From martin@suttles.sca.sfbay.sun.com

Created by martin@suttles.sca.sfbay.sun.com

Consider the following perl program

my $count = 0;
my $x = [sub() { $count },
  sub() { return $count },
  sub($) { $count += $_[0] }];
$x->[2](3);
printf "%d %d\n", $x->[0](), $x->[1]();

This prints
0 3
when it should print
3 3

The two anonymous subs $x->[0] and $x->[1] should
have exactly the same effect because `return' is
supposed to be redundant when returning the last
expression in a sub.

Flags:
    category=core
    severity=high

Site configuration information for perl v5.8.1:

Configured by martin at Sun Oct 19 13:47:11 PDT 2003.

Summary of my perl5 (revision 5.0 version 8 subversion 1) configuration:
  Platform:
    osname=solaris, osvers=2.9, archname=sun4-solaris
    uname='sunos suttles 5.9 generic_112233-05 sun4u sparc sunw,sun-blade-1000 '
    config_args='-ds -e -Dprefix=/usr/local'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O',
    cppflags='-I/usr/local/include'
    ccversion='Sun C 5.5 2003/03/12', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/lib -L/usr/ccs/lib -L/u/SUNWspro/prod/lib -L/usr/local/lib '
    libpth=/usr/lib /usr/ccs/lib /u/SUNWspro/prod/lib /usr/local/lib
    libs=-lsocket -lnsl -ldl -lm -lc
    perllibs=-lsocket -lnsl -ldl -lm -lc
    libc=/lib/libc.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='  -z ignore -z lazyload -z combreloc'
    cccdlflags='-KPIC', lddlflags=' -G  -z ignore -z lazyload -z combreloc -L/usr/lib -L/usr/ccs/lib -L/u/SUNWspro/prod/lib -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.1:
    /usr/local/lib/perl5/5.8.1/sun4-solaris
    /usr/local/lib/perl5/5.8.1
    /usr/local/lib/perl5/site_perl/5.8.1/sun4-solaris
    /usr/local/lib/perl5/site_perl/5.8.1
    /usr/local/lib/perl5/site_perl/5.8.0/sun4-solaris
    /usr/local/lib/perl5/site_perl/5.8.0
    /usr/local/lib/perl5/site_perl
    .


Environment for perl v5.8.1:
    HOME=/u/martin
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/u/martin/bin:/u/martin/bin/sun:/usr/java1.4.2/bin:/java/devtools/sparc/bin:/usr/local/bin:/usr/local/samba/bin:/usr/sfw/bin:/opt/sfw/bin:/usr/dt/bin:/usr/openwin/bin:/usr/xpg4/bin:/usr/proc/bin:/usr/ccs/bin:/usr/bin:/sbin:/usr/sbin:/java/devtools/sparc/gnucc/bin:/u/SUNWspro/bin:.
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

From @lizmat

At 23​:01 +0000 10/19/03, martin@​suttles.sca.sfbay.sun.com (via RT) wrote​:

# New Ticket Created by martin@​suttles.sca.sfbay.sun.com
# Please include the string​: [perl #24250]
# in the subject line of all future correspondence about this issue.
# <URL​: http​://rt.perl.org/rt2/Ticket/Display.html?id=24250 >

This is a bug report for perl from martin@​suttles.sca.sfbay.sun.com,
generated with the help of perlbug 1.34 running under perl v5.8.1.

-----------------------------------------------------------------
[Please enter your report here]
Consider the following perl program

my $count = 0;
my $x = [sub() { $count },
sub() { return $count },
sub($) { $count += $_[0] }];
$x->[2](3);
printf "%d %d\n", $x->[0](), $x->[1]();

This prints
0 3
when it should print
3 3

Confirmed.

Please note that if you change​:

  my $count = 0;

to​:

  our $count = 0;

the result as is expected. So this would imply a lexical pad problem to me.

Furthermore, if you change​:

  my $count = 0;

to​:

  my $count = 'Foo';

the result is _still_​:

0 3

which indicates to me the "0" is gotten from somewhere else, rather
than from $count.

Liz

From @lizmat

At 11​:44 +0200 10/20/03, Elizabeth Mattijsen wrote​:

At 23​:01 +0000 10/19/03, martin@​suttles.sca.sfbay.sun.com (via RT) wrote​:

my $count = 0;
my $x = [sub() { $count },
sub() { return $count },
sub($) { $count += $_[0] }];
$x->[2](3);
printf "%d %d\n", $x->[0](), $x->[1]();

This prints
0 3
when it should print
3 3

Hmmm... both of the following return correct results also​:

BEGIN {my $count = 'foo'}; # $foo properly initialized at compile time
my $x = [sub() { $count },
  sub() { return $count },
  sub($) { $count += $_[0] }];
$x->[2](3);
printf "%d %d\n", $x->[0](), $x->[1]();

my $count = 'foo';
my $x = [sub { $count }, # removed prototype
  sub() { return $count },
  sub($) { $count += $_[0] }];
$x->[2](3);
printf "%d %d\n", $x->[0](), $x->[1]();

So maybe this is an optimizer related problem as well?

Liz

From @lizmat

(I should stop answering my own posts ;-)

At 11​:55 +0200 10/20/03, Elizabeth Mattijsen wrote​:

Hmmm... both of the following return correct results also​:

The problem can be simplified to the following code​:

my $foo = 'foo';
my $get = sub () {$foo};
my $set = sub ($) {$foo = $_[0]};
$set->('bar');
print "foo = $foo = ".$get->()."\n";
__END__
foo = bar = foo

so this looks like an optimizer problem to me​: the $get sub is
referring to something else than the package lexical $foo.

Liz

From @iabyn

On Mon, Oct 20, 2003 at 12​:17​:24PM +0200, Elizabeth Mattijsen wrote​:

(I should stop answering my own posts ;-)

At 11​:55 +0200 10/20/03, Elizabeth Mattijsen wrote​:

Hmmm... both of the following return correct results also​:

The problem can be simplified to the following code​:

my $foo = 'foo';
my $get = sub () {$foo};
my $set = sub ($) {$foo = $_[0]};
$set->('bar');
print "foo = $foo = ".$get->()."\n";
__END__
foo = bar = foo

so this looks like an optimizer problem to me​: the $get sub is
referring to something else than the package lexical $foo.

This isn't a bug, it's a feature :-)

sub () {$lexical} is a constant sub generator, ie each call to 'sub'
returns a new constant sub rather than a closure. It doesn't seem to be
well documented though. Been around since 5.8.0.

eg

  my $x;

  for (10,20) {
  $x = $_;
  $a0 = sub(){$x};
  $a1 = sub(){$x+1};
  print $a0->(), " ", $a1->(), "\n";
  $x += 1000;
  print $a0->(), " ", $a1->(), "\n";
  }

outputs

  10 11
  10 1011
  20 21
  20 1021

since $a0 is a constant sub, but $a1 is a closure.

--
print+qq&$}$"$/$s$,$*${$}$g$s$@​$.$q$,$​:$.$q$^$,$@​$*$$;$.$q$m&if+map{m,^\d{0\,},,${$​::{$'}}=chr($"+=$&amp;||1)}q&10m22,42}6​:17*22.3@​3;^2$g3q/s"&=~m*\d\*.*g

From @lizmat

At 11​:45 +0100 10/20/03, Dave Mitchell wrote​:

On Mon, Oct 20, 2003 at 12​:17​:24PM +0200, Elizabeth Mattijsen wrote​:

(I should stop answering my own posts ;-)

At 11​:55 +0200 10/20/03, Elizabeth Mattijsen wrote​:

Hmmm... both of the following return correct results also​:

The problem can be simplified to the following code​:
so this looks like an optimizer problem to me​: the $get sub is
referring to something else than the package lexical $foo.
This isn't a bug, it's a feature :-)
sub () {$lexical} is a constant sub generator, ie each call to 'sub'
returns a new constant sub rather than a closure. It doesn't seem to be
well documented though. Been around since 5.8.0.

I always thought you would need to document a bug before it becomes a
feature? ;-)

Seriously, I haven't been able to find _any_ documentation on this
feature. It definitely is not mentioned in the 5.8.0 perldelta (or
it is described without the terms "constant" or "generator").

If this _is_ a new feature of 5.8.0, maybe it should be at least
documented in the 5.8.2 perldelta? And possible somewhere else.
Suggestions for locations where this should be documented? I'll
write up a description and submit a patch then.

Liz

From nick.ing-simmons@elixent.com

Elizabeth Mattijsen <liz@​dijkmat.nl> writes​:

At 11​:44 +0200 10/20/03, Elizabeth Mattijsen wrote​:

At 23​:01 +0000 10/19/03, martin@​suttles.sca.sfbay.sun.com (via RT) wrote​:

my $count = 0;
my $x = [sub() { $count },
sub() { return $count },
sub($) { $count += $_[0] }];
$x->[2](3);
printf "%d %d\n", $x->[0](), $x->[1]();

This prints
0 3
when it should print
3 3

Hmmm... both of the following return correct results also​:

BEGIN {my $count = 'foo'}; # $foo properly initialized at compile time

No. That has initialized a $count lexical to the BEGIN

my $x = [sub() { $count },

No lexical $count in scope here 'tis a global.

     sub\(\) \{ return $count \}\,
     sub\($\) \{ $count \+= $\_\[0\] \}\];

$x->[2](3);
printf "%d %d\n", $x->[0](), $x->[1]();

From nick.ing-simmons@elixent.com

Elizabeth Mattijsen <liz@​dijkmat.nl> writes​:

(I should stop answering my own posts ;-)

At 11​:55 +0200 10/20/03, Elizabeth Mattijsen wrote​:

Hmmm... both of the following return correct results also​:

The problem can be simplified to the following code​:

my $foo = 'foo';
my $get = sub () {$foo};
my $set = sub ($) {$foo = $_[0]};
$set->('bar');
print "foo = $foo = ".$get->()."\n";
__END__
foo = bar = foo

so this looks like an optimizer problem to me​: the $get sub is
referring to something else than the package lexical $foo.

What is a 'package lexical' ?
  - things are either in package or lexical never both...

From @lizmat

At 13​:32 +0100 10/20/03, Nick Ing-Simmons wrote​:

Elizabeth Mattijsen <liz@​dijkmat.nl> writes​:

Hmmm... both of the following return correct results also​:
BEGIN {my $count = 'foo'}; # $foo properly initialized at compile time
No. That has initialized a $count lexical to the BEGIN

Good point​: I of course meant​:

my $count;
BEGIN {$count = 'foo'}

Unfortunately, I had missed the use of printf() rather than print, so
I also missed the format specification. Had I changed if from %d to
%s, then the test would have told me more (Thanks, Yves!).

Liz

From @lizmat

At 13​:34 +0100 10/20/03, Nick Ing-Simmons wrote​:

so this looks like an optimizer problem to me​: the $get sub is
referring to something else than the package lexical $foo.
What is a 'package lexical' ?
- things are either in package or lexical never both...

A package lexical is a lexical defined at the package "scope"?

package Foo;
my $package_lexical;
{
  my $not_a_package_lexical;
}

I thought that was a pretty well known concept?

Liz

From @mjdominus

Dave Mitchell <davem@​fdgroup.com>​:

This isn't a bug, it's a feature :-)

sub () {$lexical} is a constant sub generator, ie each call to 'sub'
returns a new constant sub rather than a closure.

Oh well, so much for the principle of least astonishment.

What's the workaround for this feature?

From @demerphq

At 13​:34 +0100 10/20/03, Nick Ing-Simmons wrote​:

so this looks like an optimizer problem to me​: the $get sub is
referring to something else than the package lexical $foo.
What is a 'package lexical' ?
- things are either in package or lexical never both...

A package lexical is a lexical defined at the package "scope"?

Except that there is no such thing as package scope. There is file scope and
block scope, nothing else.

#!perl -l
package foo;
my $bar="hello";
print $bar;
package main;
print $bar;

__END__
hello
hello

package Foo;
my $package_lexical;
{
my $not_a_package_lexical;
}

I thought that was a pretty well known concept?

Well, I think its a pretty well known misnomer. But it still is a misnomer.
$package_lexical should actually be called $file_lexical.

If you want a "package lexical" youll have to wrap the whole thing in a
block.

package foo;
{
  my $package_foo_lexical;
}

or

{
  package foo;
  my $package_foo_lexical;
}

Personally I like the first one more....

Yves

From @lizmat

At 08​:58 -0400 10/20/03, Mark Jason Dominus wrote​:

Dave Mitchell <davem@​fdgroup.com>​:

This isn't a bug, it's a feature :-)

sub () {$lexical} is a constant sub generator, ie each call to 'sub'
returns a new constant sub rather than a closure.

Oh well, so much for the principle of least astonishment.

What's the workaround for this feature?

Dropping the prototype seems to work​:

my $foo = 'foo';
my $get = sub {$foo}; # look, no prototype!
my $set = sub ($) {$foo = $_[0]};
$set->('bar');
print "foo = $foo = ".$get->()."\n";
__END__
foo = bar = bar

Liz

From nick.ing-simmons@elixent.com

Elizabeth Mattijsen <liz@​dijkmat.nl> writes​:

At 13​:34 +0100 10/20/03, Nick Ing-Simmons wrote​:

so this looks like an optimizer problem to me​: the $get sub is
referring to something else than the package lexical $foo.
What is a 'package lexical' ?
- things are either in package or lexical never both...

A package lexical is a lexical defined at the package "scope"?

package Foo;
my $package_lexical;
{
my $not_a_package_lexical;
}

I thought that was a pretty well known concept?
Not by me - calling it a package lexical suggests adding a 'package'
would make a difference. It doesn't.

package Bar;
my $file_lexical;

package Foo;
{
  package Baz;
  my $nested_lexical = $file_lexical;
}

From @iabyn

On Mon, Oct 20, 2003 at 02​:47​:36PM +0200, Elizabeth Mattijsen wrote​:

At 13​:34 +0100 10/20/03, Nick Ing-Simmons wrote​:

so this looks like an optimizer problem to me​: the $get sub is
referring to something else than the package lexical $foo.
What is a 'package lexical' ?
- things are either in package or lexical never both...

A package lexical is a lexical defined at the package "scope"?

package Foo;
my $package_lexical;
{
my $not_a_package_lexical;
}

No, $package_lexical is a file-scoped lexical; it has nothing whatsoever
to do with packages.

--
In the 70's we wore flares because we didn't know any better.
What possible excuse does the current generation have?

From @iabyn

On Mon, Oct 20, 2003 at 08​:58​:05AM -0400, Mark Jason Dominus wrote​:

Dave Mitchell <davem@​fdgroup.com>​:

This isn't a bug, it's a feature :-)

sub () {$lexical} is a constant sub generator, ie each call to 'sub'
returns a new constant sub rather than a closure.

Oh well, so much for the principle of least astonishment.

What's the workaround for this feature?

Remove the ()'s.

--
To collect all the latest movies, simply place an unprotected ftp server
on the Internet, and wait for the disk to fill....

From @lizmat

At 13​:59 +0100 10/20/03, Orton, Yves wrote​:

At 13​:34 +0100 10/20/03, Nick Ing-Simmons wrote​:

so this looks like an optimizer problem to me​: the $get sub is
referring to something else than the package lexical $foo.
What is a 'package lexical' ?
- things are either in package or lexical never both...

A package lexical is a lexical defined at the package "scope"?
Except that there is no such thing as package scope. There is file scope and
block scope, nothing else.

#!perl -l
package foo;
my $bar="hello";
print $bar;
package main;
print $bar;

__END__
hello
hello

Or alternately​:

use strict;
use warnings;

package Foo;
my $foo = 1;

package Bar;
my $foo = 2;
__END__
"my" variable $foo masks earlier declaration in same scope at line 8.

I see your point.

package Foo;
my $package_lexical;
{
my $not_a_package_lexical;
}
I thought that was a pretty well known concept?
Well, I think its a pretty well known misnomer. But it still is a misnomer.
$package_lexical should actually be called $file_lexical.

If you want a "package lexical" youll have to wrap the whole thing in a
block.

package foo;
{
my $package_foo_lexical;
}

or

{
package foo;
my $package_foo_lexical;
}

Personally I like the first one more....

{
  package Foo;
  print __PACKAGE__,$/;
}
print __PACKAGE__,$/;
__END__
Foo
main

See, packages are scoped! ;-) Just kidding. I will now start to
reprogram myself to use "file lexical" rather than "package lexical".

Liz

From nick.ing-simmons@elixent.com

Mark Jason Dominus <mjd@​plover.com> writes​:

Dave Mitchell <davem@​fdgroup.com>​:

This isn't a bug, it's a feature :-)

sub () {$lexical} is a constant sub generator, ie each call to 'sub'
returns a new constant sub rather than a closure.

Oh well, so much for the principle of least astonishment.

What's the workaround for this feature?

A. Don't give the () prototype (they don't do anything useful
  on dynamic calls anyway).
B. Add a return

From nick.ing-simmons@elixent.com

Yves Orton <yves.orton@​de.mci.com> writes​:

A package lexical is a lexical defined at the package "scope"?

Except that there is no such thing as package scope. There is file scope and
block scope, nothing else.

If you want a "package lexical" youll have to wrap the whole thing in a
block.

package foo;
{
my $package_foo_lexical;
}

or

{
package foo;
my $package_foo_lexical;
}

Personally I like the first one more....

And both are surely equivalent to

{
my $invisible_lexical;
}

Yves

From @demerphq

Except that there is no such thing as package scope. There
is file scope and
block scope, nothing else.

Obviously I only meant this at the lexical level...

Personally I like the first one more....

And both are surely equivalent to

{
my $invisible_lexical;
}

Well I did assume that more would be inside the block, such as procedure
declarations....

Or did I miss your point?

Yves

From @iabyn

On Mon, Oct 20, 2003 at 02​:29​:11PM +0200, Elizabeth Mattijsen wrote​:

At 11​:45 +0100 10/20/03, Dave Mitchell wrote​:

On Mon, Oct 20, 2003 at 12​:17​:24PM +0200, Elizabeth Mattijsen wrote​:

(I should stop answering my own posts ;-)

At 11​:55 +0200 10/20/03, Elizabeth Mattijsen wrote​:

Hmmm... both of the following return correct results also​:

The problem can be simplified to the following code​:
so this looks like an optimizer problem to me​: the $get sub is
referring to something else than the package lexical $foo.
This isn't a bug, it's a feature :-)
sub () {$lexical} is a constant sub generator, ie each call to 'sub'
returns a new constant sub rather than a closure. It doesn't seem to be
well documented though. Been around since 5.8.0.

I always thought you would need to document a bug before it becomes a
feature? ;-)

Seriously, I haven't been able to find _any_ documentation on this
feature. It definitely is not mentioned in the 5.8.0 perldelta (or
it is described without the terms "constant" or "generator").

If this _is_ a new feature of 5.8.0, maybe it should be at least
documented in the 5.8.2 perldelta? And possible somewhere else.
Suggestions for locations where this should be documented? I'll
write up a description and submit a patch then.

I think the best place would be in perlsub.pod in
=head2 Constant Functions

Personally I think its a confusing feature, but it appears to be used
in lib/constant.pm - presumably as a cheap way of creating the const subs
as needed. C<use constant foo => 1> gets implemented like
  { my ($name, $val) = @​_; *$name = sub(){$val} }

Anyone feel like deprecating it ;-)

--
Little fly, thy summer's play my thoughtless hand has
terminated with extreme prejudice.
  (with apologies to William Blake)

From @lizmat

At 14​:57 +0100 10/20/03, Dave Mitchell wrote​:

On Mon, Oct 20, 2003 at 02​:29​:11PM +0200, Elizabeth Mattijsen wrote​:

Seriously, I haven't been able to find _any_ documentation on this
feature. It definitely is not mentioned in the 5.8.0 perldelta (or
it is described without the terms "constant" or "generator").

If this _is_ a new feature of 5.8.0, maybe it should be at least
documented in the 5.8.2 perldelta? And possible somewhere else.
Suggestions for locations where this should be documented? I'll
write up a description and submit a patch then.

I think the best place would be in perlsub.pod in
=head2 Constant Functions

Ok, I'll do that but first​:

Personally I think its a confusing feature, but it appears to be used
in lib/constant.pm - presumably as a cheap way of creating the const subs
as needed. C<use constant foo => 1> gets implemented like
{ my ($name, $val) = @​_; *$name = sub(){$val} }
Anyone feel like deprecating it ;-)

Well, I guess Dominus feels like me, that it is catching people by surprise.

I think this would call for a compile-time CODE attribute "​: constant"​:

{my ($name, $val) = @​_; *$name = sub : constant{$val}

Liz

From nick.ing-simmons@elixent.com

Yves Orton <yves.orton@​de.mci.com> writes​:

Except that there is no such thing as package scope. There
is file scope and
block scope, nothing else.

Obviously I only meant this at the lexical level...

Personally I like the first one more....

And both are surely equivalent to

{
my $invisible_lexical;
}

Well I did assume that more would be inside the block,

Ah - I didn't I took your examples as "boiler plate" to generate
a mythical 'package lexical'.

such as procedure
declarations....

Or did I miss your point?

No.

Yves

From @gbarr

On 20 Oct 2003, at 13​:29, Elizabeth Mattijsen wrote​:

At 11​:45 +0100 10/20/03, Dave Mitchell wrote​:

On Mon, Oct 20, 2003 at 12​:17​:24PM +0200, Elizabeth Mattijsen wrote​:

(I should stop answering my own posts ;-)

At 11​:55 +0200 10/20/03, Elizabeth Mattijsen wrote​:

Hmmm... both of the following return correct results also​:

The problem can be simplified to the following code​:
so this looks like an optimizer problem to me​: the $get sub is
referring to something else than the package lexical $foo.
This isn't a bug, it's a feature :-)
sub () {$lexical} is a constant sub generator, ie each call to 'sub'
returns a new constant sub rather than a closure. It doesn't seem to
be
well documented though. Been around since 5.8.0.

It is only supposed to generate a constant sub if there are no other
references to the variable. ie there is no way the variable can change.

I always thought you would need to document a bug before it becomes a
feature? ;-)

Seriously, I haven't been able to find _any_ documentation on this
feature. It definitely is not mentioned in the 5.8.0 perldelta (or it
is described without the terms "constant" or "generator").

Constant subs were in 5.6.0 IIRC

If this _is_ a new feature of 5.8.0, maybe it should be at least
documented in the 5.8.2 perldelta? And possible somewhere else.
Suggestions for locations where this should be documented? I'll write
up a description and submit a patch then.

I suspect it is a bug in the code that determines if a sub can be
considered a constant sub and was introduced during 5.7 development

Graham.

From @iabyn

On Mon, Oct 20, 2003 at 04​:59​:20PM +0100, Graham Barr wrote​:

It is only supposed to generate a constant sub if there are no other
references to the variable. ie there is no way the variable can change.

But it's easy to create a reference after the event​:

  sub new_constant_sub {
  my $val = shift;
  my $s = sub() {$val}; # only one ref, so $s a const sub
  $global = \$val; # whoops
  $s;
  }

I always thought you would need to document a bug before it becomes a
feature? ;-)

Seriously, I haven't been able to find _any_ documentation on this
feature. It definitely is not mentioned in the 5.8.0 perldelta (or it
is described without the terms "constant" or "generator").

Constant subs were in 5.6.0 IIRC

but anonymous 'const sub generators' were only introduced in 5.8.0, by

[ 7389] By​: jhi on 2000/10/21 14​:26​:45
  Log​: Subject​: Re​: Creating const subs for constants.
  From​: John Tobey <jtobey@​john-edwin-tobey.org>
  Date​: Fri, 20 Oct 2000 22​:03​:27 -0400 (EDT)
  Message-Id​: <m13mo0N-000FObC@​feynman.localnet>
  Branch​: perl
  ! cv.h embed.h embed.pl objXSUB.h op.c perlapi.c pod/perlapi.pod
  ! proto.h sv.c

If this _is_ a new feature of 5.8.0, maybe it should be at least
documented in the 5.8.2 perldelta? And possible somewhere else.
Suggestions for locations where this should be documented? I'll write
up a description and submit a patch then.

I suspect it is a bug in the code that determines if a sub can be
considered a constant sub and was introduced during 5.7 development

No, I think its a deliberate feature (but I could be wrong!)
In particular from that patch​:

+
+ if (CvCONST(cv)) {
+ SV* const_sv = op_const_sv(CvSTART(cv), cv);
+ assert(const_sv);
+ /* constant sub () { $x } closing over $x - see lib/constant.pm */
  ^^^^^^^^^^^^^

--
Technology is dominated by two types of people​: those who understand what
they do not manage, and those who manage what they do not understand.

From perl_dummy@bloodgate.com

-----BEGIN PGP SIGNED MESSAGE-----

Moin,

Dave Mitchell <davem@​fdgroup.com>​:

This isn't a bug, it's a feature :-)

sub () {$lexical} is a constant sub generator, ie each call to 'sub'
returns a new constant sub rather than a closure.

Oh. That got me by surprise. I thought that I followed p5p quite closely, but
someone introcuded a new bug^H^H^Hfeature while I was not looking. And a
confusing one. I think these should not be different​:

  sub () { $lexical; }
  sub () { return $lexical; }

Best wishes,

Tels

- --
Signed on Mon Oct 20 18​:54​:19 2003 with key 0x93B84C15.
Visit my photo gallery at http​://bloodgate.com/photos/
PGP key on http​://bloodgate.com/tels.asc or per email.

"Yeah, baby, yeah!"

-----BEGIN PGP SIGNATURE-----
Version​: GnuPG v1.2.2-rc1-SuSE (GNU/Linux)
Comment​: When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl.

iQEVAwUBP5QTkncLPEOTuEwVAQE85gf+KtNnAsgerq4BpXMKTCvfk7VKnvlPnA2m
neuMEtVGB8wgk1ZyC4jNzn42c2/34DtLbmtEdIjGVuumSlTDX6aYq/0FLMWcwvdj
tvWdCsOPpcpYEk84ChxYgQhokzbeXNnWBzDSlEP8cq/Eiv4R6zxIt49713OnN/qx
BfowOCZmUtcdRfQhOchiLtsZUebDRt2FXvORumse8ltK92feGTYpInho/8JPAdnr
0UkF45NFVHBYtCEGNEaj4ZFpdkY8b7/kAwPUkzr1kcq9LVq9vpC+LS6uCM6lf+TG
xFt3R7wEYgcCRKml5/d8wymRuuJNgxEoOXqvOEBmarH78pz7mm2QOA==
=XQ90
-----END PGP SIGNATURE-----

From @gbarr

On 20 Oct 2003, at 17​:48, Dave Mitchell wrote​:

On Mon, Oct 20, 2003 at 04​:59​:20PM +0100, Graham Barr wrote​:

It is only supposed to generate a constant sub if there are no other
references to the variable. ie there is no way the variable can
change.

But it's easy to create a reference after the event​:

sub new\_constant\_sub \{
my $val = shift;
my $s = sub\(\) \{$val\};    \# only one ref\, so $s a const sub

But that is not when its decided that its a constant sub. Or at least
originally it was not.

When constant subs were first introduced it was in the optimizer that
the decision was taken if a sub was constant, not when the sab was
made. The decision was "does the sub being called have a proto of ()
and is its body a single scalar with a refcnt of 1", if yes then inline
it.

I am guessing that logic has been pushed back to when the sub is
defined, which has resulted in this bug

Graham.

$global = \\$val;    \# whoops
$s;
\}

I always thought you would need to document a bug before it becomes a
feature? ;-)

Seriously, I haven't been able to find _any_ documentation on this
feature. It definitely is not mentioned in the 5.8.0 perldelta (or
it
is described without the terms "constant" or "generator").

Constant subs were in 5.6.0 IIRC

but anonymous 'const sub generators' were only introduced in 5.8.0, by

[ 7389] By​: jhi on 2000/10/21
14​:26​:45
Log​: Subject​: Re​: Creating const subs for constants.
From​: John Tobey <jtobey@​john-edwin-tobey.org>
Date​: Fri, 20 Oct 2000 22​:03​:27 -0400 (EDT)
Message-Id​: <m13mo0N-000FObC@​feynman.localnet>
Branch​: perl
! cv.h embed.h embed.pl objXSUB.h op.c perlapi.c pod/perlapi.pod
! proto.h sv.c

If this _is_ a new feature of 5.8.0, maybe it should be at least
documented in the 5.8.2 perldelta? And possible somewhere else.
Suggestions for locations where this should be documented? I'll
write
up a description and submit a patch then.

I suspect it is a bug in the code that determines if a sub can be
considered a constant sub and was introduced during 5.7 development

No, I think its a deliberate feature (but I could be wrong!)
In particular from that patch​:

+
+ if (CvCONST(cv)) {
+ SV* const_sv = op_const_sv(CvSTART(cv), cv);
+ assert(const_sv);
+ /* constant sub () { $x } closing over $x - see
lib/constant.pm */
^^^^^^^^^^^^^

--
Technology is dominated by two types of people​: those who understand
what
they do not manage, and those who manage what they do not understand.

From @ysth

On Sun, Oct 19, 2003 at 11​:01​:42PM -0000, "martin@​suttles.sca.sfbay.sun.com (via RT)" <perlbug-followup@​perl.org> wrote​:

my $count = 0;
my $x = [sub() { $count },
sub() { return $count },
sub($) { $count += $_[0] }];
$x->[2](3);
printf "%d %d\n", $x->[0](), $x->[1]();

This prints
0 3
when it should print
3 3

The two anonymous subs $x->[0] and $x->[1] should
have exactly the same effect because `return' is
supposed to be redundant when returning the last
expression in a sub.

This surprising behaviour is intentional. See perlsub/Constant
Functions. The doc there could use some improvement to mention
anonymous subs. (Note that constant.pm relies on this behaviour.)

Actually, there does seem to be a bug here. If you reverse the order
of the first two subs, $count should have an extra refcnt that should
prevent the constant sub, but it doesn't work that way. This seems to
be fixed in bleadperl.

From @iabyn

On Mon, Oct 20, 2003 at 06​:20​:13PM +0100, Graham Barr wrote​:

On 20 Oct 2003, at 17​:48, Dave Mitchell wrote​:

On Mon, Oct 20, 2003 at 04​:59​:20PM +0100, Graham Barr wrote​:

It is only supposed to generate a constant sub if there are no other
references to the variable. ie there is no way the variable can
change.

But it's easy to create a reference after the event​:

sub new_constant_sub {
my $val = shift;
my $s = sub() {$val}; # only one ref, so $s a const sub

But that is not when its decided that its a constant sub. Or at least
originally it was not.

When constant subs were first introduced it was in the optimizer that
the decision was taken if a sub was constant, not when the sab was
made. The decision was "does the sub being called have a proto of ()
and is its body a single scalar with a refcnt of 1", if yes then inline
it.

I am guessing that logic has been pushed back to when the sub is
defined, which has resulted in this bug

The 5.6.0 behaviour was that subs (named or anon) were not marked as CONST
at compile time; instead at every compile-time appearance of a sub call,
the sub was checked for having a constant value or a single lexical with
refcount<2, and if so the OP_ENTERSUB was replaced with an OP_CONST.

This meant that C<use constant> cloned a full heavyweight anon sub for
each defined constant.

In 5.8.0, sub f(){1}-style subs were detected at compile time as being
constant, and if so, replaced with CvCONSTs. sub(){$x}'s were detected
as being CONST *candidates* at compile time if the refcnt of $x was < 2.
Then each time they were cloned, the current value of $x was copied to
make a new CvCONST sub. Due to the vagarites of lexical bugs prior to
bleedperl, the compile-time test for the lexical refcnt being < 2 was not
always very DWIM.

In bleedperl, I changed it to a hybrid behaviour. At compile time, the
anon CV prototype is marked as a CONST candidate if it contains a single
outer lexical. At clone time, the cloned CV is converted into a CvCONST if
the lexical has a refcnt < 2.

Thus, the refcnt<2 test is done​:

5.6.0​: sometime after cloning, when the sub call is being compiled
5.6.0​: when the anon sub is compiled
bleed​: when the anon sub is cloned

Both the 5.8.0 and bleedperl behaviours have problems, and are very
sensitive to their environments, and will generally do things that people
find unexpected.

Thus, I like Elizabeth's suggestion of using a compile-time attribute
to explicitly turn on this special behaviour, and thus to not suprise the
ordinary user.

Dave.

--
O Unicef Clearasil!
Gibberish and Drivel!
  - "Bored of the Rings"

From @nwc10

On Mon, Oct 20, 2003 at 11​:53​:03PM +0100, Dave Mitchell wrote​:

Thus, I like Elizabeth's suggestion of using a compile-time attribute
to explicitly turn on this special behaviour, and thus to not suprise the
ordinary user.

I like it, but I'm worried that I may be hasty, so I'd like to to simmer
for a bit more before committing anything half baked to maint.
(Woohoo. Mixed metaphors. Flavour clash)

It would be nice if this (or something related) also provided a solution to
how to export constants that can be autoloaded. Currently all the core
extensions (such as Fcntl, POSIX, Socket ...) have @​EXPORT or @​EXPORT_OK
full of names of "subs" that are actually defined as needed by AUTOLOAD
(these days usually by a call into code generated by ExtUtils​::Constant,
but that's not important).

As these "subroutines" are actually constant value returning, it would
be nice if

a​: there was a way to declare them them from henceforth to be constants
  without getting the prototype redefinition warning. (And its implication
  that you're making a real change to the parsing behaviour)

  (Maybe I'm confused. Maybe we just need Exporter to declare them with
  a prototype) (Without making it slower)

b​: there was a way to make the tokenizer/optimizer whatever call the sub
  to find the value whenever it is referred to. (given that it's actually
  a constant sub)

Note that you can't simply call the AUTOLOAD routine on all the subs as
you export them, as there are bazillions, and most are not used by most
programs.

Nicholas Clark

From @druud62

On 2010-11-29 10​:22, Dave Mitchell wrote​:

On Sun, Nov 28, 2010 at 05​:16​:28PM -0800, Father Chrysostomos wrote​:

I know this is what constant.pm used to use, but it causes a bug​:

What do you think is the bug? It seems to be behaving the way I expect.

$ perl -MO=Deparse -e' BEGIN{my $x = 5; *foo = sub(){$x}; $x=6} print foo'
sub BEGIN {
my $x = 5;
*foo = sub () {
$x;
}
;
$x = 6;
}
print 5;

The compiler should patch the constsub at the '$x = 6' line.

--
Ruud

From @Abigail

On Tue, Nov 30, 2010 at 10​:34​:36PM +0100, Dr.Ruud wrote​:

On 2010-11-29 10​:22, Dave Mitchell wrote​:

On Sun, Nov 28, 2010 at 05​:16​:28PM -0800, Father Chrysostomos wrote​:

I know this is what constant.pm used to use, but it causes a bug​:

What do you think is the bug? It seems to be behaving the way I expect.

$ perl -MO=Deparse -e' BEGIN{my $x = 5; *foo = sub(){$x}; $x=6} print foo'
sub BEGIN {
my $x = 5;
*foo = sub () {
$x;
}
;
$x = 6;
}
print 5;

The compiler should patch the constsub at the '$x = 6' line.

If the compiler can detect that afterwards $x won't change - but you
need to be able to solve the halting problem to do that in general.
For instance,

  BEGIN {
  my $x = 5;
  *foo = sub () {$x};
  *bar = sub {$x = shift};
  }

the compiler could only patch the constsub if it knows bar() won't
be called.

Abigail

From @druud62

On 2010-12-01 00​:24, Abigail wrote​:

On Tue, Nov 30, 2010 at 10​:34​:36PM +0100, Dr.Ruud wrote​:

On 2010-11-29 10​:22, Dave Mitchell wrote​:

On Sun, Nov 28, 2010 at 05​:16​:28PM -0800, Father Chrysostomos wrote​:

I know this is what constant.pm used to use, but it causes a bug​:

What do you think is the bug? It seems to be behaving the way I expect.

$ perl -MO=Deparse -e' BEGIN{my $x = 5; *foo = sub(){$x}; $x=6} print foo'
sub BEGIN {
my $x = 5;
*foo = sub () {
$x;
}
;
$x = 6;
}
print 5;

The compiler should patch the constsub at the '$x = 6' line.

If the compiler can detect that afterwards $x won't change - but you
need to be able to solve the halting problem to do that in general.
For instance,

BEGIN {
my $x = 5;
*foo = sub () {$x};
*bar = sub {$x = shift};
}

the compiler could only patch the constsub if it knows bar() won't
be called.

And that is not '$x = CONSTANT', so that should then be a compile error.

--
Ruud

From @Abigail

On Wed, Dec 01, 2010 at 07​:38​:30PM +0100, Dr.Ruud wrote​:

On 2010-12-01 00​:24, Abigail wrote​:

On Tue, Nov 30, 2010 at 10​:34​:36PM +0100, Dr.Ruud wrote​:

On 2010-11-29 10​:22, Dave Mitchell wrote​:

On Sun, Nov 28, 2010 at 05​:16​:28PM -0800, Father Chrysostomos wrote​:

I know this is what constant.pm used to use, but it causes a bug​:

What do you think is the bug? It seems to be behaving the way I expect.

$ perl -MO=Deparse -e' BEGIN{my $x = 5; *foo = sub(){$x}; $x=6} print foo'
sub BEGIN {
my $x = 5;
*foo = sub () {
$x;
}
;
$x = 6;
}
print 5;

The compiler should patch the constsub at the '$x = 6' line.

If the compiler can detect that afterwards $x won't change - but you
need to be able to solve the halting problem to do that in general.
For instance,

BEGIN {
my $x = 5;
*foo = sub () {$x};
*bar = sub {$x = shift};
}

the compiler could only patch the constsub if it knows bar() won't
be called.

And that is not '$x = CONSTANT', so that should then be a compile error.

You're proposing a sub of the form 'sub {$x = shift;}' should be
a compile error?

Abigail

From @druud62

On 2010-12-02 10​:33, Abigail wrote​:

On Wed, Dec 01, 2010 at 07​:38​:30PM +0100, Dr.Ruud wrote​:

On 2010-12-01 00​:24, Abigail wrote​:

On Tue, Nov 30, 2010 at 10​:34​:36PM +0100, Dr.Ruud wrote​:

On 2010-11-29 10​:22, Dave Mitchell wrote​:

On Sun, Nov 28, 2010 at 05​:16​:28PM -0800, Father Chrysostomos wrote​:

I know this is what constant.pm used to use, but it causes a bug​:

What do you think is the bug? It seems to be behaving the way I expect.

$ perl -MO=Deparse -e' BEGIN{my $x = 5; *foo = sub(){$x}; $x=6} print foo'
sub BEGIN {
my $x = 5;
*foo = sub () {
$x;
}
;
$x = 6;
}
print 5;

The compiler should patch the constsub at the '$x = 6' line.

If the compiler can detect that afterwards $x won't change - but you
need to be able to solve the halting problem to do that in general.
For instance,

BEGIN \{
   my $x = 5;
   \*foo = sub \(\) \{$x\};
   \*bar = sub \{$x = shift\};
\}

the compiler could only patch the constsub if it knows bar() won't
be called.

And that is not '$x = CONSTANT', so that should then be a compile error.

You're proposing a sub of the form 'sub {$x = shift;}' should be
a compile error?

Indeed. The $x there, is (an alias for) a constant, and the compiler
know that. The other option is to lose the constantification of foo(),
which is unwanted.

A 'const' keyword would help.

--
Ruud

From @jkeenan

On Tue Oct 21 14​:04​:34 2003, davem wrote​:

On Tue, Oct 21, 2003 at 01​:46​:27PM -0700, Yitzchak Scott-Thoennes
wrote​:

On Tue, Oct 21, 2003 at 09​:34​:24PM +0100, Dave Mitchell
<davem@​fdgroup.com> wrote​:

In the medium-term, I want to remove the sub(){$x} feature from
Perl,
and also make constant.pm do it another way. Both are relatively
easy to
do (although in the longer term there may be even cleverer ways of
doing
it in constant.pm that would be harder to do but more efficient).
However, in the short term (ie for 5.8.2), neither of these
changes should
happen (I'm not even sure if they should happen in any 5.8.x
release).

Given that we have a report of the feature breaking code as of
5.8.0,
it would be advisable to fix it in the end-of-yearish 5.8.x release.

Perhaps, but its been broken for 1.5 years befor anyone noticed.
During
that time people may have added code that indadvertently uses the new
behaviour and so would break if reverted.

So the only real question for 5.8.2 is whether we want to add a
deprecation warning for the feature we added (undocumented) in
5.8.0
and which we intend to remove in 5.10.0 (conscensus permitting).

The point of a warning seems more to me to be for when the coder
didn't
want a constant that for when he/she did.

i *think* that's what I was saying. Every time Perl does the strange
5.8.0-ish convertion to a constant, you would get the warning. Normal
closures would be silent.

Discussion in this ticket petered out nine years ago.

Are there any code or documentation changes we should be considering
with respect to the issues raised?

If not, can we close the ticket?

Thank you very much.
Jim Keenan

From @cpansprout

On Fri Mar 30 18​:15​:13 2012, jkeenan wrote​:

On Tue Oct 21 14​:04​:34 2003, davem wrote​:

On Tue, Oct 21, 2003 at 01​:46​:27PM -0700, Yitzchak Scott-Thoennes
wrote​:

On Tue, Oct 21, 2003 at 09​:34​:24PM +0100, Dave Mitchell
<davem@​fdgroup.com> wrote​:

In the medium-term, I want to remove the sub(){$x} feature from
Perl,
and also make constant.pm do it another way. Both are relatively
easy to
do (although in the longer term there may be even cleverer ways of
doing
it in constant.pm that would be harder to do but more efficient).
However, in the short term (ie for 5.8.2), neither of these
changes should
happen (I'm not even sure if they should happen in any 5.8.x
release).

Given that we have a report of the feature breaking code as of
5.8.0,
it would be advisable to fix it in the end-of-yearish 5.8.x release.

Perhaps, but its been broken for 1.5 years befor anyone noticed.
During
that time people may have added code that indadvertently uses the new
behaviour and so would break if reverted.

So the only real question for 5.8.2 is whether we want to add a
deprecation warning for the feature we added (undocumented) in
5.8.0
and which we intend to remove in 5.10.0 (conscensus permitting).

The point of a warning seems more to me to be for when the coder
didn't
want a constant that for when he/she did.

i *think* that's what I was saying. Every time Perl does the strange
5.8.0-ish convertion to a constant, you would get the warning. Normal
closures would be silent.

Discussion in this ticket petered out nine years ago.

Are there any code or documentation changes we should be considering
with respect to the issues raised?

If not, can we close the ticket?

This is still a bug that needs fixing. See also
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=94490#txn-919044>.

--

Father Chrysostomos

From @nwc10

The question is at the end. It won't make much sense without the backstory.

On Sun, Oct 19, 2003 at 11​:01​:42PM -0000, martin@​suttles.sca.sfbay.sun.com (via RT) wrote​:

[Please enter your report here]
Consider the following perl program

my $count = 0;
my $x = [sub() { $count },
sub() { return $count },
sub($) { $count += $_[0] }];
$x->[2](3);
printf "%d %d\n", $x->[0](), $x->[1]();

This prints
0 3
when it should print
3 3

The two anonymous subs $x->[0] and $x->[1] should
have exactly the same effect because `return' is
supposed to be redundant when returning the last
expression in a sub.

And the behaviour in blead remains is the same

On Fri, Mar 30, 2012 at 10​:15​:03PM -0700, Father Chrysostomos via RT wrote​:

On Fri Mar 30 18​:15​:13 2012, jkeenan wrote​:

Discussion in this ticket petered out nine years ago.

Are there any code or documentation changes we should be considering
with respect to the issues raised?

If not, can we close the ticket?

This is still a bug that needs fixing. See also
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=94490#txn-919044>.

What's curious is that I never remember anyone hitting this problem
again. Perhaps it's not as common as we thought.

On Mon, Oct 20, 2003 at 11​:53​:03PM +0100, Dave Mitchell wrote​:

On Mon, Oct 20, 2003 at 06​:20​:13PM +0100, Graham Barr wrote​:

On 20 Oct 2003, at 17​:48, Dave Mitchell wrote​:

On Mon, Oct 20, 2003 at 04​:59​:20PM +0100, Graham Barr wrote​:

It is only supposed to generate a constant sub if there are no other
references to the variable. ie there is no way the variable can
change.

But it's easy to create a reference after the event​:

sub new_constant_sub {
my $val = shift;
my $s = sub() {$val}; # only one ref, so $s a const sub

But that is not when its decided that its a constant sub. Or at least
originally it was not.

When constant subs were first introduced it was in the optimizer that
the decision was taken if a sub was constant, not when the sab was
made. The decision was "does the sub being called have a proto of ()
and is its body a single scalar with a refcnt of 1", if yes then inline
it.

I am guessing that logic has been pushed back to when the sub is
defined, which has resulted in this bug

The 5.6.0 behaviour was that subs (named or anon) were not marked as CONST
at compile time; instead at every compile-time appearance of a sub call,
the sub was checked for having a constant value or a single lexical with
refcount<2, and if so the OP_ENTERSUB was replaced with an OP_CONST.

This meant that C<use constant> cloned a full heavyweight anon sub for
each defined constant.

In 5.8.0, sub f(){1}-style subs were detected at compile time as being
constant, and if so, replaced with CvCONSTs. sub(){$x}'s were detected
as being CONST *candidates* at compile time if the refcnt of $x was < 2.
Then each time they were cloned, the current value of $x was copied to
make a new CvCONST sub. Due to the vagarites of lexical bugs prior to
bleedperl, the compile-time test for the lexical refcnt being < 2 was not
always very DWIM.

In bleedperl, I changed it to a hybrid behaviour. At compile time, the
anon CV prototype is marked as a CONST candidate if it contains a single
outer lexical. At clone time, the cloned CV is converted into a CvCONST if
the lexical has a refcnt < 2.

Thus, the refcnt<2 test is done​:

5.6.0​: sometime after cloning, when the sub call is being compiled
5.6.0​: when the anon sub is compiled
bleed​: when the anon sub is cloned

Both the 5.8.0 and bleedperl behaviours have problems, and are very
sensitive to their environments, and will generally do things that people
find unexpected.

I forgot that you thought of this. I knew that I *implemented* it :-)

On Tue, Oct 21, 2003 at 12​:53​:12AM +0100, Dave Mitchell wrote​:

An idea occurs to me.

Ilya introduced a patch that allowed stash values to be an IV or a PV
rather than a full-blown GV+CV when a sub had only been declared rather
than defined (IV => no protoptye, PV => protoptye with the PVX containing
the protoptye string). Only later when the sub was actually defined was
the IV/PV replaced with a full GV containing a CV. This allowed Exporter
to whack lots of sub declarations into the caller's namespace without a
big memory overhead.

I wonder whether this idea could be extended - so for constant subs where
the package has no other variable of the same name, then rather than
having a GV pointing to a const CV, the stash just contains a "something"
(perhaps a PV with the right flags set) that actually holds the const
value. This PV would be shared by each packahge that imports it.

We would have to provide mechanisms
a) for the module to create all the values in its own namespace when
compiled;
b) for Exporter to cheaply copy these to the callers namespace.

Here endeth the 1am vague handwaving. Time for bed (said Zebedee).

On Tue, Oct 21, 2003 at 09​:34​:24PM +0100, Dave Mitchell wrote​:

In the medium-term, I want to remove the sub(){$x} feature from Perl,
and also make constant.pm do it another way. Both are relatively easy to
do (although in the longer term there may be even cleverer ways of doing
it in constant.pm that would be harder to do but more efficient).

constant.pm now takes advantage of the "proxy constant subroutines"
on all versions from 5.10.0 onwards. It no longer users the fragile
"feature" under discussion here, that

  *foo​::bar = sub () { $baz; }

for an unshared $baz of value 42 behaves the same as

  sub foo​::bar () {
  42;
  }

and creates a constant subroutine that the parser knows it can inline.

So I think we've done the first part of that - "another way"

So the only real question for 5.8.2 is whether we want to add a
deprecation warning for the feature we added (undocumented) in 5.8.0
and which we intend to remove in 5.10.0 (conscensus permitting).

So, do we want to remove the inlining of (inferred to be)-constant value
closures? And only inline subroutines that trivially return true constants.
And fix this 5.8.0 regression. What is relying on the regressed behaviour?

In the general case, is it impossible to detect all scenarios where the
variable within the closure can not later be modified?

(Proxy constant subroutines remain unchanged)

Nicholas Clark

From zefram@fysh.org

Nicholas Clark wrote​:

So, do we want to remove the inlining of (inferred to be)-constant value
closures?

Inlining of *actual* constant-value subs should remain, but "sub () {
$x }" should not be perceived as constant-value. We should replace
that mechanism by an explicit generator for constant-value subs of
runtime-specified value (trivially implemented in XS).

I've been wondering where the inlining really ought to be implemented.
With the call-checker mechanism, maybe the inlining of constant-value
subs should be exposed as a non-standard call checker, rather than
marring the standard code with an awkward special case.

-zefram

From @nwc10

On Fri, Apr 20, 2012 at 12​:43​:50PM +0100, Zefram wrote​:

Nicholas Clark wrote​:

So, do we want to remove the inlining of (inferred to be)-constant value
closures?

Inlining of *actual* constant-value subs should remain, but "sub () {
$x }" should not be perceived as constant-value. We should replace

To be clear, there's a specific subset of sub () { $x } which are perceived
as constant value. The bug (#24250) is that that perception code isn't 100%
accurate. I *think* Dave is saying that it can never be 100% accurate, but
I'm not sure.

Also, assuming that it could be made 100% accurate, as the inlining is
supposed to be an optimisation, is there any way it user code can spot
the difference between doing it and not doing it, other than introspecting
the optree, or benchmarking something?

that mechanism by an explicit generator for constant-value subs of
runtime-specified value (trivially implemented in XS).

What does this generator give us, that putting references-to-constants in
the symbol table doesn't already give us?

I've been wondering where the inlining really ought to be implemented.
With the call-checker mechanism, maybe the inlining of constant-value
subs should be exposed as a non-standard call checker, rather than
marring the standard code with an awkward special case.

Does this reduce complexity?

Nicholas Clark

From zefram@fysh.org

Nicholas Clark wrote​:

              The bug \(\#24250\) is that that perception code isn't 100%

accurate.

Oh really, I didn't realise there was any attempt for it to be seriously
accurate.

With PadWalker and suchlike, it's impossible for it to be certain about
a positive (no modification). Also, by virtue of the halting problem,
it's impossible for it to be certain about a negative. You can discount
PadWalker and the like, and potentially get something that has no false
positives and a useful proportion of true positives when applied to
the purest kind of pure Perl code. But that's rather a lot of work,
for a dubious payoff.

Also, assuming that it could be made 100% accurate, as the inlining is
supposed to be an optimisation, is there any way it user code can spot
the difference between doing it and not doing it, other than introspecting
the optree, or benchmarking something?

By definition, if an optimisation is done correctly then you can't tell
the difference short of such means.

What does this generator give us, that putting references-to-constants in
the symbol table doesn't already give us?

It gets you the constant-value sub as a sub, in isolation. It avoids
the need to touch a stash, which may not be where you want the sub to
end up and is a rather indirect way of getting the actual sub object.
(Maybe you want to feed the sub to Lexical​::Sub.) It would also be an
explicitly supported interface, unlike direct stash modification.

[variant call checker]

Does this reduce complexity?

Don't know yet. Probably leaves it about the same overall. That's why
I don't have any clear idea yet of whether it ought to be done.

-zefram

From @cpansprout

On Fri Apr 20 04​:38​:42 2012, nicholas wrote​:

What's curious is that I never remember anyone hitting this problem
again. Perhaps it's not as common as we thought.

Don’t we ask people to check whether bugs have been reported already? I
run into it from time to time.

On Tue, Oct 21, 2003 at 09​:34​:24PM +0100, Dave Mitchell wrote​:

In the medium-term, I want to remove the sub(){$x} feature from
Perl,
and also make constant.pm do it another way. Both are relatively
easy to
do (although in the longer term there may be even cleverer ways of
doing
it in constant.pm that would be harder to do but more efficient).

constant.pm now takes advantage of the "proxy constant subroutines"
on all versions from 5.10.0 onwards. It no longer users the fragile
"feature" under discussion here, that

\*foo​::bar = sub \(\) \{ $baz; \}

for an unshared $baz of value 42 behaves the same as

sub foo​::bar \(\) \{
    42;
\}

and creates a constant subroutine that the parser knows it can inline.

So I think we've done the first part of that - "another way"

So the only real question for 5.8.2 is whether we want to add a
deprecation warning for the feature we added (undocumented) in 5.8.0
and which we intend to remove in 5.10.0 (conscensus permitting).

So, do we want to remove the inlining of (inferred to be)-constant
value
closures?

Yes!

And only inline subroutines that trivially return true
constants.
And fix this 5.8.0 regression. What is relying on the regressed
behaviour?

Let’s put it in 5.17.early and find out!

--

Father Chrysostomos

From @cpansprout

On Fri Apr 20 04​:52​:01 2012, nicholas wrote​:

Also, assuming that it could be made 100% accurate, as the inlining is
supposed to be an optimisation, is there any way it user code can spot
the difference between doing it and not doing it, other than introspecting
the optree, or benchmarking something?

Subroutine redefinition.

--

Father Chrysostomos

From @iabyn

On Fri, Apr 20, 2012 at 12​:37​:55PM +0100, Nicholas Clark wrote​:

constant.pm now takes advantage of the "proxy constant subroutines"
on all versions from 5.10.0 onwards. It no longer users the fragile
"feature" under discussion here, that

\*foo​::bar = sub \(\) \{ $baz; \}

Well, having a quick look at constant.pm, it may use the new method as its
primary route, buts there's still plenly of places where it uses the old
mechanism, even down to

  my $const = $] > 5.009002;
  *_CAN_PCS = sub () {$const};

[ just by-the-by, I don't understand why constant.pm is written in such a
way as to be able to run under old perls; its not as if its a dual-life
CPAN distribution ]

I don't know enough about how the new method works to be abloe to say
whether it can be rewritten to avoid sub(){$x} completely, or whether
we need to add some sort of XS mechanism for the general case.

So, do we want to remove the inlining of (inferred to be)-constant value
closures? And only inline subroutines that trivially return true constants.
And fix this 5.8.0 regression. What is relying on the regressed behaviour?

In the general case, is it impossible to detect all scenarios where the
variable within the closure can not later be modified?

It's effectively impossible, and its trivially easy to later modify the
variable.
For example under 5.8.0, where the refcnt < 2 test was done at compile
time, the 1st sub would see a refcnt of 1 and mark the sub as 'convert to
const when cloning at time', then compiling the second sub retractively
allows you to modify the var​:

  {
  my $x = ...;
  sub () {$x}; # at compile time, has refcnt==1, looks safe...
  sub bad { $x++ }; # ..but it isn't
  }

After I changed it so that the refcnt < 2 test is done at runtime during
cloning, that can be defeated with

  {
  my $x = ...;
  sub () {$x}; # at runtime, has refcnt==1, looks safe...
  $​::global = \$x; # now has a refcnt of 2, so isn't safe
  }

So, I think we should remove this 'feature'.
If anyone is using it as intended, then the worst they'll get is
a performance decrease, with a const sub downgraded to a real sub that
returns the expected value. Perhaps we could issue a warning, with
perldiag.pod explaining what to do if you really wanted a const sub, and
to stick a 'return' in to defeat the warning if you didn't want it.

--
Dave's first rule of Opera​:
If something needs saying, say it​: don't warble it.

From @nwc10

On Fri, Apr 20, 2012 at 04​:55​:12PM +0100, Dave Mitchell wrote​:

On Fri, Apr 20, 2012 at 12​:37​:55PM +0100, Nicholas Clark wrote​:

constant.pm now takes advantage of the "proxy constant subroutines"
on all versions from 5.10.0 onwards. It no longer users the fragile
"feature" under discussion here, that

\*foo​::bar = sub \(\) \{ $baz; \}

Well, having a quick look at constant.pm, it may use the new method as its
primary route, buts there's still plenly of places where it uses the old
mechanism, even down to

my $const = $\] > 5\.009002;
\*\_CAN\_PCS = sub \(\) \{$const\};

Hmmm, bother. Although that can probably be solved like this​:

*_CAN_PCS = $] > 5.009002 ? sub () { 1 } : sub () { 0 };

(not tested)

[ just by-the-by, I don't understand why constant.pm is written in such a
way as to be able to run under old perls; its not as if its a dual-life
CPAN distribution ]

It is a dual-life CPAN distribution. See https://metacpan.org/module/constant

I don't know enough about how the new method works to be abloe to say
whether it can be rewritten to avoid sub(){$x} completely, or whether
we need to add some sort of XS mechanism for the general case.

I thought that it didn't use them, but there is one obvious bit that is still
live, a fallback in the else block here​:

  # The constant serves to optimise this entire block out on
  # 5.8 and earlier.
  if (_CAN_PCS && $symtab && !exists $symtab->{$name}) {
  # No typeglob yet, so we can use a reference as space-
  # efficient proxy for a constant subroutine
  # The check in Perl_ck_rvconst knows that inlinable
  # constants from cv_const_sv are read only. So we have to​:
  Internals​::SvREADONLY($scalar, 1);
  $symtab->{$name} = \$scalar;
  ++$flush_mro;
  } else {
  *$full_name = sub () { $scalar };
  }

I think that's only hard to work round in the case of naming a constant
with the same name as something else that already exists. I suspect that
that's rare, and could be replaced with a string eval, or cheating somewhat
with an intermediate assignment to known-free symbol table entry
(eg a name within %constant​::) and then "exporting" it to the right place.

I'm not so sure about this​:

  } elsif (@​_) {
  my @​list = @​_;
  *$full_name = sub () { @​list };

but did list inlining *ever* actually inline? I'm going to guess "no", given
that these lines

  In the current version of Perl, list constants are not inlined
  and some symbols may be redefined without generating a warning.

date from 5.004 when constant.pm was added

http​://perl5.git.perl.org/perl.git/blame/HEAD​:/dist/constant/lib/constant.pm#l347

So, I think we should remove this 'feature'.

Given that constant doesn't *need* it any more, there seems to be consensus
that it can go.

If anyone is using it as intended, then the worst they'll get is
a performance decrease, with a const sub downgraded to a real sub that
returns the expected value. Perhaps we could issue a warning, with
perldiag.pod explaining what to do if you really wanted a const sub, and
to stick a 'return' in to defeat the warning if you didn't want it.

I was initially thinking "do we really need a warning? No"
but then I suspect that for initial development purposes we ought to be
able to have it as a compile-time fatal error, to make sure we spot
everywhere in core that still has it. (Such as that fallback code in
constant.pm)

Nicholas Clark

From @demerphq

On Friday, 20 April 2012, Dave Mitchell wrote​:

On Fri, Apr 20, 2012 at 12​:37​:55PM +0100, Nicholas Clark wrote​:
So, I think we should remove this 'feature'.
If anyone is using it as intended, then the worst they'll get is
a performance decrease, with a const sub downgraded to a real sub that
returns the expected value. Perhaps we could issue a warning, with
perldiag.pod explaining what to do if you really wanted a const sub, and
to stick a 'return' in to defeat the warning if you didn't want it.

We use this a lot at work. We can recode to use other mechanisms to produce
constants, tho, so it's not a big deal, but my suspicion is its widely
used. Keep in mind that constants can be used to modify the optree and are
used for that as much as a value.

Dosomething() if CONSTANT;

Is used a lot.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

From @nwc10

On Sat, Apr 21, 2012 at 10​:48​:13AM +0200, demerphq wrote​:

On Friday, 20 April 2012, Dave Mitchell wrote​:

On Fri, Apr 20, 2012 at 12​:37​:55PM +0100, Nicholas Clark wrote​:
So, I think we should remove this 'feature'.
If anyone is using it as intended, then the worst they'll get is
a performance decrease, with a const sub downgraded to a real sub that
returns the expected value. Perhaps we could issue a warning, with
perldiag.pod explaining what to do if you really wanted a const sub, and
to stick a 'return' in to defeat the warning if you didn't want it.

We use this a lot at work. We can recode to use other mechanisms to produce
constants, tho, so it's not a big deal, but my suspicion is its widely
used. Keep in mind that constants can be used to modify the optree and are
used for that as much as a value.

Dosomething() if CONSTANT;

Is used a lot.

Not sure if you're aware as it's been cut - the context is *specifically*
constants of the form​:

  sub CONSTANT_FROM_VAR () { $foo; }

constants generated like this​:

  sub CONSTANT_FROM_CONSTANT () { 42 }

aren't a problem.

(And constants generated by constant.pm aren't a problem)

Nicholas Clark

From patcat88@snet.net

If this bug is about return or no return keyword in constant optimized
subs, I've run into it, http​://perlmonks.org/?node_id=913554

From @cpansprout

Fixed in 137da2b.

--

Father Chrysostomos

From [Unknown Contact. See original ticket]

Fixed in 137da2b.

--

Father Chrysostomos

@cpansprout - Status changed from 'open' to 'resolved'

From @avar

On fös 26 júl 00​:45​:03 2013, sprout wrote​:

Fixed in 137da2b.

I'm re-opening this bug because as discussed in http​://code.activestate.com/lists/perl5-porters/206933/ I think this fix should be backed out for now, and since it changes explicitly documented behavior should at the very least go through a deprecation cycle where we first add a warning indicating that subroutines that were previously inlined no longer are.

Otherwise we're just fixing a really obscure bug at the cost of performance regressions in code that relies on the previously documented behavior, and changing that behavior silently.

I think a perfectly acceptable way to deal with this bug would be to just document this special case, which I've done in 'perlsub​: Improve the "Constant Functions" documentation' which I'm about to push to a branch.

I.e. say that if you really need a subroutine that returns one variable and has a () prototype but don't want inlining just do : sub X () { return $x }

@avar - Status changed from 'resolved' to 'open'

From ambrus@math.bme.hu

On Fri, Apr 20, 2012 at 1​:51 PM, Nicholas Clark <nick@​ccl4.org> wrote​:

What does this generator give us, that putting references-to-constants in
the symbol table doesn't already give us?

It allows the constant pragma to define an inlined constant subroutine
even if the glob already exists because someone has defined an array
of the same name.

Ambrus