$AUTOLOAD inside of sub AUTOLOAD{} appears to create a memory leak #11949

p5pRT · 2012-02-09T05:48:57Z

Migrated from rt.perl.org#110248 (status was 'rejected')

Searchable as RT110248$

p5pRT · 2012-02-09T05:48:57Z

From jtbraun@CPAN.org

Created by jtbraun@CPAN.org

This is a bug report for perl from jtbraun@CPAN.org,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
use warnings;
use strict;

my $i = 0;
for my $varname ('OTHERVAR', 'AUTOLOAD') {
for my $subname ('OTHERSUB', 'AUTOLOAD') {
for my $n ( 1 .. 400 ) {
++$i;
my $superclass = 'Acme::Test::Leak';
my $subclass = sprintf("namespace%06d", $i);
my $package = $superclass . '::' . $subclass;
my $code = qq{
package $package;
use vars qw(\$$varname );
sub $subname {
print \$$varname;
};
};
eval $code;
# remove references to the namespace
delete $Acme::Test::Leak::{$subclass.'::'};

my $m e m = ‘ p s h - o s i z e$ $`;
chomp($mem);
printf( "v:%s s:%s n:%3d :: %.1f\r", $varname, $subname,
$n, $mem / 1024 );
print "\n" unless $n % 50;
}
print "\n";
}
}

__END__

The above test eval's code of the form:

package Acme::Test::Leak::namespace000001;
use vars qw( $VAR );
sub SUB {
print $VAR;
};

Where namespace always increases, $VAR takes on the values OTHERVAR or
AUTOLOAD, and SUB takes on the values OTHERSUB, AUTOLOAD. After
eval'ing this code block, the namespace is deleted by the line:

delete $Acme::Test::Leak::{$subclass.'::'};

And then the test prints out the current process memory usage. This
is repeated 400 times for each different combination of $VAR/SUB.
Memory use stays stable until the AUTOLOAD/AUTOLOAD combination, and
them memory usage steadily climbs. Example output on my system:

v:OTHERVAR s:OTHERSUB n: 50 :: 0.6
v:OTHERVAR s:OTHERSUB n:100 :: 0.6
v:OTHERVAR s:OTHERSUB n:150 :: 0.6
v:OTHERVAR s:OTHERSUB n:200 :: 0.6
v:OTHERVAR s:OTHERSUB n:250 :: 0.6
v:OTHERVAR s:OTHERSUB n:300 :: 0.6
v:OTHERVAR s:OTHERSUB n:350 :: 0.6
v:OTHERVAR s:OTHERSUB n:400 :: 0.6

v:OTHERVAR s:AUTOLOAD n: 50 :: 0.6
v:OTHERVAR s:AUTOLOAD n:100 :: 0.6
v:OTHERVAR s:AUTOLOAD n:150 :: 0.6
v:OTHERVAR s:AUTOLOAD n:200 :: 0.6
v:OTHERVAR s:AUTOLOAD n:250 :: 0.6
v:OTHERVAR s:AUTOLOAD n:300 :: 0.6
v:OTHERVAR s:AUTOLOAD n:350 :: 0.6
v:OTHERVAR s:AUTOLOAD n:400 :: 0.6

v:AUTOLOAD s:OTHERSUB n: 50 :: 0.6
v:AUTOLOAD s:OTHERSUB n:100 :: 0.6
v:AUTOLOAD s:OTHERSUB n:150 :: 0.6
v:AUTOLOAD s:OTHERSUB n:200 :: 0.6
v:AUTOLOAD s:OTHERSUB n:250 :: 0.6
v:AUTOLOAD s:OTHERSUB n:300 :: 0.6
v:AUTOLOAD s:OTHERSUB n:350 :: 0.6
v:AUTOLOAD s:OTHERSUB n:400 :: 0.6

v:AUTOLOAD s:AUTOLOAD n: 50 :: 0.7
v:AUTOLOAD s:AUTOLOAD n:100 :: 0.7
v:AUTOLOAD s:AUTOLOAD n:150 :: 0.8
v:AUTOLOAD s:AUTOLOAD n:200 :: 0.8
v:AUTOLOAD s:AUTOLOAD n:250 :: 0.8
v:AUTOLOAD s:AUTOLOAD n:300 :: 0.9
v:AUTOLOAD s:AUTOLOAD n:350 :: 0.9
v:AUTOLOAD s:AUTOLOAD n:400 :: 1.1

Perl Info


Flags:
     category=core
     severity=medium

Site configuration information for perl 5.14.2:

Configured by jtbraun at Thu Dec  1 01:10:27 PST 2011.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:

   Platform:
     osname=linux, osvers=3.0.0-13-generic, archname=i686-linux
     uname='linux pedafly 3.0.0-13-generic #22-ubuntu smp wed nov 2 
13:25:36 utc 2011 i686 i686 i386 gnulinux '
     config_args='-de 
-Dprefix=/home/jtbraun/perl5/perlbrew/perls/perl-5.14.2'
     hint=recommended, useposix=true, d_sigaction=define
     useithreads=undef, usemultiplicity=undef
     useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
     use64bitint=undef, use64bitall=undef, uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector 
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
     optimize='-O2',
     cppflags='-fno-strict-aliasing -pipe -fstack-protector 
-I/usr/local/include'
     ccversion='', gccversion='4.6.1', gccosandvers=''
     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
     ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
     alignbytes=4, prototype=define
   Linker and Libraries:
     ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
     libpth=/usr/local/lib /lib/i386-linux-gnu /lib/../lib 
/usr/lib/i386-linux-gnu /usr/lib/../lib /lib /usr/lib
     libs=-lnsl -ldl -lm -lcrypt -lutil -lc
     perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
     libc=, so=so, useshrplib=false, libperl=libperl.a
     gnulibc_version='2.13'
   Dynamic Linking:
     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
     cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib 
-fstack-protector'

Locally applied patches:



@INC for perl 5.14.2:
     /home/jtbraun/perl5/CPAN/lib/perl5/i686-linux
     /home/jtbraun/perl5/CPAN/lib/perl5
     /home/jtbraun/whome/lib/perl5
     
/home/jtbraun/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/i686-linux
     /home/jtbraun/perl5/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2
     /home/jtbraun/perl5/perlbrew/perls/perl-5.14.2/lib/5.14.2/i686-linux
     /home/jtbraun/perl5/perlbrew/perls/perl-5.14.2/lib/5.14.2
     .


Environment for perl 5.14.2:
     HOME=/home/jtbraun
     LANG=en_US.UTF-8
     LANGUAGE=en_US:en
     LC_COLLATE=en_US.UTF-8
     LC_CTYPE=en_US.UTF-8
     LC_MESSAGES=en_US.UTF-8
     LD_LIBRARY_PATH (unset)
     LOGDIR (unset)
     
PATH=/home/jtbraun/perl5/perlbrew/bin:/home/jtbraun/perl5/perlbrew/perls/perl-5.14.2/bin:/home/jtbraun/whome/bin/linux:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/home/jtbraun/perl5/CPAN/bin
     
PERL5LIB=/home/jtbraun/perl5/CPAN/lib/perl5:/home/jtbraun/whome/lib/perl5
     PERLBREW_BASHRC_VERSION=0.33
     PERLBREW_HOME=/home/jtbraun/.perlbrew
     
PERLBREW_PATH=/home/jtbraun/perl5/perlbrew/bin:/home/jtbraun/perl5/perlbrew/perls/perl-5.14.2/bin
     PERLBREW_PERL=perl-5.14.2
     PERLBREW_ROOT=/home/jtbraun/perl5/perlbrew
     PERLBREW_VERSION=0.33
     PERL_BADLANG (unset)
     PERL_CPANM_OPT=--prompt --reinstall -l ~/perl5/CPAN
     SHELL=/bin/bash

p5pRT · 2012-02-11T16:26:56Z

From @jkeenan

On Wed Feb 08 21:48:57 2012, jtbraun@CPAN.org wrote:

This is a bug report for perl from jtbraun@CPAN.org,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
use warnings;
use strict;

my $i = 0;
for my $varname ('OTHERVAR', 'AUTOLOAD') {
for my $subname ('OTHERSUB', 'AUTOLOAD') {
for my $n ( 1 .. 400 ) {
++$i;
my $superclass = 'Acme::Test::Leak';
my $subclass = sprintf("namespace%06d", $i);
my $package = $superclass . '::' . $subclass;
my $code = qq{
package $package;
use vars qw(\$$varname );
sub $subname {
print \$$varname;
};
};
eval $code;
# remove references to the namespace
delete $Acme::Test::Leak::{$subclass.'::'};
         my $mem = \`ps h \-o size $$\`;
         chomp$$mem$;
         printf$ "v&#8203;:%s s&#8203;:%s n&#8203;:%3d :&#8203;: %\.1f\\r"\, $varname\, $subname\,
$n, $mem / 1024 );
print "\n" unless $n % 50;
}
print "\n";
}
}

__END__

The above test eval's code of the form:
 package Acme&#8203;::Test&#8203;::Leak&#8203;::namespace000001;
 use vars qw\( $VAR  $;
 sub SUB \{
      print $VAR;
 \};
Where namespace always increases, $VAR takes on the values OTHERVAR or
AUTOLOAD, and SUB takes on the values OTHERSUB, AUTOLOAD. After
eval'ing this code block, the namespace is deleted by the line:
 delete $Acme&#8203;::Test&#8203;::Leak&#8203;::\{$subclass\.'&#8203;::'\};
And then the test prints out the current process memory usage. This
is repeated 400 times for each different combination of $VAR/SUB.
Memory use stays stable until the AUTOLOAD/AUTOLOAD combination, and
them memory usage steadily climbs. Example output on my system:

[snip]

 v&#8203;:AUTOLOAD s&#8203;:OTHERSUB n&#8203;:400 :&#8203;: 0\.6

 v&#8203;:AUTOLOAD s&#8203;:AUTOLOAD n&#8203;: 50 :&#8203;: 0\.7
 v&#8203;:AUTOLOAD s&#8203;:AUTOLOAD n&#8203;:100 :&#8203;: 0\.7
 v&#8203;:AUTOLOAD s&#8203;:AUTOLOAD n&#8203;:150 :&#8203;: 0\.8
 v&#8203;:AUTOLOAD s&#8203;:AUTOLOAD n&#8203;:200 :&#8203;: 0\.8
 v&#8203;:AUTOLOAD s&#8203;:AUTOLOAD n&#8203;:250 :&#8203;: 0\.8
 v&#8203;:AUTOLOAD s&#8203;:AUTOLOAD n&#8203;:300 :&#8203;: 0\.9
 v&#8203;:AUTOLOAD s&#8203;:AUTOLOAD n&#8203;:350 :&#8203;: 0\.9
 v&#8203;:AUTOLOAD s&#8203;:AUTOLOAD n&#8203;:400 :&#8203;: 1\.1

I duplicated your results on Linux/i386. However, I could not run your
code directly on Darwin/PPC, probably due to the differences in 'ps'
between Linux and BSD. On Darwin 'size' is not available, as it's not
in this list:

#####
$ ps -L
%cpu %mem acflag acflg blocked caught command cpu cputime f flags gid
ignored
inblk inblock jobc ktrace ktracep lim login logname lstart majflt minflt
msgrcv
msgsnd ni nice nivcsw nsignals nsigs nswap nvcsw nwchan oublk oublock p_ru
paddr pagein pcpu pending pgid pid pmem ppid pri pstime putime re rgid rss
rssize rsz ruid ruser sess sig sigmask sl start stat state stime svgid svuid
tdev time tpgid tsess tsiz tt tty ucomm uid upr user usrpri utime vsize vsz
wchan xstat
#####

How should I modify the 'ps' line in your code for systems other than Linux?

Thank you very much.
Jim Keenan

p5pRT · 2012-02-11T16:26:57Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2012-02-12T22:51:31Z

From @cpansprout

On Sat Feb 11 08:26:56 2012, jkeenan wrote:

How should I modify the 'ps' line in your code for systems other than
Linux?

I modified it to print out the process number and go into an infinite
loop. Then I can observe memory usage with ‘top’:

warn $$;
use warnings;
use strict;

my $i = 0;
my $varname = 'OTHERVAR';
my $subname = 'OTHERVAR';
{
++$i;
my $superclass = 'Acme::Test::Leak';
my $subclass = sprintf("namespace%06d", $i);
my $package = $superclass . '::' . $subclass;
my $code = qq{
package $package;
use vars qw(\$$varname );
sub $subname {
print \$$varname;
};
};
eval $code;
# remove references to the namespace
delete $Acme::Test::Leak::{$subclass.'::'};
redo;
}

This also shows that the problem is not specific to AUTOLOAD. But
creating a sub named OTHERVAR referencing a variable of the same name, I
can create a memory leak.

The body of the sub is referencing the *OTHERVAR glob, which references
the sub. So this is a circular reference.

--

Father Chrysostomos

p5pRT · 2012-02-13T10:02:12Z

From @demerphq

On 12 February 2012 23:51, Father Chrysostomos via RT
<perlbug-followup@perl.org> wrote:

On Sat Feb 11 08:26:56 2012, jkeenan wrote:

How should I modify the 'ps' line in your code for systems other than
Linux?

I modified it to print out the process number and go into an infinite
loop. Then I can observe memory usage with ‘top’:

warn $$;
use warnings;
use strict;

my $i = 0;
my $varname = 'OTHERVAR';
my $subname = 'OTHERVAR';
{
++$i;
my $superclass = 'Acme::Test::Leak';
my $subclass = sprintf("namespace%06d", $i);
my $package = $superclass . '::' . $subclass;
my $code = qq{
package $package;
use vars qw(\$$varname );
sub $subname {
print \$$varname;
};
};
eval $code;
# remove references to the namespace
delete $Acme::Test::Leak::{$subclass.'::'};
redo;
}

This also shows that the problem is not specific to AUTOLOAD. But
creating a sub named OTHERVAR referencing a variable of the same name, I
can create a memory leak.

The body of the sub is referencing the *OTHERVAR glob, which references
the sub. So this is a circular reference.

So you are saying that

use vars qw(\$$varname );
sub $subname {
print \$$varname;
};

produces a closure, but it should not, as $varname is a dynamic
variable. The sub should not be a closure at all, and there should be
no circular references, as dynamic variables should be looked up fresh
every time, and should not act as a closure on the glob that holds
them.

cheers
Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

p5pRT · 2012-02-13T10:07:26Z

From zefram@fysh.org

demerphq wrote:

produces a closure, but it should not,

It's not a closure in the sense of lexical variables. It's a non-cloning
sub, but it contains a reference to the glob containing the variable,
which is also the same glob that contains the sub.

                      dynamic variables should be looked up fresh

every time,

The scalar variable *is* looked up afresh every time. "local" can be used
to temporarily replace it with a different variable, which the sub will
see. It's the *glob* that's not looked up afresh. Look at the op tree.

-zefram

p5pRT · 2012-02-13T10:33:38Z

From @demerphq

On 13 February 2012 11:06, Zefram <zefram@fysh.org> wrote:

demerphq wrote:

produces a closure, but it should not,

It's not a closure in the sense of lexical variables. It's a non-cloning
sub, but it contains a reference to the glob containing the variable,
which is also the same glob that contains the sub.

Then it is a closure on the glob. Which is wrong. It should look up
the glob fresh every time (more or less).

dynamic variables should be looked up fresh
every time,

The scalar variable *is* looked up afresh every time. "local" can be used
to temporarily replace it with a different variable, which the sub will
see. It's the *glob* that's not looked up afresh. Look at the op tree.

That may be, but that it does so is a bug.

Consider the following evil script and how it behaves and how I think
it should behave. Do you disagree with what I think the expected
output should be?

package Foo;

sub AUTOLOAD {
print "Orig AUTOLOAD: $AUTOLOAD\n";
}

my $o= bless {};

$o->testing;

my $g l o b = d e l e t e$ ::Foo::{AUTOLOAD};

print $glob,"\n";

my $sub= *$glob{CODE};

if (0) {
# this does not work
*AUTOLOAD= sub {
print "New AUTOLOAD: $AUTOLOAD\n";
$sub->();
};
} else {
#but this does - but it produces buggy output
eval q[
sub AUTOLOAD {
print "New AUTOLOAD: $AUTOLOAD\n";
$sub->();
};
];
}

$o->testing_again;

__END__
"Does not work" output:

Can't locate object method "testing_again" via package "Foo" at - line 34.
Orig AUTOLOAD: Foo::testing
*Foo::AUTOLOAD

"Does work, but buggy" output:

Orig AUTOLOAD: Foo::testing
*Foo::AUTOLOAD
New AUTOLOAD: Foo::testing_again
Orig AUTOLOAD: Foo::testing
New AUTOLOAD: Foo::DESTROY
Orig AUTOLOAD: Foo::testing

Expected output:

Orig AUTOLOAD: Foo::testing
*Foo::AUTOLOAD
New AUTOLOAD: Foo::testing_again
Orig AUTOLOAD: Foo::testing_again
New AUTOLOAD: Foo::DESTROY
Orig AUTOLOAD: Foo::DESTROY

--
perl -Mre=debug -e "/just|another|perl|hacker/"

p5pRT · 2012-02-13T10:35:27Z

From jtbraun@CPAN.org

On 02/11/2012 08:26 AM, James E Keenan via RT wrote:

How should I modify the 'ps' line in your code for systems other than Linux?

I'm afraid that I don't know the answer to your question. The 'ps' line
in my test was lifted from a bug report filed against Parse::RecDescent,
which I maintain. I know that ps varies across *NIX implementations, so
my best advice is "check the manpage for ps on the system you want to
test on." Not helpful, I know.

Jeremy

p5pRT · 2012-02-13T11:53:42Z

From @demerphq

On 12 February 2012 06:56, <jtbraun@cpan.org> wrote:

On 02/11/2012 08:26 AM, James E Keenan via RT wrote:

How should I modify the 'ps' line in your code for systems other than
Linux?

I'm afraid that I don't know the answer to your question. The 'ps' line in
my test was lifted from a bug report filed against Parse::RecDescent, which

Digging further with a modified version of the original script using a
symbolic global ${"AUTOLOAD"} does not leak, and behaves as I expect
(see my other posts in this thread).

I personally thought that ${"AUTOLOAD"} and ${AUTOLOAD} and $AUTOLOAD
should all be the same thing, but apparently the latter two are
optimized in such a way that they break(?) when stash deletion is
involved.

So there is a workaround for this. The question is whether or not it
is a "wont-fix" or a "not-a-bug". I think it is a wont-fix (although
if the reason someone needed to do this was good and the work around
was not suitable I might say "lets-fix-if-we-can"), and Zefram seems
to favor "not-a-bug" as he doesn't think we make any strong
commitments about manipulating the stash.

But for your case, generating code in Parse::RecDescent, it seems to
me you can use the workaround right?

Yves

$ cat autoload_leak.pl; perl autoload_leak.pl
use warnings;
use strict;

my %seen;
my $i = 0;
for my $varname ('OTHERVAR', "{'AUTOLOAD'}", "{AUTOLOAD}", 'AUTOLOAD') {
for my $subname ( #'OTHERSUB',
'AUTOLOAD') {
for my $n ( 1 .. 400 ) {
++$i;
my $superclass = 'Acme::Test::Leak';
my $subclass = sprintf("namespace%06d", $i);
my $package = $superclass . '::' . $subclass;
my $code = <<EOFCODE;
package $package;
no strict;
no warnings;
sub $subname {
print "$package => \$$varname\\n" if \$n == 1; # \$n is $n
};
if ("$subname" eq "AUTOLOAD" and "$varname"=~/AUTOLOAD/) {
my \$o= bless {};
\$o->test_autoload
}
1;
EOFCODE
print $code,"\n" unless $seen{"$varname|$subname"}++;
eval $code
or die "$code:$@";
# remove references to the namespace
delete $Acme::Test::Leak::{$subclass.'::'};

my $m e m = ‘ p s h - o s i z e$ $`;
chomp($mem);
printf( "v:%s s:%s n:%3d :: %.1f\r", $varname, $subname,
$n, $mem / 1024 );
print "\n" unless $n % 50;
}
print "\n";
}
}
__END__
package Acme::Test::Leak::namespace000001;
no strict;
no warnings;
sub AUTOLOAD {
print "Acme::Test::Leak::namespace000001 => $OTHERVAR\n" if $n ==
1; # $n is 1
};
if ("AUTOLOAD" eq "AUTOLOAD" and "OTHERVAR"=~/AUTOLOAD/) {
my $o= bless {};
$o->test_autoload
}
1;

v:OTHERVAR s:AUTOLOAD n: 50 :: 0.6
v:OTHERVAR s:AUTOLOAD n:100 :: 0.6
v:OTHERVAR s:AUTOLOAD n:150 :: 0.6
v:OTHERVAR s:AUTOLOAD n:200 :: 0.6
v:OTHERVAR s:AUTOLOAD n:250 :: 0.6
v:OTHERVAR s:AUTOLOAD n:300 :: 0.6
v:OTHERVAR s:AUTOLOAD n:350 :: 0.6
v:OTHERVAR s:AUTOLOAD n:400 :: 0.6

package Acme::Test::Leak::namespace000401;
no strict;
no warnings;
sub AUTOLOAD {
print "Acme::Test::Leak::namespace000401 => ${'AUTOLOAD'}\n" if
$n == 1; # $n is 1
};
if ("AUTOLOAD" eq "AUTOLOAD" and "{'AUTOLOAD'}"=~/AUTOLOAD/) {
my $o= bless {};
$o->test_autoload
}
1;

Acme::Test::Leak::namespace000401 =>
Acme::Test::Leak::namespace000401::test_autoload
Acme::Test::Leak::namespace000401 => Acme::Test::Leak::namespace000401::DESTROY
v:{'AUTOLOAD'} s:AUTOLOAD n: 50 :: 0.6
v:{'AUTOLOAD'} s:AUTOLOAD n:100 :: 0.6
v:{'AUTOLOAD'} s:AUTOLOAD n:150 :: 0.6
v:{'AUTOLOAD'} s:AUTOLOAD n:200 :: 0.6
v:{'AUTOLOAD'} s:AUTOLOAD n:250 :: 0.6
v:{'AUTOLOAD'} s:AUTOLOAD n:300 :: 0.6
v:{'AUTOLOAD'} s:AUTOLOAD n:350 :: 0.6
v:{'AUTOLOAD'} s:AUTOLOAD n:400 :: 0.6

package Acme::Test::Leak::namespace000801;
no strict;
no warnings;
sub AUTOLOAD {
print "Acme::Test::Leak::namespace000801 => ${AUTOLOAD}\n" if $n
== 1; # $n is 1
};
if ("AUTOLOAD" eq "AUTOLOAD" and "{AUTOLOAD}"=~/AUTOLOAD/) {
my $o= bless {};
$o->test_autoload
}
1;

Acme::Test::Leak::namespace000801 =>
Acme::Test::Leak::namespace000801::test_autoload
Acme::Test::Leak::namespace000801 => Acme::Test::Leak::namespace000801::DESTROY
v:{AUTOLOAD} s:AUTOLOAD n: 50 :: 0.7
v:{AUTOLOAD} s:AUTOLOAD n:100 :: 0.8
v:{AUTOLOAD} s:AUTOLOAD n:150 :: 1.0
v:{AUTOLOAD} s:AUTOLOAD n:200 :: 1.0
v:{AUTOLOAD} s:AUTOLOAD n:250 :: 1.1
v:{AUTOLOAD} s:AUTOLOAD n:300 :: 1.2
v:{AUTOLOAD} s:AUTOLOAD n:350 :: 1.3
v:{AUTOLOAD} s:AUTOLOAD n:400 :: 1.3

package Acme::Test::Leak::namespace001201;
no strict;
no warnings;
sub AUTOLOAD {
print "Acme::Test::Leak::namespace001201 => $AUTOLOAD\n" if $n ==
1; # $n is 1
};
if ("AUTOLOAD" eq "AUTOLOAD" and "AUTOLOAD"=~/AUTOLOAD/) {
my $o= bless {};
$o->test_autoload
}
1;

Acme::Test::Leak::namespace001201 =>
Acme::Test::Leak::namespace001201::test_autoload
Acme::Test::Leak::namespace001201 => Acme::Test::Leak::namespace001201::DESTROY
v:AUTOLOAD s:AUTOLOAD n: 50 :: 1.5
v:AUTOLOAD s:AUTOLOAD n:100 :: 1.6
v:AUTOLOAD s:AUTOLOAD n:150 :: 1.7
v:AUTOLOAD s:AUTOLOAD n:200 :: 1.7
v:AUTOLOAD s:AUTOLOAD n:250 :: 1.8
v:AUTOLOAD s:AUTOLOAD n:300 :: 2.0
v:AUTOLOAD s:AUTOLOAD n:350 :: 2.1
v:AUTOLOAD s:AUTOLOAD n:400 :: 2.1

--
perl -Mre=debug -e "/just|another|perl|hacker/"

p5pRT · 2012-02-13T12:05:17Z

From @nwc10

On Mon, Feb 13, 2012 at 12:53:08PM +0100, demerphq wrote:

Digging further with a modified version of the original script using a
symbolic global ${"AUTOLOAD"} does not leak, and behaves as I expect
(see my other posts in this thread).

I personally thought that ${"AUTOLOAD"} and ${AUTOLOAD} and $AUTOLOAD
should all be the same thing, but apparently the latter two are
optimized in such a way that they break(?) when stash deletion is
involved.

To clarify, when you write ${"AUTOLOAD"} you mean that syntax with a
string constant? (Distinct from that syntax with an expression whose
value isn't known until runtime)

So there is a workaround for this. The question is whether or not it
is a "wont-fix" or a "not-a-bug". I think it is a wont-fix (although
if the reason someone needed to do this was good and the work around
was not suitable I might say "lets-fix-if-we-can"), and Zefram seems
to favor "not-a-bug" as he doesn't think we make any strong
commitments about manipulating the stash.

I understand why this is a difference. I also understand what the
implementation is doing that is causing this problem (at compile time,
doing the typeglob lookup for *AUTOLOAD and putting a reference to that
typeglob in the optree), and I'm really not sure which view is right.

I guess it comes down to the problem that Perl 5 isn't so much a design
as an implementation. And for any piece of emergent behaviour (such as this)
it's hard to know whether it's the implementation diverging from the
design, or the implementation matching the intent of the design.

Nicholas Clark

p5pRT · 2012-02-13T12:16:35Z

From @demerphq

On 13 February 2012 13:04, Nicholas Clark <nick@ccl4.org> wrote:

On Mon, Feb 13, 2012 at 12:53:08PM +0100, demerphq wrote:

Digging further with a modified version of the original script using a
symbolic global ${"AUTOLOAD"} does not leak, and behaves as I expect
(see my other posts in this thread).

I personally thought that ${"AUTOLOAD"} and ${AUTOLOAD} and $AUTOLOAD
should all be the same thing, but apparently the latter two are
optimized in such a way that they break(?) when stash deletion is
involved.

To clarify, when you write ${"AUTOLOAD"} you mean that syntax with a
string constant? (Distinct from that syntax with an expression whose
value isn't known until runtime)

Yes, that syntax with a string constant. I thought that ${foo} was
just the bareword equivalent of ${"foo"}.

So there is a workaround for this. The question is whether or not it
is a "wont-fix" or a "not-a-bug". I think it is a wont-fix (although
if the reason someone needed to do this was good and the work around
was not suitable I might say "lets-fix-if-we-can"), and Zefram seems
to favor "not-a-bug" as he doesn't think we make any strong
commitments about manipulating the stash.

I understand why this is a difference. I also understand what the
implementation is doing that is causing this problem (at compile time,
doing the typeglob lookup for *AUTOLOAD and putting a reference to that
typeglob in the optree), and I'm really not sure which view is right.

To me, this feels like an optimisation gotchya, meaning that at
somepoint it worked right, and the glob wasnt stored in the optree,
and then it was optimised to be faster, and since people dont delete
stash entries often nobody noticed... But I havent checked the log to
see if that is true at all.

I guess it comes down to the problem that Perl 5 isn't so much a design
as an implementation. And for any piece of emergent behaviour (such as this)
it's hard to know whether it's the implementation diverging from the
design, or the implementation matching the intent of the design.

I guess this is the kind of thing that probably Larry should decide on.

I can't help but wonder if the answer to this could have deeper
implications, but I can't put my finger on what they are. Tail
optimization for some reason comes to mind although again I cant quite
say why and maybe it is not relevant.

cheers,
Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

p5pRT · 2012-02-13T14:07:06Z

From zefram@fysh.org

demerphq wrote:

So there is a workaround for this. The question is whether or not it
is a "wont-fix" or a "not-a-bug".

To be clear, I think $foo doing glob lookup at compile time is not-a-bug,
but $foo acting as a stealth reference to &foo is won't-fix.

-zefram

p5pRT · 2012-02-13T14:39:04Z

From @nwc10

On Mon, Feb 13, 2012 at 01:15:41PM +0100, demerphq wrote:

On 13 February 2012 13:04, Nicholas Clark <nick@ccl4.org> wrote:

I understand why this is a difference. I also understand what the
implementation is doing that is causing this problem (at compile time,
doing the typeglob lookup for *AUTOLOAD and putting a reference to that
typeglob in the optree), and I'm really not sure which view is right.

To me, this feels like an optimisation gotchya, meaning that at
somepoint it worked right, and the glob wasnt stored in the optree,
and then it was optimised to be faster, and since people dont delete
stash entries often nobody noticed... But I havent checked the log to
see if that is true at all.

I don't think it ever has.

$ ./perl -Ilib -le '$foo = "old"; sub p {print

Extra close brace or missing open brace

$foo;} &amp;p; delete $

::{"foo"}; eval q{$foo = "new"; print $foo}; &p'
old
new
old

$ ~/Sandpit/5000/bin/perl -le '$foo = "old"; sub p {print

Extra close brace or missing open brace

$foo;} &amp;p; delete $

::{"foo"}; eval q{$foo = "new"; print $foo}; &p'
old
new
old

$ /usr/local/perl4/bin/perl4.036 -le '$foo = "old"; sub p {print $foo;} &p; delete $_main{"foo"}; eval q{$foo = "new"; print $foo}; &p'
old
new
old

And you learn something new (or old, or obsolete), every day. Perl 4:

The symbol table for a package happens to be stored in the associative
array of that name prepended with an underscore. The value in each
entry of the associative array is what you are referring to when you
use the *name notation.

I guess it comes down to the problem that Perl 5 isn't so much a design
as an implementation. And for any piece of emergent behaviour (such as this)
it's hard to know whether it's the implementation diverging from the
design, or the implementation matching the intent of the design.

I guess this is the kind of thing that probably Larry should decide on.

Larry eliminated typeglobs very early in the design of Perl 6.

That comment might seem at first reading to be an irrelevant or facetious
response, but it's not meant to be. It's an educated guess.

Nicholas Clark

p5pRT · 2012-02-14T00:09:49Z

From @cpansprout

On Mon Feb 13 03:53:42 2012, demerphq wrote:

So there is a workaround for this. The question is whether or not it
is a "wont-fix" or a "not-a-bug". I think it is a wont-fix (although
if the reason someone needed to do this was good and the work around
was not suitable I might say "lets-fix-if-we-can"), and Zefram seems
to favor "not-a-bug" as he doesn't think we make any strong
commitments about manipulating the stash.

There are at least half a dozen CPAN modules that rely on op-to-glob
binding, and dozens more that depend on them.

--

Father Chrysostomos

p5pRT · 2012-02-14T09:36:39Z

From jtbraun@theweirdhotel.net

Yes, I can use the workaround. Thank you all very much.

Jeremy

On Feb 13, 2012, at 3:53 AM, demerphq <demerphq@gmail.com> wrote:

But for your case, generating code in Parse::RecDescent, it seems to
me you can use the workaround right?

p5pRT · 2012-04-24T05:43:05Z

From @cpansprout

On Mon Feb 13 06:07:06 2012, zefram@fysh.org wrote:

$foo acting as a stealth reference to &foo is won't-fix.

I’ll mark it as such, then.

--

Father Chrysostomos

p5pRT · 2012-04-24T05:43:06Z

From [Unknown Contact. See original ticket]

On Mon Feb 13 06:07:06 2012, zefram@fysh.org wrote:

$foo acting as a stealth reference to &foo is won't-fix.

I’ll mark it as such, then.

--

Father Chrysostomos

p5pRT · 2012-04-24T05:43:06Z

@cpansprout - Status changed from 'open' to 'rejected'

p5pRT closed this as completed Apr 24, 2012

p5pRT added Severity Medium distro-Linux type-core wontfix labels Oct 19, 2019

p5pRT mentioned this issue Oct 19, 2019

silence warnings in build and test #12272

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

$AUTOLOAD inside of sub AUTOLOAD{} appears to create a memory leak #11949

$AUTOLOAD inside of sub AUTOLOAD{} appears to create a memory leak #11949

p5pRT commented Feb 9, 2012

p5pRT commented Feb 9, 2012

p5pRT commented Feb 11, 2012

p5pRT commented Feb 11, 2012

p5pRT commented Feb 12, 2012

p5pRT commented Feb 13, 2012

p5pRT commented Feb 13, 2012

p5pRT commented Feb 13, 2012

p5pRT commented Feb 13, 2012

p5pRT commented Feb 13, 2012

p5pRT commented Feb 13, 2012

p5pRT commented Feb 13, 2012

p5pRT commented Feb 13, 2012

p5pRT commented Feb 13, 2012

p5pRT commented Feb 14, 2012

p5pRT commented Feb 14, 2012

p5pRT commented Apr 24, 2012

p5pRT commented Apr 24, 2012

p5pRT commented Apr 24, 2012

$AUTOLOAD inside of sub AUTOLOAD{} appears to create a memory leak #11949

$AUTOLOAD inside of sub AUTOLOAD{} appears to create a memory leak #11949

Comments

p5pRT commented Feb 9, 2012

p5pRT commented Feb 9, 2012

From jtbraun@CPAN.org

Created by jtbraun@CPAN.org

p5pRT commented Feb 11, 2012

From @jkeenan

p5pRT commented Feb 11, 2012

p5pRT commented Feb 12, 2012

From @cpansprout

p5pRT commented Feb 13, 2012

From @demerphq

p5pRT commented Feb 13, 2012

From zefram@fysh.org

p5pRT commented Feb 13, 2012

From @demerphq

p5pRT commented Feb 13, 2012

From jtbraun@CPAN.org

p5pRT commented Feb 13, 2012

From @demerphq

p5pRT commented Feb 13, 2012

From @nwc10

p5pRT commented Feb 13, 2012

From @demerphq

p5pRT commented Feb 13, 2012

From zefram@fysh.org

p5pRT commented Feb 13, 2012

From @nwc10

p5pRT commented Feb 14, 2012

From @cpansprout

p5pRT commented Feb 14, 2012

From jtbraun@theweirdhotel.net

p5pRT commented Apr 24, 2012

From @cpansprout

p5pRT commented Apr 24, 2012

From [Unknown Contact. See original ticket]

p5pRT commented Apr 24, 2012