From vincent@vinc17.net

Created by vincent@vinc17.net

perl 5.12 consumes all the memory on​:

  perl -e "open FILE, '<', \*STDIN or die; <FILE>;"

perl 5.10 didn't have this problem.

Reproducible under both Mac OS X (this bug report) and GNU/Linux
(Debian), for which I also reported a bug​:

  http​://bugs.debian.org/cgi-bin/bugreport.cgi?bug=629363

Flags:
    category=core
    severity=high

Site configuration information for perl 5.12.3:

Configured by vinc17 at Fri Apr  1 23:58:52 CEST 2011.

Summary of my perl5 (revision 5 version 12 subversion 3) configuration:
   
  Platform:
    osname=darwin, osvers=8.11.0, archname=darwin-multi-2level
    uname='darwin prunille.vinc17.org 8.11.0 darwin kernel version 8.11.0: wed oct 10 18:26:00 pdt 2007; root:xnu-792.24.17~1release_ppc power macintosh powerpc '
    config_args='-D inc_version_list=5.12.2/darwin-multi-2level 5.12.2 5.12.1/darwin-multi-2level 5.12.1 5.12.0/darwin-multi-2level 5.12.0 -des -Dprefix=/opt/local -Dscriptdir=/opt/local/bin -Dcppflags=-I/opt/local/include -Dccflags=-pipe -O2 -arch ppc -Dldflags=-L/opt/local/lib -arch ppc -Dvendorprefix=/opt/local -Dusemultiplicity=y -D cc=/usr/bin/gcc-4.0 -D ld=/usr/bin/gcc-4.0 -D man1ext=1pm -D man3ext=3pm -D man1dir=/opt/local/share/man/man1p -D man3dir=/opt/local/share/man/man3p -D siteman1dir=/opt/local/share/man/man1 -D siteman3dir=/opt/local/share/man/man3 -D vendorman1dir=/opt/local/share/man/man1 -D vendorman3dir=/opt/local/share/man/man3 -D pager=/usr/bin/less -sR'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='/usr/bin/gcc-4.0', ccflags ='-pipe -O2 -arch ppc -fno-common -DPERL_DARWIN -I/opt/local/include -no-cpp-precomp -fno-strict-aliasing -I/opt/local/include',
    optimize='-O3',
    cppflags='-I/opt/local/include -no-cpp-precomp -pipe -O2 -arch ppc -fno-common -DPERL_DARWIN -I/opt/local/include -no-cpp-precomp -fno-strict-aliasing -I/opt/local/include'
    ccversion='', gccversion='4.0.1 (Apple Computer, Inc. build 5370)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='env MACOSX_DEPLOYMENT_TARGET=10.3 /usr/bin/gcc-4.0', ldflags ='-L/opt/local/lib -arch ppc'
    libpth=/opt/local/lib /usr/lib
    libs=-lgdbm -ldbm -ldl -lm -lc
    perllibs=-ldl -lm -lc
    libc=/usr/lib/libc.dylib, so=dylib, useshrplib=false, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-L/opt/local/lib -arch ppc -bundle -undefined dynamic_lookup'

Locally applied patches:
    


@INC for perl 5.12.3:
    /Users/vinc17/lib/site_perl
    /opt/local/lib/perl5/vendor_perl/5.12.3/darwin-multi-2level
    /opt/local/lib/perl5/vendor_perl/5.12.3
    /opt/local/lib/perl5/vendor_perl
    /opt/local/lib/perl5/site_perl/5.12.3/darwin-multi-2level
    /opt/local/lib/perl5/site_perl/5.12.3
    /opt/local/lib/perl5/vendor_perl/5.12.3/darwin-multi-2level
    /opt/local/lib/perl5/vendor_perl/5.12.3
    /opt/local/lib/perl5/5.12.3/darwin-multi-2level
    /opt/local/lib/perl5/5.12.3
    /opt/local/lib/perl5/site_perl
    /opt/local/lib/perl5/vendor_perl
    .


Environment for perl 5.12.3:
    DYLD_LIBRARY_PATH (unset)
    HOME=/Users/vinc17
    LANG=POSIX
    LANGUAGE (unset)
    LC_CHARMAP=ISO8859-1
    LC_COLLATE=POSIX
    LC_CTYPE=en_US.ISO8859-1
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/Users/vinc17/bin:/usr/local/bin:/opt/local/bin:/usr/bin:/bin:/usr/local/sbin:/opt/local/sbin:/usr/sbin:/sbin:/usr/X11R6/bin:.
    PERL5LIB=/Users/vinc17/lib/site_perl:/opt/local/lib/perl5/vendor_perl
    PERL_BADLANG (unset)
    SHELL=/opt/local/bin/zsh

From @cpansprout

On Sun Jun 05 15​:12​:28 2011, vincent@​vinc17.net wrote​:

perl 5.12 consumes all the memory on​:

perl -e "open FILE, '<', \*STDIN or die; <FILE>;"

perl 5.10 didn't have this problem.

If you are trying to duplicate STDIN, you need to use '<&' rather than '>'​:

$ echo "foo" | perl5.10.1 -le 'open FILE, "<&", \*STDIN or die $!; print
$_ while <FILE>;'
foo

If you use '<', on a reference, it’s meant to read the scalar, or fall
back to treating it as a file name if it’s not a scalar reference, but
in 5.10 and 5.12 there is a bug that will read random memory (which just
happens to be the glob name) if it’s a glob reference​:

$ perl -le 'open FILE, "<", \*STDIN or die $!; print $_ while <FILE>;'
*main​::STD

$ perl5.12.0 -le 'open FILE, "<", \*STDIN or die $!; print $_ while <FILE>;'
*m

It hangs for me, too, in 5.12.2. Presumably it’s trying to read one huge
line of random memory.

This was fixed in 5.14.0, which restores the 5.8.x behaviour​:

$ perl5.8.8 -le 'open FILE, "<", \*STDIN or die $!; print $_ while <FILE>;'
No such file or directory at -e line 1.
$ perl5.14.0 -le 'open FILE, "<", \*STDIN or die $!; print $_ while <FILE>;'
No such file or directory at -e line 1.

The RT System itself - Status changed from 'new' to 'open'

@cpansprout - Status changed from 'open' to 'resolved'

From @cpansprout

On Sun Jun 05 18​:05​:41 2011, sprout wrote​:

On Sun Jun 05 15​:12​:28 2011, vincent@​vinc17.net wrote​:

perl 5.12 consumes all the memory on​:

perl -e "open FILE, '<', \*STDIN or die; <FILE>;"

perl 5.10 didn't have this problem.

If you are trying to duplicate STDIN, you need to use '<&' rather than
'>'​:

$ echo "foo" | perl5.10.1 -le 'open FILE, "<&", \*STDIN or die $!; print
$_ while <FILE>;'
foo

If you use '<', on a reference, it’s meant to read the scalar, or fall
back to treating it as a file name if it’s not a scalar reference, but
in 5.10 and 5.12 there is a bug that will read random memory (which just
happens to be the glob name) if it’s a glob reference​:

$ perl -le 'open FILE, "<", \*STDIN or die $!; print $_ while <FILE>;'
*main​::STD

$ perl5.12.0 -le 'open FILE, "<", \*STDIN or die $!; print $_ while
<FILE>;'
*m

It hangs for me, too, in 5.12.2. Presumably it’s trying to read one huge
line of random memory.

This was fixed in 5.14.0, which restores the 5.8.x behaviour​:

$ perl5.8.8 -le 'open FILE, "<", \*STDIN or die $!; print $_ while
<FILE>;'
No such file or directory at -e line 1.
$ perl5.14.0 -le 'open FILE, "<", \*STDIN or die $!; print $_ while
<FILE>;'
No such file or directory at -e line 1.

Having said that, opening a glob copy (which is treated as a regular
scalar) still causes the hang and memory consumption, even in 5.14.0, so
I’m reopening this ticket.

$ perl5.14.0 -le 'open FILE, "<", \($f=*STDIN) or die $!; print $_ while
<FILE>;'

@cpansprout - Status changed from 'resolved' to 'open'

From @cpansprout

On Sun Jun 05 19​:38​:00 2011, sprout wrote​:

On Sun Jun 05 18​:05​:41 2011, sprout wrote​:

On Sun Jun 05 15​:12​:28 2011, vincent@​vinc17.net wrote​:

perl 5.12 consumes all the memory on​:

perl -e "open FILE, '<', \*STDIN or die; <FILE>;"

perl 5.10 didn't have this problem.

If you are trying to duplicate STDIN, you need to use '<&' rather than
'>'​:

$ echo "foo" | perl5.10.1 -le 'open FILE, "<&", \*STDIN or die $!; print
$_ while <FILE>;'
foo

If you use '<', on a reference, it’s meant to read the scalar, or fall
back to treating it as a file name if it’s not a scalar reference, but
in 5.10 and 5.12 there is a bug that will read random memory (which just
happens to be the glob name) if it’s a glob reference​:

$ perl -le 'open FILE, "<", \*STDIN or die $!; print $_ while <FILE>;'
*main​::STD

$ perl5.12.0 -le 'open FILE, "<", \*STDIN or die $!; print $_ while
<FILE>;'
*m

It hangs for me, too, in 5.12.2. Presumably it’s trying to read one huge
line of random memory.

This was fixed in 5.14.0, which restores the 5.8.x behaviour​:

$ perl5.8.8 -le 'open FILE, "<", \*STDIN or die $!; print $_ while
<FILE>;'
No such file or directory at -e line 1.
$ perl5.14.0 -le 'open FILE, "<", \*STDIN or die $!; print $_ while
<FILE>;'
No such file or directory at -e line 1.

Having said that, opening a glob copy (which is treated as a regular
scalar) still causes the hang and memory consumption, even in 5.14.0, so
I’m reopening this ticket.

$ perl5.14.0 -le 'open FILE, "<", \($f=*STDIN) or die $!; print $_ while
<FILE>;'

With debugging turned on, I get an assertion failure​:

Assertion failed​: (!isGV_with_GP(_svcur)), function
PerlIOScalar_get_cnt, file scalar.xs, line 244.

That should make it easy to fix....

From @cpansprout

On Sun Jun 05 22​:06​:44 2011, sprout wrote​:

On Sun Jun 05 19​:38​:00 2011, sprout wrote​:

On Sun Jun 05 18​:05​:41 2011, sprout wrote​:

On Sun Jun 05 15​:12​:28 2011, vincent@​vinc17.net wrote​:

perl 5.12 consumes all the memory on​:

perl -e "open FILE, '<', \*STDIN or die; <FILE>;"

perl 5.10 didn't have this problem.

...[O]pening a glob copy (which is treated as a regular
scalar) still causes the hang and memory consumption, even in 5.14.0, so
I’m reopening this ticket.

$ perl5.14.0 -le 'open FILE, "<", \($f=*STDIN) or die $!; print $_ while
<FILE>;'

With debugging turned on, I get an assertion failure​:

Assertion failed​: (!isGV_with_GP(_svcur)), function
PerlIOScalar_get_cnt, file scalar.xs, line 244.

That should make it easy to fix....

This is now fixed with commit 71edc89.

Is there any chance this, too, could be backported to 5.14.1?

@cpansprout - Status changed from 'open' to 'resolved'

From @cpansprout

On Sun Jun 05 22​:40​:16 2011, sprout wrote​:

On Sun Jun 05 22​:06​:44 2011, sprout wrote​:

On Sun Jun 05 19​:38​:00 2011, sprout wrote​:

On Sun Jun 05 18​:05​:41 2011, sprout wrote​:

On Sun Jun 05 15​:12​:28 2011, vincent@​vinc17.net wrote​:

perl 5.12 consumes all the memory on​:

perl -e "open FILE, '<', \*STDIN or die; <FILE>;"

perl 5.10 didn't have this problem.

...[O]pening a glob copy (which is treated as a regular
scalar) still causes the hang and memory consumption, even in
5.14.0, so
I’m reopening this ticket.

$ perl5.14.0 -le 'open FILE, "<", \($f=*STDIN) or die $!; print $_
while
<FILE>;'

With debugging turned on, I get an assertion failure​:

Assertion failed​: (!isGV_with_GP(_svcur)), function
PerlIOScalar_get_cnt, file scalar.xs, line 244.

That should make it easy to fix....

This is now fixed with commit 71edc89.

Is there any chance this, too, could be backported to 5.14.1?

I forgot to mention​: This never really worked properly (strings read
from the handle would be cut off and some random point), but it didn’t
start hanging until 5.12.2 and 5.14.0, so in that sense it’s a regression.

From [Unknown Contact. See original ticket]

On Sun Jun 05 22​:40​:16 2011, sprout wrote​:

On Sun Jun 05 22​:06​:44 2011, sprout wrote​:

On Sun Jun 05 19​:38​:00 2011, sprout wrote​:

On Sun Jun 05 18​:05​:41 2011, sprout wrote​:

On Sun Jun 05 15​:12​:28 2011, vincent@​vinc17.net wrote​:

perl 5.12 consumes all the memory on​:

perl -e "open FILE, '<', \*STDIN or die; <FILE>;"

perl 5.10 didn't have this problem.

...[O]pening a glob copy (which is treated as a regular
scalar) still causes the hang and memory consumption, even in
5.14.0, so
I’m reopening this ticket.

$ perl5.14.0 -le 'open FILE, "<", \($f=*STDIN) or die $!; print $_
while
<FILE>;'

With debugging turned on, I get an assertion failure​:

Assertion failed​: (!isGV_with_GP(_svcur)), function
PerlIOScalar_get_cnt, file scalar.xs, line 244.

That should make it easy to fix....

This is now fixed with commit 71edc89.

Is there any chance this, too, could be backported to 5.14.1?

I forgot to mention​: This never really worked properly (strings read
from the handle would be cut off and some random point), but it didn’t
start hanging until 5.12.2 and 5.14.0, so in that sense it’s a regression.

From vincent@vinc17.net

On 2011-06-05 18​:05​:41 -0700, Father Chrysostomos via RT wrote​:

If you are trying to duplicate STDIN, you need to use '<&' rather than '>'​:

$ echo "foo" | perl5.10.1 -le 'open FILE, "<&", \*STDIN or die $!; print
$_ while <FILE>;'
foo

Yes, but the main problem is the memory consumption, as my machine
didn't respond any more and I had to reboot it.

I tried '<' to see whether this worked (I couldn't see any reason
why this wouldn't be supported) because it could have slightly
simplified my script, with​:

  open FILE, '<', defined $file ? $file : \*STDIN or die;

--
Vincent Lefèvre <vincent@​vinc17.net> - Web​: <http​://www.vinc17.net/>
100% accessible validated (X)HTML - Blog​: <http​://www.vinc17.net/blog/>
Work​: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)

From @cpansprout

On Aug 28, 2011, at 10​:26 AM, Jesse Vincent wrote​:

71edc89 [perl #92258] <$fh> hangs on a glob copy

Zefram feels that this issues is too contrived to justify any backport

Also, the new behavior doesn't match the pre-broken behavior

Actually it does.

# 5.12 (osx)
(blead=10781 pod/) extracto ~/git/perl$ perl -e "open FILE, '<', \*STDIN or die; print <FILE>;"
^C
# 5.10 (osx)
130 (blead=10781 pod/) extracto ~/git/perl$ perl5.10 -e "open FILE, '<', \*STDIN or die; print <FILE>;"
*main​::STD

5.10 is what broke it originally. What you are seeing there is memory corruption. What happened to the IN in STDIN?

$ perl5.10.1 -le "open FILE, '<', \*a​::a or die; print <FILE>;"
*a​::a� �F_

That one’s reading past the glob name into random memory.

# blead
(blead=10781 pod/) extracto ~/git/perl$ ./perl -e "open FILE, '<', \*STDIN or die; print <FILE>;"
Died at -e line 1.

Same in 5.8.

But your one-liner doesn’t trigger the exact bug that 71edc89 is fixing.

Your one-liner is fixed in 5.14​:
$ perl5.14.0 -e "open FILE, '<', \*STDIN or die; print <FILE>;"

But glob copies were missed​:
$ perl5.14.0 -e 'open FILE, "<", \($f=*STDIN) or die; print <FILE>;'
^C

Both one-liners behave the same way in all previous versions.

These two go together, as the first one only partially fixed the bug​:

d34a666 [perl #91880] $_ refcounting problems in @​INC filters
339c6c6 Follow-up to d34a666

Zefram doesn't feel like this sounds serious enough for a backport

Hugo is concerned by the lack of connection between the patch and the writuep. In particular, it is unclear what the temp fiddling is trying to do, and whether it's doing it right.

The ‘whether it’s doing it right’ part makes me laugh. Several parts of the perl core weren’t ‘doing it right’, resulting in some of the bugs I’ve fixed lately.

In this particular case, that patch turns off the temp flag to avoid perpetuating an existing bug​: pp_return and pp_leavesub return as-is anything marked TEMP, instead of copying it the way they are supposed to.

Later in blead I fixed that return bug and removed the SvTEMP_off that that patch added.

It could also be backported without the SvTEMP_off, and with some of the tests removed.

These four need some explanation​:

a991bd3 Make Storable freeze COWs properly (depends on 3e)

We don't feel that this is significant enough to backport.

d5c6c69 Fix a Storable test to work in 5.8.9- (if you backport a9, you need this to maintain consistency)

No reason to backport on the merits - fixes the tests on ancient versions

To clarify​: The test in a9 relies on the bug that 3e fixes. So 3e depends on d5 iff a9 is backported.

fa819c1 Make SvIsCOW honest about globs
3e89ba1 Make Internals​::SvREADONLY smarter (depends on fa)

Zefram has some issues with 3e and wants time to evaluate it before 5.16. Because of that, this batch of four Storable-related patches probably should _not_ be backported for now.

Again, I'm sorry for the horrific delay in getting this sorted out. I know a couple of our "Don't backport this" items have already been backported. The disposition of those items should likely be discussed on p5p.

Best,
Jesse