Skip Menu |
Report information
Id: 119529
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: eda [at] waniasset.com
Cc:
AdminCc:

Operating System: Linux
PatchStatus: (no value)
Severity: medium
Type: PerlIO
Perl Version: 5.16.3
Fixed In: 5.19.5



Subject: Filehandle opened from ref to ref hangs on reading
Date: Fri, 30 Aug 2013 17:47:45 +0000
To: "perlbug [...] perl.org" <perlbug [...] perl.org>
From: Ed Avis <eda [...] waniasset.com>
Download (untitled) / with headers
text/plain 4.7k
This is a bug report for perl from eda@waniasset.com, generated with the help of perlbug 1.39 running under perl 5.16.3. ----------------------------------------------------------------- [Please describe your issue here] This causes perl to hang chewing CPU: my $x = \42; open my $fh, "<", \$x; my $got = <$fh>; [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=medium --- Site configuration information for perl 5.16.3: Configured by Red Hat, Inc. at Fri May 3 12:10:03 UTC 2013. Summary of my perl5 (revision 5 version 16 subversion 3) configuration: Platform: osname=linux, osvers=2.6.32-358.2.1.el6.x86_64, archname=x86_64-linux-thread-multi uname='linux buildvm-24.phx2.fedoraproject.org 2.6.32-358.2.1.el6.x86_64 #1 smp wed feb 20 12:17:37 est 2013 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Dccdlflags=-Wl,--enable-new-dtags -Dlddlflags=-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wl,-z,relro -DDEBUGGING=-g -Dversion=5.16.3 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl -Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin -Dusesitecustomize' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic', cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.7.2 20121109 (Red Hat 4.7.2-8)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='gcc', ldflags =' -fstack-protector' libpth=/usr/local/lib64 /lib64 /usr/lib64 libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc libc=, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.16' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--enable-new-dtags -Wl,-rpath,/usr/lib64/perl5/CORE' cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wl,-z,relro ' Locally applied patches: --- @INC for perl 5.16.3: /home/eda/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 . --- Environment for perl 5.16.3: HOME=/home/tradingsystems LANG=en_GB.UTF-8 LANGUAGE (unset) LC_COLLATE=C LC_CTYPE=en_GB.UTF-8 LC_MESSAGES=en_GB.UTF-8 LC_MONETARY=en_GB.UTF-8 LC_NUMERIC=en_GB.UTF-8 LC_TIME=en_GB.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/eda/bin:/home/eda/bin:/usr/local/bin:/usr/bin:/sbin:/usr/sbin:/sbin:/usr/sbin PERL5LIB=/home/eda/lib/perl5/ PERL_BADLANG (unset) SHELL=/bin/bash Show quoted text
______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com
______________________________________________________________________
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 107b
On Fri Aug 30 10:48:16 2013, eda@waniasset.com wrote: Show quoted text
> hang chewing CPU:
and RAM, at least under 5.19.3
Subject: Re: [perl #119529] Filehandle opened from ref to ref hangs on reading
Date: Fri, 30 Aug 2013 15:45:55 -0400
To: Perl5 Porters <perl5-porters [...] perl.org>
From: "Matthew Horsfall (alh)" <wolfsage [...] gmail.com>
Download (untitled) / with headers
text/plain 1.2k
On Fri, Aug 30, 2013 at 1:48 PM, Ed Avis <perlbug-followup@perl.org> wrote:
Show quoted text

This causes perl to hang chewing CPU:

    my $x = \42;
    open my $fh, "<", \$x;
    my $got = <$fh>;


This seems to be in sv.c, in Perl_sv_gets, in the loop starting here, at least for me:

8097  for (;;) {
8098       screamer:
8099         if (cnt > 0) {
8100             if (rslen) {
8101                 while (cnt > 0) {                    /* this     |  eat */
8102                     cnt--;
8103                     if ((*bp++ = *ptr++) == rslast)  /* really   |  dust */
8104                         goto thats_all_folks;        /* screams  |  sed :-) */
8105                 }
8106             }
8107             else {
8108                 Copy(ptr, bp, cnt, char);            /* this     |  eat */
8109                 bp += cnt;                           /* screams  |  dust */
8110                 ptr += cnt;                          /* louder   |  sed :-) */
8111                 cnt = 0;
8112                 assert (!shortbuffered);
8113                 goto cannot_be_shortbuffered;
8114             }
8115         }
8116        

I haven't chased it any further. I didn't see it on perl-5.10.1, but I do see it on perl-5.14.2.

-- Matthew Horsfall (alh)
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.3k
On Fri Aug 30 12:46:31 2013, alh wrote: Show quoted text
> On Fri, Aug 30, 2013 at 1:48 PM, Ed Avis <perlbug-followup@perl.org>
wrote: Show quoted text
>
> > > > This causes perl to hang chewing CPU: > > > > my $x = \42; > > open my $fh, "<", \$x; > > my $got = <$fh>; > > > >
> This seems to be in sv.c, in Perl_sv_gets, in the loop starting here, at > least for me: > > 8097 for (;;) { > 8098 screamer: > 8099 if (cnt > 0) { > 8100 if (rslen) { > 8101 while (cnt > 0) { /* this | > eat */ > 8102 cnt--; > 8103 if ((*bp++ = *ptr++) == rslast) /* really | > dust */ > 8104 goto thats_all_folks; /* screams | > sed :-) */ > 8105 } > 8106 } > 8107 else { > 8108 Copy(ptr, bp, cnt, char); /* this | > eat */ > 8109 bp += cnt; /* screams | > dust */ > 8110 ptr += cnt; /* louder | > sed :-) */ > 8111 cnt = 0; > 8112 assert (!shortbuffered); > 8113 goto cannot_be_shortbuffered; > 8114 } > 8115 } > 8116 > > I haven't chased it any further. I didn't see it on perl-5.10.1, but I do > see it on perl-5.14.2. > > -- Matthew Horsfall (alh)
$ ../perl.git/Porting/bisect.pl --start=v5.10.0 --end=v5.12.4 -e 'alarm 1; my $x = \42; open my $fh, "<", \$x; my $got = <$fh>;' ... 30c16bb48be7651b70d0995a08678033f9a5e942 is the first bad commit commit 30c16bb48be7651b70d0995a08678033f9a5e942 Author: David Mitchell <davem@iabyn.com> Date: Wed May 5 22:39:24 2010 +0100 RT 43789: "in memory" files don't call STORE The code in PerlIO-scalar that implements the open $fh, '>' \$buffer feature did not, apart from accidentally, support get/set magic and thus tied buffers. This patch remedies that: mostly by just blindly sprinkling SvGETMAGIC/SvSETMAGIC about, rather than doing any deep analysis and understanding of the code. One main change I did was to add a PerlIOScalar_read() function, rather than rely on the default behaviour (which implements it in terms of PerlIOScalar_get_ptr() etc), since that approach had a tendency to call FETCH multiple times -- Father Chrysostomos
Subject: RE: [perl #119529] Filehandle opened from ref to ref hangs on reading
Date: Mon, 23 Sep 2013 06:59:09 +0000
To: "perlbug-followup [...] perl.org" <perlbug-followup [...] perl.org>
From: Ed Avis <eda [...] waniasset.com>
Download (untitled) / with headers
text/plain 575b
In general I suggest that opening a filehandle giving a reference to a reference doesn't make any sense, and surely didn't do anything useful even before the change identified. It would be better to make it throw an exception. Similarly I suggest that open $fh, '<', \undef should also throw. Show quoted text
______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com
______________________________________________________________________
CC: "perlbug-followup [...] perl.org" <perlbug-followup [...] perl.org>
Subject: Re: [perl #119529] Filehandle opened from ref to ref hangs on reading
Date: Thu, 17 Oct 2013 15:57:23 +0100
To: Ed Avis <eda [...] waniasset.com>
From: Dave Mitchell <davem [...] iabyn.com>
Download (untitled) / with headers
text/plain 2.3k
On Mon, Sep 23, 2013 at 06:59:09AM +0000, Ed Avis wrote: Show quoted text
> In general I suggest that opening a filehandle giving a reference to a > reference doesn't make any sense, and surely didn't do anything useful > even before the change identified. It would be better to make it throw > an exception.
The other approach is to treat it similarly to how other non-string vars are treated when used as strings, e.g. $x = \1; $x .= 2, which sets $x to "SCALAR(0x11fd5b0)2". Show quoted text
> Similarly I suggest that open $fh, '<', \undef should also throw.
The test suite for PerlIO::scalar already explicitly expects undef to behave like an empty string *without* generating any warnings. I've pushed the following, which fixes the original bug report by going down the stringifying route. commit 552908b17471877454d99aa32f7ac5c29ea2efbb Author: David Mitchell <davem@iabyn.com> AuthorDate: Thu Oct 17 15:35:14 2013 +0100 Commit: David Mitchell <davem@iabyn.com> CommitDate: Thu Oct 17 15:46:05 2013 +0100 PerlIO::scalar: stringify refs If $s in open my $fh, "<", \$s or similar is a ref, then stringify that ref: i.e. convert it from a ref into the string "SCALAR(0x....)" or whatever. This fixes RT #119529 Filehandle opened from ref to ref hangs on reading which in this case was looping forever, since it kept thinking that the var was a string ("SCALAR.."), but whose length was 0. I haven't gone for a complete "always force var into a string" approach, since PerlIO::scalar has quite a tolerance for "bad" vars; e.g. it won't warn if $var is undef or a read-only constant number etc; and it already normalises under some circumstances and not others. So I've just increased the cases somewhat where it normalises. Also, I didn't look to closely at the code that was looping (or to put it another way, I looked at it but didn't understand it), so it could conceivably still behave badly on some other strange type of variable that manages to avoid getting normalised. -- "But Sidley Park is already a picture, and a most amiable picture too. The slopes are green and gentle. The trees are companionably grouped at intervals that show them to advantage. The rill is a serpentine ribbon unwound from the lake peaceably contained by meadows on which the right amount of sheep are tastefully arranged." -- Lady Croom, "Arcadia"
CC: "perlbug-followup [...] perl.org" <perlbug-followup [...] perl.org>
Subject: RE: [perl #119529] Filehandle opened from ref to ref hangs on reading
Date: Thu, 17 Oct 2013 14:59:57 +0000
To: 'Dave Mitchell' <davem [...] iabyn.com>
From: Ed Avis <eda [...] waniasset.com>
Download (untitled) / with headers
text/plain 608b
Thanks for fixing this. If silently treating undef as the empty string really is the desired behaviour, it would be worth documenting that. (IMHO a warning would be more useful, but I guess whoever wrote the PerlIO::scalar test suite already weighed up the pros and cons of that behaviour.) -- Ed Avis <eda@waniasset.com> Show quoted text
______________________________________________________________________ This email has been scanned by the Symantec Email Security.cloud service. For more information please visit http://www.symanteccloud.com
______________________________________________________________________
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 731b
On Thu Oct 17 07:58:17 2013, davem wrote: Show quoted text
> I've pushed the following, which fixes the original bug report by > going > down the stringifying route. > > > commit 552908b17471877454d99aa32f7ac5c29ea2efbb > Author: David Mitchell <davem@iabyn.com> > AuthorDate: Thu Oct 17 15:35:14 2013 +0100 > Commit: David Mitchell <davem@iabyn.com> > CommitDate: Thu Oct 17 15:46:05 2013 +0100
...which unfortunately breaks when reading a read-only ref in "<" mode. I pushed a better fix as 5a2bc23b. I actually wrote something similar to that a month ago, but then sat on it as real life got in the way. The amazing thing was that both Dave Mitchell and I used \42 it our tests. Great minds think alike! -- Father Chrysostomos
From: Dave Mitchell <davem [...] iabyn.com>
Subject: Re: [perl #119529] Filehandle opened from ref to ref hangs on reading
Date: Sat, 26 Oct 2013 00:25:29 +0100
CC: perl5-porters [...] perl.org
To: Father Chrysostomos via RT <perlbug-followup [...] perl.org>
Download (untitled) / with headers
text/plain 860b
On Fri, Oct 25, 2013 at 02:23:10PM -0700, Father Chrysostomos via RT wrote: Show quoted text
> On Thu Oct 17 07:58:17 2013, davem wrote:
> > I've pushed the following, which fixes the original bug report by > > going > > down the stringifying route. > > > > > > commit 552908b17471877454d99aa32f7ac5c29ea2efbb > > Author: David Mitchell <davem@iabyn.com> > > AuthorDate: Thu Oct 17 15:35:14 2013 +0100 > > Commit: David Mitchell <davem@iabyn.com> > > CommitDate: Thu Oct 17 15:46:05 2013 +0100
> > ...which unfortunately breaks when reading a read-only ref in "<" mode. I pushed a better fix as 5a2bc23b.
Ok thanks. Show quoted text
> The amazing thing was that both Dave Mitchell and I used \42 it our > tests. Great minds think alike!
Er, actually the \42 was in the sample code in the original ticket! -- That he said that that that that is is is debatable, is debatable.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org