Skip Menu |
Report information
Id: 8910
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: abigail <abigail [at] foad.org>
Cc:
AdminCc:

Operating System: Linux
PatchStatus: (no value)
Severity: low
Type: core
Perl Version: 5.6.1
Fixed In: (no value)



Date: 5 Apr 2002 14:50:30 -0000
From: abigail [...] foad.org
To: perlbug [...] perl.org
Subject: Subroutine doesn't create elements.
Cc: abigail [...] foad.org
Download (untitled) / with headers
text/plain 4.4k
This is a bug report for perl from abigail@foad.org, generated with the help of perlbug 1.33 running under perl v5.6.1. ----------------------------------------------------------------- [Please enter your report here] man perlsub states: Any arguments passed in show up in the array "@_". There- fore, if you called a function with two arguments, those would be stored in "$_[0]" and "$_[1]". The array "@_" is a local array, but its elements are aliases for the actual scalar parameters. In particular, if an element "$_[0]" is updated, the corresponding argument is updated (or an error occurs if it is not updatable). If an argument is an array or hash element which did not exist when the function was called, that element is created only when (and if) it is modified or a reference to it is taken. Now, consider this program: #!/usr/bin/perl -w use strict; sub mysub; $, = " "; $\ = "\n"; # 0 1 2 3 my @arr = ("cent", "nickel", "dime", undef); $arr [5] = "dollar"; # Auto-vivifies $arr [4] to be 'undef'. print map {defined $_ ? $_ : "^^"} @arr; mysub @arr; print map {defined $_ ? $_ : "^^"} @arr; sub mysub { $_ [0] = 0.01; $_ [1] = 0.05; $_ [2] = 0.10; $_ [3] = 0.25; $_ [4] = 0.50; $_ [5] = 1.00; } __END__ If we run this, we get: cent nickel dime ^^ ^^ dollar 0.01 0.05 0.1 0.25 ^^ 1 That is, $arr [4] isn't 0.50 after the function mysub was called, while the documentation specifies it should. bleadperl gives the same behaviour. Abigail [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=low --- Site configuration information for perl v5.6.1: Configured by abigail at Tue Jul 10 12:17:49 CEST 2001. Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration: Platform: osname=linux, osvers=2.4.5, archname=i686-linux-64int-ld uname='linux hermione 2.4.5 #6 fri jun 22 01:38:20 pdt 2001 i686 unknown ' config_args='-d -Dprefix=/opt/perl -Doptimize=-g -Dusemorebits' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef use64bitint=define use64bitall=undef uselongdouble=define Compiler: cc='cc', ccflags ='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/opt/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-g', cppflags='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/opt/local/include' ccversion='', gccversion='2.95.3 20010315 (release)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long long', ivsize=8, nvtype='long double', nvsize=12, Off_t='off_t', lseeksize=8 alignbytes=4, usemymalloc=n, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib -L/opt/local/lib' libpth=/usr/local/lib /opt/local/lib /lib /usr/lib libs=-lnsl -lndbm -lgdbm -ldl -lm -lc -lcrypt -lutil perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil libc=/lib/libc-2.2.3.so, so=so, useshrplib=false, libperl=libperl.a Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib -L/opt/local/lib' Locally applied patches: --- @INC for perl v5.6.1: /home/abigail/Perl /home/abigail/Sybase /opt/perl/lib/5.6.1/i686-linux-64int-ld /opt/perl/lib/5.6.1 /opt/perl/lib/site_perl/5.6.1/i686-linux-64int-ld /opt/perl/lib/site_perl/5.6.1 /opt/perl/lib/site_perl . --- Environment for perl v5.6.1: HOME=/home/abigail LANG (unset) LANGUAGE (unset) LC_ALL=POSIX LD_LIBRARY_PATH=/home/abigail/Lib:/usr/local/lib:/usr/lib:/lib:/usr/X11R6/lib:/opt/gnome/lib LOGDIR (unset) PATH=/home/abigail/Bin:/opt/perl/bin:/usr/local/bin:/usr/local/X11/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/X11R6/bin:/usr/games:/opt/povray/bin:/opt/gnome/bin:/opt/opera/bin:/usr/share/texmf/bin:/opt/Acrobat4/bin:/opt/java/blackdown/j2sdk1.3.1/bin:/usr/local/games/bin:/opt/gnuplot/bin:/opt/mysql/bin PERL5LIB=/home/abigail/Perl:/home/abigail/Sybase PERLDIR=/opt/perl PERL_BADLANG (unset) SHELL=/usr/bin/bash
Date: Fri, 5 Apr 2002 10:51:44 -0500
From: Michael G Schwern <schwern [...] pobox.com>
To: abigail [...] foad.org
Cc: perl5-porters [...] perl.org
Subject: Re: [ID 20020405.002] Subroutine doesn't create elements.
Download (untitled) / with headers
text/plain 2.4k
On Fri, Apr 05, 2002 at 02:50:30PM -0000, abigail@foad.org wrote: <snip> Show quoted text
> # 0 1 2 3 > my @arr = ("cent", "nickel", "dime", undef); > $arr [5] = "dollar"; # Auto-vivifies $arr [4] to be 'undef'. > > print map {defined $_ ? $_ : "^^"} @arr; > mysub @arr; > print map {defined $_ ? $_ : "^^"} @arr; > > sub mysub { > $_ [0] = 0.01; > $_ [1] = 0.05; > $_ [2] = 0.10; > $_ [3] = 0.25; > $_ [4] = 0.50; > $_ [5] = 1.00; > } > __END__ > > If we run this, we get: > cent nickel dime ^^ ^^ dollar > 0.01 0.05 0.1 0.25 ^^ 1 > > That is, $arr [4] isn't 0.50 after the function mysub was called, > while the documentation specifies it should.
Likely related to this feature: $ perl -wle '@a = ("foo ", "bar ", "woof ", undef); $a[5] = "wibble "; print map {defined $_ ? $_ : "^^ "} @a; print map {exists $a[$_] ? "exists " : "^^ "} 0..$#a;' foo bar woof ^^ ^^ wibble exists exists exists exists ^^ exists $a[3] is not defined, but it exists. $a[4] exists but is not defined. Internally, $a[3] would contain an SV with the value of undef and $a[4] contains PL_sv_undef. but since this feature was added in 5.6.0 (or there abouts) and your bug goes back to at least 5.004, I could be wrong. Devel::Peek seems to be confused by @arr. It doesn't see past element 3. SV = RV(0x10138350) at 0x1012ef44 REFCNT = 1 FLAGS = (TEMP,ROK) RV = 0x1012a2f8 SV = PVAV(0x1011f5e8) at 0x1012a2f8 REFCNT = 2 FLAGS = (PADBUSY,PADMY) IV = 0 NV = 0 ARRAY = 0x1012c8f8 FILL = 5 MAX = 11 ARYLEN = 0x0 FLAGS = (REAL) Elt No. 0 SV = PVNV(0x1013fe38) at 0x1011e1d0 REFCNT = 1 FLAGS = (NOK,pNOK) IV = 0 NV = 0.01 PV = 0x10130260 "cent"\0 CUR = 4 LEN = 5 Elt No. 1 SV = PVNV(0x1013fe58) at 0x1011e20c REFCNT = 1 FLAGS = (NOK,pNOK) IV = 0 NV = 0.05 PV = 0x10126028 "nickel"\0 CUR = 6 LEN = 7 Elt No. 2 SV = PVNV(0x1013fe78) at 0x1012a28c REFCNT = 1 FLAGS = (NOK,pNOK) IV = 0 NV = 0.1 PV = 0x10126038 "dime"\0 CUR = 4 LEN = 5 Elt No. 3 SV = NV(0x10143500) at 0x1012e064 REFCNT = 1 FLAGS = (NOK,pNOK) NV = 0.25 -- Michael G. Schwern <schwern@pobox.com> http://www.pobox.com/~schwern/ Perl Quality Assurance <perl-qa@perl.org> Kwalitee Is Job One It's Absinthe time!
From: abigail [...] foad.org
Date: Fri, 5 Apr 2002 18:26:17 +0200
To: Michael G Schwern <schwern [...] pobox.com>
Cc: perl5-porters [...] perl.org
Subject: Re: [ID 20020405.002] Subroutine doesn't create elements.
Download (untitled) / with headers
text/plain 1.6k
On Fri, Apr 05, 2002 at 10:51:44AM -0500, Michael G Schwern wrote: Show quoted text
> On Fri, Apr 05, 2002 at 02:50:30PM -0000, abigail@foad.org wrote: > <snip>
> > # 0 1 2 3 > > my @arr = ("cent", "nickel", "dime", undef); > > $arr [5] = "dollar"; # Auto-vivifies $arr [4] to be 'undef'. > > > > print map {defined $_ ? $_ : "^^"} @arr; > > mysub @arr; > > print map {defined $_ ? $_ : "^^"} @arr; > > > > sub mysub { > > $_ [0] = 0.01; > > $_ [1] = 0.05; > > $_ [2] = 0.10; > > $_ [3] = 0.25; > > $_ [4] = 0.50; > > $_ [5] = 1.00; > > } > > __END__ > > > > If we run this, we get: > > cent nickel dime ^^ ^^ dollar > > 0.01 0.05 0.1 0.25 ^^ 1 > > > > That is, $arr [4] isn't 0.50 after the function mysub was called, > > while the documentation specifies it should.
> > Likely related to this feature: > > $ perl -wle '@a = ("foo ", "bar ", "woof ", undef); $a[5] = "wibble "; > print map {defined $_ ? $_ : "^^ "} @a; > print map {exists $a[$_] ? "exists " : "^^ "} 0..$#a;' > foo bar woof ^^ ^^ wibble > exists exists exists exists ^^ exists > > $a[3] is not defined, but it exists. $a[4] exists but is not defined.
I think you mean that $a [4] neither exists, nor is defined. But that's exactly my point. The documentation says: If an argument is an array or hash element which DID NOT EXIST when the function was called, that element is created only when (and if) it is modified or a reference to it is taken. The element doesn't exist (as exists shows), is modified, yet isn't created. Abigail
Date: Fri, 5 Apr 2002 11:39:51 -0500
From: Michael G Schwern <schwern [...] pobox.com>
To: abigail [...] foad.org
Cc: perl5-porters [...] perl.org
Subject: Re: [ID 20020405.002] Subroutine doesn't create elements.
Download (untitled) / with headers
text/plain 934b
On Fri, Apr 05, 2002 at 06:26:17PM +0200, abigail@foad.org wrote: Show quoted text
> > $a[3] is not defined, but it exists. $a[4] exists but is not defined.
> > I think you mean that $a [4] neither exists, nor is defined. > > But that's exactly my point. The documentation says:
Yes, I'm not disagreeing with you (despite my brain-o, it's early for me). Just noting it's likely a the result of $a[4]'s internally having the special value of PL_sv_undef and the @_ magic not aliasing it because of this. Devel::Peek not taking this special case into account either is an omen. -- Michael G. Schwern <schwern@pobox.com> http://www.pobox.com/~schwern/ Perl Quality Assurance <perl-qa@perl.org> Kwalitee Is Job One It should indeed be said that notwithstanding the fact that I make ambulatory progress through the umbragious inter-hill mortality slot, terror sensations will no be initiated in me, due to para-etical phenomena.
Date: Fri, 5 Apr 2002 11:58:58 -0500
From: Ronald J Kimball <rjk [...] linguist.Thayer.dartmouth.edu>
To: abigail [...] foad.org
Cc: perl5-porters [...] perl.org
Subject: Re: [ID 20020405.002] Subroutine doesn't create elements.
Download (untitled) / with headers
text/plain 1.6k
On Fri, Apr 05, 2002 at 02:50:30PM -0000, abigail@foad.org wrote: Show quoted text
> man perlsub states: > > Any arguments passed in show up in the array "@_". There- > fore, if you called a function with two arguments, those > would be stored in "$_[0]" and "$_[1]". The array "@_" is > a local array, but its elements are aliases for the actual > scalar parameters. In particular, if an element "$_[0]" > is updated, the corresponding argument is updated (or an > error occurs if it is not updatable). If an argument is > an array or hash element which did not exist when the > function was called, that element is created only when > (and if) it is modified or a reference to it is taken. >
I believe you are misreading the documentation. It's referring to a situation like this: #!perl -w sub mysub1 { my $x = $_[0]; } sub mysub2 { $_[0] = 1; } mysub1($arr[0]); print "$arr[0]\n"; mysub2($arr[0]); print "$arr[0]\n"; __END__ Use of uninitialized value at tmp.pl - line 13. 1 Note that the subs are being called with "an array ... element which did not exist when the function was called", and that "that element is created only when ... it is modified" in mysub2. Keep in mind that arrays passed to subroutines are flattened, so: @arr = (0, 1); mysub(@arr); is equivalent to: @arr = (0, 1); mysub($arr[0], $arr[1]); mysub has no notion that it was called with @arr as an argument. The elements of @_ are aliased individually to the elements of @arr; @_ as a whole is not aliased to @arr as a whole. So, adding elements off the end of @_ does not add elements to @arr. Ronald
Date: Fri, 5 Apr 2002 12:05:40 -0500
From: Ronald J Kimball <rjk [...] linguist.Thayer.dartmouth.edu>
To: abigail [...] foad.org
Cc: perl5-porters [...] perl.org
Subject: Re: [ID 20020405.002] Subroutine doesn't create elements.
On Fri, Apr 05, 2002 at 11:58:58AM -0500, Ronald J Kimball wrote: Show quoted text
> On Fri, Apr 05, 2002 at 02:50:30PM -0000, abigail@foad.org wrote: >
> > man perlsub states: > > > > Any arguments passed in show up in the array "@_". There- > > fore, if you called a function with two arguments, those > > would be stored in "$_[0]" and "$_[1]". The array "@_" is > > a local array, but its elements are aliases for the actual > > scalar parameters. In particular, if an element "$_[0]" > > is updated, the corresponding argument is updated (or an > > error occurs if it is not updatable). If an argument is > > an array or hash element which did not exist when the > > function was called, that element is created only when > > (and if) it is modified or a reference to it is taken. > >
> > I believe you are misreading the documentation. It's referring to a > situation like this:
On second thought, I think I misread your bug report. Apologies. I see now what the problem is, and it does look like misbehavior. Ronald
From: Philip Newton <Philip.Newton [...] gmx.net>
To: rjk [...] linguist.Thayer.dartmouth.edu (Ronald J Kimball)
Cc: abigail [...] foad.org, perl5-porters [...] perl.org
Subject: Re: [ID 20020405.002] Subroutine doesn't create elements.
Date: Fri, 05 Apr 2002 21:26:41 +0200
Download (untitled) / with headers
text/plain 824b
On Fri, 5 Apr 2002 12:05:40 -0500, rjk@linguist.Thayer.dartmouth.edu (Ronald J Kimball) wrote: Show quoted text
> On second thought, I think I misread your bug report. Apologies. I see > now what the problem is, and it does look like misbehavior.
On the third hand, if $arr[4] is &PL_sv_undef, then assigning to $_[4] is basically the same as assigning "undef = 0.50;". Perl can't easily put something into $arr[4] because the &PL_sv_undef that got pushed on the stack could be "anything". On the fourth hand, it does mean that the documentation is incorrect (or insufficient). Not sure whether the documentation or the code should be changed, though. (I think the docs should change to reflect the code, but that's probably biassed by perceived difficulty of making the code reflect the documentation in this aspect.) Cheers, Philip
Date: Fri, 5 Apr 2002 14:34:38 -0500
From: Michael G Schwern <schwern [...] pobox.com>
To: Philip Newton <Philip.Newton [...] gmx.net>
Cc: Ronald J Kimball <rjk [...] linguist.Thayer.dartmouth.edu>, abigail [...] foad.org, perl5-porters [...] perl.org
Subject: Re: [ID 20020405.002] Subroutine doesn't create elements.
Download (untitled) / with headers
text/plain 1.1k
On Fri, Apr 05, 2002 at 09:26:41PM +0200, Philip Newton wrote: Show quoted text
> On the third hand, if $arr[4] is &PL_sv_undef, then assigning to $_[4] > is basically the same as assigning "undef = 0.50;". Perl can't easily > put something into $arr[4] because the &PL_sv_undef that got pushed on > the stack could be "anything".
$arr[4] containing &PL_sv_undef is correct. @_ aliasing is definately doing something Evil and probably not going through the AV API and likely doesn't know how to deal with an array element which is PL_sv_undef. Show quoted text
> On the fourth hand, it does mean that the documentation is incorrect (or > insufficient). Not sure whether the documentation or the code should be > changed, though. (I think the docs should change to reflect the code, > but that's probably biassed by perceived difficulty of making the code > reflect the documentation in this aspect.)
Don't document bugs as features. Then you can never fix it. -- Michael G. Schwern <schwern@pobox.com> http://www.pobox.com/~schwern/ Perl Quality Assurance <perl-qa@perl.org> Kwalitee Is Job One We're talkin' to you, weaselnuts. http://www.goats.com/archive/000831.html
From: abigail [...] foad.org
Date: Fri, 19 Apr 2002 14:47:36 +0200
To: Philip Newton <Philip.Newton [...] gmx.net>
Cc: perl5-porters [...] perl.org
Subject: Re: [ID 20020405.002] Subroutine doesn't create elements.
Download (untitled) / with headers
text/plain 1.8k
On Fri, Apr 05, 2002 at 09:26:41PM +0200, Philip Newton wrote: Show quoted text
> On Fri, 5 Apr 2002 12:05:40 -0500, rjk@linguist.Thayer.dartmouth.edu > (Ronald J Kimball) wrote: >
> > On second thought, I think I misread your bug report. Apologies. I see > > now what the problem is, and it does look like misbehavior.
> > On the third hand, if $arr[4] is &PL_sv_undef, then assigning to $_[4] > is basically the same as assigning "undef = 0.50;". Perl can't easily > put something into $arr[4] because the &PL_sv_undef that got pushed on > the stack could be "anything". > > On the fourth hand, it does mean that the documentation is incorrect (or > insufficient). Not sure whether the documentation or the code should be > changed, though. (I think the docs should change to reflect the code, > but that's probably biassed by perceived difficulty of making the code > reflect the documentation in this aspect.)
Changing the documentation means you are extending the language. From a language perspective, there is just one kind of "undef", &PL_sv_undef might make sense to many people on this list, but the language Perl doesn't know any PL_sv_undef. You'd have a hard time to explain why the following doesn't print twice the same line: #!/usr/bin/perl -w use strict; sub mysub; $, = " "; $\ = "\n"; my @arr1 = ("cent", "nickel", "dime", undef); $arr1 [5] = "dollar"; # Auto-vivifies $arr1 [4]; my @arr2 = @arr1; mysub @arr1; mysub @arr2; print map {defined $_ ? $_ : "^^^"} @arr1; print map {defined $_ ? $_ : "^^^"} @arr2; sub mysub { $_ [0] = 0.01; $_ [1] = 0.05; $_ [2] = 0.10; $_ [3] = 0.25; $_ [4] = 0.50; $_ [5] = 1.00; } __END__ This will print: 0.01 0.05 0.1 0.25 ^^^ 1 0.01 0.05 0.1 0.25 0.5 1 Abigail
To: abigail [...] foad.org
Cc: perl5-porters [...] perl.org, mjd [...] plover.com
Subject: Re: [ID 20020405.002] Subroutine doesn't create elements.
Date: Fri, 19 Apr 2002 12:14:35 -0400
From: Mark-Jason Dominus <mjd [...] plover.com>
Download (untitled) / with headers
text/plain 509b
Show quoted text
> Changing the documentation means you are extending the language.
> >From a language perspective, there is just one kind of "undef",
> &PL_sv_undef might make sense to many people on this list, but > the language Perl doesn't know any PL_sv_undef.
Unfortunately, it does. print exists($a[3]) ? "1\n" : "0\n"; undef $a[3]; print exists($a[3]) ? "1\n" : "0\n"; The output is 0 1 I still think this was a bad idea, for exactly the reason you cite, but there it is.
Subject: [perl #8910] Subroutine doesn't create elements.
Date: Wed, 11 Jun 2008 09:30:35 +0200
To: Perl5 Porters <perl5-porters [...] perl.org>, perlbug-followup [...] perl.org
From: Renée Bäcker <renee.baecker [...] smart-websolutions.de>
Download (untitled) / with headers
text/plain 1.6k
The auto-vivification doesn't create SV's at all: rbaecker@www-devel-rbaecker ~/perl510/bugs $ cat 8910.pl #!/usr/bin/perl use strict; use warnings; use Devel::Peek; my @b = (undef,undef,undef,undef); Dump \@b, 10; test(@b); print join " -> ", map {defined $_ ? $_ : "^^"} @b; print "\n\n"; my @c; $#c = 3; Dump \@c, 10; test(@c); print join " -> ", map {defined $_ ? $_ : "^^"} @c; print "\n"; sub test { $_[0] = 0; $_[1] = 1; $_[2] = 2; $_[3] = 3; } rbaecker@www-devel-rbaecker ~/perl510/bugs $ perl 8910.pl SV = RV(0x8189260) at 0x815deac REFCNT = 1 FLAGS = (TEMP,ROK) RV = 0x8142c78 SV = PVAV(0x8147284) at 0x8142c78 REFCNT = 2 FLAGS = (PADBUSY,PADMY) IV = 0 NV = 0 ARRAY = 0x81514c0 FILL = 3 MAX = 3 ARYLEN = 0x0 FLAGS = (REAL) Elt No. 0 SV = NULL(0x0) at 0x8142180 REFCNT = 1 FLAGS = () Elt No. 1 SV = NULL(0x0) at 0x814239c REFCNT = 1 FLAGS = () Elt No. 2 SV = NULL(0x0) at 0x8142c48 REFCNT = 1 FLAGS = () Elt No. 3 SV = NULL(0x0) at 0x814209c REFCNT = 1 FLAGS = () 0 -> 1 -> 2 -> 3 SV = RV(0x8189260) at 0x815de10 REFCNT = 1 FLAGS = (TEMP,ROK) RV = 0x8175b74 SV = PVAV(0x81472b0) at 0x8175b74 REFCNT = 2 FLAGS = (PADBUSY,PADMY) IV = 0 NV = 0 ARRAY = 0x8168798 FILL = 3 MAX = 3 ARYLEN = 0x815deac FLAGS = (REAL) Elt No. 0 Elt No. 1 Elt No. 2 Elt No. 3 ^^ -> ^^ -> ^^ -> ^^ rbaecker@www-devel-rbaecker ~/perl510/bugs $ -- Renée Bäcker renee.baecker@smart-websolutions.de XING: http://www.xing.com/profile/Renee_Baecker Foo-Magazin: http://foo-magazin.de
CC: perlbug-followup [...] perl.org
Subject: Re: [perl #8910] Subroutine doesn't create elements.
Date: Wed, 11 Jun 2008 11:13:47 -0500
To: renee.baecker [...] smart-websolutions.de
From: "David Nicol" <davidnicol [...] gmail.com>
Download (untitled) / with headers
text/plain 144b
On 6/11/08, Renée Bäcker <renee.baecker@smart-websolutions.de> wrote: Show quoted text
> The auto-vivification doesn't create SV's at all:
not in 5.8.8 either
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.1k
In blead, it throws an error now. Porting/bisect.pl --start v5.10.0 --end v5.22.0 --target miniperl -Dcc='ccache gcc-6' -- ./miniperl ../8910.pl ce0d59fdd1c7d145efdf6bf8da56a259fed483e4 is the first bad commit commit ce0d59fdd1c7d145efdf6bf8da56a259fed483e4 Author: Father Chrysostomos <sprout@cpan.org> Date: Tue Jul 2 13:07:45 2013 -0700 [perl #7508] Use NULL for nonexistent array elems This commit fixes bug #7508 and provides the groundwork for fixing several other bugs. Elements of @_ are aliased to the arguments, so that \$_[0] within sub foo will reference the same scalar as \$x if the sub is called as foo($x). &PL_sv_undef (the global read-only undef scalar returned by the ‘undef’ operator itself) was being used to represent nonexistent array elements. So the pattern would be broken for foo(undef), where \$_[0] would vivify a new $_[0] element, treating it as having been nonexistent. This also causes other problems with constants under ithreads (#105906) and causes a pending fix for another bug (#118691) to trig- ger this bug. This commit changes the internals to use a null pointer to represent a nonexistent element. This requires that Storable be changed to account for it. Also, IPC::Open3 was relying on the bug. So this commit patches both modules. :100644 100644 b15f6ff7b60ea28323b7bba85b7bdaae7132782f aae70bf8acfcce9fe5f71143d6039efeb8c8e2a9 M av.c :040000 040000 fa838b22ec5547df4feb63110c10681d950bb500 384bbf0307eb46120bb7a1dee84c27f3ce5e2a4d M dist :040000 040000 b2024904fd1708fbd395a2f5778f892a7955668c dc1480bd633862776023c8a1bee98c1238d5722a M ext :100644 100644 d8d9322c6055073b532f540ef5466980c93181c2 92765f0f792591f1ba8e9275f80747a1651c6e3f M pad.c :100644 100644 fd404536577cfe67f40e3effb2283e0f33e428a2 17e0f3bb5142db10ec4bb965c01e785ccbad84ce M pp.c :100644 100644 b08643fcc14028c8f319ff9a9a3da8e064b3d144 58a30831d803eeb02cf0928c5a8b73cbda3b393f M pp_hot.c :100644 100644 5f142a0332404b6689c4322a58207337044ec607 d207d0d951de3dddc27b0ebc61b1e0fe7b48c4b6 M regexec.c bisect run success That took 1492 seconds.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 278b
I probably should mention, it segfaults in that commit, it doesn't segfault in blead. Here's what it does now: dcollins@nightshade64:~/toolchain/perl$ ./perl ../8910.pl cent nickel dime ^^ ^^ dollar Modification of a read-only value attempted at ../8910.pl line 19.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 753b
On Mon Jul 04 18:02:26 2016, dcollinsn@gmail.com wrote: Show quoted text
> In blead, it throws an error now. > > Porting/bisect.pl --start v5.10.0 --end v5.22.0 --target miniperl > -Dcc='ccache gcc-6' -- ./miniperl ../8910.pl > > ce0d59fdd1c7d145efdf6bf8da56a259fed483e4 is the first bad commit > commit ce0d59fdd1c7d145efdf6bf8da56a259fed483e4 > Author: Father Chrysostomos <sprout@cpan.org> > Date: Tue Jul 2 13:07:45 2013 -0700 > > [perl #7508] Use NULL for nonexistent array elems
The solution is probably to have S_pushav in pp_hot.c push a defelem rather than &PL_sv_undef if it is in flattening lvalue context. We have to check for nulls anyway. (I have been told that defelems are too slow. This may not be the best solution.) -- Father Chrysostomos
To: perl5-porters [...] perl.org
Subject: Re: [perl #8910] Subroutine doesn't create elements.
From: Zefram <zefram [...] fysh.org>
Date: Tue, 16 Jan 2018 19:19:02 +0000
Download (untitled) / with headers
text/plain 126b
Fixed in commit 6661956a23de82b41adc406200054293d6d7aded. Fix caused some CPAN breakage, tracked as [perl #132727]. -zefram
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 605b
On Tue, 16 Jan 2018 11:19:20 -0800, zefram@fysh.org wrote: Show quoted text
> Fixed in commit 6661956a23de82b41adc406200054293d6d7aded. Fix caused > some CPAN breakage, tracked as [perl #132727].
I'd very cautiously monitor effects of this commit - as it can result in very hard to detect but measurable fall down in execution speed and increased memory consumption. And that's not mentioning new bugs. Off the top of my head, I was able to produce the following: 5.26.0 perl -e '{local $a[3] = 12; @foo=@a}; warn scalar @a' 0 at -e line 1. blead ./perl -e '{local $a[3] = 12; @foo=@a}; warn scalar @a' 3 at -e line 1.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 813b
On Tue, 16 Jan 2018 16:22:44 -0800, randir wrote: Show quoted text
> On Tue, 16 Jan 2018 11:19:20 -0800, zefram@fysh.org wrote:
> > Fixed in commit 6661956a23de82b41adc406200054293d6d7aded. Fix caused > > some CPAN breakage, tracked as [perl #132727].
> > I'd very cautiously monitor effects of this commit - as it can result > in very hard to detect but measurable fall down in execution speed and > increased memory consumption. And that's not mentioning new bugs. Off > the top of my head, I was able to produce the following: > > 5.26.0 > perl -e '{local $a[3] = 12; @foo=@a}; warn scalar @a' > 0 at -e line 1. > > blead > ./perl -e '{local $a[3] = 12; @foo=@a}; warn scalar @a' > 3 at -e line 1.
Not only that, it also breaks use of exists and delete on arrays. I think it should be reverted. -- Father Chrysostomos
Subject: Re: [perl #8910] Subroutine doesn't create elements.
Date: Wed, 17 Jan 2018 03:12:21 +0100
To: perl5-porters [...] perl.org
From: Tomasz Konojacki <me [...] xenu.pl>
On Tue, 16 Jan 2018 17:49:18 -0800 "Father Chrysostomos via RT" <perlbug-followup@perl.org> wrote: Show quoted text
> On Tue, 16 Jan 2018 16:22:44 -0800, randir wrote:
> > On Tue, 16 Jan 2018 11:19:20 -0800, zefram@fysh.org wrote:
> > > Fixed in commit 6661956a23de82b41adc406200054293d6d7aded. Fix caused > > > some CPAN breakage, tracked as [perl #132727].
> > > > I'd very cautiously monitor effects of this commit - as it can result > > in very hard to detect but measurable fall down in execution speed and > > increased memory consumption. And that's not mentioning new bugs. Off > > the top of my head, I was able to produce the following: > > > > 5.26.0 > > perl -e '{local $a[3] = 12; @foo=@a}; warn scalar @a' > > 0 at -e line 1. > > > > blead > > ./perl -e '{local $a[3] = 12; @foo=@a}; warn scalar @a' > > 3 at -e line 1.
> > Not only that, it also breaks use of exists and delete on arrays. I think it should be reverted.
It's a bit off-topic, but what's the point of exists and delete on arrays? I think they're even weirder than defined(@array) which was deprecated and removed.
Date: Wed, 17 Jan 2018 02:20:19 +0000
From: Zefram <zefram [...] fysh.org>
Subject: Re: [perl #8910] Subroutine doesn't create elements.
To: perl5-porters [...] perl.org
Father Chrysostomos via RT wrote: Show quoted text
>Not only that, it also breaks use of exists and delete on arrays.
Not really. delete() and exists() on array elements still work as documented; the `non-existent' status of an array element can still occur and be manipulated. The documentation for those operators doesn't claim that that state isn't affected by other operations. Indeed, since the array enumeration that one gets from list-context @a is an operation from the consecutive-indices model of arrays, one should not be surprised by such side effects arising from mixing the two models. Show quoted text
>I think it should be reverted.
We should not compromise the coherent consecutive-indices model of arrays in order to support the "strongly discouraged" sparse-index model. Indeed, we came close to deprecating the latter, and it only escaped that fate on the grounds that it's not in the way of anything. If it is now impeding bugfixing of the consecutive-indices mechanism, exists() on array elements is the thing that should be sacrificed. -zefram
To: perl5-porters [...] perl.org
Subject: Re: [perl #8910] Subroutine doesn't create elements.
From: Zefram <zefram [...] fysh.org>
Date: Wed, 17 Jan 2018 02:27:20 +0000
Download (untitled) / with headers
text/plain 799b
Tomasz Konojacki wrote: Show quoted text
>It's a bit off-topic, but what's the point of exists and delete on >arrays?
It was an attempt to make arrays serve two distinct purposes simultaneously. They allow one to use a Perl array as an associative array with natural-number keys. Each possible key is present or absent independent of all others; presence is detected with exists(). The more conventional view of arrays, storing a sequence of items, with indices always being an initial subsequence of the natural numbers, always predominated, and of course Perl has a perfectly good data type whose primary purpose is to handle sparse sets of keys. So the associative view of Perl arrays has always had only partial support, and the consecutive-indices model has taken precedence where conflicts arise. -zefram
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.7k
On Tue, 16 Jan 2018 18:20:42 -0800, zefram@fysh.org wrote: Show quoted text
> Father Chrysostomos via RT wrote:
> >Not only that, it also breaks use of exists and delete on arrays.
> > Not really. delete() and exists() on array elements still work as > documented; the `non-existent' status of an array element can still occur > and be manipulated. The documentation for those operators doesn't claim > that that state isn't affected by other operations. Indeed, since the > array enumeration that one gets from list-context @a is an operation from > the consecutive-indices model of arrays, one should not be surprised by > such side effects arising from mixing the two models. >
> >I think it should be reverted.
> > We should not compromise the coherent consecutive-indices model of arrays > in order to support the "strongly discouraged" sparse-index model. > Indeed, we came close to deprecating the latter, and it only escaped > that fate on the grounds that it's not in the way of anything.
It escaped that fate on the ground that there are regular users of the feature who would have to tweak thousands of lines of code if it were removed. (I still disagree with the ‘strongly discouraged’ label.) Show quoted text
> If it > is now impeding bugfixing
If a pure rvalue operation starts having observable side effects, then that is a bug. Introducing a new bug for the sake of fixing an existing one is not a reasonable compromise, considering that Perl favours backward-compatibility. You also have not responded to the localization bug that Mr. Aleynikov brought up. Flagging the op in \(@a) is easy enough to do. Surely there aren’t too many lvalue cases that are not currently flagged as lvalues? Show quoted text
> of the consecutive-indices mechanism, exists() > on array elements is the thing that should be sacrificed.
-- Father Chrysostomos
To: perl5-porters [...] perl.org
Date: Sat, 20 Jan 2018 22:43:58 +0000
From: Zefram <zefram [...] fysh.org>
Subject: Re: [perl #8910] Subroutine doesn't create elements.
Download (untitled) / with headers
text/plain 2.2k
Father Chrysostomos via RT wrote: Show quoted text
>You also have not responded to the localization bug that Mr. Aleynikov >brought up.
Is it a bug? It's a change in behaviour, but the semantics of "local" don't make clear that one behaviour or the other is correct. The effect that escapes the localisation, namely the existence of some array elements, isn't part of the object that is ostensibly being localised, a higher array element. We have a problem here that an implied side effect means that the structure of the "local" expression isn't rich enough to fully describe the extent of what should be localised. It's rather akin to this preexisting behaviour that might be considered a bug: $ perl5.27.7 -lwe '$#a = 3; print $#a; { local $a[6] = 1; print $#a; } print $#a' 3 6 -1 I'm more inclined to call this one a bug than what my edit introduced, because this one means that an empty localisation (with no operations performed in the scope of the localisation) has a side effect. I see you've edited the code in commit fd77b29b3be4108adb4a74b9c274599eaf228cb3 to use magical deferred elements. There's a defect in your edit, which you'll want to rectify: only the part for RMAGICAL arrays is effective. Your new tests only use such arrays, because you make your non-existent elements by assigning to $#a, which makes the array permanently RMAGICAL. $ perl5.27.8 -lwe '$#a = 3; print exists($a[0])||0; my @b = @a; print exists($a[0])||0' 0 0 $ perl5.27.8 -lwe '$a[3] = 3; print exists($a[0])||0; my @b = @a; print exists($a[0])||0' 0 1 I'm not a fan of the deferred-element approach. Firstly, it means that these uses of arrays in list context suffer from [perl #132729], which they didn't under my version of the code. It's introducing a bug to fix another one, though to be fair [perl #132729] needs to be addressed more generally anyway. But even if it worked, it means that the deferred elements get created *every* time the array is used in this kind of context, causing a bigger performance drain than just creating the element once. This mechanism only being required in order to support the sparse array view, it means that supporting sparse arrays is imposing a performance penalty on code that doesn't use that feature. Further reason to deprecate it. -zefram
RT-Send-CC: perl5-porters [...] perl.org
On Sat, 20 Jan 2018 14:44:17 -0800, zefram@fysh.org wrote: Show quoted text
> Father Chrysostomos via RT wrote:
> > You also have not responded to the localization bug that Mr. > > Aleynikov > > brought up.
> > Is it a bug? It's a change in behaviour, but the semantics of "local" > don't make clear that one behaviour or the other is correct. The > effect > that escapes the localisation, namely the existence of some array > elements, isn't part of the object that is ostensibly being localised, > a > higher array element. We have a problem here that an implied side > effect > means that the structure of the "local" expression isn't rich enough > to > fully describe the extent of what should be localised. It's rather > akin > to this preexisting behaviour that might be considered a bug: > > $ perl5.27.7 -lwe '$#a = 3; print $#a; { local $a[6] = 1; print $#a; } > print $#a' > 3 > 6 > -1 > > I'm more inclined to call this one a bug than what my edit introduced, > because this one means that an empty localisation (with no operations > performed in the scope of the localisation) has a side effect.
The existing behaviour is longstanding. Yes, it could be called a bug. One could argue it either way, so I don’t think it should be changed without a good reason. I have a good reason, which is off-topic for this ticket, but please let this wait until after 5.28, because my as-yet unwritten proposal changes more than just this, for the sake of consistency, and may be controversial. Show quoted text
> > I see you've edited the code in commit > fd77b29b3be4108adb4a74b9c274599eaf228cb3 to use magical deferred > elements. > There's a defect in your edit, which you'll want to rectify: only the > part for RMAGICAL arrays is effective.
Thank you for pointing that out. I have rectified it in commit 0f3591f. Show quoted text
> Your new tests only use such > arrays, because you make your non-existent elements by assigning to > $#a, > which makes the array permanently RMAGICAL.
Hmm, does that mean that support for tied arrays slows down all @a access once $#a has been mentioned? Maybe we should fix that. (Again, off-topic for this ticket.) Show quoted text
> I'm not a fan of the deferred-element approach. Firstly, it means > that > these uses of arrays in list context suffer from [perl #132729], which > they didn't under my version of the code. It's introducing a bug to > fix > another one, though to be fair [perl #132729] needs to be addressed > more > generally anyway. > > But even if it worked, it means that the deferred elements get created > *every* time the array is used in this kind of context, causing a > bigger > performance drain than just creating the element once. This mechanism > only being required in order to support the sparse array view, it > means > that supporting sparse arrays is imposing a performance penalty on > code > that doesn't use that feature. Further reason to deprecate it.
I have a fix in mind that will make it a one-time penalty and fix #132729, at least for elephants within the array. Namely, we could store the deferred elements in the array and change the way the magic works. I think that can be done for 5.28. -- Father Chrysostomos
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 580b
On Sat, 20 Jan 2018 14:44:17 -0800, zefram@fysh.org wrote: Show quoted text
> Is it a bug? It's a change in behaviour, but the semantics of "local" > don't make clear that one behaviour or the other is correct.
Ten years ago this was considered a bug: commit 4ad10a0b60fb728d1be0a9eeb1970166a3846d38 Author: Vincent Pit <perl@profvince.com> Date: Sun Dec 28 15:08:05 2008 +0100 On scope end, delete localized array elements that should not exist anymore, so that the array recovers its previous length. Honour EXISTS and DELETE for tied arrays. Has any implications changed since then?
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.7k
On Sun, 21 Jan 2018 16:24:55 -0800, sprout wrote: Show quoted text
> On Sat, 20 Jan 2018 14:44:17 -0800, zefram@fysh.org wrote:
> > I'm not a fan of the deferred-element approach. Firstly, it means > > that > > these uses of arrays in list context suffer from [perl #132729], > > which > > they didn't under my version of the code. It's introducing a bug to > > fix > > another one, though to be fair [perl #132729] needs to be addressed > > more > > generally anyway. > > > > But even if it worked, it means that the deferred elements get > > created > > *every* time the array is used in this kind of context, causing a > > bigger > > performance drain than just creating the element once. This > > mechanism > > only being required in order to support the sparse array view, it > > means > > that supporting sparse arrays is imposing a performance penalty on > > code > > that doesn't use that feature. Further reason to deprecate it.
> > I have a fix in mind that will make it a one-time penalty and fix > #132729, at least for elephants within the array. Namely, we could > store the deferred elements in the array and change the way the magic > works. I think that can be done for 5.28.
Please review the ‘Nonelem’ patch in the sprout/nonelem branch. It currently fixes #132729 only for the cases covered by this ticket (S_pushav), but it could be extended to other uses of deferred elements. However, it would only be a partial fix for #132729, not applying to elephants beyond the end of the array. Fixing *that* would be harder, but doable (but probably too intrusive for 5.28). It is worth applying a partial fix? The commit does not yet include tests for #132729 or for the memory leaks it fixes. I also have not had time yet to benchmark it. -- Father Chrysostomos
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.2k
On Sun, 21 Jan 2018 21:59:24 -0800, sprout wrote: Show quoted text
> Please review the ‘Nonelem’ patch in the sprout/nonelem branch. It > currently fixes #132729 only for the cases covered by this ticket > (S_pushav), but it could be extended to other uses of deferred > elements. However, it would only be a partial fix for #132729, not > applying to elephants beyond the end of the array. Fixing *that* > would be harder, but doable (but probably too intrusive for 5.28). It > is worth applying a partial fix?
Replying to myself, I think it is worth it, because most Perl programmers probably do not realize that arrays can have holes in them. I have applied my fix in the branch merged as 2a05854. Show quoted text
> > The commit does not yet include tests for #132729 or for the memory > leaks it fixes.
The branch includes that. Show quoted text
> I also have not had time yet to benchmark it.
I know my method is primitive, but it should not matter considering the stark difference: $ time ./miniperl -e '$#a++; for (1..5000000) { map 1, @a }' real 0m7.991s user 0m7.272s sys 0m0.703s $ time ./newminiperl -e '$#a++; for (1..5000000) { map 1, @a }' real 0m4.503s user 0m4.490s sys 0m0.006s The first is blead before the merge (with my previous, inefficient defelem fix). The second is after the merge. -- Father Chrysostomos
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 4.2k
Here's some comparison between a97021b9d2 (representing point right before this change) and 8c11620dca (representing blead): % perf stat -r 5 -e instructions,branches,branch-misses,cache-misses ./miniperl_a97021b9d2 -e 'sub foo{} $a[5] = 1; foo(@a) for (1..100_000_000)' Performance counter stats for './miniperl_a97021b9d2 -e sub foo{} $a[5] = 1; foo(@a) for (1..100_000_000)' (5 runs): 68,408,669,959 instructions ( +- 0.00% ) 13,001,633,581 branches ( +- 0.00% ) 49,824 branch-misses # 0.00% of all branches ( +- 2.59% ) 17,808 cache-misses ( +- 35.84% ) 7.886012337 seconds time elapsed ( +- 0.45% ) % perf stat -r 5 -e instructions,branches,branch-misses,cache-misses ./miniperl_8c11620dca -e 'sub foo{} $a[5] = 1; foo(@a) for (1..100_000_000)' Performance counter stats for './miniperl_8c11620dca -e sub foo{} $a[5] = 1; foo(@a) for (1..100_000_000)' (5 runs): 68,208,661,954 instructions ( +- 0.00% ) 13,601,633,162 branches ( +- 0.00% ) 50,091 branch-misses # 0.00% of all branches ( +- 1.05% ) 23,055 cache-misses ( +- 25.61% ) 7.753697959 seconds time elapsed ( +- 0.43% ) Calling subs with empty slots takes now ~4% more branches. % perf stat -r 5 -e instructions,branches,branch-misses,cache-misses ./miniperl_a97021b9d2 -e 'sub foo{} @a=(1)x5; foo(@a) for (1..100_000_000)' Performance counter stats for './miniperl_a97021b9d2 -e sub foo{} @a=(1)x5; foo(@a) for (1..100_000_000)' (5 runs): 66,508,170,242 instructions ( +- 0.00% ) 12,601,545,986 branches ( +- 0.00% ) 55,329 branch-misses # 0.00% of all branches ( +- 2.02% ) 20,607 cache-misses ( +- 31.11% ) 7.426998822 seconds time elapsed ( +- 0.09% ) % perf stat -r 5 -e instructions,branches,branch-misses,cache-misses ./miniperl_8c11620dca -e 'sub foo{} @a=(1)x5; foo(@a) for (1..100_000_000)' Performance counter stats for './miniperl_8c11620dca -e sub foo{} @a=(1)x5; foo(@a) for (1..100_000_000)' (5 runs): 66,308,249,789 instructions ( +- 0.00% ) 13,101,560,547 branches ( +- 0.00% ) 69,581 branch-misses # 0.00% of all branches ( +- 3.59% ) 27,929 cache-misses ( +- 17.41% ) 7.480210740 seconds time elapsed ( +- 0.26% ) Calling subs with dense array is, again, taking 4% more branches. Blead is chosen here to demonstrate that nothing fundamentally changed after recent patches from Father Chrysostomos, the main hit was done by 6661956a23 (though instructions improved a bit, but I didn't pinpoint that to specific commit/change): % perf stat -r 5 -e instructions,branches,branch-misses,cache-misses ./miniperl_6661956a23 -e 'sub foo{} $a[5] = 1; foo(@a) for (1..100_000_000)' Performance counter stats for './miniperl_6661956a23 -e sub foo{} $a[5] = 1; foo(@a) for (1..100_000_000)' (5 runs): 68,708,344,437 instructions ( +- 0.00% ) 13,601,578,019 branches ( +- 0.00% ) 56,758 branch-misses # 0.00% of all branches ( +- 8.07% ) 17,868 cache-misses ( +- 35.83% ) 7.620128855 seconds time elapsed ( +- 0.52% )
Download (untitled) / with headers
text/plain 317b
Thank you for filing this report. You have helped make Perl better. With the release yesterday of Perl 5.28.0, this and 185 other issues have been resolved. Perl 5.28.0 may be downloaded via: https://metacpan.org/release/XSAWYERX/perl-5.28.0 If you find that the problem persists, feel free to reopen this ticket.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org