Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure to initialize my() variables #12649

Open
p5pRT opened this issue Dec 16, 2012 · 10 comments
Open

Failure to initialize my() variables #12649

p5pRT opened this issue Dec 16, 2012 · 10 comments

Comments

@p5pRT
Copy link

p5pRT commented Dec 16, 2012

Migrated from rt.perl.org#116110 (status was 'open')

Searchable as RT116110$

@p5pRT
Copy link
Author

p5pRT commented Dec 16, 2012

From tlhackque@yahoo.com

Created by tlhackque@yahoo.com

my() variables don't initialize properly, at least as I read the doc.

Documentation in PERLSUB
  The parameter list to my() may be assigned to if desired, which allows you to initialize your variables. (If no
  initializer is given for a particular variable, it is created with the undefined value.) Commonly this is used to name
  input parameters to a subroutine. Examples​:

Camel book agrees, and is more specific​:
  The EXPR may be assigned to if desired, which allows you to initialize your lexical variables. (If no initializer is given, all scalars are initialzed to the undefined value and all arrays and hashes to the empty list.)

BUG​: Code of the form​:
  my %x = (expression) if( $q );

does not initialize %x if $q is false. In fact, you seem to get the last value of %x in the current scope.

There is no warning or error with use strict; and use warnings in effect;

This result is clearly not what the language intends. The if() modifier makes the assignement conditional. If it's false, the variable is not assigned to, and should have the default empty value.

Either the code should fail to compile, or an undefined/empty value should be supplied if the condition is false.

Leaking a previous value is clearly problematic - may have serious consequences in some usages.

Here is a reasonably small reproducer, distilled from a large project.

The final call to Dumper should show %obj1 and %obj2 with identical state. In fact, obj2 is empty.

The cause can be seen where the second printout of VAR2 shows the previous value of %present.

If the initialization of %present is made a separate statement, the correct behavior is observed.

  E.G. my %present; %present = map { $_ => 1 } @​$list if ($list);

#!/usr/bin/perl

use warnings;
use strict;

use Data​::Dumper;

sub add {
  my $a = shift;

  my $list = $a->{list};
  my %present = map { $_ => 1 } @​$list if ($list);

  print Dumper( \@​_, \%present );

  foreach my $item (@​_) {
  unless ( $present{$item} ) {
  push @​{ $a->{list} }, $item;
  $present{$item} = 1;
  }
  }
}

my %obj1;

add( \%obj1, qw/ONE/ );

my %obj2;
add( \%obj2, qw/ONE/ );

print Dumper( \%obj1, \%obj2 );

exit

Annotated output​:

$VAR1 = [ # First call​: Arg to add()
  'ONE'
  ];
$VAR2 = {}; # Initial value of conditionally-assigned hash %present
$VAR1 = [ # Second call​:
  'ONE'
  ];
$VAR2 = { # Initial value of hash - inherited from first call!
  'ONE' => 1
  };
$VAR1 = { # First object gets its value
  'list' => [
  'ONE'
  ]
  };
$VAR2 = {}; # Second does not

Perl Info

Flags:
    category=core
    severity=critical

This perlbug was built using Perl 5.14.3 in the Fedora build system.
It is being executed now by Perl 5.14.3 - Thu Oct 18 13:30:29 UTC 2012.

Site configuration information for perl 5.14.3:

Configured by Red Hat, Inc. at Thu Oct 18 13:30:29 UTC 2012.

Summary of my perl5 (revision 5 version 14 subversion 3) configuration:
   
  Platform:
    osname=linux, osvers=2.6.32-279.9.1.el6.x86_64, archname=x86_64-linux-thread-multi
    uname='linux buildvm-03.phx2.fedoraproject.org 2.6.32-279.9.1.el6.x86_64 #1 smp fri aug 31 09:04:24 edt 2012 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4  -m64 -mtune=generic -Dccdlflags=-Wl,--enable-new-dtags -Dlddlflags=-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4  -m64 -mtune=generic -Wl,-z,relro  -DDEBUGGING=-g -Dversion=5.14.3 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl -Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbin!
 perl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.7.2 20120921 (Red Hat 4.7.2-2)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -fstack-protector'
    libpth=/usr/local/lib64 /lib64 /usr/lib64
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.15'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--enable-new-dtags -Wl,-rpath,/usr/lib64/perl5/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wl,-z,relro '

Locally applied patches:
    


@INC for perl 5.14.3:
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5
    .


Environment for perl 5.14.3:
    HOME=/root
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Dec 17, 2012

From tlhackque@yahoo.com

I see from the meta ticket that (unsurprsingly), I'm not the first to
run into this in one form or another.

A little more data​:

We scanned the project with perlcritic and found 16 instances of this
construct. No particular pattern with respect to level of experience
of those who wrote them. Language lawyer and casual coders were
equally likely to fall into the trap. People who wrote one were likely
to use the construct again. So "here there be dragons" in perlsyn has
not been effective in causing people to write reliable code.

In every case, the intended semantics were my foo; foo = () if();
We found hashes, scalars, and my($x,$y) = constructs as the target of
the conditional assignments.

We didn't encounter any of the subtle scoping case
my $foo = ;
{
my $foo = $foo + 7 if();
}
(Not to say they're not out there.)

I have not seen a case in real code where someone relied (much less
reasonably relied) on an assigned value other than (undef or ()) or the
conditional assignment. But I have seen serious breakage as a result.
(The test case illustrates code that silently disabled a major feature.)

Therefore, the minimal impact to existing code is NOT to add a compile
error, warning or deprecation - but simply DWIM.

We also got a number of hits in CPAN modules that ship with our
project; these have not been investigated.

The reason I set the severity so high is that uninitialized data is a
typical breeding ground for security issues. We haven't encountered
one yet, but somewhere I wouldn't be suprised to find code of the form

my %task = ( logged_in => 1) if( $session->authenticated );

Based on all these observations, a core perl fix seems indicated.
The correct semantics seem to be​:
  "my foo; foo = () if(); - except that the scope of the new variable
materializes after the ;"

It would make sense to add a release note advising that Perl​::Critic
can detect any case where these assignments were made, and that code
should be inspected to determine if it relied on another behavior.

I hope this additional data is helpful.

On Sun Dec 16 09​:12​:33 2012, tlhackque wrote​:

This is a bug report for perl from tlhackque@​yahoo.com,
generated with the help of perlbug 1.39 running under perl 5.14.3.

-----------------------------------------------------------------
[Please describe your issue here]

my() variables don't initialize properly, at least as I read the doc.

Documentation in PERLSUB
The parameter list to my() may be assigned to if desired, which
allows you to initialize your variables. (If no
initializer is given for a particular variable, it is created
with the undefined value.) Commonly this is used to name
input parameters to a subroutine. Examples​:

Camel book agrees, and is more specific​:
The EXPR may be assigned to if desired, which allows you to
initialize your lexical variables. (If no initializer is given, all
scalars are initialzed to the undefined value and all arrays and
hashes to the empty list.)

BUG​: Code of the form​:
my %x = (expression) if( $q );

does not initialize %x if $q is false. In fact, you seem to get the
last value of %x in the current scope.

There is no warning or error with use strict; and use warnings in
effect;

This result is clearly not what the language intends. The if()
modifier makes the assignement conditional. If it's false, the
variable is not assigned to, and should have the default empty
value.

Either the code should fail to compile, or an undefined/empty value
should be supplied if the condition is false.

Leaking a previous value is clearly problematic - may have serious
consequences in some usages.

Here is a reasonably small reproducer, distilled from a large project.

The final call to Dumper should show %obj1 and %obj2 with identical
state. In fact, obj2 is empty.

The cause can be seen where the second printout of VAR2 shows the
previous value of %present.

If the initialization of %present is made a separate statement, the
correct behavior is observed.

E.G. my %present; %present = map { $_ => 1 } @​$list if ($list);

#!/usr/bin/perl

use warnings;
use strict;

use Data​::Dumper;

sub add {
my $a = shift;

my $list = $a\->\{list\};
my %present = map \{ $\_ => 1 \} @​$list if \($list\);

print Dumper\( \\@​\_\, \\%present \);

foreach my $item \(@​\_\) \{
    unless \( $present\{$item\} \) \{
        push @​\{ $a\->\{list\} \}\, $item;
        $present\{$item\} = 1;
    \}
\}

}

my %obj1;

add( \%obj1, qw/ONE/ );

my %obj2;
add( \%obj2, qw/ONE/ );

print Dumper( \%obj1, \%obj2 );

exit

Annotated output​:

$VAR1 = [ # First call​: Arg to add()
'ONE'
];
$VAR2 = {}; # Initial value of conditionally-assigned hash %present
$VAR1 = [ # Second call​:
'ONE'
];
$VAR2 = { # Initial value of hash - inherited from first call!
'ONE' => 1
};
$VAR1 = { # First object gets its value
'list' => [
'ONE'
]
};
$VAR2 = {}; # Second does not

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=critical
---
This perlbug was built using Perl 5.14.3 in the Fedora build system.
It is being executed now by Perl 5.14.3 - Thu Oct 18 13​:30​:29 UTC
2012.

Site configuration information for perl 5.14.3​:

Configured by Red Hat, Inc. at Thu Oct 18 13​:30​:29 UTC 2012.

Summary of my perl5 (revision 5 version 14 subversion 3)
configuration​:

Platform​:
osname=linux, osvers=2.6.32-279.9.1.el6.x86_64, archname=x86_64-
linux-thread-multi
uname='linux buildvm-03.phx2.fedoraproject.org 2.6.32-
279.9.1.el6.x86_64 #1 smp fri aug 31 09​:04​:24 edt 2012 x86_64
x86_64 x86_64 gnulinux '
config_args='-des -Doptimize=-O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic
-Dccdlflags=-Wl,--enable-new-dtags -Dlddlflags=-shared -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -Wl,-z,relro
-DDEBUGGING=-g -Dversion=5.14.3 -Dmyhostname=localhost
-Dperladmin=root@​localhost -Dcc=gcc -Dcf_by=Red Hat, Inc.
-Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5
-Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl
-Darchlib=/usr/lib64/perl5
-Dvendorarch=/usr/lib64/perl5/vendor_perl
-Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64
/lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads
-Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db
-Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio
-Dinstallusrbin!
perl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr
-Dd_gethostent_r_proto -Ud_endhostent_r_proto
-Ud_sethostent_r_proto -Ud_endprotoent_r_proto
-Ud_setprotoent_r_proto -Ud_endservent_r_proto
-Ud_setservent_r_proto -Dscriptdir=/usr/bin'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define,
usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.7.2 20120921 (Red Hat 4.7.2-2)',
gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define,
longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='gcc', ldflags =' -fstack-protector'
libpth=/usr/local/lib64 /lib64 /usr/lib64
libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread
-lc -lgdbm_compat
perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.15'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--
enable-new-dtags -Wl,-rpath,/usr/lib64/perl5/CORE'
cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -Wl,-z,relro '

Locally applied patches​:

---
@​INC for perl 5.14.3​:
/usr/local/lib64/perl5
/usr/local/share/perl5
/usr/lib64/perl5/vendor_perl
/usr/share/perl5/vendor_perl
/usr/lib64/perl5
/usr/share/perl5
.

---
Environment for perl 5.14.3​:
HOME=/root
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/lib64/qt-

3.3/bin​:/usr/lib64/ccache​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/
bin​:/root/bin

PERL\_BADLANG \(unset\)
SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Dec 17, 2012

From [Unknown Contact. See original ticket]

I see from the meta ticket that (unsurprsingly), I'm not the first to
run into this in one form or another.

A little more data​:

We scanned the project with perlcritic and found 16 instances of this
construct. No particular pattern with respect to level of experience
of those who wrote them. Language lawyer and casual coders were
equally likely to fall into the trap. People who wrote one were likely
to use the construct again. So "here there be dragons" in perlsyn has
not been effective in causing people to write reliable code.

In every case, the intended semantics were my foo; foo = () if();
We found hashes, scalars, and my($x,$y) = constructs as the target of
the conditional assignments.

We didn't encounter any of the subtle scoping case
my $foo = ;
{
my $foo = $foo + 7 if();
}
(Not to say they're not out there.)

I have not seen a case in real code where someone relied (much less
reasonably relied) on an assigned value other than (undef or ()) or the
conditional assignment. But I have seen serious breakage as a result.
(The test case illustrates code that silently disabled a major feature.)

Therefore, the minimal impact to existing code is NOT to add a compile
error, warning or deprecation - but simply DWIM.

We also got a number of hits in CPAN modules that ship with our
project; these have not been investigated.

The reason I set the severity so high is that uninitialized data is a
typical breeding ground for security issues. We haven't encountered
one yet, but somewhere I wouldn't be suprised to find code of the form

my %task = ( logged_in => 1) if( $session->authenticated );

Based on all these observations, a core perl fix seems indicated.
The correct semantics seem to be​:
  "my foo; foo = () if(); - except that the scope of the new variable
materializes after the ;"

It would make sense to add a release note advising that Perl​::Critic
can detect any case where these assignments were made, and that code
should be inspected to determine if it relied on another behavior.

I hope this additional data is helpful.

On Sun Dec 16 09​:12​:33 2012, tlhackque wrote​:

This is a bug report for perl from tlhackque@​yahoo.com,
generated with the help of perlbug 1.39 running under perl 5.14.3.

-----------------------------------------------------------------
[Please describe your issue here]

my() variables don't initialize properly, at least as I read the doc.

Documentation in PERLSUB
The parameter list to my() may be assigned to if desired, which
allows you to initialize your variables. (If no
initializer is given for a particular variable, it is created
with the undefined value.) Commonly this is used to name
input parameters to a subroutine. Examples​:

Camel book agrees, and is more specific​:
The EXPR may be assigned to if desired, which allows you to
initialize your lexical variables. (If no initializer is given, all
scalars are initialzed to the undefined value and all arrays and
hashes to the empty list.)

BUG​: Code of the form​:
my %x = (expression) if( $q );

does not initialize %x if $q is false. In fact, you seem to get the
last value of %x in the current scope.

There is no warning or error with use strict; and use warnings in
effect;

This result is clearly not what the language intends. The if()
modifier makes the assignement conditional. If it's false, the
variable is not assigned to, and should have the default empty
value.

Either the code should fail to compile, or an undefined/empty value
should be supplied if the condition is false.

Leaking a previous value is clearly problematic - may have serious
consequences in some usages.

Here is a reasonably small reproducer, distilled from a large project.

The final call to Dumper should show %obj1 and %obj2 with identical
state. In fact, obj2 is empty.

The cause can be seen where the second printout of VAR2 shows the
previous value of %present.

If the initialization of %present is made a separate statement, the
correct behavior is observed.

E.G. my %present; %present = map { $_ => 1 } @​$list if ($list);

#!/usr/bin/perl

use warnings;
use strict;

use Data​::Dumper;

sub add {
my $a = shift;

my $list = $a\->\{list\};
my %present = map \{ $\_ => 1 \} @​$list if \($list\);

print Dumper\( \\@​\_\, \\%present \);

foreach my $item \(@​\_\) \{
    unless \( $present\{$item\} \) \{
        push @​\{ $a\->\{list\} \}\, $item;
        $present\{$item\} = 1;
    \}
\}

}

my %obj1;

add( \%obj1, qw/ONE/ );

my %obj2;
add( \%obj2, qw/ONE/ );

print Dumper( \%obj1, \%obj2 );

exit

Annotated output​:

$VAR1 = [ # First call​: Arg to add()
'ONE'
];
$VAR2 = {}; # Initial value of conditionally-assigned hash %present
$VAR1 = [ # Second call​:
'ONE'
];
$VAR2 = { # Initial value of hash - inherited from first call!
'ONE' => 1
};
$VAR1 = { # First object gets its value
'list' => [
'ONE'
]
};
$VAR2 = {}; # Second does not

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=critical
---
This perlbug was built using Perl 5.14.3 in the Fedora build system.
It is being executed now by Perl 5.14.3 - Thu Oct 18 13​:30​:29 UTC
2012.

Site configuration information for perl 5.14.3​:

Configured by Red Hat, Inc. at Thu Oct 18 13​:30​:29 UTC 2012.

Summary of my perl5 (revision 5 version 14 subversion 3)
configuration​:

Platform​:
osname=linux, osvers=2.6.32-279.9.1.el6.x86_64, archname=x86_64-
linux-thread-multi
uname='linux buildvm-03.phx2.fedoraproject.org 2.6.32-
279.9.1.el6.x86_64 #1 smp fri aug 31 09​:04​:24 edt 2012 x86_64
x86_64 x86_64 gnulinux '
config_args='-des -Doptimize=-O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic
-Dccdlflags=-Wl,--enable-new-dtags -Dlddlflags=-shared -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -Wl,-z,relro
-DDEBUGGING=-g -Dversion=5.14.3 -Dmyhostname=localhost
-Dperladmin=root@​localhost -Dcc=gcc -Dcf_by=Red Hat, Inc.
-Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5
-Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl
-Darchlib=/usr/lib64/perl5
-Dvendorarch=/usr/lib64/perl5/vendor_perl
-Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64
/lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads
-Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db
-Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio
-Dinstallusrbin!
perl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr
-Dd_gethostent_r_proto -Ud_endhostent_r_proto
-Ud_sethostent_r_proto -Ud_endprotoent_r_proto
-Ud_setprotoent_r_proto -Ud_endservent_r_proto
-Ud_setservent_r_proto -Dscriptdir=/usr/bin'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define,
usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.7.2 20120921 (Red Hat 4.7.2-2)',
gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define,
longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='gcc', ldflags =' -fstack-protector'
libpth=/usr/local/lib64 /lib64 /usr/lib64
libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread
-lc -lgdbm_compat
perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.15'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--
enable-new-dtags -Wl,-rpath,/usr/lib64/perl5/CORE'
cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -Wl,-z,relro '

Locally applied patches​:

---
@​INC for perl 5.14.3​:
/usr/local/lib64/perl5
/usr/local/share/perl5
/usr/lib64/perl5/vendor_perl
/usr/share/perl5/vendor_perl
/usr/lib64/perl5
/usr/share/perl5
.

---
Environment for perl 5.14.3​:
HOME=/root
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/lib64/qt-

3.3/bin​:/usr/lib64/ccache​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/
bin​:/root/bin

PERL\_BADLANG \(unset\)
SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Dec 17, 2012

tlhackque@yahoo.com - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 17, 2012

From tlhackque@yahoo.com

In reviewing all the related items, I noted some confusion about what
the right semantics for "my foo = EXPR (modifier)" should be.

Here's an attempt at a precise formulation​:

When a my declaration includes an initializing expression, it is
executed AS IF the following sequence of steps is taken​:

1. For each item on the left hand side of the declaration, an
anonomyous piece of storage is allocated, of the type indicated by the
item's SIGIL. Each of these is initialized to the null value (undef or
()) matching its type.
2. The right hand side of the initializing expression is executed
  as if its left hand side was the (scalar or array of) anonymous
items.
  - A false conditional produces no result, and thus no assignment.
  - A looping modifier assigns to the storage once for each trip that
  it executes. (zero trips thru the loop => zero assignments)
3. The anonymous storage items are named according to the my at the
same sequence point as an unmodified my() is today.

Note that a my() declaration embedded in an if() or while() gets no
special treatment (beyond what it gets today).

  my( $a, $q ) = (2, 4);
  if( (my $a = $a +1 if( $nil )) || $q ) { print $a || $q; } print $a;

is well defined under this definition, and equivalent to

  print ( $nil? (2+1) : 4 );
  print 2;

This doesn't require that the implementation goes thru all 3 steps in
every case; there are ovbvious shortcuts. Implementations need only
produce equivalent results. But I think it covers all statement types
with deterministic results and all scoping matters.

Some examples​:

  my %foo = @​bar if( $nil );
  allocate an anonymous hash, initialized empty.
  evaluate %anon = @​bar if( $nil );
  name %anon %foo.

  my( $a, $b, @​c ) = ( 1, 2, @​_ ) if( $nil );
  allocate ($anon_a, $anon_b, @​anon_c) = ( undef, undef, () );
  evaluate ($anon_a, $anon_b, @​anon_c) = (1,2, @​_ ) if ( $nil );
  name $anon_a $a
  name $anon_b $b
  name @​anon_c @​c

A pretty ovbious generalization is that = can now be any sensible
assignment operator​:
  my $a ||= $b[$_] for( 0..20);

e.g. .= isn't sensible as it's guaranteed a runtime exception (undef .
anything)

  my $a = 11;
  my @​b = (...); # Also consider my @​b = (); ...
  {
  my $a += $a + $_ foreach( @​b );
  ...
  }
  allocate $anon_a = undef
  evaluate $anon_a += 11 + $_ foreach( @​b); (scalar numeric context;
undef == 0)
  name $anon_a $a;
  (...)

Some obvious shortcuts​:
  If the initializing expression is simple (no conditional or loop),
  no change from today.
  If a conditional is constant FALSE, don't bother with anonymous/rename
  If the modifier is a looping construct with no dependency on the loop
index (including the assignement operator), only assign on the last
iteration of the loop. (e.g. += 1 can't be skipped easily, but = 1
can.)

Others will come to mind.

I know next to nothing about Perl internals, so if I happened to pick
pseudo-ops with special meanings, please substitute the generic intent.

As always, thanks for all the effort you put into maintaining Perl. I
hope this note advances a solution.

@p5pRT
Copy link
Author

p5pRT commented Dec 17, 2012

From [Unknown Contact. See original ticket]

In reviewing all the related items, I noted some confusion about what
the right semantics for "my foo = EXPR (modifier)" should be.

Here's an attempt at a precise formulation​:

When a my declaration includes an initializing expression, it is
executed AS IF the following sequence of steps is taken​:

1. For each item on the left hand side of the declaration, an
anonomyous piece of storage is allocated, of the type indicated by the
item's SIGIL. Each of these is initialized to the null value (undef or
()) matching its type.
2. The right hand side of the initializing expression is executed
  as if its left hand side was the (scalar or array of) anonymous
items.
  - A false conditional produces no result, and thus no assignment.
  - A looping modifier assigns to the storage once for each trip that
  it executes. (zero trips thru the loop => zero assignments)
3. The anonymous storage items are named according to the my at the
same sequence point as an unmodified my() is today.

Note that a my() declaration embedded in an if() or while() gets no
special treatment (beyond what it gets today).

  my( $a, $q ) = (2, 4);
  if( (my $a = $a +1 if( $nil )) || $q ) { print $a || $q; } print $a;

is well defined under this definition, and equivalent to

  print ( $nil? (2+1) : 4 );
  print 2;

This doesn't require that the implementation goes thru all 3 steps in
every case; there are ovbvious shortcuts. Implementations need only
produce equivalent results. But I think it covers all statement types
with deterministic results and all scoping matters.

Some examples​:

  my %foo = @​bar if( $nil );
  allocate an anonymous hash, initialized empty.
  evaluate %anon = @​bar if( $nil );
  name %anon %foo.

  my( $a, $b, @​c ) = ( 1, 2, @​_ ) if( $nil );
  allocate ($anon_a, $anon_b, @​anon_c) = ( undef, undef, () );
  evaluate ($anon_a, $anon_b, @​anon_c) = (1,2, @​_ ) if ( $nil );
  name $anon_a $a
  name $anon_b $b
  name @​anon_c @​c

A pretty ovbious generalization is that = can now be any sensible
assignment operator​:
  my $a ||= $b[$_] for( 0..20);

e.g. .= isn't sensible as it's guaranteed a runtime exception (undef .
anything)

  my $a = 11;
  my @​b = (...); # Also consider my @​b = (); ...
  {
  my $a += $a + $_ foreach( @​b );
  ...
  }
  allocate $anon_a = undef
  evaluate $anon_a += 11 + $_ foreach( @​b); (scalar numeric context;
undef == 0)
  name $anon_a $a;
  (...)

Some obvious shortcuts​:
  If the initializing expression is simple (no conditional or loop),
  no change from today.
  If a conditional is constant FALSE, don't bother with anonymous/rename
  If the modifier is a looping construct with no dependency on the loop
index (including the assignement operator), only assign on the last
iteration of the loop. (e.g. += 1 can't be skipped easily, but = 1
can.)

Others will come to mind.

I know next to nothing about Perl internals, so if I happened to pick
pseudo-ops with special meanings, please substitute the generic intent.

As always, thanks for all the effort you put into maintaining Perl. I
hope this note advances a solution.

@p5pRT
Copy link
Author

p5pRT commented Dec 23, 2012

From @cpansprout

On Mon Dec 17 08​:06​:40 2012, tlhackque wrote​:

In reviewing all the related items, I noted some confusion about what
the right semantics for "my foo = EXPR (modifier)" should be.

Here's an attempt at a precise formulation​:

When a my declaration includes an initializing expression, it is
executed AS IF the following sequence of steps is taken​:

1. For each item on the left hand side of the declaration, an
anonomyous piece of storage is allocated, of the type indicated by the
item's SIGIL. Each of these is initialized to the null value (undef or
()) matching its type.

The problem here is that in ‘my $x if $false’ the ‘my $x’ is never executed.

‘my $x’ conceptually creates the scalar, so ‘my $x if 0; $x’ ends up
referring to a scalar that conceptually does not exist. Since perl
cheats and reuses the same scalar for efficiency, this has the effect of
leaking values from one entry of the surrounding scope to the next.

I agree this should be fixed, but I disagree that it is high priority.
I also don’t offhand see a way to fix it that would not slow down early
returns (return if $foo; my $x....). (Maybe we need SAVEt_PAD and
pp_padsv should turn off PADSTALE unconditionally; I don’t know....)

BTW, this is how you can tell perl is reusing the scalar​:

$ perl -le 'for(1..5) { print \my $x }'
SCALAR(0x826db0)
SCALAR(0x826db0)
SCALAR(0x826db0)
SCALAR(0x826db0)
SCALAR(0x826db0)

But it knows how to hide that fact when the scalar is referenced elsewhere​:

$ perl -le 'for(1..5) { print \my $x; push @​_, \$x }'
SCALAR(0x826db0)
SCALAR(0x803bb0)
SCALAR(0x826d30)
SCALAR(0x803aa0)
SCALAR(0x826df0)

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Dec 24, 2012

From @ap

* Father Chrysostomos via RT <perlbug-followup@​perl.org> [2012-12-23 22​:10]​:

The problem here is that in ‘my $x if $false’ the ‘my $x’ is never
executed.

‘my $x’ conceptually creates the scalar, so ‘my $x if 0; $x’ ends up
referring to a scalar that conceptually does not exist. Since perl
cheats and reuses the same scalar for efficiency, this has the effect
of leaking values from one entry of the surrounding scope to the next.

I agree this should be fixed, but I disagree that it is high priority.
I also don’t offhand see a way to fix it that would not slow down
early returns (return if $foo; my $x....). (Maybe we need SAVEt_PAD
and pp_padsv should turn off PADSTALE unconditionally; I don’t
know....)

Note that the problem can only possibly happen with a statement modifier
and only on its left side, or else within an expression with short-
circuit operators (which the statement modifier `if` gets compiled into,
AFAIK). Maybe it is possible to do some sort of fixup on the op tree to
lift `my` declarations from underneath the right side of conditional op
into a separate statement, to effectively turn

  my $x = "foo" if $y;

into the equivalent of

  my $x;
  $x = "foo" if $y;
?

Although now I remember groans about op tree diddling being fragile code
to write… and also faintly remember ideas about making it easier… maybe
this is something that will have to wait for that infrastructure?

(Please disregard if this is all nonsense.)

@p5pRT
Copy link
Author

p5pRT commented Dec 24, 2012

From @iabyn

On Mon, Dec 24, 2012 at 01​:30​:15AM +0100, Aristotle Pagaltzis wrote​:

* Father Chrysostomos via RT <perlbug-followup@​perl.org> [2012-12-23 22​:10]​:

The problem here is that in ‘my $x if $false’ the ‘my $x’ is never
executed.

‘my $x’ conceptually creates the scalar, so ‘my $x if 0; $x’ ends up
referring to a scalar that conceptually does not exist. Since perl
cheats and reuses the same scalar for efficiency, this has the effect
of leaking values from one entry of the surrounding scope to the next.

I agree this should be fixed, but I disagree that it is high priority.
I also don’t offhand see a way to fix it that would not slow down
early returns (return if $foo; my $x....). (Maybe we need SAVEt_PAD
and pp_padsv should turn off PADSTALE unconditionally; I don’t
know....)

Note that the problem can only possibly happen with a statement modifier
and only on its left side, or else within an expression with short-
circuit operators (which the statement modifier `if` gets compiled into,
AFAIK).

There's also

  goto FOO;
  my $x;
  FOO​:

--
The Enterprise successfully ferries an alien VIP from one place to another
without serious incident.
  -- Things That Never Happen in "Star Trek" #7

@p5pRT
Copy link
Author

p5pRT commented Dec 24, 2012

From @ap

* Dave Mitchell <davem@​iabyn.com> [2012-12-24 21​:55]​:

There's also

goto FOO;
my $x;
FOO​:

Nasty. I guess optree diddling is off the table.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants