From zefram@fysh.org

Created by zefram@fysh.org

Ran into this while tweaking Carp's localisation of $!​:

$ perl -le 'sub aa { local $!; ref($_[0]); 1 } $!=69; aa($!); print 0+$!' 
0

$! is getting squashed to 0 despite local $!. This only happens
if ref() gets applied to the $! argument, in its guise as $_[0].
Doesn't happen with ref($!) or if the argument is something else.
If $! is set to a different value within the dynamic scope of "local
$!", that new value persists after the aa() call, regardless of whether
the new value is assigned before or after the ref($_[0]). Also, if
local $! is removed then ref($_[0]) has no effect on $! (though $! can
still be directly clobbered, of course). So evidentily the ref($_[0])
is actually breaking the localisation per se.

Flags:
    category=core
    severity=low

Site configuration information for perl 5.18.1:

Configured by zefram at Thu Aug 15 19:33:26 BST 2013.

Summary of my perl5 (revision 5 version 18 subversion 1) configuration:
   
  Platform:
    osname=linux, osvers=3.2.0-4-amd64, archname=x86_64-linux-thread-multi
    uname='linux barba.rous.org 3.2.0-4-amd64 #1 smp debian 3.2.46-1 x86_64 gnulinux '
    config_args='-des -Dprefix=/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52 -Duselargefiles -Dusethreads -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dusedevel -Uversiononly -Ui_db'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.7.2', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
    libs=-lnsl -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.15'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52/lib/5.18.1/x86_64-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Locally applied patches:
    


@INC for perl 5.18.1:
    /home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52/lib/site_perl/5.18.1/x86_64-linux-thread-multi
    /home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52/lib/site_perl/5.18.1
    /home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52/lib/5.18.1/x86_64-linux-thread-multi
    /home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52/lib/5.18.1
    .


Environment for perl 5.18.1:
    HOME=/home/zefram
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52/bin:/home/zefram/usr/perl/util:/home/zefram/pub/x86_64-unknown-linux-gnu/bin:/home/zefram/pub/common/bin:/usr/bin:/bin:/usr/local/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

From victor@vsespb.ru

also​:

perl -le 'sub aa { local $!; $_[0] = 4; 1 } $!=69; aa($!); print 0+$!'
4
perl -le 'sub aa { local $x; $_[0] = 4; 1 } $x=69; aa($x); print 0+$!'
0

similar issue (but with $1, which should be auto-localized)​:
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=54728

On Mon Sep 09 12​:28​:08 2013, zefram@​fysh.org wrote​:

This is a bug report for perl from zefram@​fysh.org,
generated with the help of perlbug 1.39 running under perl 5.18.1.

-----------------------------------------------------------------
[Please describe your issue here]

Ran into this while tweaking Carp's localisation of $!​:

$ perl -le 'sub aa { local $!; ref($_[0]); 1 } $!=69; aa($!); print 0+$!'
0

$! is getting squashed to 0 despite local $!. This only happens
if ref() gets applied to the $! argument, in its guise as $_[0].
Doesn't happen with ref($!) or if the argument is something else.
If $! is set to a different value within the dynamic scope of "local
$!", that new value persists after the aa() call, regardless of
whether
the new value is assigned before or after the ref($_[0]). Also, if
local $! is removed then ref($_[0]) has no effect on $! (though $!
can
still be directly clobbered, of course). So evidentily the ref($_[0])
is actually breaking the localisation per se.

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=low
---
Site configuration information for perl 5.18.1​:

Configured by zefram at Thu Aug 15 19​:33​:26 BST 2013.

Summary of my perl5 (revision 5 version 18 subversion 1)
configuration​:

Platform​:
osname=linux, osvers=3.2.0-4-amd64, archname=x86_64-linux-thread-
multi
uname='linux barba.rous.org 3.2.0-4-amd64 #1 smp debian 3.2.46-1
x86_64 gnulinux '
config_args='-des
-Dprefix=/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52
-Duselargefiles -Dusethreads -Uafs -Ud_csh -Uusesfio -Uusenm
-Duseshrplib -Dusedevel -Uversiononly -Ui_db'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define,
usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.7.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define,
longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
libs=-lnsl -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.15'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E

-Wl,-rpath,/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52/lib/5.18.1/x86_64-linux-thread-multi/CORE'

cccdlflags='\-fPIC'\, lddlflags='\-shared \-O2 \-L/usr/local/lib

-fstack-protector'

Locally applied patches​:

---
@​INC for perl 5.18.1​:
/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-
f52/lib/site_perl/5.18.1/x86_64-linux-thread-multi
/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-
f52/lib/site_perl/5.18.1
/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-
f52/lib/5.18.1/x86_64-linux-thread-multi
/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52/lib/5.18.1
.

---
Environment for perl 5.18.1​:
HOME=/home/zefram
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-
f52/bin​:/home/zefram/usr/perl/util​:/home/zefram/pub/x86_64-unknown-
linux-

gnu/bin​:/home/zefram/pub/common/bin​:/usr/bin​:/bin​:/usr/local/bin​:/usr/games

PERL\_BADLANG \(unset\)
SHELL=/usr/bin/zsh

The RT System itself - Status changed from 'new' to 'open'

From @ikegami

On Mon, Sep 9, 2013 at 3​:36 PM, Victor Efimov via RT <
perlbug-followup@​perl.org> wrote​:

also​:

perl -le 'sub aa { local $!; $_[0] = 4; 1 } $!=69; aa($!); print 0+$!'
4
perl -le 'sub aa { local $x; $_[0] = 4; 1 } $x=69; aa($x); print 0+$!'
0

I think you meant

$ perl -le 'sub aa { local $!; $_[0] = 4; 1 } $!=69; aa($!); print 0+$!'
4
$ perl -le 'sub aa { local $x; $_[0] = 4; 1 } $x=69; aa($x); print 0+$x'
4

I'm not sure that's a problem. You're modifying the arg, and the arg is the
caller's $! or $x.

From zefram@fysh.org

Victor Efimov via RT wrote​:

similar issue (but with $1, which should be auto-localized)​:
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=54728

Not much similarity there. That's about an argument $1 surprisingly
getting the new value of $1 when it is stringified following a regexp
match in the sub. Its localisation, such as it is, works fine.
My problem, in contrast, is that certain operations on @​_ break the
localisation of $!.

The breakage doesn't require ref(). If 'my $a = shift' is put in place of
'ref($_[0])', the localisation still gets broken.

-zefram

From victor@vsespb.ru

actually my example above contain typo​: "perl -le 'sub aa { local $x;
$_[0] = 4; 1 } $x=69; aa($x); print 0+$!'"
better example​:

our $o;

sub foo {
  local $o = 'a';
  print $_[0];
  print "\n";
  $_[0] = 4;
}
$o = 123;
foo($o);
print $o;
print "\n";

prints​:
123
4

same is printed if "local $o = 'a'" removed

our $o;

sub foo {
  print $_[0];
  print "\n";
  $_[0] = 4;
}
$o = 123;
foo($o);
print $o;
print "\n";

prints​:
123
4

On Mon Sep 09 12​:36​:14 2013, vsespb wrote​:

also​:

perl -le 'sub aa { local $!; $_[0] = 4; 1 } $!=69; aa($!); print 0+$!'
4
perl -le 'sub aa { local $x; $_[0] = 4; 1 } $x=69; aa($x); print 0+$!'
0

similar issue (but with $1, which should be auto-localized)​:
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=54728

On Mon Sep 09 12​:28​:08 2013, zefram@​fysh.org wrote​:

This is a bug report for perl from zefram@​fysh.org,
generated with the help of perlbug 1.39 running under perl 5.18.1.

-----------------------------------------------------------------
[Please describe your issue here]

Ran into this while tweaking Carp's localisation of $!​:

$ perl -le 'sub aa { local $!; ref($_[0]); 1 } $!=69; aa($!); print
0+$!'
0

$! is getting squashed to 0 despite "local $!". This only happens
if ref() gets applied to the $! argument, in its guise as $_[0].
Doesn't happen with ref($!) or if the argument is something else.
If $! is set to a different value within the dynamic scope of "local
$!", that new value persists after the aa() call, regardless of
whether
the new value is assigned before or after the ref($_[0]). Also, if
"local $!" is removed then ref($_[0]) has no effect on $! (though $!
can
still be directly clobbered, of course). So evidentily the
ref($_[0])
is actually breaking the localisation per se.

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=low
---
Site configuration information for perl 5.18.1​:

Configured by zefram at Thu Aug 15 19​:33​:26 BST 2013.

Summary of my perl5 (revision 5 version 18 subversion 1)
configuration​:

Platform​:
osname=linux, osvers=3.2.0-4-amd64, archname=x86_64-linux-
thread-
multi
uname='linux barba.rous.org 3.2.0-4-amd64 #1 smp debian 3.2.46-1
x86_64 gnulinux '
config_args='-des
-Dprefix=/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52
-Duselargefiles -Dusethreads -Uafs -Ud_csh -Uusesfio -Uusenm
-Duseshrplib -Dusedevel -Uversiononly -Ui_db'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define,
usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE
-fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.7.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8,
byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define,
longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8,
Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
libs=-lnsl -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.15'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E

-Wl,-rpath,/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-f52/lib/5.18.1/x86_64-linux-thread-multi/CORE'

cccdlflags='\-fPIC'\, lddlflags='\-shared \-O2 \-L/usr/local/lib

-fstack-protector'

Locally applied patches​:

---
@​INC for perl 5.18.1​:
/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-
f52/lib/site_perl/5.18.1/x86_64-linux-thread-multi
/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-
f52/lib/site_perl/5.18.1
/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-
f52/lib/5.18.1/x86_64-linux-thread-multi
/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-
f52/lib/5.18.1
.

---
Environment for perl 5.18.1​:
HOME=/home/zefram
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/zefram/usr/perl/perl_install/perl-5.18.1-i64-
f52/bin​:/home/zefram/usr/perl/util​:/home/zefram/pub/x86_64-unknown-
linux-

gnu/bin​:/home/zefram/pub/common/bin​:/usr/bin​:/bin​:/usr/local/bin​:/usr/games

PERL\_BADLANG \(unset\)
SHELL=/usr/bin/zsh

From Eirik-Berg.Hanssen@allverden.no

On Mon, Sep 9, 2013 at 9​:36 PM, Victor Efimov via RT <
perlbug-followup@​perl.org> wrote​:

also​:

perl -le 'sub aa { local $!; $_[0] = 4; 1 } $!=69; aa($!); print 0+$!'
4
perl -le 'sub aa { local $x; $_[0] = 4; 1 } $x=69; aa($x); print 0+$!'
0

and​:

perl -le 'sub aa { local $/; $_[0] = 4; 1 } $/=69; aa($/); print 0+$/'
4
perl -le 'sub aa { local $%; $_[0] = 4; 1 } $%=69; aa($%); print 0+$%'
4

but​:
perl -le 'sub aa { local $/; ref($_[0]); 1 } $/=69; aa($/); print 0+$/'
69
perl -le 'sub aa { local $%; ref($_[0]); 1 } $%=69; aa($%); print 0+$%'
0

# oops ...

Eirik

From @cpansprout

On Mon Sep 09 13​:28​:01 2013, zefram@​fysh.org wrote​:

Victor Efimov via RT wrote​:

similar issue (but with $1, which should be auto-localized)​:
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=54728

Not much similarity there. That's about an argument $1 surprisingly
getting the new value of $1 when it is stringified following a regexp
match in the sub. Its localisation, such as it is, works fine.
My problem, in contrast, is that certain operations on @​_ break the
localisation of $!.

The breakage doesn't require ref(). If 'my $a = shift' is put in place of
'ref($_[0])', the localisation still gets broken.

No doubt it’s get-magic that triggers it. This is probably related to
#104118.

--

Father Chrysostomos

From @Leont

On Mon, Sep 9, 2013 at 9​:28 PM, Zefram <perlbug-followup@​perl.org> wrote​:

Ran into this while tweaking Carp's localisation of $!​:

$ perl -le 'sub aa { local $!; ref($_[0]); 1 } $!=69; aa($!); print 0+$!'
0

Also​:

perl -le 'sub aa { local $! = $! } $!=69; aa(); print 0+$!'
0
perl -le 'sub aa { local $! = $! + 0 } $!=69; aa(); print 0+$!'
69

$! is getting squashed to 0 despite "local $!". This only happens

if ref() gets applied to the $! argument, in its guise as $_[0].
Doesn't happen with ref($!) or if the argument is something else.
If $! is set to a different value within the dynamic scope of "local
$!", that new value persists after the aa() call, regardless of whether
the new value is assigned before or after the ref($_[0]). Also, if
"local $!" is removed then ref($_[0]) has no effect on $! (though $! can
still be directly clobbered, of course). So evidentily the ref($_[0])
is actually breaking the localisation per se.

I don't quite understand what's going on here…

Leon

From @epa

Is this not another of those bugs that would all be squashed if magic
variables like $!, $1 and so on were made pass-by-value instead of pass-
by-reference?

So when you call foo($!), foo gets its own scalar whose initial value is
copied from $!, but changes made to that scalar don't affect $!. Yes, this
is inconsistent with what happens for ordinary non-magic variables like $foo.
But I suggest it is the least bad choice and would get rid of a lot of
odd-looking behaviour and bug reports. The current pass-by-value semantics
combined with magic variables like $! are a fountain of bizarreness.

(An alternative would be for a read-only reference to be passed, so the
subroutine dies if it tries to modify one of its arguments which is really
$1 or $! or $? or whatever.)

--
Ed Avis <eda@​waniasset.com>

From @Leont

On Tue, Sep 10, 2013 at 3​:04 PM, Ed Avis <eda@​waniasset.com> wrote​:

Is this not another of those bugs that would all be squashed if magic
variables like $!, $1 and so on were made pass-by-value instead of pass-
by-reference?

perl -le '$!=69; {local $! = $! } print 0+$!'

This bug has nothing to do with either ref or function calling. It
apparently has to do with (get) magic and localization.

Leon

From @epa

Sorry, please excuse me for banging that drum inappropriately.

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http​://www.symanteccloud.com
______________________________________________________________________

From @cpansprout

On Tue Sep 10 06​:14​:56 2013, LeonT wrote​:

On Tue, Sep 10, 2013 at 3​:04 PM, Ed Avis <eda@​waniasset.com> wrote​:

Is this not another of those bugs that would all be squashed if magic
variables like $!, $1 and so on were made pass-by-value instead of pass-
by-reference?

perl -le '$!=69; {local $! = $! } print 0+$!'

You will also find that it is 0 inside the sub (bug #16235). The two
seem to be interrelate somehow.

Bug #16235 is that local $! sets $! to undef and then the $! assigned to
it from the rhs has get-magic called, which sees the same value (undef)
and assigns that.

It seems the solution (at least for local $! = $!) may be for the old $!
to have its magical flags unset.

I haven’t looked closely to see whether that is what is causing the
original report (this bug, not #16235).

--

Father Chrysostomos

From @Leont

On Tue, Sep 10, 2013 at 5​:42 PM, Father Chrysostomos via RT <
perlbug-followup@​perl.org> wrote​:

You will also find that it is 0 inside the sub (bug #16235). The two
seem to be interrelate somehow.

Bug #16235 is that local $! sets $! to undef and then the $! assigned to
it from the rhs has get-magic called, which sees the same value (undef)
and assigns that.

It seems the solution (at least for local $! = $!) may be for the old $!
to have its magical flags unset.

I haven’t looked closely to see whether that is what is causing the
original report (this bug, not #16235).

We have a localize magic 'method' that we currently aren't using for
special variables. I suspect the solution will use that.

I still don't feel like I really understand the issue, but I'm guessing the
fact that its get magic also invokes set magic is involved.

Leon

From @ikegami

On Tue, Sep 10, 2013 at 9​:13 AM, Leon Timmermans <fawaka@​gmail.com> wrote​:

On Tue, Sep 10, 2013 at 3​:04 PM, Ed Avis <eda@​waniasset.com> wrote​:

Is this not another of those bugs that would all be squashed if magic
variables like $!, $1 and so on were made pass-by-value instead of pass-
by-reference?

perl -le '$!=69; {local $! = $! } print 0+$!'

This bug has nothing to do with either ref or function calling. It
apparently has to do with (get) magic and localization.

Leon

----- BEGIN CODE -----
use feature qw( say );

{
  package M;
  sub TIESCALAR { bless \my $dummy, shift }
  sub FETCH { say "Fetching <${$_[0]}>"; ${$_[0]} }
  sub STORE { say "Storing <$_[1]>"; ${$_[0]} = $_[1] }
}

tie $x, 'M';

$x=69; say '{'; { local $x = $x; say "<<$x>>"; } say '}'; say $x;
----- END CODE -----

----- BEGIN OUTPUT -----

perl a.pl
Storing <69>
{
Fetching <69>
Storing <>
Fetching <>
Storing <>
Fetching <>
<<>>
Storing <>
}
Fetching <>
----- END OUTPUT -----

It's as if

$x=69; { local $x = $x; } say $x;

does

$x=69; { my $r = \$x; local $x; $x = $$r; } say $x;

which is clearly the wrong order.

From @Leont

On Tue, Sep 10, 2013 at 6​:17 PM, Leon Timmermans <fawaka@​gmail.com> wrote​:

I still don't feel like I really understand the issue, but I'm guessing
the fact that its get magic also invokes set magic is involved.

Now I understand.

Given​: perl -le '$!=69; {local $! = $! } print 0+$!'

What happens is this​:

Enter scope
Save SV of old $!
localize $! →
  Sets errno to 0
  Get's value of errno from old $
  Sets the value of errno to the old $! [A]
  Assign value of errno to new $!
Leave scope
  Unlocalize $!
  Maps *!{SCALAR} back to old $!
  Calls set magic on the old $!

So yes, that explains why we're seeing what we're seeing. I think that
eliminating step [A] is the most obvious solution. It may require some
dirty tricks in $!'s get magic.

Leon

From @Leont

On Tue, Sep 10, 2013 at 7​:49 PM, Leon Timmermans <fawaka@​gmail.com> wrote​:

Enter scope
Save SV of old $!
localize $! →
Sets errno to 0
Get's value of errno from old $
Sets the value of errno to the old $! [A]

No, I'm wrong. «Get's value of errno from old $!» means "Assign the value
of errno to $!". This should happen after localization, but obviously can
happen.

Assign value of errno to new $!
Leave scope
Unlocalize $!
Maps *!{SCALAR} back to old $!
Calls set magic on the old $!

So yes, that explains why we're seeing what we're seeing. I think that
eliminating step [A] is the most obvious solution. It may require some
dirty tricks in $!'s get magic.

So no, that solution won't help. Anything that will call getmagic on the
old $! will screw it up after localization. I'm not sure how to solve that.

Leon

From @cpansprout

On Mon Sep 09 12​:28​:08 2013, zefram@​fysh.org wrote​:

This is a bug report for perl from zefram@​fysh.org,
generated with the help of perlbug 1.39 running under perl 5.18.1.

-----------------------------------------------------------------
[Please describe your issue here]

Ran into this while tweaking Carp's localisation of $!​:

$ perl -le 'sub aa { local $!; ref($_[0]); 1 } $!=69; aa($!); print
0+$!'
0

$! is getting squashed to 0 despite "local $!". This only happens
if ref() gets applied to the $! argument, in its guise as $_[0].
Doesn't happen with ref($!) or if the argument is something else.
If $! is set to a different value within the dynamic scope of "local
$!", that new value persists after the aa() call, regardless of
whether
the new value is assigned before or after the ref($_[0]). Also, if
"local $!" is removed then ref($_[0]) has no effect on $! (though $!
can
still be directly clobbered, of course). So evidentily the ref($_[0])
is actually breaking the localisation per se.

I think this is all stated elsewhere in the thread, but let me recap​:

Localisation swaps out a new $! and sets the old $! aside on the savestack, it invokes set-magic when localising (setting errno to 0) and when unwinding (setting it back to the value in the old $!).

The old $! is still magical, so invoking get-magic on it will read errno into it, clobbering the only place where the old errno value is stored.

I see multiple ways to fix this, each with its own drawbacks​:

1) Make the old scalar temporarily non-magical when any magical scalar is localised. This would fix ticket #16235 ‘for free’. This could break modules like File​::chdir that export a magical scalar to multiple callers.

2) Make the old scalar temporarily non-magical, but only if it is not tied. This would require using method #3 for tied variables; also, ticket #16235 would require a separate fix still, but only for ties. The result would be more complexity and some inconsistency between built-in variables like $! and tied variables.

3) Separately store the actual value to be restored on the savestack, in addition to the old magical scalar.

#3 is the easiest to implement, and probably the safest in terms of backward compatibility.

#2 might be the best fix overall though. It would make sub { local $!; print $_[0] }-&gt;($!) just DTRT without breaking ties. The inconsistency between ties and other magic may be acceptable.

--

Father Chrysostomos

From @nwc10

On Sat, Oct 26, 2013 at 01​:16​:54AM -0700, Father Chrysostomos via RT wrote​:

I think this is all stated elsewhere in the thread, but let me recap​:

Localisation swaps out a new $! and sets the old $! aside on the savestack, it invokes set-magic when localising (setting errno to 0) and when unwinding (setting it back to the value in the old $!).

The old $! is still magical, so invoking get-magic on it will read errno into it, clobbering the only place where the old errno value is stored.

I see multiple ways to fix this, each with its own drawbacks​:

1) Make the old scalar temporarily non-magical when any magical scalar is localised. This would fix ticket #16235 ‘for free’. This could break modules like File​::chdir that export a magical scalar to multiple callers.

2) Make the old scalar temporarily non-magical, but only if it is not tied. This would require using method #3 for tied variables; also, ticket #16235 would require a separate fix still, but only for ties. The result would be more complexity and some inconsistency between built-in variables like $! and tied variables.

3) Separately store the actual value to be restored on the savestack, in addition to the old magical scalar.

#3 is the easiest to implement, and probably the safest in terms of backward compatibility.

Wouldn't that change the behaviour of code which is able to access the
localised variable by another route? eg this code would end up printing
5, 6, 5 not what we have currently​:

$ cat /tmp/local.pl
#!/usr/bin/perl -w
use strict;

$​::var = 5;

print "Before​: $​::var\n";
foo (\$​::var);
print "After​: $​::var\n";

sub foo {
  local $​::var;
  $​::var = 6;
  ${$_[0]} = 7;
  print "During​: $​::var\n";
}
__END__
$ perl /tmp/local.pl
Before​: 5
During​: 6
After​: 7

(I haven't thought very hard about this, but the whole thing does seem to be
a tricky problem. I'm not sure if the underlying mess is that for magic
variables that proxy through to some other storage, Perl's local isn't
localising that underlying storage)

Nicholas Clark

From @cpansprout

On Sun Oct 27 00​:22​:32 2013, nicholas wrote​:

On Sat, Oct 26, 2013 at 01​:16​:54AM -0700, Father Chrysostomos via RT
wrote​:

3) Separately store the actual value to be restored on the savestack,
in addition to the old magical scalar.

#3 is the easiest to implement, and probably the safest in terms of
backward compatibility.

Wouldn't that change the behaviour of code which is able to access the
localised variable by another route? eg this code would end up
printing
5, 6, 5 not what we have currently​:

$ cat /tmp/local.pl
#!/usr/bin/perl -w
use strict;

$​::var = 5;

print "Before​: $​::var\n";
foo (\$​::var);
print "After​: $​::var\n";

sub foo {
local $​::var;
$​::var = 6;
${$_[0]} = 7;
print "During​: $​::var\n";
}
__END__
$ perl /tmp/local.pl
Before​: 5
During​: 6
After​: 7

Yes, it would change that behaviour. Scratch #3 then.

You see why nobody wants to fix this? :-)

(I haven't thought very hard about this, but the whole thing does seem
to be
a tricky problem. I'm not sure if the underlying mess is that for
magic
variables that proxy through to some other storage, Perl's local isn't
localising that underlying storage)

I think that is the underlying problem. But how do you fix that for ties? I.e., how do we make your example work with Tie​::StdScalar? Do we need to introduce LOCAL and UNWIND methods for ties?

Currently with $​::var tied to Tie​::StdScalar, it prints​:

Before​: 5
During​: 7
After​: 7

And with $! it prints the same, if you print 0+$!.

--

Father Chrysostomos

From @atoomic

Created by @atoomic

We recently discovered this issue while updating Test​::Trap to 0.3.3
which by trying to localize $!, in fact corrupted it using a simple

  local ($!, $^E) = ($!, $^E);

view https://rt.cpan.org/Public/Bug/Display.html?id=127112 for more details

So I performed some basic tests using multiple perl versions from 5.10 to
5.28
and looks like they all behave in the same way (which is good).

But I think it's weird that $! cannot be localized performing a copy

perl -e'$! = 4; do { local $! = $!; 1; }; print 0 + $!'
0

Another variable would work fine and preserve its original value
perl -e'our $x= 4; do { local $x = $x; }; print 0 + $x'
4

of course a workaround could be
perl -e'$! = 4; do { local $! = $! . q//; 1; }; print 0 + $!'
4

or a less convoluted by doing a manual copy

perl -e'$! = 4; do { my $copy = $!; local $! = $copy; 1; }; print 0 + $!'
4

At this point I'm not sure we can call it a bug... (maybe it's a feature)
but I still want to know your opinion on this
and you think it is worth fixing.

Flags:
    category=core
    severity=low

Site configuration information for perl 5.26.1:

Configured by nicolas at Wed Nov 29 10:26:27 MST 2017.

Summary of my perl5 (revision 5 version 26 subversion 1) configuration:

  Platform:
    osname=darwin
    osvers=15.6.0
    archname=darwin-2level
    uname='darwin nicolas-r.local 15.6.0 darwin kernel version 15.6.0: mon
oct 2 22:20:08 pdt 2017; root:xnu-3248.71.4~1release_x86_64 x86_64 '
    config_args='-de -Dprefix=/usr/local/perl/perls/perl-5.26.1
-Aeval:scriptdir=/usr/local/perl/perls/perl-5.26.1/bin'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include'
    optimize='-O3'
    cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.0.0/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
/usr/lib
    libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=
    so=dylib
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib
-fstack-protector-strong'



@INC for perl 5.26.1:
    /Users/nicolas/.dotfiles/perl-must-have/lib
    /Users/nicolas/perl5/lib/perl5/
    /usr/local/perl/perls/perl-5.26.1/lib/site_perl/5.26.1/darwin-2level
    /usr/local/perl/perls/perl-5.26.1/lib/site_perl/5.26.1
    /usr/local/perl/perls/perl-5.26.1/lib/5.26.1/darwin-2level
    /usr/local/perl/perls/perl-5.26.1/lib/5.26.1


Environment for perl 5.26.1:
    DYLD_LIBRARY_PATH (unset)
    HOME=/Users/nicolas
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LC_CTYPE=en_US.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=/usr/local/perl/bin:/usr/local/perl/perls/perl-5.26.1/bin:/usr/local/opt/ccache/libexec:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/git/bin:/usr/local/MacGPG2/bin:/Users/nicolas/.dotfiles/bin:/Users/nicolas/perl5/bin
    PERL5DB=use Devel::NYTProf

PERL5LIB=/Users/nicolas/.dotfiles/perl-must-have/lib:/Users/nicolas/perl5/lib/perl5/
    PERLBREW_BASHRC_VERSION=0.80
    PERLBREW_HOME=/Users/nicolas/.perlbrew
    PERLBREW_MANPATH=/usr/local/perl/perls/perl-5.26.1/man
    PERLBREW_PATH=/usr/local/perl/bin:/usr/local/perl/perls/perl-5.26.1/bin
    PERLBREW_PERL=perl-5.26.1
    PERLBREW_ROOT=/usr/local/perl
    PERLBREW_VERSION=0.80
    PERL_BADLANG (unset)
    PERL_CPANM_OPT=--quiet
    SHELL=/usr/local/bin/zsh

From @Leont

On Fri, Sep 14, 2018 at 7​:21 PM Atoomic (via RT)
<perlbug-followup@​perl.org> wrote​:

We recently discovered this issue while updating Test​::Trap to 0.3.3
which by trying to localize $!, in fact corrupted it using a simple

local \($\!\, $^E\) = \($\!\, $^E\);

view https://rt.cpan.org/Public/Bug/Display.html?id=127112 for more details

So I performed some basic tests using multiple perl versions from 5.10 to
5.28
and looks like they all behave in the same way (which is good).

But I think it's weird that $! cannot be localized performing a copy

perl -e'$! = 4; do { local $! = $!; 1; }; print 0 + $!'
0

Another variable would work fine and preserve its original value
perl -e'our $x= 4; do { local $x = $x; }; print 0 + $x'
4

of course a workaround could be
perl -e'$! = 4; do { local $! = $! . q//; 1; }; print 0 + $!'
4

or a less convoluted by doing a manual copy

perl -e'$! = 4; do { my $copy = $!; local $! = $copy; 1; }; print 0 + $!'
4

At this point I'm not sure we can call it a bug... (maybe it's a feature)
but I still want to know your opinion on this
and you think it is worth fixing.

This is a duplicate of #119683

Leon

The RT System itself - Status changed from 'new' to 'open'

From @atoomic

Thanks Leon for pointing to https://rt-archive.perl.org/perl5/Ticket/Display.html?id=119683
indeed this is the exact same issue discussed & explained
so looks like we are not really to fix it soon.

The only solution for this that I have thought up would be to:

1. Store the current value somewhere when the get magic is called with PL_localize=1
2. Restore that value in the set magic when PL_localize=2

I should add that this problem exists for most magical variables than can be written to, not just $! but also $^A, $^C, $^D, $^E, $^F, $^H, $^I, $^N, $^O, $^P, $^UTF8CACHE, $^W, ^WARNING_BITS, $^WIN32_SLOPPY_STAT, $., $^, $~, $=, $-, $%, $|, $/, $\, $?, $<, $>, $(, $), $:, $$, $0.

In one (very contrived) example that could even lead to security issues:

> sudo perl -E '$> = 1000; { local $> = $> } say $>'
0

Interestingly, this was previously fixed for magical hashes and arrays

I think this is a duplicate of #5835.