$& magic can get fetched too late #11406

p5pRT · 2011-06-03T06:14:39Z

Migrated from rt.perl.org#92164 (status was 'rejected')

Searchable as RT92164$

p5pRT · 2011-06-03T06:14:40Z

From @ntyni

This is a bug report for perl from Niko Tyni <ntyni@debian.org>,
generated with the help of perlbug 1.39 running under perl 5.15.0.

In some circumstances $& magic gets fetched too late so that
another match has already overwritten the value.

./perl -e '$_="not ok"; for my $m ( (/ok/ and $&), /$_/) { print $m, "\n"; last }'
not ok

Evaluating the variable right away (with stringification for instance)
makes the bug go away.

./perl -e '$_="not ok"; for my $m ( (/ok/ and "$&"), /$_/) { print $m, "\n"; last }'
ok

I don't have a fix, but there's a TODO test in the attached patch.

Originally reported by Ian Jackson in http://bugs.debian.org/627613

Flags:
category=core
severity=low

Site configuration information for perl 5.15.0:

Configured by niko at Thu Jun 2 23:07:45 EEST 2011.

Summary of my perl5 (revision 5 version 15 subversion 0) configuration:
Commit id: ba05d9a
Platform:
osname=linux, osvers=2.6.39-1-amd64, archname=x86_64-linux
uname='linux madeleine 2.6.39-1-amd64 #1 smp tue may 24 14:34:19 utc 2011 x86_64 gnulinux '
config_args='-des -Dusedevel -DDEBUGGING=both -Doptimize=-g -O0'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-g -O0',
cppflags='-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.6.1 20110526 (prerelease)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /lib/../lib /usr/lib/../lib /lib /usr/lib /lib64 /usr/lib64
libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
libc=/lib/libc-2.13.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.13'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -g -O0 -L/usr/local/lib -fstack-protector'

Locally applied patches:

@INC for perl 5.15.0:
lib
/usr/local/lib/perl5/site_perl/5.15.0/x86_64-linux
/usr/local/lib/perl5/site_perl/5.15.0
/usr/local/lib/perl5/5.15.0/x86_64-linux
/usr/local/lib/perl5/5.15.0
.

Environment for perl 5.15.0:
HOME=/home/niko
LANG=en_US.UTF-8
LANGUAGE (unset)
LC_CTYPE=fi_FI.UTF-8
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/niko/bin:/home/niko/bin:/home/niko/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/sbin:/usr/sbin
PERL_BADLANG (unset)
SHELL=/bin/zsh

p5pRT · 2011-06-03T06:14:40Z

From @ntyni

0001-TODO-test-for-delayed-evaluation-of.patch

From 50aaee3ad389646fa1df64fb158450a116e26d79 Mon Sep 17 00:00:00 2001
From: Niko Tyni <ntyni@debian.org>
Date: Fri, 3 Jun 2011 09:02:30 +0300
Subject: [PATCH] TODO test for delayed evaluation of $&

In some circumstances $& magic gets fetched too late so that
another match has already overwritten the value.

 ./perl -e '$_="not ok"; for my $m ( (/ok/ and $&), /$_/) { print $m, "\n"; last }'
 not ok

Evaluating the variable right away (with stringification for instance)
makes the bug go away.

 ./perl -e '$_="not ok"; for my $m ( (/ok/ and "$&"), /$_/) { print $m, "\n"; last }'
 ok

Add a TODO test for this.

Originally reported by Ian Jackson in http://bugs.debian.org/627613 .
---
 t/re/pat.t |   19 ++++++++++++++++++-
 1 files changed, 18 insertions(+), 1 deletions(-)

diff --git a/t/re/pat.t b/t/re/pat.t
index 4ef9663..b778a9d 100644
--- a/t/re/pat.t
+++ b/t/re/pat.t
@@ -21,7 +21,7 @@ BEGIN {
     require './test.pl';
 }
 
-plan tests => 451;  # Update this when adding/deleting tests.
+plan tests => 454;  # Update this when adding/deleting tests.
 
 run_tests() unless caller;
 
@@ -1167,6 +1167,23 @@ sub run_tests {
         is($got,$want,'RT #84294: check that "ab" =~ /((\w+)(?{ push @got, $2 })){2}/ leaves @got in the correct state');
     }
 
+    {
+        local $_ = '12';
+
+        for my $m ( (/1/ ? 0+$& : '3'), /2/) {
+            is($m, 1, 'numified $& comes from the right match'); last;
+        }
+        for my $m ( (/1/ ? "$&" : '3'), /2/) {
+            is($m, 1, 'stringified $& comes from the right match'); last;
+        };
+      TODO: {
+      local $::TODO = '$& get magic handled too late?';
+        for my $m ( (/1/ ? $& : '3'), /2/) {
+            is($m, 1, 'plain $& comes from the right match'); last;
+        }
+      }
+    }
+
 } # End of sub run_tests
 
 1;
-- 
1.7.5.3

p5pRT · 2011-06-03T10:35:41Z

From @Leont

On Fri, Jun 3, 2011 at 8:14 AM, Niko Tyni <perlbug-followup@perl.org> wrote:

In some circumstances $& magic gets fetched too late so that
another match has already overwritten the value.

./perl -e '$_="not ok"; for my $m ( (/ok/ and $&), /$_/) { print $m, "\n"; last }'
not ok

Evaluating the variable right away (with stringification for instance)
makes the bug go away.

./perl -e '$_="not ok"; for my $m ( (/ok/ and "$&"), /$_/) { print $m, "\n"; last }'
ok

For doesn't read (or write) a variable, it aliases it; therefore
calling magic early would be wrong: get magic shouldn't be called
until a variable is actually read. "Fixing" this would create a lot
more problems than it solves I'm afaid.

Leon

p5pRT · 2011-06-03T10:35:41Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2011-06-03T10:51:20Z

From @ntyni

On Fri, Jun 03, 2011 at 12:35:00PM +0200, Leon Timmermans wrote:

On Fri, Jun 3, 2011 at 8:14 AM, Niko Tyni <perlbug-followup@perl.org> wrote:

In some circumstances $& magic gets fetched too late so that
another match has already overwritten the value.

./perl -e '$_="not ok"; for my $m ( (/ok/ and $&), /$_/) { print $m, "\n"; last }'
not ok

Evaluating the variable right away (with stringification for instance)
makes the bug go away.

./perl -e '$_="not ok"; for my $m ( (/ok/ and "$&"), /$_/) { print $m, "\n"; last }'
ok

For doesn't read (or write) a variable, it aliases it; therefore
calling magic early would be wrong: get magic shouldn't be called
until a variable is actually read. "Fixing" this would create a lot
more problems than it solves I'm afaid.

I may have refined the test case too much, sorry about that.

The original one didn't have a for loop:

my $thing = "1-word";
my $h = { Num => ($thing =~ m/^\d+/ ? $& : die),
Word => !!($thing =~ m/word/)
};
print $h->{Num};

This gives "word" instead of the expected "1".

Is hash population also a case where the values should not be actually read?
--
Niko Tyni ntyni@debian.org

p5pRT · 2011-06-03T16:33:43Z

From @ikegami

On Fri, Jun 3, 2011 at 2:14 AM, Niko Tyni <perlbug-followup@perl.org> wrote:

-----------------------------------------------------------------
In some circumstances $& magic gets fetched too late so that
another match has already overwritten the value.

./perl -e '$_="not ok"; for my $m ( (/ok/ and $&), /$_/) { print $m, "\n";
last }'
not ok

C<for> aliases, so C<$m> is an alias to C<$&>, so C<print $m> is the same as
C<print $&>, so you get the correct result.

p5pRT · 2011-06-03T16:35:35Z

@ikegami - Status changed from 'open' to 'rejected'

p5pRT · 2011-06-03T16:38:50Z

From @ikegami

The original one didn't have a for loop:

my $thing = "1-word";
my $h = { Num => ($thing =~ m/^\d+/ ? $& : die),
Word => !!($thing =~ m/word/)
};
print $h->{Num};

This gives "word" instead of the expected "1".

Assigning $& to something requires reading it and therefore a magic
fetch. The assignment happens after the the list (C<< Num => ... >>) is
built and therefore after C<< $thing =~ m/word/ >>is evaluated.

p5pRT · 2011-06-03T17:00:22Z

From ebhanssen@cpan.org

On Fri, Jun 3, 2011 at 6:38 PM, Eric Brine via RT <perlbug-followup@perl.org

wrote:

The original one didn't have a for loop:

my $thing = "1-word";
my $h = { Num => ($thing =~ m/^\d+/ ? $& : die),
Word => !!($thing =~ m/word/)
};
print $h->{Num};

This gives "word" instead of the expected "1".

Assigning $& to something requires reading it and therefore a magic
fetch. The assignment happens after the the list (C<< Num => ... >>) is
built and therefore after C<< $thing =~ m/word/ >>is evaluated.

Right. The list is built (by the comma and fat comma operators) before
the actual values are read and copied into the new hash. And just to show
this is unique neither to hashes nor to $&:

sidhekin@bluebird[18:40:43]$ perl -le '$arr = [ ++$i, ++$i ]; print
"@$arr"'
2 2
sidhekin@bluebird[18:40:44]$

As a simplification, you could say that the comma operators, as we well
know from lvalue context, preserve lvalue-ness. Yeah, both of them: C<< my
($x => $y => $z) >> is a valid lvalue. :-)

Eirik, oh, right, simplification? hang on ...

p5pRT · 2011-06-03T20:51:28Z

From @ikegami

Don't read a variable in the same expression in which it changes.

On Fri, Jun 3, 2011 at 12:38 PM, Eric Brine via RT <
perlbug-followup@perl.org> wrote:

The original one didn't have a for loop:

my $thing = "1-word";
my $h = { Num => ($thing =~ m/^\d+/ ? $& : die),
Word => !!($thing =~ m/word/)
};
print $h->{Num};

This gives "word" instead of the expected "1".

Assigning $& to something requires reading it and therefore a magic
fetch. The assignment happens after the the list (C<< Num => ... >>) is
built and therefore after C<< $thing =~ m/word/ >>is evaluated.

As a guideline, don't read a variable in the same expression in which it
changes.

p5pRT · 2011-06-04T10:39:39Z

From @iabyn

On Fri, Jun 03, 2011 at 09:38:51AM -0700, Eric Brine via RT wrote:

The original one didn't have a for loop:

my $thing = "1-word";
my $h = { Num => ($thing =~ m/^\d+/ ? $& : die),
Word => !!($thing =~ m/word/)
};
print $h->{Num};

This gives "word" instead of the expected "1".

Assigning $& to something requires reading it and therefore a magic
fetch. The assignment happens after the the list (C<< Num => ... >>) is
built and therefore after C<< $thing =~ m/word/ >>is evaluated.

Indeed. It's really not much different from

@a = (++$i, ++$i, ++$i, ++$i); print "@a\n";

which prints "4 4 4 4", not "1 2 3 4".

I think this is NOTABUG.

--
SCO - a train crash in slow motion

p5pRT closed this as completed Jun 3, 2011

p5pRT added Not a Bug Severity Low labels Oct 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

$& magic can get fetched too late #11406

$& magic can get fetched too late #11406

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

p5pRT commented Jun 4, 2011

$& magic can get fetched too late #11406

$& magic can get fetched too late #11406

Comments

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

From @ntyni

p5pRT commented Jun 3, 2011

From @ntyni

p5pRT commented Jun 3, 2011

From @Leont

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

From @ntyni

p5pRT commented Jun 3, 2011

From @ikegami

p5pRT commented Jun 3, 2011

p5pRT commented Jun 3, 2011

From @ikegami

p5pRT commented Jun 3, 2011

From ebhanssen@cpan.org

p5pRT commented Jun 3, 2011

From @ikegami

p5pRT commented Jun 4, 2011

From @iabyn