%+ not scoped properly #15638

p5pRT · 2016-10-01T00:51:27Z

Migrated from rt.perl.org#129771 (status was 'open')

Searchable as RT129771$

p5pRT · 2016-10-01T00:51:27Z

From @nkh

Created by @nkh

Hi,

based on perlre:

These special variables, like the %+ hash and the numbered match variables
($1 , $2 , $3 , etc.) are dynamically scoped until the end of the enclosing
block or until the next successful match, whichever comes first.

the following code parse a string which has the form {block1}* text*
{block2} where block may be empty.

I use named capture (b1, b2) to capture the blocks.

for each test case the following is display

test case

block1 text
block2 text
text

The problem can already be seen within the first iterations where $+{b1}
should be undefined but has kept the previous value

adding an extra block fixes the problem

for (qw(
?
{a}
a{}?
?a{}
?{a}
{a}?{b}
{a}{b}{c}
{a}{b}{c}{d}
{a}bc{d}
{a}bcd
{?}
{
}
{{}
{a}{{{{{{b}
{a}{{{{{{b
{a}{{{{{{
{a}}}}}}{b}
{a}}}}}}{b
{a}}}}}}b}
))
{
#{{{ # perl bug fix
print " $_\n" ;
my $args = $_ ;

$args =~ s/^ (?'b1' (?:\{[^\{\}]*\}) ) //x ;
print '>' . ($+{b1} // '') . "\n" ;

$args = reverse $args ;

$args =~ s/^ (?'b2' (?:\}[^\{\}]*\{) ) //x ;
print '>' . reverse($+{b2} // '') . "\n" ;

my $t = reverse $args;
print ">$t\n" ;

print "-----------------\n\n" ;
#}}}
}

Perl Info


Flags:
    category=core
    severity=high

Site configuration information for perl 5.22.1:

Configured by Debian Project at Sun Mar 13 11:54:18 UTC 2016.

Summary of my perl5 (revision 5 version 22 subversion 1) configuration:

  Platform:
    osname=linux, osvers=3.16.0, archname=x86_64-linux-gnu-thread-multi
    uname='linux localhost 3.16.0 #1 smp debian 3.16.0 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dcc=x86_64-linux-gnu-gcc
-Dcpp=x86_64-linux-gnu-cpp -Dld=x86_64-linux-gnu-gcc -Dccflags=-DDEBIAN
-Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat
-Werror=format-security -Dldflags= -Wl,-Bsymbolic-functions -Wl,-z,relro
-Dlddlflags=-shared -Wl,-Bsymbolic-functions -Wl,-z,relro
-Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.22
-Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.22 -Dvendorprefix=/usr
-Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.22 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.22.1
-Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.22.1
-Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3
-Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3
-Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager
-Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Uversiononly
-DDEBUGGING=-g -Doptimize=-O2 -dEs -Duseshrplib -Dlibperl=libperl.so.5.22.1'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='x86_64-linux-gnu-gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE
-DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv
-fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='5.3.1 20160311', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678,
doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16,
longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='x86_64-linux-gnu-gcc', ldflags =' -fstack-protector-strong
-L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=libc-2.21.so, so=so, useshrplib=true, libperl=libperl.so.5.22
    gnulibc_version='2.21'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib
-fstack-protector-strong'

Locally applied patches:
    DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS
default for modules installed from CPAN.
    DEBPKG:debian/db_file_ver - http://bugs.debian.org/340047 Remove overly
restrictive DB_File version check.
    DEBPKG:debian/doc_info - Replace generic man(1) instructions with
Debian-specific information.
    DEBPKG:debian/enc2xs_inc - http://bugs.debian.org/290336 Tweak enc2xs
to follow symlinks and ignore missing @INC directories.
    DEBPKG:debian/errno_ver - http://bugs.debian.org/343351 Remove Errno
version check due to upgrade problems with long-running processes.
    DEBPKG:debian/libperl_embed_doc - http://bugs.debian.org/186778 Note
that libperl-dev package is required for embedded linking
    DEBPKG:fixes/respect_umask - Respect umask during installation
    DEBPKG:debian/writable_site_dirs - Set umask approproately for site
install directories
    DEBPKG:debian/extutils_set_libperl_path - EU:MM: set location of
libperl.a under /usr/lib
    DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or
perllocal.pod for perl or vendor
    DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the
binary targets.
    DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packlist
files for core or vendor.
    DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as
per Debian policy.
    DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to
/etc/perl/Net as /usr may not be writable.
    DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian
    DEBPKG:debian/prune_libs - http://bugs.debian.org/128355 Prune the list
of libraries wanted to what we actually need.
    DEBPKG:fixes/net_smtp_docs - [rt.cpan.org #36038]
http://bugs.debian.org/100195 Document the Net::SMTP 'Port' option
    DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make perlivp skip
include directories in /usr/local
    DEBPKG:debian/deprecate-with-apt - http://bugs.debian.org/747628 Point
users to Debian packages of deprecated core modules
    DEBPKG:debian/squelch-locale-warnings - http://bugs.debian.org/508764
Squelch locale warnings in Debian package maintainer scripts
    DEBPKG:debian/skip-upstream-git-tests - Skip tests specific to the
upstream Git repository
    DEBPKG:debian/patchlevel - http://bugs.debian.org/567489 List packaged
patches for 5.22.1-9 in patchlevel.h
    DEBPKG:debian/skip-kfreebsd-crash - http://bugs.debian.org/628493 [perl
#96272] Skip a crashing test case in t/op/threads.t on GNU/kFreeBSD
    DEBPKG:fixes/document_makemaker_ccflags - http://bugs.debian.org/628522
[rt.cpan.org #68613] Document that CCFLAGS should include $Config{ccflags}
    DEBPKG:debian/find_html2text - http://bugs.debian.org/640479 Configure
CPAN::Distribution with correct name of html2text
    DEBPKG:debian/perl5db-x-terminal-emulator.patch -
http://bugs.debian.org/668490 Invoke x-terminal-emulator rather than xterm
in perl5db.pl
    DEBPKG:debian/cpan-missing-site-dirs - http://bugs.debian.org/688842
Fix CPAN::FirstTime defaults with nonexisting site dirs if a parent is
writable
    DEBPKG:fixes/memoize_storable_nstore - [rt.cpan.org #77790]
http://bugs.debian.org/587650 Memoize::Storable: respect 'nstore' option
not respected
    DEBPKG:debian/regen-skip - Skip a regeneration check in unrelated git
repositories
    DEBPKG:debian/makemaker-pasthru - http://bugs.debian.org/758471 Pass LD
settings through to subdirectories
    DEBPKG:fixes/pod_man_reproducible_date - http://bugs.debian.org/759405
Support POD_MAN_DATE in Pod::Man for the left-hand footer
    DEBPKG:debian/locale-robustness - http://bugs.debian.org/782068 [perl
#124310] Make t/run/locale.t survive missing locales masked by LC_ALL
    DEBPKG:fixes/podman-utc - http://bugs.debian.org/780259 Make the
embedded date from Pod::Man reproducible
    DEBPKG:fixes/podman-utc-docs - http://bugs.debian.org/780259
Documentation and test suite updates for UTC fix
    DEBPKG:fixes/podman-empty-date - http://bugs.debian.org/780259 Support
an empty POD_MAN_DATE environment variable
    DEBPKG:fixes/podman-pipe - http://bugs.debian.org/777405 Better errors
for man pages from standard input
    DEBPKG:debian/pod2man-customized - Update porting/customized.dat for
pod2man modifications
    DEBPKG:debian/makemaker-manext - http://bugs.debian.org/247370 Make
EU::MakeMaker honour MANnEXT settings in generated manpage headers
    DEBPKG:debian/makemaker_customized - Update t/porting/customized.dat
for files patched in Debian
    DEBPKG:debian/do-not-record-build-date - [6baa8db]
http://bugs.debian.org/774422 [perl #125830] Allow overriding the compile
time in "perl -V" output
    DEBPKG:fixes/podman-source-date-epoch - http://bugs.debian.org/801621
Make Pod::Man honor the SOURCE_DATE_EPOCH environment variable
    DEBPKG:fixes/podman-source-date-epoch-cleanups -
http://bugs.debian.org/801621 Coding style and documentation for
SOURCE_EPOCH_DATE
    DEBPKG:fixes/podman-source-date-epoch-testfix -
http://bugs.debian.org/807086 Guard for building with SOURCE_DATE_EPOCH or
POD_MAN_DATE set
    DEBPKG:debian/devel-ppport-reproducibility -
http://bugs.debian.org/801523 Sort the list of XS code files when
generating RealPPPort.xs
    DEBPKG:fixes/encode-unicode-bom - http://bugs.debian.org/798727 [
rt.cpan.org #107043] Address
https://rt.cpan.org/Public/Bug/Display.html?id=107043
    DEBPKG:debian/encode-unicode-bom-doc - http://bugs.debian.org/798727
Document Debian backport of Encode::Unicode fix
    DEBPKG:debian/kfreebsd-softupdates - http://bugs.debian.org/796798 Work
around Debian Bug#796798
    DEBPKG:fixes/autodie-scope - http://bugs.debian.org/798096 Fix a
scoping issue with "no autodie" and the "system" sub
    DEBPKG:debian/debugperl-compat-fix - [perl #127212]
http://bugs.debian.org/810326 Disable PERL_TRACK_MEMPOOL for debugging
builds
    DEBPKG:fixes/CVE-2015-8607_file_spec_taint_fix -
http://bugs.debian.org/810719 [perl #126862] ensure File::Spec::canonpath()
preserves taint
    DEBPKG:fixes/mkstemp-umask - http://bugs.debian.org/810924 [perl
#127322] [e57270b] Fix umask for mkstemp(3) calls
    DEBPKG:fixes/crosscompile-no-targethost - [perl #127234] Fix the
Configure escape with usecrosscompile but no targethost
    DEBPKG:fixes/podlators-no-encode - [rt.cpan.org #111156] Degrade
gracefully if utf8 is requested but Encode is not available
    DEBPKG:debian/cross-time-hires - [rt.cpan.org #111391] Add an
environment variable to skip running configuration probes
    DEBPKG:fixes/encode-unicode-pod - Unicode.pm: Fix POD error
    DEBPKG:fixes/memoize-pod - [rt.cpan.org #89441] Fix POD errors in
Memoize
    DEBPKG:fixes/ok-pod - Added encoding for pod.
    DEBPKG:fixes/CVE-2016-2381_duplicate_env - remove duplicate environment
variables from environ


@INC for perl 5.22.1:
    /home/nadim/perl5/lib/perl5/x86_64-linux-gnu-thread-multi
    /home/nadim/perl5/lib/perl5
    /etc/perl
    /usr/local/lib/x86_64-linux-gnu/perl/5.22.1
    /usr/local/share/perl/5.22.1
    /usr/lib/x86_64-linux-gnu/perl5/5.22
    /usr/share/perl5
    /usr/lib/x86_64-linux-gnu/perl/5.22
    /usr/share/perl/5.22
    /usr/local/lib/site_perl
    /usr/lib/x86_64-linux-gnu/perl-base
    .


Environment for perl 5.22.1:
    HOME=/home/nadim
    LANG=en_US.UTF-8
    LANGUAGE=
    LC_ADDRESS=fr_FR.UTF-8
    LC_IDENTIFICATION=fr_FR.UTF-8
    LC_MEASUREMENT=fr_FR.UTF-8
    LC_MONETARY=fr_FR.UTF-8
    LC_NAME=fr_FR.UTF-8
    LC_NUMERIC=fr_FR.UTF-8
    LC_PAPER=fr_FR.UTF-8
    LC_TELEPHONE=fr_FR.UTF-8
    LC_TIME=fr_FR.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=/home/nadim/bin:/home/nadim/.local/bin:/home/nadim/.qfc/bin:/home/nadim/nadim/devel/repositories/rakudo/install/share/perl6/site/bin/:/home/nadim/nadim/devel/repositories/rakudo/install/bin/:/home/nadim/perl5/bin:/home/nadim/nadim/bin/:/home/nadim/bin:/home/nadim/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/nadim/.go/bin:/home/nadim/nadim/devel/repositories/fzf/bin
    PERL5LIB=/home/nadim/perl5/lib/perl5:
    PERL_BADLANG (unset)
    PERL_LOCAL_LIB_ROOT=:/home/nadim/perl5
    PERL_MB_OPT=--install_base /home/nadim/perl5
    PERL_MM_OPT=INSTALL_BASE=/home/nadim/perl5
    SHELL=/bin/bash

p5pRT · 2016-10-03T10:09:00Z

From @iabyn

On Fri, Sep 30, 2016 at 05:51:27PM -0700, nadim khemir wrote:

based on perlre:

These special variables, like the %+ hash and the numbered match variables
($1 , $2 , $3 , etc.) are dynamically scoped until the end of the enclosing
block or until the next successful match, whichever comes first.

the following code parse a string which has the form {block1}* text*
{block2} where block may be empty.

I use named capture (b1, b2) to capture the blocks.

for each test case the following is display

test case

block1 text
block2 text
text

The problem can already be seen within the first iterations where $+{b1}
should be undefined but has kept the previous value

This is intended behaviour. From the point of view of the dynamic scope
of pattern match captures, iterating a loop isn't regarded as exiting a
scope and starting a new scope; rather it is seen as restarting the
current scope.

A similar effect can be seen with this simplified program:

for (qw(ab XY)) {
print "\nPRE: [$1][$+{foo}]\n";
/(.)(?'foo'.)/ or die;
print "POST: [$1][$+{foo}]\n";
}

which outputs:

PRE: [][]
POST: [a][b]

PRE: [a][b]
POST: [X][Y]

Adding an extra scope like

for (qw(ab XY)) {
{
print "\nPRE: [$1][$+{foo}]\n";
/(.)(?'foo'.)/ or die;
print "POST: [$1][$+{foo}]\n";
}
}

will cause the output to change to:

PRE: [][]
POST: [a][b]

PRE: [][]
POST: [X][Y]

--
Fire extinguisher (n) a device for holding open fire doors.

p5pRT · 2016-10-03T10:09:00Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2016-10-03T15:22:48Z

From @nkh

A hint in the documentation would be nice :)

maybe with this example

On Mon, Oct 3, 2016 at 12:09 PM, Dave Mitchell via RT <
perlbug-followup@perl.org> wrote:

On Fri, Sep 30, 2016 at 05:51:27PM -0700, nadim khemir wrote:

based on perlre:

These special variables, like the %+ hash and the numbered match
variables
($1 , $2 , $3 , etc.) are dynamically scoped until the end of the
enclosing
block or until the next successful match, whichever comes first.

the following code parse a string which has the form {block1}* text*
{block2} where block may be empty.

I use named capture (b1, b2) to capture the blocks.

for each test case the following is display

test case

block1 text
block2 text
text

The problem can already be seen within the first iterations where $+{b1}
should be undefined but has kept the previous value

This is intended behaviour. From the point of view of the dynamic scope
of pattern match captures, iterating a loop isn't regarded as exiting a
scope and starting a new scope; rather it is seen as restarting the
current scope.

A similar effect can be seen with this simplified program:
for $qw\(ab XY$\) \{
    print "\\nPRE&#8203;:  \[$1\]\[$\+\{foo\}\]\\n";
    /$\.$$?'foo'\.$/ or die;
    print   "POST&#8203;: \[$1\]\[$\+\{foo\}\]\\n";
\}
which outputs:
PRE&#8203;:  \[\]\[\]
POST&#8203;: \[a\]\[b\]

PRE&#8203;:  \[a\]\[b\]
POST&#8203;: \[X\]\[Y\]
Adding an extra scope like
for $qw\(ab XY$\) \{
    \{
        print "\\nPRE&#8203;:  \[$1\]\[$\+\{foo\}\]\\n";
        /$\.$$?'foo'\.$/ or die;
        print   "POST&#8203;: \[$1\]\[$\+\{foo\}\]\\n";
    \}
\}
will cause the output to change to:
PRE&#8203;:  \[\]\[\]
POST&#8203;: \[a\]\[b\]

PRE&#8203;:  \[\]\[\]
POST&#8203;: \[X\]\[Y\]
--
Fire extinguisher (n) a device for holding open fire doors.

p5pRT added Severity High distro-Linux type-core labels Oct 19, 2019

xenu removed affects-5.22 labels Nov 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

%+ not scoped properly #15638

%+ not scoped properly #15638

p5pRT commented Oct 1, 2016

p5pRT commented Oct 1, 2016

p5pRT commented Oct 3, 2016

p5pRT commented Oct 3, 2016

p5pRT commented Oct 3, 2016

Navigation Menu

%+ not scoped properly #15638

%+ not scoped properly #15638

Comments

p5pRT commented Oct 1, 2016

p5pRT commented Oct 1, 2016

From @nkh

Created by @nkh

p5pRT commented Oct 3, 2016

From @iabyn

p5pRT commented Oct 3, 2016

p5pRT commented Oct 3, 2016

From @nkh