Skip Menu |
Report information
Id: 121484
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: zefram [at] fysh.org
Cc:
AdminCc:

Operating System: Linux
PatchStatus: (no value)
Severity: High
Type:
  • core
  • regex
Perl Version: 5.19.10
Fixed In: 5.19.11



From: zefram [...] fysh.org
To: perlbug [...] perl.org
Date: Fri, 21 Mar 2014 13:00:42 +0000
Subject: /m causing false negative
CC: zefram [...] fysh.org
Download (untitled) / with headers
text/plain 3.6k
This is a bug report for perl from zefram@fysh.org, generated with the help of perlbug 1.40 running under perl 5.19.10. ----------------------------------------------------------------- [Please describe your issue here] $ perl5.19.10 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' no $ perl5.19.9 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' yes Obviously a regression. Regexp matches correctly if /m is omitted, or if either anchor is omitted, or if any of the letters "ble" in the pattern are replaced with dots. But replacing "wib" with dots keeps it falsely non-matching. This breaks the pod-coverage test for TheSchwartz, which builds up a bunch of patterns like this to describe exceptions to the coverage requirement. [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=high --- Site configuration information for perl 5.19.10: Configured by root at Fri Mar 21 09:44:41 UTC 2014. Summary of my perl5 (revision 5 version 19 subversion 10) configuration: Platform: osname=linux, osvers=2.6.32-5-686, archname=i686-linux-64int-ld uname='linux beryllium.photobox.priv 2.6.32-5-686 #1 smp fri may 10 08:33:48 utc 2013 i686 gnulinux ' config_args='-des -Dusedevel -Uversiononly -Duseshrplib -Duse64bitint -Duselongdouble -Uusethreads -Uusemultiplicity -Dprefix=/opt/perl-5.19.10 -Dsiteprefix=/opt/perl-5.19.10 -Dvendorprefix=/opt/perl-5.19.10/vendor -Doptimize=-ggdb -O2 -Dcccdlflags=-fPIC -O2 -pipe' hint=recommended, useposix=true, d_sigaction=define useithreads=undef, usemultiplicity=undef use64bitint=define, use64bitall=undef, uselongdouble=define usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-ggdb -O2', cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.7.2', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long long', ivsize=8, nvtype='long double', nvsize=12, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/i486-linux-gnu/4.7/include-fixed /usr/include/i386-linux-gnu /usr/lib /lib/i386-linux-gnu /lib/../lib /usr/lib/i386-linux-gnu /usr/lib/../lib /lib libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.13.so, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.13' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/opt/perl-5.19.10/lib/5.19.10/i686-linux-64int-ld/CORE' cccdlflags='-fPIC -O2 -pipe', lddlflags='-shared -ggdb -O2 -L/usr/local/lib -fstack-protector' --- @INC for perl 5.19.10: /opt/perl-5.19.10/lib/site_perl/5.19.10/i686-linux-64int-ld /opt/perl-5.19.10/lib/site_perl/5.19.10 /opt/perl-5.19.10/vendor/lib/vendor_perl/5.19.10/i686-linux-64int-ld /opt/perl-5.19.10/vendor/lib/vendor_perl/5.19.10 /opt/perl-5.19.10/lib/5.19.10/i686-linux-64int-ld /opt/perl-5.19.10/lib/5.19.10 . --- Environment for perl 5.19.10: HOME=/root LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/root/pub/i686-pc-linux-gnu/bin:/root/pub/common/bin:/usr/bin:/usr/sbin:/bin:/sbin:/usr/local/bin:/usr/local/sbin:/usr/games:/opt/geoip/bin:/opt/httpd/bin:/opt/perl/bin PERL_BADLANG (unset) SHELL=/usr/bin/zsh
From: Tom Hukins <tom [...] eborcom.com>
To: perl5-porters [...] perl.org
Date: Fri, 21 Mar 2014 14:16:21 +0000
Subject: Re: [perl #121484] /m causing false negative
Download (untitled) / with headers
text/plain 1.3k
On Fri, Mar 21, 2014 at 06:01:01AM -0700, Zefram wrote: Show quoted text
> $ perl5.19.10 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' > no > $ perl5.19.9 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' > yes
I've just bisected this to the following plausible-looking commit: d0d4464849e2b30aee89c175ccb5465795de10ce is the first bad commit commit d0d4464849e2b30aee89c175ccb5465795de10ce Author: David Mitchell <davem@iabyn.com> Date: Tue Mar 18 20:15:27 2014 +0000 re_intuit_start(): reduce scope of /^...$/m test Intuit has a quick reject test for a fixed pattern that is anchored at both ends. For example, with the pattern /^abcd$/, only the exact strings "abcd" or "abcd\n" will match; anything else, and the match immediately fails. A fix for [perl #115242] correctly made intuit skip the test in the presence of //m, since in this case the $ doesn't necessarily correspond to the end of the string. However, the fix was too wide in scope; it caused //m patterns to skip searching for a known string anchored just at the start, as well as one anchored at both ends. With this commit, the following code now runs in a few milliseconds rather than a few seconds on my machine: $s = "abcdefg" x 1_000_000; $s =~ /(?-m:^)abcX?fg/m for 1..100; Tom
Date: Fri, 21 Mar 2014 14:33:34 +0000
To: Perl5 Porters <perl5-porters [...] perl.org>
Subject: Re: [perl #121484] /m causing false negative
CC: bugs-bitbucket [...] rt.perl.org
From: Aaron Crane <arc [...] cpan.org>
Download (untitled) / with headers
text/plain 2.4k
Zefram <perlbug-followup@perl.org> wrote: Show quoted text
> $ perl5.19.10 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' > no > $ perl5.19.9 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' > yes
This bisects to the following commit: commit d0d4464849e2b30aee89c175ccb5465795de10ce Author: David Mitchell <davem@iabyn.com> Date: Tue Mar 18 20:15:27 2014 +0000 re_intuit_start(): reduce scope of /^...$/m test Intuit has a quick reject test for a fixed pattern that is anchored at both ends. For example, with the pattern /^abcd$/, only the exact strings "abcd" or "abcd\n" will match; anything else, and the match immediately fails. A fix for [perl #115242] correctly made intuit skip the test in the presence of //m, since in this case the $ doesn't necessarily correspond to the end of the string. However, the fix was too wide in scope; it caused //m patterns to skip searching for a known string anchored just at the start, as well as one anchored at both ends. With this commit, the following code now runs in a few milliseconds rather than a few seconds on my machine: $s = "abcdefg" x 1_000_000; $s =~ /(?-m:^)abcX?fg/m for 1..100; AFAICT from a fairly cursory look, re_intuit_start() seems to be failing to account for the "\n" at the end of the fixed check substring. This is Zefram's test case expressed in the language of t/re/re_tests: diff --git i/t/re/re_tests w/t/re/re_tests index 7ab7dc3..294032f 100644 --- i/t/re/re_tests +++ w/t/re/re_tests @@ -1884,5 +1884,21 @@ A+(*PRUNE)BC(?{}) AAABC y $& AAABC [bcd].{2,3}aaaa XbXaaaaa y - - [bcd].{2,3}aaaa Xb\x{100}aaaaa y - - +# RT #121484 - /m causing false negative +'\Awibble\z'm wibble y $& wibble +'\A.ibble\z'm wibble y $& wibble +'\Aw.bble\z'm wibble y $& wibble +'\Awi.ble\z'm wibble y $& wibble +'\Awib.le\z'm wibble y $& wibble +'\Awibb.e\z'm wibble y $& wibble +'\Awibbl.\z'm wibble y $& wibble +'\Awibble$'m wibble y $& wibble +'\A.ibble$'m wibble y $& wibble +'\Aw.bble$'m wibble y $& wibble +'\Awi.ble$'m wibble y $& wibble +'\Awib.le$'m wibble y $& wibble +'\Awibb.e$'m wibble y $& wibble +'\Awibbl.$'m wibble y $& wibble + # Keep these lines at the end of the file # vim: softtabstop=0 noexpandtab -- Aaron Crane ** http://aaroncrane.co.uk/
From: demerphq <demerphq [...] gmail.com>
Date: Fri, 21 Mar 2014 17:03:23 +0100
To: Perl5 Porteros <perl5-porters [...] perl.org>
CC: "bugs-bitbucket [...] rt.perl.org" <bugs-bitbucket [...] rt.perl.org>
Subject: Re: [perl #121484] /m causing false negative
On 21 March 2014 14:01, Zefram <perlbug-followup@perl.org> wrote: Show quoted text
> # New Ticket Created by Zefram > # Please include the string: [perl #121484] > # in the subject line of all future correspondence about this issue. > # <URL: https://rt.perl.org/Ticket/Display.html?id=121484 > > > > > This is a bug report for perl from zefram@fysh.org, > generated with the help of perlbug 1.40 running under perl 5.19.10. > > > ----------------------------------------------------------------- > [Please describe your issue here] > > $ perl5.19.10 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' > no > $ perl5.19.9 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' > yes > > Obviously a regression. Regexp matches correctly if /m is omitted, > or if either anchor is omitted, or if any of the letters "ble" in the > pattern are replaced with dots. But replacing "wib" with dots keeps it > falsely non-matching.
Why is this a regression? I call this a bug fix. It *should* have matched with /m. There are no regexp structures affected by /m in this pattern. Yves
From: demerphq <demerphq [...] gmail.com>
Subject: Re: [perl #121484] /m causing false negative
Date: Fri, 21 Mar 2014 17:05:03 +0100
To: Perl5 Porteros <perl5-porters [...] perl.org>
Download (untitled) / with headers
text/plain 1.2k
On 21 March 2014 17:03, demerphq <demerphq@gmail.com> wrote: Show quoted text
> On 21 March 2014 14:01, Zefram <perlbug-followup@perl.org> wrote:
>> # New Ticket Created by Zefram >> # Please include the string: [perl #121484] >> # in the subject line of all future correspondence about this issue. >> # <URL: https://rt.perl.org/Ticket/Display.html?id=121484 > >> >> >> >> This is a bug report for perl from zefram@fysh.org, >> generated with the help of perlbug 1.40 running under perl 5.19.10. >> >> >> ----------------------------------------------------------------- >> [Please describe your issue here] >> >> $ perl5.19.10 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' >> no >> $ perl5.19.9 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' >> yes >> >> Obviously a regression. Regexp matches correctly if /m is omitted, >> or if either anchor is omitted, or if any of the letters "ble" in the >> pattern are replaced with dots. But replacing "wib" with dots keeps it >> falsely non-matching.
> > Why is this a regression? I call this a bug fix. It *should* have > matched with /m. There are no regexp structures affected by /m in this > pattern.
I cant read. Forget this. Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Subject: Re: [perl #121484] /m causing false negative
CC: Perl5 Porters <perl5-porters [...] perl.org>, bugs-bitbucket [...] rt.perl.org
To: Aaron Crane <arc [...] cpan.org>
Date: Fri, 21 Mar 2014 20:44:30 +0100
Download (untitled) / with headers
text/plain 303b
Aaron Crane <arc@cpan.org> writes: Show quoted text
> This bisects to the following commit: > > commit d0d4464849e2b30aee89c175ccb5465795de10ce
Affected on the CPAN: FANGLY/Getopt-Euclid-0.4.4.tar.gz JKEGL/Marpa-R2-2.082000.tar.gz THALJEF/Perl-Critic-1.121.tar.gz DCONWAY/Smart-Comments-1.000005.tar.gz -- andreas
From: Dave Mitchell <davem [...] iabyn.com>
CC: perl5-porters [...] perl.org
To: Tom Hukins <tom [...] eborcom.com>
Date: Mon, 24 Mar 2014 15:43:39 +0000
Subject: Re: [perl #121484] /m causing false negative
Download (untitled) / with headers
text/plain 1.1k
On Fri, Mar 21, 2014 at 02:16:21PM +0000, Tom Hukins wrote: Show quoted text
> On Fri, Mar 21, 2014 at 06:01:01AM -0700, Zefram wrote:
> > $ perl5.19.10 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' > > no > > $ perl5.19.9 -lwe 'print "wibble" =~ /\Awibble\z/m ? "yes" : "no"' > > yes
> > I've just bisected this to the following plausible-looking commit:
Thanks. Now fixed with commit 7742aa6637b99424497eb15161fdb0a225288578 Author: David Mitchell <davem@iabyn.com> AuthorDate: Mon Mar 24 15:36:32 2014 +0000 Commit: David Mitchell <davem@iabyn.com> CommitDate: Mon Mar 24 15:36:32 2014 +0000 [perl #121484] /m causing false negative My recent commit d0d4464849e2b30aee8 in re_intuit_start() reduced the scope of a 'skip if multiline' check, so that certain optimisations weren't being unnecessarily skipped. Unfortunately it didn't reduce the scope enough, so a vital slen-- was being skipped in the SvTAIL-but-don't-fail case. This commit just moves the !multiline test further down, and updates the commentary and condition formatting a bit. -- The optimist believes that he lives in the best of all possible worlds. As does the pessimist.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org