Skip Menu |
Report information
Id: 39185
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: johnpc [at] xs4all.nl
Cc:
AdminCc:

Operating System: Linux
PatchStatus: (no value)
Severity: medium
Type: core
Perl Version: 5.8.8
Fixed In: (no value)



CC: perl [...] xs4all.nl
Subject: regex optimizer bug in qr//flags
Date: Mon, 22 May 2006 23:25:42 +0200
To: perlbug [...] perl.org
From: Jan-Pieter Cornet <johnpc [...] pcoder.xs4all.nl>
Download (untitled) / with headers
text/plain 4.5k
This is a bug report for perl from johnpc@xs4all.nl, generated with the help of perlbug 1.35 running under perl v5.8.8. ----------------------------------------------------------------- [Please enter your report here] When using flags on the qr// regex-creation operator, some regexes are badly optimized and thus fail to match. This example illustrates the problem (problem originally presented by TTY as "buzz8" during the Dutch Perl Workshop). #!/usr/bin/env perl use strict; use warnings; my $test = <<'END'; { bla } { bla bla } END my $re = qr/^\{(.*?)\}$/m; $test =~ s/$re/<$1>/g; ### match rejected by optimizer ### workarounds are: # $re = qr/(?m:^\{(.*?)\}$)/; ### OK # however: $re = qr/(?m:^\{(.*?)\}$)/m; ### FAILS too??? # chomp $test; ### OK (now $test =~ /}\n\z/) # $test .= "}" ### OK (now $test =~ /}\z/) # $test =~ s/^\{(.*?)\}$/<$1>/gm; ### OK # $test =~ s/$re/<$1>/gm; ### OK print $test; ### expect: "< bla >\n< bla bla >\n\n" __END__ The problem can be witnessed if this is run with -Mre=debug. The regex is rejected by the optimizer, but any of the suggested changes (marked "### OK") give the expected result, and make the regex match. [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=medium --- Site configuration information for perl v5.8.8: Configured by Debian Project at Tue Apr 4 22:34:25 UTC 2006. Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=linux, osvers=2.6.15.4, archname=i486-linux-gnu-thread-multi uname='linux ninsei 2.6.15.4 #1 smp preempt mon feb 20 09:48:53 pst 2006 i686 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.8 -Dsitearch=/usr/local/lib/perl/5.8.8 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.8 -Dd_dosuid -des' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include' ccversion='', gccversion='4.0.3 (Debian 4.0.3-1)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=/lib/libc-2.3.6.so, so=so, useshrplib=true, libperl=libperl.so.5.8.8 gnulibc_version='2.3.6' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib' Locally applied patches: --- @INC for perl v5.8.8: /etc/perl /usr/local/lib/perl/5.8.8 /usr/local/share/perl/5.8.8 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8 /usr/local/lib/site_perl /usr/local/lib/perl/5.8.4 /usr/local/share/perl/5.8.4 /usr/local/lib/perl/5.8.3 /usr/local/share/perl/5.8.3 /usr/local/lib/perl/5.8.0 /usr/local/share/perl/5.8.0 . --- Environment for perl v5.8.8: HOME=/home/johnpc LANG (unset) LANGUAGE (unset) LC_CTYPE=en_US.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/johnpc/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin:/usr/games:/usr/X11R6/bin PERL_BADLANG (unset) SHELL=/bin/bash
Subject: Re: [perl #39185] regex optimizer bug in qr//flags
Date: Tue, 23 May 2006 00:07:59 +0100
To: perl5-porters [...] perl.org
From: Dave Mitchell <davem [...] iabyn.com>
Download (untitled) / with headers
text/plain 1.5k
On Mon, May 22, 2006 at 02:26:40PM -0700, johnpc @ xs4all. nl wrote: Show quoted text
> When using flags on the qr// regex-creation operator, some regexes > are badly optimized and thus fail to match. This example illustrates > the problem (problem originally presented by TTY as "buzz8" during the > Dutch Perl Workshop). > > #!/usr/bin/env perl > use strict; > use warnings; > > my $test = <<'END'; > { bla } > { bla bla } > > END > > my $re = qr/^\{(.*?)\}$/m; > $test =~ s/$re/<$1>/g; ### match rejected by optimizer > ### workarounds are: > # $re = qr/(?m:^\{(.*?)\}$)/; ### OK > # however: $re = qr/(?m:^\{(.*?)\}$)/m; ### FAILS too??? > # chomp $test; ### OK (now $test =~ /}\n\z/) > # $test .= "}" ### OK (now $test =~ /}\z/) > # $test =~ s/^\{(.*?)\}$/<$1>/gm; ### OK > # $test =~ s/$re/<$1>/gm; ### OK > > print $test; ### expect: "< bla >\n< bla bla >\n\n" > __END__ > > The problem can be witnessed if this is run with -Mre=debug. The > regex is rejected by the optimizer, but any of the suggested changes > (marked "### OK") give the expected result, and make the regex match.
This appears to have been fixed between perl 5.9.1 and 5.9.2, but the fix presumably hasn't been backported to 5.8.x. -- This email is confidential, and now that you have read it you are legally obliged to shoot yourself. Or shoot a lawyer, if you prefer. If you have received this email in error, place it in its original wrapping and return for a full refund. By opening this email, you accept that Elvis lives.
CC: perl5-porters [...] perl.org
Subject: Re: [perl #39185] regex optimizer bug in qr//flags
Date: Mon, 22 May 2006 20:05:43 -0400
To: Dave Mitchell <davem [...] iabyn.com>
From: Rick Delaney <rick [...] bort.ca>
Download (untitled) / with headers
text/plain 1.5k
On Tue, May 23, 2006 at 12:07:59AM +0100, Dave Mitchell wrote: Show quoted text
> On Mon, May 22, 2006 at 02:26:40PM -0700, johnpc @ xs4all. nl wrote:
> > When using flags on the qr// regex-creation operator, some regexes > > are badly optimized and thus fail to match. This example illustrates > > the problem (problem originally presented by TTY as "buzz8" during the > > Dutch Perl Workshop). > > > > #!/usr/bin/env perl > > use strict; > > use warnings; > > > > my $test = <<'END'; > > { bla } > > { bla bla } > > > > END > > > > my $re = qr/^\{(.*?)\}$/m; > > $test =~ s/$re/<$1>/g; ### match rejected by optimizer > > ### workarounds are: > > # $re = qr/(?m:^\{(.*?)\}$)/; ### OK > > # however: $re = qr/(?m:^\{(.*?)\}$)/m; ### FAILS too??? > > # chomp $test; ### OK (now $test =~ /}\n\z/) > > # $test .= "}" ### OK (now $test =~ /}\z/) > > # $test =~ s/^\{(.*?)\}$/<$1>/gm; ### OK > > # $test =~ s/$re/<$1>/gm; ### OK > > > > print $test; ### expect: "< bla >\n< bla bla >\n\n" > > __END__ > > > > The problem can be witnessed if this is run with -Mre=debug. The > > regex is rejected by the optimizer, but any of the suggested changes > > (marked "### OK") give the expected result, and make the regex match.
> > This appears to have been fixed between perl 5.9.1 and 5.9.2, but the fix > presumably hasn't been backported to 5.8.x.
I believe it has been backported; it just didn't make it into 5.8.8. At least, it works in my copy of 5.8.x sync'ed today. Probably fixed by change 27604. -- Rick Delaney rick@bort.ca


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org