Skip Menu |
Report information
Id: 122890
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: andreas.koenig.7os6VVqR [at] franz.ak.mind.de
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: (no value)



Subject: Bleadperl v5.21.4-88-ga51d618 breaks DCONWAY/Regexp-Grammars-1.036.tar.gz
To: perlbug [...] perl.org
Date: Fri, 03 Oct 2014 08:18:32 +0200
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Download (untitled) / with headers
text/plain 3.7k
git bisect ---------- commit a51d618a82a7057c3aabb600a7a8691d27f44a34 Author: Yves Orton <demerphq@gmail.com> Date: Fri Sep 19 19:57:34 2014 +0200 rt 122283 - do not recurse into GOSUB/GOSTART when not SCF_DO_SUBSTR sample fail report ------------------ http://www.cpantesters.org/cpan/report/98605662-4578-11e4-966f-9b262c6f0924 perl -V ------- Summary of my perl5 (revision 5 version 21 subversion 5) configuration: Commit id: a83034fcec19e12f9a78fe9e85410047acb3b64c Platform: osname=linux, osvers=3.16-1-amd64, archname=x86_64-linux-thread-multi-ld uname='linux k83 3.16-1-amd64 #1 smp debian 3.16.2-2 (2014-09-08) x86_64 gnulinux ' config_args='-Dprefix=/home/sand/src/perl/repoperls/installed-perls/perl/v5.21.4-101-ga83034f/a2da -Dmyhostname=k83 -Dinstallusrbinperl=n -Uversiononly -Dusedevel -des -Ui_db -Duseithreads -Duselongdouble -DDEBUGGING=-g' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define use64bitint=define, use64bitall=define, uselongdouble=define usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2', optimize='-O2 -g', cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='', gccversion='4.9.1', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3 ivtype='long', ivsize=8, nvtype='long double', nvsize=16, Off_t='off_t', lseeksize=8 alignbytes=16, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong' Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API Built under linux Compiled at Sep 25 2014 22:31:36 %ENV: PERL5LIB="" PERL5OPT="" PERL5_CPANPLUS_IS_RUNNING="21904" PERL5_CPAN_IS_RUNNING="21904" PERL_MM_USE_DEFAULT="1" @INC: /home/sand/src/perl/repoperls/installed-perls/perl/v5.21.4-101-ga83034f/a2da/lib/site_perl/5.21.5/x86_64-linux-thread-multi-ld /home/sand/src/perl/repoperls/installed-perls/perl/v5.21.4-101-ga83034f/a2da/lib/site_perl/5.21.5 /home/sand/src/perl/repoperls/installed-perls/perl/v5.21.4-101-ga83034f/a2da/lib/5.21.5/x86_64-linux-thread-multi-ld /home/sand/src/perl/repoperls/installed-perls/perl/v5.21.4-101-ga83034f/a2da/lib/5.21.5 . -- andreas
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 653b
On Thu Oct 02 23:19:01 2014, andreas.koenig.7os6VVqR@franz.ak.mind.de wrote: Show quoted text
> git bisect > ---------- > commit a51d618a82a7057c3aabb600a7a8691d27f44a34 > Author: Yves Orton <demerphq@gmail.com> > Date: Fri Sep 19 19:57:34 2014 +0200 > > rt 122283 - do not recurse into GOSUB/GOSTART when not SCF_DO_SUBSTR > > sample fail report > ------------------ > http://www.cpantesters.org/cpan/report/98605662-4578-11e4-966f- > 9b262c6f0924
Reduced case: print scalar '7% solution' =~ m/(?: (?&solution) | % ) \Z (?(DEFINE) (?<solution>7\%\ solution) ) /msx, "\n"; Output in 5.21.4: 1 Output in blead is an empty line. -- Father Chrysostomos
RT-Send-CC: perl5-porters [...] perl.org
On Fri Oct 03 22:03:48 2014, sprout wrote: Show quoted text
> On Thu Oct 02 23:19:01 2014, andreas.koenig.7os6VVqR@franz.ak.mind.de wrote:
> > git bisect > > ---------- > > commit a51d618a82a7057c3aabb600a7a8691d27f44a34 > > Author: Yves Orton <demerphq@gmail.com> > > Date: Fri Sep 19 19:57:34 2014 +0200 > > > > rt 122283 - do not recurse into GOSUB/GOSTART when not SCF_DO_SUBSTR > > > > sample fail report > > ------------------ > > http://www.cpantesters.org/cpan/report/98605662-4578-11e4-966f- > > 9b262c6f0924
> > Reduced case: > > print scalar '7% solution' =~ > m/(?: (?&solution) | % ) \Z > (?(DEFINE) (?<solution>7\%\ solution) ) > /msx, "\n"; > > Output in 5.21.4: > 1 > > Output in blead is an empty line.
More info: The anchored-end optimisation determines that the pattern cannot start earlier than position 10 (1 character before the end of the string. Apparently it sees (?&solution) as having zero length, instead of potentially infinite length. If study_chunk (or whatever is responsible) is not going to follow (?&...) recursively, it at least needs to note that max-length optimisations need to be skipped. How does one go about that? Compiling REx "(?: (?&solution) | %% ) \Z%n (?(DEFINE) (?<solution>7\%%\"... rarest char at 0 Final program: 1: BRANCH (5) 2: GOSUB1[+12] (9) 5: BRANCH (FAIL) 6: EXACT <%> (9) 8: TAIL (9) 9: SEOL (10) 10: DEFINEP (12) 12: IFTHEN (25) 14: OPEN1 'solution' (16) 16: EXACT <7% solution> (20) 20: CLOSE1 'solution' (25) 22: LONGJMP (24) 24: TAIL (25) 25: END (0) floating ""$ at 0..1 (checking floating) minlen 0 Enabling $` $& $' support (0x7). EXECUTING... Matching REx "(?: (?&solution) | %% ) \Z%n (?(DEFINE) (?<solution>7\%%\"... against "7%% solution" Intuit: trying to determine minimum start position... Found floating substr ""$ at offset 11... (multiline anchor test skipped) try at offset... Intuit: Successfully guessed: match at offset 10 10 <7%% solutio> <n> | 1:BRANCH(5) etc. -- Father Chrysostomos
Subject: Re: [perl #122890] Bleadperl v5.21.4-88-ga51d618 breaks DCONWAY/Regexp-Grammars-1.036.tar.gz
From: demerphq <demerphq [...] gmail.com>
Date: Sun, 5 Oct 2014 11:47:56 +0200
To: Perl RT Bug Tracker <perlbug-followup [...] perl.org>
CC: Perl5 Porteros <perl5-porters [...] perl.org>
Download (untitled) / with headers
text/plain 1.5k
On 5 October 2014 03:58, Father Chrysostomos via RT <perlbug-followup@perl.org> wrote:
Show quoted text
On Fri Oct 03 22:03:48 2014, sprout wrote:
> On Thu Oct 02 23:19:01 2014, andreas.koenig.7os6VVqR@franz.ak.mind.de wrote:
> > git bisect
> > ----------
> > commit a51d618a82a7057c3aabb600a7a8691d27f44a34
> > Author: Yves Orton <demerphq@gmail.com>
> > Date:   Fri Sep 19 19:57:34 2014 +0200
> >
> > rt 122283 - do not recurse into GOSUB/GOSTART when not SCF_DO_SUBSTR
> >
> > sample fail report
> > ------------------
> > http://www.cpantesters.org/cpan/report/98605662-4578-11e4-966f-
> > 9b262c6f0924
>
> Reduced case:
>
> print scalar '7% solution' =~
>   m/(?: (?&solution) | % ) \Z
>     (?(DEFINE) (?<solution>7\%\ solution) )
>    /msx, "\n";
>
> Output in 5.21.4:
> 1
>
> Output in blead is an empty line.

More info:  The anchored-end optimisation determines that the pattern cannot start earlier than position 10 (1 character before the end of the string.  Apparently it sees (?&solution) as having zero length, instead of potentially infinite length.  If study_chunk (or whatever is responsible) is not going to follow (?&...) recursively, it at least needs to note that max-length optimisations need to be skipped.  How does one go about that?

I don't think it is that simple. There are a bunch of subtle behaviors interacting here in a suboptimal way.

I will try to find time for this this week. Your reduction to simpler case makes my life a lot easier, thanks.

For now please leave this to me.

Yves


--
perl -Mre=debug -e "/just|another|perl|hacker/"
Subject: Re: [perl #122890] Bleadperl v5.21.4-88-ga51d618 breaks DCONWAY/Regexp-Grammars-1.036.tar.gz
Date: Mon, 13 Oct 2014 08:40:36 +0200
From: demerphq <demerphq [...] gmail.com>
To: Perl RT Bug Tracker <perlbug-followup [...] perl.org>
CC: Perl5 Porteros <perl5-porters [...] perl.org>
Download (untitled) / with headers
text/plain 1.8k
On 5 October 2014 11:47, demerphq <demerphq@gmail.com> wrote:
Show quoted text
On 5 October 2014 03:58, Father Chrysostomos via RT <perlbug-followup@perl.org> wrote:
On Fri Oct 03 22:03:48 2014, sprout wrote:
> On Thu Oct 02 23:19:01 2014, andreas.koenig.7os6VVqR@franz.ak.mind.de wrote:
> > git bisect
> > ----------
> > commit a51d618a82a7057c3aabb600a7a8691d27f44a34
> > Author: Yves Orton <demerphq@gmail.com>
> > Date:   Fri Sep 19 19:57:34 2014 +0200
> >
> > rt 122283 - do not recurse into GOSUB/GOSTART when not SCF_DO_SUBSTR
> >
> > sample fail report
> > ------------------
> > http://www.cpantesters.org/cpan/report/98605662-4578-11e4-966f-
> > 9b262c6f0924
>
> Reduced case:
>
> print scalar '7% solution' =~
>   m/(?: (?&solution) | % ) \Z
>     (?(DEFINE) (?<solution>7\%\ solution) )
>    /msx, "\n";
>
> Output in 5.21.4:
> 1
>
> Output in blead is an empty line.

More info:  The anchored-end optimisation determines that the pattern cannot start earlier than position 10 (1 character before the end of the string.  Apparently it sees (?&solution) as having zero length, instead of potentially infinite length.  If study_chunk (or whatever is responsible) is not going to follow (?&...) recursively, it at least needs to note that max-length optimisations need to be skipped.  How does one go about that?

I don't think it is that simple. There are a bunch of subtle behaviors interacting here in a suboptimal way.

I will try to find time for this this week. Your reduction to simpler case makes my life a lot easier, thanks.

For now please leave this to me.


For the record I put some hours into this over the weekend. No final result yet, but I have fixed the memory leaks. The rt 122283 pattern now runs to finish in 372 minutes. :-)

Yves
 

--
perl -Mre=debug -e "/just|another|perl|hacker/"
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
To: perl5-porters [...] perl.org
Subject: Re: [perl #122890] Bleadperl v5.21.4-88-ga51d618 breaks DCONWAY/Regexp-Grammars-1.036.tar.gz
Date: Sun, 19 Oct 2014 09:51:42 +0200
Download (untitled) / with headers
text/plain 127b
also affected: SPROUT/JE-0.065.tar.gz http://www.cpantesters.org/cpan/report/785cae16-54f9-11e4-b101-b4d49ed67017 -- andreas
Date: Mon, 20 Oct 2014 08:45:15 +0200
CC: Perl5 Porteros <perl5-porters [...] perl.org>
Subject: Re: [perl #122890] Bleadperl v5.21.4-88-ga51d618 breaks DCONWAY/Regexp-Grammars-1.036.tar.gz
From: demerphq <demerphq [...] gmail.com>
To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Download (untitled) / with headers
text/plain 445b
On 19 October 2014 09:51, Andreas Koenig <andreas.koenig.7os6VVqR@franz.ak.mind.de> wrote:
Show quoted text

Last night I pushed a patch sequence which hopefully fixes this and RT # 122283.

If that fixes these modules as well then I consider this issue finally properly closed.

Yves 


--
perl -Mre=debug -e "/just|another|perl|hacker/"
Date: Mon, 20 Oct 2014 20:16:26 +0200
CC: Perl5 Porteros <perl5-porters [...] perl.org>
To: demerphq <demerphq [...] gmail.com>
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Subject: Re: [perl #122890] Bleadperl v5.21.4-88-ga51d618 breaks DCONWAY/Regexp-Grammars-1.036.tar.gz
Download (untitled) / with headers
text/plain 683b
Show quoted text
>>>>> On Mon, 20 Oct 2014 08:45:15 +0200, demerphq <demerphq@gmail.com> said:
Show quoted text
> On 19 October 2014 09:51, Andreas Koenig < > andreas.koenig.7os6VVqR@franz.ak.mind.de> wrote:
Show quoted text
Show quoted text
> Last night I pushed a patch sequence which hopefully fixes this and RT # > 122283.
Show quoted text
> If that fixes these modules as well then I consider this issue finally properly > closed.
With v5.21.4-553-gf6f22b1: DCONWAY/Regexp-Grammars-1.036.tar.gz ./Build test -- OK SPROUT/JE-0.065.tar.gz /usr/bin/make test -- OK Thank You && Cheers! -- andreas
CC: Perl5 Porteros <perl5-porters [...] perl.org>
Date: Mon, 20 Oct 2014 22:37:19 +0200
To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
From: demerphq <demerphq [...] gmail.com>
Subject: Re: [perl #122890] Bleadperl v5.21.4-88-ga51d618 breaks DCONWAY/Regexp-Grammars-1.036.tar.gz
Download (untitled) / with headers
text/plain 919b
On 20 October 2014 20:16, Andreas Koenig <andreas.koenig.7os6VVqR@franz.ak.mind.de> wrote:
Show quoted text
>>>>> On Mon, 20 Oct 2014 08:45:15 +0200, demerphq <demerphq@gmail.com> said:

  > On 19 October 2014 09:51, Andreas Koenig <
  > andreas.koenig.7os6VVqR@franz.ak.mind.de> wrote:

  >     also affected:
  >     SPROUT/JE-0.065.tar.gz
  >     http://www.cpantesters.org/cpan/report/785cae16-54f9-11e4-b101-b4d49ed67017



  > Last night I pushed a patch sequence which hopefully fixes this and RT #
  > 122283.

  > If that fixes these modules as well then I consider this issue finally properly
  > closed.

With v5.21.4-553-gf6f22b1:

  DCONWAY/Regexp-Grammars-1.036.tar.gz
  ./Build test -- OK

  SPROUT/JE-0.065.tar.gz
  /usr/bin/make test -- OK

Thank You && Cheers!

And thank you for your CPAN reports. :-)

cheers,
Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org