Skip Menu |
Report information
Id: 128170
Status: resolved
Priority: 0/
Queue: perl5

Owner: khw <khw [at] cpan.org>
Requestors: brian.carpenter [at] gmail.com
dcollinsn [at] gmail.com
Cc:
AdminCc:

Operating System: All
PatchStatus: (no value)
Severity: low
Type:
Perl Version:
  • 5.24.0
  • 5.25.2
Fixed In: (no value)



Subject: Assert fail in regcomp.c with no other symptoms: perl -e '/(?<=/'
Download (untitled) / with headers
text/plain 9.5k
Greetings Porters, I have compiled bleadperl with the afl-gcc compiler using: ./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Uuselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -Dusequadmath -des AFL_HARDEN=1 make && make test And then fuzzed the resulting binary using: AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@ After reducing testcases using `afl-tmin` and performing additional minimization by hand, I have located the following testcase that triggers an assert fail in debugging builds of the perl interpreter. The testcase is the 6-character file: /(?<=/ On normal builds, this exits with the expected error. On debug builds, this returns an assert fail. dcollins@nightshade64:~$ perl/miniperl -e '/(?<=/' Unmatched ( in regex; marked by <-- HERE in m/( <-- HERE ?<=/ at -e line 1. dcollins@nightshade64:~$ perldebug/miniperl -e '/(?<=/' miniperl: regcomp.c:10621: S_reg: Assertion `(pRExC_state->parse) < (pRExC_state->end)' failed. Aborted Debugging tool output is below. A bisect was performed and points to the following diff: cfbef7dc3bfb89e4ed2c00ea9c9e3bcfd0b170fd is the first bad commit commit cfbef7dc3bfb89e4ed2c00ea9c9e3bcfd0b170fd Author: Karl Williamson <khw@cpan.org> Date: Wed Feb 10 16:27:13 2016 -0700 regcomp.c: Fix some parsing glitches I undertook a code review of how regcomp.c parses things in light of the tickets found by the fuzzer, https://rt.perl.org/Ticket/Display.html?id=126546. This commit is the result of my efforts so far. I was not planning to push it now, but the work found a couple of new error messages that should be raised, and this has to be done before the visible changes code freeze coming up all too soon. I will add test cases after that freeze, including if to see that these changes fix all the observed issues. The audit was tedious, and may have missed some things. Several issues occurred in multiple places. One is to not advance the parse by UTF8SKIP appropriately; another is to subtract one byte from the parse and assume that that is pointing to the beginning of the previous character (which under UTF-8 it may not). Another is to assume that that the pattern is a C string, that there are no interior NULs in it. I also found unnecessary tests, given that an SV always has a terminating NUL. :040000 040000 a58a2d3154c5e346f9dfab14f7ca9897cbe06cfb 41b83ee5e555d3ddc87935c3a3b30db821741214 M pod :100644 100644 c00fdffd93dd9de69326e9b567d912dd9a1bee3f aa06bc6e8da12440f5d5b649cd5faa24a8853cc7 M regcomp.c bisect run success **GDB** dcollins@nightshade64:~$ gdb --args perldebug/miniperl -e '/(?<=/' GNU gdb (GDB) 7.10 Copyright (C) 2015 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from perldebug/miniperl...done. (gdb) run Starting program: /home/dcollins/perldebug/miniperl -e /\(\?\<=/ [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". miniperl: regcomp.c:10621: S_reg: Assertion `(pRExC_state->parse) < (pRExC_state->end)' failed. Program received signal SIGABRT, Aborted. 0x00007ffff6cf9478 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x00007ffff6cf9478 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007ffff6cfa8fa in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007ffff6cf23a7 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x00007ffff6cf2452 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x00000000006efe09 in S_reg (pRExC_state=0x7fffffffdcf0, paren=60, flagp=0x7fffffffd8bc, depth=5) at regcomp.c:10621 #5 0x00000000007186ed in S_regatom (pRExC_state=pRExC_state@entry=0x7fffffffdcf0, flagp=flagp@entry=0x7fffffffda34, depth=depth@entry=4) at regcomp.c:12354 #6 0x0000000000725d0d in S_regpiece (depth=<optimized out>, flagp=<synthetic pointer>, pRExC_state=0x7fffffffdcf0) at regcomp.c:11421 #7 S_regbranch (pRExC_state=pRExC_state@entry=0x7fffffffdcf0, flagp=flagp@entry=0x7fffffffdacc, first=first@entry=1, depth=depth@entry=2) at regcomp.c:11346 #8 0x0000000000745097 in S_reg (pRExC_state=pRExC_state@entry=0x7fffffffdcf0, flagp=flagp@entry=0x7fffffffdbbc, depth=1, paren=0) at regcomp.c:11084 #9 0x000000000077b231 in Perl_re_op_compile (patternp=<optimized out>, pat_count=<optimized out>, expr=<optimized out>, eng=0xf33160 <PL_core_reg_engine>, old_re=0x0, is_bare_re=<optimized out>, orig_rx_flags=0, pm_flags=0) at regcomp.c:7027 #10 0x00000000004c4033 in Perl_pmruntime (o=0x1202718, expr=0x12026d8, repl=0x0, isreg=isreg@entry=true, floor=<optimized out>) at op.c:5632 #11 0x000000000064b5cd in Perl_yyparse (gramtype=gramtype@entry=258) at perly.y:1028 #12 0x0000000000512111 in S_parse_body (xsinit=0xec1b00 <xs_init>, env=0x0) at perl.c:2365 #13 perl_parse (my_perl=<optimized out>, xsinit=xsinit@entry=0xec1b00 <xs_init>, argc=<optimized out>, argv=<optimized out>, env=env@entry=0x0) at perl.c:1681 #14 0x000000000040f838 in main (argc=<optimized out>, argv=<optimized out>, env=<optimized out>) at miniperlmain.c:120 (gdb) f 4 #4 0x00000000006efe09 in S_reg (pRExC_state=0x7fffffffdcf0, paren=60, flagp=0x7fffffffd8bc, depth=5) at regcomp.c:10621 10621 assert(RExC_parse < RExC_end); (gdb) info locals is_logical = false seqstart = <optimized out> endptr = 0x0 has_intervening_patws = false ret = 0x0 br = <optimized out> lastbr = <optimized out> ender = 0x0 parno = 0 flags = 0 oregflags = 0 have_branch = false is_open = false freeze_paren = 0 after_freeze = 0 num = <optimized out> parse_start = 0x1202861 "?<=" oregcomp_parse = 0x1202861 "?<=" re_debug_flags = 0 (gdb) l 10616 goto capturing_parens; 10617 } 10618 RExC_seen |= REG_LOOKBEHIND_SEEN; 10619 RExC_in_lookbehind++; 10620 RExC_parse++; 10621 assert(RExC_parse < RExC_end); 10622 /* FALLTHROUGH */ 10623 case '=': /* (?=...) */ 10624 RExC_seen_zerolen++; 10625 break; (gdb) **PERL -V** dcollins@nightshade64:~/perl$ ./perl -Ilib -V Summary of my perl5 (revision 5 version 25 subversion 1) configuration: Commit id: 8255316ac82c0fbd7ae49f2f9a217ce063d7bbfb Platform: osname=linux, osvers=4.5.0-2-amd64, archname=x86_64-linux-ld uname='linux nightshade64 4.5.0-2-amd64 #1 smp debian 4.5.3-2 (2016-05-08) x86_64 gnulinux ' config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des' hint=recommended, useposix=true, d_sigaction=define useithreads=undef, usemultiplicity=undef use64bitint=define, use64bitall=define, uselongdouble=define usemymalloc=n, bincompat5005=undef Compiler: cc='ccache afl-gcc', ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-g', cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='', gccversion='6.1.0', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3 ivtype='long', ivsize=8, nvtype='long double', nvsize=16, Off_t='off_t', lseeksize=8 alignbytes=16, prototype=define Linker and Libraries: ld='ccache afl-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/local/lib/gcc/x86_64-pc-linux-gnu/6.1.0/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.22.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.22' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong' Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_OP_PARENT PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE USE_PERLIO USE_PERL_ATOF Built under linux Compiled at May 16 2016 07:45:27 @INC: lib /usr/local/perl-afl/lib/site_perl/5.25.1/x86_64-linux-ld /usr/local/perl-afl/lib/site_perl/5.25.1 /usr/local/perl-afl/lib/5.25.1/x86_64-linux-ld /usr/local/perl-afl/lib/5.25.1 /usr/local/perl-afl/lib/site_perl/5.24.0 /usr/local/perl-afl/lib/site_perl .
Date: Sat, 21 May 2016 06:32:27 +0200
To: Perl5 Porteros <perl5-porters [...] perl.org>
CC: bugs-bitbucket [...] rt.perl.org
Subject: Re: [perl #128170] Assert fail in regcomp.c with no other symptoms: perl -e '/(?<=/'
From: demerphq <demerphq [...] gmail.com>
Download (untitled) / with headers
text/plain 10.2k
Download (untitled) / with headers
text/html 12.9k


On 17 May 2016 3:04 p.m., "Dan Collins" <perlbug-followup@perl.org> wrote:
Show quoted text

>
> # New Ticket Created by  Dan Collins
> # Please include the string:  [perl #128170]
> # in the subject line of all future correspondence about this issue.
> # <URL: https://rt.perl.org/Ticket/Display.html?id=128170 >
>
>
> Greetings Porters,
>
> I have compiled bleadperl with the afl-gcc compiler using:
>
> ./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Uuselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -Dusequadmath -des
> AFL_HARDEN=1 make && make test
>
> And then fuzzed the resulting binary using:
>
> AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@
>
> After reducing testcases using `afl-tmin` and performing additional minimization by hand, I have located the following testcase that triggers an assert fail in debugging builds of the perl interpreter. The testcase is the 6-character file:
>
> /(?<=/
>
> On normal builds, this exits with the expected error. On debug builds, this returns an assert fail.
>
> dcollins@nightshade64:~$ perl/miniperl -e '/(?<=/'
> Unmatched ( in regex; marked by <-- HERE in m/( <-- HERE ?<=/ at -e line 1.
> dcollins@nightshade64:~$ perldebug/miniperl -e '/(?<=/'
> miniperl: regcomp.c:10621: S_reg: Assertion `(pRExC_state->parse) < (pRExC_state->end)' failed.
> Aborted
>
> Debugging tool output is below. A bisect was performed and points to the following diff:
>
> cfbef7dc3bfb89e4ed2c00ea9c9e3bcfd0b170fd is the first bad commit
> commit cfbef7dc3bfb89e4ed2c00ea9c9e3bcfd0b170fd
> Author: Karl Williamson <khw@cpan.org>
> Date:   Wed Feb 10 16:27:13 2016 -0700
>
>     regcomp.c: Fix some parsing glitches
>
>     I undertook a code review of how regcomp.c parses things in light of the
>     tickets found by the fuzzer,
>     https://rt.perl.org/Ticket/Display.html?id=126546.  This commit is the
>     result of my efforts so far.  I was not planning to push it now, but the
>     work found a couple of new error messages that should be raised, and
>     this has to be done before the visible changes code freeze coming up all
>     too soon.  I will add test cases after that freeze, including if to see
>     that these changes fix all the observed issues.
>
>     The audit was tedious, and may have missed some things.  Several issues
>     occurred in multiple places.  One is to not advance the parse by
>     UTF8SKIP appropriately; another is to subtract one byte from the parse
>     and assume that that is pointing to the beginning of the previous
>     character (which under UTF-8 it may not).  Another is to assume that
>     that the pattern is a C string, that there are no interior NULs in it.
>     I also found unnecessary tests, given that an SV always has a
>     terminating NUL.

This is not correct. Xs code can and in practice does create sv's with no trailing nulls.

> :040000 040000 a58a2d3154c5e346f9dfab14f7ca9897cbe06cfb 41b83ee5e555d3ddc87935c3a3b30db821741214 M      pod
Show quoted text

> :100644 100644 c00fdffd93dd9de69326e9b567d912dd9a1bee3f aa06bc6e8da12440f5d5b649cd5faa24a8853cc7 M      regcomp.c
> bisect run success
>
>
>
> **GDB**
>
> dcollins@nightshade64:~$ gdb --args perldebug/miniperl -e '/(?<=/'
> GNU gdb (GDB) 7.10
> Copyright (C) 2015 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/>.
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from perldebug/miniperl...done.
> (gdb) run
> Starting program: /home/dcollins/perldebug/miniperl -e /\(\?\<=/
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> miniperl: regcomp.c:10621: S_reg: Assertion `(pRExC_state->parse) < (pRExC_state->end)' failed.
>
> Program received signal SIGABRT, Aborted.
> 0x00007ffff6cf9478 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> (gdb) bt
> #0  0x00007ffff6cf9478 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1  0x00007ffff6cfa8fa in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2  0x00007ffff6cf23a7 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
> #3  0x00007ffff6cf2452 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
> #4  0x00000000006efe09 in S_reg (pRExC_state=0x7fffffffdcf0, paren=60, flagp=0x7fffffffd8bc, depth=5) at regcomp.c:10621
> #5  0x00000000007186ed in S_regatom (pRExC_state=pRExC_state@entry=0x7fffffffdcf0, flagp=flagp@entry=0x7fffffffda34, depth=depth@entry=4) at regcomp.c:12354
> #6  0x0000000000725d0d in S_regpiece (depth=<optimized out>, flagp=<synthetic pointer>, pRExC_state=0x7fffffffdcf0) at regcomp.c:11421
> #7  S_regbranch (pRExC_state=pRExC_state@entry=0x7fffffffdcf0, flagp=flagp@entry=0x7fffffffdacc, first=first@entry=1, depth=depth@entry=2) at regcomp.c:11346
> #8  0x0000000000745097 in S_reg (pRExC_state=pRExC_state@entry=0x7fffffffdcf0, flagp=flagp@entry=0x7fffffffdbbc, depth=1, paren=0) at regcomp.c:11084
> #9  0x000000000077b231 in Perl_re_op_compile (patternp=<optimized out>, pat_count=<optimized out>, expr=<optimized out>, eng=0xf33160 <PL_core_reg_engine>, old_re=0x0,
>     is_bare_re=<optimized out>, orig_rx_flags=0, pm_flags=0) at regcomp.c:7027
> #10 0x00000000004c4033 in Perl_pmruntime (o=0x1202718, expr=0x12026d8, repl=0x0, isreg=isreg@entry=true, floor=<optimized out>) at op.c:5632
> #11 0x000000000064b5cd in Perl_yyparse (gramtype=gramtype@entry=258) at perly.y:1028
> #12 0x0000000000512111 in S_parse_body (xsinit=0xec1b00 <xs_init>, env=0x0) at perl.c:2365
> #13 perl_parse (my_perl=<optimized out>, xsinit=xsinit@entry=0xec1b00 <xs_init>, argc=<optimized out>, argv=<optimized out>, env=env@entry=0x0) at perl.c:1681
> #14 0x000000000040f838 in main (argc=<optimized out>, argv=<optimized out>, env=<optimized out>) at miniperlmain.c:120
> (gdb) f 4
> #4  0x00000000006efe09 in S_reg (pRExC_state=0x7fffffffdcf0, paren=60, flagp=0x7fffffffd8bc, depth=5) at regcomp.c:10621
> 10621                   assert(RExC_parse < RExC_end);
> (gdb) info locals
> is_logical = false
> seqstart = <optimized out>
> endptr = 0x0
> has_intervening_patws = false
> ret = 0x0
> br = <optimized out>
> lastbr = <optimized out>
> ender = 0x0
> parno = 0
> flags = 0
> oregflags = 0
> have_branch = false
> is_open = false
> freeze_paren = 0
> after_freeze = 0
> num = <optimized out>
> parse_start = 0x1202861 "?<="
> oregcomp_parse = 0x1202861 "?<="
> re_debug_flags = 0
> (gdb) l
> 10616                       goto capturing_parens;
> 10617                   }
> 10618                   RExC_seen |= REG_LOOKBEHIND_SEEN;
> 10619                   RExC_in_lookbehind++;
> 10620                   RExC_parse++;
> 10621                   assert(RExC_parse < RExC_end);
> 10622                   /* FALLTHROUGH */
> 10623               case '=':           /* (?=...) */
> 10624                   RExC_seen_zerolen++;
> 10625                   break;
> (gdb)
>
>
> **PERL -V**
>
> dcollins@nightshade64:~/perl$ ./perl -Ilib -V
> Summary of my perl5 (revision 5 version 25 subversion 1) configuration:
>   Commit id: 8255316ac82c0fbd7ae49f2f9a217ce063d7bbfb
>   Platform:
>     osname=linux, osvers=4.5.0-2-amd64, archname=x86_64-linux-ld
>     uname='linux nightshade64 4.5.0-2-amd64 #1 smp debian 4.5.3-2 (2016-05-08) x86_64 gnulinux '
>     config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des'
>     hint=recommended, useposix=true, d_sigaction=define
>     useithreads=undef, usemultiplicity=undef
>     use64bitint=define, use64bitall=define, uselongdouble=define
>     usemymalloc=n, bincompat5005=undef
>   Compiler:
>     cc='ccache afl-gcc', ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
>     optimize='-g',
>     cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
>     ccversion='', gccversion='6.1.0', gccosandvers=''
>     intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
>     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
>     ivtype='long', ivsize=8, nvtype='long double', nvsize=16, Off_t='off_t', lseeksize=8
>     alignbytes=16, prototype=define
>   Linker and Libraries:
>     ld='ccache afl-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
>     libpth=/usr/local/lib /usr/local/lib/gcc/x86_64-pc-linux-gnu/6.1.0/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
>     libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
>     perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
>     libc=libc-2.22.so, so=so, useshrplib=false, libperl=libperl.a
>     gnulibc_version='2.22'
>   Dynamic Linking:
>     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
>     cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong'
>
>
> Characteristics of this binary (from libperl):
>   Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE
>                         PERL_DONT_CREATE_GVSV
>                         PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
>                         PERL_OP_PARENT PERL_PRESERVE_IVUV PERL_USE_DEVEL
>                         USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
>                         USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
>                         USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE
>                         USE_PERLIO USE_PERL_ATOF
>   Built under linux
>   Compiled at May 16 2016 07:45:27
>   @INC:
>     lib
>     /usr/local/perl-afl/lib/site_perl/5.25.1/x86_64-linux-ld
>     /usr/local/perl-afl/lib/site_perl/5.25.1
>     /usr/local/perl-afl/lib/5.25.1/x86_64-linux-ld
>     /usr/local/perl-afl/lib/5.25.1
>     /usr/local/perl-afl/lib/site_perl/5.24.0
>     /usr/local/perl-afl/lib/site_perl
>     .
>

To: demerphq <demerphq [...] gmail.com>, Perl5 Porteros <perl5-porters [...] perl.org>
CC: bugs-bitbucket [...] rt.perl.org
Subject: Re: [perl #128170] Assert fail in regcomp.c with no other symptoms: perl -e '/(?<=/'
From: Karl Williamson <public [...] khwilliamson.com>
Date: Mon, 23 May 2016 14:06:16 -0600
Download (untitled) / with headers
text/plain 11.4k
On 05/20/2016 10:32 PM, demerphq wrote: Show quoted text
> > On 17 May 2016 3:04 p.m., "Dan Collins" <perlbug-followup@perl.org > <mailto:perlbug-followup@perl.org>> wrote:
> > > > # New Ticket Created by Dan Collins > > # Please include the string: [perl #128170] > > # in the subject line of all future correspondence about this issue. > > # <URL: https://rt.perl.org/Ticket/Display.html?id=128170 > > > > > > > Greetings Porters, > > > > I have compiled bleadperl with the afl-gcc compiler using: > > > > ./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache
> afl-gcc' -Uuselongdouble -Duse64bitall -Doptimize=-g -Uversiononly > -Uman1dir -Uman3dir -Dusequadmath -des
> > AFL_HARDEN=1 make && make test > > > > And then fuzzed the resulting binary using: > > > > AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@ > > > > After reducing testcases using `afl-tmin` and performing additional
> minimization by hand, I have located the following testcase that > triggers an assert fail in debugging builds of the perl interpreter. The > testcase is the 6-character file:
> > > > /(?<=/ > > > > On normal builds, this exits with the expected error. On debug
> builds, this returns an assert fail.
> > > > dcollins@nightshade64:~$ perl/miniperl -e '/(?<=/' > > Unmatched ( in regex; marked by <-- HERE in m/( <-- HERE ?<=/ at -e
> line 1.
> > dcollins@nightshade64:~$ perldebug/miniperl -e '/(?<=/' > > miniperl: regcomp.c:10621: S_reg: Assertion `(pRExC_state->parse) <
> (pRExC_state->end)' failed.
> > Aborted > > > > Debugging tool output is below. A bisect was performed and points to
> the following diff:
> > > > cfbef7dc3bfb89e4ed2c00ea9c9e3bcfd0b170fd is the first bad commit > > commit cfbef7dc3bfb89e4ed2c00ea9c9e3bcfd0b170fd > > Author: Karl Williamson <khw@cpan.org <mailto:khw@cpan.org>> > > Date: Wed Feb 10 16:27:13 2016 -0700 > > > > regcomp.c: Fix some parsing glitches > > > > I undertook a code review of how regcomp.c parses things in light
> of the
> > tickets found by the fuzzer, > > https://rt.perl.org/Ticket/Display.html?id=126546. This commit is the > > result of my efforts so far. I was not planning to push it now,
> but the
> > work found a couple of new error messages that should be raised, and > > this has to be done before the visible changes code freeze coming
> up all
> > too soon. I will add test cases after that freeze, including if
> to see
> > that these changes fix all the observed issues. > > > > The audit was tedious, and may have missed some things. Several
> issues
> > occurred in multiple places. One is to not advance the parse by > > UTF8SKIP appropriately; another is to subtract one byte from the
> parse
> > and assume that that is pointing to the beginning of the previous > > character (which under UTF-8 it may not). Another is to assume that > > that the pattern is a C string, that there are no interior NULs
> in it.
> > I also found unnecessary tests, given that an SV always has a > > terminating NUL.
> > This is not correct. Xs code can and in practice does create sv's with > no trailing nulls.
Hmm. I've written a bunch of code with that assumption, partly because a bunch of code that was already in place has that assumption. I did add assertions to this effect in the regex code at the same time as this commit was done. Can you give an example of how XS could do this? All I can think of is if it arbitrarily turns on a POK flag. Show quoted text
>
> > :040000 040000 a58a2d3154c5e346f9dfab14f7ca9897cbe06cfb
> 41b83ee5e555d3ddc87935c3a3b30db821741214 M pod
> > :100644 100644 c00fdffd93dd9de69326e9b567d912dd9a1bee3f
> aa06bc6e8da12440f5d5b649cd5faa24a8853cc7 M regcomp.c
> > bisect run success > > > > > > > > **GDB** > > > > dcollins@nightshade64:~$ gdb --args perldebug/miniperl -e '/(?<=/' > > GNU gdb (GDB) 7.10 > > Copyright (C) 2015 Free Software Foundation, Inc. > > License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> > This is free software: you are free to change and redistribute it. > > There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> > and "show warranty" for details. > > This GDB was configured as "x86_64-unknown-linux-gnu". > > Type "show configuration" for configuration details. > > For bug reporting instructions, please see: > > <http://www.gnu.org/software/gdb/bugs/>. > > Find the GDB manual and other documentation resources online at: > > <http://www.gnu.org/software/gdb/documentation/>. > > For help, type "help". > > Type "apropos word" to search for commands related to "word"... > > Reading symbols from perldebug/miniperl...done. > > (gdb) run > > Starting program: /home/dcollins/perldebug/miniperl -e /\(\?\<=/ > > [Thread debugging using libthread_db enabled] > > Using host libthread_db library
> "/lib/x86_64-linux-gnu/libthread_db.so.1".
> > miniperl: regcomp.c:10621: S_reg: Assertion `(pRExC_state->parse) <
> (pRExC_state->end)' failed.
> > > > Program received signal SIGABRT, Aborted. > > 0x00007ffff6cf9478 in raise () from /lib/x86_64-linux-gnu/libc.so.6 > > (gdb) bt > > #0 0x00007ffff6cf9478 in raise () from /lib/x86_64-linux-gnu/libc.so.6 > > #1 0x00007ffff6cfa8fa in abort () from /lib/x86_64-linux-gnu/libc.so.6 > > #2 0x00007ffff6cf23a7 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > > #3 0x00007ffff6cf2452 in __assert_fail () from
> /lib/x86_64-linux-gnu/libc.so.6
> > #4 0x00000000006efe09 in S_reg (pRExC_state=0x7fffffffdcf0,
> paren=60, flagp=0x7fffffffd8bc, depth=5) at regcomp.c:10621
> > #5 0x00000000007186ed in S_regatom
> (pRExC_state=pRExC_state@entry=0x7fffffffdcf0, > flagp=flagp@entry=0x7fffffffda34, depth=depth@entry=4) at regcomp.c:12354
> > #6 0x0000000000725d0d in S_regpiece (depth=<optimized out>,
> flagp=<synthetic pointer>, pRExC_state=0x7fffffffdcf0) at regcomp.c:11421
> > #7 S_regbranch (pRExC_state=pRExC_state@entry=0x7fffffffdcf0,
> flagp=flagp@entry=0x7fffffffdacc, first=first@entry=1, > depth=depth@entry=2) at regcomp.c:11346
> > #8 0x0000000000745097 in S_reg
> (pRExC_state=pRExC_state@entry=0x7fffffffdcf0, > flagp=flagp@entry=0x7fffffffdbbc, depth=1, paren=0) at regcomp.c:11084
> > #9 0x000000000077b231 in Perl_re_op_compile (patternp=<optimized
> out>, pat_count=<optimized out>, expr=<optimized out>, eng=0xf33160
> <PL_core_reg_engine>, old_re=0x0,
> > is_bare_re=<optimized out>, orig_rx_flags=0, pm_flags=0) at
> regcomp.c:7027
> > #10 0x00000000004c4033 in Perl_pmruntime (o=0x1202718,
> expr=0x12026d8, repl=0x0, isreg=isreg@entry=true, floor=<optimized out>) > at op.c:5632
> > #11 0x000000000064b5cd in Perl_yyparse (gramtype=gramtype@entry=258)
> at perly.y:1028
> > #12 0x0000000000512111 in S_parse_body (xsinit=0xec1b00 <xs_init>,
> env=0x0) at perl.c:2365
> > #13 perl_parse (my_perl=<optimized out>, xsinit=xsinit@entry=0xec1b00
> <xs_init>, argc=<optimized out>, argv=<optimized out>, > env=env@entry=0x0) at perl.c:1681
> > #14 0x000000000040f838 in main (argc=<optimized out>, argv=<optimized
> out>, env=<optimized out>) at miniperlmain.c:120
> > (gdb) f 4 > > #4 0x00000000006efe09 in S_reg (pRExC_state=0x7fffffffdcf0,
> paren=60, flagp=0x7fffffffd8bc, depth=5) at regcomp.c:10621
> > 10621 assert(RExC_parse < RExC_end); > > (gdb) info locals > > is_logical = false > > seqstart = <optimized out> > > endptr = 0x0 > > has_intervening_patws = false > > ret = 0x0 > > br = <optimized out> > > lastbr = <optimized out> > > ender = 0x0 > > parno = 0 > > flags = 0 > > oregflags = 0 > > have_branch = false > > is_open = false > > freeze_paren = 0 > > after_freeze = 0 > > num = <optimized out> > > parse_start = 0x1202861 "?<=" > > oregcomp_parse = 0x1202861 "?<=" > > re_debug_flags = 0 > > (gdb) l > > 10616 goto capturing_parens; > > 10617 } > > 10618 RExC_seen |= REG_LOOKBEHIND_SEEN; > > 10619 RExC_in_lookbehind++; > > 10620 RExC_parse++; > > 10621 assert(RExC_parse < RExC_end); > > 10622 /* FALLTHROUGH */ > > 10623 case '=': /* (?=...) */ > > 10624 RExC_seen_zerolen++; > > 10625 break; > > (gdb) > > > > > > **PERL -V** > > > > dcollins@nightshade64:~/perl$ ./perl -Ilib -V > > Summary of my perl5 (revision 5 version 25 subversion 1) configuration: > > Commit id: 8255316ac82c0fbd7ae49f2f9a217ce063d7bbfb > > Platform: > > osname=linux, osvers=4.5.0-2-amd64, archname=x86_64-linux-ld > > uname='linux nightshade64 4.5.0-2-amd64 #1 smp debian 4.5.3-2
> (2016-05-08) x86_64 gnulinux '
> > config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache
> afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly > -Uman1dir -Uman3dir -des'
> > hint=recommended, useposix=true, d_sigaction=define > > useithreads=undef, usemultiplicity=undef > > use64bitint=define, use64bitall=define, uselongdouble=define > > usemymalloc=n, bincompat5005=undef > > Compiler: > > cc='ccache afl-gcc', ccflags ='-fwrapv -fno-strict-aliasing -pipe
> -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE > -D_FILE_OFFSET_BITS=64',
> > optimize='-g', > > cppflags='-fwrapv -fno-strict-aliasing -pipe
> -fstack-protector-strong -I/usr/local/include'
> > ccversion='', gccversion='6.1.0', gccosandvers='' > > intsize=4, longsize=8, ptrsize=8, doublesize=8,
> byteorder=12345678, doublekind=3
> > d_longlong=define, longlongsize=8, d_longdbl=define,
> longdblsize=16, longdblkind=3
> > ivtype='long', ivsize=8, nvtype='long double', nvsize=16,
> Off_t='off_t', lseeksize=8
> > alignbytes=16, prototype=define > > Linker and Libraries: > > ld='ccache afl-gcc', ldflags =' -fstack-protector-strong
> -L/usr/local/lib'
> > libpth=/usr/local/lib
> /usr/local/lib/gcc/x86_64-pc-linux-gnu/6.1.0/include-fixed > /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib > /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
> > libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc > > perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc > > libc=libc-2.22.so <http://libc-2.22.so>, so=so, useshrplib=false,
> libperl=libperl.a
> > gnulibc_version='2.22' > > Dynamic Linking: > > dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' > > cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib
> -fstack-protector-strong'
> > > > > > Characteristics of this binary (from libperl): > > Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE > > PERL_DONT_CREATE_GVSV > > PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
> PERL_MALLOC_WRAP
> > PERL_OP_PARENT PERL_PRESERVE_IVUV PERL_USE_DEVEL > > USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES > > USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE > > USE_LOCALE_NUMERIC USE_LOCALE_TIME
> USE_LONG_DOUBLE
> > USE_PERLIO USE_PERL_ATOF > > Built under linux > > Compiled at May 16 2016 07:45:27 > > @INC: > > lib > > /usr/local/perl-afl/lib/site_perl/5.25.1/x86_64-linux-ld > > /usr/local/perl-afl/lib/site_perl/5.25.1 > > /usr/local/perl-afl/lib/5.25.1/x86_64-linux-ld > > /usr/local/perl-afl/lib/5.25.1 > > /usr/local/perl-afl/lib/site_perl/5.24.0 > > /usr/local/perl-afl/lib/site_perl > > . > >
>
From: demerphq <demerphq [...] gmail.com>
Subject: Re: [perl #128170] Assert fail in regcomp.c with no other symptoms: perl -e '/(?<=/'
To: karl williamson <public [...] khwilliamson.com>
CC: Perl5 Porteros <perl5-porters [...] perl.org>, bugs-bitbucket [...] rt.perl.org
Date: Mon, 23 May 2016 23:15:58 +0200
Download (untitled) / with headers
text/plain 3.7k


On 23 May 2016 16:06, "Karl Williamson" <public@khwilliamson.com> wrote:
Show quoted text

>
> On 05/20/2016 10:32 PM, demerphq wrote:
>>
>>
>> On 17 May 2016 3:04 p.m., "Dan Collins" <perlbug-followup@perl.org
>> <mailto:perlbug-followup@perl.org>> wrote:
>>  >
>>  > # New Ticket Created by  Dan Collins
>>  > # Please include the string:  [perl #128170]
>>  > # in the subject line of all future correspondence about this issue.
>>  > # <URL: https://rt.perl.org/Ticket/Display.html?id=128170 >
>>  >
>>  >
>>  > Greetings Porters,
>>  >
>>  > I have compiled bleadperl with the afl-gcc compiler using:
>>  >
>>  > ./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache
>> afl-gcc' -Uuselongdouble -Duse64bitall -Doptimize=-g -Uversiononly
>> -Uman1dir -Uman3dir -Dusequadmath -des
>>  > AFL_HARDEN=1 make && make test
>>  >
>>  > And then fuzzed the resulting binary using:
>>  >
>>  > AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@
>>  >
>>  > After reducing testcases using `afl-tmin` and performing additional
>> minimization by hand, I have located the following testcase that
>> triggers an assert fail in debugging builds of the perl interpreter. The
>> testcase is the 6-character file:
>>  >
>>  > /(?<=/
>>  >
>>  > On normal builds, this exits with the expected error. On debug
>> builds, this returns an assert fail.
>>  >
>>  > dcollins@nightshade64:~$ perl/miniperl -e '/(?<=/'
>>  > Unmatched ( in regex; marked by <-- HERE in m/( <-- HERE ?<=/ at -e
>> line 1.
>>  > dcollins@nightshade64:~$ perldebug/miniperl -e '/(?<=/'
>>  > miniperl: regcomp.c:10621: S_reg: Assertion `(pRExC_state->parse) <
>> (pRExC_state->end)' failed.
>>  > Aborted
>>  >
>>  > Debugging tool output is below. A bisect was performed and points to
>> the following diff:
>>  >
>>  > cfbef7dc3bfb89e4ed2c00ea9c9e3bcfd0b170fd is the first bad commit
>>  > commit cfbef7dc3bfb89e4ed2c00ea9c9e3bcfd0b170fd
>>  > Author: Karl Williamson <khw@cpan.org <mailto:khw@cpan.org>>
>>
>>  > Date:   Wed Feb 10 16:27:13 2016 -0700
>>  >
>>  >     regcomp.c: Fix some parsing glitches
>>  >
>>  >     I undertook a code review of how regcomp.c parses things in light
>> of the
>>  >     tickets found by the fuzzer,
>>  > https://rt.perl.org/Ticket/Display.html?id=126546.  This commit is the
>>  >     result of my efforts so far.  I was not planning to push it now,
>> but the
>>  >     work found a couple of new error messages that should be raised, and
>>  >     this has to be done before the visible changes code freeze coming
>> up all
>>  >     too soon.  I will add test cases after that freeze, including if
>> to see
>>  >     that these changes fix all the observed issues.
>>  >
>>  >     The audit was tedious, and may have missed some things.  Several
>> issues
>>  >     occurred in multiple places.  One is to not advance the parse by
>>  >     UTF8SKIP appropriately; another is to subtract one byte from the
>> parse
>>  >     and assume that that is pointing to the beginning of the previous
>>  >     character (which under UTF-8 it may not).  Another is to assume that
>>  >     that the pattern is a C string, that there are no interior NULs
>> in it.
>>  >     I also found unnecessary tests, given that an SV always has a
>>  >     terminating NUL.
>>
>> This is not correct. Xs code can and in practice does create sv's with
>> no trailing nulls.
>
>
> Hmm.  I've written a bunch of code with that assumption, partly because a bunch of code that was already in place has that assumption.
>
> I did add assertions to this effect in the regex code at the same time as this commit was done.
>
> Can you give an example of how XS could do this?  All I can think of is if it arbitrarily turns on a POK flag.

Yeah, exactly. But in xs that is pretty standard. I wouldnt rely on our strings always having a trailing null.

Yves

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 407b
On Mon May 23 14:16:31 2016, demerphq wrote: Show quoted text
> Yeah, exactly. But in xs that is pretty standard. I wouldnt rely on our > strings always having a trailing null.
In fact, I would like to remove that requirement (to allow substr to avoid copying the string), but someone else has pointed out that it is also common in XS to pass SvPV to system functions that expect a trailing null. -- Father Chrysostomos
Date: Tue, 24 May 2016 01:06:27 +0200
To: Perl RT Bug Tracker <perlbug-followup [...] perl.org>
CC: Perl5 Porteros <perl5-porters [...] perl.org>
From: demerphq <demerphq [...] gmail.com>
Subject: Re: [perl #128170] Assert fail in regcomp.c with no other symptoms: perl -e '/(?<=/'
Download (untitled) / with headers
text/plain 614b


On 23 May 2016 6:46 p.m., "Father Chrysostomos via RT" <perlbug-followup@perl.org> wrote:
Show quoted text

>
> On Mon May 23 14:16:31 2016, demerphq wrote:
> > Yeah, exactly. But in xs that is pretty standard. I wouldnt rely on our
> > strings always having a trailing null.
>
> In fact, I would like to remove that requirement (to allow substr to avoid copying the string), but someone else has pointed out that it is also common in XS to pass SvPV to system functions that expect a trailing null.

Maybe we should have a SvPV_null macro that DTRT for such cases and abandon any requirements that pv's are null terminated.

Yves

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 957b
On Mon May 23 16:07:03 2016, demerphq wrote: Show quoted text
> On 23 May 2016 6:46 p.m., "Father Chrysostomos via RT" < > perlbug-followup@perl.org> wrote:
> > > > On Mon May 23 14:16:31 2016, demerphq wrote:
> > > Yeah, exactly. But in xs that is pretty standard. I wouldnt rely on our > > > strings always having a trailing null.
> > > > In fact, I would like to remove that requirement (to allow substr to
> avoid copying the string), but someone else has pointed out that it is also > common in XS to pass SvPV to system functions that expect a trailing null. > > Maybe we should have a SvPV_null macro that DTRT for such cases and abandon > any requirements that pv's are null terminated.
It might have to be the other way round (SvPV guarantees a null; a new variant returns the actual string), for backward-compatibility. I have opened a separate ticket for this (#128226), since it is not directly related to the topic of this ticket. -- Father Chrysostomos
Subject: Re: [perl #128170] Assert fail in regcomp.c with no other symptoms: perl -e '/(?<=/'
From: Karl Williamson <public [...] khwilliamson.com>
CC: Perl5 Porteros <perl5-porters [...] perl.org>, bugs-bitbucket [...] rt.perl.org
To: demerphq <demerphq [...] gmail.com>
Date: Mon, 23 May 2016 22:17:29 -0600
Download (untitled) / with headers
text/plain 4.4k
On 05/23/2016 03:15 PM, demerphq wrote: Show quoted text
> > On 23 May 2016 16:06, "Karl Williamson" <public@khwilliamson.com > <mailto:public@khwilliamson.com>> wrote:
> > > > On 05/20/2016 10:32 PM, demerphq wrote:
> >> > >> > >> On 17 May 2016 3:04 p.m., "Dan Collins" <perlbug-followup@perl.org
> <mailto:perlbug-followup@perl.org>
> >> <mailto:perlbug-followup@perl.org
> <mailto:perlbug-followup@perl.org>>> wrote:
> >> > > >> > # New Ticket Created by Dan Collins > >> > # Please include the string: [perl #128170] > >> > # in the subject line of all future correspondence about this issue. > >> > # <URL: https://rt.perl.org/Ticket/Display.html?id=128170 > > >> > > >> > > >> > Greetings Porters, > >> > > >> > I have compiled bleadperl with the afl-gcc compiler using: > >> > > >> > ./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache
> >> afl-gcc' -Uuselongdouble -Duse64bitall -Doptimize=-g -Uversiononly > >> -Uman1dir -Uman3dir -Dusequadmath -des
> >> > AFL_HARDEN=1 make && make test > >> > > >> > And then fuzzed the resulting binary using: > >> > > >> > AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@ > >> > > >> > After reducing testcases using `afl-tmin` and performing additional
> >> minimization by hand, I have located the following testcase that > >> triggers an assert fail in debugging builds of the perl interpreter. The > >> testcase is the 6-character file:
> >> > > >> > /(?<=/ > >> > > >> > On normal builds, this exits with the expected error. On debug
> >> builds, this returns an assert fail.
> >> > > >> > dcollins@nightshade64:~$ perl/miniperl -e '/(?<=/' > >> > Unmatched ( in regex; marked by <-- HERE in m/( <-- HERE ?<=/ at -e
> >> line 1.
> >> > dcollins@nightshade64:~$ perldebug/miniperl -e '/(?<=/' > >> > miniperl: regcomp.c:10621: S_reg: Assertion `(pRExC_state->parse) <
> >> (pRExC_state->end)' failed.
> >> > Aborted > >> > > >> > Debugging tool output is below. A bisect was performed and points to
> >> the following diff:
> >> > > >> > cfbef7dc3bfb89e4ed2c00ea9c9e3bcfd0b170fd is the first bad commit > >> > commit cfbef7dc3bfb89e4ed2c00ea9c9e3bcfd0b170fd > >> > Author: Karl Williamson <khw@cpan.org <mailto:khw@cpan.org>
> <mailto:khw@cpan.org <mailto:khw@cpan.org>>>
> >>
> >> > Date: Wed Feb 10 16:27:13 2016 -0700 > >> > > >> > regcomp.c: Fix some parsing glitches > >> > > >> > I undertook a code review of how regcomp.c parses things in light
> >> of the
> >> > tickets found by the fuzzer, > >> > https://rt.perl.org/Ticket/Display.html?id=126546. This commit
> is the
> >> > result of my efforts so far. I was not planning to push it now,
> >> but the
> >> > work found a couple of new error messages that should be
> raised, and
> >> > this has to be done before the visible changes code freeze coming
> >> up all
> >> > too soon. I will add test cases after that freeze, including if
> >> to see
> >> > that these changes fix all the observed issues. > >> > > >> > The audit was tedious, and may have missed some things. Several
> >> issues
> >> > occurred in multiple places. One is to not advance the parse by > >> > UTF8SKIP appropriately; another is to subtract one byte from the
> >> parse
> >> > and assume that that is pointing to the beginning of the previous > >> > character (which under UTF-8 it may not). Another is to
> assume that
> >> > that the pattern is a C string, that there are no interior NULs
> >> in it.
> >> > I also found unnecessary tests, given that an SV always has a > >> > terminating NUL.
> >> > >> This is not correct. Xs code can and in practice does create sv's with > >> no trailing nulls.
> > > > > > Hmm. I've written a bunch of code with that assumption, partly
> because a bunch of code that was already in place has that assumption.
> > > > I did add assertions to this effect in the regex code at the same
> time as this commit was done.
> > > > Can you give an example of how XS could do this? All I can think of
> is if it arbitrarily turns on a POK flag. > > Yeah, exactly. But in xs that is pretty standard. I wouldnt rely on our > strings always having a trailing null. > > Yves >
I'm having trouble understanding how one would set such a thing up. At some point one would have to populate a sequence of bytes. I don't see how to populate anything but a short sequence unless the SV is already a PV. Could you outline roughly how it would be done?
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 408b
On Mon May 23 21:18:16 2016, public@khwilliamson.com wrote: Show quoted text
> I'm having trouble understanding how one would set such a thing up. > At > some point one would have to populate a sequence of bytes. I don't > see > how to populate anything but a short sequence unless the SV is already > a > PV. Could you outline roughly how it would be done?
http://grep.cpan.me/?q=SvPVX.*+%3D[+] -- Father Chrysostomos
Subject: S_reg: Assertion `(pRExC_state->parse) < (pRExC_state->end)' failed (regcomp.c:10636)
Download (untitled) / with headers
text/plain 251b
While fuzzing Perl v5.25.3 (v5.25.2-65-g3866075) with American Fuzzy Lop, it was found that perl -e '/(?<!/' triggers the following assertion failure: perl: regcomp.c:10636: S_reg: Assertion `(pRExC_state->parse) < (pRExC_state->end)' failed. Aborted
Download (untitled) / with headers
text/plain 163b
Duplicate of RT #128170, with a bisect and some discussion there. (Slightly different code - it seems both positive and negative assertions trigger this.) Merging.
Date: Tue, 5 Jul 2016 22:05:26 -0600
To: perl5-porters [...] perl.org
Subject: Re: [perl #128555] S_reg: Assertion `(pRExC_state->parse) < (pRExC_state->end)' failed (regcomp.c:10636)
From: Karl Williamson <public [...] khwilliamson.com>
Download (untitled) / with headers
text/plain 568b
On 07/05/2016 09:27 PM, Brian Carpenter (via RT) wrote: Show quoted text
> # New Ticket Created by Brian Carpenter > # Please include the string: [perl #128555] > # in the subject line of all future correspondence about this issue. > # <URL: https://rt.perl.org/Ticket/Display.html?id=128555 > > > > While fuzzing Perl v5.25.3 (v5.25.2-65-g3866075) with American Fuzzy Lop, it was found that perl -e '/(?<!/' triggers the following assertion failure: > > perl: regcomp.c:10636: S_reg: Assertion `(pRExC_state->parse) < (pRExC_state->end)' failed. > Aborted > >
I will look into it
RT-Send-CC: perl5-porters [...] perl.org
Fixed by f2e32b2c677945ce6a36531a7cd5696dc1ca2cef -- Karl Williamson
Download (untitled) / with headers
text/plain 313b
Thank you for filing this report. You have helped make Perl better. With the release today of Perl 5.26.0, this and 210 other issues have been resolved. Perl 5.26.0 may be downloaded via: https://metacpan.org/release/XSAWYERX/perl-5.26.0 If you find that the problem persists, feel free to reopen this ticket.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org