From @dur-randir

Created by @dur-randir

While fuzzing perl v5.29.8-21-gde59f38ed9 built with afl and run
under libdislocator, I found the following program (text also attached
to the ticket)

00000000 73 2c 40 30 01 7c 40 30 28 3f 7b 73 00 00 01 00 |s,@​0.|@​0(?{s....|
00000010 7d 29 2c 2c 67 |}),,g|

to cause an assertion failure​:

perl​: regexec.c​:3514​: Perl_regexec_flags​: Assertion `prog->intflags &
PREGf_GPOS_SEEN' failed.

GDB stack trace is following

#0 __GI_raise (sig=sig@​entry=6) at ../sysdeps/unix/sysv/linux/raise.c​:50
#1 0x00007ffff7c25535 in __GI_abort () at abort.c​:79
#2 0x00007ffff7c2540f in __assert_fail_base (fmt=0x7ffff7d87ee0
"%s%s%s​:%u​: %s%sAssertion `%s' failed.\n%n",
  assertion=0x5555558080e0 "prog->intflags & PREGf_GPOS_SEEN",
file=0x555555801286 "regexec.c", line=3514, function=<optimized out>)
at assert.c​:92
#3 0x00007ffff7c330f2 in __GI___assert_fail
(assertion=assertion@​entry=0x5555558080e0 "prog->intflags &
PREGf_GPOS_SEEN",
  file=file@​entry=0x555555801286 "regexec.c", line=line@​entry=3514,
  function=function@​entry=0x555555808e00 <__PRETTY_FUNCTION__.18391>
"Perl_regexec_flags") at assert.c​:101
#4 0x000055555574ca03 in Perl_regexec_flags (rx=<optimized out>,
stringarg=0x55555586a381 "", strend=<optimized out>, strbeg=<optimized
out>,
  minend=<optimized out>, sv=0x555555870e90, data=0x0, flags=152) at
inline.h​:900
#5 0x000055555567f655 in Perl_pp_subst () at inline.h​:157
#6 0x000055555564a13a in Perl_runops_debug () at dump.c​:2248
#7 0x00005555555b815e in S_run_body (oldscope=1) at perl.c​:2538
#8 perl_run (my_perl=<optimized out>) at perl.c​:2461
#9 0x0000555555584112 in main (argc=<optimized out>, argv=<optimized
out>, env=<optimized out>) at perlmain.c​:123

Bisecting it was quite funny - it's present from 5.18 onwards, but
_not_ in 5.24. This is a bisect for commit breaking it again before
5.26, which, I think, tells all the story​:

commit 5585e75
Author​: Yves Orton <demerphq@​gmail.com>
Date​: Mon Oct 31 20​:22​:37 2016 +0100

  rework perl #129903 - inf recursion from use of empty pattern in
regex codeblock

  FC didn't like my previous patch for this issue, so here is the
  one he likes better. With tests and etc. :-)

  The basic problem is that code like this​: /(?{ s!!! })/ can trigger
  infinite recursion on the C stack (not the normal perl stack) when the
  last successful pattern in scope is itself. Since the C stack overflows
  this manifests as an untrappable error/segfault, which then kills perl.

Flags:
    category=core
    severity=low

Site configuration information for perl 5.29.9:

Configured by dur-randir at Wed Feb 27 14:51:01 MSK 2019.

Summary of my perl5 (revision 5 version 29 subversion 9) configuration:
  Commit id: c1e47bad34ce1d9c84ed57c9b8978bcbd5a02e98
  Platform:
    osname=darwin
    osvers=13.4.0
    archname=darwin-thread-multi-2level
    uname='darwin isengard.local 13.4.0 darwin kernel version 13.4.0:
mon jan 11 18:17:34 pst 2016; root:xnu-2422.115.15~1release_x86_64
x86_64 '
    config_args='-de -Dusedevel -DDEBUGGING -Dusethreads'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include -DPERL_USE_SAFE_PUTENV'
    optimize='-O3 -g'
    cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -mmacosx-version-min=10.9 -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
/usr/lib
    libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=
    so=dylib
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -mmacosx-version-min=10.9 -bundle -undefined
dynamic_lookup -L/usr/local/lib -fstack-protector'



@INC for perl 5.29.9:
    lib
    /usr/local/lib/perl5/site_perl/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/site_perl/5.29.9
    /usr/local/lib/perl5/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/5.29.9


Environment for perl 5.29.9:
    DYLD_LIBRARY_PATH (unset)
    HOME=/Users/dur-randir
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin
    PERLBREW_HOME=/Users/dur-randir/.perlbrew
    PERLBREW_MANPATH=/Users/dur-randir/perlbrew/perls/perl-5.22.1/man
    PERLBREW_PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin
    PERLBREW_PERL=perl-5.22.1
    PERLBREW_ROOT=/Users/dur-randir/perlbrew
    PERLBREW_SHELLRC_VERSION=0.84
    PERLBREW_VERSION=0.84
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/zsh

From @hvds

FWIW, it doesn't need any funny characters, it reproduces just fine with eg​:
% ./miniperl -e 's{ x | (?{ s{}{x} }) }{}gx'
miniperl​: regexec.c​:3806​: Perl_regexec_flags​: Assertion `prog->intflags & 0x00000100' failed.
Aborted (core dumped)
%

Without //g, we hit a different assert​:
% ./miniperl -e 's{ x | (?{ s{}{x} }) }{}x'
miniperl​: pp_hot.c​:4283​: Perl_pp_subst​: Assertion `(((targ)->sv_flags & (0x00200000|0x00400000|0x00800000)) && Perl_mg_find(targ,'V'))' failed.
Aborted (core dumped)
%

The inner subst needs an empty match​:
% ./miniperl -e 's{ x | (?{ s{(?=)}{x} }) }{}gx'
%

The target SV needs to require upgrade to string​:
% ./miniperl -e '$x = ""; $x =~ s{ x | (?{ s{}{x} }) }{}gx'
% ./miniperl -e '$x = 0; $x =~ s{ x | (?{ s{}{x} }) }{}gx'
miniperl​: regexec.c​:3806​: Perl_regexec_flags​: Assertion `prog->intflags & 0x00000100' failed.
Aborted (core dumped)
%

I _think_ we're hitting the inner subst, finding a match, then deciding "oops, needed to upgrade the target, let's do that and try again". I'm not quite sure why we do that, but it's possible that *both* substs are remembering the need to do that, and then tripping over each other.

It's all confusing me mightily; I'll try to dig some more tomorrow, but hope that the details above might trigger a thought in someone.

Hugo

The RT System itself - Status changed from 'new' to 'open'

From @tonycoz

On Wed, Mar 27, 2019 at 12​:04​:21PM -0700, Hugo van der Sanden via RT wrote​:

FWIW, it doesn't need any funny characters, it reproduces just fine with eg​:
% ./miniperl -e 's{ x | (?{ s{}{x} }) }{}gx'
miniperl​: regexec.c​:3806​: Perl_regexec_flags​: Assertion `prog->intflags & 0x00000100' failed.
Aborted (core dumped)
%

Without //g, we hit a different assert​:
% ./miniperl -e 's{ x | (?{ s{}{x} }) }{}x'
miniperl​: pp_hot.c​:4283​: Perl_pp_subst​: Assertion `(((targ)->sv_flags & (0x00200000|0x00400000|0x00800000)) && Perl_mg_find(targ,'V'))' failed.
Aborted (core dumped)
%

The inner subst needs an empty match​:
% ./miniperl -e 's{ x | (?{ s{(?=)}{x} }) }{}gx'
%

The target SV needs to require upgrade to string​:
% ./miniperl -e '$x = ""; $x =~ s{ x | (?{ s{}{x} }) }{}gx'
% ./miniperl -e '$x = 0; $x =~ s{ x | (?{ s{}{x} }) }{}gx'
miniperl​: regexec.c​:3806​: Perl_regexec_flags​: Assertion `prog->intflags & 0x00000100' failed.
Aborted (core dumped)
%

I _think_ we're hitting the inner subst, finding a match, then deciding "oops, needed to upgrade the target, let's do that and try again". I'm not quite sure why we do that, but it's possible that *both* substs are remembering the need to do that, and then tripping over each other.

It's all confusing me mightily; I'll try to dig some more tomorrow, but hope that the details above might trigger a thought in someone.

The SV becomes SvPOKp() (and PVIV) from the SvPV_nomg() in the outer
SV, and doesn't need upgrading later.

I thought it might be because the inner subst reallocates the PV (to
fit the x in), while the parent retains a pointer to the original PV,
but valgrind doesn't complain about it.

The debugging output is confusing​:

EXECUTING...

Matching REx " x | (?{ s{}{x} }) " against "0"
  0 <> <0> | 0| 1​:BRANCH(4)
  0 <> <0> | 1| 2​:EXACT <x>(8)
  | 1| failed...
  0 <> <0> | 0| 4​:BRANCH(8)
  0 <> <0> | 1| 5​:EVAL(8)
Matching REx "" against "0"
  0 <> <0> | 0| 1​:NOTHING(2)
  0 <> <0> | 0| 2​:END(0)
Match successful!
  0 <> <0> | 2| 8​:END(0)
Match successful!
Matching REx " x | (?{ s{}{x} }) " against "0"
  0 <> <0> | 0| 1​:BRANCH(4)
  0 <> <0> | 1| 2​:EXACT <x>(8)
  | 1| failed...
  0 <> <0> | 0| 4​:BRANCH(8)
  0 <> <0> | 1| 5​:EVAL(8)
Matching REx " x | (?{ s{}{x} }) " against "x0"
  0 <> <x0> | 0| 1​:BRANCH(4)
  0 <> <x0> | 1| 2​:EXACT <x>(8)
  1 <x> <0> | 1| 8​:END(0)
Match successful!
  0 <> <0> | 2| 8​:END(0)
END​: Match possible, but length=0 is smaller than requested=1, failing!
  | 1| failed...
  | 0| BRANCH failed...
  1 <0> <> | 0| 1​:BRANCH(4)
  1 <0> <> | 1| 2​:EXACT <x>(8)
  | 1| failed...
  1 <0> <> | 0| 4​:BRANCH(8)
  1 <0> <> | 1| 5​:EVAL(8)
Matching REx " x | (?{ s{}{x} }) " against "x0"
  0 <> <x0> | 0| 1​:BRANCH(4)
  0 <> <x0> | 1| 2​:EXACT <x>(8)
  1 <x> <0> | 1| 8​:END(0)
Match successful!
  1 <0> <> | 2| 8​:END(0)
Match successful!
Matching REx " x | (?{ s{}{x} }) " against ""
  1 <0> <> | 0| 1​:BRANCH(4)
  1 <0> <> | 1| 2​:EXACT <x>(8)
  | 1| failed...
  1 <0> <> | 0| 4​:BRANCH(8)
  1 <0> <> | 1| 5​:EVAL(8)
Matching REx " x | (?{ s{}{x} }) " against "x0"
  0 <> <x0> | 0| 1​:BRANCH(4)
  0 <> <x0> | 1| 2​:EXACT <x>(8)
  1 <x> <0> | 1| 8​:END(0)
Match successful!
  1 <0> <> | 2| 8​:END(0)
Match successful!

Note that all of the inner replacements except for the first one
appear to be using the output regexp (somehow).

Tony