From @dur-randir

Created by @dur-randir

While fuzzing perl v5.29.8-21-gde59f38ed9 built with afl and run
under libdislocator, I found the program attached to this message to crash

GDB stack trace is following​:

(gdb) bt
#0 0x000055555566f938 in S_make_trie (pRExC_state=0x7fffffffd6b0,
startbranch=0x555555c795cc, first=0x555555c795cc, last=0x555555c795e0,
  tail=0x555555cb95dc, word_count=2, flags=35, depth=1) at regcomp.c​:2759
#1 0x000055555567d599 in S_study_chunk (pRExC_state=0x7fffffffd6b0,
scanp=0x7fffffffd438, minlenp=0x7fffffffd440, deltap=0x7fffffffd460,
  last=0x555555cb95e0, data=0x7fffffffda00, stopparen=-1,
recursed_depth=0, and_withp=0x555555b69f30, flags=12288, depth=0) at
regcomp.c​:4945
#2 0x000055555568cd31 in Perl_re_op_compile (patternp=0x0,
pat_count=2, expr=0x555555b59d90, eng=0x555555b21d20
<PL_core_reg_engine>, old_re=0x0,
  is_bare_re=0x0, orig_rx_flags=4, pm_flags=4) at regcomp.c​:8136
#3 0x00005555555a7081 in Perl_pmruntime (o=0x555555b597f8,
expr=0x555555b59d90, repl=0x555555b59738, flags=1, floor=0) at
op.c​:7126
#4 0x000055555565d546 in Perl_yyparse (gramtype=258) at perly.y​:1228
#5 0x00005555555d930d in S_parse_body (env=0x0, xsinit=0x55555558e1e8
<xs_init>) at perl.c​:2507
#6 0x00005555555d75df in perl_parse (my_perl=0x555555b2c260,
xsinit=0x55555558e1e8 <xs_init>, argc=2, argv=0x7fffffffe1f8, env=0x0)
at perl.c​:1798
#7 0x000055555558e12b in main (argc=2, argv=0x7fffffffe1f8,
env=0x7fffffffe210) at perlmain.c​:126

This is a regression in blead, bisect points to

commit 7c932d0
Author​: Karl Williamson <khw@​cpan.org>
Date​: Fri Oct 19 09​:48​:34 2018 -0600

  Remove sizing pass from regular expression compiler

  This commit removes the sizing pass for regular expression compilation.
  It attempts to be the minimum required to do this. Future patches are
  in the works that improve it,, and there is certainly lots more that
  could be done.

So, this may be related to https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133871.

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.29.9:

Configured by dur-randir at Wed Feb 27 14:51:01 MSK 2019.

Summary of my perl5 (revision 5 version 29 subversion 9) configuration:
  Commit id: c1e47bad34ce1d9c84ed57c9b8978bcbd5a02e98
  Platform:
    osname=darwin
    osvers=13.4.0
    archname=darwin-thread-multi-2level
    uname='darwin isengard.local 13.4.0 darwin kernel version 13.4.0:
mon jan 11 18:17:34 pst 2016; root:xnu-2422.115.15~1release_x86_64
x86_64 '
    config_args='-de -Dusedevel -DDEBUGGING -Dusethreads'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include -DPERL_USE_SAFE_PUTENV'
    optimize='-O3 -g'
    cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -mmacosx-version-min=10.9 -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
/usr/lib
    libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=
    so=dylib
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -mmacosx-version-min=10.9 -bundle -undefined
dynamic_lookup -L/usr/local/lib -fstack-protector'



@INC for perl 5.29.9:
    lib
    /usr/local/lib/perl5/site_perl/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/site_perl/5.29.9
    /usr/local/lib/perl5/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/5.29.9


Environment for perl 5.29.9:
    DYLD_LIBRARY_PATH (unset)
    HOME=/Users/dur-randir
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin
    PERLBREW_HOME=/Users/dur-randir/.perlbrew
    PERLBREW_MANPATH=/Users/dur-randir/perlbrew/perls/perl-5.22.1/man
    PERLBREW_PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin
    PERLBREW_PERL=perl-5.22.1
    PERLBREW_ROOT=/Users/dur-randir/perlbrew
    PERLBREW_SHELLRC_VERSION=0.84
    PERLBREW_VERSION=0.84
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/zsh

From @dur-randir

0113

From @dur-randir

Very similar case, but hitting another assertion​:

perl​: regcomp.c​:4444​: ssize_t S_study_chunk(RExC_state_t *, regnode **, ssize_t *, ssize_t *, regnode *, scan_data_t *, I32, U32, regnode_ssc *, U32, U32)​: Assertion `last' failed.

From @dur-randir

0113_1

From @khwilliamson

Fixed by
commit bfa9f5e
Author​: Karl Williamson <khw@​cpan.org>
Date​: Wed Mar 13 21​:25​:05 2019 -0600

  PATCH​: [perl #133921] Segfaults in regcomp.c
 
  If a regular expression pattern gets too long so that the branch
  instructions need more bits than are available, it is supposed to
  reparse and use long jumps instead of the normal ones that don't take up
  extra room. The blamed commit caused the test for this to be done too
  late. This just moves the test to do it in time; lexically later in the
  file, but just after the variable takes on the too-large value and
  before it gets used in the next loop iteration.

--
Karl Williamson

The RT System itself - Status changed from 'new' to 'open'

@khwilliamson - Status changed from 'open' to 'resolved'

From @dur-randir

The attached file still fails with the following trace​:

(gdb) bt
#0 __GI_raise (sig=sig@​entry=6) at ../sysdeps/unix/sysv/linux/raise.c​:50
#1 0x00007ffff7c25535 in __GI_abort () at abort.c​:79
#2 0x00007ffff7c2540f in __assert_fail_base (fmt=0x7ffff7d87ee0 "%s%s%s​:%u​: %s%sAssertion `%s' failed.\n%n", assertion=0x555555a7df9c "last",
  file=0x555555a7d850 "regcomp.c", line=4455, function=<optimized out>) at assert.c​:92
#3 0x00007ffff7c330f2 in __GI___assert_fail (assertion=0x555555a7df9c "last", file=0x555555a7d850 "regcomp.c", line=4455,
  function=0x555555a9ce38 <__PRETTY_FUNCTION__.21795> "S_study_chunk") at assert.c​:101
#4 0x000055555568eea2 in S_study_chunk (pRExC_state=0x7fffffffd6a0, scanp=0x7fffffffcf08, minlenp=0x7fffffffd430, deltap=0x7fffffffcf28, last=0x0,
  data=0x7fffffffd290, stopparen=-1, recursed_depth=0, and_withp=0x0, flags=10240, depth=1) at regcomp.c​:4455
#5 0x000055555568f7a0 in S_study_chunk (pRExC_state=0x7fffffffd6a0, scanp=0x7fffffffd428, minlenp=0x7fffffffd430, deltap=0x7fffffffd450,
  last=0x555555d16f18, data=0x7fffffffd9f0, stopparen=-1, recursed_depth=0, and_withp=0x0, flags=10240, depth=0) at regcomp.c​:4635
#6 0x000055555569fdcc in Perl_re_op_compile (patternp=0x0, pat_count=2, expr=0x555555b7bf20, eng=0x555555b43d20 <PL_core_reg_engine>, old_re=0x0,
  is_bare_re=0x0, orig_rx_flags=4, pm_flags=4) at regcomp.c​:8148
#7 0x00005555555ba0be in Perl_pmruntime (o=0x555555b7b818, expr=0x555555b7bf20, repl=0x555555b7b758, flags=1, floor=0) at op.c​:7131
#8 0x00005555556705e4 in Perl_yyparse (gramtype=258) at perly.y​:1228
#9 0x00005555555ec359 in S_parse_body (env=0x0, xsinit=0x5555555a11f8 <xs_init>) at perl.c​:2507
#10 0x00005555555ea62b in perl_parse (my_perl=0x555555b4e260, xsinit=0x5555555a11f8 <xs_init>, argc=2, argv=0x7fffffffe1e8, env=0x0) at perl.c​:1798
#11 0x00005555555a113b in main (argc=2, argv=0x7fffffffe1e8, env=0x7fffffffe200) at perlmain.c​:126

From @dur-randir

0129_1

From @dur-randir

Here's the original that was minimized to 0120_1 in this ticket and 0160 in #133933, but which itself fails with different assertion - regexec.c​:7812​: ssize_t S_regmatch(regmatch_info *, char *, regnode *)​: Assertion `cur_curlyx' failed.

Unfortunately, I don't have time to minimize it to exactly cur_curlyx failure, but maybe it'll go away when you fix others.

From @dur-randir

014105

From @khwilliamson

On 3/14/19 3​:17 PM, Sergey Aleynikov via RT wrote​:

Here's the original that was minimized to 0120_1 in this ticket and 0160 in #133933, but which itself fails with different assertion - regexec.c​:7812​: ssize_t S_regmatch(regmatch_info *, char *, regnode *)​: Assertion `cur_curlyx' failed.

Unfortunately, I don't have time to minimize it to exactly cur_curlyx failure, but maybe it'll go away when you fix others.

---
via perlbug​: queue​: perl5 status​: resolved
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133921

Both of your recent files now work (and have been added to pat.t) by
this commit​:

commit bf848a1
  Author​: Karl Williamson <khw@​cpan.org>
  Date​: Thu Mar 14 16​:46​:50 2019 -0600

  Add more checking for regnode offset overflowing

  This is part of the ongoing failures in [perl #133921].

  The bottom line cause is that there are generally 16 bits
available for
  the address of the next regnode. On very large patterns, this may not
  be enough. When that happens, a long jump is used instead.

  What previous commits have done is to insert tests in a loop to detect
  that overflow isn't going to occur. But it turns out that there are
  other places where such overflow could occur. The real solution
should
  be to detect overflow in the base level routine that would
otherwise get
  things wrong. This entails making that routine be able to return
  failure. It turns out that another function is used under
DEBUGGING, so
  that one must be changed as well. And the calls where it is possible
  for this to overflow are changed to look for failure return and
proceed
  appropriately, which is to set a flag that we need to use long jumps,
  and restart the parse.