Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SIGSEGV in S_study_chunk - access inside an unallocated block in arena "client" #14997

Closed
p5pRT opened this issue Oct 20, 2015 · 7 comments
Closed

Comments

@p5pRT
Copy link

p5pRT commented Oct 20, 2015

Migrated from rt.perl.org#126405 (status was 'resolved')

Searchable as RT126405$

@p5pRT
Copy link
Author

p5pRT commented Oct 20, 2015

From @dcollinsn

Greetings Porters,

I have compiled bleadperl with the afl-gcc compiler using​:

./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des
AFL_HARDEN=1 make && make test

And then fuzzed the resulting binary using​:

AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @​@​

After reducing testcases using `afl-tmin` and performing additional minimization by hand, I have located the following testcase that triggers a segmentation fault in the perl interpreter. The testcase is the file​:

/(?​:.||)(?|)000000000@​/

This one comes with a valgrind message I haven't seen before, and a search of RT reveals no open tickets referencing S_study_chunk.

**GDB**

(gdb) run
Starting program​: /home/dcollins/perldebug/perl -e /\(\?​:.\|\|\)\(\?\|\)000000000@​/
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
S_study_chunk (pRExC_state=0x7fffffffd9c0, scanp=0x40, minlenp=0x12800, deltap=0x1211ee0, last=0x40, data=0x40, stopparen=-1, recursed_depth=0, flags=11264, depth=0,
  and_withp=<optimized out>) at regcomp.c​:3922
3922 && ((PL_regkind[OP(n)] == NOTHING && (noff = NEXT_OFF(n)))
(gdb) bt
#0 S_study_chunk (pRExC_state=0x7fffffffd9c0, scanp=0x40, minlenp=0x12800, deltap=0x1211ee0, last=0x40, data=0x40, stopparen=-1, recursed_depth=0, flags=11264, depth=0,
  and_withp=<optimized out>) at regcomp.c​:3922
#1 0x00000000007a53c4 in Perl_re_op_compile (patternp=0x1211fb0, pat_count=2048, expr=0x2c00, eng=0x1211ee0, old_re=0x7fffffffd9a0, is_bare_re=0xd, orig_rx_flags=0,
  pm_flags=0) at regcomp.c​:7158
#2 0x00000000004e7ec2 in Perl_pmruntime (o=0x1211e68, expr=0x1211e28, repl=0x12800, isreg=224, floor=7948352) at op.c​:5580
#3 0x000000000066f07d in Perl_yyparse (gramtype=18947688) at perly.y​:1032
#4 0x000000000053a659 in S_parse_body (env=env@​entry=0x0, xsinit=xsinit@​entry=0x42c060 <xs_init>) at perl.c​:2307
#5 0x00000000005423e3 in perl_parse (my_perl=<optimized out>, xsinit=xsinit@​entry=0x42c060 <xs_init>, argc=<optimized out>, argv=<optimized out>, env=env@​entry=0x0)
  at perl.c​:1634
#6 0x000000000042bc88 in main (argc=3, argv=0x7fffffffe338, env=0x7fffffffe358) at perlmain.c​:114
(gdb) info locals
noff = 75776
min_subtract = 0
unfolded_multi_char = false
scan = 0x1211f28
data_fake = {pos_min = 0, pos_delta = 0, last_found = 0x0, last_end = 0, last_start_min = 0, last_start_max = 0, longest = 0x0, longest_fixed = 0x0, offset_fixed = 0,
  minlen_fixed = 0x0, lookbehind_fixed = 0, longest_float = 0x0, offset_float_min = 0, offset_float_max = 0, minlen_float = 0x0, lookbehind_float = 0, flags = 0,
  whilem_c = 0, last_closep = 0x7fffffffd9a0, start_class = 0x7fffffffd800}
re_debug_flags = 0

**VALGRIND**

==41612== Memcheck, a memory error detector
==41612== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==41612== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==41612== Command​: /home/dcollins/perldebug/perl -e /(?​:.||)(?|)000000000@​/
==41612==
==41612== Invalid read of size 1
==41612== at 0x761CA4​: S_study_chunk.constprop.51 (regcomp.c​:3922)
==41612== by 0x7A53C3​: Perl_re_op_compile (regcomp.c​:7158)
==41612== by 0x4E7EC1​: Perl_pmruntime (op.c​:5580)
==41612== by 0x66F07C​: Perl_yyparse (perly.y​:1032)
==41612== by 0x53A658​: S_parse_body (perl.c​:2307)
==41612== by 0x5423E2​: perl_parse (perl.c​:1634)
==41612== by 0x42BC87​: main (perlmain.c​:114)
==41612== Address 0x5fc67a9 is 293,625 bytes inside an unallocated block of size 4,002,608 in arena "client"
==41612==
==41612==
==41612== HEAP SUMMARY​:
==41612== in use at exit​: 118,228 bytes in 572 blocks
==41612== total heap usage​: 772 allocs, 200 frees, 143,760 bytes allocated
==41612==
==41612== LEAK SUMMARY​:
==41612== definitely lost​: 0 bytes in 0 blocks
==41612== indirectly lost​: 0 bytes in 0 blocks
==41612== possibly lost​: 0 bytes in 0 blocks
==41612== still reachable​: 118,228 bytes in 572 blocks
==41612== suppressed​: 0 bytes in 0 blocks
==41612== Rerun with --leak-check=full to see details of leaked memory
==41612==
==41612== For counts of detected and suppressed errors, rerun with​: -v
==41612== ERROR SUMMARY​: 1 errors from 1 contexts (suppressed​: 0 from 0)

**PERL -V**

Summary of my perl5 (revision 5 version 23 subversion 4) configuration​:
  Commit id​: a7dba6f
  Platform​:
  osname=linux, osvers=3.16.0-4-amd64, archname=x86_64-linux-ld
  uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux '
  config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DDEBUG_LEAKING_SCALARS -des'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  use64bitint=define, use64bitall=define, uselongdouble=define
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='ccache afl-gcc', ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-g',
  cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion='', gccversion='4.9.2', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
  ivtype='long', ivsize=8, nvtype='long double', nvsize=16, Off_t='off_t', lseeksize=8
  alignbytes=16, prototype=define
  Linker and Libraries​:
  ld='ccache afl-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.19'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
  PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL
  USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE
  USE_LOCALE_COLLATE USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE
  USE_PERLIO USE_PERL_ATOF
  Built under linux
  Compiled at Oct 15 2015 20​:45​:21
  @​INC​:
  /usr/local/perl-afl/lib/site_perl/5.23.4/x86_64-linux-ld
  /usr/local/perl-afl/lib/site_perl/5.23.4
  /usr/local/perl-afl/lib/5.23.4/x86_64-linux-ld
  /usr/local/perl-afl/lib/5.23.4
  .

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2015

From @khwilliamson

On 10/19/2015 08​:30 PM, Dan Collins (via RT) wrote​:

# New Ticket Created by Dan Collins
# Please include the string​: [perl #126405]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=126405 >

Greetings Porters,

I have compiled bleadperl with the afl-gcc compiler using​:

./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des
AFL_HARDEN=1 make && make test

And then fuzzed the resulting binary using​:

AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @​@​

After reducing testcases using `afl-tmin` and performing additional minimization by hand, I have located the following testcase that triggers a segmentation fault in the perl interpreter. The testcase is the file​:

/(?​:.||)(?|)000000000@​/

This one comes with a valgrind message I haven't seen before, and a search of RT reveals no open tickets referencing S_study_chunk.

I have bisected this to the following commit

3018b82 is the first bad commit
commit 3018b82
Author​: Karl Williamson <public@​khwilliamson.com>
Date​: Mon Dec 17 21​:37​:40 2012 -0700

  Consolidate some regex OPS

  The regular rexpression operation POSIXA works on any of the
(currently)
  16 posix classes (like \w and [​:graph​:]) under the regex modifier /a.
  This commit creates similar operations for the other modifiers​: POSIXL
  (for /l), POSIXD (for /d), POSIXU (for /u), plus their complements.

  It causes these ops to be generated instead of the ALNUM, DIGIT,
  HORIZWS, SPACE, and VERTWS ops, as well as all their variants. The net
  saving is 22 regnode types.

  The reason to do this is for maintenance. As of this commit, there are
  now 22 fewer node types for which code has to be maintained. The code
  for each variant was essentially the same logic, but on different
  operands. It would be easy to make a change to one copy and forget to
  make the corresponding change in the others. Indeed, this patch fixes
  [perl #114272] in which one copy was out of sync with others.

  This patch actually reduces the number of separate code paths to 5​:
  POSIXA, NPOSIXA, POSIXL, POSIXD, and POSIXU. The complements of the
  last 3 use the same code path as their non-complemented version, except
  that a variable is initialized differently. The code then XORs this
  variable with its result to do the complementing or not. Further, the
  POSIXD branch now just checks if the target string being matched is
  UTF-8 or not, and then jumps to either the POSIXU or POSIXA code
  respectively. So, there are effectively only 4 cases that are coded​:
  POSIXA, NPOSIXA, POSIXL, and POSIXU. (POSIXA doesn't have to worry
  about UTF-8, while NPOSIXA does, hence these for efficiency are coded
  separately.)

  Removing all this code saves memory. The output of the Linux size
  command shows that the perl executable was shrunk by 33K bytes on my
  platform compiled under -O0 (.7%) and by 18K bytes (1.3%) under -O2.

  The reason this patch was doable was previous work in numbering the
  POSIX classes, so that they could be indexed in arrays and bit
  positions. This is a large patch; I didn't see how to break it into
  smaller components.

  I chose to make this code more efficient as opposed to saving even more
  memory. Thus there is a separate loop that is jumped to after we know
  we have to load a swash; this just saves having to test if the swash is
  loaded each time through the loop. I avoid loading the swash until
  absolutely necessary. In places in the previous version of this code,
  the swash was loaded when the input was UTF-8, even if it wasn't yet
  needed (and might never be if the input didn't contain anything above
  Latin1); apparently to avoid the extra test per iteration.

  The Perl test suite runs slightly faster on my platform with this patch
  under -O0, and the speeds are indistinguishable under -O2. This is in
  spite of these new POSIX regops being unknown to the regex optimizer
  (this will be addressed in future commits), and extra machine
  instructions being required for each character (the xor, and some
  shifting and masking). I expect this is a result of better
caching, and
  not loading swashes unless absolutely necessary.

**GDB**

(gdb) run
Starting program​: /home/dcollins/perldebug/perl -e /\(\?​:.\|\|\)\(\?\|\)000000000@​/
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
S_study_chunk (pRExC_state=0x7fffffffd9c0, scanp=0x40, minlenp=0x12800, deltap=0x1211ee0, last=0x40, data=0x40, stopparen=-1, recursed_depth=0, flags=11264, depth=0,
and_withp=<optimized out>) at regcomp.c​:3922
3922 && ((PL_regkind[OP(n)] == NOTHING && (noff = NEXT_OFF(n)))
(gdb) bt
#0 S_study_chunk (pRExC_state=0x7fffffffd9c0, scanp=0x40, minlenp=0x12800, deltap=0x1211ee0, last=0x40, data=0x40, stopparen=-1, recursed_depth=0, flags=11264, depth=0,
and_withp=<optimized out>) at regcomp.c​:3922
#1 0x00000000007a53c4 in Perl_re_op_compile (patternp=0x1211fb0, pat_count=2048, expr=0x2c00, eng=0x1211ee0, old_re=0x7fffffffd9a0, is_bare_re=0xd, orig_rx_flags=0,
pm_flags=0) at regcomp.c​:7158
#2 0x00000000004e7ec2 in Perl_pmruntime (o=0x1211e68, expr=0x1211e28, repl=0x12800, isreg=224, floor=7948352) at op.c​:5580
#3 0x000000000066f07d in Perl_yyparse (gramtype=18947688) at perly.y​:1032
#4 0x000000000053a659 in S_parse_body (env=env@​entry=0x0, xsinit=xsinit@​entry=0x42c060 <xs_init>) at perl.c​:2307
#5 0x00000000005423e3 in perl_parse (my_perl=<optimized out>, xsinit=xsinit@​entry=0x42c060 <xs_init>, argc=<optimized out>, argv=<optimized out>, env=env@​entry=0x0)
at perl.c​:1634
#6 0x000000000042bc88 in main (argc=3, argv=0x7fffffffe338, env=0x7fffffffe358) at perlmain.c​:114
(gdb) info locals
noff = 75776
min_subtract = 0
unfolded_multi_char = false
scan = 0x1211f28
data_fake = {pos_min = 0, pos_delta = 0, last_found = 0x0, last_end = 0, last_start_min = 0, last_start_max = 0, longest = 0x0, longest_fixed = 0x0, offset_fixed = 0,
minlen_fixed = 0x0, lookbehind_fixed = 0, longest_float = 0x0, offset_float_min = 0, offset_float_max = 0, minlen_float = 0x0, lookbehind_float = 0, flags = 0,
whilem_c = 0, last_closep = 0x7fffffffd9a0, start_class = 0x7fffffffd800}
re_debug_flags = 0

**VALGRIND**

==41612== Memcheck, a memory error detector
==41612== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==41612== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==41612== Command​: /home/dcollins/perldebug/perl -e /(?​:.||)(?|)000000000@​/
==41612==
==41612== Invalid read of size 1
==41612== at 0x761CA4​: S_study_chunk.constprop.51 (regcomp.c​:3922)
==41612== by 0x7A53C3​: Perl_re_op_compile (regcomp.c​:7158)
==41612== by 0x4E7EC1​: Perl_pmruntime (op.c​:5580)
==41612== by 0x66F07C​: Perl_yyparse (perly.y​:1032)
==41612== by 0x53A658​: S_parse_body (perl.c​:2307)
==41612== by 0x5423E2​: perl_parse (perl.c​:1634)
==41612== by 0x42BC87​: main (perlmain.c​:114)
==41612== Address 0x5fc67a9 is 293,625 bytes inside an unallocated block of size 4,002,608 in arena "client"
==41612==
==41612==
==41612== HEAP SUMMARY​:
==41612== in use at exit​: 118,228 bytes in 572 blocks
==41612== total heap usage​: 772 allocs, 200 frees, 143,760 bytes allocated
==41612==
==41612== LEAK SUMMARY​:
==41612== definitely lost​: 0 bytes in 0 blocks
==41612== indirectly lost​: 0 bytes in 0 blocks
==41612== possibly lost​: 0 bytes in 0 blocks
==41612== still reachable​: 118,228 bytes in 572 blocks
==41612== suppressed​: 0 bytes in 0 blocks
==41612== Rerun with --leak-check=full to see details of leaked memory
==41612==
==41612== For counts of detected and suppressed errors, rerun with​: -v
==41612== ERROR SUMMARY​: 1 errors from 1 contexts (suppressed​: 0 from 0)

**PERL -V**

Summary of my perl5 (revision 5 version 23 subversion 4) configuration​:
Commit id​: a7dba6f
Platform​:
osname=linux, osvers=3.16.0-4-amd64, archname=x86_64-linux-ld
uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux '
config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DDEBUG_LEAKING_SCALARS -des'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=define
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='ccache afl-gcc', ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-g',
cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion='', gccversion='4.9.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
ivtype='long', ivsize=8, nvtype='long double', nvsize=16, Off_t='off_t', lseeksize=8
alignbytes=16, prototype=define
Linker and Libraries​:
ld='ccache afl-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.19'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
Compile-time options​: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL
USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE
USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE
USE_PERLIO USE_PERL_ATOF
Built under linux
Compiled at Oct 15 2015 20​:45​:21
@​INC​:
/usr/local/perl-afl/lib/site_perl/5.23.4/x86_64-linux-ld
/usr/local/perl-afl/lib/site_perl/5.23.4
/usr/local/perl-afl/lib/5.23.4/x86_64-linux-ld
/usr/local/perl-afl/lib/5.23.4
.

I

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Mar 7, 2016

From @khwilliamson

Thanks for reporting this.

Fixed by b297756
and tested for in 80edda0
--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented Mar 7, 2016

@khwilliamson - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented May 13, 2016

From @khwilliamson

Thank you for submitting this report. You have helped make Perl better.
 
With the release of Perl 5.24.0 on May 9, 2016, this and 149 other issues have been resolved.

Perl 5.24.0 may be downloaded via https://metacpan.org/release/RJBS/perl-5.24.0

@p5pRT
Copy link
Author

p5pRT commented May 13, 2016

@khwilliamson - Status changed from 'pending release' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant