Skip Menu |
Report information
Id: 126405
Status: resolved
Priority: 0/
Queue: perl5

Owner: khw <khw [at] cpan.org>
Requestors: dcollinsn [at] gmail.com
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: (no value)



Subject: SIGSEGV in S_study_chunk - access inside an unallocated block in arena "client"
Download (untitled) / with headers
text/plain 7.1k
Greetings Porters, I have compiled bleadperl with the afl-gcc compiler using: ./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des AFL_HARDEN=1 make && make test And then fuzzed the resulting binary using: AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@ After reducing testcases using `afl-tmin` and performing additional minimization by hand, I have located the following testcase that triggers a segmentation fault in the perl interpreter. The testcase is the file: /(?:.||)(?|)000000000@/ This one comes with a valgrind message I haven't seen before, and a search of RT reveals no open tickets referencing S_study_chunk. **GDB** (gdb) run Starting program: /home/dcollins/perldebug/perl -e /\(\?:.\|\|\)\(\?\|\)000000000@/ [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. S_study_chunk (pRExC_state=0x7fffffffd9c0, scanp=0x40, minlenp=0x12800, deltap=0x1211ee0, last=0x40, data=0x40, stopparen=-1, recursed_depth=0, flags=11264, depth=0, and_withp=<optimized out>) at regcomp.c:3922 3922 && ((PL_regkind[OP(n)] == NOTHING && (noff = NEXT_OFF(n))) (gdb) bt #0 S_study_chunk (pRExC_state=0x7fffffffd9c0, scanp=0x40, minlenp=0x12800, deltap=0x1211ee0, last=0x40, data=0x40, stopparen=-1, recursed_depth=0, flags=11264, depth=0, and_withp=<optimized out>) at regcomp.c:3922 #1 0x00000000007a53c4 in Perl_re_op_compile (patternp=0x1211fb0, pat_count=2048, expr=0x2c00, eng=0x1211ee0, old_re=0x7fffffffd9a0, is_bare_re=0xd, orig_rx_flags=0, pm_flags=0) at regcomp.c:7158 #2 0x00000000004e7ec2 in Perl_pmruntime (o=0x1211e68, expr=0x1211e28, repl=0x12800, isreg=224, floor=7948352) at op.c:5580 #3 0x000000000066f07d in Perl_yyparse (gramtype=18947688) at perly.y:1032 #4 0x000000000053a659 in S_parse_body (env=env@entry=0x0, xsinit=xsinit@entry=0x42c060 <xs_init>) at perl.c:2307 #5 0x00000000005423e3 in perl_parse (my_perl=<optimized out>, xsinit=xsinit@entry=0x42c060 <xs_init>, argc=<optimized out>, argv=<optimized out>, env=env@entry=0x0) at perl.c:1634 #6 0x000000000042bc88 in main (argc=3, argv=0x7fffffffe338, env=0x7fffffffe358) at perlmain.c:114 (gdb) info locals noff = 75776 min_subtract = 0 unfolded_multi_char = false scan = 0x1211f28 data_fake = {pos_min = 0, pos_delta = 0, last_found = 0x0, last_end = 0, last_start_min = 0, last_start_max = 0, longest = 0x0, longest_fixed = 0x0, offset_fixed = 0, minlen_fixed = 0x0, lookbehind_fixed = 0, longest_float = 0x0, offset_float_min = 0, offset_float_max = 0, minlen_float = 0x0, lookbehind_float = 0, flags = 0, whilem_c = 0, last_closep = 0x7fffffffd9a0, start_class = 0x7fffffffd800} re_debug_flags = 0 **VALGRIND** ==41612== Memcheck, a memory error detector ==41612== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==41612== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info ==41612== Command: /home/dcollins/perldebug/perl -e /(?:.||)(?|)000000000@/ ==41612== ==41612== Invalid read of size 1 ==41612== at 0x761CA4: S_study_chunk.constprop.51 (regcomp.c:3922) ==41612== by 0x7A53C3: Perl_re_op_compile (regcomp.c:7158) ==41612== by 0x4E7EC1: Perl_pmruntime (op.c:5580) ==41612== by 0x66F07C: Perl_yyparse (perly.y:1032) ==41612== by 0x53A658: S_parse_body (perl.c:2307) ==41612== by 0x5423E2: perl_parse (perl.c:1634) ==41612== by 0x42BC87: main (perlmain.c:114) ==41612== Address 0x5fc67a9 is 293,625 bytes inside an unallocated block of size 4,002,608 in arena "client" ==41612== ==41612== ==41612== HEAP SUMMARY: ==41612== in use at exit: 118,228 bytes in 572 blocks ==41612== total heap usage: 772 allocs, 200 frees, 143,760 bytes allocated ==41612== ==41612== LEAK SUMMARY: ==41612== definitely lost: 0 bytes in 0 blocks ==41612== indirectly lost: 0 bytes in 0 blocks ==41612== possibly lost: 0 bytes in 0 blocks ==41612== still reachable: 118,228 bytes in 572 blocks ==41612== suppressed: 0 bytes in 0 blocks ==41612== Rerun with --leak-check=full to see details of leaked memory ==41612== ==41612== For counts of detected and suppressed errors, rerun with: -v ==41612== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) **PERL -V** Summary of my perl5 (revision 5 version 23 subversion 4) configuration: Commit id: a7dba6f870a82e3be87bda50593cb5d2042277fd Platform: osname=linux, osvers=3.16.0-4-amd64, archname=x86_64-linux-ld uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux ' config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DDEBUG_LEAKING_SCALARS -des' hint=recommended, useposix=true, d_sigaction=define useithreads=undef, usemultiplicity=undef use64bitint=define, use64bitall=define, uselongdouble=define usemymalloc=n, bincompat5005=undef Compiler: cc='ccache afl-gcc', ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-g', cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='', gccversion='4.9.2', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3 ivtype='long', ivsize=8, nvtype='long double', nvsize=16, Off_t='off_t', lseeksize=8 alignbytes=16, prototype=define Linker and Libraries: ld='ccache afl-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong' Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Oct 15 2015 20:45:21 @INC: /usr/local/perl-afl/lib/site_perl/5.23.4/x86_64-linux-ld /usr/local/perl-afl/lib/site_perl/5.23.4 /usr/local/perl-afl/lib/5.23.4/x86_64-linux-ld /usr/local/perl-afl/lib/5.23.4 .
From: Karl Williamson <public [...] khwilliamson.com>
To: perl5-porters [...] perl.org
Subject: Re: [perl #126405] SIGSEGV in S_study_chunk - access inside an unallocated block in arena "client"
Date: Sun, 15 Nov 2015 10:24:35 -0700
Download (untitled) / with headers
text/plain 11.1k
On 10/19/2015 08:30 PM, Dan Collins (via RT) wrote: Show quoted text
> # New Ticket Created by Dan Collins > # Please include the string: [perl #126405] > # in the subject line of all future correspondence about this issue. > # <URL: https://rt.perl.org/Ticket/Display.html?id=126405 > > > > Greetings Porters, > > I have compiled bleadperl with the afl-gcc compiler using: > > ./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -des > AFL_HARDEN=1 make && make test > > And then fuzzed the resulting binary using: > > AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@ > > After reducing testcases using `afl-tmin` and performing additional minimization by hand, I have located the following testcase that triggers a segmentation fault in the perl interpreter. The testcase is the file: > > /(?:.||)(?|)000000000@/ > > This one comes with a valgrind message I haven't seen before, and a search of RT reveals no open tickets referencing S_study_chunk. >
I have bisected this to the following commit 3018b823898645e44b8c37c70ac5c6302b031381 is the first bad commit commit 3018b823898645e44b8c37c70ac5c6302b031381 Author: Karl Williamson <public@khwilliamson.com> Date: Mon Dec 17 21:37:40 2012 -0700 Consolidate some regex OPS The regular rexpression operation POSIXA works on any of the (currently) 16 posix classes (like \w and [:graph:]) under the regex modifier /a. This commit creates similar operations for the other modifiers: POSIXL (for /l), POSIXD (for /d), POSIXU (for /u), plus their complements. It causes these ops to be generated instead of the ALNUM, DIGIT, HORIZWS, SPACE, and VERTWS ops, as well as all their variants. The net saving is 22 regnode types. The reason to do this is for maintenance. As of this commit, there are now 22 fewer node types for which code has to be maintained. The code for each variant was essentially the same logic, but on different operands. It would be easy to make a change to one copy and forget to make the corresponding change in the others. Indeed, this patch fixes [perl #114272] in which one copy was out of sync with others. This patch actually reduces the number of separate code paths to 5: POSIXA, NPOSIXA, POSIXL, POSIXD, and POSIXU. The complements of the last 3 use the same code path as their non-complemented version, except that a variable is initialized differently. The code then XORs this variable with its result to do the complementing or not. Further, the POSIXD branch now just checks if the target string being matched is UTF-8 or not, and then jumps to either the POSIXU or POSIXA code respectively. So, there are effectively only 4 cases that are coded: POSIXA, NPOSIXA, POSIXL, and POSIXU. (POSIXA doesn't have to worry about UTF-8, while NPOSIXA does, hence these for efficiency are coded separately.) Removing all this code saves memory. The output of the Linux size command shows that the perl executable was shrunk by 33K bytes on my platform compiled under -O0 (.7%) and by 18K bytes (1.3%) under -O2. The reason this patch was doable was previous work in numbering the POSIX classes, so that they could be indexed in arrays and bit positions. This is a large patch; I didn't see how to break it into smaller components. I chose to make this code more efficient as opposed to saving even more memory. Thus there is a separate loop that is jumped to after we know we have to load a swash; this just saves having to test if the swash is loaded each time through the loop. I avoid loading the swash until absolutely necessary. In places in the previous version of this code, the swash was loaded when the input was UTF-8, even if it wasn't yet needed (and might never be if the input didn't contain anything above Latin1); apparently to avoid the extra test per iteration. The Perl test suite runs slightly faster on my platform with this patch under -O0, and the speeds are indistinguishable under -O2. This is in spite of these new POSIX regops being unknown to the regex optimizer (this will be addressed in future commits), and extra machine instructions being required for each character (the xor, and some shifting and masking). I expect this is a result of better caching, and not loading swashes unless absolutely necessary. Show quoted text
> **GDB** > > (gdb) run > Starting program: /home/dcollins/perldebug/perl -e /\(\?:.\|\|\)\(\?\|\)000000000@/ > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > > Program received signal SIGSEGV, Segmentation fault. > S_study_chunk (pRExC_state=0x7fffffffd9c0, scanp=0x40, minlenp=0x12800, deltap=0x1211ee0, last=0x40, data=0x40, stopparen=-1, recursed_depth=0, flags=11264, depth=0, > and_withp=<optimized out>) at regcomp.c:3922 > 3922 && ((PL_regkind[OP(n)] == NOTHING && (noff = NEXT_OFF(n))) > (gdb) bt > #0 S_study_chunk (pRExC_state=0x7fffffffd9c0, scanp=0x40, minlenp=0x12800, deltap=0x1211ee0, last=0x40, data=0x40, stopparen=-1, recursed_depth=0, flags=11264, depth=0, > and_withp=<optimized out>) at regcomp.c:3922 > #1 0x00000000007a53c4 in Perl_re_op_compile (patternp=0x1211fb0, pat_count=2048, expr=0x2c00, eng=0x1211ee0, old_re=0x7fffffffd9a0, is_bare_re=0xd, orig_rx_flags=0, > pm_flags=0) at regcomp.c:7158 > #2 0x00000000004e7ec2 in Perl_pmruntime (o=0x1211e68, expr=0x1211e28, repl=0x12800, isreg=224, floor=7948352) at op.c:5580 > #3 0x000000000066f07d in Perl_yyparse (gramtype=18947688) at perly.y:1032 > #4 0x000000000053a659 in S_parse_body (env=env@entry=0x0, xsinit=xsinit@entry=0x42c060 <xs_init>) at perl.c:2307 > #5 0x00000000005423e3 in perl_parse (my_perl=<optimized out>, xsinit=xsinit@entry=0x42c060 <xs_init>, argc=<optimized out>, argv=<optimized out>, env=env@entry=0x0) > at perl.c:1634 > #6 0x000000000042bc88 in main (argc=3, argv=0x7fffffffe338, env=0x7fffffffe358) at perlmain.c:114 > (gdb) info locals > noff = 75776 > min_subtract = 0 > unfolded_multi_char = false > scan = 0x1211f28 > data_fake = {pos_min = 0, pos_delta = 0, last_found = 0x0, last_end = 0, last_start_min = 0, last_start_max = 0, longest = 0x0, longest_fixed = 0x0, offset_fixed = 0, > minlen_fixed = 0x0, lookbehind_fixed = 0, longest_float = 0x0, offset_float_min = 0, offset_float_max = 0, minlen_float = 0x0, lookbehind_float = 0, flags = 0, > whilem_c = 0, last_closep = 0x7fffffffd9a0, start_class = 0x7fffffffd800} > re_debug_flags = 0 > > > **VALGRIND** > > ==41612== Memcheck, a memory error detector > ==41612== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. > ==41612== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info > ==41612== Command: /home/dcollins/perldebug/perl -e /(?:.||)(?|)000000000@/ > ==41612== > ==41612== Invalid read of size 1 > ==41612== at 0x761CA4: S_study_chunk.constprop.51 (regcomp.c:3922) > ==41612== by 0x7A53C3: Perl_re_op_compile (regcomp.c:7158) > ==41612== by 0x4E7EC1: Perl_pmruntime (op.c:5580) > ==41612== by 0x66F07C: Perl_yyparse (perly.y:1032) > ==41612== by 0x53A658: S_parse_body (perl.c:2307) > ==41612== by 0x5423E2: perl_parse (perl.c:1634) > ==41612== by 0x42BC87: main (perlmain.c:114) > ==41612== Address 0x5fc67a9 is 293,625 bytes inside an unallocated block of size 4,002,608 in arena "client" > ==41612== > ==41612== > ==41612== HEAP SUMMARY: > ==41612== in use at exit: 118,228 bytes in 572 blocks > ==41612== total heap usage: 772 allocs, 200 frees, 143,760 bytes allocated > ==41612== > ==41612== LEAK SUMMARY: > ==41612== definitely lost: 0 bytes in 0 blocks > ==41612== indirectly lost: 0 bytes in 0 blocks > ==41612== possibly lost: 0 bytes in 0 blocks > ==41612== still reachable: 118,228 bytes in 572 blocks > ==41612== suppressed: 0 bytes in 0 blocks > ==41612== Rerun with --leak-check=full to see details of leaked memory > ==41612== > ==41612== For counts of detected and suppressed errors, rerun with: -v > ==41612== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) > > > **PERL -V** > > Summary of my perl5 (revision 5 version 23 subversion 4) configuration: > Commit id: a7dba6f870a82e3be87bda50593cb5d2042277fd > Platform: > osname=linux, osvers=3.16.0-4-amd64, archname=x86_64-linux-ld > uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux ' > config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DDEBUG_LEAKING_SCALARS -des' > hint=recommended, useposix=true, d_sigaction=define > useithreads=undef, usemultiplicity=undef > use64bitint=define, use64bitall=define, uselongdouble=define > usemymalloc=n, bincompat5005=undef > Compiler: > cc='ccache afl-gcc', ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', > optimize='-g', > cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' > ccversion='', gccversion='4.9.2', gccosandvers='' > intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3 > d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3 > ivtype='long', ivsize=8, nvtype='long double', nvsize=16, Off_t='off_t', lseeksize=8 > alignbytes=16, prototype=define > Linker and Libraries: > ld='ccache afl-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib' > libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib > libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc > perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc > libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a > gnulibc_version='2.19' > Dynamic Linking: > dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' > cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong' > > > Characteristics of this binary (from libperl): > Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE > PERL_DONT_CREATE_GVSV > PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP > PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL > USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE > USE_LOCALE_COLLATE USE_LOCALE_CTYPE > USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE > USE_PERLIO USE_PERL_ATOF > Built under linux > Compiled at Oct 15 2015 20:45:21 > @INC: > /usr/local/perl-afl/lib/site_perl/5.23.4/x86_64-linux-ld > /usr/local/perl-afl/lib/site_perl/5.23.4 > /usr/local/perl-afl/lib/5.23.4/x86_64-linux-ld > /usr/local/perl-afl/lib/5.23.4 > . > >
I
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 156b
Thanks for reporting this. Fixed by b297756b5d78ef23f3ebf878d0342947ece424aa and tested for in 80edda0d5872f642d1b92e7581eb0882e59b9c2e -- Karl Williamson
Download (untitled) / with headers
text/plain 252b
Thank you for submitting this report. You have helped make Perl better. With the release of Perl 5.24.0 on May 9, 2016, this and 149 other issues have been resolved. Perl 5.24.0 may be downloaded via https://metacpan.org/release/RJBS/perl-5.24.0


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org