Skip Menu |
Report information
Id: 126258
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: dcollinsn [at] gmail.com
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: (no value)



Subject: grep and map lead to Perl_op_free: Assertion `!(o->op_private & ~PL_op_private_valid[type])' failed.
Download (untitled) / with headers
text/plain 6.5k
Greetings Porters, I have compiled bleadperl with the afl-gcc compiler using: ./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DPERL_POISON -des AFL_HARDEN=1 make && make test And then fuzzed the resulting binary using: AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@ After reducing testcases using `afl-tmin` and performing additional minimization by hand, I have located the following testcase that triggers an assert fail in the perl interpreter, but which (correctly) errors out in non-debugging perl. The simplest testcase is the 9-character file: grep$0,0} In a not-debugging perl, the output is: Unmatched right curly bracket at ../uncat/f3i000003 line 1, at end of line syntax error at ../uncat/f3i000003 line 1, near "0}" Execution of ../uncat/f3i000003 aborted due to compilation errors. In a debugging perl, the output is: Unmatched right curly bracket at ../uncat/f3i000003 line 1, at end of line syntax error at ../uncat/f3i000003 line 1, near "0}" Execution of ../uncat/f3i000003 aborted due to compilation errors. perl: op.c:717: Perl_op_free: Assertion `!(o->op_private & ~PL_op_private_valid[type])' failed. Aborted Other failing testcases that might help you understand what's going on here: grep$0,0} map\0,0} <<--- Literal backslash map\n0,0} <<--- Literal newline 0%grep{.}0 0%N{.}0%grep%0,0 map\nmap,0 <<--- Literal newline **GDB** dcollins@nightshade64:~/perldebug$ gdb --args ./perl -Ilib ../uncat/f3i000003 GNU gdb (GDB) 7.10 Copyright (C) 2015 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./perl...done. (gdb) run Starting program: /home/dcollins/perldebug/perl -Ilib ../uncat/f3i000003 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Unmatched right curly bracket at ../uncat/f3i000003 line 1, at end of line syntax error at ../uncat/f3i000003 line 1, near "0}" Execution of ../uncat/f3i000003 aborted due to compilation errors. perl: op.c:717: Perl_op_free: Assertion `!(o->op_private & ~PL_op_private_valid[type])' failed. Program received signal SIGABRT, Aborted. 0x00007ffff6cf4107 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x00007ffff6cf4107 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007ffff6cf54e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007ffff6ced226 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x00007ffff6ced2d2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x000000000043f359 in Perl_op_free (o=0x12118b0) at op.c:717 #5 0x0000000000529301 in perl_destruct (my_perl=<optimized out>) at perl.c:777 #6 0x000000000042c660 in main (argc=3, argv=0x7fffffffe648, env=0x7fffffffe668) at perlmain.c:127 (gdb) frame 4 #4 0x000000000043f359 in Perl_op_free (o=0x12118b0) at op.c:717 717 assert(!(o->op_private & ~PL_op_private_valid[type])); (gdb) info locals type = 169 defer_ix = <optimized out> defer_stack_alloc = <optimized out> defer_stack = <optimized out> __PRETTY_FUNCTION__ = "Perl_op_free" **PERL -V** dcollins@nightshade64:~/perldebug$ ./perl -V Summary of my perl5 (revision 5 version 23 subversion 4) configuration: Commit id: 94757bf7ac8451b3588578f287f9c511a04011bf Platform: osname=linux, osvers=3.16.0-4-amd64, archname=x86_64-linux-ld uname='linux nightshade64 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u4 (2015-09-19) x86_64 gnulinux ' config_args='-Dusedevel -Dprefix=/usr/local/perl-afl -Dcc=ccache afl-gcc -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DPERL_POISON -des' hint=recommended, useposix=true, d_sigaction=define useithreads=undef, usemultiplicity=undef use64bitint=define, use64bitall=define, uselongdouble=define usemymalloc=n, bincompat5005=undef Compiler: cc='ccache afl-gcc', ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-g', cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='', gccversion='4.9.2', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3 ivtype='long', ivsize=8, nvtype='long double', nvsize=16, Off_t='off_t', lseeksize=8 alignbytes=16, prototype=define Linker and Libraries: ld='ccache afl-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib -fstack-protector-strong' Characteristics of this binary (from libperl): Compile-time options: DEBUGGING HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Oct 2 2015 22:41:42 @INC: /usr/local/perl-afl/lib/site_perl/5.23.4/x86_64-linux-ld /usr/local/perl-afl/lib/site_perl/5.23.4 /usr/local/perl-afl/lib/5.23.4/x86_64-linux-ld /usr/local/perl-afl/lib/5.23.4 .
From: Dave Mitchell <davem [...] iabyn.com>
Date: Thu, 19 Nov 2015 16:02:15 +0000
To: perl5-porters [...] perl.org
Subject: Re: [perl #126258] grep and map lead to Perl_op_free: Assertion `!(o->op_private & ~PL_op_private_valid[type])' failed.
On Sun, Oct 04, 2015 at 04:13:28AM -0700, Dan Collins wrote: Show quoted text
> I have compiled bleadperl with the afl-gcc compiler using: > > ./Configure -Dusedevel -Dprefix='/usr/local/perl-afl' -Dcc='ccache afl-gcc' -Duselongdouble -Duse64bitall -Doptimize=-g -Uversiononly -Uman1dir -Uman3dir -DDEBUGGING -DPERL_POISON -des > AFL_HARDEN=1 make && make test > > And then fuzzed the resulting binary using: > > AFL_NO_VAR_CHECK=1 afl-fuzz -i in -o out bin/perl @@ > > After reducing testcases using `afl-tmin` and performing additional minimization by hand, I have located the following testcase that triggers an assert fail in the perl interpreter, but which (correctly) errors out in non-debugging perl. The simplest testcase is the 9-character file: > > grep$0,0}
[snip] Show quoted text
> In a debugging perl, the output is: > Unmatched right curly bracket at ../uncat/f3i000003 line 1, at end of line > syntax error at ../uncat/f3i000003 line 1, near "0}" > Execution of ../uncat/f3i000003 aborted due to compilation errors. > perl: op.c:717: Perl_op_free: Assertion `!(o->op_private & ~PL_op_private_valid[type])' failed. > Aborted
I've pushed a general fix for these sorts of issues with: commit 09681a13f07600082e5a7efbc9c978f62b24c502 Author: David Mitchell <davem@iabyn.com> AuthorDate: Thu Nov 19 15:49:03 2015 +0000 Commit: David Mitchell <davem@iabyn.com> CommitDate: Thu Nov 19 16:00:31 2015 +0000 op_free(): don't assert op_private ok when erred [perl #126258] op_free includes an assert to make sure each op's op_private field only has the flags set that are expected for that op. It's a thing I added at the same time I added the regen/op_private mechanism, and is more a general "make sure people are only setting the flags we know about" test. However, if the op tree is being freed after a compilation error, some op's flags may be in an inconsistent state; so skip the assert in that case. e.g. perl -e 'grep$0,0}' -- Modern art: "That's easy, I could have done that!" "Ah, but you didn't!"
Date: Thu, 19 Nov 2015 21:22:03 +0100
Subject: Re: [perl #126258] grep and map lead to Perl_op_free: Assertion `!(o->op_private & ~PL_op_private_valid[type])' failed.
From: Lukas Mai <plokinom [...] gmail.com>
To: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.3k
Am 19.11.2015 um 17:02 schrieb Dave Mitchell: Show quoted text
> I've pushed a general fix for these sorts of issues with: > > commit 09681a13f07600082e5a7efbc9c978f62b24c502 > Author: David Mitchell <davem@iabyn.com> > AuthorDate: Thu Nov 19 15:49:03 2015 +0000 > Commit: David Mitchell <davem@iabyn.com> > CommitDate: Thu Nov 19 16:00:31 2015 +0000 > > op_free(): don't assert op_private ok when erred > > [perl #126258] > > op_free includes an assert to make sure each op's op_private field > only has the flags set that are expected for that op. It's a thing I added > at the same time I added the regen/op_private mechanism, and is more a > general "make sure people are only setting the flags we know about" test. > > However, if the op tree is being freed after a compilation error, some > op's flags may be in an inconsistent state; so skip the assert in that case. > > e.g. > > perl -e 'grep$0,0}' >
From this description I would have expected something like: if (!(PL_parser && PL_parser->error_count)) { assert(...); } i.e. assert unless we're currently parsing and there has been an error. But the patch actually does if (PL_parser && !PL_parser->error_count) { assert(...); } Is this intentional? -- Lukas Mai <plokinom@gmail.com>
Date: Thu, 19 Nov 2015 23:27:04 +0000
From: Dave Mitchell <davem [...] iabyn.com>
Subject: Re: [perl #126258] grep and map lead to Perl_op_free: Assertion `!(o->op_private & ~PL_op_private_valid[type])' failed.
To: Lukas Mai <plokinom [...] gmail.com>
CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 890b
On Thu, Nov 19, 2015 at 09:22:03PM +0100, Lukas Mai wrote: Show quoted text
> From this description I would have expected something like: > > if (!(PL_parser && PL_parser->error_count)) { > assert(...); > } > > i.e. assert unless we're currently parsing and there has been an error. > > But the patch actually does > > if (PL_parser && !PL_parser->error_count) { > assert(...); > } > > Is this intentional?
Yes. If there's a compilation error in the main body of the program (as opposed to code that was evalled/required/used), then by the time the defective optree is is freed, PL_parser is already NULL: so we have no way of knowing whether it compiled ok or not; so always skip on a NULL PL_parser. -- "I do not resent criticism, even when, for the sake of emphasis, it parts for the time with reality". -- Winston Churchill, House of Commons, 22nd Jan 1941.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org