Skip Menu |
Report information
Id: 68348
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: dgl <dgl [at] dgl.cx>
Cc:
AdminCc:

Operating System: Linux
PatchStatus: (no value)
Severity: medium
Type:
Perl Version: 5.10.0
Fixed In: 5.25.9



Subject: Storable null pointer deref on truncated data
Date: Sun, 9 Aug 2009 14:56:00 +0100 (BST)
To: perlbug [...] perl.org
From: dgl [...] d.cx (David)
Download (untitled) / with headers
text/plain 4.2k
This is a bug report for perl from dgl@dgl.cx, generated with the help of perlbug 1.36 running under perl 5.10.0. ----------------------------------------------------------------- [Please enter your report here] When deserialising truncated storable data where the truncation is within a coderef there seems to be a null pointer dereference: Program received signal SIGSEGV, Segmentation fault. retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at Storable.xs:5438 5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv! */ text is 0x0. Code to reproduce is below, rather oddly if you uncomment the use v5.10 line it doesn't seem to segfault. # Uncommenting the line below appears to stop the segfault #use v5.10; use Storable qw(freeze thaw); $Storable::Eval = 1; $Storable::Deparse = 1; for(4000..5000) { print "$_\n"; my $s = { i => sub { "foo" }, x => "y" x $_, y => sub { "foo" }, }; thaw(substr(freeze($s), 0, 4096)); } It seems to vary slightly on different machines but this segfaults after 27-31 iterations for me. [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=library severity=medium --- Site configuration information for perl 5.10.0: Configured by Debian Project at Fri Jun 26 18:43:11 UTC 2009. Summary of my perl5 (revision 5 version 10 subversion 0) configuration: Platform: osname=linux, osvers=2.6.24-23-server, archname=i486-linux-gnu-thread-multi uname='linux rothera 2.6.24-23-server #1 smp wed apr 1 22:22:14 utc 2009 i686 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.10.0 -Dsitearch=/usr/local/lib/perl/5.10.0 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.10.0 -Dd_dosuid -des' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=undef, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2 -g', cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include' ccversion='', gccversion='4.3.3', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /usr/lib64 libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=/lib/libc-2.9.so, so=so, useshrplib=true, libperl=libperl.so.5.10.0 gnulibc_version='2.9' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib' Locally applied patches: --- @INC for perl 5.10.0: /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl . --- Environment for perl 5.10.0: HOME=/home/dgl LANG=en_GB.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/lib/jvm/java-1.5.0-sun/bin:/home/dgl/bin:/sbin:/usr/sbin:/usr/local/sbin:/sw/bin:/sw/sbin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games PERL_BADLANG (unset) SHELL=/bin/zsh
Subject: Re: [perl #68348] Storable null pointer deref on truncated data
Date: Mon, 10 Aug 2009 17:13:54 +0200
To: perl5-porters [...] perl.org
From: Vincent Pit <perl [...] profvince.com>
Download (untitled) / with headers
text/plain 4.4k
Show quoted text
> Program received signal SIGSEGV, Segmentation fault. > retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at Storable.xs:5438 > 5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv! */ > > text is 0x0. > > Code to reproduce is below, rather oddly if you uncomment the use v5.10 line it > doesn't seem to segfault. > > # Uncommenting the line below appears to stop the segfault > #use v5.10; > use Storable qw(freeze thaw); > > $Storable::Eval = 1; > $Storable::Deparse = 1; > for(4000..5000) { > print "$_\n"; > > my $s = { > i => sub { "foo" }, > x => "y" x $_, > y => sub { "foo" }, > }; > > thaw(substr(freeze($s), 0, 4096)); > } > > It seems to vary slightly on different machines but this segfaults after > 27-31 iterations for me. >
Thanks for this report. I can reproduce it down to perl 5.8.1, either with threads enabled or not (I have no 5.8.0 to test against), so it's not a regression. On 5.10.0, valgrind says : $ valgrind perl5.10.0-dbg x.pl ==4168== Memcheck, a memory error detector. ==4168== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al. ==4168== Using LibVEX rev 1906, a library for dynamic binary translation. ==4168== Copyright (C) 2004-2009, and GNU GPL'd, by OpenWorks LLP. ==4168== Using valgrind-3.5.0.SVN, a dynamic binary instrumentation framework. ==4168== Copyright (C) 2000-2009, and GNU GPL'd, by Julian Seward et al. ==4168== For more details, rerun with: -v ==4168== 4000 4001 4002 4003 4004 4005 4006 4007 4008 4009 4010 4011 4012 4013 4014 4015 4016 4017 4018 4019 4020 4021 4022 4023 4024 4025 4026 ==4168== Invalid read of size 4 ==4168== at 0x47CE9F5: retrieve_code (Storable.xs:5438) ==4168== by 0x47D26B0: retrieve (Storable.xs:5967) ==4168== by 0x47C9079: retrieve_ref (Storable.xs:4483) ==4168== by 0x47D26B0: retrieve (Storable.xs:5967) ==4168== by 0x47CDC52: retrieve_hash (Storable.xs:5214) ==4168== by 0x47D26B0: retrieve (Storable.xs:5967) ==4168== by 0x47D3725: do_retrieve (Storable.xs:6148) ==4168== by 0x47D3BE2: mretrieve (Storable.xs:6264) ==4168== by 0x47D4D19: XS_Storable_mretrieve (Storable.xs:6460) ==4168== by 0x811F702: Perl_pp_entersub (pp_hot.c:2847) ==4168== by 0x80C5E30: Perl_runops_debug (dump.c:1931) ==4168== by 0x80FE980: S_run_body (perl.c:2384) ==4168== Address 0x8 is not stack'd, malloc'd or (recently) free'd ==4168== ==4168== Process terminating with default action of signal 11 (SIGSEGV) ==4168== Access not within mapped region at address 0x8 ==4168== at 0x47CE9F5: retrieve_code (Storable.xs:5438) ==4168== by 0x47D26B0: retrieve (Storable.xs:5967) ==4168== by 0x47C9079: retrieve_ref (Storable.xs:4483) ==4168== by 0x47D26B0: retrieve (Storable.xs:5967) ==4168== by 0x47CDC52: retrieve_hash (Storable.xs:5214) ==4168== by 0x47D26B0: retrieve (Storable.xs:5967) ==4168== by 0x47D3725: do_retrieve (Storable.xs:6148) ==4168== by 0x47D3BE2: mretrieve (Storable.xs:6264) ==4168== by 0x47D4D19: XS_Storable_mretrieve (Storable.xs:6460) ==4168== by 0x811F702: Perl_pp_entersub (pp_hot.c:2847) ==4168== by 0x80C5E30: Perl_runops_debug (dump.c:1931) ==4168== by 0x80FE980: S_run_body (perl.c:2384) ==4168== If you believe this happened as a result of a stack ==4168== overflow in your program's main thread (unlikely but ==4168== possible), you can try to increase the size of the ==4168== main thread stack using the --main-stacksize= flag. ==4168== The main thread stack size used in this run was 8388608. ==4168== ==4168== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 17 from 2) ==4168== malloc/free: in use at exit: 2,450,124 bytes in 52,272 blocks. ==4168== malloc/free: 92,241 allocs, 39,969 frees, 5,604,415 bytes allocated. ==4168== For counts of detected errors, rerun with: -v ==4168== searching for pointers to 52,272 not-freed blocks. ==4168== checked 2,550,944 bytes. ==4168== ==4168== LEAK SUMMARY: ==4168== definitely lost: 176 bytes in 1 blocks. ==4168== indirectly lost: 3,393 bytes in 42 blocks. ==4168== possibly lost: 1,679,240 bytes in 42,509 blocks. ==4168== still reachable: 767,315 bytes in 9,720 blocks. ==4168== suppressed: 0 bytes in 0 blocks. ==4168== Rerun with --leak-check=full to see details of leaked memory. Indeed, if you "use 5.010", the error disappears - but not if you "use 5.008" with 5.8.x. It seems that filling the hints hash with something prevents the bug to happen, as BEGIN { $^H{foo} = 1 } use Storable qw(freeze thaw); ... also doesn't segfault and is valgrind-clean. Vincent.
Subject: Re: [perl #68348] Storable null pointer deref on truncated data
Date: Mon, 10 Aug 2009 17:27:59 +0200
To: perl5-porters [...] perl.org
From: Vincent Pit <perl [...] profvince.com>
Download (untitled) / with headers
text/plain 693b
Show quoted text
> Indeed, if you "use 5.010", the error disappears - but not if you "use > 5.008" with 5.8.x. > It seems that filling the hints hash with something prevents the bug to > happen, as > > BEGIN { $^H{foo} = 1 } > use Storable qw(freeze thaw); > ... > > also doesn't segfault and is valgrind-clean. >
More precisely, setting the hint must happen before initializing $s, as this does not produce an error : BEGIN { $^H{foo} = 1 } my $s = { i => sub { "foo" }, x => "y" x $_, y => sub { "foo" }, }; while this does : my $s = { i => sub { "foo" }, x => "y" x $_, y => sub { "foo" }, }; BEGIN { $^H{foo} = 1 } Vincent.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 513b
On Sun, 09 Aug 2009 06:56:32 -0700, dgl wrote: Show quoted text
> > When deserialising truncated storable data where the truncation is > within a > coderef there seems to be a null pointer dereference: > > Program received signal SIGSEGV, Segmentation fault. > retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at > Storable.xs:5438 > 5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv! > */ > > text is 0x0.
The patch attached to https://rt.perl.org/Ticket/Display.html?id=130098 fixes this segfault.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 845b
On Mon, 21 Nov 2016 18:27:26 GMT, lightsey@cpan.org wrote: Show quoted text
> On Sun, 09 Aug 2009 06:56:32 -0700, dgl wrote:
> > > > When deserialising truncated storable data where the truncation is > > within a > > coderef there seems to be a null pointer dereference: > > > > Program received signal SIGSEGV, Segmentation fault. > > retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at > > Storable.xs:5438 > > 5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv! > > */ > > > > text is 0x0.
> > The patch attached to > https://rt.perl.org/Ticket/Display.html?id=130098 fixes this segfault.
Please review the smoke-me/jkeenan/130098-storable branch. Can we add a regression test to t/store.t in that branch that specifically addresses the problem reported in this ticket? Thank you very much. -- James E Keenan (jkeenan@cpan.org)
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 766b
On Mon, 21 Nov 2016 18:27:26 GMT, lightsey@cpan.org wrote: Show quoted text
> On Sun, 09 Aug 2009 06:56:32 -0700, dgl wrote:
> > > > When deserialising truncated storable data where the truncation is > > within a > > coderef there seems to be a null pointer dereference: > > > > Program received signal SIGSEGV, Segmentation fault. > > retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at > > Storable.xs:5438 > > 5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv! > > */ > > > > text is 0x0.
> > The patch attached to > https://rt.perl.org/Ticket/Display.html?id=130098 fixes this segfault.
Can you confirm that commit adf9095d629bebb27169b0f3b03f75ee974da100 to blead remedies this situation? Thank you very much. -- James E Keenan (jkeenan@cpan.org)
RT-Send-CC: perl [...] profvince.com, perl5-porters [...] perl.org, lightsey [...] cpan.org, john [...] nixnuts.net, perl [...] profvince.com
Download (untitled) / with headers
text/plain 1.2k
On Sun, 01 Jan 2017 15:19:12 GMT, jkeenan wrote: Show quoted text
> On Mon, 21 Nov 2016 18:27:26 GMT, lightsey@cpan.org wrote:
> > On Sun, 09 Aug 2009 06:56:32 -0700, dgl wrote:
> > > > > > When deserialising truncated storable data where the truncation is > > > within a > > > coderef there seems to be a null pointer dereference: > > > > > > Program received signal SIGSEGV, Segmentation fault. > > > retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at > > > Storable.xs:5438 > > > 5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no > > > sv_catsv! > > > */ > > > > > > text is 0x0.
> > > > The patch attached to > > https://rt.perl.org/Ticket/Display.html?id=130098 fixes this > > segfault.
> > Can you confirm that commit adf9095d629bebb27169b0f3b03f75ee974da100 > to blead remedies this situation? > > Thank you very much.
I don't think that commit fixes the problem. When I build a threaded perl at blead (see attachment) and then run the program in the original post in this ticket, I get output like this: ##### $ ./bin/perl -Ilib ~/learn/perl/p5p/68348-storable.pl 4000 4001 4002 4003 4004 4005 4006 4007 Unable to retrieve code, at /home/jkeenan/learn/perl/p5p/68348-storable.pl line 15. ##### Any ideas? Thank you very much. -- James E Keenan (jkeenan@cpan.org)
Subject: 68348-perl-V.txt
Download 68348-perl-V.txt
text/plain 3.3k
Summary of my perl5 (revision 5 version 25 subversion 9) configuration: Commit id: 90659255338e4b2a5a99c585d4ccaf6424186d2e Platform: osname=linux osvers=4.4.0-57-generic archname=x86_64-linux-thread-multi uname='linux zareason 4.4.0-57-generic #78-ubuntu smp fri dec 9 23:50:32 utc 2016 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Dusedevel -Dusethreads -Uversiononly -Dprefix=/home/jkeenan/testing/blead -Dman1dir=none -Dman3dir=none' hint=recommended useposix=true d_sigaction=define useithreads=define usemultiplicity=define use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n bincompat5005=undef Compiler: cc='cc' ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' optimize='-O2' cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='5.4.0 20160609' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64 libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.23.so so=so useshrplib=false libperl=libperl.a gnulibc_version='2.23' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong' Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP PERL_OP_PARENT PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API Built under linux Compiled at Jan 6 2017 21:04:04 %ENV: PERLBREW_BASHRC_VERSION="0.67" PERLBREW_HOME="/home/jkeenan/.perlbrew" PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/man" PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin:/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/bin" PERLBREW_PERL="perl-5.24.0" PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew" PERLBREW_VERSION="0.67" PERL_WORKDIR="/home/jkeenan/gitwork/perl" @INC: lib /home/jkeenan/testing/blead/lib/perl5/site_perl/5.25.9/x86_64-linux-thread-multi /home/jkeenan/testing/blead/lib/perl5/site_perl/5.25.9 /home/jkeenan/testing/blead/lib/perl5/5.25.9/x86_64-linux-thread-multi /home/jkeenan/testing/blead/lib/perl5/5.25.9 .
Subject: 68348-storable.pl
Download 68348-storable.pl
text/x-perl 246b
use v5.10; use Storable qw(freeze thaw); $Storable::Eval = 1; $Storable::Deparse = 1; for(4000..5000) { print "$_\n"; my $s = { i => sub { "foo" }, x => "y" x $_, y => sub { "foo" }, }; thaw(substr(freeze($s), 0, 4096)); }
Subject: Re: [perl #68348] Storable null pointer deref on truncated data
From: John Lightsey <john [...] nixnuts.net>
To: perlbug-followup [...] perl.org
Date: Tue, 10 Jan 2017 09:03:43 -0600
Download (untitled) / with headers
text/plain 891b
On Fri, 2017-01-06 at 18:13 -0800, James E Keenan via RT wrote: Show quoted text
> On Sun, 01 Jan 2017 15:19:12 GMT, jkeenan wrote:
> > Can you confirm that commit adf9095d629bebb27169b0f3b03f75ee974da100 > > to blead remedies this situation? > >  > > Thank you very much.
> > I don't think that commit fixes the problem. > > When I build a threaded perl at blead (see attachment) and then run the > program in the original post in this ticket, I get output like this: > > ##### > $ ./bin/perl -Ilib ~/learn/perl/p5p/68348-storable.pl  > 4000 > 4001 > 4002 > 4003 > 4004 > 4005 > 4006 > 4007 > Unable to retrieve code, at /home/jkeenan/learn/perl/p5p/68348-storable.pl > line 15. > ##### > > Any ideas?
Your results show the problem is fixed. Storable is dieing instead of segfaulting now. The data is corrupted by the substr() on the 8th iteration, so it can't be loaded. The segfault was the bug.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org