Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storable null pointer deref on truncated data #9822

Closed
p5pRT opened this issue Aug 9, 2009 · 12 comments
Closed

Storable null pointer deref on truncated data #9822

p5pRT opened this issue Aug 9, 2009 · 12 comments

Comments

@p5pRT
Copy link

p5pRT commented Aug 9, 2009

Migrated from rt.perl.org#68348 (status was 'resolved')

Searchable as RT68348$

@p5pRT
Copy link
Author

p5pRT commented Aug 9, 2009

From @dgl

Created by @dgl

When deserialising truncated storable data where the truncation is within a
coderef there seems to be a null pointer dereference​:

  Program received signal SIGSEGV, Segmentation fault.
  retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at Storable.xs​:5438
  5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv! */

text is 0x0.

Code to reproduce is below, rather oddly if you uncomment the use v5.10 line it
doesn't seem to segfault.

# Uncommenting the line below appears to stop the segfault
#use v5.10;
use Storable qw(freeze thaw);

$Storable​::Eval = 1;
$Storable​::Deparse = 1;
for(4000..5000) {
  print "$_\n";

  my $s = {
  i => sub { "foo" },
  x => "y" x $_,
  y => sub { "foo" },
  };

  thaw(substr(freeze($s), 0, 4096));
}

It seems to vary slightly on different machines but this segfaults after
27-31 iterations for me.

Perl Info

Flags:
    category=library
    severity=medium

Site configuration information for perl 5.10.0:

Configured by Debian Project at Fri Jun 26 18:43:11 UTC 2009.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.6.24-23-server, archname=i486-linux-gnu-thread-multi
    uname='linux rothera 2.6.24-23-server #1 smp wed apr 1 22:22:14 utc 2009 i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.10.0 -Dsitearch=/usr/local/lib/perl/5.10.0 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.10.0 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.3.3', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib /usr/lib64
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.9.so, so=so, useshrplib=true, libperl=libperl.so.5.10.0
    gnulibc_version='2.9'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib'

Locally applied patches:
    


@INC for perl 5.10.0:
    /etc/perl
    /usr/local/lib/perl/5.10.0
    /usr/local/share/perl/5.10.0
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.10
    /usr/share/perl/5.10
    /usr/local/lib/site_perl
    .


Environment for perl 5.10.0:
    HOME=/home/dgl
    LANG=en_GB.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/lib/jvm/java-1.5.0-sun/bin:/home/dgl/bin:/sbin:/usr/sbin:/usr/local/sbin:/sw/bin:/sw/sbin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Aug 10, 2009

From perl@profvince.com

Program received signal SIGSEGV, Segmentation fault.
retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at Storable.xs​:5438
5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv! */

text is 0x0.

Code to reproduce is below, rather oddly if you uncomment the use v5.10 line it
doesn't seem to segfault.

# Uncommenting the line below appears to stop the segfault
#use v5.10;
use Storable qw(freeze thaw);

$Storable​::Eval = 1;
$Storable​::Deparse = 1;
for(4000..5000) {
print "$_\n";

my $s = {
i => sub { "foo" },
x => "y" x $_,
y => sub { "foo" },
};

thaw(substr(freeze($s), 0, 4096));
}

It seems to vary slightly on different machines but this segfaults after
27-31 iterations for me.

Thanks for this report.

I can reproduce it down to perl 5.8.1, either with threads enabled or
not (I have no 5.8.0 to test against), so it's not a regression.
On 5.10.0, valgrind says :

$ valgrind perl5.10.0-dbg x.pl
==4168== Memcheck, a memory error detector.
==4168== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==4168== Using LibVEX rev 1906, a library for dynamic binary translation.
==4168== Copyright (C) 2004-2009, and GNU GPL'd, by OpenWorks LLP.
==4168== Using valgrind-3.5.0.SVN, a dynamic binary instrumentation
framework.
==4168== Copyright (C) 2000-2009, and GNU GPL'd, by Julian Seward et al.
==4168== For more details, rerun with​: -v
==4168==
4000
4001
4002
4003
4004
4005
4006
4007
4008
4009
4010
4011
4012
4013
4014
4015
4016
4017
4018
4019
4020
4021
4022
4023
4024
4025
4026
==4168== Invalid read of size 4
==4168== at 0x47CE9F5​: retrieve_code (Storable.xs​:5438)
==4168== by 0x47D26B0​: retrieve (Storable.xs​:5967)
==4168== by 0x47C9079​: retrieve_ref (Storable.xs​:4483)
==4168== by 0x47D26B0​: retrieve (Storable.xs​:5967)
==4168== by 0x47CDC52​: retrieve_hash (Storable.xs​:5214)
==4168== by 0x47D26B0​: retrieve (Storable.xs​:5967)
==4168== by 0x47D3725​: do_retrieve (Storable.xs​:6148)
==4168== by 0x47D3BE2​: mretrieve (Storable.xs​:6264)
==4168== by 0x47D4D19​: XS_Storable_mretrieve (Storable.xs​:6460)
==4168== by 0x811F702​: Perl_pp_entersub (pp_hot.c​:2847)
==4168== by 0x80C5E30​: Perl_runops_debug (dump.c​:1931)
==4168== by 0x80FE980​: S_run_body (perl.c​:2384)
==4168== Address 0x8 is not stack'd, malloc'd or (recently) free'd
==4168==
==4168== Process terminating with default action of signal 11 (SIGSEGV)
==4168== Access not within mapped region at address 0x8
==4168== at 0x47CE9F5​: retrieve_code (Storable.xs​:5438)
==4168== by 0x47D26B0​: retrieve (Storable.xs​:5967)
==4168== by 0x47C9079​: retrieve_ref (Storable.xs​:4483)
==4168== by 0x47D26B0​: retrieve (Storable.xs​:5967)
==4168== by 0x47CDC52​: retrieve_hash (Storable.xs​:5214)
==4168== by 0x47D26B0​: retrieve (Storable.xs​:5967)
==4168== by 0x47D3725​: do_retrieve (Storable.xs​:6148)
==4168== by 0x47D3BE2​: mretrieve (Storable.xs​:6264)
==4168== by 0x47D4D19​: XS_Storable_mretrieve (Storable.xs​:6460)
==4168== by 0x811F702​: Perl_pp_entersub (pp_hot.c​:2847)
==4168== by 0x80C5E30​: Perl_runops_debug (dump.c​:1931)
==4168== by 0x80FE980​: S_run_body (perl.c​:2384)
==4168== If you believe this happened as a result of a stack
==4168== overflow in your program's main thread (unlikely but
==4168== possible), you can try to increase the size of the
==4168== main thread stack using the --main-stacksize= flag.
==4168== The main thread stack size used in this run was 8388608.
==4168==
==4168== ERROR SUMMARY​: 1 errors from 1 contexts (suppressed​: 17 from 2)
==4168== malloc/free​: in use at exit​: 2,450,124 bytes in 52,272 blocks.
==4168== malloc/free​: 92,241 allocs, 39,969 frees, 5,604,415 bytes
allocated.
==4168== For counts of detected errors, rerun with​: -v
==4168== searching for pointers to 52,272 not-freed blocks.
==4168== checked 2,550,944 bytes.
==4168==
==4168== LEAK SUMMARY​:
==4168== definitely lost​: 176 bytes in 1 blocks.
==4168== indirectly lost​: 3,393 bytes in 42 blocks.
==4168== possibly lost​: 1,679,240 bytes in 42,509 blocks.
==4168== still reachable​: 767,315 bytes in 9,720 blocks.
==4168== suppressed​: 0 bytes in 0 blocks.
==4168== Rerun with --leak-check=full to see details of leaked memory.

Indeed, if you "use 5.010", the error disappears - but not if you "use
5.008" with 5.8.x.
It seems that filling the hints hash with something prevents the bug to
happen, as

  BEGIN { $^H{foo} = 1 }
  use Storable qw(freeze thaw);
  ...

also doesn't segfault and is valgrind-clean.

Vincent.

@p5pRT
Copy link
Author

p5pRT commented Aug 10, 2009

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Aug 10, 2009

From perl@profvince.com

Indeed, if you "use 5.010", the error disappears - but not if you "use
5.008" with 5.8.x.
It seems that filling the hints hash with something prevents the bug to
happen, as

BEGIN \{ $^H\{foo\} = 1 \}
use Storable qw\(freeze thaw\);
\.\.\.

also doesn't segfault and is valgrind-clean.

More precisely, setting the hint must happen before initializing $s, as
this does not produce an error :

  BEGIN { $^H{foo} = 1 }
  my $s = {
  i => sub { "foo" },
  x => "y" x $_,
  y => sub { "foo" },
  };

while this does :

  my $s = {
  i => sub { "foo" },
  x => "y" x $_,
  y => sub { "foo" },
  };
  BEGIN { $^H{foo} = 1 }

Vincent.

@p5pRT
Copy link
Author

p5pRT commented Nov 21, 2016

From lightsey@cpan.org

On Sun, 09 Aug 2009 06​:56​:32 -0700, dgl wrote​:

When deserialising truncated storable data where the truncation is
within a
coderef there seems to be a null pointer dereference​:

Program received signal SIGSEGV, Segmentation fault.
retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at
Storable.xs​:5438
5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv!
*/

text is 0x0.

The patch attached to https://rt-archive.perl.org/perl5/Ticket/Display.html?id=130098 fixes this segfault.

@p5pRT
Copy link
Author

p5pRT commented Dec 25, 2016

From @jkeenan

On Mon, 21 Nov 2016 18​:27​:26 GMT, lightsey@​cpan.org wrote​:

On Sun, 09 Aug 2009 06​:56​:32 -0700, dgl wrote​:

When deserialising truncated storable data where the truncation is
within a
coderef there seems to be a null pointer dereference​:

Program received signal SIGSEGV, Segmentation fault.
retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at
Storable.xs​:5438
5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv!
*/

text is 0x0.

The patch attached to
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=130098 fixes this segfault.

Please review the smoke-me/jkeenan/130098-storable branch. Can we add a regression test to t/store.t in that branch that specifically addresses the problem reported in this ticket?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Jan 1, 2017

From @jkeenan

On Mon, 21 Nov 2016 18​:27​:26 GMT, lightsey@​cpan.org wrote​:

On Sun, 09 Aug 2009 06​:56​:32 -0700, dgl wrote​:

When deserialising truncated storable data where the truncation is
within a
coderef there seems to be a null pointer dereference​:

Program received signal SIGSEGV, Segmentation fault.
retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at
Storable.xs​:5438
5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no sv_catsv!
*/

text is 0x0.

The patch attached to
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=130098 fixes this segfault.

Can you confirm that commit adf9095 to blead remedies this situation?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Jan 7, 2017

From @jkeenan

On Sun, 01 Jan 2017 15​:19​:12 GMT, jkeenan wrote​:

On Mon, 21 Nov 2016 18​:27​:26 GMT, lightsey@​cpan.org wrote​:

On Sun, 09 Aug 2009 06​:56​:32 -0700, dgl wrote​:

When deserialising truncated storable data where the truncation is
within a
coderef there seems to be a null pointer dereference​:

Program received signal SIGSEGV, Segmentation fault.
retrieve_code (my_perl=0x754010, cxt=0x8c3750, cname=0x0) at
Storable.xs​:5438
5438 sv_catpv(sub, SvPV_nolen(text)); /* XXX no
sv_catsv!
*/

text is 0x0.

The patch attached to
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=130098 fixes this
segfault.

Can you confirm that commit adf9095
to blead remedies this situation?

Thank you very much.

I don't think that commit fixes the problem.

When I build a threaded perl at blead (see attachment) and then run the program in the original post in this ticket, I get output like this​:

#####
$ ./bin/perl -Ilib ~/learn/perl/p5p/68348-storable.pl
4000
4001
4002
4003
4004
4005
4006
4007
Unable to retrieve code, at /home/jkeenan/learn/perl/p5p/68348-storable.pl line 15.
#####

Any ideas?

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Jan 7, 2017

From @jkeenan

Summary of my perl5 (revision 5 version 25 subversion 9) configuration​:
  Commit id​: 9065925
  Platform​:
  osname=linux
  osvers=4.4.0-57-generic
  archname=x86_64-linux-thread-multi
  uname='linux zareason 4.4.0-57-generic #78-ubuntu smp fri dec 9 23​:50​:32 utc 2016 x86_64 x86_64 x86_64 gnulinux '
  config_args='-des -Dusedevel -Dusethreads -Uversiononly -Dprefix=/home/jkeenan/testing/blead -Dman1dir=none -Dman3dir=none'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
  optimize='-O2'
  cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='5.4.0 20160609'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64
  libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.23.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.23'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  MULTIPLICITY
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_IMPLICIT_CONTEXT
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_ITHREADS
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  USE_REENTRANT_API
  Built under linux
  Compiled at Jan 6 2017 21​:04​:04
  %ENV​:
  PERLBREW_BASHRC_VERSION="0.67"
  PERLBREW_HOME="/home/jkeenan/.perlbrew"
  PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/man"
  PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin​:/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/bin"
  PERLBREW_PERL="perl-5.24.0"
  PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew"
  PERLBREW_VERSION="0.67"
  PERL_WORKDIR="/home/jkeenan/gitwork/perl"
  @​INC​:
  lib
  /home/jkeenan/testing/blead/lib/perl5/site_perl/5.25.9/x86_64-linux-thread-multi
  /home/jkeenan/testing/blead/lib/perl5/site_perl/5.25.9
  /home/jkeenan/testing/blead/lib/perl5/5.25.9/x86_64-linux-thread-multi
  /home/jkeenan/testing/blead/lib/perl5/5.25.9
  .

@p5pRT
Copy link
Author

p5pRT commented Jan 7, 2017

From @jkeenan

68348-storable.pl

@p5pRT
Copy link
Author

p5pRT commented Jan 10, 2017

From @lightsey

On Fri, 2017-01-06 at 18​:13 -0800, James E Keenan via RT wrote​:

On Sun, 01 Jan 2017 15​:19​:12 GMT, jkeenan wrote​:

Can you confirm that commit adf9095
to blead remedies this situation?
 
Thank you very much.

I don't think that commit fixes the problem.

When I build a threaded perl at blead (see attachment) and then run the
program in the original post in this ticket, I get output like this​:

#####
$ ./bin/perl -Ilib ~/learn/perl/p5p/68348-storable.pl 
4000
4001
4002
4003
4004
4005
4006
4007
Unable to retrieve code, at /home/jkeenan/learn/perl/p5p/68348-storable.pl
line 15.
#####

Any ideas?

Your results show the problem is fixed. Storable is dieing instead of
segfaulting now.

The data is corrupted by the substr() on the 8th iteration, so it can't be
loaded. The segfault was the bug.

@p5pRT
Copy link
Author

p5pRT commented Jan 11, 2017

@iabyn - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant