Skip Menu |
 
Report information
Id: 114984
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: bohica <bohica [at] ntlworld.com>
Cc:
AdminCc:

Operating System: Linux
PatchStatus: (no value)
Severity: High
Type: library
Perl Version: 5.17.4
Fixed In: (no value)



Subject: "double free or corruption" "Invalid write of size 4" in File::Glob?
Date: Thu, 20 Sep 2012 18:47:19 +0100 (BST)
To: perlbug [...] perl.org
From: bohica [...] ntlworld.com
Download (untitled) / with headers
text/plain 9.1k
This is a bug report for perl from bohica@ntlworld.com, generated with the help of perlbug 1.39 running under perl 5.17.4. ----------------------------------------------------------------- [Please describe your issue here] *** glibc detected *** perl: double free or corruption (!prev): 0x098b2180 *** Self contained script below is the smallest I can come up with that illustrates this problem. I know it is rather specific. Changing almost any of the numbers of files created makes the problem go away. valgrind output suggests a "Invalid write of size 4". This reported via perlbug in 5.17.4 but it happens in 5.16.0 and 5.16.1 too. *** glibc detected *** perl: double free or corruption (!prev): 0x08e60240 *** ======= Backtrace: ========= /lib/i386-linux-gnu/libc.so.6(+0x6ff22)[0xb75dbf22] /lib/i386-linux-gnu/libc.so.6(+0x70bc2)[0xb75dcbc2] /lib/i386-linux-gnu/libc.so.6(cfree+0x6d)[0xb75dfcad] perl(Perl_av_extend_guts+0x18c)[0x80da7ec] perl(Perl_av_extend+0x44)[0x80da984] perl(Perl_stack_grow+0x32)[0x8106af2] perl(Perl_pp_padav+0x114)[0x80f73d4] perl(Perl_runops_standard+0xb)[0x80dc2fb] perl(perl_run+0x325)[0x807ebc5] perl(main+0x10d)[0x8061ced] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb7585113] perl[0x8061d21] ======= Memory map: ======== 08048000-08196000 r-xp 00000000 08:01 15452891 /home/martin/perl5/perlbrew/per ls/perl-5.17.4/bin/perl5.17.4 08196000-08197000 r--p 0014d000 08:01 15452891 /home/martin/perl5/perlbrew/per ls/perl-5.17.4/bin/perl5.17.4 08197000-0819a000 rw-p 0014e000 08:01 15452891 /home/martin/perl5/perlbrew/per ls/perl-5.17.4/bin/perl5.17.4 08e43000-08ec0000 rw-p 00000000 00:00 0 [heap] b7000000-b7021000 rw-p 00000000 00:00 0 b7021000-b7100000 ---p 00000000 00:00 0 b71b6000-b71d2000 r-xp 00000000 08:01 12756973 /lib/i386-linux-gnu/libgcc_s.so .1 b71d2000-b71d3000 r--p 0001b000 08:01 12756973 /lib/i386-linux-gnu/libgcc_s.so .1 b71d3000-b71d4000 rw-p 0001c000 08:01 12756973 /lib/i386-linux-gnu/libgcc_s.so .1 b71f3000-b71f8000 r-xp 00000000 08:01 16385302 /home/martin/perl5/perlbrew/per ls/perl-5.17.4/lib/5.17.4/i686-linux/auto/File/Glob/Glob.so b71f8000-b71f9000 r--p 00005000 08:01 16385302 /home/martin/perl5/perlbrew/per ls/perl-5.17.4/lib/5.17.4/i686-linux/auto/File/Glob/Glob.so b71f9000-b71fa000 rw-p 00006000 08:01 16385302 /home/martin/perl5/perlbrew/per ls/perl-5.17.4/lib/5.17.4/i686-linux/auto/File/Glob/Glob.so b71fa000-b723a000 r--p 006a5000 08:01 12665422 /usr/lib/locale/locale-archive b723a000-b736b000 r--p 00446000 08:01 12665422 /usr/lib/locale/locale-archive b736b000-b756b000 r--p 00000000 08:01 12665422 /usr/lib/locale/locale-archive b756b000-b756c000 rw-p 00000000 00:00 0 b756c000-b76e4000 r-xp 00000000 08:01 12755031 /lib/i386-linux-gnu/libc-2.13.s o b76e4000-b76e6000 r--p 00178000 08:01 12755031 /lib/i386-linux-gnu/libc-2.13.s o b76e6000-b76e7000 rw-p 0017a000 08:01 12755031 /lib/i386-linux-gnu/libc-2.13.s o b76e7000-b76ea000 rw-p 00000000 00:00 0 b76ea000-b76f2000 r-xp 00000000 08:01 12755033 /lib/i386-linux-gnu/libcrypt-2. 13.so b76f2000-b76f3000 r--p 00007000 08:01 12755033 /lib/i386-linux-gnu/libcrypt-2. 13.so b76f3000-b76f4000 rw-p 00008000 08:01 12755033 /lib/i386-linux-gnu/libcrypt-2. 13.so b76f4000-b771c000 rw-p 00000000 00:00 0 b771c000-b7744000 r-xp 00000000 08:01 12755035 /lib/i386-linux-gnu/libm-2.13.s o b7744000-b7745000 r--p 00028000 08:01 12755035 /lib/i386-linux-gnu/libm-2.13.s o b7745000-b7746000 rw-p 00029000 08:01 12755035 /lib/i386-linux-gnu/libm-2.13.s o b7746000-b7749000 r-xp 00000000 08:01 12755034 /lib/i386-linux-gnu/libdl-2.13. so b7749000-b774a000 r--p 00002000 08:01 12755034 /lib/i386-linux-gnu/libdl-2.13. so b774a000-b774b000 rw-p 00003000 08:01 12755034 /lib/i386-linux-gnu/libdl-2.13. so b7769000-b776a000 r--p 003fc000 08:01 12665422 /usr/lib/locale/locale-archive b776a000-b776c000 rw-p 00000000 00:00 0 b776c000-b776d000 r-xp 00000000 00:00 0 [vdso] b776d000-b778b000 r-xp 00000000 08:01 12755028 /lib/i386-linux-gnu/ld-2.13.so b778b000-b778c000 r--p 0001d000 08:01 12755028 /lib/i386-linux-gnu/ld-2.13.so b778c000-b778d000 rw-p 0001e000 08:01 12755028 /lib/i386-linux-gnu/ld-2.13.so bff89000-bffaa000 rw-p 00000000 00:00 0 [stack] Aborted valgrind output: ==2984== Invalid write of size 4 ==2984== at 0x4029C3E: memcpy (mc_replace_strmem.c:635) ==2984== by 0x4032E16: iterate (in /home/martin/perl5/perlbrew/perls/perl-5.1 7.4/lib/5.17.4/i686-linux/auto/File/Glob/Glob.so) ==2984== by 0x8119930: Perl_pp_glob (in /home/martin/perl5/perlbrew/perls/per l-5.17.4/bin/perl5.17.4) ==2984== by 0x40C8112: (below main) (libc-start.c:226) ==2984== Address 0x42b4568 is 0 bytes after a block of size 2,032 alloc'd ==2984== at 0x4028876: malloc (vg_replace_malloc.c:236) ==2984== by 0x80C0D48: Perl_safesysmalloc (in /home/martin/perl5/perlbrew/per ls/perl-5.17.4/bin/perl5.17.4) ==2984== by 0x80DA79F: Perl_av_extend_guts (in /home/martin/perl5/perlbrew/pe rls/perl-5.17.4/bin/perl5.17.4) ==2984== by 0x80DA983: Perl_av_extend (in /home/martin/perl5/perlbrew/perls/p erl-5.17.4/bin/perl5.17.4) ==2984== by 0x8106AF1: Perl_stack_grow (in /home/martin/perl5/perlbrew/perls/ perl-5.17.4/bin/perl5.17.4) ==2984== by 0x810FAEB: Perl_pp_flop (in /home/martin/perl5/perlbrew/perls/per l-5.17.4/bin/perl5.17.4) ==2984== by 0x80DC2FA: Perl_runops_standard (in /home/martin/perl5/perlbrew/p erls/perl-5.17.4/bin/perl5.17.4) ==2984== by 0x806ACC1: Perl_list (in /home/martin/perl5/perlbrew/perls/perl-5 .17.4/bin/perl5.17.4) ==2984== by 0x806B19F: S_listkids (in /home/martin/perl5/perlbrew/perls/perl- 5.17.4/bin/perl5.17.4) ==2984== by 0x806AD56: Perl_list (in /home/martin/perl5/perlbrew/perls/perl-5 .17.4/bin/perl5.17.4) ==2984== by 0x806D49A: Perl_newASSIGNOP (in /home/martin/perl5/perlbrew/perls /perl-5.17.4/bin/perl5.17.4) ==2984== by 0x80A04DA: Perl_yyparse (in /home/martin/perl5/perlbrew/perls/per l-5.17.4/bin/perl5.17.4) ==2984== Seems to suggest File::Glob. Has worked with various Perls before 5.16.0. Loads of other people with different Perl versions on perl monks have found it fails too - see http://www.perlmonks.org/?node_id=994686 [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=library severity=high module=File::Glob --- Site configuration information for perl 5.17.4: Configured by martin at Thu Sep 20 16:35:47 BST 2012. Summary of my perl5 (revision 5 version 17 subversion 4) configuration: Platform: osname=linux, osvers=3.0.0-17-generic-pae, archname=i686-linux uname='linux bragi 3.0.0-17-generic-pae #30-ubuntu smp thu mar 8 17:53:35 utc 2012 i686 i686 i386 gnulinux ' config_args='-de -Dprefix=/home/martin/perl5/perlbrew/perls/perl-5.17.4 -Dusedevel' hint=recommended, useposix=true, d_sigaction=define useithreads=undef, usemultiplicity=undef useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=undef, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.6.1', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib/i386-linux-gnu /lib/../lib /usr/lib/i386-linux-gnu /usr/lib/../lib /lib /usr/lib libs=-lnsl -lgdbm -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.13' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector' Locally applied patches: --- @INC for perl 5.17.4: /home/martin/perl5/perlbrew/perls/perl-5.17.4/lib/site_perl/5.17.4/i686-linux /home/martin/perl5/perlbrew/perls/perl-5.17.4/lib/site_perl/5.17.4 /home/martin/perl5/perlbrew/perls/perl-5.17.4/lib/5.17.4/i686-linux /home/martin/perl5/perlbrew/perls/perl-5.17.4/lib/5.17.4 . --- Environment for perl 5.17.4: HOME=/home/martin LANG=en_GB.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/martin/bin:/home/martin/perl5/perlbrew/bin:/home/martin/perl5/perlbrew/perls/perl-5.17.4/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/opt/real/RealPlayer PERLBREW_BASHRC_VERSION=0.42 PERLBREW_HOME=/home/martin/.perlbrew PERLBREW_MANPATH=/home/martin/perl5/perlbrew/perls/perl-5.17.4/man PERLBREW_PATH=/home/martin/perl5/perlbrew/bin:/home/martin/perl5/perlbrew/perls/perl-5.17.4/bin PERLBREW_PERL=perl-5.17.4 PERLBREW_ROOT=/home/martin/perl5/perlbrew PERLBREW_VERSION=0.42 PERL_BADLANG (unset) SHELL=/bin/bash
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 570b
Here is the script in question: use strict; use warnings; use v5.16.0; my @md = (1..305); my @mp = (1000..1205); print "market detail: ", scalar(@md), "\n"; print "market price: ", scalar(@mp), "\n"; my $path = "/tmp/x"; mkdir $path or die "making $path, $!"; foreach (@md) { open(my $f, ">", "$path/md_$_.dat"); close $f; } foreach (@mp) { open(my $f, ">", "$path/mp_$_.dat"); close $f; } chdir $path or die "failed to chdir to $path"; my @b = glob(qq{$path/mp_[0123456789]*.dat $path/md_[0123456789]*.dat}); print scalar(@b), "\n";
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 770b
chromatic on perl monks posted the following patch (see link to perlmonks above) but I've not tested it yet myself: He said: I reproduced this with bleadperl. This patch fixes the problem and all core tests pass, but it doesn't quite pass my eyeball test. It's worth filing a bug with p5p. 001 diff --git a/ext/File-Glob/Glob.xs b/ext/File-Glob/Glob.xs 002 index 3ea0590..f8bc20f 100644 003 --- a/ext/File-Glob/Glob.xs 004 +++ b/ext/File-Glob/Glob.xs 005 @@ -237,8 +237,8 @@ csh_glob(pTHX_ AV *entries, SV *patsv) 006 SV **svp = AvARRAY(patav); 007 while (items--) { 008 PUSHMARK(SP); 009 - PUTBACK; 010 doglob(aTHX_ SvPVXx(*svp++), flags); 011 + PUTBACK; 012 SPAGAIN; 013 { 014 dMARK;
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.3k
On Thu Sep 20 11:21:55 2012, bohica wrote: Show quoted text
> chromatic on perl monks posted the following patch (see link to > perlmonks above) but I've not tested it yet myself: > > He said: > > I reproduced this with bleadperl. This patch fixes the problem and all > core tests pass, but it doesn't quite pass my eyeball test. It's worth > filing a bug with p5p. > > 001 diff --git a/ext/File-Glob/Glob.xs b/ext/File-Glob/Glob.xs > 002 index 3ea0590..f8bc20f 100644 > 003 --- a/ext/File-Glob/Glob.xs > 004 +++ b/ext/File-Glob/Glob.xs > 005 @@ -237,8 +237,8 @@ csh_glob(pTHX_ AV *entries, SV *patsv) > 006 SV **svp = AvARRAY(patav); > 007 while (items--) { > 008 PUSHMARK(SP); > 009 - PUTBACK; > 010 doglob(aTHX_ SvPVXx(*svp++), flags); > 011 + PUTBACK; > 012 SPAGAIN; > 013 { > 014 dMARK;
I’m the guilty part who wrote that code. I don’t think that patch is correct. But I still don’t know what the problem is. PUTBACK is necessary to make the current stack pointer visible to doglob(). Then SPAGAIN is necessary after the doglob call to get the new stack pointer, otherwise some elements returned from doglob will not be visible. And if doglob happens to reallocate the stack, we may end up reading freed memory. Doing PUTBACK and then SPAGAIN immediately is effectively a no-op here. -- Father Chrysostomos
Subject: Re: [perl #114984] "double free or corruption" "Invalid write of size 4" in File::Glob?
Date: Thu, 20 Sep 2012 13:18:20 -0700
To: perl5-porters [...] perl.org, perlbug-followup [...] perl.org
From: chromatic <chromatic [...] wgz.org>
Download (untitled) / with headers
text/plain 796b
On Thursday, September 20, 2012 01:08:29 PM Father Chrysostomos via RT wrote: Show quoted text
> I’m the guilty part who wrote that code. I don’t think that patch is > correct. But I still don’t know what the problem is.
That patch isn't correct. I tried it again and it's obviously wrong (which is good, because it looks wrong). Show quoted text
> PUTBACK is necessary to make the current stack pointer visible to > doglob(). Then SPAGAIN is necessary after the doglob call to get the > new stack pointer, otherwise some elements returned from doglob will not > be visible. And if doglob happens to reallocate the stack, we may end > up reading freed memory. Doing PUTBACK and then SPAGAIN immediately is > effectively a no-op here.
While running the example code, doglob happens to reallocate the stack. -- c
RT-Send-CC: perl5-porters [...] perl.org, chromatic [...] wgz.org
Download (untitled) / with headers
text/plain 1010b
On Thu Sep 20 13:20:08 2012, chromatic@wgz.org wrote: Show quoted text
> On Thursday, September 20, 2012 01:08:29 PM Father Chrysostomos via RT > wrote: >
> > I’m the guilty part who wrote that code. I don’t think that patch
> is
> > correct. But I still don’t know what the problem is.
> > That patch isn't correct. I tried it again and it's obviously wrong > (which is > good, because it looks wrong). >
> > PUTBACK is necessary to make the current stack pointer visible to > > doglob(). Then SPAGAIN is necessary after the doglob call to get
> the
> > new stack pointer, otherwise some elements returned from doglob will
> not
> > be visible. And if doglob happens to reallocate the stack, we may
> end
> > up reading freed memory. Doing PUTBACK and then SPAGAIN immediately
> is
> > effectively a no-op here.
> > While running the example code, doglob happens to reallocate the > stack.
iterate() is not extending the stack before copying to it. I have a patch that I am testing. -- Father Chrysostomos
RT-Send-CC: perl5-porters [...] perl.org, chromatic [...] wgz.org
Download (untitled) / with headers
text/plain 1.1k
On Thu Sep 20 14:26:33 2012, sprout wrote: Show quoted text
> On Thu Sep 20 13:20:08 2012, chromatic@wgz.org wrote:
> > On Thursday, September 20, 2012 01:08:29 PM Father Chrysostomos via RT > > wrote: > >
> > > I’m the guilty part who wrote that code. I don’t think that patch
> > is
> > > correct. But I still don’t know what the problem is.
> > > > That patch isn't correct. I tried it again and it's obviously wrong > > (which is > > good, because it looks wrong). > >
> > > PUTBACK is necessary to make the current stack pointer visible to > > > doglob(). Then SPAGAIN is necessary after the doglob call to get
> > the
> > > new stack pointer, otherwise some elements returned from doglob will
> > not
> > > be visible. And if doglob happens to reallocate the stack, we may
> > end
> > > up reading freed memory. Doing PUTBACK and then SPAGAIN immediately
> > is
> > > effectively a no-op here.
> > > > While running the example code, doglob happens to reallocate the > > stack.
> > iterate() is not extending the stack before copying to it. I have a > patch that I am testing.
Now applied as a6636b43dc. -- Father Chrysostomos
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 111b
See also bug #121602 which is a duplicate of this bug I found because the fix isn't applied to maint-5.16 yet.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org