Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"double free or corruption" "Invalid write of size 4" in File::Glob? #12430

Closed
p5pRT opened this issue Sep 20, 2012 · 10 comments
Closed

"double free or corruption" "Invalid write of size 4" in File::Glob? #12430

p5pRT opened this issue Sep 20, 2012 · 10 comments

Comments

@p5pRT
Copy link

p5pRT commented Sep 20, 2012

Migrated from rt.perl.org#114984 (status was 'resolved')

Searchable as RT114984$

@p5pRT
Copy link
Author

p5pRT commented Sep 20, 2012

From bohica@ntlworld.com

Created by bohica@ntlworld.com

*** glibc detected *** perl​: double free or corruption (!prev)​: 0x098b2180 ***

Self contained script below is the smallest I can come up with that illustrates this problem. I know it is rather specific. Changing almost any of the numbers of files created makes the problem go away. valgrind output suggests a "Invalid write of size 4". This reported via perlbug in 5.17.4 but it happens in 5.16.0 and 5.16.1 too.

*** glibc detected *** perl​: double free or corruption (!prev)​: 0x08e60240 ***
======= Backtrace​: =========
/lib/i386-linux-gnu/libc.so.6(+0x6ff22)[0xb75dbf22]
/lib/i386-linux-gnu/libc.so.6(+0x70bc2)[0xb75dcbc2]
/lib/i386-linux-gnu/libc.so.6(cfree+0x6d)[0xb75dfcad]
perl(Perl_av_extend_guts+0x18c)[0x80da7ec]
perl(Perl_av_extend+0x44)[0x80da984]
perl(Perl_stack_grow+0x32)[0x8106af2]
perl(Perl_pp_padav+0x114)[0x80f73d4]
perl(Perl_runops_standard+0xb)[0x80dc2fb]
perl(perl_run+0x325)[0x807ebc5]
perl(main+0x10d)[0x8061ced]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb7585113]
perl[0x8061d21]
======= Memory map​: ========
08048000-08196000 r-xp 00000000 08​:01 15452891 /home/martin/perl5/perlbrew/per
ls/perl-5.17.4/bin/perl5.17.4
08196000-08197000 r--p 0014d000 08​:01 15452891 /home/martin/perl5/perlbrew/per
ls/perl-5.17.4/bin/perl5.17.4
08197000-0819a000 rw-p 0014e000 08​:01 15452891 /home/martin/perl5/perlbrew/per
ls/perl-5.17.4/bin/perl5.17.4
08e43000-08ec0000 rw-p 00000000 00​:00 0 [heap]
b7000000-b7021000 rw-p 00000000 00​:00 0
b7021000-b7100000 ---p 00000000 00​:00 0
b71b6000-b71d2000 r-xp 00000000 08​:01 12756973 /lib/i386-linux-gnu/libgcc_s.so
.1
b71d2000-b71d3000 r--p 0001b000 08​:01 12756973 /lib/i386-linux-gnu/libgcc_s.so
.1
b71d3000-b71d4000 rw-p 0001c000 08​:01 12756973 /lib/i386-linux-gnu/libgcc_s.so
.1
b71f3000-b71f8000 r-xp 00000000 08​:01 16385302 /home/martin/perl5/perlbrew/per
ls/perl-5.17.4/lib/5.17.4/i686-linux/auto/File/Glob/Glob.so
b71f8000-b71f9000 r--p 00005000 08​:01 16385302 /home/martin/perl5/perlbrew/per
ls/perl-5.17.4/lib/5.17.4/i686-linux/auto/File/Glob/Glob.so
b71f9000-b71fa000 rw-p 00006000 08​:01 16385302 /home/martin/perl5/perlbrew/per
ls/perl-5.17.4/lib/5.17.4/i686-linux/auto/File/Glob/Glob.so
b71fa000-b723a000 r--p 006a5000 08​:01 12665422 /usr/lib/locale/locale-archive
b723a000-b736b000 r--p 00446000 08​:01 12665422 /usr/lib/locale/locale-archive
b736b000-b756b000 r--p 00000000 08​:01 12665422 /usr/lib/locale/locale-archive
b756b000-b756c000 rw-p 00000000 00​:00 0
b756c000-b76e4000 r-xp 00000000 08​:01 12755031 /lib/i386-linux-gnu/libc-2.13.s
o
b76e4000-b76e6000 r--p 00178000 08​:01 12755031 /lib/i386-linux-gnu/libc-2.13.s
o
b76e6000-b76e7000 rw-p 0017a000 08​:01 12755031 /lib/i386-linux-gnu/libc-2.13.s
o
b76e7000-b76ea000 rw-p 00000000 00​:00 0
b76ea000-b76f2000 r-xp 00000000 08​:01 12755033 /lib/i386-linux-gnu/libcrypt-2.
13.so
b76f2000-b76f3000 r--p 00007000 08​:01 12755033 /lib/i386-linux-gnu/libcrypt-2.
13.so
b76f3000-b76f4000 rw-p 00008000 08​:01 12755033 /lib/i386-linux-gnu/libcrypt-2.
13.so
b76f4000-b771c000 rw-p 00000000 00​:00 0
b771c000-b7744000 r-xp 00000000 08​:01 12755035 /lib/i386-linux-gnu/libm-2.13.s
o
b7744000-b7745000 r--p 00028000 08​:01 12755035 /lib/i386-linux-gnu/libm-2.13.s
o
b7745000-b7746000 rw-p 00029000 08​:01 12755035 /lib/i386-linux-gnu/libm-2.13.s
o
b7746000-b7749000 r-xp 00000000 08​:01 12755034 /lib/i386-linux-gnu/libdl-2.13.
so
b7749000-b774a000 r--p 00002000 08​:01 12755034 /lib/i386-linux-gnu/libdl-2.13.
so
b774a000-b774b000 rw-p 00003000 08​:01 12755034 /lib/i386-linux-gnu/libdl-2.13.
so
b7769000-b776a000 r--p 003fc000 08​:01 12665422 /usr/lib/locale/locale-archive
b776a000-b776c000 rw-p 00000000 00​:00 0
b776c000-b776d000 r-xp 00000000 00​:00 0 [vdso]
b776d000-b778b000 r-xp 00000000 08​:01 12755028 /lib/i386-linux-gnu/ld-2.13.so
b778b000-b778c000 r--p 0001d000 08​:01 12755028 /lib/i386-linux-gnu/ld-2.13.so
b778c000-b778d000 rw-p 0001e000 08​:01 12755028 /lib/i386-linux-gnu/ld-2.13.so
bff89000-bffaa000 rw-p 00000000 00​:00 0 [stack]
Aborted

valgrind output​:

==2984== Invalid write of size 4
==2984== at 0x4029C3E​: memcpy (mc_replace_strmem.c​:635)
==2984== by 0x4032E16​: iterate (in /home/martin/perl5/perlbrew/perls/perl-5.1
7.4/lib/5.17.4/i686-linux/auto/File/Glob/Glob.so)
==2984== by 0x8119930​: Perl_pp_glob (in /home/martin/perl5/perlbrew/perls/per
l-5.17.4/bin/perl5.17.4)
==2984== by 0x40C8112​: (below main) (libc-start.c​:226)
==2984== Address 0x42b4568 is 0 bytes after a block of size 2,032 alloc'd
==2984== at 0x4028876​: malloc (vg_replace_malloc.c​:236)
==2984== by 0x80C0D48​: Perl_safesysmalloc (in /home/martin/perl5/perlbrew/per
ls/perl-5.17.4/bin/perl5.17.4)
==2984== by 0x80DA79F​: Perl_av_extend_guts (in /home/martin/perl5/perlbrew/pe
rls/perl-5.17.4/bin/perl5.17.4)
==2984== by 0x80DA983​: Perl_av_extend (in /home/martin/perl5/perlbrew/perls/p
erl-5.17.4/bin/perl5.17.4)
==2984== by 0x8106AF1​: Perl_stack_grow (in /home/martin/perl5/perlbrew/perls/
perl-5.17.4/bin/perl5.17.4)
==2984== by 0x810FAEB​: Perl_pp_flop (in /home/martin/perl5/perlbrew/perls/per
l-5.17.4/bin/perl5.17.4)
==2984== by 0x80DC2FA​: Perl_runops_standard (in /home/martin/perl5/perlbrew/p
erls/perl-5.17.4/bin/perl5.17.4)
==2984== by 0x806ACC1​: Perl_list (in /home/martin/perl5/perlbrew/perls/perl-5
.17.4/bin/perl5.17.4)
==2984== by 0x806B19F​: S_listkids (in /home/martin/perl5/perlbrew/perls/perl-
5.17.4/bin/perl5.17.4)
==2984== by 0x806AD56​: Perl_list (in /home/martin/perl5/perlbrew/perls/perl-5
.17.4/bin/perl5.17.4)
==2984== by 0x806D49A​: Perl_newASSIGNOP (in /home/martin/perl5/perlbrew/perls
/perl-5.17.4/bin/perl5.17.4)
==2984== by 0x80A04DA​: Perl_yyparse (in /home/martin/perl5/perlbrew/perls/per
l-5.17.4/bin/perl5.17.4)
==2984==

Seems to suggest File​::Glob.

Has worked with various Perls before 5.16.0.

Loads of other people with different Perl versions on perl monks have found it fails too - see http​://www.perlmonks.org/?node_id=994686

Perl Info

Flags:
    category=library
    severity=high
    module=File::Glob

Site configuration information for perl 5.17.4:

Configured by martin at Thu Sep 20 16:35:47 BST 2012.

Summary of my perl5 (revision 5 version 17 subversion 4) configuration:
   
  Platform:
    osname=linux, osvers=3.0.0-17-generic-pae, archname=i686-linux
    uname='linux bragi 3.0.0-17-generic-pae #30-ubuntu smp thu mar 8 17:53:35 utc 2012 i686 i686 i386 gnulinux '
    config_args='-de -Dprefix=/home/martin/perl5/perlbrew/perls/perl-5.17.4 -Dusedevel'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.6.1', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/i386-linux-gnu /lib/../lib /usr/lib/i386-linux-gnu /usr/lib/../lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.13'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Locally applied patches:
    


@INC for perl 5.17.4:
    /home/martin/perl5/perlbrew/perls/perl-5.17.4/lib/site_perl/5.17.4/i686-linux
    /home/martin/perl5/perlbrew/perls/perl-5.17.4/lib/site_perl/5.17.4
    /home/martin/perl5/perlbrew/perls/perl-5.17.4/lib/5.17.4/i686-linux
    /home/martin/perl5/perlbrew/perls/perl-5.17.4/lib/5.17.4
    .


Environment for perl 5.17.4:
    HOME=/home/martin
    LANG=en_GB.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/martin/bin:/home/martin/perl5/perlbrew/bin:/home/martin/perl5/perlbrew/perls/perl-5.17.4/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/opt/real/RealPlayer
    PERLBREW_BASHRC_VERSION=0.42
    PERLBREW_HOME=/home/martin/.perlbrew
    PERLBREW_MANPATH=/home/martin/perl5/perlbrew/perls/perl-5.17.4/man
    PERLBREW_PATH=/home/martin/perl5/perlbrew/bin:/home/martin/perl5/perlbrew/perls/perl-5.17.4/bin
    PERLBREW_PERL=perl-5.17.4
    PERLBREW_ROOT=/home/martin/perl5/perlbrew
    PERLBREW_VERSION=0.42
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Sep 20, 2012

From bohica@ntlworld.com

Here is the script in question​:

use strict;
use warnings;
use v5.16.0;

my @​md = (1..305);
my @​mp = (1000..1205);
print "market detail​: ", scalar(@​md), "\n";
print "market price​: ", scalar(@​mp), "\n";

my $path = "/tmp/x";
mkdir $path or die "making $path, $!";

foreach (@​md) {
  open(my $f, ">", "$path/md_$_.dat");
  close $f;
}

foreach (@​mp) {
  open(my $f, ">", "$path/mp_$_.dat");
  close $f;
}
chdir $path or die "failed to chdir to $path";
my @​b = glob(qq{$path/mp_[0123456789]*.dat
  $path/md_[0123456789]*.dat});
print scalar(@​b), "\n";

@p5pRT
Copy link
Author

p5pRT commented Sep 20, 2012

bohica@ntlworld.com - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Sep 20, 2012

From bohica@ntlworld.com

chromatic on perl monks posted the following patch (see link to
perlmonks above) but I've not tested it yet myself​:

He said​:

I reproduced this with bleadperl. This patch fixes the problem and all
core tests pass, but it doesn't quite pass my eyeball test. It's worth
filing a bug with p5p.

001 diff --git a/ext/File-Glob/Glob.xs b/ext/File-Glob/Glob.xs
002 index 3ea0590..f8bc20f 100644
003 --- a/ext/File-Glob/Glob.xs
004 +++ b/ext/File-Glob/Glob.xs
005 @​@​ -237,8 +237,8 @​@​ csh_glob(pTHX_ AV *entries, SV *patsv)
006 SV **svp = AvARRAY(patav);
007 while (items--) {
008 PUSHMARK(SP);
009 - PUTBACK;
010 doglob(aTHX_ SvPVXx(*svp++), flags);
011 + PUTBACK;
012 SPAGAIN;
013 {
014 dMARK;

@p5pRT
Copy link
Author

p5pRT commented Sep 20, 2012

From @cpansprout

On Thu Sep 20 11​:21​:55 2012, bohica wrote​:

chromatic on perl monks posted the following patch (see link to
perlmonks above) but I've not tested it yet myself​:

He said​:

I reproduced this with bleadperl. This patch fixes the problem and all
core tests pass, but it doesn't quite pass my eyeball test. It's worth
filing a bug with p5p.

001 diff --git a/ext/File-Glob/Glob.xs b/ext/File-Glob/Glob.xs
002 index 3ea0590..f8bc20f 100644
003 --- a/ext/File-Glob/Glob.xs
004 +++ b/ext/File-Glob/Glob.xs
005 @​@​ -237,8 +237,8 @​@​ csh_glob(pTHX_ AV *entries, SV *patsv)
006 SV **svp = AvARRAY(patav);
007 while (items--) {
008 PUSHMARK(SP);
009 - PUTBACK;
010 doglob(aTHX_ SvPVXx(*svp++), flags);
011 + PUTBACK;
012 SPAGAIN;
013 {
014 dMARK;

I’m the guilty part who wrote that code. I don’t think that patch is
correct. But I still don’t know what the problem is.

PUTBACK is necessary to make the current stack pointer visible to
doglob(). Then SPAGAIN is necessary after the doglob call to get the
new stack pointer, otherwise some elements returned from doglob will not
be visible. And if doglob happens to reallocate the stack, we may end
up reading freed memory. Doing PUTBACK and then SPAGAIN immediately is
effectively a no-op here.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Sep 20, 2012

From chromatic@wgz.org

On Thursday, September 20, 2012 01​:08​:29 PM Father Chrysostomos via RT wrote​:

I’m the guilty part who wrote that code. I don’t think that patch is
correct. But I still don’t know what the problem is.

That patch isn't correct. I tried it again and it's obviously wrong (which is
good, because it looks wrong).

PUTBACK is necessary to make the current stack pointer visible to
doglob(). Then SPAGAIN is necessary after the doglob call to get the
new stack pointer, otherwise some elements returned from doglob will not
be visible. And if doglob happens to reallocate the stack, we may end
up reading freed memory. Doing PUTBACK and then SPAGAIN immediately is
effectively a no-op here.

While running the example code, doglob happens to reallocate the stack.

-- c

@p5pRT
Copy link
Author

p5pRT commented Sep 20, 2012

From @cpansprout

On Thu Sep 20 13​:20​:08 2012, chromatic@​wgz.org wrote​:

On Thursday, September 20, 2012 01​:08​:29 PM Father Chrysostomos via RT
wrote​:

I’m the guilty part who wrote that code. I don’t think that patch
is
correct. But I still don’t know what the problem is.

That patch isn't correct. I tried it again and it's obviously wrong
(which is
good, because it looks wrong).

PUTBACK is necessary to make the current stack pointer visible to
doglob(). Then SPAGAIN is necessary after the doglob call to get
the
new stack pointer, otherwise some elements returned from doglob will
not
be visible. And if doglob happens to reallocate the stack, we may
end
up reading freed memory. Doing PUTBACK and then SPAGAIN immediately
is
effectively a no-op here.

While running the example code, doglob happens to reallocate the
stack.

iterate() is not extending the stack before copying to it. I have a
patch that I am testing.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Sep 20, 2012

From @cpansprout

On Thu Sep 20 14​:26​:33 2012, sprout wrote​:

On Thu Sep 20 13​:20​:08 2012, chromatic@​wgz.org wrote​:

On Thursday, September 20, 2012 01​:08​:29 PM Father Chrysostomos via RT
wrote​:

I’m the guilty part who wrote that code. I don’t think that patch
is
correct. But I still don’t know what the problem is.

That patch isn't correct. I tried it again and it's obviously wrong
(which is
good, because it looks wrong).

PUTBACK is necessary to make the current stack pointer visible to
doglob(). Then SPAGAIN is necessary after the doglob call to get
the
new stack pointer, otherwise some elements returned from doglob will
not
be visible. And if doglob happens to reallocate the stack, we may
end
up reading freed memory. Doing PUTBACK and then SPAGAIN immediately
is
effectively a no-op here.

While running the example code, doglob happens to reallocate the
stack.

iterate() is not extending the stack before copying to it. I have a
patch that I am testing.

Now applied as a6636b4.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Sep 20, 2012

@cpansprout - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this as completed Sep 20, 2012
@p5pRT
Copy link
Author

p5pRT commented Apr 8, 2014

From ambrus@math.bme.hu

See also bug #121602 which is a duplicate of this bug I found because the fix isn't applied to maint-5.16 yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant