Skip Menu |
Report information
Id: 44225
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: perl-5.8.0 [at] ton.iguana.be
pflanze <christian [at] pflanze.mine.nu>
reto.stamm [at] xilinx.com
xmltwig [at] gmail.com
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type:
Perl Version: (no value)
Fixed In: 5.15.0



Date: 3 May 2003 14:51:38 -0000
From: me-02 [...] ton.iguana.be
To: perlbug [...] perl.org
Subject: coredump in complex datastructure
Download (untitled) / with headers
text/plain 3.7k
This is a bug report for perl from perl-5.8.0@ton.iguana.be, generated with the help of perlbug 1.34 running under perl v5.8.0. ----------------------------------------------------------------- [Please enter your report here] The following program builds a double linked list, and then destroys it again. On exit it dumps core. Effect happens in many but not all versions off 5.8.0, no problems in 5.6.1. Increasing the 10000 to 1000000 makes more perl versions crash (e.g. my 5.6.1 begins crashing too). That version uses about 160M for me, so it's no out of memory effect. I was unable to reduce the program further. #! /usr/bin/perl -w my $head = {}; $head->{"next"} = $head; $head->{"previous"} = $head; for (1..10000) { my $node = {}; # attach node in front $prev = $head; my $next = $prev->{"next"}; $prev->{"next"} = $node; $node->{"next"} = $next; $node->{"previous"} = $prev; $next->{"previous"} = $node; } # Make nodes single connected $prev = $head; my $next = $prev->{"next"}; while ($next != $head) { undef($next->{"previous"}); $prev = $next; $next = $prev->{"next"}; } undef($prev->{"next"}); [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=medium --- Site configuration information for perl v5.8.0: Configured by ton at Tue Nov 12 01:56:18 CET 2002. Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration: Platform: osname=linux, osvers=2.4.19, archname=i686-linux-thread-multi-64int-ld uname='linux quasar 2.4.19 #5 wed oct 2 02:34:25 cest 2002 i686 unknown ' config_args='' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=define use64bitall=undef uselongdouble=define usemymalloc=y, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2 -fomit-frame-pointer', cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -I/usr/local/include' ccversion='', gccversion='2.95.3 20010315 (release)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long long', ivsize=8, nvtype='long double', nvsize=12, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lndbm -ldb -ldl -lm -lpthread -lc -lposix -lcrypt -lutil perllibs=-lnsl -ldl -lm -lpthread -lc -lposix -lcrypt -lutil libc=/lib/libc-2.2.4.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.2.4' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic' cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib' Locally applied patches: --- @INC for perl v5.8.0: /usr/lib/perl5/5.8.0/i686-linux-thread-multi-64int-ld /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i686-linux-thread-multi-64int-ld /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl . --- Environment for perl v5.8.0: HOME=/home/ton LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/ton/bin.Linux:/home/ton/bin:/home/ton/bin.SampleSetup:/usr/local/bin:/usr/local/sbin:/usr/local/jre/bin:/home/oracle/product/9.0.1/bin:/usr/local/ar/bin:/usr/games/bin:/usr/X11R6/bin:/usr/share/bin:/usr/bin:/usr/sbin:/bin:/sbin:. PERL_BADLANG (unset) SHELL=/bin/bash
Date: Mon, 5 May 2003 18:24:43 +0300
From: Enache Adrian <enache [...] rdslink.ro>
To: perl5-porters [...] perl.org
Subject: Re: [perl #22095] coredump in complex datastructure
RT-Send-Cc:
Download (untitled) / with headers
text/plain 687b
On Sat, May 03, 2003 at 02:53:08PM -0000, perl-5.8.0@ton.iguana.be (via RT) wrote: Show quoted text
> The following program builds a double linked list, and then > destroys it again. On exit it dumps core. Effect happens in > many but not all versions off 5.8.0, no problems in 5.6.1. > > Increasing the 10000 to 1000000 makes more perl versions > crash (e.g. my 5.6.1 begins crashing too). That version > uses about 160M for me, so it's no out of memory effect.
Simple hard stack overflow caused be deep recursion. (22095.pl below is your test with 10000 changed to 100000). Solution: adjust your limits. ex. $ perl 22095.pl Segmentation fault $ ulimit -s 40000 $ perl 22095.pl $ Regards, Adi
Subject: Re: [perl #22095] coredump in complex datastructure
Date: Mon, 5 May 2003 16:50:40 +0000 (UTC)
From: perl5-porters [...] ton.iguana.be (Ton Hospel)
To: perl5-porters [...] perl.org
RT-Send-Cc:
Download (untitled) / with headers
text/plain 1.5k
In article <20030505152443.GA2169@ratsnest.hole>, Enache Adrian <enache@rdslink.ro> writes: Show quoted text
> On Sat, May 03, 2003 at 02:53:08PM -0000, perl-5.8.0@ton.iguana.be (via RT) wrote:
>> The following program builds a double linked list, and then >> destroys it again. On exit it dumps core. Effect happens in >> many but not all versions off 5.8.0, no problems in 5.6.1. >> >> Increasing the 10000 to 1000000 makes more perl versions >> crash (e.g. my 5.6.1 begins crashing too). That version >> uses about 160M for me, so it's no out of memory effect.
> > Simple hard stack overflow caused be deep recursion. > (22095.pl below is your test with 10000 changed to 100000). > > Solution: adjust your limits. >
Mmm, since this happens in a module that should be always useable without doing external things, that's not so great as a general solution. I'll sever both links while walking the structure, which should also solve it though at a speed hit. Since my program has no recursion, I assume the recursion is happening inside perl, probabably code doing something like: dec_thingy { return if --self->refcount > 0; for (self->all_references) { dec_thingy($_); } free(self); } which would then be called recursively for as many times as my linear list is long ? Since e.g. on linux the default stack limit I get is only 8192, maybe worth considering making that iterative ? Otherwise this will generally bite if you drop a deep structure. This also begs the question why 5.6.1 on the same machine can handle much bigger structures before failing.
Date: Fri, 9 May 2003 11:12:39 +0200
Subject: Re: [perl #22095] coredump in complex datastructure
CC: perl5-porters [...] perl.org
To: perl5-porters [...] ton.iguana.be (Ton Hospel)
From: Arthur Bergman <sky [...] nanisky.com>
RT-Send-Cc:
Download (untitled) / with headers
text/plain 290b
On måndag, maj 5, 2003, at 18:50 Europe/Stockholm, Ton Hospel wrote: Show quoted text
> > Since e.g. on linux the default stack limit I get is only 8192, maybe > worth considering making that iterative ? Otherwise this will generally > bite if you drop a deep structure. > >
Patches welcome :) Arthur
Subject: Re: [perl #22095] coredump in complex datastructure
Date: Fri, 9 May 2003 13:20:17 +0000 (UTC)
From: perl5-porters [...] ton.iguana.be (Ton Hospel)
To: perl5-porters [...] perl.org
RT-Send-Cc:
Download (untitled) / with headers
text/plain 654b
In article <61BDB378-81FE-11D7-9B8A-000393CB5BC4@nanisky.com>, Arthur Bergman <sky@nanisky.com> writes: Show quoted text
> On måndag, maj 5, 2003, at 18:50 Europe/Stockholm, Ton Hospel wrote: >
>> >> Since e.g. on linux the default stack limit I get is only 8192, maybe >> worth considering making that iterative ? Otherwise this will generally >> bite if you drop a deep structure. >>
> Patches welcome :) >
Probably this thing should be looked into anyways even if it's kept recursive. 8192 (the default stack limit on my linux box) is 8 Meg, and the structure in my example is only 10000 entries. That seems to imply about 800 bytes per frame, which is excessive.
CC: paul.heffner [...] xilinx.com
Subject: Deep hash of hashes breaks garbage collector
Date: Wed, 22 Mar 2006 14:30:35 -0800
To: perlbug [...] perl.org
From: Reto Stamm <reto.stamm [...] xilinx.com>
Download (untitled) / with headers
text/plain 4.2k
This is a bug report for perl from reto@xilinx.com, generated with the help of perlbug 1.34 running under perl v5.8.0. ----------------------------------------------------------------- [Please enter your report here] The following script core dumps as it tries to collect $hash. # start of script { my $hash = {}; my $in_hash = $hash; for (my $i = 0; $i < 250000; $i ++) { $in_hash->{INSIDE} = {}; $in_hash = $in_hash->{INSIDE}; } } print "All cleaned up now."; # end of script This seems to happen on 32 bit Linux and 32 bit Solaris. Looking at the stack trace, it looks like it's analyzing the hash recursively. [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=high --- Site configuration information for perl v5.8.0: Configured by bhcompile' cf_email='bhcompile at Mon Sep 15 10:01:56 EDT 2003. Summary of my rderl (revision 5.0 version 8 subversion 0) configuration: Platform: osname=linux, osvers=2.4.21-1.1931.2.393.entsmp, archname=i386-linux-thread-multi uname='linux por' config_args='-des -Doptimize=-O2 -g -pipe -march=i386 -mcpu=i686 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux -Dvendorprefix=/usr -Dsiteprefix=/usr -Dotherlibdirs=/usr/lib/perl5/5.8.0 -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef' useithreads=define usemultiplicity= useperlio= d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=un uselongdouble= usemymalloc=, bincompat5005=undef Compiler: cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm', optimize='', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm' ccversion='', gccversion='3.2.3 20030502 (Red Hat Linux 3.2.3-19)', gccosandvers='' gccversion='3.2.3 200305' intsize=o, longsize=s, ptrsize=l, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long' k', ivsize=4' ivtype, nvtype='double' o_no', nvsize=, Off_t='', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='gcc' l', ldflags =' -L/usr/local/lib' ldflags_use' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil perllibs= libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libper gnulibc_version='2.3.2' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so', d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE' cccdlflags='-fPIC' ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5', lddlflags='s Unicode/Normalize XS/A' Locally applied patches: MAINT18379 --- @INC for perl v5.8.0: /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 . --- Environment for perl v5.8.0: HOME=/home/reto LANG=en_US.UTF-8 LANGUAGE (unset) LANGVAR=en_US.UTF-8 LD_LIBRARY_PATH=/home/reto/sb/HEAD/rtf/bin/lin:/build/xfndry/HEAD/rtf/bin/lin:/build/xfndry/HEAD/rtf/bin/lin LOGDIR (unset) PATH=.:/devl/swtools:/home/reto/gnu/bin:/home/reto/gnu/firefox:/usr/X/bin:/usr/kerberos/bin:/home/reto/sb/HEAD/rtf/bin/lin:/build/xfndry/HEAD/rtf/bin/lin:/home/reto/sb/HEAD/env/bin/lin:/build/xfndry/HEAD/env/bin/lin:/home/reto/sb/HEAD/rtf:/build/xfndry/HEAD/rtf:/build/xfndry/HEAD/rtf/bin/lin:/devl/swtools:/bin:/usr/bin:/usr/local/bin PERL_BADLANG (unset) SHELL=/usr/local/bin/tcsh dlflags='-share (unset)
CC: Reto Stamm (via RT) <perlbug-followup [...] perl.org>, bugs-bitbucket [...] rt.perl.org
Subject: Re: [perl #38786] Deep hash of hashes breaks garbage collector
Date: Wed, 22 Mar 2006 16:53:25 -0800
To: perl5-porters [...] perl.org
From: chromatic <chromatic [...] wgz.org>
Download (untitled) / with headers
text/plain 821b
On Wednesday 22 March 2006 14:43, Reto Stamm wrote: Show quoted text
> The following script core dumps as it tries to collect $hash. > > # start of script > { > my $hash = {}; > my $in_hash = $hash; > for (my $i = 0; $i < 250000; $i ++) { > $in_hash->{INSIDE} = {}; > $in_hash = $in_hash->{INSIDE}; > } > } > print "All cleaned up now."; > # end of script > > This seems to happen on 32 bit Linux and 32 bit Solaris. > > Looking at the stack trace, it looks like it's analyzing the hash > recursively.
Looking at the stack trace shows tens of thousands of calls. I think it's recursing past what the C stack can handle. There may be no easy fix, though rewriting the functions S_hfreeentries, Perl_hv_undef, Perl_sv_clear, Perl_sv_free2, and Perl_hv_free_ent for iteration instead of recursion might do it. -- c
Subject: Segfault in functional streams / bug in closure allocation?
Date: 28 Apr 2006 11:22:44 -0000
To: perlbug [...] perl.org
From: chris [...] elvis-jaeger.mine.nu
Download (untitled) / with headers
text/plain 5.4k
This is a bug report for perl from christian@pflanze.mine.nu, generated with the help of perlbug 1.35 running under perl v5.8.7. ----------------------------------------------------------------- [Please enter your report here] See the code below which segfaults. I don't see any cyclic reference, and running LengthOfStream (SequenceStream 15000) in an endless loop runs just fine without leaking, so cyclic references are not the problem. It is clearly related to the stack size, increasing the stack limit using ulimit raises the length of streams that can be processed. It should be possible to create such streams of infinite length, being built and deallocated while being traversed. This bug means that the functional stream programming paradigm (or probably anything else which creates closures in a repeated fashion?) cannot be applied to perl 5. use strict; sub Delay ( & ) { [ 0, $_[0] ] } sub Force ( $ ) { my $p=shift; $$p[0] ? $$p[1] : do { $$p[1]= &{ $$p[1] }; $$p[0]= 1; $$p[1] }; } sub Cons ( $ $ ) { [ @_ ] } sub Car ( $ ) { $_[0][0] } sub Cdr ( $ ) { $_[0][1] } our $Nil= []; sub LengthOfStream ( $ ) { my ($strm)=@_; my $len=0; while(1) { my $p= Force($strm); if ($p eq $Nil) { return $len; } else { $len++; $strm= Cdr($p); } } } sub SequenceStream ( $ ) { my ($n)=@_; Delay { ($n < 0) ? $Nil #: Cons $n, SequenceStream($n-1); : [ $n, SequenceStream($n-1) ];#no change }; } if (0) { while(1) { print LengthOfStream (SequenceStream 15000),"\n"; } } # above is no problem. but: if (0) { print LengthOfStream (SequenceStream 15000),"\n"; print LengthOfStream (SequenceStream 15000),"\n"; print LengthOfStream (SequenceStream 15000),"\n"; print LengthOfStream (SequenceStream 25000),"\n"; # now here a segfault happens print LengthOfStream (SequenceStream 25000),"\n"; } if (1) { my $s= (SequenceStream 100000); print LengthOfStream $s ,"\n"; print "Still alive\n"; undef $s;# <-- segfaults print "Still alive?\n"; } [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=medium --- Site configuration information for perl v5.8.7: Configured by Debian Project at Thu Jun 9 00:28:22 EST 2005. Summary of my perl5 (revision 5 version 8 subversion 7) configuration: Platform: osname=linux, osvers=2.4.27-ti1211, archname=i386-linux-thread-multi uname='linux kosh 2.4.27-ti1211 #1 sun sep 19 18:17:45 est 2004 i686 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.7 -Dsitearch=/usr/local/lib/perl/5.8.7 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.7 -Dd_dosuid -des' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include' ccversion='', gccversion='3.3.6 (Debian 1:3.3.6-6)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libperl.so.5.8.7 gnulibc_version='2.3.2' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib' Locally applied patches: --- @INC for perl v5.8.7: /etc/perl /usr/local/lib/perl/5.8.7 /usr/local/share/perl/5.8.7 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8 /usr/local/lib/site_perl /usr/local/lib/perl/5.8.4 /usr/local/share/perl/5.8.4 /usr/local/lib/perl/5.8.3 /usr/local/share/perl/5.8.3 . --- Environment for perl v5.8.7: HOME=/home/chris LANG=de_CH LANGUAGE (unset) LC_CTYPE=de_CH LC_NUMERIC=C LD_LIBRARY_PATH=:/home/chris/lib LOGDIR (unset) PATH=/usr/local/Gambit-C/bin:/opt/j2sdk_nb/j2sdk1.4.2/bin/:/home/chris/local/bin:/home/chris/bin:/root/local/bin:/root/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/bin/X11:/usr/local/sbin:/usr/local/bin:/sbin:/usr/sbin:/usr/local/sbin PERL_BADLANG (unset) SHELL=/bin/bash
Download (untitled) / with headers
text/plain 1.6k
(It looks like the below text which I already sent as followup somehow got lost, so I'm posting it through RT again.) It seems like it might be a problem with parameter passing through @_. The following change prevents the segfault in all cases. # either use this definition instead: # sub LengthOfStream ( $ ) { # my ($strm)=@_; # #@_=();# does not help # undef $_[0];# <------ helps # my $len=0; # while(1) { # my $p= Force($strm); # if ($p eq $Nil) { # return $len; # } else { # $len++; # $strm= Cdr($p); # } # } # } # or this: sub LengthOfStream_ { my ($strm,$len)=@_; #@_=(); # does not help undef $_[0];# helps. my $p= Force($strm); if ($p eq $Nil) { $len } else { @_=(Cdr($p), $len+1); goto \&LengthOfStream; # tail call } } sub LengthOfStream ( $ ) { push @_,0; goto \&LengthOfStream_; } But it has, of course, the ugly side effect of erasing the variable being passed by the caller: my $s= SequenceStream 100000; print LengthOfStream $s,"\n"; # -> 100001 # but now $s is undef I find it strange, that this case also requires the undef $_[0] (and @_=() is not enough!): print LengthOfStream SequenceStream 100000,"\n"; Any idea? What I also don't understand is why this leaks memory and eventually terminates because of an out of memory condition (it should just run forever): sub InfiniteStream { my ($startnumber)=@_; #undef $_[0]; Modification of a read-only value attempted -- of course. #@_=(); Delay { Cons $startnumber, InfiniteStream($startnumber+1); }; } LengthOfStream InfiniteStream(0); Any idea here? Christian.
Subject: Re: [perl #39017] Segfault in functional streams / bug in closure allocation?
Date: Sat, 29 Apr 2006 00:54:13 +0100
To: perl5-porters [...] perl.org
From: Dave Mitchell <davem [...] iabyn.com>
Download (untitled) / with headers
text/plain 693b
On Fri, Apr 28, 2006 at 04:23:49AM -0700, Christian wrote: Show quoted text
> See the code below which segfaults.
It's not actually closure-related; it's just that perl currently uses a recursive method to free recursive data structures, which can blow the C stack on deeply nested ones. For example the following will crash perl just as happily as your example: my $s; for (1..100_000) { $s = [$_, $s ]; } undef $s; It's on our list of things to fix one day... -- Spock (or Data) is fired from his high-ranking position for not being able to understand the most basic nuances of about one in three sentences that anyone says to him. -- Things That Never Happen in "Star Trek" #19
CC: perlbug-followup [...] perl.org
Subject: Re: [perl #39017] perlbug AutoReply: Segfault in functional streams / bug in closure allocation?
Date: Sat, 29 Apr 2006 01:30:50 +0100
To: Christian <christian [...] pflanze.mine.nu>
From: Dave Mitchell <davem [...] iabyn.com>
Download (untitled) / with headers
text/plain 479b
On Fri, Apr 28, 2006 at 02:25:25PM +0200, Christian wrote: Show quoted text
> It seems like it might be a problem with parameter passing through > @_. The following change prevents the segfault in all cases. > sub LengthOfStream_ { > my ($strm,$len)=@_; > #@_=(); # does not help > undef $_[0];# helps.
This is avoiding the segfault by avoidly building a recursive data structure. -- The optimist believes that he lives in the best of all possible worlds. As does the pessimist.
Subject: Re: [perl #39017] perlbug AutoReply: Segfault in functional streams / bug in closure allocation?
Date: Fri, 28 Apr 2006 14:25:25 +0200
To: perlbug-followup [...] perl.org
From: Christian <christian [...] pflanze.mine.nu>
Download (untitled) / with headers
text/plain 1.5k
It seems like it might be a problem with parameter passing through @_. The following change prevents the segfault in all cases. # either use this definition instead: # sub LengthOfStream ( $ ) { # my ($strm)=@_; # #@_=();# does not help # undef $_[0];# <------ helps # my $len=0; # while(1) { # my $p= Force($strm); # if ($p eq $Nil) { # return $len; # } else { # $len++; # $strm= Cdr($p); # } # } # } # or this: sub LengthOfStream_ { my ($strm,$len)=@_; #@_=(); # does not help undef $_[0];# helps. my $p= Force($strm); if ($p eq $Nil) { $len } else { @_=(Cdr($p), $len+1); goto \&LengthOfStream; # tail call } } sub LengthOfStream ( $ ) { push @_,0; goto \&LengthOfStream_; } But it has, of course, the ugly side effect of erasing the variable being passed by the caller: my $s= SequenceStream 100000; print LengthOfStream $s,"\n"; # -> 100001 # but now $s is undef I find it strange, that this case also requires the undef $_[0] (and @_=() is not enough!): print LengthOfStream SequenceStream 100000,"\n"; Any idea? What I also don't understand is why this leaks memory and eventually terminates because of an out of memory condition (it should just run forever): sub InfiniteStream { my ($startnumber)=@_; #undef $_[0]; Modification of a read-only value attempted -- of course. #@_=(); Delay { Cons $startnumber, InfiniteStream($startnumber+1); }; } LengthOfStream InfiniteStream(0); Any idea here? Christian.
Subject: perl segfaults when a variable is destroyed after using weaken a large number of times
Date: Thu, 19 Jul 2007 03:40:04 +0200
To: perlbug [...] perl.org, xmltwig [...] gmail.com
From: "Michel Rodriguez" <xmltwig [...] gmail.com>
Download (untitled) / with headers
text/plain 4.4k
This is a bug report for perl from xmltwig@gmail.com, generated with the help of perlbug 1.35 running under perl v5.8.8. ----------------------------------------------------------------- [Please enter your report here] I create a tree with a single root and thousands of children. The link from the root to the first child is strong, as is the link from each node to its next sibling. The link from each node to the root is weakened. The program crashes when the root goes out of scope. See the attached code which tries to find the exact number. Change the $NB_ELT_INIT and the $STEP values to find it. Note that the limit can vary (23790 and 23810 in my limited tests). I had the same problem with XML::Twig, which also builds a tree, at about the same number of nodes. Twig uses weaken several times for each node though, so that might be an element. Note that it is not the total number of weaken that counts, just the number in the block in which the variables are in scope. This behaviour happens in Scalar::Util 1.18 and 1.19 perl -v: This is perl, v5.8.8 built for i486-linux-gnu-thread-multi I have also had reports of bugs in XML::Twig that look suspiciously like they could come from this problem, on various machines (just today: Solaris in c.l.p.m, perl 5.8.5 and 5.8.8, and perl v5.8.7 built for i386-freebsd-64int by mail). I hope that helps -- mirod [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=medium --- Site configuration information for perl v5.8.8: Configured by Debian Project at Tue Mar 6 01:52:23 UTC 2007. Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=linux, osvers=2.6.15.7, archname=i486-linux-gnu-thread-multi uname='linux rothera 2.6.15.7 #1 smp sat sep 30 10:21:42 utc 2006 i686 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.8 -Dsitearch=/usr/local/lib/perl/5.8.8 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.8 -Dd_dosuid -des' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include' ccversion='', gccversion='4.1.2 (Ubuntu 4.1.2-0ubuntu4)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=/lib/libc-2.5.so, so=so, useshrplib=true, libperl=libperl.so.5.8.8 gnulibc_version='2.5' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib' Locally applied patches: --- @INC for perl v5.8.8: /etc/perl /usr/local/lib/perl/5.8.8 /usr/local/share/perl/5.8.8 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8 /usr/local/lib/site_perl /usr/local/lib/perl/5.8.7 /usr/local/share/perl/5.8.7 . --- Environment for perl v5.8.8: HOME=/home/mrodrigu LANG=en_US.UTF-8 LANGUAGE=en_IT:en LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/mrodrigu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games:/home/mrodrigu/bin PERL_BADLANG (unset) SHELL=/bin/bash
Download gen_internal_big_doc
application/octet-stream 751b

Message body not shown because it is not plain text.

Subject: Re: [perl #44225] perl segfaults when a variable is destroyed after using weaken a large number of times
Date: Sun, 29 Jul 2007 11:29:42 +0200
To: perl5-porters [...] perl.org
From: andreas.koenig.7os6VVqR [...] franz.ak.mind.de (Andreas J. Koenig)
Download (untitled) / with headers
text/plain 495b
Show quoted text
>>>>> On Sat, 28 Jul 2007 11:56:28 -0700, "Michel Rodriguez" (via RT) <perlbug-followup@perl.org> said:
Show quoted text
mr> The program crashes when the root goes out of scope.
Show quoted text
mr> See the attached code which tries to find the exact number. Change the mr> $NB_ELT_INIT and the $STEP values to find it.
Might be architecture dependent. I cannot reproduce under linux. I increased $NB_ELT_INIT to 100000. Program ran to an end without a crash with 5.6.2, 5.8.7, 5.8.8, maint-5.8, blead@31663. -- andreas
CC: "Andreas J. Koenig" <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Subject: Re: [perl #44225] perl segfaults when a variable is destroyed after using weaken a large number of times
Date: Sun, 29 Jul 2007 11:38:21 +0200
To: perl5-porters [...] perl.org
From: Tels <nospam-abuse [...] bloodgate.com>
Download (untitled) / with headers
text/plain 4.4k
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Moin, On Sunday 29 July 2007 11:29:42 Andreas J. Koenig wrote: Show quoted text
> >>>>> On Sat, 28 Jul 2007 11:56:28 -0700, "Michel Rodriguez" (via RT) > >>>>> <perlbug-followup@perl.org> said:
>
> mr> The program crashes when the root goes out of scope.
>
> mr> See the attached code which tries to find the exact number. Change
> the mr> $NB_ELT_INIT and the $STEP values to find it. > > Might be architecture dependent. I cannot reproduce under linux. I > increased $NB_ELT_INIT to 100000. Program ran to an end without a > crash with 5.6.2, 5.8.7, 5.8.8, maint-5.8, blead@31663.
Note that the submitter had threads :) Reproducible here: # perl bug.pl tree created tree out of scope OK (10000) tree created tree out of scope OK (15000) tree created tree out of scope OK (20000) tree created Segmentation fault # te@linux:~> perl -v This is perl, v5.8.8 built for x86_64-linux-thread-multi Copyright 1987-2006, Larry Wall All the best, Tels Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=linux, osvers=2.6.16, archname=x86_64-linux-thread-multi uname='linux dvorak 2.6.16 #1 smp mon apr 10 04:51:13 utc 2006 x86_64 x86_64 x86_64 gnulinux ' config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Duseshrplib=true -Doptimize=-O2 -fmessage-length=0 -Wall -D_FORTIFY_SOURCE=2 -g -Wall -pipe' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -pipe -Wdeclaration-after-statement -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2 -fmessage-length=0 -Wall -D_FORTIFY_SOURCE=2 -g -Wall -pipe', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -pipe -Wdeclaration-after-statement' ccversion='', gccversion='4.1.0 (SUSE Linux)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib64' libpth=/lib64 /usr/lib64 /usr/local/lib64 libs=-lm -ldl -lcrypt -lpthread perllibs=-lm -ldl -lcrypt -lpthread libc=/lib64/libc-2.4.so, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.4' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/CORE' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64' Characteristics of this binary (from libperl): Compile-time options: DEBUGGING MULTIPLICITY PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP THREADS_HAVE_PIDS USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_REENTRANT_API Built under linux Compiled at Apr 22 2006 23:33:01 @INC: /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi /usr/lib/perl5/5.8.8 /usr/lib/perl5/site_perl/5.8.8/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl . - -- Signed on Sun Jul 29 11:36:29 2007 with key 0x93B84C15. View my photo gallery: http://bloodgate.com/photos PGP key on http://bloodgate.com/tels.asc or per email. "Most people, I think, don't even know what a rootkit is, so why should they care about it?" -- Thomas Hesse, President of Sony BMG's global digital business division, 2005. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (GNU/Linux) iQEVAwUBRqxgDXcLPEOTuEwVAQIUggf+LZevVglAJVgseFy2lG46cmJ3Tu8una/V RE9tFLSOWYAwNdj1RFspbmRT5qETWtkPGCfND4FpMnCna6Wv06BgdxlMli5RmJcT ynnYya7c3Sat3jxF2tSk9yxIHphgr2TJ8IqB4M/9VmB10rugYcPBF9lU0jI2U0of +MkbRQIHYEjLNJqEyr8BvXCnRE5OVrHJzkWOwy8kdlrsTFuFnzn6b8l+cz6FVk4H pt819cijRz3QAqdlwfcJNwue0+cRdfKsCpkdoSsOR24KaUZXSFDVr5V47mhRCJio f/UWTGWJrkPPFs5QLbvLEIOSXl0ZnlUhppfBBMTtbN1FaB7r9ssBtw== =p8c9 -----END PGP SIGNATURE-----
Subject: Re: [perl #44225] perl segfaults when a variable is destroyed after using weaken a large number of times
Date: Sun, 29 Jul 2007 16:16:15 +0200
To: perl5-porters [...] perl.org
From: andreas.koenig.7os6VVqR [...] franz.ak.mind.de (Andreas J. Koenig)
Download (untitled) / with headers
text/plain 2.5k
Show quoted text
>>>>> On Sun, 29 Jul 2007 11:38:21 +0200, Tels <nospam-abuse@bloodgate.com> said:
Show quoted text
> Note that the submitter had threads :)
I cannot reproduce with threaded perls either. Show quoted text
> gnulibc_version='2.4'
I have 2.6 and this seems to be the key. I can reproduce it with bleadperl@31663 (without threads) on a box with gnulibc 2.5. The stack trace on that other box says: #0 0x080e538b in Perl_hv_undef (hv=0x850c600) at hv.c:1850 #1 0x0813363b in Perl_sv_clear (sv=0x850c600) at sv.c:5156 #2 0x081341d9 in Perl_sv_free2 (sv=0x850c600) at sv.c:5338 #3 0x0813413b in Perl_sv_free (sv=0x850c600) at sv.c:5317 #4 0x08133f3f in Perl_sv_clear (sv=0x850c628) at sv.c:5206 #5 0x081341d9 in Perl_sv_free2 (sv=0x850c628) at sv.c:5338 #6 0x0813413b in Perl_sv_free (sv=0x850c628) at sv.c:5317 #7 0x080e4866 in Perl_hv_free_ent (hv=0x850c5c4, entry=0x850b290) at hv.c:1527 #8 0x080e51c2 in S_hfreeentries (hv=0x850c5c4) at hv.c:1783 #9 0x080e5390 in Perl_hv_undef (hv=0x850c5c4) at hv.c:1850 #10 0x0813363b in Perl_sv_clear (sv=0x850c5c4) at sv.c:5156 #11 0x081341d9 in Perl_sv_free2 (sv=0x850c5c4) at sv.c:5338 #12 0x0813413b in Perl_sv_free (sv=0x850c5c4) at sv.c:5317 #13 0x08133f3f in Perl_sv_clear (sv=0x850c5ec) at sv.c:5206 #14 0x081341d9 in Perl_sv_free2 (sv=0x850c5ec) at sv.c:5338 #15 0x0813413b in Perl_sv_free (sv=0x850c5ec) at sv.c:5317 #16 0x080e4866 in Perl_hv_free_ent (hv=0x850c588, entry=0x850b278) at hv.c:1527 #17 0x080e51c2 in S_hfreeentries (hv=0x850c588) at hv.c:1783 #18 0x080e5390 in Perl_hv_undef (hv=0x850c588) at hv.c:1850 #19 0x0813363b in Perl_sv_clear (sv=0x850c588) at sv.c:5156 #20 0x081341d9 in Perl_sv_free2 (sv=0x850c588) at sv.c:5338 #21 0x0813413b in Perl_sv_free (sv=0x850c588) at sv.c:5317 #22 0x08133f3f in Perl_sv_clear (sv=0x850c5b0) at sv.c:5206 #23 0x081341d9 in Perl_sv_free2 (sv=0x850c5b0) at sv.c:5338 #24 0x0813413b in Perl_sv_free (sv=0x850c5b0) at sv.c:5317 #25 0x080e4866 in Perl_hv_free_ent (hv=0x850c54c, entry=0x850b260) at hv.c:1527 #26 0x080e51c2 in S_hfreeentries (hv=0x850c54c) at hv.c:1783 #27 0x080e5390 in Perl_hv_undef (hv=0x850c54c) at hv.c:1850 #28 0x0813363b in Perl_sv_clear (sv=0x850c54c) at sv.c:5156 #29 0x081341d9 in Perl_sv_free2 (sv=0x850c54c) at sv.c:5338 #30 0x0813413b in Perl_sv_free (sv=0x850c54c) at sv.c:5317 #31 0x08133f3f in Perl_sv_clear (sv=0x850c574) at sv.c:5206 #32 0x081341d9 in Perl_sv_free2 (sv=0x850c574) at sv.c:5338 #33 0x0813413b in Perl_sv_free (sv=0x850c574) at sv.c:5317 #34 0x080e4866 in Perl_hv_free_ent (hv=0x850c510, entry=0x850b248) at hv.c:1527 [da capo ad libitum] -- andreas
Subject: Re: [perl #44225] perl segfaults when a variable is destroyed after using weaken a large number of times
Date: Sun, 29 Jul 2007 18:58:07 +0200
To: perl5-porters [...] perl.org
From: Bo Lindbergh <blgl [...] hagernas.com>
Download (untitled) / with headers
text/plain 419b
This looks like a stack overflow caused by the recursive implementation of the reference counting scheme. You can evoke it with any sufficiently long chain of references, no weakening required: for (my $len=1; ; $len<<=1) { my $list; print STDERR "list length $len\n"; foreach (1..$len) { $list={ next => $list, }; } } /Bo Lindbergh
Download (untitled) / with headers
text/plain 497b
The issues related to perl seg-faulting when freeing deeply nested structures should now be resolved; first by the series of commits in oct 2010 that made AVs and RVs non-recursive: 5239d5c4bfde4ec02e1787e9dc9ada189ad868e5 df90f6afd9a2814197a1dd002454410c69b7fed6 b98b62bccdcb420ec5430eb831023e3d91ab2fa0 followed in May 2011 by the series of commits that made HVs non-recursive too: afbbf215a9f8470ac637c1eddc1d4f0eaa8ab090 through c2217cd33590ef654504deb38dcf3378dcdc095c


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org