From perlbug@ericgarland.com

Created by perlbug@ericgarland.com

This program will make perl 5.8.1 expand to using 350 mb on my linux
machine within 60 seconds​:

use threads;
use threads​::shared;
my @​shared : shared;
@​shared = (1) while (1);

On perl 5.8.1-1 in cygwin in Windows 2000 it will crash perl
within about 20 seconds. ActivePerl 5.8.1 isn't out but it crashes
5.8.0 build 806 just like cygwin's perl.

The ability to use threads is a major step forward for perl. With
it you can easliy use shared variables to tightly integrate
two separatly scheduled threads of execution to create complex
applications. There are a lot of things you can do with shared
scalars but shared arrays and queues are really key to this whole
threadding thing. I'm excited to see this functionality in perl
and I'm really excited to start using it but with this huge leak
I can't use it for anything but very small, infrequently used
arrays.

Am I doing someting wrong or is this a real bug? If it's a bug
are there any patches that I could try that could fix it?

Thanks!

  -Eric

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.8.1:

Configured by egarland at Sun Sep 28 11:14:35 EDT 2003.

Summary of my perl5 (revision 5.0 version 8 subversion 1) configuration:
  Platform:
    osname=linux, osvers=2.4.18-19.7.xsmp, archname=i686-linux-thread-multi-64int
    uname='linux fsdev2.ericgarland.com 2.4.18-19.7.xsmp #1 smp thu dec 12 07:56:58 est 2002 i686 unknown '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='2.96 20000731 (Red Hat Linux 7.1 2.96-98)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.2.4.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.2.4'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.1:
    /usr/local/lib/perl5/5.8.1/i686-linux-thread-multi-64int
    /usr/local/lib/perl5/5.8.1
    /usr/local/lib/perl5/site_perl/5.8.1/i686-linux-thread-multi-64int
    /usr/local/lib/perl5/site_perl/5.8.1
    /usr/local/lib/perl5/site_perl
    .


Environment for perl v5.8.1:
    HOME=/root
    LANG=en_US
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/egarland/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

From @lizmat

At 16​:27 +0000 9/28/03, Eric Garland (via RT) wrote​:

-----------------------------------------------------------------
[Please enter your report here]
This program will make perl 5.8.1 expand to using 350 mb on my linux
machine within 60 seconds​:

It takes a bit longer on my slower machines, but I have confirmed
this on both Mac OS X and Linux ;-( ;-( ;-(

Am I doing someting wrong or is this a real bug? If it's a bug
are there any patches that I could try that could fix it?

<plug mode="hopefully helpful">
Until this bug is fixed, use forks.pm (from CPAN). It is a drop-in
replacement for threads. Which should allow you to get something
working now without leaks (at the expense of CPU and latency) and
allow you to go back to the real thing once the bug is fixed.
</plug>

Liz

From @lizmat

At 16​:27 +0000 9/28/03, Eric Garland (via RT) wrote​:

use threads;
use threads​::shared;
my @​shared : shared;
@​shared = (1) while (1);

Upon further investigation, it appears that this is _not_ related to
the push / pop problem that existed in 5.8.0. The following code
does _not_ eat memory​:

use threads;
use threads​::shared;
my @​shared : shared;
while (1) {
  push( @​shared,1 );
  pop( @​shared );
}

which means that queues (which used to eat memory like hell) are safe
to use. Which is what you would use shared arrays for the most time
anyway. Right?

Liz

From @iabyn

On Sun, Sep 28, 2003 at 04​:27​:36PM -0000, Eric Garland wrote​:

This program will make perl 5.8.1 expand to using 350 mb on my linux
machine within 60 seconds​:

use threads;
use threads​::shared;
my @​shared : shared;
@​shared = (1) while (1);

It's leaking 1 shared-interpreter SV per iteration. Not sure what the SV
is yet. May not have time to investigate further.

Dave.

--
Thank God I'm an atheist.....

From enache@rdslink.ro

On Mon, Sep 29, 2003 a.d., Dave Mitchell wrote​:

On Sun, Sep 28, 2003 at 04​:27​:36PM -0000, Eric Garland wrote​:

This program will make perl 5.8.1 expand to using 350 mb on my linux
machine within 60 seconds​:

use threads;
use threads​::shared;
my @​shared : shared;
@​shared = (1) while (1);

It's leaking 1 shared-interpreter SV per iteration. Not sure what the SV
is yet. May not have time to investigate further.

Look at ext/threads/shared/shared.xs - sharedsv_elem_mg_STORE().
The svp got with 'av_fetch(..,1)' has its refcnt incremented from 1 to 2
by Perl_sharedsv_associate(). When the array is cleared, it won't be
freed.

Have fun,
Adi

From egarland@ericgarland.com

Elizabeth Mattijsen (via RT) wrote​:

At 16​:27 +0000 9/28/03, Eric Garland (via RT) wrote​:

use threads;
use threads​::shared;
my @​shared : shared;
@​shared = (1) while (1);

Upon further investigation, it appears that this is _not_ related to
the push / pop problem that existed in 5.8.0. The following code
does _not_ eat memory​:

use threads;
use threads​::shared;
my @​shared : shared;
while (1) {
push( @​shared,1 );
pop( @​shared );
}

which means that queues (which used to eat memory like hell) are safe
to use. Which is what you would use shared arrays for the most time
anyway. Right?

Aah! Excellent! I was assuming that the push/pop leak and this one were
the same thing but I tried the push/pop test and a queue enqueue/dequeue test
on both cygwin and linux and none of them leaked. It sounds like as long as I
stay away from explicitly setting a shared array my program won't leak in 5.8.1.
I should be able to work around that since most of what I want to do is
enqueue/dequeue type stuff.

Thanks!

From perlbug@wallofcuriosities.com

[elizabeth - Mon Sep 29 01​:31​:41 2003]​:

The following code does _not_ eat memory​:

use threads;
use threads​::shared;
my @​shared : shared;
while (1) {
push( @​shared,1 );
pop( @​shared );
}

the above code is fine, but the following code has the same eating
problem as the original code submitted for this bug​:

use strict;
use threads;
use threads​::shared;

while(1) {
  my @​shared : shared;
  my $var = 'blah';
  push(@​shared, $var);
 
  print scalar(@​shared) . "\n";
}

the array @​shared *should* go out of scope on each loop iteration and
in fact its value does​: the print statement prints '1' every time.
however, memory usage explodes to the same degree as before. adding an
explicit 'undef @​shared' to the end of the loop has no effect on memory
usage.

jack

From @iabyn

On Mon, Sep 29, 2003 at 08​:16​:22PM +0300, Enache Adrian wrote​:

On Mon, Sep 29, 2003 a.d., Dave Mitchell wrote​:

On Sun, Sep 28, 2003 at 04​:27​:36PM -0000, Eric Garland wrote​:

This program will make perl 5.8.1 expand to using 350 mb on my linux
machine within 60 seconds​:

use threads;
use threads​::shared;
my @​shared : shared;
@​shared = (1) while (1);

It's leaking 1 shared-interpreter SV per iteration. Not sure what the SV
is yet. May not have time to investigate further.

Look at ext/threads/shared/shared.xs - sharedsv_elem_mg_STORE().
The svp got with 'av_fetch(..,1)' has its refcnt incremented from 1 to 2
by Perl_sharedsv_associate(). When the array is cleared, it won't be
freed.

Turned out to be slightly different. The AV in the shared interpreter
wasn't geting its AvREAL flag set, so when cleared, it wasn't
REFCNTdec-ing its elements.

Fixed in bleedperl by change #21527.

Dave.

--
Please note that ash-trays are provided for the use of smokers,
whereas the floor is provided for the use of all patrons.
  - Bill Royston

From nick.ing-simmons@elixent.com

Dave Mitchell <davem@​fdgroup.com> writes​:

Turned out to be slightly different. The AV in the shared interpreter
wasn't geting its AvREAL flag set, so when cleared, it wasn't
REFCNTdec-ing its elements.

My fault I expect - I have never really grasped what AvREAL means.
Would you care to contribute a patch to one of the PODs (or even
comments in av.h) that describes what AvREAL/AvREALISH _mean_ ?

I _should_ have caught the FREETMPS thing - those I do understand!

@iabyn - Status changed from 'new' to 'resolved'