From deniz.yuret@alum.mit.edu

  split(/\W+/);

by itself in one of my functions that is called thousands of times.
When I changed it to​:

  my(@​tok) = split(/\W+/);

The leak stopped. I discovered that the following also stops the
leak​:

  for(split(/\W+/)) { ... }

i.e. Having no explicit variable but running split in the context of
the for expression.

This may be the normal behavior and I may not understand the memory
mechanism exactly. But one would think @​_ could be garbage collected
on exit from the function.

Here is my configuration info from perl -V​:

Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration​:
  Platform​:
  osname=linux, osvers=2.2.1-ac1, archname=i386-linux
  uname='linux porky.devel.redhat.com 2.2.1-ac1 #1 smp mon feb 1 17​:44​:44 est 1999 i686 unknown '
  hint=recommended, useposix=true, d_sigaction=define
  usethreads=undef useperlio=undef d_sfio=undef
  Compiler​:
  cc='cc', optimize='-O2', gccversion=egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)
  cppflags='-Dbool=char -DHAS_BOOL -I/usr/local/include'
  ccflags ='-Dbool=char -DHAS_BOOL -I/usr/local/include'
  stdchar='char', d_stdstdio=undef, usevfork=false
  intsize=4, longsize=4, ptrsize=4, doublesize=8
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
  alignbytes=4, usemymalloc=n, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -L/usr/local/lib'
  libpth=/usr/local/lib /lib /usr/lib
  libs=-lnsl -lndbm -lgdbm -ldb -ldl -lm -lc -lposix -lcrypt
  libc=, so=so, useshrplib=false, libperl=libperl.a
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
  cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Characteristics of this binary (from libperl)​:
  Built under linux
  Compiled at Apr 6 1999 23​:34​:07
  %ENV​:
  PERL5LIB="/home/deniz/local/lib/perl5/site_perl/5.005"
  @​INC​:
  /home/deniz/local/lib/perl5/site_perl/5.005
  /usr/lib/perl5/5.00503/i386-linux
  /usr/lib/perl5/5.00503
  /usr/lib/perl5/site_perl/5.005/i386-linux
  /usr/lib/perl5/site_perl/5.005
  .

From [Unknown Contact. See original ticket]

Deniz Yuret writes​:

I had a memory leak in my perl program. I traced it down to the
function split. Specifically I had the expression​:
split(/\W+/);

Implicit split to @​_ leaks, confirmed in _60.

Nat

From [Unknown Contact. See original ticket]

At 03​:33 PM 9/6/99 -0600, Nathan Torkington wrote​:

Deniz Yuret writes​:

I had a memory leak in my perl program. I traced it down to the
function split. Specifically I had the expression​:
split(/\W+/);

Implicit split to @​_ leaks, confirmed in _60.

Now that's interesting. I wonder if it's related in any way to the @​_ leaks
in threaded perl? (It seems that the elements of @​_ get over-refcounted
when passed into a thread with Thread​::new, at least in some circumstances.
Still haven't gotten a chance to track it down yet)

  Dan

----------------------------------------"it's like this"-------------------
Dan Sugalski even samurai
dan@​sidhe.org have teddy bears
  and even the teddy bears
  get drunk

From [Unknown Contact. See original ticket]

Dan Sugalski writes​:

At 03​:33 PM 9/6/99 -0600, Nathan Torkington wrote​:

Deniz Yuret writes​:

I had a memory leak in my perl program. I traced it down to the
function split. Specifically I had the expression​:
split(/\W+/);

Implicit split to @​_ leaks, confirmed in _60.

Now that's interesting. I wonder if it's related in any way to the @​_ leaks
in threaded perl? (It seems that the elements of @​_ get over-refcounted
when passed into a thread with Thread​::new, at least in some circumstances.
Still haven't gotten a chance to track it down yet)

Well, this is very easy to reproduce. Some older Perls allowed it as
simple as

perl -MDevel​::Peek -wle 'sub a {undef %a; Dump \@​_,5} $a{a} = 1; a($a{a})'
SV = UNKNOWN(0x4bb74)
  REFCNT = 0
  FLAGS = ()

[Here the corruption is obvious due to UNKNOWN and REFCNT=0]

Newer Perls have a slightly different manifestation of memory corruption​:

perl -MDevel​::Peek -wle 'sub a {undef %a; Dump \@​_,5} $a{a} = 1; a($a{a})'
SV = RV(0x131c08) at 0x104954
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x11137c
  SV = PVAV(0x11cb50) at 0x11137c
  REFCNT = 3
  FLAGS = ()
  IV = 0
  NV = 0
  ARRAY = 0x105bf0
  FILL = 0
  MAX = 3
  ARYLEN = 0x0
  FLAGS = (REIFY)
  Elt No. 0
  SV = RV(0x131c08) at 0x104954
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x11137c
  SV = PVAV(0x11cb50) at 0x11137c
  REFCNT = 3
  FLAGS = ()
  IV = 0
  NV = 0
  ARRAY = 0x105bf0
  FILL = 0
  MAX = 3
  ARYLEN = 0x0
  FLAGS = (REIFY)
  Elt No. 0
  SV = RV(0x131c08) at 0x104954
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x11137c
  SV = PVAV(0x11cb50) at 0x11137c
  REFCNT = 3
  FLAGS = ()
  IV = 0
  NV = 0
  ARRAY = 0x105bf0
  FILL = 0
  MAX = 3
  ARYLEN = 0x0
  FLAGS = (REIFY)

Note the infinite loop in @​_.

Ilya

From [Unknown Contact. See original ticket]

On Tue, 7 Sep 1999, Nathan Torkington wrote​:

Ilya Zakharevich writes​:

Well, this is very easy to reproduce. Some older Perls allowed it as
simple as

perl -MDevel​::Peek -wle 'sub a {undef %a; Dump \@​_,5} $a{a} = 1; a($a{a})'
SV = UNKNOWN(0x4bb74)
REFCNT = 0
FLAGS = ()

This seems to be a different problem from the one that Dan was
describing. He was talking about a memory leak, where scalars are
over-refcounted. You're showing under-refcounting.

Well, it's possible that the thread problem is an under-refcount.
Decrementing that zero'd wrap around to a way-too-huge number and
effectively immortalize the scalars. (I still need to go digging for
what's actually happening, so until then I'm just talking through my
hat...)

  Dan

From [Unknown Contact. See original ticket]

Ilya Zakharevich writes​:

Well, this is very easy to reproduce. Some older Perls allowed it as
simple as

perl -MDevel​::Peek -wle 'sub a {undef %a; Dump \@​_,5} $a{a} = 1; a($a{a})'
SV = UNKNOWN(0x4bb74)
REFCNT = 0
FLAGS = ()

This seems to be a different problem from the one that Dan was
describing. He was talking about a memory leak, where scalars are
over-refcounted. You're showing under-refcounting.

How long have the bugs you've shown been in Perl, Ilya? Are they
recent introductions or long-standing nastinesses?

Nat

From [Unknown Contact. See original ticket]

Nathan Torkington writes​:

Well, this is very easy to reproduce. Some older Perls allowed it as
simple as

perl -MDevel​::Peek -wle 'sub a {undef %a; Dump \@​_,5} $a{a} = 1; a($a{a})'
SV = UNKNOWN(0x4bb74)
REFCNT = 0
FLAGS = ()

This seems to be a different problem from the one that Dan was
describing. He was talking about a memory leak, where scalars are
over-refcounted. You're showing under-refcounting.

This is the *same* problem. Under-refcount happens because REFCNT is
not _inc when you enter a subroutine. Over-refcount happens because
REFCNT is not _dec when you exit a subroutine.

How long have the bugs you've shown been in Perl, Ilya? Are they
recent introductions or long-standing nastinesses?

This problem is present forever (though I did not check perl4). It is
well-known, easy-to-fix, but a fix will lead to a significant slowdown
of a function call.

Ilya

From @gsar

On Mon, 06 Sep 1999 15​:33​:00 MDT, Nathan Torkington wrote​:

Deniz Yuret writes​:

I had a memory leak in my perl program. I traced it down to the
function split. Specifically I had the expression​:
split(/\W+/);

Implicit split to @​_ leaks, confirmed in _60.

Actually a bare assignment to @​_ is enough to cause a leak.

Try these two patches and tell me if they fix it for you.

Sarathy
gsar@​activestate.com

-----------------------------------8<-----------------------------------
Change 4102 by gsar@auger on 1999/09/08 00:52:50

	fix memory leak in C<sub f { @_ = 1 } f() while 1>

Affected files ...

... //depot/perl/cop.h#28 edit
... //depot/perl/pp_hot.c#127 edit

Differences ...

==== //depot/perl/cop.h#28 (text) ====
Index: perl/cop.h
--- perl/cop.h.~1~	Tue Sep  7 17:53:56 1999
+++ perl/cop.h	Tue Sep  7 17:53:56 1999
@@ -76,6 +76,7 @@
 	    /* destroy arg array */					\
 	    av_clear(cxsub.argarray);					\
 	    AvREAL_off(cxsub.argarray);					\
+	    AvREIFY_on(cxsub.argarray);					\
 	}								\
 	if (cxsub.cv) {							\
 	    if (!(CvDEPTH(cxsub.cv) = cxsub.olddepth))			\

==== //depot/perl/pp_hot.c#127 (text) ====
Index: perl/pp_hot.c
--- perl/pp_hot.c.~1~	Tue Sep  7 17:53:56 1999
+++ perl/pp_hot.c	Tue Sep  7 17:53:56 1999
@@ -2512,6 +2512,7 @@
 	    if (AvREAL(av)) {
 		av_clear(av);
 		AvREAL_off(av);
+		AvREIFY_on(av);
 	    }
 #ifndef USE_THREADS
 	    cx->blk_sub.savearray = GvAV(PL_defgv);
End of Patch.

Change 4103 by gsar@​auger on 1999/09/08 00​:53​:50

  fix memory leak in C<sub f { split ' ', "a b" } f() while 1>

Affected files ...

... //depot/perl/pp.c#141 edit

Differences ...

==== //depot/perl/pp.c#141 (text) ====
Index​: perl/pp.c

--- perl/pp.c.~1~	Tue Sep  7 17:54:00 1999
+++ perl/pp.c	Tue Sep  7 17:54:00 1999
@@ -4870,6 +4870,7 @@
 	else {
 	    if (!AvREAL(ary)) {
 		AvREAL_on(ary);
+		AvREIFY_off(ary);
 		for (i = AvFILLp(ary); i >= 0; i--)
 		    AvARRAY(ary)[i] = &PL_sv_undef;	/* don't free mere refs */
 	    }
End of Patch.