From reza@emarq.com

perl_alloc
perl_construct
perl_destruct
perl_free

is apparently not safe.

From [Unknown Contact. See original ticket]

At 05​:33 PM 12/17/99 -0800, reza@​emarq.com wrote​:

When we embed libperl directly into C there is a huge memory leak. Even
a simple sequence of

perl_alloc
perl_construct
perl_destruct
perl_free

is apparently not safe.

How big is huge? And have you set PERL_DESTRUCT_LEVEL to something other
than zero?

  Dan

----------------------------------------"it's like this"-------------------
Dan Sugalski even samurai
dan@​sidhe.org have teddy bears and even
  teddy bears get drunk

From [Unknown Contact. See original ticket]

Dan Sugalski wrote​:

At 05​:33 PM 12/17/99 -0800, reza@​emarq.com wrote​:

When we embed libperl directly into C there is a huge memory leak. Even
a simple sequence of

perl_alloc
perl_construct
perl_destruct
perl_free

is apparently not safe.

How big is huge? And have you set PERL_DESTRUCT_LEVEL to something other
than zero?

                                    Dan

----------------------------------------"it's like this"-------------------
Dan Sugalski even samurai
dan@​sidhe.org have teddy bears and even
teddy bears get drunk

Dear Dan.
The following code appears to consume memory indefinitely​:

#include <stdlib.h>
#include <malloc.h>
#include <strstream.h>
#include <stdio.h>
#include <EXTERN.h>
#include <perl.h>
#undef TRUE
#undef FALSE
#undef Copy
#undef form

PerlInterpreter *p;

int main(int argc, char** argv, char** env)
{
  while(1)
  {
  PL_perl_destruct_level = 1;
  p=perl_alloc();
  perl_construct(p);
  perl_destruct(p);
  perl_free(p);
  cout << "\n" << mallinfo().uordblks << "\n";
  }
}

This is how I compile it​:
g++ -g perltest.C `perl -MExtUtils​::Embed -e ccopts -e ldopts`
-UHAS_BOOL -Ubool -DCRLDEBUG -DCRLVERBOSE -o PerlLeak

And my g++ and perl versions are​:

g++ -v returns​:
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/egcs-2.91.66/specs
gcc version egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)

perl -v returns​:
... 5.005_03 built for i386-linux ...

From [Unknown Contact. See original ticket]

A long time ago, reza@​emarq.com wrote​:

The following code appears to consume memory indefinitely​:

#include <stdlib.h>
#include <malloc.h>
#include <strstream.h>
#include <stdio.h>
#include <EXTERN.h>
#include <perl.h>
#undef TRUE
#undef FALSE
#undef Copy
#undef form

PerlInterpreter *p;

int main(int argc, char** argv, char** env)
{
while(1)
{
PL_perl_destruct_level = 1;
p=perl_alloc();
perl_construct(p);
perl_destruct(p);
perl_free(p);
cout << "\n" << mallinfo().uordblks << "\n";
}
}

This is how I compile it​:
g++ -g perltest.C `perl -MExtUtils​::Embed -e ccopts -e ldopts`
-UHAS_BOOL -Ubool -DCRLDEBUG -DCRLVERBOSE -o PerlLeak

This appears to still be true under bleadperl.

-spp

From @simoncozens

On Mon, Dec 11, 2000 at 10​:41​:21AM -0500, Stephen P. Potter wrote​:

A long time ago, reza@​emarq.com wrote​:

The following code appears to consume memory indefinitely​:

This appears to still be true under bleadperl.

This is because init_interp (called from perl_construct) resets the value of
Perl_perl_destruct_level to zero, which really, really defeats the whole point.

(in intrpvar.h)
  PERLVARI(Iperl_destruct_level, int, 0)

However, changing this to
  PERLVAR(Iperl_destruct_level, int)
causes a different bug, related to the destruction of PL_strtab. Oh, what fun!

Here's a fix, anyway.

--- perl.c~	Mon Dec 11 16:41:39 2000
+++ perl.c	Mon Dec 11 16:55:39 2000
@@ -648,15 +648,12 @@
     /* Now absolutely destruct everything, somehow or other, loops or no. */
     last_sv_count = 0;
     SvFLAGS(PL_fdpid) |= SVTYPEMASK;		/* don't clean out pid table now */
-    SvFLAGS(PL_strtab) |= SVTYPEMASK;		/* don't clean out strtab now */
     while (PL_sv_count != 0 && PL_sv_count != last_sv_count) {
 	last_sv_count = PL_sv_count;
 	sv_clean_all();
     }
     SvFLAGS(PL_fdpid) &= ~SVTYPEMASK;
     SvFLAGS(PL_fdpid) |= SVt_PVAV;
-    SvFLAGS(PL_strtab) &= ~SVTYPEMASK;
-    SvFLAGS(PL_strtab) |= SVt_PVHV;
 
     AvREAL_off(PL_fdpid);		/* no surviving entries */
     SvREFCNT_dec(PL_fdpid);		/* needed in io_close() */
@@ -667,7 +664,7 @@
 #endif
 
     /* Destruct the global string table. */
-    {
+    if (PL_strtab) {
 	/* Yell and reset the HeVAL() slots that are still holding refcounts,
 	 * so that sv_free() won't fail on them.
 	 */
--- ../perlio/intrpvar.h	Sun Dec 10 15:08:33 2000
+++ intrpvar.h	Mon Dec 11 16:45:06 2000
@@ -52,7 +52,7 @@
 
 /* This value may be set when embedding for full cleanup  */
 /* 0=none, 1=full, 2=full with checks */
-PERLVARI(Iperl_destruct_level,	int,	0)
+PERLVAR(Iperl_destruct_level,	int)
 
 /* magical thingies */
 PERLVAR(Ibasetime,	Time_t)		/* $^T */

From @gsar

On Mon, 11 Dec 2000 16​:57​:07 GMT, Simon Cozens wrote​:

On Mon, Dec 11, 2000 at 10​:41​:21AM -0500, Stephen P. Potter wrote​:

A long time ago, reza@​emarq.com wrote​:

The following code appears to consume memory indefinitely​:

This appears to still be true under bleadperl.

This is because init_interp (called from perl_construct) resets the value of
Perl_perl_destruct_level to zero, which really, really defeats the whole point
.

Shouldn't matter, since it then immediately sets it to 1.

(in intrpvar.h)
PERLVARI(Iperl_destruct_level, int, 0)

However, changing this to
PERLVAR(Iperl_destruct_level, int)
causes a different bug, related to the destruction of PL_strtab. Oh, what fun!

Here's a fix, anyway.

--- perl.c~ Mon Dec 11 16​:41​:39 2000
+++ perl.c Mon Dec 11 16​:55​:39 2000
@​@​ -648,15 +648,12 @​@​
/* Now absolutely destruct everything, somehow or other, loops or no. */
last_sv_count = 0;
SvFLAGS(PL_fdpid) |= SVTYPEMASK; /* don't clean out pid table no
w */
- SvFLAGS(PL_strtab) |= SVTYPEMASK; /* don't clean out strtab now *
/
while (PL_sv_count != 0 && PL_sv_count != last_sv_count) {
last_sv_count = PL_sv_count;
sv_clean_all();
}
SvFLAGS(PL_fdpid) &= ~SVTYPEMASK;
SvFLAGS(PL_fdpid) |= SVt_PVAV;
- SvFLAGS(PL_strtab) &= ~SVTYPEMASK;
- SvFLAGS(PL_strtab) |= SVt_PVHV;

AvREAL\_off\(PL\_fdpid\);        /\* no surviving entries \*/
SvREFCNT\_dec\(PL\_fdpid\);        /\* needed in io\_close\(\) \*/

@​@​ -667,7 +664,7 @​@​
#endif

/\* Destruct the global string table\. \*/

- {
+ if (PL_strtab) {

Can you explain the rationale behind those changes? They doen't look
correct to me.

(PL_strtab needs to hang around until after everything that uses
shared strings goes away. That's what the code that sets its
type to SVTYPEMASK is meant to achieve.)

Sarathy
gsar@​ActiveState.com

From @simoncozens

On Mon, Dec 11, 2000 at 12​:47​:44PM -0800, Gurusamy Sarathy wrote​:

This is because init_interp (called from perl_construct) resets the value of
Perl_perl_destruct_level to zero, which really, really defeats the whole point
.

Shouldn't matter, since it then immediately sets it to 1.

Not all the world is using MULTIPLICITY​:

#ifdef MULTIPLICITY
  init_interp();
  PL_perl_destruct_level = 1;
#else
  if (PL_perl_destruct_level > 0)
  init_interp();
#endif

Can you explain the rationale behind those changes? They doen't look
correct to me.

(PL_strtab needs to hang around until after everything that uses
shared strings goes away. That's what the code that sets its
type to SVTYPEMASK is meant to achieve.)

Yes, but PL_strtab had already gone by the time we got there.

From @gsar

On Mon, 11 Dec 2000 10​:41​:21 EST, "Stephen P. Potter" wrote​:

A long time ago, reza@​emarq.com wrote​:

The following code appears to consume memory indefinitely​:
[...]
int main(int argc, char** argv, char** env)
{
while(1)
{
PL_perl_destruct_level = 1;
p=perl_alloc();

Setting PL_perl_destruct_level before the perl_alloc() makes no
sense. Note that it expands to my_perl->Iperl_destruct_level and
similar under certain configurations.

That also proves that you're not building it with -DMULTIPLICITY,
which I'd suggest building it with.

  perl\_construct\(p\);
  perl\_destruct\(p\);
  perl\_free\(p\);
  cout \<\< "\\n" \<\< mallinfo\(\)\.uordblks \<\< "\\n"; 
\}

}

This is how I compile it​:
g++ -g perltest.C `perl -MExtUtils​::Embed -e ccopts -e ldopts`
-UHAS_BOOL -Ubool -DCRLDEBUG -DCRLVERBOSE -o PerlLeak

This appears to still be true under bleadperl.

Sarathy
gsar@​ActiveState.com

From @gsar

On Mon, 11 Dec 2000 21​:04​:17 GMT, Simon Cozens wrote​:

On Mon, Dec 11, 2000 at 12​:47​:44PM -0800, Gurusamy Sarathy wrote​:

This is because init_interp (called from perl_construct) resets the value o
f
Perl_perl_destruct_level to zero, which really, really defeats the whole po
int
.

Shouldn't matter, since it then immediately sets it to 1.

Not all the world is using MULTIPLICITY​:

If they create more than one interpreter, they should be. (This
ain't the only thing that will bite them.)

#ifdef MULTIPLICITY
init_interp();
PL_perl_destruct_level = 1;
#else
if (PL_perl_destruct_level > 0)
init_interp();
#endif

Can you explain the rationale behind those changes? They doen't look
correct to me.

(PL_strtab needs to hang around until after everything that uses
shared strings goes away. That's what the code that sets its
type to SVTYPEMASK is meant to achieve.)

Yes, but PL_strtab had already gone by the time we got there.

That looks suspicious to me (perhaps something else is broken).

Sarathy
gsar@​ActiveState.com

From [Unknown Contact. See original ticket]

Since there's a flurry of memory leak plugging, here's another one from
awhile back. I've also attached Simon's fix to the bottom, maybe someone
can look into whether it needs updated and add it.

-spp

------- Forwarded Message

A long time ago, reza@​emarq.com wrote​:

The following code appears to consume memory indefinitely​:

#include <stdlib.h>
#include <malloc.h>
#include <strstream.h>
#include <stdio.h>
#include <EXTERN.h>
#include <perl.h>
#undef TRUE
#undef FALSE
#undef Copy
#undef form

PerlInterpreter *p;

int main(int argc, char** argv, char** env)
{
while(1)
{
PL_perl_destruct_level = 1;
p=perl_alloc();
perl_construct(p);
perl_destruct(p);
perl_free(p);
cout << "\n" << mallinfo().uordblks << "\n";
}
}

This is how I compile it​:
g++ -g perltest.C `perl -MExtUtils​::Embed -e ccopts -e ldopts`
-UHAS_BOOL -Ubool -DCRLDEBUG -DCRLVERBOSE -o PerlLeak

This appears to still be true under bleadperl.

-spp

This is because init_interp (called from perl_construct) resets the value of
Perl_perl_destruct_level to zero, which really, really defeats the whole point.

(in intrpvar.h)
  PERLVARI(Iperl_destruct_level, int, 0)

However, changing this to
  PERLVAR(Iperl_destruct_level, int)
causes a different bug, related to the destruction of PL_strtab. Oh, what fun!

Here's a fix, anyway.

--- perl.c~	Mon Dec 11 16:41:39 2000
+++ perl.c	Mon Dec 11 16:55:39 2000
@@ -648,15 +648,12 @@
     /* Now absolutely destruct everything, somehow or other, loops or no. */
     last_sv_count = 0;
     SvFLAGS(PL_fdpid) |= SVTYPEMASK;		/* don't clean out pid table now */
-    SvFLAGS(PL_strtab) |= SVTYPEMASK;		/* don't clean out strtab now */
     while (PL_sv_count != 0 && PL_sv_count != last_sv_count) {
 	last_sv_count = PL_sv_count;
 	sv_clean_all();
     }
     SvFLAGS(PL_fdpid) &= ~SVTYPEMASK;
     SvFLAGS(PL_fdpid) |= SVt_PVAV;
-    SvFLAGS(PL_strtab) &= ~SVTYPEMASK;
-    SvFLAGS(PL_strtab) |= SVt_PVHV;
 
     AvREAL_off(PL_fdpid);		/* no surviving entries */
     SvREFCNT_dec(PL_fdpid);		/* needed in io_close() */
@@ -667,7 +664,7 @@
 #endif
 
     /* Destruct the global string table. */
-    {
+    if (PL_strtab) {
 	/* Yell and reset the HeVAL() slots that are still holding refcounts,
 	 * so that sv_free() won't fail on them.
 	 */
--- ../perlio/intrpvar.h	Sun Dec 10 15:08:33 2000
+++ intrpvar.h	Mon Dec 11 16:45:06 2000
@@ -52,7 +52,7 @@
 
 /* This value may be set when embedding for full cleanup  */
 /* 0=none, 1=full, 2=full with checks */
-PERLVARI(Iperl_destruct_level,	int,	0)
+PERLVAR(Iperl_destruct_level,	int)
 
 /* magical thingies */
 PERLVAR(Ibasetime,	Time_t)		/* $^T */

------- End of Forwarded Message

From @gsar

On Fri, 09 Feb 2001 11​:15​:47 EST, "Stephen P. Potter" wrote​:

Since there's a flurry of memory leak plugging, here's another one from
awhile back. I've also attached Simon's fix to the bottom, maybe someone
can look into whether it needs updated and add it.
[...]

This is how I compile it​:
g++ -g perltest.C `perl -MExtUtils​::Embed -e ccopts -e ldopts`
-UHAS_BOOL -Ubool -DCRLDEBUG -DCRLVERBOSE -o PerlLeak

This appears to still be true under bleadperl.

IIRC, this person wasn't building it with MULTIPLICITY/USE_ITHREADS.
Did you try it after enabling one of those?

This is because init_interp (called from perl_construct) resets the value of
Perl_perl_destruct_level to zero, which really, really defeats the whole point
.

(in intrpvar.h)
PERLVARI(Iperl_destruct_level, int, 0)

This is an important optimization in the non-MULTIPLICITY case, because
you don't want to blow a bunch of time freeing memory when you're going
to exit real soon anyway.

In the MULTIPLICITY/USE_ITHREADS case, perl_construct() will set
it to default to 1, so it doesn't need changing.

Sarathy
gsar@​ActiveState.com

From @AlanBurlison

"Stephen P. Potter" wrote​:

Since there's a flurry of memory leak plugging, here's another one from
awhile back. I've also attached Simon's fix to the bottom, maybe someone
can look into whether it needs updated and add it.

Patch causes SEGV​:

core 'core' of 249773​: ./miniperl configpm configpm.tmp
ff261ca0 Perl_hv_free_ent (e3158, e3bb8, ea868, ffbeece4, 0, 0) + 68
ff262190 S_hfreeentries (e3158, e3bb8, 0, 0, 0, ffbee8fd) + d0
ff26227c Perl_hv_undef (e3158, e3bb8, fffffffc, 0, 0, 6c6c0050) + 5c
ff28cca8 Perl_sv_clear (e3158, e3bb8, 0, 0, 29434, ff3bdc7c) + a68
ff28d514 Perl_sv_free (e3158, e3bb8, 3, ff3e2698, 0, 0) + 344
ff2998bc do_clean_all (e3158, e3bb8, ff3a052c, 0, ff3e2698, ff1aa235) +
a4
ff27c398 S_visit (e3158, ff299818, ff3a0018, 0, ff3e2698, 246ca) + b0
ff27c540 Perl_sv_clean_all (e3158, e3ba0, ffbeeb58, ff1c5a78, ff3e2698,
ff1989bd) + 48
ff1c354c perl_destruct (e3158, 260e8, 3, ffbeece4, 0, 0) + bbc
00026084 main (3, ffbeece4, ffbeecf4, df800, 0, 0) + 134
00025f28 _start (0, 0, 0, 0, 0, 0) + 108

Alan Burlison

From [Unknown Contact. See original ticket]

Alan Burlison <Alan.Burlison@​uk.sun.com> writes​:

"Stephen P. Potter" wrote​:

Since there's a flurry of memory leak plugging, here's another one from
awhile back. I've also attached Simon's fix to the bottom, maybe someone
can look into whether it needs updated and add it.

Patch causes SEGV​:

I meant to say I thought we fixed that one another way.

core 'core' of 249773​: ./miniperl configpm configpm.tmp
ff261ca0 Perl_hv_free_ent (e3158, e3bb8, ea868, ffbeece4, 0, 0) + 68
ff262190 S_hfreeentries (e3158, e3bb8, 0, 0, 0, ffbee8fd) + d0
ff26227c Perl_hv_undef (e3158, e3bb8, fffffffc, 0, 0, 6c6c0050) + 5c
ff28cca8 Perl_sv_clear (e3158, e3bb8, 0, 0, 29434, ff3bdc7c) + a68
ff28d514 Perl_sv_free (e3158, e3bb8, 3, ff3e2698, 0, 0) + 344
ff2998bc do_clean_all (e3158, e3bb8, ff3a052c, 0, ff3e2698, ff1aa235) +
a4
ff27c398 S_visit (e3158, ff299818, ff3a0018, 0, ff3e2698, 246ca) + b0
ff27c540 Perl_sv_clean_all (e3158, e3ba0, ffbeeb58, ff1c5a78, ff3e2698,
ff1989bd) + 48
ff1c354c perl_destruct (e3158, 260e8, 3, ffbeece4, 0, 0) + bbc
00026084 main (3, ffbeece4, ffbeecf4, df800, 0, 0) + 134
00025f28 _start (0, 0, 0, 0, 0, 0) + 108

Alan Burlison

From @smpeters

It looks like this old memory leak still exists. Included is my
valgrind output.

==18586==
==18586== ERROR SUMMARY​: 0 errors from 0 contexts (suppressed​: 0 from 0)
==18586== malloc/free​: in use at exit​: 14426410 bytes in 12403 blocks.
==18586== malloc/free​: 16404 allocs, 4001 frees, 14433412 bytes allocated.
==18586==
==18586== searching for pointers to 12403 not-freed blocks.
==18586== checked 18571140 bytes.
==18586==
==18586== 19232 bytes in 47 blocks are possibly lost in loss record 1 of 3
==18586== at 0x40029A4D​: malloc (in /usr/lib/valgrind/vgskin_memcheck.so)
==18586== by 0x402BFB8B​: Perl_safesysmalloc (in
/usr/lib/libperl.so.1.5.8)
==18586== by 0x404CE8A7​: __libc_start_main (in /lib/libc-2.3.3.so)
==18586== by 0x8048A70​: (within /home/steve/sandbox/PerlLeak)
==18586==
==18586==
==18586== 1355392 bytes in 5947 blocks are definitely lost in loss
record 2 of 3
==18586== at 0x40029A4D​: malloc (in /usr/lib/valgrind/vgskin_memcheck.so)
==18586== by 0x402BFB8B​: Perl_safesysmalloc (in
/usr/lib/libperl.so.1.5.8)
==18586==
==18586==
==18586== 13051786 bytes in 6409 blocks are still reachable in loss
record 3 of 3
==18586== at 0x40029A4D​: malloc (in /usr/lib/valgrind/vgskin_memcheck.so)
==18586== by 0x402BFB8B​: Perl_safesysmalloc (in
/usr/lib/libperl.so.1.5.8)
==18586== by 0x404CE8A7​: __libc_start_main (in /lib/libc-2.3.3.so)
==18586== by 0x8048A70​: (within /home/steve/sandbox/PerlLeak)
==18586==
==18586== LEAK SUMMARY​:
==18586== definitely lost​: 1355392 bytes in 5947 blocks.
==18586== possibly lost​: 19232 bytes in 47 blocks.
==18586== still reachable​: 13051786 bytes in 6409 blocks.
==18586== suppressed​: 0 bytes in 0 blocks.
==18586==
--18586-- TT/TC​: 0 tc sectors discarded.
--18586-- 2109 chainings, 0 unchainings.
--18586-- translate​: new 3827 (59985 -> 756025; ratio 126​:10)
--18586-- discard 0 (0 -> 0; ratio 0​:10).
--18586-- dispatch​: 14550000 jumps (bb entries), of which 1561271 (10%)
were unchained.
--18586-- 293/68713 major/minor sched events. 17935 tt_fast
misses.
--18586-- reg-alloc​: 519 t-req-spill, 139269+3490 orig+spill uis, 18131
total-reg-r.
--18586-- sanity​: 294 cheap, 12 expensive checks.
--18586-- ccalls​: 16660 C calls, 58% saves+restores avoided (57558 bytes)
--18586-- 22415 args, avg 0.88 setup instrs each (5006 bytes)
--18586-- 0% clear the stack (49980 bytes)
--18586-- 6061 retvals, 35% of reg-reg movs avoided (4158 bytes)

From @gannett-ggreer

After tweaking the C++ program source somewhat to compile under Ubuntu
10.04, I can't reproduce the memory leak. The mallinfo() call always
returns 20256 and never changes.

Source attached. I compiled with​:

g++ -g `perl -MExtUtils​::Embed -e ccopts -e ldopts` -DCRLDEBUG
-DCRLVERBOSE leak.cpp -o leak

Ubuntu's Perl is​:

Summary of my perl5 (revision 5 version 10 subversion 1) configuration​:
[...]
Characteristics of this binary (from libperl)​:
  Compile-time options​: MULTIPLICITY PERL_DONT_CREATE_GVSV
  PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
USE_64_BIT_ALL
  USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES
  USE_PERLIO USE_REENTRANT_API

--
George Greer

From @gannett-ggreer

leak.cpp

From @iabyn

The OP's code is creating multiple interpreters, and thus needs to
set PL_perl_destruct_level to 1 to ensure proper clean up. The docs were
unclear back when this ticket was created, but now it's clear in
perlembed.pod that you need to set it, and where. The example code
below demonstrates test code as a standalone C program.
On blead, remove 'PL_perl_destruct_level=1' on a non-MULTIPLICITY build,
and it will leak.

@iabyn - Status changed from 'open' to 'resolved'

From @iabyn

forgot the code​:

/* compile with
cc -o leak leak.c -I. `./perl -Ilib -MExtUtils​::Embed -e ccopts -e ldopts`
*/
#include <EXTERN.h>
#include <perl.h>

PerlInterpreter *my_perl;

int main(int argc, char** argv, char** env)
{
  for (;;) {
  my_perl = perl_alloc();
  perl_construct(my_perl);
  PL_perl_destruct_level = 1;
  perl_destruct(my_perl);
  perl_free(my_perl);
  }
}