From elmex@ta-sa.org

This is a bug report for perl from elmex@​ta-sa.org.
generated with the help of perlbug 1.36 running under perl 5.10.0.

This perl code leaks memory on all perl 5.10 I had access to​:

  while (1) { map 1, 1 }

Flags​:
  category=core
  severity=high

Site configuration information for perl 5.10.0​:

Configured by root at Wed Dec 19 23​:01​:57 CET 2007.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration​:
  Platform​:
  osname=linux, osvers=2.6.21-2-amd64, archname=x86_64-linux
  uname='linux elmex2 2.6.21-2-amd64 #1 smp tue jul 10 21​:39​:38 utc 2007 x86_64 gnulinux '
  config_args='-des -Dprefix=/opt/perl510/'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2',
  cppflags='-fno-strict-aliasing -pipe -I/usr/local/include'
  ccversion='', gccversion='4.1.3 20070518 (prerelease) (Debian 4.1.2-8)', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -L/usr/local/lib'
  libpth=/usr/local/lib /lib /usr/lib /lib64 /usr/lib64
  libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
  libc=/lib/libc-2.6.1.so, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.6.1'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib'

Locally applied patches​:
 

@​INC for perl 5.10.0​:
  /tmp/cpan/lib/perl/5.8.8/
  /tmp/cpan/share/perl/5.8.8/
  /opt/perl510/lib/5.10.0/x86_64-linux
  /opt/perl510/lib/5.10.0
  /opt/perl510/lib/site_perl/5.10.0/x86_64-linux
  /opt/perl510/lib/site_perl/5.10.0
  .

Environment for perl 5.10.0​:
  HOME=/root
  LANG=en_US.UTF-8
  LANGUAGE=en_DE​:en_US​:en_GB​:en
  LC_CTYPE=en_US.UTF-8
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/opt/jdk1.7.0/bin/​:/opt/rxvt/bin/​:/opt/perl/bin/​:/root/devel/perl​:/bin​:/usr/bin​:/sbin​:/usr/sbin​:/usr/X11R6/bin​:/usr/local/bin​:/usr/local/sbin​:/opt/bin​:/opt/sbin​:/root/bin​:/root/devel/perl/pgrab​:/opt/mplayer/bin/​:/root/devel/cvs/frel/​:/root/devel/perl/snips/​:/opt/cf/bin/​:/opt/parrot/bin/​:/opt/pike/pike/7.6.112/bin
  PERL5LIB=/tmp/cpan/lib/perl/5.8.8/​:/tmp/cpan/share/perl/5.8.8/
  PERL_BADLANG (unset)
  SHELL=/bin/bash

From @shlomif

Hi!

On Friday 18 April 2008, Robin Redeker wrote​:

# New Ticket Created by Robin Redeker
# Please include the string​: [perl #53038]
# in the subject line of all future correspondence about this issue.
# <URL​: http​://rt.perl.org/rt3/Ticket/Display.html?id=53038 >

This is a bug report for perl from elmex@​ta-sa.org.
generated with the help of perlbug 1.36 running under perl 5.10.0.

-----------------------------------------------------------------
This perl code leaks memory on all perl 5.10 I had access to​:

while (1) { map 1, 1 }

I can't see how I can comment on the bug, but I'll note that I can confirm it
with perl-base-5.10.0-13mdv2008.1 on Mandriva Cooker. As it seems, the top
output grow.

Also​:

{{{{{{{{{{{{{{{{{
shlomi​:~$ valgrind perl -e 'for my $a (1 .. 1000) { map 1,1}'
==30038== Memcheck, a memory error detector.
==30038== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
==30038== Using LibVEX rev 1804, a library for dynamic binary translation.
==30038== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
==30038== Using valgrind-3.3.0, a dynamic binary instrumentation framework.
==30038== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
==30038== For more details, rerun with​: -v
==30038==
==30038==
==30038== ERROR SUMMARY​: 0 errors from 0 contexts (suppressed​: 25 from 1)
==30038== malloc/free​: in use at exit​: 142,279 bytes in 775 blocks.
==30038== malloc/free​: 1,270 allocs, 495 frees, 173,417 bytes allocated.
==30038== For counts of detected errors, rerun with​: -v
==30038== searching for pointers to 775 not-freed blocks.
==30038== checked 280,292 bytes.
==30038==
==30038== LEAK SUMMARY​:
==30038== definitely lost​: 142,279 bytes in 775 blocks.
==30038== possibly lost​: 0 bytes in 0 blocks.
==30038== still reachable​: 0 bytes in 0 blocks.
==30038== suppressed​: 0 bytes in 0 blocks.
==30038== Rerun with --leak-check=full to see details of leaked memory.
}}}}}}}}}}}}}}}}}

I should note that I've been on the IRC when Robin reported this bug there.

Regards,

  Shlomi Fish

-----------------------------------------------------------------
---
Flags​:
category=core
severity=high
---
Site configuration information for perl 5.10.0​:

Configured by root at Wed Dec 19 23​:01​:57 CET 2007.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration​:
Platform​:
osname=linux, osvers=2.6.21-2-amd64, archname=x86_64-linux
uname='linux elmex2 2.6.21-2-amd64 #1 smp tue jul 10 21​:39​:38 utc 2007
x86_64 gnulinux ' config_args='-des -Dprefix=/opt/perl510/'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='cc', ccflags ='-fno-strict-aliasing -pipe -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2',
cppflags='-fno-strict-aliasing -pipe -I/usr/local/include'
ccversion='', gccversion='4.1.3 20070518 (prerelease) (Debian
4.1.2-8)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8,
byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define,
longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8,
Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define
Linker and Libraries​:
ld='cc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib /lib64 /usr/lib64
libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
libc=/lib/libc-2.6.1.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.6.1'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib'

Locally applied patches​:

---
@​INC for perl 5.10.0​:
/tmp/cpan/lib/perl/5.8.8/
/tmp/cpan/share/perl/5.8.8/
/opt/perl510/lib/5.10.0/x86_64-linux
/opt/perl510/lib/5.10.0
/opt/perl510/lib/site_perl/5.10.0/x86_64-linux
/opt/perl510/lib/site_perl/5.10.0
.

---
Environment for perl 5.10.0​:
HOME=/root
LANG=en_US.UTF-8
LANGUAGE=en_DE​:en_US​:en_GB​:en
LC_CTYPE=en_US.UTF-8
LD_LIBRARY_PATH (unset)
LOGDIR (unset)

PATH=/opt/jdk1.7.0/bin/​:/opt/rxvt/bin/​:/opt/perl/bin/​:/root/devel/perl​:/bin
​:/usr/bin​:/sbin​:/usr/sbin​:/usr/X11R6/bin​:/usr/local/bin​:/usr/local/sbin​:/opt
/bin​:/opt/sbin​:/root/bin​:/root/devel/perl/pgrab​:/opt/mplayer/bin/​:/root/deve
l/cvs/frel/​:/root/devel/perl/snips/​:/opt/cf/bin/​:/opt/parrot/bin/​:/opt/pike/
pike/7.6.112/bin
PERL5LIB=/tmp/cpan/lib/perl/5.8.8/​:/tmp/cpan/share/perl/5.8.8/ PERL_BADLANG
(unset)
SHELL=/bin/bash

--

Shlomi Fish shlomif@​iglu.org.il
Homepage​: http​://www.shlomifish.org/

I'm not an actor - I just play one on T.V.

The RT System itself - Status changed from 'new' to 'open'

From @nwc10

On Sat, Apr 19, 2008 at 09​:44​:02AM +0300, Shlomi Fish wrote​:

Hi!

On Friday 18 April 2008, Robin Redeker wrote​:

# New Ticket Created by Robin Redeker
# Please include the string​: [perl #53038]
# in the subject line of all future correspondence about this issue.
# <URL​: http​://rt.perl.org/rt3/Ticket/Display.html?id=53038 >

This is a bug report for perl from elmex@​ta-sa.org.
generated with the help of perlbug 1.36 running under perl 5.10.0.

-----------------------------------------------------------------
This perl code leaks memory on all perl 5.10 I had access to​:

while (1) { map 1, 1 }

I can't see how I can comment on the bug, but I'll note that I can confirm it
Replying to the e-mail message works just fine.

==30038== Rerun with --leak-check=full to see details of leaked memory.

Which says​:

==13629== 1,370,724 bytes in 649 blocks are still reachable in loss record 15 of 15
==13629== at 0x4A1B858​: malloc (vg_replace_malloc.c​:149)
==13629== by 0x4880F2​: Perl_safesysmalloc (util.c​:92)
==13629== by 0x4F2C76​: Perl_get_arena (sv.c​:707)
==13629== by 0x4F2DDC​: S_more_bodies (sv.c​:1070)
==13629== by 0x4F3E1F​: Perl_sv_upgrade (sv.c​:1352)
==13629== by 0x51349D​: Perl_sv_setpv (sv.c​:4120)
==13629== by 0x4C23A6​: perl_construct (perl.c​:319)
==13629== by 0x41C2BE​: main (perlmain.c​:107)

which appears to be bogus, because if I stick a breakpoint, here are all
the calls to S_more_bodies​:

(gdb) r
Starting program​: /home/nick/p4perl/perl/perl -e alarm\ \(5\)\;\ while\ \(1\)\ \{\ map\ 1,\ 1\ \}\

Breakpoint 1, S_more_bodies (sv_type=SVt_PV) at sv.c​:1047
1047 void ** const root = &PL_body_roots[sv_type];
(gdb) c
Continuing.

Breakpoint 1, S_more_bodies (sv_type=SVt_PVNV) at sv.c​:1047
1047 void ** const root = &PL_body_roots[sv_type];
(gdb) c
Continuing.

Breakpoint 1, S_more_bodies (sv_type=SVt_PVAV) at sv.c​:1047
1047 void ** const root = &PL_body_roots[sv_type];
(gdb) c
Continuing.

Breakpoint 1, S_more_bodies (sv_type=SVt_PVHV) at sv.c​:1047
1047 void ** const root = &PL_body_roots[sv_type];
(gdb) c
Continuing.

Breakpoint 1, S_more_bodies (sv_type=SVt_PVIV) at sv.c​:1047
1047 void ** const root = &PL_body_roots[sv_type];
(gdb) c
Continuing.

Breakpoint 1, S_more_bodies (sv_type=SVt_PVGV) at sv.c​:1047
1047 void ** const root = &PL_body_roots[sv_type];
(gdb) c
Continuing.

Breakpoint 1, S_more_bodies (sv_type=SVt_PVMG) at sv.c​:1047
1047 void ** const root = &PL_body_roots[sv_type];
(gdb) c
Continuing.

Breakpoint 1, S_more_bodies (sv_type=SVt_PVCV) at sv.c​:1047
1047 void ** const root = &PL_body_roots[sv_type];
(gdb) c
Continuing.

Breakpoint 1, S_more_bodies (sv_type=SVt_PVCV) at sv.c​:1047
1047 void ** const root = &PL_body_roots[sv_type];
(gdb) c
Continuing.

Breakpoint 1, S_more_bodies (sv_type=SVt_PVGV) at sv.c​:1047
1047 void ** const root = &PL_body_roots[sv_type];
(gdb) c
Continuing.

Breakpoint 1, S_more_bodies (sv_type=SVt_PVIO) at sv.c​:1047
1047 void ** const root = &PL_body_roots[sv_type];
(gdb) c

That isn't 649 blocks.
I'm not sure where to go from here.
Does anyone have access to any other sorts of leak-checking tools?

useithreads=undef\, usemultiplicity=undef

  ^^^^^

doesn't leak under ithreads

Nicholas Clark

From @nwc10

On Sat, Apr 19, 2008 at 10​:34​:53AM +0100, Nicholas Clark wrote​:

On Friday 18 April 2008, Robin Redeker wrote​:

This perl code leaks memory on all perl 5.10 I had access to​:

while (1) { map 1, 1 }

That isn't 649 blocks.
I'm not sure where to go from here.
Does anyone have access to any other sorts of leak-checking tools?

sticking a breakpoint on all the malloc functions seems interesting

Breakpoint 2, Perl_safesysmalloc (size=4080) at util.c​:89
89 if ((long)size < 0)
(gdb) where
#0 Perl_safesysmalloc (size=4080) at util.c​:89
#1 0x00000000004f16d5 in S_more_sv () at sv.c​:228
#2 0x0000000000516a72 in Perl_newSV (len=0) at sv.c​:4609
#3 0x0000000000660982 in Perl_gv_SVadd (gv=0x862248) at gv.c​:51
#4 0x0000000000589fd9 in Perl_pp_grepstart () at pp_ctl.c​:964
#5 0x00000000004877fb in Perl_runops_debug () at dump.c​:1984
#6 0x00000000004c86de in S_run_body (oldscope=1) at perl.c​:2392
#7 0x00000000004c7bea in perl_run (my_perl=0x85f010) at perl.c​:2312
#8 0x000000000041c309 in main (argc=3, argv=0x7fffff830298,
  env=0x7fffff8302b8) at perlmain.c​:113
(gdb) c
Continuing.

Breakpoint 2, Perl_safesysmalloc (size=4080) at util.c​:89
89 if ((long)size < 0)
(gdb) where
#0 Perl_safesysmalloc (size=4080) at util.c​:89
#1 0x00000000004f16d5 in S_more_sv () at sv.c​:228
#2 0x0000000000516a72 in Perl_newSV (len=0) at sv.c​:4609
#3 0x0000000000660982 in Perl_gv_SVadd (gv=0x862248) at gv.c​:51
#4 0x0000000000589fd9 in Perl_pp_grepstart () at pp_ctl.c​:964
#5 0x00000000004877fb in Perl_runops_debug () at dump.c​:1984
#6 0x00000000004c86de in S_run_body (oldscope=1) at perl.c​:2392
#7 0x00000000004c7bea in perl_run (my_perl=0x85f010) at perl.c​:2312
#8 0x000000000041c309 in main (argc=3, argv=0x7fffff830298,
  env=0x7fffff8302b8) at perlmain.c​:113
(gdb) c
Continuing.

And, on a macine where hardware watchpoints actually (bloody) work​:

Breakpoint 1, Perl_pp_grepstart () at pp_ctl.c​:951
951 SAVETMPS;
(gdb) call Perl_sv_dump(PL_defgv)
SV = PVGV(0x85b660) at 0x852258
  REFCNT = 2
  FLAGS = ()
  NAME = "_"
  NAMELEN = 1
  GvSTASH = 0x852168 "main"
  GP = 0x85d7f0
  SV = 0x0
  REFCNT = 1
  IO = 0x0
  FORM = 0x0
  AV = 0x852270
  HV = 0x0
  CV = 0x0
  CVGEN = 0x0
  LINE = 0
  FILE = ""
  FLAGS = 0x0
  EGV = 0x852258 "_"
(gdb) p &my_perl->Idefgv.sv_u.svu_gp->gp_sv
There is no member named Idefgv.
(gdb) p &PL_defgv.sv_u.svu_gp->gp_sv
$1 = (SV **) 0x85d7f0
(gdb) p *&PL_defgv.sv_u.svu_gp->gp_sv
$2 = (SV *) 0x0
(gdb) watch *(SV **) 0x85d7f0
Hardware watchpoint 2​: *(struct sv **) 8771568
(gdb) c
Continuing.
Hardware watchpoint 2​: *(struct sv **) 8771568

Old value = (struct sv *) 0x0
New value = (struct sv *) 0x8524e0
Perl_gv_SVadd (gv=0x852258) at gv.c​:52
52 return gv;
(gdb) c
Continuing.
Hardware watchpoint 2​: *(struct sv **) 8771568

Old value = (struct sv *) 0x8524e0
New value = (struct sv *) 0x8655e8
Perl_pp_grepstart () at pp_ctl.c​:966
966 PUTBACK;
(gdb) c
Continuing.
Hardware watchpoint 2​: *(struct sv **) 8771568

Old value = (struct sv *) 0x8655e8
New value = (struct sv *) 0x0
Perl_leave_scope (base=6) at scope.c​:804
804 break;
(gdb) where
#0 Perl_leave_scope (base=6) at scope.c​:804
#1 0x000000000057035b in Perl_pop_scope () at scope.c​:102
#2 0x000000000057fb70 in Perl_pp_mapwhile () at pp_ctl.c​:1043
#3 0x0000000000484cfc in Perl_runops_debug () at dump.c​:1984
#4 0x00000000004c5c06 in S_run_body (oldscope=1) at perl.c​:2392
#5 0x00000000004c5112 in perl_run (my_perl=0x84f010) at perl.c​:2312
#6 0x0000000000419869 in main (argc=3, argv=0x7fffee0f47d8,
  env=0x7fffee0f47f8) at perlmain.c​:113
(gdb) c
Continuing.

Breakpoint 1, Perl_pp_grepstart () at pp_ctl.c​:951
951 SAVETMPS;
(gdb) c
Continuing.
Hardware watchpoint 2​: *(struct sv **) 8771568

Old value = (struct sv *) 0x0
New value = (struct sv *) 0x852150
Perl_gv_SVadd (gv=0x852258) at gv.c​:52
52 return gv;
(gdb) p *(struct sv *) 0x852150
$3 = {sv_any = 0x0, sv_refcnt = 1, sv_flags = 0, sv_u = {svu_iv = 8774800,
  svu_uv = 8774800, svu_rv = 0x85e490, svu_pv = 0x85e490 "",
  svu_array = 0x85e490, svu_hash = 0x85e490, svu_gp = 0x85e490}}
(gdb) p *(struct sv *) 0x852150->sv_refcnt
Attempt to extract a component of a value that is not a structure pointer.
(gdb) p *((struct sv *) 0x852150)->sv_refcnt
Cannot access memory at address 0x1
(gdb) p ((struct sv *) 0x852150)->sv_refcnt
$4 = 1
(gdb) watch ((struct sv *) 0x852150)->sv_refcnt
Hardware watchpoint 3​: ((struct sv *) 8724816)->sv_refcnt
(gdb) c
Continuing.
Hardware watchpoint 2​: *(struct sv **) 8771568

Old value = (struct sv *) 0x852150
New value = (struct sv *) 0x8655e8
Perl_pp_grepstart () at pp_ctl.c​:966
966 PUTBACK;
(gdb) c
Continuing.
Hardware watchpoint 2​: *(struct sv **) 8771568

Old value = (struct sv *) 0x8655e8
New value = (struct sv *) 0x0
Perl_leave_scope (base=6) at scope.c​:804
804 break;
(gdb) call Perl_sv_dump(((struct sv *) 0x852150)
A syntax error in expression, near `'.
(gdb) call Perl_sv_dump((struct sv *) 0x852150)
SV = NULL(0x0) at 0x852150
  REFCNT = 1
  FLAGS = ()

we're leaking the SV that is assigned to DEFSV in this part of pp_grepstart​:

  SAVETMPS;
  if (PL_op->op_private & OPpGREP_LEX)
  SAVESPTR(PAD_SVl(PL_op->op_targ));
  else
  SAVE_DEFSV;
  ENTER; /* enter inner scope */
  SAVEVPTR(PL_curpm);

  src = PL_stack_base[*PL_markstack_ptr];
  SvTEMP_off(src);
  if (PL_op->op_private & OPpGREP_LEX)
  PAD_SVl(PL_op->op_targ) = src;
  else
  DEFSV = src;

I'm not actually sure whether we are leaking it, or merely "leaking" it
(because it and all the other Lintillas) are on the temps stack, and that
part of the temps stack is never cleared.

useithreads=undef\, usemultiplicity=undef

                ^^^^^

doesn't leak under ithreads

Does leak under ithreads, but it seems that it clears up before valgrind gets
a look. I assume that that's the mempool debugging code, but I didn't check.

Nicholas Clark

From p5p@spam.wizbit.be

On Fri, 18 Apr 2008 14​:18​:52 +0200, Robin Redeker
<perlbug-followup@​perl.org> wrote​:

-----------------------------------------------------------------
This perl code leaks memory on all perl 5.10 I had access to​:

while (1) { map 1, 1 }

-----------------------------------------------------------------

Could this be/Is this related to​:

[perl #48004] unacceptable memory consumption
( http​://rt.perl.org/rt3/Ticket/Display.html?id=48004 )(bug report from
december 2007)?

Kind regards,

Bram

From szbalint@inf.elte.hu

I went on a historical field-trip and tried to check what changed
between 5.8.8 and 5.10.0 to cause this memory leak and according to my
tests, r25100 made this leak visible by default.

http​://xrl.us/ooeip

Reverting to the 5.8.8 behaviour by defining PERL_CREATE_GVSV makes this
leak refrain from manifesting in 5.10.0.

This should narrow the scope of tracking down this leak, however I'm not
familiar enough with perl internals to take it further.

From @mhx

Fixed by the following change​:

  http​://public.activestate.com/cgi-bin/perlbrowse/p/34776

  34776 on 2008/11/08 by mhx@​mhx-r2d2

  Assigning to DEFSV leaks if PL_defgv's gp_sv isn't set.
  As Nicholas already noted in a FIXME, assigning to DEFSV should
  use GvSV instead of GvSVn. This change ensures that, at least
  under -DPERL_CORE, DEFSV cannot be assigned to and introduces
  a DEFSV_set macro to allow setting DEFSV.

[perl #48004] is a completely different problem.

@mhx - Status changed from 'open' to 'resolved'