Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak with named regexp captures #10702

Closed
p5pRT opened this issue Oct 7, 2010 · 24 comments
Closed

Memory leak with named regexp captures #10702

p5pRT opened this issue Oct 7, 2010 · 24 comments

Comments

@p5pRT
Copy link

p5pRT commented Oct 7, 2010

Migrated from rt.perl.org#78266 (status was 'resolved')

Searchable as RT78266$

@p5pRT
Copy link
Author

p5pRT commented Aug 18, 2010

From r.khamidullin@transcredit.ru

Please,
create the bug ticket.

I have a memory leak.

Sample with memory leak
#if ( $_ =~ /ip\=(?<ip2>\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})\,
orderInfo\=<Order><row><id>(?<orderid2>\d+)<\/id>.+?<Description>(?<rzdid2>$rzdid)<\/Description>/

Sample without memory leak
#if ( $_ =~ /ip\=(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})\,
orderInfo\=<Order><row><id>(\d+)<\/id>.+?<Description>($rzdid)<\/Description>/

--
perl -V
Summary of my perl5 (revision 5 version 10 subversion 0) configuration​:
  Platform​:
  osname=linux, osvers=2.6.27, archname=x86_64-linux-thread-multi
  uname='linux haley 2.6.27 #1 smp 2009-02-09 15​:38​:31 +0100 x86_64
x86_64 x86_64 gnulinux '
  config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr
-Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm
-Duseshrplib=true -Doptimize=-fmessage-length=0 -O2 -Wall
-D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables
-fasynchronous-unwind-tables -g -Wall -pipe
-Accflags=-DPERL_USE_SAFE_PUTENV'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV
-DDEBUGGING -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
  optimize='-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2
-fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall
-pipe',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV
-DDEBUGGING -fno-strict-aliasing -pipe'
  ccversion='', gccversion='4.3.2 [gcc-4_3-branch revision 141291]',
gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -L/usr/local/lib64'
  libpth=/lib64 /usr/lib64 /usr/local/lib64
  libs=-lm -ldl -lcrypt -lpthread
  perllibs=-lm -ldl -lcrypt -lpthread
  libc=/lib64/libc-2.9.so, so=so, useshrplib=true, libperl=libperl.so
  gnulibc_version='2.9'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E
-Wl,-rpath,/usr/lib/perl5/5.10.0/x86_64-linux-thread-multi/CORE'
  cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64'

Characteristics of this binary (from libperl)​:
  Compile-time options​: DEBUGGING MULTIPLICITY PERL_DONT_CREATE_GVSV
  PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
  PERL_TRACK_MEMPOOL PERL_USE_SAFE_PUTENV
  USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS
  USE_LARGE_FILES USE_PERLIO USE_REENTRANT_API
  Built under linux
  Compiled at Jun 10 2009 16​:23​:14
  @​INC​:
  /usr/lib/perl5/5.10.0/x86_64-linux-thread-multi
  /usr/lib/perl5/5.10.0
  /usr/lib/perl5/site_perl/5.10.0/x86_64-linux-thread-multi
  /usr/lib/perl5/site_perl/5.10.0
  /usr/lib/perl5/vendor_perl/5.10.0/x86_64-linux-thread-multi
  /usr/lib/perl5/vendor_perl/5.10.0
  /usr/lib/perl5/vendor_perl

--
С �важением,

Р���ам Хамид�ллин
�л. �пе�иали�� о�дела ин�е�не�-п�оек�ов
��� "Т�ан���еди��а��"
�ел. (495)788-0880 доб. 2701

@p5pRT
Copy link
Author

p5pRT commented Aug 18, 2010

From @iabyn

On Wed, Aug 18, 2010 at 04​:36​:10AM -0700, r.khamidullin wrote​:

I have a memory leak.

Sample with memory leak
#if ( $_ =~ /ip\=(?<ip2>\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})\,
orderInfo\=<Order><row><id>(?<orderid2>\d+)<\/id>.+?<Description>(?<rzdid2>$rzdid)<\/Description>/

Can you please supply a short, complete program that demonstrates the
leak.

--
Fire extinguisher (n) a device for holding open fire doors.

@p5pRT
Copy link
Author

p5pRT commented Aug 18, 2010

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Oct 7, 2010

From @SBECK-github

Created by @SBECK-github

There is a memory leak in both perl 5.10 and 5.12 that occurs when named
regexp captures are used.

The following script illustrates the bug​:

=============================
#!/usr/bin/perl

$a1 = '(?<a>\d)';

$a2 = $a1;
#$a2 = '(?<b>\d)';

$b = '\d';

$rx = qr/(?​:${a}${b}|${a2}​:${b})/;

#$string = "12";
$string = "1​:2";

while (1) {
  $string =~ $rx;
  $tmp = $+{a};
}

When a regexp is of the form (MATCH1|MATCH2|...) and there matches have
named captures, this leak will show up.

Every time a match occurs other than the first one, the leak can be
triggered. In other words, in the test script, if $string is "12" (which
always matches the first match), there is no leak, but when the string
"1​:2" is used, it leaks.

Also, it doesn't matter if the named captures are the same or different
in the matches. In the test script, I had the named captures in MATCH1
and MATCH2 be the same (with the first $a2 line) or different (with the
second commented out $a2 line), and both leaked.

Finally, the leak is only triggered if you actually access the %+ hash.
If you comment out the final line ($tmp = $+{a}), the script doesn't
leak, even though you are doing the match, and (I assume) populating %+.

Anyway, hope this helps.

Thanks

BTW, many thanks to BrowserUK on perlmonks for helping me track this down.

Perl Info

Flags:
     category=core
     severity=high

This perlbug was built using Perl 5.12.1 - Fri Jul 30 00:13:43 UTC 2010
It is being executed now by  Perl 5.12.1 - Fri Jul 30 00:09:43 UTC 2010.

Site configuration information for perl 5.12.1:

Configured by abuild at Fri Jul 30 00:09:43 UTC 2010.

Summary of my perl5 (revision 5 version 12 subversion 1) configuration:

   Platform:
     osname=linux, osvers=2.6.32, archname=x86_64-linux-thread-multi
     uname='linux build30 2.6.32 #1 smp 2010-01-06 16:07:25 +0100 x86_64 
x86_64 x86_64 gnulinux '
     config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr 
-Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm 
-Duseshrplib=true -Doptimize=-fmessage-length=0 -O2 -Wall 
-D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables 
-fasynchronous-unwind-tables -g -Wall -pipe 
-Accflags=-DPERL_USE_SAFE_PUTENV'
     hint=recommended, useposix=true, d_sigaction=define
     useithreads=define, usemultiplicity=define
     useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
     use64bitint=define, use64bitall=define, uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE 
-DPERL_USE_SAFE_PUTENV -DDEBUGGING -fno-strict-aliasing -pipe 
-fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
     optimize='-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 
-fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall 
-pipe',
     cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV 
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector'
     ccversion='', gccversion='4.5.0 20100604 [gcc-4_5-branch revision 
160292]', gccosandvers=''
     intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
     ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
     alignbytes=8, prototype=define
   Linker and Libraries:
     ld='cc', ldflags =' -L/usr/local/lib64 -fstack-protector'
     libpth=/lib64 /usr/lib64 /usr/local/lib64
     libs=-lm -ldl -lcrypt -lpthread
     perllibs=-lm -ldl -lcrypt -lpthread
     libc=/lib64/libc-2.11.2.so, so=so, useshrplib=true, libperl=libperl.so
     gnulibc_version='2.11.2'
   Dynamic Linking:
     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E 
-Wl,-rpath,/usr/lib/perl5/5.12.1/x86_64-linux-thread-multi/CORE'
     cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64 
-fstack-protector'

Locally applied patches:



@INC for perl 5.12.1:
     .
     
/home/sulbeck/local/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-multi
     
/home/sulbeck/local/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-multi
     /home/sulbeck/local/lib/perl5/site_perl/5.12.1
     
/home/sulbeck/local/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-multi
     /home/sulbeck/local/lib/perl5/site_perl/5.12.1
     /home/sulbeck/local/lib/perl5/site_perl
     /home/sulbeck/local/lib/perl5/5.12.1/x86_64-linux-thread-multi
     /home/sulbeck/local/lib/perl5/5.12.1/x86_64-linux-thread-multi
     /home/sulbeck/local/lib/perl5/5.12.1
     /usr/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-multi
     /usr/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-multi
     /usr/lib/perl5/site_perl/5.12.1
     /usr/lib/perl5/vendor_perl/5.12.1/x86_64-linux-thread-multi
     /usr/lib/perl5/vendor_perl/5.12.1/x86_64-linux-thread-multi
     /usr/lib/perl5/vendor_perl/5.12.1
     /usr/lib/perl5/5.12.1/x86_64-linux-thread-multi
     /usr/lib/perl5/5.12.1/x86_64-linux-thread-multi
     /usr/lib/perl5/5.12.1
     /usr/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-multi
     /usr/lib/perl5/site_perl/5.12.1
     /usr/lib/perl5/vendor_perl/5.12.1/x86_64-linux-thread-multi
     /usr/lib/perl5/vendor_perl/5.12.1
     /usr/lib/perl5/5.12.1/x86_64-linux-thread-multi
     /usr/lib/perl5/5.12.1
     .


Environment for perl 5.12.1:
     HOME=/home/sulbeck
     LANG=en_US.UTF-8
     LANGUAGE (unset)
     LC_ALL=C
     LD_LIBRARY_PATH (unset)
     LOGDIR (unset)
     
PATH=/home/sulbeck/local/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/bin:/sbin:/usr/games:/usr/lib/mit/bin:/usr/lib/mit/sbin:/usr/lib/qt3/bin:/etc:.
     PERL=/usr/bin/perl
     
PERL5LIB=.:/home/sulbeck/local/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-multi:/home/sulbeck/local/lib/perl5/site_perl/5.12.1:/home/sulbeck/local/lib/perl5/site_perl:/home/sulbeck/local/lib/perl5/5.12.1/x86_64-linux-thread-multi:/home/sulbeck/local/lib/perl5/5.12.1:/usr/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-multi:/usr/lib/perl5/site_perl/5.12.1:/usr/lib/perl5/vendor_perl/5.12.1/x86_64-linux-thread-multi:/usr/lib/perl5/vendor_perl/5.12.1:/usr/lib/perl5/5.12.1/x86_64-linux-thread-multi:/usr/lib/perl5/5.12.1
     PERL_BADLANG (unset)
     SHELL=/usr/bin/zsh


@p5pRT
Copy link
Author

p5pRT commented Feb 7, 2011

@iabyn - Status changed from 'open' to 'stalled'

@p5pRT
Copy link
Author

p5pRT commented Jul 12, 2011

From @SBECK-github

Just FYI, I just checked this against perl 5.14.1 and 5.15.0 and the
leak still exists.

@p5pRT
Copy link
Author

p5pRT commented Jul 12, 2011

@SBECK-github - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 12, 2011

From @plu

I think I'm experiencing the same bug. I've narrowed it down to a simple
script, which you can find attached here. If I can provide more help by
giving more details or anything else, please let me know.

@p5pRT
Copy link
Author

p5pRT commented Dec 12, 2011

From @plu

leak.pl

@p5pRT
Copy link
Author

p5pRT commented Dec 12, 2011

From [Unknown Contact. See original ticket]

I think I'm experiencing the same bug. I've narrowed it down to a simple
script, which you can find attached here. If I can provide more help by
giving more details or anything else, please let me know.

@p5pRT
Copy link
Author

p5pRT commented Dec 12, 2011

From @plu

Changing status from stalled to open, hope that's ok. If not, just
change it back.

@p5pRT
Copy link
Author

p5pRT commented Dec 12, 2011

@cpansprout - Status changed from 'stalled' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 12, 2011

From data@cpan.org

You may want to merge #77304 into this one, since they are the same. Or
close one of them.

On Thu Oct 07 04​:40​:37 2010, sbeck wrote​:

This is a bug report for perl from sbeck@​cpan.org,
generated with the help of perlbug 1.39 running under perl 5.12.1.

-----------------------------------------------------------------
[Please describe your issue here]

There is a memory leak in both perl 5.10 and 5.12 that occurs when
named
regexp captures are used.

The following script illustrates the bug​:

=============================
#!/usr/bin/perl

$a1 = '(?<a>\d)';

$a2 = $a1;
#$a2 = '(?<b>\d)';

$b = '\d';

$rx = qr/(?​:${a}${b}|${a2}​:${b})/;

#$string = "12";
$string = "1​:2";

while (1) {
$string =~ $rx;
$tmp = $+{a};
}

When a regexp is of the form (MATCH1|MATCH2|...) and there matches
have
named captures, this leak will show up.

Every time a match occurs other than the first one, the leak can be
triggered. In other words, in the test script, if $string is "12"
(which
always matches the first match), there is no leak, but when the string
"1​:2" is used, it leaks.

Also, it doesn't matter if the named captures are the same or
different
in the matches. In the test script, I had the named captures in MATCH1
and MATCH2 be the same (with the first $a2 line) or different (with
the
second commented out $a2 line), and both leaked.

Finally, the leak is only triggered if you actually access the %+
hash.
If you comment out the final line ($tmp = $+{a}), the script doesn't
leak, even though you are doing the match, and (I assume) populating
%+.

Anyway, hope this helps.

Thanks

BTW, many thanks to BrowserUK on perlmonks for helping me track this
down.

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=high
---
This perlbug was built using Perl 5.12.1 - Fri Jul 30 00​:13​:43 UTC
2010
It is being executed now by Perl 5.12.1 - Fri Jul 30 00​:09​:43 UTC
2010.

Site configuration information for perl 5.12.1​:

Configured by abuild at Fri Jul 30 00​:09​:43 UTC 2010.

Summary of my perl5 (revision 5 version 12 subversion 1)
configuration​:

Platform​:
osname=linux, osvers=2.6.32, archname=x86_64-linux-thread-multi
uname='linux build30 2.6.32 #1 smp 2010-01-06 16​:07​:25 +0100
x86_64
x86_64 x86_64 gnulinux '
config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr
-Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm
-Duseshrplib=true -Doptimize=-fmessage-length=0 -O2 -Wall
-D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables
-fasynchronous-unwind-tables -g -Wall -pipe
-Accflags=-DPERL_USE_SAFE_PUTENV'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define,
usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE
-DPERL_USE_SAFE_PUTENV -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2
-fstack-protector -funwind-tables -fasynchronous-unwind-tables -g
-Wall
-pipe',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector'
ccversion='', gccversion='4.5.0 20100604 [gcc-4_5-branch revision
160292]', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8,
byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define,
longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8,
Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='cc', ldflags =' -L/usr/local/lib64 -fstack-protector'
libpth=/lib64 /usr/lib64 /usr/local/lib64
libs=-lm -ldl -lcrypt -lpthread
perllibs=-lm -ldl -lcrypt -lpthread
libc=/lib64/libc-2.11.2.so, so=so, useshrplib=true,
libperl=libperl.so
gnulibc_version='2.11.2'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E
-Wl,-rpath,/usr/lib/perl5/5.12.1/x86_64-linux-thread-multi/CORE'
cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64
-fstack-protector'

Locally applied patches​:

---
@​INC for perl 5.12.1​:
.

/home/sulbeck/local/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-
multi

/home/sulbeck/local/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-
multi
/home/sulbeck/local/lib/perl5/site_perl/5.12.1

/home/sulbeck/local/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-
multi
/home/sulbeck/local/lib/perl5/site_perl/5.12.1
/home/sulbeck/local/lib/perl5/site_perl
/home/sulbeck/local/lib/perl5/5.12.1/x86_64-linux-thread-multi
/home/sulbeck/local/lib/perl5/5.12.1/x86_64-linux-thread-multi
/home/sulbeck/local/lib/perl5/5.12.1
/usr/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-multi
/usr/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-multi
/usr/lib/perl5/site_perl/5.12.1
/usr/lib/perl5/vendor_perl/5.12.1/x86_64-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.12.1/x86_64-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.12.1
/usr/lib/perl5/5.12.1/x86_64-linux-thread-multi
/usr/lib/perl5/5.12.1/x86_64-linux-thread-multi
/usr/lib/perl5/5.12.1
/usr/lib/perl5/site_perl/5.12.1/x86_64-linux-thread-multi
/usr/lib/perl5/site_perl/5.12.1
/usr/lib/perl5/vendor_perl/5.12.1/x86_64-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.12.1
/usr/lib/perl5/5.12.1/x86_64-linux-thread-multi
/usr/lib/perl5/5.12.1
.

---
Environment for perl 5.12.1​:
HOME=/home/sulbeck
LANG=en_US.UTF-8
LANGUAGE (unset)
LC_ALL=C
LD_LIBRARY_PATH (unset)
LOGDIR (unset)

PATH=/home/sulbeck/local/bin​:/usr/local/bin​:/usr/local/sbin​:/usr/bin​:/usr/sbin​:/usr/X11R6/bin​:/bin​:/sbin​:/usr/games​:/usr/lib/mit/bin​:/usr/lib/mit/sbin​:/usr/lib/qt3/bin​:/etc​:.

 PERL=/usr/bin/perl

PERL5LIB=.​:/home/sulbeck/local/lib/perl5/site_perl/5.12.1/x86_64-
linux-thread-

multi​:/home/sulbeck/local/lib/perl5/site_perl/5.12.1​:/home/sulbeck/local/lib/perl5/site_perl​:/home/sulbeck/local/lib/perl5/5.12.1/x86_64-

linux-thread-

multi​:/home/sulbeck/local/lib/perl5/5.12.1​:/usr/lib/perl5/site_perl/5.12.1/x86_64-

linux-thread-

multi​:/usr/lib/perl5/site_perl/5.12.1​:/usr/lib/perl5/vendor_perl/5.12.1/x86_64-

linux-thread-
multi​:/usr/lib/perl5/vendor_perl/5.12.1​:/usr/lib/perl5/5.12.1/x86_64-
linux-thread-multi​:/usr/lib/perl5/5.12.1
PERL_BADLANG (unset)
SHELL=/usr/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2011

From @plu

Attached a fix.

Please review that patch. I have to admit to I have no experience at all
on hacking on the Perl source code. I just discovered the issue by
compiling Perl with debug symbols and running a couple of gdb sessions.
I've checked what unused variables are still there when the function
returns and which of those unused variables have still a refcount > 0.

@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2011

From @plu

0001-Fix-78266-Memory-leak-with-named-regexp-captures.patch
From f753a93ee39594777931d507a8acbcab61183436 Mon Sep 17 00:00:00 2001
From: Johannes Plunien <plu@pqpq.de>
Date: Tue, 13 Dec 2011 11:39:52 +0400
Subject: [PATCH] Fix #78266: Memory leak with named regexp captures.

---
 regcomp.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/regcomp.c b/regcomp.c
index b1b8e6f..9e9fac4 100644
--- a/regcomp.c
+++ b/regcomp.c
@@ -5418,6 +5418,10 @@ Perl_reg_named_buff_fetch(pTHX_ REGEXP * const r, SV * const namesv,
                 return newRV_noinc(MUTABLE_SV(retarray));
         }
     }
+
+    if (ret)
+        SvREFCNT_dec(ret);
+
     return NULL;
 }
 
-- 
1.7.7

@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2011

From @plu

Added a testcase.

@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2011

From @plu

0002-Add-testcase-for-78266.patch
From 7d77e85aabc4cf50f733820cb2b793ffc240840c Mon Sep 17 00:00:00 2001
From: Johannes Plunien <plu@pqpq.de>
Date: Tue, 13 Dec 2011 12:29:47 +0400
Subject: [PATCH 2/2] Add testcase for #78266.

---
 t/op/svleak.t |   19 ++++++++++++++++++-
 1 files changed, 18 insertions(+), 1 deletions(-)

diff --git a/t/op/svleak.t b/t/op/svleak.t
index 5eb090c..4eee3c8 100644
--- a/t/op/svleak.t
+++ b/t/op/svleak.t
@@ -13,7 +13,7 @@ BEGIN {
 	or skip_all("XS::APItest not available");
 }
 
-plan tests => 19;
+plan tests => 20;
 
 # run some code N times. If the number of SVs at the end of loop N is
 # greater than (N-1)*delta at the end of loop 1, we've got a leak
@@ -141,3 +141,20 @@ leak(2, 0,
     },
     "rcatline leak"
 );
+
+{
+    my $RE = qr/
+      (?:
+        <(?<tag>
+          \s*
+          [^>\s]+
+        )>
+      )??
+    /xis;
+
+    "<html><body></body></html>" =~ m/$RE/gcs;
+
+    leak(5, 0, sub {
+        my $tag = $+{tag};
+    }, "named regexp captures");
+}
-- 
1.7.7

@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2011

From @cpansprout

On Tue Dec 13 00​:09​:08 2011, plu wrote​:

Attached a fix.

Please review that patch. I have to admit to I have no experience at all
on hacking on the Perl source code. I just discovered the issue by
compiling Perl with debug symbols and running a couple of gdb sessions.
I've checked what unused variables are still there when the function
returns and which of those unused variables have still a refcount > 0.

This patch looks correct to me, so I have applied it as b28f4af.
Thank you.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2011

@cpansprout - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this as completed Dec 13, 2011
@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2011

From @cpansprout

On Tue Dec 13 00​:31​:24 2011, plu wrote​:

Added a testcase.

And this one as a17b855.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2011

From @avar

On Tue, Dec 13, 2011 at 18​:32, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Tue Dec 13 00​:09​:08 2011, plu wrote​:

Attached a fix.

Please review that patch. I have to admit to I have no experience at all
on hacking on the Perl source code. I just discovered the issue by
compiling Perl with debug symbols and running a couple of gdb sessions.
I've checked what unused variables are still there when the function
returns and which of those unused variables have still a refcount > 0.

This patch looks correct to me, so I have applied it as b28f4af.
Thank you.

I had already pushed smoke-me/avar-plu-RT-78266 earlier today as
discussed on #p5p with a better fix for this, which I've now applied
to blead.

In this case we're leaking memory because we're allocating something,
not using it, and then not freeing it. A better solution than freeing
it is to not allocate it in the first place.

I've pushed a better fix to blead in
7402016. Since this was something we
want for 5.14.x backporting I wanted to put it in a smoke-me branch
first, but since you've pushed it to blead anyway

@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2011

From @demerphq

On 13 December 2011 19​:11, Ã�var Arnfjörð Bjarmason <avarab@​gmail.com> wrote​:

On Tue, Dec 13, 2011 at 18​:32, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Tue Dec 13 00​:09​:08 2011, plu wrote​:

Attached a fix.

Please review that patch. I have to admit to I have no experience at all
on hacking on the Perl source code. I just discovered the issue by
compiling Perl with debug symbols and running a couple of gdb sessions.
I've checked what unused variables are still there when the function
returns and which of those unused variables have still a refcount > 0.

This patch looks correct to me, so I have applied it as b28f4af.
Thank you.

I had already pushed smoke-me/avar-plu-RT-78266 earlier today as
discussed on #p5p with a better fix for this, which I've now applied
to blead.

In this case we're leaking memory because we're allocating something,
not using it, and then not freeing it. A better solution than freeing
it is to not allocate it in the first place.

I've pushed a better fix to blead in
7402016. Since this was something we
want for 5.14.x backporting I wanted to put it in a smoke-me branch
first, but since you've pushed it to blead anyway

Concur that this is a better fix. This one was my fault. Sorry people.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2011

From r.khamidullin@transcredit.ru

Johannes Plunien,

Thank You very much.

I can not check that fix at the moment.

13.12.2011 12​:09, Johannes Plunien via RT пиÑ�еÑ�​:

Attached a fix.

Please review that patch. I have to admit to I have no experience at all
on hacking on the Perl source code. I just discovered the issue by
compiling Perl with debug symbols and running a couple of gdb sessions.
I've checked what unused variables are still there when the function
returns and which of those unused variables have still a refcount> 0.

--
С �важением,

Р���ам Хамид�ллин
�л. �пе�иали�� о�дела ин�е�не�-п�оек�ов
��� "Т�ан���еди��а��"
�ел. (495)788-0880 доб. 2701

@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2011

From r.khamidullin@transcredit.ru

Johannes Plunien,

full job. :)

Regards!

13.12.2011 12​:31, Johannes Plunien via RT пиÑ�еÑ�​:

Added a testcase.

--
С �важением,

Р���ам Хамид�ллин
�л. �пе�иали�� о�дела ин�е�не�-п�оек�ов
��� "Т�ан���еди��а��"
�ел. (495)788-0880 доб. 2701

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant