Skip Menu |
Report information
Id: 132892
Status: open
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: rootkwok [at] cpan.org
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: (no value)



Date: Wed, 21 Feb 2018 23:48:59 +0800
From: 郭樂聰 <rootkwok [...] cpan.org>
Subject: Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
To: perlbug [...] perl.org
Download (untitled) / with headers
text/plain 2.3k
Hello guys,

I have a long running perl script which allows the user configure the program behavior with a hash that got pre-compiled regexps
as keys and the corresponding callback code as the value like below. Recently I have upgraded to perl v5.26 and the memory
usage grows since then, so I have fall back to perl v5.24.3 as a temporary solution. What I observed before fallback is that the
memory grows when I try to iterates the hash keys and matches them against some text.

Show quoted text
%action = (
Show quoted text
  qr/^regex here/ => sub { print "match" }, 
Show quoted text
); 

I have written a small test for this problem, you can find it in the attached perlbug.tar.xz. 20180221-mem-leak-poc.pl is the prof of
concept code. It will output the platform and perl version as the first line, and make use of the ps command to retrieve the vsz and
rss of the perl interpreter before each iteration. The test result of different platform are stored in the rslt/ sub-directory for your reference.
According to the results, with perl <v5.26, the size in memory remain unchanged; While the memory usage keeps increasing when the
script is run with perl >=v5.26. I have limited the number of iteration, the memory usage grows further if you increases the number of iteration.

`-- rslt
    |-- centos6-p5.10.1.log
    |-- centos6-p5.24.3.log
    |-- centos6-p5.26.1.log
    |-- centos6-p5.27.8.log
    |-- dfbsd-p5.24.3.log
    |-- dfbsd-p5.26.1.log
    |-- freebsd-p5.24.3.log
    `-- freebsd-p5.26.1.log

According to my empirical result, this problem occurs when using perl newer than v5.26, the first line of perl -v of the affected perl
interpreters below.
Show quoted text
This is perl 5, version 26, subversion 1 (v5.26.1) built for amd64-freebsd
This is perl 5, version 27, subversion 8 (v5.27.8) built for x86_64-linux
This is perl 5, version 26, subversion 1 (v5.26.1) built for x86_64-linux
Show quoted text
This is perl 5, version 26, subversion 1 (v5.26.1) built for x86_64-dragonfly

I have traced the problem a little bit and I *guess* the problem is in the function S_concat_pat of regcomp.c and the new SVs
copied out are immortal, so every time my program iterate through the keys of hash to match against things, it creates more
copies of the regexes. I can try to dig deeper when I am free to build the debug versions of perl, but that would be great if this
case can be taken care by the professionals here. Thanks a lot.

Best regards,
Baggio
Download code.png
image/png 49.3k
code.png
Download perlbug.tar.xz
application/x-xz 2.8k

Message body not shown because it is not plain text.

From: Shlomi Fish <shlomif [...] shlomifish.org>
CC: perl5-porters [...] perl.org
Date: Thu, 22 Feb 2018 09:44:45 +0200
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
To: "(via RT)" <perlbug-followup [...] perl.org>
Download (untitled) / with headers
text/plain 3.5k
On Wed, 21 Feb 2018 07:49:20 -0800 (via RT) <perlbug-followup@perl.org> wrote: Show quoted text
> # New Ticket Created by > # Please include the string: [perl #132892] > # in the subject line of all future correspondence about this issue. > # <URL: https://rt.perl.org/Ticket/Display.html?id=132892 > > >
Hi all! I reworked the example code to remove some potential problems, and verified that htop sees its RAM consumption increasing: #!/usr/bin/perl use strict; use warnings; # my $getmem = "ps -o vsz= -o rss= -p $$", print "== Test result on $^O perl-$^V\n"; my %h = ( qr{abcd} => [ 'ABCD', 2], qr{efgh} => [ 'EFGH', 1], qr{ijkl} => [ 'IJKL', 1], qr{mnop} => [ 'MNOP', 1], qr{qrst} => [ 'QRST', 1], qr{uvwx} => [ 'UVWX', 1], ); my @list = qw/ abcd abcd efgh ijkl mnop qrst uvwx /; while (1) { # system($getmem); for my $r (keys %h) { my $c = 0; my ($name, $expected) = @{$h{$r}}; for my $e ( @list ) { $c++ if $e =~ $r; } print ">>> Count mismatch for $name\n" if ( $c != $expected ); } } Show quoted text
> Hello guys, > > I have a long running perl script which allows the user configure the > program behavior with a hash that got pre-compiled regexps > as keys and the corresponding callback code as the value like below. > Recently I have upgraded to perl v5.26 and the memory > usage grows since then, so I have fall back to perl v5.24.3 as a temporary > solution. What I observed before fallback is that the > memory grows when I try to iterates the hash keys and matches them against > some text. > > %action = ( > > qr/^regex here/ => sub { print "match" }, > > ); > > > I have written a small test for this problem, you can find it in the > attached perlbug.tar.xz. 20180221-mem-leak-poc.pl is the prof of > concept code. It will output the platform and perl version as the first > line, and make use of the ps command to retrieve the vsz and > rss of the perl interpreter before each iteration. The test result of > different platform are stored in the rslt/ sub-directory for your reference. > According to the results, with perl <v5.26, the size in memory remain > unchanged; While the memory usage keeps increasing when the > script is run with perl >=v5.26. I have limited the number of iteration, > the memory usage grows further if you increases the number of iteration. > > |-- 20180221-mem-leak-poc.pl > `-- rslt > |-- centos6-p5.10.1.log > |-- centos6-p5.24.3.log > |-- centos6-p5.26.1.log > |-- centos6-p5.27.8.log > |-- dfbsd-p5.24.3.log > |-- dfbsd-p5.26.1.log > |-- freebsd-p5.24.3.log > `-- freebsd-p5.26.1.log > > According to my empirical result, this problem occurs when using perl newer > than v5.26, the first line of perl -v of the affected perl > interpreters below. >
> > This is perl 5, version 26, subversion 1 (v5.26.1) built for amd64-freebsd > > This is perl 5, version 27, subversion 8 (v5.27.8) built for x86_64-linux > > This is perl 5, version 26, subversion 1 (v5.26.1) built for x86_64-linux
> > This is perl 5, version 26, subversion 1 (v5.26.1) built for
> > x86_64-dragonfly
> > > I have traced the problem a little bit and I *guess* the problem is in the > function S_concat_pat of regcomp.c and the new SVs > copied out are immortal, so every time my program iterate through the keys > of hash to match against things, it creates more > copies of the regexes. I can try to dig deeper when I am free to build the > debug versions of perl, but that would be great if this > case can be taken care by the professionals here. Thanks a lot. > ​ > > ​ > Best regards, > Baggio
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 543b
Bisect points to commit b10cb25a6c86fd96fff8f2dfa6d8df3e6b51a451 Author: Yves Orton <demerphq@gmail.com> Date: Sat Sep 17 20:14:53 2016 +0200 regcomp.c: S_concat_pat: guard against missing trailing nulls The regex engine expects the pattern to have a null byte at SvEND(pat), but is not guaranteed to receive such a pattern when it is called, so S_concat_pat should guard against this case. It turns out this is only an issue when there is exactly one "argument" to the pattern. (Consider concatenation rules, etc).
To: Perl5 Porteros <perl5-porters [...] perl.org>
Date: Thu, 22 Feb 2018 12:50:37 +0100
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
From: demerphq <demerphq [...] gmail.com>
Download (untitled) / with headers
text/plain 2.8k


On 22 Feb 2018 12:39, "via RT" <perlbug-followup@perl.org> wrote:
Show quoted text
# New Ticket Created by
# Please include the string:  [perl #132892]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt.perl.org/Ticket/Display.html?id=132892 >


Hello guys,

I have a long running perl script which allows the user configure the
program behavior with a hash that got pre-compiled regexps
as keys and the corresponding callback code as the value like below.
Recently I have upgraded to perl v5.26 and the memory
usage grows since then, so I have fall back to perl v5.24.3 as a temporary
solution. What I observed before fallback is that the
memory grows when I try to iterates the hash keys and matches them against
some text.

%action = (

  qr/^regex here/ => sub { print "match" },

);

Regardless if the bug, perl hash keys are strings not objects. Perl will compile these patterns, then stringify them to store them as keys, and then when used later will recompile again. You need a better data structure.

Yves

Show quoted text


I have written a small test for this problem, you can find it in the
attached perlbug.tar.xz. 20180221-mem-leak-poc.pl is the prof of
concept code. It will output the platform and perl version as the first
line, and make use of the ps command to retrieve the vsz and
rss of the perl interpreter before each iteration. The test result of
different platform are stored in the rslt/ sub-directory for your reference.
According to the results, with perl <v5.26, the size in memory remain
unchanged; While the memory usage keeps increasing when the
script is run with perl >=v5.26. I have limited the number of iteration,
the memory usage grows further if you increases the number of iteration.

`-- rslt
    |-- centos6-p5.10.1.log
    |-- centos6-p5.24.3.log
    |-- centos6-p5.26.1.log
    |-- centos6-p5.27.8.log
    |-- dfbsd-p5.24.3.log
    |-- dfbsd-p5.26.1.log
    |-- freebsd-p5.24.3.log
    `-- freebsd-p5.26.1.log

According to my empirical result, this problem occurs when using perl newer
than v5.26, the first line of perl -v of the affected perl
interpreters below.

> This is perl 5, version 26, subversion 1 (v5.26.1) built for amd64-freebsd
> This is perl 5, version 27, subversion 8 (v5.27.8) built for x86_64-linux
> This is perl 5, version 26, subversion 1 (v5.26.1) built for x86_64-linux

This is perl 5, version 26, subversion 1 (v5.26.1) built for
> x86_64-dragonfly


I have traced the problem a little bit and I *guess* the problem is in the
function S_concat_pat of regcomp.c and the new SVs
copied out are immortal, so every time my program iterate through the keys
of hash to match against things, it creates more
copies of the regexes. I can try to dig deeper when I am free to build the
debug versions of perl, but that would be great if this
case can be taken care by the professionals here. Thanks a lot.



Best regards,
Baggio

Date: Fri, 23 Feb 2018 04:27:25 +0100
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
To: Perl RT Bug Tracker <perlbug-followup [...] perl.org>
CC: Perl5 Porteros <perl5-porters [...] perl.org>
From: demerphq <demerphq [...] gmail.com>
Download (untitled) / with headers
text/plain 813b
On 22 February 2018 at 12:29, Sergey Aleynikov via RT <perlbug-followup@perl.org> wrote: Show quoted text
> Bisect points to commit b10cb25a6c86fd96fff8f2dfa6d8df3e6b51a451 > Author: Yves Orton <demerphq@gmail.com> > Date: Sat Sep 17 20:14:53 2016 +0200 > > regcomp.c: S_concat_pat: guard against missing trailing nulls > > The regex engine expects the pattern to have a null byte at > SvEND(pat), but is not guaranteed to receive such a pattern > when it is called, so S_concat_pat should guard against this > case. It turns out this is only an issue when there is exactly > one "argument" to the pattern. (Consider concatenation rules, etc).
Thanks, I forgot an sv_2mortal in that patch, I will push a fix soon. Much obliged for the bisect! Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
From: demerphq <demerphq [...] gmail.com>
CC: Perl5 Porteros <perl5-porters [...] perl.org>
To: Perl RT Bug Tracker <perlbug-followup [...] perl.org>
Date: Fri, 23 Feb 2018 10:34:50 +0100
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
On 23 February 2018 at 04:27, demerphq <demerphq@gmail.com> wrote: Show quoted text
> On 22 February 2018 at 12:29, Sergey Aleynikov via RT > <perlbug-followup@perl.org> wrote:
>> Bisect points to commit b10cb25a6c86fd96fff8f2dfa6d8df3e6b51a451 >> Author: Yves Orton <demerphq@gmail.com> >> Date: Sat Sep 17 20:14:53 2016 +0200 >> >> regcomp.c: S_concat_pat: guard against missing trailing nulls >> >> The regex engine expects the pattern to have a null byte at >> SvEND(pat), but is not guaranteed to receive such a pattern >> when it is called, so S_concat_pat should guard against this >> case. It turns out this is only an issue when there is exactly >> one "argument" to the pattern. (Consider concatenation rules, etc).
> > Thanks, I forgot an sv_2mortal in that patch, I will push a fix soon. > > Much obliged for the bisect!
Fixed in 910a6a8be166fb3780dcd2520e3526e537383ef2 Unfortunately I don't know how to test for a memory leak, so no tests included. Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
From: Dave Mitchell <davem [...] iabyn.com>
Date: Fri, 23 Feb 2018 13:03:16 +0000
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
To: demerphq <demerphq [...] gmail.com>
CC: Perl RT Bug Tracker <perlbug-followup [...] perl.org>, Perl5 Porteros <perl5-porters [...] perl.org>
Download (untitled) / with headers
text/plain 453b
On Fri, Feb 23, 2018 at 10:34:50AM +0100, demerphq wrote: Show quoted text
> Fixed in 910a6a8be166fb3780dcd2520e3526e537383ef2 > > Unfortunately I don't know how to test for a memory leak, so no tests included.
see t/op/svleak.t. It runs a bit of code several times in a loop and tests whether PL_sv_count fails to remain constant. -- Little fly, thy summer's play my thoughtless hand has terminated with extreme prejudice. (with apologies to William Blake)
From: demerphq <demerphq [...] gmail.com>
To: Dave Mitchell <davem [...] iabyn.com>
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
Date: Sun, 25 Feb 2018 10:50:12 +0100
CC: Perl RT Bug Tracker <perlbug-followup [...] perl.org>, Perl5 Porteros <perl5-porters [...] perl.org>
Download (untitled) / with headers
text/plain 680b
On 23 February 2018 at 14:03, Dave Mitchell <davem@iabyn.com> wrote: Show quoted text
> On Fri, Feb 23, 2018 at 10:34:50AM +0100, demerphq wrote:
>> Fixed in 910a6a8be166fb3780dcd2520e3526e537383ef2 >> >> Unfortunately I don't know how to test for a memory leak, so no tests included.
> > see t/op/svleak.t. It runs a bit of code several times in a loop and > tests whether PL_sv_count fails to remain constant.
Thanks, i just pushed 06f26dc89ebc43d50835aaecc477ec160a5a5835 which does just that. FWIW, there is still an open question why this happens at all. Keys should be null terminated, and we should not fall into this code-path. Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
From: demerphq <demerphq [...] gmail.com>
To: Dave Mitchell <davem [...] iabyn.com>
Date: Sun, 25 Feb 2018 10:57:59 +0100
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
CC: Perl RT Bug Tracker <perlbug-followup [...] perl.org>, Perl5 Porteros <perl5-porters [...] perl.org>
On 25 February 2018 at 10:50, demerphq <demerphq@gmail.com> wrote: Show quoted text
> On 23 February 2018 at 14:03, Dave Mitchell <davem@iabyn.com> wrote:
>> On Fri, Feb 23, 2018 at 10:34:50AM +0100, demerphq wrote:
>>> Fixed in 910a6a8be166fb3780dcd2520e3526e537383ef2 >>> >>> Unfortunately I don't know how to test for a memory leak, so no tests included.
>> >> see t/op/svleak.t. It runs a bit of code several times in a loop and >> tests whether PL_sv_count fails to remain constant.
> > Thanks, i just pushed 06f26dc89ebc43d50835aaecc477ec160a5a5835 which > does just that. > > FWIW, there is still an open question why this happens at all. Keys > should be null terminated, and we should not fall into this code-path.
Apparently I am wrong: $ ./perl -Ilib -MDevel::Peek -le'%h=("^foo"=>1); my ($re)=keys %h; Dump $re' SV = PV(0x1009d08) at 0x103f0b8 REFCNT = 1 FLAGS = (POK,IsCOW,pPOK) PV = 0x1041478 "^foo" CUR = 4 LEN = 0 This appears to be a side-effect of shared keys. Another good reason not to put patterns in keys. :-( Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 231b
On Fri, 23 Feb 2018 01:35:02 -0800, demerphq wrote: Show quoted text
> Fixed in 910a6a8be166fb3780dcd2520e3526e537383ef2
Since this is a regression in 5.26, should it also be added to voting list for the future 5.26.2? Or there're no plans for it?
To: Perl RT Bug Tracker <perlbug-followup [...] perl.org>
Date: Sun, 25 Feb 2018 11:13:00 +0100
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
CC: Perl5 Porteros <perl5-porters [...] perl.org>
From: demerphq <demerphq [...] gmail.com>
Download (untitled) / with headers
text/plain 471b
On 25 February 2018 at 11:00, Sergey Aleynikov via RT <perlbug-followup@perl.org> wrote: Show quoted text
> On Fri, 23 Feb 2018 01:35:02 -0800, demerphq wrote:
>> Fixed in 910a6a8be166fb3780dcd2520e3526e537383ef2
> > Since this is a regression in 5.26, should it also be added to voting list for the future 5.26.2? Or there're no plans for it?
Sure. I don't remember how to do that tho. Where does the voting list live again? Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
Date: Sun, 25 Feb 2018 11:16:01 +0100
To: Dave Mitchell <davem [...] iabyn.com>
CC: Perl RT Bug Tracker <perlbug-followup [...] perl.org>, Perl5 Porteros <perl5-porters [...] perl.org>
From: demerphq <demerphq [...] gmail.com>
Download (untitled) / with headers
text/plain 1.5k
On 25 February 2018 at 10:57, demerphq <demerphq@gmail.com> wrote: Show quoted text
> On 25 February 2018 at 10:50, demerphq <demerphq@gmail.com> wrote:
>> On 23 February 2018 at 14:03, Dave Mitchell <davem@iabyn.com> wrote:
>>> On Fri, Feb 23, 2018 at 10:34:50AM +0100, demerphq wrote:
>>>> Fixed in 910a6a8be166fb3780dcd2520e3526e537383ef2 >>>> >>>> Unfortunately I don't know how to test for a memory leak, so no tests included.
>>> >>> see t/op/svleak.t. It runs a bit of code several times in a loop and >>> tests whether PL_sv_count fails to remain constant.
>> >> Thanks, i just pushed 06f26dc89ebc43d50835aaecc477ec160a5a5835 which >> does just that. >> >> FWIW, there is still an open question why this happens at all. Keys >> should be null terminated, and we should not fall into this code-path.
> > Apparently I am wrong: > > $ ./perl -Ilib -MDevel::Peek -le'%h=("^foo"=>1); my ($re)=keys %h; Dump $re' > SV = PV(0x1009d08) at 0x103f0b8 > REFCNT = 1 > FLAGS = (POK,IsCOW,pPOK) > PV = 0x1041478 "^foo" > CUR = 4 > LEN = 0 > > This appears to be a side-effect of shared keys. > > Another good reason not to put patterns in keys. :-(
Corroborating this I hacked a way to get an unshared hash, and then checked what things look like, and the keys are not shared as expected, and are null terminated. $ ./perl -Ilib -MHash::Util=new_unshared_hash -MDevel::Peek -e'my $hv=new_unshared_hash(); $hv->{foo}=1; Dump((keys %$hv)[0])' SV = PV(0x2670c58) at 0x2699680 REFCNT = 1 FLAGS = (TEMP,POK,pPOK) PV = 0x269f9e8 "foo"\0 CUR = 3 LEN = 10 Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 556b
On Sun, 25 Feb 2018 01:58:15 -0800, demerphq wrote: Show quoted text
> $ ./perl -Ilib -MDevel::Peek -le'%h=("^foo"=>1); my ($re)=keys %h; > Dump $re' > SV = PV(0x1009d08) at 0x103f0b8 > REFCNT = 1 > FLAGS = (POK,IsCOW,pPOK) > PV = 0x1041478 "^foo" > CUR = 4 > LEN = 0 > > This appears to be a side-effect of shared keys.
But the failed check in regcomp.c is not *(SvEND(msv)) == 0, but rather SvLEN(msv) > SvCUR(msv). But doesn't line 3134 in hv.c (HEK_KEY(hek)[len] = 0;) guarantee that it's implicitly null-terminated, just not shown by Devel::Peek as such?
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 191b
On Sun, 25 Feb 2018 02:13:12 -0800, demerphq wrote: Show quoted text
> Sure. I don't remember how to do that tho. Where does the voting list > live again?
Looks like votes-5.26.xml in a maint-votes branch.
CC: Perl5 Porteros <perl5-porters [...] perl.org>
Date: Sun, 25 Feb 2018 11:56:54 +0100
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
To: Perl RT Bug Tracker <perlbug-followup [...] perl.org>
From: demerphq <demerphq [...] gmail.com>
Download (untitled) / with headers
text/plain 896b
On 25 February 2018 at 11:19, Sergey Aleynikov via RT <perlbug-followup@perl.org> wrote: Show quoted text
> On Sun, 25 Feb 2018 01:58:15 -0800, demerphq wrote: >
>> $ ./perl -Ilib -MDevel::Peek -le'%h=("^foo"=>1); my ($re)=keys %h; >> Dump $re' >> SV = PV(0x1009d08) at 0x103f0b8 >> REFCNT = 1 >> FLAGS = (POK,IsCOW,pPOK) >> PV = 0x1041478 "^foo" >> CUR = 4 >> LEN = 0 >> >> This appears to be a side-effect of shared keys.
> > > But the failed check in regcomp.c is not *(SvEND(msv)) == 0, but rather SvLEN(msv) > SvCUR(msv). But doesn't line 3134 in hv.c (HEK_KEY(hek)[len] = 0;) guarantee that it's implicitly null-terminated, just not shown by Devel::Peek as such?
That sounds correct, but i dont think the regex engine has any way to know that this LEN=0 case comes from a key. Since we don't know the len we cant safely check past CUR. Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 316b
On Sun, 25 Feb 2018 02:57:09 -0800, demerphq wrote: Show quoted text
> That sounds correct, but i dont think the regex engine has any way to > know that this LEN=0 case comes from a key. Since we don't know the > len we cant safely check past CUR.
But you can check for SvIsCOW_shared_hash, which guarantees it to be from sharepvn.
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
Date: Sun, 25 Feb 2018 17:21:34 +0200
To: demerphq <demerphq [...] gmail.com>, Perl RT Bug Tracker <perlbug-followup [...] perl.org>
CC: Perl5 Porteros <perl5-porters [...] perl.org>
From: Sawyer X <xsawyerx [...] gmail.com>
Download (untitled) / with headers
text/plain 661b
On 02/25/2018 12:13 PM, demerphq wrote: Show quoted text
> On 25 February 2018 at 11:00, Sergey Aleynikov via RT > <perlbug-followup@perl.org> wrote:
>> On Fri, 23 Feb 2018 01:35:02 -0800, demerphq wrote:
>>> Fixed in 910a6a8be166fb3780dcd2520e3526e537383ef2
>> Since this is a regression in 5.26, should it also be added to voting list for the future 5.26.2? Or there're no plans for it?
It should be, yes! Show quoted text
> Sure. I don't remember how to do that tho. Where does the voting list > live again?
The branch "maint-votes" in git. I've added them here: https://perl5.git.perl.org/perl.git/commitdiff/5f21d69b722f6a73f185e533ecf138e17adeabce With a vote from both of us, Yves.
From: demerphq <demerphq [...] gmail.com>
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
Date: Sun, 25 Feb 2018 16:26:57 +0100
To: Sawyer X <xsawyerx [...] gmail.com>
CC: Perl RT Bug Tracker <perlbug-followup [...] perl.org>, Perl5 Porteros <perl5-porters [...] perl.org>
Download (untitled) / with headers
text/plain 1.1k
On 25 February 2018 at 16:21, Sawyer X <xsawyerx@gmail.com> wrote: Show quoted text
> On 02/25/2018 12:13 PM, demerphq wrote:
>> On 25 February 2018 at 11:00, Sergey Aleynikov via RT >> <perlbug-followup@perl.org> wrote:
>>> On Fri, 23 Feb 2018 01:35:02 -0800, demerphq wrote:
>>>> Fixed in 910a6a8be166fb3780dcd2520e3526e537383ef2
>>> Since this is a regression in 5.26, should it also be added to voting list for the future 5.26.2? Or there're no plans for it?
> > It should be, yes! >
>> Sure. I don't remember how to do that tho. Where does the voting list >> live again?
> > The branch "maint-votes" in git. I've added them here: > > https://perl5.git.perl.org/perl.git/commitdiff/5f21d69b722f6a73f185e533ecf138e17adeabce > > With a vote from both of us, Yves.
Thanks, I was busy trying to test Sergey Aleynikov's suggestion of using SvIsCOW_shared_hash when my perl build started throwing some weird ass errors, so assuming that the reboot and clean build i just did makes the weirdness go away, there might be one more commit required. For something like this is it preferred to backport the full patch sequence, or is it preferable to prepare a squashed version? Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
From: demerphq <demerphq [...] gmail.com>
CC: Perl5 Porteros <perl5-porters [...] perl.org>
To: Perl RT Bug Tracker <perlbug-followup [...] perl.org>
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
Date: Sun, 25 Feb 2018 17:18:02 +0100
Download (untitled) / with headers
text/plain 605b
On 25 February 2018 at 12:03, Sergey Aleynikov via RT <perlbug-followup@perl.org> wrote: Show quoted text
> On Sun, 25 Feb 2018 02:57:09 -0800, demerphq wrote: >
>> That sounds correct, but i dont think the regex engine has any way to >> know that this LEN=0 case comes from a key. Since we don't know the >> len we cant safely check past CUR.
> > But you can check for SvIsCOW_shared_hash, which guarantees it to be from sharepvn.
Thanks again for your help, i have pushed a patch to implement this suggestion: f1d945b85ac2d18ddd1ed2e1d4f72011246d905a cheers, Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
From: Sawyer X <xsawyerx [...] gmail.com>
CC: Perl RT Bug Tracker <perlbug-followup [...] perl.org>, Perl5 Porteros <perl5-porters [...] perl.org>
To: demerphq <demerphq [...] gmail.com>
Date: Sun, 25 Feb 2018 19:42:50 +0200
Subject: Re: [perl #132892] Possibly memory leak when matching strings using pre-compiled regexes stored in hash key (perl >= v5.26)
Download (untitled) / with headers
text/plain 1.2k
On 02/25/2018 05:26 PM, demerphq wrote: Show quoted text
> On 25 February 2018 at 16:21, Sawyer X <xsawyerx@gmail.com> wrote:
>> On 02/25/2018 12:13 PM, demerphq wrote:
>>> On 25 February 2018 at 11:00, Sergey Aleynikov via RT >>> <perlbug-followup@perl.org> wrote:
>>>> On Fri, 23 Feb 2018 01:35:02 -0800, demerphq wrote:
>>>>> Fixed in 910a6a8be166fb3780dcd2520e3526e537383ef2
>>>> Since this is a regression in 5.26, should it also be added to voting list for the future 5.26.2? Or there're no plans for it?
>> It should be, yes! >>
>>> Sure. I don't remember how to do that tho. Where does the voting list >>> live again?
>> The branch "maint-votes" in git. I've added them here: >> >> https://perl5.git.perl.org/perl.git/commitdiff/5f21d69b722f6a73f185e533ecf138e17adeabce >> >> With a vote from both of us, Yves.
> Thanks, I was busy trying to test Sergey Aleynikov's suggestion of > using SvIsCOW_shared_hash when my perl build started throwing some > weird ass errors, so assuming that the reboot and clean build i just > did makes the weirdness go away, there might be one more commit > required. For something like this is it preferred to backport the full > patch sequence, or is it preferable to prepare a squashed version?
I think separate patches are better. It will align with blead.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org