Skip Menu |
Report information
Id: 121437
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: alh <wolfsage [at] gmail.com>
Cc:
AdminCc:

Operating System: Linux
PatchStatus: (no value)
Severity: low
Type: core
Perl Version: 5.14.2
Fixed In:
  • 5.21.6
  • 5.22.0



Subject: "Attempt to free unreferenced scalar..." with failed regexp compilation including heredoc
Date: Fri, 14 Mar 2014 11:40:24 -0400
To: perlbug [...] perl.org
From: "Matthew Horsfall (alh)" <wolfsage [...] gmail.com>
Download (untitled) / with headers
text/plain 5.7k
This is a bug report for perl from wolfsage@gmail.com, generated with the help of perlbug 1.39 running under perl 5.14.2. ----------------------------------------------------------------- [Please describe your issue here] With perl-5.14.2: $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;' Can't find string terminator " END" anywhere before EOF at (re_eval 1) line 1. Compilation failed in regexp at -e line 1. With blead (and later Perls): $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;' Can't find string terminator "END" anywhere before EOF at -e line 1. Attempt to free unreferenced scalar: SV 0x23d9080 at -e line 1. Bisected with attached bisecter.pl: ../perl-1/Porting/bisect.pl -j 8 --start=v5.14.2 --target=miniperl -- ./miniperl -Ilib /home/mhorsfall/bisecter.pl 2>&1 | tee ~/out.txt To: 3328ab5af72319f76fe9be3910a8e07d38b14de2 is the first bad commit commit 3328ab5af72319f76fe9be3910a8e07d38b14de2 Author: Father Chrysostomos <sprout@cpan.org> Date: Wed Aug 29 12:35:49 2012 -0700 Finish fixing here-docs in re-evals This commit fixes here-docs in single-line re-evals in files (as opposed to evals) and here-docs in single-line quote-like operators inside re-evals. In both cases, the here-doc parser has to look into an outer lexing scope to find the here-doc body. And in both cases it was stomping on PL_linestr (the current line buffer) while PL_sublex_info.re_eval_start was pointing to an offset in that buffer. (re_eval_start is used to construct the string to include in the regexp's stringification once the lexer reaches the end of the re-eval.) Fixing this entails moving re_eval_start and re_eval_str to PL_parser->lex_shared, making the pre-localised values visible. This is so that the code that peeks into an outer linestr buffer to steal the here-doc body can set up re_eval_str in the right scope. (re_eval_str is used to store the re-eval text when the here- oc parser has no choice but to modify linestr; see also commit db4442662555874019.) It also entails making the stream-based parser (i.e., that reads from an input stream) leave PL_linestr alone, instead of clobbering it and then reconstructing part of it afterwards. [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=low --- Site configuration information for perl 5.14.2: Configured by Debian Project at Mon Mar 18 19:16:26 UTC 2013. Summary of my perl5 (revision 5 version 14 subversion 2) configuration: Platform: osname=linux, osvers=2.6.42-37-generic, archname=x86_64-linux-gnu-thread-multi uname='linux batsu 2.6.42-37-generic #58-ubuntu smp thu jan 24 15:28:10 utc 2013 x86_64 x86_64 x86_64 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.14 -Darchlib=/usr/lib/perl/5.14 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.14.2 -Dsitearch=/usr/local/lib/perl/5.14.2 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.14.2 -des' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2 -g', cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.6.3', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=, so=so, useshrplib=true, libperl=libperl.so.5.14.2 gnulibc_version='2.15' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector' Locally applied patches: --- @INC for perl 5.14.2: /etc/perl /usr/local/lib/perl/5.14.2 /usr/local/share/perl/5.14.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.14 /usr/share/perl/5.14 /usr/local/lib/site_perl . --- Environment for perl 5.14.2: HOME=/home/mhorsfall LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/mhorsfall/perl5/perlbrew/bin:/home/mhorsfall/bin:/home/mhorsfall/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games PERLBREW_BASHRC_VERSION=0.66 PERLBREW_HOME=/home/mhorsfall/.perlbrew PERLBREW_ROOT=/home/mhorsfall/perl5/perlbrew PERL_BADLANG (unset) SHELL=/bin/bash
Download bisecter.pl
text/x-perl 164b

Message body is not shown because sender requested not to inline it.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 878b
On Fri Mar 14 08:40:51 2014, alh wrote: Show quoted text
> This is a bug report for perl from wolfsage@gmail.com, > generated with the help of perlbug 1.39 running under perl 5.14.2. > > > ----------------------------------------------------------------- > [Please describe your issue here] > > With perl-5.14.2: > > $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;' > Can't find string terminator " > END" anywhere before EOF at (re_eval 1) line 1. > Compilation failed in regexp at -e line 1. > > With blead (and later Perls): > > $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;' > Can't find string terminator "END" anywhere before EOF at -e line 1. > Attempt to free unreferenced scalar: SV 0x23d9080 at -e line 1. >
I can confirm that I got these results in blead, which means we will get them in 5.20 as well. Thank you very much. Jim Keenan
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 866b
On Fri Mar 14 08:40:51 2014, alh wrote: Show quoted text
> This is a bug report for perl from wolfsage@gmail.com, > generated with the help of perlbug 1.39 running under perl 5.14.2. > > > ----------------------------------------------------------------- > [Please describe your issue here] > > With perl-5.14.2: > > $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;' > Can't find string terminator " > END" anywhere before EOF at (re_eval 1) line 1. > Compilation failed in regexp at -e line 1. > > With blead (and later Perls): > > $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;' > Can't find string terminator "END" anywhere before EOF at -e line 1. > Attempt to free unreferenced scalar: SV 0x23d9080 at -e line 1. >
Is this really the same issue as https://rt.perl.org/Ticket/Display.html?id=121438? Thank you very much. Jim Keenan
To: perlbug-followup [...] perl.org
Date: Tue, 27 May 2014 21:09:09 -0400
From: "Matthew Horsfall (alh)" <wolfsage [...] gmail.com>
CC: Perl5 Porters <perl5-porters [...] perl.org>
Subject: Re: [perl #121437] "Attempt to free unreferenced scalar..." with failed regexp compilation including heredoc
Download (untitled) / with headers
text/plain 372b
On Sun, May 18, 2014 at 6:58 PM, James E Keenan via RT <perlbug-followup@perl.org> wrote: Show quoted text
> Is this really the same issue as https://rt.perl.org/Ticket/Display.html?id=121438? >
I don't believe so. This issue is that Perl mishandles the heredoc, #121438 is that B::Deparse doesn't properly decode such constructs back to their original form. -- Matthew Horsfall (alh)
From: Dave Mitchell <davem [...] iabyn.com>
Date: Thu, 26 Mar 2015 13:30:55 +0000
To: perl5-porters [...] perl.org
Subject: Re: [perl #121437] "Attempt to free unreferenced scalar..." with failed regexp compilation including heredoc
Download (untitled) / with headers
text/plain 730b
On Fri, Mar 14, 2014 at 08:40:52AM -0700, Matthew Horsfall wrote: Show quoted text
> With perl-5.14.2: > > $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;' > Can't find string terminator " > END" anywhere before EOF at (re_eval 1) line 1. > Compilation failed in regexp at -e line 1. > > With blead (and later Perls): > > $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;' > Can't find string terminator "END" anywhere before EOF at -e line 1. > Attempt to free unreferenced scalar: SV 0x23d9080 at -e line 1.
This appears to have been fixed sometime between 5.21.5 and 5.21.6 -- The optimist believes that he lives in the best of all possible worlds. As does the pessimist.
From: "Matthew Horsfall (alh)" <wolfsage [...] gmail.com>
Date: Thu, 26 Mar 2015 10:44:34 -0400
To: Perl5 Porters <perl5-porters [...] perl.org>
Subject: Re: [perl #121437] "Attempt to free unreferenced scalar..." with failed regexp compilation including heredoc
Download (untitled) / with headers
text/plain 2.3k
On Thu, Mar 26, 2015 at 9:30 AM, Dave Mitchell <davem@iabyn.com> wrote: Show quoted text
> On Fri, Mar 14, 2014 at 08:40:52AM -0700, Matthew Horsfall wrote:
>> With perl-5.14.2: >> >> $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;' >> Can't find string terminator " >> END" anywhere before EOF at (re_eval 1) line 1. >> Compilation failed in regexp at -e line 1. >> >> With blead (and later Perls): >> >> $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;' >> Can't find string terminator "END" anywhere before EOF at -e line 1. >> Attempt to free unreferenced scalar: SV 0x23d9080 at -e line 1.
> > This appears to have been fixed sometime between 5.21.5 and 5.21.6
According to a bisect, fixed with: commit fd2709db56c97e05ef8ae1f7c0f586d6f61103c2 Author: Father Chrysostomos <sprout@cpan.org> Date: Sat Nov 15 13:31:40 2014 -0800 Fix double free with unterminated /(?{ <<END })/ If we are parsing from a stream (file), and we are inside a quote-like operator, and we find a here-doc marker that tries to extract the here-doc body from the stream (this is the last line of the quote-like operator, or it only has one line), and the the here-doc terminator cannot be found, then we end up trying to free a scalar twice: $ ./miniperl -e '"${ print <<END"' Can't find string terminator "END" anywhere before EOF at -e line 1. Attempt to free temp prematurely: SV 0x7fcafb82fd98, Perl interpreter: 0x7fcafb803200 at -e line 1. Attempt to free unreferenced scalar: SV 0x7fcafb82fd98, Perl interpreter: 0x7fcafb803200 at -e line 1. I caused that in v5.17.3-187-g3328ab5. The current line of code in the parser is usually stored in PL_parser->linestr (aka PL_linestr) and gets freed when the parser itself is freed. The heredoc parser, when extracting the body from a stream, tempor- arily sets aside PL_linestr, replacing it with another SV. If it doesn’t find the terminator, it frees the PL_linestr value that has been set aside, under the assumption that parser_free will take care of freeing the new value. Inside quote-like operators that does not work, because PL_linestr has been localised and set to a new value, already prospectively freed via SAVEFREESV, in sublex_push. So we can’t free that value again. -- Matthew Horsfall (alh)
Subject: Your ticket against Perl 5 has been resolved
Download (untitled) / with headers
text/plain 222b
Thanks for submitting this ticket The issue should be resolved with the release today of Perl v5.22. If you find that the problem persists, feel free to reopen this ticket -- Karl Williamson for the Perl 5 porters team


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org