From @wolfsage

Created by wolfsage@gmail.com

With perl-5.14.2​:

  $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
  Can't find string terminator "
  END" anywhere before EOF at (re_eval 1) line 1.
  Compilation failed in regexp at -e line 1.

With blead (and later Perls)​:

  $ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
  Can't find string terminator "END" anywhere before EOF at -e line 1.
  Attempt to free unreferenced scalar​: SV 0x23d9080 at -e line 1.

Bisected with attached bisecter.pl​:

  ../perl-1/Porting/bisect.pl -j 8 --start=v5.14.2 --target=miniperl
-- ./miniperl -Ilib /home/mhorsfall/bisecter.pl 2>&1 | tee ~/out.txt

To​:

  3328ab5 is the first bad commit
  commit 3328ab5
  Author​: Father Chrysostomos <sprout@​cpan.org>
  Date​: Wed Aug 29 12​:35​:49 2012 -0700

  Finish fixing here-docs in re-evals

  This commit fixes here-docs in single-line re-evals in files (as
  opposed to evals) and here-docs in single-line quote-like operators
  inside re-evals.

  In both cases, the here-doc parser has to look into an outer
  lexing scope to find the here-doc body. And in both cases it
  was stomping on PL_linestr (the current line buffer) while
  PL_sublex_info.re_eval_start was pointing to an offset in that buffer.
  (re_eval_start is used to construct the string to include in the
  regexp's stringification once the lexer reaches the end of the
  re-eval.)

  Fixing this entails moving re_eval_start and re_eval_str to
  PL_parser->lex_shared, making the pre-localised values visible.
  This is so that the code that peeks into an outer linestr buffer to
  steal the here-doc body can set up re_eval_str in the right scope.
  (re_eval_str is used to store the re-eval text when the here-
  oc parser has no choice but to modify linestr; see also commit
  db44426.)

  It also entails making the stream-based parser (i.e., that reads from
  an input stream) leave PL_linestr alone, instead of clobbering it and
  then reconstructing part of it afterwards.

Flags:
    category=core
    severity=low

Site configuration information for perl 5.14.2:

Configured by Debian Project at Mon Mar 18 19:16:26 UTC 2013.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:

  Platform:
    osname=linux, osvers=2.6.42-37-generic,
archname=x86_64-linux-gnu-thread-multi
    uname='linux batsu 2.6.42-37-generic #58-ubuntu smp thu jan 24
15:28:10 utc 2013 x86_64 x86_64 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.14 -Darchlib=/usr/lib/perl/5.14
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.14.2
-Dsitearch=/usr/local/lib/perl/5.14.2 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1
-Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh
-Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -DDEBUGGING=-g -Doptimize=-O2
-Duseshrplib -Dlibperl=libperl.so.5.14.2 -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.6.3', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=, so=so, useshrplib=true, libperl=libperl.so.5.14.2
    gnulibc_version='2.15'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib
-fstack-protector'

Locally applied patches:



@INC for perl 5.14.2:
    /etc/perl
    /usr/local/lib/perl/5.14.2
    /usr/local/share/perl/5.14.2
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.14
    /usr/share/perl/5.14
    /usr/local/lib/site_perl
    .


Environment for perl 5.14.2:
    HOME=/home/mhorsfall
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/mhorsfall/perl5/perlbrew/bin:/home/mhorsfall/bin:/home/mhorsfall/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
    PERLBREW_BASHRC_VERSION=0.66
    PERLBREW_HOME=/home/mhorsfall/.perlbrew
    PERLBREW_ROOT=/home/mhorsfall/perl5/perlbrew
    PERL_BADLANG (unset)
    SHELL=/bin/bash

From @wolfsage

bisecter.pl

From @jkeenan

On Fri Mar 14 08​:40​:51 2014, alh wrote​:

This is a bug report for perl from wolfsage@​gmail.com,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
[Please describe your issue here]

With perl-5.14.2​:

$ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
Can't find string terminator "
END" anywhere before EOF at (re_eval 1) line 1.
Compilation failed in regexp at -e line 1.

With blead (and later Perls)​:

$ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
Can't find string terminator "END" anywhere before EOF at -e line 1.
Attempt to free unreferenced scalar​: SV 0x23d9080 at -e line 1.

I can confirm that I got these results in blead, which means we will get them in 5.20 as well.

Thank you very much.
Jim Keenan

The RT System itself - Status changed from 'new' to 'open'

From @jkeenan

On Fri Mar 14 08​:40​:51 2014, alh wrote​:

This is a bug report for perl from wolfsage@​gmail.com,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
[Please describe your issue here]

With perl-5.14.2​:

$ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
Can't find string terminator "
END" anywhere before EOF at (re_eval 1) line 1.
Compilation failed in regexp at -e line 1.

With blead (and later Perls)​:

$ perl -e '/(?{print <<END\nok 64 - here-doc in re-eval\nEND\n})/;'
Can't find string terminator "END" anywhere before EOF at -e line 1.
Attempt to free unreferenced scalar​: SV 0x23d9080 at -e line 1.

Is this really the same issue as https://rt-archive.perl.org/perl5/Ticket/Display.html?id=121438?

Thank you very much.
Jim Keenan

From @wolfsage

On Sun, May 18, 2014 at 6​:58 PM, James E Keenan via RT
<perlbug-followup@​perl.org> wrote​:

Is this really the same issue as https://rt-archive.perl.org/perl5/Ticket/Display.html?id=121438?

I don't believe so.

This issue is that Perl mishandles the heredoc, #121438 is that
B​::Deparse doesn't properly decode such constructs back to their
original form.

-- Matthew Horsfall (alh)

From @iabyn

On Fri, Mar 14, 2014 at 08​:40​:52AM -0700, Matthew Horsfall wrote​:

With perl-5.14.2​:

$ perl \-e '/\(?\{print \<\<END\\nok 64 \- here\-doc in re\-eval\\nEND\\n\}\)/;'
Can't find string terminator "
END" anywhere before EOF at \(re\_eval 1\) line 1\.
Compilation failed in regexp at \-e line 1\.

With blead (and later Perls)​:

$ perl \-e '/\(?\{print \<\<END\\nok 64 \- here\-doc in re\-eval\\nEND\\n\}\)/;'
Can't find string terminator "END" anywhere before EOF at \-e line 1\.
Attempt to free unreferenced scalar&#8203;: SV 0x23d9080 at \-e line 1\.

This appears to have been fixed sometime between 5.21.5 and 5.21.6

--
The optimist believes that he lives in the best of all possible worlds.
As does the pessimist.

From @wolfsage

On Thu, Mar 26, 2015 at 9​:30 AM, Dave Mitchell <davem@​iabyn.com> wrote​:

On Fri, Mar 14, 2014 at 08​:40​:52AM -0700, Matthew Horsfall wrote​:

With perl-5.14.2​:

$ perl \-e '/\(?\{print \<\<END\\nok 64 \- here\-doc in re\-eval\\nEND\\n\}\)/;'
Can't find string terminator "
END" anywhere before EOF at \(re\_eval 1\) line 1\.
Compilation failed in regexp at \-e line 1\.

With blead (and later Perls)​:

$ perl \-e '/\(?\{print \<\<END\\nok 64 \- here\-doc in re\-eval\\nEND\\n\}\)/;'
Can't find string terminator "END" anywhere before EOF at \-e line 1\.
Attempt to free unreferenced scalar&#8203;: SV 0x23d9080 at \-e line 1\.

This appears to have been fixed sometime between 5.21.5 and 5.21.6

According to a bisect, fixed with​:

  commit fd2709d
  Author​: Father Chrysostomos <sprout@​cpan.org>
  Date​: Sat Nov 15 13​:31​:40 2014 -0800

  Fix double free with unterminated /(?{ <<END })/

  If we are parsing from a stream (file), and we are inside a quote-like
  operator, and we find a here-doc marker that tries to extract the
  here-doc body from the stream (this is the last line of the quote-like
  operator, or it only has one line), and the the here-doc terminator
  cannot be found, then we end up trying to free a scalar twice​:

  $ ./miniperl -e '"${ print <<END"'
  Can't find string terminator "END" anywhere before EOF at -e line 1.
  Attempt to free temp prematurely​: SV 0x7fcafb82fd98, Perl
interpreter​: 0x7fcafb803200 at -e line 1.
  Attempt to free unreferenced scalar​: SV 0x7fcafb82fd98, Perl
interpreter​: 0x7fcafb803200 at -e line 1.

  I caused that in v5.17.3-187-g3328ab5.

  The current line of code in the parser is usually stored in
  PL_parser->linestr (aka PL_linestr) and gets freed when the parser
  itself is freed.

  The heredoc parser, when extracting the body from a stream, tempor-
  arily sets aside PL_linestr, replacing it with another SV. If it
  doesn’t find the terminator, it frees the PL_linestr value that has
  been set aside, under the assumption that parser_free will take care
  of freeing the new value.

  Inside quote-like operators that does not work, because PL_linestr has
  been localised and set to a new value, already prospectively freed via
  SAVEFREESV, in sublex_push. So we can’t free that value again.

-- Matthew Horsfall (alh)

@iabyn - Status changed from 'open' to 'pending release'

From @khwilliamson

Thanks for submitting this ticket

The issue should be resolved with the release today of Perl v5.22. If you find that the problem persists, feel free to reopen this ticket

--
Karl Williamson for the Perl 5 porters team

@khwilliamson - Status changed from 'pending release' to 'resolved'