From @geeknik

perl -e 'qq{@​{sub{]]}}};qq{@​{sub{]]}}}}}}'

triggers a null ptr deref and segfault in perl v5.21.7 (v5.21.6-602-ge9d2bd8).

#0 Perl_pad_add_anon (func=0x0, optype=<optimized out>) at pad.c​:821
#1 0x00000000004471c3 in Perl_ck_anoncode (o=0xe2b510) at op.c​:9430
#2 0x00000000004b1d7e in Perl_newSVOP (sv=<optimized out>, flags=0, type=18) at op.c​:5850
#3 Perl_newANONATTRSUB (floor=<optimized out>, proto=<optimized out>, attrs=<optimized out>,
  block=<optimized out>) at op.c​:9298
#4 0x00000000005c690b in Perl_yyparse (gramtype=<optimized out>) at perly.y​:842
#5 0x00000000004f0875 in S_parse_body (xsinit=0x42ac70 <xs_init>, env=0x0) at perl.c​:2271
#6 perl_parse (my_perl=<optimized out>, xsinit=0x42ac70 <xs_init>, argc=<optimized out>,
  argv=<optimized out>, env=0x0) at perl.c​:1605
#7 0x000000000042a87c in main (argc=2, argv=0x7fffffffe3a8, env=0x7fffffffe3c0)
  at perlmain.c​:114
#8 0x00007ffff6f98ead in __libc_start_main (main=<optimized out>, argc=<optimized out>,
  ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>,
  rtld_fini=<optimized out>, stack_end=0x7fffffffe398) at libc-start.c​:244
#9 0x000000000042ab95 in _start ()

%%%

==55121== Invalid read of size 8
==55121== at 0x5EAA1A​: Perl_pad_add_anon (pad.c​:821)
==55121== by 0x4471C2​: Perl_ck_anoncode (op.c​:9430)
==55121== by 0x4B1D7D​: Perl_newSVOP (op.c​:5850)
==55121== by 0x4B1D7D​: Perl_newANONATTRSUB (op.c​:9298)
==55121== by 0x5C690A​: Perl_yyparse (perly.y​:842)
==55121== by 0x4F0874​: S_parse_body (perl.c​:2271)
==55121== by 0x4F0874​: perl_parse (perl.c​:1605)
==55121== by 0x42A87B​: main (perlmain.c​:114)
==55121== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==55121== Process terminating with default action of signal 11 (SIGSEGV)
==55121== Access not within mapped region at address 0x0
==55121== at 0x5EAA1A​: Perl_pad_add_anon (pad.c​:821)
==55121== by 0x4471C2​: Perl_ck_anoncode (op.c​:9430)
==55121== by 0x4B1D7D​: Perl_newSVOP (op.c​:5850)
==55121== by 0x4B1D7D​: Perl_newANONATTRSUB (op.c​:9298)
==55121== by 0x5C690A​: Perl_yyparse (perly.y​:842)
==55121== by 0x4F0874​: S_parse_body (perl.c​:2271)
==55121== by 0x4F0874​: perl_parse (perl.c​:1605)
==55121== by 0x42A87B​: main (perlmain.c​:114)

It only triggers an assertion failed in perl v5.23.4 (v5.23.3-7-ge120c24)​:
perl​: op.c​:5789​: Perl_newSVOP​: Assertion `sv' failed.
Aborted

From @geeknik

On Fri Sep 25 19​:38​:48 2015, brian.carpenter@​gmail.com wrote​:

perl -e 'qq{@​{sub{]]}}};qq{@​{sub{]]}}}}}}'

v5.25.5 (v5.25.4-110-g95c0a76)

perl​: op.c​:5966​: OP *Perl_newSVOP(I32, I32, SV *)​: Assertion `sv' failed.

From [Unknown Contact. See original ticket]

On Fri Sep 25 19​:38​:48 2015, brian.carpenter@​gmail.com wrote​:

perl -e 'qq{@​{sub{]]}}};qq{@​{sub{]]}}}}}}'

v5.25.5 (v5.25.4-110-g95c0a76)

perl​: op.c​:5966​: OP *Perl_newSVOP(I32, I32, SV *)​: Assertion `sv' failed.

From zefram@fysh.org

This bug is very sensitive to the details of the syntax errors. In the
execution that leads up to the assertion failure, the second sub{} is
treated as being nested inside the first, and this is crucial to the
failure mode. (Almost any edit to the test code leads to them being
treated as sequential, avoiding the assertion failure.)

The first newANONATTRSUB() call, for the second sub{}, succeeds.
The second newANONATTRSUB() call, for the first sub{}, then fails,
because it finds that PL_compcv is null. PL_compcv becomes null as a
side effect of the first newANONATTRSUB() call, and in normal operation
the value that PL_compcv had in the surrounding scope would immediately be
restored, with the end of compilation of the nested sub{}. Evidently the
restoration isn't happening. This is obviously for reasons tied up with
the parser's error recovery, a pretty opaque part of the code.

-zefram

The RT System itself - Status changed from 'new' to 'open'

From zefram@fysh.org

Looking a bit further, it turns out the problem is that the save stack
gets out of synch with the parser stack. Coincident with the first
syntax error, part of the save stack gets popped, to a point further
back than it was at the start of the first sub{}, while the parser state
with the pending action to handle the end of that construct remains
in place. The second sub{} reuses the same save stack positions that
the first used. As a result, the save stack popping in newATTRSUB_x()
doesn't manage to restore to a sane state, and in particular doesn't
manage to restore PL_compcv.

-zefram

From zefram@fysh.org

The thing that pops the save stack seems to be the end of the qq{}
construct, as handled in toke.c. The yacc parser doesn't realise that
any construct is ending there, because it's busy discarding tokens (up
to the semicolon) to recover from the syntax error. So the lexer and
parser get out of synch.

This problem is partly due to the way the code generated by a qq gets
embedded in the parse. The lexer generates tokens corresponding to
some builtin function calls (to join() and a notional stringification
function), and passes through the tokens of embedded ${} and @​{}
interpolations within that sequence. This works OK on correct code, but
it's fragile, and the present ticket is an illustration of the problems
that can arise. Fundamentally, delimiting the nested construct using
ordinary tokens doesn't work when it can contain arbitrary ordinary
tokens which may come in an ill-formed sequence.

In principle, a better way to handle the nested parses of qq constructs
would be to make a recursive call to the parser for the interpolations,
giving it a completely separate input stream. It would be impossible
to screw up the nesting in this case. Presumably this wasn't originally
done because the yacc structure does not lend itself to parsing different
top-level symbols. But that is a problem that I solved a few years ago,
to permit procedural calls into the parser from parse-time plugins.
Maybe it would be a good idea to reimplement qq constructs that way.
It's a bit too risky for this phase of the development cycle, though.

Incidentally, in debugging this I tried adding an assertion to
LEAVE_SCOPE(), asserting that the current stack height is at least at the
level the caller is requesting to pop to. It turns out that this isn't
a viable assertion; many exceptions have to be made. The parser's stack
clearing is heedless of the order in which it tries to leave scopes,
but there are dodgy scope-leaving calls from other places too. I was
surprised to discover this.

-zefram

From @tonycoz

On Thu, 15 Sep 2016 15​:22​:08 -0700, brian.carpenter@​gmail.com wrote​:

On Fri Sep 25 19​:38​:48 2015, brian.carpenter@​gmail.com wrote​:

perl -e 'qq{@​{sub{]]}}};qq{@​{sub{]]}}}}}}'

v5.25.5 (v5.25.4-110-g95c0a76)

perl​: op.c​:5966​: OP *Perl_newSVOP(I32, I32, SV *)​: Assertion `sv' failed.

This looks like it was fixed by bb4e4c3.

$ git describe
v5.27.8-155-gbb4e4c3869
$ ./perl ../126191.pl
syntax error at ../126191.pl line 1, near "{]"
Execution of ../126191.pl aborted due to compilation errors.

Immediately before​:

$ git describe
v5.27.8-154-g4bfb5532d3
$ ./perl ../126191.pl
perl​: op.c​:7236​: Perl_newSVOP​: Assertion `sv' failed.
Aborted

Tony

From @tonycoz

On Sun, 20 Jan 2019 20​:47​:33 -0800, tonyc wrote​:

On Thu, 15 Sep 2016 15​:22​:08 -0700, brian.carpenter@​gmail.com wrote​:

On Fri Sep 25 19​:38​:48 2015, brian.carpenter@​gmail.com wrote​:

perl -e 'qq{@​{sub{]]}}};qq{@​{sub{]]}}}}}}'

v5.25.5 (v5.25.4-110-g95c0a76)

perl​: op.c​:5966​: OP *Perl_newSVOP(I32, I32, SV *)​: Assertion `sv' failed.

This looks like it was fixed by bb4e4c3.

$ git describe
v5.27.8-155-gbb4e4c3869
$ ./perl ../126191.pl
syntax error at ../126191.pl line 1, near "{]"
Execution of ../126191.pl aborted due to compilation errors.

Immediately before​:

$ git describe
v5.27.8-154-g4bfb5532d3
$ ./perl ../126191.pl
perl​: op.c​:7236​: Perl_newSVOP​: Assertion `sv' failed.
Aborted

So closing.

Tony

@tonycoz - Status changed from 'open' to 'pending release'

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

@khwilliamson - Status changed from 'pending release' to 'resolved'