From @jkeenan

This breakage was mentioned in
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=134252, but it appears that
the breakage occurred in a different, earlier commit than the one cited
in that ticket for Data​::Alias.

The problem is that this distro's t/11loop.t now hangs indefinitely and
has to be Ctrl-C-ed. All other files in the distro's test suite PASS.

I explored a subset of the range denoted by this command​:

#####
$ git log --format=fuller --reverse
dad02c3..9d15d64
#####

Manually bisecting -- because Porting/bisect.pl doesn't, AFAIK, handle
hanging tests well -- I identified this commit as the first at which the
hang occurs​:

#####
commit 2a56a87
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Fri Apr 5 15​:38​:24 2019 +0100
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Mon Jun 24 11​:40​:06 2019 +0100

scalarvoid()​: remove anti-recursion deferring

Perl_scalarvoid() used to recursively work its way down an
optree marking ops as being in the appropriate context.

Around 5.22.0 the code was changed to avoid recursion, and
instead to malloc a buffer (if necessary) to maintain a
list of deferred child ops that need visiting. This stopped
perl crashing if the optree (and thus the recursion) was
too deep, but it introduced a potential leak. For example

use warnings FATAL => qw(void); $a = "abc"; length $a ;

The fatal warning causes scalarvoid() to leak the deferred
buffer.

This commit removes the deferred mechanism, and instead
makes use of the newish OP_PARENT mechanism to iterate over
the optree, following each kid, then back up via the parent
pointer to the next sibling etc.
#####

Thank you very much.
Jim Keenan

#####
Summary of my perl5 (revision 5 version 31 subversion 2) configuration​:
  Commit id​: 2a56a87
  Platform​:
  osname=freebsd
  osvers=11.2-stable
  archname=amd64-freebsd-thread-multi
  uname='freebsd perlmonger.nycbug.org 11.2-stable freebsd
11.2-stable #0 r339445​: sat oct 20 00​:08​:11 utc 2018
root@​perlmonger.nycbug.org​:usrobjusrsrcsysgeneric amd64 '
  config_args='-des -Dusedevel -Uversiononly
-Dprefix=/home/jkeenan/testing/2a56a87fe357165c2bf7fc0d0f54565fef60bb9a
-Dman1dir=none -Dman3dir=none -Duseithreads -Doptimize=-O2 -pipe
-fstack-protector -fno-strict-aliasing'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include
-D_FORTIFY_SOURCE=2'
  optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
  cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.2.1 Compatible FreeBSD Clang 6.0.1
(tags/RELEASE_601/final 335540)'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags ='-pthread -Wl,-E -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/lib /usr/local/lib /usr/lib/clang/6.0.1/lib /usr/lib
  libs=-lpthread -lgdbm -ldl -lm -lcrypt -lutil
  perllibs=-lpthread -ldl -lm -lcrypt -lutil
  libc=
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags=' '
  cccdlflags='-DPIC -fPIC'
  lddlflags='-shared -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  MULTIPLICITY
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_IMPLICIT_CONTEXT
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_ITHREADS
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  USE_REENTRANT_API
  Built under freebsd
  Compiled at Jul 2 2019 17​:37​:41
  %ENV​:
  PERL2DIR="/home/jkeenan/gitwork/perl2"
  PERL_WORKDIR="/home/jkeenan/gitwork/perl"
  @​INC​:
  lib

/home/jkeenan/testing/2a56a87fe357165c2bf7fc0d0f54565fef60bb9a/lib/perl5/site_perl/5.31.2/amd64-freebsd-thread-multi

/home/jkeenan/testing/2a56a87fe357165c2bf7fc0d0f54565fef60bb9a/lib/perl5/site_perl/5.31.2

/home/jkeenan/testing/2a56a87fe357165c2bf7fc0d0f54565fef60bb9a/lib/perl5/5.31.2/amd64-freebsd-thread-multi

/home/jkeenan/testing/2a56a87fe357165c2bf7fc0d0f54565fef60bb9a/lib/perl5/5.31.2
#####

From @leonerd

On Tue, 02 Jul 2019 11​:01​:42 -0700
"James E Keenan \(via RT\)" <perlbug-followup@​perl.org> wrote​:

The problem is that this distro's t/11loop.t now hangs indefinitely
and has to be Ctrl-C-ed. All other files in the distro's test suite
PASS.

...

This commit removes the deferred mechanism, and instead
makes use of the newish OP_PARENT mechanism to iterate over
the optree, following each kid, then back up via the parent
pointer to the next sibling etc.

Ah. I wonder then, whether this might be something that
Syntax​::Keyword​::Try has to fix. I do some rather awkward optree
fiddling in there, and I don't recall needing to take care of OP_PARENT
so far.

  https://metacpan.org/source/PEVANS/Syntax-Keyword-Try-0.10/lib/Syntax/Keyword/Try.xs#L257

  https://metacpan.org/source/PEVANS/Syntax-Keyword-Try-0.10/lib/Syntax/Keyword/Try.xs#L283

--
Paul "LeoNerd" Evans

leonerd@​leonerd.org.uk | https://metacpan.org/author/PEVANS
http​://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

The RT System itself - Status changed from 'new' to 'open'

From @jkeenan

On Wed, 03 Jul 2019 11​:16​:03 GMT, leonerd@​leonerd.org.uk wrote​:

On Tue, 02 Jul 2019 11​:01​:42 -0700
"James E Keenan \(via RT\)" <perlbug-followup@​perl.org> wrote​:

The problem is that this distro's t/11loop.t now hangs indefinitely
and has to be Ctrl-C-ed. All other files in the distro's test suite
PASS.

...

This commit removes the deferred mechanism, and instead
makes use of the newish OP_PARENT mechanism to iterate over
the optree, following each kid, then back up via the parent
pointer to the next sibling etc.

Ah. I wonder then, whether this might be something that
Syntax​::Keyword​::Try has to fix. I do some rather awkward optree
fiddling in there, and I don't recall needing to take care of
OP_PARENT
so far.

That's quite possible. If a distro touches op codes in its XS, then it's sensitive to changes in blead. (Unfortunately, this type of perlguts code is beyone me, so I myself don't have any useful suggestions.)

https://metacpan.org/source/PEVANS/Syntax-Keyword-Try-
0.10/lib/Syntax/Keyword/Try.xs#L257

https://metacpan.org/source/PEVANS/Syntax-Keyword-Try-
0.10/lib/Syntax/Keyword/Try.xs#L283

--
James E Keenan (jkeenan@​cpan.org)

From @leonerd

On Wed, 03 Jul 2019 04​:37​:25 -0700
"James E Keenan via RT" <perlbug-followup@​perl.org> wrote​:

That's quite possible. If a distro touches op codes in its XS, then
it's sensitive to changes in blead. (Unfortunately, this type of
perlguts code is beyone me, so I myself don't have any useful
suggestions.)

Imagining that this may be the case I've opened an rt.cpan.org ticket
on my Syntax​::Keyword​::Try.

  https://rt.cpan.org/Ticket/Display.html?id=129975

I'll investigate there. Hopefully it's a fix on my side and if so p5p
doesn't have to do anything.

I'll let you know what I find out.

--
Paul "LeoNerd" Evans

leonerd@​leonerd.org.uk | https://metacpan.org/author/PEVANS
http​://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

From @leonerd

While looking it over, I believe the relevant field is

  op->op_sibparent

which is defined at​:

  https://perl5.git.perl.org/perl.git/blob/82651abe60a8c1ca1f9eb6aae0202ecfd34bfecf:/op.h#l46

but doesn't have any documentation at

  https://perl5.git.perl.org/perl.git/blob/82651abe60a8c1ca1f9eb6aae0202ecfd34bfecf:/op.h#l12

While I'm looking at this, does someone who knows what that field is
about want to write some docs for it?

`git blame` on the field's addition has

  0f9a623 Tony Cook 2019-01-25 10​:32​:42 +1100

--
Paul "LeoNerd" Evans

leonerd@​leonerd.org.uk | https://metacpan.org/author/PEVANS
http​://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

From @ilmari

"Paul \"LeoNerd\" Evans" <leonerd@​leonerd.org.uk> writes​:

While looking it over, I believe the relevant field is

op->op_sibparent

which is defined at​:

https://perl5.git.perl.org/perl.git/blob/82651abe60a8c1ca1f9eb6aae0202ecfd34bfecf:/op.h#l46

but doesn't have any documentation at

https://perl5.git.perl.org/perl.git/blob/82651abe60a8c1ca1f9eb6aae0202ecfd34bfecf:/op.h#l12

While I'm looking at this, does someone who knows what that field is
about want to write some docs for it?

`git blame` on the field's addition has

0f9a623 Tony Cook 2019-01-25 10​:32​:42 +1100

That's not when the field was added, that's when PERL_OP_PARENT was made
mandatory.

The macros and functions to manipulate the optree are documented in
perlapi​: https://perldoc.pl/perlapi#Optree-Manipulation-Functions

Specifically the Op*SIB* macros and the op_*_(elem|list) and
op_sibling_splice functions.

See also https://perldoc.pl/perl5220delta#Internal-Changes for an
overview of the new parent linkage.

- ilmari
--
"The surreality of the universe tends towards a maximum" -- Skud's Law
"Never formulate a law or axiom that you're not prepared to live with
the consequences of." -- Skud's Meta-Law

From @leonerd

On Wed, 3 Jul 2019 12​:49​:50 +0100
"Paul \"LeoNerd\" Evans" <leonerd@​leonerd.org.uk> wrote​:

Imagining that this may be the case I've opened an rt.cpan.org ticket
on my Syntax​::Keyword​::Try.

https://rt.cpan.org/Ticket/Display.html?id=129975

I'll investigate there. Hopefully it's a fix on my side and if so p5p
doesn't have to do anything.

I'll let you know what I find out.

In summary​: there were actually two separate problems here, both of
them caused by S-K-T.xs making optree structures that weren't quite
correct, but good enough not to upset previous perls; the code
introduced in later bleadperls brought both of them to light.

These have now both been fixed and a new version uploaded to CPAN​:

  https://metacpan.org/release/PEVANS/Syntax-Keyword-Try-0.11

Passes for my local bleadperl and a test 5.31.3 on my server, so
hopefully this resolves the issue now.

--
Paul "LeoNerd" Evans

leonerd@​leonerd.org.uk | https://metacpan.org/author/PEVANS
http​://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

From @jkeenan

On Sat, 07 Sep 2019 00​:54​:51 GMT, leonerd@​leonerd.org.uk wrote​:

On Wed, 3 Jul 2019 12​:49​:50 +0100
"Paul \"LeoNerd\" Evans" <leonerd@​leonerd.org.uk> wrote​:

Imagining that this may be the case I've opened an rt.cpan.org ticket
on my Syntax​::Keyword​::Try.

https://rt.cpan.org/Ticket/Display.html?id=129975

I'll investigate there. Hopefully it's a fix on my side and if so p5p
doesn't have to do anything.

I'll let you know what I find out.

In summary​: there were actually two separate problems here, both of
them caused by S-K-T.xs making optree structures that weren't quite
correct, but good enough not to upset previous perls; the code
introduced in later bleadperls brought both of them to light.

These have now both been fixed and a new version uploaded to CPAN​:

https://metacpan.org/release/PEVANS/Syntax-Keyword-Try-0.11

Passes for my local bleadperl and a test 5.31.3 on my server, so
hopefully this resolves the issue now.

Works for me on Linux, FreeBSD-11 and FreeBSD-12.

Thank you very much for your investigation.
--
James E Keenan (jkeenan@​cpan.org)

@jkeenan - Status changed from 'open' to 'resolved'