Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build-time failures with PERL_GLOBAL_STRUCT and PERL_GLOBAL_STRUCT_PRIVATE #16794

Closed
p5pRT opened this issue Dec 23, 2018 · 28 comments
Closed

Build-time failures with PERL_GLOBAL_STRUCT and PERL_GLOBAL_STRUCT_PRIVATE #16794

p5pRT opened this issue Dec 23, 2018 · 28 comments

Comments

@p5pRT
Copy link

p5pRT commented Dec 23, 2018

Migrated from rt.perl.org#133737 (status was 'resolved')

Searchable as RT133737$

@p5pRT
Copy link
Author

p5pRT commented Dec 23, 2018

From @jkeenan

According to pod/perl5260delta.pod​:

#####
* Clang builds with "-DPERL_GLOBAL_STRUCT" or
"-DPERL_GLOBAL_STRUCT_PRIVATE" have been fixed (by disabling Thread
Safety Analysis for these configurations).
#####

However, recent smoke test reports from George Greer indicate that these
configuration options are once again preventing 'make' from completing
successfully. See, e.g., http​://perl5.test-smoke.org/report/76521.

According to Porting/bisect.pl, the first bad commit was​:

#####
commit f741678
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Sep 7 15​:21​:56 2017 -0600

Add API function Perl_langinfo()

This is designed to generally replace nl_langinfo() in XS code. It is
thread-safer, hides the quirks of perl's LC_NUMERIC handling, and can be
used on systems lacking nl_langinfo.
#####

However, at that commit the build failure occurred in locale.c. In
blead, with the same configuration options, the build failure is
occurring much earlier -- in op.c. I've built perl at several tags
corresponding to monthly development releases since then and it appears
that the error output has changed at least twice -- once for the worse,
once for the better -- since then.

I am attaching gzipped files holding the output of the 'make test_prep'
part of the following command at various points​:

#####
$ sh ./Configure -des -Dusedevel -Dcc=clang
-Accflags=-DPERL_GLOBAL_STRUCT; make test_prep
#####

With this configuration, perl-5.27.3 (2017-Aug-21) built successfully.
Given that the first bad commit was on 2018-Sep-07, build-time failures
were observed in perl-5.27.4 (2017-Sep-20). Slightly different errors
were observed in perl-5.27.5 (2017-Oct-20), but 'make' did not die until
locale.c. Starting with perl-5.27.6 (2017-Nov-20), however, the
build-time failures started to occur in op.c. These failures have
persisted since then (though the op.c line numbers at which errors have
been reported have, not surprisingly, jumped around a bit). So I'm
attaching files for 5.27.4, 5.27.5, 5.27.6 and 5.29.6. I'm also
attaching a grep for 'error​:' over my clang builds on linux.

Additional Notes​:

* When I compile with gcc rather than clang, I get different error
output but failures in the same files for the same monthly dev releases.
  Output available upon request.

* I also performed the initial bisection on FreeBSD-11.2. The bisection
pointed to the same initial "bad" commit.

* I did only one test run with PERL_GLOBAL_STRUCT_PRIVATE, so that
option might need further investigation.

* Since this was a regression between 5.26 and 5.28, we should explore
backporting any fix to 5.28 maintenance releases.

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Dec 23, 2018

@p5pRT
Copy link
Author

p5pRT commented Dec 23, 2018

@p5pRT
Copy link
Author

p5pRT commented Dec 23, 2018

@p5pRT
Copy link
Author

p5pRT commented Dec 23, 2018

@p5pRT
Copy link
Author

p5pRT commented Dec 23, 2018

@p5pRT
Copy link
Author

p5pRT commented Dec 27, 2018

From @jkeenan

On Sun, 23 Dec 2018 04​:10​:53 GMT, jkeenan@​pobox.com wrote​:

According to pod/perl5260delta.pod​:

#####
* Clang builds with "-DPERL_GLOBAL_STRUCT" or
"-DPERL_GLOBAL_STRUCT_PRIVATE" have been fixed (by disabling Thread
Safety Analysis for these configurations).
#####

However, recent smoke test reports from George Greer indicate that these
configuration options are once again preventing 'make' from completing
successfully. See, e.g., http​://perl5.test-smoke.org/report/76521.

According to Porting/bisect.pl, the first bad commit was​:

#####
commit f741678
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Sep 7 15​:21​:56 2017 -0600

Add API function Perl_langinfo()

This is designed to generally replace nl_langinfo() in XS code. It is
thread-safer, hides the quirks of perl's LC_NUMERIC handling, and can be
used on systems lacking nl_langinfo.
#####

However, at that commit the build failure occurred in locale.c. In
blead, with the same configuration options, the build failure is
occurring much earlier -- in op.c. I've built perl at several tags
corresponding to monthly development releases since then and it appears
that the error output has changed at least twice -- once for the worse,
once for the better -- since then.

I am attaching gzipped files holding the output of the 'make test_prep'
part of the following command at various points​:

#####
$ sh ./Configure -des -Dusedevel -Dcc=clang
-Accflags=-DPERL_GLOBAL_STRUCT; make test_prep
#####

With this configuration, perl-5.27.3 (2017-Aug-21) built successfully.
Given that the first bad commit was on 2018-Sep-07, build-time failures
were observed in perl-5.27.4 (2017-Sep-20). Slightly different errors
were observed in perl-5.27.5 (2017-Oct-20), but 'make' did not die until
locale.c. Starting with perl-5.27.6 (2017-Nov-20), however, the
build-time failures started to occur in op.c. These failures have
persisted since then (though the op.c line numbers at which errors have
been reported have, not surprisingly, jumped around a bit). So I'm
attaching files for 5.27.4, 5.27.5, 5.27.6 and 5.29.6. I'm also
attaching a grep for 'error​:' over my clang builds on linux.

Additional Notes​:

* When I compile with gcc rather than clang, I get different error
output but failures in the same files for the same monthly dev releases.
Output available upon request.

* I also performed the initial bisection on FreeBSD-11.2. The bisection
pointed to the same initial "bad" commit.

* I did only one test run with PERL_GLOBAL_STRUCT_PRIVATE, so that
option might need further investigation.

* Since this was a regression between 5.26 and 5.28, we should explore
backporting any fix to 5.28 maintenance releases.

Thank you very much.
Jim Keenan

Multisectioning suggests that the first commit with build-time errors was e839e6e and that additional errors appeared at f99042c. See attachment, transitions.pl.

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Dec 27, 2018

From @jkeenan

transitions.pl

@p5pRT
Copy link
Author

p5pRT commented Dec 27, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 27, 2018

From @khwilliamson

On 12/26/18 8​:30 PM, James E Keenan via RT wrote​:

On Sun, 23 Dec 2018 04​:10​:53 GMT, jkeenan@​pobox.com wrote​:

According to pod/perl5260delta.pod​:

#####
* Clang builds with "-DPERL_GLOBAL_STRUCT" or
"-DPERL_GLOBAL_STRUCT_PRIVATE" have been fixed (by disabling Thread
Safety Analysis for these configurations).
#####

However, recent smoke test reports from George Greer indicate that these
configuration options are once again preventing 'make' from completing
successfully. See, e.g., http​://perl5.test-smoke.org/report/76521.

According to Porting/bisect.pl, the first bad commit was​:

#####
commit f741678
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Sep 7 15​:21​:56 2017 -0600

Add API function Perl_langinfo()

This is designed to generally replace nl_langinfo() in XS code. It is
thread-safer, hides the quirks of perl's LC_NUMERIC handling, and can be
used on systems lacking nl_langinfo.
#####

However, at that commit the build failure occurred in locale.c. In
blead, with the same configuration options, the build failure is
occurring much earlier -- in op.c. I've built perl at several tags
corresponding to monthly development releases since then and it appears
that the error output has changed at least twice -- once for the worse,
once for the better -- since then.

I am attaching gzipped files holding the output of the 'make test_prep'
part of the following command at various points​:

#####
$ sh ./Configure -des -Dusedevel -Dcc=clang
-Accflags=-DPERL_GLOBAL_STRUCT; make test_prep
#####

With this configuration, perl-5.27.3 (2017-Aug-21) built successfully.
Given that the first bad commit was on 2018-Sep-07, build-time failures
were observed in perl-5.27.4 (2017-Sep-20). Slightly different errors
were observed in perl-5.27.5 (2017-Oct-20), but 'make' did not die until
locale.c. Starting with perl-5.27.6 (2017-Nov-20), however, the
build-time failures started to occur in op.c. These failures have
persisted since then (though the op.c line numbers at which errors have
been reported have, not surprisingly, jumped around a bit). So I'm
attaching files for 5.27.4, 5.27.5, 5.27.6 and 5.29.6. I'm also
attaching a grep for 'error​:' over my clang builds on linux.

Additional Notes​:

* When I compile with gcc rather than clang, I get different error
output but failures in the same files for the same monthly dev releases.
Output available upon request.

* I also performed the initial bisection on FreeBSD-11.2. The bisection
pointed to the same initial "bad" commit.

* I did only one test run with PERL_GLOBAL_STRUCT_PRIVATE, so that
option might need further investigation.

* Since this was a regression between 5.26 and 5.28, we should explore
backporting any fix to 5.28 maintenance releases.

Thank you very much.
Jim Keenan

Multisectioning suggests that the first commit with build-time errors was e839e6e and that additional errors appeared at f99042c. See attachment, transitions.pl.

Thank you very much.

commit e839e6e
  Author​: David Mitchell <davem@​iabyn.com>
  Date​: Tue Aug 8 18​:42​:14 2017 +0100

  Add OP_MULTICONCAT op

  commit f99042c
  Author​: Zefram <zefram@​fysh.org>
  Date​: Sat Nov 4 21​:43​:59 2017 +0000

  initialisation of simple aggregate state variables

  We now recognise the basic cases of aggregate state variables, to
  permit initialisations of the form "state @​a =" or "state @​a
:shared =".
  Initialisation of state variable lists remains forbidden, because the
  op flags don't let us distinguish "state(...)" from "(state(...))" in
  most cases.

Adding a bunch of dVARS fixes the compilation problems, as
https://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-me/khw-global_struct

indicates.

This seems to fix the problem except for PERL_GLOBAL_STRUCT_PRIVATE
gives me two test failures on Linux. Someone who is more familiar with
this should be able to figure it out quicker than me.

They are

# Failed test 1 - loading XS​::APItest in threads works at
../../t/test.pl line 1062
# got "Segmentation fault (core dumped)"
# expected "all is well"
# PROG​:
# use strict;
# use warnings;
#
# use threads;
#
# require '../../t/test.pl';
# watchdog(5);
#
# for my $t (1 .. 3) {
# threads->create(sub {
# require XS​::APItest;
# })->join;
# }
#
# print "all is well\n";
# STATUS​: 35584
../ext/XS-APItest/t/keyword_plugin_threads.t .......................
Failed 1/1 subtests

And

===========================================================
# Failed test '"make" test exited normally'
# at lib/MakeMaker/Test/Setup/XS.pm line 462.
# got​: '512'
# expected​: '0'
# make[1]​: Entering directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
# cp lib/XS/Test.pm blib/lib/XS/Test.pm
# "/home/khw/perl/global_struct/perl" -w "-I../../../../lib"
"../../../../lib/ExtUtils/xsubpp" -typemap
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic/../../../../lib/ExtUtils/typemap'
-typemap
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic/type
map' Test.xs > Test.xsc
# mv Test.xsc Test.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated
-DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing
-pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall
-Werror=declaration-after-statement -Werror=pointer-arith -Wextra
-Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\"
-DXS_VERSION=\"1.01\" -fPIC "-I../../../.." Test.c
# rm -f "blib/arch/auto/XS/Test/Test.a"
# /usr/bin/ar cr "blib/arch/auto/XS/Test/Test.a" Test.o
# /usr/bin/ranlib "blib/arch/auto/XS/Test/Test.a"
# chmod 755 blib/arch/auto/XS/Test/Test.a
# make[1]​: Leaving directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
#
# make[1]​: Entering directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
# Writing "Makefile.aperl" for this perl
# Generating a Unix-style Makefile.aperl
# Writing Makefile.aperl for XS​::Test
# Writing MYMETA.yml and MYMETA.json
# make -f Makefile.aperl perl
# make[2]​: Entering directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
# Writing perlmain.c
# mv perlmain.ct perlmain.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated
-DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing
-pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall
-Werror=declaration-after-statement -Werror=pointer-arith -Wextra
-Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\"
-DXS_VERSION=\"1.01\" -fPIC "-I../../../.." perlmain.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated
-DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing
-pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall
-Werror=declaration-after-statement -Werror=pointer-arith -Wextra
-Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\"
-DXS_VERSION=\"1.01\" -fPIC "-I../../../.." Test.c
# rm -f "blib/arch/auto/XS/Test/Test.a"
# /usr/bin/ar cr "blib/arch/auto/XS/Test/Test.a" Test.o
# /usr/bin/ranlib "blib/arch/auto/XS/Test/Test.a"
# chmod 755 blib/arch/auto/XS/Test/Test.a
# cat blib/arch/auto/XS/Test/extralibs.ld >>
blib/arch/auto/XS/Test/extralibs.all
# g++ -fstack-protector-strong -L/usr/local/lib -Wl,-E Test.o -O2 -ggdb3
-O0 ./perlmain.o -o perl "blib/arch/auto/XS/Test/Test.a"
"../../../../libperl.a" `cat blib/arch/auto/XS/Test/extralibs.all`
-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
# To install the new 'perl' binary, call
# make -f Makefile.aperl inst_perl MAP_TARGET=perl
# make -f Makefile.aperl map_clean
# make[2]​: Leaving directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
# PERL_DL_NONLAZY=1
"/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic/perl"
"-Iblib/arch" "-Iblib/lib" "-I../../../../lib" "-I../../../../lib"
"-MExtUtils​::Command​::MM" "-MTest​::Harness" "-e" "undef
*Test​::Harness​::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
# Segmentation fault (core dumped)
# Makefile​:967​: recipe for target 'test_static' failed
# make[1]​: *** [test_static] Error 139
# make[1]​: Leaving directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
# Failed test '"make" test_static exited normally'
# at lib/MakeMaker/Test/Setup/XS.pm line 462.
# got​: '512'
# expected​: '0'
# make[1]​: Entering directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'
# "/home/khw/perl/global_struct/perl" -w "-I../../../../lib"
"../../../../lib/ExtUtils/xsubpp" -typemap
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic/../../../../lib/ExtUtils/typemap'
-typemap
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic/type
map' Test.xs > Test.xsc
# mv Test.xsc Test.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated
-DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing
-pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall
-Werror=declaration-after-statement -Werror=pointer-arith -Wextra
-Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\"
-DXS_VERSION=\"1.01\" -fPIC "-I../../../.." Test.c
# rm -f "blib/arch/auto/XS/Test/Test.a"
# /usr/bin/ar cr "blib/arch/auto/XS/Test/Test.a" Test.o
# /usr/bin/ranlib "blib/arch/auto/XS/Test/Test.a"
# chmod 755 blib/arch/auto/XS/Test/Test.a
# make[1]​: Leaving directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'
#
# make[1]​: Entering directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'
# cp lib/XS/Test.pm blib/lib/XS/Test.pm
# Running Mkbootstrap for Test ()
# chmod 644 "Test.bs"
# "/home/khw/perl/global_struct/perl" -w "-I../../../../lib"
-MExtUtils​::Command​::MM -e 'cp_nonempty' -- Test.bs
blib/arch/auto/XS/Test/Test.bs 644
# rm -f blib/arch/auto/XS/Test/Test.so
# g++ -shared -O2 -ggdb3 -O0 -L/usr/local/lib -fstack-protector-strong
Test.o -o blib/arch/auto/XS/Test/Test.so \
# \
#
# chmod 755 blib/arch/auto/XS/Test/Test.so
# Writing "Makefile.aperl" for this perl
# Generating a Unix-style Makefile.aperl
# Writing Makefile.aperl for XS​::Test
# Writing MYMETA.yml and MYMETA.json
# make -f Makefile.aperl perl
# make[2]​: Entering directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'
# Writing perlmain.c
# mv perlmain.ct perlmain.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated
-DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing
-pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall
-Werror=declaration-after-statement -Werror=pointer-arith -Wextra
-Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\"
-DXS_VERSION=\"1.01\" -fPIC "-I../../../.." perlmain.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated
-DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing
-pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall
-Werror=declaration-after-statement -Werror=pointer-arith -Wextra
-Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\"
-DXS_VERSION=\"1.01\" -fPIC "-I../../../.." Test.c
# rm -f "blib/arch/auto/XS/Test/Test.a"
# /usr/bin/ar cr "blib/arch/auto/XS/Test/Test.a" Test.o
# /usr/bin/ranlib "blib/arch/auto/XS/Test/Test.a"
# chmod 755 blib/arch/auto/XS/Test/Test.a
# cat blib/arch/auto/XS/Test/extralibs.ld >>
blib/arch/auto/XS/Test/extralibs.all
# g++ -fstack-protector-strong -L/usr/local/lib -Wl,-E Test.o -O2 -ggdb3
-O0 ./perlmain.o -o perl "blib/arch/auto/XS/Test/Test.a"
"../../../../libperl.a" `cat blib/arch/auto/XS/Test/extralibs.all`
-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
# To install the new 'perl' binary, call
# make -f Makefile.aperl inst_perl MAP_TARGET=perl
# make -f Makefile.aperl map_clean
# make[2]​: Leaving directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'
# PERL_DL_NONLAZY=1
"/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic/perl"
"-Iblib/arch" "-Iblib/lib" "-I../../../../lib" "-I../../../../lib"
"-MExtUtils​::Command​::MM" "-MTest​::Harness" "-e" "undef
*Test​::Harness​::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
# Segmentation fault (core dumped)
# Makefile​:966​: recipe for target 'test_static' failed
# make[1]​: *** [test_static] Error 139
# make[1]​: Leaving directory
'/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'

@p5pRT
Copy link
Author

p5pRT commented Dec 27, 2018

From @khwilliamson

On Dec 26, 2018, at 11​:26 PM, Karl Williamson <public@​khwilliamson.com> wrote​:

On 12/26/18 8​:30 PM, James E Keenan via RT wrote​:

On Sun, 23 Dec 2018 04​:10​:53 GMT, jkeenan@​pobox.com wrote​:
According to pod/perl5260delta.pod​:

#####
* Clang builds with "-DPERL_GLOBAL_STRUCT" or
"-DPERL_GLOBAL_STRUCT_PRIVATE" have been fixed (by disabling Thread
Safety Analysis for these configurations).
#####

However, recent smoke test reports from George Greer indicate that these
configuration options are once again preventing 'make' from completing
successfully. See, e.g., http​://perl5.test-smoke.org/report/76521.

According to Porting/bisect.pl, the first bad commit was​:

#####
commit f741678
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Sep 7 15​:21​:56 2017 -0600

Add API function Perl_langinfo()

This is designed to generally replace nl_langinfo() in XS code. It is
thread-safer, hides the quirks of perl's LC_NUMERIC handling, and can be
used on systems lacking nl_langinfo.
#####

However, at that commit the build failure occurred in locale.c. In
blead, with the same configuration options, the build failure is
occurring much earlier -- in op.c. I've built perl at several tags
corresponding to monthly development releases since then and it appears
that the error output has changed at least twice -- once for the worse,
once for the better -- since then.

I am attaching gzipped files holding the output of the 'make test_prep'
part of the following command at various points​:

#####
$ sh ./Configure -des -Dusedevel -Dcc=clang
-Accflags=-DPERL_GLOBAL_STRUCT; make test_prep
#####

With this configuration, perl-5.27.3 (2017-Aug-21) built successfully.
Given that the first bad commit was on 2018-Sep-07, build-time failures
were observed in perl-5.27.4 (2017-Sep-20). Slightly different errors
were observed in perl-5.27.5 (2017-Oct-20), but 'make' did not die until
locale.c. Starting with perl-5.27.6 (2017-Nov-20), however, the
build-time failures started to occur in op.c. These failures have
persisted since then (though the op.c line numbers at which errors have
been reported have, not surprisingly, jumped around a bit). So I'm
attaching files for 5.27.4, 5.27.5, 5.27.6 and 5.29.6. I'm also
attaching a grep for 'error​:' over my clang builds on linux.

Additional Notes​:

* When I compile with gcc rather than clang, I get different error
output but failures in the same files for the same monthly dev releases.
Output available upon request.

* I also performed the initial bisection on FreeBSD-11.2. The bisection
pointed to the same initial "bad" commit.

* I did only one test run with PERL_GLOBAL_STRUCT_PRIVATE, so that
option might need further investigation.

* Since this was a regression between 5.26 and 5.28, we should explore
backporting any fix to 5.28 maintenance releases.

Thank you very much.
Jim Keenan
Multisectioning suggests that the first commit with build-time errors was e839e6e and that additional errors appeared at f99042c. See attachment, transitions.pl.
Thank you very much.

commit e839e6e
Author​: David Mitchell <davem@​iabyn.com>
Date​: Tue Aug 8 18​:42​:14 2017 +0100

Add OP\_MULTICONCAT op

commit f99042c
Author​: Zefram <zefram@​fysh.org>
Date​: Sat Nov 4 21​:43​:59 2017 +0000

initialisation of simple aggregate state variables

We now recognise the basic cases of aggregate state variables\, to
permit initialisations of the form "state @&#8203;a =" or "state @&#8203;a :shared ="\.
Initialisation of state variable lists remains forbidden\, because the
op flags don't let us distinguish "state\(\.\.\.\)" from "\(state\(\.\.\.\)\)" in
most cases\.

Adding a bunch of dVARS fixes the compilation problems, as
https://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-me/khw-global_struct

indicates.

But it seems to me that the docs indicate that a pTHX_ should do a dVAR automatically.

This seems to fix the problem except for PERL_GLOBAL_STRUCT_PRIVATE gives me two test failures on Linux. Someone who is more familiar with this should be able to figure it out quicker than me.

They are

# Failed test 1 - loading XS​::APItest in threads works at ../../t/test.pl line 1062
# got "Segmentation fault (core dumped)"
# expected "all is well"
# PROG​:
# use strict;
# use warnings;
#
# use threads;
#
# require '../../t/test.pl';
# watchdog(5);
#
# for my $t (1 .. 3) {
# threads->create(sub {
# require XS​::APItest;
# })->join;
# }
#
# print "all is well\n";
# STATUS​: 35584
../ext/XS-APItest/t/keyword_plugin_threads.t .......................
Failed 1/1 subtests

And

===========================================================
# Failed test '"make" test exited normally'
# at lib/MakeMaker/Test/Setup/XS.pm line 462.
# got​: '512'
# expected​: '0'
# make[1]​: Entering directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
# cp lib/XS/Test.pm blib/lib/XS/Test.pm
# "/home/khw/perl/global_struct/perl" -w "-I../../../../lib" "../../../../lib/ExtUtils/xsubpp" -typemap '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic/../../../../lib/ExtUtils/typemap' -typemap '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic/type map' Test.xs > Test.xsc
# mv Test.xsc Test.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated -DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall -Werror=declaration-after-statement -Werror=pointer-arith -Wextra -Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\" -DXS_VERSION=\"1.01\" -fPIC "-I../../../.." Test.c
# rm -f "blib/arch/auto/XS/Test/Test.a"
# /usr/bin/ar cr "blib/arch/auto/XS/Test/Test.a" Test.o
# /usr/bin/ranlib "blib/arch/auto/XS/Test/Test.a"
# chmod 755 blib/arch/auto/XS/Test/Test.a
# make[1]​: Leaving directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
#
# make[1]​: Entering directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
# Writing "Makefile.aperl" for this perl
# Generating a Unix-style Makefile.aperl
# Writing Makefile.aperl for XS​::Test
# Writing MYMETA.yml and MYMETA.json
# make -f Makefile.aperl perl
# make[2]​: Entering directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
# Writing perlmain.c
# mv perlmain.ct perlmain.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated -DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall -Werror=declaration-after-statement -Werror=pointer-arith -Wextra -Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\" -DXS_VERSION=\"1.01\" -fPIC "-I../../../.." perlmain.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated -DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall -Werror=declaration-after-statement -Werror=pointer-arith -Wextra -Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\" -DXS_VERSION=\"1.01\" -fPIC "-I../../../.." Test.c
# rm -f "blib/arch/auto/XS/Test/Test.a"
# /usr/bin/ar cr "blib/arch/auto/XS/Test/Test.a" Test.o
# /usr/bin/ranlib "blib/arch/auto/XS/Test/Test.a"
# chmod 755 blib/arch/auto/XS/Test/Test.a
# cat blib/arch/auto/XS/Test/extralibs.ld >> blib/arch/auto/XS/Test/extralibs.all
# g++ -fstack-protector-strong -L/usr/local/lib -Wl,-E Test.o -O2 -ggdb3 -O0 ./perlmain.o -o perl "blib/arch/auto/XS/Test/Test.a" "../../../../libperl.a" `cat blib/arch/auto/XS/Test/extralibs.all` -lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
# To install the new 'perl' binary, call
# make -f Makefile.aperl inst_perl MAP_TARGET=perl
# make -f Makefile.aperl map_clean
# make[2]​: Leaving directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
# PERL_DL_NONLAZY=1 "/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic/perl" "-Iblib/arch" "-Iblib/lib" "-I../../../../lib" "-I../../../../lib" "-MExtUtils​::Command​::MM" "-MTest​::Harness" "-e" "undef *Test​::Harness​::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
# Segmentation fault (core dumped)
# Makefile​:967​: recipe for target 'test_static' failed
# make[1]​: *** [test_static] Error 139
# make[1]​: Leaving directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Teststatic'
# Failed test '"make" test_static exited normally'
# at lib/MakeMaker/Test/Setup/XS.pm line 462.
# got​: '512'
# expected​: '0'
# make[1]​: Entering directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'
# "/home/khw/perl/global_struct/perl" -w "-I../../../../lib" "../../../../lib/ExtUtils/xsubpp" -typemap '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic/../../../../lib/ExtUtils/typemap' -typemap '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic/type map' Test.xs > Test.xsc
# mv Test.xsc Test.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated -DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall -Werror=declaration-after-statement -Werror=pointer-arith -Wextra -Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\" -DXS_VERSION=\"1.01\" -fPIC "-I../../../.." Test.c
# rm -f "blib/arch/auto/XS/Test/Test.a"
# /usr/bin/ar cr "blib/arch/auto/XS/Test/Test.a" Test.o
# /usr/bin/ranlib "blib/arch/auto/XS/Test/Test.a"
# chmod 755 blib/arch/auto/XS/Test/Test.a
# make[1]​: Leaving directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'
#
# make[1]​: Entering directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'
# cp lib/XS/Test.pm blib/lib/XS/Test.pm
# Running Mkbootstrap for Test ()
# chmod 644 "Test.bs"
# "/home/khw/perl/global_struct/perl" -w "-I../../../../lib" -MExtUtils​::Command​::MM -e 'cp_nonempty' -- Test.bs blib/arch/auto/XS/Test/Test.bs 644
# rm -f blib/arch/auto/XS/Test/Test.so
# g++ -shared -O2 -ggdb3 -O0 -L/usr/local/lib -fstack-protector-strong Test.o -o blib/arch/auto/XS/Test/Test.so \
# \
#
# chmod 755 blib/arch/auto/XS/Test/Test.so
# Writing "Makefile.aperl" for this perl
# Generating a Unix-style Makefile.aperl
# Writing Makefile.aperl for XS​::Test
# Writing MYMETA.yml and MYMETA.json
# make -f Makefile.aperl perl
# make[2]​: Entering directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'
# Writing perlmain.c
# mv perlmain.ct perlmain.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated -DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall -Werror=declaration-after-statement -Werror=pointer-arith -Wextra -Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\" -DXS_VERSION=\"1.01\" -fPIC "-I../../../.." perlmain.c
# g++ -c -D_REENTRANT -D_GNU_SOURCE -Wno-deprecated -DPERL_GLOBAL_STRUCT_PRIVATE -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -Wall -Werror=declaration-after-statement -Werror=pointer-arith -Wextra -Wwrite-strings -O2 -ggdb3 -O0 -DVERSION=\"1.01\" -DXS_VERSION=\"1.01\" -fPIC "-I../../../.." Test.c
# rm -f "blib/arch/auto/XS/Test/Test.a"
# /usr/bin/ar cr "blib/arch/auto/XS/Test/Test.a" Test.o
# /usr/bin/ranlib "blib/arch/auto/XS/Test/Test.a"
# chmod 755 blib/arch/auto/XS/Test/Test.a
# cat blib/arch/auto/XS/Test/extralibs.ld >> blib/arch/auto/XS/Test/extralibs.all
# g++ -fstack-protector-strong -L/usr/local/lib -Wl,-E Test.o -O2 -ggdb3 -O0 ./perlmain.o -o perl "blib/arch/auto/XS/Test/Test.a" "../../../../libperl.a" `cat blib/arch/auto/XS/Test/extralibs.all` -lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
# To install the new 'perl' binary, call
# make -f Makefile.aperl inst_perl MAP_TARGET=perl
# make -f Makefile.aperl map_clean
# make[2]​: Leaving directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'
# PERL_DL_NONLAZY=1 "/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic/perl" "-Iblib/arch" "-Iblib/lib" "-I../../../../lib" "-I../../../../lib" "-MExtUtils​::Command​::MM" "-MTest​::Harness" "-e" "undef *Test​::Harness​::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
# Segmentation fault (core dumped)
# Makefile​:966​: recipe for target 'test_static' failed
# make[1]​: *** [test_static] Error 139
# make[1]​: Leaving directory '/home/khw/perl/global_struct/cpan/ExtUtils-MakeMaker/t/XS-Testbasicstatic'

@p5pRT
Copy link
Author

p5pRT commented Dec 27, 2018

From @jkeenan

On Thu, 27 Dec 2018 06​:27​:35 GMT, public@​khwilliamson.com wrote​:

On 12/26/18 8​:30 PM, James E Keenan via RT wrote​:

On Sun, 23 Dec 2018 04​:10​:53 GMT, jkeenan@​pobox.com wrote​:

According to pod/perl5260delta.pod​:

#####
* Clang builds with "-DPERL_GLOBAL_STRUCT" or
"-DPERL_GLOBAL_STRUCT_PRIVATE" have been fixed (by disabling Thread
Safety Analysis for these configurations).
#####

However, recent smoke test reports from George Greer indicate that
these
configuration options are once again preventing 'make' from
completing
successfully. See, e.g., http​://perl5.test-smoke.org/report/76521.

According to Porting/bisect.pl, the first bad commit was​:

#####
commit f741678
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Sep 7 15​:21​:56 2017 -0600

Add API function Perl_langinfo()

This is designed to generally replace nl_langinfo() in XS code. It
is
thread-safer, hides the quirks of perl's LC_NUMERIC handling, and
can be
used on systems lacking nl_langinfo.
#####

However, at that commit the build failure occurred in locale.c. In
blead, with the same configuration options, the build failure is
occurring much earlier -- in op.c. I've built perl at several tags
corresponding to monthly development releases since then and it
appears
that the error output has changed at least twice -- once for the
worse,
once for the better -- since then.

I am attaching gzipped files holding the output of the 'make
test_prep'
part of the following command at various points​:

#####
$ sh ./Configure -des -Dusedevel -Dcc=clang
-Accflags=-DPERL_GLOBAL_STRUCT; make test_prep
#####

With this configuration, perl-5.27.3 (2017-Aug-21) built
successfully.
Given that the first bad commit was on 2018-Sep-07, build-time
failures
were observed in perl-5.27.4 (2017-Sep-20). Slightly different
errors
were observed in perl-5.27.5 (2017-Oct-20), but 'make' did not die
until
locale.c. Starting with perl-5.27.6 (2017-Nov-20), however, the
build-time failures started to occur in op.c. These failures have
persisted since then (though the op.c line numbers at which errors
have
been reported have, not surprisingly, jumped around a bit). So I'm
attaching files for 5.27.4, 5.27.5, 5.27.6 and 5.29.6. I'm also
attaching a grep for 'error​:' over my clang builds on linux.

Additional Notes​:

* When I compile with gcc rather than clang, I get different error
output but failures in the same files for the same monthly dev
releases.
Output available upon request.

* I also performed the initial bisection on FreeBSD-11.2. The
bisection
pointed to the same initial "bad" commit.

* I did only one test run with PERL_GLOBAL_STRUCT_PRIVATE, so that
option might need further investigation.

* Since this was a regression between 5.26 and 5.28, we should
explore
backporting any fix to 5.28 maintenance releases.

Thank you very much.
Jim Keenan

Multisectioning suggests that the first commit with build-time errors
was e839e6e and that additional errors appeared at f99042c. See
attachment, transitions.pl.

Thank you very much.

commit e839e6e
Author​: David Mitchell <davem@​iabyn.com>
Date​: Tue Aug 8 18​:42​:14 2017 +0100

Add OP_MULTICONCAT op

commit f99042c
Author​: Zefram <zefram@​fysh.org>
Date​: Sat Nov 4 21​:43​:59 2017 +0000

initialisation of simple aggregate state variables

We now recognise the basic cases of aggregate state variables, to
permit initialisations of the form "state @​a =" or "state @​a
:shared =".
Initialisation of state variable lists remains forbidden, because the
op flags don't let us distinguish "state(...)" from "(state(...))" in
most cases.

Adding a bunch of dVARS fixes the compilation problems, as
https://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-me/khw-
global_struct

indicates.

Unfortunately, the smoke-me/khw-global_struct branch breaks the build on FreeBSD and other platforms.

On FreeBSD-11.2, I compiled with my usual ./Configure options, i.e., without "-DPERL_GLOBAL_STRUCT". I got build-time failures; see attachment smoke-me.khw-global_struct.make.failure.txt.gz.

See also Carlos's smoke reports​:

http​://perl5.test-smoke.org/report/76807
http​://perl5.test-smoke.org/report/76819

Also see​:

http​://perl.develop-help.com/?b=smoke-me%2Fkhw-global_struct

[snip]

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Dec 27, 2018

@p5pRT
Copy link
Author

p5pRT commented Dec 27, 2018

From @khwilliamson

On 12/27/18 8​:46 AM, James E Keenan via RT wrote​:

On Thu, 27 Dec 2018 06​:27​:35 GMT, public@​khwilliamson.com wrote​:

On 12/26/18 8​:30 PM, James E Keenan via RT wrote​:

On Sun, 23 Dec 2018 04​:10​:53 GMT, jkeenan@​pobox.com wrote​:

According to pod/perl5260delta.pod​:

#####
* Clang builds with "-DPERL_GLOBAL_STRUCT" or
"-DPERL_GLOBAL_STRUCT_PRIVATE" have been fixed (by disabling Thread
Safety Analysis for these configurations).
#####

However, recent smoke test reports from George Greer indicate that
these
configuration options are once again preventing 'make' from
completing
successfully. See, e.g., http​://perl5.test-smoke.org/report/76521.

According to Porting/bisect.pl, the first bad commit was​:

#####
commit f741678
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Sep 7 15​:21​:56 2017 -0600

Add API function Perl_langinfo()

This is designed to generally replace nl_langinfo() in XS code. It
is
thread-safer, hides the quirks of perl's LC_NUMERIC handling, and
can be
used on systems lacking nl_langinfo.
#####

However, at that commit the build failure occurred in locale.c. In
blead, with the same configuration options, the build failure is
occurring much earlier -- in op.c. I've built perl at several tags
corresponding to monthly development releases since then and it
appears
that the error output has changed at least twice -- once for the
worse,
once for the better -- since then.

I am attaching gzipped files holding the output of the 'make
test_prep'
part of the following command at various points​:

#####
$ sh ./Configure -des -Dusedevel -Dcc=clang
-Accflags=-DPERL_GLOBAL_STRUCT; make test_prep
#####

With this configuration, perl-5.27.3 (2017-Aug-21) built
successfully.
Given that the first bad commit was on 2018-Sep-07, build-time
failures
were observed in perl-5.27.4 (2017-Sep-20). Slightly different
errors
were observed in perl-5.27.5 (2017-Oct-20), but 'make' did not die
until
locale.c. Starting with perl-5.27.6 (2017-Nov-20), however, the
build-time failures started to occur in op.c. These failures have
persisted since then (though the op.c line numbers at which errors
have
been reported have, not surprisingly, jumped around a bit). So I'm
attaching files for 5.27.4, 5.27.5, 5.27.6 and 5.29.6. I'm also
attaching a grep for 'error​:' over my clang builds on linux.

Additional Notes​:

* When I compile with gcc rather than clang, I get different error
output but failures in the same files for the same monthly dev
releases.
Output available upon request.

* I also performed the initial bisection on FreeBSD-11.2. The
bisection
pointed to the same initial "bad" commit.

* I did only one test run with PERL_GLOBAL_STRUCT_PRIVATE, so that
option might need further investigation.

* Since this was a regression between 5.26 and 5.28, we should
explore
backporting any fix to 5.28 maintenance releases.

Thank you very much.
Jim Keenan

Multisectioning suggests that the first commit with build-time errors
was e839e6e and that additional errors appeared at f99042c. See
attachment, transitions.pl.

Thank you very much.

commit e839e6e
Author​: David Mitchell <davem@​iabyn.com>
Date​: Tue Aug 8 18​:42​:14 2017 +0100

Add OP_MULTICONCAT op

commit f99042c
Author​: Zefram <zefram@​fysh.org>
Date​: Sat Nov 4 21​:43​:59 2017 +0000

initialisation of simple aggregate state variables

We now recognise the basic cases of aggregate state variables, to
permit initialisations of the form "state @​a =" or "state @​a
:shared =".
Initialisation of state variable lists remains forbidden, because the
op flags don't let us distinguish "state(...)" from "(state(...))" in
most cases.

Adding a bunch of dVARS fixes the compilation problems, as
https://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-me/khw-
global_struct

indicates.

Unfortunately, the smoke-me/khw-global_struct branch breaks the build on FreeBSD and other platforms.

On FreeBSD-11.2, I compiled with my usual ./Configure options, i.e., without "-DPERL_GLOBAL_STRUCT". I got build-time failures; see attachment smoke-me.khw-global_struct.make.failure.txt.gz.

See also Carlos's smoke reports​:

http​://perl5.test-smoke.org/report/76807
http​://perl5.test-smoke.org/report/76819

Also see​:

http​://perl.develop-help.com/?b=smoke-me%2Fkhw-global_struct

[snip]

The Configure option has no effect because for purposes of this smoke, I
made sure -DPERL_GLOBAL_STRUCT is always defined. I fixed the problem
causing the compile failures you gave, and pushed a new version to smoke

@p5pRT
Copy link
Author

p5pRT commented Dec 27, 2018

From @jkeenan

On Thu, 27 Dec 2018 16​:24​:40 GMT, public@​khwilliamson.com wrote​:

On 12/27/18 8​:46 AM, James E Keenan via RT wrote​:

On Thu, 27 Dec 2018 06​:27​:35 GMT, public@​khwilliamson.com wrote​:

On 12/26/18 8​:30 PM, James E Keenan via RT wrote​:

On Sun, 23 Dec 2018 04​:10​:53 GMT, jkeenan@​pobox.com wrote​:

According to pod/perl5260delta.pod​:

#####
* Clang builds with "-DPERL_GLOBAL_STRUCT" or
"-DPERL_GLOBAL_STRUCT_PRIVATE" have been fixed (by disabling
Thread
Safety Analysis for these configurations).
#####

However, recent smoke test reports from George Greer indicate that
these
configuration options are once again preventing 'make' from
completing
successfully. See, e.g., http​://perl5.test-
smoke.org/report/76521.

According to Porting/bisect.pl, the first bad commit was​:

#####
commit f741678
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Sep 7 15​:21​:56 2017 -0600

Add API function Perl_langinfo()

This is designed to generally replace nl_langinfo() in XS code.
It
is
thread-safer, hides the quirks of perl's LC_NUMERIC handling, and
can be
used on systems lacking nl_langinfo.
#####

However, at that commit the build failure occurred in locale.c.
In
blead, with the same configuration options, the build failure is
occurring much earlier -- in op.c. I've built perl at several
tags
corresponding to monthly development releases since then and it
appears
that the error output has changed at least twice -- once for the
worse,
once for the better -- since then.

I am attaching gzipped files holding the output of the 'make
test_prep'
part of the following command at various points​:

#####
$ sh ./Configure -des -Dusedevel -Dcc=clang
-Accflags=-DPERL_GLOBAL_STRUCT; make test_prep
#####

With this configuration, perl-5.27.3 (2017-Aug-21) built
successfully.
Given that the first bad commit was on 2018-Sep-07, build-time
failures
were observed in perl-5.27.4 (2017-Sep-20). Slightly different
errors
were observed in perl-5.27.5 (2017-Oct-20), but 'make' did not die
until
locale.c. Starting with perl-5.27.6 (2017-Nov-20), however, the
build-time failures started to occur in op.c. These failures have
persisted since then (though the op.c line numbers at which errors
have
been reported have, not surprisingly, jumped around a bit). So
I'm
attaching files for 5.27.4, 5.27.5, 5.27.6 and 5.29.6. I'm also
attaching a grep for 'error​:' over my clang builds on linux.

Additional Notes​:

* When I compile with gcc rather than clang, I get different error
output but failures in the same files for the same monthly dev
releases.
Output available upon request.

* I also performed the initial bisection on FreeBSD-11.2. The
bisection
pointed to the same initial "bad" commit.

* I did only one test run with PERL_GLOBAL_STRUCT_PRIVATE, so that
option might need further investigation.

* Since this was a regression between 5.26 and 5.28, we should
explore
backporting any fix to 5.28 maintenance releases.

Thank you very much.
Jim Keenan

Multisectioning suggests that the first commit with build-time
errors
was e839e6e and that additional errors appeared at f99042c. See
attachment, transitions.pl.

Thank you very much.

commit e839e6e
Author​: David Mitchell <davem@​iabyn.com>
Date​: Tue Aug 8 18​:42​:14 2017 +0100

Add OP_MULTICONCAT op

commit f99042c
Author​: Zefram <zefram@​fysh.org>
Date​: Sat Nov 4 21​:43​:59 2017 +0000

initialisation of simple aggregate state variables

We now recognise the basic cases of aggregate state variables, to
permit initialisations of the form "state @​a =" or "state @​a
:shared =".
Initialisation of state variable lists remains forbidden, because
the
op flags don't let us distinguish "state(...)" from "(state(...))"
in
most cases.

Adding a bunch of dVARS fixes the compilation problems, as
https://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-
me/khw-
global_struct

indicates.

Unfortunately, the smoke-me/khw-global_struct branch breaks the build
on FreeBSD and other platforms.

On FreeBSD-11.2, I compiled with my usual ./Configure options, i.e.,
without "-DPERL_GLOBAL_STRUCT". I got build-time failures; see
attachment smoke-me.khw-global_struct.make.failure.txt.gz.

See also Carlos's smoke reports​:

http​://perl5.test-smoke.org/report/76807
http​://perl5.test-smoke.org/report/76819

Also see​:

http​://perl.develop-help.com/?b=smoke-me%2Fkhw-global_struct

[snip]

The Configure option has no effect because for purposes of this smoke,
I
made sure -DPERL_GLOBAL_STRUCT is always defined. I fixed the problem
causing the compile failures you gave, and pushed a new version to
smoke

Now getting different build-time errors. See attachment with 4d74305 in name.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Dec 27, 2018

@p5pRT
Copy link
Author

p5pRT commented Dec 28, 2018

From @jkeenan

On Thu, 27 Dec 2018 17​:13​:45 GMT, jkeenan wrote​:

On Thu, 27 Dec 2018 16​:24​:40 GMT, public@​khwilliamson.com wrote​:

On 12/27/18 8​:46 AM, James E Keenan via RT wrote​:

On Thu, 27 Dec 2018 06​:27​:35 GMT, public@​khwilliamson.com wrote​:

On 12/26/18 8​:30 PM, James E Keenan via RT wrote​:

On Sun, 23 Dec 2018 04​:10​:53 GMT, jkeenan@​pobox.com wrote​:

According to pod/perl5260delta.pod​:

#####
* Clang builds with "-DPERL_GLOBAL_STRUCT" or
"-DPERL_GLOBAL_STRUCT_PRIVATE" have been fixed (by disabling
Thread
Safety Analysis for these configurations).
#####

However, recent smoke test reports from George Greer indicate
that
these
configuration options are once again preventing 'make' from
completing
successfully. See, e.g., http​://perl5.test-
smoke.org/report/76521.

According to Porting/bisect.pl, the first bad commit was​:

#####
commit f741678
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Sep 7 15​:21​:56 2017 -0600

Add API function Perl_langinfo()

This is designed to generally replace nl_langinfo() in XS code.
It
is
thread-safer, hides the quirks of perl's LC_NUMERIC handling,
and
can be
used on systems lacking nl_langinfo.
#####

However, at that commit the build failure occurred in locale.c.
In
blead, with the same configuration options, the build failure is
occurring much earlier -- in op.c. I've built perl at several
tags
corresponding to monthly development releases since then and it
appears
that the error output has changed at least twice -- once for the
worse,
once for the better -- since then.

I am attaching gzipped files holding the output of the 'make
test_prep'
part of the following command at various points​:

#####
$ sh ./Configure -des -Dusedevel -Dcc=clang
-Accflags=-DPERL_GLOBAL_STRUCT; make test_prep
#####

With this configuration, perl-5.27.3 (2017-Aug-21) built
successfully.
Given that the first bad commit was on 2018-Sep-07, build-time
failures
were observed in perl-5.27.4 (2017-Sep-20). Slightly different
errors
were observed in perl-5.27.5 (2017-Oct-20), but 'make' did not
die
until
locale.c. Starting with perl-5.27.6 (2017-Nov-20), however, the
build-time failures started to occur in op.c. These failures
have
persisted since then (though the op.c line numbers at which
errors
have
been reported have, not surprisingly, jumped around a bit). So
I'm
attaching files for 5.27.4, 5.27.5, 5.27.6 and 5.29.6. I'm also
attaching a grep for 'error​:' over my clang builds on linux.

Additional Notes​:

* When I compile with gcc rather than clang, I get different
error
output but failures in the same files for the same monthly dev
releases.
Output available upon request.

* I also performed the initial bisection on FreeBSD-11.2. The
bisection
pointed to the same initial "bad" commit.

* I did only one test run with PERL_GLOBAL_STRUCT_PRIVATE, so
that
option might need further investigation.

* Since this was a regression between 5.26 and 5.28, we should
explore
backporting any fix to 5.28 maintenance releases.

Thank you very much.
Jim Keenan

Multisectioning suggests that the first commit with build-time
errors
was e839e6e and that additional errors appeared at f99042c. See
attachment, transitions.pl.

Thank you very much.

commit e839e6e
Author​: David Mitchell <davem@​iabyn.com>
Date​: Tue Aug 8 18​:42​:14 2017 +0100

Add OP_MULTICONCAT op

commit f99042c
Author​: Zefram <zefram@​fysh.org>
Date​: Sat Nov 4 21​:43​:59 2017 +0000

initialisation of simple aggregate state variables

We now recognise the basic cases of aggregate state variables, to
permit initialisations of the form "state @​a =" or "state @​a
:shared =".
Initialisation of state variable lists remains forbidden, because
the
op flags don't let us distinguish "state(...)" from "(state(...))"
in
most cases.

Adding a bunch of dVARS fixes the compilation problems, as
https://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-
me/khw-
global_struct

indicates.

Unfortunately, the smoke-me/khw-global_struct branch breaks the
build
on FreeBSD and other platforms.

On FreeBSD-11.2, I compiled with my usual ./Configure options,
i.e.,
without "-DPERL_GLOBAL_STRUCT". I got build-time failures; see
attachment smoke-me.khw-global_struct.make.failure.txt.gz.

See also Carlos's smoke reports​:

http​://perl5.test-smoke.org/report/76807
http​://perl5.test-smoke.org/report/76819

Also see​:

http​://perl.develop-help.com/?b=smoke-me%2Fkhw-global_struct

[snip]

The Configure option has no effect because for purposes of this
smoke,
I
made sure -DPERL_GLOBAL_STRUCT is always defined. I fixed the
problem
causing the compile failures you gave, and pushed a new version to
smoke

Now getting different build-time errors. See attachment with 4d74305
in name.

Now, in the smoke-me/khw-global_struct branch, at commit 1380ecd, 'make' completes but I get these test failures​:

#####
Test Summary Report


re/pat.t (Wstat​: 0 Tests​: 851 Failed​: 1)
  Failed test​: 761
re/anyof.t (Wstat​: 0 Tests​: 1266 Failed​: 628)
  Failed tests​: 1-36, 237-240, 243-244, 259-260, 301-304
  307-308, 317-320, 323-324, 333, 339-340
  349-352, 355-356, 365, 371-372, 419-421
  423, 425-426, 428, 430-438, 447-455, 457
  459-460, 462, 464-472, 481-489, 491, 493-494
  496, 498-506, 515-523, 525, 527-528, 530
  532-540, 549-557, 559, 561-562, 564, 566-635
  638-648, 650, 652-655, 657, 660-668, 671-672
  675-677, 693-696, 710-712, 719-722, 725-727
  743-751, 763-765, 779-780, 783-785, 788-790
  802-804, 821-823, 839-842, 856-858, 860-864
  866-867, 878-879, 881-883, 916-917, 919-921
  938-940, 954-955, 957-959, 962-966, 968-970
  973-982, 984-985, 988, 991-993, 996, 998-1121
  1124-1266
  Parse errors​: Plan (1..1155) must be at the beginning or end of the TAP output
  Bad plan. You planned 1155 tests but ran 1266.
re/pat_thr.t (Wstat​: 0 Tests​: 851 Failed​: 1)
  Failed test​: 761
../ext/re/t/regop.t (Wstat​: 0 Tests​: 31 Failed​: 3)
  Failed tests​: 1, 30-31
  Parse errors​: Bad plan. You planned 54 tests but ran 31.
Files=2631, Tests=1182264, 279 wallclock secs (78.41 usr 10.23 sys + 566.38 cusr 55.54 csys = 710.57 CPU)
Result​: FAIL
*** Error code 121

Stop.
make​: stopped in /usr/home/jkeenan/gitwork/perl
#####

Thank you very much.
Jim Keenan

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Dec 28, 2018

From @jkeenan

Summary of my perl5 (revision 5 version 29 subversion 7) configuration​:
  Commit id​: 1380ecd
  Platform​:
  osname=freebsd
  osvers=11.2-stable
  archname=amd64-freebsd-thread-multi
  uname='freebsd perlmonger.nycbug.org 11.2-stable freebsd 11.2-stable #0 r339445​: sat oct 20 00​:08​:11 utc 2018 root@​perlmonger.nycbug.org​:usrobjusrsrcsysgeneric amd64 '
  config_args='-des -Dusedevel -Duseithreads -Doptimize=-O2 -pipe -fstack-protector -fno-strict-aliasing'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
  optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
  cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.2.1 Compatible FreeBSD Clang 6.0.1 (tags/RELEASE_601/final 335540)'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags ='-pthread -Wl,-E -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/lib /usr/local/lib /usr/lib/clang/6.0.1/lib /usr/lib
  libs=-lpthread -lgdbm -ldl -lm -lcrypt -lutil
  perllibs=-lpthread -ldl -lm -lcrypt -lutil
  libc=
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags=' '
  cccdlflags='-DPIC -fPIC'
  lddlflags='-shared -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  MULTIPLICITY
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_GLOBAL_STRUCT
  PERL_IMPLICIT_CONTEXT
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_ITHREADS
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  USE_REENTRANT_API
  Built under freebsd
  Compiled at Dec 27 2018 23​:39​:52
  %ENV​:
  PERL2DIR="/home/jkeenan/gitwork/perl2"
  PERL_WORKDIR="/home/jkeenan/gitwork/perl"
  @​INC​:
  lib
  /usr/local/lib/perl5/site_perl/5.29.7/amd64-freebsd-thread-multi
  /usr/local/lib/perl5/site_perl/5.29.7
  /usr/local/lib/perl5/5.29.7/amd64-freebsd-thread-multi
  /usr/local/lib/perl5/5.29.7

@p5pRT
Copy link
Author

p5pRT commented Dec 28, 2018

From @iabyn

On Thu, Dec 27, 2018 at 04​:33​:19PM -0800, James E Keenan via RT wrote​:

Now, in the smoke-me/khw-global_struct branch, at commit 1380ecd, 'make' completes but I get these test failures​:

Note that I have (unfortunately) been independently been working on getting
PERL_GLOBAL_STRUCT_PRIVATE build working again.

In an unpushd branch I also have commits that add dVARs - I haven't tested
it on non-PERL_GLOBAL_STRUCT_PRIVATE builds yet.

In addition I have commits to make t/porting/libperl.t pass. I'll carry on
working on fixing those 'tests which break under PERL_GLOBAL_STRUCT' and
ignore getting perl to build for now.

--
If life gives you lemons, you'll probably develop a citric acid allergy.

@p5pRT
Copy link
Author

p5pRT commented Dec 30, 2018

From @khwilliamson

On 12/27/18 5​:33 PM, James E Keenan via RT wrote​:

On Thu, 27 Dec 2018 17​:13​:45 GMT, jkeenan wrote​:

On Thu, 27 Dec 2018 16​:24​:40 GMT, public@​khwilliamson.com wrote​:

On 12/27/18 8​:46 AM, James E Keenan via RT wrote​:

On Thu, 27 Dec 2018 06​:27​:35 GMT, public@​khwilliamson.com wrote​:

On 12/26/18 8​:30 PM, James E Keenan via RT wrote​:

On Sun, 23 Dec 2018 04​:10​:53 GMT, jkeenan@​pobox.com wrote​:

According to pod/perl5260delta.pod​:

#####
* Clang builds with "-DPERL_GLOBAL_STRUCT" or
"-DPERL_GLOBAL_STRUCT_PRIVATE" have been fixed (by disabling
Thread
Safety Analysis for these configurations).
#####

However, recent smoke test reports from George Greer indicate
that
these
configuration options are once again preventing 'make' from
completing
successfully. See, e.g., http​://perl5.test-
smoke.org/report/76521.

According to Porting/bisect.pl, the first bad commit was​:

#####
commit f741678
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Sep 7 15​:21​:56 2017 -0600

Add API function Perl_langinfo()

This is designed to generally replace nl_langinfo() in XS code.
It
is
thread-safer, hides the quirks of perl's LC_NUMERIC handling,
and
can be
used on systems lacking nl_langinfo.
#####

However, at that commit the build failure occurred in locale.c.
In
blead, with the same configuration options, the build failure is
occurring much earlier -- in op.c. I've built perl at several
tags
corresponding to monthly development releases since then and it
appears
that the error output has changed at least twice -- once for the
worse,
once for the better -- since then.

I am attaching gzipped files holding the output of the 'make
test_prep'
part of the following command at various points​:

#####
$ sh ./Configure -des -Dusedevel -Dcc=clang
-Accflags=-DPERL_GLOBAL_STRUCT; make test_prep
#####

With this configuration, perl-5.27.3 (2017-Aug-21) built
successfully.
Given that the first bad commit was on 2018-Sep-07, build-time
failures
were observed in perl-5.27.4 (2017-Sep-20). Slightly different
errors
were observed in perl-5.27.5 (2017-Oct-20), but 'make' did not
die
until
locale.c. Starting with perl-5.27.6 (2017-Nov-20), however, the
build-time failures started to occur in op.c. These failures
have
persisted since then (though the op.c line numbers at which
errors
have
been reported have, not surprisingly, jumped around a bit). So
I'm
attaching files for 5.27.4, 5.27.5, 5.27.6 and 5.29.6. I'm also
attaching a grep for 'error​:' over my clang builds on linux.

Additional Notes​:

* When I compile with gcc rather than clang, I get different
error
output but failures in the same files for the same monthly dev
releases.
Output available upon request.

* I also performed the initial bisection on FreeBSD-11.2. The
bisection
pointed to the same initial "bad" commit.

* I did only one test run with PERL_GLOBAL_STRUCT_PRIVATE, so
that
option might need further investigation.

* Since this was a regression between 5.26 and 5.28, we should
explore
backporting any fix to 5.28 maintenance releases.

Thank you very much.
Jim Keenan

Multisectioning suggests that the first commit with build-time
errors
was e839e6e and that additional errors appeared at f99042c. See
attachment, transitions.pl.

Thank you very much.

commit e839e6e
Author​: David Mitchell <davem@​iabyn.com>
Date​: Tue Aug 8 18​:42​:14 2017 +0100

Add OP_MULTICONCAT op

commit f99042c
Author​: Zefram <zefram@​fysh.org>
Date​: Sat Nov 4 21​:43​:59 2017 +0000

initialisation of simple aggregate state variables

We now recognise the basic cases of aggregate state variables, to
permit initialisations of the form "state @​a =" or "state @​a
:shared =".
Initialisation of state variable lists remains forbidden, because
the
op flags don't let us distinguish "state(...)" from "(state(...))"
in
most cases.

Adding a bunch of dVARS fixes the compilation problems, as
https://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-
me/khw-
global_struct

indicates.

Unfortunately, the smoke-me/khw-global_struct branch breaks the
build
on FreeBSD and other platforms.

On FreeBSD-11.2, I compiled with my usual ./Configure options,
i.e.,
without "-DPERL_GLOBAL_STRUCT". I got build-time failures; see
attachment smoke-me.khw-global_struct.make.failure.txt.gz.

See also Carlos's smoke reports​:

http​://perl5.test-smoke.org/report/76807
http​://perl5.test-smoke.org/report/76819

Also see​:

http​://perl.develop-help.com/?b=smoke-me%2Fkhw-global_struct

[snip]

The Configure option has no effect because for purposes of this
smoke,
I
made sure -DPERL_GLOBAL_STRUCT is always defined. I fixed the
problem
causing the compile failures you gave, and pushed a new version to
smoke

Now getting different build-time errors. See attachment with 4d74305
in name.

Now, in the smoke-me/khw-global_struct branch, at commit 1380ecd, 'make' completes but I get these test failures​:

#####
Test Summary Report
-------------------
re/pat.t (Wstat​: 0 Tests​: 851 Failed​: 1)
Failed test​: 761
re/anyof.t (Wstat​: 0 Tests​: 1266 Failed​: 628)
Failed tests​: 1-36, 237-240, 243-244, 259-260, 301-304
307-308, 317-320, 323-324, 333, 339-340
349-352, 355-356, 365, 371-372, 419-421
423, 425-426, 428, 430-438, 447-455, 457
459-460, 462, 464-472, 481-489, 491, 493-494
496, 498-506, 515-523, 525, 527-528, 530
532-540, 549-557, 559, 561-562, 564, 566-635
638-648, 650, 652-655, 657, 660-668, 671-672
675-677, 693-696, 710-712, 719-722, 725-727
743-751, 763-765, 779-780, 783-785, 788-790
802-804, 821-823, 839-842, 856-858, 860-864
866-867, 878-879, 881-883, 916-917, 919-921
938-940, 954-955, 957-959, 962-966, 968-970
973-982, 984-985, 988, 991-993, 996, 998-1121
1124-1266
Parse errors​: Plan (1..1155) must be at the beginning or end of the TAP output
Bad plan. You planned 1155 tests but ran 1266.
re/pat_thr.t (Wstat​: 0 Tests​: 851 Failed​: 1)
Failed test​: 761
../ext/re/t/regop.t (Wstat​: 0 Tests​: 31 Failed​: 3)
Failed tests​: 1, 30-31
Parse errors​: Bad plan. You planned 54 tests but ran 31.
Files=2631, Tests=1182264, 279 wallclock secs (78.41 usr 10.23 sys + 566.38 cusr 55.54 csys = 710.57 CPU)
Result​: FAIL
*** Error code 121

Stop.
make​: stopped in /usr/home/jkeenan/gitwork/perl
#####

Thank you very much.
Jim Keenan

It turns out that there is a longstanding bug in perlvars.h which my
recent changes and George Greer's++ smokers showed. I asked on #irc for
help in figuring out what was going on, and Sergey Aleynikov++ came to
the rescue.

The problem is these lines in the middle​:

#ifdef DEBUGGING
PERLVARI(G, watch_pvx, char *, NULL)
#endif

On a PERL_GLOBAL_STRUCT non-DEBUGGING build, regcomp.o doesn't see that
entry, but re_comp.o does. That means the offsets for all entries in
the struct after watch_pvx are different. If I or gdb think I'm looking
at entry 'foo', we really are looking at entry 'bar'. Put another way,
the value of STRUCT_OFFSET(struct perl_vars, foo) differs between
regcomp.o and re_comp.o for all entries 'foo' which come in the
structure after watch_pvx.

A solution in this case is to move the watch_pvx to the end so nothing
comes after it, or to remove the #ifdef wrapping.

The #ifdef has been there since at least
  27da23d Jarkko Hietaniemi 2005-04-18

But what about the general case. Are there other inconsistencies that
can bite us? There are other #ifdef's in the file. Sergey metioned
Perl_xs_handshake(). Apparently that doesn't get called in this case,
or is buggy. Is there a porting test that could be written?

@p5pRT
Copy link
Author

p5pRT commented Dec 31, 2018

From @demerphq

I suspect this is worse for regex than for other cases as we can call into
the debugging version of the engine from the non debugging perl. In fact
this rings a bell, I think I fixed a similar issue long ago. Things used by
the debugging version must be present in the non debugging version

Yves

On Sun, 30 Dec 2018, 16​:44 Karl Williamson <public@​khwilliamson.com wrote​:

On 12/27/18 5​:33 PM, James E Keenan via RT wrote​:

On Thu, 27 Dec 2018 17​:13​:45 GMT, jkeenan wrote​:

On Thu, 27 Dec 2018 16​:24​:40 GMT, public@​khwilliamson.com wrote​:

On 12/27/18 8​:46 AM, James E Keenan via RT wrote​:

On Thu, 27 Dec 2018 06​:27​:35 GMT, public@​khwilliamson.com wrote​:

On 12/26/18 8​:30 PM, James E Keenan via RT wrote​:

On Sun, 23 Dec 2018 04​:10​:53 GMT, jkeenan@​pobox.com wrote​:

According to pod/perl5260delta.pod​:

#####
* Clang builds with "-DPERL_GLOBAL_STRUCT" or
"-DPERL_GLOBAL_STRUCT_PRIVATE" have been fixed (by disabling
Thread
Safety Analysis for these configurations).
#####

However, recent smoke test reports from George Greer indicate
that
these
configuration options are once again preventing 'make' from
completing
successfully. See, e.g., http​://perl5.test-
smoke.org/report/76521.

According to Porting/bisect.pl, the first bad commit was​:

#####
commit f741678
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Sep 7 15​:21​:56 2017 -0600

Add API function Perl_langinfo()

This is designed to generally replace nl_langinfo() in XS code.
It
is
thread-safer, hides the quirks of perl's LC_NUMERIC handling,
and
can be
used on systems lacking nl_langinfo.
#####

However, at that commit the build failure occurred in locale.c.
In
blead, with the same configuration options, the build failure is
occurring much earlier -- in op.c. I've built perl at several
tags
corresponding to monthly development releases since then and it
appears
that the error output has changed at least twice -- once for the
worse,
once for the better -- since then.

I am attaching gzipped files holding the output of the 'make
test_prep'
part of the following command at various points​:

#####
$ sh ./Configure -des -Dusedevel -Dcc=clang
-Accflags=-DPERL_GLOBAL_STRUCT; make test_prep
#####

With this configuration, perl-5.27.3 (2017-Aug-21) built
successfully.
Given that the first bad commit was on 2018-Sep-07, build-time
failures
were observed in perl-5.27.4 (2017-Sep-20). Slightly different
errors
were observed in perl-5.27.5 (2017-Oct-20), but 'make' did not
die
until
locale.c. Starting with perl-5.27.6 (2017-Nov-20), however, the
build-time failures started to occur in op.c. These failures
have
persisted since then (though the op.c line numbers at which
errors
have
been reported have, not surprisingly, jumped around a bit). So
I'm
attaching files for 5.27.4, 5.27.5, 5.27.6 and 5.29.6. I'm also
attaching a grep for 'error​:' over my clang builds on linux.

Additional Notes​:

* When I compile with gcc rather than clang, I get different
error
output but failures in the same files for the same monthly dev
releases.
Output available upon request.

* I also performed the initial bisection on FreeBSD-11.2. The
bisection
pointed to the same initial "bad" commit.

* I did only one test run with PERL_GLOBAL_STRUCT_PRIVATE, so
that
option might need further investigation.

* Since this was a regression between 5.26 and 5.28, we should
explore
backporting any fix to 5.28 maintenance releases.

Thank you very much.
Jim Keenan

Multisectioning suggests that the first commit with build-time
errors
was e839e6e and that additional errors appeared at f99042c. See
attachment, transitions.pl.

Thank you very much.

commit e839e6e
Author​: David Mitchell <davem@​iabyn.com>
Date​: Tue Aug 8 18​:42​:14 2017 +0100

Add OP_MULTICONCAT op

commit f99042c
Author​: Zefram <zefram@​fysh.org>
Date​: Sat Nov 4 21​:43​:59 2017 +0000

initialisation of simple aggregate state variables

We now recognise the basic cases of aggregate state variables, to
permit initialisations of the form "state @​a =" or "state @​a
:shared =".
Initialisation of state variable lists remains forbidden, because
the
op flags don't let us distinguish "state(...)" from "(state(...))"
in
most cases.

Adding a bunch of dVARS fixes the compilation problems, as
https://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-
me/khw-
global_struct

indicates.

Unfortunately, the smoke-me/khw-global_struct branch breaks the
build
on FreeBSD and other platforms.

On FreeBSD-11.2, I compiled with my usual ./Configure options,
i.e.,
without "-DPERL_GLOBAL_STRUCT". I got build-time failures; see
attachment smoke-me.khw-global_struct.make.failure.txt.gz.

See also Carlos's smoke reports​:

http​://perl5.test-smoke.org/report/76807
http​://perl5.test-smoke.org/report/76819

Also see​:

http​://perl.develop-help.com/?b=smoke-me%2Fkhw-global_struct

[snip]

The Configure option has no effect because for purposes of this
smoke,
I
made sure -DPERL_GLOBAL_STRUCT is always defined. I fixed the
problem
causing the compile failures you gave, and pushed a new version to
smoke

Now getting different build-time errors. See attachment with 4d74305
in name.

Now, in the smoke-me/khw-global_struct branch, at commit
1380ecd, 'make' completes but I get these
test failures​:

#####
Test Summary Report
-------------------
re/pat.t (Wstat​:
0 Tests​: 851 Failed​: 1)
Failed test​: 761
re/anyof.t (Wstat​:
0 Tests​: 1266 Failed​: 628)
Failed tests​: 1-36, 237-240, 243-244, 259-260, 301-304
307-308, 317-320, 323-324, 333, 339-340
349-352, 355-356, 365, 371-372, 419-421
423, 425-426, 428, 430-438, 447-455, 457
459-460, 462, 464-472, 481-489, 491, 493-494
496, 498-506, 515-523, 525, 527-528, 530
532-540, 549-557, 559, 561-562, 564, 566-635
638-648, 650, 652-655, 657, 660-668, 671-672
675-677, 693-696, 710-712, 719-722, 725-727
743-751, 763-765, 779-780, 783-785, 788-790
802-804, 821-823, 839-842, 856-858, 860-864
866-867, 878-879, 881-883, 916-917, 919-921
938-940, 954-955, 957-959, 962-966, 968-970
973-982, 984-985, 988, 991-993, 996, 998-1121
1124-1266
Parse errors​: Plan (1..1155) must be at the beginning or end of the
TAP output
Bad plan. You planned 1155 tests but ran 1266.
re/pat_thr.t (Wstat​:
0 Tests​: 851 Failed​: 1)
Failed test​: 761
../ext/re/t/regop.t (Wstat​:
0 Tests​: 31 Failed​: 3)
Failed tests​: 1, 30-31
Parse errors​: Bad plan. You planned 54 tests but ran 31.
Files=2631, Tests=1182264, 279 wallclock secs (78.41 usr 10.23 sys +
566.38 cusr 55.54 csys = 710.57 CPU)
Result​: FAIL
*** Error code 121

Stop.
make​: stopped in /usr/home/jkeenan/gitwork/perl
#####

Thank you very much.
Jim Keenan

It turns out that there is a longstanding bug in perlvars.h which my
recent changes and George Greer's++ smokers showed. I asked on #irc for
help in figuring out what was going on, and Sergey Aleynikov++ came to
the rescue.

The problem is these lines in the middle​:

#ifdef DEBUGGING
PERLVARI(G, watch_pvx, char *, NULL)
#endif

On a PERL_GLOBAL_STRUCT non-DEBUGGING build, regcomp.o doesn't see that
entry, but re_comp.o does. That means the offsets for all entries in
the struct after watch_pvx are different. If I or gdb think I'm looking
at entry 'foo', we really are looking at entry 'bar'. Put another way,
the value of STRUCT_OFFSET(struct perl_vars, foo) differs between
regcomp.o and re_comp.o for all entries 'foo' which come in the
structure after watch_pvx.

A solution in this case is to move the watch_pvx to the end so nothing
comes after it, or to remove the #ifdef wrapping.

The #ifdef has been there since at least
27da23d Jarkko Hietaniemi 2005-04-18

But what about the general case. Are there other inconsistencies that
can bite us? There are other #ifdef's in the file. Sergey metioned
Perl_xs_handshake(). Apparently that doesn't get called in this case,
or is buggy. Is there a porting test that could be written?

@p5pRT
Copy link
Author

p5pRT commented Dec 31, 2018

From @iabyn

On Mon, Dec 31, 2018 at 12​:35​:47PM +0100, demerphq wrote​:

I suspect this is worse for regex than for other cases as we can call into
the debugging version of the engine from the non debugging perl. In fact
this rings a bell, I think I fixed a similar issue long ago. Things used by
the debugging version must be present in the non debugging version

Yeah, we've banned '#ifdef DEBUGGING' in intrpvar.h for this exact reason.
The ban should be extended to perlvars.h too (presumably adding a comment
at the top of the file mentioning this).

On Sun, 30 Dec 2018, 16​:44 Karl Williamson <public@​khwilliamson.com wrote​:

But what about the general case. Are there other inconsistencies that
can bite us? There are other #ifdef's in the file.

I guess we'd have to audit regcomp.c and regexec.c (and any .h's included
by them) for PERL_EXT_RE_BUILD, PERL_EXT_RE_DEBUG, PERL_EXT plus anything
derived from them, such as DEBUGGING.

A quick glance at regcomp.c gives 'struct RExC_state_t' having some
DEBUGGING-only fields in the middle. I'm not clear why this isn't already
biting us. Presumably because when running under -Dr, only the re_comp.c
version of the struct is used throughout, rather than a mixture.

Sergey metioned
Perl_xs_handshake(). Apparently that doesn't get called in this case,
or is buggy. Is there a porting test that could be written?

Its not clear to me what you/he means. Perhaps it's that
Perl_xs_handshake() only checks for differences in the size the of
interpreter structure in the running compiler and in an XS module being
loaded?

Perhaps its functionality should be extended to check for the sizes of the
global vars struct under PERL_GLOBAL_STRUCT builds?

Or perhaps we should abandon PERL_GLOBAL_STRUCT altogether? My
understanding is that it was added by Jarkko to support Symbian, which is
now obsolete. I don't know whether it has any other use cases.

--
"You're so sadly neglected, and often ignored.
A poor second to Belgium, When going abroad."
  -- Monty Python, "Finland"

@p5pRT
Copy link
Author

p5pRT commented Dec 31, 2018

From @khwilliamson

On 12/31/18 5​:46 AM, Dave Mitchell wrote​:

On Mon, Dec 31, 2018 at 12​:35​:47PM +0100, demerphq wrote​:

I suspect this is worse for regex than for other cases as we can call into
the debugging version of the engine from the non debugging perl. In fact
this rings a bell, I think I fixed a similar issue long ago. Things used by
the debugging version must be present in the non debugging version

Yeah, we've banned '#ifdef DEBUGGING' in intrpvar.h for this exact reason.
The ban should be extended to perlvars.h too (presumably adding a comment
at the top of the file mentioning this).

On Sun, 30 Dec 2018, 16​:44 Karl Williamson <public@​khwilliamson.com wrote​:

But what about the general case. Are there other inconsistencies that
can bite us? There are other #ifdef's in the file.

I guess we'd have to audit regcomp.c and regexec.c (and any .h's included
by them) for PERL_EXT_RE_BUILD, PERL_EXT_RE_DEBUG, PERL_EXT plus anything
derived from them, such as DEBUGGING.

A quick glance at regcomp.c gives 'struct RExC_state_t' having some
DEBUGGING-only fields in the middle. I'm not clear why this isn't already
biting us. Presumably because when running under -Dr, only the re_comp.c
version of the struct is used throughout, rather than a mixture.

The difference is that these globals are, well, global and have effect
for the entire program. The struct you refer to is used locally inside
regcomp.c or re_comp.c and so its fine to have different versions, as
the data is not shared.

Sergey metioned
Perl_xs_handshake(). Apparently that doesn't get called in this case,
or is buggy. Is there a porting test that could be written?

Its not clear to me what you/he means. Perhaps it's that
Perl_xs_handshake() only checks for differences in the size the of
interpreter structure in the running compiler and in an XS module being
loaded?

Yes

Perhaps its functionality should be extended to check for the sizes of the
global vars struct under PERL_GLOBAL_STRUCT builds?

Yes. Other possibilities would be to grep for #ifdefs in a porting/t
test. A comment at the beginning of the file might not be seen by
someone adding something at the end. But regen/embed.pl could also look
for DEBUGGING #ifdefs.

But Sergey wondered if there are other problematic #ifdefs, not just for
regcomp.c and re_comp.c

Or perhaps we should abandon PERL_GLOBAL_STRUCT altogether? My
understanding is that it was added by Jarkko to support Symbian, which is
now obsolete. I don't know whether it has any other use cases.

And also PERL_GLOBAL_STRUCT_PRIVATE?

I've cc'd Jarkko, as last time I recall he wasn't reading p5p

@p5pRT
Copy link
Author

p5pRT commented Feb 13, 2019

From @jhi

[Karl found me gathering dust in a closet]

So yes, the global struct config was created for Symbian, which is dead
as a dodo. The probability of still existing Symbian phones getting
Perl installed is extremely low.

But more generally, the config was not Symbian-specific as such.
It was created to cover for a limitation in Symbian, and that was
"shared libraries shall not have writeable data" (they could have
data, but only read-only). In more modern operating systems the
writeable data sections of shared libraries are copy-on-write

The global struct is a bit misleading as terms go​: what it does is that
it pulls all the global data into a single struct, which can then be
heap-allocated in main, and just passed around. So it is "a struct for
globals", not a "global" "struct". The feature comes in two flavors,
vanilla and PRIVATE,the latter of which is really strict​: it leaves
not even the global struct visible, but instead a function call
through which you can access the the struct. (The description
I wrote (I think) for perlguts is still good.)

I still find the config useful as a cleanliness exercise​: do we have a
good enough understanding and tracking of the global data so that we
can move all of it to heap, making the shared library as "pure" as possible.
But then again, as the creator of the feature I am biased.

@p5pRT
Copy link
Author

p5pRT commented Feb 19, 2019

From @iabyn

On Wed, Feb 13, 2019 at 11​:04​:18AM -0800, Jarkko Hietaniemi via RT wrote​:

I still find the config useful as a cleanliness exercise​: do we have a
good enough understanding and tracking of the global data so that we
can move all of it to heap, making the shared library as "pure" as possible.
But then again, as the creator of the feature I am biased.

I've now merged the following branch into blead​:

  commit 737b460
  Author​: David Mitchell <davem@​iabyn.com>
  AuthorDate​: Tue Feb 19 15​:41​:03 2019 +0000

  [MERGE] fix PERL_GLOBAL_STRUCT builds
 
  PERL_GLOBAL_STRUCT and DPERL_GLOBAL_STRUCT_PRIVATE builds haven't been
  smoked for quite a while and the code has bit-rotted. The commits in
  this branch made them build and all test pass again, at least on Linux

It makes all tests pass (at least on Linux). It incorporates all the
changes from Karl's branch apart from​:

1) The PL_WARN_ALL/PL_WARN_NONE one,
2) and the

# define LC_NUMERIC_LOCK(cond_to_panic_if_already_locked) \
  CLANG_DIAG_IGNORE(-Wthread-safety) \
  STMT_START { \
+ dVAR; \

etc one, because neither of these seemed needed on my builds to pass.
I blindly copied the win32.c dVAR additions.

--
The Enterprise's efficient long-range scanners detect a temporal vortex
distortion in good time, allowing it to be safely avoided via a minor
course correction.
  -- Things That Never Happen in "Star Trek" #21

@p5pRT
Copy link
Author

p5pRT commented Feb 25, 2019

@iabyn - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented May 22, 2019

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

@p5pRT
Copy link
Author

p5pRT commented May 22, 2019

@khwilliamson - Status changed from 'pending release' to 'resolved'

jkeenan added a commit to jkeenan/devel-git-multibisect that referenced this issue Aug 27, 2021
This will test BuildTransitions.pm in a situation derived from
Perl/perl5#16794.  With a particular, uncommon
set of configuration options, the build was broken (5 errors reported) at the point where
the multiconcat op was introduced.  A short time later 1 additional
build-time error was introduced.  (This program currently does not
extend to the point where the build was fixed.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant