Skip Menu |
Report information
Id: 132360
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: dan.dedrick [at] gmail.com
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: core
Perl Version: 5.27.6
Fixed In: (no value)

Attachments
0001-Fix-missing-build-dependency-for-pods.patch



Date: Thu, 26 Oct 2017 20:15:38 -0400
From: dan.dedrick [...] gmail.com
Subject: [PATCH] Fix missing build dependency for pods
To: perlbug [...] perl.org
CC: ddedrick [...] lexmark.com
Download (untitled) / with headers
text/plain 4.1k
This is a bug report for perl from dan.dedrick@gmail.com, generated with the help of perlbug 1.40 running under perl 5.27.6. ----------------------------------------------------------------- [Please describe your issue here] When building perl with highly parallel options (e.g. -j 32) on a machine with many cores (e.g. 32) ocassionaly it fails with the following type of error. make[1]: Entering directory '.../cpan/podlators' Can't locate Getopt/Long.pm in @INC (you may need to install the Getopt::Long module) (@INC contains: .../cpan/AutoLoader/lib .../dist/Carp/lib .../dist/PathTools .../dist/PathTools/lib .../cpan/ExtUtils-Install/lib .../cpan/ExtUtils-MakeMaker/lib .../cpan/ExtUtils-Manifest/lib .../cpan/File-Path/lib .../ext/re .../dist/Term-ReadLine/lib .../dist/Exporter/lib .../ext/File-Find/lib .../cpan/Text-Tabs/lib .../dist/constant/lib .../cpan/version/lib .../lib ../../lib .) at .../cpan/ExtUtils-MakeMaker/lib/ExtUtils/Command/MM.pm line 109. Makefile:445: recipe for target 'manifypods' failed make[1]: *** [manifypods] Error 2 The scripts pod2man, pod2text, podchecker, podselect, and pod2usage all use Getopt-Long and since they are all part of nonxs modules this needs to be added here to prevent these build races. [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=low Type=Patch PatchStatus=HasPatch --- Site configuration information for perl 5.27.6: Configured by ddedrick at Thu Oct 26 19:54:14 EDT 2017. Summary of my perl5 (revision 5 version 27 subversion 6) configuration: Local Commit: 26cf06121b03ef60fd49af6f0f56e743b332f6b0 Ancestor: ffaed89356842ad8f2641781afa31ead7349fb7a Platform: osname=linux osvers=4.12.13-300.fc26.x86_64 archname=x86_64-linux uname='linux aeryn 4.12.13-300.fc26.x86_64 #1 smp thu sep 14 16:00:38 utc 2017 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Dprefix=/home/ddedrick/src/perl/ -Dusedevel' hint=recommended useposix=true d_sigaction=define useithreads=undef usemultiplicity=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cc' ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2' optimize='-O2' cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='7.2.1 20170915 (Red Hat 7.2.1-2)' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64 /usr/local/lib64 libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.25.so so=so useshrplib=false libperl=libperl.a gnulibc_version='2.25' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong' Locally applied patches: 26cf06121b03ef60fd49af6f0f56e743b332f6b0 --- @INC for perl 5.27.6: lib /home/ddedrick/src/perl/lib/site_perl/5.27.6/x86_64-linux /home/ddedrick/src/perl/lib/site_perl/5.27.6 /home/ddedrick/src/perl/lib/5.27.6/x86_64-linux /home/ddedrick/src/perl/lib/5.27.6 --- Environment for perl 5.27.6: HOME=/home/ddedrick LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/ddedrick/.local/bin:/home/ddedrick/bin PERL_BADLANG (unset) SHELL=/bin/bash

Message body is not shown because sender requested not to inline it.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.5k
On Fri, 27 Oct 2017 00:47:58 GMT, dan.dedrick@gmail.com wrote: Show quoted text
> This is a bug report for perl from dan.dedrick@gmail.com, > generated with the help of perlbug 1.40 running under perl 5.27.6. > > > ----------------------------------------------------------------- > [Please describe your issue here] > > When building perl with highly parallel options (e.g. -j 32) on a > machine > with many cores (e.g. 32) ocassionaly it fails with the following type > of error. > > make[1]: Entering directory '.../cpan/podlators' > Can't locate Getopt/Long.pm in @INC (you may need to install the > Getopt::Long module) (@INC contains: .../cpan/AutoLoader/lib > .../dist/Carp/lib .../dist/PathTools .../dist/PathTools/lib > .../cpan/ExtUtils-Install/lib .../cpan/ExtUtils-MakeMaker/lib > .../cpan/ExtUtils-Manifest/lib .../cpan/File-Path/lib .../ext/re > .../dist/Term-ReadLine/lib .../dist/Exporter/lib .../ext/File-Find/lib > .../cpan/Text-Tabs/lib .../dist/constant/lib .../cpan/version/lib > .../lib ../../lib .) at .../cpan/ExtUtils- > MakeMaker/lib/ExtUtils/Command/MM.pm line 109. > Makefile:445: recipe for target 'manifypods' failed > make[1]: *** [manifypods] Error 2 > > The scripts pod2man, pod2text, podchecker, podselect, and pod2usage > all use > Getopt-Long and since they are all part of nonxs modules this needs to > be > added here to prevent these build races. > >
Can you explain a bit more as to what you think is the sequence of events involved in the race condition? Thank you very much. -- James E Keenan (jkeenan@cpan.org)
Download (untitled) / with headers
text/plain 2.2k
On Fri, 27 Oct 2017 17:28:22 -0700, jkeenan wrote: Show quoted text
> On Fri, 27 Oct 2017 00:47:58 GMT, dan.dedrick@gmail.com wrote:
> > This is a bug report for perl from dan.dedrick@gmail.com, > > generated with the help of perlbug 1.40 running under perl 5.27.6. > > > > > > ----------------------------------------------------------------- > > [Please describe your issue here] > > > > When building perl with highly parallel options (e.g. -j 32) on a > > machine > > with many cores (e.g. 32) ocassionaly it fails with the following > > type > > of error. > > > > make[1]: Entering directory '.../cpan/podlators' > > Can't locate Getopt/Long.pm in @INC (you may need to install the > > Getopt::Long module) (@INC contains: .../cpan/AutoLoader/lib > > .../dist/Carp/lib .../dist/PathTools .../dist/PathTools/lib > > .../cpan/ExtUtils-Install/lib .../cpan/ExtUtils-MakeMaker/lib > > .../cpan/ExtUtils-Manifest/lib .../cpan/File-Path/lib .../ext/re > > .../dist/Term-ReadLine/lib .../dist/Exporter/lib .../ext/File- > > Find/lib > > .../cpan/Text-Tabs/lib .../dist/constant/lib .../cpan/version/lib > > .../lib ../../lib .) at .../cpan/ExtUtils- > > MakeMaker/lib/ExtUtils/Command/MM.pm line 109. > > Makefile:445: recipe for target 'manifypods' failed > > make[1]: *** [manifypods] Error 2 > > > > The scripts pod2man, pod2text, podchecker, podselect, and pod2usage > > all use > > Getopt-Long and since they are all part of nonxs modules this needs > > to > > be > > added here to prevent these build races. > > > >
> > Can you explain a bit more as to what you think is the sequence of > events involved in the race condition? > > Thank you very much.
If cpan/podlators starts building before cpan/Getopt-Long has finished it will attempt to run the pod2man, which contains "require Getopt::Long;" before Getopt-Long has been published to the lib dir. Therefore when it runs it searches its path and finds no GetOpt-Long and fails. This can naturally happen when running a very parallel make invocation (-j 32) but it is still fairly rare. It can be artificially induced by reordering the nonxs_ext variable such that cpan/podlators/pm_to_blib is before cpan/Getopt-Long/pm_to_blib (this needs to be done in a fresh workspace right after the Configure is run).
RT-Send-CC: ddedrick [...] lexmark.com, perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 3.4k
On Fri, 27 Oct 2017 18:40:10 -0700, dan.dedrick@gmail.com wrote: Show quoted text
> On Fri, 27 Oct 2017 17:28:22 -0700, jkeenan wrote:
> > On Fri, 27 Oct 2017 00:47:58 GMT, dan.dedrick@gmail.com wrote:
> > > This is a bug report for perl from dan.dedrick@gmail.com, > > > generated with the help of perlbug 1.40 running under perl 5.27.6. > > > > > > > > > ----------------------------------------------------------------- > > > [Please describe your issue here] > > > > > > When building perl with highly parallel options (e.g. -j 32) on a > > > machine > > > with many cores (e.g. 32) ocassionaly it fails with the following > > > type > > > of error. > > > > > > make[1]: Entering directory '.../cpan/podlators' > > > Can't locate Getopt/Long.pm in @INC (you may need to install the > > > Getopt::Long module) (@INC contains: .../cpan/AutoLoader/lib > > > .../dist/Carp/lib .../dist/PathTools .../dist/PathTools/lib > > > .../cpan/ExtUtils-Install/lib .../cpan/ExtUtils-MakeMaker/lib > > > .../cpan/ExtUtils-Manifest/lib .../cpan/File-Path/lib .../ext/re > > > .../dist/Term-ReadLine/lib .../dist/Exporter/lib .../ext/File- > > > Find/lib > > > .../cpan/Text-Tabs/lib .../dist/constant/lib .../cpan/version/lib > > > .../lib ../../lib .) at .../cpan/ExtUtils- > > > MakeMaker/lib/ExtUtils/Command/MM.pm line 109. > > > Makefile:445: recipe for target 'manifypods' failed > > > make[1]: *** [manifypods] Error 2 > > > > > > The scripts pod2man, pod2text, podchecker, podselect, and pod2usage > > > all use > > > Getopt-Long and since they are all part of nonxs modules this needs > > > to > > > be > > > added here to prevent these build races. > > > > > >
> > > > Can you explain a bit more as to what you think is the sequence of > > events involved in the race condition? > > > > Thank you very much.
> > If cpan/podlators starts building before cpan/Getopt-Long has finished > it will attempt to run the pod2man, which contains "require > Getopt::Long;" before Getopt-Long has been published to the lib dir. > Therefore when it runs it searches its path and finds no GetOpt-Long > and fails. This can naturally happen when running a very parallel make > invocation (-j 32) but it is still fairly rare. It can be artificially > induced by reordering the nonxs_ext variable such that > cpan/podlators/pm_to_blib is before cpan/Getopt-Long/pm_to_blib (this > needs to be done in a fresh workspace right after the Configure is > run).
I have also seen this problem for perl 5.26.1 on a parallel build with -j33 on a machine with 32 cores. The issue can be reproduced easily by adding a delay to the handling of Getopt-Long after the Makefile has been generated by the configure step. --- output/build/host-perl-5.26.1/Makefile 2018-06-06 15:18:34.905011014 +0200 +++ /tmp/Makefile.host 2018-06-06 15:19:01.200009936 +0200 @@ -583,6 +583,11 @@ $(MINIPERL) make_ext.pl $@ $(MAKE_EXT_ARGS) MAKE="$(MAKE)" LIBPERL_A=$(LIBPERL) LINKTYPE=static $(STATIC_LDFLAGS) n_dummy $(nonxs_ext): $(MINIPERL_EXE) lib/buildcustomize.pl preplibrary FORCE + @echo "------- makefile -$@-" + @if [ "$@" = "cpan/Getopt-Long/pm_to_blib" ]; then \ + echo "---- getopt" ; \ + sleep 60 ; \ + fi $(MINIPERL) make_ext.pl $@ $(MAKE_EXT_ARGS) MAKE="$(MAKE)" LIBPERL_A=$(LIBPERL) ext/Pod-Functions/pm_to_blib: cpan/Pod-Simple/pm_to_blib cpan/Pod-Escapes/pm_to_blib pod/perlfunc.pod With this above reproduction hack, the problem can even be reproduced with '-j5'. Using the patch provided by Dan Dedrick, the problem is solved. Could this patch please be applied to official releases? Thanks, Thomas
To: James E Keenan via RT <perlbug-followup [...] perl.org>
Date: Wed, 14 Nov 2018 11:53:51 +0000
Subject: Re: [perl #132360] [PATCH] Fix missing build dependency for pods
CC: ddedrick [...] lexmark.com, Perl5 Porters <perl5-porters [...] perl.org>
From: Aaron Crane <arc [...] cpan.org>
Download (untitled) / with headers
text/plain 382b
Thomas De Schampheleire via RT <perlbug-followup@perl.org> wrote: Show quoted text
> Using the patch provided by Dan Dedrick, the problem is solved. > > Could this patch please be applied to official releases?
Thanks, applied (with an extra comment, and Dan Dedrick added to the AUTHORS file) as bb6e76f0bf2e379480796380f1dc705e6cfffbff Welcome to the roster of Perl authors, Dan! -- Aaron Crane
Download (untitled) / with headers
text/plain 313b
Thank you for filing this report. You have helped make Perl better. With the release today of Perl 5.30.0, this and 160 other issues have been resolved. Perl 5.30.0 may be downloaded via: https://metacpan.org/release/XSAWYERX/perl-5.30.0 If you find that the problem persists, feel free to reopen this ticket.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org