Skip Menu |
Report information
Id: 133070
Status: open
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: bulk88 <bulk88 [at] hotmail.com>
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: core
Perl Version: 5.27.9
Fixed In: (no value)



Date: Wed, 4 Apr 2018 13:27:55 +0000
From: bulk 88 <bulk88 [...] hotmail.com>
To: "perlbug [...] perl.org" <perlbug [...] perl.org>
Subject: [PATCH] fix parallel ALL_STATIC Win32 build
Download (untitled) / with headers
text/plain 6.9k
This is a bug report for perl from bulk88@hotmail.com, generated with the help of perlbug 1.41 running under perl 5.27.9. ----------------------------------------------------------------- [Please describe your issue here] See attached patch. This is intended for 5.28. Its a follow on to problems I discussed in https://rt.perl.org/Ticket/Display.html?id=132992 . It fixes not being able to build ALL_STATIC Extensions_static target individually and relying on a race/adjacent targets to randomly finish first to make Extensions_static not fail. example of failed static Normalize compile because the mktables data wasn't generated yet ---------------------------- cl -c -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DWIN32_NO_R EGISTRY -DUSE_PERLIO -D_USE_32BIT_TIME_T -DPERLDLL -O1 -MD -Zi -DNDEBUG -GL -D VERSION=\"1.25\" -DXS_VERSION=\"1.25\" "-I..\..\lib\CORE" Collate.c Collate.c C:\perl521\srcnew\miniperl.exe "-I..\..\lib" -MExtUtils::Command -e rm_f -- "..\ ..\lib\auto\Unicode\Collate\Collate.lib" lib -ltcg -out:..\..\lib\auto\Unicode\Collate\Collate.lib Collate.obj Microsoft (R) Library Manager Version 7.10.6030 Copyright (C) Microsoft Corporation. All rights reserved. C:\perl521\srcnew\miniperl.exe "-I..\..\lib" -MExtUtils::Command -e chmod -- 755 ..\..\lib\auto\Unicode\Collate\Collate.lib gmake[1]: Leaving directory 'C:/perl521/srcnew/cpan/Unicode-Collate' Making header files for XS... Unicode::Normalize, mkheader: CombiningClass.pl not found at ./mkheader line 67. require ./mkheader called at Makefile.PL line 11 Unsuccessful Makefile.PL(dist/Unicode-Normalize): code=512 at ..\make_ext.pl lin e 518. GNUmakefile:1584: recipe for target 'Extensions_static' failed gmake: *** [Extensions_static] Error 2 ---------------------------- example of static extension failing because lib.pm not finished yet by a race ---------------------------- gmake[1]: Leaving directory 'C:/perl521/srcnew/ext/B' Can't locate lib.pm in @INC (you may need to install the lib module) (@INC conta ins: C:\perl521\srcnew\lib C:\perl521\srcnew\cpan\AutoLoader\lib C:\perl521\srcn ew\dist\Carp\lib C:\perl521\srcnew\dist\PathTools C:\perl521\srcnew\dist\PathToo ls\lib C:\perl521\srcnew\cpan\ExtUtils-Install\lib C:\perl521\srcnew\cpan\ExtUti ls-MakeMaker\lib C:\perl521\srcnew\cpan\ExtUtils-Manifest\lib C:\perl521\srcnew\ cpan\File-Path\lib C:\perl521\srcnew\ext\re C:\perl521\srcnew\dist\Term-ReadLine \lib C:\perl521\srcnew\dist\Exporter\lib C:\perl521\srcnew\ext\File-Find\lib C:\ perl521\srcnew\cpan\Text-Tabs\lib C:\perl521\srcnew\dist\constant\lib C:\perl521 \srcnew\cpan\version\lib C:\perl521\srcnew\cpan\Text-ParseWords\lib C:\perl521\s rcnew\dist\ExtUtils-ParseXS\lib C:\perl521\srcnew\cpan\Getopt-Long\lib C:\perl52 1\srcnew\cpan\parent\lib C:\perl521\srcnew\cpan\ExtUtils-Constant\lib .) at Make file.PL line 6. BEGIN failed--compilation aborted at Makefile.PL line 6. Unsuccessful Makefile.PL(cpan/Compress-Raw-Bzip2): code=512 at ..\make_ext.pl li ne 518. GNUmakefile:1584: recipe for target 'Extensions_static' failed gmake: *** [Extensions_static] Error 2 ---------------------------- [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=low --- Site configuration information for perl 5.27.9: Configured by Administrator at Tue Jan 30 20:34:30 2018. Summary of my perl5 (revision 5 version 27 subversion 9) configuration: Platform: osname=MSWin32 osvers=5.2.3790 archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended useposix=true d_sigaction=undef useithreads=define usemultiplicity=define use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cl' ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32 -D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DWIN32_NO_REGISTRY' optimize='-O1 -MD -Zi -DNDEBUG -GL' cppflags='-DWIN32' ccversion='15.00.30729.01' gccversion='' gccosandvers='' intsize=4 longsize=4 ptrsize=4 doublesize=8 byteorder=1234 doublekind=3 d_longlong=undef longlongsize=8 d_longdbl=define longdblsize=8 longdblkind=0 ivtype='long' ivsize=4 nvtype='double' nvsize=8 Off_t='__int64' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='link' ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86' libpth="C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\lib" libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib libc=msvcrt.lib so=dll useshrplib=true libperl=perl527.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs dlext=dll d_dlsymun=undef ccdlflags=' ' cccdlflags=' ' lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86' --- @INC for perl 5.27.9: lib C:/p527/srcnew/lib --- Environment for perl 5.27.9: CYGWIN=tty HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH=/usr/lib/x86:/usr/X11R6/lib LOGDIR (unset) PATH=C:\WINDOWS\system32;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN;C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin;C:\Perl\bin;C:\WINDOWS;C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE;C:\Program Files (x86)\Git\bin;C:\sp3220\c\bin; PERL_BADLANG (unset) SHELL (unset)

Message body is not shown because sender requested not to inline it.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.1k
On Thu, 05 Apr 2018 20:34:20 -0700, bulk88 wrote: Show quoted text
> This is a bug report for perl from bulk88@hotmail.com, > generated with the help of perlbug 1.41 running under perl 5.27.9. > > > ----------------------------------------------------------------- > [Please describe your issue here] > > See attached patch. This is intended for 5.28. Its a follow on to > problems I discussed in > https://rt.perl.org/Ticket/Display.html?id=132992 . It fixes not being > able to build ALL_STATIC Extensions_static target individually and > relying on a race/adjacent targets to randomly finish first to make > Extensions_static not fail.
Revised patch attach, the above one failed a regen.t test because I removed the Amiga dir from dmake mkf, thinking that will never be built on Win32 bc I was thinking the original author of gmake mkf also thought that and thats why he left it out. It turns out there a lib_cleanup.pl that was checking the dir clean list in the win32 makefiles but it didnt know about gmake makefile. So the test failed. To make things look more proper, add the amiga dir to the gmake makefile and start having that file be chked by lib_cleanup.pl like the other 2 mkfs are checked. -- bulk88 ~ bulk88 at hotmail.com
Subject: 0001-fix-parallel-ALL_STATIC-Win32-build.patch
From 63a433af1aa5ea2c6f9f7df33d7278ba15683bee Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Fri, 6 Apr 2018 09:12:48 -0400 Subject: [PATCH] fix parallel ALL_STATIC Win32 build Normalize depends on mktables's output. Mktables is a very long serial CPU intensive build product. No other modules except Normalize depend on mktables. Normalize XS module must be split out from all other XS modules to fill CPU cores with parallel work, so that the 2 longest running tagets, Mktables and all XS modules, run in parallel instead of 1 after another. Since Normalize was hardcoded as a dynamic module, in an ALL_STATIC build, Normalize was build twice, once as a static in the big Extensions_static target, and once as a dynamic module, a race happened if a parallel build was done, and if the Extensions_static target got around to trying to build Normalize static, before by chance Normalize dyn was build, Normalize static failed. This can be demostrated by cleaning the src tree and building each target individually explicitly. Fix the race by creating a Normalize static and dynamic target and dropout the deps depending on if its ALL_STATIC or not. Also the now "big" Extensions_static target when doing an ALL_STATIC build also failed because the general XS modules that need lib.pm to build didn't have lib.pm available when their Makefile.PL'es were run. For example cpan/Compress-Raw-Bzip2 needs lib.pm to build. So separate out lib.pm from Dynaloader target so both dyn extensions and static exts have lib.pm available. Technically, static exts dont need lib.pm unless its ALL_STATIC, since for dyn win32 perl Win32CORE is the only static module, but lib.pm is relativly fast and easy and low dep to build so build it anyways for static ext target in non-ALL_STATIC mode rather than special case the dep between ALL_STATIC and non-ALL_STATIC. -re.pm doesn't need lib.pm -normalize doesn't need lib.pm Reverts "Fix static builds with MinGW" commit 9999704e7ac and adds fixes. This is a follow on to [perl #132992]. To reduce the diff-ness between the dmake and gmake mkfs. Make the distclean identical. For dmkf, dont delete Storable.pm twice. And for gmfk add Amiga dir deletion since lib_cleanup.pl requires it for dmkf but the original day 1 of gmkf in commit 342634f3c8 "kmx's original GNUmakefile" never had the Amiga line, but that day 1 commit also never added GNUmakefile to lib_cleanup.pl the way makefile.mk was already added and chked for its dir clean list. So add GNUmakefile to be chked by lib_cleanup.pl the way the other 2 win32 mkfs are checked. In gmfk move Storable.pm line to match dmfk which is more psuedo-alpha sorted. Also fix the Test vs Test2 typo. Some more text is available in the ticket associated with this patch. --- regen/lib_cleanup.pl | 2 +- t/porting/regen.t | 2 +- win32/GNUmakefile | 31 +++++++++++++++++++++---------- win32/makefile.mk | 27 ++++++++++++++++++--------- 4 files changed, 41 insertions(+), 21 deletions(-) diff --git a/regen/lib_cleanup.pl b/regen/lib_cleanup.pl index d381269..5c5c4e2 100644 --- a/regen/lib_cleanup.pl +++ b/regen/lib_cleanup.pl @@ -154,7 +154,7 @@ sub edit_win32_makefile { } process('Makefile.SH', 'Makefile.SH', \&edit_makefile_SH, $TAP && '', $Verbose); -foreach ('win32/Makefile', 'win32/makefile.mk') { +foreach ('win32/Makefile', 'win32/makefile.mk', 'win32/GNUmakefile') { process($_, $_, \&edit_win32_makefile, $TAP && '', $Verbose); } diff --git a/t/porting/regen.t b/t/porting/regen.t index f296626..26639a9 100644 --- a/t/porting/regen.t +++ b/t/porting/regen.t @@ -26,7 +26,7 @@ if ( $Config{usecrosscompile} ) { skip_all( "Not all files are available during cross-compilation" ); } -my $tests = 26; # I can't see a clean way to calculate this automatically. +my $tests = 27; # I can't see a clean way to calculate this automatically. my %skip = ("regen_perly.pl" => [qw(perly.act perly.h perly.tab)], "regen/keywords.pl" => [qw(keywords.c keywords.h)], diff --git a/win32/GNUmakefile b/win32/GNUmakefile index 877c16b..82bf593 100644 --- a/win32/GNUmakefile +++ b/win32/GNUmakefile @@ -1110,11 +1110,13 @@ ifeq ($(ALL_STATIC),define) # some exclusions, unfortunately, until fixed: # - MakeMaker isn't capable enough for SDBM_File (small bug) STATIC_EXT = * !SDBM_File +NORMALIZE_STATIC = Normalize_static else # specify static extensions here, for example: # (be sure to include Win32CORE to load Win32 on demand) #STATIC_EXT = Win32CORE Cwd Compress/Raw/Zlib STATIC_EXT = Win32CORE +NORMALIZE_DYN = Normalize_dyn endif DYNALOADER = ..\DynaLoader$(o) @@ -1572,25 +1574,33 @@ endif # DynaLoader.pm, so this will have to do #most of deps of this target are in DYNALOADER and therefore omitted here -Extensions : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) MakePPPort +Extensions : $(PERLDEP) $(DYNALOADER) Extension_lib $(GLOBEXE) MakePPPort $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic !Unicode/Normalize -Extensions_normalize : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES) +Normalize_static : $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) $(UNIDATAFILES) + $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Unicode/Normalize + +Normalize_dyn : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +Unicode/Normalize Extensions_reonly : $(PERLDEP) $(DYNALOADER) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re -Extensions_static : ..\make_ext.pl list_static_libs.pl $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) MakePPPort - $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Exts_static_general : ..\make_ext.pl $(CONFIGPM) Extension_lib $(GLOBEXE) $(HAVE_COREDIR) MakePPPort + $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static !Unicode/Normalize + +Extensions_static : list_static_libs.pl Exts_static_general $(NORMALIZE_STATIC) $(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static Extensions_nonxs : ..\make_ext.pl ..\pod\perlfunc.pod $(CONFIGPM) $(GLOBEXE) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs +Extension_lib : ..\make_ext.pl $(CONFIGPM) + $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) lib + #lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building $(DYNALOADER) : ..\make_ext.pl $(CONFIGPM) $(HAVE_COREDIR) - $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib + $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader Extensions_clean : -if exist $(MINIPERL) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --all --target=clean @@ -1612,9 +1622,9 @@ PostExt : ..\lib\Storable\Limit.pm # be running in parallel like UNIDATAFILES, this target a placeholder for the # future ifeq ($(PERLSTATIC),static) -rebasePE : Extensions_normalize Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC) +rebasePE : Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC) else -rebasePE : Extensions $(PERLDLL) Extensions_normalize $(PERLEXE) +rebasePE : Extensions $(PERLDLL) $(NORMALIZE_DYN) $(PERLEXE) endif $(NOOP) @@ -1670,7 +1680,7 @@ utils: $(HAVEMINIPERL) ..\utils\Makefile $(MINIPERL) -I..\lib ..\autodoc.pl .. $(MINIPERL) -I..\lib ..\pod\perlmodlib.PL -q .. -..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs Extensions_normalize utils +..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs $(NORMALIZE_DYN) utils $(PERLEXE) -f ..\pod\buildtoc -q # Note that the pod cleanup in this next section is parsed (and regenerated @@ -1687,11 +1697,11 @@ distclean: realclean -del /f $(LIBDIR)\SDBM_File.pm $(LIBDIR)\Socket.pm $(LIBDIR)\POSIX.pm -del /f $(LIBDIR)\B.pm $(LIBDIR)\O.pm $(LIBDIR)\re.pm -del /f $(LIBDIR)\File\Glob.pm - -del /f $(LIBDIR)\Storable.pm $(LIBDIR)\Storable\Limit.pm -del /f $(LIBDIR)\Sys\Hostname.pm -del /f $(LIBDIR)\Time\HiRes.pm -del /f $(LIBDIR)\Unicode\Normalize.pm -del /f $(LIBDIR)\Math\BigInt\FastCalc.pm + -del /f $(LIBDIR)\Storable.pm $(LIBDIR)\Storable\Limit.pm -del /f $(LIBDIR)\Win32.pm -del /f $(LIBDIR)\Win32CORE.pm -del /f $(LIBDIR)\Win32API\File.pm @@ -1699,6 +1709,7 @@ distclean: realclean -del /f $(LIBDIR)\buildcustomize.pl -del /f $(DISTDIR)\XSLoader\XSLoader.pm -del /f *.def *.map + -if exist $(LIBDIR)\Amiga rmdir /s /q $(LIBDIR)\Amiga -if exist $(LIBDIR)\App rmdir /s /q $(LIBDIR)\App -if exist $(LIBDIR)\Archive rmdir /s /q $(LIBDIR)\Archive -if exist $(LIBDIR)\Attribute rmdir /s /q $(LIBDIR)\Attribute @@ -1750,7 +1761,7 @@ distclean: realclean -if exist $(LIBDIR)\TAP rmdir /s /q $(LIBDIR)\TAP -if exist $(LIBDIR)\Term rmdir /s /q $(LIBDIR)\Term -if exist $(LIBDIR)\Test rmdir /s /q $(LIBDIR)\Test - -if exist $(LIBDIR)\Test2 rmdir /s /q $(LIBDIR)\Test + -if exist $(LIBDIR)\Test2 rmdir /s /q $(LIBDIR)\Test2 -if exist $(LIBDIR)\Text rmdir /s /q $(LIBDIR)\Text -if exist $(LIBDIR)\Thread rmdir /s /q $(LIBDIR)\Thread -if exist $(LIBDIR)\threads rmdir /s /q $(LIBDIR)\threads diff --git a/win32/makefile.mk b/win32/makefile.mk index 2797b8a..265e93b 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -1084,11 +1084,13 @@ SETARGV_OBJ = setargv$(o) # some exclusions, unfortunately, until fixed: # - MakeMaker isn't capable enough for SDBM_File (small bug) STATIC_EXT = * !SDBM_File +NORMALIZE_STATIC = Normalize_static .ELSE # specify static extensions here, for example: # (be sure to include Win32CORE to load Win32 on demand) #STATIC_EXT = Win32CORE Cwd Compress/Raw/Zlib STATIC_EXT = Win32CORE +NORMALIZE_DYN = Normalize_dyn .ENDIF DYNALOADER = ..\DynaLoader$(o) @@ -1513,25 +1515,33 @@ $(PERLEXESTATIC): $(PERLSTATICLIB) $(CONFIGPM) $(PERLEXEST_OBJ) $(PERLEXE_RES) # DynaLoader.pm, so this will have to do #most of deps of this target are in DYNALOADER and therefore omitted here -Extensions : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) MakePPPort +Extensions : $(PERLDEP) $(DYNALOADER) Extension_lib $(GLOBEXE) MakePPPort $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic !Unicode/Normalize -Extensions_normalize : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES) +Normalize_static : $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) $(UNIDATAFILES) + $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Unicode/Normalize + +Normalize_dyn : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +Unicode/Normalize Extensions_reonly : $(PERLDEP) $(DYNALOADER) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re -Extensions_static : ..\make_ext.pl list_static_libs.pl $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) MakePPPort - $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Exts_static_general : ..\make_ext.pl $(CONFIGPM) Extension_lib $(GLOBEXE) $(HAVE_COREDIR) MakePPPort + $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static !Unicode/Normalize + +Extensions_static : list_static_libs.pl Exts_static_general $(NORMALIZE_STATIC) $(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static Extensions_nonxs : ..\make_ext.pl ..\pod\perlfunc.pod $(CONFIGPM) $(GLOBEXE) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs +Extension_lib : ..\make_ext.pl $(CONFIGPM) + $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) lib + #lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building $(DYNALOADER) : ..\make_ext.pl $(CONFIGPM) $(HAVE_COREDIR) - $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib + $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader Extensions_clean : -if exist $(MINIPERL) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --all --target=clean @@ -1543,9 +1553,9 @@ Extensions_realclean : # be running in parallel like UNIDATAFILES, this target a placeholder for the # future .IF "$(PERLSTATIC)"=="static" -rebasePE : Extensions_normalize Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC) +rebasePE : Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC) .ELSE -rebasePE : Extensions $(PERLDLL) Extensions_normalize $(PERLEXE) +rebasePE : Extensions $(PERLDLL) $(NORMALIZE_DYN) $(PERLEXE) .ENDIF $(NOOP) @@ -1613,7 +1623,7 @@ utils: $(HAVEMINIPERL) ..\utils\Makefile $(MINIPERL) -I..\lib ..\autodoc.pl .. $(MINIPERL) -I..\lib ..\pod\perlmodlib.PL -q .. -..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs Extensions_normalize utils +..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs $(NORMALIZE_DYN) utils $(PERLEXE) -f ..\pod\buildtoc -q # Note that the pod cleanup in this next section is parsed (and regenerated @@ -1630,7 +1640,6 @@ distclean: realclean -del /f $(LIBDIR)\SDBM_File.pm $(LIBDIR)\Socket.pm $(LIBDIR)\POSIX.pm -del /f $(LIBDIR)\B.pm $(LIBDIR)\O.pm $(LIBDIR)\re.pm -del /f $(LIBDIR)\File\Glob.pm - -del /f $(LIBDIR)\Storable.pm -del /f $(LIBDIR)\Sys\Hostname.pm -del /f $(LIBDIR)\Time\HiRes.pm -del /f $(LIBDIR)\Unicode\Normalize.pm -- 2.5.0.windows.1
To: bulk88 <bulk88 [...] hotmail.com>
From: "Konovalov, Vadim" <Vadim.Konovalov [...] dell.com>
Date: Fri, 6 Apr 2018 15:09:14 +0000
Subject: RE: [perl #133070] [PATCH] fix parallel ALL_STATIC Win32 build
CC: "perl5-porters [...] perl.org" <perl5-porters [...] perl.org>
Download (untitled) / with headers
text/plain 492b
Hi Daniel Show quoted text
> On Thu, 05 Apr 2018 20:34:20 -0700, bulk88 wrote:
> > This is > > See
> attached patch. This is intended for 5.28. Its a follow on to
> > problems I discussed in > >
... Given that you so much dived into details of this problem, maybe you can also have an idea, why all extensions happen to be stuffed into the libperl527s.a for GCC build?? I've questioned this question here: https://www.nntp.perl.org/group/perl.perl5.porters/2018/03/msg249982.html TIA
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 975b
On Fri, 06 Apr 2018 06:28:22 -0700, bulk88 wrote: Show quoted text
> Revised patch attach, the above one failed a regen.t test because I > removed the Amiga dir from dmake mkf, thinking that will never be > built on Win32 bc I was thinking the original author of gmake mkf also > thought that and thats why he left it out. It turns out there a > lib_cleanup.pl that was checking the dir clean list in the win32 > makefiles but it didnt know about gmake makefile. So the test failed. > To make things look more proper, add the amiga dir to the gmake > makefile and start having that file be chked by lib_cleanup.pl like > the other 2 mkfs are checked.
Thanks, applied as 908f2cb56527d29c9176e478fa7eee0d02a7c77e. There's still a problem if you build with STATIC_EXT=Unicode/Normalize but that might be more difficult to fix. I do wonder if the Win32 build should build miniperl, then use that to build the final config.sh/config.h and generate a Makefile for the full perl from there. Tony
RT-Send-CC: perl5-porters [...] perl.org
On Sun, 08 Apr 2018 21:03:16 -0700, tonyc wrote: Show quoted text
> Thanks, applied as 908f2cb56527d29c9176e478fa7eee0d02a7c77e. > > There's still a problem if you build with STATIC_EXT=Unicode/Normalize > but that might be more difficult to fix. > > I do wonder if the Win32 build should build miniperl, then use that to > build the final config.sh/config.h and generate a Makefile for the > full perl from there. > > Tony
What will generate the makefile? Makefile.SH would have to be rewritten in PP. IMO its pointless because Windows doesn't come with a "Unix shell" compiler and M4 compiler out of the box and switching to Visual C with Autoconf+Bash (I've built such FOSS projects before) is insane since not even Unix perl uses M4 or autoconf. I know you wrote one patch 2 yrs ago that generates an auxiliary extensions only makefile for parallel extension building so each extension can be parallelzed to infinity cores instead of the 4 "categories" of extensions building side by side they way they do now and doesn't saturate all cores. -- bulk88 ~ bulk88 at hotmail.com
Date: Tue, 17 Apr 2018 10:29:21 +1000
Subject: Re: [perl #133070] [PATCH] fix parallel ALL_STATIC Win32 build
CC: perl5-porters [...] perl.org
From: Tony Cook <tony [...] develop-help.com>
To: bulk88 via RT <perlbug-followup [...] perl.org>
Download (untitled) / with headers
text/plain 1.6k
On Mon, Apr 16, 2018 at 10:54:02AM -0700, bulk88 via RT wrote: Show quoted text
> On Sun, 08 Apr 2018 21:03:16 -0700, tonyc wrote:
> > Thanks, applied as 908f2cb56527d29c9176e478fa7eee0d02a7c77e. > > > > There's still a problem if you build with STATIC_EXT=Unicode/Normalize > > but that might be more difficult to fix. > > > > I do wonder if the Win32 build should build miniperl, then use that to > > build the final config.sh/config.h and generate a Makefile for the > > full perl from there. > > > > Tony
> > What will generate the makefile? Makefile.SH would have to be rewritten in PP.
Well, it wouldn't match Makefile.SH anyway, since it doesn't need to build miniperl. Show quoted text
> IMO its pointless because Windows doesn't come with a "Unix shell" compiler and M4 compiler out of the box and switching to Visual C with Autoconf+Bash (I've built such FOSS projects before) is insane since not even Unix perl uses M4 or autoconf. I know you wrote one patch 2 yrs ago that generates an auxiliary extensions only makefile for parallel extension building so each extension can be parallelzed to infinity cores instead of the 4 "categories" of extensions building side by side they way they do now and doesn't saturate all cores.
It would at least fix building with STATIC_EXT=Unicode/Normalize. It may be possible to improve parallel building over my extensions.mk, since with that the main makefile can't depend on individual extensions, or it will risk the sub-makefile and the main makefile trying to build the same extension at the same time, while if they're in the same file a target can depend on the .dll or $(PERL_EXE) as needed. At this point it's just brainstorming which has at least inspired a little discussion. Tony


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org