[PATCH] fix parallel ALL_STATIC Win32 build #16492
Comments
From @bulk88Created by @bulk88See attached patch. This is intended for 5.28. Its a follow on to example of failed static Normalize compile because the mktables data C:\perl521\srcnew\miniperl.exe "-I..\..\lib" -MExtUtils::Command -e require ./mkheader called at Makefile.PL line 11 example of static extension failing because lib.pm not finished yet by a ---------------------------- Perl Info |
From @bulk880001-fix-parallel-ALL_STATIC-Win32-build.patchFrom 6259ddde8c36041fc830030571c8ad7abe524b78 Mon Sep 17 00:00:00 2001
From: Daniel Dragan <bulk88@hotmail.com>
Date: Wed, 4 Apr 2018 09:20:38 -0400
Subject: [PATCH] fix parallel ALL_STATIC Win32 build
Normalize depends on mktables's output. Mktables is a very long serial CPU
intensive build product. No other modules except Normalize depend on
mktables. Normalize XS module must be split out from all other XS modules
to fill CPU cores with parallel work, so that the 2 longest running
tagets, Mktables and all XS modules, run in parallel instead of 1 after
another. Since Normalize was hardcoded as a dynamic module, in an
ALL_STATIC build, Normalize was build twice, once as a static in the
big Extensions_static target, and once as a dynamic module, a race
happened if a parallel build was done, and if the Extensions_static target
got around to trying to build Normalize static, before by chance Normalize
dyn was build, Normalize static failed. This can be demostrated by cleaning
the src tree and building each target individually explicitly. Fix the race
by creating a Normalize static and dynamic target and dropout the deps
depending on if its ALL_STATIC or not. Also the now "big" Extensions_static
target when doing an ALL_STATIC build also failed because the general XS
modules that need lib.pm to build didn't have lib.pm available when their
Makefile.PL'es were run. For example cpan/Compress-Raw-Bzip2 needs lib.pm
to build. So separate out lib.pm from Dynaloader target so both dyn
extensions and static exts have lib.pm available. Technically, static exts
dont need lib.pm unless its ALL_STATIC, since for dyn win32 perl Win32CORE
is the only static module, but lib.pm is relativly fast and easy and low
dep to build so build it anyways for static ext target in non-ALL_STATIC
mode rather than special case the dep between ALL_STATIC and
non-ALL_STATIC.
-re.pm doesn't need lib.pm
-normalize doesn't need lib.pm
Reverts "Fix static builds with MinGW" commit 9999704e7ac and adds fixes.
This is a follow on to [perl #132992].
To reduce the diff-ness between the dmake and gmake mkfs. Make the
distclean identical. For dmkf, dont delete Storable.pm twice. And for dmfk
remove Amiga dir deletion since the gmkf never had that line since day 1
of gmkf in commit 342634f3c8 "kmx's original GNUmakefile". In gmfk move
Storable.pm line to match dmfk which is more psuedo-alpha sorted. Also fix
the Test vs Test2 typo.
Some more text is available in the ticket associated with this patch.
---
win32/GNUmakefile | 30 ++++++++++++++++++++----------
win32/makefile.mk | 28 ++++++++++++++++++----------
2 files changed, 38 insertions(+), 20 deletions(-)
diff --git a/win32/GNUmakefile b/win32/GNUmakefile
index 5e70d04..bcb2e85 100644
--- a/win32/GNUmakefile
+++ b/win32/GNUmakefile
@@ -1109,11 +1109,13 @@ ifeq ($(ALL_STATIC),define)
# some exclusions, unfortunately, until fixed:
# - MakeMaker isn't capable enough for SDBM_File (small bug)
STATIC_EXT = * !SDBM_File
+NORMALIZE_STATIC = Normalize_static
else
# specify static extensions here, for example:
# (be sure to include Win32CORE to load Win32 on demand)
#STATIC_EXT = Win32CORE Cwd Compress/Raw/Zlib
STATIC_EXT = Win32CORE
+NORMALIZE_DYN = Normalize_dyn
endif
DYNALOADER = ..\DynaLoader$(o)
@@ -1571,25 +1573,33 @@ endif
# DynaLoader.pm, so this will have to do
#most of deps of this target are in DYNALOADER and therefore omitted here
-Extensions : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) MakePPPort
+Extensions : $(PERLDEP) $(DYNALOADER) Extension_lib $(GLOBEXE) MakePPPort
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic !Unicode/Normalize
-Extensions_normalize : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
+Normalize_static : $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) $(UNIDATAFILES)
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Unicode/Normalize
+
+Normalize_dyn : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +Unicode/Normalize
Extensions_reonly : $(PERLDEP) $(DYNALOADER)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re
-Extensions_static : ..\make_ext.pl list_static_libs.pl $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
- $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static
+Exts_static_general : ..\make_ext.pl $(CONFIGPM) Extension_lib $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static !Unicode/Normalize
+
+Extensions_static : list_static_libs.pl Exts_static_general $(NORMALIZE_STATIC)
$(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static
Extensions_nonxs : ..\make_ext.pl ..\pod\perlfunc.pod $(CONFIGPM) $(GLOBEXE)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs
+Extension_lib : ..\make_ext.pl $(CONFIGPM)
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) lib
+
#lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building
$(DYNALOADER) : ..\make_ext.pl $(CONFIGPM) $(HAVE_COREDIR)
- $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader
Extensions_clean :
-if exist $(MINIPERL) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --all --target=clean
@@ -1611,9 +1621,9 @@ PostExt : ..\lib\Storable\Limit.pm
# be running in parallel like UNIDATAFILES, this target a placeholder for the
# future
ifeq ($(PERLSTATIC),static)
-rebasePE : Extensions_normalize Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
+rebasePE : Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
else
-rebasePE : Extensions $(PERLDLL) Extensions_normalize $(PERLEXE)
+rebasePE : Extensions $(PERLDLL) $(NORMALIZE_DYN) $(PERLEXE)
endif
$(NOOP)
@@ -1669,7 +1679,7 @@ utils: $(HAVEMINIPERL) ..\utils\Makefile
$(MINIPERL) -I..\lib ..\autodoc.pl ..
$(MINIPERL) -I..\lib ..\pod\perlmodlib.PL -q ..
-..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs Extensions_normalize utils
+..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs $(NORMALIZE_DYN) utils
$(PERLEXE) -f ..\pod\buildtoc -q
# Note that the pod cleanup in this next section is parsed (and regenerated
@@ -1686,11 +1696,11 @@ distclean: realclean
-del /f $(LIBDIR)\SDBM_File.pm $(LIBDIR)\Socket.pm $(LIBDIR)\POSIX.pm
-del /f $(LIBDIR)\B.pm $(LIBDIR)\O.pm $(LIBDIR)\re.pm
-del /f $(LIBDIR)\File\Glob.pm
- -del /f $(LIBDIR)\Storable.pm $(LIBDIR)\Storable\Limit.pm
-del /f $(LIBDIR)\Sys\Hostname.pm
-del /f $(LIBDIR)\Time\HiRes.pm
-del /f $(LIBDIR)\Unicode\Normalize.pm
-del /f $(LIBDIR)\Math\BigInt\FastCalc.pm
+ -del /f $(LIBDIR)\Storable.pm $(LIBDIR)\Storable\Limit.pm
-del /f $(LIBDIR)\Win32.pm
-del /f $(LIBDIR)\Win32CORE.pm
-del /f $(LIBDIR)\Win32API\File.pm
@@ -1749,7 +1759,7 @@ distclean: realclean
-if exist $(LIBDIR)\TAP rmdir /s /q $(LIBDIR)\TAP
-if exist $(LIBDIR)\Term rmdir /s /q $(LIBDIR)\Term
-if exist $(LIBDIR)\Test rmdir /s /q $(LIBDIR)\Test
- -if exist $(LIBDIR)\Test2 rmdir /s /q $(LIBDIR)\Test
+ -if exist $(LIBDIR)\Test2 rmdir /s /q $(LIBDIR)\Test2
-if exist $(LIBDIR)\Text rmdir /s /q $(LIBDIR)\Text
-if exist $(LIBDIR)\Thread rmdir /s /q $(LIBDIR)\Thread
-if exist $(LIBDIR)\threads rmdir /s /q $(LIBDIR)\threads
diff --git a/win32/makefile.mk b/win32/makefile.mk
index ff82a18..acb0418 100644
--- a/win32/makefile.mk
+++ b/win32/makefile.mk
@@ -1083,11 +1083,13 @@ SETARGV_OBJ = setargv$(o)
# some exclusions, unfortunately, until fixed:
# - MakeMaker isn't capable enough for SDBM_File (small bug)
STATIC_EXT = * !SDBM_File
+NORMALIZE_STATIC = Normalize_static
.ELSE
# specify static extensions here, for example:
# (be sure to include Win32CORE to load Win32 on demand)
#STATIC_EXT = Win32CORE Cwd Compress/Raw/Zlib
STATIC_EXT = Win32CORE
+NORMALIZE_DYN = Normalize_dyn
.ENDIF
DYNALOADER = ..\DynaLoader$(o)
@@ -1512,25 +1514,33 @@ $(PERLEXESTATIC): $(PERLSTATICLIB) $(CONFIGPM) $(PERLEXEST_OBJ) $(PERLEXE_RES)
# DynaLoader.pm, so this will have to do
#most of deps of this target are in DYNALOADER and therefore omitted here
-Extensions : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) MakePPPort
+Extensions : $(PERLDEP) $(DYNALOADER) Extension_lib $(GLOBEXE) MakePPPort
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic !Unicode/Normalize
-Extensions_normalize : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
+Normalize_static : $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) $(UNIDATAFILES)
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Unicode/Normalize
+
+Normalize_dyn : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +Unicode/Normalize
Extensions_reonly : $(PERLDEP) $(DYNALOADER)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re
-Extensions_static : ..\make_ext.pl list_static_libs.pl $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
- $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static
+Exts_static_general : ..\make_ext.pl $(CONFIGPM) Extension_lib $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static !Unicode/Normalize
+
+Extensions_static : list_static_libs.pl Exts_static_general $(NORMALIZE_STATIC)
$(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static
Extensions_nonxs : ..\make_ext.pl ..\pod\perlfunc.pod $(CONFIGPM) $(GLOBEXE)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs
+Extension_lib : ..\make_ext.pl $(CONFIGPM)
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) lib
+
#lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building
$(DYNALOADER) : ..\make_ext.pl $(CONFIGPM) $(HAVE_COREDIR)
- $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader
Extensions_clean :
-if exist $(MINIPERL) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --all --target=clean
@@ -1542,9 +1552,9 @@ Extensions_realclean :
# be running in parallel like UNIDATAFILES, this target a placeholder for the
# future
.IF "$(PERLSTATIC)"=="static"
-rebasePE : Extensions_normalize Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
+rebasePE : Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
.ELSE
-rebasePE : Extensions $(PERLDLL) Extensions_normalize $(PERLEXE)
+rebasePE : Extensions $(PERLDLL) $(NORMALIZE_DYN) $(PERLEXE)
.ENDIF
$(NOOP)
@@ -1612,7 +1622,7 @@ utils: $(HAVEMINIPERL) ..\utils\Makefile
$(MINIPERL) -I..\lib ..\autodoc.pl ..
$(MINIPERL) -I..\lib ..\pod\perlmodlib.PL -q ..
-..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs Extensions_normalize utils
+..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs $(NORMALIZE_DYN) utils
$(PERLEXE) -f ..\pod\buildtoc -q
# Note that the pod cleanup in this next section is parsed (and regenerated
@@ -1629,7 +1639,6 @@ distclean: realclean
-del /f $(LIBDIR)\SDBM_File.pm $(LIBDIR)\Socket.pm $(LIBDIR)\POSIX.pm
-del /f $(LIBDIR)\B.pm $(LIBDIR)\O.pm $(LIBDIR)\re.pm
-del /f $(LIBDIR)\File\Glob.pm
- -del /f $(LIBDIR)\Storable.pm
-del /f $(LIBDIR)\Sys\Hostname.pm
-del /f $(LIBDIR)\Time\HiRes.pm
-del /f $(LIBDIR)\Unicode\Normalize.pm
@@ -1642,7 +1651,6 @@ distclean: realclean
-del /f $(LIBDIR)\buildcustomize.pl
-del /f $(DISTDIR)\XSLoader\XSLoader.pm
-del /f *.def *.map
- -if exist $(LIBDIR)\Amiga rmdir /s /q $(LIBDIR)\Amiga
-if exist $(LIBDIR)\App rmdir /s /q $(LIBDIR)\App
-if exist $(LIBDIR)\Archive rmdir /s /q $(LIBDIR)\Archive
-if exist $(LIBDIR)\Attribute rmdir /s /q $(LIBDIR)\Attribute
--
2.5.0.windows.1
|
From @bulk88On Thu, 05 Apr 2018 20:34:20 -0700, bulk88 wrote:
Revised patch attach, the above one failed a regen.t test because I removed the Amiga dir from dmake mkf, thinking that will never be built on Win32 bc I was thinking the original author of gmake mkf also thought that and thats why he left it out. It turns out there a lib_cleanup.pl that was checking the dir clean list in the win32 makefiles but it didnt know about gmake makefile. So the test failed. To make things look more proper, add the amiga dir to the gmake makefile and start having that file be chked by lib_cleanup.pl like the other 2 mkfs are checked. -- |
From @bulk880001-fix-parallel-ALL_STATIC-Win32-build.patchFrom 63a433af1aa5ea2c6f9f7df33d7278ba15683bee Mon Sep 17 00:00:00 2001
From: Daniel Dragan <bulk88@hotmail.com>
Date: Fri, 6 Apr 2018 09:12:48 -0400
Subject: [PATCH] fix parallel ALL_STATIC Win32 build
Normalize depends on mktables's output. Mktables is a very long serial CPU
intensive build product. No other modules except Normalize depend on
mktables. Normalize XS module must be split out from all other XS modules
to fill CPU cores with parallel work, so that the 2 longest running
tagets, Mktables and all XS modules, run in parallel instead of 1 after
another. Since Normalize was hardcoded as a dynamic module, in an
ALL_STATIC build, Normalize was build twice, once as a static in the
big Extensions_static target, and once as a dynamic module, a race
happened if a parallel build was done, and if the Extensions_static target
got around to trying to build Normalize static, before by chance Normalize
dyn was build, Normalize static failed. This can be demostrated by cleaning
the src tree and building each target individually explicitly. Fix the race
by creating a Normalize static and dynamic target and dropout the deps
depending on if its ALL_STATIC or not. Also the now "big" Extensions_static
target when doing an ALL_STATIC build also failed because the general XS
modules that need lib.pm to build didn't have lib.pm available when their
Makefile.PL'es were run. For example cpan/Compress-Raw-Bzip2 needs lib.pm
to build. So separate out lib.pm from Dynaloader target so both dyn
extensions and static exts have lib.pm available. Technically, static exts
dont need lib.pm unless its ALL_STATIC, since for dyn win32 perl Win32CORE
is the only static module, but lib.pm is relativly fast and easy and low
dep to build so build it anyways for static ext target in non-ALL_STATIC
mode rather than special case the dep between ALL_STATIC and
non-ALL_STATIC.
-re.pm doesn't need lib.pm
-normalize doesn't need lib.pm
Reverts "Fix static builds with MinGW" commit 9999704e7ac and adds fixes.
This is a follow on to [perl #132992].
To reduce the diff-ness between the dmake and gmake mkfs. Make the
distclean identical. For dmkf, dont delete Storable.pm twice. And for gmfk
add Amiga dir deletion since lib_cleanup.pl requires it for dmkf but the
original day 1 of gmkf in commit 342634f3c8 "kmx's original GNUmakefile"
never had the Amiga line, but that day 1 commit also never added
GNUmakefile to lib_cleanup.pl the way makefile.mk was already added and
chked for its dir clean list. So add GNUmakefile to be chked by
lib_cleanup.pl the way the other 2 win32 mkfs are checked. In gmfk move
Storable.pm line to match dmfk which is more psuedo-alpha sorted. Also
fix the Test vs Test2 typo.
Some more text is available in the ticket associated with this patch.
---
regen/lib_cleanup.pl | 2 +-
t/porting/regen.t | 2 +-
win32/GNUmakefile | 31 +++++++++++++++++++++----------
win32/makefile.mk | 27 ++++++++++++++++++---------
4 files changed, 41 insertions(+), 21 deletions(-)
diff --git a/regen/lib_cleanup.pl b/regen/lib_cleanup.pl
index d381269..5c5c4e2 100644
--- a/regen/lib_cleanup.pl
+++ b/regen/lib_cleanup.pl
@@ -154,7 +154,7 @@ sub edit_win32_makefile {
}
process('Makefile.SH', 'Makefile.SH', \&edit_makefile_SH, $TAP && '', $Verbose);
-foreach ('win32/Makefile', 'win32/makefile.mk') {
+foreach ('win32/Makefile', 'win32/makefile.mk', 'win32/GNUmakefile') {
process($_, $_, \&edit_win32_makefile, $TAP && '', $Verbose);
}
diff --git a/t/porting/regen.t b/t/porting/regen.t
index f296626..26639a9 100644
--- a/t/porting/regen.t
+++ b/t/porting/regen.t
@@ -26,7 +26,7 @@ if ( $Config{usecrosscompile} ) {
skip_all( "Not all files are available during cross-compilation" );
}
-my $tests = 26; # I can't see a clean way to calculate this automatically.
+my $tests = 27; # I can't see a clean way to calculate this automatically.
my %skip = ("regen_perly.pl" => [qw(perly.act perly.h perly.tab)],
"regen/keywords.pl" => [qw(keywords.c keywords.h)],
diff --git a/win32/GNUmakefile b/win32/GNUmakefile
index 877c16b..82bf593 100644
--- a/win32/GNUmakefile
+++ b/win32/GNUmakefile
@@ -1110,11 +1110,13 @@ ifeq ($(ALL_STATIC),define)
# some exclusions, unfortunately, until fixed:
# - MakeMaker isn't capable enough for SDBM_File (small bug)
STATIC_EXT = * !SDBM_File
+NORMALIZE_STATIC = Normalize_static
else
# specify static extensions here, for example:
# (be sure to include Win32CORE to load Win32 on demand)
#STATIC_EXT = Win32CORE Cwd Compress/Raw/Zlib
STATIC_EXT = Win32CORE
+NORMALIZE_DYN = Normalize_dyn
endif
DYNALOADER = ..\DynaLoader$(o)
@@ -1572,25 +1574,33 @@ endif
# DynaLoader.pm, so this will have to do
#most of deps of this target are in DYNALOADER and therefore omitted here
-Extensions : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) MakePPPort
+Extensions : $(PERLDEP) $(DYNALOADER) Extension_lib $(GLOBEXE) MakePPPort
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic !Unicode/Normalize
-Extensions_normalize : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
+Normalize_static : $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) $(UNIDATAFILES)
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Unicode/Normalize
+
+Normalize_dyn : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +Unicode/Normalize
Extensions_reonly : $(PERLDEP) $(DYNALOADER)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re
-Extensions_static : ..\make_ext.pl list_static_libs.pl $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
- $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static
+Exts_static_general : ..\make_ext.pl $(CONFIGPM) Extension_lib $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static !Unicode/Normalize
+
+Extensions_static : list_static_libs.pl Exts_static_general $(NORMALIZE_STATIC)
$(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static
Extensions_nonxs : ..\make_ext.pl ..\pod\perlfunc.pod $(CONFIGPM) $(GLOBEXE)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs
+Extension_lib : ..\make_ext.pl $(CONFIGPM)
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) lib
+
#lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building
$(DYNALOADER) : ..\make_ext.pl $(CONFIGPM) $(HAVE_COREDIR)
- $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader
Extensions_clean :
-if exist $(MINIPERL) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --all --target=clean
@@ -1612,9 +1622,9 @@ PostExt : ..\lib\Storable\Limit.pm
# be running in parallel like UNIDATAFILES, this target a placeholder for the
# future
ifeq ($(PERLSTATIC),static)
-rebasePE : Extensions_normalize Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
+rebasePE : Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
else
-rebasePE : Extensions $(PERLDLL) Extensions_normalize $(PERLEXE)
+rebasePE : Extensions $(PERLDLL) $(NORMALIZE_DYN) $(PERLEXE)
endif
$(NOOP)
@@ -1670,7 +1680,7 @@ utils: $(HAVEMINIPERL) ..\utils\Makefile
$(MINIPERL) -I..\lib ..\autodoc.pl ..
$(MINIPERL) -I..\lib ..\pod\perlmodlib.PL -q ..
-..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs Extensions_normalize utils
+..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs $(NORMALIZE_DYN) utils
$(PERLEXE) -f ..\pod\buildtoc -q
# Note that the pod cleanup in this next section is parsed (and regenerated
@@ -1687,11 +1697,11 @@ distclean: realclean
-del /f $(LIBDIR)\SDBM_File.pm $(LIBDIR)\Socket.pm $(LIBDIR)\POSIX.pm
-del /f $(LIBDIR)\B.pm $(LIBDIR)\O.pm $(LIBDIR)\re.pm
-del /f $(LIBDIR)\File\Glob.pm
- -del /f $(LIBDIR)\Storable.pm $(LIBDIR)\Storable\Limit.pm
-del /f $(LIBDIR)\Sys\Hostname.pm
-del /f $(LIBDIR)\Time\HiRes.pm
-del /f $(LIBDIR)\Unicode\Normalize.pm
-del /f $(LIBDIR)\Math\BigInt\FastCalc.pm
+ -del /f $(LIBDIR)\Storable.pm $(LIBDIR)\Storable\Limit.pm
-del /f $(LIBDIR)\Win32.pm
-del /f $(LIBDIR)\Win32CORE.pm
-del /f $(LIBDIR)\Win32API\File.pm
@@ -1699,6 +1709,7 @@ distclean: realclean
-del /f $(LIBDIR)\buildcustomize.pl
-del /f $(DISTDIR)\XSLoader\XSLoader.pm
-del /f *.def *.map
+ -if exist $(LIBDIR)\Amiga rmdir /s /q $(LIBDIR)\Amiga
-if exist $(LIBDIR)\App rmdir /s /q $(LIBDIR)\App
-if exist $(LIBDIR)\Archive rmdir /s /q $(LIBDIR)\Archive
-if exist $(LIBDIR)\Attribute rmdir /s /q $(LIBDIR)\Attribute
@@ -1750,7 +1761,7 @@ distclean: realclean
-if exist $(LIBDIR)\TAP rmdir /s /q $(LIBDIR)\TAP
-if exist $(LIBDIR)\Term rmdir /s /q $(LIBDIR)\Term
-if exist $(LIBDIR)\Test rmdir /s /q $(LIBDIR)\Test
- -if exist $(LIBDIR)\Test2 rmdir /s /q $(LIBDIR)\Test
+ -if exist $(LIBDIR)\Test2 rmdir /s /q $(LIBDIR)\Test2
-if exist $(LIBDIR)\Text rmdir /s /q $(LIBDIR)\Text
-if exist $(LIBDIR)\Thread rmdir /s /q $(LIBDIR)\Thread
-if exist $(LIBDIR)\threads rmdir /s /q $(LIBDIR)\threads
diff --git a/win32/makefile.mk b/win32/makefile.mk
index 2797b8a..265e93b 100644
--- a/win32/makefile.mk
+++ b/win32/makefile.mk
@@ -1084,11 +1084,13 @@ SETARGV_OBJ = setargv$(o)
# some exclusions, unfortunately, until fixed:
# - MakeMaker isn't capable enough for SDBM_File (small bug)
STATIC_EXT = * !SDBM_File
+NORMALIZE_STATIC = Normalize_static
.ELSE
# specify static extensions here, for example:
# (be sure to include Win32CORE to load Win32 on demand)
#STATIC_EXT = Win32CORE Cwd Compress/Raw/Zlib
STATIC_EXT = Win32CORE
+NORMALIZE_DYN = Normalize_dyn
.ENDIF
DYNALOADER = ..\DynaLoader$(o)
@@ -1513,25 +1515,33 @@ $(PERLEXESTATIC): $(PERLSTATICLIB) $(CONFIGPM) $(PERLEXEST_OBJ) $(PERLEXE_RES)
# DynaLoader.pm, so this will have to do
#most of deps of this target are in DYNALOADER and therefore omitted here
-Extensions : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) MakePPPort
+Extensions : $(PERLDEP) $(DYNALOADER) Extension_lib $(GLOBEXE) MakePPPort
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic !Unicode/Normalize
-Extensions_normalize : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
+Normalize_static : $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) $(UNIDATAFILES)
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Unicode/Normalize
+
+Normalize_dyn : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +Unicode/Normalize
Extensions_reonly : $(PERLDEP) $(DYNALOADER)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re
-Extensions_static : ..\make_ext.pl list_static_libs.pl $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
- $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static
+Exts_static_general : ..\make_ext.pl $(CONFIGPM) Extension_lib $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static !Unicode/Normalize
+
+Extensions_static : list_static_libs.pl Exts_static_general $(NORMALIZE_STATIC)
$(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static
Extensions_nonxs : ..\make_ext.pl ..\pod\perlfunc.pod $(CONFIGPM) $(GLOBEXE)
$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs
+Extension_lib : ..\make_ext.pl $(CONFIGPM)
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) lib
+
#lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building
$(DYNALOADER) : ..\make_ext.pl $(CONFIGPM) $(HAVE_COREDIR)
- $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib
+ $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader
Extensions_clean :
-if exist $(MINIPERL) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --all --target=clean
@@ -1543,9 +1553,9 @@ Extensions_realclean :
# be running in parallel like UNIDATAFILES, this target a placeholder for the
# future
.IF "$(PERLSTATIC)"=="static"
-rebasePE : Extensions_normalize Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
+rebasePE : Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
.ELSE
-rebasePE : Extensions $(PERLDLL) Extensions_normalize $(PERLEXE)
+rebasePE : Extensions $(PERLDLL) $(NORMALIZE_DYN) $(PERLEXE)
.ENDIF
$(NOOP)
@@ -1613,7 +1623,7 @@ utils: $(HAVEMINIPERL) ..\utils\Makefile
$(MINIPERL) -I..\lib ..\autodoc.pl ..
$(MINIPERL) -I..\lib ..\pod\perlmodlib.PL -q ..
-..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs Extensions_normalize utils
+..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs $(NORMALIZE_DYN) utils
$(PERLEXE) -f ..\pod\buildtoc -q
# Note that the pod cleanup in this next section is parsed (and regenerated
@@ -1630,7 +1640,6 @@ distclean: realclean
-del /f $(LIBDIR)\SDBM_File.pm $(LIBDIR)\Socket.pm $(LIBDIR)\POSIX.pm
-del /f $(LIBDIR)\B.pm $(LIBDIR)\O.pm $(LIBDIR)\re.pm
-del /f $(LIBDIR)\File\Glob.pm
- -del /f $(LIBDIR)\Storable.pm
-del /f $(LIBDIR)\Sys\Hostname.pm
-del /f $(LIBDIR)\Time\HiRes.pm
-del /f $(LIBDIR)\Unicode\Normalize.pm
--
2.5.0.windows.1
|
From Vadim.Konovalov@dell.comHi Daniel
... Given that you so much dived into details of this problem, maybe you can I've questioned this question here: TIA |
|
The RT System itself - Status changed from 'new' to 'open' |
From @tonycozOn Fri, 06 Apr 2018 06:28:22 -0700, bulk88 wrote:
Thanks, applied as 908f2cb. There's still a problem if you build with STATIC_EXT=Unicode/Normalize but that might be more difficult to fix. I do wonder if the Win32 build should build miniperl, then use that to build the final config.sh/config.h and generate a Makefile for the full perl from there. Tony |
From @bulk88On Sun, 08 Apr 2018 21:03:16 -0700, tonyc wrote:
What will generate the makefile? Makefile.SH would have to be rewritten in PP. IMO its pointless because Windows doesn't come with a "Unix shell" compiler and M4 compiler out of the box and switching to Visual C with Autoconf+Bash (I've built such FOSS projects before) is insane since not even Unix perl uses M4 or autoconf. I know you wrote one patch 2 yrs ago that generates an auxiliary extensions only makefile for parallel extension building so each extension can be parallelzed to infinity cores instead of the 4 "categories" of extensions building side by side they way they do now and doesn't saturate all cores. -- |
From @tonycozOn Mon, Apr 16, 2018 at 10:54:02AM -0700, bulk88 via RT wrote:
Well, it wouldn't match Makefile.SH anyway, since it doesn't need to
It would at least fix building with STATIC_EXT=Unicode/Normalize. It may be possible to improve parallel building over my extensions.mk, At this point it's just brainstorming which has at least inspired a Tony |
Migrated from rt.perl.org#133070 (status was 'open')
Searchable as RT133070$
The text was updated successfully, but these errors were encountered: