Skip Menu |
Report information
Id: 123867
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: bulk88 <bulk88 [at] hotmail.com>
Cc:
AdminCc:

Operating System: mswin32
PatchStatus: (no value)
Severity: low
Type:
Perl Version: 5.21.4
Fixed In: (no value)

Attachments
0001-add-parallel-support-4-Win32-dmake-building-part-1.patch
0001-WIP-add-parallel-support-for-Win32-dmake-building.patch
0002-add-parallel-support-4-Win32-dmake-COREDIR-paralleli.patch
0002-coredir-parallelism.patch
0003-add-parallel-support-4-Win32-dmake-uudmap-no-2nd-run.patch
0003-squash-double-dmake-run.patch
0004-squash-collapse-3-shell-runs-of-xcopy-into-1-run-of-.patch
0005-squash-less-steps-build-generate_uudmap.exe.patch
gcc463time.txt
gcc483time.txt
vc10time.txt
vc12time.txt
vc7nogltime.txt



To: perlbug [...] perl.org
Subject: [PATCH] WIP add parallel support for Win32 dmake building
From: bulk88 <bulk88 [...] hotmail.com>
Date: Wed, 18 Feb 2015 00:48:50 -0500
Download (untitled) / with headers
text/plain 11.1k
This is a bug report for perl from bulk88@hotmail.com, generated with the help of perlbug 1.40 running under perl 5.21.4. ----------------------------------------------------------------- [Please describe your issue here] See attached patch. See also http://www.nntp.perl.org/group/perl.perl5.porters/2015/02/msg225930.html and https://rt.perl.org/Ticket/Display.html?id=123854 Works with VC and GCC. Building tested with "dmake -P8 test" on a 8 core PC. Not fully smoked. I am getting 1 test failure, but I dont think it is related to this patch, the chcp stuff is from EUMM, and I'll have to reinvestigate why I am seeing it since I thought a patch went in a few months ago for "no 32 bit chcp on x64 Server 2003" into EUMM. C:\p521\srcpara\t>perl harness -v porting/utils.t porting/utils.t .. # Porting/bisect-example.sh isn't a Perl script # Porting/config_H isn't a Perl script # Porting/config.sh isn't a Perl script # Porting/corelist-diff isn't a Perl script # Porting/deparse-skips.txt isn't a Perl script # Porting/epigraphs.pod isn't a Perl script # Porting/exec-bit.txt isn't a Perl script # Porting/git-make-p4-refs isn't a Perl script # Porting/Glossary isn't a Perl script # Porting/how_to_write_a_perldelta.pod isn't a Perl script # Porting/perl5220delta.pod isn't a Perl script # Porting/perldelta_template.pod isn't a Perl script # Porting/pumpkin.pod isn't a Perl script # Porting/README.pod isn't a Perl script # Porting/README.y2038 isn't a Perl script # Porting/release_announcement_template.txt isn't a Perl script # Porting/release_managers_guide.pod isn't a Perl script # Porting/release_schedule.pod isn't a Perl script # Porting/rt_list_patches isn't a Perl script # Porting/timecheck2.c isn't a Perl script # Porting/timecheck.c isn't a Perl script # Porting/todo.pod isn't a Perl script 1..81 ok 1 - installman compiles ok 2 - installperl compiles ok 3 - regen_perly.pl compiles ok 4 - regen/uconfig_h.pl compiles ok 5 - Porting/acknowledgements.pl compiles ok 6 - Porting/add-package.pl compiles ok 7 - Porting/bench.pl compiles ok 8 - Porting/bisect.pl compiles ok 9 - Porting/bisect-runner.pl compiles ok 10 - Porting/bump-perl-version compiles ok 11 - Porting/check83.pl compiles ok 12 - Porting/checkansi.pl compiles ok 13 - Porting/checkAUTHORS.pl compiles ok 14 - Porting/checkcfgvar.pl compiles ok 15 - Porting/check-cpan-pollution compiles ok 16 - Porting/checkpodencoding.pl compiles ok 17 - Porting/checkURL.pl compiles ok 18 - Porting/checkVERSION.pl compiles ok 19 - Porting/cherrymaint compiles # Failed test 20 - Porting/cmpVERSION.pl compiles at porting/utils.t line 89 # got "\'chcp\' is not recognized as an internal or external command,\noper able program or batch file.\nPorting/cmpVERSION.pl syntax OK\n" # expected "Porting/cmpVERSION.pl syntax OK\n" # when executing perl with '-c Porting/cmpVERSION.pl' not ok 20 - Porting/cmpVERSION.pl compiles ok 21 - Porting/config_h.pl compiles # Failed test 22 - Porting/core-cpan-diff compiles at porting/utils.t line 89 # got "\'chcp\' is not recognized as an internal or external command,\noper able program or batch file.\nPorting/core-cpan-diff syntax OK\n" # expected "Porting/core-cpan-diff syntax OK\n" # when executing perl with '-c Porting/core-cpan-diff' not ok 22 - Porting/core-cpan-diff compiles # Failed test 23 - Porting/corecpan.pl compiles at porting/utils.t line 89 # got "\'chcp\' is not recognized as an internal or external command,\noper able program or batch file.\nPorting/corecpan.pl syntax OK\n" # expected "Porting/corecpan.pl syntax OK\n" # when executing perl with '-c Porting/corecpan.pl' not ok 23 - Porting/corecpan.pl compiles ok 24 - Porting/corelist-perldelta.pl compiles # Failed test 25 - Porting/corelist.pl compiles at porting/utils.t line 89 # got "\'chcp\' is not recognized as an internal or external command,\noper able program or batch file.\nPorting/corelist.pl syntax OK\n" # expected "Porting/corelist.pl syntax OK\n" # when executing perl with '-c Porting/corelist.pl' not ok 25 - Porting/corelist.pl compiles ok 26 - Porting/exercise_makedef.pl compiles ok 27 - Porting/expand-macro.pl compiles ok 28 - Porting/findrfuncs compiles ok 29 # skip Porting/git-deltatool uses Git::Wrapper, so can't test with just co re modules ok 30 - Porting/git-find-p4-change compiles ok 31 - Porting/GitUtils.pm compiles ok 32 # skip Porting/leakfinder.pl uses XS::APItest, so can't test with just cor e modules ok 33 - Porting/Maintainers compiles ok 34 - Porting/Maintainers.pl compiles ok 35 - Porting/Maintainers.pm compiles ok 36 - Porting/make_dot_patch.pl compiles ok 37 - Porting/makemeta compiles ok 38 - Porting/make_modlib_cpan.pl compiles ok 39 - Porting/makerel compiles ok 40 - Porting/make-rmg-checklist compiles ok 41 - Porting/make_snapshot.pl compiles ok 42 - Porting/manicheck compiles ok 43 - Porting/manisort compiles ok 44 - Porting/new-perldelta.pl compiles ok 45 - Porting/newtests-perldelta.pl compiles ok 46 - Porting/perlhist_calculate.pl compiles ok 47 - Porting/pod_lib.pl compiles ok 48 - Porting/pod_rules.pl compiles ok 49 # skip Porting/podtidy uses Pod::Tidy, so can't test with just core module s ok 50 - Porting/sort_perldiag.pl compiles ok 51 - Porting/sync-with-cpan compiles ok 52 - Porting/valgrindpp.pl compiles ok 53 - cpan/podlators/blib/script/pod2man compiles ok 54 - cpan/podlators/blib/script/pod2text compiles ok 55 - cpan/Pod-Usage/blib/script/pod2usage compiles ok 56 - cpan/Pod-Checker/blib/script/podchecker compiles ok 57 - cpan/Pod-Parser/blib/script/podselect compiles ok 58 - utils/c2ph compiles # Failed test 59 - utils/cpan compiles at porting/utils.t line 89 # got "\'chcp\' is not recognized as an internal or external command,\noper able program or batch file.\nutils/cpan syntax OK\n" # expected "utils/cpan syntax OK\n" # when executing perl with '-c utils/cpan' not ok 59 - utils/cpan compiles ok 60 - utils/corelist compiles ok 61 - utils/enc2xs compiles ok 62 - utils/encguess compiles ok 63 - utils/h2ph compiles ok 64 - utils/h2xs compiles # Failed test 65 - utils/instmodsh compiles at porting/utils.t line 89 # got "\'chcp\' is not recognized as an internal or external command,\noper able program or batch file.\nutils/instmodsh syntax OK\n" # expected "utils/instmodsh syntax OK\n" # when executing perl with '-c utils/instmodsh' not ok 65 - utils/instmodsh compiles ok 66 - utils/json_pp compiles # Failed test 67 - utils/libnetcfg compiles at porting/utils.t line 89 # got "\'chcp\' is not recognized as an internal or external command,\noper able program or batch file.\nutils/libnetcfg syntax OK\n" # expected "utils/libnetcfg syntax OK\n" # when executing perl with '-c utils/libnetcfg' not ok 67 - utils/libnetcfg compiles ok 68 - utils/perlbug compiles ok 69 - utils/perldoc compiles # Failed test 70 - utils/perlivp compiles at porting/utils.t line 89 # got "\'chcp\' is not recognized as an internal or external command,\noper able program or batch file.\nutils/perlivp syntax OK\n" # expected "utils/perlivp syntax OK\n" # when executing perl with '-c utils/perlivp' not ok 70 - utils/perlivp compiles ok 71 - utils/piconv compiles ok 72 - utils/pl2pm compiles ok 73 - utils/pod2html compiles ok 74 - utils/prove compiles ok 75 - utils/ptar compiles ok 76 - utils/ptardiff compiles ok 77 - utils/ptargrep compiles ok 78 - utils/shasum compiles ok 79 - utils/splain compiles ok 80 - utils/xsubpp compiles ok 81 - utils/zipdetails compiles Failed 8/81 subtests (less 3 skipped subtests: 70 okay) Test Summary Report ------------------- porting/utils.t (Wstat: 0 Tests: 81 Failed: 8) Failed tests: 20, 22-23, 25, 59, 65, 67, 70 Files=1, Tests=81, 26 wallclock secs ( 0.02 usr + 0.02 sys = 0.03 CPU) Result: FAIL C:\p521\srcpara\t> [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=low --- Site configuration information for perl 5.21.4: Configured by Owner at Thu Sep 18 12:08:58 2014. Summary of my perl5 (revision 5 version 21 subversion 4) configuration: Derived from: 7d2b2edb94ab56333b9049a3e26d15ea18445512 Ancestor: 19be3be6968e2337bcdfe480693fff795ecd1304 Platform: osname=MSWin32, osvers=5.1, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef useithreads=define, usemultiplicity=define use64bitint=undef, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cl', ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL_TEXTMODE_SCRIPTS -DPERL_HASH_FUNC_ONE_AT_A_TIME -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -D_USE_32BIT_TIME_T', optimize='-O1 -MD -Zi -DNDEBUG', cppflags='-DWIN32' ccversion='12.00.8168', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8, longdblkind=0 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -libpath:"c:\perl521\lib\CORE" -machine:x86' libpth=C:\PROGRA~1\MIAF9D~1\VC98\lib libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl521.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -libpath:"c:\perl521\lib\CORE" -machine:x86' Locally applied patches: uncommitted-changes a0fe7a7e75de29e59f1da0d6822dc06e5be658fe a261faffee83d0145642ab5d1d046c9f813bc497 6506ab86ad1602a9ca720fcd30446dce1461d23d 7d2b2edb94ab56333b9049a3e26d15ea18445512 --- @INC for perl 5.21.4: lib C:/perl521/srcnew/lib . --- Environment for perl 5.21.4: HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH= PERL_BADLANG (unset) PERL_JSON_BACKEND=Cpanel::JSON::XS PERL_YAML_BACKEND=YAML SHELL (unset)

Message body is not shown because it is too large.

Subject: Re: [perl #123867] [PATCH] WIP add parallel support for Win32 dmake building
To: "perl5-porters [...] perl.org" <perl5-porters [...] perl.org>
Date: Mon, 9 Mar 2015 09:00:23 +0000
CC: "bugs-bitbucket [...] rt.perl.org" <bugs-bitbucket [...] rt.perl.org>
From: Steve Hay <steve.m.hay [...] googlemail.com>
Download (untitled) / with headers
text/plain 758b
On 18 February 2015 at 05:49, bulk88 <perlbug-followup@perl.org> wrote: Show quoted text
> -PDBOUT is required to run multiple VC cl.exe processes, otherwise all > VC cl.exe processes will error out trying to lock and write to a > file called "vc*0.pdb", PDBOUT is empty for GCC builds since they dont > have PDB files
I haven't had time to have a good look at this patch yet, but this comment made me think of a new cl.exe option that I noticed recently when upgrading to VS2013: /FS (Force Synchronous PDB Writes). Sounds good, eh? Unfortunately, MS's own documentation (https://msdn.microsoft.com/en-us/library/dn502518.aspx) says "it doesn't prevent all errors that may occur when multiple instances of cl.exe access the PDB file at the same time"!!! Bewildering...
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.9k
On Mon Mar 09 02:00:57 2015, shay wrote: Show quoted text
> On 18 February 2015 at 05:49, bulk88 <perlbug-followup@perl.org> wrote:
> > -PDBOUT is required to run multiple VC cl.exe processes, otherwise all > > VC cl.exe processes will error out trying to lock and write to a > > file called "vc*0.pdb", PDBOUT is empty for GCC builds since they dont > > have PDB files
> > I haven't had time to have a good look at this patch yet, but this > comment made me think of a new cl.exe option that I noticed recently > when upgrading to VS2013: /FS (Force Synchronous PDB Writes). Sounds > good, eh? Unfortunately, MS's own documentation > (https://msdn.microsoft.com/en-us/library/dn502518.aspx) says "it > doesn't prevent all errors that may occur when multiple instances of > cl.exe access the PDB file at the same time"!!! Bewildering... >
Grr, I tried to use the email response feature, and I sent this to "bugs-bitbucket@rt.perl.org" and it never appeared in this ticket. Reposting through the web interface. -FS is irrelevant if it was only introduced in VS 2013, which I can't even install since XP/2003 was dropped in VS 2012. -Fd works on all VCs supported by p5p 6 to 2013. I dont exactly like the /MP option, since it is only available on VC 2008 and newer, and AFAIK, /MP means that each .c file gets the exact same -D'es (not all the perl core .c'es have the same -D'es) and must be the same langauge (win32 perl is a combo of C and C++ (for psuedofork). Also /MP parallelizes only a small portion of the workload, it can't parallel link perlglob and generate_uudmap at teh same time. You can't link perl52*.dll and perl.exe at the same time with /MP. /MP might theoretically save on I/O with some kind of caching done by cl.exe of the headers. IDK, but in general /MP is quite limited IMO. /MP also forbids /Fo, so you can't give the individual .c'es new abs paths. /Fo in /MP mode does allow /Fomini/ to set an output dir, but again it isn't as flexible as /Fo without /MP. The limits of only 1 outdir makes compiling the .c files in /win32 a lil complicated. The .c'es in / turned into .objs go into /, the .c'es in /win32 turned into .objs go into /win32. Although there are no name conflicts, and all the .objes that make up win32 miniperl go in the same /win32/mini dir, but still, MP isn't flexible. Also /MP isn't that much of a wall time saver because of -GL, so instead of ~90% of the walltime being in cl.exe, with -GL its ~10% in cl.exe, ~90 in link.exe, and /MP doesn't parallelize code gen in link.exe with -GL/-LTCG. Turning off -GL/-LTCG for release perl builds is unacceptable, although a "optimized devel (-O1, no DEBUGGING, no -GL)" profile for p5p devs is possible for a intermediate step between full optimization build and the DEBUGGING build config, i think mingw perl recently got an option like that or kmx has a patch on RT to do that. See my earlier write up of /MP in http://www.nntp.perl.org/group/perl.perl5.porters/2014/12/msg223866.html -- bulk88 ~ bulk88 at hotmail.com
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 122b
Rebased patch attached, includes a small fix for using a 64 bit dmake with 32 bit CCs. -- bulk88 ~ bulk88 at hotmail.com
Subject: 0001-WIP-add-parallel-support-for-Win32-dmake-building.patch

Message body is not shown because it is too large.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 245b
On Sun Apr 05 15:43:49 2015, bulk88 wrote: Show quoted text
> Rebased patch attached, includes a small fix for using a 64 bit dmake > with 32 bit CCs.
Revised patch attached, ..\pod\perltoc.pod target got missing deps added. -- bulk88 ~ bulk88 at hotmail.com
Subject: 0001-WIP-add-parallel-support-for-Win32-dmake-building.patch

Message body is not shown because it is too large.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 791b
On Sun May 10 20:59:50 2015, bulk88 wrote: Show quoted text
> On Sun Apr 05 15:43:49 2015, bulk88 wrote:
> > Rebased patch attached, includes a small fix for using a 64 bit dmake > > with 32 bit CCs.
> > Revised patch attached, ..\pod\perltoc.pod target got missing deps added.
Doesn't apply as is, since blead has been bumped to 5.22. I applied the rejected chunks manually, and tried a build with VC: J:\dev\perl\git\perl\win32>dmake CCTYPE=MSVC90FREE dmake: makefile.mk: line 1330: Error: -- Missing targets or attributes in rule since PERLEXPLIB isn't defined for VC builds. If I move the definition from the C< .IF "$(CCTYPE)" == "GCC" > block outside of that I get a successful parallel build. We could avoid patching cpan/HiRes.xs by making the dummy ppport.h two lines instead of one. Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 3.7k
On Mon May 11 00:22:00 2015, tonyc wrote: Show quoted text
> Doesn't apply as is, since blead has been bumped to 5.22. > > I applied the rejected chunks manually, and tried a build with VC: > > J:\dev\perl\git\perl\win32>dmake CCTYPE=MSVC90FREE > dmake: makefile.mk: line 1330: Error: -- Missing targets or > attributes in rule > > since PERLEXPLIB isn't defined for VC builds. > > If I move the definition from the C< .IF "$(CCTYPE)" == "GCC" > block > outside of > that I get a successful parallel build.
Rebased on blead/5.22 with the PERLEXPLIB stuff fixed and comments in README.win32 and comments in makefile.mk improved. Show quoted text
> We could avoid patching cpan/HiRes.xs by making the dummy ppport.h two > lines > instead of one.
I could also add the fragment (dTHXR->dTHX, aTHXR->aTHX) of 5.005 threads API that Time::HiRes wants to core headers if Time::HiRes is abandoned (which it is according to Zefram its current maintainer per IRC today) but that might lead to redefinition warnings from the CPP, unless PPPort on CPAN is patched to do "#ifndef" although that now sounds like an exercise in bureaucracy vs your 2 line ppport.h idea or core patching Time::HiRes or CPAN release of Time::HiRes. The PPPort block currently is #if (PERL_BCDVERSION < 0x5006000) # ifdef USE_THREADS # define aTHXR thr # define aTHXR_ thr, # else # define aTHXR # define aTHXR_ # endif # define dTHXR dTHR #else # define aTHXR aTHX # define aTHXR_ aTHX_ # define dTHXR dTHX #endif perms for Time::HiRes module userid type owner Time::HiRes DEWEG co-maint ZEFRAM Time::HiRes ZEFRAM modulelist ZEFRAM There are 2 ideas I did not do in this patch. First is reordering MICROCORE_SRC in compile time order (specifically ..\toke.c first ..\regcomp.c 2nd and ..\regexec.c 3rd, and the rest by alphabetic order, see http://www.nntp.perl.org/group/perl.perl5.porters/2014/12/msg223870.html for details). This is very important on my 8 core machine, since "r" and "t" are at the end of the alphabet/current MICROCORE_SRC list, and they take the longest period of time, and when compiling the interp, it winds up with 10-12 seconds spent with only 1 or 2 cores in use, just compiling those 3 ultra long compilands. All the other core .c files take 1 or 2 seconds to compile, so they all finish almost instantly, and as the dmake hits the letters t and r, and it blocks/serialized on just 2 cores which is bad. The point of this patch is compiling an interp on a 8 or 10 core machine should never take anymore wall time than the time to compile the LONGEST compiland. With the current alphabetic ordering it takes twice the longest compiland. Second idea is to reorganize make_ext.pl running so non-xs and dynamic (2 biggest ones, since Dynaloader and static (usually just Win32CORE) are noise in the scheme of things) are built together and then with some kind of load balancer .IF $(MAXPROCESS) == 1 1of1 : $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) $@ .ELIF $(MAXPROCESS) == 2 1of2 2of2 : $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) $@ .ELIF $(MAXPROCESS) == 3 1of3 2of3 3of3 : $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) $@ .ELIF $(MAXPROCESS) == 4 1of4 2of4 3of4 4of4 : $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) $@ with make_ext.pl doing modulus to know which modules to build of the global list. The other choice is hard code the module list into the win32 makefile as is done on the unix makefile, to let the make tool dispatch parallelism correctly with 1 invocation of make_ext.pl building only ONE module per invocation. -- bulk88 ~ bulk88 at hotmail.com
Subject: 0001-WIP-add-parallel-support-for-Win32-dmake-building.patch

Message body is not shown because it is too large.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 5.9k
On Mon May 11 14:29:06 2015, bulk88 wrote: Show quoted text
> On Mon May 11 00:22:00 2015, tonyc wrote:
> > Doesn't apply as is, since blead has been bumped to 5.22. > > > > I applied the rejected chunks manually, and tried a build with VC: > > > > J:\dev\perl\git\perl\win32>dmake CCTYPE=MSVC90FREE > > dmake: makefile.mk: line 1330: Error: -- Missing targets or > > attributes in rule > > > > since PERLEXPLIB isn't defined for VC builds. > > > > If I move the definition from the C< .IF "$(CCTYPE)" == "GCC" > block > > outside of > > that I get a successful parallel build.
> > Rebased on blead/5.22 with the PERLEXPLIB stuff fixed and comments in > README.win32 and comments in makefile.mk improved. > >
> > We could avoid patching cpan/HiRes.xs by making the dummy ppport.h > > two > > lines > > instead of one.
> > I could also add the fragment (dTHXR->dTHX, aTHXR->aTHX) of 5.005 > threads API that Time::HiRes wants to core headers if Time::HiRes is > abandoned (which it is according to Zefram its current maintainer per > IRC today) but that might lead to redefinition warnings from the CPP, > unless PPPort on CPAN is patched to do "#ifndef" although that now > sounds like an exercise in bureaucracy vs your 2 line ppport.h idea or > core patching Time::HiRes or CPAN release of Time::HiRes. The PPPort > block currently is > > #if (PERL_BCDVERSION < 0x5006000) > # ifdef USE_THREADS > # define aTHXR thr > # define aTHXR_ thr, > # else > # define aTHXR > # define aTHXR_ > # endif > # define dTHXR dTHR > #else > # define aTHXR aTHX > # define aTHXR_ aTHX_ > # define dTHXR dTHX > #endif
Time::HiRes isn't abandoned, just delayed. It's just: # DynaLoader.pm, so this will have to do -Extensions : ..\make_ext.pl ..\lib\buildcustomize.pl $(PERLDEP) $(CONFIGPM) $(DYNALOADER) +Extensions : ..\make_ext.pl ..\lib\buildcustomize.pl $(PERLDEP) $(CONFIGPM) $(DYNALOADER) $(COREDIR)\ppport.h $(XCOPY) ..\*.h $(COREDIR)\*.* - if not exist $(COREDIR)\ppport.h rem. > $(COREDIR)\ppport.h $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic Extensions_reonly : ..\make_ext.pl ..\lib\buildcustomize.pl $(PERLDEP) $(CONFIGPM) $(DYNALOADER) $(XCOPY) ..\*.h $(COREDIR)\*.* $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re -Extensions_static : ..\make_ext.pl ..\lib\buildcustomize.pl list_static_libs.pl $(CONFIGPM) +Extensions_static : ..\make_ext.pl ..\lib\buildcustomize.pl list_static_libs.pl $(CONFIGPM) $(COREDIR)\ppport.h $(XCOPY) ..\*.h $(COREDIR)\*.* - if not exist $(COREDIR)\ppport.h rem. > $(COREDIR)\ppport.h $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static $(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static @@ -1455,6 +1453,11 @@ Extensions_clean : Extensions_realclean : -if exist $(MINIPERL) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --all --target=realclean +$(COREDIR)\ppport.h : makefile.mk + -mkdir $(COREDIR) + @( echo #define aTHXR aTHX && \ + echo #define dTHXR dTHX ) >$@ + # all PE files need to be built by the time this target runs, PP files can still Show quoted text
> There are 2 ideas I did not do in this patch. > > First is reordering MICROCORE_SRC in compile time order (specifically > ..\toke.c first ..\regcomp.c 2nd and ..\regexec.c 3rd, and the rest by > alphabetic order, see > http://www.nntp.perl.org/group/perl.perl5.porters/2014/12/msg223870.html > for details). This is very important on my 8 core machine, since "r" > and "t" are at the end of the alphabet/current MICROCORE_SRC list, and > they take the longest period of time, and when compiling the interp, > it winds up with 10-12 seconds spent with only 1 or 2 cores in use, > just compiling those 3 ultra long compilands. All the other core .c > files take 1 or 2 seconds to compile, so they all finish almost > instantly, and as the dmake hits the letters t and r, and it > blocks/serialized on just 2 cores which is bad. The point of this > patch is compiling an interp on a 8 or 10 core machine should never > take anymore wall time than the time to compile the LONGEST compiland. > With the current alphabetic ordering it takes twice the longest > compiland.
That might help. Show quoted text
> Second idea is to reorganize make_ext.pl running so non-xs and dynamic > (2 biggest ones, since Dynaloader and static (usually just Win32CORE) > are noise in the scheme of things) are built together and then with > some kind of load balancer > > .IF $(MAXPROCESS) == 1 > 1of1 : > $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" > --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) $@ > .ELIF $(MAXPROCESS) == 2 > 1of2 2of2 : > $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" > --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) $@ > .ELIF $(MAXPROCESS) == 3 > 1of3 2of3 3of3 : > $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" > --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) $@ > .ELIF $(MAXPROCESS) == 4 > 1of4 2of4 3of4 4of4 : > $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" > --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) $@ > > with make_ext.pl doing modulus to know which modules to build of the > global list. > > The other choice is hard code the module list into the win32 makefile > as is done on the unix makefile, to let the make tool dispatch > parallelism correctly with 1 invocation of make_ext.pl building only > ONE module per invocation.
Instead of hard coding, perhaps write an make_ext_mk.pl that generates extensions.mk using FindExt to pick up which extensions need to be built similar to what Makefile.SH does, so the rules become something like: Extensions: extensions.mk $(MAKE) -f extensions.mk Extensions Extensions_nonxs: extensions.mk $(MAKE) -f extensions.mk Extensions_nonxs ... extensions.mk: makefile.mk $(MINIPERL) -I..\lib make_ext_mk.pl -dmake withe Makefile.SH rules in extensions.mk Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.5k
Adding compile timings for archive reasons, this is to change from alphabetic compile order of core intep .c files to something by time, so the longest to compile .c files are started first. NOTE ..\DynaLoader.o is running EUMM and make_ext.pl, it is not a simple call to the CC. Please ignore the timings for ..\DynaLoader.o/obj. Each line is the time of a recipe run (not a target run, a target consists of one or more recipes), the name of the line is the target name. VC7 build had -GL turned off, since -GL/LTCG skews build time by transferring nearly all of the compile time of each .c file into link time of perl522.dll and miniperl.exe, which is single threaded/single process. Yes LTCG harms build time in parallel mode, but a better binary in the end is worth far more than 10 wall seconds of CC compile time. A core win32 hacker (egh, me) who is doing high frequency rebuilds on the interp will know to turn off LTCG, and switch from -O2/-O1 to -Od/-O0, without turning on DEBUGGING, anyway. timings obtained by doing b88dmake -mr CCTYPE=MSVC7 ../perl522.dll > ../../vc.txt or b88dmake -mr ../perl522.dll > ../../gcc.txt Then processing that console log with my $f; open($f, '<', 'C:\perl521\vc.txt') || die "bad"; my $s; my $start; while($s = <$f>){ if($s =~ /^[s|e] recipe ([-+]?[0-9]*\.?[0-9]+) (.+)$/){ if(!$start) { $start = $1; } else { print sprintf('%-20s',($1-$start))." $2\n"; $start = undef; } } } then text sorting the output of that script in my word processor, which produced the 2 attached files. -- bulk88 ~ bulk88 at hotmail.com
Subject: gcc463time.txt
Download gcc463time.txt
text/plain 4.1k
6.06299996376038 ..\toke.o 5.5 mini\toke.o 5.43799996376038 ..\regcomp.o 5.20300006866455 mini\regcomp.o 4.28200006484985 ..\op.o 4.15599989891052 ..\DynaLoader.o 3.92199993133545 mini\op.o 3.84299993515015 ..\sv.o 3.70300006866455 ..\regexec.o 3.67199993133545 mini\regexec.o 3.07799983024597 mini\sv.o 2.875 ..\pp.o 2.7350001335144 mini\pp.o 2.5 ..\pp_ctl.o 2.34299993515015 mini\pp_ctl.o 1.96900010108948 ..\pp_sys.o 1.95300006866455 Extensions_static 1.73499989509583 ..\pp_pack.o 1.7039999961853 mini\pp_pack.o 1.68700003623962 ..\pp_hot.o 1.57800006866455 mini\pp_sys.o 1.57800006866455 mini\pp_hot.o 1.375 ..\gv.o 1.3600001335144 perllib.o 1.32800006866455 mini\gv.o 1.26600003242493 ..\perl.o 1.23399996757507 ..\utf8.o 1.23399996757507 ..\perlio.o 1.20300006866455 win32.o 1.18799996376038 mini\utf8.o 1.18799996376038 ..\util.o 1.18799996376038 ..\hv.o 1.17199993133545 ..\dump.o 1.14100003242493 ..\mg.o 1.14000010490417 mini\perlio.o 1.125 mini\hv.o 1.10899996757507 mini\dump.o 1.09400010108948 mini\perl.o 1.0939998626709 mini\win32.o 1.07800006866455 mini\mg.o 1.04699993133545 mini\util.o 0.875 mini\av.o 0.812999963760376 ..\doio.o 0.812000036239624 ..\universal.o 0.811999797821045 ..\pad.o 0.796999931335449 ..\git_version.h 0.765000104904175 mini\universal.o 0.75 mini\doio.o 0.75 ..\perl522.dll 0.735000133514404 ..\mathoms.o 0.734999895095825 mini\pad.o 0.733999967575073 ..\libperl522.a 0.719000101089478 mini\mathoms.o 0.717999935150146 ..\mro_core.o 0.703000068664551 ..\perly.o 0.703000068664551 ..\keywords.o 0.687999963760376 mini\mro_core.o 0.687000036239624 ..\pp_sort.o 0.686999797821045 mini\keywords.o 0.671000003814697 ..\doop.o 0.655999898910522 mini\pp_sort.o 0.655999898910522 mini\perly.o 0.655999898910522 mini\doop.o 0.625 ..\scope.o 0.594000101089478 mini\scope.o 0.562000036239624 ..\numeric.o 0.546999931335449 mini\numeric.o 0.5 ..\av.o 0.483999967575073 ..\locale.o 0.467999935150146 ..\lib\buildcustomize.pl 0.453000068664551 win32sck.o 0.453000068664551 mini\locale.o 0.452999830245972 mini\win32sck.o 0.344000101089478 ..\lib\buildcustomize.pl 0.328000068664551 ..\perl522.dll 0.328000068664551 ..\perl522.dll 0.328000068664551 ..\deb.o 0.297000169754028 mini\win32io.o 0.296999931335449 win32io.o 0.282000064849854 ..\globals.o 0.266000032424927 mini\taint.o 0.266000032424927 mini\globals.o 0.266000032424927 ..\taint.o 0.25 mini\miniperlmain.o 0.25 ..\run.o 0.235000133514404 mini\deb.o 0.233999967575073 win32thread.o 0.233999967575073 mini\win32thread.o 0.233999967575073 mini\run.o 0.233999967575073 mini\reentr.o 0.233999967575073 mini\perlapi.o 0.233999967575073 mini\caretx.o 0.233999967575073 ..\reentr.o 0.233999967575073 ..\perlapi.o 0.233999967575073 ..\caretx.o 0.187000036239624 ..\generate_uudmap.exe 0.171000003814697 ..\lib\Config.pm 0.141000032424927 mini\fcrypt.o 0.141000032424927 fcrypt.o 0.139999866485596 ..\lib\Config.pm 0.125 perldll.def 0.108999967575073 ..\lib\CORE\ppport.h 0.0940001010894775 ..\generate_uudmap.o 0.0780000686645508 ..\config.sh 0.0320000648498535 ..\libperl522.a 0.0320000648498535 ..\lib\Config.pm 0.0309998989105225 perllibst.h 0.0309998989105225 Extensions_static 0.0309998989105225 ..\bitcount.h 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 config.w32 0.0160000324249268 ..\lib\CORE\ppport.h 0.0150001049041748 mini\.exists 0.0149998664855957 mini\.exists 0.0149998664855957 mini\.exists 0 mini\.exists 0 __.NULLPRQ 0 .TARGETS 0 .ROOT 0 .INIT 0 .DONE
Subject: vc7nogltime.txt
Download vc7nogltime.txt
text/plain 4.4k
3.78200006484985 ..\DynaLoader.obj 3.6399998664856 ..\toke.obj 2.875 mini\toke.obj 2.125 ..\regexec.obj 1.9539999961853 ..\regcomp.obj 1.85899996757507 mini\regcomp.obj 1.84400010108948 mini\regexec.obj 1.7649998664856 Extensions_static 1.1560001373291 ..\sv.obj 1 ..\op.obj 0.906000137329102 mini\op.obj 0.891000032424927 mini\sv.obj 0.780999898910522 ..\git_version.h 0.719000101089478 ..\pp_pack.obj 0.717999935150146 ..\pp.obj 0.703000068664551 mini\pp_pack.obj 0.687999963760376 ..\pp_ctl.obj 0.671999931335449 mini\pp.obj 0.625 mini\pp_ctl.obj 0.546999931335449 ..\dump.obj 0.532000064849854 mini\dump.obj 0.5 ..\pp_sys.obj 0.469000101089478 ..\pp_hot.obj 0.453000068664551 mini\pp_hot.obj 0.453000068664551 ..\lib\buildcustomize.pl 0.422000169754028 ..\gv.obj 0.406000137329102 mini\gv.obj 0.405999898910522 mini\pp_sys.obj 0.375 ..\perl.obj 0.359999895095825 ..\mg.obj 0.344000101089478 mini\mg.obj 0.328999996185303 ..\perly.obj 0.328000068664551 mini\utf8.obj 0.328000068664551 ..\utf8.obj 0.328000068664551 ..\hv.obj 0.327999830245972 win32.obj 0.327999830245972 mini\perl.obj 0.327999830245972 ..\util.obj 0.312000036239624 mini\perly.obj 0.311999797821045 mini\hv.obj 0.297000169754028 ..\perlio.obj 0.297000169754028 ..\keywords.obj 0.296999931335449 perllib.obj 0.296999931335449 mini\win32.obj 0.296999931335449 mini\util.obj 0.296999931335449 ..\mro_core.obj 0.282000064849854 mini\mro_core.obj 0.282000064849854 mini\keywords.obj 0.282000064849854 ..\universal.obj 0.281000137329102 mini\perlio.obj 0.281000137329102 ..\doio.obj 0.280999898910522 ..\pp_sort.obj 0.280999898910522 ..\pad.obj 0.266000032424927 mini\universal.obj 0.266000032424927 mini\pp_sort.obj 0.266000032424927 mini\pad.obj 0.266000032424927 mini\doio.obj 0.264999866485596 ..\perl522.dll 0.233999967575073 mini\doop.obj 0.233999967575073 ..\scope.obj 0.233999967575073 ..\doop.obj 0.219000101089478 mini\scope.obj 0.219000101089478 ..\lib\buildcustomize.pl 0.217999935150146 mini\mathoms.obj 0.217999935150146 ..\mathoms.obj 0.203000068664551 mini\locale.obj 0.203000068664551 ..\numeric.obj 0.203000068664551 ..\av.obj 0.202999830245972 mini\numeric.obj 0.202999830245972 mini\av.obj 0.202999830245972 ..\locale.obj 0.172000169754028 win32io.obj 0.172000169754028 mini\win32sck.obj 0.171999931335449 win32sck.obj 0.171999931335449 mini\globals.obj 0.171999931335449 ..\globals.obj 0.157000064849854 ..\taint.obj 0.156000137329102 ..\reentr.obj 0.155999898910522 mini\win32io.obj 0.155999898910522 mini\taint.obj 0.155999898910522 mini\miniperlmain.obj 0.155999898910522 ..\run.obj 0.155999898910522 ..\deb.obj 0.141000032424927 win32thread.obj 0.141000032424927 mini\reentr.obj 0.141000032424927 mini\perlapi.obj 0.141000032424927 mini\caretx.obj 0.141000032424927 ..\lib\Config.pm 0.141000032424927 ..\lib\Config.pm 0.141000032424927 ..\caretx.obj 0.140000104904175 mini\deb.obj 0.140000104904175 ..\perlapi.obj 0.139999866485596 mini\win32thread.obj 0.139999866485596 mini\run.obj 0.108999967575073 perldll.def 0.108999967575073 ..\config.sh 0.0939998626708984 ..\lib\CORE\ppport.h 0.062999963760376 mini\fcrypt.obj 0.062000036239624 fcrypt.obj 0.0470001697540283 perllibst.h 0.0469999313354492 ..\generate_uudmap.obj 0.0320000648498535 mini\.exists 0.0320000648498535 Extensions_static 0.0310001373291016 ..\perl522.lib 0.0310001373291016 ..\generate_uudmap.exe 0.0309998989105225 ..\perl522.lib 0.0309998989105225 ..\lib\Config.pm 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 config.w32 0.0160000324249268 ..\generate_uudmap.exe 0.0150001049041748 mini\.exists 0.0150001049041748 mini\.exists 0.0150001049041748 ..\perl522.dll 0.0150001049041748 ..\lib\CORE\ppport.h 0.0149998664855957 mini\.exists 0.0149998664855957 mini\.exists 0.0149998664855957 ..\lib\buildcustomize.pl 0.0149998664855957 ..\bitcount.h 0 __.NULLPRQ 0 .TARGETS 0 .ROOT 0 .INIT 0 .DONE
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 290b
On Sat May 16 12:18:09 2015, bulk88 wrote: Show quoted text
> Adding compile timings for archive reasons, this is to change from
Adding more timings, these were done on a different, probably slower machine than the last set of timings. Includes times from VC 2013/12.0. -- bulk88 ~ bulk88 at hotmail.com
Subject: gcc483time.txt
Download gcc483time.txt
text/plain 4.1k
8.48699998855591 ..\toke.o 7.72200012207031 mini\toke.o 6.92700004577637 ..\regcomp.o 6.61400008201599 mini\regcomp.o 5.72499990463257 ..\op.o 5.30399990081787 mini\op.o 5.02300000190735 ..\sv.o 4.77400016784668 ..\regexec.o 4.61800003051758 mini\regexec.o 4.04100012779236 mini\sv.o 4.0239999294281 ..\DynaLoader.o 3.82200002670288 ..\pp.o 3.63499999046326 mini\pp.o 3.32299995422363 ..\pp_ctl.o 3.15100002288818 mini\pp_ctl.o 2.65199995040894 ..\pp_sys.o 2.27799987792969 ..\pp_pack.o 2.26200008392334 ..\pp_hot.o 2.23100018501282 mini\pp_pack.o 2.16799998283386 mini\pp_sys.o 2.15300011634827 Extensions_static 2.15299987792969 mini\pp_hot.o 2.13699984550476 ..\perl522.dll 2.02799987792969 perllib.o 1.96499991416931 ..\gv.o 1.91899991035461 mini\gv.o 1.90300011634827 ..\libperl522.a 1.77900004386902 ..\perlio.o 1.73200011253357 ..\perl.o 1.71599984169006 ..\utf8.o 1.66900014877319 win32.o 1.65300011634827 mini\utf8.o 1.62200021743774 ..\util.o 1.62199997901917 mini\perlio.o 1.59100008010864 ..\hv.o 1.57500004768372 ..\dump.o 1.55999994277954 ..\mg.o 1.52900004386902 mini\perl.o 1.52900004386902 mini\dump.o 1.51399993896484 mini\win32.o 1.51399993896484 mini\hv.o 1.48199987411499 mini\util.o 1.48199987411499 mini\mg.o 1.23200011253357 ..\perl522.dll 1.1230001449585 ..\universal.o 1.12299990653992 mini\av.o 1.12299990653992 ..\pad.o 1.10800004005432 ..\doio.o 1.07699990272522 mini\universal.o 1.06100010871887 mini\doio.o 1.02999997138977 ..\mathoms.o 1.02900004386902 mini\pad.o 0.999000072479248 ..\mro_core.o 0.997999906539917 ..\perly.o 0.983000040054321 mini\mathoms.o 0.982000112533569 mini\mro_core.o 0.950999975204468 mini\perly.o 0.950999975204468 ..\pp_sort.o 0.935999870300293 mini\pp_sort.o 0.935999870300293 ..\doop.o 0.921000003814697 mini\keywords.o 0.920000076293945 ..\keywords.o 0.919999837875366 mini\doop.o 0.889000177383423 ..\scope.o 0.858000040054321 mini\scope.o 0.811000108718872 ..\numeric.o 0.795000076293945 mini\numeric.o 0.749000072479248 ..\lib\buildcustomize.pl 0.718000173568726 ..\av.o 0.70199990272522 win32sck.o 0.70199990272522 ..\locale.o 0.670000076293945 mini\win32sck.o 0.670000076293945 mini\locale.o 0.561999797821045 ..\generate_uudmap.exe 0.483999967575073 mini\globals.o 0.467999935150146 win32io.o 0.467999935150146 mini\win32io.o 0.453000068664551 mini\caretx.o 0.453000068664551 ..\globals.o 0.436999797821045 ..\taint.o 0.421000003814697 mini\taint.o 0.421000003814697 mini\deb.o 0.406000137329102 mini\miniperlmain.o 0.404999971389771 ..\deb.o 0.390000104904175 win32thread.o 0.390000104904175 ..\reentr.o 0.390000104904175 ..\perl522.dll 0.389999866485596 ..\run.o 0.389999866485596 ..\caretx.o 0.375 mini\win32thread.o 0.375 mini\reentr.o 0.374000072479248 ..\perlapi.o 0.373999834060669 mini\run.o 0.358999967575073 mini\perlapi.o 0.218999862670898 ..\lib\buildcustomize.pl 0.187000036239624 fcrypt.o 0.156000137329102 ..\lib\Config.pm 0.155999898910522 perldll.def 0.155999898910522 mini\fcrypt.o 0.140000104904175 ..\lib\Config.pm 0.140000104904175 ..\generate_uudmap.o 0.0939998626708984 ..\config.sh 0.0469999313354492 ..\libperl522.a 0.0469999313354492 ..\lib\CORE\ppport.h 0.0319998264312744 Extensions_static 0.0310001373291016 ..\bitcount.h 0.0309998989105225 perllibst.h 0.0309998989105225 ..\git_version.h 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 ..\lib\CORE\ppport.h 0.0160000324249268 ..\lib\Config.pm 0.0159997940063477 mini\.exists 0.0150001049041748 mini\.exists 0.0150001049041748 config.w32 0.0149998664855957 mini\.exists 0 mini\.exists 0 mini\.exists 0 mini\.exists 0 mini\.exists 0 mini\.exists 0 __.NULLPRQ 0 .TARGETS 0 .ROOT 0 .INIT 0 .DONE
Subject: vc10time.txt
Download vc10time.txt
text/plain 4.4k
4.30599999427795 ..\toke.obj 3.66599988937378 mini\toke.obj 3.55699992179871 ..\DynaLoader.obj 2.49600005149841 ..\regcomp.obj 2.41799998283386 ..\regexec.obj 2.35500001907349 mini\regcomp.obj 2.19999980926514 mini\regexec.obj 1.71600008010864 Extensions_static 1.59200000762939 ..\sv.obj 1.41899991035461 ..\op.obj 1.29500007629395 mini\op.obj 1.23300004005432 mini\sv.obj 1.07599997520447 ..\pp.obj 0.997999906539917 mini\pp.obj 0.983000040054321 ..\pp_pack.obj 0.967999935150146 ..\pp_ctl.obj 0.920000076293945 mini\pp_pack.obj 0.888999938964844 mini\pp_ctl.obj 0.795000076293945 ..\pp_sys.obj 0.733000040054321 ..\pp_hot.obj 0.687000036239624 ..\dump.obj 0.671000003814697 mini\pp_hot.obj 0.671000003814697 mini\dump.obj 0.671000003814697 ..\gv.obj 0.639999866485596 perllib.obj 0.638999938964844 mini\pp_sys.obj 0.623999834060669 mini\gv.obj 0.608999967575073 ..\perl.obj 0.593000173568726 win32.obj 0.577000141143799 ..\util.obj 0.577000141143799 ..\hv.obj 0.57699990272522 ..\mg.obj 0.562000036239624 ..\utf8.obj 0.562000036239624 ..\perlio.obj 0.562000036239624 ..\keywords.obj 0.546000003814697 mini\perl.obj 0.531000137329102 mini\mg.obj 0.529999971389771 mini\win32.obj 0.529999971389771 mini\utf8.obj 0.529999971389771 mini\keywords.obj 0.514999866485596 mini\util.obj 0.514999866485596 mini\hv.obj 0.499000072479248 mini\perlio.obj 0.499000072479248 ..\perly.obj 0.499000072479248 ..\pad.obj 0.498999834060669 ..\universal.obj 0.483999967575073 ..\mro_core.obj 0.483999967575073 ..\doio.obj 0.468000173568726 mini\perly.obj 0.467999935150146 mini\doio.obj 0.467999935150146 ..\pp_sort.obj 0.453000068664551 ..\perl522.dll 0.452000141143799 mini\universal.obj 0.452000141143799 mini\mro_core.obj 0.437000036239624 mini\pad.obj 0.437000036239624 ..\mathoms.obj 0.437000036239624 ..\doop.obj 0.421999931335449 mini\pp_sort.obj 0.421000003814697 ..\scope.obj 0.406000137329102 ..\numeric.obj 0.390000104904175 mini\scope.obj 0.390000104904175 ..\av.obj 0.389999866485596 mini\mathoms.obj 0.389999866485596 mini\doop.obj 0.389999866485596 ..\lib\buildcustomize.pl 0.375 win32sck.obj 0.373999834060669 mini\numeric.obj 0.373999834060669 ..\locale.obj 0.359000205993652 mini\locale.obj 0.358999967575073 mini\av.obj 0.344000101089478 mini\win32sck.obj 0.342999935150146 win32io.obj 0.328000068664551 mini\globals.obj 0.327000141143799 ..\taint.obj 0.32699990272522 ..\globals.obj 0.312000036239624 win32thread.obj 0.312000036239624 mini\win32io.obj 0.312000036239624 mini\taint.obj 0.312000036239624 ..\run.obj 0.312000036239624 ..\perlapi.obj 0.312000036239624 ..\deb.obj 0.311999797821045 ..\reentr.obj 0.311999797821045 ..\caretx.obj 0.297000169754028 mini\reentr.obj 0.296999931335449 mini\miniperlmain.obj 0.296999931335449 mini\deb.obj 0.281000137329102 mini\run.obj 0.280999898910522 mini\perlapi.obj 0.279999971389771 mini\win32thread.obj 0.279999971389771 mini\caretx.obj 0.25 ..\lib\buildcustomize.pl 0.156000137329102 ..\lib\Config.pm 0.155999898910522 perldll.def 0.155999898910522 ..\lib\Config.pm 0.155999898910522 ..\config.sh 0.139999866485596 ..\lib\CORE\ppport.h 0.125 ..\generate_uudmap.exe 0.0780000686645508 mini\fcrypt.obj 0.0780000686645508 fcrypt.obj 0.0779998302459717 ..\perl522.dll 0.0630002021789551 ..\generate_uudmap.obj 0.062000036239624 ..\perl522.lib 0.0469999313354492 ..\lib\buildcustomize.pl 0.0460000038146973 Extensions_static 0.0320000648498535 ..\lib\Config.pm 0.0310001373291016 ..\perl522.lib 0.0310001373291016 ..\generate_uudmap.exe 0.0309998989105225 perllibst.h 0.0309998989105225 ..\git_version.h 0.0309998989105225 ..\bitcount.h 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0160000324249268 ..\lib\CORE\ppport.h 0.0150001049041748 mini\.exists 0.0149998664855957 mini\.exists 0 mini\.exists 0 mini\.exists 0 mini\.exists 0 mini\.exists 0 mini\.exists 0 config.w32 0 __.NULLPRQ 0 .TARGETS 0 .ROOT 0 .INIT 0 .DONE
Subject: vc12time.txt
Download vc12time.txt
text/plain 4.4k
4.1029999256134 ..\toke.obj 3.63499999046326 mini\toke.obj 3.19799995422363 ..\regexec.obj 3.04199981689453 mini\regexec.obj 2.12100005149841 ..\regcomp.obj 1.95000004768372 mini\regcomp.obj 1.93499994277954 ..\DynaLoader.obj 1.59100008010864 ..\sv.obj 1.40400004386902 Extensions_static 1.38900017738342 ..\op.obj 1.32599997520447 mini\sv.obj 1.15499997138977 mini\op.obj 1.04499983787537 ..\pp.obj 1.01399993896484 ..\pp_pack.obj 0.952000141143799 ..\pp_ctl.obj 0.889000177383423 mini\pp.obj 0.872999906539917 mini\pp_pack.obj 0.827000141143799 ..\dump.obj 0.794999837875366 mini\pp_ctl.obj 0.765000104904175 mini\dump.obj 0.765000104904175 ..\gv.obj 0.718000173568726 ..\keywords.obj 0.70199990272522 mini\gv.obj 0.685999870300293 mini\keywords.obj 0.685999870300293 ..\pp_sys.obj 0.671000003814697 ..\perl522.dll 0.654999971389771 ..\pp_hot.obj 0.608999967575073 ..\perly.obj 0.608000040054321 ..\hv.obj 0.607999801635742 perllib.obj 0.607999801635742 ..\util.obj 0.593000173568726 mini\pp_hot.obj 0.592000007629395 ..\mg.obj 0.578000068664551 win32.obj 0.577999830245972 mini\perly.obj 0.57699990272522 mini\pp_sys.obj 0.57699990272522 ..\perl.obj 0.546000003814697 ..\utf8.obj 0.546000003814697 ..\perlio.obj 0.546000003814697 ..\lib\buildcustomize.pl 0.530999898910522 mini\hv.obj 0.513999938964844 mini\perl.obj 0.513999938964844 mini\mg.obj 0.5 mini\win32.obj 0.5 mini\utf8.obj 0.499000072479248 mini\mro_core.obj 0.499000072479248 ..\scope.obj 0.499000072479248 ..\mro_core.obj 0.498999834060669 mini\util.obj 0.498999834060669 ..\doio.obj 0.483000040054321 mini\perlio.obj 0.468000173568726 ..\universal.obj 0.468000173568726 ..\pp_sort.obj 0.453000068664551 mini\pp_sort.obj 0.45199990272522 ..\pad.obj 0.437000036239624 mini\pad.obj 0.437000036239624 mini\doio.obj 0.437000036239624 ..\numeric.obj 0.437000036239624 ..\doop.obj 0.421999931335449 ..\locale.obj 0.421000003814697 mini\universal.obj 0.421000003814697 ..\mathoms.obj 0.421000003814697 ..\av.obj 0.406000137329102 mini\mathoms.obj 0.404999971389771 mini\scope.obj 0.404999971389771 mini\doop.obj 0.390000104904175 mini\numeric.obj 0.390000104904175 mini\locale.obj 0.374000072479248 win32sck.obj 0.358999967575073 mini\win32sck.obj 0.358999967575073 mini\av.obj 0.357999801635742 ..\globals.obj 0.343999862670898 ..\run.obj 0.343999862670898 ..\caretx.obj 0.343000173568726 ..\perlapi.obj 0.343000173568726 ..\deb.obj 0.342999935150146 win32thread.obj 0.342999935150146 win32io.obj 0.342999935150146 mini\win32io.obj 0.342999935150146 mini\globals.obj 0.328000068664551 mini\taint.obj 0.328000068664551 ..\reentr.obj 0.32699990272522 mini\deb.obj 0.32699990272522 ..\taint.obj 0.312000036239624 mini\win32thread.obj 0.312000036239624 mini\run.obj 0.312000036239624 mini\reentr.obj 0.312000036239624 mini\perlapi.obj 0.312000036239624 mini\miniperlmain.obj 0.312000036239624 mini\caretx.obj 0.248999834060669 ..\lib\buildcustomize.pl 0.156000137329102 ..\lib\Config.pm 0.155999898910522 ..\generate_uudmap.exe 0.141000032424927 ..\lib\Config.pm 0.140000104904175 perldll.def 0.108999967575073 ..\config.sh 0.0939998626708984 mini\fcrypt.obj 0.0929999351501465 ..\generate_uudmap.obj 0.0780000686645508 fcrypt.obj 0.062999963760376 ..\lib\CORE\ppport.h 0.062000036239624 ..\perl522.dll 0.0470001697540283 ..\lib\buildcustomize.pl 0.0469999313354492 ..\perl522.lib 0.0320000648498535 ..\generate_uudmap.exe 0.0310001373291016 perllibst.h 0.0310001373291016 ..\git_version.h 0.0310001373291016 ..\bitcount.h 0.0309998989105225 Extensions_static 0.0309998989105225 ..\perl522.lib 0.0309998989105225 ..\lib\Config.pm 0.0160000324249268 mini\.exists 0.0160000324249268 mini\.exists 0.0159997940063477 mini\.exists 0.0150001049041748 mini\.exists 0.0150001049041748 mini\.exists 0 mini\.exists 0 mini\.exists 0 mini\.exists 0 mini\.exists 0 mini\.exists 0 config.w32 0 __.NULLPRQ 0 .TARGETS 0 .ROOT 0 .INIT 0 .DONE 0 ..\lib\CORE\ppport.h
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 364b
On Sun May 10 20:59:50 2015, bulk88 wrote: Show quoted text
> On Sun Apr 05 15:43:49 2015, bulk88 wrote:
> > Rebased patch attached, includes a small fix for using a 64 bit dmake > > with 32 bit CCs.
> > Revised patch attached, ..\pod\perltoc.pod target got missing deps added.
Do you consider this complete at this point? I'd prefer not to have the modifications in cpan/ Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.6k
On Thu Jun 11 17:33:37 2015, tonyc wrote: Show quoted text
> On Sun May 10 20:59:50 2015, bulk88 wrote:
> > On Sun Apr 05 15:43:49 2015, bulk88 wrote:
> > > Rebased patch attached, includes a small fix for using a 64 bit dmake > > > with 32 bit CCs.
> > > > Revised patch attached, ..\pod\perltoc.pod target got missing deps added.
> > Do you consider this complete at this point? > > I'd prefer not to have the modifications in cpan/ > > Tony
No, I have to publish the version of the patch which reorders the core .c files to save ~ 10-15 seconds out of 1 minute for me, to build perl523.dll. I have to rethink a problem with how dmake picks what targets to build, its not exactly left side dep to right side dep but also "levels" on the tree. Basically I need to start static and dynaloader targets towards the beginning after miniperl is done building since they take quite long to build compared to the other .c'es, but static and dynaloader are 1 level away from core .c'es because static and dynaloader dep on CORE dir, so those 2 always run AFTER all .c files are done (and cores are NOT maxed out). If I make the .c files depend (makefile wise, not CC build wise) on CORE dir, static and dynaloader run first as I want, but "latency" wise, we are making the .c'es depend on something they dont need, delaying the start of .c building, but the time saved by starting dynaloader and static first along with regexp.c/fat .ces saves some seconds on "time until perl523.dll is built (no dyn XS/no non-XS)", so for core hacking putting the .c'es to dep on CORE dir saves time even though its insane that it does (bad dmake design). -- bulk88 ~ bulk88 at hotmail.com
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.8k
After sorting the .c'es by time (I am only moving the >1 second ones), here are the results on my 8 core with "dmake -P8 ../perl523.dll". VC with no -GL 32 seconds time sort 32 seconds time sort 32 seconds time sort 34 seconds no time sort 35 seconds no time sort 35 seconds no time sort GCC 49 seconds time sort 48 seconds time sort 48 seconds time sort 56 seconds not time sort 56 seconds not time sort 56 seconds not time sort So time sorting the .c files has very visible results. The order I decided is MICROCORE_SRC = \ ..\toke.c \ ..\regcomp.c \ ..\regexec.c \ ..\op.c \ ..\sv.c \ ..\pp.c \ ..\pp_ctl.c \ ..\pp_sys.c \ ..\pp_pack.c \ ..\pp_hot.c \ ..\gv.c \ ..\perl.c \ ..\utf8.c \ ..\dump.c \ ..\hv.c \ ..\av.c \ ..\caretx.c \ ..\deb.c \ ..\doio.c \ ..\doop.c \ ..\globals.c \ ..\mro_core.c \ ..\locale.c \ ..\keywords.c \ ..\mathoms.c \ ..\mg.c \ ..\numeric.c \ ..\pad.c \ ..\perlapi.c \ ..\perly.c \ ..\pp_sort.c \ ..\reentr.c \ ..\run.c \ ..\scope.c \ ..\taint.c \ ..\universal.c \ ..\util.c utf8.o took 1.7 seconds to create as from the earlier list. At 1.6 seconds is win32.o, but I'd have break up the organization and meaning of MICROCORE_SRC macro to put win32.c in the middle of it (win32.c is in a different macro called WIN32_SRC). These numbers also mean, that the MAXIMUM penalty, and therefore best case scenario improvement, for not time sorting every last .c file, is 1.6 seconds. DynaLoader and Extensions_static deps take 4.0 and 2.2 seconds to build, and they are currently scheduled at the end due to a poor dmake parallel algo, after all .c files, so the 1.6 sec penalty is dwarfed by poor scheduling of DynaLoader and Extensions_static targets. I will not reorganize putting DynaLoader and Extensions_static first before core .c files are started since I have to make the core .c files depend on fake dependencies for them to have the same deps as DynaLoader and Extensions_static (core .c files dont need /lib/CORE headers, XS/EUMM makefiles do) and I'd rather work on the gmake makefile and see if gmake has the same poor scheduling as dmake before I try such things. Also putting DynaLoader and Extensions_static first only speeds up reaching ../perl523.dll target, reaching test/test-prep target means running Extensions which takes many minutes and makes the wall time to build ../perl523.dll on a multi core machine irrelevant since Extensions is always the last target to finish executing. I've attached the psuedo-branch which is the final version of the parallel patch. The psuedobranch can be benchmarked for the effect of alpha sorting between commit "WIP add parallel support for Win32 dmake building " and "coredir parallelism". Once Tonyc/whoever is fine with it, I'll git squash it all together. -- bulk88 ~ bulk88 at hotmail.com
Subject: 0001-WIP-add-parallel-support-for-Win32-dmake-building.patch

Message body is not shown because it is too large.

Subject: 0002-coredir-parallelism.patch
From 7d629bbeaa1c066a78cf4401b079aa7372fc9236 Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Sun, 17 May 2015 14:18:06 -0400 Subject: [PATCH 2/5] coredir parallelism -defer the mass copy of headers to COREDIR until XS build time and dont use COREDIR for building miniperl interp or perl interp, more thing to do in parallel this way and 1 less path to search by the CC -config_h.PL requires config.h to be in COREDIR, this XCOPY in CONFIGPM targ is where COREDIR is created if it doesnt exist already -remove a bunch of XCOPY'es, copy the headers in 1 place only, and dont copy them unless they change based on timestamps -the DYNALOADER dep already contains "..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) $(HAVE_COREDIR)" dont specify them again in Extensions and Extensions_reonly, this probably reduces the size of dep graph in dmake, sincedmake explodes the graph and checks the outline of the graph/tree very inefficiently in parallel mode (architectural problem with dmake) -undo Time::HiRes in-core patch per TonyC's 2 line ppport.h idea -delete pdb files in root, otherwise switching VC version will make the next cl.exe error out that it can't write to the pdb file due to version numbers in the pdb file -reorder core .c files in a blend of VC and GCC time order, this reduces a 1:08 mins time to do "dmake -P9 ../perl522.dll" to 0:54 mins on 8 core with GCC, see details in #123867 --- Porting/Maintainers.pl | 2 -- README.win32 | 11 +++++--- cpan/Time-HiRes/HiRes.pm | 2 +- cpan/Time-HiRes/HiRes.xs | 4 +-- pod/perldelta.pod | 5 ++++ t/porting/customized.dat | 1 - win32/makefile.mk | 72 +++++++++++++++++++++++++++--------------------- 7 files changed, 55 insertions(+), 42 deletions(-) diff --git a/Porting/Maintainers.pl b/Porting/Maintainers.pl index 7de1d68..ac1e3c2 100755 --- a/Porting/Maintainers.pl +++ b/Porting/Maintainers.pl @@ -1216,8 +1216,6 @@ use File::Glob qw(:case); 'Time::HiRes' => { 'DISTRIBUTION' => 'ZEFRAM/Time-HiRes-1.9726.tar.gz', 'FILES' => q[cpan/Time-HiRes], - # CPAN release eventually - 'CUSTOMIZED' => [ 'HiRes.xs' ], }, 'Time::Local' => { diff --git a/README.win32 b/README.win32 index b9875e6..13d2a79 100644 --- a/README.win32 +++ b/README.win32 @@ -98,11 +98,14 @@ See L<Usage Hints for Perl on Windows> below for general hints about this. You need a "make" program to build the sources. If you are using Visual C++ or the Windows SDK tools, you can use nmake supplied with Visual C++ or Windows SDK. You may also use, for Visual C++ or Windows SDK, dmake instead -of nmake. dmake is open source software, but is not included with Visual C++ or -Windows SDK. If you want parallel building with Visual C++ or +of nmake. dmake is open source software, but is not included with Visual C++ or +Windows SDK. If you want parallel building with Visual C++ or Windows SDK you must use dmake instead of nmake. Builds using gcc need dmake. -nmake is not supported for gcc builds. gmake is not supported, but might be -added in the future. +nmake is not supported for gcc builds. gmake is not supported, but might be +added in the future. It is recommended to use dmake 4.13 or newer for parallel +building. Older dmakes, in parallel mode, have very high CPU usage and pound +the disk/filing system with duplicate I/O calls in an aggressive polling +loop. A port of dmake for Windows is available from: diff --git a/cpan/Time-HiRes/HiRes.pm b/cpan/Time-HiRes/HiRes.pm index 0ee2c62..cf64bc1 100644 --- a/cpan/Time-HiRes/HiRes.pm +++ b/cpan/Time-HiRes/HiRes.pm @@ -23,7 +23,7 @@ our @EXPORT_OK = qw (usleep sleep ualarm alarm gettimeofday time tv_interval stat lstat ); -our $VERSION = '1.9726_01'; +our $VERSION = '1.9726'; our $XS_VERSION = $VERSION; $VERSION = eval $VERSION; diff --git a/cpan/Time-HiRes/HiRes.xs b/cpan/Time-HiRes/HiRes.xs index 9a852c4..96640e9 100644 --- a/cpan/Time-HiRes/HiRes.xs +++ b/cpan/Time-HiRes/HiRes.xs @@ -731,7 +731,7 @@ myNVtime() static void hrstatns(UV *atime_nsec, UV *mtime_nsec, UV *ctime_nsec) { - dTHX; + dTHXR; #if TIME_HIRES_STAT == 1 *atime_nsec = PL_statcache.st_atimespec.tv_nsec; *mtime_nsec = PL_statcache.st_mtimespec.tv_nsec; @@ -1284,7 +1284,7 @@ PROTOTYPE: ;$ fakeop.op_flags = GIMME_V == G_ARRAY ? OPf_WANT_LIST : GIMME_V == G_SCALAR ? OPf_WANT_SCALAR : OPf_WANT_VOID; PL_op = &fakeop; - (void)fakeop.op_ppaddr(aTHX); + (void)fakeop.op_ppaddr(aTHXR); SPAGAIN; LEAVE; nret = SP+1 - &ST(0); diff --git a/pod/perldelta.pod b/pod/perldelta.pod index 2cb9d72..95a98b7 100644 --- a/pod/perldelta.pod +++ b/pod/perldelta.pod @@ -378,6 +378,11 @@ only execute on Vista and higher. =item * +Parallel building has been added to the dmake C<makefile.mk> makefile. All +Win32 compilers are supported. + +=item * + You can now build perl with GNU Make and GCC. [perl #123440] =item * diff --git a/t/porting/customized.dat b/t/porting/customized.dat index 974bf57..9e0bd1c 100644 --- a/t/porting/customized.dat +++ b/t/porting/customized.dat @@ -50,7 +50,6 @@ Win32API::File cpan/Win32API-File/t/file.t 124e64aa77e755235eb297644a87fac5388d3 Win32API::File cpan/Win32API-File/t/tie.t 712ea7edd0cc805ce1c0b8172c01b03dd19b583d Win32API::File cpan/Win32API-File/typemap 24bff088babeadac0873e8df390d1666d9d9db4a autodie cpan/autodie/t/mkdir.t 9e70d2282a3cc7d76a78bf8144fccba20fb37dac -Time::HiRes cpan/Time-HiRes/HiRes.xs 0a850b6dee1f2c60ade3d4c33fab7c69b02ddc45 podlators cpan/podlators/scripts/pod2man.PL f81acf53f3ff46cdcc5ebdd661c5d13eb35d20d6 podlators cpan/podlators/scripts/pod2text.PL b4693fcfe4a0a1b38a215cfb8985a65d5d025d69 version cpan/version/lib/version.pm d0923b895d57f1d669ae36fcf85c87b16db341d1 diff --git a/win32/makefile.mk b/win32/makefile.mk index 656522f..59c8819 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -472,7 +472,7 @@ a = .a # Options # -INCLUDES = -I.\include -I. -I.. -I$(COREDIR) +INCLUDES = -I.\include -I. -I.. DEFINES = -DWIN32 .IF "$(WIN64)" == "define" DEFINES += -DWIN64 -DCONSERVATIVE @@ -560,7 +560,7 @@ RSC = rc # Options # -INCLUDES = -I$(COREDIR) -I.\include -I. -I.. +INCLUDES = -I.\include -I. -I.. #PCHFLAGS = -Fpc:\temp\vcmoduls.pch -YX DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT LOCDEFS = -DPERLDLL -DPERL_CORE @@ -863,43 +863,45 @@ XCOPY = xcopy /f /r /i /d /y RCOPY = xcopy /f /r /i /e /d /y NOOP = @rem +#first ones are arrange in compile time order for faster parallel building +#see #123867 for details MICROCORE_SRC = \ + ..\toke.c \ + ..\regcomp.c \ + ..\regexec.c \ + ..\op.c \ + ..\sv.c \ + ..\pp.c \ + ..\pp_ctl.c \ + ..\pp_sys.c \ + ..\pp_pack.c \ + ..\pp_hot.c \ + ..\gv.c \ + ..\perl.c \ + ..\utf8.c \ + ..\dump.c \ + ..\hv.c \ ..\av.c \ ..\caretx.c \ ..\deb.c \ ..\doio.c \ ..\doop.c \ - ..\dump.c \ ..\globals.c \ - ..\gv.c \ ..\mro_core.c \ - ..\hv.c \ ..\locale.c \ ..\keywords.c \ ..\mathoms.c \ ..\mg.c \ ..\numeric.c \ - ..\op.c \ ..\pad.c \ - ..\perl.c \ ..\perlapi.c \ ..\perly.c \ - ..\pp.c \ - ..\pp_ctl.c \ - ..\pp_hot.c \ - ..\pp_pack.c \ ..\pp_sort.c \ - ..\pp_sys.c \ ..\reentr.c \ - ..\regcomp.c \ - ..\regexec.c \ ..\run.c \ ..\scope.c \ - ..\sv.c \ ..\taint.c \ - ..\toke.c \ ..\universal.c \ - ..\utf8.c \ ..\util.c EXTRACORE_SRC += perllib.c @@ -964,6 +966,10 @@ UUDMAP_H = ..\uudmap.h BITCOUNT_H = ..\bitcount.h MG_DATA_H = ..\mg_data.h GENERATED_HEADERS = $(UUDMAP_H) $(BITCOUNT_H) $(MG_DATA_H) +#a stub ppport.h must be generated so building XS modules, .c->.obj wise, will +#work, so this target also represents creating the COREDIR and filling it +HAVE_COREDIR = $(COREDIR)\ppport.h + MICROCORE_OBJ = $(MICROCORE_SRC:db:+$(o)) CORE_OBJ = $(MICROCORE_OBJ) $(EXTRACORE_SRC:db:+$(o)) @@ -1101,9 +1107,7 @@ regen_config_h: $(CONFIGPM): ..\config.sh config_h.PL $(MINIPERL) -I..\lib ..\configpm --chdir=.. - $(XCOPY) ..\*.h $(COREDIR)\*.* - $(XCOPY) *.h $(COREDIR)\*.* - $(RCOPY) include $(COREDIR)\*.* + $(XCOPY) config.h $(COREDIR)\*.* $(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" \ || $(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE) @@ -1379,7 +1383,7 @@ $(PERLSTATICLIB): $(PERLDLL_OBJ) Extensions_static $(LIB32) $(LIB_FLAGS) -out:$@ @Extensions_static \ @$(mktmp $(PERLDLL_OBJ)) .ENDIF - $(XCOPY) $(PERLSTATICLIB) $(COREDIR) + $(XCOPY) $(PERLSTATICLIB) $(COREDIR)\$(NULL) $(PERLEXE_RES): perlexe.rc $(PERLEXE_MANIFEST) $(PERLEXE_ICO) @@ -1390,6 +1394,14 @@ $(UUDMAP_H) $(MG_DATA_H) : $(BITCOUNT_H) $(BITCOUNT_H) : $(GENUUDMAP) $(GENUUDMAP) $(GENERATED_HEADERS) +#This generates a stub ppport.h & creates & fills /lib/CORE to allow for XS +#building .c->.obj wise (linking is a different thing). This taget is AKA +#$(HAVE_COREDIR). ppport.h can become a 0 byte file once core's Time::HiRes is +#has the patch in CPAN #102799 +$(COREDIR)\ppport.h : $(CORE_H) + $(XCOPY) *.h $(COREDIR)\*.* && $(RCOPY) include $(COREDIR)\*.* && $(XCOPY) ..\*.h $(COREDIR)\*.* + @( echo #define aTHXR aTHX&& echo #define dTHXR dTHX) >$@ + $(GENUUDMAP_OBJ) : ..\mg_raw.h $(GENUUDMAP) : $(GENUUDMAP_OBJ) @@ -1438,18 +1450,15 @@ $(PERLEXESTATIC): $(PERLSTATICLIB) $(CONFIGPM) $(PERLEXEST_OBJ) $(PERLEXE_RES) #------------------------------------------------------------------------------- # There's no direct way to mark a dependency on # DynaLoader.pm, so this will have to do -Extensions : ..\make_ext.pl ..\lib\buildcustomize.pl $(PERLDEP) $(CONFIGPM) $(DYNALOADER) - $(XCOPY) ..\*.h $(COREDIR)\*.* - if not exist $(COREDIR)\ppport.h rem. > $(COREDIR)\ppport.h + +#most of deps of this target are in DYNALOADER and therefore omitted here +Extensions : $(PERLDEP) $(DYNALOADER) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic -Extensions_reonly : ..\make_ext.pl ..\lib\buildcustomize.pl $(PERLDEP) $(CONFIGPM) $(DYNALOADER) - $(XCOPY) ..\*.h $(COREDIR)\*.* +Extensions_reonly : $(PERLDEP) $(DYNALOADER) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re -Extensions_static : ..\make_ext.pl ..\lib\buildcustomize.pl list_static_libs.pl $(CONFIGPM) - $(XCOPY) ..\*.h $(COREDIR)\*.* - if not exist $(COREDIR)\ppport.h rem. > $(COREDIR)\ppport.h +Extensions_static : ..\make_ext.pl ..\lib\buildcustomize.pl list_static_libs.pl $(CONFIGPM) $(HAVE_COREDIR) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static $(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static @@ -1457,8 +1466,7 @@ Extensions_nonxs : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) ..\pod\pe $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs #lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building -$(DYNALOADER) : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) - $(XCOPY) ..\*.h $(COREDIR)\*.* +$(DYNALOADER) : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) $(HAVE_COREDIR) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib Extensions_clean : @@ -1757,7 +1765,7 @@ _clean : -@erase ..\*$(o) ..\*$(a) ..\*.exp *$(o) *$(a) *.exp *.res -@erase ..\t\*.exe ..\t\*.dll ..\t\*.bat -@erase *.ilk - -@erase *.pdb + -@erase *.pdb ..\*.pdb -@erase Extensions_static -- 1.8.0.msysgit.0
Subject: 0003-squash-double-dmake-run.patch
From c09929acb70a27eb6213d0fd057a981844fdb7f8 Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Tue, 23 Jun 2015 00:03:04 -0400 Subject: [PATCH 3/5] squash double dmake run -in $(CONFIGPM) target, remove config_h.PL running dmake again to rebuild $(CONFIGPM) target, configpm the script does not use config.h as input. The $(CONFIGPM) target is a choke/serialization point for parallel building, so it is high value to make it as fast as possible. Maybe a long time ago, configpm read config.h but it doesn't anymore. Nearly all the dynamic config vars are determined in config_sh.PL script and ..\config.sh target. config_h.PL contains very little logic, and this logic is only for config.h, and only when you want a var to be different in config.h than it is in Config.pm/Config_heavy.pl. Putting a breakpoint ("system 'pause';") in config_h.PL, copying Config.pm/Config_heavy.pl to ".old" versions, continuing execution in config_h.PL, then diffing the 2 new Configs with the old 2 Configs before pre-config_h.PL and "$(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE)" shows no differece. When dmake runs needlessly a 2nd time it emits a warning that nothing changed. This is intentional, since configpm doesn't update the timestamps /update the tiles if the Config files are identical to the old ones to prevent mass clean+rebuild of all modules when nothing global changed. dmake: Warning: -- Target [..\lib\Config.pm] was made but the time stamp has not been updated. --- win32/makefile.mk | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/win32/makefile.mk b/win32/makefile.mk index 59c8819..221d066 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -1108,8 +1108,7 @@ regen_config_h: $(CONFIGPM): ..\config.sh config_h.PL $(MINIPERL) -I..\lib ..\configpm --chdir=.. $(XCOPY) config.h $(COREDIR)\*.* - $(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" \ - || $(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE) + -$(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" .\config.h : $(CONFIGPM) -- 1.8.0.msysgit.0
Subject: 0004-squash-collapse-3-shell-runs-of-xcopy-into-1-run-of-.patch
From 548f756ed285fe08c918d8b31df38624a08bdd98 Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Tue, 23 Jun 2015 01:26:41 -0400 Subject: [PATCH 4/5] squash collapse 3 shell runs of xcopy into 1 run of shell -collapse 3 shell runs of xcopy into 1 run of shell "timeit cmd /c "cmd /c xcopy /f /r /i /d /y ..\perl.exe ..\t\ && cmd /c xcopy /f /r /i /d /y ..\perl523.dll ..\t\ && cmd /c xcopy /f /r /i /d /y ..\perlglob.exe ..\t\"" takes 140 ms "timeit cmd /c "xcopy /f /r /i /d /y ..\perl.exe ..\t\ && xcopy /f /r /i /d /y ..\perl523.dll ..\t\ && xcopy /f /r /i /d /y ..\perlglob.exe ..\t\"" takes 78 ms on my machine --- win32/makefile.mk | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/win32/makefile.mk b/win32/makefile.mk index 221d066..8412f57 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -1687,9 +1687,8 @@ minitest : .\config.h $(HAVEMINIPERL) ..\git_version.h $(GLOBEXE) $(CONFIGPM) $( cd ..\t && perl.exe TEST base/*.t comp/*.t cmd/*.t run/*.t io/*.t re/*.t opbasic/*.t op/*.t uni/*.t perf/*.t pragma/*.t test-prep : all utils ..\pod\perltoc.pod $(TESTPREPGCC) - $(XCOPY) $(PERLEXE) ..\t\$(NULL) - $(XCOPY) $(PERLDLL) ..\t\$(NULL) - $(XCOPY) $(GLOBEXE) ..\t\$(NULL) + $(XCOPY) $(PERLEXE) ..\t\$(NULL) && $(XCOPY) $(PERLDLL) ..\t\$(NULL) \ + && $(XCOPY) $(GLOBEXE) ..\t\$(NULL) # If building with gcc versions 4.x.x or greater, then # the GCC helper DLL will also need copied to the test directory. -- 1.8.0.msysgit.0
Subject: 0005-squash-less-steps-build-generate_uudmap.exe.patch
From 615b1addb3a6b2e6c7fe6e7185c7d2c1a0e846eb Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Tue, 23 Jun 2015 12:06:21 -0400 Subject: [PATCH 5/5] squash less steps build generate_uudmap.exe -reduce number of steps to build generate_uudmap.exe. Compiling (140 ms on GCC 4.8.3) and linking (561 ms) is heavy weight compared to running generate_uudmap.exe (31 ms) and compiling globals.c (483 ms). dmake's algo for doing parallel is poor. On the main parallel pass with all the mini core .c files compiling ("MINI_OBJ"), only generate_uudmap.o will be generated as part of the chain of deps hanging off globals.o, after the 1st tier of MINI_OBJ has compiled in parallel, dmake will do another sweep of MINI_OBJ, this time it will find generate_uudmap.exe node and run linking, at this 2nd tier nothing ran except linking generate_uudmap. No parallel anything. This is a chokepoint. Then dmake does a 3rd sweep and runs the 3rd tier, which is run generate_uudmap.exe, again nothing parallel happens here. By combining compiling and linking in 1 target, the time spent serially linking generate_uudmap.exe, is moved into parallel time when the rest of MINI_OBJ compiles. Also a little bit of CPU and IO is saved by launching less processes (less cmd.exe shells, gcc/g++ process ran only once (gcc/g++ still run many standalone procs internally but now only 1 gcc was started not 2)). -suppress warnings from dmake with .UPDATEALL, the 3 generated headers are made at the same time in real life, in the makefile it was written as if they weren't dmake: Warning:--Found file corresponding to virtual target [..\uudmap.h]. dmake: Warning:--Found file corresponding to virtual target [..\mg_data.h]. -move generating the headers to the target that generates generate_uudmap.exe, this moves some serialization time, albeit small (30 ms) from right before miniperl link, to parallel time during 1st pass of MINI_OBJ --- win32/makefile.mk | 23 ++++++++--------------- 1 file changed, 8 insertions(+), 15 deletions(-) diff --git a/win32/makefile.mk b/win32/makefile.mk index 8412f57..8840a16 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -978,7 +978,6 @@ MINICORE_OBJ = $(MINIDIR)\{$(MICROCORE_OBJ:f) miniperlmain$(o) perlio$(o)} MINIWIN32_OBJ = $(MINIDIR)\{$(WIN32_OBJ:f)} MINI_OBJ = $(MINICORE_OBJ) $(MINIWIN32_OBJ) DLL_OBJ = $(DYNALOADER) -GENUUDMAP_OBJ = $(GENUUDMAP:db:+$(o)) PERLDLL_OBJ = $(CORE_OBJ) PERLEXE_OBJ = perlmain$(o) @@ -1388,9 +1387,14 @@ $(PERLEXE_RES): perlexe.rc $(PERLEXE_MANIFEST) $(PERLEXE_ICO) $(MINIDIR)\globals$(o) : $(GENERATED_HEADERS) -$(UUDMAP_H) $(MG_DATA_H) : $(BITCOUNT_H) - -$(BITCOUNT_H) : $(GENUUDMAP) +$(GENUUDMAP) $(GENERATED_HEADERS) .UPDATEALL : ..\mg_raw.h +.IF "$(CCTYPE)" == "GCC" + $(LINK32) $(CFLAGS_O) -o..\generate_uudmap.exe ..\generate_uudmap.c $(BLINK_FLAGS) \ + $(mktmp $(LKPRE) $(LIBFILES) $(LKPOST)) +.ELSE + $(CC) $(CFLAGS_O) -Fe..\generate_uudmap.exe ..\generate_uudmap.c @$(mktmp -link $(LIBFILES)) -link $(BLINK_FLAGS) + $(EMBED_EXE_MANI:s/$@/..\generate_uudmap.exe/) +.ENDIF $(GENUUDMAP) $(GENERATED_HEADERS) #This generates a stub ppport.h & creates & fills /lib/CORE to allow for XS @@ -1401,17 +1405,6 @@ $(COREDIR)\ppport.h : $(CORE_H) $(XCOPY) *.h $(COREDIR)\*.* && $(RCOPY) include $(COREDIR)\*.* && $(XCOPY) ..\*.h $(COREDIR)\*.* @( echo #define aTHXR aTHX&& echo #define dTHXR dTHX) >$@ -$(GENUUDMAP_OBJ) : ..\mg_raw.h - -$(GENUUDMAP) : $(GENUUDMAP_OBJ) -.IF "$(CCTYPE)" == "GCC" - $(LINK32) -v -o $@ $(BLINK_FLAGS) \ - $(mktmp $(LKPRE) $(GENUUDMAP_OBJ) $(LIBFILES) $(LKPOST)) -.ELSE - $(LINK32) -out:$@ $(BLINK_FLAGS) @$(mktmp $(LIBFILES) $(GENUUDMAP_OBJ)) - $(EMBED_EXE_MANI) -.ENDIF - perlmain.c : runperl.c copy runperl.c perlmain.c -- 1.8.0.msysgit.0
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 916b
On Tue Jun 23 17:01:43 2015, bulk88 wrote: Show quoted text
> After sorting the .c'es by time (I am only moving the >1 second ones), > here are the results on my 8 core with "dmake -P8 ../perl523.dll".
Unfortunately the build failed for me: 1 file(s) copied. Processing extracted\DBinaryProperties.txt 1 file(s) copied. 1 file(s) copied. Can't locate Getopt/Std.pm in @INC (you may need to install the Getopt::Std module) (@INC contains: .) at bin\pl2bat.pl line 5. BEGIN failed--compilation aborted at bin\pl2bat.pl line 5. Processing extracted\DNumValues.txt dmake: Warning: -- Found file corresponding to virtual target [config.h]. dmake: Error code 130, while making 'utils' Generating a dmake-style Makefile Writing Makefile for Cwd Processing extracted\DJoinGroup.txt I expect utils needs to depend on Extensions_nonxs, or a new directory needs to be added to writebuildcustomize.pl. Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 14.3k
On Wed Jul 15 23:58:52 2015, tonyc wrote: Show quoted text
> Unfortunately the build failed for me: > > 1 file(s) copied. > Processing extracted\DBinaryProperties.txt > 1 file(s) copied. > 1 file(s) copied. > Can't locate Getopt/Std.pm in @INC (you may need to install the > Getopt::Std module) (@INC contains: .) at bin\pl2bat.pl line 5. > BEGIN failed--compilation aborted at bin\pl2bat.pl line 5. > Processing extracted\DNumValues.txt > dmake: Warning: -- Found file corresponding to virtual target > [config.h]. > dmake: Error code 130, while making 'utils' > Generating a dmake-style Makefile > Writing Makefile for Cwd > Processing extracted\DJoinGroup.txt > > I expect utils needs to depend on Extensions_nonxs, or a new directory > needs to be > added to writebuildcustomize.pl. > > Tony
Getopt::Std is permanently installed in /lib, no buildcustomize, no Makefile.PL. http://perl5.git.perl.org/perl.git/blob/HEAD:/lib/Getopt/Std.pm pl2bat.pl is run as "..\miniperl.exe -I..\lib bin\pl2bat.pl", the -I..\lib means it sees root /lib. Putting --------------------------------------------------------- BEGIN{ $, ="\n"; print @INC; } --------------------------------------------------------- before "use Getopt::Std;" in pl2bat.pl gives me --------------------------------------------------------- C:\perl521\srcnewb4opt\lib C:\perl521\srcnewb4opt\cpan\AutoLoader\lib C:\perl521\srcnewb4opt\dist\Carp\lib C:\perl521\srcnewb4opt\dist\PathTools C:\perl521\srcnewb4opt\dist\PathTools\lib C:\perl521\srcnewb4opt\cpan\ExtUtils-Command\lib C:\perl521\srcnewb4opt\cpan\ExtUtils-Install\lib C:\perl521\srcnewb4opt\cpan\ExtUtils-MakeMaker\lib C:\perl521\srcnewb4opt\cpan\ExtUtils-Manifest\lib C:\perl521\srcnewb4opt\cpan\File-Path\lib C:\perl521\srcnewb4opt\ext\re C:\perl521\srcnewb4opt\dist\Term-ReadLine\lib C:\perl521\srcnewb4opt\dist\Exporter\lib C:\perl521\srcnewb4opt\ext\File-Find\lib C:\perl521\srcnewb4opt\cpan\Text-Tabs\lib C:\perl521\srcnewb4opt\dist\constant\lib C:\perl521\srcnewb4opt\cpan\Text-ParseWords\lib C:\perl521\srcnewb4opt\dist\ExtUtils-ParseXS\lib C:\perl521\srcnewb4opt\cpan\Getopt-Long\lib C:\perl521\srcnewb4opt\cpan\parent\lib C:\perl521\srcnewb4opt\cpan\ExtUtils-Constant\lib --------------------------------------------------------- which looks sane for miniperl. Show quoted text
> Can't locate Getopt/Std.pm in @INC (you may need to install the > Getopt::Std module) (@INC contains: .) at bin\pl2bat.pl line 5.
Is full perl somehow running pl2bat.pl? The top of pl2bat.pl has --------------------------------------------------------- eval 'exec perl -x -S "$0" ${1+"$@"}' if 0; # In case running under some shell --------------------------------------------------------- Rebased patch set on blead attached. Time::HiRes's abandonment still needs unresolved for the final single patch I create. my build log for "dmake utils CCTYPE=MSVC70" ---------------------------------------------------------------- cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_EXTERNAL_GLOB -DPE RL_IS_MINIPERL -Fomini\run.obj -Fdmini\run.pdb ..\run.c run.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_EXTERNAL_GLOB -DPE RL_IS_MINIPERL -Fomini\scope.obj -Fdmini\scope.pdb ..\scope.c scope.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_EXTERNAL_GLOB -DPE RL_IS_MINIPERL -Fomini\taint.obj -Fdmini\taint.pdb ..\taint.c taint.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_EXTERNAL_GLOB -DPE RL_IS_MINIPERL -Fomini\universal.obj -Fdmini\universal.pdb ..\universal.c universal.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_EXTERNAL_GLOB -DPE RL_IS_MINIPERL -Fomini\util.obj -Fdmini\util.pdb ..\util.c util.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_EXTERNAL_GLOB -DPE RL_IS_MINIPERL -Fomini\miniperlmain.obj -Fdmini\miniperlmain.pdb ..\miniperlmain .c miniperlmain.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_EXTERNAL_GLOB -DPE RL_IS_MINIPERL -Fomini\perlio.obj -Fdmini\perlio.pdb ..\perlio.c perlio.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_IS_MINIPERL -Fomin i\win32.obj -Fdmini\win32.pdb win32.c win32.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_IS_MINIPERL -Fomin i\win32io.obj -Fdmini\win32io.pdb win32io.c win32io.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_IS_MINIPERL -Fomin i\win32sck.obj -Fdmini\win32sck.pdb win32sck.c win32sck.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_IS_MINIPERL -Fomin i\win32thread.obj -Fdmini\win32thread.pdb win32thread.c win32thread.c cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL DLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -DPERL_IS_MINIPERL -Fomin i\fcrypt.obj -Fdmini\fcrypt.pdb fcrypt.c fcrypt.c link -out:..\miniperl.exe -subsystem:console -nologo -nodefaultlib -debug -opt:r ef,icf -libpath:"c:\perl\lib\CORE" -machine:x86 \ @C:\WINDOWS\TEMP\mk5 LINK : warning LNK4089: all references to 'WS2_32.dll' discarded by /OPT:REF if exist ..\miniperl.exe.manifest mt -nologo -manifest ..\miniperl.exe.manifest -outputresource:..\miniperl.exe;1 && if exist ..\miniperl.exe.manif est del ..\miniperl.exe.manifest ..\miniperl.exe -I..\lib -f ..\write_buildcustomize.pl .. copy config.vc config.w32 1 file(s) copied. ..\miniperl.exe -I..\lib config_sh.PL --cfgsh-option-file \ C:\WINDOWS\TEMP\mk6 config.w32 > ..\config.sh ..\miniperl.exe -I..\lib ..\configpm --chdir=.. written lib/Config.pod updated lib/Config.pm updated lib/Config_heavy.pl Warning: failed to load Config_git.pl, something strange about this perl... xcopy /f /r /i /d /y config.h ..\lib\CORE\*.* C:\perl521\srcnewb4opt\win32\config.h -> C:\perl521\srcnewb4opt\lib\CORE\config. h 1 File(s) copied ..\miniperl.exe -I..\lib config_h.PL "ARCHPREFIX=" Running config_h.PL Writing config.h Warning: failed to load Config_git.pl, something strange about this perl... config.h has changed ..\miniperl.exe -I..\lib ..\utils\Makefile.PL .. Extracting utils/Makefile (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... cd ..\utils && b88-dmake PERL=..\miniperl.exe ..\miniperl.exe -I../lib c2ph.PL Extracting c2ph (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... Linking c2ph to pstruct. ..\miniperl.exe -I../lib corelist.PL Extracting corelist (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib cpan.PL Extracting cpan (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib h2ph.PL Extracting h2ph (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib h2xs.PL Extracting h2xs (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib instmodsh.PL Extracting instmodsh (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib json_pp.PL Extracting json_pp (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib perlbug.PL Extracting perlbug (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib perldoc.PL Extracting "perldoc" (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib perlivp.PL Extracting perlivp (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib pl2pm.PL Extracting pl2pm (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib prove.PL Extracting prove (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib ptar.PL Extracting ptar (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib ptardiff.PL Extracting ptardiff (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib ptargrep.PL Extracting ptargrep (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib shasum.PL Extracting shasum (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib splain.PL Extracting splain (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib libnetcfg.PL Extracting libnetcfg (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib piconv.PL Extracting piconv (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib enc2xs.PL Extracting enc2xs (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib encguess.PL Extracting encguess (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib xsubpp.PL Extracting xsubpp (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib pod2html.PL Extracting pod2html (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I../lib zipdetails.PL Extracting zipdetails (with variable substitutions) Warning: failed to load Config_git.pl, something strange about this perl... copy ..\README.aix ..\pod\perlaix.pod 1 file(s) copied. copy ..\README.amiga ..\pod\perlamiga.pod 1 file(s) copied. copy ..\README.android ..\pod\perlandroid.pod 1 file(s) copied. copy ..\README.bs2000 ..\pod\perlbs2000.pod 1 file(s) copied. copy ..\README.ce ..\pod\perlce.pod 1 file(s) copied. copy ..\README.cn ..\pod\perlcn.pod 1 file(s) copied. copy ..\README.cygwin ..\pod\perlcygwin.pod 1 file(s) copied. copy ..\README.dos ..\pod\perldos.pod 1 file(s) copied. copy ..\README.freebsd ..\pod\perlfreebsd.pod 1 file(s) copied. copy ..\README.haiku ..\pod\perlhaiku.pod 1 file(s) copied. copy ..\README.hpux ..\pod\perlhpux.pod 1 file(s) copied. copy ..\README.hurd ..\pod\perlhurd.pod 1 file(s) copied. copy ..\README.irix ..\pod\perlirix.pod 1 file(s) copied. copy ..\README.jp ..\pod\perljp.pod 1 file(s) copied. copy ..\README.ko ..\pod\perlko.pod 1 file(s) copied. copy ..\README.linux ..\pod\perllinux.pod 1 file(s) copied. copy ..\README.macos ..\pod\perlmacos.pod 1 file(s) copied. copy ..\README.macosx ..\pod\perlmacosx.pod 1 file(s) copied. copy ..\README.netware ..\pod\perlnetware.pod 1 file(s) copied. copy ..\README.openbsd ..\pod\perlopenbsd.pod 1 file(s) copied. copy ..\README.os2 ..\pod\perlos2.pod 1 file(s) copied. copy ..\README.os390 ..\pod\perlos390.pod 1 file(s) copied. copy ..\README.os400 ..\pod\perlos400.pod 1 file(s) copied. copy ..\README.plan9 ..\pod\perlplan9.pod 1 file(s) copied. copy ..\README.qnx ..\pod\perlqnx.pod 1 file(s) copied. copy ..\README.riscos ..\pod\perlriscos.pod 1 file(s) copied. copy ..\README.solaris ..\pod\perlsolaris.pod 1 file(s) copied. copy ..\README.symbian ..\pod\perlsymbian.pod 1 file(s) copied. copy ..\README.synology ..\pod\perlsynology.pod 1 file(s) copied. copy ..\README.tru64 ..\pod\perltru64.pod 1 file(s) copied. copy ..\README.tw ..\pod\perltw.pod 1 file(s) copied. copy ..\README.vos ..\pod\perlvos.pod 1 file(s) copied. copy ..\README.win32 ..\pod\perlwin32.pod 1 file(s) copied. copy ..\pod\perldelta.pod ..\pod\perl5232delta.pod 1 file(s) copied. ..\miniperl.exe -I..\lib bin\pl2bat.pl ..\utils\h2ph ..\utils\splain ..\utils\perlbug ..\utils\pl2pm ..\utils\c2ph ..\utils\pstruct ..\utils\h2xs ..\utils\perldoc ..\utils\perlivp ..\utils\libnetcfg ..\utils\enc2xs ..\utils\encguess ..\utils\piconv ..\utils\corelist ..\utils\cpan ..\utils\xsubpp ..\utils\pod2html ..\utils\prove ..\utils\ptar ..\utils\ptardiff ..\utils\ptargrep ..\utils\zipdetails ..\utils\shasum ..\utils\instmodsh ..\utils\json_pp bin\exetype.pl bin\runperl.pl bin\pl2bat.pl bin\perlglob.pl bin\search.pl Warning: failed to load Config_git.pl, something strange about this perl... ..\miniperl.exe -I..\lib ..\autodoc.pl .. ..\miniperl.exe -I..\lib ..\pod\perlmodlib.PL -q .. C:\perl521\srcnewb4opt\win32>dmake utils CCTYPE=MSVC70 -------------------------------------------------------------------- -- bulk88 ~ bulk88 at hotmail.com
Subject: 0001-WIP-add-parallel-support-for-Win32-dmake-building.patch

Message body is not shown because it is too large.

Subject: 0002-coredir-parallelism.patch
From df63450553e8e6a8a27f417dfc8fe3e00350d647 Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Sun, 17 May 2015 14:18:06 -0400 Subject: [PATCH 2/5] coredir parallelism -defer the mass copy of headers to COREDIR until XS build time and dont use COREDIR for building miniperl interp or perl interp, more thing to do in parallel this way and 1 less path to search by the CC -config_h.PL requires config.h to be in COREDIR, this XCOPY in CONFIGPM targ is where COREDIR is created if it doesnt exist already -remove a bunch of XCOPY'es, copy the headers in 1 place only, and dont copy them unless they change based on timestamps -the DYNALOADER dep already contains "..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) $(HAVE_COREDIR)" dont specify them again in Extensions and Extensions_reonly, this probably reduces the size of dep graph in dmake, sincedmake explodes the graph and checks the outline of the graph/tree very inefficiently in parallel mode (architectural problem with dmake) -undo Time::HiRes in-core patch per TonyC's 2 line ppport.h idea -delete pdb files in root, otherwise switching VC version will make the next cl.exe error out that it can't write to the pdb file due to version numbers in the pdb file -reorder core .c files in a blend of VC and GCC time order, this reduces a 1:08 mins time to do "dmake -P9 ../perl522.dll" to 0:54 mins on 8 core with GCC, see details in #123867 --- Porting/Maintainers.pl | 2 - README.win32 | 11 ++++-- cpan/Time-HiRes/HiRes.pm | 2 +- cpan/Time-HiRes/HiRes.xs | 4 +- pod/perldelta.pod | 11 ++++++- t/porting/customized.dat | 1 - win32/makefile.mk | 72 +++++++++++++++++++++++++-------------------- 7 files changed, 59 insertions(+), 44 deletions(-) diff --git a/Porting/Maintainers.pl b/Porting/Maintainers.pl index f9c5589..fe357c6 100755 --- a/Porting/Maintainers.pl +++ b/Porting/Maintainers.pl @@ -1218,8 +1218,6 @@ use File::Glob qw(:case); 'Time::HiRes' => { 'DISTRIBUTION' => 'ZEFRAM/Time-HiRes-1.9726.tar.gz', 'FILES' => q[cpan/Time-HiRes], - # CPAN release eventually - 'CUSTOMIZED' => [ 'HiRes.xs' ], }, 'Time::Local' => { diff --git a/README.win32 b/README.win32 index b9875e6..13d2a79 100644 --- a/README.win32 +++ b/README.win32 @@ -98,11 +98,14 @@ See L<Usage Hints for Perl on Windows> below for general hints about this. You need a "make" program to build the sources. If you are using Visual C++ or the Windows SDK tools, you can use nmake supplied with Visual C++ or Windows SDK. You may also use, for Visual C++ or Windows SDK, dmake instead -of nmake. dmake is open source software, but is not included with Visual C++ or -Windows SDK. If you want parallel building with Visual C++ or +of nmake. dmake is open source software, but is not included with Visual C++ or +Windows SDK. If you want parallel building with Visual C++ or Windows SDK you must use dmake instead of nmake. Builds using gcc need dmake. -nmake is not supported for gcc builds. gmake is not supported, but might be -added in the future. +nmake is not supported for gcc builds. gmake is not supported, but might be +added in the future. It is recommended to use dmake 4.13 or newer for parallel +building. Older dmakes, in parallel mode, have very high CPU usage and pound +the disk/filing system with duplicate I/O calls in an aggressive polling +loop. A port of dmake for Windows is available from: diff --git a/cpan/Time-HiRes/HiRes.pm b/cpan/Time-HiRes/HiRes.pm index 0ee2c62..cf64bc1 100644 --- a/cpan/Time-HiRes/HiRes.pm +++ b/cpan/Time-HiRes/HiRes.pm @@ -23,7 +23,7 @@ our @EXPORT_OK = qw (usleep sleep ualarm alarm gettimeofday time tv_interval stat lstat ); -our $VERSION = '1.9726_01'; +our $VERSION = '1.9726'; our $XS_VERSION = $VERSION; $VERSION = eval $VERSION; diff --git a/cpan/Time-HiRes/HiRes.xs b/cpan/Time-HiRes/HiRes.xs index 9a852c4..96640e9 100644 --- a/cpan/Time-HiRes/HiRes.xs +++ b/cpan/Time-HiRes/HiRes.xs @@ -731,7 +731,7 @@ myNVtime() static void hrstatns(UV *atime_nsec, UV *mtime_nsec, UV *ctime_nsec) { - dTHX; + dTHXR; #if TIME_HIRES_STAT == 1 *atime_nsec = PL_statcache.st_atimespec.tv_nsec; *mtime_nsec = PL_statcache.st_mtimespec.tv_nsec; @@ -1284,7 +1284,7 @@ PROTOTYPE: ;$ fakeop.op_flags = GIMME_V == G_ARRAY ? OPf_WANT_LIST : GIMME_V == G_SCALAR ? OPf_WANT_SCALAR : OPf_WANT_VOID; PL_op = &fakeop; - (void)fakeop.op_ppaddr(aTHX); + (void)fakeop.op_ppaddr(aTHXR); SPAGAIN; LEAVE; nret = SP+1 - &ST(0); diff --git a/pod/perldelta.pod b/pod/perldelta.pod index 06f2ee2..f949597 100644 --- a/pod/perldelta.pod +++ b/pod/perldelta.pod @@ -305,9 +305,16 @@ L</Modules and Pragmata> section. =over 4 -=item XXX-some-platform +=item Win32 -XXX +=over + +=item * + +Parallel building has been added to the dmake C<makefile.mk> makefile. All +Win32 compilers are supported. + +=back =back diff --git a/t/porting/customized.dat b/t/porting/customized.dat index c3eeace..ee65785 100644 --- a/t/porting/customized.dat +++ b/t/porting/customized.dat @@ -57,7 +57,6 @@ Win32API::File cpan/Win32API-File/t/file.t 124e64aa77e755235eb297644a87fac5388d3 Win32API::File cpan/Win32API-File/t/tie.t 712ea7edd0cc805ce1c0b8172c01b03dd19b583d Win32API::File cpan/Win32API-File/typemap 24bff088babeadac0873e8df390d1666d9d9db4a autodie cpan/autodie/t/mkdir.t 9e70d2282a3cc7d76a78bf8144fccba20fb37dac -Time::HiRes cpan/Time-HiRes/HiRes.xs 0a850b6dee1f2c60ade3d4c33fab7c69b02ddc45 podlators cpan/podlators/scripts/pod2man.PL f81acf53f3ff46cdcc5ebdd661c5d13eb35d20d6 podlators cpan/podlators/scripts/pod2text.PL b4693fcfe4a0a1b38a215cfb8985a65d5d025d69 version cpan/version/lib/version.pm d0923b895d57f1d669ae36fcf85c87b16db341d1 diff --git a/win32/makefile.mk b/win32/makefile.mk index aa21094..6b5298b 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -472,7 +472,7 @@ a = .a # Options # -INCLUDES = -I.\include -I. -I.. -I$(COREDIR) +INCLUDES = -I.\include -I. -I.. DEFINES = -DWIN32 .IF "$(WIN64)" == "define" DEFINES += -DWIN64 -DCONSERVATIVE @@ -560,7 +560,7 @@ RSC = rc # Options # -INCLUDES = -I$(COREDIR) -I.\include -I. -I.. +INCLUDES = -I.\include -I. -I.. #PCHFLAGS = -Fpc:\temp\vcmoduls.pch -YX DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT LOCDEFS = -DPERLDLL -DPERL_CORE @@ -863,43 +863,45 @@ XCOPY = xcopy /f /r /i /d /y RCOPY = xcopy /f /r /i /e /d /y NOOP = @rem +#first ones are arrange in compile time order for faster parallel building +#see #123867 for details MICROCORE_SRC = \ + ..\toke.c \ + ..\regcomp.c \ + ..\regexec.c \ + ..\op.c \ + ..\sv.c \ + ..\pp.c \ + ..\pp_ctl.c \ + ..\pp_sys.c \ + ..\pp_pack.c \ + ..\pp_hot.c \ + ..\gv.c \ + ..\perl.c \ + ..\utf8.c \ + ..\dump.c \ + ..\hv.c \ ..\av.c \ ..\caretx.c \ ..\deb.c \ ..\doio.c \ ..\doop.c \ - ..\dump.c \ ..\globals.c \ - ..\gv.c \ ..\mro_core.c \ - ..\hv.c \ ..\locale.c \ ..\keywords.c \ ..\mathoms.c \ ..\mg.c \ ..\numeric.c \ - ..\op.c \ ..\pad.c \ - ..\perl.c \ ..\perlapi.c \ ..\perly.c \ - ..\pp.c \ - ..\pp_ctl.c \ - ..\pp_hot.c \ - ..\pp_pack.c \ ..\pp_sort.c \ - ..\pp_sys.c \ ..\reentr.c \ - ..\regcomp.c \ - ..\regexec.c \ ..\run.c \ ..\scope.c \ - ..\sv.c \ ..\taint.c \ - ..\toke.c \ ..\universal.c \ - ..\utf8.c \ ..\util.c EXTRACORE_SRC += perllib.c @@ -964,6 +966,10 @@ UUDMAP_H = ..\uudmap.h BITCOUNT_H = ..\bitcount.h MG_DATA_H = ..\mg_data.h GENERATED_HEADERS = $(UUDMAP_H) $(BITCOUNT_H) $(MG_DATA_H) +#a stub ppport.h must be generated so building XS modules, .c->.obj wise, will +#work, so this target also represents creating the COREDIR and filling it +HAVE_COREDIR = $(COREDIR)\ppport.h + MICROCORE_OBJ = $(MICROCORE_SRC:db:+$(o)) CORE_OBJ = $(MICROCORE_OBJ) $(EXTRACORE_SRC:db:+$(o)) @@ -1101,9 +1107,7 @@ regen_config_h: $(CONFIGPM): ..\config.sh config_h.PL $(MINIPERL) -I..\lib ..\configpm --chdir=.. - $(XCOPY) ..\*.h $(COREDIR)\*.* - $(XCOPY) *.h $(COREDIR)\*.* - $(RCOPY) include $(COREDIR)\*.* + $(XCOPY) config.h $(COREDIR)\*.* $(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" \ || $(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE) @@ -1379,7 +1383,7 @@ $(PERLSTATICLIB): $(PERLDLL_OBJ) Extensions_static $(LIB32) $(LIB_FLAGS) -out:$@ @Extensions_static \ @$(mktmp $(PERLDLL_OBJ)) .ENDIF - $(XCOPY) $(PERLSTATICLIB) $(COREDIR) + $(XCOPY) $(PERLSTATICLIB) $(COREDIR)\$(NULL) $(PERLEXE_RES): perlexe.rc $(PERLEXE_MANIFEST) $(PERLEXE_ICO) @@ -1390,6 +1394,14 @@ $(UUDMAP_H) $(MG_DATA_H) : $(BITCOUNT_H) $(BITCOUNT_H) : $(GENUUDMAP) $(GENUUDMAP) $(GENERATED_HEADERS) +#This generates a stub ppport.h & creates & fills /lib/CORE to allow for XS +#building .c->.obj wise (linking is a different thing). This taget is AKA +#$(HAVE_COREDIR). ppport.h can become a 0 byte file once core's Time::HiRes is +#has the patch in CPAN #102799 +$(COREDIR)\ppport.h : $(CORE_H) + $(XCOPY) *.h $(COREDIR)\*.* && $(RCOPY) include $(COREDIR)\*.* && $(XCOPY) ..\*.h $(COREDIR)\*.* + @( echo #define aTHXR aTHX&& echo #define dTHXR dTHX) >$@ + $(GENUUDMAP_OBJ) : ..\mg_raw.h $(GENUUDMAP) : $(GENUUDMAP_OBJ) @@ -1438,18 +1450,15 @@ $(PERLEXESTATIC): $(PERLSTATICLIB) $(CONFIGPM) $(PERLEXEST_OBJ) $(PERLEXE_RES) #------------------------------------------------------------------------------- # There's no direct way to mark a dependency on # DynaLoader.pm, so this will have to do -Extensions : ..\make_ext.pl ..\lib\buildcustomize.pl $(PERLDEP) $(CONFIGPM) $(DYNALOADER) - $(XCOPY) ..\*.h $(COREDIR)\*.* - if not exist $(COREDIR)\ppport.h rem. > $(COREDIR)\ppport.h + +#most of deps of this target are in DYNALOADER and therefore omitted here +Extensions : $(PERLDEP) $(DYNALOADER) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic -Extensions_reonly : ..\make_ext.pl ..\lib\buildcustomize.pl $(PERLDEP) $(CONFIGPM) $(DYNALOADER) - $(XCOPY) ..\*.h $(COREDIR)\*.* +Extensions_reonly : $(PERLDEP) $(DYNALOADER) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re -Extensions_static : ..\make_ext.pl ..\lib\buildcustomize.pl list_static_libs.pl $(CONFIGPM) - $(XCOPY) ..\*.h $(COREDIR)\*.* - if not exist $(COREDIR)\ppport.h rem. > $(COREDIR)\ppport.h +Extensions_static : ..\make_ext.pl ..\lib\buildcustomize.pl list_static_libs.pl $(CONFIGPM) $(HAVE_COREDIR) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static $(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static @@ -1457,8 +1466,7 @@ Extensions_nonxs : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) ..\pod\pe $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs #lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building -$(DYNALOADER) : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) - $(XCOPY) ..\*.h $(COREDIR)\*.* +$(DYNALOADER) : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) $(HAVE_COREDIR) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib Extensions_clean : @@ -1757,7 +1765,7 @@ _clean : -@erase ..\*$(o) ..\*$(a) ..\*.exp *$(o) *$(a) *.exp *.res -@erase ..\t\*.exe ..\t\*.dll ..\t\*.bat -@erase *.ilk - -@erase *.pdb + -@erase *.pdb ..\*.pdb -@erase Extensions_static -- 1.7.9.msysgit.0
Subject: 0003-squash-double-dmake-run.patch
From f4278bb6664342bd031c2d30f9665f679995a2a0 Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Tue, 23 Jun 2015 00:03:04 -0400 Subject: [PATCH 3/5] squash double dmake run -in $(CONFIGPM) target, remove config_h.PL running dmake again to rebuild $(CONFIGPM) target, configpm the script does not use config.h as input. The $(CONFIGPM) target is a choke/serialization point for parallel building, so it is high value to make it as fast as possible. Maybe a long time ago, configpm read config.h but it doesn't anymore. Nearly all the dynamic config vars are determined in config_sh.PL script and ..\config.sh target. config_h.PL contains very little logic, and this logic is only for config.h, and only when you want a var to be different in config.h than it is in Config.pm/Config_heavy.pl. Putting a breakpoint ("system 'pause';") in config_h.PL, copying Config.pm/Config_heavy.pl to ".old" versions, continuing execution in config_h.PL, then diffing the 2 new Configs with the old 2 Configs before pre-config_h.PL and "$(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE)" shows no differece. When dmake runs needlessly a 2nd time it emits a warning that nothing changed. This is intentional, since configpm doesn't update the timestamps /update the tiles if the Config files are identical to the old ones to prevent mass clean+rebuild of all modules when nothing global changed. dmake: Warning: -- Target [..\lib\Config.pm] was made but the time stamp has not been updated. --- win32/makefile.mk | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/win32/makefile.mk b/win32/makefile.mk index 6b5298b..bd086f4 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -1108,8 +1108,7 @@ regen_config_h: $(CONFIGPM): ..\config.sh config_h.PL $(MINIPERL) -I..\lib ..\configpm --chdir=.. $(XCOPY) config.h $(COREDIR)\*.* - $(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" \ - || $(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE) + -$(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" .\config.h : $(CONFIGPM) -- 1.7.9.msysgit.0
Subject: 0004-squash-collapse-3-shell-runs-of-xcopy-into-1-run-of-.patch
From 40229f56e147834bda2f8f25bbb797b32527b041 Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Tue, 23 Jun 2015 01:26:41 -0400 Subject: [PATCH 4/5] squash collapse 3 shell runs of xcopy into 1 run of shell -collapse 3 shell runs of xcopy into 1 run of shell "timeit cmd /c "cmd /c xcopy /f /r /i /d /y ..\perl.exe ..\t\ && cmd /c xcopy /f /r /i /d /y ..\perl523.dll ..\t\ && cmd /c xcopy /f /r /i /d /y ..\perlglob.exe ..\t\"" takes 140 ms "timeit cmd /c "xcopy /f /r /i /d /y ..\perl.exe ..\t\ && xcopy /f /r /i /d /y ..\perl523.dll ..\t\ && xcopy /f /r /i /d /y ..\perlglob.exe ..\t\"" takes 78 ms on my machine --- win32/makefile.mk | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/win32/makefile.mk b/win32/makefile.mk index bd086f4..1bae9de 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -1687,9 +1687,8 @@ minitest : .\config.h $(HAVEMINIPERL) ..\git_version.h $(GLOBEXE) $(CONFIGPM) $( cd ..\t && perl.exe TEST base/*.t comp/*.t cmd/*.t run/*.t io/*.t re/*.t opbasic/*.t op/*.t uni/*.t perf/*.t pragma/*.t test-prep : all utils ..\pod\perltoc.pod $(TESTPREPGCC) - $(XCOPY) $(PERLEXE) ..\t\$(NULL) - $(XCOPY) $(PERLDLL) ..\t\$(NULL) - $(XCOPY) $(GLOBEXE) ..\t\$(NULL) + $(XCOPY) $(PERLEXE) ..\t\$(NULL) && $(XCOPY) $(PERLDLL) ..\t\$(NULL) \ + && $(XCOPY) $(GLOBEXE) ..\t\$(NULL) # If building with gcc versions 4.x.x or greater, then # the GCC helper DLL will also need copied to the test directory. -- 1.7.9.msysgit.0
Subject: 0005-squash-less-steps-build-generate_uudmap.exe.patch
From 96157ef49ed5121b4c2a17abcd0068c0f2763667 Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Tue, 23 Jun 2015 12:06:21 -0400 Subject: [PATCH 5/5] squash less steps build generate_uudmap.exe -reduce number of steps to build generate_uudmap.exe. Compiling (140 ms on GCC 4.8.3) and linking (561 ms) is heavy weight compared to running generate_uudmap.exe (31 ms) and compiling globals.c (483 ms). dmake's algo for doing parallel is poor. On the main parallel pass with all the mini core .c files compiling ("MINI_OBJ"), only generate_uudmap.o will be generated as part of the chain of deps hanging off globals.o, after the 1st tier of MINI_OBJ has compiled in parallel, dmake will do another sweep of MINI_OBJ, this time it will find generate_uudmap.exe node and run linking, at this 2nd tier nothing ran except linking generate_uudmap. No parallel anything. This is a chokepoint. Then dmake does a 3rd sweep and runs the 3rd tier, which is run generate_uudmap.exe, again nothing parallel happens here. By combining compiling and linking in 1 target, the time spent serially linking generate_uudmap.exe, is moved into parallel time when the rest of MINI_OBJ compiles. Also a little bit of CPU and IO is saved by launching less processes (less cmd.exe shells, gcc/g++ process ran only once (gcc/g++ still run many standalone procs internally but now only 1 gcc was started not 2)). -suppress warnings from dmake with .UPDATEALL, the 3 generated headers are made at the same time in real life, in the makefile it was written as if they weren't dmake: Warning:--Found file corresponding to virtual target [..\uudmap.h]. dmake: Warning:--Found file corresponding to virtual target [..\mg_data.h]. -move generating the headers to the target that generates generate_uudmap.exe, this moves some serialization time, albeit small (30 ms) from right before miniperl link, to parallel time during 1st pass of MINI_OBJ --- win32/makefile.mk | 23 ++++++++--------------- 1 files changed, 8 insertions(+), 15 deletions(-) diff --git a/win32/makefile.mk b/win32/makefile.mk index 1bae9de..fdc0d28 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -978,7 +978,6 @@ MINICORE_OBJ = $(MINIDIR)\{$(MICROCORE_OBJ:f) miniperlmain$(o) perlio$(o)} MINIWIN32_OBJ = $(MINIDIR)\{$(WIN32_OBJ:f)} MINI_OBJ = $(MINICORE_OBJ) $(MINIWIN32_OBJ) DLL_OBJ = $(DYNALOADER) -GENUUDMAP_OBJ = $(GENUUDMAP:db:+$(o)) PERLDLL_OBJ = $(CORE_OBJ) PERLEXE_OBJ = perlmain$(o) @@ -1388,9 +1387,14 @@ $(PERLEXE_RES): perlexe.rc $(PERLEXE_MANIFEST) $(PERLEXE_ICO) $(MINIDIR)\globals$(o) : $(GENERATED_HEADERS) -$(UUDMAP_H) $(MG_DATA_H) : $(BITCOUNT_H) - -$(BITCOUNT_H) : $(GENUUDMAP) +$(GENUUDMAP) $(GENERATED_HEADERS) .UPDATEALL : ..\mg_raw.h +.IF "$(CCTYPE)" == "GCC" + $(LINK32) $(CFLAGS_O) -o..\generate_uudmap.exe ..\generate_uudmap.c $(BLINK_FLAGS) \ + $(mktmp $(LKPRE) $(LIBFILES) $(LKPOST)) +.ELSE + $(CC) $(CFLAGS_O) -Fe..\generate_uudmap.exe ..\generate_uudmap.c @$(mktmp -link $(LIBFILES)) -link $(BLINK_FLAGS) + $(EMBED_EXE_MANI:s/$@/..\generate_uudmap.exe/) +.ENDIF $(GENUUDMAP) $(GENERATED_HEADERS) #This generates a stub ppport.h & creates & fills /lib/CORE to allow for XS @@ -1401,17 +1405,6 @@ $(COREDIR)\ppport.h : $(CORE_H) $(XCOPY) *.h $(COREDIR)\*.* && $(RCOPY) include $(COREDIR)\*.* && $(XCOPY) ..\*.h $(COREDIR)\*.* @( echo #define aTHXR aTHX&& echo #define dTHXR dTHX) >$@ -$(GENUUDMAP_OBJ) : ..\mg_raw.h - -$(GENUUDMAP) : $(GENUUDMAP_OBJ) -.IF "$(CCTYPE)" == "GCC" - $(LINK32) -v -o $@ $(BLINK_FLAGS) \ - $(mktmp $(LKPRE) $(GENUUDMAP_OBJ) $(LIBFILES) $(LKPOST)) -.ELSE - $(LINK32) -out:$@ $(BLINK_FLAGS) @$(mktmp $(LIBFILES) $(GENUUDMAP_OBJ)) - $(EMBED_EXE_MANI) -.ENDIF - perlmain.c : runperl.c copy runperl.c perlmain.c -- 1.7.9.msysgit.0
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.2k
On Wed Jul 22 00:48:19 2015, bulk88 wrote: Show quoted text
> On Wed Jul 15 23:58:52 2015, tonyc wrote:
> > Unfortunately the build failed for me: > > > > 1 file(s) copied. > > Processing extracted\DBinaryProperties.txt > > 1 file(s) copied. > > 1 file(s) copied. > > Can't locate Getopt/Std.pm in @INC (you may need to install the > > Getopt::Std module) (@INC contains: .) at bin\pl2bat.pl line 5. > > BEGIN failed--compilation aborted at bin\pl2bat.pl line 5. > > Processing extracted\DNumValues.txt > > dmake: Warning: -- Found file corresponding to virtual target > > [config.h]. > > dmake: Error code 130, while making 'utils' > > Generating a dmake-style Makefile > > Writing Makefile for Cwd > > Processing extracted\DJoinGroup.txt > > > > I expect utils needs to depend on Extensions_nonxs, or a new > > directory > > needs to be > > added to writebuildcustomize.pl. > > > > Tony
> > Getopt::Std is permanently installed in /lib, no buildcustomize, no > Makefile.PL. > http://perl5.git.perl.org/perl.git/blob/HEAD:/lib/Getopt/Std.pm > pl2bat.pl is run as "..\miniperl.exe -I..\lib bin\pl2bat.pl", the > -I..\lib means it sees root /lib. Putting
I suspect I messed up a merge conflict fix when rebasing on blead, and lost the -I..\lib - sorry for the false alarm. It works when I apply your latest patches. Tony
RT-Send-CC: perl5-porters [...] perl.org
Final patch set attached. I kept it as 3 commits so the commit messages can be traced back to the code changes, and the commit messages dont exceed a screenful. Since Time::HiRes is now core maintained ( https://github.com/rjbs/Time-HiRes/commits/master ), I am core patching it so there isn't any unfinished business patches that needs to be revisited later when a new Time::HiRes goes in core. Nobody is going to remember to remove the 5.005 threads ppport.h hack in win32/makefile.mk for T::H when a new T::H is imported. Also I fixed a problem where the dummy ppport.h wound up in the installed perl's CORE dir. The dummy ppport.h is now deleted from installed CORE dir in installbare target. I was afraid the dummy ppport.h may somehow prevent a real full size ppport.h from being picked up by the CC, I am not going to research the -I order of EUMM and MB, if that concern is real or not, but in any case, the dummy ppport.h trick shouldn't be visible in the installed perl, only the uninstalled perl during perl building. -- bulk88 ~ bulk88 at hotmail.com
Subject: 0001-add-parallel-support-4-Win32-dmake-building-part-1.patch

Message body is not shown because it is too large.

Subject: 0002-add-parallel-support-4-Win32-dmake-COREDIR-paralleli.patch
From 327e169d000c16db8b24b4244591162735537e97 Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Sat, 15 Aug 2015 05:24:06 -0400 Subject: [PATCH 2/3] add parallel support 4 Win32 dmake-COREDIR parallelism part 2 -defer the mass copy of headers to COREDIR until XS build time and dont use COREDIR for building miniperl interp or perl interp, more thing to do in parallel this way and 1 less path to search by the CC -config_h.PL requires config.h to be in COREDIR, this XCOPY in CONFIGPM targ is where COREDIR is created if it doesnt exist already -remove a bunch of XCOPY'es, copy the headers in 1 place only, and dont copy them unless they change based on timestamps -the DYNALOADER dep already contains "..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) $(HAVE_COREDIR)" dont specify them again in Extensions and Extensions_reonly, this probably reduces the size of dep graph in dmake, since dmake explodes the graph and checks the outline of the graph/tree very inefficiently in parallel mode (architectural problem with dmake) -delete pdb files in root, otherwise switching VC version will make the next cl.exe error out that it can't write to the pdb file due to version numbers in the pdb file -reorder core .c files in a blend of VC and GCC time order, this reduces a 1:08 mins time to do "dmake -P9 ../perl522.dll" to 0:54 mins on 8 core with GCC, see details in #123867 --- pod/perldelta.pod | 11 ++++++- win32/makefile.mk | 71 +++++++++++++++++++++++++++++------------------------ 2 files changed, 48 insertions(+), 34 deletions(-) diff --git a/pod/perldelta.pod b/pod/perldelta.pod index 8ae82cf..1e676b0 100644 --- a/pod/perldelta.pod +++ b/pod/perldelta.pod @@ -322,9 +322,16 @@ L</Modules and Pragmata> section. =over 4 -=item XXX-some-platform +=item Win32 -XXX +=over + +=item * + +Parallel building has been added to the dmake C<makefile.mk> makefile. All +Win32 compilers are supported. + +=back =back diff --git a/win32/makefile.mk b/win32/makefile.mk index 13e7e57..0d3869f 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -472,7 +472,7 @@ a = .a # Options # -INCLUDES = -I.\include -I. -I.. -I$(COREDIR) +INCLUDES = -I.\include -I. -I.. DEFINES = -DWIN32 .IF "$(WIN64)" == "define" DEFINES += -DWIN64 -DCONSERVATIVE @@ -560,7 +560,7 @@ RSC = rc # Options # -INCLUDES = -I$(COREDIR) -I.\include -I. -I.. +INCLUDES = -I.\include -I. -I.. #PCHFLAGS = -Fpc:\temp\vcmoduls.pch -YX DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT LOCDEFS = -DPERLDLL -DPERL_CORE @@ -863,45 +863,47 @@ XCOPY = xcopy /f /r /i /d /y RCOPY = xcopy /f /r /i /e /d /y NOOP = @rem +#first ones are arrange in compile time order for faster parallel building +#see #123867 for details MICROCORE_SRC = \ + ..\toke.c \ + ..\regcomp.c \ + ..\regexec.c \ + ..\op.c \ + ..\sv.c \ + ..\pp.c \ + ..\pp_ctl.c \ + ..\pp_sys.c \ + ..\pp_pack.c \ + ..\pp_hot.c \ + ..\gv.c \ + ..\perl.c \ + ..\utf8.c \ + ..\dump.c \ + ..\hv.c \ ..\av.c \ ..\caretx.c \ ..\deb.c \ ..\doio.c \ ..\doop.c \ ..\dquote.c \ - ..\dump.c \ ..\globals.c \ - ..\gv.c \ ..\mro_core.c \ - ..\hv.c \ ..\locale.c \ ..\keywords.c \ ..\mathoms.c \ ..\mg.c \ ..\numeric.c \ - ..\op.c \ ..\pad.c \ - ..\perl.c \ ..\perlapi.c \ ..\perly.c \ - ..\pp.c \ - ..\pp_ctl.c \ - ..\pp_hot.c \ - ..\pp_pack.c \ ..\pp_sort.c \ - ..\pp_sys.c \ ..\reentr.c \ - ..\regcomp.c \ - ..\regexec.c \ ..\run.c \ ..\scope.c \ - ..\sv.c \ ..\taint.c \ ..\time64.c \ - ..\toke.c \ ..\universal.c \ - ..\utf8.c \ ..\util.c EXTRACORE_SRC += perllib.c @@ -966,6 +968,10 @@ UUDMAP_H = ..\uudmap.h BITCOUNT_H = ..\bitcount.h MG_DATA_H = ..\mg_data.h GENERATED_HEADERS = $(UUDMAP_H) $(BITCOUNT_H) $(MG_DATA_H) +#a stub ppport.h must be generated so building XS modules, .c->.obj wise, will +#work, so this target also represents creating the COREDIR and filling it +HAVE_COREDIR = $(COREDIR)\ppport.h + MICROCORE_OBJ = $(MICROCORE_SRC:db:+$(o)) CORE_OBJ = $(MICROCORE_OBJ) $(EXTRACORE_SRC:db:+$(o)) @@ -1103,9 +1109,7 @@ regen_config_h: $(CONFIGPM): ..\config.sh config_h.PL $(MINIPERL) -I..\lib ..\configpm --chdir=.. - $(XCOPY) ..\*.h $(COREDIR)\*.* - $(XCOPY) *.h $(COREDIR)\*.* - $(RCOPY) include $(COREDIR)\*.* + $(XCOPY) config.h $(COREDIR)\*.* $(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" \ || $(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE) @@ -1381,7 +1385,7 @@ $(PERLSTATICLIB): $(PERLDLL_OBJ) Extensions_static $(LIB32) $(LIB_FLAGS) -out:$@ @Extensions_static \ @$(mktmp $(PERLDLL_OBJ)) .ENDIF - $(XCOPY) $(PERLSTATICLIB) $(COREDIR) + $(XCOPY) $(PERLSTATICLIB) $(COREDIR)\$(NULL) $(PERLEXE_RES): perlexe.rc $(PERLEXE_MANIFEST) $(PERLEXE_ICO) @@ -1392,6 +1396,13 @@ $(UUDMAP_H) $(MG_DATA_H) : $(BITCOUNT_H) $(BITCOUNT_H) : $(GENUUDMAP) $(GENUUDMAP) $(GENERATED_HEADERS) +#This generates a stub ppport.h & creates & fills /lib/CORE to allow for XS +#building .c->.obj wise (linking is a different thing). This taget is AKA +#$(HAVE_COREDIR). +$(COREDIR)\ppport.h : $(CORE_H) + $(XCOPY) *.h $(COREDIR)\*.* && $(RCOPY) include $(COREDIR)\*.* && $(XCOPY) ..\*.h $(COREDIR)\*.* + rem. > $@ + $(GENUUDMAP_OBJ) : ..\mg_raw.h $(GENUUDMAP) : $(GENUUDMAP_OBJ) @@ -1440,18 +1451,15 @@ $(PERLEXESTATIC): $(PERLSTATICLIB) $(CONFIGPM) $(PERLEXEST_OBJ) $(PERLEXE_RES) #------------------------------------------------------------------------------- # There's no direct way to mark a dependency on # DynaLoader.pm, so this will have to do -Extensions : ..\make_ext.pl ..\lib\buildcustomize.pl $(PERLDEP) $(CONFIGPM) $(DYNALOADER) - $(XCOPY) ..\*.h $(COREDIR)\*.* - if not exist $(COREDIR)\ppport.h rem. > $(COREDIR)\ppport.h + +#most of deps of this target are in DYNALOADER and therefore omitted here +Extensions : $(PERLDEP) $(DYNALOADER) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic -Extensions_reonly : ..\make_ext.pl ..\lib\buildcustomize.pl $(PERLDEP) $(CONFIGPM) $(DYNALOADER) - $(XCOPY) ..\*.h $(COREDIR)\*.* +Extensions_reonly : $(PERLDEP) $(DYNALOADER) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re -Extensions_static : ..\make_ext.pl ..\lib\buildcustomize.pl list_static_libs.pl $(CONFIGPM) - $(XCOPY) ..\*.h $(COREDIR)\*.* - if not exist $(COREDIR)\ppport.h rem. > $(COREDIR)\ppport.h +Extensions_static : ..\make_ext.pl ..\lib\buildcustomize.pl list_static_libs.pl $(CONFIGPM) $(HAVE_COREDIR) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static $(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static @@ -1459,8 +1467,7 @@ Extensions_nonxs : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) ..\pod\pe $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs #lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building -$(DYNALOADER) : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) - $(XCOPY) ..\*.h $(COREDIR)\*.* +$(DYNALOADER) : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) $(HAVE_COREDIR) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib Extensions_clean : @@ -1760,7 +1767,7 @@ _clean : -@erase ..\*$(o) ..\*$(a) ..\*.exp *$(o) *$(a) *.exp *.res -@erase ..\t\*.exe ..\t\*.dll ..\t\*.bat -@erase *.ilk - -@erase *.pdb + -@erase *.pdb ..\*.pdb -@erase Extensions_static -- 1.7.9.msysgit.0
Subject: 0003-add-parallel-support-4-Win32-dmake-uudmap-no-2nd-run.patch
From cdbc9465c8c7c0aa3d02b363ae87b57460535fea Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Sat, 15 Aug 2015 05:39:55 -0400 Subject: [PATCH 3/3] add parallel support 4 Win32 dmake-uudmap+no 2nd run of config_h.PL part 3 -in $(CONFIGPM) target, remove config_h.PL running dmake again to rebuild $(CONFIGPM) target, configpm the script does not use config.h as input. The $(CONFIGPM) target is a choke/serialization point for parallel building, so it is high value to make it as fast as possible. Maybe a long time ago, configpm read config.h but it doesn't anymore. Nearly all the dynamic config vars are determined in config_sh.PL script and ..\config.sh target. config_h.PL contains very little logic, and this logic is only for config.h, and only when you want a var to be different in config.h than it is in Config.pm/Config_heavy.pl. Putting a breakpoint ("system 'pause';") in config_h.PL, copying Config.pm/Config_heavy.pl to ".old" versions, continuing execution in config_h.PL, then diffing the 2 new Configs with the old 2 Configs before pre-config_h.PL and "$(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE)" shows no differece. When dmake runs needlessly a 2nd time it emits a warning that nothing changed. This is intentional, since configpm doesn't update the timestamps /update the files if the Config files are identical to the old ones to prevent mass clean+rebuild of all modules when nothing global changed. dmake: Warning: -- Target [..\lib\Config.pm] was made but the time stamp has not been updated. -reduce number of steps to build generate_uudmap.exe. Compiling (140 ms on GCC 4.8.3) and linking (561 ms) is heavy weight compared to running generate_uudmap.exe (31 ms) and compiling globals.c (483 ms). dmake's algo for doing parallel is poor. On the main parallel pass with all the mini core .c files compiling ("MINI_OBJ"), only generate_uudmap.o will be generated as part of the chain of deps hanging off globals.o, after the 1st tier of MINI_OBJ has compiled in parallel, dmake will do another sweep of MINI_OBJ, this time it will find generate_uudmap.exe node and run linking, at this 2nd tier nothing ran except linking generate_uudmap. No parallel anything. This is a chokepoint. Then dmake does a 3rd sweep and runs the 3rd tier, which is run generate_uudmap.exe, again nothing parallel happens here. By combining compiling and linking in 1 target, the time spent serially linking generate_uudmap.exe, is moved into parallel time when the rest of MINI_OBJ compiles. Also a little bit of CPU and IO is saved by launching less processes (less cmd.exe shells, gcc/g++ process ran only once (gcc/g++ still run many standalone procs internally but now only 1 gcc was started not 2)). -suppress warnings from dmake with .UPDATEALL, the 3 generated headers are made at the same time in real life, in the makefile it was written as if they weren't dmake: Warning:--Found file corresponding to virtual target [..\uudmap.h]. dmake: Warning:--Found file corresponding to virtual target [..\mg_data.h]. -move generating the headers to the target that generates generate_uudmap.exe, this moves some serialization time, albeit small (30 ms) from right before miniperl link, to parallel time during 1st pass of MINI_OBJ -collapse 3 shell runs of xcopy into 1 run of shell "timeit cmd /c "cmd /c xcopy /f /r /i /d /y ..\perl.exe ..\t\ && cmd /c xcopy /f /r /i /d /y ..\perl523.dll ..\t\ && cmd /c xcopy /f /r /i /d /y ..\perlglob.exe ..\t\"" takes 140 ms "timeit cmd /c "xcopy /f /r /i /d /y ..\perl.exe ..\t\ && xcopy /f /r /i /d /y ..\perl523.dll ..\t\ && xcopy /f /r /i /d /y ..\perlglob.exe ..\t\"" takes 78 ms on my machine --- win32/makefile.mk | 31 +++++++++++-------------------- 1 files changed, 11 insertions(+), 20 deletions(-) diff --git a/win32/makefile.mk b/win32/makefile.mk index 0d3869f..188a6f9 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -980,7 +980,6 @@ MINICORE_OBJ = $(MINIDIR)\{$(MICROCORE_OBJ:f) miniperlmain$(o) perlio$(o)} MINIWIN32_OBJ = $(MINIDIR)\{$(WIN32_OBJ:f)} MINI_OBJ = $(MINICORE_OBJ) $(MINIWIN32_OBJ) DLL_OBJ = $(DYNALOADER) -GENUUDMAP_OBJ = $(GENUUDMAP:db:+$(o)) PERLDLL_OBJ = $(CORE_OBJ) PERLEXE_OBJ = perlmain$(o) @@ -1110,8 +1109,7 @@ regen_config_h: $(CONFIGPM): ..\config.sh config_h.PL $(MINIPERL) -I..\lib ..\configpm --chdir=.. $(XCOPY) config.h $(COREDIR)\*.* - $(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" \ - || $(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE) + -$(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" .\config.h : $(CONFIGPM) @@ -1391,9 +1389,14 @@ $(PERLEXE_RES): perlexe.rc $(PERLEXE_MANIFEST) $(PERLEXE_ICO) $(MINIDIR)\globals$(o) : $(GENERATED_HEADERS) -$(UUDMAP_H) $(MG_DATA_H) : $(BITCOUNT_H) - -$(BITCOUNT_H) : $(GENUUDMAP) +$(GENUUDMAP) $(GENERATED_HEADERS) .UPDATEALL : ..\mg_raw.h +.IF "$(CCTYPE)" == "GCC" + $(LINK32) $(CFLAGS_O) -o..\generate_uudmap.exe ..\generate_uudmap.c $(BLINK_FLAGS) \ + $(mktmp $(LKPRE) $(LIBFILES) $(LKPOST)) +.ELSE + $(CC) $(CFLAGS_O) -Fe..\generate_uudmap.exe ..\generate_uudmap.c @$(mktmp -link $(LIBFILES)) -link $(BLINK_FLAGS) + $(EMBED_EXE_MANI:s/$@/..\generate_uudmap.exe/) +.ENDIF $(GENUUDMAP) $(GENERATED_HEADERS) #This generates a stub ppport.h & creates & fills /lib/CORE to allow for XS @@ -1403,17 +1406,6 @@ $(COREDIR)\ppport.h : $(CORE_H) $(XCOPY) *.h $(COREDIR)\*.* && $(RCOPY) include $(COREDIR)\*.* && $(XCOPY) ..\*.h $(COREDIR)\*.* rem. > $@ -$(GENUUDMAP_OBJ) : ..\mg_raw.h - -$(GENUUDMAP) : $(GENUUDMAP_OBJ) -.IF "$(CCTYPE)" == "GCC" - $(LINK32) -v -o $@ $(BLINK_FLAGS) \ - $(mktmp $(LKPRE) $(GENUUDMAP_OBJ) $(LIBFILES) $(LKPOST)) -.ELSE - $(LINK32) -out:$@ $(BLINK_FLAGS) @$(mktmp $(LIBFILES) $(GENUUDMAP_OBJ)) - $(EMBED_EXE_MANI) -.ENDIF - perlmain.c : runperl.c copy runperl.c perlmain.c @@ -1690,9 +1682,8 @@ minitest : .\config.h $(HAVEMINIPERL) ..\git_version.h $(GLOBEXE) $(CONFIGPM) $( cd ..\t && perl.exe TEST base/*.t comp/*.t cmd/*.t run/*.t io/*.t re/*.t opbasic/*.t op/*.t uni/*.t perf/*.t pragma/*.t test-prep : all utils ..\pod\perltoc.pod $(TESTPREPGCC) - $(XCOPY) $(PERLEXE) ..\t\$(NULL) - $(XCOPY) $(PERLDLL) ..\t\$(NULL) - $(XCOPY) $(GLOBEXE) ..\t\$(NULL) + $(XCOPY) $(PERLEXE) ..\t\$(NULL) && $(XCOPY) $(PERLDLL) ..\t\$(NULL) \ + && $(XCOPY) $(GLOBEXE) ..\t\$(NULL) # If building with gcc versions 4.x.x or greater, then # the GCC helper DLL will also need copied to the test directory. -- 1.7.9.msysgit.0
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 678b
On Sat Aug 15 13:10:45 2015, bulk88 wrote: Show quoted text
> Final patch set attached. I kept it as 3 commits so the commit > messages can be traced back to the code changes, and the commit > messages dont exceed a screenful. Since Time::HiRes is now core > maintained ( https://github.com/rjbs/Time-HiRes/commits/master ), I am > core patching it so there isn't any unfinished business patches that > needs to be revisited later when a new Time::HiRes goes in core. > Nobody is going to remember to remove the 5.005 threads ppport.h hack > in win32/makefile.mk for T::H when a new T::H is imported.
As discussed in IRC, Time::HiRes isn't upstream blead, so this can't be applied as is. Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 212b
On Sun Aug 16 21:28:10 2015, tonyc wrote: Show quoted text
> As discussed in IRC, Time::HiRes isn't upstream blead, so this can't > be applied as is. > > Tony
rebased+time hires patch removed -- bulk88 ~ bulk88 at hotmail.com
Subject: 0001-add-parallel-support-4-Win32-dmake-building-part-1.patch

Message body is not shown because it is too large.

Subject: 0002-add-parallel-support-4-Win32-dmake-COREDIR-paralleli.patch
From e4233e627e94e13b0b9532aec9570a9c39a6b4b7 Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Sat, 15 Aug 2015 05:24:06 -0400 Subject: [PATCH 2/3] add parallel support 4 Win32 dmake-COREDIR parallelism part 2 -defer the mass copy of headers to COREDIR until XS build time and dont use COREDIR for building miniperl interp or perl interp, more thing to do in parallel this way and 1 less path to search by the CC -config_h.PL requires config.h to be in COREDIR, this XCOPY in CONFIGPM targ is where COREDIR is created if it doesnt exist already -remove a bunch of XCOPY'es, copy the headers in 1 place only, and dont copy them unless they change based on timestamps -the DYNALOADER dep already contains "..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) $(HAVE_COREDIR)" dont specify them again in Extensions and Extensions_reonly, this probably reduces the size of dep graph in dmake, since dmake explodes the graph and checks the outline of the graph/tree very inefficiently in parallel mode (architectural problem with dmake) -delete pdb files in root, otherwise switching VC version will make the next cl.exe error out that it can't write to the pdb file due to version numbers in the pdb file -reorder core .c files in a blend of VC and GCC time order, this reduces a 1:08 mins time to do "dmake -P9 ../perl522.dll" to 0:54 mins on 8 core with GCC, see details in #123867 --- pod/perldelta.pod | 11 ++++++++ win32/makefile.mk | 71 +++++++++++++++++++++++++++++------------------------ 2 files changed, 50 insertions(+), 32 deletions(-) diff --git a/pod/perldelta.pod b/pod/perldelta.pod index 7ce0d38..e4389c3 100644 --- a/pod/perldelta.pod +++ b/pod/perldelta.pod @@ -320,6 +320,17 @@ Comparing two strings that were both encoded in UTF-8 (or more precisely, UTF-EBCDIC) did not work properly until now. Since C<sort()> uses C<cmp()>, this fixes that as well. +=item Win32 + +=over + +=item * + +Parallel building has been added to the dmake C<makefile.mk> makefile. All +Win32 compilers are supported. + +=back + =back =head1 Internal Changes diff --git a/win32/makefile.mk b/win32/makefile.mk index c2f97bf..2eaec25 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -472,7 +472,7 @@ a = .a # Options # -INCLUDES = -I.\include -I. -I.. -I$(COREDIR) +INCLUDES = -I.\include -I. -I.. DEFINES = -DWIN32 .IF "$(WIN64)" == "define" DEFINES += -DWIN64 -DCONSERVATIVE @@ -560,7 +560,7 @@ RSC = rc # Options # -INCLUDES = -I$(COREDIR) -I.\include -I. -I.. +INCLUDES = -I.\include -I. -I.. #PCHFLAGS = -Fpc:\temp\vcmoduls.pch -YX DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT LOCDEFS = -DPERLDLL -DPERL_CORE @@ -863,45 +863,47 @@ XCOPY = xcopy /f /r /i /d /y RCOPY = xcopy /f /r /i /e /d /y NOOP = @rem +#first ones are arrange in compile time order for faster parallel building +#see #123867 for details MICROCORE_SRC = \ + ..\toke.c \ + ..\regcomp.c \ + ..\regexec.c \ + ..\op.c \ + ..\sv.c \ + ..\pp.c \ + ..\pp_ctl.c \ + ..\pp_sys.c \ + ..\pp_pack.c \ + ..\pp_hot.c \ + ..\gv.c \ + ..\perl.c \ + ..\utf8.c \ + ..\dump.c \ + ..\hv.c \ ..\av.c \ ..\caretx.c \ ..\deb.c \ ..\doio.c \ ..\doop.c \ ..\dquote.c \ - ..\dump.c \ ..\globals.c \ - ..\gv.c \ ..\mro_core.c \ - ..\hv.c \ ..\locale.c \ ..\keywords.c \ ..\mathoms.c \ ..\mg.c \ ..\numeric.c \ - ..\op.c \ ..\pad.c \ - ..\perl.c \ ..\perlapi.c \ ..\perly.c \ - ..\pp.c \ - ..\pp_ctl.c \ - ..\pp_hot.c \ - ..\pp_pack.c \ ..\pp_sort.c \ - ..\pp_sys.c \ ..\reentr.c \ - ..\regcomp.c \ - ..\regexec.c \ ..\run.c \ ..\scope.c \ - ..\sv.c \ ..\taint.c \ ..\time64.c \ - ..\toke.c \ ..\universal.c \ - ..\utf8.c \ ..\util.c EXTRACORE_SRC += perllib.c @@ -966,6 +968,10 @@ UUDMAP_H = ..\uudmap.h BITCOUNT_H = ..\bitcount.h MG_DATA_H = ..\mg_data.h GENERATED_HEADERS = $(UUDMAP_H) $(BITCOUNT_H) $(MG_DATA_H) +#a stub ppport.h must be generated so building XS modules, .c->.obj wise, will +#work, so this target also represents creating the COREDIR and filling it +HAVE_COREDIR = $(COREDIR)\ppport.h + MICROCORE_OBJ = $(MICROCORE_SRC:db:+$(o)) CORE_OBJ = $(MICROCORE_OBJ) $(EXTRACORE_SRC:db:+$(o)) @@ -1103,9 +1109,7 @@ regen_config_h: $(CONFIGPM): ..\config.sh config_h.PL $(MINIPERL) -I..\lib ..\configpm --chdir=.. - $(XCOPY) ..\*.h $(COREDIR)\*.* - $(XCOPY) *.h $(COREDIR)\*.* - $(RCOPY) include $(COREDIR)\*.* + $(XCOPY) config.h $(COREDIR)\*.* $(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" \ || $(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE) @@ -1381,7 +1385,7 @@ $(PERLSTATICLIB): $(PERLDLL_OBJ) Extensions_static $(LIB32) $(LIB_FLAGS) -out:$@ @Extensions_static \ @$(mktmp $(PERLDLL_OBJ)) .ENDIF - $(XCOPY) $(PERLSTATICLIB) $(COREDIR) + $(XCOPY) $(PERLSTATICLIB) $(COREDIR)\$(NULL) $(PERLEXE_RES): perlexe.rc $(PERLEXE_MANIFEST) $(PERLEXE_ICO) @@ -1392,6 +1396,13 @@ $(UUDMAP_H) $(MG_DATA_H) : $(BITCOUNT_H) $(BITCOUNT_H) : $(GENUUDMAP) $(GENUUDMAP) $(GENERATED_HEADERS) +#This generates a stub ppport.h & creates & fills /lib/CORE to allow for XS +#building .c->.obj wise (linking is a different thing). This taget is AKA +#$(HAVE_COREDIR). +$(COREDIR)\ppport.h : $(CORE_H) + $(XCOPY) *.h $(COREDIR)\*.* && $(RCOPY) include $(COREDIR)\*.* && $(XCOPY) ..\*.h $(COREDIR)\*.* + rem. > $@ + $(GENUUDMAP_OBJ) : ..\mg_raw.h $(GENUUDMAP) : $(GENUUDMAP_OBJ) @@ -1440,18 +1451,15 @@ $(PERLEXESTATIC): $(PERLSTATICLIB) $(CONFIGPM) $(PERLEXEST_OBJ) $(PERLEXE_RES) #------------------------------------------------------------------------------- # There's no direct way to mark a dependency on # DynaLoader.pm, so this will have to do -Extensions : ..\make_ext.pl ..\lib\buildcustomize.pl $(PERLDEP) $(CONFIGPM) $(DYNALOADER) - $(XCOPY) ..\*.h $(COREDIR)\*.* - if not exist $(COREDIR)\ppport.h rem. > $(COREDIR)\ppport.h + +#most of deps of this target are in DYNALOADER and therefore omitted here +Extensions : $(PERLDEP) $(DYNALOADER) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic -Extensions_reonly : ..\make_ext.pl ..\lib\buildcustomize.pl $(PERLDEP) $(CONFIGPM) $(DYNALOADER) - $(XCOPY) ..\*.h $(COREDIR)\*.* +Extensions_reonly : $(PERLDEP) $(DYNALOADER) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re -Extensions_static : ..\make_ext.pl ..\lib\buildcustomize.pl list_static_libs.pl $(CONFIGPM) - $(XCOPY) ..\*.h $(COREDIR)\*.* - if not exist $(COREDIR)\ppport.h rem. > $(COREDIR)\ppport.h +Extensions_static : ..\make_ext.pl ..\lib\buildcustomize.pl list_static_libs.pl $(CONFIGPM) $(HAVE_COREDIR) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static $(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static @@ -1459,8 +1467,7 @@ Extensions_nonxs : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) ..\pod\pe $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs #lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building -$(DYNALOADER) : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) - $(XCOPY) ..\*.h $(COREDIR)\*.* +$(DYNALOADER) : ..\make_ext.pl ..\lib\buildcustomize.pl $(CONFIGPM) $(HAVE_COREDIR) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib Extensions_clean : @@ -1760,7 +1767,7 @@ _clean : -@erase ..\*$(o) ..\*$(a) ..\*.exp *$(o) *$(a) *.exp *.res -@erase ..\t\*.exe ..\t\*.dll ..\t\*.bat -@erase *.ilk - -@erase *.pdb + -@erase *.pdb ..\*.pdb -@erase Extensions_static -- 1.7.9.msysgit.0
Subject: 0003-add-parallel-support-4-Win32-dmake-uudmap-no-2nd-run.patch
From 70a12f1d9a67e47e203b2341548aa1a15dfef9d3 Mon Sep 17 00:00:00 2001 From: Daniel Dragan <bulk88@hotmail.com> Date: Sat, 15 Aug 2015 05:39:55 -0400 Subject: [PATCH 3/3] add parallel support 4 Win32 dmake-uudmap+no 2nd run of config_h.PL part 3 -in $(CONFIGPM) target, remove config_h.PL running dmake again to rebuild $(CONFIGPM) target, configpm the script does not use config.h as input. The $(CONFIGPM) target is a choke/serialization point for parallel building, so it is high value to make it as fast as possible. Maybe a long time ago, configpm read config.h but it doesn't anymore. Nearly all the dynamic config vars are determined in config_sh.PL script and ..\config.sh target. config_h.PL contains very little logic, and this logic is only for config.h, and only when you want a var to be different in config.h than it is in Config.pm/Config_heavy.pl. Putting a breakpoint ("system 'pause';") in config_h.PL, copying Config.pm/Config_heavy.pl to ".old" versions, continuing execution in config_h.PL, then diffing the 2 new Configs with the old 2 Configs before pre-config_h.PL and "$(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE)" shows no differece. When dmake runs needlessly a 2nd time it emits a warning that nothing changed. This is intentional, since configpm doesn't update the timestamps /update the files if the Config files are identical to the old ones to prevent mass clean+rebuild of all modules when nothing global changed. dmake: Warning: -- Target [..\lib\Config.pm] was made but the time stamp has not been updated. -reduce number of steps to build generate_uudmap.exe. Compiling (140 ms on GCC 4.8.3) and linking (561 ms) is heavy weight compared to running generate_uudmap.exe (31 ms) and compiling globals.c (483 ms). dmake's algo for doing parallel is poor. On the main parallel pass with all the mini core .c files compiling ("MINI_OBJ"), only generate_uudmap.o will be generated as part of the chain of deps hanging off globals.o, after the 1st tier of MINI_OBJ has compiled in parallel, dmake will do another sweep of MINI_OBJ, this time it will find generate_uudmap.exe node and run linking, at this 2nd tier nothing ran except linking generate_uudmap. No parallel anything. This is a chokepoint. Then dmake does a 3rd sweep and runs the 3rd tier, which is run generate_uudmap.exe, again nothing parallel happens here. By combining compiling and linking in 1 target, the time spent serially linking generate_uudmap.exe, is moved into parallel time when the rest of MINI_OBJ compiles. Also a little bit of CPU and IO is saved by launching less processes (less cmd.exe shells, gcc/g++ process ran only once (gcc/g++ still run many standalone procs internally but now only 1 gcc was started not 2)). -suppress warnings from dmake with .UPDATEALL, the 3 generated headers are made at the same time in real life, in the makefile it was written as if they weren't dmake: Warning:--Found file corresponding to virtual target [..\uudmap.h]. dmake: Warning:--Found file corresponding to virtual target [..\mg_data.h]. -move generating the headers to the target that generates generate_uudmap.exe, this moves some serialization time, albeit small (30 ms) from right before miniperl link, to parallel time during 1st pass of MINI_OBJ -collapse 3 shell runs of xcopy into 1 run of shell "timeit cmd /c "cmd /c xcopy /f /r /i /d /y ..\perl.exe ..\t\ && cmd /c xcopy /f /r /i /d /y ..\perl523.dll ..\t\ && cmd /c xcopy /f /r /i /d /y ..\perlglob.exe ..\t\"" takes 140 ms "timeit cmd /c "xcopy /f /r /i /d /y ..\perl.exe ..\t\ && xcopy /f /r /i /d /y ..\perl523.dll ..\t\ && xcopy /f /r /i /d /y ..\perlglob.exe ..\t\"" takes 78 ms on my machine --- win32/makefile.mk | 31 +++++++++++-------------------- 1 files changed, 11 insertions(+), 20 deletions(-) diff --git a/win32/makefile.mk b/win32/makefile.mk index 2eaec25..6288ead 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -980,7 +980,6 @@ MINICORE_OBJ = $(MINIDIR)\{$(MICROCORE_OBJ:f) miniperlmain$(o) perlio$(o)} MINIWIN32_OBJ = $(MINIDIR)\{$(WIN32_OBJ:f)} MINI_OBJ = $(MINICORE_OBJ) $(MINIWIN32_OBJ) DLL_OBJ = $(DYNALOADER) -GENUUDMAP_OBJ = $(GENUUDMAP:db:+$(o)) PERLDLL_OBJ = $(CORE_OBJ) PERLEXE_OBJ = perlmain$(o) @@ -1110,8 +1109,7 @@ regen_config_h: $(CONFIGPM): ..\config.sh config_h.PL $(MINIPERL) -I..\lib ..\configpm --chdir=.. $(XCOPY) config.h $(COREDIR)\*.* - $(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" \ - || $(PLMAKE) $(MAKEMACROS) $(CONFIGPM) $(MAKEFILE) + -$(MINIPERL) -I..\lib config_h.PL "ARCHPREFIX=$(ARCHPREFIX)" .\config.h : $(CONFIGPM) @@ -1391,9 +1389,14 @@ $(PERLEXE_RES): perlexe.rc $(PERLEXE_MANIFEST) $(PERLEXE_ICO) $(MINIDIR)\globals$(o) : $(GENERATED_HEADERS) -$(UUDMAP_H) $(MG_DATA_H) : $(BITCOUNT_H) - -$(BITCOUNT_H) : $(GENUUDMAP) +$(GENUUDMAP) $(GENERATED_HEADERS) .UPDATEALL : ..\mg_raw.h +.IF "$(CCTYPE)" == "GCC" + $(LINK32) $(CFLAGS_O) -o..\generate_uudmap.exe ..\generate_uudmap.c $(BLINK_FLAGS) \ + $(mktmp $(LKPRE) $(LIBFILES) $(LKPOST)) +.ELSE + $(CC) $(CFLAGS_O) -Fe..\generate_uudmap.exe ..\generate_uudmap.c @$(mktmp -link $(LIBFILES)) -link $(BLINK_FLAGS) + $(EMBED_EXE_MANI:s/$@/..\generate_uudmap.exe/) +.ENDIF $(GENUUDMAP) $(GENERATED_HEADERS) #This generates a stub ppport.h & creates & fills /lib/CORE to allow for XS @@ -1403,17 +1406,6 @@ $(COREDIR)\ppport.h : $(CORE_H) $(XCOPY) *.h $(COREDIR)\*.* && $(RCOPY) include $(COREDIR)\*.* && $(XCOPY) ..\*.h $(COREDIR)\*.* rem. > $@ -$(GENUUDMAP_OBJ) : ..\mg_raw.h - -$(GENUUDMAP) : $(GENUUDMAP_OBJ) -.IF "$(CCTYPE)" == "GCC" - $(LINK32) -v -o $@ $(BLINK_FLAGS) \ - $(mktmp $(LKPRE) $(GENUUDMAP_OBJ) $(LIBFILES) $(LKPOST)) -.ELSE - $(LINK32) -out:$@ $(BLINK_FLAGS) @$(mktmp $(LIBFILES) $(GENUUDMAP_OBJ)) - $(EMBED_EXE_MANI) -.ENDIF - perlmain.c : runperl.c copy runperl.c perlmain.c @@ -1690,9 +1682,8 @@ minitest : .\config.h $(HAVEMINIPERL) ..\git_version.h $(GLOBEXE) $(CONFIGPM) $( cd ..\t && perl.exe TEST base/*.t comp/*.t cmd/*.t run/*.t io/*.t re/*.t opbasic/*.t op/*.t uni/*.t perf/*.t pragma/*.t test-prep : all utils ..\pod\perltoc.pod $(TESTPREPGCC) - $(XCOPY) $(PERLEXE) ..\t\$(NULL) - $(XCOPY) $(PERLDLL) ..\t\$(NULL) - $(XCOPY) $(GLOBEXE) ..\t\$(NULL) + $(XCOPY) $(PERLEXE) ..\t\$(NULL) && $(XCOPY) $(PERLDLL) ..\t\$(NULL) \ + && $(XCOPY) $(GLOBEXE) ..\t\$(NULL) # If building with gcc versions 4.x.x or greater, then # the GCC helper DLL will also need copied to the test directory. -- 1.7.9.msysgit.0
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 437b
On Thu Sep 03 22:35:03 2015, bulk88 wrote: Show quoted text
> On Sun Aug 16 21:28:10 2015, tonyc wrote:
> > As discussed in IRC, Time::HiRes isn't upstream blead, so this can't > > be applied as is. > > > > Tony
> > rebased+time hires patch removed
Thanks, applied as 3bdc51af3f9be1ab22a176fefca16c329755b094, d89ea36076afa0a17a3faa7cb33a1c9c215a26eb and c2c7bda0885d19e01f0d998f22d2248d7663e957. Thanks for all the effort you've put into this. Tony


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org