Skip Menu |
Report information
Id: 132926
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: bulk88 <bulk88 [at] hotmail.com>
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: core
Perl Version: 5.27.9
Fixed In: (no value)

Attachments
0001-dont-define-and-declare-SCX_AUX_TABLES-in-every-perl.patch
0002-remove-unused-SCX-inv-data-from-re-XS-library.patch



From: bulk 88 <bulk88 [...] hotmail.com>
To: "perlbug [...] perl.org" <perlbug [...] perl.org>
Subject: [PATCH] 10s of KB to 100s of KB binary bloat with Visual C Perl related to Perl_SCX/charclass_invlists.h
Date: Thu, 1 Mar 2018 07:42:33 +0000
Download (untitled) / with headers
text/plain 6.4k
This is a bug report for perl from bulk88@hotmail.com, generated with the help of perlbug 1.41 running under perl 5.27.9. ----------------------------------------------------------------- [Please describe your issue here] See attached patches. I noticed after compiling a new blead Perl with Visual C, then opening perl527.dll with a disassembler there were 100s of new data symbols that looked like duplicates that took lots and lots of holding page down to scroll past. I noticed none of them are used. I also noticed libperl was much fatter on disk than before (5.26/5.25). I narrowed the bloat to commit "charclasslists.h: script enums visible to CORE,EXT" https://perl5.git.perl.org/perl.git/commitdiff/fea79782ed31ac4faa71eb6193faff54b4bbe249 . Win32 GCC perl doesn't have this bloat. Its a bug/specialness of MSVC linker. The first commit at fixing these RE related "const static" unreferenced but not optimized out was https://perl5.git.perl.org/perl.git/commit/f1bfbdda19c74059bee234561325e09c9d0a9708 , but that was very minor. "charclasslists.h: script enums visible to CORE,EXT"'s bloat is very noticeable on disk (and therefore mapped memory too since these are bins). I offlist communicated with KHW but he didn't have any immediate solutions to the "charclasslists.h: script enums visible to CORE,EXT" problem where the same exact byte-wise/content-wise large const static array is declared and defined in every .o file. So I made these 2 patches. Before/after file sizes compiled with MSVC 2003 for all core XS DLLs follow. Any XS lib that did "#define PERL_EXT" got the bloat. My opinion is this MSVC binary bloat problem is a blocker for 5.28 perl. ------------------------------------ --- C:\perl521\src\b4.txt +++ C:\perl521\src\af.txt @@ -1,9 +1,9 @@ -before +after $ ls -l **/*.dll | awk '{print $5 " " $9}' -19968 lib/auto/arybase/arybase.dll -14848 lib/auto/attributes/attributes.dll -69632 lib/auto/B/B.dll +11776 lib/auto/arybase/arybase.dll +7168 lib/auto/attributes/attributes.dll +61440 lib/auto/B/B.dll 61440 lib/auto/Compress/Raw/Bzip2/Bzip2.dll 86016 lib/auto/Compress/Raw/Zlib/Zlib.dll 9216 lib/auto/Cwd/Cwd.dll @@ -26,9 +26,9 @@ 13824 lib/auto/File/Glob/Glob.dll 7680 lib/auto/Filter/Util/Call/Call.dll 9216 lib/auto/Hash/Util/FieldHash/FieldHash.dll -20480 lib/auto/Hash/Util/Util.dll +12288 lib/auto/Hash/Util/Util.dll 11264 lib/auto/IO/IO.dll -36352 lib/auto/List/Util/Util.dll +28672 lib/auto/List/Util/Util.dll 9216 lib/auto/Math/BigInt/FastCalc/FastCalc.dll 8192 lib/auto/MIME/Base64/Base64.dll 11264 lib/auto/mro/mro.dll @@ -37,8 +37,8 @@ 4096 lib/auto/PerlIO/mmap/mmap.dll 7680 lib/auto/PerlIO/scalar/scalar.dll 11264 lib/auto/PerlIO/via/via.dll -73728 lib/auto/POSIX/POSIX.dll -303104 lib/auto/re/re.dll +65536 lib/auto/POSIX/POSIX.dll +286720 lib/auto/re/re.dll 15360 lib/auto/SDBM_File/SDBM_File.dll 18944 lib/auto/Socket/Socket.dll 69632 lib/auto/Storable/Storable.dll @@ -52,7 +52,7 @@ 225280 lib/auto/Unicode/Normalize/Normalize.dll 29184 lib/auto/Win32/Win32.dll 47616 lib/auto/Win32API/File/File.dll -237568 lib/auto/XS/APItest/APItest.dll +221184 lib/auto/XS/APItest/APItest.dll 24064 lib/auto/XS/Typemap/Typemap.dll -1617920 perl527.dll -1617920 t/perl527.dll +1261568 perl527.dll +1261568 t/perl527.dll -------------------------------------------------- [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=low --- Site configuration information for perl 5.27.9: Configured by Administrator at Tue Jan 30 20:34:30 2018. Summary of my perl5 (revision 5 version 27 subversion 9) configuration: Platform: osname=MSWin32 osvers=5.2.3790 archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended useposix=true d_sigaction=undef useithreads=define usemultiplicity=define use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cl' ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32 -D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DWIN32_NO_REGISTRY' optimize='-O1 -MD -Zi -DNDEBUG -GL' cppflags='-DWIN32' ccversion='15.00.30729.01' gccversion='' gccosandvers='' intsize=4 longsize=4 ptrsize=4 doublesize=8 byteorder=1234 doublekind=3 d_longlong=undef longlongsize=8 d_longdbl=define longdblsize=8 longdblkind=0 ivtype='long' ivsize=4 nvtype='double' nvsize=8 Off_t='__int64' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='link' ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86' libpth="C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\lib" libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib libc=msvcrt.lib so=dll useshrplib=true libperl=perl527.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs dlext=dll d_dlsymun=undef ccdlflags=' ' cccdlflags=' ' lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86' --- @INC for perl 5.27.9: lib C:/p527/srcnew/lib --- Environment for perl 5.27.9: CYGWIN=tty HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH=/usr/lib/x86:/usr/X11R6/lib LOGDIR (unset) PATH=C:\WINDOWS\system32;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN;C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin;C:\Perl\bin;C:\WINDOWS;C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE;C:\Program Files (x86)\Git\bin;C:\sp3220\c\bin; PERL_BADLANG (unset) SHELL (unset)

Message body is not shown because sender requested not to inline it.

Message body is not shown because sender requested not to inline it.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 149b
Does my patch 49cd072660b3b1858be5f27df107b3b42498a3f3 which you suggested fix this for 5.28? (I haven't done any investigating) -- Karl Williamson
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 685b
On Fri, 02 Mar 2018 12:05:52 -0800, khw wrote: Show quoted text
> Does my patch 49cd072660b3b1858be5f27df107b3b42498a3f3 which you > suggested fix this for 5.28? (I haven't done any investigating)
No. Perl_SCX is still defined in every .o file and there are still 43 copies of it inside perl527.dll. mk_invlists.pl/charclass_invlists.h need to be changed to limit Perl_SCX to regexec.c only the way it was before. The SCX_enum arg being part of extern export API because of isSCRIPT_RUN is why this patch split the #define for the enum from the map/list table but now that that arg is gone from extern API, mk_invlists.pl can be changed back to the old way it was. -- bulk88 ~ bulk88 at hotmail.com
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 789b
On Fri, 02 Mar 2018 15:45:25 -0800, bulk88 wrote: Show quoted text
> On Fri, 02 Mar 2018 12:05:52 -0800, khw wrote:
> > Does my patch 49cd072660b3b1858be5f27df107b3b42498a3f3 which you > > suggested fix this for 5.28? (I haven't done any investigating)
> > No. Perl_SCX is still defined in every .o file and there are still 43 > copies of it inside perl527.dll. mk_invlists.pl/charclass_invlists.h > need to be changed to limit Perl_SCX to regexec.c only the way it was > before. The SCX_enum arg being part of extern export API because of > isSCRIPT_RUN is why this patch split the #define for the enum from the > map/list table but now that that arg is gone from extern API, > mk_invlists.pl can be changed back to the old way it was.
Was this fixed by c0221e16334efbe9723722578949dc3916d669d4 ? Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 912b
On Tue, 06 Mar 2018 19:30:53 -0800, tonyc wrote: Show quoted text
> On Fri, 02 Mar 2018 15:45:25 -0800, bulk88 wrote:
> > On Fri, 02 Mar 2018 12:05:52 -0800, khw wrote:
> > > Does my patch 49cd072660b3b1858be5f27df107b3b42498a3f3 which you > > > suggested fix this for 5.28? (I haven't done any investigating)
> > > > No. Perl_SCX is still defined in every .o file and there are still 43 > > copies of it inside perl527.dll. mk_invlists.pl/charclass_invlists.h > > need to be changed to limit Perl_SCX to regexec.c only the way it was > > before. The SCX_enum arg being part of extern export API because of > > isSCRIPT_RUN is why this patch split the #define for the enum from the > > map/list table but now that that arg is gone from extern API, > > mk_invlists.pl can be changed back to the old way it was.
> > Was this fixed by c0221e16334efbe9723722578949dc3916d669d4 ? > > Tony
Yes. -- bulk88 ~ bulk88 at hotmail.com


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org