Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storable build hangs when building 5.27.9 on Windows XP #16428

Closed
p5pRT opened this issue Feb 21, 2018 · 8 comments
Closed

Storable build hangs when building 5.27.9 on Windows XP #16428

p5pRT opened this issue Feb 21, 2018 · 8 comments

Comments

@p5pRT
Copy link

p5pRT commented Feb 21, 2018

Migrated from rt.perl.org#132893 (status was 'resolved')

Searchable as RT132893$

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2018

From @chorny

Created by @chorny

..\perl.exe -I..\lib -I. ..\dist\Storable\stacksize --core
probe for max. stack sizes...
65000 failed, try less 32550 ...
32550 failed, try less 16325 ...
16325 failed, try less 8213 ...
8213 failed, try less 4157 ...
4157 passed, try more 6185 ...
6185 passed, try more 7199 ...
7199 failed, try less 6692 ...
6692 failed, try less 6439 ...
6439 failed, try less 6312 ...
6312 failed, try less 6249 ...
6249 failed, try less 6217 ...
6217 failed, try less 6201 ...
6201 failed, try less 6193 ...
6193 passed, try more 6197 ...
6197 passed, try more 6199 ...
6199 passed, try more 6200 ...
6200 passed, try more 6200 ...
MAX_DEPTH = 6200
3100 passed, try more 4650 ...
4650 passed, try more 5425 ...
5425 passed, try more 5812 ...
Out of memory!
(hangs)

Also `stacksize` is executed on every stage - build/tests/install, so
I had to kill it 3 times. After killing hanging process, stacksize
continues search.
This is same problem as with Storable on CPAN. In CPAN version I
solved it with Win32​::Job, but it is not a core module. See
rurban/Storable#2
It should be possible to use Win32​::Process, but it is not a core module either.

cpantesters results show that with RURBAN/Storable-3.05_16.tar.gz
these values can be partially predicted. Note that values for 3.05_16
on CPAN and 3.06 in core are different. And `stacksize` in 3.05_16
currently does not hang even when not using Win32​::Job.

Windows 10 64-bit, perl 5.26.0 64-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows 10 64-bit, perl 5.22.1 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows 10 64-bit, perl 5.22.1 64-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows 10 64-bit, perl 5.24.0 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.20.1 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.18.2 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.18.2 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=512
Windows XP 32-bit inside VM, perl 5.26.0 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=512
Windows XP 32-bit inside VM, perl 5.12.2 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit, perl 5.27.9 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit, perl 5.14.0 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.16.0, 5.16.3 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257

(Total 2 installations of Windows 10 and 3 installations of Windows XP)

http​://www.cpantesters.org/cpan/report/11bf30ab-6bf4-1014-b5fc-ed671f639c1c
Windows 7 ?-bit (probably 32-bit), perl 5.14.4 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257

Perl Info

Flags:
    category=library
    severity=high
    module=Storable

Site configuration information for perl 5.27.9:

Configured by strawberry-perl at Wed Feb 21 04:27:05 2018.

Summary of my perl5 (revision 5 version 27 subversion 9) configuration:

  Platform:
    osname=MSWin32
    osvers=5.1.2600
    archname=MSWin32-x86-multi-thread-64int
    uname='Win32 strawberry-perl 5.27.9.1-beta1 #1 Wed Feb 21 04:24:03
2018 i386'
    config_args='undef'
    hint=recommended
    useposix=true
    d_sigaction=undef
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='gcc'
    ccflags =' -s -O2 -DWIN32 -D__USE_MINGW_ANSI_STDIO
-DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS
-DUSE_PERLIO -fwrapv -fno-strict-aliasing -mms-bitfields'
    optimize='-s -O2'
    cppflags='-DWIN32'
    ccversion=''
    gccversion='7.1.0'
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=4
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=12
    longdblkind=3
    ivtype='long long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='long long'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='g++.exe'
    ldflags ='-s -L"C:\strawberry\perl\lib\CORE" -L"C:\strawberry\c\lib"'
    libpth=C:\strawberry\c\lib C:\strawberry\c\i686-w64-mingw32\lib
C:\strawberry\c\lib\gcc\i686-w64-mingw32\7.1.0
    libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32
-ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32
-lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool
-lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid
-lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    libc=
    so=dll
    useshrplib=true
    libperl=libperl527.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs
    dlext=xs.dll
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags='-mdll -s -L"C:\strawberry\perl\lib\CORE" -L"C:\strawberry\c\lib"'



@INC for perl 5.27.9:
    C:/strawberry/perl/site/lib
    C:/strawberry/perl/vendor/lib
    C:/strawberry/perl/lib


Environment for perl 5.27.9:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=C:\strawberry\perl\site\bin;C:\strawberry\perl\bin;C:\strawberry\c\bin;C:\Program
Files\Far\;C:\Prg\PHP\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program
Files\ATI Technologies\ATI.ACE\Core-Static;C:\strawberry514\c\bin;C:\strawberry514\perl\site\bin;C:\strawberry514\perl\bin;C:\Program
Files\TortoiseHg\;C:\Program Files\MySQL\MySQL Server
5.5\bin;c:\util\;C:\Prg\Subversion\bin;C:\Program
Files\TortoiseSVN\bin;D:\vagrant\vagrant\bin;C:\Prg\TortoiseGit\bin;C:\Prg\Git\cmd;C:\Program
Files\Skype\Phone\;
    PERL_BADLANG (unset)
    SHELL (unset)


-- 
Alexandr Ciornii, http://chorny.net

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2018

From @jkeenan

On Wed, 21 Feb 2018 18​:46​:33 GMT, chorny wrote​:

This is a bug report for perl from alexchorny@​gmail.com,
generated with the help of perlbug 1.41 running under perl 5.27.9.

-----------------------------------------------------------------
[Please describe your issue here]

..\perl.exe -I..\lib -I. ..\dist\Storable\stacksize --core
probe for max. stack sizes...
65000 failed, try less 32550 ...
32550 failed, try less 16325 ...
16325 failed, try less 8213 ...
8213 failed, try less 4157 ...
4157 passed, try more 6185 ...
6185 passed, try more 7199 ...
7199 failed, try less 6692 ...
6692 failed, try less 6439 ...
6439 failed, try less 6312 ...
6312 failed, try less 6249 ...
6249 failed, try less 6217 ...
6217 failed, try less 6201 ...
6201 failed, try less 6193 ...
6193 passed, try more 6197 ...
6197 passed, try more 6199 ...
6199 passed, try more 6200 ...
6200 passed, try more 6200 ...
MAX_DEPTH = 6200
3100 passed, try more 4650 ...
4650 passed, try more 5425 ...
5425 passed, try more 5812 ...
Out of memory!
(hangs)

Also `stacksize` is executed on every stage - build/tests/install, so
I had to kill it 3 times. After killing hanging process, stacksize
continues search.
This is same problem as with Storable on CPAN. In CPAN version I
solved it with Win32​::Job, but it is not a core module. See
rurban/Storable#2
It should be possible to use Win32​::Process, but it is not a core
module either.

cpantesters results show that with RURBAN/Storable-3.05_16.tar.gz

Note that that's an unauthorized release​: http​://search.cpan.org/~rurban/Storable-3.05_16/

these values can be partially predicted. Note that values for 3.05_16
on CPAN and 3.06 in core are different. And `stacksize` in 3.05_16
currently does not hang even when not using Win32​::Job.

Windows 10 64-bit, perl 5.26.0 64-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows 10 64-bit, perl 5.22.1 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows 10 64-bit, perl 5.22.1 64-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows 10 64-bit, perl 5.24.0 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.20.1 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.18.2 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.18.2 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=512
Windows XP 32-bit inside VM, perl 5.26.0 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=512
Windows XP 32-bit inside VM, perl 5.12.2 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit, perl 5.27.9 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit, perl 5.14.0 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.16.0, 5.16.3 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257

(Total 2 installations of Windows 10 and 3 installations of Windows
XP)

http​://www.cpantesters.org/cpan/report/11bf30ab-6bf4-1014-b5fc-
ed671f639c1c
Windows 7 ?-bit (probably 32-bit), perl 5.14.4 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=library
severity=high
module=Storable
---
Site configuration information for perl 5.27.9​:

Configured by strawberry-perl at Wed Feb 21 04​:27​:05 2018.

Summary of my perl5 (revision 5 version 27 subversion 9)
configuration​:

Platform​:
osname=MSWin32
osvers=5.1.2600
archname=MSWin32-x86-multi-thread-64int
uname='Win32 strawberry-perl 5.27.9.1-beta1 #1 Wed Feb 21 04​:24​:03
2018 i386'
config_args='undef'
hint=recommended
useposix=true
d_sigaction=undef
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=undef
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler​:
cc='gcc'
ccflags =' -s -O2 -DWIN32 -D__USE_MINGW_ANSI_STDIO
-DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS
-DUSE_PERLIO -fwrapv -fno-strict-aliasing -mms-bitfields'
optimize='-s -O2'
cppflags='-DWIN32'
ccversion=''
gccversion='7.1.0'
gccosandvers=''
intsize=4
longsize=4
ptrsize=4
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=12
longdblkind=3
ivtype='long long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='long long'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries​:
ld='g++.exe'
ldflags ='-s -L"C​:\strawberry\perl\lib\CORE"
-L"C​:\strawberry\c\lib"'
libpth=C​:\strawberry\c\lib C​:\strawberry\c\i686-w64-mingw32\lib
C​:\strawberry\c\lib\gcc\i686-w64-mingw32\7.1.0
libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32
-ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32
-lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool
-lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid
-lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
libc=
so=dll
useshrplib=true
libperl=libperl527.a
gnulibc_version=''
Dynamic Linking​:
dlsrc=dl_win32.xs
dlext=xs.dll
d_dlsymun=undef
ccdlflags=' '
cccdlflags=' '
lddlflags='-mdll -s -L"C​:\strawberry\perl\lib\CORE"
-L"C​:\strawberry\c\lib"'

---
@​INC for perl 5.27.9​:
C​:/strawberry/perl/site/lib
C​:/strawberry/perl/vendor/lib
C​:/strawberry/perl/lib

---
Environment for perl 5.27.9​:
HOME (unset)
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=C​:\strawberry\perl\site\bin;C​:\strawberry\perl\bin;C​:\strawberry\c\bin;C​:\Program
Files\Far\;C​:\Prg\PHP\;C​:\WINDOWS\system32;C​:\WINDOWS;C​:\WINDOWS\System32\Wbem;C​:\Program
Files\ATI Technologies\ATI.ACE\Core-
Static;C​:\strawberry514\c\bin;C​:\strawberry514\perl\site\bin;C​:\strawberry514\perl\bin;C​:\Program
Files\TortoiseHg\;C​:\Program Files\MySQL\MySQL Server
5.5\bin;c​:\util\;C​:\Prg\Subversion\bin;C​:\Program
Files\TortoiseSVN\bin;D​:\vagrant\vagrant\bin;C​:\Prg\TortoiseGit\bin;C​:\Prg\Git\cmd;C​:\Program
Files\Skype\Phone\;
PERL_BADLANG (unset)
SHELL (unset)

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Feb 22, 2018

From @tonycoz

On Wed, 21 Feb 2018 10​:46​:33 -0800, chorny wrote​:

..\perl.exe -I..\lib -I. ..\dist\Storable\stacksize --core
probe for max. stack sizes...
65000 failed, try less 32550 ...
32550 failed, try less 16325 ...
16325 failed, try less 8213 ...
8213 failed, try less 4157 ...
4157 passed, try more 6185 ...
6185 passed, try more 7199 ...
7199 failed, try less 6692 ...
6692 failed, try less 6439 ...
6439 failed, try less 6312 ...
6312 failed, try less 6249 ...
6249 failed, try less 6217 ...
6217 failed, try less 6201 ...
6201 failed, try less 6193 ...
6193 passed, try more 6197 ...
6197 passed, try more 6199 ...
6199 passed, try more 6200 ...
6200 passed, try more 6200 ...
MAX_DEPTH = 6200
3100 passed, try more 4650 ...
4650 passed, try more 5425 ...
5425 passed, try more 5812 ...
Out of memory!
(hangs)

I've reproduced this. It requires USE_64_BIT_INT, I haven't managed to track down the cause yet.

Also `stacksize` is executed on every stage - build/tests/install, so
I had to kill it 3 times. After killing hanging process, stacksize
continues search.
This is same problem as with Storable on CPAN. In CPAN version I
solved it with Win32​::Job, but it is not a core module. See
rurban/Storable#2
It should be possible to use Win32​::Process, but it is not a core
module either.

I'll see if I can prevent running the probe several times, probably by having the rule in win32/(GNU)?[mM]akefile(.mk)? run a sub-make in the Storable directory instead of running it directly.

cpantesters results show that with RURBAN/Storable-3.05_16.tar.gz
these values can be partially predicted. Note that values for 3.05_16
on CPAN and 3.06 in core are different. And `stacksize` in 3.05_16
currently does not hang even when not using Win32​::Job.

Windows 10 64-bit, perl 5.26.0 64-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows 10 64-bit, perl 5.22.1 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows 10 64-bit, perl 5.22.1 64-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows 10 64-bit, perl 5.24.0 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.20.1 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.18.2 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.18.2 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=512
Windows XP 32-bit inside VM, perl 5.26.0 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=512
Windows XP 32-bit inside VM, perl 5.12.2 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit, perl 5.27.9 32-bit+64-bit int​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit, perl 5.14.0 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257
Windows XP 32-bit inside VM, perl 5.16.0, 5.16.3 32-bit​:
PST_STACK_MAX_DEPTH=513
PST_STACK_MAX_DEPTH_HASH=257

Those values are just stacksize falling back to a default.

A predefined table is going to vary based on bitness of build, debug/non-debug, whether USE_64_BIT_INT is set, version of perl, compiler, compiler version and changes to Storable itself.

The builder of perl may adjust the stack size too.

That's just too many dimensions.

I may just do what Reini did and fallback to a default on difficult platforms like XP.

Tony

@p5pRT
Copy link
Author

p5pRT commented Feb 27, 2018

From @tonycoz

On Wed, 21 Feb 2018 20​:25​:48 -0800, tonyc wrote​:

I'll see if I can prevent running the probe several times, probably by
having the rule in win32/(GNU)?[mM]akefile(.mk)? run a sub-make in the
Storable directory instead of running it directly.

Done in b58c2b0.

I'll look at the hang/memory issue next.

Tony

@p5pRT
Copy link
Author

p5pRT commented Feb 28, 2018

From @tonycoz

On Tue, 27 Feb 2018 15​:20​:46 -0800, tonyc wrote​:

On Wed, 21 Feb 2018 20​:25​:48 -0800, tonyc wrote​:

I'll see if I can prevent running the probe several times, probably by
having the rule in win32/(GNU)?[mM]akefile(.mk)? run a sub-make in the
Storable directory instead of running it directly.

Done in b58c2b0.

I'll look at the hang/memory issue next.

XP and Vista builds now skip the probe.

Tony

@p5pRT
Copy link
Author

p5pRT commented Aug 6, 2019

From @tonycoz

On Wed, 28 Feb 2018 15​:23​:26 -0800, tonyc wrote​:

On Tue, 27 Feb 2018 15​:20​:46 -0800, tonyc wrote​:

On Wed, 21 Feb 2018 20​:25​:48 -0800, tonyc wrote​:

I'll see if I can prevent running the probe several times, probably by
having the rule in win32/(GNU)?[mM]akefile(.mk)? run a sub-make in the
Storable directory instead of running it directly.

Done in b58c2b0.

I'll look at the hang/memory issue next.

XP and Vista builds now skip the probe.

All builds now skip the probe, it ended up being too flaky.

This was included in perl 5.30.

Thanks for the report.

Tony

@p5pRT
Copy link
Author

p5pRT commented Aug 6, 2019

@tonycoz - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant