Skip Menu |
Report information
Id: 132870
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: jacques.germishuys [at] gmail.com
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: critical
Type: library
Perl Version: 5.26.0
Fixed In: (no value)

Attachments
0001-perl-132870-workaround-possibe-VC2017-compiler-bug.patch



Subject: Storable dclone() broken (latest VS 2017 32-bit)
From: Jacques Germishuys <jacques.germishuys [...] gmail.com>
Date: Thu, 15 Feb 2018 15:11:08 +0200
To: perlbug [...] perl.org
Download (untitled) / with headers
text/plain 5.8k
This is a bug report for perl from jacques.germishuys@gmail.com,
generated with the help of perlbug 1.40 running under perl 5.26.0.


-----------------------------------------------------------------
[Please describe your issue here]

The following program shows that freeze() and thaw() is currently
broken with VS 2017:


use Data::Dumper;
use Storable qw/dclone/;
use Devel::Peek;

my $x =
{
value => 1
};

print Dumper ($x);
Dump ($x);

my $y = dclone ($x);

print Dumper ($y);
Dump ($y);


Output:

$VAR1 = {
  'value' => 1
};
SV = IV(0x166b3b4) at 0x166b3b8
  REFCNT = 1
  FLAGS = (ROK)
  RV = 0x164d6f8
  SV = PVHV(0x1653810) at 0x164d6f8
    REFCNT = 1
    FLAGS = (OOK,SHAREKEYS)
    AUX_FLAGS = 0
    ARRAY = 0x172e390  (0:7, 1:1)
    hash quality = 100.0%
    KEYS = 1
    FILL = 1
    MAX = 7
    RITER = -1
    EITER = 0x0
    RAND = 0x7ebeb2c7
    Elt "value" HASH = 0x9dce435e
    SV = IV(0x164d7d4) at 0x164d7d8
      REFCNT = 1
      FLAGS = (IOK,pIOK)
      IV = 1
$VAR1 = {
  'value' => -255
};
SV = IV(0x1661bfc) at 0x1661c00
  REFCNT = 1
  FLAGS = (ROK)
  RV = 0x164d808
  SV = PVHV(0x1653dd0) at 0x164d808
    REFCNT = 1
    FLAGS = (OOK,SHAREKEYS)
    AUX_FLAGS = 0
    ARRAY = 0x172e250  (0:7, 1:1)
    hash quality = 100.0%
    KEYS = 1
    FILL = 1
    MAX = 7
    RITER = -1
    EITER = 0x0
    RAND = 0x5820a183
    Elt "value" HASH = 0x9dce435e
    SV = IV(0x166114c) at 0x1661150
      REFCNT = 1
      FLAGS = (IOK,pIOK)
      IV = -255


The value of 1 got changed to -255. I've also seen 0 change to -256!


[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=library
    severity=critical
    module=Storable
---
Site configuration information for perl 5.26.0:

Configured by build at Fri Feb 2 13:01:36 2018.

Summary of my perl5 (revision 5 version 26 subversion 0) configuration:
   
  Platform:
    osname=MSWin32
    osvers=10.0
    archname=MSWin32
    uname=''
    config_args='undef'
    hint=recommended
    useposix=true
    d_sigaction=undef
    useithreads=define
    usemultiplicity=define
    use64bitint=undef
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cl'
    ccflags ='-nologo -GF -W3 -O2 -MD -Zi -DNDEBUG -GL -FS -DWIN32 -D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -D_WINSOCK_DEPRECATED_NO_WARNINGS  -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -D_USE_32BIT_TIME_T'
    optimize='-O2 -MD -Zi -DNDEBUG -GL -FS'
    cppflags='-DWIN32'
    ccversion='19.12.25835'
    gccversion=''
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=4
    doublesize=8
    byteorder=1234
    doublekind=3
    d_longlong=undef
    longlongsize=8
    d_longdbl=define
    longdblsize=8
    longdblkind=0
    ivtype='long'
    ivsize=4
    nvtype='double'
    nvsize=8
    Off_t='__int64'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='link'
    ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"C:\opt\Striata\vs-2017-perl-5.26.0-x86\\perl\lib\MSWin32\CORE" -machine:x86 -subsystem:console,"5.01"'
    libpth=\lib
    libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
    perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
    libc=ucrt.lib
    so=dll
    useshrplib=true
    libperl=perl526.lib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs
    dlext=dll
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"C:\opt\Striata\vs-2017-perl-5.26.0-x86\\perl\lib\MSWin32\CORE" -machine:x86 -subsystem:console,"5.01"'


---
@INC for perl 5.26.0:
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/vendor/lib/MSWin32
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/vendor/lib
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/lib/MSWin32
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/lib
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/site/lib/MSWin32
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/site/lib

---
Environment for perl 5.26.0:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=C:\VS2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x86;C:\VS2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x64;C:\VS2017\Common7\IDE\VC\VCPackages;C:\VS2017\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\VS2017\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;C:\VS2017\MSBuild\15.0\bin\Roslyn;C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\x64\;C:\Program Files (x86)\HTML Help Workshop;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files (x86)\Windows
Kits\10\bin\10.0.16299.0\x64;C:\VS2017\\MSBuild\15.0\bin;C:\Windows\Microsoft.NET\Framework64\v4.0.30319;C:\VS2017\Common7\IDE\;C:\VS2017\Common7\Tools\;C:\dev\vs2017-32\bin;C:\opt\Striata\vs-2017-perl-5.26.0-x86\perl\bin;C:\opt\Striata\bin\64;C:\opt\Striata\bin;C:\opt\Striata\git\cmd;C:\opt\Striata\cmake\bin;C:\opt\Striata\wix;C:\dev\vs2017-32\lib;C:\Python27\;C:\Python27\Scripts;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\jdk1.7.0_80\bin;C:\Program Files (x86)\WinSCP\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SDKs\TypeScript\1.0\;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Users\build\AppData\Local\Microsoft\WindowsApps;
    PERL_BADLANG (unset)
    SHELL (unset)

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 6.4k
On Thu, 15 Feb 2018 05:11:22 -0800, jacques.germishuys@gmail.com wrote: Show quoted text
> This is a bug report for perl from jacques.germishuys@gmail.com, > generated with the help of perlbug 1.40 running under perl 5.26.0. > > > ----------------------------------------------------------------- > [Please describe your issue here] > > The following program shows that freeze() and thaw() is currently > broken with VS 2017: > > > use Data::Dumper; > use Storable qw/dclone/; > use Devel::Peek; > > my $x = > { > value => 1 > }; > > print Dumper ($x); > Dump ($x); > > my $y = dclone ($x); > > print Dumper ($y); > Dump ($y); > > > Output: > > $VAR1 = { > 'value' => 1 > }; > SV = IV(0x166b3b4) at 0x166b3b8 > REFCNT = 1 > FLAGS = (ROK) > RV = 0x164d6f8 > SV = PVHV(0x1653810) at 0x164d6f8 > REFCNT = 1 > FLAGS = (OOK,SHAREKEYS) > AUX_FLAGS = 0 > ARRAY = 0x172e390 (0:7, 1:1) > hash quality = 100.0% > KEYS = 1 > FILL = 1 > MAX = 7 > RITER = -1 > EITER = 0x0 > RAND = 0x7ebeb2c7 > Elt "value" HASH = 0x9dce435e > SV = IV(0x164d7d4) at 0x164d7d8 > REFCNT = 1 > FLAGS = (IOK,pIOK) > IV = 1 > $VAR1 = { > 'value' => -255 > }; > SV = IV(0x1661bfc) at 0x1661c00 > REFCNT = 1 > FLAGS = (ROK) > RV = 0x164d808 > SV = PVHV(0x1653dd0) at 0x164d808 > REFCNT = 1 > FLAGS = (OOK,SHAREKEYS) > AUX_FLAGS = 0 > ARRAY = 0x172e250 (0:7, 1:1) > hash quality = 100.0% > KEYS = 1 > FILL = 1 > MAX = 7 > RITER = -1 > EITER = 0x0 > RAND = 0x5820a183 > Elt "value" HASH = 0x9dce435e > SV = IV(0x166114c) at 0x1661150 > REFCNT = 1 > FLAGS = (IOK,pIOK) > IV = -255 > > > The value of 1 got changed to -255. I've also seen 0 change to -256! > > > [Please do not change anything below this line] > ----------------------------------------------------------------- > --- > Flags: > category=library > severity=critical > module=Storable > --- > Site configuration information for perl 5.26.0: > > Configured by build at Fri Feb 2 13:01:36 2018. > > Summary of my perl5 (revision 5 version 26 subversion 0) > configuration: > > Platform: > osname=MSWin32 > osvers=10.0 > archname=MSWin32 > uname='' > config_args='undef' > hint=recommended > useposix=true > d_sigaction=undef > useithreads=define > usemultiplicity=define > use64bitint=undef > use64bitall=undef > uselongdouble=undef > usemymalloc=n > default_inc_excludes_dot=define > bincompat5005=undef > Compiler: > cc='cl' > ccflags ='-nologo -GF -W3 -O2 -MD -Zi -DNDEBUG -GL -FS -DWIN32 > -D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE > -D_CRT_NONSTDC_NO_DEPRECATE -D_WINSOCK_DEPRECATED_NO_WARNINGS > -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS > -DUSE_PERLIO -D_USE_32BIT_TIME_T' > optimize='-O2 -MD -Zi -DNDEBUG -GL -FS' > cppflags='-DWIN32' > ccversion='19.12.25835' > gccversion='' > gccosandvers='' > intsize=4 > longsize=4 > ptrsize=4 > doublesize=8 > byteorder=1234 > doublekind=3 > d_longlong=undef > longlongsize=8 > d_longdbl=define > longdblsize=8 > longdblkind=0 > ivtype='long' > ivsize=4 > nvtype='double' > nvsize=8 > Off_t='__int64' > lseeksize=8 > alignbytes=8 > prototype=define > Linker and Libraries: > ld='link' > ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg > -libpath:"C:\opt\Striata\vs-2017-perl-5.26.0- > x86\\perl\lib\MSWin32\CORE" > -machine:x86 -subsystem:console,"5.01"' > libpth=\lib > libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib > comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib > netapi32.lib > uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib > odbccp32.lib > comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib > perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib > comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib > netapi32.lib > uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib > odbccp32.lib > comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib > libc=ucrt.lib > so=dll > useshrplib=true > libperl=perl526.lib > gnulibc_version='' > Dynamic Linking: > dlsrc=dl_win32.xs > dlext=dll > d_dlsymun=undef > ccdlflags=' ' > cccdlflags=' ' > lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg > -libpath:"C:\opt\Striata\vs-2017-perl-5.26.0- > x86\\perl\lib\MSWin32\CORE" > -machine:x86 -subsystem:console,"5.01"' > > > --- > @INC for perl 5.26.0: > C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/vendor/lib/MSWin32 > C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/vendor/lib > C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/lib/MSWin32 > C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/lib > C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/site/lib/MSWin32 > C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/site/lib > > --- > Environment for perl 5.26.0: > HOME (unset) > LANG (unset) > LANGUAGE (unset) > LD_LIBRARY_PATH (unset) > LOGDIR (unset) > > PATH=C:\VS2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x86;C:\VS2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x64;C:\VS2017\Common7\IDE\VC\VCPackages;C:\VS2017\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\VS2017\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team > Explorer;C:\VS2017\MSBuild\15.0\bin\Roslyn;C:\Program Files > (x86)\Microsoft > SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\x64\;C:\Program Files > (x86)\HTML > Help Workshop;C:\Program Files (x86)\Windows > Kits\10\bin\x64;C:\Program > Files (x86)\Windows > Kits\10\bin\10.0.16299.0\x64;C:\VS2017\\MSBuild\15.0\bin;C:\Windows\Microsoft.NET\Framework64\v4.0.30319;C:\VS2017\Common7\IDE\;C:\VS2017\Common7\Tools\;C:\dev\vs2017- > 32\bin;C:\opt\Striata\vs-2017-perl-5.26.0- > x86\perl\bin;C:\opt\Striata\bin\64;C:\opt\Striata\bin;C:\opt\Striata\git\cmd;C:\opt\Striata\cmake\bin;C:\opt\Striata\wix;C:\dev\vs2017- > 32\lib;C:\Python27\;C:\Python27\Scripts;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\jdk1.7.0_80\bin;C:\Program > Files (x86)\WinSCP\;C:\Program Files\Microsoft SQL > Server\110\Tools\Binn\;C:\Program Files (x86)\Microsoft > SDKs\TypeScript\1.0\;C:\Program Files\Microsoft SQL > Server\120\Tools\Binn\;C:\Users\build\AppData\Local\Microsoft\WindowsApps; > PERL_BADLANG (unset) > SHELL (unset)
I've confirmed that the 64-bit version of the compiler has exactly the same problem. The 15.4 series of Visual Studio 2017 does not exhibit this behaviour, so far I've only seen it with the 15.5 series.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.5k
On Thu, 15 Feb 2018 05:11:22 -0800, jacques.germishuys@gmail.com wrote: Show quoted text
> $VAR1 = { > 'value' => 1 > }; > SV = IV(0x166b3b4) at 0x166b3b8 > REFCNT = 1 > FLAGS = (ROK) > RV = 0x164d6f8 > SV = PVHV(0x1653810) at 0x164d6f8 > REFCNT = 1 > FLAGS = (OOK,SHAREKEYS) > AUX_FLAGS = 0 > ARRAY = 0x172e390 (0:7, 1:1) > hash quality = 100.0% > KEYS = 1 > FILL = 1 > MAX = 7 > RITER = -1 > EITER = 0x0 > RAND = 0x7ebeb2c7 > Elt "value" HASH = 0x9dce435e > SV = IV(0x164d7d4) at 0x164d7d8 > REFCNT = 1 > FLAGS = (IOK,pIOK) > IV = 1 > $VAR1 = { > 'value' => -255 > }; > SV = IV(0x1661bfc) at 0x1661c00 > REFCNT = 1 > FLAGS = (ROK) > RV = 0x164d808 > SV = PVHV(0x1653dd0) at 0x164d808 > REFCNT = 1 > FLAGS = (OOK,SHAREKEYS) > AUX_FLAGS = 0 > ARRAY = 0x172e250 (0:7, 1:1) > hash quality = 100.0% > KEYS = 1 > FILL = 1 > MAX = 7 > RITER = -1 > EITER = 0x0 > RAND = 0x5820a183 > Elt "value" HASH = 0x9dce435e > SV = IV(0x166114c) at 0x1661150 > REFCNT = 1 > FLAGS = (IOK,pIOK) > IV = -255 > > > The value of 1 got changed to -255. I've also seen 0 change to -256!
Reproduced in blead: J:\dev\perl\git\perl>.\perl -Ilib -MStorable=dclone -MData::Dumper -le "$x = { a => 1 }; print Dumper(dclone($x))" $VAR1 = { 'a' => -255 }; J:\dev\perl\git\perl>cl Microsoft (R) C/C++ Optimizing Compiler Version 19.12.25835 for x86 Copyright (C) Microsoft Corporation. All rights reserved. Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 347b
On Sun, 18 Feb 2018 18:32:32 -0800, tonyc wrote: Show quoted text
> > The value of 1 got changed to -255. I've also seen 0 change to -256!
I suspect the compiler is being confused by the AIX compiler workaround. The attached patch prevented the issue for me. Considering tmp is a signed char (limited to -128 through 127), this can only be a compiler bug. Tony
Subject: 0001-perl-132870-workaround-possibe-VC2017-compiler-bug.patch
From 670e7f513fc6b52eddde46ff414d339dc021c02f Mon Sep 17 00:00:00 2001 From: Tony Cook <tony@develop-help.com> Date: Tue, 20 Feb 2018 16:18:05 +1100 Subject: (perl #132870) workaround possibe VC2017 compiler bug For non-debug builds MSVC could read small integers like 1 as -255. It's possible it was confused by the AIX compiler bug workaround. --- dist/Storable/Storable.xs | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/dist/Storable/Storable.xs b/dist/Storable/Storable.xs index df0c433b0e..45af7be210 100644 --- a/dist/Storable/Storable.xs +++ b/dist/Storable/Storable.xs @@ -6106,14 +6106,20 @@ static SV *retrieve_byte(pTHX_ stcxt_t *cxt, const char *cname) SV *sv; HV *stash; int siv; +#ifndef _MSC_VER signed char tmp; /* Workaround for AIX cc bug --H.Merijn Brand */ +#endif TRACEME(("retrieve_byte (#%d)", (int)cxt->tagnum)); GETMARK(siv); TRACEME(("small integer read as %d", (unsigned char) siv)); +#ifdef _MSC_VER + sv = newSViv(siv - 128); +#else tmp = (unsigned char) siv - 128; sv = newSViv(tmp); +#endif stash = cname ? gv_stashpv(cname, GV_ADD) : 0; SEEN_NN(sv, stash, 0); /* Associate this new scalar with tag "tagnum" */ -- 2.14.1.windows.1
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 530b
On Mon, 19 Feb 2018 21:22:55 -0800, tonyc wrote: Show quoted text
> On Sun, 18 Feb 2018 18:32:32 -0800, tonyc wrote:
> > > The value of 1 got changed to -255. I've also seen 0 change to > > > -256!
> > I suspect the compiler is being confused by the AIX compiler > workaround. > > The attached patch prevented the issue for me. > > Considering tmp is a signed char (limited to -128 through 127), this > can only be a compiler bug. > > Tony
Applied as 32ce30d709666239149a4f04ddcfbdec00005288 with a minor change to the commit message. Tony
Download (untitled) / with headers
text/plain 317b
Thank you for filing this report. You have helped make Perl better. With the release yesterday of Perl 5.28.0, this and 185 other issues have been resolved. Perl 5.28.0 may be downloaded via: https://metacpan.org/release/XSAWYERX/perl-5.28.0 If you find that the problem persists, feel free to reopen this ticket.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org