Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storable dclone() broken (latest VS 2017 32-bit) #16421

Closed
p5pRT opened this issue Feb 15, 2018 · 10 comments
Closed

Storable dclone() broken (latest VS 2017 32-bit) #16421

p5pRT opened this issue Feb 15, 2018 · 10 comments

Comments

@p5pRT
Copy link

p5pRT commented Feb 15, 2018

Migrated from rt.perl.org#132870 (status was 'resolved')

Searchable as RT132870$

@p5pRT
Copy link
Author

p5pRT commented Feb 15, 2018

From jacques.germishuys@gmail.com

Created by jacques.germishuys@gmail.com

The following program shows that freeze() and thaw() is currently
broken with VS 2017​:

use Data​::Dumper;
use Storable qw/dclone/;
use Devel​::Peek;

my $x =
{
value => 1
};

print Dumper ($x);
Dump ($x);

my $y = dclone ($x);

print Dumper ($y);
Dump ($y);

Output​:

$VAR1 = {
  'value' => 1
};
SV = IV(0x166b3b4) at 0x166b3b8
  REFCNT = 1
  FLAGS = (ROK)
  RV = 0x164d6f8
  SV = PVHV(0x1653810) at 0x164d6f8
  REFCNT = 1
  FLAGS = (OOK,SHAREKEYS)
  AUX_FLAGS = 0
  ARRAY = 0x172e390 (0​:7, 1​:1)
  hash quality = 100.0%
  KEYS = 1
  FILL = 1
  MAX = 7
  RITER = -1
  EITER = 0x0
  RAND = 0x7ebeb2c7
  Elt "value" HASH = 0x9dce435e
  SV = IV(0x164d7d4) at 0x164d7d8
  REFCNT = 1
  FLAGS = (IOK,pIOK)
  IV = 1
$VAR1 = {
  'value' => -255
};
SV = IV(0x1661bfc) at 0x1661c00
  REFCNT = 1
  FLAGS = (ROK)
  RV = 0x164d808
  SV = PVHV(0x1653dd0) at 0x164d808
  REFCNT = 1
  FLAGS = (OOK,SHAREKEYS)
  AUX_FLAGS = 0
  ARRAY = 0x172e250 (0​:7, 1​:1)
  hash quality = 100.0%
  KEYS = 1
  FILL = 1
  MAX = 7
  RITER = -1
  EITER = 0x0
  RAND = 0x5820a183
  Elt "value" HASH = 0x9dce435e
  SV = IV(0x166114c) at 0x1661150
  REFCNT = 1
  FLAGS = (IOK,pIOK)
  IV = -255

The value of 1 got changed to -255. I've also seen 0 change to -256!

Perl Info

Flags:
    category=library
    severity=critical
    module=Storable

Site configuration information for perl 5.26.0:

Configured by build at Fri Feb 2 13:01:36 2018.

Summary of my perl5 (revision 5 version 26 subversion 0) configuration:

  Platform:
    osname=MSWin32
    osvers=10.0
    archname=MSWin32
    uname=''
    config_args='undef'
    hint=recommended
    useposix=true
    d_sigaction=undef
    useithreads=define
    usemultiplicity=define
    use64bitint=undef
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cl'
    ccflags ='-nologo -GF -W3 -O2 -MD -Zi -DNDEBUG -GL -FS -DWIN32
-D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE
-D_CRT_NONSTDC_NO_DEPRECATE -D_WINSOCK_DEPRECATED_NO_WARNINGS
-DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS
-DUSE_PERLIO -D_USE_32BIT_TIME_T'
    optimize='-O2 -MD -Zi -DNDEBUG -GL -FS'
    cppflags='-DWIN32'
    ccversion='19.12.25835'
    gccversion=''
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=4
    doublesize=8
    byteorder=1234
    doublekind=3
    d_longlong=undef
    longlongsize=8
    d_longdbl=define
    longdblsize=8
    longdblkind=0
    ivtype='long'
    ivsize=4
    nvtype='double'
    nvsize=8
    Off_t='__int64'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='link'
    ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg
-libpath:"C:\opt\Striata\vs-2017-perl-5.26.0-x86\\perl\lib\MSWin32\CORE"
-machine:x86 -subsystem:console,"5.01"'
    libpth=\lib
    libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib
uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib
comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
    perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib
uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib
comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
    libc=ucrt.lib
    so=dll
    useshrplib=true
    libperl=perl526.lib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs
    dlext=dll
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg
-libpath:"C:\opt\Striata\vs-2017-perl-5.26.0-x86\\perl\lib\MSWin32\CORE"
-machine:x86 -subsystem:console,"5.01"'



@INC for perl 5.26.0:
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/vendor/lib/MSWin32
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/vendor/lib
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/lib/MSWin32
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/lib
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/site/lib/MSWin32
    C:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/site/lib


Environment for perl 5.26.0:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=C:\VS2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x86;C:\VS2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x64;C:\VS2017\Common7\IDE\VC\VCPackages;C:\VS2017\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\VS2017\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team
Explorer;C:\VS2017\MSBuild\15.0\bin\Roslyn;C:\Program Files (x86)\Microsoft
SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\x64\;C:\Program Files (x86)\HTML
Help Workshop;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program
Files (x86)\Windows
Kits\10\bin\10.0.16299.0\x64;C:\VS2017\\MSBuild\15.0\bin;C:\Windows\Microsoft.NET\Framework64\v4.0.30319;C:\VS2017\Common7\IDE\;C:\VS2017\Common7\Tools\;C:\dev\vs2017-32\bin;C:\opt\Striata\vs-2017-perl-5.26.0-x86\perl\bin;C:\opt\Striata\bin\64;C:\opt\Striata\bin;C:\opt\Striata\git\cmd;C:\opt\Striata\cmake\bin;C:\opt\Striata\wix;C:\dev\vs2017-32\lib;C:\Python27\;C:\Python27\Scripts;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\jdk1.7.0_80\bin;C:\Program
Files (x86)\WinSCP\;C:\Program Files\Microsoft SQL
Server\110\Tools\Binn\;C:\Program Files (x86)\Microsoft
SDKs\TypeScript\1.0\;C:\Program Files\Microsoft SQL
Server\120\Tools\Binn\;C:\Users\build\AppData\Local\Microsoft\WindowsApps;
    PERL_BADLANG (unset)
    SHELL (unset)

@p5pRT
Copy link
Author

p5pRT commented Feb 16, 2018

From jacques.germishuys@gmail.com

On Thu, 15 Feb 2018 05​:11​:22 -0800, jacques.germishuys@​gmail.com wrote​:

This is a bug report for perl from jacques.germishuys@​gmail.com,
generated with the help of perlbug 1.40 running under perl 5.26.0.

-----------------------------------------------------------------
[Please describe your issue here]

The following program shows that freeze() and thaw() is currently
broken with VS 2017​:

use Data​::Dumper;
use Storable qw/dclone/;
use Devel​::Peek;

my $x =
{
value => 1
};

print Dumper ($x);
Dump ($x);

my $y = dclone ($x);

print Dumper ($y);
Dump ($y);

Output​:

$VAR1 = {
'value' => 1
};
SV = IV(0x166b3b4) at 0x166b3b8
REFCNT = 1
FLAGS = (ROK)
RV = 0x164d6f8
SV = PVHV(0x1653810) at 0x164d6f8
REFCNT = 1
FLAGS = (OOK,SHAREKEYS)
AUX_FLAGS = 0
ARRAY = 0x172e390 (0​:7, 1​:1)
hash quality = 100.0%
KEYS = 1
FILL = 1
MAX = 7
RITER = -1
EITER = 0x0
RAND = 0x7ebeb2c7
Elt "value" HASH = 0x9dce435e
SV = IV(0x164d7d4) at 0x164d7d8
REFCNT = 1
FLAGS = (IOK,pIOK)
IV = 1
$VAR1 = {
'value' => -255
};
SV = IV(0x1661bfc) at 0x1661c00
REFCNT = 1
FLAGS = (ROK)
RV = 0x164d808
SV = PVHV(0x1653dd0) at 0x164d808
REFCNT = 1
FLAGS = (OOK,SHAREKEYS)
AUX_FLAGS = 0
ARRAY = 0x172e250 (0​:7, 1​:1)
hash quality = 100.0%
KEYS = 1
FILL = 1
MAX = 7
RITER = -1
EITER = 0x0
RAND = 0x5820a183
Elt "value" HASH = 0x9dce435e
SV = IV(0x166114c) at 0x1661150
REFCNT = 1
FLAGS = (IOK,pIOK)
IV = -255

The value of 1 got changed to -255. I've also seen 0 change to -256!

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=library
severity=critical
module=Storable
---
Site configuration information for perl 5.26.0​:

Configured by build at Fri Feb 2 13​:01​:36 2018.

Summary of my perl5 (revision 5 version 26 subversion 0)
configuration​:

Platform​:
osname=MSWin32
osvers=10.0
archname=MSWin32
uname=''
config_args='undef'
hint=recommended
useposix=true
d_sigaction=undef
useithreads=define
usemultiplicity=define
use64bitint=undef
use64bitall=undef
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler​:
cc='cl'
ccflags ='-nologo -GF -W3 -O2 -MD -Zi -DNDEBUG -GL -FS -DWIN32
-D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE
-D_CRT_NONSTDC_NO_DEPRECATE -D_WINSOCK_DEPRECATED_NO_WARNINGS
-DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS
-DUSE_PERLIO -D_USE_32BIT_TIME_T'
optimize='-O2 -MD -Zi -DNDEBUG -GL -FS'
cppflags='-DWIN32'
ccversion='19.12.25835'
gccversion=''
gccosandvers=''
intsize=4
longsize=4
ptrsize=4
doublesize=8
byteorder=1234
doublekind=3
d_longlong=undef
longlongsize=8
d_longdbl=define
longdblsize=8
longdblkind=0
ivtype='long'
ivsize=4
nvtype='double'
nvsize=8
Off_t='__int64'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries​:
ld='link'
ldflags ='-nologo -nodefaultlib -debug -opt​:ref,icf -ltcg
-libpath​:"C​:\opt\Striata\vs-2017-perl-5.26.0-
x86\\perl\lib\MSWin32\CORE"
-machine​:x86 -subsystem​:console,"5.01"'
libpth=\lib
libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib
uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib
odbccp32.lib
comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib
uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib
odbccp32.lib
comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
libc=ucrt.lib
so=dll
useshrplib=true
libperl=perl526.lib
gnulibc_version=''
Dynamic Linking​:
dlsrc=dl_win32.xs
dlext=dll
d_dlsymun=undef
ccdlflags=' '
cccdlflags=' '
lddlflags='-dll -nologo -nodefaultlib -debug -opt​:ref,icf -ltcg
-libpath​:"C​:\opt\Striata\vs-2017-perl-5.26.0-
x86\\perl\lib\MSWin32\CORE"
-machine​:x86 -subsystem​:console,"5.01"'

---
@​INC for perl 5.26.0​:
C​:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/vendor/lib/MSWin32
C​:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/vendor/lib
C​:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/lib/MSWin32
C​:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/lib
C​:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/site/lib/MSWin32
C​:/opt/Striata/vs-2017-perl-5.26.0-x86/perl/site/lib

---
Environment for perl 5.26.0​:
HOME (unset)
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)

PATH=C​:\VS2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x86;C​:\VS2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x64;C​:\VS2017\Common7\IDE\VC\VCPackages;C​:\VS2017\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C​:\VS2017\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team
Explorer;C​:\VS2017\MSBuild\15.0\bin\Roslyn;C​:\Program Files
(x86)\Microsoft
SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\x64\;C​:\Program Files
(x86)\HTML
Help Workshop;C​:\Program Files (x86)\Windows
Kits\10\bin\x64;C​:\Program
Files (x86)\Windows
Kits\10\bin\10.0.16299.0\x64;C​:\VS2017\\MSBuild\15.0\bin;C​:\Windows\Microsoft.NET\Framework64\v4.0.30319;C​:\VS2017\Common7\IDE\;C​:\VS2017\Common7\Tools\;C​:\dev\vs2017-
32\bin;C​:\opt\Striata\vs-2017-perl-5.26.0-
x86\perl\bin;C​:\opt\Striata\bin\64;C​:\opt\Striata\bin;C​:\opt\Striata\git\cmd;C​:\opt\Striata\cmake\bin;C​:\opt\Striata\wix;C​:\dev\vs2017-
32\lib;C​:\Python27\;C​:\Python27\Scripts;C​:\Windows\system32;C​:\Windows;C​:\Windows\System32\Wbem;C​:\Windows\System32\WindowsPowerShell\v1.0\;C​:\jdk1.7.0_80\bin;C​:\Program
Files (x86)\WinSCP\;C​:\Program Files\Microsoft SQL
Server\110\Tools\Binn\;C​:\Program Files (x86)\Microsoft
SDKs\TypeScript\1.0\;C​:\Program Files\Microsoft SQL
Server\120\Tools\Binn\;C​:\Users\build\AppData\Local\Microsoft\WindowsApps;
PERL_BADLANG (unset)
SHELL (unset)

I've confirmed that the 64-bit version of the compiler has exactly the same problem. The 15.4 series of Visual Studio 2017 does not exhibit this behaviour, so far I've only seen it with the 15.5 series.

@p5pRT
Copy link
Author

p5pRT commented Feb 19, 2018

From @tonycoz

On Thu, 15 Feb 2018 05​:11​:22 -0800, jacques.germishuys@​gmail.com wrote​:

$VAR1 = {
'value' => 1
};
SV = IV(0x166b3b4) at 0x166b3b8
REFCNT = 1
FLAGS = (ROK)
RV = 0x164d6f8
SV = PVHV(0x1653810) at 0x164d6f8
REFCNT = 1
FLAGS = (OOK,SHAREKEYS)
AUX_FLAGS = 0
ARRAY = 0x172e390 (0​:7, 1​:1)
hash quality = 100.0%
KEYS = 1
FILL = 1
MAX = 7
RITER = -1
EITER = 0x0
RAND = 0x7ebeb2c7
Elt "value" HASH = 0x9dce435e
SV = IV(0x164d7d4) at 0x164d7d8
REFCNT = 1
FLAGS = (IOK,pIOK)
IV = 1
$VAR1 = {
'value' => -255
};
SV = IV(0x1661bfc) at 0x1661c00
REFCNT = 1
FLAGS = (ROK)
RV = 0x164d808
SV = PVHV(0x1653dd0) at 0x164d808
REFCNT = 1
FLAGS = (OOK,SHAREKEYS)
AUX_FLAGS = 0
ARRAY = 0x172e250 (0​:7, 1​:1)
hash quality = 100.0%
KEYS = 1
FILL = 1
MAX = 7
RITER = -1
EITER = 0x0
RAND = 0x5820a183
Elt "value" HASH = 0x9dce435e
SV = IV(0x166114c) at 0x1661150
REFCNT = 1
FLAGS = (IOK,pIOK)
IV = -255

The value of 1 got changed to -255. I've also seen 0 change to -256!

Reproduced in blead​:

J​:\dev\perl\git\perl>.\perl -Ilib -MStorable=dclone -MData​::Dumper -le "$x = { a => 1 }; print Dumper(dclone($x))"
$VAR1 = {
  'a' => -255
  };

J​:\dev\perl\git\perl>cl
Microsoft (R) C/C++ Optimizing Compiler Version 19.12.25835 for x86
Copyright (C) Microsoft Corporation. All rights reserved.

Tony

@p5pRT
Copy link
Author

p5pRT commented Feb 19, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Feb 20, 2018

From @tonycoz

On Sun, 18 Feb 2018 18​:32​:32 -0800, tonyc wrote​:

The value of 1 got changed to -255. I've also seen 0 change to -256!

I suspect the compiler is being confused by the AIX compiler workaround.

The attached patch prevented the issue for me.

Considering tmp is a signed char (limited to -128 through 127), this can only be a compiler bug.

Tony

@p5pRT
Copy link
Author

p5pRT commented Feb 20, 2018

From @tonycoz

0001-perl-132870-workaround-possibe-VC2017-compiler-bug.patch
From 670e7f513fc6b52eddde46ff414d339dc021c02f Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Tue, 20 Feb 2018 16:18:05 +1100
Subject: (perl #132870) workaround possibe VC2017 compiler bug

For non-debug builds MSVC could read small integers like 1 as -255.

It's possible it was confused by the AIX compiler bug workaround.
---
 dist/Storable/Storable.xs | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/dist/Storable/Storable.xs b/dist/Storable/Storable.xs
index df0c433b0e..45af7be210 100644
--- a/dist/Storable/Storable.xs
+++ b/dist/Storable/Storable.xs
@@ -6106,14 +6106,20 @@ static SV *retrieve_byte(pTHX_ stcxt_t *cxt, const char *cname)
     SV *sv;
     HV *stash;
     int siv;
+#ifndef _MSC_VER
     signed char tmp;	/* Workaround for AIX cc bug --H.Merijn Brand */
+#endif
 
     TRACEME(("retrieve_byte (#%d)", (int)cxt->tagnum));
 
     GETMARK(siv);
     TRACEME(("small integer read as %d", (unsigned char) siv));
+#ifdef _MSC_VER
+    sv = newSViv(siv - 128);
+#else
     tmp = (unsigned char) siv - 128;
     sv = newSViv(tmp);
+#endif
     stash = cname ? gv_stashpv(cname, GV_ADD) : 0;
     SEEN_NN(sv, stash, 0);	/* Associate this new scalar with tag "tagnum" */
 
-- 
2.14.1.windows.1

@p5pRT
Copy link
Author

p5pRT commented Mar 7, 2018

From @tonycoz

On Mon, 19 Feb 2018 21​:22​:55 -0800, tonyc wrote​:

On Sun, 18 Feb 2018 18​:32​:32 -0800, tonyc wrote​:

The value of 1 got changed to -255. I've also seen 0 change to
-256!

I suspect the compiler is being confused by the AIX compiler
workaround.

The attached patch prevented the issue for me.

Considering tmp is a signed char (limited to -128 through 127), this
can only be a compiler bug.

Tony

Applied as 32ce30d with a minor change to the commit message.

Tony

@p5pRT
Copy link
Author

p5pRT commented Mar 7, 2018

@tonycoz - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented Jun 23, 2018

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release yesterday of Perl 5.28.0, this and 185 other issues have been
resolved.

Perl 5.28.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.28.0

If you find that the problem persists, feel free to reopen this ticket.

@p5pRT p5pRT closed this as completed Jun 23, 2018
@p5pRT
Copy link
Author

p5pRT commented Jun 23, 2018

@khwilliamson - Status changed from 'pending release' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant