Skip Menu |
Report information
Id: 125714
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: Steve.Hay [at] verosoftware.com
Cc: bulk88 <bulk88 [at] hotmail.com>
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: (no value)

Attachments
0001-Add-experimental-support-for-VS2015-VC-14.0.patch
0001-Add-VS2015-support-v2-with-typos-corrected-in-win32..patch
0001-Add-VS2015-support-v2.patch
0001-Add-VS2015-support-v3.patch
0001-Add-VS2015-support-v4.patch
0001-Add-VS2015-support.patch
0001-export-__pioinfo-by-force.patch
0001-more-VC-2015-fixes.patch
build.bat
clean.bat
close 18 MB per sec mem leak, no handle leak, no dbg invalid hnd excepts.txt
close.cpp
closeclosesocket 18 MB per sec mem leak, no handle leak, has invalid handle exceptions but they are trapped by winsock.txt
closeclosesocket.cpp
closesocketclose no mem leak, no handle leak, has dbg invalid hnd excepts.txt
closesocketclose.cpp
perl-5.24.1-vc14.patch
static-crt.patch
VC++ 2015 support for Perl.txt



CC: bulk88 [...] hotmail.com
Subject: Adding support for VS2015 (VC14)
Download (untitled) / with headers
text/plain 1.4k
This ticket is for centralizing discussions about how to add support for building with VS2015. There have been a couple of p5p threads already; it is better to keep it all in one place: http://www.nntp.perl.org/group/perl.perl5.porters/2015/05/msg227702.html http://www.nntp.perl.org/group/perl.perl5.porters/2015/07/msg229524.html Attached is my first attempt, originally posted in the latter thread above (saved here since http://www.nntp.perl.org doesn't seem to save attachments). Also attached is my second attempt: This does surprisingly well with the quick-and-dirty approach of simply casting a FILE* to the appropriate new CRT struct (__crt_stdio_stream_data) and accessing the members of that. It might even suffice, rather than going to the hassle of writing accessors and probably needing to separate getters from setters? However, the build currently fails to link due to __pioinfo apparently no longer being present in the new CRT (or else I'm linking the wrong libs?): win32.obj : error LNK2001: unresolved external symbol __imp____pioinfo This was added (actually restored and modified from earlier ripped out code) by: http://perl5.git.perl.org/perl.git/commit/b47a847f6284f6f98ad7509cf77a4aeb802d8fce As that commit notes, "If __pioinfo isn't exported anymore, the Perl build will break." So this was always known to be an area of danger and impending breakage; the question is: What do we do about it now that it's happened? bulk88: Do you have any ideas on the latter point?
Subject: 0001-Add-VS2015-support-v2.patch
From 062c21f5e0cbdfa1baf3cd7c7910b65e706fef36 Mon Sep 17 00:00:00 2001 From: Steve Hay <steve.m.hay@googlemail.com> Date: Tue, 28 Jul 2015 09:25:48 +0100 Subject: [PATCH] Add VS2015 support, v2 Currently not working due to at least: win32.c(3956): warning C4013: 'gets' undefined; assuming extern returning int win32.obj : error LNK2001: unresolved external symbol __imp____pioinfo --- perlio.c | 2 +- win32/Makefile | 24 ++++++++++++++++++++++++ win32/config_sh.PL | 14 ++++++++++++++ win32/makefile.mk | 24 ++++++++++++++++++++++++ win32/win32.c | 6 +++--- win32/win32.h | 38 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 104 insertions(+), 4 deletions(-) diff --git a/perlio.c b/perlio.c index ae8cbc9..16ffa47 100644 --- a/perlio.c +++ b/perlio.c @@ -3180,7 +3180,7 @@ PerlIOStdio_invalidate_fileno(pTHX_ FILE *f) structure at all */ # else - f->_file = -1; + PERLIO_FILE_file(f) = -1; # endif return 1; # else diff --git a/win32/Makefile b/win32/Makefile index a2fb2aa..27c119f 100644 --- a/win32/Makefile +++ b/win32/Makefile @@ -126,6 +126,10 @@ CCTYPE = MSVC60 #CCTYPE = MSVC120 # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) #CCTYPE = MSVC120FREE +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) +#CCTYPE = MSVC140 +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) +#CCTYPE = MSVC140FREE # # If you are using Intel C++ Compiler uncomment this @@ -445,6 +449,10 @@ CXX_FLAG = -TP -EHsc LIBC = msvcrt.lib +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +LIBC = $(LIBC) vcruntime.lib ucrt.lib +!ENDIF + !IF "$(CFG)" == "Debug" OPTIMIZE = -Od -MD -Zi -DDEBUGGING LINK_DBG = -debug @@ -489,6 +497,11 @@ OPTIMIZE = $(OPTIMIZE) -fp:precise DEFINES = $(DEFINES) -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE !ENDIF +# Likewise for deprecated Winsock APIs in VC++ 14.0 for now. +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +DEFINES = $(DEFINES) -D_WINSOCK_DEPRECATED_NO_WARNINGS +!ENDIF + # In VS 2005 (VC++ 8.0) Microsoft changes time_t from 32-bit to # 64-bit, even in 32-bit mode. It also provides the _USE_32BIT_TIME_T # preprocessor option to revert back to the old functionality for @@ -910,6 +923,17 @@ config.w32 : $(CFGSH_TMPL) @echo.>>$@ @echo #ifndef _config_h_footer_>>$@ @echo #define _config_h_footer_>>$@ +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" + @echo #undef FILE_ptr>>$@ + @echo #define FILE_ptr(fp) PERLIO_FILE_ptr(fp)>>$@ + @echo #undef FILE_cnt>>$@ + @echo #define FILE_cnt(fp) PERLIO_FILE_cnt(fp)>>$@ + @echo #undef FILE_base>>$@ + @echo #define FILE_base(fp) PERLIO_FILE_base(fp)>>$@ + @echo #undef FILE_bufsiz>>$@ + @echo #define FILE_bufsiz(fp) (PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))>>$@ + @echo #define I_STDBOOL>>$@ +!ENDIF @echo #undef Off_t>>$@ @echo #undef LSEEKSIZE>>$@ @echo #undef Off_t_size>>$@ diff --git a/win32/config_sh.PL b/win32/config_sh.PL index 98255a8..69599b1 100644 --- a/win32/config_sh.PL +++ b/win32/config_sh.PL @@ -270,6 +270,13 @@ if ($opt{cc} =~ /\bcl/ and $opt{ccversion} =~ /^(\d+)/) { if($ccversion < 13) { #VC6 $opt{ar} ='lib'; } + if ($ccversion >= 19) { # VC14 + $opt{stdio_base} = 'PERLIO_FILE_base(fp)'; + $opt{stdio_bufsiz} = '(PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))'; + $opt{stdio_cnt} = 'PERLIO_FILE_cnt(fp)'; + $opt{stdio_ptr} = 'PERLIO_FILE_ptr(fp)'; + $opt{i_stdbool} = 'define'; + } } #find out which MSVC this ICC is using elsif ($opt{cc} =~ /\bicl/) { @@ -279,6 +286,13 @@ elsif ($opt{cc} =~ /\bicl/) { $opt{sGMTIME_max} = 32535291599; $opt{sLOCALTIME_max} = 32535244799; } + if ($num_ver =~ /^(\d+)/ && $1 >= 19) { # VC14 + $opt{stdio_base} = 'PERLIO_FILE_base(fp)'; + $opt{stdio_bufsiz} = '(PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))'; + $opt{stdio_cnt} = 'PERLIO_FILE_cnt(fp)'; + $opt{stdio_ptr} = 'PERLIO_FILE_ptr(fp)'; + $opt{i_stdbool} = 'define'; + } $opt{ar} ='xilib'; } diff --git a/win32/makefile.mk b/win32/makefile.mk index c74b5bb..3a708ac 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -138,6 +138,10 @@ USE_LARGE_FILES *= define #CCTYPE = MSVC120 # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) #CCTYPE = MSVC120FREE +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) +#CCTYPE = MSVC140 +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) +#CCTYPE = MSVC140FREE # MinGW or mingw-w64 with gcc-3.4.5 or later CCTYPE *= GCC @@ -563,6 +567,10 @@ CXX_FLAG = -TP -EHsc LIBC = msvcrt.lib +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +LIBC = $(LIBC) vcruntime.lib ucrt.lib +!ENDIF + .IF "$(CFG)" == "Debug" OPTIMIZE = -Od -MD -Zi -DDEBUGGING LINK_DBG = -debug @@ -603,6 +611,11 @@ OPTIMIZE += -fp:precise DEFINES += -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE .ENDIF +# Likewise for deprecated Winsock APIs in VC++ 14.0 for now. +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +DEFINES += -D_WINSOCK_DEPRECATED_NO_WARNINGS +.ENDIF + # In VS 2005 (VC++ 8.0) Microsoft changes time_t from 32-bit to # 64-bit, even in 32-bit mode. It also provides the _USE_32BIT_TIME_T # preprocessor option to revert back to the old functionality for @@ -1074,6 +1087,17 @@ config.w32 : $(CFGSH_TMPL) @echo.>>$@ @echo #ifndef _config_h_footer_>>$@ @echo #define _config_h_footer_>>$@ +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" + @echo #undef FILE_ptr>>$@ + @echo #define FILE_ptr(fp) PERLIO_FILE_ptr(fp)>>$@ + @echo #undef FILE_cnt>>$@ + @echo #define FILE_cnt(fp) PERLIO_FILE_cnt(fp)>>$@ + @echo #undef FILE_base>>$@ + @echo #define FILE_base(fp) PERLIO_FILE_base(fp)>>$@ + @echo #undef FILE_bufsiz>>$@ + @echo #define FILE_bufsiz(fp) (PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))>>$@ + @echo #define I_STDBOOL>>$@ +.ENDIF @echo #undef Off_t>>$@ @echo #undef LSEEKSIZE>>$@ @echo #undef Off_t_size>>$@ diff --git a/win32/win32.c b/win32/win32.c index 2b883a2..75a017a 100644 --- a/win32/win32.c +++ b/win32/win32.c @@ -4130,15 +4130,15 @@ win32_fdupopen(FILE *pf) int fileno = win32_dup(win32_fileno(pf)); /* open the file in the same mode */ - if((pf)->_flag & _IOREAD) { + if (PERLIO_FILE_flag(pf) & PERLIO_FILE_flag_RD) { mode[0] = 'r'; mode[1] = 0; } - else if((pf)->_flag & _IOWRT) { + else if (PERLIO_FILE_flag(pf) & PERLIO_FILE_flag_WR) { mode[0] = 'a'; mode[1] = 0; } - else if((pf)->_flag & _IORW) { + else if (PERLIO_FILE_flag(pf) & PERLIO_FILE_flag_RW) { mode[0] = 'r'; mode[1] = '+'; mode[2] = 0; diff --git a/win32/win32.h b/win32/win32.h index 3b35b6c..f57b432 100644 --- a/win32/win32.h +++ b/win32/win32.h @@ -282,6 +282,44 @@ extern const __declspec(selectany) union { unsigned __int64 __q; double __d; } __PL_nan_u = { 0x7FF8000000000000UI64 }; # define NV_NAN ((NV)__PL_nan_u.__d) +#if defined(_MSC_VER) && _MSC_VER >= 1900 +// From corecrt_internal_stdio.h: +typedef struct +{ + union + { + FILE _public_file; + char* _ptr; + }; + + char* _base; + int _cnt; + long _flags; + long _file; + int _charbuf; + int _bufsiz; + char* _tmpfname; + CRITICAL_SECTION _lock; +} __crt_stdio_stream_data; +#define PERLIO_FILE_flag_RD 0x0001 // _IOREAD +#define PERLIO_FILE_flag_WR 0x0002 // _IOWRITE +#define PERLIO_FILE_flag_RW 0x0004 // _IOUPDATE +#define PERLIO_FILE_ptr(f) (((__crt_stdio_stream_data*)(f))->_ptr) +#define PERLIO_FILE_base(f) (((__crt_stdio_stream_data*)(f))->_base) +#define PERLIO_FILE_cnt(Ff) (((__crt_stdio_stream_data*)(f))->_cnt) +#define PERLIO_FILE_flag(f) ((int)(((__crt_stdio_stream_data*)(f))->_flags)) +#define PERLIO_FILE_file(f) ((int)(((__crt_stdio_stream_data*)(f))->_file)) +#else +#define PERLIO_FILE_flag_RD _IOREAD // 0x001 +#define PERLIO_FILE_flag_WR _IOWRT // 0x002 +#define PERLIO_FILE_flag_RW _IORW // 0x080 +#define PERLIO_FILE_ptr(f) ((f)->_ptr) +#define PERLIO_FILE_base(f) ((f)->_base) +#define PERLIO_FILE_cnt(Ff) ((f)->_cnt) +#define PERLIO_FILE_flag(f) ((f)->_flag) +#define PERLIO_FILE_file(f) ((f)->_file) +#endif + #endif /* _MSC_VER */ #ifdef __MINGW32__ /* Minimal Gnu-Win32 */ -- 1.9.5.msysgit.1
Subject: 0001-Add-VS2015-support.patch
From cba3fa1d1c4d2b6ee33621c8c282e0a43804eb9c Mon Sep 17 00:00:00 2001 From: Steve Hay <steve.m.hay@googlemail.com> Date: Sat, 25 Jul 2015 13:46:05 +0100 Subject: [PATCH] Add VS2015 support Currently not working due to at least: - f->_file access in PerlIOStdio_invalidate_fileno (perlio.c) - (pf)->_flag access in win32_fdupopen (win32/win32.c) --- win32/Makefile | 22 ++++++++++++++++++++++ win32/config_sh.PL | 12 ++++++++++-- win32/makefile.mk | 16 ++++++++++++++++ 3 files changed, 48 insertions(+), 2 deletions(-) diff --git a/win32/Makefile b/win32/Makefile index a2fb2aa..da6163c 100644 --- a/win32/Makefile +++ b/win32/Makefile @@ -126,6 +126,10 @@ CCTYPE = MSVC60 #CCTYPE = MSVC120 # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) #CCTYPE = MSVC120FREE +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) +#CCTYPE = MSVC140 +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) +#CCTYPE = MSVC140FREE # # If you are using Intel C++ Compiler uncomment this @@ -537,9 +541,15 @@ EXTRACFLAGS = $(EXTRACFLAGS) $(CXX_FLAG) !ENDIF CFLAGS = $(EXTRACFLAGS) $(INCLUDES) $(DEFINES) $(LOCDEFS) \ $(PCHFLAGS) $(OPTIMIZE) +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +LINK_FLAGS = -nologo $(LINK_DBG) \ + -libpath:"$(INST_COREDIR)" \ + -machine:$(PROCESSOR_ARCHITECTURE) +!ELSE LINK_FLAGS = -nologo -nodefaultlib $(LINK_DBG) \ -libpath:"$(INST_COREDIR)" \ -machine:$(PROCESSOR_ARCHITECTURE) +!ENDIF LIB_FLAGS = $(LIB_FLAGS) -nologo OBJOUT_FLAG = -Fo EXEOUT_FLAG = -Fe @@ -910,6 +920,18 @@ config.w32 : $(CFGSH_TMPL) @echo.>>$@ @echo #ifndef _config_h_footer_>>$@ @echo #define _config_h_footer_>>$@ +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" + @echo #undef USE_STDIO_PTR>>$@ + @echo #undef FILE_ptr>>$@ + @echo #undef STDIO_PTR_LVALUE>>$@ + @echo #undef FILE_cnt>>$@ + @echo #undef STDIO_CNT_LVALUE>>$@ + @echo #undef STDIO_PTR_LVAL_SETS_CNT>>$@ + @echo #undef STDIO_PTR_LVAL_NOCHANGE_CNT>>$@ + @echo #undef USE_STDIO_BASE>>$@ + @echo #undef FILE_base>>$@ + @echo #undef FILE_bufsiz>>$@ +!ENDIF @echo #undef Off_t>>$@ @echo #undef LSEEKSIZE>>$@ @echo #undef Off_t_size>>$@ diff --git a/win32/config_sh.PL b/win32/config_sh.PL index 98255a8..785b9c3 100644 --- a/win32/config_sh.PL +++ b/win32/config_sh.PL @@ -267,18 +267,26 @@ if ($opt{cc} =~ /\bcl/ and $opt{ccversion} =~ /^(\d+)/) { $opt{sGMTIME_max} = 32535291599; $opt{sLOCALTIME_max} = 32535244799; } - if($ccversion < 13) { #VC6 + if ($ccversion < 13) { # VC6 $opt{ar} ='lib'; } + if ($ccversion >= 19) { # VC14 + $opt{d_stdiobase} = 'undef'; + $opt{d_stdstdio} = 'undef'; + } } #find out which MSVC this ICC is using elsif ($opt{cc} =~ /\bicl/) { my $output = `cl --version 2>&1`; my $num_ver = $output =~ /^.*Version\s+([\d.]+)/ ? $1 : '?'; - if($num_ver =~ /^(\d+)/ && $1 >= 14) { + if ($num_ver =~ /^(\d+)/ && $1 >= 14) { $opt{sGMTIME_max} = 32535291599; $opt{sLOCALTIME_max} = 32535244799; } + if ($num_ver =~ /^(\d+)/ && $1 >= 19) { # VC14 + $opt{d_stdiobase} = 'undef'; + $opt{d_stdstdio} = 'undef'; + } $opt{ar} ='xilib'; } diff --git a/win32/makefile.mk b/win32/makefile.mk index c74b5bb..5bef974 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -138,6 +138,10 @@ USE_LARGE_FILES *= define #CCTYPE = MSVC120 # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) #CCTYPE = MSVC120FREE +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) +#CCTYPE = MSVC140 +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) +#CCTYPE = MSVC140FREE # MinGW or mingw-w64 with gcc-3.4.5 or later CCTYPE *= GCC @@ -1074,6 +1078,18 @@ config.w32 : $(CFGSH_TMPL) @echo.>>$@ @echo #ifndef _config_h_footer_>>$@ @echo #define _config_h_footer_>>$@ +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" + @echo #undef USE_STDIO_PTR>>$@ + @echo #undef FILE_ptr>>$@ + @echo #undef STDIO_PTR_LVALUE>>$@ + @echo #undef FILE_cnt>>$@ + @echo #undef STDIO_CNT_LVALUE>>$@ + @echo #undef STDIO_PTR_LVAL_SETS_CNT>>$@ + @echo #undef STDIO_PTR_LVAL_NOCHANGE_CNT>>$@ + @echo #undef USE_STDIO_BASE>>$@ + @echo #undef FILE_base>>$@ + @echo #undef FILE_bufsiz>>$@ +!ENDIF @echo #undef Off_t>>$@ @echo #undef LSEEKSIZE>>$@ @echo #undef Off_t_size>>$@ -- 1.9.5.msysgit.1
RT-Send-CC: perl5-porters [...] perl.org
Oops, spotted a couple of typos in win32.h in the v2 patch. Fixed in this new version.
Subject: 0001-Add-VS2015-support-v2-with-typos-corrected-in-win32..patch
From 1389bd0ab40831db87362d139998555c7b5efbd2 Mon Sep 17 00:00:00 2001 From: Steve Hay <steve.m.hay@googlemail.com> Date: Wed, 29 Jul 2015 08:56:45 +0100 Subject: [PATCH] Add VS2015 support, v2 (with typos corrected in win32.h) Currently not working due to at least: win32.c(3956): warning C4013: 'gets' undefined; assuming extern returning int win32.obj : error LNK2001: unresolved external symbol __imp____pioinfo --- perlio.c | 2 +- win32/Makefile | 24 ++++++++++++++++++++++++ win32/config_sh.PL | 14 ++++++++++++++ win32/makefile.mk | 24 ++++++++++++++++++++++++ win32/win32.c | 6 +++--- win32/win32.h | 38 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 104 insertions(+), 4 deletions(-) diff --git a/perlio.c b/perlio.c index ae8cbc9..16ffa47 100644 --- a/perlio.c +++ b/perlio.c @@ -3180,7 +3180,7 @@ PerlIOStdio_invalidate_fileno(pTHX_ FILE *f) structure at all */ # else - f->_file = -1; + PERLIO_FILE_file(f) = -1; # endif return 1; # else diff --git a/win32/Makefile b/win32/Makefile index a2fb2aa..27c119f 100644 --- a/win32/Makefile +++ b/win32/Makefile @@ -126,6 +126,10 @@ CCTYPE = MSVC60 #CCTYPE = MSVC120 # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) #CCTYPE = MSVC120FREE +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) +#CCTYPE = MSVC140 +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) +#CCTYPE = MSVC140FREE # # If you are using Intel C++ Compiler uncomment this @@ -445,6 +449,10 @@ CXX_FLAG = -TP -EHsc LIBC = msvcrt.lib +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +LIBC = $(LIBC) vcruntime.lib ucrt.lib +!ENDIF + !IF "$(CFG)" == "Debug" OPTIMIZE = -Od -MD -Zi -DDEBUGGING LINK_DBG = -debug @@ -489,6 +497,11 @@ OPTIMIZE = $(OPTIMIZE) -fp:precise DEFINES = $(DEFINES) -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE !ENDIF +# Likewise for deprecated Winsock APIs in VC++ 14.0 for now. +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +DEFINES = $(DEFINES) -D_WINSOCK_DEPRECATED_NO_WARNINGS +!ENDIF + # In VS 2005 (VC++ 8.0) Microsoft changes time_t from 32-bit to # 64-bit, even in 32-bit mode. It also provides the _USE_32BIT_TIME_T # preprocessor option to revert back to the old functionality for @@ -910,6 +923,17 @@ config.w32 : $(CFGSH_TMPL) @echo.>>$@ @echo #ifndef _config_h_footer_>>$@ @echo #define _config_h_footer_>>$@ +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" + @echo #undef FILE_ptr>>$@ + @echo #define FILE_ptr(fp) PERLIO_FILE_ptr(fp)>>$@ + @echo #undef FILE_cnt>>$@ + @echo #define FILE_cnt(fp) PERLIO_FILE_cnt(fp)>>$@ + @echo #undef FILE_base>>$@ + @echo #define FILE_base(fp) PERLIO_FILE_base(fp)>>$@ + @echo #undef FILE_bufsiz>>$@ + @echo #define FILE_bufsiz(fp) (PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))>>$@ + @echo #define I_STDBOOL>>$@ +!ENDIF @echo #undef Off_t>>$@ @echo #undef LSEEKSIZE>>$@ @echo #undef Off_t_size>>$@ diff --git a/win32/config_sh.PL b/win32/config_sh.PL index 98255a8..69599b1 100644 --- a/win32/config_sh.PL +++ b/win32/config_sh.PL @@ -270,6 +270,13 @@ if ($opt{cc} =~ /\bcl/ and $opt{ccversion} =~ /^(\d+)/) { if($ccversion < 13) { #VC6 $opt{ar} ='lib'; } + if ($ccversion >= 19) { # VC14 + $opt{stdio_base} = 'PERLIO_FILE_base(fp)'; + $opt{stdio_bufsiz} = '(PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))'; + $opt{stdio_cnt} = 'PERLIO_FILE_cnt(fp)'; + $opt{stdio_ptr} = 'PERLIO_FILE_ptr(fp)'; + $opt{i_stdbool} = 'define'; + } } #find out which MSVC this ICC is using elsif ($opt{cc} =~ /\bicl/) { @@ -279,6 +286,13 @@ elsif ($opt{cc} =~ /\bicl/) { $opt{sGMTIME_max} = 32535291599; $opt{sLOCALTIME_max} = 32535244799; } + if ($num_ver =~ /^(\d+)/ && $1 >= 19) { # VC14 + $opt{stdio_base} = 'PERLIO_FILE_base(fp)'; + $opt{stdio_bufsiz} = '(PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))'; + $opt{stdio_cnt} = 'PERLIO_FILE_cnt(fp)'; + $opt{stdio_ptr} = 'PERLIO_FILE_ptr(fp)'; + $opt{i_stdbool} = 'define'; + } $opt{ar} ='xilib'; } diff --git a/win32/makefile.mk b/win32/makefile.mk index c74b5bb..3a708ac 100644 --- a/win32/makefile.mk +++ b/win32/makefile.mk @@ -138,6 +138,10 @@ USE_LARGE_FILES *= define #CCTYPE = MSVC120 # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) #CCTYPE = MSVC120FREE +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) +#CCTYPE = MSVC140 +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) +#CCTYPE = MSVC140FREE # MinGW or mingw-w64 with gcc-3.4.5 or later CCTYPE *= GCC @@ -563,6 +567,10 @@ CXX_FLAG = -TP -EHsc LIBC = msvcrt.lib +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +LIBC = $(LIBC) vcruntime.lib ucrt.lib +!ENDIF + .IF "$(CFG)" == "Debug" OPTIMIZE = -Od -MD -Zi -DDEBUGGING LINK_DBG = -debug @@ -603,6 +611,11 @@ OPTIMIZE += -fp:precise DEFINES += -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE .ENDIF +# Likewise for deprecated Winsock APIs in VC++ 14.0 for now. +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +DEFINES += -D_WINSOCK_DEPRECATED_NO_WARNINGS +.ENDIF + # In VS 2005 (VC++ 8.0) Microsoft changes time_t from 32-bit to # 64-bit, even in 32-bit mode. It also provides the _USE_32BIT_TIME_T # preprocessor option to revert back to the old functionality for @@ -1074,6 +1087,17 @@ config.w32 : $(CFGSH_TMPL) @echo.>>$@ @echo #ifndef _config_h_footer_>>$@ @echo #define _config_h_footer_>>$@ +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" + @echo #undef FILE_ptr>>$@ + @echo #define FILE_ptr(fp) PERLIO_FILE_ptr(fp)>>$@ + @echo #undef FILE_cnt>>$@ + @echo #define FILE_cnt(fp) PERLIO_FILE_cnt(fp)>>$@ + @echo #undef FILE_base>>$@ + @echo #define FILE_base(fp) PERLIO_FILE_base(fp)>>$@ + @echo #undef FILE_bufsiz>>$@ + @echo #define FILE_bufsiz(fp) (PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))>>$@ + @echo #define I_STDBOOL>>$@ +.ENDIF @echo #undef Off_t>>$@ @echo #undef LSEEKSIZE>>$@ @echo #undef Off_t_size>>$@ diff --git a/win32/win32.c b/win32/win32.c index 2b883a2..75a017a 100644 --- a/win32/win32.c +++ b/win32/win32.c @@ -4130,15 +4130,15 @@ win32_fdupopen(FILE *pf) int fileno = win32_dup(win32_fileno(pf)); /* open the file in the same mode */ - if((pf)->_flag & _IOREAD) { + if (PERLIO_FILE_flag(pf) & PERLIO_FILE_flag_RD) { mode[0] = 'r'; mode[1] = 0; } - else if((pf)->_flag & _IOWRT) { + else if (PERLIO_FILE_flag(pf) & PERLIO_FILE_flag_WR) { mode[0] = 'a'; mode[1] = 0; } - else if((pf)->_flag & _IORW) { + else if (PERLIO_FILE_flag(pf) & PERLIO_FILE_flag_RW) { mode[0] = 'r'; mode[1] = '+'; mode[2] = 0; diff --git a/win32/win32.h b/win32/win32.h index 3b35b6c..cca03ce 100644 --- a/win32/win32.h +++ b/win32/win32.h @@ -282,6 +282,44 @@ extern const __declspec(selectany) union { unsigned __int64 __q; double __d; } __PL_nan_u = { 0x7FF8000000000000UI64 }; # define NV_NAN ((NV)__PL_nan_u.__d) +#if defined(_MSC_VER) && _MSC_VER >= 1900 +// From corecrt_internal_stdio.h: +typedef struct +{ + union + { + FILE _public_file; + char* _ptr; + }; + + char* _base; + int _cnt; + long _flags; + long _file; + int _charbuf; + int _bufsiz; + char* _tmpfname; + CRITICAL_SECTION _lock; +} __crt_stdio_stream_data; +#define PERLIO_FILE_flag_RD 0x0001 // _IOREAD +#define PERLIO_FILE_flag_WR 0x0002 // _IOWRITE +#define PERLIO_FILE_flag_RW 0x0004 // _IOUPDATE +#define PERLIO_FILE_ptr(f) (((__crt_stdio_stream_data*)(f))->_ptr) +#define PERLIO_FILE_base(f) (((__crt_stdio_stream_data*)(f))->_base) +#define PERLIO_FILE_cnt(f) (((__crt_stdio_stream_data*)(f))->_cnt) +#define PERLIO_FILE_flag(f) ((int)(((__crt_stdio_stream_data*)(f))->_flags)) +#define PERLIO_FILE_file(f) ((int)(((__crt_stdio_stream_data*)(f))->_file)) +#else +#define PERLIO_FILE_flag_RD _IOREAD // 0x001 +#define PERLIO_FILE_flag_WR _IOWRT // 0x002 +#define PERLIO_FILE_flag_RW _IORW // 0x080 +#define PERLIO_FILE_ptr(f) ((f)->_ptr) +#define PERLIO_FILE_base(f) ((f)->_base) +#define PERLIO_FILE_cnt(f) ((f)->_cnt) +#define PERLIO_FILE_flag(f) ((f)->_flag) +#define PERLIO_FILE_file(f) ((f)->_file) +#endif + #endif /* _MSC_VER */ #ifdef __MINGW32__ /* Minimal Gnu-Win32 */ -- 1.9.5.msysgit.1
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 9.9k
On Wed Jul 29 00:51:26 2015, shay wrote: Show quoted text
> This ticket is for centralizing discussions about how to add support > for building with VS2015. > > There have been a couple of p5p threads already; it is better to keep > it all in one place: > > http://www.nntp.perl.org/group/perl.perl5.porters/2015/05/msg227702.html > http://www.nntp.perl.org/group/perl.perl5.porters/2015/07/msg229524.html > > Attached is my first attempt, originally posted in the latter thread > above (saved here since http://www.nntp.perl.org doesn't seem to save > attachments).
@@ -910,6 +923,17 @@ config.w32 : $(CFGSH_TMPL) @echo.>>$@ @echo #ifndef _config_h_footer_>>$@ @echo #define _config_h_footer_>>$@ +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" + @echo #undef FILE_ptr>>$@ + @echo #define FILE_ptr(fp) PERLIO_FILE_ptr(fp)>>$@ + @echo #undef FILE_cnt>>$@ + @echo #define FILE_cnt(fp) PERLIO_FILE_cnt(fp)>>$@ + @echo #undef FILE_base>>$@ + @echo #define FILE_base(fp) PERLIO_FILE_base(fp)>>$@ + @echo #undef FILE_bufsiz>>$@ + @echo #define FILE_bufsiz(fp) (PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))>>$@ + @echo #define I_STDBOOL>>$@ +!ENDIF @echo #undef Off_t>>$@ @echo #undef LSEEKSIZE>>$@ @echo #undef Off_t_size>>$@ diff --git a/win32/config_sh.PL b/win32/config_sh.PL index 98255a8..69599b1 100644 --- a/win32/config_sh.PL +++ b/win32/config_sh.PL @@ -270,6 +270,13 @@ if ($opt{cc} =~ /\bcl/ and $opt{ccversion} =~ /^(\d+)/) { if($ccversion < 13) { #VC6 $opt{ar} ='lib'; } + if ($ccversion >= 19) { # VC14 + $opt{stdio_base} = 'PERLIO_FILE_base(fp)'; + $opt{stdio_bufsiz} = '(PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))'; + $opt{stdio_cnt} = 'PERLIO_FILE_cnt(fp)'; + $opt{stdio_ptr} = 'PERLIO_FILE_ptr(fp)'; + $opt{i_stdbool} = 'define'; + } } #find out which MSVC this ICC is using elsif ($opt{cc} =~ /\bicl/) { @@ -279,6 +286,13 @@ elsif ($opt{cc} =~ /\bicl/) { $opt{sGMTIME_max} = 32535291599; $opt{sLOCALTIME_max} = 32535244799; } + if ($num_ver =~ /^(\d+)/ && $1 >= 19) { # VC14 + $opt{stdio_base} = 'PERLIO_FILE_base(fp)'; + $opt{stdio_bufsiz} = '(PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))'; + $opt{stdio_cnt} = 'PERLIO_FILE_cnt(fp)'; + $opt{stdio_ptr} = 'PERLIO_FILE_ptr(fp)'; + $opt{i_stdbool} = 'define'; + } $opt{ar} ='xilib'; } ---------------------------- @@ -1074,6 +1087,17 @@ config.w32 : $(CFGSH_TMPL) @echo.>>$@ @echo #ifndef _config_h_footer_>>$@ @echo #define _config_h_footer_>>$@ +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" + @echo #undef FILE_ptr>>$@ + @echo #define FILE_ptr(fp) PERLIO_FILE_ptr(fp)>>$@ + @echo #undef FILE_cnt>>$@ + @echo #define FILE_cnt(fp) PERLIO_FILE_cnt(fp)>>$@ + @echo #undef FILE_base>>$@ + @echo #define FILE_base(fp) PERLIO_FILE_base(fp)>>$@ + @echo #undef FILE_bufsiz>>$@ + @echo #define FILE_bufsiz(fp) (PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))>>$@ + @echo #define I_STDBOOL>>$@ +.ENDIF @echo #undef Off_t>>$@ @echo #undef LSEEKSIZE>>$@ @echo #undef Off_t_size>>$@ ---------------------------------------------- This is overcomplicated. This should be done in win32.h #if _MSC_VER >= 1400. Why isn't PERLIO_FILE_base/etc used on every Win32 CC config including GCC and then PERLIO_FILE_base is implement one way or the other in win32.h? Those "+ @echo #undef FILE_ptr>>$@" lines are slow, each one is its own cmd.exe process launched by the make tool during the build process. It is best if there are less of them. This slowness note is from the Win32 parallel building project (see its ticket or prior ML threads). ---------------------------------------------- @@ -563,6 +567,10 @@ CXX_FLAG = -TP -EHsc LIBC = msvcrt.lib +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +LIBC = $(LIBC) vcruntime.lib ucrt.lib +!ENDIF + ---------------------------------------------- Spiritually, ucrt.lib/ucrtbase.dll is the MS libc now. VC 2015 msvcrt.lib only contains static link code for things like CPU probing and math ops. I am slightly slightly slightly afraid that lib won't be -e/-f-able anymore. This can break toolchain stuff, like when you stuff double quotes to fix a spaces in path on cmd line problem in a file path to fix something EUMM/MB/CB-ish, and now every breaks since it isn't a valid file path anymore. Maybe 2 of the 3 libs need to be put in $Config{libs} where oldnames.lib, kernel32.lib, user32.lib, gdi32.lib, etc are. I'd say ucrt.lib is the file to do probe for symbol names if you have some code that scans (not test compiles) "the libc" for what posix/c std lib symbols/functions it implements. ------------------------------------------------ @@ -603,6 +611,11 @@ OPTIMIZE += -fp:precise DEFINES += -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE .ENDIF +# Likewise for deprecated Winsock APIs in VC++ 14.0 for now. +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" +DEFINES += -D_WINSOCK_DEPRECATED_NO_WARNINGS +.ENDIF + # In VS 2005 (VC++ 8.0) Microsoft changes time_t from 32-bit to # 64-bit, even in 32-bit mode. It also provides the _USE_32BIT_TIME_T # preprocessor option to revert back to the old functionality for ------------------------------------------------ There are now 3 huge "no deprecate" -Ds on the command line. There will probably be a new no-deprecate for every version of Visual C. These should really go into win32.h. Unix perl builds have atmost 2 or 3 -Ds, not 10 or 15 -Ds. This might require reorganization of perl.h since the first CC .h is sys/types.h around line 670 in perl.h. config.h looks like the only place to the defines before sys/types.h. Some experiments need to be done if those no-deprecate defines can be moved to win32.h. Show quoted text
> Also attached is my second attempt: This does surprisingly well with > the quick-and-dirty approach of simply casting a FILE* to the > appropriate new CRT struct (__crt_stdio_stream_data) and accessing the > members of that. It might even suffice, rather than going to the > hassle of writing accessors and probably needing to separate getters > from setters?
It looks like __crt_stdio_stream_data is allocated by the plain c calloc_base() function. That means msize can be used on it. msize is used for the pioinfo/osfhnd code for VC 2005 already since the struct repeatdly changed sizes during VC 2005's lifetime. On 32 bits, __crt_stdio_stream_data is supposedly 0x38 bytes long, IDK what it is on 64. You can write an assert/DEBUGGING comparing the msize() to sizeof() as a sanity check. I support the casting+macro solution. It makes the best machine code. It can also be argued that since VC 2015 is so "new", msize should always be used because an automatic OS update can change ucrtbase.dll at any time. IDK if ucrtbase.dll is VC redist package maintained (very rare to change), Service Pack/OS upgrade only changed (medium risk) or windows updates maintained (high risk of change). Unlike the osfhnd code, where perl is only interested in the first member of the struct, with __crt_stdio_stream_data it is interested in the middle members. If MS changes the order of the members under us, we are screwed. MS did change the order to move int _cnt between pointer "_ptr" and pointer "_base" to now sitting after _base http://www.nntp.perl.org/group/perl.perl5.porters/2015/05/msg227727.html when they created the UCRT. If there is a non-DEBUGGING/always msize done on a __crt_stdio_stream_data * at start up, the only thing msize check will do is abort the perl process on startup, since we can't know what the struct will be in the future. You will only get back a working perl by downgrading the OS/uninstall a hotfix, or upgrade to the next perl maint release. At this point it is probably unlikely the struct members will be rearranged, only new ones added to the end. That doesn't affect us in reaching the old members. My final opinion is the msize should be DEBUGGING only, plus Ruby isn't doing it https://github.com/ruby/ruby/commit/3446537f1a8f6a0dbc2b27a0f25af13c7f61abf8 (IDK if Ruby has a ticket/discussion for that commit or not). Show quoted text
> However, the build currently fails to link due to __pioinfo apparently > no longer being present in the new CRT (or else I'm linking the wrong > libs?): > > win32.obj : error LNK2001: unresolved external symbol __imp____pioinfo > > This was added (actually restored and modified from earlier ripped out > code) by: > > http://perl5.git.perl.org/perl.git/commit/b47a847f6284f6f98ad7509cf77a4aeb802d8fce > > As that commit notes, "If __pioinfo isn't exported anymore, the Perl > build will break." So this was always known to be an area of danger > and impending breakage; the question is: What do we do about it now > that it's happened? > > bulk88: Do you have any ideas on the latter point?
File a bug ticket with MS at https://connect.microsoft.com/VisualStudio/ to get __pioinfo from ucrtbase.dll. In the mean time the easy but not best performance solution for is for Perl on VC 2015 to only support static UCRT from libucrt.lib since the libc/mscrt/ucrt symbols like _pioinfo can't hide from the linker when static linking. Ruby has the same exact problem as we do https://github.com/ruby/ruby/blob/trunk/win32/win32.c#L2241 IDK how they are dealing with it or they are currently broken on VC 2015. Old python used pioinfo https://github.com/python/cpython/blob/2.7/Modules/posixmodule.c#L532 they stopped using for VC 2015 https://github.com/python/cpython/blob/master/Modules/posixmodule.c#L1068 https://bugs.python.org/issue23524 , I dont like their solution of swapping the callback everywhere. They were reaching inside the ioinfo struct to check if its allocated/open FD or not, without triggering MS CRT's invalid param handler/exception/whatever. Python doesn't have the "sockets in libc FDs on win32" problem Perl and Ruby has, python sockets live in a class by themselves so they aren't FDs https://rt.perl.org/Ticket/Display.html?id=118127#txn-1357875 . The other 2 solutions I have in my head on how to get ucrtbase.dll's _pioinfo are, ummm, crazy, but would work. So any point in mentioning them or will I scare the children? -- bulk88 ~ bulk88 at hotmail.com
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 196b
This is a test case showing why the handle must be removed from the CRT fd before calling close. This test case is sorta intended to be given to MS in a ticket. -- bulk88 ~ bulk88 at hotmail.com
Subject: build.bat
Download build.bat
application/octet-stream 112b

Message body not shown because it is not plain text.

Subject: clean.bat
Download clean.bat
application/octet-stream 44b

Message body not shown because it is not plain text.

Subject: close 18 MB per sec mem leak, no handle leak, no dbg invalid hnd excepts.txt
Subject: close.cpp
Download close.cpp
text/x-c++src 536b
// PerlCrash.cpp : Defines the entry point for the console application. // #define _WIN32_WINNT 0x0500 #include <tchar.h> #include <stdio.h> #include <io.h> #include <windows.h> int _tmain(int argc, _TCHAR* argv[]) { WORD wVersionRequested = MAKEWORD(2, 2); WSADATA wsaData; int err; err = WSAStartup(wVersionRequested, &wsaData); int i = 0; for(; i< 100000000; i++) { HANDLE h = (HANDLE)socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); int fd = _open_osfhandle((intptr_t)h, 0); close(fd); } return 0; }
Subject: closeclosesocket 18 MB per sec mem leak, no handle leak, has invalid handle exceptions but they are trapped by winsock.txt
ntdll.dll!_KiRaiseUserExceptionDispatcher@0() + 0x37 ntdll.dll!_KiFastSystemCall@0() + 0x3 ntdll.dll!_NtClose@4() + 0xc mswsock.dll!_WSPCloseSocket@8() + 0x129 ws2_32.dll!_closesocket@4() + 0x47 closeclosesocket.exe!main(int argc=1, char * * argv=0x00391040) Line 22 C++ closeclosesocket.exe!mainCRTStartup() Line 259 + 0x12 C kernel32.dll!_BaseProcessStart@4() + 0x23 "Time of Day","Process Name","PID","TID","Operation","Path","Result","Detail" "1:42:00.3590502 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3592156 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3593642 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3595207 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3596690 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3598353 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3599892 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3601406 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3602943 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3604434 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3606300 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3607770 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3609130 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3610477 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3612731 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:42:00.3614352 PM","closeclosesocket.exe","4552","6140","RegCloseKey","<INVALID NAME>","INVALID HANDLE",""
Subject: closeclosesocket.cpp
Download closeclosesocket.cpp
text/x-c++src 561b
// PerlCrash.cpp : Defines the entry point for the console application. // #define _WIN32_WINNT 0x0500 #include <tchar.h> #include <stdio.h> #include <io.h> #include <windows.h> int _tmain(int argc, _TCHAR* argv[]) { WORD wVersionRequested = MAKEWORD(2, 2); WSADATA wsaData; int err; err = WSAStartup(wVersionRequested, &wsaData); int i = 0; for(; i< 100000000; i++) { HANDLE h = (HANDLE)socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); int fd = _open_osfhandle((intptr_t)h, 0); close(fd); closesocket((SOCKET)h); } return 0; }
Subject: closesocketclose no mem leak, no handle leak, has dbg invalid hnd excepts.txt
Unhandled exception at 0x7c90e4ff (ntdll.dll) in closesocketclose.exe: 0xC0000008: An invalid HANDLE was specified. ntdll.dll!_KiRaiseUserExceptionDispatcher@0() + 0x37 ntdll.dll!_KiFastSystemCall@0() + 0x3 ntdll.dll!_NtClose@4() + 0xc kernel32.dll!_CloseHandle@4() + 0x44 closesocketclose.exe!_close(int fh=3) Line 115 + 0x3a C closesocketclose.exe!main(int argc=1, char * * argv=0x00391040) Line 21 + 0xc C++ closesocketclose.exe!mainCRTStartup() Line 259 + 0x12 C kernel32.dll!_BaseProcessStart@4() + 0x23 "Time of Day","Process Name","PID","TID","Operation","Path","Result","Detail" "1:35:14.4713377 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:14.6744551 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:15.4868145 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:15.7916631 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:15.9438503 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:16.1470105 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:16.4008801 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:16.5531506 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:16.8073117 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:17.0104322 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:17.2136544 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:17.4196066 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE","" "1:35:17.6228050 PM","closesocketclose.exe","2636","4356","RegCloseKey","<INVALID NAME>","INVALID HANDLE",""
Subject: closesocketclose.cpp
Download closesocketclose.cpp
text/x-c++src 561b
// PerlCrash.cpp : Defines the entry point for the console application. // #define _WIN32_WINNT 0x0500 #include <tchar.h> #include <stdio.h> #include <io.h> #include <windows.h> int _tmain(int argc, _TCHAR* argv[]) { WORD wVersionRequested = MAKEWORD(2, 2); WSADATA wsaData; int err; err = WSAStartup(wVersionRequested, &wsaData); int i = 0; for(; i< 100000000; i++) { HANDLE h = (HANDLE)socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); int fd = _open_osfhandle((intptr_t)h, 0); closesocket((SOCKET)h); close(fd); } return 0; }
Date: Thu, 30 Jul 2015 18:10:02 +0000
From: Steve Hay <Steve.Hay [...] verosoftware.com>
To: "perlbug-followup [...] perl.org" <perlbug-followup [...] perl.org>
Subject: RE: [perl #125714] Adding support for VS2015 (VC14)
CC: "perl5-porters [...] perl.org" <perl5-porters [...] perl.org>
Download (untitled) / with headers
text/plain 9.1k
On 29 July 2015 at 13:39, bulk88 via RT <perlbug-followup@perl.org> wrote: Show quoted text
> On Wed Jul 29 00:51:26 2015, shay wrote: > ---------------------------- > @@ -1074,6 +1087,17 @@ config.w32 : $(CFGSH_TMPL) > @echo.>>$@ > @echo #ifndef _config_h_footer_>>$@ > @echo #define _config_h_footer_>>$@ > +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" > + @echo #undef FILE_ptr>>$@ > + @echo #define FILE_ptr(fp) PERLIO_FILE_ptr(fp)>>$@ > + @echo #undef FILE_cnt>>$@ > + @echo #define FILE_cnt(fp) PERLIO_FILE_cnt(fp)>>$@ > + @echo #undef FILE_base>>$@ > + @echo #define FILE_base(fp) PERLIO_FILE_base(fp)>>$@ > + @echo #undef FILE_bufsiz>>$@ > + @echo #define FILE_bufsiz(fp) (PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))>>$@ > + @echo #define I_STDBOOL>>$@ > +.ENDIF > @echo #undef Off_t>>$@ > @echo #undef LSEEKSIZE>>$@ > @echo #undef Off_t_size>>$@ > ---------------------------------------------- > This is overcomplicated. This should be done in win32.h #if _MSC_VER >= 1400. Why isn't PERLIO_FILE_base/etc used on every Win32 CC config including GCC and then PERLIO_FILE_base is implement one way or the other in win32.h? Those "+ @echo #undef FILE_ptr>>$@" lines are slow, each one is its own cmd.exe process launched by the make tool during the build process. It is best if there are less of them. This slowness note is from the Win32 parallel building project (see its ticket or prior ML threads).
I will come back to this later since it's a separate issue from getting things building with VS2015. The whole business of all these echo lines needs sorting out better somehow, not just these new lines. Show quoted text
> > ---------------------------------------------- > @@ -563,6 +567,10 @@ CXX_FLAG = -TP -EHsc > > LIBC = msvcrt.lib > > +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" > +LIBC = $(LIBC) vcruntime.lib ucrt.lib > +!ENDIF > + > ---------------------------------------------- > > Spiritually, ucrt.lib/ucrtbase.dll is the MS libc now. VC 2015 msvcrt.lib only contains static link code for things like CPU probing and math ops. I am slightly slightly slightly afraid that lib won't be -e/-f-able anymore. This can break toolchain stuff, like when you stuff double quotes to fix a spaces in path on cmd line problem in a file path to fix something EUMM/MB/CB-ish, and now every breaks since it isn't a valid file path anymore. Maybe 2 of the 3 libs need to be put in $Config{libs} where oldnames.lib, kernel32.lib, user32.lib, gdi32.lib, etc are. I'd say ucrt.lib is the file to do probe for symbol names if you have some code that scans (not test compiles) "the libc" for what posix/c std lib symbols/functions it implements.
Done in the v3 patch attached. Show quoted text
> > ------------------------------------------------ > @@ -603,6 +611,11 @@ OPTIMIZE += -fp:precise > DEFINES += -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE > .ENDIF > > +# Likewise for deprecated Winsock APIs in VC++ 14.0 for now. > +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" > +DEFINES += -D_WINSOCK_DEPRECATED_NO_WARNINGS > +.ENDIF > + > # In VS 2005 (VC++ 8.0) Microsoft changes time_t from 32-bit to > # 64-bit, even in 32-bit mode. It also provides the _USE_32BIT_TIME_T > # preprocessor option to revert back to the old functionality for > ------------------------------------------------ > > There are now 3 huge "no deprecate" -Ds on the command line. There will probably be a new no-deprecate for every version of Visual C. These should really go into win32.h. Unix perl builds have atmost 2 or 3 -Ds, not 10 or 15 -Ds. This might require reorganization of perl.h since the first CC .h is sys/types.h around line 670 in perl.h. config.h looks like the only place to the defines before sys/types.h. Some experiments need to be done if those no-deprecate defines can be moved to win32.h.
I will also return to this since it is again separate from VS2015 support. Show quoted text
>
>> Also attached is my second attempt: This does surprisingly well with >> the quick-and-dirty approach of simply casting a FILE* to the >> appropriate new CRT struct (__crt_stdio_stream_data) and accessing the >> members of that. It might even suffice, rather than going to the >> hassle of writing accessors and probably needing to separate getters >> from setters?
> > It looks like __crt_stdio_stream_data is allocated by the plain c calloc_base() function. That means msize can be used on it. msize is used for the pioinfo/osfhnd code for VC 2005 already since the struct repeatdly changed sizes during VC 2005's lifetime. On 32 bits, __crt_stdio_stream_data is supposedly 0x38 bytes long, IDK what it is on 64. You can write an assert/DEBUGGING comparing the msize() to sizeof() as a sanity check. > > I support the casting+macro solution. It makes the best machine code. > > It can also be argued that since VC 2015 is so "new", msize should always be used because an automatic OS update can change ucrtbase.dll at any time. IDK if ucrtbase.dll is VC redist package maintained (very rare to change), Service Pack/OS upgrade only changed (medium risk) or windows updates maintained (high risk of change). Unlike the osfhnd code, where perl is only interested in the first member of the struct, with __crt_stdio_stream_data it is interested in the middle members. If MS changes the order of the members under us, we are screwed. > > MS did change the order to move int _cnt between pointer "_ptr" and pointer "_base" to now sitting after _base http://www.nntp.perl.org/group/perl.perl5.porters/2015/05/msg227727.html when they created the UCRT. If there is a non-DEBUGGING/always msize done on a __crt_stdio_stream_data * at start up, the only thing msize check will do is abort the perl process on startup, since we can't know what the struct will be in the future. You will only get back a working perl by downgrading the OS/uninstall a hotfix, or upgrade to the next perl maint release. At this point it is probably unlikely the struct members will be rearranged, only new ones added to the end. That doesn't affect us in reaching the old members. My final opinion is the msize should be DEBUGGING only, plus Ruby isn't doing it https://github.com/ruby/ruby/commit/3446537f1a8f6a0dbc2b27a0f25af13c7f61abf8 (IDK if Ruby has a ticket/discussion for that commit or not).
Done in the v3 patch attached. Show quoted text
>
>> However, the build currently fails to link due to __pioinfo apparently >> no longer being present in the new CRT (or else I'm linking the wrong >> libs?): >> >> win32.obj : error LNK2001: unresolved external symbol __imp____pioinfo >> >> This was added (actually restored and modified from earlier ripped out >> code) by: >> >> http://perl5.git.perl.org/perl.git/commit/b47a847f6284f6f98ad7509cf77a4aeb802d8fce >> >> As that commit notes, "If __pioinfo isn't exported anymore, the Perl >> build will break." So this was always known to be an area of danger >> and impending breakage; the question is: What do we do about it now >> that it's happened? >> >> bulk88: Do you have any ideas on the latter point?
> > File a bug ticket with MS at https://connect.microsoft.com/VisualStudio/ to get __pioinfo from ucrtbase.dll. > > In the mean time the easy but not best performance solution for is for Perl on VC 2015 to only support static UCRT from libucrt.lib since the libc/mscrt/ucrt symbols like _pioinfo can't hide from the linker when static linking.
Done in the v3 patch attached, but I'm not very keen on this. Perl has always used the CRT DLL, which makes me very uneasy about switching to LIBC. It does get it building, though, but now crashes on startup trying to set environ[0] to NULL in S_init_postdump_symbols(). I briefly tried #defining NO_ENVIRON_ARRAY to temporarily duck that issue but the build failed because Perl_my_setenv() is then not provided by util.c, but is apparently still required by mg.c (at least)... :-/ Is this problem due to the switch from dynamic to static CRT? Or just something new in VC14 anyway? Show quoted text
> > Ruby has the same exact problem as we do https://github.com/ruby/ruby/blob/trunk/win32/win32.c#L2241 IDK how they are dealing with it or they are currently broken on VC 2015. > > Old python used pioinfo https://github.com/python/cpython/blob/2.7/Modules/posixmodule.c#L532 they stopped using for VC 2015 https://github.com/python/cpython/blob/master/Modules/posixmodule.c#L1068 https://bugs.python.org/issue23524 , I dont like their solution of swapping the callback everywhere. They were reaching inside the ioinfo struct to check if its allocated/open FD or not, without triggering MS CRT's invalid param handler/exception/whatever. Python doesn't have the "sockets in libc FDs on win32" problem Perl and Ruby has, python sockets live in a class by themselves so they aren't FDs https://rt.perl.org/Ticket/Display.html?id=118127#txn-1357875 . > > The other 2 solutions I have in my head on how to get ucrtbase.dll's _pioinfo are, ummm, crazy, but would work. So any point in mentioning them or will I scare the children? >
Since I'm not keen on the static CRT change, what are the other options that you see here?

Message body is not shown because sender requested not to inline it.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 544b
On Thu Jul 30 11:10:57 2015, Steve.Hay@verosoftware.com wrote: Show quoted text
> Since I'm not keen on the static CRT change, what are the other > options that you see here?
Can you send me your ucrtbase.dll files offlist, 32 and 64 bit ones. Also are you building 64 or 32 bit perl with 2015? As a side note, here is an interesting link http://blogs.msdn.com/b/oldnewthing/archive/2014/04/11/10516280.aspx . Here is a ruby ticket with MS about pioinfo https://connect.microsoft.com/VisualStudio/feedback/details/1279133 -- bulk88 ~ bulk88 at hotmail.com
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.8k
On Thu Jul 30 11:10:57 2015, Steve.Hay@verosoftware.com wrote: Show quoted text
> Done in the v3 patch attached, but I'm not very keen on this. Perl has > always used the CRT DLL, which makes me very uneasy about switching to > LIBC. It does get it building, though, but now crashes on startup > trying to set environ[0] to NULL in S_init_postdump_symbols(). > > I briefly tried #defining NO_ENVIRON_ARRAY to temporarily duck that > issue but the build failed because Perl_my_setenv() is then not > provided by util.c, but is apparently still required by mg.c (at > least)... :-/
The environ deref NULL SEGV is fixed by the attached patch, which goes ontop of your patch "Add VS2015 support, v3", you can rebase squash my patch with yours once you fine with it. Basically either all 3 .libs are the static ones, or the DLL ones. No mixing allowed. The actual crash was environ var in static CRT was uninitialized because this function from vcruntime wasn't running when initterm_e when through its array of C-MS-extensions/C++ global initializer functions. ---------------------------------------------------------------- _CRTALLOC(".CRT$XIAA") static _PIFV pre_c_initializer = pre_c_initialization; ************cut************ static int __cdecl pre_c_initialization() throw() { _set_app_type(main_policy::get_app_type()); ************cut************ initialize_environment(); __scrt_initialize_winrt(); return 0; } ---------------------------------------------------------------- With my patch, there are no assert fails with DEBUGGING (I think, I can a couple tests manually they were find), but harness is very broken. I am using the 32 bit RC VC 2015 from April 2015, so try it on your machine and see if you are getting this ------------------------------------------------------------------------------- perl.exe harness base/cond.t ....................................................... No subtests run base/if.t ......................................................... No subtests run base/lex.t ........................................................ No subtests run base/num.t ........................................................ No subtests run base/pat.t ........................................................ No subtests run base/rs.t ......................................................... No subtests run base/term.t ....................................................... No subtests ------------------------------------------------------------------------------- IDK where the problem is since backticks work ----------------------------------------------------------- C:\p521\src>perl -MData::Dumper -E"say Dumper(scalar(`$^X -E\"say 'hi'\"`))" $VAR1 = 'hi '; ----------------------------------------------------------- I am going to now work on getting the DLL UCRT working instead of this static CRT build. -- bulk88 ~ bulk88 at hotmail.com
Subject: 0001-more-VC-2015-fixes.patch
From 864232c88154b9b66adcfdb739e1824501b68b21 Mon Sep 17 00:00:00 2001 From: bulk88 <bulk88@hotmail.com> Date: Sat, 1 Aug 2015 04:18:26 -0400 Subject: [PATCH] more VC 2015 fixes -the msize test in Perl_win32_init was flawed, it was doing _msize(malloc(x)) == x, which will always work, instead test the size of the opaque (starting in VC 2015) FILE * -fix the ioinfo struct, CRITICAL_SECTION lock and intptr_t osfhnd were reordered by MS and osfhnd isn't the first member anymore, WIN32_DYN_IOINFO_SIZE assumes first member is always osfhnd, and would need extra work to incluse the size of CRITICAL_SECTION lock in its calculation of the location of osfhnd, so for now don't use WIN32_DYN_IOINFO_SIZE on 2015, io_sock.t was failing asserts in my_close which showed the definition of the ioinfo struct was wrong -IOINFO_L2E changed in VC 2015, this broke the WIN32_DYN_IOINFO_SIZE code which assumed the struct size will change, but the elements per array of structs will remain a constant --- win32/Makefile | 7 ++++++- win32/win32.c | 6 +++--- win32/win32.h | 39 +++++++++++++++++++++++++++++++++++++-- 3 files changed, 46 insertions(+), 6 deletions(-) diff --git a/win32/Makefile b/win32/Makefile index 963d560..654a8b1 100644 --- a/win32/Makefile +++ b/win32/Makefile @@ -546,8 +546,13 @@ LIBBASEFILES = \ netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib \ version.lib odbc32.lib odbccp32.lib comctl32.lib +# See note about ucrt.lib/libucrt.lib above !IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" -LIBBASEFILES = $(LIBBASEFILES) msvcrt.lib vcruntime.lib +! IF "$(CFG)" == "DebugFull" +LIBBASEFILES = $(LIBBASEFILES) libcmtd.lib libvcruntimed.lib +! ELSE +LIBBASEFILES = $(LIBBASEFILES) libcmt.lib libvcruntime.lib +! ENDIF !ENDIF # Avoid __intel_new_proc_init link error for libircmt. diff --git a/win32/win32.c b/win32/win32.c index 3c7c8e7..78f78ed 100644 --- a/win32/win32.c +++ b/win32/win32.c @@ -4445,17 +4445,17 @@ Perl_win32_init(int *argcp, char ***argvp) #if defined(DEBUGGING) && defined(_MSC_VER) && _MSC_VER >= 1900 { /* Sanity check on the size of our __crt_stdio_stream_data struct. */ - FILE* f = (FILE*)malloc(sizeof(FILE)); + FILE* f= fopen(w32_module_name, "r"); if (f != NULL) { __crt_stdio_stream_data* data = (__crt_stdio_stream_data*)f; Size_t data_size = _msize((void*)data); if ((SSize_t)data_size <= 0 || /* -1 is err */ - data_size != sizeof(FILE)) { + data_size != sizeof(__crt_stdio_stream_data)) { /* no interp */ fprintf(stderr, "panic: invalid __crt_stdio_stream_data size\n"); exit(1); } - free(f); + fclose(f); } } #endif diff --git a/win32/win32.h b/win32/win32.h index 1804c10..c539c0f 100644 --- a/win32/win32.h +++ b/win32/win32.h @@ -601,16 +601,31 @@ void win32_wait_for_children(pTHX); * VC uses a fixed ioinfo size. */ #if ! (_MSC_VER < 1400 || (_MSC_VER >= 1500 && _MSC_VER <= 1700) \ - || defined(__MINGW32__)) + || _MSC_VER >= 1900 || defined(__MINGW32__)) /* size of ioinfo struct is determined at runtime */ # define WIN32_DYN_IOINFO_SIZE #endif #ifndef WIN32_DYN_IOINFO_SIZE + +#if _MSC_VER >= 1900 +/* enum class __crt_lowio_text_mode : char +{ + ansi = 0, // Regular text + utf8 = 1, // UTF-8 encoded + utf16le = 2, // UTF-16LE encoded +}; */ + +typedef char __crt_lowio_text_mode; + +typedef char __crt_lowio_pipe_lookahead[3]; +#endif + /* * Control structure for lowio file handles */ typedef struct { +#if defined(__MINGW32__) || _MSC_VER < 1900 intptr_t osfhnd;/* underlying OS file HANDLE */ char osfile; /* attributes of file (e.g., open in text mode?) */ char pipech; /* one char buffer for handles opened on pipes */ @@ -636,6 +651,21 @@ typedef struct { BOOL dbcsBufferUsed; /* Bool for the lead byte buffer is used or not */ # endif # endif + +#else /* >= VC 2015 definition */ + + CRITICAL_SECTION lock; + intptr_t osfhnd; // underlying OS file HANDLE + __int64 startpos; // File position that matches buffer start + unsigned char osfile; // Attributes of file (e.g., open in text mode?) + __crt_lowio_text_mode textmode; + __crt_lowio_pipe_lookahead _pipe_lookahead; + + unsigned char unicode : 1; // Was the file opened as unicode? + unsigned char utf8translations : 1; // Buffer contains translations other than CRLF + unsigned char dbcsBufferUsed : 1; // Is the dbcsBuffer in use? + char dbcsBuffer; // Buffer for the lead byte of DBCS when converting from DBCS to Unicode +#endif } ioinfo; #else typedef intptr_t ioinfo; @@ -650,7 +680,12 @@ EXTERN_C _CRTIMP ioinfo* __pioinfo[]; * Definition of IOINFO_L2E, the log base 2 of the number of elements in each * array of ioinfo structs. */ -#define IOINFO_L2E 5 + +#if _MSC_VER >= 1900 +# define IOINFO_L2E 6 +#else +# define IOINFO_L2E 5 +#endif /* * Definition of IOINFO_ARRAY_ELTS, the number of elements in ioinfo array -- 1.9.5.msysgit.1
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.5k
On Thu Jul 30 11:10:57 2015, Steve.Hay@verosoftware.com wrote: Show quoted text
> Since I'm not keen on the static CRT change, what are the other > options that you see here?
There are more options/choices than the patch I've attached, but this patch was the easiest to create of all the choices. On Sat Aug 01 01:25:57 2015, bulk88 wrote: Show quoted text
> I am going to now work on getting the DLL UCRT working instead of this > static CRT build.
I've attached a patch for the easiest solution to no more __pioinfo export. The patch only works on 32 bit perl builds with ucrtbase.dll, 64 bits and any ucrtbased.dll are not supported ATM. Smoking the perl gave me Test Summary Report ------------------- ../cpan/IPC-Cmd/t/01_IPC-Cmd.t (Wstat: 768 Test s: 459 Failed: 3) Failed tests: 38-39, 64 Non-zero exit status: 3 ../cpan/parent/t/parent-pmc.t (Wstat: 768 Test s: 3 Failed: 3) Failed tests: 1-3 Non-zero exit status: 3 ../dist/Carp/t/errno.t (Wstat: 1024 Tes ts: 20 Failed: 4) Failed tests: 9, 12, 16, 20 Non-zero exit status: 4 Files=2391, Tests=710888, 10548 wallclock secs (76.30 usr + 4.90 sys = 81.20 CP U) Result: FAIL NMAKE : fatal error U1077: '.\perl.exe' : return code '0xa' Stop. ../cpan/parent/t/parent-pmc.t can be ignored ../dist/Carp/t/errno.t failure is unexpected, maybe it is because of my old RC VC 2015 ---------------------------------------------------------------------- C:\p521\src\t>perl harness -v ../dist/Carp/t/errno.t ../dist/Carp/t/errno.t .. 1..20 ok 1 ok 2 ok 3 ok 4 ok 5 ok 6 ok 7 ok 8 not ok 9 # Failed test at t/errno.t line 38. # got: '0' # expected: '69' ok 10 ok 11 not ok 12# Failed test at t/errno.t line 46. # got: '0' # expected: '69' ok 13 ok 14 ok 15 # Failed test at t/errno.t line 58. not ok 16 # got: '0' # expected: '69' ok 17 ok 18 ok 19 not ok 20 # Failed test at t/errno.t line 67. # got: '0' # expected: '69' # Looks like you failed 4 tests of 20. Dubious, test returned 4 (wstat 1024, 0x400) Failed 4/20 subtests Test Summary Report ------------------- ../dist/Carp/t/errno.t (Wstat: 1024 Tests: 20 Failed: 4) Failed tests: 9, 12, 16, 20 Non-zero exit status: 4 Files=1, Tests=20, 0 wallclock secs ( 0.02 usr + 0.00 sys = 0.02 CPU) Result: FAIL C:\p521\src\t> --------------------------------------------------------------- from reading errno.t, it looks like something new is calling SetLastError somewhere. Ill have to set some BPs and find out tomorrow. -- bulk88 ~ bulk88 at hotmail.com
Subject: 0001-export-__pioinfo-by-force.patch
From 3ecfddf78e3b2fe71e7e0852d403ed9885b263c4 Mon Sep 17 00:00:00 2001 From: bulk88 <bulk88@hotmail.com> Date: Sun, 2 Aug 2015 03:43:30 -0400 Subject: [PATCH] export __pioinfo by force --- win32/Makefile | 8 ++-- win32/win32.c | 130 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ win32/win32.h | 6 +++ 3 files changed, 140 insertions(+), 4 deletions(-) diff --git a/win32/Makefile b/win32/Makefile index 654a8b1..36d713e 100644 --- a/win32/Makefile +++ b/win32/Makefile @@ -453,9 +453,9 @@ CXX_FLAG = -TP -EHsc !IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" !IF "$(CFG)" == "DebugFull" -LIBC = libucrtd.lib +LIBC = ucrtd.lib !ELSE -LIBC = libucrt.lib +LIBC = ucrt.lib !ENDIF LINK_FLAG = @@ -549,9 +549,9 @@ LIBBASEFILES = \ # See note about ucrt.lib/libucrt.lib above !IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE" ! IF "$(CFG)" == "DebugFull" -LIBBASEFILES = $(LIBBASEFILES) libcmtd.lib libvcruntimed.lib +LIBBASEFILES = $(LIBBASEFILES) msvcrtd.lib vcruntimed.lib ! ELSE -LIBBASEFILES = $(LIBBASEFILES) libcmt.lib libvcruntime.lib +LIBBASEFILES = $(LIBBASEFILES) msvcrt.lib vcruntime.lib ! ENDIF !ENDIF diff --git a/win32/win32.c b/win32/win32.c index 78f78ed..48723bf 100644 --- a/win32/win32.c +++ b/win32/win32.c @@ -163,6 +163,9 @@ char w32_module_name[MAX_PATH+1]; #ifdef WIN32_DYN_IOINFO_SIZE Size_t w32_ioinfo_size;/* avoid 0 extend op b4 mul, otherwise could be a U8 */ #endif +#if _MSC_VER >= 1900 +ioinfo** __pioinfo = NULL; +#endif END_EXTERN_C static OSVERSIONINFO g_osver = {0, 0, 0, 0, 0, ""}; @@ -4396,6 +4399,61 @@ ansify_path(void) } win32_free(wide_path); } +/* this function is written using no C strings, and no other functions or other + * foreign symbols, so it is relocation free, and can be stored and moved + * around in any string */ +static ioinfo** +Perl_pioinfo_loader(unsigned char * func) { + ioinfo** pioinfo; +/* bulk88's ucrtbase.dll 10.0.10046.0, the match code also works for steveh's + * 10.0.10150.0 + * + * note by matching "sar eax,6" aka IOINFO_L2E and "imul ecx,ecx,30h" aka + * sizeof(ioinfo), we are enforcing that these 2 constants on perl side + * remain the same as in ucrtbase.dll which may one day get changed out from + * under us +__get_osfhandle: +777194A0 8B FF mov edi,edi +777194A2 55 push ebp +777194A3 8B EC mov ebp,esp +777194A5 51 push ecx +777194A6 8B 4D 08 mov ecx,dword ptr [ebp+8] +777194A9 83 F9 FE cmp ecx,0FFFFFFFEh +777194AC 75 15 jne __get_osfhandle+23h (777194C3h) +777194AE E8 1D 1B 00 00 call ___doserrno (7771AFD0h) +777194B3 83 20 00 and dword ptr [eax],0 +777194B6 E8 B5 1B 00 00 call __errno (7771B070h) +777194BB C7 00 09 00 00 00 mov dword ptr [eax],9 +777194C1 EB 43 jmp __get_osfhandle+66h (77719506h) +777194C3 85 C9 test ecx,ecx +777194C5 78 27 js __get_osfhandle+4Eh (777194EEh) +777194C7 3B 0D 28 B4 77 77 cmp ecx,dword ptr ds:[7777B428h] +777194CD 73 1F jae __get_osfhandle+4Eh (777194EEh) +777194CF 8B C1 mov eax,ecx +777194D1 83 E1 3F and ecx,3Fh +777194D4 C1 F8 06 sar eax,6 +777194D7 6B C9 30 imul ecx,ecx,30h +777194DA 8B 04 85 30 B4 77 77 mov eax,dword ptr [eax*4+7777B430h] +*/ + if(*(U32*)func == '\x8B\xFF\x55\x8B' && *(U32*)(func+=4) == '\xEC\x51\x8B\x4D' && + *(U32*)(func+=4) == '\x08\x83\xF9\xFE' && *(U8*)(func+=4) == '\x75') { + func += *(U8*)(func+=1)+1; /* get jump offset from jne ins, no point in testing call + * instructions in the failure branch */ + if(*(U32*)func == '\x85\xC9\x78\x27' && *(U16*)(func+=4) == '\x3B\x0D' && + /* 4 byte hole for address which is subject to relocation and can't be matched, although + * technically the low 2 bytes "28 B4" can be matched since Win ASLR/reloc is always + * in units of 65K but low 2 bytes will change in practically every recompile of + * ucrtbase, even with zero code changes to get_osfhandle so dont match the low 2 bytes + * so this match pattern works longer before needing to be updated again */ + *(U32*)(func+=6) == '\x73\x1F\x8B\xC1' && *(U32*)(func+=4) == '\x83\xE1\x3F\xC1' && + *(U32*)(func+=4) == '\xF8\x06\x6B\xC9' && *(U32*)(func+=4) == '\x30\x8B\x04\x85' ) { + pioinfo = (ioinfo**)*(void **)(func+=4); + return pioinfo; + } + } + /* add ucrtbased.dll */ + return NULL; +} void Perl_win32_init(int *argcp, char ***argvp) @@ -4442,6 +4500,78 @@ Perl_win32_init(int *argcp, char ***argvp) } #endif +#if _MSC_VER >= 1900 + { +/* http://code.google.com/p/chromium/issues/detail?id=330435 making a rwx + section in PE file doesn't work*/ +//#pragma section("text_rwx", read, write, execute) + + ///__declspec(allocate("text_rwx")) static char loader [4096]; + char loader [4096]; + ioinfo** (*loader_func)(unsigned char * func); + DWORD old_protect; + DWORD loaderlen = GetEnvironmentVariable("PIOINFO_LOADER", loader, sizeof(loader)); + /* convert hex string to binary */ + if(loaderlen && loaderlen < sizeof(loader)-1 && loaderlen % 2 == 0) { + unsigned int i = 0; + char * loader_p = loader; + do { + char c_hi = loader[i]; + if(isXDIGIT(c_hi)){ + char c_low; + loader[i/2] = XDIGIT_VALUE(c_hi) << 4; + c_low = loader[++i]; + if(isXDIGIT(c_low)) + loader[i/2] |= XDIGIT_VALUE(c_low); + else + goto noloader; + } + else + goto noloader; + } while (++i < loaderlen); +/* make a piece of C stack executable, otherwise DEP will SEGV the process */ + if (!VirtualProtect(loader, + sizeof(loader), + PAGE_EXECUTE_READWRITE, + &old_protect)) + goto pioinfo_loader_failed; + loader_func = (ioinfo** (*)(unsigned char * func))loader; + } else { + noloader: + loader_func = Perl_pioinfo_loader; + } + /* FF 25 is a jmp instruction/thunk, it is optional (and less efficient) for the CC + * to do this vs directly using the target function, but it is always done in -Od mode + * probably for VC's "edit & continue" feature, remove the thunk and get real func ptr + */ + { + unsigned char * f = (unsigned char *)_get_osfhandle; + if(*(unsigned short *)f == '\xFF\x25') { + f = *(unsigned char**)(f+2); /* get address of IAT entry from ins's operand */ + f = *(unsigned char**)f; /* read func pointer in IAT entry */ + } + __pioinfo = loader_func(f); + if(!__pioinfo) { + pioinfo_loader_failed: + fprintf(stderr, "panic: pioinfo loader failed\n"); /* no interp */ + exit(1); + } +/* return piece of C stack back to non-executable (the way we found it) for security/cleanlyness*/ + if(loader_func != Perl_pioinfo_loader + && !VirtualProtect(loader, + sizeof(loader), + old_protect, + &old_protect)) + goto pioinfo_loader_failed; + //{ + // HMODULE dll = LoadLibrary("C:\\Users\\Owner\\Desktop\\ucrtsteveh\\x86\\ucrtbase.dll"); + // void * func = GetProcAddress(dll, "_get_osfhandle"); + // void * stevepioinfo = Perl_pioinfo_loader(func); + //} + } + } +#endif + #if defined(DEBUGGING) && defined(_MSC_VER) && _MSC_VER >= 1900 { /* Sanity check on the size of our __crt_stdio_stream_data struct. */ diff --git a/win32/win32.h b/win32/win32.h index c539c0f..8d05b38 100644 --- a/win32/win32.h +++ b/win32/win32.h @@ -674,7 +674,13 @@ typedef intptr_t ioinfo; /* * Array of arrays of control structures for lowio files. */ + +#if _MSC_VER < 1900 EXTERN_C _CRTIMP ioinfo* __pioinfo[]; +#else +/* we obtain it, it is not exported anymore from the UCRT dll */ +ioinfo** __pioinfo; +#endif /* * Definition of IOINFO_L2E, the log base 2 of the number of elements in each -- 1.9.5.msysgit.1
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 6.3k
On Sat Aug 01 01:25:57 2015, bulk88 wrote: Show quoted text
> On Thu Jul 30 11:10:57 2015, Steve.Hay@verosoftware.com wrote:
> > Done in the v3 patch attached, but I'm not very keen on this. Perl > > has > > always used the CRT DLL, which makes me very uneasy about switching > > to > > LIBC. It does get it building, though, but now crashes on startup > > trying to set environ[0] to NULL in S_init_postdump_symbols(). > > > > I briefly tried #defining NO_ENVIRON_ARRAY to temporarily duck that > > issue but the build failed because Perl_my_setenv() is then not > > provided by util.c, but is apparently still required by mg.c (at > > least)... :-/
> > > The environ deref NULL SEGV is fixed by the attached patch, which goes > ontop of your patch "Add VS2015 support, v3", you can rebase squash my > patch with yours once you fine with it. > > Basically either all 3 .libs are the static ones, or the DLL ones. No > mixing allowed. > > The actual crash was environ var in static CRT was uninitialized > because this function from vcruntime wasn't running when initterm_e > when through its array of C-MS-extensions/C++ global initializer > functions. > > ---------------------------------------------------------------- > _CRTALLOC(".CRT$XIAA") static _PIFV pre_c_initializer = > pre_c_initialization; > > ************cut************ > static int __cdecl pre_c_initialization() throw() > { > _set_app_type(main_policy::get_app_type()); > > ************cut************ > initialize_environment(); > > __scrt_initialize_winrt(); > > return 0; > } > ---------------------------------------------------------------- > > With my patch, there are no assert fails with DEBUGGING (I think, I > can a couple tests manually they were find), but harness is very > broken. I am using the 32 bit RC VC 2015 from April 2015, so try it on > your machine and see if you are getting this > > ------------------------------------------------------------------------------- > perl.exe harness > base/cond.t ....................................................... No > subtests > run > base/if.t ......................................................... No > subtests > run > base/lex.t ........................................................ No > subtests > run > base/num.t ........................................................ No > subtests > run > base/pat.t ........................................................ No > subtests > run > base/rs.t ......................................................... No > subtests > run > base/term.t ....................................................... No > subtests > ------------------------------------------------------------------------------- > > IDK where the problem is since backticks work > ----------------------------------------------------------- > C:\p521\src>perl -MData::Dumper -E"say Dumper(scalar(`$^X -E\"say > 'hi'\"`))" > $VAR1 = 'hi > '; > ----------------------------------------------------------- > > I am going to now work on getting the DLL UCRT working instead of this > static CRT build.
Thanks for the environ fix and your other changes. I've squashed them together with mine in the attached v4 patch. With this, I got a bunch of test failures, very different from your report, and most not reproducible when I re-ran them. But three are reproducible failures: C:\Dev\Git\perl\t>.\perl harness run/locale.t run/locale.t .. 3/37 # Failed test 8 - localeconv() looks at LC_NUMERIC with and without 'use locale' at ./test.pl line 1027 # got ".." # expected ",," # PROG: # print localeconv()->{decimal_point}; # use POSIX; # use locale; # print localeconv()->{decimal_point}; # STATUS: 0 run/locale.t .. Failed 1/37 subtests (less 6 skipped subtests: 30 okay) Test Summary Report ------------------- run/locale.t (Wstat: 0 Tests: 37 Failed: 1) Failed test: 8 Files=1, Tests=37, 4 wallclock secs ( 0.05 usr + 0.00 sys = 0.05 CPU) Result: FAIL C:\Dev\Git\perl\t>.\perl harness ../ext/PerlIO-scalar/t/scalar.t ../ext/PerlIO-scalar/t/scalar.t .. 1/122 # Failed test 'check $! is updated' # at t/scalar.t line 406. # got: '0' # expected: '22' # Failed test 'check $! is updated even when we warn' # at t/scalar.t line 411. # got: '0' # expected: '22' # Failed test 'check errno set correctly' # at t/scalar.t line 444. # got: '0' # expected: '22' # Failed test 'check errno set' # at t/scalar.t line 466. # got: '0' # expected: '22' # Looks like you failed 4 tests of 122. ../ext/PerlIO-scalar/t/scalar.t .. Dubious, test returned 4 (wstat 1024, 0x400) Failed 4/122 subtests Test Summary Report ------------------- ../ext/PerlIO-scalar/t/scalar.t (Wstat: 1024 Tests: 122 Failed: 4) Failed tests: 86, 89, 100, 107 Non-zero exit status: 4 Files=1, Tests=122, 0 wallclock secs ( 0.02 usr + 0.02 sys = 0.03 CPU) Result: FAIL C:\Dev\Git\perl\t>.\perl harness ../ext/XS-Typemap/t/Typemap.t ../ext/XS-Typemap/t/Typemap.t .. 1/156 # Failed test at t/Typemap.t line 367. Label not found for "last SKIP" at ../../lib/Test/More.pm line 1314. # Looks like you planned 156 tests but ran 134. # Looks like you failed 1 test of 134 run. # Looks like your test exited with 9 just after 134. ../ext/XS-Typemap/t/Typemap.t .. Dubious, test returned 9 (wstat 2304, 0x900) Failed 23/156 subtests Test Summary Report ------------------- ../ext/XS-Typemap/t/Typemap.t (Wstat: 2304 Tests: 134 Failed: 1) Failed test: 134 Non-zero exit status: 9 Parse errors: Bad plan. You planned 156 tests but ran 134. Files=1, Tests=134, 0 wallclock secs ( 0.03 usr + 0.00 sys = 0.03 CPU) Result: FAIL I will look at these in more detail soon, but I'm inclined to commit what we've got so far so we've got something definite to base further efforts on, and so that other people trying VS2015 won't fail at the first hurdle and waste time duplicating our work here. I also want to return to the other comments that you made which I said I'd come back to, and I haven't tried out your solution to avoid using the static CRT libs yet... I had a brief look, and yes it is enough to scare the children! ;-) It makes it look like perl is doing something really naughty to have to go to such lengths. I think it would be worth submitting your other attachments (from https://rt.perl.org/Ticket/Display.html?id=125714#txn-1358960) regarding that to MS as a bug report to see if there is some better way of doing this. Do you want to do that or shall I?
Subject: 0001-Add-VS2015-support-v4.patch

Message body is not shown because it is too large.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 8.9k
On Tue Aug 04 01:07:15 2015, shay wrote: Show quoted text
> Thanks for the environ fix and your other changes. I've squashed them > together with mine in the attached v4 patch. > > With this, I got a bunch of test failures, very different from your > report, and most not reproducible when I re-ran them. But three are > reproducible failures: > > C:\Dev\Git\perl\t>.\perl harness run/locale.t > run/locale.t .. 3/37 # Failed test 8 - localeconv() looks at > LC_NUMERIC with and without 'use locale' at ./test.pl line 1027 > # got ".." > # expected ",," > # PROG: > # print localeconv()->{decimal_point}; > # use POSIX; > # use locale; > # print localeconv()->{decimal_point}; > # STATUS: 0 > run/locale.t .. Failed 1/37 subtests > (less 6 skipped subtests: 30 okay) > > Test Summary Report > ------------------- > run/locale.t (Wstat: 0 Tests: 37 Failed: 1) > Failed test: 8 > Files=1, Tests=37, 4 wallclock secs ( 0.05 usr + 0.00 sys = 0.05 > CPU) > Result: FAIL > > C:\Dev\Git\perl\t>.\perl harness ../ext/PerlIO-scalar/t/scalar.t > ../ext/PerlIO-scalar/t/scalar.t .. 1/122 > # Failed test 'check $! is updated' > # at t/scalar.t line 406. > # got: '0' > # expected: '22' > > # Failed test 'check $! is updated even when we warn' > # at t/scalar.t line 411. > # got: '0' > # expected: '22' > > # Failed test 'check errno set correctly' > # at t/scalar.t line 444. > # got: '0' > # expected: '22' > > # Failed test 'check errno set' > # at t/scalar.t line 466. > # got: '0' > # expected: '22' > # Looks like you failed 4 tests of 122. > ../ext/PerlIO-scalar/t/scalar.t .. Dubious, test returned 4 (wstat > 1024, 0x400) > Failed 4/122 subtests > > Test Summary Report > ------------------- > ../ext/PerlIO-scalar/t/scalar.t (Wstat: 1024 Tests: 122 Failed: 4) > Failed tests: 86, 89, 100, 107 > Non-zero exit status: 4 > Files=1, Tests=122, 0 wallclock secs ( 0.02 usr + 0.02 sys = 0.03 > CPU) > Result: FAIL
In my RC 2015 build, the ../dist/Carp/t/errno.t failures I am seeing is because there is an unprotect by save/restoring GLR FlsGetValue call from Perl_my_snprintf from _LocaleUpdate inside the CRT. Maybe this case was fixed by MS for gold release, but other cases of FlsGetValue are around in the UCRT. KernelBase.dll!_FlsGetValue@4() Unknown ucrtbase.dll!___acrt_FlsGetValue@4() Unknown ucrtbase.dll!___acrt_update_locale_info() Unknown ucrtbase.dll!_LocaleUpdate::_LocaleUpdate() Unknown ucrtbase.dll!common_vsprintf<__crt_stdio_output::standard_base,char>() Unknown ucrtbase.dll!___stdio_common_vsprintf() Unknown perl523.dll!_vsnprintf_l(char * const _Buffer, const unsigned int _BufferCount, const char * const _Format, __crt_locale_pointers * const _ArgList, char *) Line 1390 C perl523.dll!Perl_my_snprintf(char * buffer, const unsigned int len, const char * format, ...) Line 5137 C perl523.dll!Perl_save_re_context(interpreter * my_perl) Line 17781 C perl523.dll!S_invoke_exception_hook(interpreter * my_perl, sv * ex, bool warn) Line 1552 C perl523.dll!Perl_warn_sv(interpreter * my_perl, sv * baseex) Line 1834 C perl523.dll!Perl_pp_warn(interpreter * my_perl) Line 462 C perl523.dll!Perl_runops_standard(interpreter * my_perl) Line 41 C perl523.dll!S_run_body(interpreter * my_perl, long oldscope) Line 2448 C perl523.dll!perl_run(interpreter * my_perl) Line 2371 C perl523.dll!RunPerl(int argc, char * * argv, char * * env) Line 258 C++ [External Code] Show quoted text
> C:\Dev\Git\perl\t>.\perl harness ../ext/XS-Typemap/t/Typemap.t > ../ext/XS-Typemap/t/Typemap.t .. 1/156 > # Failed test at t/Typemap.t line 367. > Label not found for "last SKIP" at ../../lib/Test/More.pm line 1314. > # Looks like you planned 156 tests but ran 134. > # Looks like you failed 1 test of 134 run. > # Looks like your test exited with 9 just after 134. > ../ext/XS-Typemap/t/Typemap.t .. Dubious, test returned 9 (wstat 2304, > 0x900) > Failed 23/156 subtests > > Test Summary Report > ------------------- > ../ext/XS-Typemap/t/Typemap.t (Wstat: 2304 Tests: 134 Failed: 1) > Failed test: 134 > Non-zero exit status: 9 > Parse errors: Bad plan. You planned 156 tests but ran 134. > Files=1, Tests=134, 0 wallclock secs ( 0.03 usr + 0.00 sys = 0.03 > CPU) > Result: FAIL
Line 367 is --------------------------------------------------- # T_STDIO note("T_STDIO"); # open a file in XS for write my $testfile= "stdio.tmp"; # not everything below cleans up END { 1 while unlink $testfile; } my $fh = T_STDIO_open( $testfile ); ok( $fh );<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< # write to it using perl if (defined $fh) { --------------------------------------------------- Now everyone remember, Win32 Perl has NEVER been compiled with static CRT before. For any VC version. I should try a VC 2003 static CRT build and see how many problems I get. In my DLL CRT build with the pioinfo patch, in Typemap.dll _Perl_newXS_deffile(my_perl, "XS::Typemap::T_STDIO_open", XS_XS__Typemap_T_STDIO_open); and inside XS_XS__Typemap_T_STDIO_open I see v8 = v6->sv_u.sv_flags; if ( (v8 & 0x200400) == 1024 ) v9 = v6->sv_u.svu_pv; else v9 = (const char *)_Perl_sv_2pv_flags(my_perl, v6, 0, 2); v12 = _fopen(v9, "w"); v15 = _Perl_sv_newmortal(my_perl); v11 = _Perl_newGVgen_flags(v3, "XS::Typemap", 0); v10 = _PerlIO_importFILE(v12, 0); the _fopen symbol is imported from api-ms-win-crt-stdio-l1-1-0.dll which is really ucrtbase.dll. In a static CRT the fds are isolated to each DLL. Win32 perl has xsub.h and all its c std lib hooking macros that redirect to win32_* functions. In DLL CRT perl win32_* are jump stubs to CRT DLL functions, in static CRT perl the win32_* goto the internal static linked CRT in perl523.dll. perl.exe's copy of the CRT seems to play almost no role in perl interp, since the whole interp lives in perl523.dll and i think perl.exe's CRT irrelevant. In this case, XS-Typemap intentionally bypasses xsub.h in its testing ---------------------------------------------------------------------------- XS_EUPXS(XS_XS__Typemap_T_STDIO_open); /* prototype to pass -Wmissing-prototypes */ XS_EUPXS(XS_XS__Typemap_T_STDIO_open) { dVAR; dXSARGS; if (items != 1) croak_xs_usage(cv, "file"); { const char * file = (const char *)SvPV_nolen(ST(0)) ; FILE * RETVAL; #line 905 "Typemap.xs" RETVAL = xsfopen( file );<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< #line 1694 "Typemap.c" { SV * RETVALSV; RETVALSV = sv_newmortal(); { GV *gv = newGVgen("XS::Typemap"); PerlIO *fp = PerlIO_importFILE(RETVAL,0); if ( fp && do_open(gv, "+<&", 3, FALSE, 0, 0, fp) ) { SV *rv = newRV_inc((SV*)gv); rv = sv_bless(rv, GvSTASH(gv)); RETVALSV = sv_2mortal(rv); } } ST(0) = RETVALSV; } } XSRETURN(1); } --------------------------------------------------------------------- http://perl5.git.perl.org/perl.git/blob/HEAD:/ext/XS-Typemap/stdio.c --------------------------------------------------------------------- /* This provides a test of STDIO and emulates a library that has been built outside of the PerlIO system and therefore is built using FILE* rather than PerlIO * (a common occurrence for XS). Use a separate file to make sure we are not contaminated by PerlIO. */ #include <stdio.h> /* Open a file for write */ FILE * xsfopen ( const char * path ) { FILE * stream; stream = fopen( path, "w"); return stream; } --------------------------------------------------------------------------- IDK about making the test use win32_fopen instead of CRT fopen since that is violating the spirit of XS-Typemap's tests. Maybe a skip if $Config{win32_static_CRT} in ../ext/XS-Typemap/t/Typemap.t around that test? Show quoted text
> I will look at these in more detail soon, but I'm inclined to commit > what we've got so far so we've got something definite to base further > efforts on, and so that other people trying VS2015 won't fail at the > first hurdle and waste time duplicating our work here.
Other people? non-C coders building win32 perl from source or TonyC/JDB? I'd add a warning to the 2 makefiles at @@ -126,6 +126,10 @@ CCTYPE = MSVC60 #CCTYPE = MSVC120 # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) #CCTYPE = MSVC120FREE +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) +#CCTYPE = MSVC140 +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) +#CCTYPE = MSVC140FREE stating "work in progress" or "not supported" and an echo in .\config.h : $(CFGH_TMPL) -del /f config.h copy $(CFGH_TMPL) config.h @echo.>>$@ or on another VC 2015 (through ifdefs) specific target an echo warning saying "your VC 2015 is experimental". Can't have random people thinking it works 100%. I would also rename LINK_FLAG to LIBC_FLAG or CRT_FLAG or RTL_FLAG (is it the CRT or RTL MS? make up your mind). LINK_FLAG is too generic of a term. There is already a macro called LINK_FLAGS. LINK_FLAG vs LINK_FLAGS is confusing. Also LINK_FLAG is being passed to cl.exe, not link.exe, which is more confusing. -- bulk88 ~ bulk88 at hotmail.com
CC: "perl5-porters [...] perl.org" <perl5-porters [...] perl.org>
Subject: RE: [perl #125714] Adding support for VS2015 (VC14)
To: "perlbug-followup [...] perl.org" <perlbug-followup [...] perl.org>
From: Steve Hay <Steve.Hay [...] verosoftware.com>
Date: Wed, 5 Aug 2015 08:24:52 +0000
Download (untitled) / with headers
text/plain 2.3k
On 5 August 2015 at 05:06, bulk88 via RT <perlbug-followup@perl.org> wrote: Show quoted text
> Now everyone remember, Win32 Perl has NEVER been compiled with static CRT before. For any VC version. I should try a VC 2003 static CRT build and see how many problems I get. >
Good point. I should have thought of that myself: I've now tried VS2010 using /MT for the static libcmt.lib and I get the *exact* same pattern of test failures, so the failures are surely due to using the static CRT, not due to changes in VS2015. They're therefore probably not worth pursuing any further right now since I hope we can revert to the dynamic CRT once we're settled on the best (least scariest?!) way to do that, possibly after quizzing MS on our problems. Show quoted text
>> I will look at these in more detail soon, but I'm inclined to commit >> what we've got so far so we've got something definite to base further >> efforts on, and so that other people trying VS2015 won't fail at the >> first hurdle and waste time duplicating our work here.
> > Other people? non-C coders building win32 perl from source or TonyC/JDB? > > I'd add a warning to the 2 makefiles at > > @@ -126,6 +126,10 @@ CCTYPE = MSVC60 > #CCTYPE = MSVC120 > # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free version) > #CCTYPE = MSVC120FREE > +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) > +#CCTYPE = MSVC140 > +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free version) > +#CCTYPE = MSVC140FREE > > stating "work in progress" or "not supported" and an echo in > > .\config.h : $(CFGH_TMPL) > -del /f config.h > copy $(CFGH_TMPL) config.h > @echo.>>$@ > > or on another VC 2015 (through ifdefs) specific target an echo warning saying "your VC 2015 is experimental". Can't have random people thinking it works 100%.
Yes, I've added suitable warnings into the makefiles and the README.win32. I will test a few more configurations today and hopefully commit later. Show quoted text
> > I would also rename LINK_FLAG to LIBC_FLAG or CRT_FLAG or RTL_FLAG (is it the CRT or RTL MS? make up your mind). LINK_FLAG is too generic of a term. There is already a macro called LINK_FLAGS. LINK_FLAG vs LINK_FLAGS is confusing. Also LINK_FLAG is being passed to cl.exe, not link.exe, which is more confusing. >
Agreed. Renamed to LIBC_FLAG.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.8k
On Wed Aug 05 01:25:36 2015, Steve.Hay@verosoftware.com wrote: Show quoted text
> >> I will look at these in more detail soon, but I'm inclined to commit > >> what we've got so far so we've got something definite to base > >> further > >> efforts on, and so that other people trying VS2015 won't fail at the > >> first hurdle and waste time duplicating our work here.
> > > > Other people? non-C coders building win32 perl from source or > > TonyC/JDB? > > > > I'd add a warning to the 2 makefiles at > > > > @@ -126,6 +126,10 @@ CCTYPE = MSVC60 > > #CCTYPE = MSVC120 > > # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free > > version) > > #CCTYPE = MSVC120FREE > > +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) > > +#CCTYPE = MSVC140 > > +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free > > version) > > +#CCTYPE = MSVC140FREE > > > > stating "work in progress" or "not supported" and an echo in > > > > .\config.h : $(CFGH_TMPL) > > -del /f config.h > > copy $(CFGH_TMPL) config.h > > @echo.>>$@ > > > > or on another VC 2015 (through ifdefs) specific target an echo > > warning saying "your VC 2015 is experimental". Can't have random > > people thinking it works 100%.
> > Yes, I've added suitable warnings into the makefiles and the > README.win32. I will test a few more configurations today and > hopefully commit later.
I rebuilt with release VC 2015, and your V4 patch (static CRT), I am still seeing ------------------------------------------------------------------------------- perl.exe harness base/cond.t ....................................................... No subtests run base/if.t ......................................................... No subtests run base/lex.t ........................................................ No subtests run base/num.t ........................................................ No subtests run base/pat.t ........................................................ No subtests run base/rs.t ......................................................... No subtests run base/term.t ....................................................... No subtests ------------------------------------------------------------------------------- for every test. I did BUILD_STATIC and ALL_STATIC build, with fat bundled XS perl523.dll I still get "No subtests run" from harness. Only when I deleted /t/perl.exe and perl.exe and renamed perl-static.exe to perl.exe and copied fat (static CRT+static XS) perl.exe to t/perl.exe did I get normal harness running and the "No subtests run" failures went away. Changing blead (no steveh vc 2015 patches) to static CRT with VC 2003 also causes "No subtests run". Are you seeing harness fail like this? Is there something special you are doing? -- bulk88 ~ bulk88 at hotmail.com
CC: "perl5-porters [...] perl.org" <perl5-porters [...] perl.org>
Subject: RE: [perl #125714] Adding support for VS2015 (VC14)
Date: Thu, 6 Aug 2015 07:49:17 +0000
To: "perlbug-followup [...] perl.org" <perlbug-followup [...] perl.org>
From: Steve Hay <Steve.Hay [...] verosoftware.com>
Download (untitled) / with headers
text/plain 4.2k
On 6 August 2015 at 03:41, bulk88 via RT <perlbug-followup@perl.org> wrote: Show quoted text
> On Wed Aug 05 01:25:36 2015, Steve.Hay@verosoftware.com wrote:
>> >> I will look at these in more detail soon, but I'm inclined to commit >> >> what we've got so far so we've got something definite to base >> >> further >> >> efforts on, and so that other people trying VS2015 won't fail at the >> >> first hurdle and waste time duplicating our work here.
>> > >> > Other people? non-C coders building win32 perl from source or >> > TonyC/JDB? >> > >> > I'd add a warning to the 2 makefiles at >> > >> > @@ -126,6 +126,10 @@ CCTYPE = MSVC60 >> > #CCTYPE = MSVC120 >> > # Visual C++ 2013 Express Edition (aka Visual C++ 12.0) (free >> > version) >> > #CCTYPE = MSVC120FREE >> > +# Visual C++ 2015 (aka Visual C++ 14.0) (full version) >> > +#CCTYPE = MSVC140 >> > +# Visual C++ 2015 Express Edition (aka Visual C++ 14.0) (free >> > version) >> > +#CCTYPE = MSVC140FREE >> > >> > stating "work in progress" or "not supported" and an echo in >> > >> > .\config.h : $(CFGH_TMPL) >> > -del /f config.h >> > copy $(CFGH_TMPL) config.h >> > @echo.>>$@ >> > >> > or on another VC 2015 (through ifdefs) specific target an echo >> > warning saying "your VC 2015 is experimental". Can't have random >> > people thinking it works 100%.
>> >> Yes, I've added suitable warnings into the makefiles and the >> README.win32. I will test a few more configurations today and >> hopefully commit later.
> > > I rebuilt with release VC 2015, and your V4 patch (static CRT), I am still seeing > ------------------------------------------------------------------------------- > perl.exe harness > base/cond.t ....................................................... No subtests > run > base/if.t ......................................................... No subtests > run > base/lex.t ........................................................ No subtests > run > base/num.t ........................................................ No subtests > run > base/pat.t ........................................................ No subtests > run > base/rs.t ......................................................... No subtests > run > base/term.t ....................................................... No subtests > ------------------------------------------------------------------------------- > for every test. > > I did BUILD_STATIC and ALL_STATIC build, with fat bundled XS perl523.dll I still get "No subtests > run" from harness. Only when I deleted /t/perl.exe and perl.exe and renamed perl-static.exe to perl.exe and copied fat (static CRT+static XS) perl.exe to t/perl.exe did I get normal harness running and the "No subtests > run" failures went away. Changing blead (no steveh vc 2015 patches) to static CRT with VC 2003 also causes "No subtests run". Are you seeing harness fail like this? Is there something special you are doing? >
I hadn't tried BUILD_STATIC + ALL_STATIC. I just applied the v4 patch plus your fixes (which I've merged together into the attached patch, which is what I intend to apply, but will wait now until you've got it working too) and did a standard "nmake CCTYPE=MSVC140 && nmake CCTYPE=MSVC140 test" in a VC14 command prompt (set up by running C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat, with WIN64=undef set too). For a static CRT test with an old VS (I tried VS2010) I simply changed LIBC from msvcrt.lib to libcmt.lib and -MD to -MT in the Makefile and did a standard "nmake CCTYPE=MSVC100 && nmake CCTYPE=MSVC100 test" in a VC10 command prompt (C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat + WIN64=undef). I've now tried the VS2010 build with both the static CRT and BUILD_STATIC + ALL_STATIC, and that works fine too. (It hasn't finished all tests yet, but is certainly working normally -- it's done all the tests under t/ with only a couple of failures in porting/utils.t and perf/taint.t and is now well into the cpan/ sub-directories' tests.) Are you sure you're starting from a clean workspace each time? I always do "git clean -dfx" before starting a build to make sure I haven't got any stray files laying around that might interfere with things.

Message body is not shown because sender requested not to inline it.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 6.6k
On Thu Aug 06 00:50:03 2015, Steve.Hay@verosoftware.com wrote: Show quoted text
> I hadn't tried BUILD_STATIC + ALL_STATIC. I just applied the v4 patch > plus your fixes (which I've merged together into the attached patch, > which is what I intend to apply, but will wait now until you've got it > working too) and did a standard "nmake CCTYPE=MSVC140 && nmake > CCTYPE=MSVC140 test" in a VC14 command prompt (set up by running > C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat, > with WIN64=undef set too). > > For a static CRT test with an old VS (I tried VS2010) I simply changed > LIBC from msvcrt.lib to libcmt.lib and -MD to -MT in the Makefile and > did a standard "nmake CCTYPE=MSVC100 && nmake CCTYPE=MSVC100 test" in > a VC10 command prompt (C:\Program Files (x86)\Microsoft Visual Studio > 10.0\VC\vcvarsall.bat + WIN64=undef). > > I've now tried the VS2010 build with both the static CRT and > BUILD_STATIC + ALL_STATIC, and that works fine too. (It hasn't > finished all tests yet, but is certainly working normally -- it's done > all the tests under t/ with only a couple of failures in > porting/utils.t and perf/taint.t and is now well into the cpan/ sub- > directories' tests.) > > Are you sure you're starting from a clean workspace each time? I > always do "git clean -dfx" before starting a build to make sure I > haven't got any stray files laying around that might interfere with > things.
I tried blead with VC 2008 on 2K3 with ----------------------------------------------------------------------- win32/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/win32/Makefile b/win32/Makefile index 23adcd4..597e894 100644 --- a/win32/Makefile +++ b/win32/Makefile @@ -443,7 +443,7 @@ DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT LOCDEFS = -DPERLDLL -DPERL_CORE CXX_FLAG = -TP -EHsc -LIBC = msvcrt.lib +LIBC = libcmt.lib !IF "$(CFG)" == "Debug" OPTIMIZE = -Od -MD -Zi -DDEBUGGING @@ -459,7 +459,7 @@ OPTIMIZE = -Od -MDd -Zi -D_DEBUG -DDEBUGGING LINK_DBG = -debug !ELSE # -O1 yields smaller code, which turns out to be faster than -O2 on x86 and x64 -OPTIMIZE = -O1 -MD -Zi -DNDEBUG -GS- +OPTIMIZE = -Od -LD -Zi -DNDEBUG -GS- # we enable debug symbols in release builds also LINK_DBG = -debug -opt:ref,icf # you may want to enable this if you want COFF symbols in the executables ----------------------------------------------------------------------- And got harness breakage, but a little bit different from 2003 on XP and 2015 on Win 7. ------------------------------------------------------------------------------------ C:\p523\src\t> perl.exe harness base/cond.t ....................................................... No subtests run base/if.t ......................................................... 1..2 ok 1 ok 2 base/if.t ......................................................... No subtests run base/lex.t ........................................................ 1..103 #1 :x: eq :x: ok 1 ok 2 ok 3 ok 4 ok 5 ok 6 ok 7 ok 8 ok 9 ok 10 ok 11 ok 12 - make sure single quotes are honored \nnot ok ok 13 ok 14 ok 15 ok 16 ok 17 ok 18 - was the test for the deprecated use of bare << to mean <<"" ok 19 ok 20 ok 21 ok 22 ok 23 ok 24 ok 25 ok 26 ok 27 ok 28 ok 29 ok 30 ok 31 ok 32 ok 33 ok 34 ok 35 ok 36 ok 37 ok 38 ok 39 # uggedaboudit ok 40 ok 41 ok 42 ok 43 ok 44 ok 45 ok 46 ok 47 ok 48 ok 49 ok 50 ok 51 ok 52 ok 53 ok 54 ok 55 - heredoc after "" in s/// in eval ok 56 - heredoc in "" in multiline s///e in eval ok 57 - null on same line as heredoc in s/// in eval ok 58 - heredoc in "" in single-line s///e in eval ok 59 - heredoc in "" in multiline s///e outside eval ok 60 - s/// in s/// pattern ok 61 - here-doc in re-eval ok 62 - here-doc in re-eval in string eval ok 63 - eval ending with semicolon ok 64 - here-doc in single-line re-eval ok 65 - here-doc in quotes in multiline re-eval ok 66 - eval 's//<<END/' does not leave extra newlines ok 67 - # after null in s/// repl ok 68 - s//'#' . <<END/e ok 69 - s//3}->{3/e ok 70 - s//${\%x}{3}/e ok 71 - s/${foo#}//e ok 72 - listop({ok 70 => 1} + 1) ok 73 - [perl #105924] require WORD << ... ok 74 - [perl #105924] goto WORD << ... ok 75 - [perl #105924] last WORD << ... ok 76 - [perl #105924] next WORD << ... ok 77 - [perl #105924] redo WORD << ... ok 78 - [perl #105924] dump WORD << ... ok 79 - Use v[0-9]+ as a label ok 80 - Use v[0-9]+ as a label with space before colon ok 81 - call a function in package v10::foo ok 82 - colon detection after vstring does not break ? vstring : #not ok 83 - print vstring prints the vstring ok 84 - q <comment> <newline> ... ok 85 - qq <comment> <newline> ... ok 86 - qw <comment> <newline> ... ok 87 - m <comment> <newline> ... ok 88 - qr <comment> <newline> ... ok 89 - s <comment> <newline> ... ok 90 - tr <comment> <newline> ... ok 91 - y <comment> <newline> ... ok 92 - => quotes keywords across lines ok 93 - [perl #80368] qq <a\U=> ok 94 - ${...} where {...} looks like hash ok 95 - map{BEGIN... ok 96 - map{BEGIN... ok 97 - ${ <newline> =pod ok 98 - map{ <newline> =pod ok 99 - ${...} (literal triple-dot) ok 100 - map{...} (literal triple-dot) ok 101 - &{sub :lvalue...} ok 102 - map{sub :lvalue...} ok 103 - 124385 base/lex.t ........................................................ No subtests run base/num.t ........................................................ 1..53 ok 1 ok 2 ok 3 ok 4 ok 5 ok 6 ok 7 ok 8 ok 9 ok 10 ok 11 ok 12 ok 13 ok 14 ok 15 ok 16 ok 17 ok 18 ok 19 ok 20 ok 21 ok 22 ok 23 ok 24 ok 25 ok 26 ok 27 ok 28 ok 29 ok 30 ok 31 ok 32 ok 33 ok 34 ok 35 ok 36 ok 37 ok 38 ok 39 ok 40 ok 41 ok 42 ok 43 ok 44 ok 45 ok 46 ok 47 ok 48 ok 49 ok 50 ok 51 ok 52 ok 53 base/num.t ........................................................ No subtests run base/pat.t ........................................................ Terminating on signal SIGINT(2) C:\p523\src\t>1..2 ok 1 ok 2 C:\p523\src\t> ------------------------------------------------------------------------------------ My VC 2003 perl-static.exe has __app_type 1 (aka _CONSOLE_APP) encoded as a constant in .data section (no __set_app_type calls), my VC 2003 static XS+static CRT perl523.dll has __app_type 0 (aka _UNKNOWN_APP) (no __set_app_type calls). msvcr71.dll exports a function called __set_app_type that changes this. Regular DLL CRT perl.exe calls __set_app_type(1) from mainCRTStartup from perl.exe to msvcr71.dll. I need to do more research on how __app_type is selected, how that data symbol knows how to get initialized, does the VC linker special case __app_type from -subsystem https://msdn.microsoft.com/en-us/library/fcc1zstk.aspx ? IDK. Wild guess, do you have CONSOLE or _CONSOLE set as an env var when you are building? -- bulk88 ~ bulk88 at hotmail.com
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.9k
On Thu Aug 06 17:08:11 2015, bulk88 wrote: Show quoted text
> My VC 2003 perl-static.exe has __app_type 1 (aka _CONSOLE_APP) encoded > as a constant in .data section (no __set_app_type calls), my VC 2003 > static XS+static CRT perl523.dll has __app_type 0 (aka _UNKNOWN_APP) > (no __set_app_type calls). msvcr71.dll exports a function called > __set_app_type that changes this. Regular DLL CRT perl.exe calls > __set_app_type(1) from mainCRTStartup from perl.exe to msvcr71.dll. I > need to do more research on how __app_type is selected, how that data > symbol knows how to get initialized, does the VC linker special case > __app_type from -subsystem https://msdn.microsoft.com/en- > us/library/fcc1zstk.aspx ? IDK. Wild guess, do you have CONSOLE or > _CONSOLE set as an env var when you are building?
-subsystem:console,"5.01" on every call to link.exe didn't fix the harness problem or change anything about it. I solved the harness with perl523.dll with static CRT sees no child stdout data problem with this crude patch, the patch works for VC 2003 and VC 2008. The +#define _UNKNOWN_APP 0 +#define _CONSOLE_APP 1 +#define _GUI_APP 2 + +EXTERN_C _CRTIMP void __cdecl __set_app_type(int); stuff should probably be moved to the start of perllib.c or to win32.h inside PERL_CORE. -------------------------------------------------------------------- win32/Makefile | 4 ++-- win32/perllib.c | 6 ++++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/win32/Makefile b/win32/Makefile index 23adcd4..597e894 100644 --- a/win32/Makefile +++ b/win32/Makefile @@ -443,7 +443,7 @@ DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT LOCDEFS = -DPERLDLL -DPERL_CORE CXX_FLAG = -TP -EHsc -LIBC = msvcrt.lib +LIBC = libcmt.lib !IF "$(CFG)" == "Debug" OPTIMIZE = -Od -MD -Zi -DDEBUGGING @@ -459,7 +459,7 @@ OPTIMIZE = -Od -MDd -Zi -D_DEBUG -DDEBUGGING LINK_DBG = -debug !ELSE # -O1 yields smaller code, which turns out to be faster than -O2 on x86 and x64 -OPTIMIZE = -O1 -MD -Zi -DNDEBUG -GS- +OPTIMIZE = -Od -LD -Zi -DNDEBUG -GS- # we enable debug symbols in release builds also LINK_DBG = -debug -opt:ref,icf # you may want to enable this if you want COFF symbols in the executables diff --git a/win32/perllib.c b/win32/perllib.c index 0e44a24..6cee241 100644 --- a/win32/perllib.c +++ b/win32/perllib.c @@ -285,6 +285,11 @@ set_w32_module_name(void); EXTERN_C void EndSockets(void); +#define _UNKNOWN_APP 0 +#define _CONSOLE_APP 1 +#define _GUI_APP 2 + +EXTERN_C _CRTIMP void __cdecl __set_app_type(int); #ifdef __MINGW32__ EXTERN_C /* GCC in C++ mode mangles the name, otherwise */ @@ -305,6 +310,7 @@ DllMain(HINSTANCE hModule, /* DLL module handle */ w32_perldll_handle = hModule; set_w32_module_name(); + __set_app_type(_CONSOLE_APP); break; /* The DLL is detaching from a process due to ------------------------------------------------------------------------------------- How come you didn't have to do this? -- bulk88 ~ bulk88 at hotmail.com
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 332b
On Thu Aug 06 20:19:40 2015, bulk88 wrote: Show quoted text
> I solved the harness with perl523.dll with static CRT sees no child > stdout data problem with this crude patch, the patch works for VC 2003 > and VC 2008.
VC 2010 also fails for me with static CRT+harness not seeing any output from from child procs. -- bulk88 ~ bulk88 at hotmail.com
To: "perlbug-followup [...] perl.org" <perlbug-followup [...] perl.org>
From: Steve Hay <Steve.Hay [...] verosoftware.com>
Date: Fri, 7 Aug 2015 08:00:30 +0000
Subject: RE: [perl #125714] Adding support for VS2015 (VC14)
CC: "perl5-porters [...] perl.org" <perl5-porters [...] perl.org>
Download (untitled) / with headers
text/plain 3.8k
On 7 August 2015 at 01:08, bulk88 via RT <perlbug-followup@perl.org> wrote: Show quoted text
> On Thu Aug 06 00:50:03 2015, Steve.Hay@verosoftware.com wrote:
>> I hadn't tried BUILD_STATIC + ALL_STATIC. I just applied the v4 patch >> plus your fixes (which I've merged together into the attached patch, >> which is what I intend to apply, but will wait now until you've got it >> working too) and did a standard "nmake CCTYPE=MSVC140 && nmake >> CCTYPE=MSVC140 test" in a VC14 command prompt (set up by running >> C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat, >> with WIN64=undef set too). >> >> For a static CRT test with an old VS (I tried VS2010) I simply changed >> LIBC from msvcrt.lib to libcmt.lib and -MD to -MT in the Makefile and >> did a standard "nmake CCTYPE=MSVC100 && nmake CCTYPE=MSVC100 test" in >> a VC10 command prompt (C:\Program Files (x86)\Microsoft Visual Studio >> 10.0\VC\vcvarsall.bat + WIN64=undef). >> >> I've now tried the VS2010 build with both the static CRT and >> BUILD_STATIC + ALL_STATIC, and that works fine too. (It hasn't >> finished all tests yet, but is certainly working normally -- it's done >> all the tests under t/ with only a couple of failures in >> porting/utils.t and perf/taint.t and is now well into the cpan/ sub- >> directories' tests.) >> >> Are you sure you're starting from a clean workspace each time? I >> always do "git clean -dfx" before starting a build to make sure I >> haven't got any stray files laying around that might interfere with >> things.
> > I tried blead with VC 2008 on 2K3 with > ----------------------------------------------------------------------- > win32/Makefile | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/win32/Makefile b/win32/Makefile > index 23adcd4..597e894 100644 > --- a/win32/Makefile > +++ b/win32/Makefile > @@ -443,7 +443,7 @@ DEFINES = -DWIN32 -D_CONSOLE -DNO_STRICT > LOCDEFS = -DPERLDLL -DPERL_CORE > CXX_FLAG = -TP -EHsc > > -LIBC = msvcrt.lib > +LIBC = libcmt.lib > > !IF "$(CFG)" == "Debug" > OPTIMIZE = -Od -MD -Zi -DDEBUGGING > @@ -459,7 +459,7 @@ OPTIMIZE = -Od -MDd -Zi -D_DEBUG -DDEBUGGING > LINK_DBG = -debug > !ELSE > # -O1 yields smaller code, which turns out to be faster than -O2 on x86 and x64 > -OPTIMIZE = -O1 -MD -Zi -DNDEBUG -GS- > +OPTIMIZE = -Od -LD -Zi -DNDEBUG -GS- > # we enable debug symbols in release builds also > LINK_DBG = -debug -opt:ref,icf > # you may want to enable this if you want COFF symbols in the executables > ----------------------------------------------------------------------- >
I tried -MT rather than -LD, but I've just tried -LD too and it makes no difference -- mine still works fine. I notice that your diff above includes -GS- in the OPTIMIZE flags. That isn't in blead. Where has that come from? Do you have other patches in place too? Does a clean blead work with just the switch to libcmt.lib / -MT and nothing else (i.e. the attached patch)? Show quoted text
> > My VC 2003 perl-static.exe has __app_type 1 (aka _CONSOLE_APP) encoded as a constant in .data section (no __set_app_type calls), my VC 2003 static XS+static CRT perl523.dll has __app_type 0 (aka _UNKNOWN_APP) (no __set_app_type calls). msvcr71.dll exports a function called __set_app_type that changes this. Regular DLL CRT perl.exe calls __set_app_type(1) from mainCRTStartup from perl.exe to msvcr71.dll. I need to do more research on how __app_type is selected, how that data symbol knows how to get initialized, does the VC linker special case __app_type from -subsystem https://msdn.microsoft.com/en-us/library/fcc1zstk.aspx ? IDK. Wild guess, do you have CONSOLE or _CONSOLE set as an env var when you are building? >
No, I don't have either CONSOLE or _Console set in the environment.
Download static-crt.patch
text/plain 1006b

Message body is not shown because sender requested not to inline it.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 4.2k
On Fri Aug 07 01:01:08 2015, Steve.Hay@verosoftware.com wrote: Show quoted text
> I tried -MT rather than -LD, but I've just tried -LD too and it makes > no difference -- mine still works fine. > > I notice that your diff above includes -GS- in the OPTIMIZE flags. > That isn't in blead. Where has that come from? Do you have other > patches in place too? Does a clean blead work with just the switch to > libcmt.lib / -MT and nothing else (i.e. the attached patch)?
Trying a clean blead with VC 2003 on WinXP. -------------------------------------------------------------- ../cpan/JSON-PP/t/016_tied.t ...................................... Warning: una ble to close filehandle properly: Bad file descriptor during global destruction. ../cpan/JSON-PP/t/016_tied.t ...................................... No subtests run ../cpan/JSON-PP/t/017_relaxed.t ................................... Warning: una ble to close filehandle properly: Bad file descriptor during global destruction. ../cpan/JSON-PP/t/017_relaxed.t ................................... No subtests run ../cpan/JSON-PP/t/018_json_checker.t .............................. Warning: una ble to close filehandle properly: Bad file descriptor during global destruction. ../cpan/JSON-PP/t/018_json_checker.t .............................. No subtests run ../cpan/JSON-PP/t/019_incr.t ...................................... Warning: una ble to close filehandle properly: Bad file descriptor during global destruction. Terminating on signal SIGINT(2) Terminating on signal SIGINT(2) NMAKE : fatal error U1058: terminated by user Stop. C:\perl521\src\win32> C:\perl521\src\win32>git log -n3 WARNING: terminal is not fully functional commit 57da2ba98354a7c54f3141a6e5541fd50186d376 Author: Daniel Dragan <bulk88@hotmail.com> Date: Fri Aug 7 06:02:44 2015 -0400 static crt form shay commit 749f0eb1485a7bb7561d71f9539d9b7655363136 Author: Steve Hay <steve.m.hay@googlemail.com> Date: Fri Aug 7 08:39:07 2015 +0100 CPAN-Meta-YAML is now synced with 0.017-TRIAL Delete Changes file. We don't put IGNORABLEs in cpan/ folders. commit 5356d32ec564e8c2e19758f66dbcbbc38327aa75 Author: Tony Cook <tony@develop-help.com> Date: Fri Aug 7 14:20:36 2015 +1000 remove the byte-order-mark introduced to sv.c by 5488d373 This causes the build to fail on NetBSD C:\perl521\src\win32>nmake test CCTYPE=MSVC70 ---------------------------------------------------------------------- Show quoted text
> No, I don't have either CONSOLE or _Console set in the environment.
Email/offlist mail me your VC 2013 or VC 2010 (I'd prefer VC 2010), static CRT perl.exe, perl.pdb, perl523.dll and perl523.pdb files. I want to know what app_type is initially set with and where it is being changed inside you particular binaries. Push your static CRT branch somewhere public. I think problem needs to be solved by a 3rd person trying to compile static CRT perl. That 3rd person will probably be TonyC unless A. Sinan Unur comes back. Same checkout as above, VC 2003 with Win2k (a rarely used system of mine), same harness problem --------------------------------------------------------------------------------- xcopy /f /r /i /d /y ..\perlglob.exe ..\t\ C:\p523\staticcrt\perlglob.exe -> C:\p523\staticcrt\t\perlglob.exe 1 File(s) copied set PERL_STATIC_EXT=Win32CORE cd ..\t perl.exe harness base/cond.t ....................................................... No subtests run base/if.t ......................................................... No subtests run base/lex.t ........................................................ No subtests run base/num.t ........................................................ No subtests run base/pat.t ........................................................ No subtests run base/rs.t ......................................................... No subtests run base/term.t ....................................................... No subtests run base/translate.t .................................................. No subtests --------------------------------------------------------------------------------- Retried your commit with VC 2013 on Win 7, same problem remains. 4 different PCs, 5 different VCs. Not one of my PCs work for static CRT perl harness. What OS are you using? Are using cmd.exe or powershell to run nmake? -- bulk88 ~ bulk88 at hotmail.com
Subject: RE: [perl #125714] Adding support for VS2015 (VC14)
CC: "perl5-porters [...] perl.org" <perl5-porters [...] perl.org>
To: "perlbug-followup [...] perl.org" <perlbug-followup [...] perl.org>
From: Steve Hay <Steve.Hay [...] verosoftware.com>
Date: Fri, 7 Aug 2015 14:16:13 +0000
Download (untitled) / with headers
text/plain 556b
Replied offlist with requested files. This is a standard cmd.exe on Windows 8.1. I will try another machine (Win 7) tonight... Show quoted text
-----Original Message----- From: bulk88 via RT [mailto:perlbug-followup@perl.org] Sent: 07 August 2015 13:14 Cc: perl5-porters@perl.org Subject: [perl #125714] Adding support for VS2015 (VC14) Retried your commit with VC 2013 on Win 7, same problem remains. 4 different PCs, 5 different VCs. Not one of my PCs work for static CRT perl harness. What OS are you using? Are using cmd.exe or powershell to run nmake?
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 415b
On Fri Aug 07 07:16:52 2015, Steve.Hay@verosoftware.com wrote: Show quoted text
> Replied offlist with requested files. This is a standard cmd.exe on > Windows 8.1. I will try another machine (Win 7) tonight... >
I've just tried with exactly the same setup (blead + static-crt.patch using VS2010 Pro SP1) but now on Windows 7 SP1 rather than Windows 8.1 and I now see the same problem as you -- "No subtests run". Very odd indeed!
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.5k
On Fri Aug 07 13:33:31 2015, shay wrote: Show quoted text
> I've just tried with exactly the same setup (blead + static-crt.patch > using VS2010 Pro SP1) but now on Windows 7 SP1 rather than Windows 8.1 > and I now see the same problem as you -- "No subtests run". Very odd > indeed!
The binaries you sent me. perl.exe, __app_type initialized to 1 (aka _CONSOLE_APP), no explicit set funcs called perl523.dll, __app_type initialized to 0 (aka _UNKNOWN_APP), no explicit set funcs called This 2 values are identical to all static CRT perls I've built myself. If you grep the CRT source code, you will see __app_type being used in _free_osfhnd and _set_osfhnd (the unexported function, not P5's pioinfo macro). __app_type seems to be used, from what I can tell, whether fd 0/1/2 are stdout/stdin/stderr (this process has a console), or regular disk file or serial ports created with open()/fopen() (this is a GUI process with no console). __app_type controls synchronizing the CRT's view of what OS handles are IN/OUT/ERR with Win32 API/layer/subsystem stdout/stdin/stderr handles (ie, SetStdHandle()). IDK why it works on Win 8.1 for you, but fails on every win OS older than 8.1, I dont have access to 8.1 to give any comments. If the CRT never calls SetStdHandle(), the gazillion dup2/fdreopen calls that Perl core, IPC::* and TAP::Harness do to capture child proc's STDOUT wont have any effect across process boundaries since Win32 subsystem was never informed about the change/dup2, and some parts of Win32 Perl core use GetStdHandle http://perl5.git.perl.org/perl.git/blob/HEAD:/win32/win32.c#l4458 -- bulk88 ~ bulk88 at hotmail.com
RT-Send-CC: perl5-porters [...] perl.org
There has sadly been no progress on supporting building perl with VS2015, which is looking really bad now with VS2017 just around the corner. None of the solutions posted so far on this ticket seem very nice (hence why it has languished for so long). Personally, I find being able to build with current versions of VC++ more important than the fixes for #120091/#118059 which were applied in commit https://perl5.git.perl.org/perl.git/commit/b47a847f62 That commit reverted parts of https://perl5.git.perl.org/perl.git/commit/9b1f18150a and https://perl5.git.perl.org/perl.git/commit/46e77f1118, thereby reintroducing use of the ioinfo struct, which is what causes all the trouble with building with VS2015 because of the big CRT shake-up in that version. Simply reverting commit b47a847f62 (and thus undoing the #120091/#118059 fixes) allows building perl with VS2015 with few other changes required. I've attached a patch against perl-5.24.1 which does exactly this. I didn't see the dist\IO\t\cachepropagate-tcp.t failure which was the subject of #118059 when I did the build, though that was only an intermittent failure anyway. However, the underlying problem never previously caused me any trouble that I'm aware of in day-to-day use so for me this works a treat. (I think it can be a pain for smokers, though...) The importance of this for me is that I've found that linking code built with VS2015 against libraries built with VS2013 doesn't work: it causes "unresolved external symbol: ___iob_func" errors. The solution appears to be a need to rebuild the VS2013 libraries with VS2015 (e.g. see http://stackoverflow.com/questions/30412951/unresolved-external-symbol-imp-fprintf-and-imp-iob-func-sdl2 -- in particular, see MarkH's response to a seemingly simple "fix", explaining that it won't actually work). Thus, having updated all my other software from VS2013 to VS2015, I now have to rebuild perl with VS2015 as well otherwise I can't link my other software against perl524.lib. Fixing that problem is far more important than fixing #118059, hence I personally am going with the attached patch. The question is how many other people will find themselves in the same position? It's rather late in the day for 5.26 (with user-visible change freeze already gone, and full code freeze coming in three weeks) but I'm sorely tempted to apply something like the attached patch to blead so that people can actually build perl with the current VS compiler suite. It looks really bad that with VS2017 almost upon us we still haven't even got support for VS2015 in yet. If we did this then of course I'd be in support of reverting the patch sometime in the future if/when we get a decent solution for supporting VS2015 with the #120091/#118059 fixes in place as well. But until we can do both things together in a sensible manner I feel that supporting VS2015 is more important than fixing #118059 -- especially since discovering the ___iob_func problem, which rules out sticking with VS2013 for perl if you want to upgrade other software tha links against it. How do other people feel about this suggestion, and is it too late for 5.26?
Subject: perl-5.24.1-vc14.patch

Message body is not shown because it is too large.

From: Sawyer X <xsawyerx [...] gmail.com>
Date: Mon, 6 Feb 2017 20:15:29 +0100
CC: perl5-porters [...] perl.org
To: perlbug-comment [...] perl.org
Subject: Re: [perl #125714] Adding support for VS2015 (VC14)
Download (untitled) / with headers
text/plain 3.5k
On 01/27/2017 07:25 PM, Steve Hay via RT wrote: Show quoted text
> There has sadly been no progress on supporting building perl with VS2015, which is looking really bad now with VS2017 just around the corner. > > None of the solutions posted so far on this ticket seem very nice (hence why it has languished for so long). Personally, I find being able to build with current versions of VC++ more important than the fixes for #120091/#118059 which were applied in commit https://perl5.git.perl.org/perl.git/commit/b47a847f62 > > That commit reverted parts of https://perl5.git.perl.org/perl.git/commit/9b1f18150a and https://perl5.git.perl.org/perl.git/commit/46e77f1118, thereby reintroducing use of the ioinfo struct, which is what causes all the trouble with building with VS2015 because of the big CRT shake-up in that version. > > Simply reverting commit b47a847f62 (and thus undoing the #120091/#118059 fixes) allows building perl with VS2015 with few other changes required. I've attached a patch against perl-5.24.1 which does exactly this. > > I didn't see the dist\IO\t\cachepropagate-tcp.t failure which was the subject of #118059 when I did the build, though that was only an intermittent failure anyway. However, the underlying problem never previously caused me any trouble that I'm aware of in day-to-day use so for me this works a treat. (I think it can be a pain for smokers, though...) > > The importance of this for me is that I've found that linking code built with VS2015 against libraries built with VS2013 doesn't work: it causes "unresolved external symbol: ___iob_func" errors. The solution appears to be a need to rebuild the VS2013 libraries with VS2015 (e.g. see http://stackoverflow.com/questions/30412951/unresolved-external-symbol-imp-fprintf-and-imp-iob-func-sdl2 -- in particular, see MarkH's response to a seemingly simple "fix", explaining that it won't actually work). > > Thus, having updated all my other software from VS2013 to VS2015, I now have to rebuild perl with VS2015 as well otherwise I can't link my other software against perl524.lib. Fixing that problem is far more important than fixing #118059, hence I personally am going with the attached patch. > > The question is how many other people will find themselves in the same position? > > It's rather late in the day for 5.26 (with user-visible change freeze already gone, and full code freeze coming in three weeks) but I'm sorely tempted to apply something like the attached patch to blead so that people can actually build perl with the current VS compiler suite. It looks really bad that with VS2017 almost upon us we still haven't even got support for VS2015 in yet. > > If we did this then of course I'd be in support of reverting the patch sometime in the future if/when we get a decent solution for supporting VS2015 with the #120091/#118059 fixes in place as well. But until we can do both things together in a sensible manner I feel that supporting VS2015 is more important than fixing #118059 -- especially since discovering the ___iob_func problem, which rules out sticking with VS2013 for perl if you want to upgrade other software tha links against it. > > How do other people feel about this suggestion, and is it too late for 5.26?
While it's not fun to introduce things in freeze, 5.25.10 is the full freeze and until then fixes can be considered. I definitely think we should consider reverting this to allow compiling. A race condition is a problem, but it's intermittent. Not being able to run on VC2015 is a consistent problem, and a problematic one at that. :) Can anyone please weigh in on this?
CC: perl5-porters [...] perl.org
Date: Thu, 16 Feb 2017 03:12:31 +0900
To: perlbug-comment [...] perl.org
From: "K.Shirakata" <argrath [...] gmail.com>
Subject: Re: [perl #125714] Adding support for VS2015 (VC14)
Download (untitled) / with headers
text/plain 3.8k
On 2017/02/07 4:15, Sawyer X wrote: Show quoted text
> > > On 01/27/2017 07:25 PM, Steve Hay via RT wrote:
>> There has sadly been no progress on supporting building perl with VS2015, which is looking really bad now with VS2017 just around the corner. >> >> None of the solutions posted so far on this ticket seem very nice (hence why it has languished for so long). Personally, I find being able to build with current versions of VC++ more important than the fixes for #120091/#118059 which were applied in commit https://perl5.git.perl.org/perl.git/commit/b47a847f62 >> >> That commit reverted parts of https://perl5.git.perl.org/perl.git/commit/9b1f18150a and https://perl5.git.perl.org/perl.git/commit/46e77f1118, thereby reintroducing use of the ioinfo struct, which is what causes all the trouble with building with VS2015 because of the big CRT shake-up in that version. >> >> Simply reverting commit b47a847f62 (and thus undoing the #120091/#118059 fixes) allows building perl with VS2015 with few other changes required. I've attached a patch against perl-5.24.1 which does exactly this. >> >> I didn't see the dist\IO\t\cachepropagate-tcp.t failure which was the subject of #118059 when I did the build, though that was only an intermittent failure anyway. However, the underlying problem never previously caused me any trouble that I'm aware of in day-to-day use so for me this works a treat. (I think it can be a pain for smokers, though...) >> >> The importance of this for me is that I've found that linking code built with VS2015 against libraries built with VS2013 doesn't work: it causes "unresolved external symbol: ___iob_func" errors. The solution appears to be a need to rebuild the VS2013 libraries with VS2015 (e.g. see http://stackoverflow.com/questions/30412951/unresolved-external-symbol-imp-fprintf-and-imp-iob-func-sdl2 -- in particular, see MarkH's response to a seemingly simple "fix", explaining that it won't actually work). >> >> Thus, having updated all my other software from VS2013 to VS2015, I now have to rebuild perl with VS2015 as well otherwise I can't link my other software against perl524.lib. Fixing that problem is far more important than fixing #118059, hence I personally am going with the attached patch. >> >> The question is how many other people will find themselves in the same position? >> >> It's rather late in the day for 5.26 (with user-visible change freeze already gone, and full code freeze coming in three weeks) but I'm sorely tempted to apply something like the attached patch to blead so that people can actually build perl with the current VS compiler suite. It looks really bad that with VS2017 almost upon us we still haven't even got support for VS2015 in yet. >> >> If we did this then of course I'd be in support of reverting the patch sometime in the future if/when we get a decent solution for supporting VS2015 with the #120091/#118059 fixes in place as well. But until we can do both things together in a sensible manner I feel that supporting VS2015 is more important than fixing #118059 -- especially since discovering the ___iob_func problem, which rules out sticking with VS2013 for perl if you want to upgrade other software tha links against it. >> >> How do other people feel about this suggestion, and is it too late for 5.26?
> > While it's not fun to introduce things in freeze, 5.25.10 is the full > freeze and until then fixes can be considered. > > I definitely think we should consider reverting this to allow compiling. > A race condition is a problem, but it's intermittent. Not being able to > run on VC2015 is a consistent problem, and a problematic one at that. :) > > Can anyone please weigh in on this?
I think it's worth to reverting this to build on VS2015. This allow people build perl easily with MS official develop environment virtual machine. https://developer.microsoft.com/en-us/windows/downloads/virtual-machines
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 522b
This is now applied to blead by commit 1f664ef5314fb6e438137c44c95cf5ecdbdb5e9b, modified slightly to only revert the RT#120091/118059 fixes for VS2015 (and higher), as per further discussion that I had off-list with the ActivePerl and StrawberryPerl maintainers (emails now attached here for future reference). Thus, compilation with GCC and VS up to and including VS2013 should be materially the same as before this patch; compilation with VS2015 is now possible at the price of reintroducing the RT#120091/118059 bugs.
Subject: VC++ 2015 support for Perl.txt
Gmail *Steve Hay <steve.m.hay@googlemail.com>* ------------------------------------------------------------------------ *VC++ 2015 support for Perl* 9 messages ------------------------------------------------------------------------ *Steve Hay *<steve.m.hay@googlemail.com> 13 February 2017 at 08:24 To: Andy Grundman <andyg@activestate.com>, Jan Dubois <jan@jandubois.com>, kmx <kmx@atlas.cz> Hi, I realize ActivePerl and StrawberryPerl both use gcc rather than VC++, but I wondered if any of you had any feelings either way about my proposal to revert a fix for perl#120091/118059 in order to support building with VC++ 2015? I posted a patch recently here: https://rt.perl.org/Ticket/Display.html?id=125714#txn-1446150 <https://rt.perl.org/Ticket/Display.html?id=125714#txn-1446150> Nobody has replied other than Sawyer... asking for more input, but none has been forthcoming :-/ Am I the only one that cares about Perl with VC++?! The VC++ aspect might not interest you, but how would you feel about the bug fix to not close a freed socket OS handle being reverted? I think the main impact it might have is in causing an intermittent test failure in dist/IO/t/cachepropagate-tcp.t, which might cause some smoker noise (if we even have any Win32 smokers any more). Thanks, Steve ------------------------------------------------------------------------ *Jan Dubois *<jan@jandubois.com> 13 February 2017 at 20:06 To: Steve Hay <steve.m.hay@googlemail.com> Cc: Andy Grundman <andyg@activestate.com>, kmx <kmx@atlas.cz> Hi Steve, I have a hard time wrapping my head around those patches; and I still feel as uneasy about them as when they were first added. In general I consider the whole PERL_ITHREADS stuff a failure, and would consider any bugs that occur only under multiple threads less important than keeping compiler support for VS working. The only caveat is that several CPAN modules use threading for client/server network testing, so those uses should not be broken. However, I think the close socket issue potentially affects all code, so it might be bad to loose this for mingw compilation without any offsetting benefit. I have never run into it myself though, so I have no idea how common it is in practice. What do you think about conditionally reverting the change for VS compilation only? This will of course break binary compatibility between mingw and vs compiled modules, but I wonder if that is important. It only affects binary package distribution ala PPM, and if you want to use PPM, I don't see why you can't use a mingw compiled Perl. If using conditional compilation is a possible solution, then there should be some indication to mark the VS perl5xx.dll as incompatible with the mingw one. Sorry for just the high-level feedback right now, but just looking at the patches again in detail makes my head hurt... Feel free to quote my reply to p5p or Sawyer, or let me know if you want me to respond to your thread directly. Cheers, -Jan [Quoted text hidden] ------------------------------------------------------------------------ *Jan Dubois *<jan@jandubois.com> 13 February 2017 at 20:10 To: Steve Hay <steve.m.hay@googlemail.com> Cc: Andy Grundman <andyg@activestate.com>, kmx <kmx@atlas.cz> On Mon, Feb 13, 2017 at 12:06 PM, Jan Dubois <jan@jandubois.com <mailto:jan@jandubois.com>> wrote: This will of course break binary compatibility between mingw and vs compiled modules, Oops, forgot to add that not being able to compile with vs2015 at all of course breaks compatibility even harder. :) Which is why I don't consider VS and mingw producing incompatible modules a big deal anymore. Cheers, -Jan ------------------------------------------------------------------------ *Steve Hay *<steve.m.hay@googlemail.com> 14 February 2017 at 08:18 To: Jan Dubois <jan@jandubois.com> Cc: Andy Grundman <andyg@activestate.com>, kmx <kmx@atlas.cz>, Sawyer X <xsawyerx@gmail.com> Hi Jan, Thanks for the reply. I also don't use ithreads for anything much -- except mod_perl, which requires a threaded perl on Windows because Apache httpd is always threaded. So I wouldn't want to go breaking anything there either, but the close socket issue never caused me any trouble in mod_perl either as far as I can recall. However, the idea of conditional compilation is something that had also crossed my mind, but with a slightly different angle to what you suggest: I was wondering about making the close socket reversion specific to VS2015 (and above), rather than all VS versions. MinGW and VS2015+ would still be incompatible, but it would have the minor advantage of keeping binary compatibility between MinGW and VS builds for VS<=2013. The downside is that then VS<=2013 and VS>2015 would be incompatible, but (a) (as you point out) they're currently VERY incompatible anyway since VS>2015 doesn't build, and (b) I think they're incompatible anyway because of the ___iob_func problem that I mentioned in my recent post / proposal on #125714. I'm CC'ing Sawyer on this (and quoting your full replies below) because ultimately he has the unenviable task of deciding whether to jump with this, and if so then in what direction :-) Sawyer - Please feel free to quote any or all of this on p5p if you wish to make any decisions being taken here public. Thanks, Steve On 13 February 2017 at 20:10, Jan Dubois <jan@jandubois.com <mailto:jan@jandubois.com>> wrote: > > On Mon, Feb 13, 2017 at 12:06 PM, Jan Dubois <jan@jandubois.com <mailto:jan@jandubois.com>> wrote: >> >> This will of course break binary compatibility between mingw and vs >> compiled modules, > > > Oops, forgot to add that not being able to compile with vs2015 at all of > course breaks compatibility even harder. :) > > Which is why I don't consider VS and mingw producing incompatible modules a > big deal anymore. > > Cheers, > -Jan [Quoted text hidden] ------------------------------------------------------------------------ *Jan Dubois *<jan@jandubois.com> 14 February 2017 at 17:42 To: Steve Hay <steve.m.hay@googlemail.com> Cc: Andy Grundman <andyg@activestate.com>, kmx <kmx@atlas.cz>, Sawyer X <xsawyerx@gmail.com> On Tue, Feb 14, 2017 at 12:18 AM, Steve Hay <steve.m.hay@googlemail.com <mailto:steve.m.hay@googlemail.com>> wrote: However, the idea of conditional compilation is something that had also crossed my mind, but with a slightly different angle to what you suggest: I was wondering about making the close socket reversion specific to VS2015 (and above), rather than all VS versions. Yes, that sounds fine. You may still want to store a setting in Config.pm for this in case anybody ever compiles an XS module with VS2013 for a Perl compiled with VS2015+. (Actually, I've already forgotten again, but I think this change might affect module compilation too, as struct offsets may change. Or am I confused about this?) Cheers, -Jan ------------------------------------------------------------------------ *kmx *<kmx@atlas.cz> 14 February 2017 at 20:37 To: Steve Hay <steve.m.hay@googlemail.com>, Jan Dubois <jan@jandubois.com> Cc: Andy Grundman <andyg@activestate.com>, Sawyer X <xsawyerx@gmail.com> Hi Steve, from me only a short notice that the VC related changes you are considering can very unlikely affect strawberry perl builds which will stay gcc-based. I do not even have a request (or something like that) from strawberry perl users for compatibility with VC perl builds. What I am considering is to switch 64bit gcc from SJLJ to SEH in the 5.26.x series, but it is a completely different story. kmx [Quoted text hidden] ------------------------------------------------------------------------ *Jan Dubois *<jan@jandubois.com> 14 February 2017 at 20:43 To: kmx <kmx@atlas.cz> Cc: Steve Hay <steve.m.hay@googlemail.com>, Andy Grundman <andyg@activestate.com>, Sawyer X <xsawyerx@gmail.com> On Tue, Feb 14, 2017 at 12:37 PM, kmx <kmx@atlas.cz <mailto:kmx@atlas.cz>> wrote: the VC related changes you are considering can very unlikely affect strawberry perl builds which will stay gcc-based. They would affect Strawberry Perl, which is why I would prefer that they are localized via conditional compilation. What I am considering is to switch 64bit gcc from SJLJ to SEH in the 5.26.x series, but it is a completely different story. I would recommend to chat with Andy about this, in case he wants to switch over ActivePerl as well. That would keep the PPM archives working for Strawberry Perl, and would maybe also keep the PDK working with it as well. I don't know however, if this is even still working anymore, or if this is still a goal. Cheers, -Jan ------------------------------------------------------------------------ *Steve Hay *<steve.m.hay@googlemail.com> 16 February 2017 at 08:20 To: Jan Dubois <jan@jandubois.com> Cc: Andy Grundman <andyg@activestate.com>, kmx <kmx@atlas.cz>, Sawyer X <xsawyerx@gmail.com> On 14 February 2017 at 17:42, Jan Dubois <jan@jandubois.com <mailto:jan@jandubois.com>> wrote: > > On Tue, Feb 14, 2017 at 12:18 AM, Steve Hay <steve.m.hay@googlemail.com <mailto:steve.m.hay@googlemail.com>> > wrote: >> >> However, the idea of conditional compilation is something that had >> also crossed my mind, but with a slightly different angle to what you >> suggest: I was wondering about making the close socket reversion >> specific to VS2015 (and above), rather than all VS versions. > > > Yes, that sounds fine. You may still want to store a setting in Config.pm > for this in case anybody ever compiles an XS module with VS2013 for a Perl > compiled with VS2015+. I'm not sure if that's really necessary (and getting new settings into Config can be a pain unless we just do some localized Windows-only thing): It's simple enough just to compare the cc version in perl against the cc version being used to compile the module and complain if one is <2015 and one is >=2015. > (Actually, I've already forgotten again, but I think > this change might affect module compilation too, as struct offsets may > change. Or am I confused about this?) > The offsets in the struct handled by the PERLIO_FILE_*() macros will be different in <2015 compared to >=2015, but they will be incompatible anyway. For <2015 there should be no material change once I make the patch specific to >= 2015. ------------------------------------------------------------------------ *Jan Dubois *<jan@jandubois.com> 16 February 2017 at 18:32 To: Steve Hay <steve.m.hay@googlemail.com> Cc: Andy Grundman <andyg@activestate.com>, kmx <kmx@atlas.cz>, Sawyer X <xsawyerx@gmail.com> On Thu, Feb 16, 2017 at 12:20 AM, Steve Hay <steve.m.hay@googlemail.com <mailto:steve.m.hay@googlemail.com>> wrote: > Yes, that sounds fine. You may still want to store a setting in Config.pm > for this in case anybody ever compiles an XS module with VS2013 for a Perl > compiled with VS2015+. I'm not sure if that's really necessary (and getting new settings into Config can be a pain unless we just do some localized Windows-only thing): It's simple enough just to compare the cc version in perl against the cc version being used to compile the module and complain if one is <2015 and one is >=2015. Ah yes, I forgot that we have the ccversion already in Config.pm, so that is sufficient to detect the mismatch at module build time. Cheers, -Jan
Date: Sun, 19 Feb 2017 09:04:04 -0700
CC: perl5-porters [...] perl.org
To: perlbug-comment [...] perl.org
Subject: Re: [perl #125714] Adding support for VS2015 (VC14)
From: Karl Williamson <public [...] khwilliamson.com>
Download (untitled) / with headers
text/plain 141b
On 02/19/2017 08:40 AM, Steve Hay via RT wrote: Show quoted text
> This is now applied to blead by commit 1f664ef5314fb6e438137c44c95cf5ecdbdb5e9b,
shay++
Subject: Re: [perl #125714] Adding support for VS2015 (VC14)
To: perl5-porters [...] perl.org
Date: Mon, 20 Feb 2017 11:12:37 +0100
From: Sawyer X <xsawyerx [...] gmail.com>
Download (untitled) / with headers
text/plain 339b
On 02/19/2017 05:04 PM, Karl Williamson wrote: Show quoted text
> On 02/19/2017 08:40 AM, Steve Hay via RT wrote:
>> This is now applied to blead by commit >> 1f664ef5314fb6e438137c44c95cf5ecdbdb5e9b,
> > > shay++
shay++ indeed. (This issue involved additional discussions off the list to clarify some finer points. Thanks, Steve, for taking this on.)
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 554b
On Mon, 20 Feb 2017 10:14:15 GMT, xsawyerx@gmail.com wrote: Show quoted text
> > On 02/19/2017 05:04 PM, Karl Williamson wrote:
> > On 02/19/2017 08:40 AM, Steve Hay via RT wrote:
> >> This is now applied to blead by commit > >> 1f664ef5314fb6e438137c44c95cf5ecdbdb5e9b,
> > > > > > shay++
> > shay++ indeed. > > (This issue involved additional discussions off the list to clarify some > finer points. Thanks, Steve, for taking this on.) >
Do we expect that work will proceed on this endeavor during the 5.27 development cycle? -- James E Keenan (jkeenan@cpan.org)


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org