New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC on windows build status #16417
Comments
From @dur-randirCreated by @dur-randirI've recently done a Perl build on a Windows machine using VS 2017 and - perlwin32.html (aka README.win32) states "There should be no test - during both 32-bit and 64-bit builds, I've seen a lot of warnings Or is windows build currently supported only as a MinGW Strawberry Perl Info
|
From @steve-m-hayOn 12 February 2018 at 20:47, Sergey Aleynikov
I just did a build with VS2015 and found three or four tests hung. I There shouldn't be any test failures, other than what is documented in
Lots of warnings are, unfortunately, normal with some versions. I
Visual C++ builds are definitely very much still supported. |
The RT System itself - Status changed from 'new' to 'open' |
From @steve-m-hayOn 12 February 2018 at 21:49, Steve Hay <steve.m.hay@googlemail.com> wrote:
From a 32-bit VS2015 build I have only these warnings from core code: ..\mathoms.c(694): warning C4028: formal parameter 2 different from declaration and these from modules: MD5.xs(763): warning C4244: '=': conversion from 'U32' to 'char', |
From @steve-m-hayOn 19 February 2018 at 00:06, Sergey Aleynikov via RT
[...]
[...] That's about the same as I get with VS2015 or VS2017 in x64 builds.
I occasionally see certain tests (01_IPC-Cmd.t is one of them) hanging
This is MUCH more worrying. I don't get any test failures on either OS Is this run from a clean git workspace? I'm trying to picture how Are you using the full VS2017, or the free (Community Edition) Have you set CCTYPE=MSVC141? Please can you re-run some of those failed tests with the verbose |
From @sisyphus-----Original Message-----
cpan/IPC-Cmd/t/01_IPC-Cmd.t always hangs for me with my x64 mingw builds on Also, the hangs occur only when run in 'make test'. When run normally as I've pretty much given up on being bothered by these hangs. I just kill them Cheers, |
From @dur-randirOn Mon, 19 Feb 2018 01:11:35 -0800, shay wrote:
Yes, x86 build is much more clearer. I haven't compared it on a line-by-line basis, but warnings look really similar to what you've posted before.
This is indeed Windows 7, but it's a clean machine without any antivirus software installed (MS default one is switched off too). But this contradicts documented behaviour "all tests should pass".
This is a build done after 'git clean -dfx' was made from 'c:/perl-git' dir. I'll try to build from a fresh checkout. There're only two modified files: - Makefile for CCTYPE and/or WIN64 I'll check exact value set for CCTYPE, but it's not the default one (I've picked one from README.win32 recommendations).
This is a CE VS 2017.
Yes, I'm planning to report all individual failures/warnings for all modules involved, and will send detailed report for core tests here. |
From @dur-randirOn Mon, 19 Feb 2018 01:11:35 -0800, shay wrote:
So, here's a run from a clean checkout with the following config applied: c:\perl-git-fresh>git diff Inline Patchdiff --git a/win32/Makefile b/win32/Makefile
index 3889ff9ec5..4020b22621 100644
--- a/win32/Makefile
+++ b/win32/Makefile
@@ -21,7 +21,7 @@
# newly built perl.
#
INST_DRV = c:
-INST_TOP = $(INST_DRV)\perl
+INST_TOP = $(INST_DRV)\perl-blead
#
# Uncomment if you want to build a 32-bit Perl using a 32-bit compiler
@@ -112,7 +112,7 @@ DEFAULT_INC_EXCLUDES_DOT = define
# uncomment exactly one of the following
#
# Visual C++ 6.0 (aka Visual C++ 98)
-CCTYPE = MSVC60
+#CCTYPE = MSVC60
# Visual C++ .NET 2002/2003 (aka Visual C++ 7.0/7.1) (full version)
#CCTYPE = MSVC70
# Visual C++ Toolkit 2003 (aka Visual C++ 7.1) (free command-line tools)
@@ -132,7 +132,7 @@ CCTYPE = MSVC60
# Visual C++ 2015 (aka Visual C++ 14.0) (full version or Express Edition)
#CCTYPE = MSVC140
# Visual C++ 2017 (aka Visual C++ 14.1) (full version or Community Edition)
-#CCTYPE = MSVC141
+CCTYPE = MSVC141
#
# If you are using Intel C++ Compiler uncomment this
@@ -161,6 +161,7 @@ CCTYPE = MSVC60
# which causes warnings to be printed on STDERR, which in turn causes a
# few tests to fail.)
#
+
#CFG = Debug
#
diff --git a/win32/config.vc b/win32/config.vc
index a7cea270ce..3b0fc92ca4 100644
--- a/win32/config.vc
+++ b/win32/config.vc
@@ -21,7 +21,8 @@ api_revision='~PERL_API_REVISION~'
api_subversion='~PERL_API_SUBVERSION~'
api_version='~PERL_API_VERSION~'
api_versionstring='~PERL_API_REVISION~.~PERL_API_VERSION~.~PERL_API_SUBVERSION~'
-ar='lib -ltcg'
+#ar='lib -ltcg'
+ar='link /lib'
archlib='~INST_TOP~~INST_VER~\lib~INST_ARCH~'
archlibexp='~INST_TOP~~INST_VER~\lib~INST_ARCH~'
archname64=''
with the same results: Test Summary Report porting/checkcfgvar.t (Wstat: 0 Tests: 24 Failed: 1) Here're individual runs for all failing test: c:\perl-git-fresh\t>.\perl.exe harness -v porting/checkcfgvar.t Test Summary Report porting/checkcfgvar.t (Wstat: 0 Tests: 24 Failed: 1) c:\perl-git-fresh\t>.\perl.exe harness -v porting/customized.t # got "4cc48268b9316505c095dccb1523ae380b5f2f61" not ok 1 - SHA for dist/Devel-PPPort/parts/embed.fnc matches stashed SHA Test Summary Report porting/customized.t (Wstat: 0 Tests: 28 Failed: 28) c:\perl-git-fresh\t>.\perl.exe harness -v porting/pod_rules.t 1..8 Test Summary Report porting/pod_rules.t (Wstat: 0 Tests: 6 Failed: 5) c:\perl-git-fresh\t>.\perl.exe harness -v porting/regen.t not ok # Makefile.SH is up to date Test Summary Report porting/regen.t (Wstat: 65280 Tests: 11 Failed: 1) c:\perl-git-fresh\t>.\perl.exe harness -v ../cpan/CPAN-Meta-YAML/t/30_yaml_spec_tml.t Test Summary Report ../cpan/CPAN-Meta-YAML/t/30_yaml_spec_tml.t (Wstat: 512 Tests: 0 Failed: 0) ../cpan/IPC-Cmd/t/01_IPC-Cmd.t indeed passes when run as a separate test file c:\perl-git-fresh\t>.\perl.exe harness -v ../cpan/podlators/t/text/encoding.t Test Summary Report ../cpan/podlators/t/text/encoding.t (Wstat: 65280 Tests: 1 Failed: 0) ../ext/IPC-Open3/t/IPC-Open3.t also passes when run as a separate test file And here's the -V output for this build: c:\perl-git-fresh>.\perl.exe -Ilib -V Characteristics of this binary (from libperl): |
From @craigberryOn Mon, Feb 19, 2018 at 2:21 PM, Sergey Aleynikov via RT
What line ending policy do you have set in your git repository? I |
From @dur-randirOn Mon, 19 Feb 2018 12:41:35 -0800, craig.a.berry@gmail.com wrote:
All build have been done with core.autocrlf set to '' (default). I've tries another one with 'true', it changed nothing. |
From @steve-m-hayOn 19 February 2018 at 21:39, Sergey Aleynikov via RT
I agree with Craig it's highly likely to be EOL problems - that's My .gitconfig file contains [core] and all the files in my workspace have UNIX-style EOLs. Make a new workspace like that and try again? Btw, what is the reason that you change the 'ar' entry in config.vc? I |
From @demerphqOn 19 Feb 2018 19:54, "demerphq" <demerphq@gmail.com> wrote: On 19 Feb 2018 08:07, "Sergey Aleynikov via RT" <perlbug-followup@perl.org> Here's what I've got with a x64 build using MSVC 2017. Build warnings part: c:\perl-git\zaphod32_hash.h(221): warning C4267: 'initializing': conversion Fwiw, this is unexpected to me. Zaphod32 should not by default be used on I don't have the code available to check myself, but it would be nice to Steve while it's not the main focus of this ticket is there any chance you Yves |
From @steve-m-hayOn 20 February 2018 at 10:55, demerphq <demerphq@gmail.com> wrote:
hv_func.h correctly chooses STATDX, but it includes sbox32_hash.h just |
From @demerphqOn 20 Feb 2018 21:39, "Steve Hay" <steve.m.hay@googlemail.com> wrote: On 20 February 2018 at 10:55, demerphq <demerphq@gmail.com> wrote:
hv_func.h correctly chooses STATDX, but it includes sbox32_hash.h just I see. Ok, that explains it for sure. I'll see if I can figure out how to Thanks a lot, much obliged to you for digging. Yves |
From @dur-randirOn Tue, 20 Feb 2018 00:06:57 -0800, shay wrote:
Explicitly setting core.autocrlf to 'false' instead of the default '' helped indeed. Thank you.
There're some words about it in the readme, so I've changed it to what it suggests (maybe for some other compiler version?). I've tested a build without that change and it succeeded. So, now there're only IPC-under-harness failures and a lot of build warnings. Should I report them as a separate tickets for each on of the dist/ and ext/ modules, or this ticket is sufficient? As I've already reported them for all cpan/ ones. |
From @steve-m-hayOn 20 February 2018 at 22:23, Sergey Aleynikov via RT
That's in the section talking specifically about Microsoft Visual C++
I think this ticket is sufficient since dist/ and ext/ are maintained |
From @bulk88On Tue, 20 Feb 2018 06:45:46 -0800, demerphq wrote:
Every Win64 .o/XS module produces this warning described above, and stable perl can't ship if all XS module produce CC warnings in perl headers. cl -c -I. -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -fp:precise -DWIN32 -D_ .i file output of problem line static __inline U32 ; Since this is Win64 key_len is 64 bit int, that is XORed with state[2] which is a U32 and the result is assigned to a U32, truncating high bytes. Hence the warning. It seems zaphod32_hash_with_state code is unique to you (demerphq) and not FOSS code borrowed from somewhere else, so there is no upstream to go consult about the truncation. -- |
From @bulk88On Wed, 21 Feb 2018 09:05:19 -0800, bulk88 wrote:
I see a bigger problem with the new hash code. I put a __builtin_trap() right before "U32 v2 = state[2] ^ (0xC41A7AB1 * (key_len + 1));" and did a "make test". It NEVER executed. I checked the machine code, and yes zaphod32_hash_with_state/__builtin_trap is linked in and I can see how theoretically executed, but it is unreachable in real life. zaphod32_hash_with_state is ONLY referenced by sbox32_hash_with_state. static __inline U32 case 24: That is the only reference. But if I look at the callers of sbox32_hash_with_state, there are 18 callers, but all of then do the len <= 24 check. Example of usage. static __inline U32 So sbox32_hash_with_state can't be reached with a key longer than 24, yet sbox32_hash_with_state has a default: for a key longer than 24. The default: in sbox32_hash_with_state needs to be "NOT_REACHED;" and maybe an assert. That would remove zaphod32_hash_with_state() from the binary since on 64 bit OSes stadtx is used, not zaphod for long keys. I also think there is too much code not ifdefed out in hv_func.h sbox32_hash128 has no uses, it should be #if 0'ed so it doesn't reach the C->Asm stage. sbox32_seed_state128 whose only ref is sbox32_hash128 also can #if 0, since S_perl_hash_with_seed uses sbox32_seed_state96. sbox32_hash96 can also be if #0. S_perl_hash_siphash_2_4 are also in the .i file event though perl is uses the PERL_HASH_USE_SBOX32_ALSO sbox/stadtx or sbox/zaphos pair. Those need ifdefs too. -- |
From @demerphqOn 22 Feb 2018 03:04, "bulk88 via RT" <perlbug-followup@perl.org> wrote: On Wed, 21 Feb 2018 09:05:19 -0800, bulk88 wrote:
It is Foss, but it is also my work.😀 The truncation shouldn't matter, but it shouldn't throw an error either. I see a bigger problem with the new hash code. I put a __builtin_trap() This is a build option issue. Sbox hashing is very fast, but has a finite However with the way that we use sbox32 in perl we will do a similar fall zaphod32_hash_with_state is ONLY referenced by sbox32_hash_with_state. static __inline U32 case 24: That is the only reference. But if I look at the callers of sbox32_hash_with_state, there are 18 Yep. You could change when this happens with the right -D to configure. static __inline U32 So sbox32_hash_with_state can't be reached with a key longer than 24, yet Yep. Will do. I also think there is too much code not ifdefed out in hv_func.h sbox32_hash128 has no uses, it should be #if 0'ed so it doesn't reach the S_perl_hash_siphash_2_4 are also in the .i file event though perl is uses the I assumed a good compiler would elide this unused code... I'll dig further when I get a chance. Thanks, -- via perlbug: queue: perl5 status: open |
From @bulk88On Wed, 21 Feb 2018 19:04:12 -0800, demerphq wrote:
Cast to U32 before assigning to U32 will quiet MSVC.
If you dont do the NOT_REACHED (I prefer that) more macros are needed so the inside of sbox32 matches the primary long key algo. Or have the primary long key algo only from the inside and make it so there is no way to use sbox32 for long keys.
Probably nobody ever will, but still, if the fallback is from inside sbox, then the fallback should only be from sbox, not from the outer _PERL_HASH_WITH_STATE. Basically sbox CANT be compiled to be used in isolation.
Yes please.
A good linker will toss the code, a compiler CANT toss the code and had to spend CPU to generate ASM. Really bad linkers/bad OSes wont toss the code due to GOT/"debugging" reasons (call a static func explicitly with gdb).
A U32 cast is the quick and dirty way to fix it aslong as truncation is okay of that XOR, truncation is happening anyway on all 64b OSes whether a CC warns or not. -- |
From @demerphqOn 22 February 2018 at 06:34, bulk88 via RT <perlbug-followup@perl.org> wrote:
Done in: e5a5512
Er... That isnt clear to me, but i trust you. I will look into reversing
Yeah, i did the cast. I will look into a more graceful fix another day. Thanks for you help, i will come back to you when i have further patches. Yves -- |
From @bulk88On Mon, 12 Feb 2018 12:47:07 -0800, randir wrote:
cpan/IPC-Cmd/t/01_IPC-Cmd.t hangs/failures with Windows 7 (I dont see it on XP) random/race failures are caused by perl calling closesocket() on non-sockets I've talked about it at https://rt-archive.perl.org/perl5/Ticket/Display.html?id=118127 Win 7-32bit callstack of frozen with under-harness 01_IPC-Cmd.t, perl's curcop is at https://perl5.git.perl.org/perl.git/blob/c4a2ac437ed67b458e466f3724f82c10afc3eb40:/dist/IO/lib/IO/Handle.pm#l379 ntdll.dll!_KiFastSystemCallRet@0�() Unknown -- |
From @khwilliamsonOn 02/12/2018 01:47 PM, Sergey Aleynikov (via RT) wrote:
On the Windows attached to dromedary (I presume it's a VM), the build |
From @khwilliamsonOn 02/12/2018 01:47 PM, Sergey Aleynikov (via RT) wrote:
I've wondered about the common warning message from the Windows compiler |
From @khwilliamsonOn 02/24/2018 10:13 PM, Karl Williamson wrote:
This is working again; perhaps it was my fault somehow. |
Migrated from rt.perl.org#132860 (status was 'open')
Searchable as RT132860$
The text was updated successfully, but these errors were encountered: