From @bulk88

Created by @bulk88

When compiling this code from inline.h S_variant_under_utf8_count.
-------------------------
  count += ((((* (PERL_UINTMAX_T *) x) &
PERL_VARIANTS_WORD_MASK) >> 7)
  * PERL_COUNT_MULTIPLIER)
  >> ((PERL_WORDSIZE - 1) * CHARBITS);
-------------------------

I get this warning.

-----------------------
cl -c -nologo -GF -W3 -I.\include -I. -I.. -DWIN32 -D_CONSOLE
-DNO_STRICT -DPERLDLL -DPERL_CORE -O1 -MD -Zi -DNDEBUG -GL
-DPERL_EXTERNAL_GLOB -DPERL_IS_MINIPERL -Fomini\toke.obj
-Fdmini\toke.pdb ..\toke.c
toke.c
c​:\perl521\src\inline.h(646) : warning C4244​: '+=' : conversion from
'PERL_UINTMAX_T' to 'size_t', possible loss of data
-----------------------

I am using VC 2003 32b. PERL_UINTMAX_T is a 64 bit int. Var count is
"size_t" which is 32 bits.

This warning is new (fall/winter 2017) to blead perl and should be fixed
before 5.28 is released. Since KHW wrote this code, he probably knows
best how to silence the warning or change types.

Flags:
          category=core
          severity=low

Site configuration information for perl 5.27.9:

Configured by Administrator at Tue Jan 30 20:34:30 2018.

Summary of my perl5 (revision 5 version 27 subversion 9) configuration:

        Platform:
          osname=MSWin32
          osvers=5.2.3790
          archname=MSWin32-x86-multi-thread
          uname=''
          config_args='undef'
          hint=recommended
          useposix=true
          d_sigaction=undef
          useithreads=define
          usemultiplicity=define
          use64bitint=undef
          use64bitall=undef
          uselongdouble=undef
          usemymalloc=n
          default_inc_excludes_dot=define
          bincompat5005=undef
        Compiler:
          cc='cl'
          ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32
-D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE
-D_CRT_NONSTDC_NO_DEPRECATE  -DPERL_TEXTMODE_SCRIPTS
-DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DWIN32_NO_REGISTRY'
          optimize='-O1 -MD -Zi -DNDEBUG -GL'
          cppflags='-DWIN32'
          ccversion='15.00.30729.01'
          gccversion=''
          gccosandvers=''
          intsize=4
          longsize=4
          ptrsize=4
          doublesize=8
          byteorder=1234
          doublekind=3
          d_longlong=undef
          longlongsize=8
          d_longdbl=define
          longdblsize=8
          longdblkind=0
          ivtype='long'
          ivsize=4
          nvtype='double'
          nvsize=8
          Off_t='__int64'
          lseeksize=8
          alignbytes=8
          prototype=define
        Linker and Libraries:
          ld='link'
          ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg
-libpath:"c:\perl\lib\CORE"        -machine:x86'
          libpth="C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\lib"
          libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib
odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib
          perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib
odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib
          libc=msvcrt.lib
          so=dll
          useshrplib=true
          libperl=perl527.lib
          gnulibc_version=''
        Dynamic Linking:
          dlsrc=dl_win32.xs
          dlext=dll
          d_dlsymun=undef
          ccdlflags=' '
          cccdlflags=' '
          lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg
          -libpath:"c:\perl\lib\CORE"        -machine:x86'



@INC for perl 5.27.9:
          lib
          C:/p527/srcnew/lib


Environment for perl 5.27.9:
          CYGWIN=tty
          HOME (unset)
          LANG (unset)
          LANGUAGE (unset)
          LD_LIBRARY_PATH=/usr/lib/x86:/usr/X11R6/lib
          LOGDIR (unset)
          PATH=C:\WINDOWS\system32;C:\Program Files (x86)\Microsoft Visual
Studio 9.0\VC\BIN;C:\Program Files\Microsoft
SDKs\Windows\v6.0A\bin;C:\Perl\bin;C:\WINDOWS;C:\Program Files
(x86)\Microsoft Visual Studio 9.0\Common7\IDE;C:\Program Files
(x86)\Git\bin;C:\sp3220\c\bin;
          PERL_BADLANG (unset)
          SHELL (unset)

From @khwilliamson

I attempted to fixt this in
0cb2911

Please verify one way or another if it is fixed.
--
Karl Williamson

The RT System itself - Status changed from 'new' to 'open'

From @bulk88

On Tue, 03 Apr 2018 07​:56​:35 -0700, khw wrote​:

I attempted to fixt this in
0cb2911

Please verify one way or another if it is fixed.

Fixed for VC, but a new warning introduced with GCC in every .o

gcc -c -I.. -I.\include -I. -I.. -DWIN32 -DPERLDLL -DPERL_CORE -s -O2 -DPERL_T
EXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DWIN32_NO_REGISTRY
-DUSE_PERLIO -fwrapv -fno-strict-aliasing -mms-bitfields -o..\regexec.o ..\rege
xec.c
In file included from ..\perl.h​:5444​:0,
  from ..\regexec.c​:75​:
..\inline.h​: In function 'S_variant_under_utf8_count'​:
..\inline.h​:647​:21​: warning​: right shift count >= width of type
  >> ((PERL_WORDSIZE - 1) * CHARBITS);
  ^

--
bulk88 ~ bulk88 at hotmail.com

From @bulk88

On Tue, 03 Apr 2018 13​:37​:37 -0700, bulk88 wrote​:

On Tue, 03 Apr 2018 07​:56​:35 -0700, khw wrote​:

I attempted to fixt this in
0cb2911

Please verify one way or another if it is fixed.

Fixed for VC,

Actually, this made things worse, this commit (SHA-1​: 0cb2911 inline.h​: Silence compiler warning) causes malloc corruption/segv/free to wrong pool when running "re/regexp.t" and a couple other regex related .t files . Example callstack of a crash.

  perl527.dll!Perl_safesysfree(void * where=0x00dd57b4) Line 354 + 0x6 C
  perl527.dll!Perl_sv_clear(interpreter * my_perl=0x0038605c, sv * const orig_sv=0x0093c38c) Line 6737 + 0xc C
  perl527.dll!Perl_sv_free2(interpreter * my_perl=0x0038605c, sv * const sv=0x0093c38c, const unsigned long rc=1) Line 7039 + 0xd C
  perl527.dll!S_SvREFCNT_dec(interpreter * my_perl=0x0038605c, sv * sv=0x0093c38c) Line 191 + 0x11 C
  perl527.dll!Perl_pregfree2(interpreter * my_perl=0x0038605c, p5rx * rx=0x00a99a44) Line 20076 + 0x1a C
  perl527.dll!Perl_sv_clear(interpreter * my_perl=0x0038605c, sv * const orig_sv=0x00a99a44) Line 6574 + 0xd C
  perl527.dll!Perl_sv_free2(interpreter * my_perl=0x0038605c, sv * const sv=0x00a99a44, const unsigned long rc=1) Line 7039 + 0xd C
  perl527.dll!S_SvREFCNT_dec(interpreter * my_perl=0x0038605c, sv * sv=0x00a99a44) Line 191 + 0x11 C
  perl527.dll!Perl_op_clear(interpreter * my_perl=0x0038605c, op * o=0x00af9170) Line 1065 + 0x4b C
  perl527.dll!Perl_op_free(interpreter * my_perl=0x0038605c, op * o=0x00af9170) Line 869 + 0xd C
  perl527.dll!Perl_op_free(interpreter * my_perl=0x0038605c, op * o=0x00af912c) Line 852 + 0xd C
  perl527.dll!Perl_leave_scope(interpreter * my_perl=0x0038605c, long base=16) Line 1127 + 0xd C
  perl527.dll!S_pop_eval_context_maybe_croak(interpreter * my_perl=0x0038605c, context * cx=0x0038bb5c, sv * errsv=0x00000000, int action=0) Line 1632 + 0x1e C
  perl527.dll!Perl_pp_leaveeval(interpreter * my_perl=0x0038605c) Line 4554 + 0x13 C
  perl527.dll!Perl_runops_debug(interpreter * my_perl=0x0038605c) Line 2536 + 0xd C
  perl527.dll!S_run_body(interpreter * my_perl=0x0038605c, long oldscope=1) Line 2736 + 0xd C
  perl527.dll!perl_run(interpreter * my_perl=0x0038605c) Line 2660 C
  perl527.dll!RunPerl(int argc=4, char * * argv=0x00382cf0, char * * env=0x00385228) Line 252 + 0x9 C++
  perl.exe!main(int argc=4, char * * argv=0x00382cf0, char * * env=0x00383160) Line 39 + 0x12 C
  perl.exe!mainCRTStartup() Line 398 + 0xe C
  kernel32.dll!_BaseProcessStart@​4() + 0x23

--
bulk88 ~ bulk88 at hotmail.com

From @khwilliamson

On Tue, 03 Apr 2018 20​:25​:06 -0700, bulk88 wrote​:

On Tue, 03 Apr 2018 13​:37​:37 -0700, bulk88 wrote​:

On Tue, 03 Apr 2018 07​:56​:35 -0700, khw wrote​:

I attempted to fixt this in
0cb2911

Please verify one way or another if it is fixed.

Fixed for VC,

Actually, this made things worse, this commit (SHA-1​:
0cb2911 inline.h​: Silence compiler
warning) causes malloc corruption/segv/free to wrong pool when running
"re/regexp.t" and a couple other regex related .t files . Example
callstack of a crash.

-----------------
perl527.dll!Perl_safesysfree(void * where=0x00dd57b4) Line 354 + 0x6
C
perl527.dll!Perl_sv_clear(interpreter * my_perl=0x0038605c, sv * const
orig_sv=0x0093c38c) Line 6737 + 0xc C
perl527.dll!Perl_sv_free2(interpreter * my_perl=0x0038605c, sv * const
sv=0x0093c38c, const unsigned long rc=1) Line 7039 + 0xd C
perl527.dll!S_SvREFCNT_dec(interpreter * my_perl=0x0038605c, sv *
sv=0x0093c38c) Line 191 + 0x11 C
perl527.dll!Perl_pregfree2(interpreter * my_perl=0x0038605c, p5rx *
rx=0x00a99a44) Line 20076 + 0x1a C
perl527.dll!Perl_sv_clear(interpreter * my_perl=0x0038605c, sv * const
orig_sv=0x00a99a44) Line 6574 + 0xd C
perl527.dll!Perl_sv_free2(interpreter * my_perl=0x0038605c, sv * const
sv=0x00a99a44, const unsigned long rc=1) Line 7039 + 0xd C
perl527.dll!S_SvREFCNT_dec(interpreter * my_perl=0x0038605c, sv *
sv=0x00a99a44) Line 191 + 0x11 C
perl527.dll!Perl_op_clear(interpreter * my_perl=0x0038605c, op *
o=0x00af9170) Line 1065 + 0x4b C
perl527.dll!Perl_op_free(interpreter * my_perl=0x0038605c, op *
o=0x00af9170) Line 869 + 0xd C
perl527.dll!Perl_op_free(interpreter * my_perl=0x0038605c, op *
o=0x00af912c) Line 852 + 0xd C
perl527.dll!Perl_leave_scope(interpreter * my_perl=0x0038605c, long
base=16) Line 1127 + 0xd C
perl527.dll!S_pop_eval_context_maybe_croak(interpreter *
my_perl=0x0038605c, context * cx=0x0038bb5c, sv * errsv=0x00000000,
int action=0) Line 1632 + 0x1e C
perl527.dll!Perl_pp_leaveeval(interpreter * my_perl=0x0038605c) Line
4554 + 0x13 C
perl527.dll!Perl_runops_debug(interpreter * my_perl=0x0038605c) Line
2536 + 0xd C
perl527.dll!S_run_body(interpreter * my_perl=0x0038605c, long
oldscope=1) Line 2736 + 0xd C
perl527.dll!perl_run(interpreter * my_perl=0x0038605c) Line 2660
C
perl527.dll!RunPerl(int argc=4, char * * argv=0x00382cf0, char * *
env=0x00385228) Line 252 + 0x9 C++
perl.exe!main(int argc=4, char * * argv=0x00382cf0, char * *
env=0x00383160) Line 39 + 0x12 C
perl.exe!mainCRTStartup() Line 398 + 0xe C
kernel32.dll!_BaseProcessStart@​4() + 0x23
------------------

I reverted the patch that made things worse. Now, please try the attached patch and let me know if it works.
--
Karl Williamson

From @khwilliamson

From 47f6b49fadd53fac49e3c0f7e40e20a0589ddad5 Mon Sep 17 00:00:00 2001
From: Karl Williamson <khw@cpan.org>
Date: Thu, 5 Apr 2018 08:27:32 -0600
Subject: [PATCH 3/3] XXX for bulk88

---
 inline.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/inline.h b/inline.h
index 8c20e7583a..dc74c2daed 100644
--- a/inline.h
+++ b/inline.h
@@ -641,9 +641,11 @@ S_variant_under_utf8_count(const U8* const s, const U8* const e)
         /* Process per-word as long as we have at least a full word left */
         do {    /* Commit 03c1e4ab1d6ee9062fb3f94b0ba31db6698724b1 contains an
                    explanation of how this works */
-            count += ((((* (PERL_UINTMAX_T *) x) & PERL_VARIANTS_WORD_MASK) >> 7)
+            PERL_UINTMAX_T increment
+                = ((((* (PERL_UINTMAX_T *) x) & PERL_VARIANTS_WORD_MASK) >> 7)
                       * PERL_COUNT_MULTIPLIER)
                     >> ((PERL_WORDSIZE - 1) * CHARBITS);
+            count += (Size_t) increment;
             x += PERL_WORDSIZE;
         } while (x + PERL_WORDSIZE <= e);
     }
-- 
2.11.0

From @bulk88

On Thu, 05 Apr 2018 07​:31​:19 -0700, khw wrote​:

I reverted the patch that made things worse. Now, please try the
attached patch and let me know if it works.

New patch works okay on VC 32 bit and GCC 32 bit no warnings on either.

--
bulk88 ~ bulk88 at hotmail.com

From @khwilliamson

Pushed as commit e586328
--
Karl Williamson

@khwilliamson - Status changed from 'open' to 'resolved'