Skip Menu |
Report information
Id: 132177
Status: pending release
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: njh [at] bandsman.co.uk
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: medium
Type: core
Perl Version: 5.26.1
Fixed In: 5.27.4



To: perlbug [...] perl.org
From: njh [...] bandsman.co.uk
Subject: 5.26.1 sanitize=undefined
Date: Thu, 28 Sep 2017 08:19:54 -0400
Download (untitled) / with headers
text/plain 4.5k
This is a bug report for perl from njh@bandsman.co.uk, generated with the help of perlbug 1.40 running under perl 5.26.1. ----------------------------------------------------------------- I've built a Perl5.26.1 with -fsanitized=undefined and I keep seeing this: pp_hot.c:4143:6: runtime error: null pointer passed as argument 1, which is declared to never be null Here is the errant line: a call to Copy() within PP(): if (UNLIKELY(items - 1 > AvMAX(av))) { SV **ary = AvALLOC(av); Renew(ary, items, SV*); AvMAX(av) = items - 1; AvALLOC(av) = ary; AvARRAY(av) = ary; } Copy(MARK+1,AvARRAY(av),items,SV*); /* <<<<<<<<<<<<<<<<<<<<< */ AvFILLp(av) = items - 1; } [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=medium --- Site configuration information for perl 5.26.1: Configured by njh at Wed Sep 27 17:47:22 EDT 2017. Summary of my perl5 (revision 5 version 26 subversion 1) configuration: Platform: osname=linux osvers=4.12.0-2-amd64 archname=x86_64-linux-thread-multi uname='linux microcenter 4.12.0-2-amd64 #1 smp debian 4.12.13-1 (2017-09-19) x86_64 gnulinux ' config_args='-de -Dprefix=/home/njh/perl5/perlbrew/perls/perl-5.26.1-sanitize-undefined -Duseithreads -Dusedevel -Accflags=-fsanitize=undefined -g -O2 -lubsan -Acc=gcc-7 -Aldflags=-fsanitize=undefined -g -lubsan -Alddlflags=-shared -O2 -L/usr/local/lib -fstack-protector-strong -fsanitize=undefined -g -lubsan -Aeval:scriptdir=/home/njh/perl5/perlbrew/perls/perl-5.26.1-sanitize-undefined/bin' hint=recommended useposix=true d_sigaction=define useithreads=define usemultiplicity=define use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc=' gcc-7' ccflags ='-D_REENTRANT -D_GNU_SOURCE -fsanitize=undefined -g -O2 -lubsan -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2' optimize='-O2' cppflags='-D_REENTRANT -D_GNU_SOURCE -fsanitize=undefined -g -O2 -lubsan -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='7.2.0' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld=' gcc-7' ldflags =' -fsanitize=undefined -g -lubsan -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/7/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.24.so so=so useshrplib=false libperl=libperl.a gnulibc_version='2.24' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags=' -shared -O2 -L/usr/local/lib -fstack-protector-strong -fsanitize=undefined -g -lubsan' --- @INC for perl 5.26.1: /home/njh/perl5/perlbrew/perls/perl-5.26.1-sanitize-undefined/lib/site_perl/5.26.1/x86_64-linux-thread-multi /home/njh/perl5/perlbrew/perls/perl-5.26.1-sanitize-undefined/lib/site_perl/5.26.1 /home/njh/perl5/perlbrew/perls/perl-5.26.1-sanitize-undefined/lib/5.26.1/x86_64-linux-thread-multi /home/njh/perl5/perlbrew/perls/perl-5.26.1-sanitize-undefined/lib/5.26.1 --- Environment for perl 5.26.1: HOME=/home/njh LANG=en_GB.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH=/usr/local/lib:: LOGDIR (unset) PATH=/home/njh/perl5/perlbrew/bin:/home/njh/perl5/perlbrew/perls/perl-5.26.1-sanitize-undefined/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/njh/bin:/sbin PERLBREW_BASHRC_VERSION=0.80 PERLBREW_HOME=/home/njh/.perlbrew PERLBREW_MANPATH=/home/njh/perl5/perlbrew/perls/perl-5.26.1-sanitize-undefined/man PERLBREW_PATH=/home/njh/perl5/perlbrew/bin:/home/njh/perl5/perlbrew/perls/perl-5.26.1-sanitize-undefined/bin PERLBREW_PERL=perl-5.26.1-sanitize-undefined PERLBREW_ROOT=/home/njh/perl5/perlbrew PERLBREW_VERSION=0.80 PERL_BADLANG (unset) SHELL=/bin/bash
RT-Send-CC: perl5-porters [...] perl.org
On Thu, 28 Sep 2017 12:21:21 GMT, njh@bandsman.co.uk wrote: Show quoted text
> > This is a bug report for perl from njh@bandsman.co.uk, > generated with the help of perlbug 1.40 running under perl 5.26.1. > > > ----------------------------------------------------------------- > I've built a Perl5.26.1 with -fsanitized=undefined and I keep seeing > this: > > pp_hot.c:4143:6: runtime error: null pointer passed as argument 1, > which is declared to never be null > > Here is the errant line: a call to Copy() within PP(): > > if (UNLIKELY(items - 1 > AvMAX(av))) { > SV **ary = AvALLOC(av); > Renew(ary, items, SV*); > AvMAX(av) = items - 1; > AvALLOC(av) = ary; > AvARRAY(av) = ary; > } > > Copy(MARK+1,AvARRAY(av),items,SV*); /* <<<<<<<<<<<<<<<<<<<<< */ > AvFILLp(av) = items - 1; > } > >
1. This part of the codebase has been modified in the 5.27.* development series. (See commit f14cf3632 (Tony Cook 2017-08-14).) Could you re-try this on perl 5 blead? 2. In order to reproduce a problem showing up during 'make', we first look at the config_args reported in the 'perl -V' data in the ticket. Yours reads: Show quoted text
> config_args='-de -Dprefix=/home/njh/perl5/perlbrew/perls/perl- > 5.26.1-sanitize-undefined -Duseithreads -Dusedevel -Accflags=- > fsanitize=undefined -g -O2 -lubsan -Acc=gcc-7 -Aldflags=- > fsanitize=undefined -g -lubsan -Alddlflags=-shared -O2 > -L/usr/local/lib -fstack-protector-strong -fsanitize=undefined -g > -lubsan -Aeval:scriptdir=/home/njh/perl5/perlbrew/perls/perl-5.26.1- > sanitize-undefined/bin'
This is not the easiest data to work with because several of your configuration switches appear more than once: ##### -g => 3 -lubsan => 3 -O2 => 2 -Acc=gcc-7 => 1 -Accflags=-fsanitize=undefined => 1 -Aeval:scriptdir=/home/njh/perl5/perlbrew/perls/perl-5.26.1-sanitize-undefined/bin => 1 -Alddlflags=-shared => 1 -Aldflags=-fsanitize=undefined => 1 -Dprefix=/home/njh/perl5/perlbrew/perls/perl-5.26.1-sanitize-undefined => 1 -Dusedevel => 1 -Duseithreads => 1 -L/usr/local/lib => 1 -de => 1 -fsanitize=undefined => 1 -fstack-protector-strong => 1 ##### Someone attempting to reproduce your problem doesn't necessarily know what all the config args mean, nor whether or not they can appear more than once. Can the duplicates for the first 3 args listed above be cleaned up? 3. When analyzing a problem which shows up during 'make', it's often helpful to proceed by configuring with the smallest possible set of config_args that reproduces the problem. I configured perl 5 blead (commit ad3af58cb) as follows: ##### $> sh ./Configure -des -Dusedevel -Accflags=-fsanitize=undefined -Aldflags=-fsanitize=undefined ##### I then ran 'make test_harness'. I got two test failures and plenty of compilation warnings. ##### $ ./perl -Ilib t/run/fresh_perl.t 1..92 ok 1 - $a = ":="; @_ = split /($a)/o, "a:=b:=c"; print "@... not ok 2 - $cusp = ~0 ^ (~0 >> 1); # Failed test 2 - $cusp = ~0 ^ (~0 >> 1); at ./test.pl line 1059 # got "pp.c:2717:7: runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be represented in type \'long int\'\npp.c:2815:2: runtime error: negation of -9223372036854775808 cannot be represented in type \'long int\'; cast to an unsigned type to negate this value to itself\n7 0 0 8 !" # expected "7 0 0 8 !" # PROG: # $cusp = ~0 ^ (~0 >> 1); # use integer; # $, = " "; # print +($cusp - 1) % 8, $cusp % 8, -$cusp % 8, 8 | (($cusp + 1) % 8 + 7), "!\n"; # STATUS: 0 ok 3 - $foo=undef; $foo->go; ... $ ./perl -Ilib t/run/switchd.t 1..21 ok 1 - Got debugging output: 1 ... ok 19 - -d does not conflict with sort optimisations not ok 20 - $DB::single set to overload # Failed test 20 - $DB::single set to overload at t/run/switchd.t line 290 # got "pp.c:2631:7: runtime error: signed integer overflow: 9999999999 * 10000000000 cannot be represented in type \'long int\'\ndebugged\n" # expected "debugged\n" ok 21 - putenv does not interfere with PERL5OPT parsing ##### pp.c:2717:7: runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be represented in type 'long int' pp.c:2631:7: runtime error: signed integer overflow: 9999999999 * 10000000000 cannot be represented in type 'long int' pp.c:2631:7: runtime error: signed integer overflow: 9999999999 * 10000000000 cannot be represented in type 'long int' ... # Testing for subsecond file timestamps (mtime) in /home/jkeenan/gitwork/perl/dist/Time-HiRes # Subsecond file timestamps in /home/jkeenan/gitwork/perl/dist/Time-HiRes: OK pp.c:2717:7: runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be represented in type 'long int' pp.c:2815:2: runtime error: negation of -9223372036854775808 cannot be represented in type 'long int'; cast to an unsigned type to negate this value to itself pp.c:2706:7: runtime error: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long int' pp.c:2650:17: runtime error: negation of -9223372036854775808 cannot be represented in type 'long int'; cast to an unsigned type to negate this value to itself ##### Could you configure and build perl with gcc-7 with the shorter list of config_args above? Thank you very much. -- James E Keenan (jkeenan@cpan.org)
From: Nigel Horne <njh [...] bandsman.co.uk>
Subject: Re: [perl #132177] 5.26.1 sanitize=undefined
To: <perlbug-followup [...] perl.org>
Date: Sat, 30 Sep 2017 10:35:17 -0400
Download (untitled) / with headers
text/plain 303b
James: Thank you for your reply. ##### $> sh ./Configure -des -Dusedevel -Accflags=-fsanitize=undefined -Aldflags=-fsanitize=undefined ##### I have done the same on 5.26.1 and run ‘make test_harness’. The output is very large. I’ll attach it to the on-line ticket on RT. -Nigel
To: <perlbug-followup [...] perl.org>
From: Nigel Horne <njh [...] bandsman.co.uk>
Subject: Re: [perl #132177] 5.26.1 sanitize=undefined
Date: Sat, 30 Sep 2017 10:55:50 -0400
Download (untitled) / with headers
text/plain 863b
Well that didn’t work either – it bounced that the file was too big. If you need the output of make test_harness can you give me a way to send it to you? -Nigel On 30/9/17, 10:50, "Nigel Horne" <njh@bandsman.co.uk> wrote: Looks like there is no way to attach a file to the ticket! I’m therefore going to attach to this e-mail. -Nigel On 30/9/17, 10:35, "Nigel Horne" <njh@bandsman.co.uk> wrote: James: Thank you for your reply. ##### $> sh ./Configure -des -Dusedevel -Accflags=-fsanitize=undefined -Aldflags=-fsanitize=undefined ##### I have done the same on 5.26.1 and run ‘make test_harness’. The output is very large. I’ll attach it to the on-line ticket on RT. -Nigel
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 3.3k
On Sat, 30 Sep 2017 14:56:27 GMT, njh@bandsman.co.uk wrote: Show quoted text
> Well that didn’t work either – it bounced that the file was too big. > If you need the output of make test_harness can you give me a way to > send it to you? > > -Nigel > > On 30/9/17, 10:50, "Nigel Horne" <njh@bandsman.co.uk> wrote: > > Looks like there is no way to attach a file to the ticket! > > I’m therefore going to attach to this e-mail. > > -Nigel > > On 30/9/17, 10:35, "Nigel Horne" <njh@bandsman.co.uk> wrote: > > James: > > Thank you for your reply. > > ##### > $> sh ./Configure -des -Dusedevel -Accflags=-fsanitize=undefined > -Aldflags=-fsanitize=undefined > ##### > > I have done the same on 5.26.1 and run ‘make test_harness’. The > output is very large. I’ll attach it to the on-line ticket on RT. > > -Nigel > > > >
1. Here is my current hypothesis. There *are* problems when you configure with: ##### 'Accflags=-fsanitize=undefined -Aldflags=-fsanitize=undefined' ##### ... in *both* perl-5.26.1 and perl 5 blead (e.g., commit 582a8ad9). However, the problems are much *fewer* in blead (perhaps as a result of the earlier cited commit f14cf3632 (Tony Cook 2017-08-14)). I checked out the 'v5.26.1' tag into a local branch, then configured and built as follows: ##### sh ./Configure -des -Dusedevel -Accflags=-fsanitize=undefined -Aldflags=-fsanitize=undefined && make test_prep ##### I didn't record the output of 'make test_prep', but it was visibly littered with compilation warnings like this: ##### pp_hot.c:4143:6: runtime error: null pointer passed as argument 1, which is declared to never be null /usr/include/x86_64-linux-gnu/bits/string3.h:53:10: runtime error: null pointer passed as argument 1, which is declared to never be null ##### I then ran this command to capture the output (attached) of two test files: ##### cd t;./perl harness -v run/fresh_perl.t run/switchd.t 2>&1 | tee /tmp/132177-5.26.1-sanitize-undefined-output.txt; cd - ##### I then cleaned my checkout directory, checked out blead, configured and built as above -- and this time 'make test_prep' output appeared to be quite compilation-warning free. I then ran those two test files and captured their output (attached) at this commit ##### cd t;./perl harness -v run/fresh_perl.t run/switchd.t 2>&1 | tee /tmp/132177-582a8ad9-sanitize-undefined-output.txt; cd - ##### As you can see, I had test failures at both 5.26.1 and at blead -- but far fewer test failures at blead and far fewer C-level warnings during the tests at blead. 2. The above was run with gcc version 5.4.0. You are running (I think) with gcc 7.2.0. We know from smoke test reports (e.g., http://perl5.test-smoke.org/report/58240) and discussion on #p5p that gcc 7.2.* throws a ton of compilation warnings for which we have not previously accounted. So let's temporarily set aside the compilation warnings you're getting during 'make test_prep' and just focus on the FAILs in the tests. I recommend that for each of 5.26.1 and perl 5 blead, you: a. Configure and build perl with the sanitize undefined switches as above; b. Run 'make test' (or 'make test_harness') for the purpose of identifying FAILing test files; c. Then run just capture and attach the output of *only* the failing test files. Those two files should be small enough to be attachable. Thank you very much. -- James E Keenan (jkeenan@cpan.org)
Subject: 132177-5.26.1-sanitize-undefined-output.txt

Message body is not shown because it is too large.

Subject: 132177-582a8ad9-sanitize-undefined-output.txt
# Failed test 2 - $cusp = ~0 ^ (~0 >> 1); at ./test.pl line 1059 # got "pp.c:2717:7: runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be represented in type \'long int\'\npp.c:2815:2: runtime error: negation of -9223372036854775808 cannot be represented in type \'long int\'; cast to an unsigned type to negate this value to itself\n7 0 0 8 !" # expected "7 0 0 8 !" # PROG: # $cusp = ~0 ^ (~0 >> 1); # use integer; # $, = " "; # print +($cusp - 1) % 8, $cusp % 8, -$cusp % 8, 8 | (($cusp + 1) % 8 + 7), "!\n"; # STATUS: 0 # Failed test 20 - $DB::single set to overload at run/switchd.t line 290 # got "pp.c:2631:7: runtime error: signed integer overflow: 9999999999 * 10000000000 cannot be represented in type \'long int\'\ndebugged\n" # expected "debugged\n" run/switchd.t ..... 1..21 ok 1 - Got debugging output: 1 ok 2 - Got debugging output: 2 ok 3 - Got debugging output: 3 ok 4 - The debugger can see the lines of the main program under \#!perl -d ok 5 - \#!perl -d:whatever does not throw line numbers off ok 6 - Subroutine redefinition works in the debugger [perl \#48332] ok 7 - Subroutines no longer found under their names can be called ok 8 - No crash when calling orphaned subroutine via goto & ok 9 - No crash when *DB::DB exists but not &DB::DB ok 10 - No crash when &DB::DB exists but isn't actually defined ok 11 - DB::DB works after '*DB::DB if 0' ok 12 - Recursive DB::DB does not clobber its own pad ok 13 - Copy on write does not mangle ${"_<-e"}[0] [perl \#118627] ok 14 - PERL5DB with embedded newlines ok 15 - DB::goto ok 16 - %DB::lsub is not vivified ok 17 - setting breakpoints without *DB::dbline aliased ok 18 - UTF8 length caches on $DB::sub are flushed ok 19 - -d does not conflict with sort optimisations not ok 20 - $DB::single set to overload ok 21 - putenv does not interfere with PERL5OPT parsing Failed 1/21 subtests run/fresh_perl.t .. 1..92 ok 1 - $a = ":="; @_ = split /($a)/o, "a:=b:=c"; print "@... not ok 2 - $cusp = ~0 ^ (~0 >> 1); ok 3 - $foo=undef; $foo->go; ok 4 - BEGIN ok 5 - $array[128]=1 ok 6 - $x=0x0eabcd; print $x->ref; ok 7 - chop ($str .= <DATA>); ok 8 - close ($banana); ok 9 - $x=2;$y=3;$x<$y ? $x : $y += 23;print $x; ok 10 - eval 'sub bar {print "In bar"}'; ok 11 - system './perl -ne "print if eof" /dev/null' ok 12 - chop($file = <DATA>); ok 13 - package N; ok 14 - $_="foo"; ok 15 - push(@a, 1, 2, 3,) ok 16 - quotemeta "" ok 17 - for ("ABCDE") { ok 18 - package FOO;sub new {bless {FOO => BAR}}; ok 19 - $_="foo"; ok 20 - print scalar ("foo","bar") ok 21 - sub by_number { $a <=> $b; };\# inline function for... ok 22 - sub NewShell ok 23 - { ok 24 - BEGIN { die "phooey\n" } ok 25 - BEGIN { 1/$zero } ok 26 - BEGIN { undef = 0 } ok 27 - { ok 28 - my @a; $a[2] = 1; for (@a) { $_ = 2 } print "@a\n" ok 29 - \# used to attach defelem magic to all immortal val... ok 30 - @a = ($a, $b, $c, $d) = (5, 6); ok 31 - print "ok\n" if (1E2<<1 == 200 and 3E4<<3 == 24000... ok 32 - print "ok\n" if ("\0" lt "\xFF"); ok 33 - open(H,'run/fresh_perl.t'); \# must be in the 't' d... ok 34 - sub thing { 0 || return qw(now is the time) } ok 35 - $ren = 'joy'; ok 36 - $stimpy = 'happy'; ok 37 - package p; ok 38 - @list = ([ 'one', 1 ], [ 'two', 2 ]); ok 39 - ($k, $s) = qw(x 0); ok 40 - my $a = 'outer'; ok 41 - $| = 1; ok 42 - $s = 0; ok 43 - sub foo { local $_ = shift; @_ = split; @_ } ok 44 - "A" =~ /(?{"{"})/ \# Check it outside of eval too ok 45 - /(?{"{"}})/ \# Check it outside of eval too ok 46 - BEGIN { @ARGV = qw(a b c d e) } ok 47 - \# fdopen from a system descriptor to a system desc... ok 48 - sub testme { my $a = "test"; { local $a = "new tes... ok 49 - package X; ok 50 - package X; ok 51 - package X; ok 52 - package X; ok 53 - package X; ok 54 - BEGIN { ok 55 - package X; ok 56 - re(); ok 57 - use strict; ok 58 - eval ' ok 59 - if (@ARGV) { print "" } ok 60 - $x = sub {}; ok 61 - \# moved to op/lc.t ok 62 - sub f { my $a = 1; my $b = 2; my $c = 3; my $d = 4... ok 63 - \# [perl \#3066] ok 64 - print qw(ab a\b a\\b); ok 65 - \# lexicals declared after the myeval() definition ... ok 66 - \# lexicals outside an eval"" should be visible ins... ok 67 - \# [ID 20001202.002 (\#4821)] and change \#8066 added... ok 68 - \# 20001210.003 (\#4893) mjd@plover.com ok 69 - \# Inaba Hiroto ok 70 - \# Nicholas Clark ok 71 - \# Vadim Konovalov ok 72 - \# David Dyck ok 73 - \# core dump in 20000716.007 (\#3516) ok 74 - \# Bug 20010515.004 (\#6998) ok 75 - \# Bug 20010506.041 (\#6952) ok 76 - my $foo = Bar->new(); ok 77 - (?{...}) compilation bounces on PL_rs ok 78 - scalar ref to file test operator segfaults on 5.6.1 [ID 20011127.155 (\#7947)] ok 79 - [ID 20011128.159 (\#7951)] 'X' =~ /\X/ segfault in 5.6.1 ok 80 - segfault in 5.6.1 within peep() ok 81 - example from Camel 5, ch. 15, pp.406 (with my) ok 82 - example from Camel 5, ch. 15, pp.406 (with our) ok 83 - example from Camel 5, ch. 15, pp.406 (with package vars) ok 84 - example from Camel 5, ch. 15, pp.406 (with use vars) ok 85 - \# test that closures generated by eval"" hold on t... ok 86 - [ID 20020623.009 (\#9728)] nested eval/sub segfaults ok 87 - [perl \#17951] Strange UTF error ok 88 - [perl \#20667] unicode regex vs non-unicode regex ok 89 - [perl \#8760] strangeness with utf8 and warn ok 90 - "\#75146: 27e904532594b7fb (fix for \#23810) introduces a \#regression" ok 91 - [perl \#112312] crash on syntax error ok 92 - [perl \#112312] crash on syntax error - another test Failed 1/92 subtests Test Summary Report ------------------- run/switchd.t (Wstat: 0 Tests: 21 Failed: 1) Failed test: 20 run/fresh_perl.t (Wstat: 0 Tests: 92 Failed: 1) Failed test: 2 Files=2, Tests=113, 1 wallclock secs ( 0.05 usr 0.00 sys + 0.19 cusr 0.06 csys = 0.30 CPU) Result: FAIL
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 586b
On Sat, 30 Sep 2017 18:23:03 GMT, jkeenan wrote: Show quoted text
> > ##### > cd t;./perl harness -v run/fresh_perl.t run/switchd.t 2>&1 | tee > /tmp/132177-582a8ad9-sanitize-undefined-output.txt; cd - > ##### > > As you can see, I had test failures at both 5.26.1 and at blead -- but > far fewer test failures at blead and far fewer C-level warnings during > the tests at blead. >
I'm attaching a file with the parts of pp.c that are referenced in the test failures I got on blead on linux using gcc-5.4.0 with the "sanitize=undefined" configuration switches. -- James E Keenan (jkeenan@cpan.org)
Subject: problematic.sanitize.undefined.pp.c.txt
t/run/fresh_perl.t 2711 PP(pp_i_subtract) 2712 { 2713 dSP; dATARGET; 2714 tryAMAGICbin_MG(subtr_amg, AMGf_assign); 2715 { 2716 dPOPTOPiirl_ul_nomg; 2717 SETi( left - right ); 2718 RETURN; 2719 } 2720 } ... 2807 PP(pp_i_negate) 2808 { 2809 dSP; dTARGET; 2810 tryAMAGICun_MG(neg_amg, 0); 2811 if (S_negate_string(aTHX)) return NORMAL; 2812 { 2813 SV * const sv = TOPs; 2814 IV const i = SvIV_nomg(sv); 2815 SETi(-i); 2816 return NORMAL; 2817 } 2818 } t/run/switchd.t 2625 PP(pp_i_multiply) 2626 { 2627 dSP; dATARGET; 2628 tryAMAGICbin_MG(mult_amg, AMGf_assign); 2629 { 2630 dPOPTOPiirl_nomg; 2631 SETi( left * right ); 2632 RETURN; 2633 } 2634 }
From: Zefram <zefram [...] fysh.org>
Subject: Re: [perl #132177] 5.26.1 sanitize=undefined
Date: Sat, 30 Dec 2017 12:23:08 +0000
To: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 135b
The code in question was fixed by commit f14cf3632059d421de83cf901c7e849adc1fcd03 in Perl 5.27.4. This ticket can be closed. -zefram
RT-Send-CC: perl5-porters [...] perl.org
On Sat, 30 Dec 2017 12:23:26 GMT, zefram@fysh.org wrote: Show quoted text
> The code in question was fixed by commit > f14cf3632059d421de83cf901c7e849adc1fcd03 in Perl 5.27.4. This ticket > can be closed. > > -zefram
Unfortunately, I cannot confirm that the problem has been solved. This morning I followed the same procedure I described in my post of Sep 30 above. I created a branch from blead at commit dce3f5c3. I then configured as follows: ##### sh ./Configure -des -Dusedevel -Accflags=-fsanitize=undefined -Aldflags=-fsanitize=undefined ##### ... and, using my default compiler gcc-5.4.1, ran 'make test_prep', capturing the output. The output of 'make' was free of warnings. I then re-ran the two test files in question and got the same failures as reported on Sep 30; see attachments for output of 'perl -V' and 'cd t;./perl harness -v run/fresh_perl.t run/switchd.t; cd -'. I repeated the process with gcc 7.2.0. While I got some build-time warnings, all but 3 of them were in cpan-upstream code. I got exactly the same two test failures. -- James E Keenan (jkeenan@cpan.org)
Subject: 132177-dce3f5c-sanitize-undefined-freshperl-switchd.output.txt
# Failed test 2 - $cusp = ~0 ^ (~0 >> 1); at ./test.pl line 1062 # got "pp.c:2701:7: runtime error: signed integer overflow: -9223372036854775808 - 1 cannot be represented in type \'long int\'\npp.c:2799:2: runtime error: negation of -9223372036854775808 cannot be represented in type \'long int\'; cast to an unsigned type to negate this value to itself\n7 0 0 8 !" # expected "7 0 0 8 !" # PROG: # $cusp = ~0 ^ (~0 >> 1); # use integer; # $, = " "; # print +($cusp - 1) % 8, $cusp % 8, -$cusp % 8, 8 | (($cusp + 1) % 8 + 7), "!\n"; # STATUS: 0 # Failed test 20 - $DB::single set to overload at run/switchd.t line 290 # got "pp.c:2615:7: runtime error: signed integer overflow: 9999999999 * 10000000000 cannot be represented in type \'long int\'\ndebugged\n" # expected "debugged\n" run/switchd.t ..... 1..21 ok 1 - Got debugging output: 1 ok 2 - Got debugging output: 2 ok 3 - Got debugging output: 3 ok 4 - The debugger can see the lines of the main program under \#!perl -d ok 5 - \#!perl -d:whatever does not throw line numbers off ok 6 - Subroutine redefinition works in the debugger [perl \#48332] ok 7 - Subroutines no longer found under their names can be called ok 8 - No crash when calling orphaned subroutine via goto & ok 9 - No crash when *DB::DB exists but not &DB::DB ok 10 - No crash when &DB::DB exists but isn't actually defined ok 11 - DB::DB works after '*DB::DB if 0' ok 12 - Recursive DB::DB does not clobber its own pad ok 13 - Copy on write does not mangle ${"_<-e"}[0] [perl \#118627] ok 14 - PERL5DB with embedded newlines ok 15 - DB::goto ok 16 - %DB::lsub is not vivified ok 17 - setting breakpoints without *DB::dbline aliased ok 18 - UTF8 length caches on $DB::sub are flushed ok 19 - -d does not conflict with sort optimisations not ok 20 - $DB::single set to overload ok 21 - putenv does not interfere with PERL5OPT parsing Failed 1/21 subtests run/fresh_perl.t .. 1..92 ok 1 - $a = ":="; @_ = split /($a)/o, "a:=b:=c"; print "@... not ok 2 - $cusp = ~0 ^ (~0 >> 1); ok 3 - $foo=undef; $foo->go; ok 4 - BEGIN ok 5 - $array[128]=1 ok 6 - $x=0x0eabcd; print $x->ref; ok 7 - chop ($str .= <DATA>); ok 8 - close ($banana); ok 9 - $x=2;$y=3;$x<$y ? $x : $y += 23;print $x; ok 10 - eval 'sub bar {print "In bar"}'; ok 11 - system './perl -ne "print if eof" /dev/null' ok 12 - chop($file = <DATA>); ok 13 - package N; ok 14 - $_="foo"; ok 15 - push(@a, 1, 2, 3,) ok 16 - quotemeta "" ok 17 - for ("ABCDE") { ok 18 - package FOO;sub new {bless {FOO => BAR}}; ok 19 - $_="foo"; ok 20 - print scalar ("foo","bar") ok 21 - sub by_number { $a <=> $b; };\# inline function for... ok 22 - sub NewShell ok 23 - { ok 24 - BEGIN { die "phooey\n" } ok 25 - BEGIN { 1/$zero } ok 26 - BEGIN { undef = 0 } ok 27 - { ok 28 - my @a; $a[2] = 1; for (@a) { $_ = 2 } print "@a\n" ok 29 - \# used to attach defelem magic to all immortal val... ok 30 - @a = ($a, $b, $c, $d) = (5, 6); ok 31 - print "ok\n" if (1E2<<1 == 200 and 3E4<<3 == 24000... ok 32 - print "ok\n" if ("\0" lt "\xFF"); ok 33 - open(H,'run/fresh_perl.t'); \# must be in the 't' d... ok 34 - sub thing { 0 || return qw(now is the time) } ok 35 - $ren = 'joy'; ok 36 - $stimpy = 'happy'; ok 37 - package p; ok 38 - @list = ([ 'one', 1 ], [ 'two', 2 ]); ok 39 - ($k, $s) = qw(x 0); ok 40 - my $a = 'outer'; ok 41 - $| = 1; ok 42 - $s = 0; ok 43 - sub foo { local $_ = shift; @_ = split; @_ } ok 44 - "A" =~ /(?{"{"})/ \# Check it outside of eval too ok 45 - /(?{"{"}})/ \# Check it outside of eval too ok 46 - BEGIN { @ARGV = qw(a b c d e) } ok 47 - \# fdopen from a system descriptor to a system desc... ok 48 - sub testme { my $a = "test"; { local $a = "new tes... ok 49 - package X; ok 50 - package X; ok 51 - package X; ok 52 - package X; ok 53 - package X; ok 54 - BEGIN { ok 55 - package X; ok 56 - re(); ok 57 - use strict; ok 58 - eval ' ok 59 - if (@ARGV) { print "" } ok 60 - $x = sub {}; ok 61 - \# moved to op/lc.t ok 62 - sub f { my $a = 1; my $b = 2; my $c = 3; my $d = 4... ok 63 - \# [perl \#3066] ok 64 - print qw(ab a\b a\\b); ok 65 - \# lexicals declared after the myeval() definition ... ok 66 - \# lexicals outside an eval"" should be visible ins... ok 67 - \# [ID 20001202.002 (\#4821)] and change \#8066 added... ok 68 - \# 20001210.003 (\#4893) mjd@plover.com ok 69 - \# Inaba Hiroto ok 70 - \# Nicholas Clark ok 71 - \# Vadim Konovalov ok 72 - \# David Dyck ok 73 - \# core dump in 20000716.007 (\#3516) ok 74 - \# Bug 20010515.004 (\#6998) ok 75 - \# Bug 20010506.041 (\#6952) ok 76 - my $foo = Bar->new(); ok 77 - (?{...}) compilation bounces on PL_rs ok 78 - scalar ref to file test operator segfaults on 5.6.1 [ID 20011127.155 (\#7947)] ok 79 - [ID 20011128.159 (\#7951)] 'X' =~ /\X/ segfault in 5.6.1 ok 80 - segfault in 5.6.1 within peep() ok 81 - example from Camel 5, ch. 15, pp.406 (with my) ok 82 - example from Camel 5, ch. 15, pp.406 (with our) ok 83 - example from Camel 5, ch. 15, pp.406 (with package vars) ok 84 - example from Camel 5, ch. 15, pp.406 (with use vars) ok 85 - \# test that closures generated by eval"" hold on t... ok 86 - [ID 20020623.009 (\#9728)] nested eval/sub segfaults ok 87 - [perl \#17951] Strange UTF error ok 88 - [perl \#20667] unicode regex vs non-unicode regex ok 89 - [perl \#8760] strangeness with utf8 and warn ok 90 - "\#75146: 27e904532594b7fb (fix for \#23810) introduces a \#regression" ok 91 - [perl \#112312] crash on syntax error ok 92 - [perl \#112312] crash on syntax error - another test Failed 1/92 subtests Test Summary Report ------------------- run/switchd.t (Wstat: 0 Tests: 21 Failed: 1) Failed test: 20 run/fresh_perl.t (Wstat: 0 Tests: 92 Failed: 1) Failed test: 2 Files=2, Tests=113, 1 wallclock secs ( 0.05 usr 0.00 sys + 0.22 cusr 0.05 csys = 0.32 CPU) Result: FAIL
Subject: 132177-dce3f5c-sanitize-undefined-perl-V.txt
Summary of my perl5 (revision 5 version 27 subversion 8) configuration: Commit id: dce3f5c3fd788f1c2e451e3760f05a347c949eff Platform: osname=linux osvers=4.4.0-104-generic archname=x86_64-linux uname='linux zareason 4.4.0-104-generic #127-ubuntu smp mon dec 11 12:16:42 utc 2017 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Dusedevel -Accflags=-fsanitize=undefined -Aldflags=-fsanitize=undefined' hint=recommended useposix=true d_sigaction=define useithreads=undef usemultiplicity=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cc' ccflags ='-fsanitize=undefined -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' optimize='-O2' cppflags='-fsanitize=undefined -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='5.4.1 20160904' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fsanitize=undefined -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64 libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.23.so so=so useshrplib=false libperl=libperl.a gnulibc_version='2.23' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong' Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP PERL_OP_PARENT PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Dec 30 2017 08:47:37 %ENV: PERL2DIR="/home/jkeenan/gitwork/perl2" PERLBREW_BASHRC_VERSION="0.78" PERLBREW_HOME="/home/jkeenan/.perlbrew" PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.26.0/man" PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin:/home/jkeenan/perl5/perlbrew/perls/perl-5.26.0/bin" PERLBREW_PERL="perl-5.26.0" PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew" PERLBREW_VERSION="0.78" PERL_WORKDIR="/home/jkeenan/gitwork/perl" @INC: lib /usr/local/lib/perl5/site_perl/5.27.8/x86_64-linux /usr/local/lib/perl5/site_perl/5.27.8 /usr/local/lib/perl5/5.27.8/x86_64-linux /usr/local/lib/perl5/5.27.8
Subject: Re: [perl #132177] 5.26.1 sanitize=undefined
From: Zefram <zefram [...] fysh.org>
Date: Sat, 30 Dec 2017 17:34:54 +0000
To: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 943b
James E Keenan via RT wrote: Show quoted text
>I got exactly the same two test failures.
Those test failures are nothing to do with the problem that this ticket is about, which was a call to memcpy() with a null argument. That call has been fixed by the insertion of a conditional checking that there is a non-zero amount of data to copy. Your test failures are complaining of signed integer overflow, which is off the topic of this ticket, but anyway is not a bug. By default such overflow is a bug in C code, but perl has different arrangements. We rely on signed integer arithmetic wrapping, and by default we feed gcc the -fwrapv option to tell it to provide those semantics. That option doesn't appear in your ccflags, so something's wrong in how your ccflags were determined. If there's a claim that Configure failed to insert the -fwrapv option when it should have, that would be worth opening a new ticket. It's off topic for this one. -zefram


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org