From @jkeenan

Today I happened to try to compile blead on a FreeBSD-11 host on which I
don't normally test blead. Unbeknownst to me, this host was running
clang 6.0.0 as its default C-compiler. The other FreeBSD-11
installations I use (all VMs) use clang 4.0.0. During 'make' I observed
a great number of warnings, which surprised me because, in this dev
cycle, we've been extremely good about keeping our builds -- regardless
of whether we're using gcc, g++ or clang -- warnings-free. (There are
some warnings generated with some compilers during the compilation of
cpan-upstream modules that ship with the core; we don't control those.)

During smoke-testing on FreeBSD-11 ("kid51-freebsd-11-install-3"), I
normally compile perl with "ccache clang -Qunused-arguments
-fcolor-diagnostics version 4.2.1 Compatible FreeBSD Clang 4.0.0" (see​:
http​://perl5.test-smoke.org/report/66178). Similarly, when smoke
testing on FreeBSD-11, Carlos Guevara compiles with "cc version 4.2.1
Compatible FreeBSD Clang 4.0.0" (see​:
http​://perl5.test-smoke.org/report/66174). So we have a lot of
experience with clang 4.0.0 -- but apparently none with clang 6.0.0.

So to investigate I installed the "llvm60" package on that VM and
configured with 'config_args' like this​:

#####
-des -Dusedevel -Dcc=clang60 -Duseithreads -Doptimize=-O2 -pipe
-fstack-protector -fno-strict-aliasing
#####

I then called 'make' and recorded the output. I got the same number of
warnings with clang60 there as I did on the host where I originally
noticed them. The comparable number of warnings with "regular" clang
(4.0.0) was zero. 'make test_harness' passed in all cases.

I am attaching 4 files, all from the same FreeBSD-11 VM. They hold (a)
the 'make' output from compiling with clang 4.0.0 and 6.0.0,
respectively; and (b) the perl -V output from 4.0.0 and 6.0.0 respectively.

I'll leave it up to the pumpking as to whether this should be a blocker
for 5.28.0. There are more important blockers, to be sure, but it would
also be unfortunate to do a release where we have so many build-time
warnings that we could have dealt with earlier in the cycle had we been
testing with this version of the C-compiler.

Thank you very much.
Jim Keenan

From @jkeenan

Summary of my perl5 (revision 5 version 28 subversion 0) configuration​:
  Commit id​: 755a646
  Platform​:
  osname=freebsd
  osvers=11.1-release-p9
  archname=amd64-freebsd-thread-multi
  uname='freebsd kid51-freebsd-11-install-3 11.1-release-p9 freebsd 11.1-release-p9 #0​: tue apr 3 16​:59​:16 utc 2018 root@​amd64-builder.daemonology.net​:usrobjusrsrcsysgeneric amd64 '
  config_args='-des -Dusedevel -Dcc=clang60 -Duseithreads -Doptimize=-O2 -pipe -fstack-protector -fno-strict-aliasing'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='clang60'
  ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
  optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
  cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.2.1 Compatible Clang 6.0.0 (tags/RELEASE_600/final)'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='clang60'
  ldflags ='-pthread -Wl,-E -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/lib /usr/local/lib /usr/local/llvm60/lib/clang/6.0.0/lib /usr/lib
  libs=-lpthread -lgdbm -lm -lcrypt -lutil
  perllibs=-lpthread -lm -lcrypt -lutil
  libc=
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags=' '
  cccdlflags='-DPIC -fPIC'
  lddlflags='-shared -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  MULTIPLICITY
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_IMPLICIT_CONTEXT
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_ITHREADS
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  USE_REENTRANT_API
  Built under freebsd
  Compiled at May 25 2018 09​:00​:03
  %ENV​:
  PERL2DIR="/home/jkeenan/gitwork/perl2"
  PERL_WORKDIR="/home/jkeenan/gitwork/perl"
  @​INC​:
  lib
  /usr/local/lib/perl5/site_perl/5.28.0/amd64-freebsd-thread-multi
  /usr/local/lib/perl5/site_perl/5.28.0
  /usr/local/lib/perl5/5.28.0/amd64-freebsd-thread-multi
  /usr/local/lib/perl5/5.28.0

From @jkeenan

make.freebsd11.clang60.install3.output.txt

From @jkeenan

make.freebsd11.clang.install3.output.txt

From @jkeenan

Summary of my perl5 (revision 5 version 28 subversion 0) configuration​:
  Commit id​: 755a646
  Platform​:
  osname=freebsd
  osvers=11.1-release-p9
  archname=amd64-freebsd-thread-multi
  uname='freebsd kid51-freebsd-11-install-3 11.1-release-p9 freebsd 11.1-release-p9 #0​: tue apr 3 16​:59​:16 utc 2018 root@​amd64-builder.daemonology.net​:usrobjusrsrcsysgeneric amd64 '
  config_args='-des -Dusedevel -Duseithreads -Doptimize=-O2 -pipe -fstack-protector -fno-strict-aliasing'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
  optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
  cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.2.1 Compatible FreeBSD Clang 4.0.0 (tags/RELEASE_400/final 297347)'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags ='-pthread -Wl,-E -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/lib /usr/local/lib /usr/bin/../lib/clang/4.0.0/lib /usr/lib
  libs=-lpthread -lgdbm -lm -lcrypt -lutil
  perllibs=-lpthread -lm -lcrypt -lutil
  libc=
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags=' '
  cccdlflags='-DPIC -fPIC'
  lddlflags='-shared -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  MULTIPLICITY
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_IMPLICIT_CONTEXT
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_ITHREADS
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  USE_REENTRANT_API
  Built under freebsd
  Compiled at May 25 2018 08​:32​:34
  %ENV​:
  PERL2DIR="/home/jkeenan/gitwork/perl2"
  PERL_WORKDIR="/home/jkeenan/gitwork/perl"
  @​INC​:
  lib
  /usr/local/lib/perl5/site_perl/5.28.0/amd64-freebsd-thread-multi
  /usr/local/lib/perl5/site_perl/5.28.0
  /usr/local/lib/perl5/5.28.0/amd64-freebsd-thread-multi
  /usr/local/lib/perl5/5.28.0

From @jkeenan

On Fri, 25 May 2018 13​:42​:28 GMT, jkeenan@​pobox.com wrote​:

Today I happened to try to compile blead on a FreeBSD-11 host on which I
don't normally test blead. Unbeknownst to me, this host was running
clang 6.0.0 as its default C-compiler. The other FreeBSD-11
installations I use (all VMs) use clang 4.0.0. During 'make' I observed
a great number of warnings, which surprised me because, in this dev
cycle, we've been extremely good about keeping our builds -- regardless
of whether we're using gcc, g++ or clang -- warnings-free. (There are
some warnings generated with some compilers during the compilation of
cpan-upstream modules that ship with the core; we don't control those.)

During smoke-testing on FreeBSD-11 ("kid51-freebsd-11-install-3"), I
normally compile perl with "ccache clang -Qunused-arguments
-fcolor-diagnostics version 4.2.1 Compatible FreeBSD Clang 4.0.0" (see​:
http​://perl5.test-smoke.org/report/66178). Similarly, when smoke
testing on FreeBSD-11, Carlos Guevara compiles with "cc version 4.2.1
Compatible FreeBSD Clang 4.0.0" (see​:
http​://perl5.test-smoke.org/report/66174). So we have a lot of
experience with clang 4.0.0 -- but apparently none with clang 6.0.0.

So to investigate I installed the "llvm60" package on that VM and
configured with 'config_args' like this​:

#####
-des -Dusedevel -Dcc=clang60 -Duseithreads -Doptimize=-O2 -pipe
-fstack-protector -fno-strict-aliasing
#####

I then called 'make' and recorded the output. I got the same number of
warnings with clang60 there as I did on the host where I originally
noticed them. The comparable number of warnings with "regular" clang
(4.0.0) was zero. 'make test_harness' passed in all cases.

I am attaching 4 files, all from the same FreeBSD-11 VM. They hold (a)
the 'make' output from compiling with clang 4.0.0 and 6.0.0,
respectively; and (b) the perl -V output from 4.0.0 and 6.0.0 respectively.

I'll leave it up to the pumpking as to whether this should be a blocker
for 5.28.0. There are more important blockers, to be sure, but it would
also be unfortunate to do a release where we have so many build-time
warnings that we could have dealt with earlier in the cycle had we been
testing with this version of the C-compiler.

Thank you very much.
Jim Keenan

Since some of the 60+ warnings observed reported the same line of source code twice, those locations boil down to​:

#####
$ grep 'warning​:' make.freebsd11.clang60.install3.output.txt | cut -d ' ' -f1 | uniqop.c​:1218​:11​:
op.c​:5143​:33​:
op.c​:8048​:25​:
op.c​:10496​:31​:
perl.c​:1180​:10​:
perl.c​:3672​:14​:
perl.c​:3674​:30​:
perl.c​:3679​:14​:
perl.c​:3681​:30​:
regcomp.c​:16203​:36​:
regcomp.c​:16420​:23​:
util.c​:1960​:48​:
util.c​:2005​:36​:
util.c​:2008​:36​:
util.c​:2043​:3​:
mg.c​:1117​:39​:
mg.c​:1123​:51​:
mg.c​:2931​:16​:
mg.c​:2933​:32​:
mg.c​:2935​:16​:
mg.c​:2937​:46​:
sv.c​:14907​:23​:
sv.c​:15377​:33​:
scope.c​:1434​:11​:
pp_ctl.c​:2004​:23​:
pp_ctl.c​:2008​:34​:
pp_ctl.c​:3471​:34​:
pp_ctl.c​:3473​:34​:
pp_ctl.c​:3479​:6​:
utf8.c​:90​:35​:
op.c​:1218​:11​:
op.c​:5143​:33​:
op.c​:8048​:25​:
op.c​:10496​:31​:
perl.c​:1180​:10​:
perl.c​:3672​:14​:
perl.c​:3674​:30​:
perl.c​:3679​:14​:
perl.c​:3681​:30​:
B.xs​:95​:39​:
B.xs​:96​:39​:
re_comp.c​:16203​:36​:
re_comp.c​:16420​:23​:
#####

In every case the warning is​:

#####
performing pointer arithmetic on a null pointer has undefined behavior [-Wnull-pointer-arithmetic]
#####

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

The RT System itself - Status changed from 'new' to 'open'

From @iabyn

On Fri, May 25, 2018 at 06​:42​:28AM -0700, James E Keenan (via RT) wrote​:

I'll leave it up to the pumpking as to whether this should be a blocker
for 5.28.0. There are more important blockers, to be sure, but it would
also be unfortunate to do a release where we have so many build-time
warnings that we could have dealt with earlier in the cycle had we been
testing with this version of the C-compiler.

There are 69 warnings, all identical, and all resulting from the same two
macro definitions​: pWARN_ALL and pWARN_NONE.

The warnings are harmless, and we would almost certainly do more damage at
this late stage by trying to tweak the macros to silence the warnings.

--
You never really learn to swear until you learn to drive.

From @jkeenan

On Fri, 25 May 2018 14​:24​:54 GMT, davem wrote​:

On Fri, May 25, 2018 at 06​:42​:28AM -0700, James E Keenan (via RT) wrote​:

I'll leave it up to the pumpking as to whether this should be a blocker
for 5.28.0. There are more important blockers, to be sure, but it would
also be unfortunate to do a release where we have so many build-time
warnings that we could have dealt with earlier in the cycle had we been
testing with this version of the C-compiler.

There are 69 warnings, all identical, and all resulting from the same two
macro definitions​: pWARN_ALL and pWARN_NONE.

The warnings are harmless, and we would almost certainly do more damage at
this late stage by trying to tweak the macros to silence the warnings.

Okay, I'll mark this to hold for 5.29.0. FWIW, using clang *5* on Linux generated 0 build-time warnings. (clang 6 not yet available in my Linux distro.)

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

From @jkeenan

On Fri, 25 May 2018 14​:41​:25 GMT, jkeenan wrote​:

On Fri, 25 May 2018 14​:24​:54 GMT, davem wrote​:

On Fri, May 25, 2018 at 06​:42​:28AM -0700, James E Keenan (via RT)
wrote​:

I'll leave it up to the pumpking as to whether this should be a
blocker
for 5.28.0. There are more important blockers, to be sure, but it
would
also be unfortunate to do a release where we have so many build-
time
warnings that we could have dealt with earlier in the cycle had we
been
testing with this version of the C-compiler.

There are 69 warnings, all identical, and all resulting from the same
two
macro definitions​: pWARN_ALL and pWARN_NONE.

The warnings are harmless, and we would almost certainly do more
damage at
this late stage by trying to tweak the macros to silence the
warnings.

Okay, I'll mark this to hold for 5.29.0. FWIW, using clang *5* on
Linux generated 0 build-time warnings. (clang 6 not yet available in
my Linux distro.)

Thank you very much.

FreeBSD-12.0-CURRENT (development) appears to use clang-6.0.0, so we can use the smoke-testing rig used in the following report as a gauge of (any) progress on this matter.

http​://perl5.test-smoke.org/report/66194

--
James E Keenan (jkeenan@​cpan.org)

From @jkeenan

On Fri, 25 May 2018 19​:44​:46 GMT, jkeenan wrote​:

On Fri, 25 May 2018 14​:41​:25 GMT, jkeenan wrote​:

On Fri, 25 May 2018 14​:24​:54 GMT, davem wrote​:

On Fri, May 25, 2018 at 06​:42​:28AM -0700, James E Keenan (via RT)
wrote​:

I'll leave it up to the pumpking as to whether this should be a
blocker
for 5.28.0. There are more important blockers, to be sure, but
it
would
also be unfortunate to do a release where we have so many build-
time
warnings that we could have dealt with earlier in the cycle had
we
been
testing with this version of the C-compiler.

There are 69 warnings, all identical, and all resulting from the
same
two
macro definitions​: pWARN_ALL and pWARN_NONE.

The warnings are harmless, and we would almost certainly do more
damage at
this late stage by trying to tweak the macros to silence the
warnings.

Okay, I'll mark this to hold for 5.29.0. FWIW, using clang *5* on
Linux generated 0 build-time warnings. (clang 6 not yet available in
my Linux distro.)

Thank you very much.

FreeBSD-12.0-CURRENT (development) appears to use clang-6.0.0, so we
can use the smoke-testing rig used in the following report as a gauge
of (any) progress on this matter.

http​://perl5.test-smoke.org/report/66194

I believe that revisions made in the following commit address the problem​:

#####
commit 8c165a3
Author​: Karl Williamson <khw@​cpan.org>
AuthorDate​: Sun Dec 30 23​:29​:07 2018
Commit​: Karl Williamson <khw@​cpan.org>
CommitDate​: Sat Jan 5 12​:42​:26 2019

regen/warnings.pl​: Fix undefined C behavior
 
This fixes compiler warnings "performing pointer arithmetic on a null pointer has undefined behavior"
 
There are several ways to fix this. This one was suggested by Tomasz Konojacki++. Instead of trying to point to address 1 and 2, two variables are created, and we point to them. const is cast away.
#####

See also p5p thread beginning here​:

https://www.nntp.perl.org/group/perl.perl5.porters/2018/12/msg253267.html

I'm waiting on some smoke tests before closing.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

From @jkeenan

On Sun, 06 Jan 2019 02​:48​:06 GMT, jkeenan wrote​:

On Fri, 25 May 2018 19​:44​:46 GMT, jkeenan wrote​:

On Fri, 25 May 2018 14​:41​:25 GMT, jkeenan wrote​:

On Fri, 25 May 2018 14​:24​:54 GMT, davem wrote​:

On Fri, May 25, 2018 at 06​:42​:28AM -0700, James E Keenan (via RT)
wrote​:

I'll leave it up to the pumpking as to whether this should be a
blocker
for 5.28.0. There are more important blockers, to be sure, but
it
would
also be unfortunate to do a release where we have so many
build-
time
warnings that we could have dealt with earlier in the cycle had
we
been
testing with this version of the C-compiler.

There are 69 warnings, all identical, and all resulting from the
same
two
macro definitions​: pWARN_ALL and pWARN_NONE.

The warnings are harmless, and we would almost certainly do more
damage at
this late stage by trying to tweak the macros to silence the
warnings.

Okay, I'll mark this to hold for 5.29.0. FWIW, using clang *5* on
Linux generated 0 build-time warnings. (clang 6 not yet available
in
my Linux distro.)

Thank you very much.

FreeBSD-12.0-CURRENT (development) appears to use clang-6.0.0, so we
can use the smoke-testing rig used in the following report as a gauge
of (any) progress on this matter.

http​://perl5.test-smoke.org/report/66194

I believe that revisions made in the following commit address the
problem​:

#####
commit 8c165a3
Author​: Karl Williamson <khw@​cpan.org>
AuthorDate​: Sun Dec 30 23​:29​:07 2018
Commit​: Karl Williamson <khw@​cpan.org>
CommitDate​: Sat Jan 5 12​:42​:26 2019

regen/warnings.pl​: Fix undefined C behavior

This fixes compiler warnings "performing pointer arithmetic on a null
pointer has undefined behavior"

There are several ways to fix this. This one was suggested by Tomasz
Konojacki++. Instead of trying to point to address 1 and 2, two
variables are created, and we point to them. const is cast away.
#####

See also p5p thread beginning here​:

https://www.nntp.perl.org/group/perl.perl5.porters/2018/12/msg253267.html

I'm waiting on some smoke tests before closing.

Thank you very much.

Smoke tests look good. Consider these two run on the same host​:

#####
http​://perl5.test-smoke.org/report/77561
branch​: blead
git describe​: v5.29.6-87-g1454e1e843
cjg-freebsd12​: Intel(R) Xeon(R) CPU L5640 @​ 2.27GHz (amd64/1 cpus)
  on freebsd 12.0-RELEASE
  using clang version 4.2.1 Compatible FreeBSD Clang 6.0.1 (tags/RELEASE_601/final 335540) (*1)
  smoketime 4 hours 15 minutes (average 31 minutes 58 seconds)

Summary​: PASS
94 -Wnull-pointer-arithmetic warnings
#####
http​://perl5.test-smoke.org/report/77668
branch​: blead
git describe​: v5.29.6-89-g8c165a32b7
cjg-freebsd12​: Intel(R) Xeon(R) CPU L5640 @​ 2.27GHz (amd64/1 cpus)
  on freebsd 12.0-RELEASE
  using clang version 4.2.1 Compatible FreeBSD Clang 6.0.1 (tags/RELEASE_601/final 335540) (*1)
  smoketime 4 hours 17 minutes (average 32 minutes 14 seconds)

Summary​: PASS
0 warnings
#####
--
James E Keenan (jkeenan@​cpan.org)

@jkeenan - Status changed from 'open' to 'pending release'

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

@khwilliamson - Status changed from 'pending release' to 'resolved'