From nixos@crystalgamma.de

This is a bug report for perl from nixos@​crystalgamma.de,
generated with the help of perlbug 1.40 running under perl 5.24.4.

As part of making the nixpkgs/NixOS bootstrapping process reproducible, I am
working on making the build of Perl deterministic.
The upstream distribution of Perl however writes certain details about the
configuration environment like the time or uname to files like config.h or
perlbug.

This makes bitwise reproducibility difficult.
It would be a useful for every distribution aiming for reproducibility if
$cf_time were to default to the timestamp stored in SOURCE_DATE_EPOCH instead
of the
current time (see https://reproducible-builds.org/specs/source-date-epoch/ and
https://reproducible-builds.org/docs/timestamps/ ).
Furthermore, it would be useful to have a more straightforward way to replace
(or omit) the other configuration environment values (e. g. $cf_by, $myuname).

Please find an example patch for the defaulting of $cf_time attached.

Flags​:
  category=install
  severity=low

Site configuration information for perl 5.24.4​:

Configured by user at Sat Apr 14 09​:35​:01 UTC 2018.

Summary of my perl5 (revision 5 version 24 subversion 4) configuration​:
 
  Platform​:
  osname=linux, osvers=4.16.13, archname=ppc64le-linux-thread-multi
  uname='linux talos-nixos 4.16.13 #1-nixos smp wed may 30 06​:17​:45 utc 2018
ppc64le gnulinux '
  config_args='-de -Dcc=cc -Uinstallusrbinperl -Dinstallstyle=lib/perl5 -
Duseshrplib -Dlocincpth=/no-such-path/include -Dloclibpth=/nix/store/
bqz3x6hpbfxm6hmnnrkaz4bax8rna14b-glibc-2.27/lib -Dusethreads -Dprefix=/nix/
store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4 -Dman1dir=/nix/store/
5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/share/man/man1 -Dman3dir=/nix/
store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/share/man/man3'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing
-pipe -fstack-protector-strong -I/no-such-path/include -D_LARGEFILE_SOURCE -
D_FILE_OFFSET_BITS=64',
  optimize='-O1',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -
fstack-protector-strong -I/no-such-path/include'
  ccversion='', gccversion='7.3.0', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678,
doublekind=3
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16,
longdblkind=7
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector-strong -L/nix/store/
bqz3x6hpbfxm6hmnnrkaz4bax8rna14b-glibc-2.27/lib'
  libpth=
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=, so=so, useshrplib=true, libperl=libperl.so
  gnulibc_version='2.27'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-
rpath,/nix/store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/lib/
perl5/5.24.4/ppc64le-linux-thread-multi/CORE'
  cccdlflags='-fPIC', lddlflags='-shared -O1 -L/nix/store/
bqz3x6hpbfxm6hmnnrkaz4bax8rna14b-glibc-2.27/lib -fstack-protector-strong'

@​INC for perl 5.24.4​:
  /nix/store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/lib/perl5/
site_perl/5.24.4/ppc64le-linux-thread-multi
  /nix/store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/lib/perl5/
site_perl/5.24.4
  /nix/store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/lib/perl5/5.24.4/
ppc64le-linux-thread-multi
  /nix/store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/lib/perl5/5.24.4

Environment for perl 5.24.4​:
  HOME=/home/user
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/home/user/bin​:/run/wrappers/bin​:/home/user/.nix-profile/bin​:/nix/
var/nix/profiles/default/bin​:/run/current-system/sw/bin​:/etc/profiles/per-
user/user/bin
  PERL_BADLANG (unset)
  SHELL=/run/current-system/sw/bin/fish

From nixos@crystalgamma.de

--- perl-5.24.3/Configure
+++ perl-5.24.3/Configure
@@ -3862,7 +3862,10 @@
 . ./posthint.sh
 
 : who configured the system
-cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE; $date 2>&1`
+case "$SOURCE_DATE_EPOCH" in
+       "") cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE; $date 2>&1`;;
+       *) cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE; date -d "@$SOURCE_DATE_EPOCH" 2>&1`
+esac
 case "$cf_by" in
 "")
        cf_by=`(logname) 2>/dev/null`

From @jkeenan

On Wed, 15 Aug 2018 15​:18​:02 GMT, nixos@​crystalgamma.de wrote​:

This is a bug report for perl from nixos@​crystalgamma.de,
generated with the help of perlbug 1.40 running under perl 5.24.4.

-----------------------------------------------------------------
As part of making the nixpkgs/NixOS bootstrapping process
reproducible, I am
working on making the build of Perl deterministic.
The upstream distribution of Perl however writes certain details about
the
configuration environment like the time or uname to files like
config.h or
perlbug.

This makes bitwise reproducibility difficult.
It would be a useful for every distribution aiming for
reproducibility if
$cf_time were to default to the timestamp stored in SOURCE_DATE_EPOCH
instead
of the
current time (see https://reproducible-builds.org/specs/source-date-
epoch/ and
https://reproducible-builds.org/docs/timestamps/ ).
Furthermore, it would be useful to have a more straightforward way to
replace
(or omit) the other configuration environment values (e. g. $cf_by,
$myuname).

Please find an example patch for the defaulting of $cf_time attached.

There was a little bit of discussion of the question of reproducible builds at last year's Perl 5 Core Hackathon​:

https://github.com/p5h/2017/wiki/Proposed-Discussions#reproducible-builds

---
Flags​:
category=install
severity=low
---
Site configuration information for perl 5.24.4​:

Configured by user at Sat Apr 14 09​:35​:01 UTC 2018.

Summary of my perl5 (revision 5 version 24 subversion 4)
configuration​:

Platform​:
osname=linux, osvers=4.16.13, archname=ppc64le-linux-thread-multi
uname='linux talos-nixos 4.16.13 #1-nixos smp wed may 30 06​:17​:45
utc 2018
ppc64le gnulinux '
config_args='-de -Dcc=cc -Uinstallusrbinperl
-Dinstallstyle=lib/perl5 -
Duseshrplib -Dlocincpth=/no-such-path/include -Dloclibpth=/nix/store/
bqz3x6hpbfxm6hmnnrkaz4bax8rna14b-glibc-2.27/lib -Dusethreads
-Dprefix=/nix/
store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4
-Dman1dir=/nix/store/
5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/share/man/man1
-Dman3dir=/nix/
store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/share/man/man3'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-
aliasing
-pipe -fstack-protector-strong -I/no-such-path/include
-D_LARGEFILE_SOURCE -
D_FILE_OFFSET_BITS=64',
optimize='-O1',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing
-pipe -
fstack-protector-strong -I/no-such-path/include'
ccversion='', gccversion='7.3.0', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678,
doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define,
longdblsize=16,
longdblkind=7
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='cc', ldflags =' -fstack-protector-strong -L/nix/store/
bqz3x6hpbfxm6hmnnrkaz4bax8rna14b-glibc-2.27/lib'
libpth=
libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.27'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E
-Wl,-
rpath,/nix/store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/lib/
perl5/5.24.4/ppc64le-linux-thread-multi/CORE'
cccdlflags='-fPIC', lddlflags='-shared -O1 -L/nix/store/
bqz3x6hpbfxm6hmnnrkaz4bax8rna14b-glibc-2.27/lib -fstack-protector-
strong'

---
@​INC for perl 5.24.4​:
/nix/store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/lib/perl5/
site_perl/5.24.4/ppc64le-linux-thread-multi
/nix/store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-5.24.4/lib/perl5/
site_perl/5.24.4
/nix/store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-
5.24.4/lib/perl5/5.24.4/
ppc64le-linux-thread-multi
/nix/store/5vdbx07jr4v1dwdanwznnc37jcidmnjf-perl-
5.24.4/lib/perl5/5.24.4

---
Environment for perl 5.24.4​:
HOME=/home/user
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/user/bin​:/run/wrappers/bin​:/home/user/.nix-
profile/bin​:/nix/
var/nix/profiles/default/bin​:/run/current-
system/sw/bin​:/etc/profiles/per-
user/user/bin
PERL_BADLANG (unset)
SHELL=/run/current-system/sw/bin/fish

--
James E Keenan (jkeenan@​cpan.org)

The RT System itself - Status changed from 'new' to 'open'

From @jkeenan

On Wed, 15 Aug 2018 15​:18​:02 GMT, nixos@​crystalgamma.de wrote​:

This is a bug report for perl from nixos@​crystalgamma.de,
generated with the help of perlbug 1.40 running under perl 5.24.4.

-----------------------------------------------------------------
As part of making the nixpkgs/NixOS bootstrapping process
reproducible, I am
working on making the build of Perl deterministic.
The upstream distribution of Perl however writes certain details about
the
configuration environment like the time or uname to files like
config.h or
perlbug.

This makes bitwise reproducibility difficult.
It would be a useful for every distribution aiming for
reproducibility if
$cf_time were to default to the timestamp stored in SOURCE_DATE_EPOCH
instead
of the
current time (see https://reproducible-builds.org/specs/source-date-
epoch/ and
https://reproducible-builds.org/docs/timestamps/ ).
Furthermore, it would be useful to have a more straightforward way to
replace
(or omit) the other configuration environment values (e. g. $cf_by,
$myuname).

Please find an example patch for the defaulting of $cf_time attached.

The patch was drawn against perl-5.24.3, which is no longer supported. I did my best to apply it to perl 5 blead and created this branch for smoke-testing and further evaluation​:

smoke-me/jkeenan/133452/cf_time-reproducible

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

From @doughera88

On Wed, Aug 15, 2018 at 08​:18​:02AM -0700, (via RT) wrote​:

# New Ticket Created by
# Please include the string​: [perl #133452]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133452 >

-----------------------------------------------------------------
As part of making the nixpkgs/NixOS bootstrapping process reproducible, I am
working on making the build of Perl deterministic.
The upstream distribution of Perl however writes certain details about the
configuration environment like the time or uname to files like config.h or
perlbug.

This makes bitwise reproducibility difficult.

I agree that wanting to make it easy to make reproducible builds is a reasonable
goal.

It would be a useful for every distribution aiming for reproducibility if
$cf_time were to default to the timestamp stored in SOURCE_DATE_EPOCH instead
of the
current time (see https://reproducible-builds.org/specs/source-date-epoch/ and
https://reproducible-builds.org/docs/timestamps/ ).
Furthermore, it would be useful to have a more straightforward way to replace
(or omit) the other configuration environment values (e. g. $cf_by, $myuname).

cf_by can already be overridden on the Configure command line by specifying
the -Dcf_by. I'm unclear what you mean by "a more straightforward way".

Please find an example patch for the defaulting of $cf_time attached.

+case "$SOURCE_DATE_EPOCH" in
+ "") cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE; $date 2>&1`;;
+ *) cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE; date -d "@​$SOURCE_DATE_EPOCH" 2>&1`
+esac

Unfortunately, the use of date -d to print out an arbitrary date is not portable.
In general, I don't think you can count on the date command being able to output
arbitrary dates. A simpler first step, though, would be to allow a Configure
command-line override -Dcf_time. A trivial patch to do that is here​:

diff --git a/Configure b/Configure
index 0194d7f434..b93250f023 100755
--- a/Configure
+++ b/Configure
@@ -3864,8 +3864,13 @@ esac
 
 . ./posthint.sh
 
+# Allow overrides for reproducible builds
+case "$cf_time" in
+"")
+	cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE; $date 2>&1`
+	;;
+esac
 : who configured the system
-cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE; $date 2>&1`
 case "$cf_by" in
 "")
 	cf_by=`(logname) 2>/dev/null`

-- 

  Andy Dougherty doughera@​lafayette.edu

From @jkeenan

On Fri, 17 Aug 2018 20​:49​:48 GMT, doughera wrote​:

On Wed, Aug 15, 2018 at 08​:18​:02AM -0700, (via RT) wrote​:

# New Ticket Created by
# Please include the string​: [perl #133452]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133452 >

-----------------------------------------------------------------
As part of making the nixpkgs/NixOS bootstrapping process
reproducible, I am
working on making the build of Perl deterministic.
The upstream distribution of Perl however writes certain details
about the
configuration environment like the time or uname to files like
config.h or
perlbug.

This makes bitwise reproducibility difficult.

I agree that wanting to make it easy to make reproducible builds is a
reasonable
goal.

It would be a useful for every distribution aiming for
reproducibility if
$cf_time were to default to the timestamp stored in SOURCE_DATE_EPOCH
instead
of the
current time (see https://reproducible-builds.org/specs/source-date-
epoch/ and
https://reproducible-builds.org/docs/timestamps/ ).
Furthermore, it would be useful to have a more straightforward way to
replace
(or omit) the other configuration environment values (e. g. $cf_by,
$myuname).

cf_by can already be overridden on the Configure command line by
specifying
the -Dcf_by. I'm unclear what you mean by "a more straightforward
way".

Please find an example patch for the defaulting of $cf_time attached.

+case "$SOURCE_DATE_EPOCH" in
+ "") cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export
LANGUAGE; $date 2>&1`;;
+ *) cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export
LANGUAGE; date -d "@​$SOURCE_DATE_EPOCH" 2>&1`
+esac

Unfortunately, the use of date -d to print out an arbitrary date is
not portable.
In general, I don't think you can count on the date command being able
to output
arbitrary dates. A simpler first step, though, would be to allow a
Configure
command-line override -Dcf_time. A trivial patch to do that is here​:

diff --git a/Configure b/Configure
index 0194d7f434..b93250f023 100755
--- a/Configure
+++ b/Configure
@​@​ -3864,8 +3864,13 @​@​ esac

. ./posthint.sh

+# Allow overrides for reproducible builds
+case "$cf_time" in
+"")
+ cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE;
$date 2>&1`
+ ;;
+esac
: who configured the system
-cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE; $date
2>&1`
case "$cf_by" in
"")
cf_by=`(logname) 2>/dev/null`

Branch for smoke-testing​:

smoke-me/jkeenan/adougherty/133452/cf_time-reproducible

--
James E Keenan (jkeenan@​cpan.org)

From nixos@crystalgamma.de

On Friday, August 17, 2018 10​:49​:48 PM CEST you wrote​:

On Wed, Aug 15, 2018 at 08​:18​:02AM -0700, (via RT) wrote​:

# New Ticket Created by
# Please include the string​: [perl #133452]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133452 >

-----------------------------------------------------------------
As part of making the nixpkgs/NixOS bootstrapping process reproducible, I
am working on making the build of Perl deterministic.
The upstream distribution of Perl however writes certain details about the
configuration environment like the time or uname to files like config.h or
perlbug.

This makes bitwise reproducibility difficult.

I agree that wanting to make it easy to make reproducible builds is a
reasonable goal.

It would be a useful for every distribution aiming for reproducibility if
$cf_time were to default to the timestamp stored in SOURCE_DATE_EPOCH
instead of the
current time (see https://reproducible-builds.org/specs/source-date-epoch/
and https://reproducible-builds.org/docs/timestamps/ ).
Furthermore, it would be useful to have a more straightforward way to
replace (or omit) the other configuration environment values (e. g.
$cf_by, $myuname).
cf_by can already be overridden on the Configure command line by specifying
the -Dcf_by. I'm unclear what you mean by "a more straightforward way".

OK, that is definitely useful. I don't know what I was thinking when I was
asking for "a more straightforward way" … I'm kind of embarrased :(

Please find an example patch for the defaulting of $cf_time attached.

+case "$SOURCE_DATE_EPOCH" in
+ "") cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE;
$date 2>&1`;; + *) cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL;
export LANGUAGE; date -d "@​$SOURCE_DATE_EPOCH" 2>&1` +esac

Unfortunately, the use of date -d to print out an arbitrary date is not
portable. In general, I don't think you can count on the date command being
able to output arbitrary dates. A simpler first step, though, would be to
allow a Configure command-line override -Dcf_time. A trivial patch to do
that is here​:

diff --git a/Configure b/Configure
index 0194d7f434..b93250f023 100755
--- a/Configure
+++ b/Configure
@​@​ -3864,8 +3864,13 @​@​ esac

. ./posthint.sh

+# Allow overrides for reproducible builds
+case "$cf_time" in
+"")
+ cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE; $date 2>&1`
+ ;;
+esac

: who configured the system

-cf_time=`LC_ALL=C; LANGUAGE=C; export LC_ALL; export LANGUAGE; $date 2>&1`
case "$cf_by" in
"")
cf_by=`(logname) 2>/dev/null`

I feared as much. Being able to override cf_time sounds good enough though.