Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make test fails on OS X El Capitan #15057

Closed
p5pRT opened this issue Nov 21, 2015 · 15 comments
Closed

make test fails on OS X El Capitan #15057

p5pRT opened this issue Nov 21, 2015 · 15 comments

Comments

@p5pRT
Copy link

p5pRT commented Nov 21, 2015

Migrated from rt.perl.org#126706 (status was 'resolved')

Searchable as RT126706$

@p5pRT
Copy link
Author

p5pRT commented Nov 21, 2015

From dominyktiller@gmail.com

As per the subject line​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0 ./runtests choose
dyld​: Library not loaded​: /usr/local/Cellar/perl/5.22.0/lib/5.22.0/darwin-thread-multi-2level/CORE/libperl.dylib
  Referenced from​: /private/tmp/perl20151121-6532-elg51d/perl-5.22.0/t/./perl
  Reason​: image not found
./runtests​: line 70​: 26701 Trace/BPT trap​: 5 ./perl $TESTFILE $TEST_ARGS $TEST_FILES < /dev/tty
make​: *** [test] Error 133
```

OS X El Capitan has a new security feature where the DYLD_LIBRARY_PATH isn't passed down to a child process, which is probably the root cause of the failure here.

If you run​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0 env | grep DYLD
```

As a test for example, on OS X Mavericks you see​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121-6532-elg51d/perl-5.22.0
```

On OS X El Capitan you get nothing.

This has been reported in various other projects as well​:
* http​://postgresql.nabble.com/OS-X-El-Capitan-and-DYLD-LIBRARY-PATH-td5869963.html
* https://www.mail-archive.com/pgsql-hackers@postgresql.org/msg272269.html
* https://forum.qt.io/topic/59439/dyld-library-not-loaded-when-debugging-but-ok-when-running-on-os-x-el-capitan-solved/2

@p5pRT
Copy link
Author

p5pRT commented Nov 23, 2015

From @craigberry

On Sat, Nov 21, 2015 at 3​:31 PM, Dominyk Tiller
<perlbug-followup@​perl.org> wrote​:

# New Ticket Created by Dominyk Tiller
# Please include the string​: [perl #126706]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=126706 >

As per the subject line​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0 ./runtests choose
dyld​: Library not loaded​: /usr/local/Cellar/perl/5.22.0/lib/5.22.0/darwin-thread-multi-2level/CORE/libperl.dylib
Referenced from​: /private/tmp/perl20151121-6532-elg51d/perl-5.22.0/t/./perl
Reason​: image not found
./runtests​: line 70​: 26701 Trace/BPT trap​: 5 ./perl $TESTFILE $TEST_ARGS $TEST_FILES < /dev/tty
make​: *** [test] Error 133
```

OS X El Capitan has a new security feature where the DYLD_LIBRARY_PATH isn't passed down to a child process, which is probably the root cause of the failure here.

I've done numerous builds of blead and maint-5.22 on El Capitan
without any problems, but then I've never had DYLD_LIBRARY_PATH in my
environment. Can you explain why you need it in yours?

@p5pRT
Copy link
Author

p5pRT commented Nov 23, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 16, 2015

From @tonycoz

On Mon Nov 23 10​:18​:47 2015, craig.a.berry@​gmail.com wrote​:

I've done numerous builds of blead and maint-5.22 on El Capitan
without any problems,

I see the error with a -Duseshrplib build​:

TESTFILE=harness DYLD_LIBRARY_PATH=/Users/tony/dev/perl/git/perl ./runtests choose
dyld​: Library not loaded​: /usr/local/lib/perl5/5.23.6/darwin-thread-multi-2level/CORE/libperl.dylib
  Referenced from​: /Users/tony/dev/perl/git/perl/t/./perl
  Reason​: image not found
./runtests​: line 70​: 34725 Trace/BPT trap​: 5 ./perl $TESTFILE $TEST_ARGS $TEST_FILES < /dev/tty
make​: *** [test_harness] Error 133

but then I've never had DYLD_LIBRARY_PATH in my
environment. Can you explain why you need it in yours?

Configure, hints/darwin.sh and Makefile.SH conspire together to set it on darwin.

It's needed for -Duseshrplib builds, which isn't the default on Darwin, but it useful for mod_perl and other embedded perls.

The attached fixes the most immediate errors, but many more tests continue to fail due to this and they'll need to be tracked down individually if we want to support this.

Tony

@p5pRT
Copy link
Author

p5pRT commented Dec 16, 2015

From @tonycoz

0001-perl-126706-make-DYLD_LIBRARY_PATH-more-visible-on-D.patch
From 3964d88ca2b4c06f3034add5c21a7e7b53c8ea38 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Wed, 16 Dec 2015 16:28:12 +1100
Subject: [perl #126706] make DYLD_LIBRARY_PATH more visible on Darwin

On OS X 10.10, by default, system binaries, including the shell, do
not inherit DYLD_* enviroment variables.

The shell does however pass that variable onto its children, so any
place we need that visible during testing will need to set it
explicitly.
---
 runtests.SH | 8 ++++++++
 t/test.pl   | 8 ++++++++
 2 files changed, 16 insertions(+)

diff --git a/runtests.SH b/runtests.SH
index 71f41d1..feee19c 100755
--- a/runtests.SH
+++ b/runtests.SH
@@ -87,6 +87,14 @@ esac
 
 !NO!SUBS!
 
+case "$ldlibpthname" in
+    '') ;;
+    *) $spitshell >>runtests <<!GROK!THIS ;;
+$ldlibpthname=$PWD
+export $ldlibpthname
+!GROK!THIS
+esac
+
 ## In the following, dollars and backticks do need the extra backslash.
 $spitshell >>runtests <<!GROK!THIS!
 # The second branch is for testing without a tty or controlling terminal,
diff --git a/t/test.pl b/t/test.pl
index cda3840..42d96ea 100644
--- a/t/test.pl
+++ b/t/test.pl
@@ -640,6 +640,14 @@ sub _create_runperl { # Create the string to qx in runperl().
     if ($ENV{PERL_RUNPERL_DEBUG}) {
 	$runperl = "$ENV{PERL_RUNPERL_DEBUG} $runperl";
     }
+    if ($^O eq 'darwin') {
+	# OS X 10.10 drops DYLD_LIBRARY_PATH when inherited by system
+	# processes, including the shell, make sure the child perl
+	# still sees it
+	if (eval { require Config; 1 } && (my $ldlibpthname = $Config::Config{ldlibpthname})) {
+	    $runperl="$ldlibpthname=$ENV{PWD}/.. $runperl";
+	}
+    }
     unless ($args{nolib}) {
 	$runperl = $runperl . ' "-I../lib"'; # doublequotes because of VMS
     }
-- 
2.5.4 (Apple Git-61)

@p5pRT
Copy link
Author

p5pRT commented Feb 22, 2017

From @tonycoz

On Tue, 15 Dec 2015 21​:37​:28 -0800, tonyc wrote​:

The attached fixes the most immediate errors, but many more tests
continue to fail due to this and they'll need to be tracked down
individually if we want to support this.

The attached fixes tests in t/

For cpan/ and dist/ (and probably ext/) it might be useful to make something like which_perl() and which_perl_shell() visible.

Tony

@p5pRT
Copy link
Author

p5pRT commented Feb 22, 2017

From @tonycoz

0001-perl-126706-ensure-DYLD_LIBRARY_PATH-is-set-for-t-te.patch
From 379dbef097618c73e0c1e8528cb8623c9b7d8373 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Wed, 22 Feb 2017 16:03:45 +1100
Subject: (perl #126706) ensure DYLD_LIBRARY_PATH is set for t/ tests on OS X

On OS X 10.10, by default, system binaries, including the shell, do
not inherit DYLD_* enviroment variables.

So when starting a new child perl via the shell on darwin, ensure it's
set.

This fixes only core tests in t/
---
 t/base/term.t       | 12 +++++++++---
 t/io/dup.t          |  3 ++-
 t/io/fflush.t       |  2 +-
 t/io/open.t         |  2 +-
 t/io/openpid.t      |  2 +-
 t/io/through.t      |  2 +-
 t/op/alarm.t        |  2 +-
 t/op/die_exit.t     |  6 ++++--
 t/op/exec.t         |  7 ++++---
 t/op/fork.t         | 15 ++++++++++-----
 t/op/magic.t        |  7 +------
 t/op/srand.t        |  5 +++--
 t/op/taint.t        |  7 +++++++
 t/porting/authors.t |  4 +++-
 t/run/cloexec.t     |  2 +-
 t/run/fresh_perl.t  |  6 +++++-
 t/run/script.t      |  2 +-
 t/run/switcht.t     |  2 +-
 t/test.pl           | 28 ++++++++++++++++++++++++++--
 19 files changed, 82 insertions(+), 34 deletions(-)

diff --git a/t/base/term.t b/t/base/term.t
index 7a16ffd..af35b8f 100644
--- a/t/base/term.t
+++ b/t/base/term.t
@@ -16,9 +16,15 @@ elsif ($x eq chr(21)) { print "ok 1 # EBCDIC\n"; }
 else {print "not ok 1\n";}
 
 # check `` processing
-
-$x = `$^X -le "print 'hi there'"`;
-if ($x eq "hi there\n") {print "ok 2\n";} else {print "not ok 2\n";}
+if ($^O ne 'darwin') {
+    $x = `$^X -le "print 'hi there'"`;
+    if ($x eq "hi there\n") {print "ok 2\n";} else {print "not ok 2\n";}
+}
+else {
+    # the shell doesn't inherit DLYD_LIBRARY_PATH on modern Darwin
+    # and this is too early to mess with %ENV
+    print "ok 2 # SKIP need Config to reliably start a child perl here\n";
+}
 
 # check $#array
 
diff --git a/t/io/dup.t b/t/io/dup.t
index 8a8b27e..c7631d6 100644
--- a/t/io/dup.t
+++ b/t/io/dup.t
@@ -29,7 +29,8 @@ print STDOUT "ok 2\n";
 print STDERR "ok 3\n";
 
 # Since some systems don't have echo, we use Perl.
-$echo = qq{$^X -le "print q(ok %d)"};
+my $Perl = which_perl_shell();
+$echo = qq{$Perl -le "print q(ok %d)"};
 
 $cmd = sprintf $echo, 4;
 print `$cmd`;
diff --git a/t/io/fflush.t b/t/io/fflush.t
index 8e89ebb..053b743 100644
--- a/t/io/fflush.t
+++ b/t/io/fflush.t
@@ -27,7 +27,7 @@ skip_all('fflush(NULL) or equivalent not available')
 
 plan(tests => 7);
 
-my $runperl = $^X =~ m/\s/ ? qq{"$^X"} : $^X;
+my $runperl = which_perl_shell();
 $runperl .= qq{ "-I../lib"};
 
 sub file_eq {
diff --git a/t/io/open.t b/t/io/open.t
index 6be9f0e..59c4556 100644
--- a/t/io/open.t
+++ b/t/io/open.t
@@ -12,7 +12,7 @@ use Config;
 
 plan tests => 156;
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 my $afile = tempfile();
 {
diff --git a/t/io/openpid.t b/t/io/openpid.t
index d3fcf78..ccc9860 100644
--- a/t/io/openpid.t
+++ b/t/io/openpid.t
@@ -25,7 +25,7 @@ $| = 1;
 $SIG{PIPE} = 'IGNORE';
 $SIG{HUP} = 'IGNORE' if $^O eq 'interix';
 
-my $perl = which_perl();
+my $perl = which_perl_shell();
 $perl .= qq[ "-I../lib"];
 
 #
diff --git a/t/io/through.t b/t/io/through.t
index 65a64bb..6f24b01 100644
--- a/t/io/through.t
+++ b/t/io/through.t
@@ -10,7 +10,7 @@ BEGIN {
 
 use strict;
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 my $data = <<'EOD';
 x
diff --git a/t/op/alarm.t b/t/op/alarm.t
index 749482c..c2d0594 100644
--- a/t/op/alarm.t
+++ b/t/op/alarm.t
@@ -13,7 +13,7 @@ if ( !$Config{d_alarm} ) {
 }
 
 plan tests => 5;
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 my ($start_time, $end_time);
 
diff --git a/t/op/die_exit.t b/t/op/die_exit.t
index e074913..d9c76ad 100644
--- a/t/op/die_exit.t
+++ b/t/op/die_exit.t
@@ -60,14 +60,16 @@ if ($^O eq 'VMS') {
 my $tempfile = tempfile();
 open STDERR, '>', $tempfile or die "Can't open temp error file $tempfile:  $!";
 
+my $ShPerl = which_perl_shell();
+
 foreach my $test (@tests) {
     my($bang, $query, $code) = @$test;
     $code ||= 'die;';
     if ($^O eq 'MSWin32' || $^O eq 'NetWare' || $^O eq 'VMS') {
-        system(qq{$^X -e "\$! = $bang; \$? = $query; $code"});
+        system(qq{$ShPerl -e "\$! = $bang; \$? = $query; $code"});
     }
     else {
-        system(qq{$^X -e '\$! = $bang; \$? = $query; $code'});
+        system(qq{$ShPerl -e '\$! = $bang; \$? = $query; $code'});
     }
     my $exit = $?;
 
diff --git a/t/op/exec.t b/t/op/exec.t
index 1155439..5e3b959 100644
--- a/t/op/exec.t
+++ b/t/op/exec.t
@@ -38,7 +38,8 @@ my $Is_Win32 = $^O eq 'MSWin32';
 
 plan(tests => 25);
 
-my $Perl = which_perl();
+my $ListPerl = which_perl();
+my $Perl = which_perl_shell();
 
 my $exit;
 SKIP: {
@@ -59,7 +60,7 @@ is( $exit, 0, '  exited 0' );
 # On Unix its the opposite.
 my $quote = $Is_VMS || $Is_Win32 ? '"' : '';
 $tnum = curr_test();
-$exit = system $Perl, '-le', 
+$exit = system $ListPerl, '-le', 
                "${quote}print q{ok $tnum - system(PROG, LIST)}${quote}";
 next_test();
 is( $exit, 0, '  exited 0' );
@@ -157,5 +158,5 @@ TODO: {
 }
 
 my $test = curr_test();
-exec $Perl, '-le', qq{${quote}print 'ok $test - exec PROG, LIST'${quote}};
+exec $ListPerl, '-le', qq{${quote}print 'ok $test - exec PROG, LIST'${quote}};
 fail("This should never be reached if the exec() worked");
diff --git a/t/op/fork.t b/t/op/fork.t
index b69a929..444392b 100644
--- a/t/op/fork.t
+++ b/t/op/fork.t
@@ -28,8 +28,13 @@ SKIP: {
     skip "Can't set ulimit -u on this system: $probe"
 	unless $probe eq 'good';
 
+    my $perl = which_perl();
+    my $env = "";
+    if ($^O eq "darwin") {
+        $env = "DYLD_LIBRARY_PATH=$ENV{DYLD_LIBRARY_PATH} ";
+    }
     my $out = qx{
-        $shell -c 'ulimit -u 1; exec $^X -e "
+        $shell -c 'ulimit -u 1; ${env}exec $perl -e "
             print((() = fork) == 1 ? q[ok] : q[not ok])
         "'
     };
@@ -253,10 +258,10 @@ $| = 1;
 $\ = "\n";
 my $getenv;
 if ($^O eq 'MSWin32' || $^O eq 'NetWare') {
-    $getenv = qq[$^X -e "print \$ENV{TST}"];
+    $getenv = qq[$ShPerl -e "print \$ENV{TST}"];
 }
 else {
-    $getenv = qq[$^X -e 'print \$ENV{TST}'];
+    $getenv = qq[$ShPerl -e 'print \$ENV{TST}'];
 }
 $ENV{TST} = 'foo';
 if (fork) {
@@ -477,13 +482,13 @@ child: called as [main::f(foo,bar)]
 waitpid() returned ok
 ########
 # Windows 2000: https://rt.cpan.org/Ticket/Display.html?id=66016#txn-908976
-system $^X,  "-e", "if (\$pid=fork){sleep 1;kill(9, \$pid)} else {sleep 5}";
+system $Perl,  "-e", "if (\$pid=fork){sleep 1;kill(9, \$pid)} else {sleep 5}";
 print $?>>8, "\n";
 EXPECT
 0
 ########
 # Windows 7: https://rt.cpan.org/Ticket/Display.html?id=66016#txn-908976
-system $^X,  "-e", "if (\$pid=fork){kill(9, \$pid)} else {sleep 5}";
+system $Perl,  "-e", "if (\$pid=fork){kill(9, \$pid)} else {sleep 5}";
 print $?>>8, "\n";
 EXPECT
 0
diff --git a/t/op/magic.t b/t/op/magic.t
index 3f71f8e..60bb842 100644
--- a/t/op/magic.t
+++ b/t/op/magic.t
@@ -56,12 +56,7 @@ $Is_Dos      = $^O eq 'dos';
 $Is_os2      = $^O eq 'os2';
 $Is_Cygwin   = $^O eq 'cygwin';
 
-$PERL =
-   ($Is_NetWare ? 'perl'   :
-    $Is_VMS     ? $^X      :
-    $Is_MSWin32 ? '.\perl' :
-                  './perl');
-
+$PERL = which_perl_shell();
 
 sub env_is {
     my ($key, $val, $desc) = @_;
diff --git a/t/op/srand.t b/t/op/srand.t
index 09de60a..cfac4d5 100644
--- a/t/op/srand.t
+++ b/t/op/srand.t
@@ -52,9 +52,10 @@ ok( !eq_array(\@first_run, \@second_run),
 }
 
 # This test checks whether Perl called srand for you.
-@first_run  = `$^X -le "print int rand 100 for 1..100"`;
+my $perl = which_perl_shell();
+@first_run  = `$perl -le "print int rand 100 for 1..100"`;
 sleep(1); # in case our srand() is too time-dependent
-@second_run = `$^X -le "print int rand 100 for 1..100"`;
+@second_run = `$perl -le "print int rand 100 for 1..100"`;
 
 ok( !eq_array(\@first_run, \@second_run), 'srand() called automatically');
 
diff --git a/t/op/taint.t b/t/op/taint.t
index c13eaf6..08cd5af 100644
--- a/t/op/taint.t
+++ b/t/op/taint.t
@@ -55,6 +55,13 @@ my $Invoke_Perl = $Is_VMS      ? 'MCR Sys$Disk:[]Perl.exe' :
                                  './perl'               ;
 my @MoreEnv = qw/IFS CDPATH ENV BASH_ENV/;
 
+if ($^O eq "darwin") {
+    # all the tests here invoke perl through a shell, and the system shell
+    # on darwin doesn't inherit DYLD_LIBRARY_PATH...
+    my ($lib_path) = ($ENV{DYLD_LIBRARY_PATH} =~ /(.*)/);
+    $Invoke_Perl = "DYLD_LIBRARY_PATH=$lib_path $Invoke_Perl";
+}
+
 if ($Is_VMS) {
     my (%old, $x);
     for $x ('DCL$PATH', @MoreEnv) {
diff --git a/t/porting/authors.t b/t/porting/authors.t
index 563b92a..45c4e72 100644
--- a/t/porting/authors.t
+++ b/t/porting/authors.t
@@ -10,8 +10,10 @@ use strict;
 require './t/test.pl';
 find_git_or_skip('all');
 
+my $ShPerl = which_perl_shell();
+
 # This is the subset of "pretty=fuller" that checkAUTHORS.pl actually needs:
 my $quote = $^O =~ /^mswin/i ? q(") : q(');
-system("git log --pretty=format:${quote}Author: %an <%ae>%n${quote} | $^X Porting/checkAUTHORS.pl --tap -");
+system("git log --pretty=format:${quote}Author: %an <%ae>%n${quote} | $ShPerl Porting/checkAUTHORS.pl --tap -");
 
 # EOF
diff --git a/t/run/cloexec.t b/t/run/cloexec.t
index f767267..9d27ed9 100644
--- a/t/run/cloexec.t
+++ b/t/run/cloexec.t
@@ -55,7 +55,7 @@ sub make_tmp_file {
     close  FHTMP             or die "close '$fname': $!";
 }
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 my $quote = "'";
 
 my $tmperr             = tempfile();
diff --git a/t/run/fresh_perl.t b/t/run/fresh_perl.t
index 411ff04..ef05391 100644
--- a/t/run/fresh_perl.t
+++ b/t/run/fresh_perl.t
@@ -16,7 +16,7 @@ BEGIN {
 
 use strict;
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 $|=1;
 
@@ -41,6 +41,10 @@ foreach my $prog (@prgs) {
 
     my($prog,$expected) = split(/\nEXPECT\n/, $raw_prog);
     $prog .= "\n";
+
+    # this might break new tests, but there shouldn't be any new tests...
+    $prog =~ s(\./perl\b)($Perl)g;
+
     $expected = '' unless defined $expected;
 
     if ($prog =~ /^\# SKIP: (.+)/m) {
diff --git a/t/run/script.t b/t/run/script.t
index fa61a2c..066deb0 100644
--- a/t/run/script.t
+++ b/t/run/script.t
@@ -7,7 +7,7 @@ BEGIN {
     plan(3);
 }
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 my $filename = tempfile();
 
diff --git a/t/run/switcht.t b/t/run/switcht.t
index 01b9f2f..53e8591 100644
--- a/t/run/switcht.t
+++ b/t/run/switcht.t
@@ -8,7 +8,7 @@ BEGIN {
 
 plan tests => 13;
 
-my $Perl = which_perl();
+my $Perl = which_perl_shell();
 
 my $warning;
 local $SIG{__WARN__} = sub { $warning = join "\n", @_; };
diff --git a/t/test.pl b/t/test.pl
index b236103..a4b35fc 100644
--- a/t/test.pl
+++ b/t/test.pl
@@ -710,12 +710,13 @@ sub _create_runperl { # Create the string to qx in runperl().
 	$args{stdin} =~ s/\n/\\n/g;
 	$args{stdin} =~ s/\r/\\r/g;
 
+	my $shell_perl = which_perl_shell();
 	if ($is_mswin || $is_netware || $is_vms) {
-	    $runperl = qq{$Perl -e "print qq(} .
+	    $runperl = qq{$shell_perl -e "print qq(} .
 		$args{stdin} . q{)" | } . $runperl;
 	}
 	else {
-	    $runperl = qq{$Perl -e 'print qq(} .
+	    $runperl = qq{$shell_perl -e 'print qq(} .
 		$args{stdin} . q{)' | } . $runperl;
 	}
     } elsif (exists $args{stdin}) {
@@ -861,6 +862,25 @@ sub which_perl {
     return $Perl;
 }
 
+# A perl executable name suitable for use when run through the shell.
+# Needed because modern darwin system shells drop DYLD_LIBRARY_PATH
+sub which_perl_shell {
+    my $perl = which_perl();
+    if ($perl =~ m/\s/) {
+        $perl = qq{"$perl"};
+    }
+    if ($^O eq 'darwin') {
+	# OS X 10.10 drops DYLD_LIBRARY_PATH when inherited by system
+	# processes, including the shell, make sure the child perl
+	# still sees it
+	if (eval { require Config; 1 } && (my $ldlibpthname = $Config::Config{ldlibpthname})) {
+	    $perl="$ldlibpthname=$ENV{PWD}/.. $perl";
+	}
+    }
+
+    return $perl;
+}
+
 sub unlink_all {
     my $count = 0;
     foreach my $file (@_) {
@@ -1276,6 +1296,10 @@ sub run_multiple_progs {
               or die "Can't dup STDOUT->STDERR: $!;";
         }
 	};
+	if ($prog =~ m/\$(?:Sh)?Perl\b/) {
+	    print $fh "my \$Perl = q\0", which_perl(), "\0;\n";
+	    print $fh "my \$ShPerl = q\0", which_perl_shell(), "\0;\n";
+	}
 	print $fh "\n#line 1\n";  # So the line numbers don't get messed up.
 	print $fh $prog,"\n";
 	close $fh or die "Cannot close $tmpfile: $!";
-- 
2.1.4

@p5pRT
Copy link
Author

p5pRT commented Oct 4, 2018

From @tonycoz

On Sat, 21 Nov 2015 13​:31​:31 -0800, dominyktiller@​gmail.com wrote​:

As per the subject line​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0
./runtests choose
dyld​: Library not loaded​:
/usr/local/Cellar/perl/5.22.0/lib/5.22.0/darwin-thread-multi-
2level/CORE/libperl.dylib
Referenced from​: /private/tmp/perl20151121-6532-elg51d/perl-
5.22.0/t/./perl
Reason​: image not found
./runtests​: line 70​: 26701 Trace/BPT trap​: 5 ./perl $TESTFILE
$TEST_ARGS $TEST_FILES < /dev/tty
make​: *** [test] Error 133
```

OS X El Capitan has a new security feature where the DYLD_LIBRARY_PATH
isn't passed down to a child process, which is probably the root cause
of the failure here.

If you run​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0
env | grep DYLD
```

As a test for example, on OS X Mavericks you see​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121-6532-elg51d/perl-5.22.0
```

On OS X El Capitan you get nothing.

This has been reported in various other projects as well​:
* http​://postgresql.nabble.com/OS-X-El-Capitan-and-DYLD-LIBRARY-PATH-
td5869963.html
* https://www.mail-archive.com/pgsql-
hackers@​postgresql.org/msg272269.html
* https://forum.qt.io/topic/59439/dyld-library-not-loaded-when-
debugging-but-ok-when-running-on-os-x-el-capitan-solved/2

Please try the attached.

One test currently fails.

Tony

@p5pRT
Copy link
Author

p5pRT commented Oct 4, 2018

From @tonycoz

0001-perl-127606-use-rpath-to-locate-libperl.dylib-on-OS-.patch
From 1d36fa20acb0be4d1f90e8d42c29745afdcb3168 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 4 Oct 2018 14:41:03 +1000
Subject: (perl #127606) use rpath to locate libperl.dylib on OS X with SIP

SIP (System Integrity Protection) on OS X prevents the DYLD_LIBRARY_PATH
environment variable from being propagated through /bin/sh

This causes many tests to fail (and some more recent build issues.)

To avoid that, we change the way libperl.dylib is linked to perl, so
it searches an RPATH stored in the shared object, and that @rpath
name is then propagated to the executable.

During installation we modify the executable and library to use the
installed library path for the library so it can be found in the installed
directory.
---
 Configure   |  3 +++
 Makefile.SH | 38 ++++++++++++++++++++++++++++++++++++++
 installperl | 20 ++++++++++++++++++++
 3 files changed, 61 insertions(+)

diff --git a/Configure b/Configure
index 127583c..cb21bfb 100755
--- a/Configure
+++ b/Configure
@@ -8816,6 +8816,9 @@ if "$useshrplib"; then
 	cygwin)
 		# cygwin needs only ldlibpth
 		;;
+	darwin)
+		# darwin we set it at installation time
+		;;
 	*)
 		tmp_shrpenv="env LD_RUN_PATH=$shrpdir"
 		;;
diff --git a/Makefile.SH b/Makefile.SH
index 18ebd4e..8ccd178 100755
--- a/Makefile.SH
+++ b/Makefile.SH
@@ -62,6 +62,21 @@ true)
 		-compatibility_version 1 -current_version $patchlevel \
 		-prebind -seg1addr 0x27000000 -install_name \$(shrpdir)/\$@"
 		;;
+	darwin1[5-9]*|darwin[2-9]*)
+	        # from OS X 10.11 (darwin 15+), System Integrity Protection
+	        # prevents DYLD_LIBRARY_PATH from being passed to child processes
+	        # through the shell, so we build to find libperl in the build
+	        # directory, then fix the path during installation
+		shrpldflags="${ldflags} -dynamiclib \
+                            -compatibility_version \
+				${api_revision}.${api_version}.${api_subversion} \
+			     -current_version \
+				${revision}.${patchlevel}.${subversion} \
+			     -install_name @rpath/\$@ -Xlinker -headerpad_max_install_names \
+                             -Xlinker -rpath -Xlinker ` quote "$pwd" `"
+		exeldflags="-Xlinker -rpath -Xlinker ` quote "$pwd" `  -Xlinker -headerpad_max_install_names"
+		#linklibperl="-Xlinker @rpath/libperl.dylib"
+		;;
 	darwin*)
 		shrpldflags="${ldflags} -dynamiclib \
                             -compatibility_version \
@@ -69,6 +84,7 @@ true)
 			     -current_version \
 				${revision}.${patchlevel}.${subversion} \
 			     -install_name \$(shrpdir)/\$@"
+		exeldflags=""
 		;;
 	cygwin*)
 		shrpldflags="$shrpldflags -Wl,--out-implib=libperl.dll.a -Wl,--image-base,0x52000000"
@@ -339,6 +355,14 @@ MANIFEST_SRT = MANIFEST.srt
 
 !GROK!THIS!
 
+case "$useshrplib$osname" in
+truedarwin)
+	$spitshell >>$Makefile <<!GROK!THIS!
+PERL_EXE_LDFLAGS=$exeldflags
+!GROK!THIS!
+	;;
+esac
+
 case "$usecrosscompile$perl" in
 define?*)
 	$spitshell >>$Makefile <<!GROK!THIS!
@@ -1050,6 +1074,20 @@ $(PERL_EXE): $& $(perlmain_dep) $(LIBPERL) $(static_ext) ext.libs $(PERLEXPORT)
 	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(LLIBPERL) $(static_ext) `cat ext.libs` $(libs)
 !NO!SUBS!
         ;;
+
+	darwin)
+	    case "$useshrplib$osvers" in
+	    true1[5-9]*|true[2-9]*) $spitshell >>$Makefile <<'!NO!SUBS!'
+	$(SHRPENV) $(CC) -o perl $(PERL_EXE_LDFLAGS) $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
+!NO!SUBS!
+	       ;;
+	    *) $spitshell >>$Makefile <<'!NO!SUBS!'
+	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
+!NO!SUBS!
+	       ;;
+	    esac
+        ;;
+
         *) $spitshell >>$Makefile <<'!NO!SUBS!'
 	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
 !NO!SUBS!
diff --git a/installperl b/installperl
index 3bf79d2..1223d89 100755
--- a/installperl
+++ b/installperl
@@ -304,6 +304,7 @@ elsif ($^O ne 'dos') {
 	safe_unlink("$installbin/$perl_verbase$ver$exe_ext");
 	copy("perl$exe_ext", "$installbin/$perl_verbase$ver$exe_ext");
 	strip("$installbin/$perl_verbase$ver$exe_ext");
+	fix_dep_names("$installbin/$perl_verbase$ver$exe_ext");
 	chmod(0755, "$installbin/$perl_verbase$ver$exe_ext");
     }
     else {
@@ -388,6 +389,7 @@ foreach my $file (@corefiles) {
     if (copy_if_diff($file,"$installarchlib/CORE/$file")) {
 	if ($file =~ /\.(\Q$so\E|\Q$dlext\E)$/) {
 	    strip("-S", "$installarchlib/CORE/$file") if $^O eq 'darwin';
+	    fix_dep_names("$installarchlib/CORE/$file");
 	    chmod($SO_MODE, "$installarchlib/CORE/$file");
 	} else {
 	    chmod($NON_SO_MODE, "$installarchlib/CORE/$file");
@@ -791,4 +793,22 @@ sub strip
     }
 }
 
+sub fix_dep_names {
+    my $file = shift;
+
+    $^O eq "darwin" && $Config{osvers} =~ /^(1[5-9]|[2-9])/
+      && $Config{useshrplib}
+      or return;
+
+    my @opts =
+	  (
+	   "-rpath", getcwd, "$Config{archlibexp}/CORE/",
+	   $file,
+	  );
+
+    $opts{verbose} and print "  install_name_tool @opts\n";
+    system "install_name_tool", @opts
+      and die "Cannot update $file dependency paths\n";
+}
+
 # ex: set ts=8 sts=4 sw=4 et:
-- 
2.8.4 (Apple Git-73)

@p5pRT
Copy link
Author

p5pRT commented Oct 9, 2018

From @tonycoz

On Wed, 03 Oct 2018 22​:51​:15 -0700, tonyc wrote​:

On Sat, 21 Nov 2015 13​:31​:31 -0800, dominyktiller@​gmail.com wrote​:

As per the subject line​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0
./runtests choose
dyld​: Library not loaded​:
/usr/local/Cellar/perl/5.22.0/lib/5.22.0/darwin-thread-multi-
2level/CORE/libperl.dylib
Referenced from​: /private/tmp/perl20151121-6532-elg51d/perl-
5.22.0/t/./perl
Reason​: image not found
./runtests​: line 70​: 26701 Trace/BPT trap​: 5 ./perl $TESTFILE
$TEST_ARGS $TEST_FILES < /dev/tty
make​: *** [test] Error 133
```

OS X El Capitan has a new security feature where the DYLD_LIBRARY_PATH
isn't passed down to a child process, which is probably the root cause
of the failure here.

If you run​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121\-6532\-elg51d/perl\-5.22.0
env | grep DYLD
```

As a test for example, on OS X Mavericks you see​:

```
DYLD_LIBRARY_PATH=/private/tmp/perl20151121-6532-elg51d/perl-5.22.0
```

On OS X El Capitan you get nothing.

This has been reported in various other projects as well​:
* http​://postgresql.nabble.com/OS-X-El-Capitan-and-DYLD-LIBRARY-PATH-
td5869963.html
* https://www.mail-archive.com/pgsql-
hackers@​postgresql.org/msg272269.html
* https://forum.qt.io/topic/59439/dyld-library-not-loaded-when-
debugging-but-ok-when-running-on-os-x-el-capitan-solved/2

Please try the attached.

One test currently fails.

Tony

This one is better.

The original patch made the libperl path rpath relative, but that confused the embedding test build, and would likely have caused similar problems with production embedding, such as with mod_perl2.

The new patch uses absolute paths, adjusting them on installation, which fixes the embedding case.

Tony

@p5pRT
Copy link
Author

p5pRT commented Oct 9, 2018

From @tonycoz

0001-perl-127606-adjust-dependency-paths-on-installation-.patch
From ff9dd435d676c1f00fd5f2516fbf0f9a13939bb3 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 4 Oct 2018 14:41:03 +1000
Subject: (perl #127606) adjust dependency paths on installation on darwin

SIP (System Integrity Protection) on OS X prevents the
DYLD_LIBRARY_PATH environment variable from being propagated through
/bin/sh, causes many tests to fail (and some more recent build issues)
for -Duseshrplib builds.

To avoid that, we change the way libperl.dylib is linked to perl, so
for the initial build the library's id is at the build location rather
than the install location, and the generated executable also expects
to find libperl in that location.

This obviously won't work once we copy both to the installation
directory, so we adjust both the id of the library and the dependency
path in the executable to point to the new location of the library.

A previous attempt set -rpath and used @rpath in the id, but this made
the embedding test fail.
---
 Makefile.SH | 34 ++++++++++++++++++++++++++++++++--
 installperl | 25 +++++++++++++++++++++++++
 2 files changed, 57 insertions(+), 2 deletions(-)

diff --git a/Makefile.SH b/Makefile.SH
index 18ebd4e907..417c020142 100755
--- a/Makefile.SH
+++ b/Makefile.SH
@@ -67,8 +67,16 @@ true)
                             -compatibility_version \
 				${api_revision}.${api_version}.${api_subversion} \
 			     -current_version \
-				${revision}.${patchlevel}.${subversion} \
-			     -install_name \$(shrpdir)/\$@"
+				${revision}.${patchlevel}.${subversion}"
+		case "$osvers" in
+	        1[5-9]*|[2-9]*)
+			shrpldflags="$shrpldflags -install_name `pwd`/\$@ -Xlinker -headerpad_max_install_names"
+			exeldflags="-Xlinker -headerpad_max_install_names"
+			;;
+		*)
+			shrpldflags="$shrpldflags -install_name \$(shrpdir)/\$@"
+			;;
+		esac
 		;;
 	cygwin*)
 		shrpldflags="$shrpldflags -Wl,--out-implib=libperl.dll.a -Wl,--image-base,0x52000000"
@@ -339,6 +347,14 @@ MANIFEST_SRT = MANIFEST.srt
 
 !GROK!THIS!
 
+case "$useshrplib$osname" in
+truedarwin)
+	$spitshell >>$Makefile <<!GROK!THIS!
+PERL_EXE_LDFLAGS=$exeldflags
+!GROK!THIS!
+	;;
+esac
+
 case "$usecrosscompile$perl" in
 define?*)
 	$spitshell >>$Makefile <<!GROK!THIS!
@@ -1050,6 +1066,20 @@ $(PERL_EXE): $& $(perlmain_dep) $(LIBPERL) $(static_ext) ext.libs $(PERLEXPORT)
 	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(LLIBPERL) $(static_ext) `cat ext.libs` $(libs)
 !NO!SUBS!
         ;;
+
+	darwin)
+	    case "$useshrplib$osvers" in
+	    true1[5-9]*|true[2-9]*) $spitshell >>$Makefile <<'!NO!SUBS!'
+	$(SHRPENV) $(CC) -o perl $(PERL_EXE_LDFLAGS) $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
+!NO!SUBS!
+	       ;;
+	    *) $spitshell >>$Makefile <<'!NO!SUBS!'
+	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
+!NO!SUBS!
+	       ;;
+	    esac
+        ;;
+
         *) $spitshell >>$Makefile <<'!NO!SUBS!'
 	$(SHRPENV) $(CC) -o perl $(CLDFLAGS) $(CCDLFLAGS) $(perlmain_objs) $(static_ext) $(LLIBPERL) `cat ext.libs` $(libs)
 !NO!SUBS!
diff --git a/installperl b/installperl
index 3bf79d2d6f..6cd65a0923 100755
--- a/installperl
+++ b/installperl
@@ -304,6 +304,7 @@ elsif ($^O ne 'dos') {
 	safe_unlink("$installbin/$perl_verbase$ver$exe_ext");
 	copy("perl$exe_ext", "$installbin/$perl_verbase$ver$exe_ext");
 	strip("$installbin/$perl_verbase$ver$exe_ext");
+	fix_dep_names("$installbin/$perl_verbase$ver$exe_ext");
 	chmod(0755, "$installbin/$perl_verbase$ver$exe_ext");
     }
     else {
@@ -388,6 +389,7 @@ foreach my $file (@corefiles) {
     if (copy_if_diff($file,"$installarchlib/CORE/$file")) {
 	if ($file =~ /\.(\Q$so\E|\Q$dlext\E)$/) {
 	    strip("-S", "$installarchlib/CORE/$file") if $^O eq 'darwin';
+	    fix_dep_names("$installarchlib/CORE/$file");
 	    chmod($SO_MODE, "$installarchlib/CORE/$file");
 	} else {
 	    chmod($NON_SO_MODE, "$installarchlib/CORE/$file");
@@ -791,4 +793,27 @@ sub strip
     }
 }
 
+sub fix_dep_names {
+    my $file = shift;
+
+    $^O eq "darwin" && $Config{osvers} =~ /^(1[5-9]|[2-9])/
+      && $Config{useshrplib}
+      or return;
+
+    my @opts;
+    my $so = $Config{so};
+    my $libperl = "$Config{archlibexp}/CORE/libperl.$Config{so}";
+    if ($file =~ /\blibperl.\Q$Config{so}\E$/a) {
+        push @opts, -id => $libperl;
+    }
+    else {
+        push @opts, -change => getcwd . "/libperl.$so", $libperl;
+    }
+    push @opts, $file;
+
+    $opts{verbose} and print "  install_name_tool @opts\n";
+    system "install_name_tool", @opts
+      and die "Cannot update $file dependency paths\n";
+}
+
 # ex: set ts=8 sts=4 sw=4 et:
-- 
2.11.0

@p5pRT
Copy link
Author

p5pRT commented Nov 2, 2018

From @tonycoz

On Mon, 08 Oct 2018 22​:28​:39 -0700, tonyc wrote​:

This one is better.

The original patch made the libperl path rpath relative, but that
confused the embedding test build, and would likely have caused
similar problems with production embedding, such as with mod_perl2.

The new patch uses absolute paths, adjusting them on installation,
which fixes the embedding case.

Applied as 191f890.

Tony

@p5pRT
Copy link
Author

p5pRT commented Nov 2, 2018

@tonycoz - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented May 22, 2019

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

@p5pRT
Copy link
Author

p5pRT commented May 22, 2019

@khwilliamson - Status changed from 'pending release' to 'resolved'

@p5pRT p5pRT closed this as completed May 22, 2019
shogo82148 added a commit to shogo82148/devel-patchperl that referenced this issue Nov 7, 2020
bingos pushed a commit to bingos/devel-patchperl that referenced this issue Nov 7, 2020
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Jan 2, 2021
Upstream changes:
version 2.06 at 2020-12-23 17:32:27 +0000
-----------------------------------------

  Change: 3d44f83c8ccc5eba3ad3f449bab2a985464b7726
  Author: Chris 'BinGOs' Williams <chris@bingosnet.co.uk>
  Date : 2020-12-23 17:32:27 +0000

    Fixes for Big Sur

    Dynaloader fix:

    Perl/perl5@d296ead
    56711f77

    ExtUtils::MakeMaker fix:

    Perl-Toolchain-Gang/ExtUtils-MakeMaker@21a4
    41da515d55654bca4971f21ad2550de069d2

    Made adjustments to the check_patchperl tool

-----------------------------------------
version 2.04 at 2020-11-20 12:36:29 +0000
-----------------------------------------

  Change: 771e4fd9595bcc90f482f454203d84c568814966
  Author: Chris 'BinGOs' Williams <chris@bingosnet.co.uk>
  Date : 2020-11-20 12:36:29 +0000

    Tweaked hints patching

    Hints file patching now occurs before other patching and has an
    independent certified perl version.

    Updated included hints files to v5.33.4 level.

-----------------------------------------
version 2.02 at 2020-11-08 19:28:50 +0000
-----------------------------------------

  Change: f02eab39c2f0073e6aceff6bb0b73f4d03e39b8f
  Author: Chris 'BinGOs' Williams <chris@bingosnet.co.uk>
  Date : 2020-11-08 19:28:50 +0000

    This is 2.02

  Change: fa458343e6ae7f70412c466eb604633db44bf24b
  Author: Chris 'BinGOs' Williams <chris@bingosnet.co.uk>
  Date : 2020-11-08 19:19:39 +0000

    Finesse the check_patchperl script

    Suppress the git output and only show patchperl output if there is a
    problem with patching

  Change: d4fe0e9fa23d0fd78b368a5edb34550120e2c233
  Author: Ichinose Shogo <shogo82148@gmail.com>
  Date : 2020-11-08 19:18:21 +0000

    fix env command syntax

  Change: 20692277283d6eb1616eab987583b33cfb612198
  Author: Ichinose Shogo <shogo82148@gmail.com>
  Date : 2020-11-08 19:18:21 +0000

    update deprecated GitHub Actions commands

    add-path and set-env are deprecated for security reason.
    https://github.blog/changelog/2020-10-01-github-actions-deprecating-s
    et-env-and-add-path-commands/ update them to use the new environment
    files.
    https://docs.github.com/en/free-pro-team@latest/actions/reference/wor
    kflow-commands-for-github-actions#environment-files

  Change: 175ccb4bc8c88a0e3e3f4600fe1db09f023a9999
  Author: Ichinose Shogo <shogo82148@gmail.com>
  Date : 2020-11-07 09:17:00 +0000

    fix -Duseshrplib builds on darwin

    port of Perl/perl5#15057

  Change: 7c806f542ffd0b5e75d724404012fd8bc9afc4e4
  Author: Ichinose Shogo <shogo82148@gmail.com>
  Date : 2020-11-07 09:12:34 +0000

    Patch Time::HiRes in 5.24.2

    Time::HiRes is broken on threaded darwin in 5.24.2, not only 5.24.1
    and 5.24.0. bingos/devel-patchperl#26 is need
    in Perl 5.24.2.

  Change: eaf8ec2500acde369ad78aea12d1de0011480a0e
  Author: Ichinose Shogo <shogo82148@gmail.com>
  Date : 2020-06-23 15:26:13 +0000

    Perl 5.32.0 is released

  Change: 0b60d5d7d06de737cc4e033f658aad8cff906657
  Author: Chris 'BinGOs' Williams <chris@bingosnet.co.uk>
  Date : 2020-06-09 16:26:25 +0000

    Fix make depend problems with v5.9.[0-4]

  Change: df4f5009c804675bf97160490a95af8a67ded395
  Author: Chris 'BinGOs' Williams <chris@bingosnet.co.uk>
  Date : 2020-06-09 16:00:56 +0000

    Fix patching the v5.7.x series

  Change: 3dce408aaf97a3b245b29303f20b30a7584fb362
  Author: Chris 'BinGOs' Williams <chris@bingosnet.co.uk>
  Date : 2020-06-08 21:13:06 +0000

    Added a tool to check patchperl from the repository

    It checkouts the perl5 git repository and runs patchperl for all of
    the supported perl versions

  Change: 3e819851d4e28806c629ed783ef56b02f6e5ad12
  Author: Chris 'BinGOs' Williams <chris@bingosnet.co.uk>
  Date : 2020-06-08 19:09:04 +0000

    Fixed utils/h2ph.PL patching

    v5.21.9 actually already had the fix

  Change: 647e88747c8469d9c9b418a78a5b999984f17acb
  Author: Ichinose Shogo <shogo82148@gmail.com>
  Date : 2020-06-08 12:28:52 +0000

    introduce GitHub Actions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants