Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perl 5.24.1 fails to compile with DTrace enabled on FreeBSD #15718

Closed
p5pRT opened this issue Nov 15, 2016 · 20 comments
Closed

Perl 5.24.1 fails to compile with DTrace enabled on FreeBSD #15718

p5pRT opened this issue Nov 15, 2016 · 20 comments

Comments

@p5pRT
Copy link

p5pRT commented Nov 15, 2016

Migrated from rt.perl.org#130108 (status was 'resolved')

Searchable as RT130108$

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2016

From @swills

Created by @swills

This is a bug report for perl from steve@​mouf.net,
generated with the help of perlbug 1.40 running under perl 5.20.3.

-----------------------------------------------------------------
When enabling DTrace support via -Dusedtrace, perl 5.24.1 fails to
compile with
this error message​:

--- dtrace_main.o ---
dtrace​: failed to link script perldtrace.d​: No probe sites found for
declared provider

This is under FreeBSD 12.0-CURRENT, this same applies to 11.0-RELEASE. The
dtrace version is​:

dtrace​: Sun D 1.13

Looks like dtrace_main.o isn't generated becayse perlmain.o has no
probes and
unlike when dtrace_mini.o is generated, no other files with probes are
linked.

Also it seems that if at this point in the build you remove
dtrace_mini.o and
try to build it again, it fails with the same "no probes found" message.

I think this indicates that dtrace is enabled for miniperl, but not for
perl.

(Had to use perl on another system to report the error, ignore references to
5.20.3 and it's config in this bug report)

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.20.3:

Configured by perl at Sat Sep 12 19:09:14 UTC 2015.

Summary of my perl5 (revision 5 version 20 subversion 3) configuration:
     Platform:
    osname=freebsd, osvers=12.0-current, archname=amd64-freebsd-thread-multi
    uname='freebsd head-amd64-default-job-02 12.0-current freebsd
12.0-current r306554 amd64 '
    config_args='-sde -Dprefix=/usr/local -Dlibperl=libperl.so.5.20.3
-Darchlib=/usr/local/lib/perl5/5.20/mach
-Dprivlib=/usr/local/lib/perl5/5.20
-Dman3dir=/usr/local/lib/perl5/5.20/perl/man/man3
-Dman1dir=/usr/local/lib/perl5/5.20/perl/man/man1
-Dsitearch=/usr/local/lib/perl5/site_perl/mach/5.20
-Dsitelib=/usr/local/lib/perl5/site_perl -Dscriptdir=/usr/local/bin
-Dsiteman3dir=/usr/local/lib/perl5/site_perl/man/man3
-Dsiteman1dir=/usr/local/lib/perl5/site_perl/man/man1
-Dotherlibdirs=/usr/local/lib/perl5/site_perl/5.20:/usr/local/lib/perl5/site_perl/5.20/mach
-Ui_malloc -Ui_iconv -Uinstallusrbinperl -Dusenm=n -Dcc=cc -Duseshrplib
-Dinc_version_list=none -Dcf_by=perl -Dcf_email=perl@FreeBSD.org
-Dcf_time=Sat Sep 12 19:09:14 UTC 2015
-Alddlflags=-L/wrkdirs/usr/ports/lang/perl5.20/work/perl-5.20.3
-L/usr/local/lib/perl5/5.20/mach/CORE -lperl
-Dshrpldflags=$(LDDLFLAGS:N-L/wrkdirs/usr/ports/lang/perl5.20/work/perl-5.20.3:N-L/usr/local/lib/perl5/5.20/mach/CORE:N-lperl)
-Wl,-sonam!
 e,$(LIBPERL:R) -Doptimize=-O2 -pipe  -fstack-protector
-fno-strict-aliasing -Ui_gdbm -Dusemultiplicity=y -Duse64bitint
-Dusethreads=y -Dusemymalloc=n'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include',
    optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing',
    cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.2.1 Compatible FreeBSD Clang 3.8.0
(tags/RELEASE_380/final 262564)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags ='-lpthread -Wl,-E  -fstack-protector -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib /usr/bin/../lib/clang/3.8.0/lib /usr/lib
    libs=-lpthread -lm -lcrypt -lutil
    perllibs=-lpthread -lm -lcrypt -lutil
    libc=, so=so, useshrplib=true, libperl=libperl.so.5.20.3
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='
-Wl,-R/usr/local/lib/perl5/5.20/mach/CORE'
    cccdlflags='-DPIC -fPIC', lddlflags='-shared
-L/usr/local/lib/perl5/5.20/mach/CORE -lperl -L/usr/local/lib
-fstack-protector'



@INC for perl 5.20.3:
    /usr/local/lib/perl5/site_perl/mach/5.20
    /usr/local/lib/perl5/site_perl
    /usr/local/lib/perl5/5.20/mach
    /usr/local/lib/perl5/5.20
    /usr/local/lib/perl5/site_perl/5.20
    /usr/local/lib/perl5/site_perl/5.20/mach


Environment for perl 5.20.3:
    HOME=/home/swills
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LC_COLLATE=C
    LC_CTYPE=en_US.UTF-8
    LC_MESSAGE=en_US.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=/usr/local/bin:/opt/local/bin:/opt/local/sbin:/usr/X11R6/bin:/usr/local/bin:/opt/local/bin:/opt/local/sbin:/usr/X11R6/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/home/swills/bin:/usr/local/poudriere/ports/default/Tools/scripts/:/usr/local/poudriere/ports/default/Tools/scripts/
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Nov 17, 2016

From @tonycoz

On Tue, 15 Nov 2016 08​:31​:15 -0800, swills wrote​:

When enabling DTrace support via -Dusedtrace, perl 5.24.1 fails to
compile with
this error message​:

--- dtrace_main.o ---
dtrace​: failed to link script perldtrace.d​: No probe sites found for
declared provider

This is under FreeBSD 12.0-CURRENT, this same applies to 11.0-RELEASE.
The
dtrace version is​:

dtrace​: Sun D 1.13

Looks like dtrace_main.o isn't generated becayse perlmain.o has no
probes and
unlike when dtrace_mini.o is generated, no other files with probes are
linked.

Also it seems that if at this point in the build you remove
dtrace_mini.o and
try to build it again, it fails with the same "no probes found"
message.

I think this indicates that dtrace is enabled for miniperl, but not
for
perl.

(Had to use perl on another system to report the error, ignore
references to
5.20.3 and it's config in this bug report)

dtrace -G is run separately for perlmain because if perl is built with a shared perl lib (needed for dtrace on Solaris AFAIK) perlmain.o (and dtrace_main.o) is used to build the
executable while the rest of the objects (including dtrace_perllib.o) are used to build libperl.so

The build on Solaris does pick up the probes included in the
unused inline functions in inline.h, and so the build failed
before we generated a separate probes object file for perlmain.o ( see https://rt-archive.perl.org/perl5/Ticket/Display.html?id=127543 ).

On FreeBSD perlmain.o isn't picking up the probes from inline.h
and so dtrace -G fails.

The attached patch generates a dummy dtrace_main.o if dtrace -G fails on perlmain.o.

The real fix would be for dtrace -G to not fail just because a particular object in the build didn't have any probes.

Tony

@p5pRT
Copy link
Author

p5pRT commented Nov 17, 2016

From @tonycoz

0001-perl-130108-generate-a-dummy-dtrace_main.o-if-perlma.patch
From 75cab829db98cce677b3c051dff22856090af0f7 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 17 Nov 2016 22:18:30 +1100
Subject: [PATCH] (perl #130108) generate a dummy dtrace_main.o if perlmain.o
 doesn't contain probes

efc4bddfd4 added generating a probes object file for perlmain.o, since
the compiler was generating probes even for unused inline functions.

The default compiler on FreeBSD 11 however doesn't generate probes for
these unused inline functions, and dtrace -G fails because it can't
find any.

So if dtrace fails for perlmain.o generate a dummy object file to
take its place.

Similarly for XS::APItest.
---
 Makefile.SH                | 6 +++++-
 ext/XS-APItest/Makefile.PL | 3 ++-
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/Makefile.SH b/Makefile.SH
index 491d1a6..3ead3c1 100755
--- a/Makefile.SH
+++ b/Makefile.SH
@@ -866,7 +866,11 @@ $(DTRACE_PERLLIB_O): perldtrace.d $(perllib_objs_nodt)
 	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_PERLLIB_O) $(perllib_objs_nodt)
 
 $(DTRACE_MAIN_O): perldtrace.d perlmain$(OBJ_EXT)
-	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_MAIN_O) perlmain$(OBJ_EXT)
+	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_MAIN_O) perlmain$(OBJ_EXT) ||		      \
+	  ( $(ECHO) "No probes in perlmain$(OBJ_EXT), generating a dummy $(DTRACE_MAIN_O)" && \
+	    $(ECHO) >dtrace_main.c &&							      \
+	    `$(CCCMD)` $(PLDLFLAGS) dtrace_main.c &&					      \
+	     rm -f dtrace_main.c )
 
 !NO!SUBS!
 		;;
diff --git a/ext/XS-APItest/Makefile.PL b/ext/XS-APItest/Makefile.PL
index c06fac6..a104e2f 100644
--- a/ext/XS-APItest/Makefile.PL
+++ b/ext/XS-APItest/Makefile.PL
@@ -59,7 +59,8 @@ sub MY::postamble
 DTRACE_D = ../../perldtrace.d
 
 dtrace\$(OBJ_EXT): \$(DTRACE_D) core\$(OBJ_EXT)
-	$Config{dtrace} -G -s \$(DTRACE_D) -o dtrace\$(OBJ_EXT) core\$(OBJ_EXT)
+	$Config{dtrace} -G -s \$(DTRACE_D) -o dtrace\$(OBJ_EXT) core\$(OBJ_EXT) || \\
+	  ( \$(ECHO) >dtrace.c && \$(CCCMD) \$(CCCDLFLAGS) dtrace.c && rm -f dtrace.c )
 POSTAMBLE
 
     return $post;
-- 
2.1.4

@p5pRT
Copy link
Author

p5pRT commented Nov 17, 2016

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Nov 17, 2016

From @jkeenan

On Thu, 17 Nov 2016 01​:04​:23 GMT, tonyc wrote​:

On Tue, 15 Nov 2016 08​:31​:15 -0800, swills wrote​:

When enabling DTrace support via -Dusedtrace, perl 5.24.1 fails to
compile with
this error message​:

--- dtrace_main.o ---
dtrace​: failed to link script perldtrace.d​: No probe sites found for
declared provider

This is under FreeBSD 12.0-CURRENT, this same applies to 11.0-
RELEASE.
The
dtrace version is​:

dtrace​: Sun D 1.13

Looks like dtrace_main.o isn't generated becayse perlmain.o has no
probes and
unlike when dtrace_mini.o is generated, no other files with probes
are
linked.

Also it seems that if at this point in the build you remove
dtrace_mini.o and
try to build it again, it fails with the same "no probes found"
message.

I think this indicates that dtrace is enabled for miniperl, but not
for
perl.

(Had to use perl on another system to report the error, ignore
references to
5.20.3 and it's config in this bug report)

dtrace -G is run separately for perlmain because if perl is built with
a shared perl lib (needed for dtrace on Solaris AFAIK) perlmain.o (and
dtrace_main.o) is used to build the
executable while the rest of the objects (including dtrace_perllib.o)
are used to build libperl.so

The build on Solaris does pick up the probes included in the
unused inline functions in inline.h, and so the build failed
before we generated a separate probes object file for perlmain.o ( see
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=127543 ).

On FreeBSD perlmain.o isn't picking up the probes from inline.h
and so dtrace -G fails.

The attached patch generates a dummy dtrace_main.o if dtrace -G fails
on perlmain.o.

The real fix would be for dtrace -G to not fail just because a
particular object in the build didn't have any probes.

Tony

For reference, I'm attaching the tail of the output of 'make' run at commit 75cab82 in the smoke-me/tonyc/dtrace-no-main-probes branch run on FreeBSD-10.3.

sh ./Configure -des -Dusedevel -Duseithreads -Dusedtrace

Build failure here as well. Full output available if needed.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Nov 17, 2016

From @jkeenan

--- dtrace_mini.o ---
/usr/sbin/dtrace -G -s perldtrace.d -o dtrace_mini.o opmini.o perlmini.o gv.o toke.o perly.o pad.o regcomp.o dump.o util.o mg.o reentr.o mro_core.o keywords.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o regexec.o utf8.o taint.o deb.o universal.o globals.o perlio.o perlapi.o numeric.o mathoms.o locale.o pp_pack.o pp_sort.o caretx.o dquote.o time64.o miniperlmain.o
--- lib/buildcustomize.pl ---
cc -pthread -Wl,-E -fstack-protector -L/usr/local/lib -o miniperl opmini.o perlmini.o gv.o toke.o perly.o pad.o regcomp.o dump.o util.o mg.o reentr.o mro_core.o keywords.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o regexec.o utf8.o taint.o deb.o universal.o globals.o perlio.o perlapi.o numeric.o mathoms.o locale.o pp_pack.o pp_sort.o caretx.o dquote.o time64.o miniperlmain.o dtrace_mini.o -lpthread -lm -lcrypt -lutil
dtrace_mini.o​: In function `dtrace_dof_init'​:
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0xd7)​: undefined reference to `elf_version'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x10f)​: undefined reference to `elf_begin'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x131)​: undefined reference to `elf_getshdrstrndx'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x13b)​: undefined reference to `elf_nextscn'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x1a2)​: undefined reference to `elf_getdata'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x1ce)​: undefined reference to `elf_getdata'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x1e1)​: undefined reference to `elf_getdata'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x207)​: undefined reference to `gelf_getshdr'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x22b)​: undefined reference to `elf_strptr'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x24e)​: undefined reference to `elf_nextscn'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x5ef)​: undefined reference to `elf_end'
dtrace_mini.o​: In function `fixsymbol'​:
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x7ab)​: undefined reference to `gelf_getsym'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x7e3)​: undefined reference to `elf_strptr'
cc​: error​: linker command failed with exit code 1 (use -v to see invocation)
*** [lib/buildcustomize.pl] Error code 1

make​: stopped in /usr/home/jkeenan/gitwork/perl
1 error

make​: stopped in /usr/home/jkeenan/gitwork/perl

@p5pRT
Copy link
Author

p5pRT commented Nov 17, 2016

From @swills

Hi,

Thanks, that fixes the build. But I only see 3 probes in perl​:

70179 perl99307 libperl.so.5.24 perl_run
op-entry
70180 perl99307 libperl.so.5.24 perl_parse
op-entry
70181 perl99307 libperl.so.5.24 perl_destruct
op-entry

When I run miniperl I see many more​:

70179 perl99684 miniperl Perl_dtrace_probe_load
loaded-file
70180 perl99684 miniperl Perl_dtrace_probe_load
loading-file
70181 perl99684 miniperl Perl_runops_standard
op-entry
70182 perl99684 miniperl Perl_dtrace_probe_op
op-entry
70183 perl99684 miniperl Perl_runops_debug
op-entry
70184 perl99684 miniperl perl_run
op-entry
70185 perl99684 miniperl perl_parse
op-entry
70186 perl99684 miniperl perl_destruct
op-entry
70187 perl99684 miniperl Perl_dtrace_probe_phase
phase-change
70188 perl99684 miniperl Perl_pp_sort
sub-entry
70189 perl99684 miniperl S_regmatch
sub-entry
70190 perl99684 miniperl Perl_pp_require
sub-entry
70191 perl99684 miniperl Perl_pp_dbstate
sub-entry
70192 perl99684 miniperl S_cx_popsub
sub-entry
70193 perl99684 miniperl Perl_pp_entersub
sub-entry
70194 perl99684 miniperl Perl_pp_leavesub
sub-entry
70195 perl99684 miniperl Perl_dtrace_probe_call
sub-entry
70196 perl99684 miniperl Perl_dtrace_probe_call
sub-return

Would be nice to have all the probes in perl.

Steve

On 11/16/2016 20​:04, Tony Cook via RT wrote​:

On Tue, 15 Nov 2016 08​:31​:15 -0800, swills wrote​:

When enabling DTrace support via -Dusedtrace, perl 5.24.1 fails to
compile with
this error message​:

--- dtrace_main.o ---
dtrace​: failed to link script perldtrace.d​: No probe sites found for
declared provider

This is under FreeBSD 12.0-CURRENT, this same applies to 11.0-RELEASE.
The
dtrace version is​:

dtrace​: Sun D 1.13

Looks like dtrace_main.o isn't generated becayse perlmain.o has no
probes and
unlike when dtrace_mini.o is generated, no other files with probes are
linked.

Also it seems that if at this point in the build you remove
dtrace_mini.o and
try to build it again, it fails with the same "no probes found"
message.

I think this indicates that dtrace is enabled for miniperl, but not
for
perl.

(Had to use perl on another system to report the error, ignore
references to
5.20.3 and it's config in this bug report)
dtrace -G is run separately for perlmain because if perl is built with a shared perl lib (needed for dtrace on Solaris AFAIK) perlmain.o (and dtrace_main.o) is used to build the
executable while the rest of the objects (including dtrace_perllib.o) are used to build libperl.so

The build on Solaris does pick up the probes included in the
unused inline functions in inline.h, and so the build failed
before we generated a separate probes object file for perlmain.o ( see https://rt-archive.perl.org/perl5/Ticket/Display.html?id=127543 ).

On FreeBSD perlmain.o isn't picking up the probes from inline.h
and so dtrace -G fails.

The attached patch generates a dummy dtrace_main.o if dtrace -G fails on perlmain.o.

The real fix would be for dtrace -G to not fail just because a particular object in the build didn't have any probes.

Tony

@p5pRT
Copy link
Author

p5pRT commented Nov 17, 2016

From @tonycoz

On Wed, 16 Nov 2016 19​:44​:45 -0800, jkeenan wrote​:

sh ./Configure -des -Dusedevel -Duseithreads -Dusedtrace

Build failure here as well. Full output available if needed.

Thank you very much.

/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x7ab)​: undefined reference to `gelf_getsym'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x7e3)​: undefined reference to `elf_strptr'

The attached should fix that. I haven't tested 9.x, it may need the
same treatment.

I'll take a look at the missing probes next week.

Tony

@p5pRT
Copy link
Author

p5pRT commented Nov 17, 2016

From @tonycoz

0001-add-elf-to-libswanted-on-FreeBSD-10.x.patch
From 92c8ceadcd74e0a4750b506995bdf7664e823064 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Fri, 18 Nov 2016 03:31:06 +1100
Subject: [PATCH] add elf to libswanted on FreeBSD 10.x

usedtrace builds add references to libelf symbols, causing link
failures without it.

at hints time we don't know if the user will interactively select
dtrace and there's no CBU, so it's added unconditionally on 10.x
---
 hints/freebsd.sh | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/hints/freebsd.sh b/hints/freebsd.sh
index 135129f..35fea6a 100644
--- a/hints/freebsd.sh
+++ b/hints/freebsd.sh
@@ -105,6 +105,15 @@ case "$osvers" in
 	;;
 esac
 
+case "$osvers" in
+10.*)
+	# dtrace on 10.x needs libelf symbols, but we don't know if the
+	# user is going to request usedtrace and there's no .cbu for usedtrace
+	libswanted="$libswanted elf"
+	echo "libswanted $libswanted" >&4
+	;;
+esac     
+
 # Dynamic Loading flags have not changed much, so they are separated
 # out here to avoid duplicating them everywhere.
 case "$osvers" in
@@ -312,3 +321,4 @@ d_printf_format_null='undef'
 
 # As of 10.3-RELEASE FreeBSD.  See [perl #128867]
 d_uselocale='undef'
+
-- 
2.1.4

@p5pRT
Copy link
Author

p5pRT commented Nov 17, 2016

From @jkeenan

On Thu, 17 Nov 2016 05​:37​:32 GMT, tonyc wrote​:

On Wed, 16 Nov 2016 19​:44​:45 -0800, jkeenan wrote​:

sh ./Configure -des -Dusedevel -Duseithreads -Dusedtrace

Build failure here as well. Full output available if needed.

Thank you very much.

/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x7ab)​:
undefined reference to `gelf_getsym'
/usr/src/cddl/lib/drti/../../../cddl/contrib/opensolaris/lib/libdtrace/common/drti.c​:(.text+0x7e3)​:
undefined reference to `elf_strptr'

The attached should fix that. I haven't tested 9.x, it may need the
same treatment.

I'll take a look at the missing probes next week.

Tony

I applied this patch on FreeBSD-10.3 to the smoke-me/tonyc/dtrace-no-main-probes branch. This time 'make' completed successfully and 'make test' completed with failures (expected) only in lib/locale.t.

I'll send you typescripts of the runs off list.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Nov 21, 2016

From @tonycoz

On Wed, 16 Nov 2016 20​:36​:28 -0800, swills wrote​:

Hi,

Thanks, that fixes the build. But I only see 3 probes in perl​:

70179 perl99307 libperl.so.5.24 perl_run
op-entry
70180 perl99307 libperl.so.5.24 perl_parse
op-entry
70181 perl99307 libperl.so.5.24 perl_destruct
op-entry

This looks like you're doing a -Duseshrplib build.

I tried that and couldn't even build miniperl, getting an unhelpful error​:

/usr/sbin/dtrace -G -s perldtrace.d -o dtrace_mini.o opmini.o perlmini.o gv.o toke.o perly.o pad.o regcomp.o dump.o util.o mg.o reentr.o mro_core.o keywords.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o regexec.o utf8.o taint.o deb.o universal.o globals.o perlio.o perlapi.o numeric.o mathoms.o locale.o pp_pack.o pp_sort.o caretx.o dquote.o time64.o miniperlmain.o
dtrace​: failed to link script perldtrace.d​: an error was encountered while processing util.o

I tried a non -Duseshrplib and got no probes at all, the generated executable does include dtrace entry point symbols
but show no probes.

The miniperl generated from largely the same object files did have probes.

Unfortunately dtrace is opaque enough that there's really no way for us to fix this.

I'll do some checking on Solaris, but I suspect this will need
some support from FreeBSD people who know there way around userland dtrace.

Tony

@p5pRT
Copy link
Author

p5pRT commented Nov 22, 2016

From @swills

The reason perl has no probes is because when dtrace -G is called, it
overwrites the probe sites in the .o files with NOPs. Which means when
the same .o files that were used for miniperl are used for perl, they
lack probes.

Steve

On 11/20/2016 20​:04, Tony Cook via RT wrote​:

On Wed, 16 Nov 2016 20​:36​:28 -0800, swills wrote​:

Hi,

Thanks, that fixes the build. But I only see 3 probes in perl​:

70179 perl99307 libperl.so.5.24 perl_run
op-entry
70180 perl99307 libperl.so.5.24 perl_parse
op-entry
70181 perl99307 libperl.so.5.24 perl_destruct
op-entry
This looks like you're doing a -Duseshrplib build.

I tried that and couldn't even build miniperl, getting an unhelpful error​:

/usr/sbin/dtrace -G -s perldtrace.d -o dtrace_mini.o opmini.o perlmini.o gv.o toke.o perly.o pad.o regcomp.o dump.o util.o mg.o reentr.o mro_core.o keywords.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o regexec.o utf8.o taint.o deb.o universal.o globals.o perlio.o perlapi.o numeric.o mathoms.o locale.o pp_pack.o pp_sort.o caretx.o dquote.o time64.o miniperlmain.o
dtrace​: failed to link script perldtrace.d​: an error was encountered while processing util.o

I tried a non -Duseshrplib and got no probes at all, the generated executable does include dtrace entry point symbols
but show no probes.

The miniperl generated from largely the same object files did have probes.

Unfortunately dtrace is opaque enough that there's really no way for us to fix this.

I'll do some checking on Solaris, but I suspect this will need
some support from FreeBSD people who know there way around userland dtrace.

Tony

@p5pRT
Copy link
Author

p5pRT commented Nov 28, 2016

From @tonycoz

On Mon, 21 Nov 2016 18​:31​:57 -0800, swills wrote​:

The reason perl has no probes is because when dtrace -G is called, it
overwrites the probe sites in the .o files with NOPs. Which means when
the same .o files that were used for miniperl are used for perl, they
lack probes.

Attached the patches we've been testing.

A single file to simplify working with it.

Tony

@p5pRT
Copy link
Author

p5pRT commented Nov 28, 2016

From @tonycoz

freebsd-dtrace.patch
From 246444ecc9db2d8dc5b522124900e771b660dceb Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 17 Nov 2016 22:18:30 +1100
Subject: (perl #130108) generate a dummy dtrace_main.o if perlmain.o doesn't
 contain probes

efc4bddfd4 added generating a probes object file for perlmain.o, since
the compiler was generating probes even for unused inline functions.

The default compiler on FreeBSD 11 however doesn't generate probes for
these unused inline functions, and dtrace -G fails because it can't
find any.

So if dtrace fails for perlmain.o generate a dummy object file to
take its place.

Similarly for XS::APItest.
---
 Makefile.SH                | 6 +++++-
 ext/XS-APItest/Makefile.PL | 3 ++-
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/Makefile.SH b/Makefile.SH
index a094f87..b295a48 100755
--- a/Makefile.SH
+++ b/Makefile.SH
@@ -866,7 +866,11 @@ $(DTRACE_PERLLIB_O): perldtrace.d $(perllib_objs_nodt)
 	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_PERLLIB_O) $(perllib_objs_nodt)
 
 $(DTRACE_MAIN_O): perldtrace.d perlmain$(OBJ_EXT)
-	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_MAIN_O) perlmain$(OBJ_EXT)
+	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_MAIN_O) perlmain$(OBJ_EXT) ||		      \
+	  ( $(ECHO) "No probes in perlmain$(OBJ_EXT), generating a dummy $(DTRACE_MAIN_O)" && \
+	    $(ECHO) >dtrace_main.c &&							      \
+	    `$(CCCMD)` $(PLDLFLAGS) dtrace_main.c &&					      \
+	     rm -f dtrace_main.c )
 
 !NO!SUBS!
 		;;
diff --git a/ext/XS-APItest/Makefile.PL b/ext/XS-APItest/Makefile.PL
index c06fac6..a104e2f 100644
--- a/ext/XS-APItest/Makefile.PL
+++ b/ext/XS-APItest/Makefile.PL
@@ -59,7 +59,8 @@ sub MY::postamble
 DTRACE_D = ../../perldtrace.d
 
 dtrace\$(OBJ_EXT): \$(DTRACE_D) core\$(OBJ_EXT)
-	$Config{dtrace} -G -s \$(DTRACE_D) -o dtrace\$(OBJ_EXT) core\$(OBJ_EXT)
+	$Config{dtrace} -G -s \$(DTRACE_D) -o dtrace\$(OBJ_EXT) core\$(OBJ_EXT) || \\
+	  ( \$(ECHO) >dtrace.c && \$(CCCMD) \$(CCCDLFLAGS) dtrace.c && rm -f dtrace.c )
 POSTAMBLE
 
     return $post;
-- 
2.1.4


From 65a4e4e1ad005cf1ff838309a161454e9d6591cd Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Mon, 28 Nov 2016 15:22:16 +1100
Subject: (perl #130108) add elf to libswanted on FreeBSD 10.x

usedtrace builds add references to libelf symbols, causing link
failures without it.

at hints time we don't know if the user will interactively select
dtrace and there's no CBU, so it's added unconditionally on 10.x
---
 hints/freebsd.sh | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/hints/freebsd.sh b/hints/freebsd.sh
index 135129f..06b0bfd 100644
--- a/hints/freebsd.sh
+++ b/hints/freebsd.sh
@@ -105,6 +105,14 @@ case "$osvers" in
 	;;
 esac
 
+case "$osvers" in
+10.*)
+	# dtrace on 10.x needs libelf symbols, but we don't know if the
+	# user is going to request usedtrace and there's no .cbu for usedtrace
+	libswanted="$libswanted elf"
+	;;
+esac
+
 # Dynamic Loading flags have not changed much, so they are separated
 # out here to avoid duplicating them everywhere.
 case "$osvers" in
-- 
2.1.4


From b8c2ffb6afd2b7041addf55107e5a775fd2de236 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Mon, 28 Nov 2016 15:19:12 +1100
Subject: (perl #130108) separate compiled objects from dtrace modified objects

When generating an object file with "dtrace -G", dtrace also modifies
the input object files.

This causes two problems:

1) Since the objects are modified, their modification times are updated
which may break dependency checks by make.

2) on FreeBSD at least, once the object has been processed it can't be
processed again by dtrace -G
---
 Makefile.SH | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 58 insertions(+), 7 deletions(-)

diff --git a/Makefile.SH b/Makefile.SH
index b295a48..7a4ed42 100755
--- a/Makefile.SH
+++ b/Makefile.SH
@@ -518,10 +518,52 @@ main_only_objs = op$(OBJ_EXT)     perl$(OBJ_EXT)
 miniperl_objs_nodt = $(mini_only_objs) $(common_objs) miniperlmain$(OBJ_EXT)
 perllib_objs_nodt  = $(main_only_objs) $(common_objs)
 
+!NO!SUBS!
+
+# dtrace with -G modifies the source object files, which can cause
+# dependency issues, and can cause the dtrace -G to fail on FreeBSD
+# so separate the objects generated by $(CC) from those used to link
+# the executable when dtrace -G is involved.
+#
+# $(FOO:op%os=np%ns) isn't generally portable but is portable to
+# the makes on darwin, Solaris, FreeBSD and Linux, which is where we
+# use dtrace
+
+case "$usedtrace:$dtraceobject" in
+define:define)
+    $spitshell >>$Makefile <<'!NO!SUBS!'
+
+miniperl_dtrace_objs = $(miniperl_objs_nodt:%=mpdtrace/%)
+perllib_dtrace_objs = $(perllib_objs_nodt:%=libpdtrace/%)
+perlmain_dtrace_objs = maindtrace/perlmain$(OBJ_EXT)
+
+miniperl_objs = $(miniperl_dtrace_objs) $(DTRACE_MINI_O)
+perllib_objs  = $(perllib_dtrace_objs) $(DTRACE_PERLLIB_O)
+perlmain_objs = $(perlmain_dtrace_objs) $(DTRACE_MAIN_O)
+
+miniperl_dep = $(DTRACE_MINI_O)
+perllib_dep = $(DTRACE_PERLLIB_O)
+perlmain_dep = $(DTRACE_MAIN_O)
+
+!NO!SUBS!
+    ;;
+*)
+    $spitshell >>$Makefile <<'!NO!SUBS!'
+
 miniperl_objs = $(miniperl_objs_nodt) $(DTRACE_MINI_O)
 perllib_objs  = $(perllib_objs_nodt) $(DTRACE_PERLLIB_O)
 perlmain_objs = perlmain$(OBJ_EXT) $(DTRACE_MAIN_O)
 
+miniperl_dep = $(miniperl_objs)
+perllib_dep = $(perllib_objs)
+perlmain_dep = $(perlmain_objs)
+
+!NO!SUBS!
+    ;;
+esac
+
+$spitshell >>$Makefile <<'!NO!SUBS!'
+
 perltoc_pod_prereqs = extra.pods pod/perl5258delta.pod pod/perlapi.pod pod/perlintern.pod pod/perlmodlib.pod pod/perluniprops.pod
 generated_pods = pod/perltoc.pod $(perltoc_pod_prereqs)
 generated_headers = uudmap.h bitcount.h mg_data.h
@@ -860,13 +902,22 @@ mydtrace.h: $(DTRACE_H)
 	define)
 		$spitshell >>$Makefile <<'!NO!SUBS!'
 $(DTRACE_MINI_O): perldtrace.d $(miniperl_objs_nodt)
-	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_MINI_O) $(miniperl_objs_nodt)
+	-rm -rf mpdtrace
+	mkdir mpdtrace
+	cp $(miniperl_objs_nodt) mpdtrace/
+	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_MINI_O) $(miniperl_dtrace_objs)
 
 $(DTRACE_PERLLIB_O): perldtrace.d $(perllib_objs_nodt)
-	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_PERLLIB_O) $(perllib_objs_nodt)
+	-rm -rf libpdtrace
+	mkdir libpdtrace
+	cp $(perllib_objs_nodt) libpdtrace/
+	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_PERLLIB_O) $(perllib_dtrace_objs)
 
 $(DTRACE_MAIN_O): perldtrace.d perlmain$(OBJ_EXT)
-	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_MAIN_O) perlmain$(OBJ_EXT) ||		      \
+	-rm -rf maindtrace
+	mkdir maindtrace
+	cp perlmain$(OBJ_EXT) maindtrace/
+	$(DTRACE) -G -s perldtrace.d -o $(DTRACE_MAIN_O) $(perlmain_dtrace_objs) ||	      \
 	  ( $(ECHO) "No probes in perlmain$(OBJ_EXT), generating a dummy $(DTRACE_MAIN_O)" && \
 	    $(ECHO) >dtrace_main.c &&							      \
 	    `$(CCCMD)` $(PLDLFLAGS) dtrace_main.c &&					      \
@@ -876,7 +927,7 @@ $(DTRACE_MAIN_O): perldtrace.d perlmain$(OBJ_EXT)
 		;;
     esac
 	$spitshell >>$Makefile <<'!NO!SUBS!'
-$(LIBPERL): $& $(perllib_objs) $(DYNALOADER) $(LIBPERLEXPORT)
+$(LIBPERL): $& $(perllib_dep) $(DYNALOADER) $(LIBPERLEXPORT)
 !NO!SUBS!
 	case "$useshrplib" in
 	true)
@@ -977,7 +1028,7 @@ lib/buildcustomize.pl: $& $(miniperl_objs) write_buildcustomize.pl
 	*)
 		if test "X$hostperl" != X; then
 			$spitshell >>$Makefile <<!GROK!THIS!
-lib/buildcustomize.pl: \$& \$(miniperl_objs) write_buildcustomize.pl
+lib/buildcustomize.pl: \$& \$(miniperl_dep) write_buildcustomize.pl
 	-@rm -f miniperl.xok
 	-@rm \$(MINIPERL_EXE)
 	\$(LNS) \$(HOST_PERL) \$(MINIPERL_EXE)
@@ -986,7 +1037,7 @@ lib/buildcustomize.pl: \$& \$(miniperl_objs) write_buildcustomize.pl
 !GROK!THIS!
 		else
 			$spitshell >>$Makefile <<'!NO!SUBS!'
-lib/buildcustomize.pl: $& $(miniperl_objs) write_buildcustomize.pl
+lib/buildcustomize.pl: $& $(miniperl_dep) write_buildcustomize.pl
 	-@rm -f miniperl.xok
 	$(CC) $(CLDFLAGS) -o $(MINIPERL_EXE) \
 	    $(miniperl_objs) $(libs)
@@ -999,7 +1050,7 @@ lib/buildcustomize.pl: $& $(miniperl_objs) write_buildcustomize.pl
 
 	$spitshell >>$Makefile <<'!NO!SUBS!'
 
-$(PERL_EXE): $& $(perlmain_objs) $(LIBPERL) $(static_ext) ext.libs $(PERLEXPORT) write_buildcustomize.pl
+$(PERL_EXE): $& $(perlmain_dep) $(LIBPERL) $(static_ext) ext.libs $(PERLEXPORT) write_buildcustomize.pl
 	-@rm -f miniperl.xok
 !NO!SUBS!
 
-- 
2.1.4


From 8994a9f2e42dcf6cedf01893695f38ed7942df4d Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Mon, 28 Nov 2016 15:15:18 +1100
Subject: (perl #130108) check if dtrace accepts -xnolibs and use it if
 available

dtrace without -xnolibs fails in a FreeBSD jail, so we need to supply
it on FreeBSD.

Unfortunately systemtap's dtrace emulation doesn't support -xnolibs so
we need to test if it's available.
---
 Configure        | 34 +++++++++++++++++++++++++++++++---
 Makefile.SH      |  7 ++++++-
 Porting/Glossary |  5 +++++
 3 files changed, 42 insertions(+), 4 deletions(-)

diff --git a/Configure b/Configure
index 845fc43..9d91a81 100755
--- a/Configure
+++ b/Configure
@@ -949,6 +949,7 @@ lddlflags=''
 usedl=''
 doublesize=''
 dtraceobject=''
+dtracexnolibs=''
 ebcdic=''
 fflushNULL=''
 fflushall=''
@@ -20966,12 +20967,38 @@ randseedtype=U32
 : object file that uses at least one of the probes defined in the .d file
 case "$usedtrace" in
 $define)
+    case "$dtracexnolibs" in
+    $define|true|[yY]*)
+        dtracexnolibs=$define
+	$dtrace -h -xnolibs -s ../perldtrace.d -o perldtrace.h
+	;;
+    ' '|'')
+        if $dtrace -h -xnolibs -s ../perldtrace.d -o perldtrace.h 2>&1 ; then
+	     dtracexnolibs=$define
+	     echo "Your dtrace accepts -xnolibs"
+	elif $dtrace -h -s ../perldtrace.d -o perldtrace.h 2>&1 ; then
+	     dtracexnolibs=$undef
+	     echo "Your dtrace doesn't accept -xnolibs"
+	else
+             echo "Your dtrace doesn't work at all, try building without dtrace support" >&4
+	     exit 1
+	fi
+	;;
+    *)
+        dtracexnolibs=$undef
+	$dtrace -h -s ../perldtrace.d -o perldtrace.h
+	;;
+    esac
+    case $dtracexnolibs in
+    $define) xnolibs=-xnolibs ;;
+    *) xnolibs= ;;
+    esac
+
     case "$dtraceobject" in
     $define|true|[yY]*)
         dtraceobject=$define
         ;;
     ' '|'')
-        $dtrace -h -s ../perldtrace.d -o perldtrace.h
         $cat >try.c <<EOM
 #include "perldtrace.h"
 int main(void) {
@@ -20981,14 +21008,14 @@ int main(void) {
 EOM
         dtraceobject=$undef
         if $cc -c -o try.o $optimize $ccflags try.c \
-                    && $dtrace -G -s ../perldtrace.d try.o >/dev/null 2>&1; then
+                    && $dtrace -G $xnolibs -s ../perldtrace.d try.o >/dev/null 2>&1; then
                 dtraceobject=$define
             echo "Your dtrace builds an object file"
         fi
-        $rm -f try.c try.o perldtrace.o
         ;;
     *) dtraceobject=$undef ;;
     esac
+    $rm -f try.c try.o perldtrace.o perldtrace.h
 esac
 
 : Determine if this is an EBCDIC system
@@ -24954,6 +24981,7 @@ drand01='$drand01'
 drand48_r_proto='$drand48_r_proto'
 dtrace='$dtrace'
 dtraceobject='$dtraceobject'
+dtracexnolibs='$dtracexnolibs'
 dynamic_ext='$dynamic_ext'
 eagain='$eagain'
 ebcdic='$ebcdic'
diff --git a/Makefile.SH b/Makefile.SH
index 7a4ed42..97f3c5a 100755
--- a/Makefile.SH
+++ b/Makefile.SH
@@ -390,8 +390,13 @@ VG_TEST  ?= ./perl -e 1 2>/dev/null
 	;;
 esac
 
+case "$dtracexnolibs" in
+define) xnolibs=-xnolibs ;;
+*) xnolibs= ;;
+esac
+
 $spitshell >>$Makefile <<!GROK!THIS!
-DTRACE = $dtrace
+DTRACE = $dtrace $xnolibs
 DTRACE_H = $dtrace_h
 
 DTRACE_PERLLIB_O = $dtrace_perllib_o # "dtrace -G" output for perllib_objs
diff --git a/Porting/Glossary b/Porting/Glossary
index 1d2a6ea..a94eaab 100644
--- a/Porting/Glossary
+++ b/Porting/Glossary
@@ -3031,6 +3031,11 @@ dtrace (usedtrace.U):
 dtraceobject (dtraceobject.U):
 	Whether we need to build an object file with the dtrace tool.
 
+dtracexnolibs (dtraceobject.U):
+	Whether dtrace accepts -xnolibs.  If available we call dtrace -h
+	and dtrace -G with -xnolibs to allow dtrace to run in a jail on
+	FreeBSD.
+
 dynamic_ext (Extensions.U):
 	This variable holds a list of XS extension files we want to
 	link dynamically into the package.  It is used by Makefile.
-- 
2.1.4

@p5pRT
Copy link
Author

p5pRT commented Dec 19, 2016

From @swills

Hi,

On 11/27/2016 23​:46, Tony Cook via RT wrote​:

On Mon, 21 Nov 2016 18​:31​:57 -0800, swills wrote​:

The reason perl has no probes is because when dtrace -G is called, it
overwrites the probe sites in the .o files with NOPs. Which means when
the same .o files that were used for miniperl are used for perl, they
lack probes.
Attached the patches we've been testing.

A single file to simplify working with it.

Thanks, I believe these are working now.

Steve

@p5pRT
Copy link
Author

p5pRT commented Dec 23, 2016

From @jkeenan

On Mon, 19 Dec 2016 00​:35​:06 GMT, swills wrote​:

Hi,

On 11/27/2016 23​:46, Tony Cook via RT wrote​:

On Mon, 21 Nov 2016 18​:31​:57 -0800, swills wrote​:

The reason perl has no probes is because when dtrace -G is called, it
overwrites the probe sites in the .o files with NOPs. Which means when
the same .o files that were used for miniperl are used for perl, they
lack probes.
Attached the patches we've been testing.

A single file to simplify working with it.

Thanks, I believe these are working now.

Steve

Tony C, can you describe the next steps for this RT?

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Jan 4, 2017

From @tonycoz

On Fri, 23 Dec 2016 05​:07​:41 -0800, jkeenan wrote​:

Tony C, can you describe the next steps for this RT?

It needs to be rebased and applied to blead, which I've done in 47a83dc, c345ac2, 728ecd1 and f2a60c1.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jan 4, 2017

@tonycoz - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented May 30, 2017

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.26.0, this and 210 other issues have been
resolved.

Perl 5.26.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.26.0

If you find that the problem persists, feel free to reopen this ticket.

@p5pRT
Copy link
Author

p5pRT commented May 30, 2017

@khwilliamson - Status changed from 'pending release' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant