Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ppport.h breaks croak_xs_usage() on old Perls #16424

Closed
p5pRT opened this issue Feb 17, 2018 · 8 comments
Closed

ppport.h breaks croak_xs_usage() on old Perls #16424

p5pRT opened this issue Feb 17, 2018 · 8 comments

Comments

@p5pRT
Copy link

p5pRT commented Feb 17, 2018

Migrated from rt.perl.org#132876 (status was 'open')

Searchable as RT132876$

@p5pRT
Copy link
Author

p5pRT commented Feb 17, 2018

From zefram@fysh.org

Created by zefram@fysh.org

The current version of ppport.h will define croak_xs_usage() if it
is used on a pre-5.10.1 Perl. This definition takes precedence over
the definition generated by ExtUtils​::ParseXS, and so will be used
by generated XS boilerplate code, not just where it is explicitly
called by user code. However, by default the ppport.h definition
of croak_xs_usage() is broken, referring to an external symbol
"DPPP_my_croak_xs_usage" that is never defined. This will break almost
any XS code that uses such a recent ppport.h, when compiled with an at
all recent ExtUtils​::ParseXS on a pre-5.10.1 Perl.

An individual XS module can avoid this problem by defining the flag macro
"NEED_croak_xs_usage" before #including ppport.h. This causes ppport.h to
provide a working definition of croak_xs_usage() instead of the default
broken one. Observe that I applied this workaround to PathTools in
commit 0840107, anticipating that it
might get a CPAN release soon and that that release might bundle the
current ppport.h. (PathTools has an explicit call to croak_xs_usage(),
not just the ones implicit in the XS; this is a red herring.)

Individual XS modules should not have to define this flag macro if they
are not making explicit calls to croak_xs_usage(). ppport.h should not
by default break the code generated by ExtUtils​::ParseXS.

Perl Info

Flags:
    category=library
    severity=medium
    module=Devel::PPPort

Site configuration information for perl 5.27.8:

Configured by zefram at Sat Jan 20 03:46:58 GMT 2018.

Summary of my perl5 (revision 5 version 27 subversion 8) configuration:
   
  Platform:
    osname=linux
    osvers=3.16.0-4-amd64
    archname=x86_64-linux-thread-multi
    uname='linux barba.rous.org 3.16.0-4-amd64 #1 smp debian 3.16.43-2+deb8u2 (2017-06-26) x86_64 gnulinux '
    config_args='-des -Dprefix=/home/zefram/usr/perl/perl_install/perl-5.27.8-i64-f52 -Duselargefiles -Dusethreads -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dusedevel -Uversiononly -Ui_db'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O2'
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='4.9.2'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.19.so
    so=so
    useshrplib=true
    libperl=libperl.so
    gnulibc_version='2.19'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E -Wl,-rpath,/home/zefram/usr/perl/perl_install/perl-5.27.8-i64-f52/lib/5.27.8/x86_64-linux-thread-multi/CORE'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'



@INC for perl 5.27.8:
    /home/zefram/usr/perl/perl_install/perl-5.27.8-i64-f52/lib/site_perl/5.27.8/x86_64-linux-thread-multi
    /home/zefram/usr/perl/perl_install/perl-5.27.8-i64-f52/lib/site_perl/5.27.8
    /home/zefram/usr/perl/perl_install/perl-5.27.8-i64-f52/lib/5.27.8/x86_64-linux-thread-multi
    /home/zefram/usr/perl/perl_install/perl-5.27.8-i64-f52/lib/5.27.8


Environment for perl 5.27.8:
    HOME=/home/zefram
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/zefram/usr/perl/perl_install/perl-5.27.8-i64-f52/bin:/home/zefram/usr/perl/util:/home/zefram/pub/x86_64-unknown-linux-gnu/bin:/home/zefram/pub/common/bin:/usr/bin:/bin:/usr/local/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2018

From @tonycoz

On Sat, 17 Feb 2018 10​:51​:15 -0800, zefram@​fysh.org wrote​:

The current version of ppport.h will define croak_xs_usage() if it
is used on a pre-5.10.1 Perl. This definition takes precedence over
the definition generated by ExtUtils​::ParseXS, and so will be used
by generated XS boilerplate code, not just where it is explicitly
called by user code. However, by default the ppport.h definition
of croak_xs_usage() is broken, referring to an external symbol
"DPPP_my_croak_xs_usage" that is never defined. This will break
almost
any XS code that uses such a recent ppport.h, when compiled with an at
all recent ExtUtils​::ParseXS on a pre-5.10.1 Perl.

An individual XS module can avoid this problem by defining the flag
macro
"NEED_croak_xs_usage" before #including ppport.h. This causes
ppport.h to
provide a working definition of croak_xs_usage() instead of the
default
broken one. Observe that I applied this workaround to PathTools in
commit 0840107, anticipating that it
might get a CPAN release soon and that that release might bundle the
current ppport.h. (PathTools has an explicit call to
croak_xs_usage(),
not just the ones implicit in the XS; this is a red herring.)

Individual XS modules should not have to define this flag macro if
they
are not making explicit calls to croak_xs_usage(). ppport.h should
not
by default break the code generated by ExtUtils​::ParseXS.

Do the attached fix this for you?

Tony

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2018

From @tonycoz

0001-perl-132876-only-define-croak_xs_usage-s-assert-macr.patch
From 727015c782d1c37af4fc8d21dfd1a00e21b83fbc Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Wed, 21 Feb 2018 11:24:00 +1100
Subject: (perl #132876) only define croak_xs_usage()'s assert macro if cxu
 requested

This avoids an interaction with the code ExtUtils::ParseXS generates to
define a fallback croak_xs_usage().

That code checks whether the assert macro is defined rather than checking
if croak_xs_usage() is defined, and since ppport.h was always defining
the assert macro, the EU::PXS code would fail to define its own
croak_xs_usage() fallback.
---
 dist/Devel-PPPort/parts/inc/mess | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/dist/Devel-PPPort/parts/inc/mess b/dist/Devel-PPPort/parts/inc/mess
index eb2de7b15e..e4723df9ba 100644
--- a/dist/Devel-PPPort/parts/inc/mess
+++ b/dist/Devel-PPPort/parts/inc/mess
@@ -199,12 +199,13 @@ mess_sv(pTHX_ SV *basemsg, bool consume)
 #endif
 #endif
 
+#ifndef croak_xs_usage
+#if { NEED croak_xs_usage }
+
 #ifndef PERL_ARGS_ASSERT_CROAK_XS_USAGE
 #define PERL_ARGS_ASSERT_CROAK_XS_USAGE assert(cv); assert(params)
 #endif
 
-#ifndef croak_xs_usage
-#if { NEED croak_xs_usage }
 void
 croak_xs_usage(const CV *const cv, const char *const params)
 {
-- 
2.11.0

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2018

From @tonycoz

0002-perl-132876-define-API-macros-on-if-the-NEED_-macro-.patch
From 5e5e223929f29119500eee8eb7a35472294882c0 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Wed, 21 Feb 2018 14:50:13 +1100
Subject: (perl #132876) define API macros on if the NEED_ macro is defined

Rather then always defining an API macro such as "croak_xs_usage",
only define it if the NEED_ macro is defined.

This means that code that checks if the API macro is defined won't
get a false positive and try to use an API that's neither available nor
emulated.

If the including code attempts to define its own fallback, as EU::PXS
code does, it also prevents a macro redefinition, most likely with a
different token sequence which the C standard forbids.

This change has the (I think unlikely) risk that a user of ppport.h
might include ppport.h for the macro definition and then define their
own DPPP_some_api() function.  I attempted to check via grep.cpan.me,
but it wasn't working at the time.
---
 dist/Devel-PPPort/PPPort_pm.PL | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/dist/Devel-PPPort/PPPort_pm.PL b/dist/Devel-PPPort/PPPort_pm.PL
index 15cfe63405..d0b2ca5e9f 100644
--- a/dist/Devel-PPPort/PPPort_pm.PL
+++ b/dist/Devel-PPPort/PPPort_pm.PL
@@ -293,9 +293,9 @@ sub expand_pp_expr
          . "extern $proto;\n"
          . "#endif\n"
          . "\n"
-         . "$embed\n"
+         . "#if defined(NEED_$func) || defined(NEED_${func}_GLOBAL)\n"
          . "\n"
-         . "#if defined(NEED_$func) || defined(NEED_${func}_GLOBAL)";
+         . "$embed\n";
   }
 
   die "cannot expand preprocessor expression '$expr'\n";
-- 
2.11.0

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Feb 24, 2018

From zefram@fysh.org

Tony Cook via RT wrote​:

Do the attached fix this for you?

No, that's not a correct fix. By avoiding defining
PERL_ARGS_ASSERT_CROAK_XS_USAGE it avoids suppressing the XS boilerplate
definition of croak_xs_usage(), but ppport.h still defines its own
croak_xs_usage(). The two definitions then conflict​:

Cwd.c​:678​:0​: warning​: "croak_xs_usage" redefined
#define croak_xs_usage S_croak_xs_usage
^
In file included from Cwd.xs​:14​:0​:
ppport.h​:5342​:0​: note​: this is the location of the previous definition
#define croak_xs_usage DPPP_(my_croak_xs_usage)
^

gcc only warns, but non-identical macro redefinition is not legal C,
so a compiler would be entitled to error. To avoid this, ppport.h
must refrain from #defining croak_xs_usage() if it's not doing
the whole job​: croak_xs_usage() definition that actually works, and
PERL_ARGS_ASSERT_CROAK_XS_USAGE also #defined. This would be good advice
for ppport.h in general​: #defining to external symbols may be a useful
option, but it's a terrible default, because by default it's broken.

I take no position on whether ppport.h should be defining croak_xs_usage()
by default. It's just got to be either the complete definition or none
at all.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Feb 27, 2018

From @tonycoz

On Sat, 24 Feb 2018 15​:54​:52 -0800, zefram@​fysh.org wrote​:

Tony Cook via RT wrote​:

Do the attached fix this for you?

No, that's not a correct fix. By avoiding defining
PERL_ARGS_ASSERT_CROAK_XS_USAGE it avoids suppressing the XS boilerplate
definition of croak_xs_usage(), but ppport.h still defines its own
croak_xs_usage(). The two definitions then conflict​:

Cwd.c​:678​:0​: warning​: "croak_xs_usage" redefined
#define croak_xs_usage S_croak_xs_usage
^
In file included from Cwd.xs​:14​:0​:
ppport.h​:5342​:0​: note​: this is the location of the previous definition
#define croak_xs_usage DPPP_(my_croak_xs_usage)
^

gcc only warns, but non-identical macro redefinition is not legal C,

The notes on my second patch say​:

If the including code attempts to define its own fallback, as EU​::PXS code does, it also prevents a macro redefinition, most likely with a different token sequence which the C standard forbids.

Did you apply both patches?

Tony

@p5pRT
Copy link
Author

p5pRT commented Mar 31, 2018

From zefram@fysh.org

Tony Cook via RT wrote​:

Did you apply both patches?

Sorry, I missed that there were two patches, and only applied the
first one. The two taken together do fix this issue.

-zefram

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants