Skip Menu |
Report information
Id: 127212
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: ntyni [at] debian.org
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: (no value)



To: perlbug [...] perl.org
Date: Fri, 08 Jan 2016 15:13:01 +0200
From: Niko Tyni <ntyni [...] debian.org>
Subject: ABI incompatibility with -DDEBUGGING vs. without, caused by memory_debug_header
Download (untitled) / with headers
text/plain 10.7k
This is a bug report for perl from Niko Tyni <ntyni@debian.org>, generated with the help of perlbug 1.40 running under perl 5.22.1. ----------------------------------------------------------------- The strict XS binary interface checking in Perl_xs_handshake(), introduced in the 5.21 cycle, has pinpointed a long standing ABI incompatibility in -DDEBUGGING vs. normal builds. According to at least commit f2b88940d81576 such builds are intended to be binary compatible, but we're getting % debugperl -e 'use Cwd' Cwd.c: loadable library and perl binaries are mismatched (got handshake key 0xdb00080, needed 0xdc80080) where 'debugperl' is a 5.22.1 -DDEBUGGING one, configured otherwise identically to the 'normal' perl that the extension is built for. The handshake check takes the interpreter size into account, but the sizes differ by 24 bytes here on x86_64-linux-gnu, because of the 'memory_debug_header' interpreter member that's conditional on PERL_TRACK_MEMPOOL, enabled by -DDEBUGGING. While 'memory_debug_header' is inserted almost at the end, it does move 'sv_consts' and 'random_state' forward, making this a "real" ABI incompatibily, albeit apparently minor enough that it hasn't been noticed earlier. I suppose either PERL_TRACK_MEMPOOL needs to not get enabled by -DDEBUGGING, or the interpreter variables need a similarly sized placeholder for the non-debugging case. FWIW, I intend to do the former for the Debian 5.22 packages, where we would very much like to keep the 'debugperl' variant able to load extensions built for the normal non-debugging perl. (This is originally https://bugs.debian.org/810326) ----------------------------------------------------------------- --- Flags: category=core severity=medium --- Site configuration information for perl 5.22.1: Configured by Debian Project at Sat Dec 26 07:28:43 UTC 2015. Summary of my perl5 (revision 5 version 22 subversion 1) configuration: Platform: osname=linux, osvers=3.16.0, archname=x86_64-linux-gnu-thread-multi uname='linux localhost 3.16.0 #1 smp debian 3.16.0 x86_64 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dcc=x86_64-linux-gnu-gcc -Dcpp=x86_64-linux-gnu-cpp -Dld=x86_64-linux-gnu-gcc -Dccflags=-DDEBIAN -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl,-z,relro -Dlddlflags=-shared -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.22 -Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.22 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.22 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.22.1 -Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.22.1 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.22.1 -des' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='x86_64-linux-gnu-gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2 -g', cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include' ccversion='', gccversion='5.3.1 20151219', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='x86_64-linux-gnu-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=libc-2.21.so, so=so, useshrplib=true, libperl=libperl.so.5.22 gnulibc_version='2.21' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector-strong' Locally applied patches: DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS default for modules installed from CPAN. DEBPKG:debian/db_file_ver - http://bugs.debian.org/340047 Remove overly restrictive DB_File version check. DEBPKG:debian/doc_info - Replace generic man(1) instructions with Debian-specific information. DEBPKG:debian/enc2xs_inc - http://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @INC directories. DEBPKG:debian/errno_ver - http://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes. DEBPKG:debian/libperl_embed_doc - http://bugs.debian.org/186778 Note that libperl-dev package is required for embedded linking DEBPKG:fixes/respect_umask - Respect umask during installation DEBPKG:debian/writable_site_dirs - Set umask approproately for site install directories DEBPKG:debian/extutils_set_libperl_path - EU:MM: set location of libperl.a under /usr/lib DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or perllocal.pod for perl or vendor DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets. DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor. DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy. DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to /etc/perl/Net as /usr may not be writable. DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian DEBPKG:debian/prune_libs - http://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need. DEBPKG:fixes/net_smtp_docs - [rt.cpan.org #36038] http://bugs.debian.org/100195 Document the Net::SMTP 'Port' option DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local DEBPKG:debian/deprecate-with-apt - http://bugs.debian.org/747628 Point users to Debian packages of deprecated core modules DEBPKG:debian/squelch-locale-warnings - http://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts DEBPKG:debian/skip-upstream-git-tests - Skip tests specific to the upstream Git repository DEBPKG:debian/patchlevel - http://bugs.debian.org/567489 List packaged patches for 5.22.1-3 in patchlevel.h DEBPKG:debian/skip-kfreebsd-crash - http://bugs.debian.org/628493 [perl #96272] Skip a crashing test case in t/op/threads.t on GNU/kFreeBSD DEBPKG:fixes/document_makemaker_ccflags - http://bugs.debian.org/628522 [rt.cpan.org #68613] Document that CCFLAGS should include $Config{ccflags} DEBPKG:debian/find_html2text - http://bugs.debian.org/640479 Configure CPAN::Distribution with correct name of html2text DEBPKG:debian/perl5db-x-terminal-emulator.patch - http://bugs.debian.org/668490 Invoke x-terminal-emulator rather than xterm in perl5db.pl DEBPKG:debian/cpan-missing-site-dirs - http://bugs.debian.org/688842 Fix CPAN::FirstTime defaults with nonexisting site dirs if a parent is writable DEBPKG:fixes/memoize_storable_nstore - [rt.cpan.org #77790] http://bugs.debian.org/587650 Memoize::Storable: respect 'nstore' option not respected DEBPKG:debian/regen-skip - Skip a regeneration check in unrelated git repositories DEBPKG:debian/makemaker-pasthru - http://bugs.debian.org/758471 Pass LD settings through to subdirectories DEBPKG:fixes/pod_man_reproducible_date - http://bugs.debian.org/759405 Support POD_MAN_DATE in Pod::Man for the left-hand footer DEBPKG:debian/locale-robustness - http://bugs.debian.org/782068 [perl #124310] Make t/run/locale.t survive missing locales masked by LC_ALL DEBPKG:fixes/podman-utc - http://bugs.debian.org/780259 Make the embedded date from Pod::Man reproducible DEBPKG:fixes/podman-utc-docs - http://bugs.debian.org/780259 Documentation and test suite updates for UTC fix DEBPKG:fixes/podman-empty-date - http://bugs.debian.org/780259 Support an empty POD_MAN_DATE environment variable DEBPKG:fixes/podman-pipe - http://bugs.debian.org/777405 Better errors for man pages from standard input DEBPKG:debian/pod2man-customized - Update porting/customized.dat for pod2man modifications DEBPKG:debian/makemaker-manext - http://bugs.debian.org/247370 Make EU::MakeMaker honour MANnEXT settings in generated manpage headers DEBPKG:debian/makemaker_customized - Update t/porting/customized.dat for files patched in Debian DEBPKG:debian/do-not-record-build-date - [6baa8db] http://bugs.debian.org/774422 [perl #125830] Allow overriding the compile time in "perl -V" output DEBPKG:fixes/podman-source-date-epoch - http://bugs.debian.org/801621 Make Pod::Man honor the SOURCE_DATE_EPOCH environment variable DEBPKG:fixes/podman-source-date-epoch-cleanups - http://bugs.debian.org/801621 Coding style and documentation for SOURCE_EPOCH_DATE DEBPKG:fixes/podman-source-date-epoch-testfix - http://bugs.debian.org/807086 Guard for building with SOURCE_DATE_EPOCH or POD_MAN_DATE set DEBPKG:debian/devel-ppport-reproducibility - http://bugs.debian.org/801523 Sort the list of XS code files when generating RealPPPort.xs DEBPKG:fixes/encode-unicode-bom - http://bugs.debian.org/798727 [rt.cpan.org #107043] Address https://rt.cpan.org/Public/Bug/Display.html?id=107043 DEBPKG:debian/encode-unicode-bom-doc - http://bugs.debian.org/798727 Document Debian backport of Encode::Unicode fix DEBPKG:debian/kfreebsd-softupdates - http://bugs.debian.org/796798 Work around Debian Bug#796798 DEBPKG:fixes/autodie-scope - http://bugs.debian.org/798096 Fix a scoping issue with "no autodie" and the "system" sub --- @INC for perl 5.22.1: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base . --- Environment for perl 5.22.1: HOME=/home/niko LANG=en_US.UTF-8 LANGUAGE (unset) LC_CTYPE=fi_FI.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/niko/bin:/home/niko/bin:/home/niko/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/sbin:/usr/sbin PERL_BADLANG (unset) SHELL=/bin/zsh
Subject: Re: [perl #127212] ABI incompatibility with -DDEBUGGING vs. without, caused by memory_debug_header
To: Perl5 Porters <perl5-porters [...] perl.org>
CC: "bugs-bitbucket [...] rt.perl.org" <bugs-bitbucket [...] rt.perl.org>
Date: Sun, 17 Jan 2016 14:01:40 +0000
From: Aaron Crane <arc [...] cpan.org>
Niko Tyni <perlbug-followup@perl.org> wrote: Show quoted text
> The strict XS binary interface checking in Perl_xs_handshake(), introduced > in the 5.21 cycle, has pinpointed a long standing ABI incompatibility in > -DDEBUGGING vs. normal builds. According to at least commit f2b88940d81576 > such builds are intended to be binary compatible, but we're getting > > % debugperl -e 'use Cwd' > Cwd.c: loadable library and perl binaries are mismatched (got handshake key 0xdb00080, needed 0xdc80080) > > where 'debugperl' is a 5.22.1 -DDEBUGGING one, configured otherwise > identically to the 'normal' perl that the extension is built for.
Thanks for the report. This is fixed as of 5ca5a62834a7fccb204c7fd4b35a11873d1bdd3b. As far as I can tell, we can't fix this for any maint release, as it would introduce a binary incompatibility between non-DEBUGGING releases in the relevant maint. So I think Debian's builds of 5.22.x will need to retain the local patch found in Debian #810326. But that patch shouldn't be necessary at such time as 5.24 goes into Debian. -- Aaron Crane ** http://aaroncrane.co.uk/
From: ilmari [...] ilmari.org (Dagfinn Ilmari Mannsåker)
Date: Sun, 17 Jan 2016 14:36:03 +0000
To: perl5-porters [...] perl.org
Subject: Re: [perl #127212] ABI incompatibility with -DDEBUGGING vs. without, caused by memory_debug_header
Download (untitled) / with headers
text/plain 1.8k
Aaron Crane <arc@cpan.org> writes: Show quoted text
> Niko Tyni <perlbug-followup@perl.org> wrote:
>> The strict XS binary interface checking in Perl_xs_handshake(), introduced >> in the 5.21 cycle, has pinpointed a long standing ABI incompatibility in >> -DDEBUGGING vs. normal builds. According to at least commit f2b88940d81576 >> such builds are intended to be binary compatible, but we're getting >> >> % debugperl -e 'use Cwd' >> Cwd.c: loadable library and perl binaries are mismatched (got handshake key 0xdb00080, needed 0xdc80080) >> >> where 'debugperl' is a 5.22.1 -DDEBUGGING one, configured otherwise >> identically to the 'normal' perl that the extension is built for.
> > Thanks for the report. This is fixed as of > 5ca5a62834a7fccb204c7fd4b35a11873d1bdd3b.
This breaks builds without PERL_IMPLICIT_CONTEXT (i.e. threads) or PERL_DEBUG_READONLY_COW defined, since struct perl_memory_debug_header is not defined otherwise. echo @`sh cflags "optimize='-O2'" perlmini.o` -DPERL_IS_MINIPERL -DPERL_EXTERNAL_GLOB perlmini.c @cc -c -DPERL_CORE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -O2 -Wall -DPERL_IS_MINIPERL -DPERL_EXTERNAL_GLOB perlmini.c In file included from perl.c:33:0: intrpvar.h:787:9: error: field ‘Imemory_debug_header’ has incomplete type PERLVAR(I, memory_debug_header, struct perl_memory_debug_header) ^ perl.h:5633:41: note: in definition of macro ‘PERLVAR’ # define PERLVAR(prefix,var,type) type prefix##var; ^ makefile:320: recipe for target 'perlmini.o' failed make: *** [perlmini.o] Error 1 -- "I use RMS as a guide in the same way that a boat captain would use a lighthouse. It's good to know where it is, but you generally don't want to find yourself in the same spot." - Tollef Fog Heen
Date: Sun, 17 Jan 2016 14:54:16 +0000
From: Aaron Crane <arc [...] cpan.org>
Subject: Re: [perl #127212] ABI incompatibility with -DDEBUGGING vs. without, caused by memory_debug_header
To: Perl5 Porters <perl5-porters [...] perl.org>
Download (untitled) / with headers
text/plain 483b
Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> wrote: Show quoted text
> Aaron Crane <arc@cpan.org> writes:
>> >> Thanks for the report. This is fixed as of >> 5ca5a62834a7fccb204c7fd4b35a11873d1bdd3b.
> > This breaks builds without PERL_IMPLICIT_CONTEXT (i.e. threads) or > PERL_DEBUG_READONLY_COW defined, since struct perl_memory_debug_header > is not defined otherwise.
D'oh. Thanks for the report; fixed in 0419d9784dfda8adf6980bd60e782b144126dd72. -- Aaron Crane ** http://aaroncrane.co.uk/
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 786b
On Sun Jan 17 06:54:53 2016, arc wrote: Show quoted text
> Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> wrote:
> > Aaron Crane <arc@cpan.org> writes:
> >> > >> Thanks for the report. This is fixed as of > >> 5ca5a62834a7fccb204c7fd4b35a11873d1bdd3b.
> > > > This breaks builds without PERL_IMPLICIT_CONTEXT (i.e. threads) or > > PERL_DEBUG_READONLY_COW defined, since struct > > perl_memory_debug_header > > is not defined otherwise.
> > D'oh. Thanks for the report; fixed in > 0419d9784dfda8adf6980bd60e782b144126dd72.
Your fix does not fix ABI compatibility between a DEBUGGING XS module and non-DEBUGGING interp because of Perl_pad_sv. See https://rt.perl.org/Ticket/Display.html?id=123007#txn-1314815 and https://rt.perl.org/Public/Bug/Display.html?id=122032 . -- bulk88 ~ bulk88 at hotmail.com


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org