Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

interp struct difference, PERL_EXT_RE_DEBUG breaks ABI between re:: and core #14169

Closed
p5pRT opened this issue Oct 19, 2014 · 9 comments
Closed

Comments

@p5pRT
Copy link

p5pRT commented Oct 19, 2014

Migrated from rt.perl.org#123007 (status was 'open')

Searchable as RT123007$

@p5pRT
Copy link
Author

p5pRT commented Oct 19, 2014

From @bulk88

Created by @bulk88

I am running 5.21.5 blead perl. In testing a new perl API version check
to stop mismatches between XS libraries and interp core, I found that
the XS module re​:: has a different size and definition of "struct
interpreter". The struct in core on my build/system is 0xA58 bytes long,
re​:: reports 0xA68 bytes long. This is unacceptable since any struct
members after a certain point in the interpreter struct will get garbage
written to then if re​:: touches those members, since the offsets of some
members are now different between re​::* and core. Before ~5.14, "public"
XS modules used C functions to get the offsets to the interp struct
members for cross major release ABI compatibility. Core modules didn't
do that because PERL_CORE macro turned off the indirection. IDK if re​::*
at that time used or didn't use the interp struct indirection macros.
Doing a diff between the post CPP .i files between re​:: and core shows
"struct perl_memory_debug_header Imemory_debug_header;" exists in re​::*
but is missing in core. intrpvar.h says

------------------------
#ifdef PERL_TRACK_MEMPOOL
/* For use with the memory debugging code in util.c */
PERLVAR(I, memory_debug_header, struct perl_memory_debug_header)
#endif
------------------------

re.xs at the top starts with

------------------------
#if defined(PERL_EXT_RE_DEBUG) && !defined(DEBUGGING)
# define DEBUGGING
#endif
------------------------

and re​::'s Makefile.PL says "my $defines = '-DPERL_EXT_RE_BUILD
-DPERL_EXT_RE_DEBUG -DPERL_EXT';" . I dont understand why this code is
here and why DEBUGGING is on, but this is unacceptable in the current state.

The DEBUGGING was added in

"Re​: [ID 20011117.127] Not OK​: perl v5.7.2 +DEVEL13048 on i686-linux
2.4.15-pre6"
author Andy Dougherty <doughera@​lafayette.edu>
  Tue, 20 Nov 2001 12​:45​:32 +0000 (07​:45 -0500)
committer Jarkko Hietaniemi <jhi@​iki.fi>
  Tue, 20 Nov 2001 16​:59​:54 +0000 (16​:59 +0000)
41b1671
http​://www.nntp.perl.org/group/perl.perl5.porters/2001/11/msg47076.html

To go back further, I found

"make ext/re play nice with DEBUGGING override"
author Craig A. Berry <craigberry@​mac.com>
  Tue, 12 Jun 2001 23​:55​:26 +0000 (18​:55 -0500)
committer Jarkko Hietaniemi <jhi@​iki.fi>
  Wed, 13 Jun 2001 13​:55​:10 +0000 (13​:55 +0000)
fd3cca2
http​://www.nntp.perl.org/group/perl.perl5.porters/2001/06/msg38315.html

And the original code that started using DEBUGGING was

"added suggested patch (via PM), tweaked to implicitly specify -DDEBUGGING"
author Andy Dougherty <doughera@​lafayette.edu>
  Mon, 13 Jul 1998 16​:50​:55 +0000 (12​:50 -0400)
committer Gurusamy Sarathy <gsar@​cpan.org>
  Tue, 14 Jul 1998 04​:23​:28 +0000 (04​:23 +0000)
b9d5759

I can't find the ML posts associated with that patch.

struct interpreter {
-#line 1 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
-
-#line 1 "c:\\perl521\\srcnewb4opt\\lib\\core\\handy.h"
-
-#line 12 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 1 "c:\\perl521\\srcnewb4opt\\intrpvar.h"
+
+#line 1 "c:\\perl521\\srcnewb4opt\\handy.h"
+
+#line 12 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

SV **Istack_sp;
OP *Iop;
@@ -26968,9 +27271,9 @@
char Isawalias;

U8 Ihash_rand_bits_enabled;
-#line 70 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 70 "c:\\perl521\\srcnewb4opt\\intrpvar.h"
UV Ihash_rand_bits;
-#line 72 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 72 "c:\\perl521\\srcnewb4opt\\intrpvar.h"
HV *Istrtab;

PMOP *Icurpm;
@@ -26984,7 +27287,9 @@

U8 Idowarn;

-#line 99 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 97 "c:\\perl521\\srcnewb4opt\\intrpvar.h"
+
+#line 99 "c:\\perl521\\srcnewb4opt\\intrpvar.h"
I8 Iutf8cache;

HV *Idefstash;
@@ -27024,7 +27329,7 @@
SV *Istatname;

struct tms Itimesbuf;
-#line 180 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 180 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

SV *Irs;
GV *Ilast_in_gv;
@@ -27111,7 +27416,7 @@
I32 Istatusvalue;

I32 Istatusvalue_posix;
-#line 347 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 347 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

int Isig_pending;
int *Ipsig_pend;
@@ -27195,7 +27500,7 @@

struct interp_intern Isys_intern;

-#line 484 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 484 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

CV *IDBcv;
int Igeneration;
@@ -27215,7 +27520,7 @@

U32 Ibreakable_sub_gen;

-#line 512 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 512 "c:\\perl521\\srcnewb4opt\\intrpvar.h"
U32 Icop_seqmax;

U32 Ievalseq;
@@ -27252,7 +27557,7 @@
U32 Icollation_ix;
char Icollation_standard;

-#line 553 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 553 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

char Iunsafe;
char Icolorset;
@@ -27270,7 +27575,7 @@
char *Inumeric_name;
SV *Inumeric_radix_sv;

-#line 583 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 583 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

SV *ILatin1;
SV *IUpperLatin1;
@@ -27300,7 +27605,7 @@
U8 Ilast_swash_key[13 - 1];
U8 Ilast_swash_klen;

-#line 619 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 619 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

char Ipad_reset_pending;
char Isrand_called;
@@ -27318,7 +27623,7 @@
struct IPerlDir *IDir;
struct IPerlSock *ISock;
struct IPerlProc *IProc;
-#line 642 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 642 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

PTR_TBL_t *Iptr_table;
AV *Ibeginav_save;
@@ -27332,7 +27637,7 @@
HV **Istashpad;
PADOFFSET Istashpadmax;
PADOFFSET Istashpadix;
-#line 661 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 661 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

HV *Icustom_op_names;
HV *Icustom_op_descs;
@@ -27340,7 +27645,7 @@
PerlIOl *Iperlio;
PerlIO_list_t *Iknown_layers;
PerlIO_list_t *Idef_layerlist;
-#line 674 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 674 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

SV *Iencoding;

@@ -27361,7 +27666,7 @@
share_proc_t Isharehook;
share_proc_t Ilockhook;

-#line 701 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 701 "c:\\perl521\\srcnewb4opt\\intrpvar.h"
share_proc_t Iunlockhook;

thrhook_proc_t Ithreadhook;
@@ -27369,7 +27674,7 @@
destroyable_proc_t Idestroyhook;

despatch_signals_proc_t Isignalhook;
-#line 711 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 711 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

HV *Iisarev;

@@ -27394,68 +27699,67 @@
void **Imy_cxt_list;
int Imy_cxt_size;
-#line 757 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
-
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
- struct perl_memory_debug_header Imemory_debug_header;
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
-#line 762 "c:\\perl521\\srcnewb4opt\\lib\\core\\intrpvar.h"
+#line 757 "c:\\perl521\\srcnewb4opt\\intrpvar.h"

SV *Isv_consts[35];

perl_drand48_t Irandom_state;

-#line 5370 "..\\..\\lib\\CORE\\perl.h"
+#line 5370 "c:\\perl521\\srcnewb4opt\\perl.h"
};
Perl Info

Flags:
    category=library
    severity=high
    module=re

Site configuration information for perl 5.21.2:

Configured by Owner at Tue Sep 23 02:37:36 2014.

Summary of my perl5 (revision 5 version 21 subversion 2) configuration:
  Derived from: 101c6642b743a0f82b7806d5a14d645731f1509c
  Ancestor: 8eaff90c13544b96799fc27a09c05dec3d2706c9
  Platform:
    osname=MSWin32, osvers=5.1, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    useithreads=define, usemultiplicity=define
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -GF -W3 -Od -MD -Zi -DDEBUGGING -DWIN32 
-D_CONSOLE -DNO_STRICT  -DPERL_TEXTMODE_SCRIPTS 
-DPERL_HASH_FUNC_ONE_AT_A_TIME -DPERL_IMPLICIT_CONTEXT 
-DPERL_IMPLICIT_SYS -DUSE_PERLIO -D_USE_32BIT_TIME_T',
    optimize='-Od -MD -Zi -DDEBUGGING',
    cppflags='-DWIN32'
    ccversion='13.10.6030', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', 
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -debug  
-libpath:"c:\perl521\lib\CORE"  -machine:x86'
    libpth="C:\Program Files\Microsoft Visual Studio .NET 2003\VC7\lib"
    libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  
netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib 
odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib
    perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib 
winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib 
oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  
version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl521.lib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug  
-libpath:"c:\perl521\lib\CORE"  -machine:x86'

Locally applied patches:
    uncommitted-changes
    22eb0703e4f1ca0305ca22390770b019a4b0ec4c
    ab36ae0a3ee81e650c6775a6e779fd961d14a2df
    7374111ae13f4be3f77c384a6ed74d26608a6d94
    543c0e688ee3c5a452b9203e04eee8a0f448ecfa
    68a4b64dc9a9688bb38aabd2dd076a93805c5d0a
    2bfbae23de43234000b13d68be2d6c2e17642f52
    d56c627eaf78528a30b6e2e11f11f4a310f6b738
    568f0b08dbad76dcd72664e258a9504bcc2b1246
    101c6642b743a0f82b7806d5a14d645731f1509c


@INC for perl 5.21.2:
    C:/perl521/site/lib
    C:/perl521/lib
    .


Environment for perl 5.21.2:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=C:\perl521\bin;C:\Program Files\Microsoft Visual Studio .NET 
2003\Common7\IDE;C:\Program Files\Microsoft Visual Studio .NET 
2003\VC7\BIN;C:\Program Files\Microsoft Visual Studio .NET 
2003\Common7\Tools;C:\Program Files\Microsoft Visual Studio .NET 
2003\Common7\Tools\bin\prerelease;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\system32\wbem;
    PERL_BADLANG (unset)
    SHELL (unset)


@p5pRT
Copy link
Author

p5pRT commented Oct 20, 2014

From @iabyn

On Sat, Oct 18, 2014 at 05​:35​:10PM -0700, bulk88 wrote​:

I am running 5.21.5 blead perl. In testing a new perl API version check
to stop mismatches between XS libraries and interp core, I found that
the XS module re​:: has a different size and definition of "struct
interpreter".
[snip]
intrpvar.h says

------------------------
#ifdef PERL_TRACK_MEMPOOL
/* For use with the memory debugging code in util.c */
PERLVAR(I, memory_debug_header, struct perl_memory_debug_header)
#endif
------------------------

re.xs at the top starts with

------------------------
#if defined(PERL_EXT_RE_DEBUG) && !defined(DEBUGGING)
# define DEBUGGING
#endif
------------------------

and re​::'s Makefile.PL says "my $defines = '-DPERL_EXT_RE_BUILD
-DPERL_EXT_RE_DEBUG -DPERL_EXT';" . I dont understand why this code is
here and why DEBUGGING is on, but this is unacceptable in the current state.

Adding DEBUGGING is the raison d'etre of the re module; it allows you
to execute variants of the regex engine's normal API functions with
debugging enabled, even on non-debugging builds; so on a non-debugging
perl you can do​:

  perl -Mre=Debug,EXECUTE -e'"abc" =~ /a+/'

Because of this, its always been the case that the interpreter struct must
be the same on debugging and non-debugging builds; see for example this
comment in intrpvar.h​:

  /* name of the scopes we've ENTERed. Only used with -DDEBUGGING, but needs to be
  present always, as -DDEBUGGING must be binary compatible with non. */
  PERLVARI(I, scopestack_name, const char * *, NULL)

Probably the best way to fix is this to add to the two #if's above, i.e.

- #ifdef PERL_TRACK_MEMPOOL
+ #if defined(PERL_TRACK_MEMPOOL) || defined(DEBUGGING)

- #if defined(PERL_EXT_RE_DEBUG) && !defined(DEBUGGING)
+ #if (defined(PERL_EXT_RE_DEBUG) || defined(PERL_TRACK_MEMPOOL)) && !defined(DEBUGGING)

Then add a test to stop this happening again. Probably add a function to
both re.xs and apitest.xs that returns sizeof(struct PerlInterpreter),
then check that they match.

--
Dave's first rule of Opera​:
If something needs saying, say it​: don't warble it.

@p5pRT
Copy link
Author

p5pRT commented Oct 20, 2014

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Oct 21, 2014

From @bulk88

The problem is, in perl's headers, DEBUGGING means you SHALL have PERL_TRACK_MEMPOOL, following is from perl.h

#ifdef PERL_IMPLICIT_CONTEXT
# ifndef MULTIPLICITY
# define MULTIPLICITY
# endif
# define tTHX PerlInterpreter*
# define pTHX tTHX my_perl PERL_UNUSED_DECL
# define aTHX my_perl
# define aTHXa(a) aTHX = (tTHX)a
........................
# define pTHX_12 13
# if defined(DEBUGGING) && !defined(PERL_TRACK_MEMPOOL)
# define PERL_TRACK_MEMPOOL
# endif
#else
# undef PERL_TRACK_MEMPOOL
#endif

That code is from e8dda94
"Reworked PERL_TRACK_MEMPOOL patch"
http​://www.nntp.perl.org/group/perl.perl5.porters/2005/11/msg106694.html and is Jan Dubois's code from 2005.

From reading that thread, it makes me think that the concept of DEBUGGING is broken, or re​::/regexp core code is broken. DEBUGGING XS and non-DEBUGGING core do not have ABI compatibility. A -DDEBUGING to CC will cause XS modules to use Perl_pad_sv (used in dXSTARG), and can't load with a non-DEBUGGING libperl since that doesn't export Perl_pad_sv function as of commit/fb8a983643e8a4ec3e57626fb8754442ebe19278 http​://www.nntp.perl.org/group/perl.perl5.porters/2005/12/msg107232.html . Perl_pad_sv vs PL_curpad[po] comes from Perl 5.0 alpha 2. It was last heavily edited (not sure if changed) in dd2155a "move all pad-related code to its own src file" and origination commit (DEBUGGING pad get vs non DEBUGGING pad get) is alpha 2 7907280 .

IMO regexec.c (and therefore re​::) should not use DEBUGGING macro, but instead the existing PERL_EXT_RE_DEBUG or PERL_IN_XSUB_RE or a new PERL_RE_DEBUG, and PERL_RE_DEBUG becomes defined if DEBUGGING (from core)or if PERL_EXT_RE_DEBUG (from re​::) is defined.

Another idea I dont particularly like is for non-DEBUGGING and DEBUGGING interp struct to have PL_dbgext, which is a pointer to a Newx block which contains a struct that contains the DEBUGGING members. Mem is saved this way by not having alloced DEBUGGING members be alloced on non-DEBUGGING and never used. SEGV on NULL protects against errors.

Other choice is to abandon DEBUGGING and non-DEBUGGING interp equivalence and sizeof(PerlInterpreter) means nothing and length of the struct variable and put all the DEBUGGING members on the end and its uninit memory to use them on non-DEBUGGING.

Then add a test to stop this happening again. Probably add a function
to
both re.xs and apitest.xs that returns sizeof(struct PerlInterpreter),
then check that they match.

I am preparing some code/patch that does that (early version see attachment in http​://markmail.org/thread/knydxxhzmmprbwz3 ). That is how I found out about this.

--
bulk88 ~ bulk88 at hotmail.com

@toddr
Copy link
Member

toddr commented Feb 13, 2020

The markmail link is this:

I am refactoring how XS_APIVERSION_BOOTCHECK and XS_VERSION_BOOTCHECK
and BOOT:/ParseXS work (see attachment for WIP). The ultimate plan is to
merge XS_APIVERSION_BOOTCHECK and XS_VERSION_BOOTCHECK into 1 function
call and push/move less outgoing C args to the now 1 and only function
call. Also some of dXSARGS will be factored out into the new 1 function
call. So far I ran into a test that tests bad behavior but breaks my
code's assumptions.

f9cc56f

"xs_version_bootcheck() must use mortals, as {new,upg}_version() can croak."

Currently if XS_VERSION is CPP undef, the version check doesn't happen.
Currently if XS_VERSION is CPP defined but empty string litteral (""),
the code croaks with "Invalid version format (version required)" which
is from version::'s guts, not ****version_bootcheck()'s code.

XSUB.h

#ifdef XS_VERSION
#  define XS_VERSION_BOOTCHECK					\
      Perl_xs_version_bootcheck(aTHX_ items, ax, STR_WITH_LEN(XS_VERSION))
#else
#  define XS_VERSION_BOOTCHECK
#endif

XSUB-redefined-macros.xs

/* We have to be in a different .xs so that we can do this:  */


#undef XS_VERSION
#define XS_VERSION ""
#undef PERL_API_VERSION_STRING
#define PERL_API_VERSION_STRING "1.0.16"
#include "XSUB.h"


/* This can't be "MODULE = XS::APItest" as then we get duplicate 
bootstraps.  */
MODULE = XS::APItest::XSUB1	PACKAGE = XS::APItest::XSUB


PROTOTYPES: DISABLE


EXPORT_XSUB_SYMBOLS: ENABLE


void
XS_VERSION_empty(...)
     PPCODE:
         XS_VERSION_BOOTCHECK;
         XSRETURN_EMPTY;


void
XS_APIVERSION_invalid(...)
     PPCODE:
         XS_APIVERSION_BOOTCHECK;
         XSRETURN_EMPTY;

My new code assumes that if XS_VERSION is CPP undef, or sizeof(""
XS_VERSION "")-1 == 0, to not do the XS_VERSION api. Can I change the
test and make "" for XS_VERSION stop being a fatal error/test fail?

From d57f1f098221cb075f7cca397fbe224a76220447 Mon Sep 17 00:00:00 2001
From: Daniel Dragan <bul...@hotmail.com>
Date: Thu, 16 Oct 2014 03:49:27 -0400
Subject: [PATCH] WIP handshake api


---
 XSUB.h           |   10 +++++--
 embed.fnc        |    2 +-
 perl.h           |   20 ++++++++++++++
 pod/perldiag.pod |    6 ++++
 proto.h          |    5 +++
 util.c           |   73 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 util.h           |   39 ++++++++++++++++++++++++++++
 7 files changed, 150 insertions(+), 5 deletions(-)


diff --git a/XSUB.h b/XSUB.h
index d0fb253..ea94dfe 100644
--- a/XSUB.h
+++ b/XSUB.h
@@ -324,14 +324,18 @@ Rethrows a previously caught exception.  See L<perlguts/"Exception Handling">.
 #define newXSproto(a,b,c,d)	newXS_flags(a,b,c,d,0)


 #ifdef XS_VERSION
+//#  define XS_VERSION_BOOTCHECK						\
+//    Perl_xs_version_bootcheck(aTHX_ items, ax, STR_WITH_LEN())
 #  define XS_VERSION_BOOTCHECK						\
-    Perl_xs_version_bootcheck(aTHX_ items, ax, STR_WITH_LEN(XS_VERSION))
+    Perl_xs_handshake(HS_KEY(FALSE, "", XS_VERSION), HS_CXT, items, ax, XS_VERSION)
 #else
 #  define XS_VERSION_BOOTCHECK
 #endif


-#define XS_APIVERSION_BOOTCHECK						\
-    Perl_xs_apiversion_bootcheck(aTHX_ ST(0), STR_WITH_LEN("v" PERL_API_VERSION_STRING))
+//#define XS_APIVERSION_BOOTCHECK						\
+//    Perl_xs_apiversion_bootcheck(aTHX_ ST(0), STR_WITH_LEN("v" PERL_API_VERSION_STRING))
+#  define XS_APIVERSION_BOOTCHECK						\
+    Perl_xs_handshake(HS_KEY(FALSE, "v" PERL_API_VERSION_STRING, ""), HS_CXT, items, ax, "v" PERL_API_VERSION_STRING)


 #ifdef NO_XSLOCKS
 #  define dXCPT             dJMPENV; int rEtV = 0
diff --git a/embed.fnc b/embed.fnc
index 9e6ac27..b0bec45 100644
--- a/embed.fnc
+++ b/embed.fnc
@@ -2693,7 +2693,7 @@ Xpo	|void	|xs_version_bootcheck|U32 items|U32 ax|NN const
char *xs_p \
 : XS_APIVERSION_BOOTCHECK
 Xpo	|void	|xs_apiversion_bootcheck|NN SV *module|NN const char *api_p \
 				|STRLEN api_len
-
+Xpon	|I32	|xs_handshake|U32 key|NN void * v_my_perl|...
 #ifndef HAS_STRLCAT
 Apnod	|Size_t	|my_strlcat	|NULLOK char *dst|NULLOK const char *src|Size_t size
 #endif
diff --git a/perl.h b/perl.h
index 95498fc..64fc409 100644
--- a/perl.h
+++ b/perl.h
@@ -5508,6 +5508,26 @@ END_EXTERN_C
 #undef PERLVARI
 #undef PERLVARIC


+#if !defined(MULTIPLICITY)
+/* Set up PERLVAR macros for populating structs */
+#  define PERLVAR(prefix,var,type) type prefix##var;
+/* 'var' is an array of length 'n' */
+#  define PERLVARA(prefix,var,n,type) type prefix##var[n];
+/* initialize 'var' to init' */
+#  define PERLVARI(prefix,var,type,init) type prefix##var;
+/* like PERLVARI, but make 'var' a const */
+#  define PERLVARIC(prefix,var,type,init) type prefix##var;
+
+/* this is never instantiated, is it just used for sizeof(struct
PerlHandShakeInterpreter) */
+struct PerlHandShakeInterpreter {
+#  include "intrpvar.h"
+};
+#  undef PERLVAR
+#  undef PERLVARA
+#  undef PERLVARI
+#  undef PERLVARIC
+#endif
+
 START_EXTERN_C


 /* dummy variables that hold pointers to both runops functions, thus forcing
diff --git a/pod/perldiag.pod b/pod/perldiag.pod
index a163937..40e9b1e 100644
--- a/pod/perldiag.pod
+++ b/pod/perldiag.pod
@@ -536,6 +536,12 @@ copiable.
 (P) When starting a new thread or returning values from a thread, Perl
 encountered an invalid data type.


+=item BOOT:: Invalid handshake key got %X needed %X, binaries are mismatched
+
+(P) A dynamic loading library C<.so> or C<.dll> was being loaded into the
+process that was built against a different build of perl than the said
+library was compiled against.
+
 =item Buffer overflow in prime_env_iter: %s


 (W internal) A warning peculiar to VMS.  While Perl was preparing to
diff --git a/proto.h b/proto.h
index 3cecf16..4796baa 100644
--- a/proto.h
+++ b/proto.h
@@ -5168,6 +5168,11 @@ PERL_CALLCONV void	Perl_xs_apiversion_bootcheck(pTHX_ SV
*module, const char *ap
 #define PERL_ARGS_ASSERT_XS_APIVERSION_BOOTCHECK	\
 	assert(module); assert(api_p)


+PERL_CALLCONV I32	Perl_xs_handshake(U32 key, void * v_my_perl, ...)
+			__attribute__nonnull__(2);
+#define PERL_ARGS_ASSERT_XS_HANDSHAKE	\
+	assert(v_my_perl)
+
 PERL_CALLCONV void	Perl_xs_version_bootcheck(pTHX_ U32 items, U32 ax, const
char *xs_p, STRLEN xs_len)
 			__attribute__nonnull__(pTHX_3);
 #define PERL_ARGS_ASSERT_XS_VERSION_BOOTCHECK	\
diff --git a/util.c b/util.c
index b0d6560..f52350d 100644
--- a/util.c
+++ b/util.c
@@ -5335,6 +5335,76 @@ Perl_my_cxt_init(pTHX_ const char *my_cxt_key, size_t
size)
 #endif /* #ifndef PERL_GLOBAL_STRUCT_PRIVATE */
 #endif /* PERL_IMPLICIT_CONTEXT */


+
+/* v_my_perl is my_perl if PERL_IMPLICIT_CONTEXT and otherwise &PL_stack_sp
+  this will catch threaded perl520.dll calling IO.dll, and IO.dll was linked
with threaded perl510.dll
+  the Perl_get_context from perl510.dll wont match perl520's my_perl in the
same process.
+  On ELF &PL_stack_sp ????? or pass a function pointer that came from
pp_entersub or something,
+  
+  my_perl can not be the first arg since then key will be out of place in a
+  threaded vs non-threaded mixup and analyzing the key number's bitfields won't
reveal the problem
+  it will be a valid key (unthreaded perl) vs gibberish (my_perl arg)
+  (threaded XS mod)*/
+/* Perl_xs_handshake(U32 key, void * v_my_perl, [U32 items, U32 ax], [char *
api_version], [char * xs_version]) */
+I32
+Perl_xs_handshake(const U32 key, void * v_my_perl, ...)
+{
+    va_list args;
+    U32 items, ax;
+#ifdef PERL_IMPLICIT_CONTEXT
+    dTHX;
+#endif
+    PERL_ARGS_ASSERT_XS_HANDSHAKE;
+    DebugBreak();
+    va_start(args, v_my_perl);
+    
+    if((key & HSm_KEY_MATCH) != (HS_KEY(FALSE, "", "") & HSm_KEY_MATCH))
+        Perl_croak_nocontext("BOOT:: Invalid handshake key got %X needed %X,
binaries are mismatched",
+                             (key & HSm_KEY_MATCH), (HS_KEY(FALSE, "", "") &
HSm_KEY_MATCH));
+/* try to catch where a threaded perl interp DLL is loaded into a process by a
+   XS DLL, and the threaded perl interp DLL never initialized its
TLS/PERL_SYS_INIT3 */
+#ifdef PERL_IMPLICIT_CONTEXT
+    if(my_perl != (tTHX)v_my_perl)
+        Perl_croak_nocontext("BOOT:: Invalid handshake key got %X needed %X,
binaries are mismatched",
+                             v_my_perl, my_perl);
+#else
+    if(&PL_stack_sp != (SV ***)v_my_perl)
+        Perl_croak_nocontext("BOOT:: Invalid handshake key got %X needed %X,
binaries are mismatched",
+                             v_my_perl, &PL_stack_sp);
+#endif
+
+
+    if(key & HSf_POPMARK) {
+        ax = POPMARK;
+	{   SV **mark = PL_stack_base + ax++;
+            {   dSP;
+                items = (I32)(SP - MARK);
+            }
+        }
+    } else {
+        items = va_arg(args, U32);
+        ax = va_arg(args, U32);
+    }
+    {
+        U8 apiverlen;
+        assert(HS_GETAPIVERLEN(key) <= UCHAR_MAX);
+        if(apiverlen = (U8)HS_GETAPIVERLEN(key))
+            Perl_xs_apiversion_bootcheck(aTHX_
+                PL_stack_base[ax + 0], va_arg(args, char*), apiverlen);
+    }
+    {
+        U8 xsverlen; /*revise to 32 bits for optimization ? */
+        assert(HS_GETXSVERLEN(key) <= UCHAR_MAX && HS_GETXSVERLEN(key) <=
HS_APIVERLEN_MAX);
+        if(xsverlen = (U8)HS_GETXSVERLEN(key))
+            Perl_xs_version_bootcheck(aTHX_
+                items, ax, va_arg(args, char*), xsverlen);
+    }
+    va_end(args);
+    if(key & HSf_POPMARK) /* so BOOT caller can calc items */
+        ax--;
+    return ax;
+}
+
 void
 Perl_xs_version_bootcheck(pTHX_ U32 items, U32 ax, const char *xs_p,
 			  STRLEN xs_len)
@@ -5392,7 +5462,8 @@ Perl_xs_apiversion_bootcheck(pTHX_ SV *module, const char
*api_p,
     SV *runver;


     PERL_ARGS_ASSERT_XS_APIVERSION_BOOTCHECK;
-
+/* XXX should this be a memcmp/strcmp? both strings are controlled by perls
headers
+  0 padding equivelency rules don't apply */
     /* This might croak  */
     compver = upg_version(compver, 0);
     /* This should never croak */
diff --git a/util.h b/util.h
index 736f978..98435f0 100644
--- a/util.h
+++ b/util.h
@@ -163,6 +163,45 @@ typedef struct {


 #endif /* USE_C_BACKTRACE */


+/* use a packed 32 bit constant to start the handshake */
+#define HSm_APIVERLEN 0x0000003F /* perl version string won't be more than 63
chars */
+#define HS_APIVERLEN_MAX HSm_APIVERLEN
+#define HSm_XSVERLEN 0x0000FF00 /* if 0, not present, dont check, die if over
255*/
+#define HS_XSVERLEN_MAX 0xFF
+#define HSf_POPMARK 0x00000040 /* popmark mode or you must supply ax and items
*/
+#define HSf_IMP_CXT 0x00000080 /* threaded, pTHX_ present */
+#define HSm_INTRPSIZE 0xFFFF0000 /* interp struct size */
+#define HS_GETINTERPSIZE(key) ((key) >> 16)
+#define HS_GETXSVERLEN(key) (((key) & HSm_XSVERLEN) >> 8)
+#define HS_GETAPIVERLEN(key) ((key) & HSm_APIVERLEN)
+/* and maybe HSm_APIVERLEN one day if Perl_xs_apiversion_bootcheck is changed
to a memcmp */
+#define HSm_KEY_MATCH (HSm_INTRPSIZE|HSf_IMP_CXT) /* these bits must always
match between a XS mod and interp */
+/* all args must be const somethings */
+/* U32 return = (U16 interpsize, bool cxt, bool popmark, U6 (SIX!) apiverlen,
U8 xsverlen) */
+#define HS_KEYp(interpsize, cxt, popmark, apiverlen, xsverlen) \
+    (((interpsize)  << 16) \
+    | ((xsverlen) > HS_XSVERLEN_MAX \
+        ? (Perl_croak_nocontext("panic: handshake overflow"), HS_XSVERLEN_MAX)
\
+        : (xsverlen) << 8) \
+    | (cBOOL(cxt) ? HSf_IMP_CXT : 0) \
+    | (cBOOL(popmark) ? HSf_POPMARK : 0) \
+    | ((apiverlen) > HS_APIVERLEN_MAX \
+        ? (Perl_croak_nocontext("panic: handshake overflow"), HS_APIVERLEN_MAX)
\
+        : (apiverlen)))
+
+/* U32 return = (bool popmark, "litteral_string_api_ver",
"litteral_string_xs_ver") */
+#ifdef PERL_IMPLICIT_CONTEXT
+#  define HS_KEY(popmark, apiver, xsver) \
+    HS_KEYp(sizeof(PerlInterpreter), TRUE, popmark, \
+    sizeof("" apiver "")-1, sizeof("" xsver "")-1)
+#  define HS_CXT my_perl
+#else
+#  define HS_KEY(popmark, apiver, xsver) \
+    HS_KEYp(sizeof(struct PerlHandShakeInterpreter), FALSE, popmark, \
+    sizeof("" apiver "")-1, sizeof("" xsver "")-1)
+#  define HS_CXT &PL_stack_sp
+#endif
+
 /*
  * Local variables:
  * c-indentation-style: bsd

-- 
1.7.9.msysgit.0

@toddr
Copy link
Member

toddr commented Feb 13, 2020

However the patch has line wrap on it so it is useless. I'm closing this unless @bulk88 wants to revive it.

@toddr toddr closed this as completed Feb 13, 2020
@khwilliamson
Copy link
Contributor

Note that there is a comment in ext/re/re.xs
/* skip API version checking due to different interp struct size but,
this hack is until #123007 is resolved */
#define dXSBOOTARGSXSAPIVERCHK dXSBOOTARGSNOVERCHK

@toddr
Copy link
Member

toddr commented Feb 25, 2020

That comment was added in db6e00b

@toddr
Copy link
Member

toddr commented Feb 25, 2020

Which was introduced via #14212.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants