Skip Menu |
Report information
Id: 44051
Status: open
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: perlbug [at] daveola.com
Cc: bulk88 <bulk88 [at] hotmail.com>
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: (no value)



Subject: panic: top_env from embedded perl using 'do' with unused XS calls
Date: Thu, 19 Jul 2007 08:14:58 -0700
To: perlbug [...] perl.org
From: David Ljung Madison <daveMail [...] getdave.com>
Download (untitled) / with headers
text/plain 906b
This is a bug report for perl from perlbug@daveola.com, generated with the help of perlbug 1.35 running under perl v5.8.8. ----------------------------------------------------------------- [Please enter your report here] I'm getting a panic: top_env I've managed to reduce the test case to the following: 1. From a C program, create/start a perl interpreter object 2. Load a perl script (using eval_pv "do ...") 3. Setup an XS routine (which, incidentally, never gets used) 4. Call a perl routine in the perl script 5. From that perl routine, execute another perl script (using 'do file') 6. That perl script needs to define four subs, two which must call the XS routine at least four times, and two which must call it at least once - though again note that none of the routines are called. The attempt to 'do file' in step 5 causes the panic. Here's the testcase and Makefile
Download bug.tar.gz
application/octet-stream 1k

Message body not shown because it is not plain text.

RT-Send-CC: perl5-porters [...] perl.org
I can't reproduce this with perl 5.16.0. Is this still an issue?
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.1k
On Fri Jun 22 10:01:49 2012, doy wrote: Show quoted text
> I can't reproduce this with perl 5.16.0. Is this still an issue?
I (Ubuntu) has finally updated to perl 5.14.2 The original issue is gone with 5.14.2, but it reveals new problems. If we actually try to call a routine in the eval/do/require'd script that calls any of the XS functions, then we have a problem. I've made a simpler example. We have embedded perl that 'eval's another script that has a function that calls an XS call. If we eval/do, then when we try to call that function, perl complains about an 'Undefined subroutine' even though the eval didn't have any errors. If we change it to a require, then the require complains when it hits the XS call saying that we've got too many arguments for the call, even though it's the correct number. If we remove the arguments from the XS call, then the require works and the XS call is made, but it will complain about not getting enough arguments. It seems the XS information is lost across the eval. I've attached a new example tar. -- David Ljung Madison http://GetDave.com/ http://MarginalHacks.com/ http://DaveSource.com/
Download bug.tar.gz
application/x-gzip 1k

Message body not shown because it is not plain text.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.1k
On Fri Jun 22 10:01:49 2012, doy wrote: Show quoted text
> I can't reproduce this with perl 5.16.0. Is this still an issue?
I (Ubuntu) has finally updated to perl 5.14.2 The original issue is gone with 5.14.2, but it reveals new problems. If we actually try to call a routine in the eval/do/require'd script that calls any of the XS functions, then we have a problem. I've made a simpler example. We have embedded perl that 'eval's another script that has a function that calls an XS call. If we eval/do, then when we try to call that function, perl complains about an 'Undefined subroutine' even though the eval didn't have any errors. If we change it to a require, then the require complains when it hits the XS call saying that we've got too many arguments for the call, even though it's the correct number. If we remove the arguments from the XS call, then the require works and the XS call is made, but it will complain about not getting enough arguments. It seems the XS information is lost across the eval. I've attached a new example tar. -- David Ljung Madison http://GetDave.com/ http://MarginalHacks.com/ http://DaveSource.com/
Download bug.tar.gz
application/x-gzip 1k

Message body not shown because it is not plain text.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 4.4k
On Sat Nov 03 00:27:16 2012, daveola wrote: Show quoted text
> On Fri Jun 22 10:01:49 2012, doy wrote:
> > I can't reproduce this with perl 5.16.0. Is this still an issue?
> > I (Ubuntu) has finally updated to perl 5.14.2 > > The original issue is gone with 5.14.2, but it reveals new problems. > > If we actually try to call a routine in the eval/do/require'd script > that calls any of the XS functions, then we have a problem. > > I've made a simpler example. We have embedded perl that 'eval's another > script that has a function that calls an XS call. If we eval/do, then > when we try to call that function, perl complains about an 'Undefined > subroutine' even though the eval didn't have any errors. > > If we change it to a require, then the require complains when it hits > the XS call saying that we've got too many arguments for the call, even > though it's the correct number. If we remove the arguments from the XS > call, then the require works and the XS call is made, but it will > complain about not getting enough arguments. > > It seems the XS information is lost across the eval. > > I've attached a new example tar. > > -- > David Ljung Madison > http://GetDave.com/ http://MarginalHacks.com/ > http://DaveSource.com/
I got something else from the code you supplied. On Win32 Perl I got a crash. Show quoted text
___________________________________________________ ntdll.dll!_RtlpWaitForCriticalSection@4() + 0x5b ntdll.dll!_RtlEnterCriticalSection@4() + 0x46
> perl517.dll!PerlIOUnix_refcnt_inc(int fd=0) Line 2374 C
perl517.dll!PerlIOUnix_setfd(interpreter * my_perl=0x008647bc, _PerlIO * * f=0x00913254, int fd=0, int imode=0) Line 2654 + 0x9 C perl517.dll!PerlIOUnix_open(interpreter * my_perl=0x008647bc, _PerlIO_funcs * self=0x2819ee20, PerlIO_list_s * layers=0x00913104, long n=0, const char * mode=0x281921d5, int fd=0, int imode=0, int perm=0, _PerlIO * * f=0x00913254, int narg=0, sv * * args=0x00000000) Line 2735 + 0x15 C perl517.dll!PerlIOBuf_open(interpreter * my_perl=0x008647bc, _PerlIO_funcs * self=0x2819efe0, PerlIO_list_s * layers=0x00913104, long n=1, const char * mode=0x281921d4, int fd=0, int imode=0, int perm=0, _PerlIO * * f=0x00000000, int narg=0, sv * * args=0x00000000) Line 3860 + 0x35 C perl517.dll!PerlIO_openn(interpreter * my_perl=0x008647bc, const char * layers=0x00000000, const char * mode=0x281921d4, int fd=0, int imode=0, int perm=0, _PerlIO * * f=0x00000000, int narg=0, sv * * args=0x00000000) Line 1649 + 0x32 C perl517.dll!PerlIO_fdopen(int fd=0, const char * mode=0x281921d4) Line 4884 + 0x1d C perl517.dll!PerlIO_stdstreams(interpreter * my_perl=0x008647bc) Line 1221 + 0xc C perl517.dll!PerlIO_resolve_layers(interpreter * my_perl=0x008647bc, const char * layers=0x00000000, const char * mode=0x28189294, int narg=1, sv * * args=0x0012fde4) Line 1539 + 0x9 C perl517.dll!PerlIO_openn(interpreter * my_perl=0x008647bc, const char * layers=0x00000000, const char * mode=0x28189294, int fd=-1, int imode=0, int perm=0, _PerlIO * * f=0x00000000, int narg=1, sv * * args=0x0012fde4) Line 1620 + 0x19 C perl517.dll!PerlIO_open(const char * path=0x2818a0b0, const char * mode=0x28189294) Line 4893 + 0x1d C perl517.dll!S_open_script(interpreter * my_perl=0x008647bc, const char * scriptname=0x2818a0b0, char dosearch=0, char * suidscript=0x0012fe53) Line 3705 + 0xe C perl517.dll!S_parse_body(interpreter * my_perl=0x008647bc, char * * env=0x00000000, void (interpreter *)* xsinit=0x00000000) Line 2062 + 0x15 C perl517.dll!perl_parse(interpreter * my_perl=0x008647bc, void (interpreter *)* xsinit=0x00000000, int argc=3, char * * argv=0x0012ff50, char * * env=0x00000000) Line 1621 + 0x11 C main.exe!main() Line 28 + 0x14 C main.exe!mainCRTStartup() Line 398 + 0xe C kernel32.dll!_BaseProcessStart@4() + 0x23
___________________________________________________ PL_perlio_mutex is not set up correctly since DebugInfo should be a pointer not null. DebugInfo is part of a link list debugging thing Windows does in critical section structs.
___________________________________________________ - PL_perlio_mutex {DebugInfo=0x00000000 {Type=??? CreatorBackTraceIndex=??? CriticalSection=??? ...} LockCount=1 RecursionCount=0 ...} _RTL_CRITICAL_SECTION + DebugInfo 0x00000000 {Type=??? CreatorBackTraceIndex=??? CriticalSection=??? ...} _RTL_CRITICAL_SECTION_DEBUG * LockCount 1 long RecursionCount 0 long OwningThread 0x00000000 void * LockSemaphore 0x000007bc void * SpinCount 0 unsigned long
___________________________________________________
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 4.1k
On Wed Nov 07 21:40:01 2012, bulk88 wrote: Show quoted text
> > I've attached a new example tar. > > > > -- > > David Ljung Madison > > http://GetDave.com/ http://MarginalHacks.com/ > > http://DaveSource.com/
> > I got something else from the code you supplied. On Win32 Perl I got a > crash. > ___________________________________________________ > ntdll.dll!_RtlpWaitForCriticalSection@4() + 0x5b > ntdll.dll!_RtlEnterCriticalSection@4() + 0x46
> > perl517.dll!PerlIOUnix_refcnt_inc(int fd=0) Line 2374 C
> perl517.dll!PerlIOUnix_setfd(interpreter * my_perl=0x008647bc, _PerlIO > * * f=0x00913254, int fd=0, int imode=0) Line 2654 + 0x9 C > perl517.dll!PerlIOUnix_open(interpreter * my_perl=0x008647bc, > _PerlIO_funcs * self=0x2819ee20, PerlIO_list_s * layers=0x00913104, long > n=0, const char * mode=0x281921d5, int fd=0, int imode=0, int perm=0, > _PerlIO * * f=0x00913254, int narg=0, sv * * args=0x00000000) Line > 2735 + 0x15 C > perl517.dll!PerlIOBuf_open(interpreter * my_perl=0x008647bc, > _PerlIO_funcs * self=0x2819efe0, PerlIO_list_s * layers=0x00913104, long > n=1, const char * mode=0x281921d4, int fd=0, int imode=0, int perm=0, > _PerlIO * * f=0x00000000, int narg=0, sv * * args=0x00000000) Line > 3860 + 0x35 C > perl517.dll!PerlIO_openn(interpreter * my_perl=0x008647bc, const char > * layers=0x00000000, const char * mode=0x281921d4, int fd=0, int > imode=0, int perm=0, _PerlIO * * f=0x00000000, int narg=0, sv * * > args=0x00000000) Line 1649 + 0x32 C > perl517.dll!PerlIO_fdopen(int fd=0, const char * mode=0x281921d4) > Line 4884 + 0x1d C > perl517.dll!PerlIO_stdstreams(interpreter * my_perl=0x008647bc) Line > 1221 + 0xc C > perl517.dll!PerlIO_resolve_layers(interpreter * my_perl=0x008647bc, > const char * layers=0x00000000, const char * mode=0x28189294, int > narg=1, sv * * args=0x0012fde4) Line 1539 + 0x9 C > perl517.dll!PerlIO_openn(interpreter * my_perl=0x008647bc, const char > * layers=0x00000000, const char * mode=0x28189294, int fd=-1, int > imode=0, int perm=0, _PerlIO * * f=0x00000000, int narg=1, sv * * > args=0x0012fde4) Line 1620 + 0x19 C > perl517.dll!PerlIO_open(const char * path=0x2818a0b0, const char * > mode=0x28189294) Line 4893 + 0x1d C > perl517.dll!S_open_script(interpreter * my_perl=0x008647bc, const char > * scriptname=0x2818a0b0, char dosearch=0, char * suidscript=0x0012fe53) > Line 3705 + 0xe C > perl517.dll!S_parse_body(interpreter * my_perl=0x008647bc, char * * > env=0x00000000, void (interpreter *)* xsinit=0x00000000) Line 2062
+ 0x15 C Show quoted text
> perl517.dll!perl_parse(interpreter * my_perl=0x008647bc, void > (interpreter *)* xsinit=0x00000000, int argc=3, char * * > argv=0x0012ff50, char * * env=0x00000000) Line 1621 + 0x11 C > main.exe!main() Line 28 + 0x14 C > main.exe!mainCRTStartup() Line 398 + 0xe C > kernel32.dll!_BaseProcessStart@4() + 0x23 > ___________________________________________________ > PL_perlio_mutex is not set up correctly since DebugInfo should be a > pointer not null. DebugInfo is part of a link list debugging thing > Windows does in critical section structs. > ___________________________________________________ > - PL_perlio_mutex {DebugInfo=0x00000000 {Type=??? > CreatorBackTraceIndex=??? CriticalSection=??? ...} LockCount=1 > RecursionCount=0 ...} _RTL_CRITICAL_SECTION > + DebugInfo 0x00000000 {Type=??? CreatorBackTraceIndex=??? > CriticalSection=??? ...} _RTL_CRITICAL_SECTION_DEBUG * > LockCount 1 long > RecursionCount 0 long > OwningThread 0x00000000 void * > LockSemaphore 0x000007bc void * > SpinCount 0 unsigned long > ___________________________________________________
The crash on windows is caused by a missing (see perlembed ) "PERL_SYS_INIT3(&argc,&argv,&env);" and "PERL_SYS_TERM();" in main.c . On threaded Win32 Perl, File IO definitely won't work without those 2 calls. I dont know what those 2 do under the hood on Unix, so that might explain why there is more subtle behavior on Unix. Try adding those 2 as described in perlembed and see if your problem goes away. I deleted the files I used to do the above a while ago and forgot about this post so I can't test the suggestion to see if it stops crashing on Win32 Perl. -- bulk88 ~ bulk88 at hotmail.com
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 272b
Show quoted text
> The crash on windows is caused by a missing (see perlembed ) > "PERL_SYS_INIT3(&argc,&argv,&env);" and "PERL_SYS_TERM();" in main.c .
I've added those to the second example (the one showing a missing subroutine) and I still see the problem. A third tar is attached.
Download bug3.tar.z
application/x-compress 1.1k

Message body not shown because it is not plain text.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 406b
Show quoted text
> The crash on windows is caused by a missing (see perlembed ) > "PERL_SYS_INIT3(&argc,&argv,&env);" and "PERL_SYS_TERM();" in main.c
It doesn't seem that the docs really explain what PERL_SYS_INIT3 does. If I don't have the env, can I supply null or perhaps a pointer to an empty array of char pointers? This code is being called from another language, so we don't have a main or env in the C code.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 3.4k
On Mon Mar 11 21:08:22 2013, daveola wrote: Show quoted text
> > The crash on windows is caused by a missing (see perlembed ) > > "PERL_SYS_INIT3(&argc,&argv,&env);" and "PERL_SYS_TERM();" in main.c
> > It doesn't seem that the docs really explain what PERL_SYS_INIT3 does. > If I don't have the env, can I supply null or perhaps a pointer to an > empty array of char pointers? This code is being called from another > language, so we don't have a main or env in the C code.
Tried your new version. No crashes on 5.17.7. Used (win32 platform) Show quoted text
__________________________________________________________________ cl -c -nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -GL -G7 -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -D_USE_32BIT_TIME_T -MD -Zi -DNDEBUG -O1 -GL -G7 "-IC:\perl517\lib\CORE" main.c link -out:main.exe -nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"c:\perl517\lib\CORE" -machine:x86 main.obj C:\perl517\lib\CORE\perl517.lib oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib
__________________________________________________________________ to compile. Running your code unmodified gave
__________________________________________________________________ C:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>main Evaluate builtin.pl Setup the XS routine Call builtinInit do script.pl Eval returns: scriptpl and Undefined subroutine &main::fubar0 called at builtin.pl line 5. C:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>
___________________________________________________________________ In builtin.pl you wrote " my $ret = eval 'script.pl';" That doesn't do what I think you think it does. You probably want a do() or a require(). The error is appropriate since script.pl was never read. Changing the eval to a require so the line reads, " my $ret = require 'script.pl';", gives
____________________________________________________________________ C:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>main Evaluate builtin.pl Setup the XS routine Call builtinInit do script.pl Too many arguments for main::iout at script.pl line 5, near ""Hi")" Compilation failed in require at builtin.pl line 3. C:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>
____________________________________________________________________ "Too many arguments for " is an error from perl prototypes system. Looking in main.c I see " newXSproto("iout",XS_iout,__FILE__,"");", if I change the line to " newXSproto("iout",XS_iout,__FILE__,NULL);" and recompile, I get
____________________________________________________________________ C:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>main Evaluate builtin.pl Setup the XS routine Call builtinInit do script.pl Eval returns: 1 and in fubar0 HiDone with script.pl Done C:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>
_____________________________________________________________________ I dont see a C bug in Perl here. You used "newXSproto" and gave a prototype. If there is a bug, it is the POD for newXSproto at http://perldoc.perl.org/perlapi.html isnt listing a C prototype nor describes the params that func/macro takes (I think that should be fixed). -- bulk88 ~ bulk88 at hotmail.com
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.2k
Show quoted text
> In builtin.pl you wrote " my $ret = eval 'script.pl';"
Now that's humiliating. That's a typo from when I tried to simplify the case down from the much larger case (which was using a 'do'). Show quoted text
> Looking in main.c I see "
newXSproto("iout",XS_iout,__FILE__,"");", if I Show quoted text
> change the line to " newXSproto("iout",XS_iout,__FILE__,NULL);" and
Thanks for the fix on that. Show quoted text
> If there is a bug, it is the POD for newXSproto at > http://perldoc.perl.org/perlapi.html isnt listing a C prototype nor > describes the params that func/macro takes (I think that should be
fixed). I agree with that. There's certainly documentation missing. Should a separate bug be filed, and would it accomplish anything, or is this probably always going to be under-documented? Also - it seems near impossible to find out what is actually happening in PERL_SYS_INIT3 and PERL_SYS_TERM, I've found some Windows code for them, though I'm on UNIX. And the problem is I don't really think I have a way to call PERL_SYS_TERM, since it should only be called on the last ending perl, and I have no way of knowing when that is (the main loop for this is in code that is not under my control, I'm writing a C library which uses Perl. Is it vital that I call PERL_SYS_TERM?
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 434b
I wrote: Show quoted text
> I don't really think I have a way to call PERL_SYS_TERM, since it > should only be called on the last ending perl, and I have no way of > knowing when that is (the main loop for this is in code that is not > under my control, I'm writing a C library which uses Perl.
It seems I am not alone, Jan brings up similar issues with PERL_SYS_TERM: http://www.nntp.perl.org/group/perl.perl5.porters/2007/01/msg120081.html
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.7k
On Tue Mar 12 15:06:14 2013, daveola wrote: Show quoted text
> > If there is a bug, it is the POD for newXSproto at > > http://perldoc.perl.org/perlapi.html isnt listing a C prototype nor > > describes the params that func/macro takes (I think that should be
> fixed). > > I agree with that. There's certainly documentation missing. Should a > separate bug be filed, and would it accomplish anything, or is this > probably always going to be under-documented?
It should not always be under-documented. A git patch is welcome to the POD. Some things to do is look at newXS, see what it says about its params. Look at the source code for newXS_flags (???), see if newXS and newXSproto need more things added to their POD from looking at their source. Like who owns the CV * newXS returns? Do you have to do a refcnt_dec on it? (no). Show quoted text
> > Also - it seems near impossible to find out what is actually happening > in PERL_SYS_INIT3 and PERL_SYS_TERM, I've found some Windows code for > them, though I'm on UNIX. And the problem is I don't really think I > have a way to call PERL_SYS_TERM, since it should only be called on > the last ending perl, and I have no way of knowing when that is (the > main loop for this is in code that is not under my control, I'm writing > a C library which uses Perl. > > Is it vital that I call PERL_SYS_TERM? >
Yes. What it does is different on every OS. Basically it shuts down the per process (not per interp) perl systems. INIT starts up the per process stuff. Call each only once per process regardless of number interps created/destroyed. Make a C static counter. Each interp increases it, each interp decreases it. Mutex it or use atomic ++/__ funcs. When it hits zero do the term. If you go from 0 to 1, do an INIT. -- bulk88 ~ bulk88 at hotmail.com
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 506b
Show quoted text
> It should not always be under-documented. A git patch is welcome to the > POD. Some things to do is look at newXS, see what it says about its > params. Look at the source code for newXS_flags (???), see if newXS and
If I understood the perlembed system I would happily do that patch, but clearly I'm not the person to ask. Show quoted text
> Make a C static counter... If you go from 0 to 1, do an INIT.
Is it okay to do a PERL_SYS_TERM if later other perl interpreters are created (following a new PERL_SYS_INIT3?)


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org