Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DynaLoader doesn't dlclose XS code on interpreter exit #1046

Closed
p5pRT opened this issue Jan 18, 2000 · 18 comments
Closed

DynaLoader doesn't dlclose XS code on interpreter exit #1046

p5pRT opened this issue Jan 18, 2000 · 18 comments

Comments

@p5pRT
Copy link

p5pRT commented Jan 18, 2000

Migrated from rt.perl.org#1997 (status was 'resolved')

Searchable as RT1997$

@p5pRT
Copy link
Author

p5pRT commented Jan 18, 2000

From root@mongo.uk.sun.com

DynaLoader assumes that exiting of the perl interpreter is the same thing as
exit of the entire application. In the case where perl is built as a .so and
embedded inside something else (e.g. Apache/mod_perl) it is certainly not the
same thing.

The failure of DynaLoader to dlclose the XS modules leads to a very insiduous
bug. The XS modules are left loaded in memory, and if the perl libperl.so is
subsequently reloaded via dlopen, it is often loaded into a different address.
The orphaned XS .so files are still in memory, linked to a 'ghost' of the perl
libperl.so. This causes all sorts of insiduous memory and heap corruption
problems.

DynaLoader maintains a list of opened .so files in dl_librefs - on exit it
should dlunload all the objects in this list.

Perl Info


Site configuration information for perl 5.00503:

Configured by root at Tue May 11 08:54:38 BST 1999.

Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration:
  Platform:
    osname=solaris, osvers=2.7, archname=sun4-solaris
    uname='sunos mongo 5.7 generic sun4u sparc sunw,ultra-1 '
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef useperlio=undef d_sfio=undef
  Compiler:
    cc='cc', optimize='-fast', gccversion=
    cppflags='-I/usr/local/include'
    ccflags ='-I/usr/local/include'
    stdchar='char', d_stdstdio=define, usevfork=false
    intsize=4, longsize=4, ptrsize=4, doublesize=8
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    alignbytes=8, usemymalloc=y, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib
    libs=-lsocket -lnsl -ldl -lm -lc -lcrypt
    libc=/lib/libc.so, so=so, useshrplib=true, libperl=libperl.so
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='  -R /usr/local/lib/perl5/5.00503/sun4-solaris/CORE'
    cccdlflags='-KPIC', lddlflags='-G -L/usr/local/lib'

Locally applied patches:
    


@INC for perl 5.00503:
    /usr/local/lib/perl5/5.00503/sun4-solaris
    /usr/local/lib/perl5/5.00503
    /usr/local/lib/perl5/site_perl/5.005/sun4-solaris
    /usr/local/lib/perl5/site_perl/5.005
    .


Environment for perl 5.00503:
    HOME=/home1/rootal
    LANG=C
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home1/rootal/bin:/usr/local/bin:/bin:/usr/bin:/usr/sbin:/opt/gcc/bin:/opt/SUNWspro/bin:/usr/ccs/bin:/usr/ucb:/usr/dt/bin:/usr/openwin/bin:/opt/SUNWisdn/bin:/opt/SUNWisdn/tools:.
    PERL_BADLANG (unset)
    SHELL=/bin/ksh

@p5pRT
Copy link
Author

p5pRT commented Jan 18, 2000

From [Unknown Contact. See original ticket]

Super-User writes​:

The failure of DynaLoader to dlclose the XS modules leads to a very insiduous
bug. The XS modules are left loaded in memory, and if the perl libperl.so is
subsequently reloaded via dlopen, it is often loaded into a different address.
The orphaned XS .so files are still in memory, linked to a 'ghost' of the perl
libperl.so. This causes all sorts of insiduous memory and heap corruption
problems.

Looks like "do not do this" category. Though may be it is safe to
dlclose() things at the end of "global destruction"...

DynaLoader maintains a list of opened .so files in dl_librefs - on exit it
should dlunload all the objects in this list.

Is not this list dead at the end of global destruction? 1/2 ;-)

Ilya

@p5pRT
Copy link
Author

p5pRT commented Jan 19, 2000

From @AlanBurlison

Ilya Zakharevich wrote​:

Super-User writes​:

The failure of DynaLoader to dlclose the XS modules leads to a very insiduous
bug. The XS modules are left loaded in memory, and if the perl libperl.so is
subsequently reloaded via dlopen, it is often loaded into a different address.
The orphaned XS .so files are still in memory, linked to a 'ghost' of the perl
libperl.so. This causes all sorts of insiduous memory and heap corruption
problems.

Looks like "do not do this" category. Though may be it is safe to
dlclose() things at the end of "global destruction"...

Right. Tell that to the Apache developers.
Actually, I think that if we claim that perl can be embedded inside
another program, it is reasonable to expect it to clean up after itself
when asked to exit. This includes removing any .so files it has
dlopened.

DynaLoader maintains a list of opened .so files in dl_librefs - on exit it
should dlunload all the objects in this list.

Is not this list dead at the end of global destruction? 1/2 ;-)

Couldn't it be done in an END block inside DynaLoader?

Alan Burlison

@p5pRT
Copy link
Author

p5pRT commented Jan 19, 2000

From @AlanBurlison

Alan Burlison wrote​:

DynaLoader maintains a list of opened .so files in dl_librefs - on exit it
should dlunload all the objects in this list.

Is not this list dead at the end of global destruction? 1/2 ;-)

Couldn't it be done in an END block inside DynaLoader?

I've a patch to do this for 5.005_63 (attatched) for at least those
platforms that support dlopen/dlclose - on others it is a no-op (I
hope).

Unfortunately I can't actually get mod_perl + a load of other stuff to
build on 5.005_63, so I can't see if it cures the mod_perl problem.

Could a DynaLoader guru have a quick look at the patch and let me know
if it is complete garbage?

I have backported the patch to 5.005_03, and it appears that mod_perl
doesn't actually destroy the perl interpreter correctly when Apache does
a restart.

Sigh. What a mess.

Alan Burlison

@p5pRT
Copy link
Author

p5pRT commented Jan 19, 2000

From @AlanBurlison

DynaLoader.patch
*** ext/DynaLoader/DynaLoader_pm.PL.orig	Wed Jan 19 13:27:35 2000
--- ext/DynaLoader/DynaLoader_pm.PL	Wed Jan 19 16:02:57 2000
***************
*** 251,256 ****
--- 251,264 ----
  #}
  
  
+ sub END {
+     return if (! defined(&dl_unload_file));
+     while (my $libref = pop(@dl_librefs)) {
+         dl_unload_file($libref);
+     }
+ }
+ 
+ 
  # Let autosplit and the autoloader deal with these functions:
  __END__
  
***************
*** 466,471 ****
--- 474,480 ----
    @symbols = dl_undef_symbols()                        C
    dl_install_xsub($name, $symref [, $filename])        C
    $message = dl_error                                  C
+   dl_unload_file($libref)                              C
  
  =over 4
  
***************
*** 709,714 ****
--- 718,735 ----
  $filename is not defined then "DynaLoader" will be used.
  
  
+ =item dl_unload_file()
+ 
+ Syntax:
+ 
+     dl_unload_file($libref)
+ 
+ Unloads the library referred to by $libref.  Not normally called explicitly,
+ but by an END block in DynaLoader.pm that unloads all the modules whos handles
+ are stored in @dl_librefs.  This function may be absent on platforms that don't
+ require this functionality.
+ 
+ 
  =item bootstrap()
  
  Syntax:
*** ext/DynaLoader/XSLoader_pm.PL.orig	Wed Jan 19 13:56:48 2000
--- ext/DynaLoader/XSLoader_pm.PL	Wed Jan 19 15:48:54 2000
***************
*** 126,131 ****
--- 126,140 ----
      goto &DynaLoader::bootstrap_inherit;
  }
  
+ 
+ sub END {
+     package DynaLoader;
+     return if (! defined(&dl_unload_file));
+     while (my $libref = pop(@dl_librefs)) {
+         dl_unload_file($libref);
+     }
+ }
+ 
  __END__
  
  =head1 NAME
*** ext/DynaLoader/dl_dlopen.xs.orig	Wed Jan 19 13:27:47 2000
--- ext/DynaLoader/dl_dlopen.xs	Wed Jan 19 13:33:38 2000
***************
*** 169,174 ****
--- 169,185 ----
  	sv_setiv( ST(0), PTR2IV(RETVAL));
  
  
+ int
+ dl_unload_file(libhandle)
+     void *	libhandle
+     CODE:
+     DLDEBUG(2, PerlIO_printf(Perl_debug_log, "dl_unload_file(handle=%lx)\n",
+ 			     (unsigned long) libhandle));
+     RETVAL = dlclose(libhandle);
+     if (RETVAL != 0)
+ 	SaveError(aTHX_ "%s",dlerror()) ;
+ 
+ 
  void *
  dl_find_symbol(libhandle, symbolname)
      void *	libhandle

@p5pRT
Copy link
Author

p5pRT commented Jan 19, 2000

From [Unknown Contact. See original ticket]

Alan Burlison writes​:

Couldn't it be done in an END block inside DynaLoader?

Too early. The only safe place (if any) should be
after-the-end-of-global-destruction (assuming the list is kept in a C
structure, thus is indestructible).

Ilya

@p5pRT
Copy link
Author

p5pRT commented Jan 20, 2000

From @timbunce

On Wed, Jan 19, 2000 at 02​:38​:18PM -0500, Ilya Zakharevich wrote​:

Alan Burlison writes​:

Couldn't it be done in an END block inside DynaLoader?

Too early. The only safe place (if any) should be
after-the-end-of-global-destruction (assuming the list is kept in a C
structure, thus is indestructible).

Good point, but I'm not so sure. Objects have been cleaned, via
sv_clean_objs(), before the END blocks are called. (Umm, I don't see
any mention in the docs that objects will have been destroyed before
END blocks are called.)

If destruct_level==0 then that's just about the last thing
perl_destruct() does before calling return.

If destruct_level > 0 then there's lots of SvREFCNT_dec'ing going on
and sv_clean_all() etc but I don't think they'd trigger execution of
any XS code.

Also the dlclose's should be skipped unless perl_destruct_level > 0.
Otherwise it'll slowdown perl's overall run time for everyone.

Other random observation, I see that the PERL_DESTRUCT_LEVEL env var
value is checked in perl_destruct() but not propogated to the
PL_perl_destruct_level global.

Tim.

@p5pRT
Copy link
Author

p5pRT commented Jan 20, 2000

From [Unknown Contact. See original ticket]

On Thu, Jan 20, 2000 at 11​:16​:00AM +0000, Tim Bunce wrote​:

Couldn't it be done in an END block inside DynaLoader?

Too early. The only safe place (if any) should be
after-the-end-of-global-destruction (assuming the list is kept in a C
structure, thus is indestructible).

Good point, but I'm not so sure. Objects have been cleaned, via
sv_clean_objs(), before the END blocks are called. (Umm, I don't see
any mention in the docs that objects will have been destroyed before
END blocks are called.)

Do not think so​:

perl -wle 'sub a​::new {bless [],"a"}; my $a="a"->new;END{print "<$a>"}'
<a=ARRAY(0xe2064)>

If destruct_level==0 then that's just about the last thing
perl_destruct() does before calling return.

destruct_level==0 has nothing to do with embedding anyway...

Ilya

@p5pRT
Copy link
Author

p5pRT commented Jan 20, 2000

From @timbunce

On Thu, Jan 20, 2000 at 01​:14​:42PM -0500, Ilya Zakharevich wrote​:

On Thu, Jan 20, 2000 at 11​:16​:00AM +0000, Tim Bunce wrote​:

Couldn't it be done in an END block inside DynaLoader?

Too early. The only safe place (if any) should be
after-the-end-of-global-destruction (assuming the list is kept in a C
structure, thus is indestructible).

Good point, but I'm not so sure. Objects have been cleaned, via
sv_clean_objs(), before the END blocks are called. (Umm, I don't see
any mention in the docs that objects will have been destroyed before
END blocks are called.)

Do not think so​:

perl -wle 'sub a​::new {bless [],"a"}; my $a="a"->new;END{print "<$a>"}'
<a=ARRAY(0xe2064)>

Ah, I see I confused PL_exitlist with PL_endav.

Looks like DynaLoader​::bootstrap just needs to call perl_atexit(...)
to register a callback to do the unloading after gobal destruction.
(For versions of perl that have perl_atexit()).

If destruct_level==0 then that's just about the last thing
perl_destruct() does before calling return.

destruct_level==0 has nothing to do with embedding anyway...

I didn't say it did (though I'd disagree with "nothing to do").

Tim.

@p5pRT
Copy link
Author

p5pRT commented Jan 20, 2000

From [Unknown Contact. See original ticket]

On Thu, Jan 20, 2000 at 11​:18​:42PM +0000, Tim Bunce wrote​:

On Thu, Jan 20, 2000 at 01​:14​:42PM -0500, Ilya Zakharevich wrote​:

Do not think so​:

perl -wle 'sub a​::new {bless [],"a"}; my $a="a"->new;END{print "<$a>"}'
<a=ARRAY(0xe2064)>

Ah, I see I confused PL_exitlist with PL_endav.

Looks like DynaLoader​::bootstrap just needs to call perl_atexit(...)
to register a callback to do the unloading after gobal destruction.
(For versions of perl that have perl_atexit()).

In which case, I propose this​: mod_perl can save a copy of
@​DynaLoader​::dl_librefs, and then call perl_destruct. After that it
can attempt to dlclose() them; the dlclose() calls will fail (silently)
if the modules are already unloaded, and do the right thing if they
weren't.

It's a bit of a messy hack, but how's that sound for coping with
fixed/unfixed versions of DynaLoader?

Dan

/--------------------------------\ /--------------------------------\
| Daniel Jacobowitz |__| SCS Class of 2002 |
| Debian GNU/Linux Developer __ Carnegie Mellon University |
| dan@​debian.org | | dmj+@​andrew.cmu.edu |
\--------------------------------/ \--------------------------------/

@p5pRT
Copy link
Author

p5pRT commented Jan 21, 2000

From @timbunce

On Thu, Jan 20, 2000 at 08​:40​:23PM -0500, Daniel Jacobowitz wrote​:

On Thu, Jan 20, 2000 at 11​:18​:42PM +0000, Tim Bunce wrote​:

On Thu, Jan 20, 2000 at 01​:14​:42PM -0500, Ilya Zakharevich wrote​:

Do not think so​:

perl -wle 'sub a​::new {bless [],"a"}; my $a="a"->new;END{print "<$a>"}'
<a=ARRAY(0xe2064)>

Ah, I see I confused PL_exitlist with PL_endav.

Looks like DynaLoader​::bootstrap just needs to call perl_atexit(...)
to register a callback to do the unloading after gobal destruction.
(For versions of perl that have perl_atexit()).

In which case, I propose this​: mod_perl can save a copy of
@​DynaLoader​::dl_librefs, and then call perl_destruct. After that it
can attempt to dlclose() them; the dlclose() calls will fail (silently)
if the modules are already unloaded, and do the right thing if they
weren't.

That's a presumption, but probably true on most platforms. For some
you might get a core dump. You could configure your way around that.

Tim.

@p5pRT
Copy link
Author

p5pRT commented Jan 22, 2000

From [Unknown Contact. See original ticket]

Tim Bunce <Tim.Bunce@​ig.co.uk> writes​:

Good point, but I'm not so sure. Objects have been cleaned, via
sv_clean_objs(), before the END blocks are called.

I don't think so. But if true that might explain why I get
uses of undefined values in my END blocks ...

(Umm, I don't see
any mention in the docs that objects will have been destroyed before
END blocks are called.)

--
Nick Ing-Simmons

@p5pRT
Copy link
Author

p5pRT commented Jan 22, 2000

From @gsar

On Sat, 22 Jan 2000 16​:56​:29 GMT, Nick Ing-Simmons wrote​:

Tim Bunce <Tim.Bunce@​ig.co.uk> writes​:

Good point, but I'm not so sure. Objects have been cleaned, via
sv_clean_objs(), before the END blocks are called.

I don't think so. But if true that might explain why I get
uses of undefined values in my END blocks ...

END blocks are called just before perl_run() returns. Global object
destruction happens in perl_destruct().

Scheduling dlclose()s using call_atexit() ought to work. I'm going
to let the interested parties test that and send me a patch.

Sarathy
gsar@​ActiveState.com

@p5pRT
Copy link
Author

p5pRT commented Jan 26, 2000

From @gsar

On Wed, 26 Jan 2000 16​:06​:29 GMT, Alan Burlison wrote​:

Gurusamy Sarathy wrote​:

Scheduling dlclose()s using call_atexit() ought to work. I'm going
to let the interested parties test that and send me a patch.

Is anyone up for this? I'm in the US this week, so I'm not able to
contribute much (plus I'd probably just break something ;-) Gurusamay,
do you have a timescale for when you need this?

Some time before Feb 29 would be good. Today would be better. :-)

Sarathy
gsar@​ActiveState.com

@p5pRT
Copy link
Author

p5pRT commented Jan 26, 2000

From @AlanBurlison

Gurusamy Sarathy wrote​:

Scheduling dlclose()s using call_atexit() ought to work. I'm going
to let the interested parties test that and send me a patch.

Is anyone up for this? I'm in the US this week, so I'm not able to
contribute much (plus I'd probably just break something ;-) Gurusamay,
do you have a timescale for when you need this?

Thanks,

Alan Burlison

@p5pRT
Copy link
Author

p5pRT commented Jul 13, 2005

From @schwern

[root@​mongo.uk.sun.com - Tue Jan 18 07​:49​:50 2000]​:

DynaLoader assumes that exiting of the perl interpreter is the same
thing as
exit of the entire application. In the case where perl is built as a
.so and
embedded inside something else (e.g. Apache/mod_perl) it is certainly
not the
same thing.

The failure of DynaLoader to dlclose the XS modules leads to a very
insiduous
bug. The XS modules are left loaded in memory, and if the perl
libperl.so is
subsequently reloaded via dlopen, it is often loaded into a different
address.
The orphaned XS .so files are still in memory, linked to a 'ghost' of
the perl
libperl.so. This causes all sorts of insiduous memory and heap
corruption
problems.

DynaLoader maintains a list of opened .so files in dl_librefs - on
exit it
should dlunload all the objects in this list.

Looking at DynaLoader and skimming the discussion of this bug it seems
like it has been applied and fixed. Could someone more familiar with
DynaLoader please confirm?

@p5pRT
Copy link
Author

p5pRT commented Apr 19, 2006

From @smpeters

[schwern - Tue Jul 12 19​:40​:34 2005]​:

[root@​mongo.uk.sun.com - Tue Jan 18 07​:49​:50 2000]​:

DynaLoader assumes that exiting of the perl interpreter is the same
thing as
exit of the entire application. In the case where perl is built as a
.so and
embedded inside something else (e.g. Apache/mod_perl) it is certainly
not the
same thing.

The failure of DynaLoader to dlclose the XS modules leads to a very
insiduous
bug. The XS modules are left loaded in memory, and if the perl
libperl.so is
subsequently reloaded via dlopen, it is often loaded into a different
address.
The orphaned XS .so files are still in memory, linked to a 'ghost' of
the perl
libperl.so. This causes all sorts of insiduous memory and heap
corruption
problems.

DynaLoader maintains a list of opened .so files in dl_librefs - on
exit it
should dlunload all the objects in this list.

Looking at DynaLoader and skimming the discussion of this bug it seems
like it has been applied and fixed. Could someone more familiar with
DynaLoader please confirm?

It looks as if the the following change fixed this...

Change 5381 by gsar@​auger on 2000/03/01 00​:46​:44

  unload extension shared objects when exiting, implemented
  only for dl_dlopen.xs (from Alan Burlison)

Affected files ...

... //depot/perl/ext/DynaLoader/DynaLoader_pm.PL#10 edit
... //depot/perl/ext/DynaLoader/Makefile.PL#12 edit
... //depot/perl/ext/DynaLoader/dl_dlopen.xs#16 edit
... //depot/perl/ext/DynaLoader/dlutils.c#14 edit

@p5pRT
Copy link
Author

p5pRT commented Apr 19, 2006

@smpeters - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant