New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Objects destroyed in the wrong order during global destruction #7628
Comments
From francois@fdesar.netHello, all of you. It's been a long time since I last posted, but As it shows both on 5.6.0 and 5.6.1, I prefer to I finally got a minimalistic demonstartion of the First of all, the stubs have been generated using h2xs -cfn Foo Then you can use this file as a patch file against After the usual 'perl Makefile.PL' and 'make', here -------------------------------------------- The first 'Foo::DESTROY' is ok. It destroys the $y variable, - Using 'my' variables doesn't show the problem. Cheers, François Start of patch ----------------------------- Inline Patch--- Foo.xs.orig Wed Apr 11 19:13:58 2001
+++ Foo.xs Wed Apr 11 19:00:34 2001
@@ -2,6 +2,16 @@
#include "perl.h"
#include "XSUB.h"
+typedef int BAR;
+
+static BAR bar;
MODULE = Foo PACKAGE = Foo
+BAR *
+_create(init)
+ char * init
+ CODE:
+ RETVAL=&bar;
+ OUTPUT:
+ RETVAL
Start of patch ----------------------------- Inline Patch--- Foo.pm.orig Wed Apr 11 19:13:39 2001
+++ Foo.pm Wed Apr 11 20:56:16 2001
@@ -34,6 +34,23 @@
# Autoload methods go after =cut, and are processed by the autosplit
+sub new { Start of patch ----------------------------- Inline Patch--- test.pl.orig Wed Apr 11 19:19:18 2001
+++ test.pl Wed Apr 11 18:18:46 2001
@@ -15,3 +15,5 @@
# Insert your test code below, the Test module is use()ed here so read
# its man page ( perldoc Test ) for help writing this test script.
+$x = new Foo();
+$y = new Foo();
End of patch ------------------------------- Perl Info
|
From @timj
I assume you are using T_PTROBJ in your local typemap? Can you check that you really are getting a different object each time. It seems that the destruction order is: Object $x but you need to check this (I haven't got time to actual build this Tim
-- |
From [Unknown Contact. See original ticket]Oooops yes. The typemap is BAR * T_PTROBJ
Yes, I get a different object each time and a diffierent scalar for Foo::new >>Foo=HASH(0x80f59e8)<<>>BARPtr=SCALAR(0x80f5a30)<< I attached a static pointer just to simplify the code, but having I add the 'diff -u' of Foo.pm that generates the output above, just in François Start of patch ------------------------------------ Inline Patch--- Foo.pm.orig Thu Apr 12 11:09:44 2001
+++ Foo.pm Thu Apr 12 11:05:21 2001
@@ -34,6 +34,26 @@
# Autoload methods go after =cut, and are processed by the autosplit
+sub new { |
From @timj
Add a destructor in package BARPtr. MODULE = Foo PACKAGE = BARPtr void or something. This will tell you when perl is destroying the pointer As I said yesterday, it sounds like you are being bitten by the -- |
From [Unknown Contact. See original ticket]First, I want to thank you very much for trying to help me. But I'm not bitten by anything : I just submitted this bug report Actually, my XS module is on its way and doesn't share anything from I just came across this insanity and posted only for the sake of Perlishly yours, François Tim Jenness a écrit :
|
From @SignumCreated by @SignumThis is a bug report for perl from email@christoph-haas.de, ----------------------------------------------------------------- The "Helper.pm" declares a simple class that does nothing really NEW:$VAR1 = { So the CGI object (as a member of the Helper class) is created When I call $helper->DESTROY then the CGI class is still Interesting though: when I declare a lexical (local) variable I could verify that in Perl 5.6.0 this works as expected. Two Perl Info
|
From @mjdominusCreated by @mjdominusThe following program manufactures an "Outer" object that contains an #!/usr/bin/perl use Data::Dumper; sub Outer::new sub Outer::DESTROY $helper = new Outer; sub test Here Outer::DESTROY is expecting that its subobject will still be Hey, where is my subobject? at helper line 13 during global destruction. The bug is highly peturbable. Removing the (unused) "use This bug was reported by Christoph Haas under 5.8.4; his bug report Perl Info
|
From @demerphq
I thought order of destruction in perl in general was undefined? At least "Perl's notion of the right time to call a destructor is not well-defined There is also something on the subject in perlobj.pod but it doesn't say Im wondering why Perl doesn't have a counter whose value is incremented and Yves |
The RT System itself - Status changed from 'new' to 'open' |
From @mjdominus
That is talking about something else that is irrelevant in this case. { my $obj = Class->new; B; The question here is whether Class::DESTROY is called immediately at However, the problem I am reporting is a different situation. Here, a In my example, Perl's premature execution of the inner object while
It maintains a count of the number of references to each object; this |
From @demerphqMJD wrote:
Well I read that as a general observation that you cant count on DESTROY
Only in normal destruction. It _cant_ make the guarantee that you say it my $x=bless {},'Foo'; my $z; sub Foo::Destroy { print "Destroying: @_\n" }
Not in global destruction though. Refcounting can't help there at all. yves |
From @mjdominus
Yes, you're right. But in this case there is no circular structure. As I mentioned to Christoph Haas, our surprise at this situation might
I misunderstood your suggestion. Thanks for explaining it. It still seems to me that Perl's final destruction should have a first |
From @lizmatAt 1:21 PM -0500 12/2/04, Mark Jason Dominus wrote:
I think it would be possible to create a module (pragma) that would Liz |
From @pjcjOn Thu, Dec 02, 2004 at 10:03:59PM +0100, Elizabeth Mattijsen wrote:
Devel::Cover does just this. It's something of an unholy mix of XS and The way it works is that in a CHECK block an XS sub is called which It's probably easier to look at the code. It's on CPAN. But if you do this, our two modules will fight. Which means I'll have -- |
From @ysthOn Fri, Dec 03, 2004 at 01:01:20AM +0100, Paul Johnson <paul@pjcj.net> wrote:
Does the first END sub do something else also? Why not skip the CHECK No dedicated XS needed, at least as of 5.8.1: perl -we'use B qw/end_av/; END { print "baz" } END {print "bar"; push @{end_av()->object_2svref}, sub { print "quux" } } END { print "foo" }' |
From @lizmatAt 1:01 AM +0100 12/3/04, Paul Johnson wrote:
How about if Devel::Cover would supply a method/subroutine to install On the other hand, having Devel::Cover as a prerequisite may be quite Liz |
From @lizmatAt 11:20 PM -0800 12/2/04, Yitzchak Scott-Thoennes wrote:
That would be my plan, yes.
Wow. But is having B as a prerequisite a good idea? Liz |
From @rgsElizabeth Mattijsen wrote:
Well, B is core :) |
From @lizmatAt 11:54 AM +0100 12/3/04, Rafael Garcia-Suarez wrote:
Ah, ok! Well, I've just uploaded this to CPAN: =head1 NAME ogd - ordered global destruction of objects =head1 SYNOPSIS use ogd; ogd->register( @object ); # for objects from XSUBs only =head1 DESCRIPTION This module adds ordered destruction of object in LIFO order during global =head1 CLASS METHODS =head2 register ogd->register( @object ); # only for blessed objects created in XSUBs Not all blessed objects in Perl are necessarily created with "bless": they can =head1 REQUIRED MODULES B (any) =head1 ORDER OF LOADING The C<ogd> pragma installs its own version of the "bless" system function. =head1 TODO Examples should be added. =head1 AUTHOR Elizabeth Mattijsen, <liz@dijkmat.nl>. Please report bugs to <perlbugs@dijkmat.nl>. =head1 ACKNOWLEDGEMENTS Mark Jason Dominus for the initial impetus. Yitzchak Scott-Thoennes for =head1 COPYRIGHT Copyright (c) 2004 Elizabeth Mattijsen <liz@dijkmat.nl>. All rights =head1 SEE ALSO L<Thread::Bless>. =cut For those who cannot wait: it's also available from: http://www.liz.nl/CPAN/ogd-0.01.tar.gz Liz |
From @ysthOn Fri, Dec 03, 2004 at 04:03:42PM +0100, Elizabeth Mattijsen <liz@dijkmat.nl> wrote:
Before 5.8.1, there wasn't a B::object_2svref, so either put in a |
From @lizmatAt 11:06 AM -0800 12/3/04, Yitzchak Scott-Thoennes wrote:
I've put that in the Makefile.PL, to avoid the overhead during Liz |
From @dcollinsnBased on the above suggestions to make a pragma and upload it to cpan, and the report that Liz has done so, I believe this ticket can be closed? I intend to close resolved in a week unless there is some objection. |
@dcollinsn - Status changed from 'open' to 'resolved' |
From @nwellnhofOn Thu, 07 Jul 2016 09:49:52 -0700, dcollinsn@gmail.com wrote:
I object to closing this bug. Perl really should run global destruction in two phases: 1. Decrease the refcount of globals like "our" variables which will run destructors in correct order unless there are circular references or refcount leaks. 2. Only then forcibly destroy the remaining objects. Ideally, it should be possible to tell which of the two phases is run. Some code may want to skip forced destruction in phase 2. There's no need for a pragma that adds additional overhead. Besides, "use ogd" doesn't seem to be very useful. In most cases you don't want LIFO order, but make sure that an object that is still referred to isn't destroyed before the referrer. |
From @LeontOn Thu, Mar 9, 2017 at 9:55 PM, Nick Wellnhofer via RT <
Destructors can and do depend on package variables, this would not solve The proper solution would be to topologically sort the objects and then Leon |
From @nwellnhofOn Fri, 10 Mar 2017 02:16:13 -0800, LeonT wrote:
Right, but it would solve a large class of cases. Also note that Perl seems to do the right thing for "my" variables with package scope. Only for "our" variables, it seems impossible to get a sane destruction order. If you run the program below, $my1 if always destroyed before $my2, regardless of initialization order. But "our" variables are always destroyed in the DESTRUCT phase without taking references between objects into account.
Just because a solution isn't perfect doesn't mean that it shouldn't be implemented. Anyway, I'm only asking to acknowledge that this bug isn't fixed and to reopen it. #!/usr/bin/perl package TestObj; sub new { sub set_ref { sub DESTROY { my $my1 = TestObj->new('my1'); our $our1 = TestObj->new('our1'); |
From @cpansproutOn Fri, 10 Mar 2017 07:38:02 -0800, wellnhofer@aevum.de wrote:
Seems fair enough. I am doing exactly that. But let me note that simply destroying ‘our’ variables before the general reference sweep is not so straightforward. If you start by deleting everything stored by globs, then you end up deleting the destructors themselves. Again, deleting everything from globs except subs would only work for some code, and not other code. What Leon Timmermans said about a topological order would probably be the only solution.
-- Father Chrysostomos |
@cpansprout - Status changed from 'resolved' to 'open' |
From @avarOn Sat, Mar 11, 2017 at 2:41 AM, Father Chrysostomos via RT
On a related note. At Booking.com in our web applications we don't run So uWSGI has an option now to simply skip the perl_destruct() & I can't think of a use-case for this, but is it a good idea to provide I don't know if that's possible or even a mad idea, but since you're
|
From @cpansproutOn Sat, 11 Mar 2017 04:33:24 -0800, avarab@gmail.com wrote:
Well, there is sv_clean_objs. I think it is an API function.
I’m not exactly poking it. I *have* poked it in the past, so I thought I could give some input (which I fear proved no more helpful than what was already said). -- Father Chrysostomos |
From @LeontOn Sat, Mar 11, 2017 at 1:32 PM, Ævar Arnfjörð Bjarmason
Perl already has a concept of destruct level, current valid values are 0 destructs much of the interpreter and all objects, this is the I'm not sure what the value for PL_perl_destruct_level is in your Leon |
From @iabynOn Fri, Mar 10, 2017 at 07:38:02AM -0800, Nick Wellnhofer via RT wrote:
But has just been pointed out, freeing package vars before calling
file-scoped lexical variables are freed (or rather their reference count Package variables have no such concepts (well, subs tend to have ops that -- |
From @nwellnhofOn Mon, 13 Mar 2017 02:31:15 -0700, davem wrote:
I realize that this problem is harder than I initially thought. But I think it could be solved by deferring freeing of package vars during global destruction. So if a refcount drops to zero, DESTROY is invoked, the object is not freed, but refcounts of other objects that are referred to are decreased. In other words, you simulate freeing an object just for refcount side effects. Then, in a second pass, DESTROY is invoked on objects that still have a refcount (because of circular references or refcount leaks). |
From @iabynOn Mon, Mar 13, 2017 at 08:52:09AM -0700, Nick Wellnhofer via RT wrote:
Can you provide a short, self-contained perl program which demonstrates -- |
From @nwellnhofOn Tue, 14 Mar 2017 01:46:17 -0700, davem wrote:
I already did so here: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=32714#txn-1453380 I would expect $our1->DESTROY to be run before $our2->DESTROY because $our1 holds a reference to $our2. But (at least with Perl 5.22.1), the DESTROY method is first invoked on $our2. |
From @demerphqOn 11 March 2017 at 22:16, Leon Timmermans <fawaka@gmail.com> wrote:
Those models dont match the problem Booking is solving. They may match The issue here is that in a multi-forked process model you want to add So consider a scenario where we have a mother process which owns an What happens in a normal scenario is that the refcount page holding If you are using a fork/exec model, the costs of tearing down all the So we don't tear down, we just kill everything. Avar would like a more Examples of this abound btw, for instance consider PL_strtab. Every Yves |
From @avarOn Tue, Mar 14, 2017 at 1:22 PM, demerphq <demerphq@gmail.com> wrote:
Everything you said makes sense & is a good summary of our situation But just to clarify: I'm not particularly looking for a graceful I.e. at Booking's web serving processes we're perfectly happy to just But this is an interesting question for perl embedding in general. Or perhaps out of those 10GB of variables/hashes/data you'd just like Another thing I didn't have time to look into as well is to what |
From @demerphqOn 14 March 2017 at 13:40, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
Ok, fair enough. On the other hand, if someone could offer us a
The SvHeader holds the recount. And sv heads are packed into pages. An Yves -- |
From @iabynOn Tue, Mar 14, 2017 at 05:20:38AM -0700, Nick Wellnhofer via RT wrote:
Ah sorry, I completely failed to spot that when I looked.
Note that the effect demonstrated by your code is not caused by global vars What is actually happening is that at that point during global So its a bit like your code having this in it: END { undef $our1; undef $our2 } except that the order is non-deterministic, so it could equally as well be END { undef $our2; undef $our1 } and note that its also unrelated to package vars: it's *any* RV that can our $x; (where each hash is blessed) might have a destruction equivalent to any of END { undef $x->{l1}{l2}; undef $x } Now having explained all that, I don't really understand your proposal -- |
From @nwellnhofOn Thu, 16 Mar 2017 04:09:36 -0700, davem wrote:
Yes, it seems that these RVs are undeffed, but something with the order of DESTROY calls isn't right.
There must be something else going on. With regard to destructors, the order of undeffing shouldn't matter. When inserted into my test program, both of the END blocks above result in the correct order of DESTROY calls.
I don't care when package vars are freed, I'm mostly concerned with the order in which DESTROY methods are called. I want to avoid DESTROY being invoked on an object that still has references pointing to it, unless it's absolutely necessary because of a reference cycle. Simply setting each package scalar to undef and clearing each package array or hash *should already* result in the correct order, but for some reason it doesn't. (Unless something was fixed since Perl 5.22.1.) My proposal above only takes it one step further and makes sure that package vars don't lose their content too early when other destructors still might use them. As I understood Leon's ("Destructors can and do depend on package variables, this would not solve all non-circular cases.") and your comment ("But has just been pointed out, freeing package vars before calling destructors breaks everything."), this is desirable. But maybe this was a misunderstanding. As it seems, Perl simply undefs package vars in random order, so a destructor can't rely on their contents anyway. Personally, I don't really care about the exact behavior. Another useful feature, especially for XS code, would be a mechanism to tell when DESTROY is invoked forcibly on an object that still has references during global destruction. In this case, calling destructors of an external C library could can lead to crashes. Right now, the best heuristic I could come up with is (PL_dirty && SvREFCNT(sv) > 1) which isn't completely reliable in my particular case. |
From @nwellnhofOn Fri, 17 Mar 2017 14:09:51 -0700, wellnhofer@aevum.de wrote:
OK, now I understand. What's happening in my test program is essentially: END { As Dave already explained, in an early stage of global destruction *every* blessed RV is undeffed in random order: package var, hash value, whatever. So all bets are off. This seems so unpredictable, I wonder why we bother running destructors in the DESTRUCT phase at all. |
From @iabynOn Fri, Mar 17, 2017 at 02:09:52PM -0700, Nick Wellnhofer via RT wrote:
Yes, but I still don't understand exactly what your proposal is. That is -- |
From @demerphqOn 17 March 2017 at 23:27, Nick Wellnhofer via RT
I don't really understand why freeing the stash is any different from Or is this is a case where there are reference cycles, and we are in Sorry if I am asking question already answered... Yves -- |
From @iabynOn Wed, Mar 22, 2017 at 11:46:06AM +0100, demerphq wrote:
No, we're discussing a relatively early phase of destruction. From * call END blocksd; At this point if PL_destruct_level == 0 and we're not a child thread, * free PL_regex_padav; At the point we're discussing, we trigger the calling of DESTROY on all If we instead went for the approach of triggering destructors by freeing I *think* the topic of this thread is how to go about undeffing refs to -- |
Migrated from rt.perl.org#32714 (status was 'open')
Searchable as RT32714$
The text was updated successfully, but these errors were encountered: