New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SDBM_File attempts to double-free DB pointer when threads are cleaned up #9611
Comments
From chrisb@debian.orgCreated by chrisb@debian.orgThis is a bug report for perl from chrisb@debian.org, ----------------------------------------------------------------- The SDBM_File module doesn't seem to be thread-safe. If a file is tied before a The following script reproduces the problem for me: ================================== use strict; my %dbtest; for (1 .. 2) sub testThread
|
From @jmdhOn Fri Jan 02 08:42:23 2009, crispygoth wrote:
This test script fails for me on Debian 5.12 and passes on Debian 5.14, |
The RT System itself - Status changed from 'new' to 'open' |
From @ppisarDne Út 31.Květen.2011 13:52:08, dom napsal(a):
It happens even when spawning a single thread.
I don't think so. It also crashes with perl 5.18.2. And with blead too. -- Petr |
From @ppisarDne Pá 02.led.2009 08:42:23, crispygoth napsal(a):
The GDBM_File crashes the same way. ODBM_File does not crash, but perl with enabled debugging warns on safefree(db) called at the end of DESTROY(): (in cleanup) panic: free from wrong pool, 2095010!=22068e0 at ../sdbm_threads-bug1104827/test.odbm line 11. The same warning produce SDBM_File and GDBM_File. Only the DB_File does not crash neither warns. But that's because modern BerkeleyDB is somewhat thread-safe and the DB_File.xs. I think about using my_cxt_t to register each opened database, associating a counter there and increasing the counter on each CLONE(). Then each DESTROY() would check the counter and close the database only in last thread. However I feel like reinventing a wheel. There must already exist a tooling for that in the perl. -- Petr |
From @ppisarDne Pá 02.led.2009 08:42:23, crispygoth napsal(a):
At the end, I decided for different fix. I track the interpreter which allocated the object and deallocate it only from the original thread. This prevents from double-free as well as wrong-memory-pool panics, whereas it does not need any reference counting. The fix developed against v5.21.0-269-g98db13b is attached. -- Petr |
From @ppisarperl-5.21.0-Destroy-GDBM-NDBM-ODBM-SDBM-_File-objects-only-from-.patchFrom fa7f2d3b340eb1056e24d083b45c94b380af65ed Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Petr=20P=C3=ADsa=C5=99?= <ppisar@redhat.com>
Date: Fri, 6 Jun 2014 14:31:59 +0200
Subject: [PATCH] Destroy {GDBM,NDBM,ODBM,SDBM}_File objects only from original
thread context
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This patch fixes a crash when destroing a hash tied to a *_File
database after spawning a thread:
use Fcntl;
use SDBM_File;
use threads;
tie(my %dbtest, 'SDBM_File', "test.db", O_RDWR|O_CREAT, 0666);
threads->new(sub {})->join;
This crashed or paniced depending on how perl was configured.
Closes RT#61912.
Signed-off-by: Petr Písař <ppisar@redhat.com>
---
ext/GDBM_File/GDBM_File.xs | 16 ++++++++++------
ext/NDBM_File/NDBM_File.xs | 16 ++++++++++------
ext/ODBM_File/ODBM_File.xs | 18 +++++++++++-------
ext/SDBM_File/SDBM_File.xs | 4 +++-
t/lib/dbmt_common.pl | 35 +++++++++++++++++++++++++++++++++++
5 files changed, 69 insertions(+), 20 deletions(-)
diff --git a/ext/GDBM_File/GDBM_File.xs b/ext/GDBM_File/GDBM_File.xs
index 33e08e2..7160f54 100644
--- a/ext/GDBM_File/GDBM_File.xs
+++ b/ext/GDBM_File/GDBM_File.xs
@@ -13,6 +13,7 @@
#define store_value 3
typedef struct {
+ tTHX owner;
GDBM_FILE dbp ;
SV * filter[4];
int filtering ;
@@ -89,6 +90,7 @@ gdbm_TIEHASH(dbtype, name, read_write, mode)
if ((dbp = gdbm_open(name, GDBM_BLOCKSIZE, read_write, mode,
(FATALFUNC) croak_string))) {
RETVAL = (GDBM_File)safecalloc(1, sizeof(GDBM_File_type)) ;
+ RETVAL->owner = aTHX;
RETVAL->dbp = dbp ;
}
@@ -109,12 +111,14 @@ gdbm_DESTROY(db)
PREINIT:
int i = store_value;
CODE:
- gdbm_close(db);
- do {
- if (db->filter[i])
- SvREFCNT_dec(db->filter[i]);
- } while (i-- > 0);
- safefree(db);
+ if (db && db->owner == aTHX) {
+ gdbm_close(db);
+ do {
+ if (db->filter[i])
+ SvREFCNT_dec(db->filter[i]);
+ } while (i-- > 0);
+ safefree(db);
+ }
#define gdbm_FETCH(db,key) gdbm_fetch(db->dbp,key)
datum_value
diff --git a/ext/NDBM_File/NDBM_File.xs b/ext/NDBM_File/NDBM_File.xs
index e3adf3f..aa3ae6c 100644
--- a/ext/NDBM_File/NDBM_File.xs
+++ b/ext/NDBM_File/NDBM_File.xs
@@ -33,6 +33,7 @@ END_EXTERN_C
#define store_value 3
typedef struct {
+ tTHX owner;
DBM * dbp ;
SV * filter[4];
int filtering ;
@@ -71,6 +72,7 @@ ndbm_TIEHASH(dbtype, filename, flags, mode)
RETVAL = NULL ;
if ((dbp = dbm_open(filename, flags, mode))) {
RETVAL = (NDBM_File)safecalloc(1, sizeof(NDBM_File_type));
+ RETVAL->owner = aTHX;
RETVAL->dbp = dbp ;
}
@@ -84,12 +86,14 @@ ndbm_DESTROY(db)
PREINIT:
int i = store_value;
CODE:
- dbm_close(db->dbp);
- do {
- if (db->filter[i])
- SvREFCNT_dec(db->filter[i]);
- } while (i-- > 0);
- safefree(db);
+ if (db && db->owner == aTHX) {
+ dbm_close(db->dbp);
+ do {
+ if (db->filter[i])
+ SvREFCNT_dec(db->filter[i]);
+ } while (i-- > 0);
+ safefree(db);
+ }
#define ndbm_FETCH(db,key) dbm_fetch(db->dbp,key)
datum_value
diff --git a/ext/ODBM_File/ODBM_File.xs b/ext/ODBM_File/ODBM_File.xs
index d1ece7f..f7e00a0 100644
--- a/ext/ODBM_File/ODBM_File.xs
+++ b/ext/ODBM_File/ODBM_File.xs
@@ -45,6 +45,7 @@ datum nextkey(datum key);
#define store_value 3
typedef struct {
+ tTHX owner;
void * dbp ;
SV * filter[4];
int filtering ;
@@ -112,6 +113,7 @@ odbm_TIEHASH(dbtype, filename, flags, mode)
}
dbp = (void*)(dbminit(filename) >= 0 ? &dbmrefcnt : 0);
RETVAL = (ODBM_File)safecalloc(1, sizeof(ODBM_File_type));
+ RETVAL->owner = aTHX;
RETVAL->dbp = dbp ;
}
OUTPUT:
@@ -124,13 +126,15 @@ DESTROY(db)
dMY_CXT;
int i = store_value;
CODE:
- dbmrefcnt--;
- dbmclose();
- do {
- if (db->filter[i])
- SvREFCNT_dec(db->filter[i]);
- } while (i-- > 0);
- safefree(db);
+ if (db && db->owner == aTHX) {
+ dbmrefcnt--;
+ dbmclose();
+ do {
+ if (db->filter[i])
+ SvREFCNT_dec(db->filter[i]);
+ } while (i-- > 0);
+ safefree(db);
+ }
datum_value
odbm_FETCH(db, key)
diff --git a/ext/SDBM_File/SDBM_File.xs b/ext/SDBM_File/SDBM_File.xs
index 070f074..261031d 100644
--- a/ext/SDBM_File/SDBM_File.xs
+++ b/ext/SDBM_File/SDBM_File.xs
@@ -10,6 +10,7 @@
#define store_value 3
typedef struct {
+ tTHX owner;
DBM * dbp ;
SV * filter[4];
int filtering ;
@@ -49,6 +50,7 @@ sdbm_TIEHASH(dbtype, filename, flags, mode, pagname=NULL)
}
if (dbp) {
RETVAL = (SDBM_File)safecalloc(1, sizeof(SDBM_File_type));
+ RETVAL->owner = aTHX;
RETVAL->dbp = dbp ;
}
@@ -60,7 +62,7 @@ void
sdbm_DESTROY(db)
SDBM_File db
CODE:
- if (db) {
+ if (db && db->owner == aTHX) {
int i = store_value;
sdbm_close(db->dbp);
do {
diff --git a/t/lib/dbmt_common.pl b/t/lib/dbmt_common.pl
index 5d4098c..a0a4d52 100644
--- a/t/lib/dbmt_common.pl
+++ b/t/lib/dbmt_common.pl
@@ -511,5 +511,40 @@ unlink <Op_dbmx*>, $Dfile;
unlink <Op1_dbmx*>;
}
+{
+ # Check DBM back-ends do not destroy objects from then-spawned threads.
+ # RT#61912.
+ SKIP: {
+ my $threads_count = 2;
+ skip 'Threads are disabled', 3 + 2 * $threads_count
+ unless $Config{usethreads};
+ use_ok('threads');
+
+ my %h;
+ unlink <Op1_dbmx*>;
+
+ my $db = tie %h, $DBM_Class, 'Op1_dbmx', $create, 0640;
+ isa_ok($db, $DBM_Class);
+
+ for (1 .. 2) {
+ ok(threads->create(
+ sub {
+ $SIG{'__WARN__'} = sub { fail(shift) }; # debugging perl panics
+ # report it by spurious TAP line
+ 1;
+ }), "Thread $_ created");
+ }
+ for (threads->list) {
+ is($_->join, 1, "A thread exited successfully");
+ }
+
+ pass("Tied object survived exiting threads");
+
+ undef $db;
+ untie %h;
+ unlink <Op1_dbmx*>;
+ }
+}
+
done_testing();
1;
--
1.9.3
|
From @ppisarDne Pá 06.čen.2014 00:19:17, ppisar napsal(a):
I had faulty test. DB_File crashes too. Reported to CPAN <https://rt.cpan.org/Public/Bug/Display.html?id=96357>. |
From @iabynOn Tue, Jun 10, 2014 at 12:16:52AM -0700, Petr Pisar via RT wrote:
I don't think that approach is completely safe. It's possible with threads But even if we fix that, I should imagine that since the underlying *dbm So I think the approach would need to be something along the lines of: Each perl-level object points to a shared struct (allocated using DESTROY frees the db handle (and sets the pointer to NULL) only if the THX When DB methods are called, it croaks unless the THXes match - so only the -- |
From @ppisarOn 2014-06-17, Dave Mitchell <davem@iabyn.com> wrote:
(I believe you talk about thread-childs. Not about fork-childs.) Well, my patch deals only with the destructor to fix the surprise on I believe it behaves correctly even if the parent keeps orphaned
*dbm libraries are not thread-safe as whole, so there is no new issue.
I cannot find documentation for the PerlMemShared_malloc(). Is it
I other words, you ask me to ban any inter-thread access to the tied -- Petr |
Migrated from rt.perl.org#61912 (status was 'open')
Searchable as RT61912$
The text was updated successfully, but these errors were encountered: