Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault in Perl_csighandler when receiving SIGCHLD during thread creation #9570

Closed
p5pRT opened this issue Nov 21, 2008 · 9 comments

Comments

@p5pRT
Copy link

p5pRT commented Nov 21, 2008

Migrated from rt.perl.org#60724 (status was 'resolved')

Searchable as RT60724$

@p5pRT
Copy link
Author

p5pRT commented Nov 21, 2008

From mail@pascalhofmann.de

I'have a program which uses threads and has a custom signal handler for
SIGCHLD. If a SIGCHLD is received during thread creation perl produces a
segfault. Here is the stacktrace from two gdb-sessions​:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7ee08c0 (LWP 1328)]
0x0809ad07 in Perl_csighandler ()
(gdb) bt
#0 0x0809ad07 in Perl_csighandler ()
#1 <signal handler called>
#2 0xb7f8538c in ?? () from /lib/tls/i686/cmov/libc.so.6
#3 0xb7f86865 in malloc () from /lib/tls/i686/cmov/libc.so.6
#4 0x0809498c in Perl_safesysmalloc ()
#5 0x080a8373 in Perl_av_extend ()
#6 0x080a8e13 in Perl_av_store ()
#7 0x080a9523 in Perl_av_push ()
#8 0x080cc700 in perl_clone ()
#9 0xb7dbafc4 in XS_threads_create () from
/usr/lib/perl/5.10/auto/threads/threads.so
#10 0x080b3f02 in Perl_pp_entersub ()

#11 0x080b22e9 in Perl_runops_standard ()

#12 0x080b0750 in perl_run ()

#13 0x08063ebd in main ()


Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7df18c0 (LWP 16487)]
0x0809ad07 in Perl_csighandler ()
(gdb) bt
#0 0x0809ad07 in Perl_csighandler ()
#1 <signal handler called>
#2 0xb7e9cb56 in memcpy () from /lib/tls/i686/cmov/libc.so.6
#3 0x09804638 in ?? ()
#4 0x080bc91f in Perl_sv_dup ()
#5 0x080bd13d in Perl_sv_dup ()
#6 0x080bd13d in Perl_sv_dup ()
#7 0x080bd196 in Perl_sv_dup ()
#8 0x080bdfa1 in Perl_gp_dup ()
#9 0x080bd2c6 in Perl_sv_dup ()
#10 0x080a73da in Perl_he_dup ()
#11 0x080bcd54 in Perl_sv_dup ()
#12 0x080bc970 in Perl_sv_dup ()
#13 0x080bda18 in Perl_rvpv_dup ()
#14 0x080bcfb7 in Perl_sv_dup ()
#15 0x080bdeaa in Perl_gp_dup ()
#16 0x080bd2c6 in Perl_sv_dup ()
#17 0x080bd13d in Perl_sv_dup ()
#18 0x080bd13d in Perl_sv_dup ()
#19 0x080bd196 in Perl_sv_dup ()
#20 0x080bd093 in Perl_sv_dup ()
#21 0x080bda18 in Perl_rvpv_dup ()
#22 0x080bc91f in Perl_sv_dup ()
#23 0x080a73da in Perl_he_dup ()
#24 0x080bcd54 in Perl_sv_dup ()
#25 0x080bdf59 in Perl_gp_dup ()
#26 0x080bd2c6 in Perl_sv_dup ()
#27 0x080bd13d in Perl_sv_dup ()
#28 0x080bd13d in Perl_sv_dup ()
#29 0x080bd196 in Perl_sv_dup ()
#30 0x080bdfa1 in Perl_gp_dup ()
#31 0x080bd2c6 in Perl_sv_dup ()
#32 0x080a73da in Perl_he_dup ()
#33 0x080bcd54 in Perl_sv_dup ()
#34 0x080bdf59 in Perl_gp_dup ()
#35 0x080bd2c6 in Perl_sv_dup ()
#36 0x080a73da in Perl_he_dup ()
#37 0x080bcd54 in Perl_sv_dup ()
#38 0x080bca54 in Perl_sv_dup ()
#39 0x080bdfa1 in Perl_gp_dup ()
#40 0x080bd2c6 in Perl_sv_dup ()
#41 0x080a73da in Perl_he_dup ()
#42 0x080bcd54 in Perl_sv_dup ()
#43 0x080bca54 in Perl_sv_dup ()
#44 0x080bdfa1 in Perl_gp_dup ()
#45 0x080bd2c6 in Perl_sv_dup ()
#46 0x080a73da in Perl_he_dup ()
#47 0x080bcd54 in Perl_sv_dup ()
#48 0x080bc970 in Perl_sv_dup ()
#49 0x080bded5 in Perl_gp_dup ()
#50 0x080bd2c6 in Perl_sv_dup ()
#51 0x080bd13d in Perl_sv_dup ()
#52 0x080bd13d in Perl_sv_dup ()
#53 0x080bd196 in Perl_sv_dup ()
#54 0x080bdfa1 in Perl_gp_dup ()
#55 0x080bd2c6 in Perl_sv_dup ()
#56 0x080a73da in Perl_he_dup ()
#57 0x080bcd54 in Perl_sv_dup ()
#58 0x080bdf59 in Perl_gp_dup ()
#59 0x080bd2c6 in Perl_sv_dup ()
#60 0x080a73da in Perl_he_dup ()
#61 0x080a7380 in Perl_he_dup ()
#62 0x080a7380 in Perl_he_dup ()
#63 0x080bcd54 in Perl_sv_dup ()
#64 0x080bca54 in Perl_sv_dup ()
#65 0x080bdfa1 in Perl_gp_dup ()
#66 0x080bd2c6 in Perl_sv_dup ()
#67 0x080a73da in Perl_he_dup ()
#68 0x080bcd54 in Perl_sv_dup ()
#69 0x080bd2ab in Perl_sv_dup ()
#70 0x080cc2cc in perl_clone ()
#71 0xb7ccbfc4 in XS_threads_create () from
/usr/lib/perl/5.10/auto/threads/threads.so
#72 0x080b3f02 in Perl_pp_entersub ()

#73 0x080b22e9 in Perl_runops_standard ()

#74 0x080b0750 in perl_run ()

#75 0x08063ebd in main ()


Flags​:

  category=

  severity=


Site configuration information for perl 5.10.0​:

Configured by Debian Project at Thu Jul 24 09​:01​:44 UTC 2008.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration​:
  Platform​:
  osname=linux, osvers=2.6.24-15-server,
archname=i486-linux-gnu-thread-multi
  uname='linux palmer 2.6.24-15-server #1 smp tue apr 8 01​:08​:17 utc
2008 i686 gnulinux '
  config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.10.0
-Dsitearch=/usr/local/lib/perl/5.10.0 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio
-Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib
-Dlibperl=libperl.so.5.10.0 -Dd_dosuid -des'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=undef, use64bitall=undef, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN
-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
  optimize='-O2 -g',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing
-pipe -I/usr/local/include'
  ccversion='', gccversion='4.3.1', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=4, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -L/usr/local/lib'
  libpth=/usr/local/lib /lib /usr/lib /usr/lib64
  libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
  perllibs=-ldl -lm -lpthread -lc -lcrypt
  libc=/lib/libc-2.8.90.so, so=so, useshrplib=true,
libperl=libperl.so.5.10.0
  gnulibc_version='2.8.90'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib'

Locally applied patches​:


@​INC for perl 5.10.0​:
  /etc/perl
  /usr/local/lib/perl/5.10.0
  /usr/local/share/perl/5.10.0
  /usr/lib/perl5
  /usr/share/perl5
  /usr/lib/perl/5.10
  /usr/share/perl/5.10
  /usr/local/lib/site_perl
  .


Environment for perl 5.10.0​:
  HOME=/home/pascal
  LANG=de_DE.UTF-8
  LANGUAGE=de
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)

PATH=/home/pascal/bin​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/sbin​:/bin​:/usr/games
  PERL_BADLANG (unset)
  SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented May 9, 2009

From john.wright@hp.com

On Fri Nov 21 00​:39​:51 2008, mail@​pascalhofmann.de wrote​:

I'have a program which uses threads and has a custom signal handler
for SIGCHLD. If a SIGCHLD is received during thread creation perl
produces a segfault.

This bug lies in the fact that Perl_csighandler() relies on the
thread-specific my_perl struct being in a consistent state. But unless
we block signals during certain points in the code, that's not
guaranteed to be the case.

I have attached a reproducer, crash-with-threads.pl, based on one
written by Pascal Hofmann. It crashes variously in two ways​:

1. Crash when trying to do "PL_psig_pend[sig]++;". This happens
  sometimes when Perl is cloning the interpreter for a new thread (as
  in the backtrace attached here), and sometimes when it's tearing
  down a thread, with the same effect.
 
  When it's cloning the interpreter, it changes its context to the new
  interpreter before it's actually fully initialized, so if the signal
  handler fires during this time, it gets the cloned interpreter
  (which hasn't been actually assigned to a new thread yet), and tries
  to access a pointer inside that struct that is yet NULL.

  It can also get a signal while it's tearing down a thread. I've
  seen this happen both in the "parent" thread (where it changes its
  context like above), and in the thread that the victim interpreter
  was actually running in (where it happens to receive a signal after
  the parent starts destroying the interpreter and before the thread
  goes away).

  My solution here is to wrap the potentially dangerous sections
  (either around where the code changes its context before the
  interpreter is ready or before the interpreter will get destroyed)
  with some code that masks most of the signals, and restores to the
  orginal signal mask state afterwards.

2. Crashing when accessing PL_signals (which expands to
  my_perl->Isignals for me). This occurs when a new thread has been
  created (using pthread_create) but the thread hasn't had a chance to
  set its interpreter context with PERL_SET_CONTEXT(). Even though
  this is the first thing the thread does, it could receive a signal
  while pthread_create is doing its thing.

  My solution here is to block signals in the calling thread before
  calling pthread_create (actually, while it's setting up the new
  interpreter struct -- see case 1), setting the old signal mask to a
  new member of the thread struct, and then restore the signal mask
  when everything is clear in the new thread, and in the "parent"
  thread right after thread creation.

I also ran into a related bug, in the non-threaded case​: If Perl is in
the middle of exiting (after it is finished with actually interpreting
the Perl code), and it receivs a signal that had a registered handler in
the script, it is possible for the struct representing the interpreter
to have been torn down and its thread-local reference already removed.
Since the signal handler refers to this, it segfaults. (See
crash-without-threads.pl.

My solution to this is to de-register the signal handler before
destroying the Perl interpreter.

I have attached, as well as the reproducers above, two patches which fix
the problems for me. I'm sure the threading one needs some preprocessor
work, since it relies on pthread_sigmask(). I'm not familiar with any
of the other threading libraries Perl supports.

@p5pRT
Copy link
Author

p5pRT commented May 9, 2009

From john.wright@hp.com

0002-Mask-signals-in-thread-creation-and-destruction-to-a.patch
From 4c0ed7b7d0b87ebd1cd194a5232ea8b2813edaec Mon Sep 17 00:00:00 2001
From: John Wright <john.wright@hp.com>
Date: Wed, 6 May 2009 15:48:12 -0600
Subject: [PATCH 2/2] Mask signals in thread creation and destruction to avoid a segfault

If our signal handler gets called before a thread got a chance to run
PERL_SET_CONTEXT(), the call to get the thread-specific interpreter will
fail, and we'll end up with aTHX == NULL.  Prevent this from happening
by blocking most signals right before creating the new thread.  This
way, the new thread starts out with most signals blocked, and it
unblocks them when it's ready to handle them.

This required saving the original signal mask somewhere - I put it in
the thread struct.

In several places, PERL_SET_CONTEXT() is called before the target perl
interpreter struct has been fully initialized, so this patch adds code
to block signals around those blocks of code.
---
 ext/threads/threads.xs |   82 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 82 insertions(+), 0 deletions(-)
 mode change 100644 => 100755 ext/threads/threads.xs

diff --git a/ext/threads/threads.xs b/ext/threads/threads.xs
old mode 100644
new mode 100755
index a15f7ec..9574653
--- a/ext/threads/threads.xs
+++ b/ext/threads/threads.xs
@@ -75,6 +75,7 @@ typedef struct _ithread {
     IV stack_size;
     SV *err;                    /* Error from abnormally terminated thread */
     char *err_class;            /* Error object's classname if applicable */
+    sigset_t initial_sigmask;   /* Thread wakes up with signals blocked */
 } ithread;
 
 
@@ -114,6 +115,44 @@ typedef struct {
 
 #define MY_POOL (*my_poolp)
 
+/* Block most signals for calling thread, setting the old signal mask to
+ * oldmask, if it is not NULL */
+STATIC int
+S_block_most_signals(sigset_t *oldmask)
+{
+    sigset_t newmask;
+
+    sigfillset(&newmask);
+    /* Don't block certain "important" signals (stolen from mg.c) */
+#ifdef SIGILL
+    sigdelset(&newmask, SIGILL);
+#endif
+#ifdef SIGBUS
+    sigdelset(&newmask, SIGBUS);
+#endif
+#ifdef SIGSEGV
+    sigdelset(&newmask, SIGSEGV);
+#endif
+
+#ifdef WIN32
+    /* XXX: How to do this on win32? */
+    return 0;
+#else
+    return pthread_sigmask(SIG_BLOCK, &newmask, oldmask);
+#endif /* WIN32 */
+}
+
+/* Set the signal mask for this thread to newmask */
+STATIC int
+S_set_sigmask(sigset_t *newmask)
+{
+#ifdef WIN32
+    /* XXX: How to do this on win32? */
+    return 0;
+#else
+    return pthread_sigmask(SIG_SETMASK, newmask, NULL);
+#endif /* WIN32 */
+}
 
 /* Used by Perl interpreter for thread context switching */
 STATIC void
@@ -142,12 +181,19 @@ STATIC void
 S_ithread_clear(pTHX_ ithread *thread)
 {
     PerlInterpreter *interp;
+    sigset_t origmask;
 
     assert(((thread->state & PERL_ITHR_FINISHED) &&
             (thread->state & PERL_ITHR_UNCALLABLE))
                 ||
            (thread->state & PERL_ITHR_NONVIABLE));
 
+    /* We temporarily set the interpreter context to the interpreter being
+     * destroyed.  It's in no condition to handle signals while it's being
+     * taken apart.
+     */
+    S_block_most_signals(&origmask);
+
     interp = thread->interp;
     if (interp) {
         dTHXa(interp);
@@ -169,6 +215,7 @@ S_ithread_clear(pTHX_ ithread *thread)
     }
 
     PERL_SET_CONTEXT(aTHX);
+    S_set_sigmask(&origmask);
 }
 
 
@@ -415,6 +462,11 @@ S_ithread_run(void * arg)
     PERL_SET_CONTEXT(thread->interp);
     S_ithread_set(aTHX_ thread);
 
+    /* Thread starts with most signals blocked - restore the signal mask from
+     * the ithread struct.
+     */
+    S_set_sigmask(&thread->initial_sigmask);
+
     PL_perl_destruct_level = 2;
 
     {
@@ -448,6 +500,12 @@ S_ithread_run(void * arg)
         }
         JMPENV_POP;
 
+        /* The interpreter is finished, so this thread can stop receiving
+         * signals.  This way, our signal handler doesn't get called in the
+         * middle of our parent thread calling perl_destruct()...
+         */
+        S_block_most_signals(NULL);
+
         /* Remove args from stack and put back in params array */
         SPAGAIN;
         for (ii=len-1; ii >= 0; ii--) {
@@ -660,6 +718,25 @@ S_ithread_create(
     PL_srand_called = FALSE;   /* Set it to false so we can detect if it gets
                                   set during the clone */
 
+    /* perl_clone() will leave us the new interpreter's context.  This poses
+     * two problems for our signal handler.  First, it sets the new context
+     * before the new interpreter struct is fully initialized, so our signal
+     * handler might find bogus data in the interpreter struct it gets.
+     * Second, even if the interpreter is initialized before a signal comes in,
+     * we would like to avoid that interpreter receiving notifications for
+     * signals (especially when they ought to be for the one running in this
+     * thread), until it is running in its own thread.  Another problem is that
+     * the new thread will not have set the context until some time after it
+     * has started, so it won't be safe for our signal handler to run until
+     * that time.
+     *
+     * So we block most signals here, so the new thread will inherit the signal
+     * mask, and unblock them right after the thread creation.  The original
+     * mask is saved in the thread struct so that the new thread can restore
+     * the original mask.
+     */
+    S_block_most_signals(&thread->initial_sigmask);
+
 #ifdef WIN32
     thread->interp = perl_clone(aTHX, CLONEf_KEEP_PTR_TABLE | CLONEf_CLONE_HOST);
 #else
@@ -774,6 +851,11 @@ S_ithread_create(
 #  endif
         }
 
+    /* Now it's safe to accept signals, since we're in our own interpreter's
+     * context and we have created the thread.
+     */
+    S_set_sigmask(&thread->initial_sigmask);
+
 #  ifdef _POSIX_THREAD_ATTR_STACKSIZE
         /* Try to get thread's actual stack size */
         {
-- 
1.5.6.5

@p5pRT
Copy link
Author

p5pRT commented May 9, 2009

@p5pRT
Copy link
Author

p5pRT commented May 9, 2009

@p5pRT
Copy link
Author

p5pRT commented May 9, 2009

From john.wright@hp.com

0001-main-Unregister-signal-handler-before-destroying-my.patch
From a9450face7394716acd9ca8403d0f50e16b60b0a Mon Sep 17 00:00:00 2001
From: John Wright <john@johnwright.org>
Date: Wed, 6 May 2009 00:47:15 -0600
Subject: [PATCH 1/2] main: Unregister signal handler before destroying my_perl

If the signal handler runs after perl_destruct() has been called, it
will get an invalid (or NULL) my_perl when it asks for the
thread-specific interpreter struct.  This patch resets the signal
handler for any signal previously handled by PL_csighandlerp to SIG_DFL
before calling perl_destruct().
---
 miniperlmain.c |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/miniperlmain.c b/miniperlmain.c
index f60a3e0..f2302c2 100644
--- a/miniperlmain.c
+++ b/miniperlmain.c
@@ -67,7 +67,7 @@ main(int argc, char **argv, char **env)
 #endif
 {
     dVAR;
-    int exitstatus;
+    int exitstatus, i;
 #ifdef PERL_GLOBAL_STRUCT
     struct perl_vars *plvarsp = init_global_struct();
 #  ifdef PERL_GLOBAL_STRUCT_PRIVATE
@@ -116,6 +116,13 @@ main(int argc, char **argv, char **env)
     if (!exitstatus)
         perl_run(my_perl);
 
+    /* Unregister our signal handler before destroying my_perl */
+    for (i = 0; PL_sig_name[i]; i++) {
+	if (rsignal_state(PL_sig_num[i]) == (Sighandler_t) PL_csighandlerp) {
+	    rsignal(PL_sig_num[i], (Sighandler_t) SIG_DFL);
+	}
+    }
+
     exitstatus = perl_destruct(my_perl);
 
     perl_free(my_perl);
-- 
1.5.6.5

@p5pRT
Copy link
Author

p5pRT commented May 9, 2009

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2009

From @rgs

Fixed by those patches :

commit b762d86
Author​: John Wright <john.wright@​hp.com>
Date​: Wed May 6 15​:48​:12 2009 -0600

  Mask signals in thread creation and destruction to avoid a segfault
 
  If our signal handler gets called before a thread got a chance to run
  PERL_SET_CONTEXT(), the call to get the thread-specific interpreter will
  fail, and we'll end up with aTHX == NULL. Prevent this from happening
  by blocking most signals right before creating the new thread. This
  way, the new thread starts out with most signals blocked, and it
  unblocks them when it's ready to handle them.
 
  This required saving the original signal mask somewhere - I put it in
  the thread struct.
 
  In several places, PERL_SET_CONTEXT() is called before the target perl
  interpreter struct has been fully initialized, so this patch adds code
  to block signals around those blocks of code.

ext/threads/threads.xs | 82
++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 82 insertions(+), 0 deletions(-)

commit 01be072
Author​: John Wright <john@​johnwright.org>
Date​: Wed May 6 00​:47​:15 2009 -0600

  main​: Unregister signal handler before destroying my_perl
 
  If the signal handler runs after perl_destruct() has been called, it
  will get an invalid (or NULL) my_perl when it asks for the
  thread-specific interpreter struct. This patch resets the signal
  handler for any signal previously handled by PL_csighandlerp to SIG_DFL
  before calling perl_destruct().

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2009

@rgs - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant