Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PATCH] Reap child in case where exception has been thrown #12371

Closed
p5pRT opened this issue Sep 3, 2012 · 10 comments
Closed

[PATCH] Reap child in case where exception has been thrown #12371

p5pRT opened this issue Sep 3, 2012 · 10 comments
Labels

Comments

@p5pRT
Copy link

p5pRT commented Sep 3, 2012

Migrated from rt.perl.org#114722 (status was 'resolved')

Searchable as RT114722$

@p5pRT
Copy link
Author

p5pRT commented Sep 3, 2012

From philip.boulain@smoothwall.net

This is a bug report for perl from philip.boulain@​smoothwall.net,
generated with the help of perlbug 1.39 running under perl 5.14.2.


IPC​::Open3 does not behave as documented for exec failures in the
child; in particular, if the command to be executed does not
exist.

As documented, open3 should return the child PID, which should
then die with a complaint to its error handle. It would then be
the responsibility of the calling code to reap the child, as per
normal.

In practice, open3 throws an exception, as for other errors (I
believe this was introduced by feature request #72016). This
means that the calling code cannot learn the child pid, since
open3 never returns. Hence only open3 can reap the child, but it
does not appear to do so.

There are therefore two problems​:
- exec failures leave zombies
- the documentation for exec failures is out of date

Attached​:
* open3bug - quick script to demonstrate/reproduce the problem,
  just for bug context; not for merge
* 0001-Reap-child-in-case-where-exception-has-been-thrown.patch
  - test case and proposed fix for the issue
  (reap the child and update the documentation)

The test case failed correctly without the fix, and passes with
the fix, but please be aware that I have only been able to test
under Linux.



Flags​:
  category=library
  severity=low
  module=IPC​::Open3


Site configuration information for perl 5.14.2​:

Configured by Debian Project at Fri Aug 10 21​:43​:06 UTC 2012.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration​:

  Platform​:
  osname=linux, osvers=2.6.42-26-generic,
archname=x86_64-linux-gnu-thread-multi
  uname='linux allspice 2.6.42-26-generic #41-ubuntu smp thu jun 14
17​:49​:24 utc 2012 x86_64 x86_64 x86_64 gnulinux '
  config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.14 -Darchlib=/usr/lib/perl/5.14
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.14.2
-Dsitearch=/usr/local/lib/perl/5.14.2 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1
-Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh
-Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -DDEBUGGING=-g -Doptimize=-O2
-Duseshrplib -Dlibperl=libperl.so.5.14.2 -des'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2 -g',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.6.3', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
  libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
  perllibs=-ldl -lm -lpthread -lc -lcrypt
  libc=, so=so, useshrplib=true, libperl=libperl.so.5.14.2
  gnulibc_version='2.15'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib
-fstack-protector'

Locally applied patches​:


@​INC for perl 5.14.2​:
  /etc/perl
  /usr/local/lib/perl/5.14.2
  /usr/local/share/perl/5.14.2
  /usr/lib/perl5
  /usr/share/perl5
  /usr/lib/perl/5.14
  /usr/share/perl/5.14
  /usr/local/lib/site_perl
  .


Environment for perl 5.14.2​:
  HOME=/home/philip
  LANG=en_GB.UTF-8
  LANGUAGE=en_GB​:en
  LD_LIBRARY_PATH=/home/philip/opt/lib​:
  LOGDIR (unset)
  PATH=/home/philip/opt/bin​:/usr/lib/lightdm/lightdm​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/sbin​:/bin​:/usr/games
  PERL_BADLANG (unset)
  SHELL=/bin/bash

--
Dr. Philip Boulain
Developer
philip.boulain@​smoothwall.net

Smoothwall Ltd
Phone​: +44 (0) 8701 999500
www.smoothwall.net

Smoothwall Limited is registered in England, Company Number​: 4298247
and whose registered address is 1 John Charles Way, Leeds, LS12 6QA
United Kingdom
This email and any attachments transmitted with it are confidential to
the intended recipient(s) and may not be communicated to any other
person or published by any means without the permission of Smoothwall
Limited. Any opinions stated in this message are solely those of the
author.

@p5pRT
Copy link
Author

p5pRT commented Sep 3, 2012

@p5pRT
Copy link
Author

p5pRT commented Sep 3, 2012

From philip.boulain@smoothwall.net

0001-Reap-child-in-case-where-exception-has-been-thrown.patch
From 7be72bf92786362968ed95b3a040288b2f6a5e0d Mon Sep 17 00:00:00 2001
From: Philip Boulain <philip.boulain@smoothwall.net>
Date: Mon, 3 Sep 2012 15:16:26 +0100
Subject: [PATCH] Reap child in case where exception has been thrown

If open3 throws due to an issue such as an exec failure, the caller
cannot know the child PID to wait for. Therefore it is our
responsibility to reap it.

Also update POD, since on some platforms exec failures now ARE raised as
exceptions (since perlbug #72016).
---
 ext/IPC-Open3/lib/IPC/Open3.pm |    4 +++-
 ext/IPC-Open3/t/IPC-Open3.t    |    7 ++++++-
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/ext/IPC-Open3/lib/IPC/Open3.pm b/ext/IPC-Open3/lib/IPC/Open3.pm
index 31c68af..8a2264e 100644
--- a/ext/IPC-Open3/lib/IPC/Open3.pm
+++ b/ext/IPC-Open3/lib/IPC/Open3.pm
@@ -57,7 +57,8 @@ as file descriptors.
 open3() returns the process ID of the child process.  It doesn't return on
 failure: it just raises an exception matching C</^open3:/>.  However,
 C<exec> failures in the child (such as no such file or permission denied),
-are just reported to CHLD_ERR, as it is not possible to trap them.
+are just reported to CHLD_ERR under Windows and OS/2, as it is not possible
+to trap them.
 
 If the child process dies for any reason, the next write to CHLD_IN is
 likely to generate a SIGPIPE in the parent, which is fatal by default.
@@ -295,6 +296,7 @@ sub _open3 {
 	    if ($bytes_read) {
 		(my $bang, $to_read) = unpack('II', $buf);
 		read($stat_r, my $err = '', $to_read);
+		waitpid $kidpid, 0; # Reap child which should have exited
 		if ($err) {
 		    utf8::decode $err if $] >= 5.008;
 		} else {
diff --git a/ext/IPC-Open3/t/IPC-Open3.t b/ext/IPC-Open3/t/IPC-Open3.t
index 7b85b82..6ab519d 100644
--- a/ext/IPC-Open3/t/IPC-Open3.t
+++ b/ext/IPC-Open3/t/IPC-Open3.t
@@ -14,10 +14,11 @@ BEGIN {
 }
 
 use strict;
-use Test::More tests => 37;
+use Test::More tests => 38;
 
 use IO::Handle;
 use IPC::Open3;
+use POSIX ":sys_wait_h";
 
 my $perl = $^X;
 
@@ -154,6 +155,10 @@ $TB->current_test($test);
     isnt($@, '',
 	 'open3 of a non existent program fails with an exception in the parent')
 	or do {waitpid $pid, 0};
+    SKIP: {
+	skip 'open3 returned, our responsibility to reap', 1 unless $@;
+	is(waitpid(-1, WNOHANG), -1, 'failed exec child is reaped');
+    }
 }
 
 $pid = eval { open3 'WRITE', '', 'ERROR', '/non/existent/program'; };
-- 
1.7.9.5

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2012

From @jkeenan

On Mon Sep 03 08​:42​:06 2012, philip.boulain@​smoothwall.net wrote​:

This is a bug report for perl from philip.boulain@​smoothwall.net,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
IPC​::Open3 does not behave as documented for exec failures in the
child; in particular, if the command to be executed does not
exist.

As documented, open3 should return the child PID, which should
then die with a complaint to its error handle. It would then be
the responsibility of the calling code to reap the child, as per
normal.

In practice, open3 throws an exception, as for other errors (I
believe this was introduced by feature request #72016). This
means that the calling code cannot learn the child pid, since
open3 never returns. Hence only open3 can reap the child, but it
does not appear to do so.

There are therefore two problems​:
- exec failures leave zombies
- the documentation for exec failures is out of date

Attached​:
* open3bug - quick script to demonstrate/reproduce the problem,
just for bug context; not for merge
* 0001-Reap-child-in-case-where-exception-has-been-thrown.patch
- test case and proposed fix for the issue
(reap the child and update the documentation)

The test case failed correctly without the fix, and passes with
the fix, but please be aware that I have only been able to test
under Linux.

Your demonstration program, open3bug, uses 'pstree', which appears to be
Linux-specific. Would you have an equivalent way of demonstrating the
bug on a system that lacks 'pstree' (as my Darwin does)? That might
elicit more responses.

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2012

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2012

From philip.boulain@smoothwall.net

On 6 September 2012 02​:35, James E Keenan via RT
<perlbug-followup@​perl.org> wrote​:

Your demonstration program, open3bug, uses 'pstree', which appears to be
Linux-specific. Would you have an equivalent way of demonstrating the
bug on a system that lacks 'pstree' (as my Darwin does)? That might
elicit more responses.

Hmm, sorry, I don't see how to get comparable behaviour from BSD
ps-like tools. The test case should still work, though, since it
checks for the presence of the child with waitpid.

If you have access to GNU ps (BSD has no --ppid-alike I can find), you
could change show_pstree() in open3bug to​:

sub show_pstree { system ('ps', 'ufp', $$, '--ppid', $$); }

Failing that, here's the output on my machine (configuration as per
perlbug's output above) with that change​:

====
Before any IPC​:

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
philip 4013 0.0 0.0 23400 3340 pts/6 S+ 10​:30 0​:00
/usr/bin/perl /home/philip/open3bug
philip 4014 0.0 0.0 18132 1156 pts/6 R+ 10​:30 0​:00 \_
ps ufp 4013 --ppid 4013

open3 threw an exception!
(open3​: exec of /tmp/does_not_exist --version failed at
/home/philip/open3bug line 19)
As a result, pid is not set!
If open3 has not reaped the child itself, this is a bug, since this
code cannot possibly (safely) reap an unknown zombie.

After IPC, before waitpid (if known)​:

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
philip 4013 0.5 0.0 23528 3428 pts/6 S+ 10​:30 0​:00
/usr/bin/perl /home/philip/open3bug
philip 4015 0.0 0.0 0 0 pts/6 Z+ 10​:30 0​:00 \_
[open3bug] <defunct>
philip 4016 0.0 0.0 18132 1156 pts/6 R+ 10​:30 0​:00 \_
ps ufp 4013 --ppid 4013

No child PID known; can't waitpid.
Final state​:

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
philip 4013 0.5 0.0 23528 3428 pts/6 S+ 10​:30 0​:00
/usr/bin/perl /home/philip/open3bug
philip 4015 0.0 0.0 0 0 pts/6 Z+ 10​:30 0​:00 \_
[open3bug] <defunct>
philip 4017 0.0 0.0 18132 1156 pts/6 R+ 10​:30 0​:00 \_
ps ufp 4013 --ppid 4013

(I added a very short sleep after the exception warning on line 26, so
that the child has a chance to exit and the ps output is a little
clearer.)

--
Dr. Philip Boulain
Developer
philip.boulain@​smoothwall.net

Smoothwall Ltd
Phone​: +44 (0) 8701 999500
www.smoothwall.net

Smoothwall Limited is registered in England, Company Number​: 4298247 and
whose registered address is 1 John Charles Way, Leeds, LS12 6QA United
Kingdom
Any opinions stated in this message are solely those of the author.

@p5pRT
Copy link
Author

p5pRT commented Jul 25, 2013

From @tonycoz

On Wed Sep 05 18​:35​:49 2012, jkeenan wrote​:

Your demonstration program, open3bug, uses 'pstree', which appears to be
Linux-specific. Would you have an equivalent way of demonstrating the
bug on a system that lacks 'pstree' (as my Darwin does)? That might
elicit more responses.

A simpler way to demonstrate it, in one terminal​:

# cwd is a built blead
pallas​:perl tony$ ./perl -Ilib -MIPC​::Open3 -le 'for (1..10) { my ($in,
$out); eval { my $pid = open3($in, $out, "", "unknownprogram");
waitpid($pid, 0) }; } sleep 100'

in another​:

pallas​:~ tony$ ps w
  PID TT STAT TIME COMMAND
7361 s002 Ss 0​:00.03 -bash
40376 s002 S+ 0​:00.02 ./perl -Ilib -MIPC​::Open3 -le for (1..10) {
my ($in, $out); eval { my $pid = open3($in, $out, "", "unkno
40377 s002 Z+ 0​:00.00 (perl)
40378 s002 Z+ 0​:00.00 (perl)
40379 s002 Z+ 0​:00.00 (perl)
40380 s002 Z+ 0​:00.00 (perl)
40381 s002 Z+ 0​:00.00 (perl)
40382 s002 Z+ 0​:00.00 (perl)
40383 s002 Z+ 0​:00.00 (perl)
40384 s002 Z+ 0​:00.00 (perl)
40385 s002 Z+ 0​:00.00 (perl)
40386 s002 Z+ 0​:00.00 (perl)
40303 s003 Ss 0​:00.01 -bash

(the processes with the Z+ flags are zombies)

The patch fixes the problem.

I'll apply this in a couple of days unless someone points out a problem
with it (along with an AUTHORS update and version bump)

Tony

@p5pRT
Copy link
Author

p5pRT commented Jul 29, 2013

From @tonycoz

On Wed Jul 24 23​:07​:35 2013, tonyc wrote​:

On Wed Sep 05 18​:35​:49 2012, jkeenan wrote​:

Your demonstration program, open3bug, uses 'pstree', which appears to be
Linux-specific. Would you have an equivalent way of demonstrating the
bug on a system that lacks 'pstree' (as my Darwin does)? That might
elicit more responses.

A simpler way to demonstrate it, in one terminal​:

# cwd is a built blead
pallas​:perl tony$ ./perl -Ilib -MIPC​::Open3 -le 'for (1..10) { my ($in,
$out); eval { my $pid = open3($in, $out, "", "unknownprogram");
waitpid($pid, 0) }; } sleep 100'

in another​:

pallas​:~ tony$ ps w
PID TT STAT TIME COMMAND
7361 s002 Ss 0​:00.03 -bash
40376 s002 S+ 0​:00.02 ./perl -Ilib -MIPC​::Open3 -le for (1..10) {
my ($in, $out); eval { my $pid = open3($in, $out, "", "unkno
40377 s002 Z+ 0​:00.00 (perl)
40378 s002 Z+ 0​:00.00 (perl)
40379 s002 Z+ 0​:00.00 (perl)
40380 s002 Z+ 0​:00.00 (perl)
40381 s002 Z+ 0​:00.00 (perl)
40382 s002 Z+ 0​:00.00 (perl)
40383 s002 Z+ 0​:00.00 (perl)
40384 s002 Z+ 0​:00.00 (perl)
40385 s002 Z+ 0​:00.00 (perl)
40386 s002 Z+ 0​:00.00 (perl)
40303 s003 Ss 0​:00.01 -bash

(the processes with the Z+ flags are zombies)

The patch fixes the problem.

I'll apply this in a couple of days unless someone points out a problem
with it (along with an AUTHORS update and version bump)

Thanks, applied as cccbbce.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jul 29, 2013

@tonycoz - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this as completed Jul 29, 2013
@p5pRT
Copy link
Author

p5pRT commented Jul 29, 2013

From @ikegami

On Thu, Jul 25, 2013 at 2​:07 AM, Tony Cook via RT <perlbug-followup@​perl.org

wrote​:

The patch fixes the problem.

I'll apply this in a couple of days unless someone points out a problem
with it

It changes my code and it looks right to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant