From joknarf@free.fr

Created by joknarf@free.fr

With this piece of code sometimes deadlock for some threads (more threads=more often, maybe some race condition ?)
The same code does not causes deadlock with perl version 5.8.7 and below
Same deadlocks occurs with perl 5.10
Tested with same result on RedHat Linux 5.3 / Solaris 10 sparc / xubuntu / knoppix

#!/usr/bin/perl
use strict;
use warnings;
use threads;

my $Parallel=$ARGV[0];
threads->create(\&jobq, $_) for 1..$Parallel;
$_->join foreach threads->list;

sub jobq {
  my $Q=$_[0];
  printf("Queue %s started\n",$Q);
  open( PIPE, "| cat >/dev/null 2>&1");
  print PIPE "data";
  printf("Queue %s closing\n",$Q);
  close(PIPE);
  printf("Queue %s closed\n",$Q);
}

The purpose of the script is to distribute jobs to threads that are piped to another command

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.10.0:

Configured by Debian Project at Thu Jan  1 12:43:38 UTC 2009.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.6.26-1-686, archname=i486-linux-gnu-thread-multi
    uname='linux rebekka 2.6.26-1-686 #1 smp mon dec 15 18:15:07 utc 2008 i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.10.0 -Dsitearch=/usr/local/lib/perl/5.10.0 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.10.0 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.3.2', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib /usr/lib64
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.7.so, so=so, useshrplib=true, libperl=libperl.so.5.10.0
    gnulibc_version='2.7'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib'

Locally applied patches:
    


@INC for perl 5.10.0:
    /etc/perl
    /usr/local/lib/perl/5.10.0
    /usr/local/share/perl/5.10.0
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.10
    /usr/share/perl/5.10
    /usr/local/lib/site_perl
    .


Environment for perl 5.10.0:
    HOME=/home/knoppix
    LANG=C
    LANGUAGE=C
    LC_ALL=C
    LC_MESSAGES=C
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/bash 

From @iabyn

On Wed, Mar 04, 2009 at 11​:49​:11PM -0800, JO Knarf wrote​:

With this piece of code sometimes deadlock for some threads (more threads=more often, maybe some race condition ?)
The same code does not causes deadlock with perl version 5.8.7 and below
Same deadlocks occurs with perl 5.10

I also see deadlocks with perl 5.8.1 and 5.8.7 (for ARGV[0]=10).
This is Linux, 2-CPU i686.

Tested with same result on RedHat Linux 5.3 / Solaris 10 sparc / xubuntu / knoppix

#!/usr/bin/perl
use strict;
use warnings;
use threads;

my $Parallel=$ARGV[0];
threads->create(\&jobq, $_) for 1..$Parallel;
$_->join foreach threads->list;

sub jobq {
my $Q=$_[0];
printf("Queue %s started\n",$Q);
open( PIPE, "| cat >/dev/null 2>&1");
print PIPE "data";
printf("Queue %s closing\n",$Q);
close(PIPE);
printf("Queue %s closed\n",$Q);
}

The deadlock appears to be that the parents (i.e. the threads that hold the
write end of the main pipe) are waiting to read any error status back
from the forked children via an auxiliary pipe, but although the
children have exec'd sh (which then runs cat), and so the write end of the
auxiliary pipe should have been closed (its made close-on-exec), the
parents haven't been given a 0 return from the read() system call.

Might be some interaction between POSIX threads and fork, but I don't
really have time to look in further now.

--
The Enterprise is involved in a bizarre time-warp experience which is in
some way unconnected with the Late 20th Century.
  -- Things That Never Happen in "Star Trek" #14

The RT System itself - Status changed from 'new' to 'open'

From starlight@binnacle.cx

may be same as bug 72120

From starlight@binnacle.cx

deadlock caused by race in pipe open logic in need of mutex​:

pipe()
fcntl(F_SETFD(FD_CLOEXEC))
fork()

without mutex the child processes will inherit
each other's pipes from the parent preventing
proper EOF detection by parent threads and
child processes

From starlight@binnacle.cx

forgot to add​:

one can work around this problem by wrapping
the pipe-open statement in a perl lock; this
single-threads the aforementioned system call
sequence and eliminates sibling pipe inheritance

From starlight@binnacle.cx

On Sat Jan 16 04​:38​:45 2010, starlight wrote​:

forgot to add​:

one can work around this problem by wrapping
the pipe-open statement in a perl lock; this
single-threads the aforementioned system call
sequence and eliminates sibling pipe inheritance

Oops--not true. Was able to induce deadlocks even with perl locks
in place, though a much higher number of instances is required.

Perl locks do appear to fix the problem if the threads live
forever as with our script. A semaphore is used to wake up the
workers when needed, and the lock appears to prevent the sibling
pipe inheritance.