From @jdhedden

Created by @jdhedden

A pipe opened before a thread is created will cause the application
to hang when the thread is subsequently joined​:

  open(my $OUT, '| cat') || die("ERROR​: $!");
  threads->create(sub { })->join(); # <--- hangs

Complete sample script attached.

This may be related to [perl #63662]
http​://rt.perl.org/rt3/Ticket/Display.html?id=63662

Flags:
    category=core
    severity=high

Site configuration information for perl 5.12.2:

Configured by Jerry at Tue Oct 19 09:58:12 EDT 2010.

Summary of my perl5 (revision 5 version 12 subversion 2 patch 43208)
configuration:
  Snapshot of: 4ffae9cec64e9dd4150c249b8d63bdceb4e65fb2
  Platform:
    osname=cygwin, osvers=1.5.25(0.15642), archname=cygwin-thread-multi-64int
    uname='cygwin_nt-5.1 pn100-02-2-364p 1.5.25(0.15642) 2008-06-12
19:34 i686 cygwin '
    config_args='-de -Duse64bitint -Dusethreads
-Dinc_version_list=none -Dnoextensions=IPC/SysV Sys/Syslog Devel/DProf
Devel/Peek re GDBM_File NDBM_File ODBM_File Text/Soundex
Math/BigInt/FastCalc Time/Piece -A append:ccflags= -DNO_MATHOMS  -A
define:optimize=-Os -pipe -funit-at-a-time -march=pentium4
-mfpmath=sse -mieee-fp -mmmx -msse -msse2'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=undef, uselongdouble=undef
    usemymalloc=y, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__
-DNO_MATHOMS -fno-strict-aliasing -pipe',
    optimize='-Os -pipe -funit-at-a-time -march=pentium4 -mfpmath=sse
-mieee-fp -mmmx -msse -msse2',
    cppflags='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -DNO_MATHOMS
-fno-strict-aliasing -pipe'
    ccversion='', gccversion='3.4.4 (cygming special, gdc 0.12, using
dmd 0.125)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long long', ivsize=8, nvtype='double', nvsize=8,
Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='g++', ldflags =' -Wl,--enable-auto-import
-Wl,--export-all-symbols -Wl,--stack,8388608
-Wl,--enable-auto-image-base -s -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib /lib
    libs=-lgdbm -ldl -lcrypt -lgdbm_compat
    perllibs=-ldl -lcrypt
    libc=/usr/lib/libc.a, so=dll, useshrplib=true, libperl=cygperl5_12_2.dll
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' -s'
    cccdlflags=' ', lddlflags=' --shared  -Wl,--enable-auto-import
-Wl,--export-all-symbols -Wl,--stack,8388608
-Wl,--enable-auto-image-base -s -L/usr/local/lib'

Locally applied patches:



@INC for perl 5.12.2:
    /usr/lib/perl5/site_perl/5.12.2/cygwin
    /usr/lib/perl5/site_perl/5.12.2
    /usr/lib/perl5/5.12.2/cygwin
    /usr/lib/perl5/5.12.2
    .


Environment for perl 5.12.2:
    CYGWIN=ntsec
    HOME=/home/jhedden
    LANG=C
    LANGUAGE=C
    LC_ALL=C
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/jhedden/bin:/usr/local/src/perl/bin:/link/work/bin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/c/Program
Files/WiX:/c/djgpp/bin:/c/bb/tools/apache-ant-1.6.5/bin:/c/Program
Files/nant-0.85/bin:/c/bb/tools/jse5/bin:/c/dev-cpp/bin:/c/WINDOWS/system32:/c/WINDOWS:/c/WINDOWS/system32/WBEM:/c/blp/API/dde:/c/blp/API:/c/oracle/ora92/bin:/c/Program
Files/Oracle/jre/1.3.1/bin:/c/Program
Files/Oracle/jre/1.1.8/bin:/c/Program
Files/Hummingbird/Connectivity/7.10/Accessories:/usr/bin:/c/Program
Files/Windows Imaging/:/c/Program Files/Citrix/ICAService/:/c/Program
Files/Citrix/System32/:.
    PERLIO=perlio
    PERL_BADLANG (unset)
    SHELL (unset)

From @jdhedden

hang.pl

From matthew.kidd@ghctechnologies.com

I am the original source of Bug Id 78494​:

http​://rt.perl.org/rt3//Public/Bug/Display.html?id=78494

I first reported it as a bug against the threads package​:

https://rt.cpan.org/Public/Bug/Display.html?id=61705

Jerry Hedden was able to greatly simplify my bug report in the
report he filed against perlcore. But I think his simplification
makes the bug seem unimportant and that perhaps led to the low
severity assignment.

In practice the bug is much more problematic because it means
compressed I/O (e.g. gzip) can not be handled in a threaded
environment. This is a big deal in an increasingly multi-core
world where one is handling a lot of data. It affects me in a
bioinformatics context but I can imagine many data mining
scenarios that would also be adversely affected.

In some cases it is possible to work around the bug by piping
compressed input into Perl and compressing the output. But this
only works if there is a singled input and/or output stream.

Accordingly, I am requesting that the priority of Bug Id 78494
be boosted.

  - Matthew Kidd
  GHC Technologies

From @cpansprout

On Thu Oct 21 07​:31​:08 2010, jdhedden@​cpan.org wrote​:

A pipe opened before a thread is created will cause the application
to hang when the thread is subsequently joined​:

open(my $OUT, '| cat') || die("ERROR​: $!");
threads->create(sub { })->join(); # <--- hangs

close($OUT) within the thread also hangs.

The same thing happens with this, too​:

open(my $OUT, "| cat -n");
open my $OUT2, ">&", $OUT;
close $OUT;

Closing $OUT2 first works, because it has somehow been marked as not
needing to wait for a process. (I believe its entry in PL_fdpid is
empty, but have not checked.) Closing $OUT hangs, because it’s waiting
for the process to end while $OUT2 is still attached to it.

PerlIOUnix_dup and PerlIOStdio_dup do not call PerlLIO_dup during
cloning, but only for ">&" (PERLIO_DUP_CLONE vs. PERLIO_DUP_FD). So we
end up with two file handles (PerlIO* things) with the same fileno. I
don’t know how that is meant to work. When is the C-level file
descriptor freed? (I know very little about I/O in C. That probably shows.)

Also, PL_fdpid is a per-thread variable, so I am completely at a loss
how to solve this.

It seems that any pipe duplication is doomed to cause a hang somewhere.
Why does perl have to wait for the process to end anyway? Is it to avoid
zombies?

It also seems that any existing pipes must be closed before threads are
created (just like dir handles in 5.12.x and earlier), because then a
deadlock will be guaranteed.

The RT System itself - Status changed from 'new' to 'open'

From @Leont

On Sun, Feb 13, 2011 at 10​:18 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

PerlIOUnix_dup and PerlIOStdio_dup do not call PerlLIO_dup during
cloning, but only for ">&" (PERLIO_DUP_CLONE vs. PERLIO_DUP_FD). So we
end up with two file handles (PerlIO* things) with the same fileno. I
don’t know how that is meant to work. When is the C-level file
descriptor freed? (I know very little about I/O in C. That probably shows.)

PerlIO does reference counting on file descriptors, see
PerlIOUnix_refcnt_inc and PerlIOUnix_refcnt_dec in perlio.c. Having
said that, I do suspect it leaks on some circumstances, though I
haven't had the time yet to confirm my suspicions.

Leon

From @cpansprout

I’ve just used the fd-refcounting mechanism to solve this for threads
with commit 2e0cfa1.

The other case I discovered involving ">&" is unresolved, but I will put
that in another ticket.

@cpansprout - Status changed from 'open' to 'resolved'

From @cpansprout

On Tue Feb 15 16​:38​:11 2011, sprout wrote​:

I’ve just used the fd-refcounting mechanism to solve this for threads
with commit 2e0cfa1.

That does not solve #63662, so these appear to be two separate issues.