Perl hangs when when using backticks or popen from a thread on Win32 #8489

p5pRT · 2006-06-19T16:51:23Z

Migrated from rt.perl.org#39531 (status was 'resolved')

Searchable as RT39531$

p5pRT · 2006-06-19T16:51:24Z

From KDoyle@emdeon.com

This is a bug report for perl from kdoyle@emdeon.com,
generated with the help of perlbug 1.35 running under perl v5.8.7.

Discovered that in ActiveState Perl build 813 & 817 will often hang when
multiple threads are running that use either backtic or popen to execute an
external to capture the output. Have reproduced it on 5 machines in-house,
and just received confirmation from the PerlMonk's site from another user
whose been able to reproduce. Has been reproduced on Windows XP SP1 & SP2,
and on single thread and multi-thread machines.

Some machines will hang immediately, some will require looping the test a
few times, but on all machines tested the loop would produce a hang very
quickly, within about a dozen iterations. Also tested on Debian Linux,
works without problems, so it appears to be unique to Windows.

===========example code follows: thread2.pl===============================
#! perl -slw
use strict;
use threads;

sub test {
my $t = shift;

my $cmd = "c:/windows/system32/sort.exe test$t.dat";

sleep 1; # kludge to insure all threads are up...

system "$cmd >sorted$t.dat";

open F, "<sorted$t.dat" or die $!;
print "Via system:\n", do{ local $/; <F> };
close F;

print "Via backticks\n", `$cmd`;

open F, "$cmd|" or die $!;
print "Via pipe\n", <F>;
close F;
}

my $t1 = async{ \&test(1); };
my $t2 = async{ \&test(2); };
my $t3 = async{ \&test(3); };
my $t4 = async{ \&test(4); };

$t1->join;
$t2->join;
$t3->join;
$t4->join;
=====Test data file follows (all 4 test files are the same,
test[1-4].dat=====
world
hello
===========Windows .bat file
tester===========================================
:LOOP
perl thread2.pl
goto LOOP

The test program runs window's "sort" in four background threads to sort
four test files. The path to Windows sort may differ on some machines (some
have a c:/winnt directory instead of c:/windows for example). It doesn't
matter what the background program is, sort was used because it was
convenient to reproduce the problem with.

Each of the threads test the three methods of executing external binaries,
first with "system", then with backtic, and finally with popen. If system
is the only one used it won't hang, either backtic or popen will hang if run
by themselves. Consequently, I'm currently using system as a workaround,
but it's unfortunate that it requires additional disk I/O as in my
application performance is important and disk I/O is a primary bottleneck.

--

Keith Doyle

Emdeon

Senior Systems and Applications Engineer
Emdeon Practice Services
22010 Wilmington Ave #109 - Carson, CA 90745
Office: 877-932-6301 x25155 - Email: kdoyle@emdeon.com
www.emdeon.com

Flags:
category=core
severity=medium

Site configuration information for perl v5.8.7:

Configured by builder at Mon Jun 6 13:36:05 2005.

Summary of my perl5 (revision 5 version 8 subversion 7) configuration:
Platform:
osname=MSWin32, osvers=5.0, archname=MSWin32-x86-multi-thread
uname=''
config_args='undef'
hint=recommended, useposix=true, d_sigaction=undef
usethreads=define use5005threads=undef useithreads=define
usemultiplicity=define
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cl', ccflags ='-nologo -Gf -W3 -MD -Zi -DNDEBUG -O1 -DWIN32
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DBUILT_BY_ACTIVESTATE
-DNO_HASH_SEED -DUSE_SITECUSTOMIZE -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
optimize='-MD -Zi -DNDEBUG -O1',
cppflags='-DWIN32'
ccversion='12.00.8804', gccversion='', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf
-libpath:"C:\Perl\lib\CORE" -machine:x86'
libpth=\lib
libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib
uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib
msvcrt.lib
perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib
uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib
msvcrt.lib
libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
gnulibc_version='undef'
Dynamic Linking:
dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug
-opt:ref,icf -libpath:"C:\Perl\lib\CORE" -machine:x86'

Locally applied patches:
ACTIVEPERL_LOCAL_PATCHES_ENTRY
# if !defined(PERL_DARWIN)
Iin_load_module moved for compatibility with build 806
# endif
# if defined(__hpux)
Avoid signal flag SA_RESTART for older versions of HP-UX
# endif
PerlEx hacks for CGI::Carp
Less verbose ExtUtils::Install and Pod::Find
instmodsh upgraded from ExtUtils-MakeMaker-6.25
24699 ICMP_UNREACHABLE handling in Net::Ping
21540 Fix backward-compatibility issues in if.pm

@INC for perl v5.8.7:
C:\Program Files\ActiveState Perl Dev Kit 6.0\lib\
C:/perl/lib
C:/perl/site/lib
.

Environment for perl v5.8.7:
HOME=c:\MKSTools
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=C:/Program Files/ActiveState Perl Dev Kit
6.0/bin;C:/MKSTools/mksnt;C:/PROGRA~~1/ACTIVE~~1.0�in/;C:/Perl�in/;C:/MKStools
/Bin;C:/Perl/Bin;C:/PROGRA~~1/WI8DE7~~1/Tools/;C:/WINDOWS/system32;C:/WINDOWS;
C:/WINDOWS/System32/Wbem;C:/usr/local/wbin;C:/PROGRA1/Vim/Vim63;C:/ORANT/BI
N;C:/PROGRA1/HUMMIN~~1/CONNEC~~1/7.11/ACCESS~~1/;.;C:/PROGRA~~1/MI68411/80/Too
ls/Binn/;C:/PROGRA1/MI6841~~1/90/DTS/Binn/;C:/PROGRA~~1/MI68411/90/Tools/Bin
n/;C:/PROGRA1/MI6841~~1/90/Tools/Binn/VSShell/Common7/IDE/;C:/PROGRA~~1/MID05
A~~1/Common7/IDE/PRIVAT~~1/;C:/MKStools/mksnt;C:/PROGRA1/BORLAND/DELPHI4/BIN;
C:/PROGRA1/BORLAND/VBROKER/BIN;C:/PROGRA~1/BORLAND/VBROKER/JRE/BIN;.
PERL5LIB=C:\Program Files\ActiveState Perl Dev Kit 6.0\lib\
PERLDB_OPTS=RemotePort=127.0.0.1:2000
PERL_BADLANG (unset)
SHELL=C:/MKSTOOLS/MKSNT/SH.EXE

p5pRT · 2013-05-27T13:12:12Z

From @jkeenan

On Mon Jun 19 09:51:24 2006, KDoyle@emdeon.com wrote:

This is a bug report for perl from kdoyle@emdeon.com,
generated with the help of perlbug 1.35 running under perl v5.8.7.

-----------------------------------------------------------------

Discovered that in ActiveState Perl build 813 & 817 will often hang when
multiple threads are running that use either backtic or popen to
execute an
external to capture the output. Have reproduced it on 5 machines
in-house,
and just received confirmation from the PerlMonk's site from another user
whose been able to reproduce. Has been reproduced on Windows XP SP1 & SP2,
and on single thread and multi-thread machines.

Some machines will hang immediately, some will require looping the test a
few times, but on all machines tested the loop would produce a hang very
quickly, within about a dozen iterations. Also tested on Debian Linux,
works without problems, so it appears to be unique to Windows.

===========example code follows: thread2.pl===============================
#! perl -slw
use strict;
use threads;

sub test {
my $t = shift;
    my $cmd = "c&#8203;:/windows/system32/sort\.exe test$t\.dat";

sleep 1;    \# kludge to insure all threads are up\.\.\.

    system "$cmd >sorted$t\.dat";

    open F\, "\<sorted$t\.dat" or die $\!;
    print "Via system&#8203;:\\n"\, do\{ local $/; \<F> \};
    close F;
    
    print "Via backticks\\n"\, \`$cmd\`;

    open F\, "$cmd|" or die $\!;
    print "Via pipe\\n"\, \<F>;
    close F;
}

my $t1 = async{ \&test(1); };
my $t2 = async{ \&test(2); };
my $t3 = async{ \&test(3); };
my $t4 = async{ \&test(4); };

$t1->join;
$t2->join;
$t3->join;
$t4->join;
=====Test data file follows (all 4 test files are the same,
test[1-4].dat=====
world
hello
===========Windows .bat file
tester===========================================
:LOOP
perl thread2.pl
goto LOOP

The test program runs window's "sort" in four background threads to sort
four test files. The path to Windows sort may differ on some machines
(some
have a c:/winnt directory instead of c:/windows for example). It doesn't
matter what the background program is, sort was used because it was
convenient to reproduce the problem with.

Each of the threads test the three methods of executing external binaries,
first with "system", then with backtic, and finally with popen. If system
is the only one used it won't hang, either backtic or popen will hang
if run
by themselves. Consequently, I'm currently using system as a workaround,
but it's unfortunate that it requires additional disk I/O as in my
application performance is important and disk I/O is a primary bottleneck.

--

Keith Doyle

In the course of reviewing older tickets this morning, I came across
this one.

Keith, were you ever able to resolve this problem? Was the vendor with
whom you corresponded able to respond?

Thank you very much.
Jim Keenan

p5pRT · 2013-05-27T13:12:13Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2018-08-17T02:06:09Z

From vlad.romascanu@gmail.com

Likely duplicate of Bugs 38723, 77672, fixed in v5.19.8.

p5pRT · 2018-08-17T02:06:09Z

From [Unknown Contact. See original ticket]

Likely duplicate of Bugs 38723, 77672, fixed in v5.19.8.

p5pRT · 2018-08-17T12:37:43Z

From @jkeenan

On Fri, 17 Aug 2018 02:06:09 GMT, vlad.romascanu@gmail.com wrote:

Likely duplicate of Bugs 38723, 77672, fixed in v5.19.8.

Thanks for spotting that. And since the OP hasn't responded to my 2013 request for an update, I'm closing this ticket.

Thank you very much.
--
James E Keenan (jkeenan@cpan.org)

p5pRT · 2018-08-17T12:37:45Z

@jkeenan - Status changed from 'open' to 'resolved'

p5pRT closed this as completed Aug 17, 2018

p5pRT added Severity Medium distro-mswin32 type-OS-interaction type-core type-ithreads labels Oct 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Perl hangs when when using backticks or popen from a thread on Win32 #8489

Perl hangs when when using backticks or popen from a thread on Win32 #8489

p5pRT commented Jun 19, 2006

p5pRT commented Jun 19, 2006

p5pRT commented May 27, 2013

$t1->join;
$t2->join;
$t3->join;
$t4->join;
=====Test data file follows (all 4 test files are the same,
test[1-4].dat=====
world
hello
===========Windows .bat file
tester===========================================
:LOOP
perl thread2.pl
goto LOOP

p5pRT commented May 27, 2013

p5pRT commented Aug 17, 2018

p5pRT commented Aug 17, 2018

p5pRT commented Aug 17, 2018

p5pRT commented Aug 17, 2018

Perl hangs when when using backticks or popen from a thread on Win32 #8489

Perl hangs when when using backticks or popen from a thread on Win32 #8489

Comments

p5pRT commented Jun 19, 2006

p5pRT commented Jun 19, 2006

From KDoyle@emdeon.com

$t1->join; $t2->join; $t3->join; $t4->join; =====Test data file follows (all 4 test files are the same, test[1-4].dat===== world hello ===========Windows .bat file tester=========================================== :LOOP perl thread2.pl goto LOOP

p5pRT commented May 27, 2013

From @jkeenan

$t1->join; $t2->join; $t3->join; $t4->join; =====Test data file follows (all 4 test files are the same, test[1-4].dat===== world hello ===========Windows .bat file tester=========================================== :LOOP perl thread2.pl goto LOOP

p5pRT commented May 27, 2013

p5pRT commented Aug 17, 2018

From vlad.romascanu@gmail.com

p5pRT commented Aug 17, 2018

From [Unknown Contact. See original ticket]

p5pRT commented Aug 17, 2018

From @jkeenan

p5pRT commented Aug 17, 2018

$t1->join;
$t2->join;
$t3->join;
$t4->join;
=====Test data file follows (all 4 test files are the same,
test[1-4].dat=====
world
hello
===========Windows .bat file
tester===========================================
:LOOP
perl thread2.pl
goto LOOP

$t1->join;
$t2->join;
$t3->join;
$t4->join;
=====Test data file follows (all 4 test files are the same,
test[1-4].dat=====
world
hello
===========Windows .bat file
tester===========================================
:LOOP
perl thread2.pl
goto LOOP