From johnh@isi.edu

Created by johnh@isi.edu

IO on an active pipe fails after a thread fork,
making

cat pipe_input.in | ./pipe_test.pl

behave DIFFERENTLY than

./pipe_test.pl < cat_input.in

This result appears to be because perl tries to lseek to the current
position of the stream after fork and lseek fails with EISPIPE in the
first (cat) case. I'm guessing it considers the failed lseek to make
the filehandle bad. (This speculation is from strace.)

It seems to me (not having looked at perl's guts) that there is no
need to lseek, at least for pipes, so perl should produce identical
output in this case. I.e., this difference is a bug.

This behavior is reproducable with the test case below.

The problem only happens when the file is active (i.e., one line of input
has been read before the thread fork).

----------------------------------------
#!/usr/bin/perl -w

# save as pipe_test.pl

use IO​::Handle;
use threads;

my $in = new IO​::Handle;
$in->fdopen(fileno(STDIN), "r");

# comment out the next line and it works
my($header) = $in->getline;
print "Header, always correct​: $header";

my $worker_thread = threads->new( sub {
  my $count = 0;
  my $line;
  while ($line = $in->getline) {
  print "$count​: $line";
  $count++;
  };
  return $count;
  } );

my $result = $worker_thread->join;

print "\nline count​: $result\n\n";
print "Count will incorrectly be 0 if run with cat pipe_input.in | perl pipe_test.pl,\n" .
  "or 2 if run with perl pipe_test.pl <pipe_input.in.\n" .
  "It should be the same in both cases.\n" .
  "Suspected bug​: after fork, perl lseeks on \$in, which files if\n" .
  "it is a pipe (OS​: linux 2.6.24 on Fedora 8, perl-5.8.8).\n" .
  "(Note​: it works if there is no pre-existing IO on the file,\n" .
  "i.e., if first getline is commented out.)\n";
 

exit 0;
----------------------------------------

sample input file (pipe_input.in)​:

#h header
line one
line two

----------------------------------------

sample tail of the strace -f output​:

...
read(3, "", 4096) = 0
close(3) = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfb7b9e8) = -1 EINVAL (Invalid argument)
_llseek(0, 0, 0xbfb7ba10, SEEK_CUR) = -1 ESPIPE (Illegal seek)
fstat64(0, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
fcntl64(0, F_SETFD, 0) = 0
read(0, "#h header\nline one\nline two\n", 4096) = 28
write(1, "Header, always correct​: #h heade"..., 34) = 34
_llseek(0, 10, 0xbfb7bb60, SEEK_SET) = -1 ESPIPE (Illegal seek)
mmap2(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7cfa000
brk(0x9ada000) = 0x9ada000
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfb7ba48) = -1 EINVAL (Invalid argument)
_llseek(0, 0, 0xbfb7ba70, SEEK_CUR) = -1 ESPIPE (Illegal seek)
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
_llseek(1, 0, 0xbfb7ba70, SEEK_CUR) = -1 ESPIPE (Illegal seek)
ioctl(2, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfb7ba48) = -1 ENOTTY (Inappropriate ioctl for device)
_llseek(2, 0, [98043], SEEK_CUR) = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfb7ba48) = -1 EINVAL (Invalid argument)
_llseek(0, 0, 0xbfb7ba70, SEEK_CUR) = -1 ESPIPE (Illegal seek)
brk(0x9afb000) = 0x9afb000
brk(0x9b1c000) = 0x9b1c000
brk(0x9b3d000) = 0x9b3d000
mmap2(NULL, 10489856, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb72f9000
mprotect(0xb72f9000, 4096, PROT_NONE) = 0
clone(Process 29195 attached (waiting for parent)
Process 29195 resumed (parent 29194 ready)
child_stack=0xb7cf94b4, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0xb7cf9bd8, {entry_number​:6, base_addr​:0xb7cf9b90, limit​:1048575, seg_32bit​:1, contents​:0, read_exec_only​:0, limit_in_pages​:1, seg_not_present​:0, useable​:1}, child_tidptr=0xb7cf9bd8) = 29195
[pid 29195] set_robust_list(0xb7cf9be0, 0xc) = 0
[pid 29195] read(0, "", 4096) = 0
[pid 29195] _exit(0) = ?
Process 29195 detached
munmap(0xb7cfa000, 135168) = 0
write(1, "\nline count​: 0\n\n", 16) = 16
write(1, "Count will be 0 if run with cat "..., 231) = 231
_llseek(0, 10, 0xbfb7b9e0, SEEK_SET) = -1 ESPIPE (Illegal seek)
munmap(0xb7d1b000, 135168) = 0
exit_group(0) = ?

Flags:
    category=core
    severity=medium

This perlbug was built using Perl v5.8.8 in the Red Hat build system.
It is being executed now by Perl v5.8.8 - Tue Apr 29 02:54:41 EDT 2008.

Site configuration information for perl v5.8.8:

Configured by Red Hat, Inc. at Tue Apr 29 02:54:41 EDT 2008.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.18-53.1.14.el5xen, archname=i386-linux-thread-multi
    uname='linux xenbuilder2.fedora.redhat.com 2.6.18-53.1.14.el5xen #1 smp tue feb 19 07:33:17 est 2008 i686 i686 i386 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -Dversion=5.8.8 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux -Dvendorprefix=/usr -Dsiteprefix=/usr -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dinc_version_list=5.8.7 5.8.6 5.8.5 -Dscriptdir=/usr/bin'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='4.1.2 20070925 (Red Hat 4.1.2-33)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.7.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.7'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.8:
    /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.7/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.8
    /usr/lib/perl5/site_perl/5.8.7
    /usr/lib/perl5/site_perl/5.8.6
    /usr/lib/perl5/site_perl/5.8.5
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.8
    /usr/lib/perl5/vendor_perl/5.8.7
    /usr/lib/perl5/vendor_perl/5.8.6
    /usr/lib/perl5/vendor_perl/5.8.5
    /usr/lib/perl5/vendor_perl
    /usr/lib/perl5/5.8.8/i386-linux-thread-multi
    /usr/lib/perl5/5.8.8
    .


Environment for perl v5.8.8:
    HOME=/home/johnh
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/usr/local/lib
    LOGDIR (unset)
    PATH=/etc/alternatives/java_sdk_1.6.0:/home/johnh/BIN:/home/johnh/BIN/LINUX:/usr/local/bin:/bin:/usr/bin:/home/johnh/BIN/WP:/home/johnh/BIN/HOSTS:/home/johnh/BIN/DB:/usr/X11R6/bin:/usr/local/etc:/usr/local/sbin:/etc:/usr/etc:/sbin:/usr/sbin:/usr/hosts:/usr/local/games:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/bash

From johnh@isi.edu

The test case I posted for perl-5.8.8 produces the same results on
perl-5.10.0 on Fedora 9.

  -John Heidemann

From @jkeenan

On Tue May 27 10​:52​:24 2008, johnh@​isi.edu wrote​:

This is a bug report for perl from johnh@​isi.edu,
generated with the help of perlbug 1.35 running under perl v5.8.8.

-----------------------------------------------------------------
[Please enter your report here]

IO on an active pipe fails after a thread fork,
making

cat pipe_input.in | ./pipe_test.pl

behave DIFFERENTLY than

./pipe_test.pl < cat_input.in

This result appears to be because perl tries to lseek to the current
position of the stream after fork and lseek fails with EISPIPE in the
first (cat) case. I'm guessing it considers the failed lseek to make
the filehandle bad. (This speculation is from strace.)

It seems to me (not having looked at perl's guts) that there is no
need to lseek, at least for pipes, so perl should produce identical
output in this case. I.e., this difference is a bug.

This behavior is reproducable with the test case below.

The problem only happens when the file is active (i.e., one line of
input
has been read before the thread fork).

----------------------------------------
#!/usr/bin/perl -w

# save as pipe_test.pl

use IO​::Handle;
use threads;

my $in = new IO​::Handle;
$in->fdopen(fileno(STDIN), "r");

# comment out the next line and it works
my($header) = $in->getline;
print "Header, always correct​: $header";

my $worker_thread = threads->new( sub {
my $count = 0;
my $line;
while ($line = $in->getline) {
print "$count​: $line";
$count++;
};
return $count;
} );

my $result = $worker_thread->join;

print "\nline count​: $result\n\n";
print "Count will incorrectly be 0 if run with cat pipe_input.in |
perl pipe_test.pl,\n" .
"or 2 if run with perl pipe_test.pl <pipe_input.in.\n" .
"It should be the same in both cases.\n" .
"Suspected bug​: after fork, perl lseeks on \$in, which files if\n"
.
"it is a pipe (OS​: linux 2.6.24 on Fedora 8, perl-5.8.8).\n" .
"(Note​: it works if there is no pre-existing IO on the file,\n" .
"i.e., if first getline is commented out.)\n";

exit 0;
----------------------------------------

sample input file (pipe_input.in)​:

#h header
line one
line two

----------------------------------------

sample tail of the strace -f output​:

...
read(3, "", 4096) = 0
close(3) = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfb7b9e8) = -1 EINVAL
(Invalid argument)
_llseek(0, 0, 0xbfb7ba10, SEEK_CUR) = -1 ESPIPE (Illegal seek)
fstat64(0, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
fcntl64(0, F_SETFD, 0) = 0
read(0, "#h header\nline one\nline two\n", 4096) = 28
write(1, "Header, always correct​: #h heade"..., 34) = 34
_llseek(0, 10, 0xbfb7bb60, SEEK_SET) = -1 ESPIPE (Illegal seek)
mmap2(NULL, 135168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) = 0xb7cfa000
brk(0x9ada000) = 0x9ada000
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfb7ba48) = -1 EINVAL
(Invalid argument)
_llseek(0, 0, 0xbfb7ba70, SEEK_CUR) = -1 ESPIPE (Illegal seek)
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo
...}) = 0
_llseek(1, 0, 0xbfb7ba70, SEEK_CUR) = -1 ESPIPE (Illegal seek)
ioctl(2, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfb7ba48) = -1 ENOTTY
(Inappropriate ioctl for device)
_llseek(2, 0, [98043], SEEK_CUR) = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfb7ba48) = -1 EINVAL
(Invalid argument)
_llseek(0, 0, 0xbfb7ba70, SEEK_CUR) = -1 ESPIPE (Illegal seek)
brk(0x9afb000) = 0x9afb000
brk(0x9b1c000) = 0x9b1c000
brk(0x9b3d000) = 0x9b3d000
mmap2(NULL, 10489856, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) = 0xb72f9000
mprotect(0xb72f9000, 4096, PROT_NONE) = 0
clone(Process 29195 attached (waiting for parent)
Process 29195 resumed (parent 29194 ready)
child_stack=0xb7cf94b4,

flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,

parent_tidptr=0xb7cf9bd8, {entry_number​:6, base_addr​:0xb7cf9b90,
limit​:1048575, seg_32bit​:1, contents​:0, read_exec_only​:0,
limit_in_pages​:1, seg_not_present​:0, useable​:1},
child_tidptr=0xb7cf9bd8) = 29195
[pid 29195] set_robust_list(0xb7cf9be0, 0xc) = 0
[pid 29195] read(0, "", 4096) = 0
[pid 29195] _exit(0) = ?
Process 29195 detached
munmap(0xb7cfa000, 135168) = 0
write(1, "\nline count​: 0\n\n", 16) = 16
write(1, "Count will be 0 if run with cat "..., 231) = 231
_llseek(0, 10, 0xbfb7b9e0, SEEK_SET) = -1 ESPIPE (Illegal seek)
munmap(0xb7d1b000, 135168) = 0
exit_group(0) = ?

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=medium
---
This perlbug was built using Perl v5.8.8 in the Red Hat build system.
It is being executed now by Perl v5.8.8 - Tue Apr 29 02​:54​:41 EDT
2008.

Could someone familiar with threads try this out on Perl 5.18 and make
some suggestions?

Thank you very much.
Jim Keenan

The RT System itself - Status changed from 'new' to 'open'

From @bulk88

On Sun May 26 09​:51​:31 2013, jkeenan wrote​:

Could someone familiar with threads try this out on Perl 5.18 and make
some suggestions?

Thank you very much.
Jim Keenan

Script as posted above
________________________________________________________________________
C​:\Documents and Settings\Owner\Desktop>perl pipe_test.pl < pipe_input.in
Header, always correct​: #h header
0​: line one
1​: line two
line count​: 2

Count will incorrectly be 0 if run with cat pipe_input.in | perl
pipe_test.pl,
or 2 if run with perl pipe_test.pl <pipe_input.in.
It should be the same in both cases.
Suspected bug​: after fork, perl lseeks on $in, which files if
it is a pipe (OS​: linux 2.6.24 on Fedora 8, perl-5.8.8).
(Note​: it works if there is no pre-existing IO on the file,
i.e., if first getline is commented out.)

C​:\Documents and Settings\Owner\Desktop>type pipe_input.in | perl
pipe_test.pl
Header, always correct​: #h header

line count​: 0

Count will incorrectly be 0 if run with cat pipe_input.in | perl
pipe_test.pl,
or 2 if run with perl pipe_test.pl <pipe_input.in.
It should be the same in both cases.
Suspected bug​: after fork, perl lseeks on $in, which files if
it is a pipe (OS​: linux 2.6.24 on Fedora 8, perl-5.8.8).
(Note​: it works if there is no pre-existing IO on the file,
i.e., if first getline is commented out.)

C​:\Documents and Settings\Owner\Desktop>
____________________________________________________________________

Below is with "my($header) = $in->getline;" commented out.
___________________________________________________________________
C​:\Documents and Settings\Owner\Desktop>perl pipe_test.pl < pipe_input.in
Header, always correct​: 0​: #h header
1​: line one
2​: line two
line count​: 3

Count will incorrectly be 0 if run with cat pipe_input.in | perl
pipe_test.pl,
or 2 if run with perl pipe_test.pl <pipe_input.in.
It should be the same in both cases.
Suspected bug​: after fork, perl lseeks on $in, which files if
it is a pipe (OS​: linux 2.6.24 on Fedora 8, perl-5.8.8).
(Note​: it works if there is no pre-existing IO on the file,
i.e., if first getline is commented out.)

C​:\Documents and Settings\Owner\Desktop>type pipe_input.in | perl
pipe_test.pl
Header, always correct​: 0​: #h header
1​: line one
2​: line two
line count​: 3

Count will incorrectly be 0 if run with cat pipe_input.in | perl
pipe_test.pl,
or 2 if run with perl pipe_test.pl <pipe_input.in.
It should be the same in both cases.
Suspected bug​: after fork, perl lseeks on $in, which files if
it is a pipe (OS​: linux 2.6.24 on Fedora 8, perl-5.8.8).
(Note​: it works if there is no pre-existing IO on the file,
i.e., if first getline is commented out.)

C​:\Documents and Settings\Owner\Desktop>
______________________________________________________

tested with "Summary of my perl5 (revision 5 version 19 subversion 0)"
on Win32 32 bits, VC. It is beyond my knowledge set to comment if there
is a bug in Perl or not.

--
bulk88 ~ bulk88 at hotmail.com