Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

readline of a not NL-terminated last line results in Bad file descriptor #8431

Closed
p5pRT opened this issue May 3, 2006 · 14 comments
Closed

Comments

@p5pRT
Copy link

p5pRT commented May 3, 2006

Migrated from rt.perl.org#39060 (status was 'resolved')

Searchable as RT39060$

@p5pRT
Copy link
Author

p5pRT commented May 3, 2006

From Mark.Martinec@ijs.si

Created by Mark.Martinec@ijs.si

This is a bug report for perl from Mark.Martinec@​ijs.si,
generated with the help of perlbug 1.35 running under perl v5.8.8.

-----------------------------------------------------------------
When reading line-by-line from a file where the last line is not
newline-terminated, the resulting status is "Bad file descriptor"
(EBADF).

Using the <>. readline or IO​::File​::getline behave the same,
reading form a file or stdin behaves the same.

To reproduce​:

  echo -n "test" | \
  perl -e '$!=0; while(<>){print; $!=0}; $!==0 or die "Err​: $!"'
  Err​: Bad file descriptor at -e line 1, <> line 1.
  test

Interestingly the problem occurs only on Linux, but not on
FreeBSD or NetBSD. The problem is not new with 5.8.8, the
first time I know it was reported on June 2005, Perl 5.8.3.
I'm not sure with which version of Perl it was introduced,
probably with the invention of PerlIO.

From the documentation​:
  In scalar context, evaluating a filehandle in angle brackets yields
  the next line from that file (the newline, if any, included),
  or undef at end-of-file or on error.
...
  If readline encounters an operating system error, $! will be set
  with the corresponding error message. It can be helpful to check $!
  when you are reading from filehandles you don't trust, ...

So testing for $! after a readline returing an undef is valid and is
supposed to be used. The reason why the problem is not seen more often
is that people are sloppy and neglect to test status of operations.

Perl Info

Flags:
    category=core
    severity=medium

This perlbug was built using Perl v5.8.8 in the Red Hat build system.
It is being executed now by Perl v5.8.8 - Sun Feb  5 22:59:04 CET 2006.

Site configuration information for perl v5.8.8:

Configured by cmi at Sun Feb  5 22:59:04 CET 2006.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.15-2.2.0.cmi, archname=i386-linux-thread-multi
    uname='linux alice.ijs.si 2.6.15-2.2.0.cmi #1 sun feb 5 22:42:11 cet 2006 i686 athlon i386 gnulinux '
    config_args='-des -Doptimize=-O2 -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -m32 -march=i386 -mtune=pentium4 -fasynchronous-unwind-tables -Dversion=5.8.8 -Dmyhostname=localhost -Dperladmin=Janez.Srakar@ijs.si -Dcc=gcc -Dcf_by=cmi -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux -Dvendorprefix=/usr -Dsiteprefix=/usr -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dinc_version_list=5.8.7 -Dscriptdir=/usr/bin'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-O2 -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -m32 -march=i386 -mtune=pentium4 -fasynchronous-unwind-tables',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='4.0.2 20051125 (CMI 4.0.2-8.3.cmi)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.3.5.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.3.5'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.8:
    /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.7/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.8
    /usr/lib/perl5/site_perl/5.8.7
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.8
    /usr/lib/perl5/vendor_perl/5.8.7
    /usr/lib/perl5/vendor_perl
    /usr/lib/perl5/5.8.8/i386-linux-thread-multi
    /usr/lib/perl5/5.8.8
    .


Environment for perl v5.8.8:
    HOME=/home/mark
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/java/bin:/usr/X11R6/bin:/usr/bin/X11:/bin:/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/home/mark/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2006

From @andk

On Wed, 03 May 2006 07​:37​:17 -0700, Mark Martinec (via RT) <perlbug-followup@​perl.org> said​:

blame analysis shows​:

----Program----
if (open FH, "-|") {
  $!=0;
  while(<FH>){
  $!=0
  }
  $!==0 or die "Err​: $!";
} else {
  print "test";
}

----Output of .../pBlBYbJ/perl-5.8.0@​19461/bin/perl----

----EOF ($?='0')----
----Output of .../paw8pq2/perl-5.8.0@​19462/bin/perl----
Err​: Invalid argument at tests/markm-eof.pl line 6, <FH> line 1.

----EOF ($?='5632')----

Change 19462 by jhi@​kosh on 2003/05/09 18​:53​:27

  Try to comprehensively have a plan B if a PerlIO
  layer doesn't have a particular function.
  (1) If there's a corresponding PerlIOBase, call it.
  (2) If not having the function is 'harmless', be silently happy.
  (Currently only Flush is in this category.)
  (3) Otherwise set errno and return failure.

--
andreas

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2006

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Nov 17, 2006

From Mark.Martinec@ijs.si

It gets more interesting, thanks to recent troubleshooting by Bradley M
Alexander (see amavisd-user mailing list, 2006-11-17). The "Bad file
descriptor" status can show even when lines are correctly terminated
with a NL. The problem is reproducible with Perl 5.8.8 on several
brands of Linux, and can not be reproduced on any FreeBSD box within my
reach.

New test cases (distilled from more complex real application)​:

$ echo "test" | perl -e '
  $!=0; while(&lt;&gt;){print}; $!==0 or die "Err​: $!"'
test

$ echo "test" | PERLIO="perlio" perl -e '
  $!=0; while(&lt;&gt;){print}; $!==0 or die "Err​: $!"'
test

$ echo "test" | PERLIO="stdio" perl -e '
  $!=0; while(&lt;&gt;){print}; $!==0 or die "Err​: $!"'
test
Err​: Bad file descriptor at -e line 1, <> line 1.

Unfortunately some Linux distributions seem to set PERLIO="stdio"
in the environment gratuitously, so some people stumble across
the problem, while most others do not.

  Mark

@p5pRT
Copy link
Author

p5pRT commented May 16, 2008

p5p@spam.wizbit.be - Status changed from 'open' to 'stalled'

@p5pRT
Copy link
Author

p5pRT commented Sep 23, 2008

From Mark.Martinec@ijs.si

$ echo "test" | PERLIO="stdio" perl -e '
$!=0; while(&lt;&gt;){print}; $!==0 or die "Err​: $!"'
test
Err​: Bad file descriptor at -e line 1, <> line 1.

Unfortunately some Linux distributions seem to set PERLIO="stdio"
in the environment gratuitously, so some people stumble across
the problem, while most others do not.

The bug is still present in 5.10.0!
  (apparently introduced by Change 19462, jhi@​kosh, 2003/05/09).

Its presence forces people to intentionally skip status checks
on readline, which is a bad programming practice, hindering
program maintainability/troubleshooting.

It would be very much desirable to get it finally fixed,
both for perl 5.8.8+ and for 5.10.

@p5pRT
Copy link
Author

p5pRT commented Sep 23, 2008

The RT System itself - Status changed from 'stalled' to 'open'

@p5pRT
Copy link
Author

p5pRT commented May 26, 2009

From skasal@redhat.com

The probelm has also been reported to Fedora​:
http​://bugzilla.redhat.com/221113

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2009

From skasal@redhat.com

I traced the problem to PerlIO_fast_gets​: if PERLIO_F_FASTGETS bit is
unset, the function correctly returns 0, but incorretly sets errno=EBADF.

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2009

From skasal@redhat.com

0001-fix-RT-39060-errno-incorrectly-set-in-perlio.patch
From e57cc2468d765872b20810478b94ead3906f1912 Mon Sep 17 00:00:00 2001
From: Stepan Kasal <skasal@redhat.com>
Date: Wed, 3 Jun 2009 12:03:55 +0200
Subject: [PATCH] fix RT 39060, errno incorrectly set in perlio

---
 MANIFEST     |    1 +
 perlio.c     |   12 +++++++-----
 t/io/errno.t |   26 ++++++++++++++++++++++++++
 3 files changed, 34 insertions(+), 5 deletions(-)
 create mode 100644 t/io/errno.t

diff --git a/MANIFEST b/MANIFEST
index b7c9341..be3be43 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -3899,6 +3899,7 @@ t/io/binmode.t			See if binmode() works
 t/io/crlf.t			See if :crlf works
 t/io/crlf_through.t		See if pipe passes data intact with :crlf
 t/io/dup.t			See if >& works right
+t/io/errno.t			See if $! is correctly set
 t/io/fflush.t			See if auto-flush on fork/exec/system/qx works
 t/io/fs.t			See if directory manipulations work
 t/io/inplace.t			See if inplace editing works
diff --git a/perlio.c b/perlio.c
index 0a086a8..e92a32a 100644
--- a/perlio.c
+++ b/perlio.c
@@ -1784,12 +1784,14 @@ PerlIO_has_base(PerlIO *f)
 int
 PerlIO_fast_gets(PerlIO *f)
 {
-    if (PerlIOValid(f) && (PerlIOBase(f)->flags & PERLIO_F_FASTGETS)) {
-	 const PerlIO_funcs * const tab = PerlIOBase(f)->tab;
+    if (PerlIOValid(f)) {
+	 if (PerlIOBase(f)->flags & PERLIO_F_FASTGETS) {
+	     const PerlIO_funcs * const tab = PerlIOBase(f)->tab;
 
-	 if (tab)
-	      return (tab->Set_ptrcnt != NULL);
-	 SETERRNO(EINVAL, LIB_INVARG);
+	     if (tab)
+		  return (tab->Set_ptrcnt != NULL);
+	     SETERRNO(EINVAL, LIB_INVARG);
+	 }
     }
     else
 	 SETERRNO(EBADF, SS_IVCHAN);
diff --git a/t/io/errno.t b/t/io/errno.t
new file mode 100644
index 0000000..b55e3db
--- /dev/null
+++ b/t/io/errno.t
@@ -0,0 +1,26 @@
+#!./perl
+# vim: ts=4 sts=4 sw=4:
+
+# $! may not be set if EOF was reached without any error.
+# http://rt.perl.org/rt3/Ticket/Display.html?id=39060
+
+use strict;
+require './test.pl';
+
+plan( tests => 16 );
+
+my $test_prog = 'while(<>){print}; print $!';
+
+for my $perlio ('perlio', 'stdio') {
+    $ENV{PERLIO} = $perlio;
+    for my $test_in ("test\n", "test") {
+		my $test_in_esc = $test_in;
+		$test_in_esc =~ s/\n/\\n/g;
+		for my $rs_code ('', '$/=undef', '$/=\2', '$/=\1024') {
+			is( runperl( prog => "$rs_code; $test_prog",
+						 stdin => $test_in, stderr => 1),
+				$test_in,
+				"Wrong errno, PERLIO=$ENV{PERLIO} stdin='$test_in_esc'");
+		}
+	}
+}
-- 
1.6.2

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2009

From skasal@redhat.com

On Wed Jun 03 07​:26​:55 2009, kasal wrote​:

I traced the problem to PerlIO_fast_gets​: if PERLIO_F_FASTGETS bit is
unset, the function correctly returns 0, but incorretly sets errno=EBADF.

I forgot to mention that I verified that the patch does not break the
test suite.

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2009

From skasal@redhat.com

Giving due credit​: all the test combinations are inspired by test
scripts written by Wojciech Pilorz, attached to the Fedora incarnation
of this bug, https://bugzilla.redhat.com/show_bug.cgi?id=221113

@p5pRT
Copy link
Author

p5pRT commented Jun 8, 2009

From @rgs

2009/6/3 Stepan Kasal via RT <perlbug-followup@​perl.org>​:

I traced the problem to PerlIO_fast_gets​: if PERLIO_F_FASTGETS bit is
unset, the function correctly returns 0, but incorretly sets errno=EBADF.

Thanks, applied as d7dfc38

@p5pRT
Copy link
Author

p5pRT commented Jun 8, 2009

@rgs - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant