IO::Handle setvbuf bug #3523

p5pRT · 2001-03-05T16:00:00Z

Migrated from rt.perl.org#5971 (status was 'open')

Searchable as RT5971$

p5pRT · 2001-03-05T16:00:00Z

From me-01@ton.iguana.be

I Was doing some straces of perl's output:

strace perl -wle 'print STDERR "hoi", "wow"'

gives(as expected):

write(2, "hoi", 3) = 3
write(2, "wow", 3) = 3
write(2, "\n", 1

Because I needed lines in my application, I wanted line buffering, so I
tried:

strace perl -wle 'use IO::Handle '_IOLBF'; STDERR->setvbuf($buffer_var, _IOLBF, 1024); print STDERR "hoidyhu", "wowgrrr"'

which gives (totally unexpected):

write(2, "h", 1) = 1
write(2, "oidyhu", 6) = 6
write(2, "w", 1) = 1
write(2, "owgrrr", 6) = 6
write(2, "\n", 1

However, this:

strace perl -wle 'use IO::Handle '_IOLBF'; STDERR->setvbuf(my $buffer_var, _IOLBF, 1024); print STDERR "hoidyhu", "wowgrrr"'

gives the expected:

write(2, "hoidyhuwowgrrr\n", 15) = 15

I think there's something very wrong if it matters if the buffer
argument is a global or a lexical variable. Things like that make
me worry about what happens in memory.

Perl Info


Flags:
    category=library
    severity=medium

Site configuration information for perl v5.6.0:

Configured by ton at Wed Aug  9 23:24:07 CEST 2000.

Summary of my perl5 (revision 5.0 version 6 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.4.0-test5, archname=i686-linux
    uname='linux quasar 2.4.0-test5 #2 sat jul 29 04:45:14 cest 2000 i686 unknown '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=undef d_sfio=undef uselargefiles=undef 
    use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=undef
  Compiler:
    cc='cc', optimize='-O2 -fomit-frame-pointer', gccversion=egcs-2.91.66 19990314 (egcs-1.1.2 release), gccosandvers=
    cppflags='-fno-strict-aliasing -I/usr/local/include'
    ccflags ='-fno-strict-aliasing -I/usr/local/include'
    stdchar='char', d_stdstdio=define, usevfork=false
    intsize=4, longsize=4, ptrsize=4, doublesize=8
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=4
    alignbytes=4, usemymalloc=y, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib -L/opt/gnu/lib'
    libpth=/usr/local/lib /opt/gnu/lib /lib /usr/lib
    libs=-lnsl -lndbm -ldl -lm -lc -lposix -lcrypt -lutil
    libc=/lib/libc-2.1.1.so, so=so, useshrplib=true, libperl=libperl.so
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5/5.6.0/i686-linux/CORE'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib -L/opt/gnu/lib'

Locally applied patches:
    


@INC for perl v5.6.0:
    /usr/lib/perl5/5.6.0/i686-linux
    /usr/lib/perl5/5.6.0
    /usr/lib/perl5/site_perl/5.6.0/i686-linux
    /usr/lib/perl5/site_perl/5.6.0
    /usr/lib/perl5/site_perl
    .


Environment for perl v5.6.0:
    HOME=/home/ton
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/ton/bin.Linux:/home/ton/bin:/home/ton/bin.SampleSetup:/usr/local/bin:/usr/local/sbin:/usr/local/jre/bin:/usr/local/jdk116_v5/bin:/home/oracle/product/8.1.5/bin:/usr/games/bin:/usr/X11R6/bin:/usr/share/bin:/usr/bin:/usr/sbin:/bin:/sbin:.
    PERL_BADLANG (unset)
    SHELL=/bin/bash

p5pRT · 2010-07-29T03:27:39Z

From @gannett-ggreer

This isn't "PerlIO" per se because the documentation notes:

- - - 8< - - - 8< - - -
WARNING: The IO::Handle::setvbuf() is not available by default on Perls
5.8.0 and later because setvbuf() is rather specific to using the stdio
library, while Perl prefers the new perlio subsystem instead.
- - - 8< - - - 8< - - -

Indeed on Linux I get:
IO::Handle::setvbuf not implemented on this architecture at -e line 1.

--
George Greer

p5pRT · 2012-05-05T11:59:01Z

From @Hugmeir

On Wed Jul 28 20:27:39 2010, greerga wrote:

This isn't "PerlIO" per se because the documentation notes:

- - - 8< - - - 8< - - -
WARNING: The IO::Handle::setvbuf() is not available by default on Perls
5.8.0 and later because setvbuf() is rather specific to using the stdio
library, while Perl prefers the new perlio subsystem instead.
- - - 8< - - - 8< - - -

Indeed on Linux I get:
IO::Handle::setvbuf not implemented on this architecture at -e line 1.

How would I need to compile Perl to test if this is still present?

p5pRT · 2012-05-07T23:54:51Z

From @tonycoz

On Sat, May 05, 2012 at 04:59:02AM -0700, Brian Fraser via RT wrote:

On Wed Jul 28 20:27:39 2010, greerga wrote:

This isn't "PerlIO" per se because the documentation notes:

- - - 8< - - - 8< - - -
WARNING: The IO::Handle::setvbuf() is not available by default on Perls
5.8.0 and later because setvbuf() is rather specific to using the stdio
library, while Perl prefers the new perlio subsystem instead.
- - - 8< - - - 8< - - -

Indeed on Linux I get:
IO::Handle::setvbuf not implemented on this architecture at -e line 1.

How would I need to compile Perl to test if this is still present?

You'd have to build perl without perlio, but Configure now always
enables perlio.

Tony

p5pRT · 2012-05-08T13:52:25Z

From @nwc10

On Tue, May 08, 2012 at 09:54:20AM +1000, Tony Cook wrote:

On Sat, May 05, 2012 at 04:59:02AM -0700, Brian Fraser via RT wrote:

On Wed Jul 28 20:27:39 2010, greerga wrote:

This isn't "PerlIO" per se because the documentation notes:

- - - 8< - - - 8< - - -
WARNING: The IO::Handle::setvbuf() is not available by default on Perls
5.8.0 and later because setvbuf() is rather specific to using the stdio
library, while Perl prefers the new perlio subsystem instead.
- - - 8< - - - 8< - - -

Indeed on Linux I get:
IO::Handle::setvbuf not implemented on this architecture at -e line 1.

How would I need to compile Perl to test if this is still present?

You'd have to build perl without perlio, but Configure now always
enables perlio.

"Now" as in post 5.15.0

If I build 5.14.0 like this:

./Configure -de -Uuseperlio && make -j5 test_prep

I see the same confusing difference:

$ strace -oout -ewrite ./perl -Ilib -wle 'use IO::Handle '_IOLBF'; STDERR->setvbuf(my $buffer_var, _IOLBF, 1024); print STDERR "hoidyhu", "wowgrrr"'
hoidyhuwowgrrr
$ cat out
write(2, "hoidyhuwowgrrr\n", 15) = 15

$ strace -oout -ewrite ./perl -Ilib -wle 'use IO::Handle '_IOLBF'; STDERR->setvbuf($buffer_var, _IOLBF, 1024); print STDERR "hoidyhu", "wowgrrr"'
Name "main::buffer_var" used only once: possible typo at -e line 1.
hoidyhuwowgrrr
$ cat out
write(2, "Name \"main::buffer_var\" used onl"..., 68) = 68
write(2, "h", 1) = 1
write(2, "oidyhu", 6) = 6
write(2, "w", 1) = 1
write(2, "owgrrr", 6) = 6
write(2, "\n", 1) = 1

Nicholas Clark

p5pRT · 2012-05-08T15:03:57Z

From @nwc10

On Tue, May 08, 2012 at 02:51:51PM +0100, Nicholas Clark wrote:

On Tue, May 08, 2012 at 09:54:20AM +1000, Tony Cook wrote:

On Sat, May 05, 2012 at 04:59:02AM -0700, Brian Fraser via RT wrote:

On Wed Jul 28 20:27:39 2010, greerga wrote:

This isn't "PerlIO" per se because the documentation notes:

- - - 8< - - - 8< - - -
WARNING: The IO::Handle::setvbuf() is not available by default on Perls
5.8.0 and later because setvbuf() is rather specific to using the stdio
library, while Perl prefers the new perlio subsystem instead.
- - - 8< - - - 8< - - -

Indeed on Linux I get:
IO::Handle::setvbuf not implemented on this architecture at -e line 1.

How would I need to compile Perl to test if this is still present?

You'd have to build perl without perlio, but Configure now always
enables perlio.

"Now" as in post 5.15.0

If I build 5.14.0 like this:

./Configure -de -Uuseperlio && make -j5 test_prep

I see the same confusing difference:

$ strace -oout -ewrite ./perl -Ilib -wle 'use IO::Handle '_IOLBF'; STDERR->setvbuf(my $buffer_var, _IOLBF, 1024); print STDERR "hoidyhu", "wowgrrr"'
hoidyhuwowgrrr
$ cat out
write(2, "hoidyhuwowgrrr\n", 15) = 15

$ strace -oout -ewrite ./perl -Ilib -wle 'use IO::Handle '_IOLBF'; STDERR->setvbuf($buffer_var, _IOLBF, 1024); print STDERR "hoidyhu", "wowgrrr"'
Name "main::buffer_var" used only once: possible typo at -e line 1.
hoidyhuwowgrrr
$ cat out
write(2, "Name \"main::buffer_var\" used onl"..., 68) = 68
write(2, "h", 1) = 1
write(2, "oidyhu", 6) = 6
write(2, "w", 1) = 1
write(2, "owgrrr", 6) = 6
write(2, "\n", 1) = 1

Ho ho ho.

$ strace -ofixed -ewrite ./perl -Ilib -wle 'use IO::Handle '_IOLBF'; STDERR->setvbuf($buffer_var, _IOLBF, 1024); print STDERR "hoidyhu", "wowgrrr"; 0 if $buffer_var;'
hoidyhuwowgrrr
$ cat fixed
write(2, "hoidyhuwowgrrr\n", 15) = 15

From the C library man page:

Except for unbuffered files, the buf argument should point to a buffer
at least size bytes long; this buffer will be used instead of the cur-
rent buffer. If the argument buf is NULL, only the mode is affected; a
new buffer will be allocated on the next read or write operation. The
setvbuf() function may only be used after opening a stream and before
any other operations have been performed on it.

What's going wrong in the test case is that the used only once warning is
causing output to STDERR before setvbuf() is called, which violates the
calling constraints.

IO::Handle's documentation says.

You should only change the buffer before any I/O, or immediately after
calling flush.

This actually seems to be erroneous, as calling flush on the handle *doesn't*
make setvbuf "work".

$ strace -obug -ewrite ./perl -Ilib -wle 'use IO::Handle '_IOLBF'; STDERR->flush or die; STDERR->setvbuf($buffer_var, _IOLBF, 1024); print STDERR "hoidyhu", "wowgrrr";'
Name "main::buffer_var" used only once: possible typo at -e line 1.
hoidyhuwowgrrr
$ cat bug
write(2, "Name \"main::buffer_var\" used onl"..., 68) = 68
write(2, "h", 1) = 1
write(2, "oidyhu", 6) = 6
write(2, "w", 1) = 1
write(2, "owgrrr", 6) = 6
write(2, "\n", 1) = 1

Looks like a fix to IO::Handle's documentation is needed.

Also, I guess that we should take this part out of its synopsis:

# setvbuf is not available by default on Perls 5.8.0 and later.
use IO::Handle '_IOLBF';
$io->setvbuf($buffer_var, _IOLBF, 1024);

as it's really not useful any more.

Nicholas Clark

Leont · 2020-03-04T23:27:51Z

I think we can close this as WONTFIX?

tonycoz · 2020-03-09T23:54:40Z

Yes, the setvbuf() being unavailable is documented so I think it's closable (and closed).

p5pRT added Severity Medium distro-Linux type-PerlIO type-library labels Oct 18, 2019

Leont added the Closable? We might be able to close this ticket, but we need to check with the reporter label Mar 4, 2020

tonycoz closed this as completed Mar 9, 2020

jkeenan removed the Closable? We might be able to close this ticket, but we need to check with the reporter label Mar 23, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IO::Handle setvbuf bug #3523

IO::Handle setvbuf bug #3523

p5pRT commented Mar 5, 2001

p5pRT commented Mar 5, 2001

p5pRT commented Jul 29, 2010

p5pRT commented May 5, 2012

p5pRT commented May 7, 2012

p5pRT commented May 8, 2012

p5pRT commented May 8, 2012

Leont commented Mar 4, 2020

tonycoz commented Mar 9, 2020

IO::Handle setvbuf bug #3523

IO::Handle setvbuf bug #3523

Comments

p5pRT commented Mar 5, 2001

p5pRT commented Mar 5, 2001

From me-01@ton.iguana.be

p5pRT commented Jul 29, 2010

From @gannett-ggreer

p5pRT commented May 5, 2012

From @Hugmeir

p5pRT commented May 7, 2012

From @tonycoz

p5pRT commented May 8, 2012

From @nwc10

p5pRT commented May 8, 2012

From @nwc10

Leont commented Mar 4, 2020

tonycoz commented Mar 9, 2020