From @dmcbride

Created by @dmcbride

(Originally found on AIX with 5.10.1, but can be reproduced in 5.16.0
on Linux.)

The following code has two surprises.

a) the layers for STDOUT are different for input vs output.
b) the layers for STDOUT's output are missing the utf8 layer after
  the reopening. (They're missing for $fh, too.)

(Thanks to #perl for pointing me to PerlIO and that the layers can
be different, and to leont on #p5p for pointing me to re-opening issues.)

---

#!/bin/env perl5.16.0

use strict;
use warnings;
use 5.10.1;

say "Testing on $^V​:";

binmode STDOUT, '​:utf8';

say "Layers before​:";
say " $_" for PerlIO​::get_layers(*STDOUT);
say " --------";
say " $_" for PerlIO​::get_layers(*STDOUT, output => 1);
'utf8' ~~ [ PerlIO​::get_layers(*STDOUT, output => 1) ] or die "Huh?";

open my $fh, '>&STDOUT' or die "can't dup"; open STDOUT, '>&', $fh or die "can't dup2";

say "Layers after​:";
say " $_" for PerlIO​::get_layers(*STDOUT);
say " --------";
say " $_" for PerlIO​::get_layers(*STDOUT, output => 1);
'utf8' ~~ [ PerlIO​::get_layers(*STDOUT, output => 1) ] or die "Huh2?";

---

Output​:

$ perl5.16.0 ./x.pl
Testing on v5.16.0​:
Layers before​:
  unix
  perlio
  utf8
--------
  unix
  perlio
  utf8
Layers after​:
  unix
  perlio
  utf8
--------
  unix
  perlio

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.16.0:

Configured by nobody at Wed Jun 13 14:08:00 MDT 2012.

Summary of my perl5 (revision 5 version 16 subversion 0) configuration:
   
  Platform:
    osname=linux, osvers=3.2.12-gentoo, archname=x86_64-linux-thread-multi
    uname='linux naboo 3.2.12-gentoo #2 smp sun jun 3 01:06:44 mdt 2012 x86_64 intel(r) core(tm) i7 cpu 930 @ 2.80ghz genuineintel gnulinux '
    config_args='-desr -Duse64bitall -Dusethreads -Dprefix=/opt/myperl/5.16.0'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector'
    ccversion='', gccversion='4.5.3', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /usr/lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.14.1.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.14.1'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Locally applied patches:
    


@INC for perl 5.16.0:
    /opt/myperl/5.16.0/lib/site_perl/5.16.0/x86_64-linux-thread-multi
    /opt/myperl/5.16.0/lib/site_perl/5.16.0
    /opt/myperl/5.16.0/lib/5.16.0/x86_64-linux-thread-multi
    /opt/myperl/5.16.0/lib/5.16.0
    .


Environment for perl 5.16.0:
    HOME=/home/dmcbride
    LANG=en_US.utf8
    LANGUAGE=
    LC_ALL=en_US.utf8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/bin:/share/cvs/work:/home/dmcbride/bin:/usr/lib/distcc/bin:/usr/bin:/bin:/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.5.3:/usr/x86_64-pc-linux-gnu/i686-pc-linux-gnu/gcc-bin/4.5.3:/share/cvs/bin:/usr/games/bin:/share/bin:/share/darin/bin:/share/cvs/work/shared
    PERL_BADLANG (unset)
    SHELL=/bin/bash

From ambrus@math.bme.hu

On 7/4/12, Darin McBride <perlbug-followup@​perl.org> wrote​:

# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=113982 >

b) the layers for STDOUT's output are missing the utf8 layer after
the reopening. (They're missing for $fh, too.)

I believe this might be the intended semantics.

If you duplicate a filehandle with the "&" or "&=" open mode, that
necessarily means to duplicate the filehandle structures, such as the
output buffer. The two handles share only the underlying file
description or file descriptor respectively.

If you wanted two handles that share their internal state, you
wouldn't dup, you'd just copy a reference. Eg.

$ perl -we 'open F, ">&=", STDOUT or die; for (1..10) { print F "foo";
print STDOUT "bar"; } print F "\n"; print STDOUT "\n";'
foofoofoofoofoofoofoofoofoofoo
barbarbarbarbarbarbarbarbarbar
$ perl -we '*F = *STDOUT{IO}; for (1..10) { print F "foo"; print
STDOUT "bar"; } print F "\n"; print STDOUT "\n";'
$

(Remark. I imagine there are cases when you may need to share a lower
layer but have separate output layer in dupes of filehandles. For
example, you may want to have two handles with different :encodings
that share an underlying ssl layer. But I believe perlio doesn't
support that natively​: you'd need a special perlio layer that forwards
data from one handle to another instead.)

Btw, report [perl #34595] is somewhat similar to this, but I believe
they're not actually the same.

Ambrus

The RT System itself - Status changed from 'new' to 'open'

From @dmcbride

On Wednesday July 4 2012 12​:28​:41 PM you wrote​:

If you wanted two handles that share their internal state, you
wouldn't dup, you'd just copy a reference. Eg.

Here's the crux of the problem - there are many times where you can't simply
copy a reference. STD* are probably the primary cases of this​: if you need to
redirect them, you will (eventually) need to redirect them back. This is so
the filehandles stay opened to the correct terminal when you're done.

Some modules do this somewhat hidden, others do it right (i.e., fork first -
redirecting back then becomes implicit by the OS when you exit).

But I do think this is surprising. Especially since this is the example in
perlfunc under "open" :-/

From @Leont

On Wed Jul 04 07​:59​:23 2012, dmcbride@​cpan.org wrote​:

The following code has two surprises.

a) the layers for STDOUT are different for input vs output.
b) the layers for STDOUT's output are missing the utf8 layer after
the reopening. (They're missing for $fh, too.)

(Thanks to #perl for pointing me to PerlIO and that the layers can
be different, and to leont on #p5p for pointing me to re-opening
issues.)

I found this lovely explanation in do_openn (doio.c​:597)​:

  /* Eeek - FIXME !!!
  * If this is a standard handle we discard all the layer stuff
  * and just dup the fd into whatever was on the handle before !
  */

This has been there since 2001 (e99cca9) :-(

Leon