use open doesn't affect magic diamond operator #15311

p5pRT · 2016-05-07T20:06:13Z

Migrated from rt.perl.org#128096 (status was 'open')

Searchable as RT128096$

p5pRT · 2016-05-07T20:06:13Z

From ambs@zbr.pt

Created by ambs@cpan.org

In perl 5.22.0,
use open ':encoding(utf-8)';

does not affect the "filehandle" ARGV.

Perl Info


Flags:
    category=core
    severity=medium

Summary of my perl5 (revision 5 version 22 subversion 0) configuration:

  Platform:
    osname=darwin, osvers=14.1.0, archname=darwin-2level
    uname='darwin stravinski.home 14.1.0 darwin kernel version 14.1.0: thu feb 26 19:26:47 pst 2015; root:xnu-2782.10.73~1release_x86_64 x86_64 '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include',
    optimize='-O3',
    cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion='', gccversion='4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.1.0/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib
    libs=-lpthread -lgdbm -ldbm -ldb -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=, so=dylib, useshrplib=false, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl):
  Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
                        PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
                        USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
                        USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
                        USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
                        USE_PERL_ATOF
  Built under darwin
  Compiled at Jun  6 2015 17:38:31
  %ENV:
    PERL_AUTOINSTALL_PREFER_CPAN="1"
    PERL_CPANM_OPT="-S"
  @INC:
    /opt/perl/lib/site_perl/5.22.0/darwin-2level
    /opt/perl/lib/site_perl/5.22.0
    /opt/perl/lib/5.22.0/darwin-2level
    /opt/perl/lib/5.22.0
    .
Summary of my perl5 (revision 5 version 22 subversion 0) configuration:

  Platform:
    osname=darwin, osvers=14.1.0, archname=darwin-2level
    uname='darwin stravinski.home 14.1.0 darwin kernel version 14.1.0: thu feb 26 19:26:47 pst 2015; root:xnu-2782.10.73~1release_x86_64 x86_64 '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include',
    optimize='-O3',
    cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion='', gccversion='4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.1.0/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib
    libs=-lpthread -lgdbm -ldbm -ldb -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=, so=dylib, useshrplib=false, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl):
  Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
                        PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
                        USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
                        USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
                        USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
                        USE_PERL_ATOF
  Built under darwin
  Compiled at Jun  6 2015 17:38:31
  %ENV:
    PERL_AUTOINSTALL_PREFER_CPAN="1"
    PERL_CPANM_OPT="-S"
  @INC:
    /opt/perl/lib/site_perl/5.22.0/darwin-2level
    /opt/perl/lib/site_perl/5.22.0
    /opt/perl/lib/5.22.0/darwin-2level
    /opt/perl/lib/5.22.0
    .
Summary of my perl5 (revision 5 version 22 subversion 0) configuration:

  Platform:
    osname=darwin, osvers=14.1.0, archname=darwin-2level
    uname='darwin stravinski.home 14.1.0 darwin kernel version 14.1.0: thu feb 26 19:26:47 pst 2015; root:xnu-2782.10.73~1release_x86_64 x86_64 '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include',
    optimize='-O3',
    cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion='', gccversion='4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.1.0/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib
    libs=-lpthread -lgdbm -ldbm -ldb -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=, so=dylib, useshrplib=false, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl):
  Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
                        PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
                        USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
                        USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
                        USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
                        USE_PERL_ATOF
  Built under darwin
  Compiled at Jun  6 2015 17:38:31
  %ENV:
    PERL_AUTOINSTALL_PREFER_CPAN="1"
    PERL_CPANM_OPT="-S"
  @INC:
    /opt/perl/lib/site_perl/5.22.0/darwin-2level
    /opt/perl/lib/site_perl/5.22.0
    /opt/perl/lib/5.22.0/darwin-2level
    /opt/perl/lib/5.22.0
    .

Configured by ambs at Wed Jan  7 16:50:26 WET 2015.

Summary of my perl5 (revision 5 version 20 subversion 1) configuration:
   
  Platform:
    osname=linux, osvers=3.2.0-4-amd64, archname=x86_64-linux
    uname='linux per-fide 3.2.0-4-amd64 #1 smp debian 3.2.63-2+deb7u2 x86_64 gnulinux '
    config_args='-ds -e -Dprefix=/opt/perl-5.20.1/'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.7.2', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.7/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lnsl -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.13.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.13'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'



@INC for perl 5.20.1:
    /opt/perl-5.20.1/lib/site_perl/5.20.1/x86_64-linux
    /opt/perl-5.20.1/lib/site_perl/5.20.1
    /opt/perl-5.20.1/lib/5.20.1/x86_64-linux
    /opt/perl-5.20.1/lib/5.20.1
    .


Environment for perl 5.20.1:
    HOME=/home/ambs
    LANG=en_US.UTF-8
    LANGUAGE=en_US:en
    LC_COLLATE=pt_PT.UTF-8
    LC_CTYPE=pt_PT.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/opt/jre1.8.0_51/bin/:/opt/vim74/bin:/opt/cwb-3.0.0/bin/:/opt/perl-5.20.1/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/ambs/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

p5pRT · 2016-05-07T20:14:28Z

From ambs@zbr.pt

Sorry, probably I pushed the trigger too fast, but:

while (<>) { say } # seems to print everything good (byte wise).

///

use open ":encoding(utf-8)";
while (<>) { say } # prints something strange
# file -i says
# text/plain; charset=unknown-8bit

///

binmode STDOUT, ":encoding(utf-8)";
while (<>) { say } # double encoding

So, I would say there *is* some bug...

On 07/05/16 21:06, ambs@cpan.org wrote:

# New Ticket Created by ambs@cpan.org
# Please include the string: [perl #128096]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=128096 >

This is a bug report for perl from ambs@cpan.org,
generated with the help of perlbug 1.40 running under perl 5.20.1.

-----------------------------------------------------------------
[Please describe your issue here]

In perl 5.22.0,
use open ':encoding(utf-8)';

does not affect the "filehandle" ARGV.

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
category=core
severity=medium
---
Summary of my perl5 (revision 5 version 22 subversion 0) configuration:

Platform:
osname=darwin, osvers=14.1.0, archname=darwin-2level
uname='darwin stravinski.home 14.1.0 darwin kernel version 14.1.0: thu feb 26 19:26:47 pst 2015; root:xnu-2782.10.73~1release_x86_64 x86_64 '
config_args=''
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include',
optimize='-O3',
cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion='', gccversion='4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.1.0/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib
libs=-lpthread -lgdbm -ldbm -ldb -ldl -lm -lutil -lc
perllibs=-lpthread -ldl -lm -lutil -lc
libc=, so=dylib, useshrplib=false, libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
USE_PERL_ATOF
Built under darwin
Compiled at Jun 6 2015 17:38:31
%ENV:
PERL_AUTOINSTALL_PREFER_CPAN="1"
PERL_CPANM_OPT="-S"
@INC:
/opt/perl/lib/site_perl/5.22.0/darwin-2level
/opt/perl/lib/site_perl/5.22.0
/opt/perl/lib/5.22.0/darwin-2level
/opt/perl/lib/5.22.0
.
Summary of my perl5 (revision 5 version 22 subversion 0) configuration:

Platform:
osname=darwin, osvers=14.1.0, archname=darwin-2level
uname='darwin stravinski.home 14.1.0 darwin kernel version 14.1.0: thu feb 26 19:26:47 pst 2015; root:xnu-2782.10.73~1release_x86_64 x86_64 '
config_args=''
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include',
optimize='-O3',
cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion='', gccversion='4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.1.0/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib
libs=-lpthread -lgdbm -ldbm -ldb -ldl -lm -lutil -lc
perllibs=-lpthread -ldl -lm -lutil -lc
libc=, so=dylib, useshrplib=false, libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
USE_PERL_ATOF
Built under darwin
Compiled at Jun 6 2015 17:38:31
%ENV:
PERL_AUTOINSTALL_PREFER_CPAN="1"
PERL_CPANM_OPT="-S"
@INC:
/opt/perl/lib/site_perl/5.22.0/darwin-2level
/opt/perl/lib/site_perl/5.22.0
/opt/perl/lib/5.22.0/darwin-2level
/opt/perl/lib/5.22.0
.
Summary of my perl5 (revision 5 version 22 subversion 0) configuration:

Platform:
osname=darwin, osvers=14.1.0, archname=darwin-2level
uname='darwin stravinski.home 14.1.0 darwin kernel version 14.1.0: thu feb 26 19:26:47 pst 2015; root:xnu-2782.10.73~1release_x86_64 x86_64 '
config_args=''
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include',
optimize='-O3',
cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion='', gccversion='4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.1.0/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib
libs=-lpthread -lgdbm -ldbm -ldb -ldl -lm -lutil -lc
perllibs=-lpthread -ldl -lm -lutil -lc
libc=, so=dylib, useshrplib=false, libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
USE_PERL_ATOF
Built under darwin
Compiled at Jun 6 2015 17:38:31
%ENV:
PERL_AUTOINSTALL_PREFER_CPAN="1"
PERL_CPANM_OPT="-S"
@INC:
/opt/perl/lib/site_perl/5.22.0/darwin-2level
/opt/perl/lib/site_perl/5.22.0
/opt/perl/lib/5.22.0/darwin-2level
/opt/perl/lib/5.22.0
.

Configured by ambs at Wed Jan 7 16:50:26 WET 2015.

Summary of my perl5 (revision 5 version 20 subversion 1) configuration:

Platform:
osname=linux, osvers=3.2.0-4-amd64, archname=x86_64-linux
uname='linux per-fide 3.2.0-4-amd64 #1 smp debian 3.2.63-2+deb7u2 x86_64 gnulinux '
config_args='-ds -e -Dprefix=/opt/perl-5.20.1/'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.7.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.7/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.13.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.13'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

---
@INC for perl 5.20.1:
/opt/perl-5.20.1/lib/site_perl/5.20.1/x86_64-linux
/opt/perl-5.20.1/lib/site_perl/5.20.1
/opt/perl-5.20.1/lib/5.20.1/x86_64-linux
/opt/perl-5.20.1/lib/5.20.1
.

---
Environment for perl 5.20.1:
HOME=/home/ambs
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_COLLATE=pt_PT.UTF-8
LC_CTYPE=pt_PT.UTF-8
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/opt/jre1.8.0_51/bin/:/opt/vim74/bin:/opt/cwb-3.0.0/bin/:/opt/perl-5.20.1/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/ambs/bin
PERL_BADLANG (unset)
SHELL=/bin/bash

p5pRT · 2016-05-07T20:14:28Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2016-05-09T02:31:13Z

From ambs@zbr.pt

On 07/05/16 21:14, Alberto Simoes via RT wrote:

Sorry, probably I pushed the trigger too fast, but:

while (<>) { say } # seems to print everything good (byte wise).

///

use open ":encoding(utf-8)";
while (<>) { say } # prints something strange
# file -i says
# text/plain; charset=unknown-8bit

///

binmode STDOUT, ":encoding(utf-8)";
while (<>) { say } # double encoding

Extra one:

use open ":encoding(utf-8)";
binmode STDOUT, ":encoding(utf-8)";

this combo outputs utf-8 correctly.

So, I would say there *is* some bug...

On 07/05/16 21:06, ambs@cpan.org wrote:

# New Ticket Created by ambs@cpan.org
# Please include the string: [perl #128096]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=128096 >

This is a bug report for perl from ambs@cpan.org,
generated with the help of perlbug 1.40 running under perl 5.20.1.

-----------------------------------------------------------------
[Please describe your issue here]

In perl 5.22.0,
use open ':encoding(utf-8)';

does not affect the "filehandle" ARGV.

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
category=core
severity=medium
---
Summary of my perl5 (revision 5 version 22 subversion 0) configuration:

Platform:
osname=darwin, osvers=14.1.0, archname=darwin-2level
uname='darwin stravinski.home 14.1.0 darwin kernel version 14.1.0: thu feb 26 19:26:47 pst 2015; root:xnu-2782.10.73~1release_x86_64 x86_64 '
config_args=''
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include',
optimize='-O3',
cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion='', gccversion='4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.1.0/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib
libs=-lpthread -lgdbm -ldbm -ldb -ldl -lm -lutil -lc
perllibs=-lpthread -ldl -lm -lutil -lc
libc=, so=dylib, useshrplib=false, libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
USE_PERL_ATOF
Built under darwin
Compiled at Jun 6 2015 17:38:31
%ENV:
PERL_AUTOINSTALL_PREFER_CPAN="1"
PERL_CPANM_OPT="-S"
@INC:
/opt/perl/lib/site_perl/5.22.0/darwin-2level
/opt/perl/lib/site_perl/5.22.0
/opt/perl/lib/5.22.0/darwin-2level
/opt/perl/lib/5.22.0
.
Summary of my perl5 (revision 5 version 22 subversion 0) configuration:

Platform:
osname=darwin, osvers=14.1.0, archname=darwin-2level
uname='darwin stravinski.home 14.1.0 darwin kernel version 14.1.0: thu feb 26 19:26:47 pst 2015; root:xnu-2782.10.73~1release_x86_64 x86_64 '
config_args=''
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include',
optimize='-O3',
cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion='', gccversion='4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.1.0/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib
libs=-lpthread -lgdbm -ldbm -ldb -ldl -lm -lutil -lc
perllibs=-lpthread -ldl -lm -lutil -lc
libc=, so=dylib, useshrplib=false, libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
USE_PERL_ATOF
Built under darwin
Compiled at Jun 6 2015 17:38:31
%ENV:
PERL_AUTOINSTALL_PREFER_CPAN="1"
PERL_CPANM_OPT="-S"
@INC:
/opt/perl/lib/site_perl/5.22.0/darwin-2level
/opt/perl/lib/site_perl/5.22.0
/opt/perl/lib/5.22.0/darwin-2level
/opt/perl/lib/5.22.0
.
Summary of my perl5 (revision 5 version 22 subversion 0) configuration:

Platform:
osname=darwin, osvers=14.1.0, archname=darwin-2level
uname='darwin stravinski.home 14.1.0 darwin kernel version 14.1.0: thu feb 26 19:26:47 pst 2015; root:xnu-2782.10.73~1release_x86_64 x86_64 '
config_args=''
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include',
optimize='-O3',
cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion='', gccversion='4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /Library/Developer/CommandLineTools/usr/bin/../lib/clang/6.1.0/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib
libs=-lpthread -lgdbm -ldbm -ldb -ldl -lm -lutil -lc
perllibs=-lpthread -ldl -lm -lutil -lc
libc=, so=dylib, useshrplib=false, libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
USE_PERL_ATOF
Built under darwin
Compiled at Jun 6 2015 17:38:31
%ENV:
PERL_AUTOINSTALL_PREFER_CPAN="1"
PERL_CPANM_OPT="-S"
@INC:
/opt/perl/lib/site_perl/5.22.0/darwin-2level
/opt/perl/lib/site_perl/5.22.0
/opt/perl/lib/5.22.0/darwin-2level
/opt/perl/lib/5.22.0
.

Configured by ambs at Wed Jan 7 16:50:26 WET 2015.

Summary of my perl5 (revision 5 version 20 subversion 1) configuration:

Platform:
osname=linux, osvers=3.2.0-4-amd64, archname=x86_64-linux
uname='linux per-fide 3.2.0-4-amd64 #1 smp debian 3.2.63-2+deb7u2 x86_64 gnulinux '
config_args='-ds -e -Dprefix=/opt/perl-5.20.1/'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.7.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.7/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.13.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.13'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

---
@INC for perl 5.20.1:
/opt/perl-5.20.1/lib/site_perl/5.20.1/x86_64-linux
/opt/perl-5.20.1/lib/site_perl/5.20.1
/opt/perl-5.20.1/lib/5.20.1/x86_64-linux
/opt/perl-5.20.1/lib/5.20.1
.

---
Environment for perl 5.20.1:
HOME=/home/ambs
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_COLLATE=pt_PT.UTF-8
LC_CTYPE=pt_PT.UTF-8
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/opt/jre1.8.0_51/bin/:/opt/vim74/bin:/opt/cwb-3.0.0/bin/:/opt/perl-5.20.1/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/ambs/bin
PERL_BADLANG (unset)
SHELL=/bin/bash

p5pRT · 2016-05-09T14:27:29Z

From @ikegami

On Sat, May 7, 2016 at 4:16 PM, Alberto Simoes <ambs@perl-hackers.net>
wrote:

On 07/05/16 21:14, Alberto Simoes via RT wrote:

Sorry, probably I pushed the trigger too fast, but:

while (<>) { say } # seems to print everything good (byte wise).

///

use open ":encoding(utf-8)";
while (<>) { say } # prints something strange
# file -i says
# text/plain; charset=unknown-8bit

///

binmode STDOUT, ":encoding(utf-8)";
while (<>) { say } # double encoding

Extra one:

use open ":encoding(utf-8)";
binmode STDOUT, ":encoding(utf-8)";

this combo outputs utf-8 correctly.

One normally wants:

use open ":std", ":encoding(utf-8)";

Set the default encoding for open(), and binmodes STDIN, STDOUT and STDERR
to the specified encoding.

p5pRT · 2016-05-09T17:36:12Z

From ambs@zbr.pt

One normally wants:

use open ":std", ":encoding(utf-8)";

Set the default encoding for open(), and binmodes STDIN, STDOUT and
STDERR to the specified encoding.

OK, reading the docs, I get:

The ":std" subpragma on its own has no effect, but if combined with the
":utf8" or ":encoding" subpragmas, it converts the standard filehandles
(STDIN, STDOUT, STDERR) to comply with encoding selected for
input/output handles. For example, if both input and out are chosen to
be ":encoding(utf8)", a ":std" will mean that STDIN, STDOUT, and STDERR
are also in ":encoding(utf8)". On the other hand, if only output is
chosen to be in ":encoding(koi8r)", a ":std" will cause only the STDOUT
and STDERR to be in "koi8r". The ":locale" subpragma implicitly turns on
":std".

*BUT* synopsis has a line with:

use open ':std';

So, please, remove that line, or add one like you suggest:

use open ":std", ":encoding(utf-8)";

and probably add a comment on each synopsis line.

And the sentence:

The "open" pragma serves as one of the interfaces to declare default
"layers" (also known as "disciplines") for all I/O.

might be changed to

The "open" pragma serves as one of the interfaces to declare default
"layers" (also known as "disciplines") for all new I/O file handles
(for STDOUT, STDERR and STDIN use also the :std pragma).

Am I reading the docs correctly?
Cheers

p5pRT · 2016-05-17T05:16:50Z

From @tonycoz

On Sat May 07 13:14:28 2016, ambs wrote:

Sorry, probably I pushed the trigger too fast, but:

while (<>) { say } # seems to print everything good (byte wise).

///

use open ":encoding(utf-8)";
while (<>) { say } # prints something strange
# file -i says
# text/plain; charset=unknown-8bit

///

binmode STDOUT, ":encoding(utf-8)";
while (<>) { say } # double encoding

So, I would say there *is* some bug...

Am I right in assuming you're only talking about when <> falls back to STDIN.

It seems to work ok when filename are supplied:

$ ./perl -Ilib -le 'use open ":encoding(UTF-8)"; while (<>) { print ord }' foo
257

(foo contains a single "\x{101}")

Tony

p5pRT added Severity Medium distro-Linux type-core labels Oct 19, 2019

xenu removed the Severity Medium label Dec 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use open doesn't affect magic diamond operator #15311

use open doesn't affect magic diamond operator #15311

p5pRT commented May 7, 2016

p5pRT commented May 7, 2016

p5pRT commented May 7, 2016

p5pRT commented May 7, 2016

p5pRT commented May 9, 2016

p5pRT commented May 9, 2016

p5pRT commented May 9, 2016

p5pRT commented May 17, 2016

use open doesn't affect magic diamond operator #15311

use open doesn't affect magic diamond operator #15311

Comments

p5pRT commented May 7, 2016

p5pRT commented May 7, 2016

From ambs@zbr.pt

Created by ambs@cpan.org

p5pRT commented May 7, 2016

From ambs@zbr.pt

p5pRT commented May 7, 2016

p5pRT commented May 9, 2016

From ambs@zbr.pt

p5pRT commented May 9, 2016

From @ikegami

p5pRT commented May 9, 2016

From ambs@zbr.pt

p5pRT commented May 17, 2016

From @tonycoz