Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Perl warnings may generate "Wide character in print" warnings #14747

Open
p5pRT opened this issue Jun 10, 2015 · 6 comments
Open

Perl warnings may generate "Wide character in print" warnings #14747

p5pRT opened this issue Jun 10, 2015 · 6 comments
Labels
distro-Linux type-core

Comments

@p5pRT
Copy link

p5pRT commented Jun 10, 2015

Migrated from rt.perl.org#125379 (status was 'open')

Searchable as RT125379$
@p5pRT
Copy link
Author

p5pRT commented Jun 10, 2015

From @wolfsage

Created by wolfsage@gmail.com

Perl's own warnings will generate utf8 warnings. This seems very minor,
but it's mildly confusing as it claims "print at line 1" when I didn't
use print.

  mhorsfall@​dumai​:~$ perl -Mutf8 -e 'my $x = 5+3*ʂʂ;'
  Wide character in print at -e line 1.
  Operator or semicolon missing before *ʂʂ at -e line 1.
  Ambiguous use of * resolved as operator * at -e line 1.

Perl Info 

Flags:
    category=core
    severity=low

Site configuration information for perl 5.20.1:

Configured by Debian Project at Fri Oct 10 14:16:26 UTC 2014.

Summary of my perl5 (revision 5 version 20 subversion 1) configuration:

  Platform:
    osname=linux, osvers=3.2.0-37-generic,
archname=x86_64-linux-gnu-thread-multi
    uname='linux batsu 3.2.0-37-generic #58-ubuntu smp thu jan 24
15:28:10 utc 2013 x86_64 x86_64 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat
-Werror=format-security -Dldflags= -Wl,-Bsymbolic-functions
-Wl,-z,relro -Dlddlflags=-shared -Wl,-Bsymbolic-functions -Wl,-z,relro
-Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.20
-Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.20 -Dvendorprefix=/usr
-Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.20
-Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.20.1
-Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.20.1
-Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3
-Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3
-Duse64bitint -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio
-Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -Doptimize=-O2
-Duseshrplib -Dlibperl=libperl.so.5.20.1 -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv
-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv
-fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.9.1', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib
/usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu
/lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=libc-2.19.so, so=so, useshrplib=true, libperl=libperl.so.5.20
    gnulibc_version='2.19'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector'

Locally applied patches:
    DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS
default for modules installed from CPAN.
    DEBPKG:debian/db_file_ver - http://bugs.debian.org/340047 Remove
overly restrictive DB_File version check.
    DEBPKG:debian/doc_info - Replace generic man(1) instructions with
Debian-specific information.
    DEBPKG:debian/enc2xs_inc - http://bugs.debian.org/290336 Tweak
enc2xs to follow symlinks and ignore missing @INC directories.
    DEBPKG:debian/errno_ver - http://bugs.debian.org/343351 Remove
Errno version check due to upgrade problems with long-running
processes.
    DEBPKG:debian/libperl_embed_doc - http://bugs.debian.org/186778
Note that libperl-dev package is required for embedded linking
    DEBPKG:fixes/respect_umask - Respect umask during installation
    DEBPKG:debian/writable_site_dirs - Set umask approproately for
site install directories
    DEBPKG:debian/extutils_set_libperl_path - EU:MM: set location of
libperl.a under /usr/lib
    DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or
perllocal.pod for perl or vendor
    DEBPKG:debian/prefix_changes - Fiddle with *PREFIX and variables
written to the makefile
    DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to
the binary targets.
    DEBPKG:debian/instmodsh_doc - Debian policy doesn't install
.packlist files for core or vendor.
    DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH
as per Debian policy.
    DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to
/etc/perl/Net as /usr may not be writable.
    DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian
    DEBPKG:debian/module_build_man_extensions -
http://bugs.debian.org/479460 Adjust Module::Build manual page
extensions for the Debian Perl policy
    DEBPKG:debian/prune_libs - http://bugs.debian.org/128355 Prune the
list of libraries wanted to what we actually need.
    DEBPKG:fixes/net_smtp_docs - [rt.cpan.org #36038]
http://bugs.debian.org/100195 Document the Net::SMTP 'Port' option
    DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make perlivp
skip include directories in /usr/local
    DEBPKG:debian/deprecate-with-apt - http://bugs.debian.org/747628
Point users to Debian packages of deprecated core modules
    DEBPKG:debian/squelch-locale-warnings -
http://bugs.debian.org/508764 Squelch locale warnings in Debian
package maintainer scripts
    DEBPKG:debian/skip-upstream-git-tests - Skip tests specific to the
upstream Git repository
    DEBPKG:debian/patchlevel - http://bugs.debian.org/567489 List
packaged patches for 5.20.1-1 in patchlevel.h
    DEBPKG:debian/skip-kfreebsd-crash - http://bugs.debian.org/628493
[perl #96272] Skip a crashing test case in t/op/threads.t on
GNU/kFreeBSD
    DEBPKG:fixes/document_makemaker_ccflags -
http://bugs.debian.org/628522 [rt.cpan.org #68613] Document that
CCFLAGS should include $Config{ccflags}
    DEBPKG:debian/find_html2text - http://bugs.debian.org/640479
Configure CPAN::Distribution with correct name of html2text
    DEBPKG:debian/perl5db-x-terminal-emulator.patch -
http://bugs.debian.org/668490 Invoke x-terminal-emulator rather than
xterm in perl5db.pl
    DEBPKG:debian/cpan-missing-site-dirs -
http://bugs.debian.org/688842 Fix CPAN::FirstTime defaults with
nonexisting site dirs if a parent is writable
    DEBPKG:fixes/memoize_storable_nstore - [rt.cpan.org #77790]
http://bugs.debian.org/587650 Memoize::Storable: respect 'nstore'
option not respected
    DEBPKG:debian/regen-skip - Skip a regeneration check in unrelated
git repositories
    DEBPKG:fixes/regcomp-mips-optim - [perl #122817]
http://bugs.debian.org/754054 Downgrade the optimization of regcomp.c
on mips and mipsel due to a gcc-4.9 bug
    DEBPKG:debian/makemaker-pasthru - http://bugs.debian.org/758471
Pass LD settings through to subdirectories
    DEBPKG:fixes/perldoc-less-R - [rt.cpan.org #98636]
http://bugs.debian.org/758689 Tell the 'less' pager to allow terminal
escape sequences
    DEBPKG:fixes/hurd_test_todo_socket.t - [perl #122657]
http://bugs.debian.org/758718 Disable failing GNU/Hurd test in
t/io/socket.t
    DEBPKG:fixes/pod_man_reproducible_date -
http://bugs.debian.org/759405 Support POD_MAN_DATE in Pod::Man for the
left-hand footer
    DEBPKG:fixes/data_dump_infinite_recurse - [19be3be] don't recurse
infinitely in Data::Dumper


@INC for perl 5.20.1:
    /etc/perl
    /usr/local/lib/x86_64-linux-gnu/perl/5.20.1
    /usr/local/share/perl/5.20.1
    /usr/lib/x86_64-linux-gnu/perl5/5.20
    /usr/share/perl5
    /usr/lib/x86_64-linux-gnu/perl/5.20
    /usr/share/perl/5.20
    /usr/local/lib/site_perl
    .


Environment for perl 5.20.1:
    HOME=/home/mhorsfall
    LANG=en_US.UTF-8
    LANGUAGE=en_US
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/mhorsfall/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
    PERLDOC=-oman
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Jun 25, 2015

From @jkeenan

On Wed Jun 10 08​:17​:49 2015, alh wrote​:

This is a bug report for perl from wolfsage@​gmail.com,
generated with the help of perlbug 1.40 running under perl 5.20.1.

-----------------------------------------------------------------
[Please describe your issue here]

Perl's own warnings will generate utf8 warnings. This seems very
minor,
but it's mildly confusing as it claims "print at line 1" when I didn't
use print.

mhorsfall@​dumai​:~$ perl -Mutf8 -e 'my $x = 5+3*ʂʂ;'
Wide character in print at -e line 1.
Operator or semicolon missing before *ʂʂ at -e line 1.
Ambiguous use of * resolved as operator * at -e line 1.

Confirmed for 5.20.1, 5.22.0 and blead (e67e8a4).

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=low
---
Site configuration information for perl 5.20.1​:

Configured by Debian Project at Fri Oct 10 14​:16​:26 UTC 2014.

Summary of my perl5 (revision 5 version 20 subversion 1)
configuration​:

Platform​:
osname=linux, osvers=3.2.0-37-generic,
archname=x86_64-linux-gnu-thread-multi
uname='linux batsu 3.2.0-37-generic #58-ubuntu smp thu jan 24
15​:28​:10 utc 2013 x86_64 x86_64 x86_64 gnulinux '
config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat
-Werror=format-security -Dldflags= -Wl,-Bsymbolic-functions
-Wl,-z,relro -Dlddlflags=-shared -Wl,-Bsymbolic-functions -Wl,-z,relro
-Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.20
-Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.20 -Dvendorprefix=/usr
-Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.20
-Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.20.1
-Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.20.1
-Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3
-Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3
-Duse64bitint -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio
-Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -Doptimize=-O2
-Duseshrplib -Dlibperl=libperl.so.5.20.1 -des'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv
-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
optimize='-O2 -g',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv
-fno-strict-aliasing -pipe -I/usr/local/include'
ccversion='', gccversion='4.9.1', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib
/usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu
/lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
perllibs=-ldl -lm -lpthread -lc -lcrypt
libc=libc-2.19.so, so=so, useshrplib=true, libperl=libperl.so.5.20
gnulibc_version='2.19'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -fstack-
protector'

Locally applied patches​:
DEBPKG​:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS
default for modules installed from CPAN.
DEBPKG​:debian/db_file_ver - http​://bugs.debian.org/340047 Remove
overly restrictive DB_File version check.
DEBPKG​:debian/doc_info - Replace generic man(1) instructions with
Debian-specific information.
DEBPKG​:debian/enc2xs_inc - http​://bugs.debian.org/290336 Tweak
enc2xs to follow symlinks and ignore missing @​INC directories.
DEBPKG​:debian/errno_ver - http​://bugs.debian.org/343351 Remove
Errno version check due to upgrade problems with long-running
processes.
DEBPKG​:debian/libperl_embed_doc - http​://bugs.debian.org/186778
Note that libperl-dev package is required for embedded linking
DEBPKG​:fixes/respect_umask - Respect umask during installation
DEBPKG​:debian/writable_site_dirs - Set umask approproately for
site install directories
DEBPKG​:debian/extutils_set_libperl_path - EU​:MM​: set location of
libperl.a under /usr/lib
DEBPKG​:debian/no_packlist_perllocal - Don't install .packlist or
perllocal.pod for perl or vendor
DEBPKG​:debian/prefix_changes - Fiddle with *PREFIX and variables
written to the makefile
DEBPKG​:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to
the binary targets.
DEBPKG​:debian/instmodsh_doc - Debian policy doesn't install
.packlist files for core or vendor.
DEBPKG​:debian/ld_run_path - Remove standard libs from LD_RUN_PATH
as per Debian policy.
DEBPKG​:debian/libnet_config_path - Set location of libnet.cfg to
/etc/perl/Net as /usr may not be writable.
DEBPKG​:debian/mod_paths - Tweak @​INC ordering for Debian
DEBPKG​:debian/module_build_man_extensions -
http​://bugs.debian.org/479460 Adjust Module​::Build manual page
extensions for the Debian Perl policy
DEBPKG​:debian/prune_libs - http​://bugs.debian.org/128355 Prune the
list of libraries wanted to what we actually need.
DEBPKG​:fixes/net_smtp_docs - [rt.cpan.org #36038]
http​://bugs.debian.org/100195 Document the Net​::SMTP 'Port' option
DEBPKG​:debian/perlivp - http​://bugs.debian.org/510895 Make perlivp
skip include directories in /usr/local
DEBPKG​:debian/deprecate-with-apt - http​://bugs.debian.org/747628
Point users to Debian packages of deprecated core modules
DEBPKG​:debian/squelch-locale-warnings -
http​://bugs.debian.org/508764 Squelch locale warnings in Debian
package maintainer scripts
DEBPKG​:debian/skip-upstream-git-tests - Skip tests specific to the
upstream Git repository
DEBPKG​:debian/patchlevel - http​://bugs.debian.org/567489 List
packaged patches for 5.20.1-1 in patchlevel.h
DEBPKG​:debian/skip-kfreebsd-crash - http​://bugs.debian.org/628493
[perl #96272] Skip a crashing test case in t/op/threads.t on
GNU/kFreeBSD
DEBPKG​:fixes/document_makemaker_ccflags -
http​://bugs.debian.org/628522 [rt.cpan.org #68613] Document that
CCFLAGS should include $Config{ccflags}
DEBPKG​:debian/find_html2text - http​://bugs.debian.org/640479
Configure CPAN​::Distribution with correct name of html2text
DEBPKG​:debian/perl5db-x-terminal-emulator.patch -
http​://bugs.debian.org/668490 Invoke x-terminal-emulator rather than
xterm in perl5db.pl
DEBPKG​:debian/cpan-missing-site-dirs -
http​://bugs.debian.org/688842 Fix CPAN​::FirstTime defaults with
nonexisting site dirs if a parent is writable
DEBPKG​:fixes/memoize_storable_nstore - [rt.cpan.org #77790]
http​://bugs.debian.org/587650 Memoize​::Storable​: respect 'nstore'
option not respected
DEBPKG​:debian/regen-skip - Skip a regeneration check in unrelated
git repositories
DEBPKG​:fixes/regcomp-mips-optim - [perl #122817]
http​://bugs.debian.org/754054 Downgrade the optimization of regcomp.c
on mips and mipsel due to a gcc-4.9 bug
DEBPKG​:debian/makemaker-pasthru - http​://bugs.debian.org/758471
Pass LD settings through to subdirectories
DEBPKG​:fixes/perldoc-less-R - [rt.cpan.org #98636]
http​://bugs.debian.org/758689 Tell the 'less' pager to allow terminal
escape sequences
DEBPKG​:fixes/hurd_test_todo_socket.t - [perl #122657]
http​://bugs.debian.org/758718 Disable failing GNU/Hurd test in
t/io/socket.t
DEBPKG​:fixes/pod_man_reproducible_date -
http​://bugs.debian.org/759405 Support POD_MAN_DATE in Pod​::Man for the
left-hand footer
DEBPKG​:fixes/data_dump_infinite_recurse - [19be3be] don't recurse
infinitely in Data​::Dumper

---
@​INC for perl 5.20.1​:
/etc/perl
/usr/local/lib/x86_64-linux-gnu/perl/5.20.1
/usr/local/share/perl/5.20.1
/usr/lib/x86_64-linux-gnu/perl5/5.20
/usr/share/perl5
/usr/lib/x86_64-linux-gnu/perl/5.20
/usr/share/perl/5.20
/usr/local/lib/site_perl
.

---
Environment for perl 5.20.1​:
HOME=/home/mhorsfall
LANG=en_US.UTF-8
LANGUAGE=en_US
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/mhorsfall/bin​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/sbin​:/bin​:/usr/games​:/usr/local/games
PERLDOC=-oman
PERL_BADLANG (unset)
SHELL=/bin/bash

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Jun 25, 2015

The RT System itself - Status changed from 'new' to 'open'

@toddr toddr removed the khw label Oct 25, 2019
@khwilliamson
Copy link
Contributor

I think it's possible that the wide characters could come out as mojibake if the terminal encoding isn't set to expect UTF-8. Then the Wide character warning serves a useful purpose, explaining why.

@jkeenan
Copy link
Contributor

jkeenan commented Dec 8, 2019

@wolfsage wrote in 2015:

Perl's own warnings will generate utf8 warnings. This seems very minor,
but it's mildly confusing as it claims "print at line 1" when I didn't
use print.

mhorsfall@​dumai​:~$ perl -Mutf8 -e 'my $x = 5+3*ʂʂ;'
Wide character in print at -e line 1.
Operator or semicolon missing before *ʂʂ at -e line 1.
Ambiguous use of * resolved as operator * at -e line 1.

@khwilliamson wrote:

I think it's possible that the wide characters could come out as mojibake if the terminal encoding isn't set to expect UTF-8. Then the Wide character warning serves a useful purpose, explaining why.

Other than noting this anomaly in documentation, is there anything we can do?

@Grinnz
Copy link
Contributor

Grinnz commented Dec 8, 2019

I don't think a fully correct solution is possible here. The warning message is generated from source code, which may have been decoded from UTF-8 bytes. But where it's going may expect UTF-8 bytes, it may be a handle that expects characters and has an encoding layer, it may not even be a handle at all (e.g. with a $SIG{__WARN__} handler). I do however think that the extra warning is not actually useful. But avoiding it depends how many layers of indirection there are between where the warning is generated and where it outputs.

@xenu xenu removed the affects-5.20 label Nov 19, 2021
@xenu xenu removed the Severity Low label Dec 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
distro-Linux type-core
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@jkeenan @toddr @khwilliamson @xenu @Grinnz @p5pRT