use open :locale doesn't work in Windows #9657

p5pRT · 2009-02-22T21:16:41Z

Migrated from rt.perl.org#63402 (status was 'open')

Searchable as RT63402$

p5pRT · 2009-02-22T21:16:46Z

From @ikegami

Created by @ikegami

[ Getting HTTP 500 errors trying to search rt.perl.org to see if this bug
was previously submitted ]

"use open ':std', ':locale';" doesn't work in Windows.

chcp
Active code page: 850

perl -we"use open ':std', ':encoding(cp850)'; print chr 0xB6" | perl -l
-0777 -e"print unpack 'H*', <>"
f4
(correct)

perl -we"use open ':std', ':locale'; print chr 0xB6" | perl -l
-0777 -e"print unpack 'H*', <>"
Cannot figure out an encoding to use at -e line 1
b6
(should be "f4" with no warnings)

perl -we"use open ':std', ':encoding(cp850)'; print chr 0x2591" | perl -l
-0777 -e"print unpack 'H*', <>"
b0
(correct)

perl -we"use open ':std', ':locale'; print chr 0x2591" | perl -l
-0777 -e"print unpack 'H*', <>"
Cannot figure out an encoding to use at -e line 1
Wide character in print at -e line 1.
e29691
(should be "b0" with no warnings)

Perl Info


Flags:
    category=library
    severity=medium

Site configuration information for perl 5.10.0:

Configured by SYSTEM at Tue Dec 18 08:45:53 2007.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
  Platform:
    osname=MSWin32, osvers=5.00, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_SITECUSTOMIZE
-DPRIVLIB_LAST_IN_INC -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS
-DUSE_PERLIO -DPERL_MSVCRT_READFIX',
    optimize='-MD -Zi -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='12.00.8804', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf
-libpath:"C:\progs\perl5100\lib\CORE"  -machine:x86'
    libpth=\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib
uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib
msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib
uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib
msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl510.lib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug
-opt:ref,icf  -libpath:"C:\progs\perl5100\lib\CORE"  -machine:x86'

Locally applied patches:
    ACTIVEPERL_LOCAL_PATCHES_ENTRY


@INC for perl 5.10.0:
    c:/progs/perl5100/site/lib
    c:/progs/perl5100/lib
    .


Environment for perl 5.10.0:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=c:\bin;c:\progs\perl5100\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\WBEM
    PERL_BADLANG (unset)
    SHELL (unset)

p5pRT · 2013-05-26T17:03:07Z

From @jkeenan

On Sun Feb 22 13:16:46 2009, ikegami@adaelis.com wrote:

This is a bug report for perl from ikegami@adaelis.com,
generated with the help of perlbug 1.36 running under perl 5.10.0.

-----------------------------------------------------------------
[Please enter your report here]

[ Getting HTTP 500 errors trying to search rt.perl.org to see if this
bug
was previously submitted ]

"use open ':std', ':locale';" doesn't work in Windows.

chcp
Active code page: 850

perl -we"use open ':std', ':encoding(cp850)'; print chr 0xB6" | perl
-l
-0777 -e"print unpack 'H*', <>"
f4
(correct)

perl -we"use open ':std', ':locale'; print chr 0xB6" | perl
-l
-0777 -e"print unpack 'H*', <>"
Cannot figure out an encoding to use at -e line 1
b6
(should be "f4" with no warnings)

perl -we"use open ':std', ':encoding(cp850)'; print chr 0x2591" |
perl -l
-0777 -e"print unpack 'H*', <>"
b0
(correct)

perl -we"use open ':std', ':locale'; print chr 0x2591" |
perl -l
-0777 -e"print unpack 'H*', <>"
Cannot figure out an encoding to use at -e line 1
Wide character in print at -e line 1.
e29691
(should be "b0" with no warnings)

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
category=library
severity=medium
---
Site configuration information for perl 5.10.0:

Does this problem still persist with Perl 5.18 on Windows?

Thank you very much.
Jim Keenan

p5pRT · 2013-05-26T17:03:07Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2013-05-26T17:40:04Z

From @bulk88

On Sun May 26 10:03:07 2013, jkeenan wrote:

Does this problem still persist with Perl 5.18 on Windows?

Thank you very much.
Jim Keenan

No change in behavior the reporter observed on 5.19.0. I can't comment
whether there is a bug here or not.

____________________________________________________________________

C:\Documents and Settings\Owner\Desktop>chcp
Active code page: 1252

C:\Documents and Settings\Owner\Desktop>perl -we"use open ':std',
':encoding(cp8
50)'; print chr 0xB6" | perl -l -0777 -e"print unpack 'H*', <>"
f4

C:\Documents and Settings\Owner\Desktop>perl -we"use open ':std',
':locale'; pri
nt chr 0xB6" | perl -l -0777 -e"print unpack 'H*', <>"
Cannot figure out an encoding to use at -e line 1.
b6

C:\Documents and Settings\Owner\Desktop>perl -we"use open ':std',
':encoding(cp8
50)'; print chr 0x2591" | perl -l -0777 -e"print unpack 'H*', <>"
b0

C:\Documents and Settings\Owner\Desktop>perl -we"use open ':std',
':locale'; pri
nt chr 0x2591" | perl -l -0777 -e"print unpack 'H*', <>"
Cannot figure out an encoding to use at -e line 1.
Wide character in print at -e line 1.
e29691

C:\Documents and Settings\Owner\Desktop>
____________________________________________________________________

--
bulk88 ~ bulk88 at hotmail.com

p5pRT · 2013-05-27T04:00:32Z

From @ikegami

On Sun, May 26, 2013 at 1:40 PM, bulk88 via RT <perlbug-followup@perl.org>wrote:

No change in behavior the reporter observed on 5.19.0. I can't comment
whether there is a bug here or not.

Well, I don't think having the following lead every portable script is a
proper expectation of Perl programmers:

my $encoding;
BEGIN {
if ($^O eq 'MSWin32') {
require Win32;
my $cp = Win32::GetConsoleCP();
$encoding = ":encoding(cp$cp)";
} else {
$encoding = ':locale';
}
}

use open ':std', $encoding;

The solution may not be to change :locale.

p5pRT · 2013-05-27T11:14:11Z

From @Leont

On Sun, May 26, 2013 at 7:03 PM, James E Keenan via RT
<perlbug-followup@perl.org> wrote:

Does this problem still persist with Perl 5.18 on Windows?

Thank you very much.
Jim Keenan

":locale" is a mess. open.pm is faking it, since there didn't use to
be a :locale layer. Now there is one, but it doesn't pass tests on
Windows (that may be a test issue though).

Leon

p5pRT · 2013-05-28T01:39:03Z

From @ikegami

On Sun, May 26, 2013 at 11:59 PM, Eric Brine <ikegami@adaelis.com> wrote:

On Sun, May 26, 2013 at 1:40 PM, bulk88 via RT <perlbug-followup@perl.org>wrote:

No change in behavior the reporter observed on 5.19.0. I can't comment
whether there is a bug here or not.

Well, I don't think having the following lead every portable script is a
proper expectation of Perl programmers:
my $encoding;
BEGIN \{
   if $$^O eq 'MSWin32'$ \{
      require Win32;
      my $cp = Win32&#8203;::GetConsoleCP;
      $encoding = "&#8203;:encoding$cp$cp$";
   \} else \{
      $encoding = '&#8203;:locale';
   \}
\}

use open '&#8203;:std'\, $encoding;
The solution may not be to change :locale.

In fact, the solution can be implemented in a module. Given this and given
the fact that it's very unlikely that this ticket will motivate anyone to
work on this problem, so you can close this ticket if you want.

p5pRT · 2013-05-28T08:43:53Z

From @nwc10

On Mon, May 27, 2013 at 09:38:15PM -0400, Eric Brine wrote:

On Sun, May 26, 2013 at 11:59 PM, Eric Brine <ikegami@adaelis.com> wrote:
On Sun, May 26, 2013 at 1:40 PM, bulk88 via RT <perlbug-followup@perl.org>wrote:

No change in behavior the reporter observed on 5.19.0. I can't comment
whether there is a bug here or not.

Well, I don't think having the following lead every portable script is a
proper expectation of Perl programmers:
my $encoding;
BEGIN \{
   if $$^O eq 'MSWin32'$ \{
      require Win32;
      my $cp = Win32&#8203;::GetConsoleCP;
      $encoding = "&#8203;:encoding$cp$cp$";
   \} else \{
      $encoding = '&#8203;:locale';
   \}
\}

use open '&#8203;:std'\, $encoding;
The solution may not be to change :locale.
In fact, the solution can be implemented in a module. Given this and given
the fact that it's very unlikely that this ticket will motivate anyone to
work on this problem, so you can close this ticket if you want.

However, it's not documented in open.pm's Pod that :locale only works on
*nix. So if we decide that we're not going to change the code, we should at
least document the current behaviour/limitations.

I guessing that it's sort of *implied* by this:

The logic of C<:locale> is described in full in L<encoding>,
but in short it is first trying nl_langinfo(CODESET) and then
guessing from the LC_ALL and LANG locale environment variables.

but only by reversing out from this bug report and inferring that none of
those approaches work on Win32.

Also, given that :locale uses encoding, and we've said we're going to remove
encoding.pm, doesn't that mean that :locale itself needs deprecating? Or, at
least, rethinking?

Nicholas Clark

p5pRT · 2013-05-28T15:18:13Z

From @ikegami

On Tue, May 28, 2013 at 4:43 AM, Nicholas Clark via RT <
perlbug-followup@perl.org> wrote:

Also, given that :locale uses encoding, and we've said we're going to
remove
encoding.pm, doesn't that mean that :locale itself needs deprecating? Or,
at
least, rethinking?

No, it doesn't need deprecating. :locale uses :encoding layers to perform
its encoding/decoding and none of encoding.pm's deprecated core
functionality.

encoding.pm is only used because it already had a function to determine the
encoding a local uses (_get_locale_encoding). It's a simply a case of
improper modularisation,

- Eric

p5pRT added the Severity Low label Oct 18, 2019

xenu added the distro-mswin32 label Oct 20, 2021

xenu removed the Severity Low label Dec 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use open :locale doesn't work in Windows #9657

use open :locale doesn't work in Windows #9657

p5pRT commented Feb 22, 2009

p5pRT commented Feb 22, 2009

p5pRT commented May 26, 2013

p5pRT commented May 26, 2013

p5pRT commented May 26, 2013

p5pRT commented May 27, 2013

p5pRT commented May 27, 2013

p5pRT commented May 28, 2013

p5pRT commented May 28, 2013

p5pRT commented May 28, 2013

use open :locale doesn't work in Windows #9657

use open :locale doesn't work in Windows #9657

Comments

p5pRT commented Feb 22, 2009

p5pRT commented Feb 22, 2009

From @ikegami

Created by @ikegami

p5pRT commented May 26, 2013

From @jkeenan

p5pRT commented May 26, 2013

p5pRT commented May 26, 2013

From @bulk88

p5pRT commented May 27, 2013

From @ikegami

p5pRT commented May 27, 2013

From @Leont

p5pRT commented May 28, 2013

From @ikegami

p5pRT commented May 28, 2013

From @nwc10

p5pRT commented May 28, 2013

From @ikegami