Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

a problem with encoding and -i #6610

Closed
p5pRT opened this issue Jul 14, 2003 · 5 comments
Closed

a problem with encoding and -i #6610

p5pRT opened this issue Jul 14, 2003 · 5 comments
Labels
distro-Linux type-library type-Unicode

Comments

@p5pRT
Copy link

p5pRT commented Jul 14, 2003

Migrated from rt.perl.org#22958 (status was 'rejected')

Searchable as RT22958$
@p5pRT
Copy link
Author

p5pRT commented Jul 14, 2003

From chihchun@kalug.linux.org.tw

Created by chihchun@kalug.linux.org.tw

I'm trying to use perl for convert big5 charset to utf8. The foo file
contains some Chinese BIG5 chars.

perl -M'encoding "big5", STDOUT => "utf8";' -e 'while(<>){print};' foo

The result print utf8 chars, it works fine.

I added -i.bak the file "foo" is big5 and "foo.bak" is big5 too.
my commands are

perl -M'encoding "big5", STDOUT => "utf8";' -e 'while(<>){print};' -i.bak foo
perl -i.bak -M'encoding "big5", STDOUT => "utf8";' -e 'while(<>){print};' foo

Perl Info 

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.8.0:

Configured by Debian Project at Fri Jun  6 00:10:15 EST 2003.

Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.4.20-xfs+ti1211, archname=i386-linux-thread-multi
    uname='linux kosh 2.4.20-xfs+ti1211 #1 sat nov 30 19:19:08 est 2002 i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8.0 -Darchlib=/usr/lib/perl/5.8.0 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.0 -Dsitearch=/usr/local/lib/perl/5.8.0 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.0 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O3',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing'
    ccversion='', gccversion='3.3 (Debian)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.3.1.so, so=so, useshrplib=true, libperl=libperl.so.5.8.0
    gnulibc_version='2.3.1'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.0:
    /etc/perl
    /usr/local/lib/perl/5.8.0
    /usr/local/share/perl/5.8.0
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.8.0
    /usr/share/perl/5.8.0
    /usr/local/lib/site_perl
    .


Environment for perl v5.8.0:
    HOME=/home/chihchun
    LANG=zh_TW.Big5
    LANGUAGE (unset)
    LC_ALL=zh_TW.Big5
    LC_CTYPE=zh_TW.Big5
    LD_LIBRARY_PATH=/usr/local/lib:
    LOGDIR (unset)
    PATH=/usr/bin:/bin:/usr/bin/X11:/usr/games:/home/chihchun/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Jul 18, 2003

From ben.goldberg@hotpop.com

"Rex Tsai (via RT)" wrote​:
[snip]

I'm trying to use perl for convert big5 charset to utf8. The foo file
contains some Chinese BIG5 chars.

perl -M'encoding "big5", STDOUT => "utf8";' -e 'while(<>){print};' foo

The result print utf8 chars, it works fine.

I added -i.bak the file "foo" is big5 and "foo.bak" is big5 too.
my commands are

perl -M'encoding "big5", STDOUT => "utf8";' \
-e 'while(<>){print};' -i.bak foo
perl -i.bak -M'encoding "big5", STDOUT => "utf8";' \
-e 'while(<>){print};' foo

The problem is that when you print, it is going to the currently
select()ed filehandle, which is not necessarily *STDOUT. This is
actually a good thing... however, you are only changing STDOUT to utf8,
and are not changing the select()ed filehandle to utf8.

Since your default open mode was "​:encoding(big5)", the select()ed
filehandle (which is *ARGVOUT, in this case) was opened with that
encoding.

Since you need the default open modes for input and output to be
different (big5 for input, and utf8 for output), and since encoding.pm
doesn't support this, you need the open pragma.

Try the following​:

  perl -M"open=IN,​:encoding(big5),OUT,​:utf8" \
  -e "print while <>" -i.bak foo

[untested]

--
$a=24;split//,240513;s/\B/ => /for@​@​=qw(ac ab bc ba cb ca
);{push(@​b,$a),($a-=6)^=1 for 2..$a/6x--$|;print "$@​[$a%6
]\n";((6<=($a-=6))?$a+=$_[$a%6]-$a%6​:($a=pop @​b))&&redo;}

@p5pRT
Copy link
Author

p5pRT commented Jul 20, 2003

From chihchun@kalug.linux.org.tw

[ben.goldberg <!--c--> <i>at</i> <!--a--> hotpop.com - Fri Jul 18
10​:54​:19 2003]​:

The problem is that when you print, it is going to the currently
select()ed filehandle, which is not necessarily *STDOUT. This is
actually a good thing... however, you are only changing STDOUT to utf8,
and are not changing the select()ed filehandle to utf8.

Since your default open mode was "​:encoding(big5)", the select()ed
filehandle (which is *ARGVOUT, in this case) was opened with that
encoding.

Since you need the default open modes for input and output to be
different (big5 for input, and utf8 for output), and since encoding.pm
doesn't support this, you need the open pragma.

Try the following​:

perl -M"open=IN,​:encoding(big5),OUT,​:utf8" \
-e "print while <>" -i.bak foo

Without -i.bak , this command print correct utf8 string,
but it's still the big5 string in foo and foo.bak with -i.bak.

@p5pRT
Copy link
Author

p5pRT commented Jan 16, 2018

From zefram@fysh.org

There is no Perl bug here. This ticket should be closed.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Jan 16, 2018

@tonycoz - Status changed from 'open' to 'rejected'

@p5pRT p5pRT closed this as completed Jan 16, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
distro-Linux type-library type-Unicode
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@p5pRT