Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encode() with FB_CROAK may destroy its argument #9391

Closed
p5pRT opened this issue Jun 25, 2008 · 5 comments
Closed

encode() with FB_CROAK may destroy its argument #9391

p5pRT opened this issue Jun 25, 2008 · 5 comments

Comments

@p5pRT
Copy link

p5pRT commented Jun 25, 2008

Migrated from rt.perl.org#56340 (status was 'rejected')

Searchable as RT56340$

@p5pRT
Copy link
Author

p5pRT commented Jun 25, 2008

From alex@kapranoff.ru

Created by alex@kapranoff.ru

Standard Encode​::encode() function with non-FB_DEFAULT argument
and 'utf-7' encoding always kills its data argument.

use Encode;

my $s = 'test';
my $s2 = $s;

my $enc = find_encoding('utf-7');

$enc->encode($s);
warn "encode '$s' ne '$s2'\n" if $s ne $s2;

$enc->encode($s, Encode​::FB_CROAK | Encode​::LEAVE_SRC);
warn "encode w. FB_CROAK '$s' ne '$s2'\n" if $s ne $s2;

Encode POD suggests using LEAVE_SRC bit, but it's not used inside
utf-7 codec so it has no effect.

Here's a patch against bleadperl as of today​:

Inline Patch
--- UTF7.pm.orig	2008-06-25 16:25:57.000000000 +0400
+++ UTF7.pm	2008-06-25 17:01:58.000000000 +0400
@@ -52,7 +52,7 @@
              die "This should not happen! (pos=" . pos($str) . ")";
          }
      }
-    $_[1] = '' if $chk;
+    $_[1] = '' if $chk && ($chk & Encode::LEAVE_SRC) == 0;
      return $bytes;
  }
Perl Info

Flags:
     category=core
     severity=critical

Site configuration information for perl 5.10.0:

Configured by kappa at Tue Dec 18 22:42:37 MSK 2007.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
   Platform:
     osname=linux, osvers=2.6.22-14-generic, 
archname=i686-linux-thread-multi-64int-ld
     uname='linux aukland 2.6.22-14-generic #1 smp sun oct 14 23:05:12 
gmt 2007 i686 gnulinux '
     config_args=''
     hint=recommended, useposix=true, d_sigaction=define
     useithreads=define, usemultiplicity=define
     useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
     use64bitint=define, use64bitall=undef, uselongdouble=define
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing 
-pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
     optimize='-O2',
     cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe 
-I/usr/local/include'
     ccversion='', gccversion='4.1.3 20070929 (prerelease) (Ubuntu 
4.1.2-16ubuntu2)', gccosandvers=''
     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
     ivtype='long long', ivsize=8, nvtype='long double', nvsize=12, 
Off_t='off_t', lseeksize=8
     alignbytes=4, prototype=define
   Linker and Libraries:
     ld='cc', ldflags =' -L/usr/local/lib'
     libpth=/usr/local/lib /lib /usr/lib /usr/lib64
     libs=-lnsl -lgdbm -ldl -lm -lcrypt -lutil -lpthread -lc
     perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
     libc=/lib/libc-2.6.1.so, so=so, useshrplib=false, libperl=libperl.a
     gnulibc_version='2.6.1'
   Dynamic Linking:
     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
     cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib'

Locally applied patches:



@INC for perl 5.10.0:
     /site/lib
     /usr/local/lib/perl5/5.10.0/i686-linux-thread-multi-64int-ld
     /usr/local/lib/perl5/5.10.0
     /usr/local/lib/perl5/site_perl/5.10.0/i686-linux-thread-multi-64int-ld
     /usr/local/lib/perl5/site_perl/5.10.0
     .


Environment for perl 5.10.0:
     HOME=/home/kappa
     LANG=ru_RU.UTF-8
     LANGUAGE (unset)
     LD_LIBRARY_PATH (unset)
     LOGDIR (unset)
 
PATH=/home/kappa/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
     PERLLIB=/site/lib
     PERL_BADLANG (unset)
     SHELL=/usr/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Dec 19, 2011

From @jkeenan

On Wed Jun 25 06​:07​:14 2008, kappa wrote​:

Standard Encode​::encode() function with non-FB_DEFAULT argument
and 'utf-7' encoding always kills its data argument.

use Encode;

my $s = 'test';
my $s2 = $s;

my $enc = find_encoding('utf-7');

$enc->encode($s);
warn "encode '$s' ne '$s2'\n" if $s ne $s2;

$enc->encode($s, Encode​::FB_CROAK | Encode​::LEAVE_SRC);
warn "encode w. FB_CROAK '$s' ne '$s2'\n" if $s ne $s2;

Encode POD suggests using LEAVE_SRC bit, but it's not used inside
utf-7 codec so it has no effect.

Here's a patch against bleadperl as of today​:

--- UTF7.pm.orig 2008-06-25 16​:25​:57.000000000 +0400
+++ UTF7.pm 2008-06-25 17​:01​:58.000000000 +0400
@​@​ -52,7 +52,7 @​@​
die "This should not happen! (pos=" . pos($str) . ")";
}
}
- $_[1] = '' if $chk;
+ $_[1] = '' if $chk && ($chk & Encode​::LEAVE_SRC) == 0;
return $bytes;
}

Is this the same issue that was addressed in
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=22834 (closed in 2003)?

And, since we're on the subject of Encode and FB_CROAK, is it related to
this ticket still open since 2004?
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=30774

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Dec 19, 2011

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 19, 2011

From @rjbs

This bug is *NOT* the same as
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=22834

It does not appear to be related to
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=30774

The problem APPEARS to be that the user is relying on the $check
semantics from Encode​::encode, which do not APPEAR to be the same as
those of an Encode​::Encoding object's ->encode method. There, $check
seems to be a simple boolean and any true value for $check means that
the code should-or-must alter the given string in place.

The following simplified test passes, showing that the semantics for
$check in the function do seem to work as promised​:

  use Encode;
  my $s = 'test';
  my $s2 = $s;
  encode('utf-7', $s, Encode​::FB_CROAK | Encode​::LEAVE_SRC );
  warn "encode w. FB_CROAK '$s' ne '$s2'\n" if $s ne $s2;

I believe this is a non-bug.

@p5pRT
Copy link
Author

p5pRT commented Apr 20, 2012

@cpansprout - Status changed from 'open' to 'rejected'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant