From vvv@vsu.ru

Created by vvv@vsu.ru

in the below test file, the string $plain is produced by the decrypt method
of Crypt​::Rijndael, and $plain is equal to "This is a test\002\002",
but nevertheless attempting to strip the trailing padding with
s/\002+$//; does not work, and also if i would use $plain =~ m/(.)$/;
perl would not set $1 to \002 but will leave it undefined.
so the string $plain has some "strange" representation inside the perl.

it may be caused by Crypt​::Rijndael, but in any case it looks like perl's core
behavior is broken, if such things are possible.

#############################################################################
#!/usr/bin/perl

use strict;
use Crypt​::Rijndael;

my $encrypted = pack("H*","89bc70c0c151354857d0fe5d2976fc36");
my $key = pack("H*","430b26a3d7102f725ef604373651c6f0");
my $cipher = Crypt​::Rijndael->new($key, Crypt​::Rijndael​::MODE_CBC());
my $plain = $cipher->decrypt($encrypted);
die if $plain ne "This is a test\002\002";
# so $plain ends with \002\002 indeed
print "plain​: ".unpack("H*",$plain)."\n";
$plain =~ s/\002+$//;
# the above should have stripped two trailing \002 characters but it did not​:
print "plain​: ".unpack("H*",$plain)."\n";
#############################################################################

Flags:
    category=library
    severity=high

Site configuration information for perl 5.10.0:

Configured by Debian Project at Tue Sep 30 16:11:07 UTC 2008.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.6.26.1, archname=i486-linux-gnu-thread-multi
    uname='linux ninsei 2.6.26.1 #1 smp preempt sun aug 3 22:34:07 pdt 2008 i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.10.0 -Dsitearch=/usr/local/lib/perl/5.10.0 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.10.0 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.3.2', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib /usr/lib64
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.7.so, so=so, useshrplib=true, libperl=libperl.so.5.10.0
    gnulibc_version='2.7'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib'

Locally applied patches:
    


@INC for perl 5.10.0:
    /etc/perl
    /usr/local/lib/perl/5.10.0
    /usr/local/share/perl/5.10.0
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.10
    /usr/share/perl/5.10
    /usr/local/lib/site_perl
    .


Environment for perl 5.10.0:
    HOME=/home/vvv
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/vvv/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/bash

From @nwc10

Thanks for the bug report.

On Fri, Oct 31, 2008 at 12​:47​:51AM -0700, vvv@​vsu.ru (via RT) wrote​:

in the below test file, the string $plain is produced by the decrypt method
of Crypt​::Rijndael, and $plain is equal to "This is a test\002\002",
but nevertheless attempting to strip the trailing padding with
s/\002+$//; does not work, and also if i would use $plain =~ m/(.)$/;
perl would not set $1 to \002 but will leave it undefined.
so the string $plain has some "strange" representation inside the perl.

it may be caused by Crypt​::Rijndael, but in any case it looks like perl's core
behavior is broken, if such things are possible.

Well, it is caused by Crypt​::Rijndael, but arguably also the core is broken.

As you may already know, internally perl stores strings as a buffer of bytes,
and a length. C stores strings as a sequence of bytes terminated by an ASCII
NUL. But for convenience perl also adds a '\0' after the bytes in the buffer,
to make it easy to pass the buffer to C functions. Well, that's what's
supposed to be there​:

If I tweak your test slighty, I get the following​:

$ cat ~/test/60246_tweaked
#!/usr/bin/perl

use strict;
use Crypt​::Rijndael;

my $encrypted = pack("H*","89bc70c0c151354857d0fe5d2976fc36");
my $key = pack("H*","430b26a3d7102f725ef604373651c6f0");
my $cipher = Crypt​::Rijndael->new($key, Crypt​::Rijndael​::MODE_CBC());
my $plain = $cipher->decrypt($encrypted);
die if $plain ne "This is a test\002\002";
use Devel​::Peek;

Dump ($plain);
$plain =~ s/..$//;
Dump ($plain);

$plain .= '';

Dump ($plain);
$plain =~ s/..$//;
Dump ($plain);

$ ~/Sandpit/588/bin/perl ~/test/60246_tweaked
SV = PV(0x668df8) at 0x681af0
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,POK,pPOK)
  PV = 0x6e0510 "This is a test\2\2"
  CUR = 16
  LEN = 24
SV = PV(0x668df8) at 0x681af0
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,POK,pPOK)
  PV = 0x6e0510 "This is a test\2\2"
  CUR = 16
  LEN = 24
SV = PV(0x668df8) at 0x681af0
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,POK,pPOK)
  PV = 0x6e0510 "This is a test\2\2"\0
  CUR = 16
  LEN = 24
SV = PV(0x668df8) at 0x681af0
  REFCNT = 1
  FLAGS = (PADBUSY,PADMY,POK,pPOK)
  PV = 0x6e0510 "This is a test"\0
  CUR = 14
  LEN = 24

The first pair of dumps show that Crypt​::Rijndael has returned a scalar
which doesn't have the "\0" at the end. Clearly Perl's eq operator doesn't
notice, but your test case shows that for some reason the regexp engine
*does*.

I tweaked the test case to append an empty string. This has the effect of
"normalising" the buffer, with a "\0" afterwards, as you can see from the
third dump. The fourth dump shows that the regexp engine correctly removes
the two characters once it's there.

So Crypt​::Rijndael needs fixing to add the "\0" in to make well formed
scalars. (Else users of Crypt​::Rijndael are going to come very unstuck
when they try to use its output as filenames in open statements.)

But also, the core regexp engine needs fixing, as it should be doing
everything on explicit lengths, and not doing anything with C style strcmp()
I'm guessing that if it's using strcmp() somewhere, then it's possible to
get it to produce wrong results for strings containing "\0" somewhere in
the middle.

Nicholas Clark

The RT System itself - Status changed from 'new' to 'open'

From @cpansprout

Fixed by 7016d6e.

@cpansprout - Status changed from 'open' to 'resolved'