bug with index() matching beyond end of string when \0 bytes (00000000) are involved #9315

p5pRT · 2008-05-05T16:43:02Z

Migrated from rt.perl.org#53746 (status was 'resolved')

Searchable as RT53746$

p5pRT · 2008-05-05T16:43:03Z

From insana@mediaglyphs.org

This is a bug report for perl from insana@ebi.ac.uk,
generated with the help of perlbug 1.36 running under perl 5.10.0.

# by Giuseppe Insana <insana@ebi.ac.uk>
# May 5 2008

$msg="aaZaa";
$string1="aa";
$string2="aaa";
print "msg (bits): ",unpack('b*',$msg),"\n";
print "indexing string1 ($string1) and string2 ($string2) inside msg
($msg)\n";
print index($msg,$string1,0),"\n";
print index($msg,$string2,0),"\n";

$msg="\0\0Z\0\0";
$string1="\0\0";
$string2="\0\0\0";
print "msg (bits): ",unpack('b*',$msg),"\n";
print "indexing string1 ($string1) and string2 ($string2) inside msg
($msg)\n";
print index($msg,$string1,0),"\n";
print index($msg,$string2,0),"\n";

print "\n result is different, but actually for string2 index should
always return -1 in both cases, shouldn't it??\n\n";

Flags:
category=core
severity=medium

Site configuration information for perl 5.10.0:

Configured by SYSTEM at Thu Jan 10 11:00:30 2008.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
Platform:
osname=MSWin32, osvers=5.00, archname=MSWin32-x86-multi-thread
uname=''
config_args='undef'
hint=recommended, useposix=true, d_sigaction=undef
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=undef, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_SITECUSTOMIZE
-DPRIVLIB_LAST_IN_INC -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS
-DUSE_PERLIO -DPERL_MSVCRT_READFIX',
optimize='-MD -Zi -DNDEBUG -O1',
cppflags='-DWIN32'
ccversion='12.00.8804', gccversion='', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
ivtype='long', ivsize=4, nvtype='double', nvsize=8,
Off_t='__int64', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf
-libpath:"D:\apps\Perl\lib\CORE" -machine:x86'
libpth=\lib
libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib
odbc32.lib odbccp32.lib msvcrt.lib
perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib
version.lib odbc32.lib odbccp32.lib msvcrt.lib
libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl510.lib
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug
-opt:ref,icf -libpath:"D:\apps\Perl\lib\CORE" -machine:x86'

Locally applied patches:
ACTIVEPERL_LOCAL_PATCHES_ENTRY
32809 Load 'loadable object' with non-default file extension
32728 64-bit fix for Time::Local

@INC for perl 5.10.0:
D:/apps/Perl/site/lib
D:/apps/Perl/lib
.

Environment for perl 5.10.0:
HOME=D:\HOME
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)

PATH=D:\apps\Perl\site\bin;D:\apps\Perl\bin;D:\apps\PHP\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program
Files\GNU\GnuPG;D:\apps\cvsnt;C:\USR\BIN;D:\apps\Python25;
PERL_BADLANG (unset)
SHELL=/bin/sh

p5pRT · 2008-05-05T18:50:42Z

From p5p@spam.wizbit.be

Easier example:

perl5.8.8 -wle 'my $s = "\0\0\0"; print index("\0\0Z\0\0", $s, 0);'
# Output: -1

perl5.10.0 -wle 'my $s = "\0\0\0"; print index("\0\0Z\0\0", $s, 0);'
# Output: 3

perl-blead -wle 'my $s = "\0\0\0"; print index("\0\0Z\0\0", $s, 0);'
# Output: 3

Note that:

perl-blead -wle 'print index("\0\0Z\0\0", "\0\0\0", 0);'
# Output: -1

perl-blead -wle 'my $s = "\0\0\0\0"; print index("\0\0Z\0\0", $s, 0);'
# Output: -1

p5pRT · 2008-05-05T18:50:44Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2008-05-06T12:58:16Z

From @Abigail

On Mon, May 05, 2008 at 11:50:44AM -0700, Bram via RT wrote:

Easier example:

perl5.8.8 -wle 'my $s = "\0\0\0"; print index("\0\0Z\0\0", $s, 0);'
# Output: -1

perl5.10.0 -wle 'my $s = "\0\0\0"; print index("\0\0Z\0\0", $s, 0);'
# Output: 3

perl-blead -wle 'my $s = "\0\0\0"; print index("\0\0Z\0\0", $s, 0);'
# Output: 3

Attached is a set of tests.

Abigail

p5pRT · 2008-05-06T12:58:16Z

From @Abigail

Inline Patch

--- t/op/index.t.orig	2008-05-06 14:08:20.000000000 +0200
+++ t/op/index.t	2008-05-06 14:56:41.000000000 +0200
@@ -7,7 +7,7 @@
 }
 
 use strict;
-plan( tests => 69 );
+plan( tests => 111 );
 
 run_tests() unless caller;
 
@@ -160,4 +160,43 @@
     is(index($t, 'xyz'), 4, "0xfffffffd and utf8cache");
 }
 
+
+# Tests for NUL characters.
+{
+    my @tests = (
+        ["",            -1, -1, -1],
+        ["foo",         -1, -1, -1],
+        ["\0",           0, -1, -1],
+        ["\0\0",         0,  0, -1],
+        ["\0\0\0",       0,  0,  0],
+        ["foo\0",        3, -1, -1],
+        ["foo\0foo\0\0", 3,  7, -1],
+    );
+    foreach my $l (1 .. 3) {
+        my $q = "\0" x $l;
+        my $i = 0;
+        foreach my $test (@tests) {
+            $i ++;
+            my $str = $$test [0];
+            my $res = $$test [$l];
+
+            {
+                local $::TODO = ($l == 3 && $i == 7 ? "Bug #53746" : "");
+                is (index ($str, $q), $res, "Find NUL character(s)");
+            }
+
+            #
+            # Bug #53746 shows a difference between variables and literals,
+            # so test literals as well.
+            #
+            my $test_str = qq {is (index ("$str", "$q"), $res, } .
+                           qq {"Find NUL character(s)")};
+               $test_str =~ s/\0/\\0/g;
+
+            eval $test_str;
+            die $@ if $@;
+        }
+    }
+}
+
 }

p5pRT · 2008-05-30T21:44:48Z

From @iabyn

On Tue, May 06, 2008 at 02:57:36PM +0200, Abigail wrote:

On Mon, May 05, 2008 at 11:50:44AM -0700, Bram via RT wrote:

Easier example:

perl5.8.8 -wle 'my $s = "\0\0\0"; print index("\0\0Z\0\0", $s, 0);'
# Output: -1

perl5.10.0 -wle 'my $s = "\0\0\0"; print index("\0\0Z\0\0", $s, 0);'
# Output: 3

perl-blead -wle 'my $s = "\0\0\0"; print index("\0\0Z\0\0", $s, 0);'
# Output: 3

Attached is a set of tests.

Thanks, applied as change #33951.
I've fixed the bug itself as change #33952.
(Whoo hoo! A bug that was easy to locate, fix, and test for. You don't
see that very often :-).

--
Overhead, without any fuss, the stars were going out.
-- Arthur C Clarke

p5pRT · 2008-05-31T08:46:44Z

p5p@spam.wizbit.be - Status changed from 'open' to 'resolved'

p5pRT closed this as completed May 31, 2008

p5pRT added Severity Medium distro-mswin32 type-core labels Oct 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bug with index() matching beyond end of string when \0 bytes (00000000) are involved #9315

bug with index() matching beyond end of string when \0 bytes (00000000) are involved #9315

p5pRT commented May 5, 2008

p5pRT commented May 5, 2008

p5pRT commented May 5, 2008

p5pRT commented May 5, 2008

p5pRT commented May 6, 2008

p5pRT commented May 6, 2008

p5pRT commented May 30, 2008

p5pRT commented May 31, 2008

bug with index() matching beyond end of string when \0 bytes (00000000) are involved #9315

bug with index() matching beyond end of string when \0 bytes (00000000) are involved #9315

Comments

p5pRT commented May 5, 2008

p5pRT commented May 5, 2008

From insana@mediaglyphs.org

p5pRT commented May 5, 2008

From p5p@spam.wizbit.be

p5pRT commented May 5, 2008

p5pRT commented May 6, 2008

From @Abigail

p5pRT commented May 6, 2008

From @Abigail

p5pRT commented May 30, 2008

From @iabyn

p5pRT commented May 31, 2008