Skip Menu |
Report information
Id: 131575
Status: pending release
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: ntyni [at] debian.org
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: 5.27.1



To: perlbug [...] perl.org
Subject: s// on utf8 string occasionally crashes with "Malformed UTF-8 character"
Date: Wed, 14 Jun 2017 20:32:27 +0100
From: Niko Tyni <ntyni [...] debian.org>
Download (untitled) / with headers
text/plain 3.7k
This is a bug report for perl from Niko Tyni <ntyni@debian.org>, generated with the help of perlbug 1.40 running under perl 5.27.1. ----------------------------------------------------------------- As reported by Benjamin Bayart in https://bugs.debian.org/864782 a regression between 5.22 and 5.24 made s// on a valid utf8 string start to crash occasionally with 'Malformed UTF-8 character' diagnostics. I was able to get it down to #!perl my $text = "%t%\x{6bce}"; $text =~ s{~*%[a-z]%}{}g; print "ok\n"; which crashes for me on blead ~50% of the time with something like Malformed UTF-8 character: \x91 (unexpected continuation byte 0x91, with no preceding start byte) in substitution (s///) at /home/ntyni/864782.pl line 3. Malformed UTF-8 character (fatal) at /home/ntyni/864782.pl line 3. Running this with valgrind shows invalid reads, and bisecting indicates that those errors started with v5.23.3-110-g147f21b5b8 : commit 147f21b5b8054c559a1ffb568dbf310244fa0c91 Author: David Mitchell <davem@iabyn.com> Date: Sat Sep 26 13:12:40 2015 +0100 make /fixed-substr/ much faster. ----------------------------------------------------------------- --- Flags: category=core severity=medium --- Site configuration information for perl 5.27.1: Configured by ntyni at Wed Jun 14 20:16:03 BST 2017. Summary of my perl5 (revision 5 version 27 subversion 1) configuration: Commit id: b50535da2f4aaf97d13e96cda0069755fb6bbf76 Platform: osname=linux osvers=4.9.0-3-amd64 archname=x86_64-linux uname='linux carme 4.9.0-3-amd64 #1 smp debian 4.9.25-1 (2017-05-02) x86_64 gnulinux ' config_args='-des -Dusedevel -DDEBUGGING' hint=recommended useposix=true d_sigaction=define useithreads=undef usemultiplicity=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cc' ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' optimize='-O2 -g' cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='6.3.0 20170516' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.24.so so=so useshrplib=false libperl=libperl.a gnulibc_version='2.24' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong' --- @INC for perl 5.27.1: lib /usr/local/lib/perl5/site_perl/5.27.1/x86_64-linux /usr/local/lib/perl5/site_perl/5.27.1 /usr/local/lib/perl5/5.27.1/x86_64-linux /usr/local/lib/perl5/5.27.1 --- Environment for perl 5.27.1: HOME=/home/ntyni LANG=en_GB.UTF-8 LANGUAGE=en_GB:en LC_CTYPE=fi_FI.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/local/bin:/usr/bin:/bin:/usr/games PERL_BADLANG (unset) SHELL=/bin/zsh
Subject: Re: [perl #131575] s// on utf8 string occasionally crashes with "Malformed UTF-8 character"
To: perl5-porters [...] perl.org
Date: Fri, 16 Jun 2017 15:57:24 +0100
From: Dave Mitchell <davem [...] iabyn.com>
On Wed, Jun 14, 2017 at 01:00:12PM -0700, Niko Tyni wrote: Show quoted text
> As reported by Benjamin Bayart in https://bugs.debian.org/864782 a > regression between 5.22 and 5.24 made s// on a valid utf8 string start > to crash occasionally with 'Malformed UTF-8 character' diagnostics. > > I was able to get it down to > > #!perl > my $text = "%t%\x{6bce}"; > $text =~ s{~*%[a-z]%}{}g; > print "ok\n"; > > which crashes for me on blead ~50% of the time with something like > > Malformed UTF-8 character: \x91 (unexpected continuation byte 0x91, with no preceding start byte) in substitution (s///) at /home/ntyni/864782.pl line 3. > Malformed UTF-8 character (fatal) at /home/ntyni/864782.pl line 3. > > Running this with valgrind shows invalid reads, and bisecting > indicates that those errors started with v5.23.3-110-g147f21b5b8 : > > commit 147f21b5b8054c559a1ffb568dbf310244fa0c91 > Author: David Mitchell <davem@iabyn.com> > Date: Sat Sep 26 13:12:40 2015 +0100 > > make /fixed-substr/ much faster.
Now fixed by v5.27.0-322-gbb152a4 -- Monto Blanco... scorchio!


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org