Skip Menu |
Report information
Id: 131575
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: ntyni [at] debian.org
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: 5.27.1



To: perlbug [...] perl.org
Subject: s// on utf8 string occasionally crashes with "Malformed UTF-8 character"
Date: Wed, 14 Jun 2017 20:32:27 +0100
From: Niko Tyni <ntyni [...] debian.org>
Download (untitled) / with headers
text/plain 3.7k
This is a bug report for perl from Niko Tyni <ntyni@debian.org>, generated with the help of perlbug 1.40 running under perl 5.27.1. ----------------------------------------------------------------- As reported by Benjamin Bayart in https://bugs.debian.org/864782 a regression between 5.22 and 5.24 made s// on a valid utf8 string start to crash occasionally with 'Malformed UTF-8 character' diagnostics. I was able to get it down to #!perl my $text = "%t%\x{6bce}"; $text =~ s{~*%[a-z]%}{}g; print "ok\n"; which crashes for me on blead ~50% of the time with something like Malformed UTF-8 character: \x91 (unexpected continuation byte 0x91, with no preceding start byte) in substitution (s///) at /home/ntyni/864782.pl line 3. Malformed UTF-8 character (fatal) at /home/ntyni/864782.pl line 3. Running this with valgrind shows invalid reads, and bisecting indicates that those errors started with v5.23.3-110-g147f21b5b8 : commit 147f21b5b8054c559a1ffb568dbf310244fa0c91 Author: David Mitchell <davem@iabyn.com> Date: Sat Sep 26 13:12:40 2015 +0100 make /fixed-substr/ much faster. ----------------------------------------------------------------- --- Flags: category=core severity=medium --- Site configuration information for perl 5.27.1: Configured by ntyni at Wed Jun 14 20:16:03 BST 2017. Summary of my perl5 (revision 5 version 27 subversion 1) configuration: Commit id: b50535da2f4aaf97d13e96cda0069755fb6bbf76 Platform: osname=linux osvers=4.9.0-3-amd64 archname=x86_64-linux uname='linux carme 4.9.0-3-amd64 #1 smp debian 4.9.25-1 (2017-05-02) x86_64 gnulinux ' config_args='-des -Dusedevel -DDEBUGGING' hint=recommended useposix=true d_sigaction=define useithreads=undef usemultiplicity=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cc' ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' optimize='-O2 -g' cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='6.3.0 20170516' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.24.so so=so useshrplib=false libperl=libperl.a gnulibc_version='2.24' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong' --- @INC for perl 5.27.1: lib /usr/local/lib/perl5/site_perl/5.27.1/x86_64-linux /usr/local/lib/perl5/site_perl/5.27.1 /usr/local/lib/perl5/5.27.1/x86_64-linux /usr/local/lib/perl5/5.27.1 --- Environment for perl 5.27.1: HOME=/home/ntyni LANG=en_GB.UTF-8 LANGUAGE=en_GB:en LC_CTYPE=fi_FI.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/local/bin:/usr/bin:/bin:/usr/games PERL_BADLANG (unset) SHELL=/bin/zsh
Subject: Re: [perl #131575] s// on utf8 string occasionally crashes with "Malformed UTF-8 character"
To: perl5-porters [...] perl.org
Date: Fri, 16 Jun 2017 15:57:24 +0100
From: Dave Mitchell <davem [...] iabyn.com>
On Wed, Jun 14, 2017 at 01:00:12PM -0700, Niko Tyni wrote: Show quoted text
> As reported by Benjamin Bayart in https://bugs.debian.org/864782 a > regression between 5.22 and 5.24 made s// on a valid utf8 string start > to crash occasionally with 'Malformed UTF-8 character' diagnostics. > > I was able to get it down to > > #!perl > my $text = "%t%\x{6bce}"; > $text =~ s{~*%[a-z]%}{}g; > print "ok\n"; > > which crashes for me on blead ~50% of the time with something like > > Malformed UTF-8 character: \x91 (unexpected continuation byte 0x91, with no preceding start byte) in substitution (s///) at /home/ntyni/864782.pl line 3. > Malformed UTF-8 character (fatal) at /home/ntyni/864782.pl line 3. > > Running this with valgrind shows invalid reads, and bisecting > indicates that those errors started with v5.23.3-110-g147f21b5b8 : > > commit 147f21b5b8054c559a1ffb568dbf310244fa0c91 > Author: David Mitchell <davem@iabyn.com> > Date: Sat Sep 26 13:12:40 2015 +0100 > > make /fixed-substr/ much faster.
Now fixed by v5.27.0-322-gbb152a4 -- Monto Blanco... scorchio!
Download (untitled) / with headers
text/plain 317b
Thank you for filing this report. You have helped make Perl better. With the release yesterday of Perl 5.28.0, this and 185 other issues have been resolved. Perl 5.28.0 may be downloaded via: https://metacpan.org/release/XSAWYERX/perl-5.28.0 If you find that the problem persists, feel free to reopen this ticket.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org