Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Malformed UTF-8 character in bitwise xor when using $1 #9972

Closed
p5pRT opened this issue Nov 19, 2009 · 11 comments
Closed

Malformed UTF-8 character in bitwise xor when using $1 #9972

p5pRT opened this issue Nov 19, 2009 · 11 comments

Comments

@p5pRT
Copy link

p5pRT commented Nov 19, 2009

Migrated from rt.perl.org#70652 (status was 'open')

Searchable as RT70652$

@p5pRT
Copy link
Author

p5pRT commented Nov 19, 2009

From perl@marc-s.de

This is a bug report for perl from perl@​marc-s.de,
generated with the help of perlbug 1.35 running under perl v5.8.8.

#!/usr/bin/perl -w

use warnings;
use strict;

#No errors when using bytes!!!
#use bytes;

#Unicode bug

my $unicodestring = "\x{5454}\x{6655}";
my $normalstring="0\36\4\13\200\0\31V\3\0\320\225\342\26\365\4\0\240\r\2\3\0\242_\2\1\0\2\1\0000\0\b\b\b\b\b\b\b\b";
my $iv="\246\205\236\367]\257\304\276";

print "First we need \$1 to be unicode, otherwise the bug won't occour​:\n";
$unicodestring=~/(.)/;
print "\$1 is assigned but not yet unicode​: UTF8-Flag(\$1)​: ",utf8​::is_utf8($1),"\n";
my $copy=$1;
print "After we copy \$1 the Flag is on​: UTF8-Flag(\$1)​: ",utf8​::is_utf8($1),"\n";

print "\nNow we take 8 Bytes of a normal string with m/(.{8})/\n";
print "\$normalstring​: UTF8-Flag​: ",utf8​::is_utf8($normalstring),"\n";

$normalstring=~/(.{8})/;
print "The UTF-8 Flag of \$1 is still on​: UTF8-Flag(\$1)​: ",utf8​::is_utf8($1),"\n";
print "We have a second value called (\$iv) without an UTF-8 Flag : UTF8-Flag(\$iv)​: ",utf8​::is_utf8($iv),"\n";
print "Now the UTF-8 Flag of \$1 is off​: UTF8-Flag(\$1)​: ",utf8​::is_utf8($1),"\n";
print "Then we do a bitwise ^ opreation of both into \$x (my \$x=\$1 ^ \$iv), which results in a warning.\n\n\n";
# Uncomment this for a fix​:
#my $c=$1;

my $x=$1 ^ $iv;
print "\n\n\$1 is now not UTF-8 anymore UTF8-Flag (\$1)​: ",utf8​::is_utf8($1),"\n";
print "\$x is now UTF-8​: UTF8-Flag (\$x)​: ",utf8​::is_utf8($x),"\n";
print "\$iv suddenly is also UTF-8​: UTF8-Flag (\$iv)​: ",utf8​::is_utf8($iv),"\n";

print "How come 2 non UTF-8 flagged scalars suddenly turn into UTF-8 when a previously UTF-8 flagged \$1 is involved?\n";
print "This will also not happen if \$1 was not UTF-8 before the second m//";

print "This will not happen if we copy \$1 first into another variable or print \$1 before the operation\nThen the UTF-8 Flag will be removed\n";

# The problem occours in Net​::SNMP's encrypt_des routine.


Flags​:
  category=core
  severity=low


Site configuration information for perl v5.8.8​:

Configured by Debian Project at Fri Dec 19 00​:43​:54 EST 2008.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration​:
  Platform​:
  osname=linux, osvers=2.6.18-6-686, archname=i486-linux-gnu-thread-multi
  uname='linux etch 2.6.18-6-686 #1 smp fri dec 12 16​:48​:28 utc 2008 i686 gnulinux '
  config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.8 -Dsitearch=/usr/local/lib/perl/5.8.8 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.8 -Dd_dosuid -des'
  hint=recommended, useposix=true, d_sigaction=define
  usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
  useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
  use64bitint=undef use64bitall=undef uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include'
  ccversion='', gccversion='4.1.2 20061115 (prerelease) (Debian 4.1.1-21)', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=4, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -L/usr/local/lib'
  libpth=/usr/local/lib /lib /usr/lib
  libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
  perllibs=-ldl -lm -lpthread -lc -lcrypt
  libc=/lib/libc-2.3.6.so, so=so, useshrplib=true, libperl=libperl.so.5.8.8
  gnulibc_version='2.3.6'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches​:
 


@​INC for perl v5.8.8​:
  /etc/perl
  /usr/local/lib/perl/5.8.8
  /usr/local/share/perl/5.8.8
  /usr/lib/perl5
  /usr/share/perl5
  /usr/lib/perl/5.8
  /usr/share/perl/5.8
  /usr/local/lib/site_perl
  .


Environment for perl v5.8.8​:
  HOME=/root
  LANG=de_DE.UTF-8
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/sbin​:/bin
  PERL_BADLANG (unset)
  SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Nov 20, 2009

From zefram@fysh.org

perl@​marc-s.de (via RT) wrote​:

# The problem occours in Net​::SNMP's encrypt_des routine.

In that case Net​::SNMP has a bug. Bitwise ^ on a string containing
non-octets is not a sane operation for cryptographic use. You want to
either limit the input to Latin-1 (or ASCII) or explicitly encode it
into octets using UTF-8/UTF-16/whatever.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Nov 20, 2009

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Nov 20, 2009

From @ap

* Zefram <zefram@​fysh.org> [2009-11-20 11​:20]​:

Bitwise ^ on a string containing non-octets is not a sane
operation for cryptographic use.

Is it a sane operation at all? Should it be permitted? Silently?

(Wondering aloud.)

Regards,
--
Aristotle Pagaltzis // <http​://plasmasturm.org/>

@p5pRT
Copy link
Author

p5pRT commented Nov 20, 2009

From @ikegami

On Thu, Nov 19, 2009 at 4​:14 PM, perl@​marc-s.de (via RT) <
perlbug-followup@​perl.org> wrote​:

# New Ticket Created by perl@​marc-s.de
# Please include the string​: [perl #70652]
# in the subject line of all future correspondence about this issue.
# <URL​: http​://rt.perl.org/rt3/Ticket/Display.html?id=70652 >

This is a bug report for perl from perl@​marc-s.de,
generated with the help of perlbug 1.35 running under perl v5.8.8.

I can't replicate the behaviour you describe in 5.8.0 (had to get is_utf8
from Encode), 5.8.8, 5.8.9, 5.10.0 or 5.10.1. Specifically,

#!/usr/bin/perl -w

use warnings;
use strict;

#No errors when using bytes!!!
#use bytes;

#Unicode bug

my $unicodestring = "\x{5454}\x{6655}";
my
$normalstring="0\36\4\13\200\0\31V\3\0\320\225\342\26\365\4\0\240\r\2\3\0\242_\2\1\0\2\1\0000\0\b\b\b\b\b\b\b\b";
my $iv="\246\205\236\367]\257\304\276";

print "First we need \$1 to be unicode, otherwise the bug won't occour​:\n";
$unicodestring=~/(.)/;
print "\$1 is assigned but not yet unicode​: UTF8-Flag(\$1)​:
",utf8​::is_utf8($1),"\n";
my $copy=$1;
print "After we copy \$1 the Flag is on​: UTF8-Flag(\$1)​:
",utf8​::is_utf8($1),"\n";

print "\nNow we take 8 Bytes of a normal string with m/(.{8})/\n";
print "\$normalstring​: UTF8-Flag​: ",utf8​::is_utf8($normalstring),"\n";

$normalstring=~/(.{8})/;
print "The UTF-8 Flag of \$1 is still on​: UTF8-Flag(\$1)​:
",utf8​::is_utf8($1),"\n";
print "We have a second value called (\$iv) without an UTF-8 Flag :
UTF8-Flag(\$iv)​: ",utf8​::is_utf8($iv),"\n";
print "Now the UTF-8 Flag of \$1 is off​: UTF8-Flag(\$1)​:
",utf8​::is_utf8($1),"\n";

No​: Now the UTF-8 Flag of $1 is off​: UTF8-Flag($1)​: 1

print "Then we do a bitwise ^ opreation of both into \$x (my \$x=\$1 ^
\$iv), which results in a warning.\n\n\n";

No warning

# Uncomment this for a fix​:
#my $c=$1;

my $x=$1 ^ $iv;
print "\n\n\$1 is now not UTF-8 anymore UTF8-Flag (\$1)​:
",utf8​::is_utf8($1),"\n";
print "\$x is now UTF-8​: UTF8-Flag (\$x)​: ",utf8​::is_utf8($x),"\n";

No​: $x is now UTF-8​: UTF8-Flag ($x)​:

print "\$iv suddenly is also UTF-8​: UTF8-Flag (\$iv)​:
",utf8​::is_utf8($iv),"\n";

No​: $iv suddenly is also UTF-8​: UTF8-Flag ($iv)​:

print "How come 2 non UTF-8 flagged scalars suddenly turn into UTF-8 when a
previously UTF-8 flagged \$1 is involved?\n";
print "This will also not happen if \$1 was not UTF-8 before the second
m//";

print "This will not happen if we copy \$1 first into another variable or
print \$1 before the operation\nThen the UTF-8 Flag will be removed\n";

@p5pRT
Copy link
Author

p5pRT commented Aug 17, 2016

From @dcollinsn

I am able to partially reproduce this on 5.8.8, even less partially reproduce it on 5.10 and 5.12, and it appears to have been "fixed" in 5.14. My 5.8.8 differs from the reporter's in that $1 never stops being UTF8. In 5.10.0, the bitwise warning disappears, and nothing is UTF8 after that line. In 5.14.0, $1 stops being UTF8 as well

  1) UTF-8 flag of $1 after m//
  2) UTF-8 flag of $1 after utf8​::is_utf8($iv)
  3) Warning after bitwise ^
  4) UTF-8 flag of $1 after ^
  5) UTF-8 flag of $x after ^
  6) UTF-8 flag of $iv after ^

  +----------------------+---+---+---+---+---+---+
  | TEST | 1 | 2 | 3 | 4 | 5 | 6 |
  +----------------------+---+---+---+---+---+---+
  | 5.8.8 (Expected?) | 0 | 0 | 0 | 0 | 0 | 0 |
  | 5.8.8 (Reporter) | 1 | 0 | 1 | 0 | 1 | 1 |
  | 5.8.8 (Mine) | 1 | 1 | 1 | 0 | 1 | 1 |
  | 5.10.0 (Mine) | 1 | 1 | 0 | 0 | 0 | 0 |
  | 5.12.0 (Mine) | 1 | 1 | 0 | 0 | 0 | 0 |
  | 5.14.0 (Mine) | 0 | 0 | 0 | 0 | 0 | 0 |
  | ...no changes... | | | | | | |
  | 5.25.2 (Mine) | 0 | 0 | 0 | 0 | 0 | 0 |
  +----------------------+---+---+---+---+---+---+

I'm choosing not to bisect this because there are at least two steps to this fix, and perl versions before 5.12ish don't build on my computer without manual assistance. It does appear that all the issues raised by this ticket have been resolved in every currently maintained perl.

--
Respectfully,
Dan Collins

@p5pRT
Copy link
Author

p5pRT commented Aug 17, 2016

@dcollinsn - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this as completed Aug 17, 2016
@p5pRT
Copy link
Author

p5pRT commented Aug 17, 2016

From @xsawyerx

On 08/17/2016 05​:12 PM, Dan Collins via RT wrote​:

[...]
I'm choosing not to bisect this because there are at least two steps to this fix, and perl versions before 5.12ish don't build on my computer without manual assistance. It does appear that all the issues raised by this ticket have been resolved in every currently maintained perl.

If it's fixed in all the supported versions (heck, since 5.14), I don't
see a reason to not just resolve this issue.

@p5pRT
Copy link
Author

p5pRT commented Aug 17, 2016

From @Tux

On Wed, 17 Aug 2016 17​:15​:11 +0200, Sawyer X <xsawyerx@​gmail.com> wrote​:

On 08/17/2016 05​:12 PM, Dan Collins via RT wrote​:

[...]
I'm choosing not to bisect this because there are at least two
steps to this fix, and perl versions before 5.12ish don't build on
my computer without manual assistance. It does appear that all the
issues raised by this ticket have been resolved in every currently
maintained perl.

If it's fixed in all the supported versions (heck, since 5.14), I
don't see a reason to not just resolve this issue.

For completeness sake​:

$ cat rt70652.pl
use strict;
use warnings;

my $unicodestring = "\x{5454}\x{6655}";
my $normalstring = "0\36\4\13\200\0\31V\3\0\320\225\342\26\365\4\0\240\r\2\3\0\242_\2\1\0\2\1\0000\0\b\b\b\b\b\b\b\b";
my $iv = "\246\205\236\367]\257\304\276";

# First we need $1 to be unicode, otherwise the bug won't occur
$unicodestring =~ m/(.)/;

my @​t;

# $1 is assigned but not yet unicode​: UTF8-Flag ($1)
push @​t, utf8​::is_utf8 ($1);

# After we copy $1 the Flag is on​: UTF8-Flag ($1)
my $copy = $1;
push @​t, utf8​::is_utf8 ($1);

# Now we take 8 Bytes of a normal string with m/(.{8})/
push @​t, utf8​::is_utf8 ($normalstring);

$normalstring =~ m/(.{8})/;

# The UTF-8 Flag of $1 is still on​: UTF8-Flag ($1)
push @​t, utf8​::is_utf8 ($1);
# We have a second value called ($iv) without an UTF-8 Flag : UTF8-Flag ($iv)
push @​t, utf8​::is_utf8 ($iv);

# Now the UTF-8 Flag of $1 is off​: UTF8-Flag ($1)
push @​t, utf8​::is_utf8 ($1);

my $x = $1 ^ $iv;
# $1 is now not UTF-8 anymore UTF8-Flag ($1)
push @​t, utf8​::is_utf8 ($1);
# $x is now UTF-8​: UTF8-Flag ($x)
push @​t, utf8​::is_utf8 ($x);
# $iv suddenly is also UTF-8​: UTF8-Flag ($iv)
push @​t, utf8​::is_utf8 ($iv);

print join "​:" => $], @​t, "\n";

5.6.0 .. 5.8.0 will warn and die​:
Undefined subroutine &utf8​::is_utf8 called at rt70652.pl line 16.

5.8.1 .. 5.8.8 will warn​:
Malformed UTF-8 character (unexpected continuation byte 0x80, with no preceding start byte) in bitwise xor (^) at rt70652.pl line 35.

$ perl-all rt70652.pl | & grep ^5 | sort -u
5.008001​::1​::1​::1​::1​:1​:
5.008002​::1​::1​::1​::1​:1​:
5.008003​::1​::1​::1​::1​:1​:
5.008004​::1​::1​::1​::1​:1​:
5.008005​::1​::1​::1​::1​:1​:
5.008006​::1​::1​::1​::1​:1​:
5.008007​::1​::1​::1​::1​:1​:
5.008008​::1​::1​::1​::1​:1​:
5.008009​::1​::1​::1​::​::
5.010000​::1​::1​::1​::​::
5.010001​::1​::1​::1​::​::
5.011000​::1​::1​::1​::​::
5.011001​::1​::1​::1​::​::
5.011002​::1​::1​::1​::​::
5.011003​::1​::1​::1​::​::
5.011004​::1​::1​::1​::​::
5.011005​::1​::1​::1​::​::
5.012000​::1​::1​::1​::​::
5.012001​:1​:1​::​::​::​::
5.012002​:1​:1​::​::​::​::
5.012003​:1​:1​::​::​::​::
5.012004​:1​:1​::​::​::​::
5.012005​:1​:1​::​::​::​::
5.013000​::1​::1​::1​::​::
5.013001​:1​:1​::​::​::​::
5.013002​:1​:1​::​::​::​::
5.013003​:1​:1​::​::​::​::
5.013004​:1​:1​::​::​::​::
5.013005​:1​:1​::​::​::​::
5.013006​:1​:1​::​::​::​::
5.013007​:1​:1​::​::​::​::
5.013008​:1​:1​::​::​::​::
5.013009​:1​:1​::​::​::​::
5.013010​:1​:1​::​::​::​::
5.013011​:1​:1​::​::​::​::
5.014000​:1​:1​::​::​::​::
5.014001​:1​:1​::​::​::​::
5.014002​:1​:1​::​::​::​::
5.014003​:1​:1​::​::​::​::
5.014004​:1​:1​::​::​::​::
5.015000​:1​:1​::​::​::​::
5.015001​:1​:1​::​::​::​::
5.015002​:1​:1​::​::​::​::
5.015003​:1​:1​::​::​::​::
5.015004​:1​:1​::​::​::​::
5.015005​:1​:1​::​::​::​::
5.015006​:1​:1​::​::​::​::
5.015007​:1​:1​::​::​::​::
5.015008​:1​:1​::​::​::​::
5.015009​:1​:1​::​::​::​::
5.016000​:1​:1​::​::​::​::
5.016001​:1​:1​::​::​::​::
5.016002​:1​:1​::​::​::​::
5.016003​:1​:1​::​::​::​::
5.017000​:1​:1​::​::​::​::
5.017001​:1​:1​::​::​::​::
5.017002​:1​:1​::​::​::​::
5.017003​:1​:1​::​::​::​::
5.017004​:1​:1​::​::​::​::
5.017005​:1​:1​::​::​::​::
5.017006​:1​:1​::​::​::​::
5.017007​:1​:1​::​::​::​::
5.017008​:1​:1​::​::​::​::
5.017009​:1​:1​::​::​::​::
5.017010​:1​:1​::​::​::​::
5.017011​:1​:1​::​::​::​::
5.018000​:1​:1​::​::​::​::
5.018001​:1​:1​::​::​::​::
5.018002​:1​:1​::​::​::​::
5.018003​:1​:1​::​::​::​::
5.018004​:1​:1​::​::​::​::
5.019000​:1​:1​::​::​::​::
5.019001​:1​:1​::​::​::​::
5.019002​:1​:1​::​::​::​::
5.019003​:1​:1​::​::​::​::
5.019004​:1​:1​::​::​::​::
5.019005​:1​:1​::​::​::​::
5.019006​:1​:1​::​::​::​::
5.019007​:1​:1​::​::​::​::
5.019008​:1​:1​::​::​::​::
5.019009​:1​:1​::​::​::​::
5.019010​:1​:1​::​::​::​::
5.019011​:1​:1​::​::​::​::
5.020000​:1​:1​::​::​::​::
5.020001​:1​:1​::​::​::​::
5.020002​:1​:1​::​::​::​::
5.020003​:1​:1​::​::​::​::
5.021000​:1​:1​::​::​::​::
5.021001​:1​:1​::​::​::​::
5.021002​:1​:1​::​::​::​::
5.021003​:1​:1​::​::​::​::
5.021004​:1​:1​::​::​::​::
5.021005​:1​:1​::​::​::​::
5.021006​:1​:1​::​::​::​::
5.021007​:1​:1​::​::​::​::
5.021008​:1​:1​::​::​::​::
5.021009​:1​:1​::​::​::​::
5.021010​:1​:1​::​::​::​::
5.021011​:1​:1​::​::​::​::
5.022000​:1​:1​::​::​::​::
5.022001​:1​:1​::​::​::​::
5.022002​:1​:1​::​::​::​::
5.022003​:1​:1​::​::​::​::
5.023000​:1​:1​::​::​::​::
5.023001​:1​:1​::​::​::​::
5.023002​:1​:1​::​::​::​::
5.023003​:1​:1​::​::​::​::
5.023004​:1​:1​::​::​::​::
5.023005​:1​:1​::​::​::​::
5.023006​:1​:1​::​::​::​::
5.023007​:1​:1​::​::​::​::
5.023008​:1​:1​::​::​::​::
5.023009​:1​:1​::​::​::​::
5.024000​:1​:1​::​::​::​::
5.024001​:1​:1​::​::​::​::
5.025000​:1​:1​::​::​::​::
5.025001​:1​:1​::​::​::​::
5.025002​:1​:1​::​::​::​::
5.025003​:1​:1​::​::​::​::

--
H.Merijn Brand http​://tux.nl Perl Monger http​://amsterdam.pm.org/
using perl5.00307 .. 5.25 porting perl5 on HP-UX, AIX, and openSUSE
http​://mirrors.develooper.com/hpux/ http​://www.test-smoke.org/
http​://qa.perl.org http​://www.goldmark.org/jeff/stupid-disclaimers/

@p5pRT
Copy link
Author

p5pRT commented Aug 17, 2016

From @xsawyerx

[Top-posted]

We can still add this test as a regression test.

On 08/17/2016 05​:46 PM, H.Merijn Brand wrote​:

On Wed, 17 Aug 2016 17​:15​:11 +0200, Sawyer X <xsawyerx@​gmail.com> wrote​:

On 08/17/2016 05​:12 PM, Dan Collins via RT wrote​:

[...]
I'm choosing not to bisect this because there are at least two
steps to this fix, and perl versions before 5.12ish don't build on
my computer without manual assistance. It does appear that all the
issues raised by this ticket have been resolved in every currently
maintained perl.
If it's fixed in all the supported versions (heck, since 5.14), I
don't see a reason to not just resolve this issue.
For completeness sake​:

$ cat rt70652.pl
use strict;
use warnings;

my $unicodestring = "\x{5454}\x{6655}";
my $normalstring = "0\36\4\13\200\0\31V\3\0\320\225\342\26\365\4\0\240\r\2\3\0\242_\2\1\0\2\1\0000\0\b\b\b\b\b\b\b\b";
my $iv = "\246\205\236\367]\257\304\276";

# First we need $1 to be unicode, otherwise the bug won't occur
$unicodestring =~ m/(.)/;

my @​t;

# $1 is assigned but not yet unicode​: UTF8-Flag ($1)
push @​t, utf8​::is_utf8 ($1);

# After we copy $1 the Flag is on​: UTF8-Flag ($1)
my $copy = $1;
push @​t, utf8​::is_utf8 ($1);

# Now we take 8 Bytes of a normal string with m/(.{8})/
push @​t, utf8​::is_utf8 ($normalstring);

$normalstring =~ m/(.{8})/;

# The UTF-8 Flag of $1 is still on​: UTF8-Flag ($1)
push @​t, utf8​::is_utf8 ($1);
# We have a second value called ($iv) without an UTF-8 Flag : UTF8-Flag ($iv)
push @​t, utf8​::is_utf8 ($iv);

# Now the UTF-8 Flag of $1 is off​: UTF8-Flag ($1)
push @​t, utf8​::is_utf8 ($1);

my $x = $1 ^ $iv;
# $1 is now not UTF-8 anymore UTF8-Flag ($1)
push @​t, utf8​::is_utf8 ($1);
# $x is now UTF-8​: UTF8-Flag ($x)
push @​t, utf8​::is_utf8 ($x);
# $iv suddenly is also UTF-8​: UTF8-Flag ($iv)
push @​t, utf8​::is_utf8 ($iv);

print join "​:" => $], @​t, "\n";

5.6.0 .. 5.8.0 will warn and die​:
Undefined subroutine &utf8​::is_utf8 called at rt70652.pl line 16.

5.8.1 .. 5.8.8 will warn​:
Malformed UTF-8 character (unexpected continuation byte 0x80, with no preceding start byte) in bitwise xor (^) at rt70652.pl line 35.

$ perl-all rt70652.pl | & grep ^5 | sort -u
5.008001​::1​::1​::1​::1​:1​:
5.008002​::1​::1​::1​::1​:1​:
5.008003​::1​::1​::1​::1​:1​:
5.008004​::1​::1​::1​::1​:1​:
5.008005​::1​::1​::1​::1​:1​:
5.008006​::1​::1​::1​::1​:1​:
5.008007​::1​::1​::1​::1​:1​:
5.008008​::1​::1​::1​::1​:1​:
5.008009​::1​::1​::1​::​::
5.010000​::1​::1​::1​::​::
5.010001​::1​::1​::1​::​::
5.011000​::1​::1​::1​::​::
5.011001​::1​::1​::1​::​::
5.011002​::1​::1​::1​::​::
5.011003​::1​::1​::1​::​::
5.011004​::1​::1​::1​::​::
5.011005​::1​::1​::1​::​::
5.012000​::1​::1​::1​::​::
5.012001​:1​:1​::​::​::​::
5.012002​:1​:1​::​::​::​::
5.012003​:1​:1​::​::​::​::
5.012004​:1​:1​::​::​::​::
5.012005​:1​:1​::​::​::​::
5.013000​::1​::1​::1​::​::
5.013001​:1​:1​::​::​::​::
5.013002​:1​:1​::​::​::​::
5.013003​:1​:1​::​::​::​::
5.013004​:1​:1​::​::​::​::
5.013005​:1​:1​::​::​::​::
5.013006​:1​:1​::​::​::​::
5.013007​:1​:1​::​::​::​::
5.013008​:1​:1​::​::​::​::
5.013009​:1​:1​::​::​::​::
5.013010​:1​:1​::​::​::​::
5.013011​:1​:1​::​::​::​::
5.014000​:1​:1​::​::​::​::
5.014001​:1​:1​::​::​::​::
5.014002​:1​:1​::​::​::​::
5.014003​:1​:1​::​::​::​::
5.014004​:1​:1​::​::​::​::
5.015000​:1​:1​::​::​::​::
5.015001​:1​:1​::​::​::​::
5.015002​:1​:1​::​::​::​::
5.015003​:1​:1​::​::​::​::
5.015004​:1​:1​::​::​::​::
5.015005​:1​:1​::​::​::​::
5.015006​:1​:1​::​::​::​::
5.015007​:1​:1​::​::​::​::
5.015008​:1​:1​::​::​::​::
5.015009​:1​:1​::​::​::​::
5.016000​:1​:1​::​::​::​::
5.016001​:1​:1​::​::​::​::
5.016002​:1​:1​::​::​::​::
5.016003​:1​:1​::​::​::​::
5.017000​:1​:1​::​::​::​::
5.017001​:1​:1​::​::​::​::
5.017002​:1​:1​::​::​::​::
5.017003​:1​:1​::​::​::​::
5.017004​:1​:1​::​::​::​::
5.017005​:1​:1​::​::​::​::
5.017006​:1​:1​::​::​::​::
5.017007​:1​:1​::​::​::​::
5.017008​:1​:1​::​::​::​::
5.017009​:1​:1​::​::​::​::
5.017010​:1​:1​::​::​::​::
5.017011​:1​:1​::​::​::​::
5.018000​:1​:1​::​::​::​::
5.018001​:1​:1​::​::​::​::
5.018002​:1​:1​::​::​::​::
5.018003​:1​:1​::​::​::​::
5.018004​:1​:1​::​::​::​::
5.019000​:1​:1​::​::​::​::
5.019001​:1​:1​::​::​::​::
5.019002​:1​:1​::​::​::​::
5.019003​:1​:1​::​::​::​::
5.019004​:1​:1​::​::​::​::
5.019005​:1​:1​::​::​::​::
5.019006​:1​:1​::​::​::​::
5.019007​:1​:1​::​::​::​::
5.019008​:1​:1​::​::​::​::
5.019009​:1​:1​::​::​::​::
5.019010​:1​:1​::​::​::​::
5.019011​:1​:1​::​::​::​::
5.020000​:1​:1​::​::​::​::
5.020001​:1​:1​::​::​::​::
5.020002​:1​:1​::​::​::​::
5.020003​:1​:1​::​::​::​::
5.021000​:1​:1​::​::​::​::
5.021001​:1​:1​::​::​::​::
5.021002​:1​:1​::​::​::​::
5.021003​:1​:1​::​::​::​::
5.021004​:1​:1​::​::​::​::
5.021005​:1​:1​::​::​::​::
5.021006​:1​:1​::​::​::​::
5.021007​:1​:1​::​::​::​::
5.021008​:1​:1​::​::​::​::
5.021009​:1​:1​::​::​::​::
5.021010​:1​:1​::​::​::​::
5.021011​:1​:1​::​::​::​::
5.022000​:1​:1​::​::​::​::
5.022001​:1​:1​::​::​::​::
5.022002​:1​:1​::​::​::​::
5.022003​:1​:1​::​::​::​::
5.023000​:1​:1​::​::​::​::
5.023001​:1​:1​::​::​::​::
5.023002​:1​:1​::​::​::​::
5.023003​:1​:1​::​::​::​::
5.023004​:1​:1​::​::​::​::
5.023005​:1​:1​::​::​::​::
5.023006​:1​:1​::​::​::​::
5.023007​:1​:1​::​::​::​::
5.023008​:1​:1​::​::​::​::
5.023009​:1​:1​::​::​::​::
5.024000​:1​:1​::​::​::​::
5.024001​:1​:1​::​::​::​::
5.025000​:1​:1​::​::​::​::
5.025001​:1​:1​::​::​::​::
5.025002​:1​:1​::​::​::​::
5.025003​:1​:1​::​::​::​::

@p5pRT
Copy link
Author

p5pRT commented Aug 17, 2016

@dcollinsn - Status changed from 'resolved' to 'open'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant