Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent use of strings with code points over 0xFF as arguments to "vec" #17021

Closed
p5pRT opened this issue May 25, 2019 · 7 comments
Closed

Prevent use of strings with code points over 0xFF as arguments to "vec" #17021

p5pRT opened this issue May 25, 2019 · 7 comments
Milestone

Comments

@p5pRT
Copy link

p5pRT commented May 25, 2019

Migrated from rt.perl.org#134139 (status was 'pending release')

Searchable as RT134139$

@p5pRT
Copy link
Author

p5pRT commented May 25, 2019

From @jkeenan

pod/perldeprecation.pod contains this entry for perl-5.32​:

#####
Use of strings with code points over 0xFF as arguments to
"vec"

"vec" views its string argument as a sequence of bits. A
string containing a code point over 0xFF is nonsensical.
This usage is deprecated in Perl 5.28, and will be removed
in Perl 5.32.
#####

This entry was (mostly) added in this commit​:

#####
commit 76aae38
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Jun 1 16​:10​:48 2017 -0600

perldeprecation​: Add vec() with above-FF code points
#####

Make it so.

@p5pRT
Copy link
Author

p5pRT commented May 25, 2019

From @jkeenan

Summary of my perl5 (revision 5 version 31 subversion 0) configuration​:
  Commit id​: 58f4626
  Platform​:
  osname=linux
  osvers=4.15.0-50-generic
  archname=x86_64-linux
  uname='linux zareason 4.15.0-50-generic #54-ubuntu smp mon may 6 18​:46​:08 utc 2019 x86_64 x86_64 x86_64 gnulinux '
  config_args='-des -Dusedevel -Dusemymalloc=y'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=y
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
  optimize='-O2'
  cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='7.4.0'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/7/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64
  libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.27.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.27'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  MYMALLOC
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  Built under linux
  Compiled at May 22 2019 14​:58​:04
  %ENV​:
  PERL2DIR="/home/jkeenan/gitwork/perl2"
  PERLBREW_HOME="/home/jkeenan/.perlbrew"
  PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.28.0/man"
  PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin​:/home/jkeenan/perl5/perlbrew/perls/perl-5.28.0/bin"
  PERLBREW_PERL="perl-5.28.0"
  PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew"
  PERLBREW_SHELLRC_VERSION="0.84"
  PERLBREW_VERSION="0.84"
  PERL_WORKDIR="/home/jkeenan/gitwork/perl"
  @​INC​:
  lib
  /usr/local/lib/perl5/site_perl/5.31.0/x86_64-linux
  /usr/local/lib/perl5/site_perl/5.31.0
  /usr/local/lib/perl5/5.31.0/x86_64-linux
  /usr/local/lib/perl5/5.31.0

@p5pRT
Copy link
Author

p5pRT commented May 27, 2019

From @jkeenan

On Sat, 25 May 2019 02​:08​:59 GMT, jkeenan@​pobox.com wrote​:

pod/perldeprecation.pod contains this entry for perl-5.32​:

#####
Use of strings with code points over 0xFF as arguments to
"vec"

"vec" views its string argument as a sequence of bits. A
string containing a code point over 0xFF is nonsensical.
This usage is deprecated in Perl 5.28, and will be removed
in Perl 5.32.
#####

This entry was (mostly) added in this commit​:

#####
commit 76aae38
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Jun 1 16​:10​:48 2017 -0600

perldeprecation​: Add vec() with above-FF code points
#####

Make it so.

Please review the patch attached. This is available for smoking in the following branch​:

smoke-me/jkeenan/rt-134139-vec

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented May 27, 2019

From @jkeenan

0001-Use-of-strings-with-code-points-over-0xFF-as-argumen.patch
From b137856f094821c8ad330f6a5768544ae59555d0 Mon Sep 17 00:00:00 2001
From: James E Keenan <jkeenan@cpan.org>
Date: Sun, 26 May 2019 22:30:40 -0400
Subject: [PATCH] Use of strings with code points over 0xFF as arguments to
 "vec"

Implement scheduled fatalization.  Adapt existing tests in t/op/vec.t.
Eliminate t/lib/warnings/doop and move one test to t/op/vec.t.

Document this fatalization in perldiag and perlfunc.

For: RT # 134139
---
 MANIFEST            |  1 -
 doop.c              |  5 +----
 pod/perldiag.pod    |  7 +++----
 pod/perlfunc.pod    |  6 ++----
 t/lib/warnings/doop | 14 --------------
 t/op/vec.t          | 26 +++++++++++++++++---------
 6 files changed, 23 insertions(+), 36 deletions(-)
 delete mode 100644 t/lib/warnings/doop

diff --git a/MANIFEST b/MANIFEST
index de891fc8ff..830340475f 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -5507,7 +5507,6 @@ t/lib/warnings/9enabled		Tests warnings
 t/lib/warnings/9uninit		Tests "Use of uninitialized" warnings
 t/lib/warnings/av		Tests for av.c for warnings.t
 t/lib/warnings/doio		Tests for doio.c for warnings.t
-t/lib/warnings/doop		Tests for doop.c for warnings.t
 t/lib/warnings/gv		Tests for gv.c for warnings.t
 t/lib/warnings/hv		Tests for hv.c for warnings.t
 t/lib/warnings/malloc		Tests for malloc.c for warnings.t
diff --git a/doop.c b/doop.c
index 54e35f10a6..3b4377c652 100644
--- a/doop.c
+++ b/doop.c
@@ -763,10 +763,7 @@ Perl_do_vecget(pTHX_ SV *sv, STRLEN offset, int size)
             s = (unsigned char *) SvPV_flags(sv, srclen, svpv_flags);
         }
         else {
-            Perl_ck_warner_d(aTHX_ packWARN(WARN_DEPRECATED),
-                                "Use of strings with code points over 0xFF as"
-                                " arguments to vec is deprecated. This will"
-                                " be a fatal error in Perl 5.32");
+	        Perl_croak(aTHX_ "Use of strings with code points over 0xFF as arguments to vec is forbidden");
         }
     }
 
diff --git a/pod/perldiag.pod b/pod/perldiag.pod
index f69b1b8367..11339f0e9b 100644
--- a/pod/perldiag.pod
+++ b/pod/perldiag.pod
@@ -7340,13 +7340,12 @@ operators treat their operands as strings of bytes, and values beyond
 
 This became fatal in Perl 5.28.
 
-=item Use of strings with code points over 0xFF as arguments to C<vec>
-is deprecated. This will be a fatal error in Perl 5.32
+=item Use of strings with code points over 0xFF as arguments to vec is forbidden
 
-(D deprecated) You tried to use L<C<vec>|perlfunc/vec EXPR,OFFSET,BITS>
+(F) You tried to use L<C<vec>|perlfunc/vec EXPR,OFFSET,BITS>
 on a string containing a code point over 0xFF, which is nonsensical here.
 
-Such usage will be a fatal error in Perl 5.32.
+This became fatal in Perl 5.32.
 
 =item Use of tainted arguments in %s is deprecated
 
diff --git a/pod/perlfunc.pod b/pod/perlfunc.pod
index 196a88d71f..0e50132907 100644
--- a/pod/perlfunc.pod
+++ b/pod/perlfunc.pod
@@ -9669,10 +9669,8 @@ to try to write off the beginning of the string (i.e., negative OFFSET).
 If the string happens to be encoded as UTF-8 internally (and thus has
 the UTF8 flag set), L<C<vec>|/vec EXPR,OFFSET,BITS> tries to convert it
 to use a one-byte-per-character internal representation. However, if the
-string contains characters with values of 256 or higher, that conversion
-will fail, and a deprecation message will be raised.  In that situation,
-C<vec> will operate on the underlying buffer regardless, in its internal
-UTF-8 representation.  In Perl 5.32, this will be a fatal error.
+string contains characters with values of 256 or higher, a fatal error
+will occur.
 
 Strings created with L<C<vec>|/vec EXPR,OFFSET,BITS> can also be
 manipulated with the logical
diff --git a/t/lib/warnings/doop b/t/lib/warnings/doop
deleted file mode 100644
index 09db146737..0000000000
--- a/t/lib/warnings/doop
+++ /dev/null
@@ -1,14 +0,0 @@
-__END__
-# doop.c
-use utf8 ;
-$_ = "\x80  \xff" ;
-chop ;
-EXPECT
-########
-# NAME vec with above ff code points is deprecated
-my $foo = "\x{100}" . "\xff\xfe";
-eval { vec($foo, 1, 8) };
-no warnings 'deprecated';
-eval { vec($foo, 1, 8) };
-EXPECT
-Use of strings with code points over 0xFF as arguments to vec is deprecated. This will be a fatal error in Perl 5.32 at - line 2.
diff --git a/t/op/vec.t b/t/op/vec.t
index d3bb57dcd7..aa965785d3 100644
--- a/t/op/vec.t
+++ b/t/op/vec.t
@@ -10,6 +10,7 @@ use Config;
 
 plan(tests => 78);
 
+my $exception_134139 = "Use of strings with code points over 0xFF as arguments to vec is forbidden";
 
 is(vec($foo,0,1), 0);
 is(length($foo), undef);
@@ -65,18 +66,15 @@ $x = substr $foo, 1;
 is(vec($x, 0, 8), 255);
 $@ = undef;
 {
-    no warnings 'deprecated';
+    local $@;
     eval { vec($foo, 1, 8) };
-    ok(! $@);
+    like($@, qr/$exception_134139/,
+        "Caught exception: code point over 0xFF used as argument to vec");
     $@ = undef;
     eval { vec($foo, 1, 8) = 13 };
-    ok(! $@);
-    if ($::IS_EBCDIC) {
-        is($foo, "\x8c\x0d\xff\x8a\x69");
-    }
-    else {
-        is($foo, "\xc4\x0d\xc3\xbf\xc3\xbe");
-    }
+    #ok(! $@);
+    like($@, qr/$exception_134139/,
+        "Caught exception: code point over 0xFF used as argument to vec");
 }
 $foo = "\x{100}" . "\xff\xfe";
 $x = substr $foo, 1;
@@ -244,3 +242,13 @@ like($@, qr/^Modification of a read-only value attempted at /,
     $v = eval { RT131083(1, vec($s, $off, 8)); };
     like($@, qr/Out of memory!/, "RT131083 lval ~0");
 }
+
+{
+    # Adapting test formerly in t/lib/warnings/doop
+
+    local $@;
+    my $foo = "\x{100}" . "\xff\xfe";
+    eval { vec($foo, 1, 8) };
+    like($@, qr/$exception_134139/,
+        "RT 134139: Use of strings with code points over 0xFF as arguments to 'vec' is now forbidden");
+}
-- 
2.17.1

@p5pRT
Copy link
Author

p5pRT commented May 27, 2019

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented May 30, 2019

From @jkeenan

On Mon, 27 May 2019 14​:08​:14 GMT, jkeenan wrote​:

On Sat, 25 May 2019 02​:08​:59 GMT, jkeenan@​pobox.com wrote​:

pod/perldeprecation.pod contains this entry for perl-5.32​:

#####
Use of strings with code points over 0xFF as arguments to
"vec"

"vec" views its string argument as a sequence of bits. A
string containing a code point over 0xFF is nonsensical.
This usage is deprecated in Perl 5.28, and will be removed
in Perl 5.32.
#####

This entry was (mostly) added in this commit​:

#####
commit 76aae38
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu Jun 1 16​:10​:48 2017 -0600

perldeprecation​: Add vec() with above-FF code points
#####

Make it so.

Please review the patch attached. This is available for smoking in
the following branch​:

smoke-me/jkeenan/rt-134139-vec

Thank you very much.

Reviewed by khw. Merged to blead with minor edits in commit da5a0da.

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented May 30, 2019

@jkeenan - Status changed from 'open' to 'pending release'

@p5pRT p5pRT closed this as completed May 30, 2019
@toddr toddr added this to the 5.32.0 milestone Oct 25, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants