perluniprops out of date regarding \p{Word} #16255

p5pRT · 2017-11-18T05:01:31Z

Migrated from rt.perl.org#132463 (status was 'resolved')

Searchable as RT132463$

p5pRT · 2017-11-18T05:01:31Z

From @mauke

Created by @mauke

According to perluniprops, \w (a.k.a. \p{Word}) consists of \p{Alnum} +
\pM + \p{Pc}.

But in 5.18 \w was changed to also match U+200C and U+200D. This is
mentioned in perl5180delta.

perluniprops should be updated to match the current code.

Perl Info


Flags:
    category=docs
    severity=low

Site configuration information for perl 5.26.0:

Configured by mauke at Fri Sep 22 13:28:36 CEST 2017.

Summary of my perl5 (revision 5 version 26 subversion 0) configuration:
   
  Platform:
    osname=linux
    osvers=4.9.41-1-lts
    archname=i686-linux
    uname='linux simplicio 4.9.41-1-lts #1 smp mon aug 7 17:57:02 cest 2017 i686 gnulinux '
    config_args=''
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=undef
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    optimize='-O2 -march=native'
    cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='7.2.0'
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=4
    doublesize=8
    byteorder=1234
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=12
    longdblkind=3
    ivtype='long'
    ivsize=4
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=4
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags ='-fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/i686-pc-linux-gnu/7.2.0/include-fixed /usr/lib /lib
    libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.26.so
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.26'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -march=native -L/usr/local/lib -fstack-protector-strong'



@INC for perl 5.26.0:
    /home/mauke/usr/lib/perl5/site_perl/5.26.0/i686-linux
    /home/mauke/usr/lib/perl5/site_perl/5.26.0
    /home/mauke/usr/lib/perl5/5.26.0/i686-linux
    /home/mauke/usr/lib/perl5/5.26.0


Environment for perl 5.26.0:
    HOME=/home/mauke
    LANG=en_US.UTF-8
    LANGUAGE=en_US
    LC_COLLATE=C
    LC_MONETARY=de_DE.UTF-8
    LC_TIME=de_DE.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/mauke/perl5/perlbrew/bin:/home/mauke/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
    PERLBREW_BASHRC_VERSION=0.73
    PERLBREW_HOME=/home/mauke/.perlbrew
    PERLBREW_ROOT=/home/mauke/perl5/perlbrew
    PERL_BADLANG (unset)
    PERL_UNICODE=SAL
    SHELL=/bin/bash

p5pRT · 2017-11-18T16:48:54Z

@khwilliamson - Status changed from 'new' to 'open'

p5pRT · 2017-11-18T16:49:22Z

From @khwilliamson

Fixed by

commit 9b79e9e
Author: Karl Williamson <khw@cpan.org>
Date: Sat Nov 18 09:35:25 2017 -0700

PATCH: [perl #132463] perluniprops for \p{Word}

perluniprops was not updated to reflect the changes made to what
\p{Word} contains as of 5.18. What was added was the code points that
have the Join_Control property, which, so far, only contain U+200C and
U+200D. This commit uses Join Control instead of the hard-coded code
point numbers, so that when Unicode changes it, it automatically will
still be valid.

Thanks for spotting this.
--
Karl Williamson

p5pRT · 2017-11-18T16:49:23Z

@khwilliamson - Status changed from 'open' to 'pending release'

p5pRT · 2017-11-19T00:33:13Z

From @karenetheridge

This fix should also be backported to whatever other perl releases are
still being maintained (5.24.x, 5.26.x)

On Sat, Nov 18, 2017 at 8:49 AM, Karl Williamson via RT <
perlbug-followup@perl.org> wrote:

Fixed by

commit 9b79e9e
Author: Karl Williamson <khw@cpan.org>
Date: Sat Nov 18 09:35:25 2017 -0700
 PATCH&#8203;: \[perl \#132463\] perluniprops for \\p\{Word\}

 perluniprops was not updated to reflect the changes made to what
 \\p\{Word\} contains as of 5\.18\.  What was added was the code points that
 have the Join\_Control property\, which\, so far\, only contain U\+200C and
 U\+200D\.  This commit uses Join Control instead of the hard\-coded code
 point numbers\, so that when Unicode changes it\, it automatically will
 still be valid\.

 Thanks for spotting this\.
--
Karl Williamson

---
via perlbug: queue: perl5 status: open
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132463

p5pRT · 2017-11-19T14:15:03Z

From @xsawyerx

https://perl5.git.perl.org/perl.git/commitdiff/b183555123f84eb6ce65eb8af346d9a9579781ab

On 11/19/2017 01:32 AM, Karen Etheridge wrote:

This fix should also be backported to whatever other perl releases are
still being maintained (5.24.x, 5.26.x)

On Sat, Nov 18, 2017 at 8:49 AM, Karl Williamson via RT
<perlbug-followup@perl.org <mailto:perlbug-followup@perl.org>> wrote:

Fixed by

commit 9b79e9e3431d11e79a4f85268f70130b7b4369f6
 Author&#8203;: Karl Williamson \<khw@&#8203;cpan\.org \<mailto&#8203;:khw@&#8203;cpan\.org>>
 Date&#8203;:   Sat Nov 18 09&#8203;:35&#8203;:25 2017 \-0700

     PATCH&#8203;: \[perl \#132463\] perluniprops for \\p\{Word\}

     perluniprops was not updated to reflect the changes made to what
     \\p\{Word\} contains as of 5\.18\.  What was added was the code
points that
     have the Join\_Control property\, which\, so far\, only contain
U\+200C and
     U\+200D\.  This commit uses Join Control instead of the
hard\-coded code
     point numbers\, so that when Unicode changes it\, it
automatically will
     still be valid\.

     Thanks for spotting this\.
\-\-
Karl Williamson

\-\-\-
via perlbug&#8203;:  queue&#8203;: perl5 status&#8203;: open
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132463
\<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132463>

p5pRT · 2017-11-23T02:52:21Z

From @khwilliamson

On 11/19/2017 07:14 AM, Sawyer X wrote:

https://perl5.git.perl.org/perl.git/commitdiff/b183555123f84eb6ce65eb8af346d9a9579781ab

I thought we decided to get rid of the voting procedure, but I don't
remember what the replacement was to be. Perhaps this could be a guinea
pig.

On 11/19/2017 01:32 AM, Karen Etheridge wrote:

This fix should also be backported to whatever other perl releases are
still being maintained (5.24.x, 5.26.x)

On Sat, Nov 18, 2017 at 8:49 AM, Karl Williamson via RT
<perlbug-followup@perl.org <mailto:perlbug-followup@perl.org>> wrote:

 Fixed by

 commit 9b79e9e3431d11e79a4f85268f70130b7b4369f6
  Author&#8203;: Karl Williamson \<khw@&#8203;cpan\.org \<mailto&#8203;:khw@&#8203;cpan\.org>>
  Date&#8203;:   Sat Nov 18 09&#8203;:35&#8203;:25 2017 \-0700

      PATCH&#8203;: \[perl \#132463\] perluniprops for \\p\{Word\}

      perluniprops was not updated to reflect the changes made to what
      \\p\{Word\} contains as of 5\.18\.  What was added was the code
 points that
      have the Join\_Control property\, which\, so far\, only contain
 U\+200C and
      U\+200D\.  This commit uses Join Control instead of the
 hard\-coded code
      point numbers\, so that when Unicode changes it\, it
 automatically will
      still be valid\.

      Thanks for spotting this\.
 \-\-
 Karl Williamson

 \-\-\-
 via perlbug&#8203;:  queue&#8203;: perl5 status&#8203;: open
 https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132463
 \<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132463>

p5pRT · 2017-11-23T07:41:24Z

From @xsawyerx

On 23 November 2017 at 03:51, Karl Williamson <public@khwilliamson.com>
wrote:

On 11/19/2017 07:14 AM, Sawyer X wrote:

https://perl5.git.perl.org/perl.git/commitdiff/b183555123f84
eb6ce65eb8af346d9a9579781ab

I thought we decided to get rid of the voting procedure, but I don't
remember what the replacement was to be. Perhaps this could be a guinea
pig.

I don't think anyone wrote down exactly how we want to do it. I remembered
the majority of it and raised it with Steve Hay, who had a few questions
that I overlooked. We can definitely try with this issue though.

p5pRT · 2018-06-23T15:32:16Z

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release yesterday of Perl 5.28.0, this and 185 other issues have been
resolved.

Perl 5.28.0 may be downloaded via:
https://metacpan.org/release/XSAWYERX/perl-5.28.0

If you find that the problem persists, feel free to reopen this ticket.

p5pRT · 2018-06-23T15:32:17Z

@khwilliamson - Status changed from 'pending release' to 'resolved'

p5pRT closed this as completed Jun 23, 2018

p5pRT added Severity Low documentation labels Oct 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perluniprops out of date regarding \p{Word} #16255

perluniprops out of date regarding \p{Word} #16255

p5pRT commented Nov 18, 2017

p5pRT commented Nov 18, 2017

p5pRT commented Nov 18, 2017

p5pRT commented Nov 18, 2017

p5pRT commented Nov 18, 2017

p5pRT commented Nov 19, 2017

p5pRT commented Nov 19, 2017

p5pRT commented Nov 23, 2017

p5pRT commented Nov 23, 2017

p5pRT commented Jun 23, 2018

p5pRT commented Jun 23, 2018

perluniprops out of date regarding \p{Word} #16255

perluniprops out of date regarding \p{Word} #16255

Comments

p5pRT commented Nov 18, 2017

p5pRT commented Nov 18, 2017

From @mauke

Created by @mauke

p5pRT commented Nov 18, 2017

p5pRT commented Nov 18, 2017

From @khwilliamson

p5pRT commented Nov 18, 2017

p5pRT commented Nov 19, 2017

From @karenetheridge

p5pRT commented Nov 19, 2017

From @xsawyerx

p5pRT commented Nov 23, 2017

From @khwilliamson

p5pRT commented Nov 23, 2017

From @xsawyerx

p5pRT commented Jun 23, 2018

From @khwilliamson

p5pRT commented Jun 23, 2018