Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blead Breaks CPAN: CFAERBER/Net-IDN-Encode-2.500.tar.gz #16850

Closed
p5pRT opened this issue Feb 21, 2019 · 44 comments
Closed

Blead Breaks CPAN: CFAERBER/Net-IDN-Encode-2.500.tar.gz #16850

p5pRT opened this issue Feb 21, 2019 · 44 comments
Labels
BBC Blead Breaks CPAN - changes in blead broke a cpan module(s) distro-All

Comments

@p5pRT
Copy link

p5pRT commented Feb 21, 2019

Migrated from rt.perl.org#133860 (status was 'resolved')

Searchable as RT133860$

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2019

From @eserte

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.40 running under perl 5.29.8.


Starting with perl 5.29.8 the test suite of Net-IDN-Encode-2.500
started to fail. There are no problems with 5.29.7 or earlier perl
versions.

Test reports will appear in an hour or so at CPAN Testers.



Flags​:
  category=core
  severity=low


This perlbug was built using Perl 5.20.2 - Mon Sep 18 18​:13​:32 UTC 2017
It is being executed now by Perl 5.29.8 - Thu Feb 21 18​:36​:21 CET 2019.

Site configuration information for perl 5.29.8​:

Configured by eserte at Thu Feb 21 18​:36​:21 CET 2019.

Summary of my perl5 (revision 5 version 29 subversion 8) configuration​:
 
  Platform​:
  osname=linux
  osvers=3.16.0-4-amd64
  archname=x86_64-linux
  uname='linux cabulja 3.16.0-4-amd64 #1 smp debian 3.16.51-3 (2017-12-13) x86_64 gnulinux '
  config_args='-ds -e -Dprefix=/opt/perl-5.29.8 -Dusedevel -Dusemallocwrap=no -Dcf_email=srezic@​cpan.org'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
  optimize='-O2'
  cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.9.2'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.19.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.19'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'


@​INC for perl 5.29.8​:
  /opt/perl-5.29.8/lib/site_perl/5.29.8/x86_64-linux
  /opt/perl-5.29.8/lib/site_perl/5.29.8
  /opt/perl-5.29.8/lib/5.29.8/x86_64-linux
  /opt/perl-5.29.8/lib/5.29.8


Environment for perl 5.29.8​:
  HOME=/home/eserte
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/eserte/bin/linux-gnu​:/home/eserte/bin/sh​:/home/eserte/bin​:/home/eserte/bin/pistachio-perl/bin​:/usr/games​:/home/eserte/devel
  PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
  PERL_BADLANG (unset)
  SHELL=/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2019

From @eserte

Dana Thu, 21 Feb 2019 12​:53​:01 -0800, slaven@​rezic.de reče​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.40 running under perl 5.29.8.

-----------------------------------------------------------------
Starting with perl 5.29.8 the test suite of Net-IDN-Encode-2.500
started to fail. There are no problems with 5.29.7 or earlier perl
versions.

Test reports will appear in an hour or so at CPAN Testers.

Two sample fail reports​:
* http​://www.cpantesters.org/cpan/report/f1f6e6cc-361a-11e9-96a6-b1a1837a2169
* http​://www.cpantesters.org/cpan/report/c76e7d20-361a-11e9-96a6-b1a1837a2169

-----------------------------------------------------------------
---
Flags​:
category=core
severity=low
---
This perlbug was built using Perl 5.20.2 - Mon Sep 18 18​:13​:32 UTC
2017
It is being executed now by Perl 5.29.8 - Thu Feb 21 18​:36​:21 CET
2019.

Site configuration information for perl 5.29.8​:

Configured by eserte at Thu Feb 21 18​:36​:21 CET 2019.

Summary of my perl5 (revision 5 version 29 subversion 8)
configuration​:

Platform​:
osname=linux
osvers=3.16.0-4-amd64
archname=x86_64-linux
uname='linux cabulja 3.16.0-4-amd64 #1 smp debian 3.16.51-3 (2017-
12-13) x86_64 gnulinux '
config_args='-ds -e -Dprefix=/opt/perl-5.29.8 -Dusedevel
-Dusemallocwrap=no -Dcf_email=srezic@​cpan.org'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler​:
cc='cc'
ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-
strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
-D_FORTIFY_SOURCE=2'
optimize='-O2'
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-
strong -I/usr/local/include'
ccversion=''
gccversion='4.9.2'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries​:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-
fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu
/lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
-lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.19.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.19'
Dynamic Linking​:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

---
@​INC for perl 5.29.8​:
/opt/perl-5.29.8/lib/site_perl/5.29.8/x86_64-linux
/opt/perl-5.29.8/lib/site_perl/5.29.8
/opt/perl-5.29.8/lib/5.29.8/x86_64-linux
/opt/perl-5.29.8/lib/5.29.8

---
Environment for perl 5.29.8​:
HOME=/home/eserte
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/eserte/bin/linux-
gnu​:/home/eserte/bin/sh​:/home/eserte/bin​:/home/eserte/bin/pistachio-
perl/bin​:/usr/games​:/home/eserte/devel
PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
PERL_BADLANG (unset)
SHELL=/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2019

From @jkeenan

On Thu, 21 Feb 2019 20​:53​:01 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.40 running under perl 5.29.8.

-----------------------------------------------------------------
Starting with perl 5.29.8 the test suite of Net-IDN-Encode-2.500
started to fail. There are no problems with 5.29.7 or earlier perl
versions.

Test reports will appear in an hour or so at CPAN Testers.

Bisection points to this commit​:

#####
Author​: Karl Williamson <khw@​cpan.org>
AuthorDate​: Mon Aug 20 18​:31​:04 2018 -0600
Commit​: Karl Williamson <khw@​cpan.org>
CommitDate​: Thu Feb 14 22​:12​:44 2019 -0700

Move \p{user-defined} to core from utf8_heavy.pl

This large commit moves the handling of user-defined properties to C code. This should speed it up, but the main reason to do this is to stop using swashes in this case, leaving only tr/// using them. Once that too is converted, all swash handling can be ripped out of perl.

Doing this in perl has caused some nasty interactions that will now be fixed automatically.

The change is not entirely transparent, however (besides speed and the possibility of removing these interactions). perldelta in this commit details these.
#####

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2019

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Feb 21, 2019

From @jkeenan

On Thu, 21 Feb 2019 22​:54​:00 GMT, jkeenan wrote​:

On Thu, 21 Feb 2019 20​:53​:01 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.40 running under perl 5.29.8.

-----------------------------------------------------------------
Starting with perl 5.29.8 the test suite of Net-IDN-Encode-2.500
started to fail. There are no problems with 5.29.7 or earlier perl
versions.

Test reports will appear in an hour or so at CPAN Testers.

Bisection points to this commit​:

commit 73b9584

#####
Author​: Karl Williamson <khw@​cpan.org>
AuthorDate​: Mon Aug 20 18​:31​:04 2018 -0600
Commit​: Karl Williamson <khw@​cpan.org>
CommitDate​: Thu Feb 14 22​:12​:44 2019 -0700

Move \p{user-defined} to core from utf8_heavy.pl

This large commit moves the handling of user-defined properties to C
code. This should speed it up, but the main reason to do this is to
stop using swashes in this case, leaving only tr/// using them. Once
that too is converted, all swash handling can be ripped out of perl.

Doing this in perl has caused some nasty interactions that will now be
fixed automatically.

The change is not entirely transparent, however (besides speed and the
possibility of removing these interactions). perldelta in this commit
details these.
#####

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Feb 22, 2019

From @khwilliamson

On 2/21/19 3​:54 PM, James E Keenan via RT wrote​:

On Thu, 21 Feb 2019 20​:53​:01 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.40 running under perl 5.29.8.

-----------------------------------------------------------------
Starting with perl 5.29.8 the test suite of Net-IDN-Encode-2.500
started to fail. There are no problems with 5.29.7 or earlier perl
versions.

Test reports will appear in an hour or so at CPAN Testers.

Bisection points to this commit​:

#####
Author​: Karl Williamson <khw@​cpan.org>
AuthorDate​: Mon Aug 20 18​:31​:04 2018 -0600
Commit​: Karl Williamson <khw@​cpan.org>
CommitDate​: Thu Feb 14 22​:12​:44 2019 -0700

Move \p{user-defined} to core from utf8_heavy.pl

This large commit moves the handling of user-defined properties to C code. This should speed it up, but the main reason to do this is to stop using swashes in this case, leaving only tr/// using them. Once that too is converted, all swash handling can be ripped out of perl.

Doing this in perl has caused some nasty interactions that will now be fixed automatically.

The change is not entirely transparent, however (besides speed and the possibility of removing these interactions). perldelta in this commit details these.
#####

Thank you very much.

I have found the problem, but don't know what to do about it.

The module turns out to be relying on a bug in the earlier core.
User-defined property functions are only called once, their return is
cached, and that is used from then on instead of calling the function
again. This was done in part for security reasons, and cannot be
changed without breaking other code.

In this module, the function IsIgnored() is somehow being called from a
BEGIN block. That function uses data that isn't initialized until run
time, so returns empty. And all future uses also return empty.

The reason it used to work is that a bug in the core caused the first
call of it to look in the wrong package. That means it wasn't found, so
actually didn't get called. When called later, with the correct
package, the data has been initialized, and the function returns real
data which gets cached and returned in future calls.

What can be done with this module is to remove the BEGIN block call, or
place the data initialization also in a BEGIN block.

But how much other code out there also relies on this former bug? I
don't know how to fix it generally. One idea is to not consider a
function defined until it returns something. I don't know the security
implications of this. And would that break other code?

@p5pRT
Copy link
Author

p5pRT commented Feb 23, 2019

From @jkeenan

On Thu, 21 Feb 2019 22​:58​:10 GMT, jkeenan wrote​:

On Thu, 21 Feb 2019 22​:54​:00 GMT, jkeenan wrote​:

On Thu, 21 Feb 2019 20​:53​:01 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.40 running under perl 5.29.8.

-----------------------------------------------------------------
Starting with perl 5.29.8 the test suite of Net-IDN-Encode-2.500
started to fail. There are no problems with 5.29.7 or earlier perl
versions.

Test reports will appear in an hour or so at CPAN Testers.

Bisection points to this commit​:

commit 73b9584

#####
Author​: Karl Williamson <khw@​cpan.org>
AuthorDate​: Mon Aug 20 18​:31​:04 2018 -0600
Commit​: Karl Williamson <khw@​cpan.org>
CommitDate​: Thu Feb 14 22​:12​:44 2019 -0700

Move \p{user-defined} to core from utf8_heavy.pl

This large commit moves the handling of user-defined properties to C
code. This should speed it up, but the main reason to do this is to
stop using swashes in this case, leaving only tr/// using them. Once
that too is converted, all swash handling can be ripped out of perl.

Doing this in perl has caused some nasty interactions that will now be
fixed automatically.

The change is not entirely transparent, however (besides speed and the
possibility of removing these interactions). perldelta in this commit
details these.
#####

Thank you very much.

I also encountered this problem on FreeBSD-11.1 while doing monthly "CPAN River 3000" testing. Attaching output.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Feb 23, 2019

@p5pRT
Copy link
Author

p5pRT commented Mar 5, 2019

From @khwilliamson

On Fri, 22 Feb 2019 12​:09​:22 -0800, public@​khwilliamson.com wrote​:

On 2/21/19 3​:54 PM, James E Keenan via RT wrote​:

On Thu, 21 Feb 2019 20​:53​:01 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.40 running under perl 5.29.8.

-----------------------------------------------------------------
Starting with perl 5.29.8 the test suite of Net-IDN-Encode-2.500
started to fail. There are no problems with 5.29.7 or earlier perl
versions.

Test reports will appear in an hour or so at CPAN Testers.

Bisection points to this commit​:

#####
Author​: Karl Williamson <khw@​cpan.org>
AuthorDate​: Mon Aug 20 18​:31​:04 2018 -0600
Commit​: Karl Williamson <khw@​cpan.org>
CommitDate​: Thu Feb 14 22​:12​:44 2019 -0700

Move \p{user-defined} to core from utf8_heavy.pl

This large commit moves the handling of user-defined properties to C
code. This should speed it up, but the main reason to do this is to
stop using swashes in this case, leaving only tr/// using them. Once
that too is converted, all swash handling can be ripped out of perl.

Doing this in perl has caused some nasty interactions that will now
be fixed automatically.

The change is not entirely transparent, however (besides speed and
the possibility of removing these interactions). perldelta in this
commit details these.
#####

Thank you very much.

I have found the problem, but don't know what to do about it.

The module turns out to be relying on a bug in the earlier core.
User-defined property functions are only called once, their return is
cached, and that is used from then on instead of calling the function
again. This was done in part for security reasons, and cannot be
changed without breaking other code.

In this module, the function IsIgnored() is somehow being called from
a
BEGIN block. That function uses data that isn't initialized until run
time, so returns empty. And all future uses also return empty.

The reason it used to work is that a bug in the core caused the first
call of it to look in the wrong package. That means it wasn't found,
so
actually didn't get called. When called later, with the correct
package, the data has been initialized, and the function returns real
data which gets cached and returned in future calls.

What can be done with this module is to remove the BEGIN block call,
or
place the data initialization also in a BEGIN block.

But how much other code out there also relies on this former bug? I
don't know how to fix it generally. One idea is to not consider a
function defined until it returns something. I don't know the
security
implications of this. And would that break other code?

I have created a PR for this module. If you would like to comment on it, see
cfaerber/Net-IDN-Encode#8
--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented Mar 30, 2019

From @andk

Also affected​: BDFOY/Roman-Unicode-1.031.tar.gz
--
andreas

@p5pRT
Copy link
Author

p5pRT commented Mar 31, 2019

From @jkeenan

On Sat, 30 Mar 2019 21​:15​:56 GMT, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Also affected​: BDFOY/Roman-Unicode-1.031.tar.gz

briandfoy/roman-unicode#1

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Mar 31, 2019

From [Unknown Contact. See original ticket]

On Sat, 30 Mar 2019 21​:15​:56 GMT, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Also affected​: BDFOY/Roman-Unicode-1.031.tar.gz

briandfoy/roman-unicode#1

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Apr 1, 2019

From @khwilliamson

On Sat, 30 Mar 2019 17​:56​:15 -0700, jkeenan wrote​:

On Sat, 30 Mar 2019 21​:15​:56 GMT,
andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Also affected​: BDFOY/Roman-Unicode-1.031.tar.gz

briandfoy/roman-unicode#1

Because the actual trajectory of these two issues with the same commit are so divergent, I created a new ticket to track the Roman​::Unicode problem.

https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133970
--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented Apr 1, 2019

From @jkeenan

On Mon, 01 Apr 2019 04​:56​:35 GMT, khw wrote​:

On Sat, 30 Mar 2019 17​:56​:15 -0700, jkeenan wrote​:

On Sat, 30 Mar 2019 21​:15​:56 GMT,
andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Also affected​: BDFOY/Roman-Unicode-1.031.tar.gz

briandfoy/roman-unicode#1

Because the actual trajectory of these two issues with the same commit
are so divergent, I created a new ticket to track the Roman​::Unicode
problem.

https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133970

Does that mean that *this* RT (133860 -- dealing with Net-IDN-Encode) is closable?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Apr 2, 2019

From @khwilliamson

On 4/1/19 4​:54 PM, James E Keenan via RT wrote​:

On Mon, 01 Apr 2019 04​:56​:35 GMT, khw wrote​:

On Sat, 30 Mar 2019 17​:56​:15 -0700, jkeenan wrote​:

On Sat, 30 Mar 2019 21​:15​:56 GMT,
andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Also affected​: BDFOY/Roman-Unicode-1.031.tar.gz

briandfoy/roman-unicode#1

Because the actual trajectory of these two issues with the same commit
are so divergent, I created a new ticket to track the Roman​::Unicode
problem.

https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133970

Does that mean that *this* RT (133860 -- dealing with Net-IDN-Encode) is closable?

Thank you very much.

I submitted a PR, that means it isn't a blocker. But if a bunch of
other distros have the same issue, it may mean that that we'd have to do
something to get them to not break.

@p5pRT
Copy link
Author

p5pRT commented Apr 7, 2019

From @xsawyerx

On 4/2/19 3​:04 AM, Karl Williamson wrote​:

On 4/1/19 4​:54 PM, James E Keenan via RT wrote​:

On Mon, 01 Apr 2019 04​:56​:35 GMT, khw wrote​:

On Sat, 30 Mar 2019 17​:56​:15 -0700, jkeenan wrote​:

On Sat, 30 Mar 2019 21​:15​:56 GMT,
andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Also affected​: BDFOY/Roman-Unicode-1.031.tar.gz

briandfoy/roman-unicode#1

Because the actual trajectory of these two issues with the same commit
are so divergent, I created a new ticket to track the Roman​::Unicode
problem.

https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133970

Does that mean that *this* RT (133860 -- dealing with Net-IDN-Encode)
is closable?

Thank you very much.

I submitted a PR, that means it isn't a blocker.  But if a bunch of
other distros have the same issue, it may mean that that we'd have to
do something to get them to not break.

Any ideas of what we could do, other than reintroducing the original bug?

@p5pRT
Copy link
Author

p5pRT commented Apr 7, 2019

From @khwilliamson

On 4/6/2019 11​:49 PM, Sawyer X wrote​:

On 4/2/19 3​:04 AM, Karl Williamson wrote​:

On 4/1/19 4​:54 PM, James E Keenan via RT wrote​:

On Mon, 01 Apr 2019 04​:56​:35 GMT, khw wrote​:

On Sat, 30 Mar 2019 17​:56​:15 -0700, jkeenan wrote​:

On Sat, 30 Mar 2019 21​:15​:56 GMT,
andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Also affected​: BDFOY/Roman-Unicode-1.031.tar.gz

briandfoy/roman-unicode#1

Because the actual trajectory of these two issues with the same commit
are so divergent, I created a new ticket to track the Roman​::Unicode
problem.

https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133970

Does that mean that *this* RT (133860 -- dealing with Net-IDN-Encode)
is closable?

Thank you very much.

I submitted a PR, that means it isn't a blocker.  But if a bunch of
other distros have the same issue, it may mean that that we'd have to
do something to get them to not break.

Any ideas of what we could do, other than reintroducing the original bug?

If a sub returned nothing, we could consider it still undefined, and
keep calling it until the first time (if ever) it did return something.

@p5pRT
Copy link
Author

p5pRT commented Apr 7, 2019

From @eserte

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.29.9.


The test suite of ANNO/Unicode-CharWidth-1.05.tar.gz fails
since perl 5.29.8. A sample fail report on CPAN Testers​:
http​://www.cpantesters.org/cpan/report/ca2cd882-5801-11e9-a3fc-1703e4e5c01a



Flags​:
  category=core
  severity=low


Site configuration information for perl 5.29.9​:

Configured by eserte at Thu Mar 21 21​:10​:55 UTC 2019.

Summary of my perl5 (revision 5 version 29 subversion 9) configuration​:
 
  Platform​:
  osname=linux
  osvers=4.15.0-46-generic
  archname=x86_64-linux
  uname='linux ubuntu18 4.15.0-46-generic #49-ubuntu smp wed feb 6 09​:33​:07 utc 2019 x86_64 x86_64 x86_64 gnulinux '
  config_args='-ds -e -Dprefix=/opt/perl-5.29.9 -Dusedevel -Dusemallocwrap=no -Dcf_email=srezic@​cpan.org'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
  optimize='-O2'
  cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='7.3.0'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/7/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.27.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.27'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'


@​INC for perl 5.29.9​:
  /opt/perl-5.29.9/lib/site_perl/5.29.9/x86_64-linux
  /opt/perl-5.29.9/lib/site_perl/5.29.9
  /opt/perl-5.29.9/lib/5.29.9/x86_64-linux
  /opt/perl-5.29.9/lib/5.29.9


Environment for perl 5.29.9​:
  HOME=/home/eserte
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LC_ALL=de_DE.UTF-8
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/eserte/bin/linux-gnu​:/home/eserte/bin/sh​:/home/eserte/bin​:/home/eserte/bin/pistachio-perl/bin​:/usr/games​:/home/eserte/devel
  PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
  PERL_BADLANG (unset)
  SHELL=/usr/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Apr 7, 2019

From @jkeenan

On Sun, 07 Apr 2019 19​:31​:16 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.29.9.

-----------------------------------------------------------------
The test suite of ANNO/Unicode-CharWidth-1.05.tar.gz fails
since perl 5.29.8. A sample fail report on CPAN Testers​:
http​://www.cpantesters.org/cpan/report/ca2cd882-5801-11e9-a3fc-
1703e4e5c01a

With this bisection command​:

#####
perl Porting/bisect.pl --start=7b97bf55c0 --end=ca8b93afd02ddde55c1aa9e6fbff9acdad31593d --module=Unicode​::CharWidth
#####

... I got this bisection result​:

#####
73b9584 is the first bad commit commit 73b9584 Author​: Karl Williamson <khw@​cpan.org> Date​: Mon Aug 20 18​:31​:04 2018 -0600 Move \p{user-defined} to core from utf8_heavy.pl
#####

... which is the same commit where breakage was reported in https://rt.perl.org/Public/Bug/Display.html?id=133860. So I recommend that this ticket be merged into that one.
#####

-----------------------------------------------------------------
---
Flags​:
category=core
severity=low
---
Site configuration information for perl 5.29.9​:

Configured by eserte at Thu Mar 21 21​:10​:55 UTC 2019.

Summary of my perl5 (revision 5 version 29 subversion 9)
configuration​:

Platform​:
osname=linux
osvers=4.15.0-46-generic
archname=x86_64-linux
uname='linux ubuntu18 4.15.0-46-generic #49-ubuntu smp wed feb 6
09​:33​:07 utc 2019 x86_64 x86_64 x86_64 gnulinux '
config_args='-ds -e -Dprefix=/opt/perl-5.29.9 -Dusedevel
-Dusemallocwrap=no -Dcf_email=srezic@​cpan.org'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler​:
cc='cc'
ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-
strong -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64'
optimize='-O2'
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-
strong -I/usr/local/include'
ccversion=''
gccversion='7.3.0'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries​:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/7/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu
/lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.27.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.27'
Dynamic Linking​:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

---
@​INC for perl 5.29.9​:
/opt/perl-5.29.9/lib/site_perl/5.29.9/x86_64-linux
/opt/perl-5.29.9/lib/site_perl/5.29.9
/opt/perl-5.29.9/lib/5.29.9/x86_64-linux
/opt/perl-5.29.9/lib/5.29.9

---
Environment for perl 5.29.9​:
HOME=/home/eserte
LANG=en_US.UTF-8
LANGUAGE (unset)
LC_ALL=de_DE.UTF-8
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/eserte/bin/linux-
gnu​:/home/eserte/bin/sh​:/home/eserte/bin​:/home/eserte/bin/pistachio-
perl/bin​:/usr/games​:/home/eserte/devel
PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
PERL_BADLANG (unset)
SHELL=/usr/bin/zsh

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Apr 7, 2019

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Apr 8, 2019

From @xsawyerx

On 4/7/19 6​:18 PM, Karl Williamson wrote​:

On 4/6/2019 11​:49 PM, Sawyer X wrote​:

On 4/2/19 3​:04 AM, Karl Williamson wrote​:

On 4/1/19 4​:54 PM, James E Keenan via RT wrote​:

On Mon, 01 Apr 2019 04​:56​:35 GMT, khw wrote​:

On Sat, 30 Mar 2019 17​:56​:15 -0700, jkeenan wrote​:

On Sat, 30 Mar 2019 21​:15​:56 GMT,
andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Also affected​: BDFOY/Roman-Unicode-1.031.tar.gz

briandfoy/roman-unicode#1

Because the actual trajectory of these two issues with the same
commit
are so divergent, I created a new ticket to track the Roman​::Unicode
problem.

https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133970

Does that mean that *this* RT (133860 -- dealing with Net-IDN-Encode)
is closable?

Thank you very much.

I submitted a PR, that means it isn't a blocker.  But if a bunch of
other distros have the same issue, it may mean that that we'd have to
do something to get them to not break.

Any ideas of what we could do, other than reintroducing the original
bug?

If a sub returned nothing, we could consider it still undefined, and
keep calling it until the first time (if ever) it did return something.

Similar to introducing the buggy behavior, but in a single function?

Sounds like a temporary fix, though. This module will still need to be
fixed, though, no?

@p5pRT
Copy link
Author

p5pRT commented Apr 8, 2019

From @xsawyerx

On 4/8/19 2​:46 AM, James E Keenan via RT wrote​:

On Sun, 07 Apr 2019 19​:31​:16 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.29.9.

-----------------------------------------------------------------
The test suite of ANNO/Unicode-CharWidth-1.05.tar.gz fails
since perl 5.29.8. A sample fail report on CPAN Testers​:
http​://www.cpantesters.org/cpan/report/ca2cd882-5801-11e9-a3fc-
1703e4e5c01a

With this bisection command​:

#####
perl Porting/bisect.pl --start=7b97bf55c0 --end=ca8b93afd02ddde55c1aa9e6fbff9acdad31593d --module=Unicode​::CharWidth
#####

... I got this bisection result​:

#####
73b9584 is the first bad commit commit 73b9584 Author​: Karl Williamson <khw@​cpan.org> Date​: Mon Aug 20 18​:31​:04 2018 -0600 Move \p{user-defined} to core from utf8_heavy.pl
#####

... which is the same commit where breakage was reported in https://rt.perl.org/Public/Bug/Display.html?id=133860. So I recommend that this ticket be merged into that one.

I was going to suggest the same.

@p5pRT
Copy link
Author

p5pRT commented Apr 8, 2019

From @khwilliamson

On 4/7/19 11​:25 PM, Sawyer X wrote​:

On 4/7/19 6​:18 PM, Karl Williamson wrote​:

On 4/6/2019 11​:49 PM, Sawyer X wrote​:

On 4/2/19 3​:04 AM, Karl Williamson wrote​:

On 4/1/19 4​:54 PM, James E Keenan via RT wrote​:

On Mon, 01 Apr 2019 04​:56​:35 GMT, khw wrote​:

On Sat, 30 Mar 2019 17​:56​:15 -0700, jkeenan wrote​:

On Sat, 30 Mar 2019 21​:15​:56 GMT,
andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Also affected​: BDFOY/Roman-Unicode-1.031.tar.gz

briandfoy/roman-unicode#1

Because the actual trajectory of these two issues with the same
commit
are so divergent, I created a new ticket to track the Roman​::Unicode
problem.

https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133970

Does that mean that *this* RT (133860 -- dealing with Net-IDN-Encode)
is closable?

Thank you very much.

I submitted a PR, that means it isn't a blocker.  But if a bunch of
other distros have the same issue, it may mean that that we'd have to
do something to get them to not break.

Any ideas of what we could do, other than reintroducing the original
bug?

If a sub returned nothing, we could consider it still undefined, and
keep calling it until the first time (if ever) it did return something.

Similar to introducing the buggy behavior, but in a single function?

Sounds like a temporary fix, though. This module will still need to be
fixed, though, no?

Not if we did what I said we could do. I'm not advocating it, and if
this is the only module that has that behavior, I think it should
change, and have made a PR to do so.

@p5pRT
Copy link
Author

p5pRT commented Apr 8, 2019

From @khwilliamson

On 4/7/19 11​:29 PM, Sawyer X wrote​:

On 4/8/19 2​:46 AM, James E Keenan via RT wrote​:

On Sun, 07 Apr 2019 19​:31​:16 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.29.9.

-----------------------------------------------------------------
The test suite of ANNO/Unicode-CharWidth-1.05.tar.gz fails
since perl 5.29.8. A sample fail report on CPAN Testers​:
http​://www.cpantesters.org/cpan/report/ca2cd882-5801-11e9-a3fc-
1703e4e5c01a

With this bisection command​:

#####
perl Porting/bisect.pl --start=7b97bf55c0 --end=ca8b93afd02ddde55c1aa9e6fbff9acdad31593d --module=Unicode​::CharWidth
#####

... I got this bisection result​:

#####
73b9584 is the first bad commit commit 73b9584 Author​: Karl Williamson <khw@​cpan.org> Date​: Mon Aug 20 18​:31​:04 2018 -0600 Move \p{user-defined} to core from utf8_heavy.pl
#####

... which is the same commit where breakage was reported in https://rt.perl.org/Public/Bug/Display.html?id=133860. So I recommend that this ticket be merged into that one.

I was going to suggest the same.

There are at least two causes for this breakage, so merging is premature

@p5pRT
Copy link
Author

p5pRT commented Apr 8, 2019

From @jkeenan

On 4/8/19 11​:57 AM, Karl Williamson wrote​:

On 4/7/19 11​:29 PM, Sawyer X wrote​:

On 4/8/19 2​:46 AM, James E Keenan via RT wrote​:

On Sun, 07 Apr 2019 19​:31​:16 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.29.9.

-----------------------------------------------------------------
The test suite of ANNO/Unicode-CharWidth-1.05.tar.gz fails
since perl 5.29.8. A sample fail report on CPAN Testers​:
http​://www.cpantesters.org/cpan/report/ca2cd882-5801-11e9-a3fc-
1703e4e5c01a

With this bisection command​:

#####
perl Porting/bisect.pl --start=7b97bf55c0
--end=ca8b93afd02ddde55c1aa9e6fbff9acdad31593d
--module=Unicode​::CharWidth
#####

... I got this bisection result​:

#####
73b9584 is the first bad commit
commit 73b9584 Author​: Karl
Williamson <khw@​cpan.org> Date​: Mon Aug 20 18​:31​:04 2018 -0600 Move
\p{user-defined} to core from utf8_heavy.pl
#####

... which is the same commit where breakage was reported in
https://rt.perl.org/Public/Bug/Display.html?id=133860. So I recommend
that this ticket be merged into that one.

I was going to suggest the same.

There are at least two causes for this breakage, so merging is premature

Too late; I acted on sawyer's +1 about 4 hours ago. Suggestions?

jimk

@p5pRT
Copy link
Author

p5pRT commented Apr 8, 2019

From @khwilliamson

On 4/8/2019 11​:10 AM, James E Keenan wrote​:

On 4/8/19 11​:57 AM, Karl Williamson wrote​:

On 4/7/19 11​:29 PM, Sawyer X wrote​:

On 4/8/19 2​:46 AM, James E Keenan via RT wrote​:

On Sun, 07 Apr 2019 19​:31​:16 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.29.9.

-----------------------------------------------------------------
The test suite of ANNO/Unicode-CharWidth-1.05.tar.gz fails
since perl 5.29.8. A sample fail report on CPAN Testers​:
http​://www.cpantesters.org/cpan/report/ca2cd882-5801-11e9-a3fc-
1703e4e5c01a

With this bisection command​:

#####
perl Porting/bisect.pl --start=7b97bf55c0
--end=ca8b93afd02ddde55c1aa9e6fbff9acdad31593d
--module=Unicode​::CharWidth
#####

... I got this bisection result​:

#####
73b9584 is the first bad commit
commit 73b9584 Author​: Karl
Williamson <khw@​cpan.org> Date​: Mon Aug 20 18​:31​:04 2018 -0600 Move
\p{user-defined} to core from utf8_heavy.pl
#####

... which is the same commit where breakage was reported in
https://rt.perl.org/Public/Bug/Display.html?id=133860. So I
recommend that this ticket be merged into that one.

I was going to suggest the same.

There are at least two causes for this breakage, so merging is premature

Too late; I acted on sawyer's +1 about 4 hours ago.  Suggestions?

jimk

I don't think it's possible to unmerge. It will cause extra work for me
if it's not the same cause. Just don't merge any more to this ticket or
the other one that has the same bisect result

@p5pRT
Copy link
Author

p5pRT commented Apr 9, 2019

From @khwilliamson

On 4/8/19 12​:02 PM, Karl Williamson wrote​:

On 4/8/2019 11​:10 AM, James E Keenan wrote​:

On 4/8/19 11​:57 AM, Karl Williamson wrote​:

On 4/7/19 11​:29 PM, Sawyer X wrote​:

On 4/8/19 2​:46 AM, James E Keenan via RT wrote​:

On Sun, 07 Apr 2019 19​:31​:16 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.29.9.

-----------------------------------------------------------------
The test suite of ANNO/Unicode-CharWidth-1.05.tar.gz fails
since perl 5.29.8. A sample fail report on CPAN Testers​:
http​://www.cpantesters.org/cpan/report/ca2cd882-5801-11e9-a3fc-
1703e4e5c01a

With this bisection command​:

#####
perl Porting/bisect.pl --start=7b97bf55c0
--end=ca8b93afd02ddde55c1aa9e6fbff9acdad31593d
--module=Unicode​::CharWidth
#####

... I got this bisection result​:

#####
73b9584 is the first bad commit
commit 73b9584 Author​: Karl
Williamson <khw@​cpan.org> Date​: Mon Aug 20 18​:31​:04 2018 -0600 Move
\p{user-defined} to core from utf8_heavy.pl
#####

... which is the same commit where breakage was reported in
https://rt.perl.org/Public/Bug/Display.html?id=133860. So I
recommend that this ticket be merged into that one.

I was going to suggest the same.

There are at least two causes for this breakage, so merging is premature

Too late; I acted on sawyer's +1 about 4 hours ago.  Suggestions?

jimk

I don't think it's possible to unmerge.  It will cause extra work for me
if it's not the same cause.  Just don't merge any more to this ticket or
the other one that has the same bisect result

It turns out the root cause of this problem is completely different from
the one the ticket it got merged into is. It would be good if we could
unmerge. Does anyone have any suggestions?

And I'm not sure how best to fix this. I thought that I could use
cv_name() to get a fully qualified name for a sub this is globally
unique. It turns out that's not true. The module is creating anonymous
subs with the precise same name as others. I don't know if the
addresses are stable during a single run. Suggestions would be
appreciated here as well.

@p5pRT
Copy link
Author

p5pRT commented Apr 9, 2019

From @jkeenan

On Tue, 09 Apr 2019 07​:45​:35 GMT, public@​khwilliamson.com wrote​:

On 4/8/19 12​:02 PM, Karl Williamson wrote​:

On 4/8/2019 11​:10 AM, James E Keenan wrote​:

On 4/8/19 11​:57 AM, Karl Williamson wrote​:

On 4/7/19 11​:29 PM, Sawyer X wrote​:

On 4/8/19 2​:46 AM, James E Keenan via RT wrote​:

On Sun, 07 Apr 2019 19​:31​:16 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.29.9.

-----------------------------------------------------------------
The test suite of ANNO/Unicode-CharWidth-1.05.tar.gz fails
since perl 5.29.8. A sample fail report on CPAN Testers​:
http​://www.cpantesters.org/cpan/report/ca2cd882-5801-11e9-a3fc-
1703e4e5c01a

With this bisection command​:

#####
perl Porting/bisect.pl --start=7b97bf55c0
--end=ca8b93afd02ddde55c1aa9e6fbff9acdad31593d
--module=Unicode​::CharWidth
#####

... I got this bisection result​:

#####
73b9584 is the first bad commit
commit 73b9584 Author​: Karl
Williamson <khw@​cpan.org> Date​: Mon Aug 20 18​:31​:04 2018 -0600 Move
\p{user-defined} to core from utf8_heavy.pl
#####

... which is the same commit where breakage was reported in
https://rt.perl.org/Public/Bug/Display.html?id=133860. So I
recommend that this ticket be merged into that one.

I was going to suggest the same.

There are at least two causes for this breakage, so merging is premature

Too late; I acted on sawyer's +1 about 4 hours ago.  Suggestions?

jimk

I don't think it's possible to unmerge.  It will cause extra work for me
if it's not the same cause.  Just don't merge any more to this ticket or
the other one that has the same bisect result

It turns out the root cause of this problem is completely different from
the one the ticket it got merged into is. It would be good if we could
unmerge. Does anyone have any suggestions?

Well, what if I created a new ticket with most of the same content as in the original RT 134004 (that content now found at https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133860#txn-1624393)?

If I did so I, would like to include a note about why we're deviating from our normal practice of merging all BBC tickets associated with a single commit into the first such ticket created. If you were to provide me with a couple sentences for that reasoning, I would include that in the RT's first post.

And I'm not sure how best to fix this. I thought that I could use
cv_name() to get a fully qualified name for a sub this is globally
unique. It turns out that's not true. The module is creating anonymous
subs with the precise same name as others. I don't know if the
addresses are stable during a single run. Suggestions would be
appreciated here as well.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Apr 9, 2019

From @khwilliamson

On 4/9/19 5​:09 AM, James E Keenan via RT wrote​:

On Tue, 09 Apr 2019 07​:45​:35 GMT, public@​khwilliamson.com wrote​:

On 4/8/19 12​:02 PM, Karl Williamson wrote​:

On 4/8/2019 11​:10 AM, James E Keenan wrote​:

On 4/8/19 11​:57 AM, Karl Williamson wrote​:

On 4/7/19 11​:29 PM, Sawyer X wrote​:

On 4/8/19 2​:46 AM, James E Keenan via RT wrote​:

On Sun, 07 Apr 2019 19​:31​:16 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.29.9.

-----------------------------------------------------------------
The test suite of ANNO/Unicode-CharWidth-1.05.tar.gz fails
since perl 5.29.8. A sample fail report on CPAN Testers​:
http​://www.cpantesters.org/cpan/report/ca2cd882-5801-11e9-a3fc-
1703e4e5c01a

With this bisection command​:

#####
perl Porting/bisect.pl --start=7b97bf55c0
--end=ca8b93afd02ddde55c1aa9e6fbff9acdad31593d
--module=Unicode​::CharWidth
#####

... I got this bisection result​:

#####
73b9584 is the first bad commit
commit 73b9584 Author​: Karl
Williamson <khw@​cpan.org> Date​: Mon Aug 20 18​:31​:04 2018 -0600 Move
\p{user-defined} to core from utf8_heavy.pl
#####

... which is the same commit where breakage was reported in
https://rt.perl.org/Public/Bug/Display.html?id=133860. So I
recommend that this ticket be merged into that one.

I was going to suggest the same.

There are at least two causes for this breakage, so merging is premature

Too late; I acted on sawyer's +1 about 4 hours ago.  Suggestions?

jimk

I don't think it's possible to unmerge.  It will cause extra work for me
if it's not the same cause.  Just don't merge any more to this ticket or
the other one that has the same bisect result

It turns out the root cause of this problem is completely different from
the one the ticket it got merged into is. It would be good if we could
unmerge. Does anyone have any suggestions?

Well, what if I created a new ticket with most of the same content as in the original RT 134004 (that content now found at https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133860#txn-1624393)?

If I did so I, would like to include a note about why we're deviating from our normal practice of merging all BBC tickets associated with a single commit into the first such ticket created. If you were to provide me with a couple sentences for that reasoning, I would include that in the RT's first post.

This large commit has multiple issues.

[perl #133860] is to track modules that were relying on previous
incorrect behavior. So far there is only one instance of this. But if
enough modules break for this reason, we'd have to consider emulating
that improper behavior to some extent.

[perl #133970] is because the commit needed to do save and restore the
stack, and other housekeeping.

This ticket is because the commit wrongly assumed that a fully qualified
subroutine name uniquely identified a subroutine.

The fixes for these are completely independent, and doing any one will
get the tests passing again for the modules affected by each issue.

And I'm not sure how best to fix this. I thought that I could use
cv_name() to get a fully qualified name for a sub this is globally
unique. It turns out that's not true. The module is creating anonymous
subs with the precise same name as others. I don't know if the
addresses are stable during a single run. Suggestions would be
appreciated here as well.

Thank you very much.

@p5pRT
Copy link
Author

p5pRT commented Apr 9, 2019

From @dur-randir

On Tue, 09 Apr 2019 00​:45​:35 -0700, public@​khwilliamson.com wrote​:

And I'm not sure how best to fix this. I thought that I could use
cv_name() to get a fully qualified name for a sub this is globally
unique. It turns out that's not true. The module is creating anonymous
subs with the precise same name as others. I don't know if the
addresses are stable during a single run. Suggestions would be
appreciated here as well.

CV* are not moved, but it's address may be reused, if the sub is freed.

@p5pRT
Copy link
Author

p5pRT commented Apr 9, 2019

From @xsawyerx

On 4/9/19 10​:45 AM, Karl Williamson wrote​:

On 4/8/19 12​:02 PM, Karl Williamson wrote​:

On 4/8/2019 11​:10 AM, James E Keenan wrote​:

On 4/8/19 11​:57 AM, Karl Williamson wrote​:

On 4/7/19 11​:29 PM, Sawyer X wrote​:

On 4/8/19 2​:46 AM, James E Keenan via RT wrote​:

On Sun, 07 Apr 2019 19​:31​:16 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.29.9.

-----------------------------------------------------------------
The test suite of ANNO/Unicode-CharWidth-1.05.tar.gz fails
since perl 5.29.8. A sample fail report on CPAN Testers​:
http​://www.cpantesters.org/cpan/report/ca2cd882-5801-11e9-a3fc-
1703e4e5c01a

With this bisection command​:

#####
perl Porting/bisect.pl --start=7b97bf55c0
--end=ca8b93afd02ddde55c1aa9e6fbff9acdad31593d
--module=Unicode​::CharWidth
#####

... I got this bisection result​:

#####
73b9584 is the first bad commit
commit 73b9584 Author​: Karl
Williamson <khw@​cpan.org> Date​: Mon Aug 20 18​:31​:04 2018 -0600
Move \p{user-defined} to core from utf8_heavy.pl
#####

... which is the same commit where breakage was reported in
https://rt.perl.org/Public/Bug/Display.html?id=133860. So I
recommend that this ticket be merged into that one.

I was going to suggest the same.

There are at least two causes for this breakage, so merging is
premature

Too late; I acted on sawyer's +1 about 4 hours ago.  Suggestions?

jimk

I don't think it's possible to unmerge.  It will cause extra work for
me if it's not the same cause.  Just don't merge any more to this
ticket or the other one that has the same bisect result

It turns out the root cause of this problem is completely different
from the one the ticket it got merged into is.  It would be good if we
could unmerge.

Sorry about that. I figured "same commit -- same breakage."

  Does anyone have any suggestions?

As Jim noted, creating a new ticket that points to the original way,
clarifying this is a separate ticket to follow-up on a particular cause
that is not shared.

@p5pRT
Copy link
Author

p5pRT commented Apr 9, 2019

From @dur-randir

On Tue, 09 Apr 2019 08​:49​:44 -0700, randir wrote​:

CV* are not moved, but it's address may be reused, if the sub is freed.

As an afterthought, thread creation changes all pointers in a new thread, if it matters in this case.

@p5pRT
Copy link
Author

p5pRT commented Apr 13, 2019

From @khwilliamson

The #134004 portion of this merged ticket is fixed by

commit ef80af0
Author​: Karl Williamson <khw@​cpan.org>
Date​: Fri Apr 12 15​:45​:32 2019 -0600

* PATCH​: [perl #134004] BBC breaks Unicode​::CharWidth
 
  A user-defined property \p{IsFoo} is package specific, and can be
  specified with :​: package qualifiers \p{pkg1​::pkg2​::...​::IsFoo}. Some
  other package can also define an IsFoo which is totally independent of
  the first. These properties are implemented by definining a sub IsFoo()
  in the proper package. I used cv_name() to get the fully qualified name
  of the sub. The problem with that is that it can evaluate to
  pkg1​::pkg2​::...​::_ANON_, for example. What I really want is the
  property name IsFoo, fully qualified. This commit changes to do that.
--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented Apr 22, 2019

From @khwilliamson

On Tue, 09 Apr 2019 07​:16​:02 -0700, public@​khwilliamson.com wrote​:

I've removed this from the blockers list. The only distro that has showed up so far that relied on previous buggy behavior is the one in #133860, and a PR request has been issued for it.

@p5pRT
Copy link
Author

p5pRT commented May 16, 2019

From @ppisar

Created by @ppisar

I noticed Net-IDN-Encode tests fail with Perl 5.30-RC1
<https://rt.cpan.org/Ticket/Display.html?id=129588>. According to CPAN testers
the failure emerged with 5.29.8. (Though the exact way of failing changed
then.) When debugging it I reduced it to the point where I believe it is a bug
in the interpreter.

This program​:

#!/usr/bin/perl
package Foo;

sub make {
  my @​lines;
  while( my($c) = splice(@​_,0,1) ) {
  warn "HIT";
  push @​lines, sprintf("%04X", $c);
  }
  return join "\n", @​lines;
}

my @​characters = ( 0x0061 );
sub IsProperty { make(@​characters); };

if ('a' =~ /\p{IsProperty}/) {
  print "ok\n";
} else {
  print "not ok\n";
}
__END__

correctly recognizes 'a' character as \x0061 code point on Perl 5.28.2 because
the while-loop iterates over @​characters list​:

$ perl /tmp/test
HIT at /tmp/test line 7.
ok

However, this not true on Perl 5.30​:

$ perl /tmp/test
not ok

Calling make(@​characters) directly works as expected. Replacing make argument
with a literal list (make(0x0061)) works. Replacing the while-splice with
a for(@​_) also works. Thus I conclude there is some strange interaction
between user-defined unicode properties and splice.

-- Petr

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl 5.30.0:

Configured by petr at Thu May 16 17:35:41 CEST 2019.

Summary of my perl5 (revision 5 version 30 subversion 0) configuration:
  Commit id: 3271cf246fc5e8e632a91eaeadd71a58c7eaebb8
  Platform:
    osname=linux
    osvers=5.0.14-200.fc29.x86_64
    archname=x86_64-linux-thread-multi
    uname='linux dhcp-0-146.brq.redhat.com 5.0.14-200.fc29.x86_64 #1 smp thu may 9 10:46:15 utc 2019 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Dusedevel -Dusethreads -Duseithreads -Dldflags=-lm -lpthread -Accflags=-I/usr/src/kernels/5.0.14-200.fc29.x86_64/arch/x86/include'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-D_REENTRANT -D_GNU_SOURCE -I/usr/src/kernels/5.0.14-200.fc29.x86_64/arch/x86/include -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O2'
    cppflags='-D_REENTRANT -D_GNU_SOURCE -I/usr/src/kernels/5.0.14-200.fc29.x86_64/arch/x86/include -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='8.3.1 20190223 (Red Hat 8.3.1-2)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags ='-lm -lpthread -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lpthread -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.28.so
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.28'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'



@INC for perl 5.30.0:
    lib
    /usr/local/lib/perl5/site_perl/5.30.0/x86_64-linux-thread-multi
    /usr/local/lib/perl5/site_perl/5.30.0
    /usr/local/lib/perl5/5.30.0/x86_64-linux-thread-multi
    /usr/local/lib/perl5/5.30.0


Environment for perl 5.30.0:
    HOME=/home/petr
    LANG=cs_CZ.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/petr/bin:/usr/share/Modules/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented May 16, 2019

From @jkeenan

On Thu, 16 May 2019 16​:14​:21 GMT, ppisar wrote​:

This is a bug report for perl from ppisar@​redhat.com,
generated with the help of perlbug 1.41 running under perl 5.30.0.

-----------------------------------------------------------------
[Please describe your issue here]

I noticed Net-IDN-Encode tests fail with Perl 5.30-RC1
<https://rt.cpan.org/Ticket/Display.html?id=129588>. According to CPAN
testers
the failure emerged with 5.29.8. (Though the exact way of failing
changed
then.) When debugging it I reduced it to the point where I believe it
is a bug
in the interpreter.

This program​:

#!/usr/bin/perl
package Foo;

The line above is relevant. If you comment it out, this test program prints "not ok" on both 5.28 and blead.

Do you know why that would be so?

sub make {
my @​lines;
while( my($c) = splice(@​_,0,1) ) {
warn "HIT";
push @​lines, sprintf("%04X", $c);
}
return join "\n", @​lines;
}

my @​characters = ( 0x0061 );
sub IsProperty { make(@​characters); };

if ('a' =~ /\p{IsProperty}/) {
print "ok\n";
} else {
print "not ok\n";
}
__END__

correctly recognizes 'a' character as \x0061 code point on Perl 5.28.2
because
the while-loop iterates over @​characters list​:

$ perl /tmp/test
HIT at /tmp/test line 7.
ok

However, this not true on Perl 5.30​:

$ perl /tmp/test
not ok

Calling make(@​characters) directly works as expected. Replacing make
argument
with a literal list (make(0x0061)) works. Replacing the while-splice
with
a for(@​_) also works. Thus I conclude there is some strange
interaction
between user-defined unicode properties and splice.

-- Petr

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented May 16, 2019

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented May 16, 2019

From @jkeenan

On Thu, 16 May 2019 17​:20​:07 GMT, jkeenan wrote​:

On Thu, 16 May 2019 16​:14​:21 GMT, ppisar wrote​:

This is a bug report for perl from ppisar@​redhat.com,
generated with the help of perlbug 1.41 running under perl 5.30.0.

-----------------------------------------------------------------
[Please describe your issue here]

I noticed Net-IDN-Encode tests fail with Perl 5.30-RC1
<https://rt.cpan.org/Ticket/Display.html?id=129588>. According to
CPAN
testers
the failure emerged with 5.29.8. (Though the exact way of failing
changed
then.) When debugging it I reduced it to the point where I believe it
is a bug
in the interpreter.

This program​:

#!/usr/bin/perl
package Foo;

The line above is relevant. If you comment it out, this test program
prints "not ok" on both 5.28 and blead.

Do you know why that would be so?

sub make {
my @​lines;
while( my($c) = splice(@​_,0,1) ) {
warn "HIT";
push @​lines, sprintf("%04X", $c);
}
return join "\n", @​lines;
}

my @​characters = ( 0x0061 );
sub IsProperty { make(@​characters); };

if ('a' =~ /\p{IsProperty}/) {
print "ok\n";
} else {
print "not ok\n";
}
__END__

correctly recognizes 'a' character as \x0061 code point on Perl
5.28.2
because
the while-loop iterates over @​characters list​:

$ perl /tmp/test
HIT at /tmp/test line 7.
ok

However, this not true on Perl 5.30​:

$ perl /tmp/test
not ok

Calling make(@​characters) directly works as expected. Replacing make
argument
with a literal list (make(0x0061)) works. Replacing the while-splice
with
a for(@​_) also works. Thus I conclude there is some strange
interaction
between user-defined unicode properties and splice.

-- Petr

Thank you very much.

Original problem confirmed on FreeBSD-11.2.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented May 16, 2019

From @jkeenan

On Thu, 16 May 2019 17​:29​:19 GMT, jkeenan wrote​:

On Thu, 16 May 2019 17​:20​:07 GMT, jkeenan wrote​:

On Thu, 16 May 2019 16​:14​:21 GMT, ppisar wrote​:

This is a bug report for perl from ppisar@​redhat.com,
generated with the help of perlbug 1.41 running under perl 5.30.0.

-----------------------------------------------------------------
[Please describe your issue here]

I noticed Net-IDN-Encode tests fail with Perl 5.30-RC1
<https://rt.cpan.org/Ticket/Display.html?id=129588>. According to
CPAN
testers
the failure emerged with 5.29.8. (Though the exact way of failing
changed
then.) When debugging it I reduced it to the point where I believe it
is a bug
in the interpreter.

This program​:

#!/usr/bin/perl
package Foo;

The line above is relevant. If you comment it out, this test program
prints "not ok" on both 5.28 and blead.

Do you know why that would be so?

sub make {
my @​lines;
while( my($c) = splice(@​_,0,1) ) {
warn "HIT";
push @​lines, sprintf("%04X", $c);
}
return join "\n", @​lines;
}

my @​characters = ( 0x0061 );
sub IsProperty { make(@​characters); };

if ('a' =~ /\p{IsProperty}/) {
print "ok\n";
} else {
print "not ok\n";
}
__END__

correctly recognizes 'a' character as \x0061 code point on Perl
5.28.2
because
the while-loop iterates over @​characters list​:

$ perl /tmp/test
HIT at /tmp/test line 7.
ok

However, this not true on Perl 5.30​:

$ perl /tmp/test
not ok

Calling make(@​characters) directly works as expected. Replacing make
argument
with a literal list (make(0x0061)) works. Replacing the while-splice
with
a for(@​_) also works. Thus I conclude there is some strange
interaction
between user-defined unicode properties and splice.

-- Petr

Thank you very much.

Original problem confirmed on FreeBSD-11.2.

FWIW, bisection pointed to this commit​:

#####
73b9584 is the first bad commit
commit 73b9584
Author​: Karl Williamson <khw@​cpan.org>
Date​: Mon Aug 20 18​:31​:04 2018 -0600

Move \p{user-defined} to core from utf8_heavy.pl
#####

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented May 17, 2019

From @khwilliamson

Now fixed by
commit 4531f51
Author​: Karl Williamson <khw@​cpan.org>
Date​: Thu May 16 15​:47​:20 2019 -0600

  PATCH​: [perl #133860] 5.30 regression
 
  These bugs stem from trying to compile a user-defined \p{IsProperty}
  before the data for the property is available. In the past, a bug used
  the wrong package for IsProperty, and it wasn't found, so its expansion
  was delayed until runtime. But that bug got fixed, and now it finds the
  property and thinks its deliberately empty, at compile time.
 
  This is a change in behavior, even if it is fixing a bug, where the real
  problem is unobvious. The solution adopted in this commit is to defer
  all empty properties at pattern compilation time. If they are still
  empty at runtime, that's what the expansion will be.
--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented May 17, 2019

@khwilliamson - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this as completed May 17, 2019
@p5pRT
Copy link
Author

p5pRT commented May 19, 2019

From @khwilliamson

On 5/17/19 8​:31 AM, Karl Williamson wrote​:

I'm thinking this should be in RC2.

It is now in blead, and very soon RC2 will be released.

Thank you for pursuing this problem. This ticket is a duplicate of
#133860, and I merged them together, and both are now resolved.

Earlier when this bug surfaced, as 133860, that appeared to be the only
affected module,and I issued a PR, which has yet to be acted upon.  It
is somewhat arguable that this is a bug in the program, calling a
function earlier in the compilation process than the data it relies on
is available.  But this is a silent change of behavior in Perl, and its
not obvious, even to very experienced programmers what is happening.

The branch smoke-me/khw-petr has a patch to fix this, now including a
test.  I think it should be applied to blead, and an RC2 started ASAP.

On 5/17/19 2​:05 AM, Petr Pisar wrote​:

On Thu, May 16, 2019 at 03​:57​:43PM -0600, Karl Williamson wrote​:

Note that the RT/email system is broken again, so make sure all
recipients
you want to be sure to get this are on the cc​: list

On 5/16/19 12​:20 PM, Karl Williamson wrote​:

On 5/16/19 11​:29 AM, James E Keenan via RT wrote​:

On Thu, 16 May 2019 17​:20​:07 GMT, jkeenan wrote​:

On Thu, 16 May 2019 16​:14​:21 GMT, ppisar wrote​:

This is a bug report for perl from ppisar@​redhat.com,
generated with the help of perlbug 1.41 running under perl 5.30.0.

-----------------------------------------------------------------
[Please describe your issue here]

I noticed Net-IDN-Encode tests fail with Perl 5.30-RC1
<https://rt.cpan.org/Ticket/Display.html?id=129588>. According to
CPAN
testers
the failure emerged with 5.29.8. (Though the exact way of failing
changed
then.) When debugging it I reduced it to the point where I
believe it
is a bug
in the interpreter.

This program​:

#!/usr/bin/perl
package Foo;

The line above is relevant.  If you comment it out, this test program
prints "not ok" on both 5.28 and blead.

Do you know why that would be so?

sub make {
      my @​lines;
      while( my($c) = splice(@​_,0,1) ) {
          warn "HIT";
          push @​lines, sprintf("%04X", $c);
      }
      return join "\n", @​lines;
}

my @​characters = ( 0x0061 );
sub IsProperty { make(@​characters); };

if ('a' =~ /\p{IsProperty}/) {
      print "ok\n";
} else {
      print "not ok\n";
}
__END__

correctly recognizes 'a' character as \x0061 code point on Perl
5.28.2
because
the while-loop iterates over @​characters list​:

$ perl /tmp/test
HIT at /tmp/test line 7.
ok

However, this not true on Perl 5.30​:

$ perl /tmp/test
not ok

Calling make(@​characters) directly works as expected. Replacing make
argument
with a literal list (make(0x0061)) works. Replacing the while-splice
with
a for(@​_) also works. Thus I conclude there is some strange
interaction
between user-defined unicode properties and splice.

I don't see any difference in behavior between the while() and the
for().  Are you sure?

You are right. It also does not work with for(). I probably made a
mistake.

I still don't see any differences in behaviors.  I now think we
should fix
this.  And the only way how that I can think of is the same thing I
thought
of earlier when we were seeing if the Net-IDN-Encode tests were the only
things failing

https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133860

And the way I see to fix it, is to not presume that the function is
fully
operational if it returns nothing when called at pattern compilation
time,
but to defer it to runtime, like we do any function whose definition we
haven't yet encountered.

If it still is empty at runtime, then assume that that was what was
meant.

Attached is a patch that does this, and it is smoking on the branch
smoke-me/khw-petr.

I confirm the patch works for me.

-- Petr

@p5pRT
Copy link
Author

p5pRT commented May 19, 2019

From @ppisar

On Thu, May 16, 2019 at 03​:57​:43PM -0600, Karl Williamson wrote​:

Note that the RT/email system is broken again, so make sure all recipients
you want to be sure to get this are on the cc​: list

On 5/16/19 12​:20 PM, Karl Williamson wrote​:

On 5/16/19 11​:29 AM, James E Keenan via RT wrote​:

On Thu, 16 May 2019 17​:20​:07 GMT, jkeenan wrote​:

On Thu, 16 May 2019 16​:14​:21 GMT, ppisar wrote​:

This is a bug report for perl from ppisar@​redhat.com,
generated with the help of perlbug 1.41 running under perl 5.30.0.

-----------------------------------------------------------------
[Please describe your issue here]

I noticed Net-IDN-Encode tests fail with Perl 5.30-RC1
<https://rt.cpan.org/Ticket/Display.html?id=129588>. According to
CPAN
testers
the failure emerged with 5.29.8. (Though the exact way of failing
changed
then.) When debugging it I reduced it to the point where I believe it
is a bug
in the interpreter.

This program​:

#!/usr/bin/perl
package Foo;

The line above is relevant.  If you comment it out, this test program
prints "not ok" on both 5.28 and blead.

Do you know why that would be so?

sub make {
     my @​lines;
     while( my($c) = splice(@​_,0,1) ) {
         warn "HIT";
         push @​lines, sprintf("%04X", $c);
     }
     return join "\n", @​lines;
}

my @​characters = ( 0x0061 );
sub IsProperty { make(@​characters); };

if ('a' =~ /\p{IsProperty}/) {
     print "ok\n";
} else {
     print "not ok\n";
}
__END__

correctly recognizes 'a' character as \x0061 code point on Perl
5.28.2
because
the while-loop iterates over @​characters list​:

$ perl /tmp/test
HIT at /tmp/test line 7.
ok

However, this not true on Perl 5.30​:

$ perl /tmp/test
not ok

Calling make(@​characters) directly works as expected. Replacing make
argument
with a literal list (make(0x0061)) works. Replacing the while-splice
with
a for(@​_) also works. Thus I conclude there is some strange
interaction
between user-defined unicode properties and splice.

I don't see any difference in behavior between the while() and the
for().  Are you sure?

You are right. It also does not work with for(). I probably made a mistake.

I still don't see any differences in behaviors. I now think we should fix
this. And the only way how that I can think of is the same thing I thought
of earlier when we were seeing if the Net-IDN-Encode tests were the only
things failing

https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133860

And the way I see to fix it, is to not presume that the function is fully
operational if it returns nothing when called at pattern compilation time,
but to defer it to runtime, like we do any function whose definition we
haven't yet encountered.

If it still is empty at runtime, then assume that that was what was meant.

Attached is a patch that does this, and it is smoking on the branch
smoke-me/khw-petr.

I confirm the patch works for me.

-- Petr

@p5pRT
Copy link
Author

p5pRT commented May 19, 2019

From @khwilliamson

I'm thinking this should be in RC2.

Earlier when this bug surfaced, as 133860, that appeared to be the only
affected module,and I issued a PR, which has yet to be acted upon. It
is somewhat arguable that this is a bug in the program, calling a
function earlier in the compilation process than the data it relies on
is available. But this is a silent change of behavior in Perl, and its
not obvious, even to very experienced programmers what is happening.

The branch smoke-me/khw-petr has a patch to fix this, now including a
test. I think it should be applied to blead, and an RC2 started ASAP.

On 5/17/19 2​:05 AM, Petr Pisar wrote​:

On Thu, May 16, 2019 at 03​:57​:43PM -0600, Karl Williamson wrote​:

Note that the RT/email system is broken again, so make sure all recipients
you want to be sure to get this are on the cc​: list

On 5/16/19 12​:20 PM, Karl Williamson wrote​:

On 5/16/19 11​:29 AM, James E Keenan via RT wrote​:

On Thu, 16 May 2019 17​:20​:07 GMT, jkeenan wrote​:

On Thu, 16 May 2019 16​:14​:21 GMT, ppisar wrote​:

This is a bug report for perl from ppisar@​redhat.com,
generated with the help of perlbug 1.41 running under perl 5.30.0.

-----------------------------------------------------------------
[Please describe your issue here]

I noticed Net-IDN-Encode tests fail with Perl 5.30-RC1
<https://rt.cpan.org/Ticket/Display.html?id=129588>. According to
CPAN
testers
the failure emerged with 5.29.8. (Though the exact way of failing
changed
then.) When debugging it I reduced it to the point where I believe it
is a bug
in the interpreter.

This program​:

#!/usr/bin/perl
package Foo;

The line above is relevant.  If you comment it out, this test program
prints "not ok" on both 5.28 and blead.

Do you know why that would be so?

sub make {
     my @​lines;
     while( my($c) = splice(@​_,0,1) ) {
         warn "HIT";
         push @​lines, sprintf("%04X", $c);
     }
     return join "\n", @​lines;
}

my @​characters = ( 0x0061 );
sub IsProperty { make(@​characters); };

if ('a' =~ /\p{IsProperty}/) {
     print "ok\n";
} else {
     print "not ok\n";
}
__END__

correctly recognizes 'a' character as \x0061 code point on Perl
5.28.2
because
the while-loop iterates over @​characters list​:

$ perl /tmp/test
HIT at /tmp/test line 7.
ok

However, this not true on Perl 5.30​:

$ perl /tmp/test
not ok

Calling make(@​characters) directly works as expected. Replacing make
argument
with a literal list (make(0x0061)) works. Replacing the while-splice
with
a for(@​_) also works. Thus I conclude there is some strange
interaction
between user-defined unicode properties and splice.

I don't see any difference in behavior between the while() and the
for().  Are you sure?

You are right. It also does not work with for(). I probably made a mistake.

I still don't see any differences in behaviors. I now think we should fix
this. And the only way how that I can think of is the same thing I thought
of earlier when we were seeing if the Net-IDN-Encode tests were the only
things failing

https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133860

And the way I see to fix it, is to not presume that the function is fully
operational if it returns nothing when called at pattern compilation time,
but to defer it to runtime, like we do any function whose definition we
haven't yet encountered.

If it still is empty at runtime, then assume that that was what was meant.

Attached is a patch that does this, and it is smoking on the branch
smoke-me/khw-petr.

I confirm the patch works for me.

-- Petr

@p5pRT p5pRT added BBC Blead Breaks CPAN - changes in blead broke a cpan module(s) Severity Low distro-All labels Oct 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BBC Blead Breaks CPAN - changes in blead broke a cpan module(s) distro-All
Projects
None yet
Development

No branches or pull requests

1 participant