Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bleadperl v5.25.8-36-g6cdc5cd8f3 breaks HURRICUP/DTL-Fast-1.622.tar.gz #15834

Closed
p5pRT opened this issue Jan 25, 2017 · 20 comments
Closed

Bleadperl v5.25.8-36-g6cdc5cd8f3 breaks HURRICUP/DTL-Fast-1.622.tar.gz #15834

p5pRT opened this issue Jan 25, 2017 · 20 comments

Comments

@p5pRT
Copy link

p5pRT commented Jan 25, 2017

Migrated from rt.perl.org#130641 (status was 'resolved')

Searchable as RT130641$

@p5pRT
Copy link
Author

p5pRT commented Jan 25, 2017

From @andk

Kudos to Slaven, who discovered the two distros!

bisect


commit 6cdc5cd
Author​: Karl Williamson <khw@​cpan.org>
Date​: Fri Dec 2 09​:35​:53 2016 -0700

  PATCH​: [perl #126310] single quote UTF-8 malformation detection

also affected


STEFFENW/Locale-TextDomain-OO-1.023.tar.gz

diagnostics


http​://www.cpantesters.org/cpan/report/76450368

More reports are on their way but not yet available on cpantesters.

perl -V


Summary of my perl5 (revision 5 version 25 subversion 9) configuration​:
  Commit id​: 6cdc5cd
  Platform​:
  osname=linux
  osvers=4.8.0-2-amd64
  archname=x86_64-linux-thread-multi
  uname='linux k93msid 4.8.0-2-amd64 #1 smp debian 4.8.11-1 (2016-12-02) x86_64 gnulinux '
  config_args='-Dprefix=/home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.25.8-36-g6cdc5cd8f3/f11c -Dmyhostname=k93msid -Dinstallusrbinperl=n -Uversiononly -Dusedevel -des -Ui_db -Dlibswanted=cl pthread socket inet nsl gdbm dbm malloc dl ld sun m crypt sec util c cposix posix ucb BSD gdbm_compat -Duseithreads -Uuselongdouble -DDEBUGGING=-g'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
  optimize='-O2 -g'
  cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='6.2.1 20161124'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.24.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.24'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  MULTIPLICITY
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_IMPLICIT_CONTEXT
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_ITHREADS
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  USE_REENTRANT_API
  Built under linux
  Compiled at Dec 23 2016 20​:47​:47
  @​INC​:
  /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.25.8-36-g6cdc5cd8f3/f11c/lib/site_perl/5.25.9/x86_64-linux-thread-multi
  /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.25.8-36-g6cdc5cd8f3/f11c/lib/site_perl/5.25.9
  /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.25.8-36-g6cdc5cd8f3/f11c/lib/5.25.9/x86_64-linux-thread-multi
  /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.25.8-36-g6cdc5cd8f3/f11c/lib/5.25.9
  .

--
andreas

@p5pRT
Copy link
Author

p5pRT commented Jan 25, 2017

From @iabyn

On Wed, Jan 25, 2017 at 04​:46​:25AM -0800, Andreas J. Koenig via RT wrote​:

# New Ticket Created by (Andreas J. Koenig)
# Please include the string​: [perl #130641]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=130641 >

Kudos to Slaven, who discovered the two distros!

bisect
------
commit 6cdc5cd
Author​: Karl Williamson <khw@​cpan.org>
Date​: Fri Dec 2 09​:35​:53 2016 -0700

PATCH&#8203;: \[perl \#126310\] single quote UTF\-8 malformation detection

For DTL-Fast, the test file t/xs/count_lines.t has the equivalent of the
following in it​:

  use utf8;
  my $s = <<EOF;
  ... five bytes with values \xfa\xfc\xfd\xfe\xff ...
  EOF

where the bytes making up the heredoc string aren't valid utf8. So in this
case it seems that perl is correctly flagging up illegal input in the
src file.

also affected
-------------
STEFFENW/Locale-TextDomain-OO-1.023.tar.gz

I haven't looked at that one.

--
Indomitable in retreat, invincible in advance, insufferable in victory
  -- Churchill on Montgomery

@p5pRT
Copy link
Author

p5pRT commented Jan 25, 2017

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jan 26, 2017

From @khwilliamson

On 01/25/2017 06​:43 AM, Dave Mitchell wrote​:

On Wed, Jan 25, 2017 at 04​:46​:25AM -0800, Andreas J. Koenig via RT wrote​:

# New Ticket Created by (Andreas J. Koenig)
# Please include the string​: [perl #130641]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=130641 >

Kudos to Slaven, who discovered the two distros!

bisect
------
commit 6cdc5cd
Author​: Karl Williamson <khw@​cpan.org>
Date​: Fri Dec 2 09​:35​:53 2016 -0700

PATCH&#8203;: \[perl \#126310\] single quote UTF\-8 malformation detection

For DTL-Fast, the test file t/xs/count_lines.t has the equivalent of the
following in it​:

use utf8;
my $s = \<\<EOF;
\.\.\. five bytes with values \\xfa\\xfc\\xfd\\xfe\\xff \.\.\.
EOF

where the bytes making up the heredoc string aren't valid utf8. So in this
case it seems that perl is correctly flagging up illegal input in the
src file.

also affected
-------------
STEFFENW/Locale-TextDomain-OO-1.023.tar.gz

I haven't looked at that one.

I could not easily get L​::TD​::OO to install on my system. And I'm not
sure why. It looks like some pre-reqs are not listed, but I still
couldn't get it to work when I installed some of them by hand. Is there
a log where the failure is shown?

@p5pRT
Copy link
Author

p5pRT commented Jan 26, 2017

From @andk

On Thu, 26 Jan 2017 11​:39​:07 -0800, "karl williamson via RT" <perlbug-followup@​perl.org> said​:

STEFFENW/Locale-TextDomain-OO-1.023.tar.gz

I haven't looked at that one.

  > I could not easily get L​::TD​::OO to install on my system. And I'm not
  > sure why. It looks like some pre-reqs are not listed, but I still
  > couldn't get it to work when I installed some of them by hand. Is there
  > a log where the failure is shown?

One test from Dan Collins has arrived, it looks the same as my results​:

http​://www.cpantesters.org/cpan/report/92113ba8-cd6d-11e6-8b63-3f8052daa5e1

--
andreas

@p5pRT
Copy link
Author

p5pRT commented Jan 26, 2017

From @iabyn

On Thu, Jan 26, 2017 at 09​:50​:19PM +0100, Andreas Koenig wrote​:

On Thu, 26 Jan 2017 11​:39​:07 -0800, "karl williamson via RT" <perlbug-followup@​perl.org> said​:

STEFFENW/Locale-TextDomain-OO-1.023.tar.gz

I haven't looked at that one.

I could not easily get L​::TD​::OO to install on my system. And I'm not
sure why. It looks like some pre-reqs are not listed, but I still
couldn't get it to work when I installed some of them by hand. Is there
a log where the failure is shown?

One test from Dan Collins has arrived, it looks the same as my results​:

http​://www.cpantesters.org/cpan/report/92113ba8-cd6d-11e6-8b63-3f8052daa5e1

I didn't have any problem building it on my system. Its a fault in
t/51_test_examples.t, line 172, which has the rough equivalent of

  use utf8;
  ...
  <<EOT
  ...
  ... a line containing the following bytes​:
  a7 20 42 75 63 68 0d 0a
  ...
  EOT

So again, the new check in perl is doing the Right Thing.

--
Never do today what you can put off till tomorrow.

@p5pRT
Copy link
Author

p5pRT commented Jan 26, 2017

From @jkeenan

On Thu, 26 Jan 2017 20​:51​:14 GMT, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

On Thu, 26 Jan 2017 11​:39​:07 -0800, "karl williamson via RT"
<perlbug-followup@​perl.org> said​:

STEFFENW/Locale-TextDomain-OO-1.023.tar.gz

I haven't looked at that one.

I could not easily get L​::TD​::OO to install on my system. And I'm
not
sure why. It looks like some pre-reqs are not listed, but I still
couldn't get it to work when I installed some of them by hand. Is
there
a log where the failure is shown?

One test from Dan Collins has arrived, it looks the same as my
results​:

http​://www.cpantesters.org/cpan/report/92113ba8-cd6d-11e6-8b63-
3f8052daa5e1

I also got the failure using cpanm to install against a threaded perl built at commit c1b4bd9 (committed Jan 11)​:

#####
t/43_singleton_gettext_mo.t ............ ok
Malformed UTF-8 character​: \xa7 (unexpected continuation byte 0xa7, with no preceding start byte) at t/51_test_examples.t line 172.
Malformed UTF-8 character (fatal) at t/51_test_examples.t line 172.
t/51_test_examples.t ...................
Dubious, test returned 255 (wstat 65280, 0xff00)
No subtests run
t/52_test_examples_utf-8.t ............. skipped​: Set $ENV{AUTHOR_TESTING} to run this test.
...
Test Summary Report


t/51_test_examples.t (Wstat​: 65280 Tests​: 0 Failed​: 0)
  Non-zero exit status​: 255
  Parse errors​: No plan found in TAP output
Files=40, Tests=452, 5 wallclock secs ( 0.12 usr 0.05 sys + 4.52 cusr 0.30 csys = 4.99 CPU)
Result​: FAIL
Failed 1/40 test programs. 0/452 subtests failed.
Makefile​:924​: recipe for target 'test_dynamic' failed
make​: *** [test_dynamic] Error 255
-> FAIL Installing Locale​::TextDomain​::OO failed. See /home/jkeenan/.cpanm/work/1485467623.20662/build.log for details. Retry with --force to force install it.
Expiring 5 work directories.
68 distributions installed
#####
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Jan 27, 2017

From @khwilliamson

On 01/26/2017 03​:02 PM, Dave Mitchell wrote​:

On Thu, Jan 26, 2017 at 09​:50​:19PM +0100, Andreas Koenig wrote​:

On Thu, 26 Jan 2017 11​:39​:07 -0800, "karl williamson via RT" <perlbug-followup@​perl.org> said​:

STEFFENW/Locale-TextDomain-OO-1.023.tar.gz

I haven't looked at that one.

I could not easily get L​::TD​::OO to install on my system. And I'm not
sure why. It looks like some pre-reqs are not listed, but I still
couldn't get it to work when I installed some of them by hand. Is there
a log where the failure is shown?

One test from Dan Collins has arrived, it looks the same as my results​:

http​://www.cpantesters.org/cpan/report/92113ba8-cd6d-11e6-8b63-3f8052daa5e1

I didn't have any problem building it on my system. Its a fault in
t/51_test_examples.t, line 172, which has the rough equivalent of

use utf8;
\.\.\.
\<\<EOT
\.\.\.
\.\.\. a line containing the following bytes&#8203;:
a7  20  42  75  63  68  0d  0a
\.\.\.
EOT

So again, the new check in perl is doing the Right Thing.

Ok, then, we should reject this ticket, and write ones on the distros.
Who does that?

@p5pRT
Copy link
Author

p5pRT commented Jan 27, 2017

From @andk

On Thu, 26 Jan 2017 19​:31​:43 -0800, "karl williamson via RT" <perlbug-followup@​perl.org> said​:

  > Ok, then, we should reject this ticket, and write ones on the distros.
  > Who does that?

My favorite workflow would be there is no reject for BBCs unless the BBC
itself is faulty, say, the reporter had dirt in his eyes.

If it is a valid BBC, either perl is broken or modules on CPAN. The next
release of perl needs to deal with it some way. The ticket should remain
open until we have developed confidence that enough repair has been
applied and enough time has passed. The time factor is necessary because
we expect some "also affected" messages to arrive a bit later than the
original ticket.

After a BBC, we must face it, there is a need to improve either perl or
CPAN. If it is CPAN, module authors need help, and there's no way to
declare that as somebody else's problem.

Alternate views welcome.

--
andreas

@p5pRT
Copy link
Author

p5pRT commented Jan 27, 2017

From @iabyn

On Fri, Jan 27, 2017 at 08​:29​:39AM +0100, Andreas Koenig wrote​:

After a BBC, we must face it, there is a need to improve either perl or
CPAN. If it is CPAN, module authors need help, and there's no way to
declare that as somebody else's problem.

I suppose the main thing here is who is "responsible" for opening
RT/github tickets for those distributions? The diagnoses found here can
then be easily relayed to the authors. In the past such tickets normally
seem to magically appear - but I don't know whether anyone here considers
it one of their normal roles (in the same way that I don't personally
worry about whether the latest versions of CPAN modules have been brought
into blead; people like Chris Williams seem to just magically do it.)

--
Never work with children, animals, or actors.

@p5pRT
Copy link
Author

p5pRT commented Jan 27, 2017

From @andk

also affected​:

DOCCY/Config-FileManager-1.6.tar.gz
KIRILLOVA/Lingua-Numending-0.01.tar.gz

--
andreas

@p5pRT
Copy link
Author

p5pRT commented Jan 31, 2017

From @khwilliamson

On Thu, 26 Jan 2017 23​:30​:29 -0800, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

On Thu, 26 Jan 2017 19​:31​:43 -0800, "karl williamson via RT"
<perlbug-followup@​perl.org> said​:

Ok, then, we should reject this ticket, and write ones on the
distros.
Who does that?

My favorite workflow would be there is no reject for BBCs unless the
BBC
itself is faulty, say, the reporter had dirt in his eyes.

If it is a valid BBC, either perl is broken or modules on CPAN. The
next
release of perl needs to deal with it some way. The ticket should
remain
open until we have developed confidence that enough repair has been
applied and enough time has passed. The time factor is necessary
because
we expect some "also affected" messages to arrive a bit later than the
original ticket.

After a BBC, we must face it, there is a need to improve either perl
or
CPAN. If it is CPAN, module authors need help, and there's no way to
declare that as somebody else's problem.

Alternate views welcome.

Are confirmation views welcome? :)

I agree with your workflow.
--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented Jan 31, 2017

From @andk

On Mon, 30 Jan 2017 18​:56​:14 -0800, "Karl Williamson via RT" <perlbug-followup@​perl.org> said​:

Alternate views welcome.

  > Are confirmation views welcome? :)

  > I agree with your workflow.

:)
--
andreas

@p5pRT
Copy link
Author

p5pRT commented Feb 1, 2017

From @xsawyerx

On 01/27/2017 04​:46 PM, Dave Mitchell wrote​:

On Fri, Jan 27, 2017 at 08​:29​:39AM +0100, Andreas Koenig wrote​:

After a BBC, we must face it, there is a need to improve either perl or
CPAN. If it is CPAN, module authors need help, and there's no way to
declare that as somebody else's problem.
I suppose the main thing here is who is "responsible" for opening
RT/github tickets for those distributions? The diagnoses found here can
then be easily relayed to the authors. In the past such tickets normally
seem to magically appear - but I don't know whether anyone here considers
it one of their normal roles (in the same way that I don't personally
worry about whether the latest versions of CPAN modules have been brought
into blead; people like Chris Williams seem to just magically do it.)

I definitely agree Chris Williams is magical. :)

As for the BBC workflow​: I believe Andreas phrased it well​: If there's a
BBC, either perl is broken and needs fixing or a module is broken and
needs fixing. The latter includes tests that are written incorrectly, as
it seems this case might be.

I don't know if we ever clarified a strict BBC handling policy, but the
following seems to me what is happening​:
* A smoker opens a BBC ticket.
* In the ticket we determine if perl is doing the right thing or not.
* If we're doing the right thing, we (usually Andreas, Slaven, Jim - for
which I'm very grateful) open a ticket with the module. Sometimes this
is accompanied with a patch.
* We make a decision on whether we should revert or not, depending on
the amount of breakage and the importance of the change we've made.

This seems to me like a very good workflow, as it really tries to
balance requirements and to serve the community well. We know of the
affect our changes have on modules and module authors know if they made
any mistakes.

As to Dave's question​: Whose responsibility is it to open the RT/Github
ticket for the distribution? I think we should be responsible of it.
This serves the purpose of both alerting the author, but also alerting
the users of the offending distribution, and finally allows us to easily
track and prove the interaction with the author to try and accommodate
this situation.

Yves once made a point I keep thinking about​: Issue trackers are not
just for the author. They are primarily for the user, so they have a
place to both interact and report issues, but also a resource for
coordinating on fixes. It makes me think of how often I find in RT
queues a patch that wasn't applied yet, or an explanation of what I did
wrong. This is what I mean by helping the users of the distribution. The
other reasons (helping the author and us) I think are pretty clear.

I am willing to usurp the role of making sure all BBCs have a
correlating ticket in the RT or Github issues queue of the associated
distribution, if this is something we see falling through the cracks.

@p5pRT
Copy link
Author

p5pRT commented Feb 20, 2017

From @tonycoz

On Wed, 01 Feb 2017 03​:27​:00 -0800, xsawyerx@​gmail.com wrote​:

I am willing to usurp the role of making sure all BBCs have a
correlating ticket in the RT or Github issues queue of the associated
distribution, if this is something we see falling through the cracks.

The dists mentioned here have the following tickets (not opened by me)​:

HURRICUP/DTL-Fast-1.622.tar.gz
  hurricup/DTL-Fast#116

STEFFENW/Locale-TextDomain-OO-1.023.tar.gz
  https://rt.cpan.org/Ticket/Display.html?id=120137

DOCCY/Config-FileManager-1.6.tar.gz
  https://rt.cpan.org/Ticket/Display.html?id=120136

KIRILLOVA/Lingua-Numending-0.01.tar.gz
  https://rt.cpan.org/Ticket/Display.html?id=120135

Tony

@p5pRT
Copy link
Author

p5pRT commented Mar 20, 2017

From @iabyn

On Mon, Feb 20, 2017 at 12​:36​:52AM -0800, Tony Cook via RT wrote​:

On Wed, 01 Feb 2017 03​:27​:00 -0800, xsawyerx@​gmail.com wrote​:

I am willing to usurp the role of making sure all BBCs have a
correlating ticket in the RT or Github issues queue of the associated
distribution, if this is something we see falling through the cracks.

The dists mentioned here have the following tickets (not opened by me)​:

HURRICUP/DTL-Fast-1.622.tar.gz
hurricup/DTL-Fast#116

This has been fixed in DTL-Fast-1.623

STEFFENW/Locale-TextDomain-OO-1.023.tar.gz
https://rt.cpan.org/Ticket/Display.html?id=120137

This has been fixed in Locale-TextDomain-OO-1.026

DOCCY/Config-FileManager-1.6.tar.gz
https://rt.cpan.org/Ticket/Display.html?id=120136

Ticket is still new, no new release.

KIRILLOVA/Lingua-Numending-0.01.tar.gz
https://rt.cpan.org/Ticket/Display.html?id=120135

Ticket is still new, no new release.

Since there's nothing needing fixing in blead, and I don't think anyone is
suggesting reverting the new 'Malformed UTF-8 character' fatal error,
I think this ticket should be removed from the blockers list. I'll do this
in a few days time if no-one objects.

--
My Dad used to say 'always fight fire with fire', which is probably why
he got thrown out of the fire brigade.

@p5pRT
Copy link
Author

p5pRT commented Mar 27, 2017

From @iabyn

On Mon, Mar 20, 2017 at 03​:20​:41PM +0000, Dave Mitchell wrote​:

I think this ticket should be removed from the blockers list. I'll do this
in a few days time if no-one objects.

Now removed.

--
The crew of the Enterprise encounter an alien life form which is
surprisingly neither humanoid nor made from pure energy.
  -- Things That Never Happen in "Star Trek" #22

@p5pRT
Copy link
Author

p5pRT commented Mar 27, 2017

From @khwilliamson

On Mon, 27 Mar 2017 03​:39​:51 -0700, davem wrote​:

On Mon, Mar 20, 2017 at 03​:20​:41PM +0000, Dave Mitchell wrote​:

I think this ticket should be removed from the blockers list. I'll do this
in a few days time if no-one objects.

Now removed.

Since the affected distributions all have tickets opened for them, I think this ticket can be closed, and will do so in a week unless there is reasonable objection
--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented Apr 3, 2017

From @khwilliamson

Resolved as scheduled
--
Karl Williamson

@p5pRT p5pRT closed this as completed Apr 3, 2017
@p5pRT
Copy link
Author

p5pRT commented Apr 3, 2017

@khwilliamson - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant