Modules broken by hash iterator randomization #12877

p5pRT · 2013-03-23T18:25:37Z

Migrated from rt.perl.org#117313 (status was 'resolved')

Searchable as RT117313$

p5pRT · 2013-03-23T18:25:37Z

From @demerphq

[Separating this from the bug/thread about JSON::XS]

(Andreas J. Koenig) (via RT) <perlbug-followup@perl.org> writes:

# <URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=117239 >
Harden hashes against hash seed discovery by randomizing hash iteration

Other candidates that start failing around the same time and should get
a closer examination:

JEEN/Acme-CPANAuthors-Korean-0.09.tar.gz
SHARYANTO/Data-Schema-0.135.tar.gz
SMIRNIOS/DBD-SQLAnywhere-2.08.tar.gz
TIMB/DBI-1.623.tar.gz
ANDK/Devel-Symdump-2.08.tar.gz
OPI/IO-Handle-Record-0.14.tar.gz
MAKAMAKA/JSON-PPdev-2.27100.tar.gz
JEEN/Lingua-KO-TypoCorrector-0.03.tar.gz
JROBINSON/Locale-Object-0.79.tar.gz
PSCUST/Parse-FSM-1.06.tar.gz
MAGGIEXYZ/PDL-Stats-0.6.2.tar.gz
ADAMK/Perl-Squish-1.06.tar.gz
SATOH/Plack-Middleware-StaticShared-0.05.tar.gz
VOJ/RDF-NS-20130208.tar.gz
MWS/ResourcePool-1.0106.tar.gz
JSIRACUSA/Rose-HTML-Objects-0.617.tar.gz
MKUTTER/SOAP-Lite-0.714.tar.gz
MKUTTER/SOAP-Transport-TCP-0.715.tar.gz
TADAM/Test-Mock-LWP-Dispatch-0.03.tar.gz
MSCHWERN/Time-y2038-20100403.tar.gz
JEEN/WebService-Aladdin-0.0706.tar.gz

I have not had time to verify that they all start failing on the same
commit. At least for DBI I have evidence that it starts failing on
v5.17.9-201-g3078e10. But since randomness is involved this finding may
be biased.

I have not verified others due to lack of time.

--
andreas

--
perl -Mre=debug -e "/just|another|perl|hacker/"

p5pRT · 2013-03-23T19:15:49Z

From @TFoertsch

On 03/23/2013 07:25 PM, yves orton (via RT) wrote:

Other candidates that start failing around the same time and should get
a closer examination:

OPI/IO-Handle-Record-0.14.tar.gz

This one should already be fixed in version 0.15. In a test it compared
Data::Dumper::Dumper(\%result) with Data::Dumper::Dumper(\%expected).

Torsten

p5pRT · 2013-03-23T19:15:50Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2013-03-23T20:24:35Z

From @ilmari

yves orton (via RT) <perlbug-followup@perl.org> writes:

TIMB/DBI-1.623.tar.gz

A fixed DBI-1.624 has been uploaded

ANDK/Devel-Symdump-2.08.tar.gz

Patch in RT: <https://rt.cpan.org/Public/Bug/Display.html?id=84139>

--
"A disappointingly low fraction of the human race is,
at any given time, on fire." - Stig Sandbeck Mathisen

p5pRT · 2013-03-23T20:40:49Z

From @ilmari

yves orton (via RT) <perlbug-followup@perl.org> writes:

SATOH/Plack-Middleware-StaticShared-0.05.tar.gz

This looks like a false positive. It hasn't failed once in over a
hundred runs here, and the only non-PASS on cpantesters is an UNKNOWN
due to a failure to load a module from Makefile.PL (which is doing weird
stuff with author deps in non-author situations).

--
"A disappointingly low fraction of the human race is,
at any given time, on fire." - Stig Sandbeck Mathisen

p5pRT · 2013-03-28T08:35:03Z

From @sisyphus

-----Original Message-----
From: yves orton (via RT)

Other candidates that start failing around the same time and should get
a closer examination:

[snip]

MAGGIEXYZ/PDL-Stats-0.6.2.tar.gz
[snip]

I can't find any FAIL reports for PDL-Stats-0.6.2 at all. (Do you have a
link to one ? I'd be interested to take a look.)

It builds fine for me on 5.17.10 (MS Windows).

Cheers,
Rob

p5pRT · 2013-03-28T09:44:44Z

From @andk

<sisyphus1@optusnet.com.au> writes:

-----Original Message-----
From: yves orton (via RT)

Other candidates that start failing around the same time and should get
a closer examination:

My wording was wrong. Should have said: Other candidates that might have
had to suffer from this change directly or indirectly should get a
closer examination.

[snip]

MAGGIEXYZ/PDL-Stats-0.6.2.tar.gz
[snip]

I can't find any FAIL reports for PDL-Stats-0.6.2 at all. (Do you have
a link to one ? I'd be interested to take a look.)

It turns out that PDL::Stats fails during 'perl Makefile.PL' when PDL is
not installed. Failing in this phase does not lead to cpantesters
reports. Here is the diagnostics:

Warning: prerequisite PDL 2.4.4 not found.
ERROR from evaluation of /tmp/loop_over_bdir-28207-apZvLO/PDL-Stats-0.6.2-GsPLmR/Basic/Makefile.PL: Can't locate PDL/Core/Dev.pm in @INC (you may need to install the PDL::Core::Dev module) (@INC contains: /tmp/CPAN-Reporter-lib-wuU5 /home/sand/src/perl/repoperls/installed-perls/perl/v5.17.10-36-g3a1b438/127e/lib/site_perl/5.17.11/x86_64-linux-ld /home/sand/src/perl/repoperls/installed-perls/perl/v5.17.10-36-g3a1b438/127e/lib/site_perl/5.17.11 /home/sand/src/perl/repoperls/installed-perls/perl/v5.17.10-36-g3a1b438/127e/lib/5.17.11/x86_64-linux-ld /home/sand/src/perl/repoperls/installed-perls/perl/v5.17.10-36-g3a1b438/127e/lib/5.17.11 /tmp/loop_over_bdir-28207-apZvLO/PDL-Stats-0.6.2-GsPLmR .) at ./Makefile.PL line 1.

It builds fine for me on 5.17.10 (MS Windows).

PDL failed for me with v5.17.9-203-ga7b39f8 in t/hdrs.t. Maybe a
randomness effect. Report is here:

http://www.cpantesters.org/cpan/report/92939084-90f0-11e2-9023-fb6e3b384401

HTH,
--
andreas

p5pRT · 2013-03-28T11:23:42Z

From zefram@fysh.org

Andreas Koenig wrote:

PDL failed for me with v5.17.9-203-ga7b39f8 in t/hdrs.t. Maybe a
randomness effect. Report is here:

http://www.cpantesters.org/cpan/report/92939084-90f0-11e2-9023-fb6e3b384401

That report says:

# Failed test 42 in t/niceslice.t at line 211
# t/niceslice.t line 211 is: ok(!$@ and join("",%{$b->gethdr}) eq join("",%{$h}));

That code plainly is dependent on hash ordering, and so needs an update.

-zefram

p5pRT · 2013-03-29T06:23:31Z

From @sisyphus

-----Original Message-----
From: Andreas Koenig

It builds fine for me on 5.17.10 (MS Windows).

PDL failed for me with v5.17.9-203-ga7b39f8 in t/hdrs.t. Maybe a
randomness effect. Report is here:

http://www.cpantesters.org/cpan/report/92939084-90f0-11e2-9023-fb6e3b384401

Oh yes, I get the same failures with PDL on 5.17.10 - but I installed it
anyway.

My hunch is that these failures *are* a result of the randomization of the
hash iterator, but I haven't really looked at it yet.
The t/hdrs.t test script even fails in a random fashion for me - any
combination of tests 3, 5, 6 & 7 can fail (or pass), and results typically
vary from one running of the script to the next.

Thanks for clarifying, Andreas.

Cheers,
Rob

p5pRT · 2013-03-29T10:47:44Z

From devel.chm.01@gmail.com

Fixes for the problem tests have been
pushed to PDL git. Thanks for the pointer,
Andreas.

--Chris

On Fri, Mar 29, 2013 at 2:21 AM, <sisyphus1@optusnet.com.au> wrote:

-----Original Message----- From: Andreas Koenig

It builds fine for me on 5.17.10 (MS Windows).

PDL failed for me with v5.17.9-203-ga7b39f8 in t/hdrs.t. Maybe a
randomness effect. Report is here:

http://www.cpantesters.org/cpan/report/92939084-90f0-11e2-9023-fb6e3b384401

Oh yes, I get the same failures with PDL on 5.17.10 - but I installed it
anyway.

My hunch is that these failures *are* a result of the randomization of the
hash iterator, but I haven't really looked at it yet.
The t/hdrs.t test script even fails in a random fashion for me - any
combination of tests 3, 5, 6 & 7 can fail (or pass), and results typically
vary from one running of the script to the next.

Thanks for clarifying, Andreas.

Cheers,
Rob

p5pRT · 2013-03-29T10:58:27Z

From @andk

The distros below had been nominated by me as maybe-victims of recent
hash randomization commits. I've now visited them all and found them in
this state:

--
andreas

p5pRT · 2013-03-29T12:36:24Z

From @rjbs

* Andreas Koenig <andreas.koenig.7os6VVqR@franz.ak.mind.de> [2013-03-29T06:57:40]

The distros below had been nominated by me as maybe-victims of recent
hash randomization commits. I've now visited them all and found them in
this state:

Thanks very much Andreas, you are a treasure!

--
rjbs

p5pRT · 2013-03-29T15:32:33Z

From @timbunce

On Fri, Mar 29, 2013 at 11:57:40AM +0100, Andreas Koenig wrote:

TIMB/DBI-1.623.tar.gz | fixed in 1.624 |

Yeap, fixed.

Devel::NYTProf also had a bad test. I hope to release v5 soonish.

Tim.

p5pRT · 2013-03-30T01:28:16Z

From @jkeenan

On Fri Mar 29 03:58:27 2013, andreas.koenig.7os6VVqR@franz.ak.mind.de wrote:

The distros below had been nominated by me as maybe-victims of recent
hash randomization commits. I've now visited them all and found them
in
this state:

distro | ticket |
needs patch?

-------------------------------------------------+-------------------+----------------
[snip]

JSIRACUSA/Rose-HTML-Objects-0.617.tar.gz | 84279 |
needs patch

https://rt.cpan.org/Public/Bug/Display.html?id=84279#txn-1196066

Author reports corrected in 0.618, uploaded to CPAN.

p5pRT · 2013-05-20T01:24:33Z

From @rjbs

I believe we no longer need to keep track of these.

--
rjbs

p5pRT · 2013-05-20T01:24:33Z

From [Unknown Contact. See original ticket]

I believe we no longer need to keep track of these.

--
rjbs

p5pRT · 2013-05-20T01:24:34Z

@rjbs - Status changed from 'open' to 'resolved'

p5pRT closed this as completed May 20, 2013

p5pRT added the Severity Low label Oct 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Modules broken by hash iterator randomization #12877

Modules broken by hash iterator randomization #12877

p5pRT commented Mar 23, 2013

p5pRT commented Mar 23, 2013

p5pRT commented Mar 23, 2013

p5pRT commented Mar 23, 2013

p5pRT commented Mar 23, 2013

p5pRT commented Mar 23, 2013

p5pRT commented Mar 28, 2013

p5pRT commented Mar 28, 2013

p5pRT commented Mar 28, 2013

p5pRT commented Mar 29, 2013

p5pRT commented Mar 29, 2013

p5pRT commented Mar 29, 2013

p5pRT commented Mar 29, 2013

p5pRT commented Mar 29, 2013

p5pRT commented Mar 30, 2013

p5pRT commented May 20, 2013

p5pRT commented May 20, 2013

p5pRT commented May 20, 2013

Modules broken by hash iterator randomization #12877

Modules broken by hash iterator randomization #12877

Comments

p5pRT commented Mar 23, 2013

p5pRT commented Mar 23, 2013

From @demerphq

p5pRT commented Mar 23, 2013

From @TFoertsch

p5pRT commented Mar 23, 2013

p5pRT commented Mar 23, 2013

From @ilmari

p5pRT commented Mar 23, 2013

From @ilmari

p5pRT commented Mar 28, 2013

From @sisyphus

p5pRT commented Mar 28, 2013

From @andk

p5pRT commented Mar 28, 2013

From zefram@fysh.org

p5pRT commented Mar 29, 2013

From @sisyphus

p5pRT commented Mar 29, 2013

From devel.chm.01@gmail.com

p5pRT commented Mar 29, 2013

From @andk

p5pRT commented Mar 29, 2013

From @rjbs

p5pRT commented Mar 29, 2013

From @timbunce

p5pRT commented Mar 30, 2013

From @jkeenan

p5pRT commented May 20, 2013

From @rjbs

p5pRT commented May 20, 2013

From [Unknown Contact. See original ticket]

p5pRT commented May 20, 2013