From @atoomic

This needs investigation, unfortunately, I do not have more details than this at this time.
I hope to be able to track it pretty soon.

This is coming from a Travis CI run​: https://travis-ci.org/p5h/perl5demo/jobs/302868505

HEAD is blead + a minor commit which does not seem related to it, and the test is passing on other macosx builds, view https://travis-ci.org/p5h/perl5demo/builds/302868502

I assume that the server was running with a high load, and everything was slow down...

HEAD was here p5h/perl5demo@c4a1841
which is blead=4af7d876a8f36e9eda38116fb0b3d5e3fde025f8 + e1f53cba21c7687c50d535433ea1afc8b78259dd

I suggest to identify the test and skip it when ENV{CI} is set, same as done in cd2a978 ( view PACKAGING for CI option ).

Relevant output from the travis CI log.

____________________________________
# Looks like you failed 1 test of 10.
../dist/Time-HiRes/t/alarm.t .......................................
Dubious, test returned 1 (wstat 256, 0x100)

Test Summary Report

../dist/Time-HiRes/t/alarm.t (Wstat​: 256 Tests​: 10 Failed​: 1)
  Failed test​: 2
  Non-zero exit status​: 1
Files=2627, Tests=1143283, 590 wallclock secs (135.12 usr 16.35 sys + 696.97 cusr 123.73 csys = 972.17 CPU)
Result​: FAIL
make​: *** [test_harness_notty] Error 1

From @jkeenan

On Thu, 16 Nov 2017 20​:14​:32 GMT, atoomic@​cpan.org wrote​:

This needs investigation, unfortunately, I do not have more details
than this at this time.
I hope to be able to track it pretty soon.

This is coming from a Travis CI run​: https://travis-
ci.org/p5h/perl5demo/jobs/302868505

HEAD is blead + a minor commit which does not seem related to it, and
the test is passing on other macosx builds, view https://travis-
ci.org/p5h/perl5demo/builds/302868502

I assume that the server was running with a high load, and everything
was slow down...

HEAD was here
p5h/perl5demo@c4a1841
which is blead=4af7d876a8f36e9eda38116fb0b3d5e3fde025f8 +
e1f53cba21c7687c50d535433ea1afc8b78259dd

I suggest to identify the test and skip it when ENV{CI} is set, same
as done in cd2a978 ( view PACKAGING
for CI option ).

Relevant output from the travis CI log.

____________________________________
# Looks like you failed 1 test of 10.
../dist/Time-HiRes/t/alarm.t .......................................
Dubious, test returned 1 (wstat 256, 0x100)

Test Summary Report
-------------------
../dist/Time-HiRes/t/alarm.t
(Wstat​: 256 Tests​: 10 Failed​: 1)
Failed test​: 2
Non-zero exit status​: 1
Files=2627, Tests=1143283, 590 wallclock secs (135.12 usr 16.35 sys +
696.97 cusr 123.73 csys = 972.17 CPU)
Result​: FAIL
make​: *** [test_harness_notty] Error 1

I don't think this warrants any changes in our source code. dist/Time-HiRes/t/alarm.t is a test that not infrequently fails in resource-constrained environments.

For example, it fails from time to time in the FreeBSD-10.3 VM I maintain on my laptop -- often enough that it's one of four resource-related test failures I cite in a note when I run smoke tests. (See, e.g., http​://perl5.test-smoke.org/report/59803.)

While we may want to have some discussion (on p5p list rather than in RT) of how we handle resource-constrained tests in general, I don't see a need at this time to change this particular test.

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

The RT System itself - Status changed from 'new' to 'open'

From @xsawyerx

On 11/20/2017 03​:56 PM, James E Keenan via RT wrote​:

On Thu, 16 Nov 2017 20​:14​:32 GMT, atoomic@​cpan.org wrote​:

This needs investigation, unfortunately, I do not have more details
than this at this time.
I hope to be able to track it pretty soon.

This is coming from a Travis CI run​: https://travis-
ci.org/p5h/perl5demo/jobs/302868505

HEAD is blead + a minor commit which does not seem related to it, and
the test is passing on other macosx builds, view https://travis-
ci.org/p5h/perl5demo/builds/302868502

I assume that the server was running with a high load, and everything
was slow down...

HEAD was here
p5h/perl5demo@c4a1841
which is blead=4af7d876a8f36e9eda38116fb0b3d5e3fde025f8 +
e1f53cba21c7687c50d535433ea1afc8b78259dd

I suggest to identify the test and skip it when ENV{CI} is set, same
as done in cd2a978 ( view PACKAGING
for CI option ).

Relevant output from the travis CI log.

____________________________________
# Looks like you failed 1 test of 10.
../dist/Time-HiRes/t/alarm.t .......................................
Dubious, test returned 1 (wstat 256, 0x100)

Test Summary Report
-------------------
../dist/Time-HiRes/t/alarm.t
(Wstat​: 256 Tests​: 10 Failed​: 1)
Failed test​: 2
Non-zero exit status​: 1
Files=2627, Tests=1143283, 590 wallclock secs (135.12 usr 16.35 sys +
696.97 cusr 123.73 csys = 972.17 CPU)
Result​: FAIL
make​: *** [test_harness_notty] Error 1
I don't think this warrants any changes in our source code. dist/Time-HiRes/t/alarm.t is a test that not infrequently fails in resource-constrained environments.

The problem with this test failing this way (and particularly flapping
behavior) is that we do not know whether the function is buggy or
whether the system is loaded. Normalizing false positives means ignoring
true positives.

I would be happy to detect and prevent these so we could treat a "fail"
as "no, seriously, this is a bug in the code."

From @toddr

On Mon, 20 Nov 2017 06​:56​:21 -0800, jkeenan wrote​:

While we may want to have some discussion (on p5p list rather than in
RT) of how we handle resource-constrained tests in general, I don't
see a need at this time to change this particular test.

Jim,

Don't ticket comments relay an email to the p5p list and vice-versa?

+1 to adding the other 3 tests to TODO or whichever on a resource constrained system.

Todd