Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.dir returns corrupted IO::Paths under concurrent load #5742

Closed
p6rt opened this issue Oct 10, 2016 · 10 comments
Closed

.dir returns corrupted IO::Paths under concurrent load #5742

p6rt opened this issue Oct 10, 2016 · 10 comments

Comments

@p6rt
Copy link

p6rt commented Oct 10, 2016

Migrated from rt.perl.org#129845 (status was 'resolved')

Searchable as RT129845$

@p6rt
Copy link
Author

p6rt commented Oct 10, 2016

From @smls

Calling `.dir` concurrently from multiple threads, results in some of
the IO​::Path objects it returns to be corrupted.

Golfed test script that pretty reliably demonstrates the bug​:
https://gist.github.com/smls/3c59b3f818d27f3854852afd361e9b95

The corruption presents itself in the fact that the .Str method on
such an IO​::Path says it's one file, while its .gist (and other
methods) think it's another file, as exhibited by the following sample
output from the above script​:

  Str​: /tmp/concurrent_dir_bug/sss
  absolute​: /tmp/concurrent_dir_bug/vvv
  gist​: "/tmp/concurrent_dir_bug/vvv".IO
  perl​: "/tmp/concurrent_dir_bug/vvv".IO(​:SPEC(IO​::Spec​::Unix))

The script almost always fails with an output like this (not always
with the same file names of course), but *sometimes* it fails with the
following error instead (where the line number points to the `await`)​:

  Type check failed in binding; expected Str but got Mu (Mu)
  in block <unit> at /home/sam/dev/concurrent_dir_bug.p6 line 13

And *one* time I got the following instead​:

  [1] 24237 segmentation fault (core dumped) ~/dev/test.p6

(It's possible that the latter two failure modes are just general
concurrency bugginess, possibly related to RT# 129781​:
https://rt-archive.perl.org/perl6/Ticket/Display.html?id=129781 )

@p6rt
Copy link
Author

p6rt commented Oct 10, 2016

From @smls

Forgot to state the Rakudo version, sorry​:
 
  Rakudo version 2016.09-163-g92c0921 built on MoarVM version 2016.09-39-g688796b
  implementing Perl 6.c.

Btw, here's a simpler test-case that almost always exhibits the *second* failure mode​:

  ➜ await do for ^20 { start dir("/").eager }
  Type check failed in binding; expected Str but got Mu (Mu)

@p6rt
Copy link
Author

p6rt commented Oct 10, 2016

From @lizmat

This appears to be a MoarVM specific issue​: I can’t get this script to fail on the JVM backend.

The applicable code appears to be MVM_dir_read in src/io/dirops.c . Which appears to take a thread context. On non-Win32 systems, it appears to be using readdir_r() . For which I found a rather recent​:

h2o/h2o#1046
h2o/h2o#1052

So it would appear we need to replace readdir_r by readdir?

On 10 Oct 2016, at 17​:58, Sam S. (via RT) <perl6-bugs-followup@​perl.org> wrote​:

# New Ticket Created by Sam S.
# Please include the string​: [perl #​129845]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=129845 >

Calling `.dir` concurrently from multiple threads, results in some of
the IO​::Path objects it returns to be corrupted.

Golfed test script that pretty reliably demonstrates the bug​:
https://gist.github.com/smls/3c59b3f818d27f3854852afd361e9b95

The corruption presents itself in the fact that the .Str method on
such an IO​::Path says it's one file, while its .gist (and other
methods) think it's another file, as exhibited by the following sample
output from the above script​:

   Str&#8203;: /tmp/concurrent\_dir\_bug/sss

absolute​: /tmp/concurrent_dir_bug/vvv
gist​: "/tmp/concurrent_dir_bug/vvv".IO
perl​: "/tmp/concurrent_dir_bug/vvv".IO(​:SPEC(IO​::Spec​::Unix))

The script almost always fails with an output like this (not always
with the same file names of course), but *sometimes* it fails with the
following error instead (where the line number points to the `await`)​:

Type check failed in binding; expected Str but got Mu (Mu)
in block <unit> at /home/sam/dev/concurrent_dir_bug.p6 line 13

And *one* time I got the following instead​:

[1] 24237 segmentation fault (core dumped) ~/dev/test.p6

(It's possible that the latter two failure modes are just general
concurrency bugginess, possibly related to RT# 129781​:
https://rt-archive.perl.org/perl6/Ticket/Display.html?id=129781 )

@p6rt
Copy link
Author

p6rt commented Oct 10, 2016

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Aug 26, 2017

From @smls

Both the demo script (from the top post) and the one-line test case (2nd post) now pass without error!

Bisectable identifies a commit from last week as the fix​:

  https://gist.github.com/97d96022fdfed416e67ecadf5b2b9353
  rakudo/rakudo@231cb3f

Is there a way to write a spectest for a probabilistic bug like this?

@p6rt
Copy link
Author

p6rt commented Aug 26, 2017

From @AlexDaniel

It is more or less reliable actually. I think writing a stresstest will do.
On 2017-08-26 05​:33​:17, smls75@​gmail.com wrote​:

Both the demo script (from the top post) and the one-line test case
(2nd post) now pass without error!

Bisectable identifies a commit from last week as the fix​:

https://gist.github.com/97d96022fdfed416e67ecadf5b2b9353
rakudo/rakudo@231cb3f

Is there a way to write a spectest for a probabilistic bug like this?

@p6rt
Copy link
Author

p6rt commented Oct 13, 2017

From @zoffixznet

Tests in Raku/roast@bcaaad4d42

@p6rt
Copy link
Author

p6rt commented Oct 13, 2017

From @zoffixznet

On Thu, 12 Oct 2017 17​:17​:54 -0700, cpan@​zoffix.com wrote​:

Tests in Raku/roast@bcaaad4d42

@p6rt
Copy link
Author

p6rt commented Oct 13, 2017

@zoffixznet - Status changed from 'open' to 'resolved'

@p6rt p6rt closed this as completed Oct 13, 2017
@p6rt
Copy link
Author

p6rt commented Oct 13, 2017

From @zoffixznet

On Thu, 12 Oct 2017 17​:17​:54 -0700, cpan@​zoffix.com wrote​:

Tests in Raku/roast@bcaaad4d42

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant