Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proc with .in.close and .out.slurp-rest in different threads, hangs #5749

Closed
p6rt opened this issue Oct 14, 2016 · 9 comments
Closed

Proc with .in.close and .out.slurp-rest in different threads, hangs #5749

p6rt opened this issue Oct 14, 2016 · 9 comments

Comments

@p6rt
Copy link

p6rt commented Oct 14, 2016

Migrated from rt.perl.org#129882 (status was 'resolved')

Searchable as RT129882$

@p6rt
Copy link
Author

p6rt commented Oct 14, 2016

From @smls

The following code attempts to pipe a large Blob (larger than a pipe's
buffer size) through an external command.
However, it hangs on the `.in.close` line and never finishes​:

  my $proc = run «cat -», :in, :out, :bin;
  my $input = ("a" x 1000000).encode;
  my $promise = start {
  $proc.in.write​: $input;
  $proc.in.close;
  }
  say $proc.out.slurp-rest(​:close, :bin).bytes;
  await $promise;

More verbose version of the code with debug output showing where it blocks​:
https://gist.github.com/smls/75518c746ca65b3287482c491971eac0

IRC comments​:

  <timotimo> moar sometimes gets confused, i think
  <timotimo> or maybe it's libuv

  <RabidGravy> shouldn't that give an error as other handles across threads do?

  <gfldex> you get the same behaviour with
  https://rt.perl.org/Public/Bug/Display.html?id=129787

Note that without the `start` (i.e. when writing to `.in` and then
reading from `.out` in the same thread), it already hangs on the
`.in.write`, as geekosaur explains​:

  <geekosaur> it will block if the process is not reading, yes. pipes have
  finite buffer space
  <geekosaur> whether this block is happening at OS level or if libuv is
  supposed to be using non-blocking operations and return a "would
  block, try again" and some higher level is blocking based on
  that, I cannot say

@p6rt
Copy link
Author

p6rt commented Oct 19, 2016

From @jnthn

On Fri Oct 14 16​:40​:27 2016, smls75@​gmail.com wrote​:

The following code attempts to pipe a large Blob (larger than a pipe's
buffer size) through an external command.
However, it hangs on the `.in.close` line and never finishes​:

my $proc = run «cat -», :in, :out, :bin;
my $input = ("a" x 1000000).encode;
my $promise = start {
$proc.in.write​: $input;
$proc.in.close;
}
say $proc.out.slurp-rest(​:close, :bin).bytes;
await $promise;

For now, if using threads, stick with Proc​::Async (in the future, we'll probably re-implement Proc/run etc. in terms of Proc​::Async).

/jnthn

@p6rt
Copy link
Author

p6rt commented Oct 19, 2016

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Jul 9, 2017

From @dogbert17

On Wed, 19 Oct 2016 05​:57​:04 -0700, jnthn@​jnthn.net wrote​:

On Fri Oct 14 16​:40​:27 2016, smls75@​gmail.com wrote​:

The following code attempts to pipe a large Blob (larger than a
pipe's
buffer size) through an external command.
However, it hangs on the `.in.close` line and never finishes​:

my $proc = run «cat -», :in, :out, :bin;
my $input = ("a" x 1000000).encode;
my $promise = start {
$proc.in.write​: $input;
$proc.in.close;
}
say $proc.out.slurp-rest(​:close, :bin).bytes;
await $promise;

For now, if using threads, stick with Proc​::Async (in the future,
we'll probably re-implement Proc/run etc. in terms of Proc​::Async).

/jnthn

jnthn recently made the above mentioned re-implementation so the code shown in the report now works.

Closing issue

@p6rt
Copy link
Author

p6rt commented Jul 9, 2017

@dogbert17 - Status changed from 'open' to 'resolved'

@p6rt
Copy link
Author

p6rt commented Jul 10, 2017

From @AlexDaniel

That's pretty good. But issues are not closed without tests, unless there's a good reason not to add a test. No reason was mentioned, therefore reopening this with “testneeded” tag.

On 2017-07-09 15​:49​:46, jan-olof.hendig@​bredband.net wrote​:

On Wed, 19 Oct 2016 05​:57​:04 -0700, jnthn@​jnthn.net wrote​:

On Fri Oct 14 16​:40​:27 2016, smls75@​gmail.com wrote​:

The following code attempts to pipe a large Blob (larger than a
pipe's
buffer size) through an external command.
However, it hangs on the `.in.close` line and never finishes​:

my $proc = run «cat -», :in, :out, :bin;
my $input = ("a" x 1000000).encode;
my $promise = start {
$proc.in.write​: $input;
$proc.in.close;
}
say $proc.out.slurp-rest(​:close, :bin).bytes;
await $promise;

For now, if using threads, stick with Proc​::Async (in the future,
we'll probably re-implement Proc/run etc. in terms of Proc​::Async).

/jnthn

jnthn recently made the above mentioned re-implementation so the code
shown in the report now works.

Closing issue

@p6rt
Copy link
Author

p6rt commented Jul 10, 2017

@AlexDaniel - Status changed from 'resolved' to 'open'

@p6rt
Copy link
Author

p6rt commented Feb 2, 2018

From @AlexDaniel

Test in Raku/roast@195227f

On 2017-07-09 19​:40​:57, alex.jakimenko@​gmail.com wrote​:

That's pretty good. But issues are not closed without tests, unless
there's a
good reason not to add a test. No reason was mentioned, therefore
reopening
this with “testneeded” tag.

On 2017-07-09 15​:49​:46, jan-olof.hendig@​bredband.net wrote​:

On Wed, 19 Oct 2016 05​:57​:04 -0700, jnthn@​jnthn.net wrote​:

On Fri Oct 14 16​:40​:27 2016, smls75@​gmail.com wrote​:

The following code attempts to pipe a large Blob (larger than a
pipe's
buffer size) through an external command.
However, it hangs on the `.in.close` line and never finishes​:

my $proc = run «cat -», :in, :out, :bin;
my $input = ("a" x 1000000).encode;
my $promise = start {
$proc.in.write​: $input;
$proc.in.close;
}
say $proc.out.slurp-rest(​:close, :bin).bytes;
await $promise;

For now, if using threads, stick with Proc​::Async (in the future,
we'll probably re-implement Proc/run etc. in terms of Proc​::Async).

/jnthn

jnthn recently made the above mentioned re-implementation so the code
shown in the report now works.

Closing issue

@p6rt
Copy link
Author

p6rt commented Feb 2, 2018

@AlexDaniel - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant