Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.unique on Supply after .grep with IO object causes high cpu usage and breaking of supply chain #4538

Closed
p6rt opened this issue Sep 16, 2015 · 7 comments

Comments

@p6rt
Copy link

p6rt commented Sep 16, 2015

Migrated from rt.perl.org#126073 (status was 'resolved')

Searchable as RT126073$

@p6rt
Copy link
Author

p6rt commented Sep 16, 2015

From muraiki@pair.com

If I create a Supply chain where I .unique after .grep(*.IO.e), I end up
with very high CPU usage (180%) within a second or two. I also never end
up seeing the result of the tap even when I've created the right
conditions to trigger it.

  IO​::Notification.watch-path('.')\
  .map(*.path)\
  .grep(* ~~ /test_file\.txt/)\
  .grep(*.IO.e)\
  .unique(​:expires(1))\
  .tap​: -> $foo { say $foo };

  sleep(5);

Also, the program doesn't actually terminate after 5 seconds, despite
the sleep(5).

If I uncomment the unique or the grep(*.IO.e), it works without a
problem. I admit that I might be doing something wrong. Am I correct in
understanding that the .grep(*.IO.e) line should convert the path string
to an IO object, test for existence, and if it does exist it should then
emit the path string?

I feel like I had done something similar pre-GLR and had unexpected
results, but not high cpu usage. I'm not saying that the GLR is the
source of this, but rather am trying to convey a timeframe for when the
problem might have started to happen.

@p6rt
Copy link
Author

p6rt commented Sep 17, 2015

From @lizmat

On 16 Sep 2015, at 22​:01, Erik Ferguson (via RT) <perl6-bugs-followup@​perl.org> wrote​:

# New Ticket Created by Erik Ferguson
# Please include the string​: [perl #​126073]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=126073 >

If I create a Supply chain where I .unique after .grep(*.IO.e), I end up
with very high CPU usage (180%) within a second or two. I also never end
up seeing the result of the tap even when I've created the right
conditions to trigger it.

IO​::Notification.watch-path('.')\
.map(*.path)\
.grep(* ~~ /test_file\.txt/)\
.grep(*.IO.e)\
.unique(​:expires(1))\
.tap​: -> $foo { say $foo };

sleep(5);

Also, the program doesn't actually terminate after 5 seconds, despite
the sleep(5).

If I uncomment the unique or the grep(*.IO.e), it works without a
problem. I admit that I might be doing something wrong. Am I correct in
understanding that the .grep(*.IO.e) line should convert the path string
to an IO object, test for existence, and if it does exist it should then
emit the path string?

Yes, it should. Looks like it gets into an infinite loop there. Checking now.

I feel like I had done something similar pre-GLR and had unexpected
results, but not high cpu usage. I'm not saying that the GLR is the
source of this, but rather am trying to convey a timeframe for when the
problem might have started to happen.

The high CPU usage is because it’s eating memory at an alarming rate, at about 200MB / second in my machine.

Liz

@p6rt
Copy link
Author

p6rt commented Sep 17, 2015

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Sep 18, 2015

From @lizmat

Fixed with d25a07a09d4ad4fa8d9e , test added with 3165b8c85b30c9710e2, can be closed

On 16 Sep 2015, at 22​:01, Erik Ferguson (via RT) <perl6-bugs-followup@​perl.org> wrote​:

# New Ticket Created by Erik Ferguson
# Please include the string​: [perl #​126073]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=126073 >

If I create a Supply chain where I .unique after .grep(*.IO.e), I end up
with very high CPU usage (180%) within a second or two. I also never end
up seeing the result of the tap even when I've created the right
conditions to trigger it.

IO​::Notification.watch-path('.')\
.map(*.path)\
.grep(* ~~ /test_file\.txt/)\
.grep(*.IO.e)\
.unique(​:expires(1))\
.tap​: -> $foo { say $foo };

sleep(5);

Also, the program doesn't actually terminate after 5 seconds, despite
the sleep(5).

If I uncomment the unique or the grep(*.IO.e), it works without a
problem. I admit that I might be doing something wrong. Am I correct in
understanding that the .grep(*.IO.e) line should convert the path string
to an IO object, test for existence, and if it does exist it should then
emit the path string?

I feel like I had done something similar pre-GLR and had unexpected
results, but not high cpu usage. I'm not saying that the GLR is the
source of this, but rather am trying to convey a timeframe for when the
problem might have started to happen.

@p6rt
Copy link
Author

p6rt commented Sep 18, 2015

From @usev6

On Fri Sep 18 01​:57​:50 2015, elizabeth wrote​:

Fixed with d25a07a09d4ad4fa8d9e , test added with 3165b8c85b30c9710e2,
can be closed

Great! I'm closing this ticket as 'resolved'.

@p6rt p6rt closed this as completed Sep 18, 2015
@p6rt
Copy link
Author

p6rt commented Sep 18, 2015

@usev6 - Status changed from 'open' to 'resolved'

@p6rt
Copy link
Author

p6rt commented Sep 19, 2015

From muraiki@pair.com

Great, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant