Skip Menu |
Report information
Id: 132292
Status: open
Priority: 0/
Queue: perl6

Owner: Nobody
Requestors: alex.jakimenko [at] gmail.com
Cc:
AdminCc:

Severity: (no value)
Tag: (no value)
Platform: (no value)
Patch Status: (no value)
VM: (no value)



Subject: [REGRESSION] Recursively .emit-ing from the tap of the same supply bails out
Download (untitled) / with headers
text/plain 622b
Code: my $s1 = Supplier.new; $s1.Supply.tap: { say $_; $s1.emit(2) if $++ < 5; say "here" }; $s1.emit(1) ¦2017.06: 1 2 2 2 2 2 here here here here here here ¦HEAD(012c80f): 1 here 2 Possible IRC discussion: https://irclog.perlgeek.de/perl6-dev/2017-09-21#i_15197905 The behavior change two times, first it started hanging after (2017-09-18) https://github.com/rakudo/rakudo/commit/4a8038c2956e863bc661a2a00e8371eb98002608 And then the hang was resolved (incorrectly?) in (2017-09-22) https://github.com/rakudo/rakudo/commit/547839200a772e26ea164e9d1fd8c9cd4a5c2d9f I think the output on 2017.06 makes more sense.
RT-Send-CC: perl6-compiler [...] perl.org
On Fri, 13 Oct 2017 20:43:42 -0700, alex.jakimenko@gmail.com wrote: Show quoted text
> Code: > my $s1 = Supplier.new; $s1.Supply.tap: { say $_; $s1.emit(2) if $++ < > 5; say "here" }; $s1.emit(1) > > > ¦2017.06: > 1 > 2 > 2 > 2 > 2 > 2 > here > here > here > here > here > here > > ¦HEAD(012c80f): > 1 > here > 2 > > > Possible IRC discussion: https://irclog.perlgeek.de/perl6-dev/2017-09- > 21#i_15197905 > > The behavior change two times, first it started hanging after > (2017-09-18) > https://github.com/rakudo/rakudo/commit/4a8038c2956e863bc661a2a00e8371eb98002608 > And then the hang was resolved (incorrectly?) in > (2017-09-22) > https://github.com/rakudo/rakudo/commit/547839200a772e26ea164e9d1fd8c9cd4a5c2d9f > > > I think the output on 2017.06 makes more sense.
Actually the 2017.06 behavior was clearly wrong, because it violates the principle that a Supply chain will process a message at a time. That every "here" comes out at the end illustrates that the tap block was reentered. That was not an intended behavior, but rather an accident resulting through use of a reentrant mutex for some (not all) Supply concurrency management. The commits in question and those around them introduced a unified concurrency model for all Supply operations, including `supply` blocks, based around Lock::Async. The changes fixed many other issues, but also forced a revisit of the question of recursion - effectively, a supply sending a message to itself. This is a tricky problem, because there's some competing design goals around supplies: 1. Serial message processing (as mentioned above) 2. Back-pressure: those who emit into a Supply chain pay the cost of the message processing. 3. Fairness: Messages are processed in the order they arrive. 4. No concurrency unless requested I think 1 is pretty non-negotiable, because it's hard to write reliable concurrent code if you don't know what your "transaction scope" is. I really should have noticed the issue with reentrant mutexes sooner, though I guess that's my own usage biases to blame: I very rarely use `.tap` and instead use supply/react/whenever where this issue could never happen (but - big issue - the old supply block mechanism violated goal 2 and arguably 3). So the interesting question is how many we can have out of 2, 3, and 4. The current solution, on recursion, is to schedule the recursive message handling using the current $*SCHEDULER. This makes sure that we get 1 and 3. Unfortunately, it violates 2 (if we expect it to be transitively applied) and 4, and it's 4 that results in the effects reported here. Note that if it's rewritten as: my $s1 = Supplier.new; react { whenever $s1 { say $_; $++ < 5 ?? $s1.emit(2) !! $s1.done; say "here" }; $s1.emit(1) } Then it works fine: 1 here 2 here 2 here 2 here 2 here 2 here Also if you sleep a bit after the original: my $s1 = Supplier.new; $s1.Supply.tap: { say $_; $s1.emit(2) if $++ < 5; say "here" }; $s1.emit(1); sleep 1 Then the output is the same as the above also (of course, the react block way is the correct one, not sleeping!) So the question is if we can find a way to have 2 and 4, while retaining 1 and 3, and at what cost.
Download (untitled) / with headers
text/plain 851b
On Mon, 16 Oct 2017 07:42:06 -0700, jnthn@jnthn.net wrote: Show quoted text
> So the question is if we can find a way to have 2 and 4, while > retaining 1 and 3, and at what cost.
Also noting that in order to preserve 3, then in a situation like: * "Thread" 1 sends message A * Handler for message A starts running * "Thread" 2 sends message B * Handler for message A emits a recursive message C * Handler for message A completes Then at this point, the next thing that needs to happen is for message B to be processed, for fairness. This means that "Thread" 1 needs to (non-blockingly, if in the pool) wait until message B has been processed, before it can do message C. If we can arrange for that to happen then I guess things would work out OK enough: the sender of A will have to wait a while, but the recursive message send of C was "its fault" for sending A.
Right. Then I guess it's not a regression. Tag removed.

On 2017-10-16 07:42:06, jnthn@jnthn.net wrote:
Show quoted text
> On Fri, 13 Oct 2017 20:43:42 -0700, alex.jakimenko@gmail.com wrote:
> > Code:
> > my $s1 = Supplier.new; $s1.Supply.tap: { say $_; $s1.emit(2) if $++ <
> > 5; say "here" }; $s1.emit(1)
> >
> >
> > ¦2017.06:
> > 1
> > 2
> > 2
> > 2
> > 2
> > 2
> > here
> > here
> > here
> > here
> > here
> > here
> >
> > ¦HEAD(012c80f):
> > 1
> > here
> > 2
> >
> >
> > Possible IRC discussion: https://irclog.perlgeek.de/perl6-dev/2017-
> > 09-
> > 21#i_15197905
> >
> > The behavior change two times, first it started hanging after
> > (2017-09-18)
> > https://github.com/rakudo/rakudo/commit/4a8038c2956e863bc661a2a00e8371eb98002608
> > And then the hang was resolved (incorrectly?) in
> > (2017-09-22)
> > https://github.com/rakudo/rakudo/commit/547839200a772e26ea164e9d1fd8c9cd4a5c2d9f
> >
> >
> > I think the output on 2017.06 makes more sense.
>
> Actually the 2017.06 behavior was clearly wrong, because it violates
> the principle that a Supply chain will process a message at a time.
> That every "here" comes out at the end illustrates that the tap block
> was reentered. That was not an intended behavior, but rather an
> accident resulting through use of a reentrant mutex for some (not all)
> Supply concurrency management.
>
> The commits in question and those around them introduced a unified
> concurrency model for all Supply operations, including `supply`
> blocks, based around Lock::Async. The changes fixed many other issues,
> but also forced a revisit of the question of recursion - effectively,
> a supply sending a message to itself. This is a tricky problem,
> because there's some competing design goals around supplies:
>
> 1. Serial message processing (as mentioned above)
> 2. Back-pressure: those who emit into a Supply chain pay the cost of
> the message processing.
> 3. Fairness: Messages are processed in the order they arrive.
> 4. No concurrency unless requested
>
> I think 1 is pretty non-negotiable, because it's hard to write
> reliable concurrent code if you don't know what your "transaction
> scope" is. I really should have noticed the issue with reentrant
> mutexes sooner, though I guess that's my own usage biases to blame: I
> very rarely use `.tap` and instead use supply/react/whenever where
> this issue could never happen (but - big issue - the old supply block
> mechanism violated goal 2 and arguably 3).
>
> So the interesting question is how many we can have out of 2, 3, and
> 4. The current solution, on recursion, is to schedule the recursive
> message handling using the current $*SCHEDULER. This makes sure that
> we get 1 and 3. Unfortunately, it violates 2 (if we expect it to be
> transitively applied) and 4, and it's 4 that results in the effects
> reported here.
>
> Note that if it's rewritten as:
>
> my $s1 = Supplier.new; react { whenever $s1 { say $_; $++ < 5 ??
> $s1.emit(2) !! $s1.done; say "here" }; $s1.emit(1) }
>
> Then it works fine:
>
> 1
> here
> 2
> here
> 2
> here
> 2
> here
> 2
> here
> 2
> here
>
> Also if you sleep a bit after the original:
>
> my $s1 = Supplier.new; $s1.Supply.tap: { say $_; $s1.emit(2) if $++ <
> 5; say "here" }; $s1.emit(1); sleep 1
>
> Then the output is the same as the above also (of course, the react
> block way is the correct one, not sleeping!)
>
> So the question is if we can find a way to have 2 and 4, while
> retaining 1 and 3, and at what cost.




This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org