Skip Menu |
Report information
Id: 131940
Status: open
Priority: 0/
Queue: perl6

Owner: Nobody
Requestors: shoichikaji [at] gmail.com
Cc:
AdminCc:

Severity: (no value)
Tag: (no value)
Platform: (no value)
Patch Status: (no value)
VM: (no value)



To: rakudobug [...] perl.org
Date: Mon, 21 Aug 2017 20:38:42 +0900
Subject: Proc::Async.bind-stdin leaves a unix socket open
From: Shoichi Kaji <shoichikaji [...] gmail.com>
Download (untitled) / with headers
text/plain 1.3k
(Please note that the following is available at gist too. https://gist.github.com/skaji/b0b210304e932445890f2a28cf7f53b0 ) I found that Proc::Async.bind-stdin left a unix socket open. Should I call some methods on Proc::Async if I use bind-stdin? ## How to reproduce this issue The following script demonstrates this issue.https://gist.github.com/skaji/b0b210304e932445890f2a28cf7f53b0#file-bind-stdin-test-p6 --- ❯ perl6 bind-stdin-test.p6 20 >/dev/null run 20 tests, leaves 152 files open ❯ perl6 bind-stdin-test.p6 21 >/dev/null run 21 tests, leaves 153 files open ❯ perl6 bind-stdin-test.p6 22 >/dev/null run 22 tests, leaves 154 files open ❯ perl6 bind-stdin-test.p6 23 >/dev/null run 23 tests, leaves 155 files open --- This shows everytime we call bind-stdin method, the number of open files increases. The output of lsof -p PID in macos is below: --- ... moar 44821 skaji 139u unix 0xf2addbda53e4c6b7 0t0 ->(none) moar 44821 skaji 140u unix 0xf2addbda53e4c077 0t0 ->(none) moar 44821 skaji 141u unix 0xf2addbda53e4c90f 0t0 ->0xf2addbda53e4c77f --- See https://gist.github.com/skaji/b0b210304e932445890f2a28cf7f53b0#file-zz-result-on-macos-txt ## My environment * This is Rakudo version 2017.07-202-ge3e29c5 built on MoarVM version 2017.07-444-g88c6851 * Both macOS Sierra (10.12.6) and Ubuntu 16.04 64bit (linux 4.4.0)
Download (untitled) / with headers
text/plain 1.9k
On Mon, 21 Aug 2017 04:38:48 -0700, shoichikaji@gmail.com wrote: Show quoted text
> This shows everytime we call bind-stdin method, the number of open > files increases.
I've been able to confirm that the particular file handles being leaked are the parent process's write uv_pipes towards the bound stdin. Obviously the parent process does not need these filehandles anymore after building a pipeline between children, but it ends up keeping them anyway. I don't see anything else in the Proc::Async API that would help to close these descriptors. The .close-stdin method demands that the (incompatible with .bind-stdin) :w option has been supplied. Internally, handles are fed to a uv_process_t. This libuv object has a uv_close, but not for particular filehandles... and I haven't used enough libuv to know if that also kills the process. How this object actually stores these handles is opaque, so we have no guarantee that we can reach in and grab the handles and close them without causing re-use/async/signal issues. Note that if we are (unlike in the example) interested in the output of the whole pipeline, we would need to avoid also closing the stdout of the final stage, and depending on libuv implementation this might be made tricky. The Proc::Async class and its associated MoarVM backend are pretty convoluted code with lots of promises being handed about.,, as any async abstraction implementation is want to be. Also it is not that easy to work on from outside the core tree given highly restrictive encapsulation. So far I have not seen any indication of anything that is intended to clean up these particular filehandles. It might be as simple as: - @blockers.push($!stdin-fd.then({ $!stdin-fd := .result })); + @blockers.push($!stdin-fd.then({ $!stdin-fd := .result; $!ready_promise := $!ready_promise.then({ # find and close the C filehandle and alter the libuv object so it knows it was closed }) ...but that supposes a lot of things are safe to do which probably are not safe to do.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org