New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Using Proc::Async with tap leads to memory leak #5974
Comments
From @ronaldxsThe example below is believed to be a simplification of memory leak The program just echoes numbered lines through the UNIX 'cat' program This issue occurs with Perl 6 doc generation of pod code blocks. At my $proc = Proc::Async.new('/usr/bin/env', 'cat', :r, :w); my $last; say $i if $i %% 1_000; my $p = Promise.new; my $tap = $proc-supply.tap( -> $cat-echo { $proc.say('line ' ~ $i); # $tmp_fname); say $last;
Links: [1] Raku/doc#1104 |
From @zoffixznetOn Tue, 03 Jan 2017 10:33:31 -0800, ronaldxs wrote:
Confirmed on Rakudo version 2016.12-52-g9eed276 built on MoarVM version 2016.12-6-g65acd55 on running on Linux. Memory keeps slowly increasing; got up to 1.1GB by 48000 iterations. |
The RT System itself - Status changed from 'new' to 'open' |
From @jnthnOn Tue, 03 Jan 2017 12:36:10 -0800, cpan@zoffix.com wrote:
Worth retrying on HEAD, which has a number of fixes and improvements resulting from analyzing this issue. /jnthn |
From @ronaldxs
Both code sample and htmlify.p6 still leak for me. Sorry. ron@ron-laptop:~/.rakudobrew/moar-nom$ perl6 -v |
From @ronaldxsJust to try to quantify for future reference or anyone interested ... my $proc-prom = $proc.start; my $last; my $p = Promise.new; my $tap = $proc-supply.tap( -> $cat-echo { $proc.say('line ' ~ $i); # $tmp_fname); say $last; /*** prints ***/ pmap after 0 iterations |
From @jnthnOn Tue, 10 Jan 2017 14:30:27 -0800, ronaldxs wrote:
Yes, I wasn't entirely clear - there was more than one issue, and so fixing the first couple of issues only improved things rather than fully resolved them. I just bumped MOAR_REVISION again to get another set of fixes, which almost completely address the leakage. There is a tiny leak remaining that I'm aware of (about 8 bytes per write), so I'll leave this open for now, but by my measurements it seems to converge around 1000 iterations and leak very little beyond then. (This kind of initial growth and convergence is normal behavior.) /jnthn |
From @ronaldxsHi. I now have 2 ubuntu/Rakudo computers, a desktop and an old laptop, and after upgrading both to a current Rakudo, Rakudo version 2017.01-52-g42a1eff built on MoarVM, I see differing results. On the older laptop that showed the problem before I see fairly identical behavior to before the fix. Memory goes from 100k to 1.3Gig at 10_000 iterations and when I step to 18_000 iterations I get 2.3Gig. On the new ubuntu/rakudo running on an i7 under a Windows 10 hypervisor I get the behavior you describe of going to 1Gig before 2000 iterations and then staying stable up through 18_000 iterations. So AFAIK both machines are running rakudo from today / this afternoon ET, and both are running ubuntu 14.04. The computer that gets the results you describe is an intel i7 with 16Gig Ram running Windows 10 and running ubuntu under a hypervisor. The hypervisor has some say in memory management is configured for dynamic memory allocation. The computer that still seems to show a memory leak is a dual pentium laptop with 2Gig of Ram. Both systems have swap. "make html" still runs out of memory in the doc repository on the notebook. If you want me to run any test or diagnostics on either computer please let me know. |
From @ronaldxsOne other difference between the laptop and desktop computers that had differing memory leak behavior is that the laptop was running 32 bit Ubuntu and the desktop 64 bit Ubuntu. I retested after noticing the following release note for 2017-02: Fixed overflow during full GC collection on 32-bit systems The fix made me aware of the possibility that 32 bit and 64 bit systems might have different memory management issues. Running with rakudo 2017-02 (slightly different versions) 64 bit shows the behavior I described on my last note of stabilizing at 1 Gig. With 2017-02 32 bit now stabilizes around 460k. I sort of understand that doubling the size of a memory word may double the amount of memory but wonder if there may not be some further room for optimization on 64 bit given 32 bit can get by with half the memory. There is still some slow leakage of a few (~3%) over ten thousand iterations. Also htmlify.p6 can now run on my 2Gig laptop without running out of memory. At this point this leak seams to be at least lower priority and possibly fixed. |
From @ronaldxsjnthn mentioned on irc awareness of at least one more leak (presumably) related to this ticket and so ticket waits on news of fixing further leak(s). |
From @dogbert17On Mon, 20 Feb 2017 06:39:32 -0800, ronaldxs wrote:
Running the example above one a 32-bit Linux VM I get: This Rakudo version is 2018.03.136.g.768.cf.9.f.29 built on MoarVM version 2018.03.58.g.0.aaea.3.c.4.f, pmap after 0 iterations So it looks as if things have improved considerably |
Migrated from rt.perl.org#130494 (status was 'open')
Searchable as RT130494$
The text was updated successfully, but these errors were encountered: