Skip Menu |
Report information
Id: 132104
Status: open
Priority: 0/
Queue: perl6

Owner: Nobody
Requestors: alex.jakimenko [at] gmail.com
Cc:
AdminCc:

Severity: (no value)
Tag: (no value)
Platform: (no value)
Patch Status: (no value)
VM: JVM



Subject: [JVM] EvalServer seems to leak memory
Download (untitled) / with headers
text/plain 188b
From this discussion https://irclog.perlgeek.de/perl6-dev/2017-09-16#i_15171820 4) the EvalServer seems to leak memory. it's no longer possible to run 'make spectest', even with -Xmx6000m
RT-Send-CC: perl6-compiler [...] perl.org
Download (untitled) / with headers
text/plain 1008b
On Sat, 16 Sep 2017 07:31:18 -0700, alex.jakimenko@gmail.com wrote: Show quoted text
> From this discussion https://irclog.perlgeek.de/perl6-dev/2017-09- > 16#i_15171820 > > 4) the EvalServer seems to leak memory. it's no longer possible to run > 'make spectest', even with -Xmx6000m
My findings so far: 1) The EvalServer does not leak memory per se. Assuming a clean EvalServer instance has been started with './perl6-eval-server -cookie TESTTOKEN -app ./perl6.jar' the following does not leak memory: $ echo 'say 42;' > foo.p6 $ for i in {1..500}; do ./eval-client.pl TESTTOKEN run ./foo.p6; done 2) Using 'run' (or calling Proc::Async directly) does not leak memory: $ ./perl6-j -e 'for ^5000 { run("echo", "42") }' 3) Feeding the EvalServer with a program that calls 'run' (or Proc::Async directly) does heavily leak memory, threads and what not (again assuming a clean EvalServer instance started as above): $ echo 'run("echo", "42");' > foo.p6 $ for i in {1..500}; do ./eval-client.pl TESTTOKEN run foo.p6; done
Download (untitled) / with headers
text/plain 894b
On Sat, 23 Sep 2017 12:28:53 -0700, bartolin@gmx.de wrote: Show quoted text
> 3) Feeding the EvalServer with a program that calls 'run' (or > Proc::Async directly) does heavily leak memory, threads and what not > (again assuming a clean EvalServer instance started as above): > > $ echo 'run("echo", "42");' > foo.p6 > $ for i in {1..500}; do ./eval-client.pl TESTTOKEN run foo.p6; done
I'm still trying to find the cause of the memory leak. Part of the problem seems to be that with each invokation of './eval-client.pl TESTTOKEN run foo.p6' three or four additional threads are started. Those threads are never stopped. Some debug statements in src/core/ThreadPoolScheduler.pm indicate that with each invokation (s.a.) a new AffinityWorker instance and a new GeneralWorker instance are created (and at least one other thread). I guess those newly started Threads stay around until the EvalServer is killed.
RT-Send-CC: perl6-compiler [...] perl.org
Download (untitled) / with headers
text/plain 1.1k
On Thu, 12 Oct 2017 13:35:27 -0700, bartolin@gmx.de wrote: Show quoted text
> On Sat, 23 Sep 2017 12:28:53 -0700, bartolin@gmx.de wrote:
> > 3) Feeding the EvalServer with a program that calls 'run' (or > > Proc::Async directly) does heavily leak memory, threads and what not > > (again assuming a clean EvalServer instance started as above): > > > > $ echo 'run("echo", "42");' > foo.p6 > > $ for i in {1..500}; do ./eval-client.pl TESTTOKEN run foo.p6; done
> > I'm still trying to find the cause of the memory leak. Part of the > problem seems to be that with each invokation of './eval-client.pl > TESTTOKEN run foo.p6' three or four additional threads are started. > Those threads are never stopped. > > Some debug statements in src/core/ThreadPoolScheduler.pm indicate that > with each invokation (s.a.) a new AffinityWorker instance and a new > GeneralWorker instance are created (and at least one other thread). I > guess those newly started Threads stay around until the EvalServer is > killed.
I've pushed some improvements for this issue in NQP commit b88de49aad. As the commit message notes, there's probably more out there, but this gets us through spectest on hack.p6c.org at least, with a max RES of around 4.5gb.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org