Skip Menu |
Report information
Id: 131915
Status: resolved
Priority: 0/
Queue: perl6

Owner: Nobody
Requestors: scolinet [at] gmail.com
Cc:
AdminCc:

Severity: (no value)
Tag: (no value)
Platform: (no value)
Patch Status: (no value)
VM: (no value)



Subject: [Regression] Proc using more memory since being a Proc::Async
Download (untitled) / with headers
text/plain 543b
Similar to https://rt.perl.org/m/ticket/show?id=131914 it was noticed that there is a huge increase of memory usage after running a simple echo command. Result are in Kbytes. <AlexDaniel> commit: 92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec sub mem { "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024 }; say mem(); run 'echo'; say mem() <committable6test> AlexDaniel, ¦92bd7e4f54a9^: «56444␤␤59820» ¦92bd7e4: «56568␤␤215976» ¦9658dd98c9dd8ec^^: «59212␤␤216660» ¦9658dd9: «139024␤␤305320»
TL;DR it went from 59820 to 215976 after this commit: https://github.com/rakudo/rakudo/commit/92bd7e4f54a92fa660f99b4d056d33a08fb98bd2

Bisect log:
https://gist.github.com/678ce88a13272eab454a22568f3bddc7

On 2017-08-16 14:24:48, scolinet@gmail.com wrote:
Show quoted text
> Similar to https://rt.perl.org/m/ticket/show?id=131914 it was noticed
> that there is a huge increase of memory usage after running a simple
> echo command.
> Result are in Kbytes.
>
> <AlexDaniel> commit:
> 92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec sub mem {
> "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024 }; say
> mem(); run 'echo'; say mem()
> <committable6test> AlexDaniel, ¦92bd7e4f54a9^: «56444␤␤59820»
> ¦92bd7e4: «56568␤␤215976» ¦9658dd98c9dd8ec^^: «59212␤␤216660»
> ¦9658dd9: «139024␤␤305320»


https://irclog.perlgeek.de/perl6-dev/2017-08-17#i_15032963

On 2017-08-16 14:30:42, alex.jakimenko@gmail.com wrote:
Show quoted text
> TL;DR it went from 59820 to 215976 after this commit:
> https://github.com/rakudo/rakudo/commit/92bd7e4f54a92fa660f99b4d056d33a08fb98bd2
>
> Bisect log:
> https://gist.github.com/678ce88a13272eab454a22568f3bddc7
>
> On 2017-08-16 14:24:48, scolinet@gmail.com wrote:
> > Similar to https://rt.perl.org/m/ticket/show?id=131914 it was noticed
> > that there is a huge increase of memory usage after running a simple
> > echo command.
> > Result are in Kbytes.
> >
> > <AlexDaniel> commit:
> > 92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec sub mem
> > {
> > "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024 }; say
> > mem(); run 'echo'; say mem()
> > <committable6test> AlexDaniel, ¦92bd7e4f54a9^: «56444␤␤59820»
> > ¦92bd7e4: «56568␤␤215976» ¦9658dd98c9dd8ec^^: «59212␤␤216660»
> > ¦9658dd9: «139024␤␤305320»


RT-Send-CC: perl6-compiler [...] perl.org
Download (untitled) / with headers
text/plain 1.3k
On Wed, 16 Aug 2017 14:24:48 -0700, scolinet@gmail.com wrote: Show quoted text
> Similar to https://rt.perl.org/m/ticket/show?id=131914 it was noticed > that there is a huge increase of memory usage after running a simple > echo command. > Result are in Kbytes. > > <AlexDaniel> commit: > 92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec sub mem { > "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024 }; say > mem(); run 'echo'; say mem() > <committable6test> AlexDaniel, ¦92bd7e4f54a9^: «56444␤␤59820» > ¦92bd7e4: «56568␤␤215976» ¦9658dd98c9dd8ec^^: «59212␤␤216660» > ¦9658dd9: «139024␤␤305320»
We won't get it back to where it was, because threads cost something. And we'd not really want to go back to where we were, because Proc had all kinds of ways of not working. It's easy to be cheaper if you can be brokener too. The cost is now very significantly lower, however, thanks to: * The new scheduler, which doesn't wastefully start a load of threads for a single Proc::Async usage. This was the major win. * Smarter handling of memory for new threads in MoarVM; now they start with a 128KB nursery and grow it as needed, as opposed to starting with a 4MB nursery. So rather than it causing a 3.8 times growth in program memory size, I now measure it as less than 1.5 (and, better, it won't increase with follow-up calls to run the way it used to either).


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org