Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proc using more memory since being a Proc::Async #6449

Closed
p6rt opened this issue Aug 16, 2017 · 6 comments
Closed

Proc using more memory since being a Proc::Async #6449

p6rt opened this issue Aug 16, 2017 · 6 comments
Labels
regression Issue did not exist previously

Comments

@p6rt
Copy link

p6rt commented Aug 16, 2017

Migrated from rt.perl.org#131915 (status was 'resolved')

Searchable as RT131915$

@p6rt
Copy link
Author

p6rt commented Aug 16, 2017

From @Skarsnik

Similar to https://rt.perl.org/m/ticket/show?id=131914 it was noticed that there is a huge increase of memory usage after running a simple echo command.
Result are in Kbytes.

<AlexDaniel> commit​: 92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec sub mem { "/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024 }; say mem(); run 'echo'; say mem()
<committable6test> AlexDaniel, ¦92bd7e4f54a9^​: «56444␤␤59820» ¦92bd7e4​: «56568␤␤215976» ¦9658dd98c9dd8ec^^​: «59212␤␤216660» ¦9658dd9​: «139024␤␤305320»

@p6rt
Copy link
Author

p6rt commented Aug 16, 2017

From @AlexDaniel

TL;DR it went from 59820 to 215976 after this commit​: rakudo/rakudo@92bd7e4

Bisect log​:
https://gist.github.com/678ce88a13272eab454a22568f3bddc7

On 2017-08-16 14​:24​:48, scolinet@​gmail.com wrote​:

Similar to https://rt.perl.org/m/ticket/show?id=131914 it was noticed
that there is a huge increase of memory usage after running a simple
echo command.
Result are in Kbytes.

<AlexDaniel> commit​:
92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec sub mem {
"/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024 }; say
mem(); run 'echo'; say mem()
<committable6test> AlexDaniel, ¦92bd7e4f54a9^​: «56444␤␤59820»
¦92bd7e4​: «56568␤␤215976» ¦9658dd98c9dd8ec^^​: «59212␤␤216660»
¦9658dd9​: «139024␤␤305320»

@p6rt
Copy link
Author

p6rt commented Aug 16, 2017

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Aug 17, 2017

From @AlexDaniel

https://irclog.perlgeek.de/perl6-dev/2017-08-17#i_15032963

On 2017-08-16 14​:30​:42, alex.jakimenko@​gmail.com wrote​:

TL;DR it went from 59820 to 215976 after this commit​:
rakudo/rakudo@92bd7e4

Bisect log​:
https://gist.github.com/678ce88a13272eab454a22568f3bddc7

On 2017-08-16 14​:24​:48, scolinet@​gmail.com wrote​:

Similar to https://rt.perl.org/m/ticket/show?id=131914 it was noticed
that there is a huge increase of memory usage after running a simple
echo command.
Result are in Kbytes.

<AlexDaniel> commit​:
92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec sub mem
{
"/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024 }; say
mem(); run 'echo'; say mem()
<committable6test> AlexDaniel, ¦92bd7e4f54a9^​: «56444␤␤59820»
¦92bd7e4​: «56568␤␤215976» ¦9658dd98c9dd8ec^^​: «59212␤␤216660»
¦9658dd9​: «139024␤␤305320»

@p6rt
Copy link
Author

p6rt commented Oct 5, 2017

From @jnthn

On Wed, 16 Aug 2017 14​:24​:48 -0700, scolinet@​gmail.com wrote​:

Similar to https://rt.perl.org/m/ticket/show?id=131914 it was noticed
that there is a huge increase of memory usage after running a simple
echo command.
Result are in Kbytes.

<AlexDaniel> commit​:
92bd7e4f54a9^,92bd7e4f54a9,9658dd98c9dd8ec^^,9658dd98c9dd8ec sub mem {
"/proc/$*PID/statm".IO.lines[0].split(/\s/)[5] * 4096 / 1024 }; say
mem(); run 'echo'; say mem()
<committable6test> AlexDaniel, ¦92bd7e4f54a9^​: «56444␤␤59820»
¦92bd7e4​: «56568␤␤215976» ¦9658dd98c9dd8ec^^​: «59212␤␤216660»
¦9658dd9​: «139024␤␤305320»

We won't get it back to where it was, because threads cost something. And we'd not really want to go back to where we were, because Proc had all kinds of ways of not working. It's easy to be cheaper if you can be brokener too.

The cost is now very significantly lower, however, thanks to​:

* The new scheduler, which doesn't wastefully start a load of threads for a single Proc​::Async usage. This was the major win.
* Smarter handling of memory for new threads in MoarVM; now they start with a 128KB nursery and grow it as needed, as opposed to starting with a 4MB nursery.

So rather than it causing a 3.8 times growth in program memory size, I now measure it as less than 1.5 (and, better, it won't increase with follow-up calls to run the way it used to either).

@p6rt
Copy link
Author

p6rt commented Oct 5, 2017

@jnthn - Status changed from 'open' to 'resolved'

@p6rt p6rt closed this as completed Oct 5, 2017
@p6rt p6rt added the regression Issue did not exist previously label Jan 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
regression Issue did not exist previously
Projects
None yet
Development

No branches or pull requests

1 participant