Skip Menu |
Report information
Id: 131515
Status: open
Priority: 0/
Queue: perl6

Owner: Nobody
Requestors: cpan [at] zoffix.com
Cc:
AdminCc:

Severity: (no value)
Tag: (no value)
Platform: (no value)
Patch Status: (no value)
VM: (no value)



Subject: [PERF] Performance loss in some commits
Download (untitled) / with headers
text/plain 1.3k
Recently, we have had about ~2.7% overall performance loss, when running stresstest/spectest that a couple of devs noticed. Tux's bench also showed a slowdown. Based on stresstest timings I post ( https://irclog.perlgeek.de/perl6-dev/search/?nick=&q=ZOFVM ), the slowdown happened in some commits made between 2017-05-31 and 2017-06-01. I tried narrowing down what brought the slowdown, but couldn't quite pinpoint it, because there's some randomness in timings of some tests. Here's my results of test times: SHA Stresstest (s) Spectest (s) aebd0fa 123 01d948d 122 2500e50 121 / 122 86 79b8ab9 119 / 118 83 b879060 117 / 119 82 / 82 3f5cc5a 117 3488914 118 fb7dd8a 119 As you can see, the middle commits is where the slowage appears to occur, with 2500e50 being the most likely culprit, even though, the code itself isn't something that would be even remotely hot. I went and rewrote it in NQP (attached) to see if it drops the numbers at all, and... it didn't really. I got a 120s stresstest and a 85s spectest. I gave up with this. I think we need some extensive bench without any random sleeps or delays in it and comprehensive enough to test more than just a few text methods. Perhaps, something that can be cooked up with https://map.perl6.party/ And with such a bench in hand, it might become easier to hunt commits that slow stuff down considerably.
Subject: exception.diff
Download exception.diff
application/octet-stream 5.3k

Message body not shown because it is not plain text.

Subject: Re: [perl #131515] [PERF] Performance loss in some commits
To: "Zoffix Znet (via RT)" <perl6-bugs-followup [...] perl.org>
From: Elizabeth Mattijsen <liz [...] dijkmat.nl>
Date: Mon, 5 Jun 2017 22:27:47 +0200
Show quoted text
> On 5 Jun 2017, at 19:30, Zoffix Znet (via RT) <perl6-bugs-followup@perl.org> wrote: > > # New Ticket Created by Zoffix Znet > # Please include the string: [perl #131515] > # in the subject line of all future correspondence about this issue. > # <URL: https://rt.perl.org/Ticket/Display.html?id=131515 > > > > Recently, we have had about ~2.7% overall performance loss, when running stresstest/spectest that a couple of devs noticed. > Tux's bench also showed a slowdown. > > Based on stresstest timings I post ( https://irclog.perlgeek.de/perl6-dev/search/?nick=&q=ZOFVM ), the slowdown happened in > some commits made between 2017-05-31 and 2017-06-01. I tried narrowing down what brought the slowdown, but couldn't quite > pinpoint it, because there's some randomness in timings of some tests. Here's my results of test times: > > SHA Stresstest (s) Spectest (s) > aebd0fa 123 > 01d948d 122 > 2500e50 121 / 122 86 > 79b8ab9 119 / 118 83 > b879060 117 / 119 82 / 82 > 3f5cc5a 117 > 3488914 118 > fb7dd8a 119 > > As you can see, the middle commits is where the slowage appears to occur, with 2500e50 being the most likely culprit, > even though, the code itself isn't something that would be even remotely hot. > > I went and rewrote it in NQP (attached) to see if it drops the numbers at all, and... it didn't really. > I got a 120s stresstest and a 85s spectest. > > I gave up with this. > > I think we need some extensive bench without any random sleeps or delays in it and comprehensive enough to test more than > just a few text methods. Perhaps, something that can be cooked up with https://map.perl6.party/ > > And with such a bench in hand, it might become easier to hunt commits that slow stuff down considerably.<exception.diff>
Generally, we don’t care about performance once we have an unrecoverable exception that needs to be reported. If the stresstest regression is caused by trying to do Levenstein on method names (which can be a lot on some objects), then maybe the performance of spectest/stresstest can be accounted for by *just* the tests that throw Method Not Found errors?
RT-Send-CC: perl6-compiler [...] perl.org
Download (untitled) / with headers
text/plain 623b
On Mon, 05 Jun 2017 13:28:00 -0700, elizabeth wrote: Show quoted text
> Generally, we don’t care about performance once we have an > unrecoverable exception that needs to be reported. If the stresstest > regression is caused by trying to do Levenstein on method names (which > can be a lot on some objects), then maybe the performance of > spectest/stresstest can be accounted for by *just* the tests that > throw Method Not Found errors?
Yeah, I think the nqp version of the exception is proof that's the wrong commit, otherwise it'd've got faster than even pre-commit. Something else was the cause, but I ran out of steam hunting it.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org