New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory Leak in perl 5.28 and 5.30 #17105
Comments
From a.shikov@dtel-ix.netCreated by a.shikov@dtel-ix.netHello! Recently we migrated some Perl scripts from old to another server. On a new server we've noticed an intensive memory leak. After investigation --------------------------------------------------------------------------- use strict; openlog('syslog-demo', 'ndelay', LOG_LOCAL2); my $c = 0; my @chars = ("A".."Z", "a".."z", "0".."9", "." , "-", "_" ); while ($c++ <= 10000) { Add the line below to syslog.conf to save log to file: Then start script and check out the log file with tail -f. Right after script is started 'top' shows that memory utilization starts to grow rapidly: last pid: 44500; load averages: 1.12, 1.19, 1.22 up 21+05:23:33 15:21:28 PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 45209 root 1 22 0 196M 186M nanslp 1 0:04 4.50% perl 45209 root 1 22 0 266M 258M nanslp 0 0:05 4.64% perl 45209 root 1 22 0 541M 534M nanslp 0 0:11 2.29% perl Tested on physical machine, on virtual machine - the situation is the same on both. The problem is not with Sys::Syslog module, because with Unix::Syslog the problem is the same. After that I tested this script with three major versions of Perl: 5.26.3, 5.28.1, 5.28.2 and 5.30.0 Only with 5.26.3 the script does not eats memory. The rest three versions have this Any suggestions are kindly appreciated. Thanks! Perl Info
|
From @jkeenanOn Tue, 23 Jul 2019 17:22:46 GMT, a.shikov@dtel-ix.net wrote:
1. I'm a bit confused by this report. Near the top you say, "we've found that memory leak is caused by Sys::Syslog". But later on, you say, "The problem is not with Sys::Syslog module, because with Unix::Syslog the problem is the same." Which is it? 2. Have you tried your diagnostic script on FreeBSD-11 with perl-5.26, 5.28 and 5.30? What results do you get there? (We have to try to distinguish between changes in Perl and changes in FreeBSD.) Thank you very much. -- |
The RT System itself - Status changed from 'new' to 'open' |
From @jkeenanOn Tue, 23 Jul 2019 17:22:46 GMT, a.shikov@dtel-ix.net wrote:
I tried that (as well as another spelling including this ticket number) but each time the output was simply appended to /var/log/debug.log. How can I get this to go to a file just for this problem? Thank you very much. -- |
From @rootkwokHello, Interesting case, looks like it is a problem with locale implementation. I Best regards, (via RT) <perlbug-followup@perl.org> � 2019年7�26� �� ��6:50寫��
|
From a.shikov@dtel-ix.netOn Fri, Jul 26, 2019 at 11:50:04 +0800, é�æ¨�è�° wrote:
Hello! Dear Baggio, yes, you're right. With env LC_ALL=C script executes without leak:
O_o
-- |
From a.shikov@dtel-ix.netHello! On Wed, Jul 24, 2019 at 15:07:48 -0700, James E Keenan via RT wrote:
It does not matter. I just described closely our environment but problem can I repeated a test some minutes ago. # ps ax|grep syslog-demo : The results are: <9:46:18> # top -b | grep perl <9:46:19> # top -b | grep perl <9:46:20> # top -b | grep perl <9:46:21> # top -b | grep perl <9:46:22> # top -b | grep perl <9:46:23> # top -b | grep perl Also I tested that on FreeBSD 11.2-RELEASE. On 11.2 the problem is absent. 1. The problem is present in FreeBSD 12.0-RELEASE. Thanks. -- |
From @tonycozOn Tue, 23 Jul 2019 10:22:46 -0700, a.shikov@dtel-ix.net wrote:
This simplifies down to: use POSIX qw(setlocale LC_TIME); while (1) { Unfortunately valgrind[1] says there's no leaks: ==28285== HEAP SUMMARY: I see a similar issue with the following C code: #include <locale.h> int main() { which I believe is handling the locale objects correctly, so I suspect this is a FreeBSD libc issue. Tony [1] from pkg install valgrind |
From a.shikov@dtel-ix.netHello! On Mon, Jul 29, 2019 at 22:35:27 -0700, Tony Cook via RT wrote:
Thank you for pointing this out. And what's strange is that your C code results to same memory leak on both 11.2-RELEASE I've filled the bug in FreeBSD Bugzilla. Let's see the findings there as well. -- |
From @tonycozOn Tue, 30 Jul 2019 21:23:10 -0700, a.shikov@dtel-ix.net wrote:
Yes, this has changed to make locale handling efficiently thread-safe. The C setlocale() changes the locale across all threads, so locks are needed to ensure that one thread doesn't step on the locale usage of another thread. uselocale() etc are thread local, so it's not a problem.
hints/freebsd.sh blocks usage of those APIs in 11.2, see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211743 and https://rt-archive.perl.org/perl5/Ticket/Display.html?id=128867.
Thanks. |
There has been no real action taken by freebsd on https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239520. This isn't a bug in Perl. Is there anything left for us to do? If not, can we close this ticket |
On 3/20/20 11:39 AM, Karl Williamson wrote:
There has been no real action taken by freebsd on
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239520 <url>. This
isn't a bug in Perl. Is there anything left for us to do? If not, can we
close this ticket
Please leave open for a while. I just posted in the FreeBSD ticket and
that stirred a response.
jimk
|
FreeBSD is making a change ... |
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=239520#c11 "This will be fixed in [FreeBSD-]11.4 and 12.2." |
So what does Perl do in response. Re-reading the thread, it seems to me we should set the hints file to not use thread safe locale handling before the fixed versions. Or is thread safety so important that we live with the leak in them. |
I think updating the hints is the correct thing to do. If I understand correctly, without uselocale() etc we just fall back to setlocale() and locking which should be safe if only perl is touching/using the locale in the process. |
On 10/28/20 5:47 PM, Tony Cook wrote:
So what does Perl do in response. Re-reading the thread, it seems to
me we should set the hints file to not use thread safe locale
handling before the fixed versions. Or is thread safety so important
that we live with the leak in them.
I think updating the hints is the correct thing to do.
If I understand correctly, without uselocale() etc we just fall back to
setlocale() and locking which should be safe if only perl is
touching/using the locale in the process.
It's safe as far as races, but setlocale() changes the locale in all
threads simultaneously.
|
@tonycoz @khwilliamson, the FreeBSD Bugzilla ticket tracking this problem has been closed, with the problem reported to be "Fixed in all currently supported releases". Can our ticket now be closed? |
as far as I'm concerned, yes |
Migrated from rt.perl.org#134305 (status was 'open')
Searchable as RT134305$
The text was updated successfully, but these errors were encountered: