Skip Menu |
Report information
Id: 132849
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: randir <sergey.aleynikov [at] gmail.com>
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: High
Type: library
Perl Version: 5.27.9
Fixed In: (no value)



To: perlbug [...] perl.org
From: Sergey Aleynikov <sergey.aleynikov [...] gmail.com>
Date: Sun, 11 Feb 2018 01:29:14 +0300
Subject: Perl build process leaves random files in system
Download (untitled) / with headers
text/plain 4.5k
This is a bug report for perl from sergey.aleynikov@gmail.com, generated with the help of perlbug 1.41 running under perl 5.27.9. ----------------------------------------------------------------- [Please describe your issue here] Starting with the commit c0e3b4b51c, perl build process now leaves (on some systems) files outside of it's build tree: dorothy at ~/afl-bisect ±(c0e3b4b51c) ❯ ls -al /home/core total 32 drwxrwxrwx 2 root root 24576 Aug 9 2017 . drwxr-xr-x 5 root root 4096 May 19 2017 .. dorothy at ~/afl-bisect ±(c0e3b4b51c) ❯ git clean -dfx && ./Configure -de -Dusedevel && make -j20 test_prep <snip> cd t && (rm -f perl; /bin/ln -s ../perl perl) drwxrwxrwx 2 root root 24576 Feb 11 01:22 . drwxr-xr-x 5 root root 4096 May 19 2017 .. -rw------- 1 afl afl 19984384 Feb 11 01:22 core.40209 -rw------- 1 afl afl 16064512 Feb 11 01:22 core.40211 -rw------- 1 afl afl 15572992 Feb 11 01:22 core.40223 -rw------- 1 afl afl 15327232 Feb 11 01:22 core.40233 -rw------- 1 afl afl 15294464 Feb 11 01:22 core.40247 -rw------- 1 afl afl 15294464 Feb 11 01:22 core.40267 -rw------- 1 afl afl 15708160 Feb 11 01:22 core.40352 -rw------- 1 afl afl 15101952 Feb 11 01:22 core.40392 -rw------- 1 afl afl 15101952 Feb 11 01:22 core.40479 This pollutes users' systems. This leads to things like https://rt.perl.org/Ticket/Display.html?id=132843. This can break in other ways on restricted systems. I do not find this behavior nor desirable nor sane. [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=library severity=high module=Storable --- Site configuration information for perl 5.27.9: Configured by root at Sun Feb 11 01:07:53 MSK 2018. Summary of my perl5 (revision 5 version 27 subversion 9) configuration: Commit id: c0e3b4b51cabf15ed8fc5f564dfeea31c25f5239 Platform: osname=linux osvers=4.9.0-5-amd64 archname=x86_64-linux uname='linux dorothy 4.9.0-5-amd64 #1 smp debian 4.9.65-3+deb9u2 (2018-01-04) x86_64 gnulinux ' config_args='-de -Dusedevel' hint=recommended useposix=true d_sigaction=define useithreads=undef usemultiplicity=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cc' ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2' optimize='-O2' cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='6.3.0 20170516' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.24.so so=so useshrplib=false libperl=libperl.a gnulibc_version='2.24' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong' --- @INC for perl 5.27.9: lib /usr/local/lib/perl5/site_perl/5.27.9/x86_64-linux /usr/local/lib/perl5/site_perl/5.27.9 /usr/local/lib/perl5/5.27.9/x86_64-linux /usr/local/lib/perl5/5.27.9 --- Environment for perl 5.27.9: HOME=/home/afl LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_CTYPE=en_US.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/afl/perlbrew/bin:/home/afl/perlbrew/perls/perl-5.20.2/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games PERLBREW_BASHRC_VERSION=0.78 PERLBREW_HOME=/home/afl/.perlbrew PERLBREW_MANPATH=/home/afl/perlbrew/perls/perl-5.20.2/man PERLBREW_PATH=/home/afl/perlbrew/bin:/home/afl/perlbrew/perls/perl-5.20.2/bin PERLBREW_PERL=perl-5.20.2 PERLBREW_ROOT=/home/afl/perlbrew PERLBREW_VERSION=0.78 PERL_BADLANG (unset) SHELL=/usr/bin/zsh
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.2k
On Sat, 10 Feb 2018 22:29:20 GMT, randir wrote: Show quoted text
> This is a bug report for perl from sergey.aleynikov@gmail.com, > generated with the help of perlbug 1.41 running under perl 5.27.9. > > > ----------------------------------------------------------------- > [Please describe your issue here] > > Starting with the commit c0e3b4b51c, perl build process now leaves (on > some systems) files outside of it's build tree: > > dorothy at ~/afl-bisect ±(c0e3b4b51c) ❯ ls -al /home/core > total 32 > drwxrwxrwx 2 root root 24576 Aug 9 2017 . > drwxr-xr-x 5 root root 4096 May 19 2017 .. > > dorothy at ~/afl-bisect ±(c0e3b4b51c) ❯ git clean -dfx && ./Configure > -de -Dusedevel && make -j20 test_prep > <snip> > cd t && (rm -f perl; /bin/ln -s ../perl perl) > > drwxrwxrwx 2 root root 24576 Feb 11 01:22 . > drwxr-xr-x 5 root root 4096 May 19 2017 .. > -rw------- 1 afl afl 19984384 Feb 11 01:22 core.40209 > -rw------- 1 afl afl 16064512 Feb 11 01:22 core.40211 > -rw------- 1 afl afl 15572992 Feb 11 01:22 core.40223 > -rw------- 1 afl afl 15327232 Feb 11 01:22 core.40233 > -rw------- 1 afl afl 15294464 Feb 11 01:22 core.40247 > -rw------- 1 afl afl 15294464 Feb 11 01:22 core.40267 > -rw------- 1 afl afl 15708160 Feb 11 01:22 core.40352 > -rw------- 1 afl afl 15101952 Feb 11 01:22 core.40392 > -rw------- 1 afl afl 15101952 Feb 11 01:22 core.40479 > > This pollutes users' systems. This leads to things like > https://rt.perl.org/Ticket/Display.html?id=132843. This can break in > other ways on restricted systems. > > I do not find this behavior nor desirable nor sane. >
While I share your concern about failure to clean up files intended to be temporary, I'm having trouble replicating your results. At commit 199fc8cde4ec30a56626c0765b3b0efacb327664 I cleaned my work directory, then called Configure and make test_prep as I ordinarily do. When those completed, I checked the following directories for un-cleaned-up files: / /tmp/ /home/ ~/ ~/tmp/ I didn't find any core.* files in any of those locations. Can you tell me where I should have been looking? Or explain a bit more about the circumstances in which these files might not be cleaned up? Thank you very much. -- James E Keenan (jkeenan@cpan.org)
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.4k
On Sat, 10 Feb 2018 17:07:56 -0800, jkeenan wrote: Show quoted text
> While I share your concern about failure to clean up files intended to > be temporary, I'm having trouble replicating your results.
Show quoted text
> I didn't find any core.* files in any of those locations. Can you > tell me where I should have been looking? Or explain a bit more about > the circumstances in which these files might not be cleaned up?
So, here's a full picture. On linux systems, location for core files is controlled by a system-wide tunable in /proc/sys/kernel/core_pattern. Default value for it varies across different Linux distros. But, since it it's system-wide, it can only be set by root and software can't rely on it's specific value. Then, there's a second layer of defense - to allow programs some control, whether they want to leave core files after them, or not. There is a per-process limit, named 'core limit', which when set to 0 disables generating core files for this process altogether. It can be set, for example, in a shell by executing 'limit -c 0' to suppress core files from all subsequently run commands from this shell (or 'limit -c unlimited' to enable core generation for processes of all sizes). So, to reproduce this, you should: - save somewhere value of /proc/sys/kernel/core_pattern (to restore it later) - set, as root, that tunable to '/home/core/%p.core', where %p means process pid (or anywhere else you'd like it to point for a test) - execute 'ulimit -c unlimited' in a shell where you'd run 'make' - build perl
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.9k
On Sun, 11 Feb 2018 10:11:34 GMT, randir wrote: Show quoted text
> On Sat, 10 Feb 2018 17:07:56 -0800, jkeenan wrote:
> > While I share your concern about failure to clean up files intended > > to > > be temporary, I'm having trouble replicating your results.
>
> > I didn't find any core.* files in any of those locations. Can you > > tell me where I should have been looking? Or explain a bit more > > about > > the circumstances in which these files might not be cleaned up?
> > So, here's a full picture. > > On linux systems, location for core files is controlled by a system- > wide tunable in /proc/sys/kernel/core_pattern. Default value for it > varies across different Linux distros. But, since it it's system-wide, > it can only be set by root and software can't rely on it's specific > value. > > Then, there's a second layer of defense - to allow programs some > control, whether they want to leave core files after them, or not. > There is a per-process limit, named 'core limit', which when set to 0 > disables generating core files for this process altogether. It can be > set, for example, in a shell by executing 'limit -c 0' to suppress > core files from all subsequently run commands from this shell (or > 'limit -c unlimited' to enable core generation for processes of all > sizes). > > So, to reproduce this, you should: > > - save somewhere value of /proc/sys/kernel/core_pattern (to restore it > later) > - set, as root, that tunable to '/home/core/%p.core', where %p means > process pid (or anywhere else you'd like it to point for a test) > - execute 'ulimit -c unlimited' in a shell where you'd run 'make' > - build perl
Thanks for that explanation. I was previously unaware of /proc/sys/kernel/core_pattern. On Ubuntu that pipes a crash report to the 'apport' program described here: https://wiki.ubuntu.com/Apport. Of course, the point you made in your original post is still a concern. I don't know how to address it best. Thank you very much. -- James E Keenan (jkeenan@cpan.org)
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.5k
On Sun, 11 Feb 2018 02:11:34 -0800, randir wrote: Show quoted text
> On Sat, 10 Feb 2018 17:07:56 -0800, jkeenan wrote:
> > While I share your concern about failure to clean up files intended > > to > > be temporary, I'm having trouble replicating your results.
>
> > I didn't find any core.* files in any of those locations. Can you > > tell me where I should have been looking? Or explain a bit more > > about > > the circumstances in which these files might not be cleaned up?
> > So, here's a full picture. > > On linux systems, location for core files is controlled by a system- > wide tunable in /proc/sys/kernel/core_pattern. Default value for it > varies across different Linux distros. But, since it it's system-wide, > it can only be set by root and software can't rely on it's specific > value. > > Then, there's a second layer of defense - to allow programs some > control, whether they want to leave core files after them, or not. > There is a per-process limit, named 'core limit', which when set to 0 > disables generating core files for this process altogether. It can be > set, for example, in a shell by executing 'limit -c 0' to suppress > core files from all subsequently run commands from this shell (or > 'limit -c unlimited' to enable core generation for processes of all > sizes).
In 7ec112d2a5ff7c80b00801e1c94308e3709d68df I've added code to dist/Storable/stacksize to "ulimit -c 0"* before running the stack overflow tests. Please let me know if this fixes the problem for you. If it doesn't I'll disable this probing by default. Thanks, Tony * where available
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 400b
On Sun, 11 Feb 2018 16:50:16 -0800, tonyc wrote: Show quoted text
> Please let me know if this fixes the problem for you. > > If it doesn't I'll disable this probing by default.
Now it leaves no traces on linux. But then I've checked windows x64 and x86 builds using VS 2017. x64 passed flawlessly, but for x86 i've got this: - http://i.imgur.com/9WIXn60.png - (after pressing close) http://i.imgur.com/ehVen2N.png
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 268b
On Mon, 12 Feb 2018 12:27:01 -0800, randir wrote: Show quoted text
> x64 passed flawlessly, but for x86 i've > got this:
I've done this again from a clean checkout and now also compiles. Can it be that 'nmake clean' leaves something behind, as I've built 64-bit first and then 32-bit?
From: Tomasz Konojacki <me [...] xenu.pl>
To: perl5-porters [...] perl.org
Subject: Re: [perl #132849] Perl build process leaves random files in system
Date: Mon, 12 Feb 2018 23:28:19 +0100
Download (untitled) / with headers
text/plain 467b
On Mon, 12 Feb 2018, at 22:14, Sergey Aleynikov via RT wrote: Show quoted text
> On Mon, 12 Feb 2018 12:27:01 -0800, randir wrote:
> > x64 passed flawlessly, but for x86 i've > > got this:
> > I've done this again from a clean checkout and now also compiles. Can it > be that 'nmake clean' leaves something behind, as I've built 64-bit > first and then 32-bit?
My experience says that (n)make clean can't be trusted. git clean -dxf is the most foolproof way to clean source tree.
From: Karl Williamson <public [...] khwilliamson.com>
To: perlbug-followup [...] perl.org
Date: Mon, 12 Feb 2018 15:28:22 -0700
Subject: Re: [perl #132849] Perl build process leaves random files in system
CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1023b
On 02/12/2018 01:27 PM, Sergey Aleynikov via RT wrote: Show quoted text
> On Sun, 11 Feb 2018 16:50:16 -0800, tonyc wrote:
>> Please let me know if this fixes the problem for you. >> >> If it doesn't I'll disable this probing by default.
> > Now it leaves no traces on linux. But then I've checked windows x64 and x86 builds using VS 2017. x64 passed flawlessly, but for x86 i've got this: > > - http://i.imgur.com/9WIXn60.png > - (after pressing close) http://i.imgur.com/ehVen2N.png
I was getting those errors last week on dromedary's win32 (whatever that is, maybe a VM?), and I "solved" it by cloning a new repository. I'm pretty sure it happened before Storable was pushed to blead. So some file, maybe it's just config data, is getting left around by a git clean -dfx that doesn't show up under git status. I looked up error 5 and it was something about no access rights IIRC. I don't know about the high-order 0xC. Show quoted text
> > --- > via perlbug: queue: perl5 status: open > https://rt.perl.org/Ticket/Display.html?id=132849 >
CC: ;, perl5-porters [...] perl.org
From: Tony Cook <tony [...] develop-help.com>
Date: Tue, 13 Feb 2018 10:13:35 +1100
Subject: Re: [perl #132849] Perl build process leaves random files in system
To: Sergey Aleynikov via RT <perlbug-followup [...] perl.org>
Download (untitled) / with headers
text/plain 617b
On Mon, Feb 12, 2018 at 12:27:01PM -0800, Sergey Aleynikov via RT wrote: Show quoted text
> On Sun, 11 Feb 2018 16:50:16 -0800, tonyc wrote:
> > Please let me know if this fixes the problem for you. > > > > If it doesn't I'll disable this probing by default.
> > Now it leaves no traces on linux. But then I've checked windows x64 and x86 builds using VS 2017. x64 passed flawlessly, but for x86 i've got this: > > - http://i.imgur.com/9WIXn60.png > - (after pressing close) http://i.imgur.com/ehVen2N.png
I expected the code I'm using to suppress that error worked on both 32 and 64-bit Windows, running some tests myself. Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.9k
On Mon, 12 Feb 2018 15:13:53 -0800, tonyc wrote: Show quoted text
> On Mon, Feb 12, 2018 at 12:27:01PM -0800, Sergey Aleynikov via RT > wrote:
> > On Sun, 11 Feb 2018 16:50:16 -0800, tonyc wrote:
> > > Please let me know if this fixes the problem for you. > > > > > > If it doesn't I'll disable this probing by default.
> > > > Now it leaves no traces on linux. But then I've checked windows x64 > > and x86 builds using VS 2017. x64 passed flawlessly, but for x86 i've > > got this: > > > > - http://i.imgur.com/9WIXn60.png > > - (after pressing close) http://i.imgur.com/ehVen2N.png
> > I expected the code I'm using to suppress that error worked on both 32 > and 64-bit Windows, running some tests myself.
I'm not seeing failures in the build for x32 on VC 2015, I don't have VC 2017 installed: J:\dev\perl\git\perl\miniperl.exe "-I..\..\lib" -MExtUtils::Command -e c hmod -- 755 ..\..\lib\auto\threads\shared\shared.dll ..\perl.exe -I..\lib -I. ..\dist\Storable\stacksize --core probe for max. stack sizes... 65000 failed, try less 32550 ... 32550 failed, try less 16325 ... 16325 failed, try less 8213 ... 8213 failed, try less 4157 ... 4157 passed, try more 6185 ... 6185 passed, try more 7199 ... 7199 failed, try less 6692 ... 6692 failed, try less 6439 ... 6439 passed, try more 6565 ... 6565 failed, try less 6502 ... 6502 failed, try less 6471 ... 6471 failed, try less 6455 ... 6455 passed, try more 6463 ... 6463 failed, try less 6459 ... 6459 passed, try more 6461 ... 6461 passed, try more 6462 ... 6462 passed, try more 6462 ... MAX_DEPTH = 6462 3000 passed, try more 3000 ... MAX_DEPTH_HASH = 3000 if not exist ..\lib\Storable mkdir ..\lib\Storable copy ..\dist\Storable\lib\Storable\Limit.pm ..\lib\Storable\Limit.pm 1 file(s) copied. So I don't know what's going on. I tested: J:\dev\perl\git\perl\win32>git describe v5.27.8-253-g141513f I ran the build with: nmake CCTYPE=MSVC140 WIN64=undef test-prep Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 311b
On Mon, 12 Feb 2018 15:55:00 -0800, tonyc wrote: Show quoted text
> I'm not seeing failures in the build for x32 on VC 2015, I don't have > VC 2017 installed:
Show quoted text
> nmake CCTYPE=MSVC140 WIN64=undef test-prep
My second try to build 32-bit version also succeeded, so I think it's fine. Seems like I've messed with consequent builds.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 401b
On Mon, 12 Feb 2018 16:04:20 -0800, randir wrote: Show quoted text
> On Mon, 12 Feb 2018 15:55:00 -0800, tonyc wrote:
> > I'm not seeing failures in the build for x32 on VC 2015, I don't have > > VC 2017 installed:
>
> > nmake CCTYPE=MSVC140 WIN64=undef test-prep
> > My second try to build 32-bit version also succeeded, so I think it's > fine. Seems like I've messed with consequent builds.
Thanks, closing. Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 271b
It is true that the perl build does not leave core files anymore, but it still emits "Segmentation fault (core dumped)" message on my environment. https://gist.github.com/skaji/4fb507113bb7a05a9514e323a001b9a3 It would be nice if it does not emit such messages either.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 412b
On Wed, 21 Feb 2018 08:13:29 -0800, skaji@cpan.org wrote: Show quoted text
> It is true that the perl build does not leave core files anymore, > but it still emits "Segmentation fault (core dumped)" message on my > environment. > https://gist.github.com/skaji/4fb507113bb7a05a9514e323a001b9a3 > > It would be nice if it does not emit such messages either.
That should be fixed by 3d79e577a768f4ffbe0d25be5bcf58ae9d112001. Tony
To: perlbug-followup [...] perl.org
Subject: Re: [perl #132849] Perl build process leaves random files in system
From: Shoichi Kaji <shoichikaji [...] gmail.com>
Date: Sun, 06 May 2018 07:12:58 +0000
Download (untitled) / with headers
text/plain 973b
Thanks Tony. On the other hand, I also found that "segfault" messages might be emitted to syslog. Please look at https://gist.github.com/skaji/1d7a5d57da120e8ca6c2807b4e1a0307 Because people often monitor syslog data and alert themselves when a log pattern is detected, I think we should stop these messages if possible (but I don't know how). On Mon, Feb 26, 2018 at 4:11 PM Tony Cook via RT <perlbug-followup@perl.org> wrote: Show quoted text
> On Wed, 21 Feb 2018 08:13:29 -0800, skaji@cpan.org wrote:
> > It is true that the perl build does not leave core files anymore, > > but it still emits "Segmentation fault (core dumped)" message on my > > environment. > > https://gist.github.com/skaji/4fb507113bb7a05a9514e323a001b9a3 > > > > It would be nice if it does not emit such messages either.
Show quoted text
> That should be fixed by 3d79e577a768f4ffbe0d25be5bcf58ae9d112001.
Show quoted text
> Tony
Show quoted text
> --- > via perlbug: queue: perl5 status: pending release > https://rt.perl.org/Ticket/Display.html?id=132849
Download (untitled) / with headers
text/plain 317b
Thank you for filing this report. You have helped make Perl better. With the release yesterday of Perl 5.28.0, this and 185 other issues have been resolved. Perl 5.28.0 may be downloaded via: https://metacpan.org/release/XSAWYERX/perl-5.28.0 If you find that the problem persists, feel free to reopen this ticket.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org