Skip Menu |
Report information
Id: 132358
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: shay <steve.m.hay [at] googlemail.com>
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: High
Type: core
Perl Version: 5.27.6
Fixed In: (no value)



From: Steve Hay <steve.m.hay [...] googlemail.com>
Subject: Implicit filehandle close not working when STDERR closed
Date: Wed, 25 Oct 2017 14:08:21 +0100
To: perlbug [...] perl.org
Download (untitled) / with headers
text/plain 4.6k
This is a bug report for perl from steve.m.hay@googlemail.com, generated with the help of perlbug 1.40 running under perl 5.27.6. ----------------------------------------------------------------- [Please describe your issue here] I have recently noticed temporary files being left behind by ExtUtils::CBuilder's test suite, which I traced to the $SCRIPT filehandle in ExtUtils::CBuilder::Platform::Windows::MSVC::write_compiler_script(). It should be automatically closed when that function exits, but it isn't - causing the attempted later deletion of it to fail because of a familiar Windows problem of not being able to delete an open file. Furthermore, I found that the problem weirdly only occurs with the changes made by https://perl5.git.perl.org/perl.git/commit/581ae9ba65 in place. The closure of STDERR is the key thing in that change, and the problem is easily reproduced as follows: use strict; use warnings; my $file = 'test.txt'; close STDERR; { my $fh; if (open $fh, '>', $file) { print "Opened $file. Press <ENTER> to close and unlink it...\n"; <STDIN>; } else { print "Open $file failed: $!"; exit; } } if (unlink $file) { print "Unlinked $file\n"; } else { print "Unlink $file failed: $!\n"; } On Windows, the $file does not get deleted at the end and the output is: Opened test.txt. Press <ENTER> to close and unlink it... Unlink test.txt failed: Permission denied (Unfortunately, you probably won't see the problem as easily as that on other OSes since they mostly don't have an issue deleting an open file. But presumably the problem of $fh still being open should still be evident by other means of inspection.) If I simply comment out the "close STDERR;" line then the $file does get deleted and the output is as expected: Opened test.txt. Press <ENTER> to close and unlink it... Unlinked test.txt [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=high --- Site configuration information for perl 5.27.6: Configured by shay at Wed Oct 25 13:40:39 2017. Summary of my perl5 (revision 5 version 27 subversion 6) configuration: Commit id: 0cdc775ef423ad6415e6f80b9244c17a52bf5149 Platform: osname=MSWin32 osvers=6.3 archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended useposix=true d_sigaction=undef useithreads=define usemultiplicity=define use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cl' ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32 -D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS' optimize='-O1 -MD -Zi -DNDEBUG -GL' cppflags='-DWIN32' ccversion='16.00.40219.01' gccversion='' gccosandvers='' intsize=4 longsize=4 ptrsize=4 doublesize=8 byteorder=1234 doublekind=3 d_longlong=undef longlongsize=8 d_longdbl=define longdblsize=8 longdblkind=0 ivtype='long' ivsize=4 nvtype='double' nvsize=8 Off_t='__int64' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='link' ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86' libpth="C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\\lib" libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib libc=msvcrt.lib so=dll useshrplib=true libperl=perl527.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs dlext=dll d_dlsymun=undef ccdlflags=' ' cccdlflags=' ' lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86' --- @INC for perl 5.27.6: C:/perl/site/lib C:/perl/lib --- Environment for perl 5.27.6: HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=C:\Windows\system32;C:\Windows;C:\Windows\system32\wbem;C:\perl\bin PERL_BADLANG (unset) SHELL (unset)
Download test.pl
text/x-perl 356b

Message body is not shown because sender requested not to inline it.

To: perl5-porters [...] perl.org
From: Zefram <zefram [...] fysh.org>
Subject: Re: [perl #132358] Implicit filehandle close not working when STDERR closed
Date: Wed, 25 Oct 2017 15:25:21 +0100
Download (untitled) / with headers
text/plain 540b
Steve Hay wrote: Show quoted text
>The closure of STDERR is the key thing in that change,
Since you didn't open a new standard error stream deliberately, the next thing you open gets that file descriptor, becoming the process's new standard error stream (even though not mapped as STDERR). perl quite sensibly declines to implicitly close this. Rather than just close STDERR, you should instead reopen it pointing to something innocuous. Or if you really want STDERR closed, you should at least point the file descriptor to something innocuous. -zefram
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 667b
On Wed, 25 Oct 2017 07:25:36 -0700, zefram@fysh.org wrote: Show quoted text
> Steve Hay wrote:
> >The closure of STDERR is the key thing in that change,
> > Since you didn't open a new standard error stream deliberately, the next > thing you open gets that file descriptor, becoming the process's new > standard error stream (even though not mapped as STDERR). perl quite > sensibly declines to implicitly close this. Rather than just close > STDERR, you should instead reopen it pointing to something innocuous. > Or if you really want STDERR closed, you should at least point the file > descriptor to something innocuous.
Or just close $fh explicitly. -- Father Chrysostomos
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 729b
On Wed, 25 Oct 2017 07:25:36 -0700, zefram@fysh.org wrote: Show quoted text
> Steve Hay wrote:
> >The closure of STDERR is the key thing in that change,
> > Since you didn't open a new standard error stream deliberately, the next > thing you open gets that file descriptor, becoming the process's new > standard error stream (even though not mapped as STDERR). perl quite > sensibly declines to implicitly close this. Rather than just close > STDERR, you should instead reopen it pointing to something innocuous. > Or if you really want STDERR closed, you should at least point the file > descriptor to something innocuous. > > -zefram
Thanks. I should have thought of that. Now fixed in https://perl5.git.perl.org/perl.git/commit/b2b30beea3
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.4k
On Wed, 25 Oct 2017 08:45:45 -0700, sprout wrote: Show quoted text
> On Wed, 25 Oct 2017 07:25:36 -0700, zefram@fysh.org wrote:
> > Steve Hay wrote:
> > >The closure of STDERR is the key thing in that change,
> > > > Since you didn't open a new standard error stream deliberately, the next > > thing you open gets that file descriptor, becoming the process's new > > standard error stream (even though not mapped as STDERR). perl quite > > sensibly declines to implicitly close this. Rather than just close > > STDERR, you should instead reopen it pointing to something innocuous. > > Or if you really want STDERR closed, you should at least point the file > > descriptor to something innocuous.
> > Or just close $fh explicitly. >
Yes, that also works, but it's a shame that code such as that in write_compiler_script() can be affected from a distance like this. I've never considered it before, but clearly you can't rely on implicit close always working: due to possible interference from afar you must always explicitly close() if you really need the file to be closed. I'm wondering if a warning somewhere in the system might have helped here? Either when opening the file and finding that the stderr file descriptor has been used for it, or when refraining from doing the implicit close on that stream? Neither thing was what was expected/wanted in this case, but warnings would be annoying in other cases where these actions were intended.
Date: Wed, 25 Oct 2017 10:54:27 -0700
CC: Perl5 Porteros <perl5-porters [...] perl.org>
To: Max Maischein <perlbug-followup [...] perl.org>
Subject: Re: [perl #132358] Implicit filehandle close not working when STDERR closed
From: Chad Granum <exodist7 [...] gmail.com>
Download (untitled) / with headers
text/plain 2.2k
Would be nice to have some control over file descriptors used in open.  For instance I have code that intentionally closes STDOUT and STDERR and opens fh's with the intention of getting the file descriptors 1 and 2. I have to make sure to close/open in the correct order and rely on the side-effect that they get the recently closed numbers 1 and 2. Would be nice to be able to tell open that I want to get a specific descriptor and have it fail if it is already used.

Would also be nice to be able to tell open not to use any "special" descriptors like 0 1 and 2.

-Chad

On Wed, Oct 25, 2017 at 9:59 AM, Steve Hay via RT <perlbug-followup@perl.org> wrote:
Show quoted text
On Wed, 25 Oct 2017 08:45:45 -0700, sprout wrote:
> On Wed, 25 Oct 2017 07:25:36 -0700, zefram@fysh.org wrote:
> > Steve Hay wrote:
> > >The closure of STDERR is the key thing in that change,
> >
> > Since you didn't open a new standard error stream deliberately, the next
> > thing you open gets that file descriptor, becoming the process's new
> > standard error stream (even though not mapped as STDERR).  perl quite
> > sensibly declines to implicitly close this.  Rather than just close
> > STDERR, you should instead reopen it pointing to something innocuous.
> > Or if you really want STDERR closed, you should at least point the file
> > descriptor to something innocuous.
>
> Or just close $fh explicitly.
>

Yes, that also works, but it's a shame that code such as that in write_compiler_script() can be affected from a distance like this. I've never considered it before, but clearly you can't rely on implicit close always working: due to possible interference from afar you must always explicitly close() if you really need the file to be closed.

I'm wondering if a warning somewhere in the system might have helped here? Either when opening the file and finding that the stderr file descriptor has been used for it, or when refraining from doing the implicit close on that stream? Neither thing was what was expected/wanted in this case, but warnings would be annoying in other cases where these actions were intended.

---
via perlbug:  queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=132358

From: Leon Timmermans <fawaka [...] gmail.com>
Subject: Re: [perl #132358] Implicit filehandle close not working when STDERR closed
Date: Wed, 25 Oct 2017 20:50:09 +0200
CC: Max Maischein <perlbug-followup [...] perl.org>, Perl5 Porteros <perl5-porters [...] perl.org>
To: Chad Granum <exodist7 [...] gmail.com>
Download (untitled) / with headers
text/plain 751b
On Wed, Oct 25, 2017 at 7:54 PM, Chad Granum <exodist7@gmail.com> wrote:
Show quoted text
Would be nice to have some control over file descriptors used in open.  For instance I have code that intentionally closes STDOUT and STDERR and opens fh's with the intention of getting the file descriptors 1 and 2. I have to make sure to close/open in the correct order and rely on the side-effect that they get the recently closed numbers 1 and 2. Would be nice to be able to tell open that I want to get a specific descriptor and have it fail if it is already used.

Would also be nice to be able to tell open not to use any "special" descriptors like 0 1 and 2.

perl -e 'open STDOUT, ">", "foo.txt" or die "Couldnt open: $!"; warn fileno STDOUT'

Leon


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org