Win32: -e '"' always returns true #12431

p5pRT · 2012-09-21T04:59:47Z

Migrated from rt.perl.org#114986 (status was 'open')

Searchable as RT114986$

p5pRT · 2012-09-21T04:59:47Z

From @sisyphus

Subject: Win32: -e '"' always returns true
Message-Id: <5.16.0_1816_1348202385@desktop2>
Reply-To: sisyphus1@optusnet.com.au
To: perlbug@perl.org
From: sisyphus1@optusnet.com.au

This is a bug report for perl from sisyphus1@optusnet.com.au,
generated with the help of perlbug 1.39 running under perl 5.16.0.

Hi,

This (raised recently at http://www.perlmonks.org/index.pl?node_id=979494)
must surely have come up before ?

It's behaviour that's specific to MS Windows - for all versions of perl back
to 5.6.2 (and presumably even further back).

###############################

use warnings;
use strict;

print "1: huh?\n" if -e '"';
print "2: huh?\n" if -e '""';
print "3: huh?\n" if -e '"""';
print "4: huh?\n" if -e '""""';
print "5: huh?\n" if -e '"""""';
print "6: huh?\n" if -e '""""""""""""""';
print "7: huh?\n" if defined -f '"';
print "8: huh?\n" if defined -f '""';
print "9: huh?\n" if defined -f '"""';
print "10: huh?\n" if defined -f '""""';
print "11: huh?\n" if defined -f '"""""';
print "12: huh?\n" if defined -f '""""""""""""""';

__END__
Outputs:

1: huh?
2: huh?
3: huh?
4: huh?
5: huh?
6: huh?
7: huh?
8: huh?
9: huh?
10: huh?
11: huh?
12: huh?

###############################

That's pretty astounding at first glance - given that the doublequote symbol
is an illegal filename character.

Apparently this behaviour results from a Windows system practice of treating
the doublequote symbol as meaning 'everything', and some people might hold
the view that the output of the above demo is correct behaviour.
As I understand it, -e '"' will currently return true on Windows because
it's effectively asking "does anything exist in the cwd?" ... and something
always exists in the cwd (even if it's just the '.' and '..' directories).

For me, this behaviour just doesn't DWIM at all, and it also doesn't match
the -X documentation, imo.

Dunno ... I'll leave it up to p5p to decide how this should be handled.

My view is that if the behaviour is to be kept as is, then we should at
least display a notice about this in the -X docs. Something like:

NOTE: On MS windows, if the specified filename consists solely of one
or more doublequote symbols (") then -e will return true, and
-f will return false but defined.

And perhaps a similar notice for 'stat' in the perlfunc and perlport docs.
Note that stat('"') returns the same as stat('.').

Cheers,
Rob

Flags:
category=core
severity=medium

Site configuration information for perl 5.16.0:

Configured by Rob at Mon May 28 14:31:11 2012.

Summary of my perl5 (revision 5 version 16 subversion 0) configuration:

Platform:
osname=MSWin32, osvers=6.0, archname=MSWin32-x86-multi-thread
uname=''
config_args='undef'
hint=recommended, useposix=true, d_sigaction=undef
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=undef, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='gcc', ccflags
=' -s -O2 -DWIN32 -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS
-fno-strict-aliasing -mms-bitfields',
optimize='-s -O2',
cppflags='-DWIN32'
ccversion='', gccversion='4.5.2', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='long long',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='g++', ldflags ='-s -L"c:\MinGW\perl516\lib\CORE" -L"C:\MinGW\lib"'
libpth=C:\MinGW\lib C:\MinGW\msys\1.0\local\lib
C:\MinGW\msys\1.0\local\ssl\lib
libs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32
-lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion
-lodbc32 -lodbccp32 -lcomctl32
perllibs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32
-lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion
-lodbc32 -lodbccp32 -lcomctl32
libc=, so=dll, useshrplib=true, libperl=libperl516.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ',
lddlflags='-mdll -s -L"c:\MinGW\perl516\lib\CORE" -L"C:\MinGW\lib"'

Locally applied patches:

@INC for perl 5.16.0:
C:\_32\mpfr_dec64_516\lib\perl5\MSWin32-x86-multi-thread
C:/MinGW/perl516/site/lib
C:/MinGW/perl516/lib
.

Environment for perl 5.16.0:
HOME (unset)
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=C:\Program Files\Windows
NT\Accessories;C:\MinGW\bin;C:\_32\dmake;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\batch;C:\_32\lzma;C:\MinGW\perl516\bin;C:\MinGW\msys\1.0\bin;C:\_32\Gnuplot\binary;C:\sisyphusion\libgcc_straw32
PERL5LIB=;C:\_32\mpfr_dec64_516\lib\perl5\MSWin32-x86-multi-thread
PERL_BADLANG (unset)
SHELL (unset)

p5pRT · 2012-09-21T20:04:16Z

From @ikegami

It performs other shell interpolation too.

dir /b foo bar
foo

set XX=foo

perl -E"say -e qq{%XX%} ?1:0"
1

set XX=bar

perl -E"say -e qq{%XX%} ?1:0"
0

p5pRT · 2012-09-21T20:04:17Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2012-09-21T20:05:50Z

From @ikegami

On Fri, Sep 21, 2012 at 4:03 PM, Eric Brine <ikegami@adaelis.com> wrote:

It performs other shell interpolation too.

dir /b foo bar
foo

set XX=foo

perl -E"say -e qq{%XX%} ?1:0"
1

set XX=bar

perl -E"say -e qq{%XX%} ?1:0"
0

Nevermind. Bad test.

p5pRT · 2012-09-24T06:36:57Z

From vadim.konovalov@alcatel-lucent.com

Apparently this behaviour results from a Windows system
practice of treating
the doublequote symbol as meaning 'everything', and some
people might hold
the view that the output of the above demo is correct behaviour.
As I understand it, -e '"' will currently return true on
Windows because
it's effectively asking "does anything exist in the cwd?" ...
and something
always exists in the cwd (even if it's just the '.' and '..'
directories).

As for my knowledge, doublequote does not mean anything,

My view is that if the behaviour is to be kept as is, then we
should at
least display a notice about this in the -X docs. Something like:

IMO -f '"' should be undef

NOTE: On MS windows, if the specified filename consists solely of one
or more doublequote symbols (") then -e will return true, and
-f will return false but defined.

it appears that doublequote is prohibited in filenames on Windows, similarily
to ':', '/', '*' etc.
Also, spaces (' ') are stripped at the end and start of file name.

So - from one side, -f should return false in all such cases, because a file
with such a name does not exist.
Trying to still find a file with same name but doublequotes stripped is wrong, IMO.

Another question - is it OS or filesystem?

p5pRT · 2012-09-24T07:48:24Z

From @sisyphus

----- Original Message -----
From: "Konovalov, Vadim (Vadim)** CTR **"
<vadim.konovalov@alcatel-lucent.com>
To: <perl5-porters@perl.org>; <bugs-bitbucket@rt.perl.org>
Sent: Monday, September 24, 2012 4:35 PM
Subject: RE: [perl #114986] Win32: -e '"' always returns true

Apparently this behaviour results from a Windows system
practice of treating
the doublequote symbol as meaning 'everything', and some
people might hold
the view that the output of the above demo is correct behaviour.
As I understand it, -e '"' will currently return true on
Windows because
it's effectively asking "does anything exist in the cwd?" ...
and something
always exists in the cwd (even if it's just the '.' and '..'
directories).

As for my knowledge, doublequote does not mean anything,

My view is that if the behaviour is to be kept as is, then we
should at
least display a notice about this in the -X docs. Something like:

IMO -f '"' should be undef

And the way I read it, that's what the documentation implies.
Interestingly, both -T '"' and -B '"' return undef, it's just -f '"' that
returns a false but defined value.

AFAIK, -e and -f are the only switches that contravene the -X documentation.

NOTE: On MS windows, if the specified filename consists solely of one
or more doublequote symbols (") then -e will return true, and
-f will return false but defined.

it appears that doublequote is prohibited in filenames on Windows,
similarily
to ':', '/', '*' etc.

Yes, the doublequote is prohibited.

Also, spaces (' ') are stripped at the end and start of file name.

So - from one side, -f should return false in all such cases, because a
file
with such a name does not exist.
Trying to still find a file with same name but doublequotes stripped is
wrong, IMO.

Another question - is it OS or filesystem?

I think it's OS .... but I don't really know for sure.
How does one test this ?

Cheers,
Rob

p5pRT · 2012-09-24T17:18:27Z

From @bulk88

From: sisyphus1@optusnet.com.au
To: vadim.konovalov@alcatel-lucent.com; perl5-porters@perl.org; bugs-bitbucket@rt.perl.org
Subject: Re: [perl #114986] Win32: -e '"' always returns true
Date: Mon, 24 Sep 2012 17:47:02 +1000

Another question - is it OS or filesystem?

I think it's OS .... but I don't really know for sure.
How does one test this ?

Cheers,
Rob

Use a C debugger. Win32 Perl has alot of file path manipulation code in perlhost.h, win32io.c and win32.c to enable POSIX paths or unclean (directory name without or without a trailing "\") to work on short file name DOS Windows and other little bugs and strange things around the Win32 API. PerlDir_mapA is a common function that is called. Rather than take a "read your OS's CLib manpage" approach the way perl does on Unix (I think thats the usually explanation Perl people give for "this works on Linux but not Solaris/AIX/OSX"), Win32 Perl tries to create a POSIX compatibility layer over Win32 so alot of Unix Perl scripts will work unmodified on Win32. Plus remember the all file IO goes through the MS CRT which itself has some "posix compatibility layer" stuff going on. So there are 3 main suspects and 2 unlikely suspects for what causes q!"! to be a valid file name, Perl's /win32 directory in src code, the MS CRT, or the Kernel32 API, and as a semi-jk let us throw in NTDLL/Native API's path processing, or the NTFS or FAT32 FS driver's path processing too.

p5pRT · 2012-09-25T18:37:47Z

From @jandubois

On Mon, 24 Sep 2012, bulk 88 wrote:

PerlDir_mapA is a common function that is called.

I'm pretty sure that the PerlDir_mapA() processing is the underlying
cause. Perl on Windows uses the perlhost mechanism to provide a
virtual environment for each Perl interpreter inside the same process.
So each interpreter will pretend to have their own environment variables
*and* their own working directory.

To maintain this illusion Perl cannot rely on the system CWD to resolve
relative paths but must turn them into absolute paths relative to the CWD
of the current interpreter itself. This happens in PerlDir_mapA. So I
suspect the '"' characters will simply be ignored and the stat() call will
be made on the CWD instead.

Cheers,
-Jan

toddr · 2020-02-13T15:40:37Z

I understand the underlying problem hasn't been resolved, but I don't see anything we can do. The last comment from @jandubois seems to indicate this is not going to be fixed, right?

richardleach · 2020-02-13T17:06:36Z

Possibly something that could be looked at as part of any future revamp of filename handling on Windows to fix #9578 and related "Unicode and System Calls" limitations?

As discussed on the mailing list here: https://www.nntp.perl.org/group/perl.perl5.porters/2020/10/msg258453.html This just removes the declaration that we support the very old versions of Windows that have long since been EOLed. For reference of problems related to maintaining the EOLed versions: Perl#4145 Perl#6080 Perl#7410 Perl#8502 Perl#9025 Perl#12431 Perl#14687

tonycoz · 2020-12-03T03:41:10Z

This is fixed in blead, probably by e935ef3.

I expect it was also fixed in builds with recent MSVC, since the UCRT uses CreateFile() to open the file to stat rather than FindFirstFile() as MSVCRT does.

p5pRT added Severity Medium distro-mswin32 type-OS-interaction type-portability labels Oct 19, 2019

toddr added bulk88-query Closable? We might be able to close this ticket, but we need to check with the reporter labels Feb 13, 2020

toddr mentioned this issue Feb 13, 2020

-e tests not reliable under Win32 #7410

Open

genio mentioned this issue Oct 15, 2020

Remove EOL Windows Versions From Supported List. #18242

Closed

tonycoz closed this as completed Dec 3, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Win32: -e '"' always returns true #12431

Win32: -e '"' always returns true #12431

p5pRT commented Sep 21, 2012

p5pRT commented Sep 21, 2012

p5pRT commented Sep 21, 2012

p5pRT commented Sep 21, 2012

p5pRT commented Sep 21, 2012

p5pRT commented Sep 24, 2012

p5pRT commented Sep 24, 2012

p5pRT commented Sep 24, 2012

p5pRT commented Sep 25, 2012

toddr commented Feb 13, 2020

richardleach commented Feb 13, 2020

tonycoz commented Dec 3, 2020

Win32: -e '"' always returns true #12431

Win32: -e '"' always returns true #12431

Comments

p5pRT commented Sep 21, 2012

p5pRT commented Sep 21, 2012

From @sisyphus

p5pRT commented Sep 21, 2012

From @ikegami

p5pRT commented Sep 21, 2012

p5pRT commented Sep 21, 2012

From @ikegami

p5pRT commented Sep 24, 2012

From vadim.konovalov@alcatel-lucent.com

p5pRT commented Sep 24, 2012

From @sisyphus

p5pRT commented Sep 24, 2012

From @bulk88

p5pRT commented Sep 25, 2012

From @jandubois

toddr commented Feb 13, 2020

richardleach commented Feb 13, 2020

tonycoz commented Dec 3, 2020