Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with filetest -x _ on Win2k AS Perl build 626 #4145

Closed
p5pRT opened this issue Jun 27, 2001 · 8 comments
Closed

Problem with filetest -x _ on Win2k AS Perl build 626 #4145

p5pRT opened this issue Jun 27, 2001 · 8 comments

Comments

@p5pRT
Copy link

p5pRT commented Jun 27, 2001

Migrated from rt.perl.org#7194 (status was 'open')

Searchable as RT7194$

@p5pRT
Copy link
Author

p5pRT commented Jun 27, 2001

From Rudif@bluemail.ch

Created by rudif@bluemail.ch

On Win2k, AS Perl build 626 - but not on Linux -

I have found that executing the filetest -T _ or -B _ has the side effect of
causing the subsequent filetest -x _ to give the wrong answer (false where it should be true).

The script below demonstrates the problem.

Specifically, subs filetest2() and filetest3() below fail to identify files
c:/perl/bin/pod2html.bat c:/perl/bin/perl.exe as Executable, while the
reference sub filetest0() identifies them correctly.
Also, there are two workarounds for the problem, in subs filetest1() and filetest4():
use -x _ BEFORE -T or -B, or use -x $file.

I have discovered this problem while I was running Version 0.21 Pod::Find::pod_find() to
list pod-containing files in C:/perl/bin, where I expected to see several .bat files listed.

But there were none, although several of them contain pod (e.g. pod2html.bat).

I looked into Pod::Find, drilled down into _check_and_extract_name() and I found the code ...

sub _check_and_extract_name {
my ($file, $verbose, $root_rx) = @_;

# check extension or executable flag
# this involves testing the .bat extension on Win32!
unless(-f $file && -T _ && ($file =~ /\.(pod|pm|plx?)\z/i || -x _ )) {
return undef;
}

return undef unless contains_pod($file,$verbose);

# ... proceed with adding the accepted file to the lost of pods ...

... that decides to accept or reject a file, based on the extension and on file tests.
In the case of .bat files, this code was returning undef.

After some experimenting I found two workarounds that fix the problem:

unless(-f $file _ && ($file =~ /\.(pod|pm|plx?)\z/i || -x _ ) && -T) {
return undef;

unless(-f $file && -T _ && ($file =~ /\.(pod|pm|plx?)\z/i || -x $file )) {
return undef;

These are consistent with workarounds shown in the script below.

HTH
Rudi Farkas

The test script:

#! /usr/bin/perl -w

use strict;

#
# This script checks the behavior of filetest -x _
# and demonstrates a problem in Win2k where filetest2() and filetest3() below
# fail to classify the 2 test files (a script and the perl interpreter) as executable.
# By rudif@bluemail.ch 26 Jun 2001
#

my ($script, $exe);
if ($^O =~ /Win/) {
# assume the usual directory - please edit if different on your machine
($script, $exe) = qw ( c:/perl/bin/pod2html.bat c:/perl/bin/perl.exe );
}
else {
# assume *nix
# assume the usual directory - please edit if different on your machine
($script, $exe) = qw ( /usr/bin/pod2html /usr/bin/perl );
}

die "no such file $script" unless -f $script;
die "no such file $exe" unless -f $exe;

printf "OS $^O, perl %vd\n\n", $^V;

compare(filetest0($exe), filetest1($exe));
compare(filetest0($exe), filetest2($exe));
compare(filetest0($exe), filetest3($exe));
compare(filetest0($exe), filetest4($exe));
print "\n";

compare(filetest0($script), filetest1($script));
compare(filetest0($script), filetest2($script));
compare(filetest0($script), filetest3($script));
compare(filetest0($script), filetest4($script));
print "\n";

sub filetest0 { # reference - not using _
my $file = shift;
my @props;
push @props, "Readable" if -r $file;
push @props, "Writable" if -w $file;
push @props, "Executable" if -x $file;
push @props, "Binary" if -B $file;
push @props, "Text" if -T $file;
join ' ', "filetest0 $file: ", sort @props;
}

sub filetest1 {
my $file = shift;
stat($file);
my @props;
push @props, "Readable" if -r _;
push @props, "Writable" if -w _;
push @props, "Executable" if -x _; # before -B and -T
push @props, "Binary" if -B _;
push @props, "Text" if -T _;
join ' ', "filetest1 $file: ", sort @props;
}

sub filetest2 {
my $file = shift;
stat($file);
my @props;
push @props, "Readable" if -r _;
push @props, "Writable" if -w _;
push @props, "Text" if -T _;
push @props, "Executable" if -x _; # after -T _
push @props, "Binary" if -B _;
join ' ', "filetest2 $file: ", sort @props;
}

sub filetest3 {
my $file = shift;
stat($file);
my @props;
push @props, "Readable" if -r _;
push @props, "Writable" if -w _;
push @props, "Text" if -T $file;
push @props, "Executable" if -x _; # after -T $file
push @props, "Binary" if -B _;
join ' ', "filetest3 $file: ", sort @props;
}

sub filetest4 {
my $file = shift;
stat($file);
my @props;
push @props, "Readable" if -r _;
push @props, "Writable" if -w _;
push @props, "Text" if -T _;
push @props, "Executable" if -x $file; # not using _
push @props, "Binary" if -B _;
join ' ', "filetest4 $file: ", sort @props;
}

sub compare {
my ($ref, $other) = @_;
(my $_ref = $ref) =~ s/.*://;
(my $_other = $other) =~ s/.*://;
if ($_ref eq $_other) {
printf " ok $other\n",
}
else {
printf "not ok $other\n",
}
}


__END__

# output on Win2k

OS MSWin32, perl 5.6.1

ok filetest1 c:/perl/bin/perl.exe: Binary Executable Readable Writable
not ok filetest2 c:/perl/bin/perl.exe: Binary Readable Writable
not ok filetest3 c:/perl/bin/perl.exe: Binary Readable Writable
ok filetest4 c:/perl/bin/perl.exe: Binary Executable Readable Writable

ok filetest1 c:/perl/bin/pod2html.bat: Executable Readable Text Writable
not ok filetest2 c:/perl/bin/pod2html.bat: Readable Text Writable
not ok filetest3 c:/perl/bin/pod2html.bat: Readable Text Writable
ok filetest4 c:/perl/bin/pod2html.bat: Executable Readable Text Writable
Perl Info
---
Flags:
    category=core
    severity=medium
---
Site configuration information for perl v5.6.1:

Configured by rudolf.farkas at Wed May  2 01:31:01 2001.

Summary of my perl5 (revision 5 version 6 subversion 1) configuration:
  Platform:
    osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    usethreads=undef use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=undef d_sfio=undef uselargefiles=undef usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
  Compiler:
    cc='cl', ccflags ='-nologo -O1 -MD -DNDEBUG -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT  -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DPERL_MSVCRT_READFIX',
    optimize='-O1 -MD -DNDEBUG',
    cppflags='-DWIN32'
    ccversion='', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=4
    alignbytes=8, usemymalloc=n, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -release  -libpath:"C:\Perl\lib\CORE"  -machine:x86'
    libpth="C:\Program Files\Microsoft Visual Studio\VC98\mfc\lib" "C:\Program Files\Microsoft Visual Studio\VC98\lib" "C:\Perl\lib\CORE"
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl56.lib
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release  -libpath:"C:\Perl\lib\CORE"  -machine:x86'

Locally applied patches:
    ACTIVEPERL_LOCAL_PATCHES_ENTRY

---
@INC for perl v5.6.1:
    C:/Perl/lib
    C:/Perl/site/lib
    .

---
Environment for perl v5.6.1:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=H:\Perl\bin\;;H:\devperl\u;C:\WINNT\system32;C:\WINNT;C:\WINNT\System32\Wbem;C:\Program Files\Perforce;C:\usr\local\wbin;C:\Program Files\Microsoft Visual Studio\Common\Tools\WinNT;C:\Program Files\Microsoft Visual Studio\Common\MSDev98\Bin;C:\Program Files\Microsoft Visual Studio\Common\Tools;C:\Program Files\Microsoft Visual Studio\VC98\bin;C:\Program Files\Microsoft Platform SDK\Bin\;C:\Program Files\Microsoft Platform SDK\Bin\WinNT;
    PERL5LIB=H:\tools\pl;
    PERL_BADLANG (unset)
    SHELL (unset)

@p5pRT
Copy link
Author

p5pRT commented Apr 30, 2010

From @chorny

Same result on 5.12.0 (Strawberry, Win XP). Attaching better test program.

--
Alexandr Ciornii, http​://chorny.net

@p5pRT
Copy link
Author

p5pRT commented Apr 30, 2010

From @chorny

#! /usr/bin/perl -w

use strict;
use Config;
#
# This script checks the behavior of filetest -x _ 
# and demonstrates a problem in Win2k where filetest2() and filetest3() below
# fail to classify the 2 test files (a script and the perl interpreter) as executable.
# By rudif@bluemail.ch 26 Jun 2001
# 

my ($script, $exe);
if ($^O =~ /Win/) { 
    ($script, $exe) = ( "$Config{bin}\\pod2html.bat","$Config{bin}\\perl.exe" );
}
else { 
    # assume *nix
    # assume the usual directory - please edit if different on your machine
    ($script, $exe) = qw ( /usr/bin/pod2html /usr/bin/perl );
}

die "no such file $script" unless -f $script;
die "no such file $exe" unless -f $exe;

printf "OS $^O, perl %vd\n\n", $^V;

compare(filetest0($exe), filetest1($exe));
compare(filetest0($exe), filetest2($exe));
compare(filetest0($exe), filetest3($exe));
compare(filetest0($exe), filetest4($exe));
print "\n";

compare(filetest0($script), filetest1($script));
compare(filetest0($script), filetest2($script));
compare(filetest0($script), filetest3($script));
compare(filetest0($script), filetest4($script));
print "\n";

sub filetest0 {     # reference - not using _
    my $file = shift;
    my @props;
    push @props, "Readable" if -r $file;
    push @props, "Writable" if -w $file;
    push @props, "Executable" if -x $file;
    push @props, "Binary" if -B $file;
    push @props, "Text" if -T $file;
    join ' ', "filetest0 $file: ", sort @props; 
}

sub filetest1 {     
    my $file = shift;
    stat($file);
    my @props;
    push @props, "Readable" if -r _;
    push @props, "Writable" if -w _;
    push @props, "Executable" if -x _;  # before -B and -T
    push @props, "Binary" if -B _;
    push @props, "Text" if -T _;
    join ' ', "filetest1 $file: ", sort @props; 
}

sub filetest2 {     
    my $file = shift;
    stat($file);
    my @props;
    push @props, "Readable" if -r _;
    push @props, "Writable" if -w _;
    push @props, "Text" if -T _;
    push @props, "Executable" if -x _; # after -T _
    push @props, "Binary" if -B _;
    join ' ', "filetest2 $file: ", sort @props; 
}

sub filetest3 {
    my $file = shift;
    stat($file);
    my @props;
    push @props, "Readable" if -r _;
    push @props, "Writable" if -w _;
    push @props, "Text" if -T $file; 
    push @props, "Executable" if -x _;  # after -T $file
    push @props, "Binary" if -B _;
    join ' ', "filetest3 $file: ", sort @props; 
}

sub filetest4 {
    my $file = shift;
    stat($file);
    my @props;
    push @props, "Readable" if -r _;
    push @props, "Writable" if -w _;
    push @props, "Text" if -T _;
    push @props, "Executable" if -x $file; # not using _
    push @props, "Binary" if -B _;
    join ' ', "filetest4 $file: ", sort @props; 
}

sub compare {
    my ($ref, $other) = @_;
    (my $_ref = $ref) =~ s/.*://;
    (my $_other = $other) =~ s/.*://;
    if ($_ref eq $_other) {
        printf "    ok $other\n", 
    }
    else {
        printf "not ok $other\n", 
    }
}


__END__

@p5pRT
Copy link
Author

p5pRT commented Apr 30, 2010

From [Unknown Contact. See original ticket]

Same result on 5.12.0 (Strawberry, Win XP). Attaching better test program.

--
Alexandr Ciornii, http​://chorny.net

@richardleach
Copy link
Contributor

Still occurs with Strawberry v5.30.0 on Win 10.

Notes for digging into this:

  • In pp_sys.c, -B is PP(pp_ftbinary) and -T is PP(pp_fttext)
  • PP(pp_ftbinary) calls PP(pp_fttext), so the bug is probably in PP(pp_fttext)

@richardleach
Copy link
Contributor

richardleach commented Oct 27, 2019

Ah, #4146 was initially a duplicate of this ticket and therefore was resolved, but it contained further diagnosis from the OP not present in this ticket. Specifically, that the behaviour is:

due to discrepancy in st_mode values returned by Microsoft
functions _stat() and _fstat(). Specifically, I found that _stat() sets
the 3
Execute bits (mask 0111 octal) to 1 when it sees an executable file, while
_fstat() sets these bits to 0 when looking at the same file.

Modern VS2019 documentation follows what the OP described:

In that other ticket, OP was keen that perl should work around this MS behavior such that perl's behavior is consistent across platforms.

Seems like our 2019 options are either:

  • put a Windows-specific extra calls to stat() in PP(pp_fttext) - not sure if this is practical, but at least the filename is known.
  • amend win32_fstat() to get the filename and then mock up the missing behaviour. We would have to get the filename from the file descriptor though, which looks hairy unless we only fix the behaviour for Windows releases from Windows Vista and Windows Server 2008 onwards (i.e. actual current versions of Windows): https://docs.microsoft.com/en-us/windows/win32/memory/obtaining-a-file-name-from-a-file-handle
  • amend perlfunc to describe this behaviour

Given the limitations around the use of these operators anyway (i.e. no actual DACL parsing, heuristic determination of executability), amending perlfunc seems to me like the appropriate effort. (Although willing to look into the feasibility of the first option when time allows.)

@toddr
Copy link
Member

toddr commented Feb 13, 2020

Given the state of things, documenting might be a good choice. Is there a replacement perl module for windows that is a better choice?

genio added a commit to genio/perl5 that referenced this issue Oct 15, 2020
As discussed on the mailing list here:
https://www.nntp.perl.org/group/perl.perl5.porters/2020/10/msg258453.html

This just removes the declaration that we support the very old versions
of Windows that have long since been EOLed.

For reference of problems related to maintaining the EOLed versions:
Perl#4145
Perl#6080
Perl#7410
Perl#8502
Perl#9025
Perl#12431
Perl#14687
@tonycoz
Copy link
Contributor

tonycoz commented Dec 3, 2020

This is fixed by e935ef3 at least on Vista or later.

Older versions of Windows don't have a function to retrieve the filename for an open file handle.

@tonycoz tonycoz closed this as completed Dec 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants