File::Glob sometimes returns pattern if no match, regardless of GLOB_NOCHECK #14702

p5pRT · 2015-05-13T12:20:25Z

Migrated from rt.perl.org#125173 (status was 'open')

Searchable as RT125173$

p5pRT · 2015-05-13T12:20:25Z

From @epa

Created by @epa

The documentation for File::Glob states

File::Glob::bsd_glob() implements the FreeBSD glob(3) routine,

The POSIX defined flags for bsd_glob() are:
...
GLOB_NOCHECK
If the pattern does not match any pathname, then bsd_glob()
returns a list consisting of only the pattern.

This implies that the behaviour of returning the pattern back on no
match is governed by this flag. If the flag is not given, then a
pattern which matches no filenames would be expected to return an
empty list. But in fact this sometimes happens and sometimes not:

% mkdir new
% cd new
% perl -MFile::Glob -E 'say File::Glob::bsd_glob("x")'
x
% perl -MFile::Glob -E 'say File::Glob::bsd_glob("x", File::Glob::GLOB_NOCHECK())'
x
% perl -MFile::Glob -E 'say File::Glob::bsd_glob("x*")'

% perl -MFile::Glob -E 'say File::Glob::bsd_glob("x*", File::Glob::GLOB_NOCHECK())'
x*

The first of the four cases above is odd. GLOB_NOCHECK was not
specified; the pattern matched no files; and yet the pattern is being
returned back.

While this is consistent with Perl's builtin glob(), it seems
inconsistent with File::Glob's documentation and with the FreeBSD
glob(3) it purports to implement - at least as far as I can tell from
for example
<https://www.freebsd.org/cgi/man.cgi?query=glob&sektion=3>.

I suggest fixing the behaviour of bsd_glob() to be as documented - so
a pattern with no matches returns nothing, unless GLOB_NOCHECK is
given, in which case the pattern itself is returned as the only match.

(That does not imply that Perl's builtin glob() should change its
behaviour, since glob() is defined to do what /bin/csh would do.
Currently, glob() is implemented in terms of bsd_glob(), but that
would need to change a little bit.)

If changing what bsd_glob does is too great a break with bugwards
compatibility, a new flag GLOB_CHECK (not present in FreeBSD or Linux)
could be added to request the more consistent behaviour.

Perl Info


Flags:
    category=library
    severity=medium
    module=File::Glob

Site configuration information for perl 5.18.4:

Configured by Red Hat, Inc. at Thu Apr  2 16:17:20 UTC 2015.

Summary of my perl5 (revision 5 version 18 subversion 4) configuration:
   
  Platform:
    osname=linux, osvers=3.19.1-201.fc21.x86_64, archname=x86_64-linux-thread-multi
    uname='linux buildvm-23.phx2.fedoraproject.org 3.19.1-201.fc21.x86_64 #1 smp wed mar 18 04:29:24 utc 2015 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mtune=generic -Dccdlflags=-Wl,--enable-new-dtags -Dlddlflags=-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mtune=generic -Wl,-z,relro  -Dshrpdir=/usr/lib64 -DDEBUGGING=-g -Dversion=5.18.4 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl -Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin -Dusesitecustomize'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.8.3 20140911 (Red Hat 4.8.3-7)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -fstack-protector'
    libpth=/usr/local/lib64 /lib64 /usr/lib64
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.18'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--enable-new-dtags'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -Wl,-z,relro '

Locally applied patches:
    Fedora Patch1: Removes date check, Fedora/RHEL specific
    Fedora Patch3: support for libdir64
    Fedora Patch4: use libresolv instead of libbind
    Fedora Patch5: USE_MM_LD_RUN_PATH
    Fedora Patch6: Skip hostname tests, due to builders not being network capable
    Fedora Patch7: Dont run one io test due to random builder failures
    Fedora Patch9: Fix find2perl to translate ? glob properly (RT#113054)
    Fedora Patch10: Update h2ph(1) documentation (RT#117647)
    Fedora Patch11: Update pod2html(1) documentation (RT#117623)
    Fedora Patch12: Disable ornaments on perl5db AutoTrace tests (RT#118817)
    Fedora Patch14: Do not use system Term::ReadLine::Gnu in tests (RT#118821)
    Fedora Patch15: Define SONAME for libperl.so
    Fedora Patch16: Install libperl.so to -Dshrpdir value
    Fedora Patch18: Fix crash with \\&$glob_copy (RT#119051)
    Fedora Patch19: Fix coreamp.t rand test (RT#118237)
    Fedora Patch20: Reap child in case where exception has been thrown (RT#114722)
    Fedora Patch21: Fix using regular expressions containing multiple code blocks (RT#117917)
    Fedora Patch22: Create site paths by cpan for the first time (CPAN RT#99905)
    Fedora Patch200: Link XS modules to libperl.so with EU::CBuilder on Linux
    Fedora Patch201: Link XS modules to libperl.so with EU::MM on Linux


@INC for perl 5.18.4:
    /home/eda/lib/perl5/
    /home/eda/lib64/perl5/
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5
    .


Environment for perl 5.18.4:
    HOME=/home/eda
    LANG=en_GB.UTF-8
    LANGUAGE (unset)
    LC_COLLATE=C
    LC_CTYPE=en_GB.UTF-8
    LC_MESSAGES=en_GB.UTF-8
    LC_MONETARY=en_GB.UTF-8
    LC_NUMERIC=en_GB.UTF-8
    LC_TIME=en_GB.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/eda/bin:/home/eda/bin:/usr/local/bin:/usr/bin:/sbin:/usr/sbin:/sbin:/usr/sbin
    PERL5LIB=/home/eda/lib/perl5/:/home/eda/lib64/perl5/
    PERL_BADLANG (unset)
    SHELL=/bin/bash

-- 
Ed Avis <eda@waniasset.com>
Please ignore autogenerated disclaimer below this point.


This email is intended only for the person to whom it is addressed and may contain confidential information. Any retransmission, copying, disclosure or other use of, this information by persons other than the intended recipient is prohibited. If you received this email in error, please contact the sender and delete the material. This email is for information only and is not intended as an offer or solicitation for the purchase or sale of any financial instrument. Wadhwani Asset Management LLP is a Limited Liability Partnership registered in England (OC303168) with registered office at 40 Berkeley Square, 3rd Floor, London, W1J 5AL. It is authorised and regulated by the Financial Conduct Authority.

p5pRT · 2015-05-14T11:03:36Z

From @ap

* Ed Avis <perlbug-followup@perl.org> [2015-05-13 14:25]:

While this is consistent with Perl's builtin glob(), it seems
inconsistent with File::Glob's documentation and with the FreeBSD
glob(3) it purports to implement - at least as far as I can tell from
for example
<https://www.freebsd.org/cgi/man.cgi?query=glob&sektion=3>.

Within an empty directory the attached C program yields the following
output on my FreeBSD box:

(no flags) x:
No such file or directory

(GLOB_NOCHECK) x:
x

(no flags) x*:
No error: 0

(GLOB_NOCHECK) x*:
x*

p5pRT · 2015-05-14T11:03:36Z

From @ap

#include <stdio.h>
#include <glob.h>

void
print_glob (char * description, char * pattern, int flags)
{
	glob_t g;
	int i;

	printf("\n(%s) %s:\n", description, pattern);

	if (glob(pattern, flags, NULL, &g))
		return (void) perror("");

	for (i = 0; i < g.gl_pathc; ++i)
		printf("%s\n", g.gl_pathv[i]);
}

int
main ()
{
	print_glob("no flags",     "x", 0);
	print_glob("GLOB_NOCHECK", "x", GLOB_NOCHECK);
	print_glob("no flags",     "x*", 0);
	print_glob("GLOB_NOCHECK", "x*", GLOB_NOCHECK);
}

p5pRT · 2015-05-14T11:03:36Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2015-05-14T11:11:35Z

From @epa

Thanks for the test program. But I think it is missing the fact that
(according to my reading of the FreeBSD manual page) the glob() function
always sets the array of results, even if there was an error. Only the
GLOB_ERR flag or a callback error handler will make it abort on error.

Please try this expanded test program:

#include <stdio.h>
#include <glob.h>

void
print_glob (char * description, char * pattern, int flags)
{
glob_t g;
int i;

printf("\n(%s) %s:\n", description, pattern);

if (glob(pattern, flags, NULL, &g))
(void) perror("");
else
puts("no error from glob");

printf("%d results returned\n", g.gl_pathc);
for (i = 0; i < g.gl_pathc; ++i)
printf("result %d: %s\n", i, g.gl_pathv[i]);
}

int
main ()
{
print_glob("no flags", "x", 0);
print_glob("GLOB_NOCHECK", "x", GLOB_NOCHECK);
print_glob("no flags", "x*", 0);
print_glob("GLOB_NOCHECK", "x*", GLOB_NOCHECK);
}

--
Ed Avis <eda@waniasset.com>

p5pRT · 2016-10-13T13:18:41Z

From @epa

Please could I ask the perl5-porters to have another look at this bug?

epa · 2023-12-21T16:13:42Z

As this bug was originally filed against an older perl version, I checked it with a somewhat newer one. Using 5.26.3 I can still reproduce the same behaviour:

% perl -MFile::Glob -E 'say File::Glob::bsd_glob("x")'
x

and the File::Glob documentation still says

GLOB_NOCHECK
If the pattern does not match any pathname, then bsd_glob() returns
a list consisting of only the pattern.

So I believe the behaviour is still inconsistent with the documentation (which at least implies that if GLOB_NOCHECK is not given then you will not get the pattern back on no match).

The C test program I suggested

#include <stdio.h>
#include <glob.h>

void
print_glob (char * description, char * pattern, int flags)
{
  glob_t g;
  int i;

  printf("\n(%s) %s:\n", description, pattern);

  if (glob(pattern, flags, NULL, &g))
  (void) perror("");
  else
  puts("no error from glob");

  printf("%d results returned\n", g.gl_pathc);
  for (i = 0; i < g.gl_pathc; ++i)
  printf("result %d: %s\n", i, g.gl_pathv[i]);
}

int
main ()
{
  print_glob("no flags", "x", 0);
  print_glob("GLOB_NOCHECK", "x", GLOB_NOCHECK);
  print_glob("no flags", "x*", 0);
  print_glob("GLOB_NOCHECK", "x*", GLOB_NOCHECK);
}

produces these results on Linux 4.18 with glibc 2.28:

(no flags) x:
No such file or directory
0 results returned

(GLOB_NOCHECK) x:
no error from glob
1 results returned
result 0: x

(no flags) x*:
No such file or directory
0 results returned

(GLOB_NOCHECK) x*:
no error from glob
1 results returned
result 0: x*

Note the first case, where the pattern did not match, GLOB_NOCHECK was not set, and the result from glob() was a zero return value, setting errno, and no results returned. I believe File::Glob's behaviour is inconsistent not only with its own docs, but with C glob().

p5pRT added Severity Medium distro-Linux type-library labels Oct 19, 2019

xenu removed the affects-5.18 label Nov 19, 2021

xenu removed the Severity Medium label Dec 15, 2021

jkeenan removed the distro-Linux label Dec 21, 2023

jkeenan mentioned this issue Dec 21, 2023

Behavior of File::Glob export tags is poorly tested #21744

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

File::Glob sometimes returns pattern if no match, regardless of GLOB_NOCHECK #14702

File::Glob sometimes returns pattern if no match, regardless of GLOB_NOCHECK #14702

p5pRT commented May 13, 2015

p5pRT commented May 13, 2015

p5pRT commented May 14, 2015

p5pRT commented May 14, 2015

p5pRT commented May 14, 2015

p5pRT commented May 14, 2015

p5pRT commented Oct 13, 2016

epa commented Dec 21, 2023

File::Glob sometimes returns pattern if no match, regardless of GLOB_NOCHECK #14702

File::Glob sometimes returns pattern if no match, regardless of GLOB_NOCHECK #14702

Comments

p5pRT commented May 13, 2015

p5pRT commented May 13, 2015

From @epa

Created by @epa

p5pRT commented May 14, 2015

From @ap

p5pRT commented May 14, 2015

From @ap

p5pRT commented May 14, 2015

p5pRT commented May 14, 2015

From @epa

p5pRT commented Oct 13, 2016

From @epa

epa commented Dec 21, 2023