From @khwilliamson

This is a bug report for perl from khw@​khw.(none),
generated with the help of perlbug 1.40 running under perl 5.26.0.

glibc 2.24 deprecates readdir_r. And, compiling perl creates annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if two
threads share a stream. For those, it recommends using readdir with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.

Flags​:
  category=core
  severity=low

Site configuration information for perl 5.26.0​:

Configured by khw at Mon Apr 24 08​:31​:45 MDT 2017.

Summary of my perl5 (revision 5 version 26 subversion 0) configuration​:
  Commit id​: 63e6b12
  Platform​:
  osname=linux
  osvers=4.10.0-19-generic
  archname=x86_64-linux-thread-multi-ld
  uname='linux khw 4.10.0-19-generic #21-ubuntu smp thu apr 6
17​:04​:57 utc 2017 x86_64 x86_64 x86_64 gnulinux '
  config_args='-des -Uversiononly -Dprefix=/home/khw/blead -Dusedevel
-D'optimize=-ggdb3' -A'optimize=-ggdb3' -A'optimize=-O0'
-Accflags='-DPERL_BOOL_AS_CHAR' -Accflags='-DPERL_EXTERNAL_GLOB'
-Dman1dir=none -Dman3dir=none -Dcc=g++ -DDEBUGGING -Dusemorebits
-Dusecbacktrace -Dusethreads'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=define
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='g++'
  ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_BOOL_AS_CHAR
-DPERL_EXTERNAL_GLOB -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
  optimize='-ggdb3 -ggdb3 -O0'
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_BOOL_AS_CHAR
-DPERL_EXTERNAL_GLOB -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='6.3.0 20170406'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='long double'
  nvsize=16
  Off_t='off_t'
  lseeksize=8
  alignbytes=16
  prototype=define
  Linker and Libraries​:
  ld='g++'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/include/c++/6 /usr/include/x86_64-linux-gnu/c++/6
/usr/include/c++/6/backward /usr/local/lib
/usr/lib/gcc/x86_64-linux-gnu/6/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.24.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.24'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -ggdb3 -ggdb3 -O0 -L/usr/local/lib
-fstack-protector-strong'

@​INC for perl 5.26.0​:
  /home/khw/perl/blead/lib
  /home/khw/perl/blead/t
  /home/khw/blead/lib/perl5/site_perl/5.26.0/x86_64-linux-thread-multi-ld
  /home/khw/blead/lib/perl5/site_perl/5.26.0
  /home/khw/blead/lib/perl5/5.26.0/x86_64-linux-thread-multi-ld
  /home/khw/blead/lib/perl5/5.26.0
  /home/khw/blead/lib/perl5/site_perl/5.25.12
  /home/khw/blead/lib/perl5/site_perl/5.25.11
  /home/khw/blead/lib/perl5/site_perl/5.25.10
  /home/khw/blead/lib/perl5/site_perl/5.25.9
  /home/khw/blead/lib/perl5/site_perl/5.25.8
  /home/khw/blead/lib/perl5/site_perl/5.25.7
  /home/khw/blead/lib/perl5/site_perl/5.25.6
  /home/khw/blead/lib/perl5/site_perl/5.25.5
  /home/khw/blead/lib/perl5/site_perl/5.25.4
  /home/khw/blead/lib/perl5/site_perl/5.25.3
  /home/khw/blead/lib/perl5/site_perl/5.25.2
  /home/khw/blead/lib/perl5/site_perl/5.25.1
  /home/khw/blead/lib/perl5/site_perl/5.24.0
  /home/khw/blead/lib/perl5/site_perl/5.23.10
  /home/khw/blead/lib/perl5/site_perl/5.23.9
  /home/khw/blead/lib/perl5/site_perl/5.23.8
  /home/khw/blead/lib/perl5/site_perl/5.23.7
  /home/khw/blead/lib/perl5/site_perl/5.23.6
  /home/khw/blead/lib/perl5/site_perl/5.23.5
  /home/khw/blead/lib/perl5/site_perl/5.23.4
  /home/khw/blead/lib/perl5/site_perl/5.23.3
  /home/khw/blead/lib/perl5/site_perl/5.23.2
  /home/khw/blead/lib/perl5/site_perl/5.23.1
  /home/khw/blead/lib/perl5/site_perl/5.23.0
  /home/khw/blead/lib/perl5/site_perl/5.22.0
  /home/khw/blead/lib/perl5/site_perl/5.21.12
  /home/khw/blead/lib/perl5/site_perl/5.21.11
  /home/khw/blead/lib/perl5/site_perl/5.21.10
  /home/khw/blead/lib/perl5/site_perl/5.21.9
  /home/khw/blead/lib/perl5/site_perl/5.21.8
  /home/khw/blead/lib/perl5/site_perl/5.21.7
  /home/khw/blead/lib/perl5/site_perl/5.21.6
  /home/khw/blead/lib/perl5/site_perl/5.21.5
  /home/khw/blead/lib/perl5/site_perl/5.21.4
  /home/khw/blead/lib/perl5/site_perl/5.21.3
  /home/khw/blead/lib/perl5/site_perl/5.21.2
  /home/khw/blead/lib/perl5/site_perl/5.21.1
  /home/khw/blead/lib/perl5/site_perl/5.20.0
  /home/khw/blead/lib/perl5/site_perl/5.19.12
  /home/khw/blead/lib/perl5/site_perl/5.19.11
  /home/khw/blead/lib/perl5/site_perl/5.19.10
  /home/khw/blead/lib/perl5/site_perl

Environment for perl 5.26.0​:
  HOME=/home/khw
  LANG=en_US.UTF-8
  LANGUAGE=en_US
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)

PATH=/usr/lib/ccache​:/home/khw/bin​:/home/khw/perl5/perlbrew/bin​:/home/khw/print/bin​:/bin​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/sbin​:/usr/games​:/usr/local/games​:/home/khw/iands/www​:/home/khw/cxoffice/bin
  PERL5OPT=-w
  PERL_BADLANG (unset)
  PERL_DIFF_TOOL=wgdiff
  PERL_POD_PEDANTIC=1
  SHELL=/bin/ksh

From @khwilliamson

READDIR_R(3) Linux Programmer's Manual READDIR_R(3)

NAME
  readdir_r - read a directory

SYNOPSIS
  #include <dirent.h>

  int readdir_r(DIR *dirp, struct dirent *entry, struct dirent **result);

  Feature Test Macro Requirements for glibc (see feature_test_macros(7))​:

  readdir_r()​:
  _POSIX_C_SOURCE
  || /* Glibc versions <= 2.19​: */ _BSD_SOURCE || _SVID_SOURCE

DESCRIPTION
  This function is deprecated; use readdir(3) instead.

  The readdir_r() function was invented as a reentrant version of read‐
  dir(3). It reads the next directory entry from the directory stream
  dirp, and returns it in the caller-allocated buffer pointed to by
  entry. For details of the dirent structure, see readdir(3).

  A pointer to the returned buffer is placed in *result; if the end of
  the directory stream was encountered, then NULL is instead returned in
  *result.

  It is recommended that applications use readdir(3) instead of read‐
  dir_r(). Furthermore, since version 2.24, glibc deprecates read‐
  dir_r(). The reasons are as follows​:

  * On systems where NAME_MAX is undefined, calling readdir_r() may be
  unsafe because the interface does not allow the caller to specify
  the length of the buffer used for the returned directory entry.

  * On some systems, readdir_r() can't read directory entries with very
  long names. When the glibc implementation encounters such a name,
  readdir_r() fails with the error ENAMETOOLONG after the final direc‐
  tory entry has been read. On some other systems, readdir_r() may
  return a success status, but the returned d_name field may not be
  null terminated or may be truncated.

  * In the current POSIX.1 specification (POSIX.1-2008), readdir(3) is
  not required to be thread-safe. However, in modern implementations
  (including the glibc implementation), concurrent calls to readdir(3)
  that specify different directory streams are thread-safe. There‐
  fore, the use of readdir_r() is generally unnecessary in multi‐
  threaded programs. In cases where multiple threads must read from
  the same directory stream, using readdir(3) with external synchro‐
  nization is still preferable to the use of readdir_r(), for the rea‐
  sons given in the points above.

  * It is expected that a future version of POSIX.1 will make read‐
  dir_r() obsolete, and require that readdir(3) be thread-safe when
  concurrently employed on different directory streams.

RETURN VALUE
  The readdir_r() function returns 0 on success. On error, it returns a
  positive error number (listed under ERRORS). If the end of the direc‐
  tory stream is reached, readdir_r() returns 0, and returns NULL in
  *result.

ERRORS
  EBADF Invalid directory stream descriptor dirp.

  ENAMETOOLONG
  A directory entry whose name was too long to be read was encoun‐
  tered.

ATTRIBUTES
  For an explanation of the terms used in this section, see
  attributes(7).

  ┌────────────┬───────────────┬─────────┐
  │Interface │ Attribute │ Value │
  ├────────────┼───────────────┼─────────┤
  │readdir_r() │ Thread safety │ MT-Safe │
  └────────────┴───────────────┴─────────┘
CONFORMING TO
  POSIX.1-2001, POSIX.1-2008.

SEE ALSO
  readdir(3)

COLOPHON
  This page is part of release 4.09 of the Linux man-pages project. A
  description of the project, information about reporting bugs, and the
  latest version of this page, can be found at
  https://www.kernel.org/doc/man-pages/.

  2016-03-01 READDIR_R(3)

From @tonycoz

On Wed, 26 Apr 2017 11​:24​:08 -0700, public@​khwilliamson.com wrote​:

glibc 2.24 deprecates readdir_r. And, compiling perl creates annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if two
threads share a stream. For those, it recommends using readdir with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.

This is something we'd need to detect, somehow.

POSIX still defines readdir_r() and doesn't require that readdir() be thread-safe (though that may change, see http​://austingroupbugs.net/view.php?id=696 )

Tony

The RT System itself - Status changed from 'new' to 'open'

From @Tux

On Tue, 23 May 2017 19​:11​:24 -0700, "Tony Cook via RT"
<perlbug-followup@​perl.org> wrote​:

On Wed, 26 Apr 2017 11​:24​:08 -0700, public@​khwilliamson.com wrote​:

glibc 2.24 deprecates readdir_r. And, compiling perl creates annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if two
threads share a stream. For those, it recommends using readdir with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.

This is something we'd need to detect, somehow.

POSIX still defines readdir_r() and doesn't require that readdir() be thread-safe (though that may change, see http​://austingroupbugs.net/view.php?id=696 )

This has already been addressed in branch arc_readdir_after5260
Aaron promised to merge after 5.26.0

--
H.Merijn Brand http​://tux.nl Perl Monger http​://amsterdam.pm.org/
using perl5.00307 .. 5.25 porting perl5 on HP-UX, AIX, and openSUSE
http​://mirrors.develooper.com/hpux/ http​://www.test-smoke.org/
http​://qa.perl.org http​://www.goldmark.org/jeff/stupid-disclaimers/

From @xsawyerx

On 05/24/2017 09​:46 AM, H.Merijn Brand wrote​:

On Tue, 23 May 2017 19​:11​:24 -0700, "Tony Cook via RT"
<perlbug-followup@​perl.org> wrote​:

On Wed, 26 Apr 2017 11​:24​:08 -0700, public@​khwilliamson.com wrote​:

glibc 2.24 deprecates readdir_r. And, compiling perl creates annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if two
threads share a stream. For those, it recommends using readdir with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.
This is something we'd need to detect, somehow.

POSIX still defines readdir_r() and doesn't require that readdir() be thread-safe (though that may change, see http​://austingroupbugs.net/view.php?id=696 )
This has already been addressed in branch arc_readdir_after5260
Aaron promised to merge after 5.26.0

This can be merged into blead-next.

From @khwilliamson

On Sat, 27 May 2017 10​:15​:59 -0700, xsawyerx@​gmail.com wrote​:

On 05/24/2017 09​:46 AM, H.Merijn Brand wrote​:

On Tue, 23 May 2017 19​:11​:24 -0700, "Tony Cook via RT"
<perlbug-followup@​perl.org> wrote​:

On Wed, 26 Apr 2017 11​:24​:08 -0700, public@​khwilliamson.com wrote​:

glibc 2.24 deprecates readdir_r. And, compiling perl creates
annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if
two
threads share a stream. For those, it recommends using readdir
with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.
This is something we'd need to detect, somehow.

POSIX still defines readdir_r() and doesn't require that readdir()
be thread-safe (though that may change, see
http​://austingroupbugs.net/view.php?id=696 )
This has already been addressed in branch arc_readdir_after5260
Aaron promised to merge after 5.26.0

This can be merged into blead-next.

This is on our blockers list for 5.28. Did this merge ever happen?
--
Karl Williamson

From @xsawyerx

On 03/20/2018 01​:23 AM, Karl Williamson via RT wrote​:

On Sat, 27 May 2017 10​:15​:59 -0700, xsawyerx@​gmail.com wrote​:

On 05/24/2017 09​:46 AM, H.Merijn Brand wrote​:

On Tue, 23 May 2017 19​:11​:24 -0700, "Tony Cook via RT"
<perlbug-followup@​perl.org> wrote​:

On Wed, 26 Apr 2017 11​:24​:08 -0700, public@​khwilliamson.com wrote​:

glibc 2.24 deprecates readdir_r. And, compiling perl creates
annoying
warning noise.

Starting with that version, plain readdir is reentrant, but not if
two
threads share a stream. For those, it recommends using readdir
with
synchronization exclusion (i.e., mutexes I suppose).

A man page is attached.
This is something we'd need to detect, somehow.

POSIX still defines readdir_r() and doesn't require that readdir()
be thread-safe (though that may change, see
http​://austingroupbugs.net/view.php?id=696 )
This has already been addressed in branch arc_readdir_after5260
Aaron promised to merge after 5.26.0
This can be merged into blead-next.
This is on our blockers list for 5.28. Did this merge ever happen?

I found this​: c6709bf.

From @khwilliamson

I'm closing it then.
--
Karl Williamson

@khwilliamson - Status changed from 'open' to 'pending release'

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release yesterday of Perl 5.28.0, this and 185 other issues have been
resolved.

Perl 5.28.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.28.0

If you find that the problem persists, feel free to reopen this ticket.

@khwilliamson - Status changed from 'pending release' to 'resolved'