readline/readdir does not assign to $_ implicitly. #16825

p5pRT · 2019-01-24T03:02:35Z

Migrated from rt.perl.org#133795 (status was 'open')

Searchable as RT133795$

p5pRT · 2019-01-24T03:02:36Z

From @toddr

Created by @toddr

We've been experimenting with virtualizing directory interaction in
Test::MockFile by overriding Core::GLOBAL::readdir.

I got a report about a problem with implcit assignment to $_ not working
via cpanel/Test-MockFile#32

To try to clarify the problem we came up with this perl script.

use strict;
use warnings;

BEGIN { *CORE::GLOBAL::readdir = sub { $_[0] } }

print "*** readdir assigns to \$_ implicitly\n";
while (readdir 'foo') {
  print;
  last;
}

print "*** readdir undef\n";
while (readdir undef) { # doesn't implicitly check defined()
  print;
  last;
}

print "*** \$_ = readdir 'foo'\n";
while ($_ = readdir 'foo') {
  print;
  last;
}

I would have expected foo 2 times but instead I got this. The uninitialized
check should't have happened either.

*** readdir assigns to $_ implicitly
Use of uninitialized value $_ in print at projects/Test-MockFile/foo.pl
line 8.
*** readdir undef
*** $_ = readdir 'foo'

Perl Info


Flags:
    category=core
    severity=low

Site configuration information for perl 5.28.0:

Configured by cPanel at Tue Nov 27 16:50:45 CST 2018.

Summary of my perl5 (revision 5 version 28 subversion 0) configuration:

  Platform:
    osname=linux
    osvers=3.10.0-123.20.1.el7.x86_64
    archname=x86_64-linux-64int
    uname='linux rpmbuild-64-centos-7.dev.cpanel.net
3.10.0-123.20.1.el7.x86_64 #1 smp thu jan 29 18:05:33 utc 2015 x86_64
x86_64 x86_64 gnulinux '
    config_args='-des -Dusedevel -Darchname=x86_64-linux-64int
-Dcc=/usr/bin/gcc -Dcpp=/usr/bin/cpp -Dusemymalloc=n -DDEBUGGING=none
-Doptimize=-Os -Accflags=-m64 -Dccflags=-DPERL_DISABLE_PMC -fPIC -DPIC
-Duseshrplib -Duselargefiles=yes -Duseposix=true -Dhint=recommended
-Duseperlio=yes -Dprefix=/usr/local/cpanel/3rdparty/perl/528
-Dsiteprefix=/opt/cpanel/perl5/528 -Dsitebin=/opt/cpanel/perl5/528/bin
-Dsitelib=/opt/cpanel/perl5/528/site_lib -Dusevendorprefix=true
-Dvendorbin=/usr/local/cpanel/3rdparty/perl/528/bin
-Dvendorprefix=/usr/local/cpanel/3rdparty/perl/528/lib/perl5
-Dvendorlib=/usr/local/cpanel/3rdparty/perl/528/lib/perl5/cpanel_lib
-Dprivlib=/usr/local/cpanel/3rdparty/perl/528/lib/perl5/5.28.0
-Dman1dir=none -Dman3dir=none
-Dscriptdir=/usr/local/cpanel/3rdparty/perl/528/bin
-Dscriptdirexp=/usr/local/cpanel/3rdparty/perl/528/bin -Dsiteman1dir=none
-Dsiteman3dir=none
-Dinstallman1dir=none -Dversiononly=no -Dinstallusrbinperl=no
-Dcf_by=cPanel -Dmyhostname=localhost -Dperladmin=root@localhost -Dcf_email=
support@cpanel.net -DDB_File=true -Ud_dosuid -Uuserelocatableinc -Umad
-Uusethreads -Uusemultiplicity -Uusesocks -Uuselongdouble -Duse64bitint
-Uuse64bitall'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='/usr/bin/gcc'
    ccflags ='-DPERL_DISABLE_PMC -fPIC -DPIC -m64 -fwrapv
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-Os'
    cppflags='-DPERL_DISABLE_PMC -fPIC -DPIC -m64 -fwrapv
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='4.8.2 20140120 (Red Hat 4.8.2-16)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='/usr/bin/gcc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib
/lib64 /usr/lib64 /usr/local/lib64
    libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
-lgdbm_compat
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.17.so
    so=so
    useshrplib=true
    libperl=libperl.so
    gnulibc_version='2.17'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E
-Wl,-rpath,/usr/local/cpanel/3rdparty/perl/528/lib/perl5/5.28.0/x86_64-linux-64int/CORE'
    cccdlflags='-fPIC'
    lddlflags='-shared -Os -L/usr/local/lib -fstack-protector-strong'

Locally applied patches:
    cPanel - disable man installs
    cPanel - cPanel INC PATH


@INC for perl 5.28.0:
    /usr/local/cpanel

/usr/local/cpanel/3rdparty/perl/528/lib/perl5/cpanel_lib/x86_64-linux-64int
    /usr/local/cpanel/3rdparty/perl/528/lib/perl5/cpanel_lib
    /usr/local/cpanel/3rdparty/perl/528/lib/perl5/5.28.0/x86_64-linux-64int
    /usr/local/cpanel/3rdparty/perl/528/lib/perl5/5.28.0
    /opt/cpanel/perl5/528/site_lib/x86_64-linux-64int
    /opt/cpanel/perl5/528/site_lib


Environment for perl 5.28.0:
    HOME=/root
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/bin
    PERL_BADLANG (unset)
    PERL_USE_UNSAFE_INC=0
    SHELL=/bin/zsh

p5pRT · 2019-01-24T03:08:59Z

From @tonycoz

Subject: readline does not assign to $_ implicitly.

On Wed, 23 Jan 2019 19:02:36 -0800, TODDR wrote:

We've been experimenting with virtualizing directory interaction in
Test::MockFile by overriding Core::GLOBAL::readdir.
(and no mention of readline)

For anyone confused by the mismatch between the subject and body,
this occurs for both readline and readdir.

Tony

p5pRT · 2019-01-24T03:08:59Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2019-01-24T03:39:30Z

From @Grinnz

On Wed, 23 Jan 2019 19:02:36 -0800, TODDR wrote:

use strict;
use warnings;

BEGIN { *CORE::GLOBAL::readdir = sub { $_[0] } }

print "*** readdir assigns to \$_ implicitly\n";
while (readdir 'foo') {
print;
last;
}

print "*** readdir undef\n";
while (readdir undef) { # doesn't implicitly check defined()
print;
last;
}

print "*** \$_ = readdir 'foo'\n";
while ($_ = readdir 'foo') {
print;
last;
}
---

I would have expected foo 2 times but instead I got this. The
uninitialized
check should't have happened either.

*** readdir assigns to $_ implicitly
Use of uninitialized value $_ in print at projects/Test-
MockFile/foo.pl
line 8.
*** readdir undef
*** $_ = readdir 'foo'

The warning is from trying to print undef because the first call did not assign anything to $_. The second loop does not run at all. That's actually correct behavior since it returns undef, but if you instead use a defined false value like '' or 0, the condition still fails; this shows that the implicit defined check is also not being applied (only returning undef should fail the condition).

-Dan

p5pRT · 2019-02-04T05:08:46Z

From @tonycoz

On Wed, 23 Jan 2019 19:02:36 -0800, TODDR wrote:

We've been experimenting with virtualizing directory interaction in
Test::MockFile by overriding Core::GLOBAL::readdir.

I got a report about a problem with implcit assignment to $_ not
working
via cpanel/Test-MockFile#32

To try to clarify the problem we came up with this perl script.

As discussed in IRC this happens due to the order in which ops are replaced with functions relative to where the "while (readline...)" -> "while ($_ = readline)" and "while ($x = readline)" => "while (defined($x = readline))" transformation is done.

An op like readdir or readline is replaced with it's override in the block in toke.c around line 7211, ie. during tokenization, so the checks in Perl_newLOOPOP() and Perl_newWHILEOP() see an entersub instead of a readdir/readline.

To fix it, I think we'd need to prevent readdir/readline being replaced in toke.c and do the check later, either in the new(WHILE|LOOP)OP(), or in a ck function.

Tony

toddr · 2020-03-23T22:21:48Z

@tonycoz How significantly would we be de-optimizing this to make it work? I guess the question I'm asking myself is: "Should we?"

tonycoz · 2020-03-25T03:15:03Z

It would slow compilation a little, but I doubt it would make a noticeable difference.

It shouldn't make any difference to runtime, except for the desired extra defined($_ = ... ) ops.

The current behaviour is fairly surprising, so I don't think this should be WONTFIX.

jkeenan · 2023-10-29T22:47:17Z

It would slow compilation a little, but I doubt it would make a noticeable difference.

It shouldn't make any difference to runtime, except for the desired extra defined($_ = ... ) ops.

The current behaviour is fairly surprising, so I don't think this should be WONTFIX.

@tonycoz, @toddr, are we going to take any further action with respect to this ticket?

tonycoz · 2023-10-29T23:10:43Z

I think it's still a bug (or at least surprising.)

toddr · 2023-10-30T00:07:23Z

My bug in Test::MockFile is unfixable because of this if I recall correctly.

p5pRT added Severity Low type-core labels Oct 19, 2019

toddr changed the title ~~readline does not assign to $_ implicitly.~~ readline/readdir does not assign to $_ implicitly. Oct 19, 2019

toddr assigned toddr and tonycoz and unassigned toddr Mar 23, 2020

xenu removed the affects-5.28 label Nov 19, 2021

xenu removed the Severity Low label Dec 29, 2021

jkeenan added the Closable? label Oct 29, 2023

jkeenan removed the Closable? label Oct 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

readline/readdir does not assign to $_ implicitly. #16825

readline/readdir does not assign to $_ implicitly. #16825

p5pRT commented Jan 24, 2019

p5pRT commented Jan 24, 2019 •

edited by toddr

Loading

p5pRT commented Jan 24, 2019

p5pRT commented Jan 24, 2019

p5pRT commented Jan 24, 2019

p5pRT commented Feb 4, 2019

toddr commented Mar 23, 2020

tonycoz commented Mar 25, 2020

jkeenan commented Oct 29, 2023

tonycoz commented Oct 29, 2023

toddr commented Oct 30, 2023

readline/readdir does not assign to $_ implicitly. #16825

readline/readdir does not assign to $_ implicitly. #16825

Comments

p5pRT commented Jan 24, 2019

p5pRT commented Jan 24, 2019 • edited by toddr Loading

From @toddr

Created by @toddr

p5pRT commented Jan 24, 2019

From @tonycoz

p5pRT commented Jan 24, 2019

p5pRT commented Jan 24, 2019

From @Grinnz

p5pRT commented Feb 4, 2019

From @tonycoz

toddr commented Mar 23, 2020

tonycoz commented Mar 25, 2020

jkeenan commented Oct 29, 2023

tonycoz commented Oct 29, 2023

toddr commented Oct 30, 2023

p5pRT commented Jan 24, 2019 •

edited by toddr

Loading