From agalama@mailfish.de

Created by galama@mailfish.de

When called with a dereferenced undefined scalar variable, keys()
and values() create a reference to an empty hash​:

$ perl -MData​::Dumper -e 'print Dumper($h); keys %$h; print Dumper($h)'
$VAR1 = undef;
$VAR1 = {};
$ perl -MData​::Dumper -e 'print Dumper($h); values %$h; print Dumper($h)'
$VAR1 = undef;
$VAR1 = {};
$

The documentation says​: "As a side effect, calling keys()/values()
resets the internal interator of the HASH or ARRAY (see "each")."

I am not sure if this is meant to imply that $h should point to a newly
created empty hash afterwards in the above case. If so, this should be
stated explicitly. I found this very surprising.

Flags:
    category=core
    severity=low

This perlbug was built using Perl 5.14.4 in the Fedora build system.
It is being executed now by Perl 5.14.4 - Thu Apr 11 12:56:04 UTC 2013.

Site configuration information for perl 5.14.4:

Configured by Red Hat, Inc. at Thu Apr 11 12:56:04 UTC 2013.

Summary of my perl5 (revision 5 version 14 subversion 4) configuration:

  Platform:
    osname=linux, osvers=2.6.32-358.2.1.el6.x86_64,
archname=x86_64-linux-thread-multi
    uname='linux buildvm-11.phx2.fedoraproject.org
2.6.32-358.2.1.el6.x86_64 #1 smp wed feb 20 12:17:37 est 2013 x86_64
x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4  -m64 -mtune=generic
-Dccdlflags=-Wl,--enable-new-dtags -Dlddlflags=-shared -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4  -m64 -mtune=generic -Wl,-z,relro
-DDEBUGGING=-g -Dversion=5.14.4 -Dmyhostname=localhost
-Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr
-Dvendorprefix=/usr -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5
-Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl
-Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl
-Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64
/usr/lib64 -Duseshrplib -Dusethreads -Duseithreads
-Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db
-Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio
-Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly
-Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto
-Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto
-Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.7.2 20120921 (Red Hat 4.7.2-2)',
gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -fstack-protector'
    libpth=/usr/local/lib64 /lib64 /usr/lib64
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread
-lc -lgdbm_compat
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.15'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef,
ccdlflags='-Wl,--enable-new-dtags -Wl,-rpath,/usr/lib64/perl5/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -Wl,-z,relro '

Locally applied patches:



@INC for perl 5.14.4:
    /home/xxx/src/perl
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5
    .


Environment for perl 5.14.4:
    HOME=/home/xxx
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LC_MEASUREMENT=de_DE.utf8
    LC_MONETARY=de_DE.utf8
    LC_NUMERIC=de_DE.utf8
    LC_TIME=de_DE.utf8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/xxx/.local/bin:/home/xxx/bin
    PERL5LIB=/home/xxx/src/perl
    PERL_BADLANG (unset)
    SHELL=/bin/bash

From @epa

It is certainly odd that keys or values succeeds when a simple
dereference in list context (giving key,value pairs) would fail.

% perl -E 'use warnings; use strict; my $a; my @​x = keys %$a; say "ok"'
ok
% perl -E 'use warnings; use strict; my $a; my @​x = %$a; say "ok"'
perl -E 'use warnings; use strict; my $a; my @​x = %$a; say "ok"'
Can't use an undefined value as a HASH reference at -e line 1.

On the other hand, dereferencing and looking up a value does autovivify​:

% perl -E 'use warnings; use strict; my $a; my $x = $a->{a}; say $a'
HASH(0x233aa68)

The RT System itself - Status changed from 'new' to 'open'

From @ikegami

On Tue, Jul 23, 2013 at 7​:57 AM, Ed Avis via RT
<perlbug-followup@​perl.org>wrote​:

It is certainly odd that keys or values succeeds when a simple
dereference in list context (giving key,value pairs) would fail.

Sub arguments and some function arguments are lvalues. lvalue derefs
autovivify. rvalue derefs croak.

This is behaving properly.

perl -E"sub { }->(%$a); say $a;"
HASH(0x2cd048)

perl -E"keys(%$a); say $a;"
HASH(0x25d068)

From @ikegami

On Tue, Jul 23, 2013 at 10​:36 AM, Eric Brine <ikegami@​adaelis.com> wrote​:

On Tue, Jul 23, 2013 at 7​:57 AM, Ed Avis via RT <perlbug-followup@​perl.org

wrote​:

It is certainly odd that keys or values succeeds when a simple
dereference in list context (giving key,value pairs) would fail.

Sub arguments and some function arguments are lvalues. lvalue derefs
autovivify. rvalue derefs croak.

This is behaving properly.

perl -E"sub { }->(%$a); say $a;"
HASH(0x2cd048)

perl -E"keys(%$a); say $a;"
HASH(0x25d068)

perl -E"values(%$a); say $a;"
HASH(0x5ad028)

perl -E"map +(), %$a; say $a;"
HASH(0x61d068)

perl -E"for (%$a) { } say $a;"
HASH(0x21d068)

From @epa

Eric Brine <ikegami <at> adaelis.com> writes​:

It is certainly odd that keys or values succeeds when a simple
dereference in list context (giving key,value pairs) would fail.

Sub arguments and some function arguments are lvalues. lvalue derefs
autovivify. rvalue derefs croak.

That doesn't really explain *why* the argument to keys and values should be
an lvalue. Which I suppose is what this bug is really about. The perlfunc
documentation for keys and values does not say whether its argument is an
lvalue or an rvalue. Or are you saying that these builtins follow the
general rule for subroutines?

At first sight, getting the keys and values together with %$a doesn't seem
much different from (keys(%$a), values(%$a)). Just a different ordering.
Yet one autovivifies and the other does not.

The 'scalar' builtin used to get array length doesn't autovivify​:

% perl -E 'use warnings; use strict; my $a; my $x = scalar @​$a'
Can't use an undefined value as an ARRAY reference at -e line 1.

Is it documented which builtins take lvalues and which rvalues?

--
Ed Avis <eda@​waniasset.com>

From @ikegami

On Tue, Jul 23, 2013 at 11​:19 AM, Ed Avis <eda@​waniasset.com> wrote​:

Or are you saying that these builtins follow the
general rule for subroutines?

Yes. But it turns out that most named ops don't.

perl -E"length($$a); say $a ? 'lvalue' : 'rvalue';"
rvalue

perl -E"oct($$a); say $a ? 'lvalue' : 'rvalue';"
rvalue

perl -E"substr($$a, 0); say $a ? 'lvalue' : 'rvalue';"
rvalue

perl -E"push @​a, $$a; say $a ? 'lvalue' : 'rvalue';"
rvalue

So it doesn't appear to be as simple as I thought it was.

From @epa

...and to add another wrinkle, the autovivification for keys and values does
not happen when using the new auto-deref syntax​:

% perl -E 'use warnings; use strict; my $a = {}; my @​x = keys $a; say "ok"'
ok
% perl -E 'use warnings; use strict; my $a; my @​x = keys $a; say "ok"'
Type of argument to keys on reference must be unblessed hashref or arrayref

But this might have been a deliberate decision for auto-deref, even if it
does mean that 'keys $a' has different semantics to 'keys %$a'.

--
Ed Avis <eda@​waniasset.com>

From @ikegami

On Wed, Jul 24, 2013 at 3​:12 AM, Ed Avis <eda@​waniasset.com> wrote​:

But this might have been a deliberate decision for auto-deref, even if it
does mean that 'keys $a' has different semantics to 'keys %$a'.

It's never keys that does the autovivification, it's the dereference. In
this new code, there's no dereference, and keys itself can't dereference
until it known whether it hash a hash ref or an array ref.