Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

each on an anonymous array constructor doesn't complain, but sorta works #11163

Open
p5pRT opened this issue Mar 1, 2011 · 21 comments
Open

each on an anonymous array constructor doesn't complain, but sorta works #11163

p5pRT opened this issue Mar 1, 2011 · 21 comments

Comments

@p5pRT
Copy link

p5pRT commented Mar 1, 2011

Migrated from rt.perl.org#85150 (status was 'open')

Searchable as RT85150$

@p5pRT
Copy link
Author

p5pRT commented Mar 1, 2011

From @briandfoy

I thought I'd be clever with each() on an anonymous array that's
not in a variable​:

  use Unicode​::Normalize;

  my $string = "éáabcáá\x{65}\x{301}í";

  while( my( $index, $grapheme ) = each [$string =~ m/(\X)/g] ) {
  my $decomposed = NFD( $grapheme );
  print "Found an accent at $index ($grapheme)\n"
  if -1 < index( $decomposed, "\x{301}" );
  }

I know that each() is an array operator and this had little hope of working,
but I figured I'd give it a shot. It doesn't complain, and it always
returns the first element, probably because it's a new anonymous array
every time. There's probably no chance of it working because there's
no variable to hang state on (but I would love if if could).

I think there should be a warning here, and maybe even a compilation error.


Summary of my perl5 (revision 5 version 13 subversion 10) configuration​:

  Platform​:
  osname=darwin, osvers=10.6.0, archname=darwin-2level
  uname='darwin roscoe.local 10.6.0 darwin kernel version 10.6.0​: wed nov
10 18​:13​:17 pst 2010; root​:xnu-1504.9.26~3release_i386 i386 i386 '
  config_args='-des -Dprefix=/usr/local/perls/perl-5.13.10 -Dusedevel'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-fno-common -DPERL_DARWIN -no-cpp-precomp
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include',
  optimize='-O3',
  cppflags='-no-cpp-precomp -fno-common -DPERL_DARWIN -no-cpp-precomp
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.2.1 (Apple Inc. build 5664)',
gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector
-L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib
  libs=-ldbm -ldl -lm -lutil -lc
  perllibs=-ldl -lm -lutil -lc
  libc=/usr/lib/libc.dylib, so=dylib, useshrplib=false, libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
  cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup
-L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl)​:
  Compile-time options​: PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP
PERL_USE_DEVEL
  USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES
  USE_PERLIO USE_PERL_ATOF
  Built under darwin
  Compiled at Feb 25 2011 03​:21​:17
  @​INC​:
  /usr/local/perls/perl-5.13.10/lib/site_perl/5.13.10/darwin-2level
  /usr/local/perls/perl-5.13.10/lib/site_perl/5.13.10
  /usr/local/perls/perl-5.13.10/lib/5.13.10/darwin-2level
  /usr/local/perls/perl-5.13.10/lib/5.13.10
  .

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From @jkeenan

On Mon Feb 28 22​:34​:48 2011, comdog wrote​:

I thought I'd be clever with each() on an anonymous array that's
not in a variable​:

I know that each() is an array operator and this had little hope of
working,
but I figured I'd give it a shot. It doesn't complain, and it always
returns the first element, probably because it's a new anonymous array
every time. There's probably no chance of it working because there's
no variable to hang state on (but I would love if if could).

I think there should be a warning here, and maybe even a compilation
error.

In Perl 5.14, 'perldoc -f each' states (near the end of the doc)​:

#####
Starting with Perl 5.14, "each" can take a scalar EXPR, which
must hold reference to an unblessed hash or array. The
argument will be dereferenced automatically. This aspect of
"each" is considered highly experimental. The exact behaviour
may change in a future version of Perl.
#####

Do you think we need warnings or fatals above and beyond this?

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From @briandfoy

#####
Starting with Perl 5.14, "each" can take a scalar EXPR, which
must hold reference to an unblessed hash or array.  The
argument will be dereferenced automatically.  This aspect of
"each" is considered highly experimental.  The exact behaviour
may change in a future version of Perl.
#####

Do you think we need warnings or fatals above and beyond this?

Yes, which is why I said so. It doesn't take an EXPR. It takes a VAR.
It has to be a variable. It should n't try to silently work otherwise.

--
brian d foy <brian.d.foy@​gmail.com>
http​://www.pair.com/~comdog/

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From @ikegami

On Mon, Jan 2, 2012 at 10​:44 PM, brian d foy <brian.d.foy@​gmail.com> wrote​:

Yes, which is why I said so. It doesn't take an EXPR. It takes a VAR.
It has to be a variable. It should n't try to silently work otherwise.

No, it doesn't just take a "VAR" anymore.

$ perl -wE'@​a=qw( a b c ); say while ($_) = each(\@​a);'
0
1
2

$ perl -wE'@​a=qw( a b c ); sub f { \@​a } say while ($_) = each(f());'
0
1
2

It's even possible to have buggy code when passing a "VAR".

$ perl -wE'say while ($_) = each(@​{ [qw( a b c )] });'
0
0
0
0
0
0
...

Given that C<< each EXPR >> isn't necessarily wrong and C<< each VAR >>
isn't necessarily right, I'm not sure what kind of warnings could be added
safely.

Given

  - some pretty fuzzy heuristics would need to be employed,
  - few people have stumbled on this (It's come up once or twice on
  PerlMonks where the problem was promptly identified),
  - the buggy code exhibits the problem on practically every run, and
  - it's easy to isolate the buggy code when it occurs,

I'm not sure that anything needs to be done.
- Eric

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From @cpansprout

On Tue Jan 03 12​:59​:01 2012, ikegami@​adaelis.com wrote​:

On Mon, Jan 2, 2012 at 10​:44 PM, brian d foy <brian.d.foy@​gmail.com>
wrote​:

Yes, which is why I said so. It doesn't take an EXPR. It takes a VAR.
It has to be a variable. It should n't try to silently work otherwise.

No, it doesn't just take a "VAR" anymore.

$ perl -wE'@​a=qw( a b c ); say while ($_) = each(\@​a);'
0
1
2

$ perl -wE'@​a=qw( a b c ); sub f { \@​a } say while ($_) = each(f());'
0
1
2

It's even possible to have buggy code when passing a "VAR".

$ perl -wE'say while ($_) = each(@​{ [qw( a b c )] });'
0
0
0
0
0
0
...

Given that C<< each EXPR >> isn't necessarily wrong and C<< each VAR >>
isn't necessarily right, I'm not sure what kind of warnings could be added
safely.

Given

- some pretty fuzzy heuristics would need to be employed,
- few people have stumbled on this (It's come up once or twice on
PerlMonks where the problem was promptly identified),
- the buggy code exhibits the problem on practically every run, and
- it's easy to isolate the buggy code when it occurs,

I'm not sure that anything needs to be done.
- Eric

We could do something similar to this, based on SvTEMP and SvREFCNT​:

$ perl5.15.6 -we '+sub​:lvalue{my$x}->()=3'
Useless assignment to a temporary at -e line 1.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From @briandfoy

$ perl -wE'@​a=qw( a b c ); say while ($_) = each(\@​a);'

$ perl -wE'@​a=qw( a b c ); sub f { \@​a } say while ($_) = each(f());'

Both of these still have a named variable attached to the data.

$ perl -wE'say while ($_) = each(@​{ [qw( a b c )] });'

This does not have a named variable and is the point of my original
report. This doesn't work, but implying that each takes an EXPR
implies that it should. Either the docs need to be fixed to note the
limitation or each needs to work on anonymous references.

That is, you've said what I initially reported.

There are instances where the use of each has no hope of working but
Perl will silently try anyway. I disagree that there's nothing to be
done about that.

--
brian d foy <brian.d.foy@​gmail.com>
http​://www.pair.com/~comdog/

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From zefram@fysh.org

brian d foy wrote​:

$ perl -wE'say while ($_) = each(@​{ [qw( a b c )] });'

   This doesn't work\,

On the contrary, it works perfectly, doing exactly what the statement
appears to say. Each time round the loop it constructs a new array and
takes a reference to it ([...]), dereferences to get the array (@​{...}),
and then commences iteration over the array (each(...)). Since you
told it to construct a new array each time, the iteration over that
array necessarily starts afresh each time. The endless zeroes tell you,
correctly, that each of the arrays you asked for begins with an element
numbered zero.

The loop will end as soon as one of the constructed arrays has
no elements. You can get this immediately by using the constructor
expression [] rather than [qw(a b c)], or you can get more complicated
behaviour with something like [rand(1)<0.8 ? qw(a b c) : ()].

                   Either the docs need to be fixed to note the

limitation or each needs to work on anonymous references.

It works perfectly, and has no surprising limitation.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From @briandfoy

On the contrary, it works perfectly, doing exactly what the statement
appears to say.

That's why I suggested it should be a warning. It's probably not what
people want.

As for surprising limitations, well, I was surprised. It's another
thing I need to explain with each() when I teach it because I know a
couple of students in each class will try it. It doesn't matter to you
if that makes sense or isn't the way you think. Stupid stuff happens
all of the time. Ignoring that it's going to happen doesn't make it go
away. We could remove most warnings using the same premise, but we
don't because they are a salve for the stupidity we know will happen.

However, this is the end of it for me. Decide what you like and I'll
just add it to the list of quirks that people have to know.

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From @ikegami

On Tue, Jan 3, 2012 at 4​:14 PM, brian d foy <brian.d.foy@​gmail.com> wrote​:

$ perl -wE'@​a=qw( a b c ); say while ($_) = each(\@​a);'

$ perl -wE'@​a=qw( a b c ); sub f { \@​a } say while ($_) = each(f());'

Both of these still have a named variable attached to the data.

Noone said "named variable". We were talking of VAR.

$ perl -wE'say while ($_) = each(@​{ [qw( a b c )] });'

This does not have a named variable

Noone said "named variable". We were talking of VAR.

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From @rjbs

* Eric Brine <ikegami@​adaelis.com> [2012-01-03T17​:10​:52]

Noone said "named variable". We were talking of VAR.

perlfunc uses EXPR, LIST, VAR, VARIABLE, SOCKET, and so on. There's an
explanation of explanation of LIST, but nothing else. If we're going to get
picky about exact terminology, maybe the terminology should be better
explained. I don't see anything anywhere to help with this. Am I missing it?

--
rjbs

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From @rjbs

* brian d foy <brian.d.foy@​gmail.com> [2012-01-03T17​:09​:17]

On the contrary, it works perfectly, doing exactly what the statement
appears to say.

That's why I suggested it should be a warning. It's probably not what
people want.

I agree with both of these bits. Yes, if you have a very good mental model of
the subtleties of what is happening, this very simple construct works. The
problem is that you shouldn't need to understand subtleties for something that
looks this simple.

How hard is a warning here?

Also, I think Jim's original question was on point​: there is a big "warning,"
which is that each ARRAYREF is experimental. I look forward to future
experiments requiring that someone explicitly allow them to be run in his or
her code.

--
rjbs

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From @davidnicol

each(X) is almost always a mistake when X is built fresh each time
each(X) is invoked -- it might make sense when we know that X will be
either a hashref or an arrayref but not which and we want the first
elt if its an array or some random pair when it's a hashref; that's
highly contrived and would not be clear at all. So reference-counting
the argument could be good grounds for the warning, unless that
produce a lot of false warnings without reference-counting the stack.
Automatic dereferencing is not the issue here, it's the failure of the
object of the each method to persist between invocations.

@p5pRT
Copy link
Author

p5pRT commented Jan 3, 2012

From zefram@fysh.org

Ricardo Signes wrote​:

How hard is a warning here?

What is the proposed scope of a warning? What's been misunderstood
here isn't "each", it's "while". You could get an essentially identical
effect from "... while pop @​{[1,2,3]}" or from any other builtin or sub
that side-effects an array or hash argument.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Jan 4, 2012

From @davidnicol

On Tue, Jan 3, 2012 at 5​:47 PM, Zefram <zefram@​fysh.org> wrote​:

Ricardo Signes wrote​:

How hard is a warning here?

What is the proposed scope of a warning?  What's been misunderstood
here isn't "each", it's "while".  You could get an essentially identical
effect from "... while pop @​{[1,2,3]}" or from any other builtin or sub
that side-effects an array or hash argument.

-zefram

the warning could say "mutator used in boolean context" for instance,
and it might go off whenever a returned value gets downgraded to
true/false. Discouraging that could go a long way towards eventually
making all that "0e0" and "0but_true" a memory.

@p5pRT
Copy link
Author

p5pRT commented Jan 4, 2012

From @ikegami

On Tue, Jan 3, 2012 at 6​:47 PM, Zefram <zefram@​fysh.org> wrote​:

Ricardo Signes wrote​:

How hard is a warning here?

What is the proposed scope of a warning? What's been misunderstood
here isn't "each", it's "while". You could get an essentially identical
effect from "... while pop @​{[1,2,3]}" or from any other builtin or sub
that side-effects an array or hash argument.

-zefram

*If* C<pop> warns on a SvTEMP and SvREFCNT check, perldiag should suggest

(@​{ TEMP_EXPR })[-1]

as an alternative to

pop @​{ TEMP_EXPR }

@p5pRT
Copy link
Author

p5pRT commented Jan 4, 2012

From @ikegami

On Tue, Jan 3, 2012 at 6​:22 PM, Ricardo Signes <perl.p5p@​rjbs.manxome.org>wrote​:

* Eric Brine <ikegami@​adaelis.com> [2012-01-03T17​:10​:52]

Noone said "named variable". We were talking of VAR.

perlfunc uses EXPR, LIST, VAR, VARIABLE, SOCKET, and so on. There's an
explanation of explanation of LIST, but nothing else. If we're going to
get
picky about exact terminology, maybe the terminology should be better
explained. I don't see anything anywhere to help with this. Am I missing
it?

Yes, my reply didn't make much sense.

Basically, I didn't want to guess what he meant by named variables, since
he claimed there were named variable where there clearly weren't. Neither a
deref or a sub call are named variable.

And anything that starts with @​ or % is considered a named variable (at
least when prototypes and pre-5.14 C<each> are concerned), so yes, there is
a named variable in the last example I gave.

As a Father C pointed out, the correct check relates to TEMP and REFCNT,
not "named".

- Eric

@p5pRT
Copy link
Author

p5pRT commented Jan 4, 2012

From @rjbs

* Ricardo Signes <perl.p5p@​rjbs.manxome.org> [2012-01-03T18​:28​:55]

Also, I think Jim's original question was on point​: there is a big "warning,"
which is that each ARRAYREF is experimental. I look forward to future
experiments requiring that someone explicitly allow them to be run in his or
her code.

...except it wasn't, of course, because this happens with non-reference
anonymous arrays, too, as already demonstrated in many other posts, but here
too​:

  perl -E 'say while ($_) = each @​{[ 1 ]}

--
rjbs

@p5pRT
Copy link
Author

p5pRT commented Jan 4, 2012

From @rjbs

* Zefram <zefram@​fysh.org> [2012-01-03T18​:47​:01]

Ricardo Signes wrote​:

How hard is a warning here?

What is the proposed scope of a warning? What's been misunderstood
here isn't "each", it's "while".

Yes, thanks -- while I /knew/ that, it hadn't hit home.

I'll try to think about whether I think this is worth pursuing from a
user-experience perspective. Thing is, I think "each" and "pop" are going to
seem Just Different to the student.

My initial feeling is that this is not going to be worth dealing with in perl.
That's just a hunch.

--
rjbs

@p5pRT
Copy link
Author

p5pRT commented Jan 4, 2012

From @b2gills

This should warn, or the anonymous array/hash reference should be the
same on every iteration.

  while( my($k,$v) = each [0..3] ){ ... }
  while( my($k,$v) = each {0..3} ){ ... }

If we make the reference the same on every iteration, it should
possibly warn if the user doesn't declare a minimum version.


These should at least warn, since there is no way to break out of the
loop without `goto`, or `last`.
There is also no possible change in each iteration.

( These are anonymous array, or hash references created with constant lists )

  while( my($k,$v) = each [0..3] ){ ... }
  while( my($k,$v) = each @​{[0..3]} ){ ... }

  while( my($k,$v) = each {0..3} ){ ... }
  while( my($k,$v) = each %{{0..3}} ){ ... }

  while( my($k,$v) = pop [0..3] ){ ... }
  while( my($k,$v) = pop @​{[0..3]} ){ ... }
  while( my($k,$v) = shift [0..3] ){ ... }
  while( my($k,$v) = shift @​{[0..3]} ){ ... }
  while( my $n = unshift [0..3], 0 ){ ... }
  while( my $n = unshift @​{[0..3]}, 0 ){ ... }

(possibly other builtins should be added to this list.)


These might be worth warning as well since it suffers the same problem.
I can see this being used by a Master Perl Guru to do something amazing though.

  while( my($k,$v) = each my $arr = [0..3] ){ ... }
  while( my($k,$v) = each @​{ my $arr = [0..3] } ){ ... }

  while( my($k,$v) = each my $hash = {0..3} ){ ... }
  while( my($k,$v) = each %{ my $hash = {0..3} } ){ ... }

  while( my($k,$v) = pop my $arr = [0..3] ){ ... }
  while( my($k,$v) = pop @​{ my $arr = [0..3] } ){ ... }
  while( my($k,$v) = shift my $arr = [0..3] ){ ... }
  while( my($k,$v) = shift @​{ my $arr = [0..3] } ){ ... }
  while( my $n = unshift my $arr = [0..3], 0 ){ ... }
  while( my $n = unshift @​{ my $arr = [0..3] }, 0 ){ ... }


These possibly shouldn't warn, or be modified since the value will
likely be different on each run.

  while( my($k,$v) = each [rand] ){ ... }
  while( my($k,$v) = each @​{[rand]} ){ ... }
  while( my($k,$v) = each [subroutine()] ){ ... }
  while( my($k,$v) = each @​{[subroutine()]} ){ ... }
  etc.


These possibly shouldn't warn, or be modified since the value can
change in the loop, and thus be different on subsequent runs.

  while( my($k,$v) = each [@​ARGV] ){ ... }
  while( my($k,$v) = each [@​array] ){ ... }
  while( my($k,$v) = each [%hash] ){ ... }
  while( my($k,$v) = each [$_] ){ ... }
  while( my($k,$v) = each [$scalar] ){ ... }
  etc.


The warning should mention that it is because of interactions of a
while loop and an anonymous array/hash reference with a constant list

@p5pRT
Copy link
Author

p5pRT commented Jan 4, 2012

From @jkeenan

On Mon Jan 02 19​:17​:02 2012, jkeenan wrote​:

In Perl 5.14, 'perldoc -f each' states (near the end of the doc)​:

#####
Starting with Perl 5.14, "each" can take a scalar EXPR, which
must hold reference to an unblessed hash or array. The
argument will be dereferenced automatically. This aspect of
"each" is considered highly experimental. The exact behaviour
may change in a future version of Perl.
#####

Historical side-note​: Until I had to look at the current documentation
for 'each' for the purpose of commenting on this ticket, not only was I
unaware that you could say 'each <EXPR>' -- where EXPR is an array or
hash reference -- but I was completely unaware that you could say 'each
<LIST>' and have something meaningful happen.

I guess that reflects the fact that I learned 'each' 11 years ago at a
time when the Camel book only documented 'each <HASH>'. And I've never
had occasion to want to say 'each' on anything but a hash.

Now, back to the main thread ...

jimk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants