Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Double quotes in map call induce a syntax error #14781

Open
p5pRT opened this issue Jul 1, 2015 · 15 comments
Open

Double quotes in map call induce a syntax error #14781

p5pRT opened this issue Jul 1, 2015 · 15 comments

Comments

@p5pRT
Copy link

p5pRT commented Jul 1, 2015

Migrated from rt.perl.org#125522 (status was 'open')

Searchable as RT125522$

@p5pRT
Copy link
Author

p5pRT commented Jul 1, 2015

From the.rob.dixon@gmail.com

This is a bug report for perl from the.rob.dixon@​gmail.com,
generated with the help of perlbug 1.40 running under perl 5.22.0.


There is a strange issue with double-quoted variables when using `map`

This is fine

  %h = map { $k => 1 } 1;

but this

  %h = map { "$k" => 1 } 1;

throws a syntax error 'near "} 1"'



Flags​:
  category=core
  severity=low


Site configuration information for perl 5.22.0​:

Configured by strawberry-perl at Mon Jun 1 20​:06​:45 2015.

Summary of my perl5 (revision 5 version 22 subversion 0) configuration​:

  Platform​:
  osname=MSWin32, osvers=6.3, archname=MSWin32-x86-multi-thread-64int
  uname='Win32 strawberry-perl 5.22.0.1 #1 Mon Jun 1 20​:04​:50 2015 i386'
  config_args='undef'
  hint=recommended, useposix=true, d_sigaction=undef
  useithreads=define, usemultiplicity=define
  use64bitint=define, use64bitall=undef, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='gcc', ccflags =' -s -O2 -DWIN32 -DPERL_TEXTMODE_SCRIPTS
-DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -fwrapv -fno-strict-aliasing
-mms-bitfields',
  optimize='-s -O2',
  cppflags='-DWIN32'
  ccversion='', gccversion='4.9.2', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678,
doublekind=3
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8,
longdblkind=3
  ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='long
long', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='g++', ldflags ='-s -L"C​:\STRAWB1\perl\lib\CORE"
-L"C​:\STRAWB
1\c\lib"'
  libpth=C​:\STRAWB1\c\lib C​:\STRAWB1\c\i686-w64-mingw32\lib
C​:\STRAWB1\c\lib\gcc\i686-w64-mingw32\4.9.2
  libs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32
-ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr
-lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
  perllibs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32
-ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr
-lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
  libc=, so=dll, useshrplib=true, libperl=libperl522.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_win32.xs, dlext=xs.dll, d_dlsymun=undef, ccdlflags=' '
  cccdlflags=' ', lddlflags='-mdll -s -L"C​:\STRAWB
1\perl\lib\CORE"
-L"C​:\STRAWB~1\c\lib"'


@​INC for perl 5.22.0​:
  C​:/Strawberry/perl/site/lib/MSWin32-x86-multi-thread-64int
  C​:/Strawberry/perl/site/lib
  C​:/Strawberry/perl/vendor/lib
  C​:/Strawberry/perl/lib
  .


Environment for perl 5.22.0​:
  HOME (unset)
  LANG (unset)
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)

PATH=C​:\Windows\system32;C​:\Windows;C​:\Windows\System32\Wbem;C​:\Windows\System32\WindowsPowerShell\v1.0\;C​:\Strawberry\c\bin;C​:\Strawberry\perl\site\bin;C​:\Strawberry\perl\bin;C​:\ffmpeg\bin;C​:\Program
Files (x86)\Git\cmd;C​:\Program Files (x86)\NVIDIA
Corporation\PhysX\Common;C​:\Program Files (x86)\EaseUS\Todo Backup\bin\x64\
  PERL_BADLANG (unset)
  SHELL (unset)

@p5pRT
Copy link
Author

p5pRT commented Jul 1, 2015

From @ilmari

Rob Dixon (via RT) <perlbug-followup@​perl.org> writes​:

There is a strange issue with double-quoted variables when using `map`

This is fine

%h = map \{ $k => 1 \} 1;

but this

%h = map \{ "$k" => 1 \} 1;

throws a syntax error 'near "} 1"'

This is documented in the perlfunc entry for map​:

  { starts both hash references and blocks, so map { ... could be
  either the start of map BLOCK LIST or map EXPR, LIST. Because Perl
  doesn't look ahead for the closing } it has to take a guess at which
  it's dealing with based on what it finds just after the {. Usually
  it gets it right, but if it doesn't it won't realize something is
  wrong until it gets to the } and encounters the missing (or
  unexpected) comma. The syntax error will be reported close to the },
  but you'll need to change something near the { such as using a unary
  + or semicolon to give Perl some help​:

  %hash = map { "\L$_" => 1 } @​array # perl guesses EXPR. wrong
  %hash = map { +"\L$_" => 1 } @​array # perl guesses BLOCK. right
  %hash = map {; "\L$_" => 1 } @​array # this also works
  %hash = map { ("\L$_" => 1) } @​array # as does this
  %hash = map { lc($_) => 1 } @​array # and this.
  %hash = map +( lc($_) => 1 ), @​array # this is EXPR and works!

  %hash = map ( lc($_), 1 ), @​array # evaluates to (1, @​array)

http​://perldoc.perl.org/functions/map.html

--
"The surreality of the universe tends towards a maximum" -- Skud's Law
"Never formulate a law or axiom that you're not prepared to live with
the consequences of." -- Skud's Meta-Law

@p5pRT
Copy link
Author

p5pRT commented Jul 1, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jul 1, 2015

From zefram@fysh.org

Rob Dixon wrote​:

There is a strange issue with double-quoted variables when using `map`

Not specific to map​:

$ perl -le '{ $k => 1 } 1; print "OK"'
OK
$ perl -le '{ "$k" => 1 } 1; print "OK"'
Number found where operator expected at -e line 1, near "} 1"
  (Missing operator before 1?)
syntax error at -e line 1, near "} 1"
Execution of -e aborted due to compilation errors.

The difference in parsing is whether the braced group is interpreted as
a code block (without the quotes) or as a hash constructor expression
(with the quotes). In the map case and the above you're seeking code
block treatment. Here's what happens if you want expression treatment​:

$ perl -le '{ $k => 1 } ^ 1; print "OK"'
syntax error at -e line 1, near "} ^"
Execution of -e aborted due to compilation errors.
$ perl -le '{ "$k" => 1 } ^ 1; print "OK"'
OK

This problem arises anywhere that the next thing can be either a code
block or an expression. In the case of map, the ambiguity arises
because of the "map EXPR, LIST" form. Because the ambiguity resolution
is unavoidably heuristic, it's not a bug that it didn't DWYM in a
particular case. The idiomatic solution is to disambiguate yourself,
by adding a semicolon after the opening brace for a block, or by adding
a plus before the opening brace of a hash constructor​:

$ perl -le '{; $k => 1 } 1; print "OK"'
OK
$ perl -le '{; "$k" => 1 } 1; print "OK"'
OK
$ perl -le '+{ $k => 1 } ^ 1; print "OK"'
OK
$ perl -le '+{ "$k" => 1 } ^ 1; print "OK"'
OK

-zefram

@p5pRT
Copy link
Author

p5pRT commented Jul 1, 2015

From cm.perl@abtela.com

Le 01/07/2015 17​:50, Rob Dixon (via RT) a écrit :

There is a strange issue with double-quoted variables when using `map`

This is fine

 %h = map \{ $k => 1 \} 1;

but this

 %h = map \{ "$k" => 1 \} 1;

throws a syntax error 'near "} 1"'

Not a bug IMO.

This is because of the inherent ambiguity of { ... } as either an
expression (a hasref constructor) or a block, in relation to the two
possible forms for map : map expr, list or map block list.

Perl does not always succeed in guessing the intended nature of the
braces. In the second case above it guesses wrong (expression), and
throws an error because of the missing comma.

You can force the block interpretation by adding a semicolumn after the
opening brace :

$ perl -MData​::Dump -E'%h = map {; "$k" => 1 } 1; dd \%h';
{ "" => 1 }
$

--Christian

@p5pRT
Copy link
Author

p5pRT commented Jul 1, 2015

From perl5-porters@perl.org

Christian Millour wrote​:

Not a bug IMO.

No, but it is a fixable misfeature.

@p5pRT
Copy link
Author

p5pRT commented Jul 1, 2015

From zefram@fysh.org

Father Chrysostomos wrote​:

No, but it is a fixable misfeature.

It's impossible to always disambiguate correctly, where the ambiguity
arises. The parser needs to commit to whether the brace is opening a
nested lexical scope before arbitrary expressions inside the braces
can be parsed, and the procedural aspects of parsing generally mean
that parsing of things inside the braces can't be deferred or rerun.
It's thus unavoidable in the general case to make a decision on the
opening brace long before the closing brace comes into sight.

What we *can* improve is the error message. We could tag the braced
group to indicate the use of heuristics, and in the event of syntax
error just after the closing brace this could trigger the addition of
helpful text to the error message. Something like "Might be a code block
misinterpreted as a hash constructor", akin to the "runaway multi-line
string" diagnostic. perldiag could then advise about disambiguation.

The other fixing that's conceivable is to remove some cases of the
ambiguity. We could perhaps deprecate "map EXPR, LIST", so that
eventually "map {" will be unambiguously parsed as a code block. Or,
less disruptively, make the availability of "map EXPR, LIST" subject to
a feature flag, with that syntax disabled by the feature bundles for
sufficiently high perl versions. This would only address map and the
like; there's no getting away from the ambiguity in statement-starting
position.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Jul 1, 2015

From @rjbs

* Zefram <zefram@​fysh.org> [2015-07-01T16​:01​:09]

The other fixing that's conceivable is to remove some cases of the
ambiguity. We could perhaps deprecate "map EXPR, LIST", so that
eventually "map {" will be unambiguously parsed as a code block. Or,
less disruptively, make the availability of "map EXPR, LIST" subject to
a feature flag, with that syntax disabled by the feature bundles for
sufficiently high perl versions. This would only address map and the
like; there's no getting away from the ambiguity in statement-starting
position.

I was just wondering whether anybody but rjbs would benefit from something like
that. I never use the EXPR form, and always write "map {; ... } LIST". Would
I be willing to throw "no ambiguous 'exprblockmapgrep'" in my boilerplate
policy module? Maybe, but I am already used to typing this.

Would we want to enforce it on new users? Well, I'd put it in the style guide
of any code base I managed, but it seems like it would lead to huge despair, or
at least annoyance, among many, without really justifying the cost. Of course,
maybe I am wrong and absolutely nobody wants to use the EXPR form ever anymore.

Anyway, I think that a better diagnostic on these cases would be just great.

--
rjbs

@p5pRT
Copy link
Author

p5pRT commented Jul 1, 2015

From @Abigail

On Wed, Jul 01, 2015 at 04​:17​:03PM -0400, Ricardo Signes wrote​:

* Zefram <zefram@​fysh.org> [2015-07-01T16​:01​:09]

The other fixing that's conceivable is to remove some cases of the
ambiguity. We could perhaps deprecate "map EXPR, LIST", so that
eventually "map {" will be unambiguously parsed as a code block. Or,
less disruptively, make the availability of "map EXPR, LIST" subject to
a feature flag, with that syntax disabled by the feature bundles for
sufficiently high perl versions. This would only address map and the
like; there's no getting away from the ambiguity in statement-starting
position.

I was just wondering whether anybody but rjbs would benefit from something like
that. I never use the EXPR form, and always write "map {; ... } LIST". Would
I be willing to throw "no ambiguous 'exprblockmapgrep'" in my boilerplate
policy module? Maybe, but I am already used to typing this.

I almost always use "map {BLOCK} LIST" as well, unless I really need
the speed of map EXPR, LIST [1]. But I get tripped by the heuristics
getting it wrong in so few cases, that I'm not writing "map {;" by
default. [2]

Would we want to enforce it on new users? Well, I'd put it in the style guide
of any code base I managed, but it seems like it would lead to huge despair, o
at least annoyance, among many, without really justifying the cost. Of course
maybe I am wrong and absolutely nobody wants to use the EXPR form ever anymore

I've encountered enough people who prefer "map EXPR, LIST" over
"map {BLOCK} LIST" that I think that in general, such a style guide
isn't going to fly. Deprecating "map EXPR, LIST" is, IMO, not worth
the hassle.

Anyway, I think that a better diagnostic on these cases would be just great.

That I fully support.

[1] At $WORK we used to have hot code where this difference mattered --
  up to the point we were replacing foreach loops with maps (and greps!)
  in void context. (And at that point, it made sense to spend a lot
  of resources to just preprocess every possible input).
[2] People have claimed I've invented the "map {;" idiom, and that I
  always write "map {;" -- neither is true. (I learned the idiom from
  Larry Rossler).

Abigail

@p5pRT
Copy link
Author

p5pRT commented Jul 2, 2015

From @ap

* Zefram <zefram@​fysh.org> [2015-07-01 22​:05]​:

What we *can* improve is the error message. We could tag the braced
group to indicate the use of heuristics, and in the event of syntax
error just after the closing brace this could trigger the addition of
helpful text to the error message. Something like "Might be a code
block misinterpreted as a hash constructor", akin to the "runaway
multi-line string" diagnostic. perldiag could then advise about
disambiguation.

Yes please! That would be excellent.

* Ricardo Signes <perl.p5p@​rjbs.manxome.org> [2015-07-01 22​:20]​:

I never use the EXPR form, and always write "map {; ... } LIST". […]
Of course, maybe […] absolutely nobody wants to use the EXPR form ever
anymore.

I do prefer `map BLOCK` but still use `map EXPR` liberally.

If I’m mapping to a short expression, the extra braces feel like an
overdose of line noise. But then I also drop a lot more parentheses
on function calls than many people seem to, so… e.g. somewhere I’ve
got a `map [ split ' ' ], ...` that I wouldn’t want to write in any
other way, it’s so clear and *concise*. Adding all the boilerplate
delimiters and verbiage to make it explicit would just detract from
its readability, not help.

I was going to say I still use `map BLOCK` far more, but then I ran
a few greps and it turns out the ratio is closer to 2​:1 in its favour,
not nearly as lopsided as my impression. Then again that includes some
codebases I’ve been working on for a long time, and my inclination has
been tilting for some time, so it’s possible that is indeed far more
lopsided now, though likely still less so than I thought.

Also, I knew I don’t use the `map {;` disambiguation all the time, but
wasn’t sure when I do. Turns out I do that only when I’m writing a map
of the form `map {; TERM, TERM }` (with either lean or fat comma). Makes
sense​: I can’t be arsed to memorise the rules for that heuristic, so
I forcibly suppress it in the one case it matters. Hooray for laziness.

But, conversely, when I do mean to map each list element to a hash,
I *always* use the `map +{ ... }, LIST` form for that. I think that is
helpful not simply to hint-hint perl but actually important for human
readers, because it explicitly codifies intent. (Besides the fact that
while humans may be able to employ lookahead far better than perl can,
that doesn’t mean their capacity is unlimited, or that it doesn’t slow
them down.)

Obviously I would never object to some kind of optional `no mapexpr`
pragma for those who want the discipline. I don’t even know that I’d
object to its eventual defaultness, but it would probably dismay me
at the present time.

Regards,
--
Aristotle Pagaltzis // <http​://plasmasturm.org/>

@p5pRT
Copy link
Author

p5pRT commented Jul 2, 2015

From @epa

You could tweak the DWIMmery slightly so that a map call is always assumed
to be of the BLOCK kind if the first argument is of form {..}. Mapping
an anonymous hash constructor over a list must be pretty rare.

These (not-a-)bugs come up quite often.

--
Ed Avis <eda@​waniasset.com>

@p5pRT
Copy link
Author

p5pRT commented Jul 2, 2015

From @Abigail

On Thu, Jul 02, 2015 at 07​:21​:59AM +0000, Ed Avis wrote​:

You could tweak the DWIMmery slightly so that a map call is always assumed
to be of the BLOCK kind if the first argument is of form {..}. Mapping
an anonymous hash constructor over a list must be pretty rare.

Changing the current behaviour runs the risk of breaking existing
code, while the gain is every now and then someone doesn't get an
easy to fix compilation error.

It's a trade-off I prefer not to change.

These (not-a-)bugs come up quite often.

Abigail

@p5pRT
Copy link
Author

p5pRT commented Jul 2, 2015

From cm.perl@abtela.com

Le 01/07/2015 22​:44, Abigail a écrit :

[1] At $WORK we used to have hot code where this difference mattered --
up to the point we were replacing foreach loops with maps (and greps!)
in void context. (And at that point, it made sense to spend a lot
of resources to just preprocess every possible input).

Interesting. I was led to believe that postfix foreach was faster than
block foreach, itself faster that expression map in void context, with
block map in v.c. the slowest of all. A trivial benchmark seems to
validate this, for both true lists

$ /usr/local/bin/perl -MBenchmark=timethese,cmpthese -E'
@​a = 1 .. 100_000;
$r = timethese(1000, {
  lpfor => sub { my $x; $x += $_ for @​a },
  lbfor => sub { my $x; for (@​a) { $x += $_ } },
  lemap => sub { my $x; map +($x += $_), @​a },
  lbmap => sub { my $x; map { $x += $_ } @​a }
} );
cmpthese($r)'
Benchmark​: timing 1000 iterations of lbfor, lbmap, lemap, lpfor...
  lbfor​: 10 wallclock secs ( 9.93 usr + 0.01 sys = 9.94 CPU) @​
100.60/s (n=1000)
  lbmap​: 11 wallclock secs (10.50 usr + 0.00 sys = 10.50 CPU) @​
95.24/s (n=1000)
  lemap​: 10 wallclock secs (10.43 usr + 0.00 sys = 10.43 CPU) @​
95.88/s (n=1000)
  lpfor​: 9 wallclock secs ( 8.28 usr + 0.01 sys = 8.29 CPU) @​
120.63/s (n=1000)
  Rate lbmap lemap lbfor lpfor
lbmap 95.2/s -- -1% -5% -21%
lemap 95.9/s 1% -- -5% -21%
lbfor 101/s 6% 5% -- -17%
lpfor 121/s 27% 26% 20% --
$

and ranges

$ /usr/local/bin/perl -MBenchmark=timethese,cmpthese -E'
$r = timethese(1000, {
  rpfor => sub { my $x; $x += $_ for 1 .. 100_000 },
  rbfor => sub { my $x; for (1 .. 100_000) { $x += $_ } },
  remap => sub { my $x; map +($x += $_), 1 .. 100_000 },
  rbmap => sub { my $x; map { $x += $_ } 1 .. 100_000 }
} );
cmpthese($r)'
Benchmark​: timing 1000 iterations of rbfor, rbmap, remap, rpfor...
  rbfor​: 10 wallclock secs ( 9.78 usr + 0.00 sys = 9.78 CPU) @​
102.25/s (n=1000)
  rbmap​: 16 wallclock secs (15.30 usr + 0.00 sys = 15.30 CPU) @​
65.36/s (n=1000)
  remap​: 15 wallclock secs (15.50 usr + 0.00 sys = 15.50 CPU) @​
64.52/s (n=1000)
  rpfor​: 9 wallclock secs ( 8.45 usr + 0.00 sys = 8.45 CPU) @​
118.34/s (n=1000)
  Rate remap rbmap rbfor rpfor
remap 64.5/s -- -1% -37% -45%
rbmap 65.4/s 1% -- -36% -45%
rbfor 102/s 58% 56% -- -14%
rpfor 118/s 83% 81% 16% --
$

I have experimented with list sizes ranging from 10 to 100_000 with
rpfor and lpfor always the fastest. So what am I overlooking here ?

TIA,

--Christian,

(FWIW, the benchmarks above were run on an old but very lightly used
server, with the following perl :)

$ /usr/local/bin/perl -V
Summary of my perl5 (revision 5 version 22 subversion 0) configuration​:

  Platform​:
  osname=linux, osvers=2.6.18-53.1.19.el5.028stab053.14,
archname=i686-linux
  uname='linux immoouest_server1.abtela.com
2.6.18-53.1.19.el5.028stab053.14 #1 smp thu may 8 20​:43​:27 msd 2008 i686
i686 i386 gnulinux '
  config_args='-de'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  use64bitint=undef, use64bitall=undef, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-fno-strict-aliasing -pipe -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2',
  cppflags='-fno-strict-aliasing -pipe -I/usr/local/include'
  ccversion='', gccversion='3.4.6 20060404 (Red Hat 3.4.6-9)',
gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234,
doublekind=3
  d_longlong=define, longlongsize=8, d_longdbl=define,
longdblsize=12, longdblkind=3
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=4, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib /lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.3.4.so, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.3.4'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib'

Characteristics of this binary (from libperl)​:
  Compile-time options​: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
  PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
  USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME
  USE_PERLIO USE_PERL_ATOF
  Built under linux
  Compiled at Jun 8 2015 00​:47​:23
  @​INC​:
  /usr/local/lib/perl5/site_perl/5.22.0/i686-linux
  /usr/local/lib/perl5/site_perl/5.22.0
  /usr/local/lib/perl5/5.22.0/i686-linux
  /usr/local/lib/perl5/5.22.0
  .
$

@p5pRT
Copy link
Author

p5pRT commented Jul 2, 2015

From @Abigail

On Thu, Jul 02, 2015 at 01​:44​:35PM +0200, Christian Millour wrote​:

Le 01/07/2015 22​:44, Abigail a écrit :

[1] At $WORK we used to have hot code where this difference mattered --
up to the point we were replacing foreach loops with maps (and greps!)
in void context. (And at that point, it made sense to spend a lot
of resources to just preprocess every possible input).

Interesting. I was led to believe that postfix foreach was faster than
block foreach, itself faster that expression map in void context, with
block map in v.c. the slowest of all. A trivial benchmark seems to
validate this, for both true lists

[Snip]

I have experimented with list sizes ranging from 10 to 100_000 with
rpfor and lpfor always the fastest. So what am I overlooking here ?

All you have proven is that for this particular expression, on your
particular box, with your version of perl, compiled with your specific
set of compilation parameters, rbfor and rpfor are, in a tiny benchmark,
faster. On top of that, you used Benchmark.pm. (You have lies, damned
lies, statistics, and then Benchmark.pm).

Things may be different if you have a different OS, a different version
of Perl, run on a heavy utilized box, or use a different expression.

If it matters, always benchmark what you're actually doing, under real
operating conditions.

Abigail

@p5pRT
Copy link
Author

p5pRT commented Jul 2, 2015

From cm.perl@abtela.com

Le 02/07/2015 13​:58, Abigail a écrit :

On Thu, Jul 02, 2015 at 01​:44​:35PM +0200, Christian Millour wrote​:

Le 01/07/2015 22​:44, Abigail a écrit :

[1] At $WORK we used to have hot code where this difference mattered --
up to the point we were replacing foreach loops with maps (and greps!)
in void context. (And at that point, it made sense to spend a lot
of resources to just preprocess every possible input).

Interesting. I was led to believe that postfix foreach was faster than
block foreach, itself faster that expression map in void context, with
block map in v.c. the slowest of all. A trivial benchmark seems to
validate this, for both true lists

[Snip]

I have experimented with list sizes ranging from 10 to 100_000 with
rpfor and lpfor always the fastest. So what am I overlooking here ?

All you have proven is that for this particular expression, on your
particular box, with your version of perl, compiled with your specific
set of compilation parameters, rbfor and rpfor are, in a tiny benchmark,
faster. On top of that, you used Benchmark.pm. (You have lies, damned
lies, statistics, and then Benchmark.pm).

To be honest I am not familiar with Benchmark.pm. I tried to mitigate
any vagaries by using both timethese (to get and check the wallclock
seconds reported) and compthese, with enough iterations to balance any
temporary external load, on a lightly used server; I varied the list
size to account for possible paging issues; I chose a very simple
expression on purpose; all that in the hope that it would better
showcase the time spent in the iteration scaffolding for each looping
method. Maybe that was misdirected.

Things may be different if you have a different OS, a different version
of Perl, run on a heavy utilized box, or use a different expression.

Definitely, and my question was in no way a rebuttal or your own
experience. It is just that over the years, with various versions of
perl (mostly unthreaded), running on solid silicon or in virtual
machines, under linux or windows, and when I bothered if at all to try
and measure such stuff (with context-relevant metrics, such as actual
response time or number of documents processed per second), postfix
foreach always finished first. I was just curious about the set of
conditions under which that might not be true.

This has impacted my programming style : I tend to favor postfix
foreach, and use the topic $_ rather than declaring an iteration
variable. I'll freely admit to being a bit insecure about both, and so
welcome arguments as well as hard facts to broaden my views...

If it matters, always benchmark what you're actually doing, under real
operating conditions.

amen to that :-). Cheers,

--Christian

Abigail

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants