Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Still "keys" performance regression between 5.10.1 and 5.12.3-RC1 #11028

Closed
p5pRT opened this issue Jan 11, 2011 · 23 comments
Closed

Still "keys" performance regression between 5.10.1 and 5.12.3-RC1 #11028

p5pRT opened this issue Jan 11, 2011 · 23 comments

Comments

@p5pRT
Copy link

p5pRT commented Jan 11, 2011

Migrated from rt.perl.org#82110 (status was 'resolved')

Searchable as RT82110$

@p5pRT
Copy link
Author

p5pRT commented Jan 11, 2011

From btik-fbsd@scoubidou.com

Created by btik-fbsd@scoubidou.com

There is a performance regression since 5.10.1 when using core keys
function on dereferenced hashes.

my $hash = { 1 .. 1000 };
my %hash = ( 1 .. 1000 );

my $count = -10;
cmpthese( $count, {
  'keys %$hash' => sub { my @​array = keys %$hash },
  'keys %hash' => sub { my @​array = keys %hash },
  });

On the same host, FreeBSD 8.1-RELEASE amd64​:

* with perl 5.10.1 (non threaded perl, FreeBSD ports version)
  Rate keys %hash keys %$hash
keys %hash 6990/s -- -0%
keys %$hash 7021/s 0% --

* with last perl 5.12.3-RC1
  Rate keys %$hash keys %hash
keys %$hash 5393/s -- -27%
keys %hash 7396/s 37% --

Perl Info

Flags:
    category=core
    severity=medium

This perlbug was built using Perl 5.12.3 - Tue Jan 11 17:55:15 CET 2011
It is being executed now by  Perl 5.10.1 - Thu Nov 25 11:37:11 CET 2010.

Site configuration information for perl 5.10.1:

Configured by max at Thu Nov 25 11:37:11 CET 2010.

Summary of my perl5 (revision 5 version 10 subversion 1) configuration:
   
  Platform:
    osname=freebsd, osvers=8.1-release, archname=amd64-freebsd
    uname='freebsd auber.mobigard.com 8.1-release freebsd 8.1-release #0: mon jul 19 02:36:49 utc 2010 root@mason.cse.buffalo.edu:usrobjusrsrcsysgeneric amd64 '
    config_args='-sde -Dprefix=/usr/local -Darchlib=/usr/local/lib/perl5/5.10.1/mach -Dprivlib=/usr/local/lib/perl5/5.10.1 -Dman3dir=/usr/local/lib/perl5/5.10.1/perl/man/man3 -Dman1dir=/usr/local/man/man1 -Dsitearch=/usr/local/lib/perl5/site_perl/5.10.1/mach -Dsitelib=/usr/local/lib/perl5/site_perl/5.10.1 -Dscriptdir=/usr/local/bin -Dsiteman3dir=/usr/local/lib/perl5/5.10.1/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Ui_malloc -Ui_iconv -Uinstallusrbinperl -Dcc=cc -Duseshrplib -Dinc_version_list=none -Dccflags=-DAPPLLIB_EXP="/usr/local/lib/perl5/5.10.1/BSDPAN" -Doptimize=-O2 -pipe -march=nocona -fno-strict-aliasing -Ud_dosuid -Ui_gdbm -Dusethreads=n -Dusemymalloc=n -Duse64bitint'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.10.1/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include',
    optimize='-O2 -pipe -march=nocona -fno-strict-aliasing',
    cppflags='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.10.1/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.2.1 20070719  [FreeBSD]', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -Wl,-E  -fstack-protector -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib
    libs=-lgdbm -lm -lcrypt -lutil
    perllibs=-lm -lcrypt -lutil
    libc=, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='  -Wl,-R/usr/local/lib/perl5/5.10.1/mach/CORE'
    cccdlflags='-DPIC -fPIC', lddlflags='-shared  -L/usr/local/lib -fstack-protector'

Locally applied patches:
    RC1


@INC for perl 5.10.1:
    /usr/local/lib/perl5/5.10.1/BSDPAN
    /usr/local/lib/perl5/site_perl/5.10.1/mach
    /usr/local/lib/perl5/site_perl/5.10.1
    /usr/local/lib/perl5/5.10.1/mach
    /usr/local/lib/perl5/5.10.1
    .


Environment for perl 5.10.1:
    HOME=/home/max
    LANG=fr_FR.ISO8859-1
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin:/home/max/bin:/home/max/.madde/0.6.14/bin
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Jan 11, 2011

From btik-fbsd@scoubidou.com

Note that this problem affects 5.12.3-RC1, not 5.10.1.

I did the benchmark without installing 5.12.3, and called perlbug as
./utils/perlbug but it took the installed version (5.10.1) to make the
report...

Sorry...

@p5pRT
Copy link
Author

p5pRT commented Jan 11, 2011

btik-fbsd@scoubidou.com - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jan 11, 2011

From btik-fbsd@scoubidou.com

Just created a new report specific to 5.12.3-RC1.

Please, forget this one and sorry for the noise.

@p5pRT
Copy link
Author

p5pRT commented Jan 11, 2011

From btik-fbsd@scoubidou.com

This is a bug report for perl from btik-fbsd@​scoubidou.com,
generated with the help of perlbug 1.39 running under perl 5.12.3.


There is a performance regression since 5.10.1 when using core keys
function on dereferenced hashes.

use strict;
use warnings;
use Benchmark qw(​:all);

my $hash = { 1 .. 1000 };
my %hash = ( 1 .. 1000 );

my $count = -10;
cmpthese( $count, {
  'keys %$hash' => sub { my @​array = keys %$hash },
  'keys %hash' => sub { my @​array = keys %hash },
});

On the same host, FreeBSD 8.1-RELEASE amd64​:

* with perl 5.10.1 (non threaded perl, FreeBSD ports version)
  Rate keys %hash keys %$hash
keys %hash 6990/s -- -0%
keys %$hash 7021/s 0% --

* with last perl 5.12.3-RC1
  Rate keys %hash keys %$hash
keys %$hash 5393/s -- -27%
keys %hash 7396/s 37% --

Max.



Flags​:
  category=core
  severity=medium


Site configuration information for perl 5.12.3​:

Configured by max at Tue Jan 11 22​:58​:26 CET 2011.

Summary of my perl5 (revision 5 version 12 subversion 3) configuration​:
 
  Platform​:
  osname=freebsd, osvers=8.1-release, archname=amd64-freebsd
  uname='freebsd zetta.scoubidou.com 8.1-release freebsd 8.1-release #2​: thu aug 19 23​:09​:12 cest 2010 root@​mfsbsd​:usrobjusrsrcsyszetta amd64 '
  config_args='-des -Dprefix=/home/max/local/perl'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include',
  optimize='-O',
  cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.2.1 20070719 [FreeBSD]', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags ='-Wl,-E -fstack-protector -L/usr/local/lib'
  libpth=/usr/lib /usr/local/lib
  libs=-lgdbm -lm -lcrypt -lutil -lc
  perllibs=-lm -lcrypt -lutil -lc
  libc=, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
  cccdlflags='-DPIC -fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector'

Locally applied patches​:
  RC1


@​INC for perl 5.12.3​:
  /home/max/local/perl/lib/site_perl/5.12.3/amd64-freebsd
  /home/max/local/perl/lib/site_perl/5.12.3
  /home/max/local/perl/lib/5.12.3/amd64-freebsd
  /home/max/local/perl/lib/5.12.3
  .


Environment for perl 5.12.3​:
  HOME=/root
  LANG=fr_FR.ISO8859-15
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/sbin​:/bin​:/usr/sbin​:/usr/bin​:/usr/local/bin​:/usr/X11R6/bin​:/usr/local/sbin​:/usr/local/pilot/bin​:/root/bin​:/root/bin
  PERL_BADLANG (unset)
  SHELL=/bin/csh

@p5pRT
Copy link
Author

p5pRT commented Jan 12, 2011

From @nwc10

On Tue, Jan 11, 2011 at 03​:13​:40PM -0800, btik-fbsd @​ scoubidou. com wrote​:

There is a performance regression since 5.10.1 when using core keys
function on dereferenced hashes.

There is a performance regression, but it's not directly to do with
dereferenced hashes.

If I take your example​:

use strict;
use warnings;
use Benchmark qw(​:all);

my $hash = { 1 .. 1000 };
my %hash = ( 1 .. 1000 );

my $count = -10;
cmpthese( $count, {
'keys %$hash' => sub { my @​array = keys %$hash },
'keys %hash' => sub { my @​array = keys %hash },
});

I can replicate it​:

$ cat 82100.pl
use strict;
use warnings;
use Benchmark qw(​:all);

my $hash = { 1 .. 1000 };
my %hash = ( 1 .. 1000 );

my $count = -10;
cmpthese( $count, {
  'keys %$hash' => sub { my @​array = keys %$hash },
  'keys %hash' => sub { my @​array = keys %hash },
});
$ ~/Sandpit/5101/bin/perl5.10.1 82100.pl
  Rate keys %hash keys %$hash
keys %hash 6949/s -- -1%
keys %$hash 7031/s 1% --
$ ~/Sandpit/5123-RC1/bin/perl5.12.3 82100.pl
  Rate keys %$hash keys %hash
keys %$hash 5013/s -- -26%
keys %hash 6776/s 35% --

If I switch from lexical to package variables​:

$ cat 82100p.pl
use strict;
use warnings;
use Benchmark qw(​:all);

my $hash = { 1 .. 1000 };
my %hash = ( 1 .. 1000 );

my $count = -10;
cmpthese( $count, {
  'keys %$hash' => sub { @​​::array = keys %$hash },
  'keys %hash' => sub { @​​::array = keys %hash },
});
$ ~/Sandpit/5101/bin/perl5.10.1 82100p.pl
  Rate keys %$hash keys %hash
keys %$hash 5221/s -- -26%
keys %hash 7098/s 36% --
$ ~/Sandpit/5123-RC1/bin/perl5.12.3 82100p.pl
  Rate keys %$hash keys %hash
keys %$hash 4975/s -- -26%
keys %hash 6719/s 35% --

there's no change. So it can't directly be anything to do with the use of
dereferenced hashes.

I believe that the speed change is actually due to the removal of an
overly-aggressive optimisation in the list assignment code (in change
0f907b9, fixes bug #70171, over-optimisation added in
fafafba)

I'm not sure what can be done about that. The optimisation code for list
assignment is self contained - it isn't aware that it's in the same statement
as the declaration lexicals.

Nicholas Clark

@p5pRT
Copy link
Author

p5pRT commented Jan 12, 2011

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jan 12, 2011

From @iabyn

On Tue, Jan 11, 2011 at 09​:21​:32AM -0800, btik-fbsd@​scoubidou.com wrote​:

my $hash = { 1 .. 1000 };
my %hash = ( 1 .. 1000 );

my $count = -10;
cmpthese( $count, {
'keys %$hash' => sub { my @​array = keys %$hash },
'keys %hash' => sub { my @​array = keys %hash },
});

On the same host, FreeBSD 8.1-RELEASE amd64​:

* with perl 5.10.1 (non threaded perl, FreeBSD ports version)
Rate keys %hash keys %$hash
keys %hash 6990/s -- -0%
keys %$hash 7021/s 0% --

* with last perl 5.12.3-RC1
Rate keys %$hash keys %hash
keys %$hash 5393/s -- -27%
keys %hash 7396/s 37% --

This is due to the COMMON-flag pessimisation of the array assignment op
in C< @​array = keys %$hash>.
This flag gets set in​:

  5.10.0
  5.12.x
  5.13.x

and *doesn't* get set in

  5.8.x
  5.10.1

so 5.10.1 (and 5.8.x) are faster.

Now, I remember there was a bug related to the COMMON flag on 5.10.0 that
made C<my ($x,$y) = @​_> slow, and that bug was fixed in 5.10.1, but also
in 5.12.0 onwards, so I'm a bit bemused how this new issue got fixed in
5.10.1 but not 5.12.0 onwards.

Anyway, its not a regression from any earlier 5.12.x, so isn't a blocker
for 5.12.3.

--
Monto Blanco... scorchio!

@p5pRT
Copy link
Author

p5pRT commented Jan 16, 2011

From @cpansprout

On Wed Jan 12 06​:44​:31 2011, davem wrote​:

On Tue, Jan 11, 2011 at 09​:21​:32AM -0800, btik-fbsd@​scoubidou.com wrote​:

my $hash = { 1 .. 1000 };
my %hash = ( 1 .. 1000 );

my $count = -10;
cmpthese( $count, {
'keys %$hash' => sub { my @​array = keys %$hash },
'keys %hash' => sub { my @​array = keys %hash },
});

On the same host, FreeBSD 8.1-RELEASE amd64​:

* with perl 5.10.1 (non threaded perl, FreeBSD ports version)
Rate keys %hash keys %$hash
keys %hash 6990/s -- -0%
keys %$hash 7021/s 0% --

* with last perl 5.12.3-RC1
Rate keys %$hash keys %hash
keys %$hash 5393/s -- -27%
keys %hash 7396/s 37% --

This is due to the COMMON-flag pessimisation of the array assignment op
in C< @​array = keys %$hash>.
This flag gets set in​:

5\.10\.0
5\.12\.x
5\.13\.x

and *doesn't* get set in

5\.8\.x
5\.10\.1

so 5.10.1 (and 5.8.x) are faster.

Now, I remember there was a bug related to the COMMON flag on 5.10.0 that
made C<my ($x,$y) = @​_> slow, and that bug was fixed in 5.10.1,

by change fafafba, which caused bug #70171.

but also
in 5.12.0 onwards, so I'm a bit bemused how this new issue got fixed in
5.10.1 but not 5.12.0 onwards.

Probably because #70171 was fixed by 0f907b9, which got into 5.12.

An expression like ‘my @​array = ...’ might also reference @​array on the
right hand side (if named subroutine closes over @​array and its return
value is on the rhs), so the no-common-vars optimisation is disabled for
‘my @​array =’.

If the RHS has a hash dereference, keys %hash or similar, we could
re-enable the no-common-vars optimisation. But that part of the code is
already scarily complex as it is. :-)

Anyway, its not a regression from any earlier 5.12.x, so isn't a blocker
for 5.12.3.

@p5pRT
Copy link
Author

p5pRT commented Jan 18, 2011

From @iabyn

On Sun, Jan 16, 2011 at 01​:28​:11PM -0800, Father Chrysostomos via RT wrote​:

An expression like ‘my @​array = ...’ might also reference @​array on the
right hand side (if named subroutine closes over @​array and its return
value is on the rhs), so the no-common-vars optimisation is disabled for
‘my @​array =’.

But we should be able to tell whether the lexical is closed over by its
reference count​:

$ p -MDevel​::Peek -we'my $x; Dump $x'
ALLOCATED at -e​:1 for padsv (parent 0x0); serial 1763
SV = NULL(0x0) at 0x2171f80
  REFCNT = 1
  FLAGS = (PADMY)

$ p -MDevel​::Peek -we'my $x; Dump $x; sub f {$x+1}'
ALLOCATED at -e​:1 for padsv (parent 0x0); serial 1763
SV = NULL(0x0) at 0x283ef90
  REFCNT = 2
  FLAGS = (PADMY)

--
Nothing ventured, nothing lost.

@p5pRT
Copy link
Author

p5pRT commented Feb 6, 2011

From @cpansprout

On Tue Jan 18 06​:53​:20 2011, davem wrote​:

On Sun, Jan 16, 2011 at 01​:28​:11PM -0800, Father Chrysostomos via RT
wrote​:

An expression like ‘my @​array = ...’ might also reference @​array on the
right hand side (if named subroutine closes over @​array and its return
value is on the rhs), so the no-common-vars optimisation is disabled for
‘my @​array =’.

But we should be able to tell whether the lexical is closed over by its
reference count​:

$ p -MDevel​::Peek -we'my $x; Dump $x'
ALLOCATED at -e​:1 for padsv (parent 0x0); serial 1763
SV = NULL(0x0) at 0x2171f80
REFCNT = 1
FLAGS = (PADMY)

$ p -MDevel​::Peek -we'my $x; Dump $x; sub f {$x+1}'
ALLOCATED at -e​:1 for padsv (parent 0x0); serial 1763
SV = NULL(0x0) at 0x283ef90
REFCNT = 2
FLAGS = (PADMY)

At compile time, the OPpASSIGN_COMMON flag is set when the list
assignment is compiled. If the list assignment comes before the
subroutine that closes over the lexical, it’s not possible to know at
that time.
It’s not possible to use a reference count in pp_aassign, either,
because the stack is not reference-counted. (I tried that and got an
incredible speed-up, but ($a,$b)=($b,$a) stopped working.)

@p5pRT
Copy link
Author

p5pRT commented Mar 10, 2011

From @iabyn

On Sun, Feb 06, 2011 at 01​:07​:25PM -0800, Father Chrysostomos via RT wrote​:

On Tue Jan 18 06​:53​:20 2011, davem wrote​:

On Sun, Jan 16, 2011 at 01​:28​:11PM -0800, Father Chrysostomos via RT
wrote​:

An expression like ‘my @​array = ...’ might also reference @​array on the
right hand side (if named subroutine closes over @​array and its return
value is on the rhs), so the no-common-vars optimisation is disabled for
‘my @​array =’.

But we should be able to tell whether the lexical is closed over by its
reference count​:

$ p -MDevel​::Peek -we'my $x; Dump $x'
ALLOCATED at -e​:1 for padsv (parent 0x0); serial 1763
SV = NULL(0x0) at 0x2171f80
REFCNT = 1
FLAGS = (PADMY)

$ p -MDevel​::Peek -we'my $x; Dump $x; sub f {$x+1}'
ALLOCATED at -e​:1 for padsv (parent 0x0); serial 1763
SV = NULL(0x0) at 0x283ef90
REFCNT = 2
FLAGS = (PADMY)

At compile time, the OPpASSIGN_COMMON flag is set when the list
assignment is compiled. If the list assignment comes before the
subroutine that closes over the lexical, it’s not possible to know at
that time.
It’s not possible to use a reference count in pp_aassign, either,
because the stack is not reference-counted. (I tried that and got an
incredible speed-up, but ($a,$b)=($b,$a) stopped working.)

And thinking further, you can get this behaviour without a closure too​:

  my $r;
  again​:
  my @​x = @​$r; # *** common assignment on 2nd attempt!
  @​x = (1,2,3);
  $r = \@​x;
  goto again unless $i++;

So I think this assignment is correctly pessimised, and shouldn't
be regarded as a bug.

I'll close the ticket.

--
But Pity stayed his hand. "It's a pity I've run out of bullets",
he thought. -- "Bored of the Rings"

@p5pRT
Copy link
Author

p5pRT commented Mar 11, 2011

From @iabyn

I've added a couple of extra tests with commit
ade1cea

@p5pRT
Copy link
Author

p5pRT commented Mar 11, 2011

@iabyn - Status changed from 'open' to 'rejected'

@p5pRT
Copy link
Author

p5pRT commented Mar 12, 2011

From @demerphq

On 10 March 2011 18​:04, Dave Mitchell <davem@​iabyn.com> wrote​:

On Sun, Feb 06, 2011 at 01​:07​:25PM -0800, Father Chrysostomos via RT wrote​:

On Tue Jan 18 06​:53​:20 2011, davem wrote​:

On Sun, Jan 16, 2011 at 01​:28​:11PM -0800, Father Chrysostomos via RT
wrote​:

An expression like ‘my @​array = ...’ might also reference @​array on the
right hand side (if named subroutine closes over @​array and its return
value is on the rhs), so the no-common-vars optimisation is disabled for
‘my @​array =’.

But we should be able to tell whether the lexical is closed over by its
reference count​:

$ p -MDevel​::Peek -we'my $x; Dump $x'
ALLOCATED at -e​:1 for padsv (parent 0x0); serial 1763
SV = NULL(0x0) at 0x2171f80
  REFCNT = 1
  FLAGS = (PADMY)

$ p -MDevel​::Peek -we'my $x; Dump $x; sub f {$x+1}'
ALLOCATED at -e​:1 for padsv (parent 0x0); serial 1763
SV = NULL(0x0) at 0x283ef90
  REFCNT = 2
  FLAGS = (PADMY)

At compile time, the OPpASSIGN_COMMON flag is set when the list
assignment is compiled. If the list assignment comes before the
subroutine that closes over the lexical, it’s not possible to know at
that time.
It’s not possible to use a reference count in pp_aassign, either,
because the stack is not reference-counted. (I tried that and got an
incredible speed-up, but ($a,$b)=($b,$a) stopped working.)

And thinking further, you can get this behaviour without a closure too​:

   my $r;
 again​:
   my @​x = @​$r; # *** common assignment on 2nd attempt!
   @​x = (1,2,3);
   $r = \@​x;
   goto again unless $i++;

So I think this assignment is correctly pessimised, and shouldn't
be regarded as a bug.

I'll close the ticket.

Am I right in understanding that our position is that a 30% slowdown
in "my @​array= list " is an acceptable cost to make code like this​:

  my $r;
again​:
  my @​x = @​$r; # *** common assignment on 2nd attempt!
  print "@​x\n";
  @​x = (1,2,3);
  $r = \@​x;
  goto again unless $i++;

work correctly and not result in "semi-panic​: attempt to dup freed
string at /tmp/b.pl line 2." type errors?

I personally do not think that this is so clear cut, given that they
can do this​:

  my $r;
again​:
  my @​x;
  @​x = @​$r; # *** common assignment on 2nd attempt!
  print "@​x\n";
  @​x = (1,2,3);
  $r = \@​x;
  goto again unless $i++;

And have it work correctly?

I have not fully understood FC's closure case properly, so maybe this
is an easier trap to fall into than I think, but right now I'm
thinking this is a pessimisation that is much akin to make $1 "safe"
after a //g in scalar context, a case where we decided that simply
saying "don't do that" was better than making every //g match slower.

IOW, I'm not sure that sacrificing performance on a very common
construct to make what seems to me to be a pretty unusual edge case
work properly is a good trade off especially given there is an easy
work around.

While I don't think id go so far as to say we should actually choosing
speed over correctness is right here, I have to admit I'm tempted, and
do think at the very least we should keep this bug open.

cheers,
Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Mar 12, 2011

From @iabyn

On Sat, Mar 12, 2011 at 01​:51​:46PM +0100, demerphq wrote​:

Am I right in understanding that our position is that a 30% slowdown
in "my @​array= list " is an acceptable cost

Note its only these two

  my @​array = @​$arrayref;
  my %hash = %$hashref;

that get the pessimisation, and not the more general versions of

  my @​array = something;
  my %hash = something;

I personally do not think that this is so clear cut, given that they
can do this​:

 my $r;

again​:
my @​x;
@​x = @​$r; # *** common assignment on 2nd attempt!
print "@​x\n";
@​x = (1,2,3);
$r = \@​x;
goto again unless $i++;

And have it work correctly?

I'm not sure what point you're trying to make here. In that code, the
@​x = @​$r is pessimised, and always has been. The only change made in 5.1.20
onwards is that formerly, it was assumed that code starting with a my
declaration, i.e.
  my @​x = ...
could always be optimised, because it was impossible at that point for @​x
to have any elements. FC's closure example, and then my goto example, both
demonstrate that it is possible at the point of executing the PADAV/INTRO,
for @​x to already have gained elements. Thus we now recognise that there
are some cases where this assumption isn't true.

I have not fully understood FC's closure case properly, so maybe this is
an easier trap to fall into than I think, but right now I'm thinking
this is a pessimisation that is much akin to make $1 "safe" after a //g
in scalar context, a case where we decided that simply saying "don't do
that" was better than making every //g match slower.

IOW, I'm not sure that sacrificing performance on a very common
construct to make what seems to me to be a pretty unusual edge case work
properly is a good trade off especially given there is an easy work
around.

I don't understand what you mean by a workaround.

While I don't think id go so far as to say we should actually choosing
speed over correctness is right here, I have to admit I'm tempted, and
do think at the very least we should keep this bug open.

We're not talking speed over correctness here. We're talking speed over
panics and crashing.

--
Diplomacy is telling someone to go to hell in such a way that they'll
look forward to the trip

@p5pRT
Copy link
Author

p5pRT commented Mar 12, 2011

From @lizmat

On Mar 12, 2011, at 9​:58 PM, Dave Mitchell wrote​:

On Sat, Mar 12, 2011 at 01​:51​:46PM +0100, demerphq wrote​:

Am I right in understanding that our position is that a 30% slowdown
in "my @​array= list " is an acceptable cost

Note its only these two

my @​array = @​$arrayref;
my %hash = %$hashref;

that get the pessimisation,

FWIW, in the cases where I've used this construct, it was for creating local copies of a parameter hash. On the one hand, that would always be small hashes. If such a subroutine gets called a lot, it would probably make sense to change the input parameters anyway.

Since there is almost no performance difference between $foo[...] / $foo{...} and $foo->[...] / $foo->{...}, I don't see a point why anyone would need to make copies just for that reason. If there are other reasons you need a copy, then I guess you will have to pay for that.

and not the more general versions of

my @​array = something;
my %hash = something;

Good to know. :-)

Liz

@p5pRT
Copy link
Author

p5pRT commented Mar 12, 2011

From @demerphq

On 12 March 2011 21​:58, Dave Mitchell <davem@​iabyn.com> wrote​:

On Sat, Mar 12, 2011 at 01​:51​:46PM +0100, demerphq wrote​:

Am I right in understanding that our position is that a 30% slowdown
in "my @​array= list " is an acceptable cost

Note its only these two

   my @​array = @​$arrayref;
   my %hash  = %$hashref;

that get the pessimisation, and not the more general versions of

   my @​array = something;
   my %hash  = something;

So then this doesn't explain​:

my @​array= keys %$hash;

being slower? Isn't that what this thread/bug is about?

I personally do not think that this is so clear cut, given that they
can do this​:

     my $r;
again​:
     my @​x;
     @​x = @​$r; # *** common assignment on 2nd attempt!
     print "@​x\n";
     @​x = (1,2,3);
     $r = \@​x;
     goto again unless $i++;

And have it work correctly?

I'm not sure what point you're trying to make here. In that code, the
@​x = @​$r is pessimised, and always has been. The only change made in 5.1.20
onwards is that formerly, it was assumed that code starting with a my
declaration, i.e.
   my @​x = ...
could always be optimised, because it was impossible at that point for @​x
to have any elements. FC's closure example, and then my goto example, both
demonstrate that it is possible at the point of executing the PADAV/INTRO,
for @​x to already have gained elements. Thus we now recognise that there
are some cases where this assumption isn't true.

I have not fully understood FC's closure case properly, so maybe this is
an easier trap to fall into than I think, but right now I'm thinking
this is a pessimisation that is much akin to make $1 "safe" after a //g
in scalar context, a case where we decided that simply saying "don't do
that" was better than making every //g match slower.

IOW, I'm not sure that sacrificing performance on a very common
construct to make what seems to me to be a pretty unusual edge case work
properly is a good trade off especially given there is an easy work
around.

I don't understand what you mean by a workaround.

By workaround I meant that moving the my one line up "makes it work ok".

While I don't think id go so far as to say we should actually choosing
speed over correctness is right here, I have to admit I'm tempted, and
do think at the very least we should keep this bug open.

We're not talking speed over correctness here. We're talking speed over
panics and crashing.

Yes, the panic is why I was being so mealy-mouthed about things.

Thanks for explaining things.

cheers,
Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Mar 12, 2011

From @iabyn

On Sat, Mar 12, 2011 at 10​:17​:51PM +0100, demerphq wrote​:

So then this doesn't explain​:

my @​array= keys %$hash;

being slower? Isn't that what this thread/bug is about?

The important point is the hash deref on the RHS.

However, this is all happily now irrelevant, with this commit I've just
pushed, which speeds this case up again​:

commit f403e0db17c184bb595ab0dd953ac809957ea768
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Sat Mar 12 22​:01​:26 2011 +0000
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Sat Mar 12 22​:09​:52 2011 +0000

  [perl #82111] de-pessimise some my @​array = ...
 
  Due to obscure closure and goto tricks, it's sometimes possible for the
  array or hash in the LHS of 'my @​a = ...' and 'my %h = ...' to be
  non-empty. At compile-time, these conditions are detected and the assign
  op is compiled with the OPpASSIGN_COMMON, making the assignment slower.
 
  This commit speeds it up again by adding a run-time check to pp_aassign
  to only do the OPpASSIGN_COMMON code-branch if the LHS isn't an empty
  array or hash.
 
  See also #70171.

Affected files ...
 
  M pp_hot.c

Differences ...

Inline Patch
diff --git a/pp_hot.c b/pp_hot.c
index 852ff80..f8a61cb 100644
--- a/pp_hot.c
+++ b/pp_hot.c
@@ -989,8 +989,19 @@ PP(pp_aassign)
     /* If there's a common identifier on both sides we have to take
      * special care that assigning the identifier on the left doesn't
      * clobber a value on the right that's used later in the list.
+     * Don't bother if LHS is just an empty hash or array.
      */
-    if (PL_op->op_private & (OPpASSIGN_COMMON)) {
+
+    if (    (PL_op->op_private & OPpASSIGN_COMMON)
+	&&  (
+	       firstlelem != lastlelem
+	    || ! ((sv = *firstlelem))
+	    || SvMAGICAL(sv)
+	    || ! (SvTYPE(sv) == SVt_PVAV || SvTYPE(sv) == SVt_PVHV)
+	    || (SvTYPE(sv) == SVt_PVAV && AvFILL((AV*)sv) != -1)
+	    || (SvTYPE(sv) == SVt_PVHV && HvKEYS((HV*)sv) != 0)
+	    )
+    ) {
 	EXTEND_MORTAL(lastrelem - firstrelem + 1);
 	for (relem = firstrelem; relem <= lastrelem; relem++) {
 	    if ((sv = *relem)) {


-- 

O Unicef Clearasil!
Gibberish and Drivel!
  -- "Bored of the Rings"

@p5pRT
Copy link
Author

p5pRT commented Mar 12, 2011

@iabyn - Status changed from 'rejected' to 'resolved'

@p5pRT p5pRT closed this as completed Mar 12, 2011
@p5pRT
Copy link
Author

p5pRT commented Mar 12, 2011

From @demerphq

On 12 March 2011 23​:12, Dave Mitchell <davem@​iabyn.com> wrote​:

On Sat, Mar 12, 2011 at 10​:17​:51PM +0100, demerphq wrote​:

So then this doesn't explain​:

my @​array= keys %$hash;

being slower? Isn't that what this thread/bug is about?

The important point is the hash deref on the RHS.

However, this is all happily now irrelevant, with this commit I've just
pushed, which speeds this case up again​:

Cool, thanks!

Yves
--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Mar 12, 2011

From @demerphq

are you sure you pushed it? i dont see a mail, and when i pulled i
didnt see any changes...

On 12 March 2011 23​:12, Dave Mitchell <davem@​iabyn.com> wrote​:

On Sat, Mar 12, 2011 at 10​:17​:51PM +0100, demerphq wrote​:

So then this doesn't explain​:

my @​array= keys %$hash;

being slower? Isn't that what this thread/bug is about?

The important point is the hash deref on the RHS.

However, this is all happily now irrelevant, with this commit I've just
pushed, which speeds this case up again​:

commit f403e0db17c184bb595ab0dd953ac809957ea768
Author​:     David Mitchell <davem@​iabyn.com>
AuthorDate​: Sat Mar 12 22​:01​:26 2011 +0000
Commit​:     David Mitchell <davem@​iabyn.com>
CommitDate​: Sat Mar 12 22​:09​:52 2011 +0000

   [perl #82111] de-pessimise some my @​array = ...

   Due to obscure closure and goto tricks, it's sometimes possible for the
   array or hash in the LHS of 'my @​a = ...' and 'my %h = ...' to be
   non-empty. At compile-time, these conditions are detected and the assign
   op is compiled with the OPpASSIGN_COMMON, making the assignment slower.

   This commit speeds it up again by adding a run-time check to pp_aassign
   to only do the OPpASSIGN_COMMON code-branch if the LHS isn't an empty
   array or hash.

   See also #70171.

Affected files ...

   M   pp_hot.c

Differences ...

diff --git a/pp_hot.c b/pp_hot.c
index 852ff80..f8a61cb 100644
--- a/pp_hot.c
+++ b/pp_hot.c
@​@​ -989,8 +989,19 @​@​ PP(pp_aassign)
    /* If there's a common identifier on both sides we have to take
     * special care that assigning the identifier on the left doesn't
     * clobber a value on the right that's used later in the list.
+     * Don't bother if LHS is just an empty hash or array.
     */
-    if (PL_op->op_private & (OPpASSIGN_COMMON)) {
+
+    if (    (PL_op->op_private & OPpASSIGN_COMMON)
+       &&  (
+              firstlelem != lastlelem
+           || ! ((sv = *firstlelem))
+           || SvMAGICAL(sv)
+           || ! (SvTYPE(sv) == SVt_PVAV || SvTYPE(sv) == SVt_PVHV)
+           || (SvTYPE(sv) == SVt_PVAV && AvFILL((AV*)sv) != -1)
+           || (SvTYPE(sv) == SVt_PVHV && HvKEYS((HV*)sv) != 0)
+           )
+    ) {
       EXTEND_MORTAL(lastrelem - firstrelem + 1);
       for (relem = firstrelem; relem <= lastrelem; relem++) {
           if ((sv = *relem)) {

--
O Unicef Clearasil!
Gibberish and Drivel!
   -- "Bored of the Rings"

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Mar 12, 2011

From @iabyn

On Sat, Mar 12, 2011 at 11​:35​:12PM +0100, demerphq wrote​:

are you sure you pushed it? i dont see a mail, and when i pulled i
didnt see any changes...

Whoops. Now pushed, as commit acdea6f

--
Atheism is a religion like not collecting stamps is a hobby

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant