Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmark feature request: interleave code runs #13662

Open
p5pRT opened this issue Mar 14, 2014 · 3 comments
Open

Benchmark feature request: interleave code runs #13662

p5pRT opened this issue Mar 14, 2014 · 3 comments
Labels
distro-Linux type-library Wishlist

Comments

@p5pRT
Copy link

p5pRT commented Mar 14, 2014

Migrated from rt.perl.org#121439 (status was 'open')

Searchable as RT121439$
@p5pRT
Copy link
Author

p5pRT commented Mar 14, 2014

From perlbug@ch.pkts.ca

Created by perlbug@ch.pkts.ca

I've been running a benchmark between two short pieces of code, and
the run-to-run variation between them is significant. For instance,
in the first run of the benchmark, the second piece was 3% faster;
in the next run the first piece was 2% faster.

Instead of running the first piece X times, then running the second piece
X times, how about running the first piece followed by the second piece,
X times, and sum the individual times?

This will help to get rid of variation in the results due to outside
influences, such as some other program starting up and using cpu, disk,
memory or net only during one half of the benchmark.

Thanks!

(Here's the code I was testing​:)
#!/usr/bin/perl -w
use strict;

use Benchmark qw(​:all);

my @​toDisplay = qw( exceedlimit startdate folder useracls from from_groups_choice
  tocc to_groups_choice subject attfilename attachment_extension
  attsize_combined priorityid rmcm_features bloomberg_type
  linkedin_features twitter_features labelid filterflags
  afr_widget_flag event_type afr_widget_event keywords
  headername
  headervalue grouptag retention indexerror
  mobilelogs ); #labelid/commentid

my %searchcriteria = (
  exceedlimit => 1,
  twitter_features => 1,
  );

cmpthese(360000*3, {
  'Original' => 'original();',
  'Modified' => 'modified();',
  });

sub original {
  my $aa;
  foreach my $crit ( @​toDisplay ) {
  if ($searchcriteria{$crit}) {
  $aa="foo".$crit.time();
  }
  }
}

sub modified {
  my $aa;
  foreach my $crit ( grep {$searchcriteria{$_}} @​toDisplay ) {
  $aa="foo".$crit.time();
  }
}

Perl Info 

Flags:
    category=library
    severity=wishlist
    module=Benchmark

Site configuration information for perl 5.14.2:

Configured by Debian Project at Tue Feb  4 23:09:53 UTC 2014.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:
   
  Platform:
    osname=linux, osvers=2.6.42-37-generic, archname=x86_64-linux-gnu-thread-multi
    uname='linux panlong 2.6.42-37-generic #58-ubuntu smp thu jan 24 15:28:10 utc 2013 x86_64 x86_64 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.14 -Darchlib=/usr/lib/perl/5.14 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.14.2 -Dsitearch=/usr/local/lib/perl/5.14.2 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.14.2 -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.6.3', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=, so=so, useshrplib=true, libperl=libperl.so.5.14.2
    gnulibc_version='2.15'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector'

Locally applied patches:
    


@INC for perl 5.14.2:
    /root/systemscripts/SVN/GlobalRelay-FilterSystem/lib
    /etc/perl
    /usr/local/lib/perl/5.14.2
    /usr/local/share/perl/5.14.2
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.14
    /usr/share/perl/5.14
    /usr/local/lib/site_perl
    .


Environment for perl 5.14.2:
    HOME=/home/charles.howes
    LANG=en_CA.UTF-8
    LANGUAGE=en_CA:en
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/charles.howes/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/home/charles.howes/bin:/usr/local/bin:/opt/devel:/opt/csw/bin:/opt/csw/sbin:/usr/sfw/bin:/usr/sfw/sbin:/bin:/usr/bin:/opt/bin:/sbin:/usr/sbin:/usr/ccs/bin:/opt/httpd/bin:
    PERL5LIB=:/root/systemscripts/SVN/GlobalRelay-FilterSystem/lib
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented May 18, 2014

From @jkeenan

On Fri Mar 14 11​:22​:44 2014, perlbug@​ch.pkts.ca wrote​:

This is a bug report for perl from perlbug@​ch.pkts.ca,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
[Please describe your issue here]
I've been running a benchmark between two short pieces of code, and
the run-to-run variation between them is significant. For instance,
in the first run of the benchmark, the second piece was 3% faster;
in the next run the first piece was 2% faster.

Instead of running the first piece X times, then running the second
piece
X times, how about running the first piece followed by the second
piece,
X times, and sum the individual times?

This will help to get rid of variation in the results due to outside
influences, such as some other program starting up and using cpu,
disk,
memory or net only during one half of the benchmark.

I am not an expert in the theory or practice of benchmarking, so I can't say whether your request is plausible or not. However, even if it were plausible, my hunch is that the Perl 5 Porters -- almost entirely volunteers -- would not have the resources to do this by modifying the Benchmark library distributed with the Perl 5 core and then maintaining it on a permanent basis..

I would recommend that you try this approach out via by cloning Benchmark.pm, getting the clone to work as desired, and then posting the result on CPAN.

Thank you very much.
JIm Keenan

@p5pRT
Copy link
Author

p5pRT commented May 18, 2014

The RT System itself - Status changed from 'new' to 'open'

@xenu xenu removed the affects-5.14 label Nov 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
distro-Linux type-library Wishlist
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@xenu @p5pRT