Skip Menu |
Report information
Id: 122000
Status: resolved
Priority: 0/
Queue: perl5

Owner: tonyc <tony [at] develop-help.com>
Requestors: afresh1 [at] openbsd.org
Cc:
AdminCc:

Operating System: openbsd
PatchStatus: (no value)
Severity: low
Type: core
Perl Version: 5.18.2
Fixed In: (no value)

Attachments
0001-Default-to-using-OpenBSD-s-malloc.patch



From: afresh1 [...] openbsd.org
Subject: Use OpenBSD's malloc by default
To: perlbug [...] perl.org
Date: Sat, 31 May 2014 12:01:52 -0700
Download (untitled) / with headers
text/plain 7.9k
This is a bug report for perl from afresh1@openbsd.org, generated with the help of perlbug 1.39 running under perl 5.18.2. ----------------------------------------------------------------- [Please describe your issue here] OpenBSD would prefer that perl default to using the system malloc. In perlbug #75742 it was successfully argued that the speed of perl's malloc was reason enough to default to using it instead of the system malloc on OpenBSD. While the system malloc on OpenBSD is slower than the perl provided malloc, the OpenBSD malloc provides many security features that people using OpenBSD expect to have. OpenBSD users, by virtue of having chosen OpenBSD, have made the decision that security is more important than speed, so I believe this patch will do what most users of perl on OpenBSD would expect. This has always been the default in the base system perl in OpenBSD. Perl users who do need the additional speed can still build with mymalloc. These speed tests are with perl-5.18.2 as that is what I had easily available. The speed of the OpenBSD malloc is not significantly slower in most cases except the array case. OpenBSD malloc: Benchmark: timing 1000 iterations of 1E5 chars + 1E4 x 1E2 chars, 1E6 chars + 1E4 x 1E2 chars, 1E7 (1E5 x 1E2 chars) array + 1E4 x 1E2 chars , 1E7 chars (pre-extend to 2E7) + 1E4 x 1E2 chars, 1E7 chars + 1E1 x 1E5 chars, 1E7 chars + 1E2 x 1E4 chars, 1E7 chars + 1E3 x 1E3 chars, 1E7 chars + 1E4 x 1E2 chars, 1E7 chars + 1E5 x 1E1 chars... 1E5 chars + 1E4 x 1E2 chars: 1.0778 wallclock secs ( 1.08 usr + 0.00 sys = 1.08 CPU) @ 925.93/s (n=1000) 1E6 chars + 1E4 x 1E2 chars: 1.18271 wallclock secs ( 1.19 usr + 0.00 sys = 1.19 CPU) @ 840.34/s (n=1000) 1E7 (1E5 x 1E2 chars) array + 1E4 x 1E2 chars : 63.6428 wallclock secs (36.31 usr + 27.30 sys = 63.61 CPU) @ 15.72/s (n=1000) 1E7 chars (pre-extend to 2E7) + 1E4 x 1E2 chars: 1.63427 wallclock secs ( 1.14 usr + 0.49 sys = 1.63 CPU) @ 613.50/s (n=1000) 1E7 chars + 1E1 x 1E5 chars: 2.01161 wallclock secs ( 2.01 usr + 0.00 sys = 2.01 CPU) @ 497.51/s (n=1000) 1E7 chars + 1E2 x 1E4 chars: 2.00405 wallclock secs ( 2.00 usr + 0.00 sys = 2.00 CPU) @ 500.00/s (n=1000) 1E7 chars + 1E3 x 1E3 chars: 2.08054 wallclock secs ( 2.08 usr + 0.00 sys = 2.08 CPU) @ 480.77/s (n=1000) 1E7 chars + 1E4 x 1E2 chars: 3.05625 wallclock secs ( 3.07 usr + 0.00 sys = 3.07 CPU) @ 325.73/s (n=1000) 1E7 chars + 1E5 x 1E1 chars: 10.0508 wallclock secs (10.01 usr + 0.00 sys = 10.01 CPU) @ 99.90/s (n=1000) perl malloc: Benchmark: timing 1000 iterations of 1E5 chars + 1E4 x 1E2 chars, 1E6 chars + 1E4 x 1E2 chars, 1E7 (1E5 x 1E2 chars) array + 1E4 x 1E2 chars , 1E7 chars (pre-extend to 2E7) + 1E4 x 1E2 chars, 1E7 chars + 1E1 x 1E5 chars, 1E7 chars + 1E2 x 1E4 chars, 1E7 chars + 1E3 x 1E3 chars, 1E7 chars + 1E4 x 1E2 chars, 1E7 chars + 1E5 x 1E1 chars... 1E5 chars + 1E4 x 1E2 chars: 1.04273 wallclock secs ( 1.04 usr + 0.00 sys = 1.04 CPU) @ 961.54/s (n=1000) 1E6 chars + 1E4 x 1E2 chars: 1.13512 wallclock secs ( 1.13 usr + 0.00 sys = 1.13 CPU) @ 884.96/s (n=1000) 1E7 (1E5 x 1E2 chars) array + 1E4 x 1E2 chars : 13.2705 wallclock secs (13.24 usr + 0.00 sys = 13.24 CPU) @ 75.53/s (n=1000) 1E7 chars (pre-extend to 2E7) + 1E4 x 1E2 chars: 1.57271 wallclock secs ( 1.16 usr + 0.43 sys = 1.59 CPU) @ 628.93/s (n=1000) 1E7 chars + 1E1 x 1E5 chars: 2.00015 wallclock secs ( 2.00 usr + 0.00 sys = 2.00 CPU) @ 500.00/s (n=1000) 1E7 chars + 1E2 x 1E4 chars: 1.99175 wallclock secs ( 1.98 usr + 0.00 sys = 1.98 CPU) @ 505.05/s (n=1000) 1E7 chars + 1E3 x 1E3 chars: 2.0531 wallclock secs ( 2.04 usr + 0.00 sys = 2.04 CPU) @ 490.20/s (n=1000) 1E7 chars + 1E4 x 1E2 chars: 2.90756 wallclock secs ( 2.90 usr + 0.00 sys = 2.90 CPU) @ 344.83/s (n=1000) 1E7 chars + 1E5 x 1E1 chars: 8.85005 wallclock secs ( 8.85 usr + 0.01 sys = 8.86 CPU) @ 112.87/s (n=1000) This uses a modified version of a malloc speed test I found on the Internet. https://groups.google.com/forum/#!msg/comp.lang.perl.misc/t8kTP_IACfI/4w02x-dWPB4J use strict; use warnings; use Benchmark qw( timethese :hireswallclock ); my $c1E1 = '#' x 1E1; my $c1E2 = '#' x 1E2; my $c1E3 = '#' x 1E3; my $c1E4 = '#' x 1E4; my $c1E5 = '#' x 1E5; my $str1 = '#' x 1E5; my $str2 = '#' x 1E6; my $str3 = '#' x 1E7; my $str4 = '#' x 1E7; my $str5 = '#' x 1E7; my $str6 = '#' x 1E7; my $str7 = '#' x 1E7; my $str8 = '#' x 1E7; my $str9 = '#' x 2E7; $str9 = '#' x 1E7; my @ar1 = map{ $c1E2 } 1..1E5; timethese( 1000, { '1E5 chars + 1E4 x 1E2 chars' => sub{ my $str = $str1; $str .= $c1E2 for 1..1E4 }, '1E6 chars + 1E4 x 1E2 chars' => sub{ my $str = $str2; $str .= $c1E2 for 1..1E4 }, '1E7 chars + 1E4 x 1E2 chars' => sub{ my $str = $str3; $str .= $c1E2 for 1..1E4 }, '1E7 chars + 1E5 x 1E1 chars' => sub{ my $str = $str4; $str .= $c1E1 for 1..1E5 }, '1E7 chars + 1E4 x 1E2 chars' => sub{ my $str = $str5; $str .= $c1E2 for 1..1E4 }, '1E7 chars + 1E3 x 1E3 chars' => sub{ my $str = $str6; $str .= $c1E3 for 1..1E3 }, '1E7 chars + 1E2 x 1E4 chars' => sub{ my $str = $str7; $str .= $c1E4 for 1..1E2 }, '1E7 chars + 1E1 x 1E5 chars' => sub{ my $str = $str8; $str .= $c1E5 for 1..1E1 }, '1E7 chars (pre-extend to 2E7) + 1E4 x 1E2 chars' => sub{ $str9 .= $c1E2 for 1..1E4 }, '1E7 (1E5 x 1E2 chars) array + 1E4 x 1E2 chars ' => sub{ my @ar = @ar1; push @ar, $c1E2 for 1..1E4 }, }); [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=low --- Site configuration information for perl 5.18.2: Configured by root at Thu Jan 1 0:00:00 UTC 1970. Summary of my perl5 (revision 5 version 18 subversion 2) configuration: Platform: osname=openbsd, osvers=5.5, archname=amd64-openbsd uname='openbsd' config_args='-dsE -Dopenbsd_distribution=defined -Dccflags=-DNO_LOCALE_NUMERIC -DNO_LOCALE_COLLATE -Dmksymlinks' hint=recommended, useposix=true, d_sigaction=define useithreads=undef, usemultiplicity=undef useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-DNO_LOCALE_NUMERIC -DNO_LOCALE_COLLATE -fno-strict-aliasing -fno-delete-null-pointer-checks -pipe -fstack-protector -I/usr/local/include', optimize='-O2', cppflags='-DNO_LOCALE_NUMERIC -DNO_LOCALE_COLLATE -fno-strict-aliasing -fno-delete-null-pointer-checks -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.2.1 20070719 ', gccosandvers='openbsd5.5' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags ='-Wl,-E -fstack-protector' libpth=/usr/lib libs=-lm -lutil -lc perllibs=-lm -lutil -lc libc=/usr/lib/libc.so.74.2, so=so, useshrplib=true, libperl=libperl.so.15.0 gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-R/usr/libdata/perl5/amd64-openbsd/5.18.2/CORE' cccdlflags='-DPIC -fPIC ', lddlflags='-shared -fPIC -fstack-protector' Locally applied patches: --- @INC for perl 5.18.2: /usr/local/libdata/perl5/site_perl/amd64-openbsd /usr/libdata/perl5/site_perl/amd64-openbsd /usr/local/libdata/perl5/site_perl /usr/libdata/perl5/site_perl /usr/libdata/perl5/amd64-openbsd/5.18.2 /usr/local/libdata/perl5/amd64-openbsd/5.18.2 /usr/libdata/perl5 /usr/local/libdata/perl5 . --- Environment for perl 5.18.2: HOME=/home/afresh1 LANG (unset) LANGUAGE (unset) LC_CTYPE=en_US.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/afresh1/.plenv/bin:/home/afresh1/bin:/home/afresh1/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin:/usr/games:/home/afresh1/.plenv/bin:. PERL_BADLANG (unset) SHELL=/bin/ksh
From de5afa1c9bfe274e36e94aec99e866ba8d2c0a15 Mon Sep 17 00:00:00 2001 From: Andrew Fresh <andrew@afresh1.com> Date: Sat, 31 May 2014 10:30:40 -0700 Subject: [PATCH] Default to using OpenBSD's malloc While perl bug #75742 successfully convinced p5p that the speed of the OpenBSD malloc was a good enough reason to instead use perl's malloc, the system malloc has many security features that OpenBSD users expect. The system malloc has always been used in the base system perl. While the is slower than the perl malloc, the OpenBSD project believes the tradeoff is worth it and would prefer that users who need the speed specifically ask for it. --- hints/openbsd.sh | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/hints/openbsd.sh b/hints/openbsd.sh index fd29c71..26a8fe6 100644 --- a/hints/openbsd.sh +++ b/hints/openbsd.sh @@ -8,12 +8,8 @@ # ./Configure -des -Dopenbsd_distribution=defined # -# In OpenBSD > 3.7, use perl's malloc [perl #75742] -case "$osvers" in -3.[89]*|[4-9]*) - test "$usemymalloc" || usemymalloc=y - ;; -esac +# OpenBSD has a better malloc than perl... +test "$usemymalloc" || usemymalloc='n' # malloc wrap works case "$usemallocwrap" in -- 1.9.2
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.1k
Show quoted text
> OpenBSD would prefer that perl default to using the system malloc. > > In perlbug #75742 it was successfully argued that the speed of perl's > malloc > was reason enough to default to using it instead of the system malloc > on > OpenBSD. While the system malloc on OpenBSD is slower than the perl > provided > malloc, the OpenBSD malloc provides many security features that people > using > OpenBSD expect to have. > > OpenBSD users, by virtue of having chosen OpenBSD, have made the > decision that > security is more important than speed, so I believe this patch will do > what > most users of perl on OpenBSD would expect. > > This has always been the default in the base system perl in OpenBSD. > > Perl users who do need the additional speed can still build with > mymalloc. > > > These speed tests are with perl-5.18.2 as that is what I had easily > available. > > The speed of the OpenBSD malloc is not significantly slower in most > cases > except the array case. >
I would like to +1 this sentiment. Since the user has installed OpenBSD, the sane default assumption is that the user desires to experience the full breadth of security features built into the OS.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 507b
On Sat May 31 15:03:26 2014, eric@lalonde.com wrote: Show quoted text
> > OpenBSD would prefer that perl default to using the system malloc.
> > I would like to +1 this sentiment. Since the user has installed > OpenBSD, the sane default assumption is that the user desires to > experience the full breadth of security features built into the OS.
perl mymalloc is not thread-safe. I wouldn't trust it. I found it odd that openbsd still has to use mymalloc. cygwin switched to system malloc some time ago. -- Reini Urban
CC: Perl5 Porters <perl5-porters [...] perl.org>
Subject: Re: [perl #122000] Use OpenBSD's malloc by default
From: Brad Gilbert <b2gills [...] gmail.com>
Date: Sun, 1 Jun 2014 01:16:30 -0500
To: PerlBug <perlbug-followup [...] perl.org>
Download (untitled) / with headers
text/plain 784b
On Sun, Jun 1, 2014 at 1:04 AM, Reini Urban via RT <perlbug-followup@perl.org> wrote: Show quoted text
> On Sat May 31 15:03:26 2014, eric@lalonde.com wrote:
>> > OpenBSD would prefer that perl default to using the system malloc.
>> >> I would like to +1 this sentiment. Since the user has installed >> OpenBSD, the sane default assumption is that the user desires to >> experience the full breadth of security features built into the OS.
> > perl mymalloc is not thread-safe. I wouldn't trust it. > > I found it odd that openbsd still has to use mymalloc. > cygwin switched to system malloc some time ago. > > -- > Reini Urban
I would also like to see this happen, as it would likely tease out any use after free bugs. ( This was the main reason OpenSSL couldn't use the system malloc on OpenBSD )
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 226b
On Sat May 31 12:02:07 2014, afresh1@openbsd.org wrote: Show quoted text
> OpenBSD would prefer that perl default to using the system malloc.
Thanks, applied as 9be9e8a734382a4f2852efc22debe8e98e91eee9 with the discussed author change. Tony


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org