From zefram@fysh.org

Created by zefram@fysh.org

On a Perl with 64-bit IV and 64-bit NV (1+52 bit significand)​:

$ perl -lwe '$a = 18446744073709549568; printf "%u\n", $a-1'
18446744073709549567
$ perl -lwe '$a = 2**64 - 2**11; printf "%u\n", $a-1'
18446744073709549568

$a gets the same numerical value in both cases, but the first time
it is as a native integer and the second time as a native float.
perlnumber(1) says​:

# The binary operators ... "-" ... will attempt to convert arguments to
# integers. If both conversions are possible without loss of precision,
# and the operation can be performed without loss of precision then the
# integer result is used.

So I'd expect that in both cases the subtraction would be done in
integer arithmetic and yield a mathematically exact integer result.
That actually only happens in the first case. In the second case it
appears that the subtraction was done in float arithmetic, where the
exact result is not representable.

This only happens at the extreme positive end of the native integer range.
Around +2^63 everything works nominally, and also around -2^63 at the
extreme negative end of the native integer range.

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.8.8:

Configured by zefram at Tue Jan 16 13:21:19 GMT 2007.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.14-gentoo-r5, archname=i686-linux-thread-multi-64int
    uname='linux shilling.local 2.6.14-gentoo-r5 #1 smp preempt thu jan 12 11:40:10 gmt 2006 i686 intel(r) pentium(r) 4 cpu 3.20ghz genuineintel gnulinux '
    config_args='-des -Darchname=i686-linux-thread -Dcccdlflags=-fPIC -Dccdlflags=-rdynamic -Dcc=i686-pc-linux-gnu-gcc -Dprefix=/home/zefram/tmp/perl64i -Dlocincpth= -Doptimize=-O3 -march=pentium4 -mtune=pentium4 --force-addr -momit-leaf-frame-pointer -fomit-frame-pointer -ftracer -pipe -Duselargefiles -Dd_semctl_semun -Dman1ext=1 -Dman3ext=3pm -Ud_csh -Dusethreads -Di_ndbm -Di_gdbm -Di_db -Duse64bitint'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='i686-pc-linux-gnu-gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -pipe -Wdeclaration-after-statement -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O3 -march=pentium4 -mtune=pentium4 --force-addr -momit-leaf-frame-pointer -fomit-frame-pointer -ftracer -pipe',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -pipe -Wdeclaration-after-statement'
    ccversion='', gccversion='3.4.5 (Gentoo 3.4.5, ssp-3.4.5-1.0, pie-8.7.9)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='i686-pc-linux-gnu-gcc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lndbm -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.3.6.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.3.6'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.8:
    /home/zefram/tmp/perl64i/lib/5.8.8/i686-linux-thread-multi-64int
    /home/zefram/tmp/perl64i/lib/5.8.8
    /home/zefram/tmp/perl64i/lib/site_perl/5.8.8/i686-linux-thread-multi-64int
    /home/zefram/tmp/perl64i/lib/site_perl/5.8.8
    /home/zefram/tmp/perl64i/lib/site_perl
    .


Environment for perl v5.8.8:
    HOME=/home/zefram
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/zefram/pub/i686-pc-linux-gnu/bin:/home/zefram/pub/common/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/local/bin:/usr/games/bin:/opt/exim/bin:/opt/opera/bin:/opt/sun-jdk-1.4.2.10/bin:/opt/tomcat5/bin
    PERL_BADLANG (unset)
    SHELL=/home/zefram/pub/i686-pc-linux-gnu/bin/zsh

From @nwc10

On Thu, Jan 18, 2007 at 06​:57​:57AM -0800, Zefram wrote​:

On a Perl with 64-bit IV and 64-bit NV (1+52 bit significand)​:

$ perl -lwe '$a = 18446744073709549568; printf "%u\n", $a-1'
18446744073709549567
$ perl -lwe '$a = 2**64 - 2**11; printf "%u\n", $a-1'
18446744073709549568

$a gets the same numerical value in both cases, but the first time
it is as a native integer and the second time as a native float.
perlnumber(1) says​:

# The binary operators ... "-" ... will attempt to convert arguments to
# integers. If both conversions are possible without loss of precision,
# and the operation can be performed without loss of precision then the
# integer result is used.

So I'd expect that in both cases the subtraction would be done in
integer arithmetic and yield a mathematically exact integer result.
That actually only happens in the first case. In the second case it
appears that the subtraction was done in float arithmetic, where the
exact result is not representable.

This only happens at the extreme positive end of the native integer range.
Around +2^63 everything works nominally, and also around -2^63 at the
extreme negative end of the native integer range.

You're hitting the limits of the implementation.
The documentation perhaps is a little over-terse and so inaccurate.
Conversion to integers for integer arithmetic (from floating point) only
happens if the value is in the range for which the floating point
representation can hold integers exactly.

I forget why this was needed, but the alternative (convert whenever it can
be) turned out to cause more problems, because things like 2**64 - 2046
and 2**64 - 2048 (which both round to the same floating point value with a
53 bit mantissa), so you don't actually know if the value
18446744073709549568.0 really was 18446744073709549568, or actually
18446744073709549570
So we decided to keep it as floating point, because it's inexact, rather
than giving an illusion of accuracy.

Nicholas Clark

The RT System itself - Status changed from 'new' to 'open'

From zefram@fysh.org

Wolfgang Laun wrote​:

Maybe I'm just dense, but I think 2**64 *cannot* be converted to an
integer, as
this is a binary 1 followed by 64 zeros, so it requires 65 bits.

This is correct, but that's not the value that I expected to be converted
to integer. The subtraction 2**64 - 2**11 necessarily takes place
in floating point. The result of this *is* exactly representable as
an integer. The subtraction $a - 1 should then take place in integer
arithmetic. Sorry I wasn't clearer about which subtraction I meant.

-zefram

From @doy

For what it's worth, I agree with Nicholas's reasoning here.

-doy

From @nwc10

On Fri, Jul 06, 2012 at 07​:25​:09PM -0700, Jesse Luehrs via RT wrote​:

For what it's worth, I agree with Nicholas's reasoning here.

I have no attachment to the approach we currently have - it seemed to be
the only think that worked. If someone can figure out an approach that works
better than the one we currently have, great.

"better" isn't intended to be a weasel word, but I guess that it is.
It's about 10 years now, and I really can't remember the details, but
various combinations of integer size, floating point mantissa size, and
sprintf rounding tended to really upset the expectations of some regression
tests (and some code)

Life is probably sort of easier now because

a) I think most platforms that support long doubles (eg FreeBSD) now
  *actually* properly support long doubles (eg sinl and sqrtl) - they didn't
  then

b) IIRC the platform with the most troublesome sprintf is pretty much
  dead (I think it was Tru64, but it might have been DG/UX). Certainly, on
  one 64 bit platform it was impossible to find *any* printf format that
  would actually output the full digits unrounded​:

  $ perl -e 'printf "%20.0f\n", 2**64'
  18446744073709551616

  Instead it insisted on giving something like 18446744073709550000

I don't think that we actually had any problems from the "interesting"
Cray architecture on this one. (the one with the 53 bit "integer" divide,
and sizeof(short) == 8, among other things, now sadly no longer available
to us masochists.)

Nicholas Clark