Skip Menu |

To: perlbug [...] perl.org
Subject: printf %a mishandles subnormals
Date: Thu, 04 Aug 2016 21:50:53 +0100
From: zefram [...] fysh.org
CC: zefram [...] fysh.org
Download (untitled) / with headers
text/plain 3.8k
This is a bug report for perl from zefram@fysh.org, generated with the help of perlbug 1.40 running under perl 5.24.0. ----------------------------------------------------------------- [Please describe your issue here] $ perl -lwe 'printf "%a\n", 3e-320' 0x1.00000000017b8p-1062 This output is numerically incorrect. This happens for any subnormal floating-point value. The output is a mixture of two ways of correctly describing the value: it is 0x0.00000000017b8p-1022 (displaying the physical structure of a subnormal) or 0x1.7b8p-1062 (consistent scientific style). $ perl -MData::Float=float_hex -lwe 'print float_hex(3e-320, {subnormal_strategy=>$_}) for qw(SUBNORMAL NORMAL)' +0x0.00000000017b8p-1022 +0x1.7b80000000000p-1062 [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=low --- Site configuration information for perl 5.24.0: Configured by zefram at Mon May 9 19:42:55 BST 2016. Summary of my perl5 (revision 5 version 24 subversion 0) configuration: Platform: osname=linux, osvers=3.16.0-4-amd64, archname=x86_64-linux-thread-multi uname='linux barba.rous.org 3.16.0-4-amd64 #1 smp debian 3.16.7-ckt11-1+deb8u6 (2015-11-09) x86_64 gnulinux ' config_args='-des -Dprefix=/home/zefram/usr/perl/perl_install/perl-5.24.0-i64-f52 -Duselargefiles -Dusethreads -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dusedevel -Uversiononly -Ui_db' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2', optimize='-O2', cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='', gccversion='4.9.2', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.19.so, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.19' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/home/zefram/usr/perl/perl_install/perl-5.24.0-i64-f52/lib/5.24.0/x86_64-linux-thread-multi/CORE' cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong' --- @INC for perl 5.24.0: /home/zefram/usr/perl/perl_install/perl-5.24.0-i64-f52/lib/site_perl/5.24.0/x86_64-linux-thread-multi /home/zefram/usr/perl/perl_install/perl-5.24.0-i64-f52/lib/site_perl/5.24.0 /home/zefram/usr/perl/perl_install/perl-5.24.0-i64-f52/lib/5.24.0/x86_64-linux-thread-multi /home/zefram/usr/perl/perl_install/perl-5.24.0-i64-f52/lib/5.24.0 . --- Environment for perl 5.24.0: HOME=/home/zefram LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/zefram/usr/perl/perl_install/perl-5.24.0-i64-f52/bin:/home/zefram/usr/perl/util:/home/zefram/pub/x86_64-unknown-linux-gnu/bin:/home/zefram/pub/common/bin:/usr/bin:/bin:/usr/local/bin:/usr/games PERL_BADLANG (unset) SHELL=/usr/bin/zsh
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 806b
On Thu Aug 04 13:51:17 2016, zefram@fysh.org wrote: Show quoted text
> $ perl -lwe 'printf "%a\n", 3e-320' > 0x1.00000000017b8p-1062 > > This output is numerically incorrect. This happens for any subnormal > floating-point value. The output is a mixture of two ways of > correctly > describing the value: it is 0x0.00000000017b8p-1022 (displaying > the physical structure of a subnormal) or 0x1.7b8p-1062 (consistent > scientific style).
The handling of the implicit one bit is incorrect too. Show quoted text
> $ perl -MData::Float=float_hex -lwe 'print float_hex(3e-320, > {subnormal_strategy=>$_}) for qw(SUBNORMAL NORMAL)' > +0x0.00000000017b8p-1022 > +0x1.7b80000000000p-1062
It looks like subnormals simply weren't implemented: /* XXX Inf/NaN/denormal handling in the HEXTRACT_IMPLICIT_BIT, * and elsewhere. */ Tony
To: perl5-porters [...] perl.org
Subject: Re: [perl #128843] printf %a mishandles subnormals
Date: Sun, 14 Aug 2016 00:18:23 +0100
From: Zefram <zefram [...] fysh.org>
Download (untitled) / with headers
text/plain 3.8k
Jarkko Hietaniemi via RT wrote: Show quoted text
When I first reviewed this, before Jarkko merged it to blead, I thought it fixed the problem, and declared it good. Actually it's not correct: it's changed the problem into a different form. Sorry for not spotting it first time. With IEEE double: $ perl -lwe 'printf "%a\n", 0x1.fffffffffffffp-1022' 0x1.fffffffffffffp-1022 $ perl -lwe 'printf "%a\n", 0x0.fffffffffffffp-1022' 0x1.fffffffffffffp-1023 $ perl -lwe 'printf "%a\n", 0x0.7ffffffffffffp-1022' 0x1.7ffffffffffffp-1024 $ perl -lwe 'printf "%a\n", 0x0.3ffffffffffffp-1022' 0x1.3ffffffffffffp-1025 $ perl -lwe 'printf "%a\n", 0x0.1ffffffffffffp-1022' 0x1.1ffffffffffffp-1026 $ perl -lwe 'printf "%a\n", 0x0.0ffffffffffffp-1022' 0x1.fffffffffffep-1027 The first one here (which is normalised) is OK, but the subnormals have gone awry. The second should be "0x1.ffffffffffffep-1023", the third should be "0x1.ffffffffffffcp-1024", the fourth should be "0x1.ffffffffffff8p-1025", and the fifth should be "0x1.ffffffffffffp-1026". The sixth output, also subnormal, is correct. Smaller subnormals of the same pattern are all correct, down to the smallest positive value. All of the incorrect ones have the pattern of showing a normalised-style integer part and exponent but the fractional part from subnormal style. That's precisely what was wrong originally for all subnormals. As far as I can see, the problem arises with any magnitude in the range [2**-1026, 2**-1022), and not for anything outside that range. I expect that the reason for exactly four binary orders of magnitude being affected is down to that being a single order of magnitude of the output radix. With x86 80-bit long double, the behaviour is stranger: $ perl -lwe 'printf "%a\n", 0x1.fffffffffffffffep0 * 2**-16382' Hexadecimal float: mantissa overflow at -e line 1. 0xf.fffffffffffffffp-16385 $ perl -lwe 'printf "%a\n", 0x0.fffffffffffffffep0 * 2**-16382' 0x7.fffffffffffffffp-16386 $ perl -lwe 'printf "%a\n", 0x0.7ffffffffffffffep0 * 2**-16382' 0x3.fffffffffffffffp-16387 $ perl -lwe 'printf "%a\n", 0x0.3ffffffffffffffep0 * 2**-16382' 0x1.fffffffffffffffp-16388 $ perl -lwe 'printf "%a\n", 0x0.1ffffffffffffffep0 * 2**-16382' 0x0.fffffffffffffffp-16389 $ perl -lwe 'printf "%a\n", 0x0.0ffffffffffffffep0 * 2**-16382' 0x0.7ffffffffffffffp-16390 $ perl -lwe 'printf "%a\n", 0x0.07fffffffffffffep0 * 2**-16382' 0x0.3ffffffffffffffp-16391 $ perl -lwe 'printf "%a\n", 0x0.03fffffffffffffep0 * 2**-16382' 0x0.1ffffffffffffffp-16392 $ perl -lwe 'printf "%a\n", 0x0.01fffffffffffffep0 * 2**-16382' 0x0.0ffffffffffffffp-16393 $ perl -lwe 'printf "%a\n", 0x0.00fffffffffffffep0 * 2**-16382' 0x0.07fffffffffffffp-16394 Subnormals here are consistently showing the integer and fractional part from subnormal style but the exponent of normalised style. This is different from the manifestation that was seen with IEEE double. 5.24.0 shows the same behaviour as blead here. Thus the manifestation of the subnormal problem was always different depending on the NV format, and Jarkko's patch hasn't affected the behaviour on x86 80-bit long double. Things get even weirder for certain subnormals in this format: $ perl -lwe 'printf "%a\n", 0x0.1000000000000000p0 * 2**-16382' 0x08p-16389 $ perl -lwe 'printf "%a\n", 0x0.1e00000000000000p0 * 2**-16382' 0x0fp-16389 $ perl -lwe 'printf "%a\n", 0x0.0000000000000040p0 * 2**-16382' 0x000000000000002p-16443 $ perl -lwe 'printf "%a\n", 0x0.0000000000000020p0 * 2**-16382' 0x000000000000001p-16444 $ perl -lwe 'printf "%a\n", 0x0.000000000000001ep0 * 2**-16382' 0x000000000000000fp-16445 Where only one output hex digit is non-zero, in addition to the initial subnormal bug the output is missing its radix point. This too hasn't changed between 5.24.0 and blead. This is presumably because if the values were output correctly normalised these ones would not require a radix point. -zefram
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.4k
On Sat Aug 13 16:19:16 2016, zefram@fysh.org wrote: Show quoted text
> Jarkko Hietaniemi via RT wrote: > > When I first reviewed this, before Jarkko merged it to blead, I > thought > it fixed the problem, and declared it good. Actually it's not > correct: > it's changed the problem into a different form. Sorry for not > spotting > it first time. >
I've been following the discussion in this RT, as well as noting the filing of 5 other bug reports related to 'printf'. Those other reports are: https://rt.perl.org/Ticket/Display.html?id=128888 https://rt.perl.org/Ticket/Display.html?id=128893 https://rt.perl.org/Ticket/Display.html?id=128889 https://rt.perl.org/Ticket/Display.html?id=128890 https://rt.perl.org/Ticket/Display.html?id=128909 (Each of the above is still in New status.) I have to admit I'm confused as to where the discussion in going. In particular, I'm concerned about the fact that no one has cited any standards for the behavior of 'printf' that we presumably have failed to meet up until now and ought to be meeting going forward. It's not clear on what basis, other than personal knowledge, participants in the discussion are making their claims or making code changes. For those following along at home, this is unsatisfying. Could we step back from the discussion for a moment and provide some links to standards we ought to be meeting? Thank you very much. -- James E Keenan (jkeenan@cpan.org)
Subject: Re: [perl #128843] printf %a mishandles subnormals
To: perl5-porters [...] perl.org
Date: Sun, 14 Aug 2016 14:48:31 +0100
From: Zefram <zefram [...] fysh.org>
Download (untitled) / with headers
text/plain 706b
James E Keenan via RT wrote: Show quoted text
>In particular, I'm concerned about the fact that no one has cited any >standards for the behavior of 'printf' that we presumably have failed >to meet up until now and ought to be meeting going forward.
I've based my expectations on the C standard, to the extent that it's not overridden by deliberate differences in Perl. I've mentioned it in a couple of the bug reports. The actual standard isn't freely available (in either sense of "free"), but a late draft of C99 is available at <http://std.dkuug.dk/JTC1/SC22/WG14/www/docs/n843.pdf>. In that document, the description of printf-style formatting starts on page 271, section 7.19.6.1 "The fprintf function". -zefram
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.5k
I've been following the discussion in this RT, as well as noting the Show quoted text
> filing of 5 other bug reports related to 'printf'. Those other > reports are: > > https://rt.perl.org/Ticket/Display.html?id=128888 > https://rt.perl.org/Ticket/Display.html?id=128893 > https://rt.perl.org/Ticket/Display.html?id=128889 > https://rt.perl.org/Ticket/Display.html?id=128890 > https://rt.perl.org/Ticket/Display.html?id=128909 > > (Each of the above is still in New status.) > > I have to admit I'm confused as to where the discussion in going. In > particular, I'm concerned about the fact that no one has cited any > standards for the behavior of 'printf' that we presumably have failed > to meet up until now and ought to be meeting going forward. It's not > clear on what basis, other than personal knowledge, participants in > the discussion are making their claims or making code changes. For > those following along at home, this is unsatisfying. > > Could we step back from the discussion for a moment and provide some > links to standards we ought to be meeting?
Yeah, sorry. I see Zefram already pointed out some of the relevant docs. In addition to the C99 (the first one in which the hexfp literals or the %a came along), I'm following what the majority of platforms seems to have implemented. There often seems to be some differences in the interpretation of the standards, but if two our of three independent implementations seem to agree on a behavior, I'm happy to follow it, unless of course the behavior clearly disagrees with some standard.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 292b
Show quoted text
> When I first reviewed this, before Jarkko merged it to blead, I > thought > it fixed the problem, and declared it good. Actually it's not > correct: > it's changed the problem into a different form. Sorry for not > spotting > it first time.
Hrm. I'll try to take a look again tonight.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 251b
Zefram, before I go off and try to fix the new findings, could you please check t/op/sprintf2.t and t/op/hexfp.t as of blead 73013786, and see that there are no accidentally wrong behaviors in the newly added tests, especially the subnormals ones?
From: Zefram <zefram [...] fysh.org>
Date: Sun, 14 Aug 2016 18:32:20 +0100
Subject: Re: [perl #128843] printf %a mishandles subnormals
To: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 553b
Jarkko Hietaniemi via RT wrote: Show quoted text
>Zefram, before I go off and try to fix the new findings, could you >please check t/op/sprintf2.t and t/op/hexfp.t as of blead 73013786,
sprintf2.t for double-double: [ '%a', '-1', '-0x0p+0' ], Expected result should be '-0x1p+0'. sprintf2.t has a comment about tests for this ticket (#128843) for the 80-bit format, but it doesn't actually have any tests for values that are subnormal in that format. It only tests 3e-320, which is normal in 80-bit. Other than that the %a tests look fine. -zefram
RT-Send-CC: perl5-porters [...] perl.org
Marking resolved until proven otherwise.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org