From zefram@fysh.org

(9.999e-5 * 2e0**66).Int * 5**8 - 9999 * 2**58
287369
(9.99899999999999995e-5 * 2e0**66).Int * 5**8 - 9999 * 2**58
-103256

The above computations show, scaled up, the difference between a Num value
and the exact value 9.999e-5. As you can see, the Num that arose from
the "9.99899999999999995e-5" literal was slightly closer than the Num
that arose from the "9.999e-5" literal. The closer of these two is in
fact the closest representable IEEE double precision value to 9.999e-5.
Thus this literal "9.999e-5" has not yielded the closest available Num
to its nominal value; this is a bug. glibc's strtod() handles this case
fine, yielding the closer value.

Note that the only rounding occurring in this test case is on the
decimal->float conversion. The multiplication by a power of two,
conversion of integer-valued float to Int, and all the Int arithmetic,
are all exact.

-zefram

From @coke

On Fri, 12 Aug 2016 09​:46​:38 -0700, zefram@​fysh.org wrote​:

(9.999e-5 * 2e0**66).Int * 5**8 - 9999 * 2**58
287369
(9.99899999999999995e-5 * 2e0**66).Int * 5**8 - 9999 * 2**58
-103256

The above computations show, scaled up, the difference between a Num value
and the exact value 9.999e-5. As you can see, the Num that arose from
the "9.99899999999999995e-5" literal was slightly closer than the Num
that arose from the "9.999e-5" literal. The closer of these two is in
fact the closest representable IEEE double precision value to 9.999e-5.
Thus this literal "9.999e-5" has not yielded the closest available Num
to its nominal value; this is a bug. glibc's strtod() handles this case
fine, yielding the closer value.

Note that the only rounding occurring in this test case is on the
decimal->float conversion. The multiplication by a power of two,
conversion of integer-valued float to Int, and all the Int arithmetic,
are all exact.

-zefram

as of b5aa3c5, these both output 287369 now.

--
Will "Coke" Coleda

The RT System itself - Status changed from 'new' to 'open'

From zefram@fysh.org

Will Coleda via RT wrote​:

as of b5aa3c5, these both output 287369 now.

For clarity, that's still bad rounding, then.

-zefram

From @zoffixznet

On Fri, 12 Aug 2016 09​:46​:38 -0700, zefram@​fysh.org wrote​:

(9.999e-5 * 2e0**66).Int * 5**8 - 9999 * 2**58
287369
(9.99899999999999995e-5 * 2e0**66).Int * 5**8 - 9999 * 2**58
-103256

The above computations show, scaled up, the difference between a Num value
and the exact value 9.999e-5. As you can see, the Num that arose from
the "9.99899999999999995e-5" literal was slightly closer than the Num
that arose from the "9.999e-5" literal. The closer of these two is in
fact the closest representable IEEE double precision value to 9.999e-5.
Thus this literal "9.999e-5" has not yielded the closest available Num
to its nominal value; this is a bug. glibc's strtod() handles this case
fine, yielding the closer value.

Note that the only rounding occurring in this test case is on the
decimal->float conversion. The multiplication by a power of two,
conversion of integer-valued float to Int, and all the Int arithmetic,
are all exact.

-zefram

Thank you for the report. This is now fixed.

Fix​: MoarVM/MoarVM@067c0594103a025
  MoarVM/MoarVM@8841c4241b4faa8
  MoarVM/MoarVM@af2eb8a7f7d4344
  MoarVM/MoarVM@4d3fc2818d0032b
  rakudo/rakudo@8422d7b4e23678b
  rakudo/rakudo@a2a2a745c4242d1
Test​: Raku/roast@7fdd5372b85172fcc

@zoffixznet - Status changed from 'open' to 'resolved'