Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

number parsing #16073

Closed
p5pRT opened this issue Jul 9, 2017 · 10 comments
Closed

number parsing #16073

p5pRT opened this issue Jul 9, 2017 · 10 comments

Comments

@p5pRT
Copy link

p5pRT commented Jul 9, 2017

Migrated from rt.perl.org#131725 (status was 'resolved')

Searchable as RT131725$

@p5pRT
Copy link
Author

p5pRT commented Jul 9, 2017

From ruud.vantol@booking.com

Created by ruud.vantol@booking.com

perl -wE'say 1e--5'
-4

I think it should die if there is no digit directly after the 'e' and sign.

Perl Info

Flags:
    category=core
    severity=low


@p5pRT
Copy link
Author

p5pRT commented Aug 3, 2017

From @tonycoz

On Sun, 09 Jul 2017 05​:11​:37 -0700, ruud.vantol@​booking.com wrote​:

perl -wE'say 1e--5'
-4

I think it should die if there is no digit directly after the 'e' and sign.

The attached prevents consuming the "e-" if there's no digits in the exponent.

Tony

@p5pRT
Copy link
Author

p5pRT commented Aug 3, 2017

From @tonycoz

0001-perl-131725-ignore-the-exponent-on-a-decimal-float-i.patch
From 01378fadacb66c3905dd881fc28f30f573547aee Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 3 Aug 2017 12:11:56 +1000
Subject: (perl #131725) ignore the exponent on a decimal float if no digits

Previously the "1e-" in "1e--5" would be treated as "1", but consumed
the "e-".

This wasn't an issue for hex floats.

I considered (and implemented) croaking instead, but this was
inconsistent with the behaviour for hex floats, which only reach this
code if a full hex float has been parsed.
---
 t/lib/croak/toke | 10 ++++++++++
 toke.c           | 21 +++++++++++++++++++--
 2 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/t/lib/croak/toke b/t/lib/croak/toke
index 2603224..c477be0 100644
--- a/t/lib/croak/toke
+++ b/t/lib/croak/toke
@@ -394,3 +394,13 @@ $a = <<~ ;
 
 EXPECT
 Use of bare << to mean <<"" is forbidden at - line 1.
+########
+# NAME incomplete floating point decimal exponent (#131725)
+1e--5
+EXPECT
+Bareword found where operator expected at - line 1, near "1e"
+	(Missing operator before e?)
+Number found where operator expected at - line 1, near "--5"
+	(Missing operator before 5?)
+syntax error at - line 1, near "1e"
+Execution of - aborted due to compilation errors.
diff --git a/toke.c b/toke.c
index 6aa5f26..6de7d09 100644
--- a/toke.c
+++ b/toke.c
@@ -11182,9 +11182,11 @@ Perl_scan_num(pTHX_ const char *start, YYSTYPE* lvalp)
               || UNLIKELY(hexfp && isALPHA_FOLD_EQ(*s, 'p')))
             && strchr("+-0123456789_", s[1]))
         {
-            floatit = TRUE;
+            int exp_digits = 0;
+            const char *save_s = s;
+            char * save_d = d;
 
-	    /* regardless of whether user said 3E5 or 3e5, use lower 'e',
+            /* regardless of whether user said 3E5 or 3e5, use lower 'e',
                ditto for p (hexfloats) */
             if ((isALPHA_FOLD_EQ(*s, 'e'))) {
 		/* At least some Mach atof()s don't grok 'E' */
@@ -11216,6 +11218,7 @@ Perl_scan_num(pTHX_ const char *start, YYSTYPE* lvalp)
 	    /* read digits of exponent */
 	    while (isDIGIT(*s) || *s == '_') {
 	        if (isDIGIT(*s)) {
+                    ++exp_digits;
 		    if (d >= e)
 		        Perl_croak(aTHX_ "%s", number_too_long);
 		    *d++ = *s++;
@@ -11227,6 +11230,20 @@ Perl_scan_num(pTHX_ const char *start, YYSTYPE* lvalp)
 		   lastub = s++;
 		}
 	    }
+
+            if (!exp_digits) {
+                /* no exponent digits, the [eEpP] could be for something else,
+                 * though in practice we don't get here for p since that's preparsed
+                 * earlier, and results in only the 0xX being consumed, so behave similarly
+                 * for decimal floats and consume only the D.DD, leaving the [eE] to the
+                 * next token.
+                 */
+                s = save_s;
+                d = save_d;
+            }
+            else {
+                floatit = TRUE;
+            }
 	}
 
 
-- 
2.1.4

@p5pRT
Copy link
Author

p5pRT commented Aug 3, 2017

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Aug 14, 2017

From @tonycoz

On Wed, 02 Aug 2017 19​:15​:25 -0700, tonyc wrote​:

On Sun, 09 Jul 2017 05​:11​:37 -0700, ruud.vantol@​booking.com wrote​:

perl -wE'say 1e--5'
-4

I think it should die if there is no digit directly after the 'e' and
sign.

The attached prevents consuming the "e-" if there's no digits in the
exponent.

Applied as adb0f5c.

If it ends up causing lots of CPAN failures it may need to be changed to a deprecation.

Tony

@p5pRT
Copy link
Author

p5pRT commented Aug 14, 2017

From @xsawyerx

On 08/14/2017 07​:59 AM, Tony Cook via RT wrote​:

On Wed, 02 Aug 2017 19​:15​:25 -0700, tonyc wrote​:

On Sun, 09 Jul 2017 05​:11​:37 -0700, ruud.vantol@​booking.com wrote​:

perl -wE'say 1e--5'
-4

I think it should die if there is no digit directly after the 'e' and
sign.
The attached prevents consuming the "e-" if there's no digits in the
exponent.
Applied as adb0f5c.

If it ends up causing lots of CPAN failures it may need to be changed to a deprecation.

I think that's a good idea.

@p5pRT
Copy link
Author

p5pRT commented Jan 7, 2019

From @tonycoz

On Sun, 13 Aug 2017 22​:59​:59 -0700, tonyc wrote​:

On Wed, 02 Aug 2017 19​:15​:25 -0700, tonyc wrote​:

On Sun, 09 Jul 2017 05​:11​:37 -0700, ruud.vantol@​booking.com wrote​:

perl -wE'say 1e--5'
-4

I think it should die if there is no digit directly after the 'e'
and
sign.

The attached prevents consuming the "e-" if there's no digits in the
exponent.

Applied as adb0f5c.

If it ends up causing lots of CPAN failures it may need to be changed
to a deprecation.

I didn't see any reported issues, closing.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jan 7, 2019

@tonycoz - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented May 22, 2019

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

@p5pRT
Copy link
Author

p5pRT commented May 22, 2019

@khwilliamson - Status changed from 'pending release' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant