Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unicode Character 'PARAGRAPH SEPARATOR' (U+2029) cannot be used to separate lines #3449

Closed
p6rt opened this issue Jul 20, 2014 · 7 comments
Closed

Comments

@p6rt
Copy link

p6rt commented Jul 20, 2014

Migrated from rt.perl.org#122341 (status was 'resolved')

Searchable as RT122341$

@p6rt
Copy link
Author

p6rt commented Jul 20, 2014

From @coke

S02-lexical-conventions/unicode.t has a test​:

eval_lives_ok "\{ 1 \} \x2029 \{ 1 \}"

which currently fails.

--
Will "Coke" Coleda

@p6rt
Copy link
Author

p6rt commented Jul 21, 2014

From @FROGGS

Fixed for MoarVM and JVM​:
MoarVM/MoarVM@503f76ac8b
Raku/nqp@f1b098fea6

Tests​:
Raku/roast@efcdc6b724

Attempt to fix it for parrot​:

Inline Patch
diff --git a/tools/dev/gen_charset_tables.pl b/tools/dev/gen_charset_tables.pl
index 2d39969..fb65f19 100755
--- a/tools/dev/gen_charset_tables.pl
+++ b/tools/dev/gen_charset_tables.pl
@@ -77,7 +77,7 @@ sub classify {
     $ret |= 0x0200 if $chr =~ /^[[:cntrl:]]$/;            # CCLASS_CONTROL
     $ret |= 0x0400 if $chr =~ /^[[:punct:]]$/;            # CCLASS_PUNCTUATION
     $ret |= 0x0800 if $chr =~ /^[[:alnum:]]$/;            # CCLASS_ALPHANUMERIC
-    $ret |= 0x1000 if $chr =~ /^[\n\r\f\x85]$/;           # CCLASS_NEWLINE
+    $ret |= 0x1000 if $chr =~ /^[\n\x0b\r\f\x85\x{2028}\x{2029}]$/; # CCLASS_NEWLINE
     $ret |= 0x2000 if $chr =~ /^[[:alnum:]_]$/;           # CCLASS_WORD
 
     return $ret;


Which sadly explodes like: /home/froggs/dev/nqp/install/bin/nqp\-p \-\-target=pir \-\-output=src/gen/perl6\-grammar\.pir \-\-encoding=utf8 src/Perl6/Grammar\.nqp Unable to parse expression in metachar​:sym\; couldn't find final '>' at line 384, near "\+\[\\\\\< \\\\\[ \\\\\\x\{ab\}"

@p6rt
Copy link
Author

p6rt commented Jul 21, 2014

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Jul 22, 2014

From @FROGGS

Parrot PR added​: parrot/parrot#1087

@p6rt
Copy link
Author

p6rt commented Nov 9, 2014

From @usev6

With the latest Parrot the tests pass on all backends again.

I unfudged the tests with commit Raku/roast@d6e7e47647 and I'm closing this ticket now.

1 similar comment
@p6rt
Copy link
Author

p6rt commented Nov 9, 2014

From @usev6

With the latest Parrot the tests pass on all backends again.

I unfudged the tests with commit Raku/roast@d6e7e47647 and I'm closing this ticket now.

@p6rt
Copy link
Author

p6rt commented Nov 9, 2014

@usev6 - Status changed from 'open' to 'resolved'

@p6rt p6rt closed this as completed Nov 9, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant