Skip Menu |
Report information
Id: 127075
Status: resolved
Priority: 0/
Queue: perl6

Owner: Nobody
Requestors: rubio.terra [at] gmail.com
Cc:
AdminCc:

Severity: (no value)
Tag: (no value)
Platform: (no value)
Patch Status: (no value)
VM: (no value)



Subject: Error referencing non-ascii tokens on grammar
Download (untitled) / with headers
text/plain 712b
I can declare non-ascii tokens without problems on a grammar but I get an encoding error if I refer to the token using the <> syntax. This happens on MoarVM, it works fine on JVM. This runs OK: perl6 -c -e 'grammar { token TOP { <numero>+ }; token número {<< \d+ >>} }' Syntax OK Now, if I reference the new token (número) I get an error: perl6 -c -e 'grammar { token TOP { <número>+ }; token número {<< \d+ >>} }' ===SORRY!=== Error encoding ASCII string: could not encode codepoint 250 The same code on JVM runs OK: perl6-j -c -e 'grammar { token TOP { <número>+ }; token número {<< \d+ >>} }' Syntax OK I'm running Linux Fedora 22. Both Rakudo 2015.11 and 2015.12 exhibit the error.
From: Elizabeth Mattijsen <liz [...] dijkmat.nl>
To: "Rúbio Terra (via RT)" <perl6-bugs-followup [...] perl.org>
Date: Thu, 31 Dec 2015 15:12:29 +0100
Subject: Re: [perl #127075] Error referencing non-ascii tokens on grammar
Download (untitled) / with headers
text/plain 1.7k
Show quoted text
> On 29 Dec 2015, at 17:35, Rúbio Terra (via RT) <perl6-bugs-followup@perl.org> wrote: > > # New Ticket Created by Rúbio Terra > # Please include the string: [perl #127075] > # in the subject line of all future correspondence about this issue. > # <URL: https://rt.perl.org/Ticket/Display.html?id=127075 > > > > I can declare non-ascii tokens without problems on a grammar but I get an encoding error if I refer to the token using the <> syntax. This happens on MoarVM, it works fine on JVM. > > This runs OK: > > perl6 -c -e 'grammar { token TOP { <numero>+ }; token número {<< \d+ >>} }' > Syntax OK > > Now, if I reference the new token (número) I get an error: > > perl6 -c -e 'grammar { token TOP { <número>+ }; token número {<< \d+ >>} }' > ===SORRY!=== > Error encoding ASCII string: could not encode codepoint 250 > > The same code on JVM runs OK: > > perl6-j -c -e 'grammar { token TOP { <número>+ }; token número {<< \d+ >>} }' > Syntax OK > > I'm running Linux Fedora 22. Both Rakudo 2015.11 and 2015.12 exhibit the error.
This looks like it is happening deep in the bowels of NQP: $ 6l 'grammar { token TOP { <número>+ } }' Error encoding ASCII string: could not encode codepoint 250 at gen/moar/stage2/NQPP6QRegex.nqp:1382 (/Users/liz/Github/rakudo.moar/install/share/nqp/lib/NQPP6QRegex.moarvm:capnames:293) from gen/moar/stage2/NQPP6QRegex.nqp:1397 (/Users/liz/Github/rakudo.moar/install/share/nqp/lib/NQPP6QRegex.moarvm:capnames:478) from gen/moar/stage2/NQPP6QRegex.nqp:1361 (/Users/liz/Github/rakudo.moar/install/share/nqp/lib/NQPP6QRegex.moarvm:capnames:52) from gen/moar/stage2/NQPP6QRegex.nqp:1313 (/Users/liz/Github/rakudo.moar/install/share/nqp/lib/NQPP6QRegex.moarvm:qbuildsub:112) Hope somebody with NQP internals knowledge can pick this up. Liz


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org