New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
IO::Handle.read-internal cannot handle fancy Unicode chars on TTY handles #6637
Comments
From @lefthNote: results are from Windows 10. Same results from CMD and from Cmder. Working: But in the REPL, I tried to run:
Without Linenoise installed, Rakudo endlessly repeats "> Decoder may not be With Linenoise installed, I get this error: Malformed UTF-8 at line 1 col 6 I got similar error when using these characters as quoting operators, such Note: the obvious workaround is to substitute "<<" and ">>" for "«" and "»". |
From @AlexDanielMarked as 「SEVERE」 because it's very unfortunate that non-ascii stuff does not work. I don't know why it happens though, so people should feel free to retag as needed. On 2017-11-13 20:29:52, dan@zwell.net wrote:
|
The RT System itself - Status changed from 'new' to 'open' |
From @zoffixznetOn Mon, 13 Nov 2017 20:29:52 -0800, dan@zwell.net wrote:
That looks like it might be working correctly, except cmd.exe wasn't told to UTF-8 code page (need to run `chcp 65001`).
On 2017.07 on Win7 with 65001 code page enabled, the » char doesn't show up at all. Just seems to get removed from the content if I paste it into the terminal. The default REPL uses `prompt` to read stuff and here's another datapoint: prompt() returns Nil if the data it reads has fancy Unicode chars like » or 「」: C:\Users\zoffi>perl6 -e "say prompt" C:\Users\zoffi>perl6 -e "say prompt" C:\Users\zoffi>perl6 -e "say prompt" Looks like it might be a good place to start looking at for the OP bug. |
From @zoffixznetOn Thu, 16 Nov 2017 09:53:46 -0800, cpan@zoffix.com wrote:
looks to be a problem in moarvm. perl6 -e "say $*IN.read-internal(1000)" gives an empty buf (Buf[uint8]:0x<>) if the input got fancypants chars |
From @zoffixznetOn Thu, 16 Nov 2017 09:53:46 -0800, cpan@zoffix.com wrote:
Starting to think this might be a limitation of cmd.exe. Though strangely, For example, Perl 5 has the exact same problem that the fancy chars get stripped: C:\rakudo>perl -wlE "say '[' . scalar(readline) . ']'" I also cobbled together this C program from MSDN's code examples and the same #include <fcntl.h> int main( void ) { result = _setmode(_fileno(stdin), _O_BINARY); if( ( bytesread = _read( _fileno( stdin ), buffer, nbytes ) ) <= 0 ) printf("Read this: `"); return 1; Anything fancy gets read as a nul byte instead of the proper bytes for that char: C:\rakudo>gcc test.c && a.exe C:\rakudo> |
From @geekosaurOn Mon, Dec 25, 2017 at 1:07 AM, Zoffix Znet via RT <
IIRC this is known, and not really fixable. It's not even cmd.exe but a -- |
From @geekosaurOn Tue, Dec 26, 2017 at 12:15 AM, Brandon Allbery via RT <
Come to think of it, there should be existing mention of this on the moarvm -- |
Migrated from rt.perl.org#132441 (status was 'open')
Searchable as RT132441$
The text was updated successfully, but these errors were encountered: