Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

utf8-c8 generates spurious NUL #5408

Closed
p6rt opened this issue Jul 1, 2016 · 6 comments
Closed

utf8-c8 generates spurious NUL #5408

p6rt opened this issue Jul 1, 2016 · 6 comments

Comments

@p6rt
Copy link

p6rt commented Jul 1, 2016

Migrated from rt.perl.org#128511 (status was 'resolved')

Searchable as RT128511$

@p6rt
Copy link
Author

p6rt commented Jul 1, 2016

From zefram@fysh.org

A decode-then-encode cycle through the utf8-c8 encoding is meant to
round-trip an octet string. But sometimes it adds a spurious NUL to
the end of the string​:

Blob[uint8].new(233).decode("utf8-c8").encode("utf8-c8").perl
Blob[uint8].new(233,0)
Blob[uint8].new(233, 128).decode("utf8-c8").encode("utf8-c8").perl
Blob[uint8].new(233,128,0)

This seems to happen whenever the end of the input string is a truncation
of the UTF-8 representation of a character in the Unicode range, and not
in any other case. In particular, for single-octet strings it happens
iff the octet value is between 194 (0xc2) and 244 (0xf4) inclusive.

-zefram

@p6rt
Copy link
Author

p6rt commented Feb 27, 2017

From zefram@fysh.org

This problem no longer occurs.

-zefram

@p6rt
Copy link
Author

p6rt commented Mar 14, 2018

From @dogbert17

On Sun, 26 Feb 2017 23​:02​:00 -0800, zefram@​fysh.org wrote​:

This problem no longer occurs.

-zefram

This was fixed with commit (2017-01-02) rakudo/rakudo@7c8b705

@p6rt
Copy link
Author

p6rt commented Mar 14, 2018

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Mar 17, 2018

From @dogbert17

On Wed, 14 Mar 2018 13​:26​:11 -0700, jan-olof.hendig@​bredband.net wrote​:

On Sun, 26 Feb 2017 23​:02​:00 -0800, zefram@​fysh.org wrote​:

This problem no longer occurs.

-zefram

This was fixed with commit (2017-01-02)
rakudo/rakudo@7c8b705

Tests added with commit Raku/roast@d0d4609

@p6rt
Copy link
Author

p6rt commented Mar 17, 2018

@dogbert17 - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant