Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON::Tiny tests output bad text on latest build #6440

Closed
p6rt opened this issue Aug 10, 2017 · 7 comments
Closed

JSON::Tiny tests output bad text on latest build #6440

p6rt opened this issue Aug 10, 2017 · 7 comments
Labels
regression Issue did not exist previously

Comments

@p6rt
Copy link

p6rt commented Aug 10, 2017

Migrated from rt.perl.org#131881 (status was 'resolved')

Searchable as RT131881$

@p6rt
Copy link
Author

p6rt commented Aug 10, 2017

From @jdv

I updated to​:

[jdv@​localhost ~]$ perl6 -v
This is Rakudo version 2017.07-144-gec7bc25 built on MoarVM version
2017.07-365-gc0f7a3b
implementing Perl 6.c.
[jdv@​localhost ~]$

from a build from last weekend or so and the json tiny tests failed and
spit out some bad bytes​:

# Failed test 'JSON string «["̥̞ͫͩ̉̎͊ỏ̟͙̞̻̻̹̹r̵̨͇̫̦̺̮̼ͭ,̷̧̬̲͈̤̤̅̉͛ͫ͠
̣͕̝̳̗̲̩̣͗̋͘e̼ͩ̔̿̾̋̾͘a̸̙͙͔̜͙̳̻͊ͯͪͤͬ͊͆͠c̦̃̓̈̾͗́͝͝ḩ͉͓̦̯̞̆
̵͇͍ͪ̀́w̷ͩ͗̃͒͏̲̼̯̕i̞̫̜̮̣̰͎̅̂͋̀ͅń̢̨̪̭̠̼͕͎̐́͆̅ͅd̰̤̝͎̹͕ͩ̃̏̃̍͊̔ȏ̶̼̺̪̾ͨ̈̄ͤ̄̚̕w̙̻̱̙̺͎͚̙ͧͪ̊̒̕͝
̴̼̰̱͈̠ͤ̋̏̇͛&̠̺̟̞͎̳̻̙̽̓́̔̇ͩ͛ͧ̚ ̡̡̫̻͐̌̐̌ͬͦ̂͞"]» parsed'
# at t/01-parse.t line 208
# Looks like you failed 1 test of 93
Input (2 characters) is not a valid JSON string
  in sub from-json at
/Users/ugexe/.zef/store/json.git/957740bb22b122b711363328e224b56935873a61/lib/JSON/Tiny.pm
(JSON​::Tiny) line 59
  in block <unit> at t/03-unicode.t line 15

and then this happens​:

Aug 10 14​:29​:48 localhost.localdomain gnome-terminal-server[3375]​: **
Aug 10 14​:29​:48 localhost.localdomain gnome-terminal-server[3375]​:
Pango​:ERROR​:pango-glyph-item.c​:319​:pango_glyph_item_iter_next_cluster​:
assertion failed​: (iter->start_char < iter->end_char)
Aug 10 14​:29​:48 localhost.localdomain audit[3375]​: ANOM_ABEND auid=1000
uid=1000 gid=1000 ses=5 pid=3375 comm="gnome-terminal-"
exe="/usr/libexec/gnome-terminal-server" sig=6 res=1
Aug 10 14​:29​:48 localhost.localdomain abrt-hook-ccpp[3910]​: Process
3375 (gnome-terminal-server) of user 1000 killed by SIGABRT - dumping
core
Aug 10 14​:29​:48 localhost.localdomain systemd[2268]​:
gnome-terminal-server.service​: Main process exited, code=dumped,
status=6/ABRT
Aug 10 14​:29​:48 localhost.localdomain systemd[2268]​:
gnome-terminal-server.service​: Unit entered failed state.
Aug 10 14​:29​:48 localhost.localdomain systemd[2268]​:
gnome-terminal-server.service​: Failed with result 'core-dump'.

which is the log version of "all my gnome-terminal instances just
vanished...​:(".

@p6rt
Copy link
Author

p6rt commented Aug 10, 2017

@jdv - Status changed from 'new' to 'deleted'

@p6rt
Copy link
Author

p6rt commented Aug 10, 2017

@jdv - Status changed from 'deleted' to 'open'

@p6rt
Copy link
Author

p6rt commented Aug 10, 2017

From @AlexDaniel

It was bisected to rakudo/rakudo@f590863

Example (before and after)​: https://gist.github.com/Whateverable/2a9088ddcff37fd6f748b77ba3339af2

On 2017-08-10 14​:54​:24, jdv79 wrote​:

I updated to​:

[jdv@​localhost ~]$ perl6 -v
This is Rakudo version 2017.07-144-gec7bc25 built on MoarVM version
2017.07-365-gc0f7a3b
implementing Perl 6.c.
[jdv@​localhost ~]$

from a build from last weekend or so and the json tiny tests failed
and
spit out some bad bytes​:

# Failed test 'JSON string «["̥̞ͫͩ̉̎͊ỏ̟͙̞̻̻̹̹r̵̨͇̫̦̺̮̼ͭ,̷̧̬̲͈̤̤̅̉͛ͫ͠
̣͕̝̳̗̲̩̣͗̋͘e̼ͩ̔̿̾̋̾͘a̸̙͙͔̜͙̳̻͊ͯͪͤͬ͊͆͠c̦̃̓̈̾͗́͝͝ḩ͉͓̦̯̞̆
̵͇͍ͪ̀́w̷ͩ͗̃͒͏̲̼̯̕i̞̫̜̮̣̰͎̅̂͋̀ͅń̢̨̪̭̠̼͕͎̐́͆̅ͅd̰̤̝͎̹͕ͩ̃̏̃̍͊̔ȏ̶̼̺̪̾ͨ̈̄ͤ̄̚̕w̙̻̱̙̺͎͚̙ͧͪ̊̒̕͝
̴̼̰̱͈̠ͤ̋̏̇͛&̠̺̟̞͎̳̻̙̽̓́̔̇ͩ͛ͧ̚ ̡̡̫̻͐̌̐̌ͬͦ̂͞"]» parsed'
# at t/01-parse.t line 208
# Looks like you failed 1 test of 93
Input (2 characters) is not a valid JSON string
in sub from-json at
/Users/ugexe/.zef/store/json.git/957740bb22b122b711363328e224b56935873a61/lib/JSON/Tiny.pm
(JSON​::Tiny) line 59
in block <unit> at t/03-unicode.t line 15

and then this happens​:

Aug 10 14​:29​:48 localhost.localdomain gnome-terminal-server[3375]​: **
Aug 10 14​:29​:48 localhost.localdomain gnome-terminal-server[3375]​:
Pango​:ERROR​:pango-glyph-
item.c​:319​:pango_glyph_item_iter_next_cluster​:
assertion failed​: (iter->start_char < iter->end_char)
Aug 10 14​:29​:48 localhost.localdomain audit[3375]​: ANOM_ABEND
auid=1000
uid=1000 gid=1000 ses=5 pid=3375 comm="gnome-terminal-"
exe="/usr/libexec/gnome-terminal-server" sig=6 res=1
Aug 10 14​:29​:48 localhost.localdomain abrt-hook-ccpp[3910]​: Process
3375 (gnome-terminal-server) of user 1000 killed by SIGABRT - dumping
core
Aug 10 14​:29​:48 localhost.localdomain systemd[2268]​:
gnome-terminal-server.service​: Main process exited, code=dumped,
status=6/ABRT
Aug 10 14​:29​:48 localhost.localdomain systemd[2268]​:
gnome-terminal-server.service​: Unit entered failed state.
Aug 10 14​:29​:48 localhost.localdomain systemd[2268]​:
gnome-terminal-server.service​: Failed with result 'core-dump'.

which is the log version of "all my gnome-terminal instances just
vanished...​:(".

@p6rt
Copy link
Author

p6rt commented Aug 11, 2017

From @moritz

Background of JSON​::Tiny's weird code​:

JSON allows you to have quoted strings, and these strings can start with basically any Unicode codepoint, including combining characters. JSON​::Tiny failed to parse these strings, because Perl 6 combines the opening quote with the following (combining) codepoint to a single grapheme.

Since regexes don't have a codepoint mode (which would be appropriate for parsing JSON), the worakround was to parse the opening quote with :ignoremark. In the old rakudo version, the regex / (​:ignoremark '"') / parsed the quote plus combining character(s) in a single capture, and then transfered the combining characters to the start of the quoted string in the action method.

I would expect the following three lines to all print "True\n"​:

say so qq{"\c[ZERO WIDTH JOINER]a"} ~~ rx​:ignoremark/^ '"' /;
say so qq{"\c[ZERO WIDTH JOINER]a"} ~~ / :ignoremark ^ '"' /;
say so qq{"\c[ZERO WIDTH JOINER]a"} ~~ / (​:ignoremark ^ '"' ) /;

If somebody objects, please then clarify what you think the proper way is to parse JSON with a Perl 6 grammar.

@p6rt
Copy link
Author

p6rt commented Aug 14, 2017

From @samcv

I have fixed this as of this MoarVM commit​:

MoarVM/MoarVM@712cff3

Full explaination in the commit description.

Thanks a lot for reporting this bug :)

@p6rt
Copy link
Author

p6rt commented Aug 14, 2017

@samcv - Status changed from 'open' to 'resolved'

@p6rt p6rt closed this as completed Aug 14, 2017
@p6rt p6rt added the regression Issue did not exist previously label Jan 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
regression Issue did not exist previously
Projects
None yet
Development

No branches or pull requests

1 participant