New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
utf8.c:832: S_unexpected_non_continuation_text: Assertion `expect_len == UTF8SKIP(s)' failed. #16039
Comments
From @dur-randirCreated by @dur-randirWhile fuzzing perl v5.27.1-37-g4c95ee9f29 built with afl and run for(uc 0..t){0~~pack"UXp>",exp} to cause an assertion failure. This is a regression in 5.26, bisect points to: commit 7cf8d05 Add details to UTF-8 malformation error messages GDB info about the crash location is: (gdb) bt Perl Info
|
@khwilliamson - Status changed from 'new' to 'open' |
From @khwilliamsonThanks, fixed by I'm adding my vote for this to go into 5.26.1 -- |
@khwilliamson - Status changed from 'open' to 'pending release' |
From @tonycozOn Sat, 24 Jun 2017 11:09:30 -0700, khw wrote:
The test case: for(uc 0..t){0~~pack"UXp>",exp} here is fragile. The pack pattern here is: - pack as unicode (the error case triggers on exp(5)) So the test case depends on the high-byte of the PV in PL_sv_no being zero, which may not be the case, especially on 32-bit systems. Changing the pattern to: "UXc" should be less fragile (a zero byte will be packed instead of the high-byte of a pointer.) I discovered this while testing some other pack changes. Patch attached. Tony |
From @tonycoz0001-perl-131646-make-the-test-less-fragile.patchFrom 8e61f24b2b152a5e6a8bf22ddbab304ccdb413aa Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@debian9-x32.tony.develop-help.com>
Date: Tue, 8 Aug 2017 11:09:02 +1000
Subject: (perl #131646) make the test less fragile
The original pattern "UXp>" with the $_ that causes the failure, 5,
so we end up packing exp(5) or 148.... with U packs:
- U (148), producing C2 94, with the UTF8 flag set
- X - back up a byte,
- p> - write the address of PL_sv_no's PV in big-ending
The final p> will typically overwrite the 94 with a zero on 64-bit
systems, but with the smaller address space of 32-bit systems that
high-byte is much less likely to be 0, causing the comparison to fail.
Instead just pack a zero byte.
---
t/lib/warnings/utf8 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/t/lib/warnings/utf8 b/t/lib/warnings/utf8
index dfc58c12db..a9a6388d31 100644
--- a/t/lib/warnings/utf8
+++ b/t/lib/warnings/utf8
@@ -779,7 +779,7 @@ BEGIN{
}
no warnings;
use warnings 'utf8';
-for(uc 0..t){0~~pack"UXp>",exp}
+for(uc 0..t){0~~pack"UXc",exp}
EXPECT
OPTIONS regex
Malformed UTF-8 character: \\x([[:xdigit:]]{2})\\x([[:xdigit:]]{2}) \(unexpected non-continuation byte 0x\2, immediately after start byte 0x\1; need 2 bytes, got 1\) in smart match at - line 9.
--
2.11.0
|
From @tonycozOn Mon, 07 Aug 2017 18:13:07 -0700, tonyc wrote:
Actually, it doesn't depend on the high byte being zero, but it does depend on the high-byte not being a valid continuation byte, which might not be the case. Tony |
From @tonycozOn Mon, 07 Aug 2017 18:23:00 -0700, tonyc wrote:
Patch applied as 9c6b56d with a changed comment to reflect the above. Tony |
From @khwilliamsonThank you for filing this report. You have helped make Perl better. With the release yesterday of Perl 5.28.0, this and 185 other issues have been Perl 5.28.0 may be downloaded via: If you find that the problem persists, feel free to reopen this ticket. |
@khwilliamson - Status changed from 'pending release' to 'resolved' |
Migrated from rt.perl.org#131646 (status was 'resolved')
Searchable as RT131646$
The text was updated successfully, but these errors were encountered: