Skip Menu |
Report information
Id: 128701
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: ajs <ajs [at] ajs.com>
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: (no value)



From: Aaron Sherman <ajs [...] ajs.com>
Subject: Unicode handling of warnings appears broken in 5.18.2 and 5.24.0
To: perlbug [...] perl.org
Date: Fri, 22 Jul 2016 13:35:59 -0400
Download (untitled) / with headers
text/plain 685b
$ perl -C x.pl
Can't find string terminator "Â" anywhere before EOF at x.pl line 4.
$ cat x.pl
use v5.18.0;
use utf8;

say q« foo4 » ;
$perl -v
This is perl 5, version 18, subversion 2 (v5.18.2) built for x86_64-linux-gnu-thread-multi

Expected:
Can't find string terminator "«" anywhere before EOF at x.pl line 4.

Or at least an error about the use of an invalid punctuation character as a delimiter (perhaps "what do you think this is, Perl 6?!"). But definitely not munged latin-1...

Also tried this under 5.24.0 and got the same output.

--
Aaron Sherman, M.:
P: 617-440-4332 Google Talk, Email and Google Plus: ajs@ajs.com
Toolsmith, developer, gamer and life-long student.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.9k
On Fri Jul 22 10:36:25 2016, ajs wrote: Show quoted text
> $ perl -C x.pl > Can't find string terminator "Â" anywhere before EOF at x.pl line 4. > $ cat x.pl > use v5.18.0; > use utf8; > > say q« foo4 » ; > $perl -v > This is perl 5, version 18, subversion 2 (v5.18.2) built for > x86_64-linux-gnu-thread-multi > > Expected: > Can't find string terminator "«" anywhere before EOF at x.pl line 4. > > Or at least an error about the use of an invalid punctuation character as a > delimiter (perhaps "what do you think this is, Perl 6?!"). But definitely > not munged latin-1... > > Also tried this under 5.24.0 and got the same output.
The same bug occurs with the ‘(Might be a runaway multi-line...)’ hint that sometimes accompanies a syntax error. In a utf-8 terminal, with 5.12.4: $ perl -CS -e 'use utf8; q Ω' -e 'Ω >' syntax error at -e line 2, at EOF (Might be a runaway multi-line ?????????????????????????? string starting on line 1) Execution of -e aborted due to compilation errors. And 5.25.3: $ ./perl -Ilib -CS -e 'use utf8; q Ω' -e 'Ω >' Use of code point 0xFFFFFFFFFFFFFFCE is deprecated; the permissible max is 0x7FFFFFFFFFFFFFFF at -e line 2. Use of code point 0xFFFFFFFFFFFFFFCE is deprecated; the permissible max is 0x7FFFFFFFFFFFFFFF at -e line 2. Use of code point 0xFFFFFFFFFFFFFFCE is deprecated; the permissible max is 0x7FFFFFFFFFFFFFFF at -e line 2. Use of code point 0xFFFFFFFFFFFFFFCE is deprecated; the permissible max is 0x7FFFFFFFFFFFFFFF at -e line 2. syntax error at -e line 2, at EOF (Might be a runaway multi-line ?????????????????????????? string starting on line 1) Execution of -e aborted due to compilation errors. Ow! (Another thing I might note is that the runaway quote hint should include the operator if the delimiter is non-standard, because this is unhelpful: $ perl -e 'q x' -e 'x >' syntax error at -e line 2, at EOF (Might be a runaway multi-line xx string starting on line 1) Execution of -e aborted due to compilation errors. ) -- Father Chrysostomos
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.4k
On Tue Jul 26 00:04:19 2016, sprout wrote: Show quoted text
> On Fri Jul 22 10:36:25 2016, ajs wrote:
> > $ perl -C x.pl > > Can't find string terminator "Â" anywhere before EOF at x.pl line 4. > > $ cat x.pl > > use v5.18.0; > > use utf8; > > > > say q« foo4 » ; > > $perl -v > > This is perl 5, version 18, subversion 2 (v5.18.2) built for > > x86_64-linux-gnu-thread-multi > > > > Expected: > > Can't find string terminator "«" anywhere before EOF at x.pl line 4.
This is fixed in cb6501357. But there are problems with heredoc terminators (handled by the same code, more or less), so there is still some more fixing to do: $ ./perl -Ilib -CS -e 'use utf8; <<"«"' Can't find string terminator "«" anywhere before EOF at -e line 1. Show quoted text
> The same bug occurs with the ‘(Might be a runaway multi-line...)’ hint > that sometimes accompanies a syntax error. > > In a utf-8 terminal, with 5.12.4: > > $ perl -CS -e 'use utf8; q Ω' -e 'Ω >' > syntax error at -e line 2, at EOF > (Might be a runaway multi-line ?????????????????????????? string > starting on line 1) > Execution of -e aborted due to compilation errors.
That is not yet fixed. Show quoted text
> (Another thing I might note is that the runaway quote hint should > include the operator if the delimiter is non-standard, because this is > unhelpful: > > $ perl -e 'q x' -e 'x >' > syntax error at -e line 2, at EOF > (Might be a runaway multi-line xx string starting on line 1) > Execution of -e aborted due to compilation errors. > )
Neither is that. -- Father Chrysostomos
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.2k
On Tue Jul 26 01:55:10 2016, sprout wrote: Show quoted text
> On Tue Jul 26 00:04:19 2016, sprout wrote:
> > On Fri Jul 22 10:36:25 2016, ajs wrote:
> > > $ perl -C x.pl > > > Can't find string terminator "Â" anywhere before EOF at x.pl line > > > 4. > > > $ cat x.pl > > > use v5.18.0; > > > use utf8; > > > > > > say q« foo4 » ; > > > $perl -v > > > This is perl 5, version 18, subversion 2 (v5.18.2) built for > > > x86_64-linux-gnu-thread-multi > > > > > > Expected: > > > Can't find string terminator "«" anywhere before EOF at x.pl line > > > 4.
> > This is fixed in cb6501357. But there are problems with heredoc > terminators (handled by the same code, more or less), so there is > still some more fixing to do: > > $ ./perl -Ilib -CS -e 'use utf8; <<"«"' > Can't find string terminator "«" anywhere before EOF at -e line 1.
That one is fixed now in commit 7f7f7d0. Show quoted text
>
> > The same bug occurs with the ‘(Might be a runaway multi-line...)’ > > hint > > that sometimes accompanies a syntax error. > > > > In a utf-8 terminal, with 5.12.4: > > > > $ perl -CS -e 'use utf8; q Ω' -e 'Ω >' > > syntax error at -e line 2, at EOF > > (Might be a runaway multi-line ?????????????????????????? string > > starting on line 1) > > Execution of -e aborted due to compilation errors.
> > That is not yet fixed.
Actually, cb6501357 *did* fix it. I didn’t realise that Perl_sv_catpvf supports %c with numbers > 255, but apparently it does. However, it only goes up to 32 bits (at least on a Mac), because %c takes an int (which is 32 bits). That means you get garbled output for heredoc delimiters above \x{7fffffff}, but those are not valid Unicode characters anyway, and hence cannot display in a terminal. So I’m not going to worry about it. (I started to write a patch, but it adds to much complexity to the code.) Show quoted text
>
> > (Another thing I might note is that the runaway quote hint should > > include the operator if the delimiter is non-standard, because this > > is > > unhelpful: > > > > $ perl -e 'q x' -e 'x >' > > syntax error at -e line 2, at EOF > > (Might be a runaway multi-line xx string starting on line 1) > > Execution of -e aborted due to compilation errors. > > )
> > Neither is that.
I have decided to leave this alone. It’s not easy to fix, and nobody has complained about it. -- Father Chrysostomos
Download (untitled) / with headers
text/plain 313b
Thank you for filing this report. You have helped make Perl better. With the release today of Perl 5.26.0, this and 210 other issues have been resolved. Perl 5.26.0 may be downloaded via: https://metacpan.org/release/XSAWYERX/perl-5.26.0 If you find that the problem persists, feel free to reopen this ticket.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org