Error parsing "[~" #9682

p5pRT · 2009-03-14T05:43:59Z

Migrated from rt.perl.org#63854 (status was 'resolved')

Searchable as RT63854$

p5pRT · 2009-03-14T05:44:01Z

From @ig3

Created by ian@alula.local

perl -e '$array[~0xFFFFFFFE]'

produces

syntax error at -e line 1, near "0xFFFFFFFE]"
Execution of -e aborted due to compilation errors.

Perl Info


Flags:
    category=core
    severity=low

Site configuration information for perl 5.11.0:

Configured by ian at Sat Mar 14 11:06:39 NZDT 2009.

Summary of my perl5 (revision 5 version 11 subversion 0) configuration:
  Commit id: 43eb98159eec9ba37935203e5bf9f3a4199380c5
  Platform:
    osname=linux, osvers=2.6.18-92.el5, archname=i686-linux
    uname='linux alula.local 2.6.18-92.el5 #1 smp tue jun 10 18:49:47 
edt 2008 i686 i686 i386 gnulinux '
    config_args='-Dprefix=/home/ian/localperl'
    hint=previous, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-DDEBUGGING -fno-strict-aliasing -pipe 
-fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE 
-D_FILE_OFFSET_BITS=64',
    optimize='-g',
    cppflags='-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector 
-I/usr/local/include -DDEBUGGING -fno-strict-aliasing -pipe 
-fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE 
-D_FILE_OFFSET_BITS=64'
    ccversion='', gccversion='4.1.2 20071124 (Red Hat 4.1.2-42)', 
gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=/lib/libc-2.5.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.5'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib 
-fstack-protector'

Locally applied patches:
    PERL_GIT_UNPUSHED_COMMITS        /* do not remove this line */
    PERL_GIT_UNCOMMITTED_CHANGES    /* do not remove this line */


@INC for perl 5.11.0:
    /home/ian/localperl/lib/site_perl/5.11.0/i686-linux
    /home/ian/localperl/lib/site_perl/5.11.0
    /home/ian/localperl/lib/5.11.0/i686-linux
    /home/ian/localperl/lib/5.11.0
    .


Environment for perl 5.11.0:
    HOME=/home/ian
    LANG=en_NZ.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    
PATH=/usr/local/java/jdk/bin:/usr/local/java/ant/bin:/usr/kerberos/bin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/usr/local/Trolltech/Qt-4.4.3/bin:/home/ian/bin:/sbin:/usr/local/Trolltech/Qt-4.4.3/bin
    PERLDOC_PAGER=less
    PERL_BADLANG (unset)
    SHELL=/bin/bash

p5pRT · 2009-03-14T06:16:19Z

From @ig3

The attached patch fixes the problem (apologies for the name) but I
don't know enough to suggest it might be the best solution.

With this patch and default configuration on linux (CentOS 5.2) the
problem was resolved and perl still passed all tests:

All tests successful.
u=10.92 s=2.08 cu=747.90 cs=55.63 scripts=1549 tests=208175

p5pRT · 2009-03-14T06:16:19Z

From @ig3

0001-Return-OPERATOR-for-without-falling-through.patch

From 7fa3894b37ebedd573cb0d33cffdf50bf63d1588 Mon Sep 17 00:00:00 2001
From: Ian Goodacre <Ian.Goodacre@xtra.co.nz>
Date: Sat, 14 Mar 2009 19:04:12 +1300
Subject: [PATCH] Return OPERATOR('[') for '[' without falling through to case '~',
 avoiding misinterpreting "[~" as an OP_SMARTMATCH.

---
 toke.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/toke.c b/toke.c
index 3c3e0ad..8f5795a 100644
--- a/toke.c
+++ b/toke.c
@@ -4287,7 +4287,10 @@ Perl_yylex(pTHX)
 	BOop(OP_BIT_XOR);
     case '[':
 	PL_lex_brackets++;
-	/* FALL THROUGH */
+	{
+	    const char tmp = *s++;
+	    OPERATOR(tmp);
+	}
     case '~':
 	if (s[1] == '~'
 	    && (PL_expect == XOPERATOR || PL_expect == XTERMORDORDOR))
-- 
1.6.1

p5pRT · 2009-03-14T09:38:26Z

From @ig3

FWIW, here are a couple of patches that add a new test to t/base/lex.t
to test that "[~" is scanned correctly.

Regards,
Ian Goodacre

p5pRT · 2009-03-14T09:42:39Z

From @ig3

FWIW, here are a couple of patches that add a new test to t/base/lex.t
to test that "[~" is scanned correctly.

Regards,
Ian Goodacre

p5pRT · 2009-03-14T09:42:39Z

From @ig3

0002-New-test-for-correct-scanning-of.patch

From a0e5ec1395c9ae9e9e648053f6c0e9509ec75e51 Mon Sep 17 00:00:00 2001
From: Ian Goodacre <Ian.Goodacre@xtra.co.nz>
Date: Sat, 14 Mar 2009 22:12:25 +1300
Subject: [PATCH 2/3] New test for correct scanning of "[~"

---
 t/base/lex.t |   10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)

diff --git a/t/base/lex.t b/t/base/lex.t
index f45e56c..1edcf01 100755
--- a/t/base/lex.t
+++ b/t/base/lex.t
@@ -1,6 +1,6 @@
 #!./perl
 
-print "1..56\n";
+print "1..57\n";
 
 $x = 'x';
 
@@ -267,3 +267,11 @@ foo::::::bar;
 eval "\$x =\xE2foo";
 if ($@ =~ /Unrecognized character \\xE2 in column 5/) { print "ok $test\n"; } else { print "not ok $test\n"; }
 $test++;
+
+# Is "[~" scanned correctly?
+eval {
+    my @a;
+    my $x = $a[~1]
+};
+print "not " if($@);
+print "ok 57\n";
-- 
1.6.1

p5pRT · 2009-03-14T09:42:39Z

From @ig3

0003-Corrected-test-for-scan-of.patch

From f384e88df46dfc474f629a29d39bacf0641676ae Mon Sep 17 00:00:00 2001
From: Ian Goodacre <Ian.Goodacre@xtra.co.nz>
Date: Sat, 14 Mar 2009 22:30:56 +1300
Subject: [PATCH 3/3] Corrected test for scan of "[~"

---
 t/base/lex.t |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/base/lex.t b/t/base/lex.t
index 1edcf01..9892df5 100755
--- a/t/base/lex.t
+++ b/t/base/lex.t
@@ -269,9 +269,9 @@ if ($@ =~ /Unrecognized character \\xE2 in column 5/) { print "ok $test\n"; } el
 $test++;
 
 # Is "[~" scanned correctly?
-eval {
+eval '
     my @a;
     my $x = $a[~1]
-};
+';
 print "not " if($@);
 print "ok 57\n";
-- 
1.6.1

p5pRT · 2009-03-14T12:01:05Z

From bitcard@profvince.com

This is indeed a 5.10 regression introduced with smart maching in
0d86345. Looks like the good way to fix it. Thanks, applied as df3467d.

p5pRT · 2009-03-14T12:01:06Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2009-03-14T19:56:47Z

From @ig3

Vincent Pit via RT wrote:

This is indeed a 5.10 regression introduced with smart maching in
0d86345. Looks like the good way to fix it. Thanks, applied as df3467d.

It occurred to me that the test I wrote wasn't very good. I think the
following is better. Sorry it's not as a patch - I don't know how to get
git to produce a single patch and I now have three changes. If I could
do better, I would be happy to have feedback or guidance.

Inline Patch

diff --git a/t/base/lex.t b/t/base/lex.t
index f45e56c..1b8045b 100755
--- a/t/base/lex.t
+++ b/t/base/lex.t
@@ -1,6 +1,6 @@
 #!./perl
 
-print "1..56\n";
+print "1..57\n";
 
 $x = 'x';
 
@@ -267,3 +267,8 @@ foo::::::bar;
 eval "\$x =\xE2foo";
 if ($@ =~ /Unrecognized character \\xE2 in column 5/) { print "ok

$test\n"; } e $test\+\+; \+ \+\# Is "\[~" scanned correctly? \+@a = $1\,2\,3$; \+print "not " unless$$a\[~~2\] == 3$; \+print "ok 57\\n";

Inline Patch

diff --git a/toke.c b/toke.c
index 3c3e0ad..8f5795a 100644
--- a/toke.c
+++ b/toke.c
@@ -4287,7 +4287,10 @@ Perl_yylex(pTHX)
        BOop(OP_BIT_XOR);
     case '[':
        PL_lex_brackets++;
-       /* FALL THROUGH */
+       {
+           const char tmp = *s++;
+           OPERATOR(tmp);
+       }
     case '~':
        if (s[1] == '~'
            && (PL_expect == XOPERATOR || PL_expect == XTERMORDORDOR))

p5pRT · 2009-03-14T22:30:57Z

@rgs - Status changed from 'open' to 'resolved'

p5pRT · 2009-03-14T22:30:59Z

From @rgs

2009/3/14 Ian Goodacre <Ian.Goodacre@xtra.co.nz>:

Vincent Pit via RT wrote:

This is indeed a 5.10 regression introduced with smart maching in
0d86345. Looks like the good way to fix it. Thanks, applied as df3467d.

It occurred to me that the test I wrote wasn't very good. I think the
following is better. Sorry it's not as a patch - I don't know how to get git
to produce a single patch and I now have three changes. If I could do
better, I would be happy to have feedback or guidance.

Thanks, I adjusted the test accordingly. (f50fa36)

So squash a series of commits, just rebase your changes on the current
origin head with the --squash option. But it's probably easier to just
commit with the --amend option, that allows to amend the previous
commit.

p5pRT closed this as completed Mar 14, 2009

p5pRT added Severity Low distro-Linux type-core labels Oct 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error parsing "[~" #9682

Error parsing "[~" #9682

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

Error parsing "[~" #9682

Error parsing "[~" #9682

Comments

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

From @ig3

Created by ian@alula.local

p5pRT commented Mar 14, 2009

From @ig3

p5pRT commented Mar 14, 2009

From @ig3

p5pRT commented Mar 14, 2009

From @ig3

p5pRT commented Mar 14, 2009

From @ig3

p5pRT commented Mar 14, 2009

From @ig3

p5pRT commented Mar 14, 2009

From @ig3

p5pRT commented Mar 14, 2009

From bitcard@profvince.com

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

From @ig3

p5pRT commented Mar 14, 2009

p5pRT commented Mar 14, 2009

From @rgs