Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PATCH] Remove support for ?PATTERN? without explicit 'm' operator #13504

Closed
p5pRT opened this issue Jan 1, 2014 · 19 comments
Closed

[PATCH] Remove support for ?PATTERN? without explicit 'm' operator #13504

p5pRT opened this issue Jan 1, 2014 · 19 comments
Labels

Comments

@p5pRT
Copy link

p5pRT commented Jan 1, 2014

Migrated from rt.perl.org#120912 (status was 'resolved')

Searchable as RT120912$

@p5pRT
Copy link
Author

p5pRT commented Jan 1, 2014

From @ilmari

This has issued a deprecation warning since Perl v5.14 (commit
725a61d), and precludes using ? as an operator after a unary operator
that defaults to $_, such as​:

  ref ? $_ : [$_]

--
"A disappointingly low fraction of the human race is,
at any given time, on fire." - Stig Sandbeck Mathisen

@p5pRT
Copy link
Author

p5pRT commented Jan 1, 2014

From @ilmari

0001-Remove-support-for-PATTERN-without-explicit-m-operat.patch
From 98ef8c47630cdba5f44c04ecb006dfd82eb8e1f8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilmari@ilmari.org>
Date: Wed, 1 Jan 2014 03:41:25 +0100
Subject: [PATCH] Remove support for ?PATTERN? without explicit 'm' operator

This has issued a deprecation warning since v5.14 (commit 725a61d70),
and precludes using ? as an operator after a unary operator that
defaults to $_, such as:

   ref ? $_ : [$_]
---
 pod/perldelta.pod   |  8 ++++++
 pod/perldiag.pod    |  9 ------
 t/lib/warnings/toke | 15 +++++-----
 toke.c              | 80 +++++++++++++++++++++--------------------------------
 4 files changed, 47 insertions(+), 65 deletions(-)

diff --git a/pod/perldelta.pod b/pod/perldelta.pod
index e36ae85..48b11ea 100644
--- a/pod/perldelta.pod
+++ b/pod/perldelta.pod
@@ -74,6 +74,14 @@ possible code points; that is, it is equivalent to C<qr/./s>.  Thus
 C<\p{All}> is no longer synonymous with C<\p{Any}>, which continues to
 match just the Unicode code points, as Unicode says it should.
 
+=head2 Support for C<?PATTERN?> without explicit operator has been removed
+
+Starting regular expressions matching only once directly with the
+question mark delimiter is now a syntax error, so that the question mark
+can be available for use in new operators.  Write C<m?PATTERN?> instead,
+explicitly using the C<m> operator: the question mark delimiter still
+invokes match-once behaviour.
+
 =head1 Deprecations
 
 XXX Any deprecated features, syntax, modules etc. should be listed here.
diff --git a/pod/perldiag.pod b/pod/perldiag.pod
index 61d144a..b979e6e 100644
--- a/pod/perldiag.pod
+++ b/pod/perldiag.pod
@@ -6218,15 +6218,6 @@ no way to tie the scalar itself when it held a typeglob, and no way to
 untie a scalar that had had a typeglob assigned to it.  If you see this
 message, you must be using an older version.
 
-=item Use of ?PATTERN? without explicit operator is deprecated
-
-(D deprecated) You have written something like C<?\w?>, for a regular
-expression that matches only once.  Starting this term directly with
-the question mark delimiter is now deprecated, so that the question mark
-will be available for use in new operators in the future.  Write C<m?\w?>
-instead, explicitly using the C<m> operator: the question mark delimiter
-still invokes match-once behaviour.
-
 =item Use of reference "%s" as array index
 
 (W misc) You tried to use a reference as an array index; this probably
diff --git a/t/lib/warnings/toke b/t/lib/warnings/toke
index 02c6ccc..56fbfe9 100644
--- a/t/lib/warnings/toke
+++ b/t/lib/warnings/toke
@@ -1229,7 +1229,6 @@ $_ = $a = 1;
 $a !=~  /1/;
 $a !=~ m#1#;
 $a !=~/1/;
-$a !=~ ?/?;
 $a !=~ y/1//;
 $a !=~ tr/1//;
 $a !=~ s/1//;
@@ -1238,7 +1237,6 @@ no warnings "syntax";
 $a !=~  /1/;
 $a !=~ m#1#;
 $a !=~/1/;
-$a !=~ ?/?;
 $a !=~ y/1//;
 $a !=~ tr/1//;
 $a !=~ s/1//;
@@ -1249,7 +1247,6 @@ EXPECT
 !=~ should be !~ at - line 7.
 !=~ should be !~ at - line 8.
 !=~ should be !~ at - line 9.
-!=~ should be !~ at - line 10.
 ########
 # toke.c
 our $foo :unique;
@@ -1351,13 +1348,17 @@ Non-octal character '8'.  Resolved as "\o{123}" at - line 3.
 ########
 # toke.c
 use warnings;
-my $a = "foo";
-print $a =~ ?f? ? "yes\n" : "no\n" foreach 0..2;
+print ref ? "yes\n" : "no\n" foreach [], ''; # ? is unambiguosly an operator
 EXPECT
-Use of ?PATTERN? without explicit operator is deprecated at - line 4.
 yes
 no
-no
+########
+# toke .c
+use warnings;
+$a =~ ?rand?; # ? is not a regex match
+EXPECT
+syntax error at - line 3, near "=~ ?"
+Execution of - aborted due to compilation errors.
 ########
 # toke.c
 use warnings;
diff --git a/toke.c b/toke.c
index 8ac0f31..90ee12b 100644
--- a/toke.c
+++ b/toke.c
@@ -6748,61 +6748,43 @@ If the next character is in (or extends into) the next chunk of input
 	TERM('@');
 
      case '/':			/* may be division, defined-or, or pattern */
-	if (PL_expect == XTERMORDORDOR && s[1] == '/') {
+	if ((PL_expect == XOPERATOR || PL_expect == XTERMORDORDOR) && s[1] == '/') {
 	    if (!PL_lex_allbrackets && PL_lex_fakeeof >=
 		    (s[2] == '=' ? LEX_FAKEEOF_ASSIGN : LEX_FAKEEOF_LOGIC))
 		TOKEN(0);
 	    s += 2;
 	    AOPERATOR(DORDOR);
 	}
-     case '?':			/* may either be conditional or pattern */
-	if (PL_expect == XOPERATOR) {
-	     char tmp = *s++;
-	     if(tmp == '?') {
-		if (!PL_lex_allbrackets &&
-			PL_lex_fakeeof >= LEX_FAKEEOF_IFELSE) {
-		    s--;
-		    TOKEN(0);
-		}
-		PL_lex_allbrackets++;
-		OPERATOR('?');
-	     }
-             else {
-	         tmp = *s++;
-	         if(tmp == '/') {
-	             /* A // operator. */
-		    if (!PL_lex_allbrackets && PL_lex_fakeeof >=
-			    (*s == '=' ? LEX_FAKEEOF_ASSIGN :
-					    LEX_FAKEEOF_LOGIC)) {
-			s -= 2;
-			TOKEN(0);
-		    }
-	            AOPERATOR(DORDOR);
-	         }
-	         else {
-	             s--;
-		     if (*s == '=' && !PL_lex_allbrackets &&
-			     PL_lex_fakeeof >= LEX_FAKEEOF_ASSIGN) {
-			 s--;
-			 TOKEN(0);
-		     }
-	             Mop(OP_DIVIDE);
-	         }
-	     }
-	 }
-	 else {
-	     /* Disable warning on "study /blah/" */
-	     if (PL_oldoldbufptr == PL_last_uni
-	      && (*PL_last_uni != 's' || s - PL_last_uni < 5
-	          || memNE(PL_last_uni, "study", 5)
-	          || isWORDCHAR_lazy_if(PL_last_uni+5,UTF)
-	      ))
-	         check_uni();
-	     if (*s == '?')
-		 deprecate("?PATTERN? without explicit operator");
-	     s = scan_pat(s,OP_MATCH);
-	     TERM(sublex_start());
-	 }
+	else if (PL_expect == XOPERATOR) {
+	    s++;
+	    if (*s == '=' && !PL_lex_allbrackets &&
+		PL_lex_fakeeof >= LEX_FAKEEOF_ASSIGN) {
+		s--;
+		TOKEN(0);
+	    }
+	    Mop(OP_DIVIDE);
+        }
+	else {
+	    /* Disable warning on "study /blah/" */
+	    if (PL_oldoldbufptr == PL_last_uni
+	     && (*PL_last_uni != 's' || s - PL_last_uni < 5
+	         || memNE(PL_last_uni, "study", 5)
+	         || isWORDCHAR_lazy_if(PL_last_uni+5,UTF)
+	     ))
+	        check_uni();
+	    s = scan_pat(s,OP_MATCH);
+	    TERM(sublex_start());
+	}
+
+     case '?':			/* conditional */
+	s++;
+	if (!PL_lex_allbrackets &&
+	    PL_lex_fakeeof >= LEX_FAKEEOF_IFELSE) {
+	    s--;
+	    TOKEN(0);
+	}
+	PL_lex_allbrackets++;
+	OPERATOR('?');
 
     case '.':
 	if (PL_lex_formbrack && PL_lex_brackets == PL_lex_formbrack
-- 
1.8.3.2

@p5pRT
Copy link
Author

p5pRT commented Jan 2, 2014

From @Leont

On Wed, Jan 1, 2014 at 3​:12 PM, Dagfinn Ilmari Mannsåker <
perlbug-followup@​perl.org> wrote​:

This has issued a deprecation warning since Perl v5.14 (commit
725a61d), and precludes using ? as an operator after a unary operator
that defaults to $_, such as​:

ref ? $_ : [$_]

--
"A disappointingly low fraction of the human race is,
at any given time, on fire." - Stig Sandbeck Mathisen

From 98ef8c4 Mon Sep 17 00​:00​:00 2001
From​: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilmari@​ilmari.org>
Date​: Wed, 1 Jan 2014 03​:41​:25 +0100
Subject​: [PATCH] Remove support for ?PATTERN? without explicit 'm' operator

This has issued a deprecation warning since v5.14 (commit 725a61d),
and precludes using ? as an operator after a unary operator that
defaults to $_, such as​:

ref ? $_ : [$_]
---
pod/perldelta.pod | 8 ++++++
pod/perldiag.pod | 9 ------
t/lib/warnings/toke | 15 +++++-----
toke.c | 80
+++++++++++++++++++++--------------------------------
4 files changed, 47 insertions(+), 65 deletions(-)

diff --git a/pod/perldelta.pod b/pod/perldelta.pod
index e36ae85..48b11ea 100644
--- a/pod/perldelta.pod
+++ b/pod/perldelta.pod
@​@​ -74,6 +74,14 @​@​ possible code points; that is, it is equivalent to
C<qr/./s>. Thus
C<\p{All}> is no longer synonymous with C<\p{Any}>, which continues to
match just the Unicode code points, as Unicode says it should.

+=head2 Support for C<?PATTERN?> without explicit operator has been removed
+
+Starting regular expressions matching only once directly with the
+question mark delimiter is now a syntax error, so that the question mark
+can be available for use in new operators. Write C<m?PATTERN?> instead,
+explicitly using the C<m> operator​: the question mark delimiter still
+invokes match-once behaviour.
+
=head1 Deprecations

XXX Any deprecated features, syntax, modules etc. should be listed here.
diff --git a/pod/perldiag.pod b/pod/perldiag.pod
index 61d144a..b979e6e 100644
--- a/pod/perldiag.pod
+++ b/pod/perldiag.pod
@​@​ -6218,15 +6218,6 @​@​ no way to tie the scalar itself when it held a
typeglob, and no way to
untie a scalar that had had a typeglob assigned to it. If you see this
message, you must be using an older version.

-=item Use of ?PATTERN? without explicit operator is deprecated
-
-(D deprecated) You have written something like C<?\w?>, for a regular
-expression that matches only once. Starting this term directly with
-the question mark delimiter is now deprecated, so that the question mark
-will be available for use in new operators in the future. Write C<m?\w?>
-instead, explicitly using the C<m> operator​: the question mark delimiter
-still invokes match-once behaviour.
-
=item Use of reference "%s" as array index

(W misc) You tried to use a reference as an array index; this probably
diff --git a/t/lib/warnings/toke b/t/lib/warnings/toke
index 02c6ccc..56fbfe9 100644
--- a/t/lib/warnings/toke
+++ b/t/lib/warnings/toke
@​@​ -1229,7 +1229,6 @​@​ $_ = $a = 1;
$a !=~ /1/;
$a !=~ m#1#;
$a !=/1/;
-$a !=
?/?;
$a !=~ y/1//;
$a !=~ tr/1//;
$a !=~ s/1//;
@​@​ -1238,7 +1237,6 @​@​ no warnings "syntax";
$a !=~ /1/;
$a !=~ m#1#;
$a !=/1/;
-$a !=
?/?;
$a !=~ y/1//;
$a !=~ tr/1//;
$a !=~ s/1//;
@​@​ -1249,7 +1247,6 @​@​ EXPECT
!=~ should be !~ at - line 7.
!=~ should be !~ at - line 8.
!=~ should be !~ at - line 9.
-!=~ should be !~ at - line 10.
########
# toke.c
our $foo :unique;
@​@​ -1351,13 +1348,17 @​@​ Non-octal character '8'. Resolved as "\o{123}" at
- line 3.
########
# toke.c
use warnings;
-my $a = "foo";
-print $a =~ ?f? ? "yes\n" : "no\n" foreach 0..2;
+print ref ? "yes\n" : "no\n" foreach [], ''; # ? is unambiguosly an
operator
EXPECT
-Use of ?PATTERN? without explicit operator is deprecated at - line 4.
yes
no
-no
+########
+# toke .c
+use warnings;
+$a =~ ?rand?; # ? is not a regex match
+EXPECT
+syntax error at - line 3, near "=~ ?"
+Execution of - aborted due to compilation errors.
########
# toke.c
use warnings;
diff --git a/toke.c b/toke.c
index 8ac0f31..90ee12b 100644
--- a/toke.c
+++ b/toke.c
@​@​ -6748,61 +6748,43 @​@​ If the next character is in (or extends into) the
next chunk of input
TERM('@​');

  case '/'&#8203;:                 /\* may be division\, defined\-or\, or pattern

*/
- if (PL_expect == XTERMORDORDOR && s[1] == '/') {
+ if ((PL_expect == XOPERATOR || PL_expect == XTERMORDORDOR) && s[1]
== '/') {
if (!PL_lex_allbrackets && PL_lex_fakeeof >=
(s[2] == '=' ? LEX_FAKEEOF_ASSIGN : LEX_FAKEEOF_LOGIC))
TOKEN(0);
s += 2;
AOPERATOR(DORDOR);
}
- case '?'​: /* may either be conditional or pattern */
- if (PL_expect == XOPERATOR) {
- char tmp = *s++;
- if(tmp == '?') {
- if (!PL_lex_allbrackets &&
- PL_lex_fakeeof >= LEX_FAKEEOF_IFELSE) {
- s--;
- TOKEN(0);
- }
- PL_lex_allbrackets++;
- OPERATOR('?');
- }
- else {
- tmp = *s++;
- if(tmp == '/') {
- /* A // operator. */
- if (!PL_lex_allbrackets && PL_lex_fakeeof >=
- (*s == '=' ? LEX_FAKEEOF_ASSIGN :
- LEX_FAKEEOF_LOGIC)) {
- s -= 2;
- TOKEN(0);
- }
- AOPERATOR(DORDOR);
- }
- else {
- s--;
- if (*s == '=' && !PL_lex_allbrackets &&
- PL_lex_fakeeof >= LEX_FAKEEOF_ASSIGN) {
- s--;
- TOKEN(0);
- }
- Mop(OP_DIVIDE);
- }
- }
- }
- else {
- /* Disable warning on "study /blah/" */
- if (PL_oldoldbufptr == PL_last_uni
- && (*PL_last_uni != 's' || s - PL_last_uni < 5
- || memNE(PL_last_uni, "study", 5)
- || isWORDCHAR_lazy_if(PL_last_uni+5,UTF)
- ))
- check_uni();
- if (*s == '?')
- deprecate("?PATTERN? without explicit operator");
- s = scan_pat(s,OP_MATCH);
- TERM(sublex_start());
- }
+ else if (PL_expect == XOPERATOR) {
+ s++;
+ if (*s == '=' && !PL_lex_allbrackets &&
+ PL_lex_fakeeof >= LEX_FAKEEOF_ASSIGN) {
+ s--;
+ TOKEN(0);
+ }
+ Mop(OP_DIVIDE);
+ }
+ else {
+ /* Disable warning on "study /blah/" */
+ if (PL_oldoldbufptr == PL_last_uni
+ && (*PL_last_uni != 's' || s - PL_last_uni < 5
+ || memNE(PL_last_uni, "study", 5)
+ || isWORDCHAR_lazy_if(PL_last_uni+5,UTF)
+ ))
+ check_uni();
+ s = scan_pat(s,OP_MATCH);
+ TERM(sublex_start());
+ }
+
+ case '?'​: /* conditional */
+ s++;
+ if (!PL_lex_allbrackets &&
+ PL_lex_fakeeof >= LEX_FAKEEOF_IFELSE) {
+ s--;
+ TOKEN(0);
+ }
+ PL_lex_allbrackets++;
+ OPERATOR('?');

 case '\.'&#8203;:
    if \(PL\_lex\_formbrack && PL\_lex\_brackets == PL\_lex\_formbrack

--
1.8.3.2

+1

This syntax is very much standing in the way. I'd love us to get rid of it.

Leon

@p5pRT
Copy link
Author

p5pRT commented Jan 2, 2014

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jan 2, 2014

From @cpansprout

On Thu Jan 02 01​:03​:54 2014, LeonT wrote​:

+1

This syntax is very much standing in the way. I'd love us to get rid
of it.

Standing in the way of what?

IIRC, this was deprecated so that we could introduce new prefix opera-
tors beginning with ?.

If we simply remove this, then ref ? foo : bar becomes valid syntax,
and we will no longer be able to introduce prefix operators without
breaking existing code.

So getting rid of this syntax now to get it ‘out of the way’ will
actually cause backward compatibility problems later, unless we decide
*now* what we want to do with that syntactic niche.

Maybe we should wait till after 5.20, so we have plenty of time to
think about it and possibly revert it.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jan 2, 2014

From @ilmari

The two attached patches remove the now obsolete "ternary operator
parsed as search pattern" warning and fix B​::Deparse to not emit
?PATTERN? for m?PATTERN?.

Feel free to apply the latter before the original patch in this ticket
(rewording the commit message to future tense) if desired. Not doing so
does not break the build, since there were no tests for deparsing
m?PATTERN?.

--
- Twitter seems more influential [than blogs] in the 'gets reported in
  the mainstream press' sense at least. - Matt McLeod
- That'd be because the content of a tweet is easier to condense down
  to a mainstream media article. - Calle Dybedahl

@p5pRT
Copy link
Author

p5pRT commented Jan 2, 2014

From @ilmari

0002-Remove-ternary-operator-parsed-as-search-pattern-war.patch
From 70f2deab27dbc20b6c6d13dcd960a6160179efc4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilmari@ilmari.org>
Date: Thu, 2 Jan 2014 14:06:51 +0100
Subject: [PATCH 2/3] Remove "ternary operator parsed as search pattern"
 warning

Since ?PATTERN? without 'm' is no longer supported, there is no
ambiguity any more.
---
 pod/perldiag.pod | 10 ----------
 toke.c           | 10 ++--------
 2 files changed, 2 insertions(+), 18 deletions(-)

diff --git a/pod/perldiag.pod b/pod/perldiag.pod
index b979e6e..41b3050 100644
--- a/pod/perldiag.pod
+++ b/pod/perldiag.pod
@@ -4711,16 +4711,6 @@ construct, not just the empty search pattern.  Therefore code written
 in Perl 5.9.0 or later that uses the // as the I<defined-or> can be
 misparsed by pre-5.9.0 Perls as a non-terminated search pattern.
 
-=item Search pattern not terminated or ternary operator parsed as search pattern
-
-(F) The lexer couldn't find the final delimiter of a C<?PATTERN?>
-construct.
-
-The question mark is also used as part of the ternary operator (as in
-C<foo ? 0 : 1>) leading to some ambiguous constructions being wrongly
-parsed.  One way to disambiguate the parsing is to put parentheses around
-the conditional expression, i.e. C<(foo) ? 0 : 1>.
-
 =item seekdir() attempted on invalid dirhandle %s
 
 (W io) The dirhandle you are doing a seekdir() on is either closed or not
diff --git a/toke.c b/toke.c
index 90ee12b..28cc9e5 100644
--- a/toke.c
+++ b/toke.c
@@ -9701,14 +9701,8 @@ If the next character is in (or extends into) the next chunk of input
     s = scan_str(start,!!PL_madskills,FALSE, (PL_in_eval & EVAL_RE_REPARSING),
                        TRUE /* look for escaped bracketed metas */, NULL);
 
-    if (!s) {
-	const char * const delimiter = skipspace(start);
-	Perl_croak(aTHX_
-		   (const char *)
-		   (*delimiter == '?'
-		    ? "Search pattern not terminated or ternary operator parsed as search pattern"
-		    : "Search pattern not terminated" ));
-    }
+    if (!s)
+	Perl_croak(aTHX_ "Search pattern not terminated");
 
     pm = (PMOP*)newPMOP(type, 0);
     if (PL_multi_open == '?') {
-- 
1.8.3.2

@p5pRT
Copy link
Author

p5pRT commented Jan 2, 2014

From @ilmari

0003-Preserve-explicit-m-when-deparsing-m-PATTERN.patch
From 224a1d70c6b2efa2cf421d82c72abecc72d87735 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilmari@ilmari.org>
Date: Thu, 2 Jan 2014 14:18:18 +0100
Subject: [PATCH 3/3] Preserve explicit 'm' when deparsing m?PATTERN?

Bare ?PATTERN? is no longer allowed
---
 lib/B/Deparse.pm | 2 +-
 lib/B/Deparse.t  | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/B/Deparse.pm b/lib/B/Deparse.pm
index 31ad998..d1f2385 100644
--- a/lib/B/Deparse.pm
+++ b/lib/B/Deparse.pm
@@ -4774,7 +4774,7 @@ sub matchop {
     $flags = $matchwords{$flags} if $matchwords{$flags};
     if ($pmflags & PMf_ONCE) { # only one kind of delimiter works here
 	$re =~ s/\?/\\?/g;
-	$re = "?$re?";
+	$re = "m?$re?";        # explicit 'm' is required
     } elsif ($quote) {
 	$re = single_delim($name, $delim, $re);
     }
diff --git a/lib/B/Deparse.t b/lib/B/Deparse.t
index fd727c2..9b1ec41 100644
--- a/lib/B/Deparse.t
+++ b/lib/B/Deparse.t
@@ -1423,3 +1423,6 @@ print f();
 ####
 # Elements of %# should not be confused with $#{ array }
 () = ${#}{'foo'};
+####
+# 'm' must be preserved in m??
+m??;
-- 
1.8.3.2

@p5pRT
Copy link
Author

p5pRT commented Feb 10, 2014

From @rjbs

On Wed Jan 01 06​:12​:06 2014, ilmari wrote​:

This has issued a deprecation warning since Perl v5.14 (commit
725a61d), and precludes using ? as an operator after a unary operator
that defaults to $_, such as​:

ref ? $_ : [$_]

I am in favor of merging this in 5.21.0, and have marked this as a 5.22.0 blocker.

--
rjbs

@p5pRT
Copy link
Author

p5pRT commented Jun 6, 2014

From @ilmari

Ricardo Signes <perl.p5p@​rjbs.manxome.org> writes​:

Default-on deprecations warnings have been issued for ?PATTERN? since 5.14.0
(2011). If we're going to fatalize this behavior this series, best to do it
early.

Attached are the patches from
https://rt.perl.org/Public/Bug/Display.html?id=120912 rebased onto
current blead.

--
"I use RMS as a guide in the same way that a boat captain would use
a lighthouse. It's good to know where it is, but you generally
don't want to find yourself in the same spot." - Tollef Fog Heen

@p5pRT
Copy link
Author

p5pRT commented Jun 6, 2014

From @ilmari

0001-Remove-support-for-PATTERN-without-explicit-m-operat.patch
From c71198879c83025d888dde3f623e90a0d9fab3ec Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilmari@ilmari.org>
Date: Wed, 1 Jan 2014 03:41:25 +0100
Subject: [PATCH 1/3] Remove support for ?PATTERN? without explicit 'm'
 operator

This has issued a deprecation warning since v5.14 (commit 725a61d70),
and precludes using ? as an operator after a unary operator that
defaults to $_, such as:

   ref ? $_ : [$_]
---
 pod/perldelta.pod   |  8 ++++++
 pod/perldiag.pod    |  9 ------
 t/lib/warnings/toke | 15 +++++-----
 toke.c              | 81 ++++++++++++++++++++---------------------------------
 4 files changed, 47 insertions(+), 66 deletions(-)

diff --git a/pod/perldelta.pod b/pod/perldelta.pod
index 21e59eb..1b0a651 100644
--- a/pod/perldelta.pod
+++ b/pod/perldelta.pod
@@ -117,6 +117,14 @@ be affected by locale; only C<$!> and C<$^E> stringification.  The
 'bytes' pragma causes the UTF-8 flag to not be set, just as in previous
 Perl releases.  This resolves [perl #112208].
 
+=head2 Support for C<?PATTERN?> without explicit operator has been removed
+
+Starting regular expressions matching only once directly with the
+question mark delimiter is now a syntax error, so that the question mark
+can be available for use in new operators.  Write C<m?PATTERN?> instead,
+explicitly using the C<m> operator: the question mark delimiter still
+invokes match-once behaviour.
+
 =head1 Deprecations
 
 XXX Any deprecated features, syntax, modules etc. should be listed here.
diff --git a/pod/perldiag.pod b/pod/perldiag.pod
index 74f2e49..aa868f0 100644
--- a/pod/perldiag.pod
+++ b/pod/perldiag.pod
@@ -6379,15 +6379,6 @@ no way to tie the scalar itself when it held a typeglob, and no way to
 untie a scalar that had had a typeglob assigned to it.  If you see this
 message, you must be using an older version.
 
-=item Use of ?PATTERN? without explicit operator is deprecated
-
-(D deprecated) You have written something like C<?\w?>, for a regular
-expression that matches only once.  Starting this term directly with
-the question mark delimiter is now deprecated, so that the question mark
-will be available for use in new operators in the future.  Write C<m?\w?>
-instead, explicitly using the C<m> operator: the question mark delimiter
-still invokes match-once behaviour.
-
 =item Use of reference "%s" as array index
 
 (W misc) You tried to use a reference as an array index; this probably
diff --git a/t/lib/warnings/toke b/t/lib/warnings/toke
index 92e94ad..7f2b730 100644
--- a/t/lib/warnings/toke
+++ b/t/lib/warnings/toke
@@ -1229,7 +1229,6 @@ $_ = $a = 1;
 $a !=~  /1/;
 $a !=~ m#1#;
 $a !=~/1/;
-$a !=~ ?/?;
 $a !=~ y/1//;
 $a !=~ tr/1//;
 $a !=~ s/1//;
@@ -1238,7 +1237,6 @@ no warnings "syntax";
 $a !=~  /1/;
 $a !=~ m#1#;
 $a !=~/1/;
-$a !=~ ?/?;
 $a !=~ y/1//;
 $a !=~ tr/1//;
 $a !=~ s/1//;
@@ -1249,7 +1247,6 @@ EXPECT
 !=~ should be !~ at - line 7.
 !=~ should be !~ at - line 8.
 !=~ should be !~ at - line 9.
-!=~ should be !~ at - line 10.
 ########
 # toke.c
 our $foo :unique;
@@ -1351,13 +1348,17 @@ Non-octal character '8'.  Resolved as "\o{123}" at - line 3.
 ########
 # toke.c
 use warnings;
-my $a = "foo";
-print $a =~ ?f? ? "yes\n" : "no\n" foreach 0..2;
+print ref ? "yes\n" : "no\n" foreach [], ''; # ? is unambiguosly an operator
 EXPECT
-Use of ?PATTERN? without explicit operator is deprecated at - line 4.
 yes
 no
-no
+########
+# toke .c
+use warnings;
+$a =~ ?rand?; # ? is not a regex match
+EXPECT
+syntax error at - line 3, near "=~ ?"
+Execution of - aborted due to compilation errors.
 ########
 # toke.c
 use warnings;
diff --git a/toke.c b/toke.c
index ddb5cd4..8c9e63c 100644
--- a/toke.c
+++ b/toke.c
@@ -6763,62 +6763,43 @@ Perl_yylex(pTHX)
 	TERM('@');
 
      case '/':			/* may be division, defined-or, or pattern */
-	if (PL_expect == XTERMORDORDOR && s[1] == '/') {
+	if ((PL_expect == XOPERATOR || PL_expect == XTERMORDORDOR) && s[1] == '/') {
 	    if (!PL_lex_allbrackets && PL_lex_fakeeof >=
 		    (s[2] == '=' ? LEX_FAKEEOF_ASSIGN : LEX_FAKEEOF_LOGIC))
 		TOKEN(0);
 	    s += 2;
 	    AOPERATOR(DORDOR);
 	}
-	/* FALLTHROUGH */
-     case '?':			/* may either be conditional or pattern */
-	if (PL_expect == XOPERATOR) {
-	     char tmp = *s++;
-	     if(tmp == '?') {
-		if (!PL_lex_allbrackets &&
-			PL_lex_fakeeof >= LEX_FAKEEOF_IFELSE) {
-		    s--;
-		    TOKEN(0);
-		}
-		PL_lex_allbrackets++;
-		OPERATOR('?');
-	     }
-             else {
-	         tmp = *s++;
-	         if(tmp == '/') {
-	             /* A // operator. */
-		    if (!PL_lex_allbrackets && PL_lex_fakeeof >=
-			    (*s == '=' ? LEX_FAKEEOF_ASSIGN :
-					    LEX_FAKEEOF_LOGIC)) {
-			s -= 2;
-			TOKEN(0);
-		    }
-	            AOPERATOR(DORDOR);
-	         }
-	         else {
-	             s--;
-		     if (*s == '=' && !PL_lex_allbrackets &&
-			     PL_lex_fakeeof >= LEX_FAKEEOF_ASSIGN) {
-			 s--;
-			 TOKEN(0);
-		     }
-	             Mop(OP_DIVIDE);
-	         }
-	     }
-	 }
-	 else {
-	     /* Disable warning on "study /blah/" */
-	     if (PL_oldoldbufptr == PL_last_uni
-	      && (*PL_last_uni != 's' || s - PL_last_uni < 5
-	          || memNE(PL_last_uni, "study", 5)
-	          || isWORDCHAR_lazy_if(PL_last_uni+5,UTF)
-	      ))
-	         check_uni();
-	     if (*s == '?')
-		 deprecate("?PATTERN? without explicit operator");
-	     s = scan_pat(s,OP_MATCH);
-	     TERM(sublex_start());
-	 }
+	else if (PL_expect == XOPERATOR) {
+	    s++;
+	    if (*s == '=' && !PL_lex_allbrackets &&
+		PL_lex_fakeeof >= LEX_FAKEEOF_ASSIGN) {
+		s--;
+		TOKEN(0);
+	    }
+	    Mop(OP_DIVIDE);
+        }
+	else {
+	    /* Disable warning on "study /blah/" */
+	    if (PL_oldoldbufptr == PL_last_uni
+	     && (*PL_last_uni != 's' || s - PL_last_uni < 5
+	         || memNE(PL_last_uni, "study", 5)
+	         || isWORDCHAR_lazy_if(PL_last_uni+5,UTF)
+	     ))
+	        check_uni();
+	    s = scan_pat(s,OP_MATCH);
+	    TERM(sublex_start());
+	}
+
+     case '?':			/* conditional */
+	s++;
+	if (!PL_lex_allbrackets &&
+	    PL_lex_fakeeof >= LEX_FAKEEOF_IFELSE) {
+	    s--;
+	    TOKEN(0);
+	}
+	PL_lex_allbrackets++;
+	OPERATOR('?');
 
     case '.':
 	if (PL_lex_formbrack && PL_lex_brackets == PL_lex_formbrack
-- 
2.0.0

@p5pRT
Copy link
Author

p5pRT commented Jun 6, 2014

From @ilmari

0002-Remove-ternary-operator-parsed-as-search-pattern-war.patch
From cbc31cc28be7cfe01663507c150b06798f9017e3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilmari@ilmari.org>
Date: Thu, 2 Jan 2014 14:06:51 +0100
Subject: [PATCH 2/3] Remove "ternary operator parsed as search pattern"
 warning

Since ?PATTERN? without 'm' is no longer supported, there is no
ambiguity any more.
---
 pod/perldiag.pod | 10 ----------
 toke.c           | 10 ++--------
 2 files changed, 2 insertions(+), 18 deletions(-)

diff --git a/pod/perldiag.pod b/pod/perldiag.pod
index aa868f0..19aee77 100644
--- a/pod/perldiag.pod
+++ b/pod/perldiag.pod
@@ -4785,16 +4785,6 @@ construct, not just the empty search pattern.  Therefore code written
 in Perl 5.10.0 or later that uses the // as the I<defined-or> can be
 misparsed by pre-5.10.0 Perls as a non-terminated search pattern.
 
-=item Search pattern not terminated or ternary operator parsed as search pattern
-
-(F) The lexer couldn't find the final delimiter of a C<?PATTERN?>
-construct.
-
-The question mark is also used as part of the ternary operator (as in
-C<foo ? 0 : 1>) leading to some ambiguous constructions being wrongly
-parsed.  One way to disambiguate the parsing is to put parentheses around
-the conditional expression, i.e. C<(foo) ? 0 : 1>.
-
 =item seekdir() attempted on invalid dirhandle %s
 
 (W io) The dirhandle you are doing a seekdir() on is either closed or not
diff --git a/toke.c b/toke.c
index 8c9e63c..1a0fc36 100644
--- a/toke.c
+++ b/toke.c
@@ -9715,14 +9715,8 @@ S_scan_pat(pTHX_ char *start, I32 type)
     s = scan_str(start,!!PL_madskills,FALSE, (PL_in_eval & EVAL_RE_REPARSING),
                        TRUE /* look for escaped bracketed metas */, NULL);
 
-    if (!s) {
-	const char * const delimiter = skipspace(start);
-	Perl_croak(aTHX_
-		   (const char *)
-		   (*delimiter == '?'
-		    ? "Search pattern not terminated or ternary operator parsed as search pattern"
-		    : "Search pattern not terminated" ));
-    }
+    if (!s)
+	Perl_croak(aTHX_ "Search pattern not terminated");
 
     pm = (PMOP*)newPMOP(type, 0);
     if (PL_multi_open == '?') {
-- 
2.0.0

@p5pRT
Copy link
Author

p5pRT commented Jun 6, 2014

From @ilmari

0003-Preserve-explicit-m-when-deparsing-m-PATTERN.patch
From 6a12688487ab24dc21b298bd03b331bbdba5a4f2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilmari@ilmari.org>
Date: Thu, 2 Jan 2014 14:18:18 +0100
Subject: [PATCH 3/3] Preserve explicit 'm' when deparsing m?PATTERN?

Bare ?PATTERN? is no longer allowed
---
 lib/B/Deparse.pm | 2 +-
 lib/B/Deparse.t  | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/B/Deparse.pm b/lib/B/Deparse.pm
index e7eb8cf..43a7c35 100644
--- a/lib/B/Deparse.pm
+++ b/lib/B/Deparse.pm
@@ -4789,7 +4789,7 @@ sub matchop {
     $flags = $matchwords{$flags} if $matchwords{$flags};
     if ($pmflags & PMf_ONCE) { # only one kind of delimiter works here
 	$re =~ s/\?/\\?/g;
-	$re = "?$re?";
+	$re = "m?$re?";        # explicit 'm' is required
     } elsif ($quote) {
 	$re = single_delim($name, $delim, $re);
     }
diff --git a/lib/B/Deparse.t b/lib/B/Deparse.t
index 811f960..c804b70 100644
--- a/lib/B/Deparse.t
+++ b/lib/B/Deparse.t
@@ -1447,3 +1447,6 @@ package foo;
 my @b;
 $x = $b[-256] + $b[-255] + $b[-129] + $b[-128] + $b[-127] + $b[-1] + $b[0];
 $x = $b[1] + $b[126] + $b[127] + $b[128] + $b[255] + $b[256];
+####
+# 'm' must be preserved in m??
+m??;
-- 
2.0.0

@p5pRT
Copy link
Author

p5pRT commented Jun 12, 2014

From @wolfsage

On Fri, Jun 6, 2014 at 11​:22 AM, Dagfinn Ilmari Mannsåker
<ilmari@​ilmari.org> wrote​:

Ricardo Signes <perl.p5p@​rjbs.manxome.org> writes​:

Default-on deprecations warnings have been issued for ?PATTERN? since 5.14.0
(2011). If we're going to fatalize this behavior this series, best to do it
early.

Attached are the patches from
https://rt.perl.org/Public/Bug/Display.html?id=120912 rebased onto
current blead.

Once I'm caught up with deltas (getting there) I'll apply this,
probably tonight.

-- Matthew Horsfall (alh)

@p5pRT
Copy link
Author

p5pRT commented Jun 12, 2014

From @rurban

And maybe you can fix the misleading headline in the bugreport.
It's a RT presentation issue skipping, not displaying "without
explicit 'm' operator" on chrome.

It reads "Bug #120912 for perl5​: [PATCH] Remove support for ?PATTERN?"
which reads that match_once is deprecated, which is worrysome to some.
The pod changes look good

@p5pRT
Copy link
Author

p5pRT commented Jun 12, 2014

From @wolfsage

On Thu, Jun 12, 2014 at 7​:14 AM, Matthew Horsfall (alh)
<wolfsage@​gmail.com> wrote​:

Once I'm caught up with deltas (getting there) I'll apply this,
probably tonight.

-- Matthew Horsfall (alh)

For now, pushed as a smoke-me branch​:

http​://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-me/alh/remove-pattern-without-m

Would any other committers mind taking a look and giving a +1 if
they're happy with it? I intend to merge this in for 5.21.1 if smoke
looks good.

Thanks,

-- Matthew Horsfall (alh)

@p5pRT
Copy link
Author

p5pRT commented Jun 13, 2014

From @jkeenan

On 6/12/14 7​:22 PM, Matthew Horsfall (alh) wrote​:

On Thu, Jun 12, 2014 at 7​:14 AM, Matthew Horsfall (alh)
<wolfsage@​gmail.com> wrote​:

Once I'm caught up with deltas (getting there) I'll apply this,
probably tonight.

-- Matthew Horsfall (alh)

For now, pushed as a smoke-me branch​:

http​://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-me/alh/remove-pattern-without-m

Would any other committers mind taking a look and giving a +1 if
they're happy with it? I intend to merge this in for 5.21.1 if smoke
looks good.

This correspondence didn't make it to the list​:

On Thu, Jun 12, 2014 at 7​:51 PM, James E Keenan <jkeen@​verizon.net> wrote​:

Is there any test which demonstrates that ?? without an explicit
'm' will,
in some sense, fail?
Yup, take a look at
http​://perl5.git.perl.org/perl.git/commitdiff/d752f69d9b2a761b13bc2f6b464efa3205049ac7#patch3

+########
+# toke .c
+use warnings;
+$a =~ ?rand?; # ? is not a regex match
+EXPECT
+syntax error at - line 3, near "=~ ?"
+Execution of - aborted due to compilation errors

-- Matthew Horsfall (alh)

@p5pRT
Copy link
Author

p5pRT commented Jun 17, 2014

From @wolfsage

On Thu, Jun 12, 2014 at 7​:22 PM, Matthew Horsfall (alh)
<wolfsage@​gmail.com> wrote​:

For now, pushed as a smoke-me branch​:

http​://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-me/alh/remove-pattern-without-m

Would any other committers mind taking a look and giving a +1 if
they're happy with it? I intend to merge this in for 5.21.1 if smoke
looks good.

Thanks,

Smokes looked good. This has been merged into blead.

-- Matthew Horsfall (alh)

@p5pRT
Copy link
Author

p5pRT commented Jun 17, 2014

@wolfsage - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant