Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify parser to allow empty semicolon ';' terminated statement #127

Closed
p6rt opened this issue Jun 10, 2008 · 11 comments
Closed

Modify parser to allow empty semicolon ';' terminated statement #127

p6rt opened this issue Jun 10, 2008 · 11 comments

Comments

@p6rt
Copy link

p6rt commented Jun 10, 2008

Migrated from rt.perl.org#55594 (status was 'resolved')

Searchable as RT55594$

@p6rt
Copy link
Author

p6rt commented Jun 10, 2008

From @ronaldxs

Perl 5 allows empty semicolon terminated statements. perl6 currently
allows for empty closures but treats a semicolon that follows nothing or
whitespace as a syntax error. The attached patch to grammar.pg provides
a fix for this discrepancy between perl 5 and perl 6. Also attached is
an attempt at a test suite for some statement termination cases.

The patch is small and passes "make fulltest" under ubuntu and "make
spectest", as well as the parts of "make test" in the perl6 directory,
under cygwin. Modifying statementlist this way does not appear to be
the most elegant solution to the problem but approaches like
[<statement>?<.eat_terminator> ]* seemed to fail with relatively serious
errors.

The terminator.t test file is vaguely intended for the
t/spec/S04-statements/ directory.

@p6rt
Copy link
Author

p6rt commented Jun 10, 2008

From @ronaldxs

empty_stmt.diff
Index: t/spectest_regression.data
===================================================================
--- t/spectest_regression.data	(revision 28227)
+++ t/spectest_regression.data	(working copy)
@@ -15,6 +15,7 @@
 S03-operators/not.t                             # pure
 S03-operators/relational.t                      # pure
 S03-operators/true.t                            # pure
+S04-statements/terminator.t                     # pure
 S04-statements/try.t
 S04-statements/until.t                          # pure
 S04-statements/while.t
Index: src/parser/grammar.pg
===================================================================
--- src/parser/grammar.pg	(revision 28227)
+++ src/parser/grammar.pg	(working copy)
@@ -151,8 +151,10 @@
 
 
 rule statementlist {
-    [<statement><.eat_terminator> ]*
-    {*}
+    | [
+        ||   ';'
+        ||   <statement><.eat_terminator>
+    ]*    {*}    
 }
 
 ##  The eat_terminator detects when we're at a valid

@p6rt
Copy link
Author

p6rt commented Jun 10, 2008

From @ronaldxs

use v6;

use Test;

plan 9;

# L<S04/"Statement parsing"/"statement termination">

# the 'empty statement' case responsible for the creation of this test file
eval_lives_ok(';', 'empty statement');

eval_lives_ok('my $x = 2', 'simple statement no semi');
eval_lives_ok('my $x =
9', 'simple statement on two lines no semi');
eval_lives_ok('my $x = 2;', 'simple statement with semi');
eval_lives_ok('{my $x = 2}', 'end of closure terminator');
eval_lives_ok('{my $x =
2;}', 'double terminator');
eval_lives_ok(';my $x = 2;{my $x = 2;;};', 'extra terminators');

eval_dies_ok('{my $x = 2;', 'open closure');
eval_dies_ok('my $x = ', 'incomplete expression');

@p6rt
Copy link
Author

p6rt commented Jun 11, 2008

From @pmichaud

On Tue, Jun 10, 2008 at 01​:07​:45PM -0700, Ron Schmidt wrote​:

Perl 5 allows empty semicolon terminated statements. perl6 currently
allows for empty closures but treats a semicolon that follows nothing or
whitespace as a syntax error. The attached patch to grammar.pg provides
a fix for this discrepancy between perl 5 and perl 6. Also attached is
an attempt at a test suite for some statement termination cases.

The patch is small and passes "make fulltest" under ubuntu and "make
spectest", as well as the parts of "make test" in the perl6 directory,
under cygwin. Modifying statementlist this way does not appear to be
the most elegant solution to the problem but approaches like
[<statement>?<.eat_terminator> ]* seemed to fail with relatively serious
errors.

Thanks for the patch. In cases of changes to the grammar, however,
we're trying to keep Rakudo's grammar as closely aligned to STD.pm
as we can -- here's STD.pm's definition for statementlist (line ~570)​:

  rule statementlist {
  :my StrPos $endstmt is context<rw> = -1;
  [<statement><.eat_terminator> ]*
  {*}
  }

So, since Rakudo's current grammar pretty much matches what STD.pm
has here, I'd prefer that we figure out how STD.pm is handling
empty statements and do that.

Thanks again!

Pm

@p6rt
Copy link
Author

p6rt commented Jun 11, 2008

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Jun 13, 2008

From @ronaldxs

So, since Rakudo's current grammar pretty much matches what STD.pm
has here, I'd prefer that we figure out how STD.pm is handling
empty statements and do that.

Thanks again!

Pm

As far as I can tell, STD.pm handles this problem in "token statement",
as shown below. If you modify the statement rule in grammar.pg the same
way and make a small change to actions.pm, parrot/perl6 seems to handle
empty ; statements fine and passes the small, included, test suite. The
revised patches are attached.

token statement {
  ...
  | <?before ';'> {*} #= null
  ...
}

@p6rt
Copy link
Author

p6rt commented Jun 13, 2008

From @ronaldxs

empty_stmt_r2.diff
Index: src/parser/grammar.pg
===================================================================
--- src/parser/grammar.pg	(revision 28329)
+++ src/parser/grammar.pg	(working copy)
@@ -203,6 +203,7 @@
             {*}                                  #= statement_mod_cond
         || {*}                                   #= expr
         ]
+    | <?before ';'> {*}                          #= null
 }
 
 rule statement_control {
Index: src/parser/actions.pm
===================================================================
--- src/parser/actions.pm	(revision 28329)
+++ src/parser/actions.pm	(working copy)
@@ -111,6 +111,9 @@
     if $key eq 'statement_control' {
         $past := $( $<statement_control> );
     }
+    elsif $key eq 'null' {
+        $past := PAST::Stmts.new();  # empty stmts seem eliminated by TGE
+    }    
     else {
         my $expr := $( $<expr> );
         if $expr.WHAT() eq 'Block' && !$expr.blocktype() {
Index: t/spectest_regression.data
===================================================================
--- t/spectest_regression.data	(revision 28329)
+++ t/spectest_regression.data	(working copy)
@@ -15,6 +15,7 @@
 S03-operators/not.t                             # pure
 S03-operators/relational.t                      # pure
 S03-operators/true.t                            # pure
+S04-statements/terminator.t                     # pure
 S04-statements/try.t
 S04-statements/until.t                          # pure
 S04-statements/while.t

@p6rt
Copy link
Author

p6rt commented Jun 13, 2008

From @ronaldxs

use v6;

use Test;

plan 9;

# L<S04/"Statement parsing"/"statement termination">

# the 'empty statement' case responsible for the creation of this test file
eval_lives_ok(';', 'empty statement');

eval_lives_ok('my $x = 2', 'simple statement no semi');
eval_lives_ok('my $x =
9', 'simple statement on two lines no semi');
eval_lives_ok('my $x = 2;', 'simple statement with semi');
eval_lives_ok('{my $x = 2}', 'end of closure terminator');
eval_lives_ok('{my $x =
2;}', 'double terminator');
eval_lives_ok(';my $x = 2;{my $x = 2;;};', 'extra terminators');

eval_dies_ok('{my $x = 2;', 'open closure');
eval_dies_ok('my $x = ', 'incomplete expression');

@p6rt
Copy link
Author

p6rt commented Jun 21, 2008

From @pmichaud

Applied in r28597, thanks!

Pm

1 similar comment
@p6rt
Copy link
Author

p6rt commented Jun 21, 2008

From @pmichaud

Applied in r28597, thanks!

Pm

@p6rt
Copy link
Author

p6rt commented Jun 21, 2008

@pmichaud - Status changed from 'open' to 'resolved'

@p6rt p6rt closed this as completed Jun 21, 2008
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant