Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does ‘my sub f; eval 'sub f{}'’ make sense? #14303

Closed
p5pRT opened this issue Dec 4, 2014 · 9 comments
Closed

Does ‘my sub f; eval 'sub f{}'’ make sense? #14303

p5pRT opened this issue Dec 4, 2014 · 9 comments

Comments

@p5pRT
Copy link

p5pRT commented Dec 4, 2014

Migrated from rt.perl.org#123367 (status was 'resolved')

Searchable as RT123367$

@p5pRT
Copy link
Author

p5pRT commented Dec 4, 2014

From @cpansprout

I thought I had submitted this already, but I couldnâ��t find it in RT​:

I know I was the one that implemented it and made eval 'sub f{}' work with �my sub f� in scope, because it seemed that one might expect it to work. But on second thought I�m not sure it does make sense.

�sub f {}� is not actually evaluated at run time. It is a compile-time declaration. For a lexical �my� symbol, it creates, at compile time, a protosub that later gets cloned on scope entry.

In â��my sub f; eval 'sub f{}'â��, the sub declaration should create a protosub that gets cloned the *next* time the enclosing scope is entered. Right? It least thatâ��s how it should work if we want it consistent. But this is what currently happens​:

$ ./perl -XIlib -e 'use feature "​:all"; for(1,0) { my sub f; eval "sub f{warn 42}" if $_; f }'
42 at (eval 1) line 1.
Undefined subroutine &f called at -e line 1.

Now, I�m not sure that this eval-after-my even makes that much sense, and lexical subs are experimental, so that is an argument for changing it.

I *do* think this code makes sense, but it doesnâ��t work​:

$ ./perl -Ilib -e 'use feature "​:all"; my sub f; BEGIN { eval "sub f{warn 42}"} f'
The lexical_subs feature is experimental at -e line 1.
Undefined subroutine &f called at -e line 1.

BEGIN time (when subs are being defined) seems a logical place to decide which sub to install.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jan 8, 2015

From @ap

* Father Chrysostomos <perlbug-followup@​perl.org> [2014-12-04 19​:45]​:

On second thought I�m not sure it does make sense.

You asked me to look at this a while ago. After repeat reading, with
some lengthy pauses included, I still remain half-confused about what
the problem is or what you think is the way it ought to work.

All I can say is that to my (possibly naïve) mind, `my sub f {...}`
ought to work just as if it said `my $f = sub {...};`, with subsequent
invocations of `f(...)` being exactly equivalent to `$f->(...)`. And of
course a `sub f {...}` within scope of `my sub f` would likewise amount
to `$f = sub {...};` � maybe even down to the lack of redefine warning.

This is the only way of defining this feature that I can come up with to
produce behaviour with a coherence and self-consistence that I will find
it graspable and predictable even in strange corner cases.

By that line of thinking, then,

$ ./perl -XIlib -e 'use feature "​:all"; for(1,0) { my sub f; eval "sub f{warn 42}" if $_; f }'
42 at (eval 1) line 1.
Undefined subroutine &f called at -e line 1.

� this is exactly what one would expect. And by the same token,

$ ./perl -Ilib -e 'use feature "​:all"; my sub f; BEGIN { eval "sub f{warn 42}"} f'
The lexical_subs feature is experimental at -e line 1.
Undefined subroutine &f called at -e line 1.

� this *ought* to work.

Is the strict equivalence of these forms in my mind the reason I might
be thinking of this differently than you do? The following appears to
suggest it​:

�sub f {}� is not actually evaluated at run time. It is a compile-time
declaration. For a lexical �my� symbol, it creates, at compile time,
a protosub that later gets cloned on scope entry.

I�m not steeped enough in the implementation to talk about it in terms
of protosub etc, but again, based on the strict equivalence, `my sub f`
basically ought to behave essentially like `my $f` does WRT the compile
time/runtime effects split.

Is this opinion useful to you?

Regards,
--
Aristotle Pagaltzis // <http​://plasmasturm.org/>

@p5pRT
Copy link
Author

p5pRT commented Jan 8, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jan 9, 2015

From @cpansprout

On Wed Jan 07 22​:08​:03 2015, aristotle wrote​:

* Father Chrysostomos <perlbug-followup@​perl.org> [2014-12-04 19​:45]​:

On second thought I�m not sure it does make sense.

You asked me to look at this a while ago. After repeat reading, with
some lengthy pauses included, I still remain half-confused about what
the problem is or what you think is the way it ought to work.

All I can say is that to my (possibly naïve) mind, `my sub f {...}`
ought to work just as if it said `my $f = sub {...};`, with subsequent
invocations of `f(...)` being exactly equivalent to `$f->(...)`. And
of
course a `sub f {...}` within scope of `my sub f` would likewise
amount
to `$f = sub {...};` � maybe even down to the lack of redefine
warning.

This is the only way of defining this feature that I can come up with
to
produce behaviour with a coherence and self-consistence that I will
find
it graspable and predictable even in strange corner cases.

With regular subs, sub foo{} is mostly equivalent to​:

  BEGIN { *foo = sub{} }

Similarly, state sub foo{} is like​:

  state sub foo;
  BEGIN { \&foo = sub{} }

My-subs also have a compile-time action, in that the declaration creates a protosub that will be cloned at run time, just as sub{...} does.

By that line of thinking, then,

$ ./perl -XIlib -e 'use feature "​:all"; for(1,0) { my sub f; eval
"sub f{warn 42}" if $_; f }'
42 at (eval 1) line 1.
Undefined subroutine &f called at -e line 1.

� this is exactly what one would expect. And by the same token,

$ ./perl -Ilib -e 'use feature "​:all"; my sub f; BEGIN { eval "sub
f{warn 42}"} f'
The lexical_subs feature is experimental at -e line 1.
Undefined subroutine &f called at -e line 1.

� this *ought* to work.

Is the strict equivalence of these forms in my mind the reason I might
be thinking of this differently than you do? The following appears to
suggest it​:

�sub f {}� is not actually evaluated at run time. It is a compile-
time
declaration. For a lexical �my� symbol, it creates, at compile time,
a protosub that later gets cloned on scope entry.

I�m not steeped enough in the implementation

Neither am I. I recently had a look at the code in newMYSUB, and I don�t understand it. This is coming from the person who *wrote* it. So that suggests that the number of people who understand that code is zero, which is a scary thought.

to talk about it in terms
of protosub etc, but again, based on the strict equivalence, `my sub
f`
basically ought to behave essentially like `my $f` does WRT the
compile
time/runtime effects split.

Is this opinion useful to you?

I don�t really know the answer to that. :-)

Taking this example that you say ought to work​:

$ ./perl -Ilib -e 'use feature "​:all"; my sub f; BEGIN { eval "sub
f{warn 42}"} f'
The lexical_subs feature is experimental at -e line 1.
Undefined subroutine &f called at -e line 1.

Is that equivalent to

  my sub f;
  BEGIN { eval '\&f = sub { warn 42 }' }

?

If so, then we will have aliased the sub at compile time before the lexical scope is entered. Since all lexical variables are implicitly reset at scope entry (or exit, depending on how you look at it), at some point sub f has to revert back to being undefined.

Or, if

  my sub f { warn 41 }
  BEGIN { sub { warn 42 } }

is equivalent to

  my sub f;
  \&f = sub { warn 41 };
  BEGIN { \&f = sub { warn 42 } }

then the BEGIN block is a no-op, since the run-time 41 assignment would take precedence.

Similarly, the preceding example would be equivalent to having \&f = <undefined sub> outside the BEGIN block.

So the assignment model doesn�t work here.

Brad Gilbert wrote​:

FWIW, in Perl6 it also works​:

$ perl6 \-e ' my &f //= sub \(\)\{???\}; f; EVAL q\[&f = sub \(\)\{say "eval"\}\]; f'
Stub code executed  in sub  at \-e&#8203;:1

eval
$ perl6 \-e ' my &f //= sub \(\)\{???\}; f; BEGIN EVAL q\[&f = sub \(\)\{say "eval"\}\]; f '
eval
eval

I notice you are using //=, which avoids all the problems we are dealing with here.

What if you replace that with �my sub f {???}�?

Using \&f=... assignment in Perl 5 I can make it work very similarly to your Perl 6 example. It is the declarative syntax I am asking about.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jan 10, 2015

From @cpansprout

On Wed Jan 07 22​:08​:03 2015, aristotle wrote​:

* Father Chrysostomos <perlbug-followup@​perl.org> [2014-12-04 19​:45]​:

On second thought I�m not sure it does make sense.

You asked me to look at this a while ago. After repeat reading, with
some lengthy pauses included, I still remain half-confused about what
the problem is or what you think is the way it ought to work.

All I can say is that to my (possibly naïve) mind, `my sub f {...}`
ought to work just as if it said `my $f = sub {...};`, with subsequent
invocations of `f(...)` being exactly equivalent to `$f->(...)`. And
of
course a `sub f {...}` within scope of `my sub f` would likewise
amount
to `$f = sub {...};` � maybe even down to the lack of redefine
warning.

This is the only way of defining this feature that I can come up with
to
produce behaviour with a coherence and self-consistence that I will
find
it graspable and predictable even in strange corner cases.

Here is another way of looking at it. This code​:

my $f = sub {...}

involves three distinct steps.

1. A protosub is created at compile time (you can observe this with
  attributes).
2. The sub is cloned.
3. The clone is assigned to $f.

With 'my sub f {...}' it is clear that the scoping is identical to 'my $f = sub {...}'.

It is when the declaration precedes the definition that we can no longer us the 'my $f =' model to explain it.

my sub f;
f();
sub f {...};

In this case step 1 happens at compile time, as before, but on line 3. Steps 2 and three happen on line 1, allowing the f() call on line 2 to work.

This later definition of the sub works even if it is inside an inner sub​:

my sub f;
f();
sub { sub f {...} }

my sub f;
f();
sub BEGIN { sub f {...} }

It is when we have string eval that things get confusing.

my sub f;
eval 'sub f {...}';

In that code, steps 2 and 3 happen first, and then all three steps happen at the same time inside the eval. (That the internal code that handles sub definitions should have to handle cloning as well really complicates things.) This is an exception to the general rule, which is why I question whether it was a good idea. Usually steps 2 and 3 happen only when the enclosing block is entered.

When you have a BEGIN block with an eval inside it, then things get a little screwy.

my sub f;
BEGIN { eval 'sub f {...}'; }

Because of the way lexical-sub-definition-in-eval has been special-cased to work, you get all three steps happening at once at compile time, and then steps 2 and 3 repeat at run time.

Now this may all sound like internal jargon, but it is all observable behaviour. Steps 2 and 3 always happen together with my-subs, but step 1 happens separately. One can observe step 1 through attributes (i.e., based on when MODIFY_CODE_ATTRIBUTES is called.) One can observe steps 2 and 3 (actually a single step) by the effects of calling the sub.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jan 10, 2015

From @cpansprout

On Fri Jan 09 18​:14​:33 2015, sprout wrote​:

On Wed Jan 07 22​:08​:03 2015, aristotle wrote​:

* Father Chrysostomos <perlbug-followup@​perl.org> [2014-12-04 19​:45]​:

On second thought I�m not sure it does make sense.

You asked me to look at this a while ago. After repeat reading, with
some lengthy pauses included, I still remain half-confused about what
the problem is or what you think is the way it ought to work.

All I can say is that to my (possibly naïve) mind, `my sub f {...}`
ought to work just as if it said `my $f = sub {...};`, with
subsequent
invocations of `f(...)` being exactly equivalent to `$f->(...)`. And
of
course a `sub f {...}` within scope of `my sub f` would likewise
amount
to `$f = sub {...};` � maybe even down to the lack of redefine
warning.

This is the only way of defining this feature that I can come up with
to
produce behaviour with a coherence and self-consistence that I will
find
it graspable and predictable even in strange corner cases.

Here is another way of looking at it. This code​:

my $f = sub {...}

involves three distinct steps.

1. A protosub is created at compile time (you can observe this with
attributes).
2. The sub is cloned.
3. The clone is assigned to $f.

With 'my sub f {...}' it is clear that the scoping is identical to 'my
$f = sub {...}'.

It is when the declaration precedes the definition that we can no
longer us the 'my $f =' model to explain it.

my sub f;
f();
sub f {...};

In this case step 1 happens at compile time, as before, but on line 3.
Steps 2 and three happen on line 1, allowing the f() call on line 2 to
work.

This later definition of the sub works even if it is inside an inner
sub​:

my sub f;
f();
sub { sub f {...} }

my sub f;
f();
sub BEGIN { sub f {...} }

It is when we have string eval that things get confusing.

my sub f;
eval 'sub f {...}';

In that code, steps 2 and 3 happen first, and then all three steps
happen at the same time inside the eval. (That the internal code that
handles sub definitions should have to handle cloning as well really
complicates things.) This is an exception to the general rule, which
is why I question whether it was a good idea. Usually steps 2 and 3
happen only when the enclosing block is entered.

When you have a BEGIN block with an eval inside it, then things get a
little screwy.

my sub f;
BEGIN { eval 'sub f {...}'; }

Because of the way lexical-sub-definition-in-eval has been special-
cased to work, you get all three steps happening at once at compile
time, and then steps 2 and 3 repeat at run time.

Now this may all sound like internal jargon, but it is all observable
behaviour. Steps 2 and 3 always happen together with my-subs, but
step 1 happens separately. One can observe step 1 through attributes
(i.e., based on when MODIFY_CODE_ATTRIBUTES is called.) One can
observe steps 2 and 3 (actually a single step) by the effects of
calling the sub.

Oh, I think I know what I was missing. When the sub is defined in a string eval and declared outside the eval (this is the special case referred to above), the sub is immediately cloned and the clone is installed in the pad. Otherwise, the protosub is set aside for the sake of future cloning. Maybe we need *both* to happen in string eval, and everybody�s expectations will be met. Or maybe not everybody�s.

  for (1,2) {
  my sub f { print "42\n" };
  f();
  eval 'sub f { print "43\n" }';
  f();
  }

That would print​:

42
43
43
43

But it would allow BEGIN-time string eval to set things up properly.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented May 19, 2016

From @cpansprout

On Wed Jan 07 22​:08​:03 2015, aristotle wrote​:

* Father Chrysostomos <perlbug-followup@​perl.org> [2014-12-04 19​:45]​:

$ ./perl -Ilib -e 'use feature "​:all"; my sub f; BEGIN { eval "sub
f{warn 42}"} f'
The lexical_subs feature is experimental at -e line 1.
Undefined subroutine &f called at -e line 1.

� this *ought* to work.

And it started working with commit v5.21.6-197-g0f94cb1, but I do not know why. (That commit was not supposed to change behaviour at all.) It needs a test.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented May 20, 2016

From @cpansprout

On Wed May 18 22​:22​:33 2016, sprout wrote​:

On Wed Jan 07 22​:08​:03 2015, aristotle wrote​:

* Father Chrysostomos <perlbug-followup@​perl.org> [2014-12-04 19​:45]​:

$ ./perl -Ilib -e 'use feature "​:all"; my sub f; BEGIN { eval "sub
f{warn 42}"} f'
The lexical_subs feature is experimental at -e line 1.
Undefined subroutine &f called at -e line 1.

� this *ought* to work.

And it started working with commit v5.21.6-197-g0f94cb1, but I do not
know why. (That commit was not supposed to change behaviour at all.)
It needs a test.

Much of what I said in this ticket was based on a misunderstanding of how newMYSUB determines how to install the sub.¹ Whether a newly-defined lexical sub is set aside for cloning later or cloned and installed in the lexical scope immediately is determined by whether the scope in which it is declared² is currently active (the sub at that scope level is running).

So, that

  my sub f;
  BEGIN { eval "sub f {...}" }

did not work was not due to the model followed, but simply due to a bug in the code. I do not think it is worth poring through the code to find out exactly why it used to fail. But it probably had something to do with the more complex structure of SV + magic not being handled correctly somewhere. v5.21.6-197-g0f94cb1 simplified the data structure and apparently fixed the mishandling thereof at the same time.

All this is to say that this bug can be closed and that the current model, which I suggested changing, is fine. I added a test in 38fe8a0.

¹I know, I even *wrote* the code. But by the time this ticket came up, I had forgotten it. Also, just *look* at the code in newMYSUB. That�s enough to hurt anyone�s head. Maybe I should plead insanity, but not my own in sanity; rather, insanity in Perl�s run-time-vs-compile-time scoping model.

²Note the distinction between declaration and definition. The former refers to �my sub�, which establishes the lexical scope. The latter refers to the body, which defines what the sub does.

--

Father Chrysostomos

@p5pRT p5pRT closed this as completed May 20, 2016
@p5pRT
Copy link
Author

p5pRT commented May 20, 2016

@cpansprout - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant