Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

silence of IterationEnd failures #3147

Closed
p6rt opened this issue Sep 24, 2015 · 10 comments
Closed

silence of IterationEnd failures #3147

p6rt opened this issue Sep 24, 2015 · 10 comments
Labels

Comments

@p6rt
Copy link

p6rt commented Sep 24, 2015

Migrated from rt.perl.org#126163 (status was 'open')

Searchable as RT126163$

@p6rt
Copy link
Author

p6rt commented Sep 24, 2015

From zefram@fysh.org

Even if [perl #​126146] and [perl #​126147] are not bugs, that .map et
al go wrong *silently* is less than awesome. It would be better if an
error were signalled whenever they're used outside their intended scope.
Maybe this could be factored out to the iteration system, requiring checks
in only a small number of places. Whenever an iteration construct gets
an out-of-scope input, or generates an out-of-scope output, it should
detect that and signal an error.

When the spec documentation addresses the issue of what values can be
passed through the iteration system [perl #​126159], the checks should
of course match the documentation. The check might be .isa(Any), for
example. At minimum, the check needs to reject anything that downstream
iteration code would misinterpret, so !=​:=IterationEnd.

-zefram

@p6rt
Copy link
Author

p6rt commented May 20, 2016

From @smls

I'm inclined to reject this ticket, since the implementors (jnthn, lizmat) have already ruled (in the two referenced tickets) that doing anything other than the intended =​:= check to an IterationEnd is unsupported and a case of DIHWIDT [1].

Storing IterationEnd in an array or passing it to a &map, would certainly qualify as "doing something other than the intended =​:= check" to it, so... just don't do those things.

Furthermore, the moved goalpost of this new ticket seems impractical to me.
How do you intend for &map and &reduce to do a preliminary check to make sure that no input element is IterationEnd, without iterating over the input list?
And if it iterates over the input list, then it *has* to quietly stop when it encounters an IterationEnd, because for all it knows the input list could be generated by an Iterable object that legitimately signals its end using IterationEnd.


[1] http://design.perl6.org/S99.html#DIHWIDT

@p6rt
Copy link
Author

p6rt commented May 20, 2016

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented May 21, 2016

From zefram@fysh.org

Sam S. via RT wrote​:

Storing IterationEnd in an array or passing it to a &map, would certainly
qualify as "doing something other than the intended =​:= check" to it,
so... just don't do those things.

Don't introspect? Don't metaprogram?

How do you intend for &map and &reduce to do a preliminary check to
make sure that no input element is IterationEnd, without iterating over
the input list?

I imagine that the check would be performed by whatever puts values
into the IterationEnd-delimited context, where they have come from
something else. So normally this would be a check when *outputting*
from an iterator, in its pull-one method. One can always check that the
value being returned by pull-one isn't IterationEnd in the cases where
one doesn't intend to return the sentinel to end the iteration.

For example, a List knows how many elements it has, even if some
of those elements are IterationEnd. The iterator returned by the
List.iterator method transfers values from the counted context to the
IterationEnd-delimited context, internally maintaining an index that it
compares to the element count. (This happens in methods pull-one and
reify-and-pull-one in the anonymous iterator class in List.iterator).
That iterator code could check whether a value that it has retrieved by
index is IterationEnd, and signal an error if it is.

In the case of map, obviously it can't check its input values,
because they're already in an IterationEnd-delimited context, coming
from an iterator. But map can check its output​: it can look at what
the user-supplied iteration function returned, and signal an error if
that's IterationEnd.

-zefram

@p6rt
Copy link
Author

p6rt commented May 21, 2016

From @smls

Don't introspect? Don't metaprogram?

Can you describe an actual example of something a Perl 6 module author might want to do that could cause IteratorEnd to be leaked outside of its intended context?

Based on jnthn's comments, my understanding is that​:

1) The only place that is allowed to return IterationEnd, is the .pull-one method (and friends) of a class that implements the Iterator role.

2) Any code that consumes an Iterator, is responsible for doing the `=​:= IteratorEnd` check.

I.e. it exists only for this one interface - The supplier side of the interface may generate the sentinel, and the receiving side immediately handles it. It never reaches the "outside world", as long as implementers follow this protocol.

(Which we can expect them to, since this is a very low-level API that is only public in order to give module authors who know what they're doing, a way to add low-level functionality to Perl 6 that is compatible with the existing iteration features of the language.
If you're using the low-level Iterator API in normal code because you're trying to use Perl 6 as if it was Python, then again, DIHWIDT. If the documentation does not sufficiently deter people from trying this, then maybe this should be a p6doc ticket instead.)

So if you want to litter other parts of Perl 6 with (potentially performance-degrading) checks for the sentinel value used in this particular interface, you would certainly bolster your case if you demonstrated how the sentinel value could legitimately end up in those places, or why it would be beneficial to allow it to do so.

@p6rt
Copy link
Author

p6rt commented May 21, 2016

From zefram@fysh.org

Sam S. via RT wrote​:

Can you describe an actual example of something a Perl 6 module author
might want to do that could cause IteratorEnd to be leaked outside of
its intended context?

Introspect on the CORE​:: stash. Perhaps in order to parse code that
refers to objects by their Perl 6 names, and which makes use of the
public iterator API including the approved "=​:= IteratorEnd" check.

It never reaches the "outside world", as long as implementers follow
this protocol.

So, say, putting the object into a public namespace is an unapproved
use, and has to be unapproved because it would cause such a leak.
Oops, the core implementation is breaking the protocol. Better remove
CORE​::<IterationEnd>.

If the documentation does not sufficiently deter people from trying this,
then maybe this should be a p6doc ticket instead.

If you want to maintain the situation of IterationEnd being reified as
a named object, but not able to be processed by the language's general
mechanisms, then the documentation certainly needs to prohibit more
than just direct, deliberate references to it. Prohibit name lookups
from stashes? Prohibit metaprogramming of iterator code? It's not at
all obvious which things are intended to be unsupported.

So if you want to litter other parts of Perl 6 with (potentially
performance-degrading) checks for the sentinel value

The checks are not free, of course, but they're cheap. As jnthn has
explained, you're using a sentinel-delimited API specifically because
that identity comparison is cheap.

-zefram

@p6rt
Copy link
Author

p6rt commented May 21, 2016

From @smls

Introspect on the CORE​:: stash.

I gave it a try and you're right, the `IterationEnd` value breaks iterating over the values of that stash​:

  say CORE​::.keys.map(*.perl).elems; # 712
  say CORE​::.values.map(*.perl).elems; # 193

That doesn't make it impossible to work with (just make sure you iterate over .keys or .pairs), but I agree that it would be a WAT for someone doing this unsuspectingly.

Better remove CORE​::<IterationEnd>.

Maybe you're right, and `IterationEnd` should be a special built-in thing that does not live in `CORE​::`?

Still, it seems like a contained issue. Or is there anything else other than the `CORE​::` object that would leak an `IterationEnd` value into a Seq/List passed to user code? Maybe special-casing the behavior of this object would be easier/cheaper than adding additional checks to all Seq/List producers everywhere.

But those discussions are out of my depth... :)
I'm just here to follow up on RT tickets, and since this one is not as clearly closeable as I thought, I will leave it be and hope that jnthn & co get around to it some time.

@p6rt
Copy link
Author

p6rt commented May 21, 2016

From zefram@fysh.org

Sam S. via RT wrote​:

is there anything else other than the `CORE​::` object that would leak an
`IterationEnd` value into a Seq/List passed to user code?

There are some kinds of programming that would be liable to innocently run
into the actual object even if it doesn't have a regular name. It looks
like Perl 6 broadly intends to enable the affected kinds of programming,
but this is where my Perl 6 knowledge runs out​: I don't know which
specifically exist now or how to do them. Still, I think the prospects
are worrying enough even where Perl 6 doesn't actually do them yet.

First, there's MOP stuff, where metaclass code might wrap method calls or
otherwise tweak their implementation. This kind of code would naturally
need to handle all the kinds of value that can be passed to or returned
from a method. So it can see the IterationEnd value in the operation
of valid iterator code, without specifically knowing that it's dealing
with iterator code.

In a related vein, think about introspection into the call records of a
running program. For example, one might want to produce a stack trace
that shows the arguments to each call, as does Perl 5's Carp​::confess().
Or one might run a program under a Perl 6 debugger, trying to emit trace
data for each call and return.

One can also consider introspection into the non-running code reified
in Sub objects and the like, or plugging extensions into the compiler.
Where the code has a "=​:= IterationEnd" operation, the introspection
would be liable to see the IterationEnd value, even if it wasn't
spelled with a regular name. Reformulating "$a =​:= IterationEnd" as
"is_IterationEnd($a)" could avoid this problem for the comparisons,
but it still arises for code that returns the sentinel.

-zefram

@JJ
Copy link
Contributor

JJ commented Jan 7, 2020

This might be a trap or in any case, a documentation issue, not an implementation issue. Moving it to the doc repo.

@JJ JJ transferred this issue from Raku/old-issue-tracker Jan 7, 2020
@JJ JJ added the trap label Feb 8, 2020
@JJ
Copy link
Contributor

JJ commented Aug 5, 2020

I'll try to summarize it here. Since IterationEnd marks the end of a loop, anything that contains it will silently end the loop when it bumps into it.

.say for ["foo",IterationEnd, "baz"] # foo

It does not happen in map, however.

@JJ JJ closed this as completed in beb441c Aug 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants