Skip Menu |
Report information
Id: 126163
Status: open
Priority: 0/
Queue: perl6

Owner: Nobody
Requestors: zefram [at] fysh.org
Cc:
AdminCc:

Severity: (no value)
Tag: (no value)
Platform: (no value)
Patch Status: (no value)
VM: (no value)



Subject: [LTA] silence of IterationEnd failures
From: Zefram <zefram [...] fysh.org>
Date: Thu, 24 Sep 2015 13:41:39 +0100
To: rakudobug [...] perl.org
Download (untitled) / with headers
text/plain 812b
Even if [perl #126146] and [perl #126147] are not bugs, that .map et al go wrong *silently* is less than awesome. It would be better if an error were signalled whenever they're used outside their intended scope. Maybe this could be factored out to the iteration system, requiring checks in only a small number of places. Whenever an iteration construct gets an out-of-scope input, or generates an out-of-scope output, it should detect that and signal an error. When the spec documentation addresses the issue of what values can be passed through the iteration system [perl #126159], the checks should of course match the documentation. The check might be .isa(Any), for example. At minimum, the check needs to reject anything that downstream iteration code would misinterpret, so !=:=IterationEnd. -zefram
Download (untitled) / with headers
text/plain 943b
I'm inclined to reject this ticket, since the implementors (jnthn, lizmat) have already ruled (in the two referenced tickets) that doing anything other than the intended =:= check to an IterationEnd is unsupported and a case of DIHWIDT [1]. Storing IterationEnd in an array or passing it to a &map, would certainly qualify as "doing something other than the intended =:= check" to it, so... just don't do those things. Furthermore, the moved goalpost of this new ticket seems impractical to me. How do you intend for &map and &reduce to do a preliminary check to make sure that no input element is IterationEnd, without iterating over the input list? And if it iterates over the input list, then it *has* to quietly stop when it encounters an IterationEnd, because for all it knows the input list could be generated by an Iterable object that legitimately signals its end using IterationEnd. --- [1] http://design.perl6.org/S99.html#DIHWIDT
Date: Sat, 21 May 2016 01:18:57 +0100
To: "Sam S. via RT" <perl6-bugs-followup [...] perl.org>
Subject: Re: [perl #126163] [LTA] silence of IterationEnd failures
From: Zefram <zefram [...] fysh.org>
Download (untitled) / with headers
text/plain 1.6k
Sam S. via RT wrote: Show quoted text
>Storing IterationEnd in an array or passing it to a &map, would certainly >qualify as "doing something other than the intended =:= check" to it, >so... just don't do those things.
Don't introspect? Don't metaprogram? Show quoted text
>How do you intend for &map and &reduce to do a preliminary check to >make sure that no input element is IterationEnd, without iterating over >the input list?
I imagine that the check would be performed by whatever puts values into the IterationEnd-delimited context, where they have come from something else. So normally this would be a check when *outputting* from an iterator, in its pull-one method. One can always check that the value being returned by pull-one isn't IterationEnd in the cases where one doesn't intend to return the sentinel to end the iteration. For example, a List knows how many elements it has, even if some of those elements are IterationEnd. The iterator returned by the List.iterator method transfers values from the counted context to the IterationEnd-delimited context, internally maintaining an index that it compares to the element count. (This happens in methods pull-one and reify-and-pull-one in the anonymous iterator class in List.iterator). That iterator code could check whether a value that it has retrieved by index is IterationEnd, and signal an error if it is. In the case of map, obviously it can't check its input values, because they're already in an IterationEnd-delimited context, coming from an iterator. But map can check its output: it can look at what the user-supplied iteration function returned, and signal an error if that's IterationEnd. -zefram
Download (untitled) / with headers
text/plain 1.5k
Show quoted text
> Don't introspect? Don't metaprogram?
Can you describe an actual example of something a Perl 6 module author might want to do that could cause IteratorEnd to be leaked outside of its intended context? Based on jnthn's comments, my understanding is that: 1) The only place that is allowed to return IterationEnd, is the .pull-one method (and friends) of a class that implements the Iterator role. 2) Any code that consumes an Iterator, is responsible for doing the `=:= IteratorEnd` check. I.e. it exists only for this one interface - The supplier side of the interface may generate the sentinel, and the receiving side immediately handles it. It never reaches the "outside world", as long as implementers follow this protocol. (Which we can expect them to, since this is a very low-level API that is only public in order to give module authors who know what they're doing, a way to add low-level functionality to Perl 6 that is compatible with the existing iteration features of the language. If you're using the low-level Iterator API in normal code because you're trying to use Perl 6 as if it was Python, then again, DIHWIDT. If the documentation does not sufficiently deter people from trying this, then maybe this should be a p6doc ticket instead.) So if you want to litter other parts of Perl 6 with (potentially performance-degrading) checks for the sentinel value used in this particular interface, you would certainly bolster your case if you demonstrated how the sentinel value could legitimately end up in those places, or why it would be beneficial to allow it to do so.
From: Zefram <zefram [...] fysh.org>
Subject: Re: [perl #126163] [LTA] silence of IterationEnd failures
To: "Sam S. via RT" <perl6-bugs-followup [...] perl.org>
Date: Sat, 21 May 2016 15:49:21 +0100
Download (untitled) / with headers
text/plain 1.5k
Sam S. via RT wrote: Show quoted text
>Can you describe an actual example of something a Perl 6 module author >might want to do that could cause IteratorEnd to be leaked outside of >its intended context?
Introspect on the CORE:: stash. Perhaps in order to parse code that refers to objects by their Perl 6 names, and which makes use of the public iterator API including the approved "=:= IteratorEnd" check. Show quoted text
>It never reaches the "outside world", as long as implementers follow >this protocol.
So, say, putting the object into a public namespace is an unapproved use, and has to be unapproved because it would cause such a leak. Oops, the core implementation is breaking the protocol. Better remove CORE::<IterationEnd>. Show quoted text
>If the documentation does not sufficiently deter people from trying this, >then maybe this should be a p6doc ticket instead.
If you want to maintain the situation of IterationEnd being reified as a named object, but not able to be processed by the language's general mechanisms, then the documentation certainly needs to prohibit more than just direct, deliberate references to it. Prohibit name lookups from stashes? Prohibit metaprogramming of iterator code? It's not at all obvious which things are intended to be unsupported. Show quoted text
>So if you want to litter other parts of Perl 6 with (potentially >performance-degrading) checks for the sentinel value
The checks are not free, of course, but they're cheap. As jnthn has explained, you're using a sentinel-delimited API specifically because that identity comparison is cheap. -zefram
Show quoted text
> Introspect on the CORE:: stash.
I gave it a try and you're right, the `IterationEnd` value breaks iterating over the values of that stash: say CORE::.keys.map(*.perl).elems; # 712 say CORE::.values.map(*.perl).elems; # 193 That doesn't make it impossible to work with (just make sure you iterate over .keys or .pairs), but I agree that it would be a WAT for someone doing this unsuspectingly. Show quoted text
> Better remove CORE::<IterationEnd>.
Maybe you're right, and `IterationEnd` should be a special built-in thing that does not live in `CORE::`? Still, it seems like a contained issue. Or is there anything else other than the `CORE::` object that would leak an `IterationEnd` value into a Seq/List passed to user code? Maybe special-casing the behavior of this object would be easier/cheaper than adding additional checks to all Seq/List producers everywhere. But those discussions are out of my depth... :) I'm just here to follow up on RT tickets, and since this one is not as clearly closeable as I thought, I will leave it be and hope that jnthn & co get around to it some time.
Date: Sat, 21 May 2016 18:56:53 +0100
From: Zefram <zefram [...] fysh.org>
Subject: Re: [perl #126163] [LTA] silence of IterationEnd failures
To: "Sam S. via RT" <perl6-bugs-followup [...] perl.org>
Download (untitled) / with headers
text/plain 1.7k
Sam S. via RT wrote: Show quoted text
>is there anything else other than the `CORE::` object that would leak an >`IterationEnd` value into a Seq/List passed to user code?
There are some kinds of programming that would be liable to innocently run into the actual object even if it doesn't have a regular name. It looks like Perl 6 broadly intends to enable the affected kinds of programming, but this is where my Perl 6 knowledge runs out: I don't know which specifically exist now or how to do them. Still, I think the prospects are worrying enough even where Perl 6 doesn't actually do them yet. First, there's MOP stuff, where metaclass code might wrap method calls or otherwise tweak their implementation. This kind of code would naturally need to handle all the kinds of value that can be passed to or returned from a method. So it can see the IterationEnd value in the operation of valid iterator code, without specifically knowing that it's dealing with iterator code. In a related vein, think about introspection into the call records of a running program. For example, one might want to produce a stack trace that shows the arguments to each call, as does Perl 5's Carp::confess(). Or one might run a program under a Perl 6 debugger, trying to emit trace data for each call and return. One can also consider introspection into the non-running code reified in Sub objects and the like, or plugging extensions into the compiler. Where the code has a "=:= IterationEnd" operation, the introspection would be liable to see the IterationEnd value, even if it wasn't spelled with a regular name. Reformulating "$a =:= IterationEnd" as "is_IterationEnd($a)" could avoid this problem for the comparisons, but it still arises for code that returns the sentinel. -zefram


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org