Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lines() isn't lazy in Rakudo #2742

Closed
p6rt opened this issue May 1, 2012 · 10 comments
Closed

lines() isn't lazy in Rakudo #2742

p6rt opened this issue May 1, 2012 · 10 comments

Comments

@p6rt
Copy link

p6rt commented May 1, 2012

Migrated from rt.perl.org#112716 (status was 'resolved')

Searchable as RT112716$

@p6rt
Copy link
Author

p6rt commented May 1, 2012

From @masak

<tadzik> r​: for $*IN.lines -> $line { say $line.lc }
<p6eval> rakudo 4c241c​: OUTPUT«land der berge, land am strome, [...]
<goraki> tadzik​: masak​: when I run either I don't get any output until
I hit ctrl-d to "end" the input.
<masak> goraki​: let my try locally.
<goraki> masak​: ta.
<masak> my gosh, you're right!
* masak submits rakudobug
<masak> goraki​: your request has been filed. thank you for noticing.
<PerlJam> wait... that's a bug?
<masak> sure.
<tadzik> sure
<masak> lines() is supposed to be lazy.

@p6rt
Copy link
Author

p6rt commented May 3, 2012

From @pmichaud

On Tue, May 01, 2012 at 09​:11​:22AM -0700, Carl Mäsak wrote​:

<tadzik> r​: for $*IN.lines -> $line { say $line.lc }
<p6eval> rakudo 4c241c​: OUTPUT«land der berge, land am strome, [...]
<goraki> tadzik​: masak​: when I run either I don't get any output until
I hit ctrl-d to "end" the input.
<masak> goraki​: let my try locally.
<masak> my gosh, you're right!
* masak submits rakudobug
[...]
<masak> lines() is supposed to be lazy.

Actually, as implemented lines() is already "lazy"​:

  pmichaud@​kiwi​:~/p6/rakudo$ ./perl6
  > my @​a := $*IN.lines().map( { .lc.say } ); 1;
  1
  > say @​a[0]
  HELLO
  hello
  True
  >

The problem is actually in the .map code itself (I suspect
a misoptimization) -- when map is attempting to reify multiple
elements, it's batching up the required number of elements
before processing any of them​:

  > @​a[3]
  WORLD
  GOODBYE
  WORLD
  world
  goodbye
  world
  >

The problem line is in src/core/MapIter.pm​:

  my $munched := $!list.munch($argc * $count);

This causes all of the requested elements of the source $!list to
be reified before any of them are processed, thus .map isn't
properly lazy. This line was added as an optimization to make
some .map operations faster, but lost some of the lazy aspects
of map (as well as other things).

The correct approach is to make .munch sufficiently performant
over repeated calls (avoiding repeated shifts of the
underlying list structures), which I plan to take care of
shortly.

Pm

@p6rt
Copy link
Author

p6rt commented May 3, 2012

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Jul 1, 2012

From @ronaldxs

See from IRC http://irclog.perlgeek.de/perl6/2012-06-18#i_5738718

A test might look something like the code below which currently results
in a "not OK".

use Test;

my $i = 0;
my @​a = (10, 20 ,30 ,40);
my @​l := gather { for @​a {$i++; take $_ } };
my @​pull;
push @​pull, $i, $_ for @​l;
is_deeply(@​pull, [ 0, 10, 1, 20, 2, 30, 3, 40]);

@p6rt
Copy link
Author

p6rt commented Jul 1, 2012

From @ronaldxs

On second thought a simpler and better test might be​:

use Test;

my $i = 0;
my @​a = (10, 20 ,30 ,40);
my @​l := gather { for @​a {++$i; take $_ } };
my @​pull;
push @​pull, $_, $i for @​l;
is(@​pull, [ 10, 1, 20, 2, 30, 3, 40, 4]);

@p6rt
Copy link
Author

p6rt commented Jul 1, 2012

From @pmichaud

On Thu May 03 01​:53​:21 2012, pmichaud wrote​:

[...]
The correct approach is to make .munch sufficiently performant
over repeated calls (avoiding repeated shifts of the
underlying list structures), which I plan to take care of
shortly.

Although I've improved MapIter so that it's appropriately lazy,
there's still a bit too much "workahead" hanging around in the
ListIter code. I'm working on this now, which should resolve the
remaining issues with the overly-eager 'for' statements.

Pm

@p6rt
Copy link
Author

p6rt commented Mar 6, 2014

From @ronaldxs

On Sun Jul 01 09​:53​:36 2012, pmichaud wrote​:

On Thu May 03 01​:53​:21 2012, pmichaud wrote​:

[...]
Although I've improved MapIter so that it's appropriately lazy,
there's still a bit too much "workahead" hanging around in the
ListIter code. I'm working on this now, which should resolve the
remaining issues with the overly-eager 'for' statements.

Pm

Re-tested with a recent rakudo build and it looks like the examples work. Wondering if we can't close this one.

Ron Schmidt

@p6rt
Copy link
Author

p6rt commented Mar 8, 2014

From @ronaldxs

After chatting with Masak on IRC I am updating the ticket again. There is an existing test in

https://github.com/perl6/roast/blob/master/S04-statements/gather.t

That seems to me to cover the same ground as the 10, 20, 30, 40 example.

The test is​:

# lazy gather
{
  my $count = 0;
  my @​list := gather {
  for 1 .. 10 -> $a {
  take $a;
  $count++
  }
  };
  my $result = @​list[2];
  is($count, 2, "gather is lazy");
}

So I don't know if or how one ads the *IN example to testing but I believe the 10, 20, 30, 40 test is already in.

Thanks,
Ron

@p6rt
Copy link
Author

p6rt commented Sep 14, 2015

From @niner

Test added in roast commit eaaec7af0a924fd14e97bf24ccb71e458d19f439
Current implementation in Rakudo really is lazy enough.

@p6rt p6rt closed this as completed Sep 14, 2015
@p6rt
Copy link
Author

p6rt commented Sep 14, 2015

@niner - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant