Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moar bug flattening Arg in Baggy with slurpy_positional #3528

Closed
p6rt opened this issue Sep 19, 2014 · 7 comments
Closed

Moar bug flattening Arg in Baggy with slurpy_positional #3528

p6rt opened this issue Sep 19, 2014 · 7 comments

Comments

@p6rt
Copy link

p6rt commented Sep 19, 2014

Migrated from rt.perl.org#122810 (status was 'resolved')

Searchable as RT122810$

@p6rt
Copy link
Author

p6rt commented Sep 19, 2014

From grtodd@gmail.com

Using masak's "p6-golfed" word counter​:

  perl6 -e `.say for (bag slurp.words).pairs.sort(*.value).reverse[^10]'

on this file http://www.gutenberg.org/files/24007/24007-8.txt

which I have saved as UTF-8 text in La_psychologie_des_foules.txt

iconv -f ISO-8859-1 -t UTF-8 24007-8.txt > La_psychologie_des_foules.txt
perl6 -e `.say for (bag slurp.words).pairs.sort(*.value).reverse[^10]'
La_psychologie_des_foules.txt

I get this error​:

Arg has not been flattened in slurpy_positional
  in method new at src/gen/m-CORE.setting​:17249
  in sub bag at src/gen/m-CORE.setting​:17906

src/gen/m-CORE.setting​:17249 comes from
src/core/Baggy.pm
https://github.com/rakudo/rakudo/blob/nom/src/core/Baggy.pm#L24

"slurpy_positional" shows up under nqp/MoarVM/​:
nqp/MoarVM/src/core/interp.c
and nqp/MoarVM/src/core/args.c in a while(){} around linke 515 we have​:

if (arg_info.flags & MVM_CALLSITE_ARG_FLAT) {,
  MVM_exception_throw_adhoc(tc, "Arg has not been flattened
in slurpy_positional");,
},

so my guess is that this is a Baggy/Arg <--> moar bug or a not yet
implemented bit in the switch/case statements ( MVM_CALLSITE_ARG_etc )
in that file.

Neither perl6-j and perl6-p have this error.

Other data points​:

The file is 6294 lines long and has many words to count​:
wc La_psychologie_des_foules.txt
6294 53930 369219 La_psychologie_des_foules.txt

If I reduce the size of the file this the code works without the error

head -n 5655 > short_La_psychologie_des_foules.txt
wc short_La_psychologie_des_foules.txt
5655 49190 334666 short_La_psychologie_des_foules.txt

This seems to predictably occur when the file is > 5654 lines long.
Creating a file with 5655 lines and removing characters gradually you
can create a file at the exact size limit to provoke a crash.
Larger files with more lines and fewer "words" also work. Trying to
find a pattern by aliasing the above code to p6wordcount and then
passing it lots of text files​:

  for i in *txt; do echo ""; wc $i ; p6wordcount $i; done

gives only faint glimmers of light ... it seems where files have
numbers of words > ~49000 the error appears. Files with large numbers
of characters don't seem to have the same effect though, i.e. as long
as the number of words is within the limit.

Hope that helps someone track this down.

@p6rt
Copy link
Author

p6rt commented Sep 21, 2014

From grtodd@gmail.com

This commit appears to fix the issue​:

rakudo/rakudo@a8267b418c

A test is needed to confirm

@p6rt
Copy link
Author

p6rt commented Sep 21, 2014

grtodd@gmail.com - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Sep 21, 2014

From grtodd@gmail.com

On Sun Sep 21 07​:28​:51 2014, grtodd@​gmail.com wrote​:

This commit appears to fix the issue​:

[SNIP]

A test is needed to confirm.

Bug in bug report! Wrong commit mentioned!!

This is the commit that has the fix​:

rakudo/rakudo@0d4bc7031f

@p6rt
Copy link
Author

p6rt commented Sep 26, 2014

From grtodd@gmail.com

With some help from #perl6 I seem to have found a reliable way to trip this bug that could comprise a test. Adding this note here so I don't forget.

Building @​commit e630782

Passes​:
my @​arr_works = ('a'...*)[^49000] ;
.say for (bag @​arr_works.words).pairs.sort(*.value).reverse[^5]
"btlp" => 1
"btlo" => 1
"btln" => 1
"btlm" => 1
"btll" => 1

Fails​:
my @​arr_dies = ('a'...*)[^50000] ;
a b c d e f g h i j k l m n o p q r s t u v w x y z aa ab ac ad ae af ag ah ai aj ak al am an ao ap aq ar as at au av aw ax ay az ba bb bc bd be bf bg bh bi bj bk bl bm bn bo bp bq br bs bt bu bv bw bx by bz ca cb cc cd ce cf cg ch ci cj ck cl cm cn co cp cq cr cs ct cu cv ...

.say for (bag @​arr_dies.words).pairs.sort(*.value).reverse[^5]
Arg has not been flattened in slurpy_positional

Builds from after the fix committed by jnth (noted above) pass with very large arrays (e.g. my @​arr = ('a'...*)[^2000000] ; ) so a test could possibly be written for this bug using some appropriate value.

@p6rt
Copy link
Author

p6rt commented Sep 29, 2014

From grtodd@gmail.com

w/ a test​: this fails on older build and passes with the fix noted above​:

ok bag(my @​big_arr = ("a"...*)[^50000]), "big array goes into a bag";

pushed as perl6/roast/commit/d720ab5 to S03-operators/bag.t

@p6rt p6rt closed this as completed Sep 29, 2014
@p6rt
Copy link
Author

p6rt commented Sep 29, 2014

grtodd@gmail.com - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant