Skip Menu |
Report information
Id: 132447
Status: open
Priority: 0/
Queue: perl6

Owner: Nobody
Requestors: dan [at] zwell.net
Cc:
AdminCc:

Severity: (no value)
Tag: testneeded
Platform: (no value)
Patch Status: (no value)
VM: (no value)



Date: Tue, 14 Nov 2017 18:59:25 +0000
Subject: IO.slurp throws exception when run in threads
To: "rakudobug [...] perl.org" <rakudobug [...] perl.org>
From: Dan Zwell <dan [...] zwell.net>
Download (untitled) / with headers
text/plain 1.1k
The exception is:
Cannot assign to an immutable value  in method slurp at SETTING::src/core/IO/Handle.pm line 698
  in method slurp at SETTING::src/core/IO/Path.pm line 603
  in block  at ./concurrency-test.p6 line 40
  in block  at SETTING::src/core/Promise.pm line 217
  in block  at SETTING::src/core/ThreadPoolScheduler.pm line 284
  in block  at SETTING::src/core/ThreadPoolScheduler.pm line 173
  in block  at SETTING::src/core/ThreadPoolScheduler.pm line 166
  in block  at SETTING::src/core/ThreadPoolScheduler.pm line 163

I've observed this error on both Linux and Windows 10, on the September and October Rakudo releases. It's intermittent, so I use a shell loop to run my test case repeatedly until the problem occurs:

while perl6 ./concurrency-test.p6 $DIR; do :; done
(where $DIR is any directory that contains several thousand files.)

This problem only occurs with slurp. When I instead open a file handle, read its full contents with `.lines.cache`, then close the handle, there is no exception.

The test case I used to show this issue is uploaded here:
https://gist.github.com/lefth/6d71ca714ca2dc184220a91ceb41334d
RT-Send-CC: perl6-compiler [...] perl.org
Download (untitled) / with headers
text/plain 845b
Hi, Having trouble reproducing this on Linux on 2017.10-177-g77048b6 in a dir with 33k files that have one line of text each. - Can you inlcude the exact version (perl6 -v) to go with the --ll-exception stack trace so we can examine the referenced core code? I tried following the line numbers in the trace you gave with 2017.09 and 2017.10 releases, but they don't seem to quite match up - Does your directory also have other directories? Does the crash happen with just files in the directory? - What's the typical size of the files in your directory? Show quoted text
> This problem only occurs with slurp. When I instead open a file handle, > read its full contents with `.lines.cache`, then close the handle, there is > no exception.
Does it happen if you use the handle's .slurp method? i.e. open($file-path).slurp: :close; Let us know. Thanks.
To: perl6-bugs-followup [...] perl.org
From: Dan Zwell <dan [...] zwell.net>
Date: Wed, 15 Nov 2017 18:06:55 +0000
Subject: Re: [perl #132447] AutoReply: IO.slurp throws exception when run in threads
It seems this has already been resolved on the master branch. After making a new build, I cannot reproduce the problem. Can you close the bug report? Thanks!

On Wed, Nov 15, 2017 at 2:59 AM perl6 via RT <perl6-bugs-followup@perl.org> wrote:
Show quoted text
Greetings,

This message has been automatically generated in response to the
creation of a trouble ticket regarding:
        "IO.slurp throws exception when run in threads",
a summary of which appears below.

There is no need to reply to this message right now.  Your ticket has been
assigned an ID of [perl #132447].

Please include the string:

         [perl #132447]

in the subject line of all future correspondence about this issue. To do so,
you may reply to this message.

                        Thank you,
                        perl6-bugs-followup@perl.org

-------------------------------------------------------------------------
The exception is:
Cannot assign to an immutable value  in method slurp at
SETTING::src/core/IO/Handle.pm line 698
  in method slurp at SETTING::src/core/IO/Path.pm line 603
  in block  at ./concurrency-test.p6 line 40
  in block  at SETTING::src/core/Promise.pm line 217
  in block  at SETTING::src/core/ThreadPoolScheduler.pm line 284
  in block  at SETTING::src/core/ThreadPoolScheduler.pm line 173
  in block  at SETTING::src/core/ThreadPoolScheduler.pm line 166
  in block  at SETTING::src/core/ThreadPoolScheduler.pm line 163

I've observed this error on both Linux and Windows 10, on the September and
October Rakudo releases. It's intermittent, so I use a shell loop to run my
test case repeatedly until the problem occurs:

while perl6 ./concurrency-test.p6 $DIR; do :; done
(where $DIR is any directory that contains several thousand files.)

This problem only occurs with slurp. When I instead open a file handle,
read its full contents with `.lines.cache`, then close the handle, there is
no exception.

The test case I used to show this issue is uploaded here:
https://gist.github.com/lefth/6d71ca714ca2dc184220a91ceb41334d

Marked as 「testneeded」 to make sure we have tests for it.

If there are tests already, then it should be linked and the ticket can be closed.


On 2017-11-15 14:43:51, dan@zwell.net wrote:
Show quoted text
> It seems this has already been resolved on the master branch. After
> making
> a new build, I cannot reproduce the problem. Can you close the bug
> report?
> Thanks!
>
> On Wed, Nov 15, 2017 at 2:59 AM perl6 via RT <perl6-bugs-
> followup@perl.org>
> wrote:
>
> > Greetings,
> >
> > This message has been automatically generated in response to the
> > creation of a trouble ticket regarding:
> > "IO.slurp throws exception when run in threads",
> > a summary of which appears below.
> >
> > There is no need to reply to this message right now. Your ticket has
> > been
> > assigned an ID of [perl #132447].
> >
> > Please include the string:
> >
> > [perl #132447]
> >
> > in the subject line of all future correspondence about this issue. To
> > do
> > so,
> > you may reply to this message.
> >
> > Thank you,
> > perl6-bugs-followup@perl.org
> >
> > -------------------------------------------------------------------------
> > The exception is:
> > Cannot assign to an immutable value in method slurp at
> > SETTING::src/core/IO/Handle.pm line 698
> > in method slurp at SETTING::src/core/IO/Path.pm line 603
> > in block at ./concurrency-test.p6 line 40
> > in block at SETTING::src/core/Promise.pm line 217
> > in block at SETTING::src/core/ThreadPoolScheduler.pm line 284
> > in block at SETTING::src/core/ThreadPoolScheduler.pm line 173
> > in block at SETTING::src/core/ThreadPoolScheduler.pm line 166
> > in block at SETTING::src/core/ThreadPoolScheduler.pm line 163
> >
> > I've observed this error on both Linux and Windows 10, on the
> > September and
> > October Rakudo releases. It's intermittent, so I use a shell loop to
> > run my
> > test case repeatedly until the problem occurs:
> >
> > while perl6 ./concurrency-test.p6 $DIR; do :; done
> > (where $DIR is any directory that contains several thousand files.)
> >
> > This problem only occurs with slurp. When I instead open a file
> > handle,
> > read its full contents with `.lines.cache`, then close the handle,
> > there is
> > no exception.
> >
> > The test case I used to show this issue is uploaded here:
> > https://gist.github.com/lefth/6d71ca714ca2dc184220a91ceb41334d
> >
> >


Date: Fri, 17 Nov 2017 10:59:05 +0000
Subject: Re: [perl #132447] IO.slurp throws exception when run in threads
To: perl6-bugs-followup [...] perl.org
From: Dan Zwell <dan [...] zwell.net>
Download (untitled) / with headers
text/plain 5.4k
After more careful checking, I found the bug fix did make it into the October release. A bisect showed it was fixed it in commit 6af44f8d38a02bbd0d68cfd014165d6e33e4d89a. So, in the prior commit, the --version 2017.09-490-g3f595acfb built on MoarVM version 2017.10, and the exception (with --ll-exception) is:

Got exception: Cannot assign to an immutable value  in method slurp at SETTING::src/core/IO/Handle.pm line 735
  in method slurp at SETTING::src/core/IO/Path.pm line 609
  in block  at ./concurrency-test.p6 line 40
  in block  at SETTING::src/core/Promise.pm line 241
  in block  at SETTING::src/core/ThreadPoolScheduler.pm line 659
  in block  at SETTING::src/core/ThreadPoolScheduler.pm line 224
  in method run-one at SETTING::src/core/ThreadPoolScheduler.pm line 224
  in method dispatch:<!> at SETTING::src/core/Mu.pm line 806
  in block  at SETTING::src/core/ThreadPoolScheduler.pm line 258

Died
   at SETTING::src/core/Exception.pm:57  (D:\cygwin64\home\p6\rakudo\install\share/perl6/runtime/CORE.setting.moarvm:throw)
 from SETTING::src/core/control.pm:170  (D:\cygwin64\home\p6\rakudo\install\share/perl6/runtime/CORE.setting.moarvm:die)
 from SETTING::src/core/control.pm:166  (D:\cygwin64\home\p6\rakudo\install\share/perl6/runtime/CORE.setting.moarvm:die)
 from ./concurrency-test.p6:54  (<ephemeral file>:MAIN)
 from SETTING::src/core/Main.pm:198  (D:\cygwin64\home\p6\rakudo\install\share/perl6/runtime/CORE.setting.moarvm:MAIN_HELPER)
 from ./concurrency-test.p6:33  (<ephemeral file>:<unit>)
 from ./concurrency-test.p6:1  (<ephemeral file>:<unit-outer>)
 from gen\moar\stage2\NQPHLL.nqp:1542  (D:\cygwin64\home\p6\rakudo\install\share\nqp\lib/NQPHLL.moarvm:eval)
 from gen\moar\stage2\NQPHLL.nqp:1779  (D:\cygwin64\home\p6\rakudo\install\share\nqp\lib/NQPHLL.moarvm:evalfiles)
 from gen\moar\stage2\NQPHLL.nqp:1704  (D:\cygwin64\home\p6\rakudo\install\share\nqp\lib/NQPHLL.moarvm:command_eval)
 from src/Perl6/Compiler.nqp:42  (D:\cygwin64\home\p6\rakudo\install\share\nqp\lib/Perl6/Compiler.moarvm:command_eval)
 from gen\moar\stage2\NQPHLL.nqp:1630  (D:\cygwin64\home\p6\rakudo\install\share\nqp\lib/NQPHLL.moarvm:command_line)
 from gen/moar/main.nqp:47  (D:\cygwin64\home\p6\rakudo\install\share\perl6\runtime\perl6.moarvm:MAIN)
 from gen/moar/main.nqp:38  (D:\cygwin64\home\p6\rakudo\install\share\perl6\runtime\perl6.moarvm:<mainline>)
 from <unknown>:1  (D:\cygwin64\home\p6\rakudo\install\share\perl6\runtime\perl6.moarvm:<main>)
 from <unknown>:1  (D:\cygwin64\home\p6\rakudo\install\share\perl6\runtime\perl6.moarvm:<entry>)

I found I can reproduce the error within 1-4 iterations when running the script on the Rakudo tree (built with --gen-moar and --gen-nqp and installed without a prefix to ./install/).

Slurping on the file handle you described still produces the exception: open($file-path).slurp: :close;
but slurping from a filehandle without :close, followed by $fh.close, is fine.



On Thu, Nov 16, 2017 at 7:58 AM Aleks-Daniel Jakimenko-Aleksejev via RT <perl6-bugs-followup@perl.org> wrote:
Show quoted text
Marked as 「testneeded」 to make sure we have tests for it.

If there are tests already, then it should be linked and the ticket can be
closed.

On 2017-11-15 14:43:51, dan@zwell.net wrote:
> It seems this has already been resolved on the master branch. After
> making
> a new build, I cannot reproduce the problem. Can you close the bug
> report?
> Thanks!
>
> On Wed, Nov 15, 2017 at 2:59 AM perl6 via RT <perl6-bugs-
> followup@perl.org>
> wrote:
>
> > Greetings,
> >
> > This message has been automatically generated in response to the
> > creation of a trouble ticket regarding:
> > "IO.slurp throws exception when run in threads",
> > a summary of which appears below.
> >
> > There is no need to reply to this message right now. Your ticket has
> > been
> > assigned an ID of [perl #132447].
> >
> > Please include the string:
> >
> > [perl #132447]
> >
> > in the subject line of all future correspondence about this issue. To
> > do
> > so,
> > you may reply to this message.
> >
> > Thank you,
> > perl6-bugs-followup@perl.org
> >
> > -------------------------------------------------------------------------
> > The exception is:
> > Cannot assign to an immutable value in method slurp at
> > SETTING::src/core/IO/Handle.pm line 698
> > in method slurp at SETTING::src/core/IO/Path.pm line 603
> > in block at ./concurrency-test.p6 line 40
> > in block at SETTING::src/core/Promise.pm line 217
> > in block at SETTING::src/core/ThreadPoolScheduler.pm line 284
> > in block at SETTING::src/core/ThreadPoolScheduler.pm line 173
> > in block at SETTING::src/core/ThreadPoolScheduler.pm line 166
> > in block at SETTING::src/core/ThreadPoolScheduler.pm line 163
> >
> > I've observed this error on both Linux and Windows 10, on the
> > September and
> > October Rakudo releases. It's intermittent, so I use a shell loop to
> > run my
> > test case repeatedly until the problem occurs:
> >
> > while perl6 ./concurrency-test.p6 $DIR; do :; done
> > (where $DIR is any directory that contains several thousand files.)
> >
> > This problem only occurs with slurp. When I instead open a file
> > handle,
> > read its full contents with `.lines.cache`, then close the handle,
> > there is
> > no exception.
> >
> > The test case I used to show this issue is uploaded here:
> > https://gist.github.com/lefth/6d71ca714ca2dc184220a91ceb41334d
> >
> >

RT-Send-CC: perl6-compiler [...] perl.org
On Fri, 17 Nov 2017 09:26:10 -0800, dan@zwell.net wrote: Show quoted text
> After more careful checking, I found the bug fix did make it into the > October release. A bisect showed it was fixed it in > commit 6af44f8d38a02bbd0d68cfd014165d6e33e4d89a. > [...] > Slurping on the file handle you described still produces the > exception: open($file-path).slurp: :close; > but slurping from a filehandle without :close, followed by $fh.close, is fine.
Great. The behaviour you describe matches the commit; without :close the buggy path is not taken. Show quoted text
> I found I can reproduce the error within 1-4 iterations when running > the > script on the Rakudo tree (built with --gen-moar and --gen-nqp and > installed without a prefix to ./install/).
I tried writing the test to cover this bug, but out of dozens of things I tried (including trying your original script on rakudo's dir), I just can't reproduce the failure. Does this crash on your computer when using commits before the fix, by any chance? given $*TMPDIR.add: 'RT132447.test' { $^p.spurt: ''; await (start $p.slurp) xx 40_000 };
Subject: Re: [perl #132447] IO.slurp throws exception when run in threads
Date: Sat, 18 Nov 2017 08:03:35 +0000
To: perl6-bugs-followup [...] perl.org
From: Dan Zwell <dan [...] zwell.net>
Oh, and when the list of filenames is the simpler `"$*TMPDIR/RT132447.test" xx 100`, the problem also appears, but it seemed to take many more iterations to crash. That could be just chance, though.

On Sat, Nov 18, 2017 at 4:01 PM Dan Zwell <dan@zwell.net> wrote:
Show quoted text
Thanks for taking the time to look into this! I can't reproduce it with that snippet, even if I make the file nonempty. But I can reproduce it with the following two snippets. (I could not reproduce when I populated the input file in the same script that does the await loop.)

perl6 -e '"$*TMPDIR/RT132447.test".IO.spurt: "a" x 400_000;'
while perl6 --ll-exception -e 'my @files = gather { for ^100 { take "$*TMPDIR/RT132447.test" } }; await do for @files { start .IO.slurp }'; do echo iteration done; done

On Sat, Nov 18, 2017 at 3:28 AM Zoffix Znet via RT <perl6-bugs-followup@perl.org> wrote:
On Fri, 17 Nov 2017 09:26:10 -0800, dan@zwell.net wrote:
> After more careful checking, I found the bug fix did make it into the
> October release. A bisect showed it was fixed it in
> commit 6af44f8d38a02bbd0d68cfd014165d6e33e4d89a.
> [...]
> Slurping on the file handle you described still produces the
> exception: open($file-path).slurp: :close;
> but slurping from a filehandle without :close, followed by $fh.close, is fine.

Great. The behaviour you describe matches the commit; without :close the buggy path is not taken.


> I found I can reproduce the error within 1-4 iterations when running
> the
> script on the Rakudo tree (built with --gen-moar and --gen-nqp and
> installed without a prefix to ./install/).

I tried writing the test to cover this bug, but out of dozens of things I tried (including trying your original script on rakudo's dir), I just can't reproduce the failure.

Does this crash on your computer when using commits before the fix, by any chance?

    given $*TMPDIR.add: 'RT132447.test' { $^p.spurt: ''; await (start $p.slurp) xx 40_000 };


Subject: Re: [perl #132447] IO.slurp throws exception when run in threads
Date: Sat, 18 Nov 2017 08:01:04 +0000
From: Dan Zwell <dan [...] zwell.net>
To: perl6-bugs-followup [...] perl.org
Download (untitled) / with headers
text/plain 1.6k
Thanks for taking the time to look into this! I can't reproduce it with that snippet, even if I make the file nonempty. But I can reproduce it with the following two snippets. (I could not reproduce when I populated the input file in the same script that does the await loop.)

perl6 -e '"$*TMPDIR/RT132447.test".IO.spurt: "a" x 400_000;'
while perl6 --ll-exception -e 'my @files = gather { for ^100 { take "$*TMPDIR/RT132447.test" } }; await do for @files { start .IO.slurp }'; do echo iteration done; done

On Sat, Nov 18, 2017 at 3:28 AM Zoffix Znet via RT <perl6-bugs-followup@perl.org> wrote:
Show quoted text
On Fri, 17 Nov 2017 09:26:10 -0800, dan@zwell.net wrote:
> After more careful checking, I found the bug fix did make it into the
> October release. A bisect showed it was fixed it in
> commit 6af44f8d38a02bbd0d68cfd014165d6e33e4d89a.
> [...]
> Slurping on the file handle you described still produces the
> exception: open($file-path).slurp: :close;
> but slurping from a filehandle without :close, followed by $fh.close, is fine.

Great. The behaviour you describe matches the commit; without :close the buggy path is not taken.


> I found I can reproduce the error within 1-4 iterations when running
> the
> script on the Rakudo tree (built with --gen-moar and --gen-nqp and
> installed without a prefix to ./install/).

I tried writing the test to cover this bug, but out of dozens of things I tried (including trying your original script on rakudo's dir), I just can't reproduce the failure.

Does this crash on your computer when using commits before the fix, by any chance?

    given $*TMPDIR.add: 'RT132447.test' { $^p.spurt: ''; await (start $p.slurp) xx 40_000 };




This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org