Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pushing into the same array from two threads segfaults reliably #5546

Open
p6rt opened this issue Aug 8, 2016 · 5 comments
Open

Pushing into the same array from two threads segfaults reliably #5546

p6rt opened this issue Aug 8, 2016 · 5 comments
Labels
SEGV Segmentation fault, bus error, etc.

Comments

@p6rt
Copy link

p6rt commented Aug 8, 2016

Migrated from rt.perl.org#128870 (status was 'open')

Searchable as RT128870$

@p6rt
Copy link
Author

p6rt commented Aug 8, 2016

From @AlexDaniel

While it is probably a bad idea to push values into the same array from two threads, if I recall correctly rakudo is supposed to not crash like that no matter what.

Code​:
my @​a;
start loop { @​a.push​: rand };
start loop { @​a.push​: rand };
sleep 1

Result​:
“corrupted double-linked list”, or “double free or corruption”, or “.realloc()​: invalid next size”, etc. Usually with memory map dumped.

@p6rt
Copy link
Author

p6rt commented Aug 8, 2016

From @nwc10

On Mon, Aug 08, 2016 at 12​:25​:52AM -0700, Aleks-Daniel Jakimenko-Aleksejev wrote​:

While it is probably a bad idea to push values into the same array from two
threads, if I recall correctly rakudo is supposed to not crash like that no
matter what.

Code​:
my @​a;
start loop { @​a.push​: rand };
start loop { @​a.push​: rand };
sleep 1

Result​:
“corrupted double-linked list”, or “double free or corruption”, or “.realloc()​:
invalid next size”, etc. Usually with memory map dumped.

Thanks for the report. Yes, I also don't think that it should crash at a VM
level. Ugly stuff like that is potentially exploitable.

ASAN seems to be pretty consistent​:

$ ./perl6-m -Ilib -e 'my @​a; start loop { @​a.push​: rand }; start loop { @​a.push​: rand }; sleep 1'

==25229==ERROR​: AddressSanitizer​: attempting double-free on 0x60c000173740 in thread T1​:
  #​0 0x7fb80293b8e6 in __interceptor_realloc ../../.././libsanitizer/asan/asan_malloc_linux.cc​:93
  #​1 0x7fb801e6de7e in MVM_realloc src/core/alloc.h​:20
  #​2 0x7fb801e70c67 in set_size_internal src/6model/reprs/MVMArray.c​:334
  #​3 0x7fb801e71b77 in push src/6model/reprs/MVMArray.c​:437
  #​4 0x7fb801d385ee in MVM_interp_run src/core/interp.c​:2163
  #​5 0x7fb801dbd862 in start_thread src/core/threads.c​:77
  #​6 0x7fb80208460d in uv__thread_start 3rdparty/libuv/src/unix/thread.c​:49
  #​7 0x7fb80111daa0 in start_thread (/lib64/libpthread.so.0+0x7aa0)
  #​8 0x7fb801623aac in __clone (/lib64/libc.so.6+0xe8aac)

0x60c000173740 is located 0 bytes inside of 128-byte region [0x60c000173740,0x60c0001737c0)
freed by thread T2 here​:
  #​0 0x7fb80293b8e6 in __interceptor_realloc ../../.././libsanitizer/asan/asan_malloc_linux.cc​:93
  #​1 0x7fb801e6de7e in MVM_realloc src/core/alloc.h​:20
  #​2 0x7fb801e70c67 in set_size_internal src/6model/reprs/MVMArray.c​:334
  #​3 0x7fb801e71b77 in push src/6model/reprs/MVMArray.c​:437
  #​4 0x7fb801d385ee in MVM_interp_run src/core/interp.c​:2163
  #​5 0x7fb801dbd862 in start_thread src/core/threads.c​:77
  #​6 0x7fb80208460d in uv__thread_start 3rdparty/libuv/src/unix/thread.c​:49
  #​7 0x7fb80111daa0 in start_thread (/lib64/libpthread.so.0+0x7aa0)

previously allocated by thread T1 here​:
  #​0 0x7fb80293b8e6 in __interceptor_realloc ../../.././libsanitizer/asan/asan_malloc_linux.cc​:93
  #​1 0x7fb801e6de7e in MVM_realloc src/core/alloc.h​:20
  #​2 0x7fb801e70c67 in set_size_internal src/6model/reprs/MVMArray.c​:334
  #​3 0x7fb801e71b77 in push src/6model/reprs/MVMArray.c​:437
  #​4 0x7fb801d385ee in MVM_interp_run src/core/interp.c​:2163
  #​5 0x7fb801dbd862 in start_thread src/core/threads.c​:77
  #​6 0x7fb80208460d in uv__thread_start 3rdparty/libuv/src/unix/thread.c​:49
  #​7 0x7fb80111daa0 in start_thread (/lib64/libpthread.so.0+0x7aa0)

Thread T1 created by T0 here​:
  #​0 0x7fb80290a6ea in __interceptor_pthread_create ../../.././libsanitizer/asan/asan_interceptors.cc​:183
  #​1 0x7fb802084712 in uv_thread_create 3rdparty/libuv/src/unix/thread.c​:66
  #​2 0x7fb801dbdc13 in MVM_thread_run src/core/threads.c​:129
  #​3 0x7fb801d63435 in MVM_interp_run src/core/interp.c​:3964
  #​4 0x7fb802018c7c in MVM_vm_run_file src/moar.c​:304
  #​5 0x401a4f in main src/main.c​:191
  #​6 0x7fb801559d1c in __libc_start_main (/lib64/libc.so.6+0x1ed1c)

Thread T2 created by T0 here​:
  #​0 0x7fb80290a6ea in __interceptor_pthread_create ../../.././libsanitizer/asan/asan_interceptors.cc​:183
  #​1 0x7fb802084712 in uv_thread_create 3rdparty/libuv/src/unix/thread.c​:66
  #​2 0x7fb801dbdc13 in MVM_thread_run src/core/threads.c​:129
  #​3 0x7fb801d63435 in MVM_interp_run src/core/interp.c​:3964
  #​4 0x7fb802018c7c in MVM_vm_run_file src/moar.c​:304
  #​5 0x401a4f in main src/main.c​:191
  #​6 0x7fb801559d1c in __libc_start_main (/lib64/libc.so.6+0x1ed1c)

SUMMARY​: AddressSanitizer​: double-free ../../.././libsanitizer/asan/asan_malloc_linux.cc​:93 __interceptor_realloc
==25229==ABORTING

Nicholas Clark

@p6rt
Copy link
Author

p6rt commented Aug 8, 2016

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Aug 8, 2016

From @lizmat

As far as I understand Jonathan’s position on this, is that you shouldn’t do that. If you want to push to an array from multiple threads, you probably should use a Channel, or use Supplies with an .act block.

As to the security implications of this behaviour, I must admit I haven’t thought about that in this context myself yet. In any case, having any array automatically work correctly when being pushed to from multiple threads, is a ticket to deadlock hell (if I remember Jonathan’s wording on this subject correctly).

Liz

On 08 Aug 2016, at 12​:16, Nicholas Clark <nick@​ccl4.org> wrote​:

On Mon, Aug 08, 2016 at 12​:25​:52AM -0700, Aleks-Daniel Jakimenko-Aleksejev wrote​:

While it is probably a bad idea to push values into the same array from two
threads, if I recall correctly rakudo is supposed to not crash like that no
matter what.

Code​:
my @​a;
start loop { @​a.push​: rand };
start loop { @​a.push​: rand };
sleep 1

Result​:
“corrupted double-linked list”, or “double free or corruption”, or “.realloc()​:
invalid next size”, etc. Usually with memory map dumped.

Thanks for the report. Yes, I also don't think that it should crash at a VM
level. Ugly stuff like that is potentially exploitable.

ASAN seems to be pretty consistent​:

$ ./perl6-m -Ilib -e 'my @​a; start loop { @​a.push​: rand }; start loop { @​a.push​: rand }; sleep 1'

==25229==ERROR​: AddressSanitizer​: attempting double-free on 0x60c000173740 in thread T1​:
#​0 0x7fb80293b8e6 in __interceptor_realloc ../../.././libsanitizer/asan/asan_malloc_linux.cc​:93
#​1 0x7fb801e6de7e in MVM_realloc src/core/alloc.h​:20
#​2 0x7fb801e70c67 in set_size_internal src/6model/reprs/MVMArray.c​:334
#​3 0x7fb801e71b77 in push src/6model/reprs/MVMArray.c​:437
#​4 0x7fb801d385ee in MVM_interp_run src/core/interp.c​:2163
#​5 0x7fb801dbd862 in start_thread src/core/threads.c​:77
#​6 0x7fb80208460d in uv__thread_start 3rdparty/libuv/src/unix/thread.c​:49
#​7 0x7fb80111daa0 in start_thread (/lib64/libpthread.so.0+0x7aa0)
#​8 0x7fb801623aac in __clone (/lib64/libc.so.6+0xe8aac)

0x60c000173740 is located 0 bytes inside of 128-byte region [0x60c000173740,0x60c0001737c0)
freed by thread T2 here​:
#​0 0x7fb80293b8e6 in __interceptor_realloc ../../.././libsanitizer/asan/asan_malloc_linux.cc​:93
#​1 0x7fb801e6de7e in MVM_realloc src/core/alloc.h​:20
#​2 0x7fb801e70c67 in set_size_internal src/6model/reprs/MVMArray.c​:334
#​3 0x7fb801e71b77 in push src/6model/reprs/MVMArray.c​:437
#​4 0x7fb801d385ee in MVM_interp_run src/core/interp.c​:2163
#​5 0x7fb801dbd862 in start_thread src/core/threads.c​:77
#​6 0x7fb80208460d in uv__thread_start 3rdparty/libuv/src/unix/thread.c​:49
#​7 0x7fb80111daa0 in start_thread (/lib64/libpthread.so.0+0x7aa0)

previously allocated by thread T1 here​:
#​0 0x7fb80293b8e6 in __interceptor_realloc ../../.././libsanitizer/asan/asan_malloc_linux.cc​:93
#​1 0x7fb801e6de7e in MVM_realloc src/core/alloc.h​:20
#​2 0x7fb801e70c67 in set_size_internal src/6model/reprs/MVMArray.c​:334
#​3 0x7fb801e71b77 in push src/6model/reprs/MVMArray.c​:437
#​4 0x7fb801d385ee in MVM_interp_run src/core/interp.c​:2163
#​5 0x7fb801dbd862 in start_thread src/core/threads.c​:77
#​6 0x7fb80208460d in uv__thread_start 3rdparty/libuv/src/unix/thread.c​:49
#​7 0x7fb80111daa0 in start_thread (/lib64/libpthread.so.0+0x7aa0)

Thread T1 created by T0 here​:
#​0 0x7fb80290a6ea in __interceptor_pthread_create ../../.././libsanitizer/asan/asan_interceptors.cc​:183
#​1 0x7fb802084712 in uv_thread_create 3rdparty/libuv/src/unix/thread.c​:66
#​2 0x7fb801dbdc13 in MVM_thread_run src/core/threads.c​:129
#​3 0x7fb801d63435 in MVM_interp_run src/core/interp.c​:3964
#​4 0x7fb802018c7c in MVM_vm_run_file src/moar.c​:304
#​5 0x401a4f in main src/main.c​:191
#​6 0x7fb801559d1c in __libc_start_main (/lib64/libc.so.6+0x1ed1c)

Thread T2 created by T0 here​:
#​0 0x7fb80290a6ea in __interceptor_pthread_create ../../.././libsanitizer/asan/asan_interceptors.cc​:183
#​1 0x7fb802084712 in uv_thread_create 3rdparty/libuv/src/unix/thread.c​:66
#​2 0x7fb801dbdc13 in MVM_thread_run src/core/threads.c​:129
#​3 0x7fb801d63435 in MVM_interp_run src/core/interp.c​:3964
#​4 0x7fb802018c7c in MVM_vm_run_file src/moar.c​:304
#​5 0x401a4f in main src/main.c​:191
#​6 0x7fb801559d1c in __libc_start_main (/lib64/libc.so.6+0x1ed1c)

SUMMARY​: AddressSanitizer​: double-free ../../.././libsanitizer/asan/asan_malloc_linux.cc​:93 __interceptor_realloc
==25229==ABORTING

Nicholas Clark

@p6rt
Copy link
Author

p6rt commented Aug 9, 2016

From @jnthn

On Mon Aug 08 04​:09​:13 2016, elizabeth wrote​:

As far as I understand Jonathan’s position on this, is that you
shouldn’t do that.

Well, you shouldn't do it without concurrency control if you want correct results. But SEGV isn't an OK failure mode. So it wants addressing up to that point (and I know how to do it, just didn't get to it yet).

If you want to push to an array from multiple
threads, you probably should use a Channel, or use Supplies with an
.act block.

Or re-think your problem in a less imperative way. :-)

In any case,
having any array automatically work correctly when being pushed to
from multiple threads, is a ticket to deadlock hell (if I remember
Jonathan’s wording on this subject correctly).

Well, I had many words, but the other notable issue is that it gives a false sense of security (even if array operations were to be individually safe, auto-viv would still a race, as would `while @​foo { @​foo.pop }`) and just leads people towards high-contention designs when a different framing of the problem would give a lower-contention design.

/jnthn

@p6rt p6rt added the SEGV Segmentation fault, bus error, etc. label Jan 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
SEGV Segmentation fault, bus error, etc.
Projects
None yet
Development

No branches or pull requests

1 participant