Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal to link perl with system threads library by default #14133

Closed
p5pRT opened this issue Oct 4, 2014 · 21 comments
Closed

Proposal to link perl with system threads library by default #14133

p5pRT opened this issue Oct 4, 2014 · 21 comments

Comments

@p5pRT
Copy link

p5pRT commented Oct 4, 2014

Migrated from rt.perl.org#122906 (status was 'resolved')

Searchable as RT122906$

@p5pRT
Copy link
Author

p5pRT commented Oct 4, 2014

From verdrehung@gmail.com

Created by oleg@cpan.org

For now we have two perl types​: threaded and non-threaded.
Threaded is linked with threads library, non-threaded is not.
The problem comes when we want to make XS module which
will use system threads. In fact this module will crash on all
non-threaded perls, because threads library is not linked with
this perl and proper initialization will not be done on perl start.
See​: http​://stackoverflow.com/questions/13587325/threads-vs-pthread-in-perl

Of course someone can use '-A prepend​:libswanted="pthread "'
to compile non-threaded perl linked with pthreads, but nobody
do this. The only available solution to writer of the module is
to say to the user​: if it will crash recompile your perl linked with
pthreads or use LD_PRELOAD=/path/to/pthreads.so hack.

So, my proposal is to always link perl with system threads library
to allow to write threaded XS modules for all types of perl.

Perl Info

Flags:
    category=core
    severity=wishlist

Site configuration information for perl 5.14.2:

Configured by Debian Project at Mon Sep 30 03:43:22 UTC 2013.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:

  Platform:
    osname=linux, osvers=3.2.0-4-amd64, archname=x86_64-linux-gnu-thread-multi
    uname='linux babin 3.2.0-4-amd64 #1 smp debian 3.2.46-1+deb7u1
x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector --param=ssp-buffer-size=4
-Wformat -Werror=format-security -Dldflags= -Wl,-z,relro
-Dlddlflags=-shared -Wl,-z,relro -Dcccdlflags=-fPIC
-Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.14 -Darchlib=/usr/lib/perl/5.14
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.14.2
-Dsitearch=/usr/local/lib/perl/5.14.2 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1
-Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh
-Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -DDEBUGGING=-g -Doptimize=-O2
-Duseshrplib -Dlibperl=libperl.so.5.14.2 -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN
-fstack-protector -fno-strict-aliasing -pipe -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fstack-protector
-fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.7.2', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=, so=so, useshrplib=true, libperl=libperl.so.5.14.2
    gnulibc_version='2.13'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector'


@INC for perl 5.14.2:
    /etc/perl
    /usr/local/lib/perl/5.14.2
    /usr/local/share/perl/5.14.2
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.14
    /usr/share/perl/5.14
    /usr/local/lib/site_perl
    .


Environment for perl 5.14.2:
    HOME=/home/oleg
    LANG=C
    LANGUAGE=
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Oct 5, 2014

From @craigberry

On Sat, Oct 4, 2014 at 8​:49 AM, Ð�лег Ð� <perlbug-followup@​perl.org> wrote​:

# New Ticket Created by �лег �
# Please include the string​: [perl #122906]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=122906 >

This is a bug report for perl from oleg@​cpan.org,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
[Please describe your issue here]
For now we have two perl types​: threaded and non-threaded.
Threaded is linked with threads library, non-threaded is not.
The problem comes when we want to make XS module which
will use system threads. In fact this module will crash on all
non-threaded perls, because threads library is not linked with
this perl and proper initialization will not be done on perl start.
See​: http​://stackoverflow.com/questions/13587325/threads-vs-pthread-in-perl

Of course someone can use '-A prepend​:libswanted="pthread "'
to compile non-threaded perl linked with pthreads, but nobody
do this. The only available solution to writer of the module is
to say to the user​: if it will crash recompile your perl linked with
pthreads or use LD_PRELOAD=/path/to/pthreads.so hack.

So, my proposal is to always link perl with system threads library
to allow to write threaded XS modules for all types of perl.

If you can propose how to guarantee that all Perl ops are executed in
the main thread, or, more to the point, how Perl's global state is
only visible from a single thread, then yes, it might make sense to
have the non-threaded Perl executable capable of loading and running
threaded components. Such a component could really not be properly
called a "threaded XS module." XS code does lots of manipulation of
Perl's global data, and the only sanctioned way to do that under
threads is to clone most of that data once per thread and then
carefully synchronize via mutexes those few remaining bits of global
data that are not cloned. Please correct me if I've misunderstood
what you're asking.

@p5pRT
Copy link
Author

p5pRT commented Oct 5, 2014

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Oct 5, 2014

From @bulk88

Unthreaded perl doesn't need to know about OS threads. If your app is crashing because an unthreaded perl is running in 2 different OS threads simultaneously, well, then you need to fix the XS code that passed a function pointer that contains a call_sv() to some non-Perl C function which started a thread, or ran the function pointer in a thread pool thread. It is completely safe to run a perl interp on different OS threads provided that all perl C calls (and eventually Perl language calls in the runloop) are serialized. They have to be serialized using OS specific APIs (mutex, critical section, semaphore, whatever).

Read http​://perlmonks.org/?node_id=973277 and http​://perlmonks.org/?node=870516

--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

p5pRT commented Oct 5, 2014

From @Tux

On Sat, 4 Oct 2014 06​:49​:35 -0700, _______ _ (via RT)
<perlbug-followup@​perl.org> wrote​:

For now we have two perl types​: threaded and non-threaded.
Threaded is linked with threads library, non-threaded is not.

Most of them are not. Some are.

e.g. on HP-UX all perls are linked with -lpthread, as that is required
by some XS (notably Oracle) even if neither is actually using threads

The problem comes when we want to make XS module which
will use system threads.

s/use/require/

If an .so is linked with -lpthread, it needs it for loading, even if
none of the functionality of the lib is used.

In fact this module will crash on all non-threaded perls, because

s/crash/not load/

threads library is not linked with this perl and proper
initialization will not be done on perl start.
See​: http​://stackoverflow.com/questions/13587325/threads-vs-pthread-in-perl

Of course someone can use '-A prepend​:libswanted="pthread "'

yes, that is mentioned in several places in the docs

to compile non-threaded perl linked with pthreads, but nobody
do this.

I do (for all my HP-UX perl distributions)

The only available solution to writer of the module is to say to
the user​: if it will crash recompile your perl linked with pthreads
or use LD_PRELOAD=/path/to/pthreads.so hack.

So, my proposal is to always link perl with system threads library
to allow to write threaded XS modules for all types of perl.

I will not object, but it should *NOT* block the build if
libpthread.$so_ext is not available. As in, it can be added to
$libs_wanted, but it shall not be a requirement. There are e.g. still
systems that have/use pth.sl instead of pthread.so

--
H.Merijn Brand http​://tux.nl Perl Monger http​://amsterdam.pm.org/
using perl5.00307 .. 5.19 porting perl5 on HP-UX, AIX, and openSUSE
http​://mirrors.develooper.com/hpux/ http​://www.test-smoke.org/
http​://qa.perl.org http​://www.goldmark.org/jeff/stupid-disclaimers/

@p5pRT
Copy link
Author

p5pRT commented Oct 5, 2014

From verdrehung@gmail.com

In fact this report is not about manipulating perl structures in
separate thread. Even if your XS code creates thread for its own needs
and just calls some functions from libc in this thread it may cause
perl to crash if perl itself is not linked with threads library.
Example with my module​:

use strict;
use Net​::DNS​::Native;

my $dns = Net​::DNS​::Native->new(pool => 3);

my @​sock;

for (1..300) {
  my $sock = $dns->getaddrinfo("localhost")
;
  push @​sock, $sock;
}

my $buf;
for my $sock (@​sock) {
  sysread($sock, $buf, 1);
  $dns->get_result($sock);
}

This will work successfully if perl linked with pthreads and always
crashes with segfault or "double free or corruption" if not.
Furthermore this is not specific to perl. This plain C example will
also crash if we will not link mainc.c with pthreads​:
https://github.com/olegwtf/sandbox/tree/master/threaded-so-for-non-threaded-app
In all this examples i use getaddrinfo(3), but i think this is not
only the case and it will crash with other libc functions too.
Moreover, on NetBSD, for example, abort() will be called immediately
after your shared library will try to create thread and main app
loaded this library is not linked with pthreads.

2014-10-05 9​:48 GMT+07​:00 Craig Berry via RT <perlbug-followup@​perl.org>​:

On Sat, Oct 4, 2014 at 8​:49 AM, Ð�лег Ð� <perlbug-followup@​perl.org> wrote​:

# New Ticket Created by �лег �
# Please include the string​: [perl #122906]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=122906 >

This is a bug report for perl from oleg@​cpan.org,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
[Please describe your issue here]
For now we have two perl types​: threaded and non-threaded.
Threaded is linked with threads library, non-threaded is not.
The problem comes when we want to make XS module which
will use system threads. In fact this module will crash on all
non-threaded perls, because threads library is not linked with
this perl and proper initialization will not be done on perl start.
See​: http​://stackoverflow.com/questions/13587325/threads-vs-pthread-in-perl

Of course someone can use '-A prepend​:libswanted="pthread "'
to compile non-threaded perl linked with pthreads, but nobody
do this. The only available solution to writer of the module is
to say to the user​: if it will crash recompile your perl linked with
pthreads or use LD_PRELOAD=/path/to/pthreads.so hack.

So, my proposal is to always link perl with system threads library
to allow to write threaded XS modules for all types of perl.

If you can propose how to guarantee that all Perl ops are executed in
the main thread, or, more to the point, how Perl's global state is
only visible from a single thread, then yes, it might make sense to
have the non-threaded Perl executable capable of loading and running
threaded components. Such a component could really not be properly
called a "threaded XS module." XS code does lots of manipulation of
Perl's global data, and the only sanctioned way to do that under
threads is to clone most of that data once per thread and then
carefully synchronize via mutexes those few remaining bits of global
data that are not cloned. Please correct me if I've misunderstood
what you're asking.

@p5pRT
Copy link
Author

p5pRT commented Oct 5, 2014

From verdrehung@gmail.com

In fact this report is not about manipulating perl structures in
separate thread. Even if your XS code creates thread for its own needs
and just calls some functions from libc in this thread it may cause
perl to crash if perl itself is not linked with threads library.
Example with my module​:

use strict;
use Net​::DNS​::Native;

my $dns = Net​::DNS​::Native->new(pool => 3);

my @​sock;

for (1..300) {
  my $sock = $dns->getaddrinfo("localhost");
  push @​sock, $sock;
}

my $buf;
for my $sock (@​sock) {
  sysread($sock, $buf, 1);
  $dns->get_result($sock);
}

This will work successfully if perl linked with pthreads and always
crashes with segfault or "double free or corruption" if not.
Furthermore this is not specific to perl. This plain C example will
also crash if we will not link mainc.c with pthreads​:
https://github.com/olegwtf/sandbox/tree/master/threaded-so-for-non-threaded-app
In all this examples i use getaddrinfo(3), but i think this is not
only the case and it will crash with other libc functions too.
Moreover, on NetBSD, for example, abort() will be called immediately
after your shared library will try to create thread and main app
loaded this library not linked with pthreads.

2014-10-05 12​:51 GMT+07​:00 bulk88 via RT <perlbug-followup@​perl.org>​:

Unthreaded perl doesn't need to know about OS threads. If your app is crashing because an unthreaded perl is running in 2 different OS threads simultaneously, well, then you need to fix the XS code that passed a function pointer that contains a call_sv() to some non-Perl C function which started a thread, or ran the function pointer in a thread pool thread. It is completely safe to run a perl interp on different OS threads provided that all perl C calls (and eventually Perl language calls in the runloop) are serialized. They have to be serialized using OS specific APIs (mutex, critical section, semaphore, whatever).

Read http​://perlmonks.org/?node_id=973277 and http​://perlmonks.org/?node=870516

--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

p5pRT commented Oct 5, 2014

From verdrehung@gmail.com

Ð� Ñ�ообÑ�ении оÑ� 5 окÑ�Ñ�бÑ�Ñ� 2014 09​:48​:32 авÑ�оÑ� Craig Berry via RT напиÑ�ал​:

If you can propose how to guarantee that all Perl ops are executed in
the main thread, or, more to the point, how Perl's global state is
only visible from a single thread, then yes, it might make sense to
have the non-threaded Perl executable capable of loading and running
threaded components. Such a component could really not be properly
called a "threaded XS module." XS code does lots of manipulation of
Perl's global data, and the only sanctioned way to do that under
threads is to clone most of that data once per thread and then
carefully synchronize via mutexes those few remaining bits of global
data that are not cloned. Please correct me if I've misunderstood
what you're asking.

In fact this report is not about manipulating perl structures in
separate thread. Even if your XS code creates thread for its own needs
and just calls some functions from libc in this thread it may cause
perl to crash if perl itself is not linked with threads library.
Example with my module​:

use strict;
use Net​::DNS​::Native;

my $dns = Net​::DNS​::Native->new(pool => 3);

my @​sock;

for (1..300) {
  my $sock = $dns->getaddrinfo("localhost");
  push @​sock, $sock;
}

my $buf;
for my $sock (@​sock) {
  sysread($sock, $buf, 1);
  $dns->get_result($sock);
}

This will work successfully if perl linked with pthreads and always
crashes with segfault or "double free or corruption" if not.
Furthermore this is not specific to perl. This plain C example will
also crash if we will not link mainc.c with pthreads​:
https://github.com/olegwtf/sandbox/tree/master/threaded-so-for-non-threaded-
app
In all this examples i used getaddrinfo(3), but i think this is not
only the case and it will crash with other libc functions too.
Moreover, on NetBSD, for example, abort() will be called immediately
after your shared library will try to create thread and main app
loaded this library is not linked with pthreads.

@p5pRT
Copy link
Author

p5pRT commented Oct 6, 2014

From @Leont

On Sat, Oct 4, 2014 at 3​:49 PM, Ð�лег Ð� <perlbug-followup@​perl.org> wrote​:

For now we have two perl types​: threaded and non-threaded.
Threaded is linked with threads library, non-threaded is not.
The problem comes when we want to make XS module which
will use system threads. In fact this module will crash on all
non-threaded perls, because threads library is not linked with
this perl and proper initialization will not be done on perl start.
See​:
http​://stackoverflow.com/questions/13587325/threads-vs-pthread-in-perl

Of course someone can use '-A prepend​:libswanted="pthread "'
to compile non-threaded perl linked with pthreads, but nobody
do this. The only available solution to writer of the module is
to say to the user​: if it will crash recompile your perl linked with
pthreads or use LD_PRELOAD=/path/to/pthreads.so hack.

So, my proposal is to always link perl with system threads library
to allow to write threaded XS modules for all types of perl.

As far as I know, that issue is only common on a some platforms, but I may
be mistaken.

There is another reason to do this though​: libpthread may override symbols
from libc. On current Linux the only example seems to be raise(), but
historically it has included fork (because pthread_atfork AFAICT), vfork,
longjmp/siglongjmp and system. In this particular case not linking to
pthread in an executable that is indirectly multi-threaded may mean that a
signal is raised to the wrong thread in the process. Other similar issues
may exist on other platforms.

Leon

@p5pRT
Copy link
Author

p5pRT commented Oct 7, 2014

From verdrehung@gmail.com

Ð� Ñ�ообÑ�ении оÑ� 5 окÑ�Ñ�бÑ�Ñ� 2014 21​:10​:46 авÑ�оÑ� Oleg G напиÑ�ал​:

Ð� Ñ�ообÑ�ении оÑ� 5 окÑ�Ñ�бÑ�Ñ� 2014 09​:48​:32 авÑ�оÑ� Craig Berry via RT напиÑ�ал​:

If you can propose how to guarantee that all Perl ops are executed in
the main thread, or, more to the point, how Perl's global state is
only visible from a single thread, then yes, it might make sense to
have the non-threaded Perl executable capable of loading and running
threaded components. Such a component could really not be properly
called a "threaded XS module." XS code does lots of manipulation of
Perl's global data, and the only sanctioned way to do that under
threads is to clone most of that data once per thread and then
carefully synchronize via mutexes those few remaining bits of global
data that are not cloned. Please correct me if I've misunderstood
what you're asking.

In fact this report is not about manipulating perl structures in
separate thread. Even if your XS code creates thread for its own needs
and just calls some functions from libc in this thread it may cause
perl to crash if perl itself is not linked with threads library.
Example with my module​:

use strict;
use Net​::DNS​::Native;

my $dns = Net​::DNS​::Native->new(pool => 3);

my @​sock;

for (1..300) {
my $sock = $dns->getaddrinfo("localhost");
push @​sock, $sock;
}

my $buf;
for my $sock (@​sock) {
sysread($sock, $buf, 1);
$dns->get_result($sock);
}

This will work successfully if perl linked with pthreads and always
crashes with segfault or "double free or corruption" if not.
Furthermore this is not specific to perl. This plain C example will
also crash if we will not link mainc.c with pthreads​:
https://github.com/olegwtf/sandbox/tree/master/threaded-so-for-non-threaded-
app
In all this examples i used getaddrinfo(3), but i think this is not
only the case and it will crash with other libc functions too.
Moreover, on NetBSD, for example, abort() will be called immediately
after your shared library will try to create thread and main app
loaded this library is not linked with pthreads.

Today i played with gdb a little to find out what's going on here​:
https://github.com/olegwtf/sandbox/tree/master/threaded-so-for-non-threaded-app

And as gdb showed, app always crashed at rewind.c line 36​: http​://www.eglibc.org/cgi-bin/viewvc.cgi/branches/eglibc-2_13/libc/libio/rewind.c?annotate=12752
When i grepped elibc source i found two places where this macro was defined​:
*bits/stdio-lock.h line 49​: http​://www.eglibc.org/cgi-bin/viewvc.cgi/branches/eglibc-2_13/libc/bits/stdio-lock.h?annotate=12752
*sysdeps/pthread/bits/stdio-lock.h line 91​: http​://www.eglibc.org/cgi-bin/viewvc.cgi/branches/eglibc-2_13/libc/nptl/sysdeps/pthread/bits/stdio-lock.h?annotate=12752

So, first is default and second for threaded code. And if i understood correctly this is how it works​:
1. when our main app not linked with pthreads it loads default implementation of several macroses (i think not only stdio-lock has dual version)
2. when our threaded shared library loaded it starts using this non threads compatible default implementation
3. since this implementation is not thread safe it crashes

I think other libc implementations has same behavior. And only the case is to link main application (perl in our case) with threads library

P.S.
Sorry for duplicates of my previous message. You can delete two of three

@p5pRT
Copy link
Author

p5pRT commented Oct 7, 2014

From @Leont

On Mon, Oct 6, 2014 at 7​:34 PM, Oleg G <verdrehung@​gmail.com> wrote​:

Today i played with gdb a little to find out what's going on here​:

https://github.com/olegwtf/sandbox/tree/master/threaded-so-for-non-threaded-app

And as gdb showed, app always crashed at rewind.c line 36​:
http​://www.eglibc.org/cgi-bin/viewvc.cgi/branches/eglibc-2_13/libc/libio/rewind.c?annotate=12752
When i grepped elibc source i found two places where this macro was
defined​:
*bits/stdio-lock.h line 49​:
http​://www.eglibc.org/cgi-bin/viewvc.cgi/branches/eglibc-2_13/libc/bits/stdio-lock.h?annotate=12752
*sysdeps/pthread/bits/stdio-lock.h line 91​:
http​://www.eglibc.org/cgi-bin/viewvc.cgi/branches/eglibc-2_13/libc/nptl/sysdeps/pthread/bits/stdio-lock.h?annotate=12752

So, first is default and second for threaded code. And if i understood
correctly this is how it works​:
1. when our main app not linked with pthreads it loads default
implementation of several macroses (i think not only stdio-lock has dual
version)
2. when our threaded shared library loaded it starts using this non
threads compatible default implementation
3. since this implementation is not thread safe it crashes

I think other libc implementations has same behavior. And only the case is
to link main application (perl in our case) with threads library

This all makes sense to me.

Leon

@p5pRT
Copy link
Author

p5pRT commented Oct 12, 2014

From verdrehung@gmail.com

Ð� Ñ�ообÑ�ении оÑ� 4 окÑ�Ñ�бÑ�Ñ� 2014 20​:49​:15 авÑ�оÑ� Ð�лег Ð� напиÑ�ал​:

This is a bug report for perl from oleg@​cpan.org,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
[Please describe your issue here]
For now we have two perl types​: threaded and non-threaded.
Threaded is linked with threads library, non-threaded is not.
The problem comes when we want to make XS module which
will use system threads. In fact this module will crash on all
non-threaded perls, because threads library is not linked with
this perl and proper initialization will not be done on perl start.
See​: http​://stackoverflow.com/questions/13587325/threads-vs-pthread-in-perl

Of course someone can use '-A prepend​:libswanted="pthread "'
to compile non-threaded perl linked with pthreads, but nobody
do this. The only available solution to writer of the module is
to say to the user​: if it will crash recompile your perl linked with
pthreads or use LD_PRELOAD=/path/to/pthreads.so hack.

So, my proposal is to always link perl with system threads library
to allow to write threaded XS modules for all types of perl.

I made a simple patch. See attachment.
Is it correct?

Is this acceptable feature in general?

@p5pRT
Copy link
Author

p5pRT commented Oct 12, 2014

From verdrehung@gmail.com

0001-Link-with-pthread-always-if-it-is-available-https-rt.patch
From e8cd2fac0f79c44ab8e018ba0f702d3ba626ae13 Mon Sep 17 00:00:00 2001
From: Oleg G <verdrehung@gmail.com>
Date: Sun, 12 Oct 2014 19:37:20 +0700
Subject: [PATCH] Link with pthread always if it is available:
 https://rt.perl.org/Public/Bug/Display.html?id=122906

---
 Configure |    8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Configure b/Configure
index 95909f2..85e11c9 100755
--- a/Configure
+++ b/Configure
@@ -16841,6 +16841,14 @@ else
     old_pthread_create_joinable=""
 fi
 
+: do we need to link with pthreads
+if test "X$usethreads" != "X$define" -a "X$i_pthread" = "X$define"; then
+	case "$libs" in 
+		*"-lpthread"*);;
+		*) libs="$libs -lpthread";;
+	esac
+fi
+
 : see if pause exists
 set pause d_pause
 eval $inlibc
-- 
1.7.10.4

@p5pRT
Copy link
Author

p5pRT commented Oct 12, 2014

From @Leont

On Sun, Oct 12, 2014 at 3​:05 PM, Oleg G <verdrehung@​gmail.com> wrote​:

I made a simple patch. See attachment.
Is it correct?

No. It should only do this if there is in fact a libpthread available, and
even then there should be a way to disable it (I imagine embedded
environments may not carry libpthread for example)

Leon

@p5pRT
Copy link
Author

p5pRT commented Oct 13, 2014

From @Tux

On Sun, 12 Oct 2014 20​:05​:13 +0700, Oleg G <verdrehung@​gmail.com> wrote​:

Ð� Ñ�ообÑ�ении оÑ� 4 окÑ�Ñ�бÑ�Ñ� 2014 20​:49​:15 авÑ�оÑ� Ð�лег Ð� напиÑ�ал​:

This is a bug report for perl from oleg@​cpan.org,
generated with the help of perlbug 1.39 running under perl 5.14.2.

-----------------------------------------------------------------
[Please describe your issue here]
For now we have two perl types​: threaded and non-threaded.
Threaded is linked with threads library, non-threaded is not.
The problem comes when we want to make XS module which
will use system threads. In fact this module will crash on all
non-threaded perls, because threads library is not linked with
this perl and proper initialization will not be done on perl start.
See​: http​://stackoverflow.com/questions/13587325/threads-vs-pthread-in-perl

Of course someone can use '-A prepend​:libswanted="pthread "'
to compile non-threaded perl linked with pthreads, but nobody
do this. The only available solution to writer of the module is
to say to the user​: if it will crash recompile your perl linked with
pthreads or use LD_PRELOAD=/path/to/pthreads.so hack.

So, my proposal is to always link perl with system threads library
to allow to write threaded XS modules for all types of perl.

I made a simple patch. See attachment.
Is it correct?

No, pthread should be added to $libswanted like

Inline Patch
--- a/Configure  2014-10-13 08:21:29.769163741 +0200
+++ b/Configure  2014-10-13 08:22:22.831782753 +0200
@@ -1408,7 +1408,7 @@ libswanted_uselargefiles=''
 : List of libraries we want.
 : If anyone needs extra -lxxx, put those in a hint file.
 libswanted="socket bind inet nsl nm ndbm gdbm dbm db malloc dl ld sun"
-libswanted="$libswanted m crypt sec util c cposix posix ucb bsd BSD"
+libswanted="$libswanted m crypt sec util c cposix posix ucb bsd BSD pthread"
 : We probably want to search /usr/shlib before most other libraries.
 : This is only used by the lib/ExtUtils/MakeMaker.pm routine extliblist.
 glibpth=`echo " $glibpth " | sed -e 's! /usr/shlib ! !'`

> Is this acceptable feature in general?

--
H.Merijn Brand http​://tux.nl Perl Monger http​://amsterdam.pm.org/
using perl5.00307 .. 5.19 porting perl5 on HP-UX, AIX, and openSUSE
http​://mirrors.develooper.com/hpux/ http​://www.test-smoke.org/
http​://qa.perl.org http​://www.goldmark.org/jeff/stupid-disclaimers/

@p5pRT
Copy link
Author

p5pRT commented Oct 20, 2014

From verdrehung@gmail.com

2014-10-13 5​:19 GMT+07​:00 Leon Timmermans via RT <perlbug-followup@​perl.org>​:

On Sun, Oct 12, 2014 at 3​:05 PM, Oleg G <verdrehung@​gmail.com> wrote​:

I made a simple patch. See attachment.
Is it correct?

No. It should only do this if there is in fact a libpthread available, and
even then there should be a way to disable it (I imagine embedded
environments may not carry libpthread for example)

Leon

Thanks. I reworked my patch a little. So now it checks is libpthread
available and someone can
disable linking with pthreads by appending -U libpthread option to
Configure script. I am still
directly use $libs instead of $libswanted. Don't know is there any
reason to use $libswanted

@p5pRT
Copy link
Author

p5pRT commented Oct 20, 2014

From verdrehung@gmail.com

0001-Always-link-with-libpthread-if-it-is-available-https.patch
From 79503b158efe0204c5c597ef90295a7f0ac75a86 Mon Sep 17 00:00:00 2001
From: Oleg <verdrehung@gmail.com>
Date: Mon, 20 Oct 2014 14:32:20 +0700
Subject: [PATCH] Always link with libpthread if it is available:
 https://rt.perl.org/Public/Bug/Display.html?id=122906

---
 Configure | 29 ++++++++++++++++++++++++++++-
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/Configure b/Configure
index caa93a0..322bf7d 100755
--- a/Configure
+++ b/Configure
@@ -1025,6 +1025,7 @@ longsize=''
 shortsize=''
 issymlink=''
 libc=''
+libpthread=''
 ldlibpthname=''
 libperl=''
 shrpenv=''
@@ -1386,6 +1387,10 @@ test -d /usr/lib64         && glibpth="$glibpth /lib64 /usr/lib64 /usr/local/lib
 : machines, like the mips.  Usually, it should be empty.
 plibpth=''
 
+: On most systems perl should be linked with pthreads to make available
+: using of threaded shared libraries
+libpthread="$define"
+
 : default library list
 libswanted=''
 : some systems want to use only the non-versioned libso:s
@@ -16844,6 +16849,28 @@ else
     old_pthread_create_joinable=""
 fi
 
+: do we need to link with pthreads
+case "$libs" in
+	*"-lpthread"*) libpthread="$define";;
+	*)
+	if test "X$libpthread" = "X$define"; then
+		$cat >try.c <<EOM
+int main (int argc, char *argv[])
+{
+    return 0;
+}
+EOM
+		if $cc -o try $ccflags $ldflags -lpthread try.c >/dev/null 2>&1; then
+			echo "You has libpthread. Appending to \$libs." >&4
+			libs="$libs -lpthread"
+		else
+			libpthread=''
+		fi
+		$rm -f try try.*
+	fi
+	;;
+esac
+
 : see if pause exists
 set pause d_pause
 eval $inlibc
@@ -24168,6 +24195,7 @@ lib_ext='$lib_ext'
 libc='$libc'
 libperl='$libperl'
 libpth='$libpth'
+libpthread='$libpthread'
 libs='$libs'
 libsdirs='$libsdirs'
 libsfiles='$libsfiles'
@@ -24615,4 +24643,3 @@ $rm -f kit*isdone ark*isdone
 $rm -rf UU
 
 : End of Configure
-
-- 
1.8.4.4

@p5pRT
Copy link
Author

p5pRT commented Oct 30, 2014

From verdrehung@gmail.com

2014-10-20 14​:45 GMT+07​:00 Ð�лег Ð� <verdrehung@​gmail.com>​:

2014-10-13 5​:19 GMT+07​:00 Leon Timmermans via RT <perlbug-followup@​perl.org>​:

On Sun, Oct 12, 2014 at 3​:05 PM, Oleg G <verdrehung@​gmail.com> wrote​:

I made a simple patch. See attachment.
Is it correct?

No. It should only do this if there is in fact a libpthread available, and
even then there should be a way to disable it (I imagine embedded
environments may not carry libpthread for example)

Leon

Thanks. I reworked my patch a little. So now it checks is libpthread
available and someone can
disable linking with pthreads by appending -U libpthread option to
Configure script. I am still
directly use $libs instead of $libswanted. Don't know is there any
reason to use $libswanted

Does anyone interested in this feature? If no, feel free to reject this ticket.
The worst thing, I think, is to leave this ticket opened for a years.

@p5pRT
Copy link
Author

p5pRT commented Dec 11, 2014

@Tux - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented Jun 2, 2015

From @khwilliamson

Thanks for submitting this ticket

The issue should be resolved with the release today of Perl v5.22. If you find that the problem persists, feel free to reopen this ticket

--
Karl Williamson for the Perl 5 porters team

@p5pRT
Copy link
Author

p5pRT commented Jun 2, 2015

@khwilliamson - Status changed from 'pending release' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant