Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch some core modules to XSLoader #16147

Closed
p5pRT opened this issue Sep 13, 2017 · 21 comments
Closed

Switch some core modules to XSLoader #16147

p5pRT opened this issue Sep 13, 2017 · 21 comments

Comments

@p5pRT
Copy link

p5pRT commented Sep 13, 2017

Migrated from rt.perl.org#132080 (status was 'resolved')

Searchable as RT132080$

@p5pRT
Copy link
Author

p5pRT commented Sep 13, 2017

From @atoomic

This is a bug report for perl from atoomic@​cpan.org,
generated with the help of perlbug 1.40 running under perl 5.26.0.


The traditional boiler plate to use XSLoader or DynaLoader depending on the
version of perl ( for versions <5.006) do not make sense for core modules.

This is ok for CPAN modules which can be installed on any perl version, but
when we know we are using a modern perl version we should simply try to use
XSLoader.

IMO with perl 5.28+ we have no reason to keep that legacy pattern.

Also note that the extra eval was removed in PathTools as part of this
commit.

The commit is attached to this message and can also be read online at​:
atoomic@b917b36



Flags​:
  category=core
  severity=low


Site configuration information for perl 5.26.0​:

Configured by cPanel at Thu Aug 31 17​:31​:39 CDT 2017.

Summary of my perl5 (revision 5 version 26 subversion 0) configuration​:

  Platform​:
  osname=linux
  osvers=3.10.0-123.20.1.el7.x86_64
  archname=x86_64-linux-64int
  uname='linux rpmbuild-64-centos-7.dev.cpanel.net
3.10.0-123.20.1.el7.x86_64 #1 smp thu jan 29 18​:05​:33 utc 2015 x86_64
x86_64 x86_64 gnulinux '
  config_args='-des -Dusedevel -Darchname=x86_64-linux-64int
-Dcc=/usr/bin/gcc -Dcpp=/usr/bin/cpp -Dusemymalloc=n -DDEBUGGING=none
-Doptimize=-Os -Accflags=-m64 -Dccflags=-DPERL_DISABLE_PMC -fPIC -DPIC
-I/usr/local/cpanel/3rdparty/perl/526/include
-I/usr/local/cpanel/3rdparty/include -L/usr/local/cpanel/3rdparty/lib64
-Duseshrplib -Duselargefiles=yes -Duseposix=true -Dhint=recommended
-Duseperlio=yes -Dcppflags=-I/usr/local/cpanel/3rdparty/perl/526/include
-I/usr/local/cpanel/3rdparty/include -L/usr/local/cpanel/3rdparty/lib64
-Dldflags=-L/usr/local/cpanel/3rdparty/lib64
-Dprefix=/usr/local/cpanel/3rdparty/perl/526
-Dsiteprefix=/opt/cpanel/perl5/526 -Dsitebin=/opt/cpanel/perl5/526/bin
-Dsitelib=/opt/cpanel/perl5/526/site_lib -Dusevendorprefix=true
-Dvendorbin=/usr/local/cpanel/3rdparty/perl/526/bin
-Dvendorprefix=/usr/local/cpanel/3rdparty/perl/526/lib64/perl5
-Dvendorlib=/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib
-Dprivlib=/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/5.26.0
-Dman1dir=none -Dman3dir=none
-Dscriptdir=/usr/local/cpanel/3rdparty/perl/526/bin
-Dscriptdirexp=/usr/local/cpanel/3rdparty/perl/526/bin -Dsiteman1dir=none
-Dsiteman3dir=none -Dinstallman1dir=none -Dversiononly=no
-Dinstallusrbinperl=no -Dcf_by=cPanel -Dmyhostname=localhost
-Dperladmin=root@​localhost -Dcf_email=support@​cpanel.net
-Di_dbm=/usr/local/cpanel/3rdparty/include
-Di_gdbm=/usr/local/cpanel/3rdparty/include
-Di_ndbm=/usr/local/cpanel/3rdparty/include -DDB_File=true -Ud_dosuid
-Uuserelocatableinc -Umad -Uusethreads -Uusemultiplicity -Uusesocks
-Uuselongdouble -Aldflags=-L/usr/local/cpanel/3rdparty/lib64 -L/usr/lib64
-L/lib64 -lgdbm -Dlocincpth=/usr/local/cpanel/3rdparty/perl/526/include
/usr/local/cpanel/3rdparty/include /usr/local/include
-Duse64bitint -Uuse64bitall -Dlibpth=/usr/local/cpanel/3rdparty/lib64
/usr/local/lib64 /usr/local/lib /lib64 /usr/lib64 '
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=undef
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='/usr/bin/gcc'
  ccflags ='-DPERL_DISABLE_PMC -fPIC -DPIC
-I/usr/local/cpanel/3rdparty/perl/526/include
-I/usr/local/cpanel/3rdparty/include -L/usr/local/cpanel/3rdparty/lib64
-m64 -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
-D_FORTIFY_SOURCE=2'
  optimize='-Os'
  cppflags='-I/usr/local/cpanel/3rdparty/perl/526/include
-I/usr/local/cpanel/3rdparty/include -L/usr/local/cpanel/3rdparty/lib64
-DPERL_DISABLE_PMC -fPIC -DPIC
-I/usr/local/cpanel/3rdparty/perl/526/include
-I/usr/local/cpanel/3rdparty/include -L/usr/local/cpanel/3rdparty/lib64
-m64 -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong
-I/usr/local/include'
  ccversion=''
  gccversion='4.8.2 20140120 (Red Hat 4.8.2-16)'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='/usr/bin/gcc'
  ldflags ='-L/usr/local/cpanel/3rdparty/lib64
-L/usr/local/cpanel/3rdparty/lib64 -L/usr/lib64 -L/lib64 -lgdbm
-fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/cpanel/3rdparty/lib64 /usr/local/lib64 /usr/local/lib
/lib64 /usr/lib64 /usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64
/lib
  libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
-lgdbm_compat
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.17.so
  so=so
  useshrplib=true
  libperl=libperl.so
  gnulibc_version='2.17'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E
-Wl,-rpath,/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/5.26.0/x86_64-linux-64int/CORE'
  cccdlflags='-fPIC'
  lddlflags='-shared -Os -L/usr/local/cpanel/3rdparty/lib64 -L/usr/lib64
-L/lib64 -L/usr/local/lib -fstack-protector-strong'

Locally applied patches​:
  cPanel patches
  cPanel INC path changes
  cPanel performance improvements to modules
  cPanel Immortal COW
  cPanel B and O performance fixups
  cPanel B​::C Declare Static Memory malloc patches
  cPanel Disable XS handshake


@​INC for perl 5.26.0​:
  /root/.dotfiles/perl-must-have/lib
  /root/perl5/lib/perl5/
  /usr/local/cpanel

/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/x86_64-linux-64int
  /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib

/usr/local/cpanel/3rdparty/perl/526/lib64/perl5/5.26.0/x86_64-linux-64int
  /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/5.26.0
  /opt/cpanel/perl5/526/site_lib/x86_64-linux-64int
  /opt/cpanel/perl5/526/site_lib


Environment for perl 5.26.0​:
  HOME=/root
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)

PATH=/usr/local/cpanel/3rdparty/perl/526/bin​:/usr/local/cpanel/3rdparty/perl/524/bin​:/usr/local/cpanel/3rdparty/perl/522/bin​:/usr/local/cpanel/3rdparty/perl/514/bin​:/usr/local/cpanel/3rdparty/bin​:/root/bin/​:/opt/local/bin​:/opt/local/sbin​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/opt/cpanel/composer/bin​:/root/.dotfiles/bin​:/root/perl5/bin​:/root/.rvm/bin​:/root/bin
  PERL5DB=use Devel​::NYTProf
  PERL5LIB=/root/.dotfiles/perl-must-have/lib​::/root/perl5/lib/perl5/
  PERL_BADLANG (unset)
  PERL_CPANM_OPT=--quiet
  SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Sep 13, 2017

From @atoomic

0001-Switch-some-core-modules-to-XSLoader.patch
From b917b3622159892d1fc540f100c7536700512b70 Mon Sep 17 00:00:00 2001
From: Nicolas R <atoomic@cpan.org>
Date: Wed, 13 Sep 2017 11:22:32 -0600
Subject: [PATCH] Switch some core modules to XSLoader

Remove perl 5.006 compatibilities with DynaLoader
and use XSLoader directly.

The traditional boiler plate to use XSLoader
for perl > 5.006 or DynaLoader, do not make sense
for core modules in perl 5.28+.

This is wanted for CPAN modules which can be install
on any perl version.
---
 dist/PathTools/Cwd.pm                | 18 ++++--------------
 dist/PathTools/lib/File/Spec/Unix.pm | 14 +++-----------
 dist/Time-HiRes/HiRes.pm             |  8 ++++----
 dist/Unicode-Normalize/Normalize.pm  |  8 ++++----
 4 files changed, 15 insertions(+), 33 deletions(-)

diff --git a/dist/PathTools/Cwd.pm b/dist/PathTools/Cwd.pm
index cc77e58e2b..90d9e6e47c 100644
--- a/dist/PathTools/Cwd.pm
+++ b/dist/PathTools/Cwd.pm
@@ -3,7 +3,7 @@ use strict;
 use Exporter;
 use vars qw(@ISA @EXPORT @EXPORT_OK $VERSION);
 
-$VERSION = '3.68';
+$VERSION = '3.69';
 my $xs_version = $VERSION;
 $VERSION =~ tr/_//d;
 
@@ -77,19 +77,9 @@ sub _vms_efs {
 
 
 # If loading the XS stuff doesn't work, we can fall back to pure perl
-if(! defined &getcwd && defined &DynaLoader::boot_DynaLoader) {
-  eval {#eval is questionable since we are handling potential errors like
-        #"Cwd object version 3.48 does not match bootstrap parameter 3.50
-        #at lib/DynaLoader.pm line 216." by having this eval
-    if ( $] >= 5.006 ) {
-      require XSLoader;
-      XSLoader::load( __PACKAGE__, $xs_version);
-    } else {
-      require DynaLoader;
-      push @ISA, 'DynaLoader';
-      __PACKAGE__->bootstrap( $xs_version );
-    }
-  };
+if(! defined &getcwd && defined &DynaLoader::boot_DynaLoader) { # skipped on miniperl
+    require XSLoader;
+    XSLoader::load( __PACKAGE__, $xs_version);
 }
 
 # Big nasty table of function aliases
diff --git a/dist/PathTools/lib/File/Spec/Unix.pm b/dist/PathTools/lib/File/Spec/Unix.pm
index e1a30f8ca1..6249843fd0 100644
--- a/dist/PathTools/lib/File/Spec/Unix.pm
+++ b/dist/PathTools/lib/File/Spec/Unix.pm
@@ -3,22 +3,14 @@ package File::Spec::Unix;
 use strict;
 use vars qw($VERSION);
 
-$VERSION = '3.68';
+$VERSION = '3.69';
 my $xs_version = $VERSION;
 $VERSION =~ tr/_//d;
 
-#dont try to load XSLoader and DynaLoader only to ultimately fail on miniperl
-if(!defined &canonpath && defined &DynaLoader::boot_DynaLoader) {
-  eval {#eval is questionable since we are handling potential errors like
-        #"Cwd object version 3.48 does not match bootstrap parameter 3.50
-        #at lib/DynaLoader.pm line 216." by having this eval
-    if ( $] >= 5.006 ) {
+#dont try to load XSLoader only to ultimately fail on miniperl
+if(!defined &canonpath && defined &DynaLoader::boot_DynaLoader) { # skipped on miniperl
 	require XSLoader;
 	XSLoader::load("Cwd", $xs_version);
-    } else {
-	require Cwd;
-    }
-  };
 }
 
 =head1 NAME
diff --git a/dist/Time-HiRes/HiRes.pm b/dist/Time-HiRes/HiRes.pm
index fbdc2b4aca..a57c77147d 100644
--- a/dist/Time-HiRes/HiRes.pm
+++ b/dist/Time-HiRes/HiRes.pm
@@ -4,9 +4,9 @@ package Time::HiRes;
 use strict;
 
 require Exporter;
-require DynaLoader;
+use XSLoader ();
 
-our @ISA = qw(Exporter DynaLoader);
+our @ISA = qw(Exporter);
 
 our @EXPORT = qw( );
 our @EXPORT_OK = qw (usleep sleep ualarm alarm gettimeofday time tv_interval
@@ -28,7 +28,7 @@ our @EXPORT_OK = qw (usleep sleep ualarm alarm gettimeofday time tv_interval
 		 stat lstat utime
 		);
 
-our $VERSION = '1.9743';
+our $VERSION = '1.9744';
 our $XS_VERSION = $VERSION;
 $VERSION = eval $VERSION;
 
@@ -69,7 +69,7 @@ sub import {
     Time::HiRes->export_to_level(1, $this, @_);
 }
 
-bootstrap Time::HiRes;
+XSLoader::load( 'Time::HiRes', $XS_VERSION );
 
 # Preloaded methods go here.
 
diff --git a/dist/Unicode-Normalize/Normalize.pm b/dist/Unicode-Normalize/Normalize.pm
index b53f5c728b..adf3db50d8 100644
--- a/dist/Unicode-Normalize/Normalize.pm
+++ b/dist/Unicode-Normalize/Normalize.pm
@@ -16,7 +16,7 @@ use Carp;
 
 no warnings 'utf8';
 
-our $VERSION = '1.25';
+our $VERSION = '1.26';
 our $PACKAGE = __PACKAGE__;
 
 our @EXPORT = qw( NFC NFD NFKC NFKD );
@@ -56,9 +56,9 @@ require Exporter;
 
 ##### The above part is common to XS and PP #####
 
-our @ISA = qw(Exporter DynaLoader);
-require DynaLoader;
-bootstrap Unicode::Normalize $VERSION;
+our @ISA = qw(Exporter);
+use XSLoader ();
+XSLoader::load( 'Unicode::Normalize', $VERSION );
 
 ##### The below part is common to XS and PP #####
 
-- 
2.14.1

@p5pRT
Copy link
Author

p5pRT commented Sep 13, 2017

From zefram@fysh.org

Nicolas R. wrote​:

The traditional boiler plate to use XSLoader or DynaLoader depending on the
version of perl ( for versions <5.006) do not make sense for core modules.

All the modules you've removed this from are dual-life, and so in
their CPAN incarnations can potentially be installed on pre-5.6 perls.
Coreness is only an argument against version portability for *core-only*
modules (ext/).

However, there's an independent argument that the value in portability to
pre-5.6 perls is negligible. (This applies to any CPAN module, whether
CPAN-only or dual-life.) It is reasonable to remove the boilerplate from
the dist/ modules simply for cleanliness and to reduce the maintenance
burden. The same argument also applies to the cpan/ modules, but those
need to be edited in their CPAN incarnation first.

Also note that the extra eval was removed in PathTools as part of this
commit.

PathTools is dual-life, and the eval supports installing from CPAN in
places where XS compilation isn't possible. This should not be removed
short of an explicit decision to drop compatibility to compilerless
installations.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Sep 13, 2017

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2017

From @atoomic

On Wed, 13 Sep 2017 16​:58​:02 -0700, zefram@​fysh.org wrote​:

Nicolas R. wrote​:

The traditional boiler plate to use XSLoader or DynaLoader depending on the
version of perl ( for versions <5.006) do not make sense for core modules.

All the modules you've removed this from are dual-life, and so in
their CPAN incarnations can potentially be installed on pre-5.6 perls.
Coreness is only an argument against version portability for *core-only*
modules (ext/).

Thanks for the correction indeed, all the modules updated as part of this patch are dual-life.

However, there's an independent argument that the value in portability to
pre-5.6 perls is negligible. (This applies to any CPAN module, whether
CPAN-only or dual-life.) It is reasonable to remove the boilerplate from
the dist/ modules simply for cleanliness and to reduce the maintenance
burden. The same argument also applies to the cpan/ modules, but those
need to be edited in their CPAN incarnation first.

Concerning the CPAN modules, I've already patched most of them in our local cpan mirror, to switch them to XSLoader. But as you mentioned this involve dropping support for versions earlier than 5.6 and I think opinions might differ.

Also note that the extra eval was removed in PathTools as part of this
commit.

PathTools is dual-life, and the eval supports installing from CPAN in
places where XS compilation isn't possible. This should not be removed
short of an explicit decision to drop compatibility to compilerless
installations.

I can restore the extra 'eval' protection from PathTools which I thought was mainly there to protect against miniperl, and was a duplicate of the logic in the if statement.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2017

From @toddr

On Wed, 13 Sep 2017 16​:58​:02 -0700, zefram@​fysh.org wrote​:

However, there's an independent argument that the value in portability to
pre-5.6 perls is negligible. (This applies to any CPAN module, whether
CPAN-only or dual-life.) It is reasonable to remove the boilerplate from
the dist/ modules simply for cleanliness and to reduce the maintenance
burden.

Yes I think that's the proposal here. For the handful of people using 5.6, isn't that what barnyard is for?

http​://cp5.6.2an.barnyard.co.uk/

Also note that the extra eval was removed in PathTools as part of this
commit.

PathTools is dual-life, and the eval supports installing from CPAN in
places where XS compilation isn't possible. This should not be removed
short of an explicit decision to drop compatibility to compilerless
installations.

I'd challenge this too. Is there really a big demand for compilerless use of PathTools? I'm not aware of many other modules that have this behavior or if they do, they solve it with Cwd​::PP right?

I'm guessing this was put in long ago and even when it was there was no evidence it had value.

Todd

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2017

From @Leont

On Thu, Sep 14, 2017 at 1​:42 AM, Nicolas R. <perlbug-followup@​perl.org>
wrote​:

The traditional boiler plate to use XSLoader or DynaLoader depending on the
version of perl ( for versions <5.006) do not make sense for core modules.

This is ok for CPAN modules which can be installed on any perl version, but
when we know we are using a modern perl version we should simply try to use
XSLoader.

IMO with perl 5.28+ we have no reason to keep that legacy pattern.

Also note that the extra eval was removed in PathTools as part of this
commit.

The commit is attached to this message and can also be read online at​:
atoomic@b917b36
6700512b70

ExtUtils​::MakeMaker dropped 5.5 support almost a decade ago, and so did
Test​::More. If anyone is still running 5.5 (I'm sure someone still is) they
are in no position to consider upgrading almost anything, really. Even
without that, any code containing «use warnings;» or 3-arg open will not
work on 5.5 (which probably includes a lot of the code we're discussing
right now).

5.6 is on intensive care, but 5.5 is dead and buried.

Leon

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2017

From zefram@fysh.org

Todd Rinaldo via RT wrote​:

For the handful of people using 5.6,

They'll still be OK because 5.6 supports "our". The support being dropped
is for perls *preceding* 5.6. It's been a very long time since a full
CPAN toolchain ran on pre-5.6 perls.

I'd challenge this too. Is there really a big demand for compilerless
use of PathTools?

Compilerless installations are regrettably common. Reducing the support
for them is no small matter.

I'm not aware of many other modules that have this behavior

Some of mine have such dual implementation in a single distro, such as
Params​::Classify and XML​::Easy. PathTools is exactly the kind of distro
for which this is a useful technique​: everything it does can be done in
pure Perl, but big performance gains are available by doing it in XS.
Going pure Perl only would limit the utility of the distro to people with
fully-capable installations, but going XS-only would exclude a sizeable
chunk of the audience, in either case making the distro less attractive
both to end users and as a dependency for other CPAN distros.

if they do, they solve it with Cwd​::PP right?

There are a few arranged like that (or the other way, with a :​:XS),
but it's inconvenient to the user to do that, so it's to be avoided
where possible. The problem is that it puts the onus of selecting
the implementation on the user, and if the immediate user is a CPAN
module then they don't necessarily know whether the end user can handle
an XS dependency. If there's to be automatic selection between two
implementations, again it's the using module that has to manage that.
Also, especially when the two implementations come from different authors
(which is a primary reason to package them separately in the first
place), there's a great risk of the two module interfaces diverging.
Many :​:PP/​::XS arrangements don't even attempt to make the interfaces
match, causing yet more pain to the user who tries to automatically
select between them.

Putting the two implementations in the same distro, behind the same
module interface, tightly couples them and requires that the author take
great care to keep their behaviour equivalent. But it also makes it as
easy as possible to keep them equivalent​: they can both be subjected
to the same test suite. It's worth this effort, to make the multiple
implementations invisible at the API level.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Sep 19, 2017

From @toddr

On Thu, 14 Sep 2017 15​:10​:00 -0700, zefram@​fysh.org wrote​:

Todd Rinaldo via RT wrote​:

For the handful of people using 5.6,

They'll still be OK because 5.6 supports "our". The support being dropped
is for perls *preceding* 5.6. It's been a very long time since a full
CPAN toolchain ran on pre-5.6 perls.

I'd challenge this too. Is there really a big demand for compilerless
use of PathTools?

Compilerless installations are regrettably common. Reducing the support
for them is no small matter.

I'm not aware of many other modules that have this behavior

Some of mine have such dual implementation in a single distro, such as
Params​::Classify and XML​::Easy. PathTools is exactly the kind of distro
for which this is a useful technique​: everything it does can be done in
pure Perl, but big performance gains are available by doing it in XS.
Going pure Perl only would limit the utility of the distro to people with
fully-capable installations, but going XS-only would exclude a sizeable
chunk of the audience, in either case making the distro less attractive
both to end users and as a dependency for other CPAN distros.

I'm going to bring this up for discussion since I've never seen it discussed before. It is not meant as a spark for flameage.

We seem to take the stance on dist modules that we have to support an unspecified lesser version of Perl. I think the lack of policy makes for some confusion when we discuss patches like this.

I suggest a rather draconian policy​: "The modules maintained in the dist tree of the perl source code should support blead and *MAYBE* the 2 maint versions but not have to consider support for older versions of Perl."

IMO and within reason, I suggest that it should be the dual life maintainer's job to get the dist module working on CPAN. This makes it so that core does not have to make its code do back flips in blead perl to work with perl 5.6. I think this could potentially simplify the code and potentially make programs smaller/faster.

Keep in mind that we are not spectacular at keeping dist up with what is on CPAN. IMO, that's perfectly ok. If you want the new hotness, upgrade! If module X has a versioning issue working on the blead version vs the CPAN version, then do a release or ask for a release.

This may be a weak example, but version.pm seems to do just this the last time I looked at it. Its CPAN version looks nothing like the code in CORE since some of the work its CPAN counterpart does is built in to modern versions of Perl.

Todd

@p5pRT
Copy link
Author

p5pRT commented Sep 19, 2017

From zefram@fysh.org

Todd Rinaldo via RT wrote​:

IMO and within reason, I suggest that it should be the dual life
maintainer's job to get the dist module working on CPAN. This makes it
so that core does not have to make its code do back flips in blead perl
to work with perl 5.6.

A basic aspect of how we manage dual-life modules is that the code in
blead is identical to what goes on CPAN. (We omit some CPAN distro
files entirely from the blead tree, but that's very simple to manage.)
We don't want to maintain a bunch of patches that have to be repeatedly
applied to make CPAN releases. We also don't want to persistently
have two different versions of the module; that's not really dual-life,
and would cause a lot of hassle with version numbers.

Backward compatibility of module code is a CPAN authoring issue, and for
the dist/ modules the author is the p5p collective. We do in principle
need to decide how far back we're willing to push the compatibility, and
we have to do the work to implement that compatibility. But it doesn't
necessarily need to be an explicit policy, and it certainly doesn't
have to be the same version threshold for all modules. It's sensible
for the threshold to tacitly be a limited amount of effort, rather than
a limited age of supported perl versions.

What actually happens is that, because we don't smoke dist/ modules
against older perls, the backcompat periodically rots, and eventually
gets fixed when someone can be bothered to look. (I fixed backcompat
for a few modules, especially to 5.6, a couple of months ago.) Actual
compatibility of what gets released to CPAN is variable, a random sample
of the fluctuating state of the blead code. Taken together, p5p acts
as a CPAN author who isn't at all systematic about testing perl version
compatibility.

Generally, backcompat code in a module isn't expensive on new perl
versions, because the only part of it that gets run on new versions is a
load-time version check. If you have a specific case where there seems to
be unwarranted expense, bring it up and we can see about refactoring it.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Sep 23, 2017

From @xsawyerx

On 09/19/2017 05​:52 PM, Todd Rinaldo via RT wrote​:

On Thu, 14 Sep 2017 15​:10​:00 -0700, zefram@​fysh.org wrote​:

Todd Rinaldo via RT wrote​:

For the handful of people using 5.6,
They'll still be OK because 5.6 supports "our". The support being dropped
is for perls *preceding* 5.6. It's been a very long time since a full
CPAN toolchain ran on pre-5.6 perls.

I'd challenge this too. Is there really a big demand for compilerless
use of PathTools?
Compilerless installations are regrettably common. Reducing the support
for them is no small matter.

I'm not aware of many other modules that have this behavior
Some of mine have such dual implementation in a single distro, such as
Params​::Classify and XML​::Easy. PathTools is exactly the kind of distro
for which this is a useful technique​: everything it does can be done in
pure Perl, but big performance gains are available by doing it in XS.
Going pure Perl only would limit the utility of the distro to people with
fully-capable installations, but going XS-only would exclude a sizeable
chunk of the audience, in either case making the distro less attractive
both to end users and as a dependency for other CPAN distros.
I'm going to bring this up for discussion since I've never seen it discussed before. It is not meant as a spark for flameage.

We seem to take the stance on dist modules that we have to support an unspecified lesser version of Perl. I think the lack of policy makes for some confusion when we discuss patches like this.

I suggest a rather draconian policy​: "The modules maintained in the dist tree of the perl source code should support blead and *MAYBE* the 2 maint versions but not have to consider support for older versions of Perl."

While I understand the desire for such a policy, I'm not sure it is in
the best interest of the Perl users community. I would rather have a
decision made per module when it comes to it. There's no reason to fix
before there's a problem.

Personally, I fail to understand why we should support 5.6, but I would
be happy to listen to a party making that claim.

In any case where such backcompat is in the way of a patch and that
patch cannot be easily revised, I would be happy to consider simply
removing the support for it. Some modules would require more leniency
(base.pm, parent.pm, and other core functionality ones) but otherwise,
we should stay open to removing support for grossly outdated versions
when we deem it either a major pain to support or have no one who is
able and willing to do the work.

@p5pRT
Copy link
Author

p5pRT commented Sep 23, 2017

From @Leont

On Tue, Sep 19, 2017 at 5​:52 PM, Todd Rinaldo via RT <
perlbug-followup@​perl.org> wrote​:

I'm going to bring this up for discussion since I've never seen it
discussed before. It is not meant as a spark for flameage.

We seem to take the stance on dist modules that we have to support an
unspecified lesser version of Perl. I think the lack of policy makes for
some confusion when we discuss patches like this.

I suggest a rather draconian policy​: "The modules maintained in the dist
tree of the perl source code should support blead and *MAYBE* the 2 maint
versions but not have to consider support for older versions of Perl."

IMO and within reason, I suggest that it should be the dual life
maintainer's job to get the dist module working on CPAN. This makes it so
that core does not have to make its code do back flips in blead perl to
work with perl 5.6. I think this could potentially simplify the code and
potentially make programs smaller/faster.

Keep in mind that we are not spectacular at keeping dist up with what is
on CPAN. IMO, that's perfectly ok. If you want the new hotness, upgrade! If
module X has a versioning issue working on the blead version vs the CPAN
version, then do a release or ask for a release.

That's rather missing an essential point though​: why would they drop
compatibility at all? Almost all of our modules are older than 5.6 anyway,
and hence are compatible with it. It's not that maintaining compatibility
is generally speaking work, it's "modernizing" code that is. Dropping 5.5
support allowed us to use warnings (which is a good trade-off to me), and
dropping 5.6 can give one proper unicode support, but what would dropping
5.8 support give us? Defined-or? say?

This may be a weak example, but version.pm seems to do just this the last

time I looked at it. Its CPAN version looks nothing like the code in CORE
since some of the work its CPAN counterpart does is built in to modern
versions of Perl.

version.pm is special it's functionality is used in implementing core
functionality (->VERSION($version) checking). And TBH I don't think we're
managing it well right now.

Leon

@p5pRT
Copy link
Author

p5pRT commented Sep 23, 2017

From zefram@fysh.org

Sawyer X wrote​:

Personally, I fail to understand why we should support 5.6,

Because some people still use it, and the cost of supporting it is
not extravagant. Though the cost rises over time, as toolchain and
infrastructure modules gradually lose support, to the point that testing
on 5.6 is now slipping beyond the bounds of practicality. Supporting
anything earlier than 5.6 would mean giving up "use warnings" and "our",
as well as establishing a very antiquated toolchain for testing, so few
people bother with that. 5.6.1 has been the minimum practical version
to port to and test on for a *long* time.

Relative to 5.6, the main gain of a 5.8 minimum is proper handling of
Unicode in basic string operations. This is becoming the new minimum
for the toolchain, with Unicode handling in META.json now taken to be
mandatory. It is this compelling single feature, rather than the large
number of small conveniences and bugfixes, that is driving the dropping of
5.6 support. Projecting this pattern into the future, there is no obvious
compelling feature that would drive the adoption of 5.10 or anything
later as the new minimum. Perhaps some of the new syntax stuff enabled
by 5.12 or 5.14 will become sufficiently widely used to make one of them
the practical minimum, but at the moment we're not seeing that happen.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Sep 24, 2017

From @xsawyerx

On 09/23/2017 06​:41 PM, Zefram wrote​:

Sawyer X wrote​:

Personally, I fail to understand why we should support 5.6,
Because some people still use it, and the cost of supporting it is
not extravagant.

True, this was a general statement, but I hoped to provide enough
context around it to provide a suggestion closer to your phrasing here.
While I "don't understand it," I also do not see a reason to avoid it if
the cost (in time, effort, resources - people who are able to do the
work) is "not extravagant."

Though the cost rises over time, as toolchain and
infrastructure modules gradually lose support, to the point that testing
on 5.6 is now slipping beyond the bounds of practicality. Supporting
anything earlier than 5.6 would mean giving up "use warnings" and "our",
as well as establishing a very antiquated toolchain for testing, so few
people bother with that. 5.6.1 has been the minimum practical version
to port to and test on for a *long* time.

"Impractical" in this context could also mean "too high a cost."

Relative to 5.6, the main gain of a 5.8 minimum is proper handling of
Unicode in basic string operations. This is becoming the new minimum
for the toolchain, with Unicode handling in META.json now taken to be
mandatory. It is this compelling single feature, rather than the large
number of small conveniences and bugfixes, that is driving the dropping of
5.6 support.

It appears toolchain is putting a lot of effort in tracking the lowest
viable version of Perl to support and I believe we should keep what
toolchain in mind when consider the versions we are willing to work hard
to support. If toolchain decides to bring the minimum version to 5.8, we
should at the very least discuss core module support for 5.6, possibly
dropping it. (This is not a commitment, of course.)

Projecting this pattern into the future, there is no obvious
compelling feature that would drive the adoption of 5.10 or anything
later as the new minimum. Perhaps some of the new syntax stuff enabled
by 5.12 or 5.14 will become sufficiently widely used to make one of them
the practical minimum, but at the moment we're not seeing that happen.

Agreed.

I wouldn't mind having a feature that promotes everyone to upgrade just
to have it, rendering all versions under 5.22 unused. :)

@p5pRT
Copy link
Author

p5pRT commented Oct 3, 2017

From @toddr

On Sun, 24 Sep 2017 13​:16​:57 -0700, xsawyerx@​gmail.com wrote​:

It appears toolchain is putting a lot of effort in tracking the lowest
viable version of Perl to support and I believe we should keep what
toolchain in mind when consider the versions we are willing to work hard
to support. If toolchain decides to bring the minimum version to 5.8, we
should at the very least discuss core module support for 5.6, possibly
dropping it. (This is not a commitment, of course.)

Perhaps it was a mistake to bring up this policy with regards to these patches. The patches listed are 5.6 compatible which seems to be our minimum toolchain version support is. Is there another reason not to merge this then?

The value here is that as long as Perl ships with modules that suggest Dynaloader is still needed, it's hard to set a good example elsewhere on CPAN.

Todd

@p5pRT
Copy link
Author

p5pRT commented Nov 2, 2017

From @toddr

On Tue, 03 Oct 2017 13​:50​:15 -0700, TODDR wrote​:

Perhaps it was a mistake to bring up this policy with regards to these
patches. The patches listed are 5.6 compatible which seems to be our
minimum toolchain version support is. Is there another reason not to
merge this then?

The value here is that as long as Perl ships with modules that suggest
Dynaloader is still needed, it's hard to set a good example elsewhere
on CPAN.

Barring any further concerns, we're going to merge this on on Monday.

Thanks,
Todd

@p5pRT
Copy link
Author

p5pRT commented Nov 13, 2017

From @toddr

Pushed to blead with commit b9a5a78

@p5pRT
Copy link
Author

p5pRT commented Nov 13, 2017

@toddr - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented Jun 23, 2018

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release yesterday of Perl 5.28.0, this and 185 other issues have been
resolved.

Perl 5.28.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.28.0

If you find that the problem persists, feel free to reopen this ticket.

@p5pRT
Copy link
Author

p5pRT commented Jun 23, 2018

@khwilliamson - Status changed from 'pending release' to 'resolved'

@p5pRT p5pRT closed this as completed Jun 23, 2018
@p5pRT
Copy link
Author

p5pRT commented Jun 25, 2018

From @atoomic

Thank you!

Le sam. 23 juin 2018 à 09​:32, Karl Williamson via RT <
perlbug-followup@​perl.org> a écrit :

According to our records, your request regarding
"Switch some core modules to XSLoader"
has been resolved.

If you have any further questions or concerns, please respond to this
message.

For other topics, please create a new ticket.

<URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132080 >

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant