Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

binaries mismatched again #16654

Open
p5pRT opened this issue Aug 10, 2018 · 41 comments
Open

binaries mismatched again #16654

p5pRT opened this issue Aug 10, 2018 · 41 comments

Comments

@p5pRT
Copy link

p5pRT commented Aug 10, 2018

Migrated from rt.perl.org#133440 (status was 'open')

Searchable as RT133440$

@p5pRT
Copy link
Author

p5pRT commented Aug 10, 2018

From frederik@ofb.net

Created by frederik@ofb.net

I was able to find that I already reported this bug, sorry​:

https://rt.perl.org/Public/Bug/Display.html?id=130718

Maybe I have a bad memory, but it came up again and I wrote a new report before I remembered the old one. Here it is​:

Every time I upgrade my system, I run into this problem. But I don't upgrade often enough to remember what the solution is.

The problem is that when running Perl code I see error messages containing the text "loadable library and perl binaries are mismatched".

These messages don't point to a remedy, or even a specific module; instead they give the name of a C source file in a module. Googling is not very helpful.

Looking in my shell history, I see that when I upgrade I should do something like "cpan-outdated --exclude-core -p | cpanm".

However, now this command gives such an error​:

$ cpan-outdated --exclude-core -p
Zlib.c​: loadable library and perl binaries are mismatched (got handshake key 0xde00080, needed 0xce00080)

After using 'locate' to search my filesystem for Zlib.c, I was able to figure out the offending module from the path name, and remove it.

$ cpanm -U Compress​::Raw​::Zlib

This made cpan-outdated work again.

However, I think this is all a bit too complicated for casual users to understand. Where is the "discoverability"...?

Am I an atypical user for having basic stuff like Compress​::Raw​::Zlib installed "locally" (i.e. in my home directory)? (I'm not sure how it got there)

Or maybe everyone who encounters this problem knows what to do with a C file name? Or maybe cpan/cpanm are for experts-only? Or maybe Perl itself is experts-only these days?

I should note that I'm using Arch Linux; is there a better situation on Debian-based distros?

I'm not trying to be facetious, I just have a confusing situation and I don't know why other people are not also finding it as confusing as I do. Maybe there is a simple answer.

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.28.0:

Configured by builduser at Wed Aug  1 10:43:08 CEST 2018.

Summary of my perl5 (revision 5 version 28 subversion 0) configuration:
   
  Platform:
    osname=linux
    osvers=4.17.11-arch1
    archname=x86_64-linux-thread-multi
    uname='linux flo-64s 4.17.11-arch1 #1 smp preempt sun jul 29 10:11:16 utc 2018 x86_64 gnulinux '
    config_args='-des -Dusethreads -Duseshrplib -Doptimize=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -Dprefix=/usr -Dvendorprefix=/usr -Dprivlib=/usr/share/perl5/core_perl -Darchlib=/usr/lib/perl5/5.28/core_perl -Dsitelib=/usr/share/perl5/site_perl -Dsitearch=/usr/lib/perl5/5.28/site_perl -Dvendorlib=/usr/share/perl5/vendor_perl -Dvendorarch=/usr/lib/perl5/5.28/vendor_perl -Dscriptdir=/usr/bin/core_perl -Dsitescript=/usr/bin/site_perl -Dvendorscript=/usr/bin/vendor_perl -Dinc_version_list=none -Dman1ext=1perl -Dman3ext=3perl -Dcccdlflags='-fPIC' -Dlddlflags=-shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -Dldflags=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    optimize='-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt'
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='8.1.1 20180531'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags ='-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/x86_64-pc-linux-gnu/8.1.1/include-fixed /usr/lib /lib/../lib /usr/lib/../lib /lib /lib64 /usr/lib64
    libs=-lpthread -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.27.so
    so=so
    useshrplib=true
    libperl=libperl.so
    gnulibc_version='2.27'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.28/core_perl/CORE'
    cccdlflags='-fPIC'
    lddlflags='-shared -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -L/usr/local/lib -fstack-protector-strong'



@INC for perl 5.28.0:
    /home/frederik/scripts-misc/perl
    /home/frederik/.local/lib/perl5/x86_64-linux-thread-multi
    /home/frederik/.local/lib/perl5
    /usr/lib/perl5/5.28/site_perl
    /usr/share/perl5/site_perl
    /usr/lib/perl5/5.28/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib/perl5/5.28/core_perl
    /usr/share/perl5/core_perl


Environment for perl 5.28.0:
    HOME=/home/frederik
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/home/frederik/.local/arch/x86_64/lib:/home/frederik/.local/lib:/usr/local/lib
    LOGDIR (unset)
    PATH=/home/frederik/.local/bin:/home/frederik/projects/mailproc:/home/frederik/scripts-misc:/home/frederik/.local/arch/x86_64/bin:/usr/bin/core_perl:/usr/bin/vendor_perl:/usr/bin/site_perl:/usr/local/bin:/usr/local/sbin:/usr/bin
    PERL5LIB=/home/frederik/scripts-misc/perl:/home/frederik/.local/lib/perl5:
    PERL_BADLANG (unset)
    PERL_LOCAL_LIB_ROOT=/home/frederik/.local/:/home/frederik/.local/:/home/frederik/.local/:/home/frederik/.local/
    PERL_MB_OPT=--install_base "/home/frederik/.local/"
    PERL_MM_OPT=INSTALL_BASE=/home/frederik/.local/
    SHELL=/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Aug 10, 2018

From @doughera88

On Fri, Aug 10, 2018 at 11​:24​:20AM -0700, (via RT) wrote​:

# New Ticket Created by
# Please include the string​: [perl #133440]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133440 >

This is a bug report for perl from frederik@​ofb.net,
generated with the help of perlbug 1.41 running under perl 5.28.0.

-----------------------------------------------------------------
[Please describe your issue here]

I was able to find that I already reported this bug, sorry​:

https://rt.perl.org/Public/Bug/Display.html?id=130718

The problem is that when running Perl code I see error messages containing the text "loadable library and perl binaries are mismatched".

Am I an atypical user for having basic stuff like Compress​::Raw​::Zlib installed "locally" (i.e. in my home directory)? (I'm not sure how it got there)

I'm not trying to be facetious, I just have a confusing situation and I don't know why other people are not also finding it as confusing as I do. Maybe there is a simple answer.

The immediate problem is that you have version-specific stuff stored
in a non-version-specific directory. That is, looking at your @​INC,
you have "locally" installed modules in

/home/frederik/\.local/lib/perl5/x86\_64\-linux\-thread\-multi

Perl's configuration defaults are to store version-specific stuff in
version-specific directories. Lots more details are in the INSTALL file,
under the section "Coexistence with earlier versions of perl 5".

I'm not sure why you have Compress​::Raw​::Zlib installed locally, since
it has been bundled with perl since version 5.9.4. Of course you are
certainly welcome to install different versions than those bundled with
perl, but if you are going to do so, and if you're going to store them in
a non-version-specific directory, you'll need to recompile them when
you upgrade to a new major version. Upgrade time is also a good time
to re-visit the question of whether you want to continue overriding the
standard version with your own local version.

Hope that helps a little,

--
  Andy Dougherty doughera@​lafayette.edu

@p5pRT
Copy link
Author

p5pRT commented Aug 10, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Aug 11, 2018

From frederik@ofb.net

The immediate problem is that you have version-specific stuff stored
in a non-version-specific directory. That is, looking at your @​INC,
you have "locally" installed modules in

/home/frederik/\.local/lib/perl5/x86\_64\-linux\-thread\-multi

Perl's configuration defaults are to store version-specific stuff in
version-specific directories. Lots more details are in the INSTALL file,
under the section "Coexistence with earlier versions of perl 5".

I'm not sure why you have Compress​::Raw​::Zlib installed locally, since
it has been bundled with perl since version 5.9.4. Of course you are
certainly welcome to install different versions than those bundled with
perl, but if you are going to do so, and if you're going to store them in
a non-version-specific directory, you'll need to recompile them when
you upgrade to a new major version. Upgrade time is also a good time
to re-visit the question of whether you want to continue overriding the
standard version with your own local version.

Hope that helps a little,

Thank you, it does. I already know a bit about the technical reason
for the problem. Perhaps I installed something that depended on
Compress​::Raw​::Zlib over 10 years ago, then that module ended up in a
list of modules that I think I need to run various personal scripts
and this is why I have it installed locally. What's this about INSTALL
and version-specific directories? Did I configure things wrong when I
set up cpanm?

I guess I'm wondering why more people aren't coming to you with
questions about this. And why not change the error message to point to
a manual page which explains what to do. By "discoverability" I meant,
a way for users to understand what they're seeing by reading the
documentation that logically presents itself, for instance, if a
module author says "you can install my module using the cpan tool",
then two years later the user gets this error message, what is he
expected to do next, given lack of omniscience - i.e. only knowing
what he had to know to install cpan, and knowing the text of the error
message.

Reading what you wrote, I downloaded 'perl' and looked at INSTALL, at
all the lines containing 'version.*directory' (I didn't have time to
read all 3000 lines yet), and it didn't really answer my question. But
this is not something I think most users would be expected to do; what
happens is you get Perl from your distro and then you see there is a
module in CPAN but not in the distro, so you install it locally using
cpanm, etc. Then things break.

I would like you to say, "Frederick, here is where you went wrong, you
made a mistake that other users are not making when you did X".
Because that would explain why you haven't been getting other bug
reports about this. Lacking such an explanation, I'm kind of fearful
that the answer is that Perl has just become a tool of an older
generation of power users, many young programmers these days haven't
even heard of it, they use bland corporate languages with fewer
"gotchas"... (I'm only guessing here, because I don't use them
myself...) But maybe that's off-topic.

By the way I think I set up cpanm entirely by adding lines from `perl
-Mlocal​::lib=.local` to my shell profile.

Best wishes,

Frederick

@p5pRT
Copy link
Author

p5pRT commented Aug 12, 2018

From @bulk88

On Fri, 10 Aug 2018 11​:24​:20 -0700, frederik@​ofb.net wrote​:

Maybe I have a bad memory, but it came up again and I wrote a new
report before I remembered the old one. Here it is​:

Every time I upgrade my system, I run into this problem. But I don't
upgrade often enough to remember what the solution is.

The problem is that when running Perl code I see error messages
containing the text "loadable library and perl binaries are
mismatched".

These messages don't point to a remedy, or even a specific module;
instead they give the name of a C source file in a module. Googling is
not very helpful.

https://perldoc.perl.org/perldiag.html#List-form-of-piped-open-not-implemented

%s​: loadable library and perl binaries are mismatched (got handshake key %p, needed %p)
(P) A dynamic loading library .so or .dll was being loaded into the process that was built against a different build of perl than the said library was compiled against. Reinstalling the XS module will likely fix this error.

--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

p5pRT commented Aug 13, 2018

From @Leont

On Sat, Aug 11, 2018 at 6​:23 AM, <frederik@​ofb.net> wrote​:

I guess I'm wondering why more people aren't coming to you with
questions about this. And why not change the error message to point to
a manual page which explains what to do. By "discoverability" I meant,
a way for users to understand what they're seeing by reading the
documentation that logically presents itself, for instance, if a
module author says "you can install my module using the cpan tool",
then two years later the user gets this error message, what is he
expected to do next, given lack of omniscience - i.e. only knowing
what he had to know to install cpan, and knowing the text of the error
message.

Reading what you wrote, I downloaded 'perl' and looked at INSTALL, at
all the lines containing 'version.*directory' (I didn't have time to
read all 3000 lines yet), and it didn't really answer my question. But
this is not something I think most users would be expected to do; what
happens is you get Perl from your distro and then you see there is a
module in CPAN but not in the distro, so you install it locally using
cpanm, etc. Then things break.

I would like you to say, "Frederick, here is where you went wrong, you
made a mistake that other users are not making when you did X".
Because that would explain why you haven't been getting other bug
reports about this. Lacking such an explanation, I'm kind of fearful
that the answer is that Perl has just become a tool of an older
generation of power users, many young programmers these days haven't
even heard of it, they use bland corporate languages with fewer
"gotchas"... (I'm only guessing here, because I don't use them
myself...) But maybe that's off-topic.

By the way I think I set up cpanm entirely by adding lines from `perl
-Mlocal​::lib=.local` to my shell profile.

I think most people avoid configurations where incompatible perl
builds share the same arch directories. Judging by your @​INC my first
guess is that you're reusing your old perl's local​::lib directories,
which would indeed blow up in your face.

Leon

@p5pRT
Copy link
Author

p5pRT commented Aug 14, 2018

From @iabyn

On Mon, Aug 13, 2018 at 11​:56​:26PM +0200, Leon Timmermans wrote​:

I think most people avoid configurations where incompatible perl
builds share the same arch directories. Judging by your @​INC my first
guess is that you're reusing your old perl's local​::lib directories,
which would indeed blow up in your face.

From a quick perusal of local​::lib's documentation, it appears that
it doesn't use version-specific paths under the local directory, which
on the face of it seems like a design flaw.

--
If life gives you lemons, you'll probably develop a citric acid allergy.

@p5pRT
Copy link
Author

p5pRT commented Aug 14, 2018

From @eserte

Dana Tue, 14 Aug 2018 01​:42​:09 -0700, davem reče​:

On Mon, Aug 13, 2018 at 11​:56​:26PM +0200, Leon Timmermans wrote​:

I think most people avoid configurations where incompatible perl
builds share the same arch directories. Judging by your @​INC my first
guess is that you're reusing your old perl's local​::lib directories,
which would indeed blow up in your face.

From a quick perusal of local​::lib's documentation, it appears that
it doesn't use version-specific paths under the local directory, which
on the face of it seems like a design flaw.

Interestingly, local​::lib creates version-specific paths on first-time initialization. But I think the problem is that by using ExtUtils​::MakeMaker's INSTALL_BASE no installation to version-specific paths happens, so the design flaw is probably there. (Did not check Module​::Build's --install_base)

@p5pRT
Copy link
Author

p5pRT commented Aug 14, 2018

From @Leont

On Tue, Aug 14, 2018 at 10​:41 AM, Dave Mitchell <davem@​iabyn.com> wrote​:

On Mon, Aug 13, 2018 at 11​:56​:26PM +0200, Leon Timmermans wrote​:

I think most people avoid configurations where incompatible perl
builds share the same arch directories. Judging by your @​INC my first
guess is that you're reusing your old perl's local​::lib directories,
which would indeed blow up in your face.

From a quick perusal of local​::lib's documentation, it appears that
it doesn't use version-specific paths under the local directory, which
on the face of it seems like a design flaw.

Perl doesn't quite provide enough rope to tie that knot. Or at least
it doesn't for the common scenario where one has multiple
(incompatible) perls and at login time one doesn't know yet which to
use when. The only primitive we provide (PERL5LIB) does "prepend these
dirs to @​INC". I can sort of imagine a tool that does what you
suggest, but the additional complexity and fragility sound
uncomfortable.

It's a real problem, but I don't think it can be solved on the
local​::lib side alone.

Leon

@p5pRT
Copy link
Author

p5pRT commented Aug 15, 2018

From frederik@ofb.net

On Wed, Aug 15, 2018 at 01​:19​:16AM +0200, Leon Timmermans wrote​:

On Tue, Aug 14, 2018 at 10​:41 AM, Dave Mitchell <davem@​iabyn.com> wrote​:

On Mon, Aug 13, 2018 at 11​:56​:26PM +0200, Leon Timmermans wrote​:

I think most people avoid configurations where incompatible perl
builds share the same arch directories. Judging by your @​INC my first
guess is that you're reusing your old perl's local​::lib directories,
which would indeed blow up in your face.

From a quick perusal of local​::lib's documentation, it appears that
it doesn't use version-specific paths under the local directory, which
on the face of it seems like a design flaw.

Perl doesn't quite provide enough rope to tie that knot. Or at least
it doesn't for the common scenario where one has multiple
(incompatible) perls and at login time one doesn't know yet which to
use when. The only primitive we provide (PERL5LIB) does "prepend these
dirs to @​INC". I can sort of imagine a tool that does what you
suggest, but the additional complexity and fragility sound
uncomfortable.

It's a real problem, but I don't think it can be solved on the
local​::lib side alone.

Thanks for the discussion. I don't have two different versions of Perl
installed, just one version from my distribution Arch, which gets
updated occasionally.

I'm not sure where this is headed but I just wanted us to keep in mind
what it looks like for a new user who is doing the standard (?) thing
of updating Perl via his distribution packaging system, while having
CPAN modules installed locally (e.g. in his home directory, is this
not typical?).

There seems to be no discussion of changing the mismatched binary
error message to point to documentation about e.g. updating CPAN
modules.

If the default setup were fixed so that version-specific paths were
used for local modules, then what would our user experience consist of
- user installs CPAN modules to make his scripts work, then a year
later he upgrades his distribution to a new version of Perl and his
scripts stop working with "Can't locate XXX.pm in @​INC"? I guess that
is better than the error message I saw, because then the user at least
has a module name to start with, but I think it would be preferable to
have an error that somehow ultimately leads (e.g. via a reference to a
man page) to instructions on upgrading locally-installed CPAN modules.

To make this concrete, what if you could change the error​:

  Zlib.c​: loadable library and perl binaries are mismatched (got handshake key
0xde00080, needed 0xce00080)

to say​:

  In Perl module Compress​::Raw​::Zlib (Zlib.c)​: loadable library and perl binaries are mismatched (got handshake key 0xde00080, needed 0xce00080). Do you need to recompile this module for a new Perl version? See `man perlmodupgrade`.

(Actually, I'm noticing that the "perlmodinstall" manual page doesn't
mention cpanm or cpan tools, I wonder if those should be there too)

Hope these suggestions aren't too annoying,

Frederick

@p5pRT
Copy link
Author

p5pRT commented Aug 15, 2018

From @Grinnz

On Tue, Aug 14, 2018 at 8​:16 PM <frederik@​ofb.net> wrote​:

Thanks for the discussion. I don't have two different versions of Perl
installed, just one version from my distribution Arch, which gets
updated occasionally.

I'm not sure where this is headed but I just wanted us to keep in mind
what it looks like for a new user who is doing the standard (?) thing
of updating Perl via his distribution packaging system, while having
CPAN modules installed locally (e.g. in his home directory, is this
not typical?).

There seems to be no discussion of changing the mismatched binary
error message to point to documentation about e.g. updating CPAN
modules.

If the default setup were fixed so that version-specific paths were
used for local modules, then what would our user experience consist of
- user installs CPAN modules to make his scripts work, then a year
later he upgrades his distribution to a new version of Perl and his
scripts stop working with "Can't locate XXX.pm in @​INC"? I guess that
is better than the error message I saw, because then the user at least
has a module name to start with, but I think it would be preferable to
have an error that somehow ultimately leads (e.g. via a reference to a
man page) to instructions on upgrading locally-installed CPAN modules.

To make this concrete, what if you could change the error​:

Zlib\.c&#8203;: loadable library and perl binaries are mismatched \(got

handshake key
0xde00080, needed 0xce00080)

to say​:

In Perl module Compress&#8203;::Raw&#8203;::Zlib \(Zlib\.c\)&#8203;: loadable library and perl

binaries are mismatched (got handshake key 0xde00080, needed 0xce00080). Do
you need to recompile this module for a new Perl version? See `man
perlmodupgrade`.

(Actually, I'm noticing that the "perlmodinstall" manual page doesn't
mention cpanm or cpan tools, I wonder if those should be there too)

Hope these suggestions aren't too annoying,

Frederick

These are good ideas. The fundamental issue though is that modules you
installed for one major version of Perl must always be installed again for
another, and users don't tend to expect this whether they use local​::lib or
not, it will always be a problem if you install modules without using your
package manager in a Perl managed by your package manager.

-Dan

@p5pRT
Copy link
Author

p5pRT commented Aug 15, 2018

From @eserte

Dana Tue, 14 Aug 2018 16​:19​:54 -0700, LeonT reče​:

On Tue, Aug 14, 2018 at 10​:41 AM, Dave Mitchell <davem@​iabyn.com> wrote​:

On Mon, Aug 13, 2018 at 11​:56​:26PM +0200, Leon Timmermans wrote​:

I think most people avoid configurations where incompatible perl
builds share the same arch directories. Judging by your @​INC my first
guess is that you're reusing your old perl's local​::lib directories,
which would indeed blow up in your face.

From a quick perusal of local​::lib's documentation, it appears that
it doesn't use version-specific paths under the local directory, which
on the face of it seems like a design flaw.

Perl doesn't quite provide enough rope to tie that knot. Or at least
it doesn't for the common scenario where one has multiple
(incompatible) perls and at login time one doesn't know yet which to
use when. The only primitive we provide (PERL5LIB) does "prepend these
dirs to @​INC". I can sort of imagine a tool that does what you
suggest, but the additional complexity and fragility sound
uncomfortable.

The PERL5LIB mechanism is sufficient here. If set user sets something like

  PERL5LIB=/root/perl5/lib/perl5

then @​INC contains automatically

  /root/perl5/lib/perl5/5.28.0/x86_64-linux-thread-multi
  /root/perl5/lib/perl5/5.28.0
  /root/perl5/lib/perl5/x86_64-linux-thread-multi
  /root/perl5/lib/perl5

so architecture and perl api (version) specific files have their place.

The problem is, as I said, that INSTALL_BASE does not use these version-specific files.

@p5pRT
Copy link
Author

p5pRT commented Aug 15, 2018

From frederik@ofb.net

On Tue, Aug 14, 2018 at 05​:28​:03PM -0700, Dan Book via RT wrote​:

On Tue, Aug 14, 2018 at 8​:16 PM <frederik@​ofb.net> wrote​:

Thanks for the discussion. I don't have two different versions of Perl
installed, just one version from my distribution Arch, which gets
updated occasionally.

I'm not sure where this is headed but I just wanted us to keep in mind
what it looks like for a new user who is doing the standard (?) thing
of updating Perl via his distribution packaging system, while having
CPAN modules installed locally (e.g. in his home directory, is this
not typical?).

There seems to be no discussion of changing the mismatched binary
error message to point to documentation about e.g. updating CPAN
modules.

If the default setup were fixed so that version-specific paths were
used for local modules, then what would our user experience consist of
- user installs CPAN modules to make his scripts work, then a year
later he upgrades his distribution to a new version of Perl and his
scripts stop working with "Can't locate XXX.pm in @​INC"? I guess that
is better than the error message I saw, because then the user at least
has a module name to start with, but I think it would be preferable to
have an error that somehow ultimately leads (e.g. via a reference to a
man page) to instructions on upgrading locally-installed CPAN modules.

To make this concrete, what if you could change the error​:

Zlib\.c&#8203;: loadable library and perl binaries are mismatched \(got

handshake key
0xde00080, needed 0xce00080)

to say​:

In Perl module Compress&#8203;::Raw&#8203;::Zlib \(Zlib\.c\)&#8203;: loadable library and perl

binaries are mismatched (got handshake key 0xde00080, needed 0xce00080). Do
you need to recompile this module for a new Perl version? See `man
perlmodupgrade`.

(Actually, I'm noticing that the "perlmodinstall" manual page doesn't
mention cpanm or cpan tools, I wonder if those should be there too)

Hope these suggestions aren't too annoying,

Frederick

These are good ideas. The fundamental issue though is that modules you
installed for one major version of Perl must always be installed again for
another, and users don't tend to expect this whether they use local​::lib or
not, it will always be a problem if you install modules without using your
package manager in a Perl managed by your package manager.

Thanks. So people who use cpanm tend to have Perl compiled locally?
I'm still trying to figure out the common use case.

I didn't realize that you can do "cpanm Perl" but it looks like it fails​:

  Perl.xs​: At top level​:
  Perl.xs​:33​:27​: error​: ‘PL_tokenbuf’ undeclared here (not in a function); did you mean ‘PL_top_env’?
  char ptokenbuf[sizeof(PL_tokenbuf)];
  ^~~~~~~~~~~
  PL_top_env
 
I wonder how much space this ends up taking up were it to succeed. I
think the idea with having modules installed in my home directory is
that other scripts in my home directory can then depend on them, and I
can sync the scripts between different hosts along with my shell
config and so on.

But that would argue for version-specific paths (because the different
hosts might have different Perl versions).

Thank you,

Frederick

@p5pRT
Copy link
Author

p5pRT commented Aug 15, 2018

From @andk

On Tue, 14 Aug 2018 22​:58​:52 -0700, frederik@​ofb.net said​:

  > But that would argue for version-specific paths (because the different
  > hosts might have different Perl versions).

You can sync your .local directory to other machines only if the two
machines are perfectly in sync with regard to all involved libraries.

As I read the whole thread here, your problem is recompilation after
some library upgrade. The shell that comes with CPAN.pm has a commend
for that, it is called `recompile`. Maye you try that out. Disclaimer​: I
wrote it. Let me know if you have questions about it.

--
andreas

@p5pRT
Copy link
Author

p5pRT commented Aug 15, 2018

From frederik@ofb.net

On Tue, Aug 14, 2018 at 11​:47​:54PM -0700, (Andreas J. Koenig) via RT wrote​:

On Tue, 14 Aug 2018 22​:58​:52 -0700, frederik@​ofb.net said​:

But that would argue for version-specific paths (because the different
hosts might have different Perl versions).

You can sync your .local directory to other machines only if the two
machines are perfectly in sync with regard to all involved libraries.

As I read the whole thread here, your problem is recompilation after
some library upgrade. The shell that comes with CPAN.pm has a commend
for that, it is called `recompile`. Maye you try that out. Disclaimer​: I
wrote it. Let me know if you have questions about it.

That's good advice, although I would say that the main problem for me
is users being faced with an error message that doesn't point to a
clear solution. But maybe that's what Stack Exchange or IRC is for?
And maybe I'm the only one that's confused? Nevertheless one would
hope that the local documentation would be sufficient for something
this basic.

FWIW I tried your 'recompile' and I got a bunch of errors. I'm
attaching the output. I'm rerunning it after "cpanm -U Digest​::SHA"
and it seems to be doing something productive.

The approach I tried earlier, which I think I mentioned, was to run
cpan-outdated, but I guess this is something different - its man page
doesn't say anything about helping you recompile binary modules which
may not have actually changed versions. Your suggestion seems more
promising.

Thank you,

Frederick

@p5pRT
Copy link
Author

p5pRT commented Aug 15, 2018

From frederik@ofb.net

cpan.out

@p5pRT
Copy link
Author

p5pRT commented Aug 15, 2018

From @iabyn

On Tue, Aug 14, 2018 at 05​:14​:41PM -0700, frederik@​ofb.net wrote​:

Thanks for the discussion. I don't have two different versions of Perl
installed, just one version from my distribution Arch, which gets
updated occasionally.

I'm not sure where this is headed but I just wanted us to keep in mind
what it looks like for a new user who is doing the standard (?) thing
of updating Perl via his distribution packaging system, while having
CPAN modules installed locally (e.g. in his home directory, is this
not typical?).

The more typical approach is to have a personal perl installation separate
from the system's one​: this would contain both the perl binary plus any
additional modules you have installed. That way you have complete control
over what gets upgraded and when. Using the system perl in combination with
privately built/installed modules puts you at the mercy of whatever your
OS's update/upgrade procedures do to the system perl.

If the default setup were fixed so that version-specific paths were
used for local modules,

Note that the default behaviour for perl *is* always to install in
version-specific directories. The behaviour you are seeing is that of a
third-party module, local​::lib, which is not bundled with perl, and which
we (p5p) have no control over. (Although elsewhere in this thread it
appears that MakeMaker etc may be to blame too.)

then what would our user experience consist of
- user installs CPAN modules to make his scripts work, then a year
later he upgrades his distribution to a new version of Perl and his
scripts stop working with "Can't locate XXX.pm in @​INC"? I guess that
is better than the error message I saw, because then the user at least
has a module name to start with, but I think it would be preferable to
have an error that somehow ultimately leads (e.g. via a reference to a
man page) to instructions on upgrading locally-installed CPAN modules.

To make this concrete, what if you could change the error​:

Zlib\.c&#8203;: loadable library and perl binaries are mismatched \(got handshake key

0xde00080, needed 0xce00080)

to say​:

In Perl module Compress&#8203;::Raw&#8203;::Zlib \(Zlib\.c\)&#8203;: loadable library and perl binaries are mismatched \(got handshake key 0xde00080\, needed 0xce00080\)\. Do you need to recompile this module for a new Perl version? See \`man perlmodupgrade\`\.

(Actually, I'm noticing that the "perlmodinstall" manual page doesn't
mention cpanm or cpan tools, I wonder if those should be there too)

Some observations.

As I pointed out earlier, I'm not sure that it will be possible (or at
least easy) to display the module name as part of the error message.

I'm very much in favour of removing the 'handshake' part of the message,
and replacing it with something informative and useful - i.e. to actually
decode the hex values and explain what was mismatched.

I'm also not opposed to the error message referring to a man page;
although note that if your perl installation is borked due to version
mismatches, then it's possible that the tool you use for viewing man pages
(such as perldoc) also wont work!

It may be overkill to have a complete new man page - perhaps just an extra
section in an existing document instead?

It will be quite difficult to write such a perlmodupgrade man page. Bear
in mind that the error message you got refers to any sort of mismatch
between the perl binary you are executing, and an XS module which that perl
is trying to load. Your particular scenario is just one of many, and
reinstalling the offending module isn't always the correct solution. Such
a manual page would have to explain many possible scenarios, including
interactions with third-party solutions such as OS packages, perlbrew,
local​::lib etc.

Personally I don't feel I have the expertise to write all of such a
document.

--
Any [programming] language that doesn't occasionally surprise the
novice will pay for it by continually surprising the expert.
  -- Larry Wall

@p5pRT
Copy link
Author

p5pRT commented Aug 15, 2018

From @Grinnz

On Wed, Aug 15, 2018 at 6​:36 AM Dave Mitchell <davem@​iabyn.com> wrote​:

The more typical approach is to have a personal perl installation separate
from the system's one​: this would contain both the perl binary plus any
additional modules you have installed. That way you have complete control
over what gets upgraded and when. Using the system perl in combination with
privately built/installed modules puts you at the mercy of whatever your
OS's update/upgrade procedures do to the system perl.

To add to this​: typical methods of doing this are Perl​::Build, perlbrew,
and plenv. The first simply installs a Perl somewhere, the other two are
Perl-based and shell-based multiple-Perl managers.

-Dan

@p5pRT
Copy link
Author

p5pRT commented Aug 17, 2018

From frederik@ofb.net

Just wanted to briefly follow this up...

Andreas​: "cpan[1]> recompile" worked fine on both my machines. I also
noticed that the second time I run it, it notices nothing needs to be
done, which is great.

Dave​: Thanks for the observations. I'm happy to look at the code and
send a patch for a better error message when I have time, which is
probably not going to be within the next month or so. Also I think it
should be worthwhile to investigate a way to get the full module name
in the message, presumably whatever compiles modules (MakeMaker?) can
be changed to provide a CPP macro with the module name, and I think
this could be done in a backwards-compatible manner - whatever that
means.

Dan​: I can look at perlbrew et al (it was also recommended off-list),
I'd of course like to see how far I can go with the binaries provided
by my distro.

One problem with my argument that the error message isn't
user-friendly, is that anyone can easily submit a bug for it and get
friendly and helpful replies from the developers, as I have done.
Still, I'm interested in improving things and I hope that I myself or
someone else can find time to work on this in the not-too-distant
future.

Thanks,

Frederick

On Tue, Aug 14, 2018 at 11​:47​:54PM -0700, (Andreas J. Koenig) via RT wrote​:

On Tue, 14 Aug 2018 22​:58​:52 -0700, frederik@​ofb.net said​:

But that would argue for version-specific paths (because the different
hosts might have different Perl versions).

You can sync your .local directory to other machines only if the two
machines are perfectly in sync with regard to all involved libraries.

As I read the whole thread here, your problem is recompilation after
some library upgrade. The shell that comes with CPAN.pm has a commend
for that, it is called `recompile`. Maye you try that out. Disclaimer​: I
wrote it. Let me know if you have questions about it.

--
andreas

On Wed, Aug 15, 2018 at 03​:35​:57AM -0700, Dave Mitchell via RT wrote​:

On Tue, Aug 14, 2018 at 05​:14​:41PM -0700, frederik@​ofb.net wrote​:

Thanks for the discussion. I don't have two different versions of Perl
installed, just one version from my distribution Arch, which gets
updated occasionally.

I'm not sure where this is headed but I just wanted us to keep in mind
what it looks like for a new user who is doing the standard (?) thing
of updating Perl via his distribution packaging system, while having
CPAN modules installed locally (e.g. in his home directory, is this
not typical?).

The more typical approach is to have a personal perl installation separate
from the system's one​: this would contain both the perl binary plus any
additional modules you have installed. That way you have complete control
over what gets upgraded and when. Using the system perl in combination with
privately built/installed modules puts you at the mercy of whatever your
OS's update/upgrade procedures do to the system perl.

If the default setup were fixed so that version-specific paths were
used for local modules,

Note that the default behaviour for perl *is* always to install in
version-specific directories. The behaviour you are seeing is that of a
third-party module, local​::lib, which is not bundled with perl, and which
we (p5p) have no control over. (Although elsewhere in this thread it
appears that MakeMaker etc may be to blame too.)

then what would our user experience consist of
- user installs CPAN modules to make his scripts work, then a year
later he upgrades his distribution to a new version of Perl and his
scripts stop working with "Can't locate XXX.pm in @​INC"? I guess that
is better than the error message I saw, because then the user at least
has a module name to start with, but I think it would be preferable to
have an error that somehow ultimately leads (e.g. via a reference to a
man page) to instructions on upgrading locally-installed CPAN modules.

To make this concrete, what if you could change the error​:

Zlib\.c&#8203;: loadable library and perl binaries are mismatched \(got handshake key

0xde00080, needed 0xce00080)

to say​:

In Perl module Compress&#8203;::Raw&#8203;::Zlib \(Zlib\.c\)&#8203;: loadable library and perl binaries are mismatched \(got handshake key 0xde00080\, needed 0xce00080\)\. Do you need to recompile this module for a new Perl version? See \`man perlmodupgrade\`\.

(Actually, I'm noticing that the "perlmodinstall" manual page doesn't
mention cpanm or cpan tools, I wonder if those should be there too)

Some observations.

As I pointed out earlier, I'm not sure that it will be possible (or at
least easy) to display the module name as part of the error message.

I'm very much in favour of removing the 'handshake' part of the message,
and replacing it with something informative and useful - i.e. to actually
decode the hex values and explain what was mismatched.

I'm also not opposed to the error message referring to a man page;
although note that if your perl installation is borked due to version
mismatches, then it's possible that the tool you use for viewing man pages
(such as perldoc) also wont work!

It may be overkill to have a complete new man page - perhaps just an extra
section in an existing document instead?

It will be quite difficult to write such a perlmodupgrade man page. Bear
in mind that the error message you got refers to any sort of mismatch
between the perl binary you are executing, and an XS module which that perl
is trying to load. Your particular scenario is just one of many, and
reinstalling the offending module isn't always the correct solution. Such
a manual page would have to explain many possible scenarios, including
interactions with third-party solutions such as OS packages, perlbrew,
local​::lib etc.

Personally I don't feel I have the expertise to write all of such a
document.

--
Any [programming] language that doesn't occasionally surprise the
novice will pay for it by continually surprising the expert.
-- Larry Wall

@p5pRT
Copy link
Author

p5pRT commented Aug 17, 2018

From @andk

On Thu, 16 Aug 2018 21​:51​:39 -0700, frederik@​ofb.net said​:

  > Just wanted to briefly follow this up...
  > Andreas​: "cpan[1]> recompile" worked fine on both my machines. I also
  > noticed that the second time I run it, it notices nothing needs to be
  > done, which is great.

Thanks for letting me know and for sending the report how noisy it was
during its performance. I'm sorry for this bug, I've since fixed it in
the repository so it will appear in version 2.21.

Besides that I have also found the original ticket in which we received
the first guide that explained the "got handshake key" message. It was
https://rt.perl.org/Public/Bug/Display.html?id=125236. Maybe the two
tickets would be suited to be folded into one, I'm not sure.

--
andreas

@p5pRT
Copy link
Author

p5pRT commented Aug 19, 2018

From @karenetheridge

On Wed, Aug 15, 2018 at 3​:35 AM, Dave Mitchell <davem@​iabyn.com> wrote​:

Note that the default behaviour for perl *is* always to install in
version-specific directories. The behaviour you are seeing is that of a
third-party module, local​::lib, which is not bundled with perl, and which
we (p5p) have no control over. (Although elsewhere in this thread it
appears that MakeMaker etc may be to blame too.)

But the toolchain gang does, and there's a large overlap between that group
and p5p. We should not go "oh well, not in core, not our problem" and leave
it to the original bug reporter to have to pursue the issue in another bug
queue (and attempt to explain the problem again, which leads to a game of
telephone). We can do better than that.

All local​::lib does is set some environment variables, e.g.​:

$ cd ~; perl -Mlocal​::lib=local
Attempting to create directory /Users/ether/local
PATH="/Users/ether/local/bin${PATH​:+​:${PATH}}"; export PATH;
PERL5LIB="/Users/ether/local/lib/perl5${PERL5LIB​:+​:${PERL5LIB}}"; export
PERL5LIB;
PERL_LOCAL_LIB_ROOT="/Users/ether/local${PERL_LOCAL_LIB_ROOT​:+​:${PERL_LOCAL_LIB_ROOT}}";
export PERL_LOCAL_LIB_ROOT;
PERL_MB_OPT="--install_base \"/Users/ether/local\""; export PERL_MB_OPT;
PERL_MM_OPT="INSTALL_BASE=/Users/ether/local"; export PERL_MM_OPT;
Perhaps those environment variables should always contain the perl
version. There would be some backcompat issues, but we could possibly
handle that by putting the old non-versioned directory into PERL5LIB after
the new versioned one (or something clever with symlinks, or.. well, we'd
have to discuss it.)

-ether@​cpan.org

@p5pRT
Copy link
Author

p5pRT commented Aug 19, 2018

From frederik@ofb.net

Thanks for your reply. While trying to downgrade my system (debugging
another issue) I discovered a possible reason why I have problematic
binary modules in my home directory which are also in Perl core
(causing cpan to break when I upgrade Perl). The reason appears to be
that the "recompile" cpan command reinstalls modules which I had
removed earlier using 'cpanm'.

$ PERL5LIB= cpanm -U Digest​::SHA
...
$ PERL5LIB= cpanm -U Time​::HiRes
...
$ perldoc -l Time​::HiRes
/usr/lib/perl5/5.28/core_perl/Time/HiRes.pm
$ perldoc -l Digest​::SHA
/usr/lib/perl5/5.28/core_perl/Digest/SHA.pm

(after cpan recompile)

$ perldoc -l Time​::HiRes
/home/frederik/.local/lib/perl5/x86_64-linux-thread-multi/Time/HiRes.pm
$ perldoc -l Digest​::SHA
/home/frederik/.local/lib/perl5/x86_64-linux-thread-multi/Digest/SHA.pm

I don't know if this is a bug or just an annoyance, but it may go a
little way towards solving the mystery of why I have these modules
around, since I think I recall removing them earlier. Obviously, I
don't look too carefully at the ouput of 'cpan -r', because I have
better things to do, I just assume it's doing the "right thing". By
the time I realize that I have duplicate modules installed, it's
probably a year later and I don't remember having removed them.

Also, I notice that if I ask 'cpanm' to install a module which is
already in core, then it doesn't warn me​:

$ perldoc -l DB_File
/usr/lib/perl5/5.28/core_perl/DB_File.pm
$ cpanm DB_File
--> Working on DB_File
Fetching http​://www.cpan.org/authors/id/P/PM/PMQS/DB_File-1.842.tar.gz ... OK
Configuring DB_File-1.842 ... ^C

I haven't checked what happens if I use cpanm to install a module
which depends on modules in core, but if you give me an example of a
CPAN module with dependencies in core then I can try this out.

I couldn't figure out how to use the 'cpan' tool to remove modules,
but 'cpanm' has a nice simple interface for doing that so I thought it
would be compatible with 'cpan'.

Also, I see 'cpan -r -T' runs lots of tests, I thought '-T' was to
supposed to skip the tests.

Hope this helps,

Frederick

On Fri, Aug 17, 2018 at 01​:40​:52PM -0700, (Andreas J. Koenig) via RT wrote​:

On Thu, 16 Aug 2018 21​:51​:39 -0700, frederik@​ofb.net said​:

Just wanted to briefly follow this up...
Andreas​: "cpan[1]> recompile" worked fine on both my machines. I also
noticed that the second time I run it, it notices nothing needs to be
done, which is great.

Thanks for letting me know and for sending the report how noisy it was
during its performance. I'm sorry for this bug, I've since fixed it in
the repository so it will appear in version 2.21.

Besides that I have also found the original ticket in which we received
the first guide that explained the "got handshake key" message. It was
https://rt.perl.org/Public/Bug/Display.html?id=125236. Maybe the two
tickets would be suited to be folded into one, I'm not sure.

@p5pRT
Copy link
Author

p5pRT commented Aug 19, 2018

From @Grinnz

On Sat, Aug 18, 2018 at 11​:26 PM <frederik@​ofb.net> wrote​:

Thanks for your reply. While trying to downgrade my system (debugging
another issue) I discovered a possible reason why I have problematic
binary modules in my home directory which are also in Perl core
(causing cpan to break when I upgrade Perl). The reason appears to be
that the "recompile" cpan command reinstalls modules which I had
removed earlier using 'cpanm'.

$ PERL5LIB= cpanm -U Digest​::SHA
...
$ PERL5LIB= cpanm -U Time​::HiRes
...
$ perldoc -l Time​::HiRes
/usr/lib/perl5/5.28/core_perl/Time/HiRes.pm
$ perldoc -l Digest​::SHA
/usr/lib/perl5/5.28/core_perl/Digest/SHA.pm

(after cpan recompile)

$ perldoc -l Time​::HiRes
/home/frederik/.local/lib/perl5/x86_64-linux-thread-multi/Time/HiRes.pm
$ perldoc -l Digest​::SHA
/home/frederik/.local/lib/perl5/x86_64-linux-thread-multi/Digest/SHA.pm

I don't know if this is a bug or just an annoyance, but it may go a
little way towards solving the mystery of why I have these modules
around, since I think I recall removing them earlier. Obviously, I
don't look too carefully at the ouput of 'cpan -r', because I have
better things to do, I just assume it's doing the "right thing". By
the time I realize that I have duplicate modules installed, it's
probably a year later and I don't remember having removed them.

Also, I notice that if I ask 'cpanm' to install a module which is
already in core, then it doesn't warn me​:

$ perldoc -l DB_File
/usr/lib/perl5/5.28/core_perl/DB_File.pm
$ cpanm DB_File
--> Working on DB_File
Fetching http​://www.cpan.org/authors/id/P/PM/PMQS/DB_File-1.842.tar.gz
... OK
Configuring DB_File-1.842 ... ^C

I haven't checked what happens if I use cpanm to install a module
which depends on modules in core, but if you give me an example of a
CPAN module with dependencies in core then I can try this out.

I couldn't figure out how to use the 'cpan' tool to remove modules,
but 'cpanm' has a nice simple interface for doing that so I thought it
would be compatible with 'cpan'.

These are dual-life modules; it is expected and often desired to install
them a second time in site or vendor libs to upgrade them from the core
module version. Since Perl 5.12, site and vendor libs take precedence
in @​INC for that reason. local​::lib of course takes precedence over all
standard lib directories. cpanm's uninstall command relies on packlists,
and thus can only uninstall modules installed by a cpan client (whether
cpanm or cpan).

-Dan

@p5pRT
Copy link
Author

p5pRT commented Aug 21, 2018

From frederik@ofb.net

These are dual-life modules; it is expected and often desired to install
them a second time in site or vendor libs to upgrade them from the core
module version. Since Perl 5.12, site and vendor libs take precedence
in @​INC for that reason. local​::lib of course takes precedence over all
standard lib directories. cpanm's uninstall command relies on packlists,
and thus can only uninstall modules installed by a cpan client (whether
cpanm or cpan).

You're saying that these modules are purposefully installed by 'cpan'
even when compatible versions already exist in 'core'?

I guess not everyone expects this, because when I first submitted this
bug report I got a response from Andy Dougherty saying "I'm not sure
why you have Compress​::Raw​::Zlib installed locally, since it has been
bundled with perl since version 5.9.4." And I have in my shell history
that I uninstalled it, while now it is back in ~/.local. Although when
I removed it again and re-ran 'cpan -r', it didn't reappear, so I'm
not entirely sure what's happening.

Thanks,

Frederick

@p5pRT
Copy link
Author

p5pRT commented Aug 22, 2018

From @Grinnz

On Tue, Aug 21, 2018 at 7​:45 PM <frederik@​ofb.net> wrote​:

You're saying that these modules are purposefully installed by 'cpan'
even when compatible versions already exist in 'core'?

I guess not everyone expects this, because when I first submitted this
bug report I got a response from Andy Dougherty saying "I'm not sure
why you have Compress​::Raw​::Zlib installed locally, since it has been
bundled with perl since version 5.9.4." And I have in my shell history
that I uninstalled it, while now it is back in ~/.local. Although when
I removed it again and re-ran 'cpan -r', it didn't reappear, so I'm
not entirely sure what's happening.

That is correct. His response seemed to account for this possibility but
was asking why you had installed a newer version in a local lib without the
use of version specific directories, which could be considered a deficiency
in the toolchain. Regardless, upgrades to dual-life modules are installed
in sitelib or local libs very commonly, this is the reason they are
available from CPAN. I can't comment on how CPAN.pm's recompile function
interacts with dual-life modules or local libs.

-Dan

@p5pRT
Copy link
Author

p5pRT commented Aug 22, 2018

From frederik@ofb.net

By the way I never took the time to thank you for pointing these
things out and trying to up the responsibility level a bit here.

Thanks!

Frederick

On Sat, Aug 18, 2018 at 07​:55​:29PM -0700, Karen Etheridge via RT wrote​:

On Wed, Aug 15, 2018 at 3​:35 AM, Dave Mitchell <davem@​iabyn.com> wrote​:

Note that the default behaviour for perl *is* always to install in
version-specific directories. The behaviour you are seeing is that of a
third-party module, local​::lib, which is not bundled with perl, and which
we (p5p) have no control over. (Although elsewhere in this thread it
appears that MakeMaker etc may be to blame too.)

But the toolchain gang does, and there's a large overlap between that group
and p5p. We should not go "oh well, not in core, not our problem" and leave
it to the original bug reporter to have to pursue the issue in another bug
queue (and attempt to explain the problem again, which leads to a game of
telephone). We can do better than that.

All local​::lib does is set some environment variables, e.g.​:

$ cd ~; perl -Mlocal​::lib=local
Attempting to create directory /Users/ether/local
PATH="/Users/ether/local/bin${PATH​:+​:${PATH}}"; export PATH;
PERL5LIB="/Users/ether/local/lib/perl5${PERL5LIB​:+​:${PERL5LIB}}"; export
PERL5LIB;
PERL_LOCAL_LIB_ROOT="/Users/ether/local${PERL_LOCAL_LIB_ROOT​:+​:${PERL_LOCAL_LIB_ROOT}}";
export PERL_LOCAL_LIB_ROOT;
PERL_MB_OPT="--install_base \"/Users/ether/local\""; export PERL_MB_OPT;
PERL_MM_OPT="INSTALL_BASE=/Users/ether/local"; export PERL_MM_OPT;
Perhaps those environment variables should always contain the perl
version. There would be some backcompat issues, but we could possibly
handle that by putting the old non-versioned directory into PERL5LIB after
the new versioned one (or something clever with symlinks, or.. well, we'd
have to discuss it.)

-ether@​cpan.org

@p5pRT
Copy link
Author

p5pRT commented Aug 22, 2018

From @iabyn

On Sat, Aug 18, 2018 at 07​:55​:14PM -0700, Karen Etheridge wrote​:

On Wed, Aug 15, 2018 at 3​:35 AM, Dave Mitchell <davem@​iabyn.com> wrote​:

Note that the default behaviour for perl *is* always to install in
version-specific directories. The behaviour you are seeing is that of a
third-party module, local​::lib, which is not bundled with perl, and which
we (p5p) have no control over. (Although elsewhere in this thread it
appears that MakeMaker etc may be to blame too.)

But the toolchain gang does, and there's a large overlap between that group
and p5p. We should not go "oh well, not in core, not our problem" and leave
it to the original bug reporter to have to pursue the issue in another bug
queue (and attempt to explain the problem again, which leads to a game of
telephone). We can do better than that.

You quoted one paragraph of mine from a long email, where I had clearly
spent some some and thought on the issue, and was attempting to narrow the
issues, and trying to find out which bits need fixing and by who etc. At
no point did I say or imply "not a perl issue, go bug the owner of
local​::lib instead; I'm closing this ticket".

What I said in that paragraph is also completely true - whatever
discussions we might have here, and what grand solutions we might propose
to fix local​::lib, ultimately the owner of the cpan module is free to have
their own ideas as to what what their module should do and be irritated
that p5p is drying to dictate solutions to them.

So I'm mildly peeved by your response, and the OP's follow up of "oh at
last someone's taking some responsibility" - after about 7 porters have
already taken time and effort to try and diagnose the issue or suggest
workarounds and/or long-term solutions.

--
"There's something wrong with our bloody ships today, Chatfield."
  -- Admiral Beatty at the Battle of Jutland, 31st May 1916.

@p5pRT
Copy link
Author

p5pRT commented Aug 22, 2018

From frederik@ofb.net

So I'm mildly peeved by your response, and the OP's follow up of "oh at
last someone's taking some responsibility" - after about 7 porters have
already taken time and effort to try and diagnose the issue or suggest
workarounds and/or long-term solutions.

I'm sorry Dave, my reply to Karen was meant to be off-list, but I
failed to note the "via RT" in the email address. (Apologies to Karen
as well)

In any case I wasn't thanking her for taking responsibility, but for
encouraging P5P to do so. I'm also very familiar with the annoyance of
having to "pursue the issue in another bug queue", although I'm not
sure where I would send what suggestions, as I think we haven't really
discussed that yet.

I'm attaching some proposed patches for P5P, maybe I have enough
information to take some responsibility as well at this point.
However, these are not meant to be a "finished product", I'm sure they
will need some additional work before they can be applied.

I would hope that eventually the full module name can be reported by
the util.c error message, but rather than dig around to figure out how
to do that (I am badly unfamiliar with the software) I decided that my
documentation patch would just explain how to search for the module
using the information supplied by the current message.

Thank you,

Frederick

@p5pRT
Copy link
Author

p5pRT commented Aug 22, 2018

From frederik@ofb.net

util.c.diff
--- perl-5.28.0/util.c	2018-05-21 05:29:23.000000000 -0700
+++ /home/frederik/util.c-mod	2018-08-22 09:19:26.797143798 -0700
@@ -5337,7 +5337,8 @@
 	bad_handshake:/* recycle branch and string from above */
 	if(got != (void *)HSf_NOCHK)
 	    noperl_die("%s: loadable library and perl binaries are mismatched"
-                       " (got handshake key %p, needed %p)\n",
+                       " (got handshake key %p, needed %p)."
+                       " See perldiag(1) (do you need to recompile?)\n",
 		file, got, need);
     }
 

@p5pRT
Copy link
Author

p5pRT commented Aug 22, 2018

From frederik@ofb.net

perldiag.diff
--- perl-5.28.0/pod/perldiag.pod	2018-05-21 05:29:23.000000000 -0700
+++ /home/frederik/perldiag-mod	2018-08-22 09:34:08.050699448 -0700
@@ -3365,11 +3365,52 @@
 
 =item %s: loadable library and perl binaries are mismatched (got handshake key %p, needed %p)
 
-(P) A dynamic loading library C<.so> or C<.dll> was being loaded into the
-process that was built against a different build of perl than the
-said library was compiled against.  Reinstalling the XS module will
+(P) A dynamic loading library C<.so> or C<.dll> was being loaded into
+the process that was built against a different build of perl than the
+said library was compiled against. Reinstalling the XS module will
 likely fix this error.
 
+Typically this error occurs after an OS distribution upgrade, in which
+a new Perl version has been installed system-wide, but modules
+installed locally (e.g. in the user's home directory) have not been
+recompiled yet. In this case the problem can be fixed by telling the
+cpan tool to recompile all local modules. For example, on Unix-based
+installations:
+
+    $ PERL5LIB= cpan -r
+
+(the "PERL5LIB=" prefix ensures that cpan doesn't try to use local
+modules which are assumed to be broken in this case). This command
+should be run after every distribution upgrade.
+
+Alternatively, heavy users of modules downloaded from CPAN can install
+a local Perl instance using such tools as Perl::Build, perlbrew, and
+plenv, and use these when running local scripts. This will allow
+locally-installed Perl software to keep working when the system-wide
+Perl is upgraded, by managing two (or more) distinct versions of the
+Perl interpreter in parallel.
+
+To deal with this error in a module-specific way, the location of the
+offending module can be found by searching in your PERL5LIB for a file
+with the same name as is listed in the error message, for example:
+
+    Parser.c: loadable library and perl binaries are mismatched (got handshake key 0xce00080, needed 0xde00080)
+    # Find the DLL, looks like it belongs to the HTML::Parser module
+    $ find .local/lib/perl5 | grep Parser.so
+    .local/lib/perl5/x86_64-linux-thread-multi/auto/HTML/Parser/Parser.so
+    # Check that this is really where the module is
+    $ perldoc -l HTML::Parser
+    ~/.local/lib/perl5/x86_64-linux-thread-multi/HTML/Parser.pm
+    # Notice that the module is also installed system-wide
+    $ PERL5LIB= perldoc -l HTML::Parser
+    /usr/lib/perl5/5.26/vendor_perl/HTML/Parser.pm
+    # Remove the local version of the library (if installed via cpan)
+    $ PERL5LIB= cpanm -U HTML::Parser
+    ...
+    # Download, compile and install the newest version of the library (if
+    # needed)
+    $ PERL5LIB= cpanm HTML::Parser
+
 =item Locale '%s' contains (at least) the following characters which
 have unexpected meanings: %s  The Perl program will use the expected
 meanings

@p5pRT
Copy link
Author

p5pRT commented Aug 27, 2018

From frederik@ofb.net

Here is an updated version of the documentation patch. I made some
clarifications and also added a paragraph suggesting that users prefer
distribution packages of perl modules.

After I learned that 'cpan -r' and other installation commands cause
core modules to get reinstalled locally, I'm interested in learning
how to prevent that from happening.

The 'cpanm' man page has some promising options for excluding core and
vendor modules but when I try these options with such a module, then
the module still gets installed​:

  $ cpanm --self-contained --exclude-vendor HTTP​::Date
  --> Working on HTTP​::Date
  Fetching http​://www.cpan.org/authors/id/G/GA/GAAS/HTTP-Date-6.02.tar.gz ... OK

If anyone can comment on this...

I also welcome comments on the patches.

Thanks,

Frederick

@p5pRT
Copy link
Author

p5pRT commented Aug 28, 2018

From @iabyn

On Mon, Aug 27, 2018 at 09​:26​:35AM -0700, frederik@​ofb.net wrote​:

Here is an updated version of the documentation patch.

I'm not seeing anything attached.

--
But Pity stayed his hand. "It's a pity I've run out of bullets",
he thought. -- "Bored of the Rings"

@p5pRT
Copy link
Author

p5pRT commented Aug 28, 2018

From frederik@ofb.net

Apologies, trying that again...

On Tue, Aug 28, 2018 at 12​:48​:29AM -0700, Dave Mitchell via RT wrote​:

On Mon, Aug 27, 2018 at 09​:26​:35AM -0700, frederik@​ofb.net wrote​:

Here is an updated version of the documentation patch.

I'm not seeing anything attached.

--
But Pity stayed his hand. "It's a pity I've run out of bullets",
he thought. -- "Bored of the Rings"

@p5pRT
Copy link
Author

p5pRT commented Aug 28, 2018

From frederik@ofb.net

perldiag.2.diff
--- /home/frederik/pkg-tmp/packages/perl/trunk/src/perl-5.28.0/pod/perldiag.pod	2018-05-21 05:29:23.000000000 -0700
+++ /home/frederik/perldiag-mod	2018-08-27 09:19:20.502177824 -0700
@@ -3370,6 +3370,64 @@
 said library was compiled against.  Reinstalling the XS module will
 likely fix this error.
 
+Typically this error occurs after an OS distribution upgrade, in which
+a new Perl version has been installed system-wide, but modules
+installed locally (e.g. in the user's home directory) have not been
+recompiled yet. In the common case where the modules have been
+installed using the 'cpan' or 'cpanm' tool, the problem can be fixed
+by telling 'cpan' to recompile all local modules. For example, on
+Unix-based installations:
+
+    $ PERL5LIB= cpan -r
+
+(the "PERL5LIB=" prefix ensures that cpan doesn't try to use local
+modules, which are assumed to be broken in this case). This command
+should be run after every distribution upgrade.
+
+Alternatively, heavy users of locally-compiled modules can install a
+local Perl instance using such tools as Perl::Build, perlbrew, and
+plenv, and use these when running local scripts. This will allow
+locally-installed Perl software to keep working when the system-wide
+Perl is upgraded, by managing two (or more) distinct versions of the
+Perl interpreter in parallel.
+
+Consider using modules packaged by your distribution when possible,
+rather than downloading and compiling them using the 'cpan' tool.
+Unlike locally-compiled modules, these will be automatically upgraded
+when you upgrade your distribution's perl package. For example, rather
+than installing BerkeleyDB or CGI from CPAN, check whether your
+distribution has a package called perl-berkeleydb or perl-cgi, and
+install that package instead.
+
+If you prefer to address this error on a per-module basis, you can
+find the location of the offending module by searching directories in
+your PERL5LIB for a file with the same name as is listed in the error
+message, for example:
+
+    $ some-perl-script
+    Parser.c: loadable library and perl binaries are mismatched ...
+
+    ## Find the DLL, looks like it belongs to the HTML::Parser module
+    $ find .local/lib/perl5 | grep Parser.so
+    .local/lib/perl5/x86_64-linux-thread-multi/auto/HTML/Parser/Parser.so
+
+    ## Check that this is really where the module is
+    $ perldoc -l HTML::Parser
+    ~/.local/lib/perl5/x86_64-linux-thread-multi/HTML/Parser.pm
+
+    ## Notice that the module is also installed system-wide
+    $ PERL5LIB= perldoc -l HTML::Parser
+    /usr/lib/perl5/5.26/vendor_perl/HTML/Parser.pm
+
+    ## Remove the local version of the library (if installed via cpan).
+    ## After this, the system version will be used.
+    $ PERL5LIB= cpanm -U HTML::Parser
+    ...
+
+    ## Download, compile and install the newest version of the library
+    ## (if the system version isn't new enough for you)
+    $ PERL5LIB= cpanm HTML::Parser
+
 =item Locale '%s' contains (at least) the following characters which
 have unexpected meanings: %s  The Perl program will use the expected
 meanings

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2018

From frederik@ofb.net

Any comments on the patch?

@p5pRT
Copy link
Author

p5pRT commented Sep 17, 2018

From @iabyn

On Tue, Aug 28, 2018 at 08​:23​:57AM -0700, frederik@​ofb.net wrote​:

+Typically this error occurs after an OS distribution upgrade, in which
+a new Perl version has been installed system-wide, but modules
+installed locally (e.g. in the user's home directory) have not been
+recompiled yet.

I dislike this text (and by extension the whole patch). You are mentioning
one particular scenario, which happened to occur to you, and are assuming
that this is the common case.

There are many reasons why a user might get this error, and your case was
just one of them.

--
Indomitable in retreat, invincible in advance, insufferable in victory
  -- Churchill on Montgomery

@p5pRT
Copy link
Author

p5pRT commented Sep 17, 2018

From frederik@ofb.net

On Mon, Sep 17, 2018 at 12​:24​:31PM +0100, Dave Mitchell wrote​:

On Tue, Aug 28, 2018 at 08​:23​:57AM -0700, frederik@​ofb.net wrote​:

+Typically this error occurs after an OS distribution upgrade, in which
+a new Perl version has been installed system-wide, but modules
+installed locally (e.g. in the user's home directory) have not been
+recompiled yet.

I dislike this text (and by extension the whole patch). You are mentioning
one particular scenario, which happened to occur to you, and are assuming
that this is the common case.

There are many reasons why a user might get this error, and your case was
just one of them.

Thanks for the feedback. Do you use Linux From Scratich or something?

I guess my patch makes a lot of assumptions about how people would get
this particular error. I can try to change the wording if you give me
more guidance. The underlying goal is to make it so that people can
figure out how to fix their systems and get them into a working state
again without using Google, i.e. just by using the manual pages and
other documentation that comes with Perl. I guess another assumption
is that a user doesn't have to have read and remembered all the
documentation that comes with all the software he installed, because
otherwise you could argue that this is already the case.

Another thing which is confusing is that the software, while
presumably knowing the name of the offending module which it was
attempting to load at the time of the error, did not report this name
to the user. That's confusing because it makes it seem like the
problem is not with a module. The documentation and diagnostics
changes I suggested were also partly meant to compensate for this
information gap.

If it is just that one sentence that's a problem, then it would
probably be easy to rephrase it in a way that preserves the intent
while broadening the scope​:

"Typically this error occurs after a new version of Perl has been
installed, for example during an operating system upgrade. Any
non-core modules with XS code will need to be recompiled at the same
time, or they will trigger this error when loaded. In the common case
where such modules have been installed using the 'cpan' or 'cpanm'
tool [...]"

Also I don't know the "pod" format, I'm assuming that it would be
easier for whoever applies the patch to fix any formatting problems
himself rather than engage in further back-and-forth, but let me know
if that is not the case.

Thanks,

Frederick

@p5pRT
Copy link
Author

p5pRT commented Sep 17, 2018

From @xenu

On Mon, 17 Sep 2018, at 13​:24, Dave Mitchell wrote​:

On Tue, Aug 28, 2018 at 08​:23​:57AM -0700, frederik@​ofb.net wrote​:

+Typically this error occurs after an OS distribution upgrade, in which
+a new Perl version has been installed system-wide, but modules
+installed locally (e.g. in the user's home directory) have not been
+recompiled yet.

I dislike this text (and by extension the whole patch). You are mentioning
one particular scenario, which happened to occur to you, and are assuming
that this is the common case.

There are many reasons why a user might get this error, and your case was
just one of them.

As someone who spends a lot of time on perl support irc channels, I'd say it's the most
common case *by far*. It happens all the time to users of rolling-release linux
distributions like arch, gentoo, opensuse tumbleweed etc.

@p5pRT
Copy link
Author

p5pRT commented Sep 18, 2018

From @iabyn

On Mon, Sep 17, 2018 at 07​:15​:55AM -0700, frederik@​ofb.net wrote​:

If it is just that one sentence that's a problem, then it would
probably be easy to rephrase it in a way that preserves the intent
while broadening the scope​:

No, I think the general thrust of the patch is misguided. As I said a lot
earlier in this thread, I would find it hard to write such a patch.

The particular scenario you encountered seems to me to be relatively rare.
It required the following combination of circumstances​:

1) You were using the OS's perl installation.
2) You didn't use the OS's packing system to add extra CPAN modules​:
  for example on Fedora to install the Time​::ParseDate module,
  I would install the perl-Time-ParseDate RPM provided by Fedora.
3) You didn't use cpan or similar in a way that installs the extra packages
  as part of the perl installation which the cpan tool is a part of.
4) You used the third-party local​::lib tool install the extra CPAN modules
  under your home directory, which doesn't install them in
  version-specific paths.

Change any of those conditions and you likely wouldn't have seen that
error message (most likely you would instead have seen an error message
about not being able to find the module).

Some other ways of getting that error message include​:
* having both a system perl and another perl installed, and an incorrect
  setting of PERL5LIB or similar causes one perl to pick up modules from
  paths intended for the other perl.
* Or rebuilding the same version of perl, but with different build options.
* Or someone thinking they can "install" a module by just copying the *.pm
  and *.so files from one system to another.

All you can really say about that error message, is that a perl
interpreter is trying to load an XS module which was built using a
different perl interpreter (but not necessarily a different version).

--
Spock (or Data) is fired from his high-ranking position for not being able
to understand the most basic nuances of about one in three sentences that
anyone says to him.
  -- Things That Never Happen in "Star Trek" #19

@p5pRT
Copy link
Author

p5pRT commented Sep 18, 2018

From frederik@ofb.net

Hi Dave,

I'm looking over the messages in this thread and I see that you said
you would find it difficult to write a manual page about upgrading
perl modules, but you also said that you were not opposed to having
the error message refer to a manual page, and you suggested adding an
extra section in an existing page. Then I submitted a patch along the
lines of your suggestions - modifying the error message to refer to a
manual page and adding additional material to a section in an existing
manual page.

Are you still interested in brainstorming about ways to fix this
particular issue? If so, then I'm happy to continue modifying what
I've done to satisfy the requirements of the situation. Just let me
know in what general direction you would like to see this go.

Thanks,

Frederick

On Tue, Sep 18, 2018 at 10​:43​:27AM +0100, Dave Mitchell wrote​:

On Mon, Sep 17, 2018 at 07​:15​:55AM -0700, frederik@​ofb.net wrote​:

If it is just that one sentence that's a problem, then it would
probably be easy to rephrase it in a way that preserves the intent
while broadening the scope​:

No, I think the general thrust of the patch is misguided. As I said a lot
earlier in this thread, I would find it hard to write such a patch.

The particular scenario you encountered seems to me to be relatively rare.
It required the following combination of circumstances​:

1) You were using the OS's perl installation.
2) You didn't use the OS's packing system to add extra CPAN modules​:
for example on Fedora to install the Time​::ParseDate module,
I would install the perl-Time-ParseDate RPM provided by Fedora.
3) You didn't use cpan or similar in a way that installs the extra packages
as part of the perl installation which the cpan tool is a part of.
4) You used the third-party local​::lib tool install the extra CPAN modules
under your home directory, which doesn't install them in
version-specific paths.

Change any of those conditions and you likely wouldn't have seen that
error message (most likely you would instead have seen an error message
about not being able to find the module).

Some other ways of getting that error message include​:
* having both a system perl and another perl installed, and an incorrect
setting of PERL5LIB or similar causes one perl to pick up modules from
paths intended for the other perl.
* Or rebuilding the same version of perl, but with different build options.
* Or someone thinking they can "install" a module by just copying the *.pm
and *.so files from one system to another.

All you can really say about that error message, is that a perl
interpreter is trying to load an XS module which was built using a
different perl interpreter (but not necessarily a different version).

@p5pRT
Copy link
Author

p5pRT commented Sep 18, 2018

From @Leont

On Tue, Sep 18, 2018 at 11​:44 AM Dave Mitchell <davem@​iabyn.com> wrote​:

On Mon, Sep 17, 2018 at 07​:15​:55AM -0700, frederik@​ofb.net wrote​:

If it is just that one sentence that's a problem, then it would
probably be easy to rephrase it in a way that preserves the intent
while broadening the scope​:

No, I think the general thrust of the patch is misguided. As I said a lot
earlier in this thread, I would find it hard to write such a patch.

The particular scenario you encountered seems to me to be relatively rare.
It required the following combination of circumstances​:

1) You were using the OS's perl installation.
2) You didn't use the OS's packing system to add extra CPAN modules​:
for example on Fedora to install the Time​::ParseDate module,
I would install the perl-Time-ParseDate RPM provided by Fedora.
3) You didn't use cpan or similar in a way that installs the extra packages
as part of the perl installation which the cpan tool is a part of.
4) You used the third-party local​::lib tool install the extra CPAN modules
under your home directory, which doesn't install them in
version-specific paths.

Change any of those conditions and you likely wouldn't have seen that
error message (most likely you would instead have seen an error message
about not being able to find the module).

Some other ways of getting that error message include​:
* having both a system perl and another perl installed, and an incorrect
setting of PERL5LIB or similar causes one perl to pick up modules from
paths intended for the other perl.
* Or rebuilding the same version of perl, but with different build options.
* Or someone thinking they can "install" a module by just copying the *.pm
and *.so files from one system to another.

Yeah, especially the first option is quite easily triggered in my
experience (and the reason I don't mix perlbrew and local​::lib). I
think it's useful to list various reasons why this can happen.

That said, there is one thing about the case Frederik describes that
sets it apart from the others​: it's a "but it worked yesterday"
scenario. One can trigger it without the end-user doing anything perl
related (unlike compiling a new and incompatible perl). That probably
explains Tomasz' observation that questions on IRC tend to be about
this scenario.

Leon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants