From @nwc10

Created by @nwc10

There isn't a ticket for this yet​:

Need to provide patches for CPAN and CPANPLUS so that

a​: 'install Switch' (etc)
  automatically behaves like 'force install Switch' when Switch is marked as
  deprecated in core, so that it grabs the CPAN version
b​: a dependency on Switch (etc) causes an install of Switch from CPAN, if it
  was loaded from the core library paths.

Are there any other "loopholes" we need to close, so that the toolchain treats
"Switch in core" as if it's not actually installed?

Deprecating modules from core for the 5.12.0 release doesn't work until we've
resolved this.

Nicholas Clark

Flags:
    category=library
    severity=high

Site configuration information for perl 5.11.0.42:

Configured by nick at Fri Oct 16 20:36:59 BST 2009.

Summary of my perl5 (revision 5 version 11 subversion 0) configuration:
  Commit id: 20d0b1e9c410d995ea730a00781152c652d4b672
  Platform:
    osname=linux, osvers=2.6.18-xenu, archname=x86_64-linux-thread-multi
    uname='linux zazen 2.6.18-xenu #1 smp thu oct 4 12:23:41 bst 2007 x86_64 gnulinux '
    config_args='-Dusedevel=y -Dcc=ccache g++ -Dld=g++ -Ubincompat5005 -Uinstallusrbinperl -Dcf_email=nick@ccl4.org -Dperladmin=nick@ccl4.org -Dinc_version_list=  -Dinc_version_list_init=0 -Doptimize=-Os -Dusethreads -Duse64bitall -Uusemymalloc -Duseperlio -Dprefix=~/Sandpit/snap5.9.x-v5.11.0-218-g20d0b1e -Uusevendorprefix -Uvendorprefix=~/Sandpit/snap5.9.x-v5.11.0-218-g20d0b1e -Dinstallman1dir=none -Dinstallman3dir=none -Uuserelocatableinc -de'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='ccache g++', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-Os',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.3.2', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='g++', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib /lib64 /usr/lib64
    libs=-lnsl -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc.so.6, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.7'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -Os -L/usr/local/lib -fstack-protector'

Locally applied patches:
    


@INC for perl 5.11.0.42:
    lib
    /home/nick/Sandpit/snap5.9.x-v5.11.0-218-g20d0b1e/lib/perl5/site_perl/5.11.0/x86_64-linux-thread-multi
    /home/nick/Sandpit/snap5.9.x-v5.11.0-218-g20d0b1e/lib/perl5/site_perl/5.11.0
    /home/nick/Sandpit/snap5.9.x-v5.11.0-218-g20d0b1e/lib/perl5/5.11.0/x86_64-linux-thread-multi
    /home/nick/Sandpit/snap5.9.x-v5.11.0-218-g20d0b1e/lib/perl5/5.11.0
    .


Environment for perl 5.11.0.42:
    HOME=/home/nick
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/nick/bin:/usr/local/bin:/usr/bin:/bin:/usr/games:/usr/local/sbin:/sbin:/usr/sbin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

From @xdg

CPAN.pm doesn't care if a module is core or not. If a requested module
is not installed with a sufficiently high version, CPAN.pm attempts to
retrieve it from CPAN.

As long as the version on CPAN is newer than the deprecated version in
core, I think this is a non-issue with respect to CPAN.pm.

-- David

The RT System itself - Status changed from 'new' to 'open'

From @nwc10

On Mon, Oct 19, 2009 at 01​:36​:40PM -0700, David Golden via RT wrote​:

CPAN.pm doesn't care if a module is core or not. If a requested module
is not installed with a sufficiently high version, CPAN.pm attempts to
retrieve it from CPAN.

As long as the version on CPAN is newer than the deprecated version in
core, I think this is a non-issue with respect to CPAN.pm.

The module on CPAN will be the *same* version as in core.

This is deliberate.

If CPAN is intentionally 1 ahead of core then we get the following issues

Start with​: Core has 1.23, CPAN has 1.24

Dual life module author makes bug fix. Forgets this little constraint
Puts 1.25 on CPAN.

Now, can't integrate *that* to blead because it's the same version

So, put out 1.27 on CPAN, which is just a version bump, so that 1.26 can go
into core.

Alternative, also not good​:

Have a 1.26 in core that has the code that warns that it is depreciated
Have a 1.26 on CPAN that doesn't

The PAUSE indexer is now unhappy (or Andreas isn't) because we have two modules
claiming to be the same version, but they aren't identical.

The design is to have byte for byte *identical* versions on CPAN and core,
so that this part is a non-issue, and so that the PAUSE indexer doesn't have
kittens.

The trick to having this work is that the toolchain modules

1​: know which modules are deprecated in core
2​: treat any of these modules that are installed by core, but not in sitelib,
  as if they're actually missing
3​: download and build them from CPAN, just like any other missing dependency
  (and the distribution from sitelib will install to "site", because authors
  will be doing that, because they will politely sent patches to fix this)

I know that CPAN can already do most of this, because "force install" works.
Effectively what is needed is to have "install Switch" behave automatically
as "force install Switch", if Switch is a module in core but deprecated from
core.

Nicholas Clark

From @xdg

On Mon, Oct 19, 2009 at 4​:45 PM, Nicholas Clark <nick@​ccl4.org> wrote​:

Alternative, also not good​:

Have a 1.26 in core that has the code that warns that it is depreciated
Have a 1.26 on CPAN that doesn't

I thought the strategy was to have "deprecated" used to indicate
modules that were deprecated, right? Has that changed?

My thought is that anything deprecated get a version in core marked
deprecated and then no further bug fixes are brought into core. Any
bug fixes only exist on CPAN.

Deprecating a module then just requires​:

* Bump version slightly in core (could be a dev version) and add
'deprecated.pm'.
* Bump version to higher than core (*not* a dev), remove deprecated.pm.
* All future updates *only* go to CPAN

Does that solve the problem without adding special cases in the toolchain?

The trick to having this work is that the toolchain modules

1​: know which modules are deprecated in core

If we really have to go this way, add this to Module​::CoreList (or
create Module​::CoreList​::Deprecated). I don't want to maintain a
one-off list in the toolchain.

-- David

From @nwc10

On Mon, Oct 19, 2009 at 04​:57​:24PM -0400, David Golden wrote​:

On Mon, Oct 19, 2009 at 4​:45 PM, Nicholas Clark <nick@​ccl4.org> wrote​:

Alternative, also not good​:

Have a 1.26 in core that has the code that warns that it is depreciated
Have a 1.26 on CPAN that doesn't

I thought the strategy was to have "deprecated" used to indicate
modules that were deprecated, right? Has that changed?

Yes, No.

The code of deprecate.pm consults %INC to see where the module was laoded from.
If sitelib and archlib have been configured to be the the same, it does nothing
(because you can't tell anything. This is not a standard configuration)
If they differ, then
  If the module is loaded from archlib, it's from the core, and it warns
  If the module is loaded from sitelib, it's been installed by CPAN, and it is
  silent

As sitelib is now ahead of archlib in @​INC, the same module installed in both
will be loaded from sitelib.
Downloading and installing the module from CPAN will install it to sitelib
(Because the author will have changed Makefile.PL to do this)

Hence code will work, with warnings, for the duration of the deprecation
period, but the warnings will go away, if the module is properly installed
from CPAN (manually, or as a correctly given dependency)

Hence when, after the deprecation period the module is removed, packages now
with correctly specified dependencies will work.

My thought is that anything deprecated get a version in core marked
deprecated and then no further bug fixes are brought into core. Any
bug fixes only exist on CPAN.

I was assuming a general solution. We may well deprecate from core actively
developed modules, or modules that end up with CVEs.

Given that the aggregation of the current plans for blead and maint amount to​:

  The combination of all 3 as proposed is that only .0 stable releases have
  the non-critical bug fixes.

  There *aren't* going to be that many stable releases with bug fixes - at
  most once a year, maybe not that frequently. And that release is going to
  be 5.$x.0, which has a superstition attached.
 

which, I believe, isn't actually what people want, but is right now the *best*
that is going to happen. [And it's not clear whether Dave is only volunteering
to do a "critical bugfix style 5.10.2, 5.10.3, or is also doing prepared to do
the same for 5.12.1 etc, so it may be that *no-one* will do maint-5.12 or
maint-5.14]

So I'm not assuming that this plan will stick, given that Dave has *also*
offered to provide time for a more traditional "maint" at commercial rates.
At which point a general solution would be needed, because maint would want
to pull in current modules.

Does that solve the problem without adding special cases in the toolchain?

Yes, but irritates me because I worked hard to make a general solution
possible, and both Andreas and Jos agreed to my plan.

The trick to having this work is that the toolchain modules

1​: know which modules are deprecated in core

If we really have to go this way, add this to Module​::CoreList (or
create Module​::CoreList​::Deprecated). I don't want to maintain a
one-off list in the toolchain.

Agree. I had assumed that the toolchain gets the version of installed modules,
by loading them. Am I wrong, in that it finds the file on disk and parses it?
The thought being that by loading the module, deprecate.pm could build up a
data structure of which module declared themselves to be deprecated.

Either way, yes, it's better to put the data in Module​::CoreList.

Nicholas Clark

From @xdg

On Mon, Oct 19, 2009 at 5​:22 PM, Nicholas Clark <nick@​ccl4.org> wrote​:

Given that the aggregation of the current plans for blead and maint amount to​:

   The combination of all 3 as proposed is that only .0 stable releases have
   the non-critical bug fixes.

   There *aren't* going to be that many stable releases with bug fixes - at
   most once a year, maybe not that frequently. And that release is going to
   be 5.$x.0, which has a superstition attached.

I need to jump back to those threads, but I understood it to be only
*maint* that would be non-critical releases, with blead continuing to
churn forward (just with timeboxed releases). It still means that
people will have to live with a maint release for a year before
anything major changes, yes.

Does that solve the problem without adding special cases in the toolchain?

Yes, but irritates me because I worked hard to make a general solution
possible, and both Andreas and Jos agreed to my plan.

Sorry. I'm in 'special-case' hell maintaining M​::B​::Compat (and much
of the rest of the toolchain bits), so I get really squeamish of doing
extra work when we can avoid it. But then, I'm on the "just break
stuff" side of the p5p debates, so YMMV. :-)

The trick to having this work is that the toolchain modules

1​: know which modules are deprecated in core

If we really have to go this way, add this to Module​::CoreList (or
create Module​::CoreList​::Deprecated).  I don't want to maintain a
one-off list in the toolchain.

Agree. I had assumed that the toolchain gets the version of installed modules,
by loading them. Am I wrong, in that it finds the file on disk and parses it?
The thought being that by loading the module, deprecate.pm could build up a
data structure of which module declared themselves to be deprecated.

Either way, yes, it's better to put the data in Module​::CoreList.

Generally speaking, I think most of the toolchain searches @​INC for
what would be loaded and then calls MM->parse_version on the resulting
file. So only the $VERSION line gets eval'd. M​::B has its own
parse_version implementation but it's the same approach. Very little
actually gets loaded unless absolutely necessary.

-- David

From @xdg

On Mon, Oct 19, 2009 at 5​:45 PM, David Golden <dagolden@​cpan.org> wrote​:

If we really have to go this way, add this to Module​::CoreList (or
create Module​::CoreList​::Deprecated).  I don't want to maintain a
one-off list in the toolchain.

Agree. I had assumed that the toolchain gets the version of installed modules,
by loading them. Am I wrong, in that it finds the file on disk and parses it?
The thought being that by loading the module, deprecate.pm could build up a
data structure of which module declared themselves to be deprecated.

Either way, yes, it's better to put the data in Module​::CoreList.

Generally speaking, I think most of the toolchain searches @​INC for
what would be loaded and then calls MM->parse_version on the resulting
file.  So only the $VERSION line gets eval'd.  M​::B has its own
parse_version implementation but it's the same approach.  Very little
actually gets loaded unless absolutely necessary.

I think I finally grok what you're asking for. You want deprecated
core modules to be "invisible" during dependency resolution.
Specifically​:

* When CPAN/PLUS check for the "installed" version of a
core-deprecated module, they should *ignore* it if it is installed in
$Config{installprivlib} or $Config{installarchlib}

This means that while anything that happens to specify the deprecated
module as a dependency will result in a new copy of the deprecated
module being installed from CPAN.

The only tricky bit I see is that when users search for a module (e.g.
"m Switch" in the CPAN Shell), if the installed version is ignored
there, then CPAN.pm is lying to the user about it being installed. So
it can't just be invisible in the "check if installed" routine -- but
the caller may need to be informed of deprecation status.

As for Module​::CoreList, this is what I want as an API, I think​:

  if ( Module​::CoreList->is_deprecated( $module, $perl_version ) ) { ... }

With $perl_version defaulting to $] if not defined.

-- David

From @andk

David, the commit 22a8f074 in cpampm repo was already trying to do (a)
and I think it does so correctly and does not need the suggested API.
I also think that this commit already does (b) but I have no test case
at hand. If CPAN.pm really does both (a) and (b) we may not need the
suggested API although it looks very elegant.

From @bingos

On Mon, Oct 19, 2009 at 08​:47​:07PM -0400, David Golden wrote​:

As for Module​::CoreList, this is what I want as an API, I think​:

if ( Module​::CoreList->is_deprecated( $module, $perl_version ) ) { ... }

With $perl_version defaulting to $] if not defined.

-- David

Seconded.

CPANPLUS would need amending it at least two places from what I can see​:

CPANPLUS​::Module
CPANPLUS​::Dist

Module​::CoreList is already used by CPANPLUS.

CPANPLUS delegates checking whether a module is installed already or not to
Module​::Load​::Conditional->check_install(). That returns enough information
to determine the installation path, etc.

I see blead already has Switch deprecated and @​INC set appropriately, cool.

All I need is the M​::CL function then.

Cheers,

--
Chris Williams
aka BinGOs
PGP ID 0x4658671F
http​://www.gumbynet.org.uk

From @xdg

Module​::CoreList​::is_deprecated is implemented in perl Core.

CPAN.pm in repository has been updated to use it and the next time
CPAN.pm syncs to blead, it will be included.

Switch needs a new CPAN release from blead that includes the newly
updated INSTALLDIRS.

Shell needs change INSTALLDIRS and re-release to CPAN. C.f.
https://rt.cpan.org/Ticket/Display.html?id=50729

That's it from the CPAN.pm side, I think.

-- David

From @obra

Tested against 5.11.3tobe with now-current CPAN and CPANPLUS.

Each will install Switch from CPAN the first time it's asked to and
then report that it's already up to date. After running the
installation, there's a new copy of switch in site_perl.

Resolving

From [Unknown Contact. See original ticket]

Tested against 5.11.3tobe with now-current CPAN and CPANPLUS.

Each will install Switch from CPAN the first time it's asked to and
then report that it's already up to date. After running the
installation, there's a new copy of switch in site_perl.

Resolving

@obra - Status changed from 'open' to 'resolved'