Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"." in @INC prose for perl5.26 perldelta #15977

Closed
p5pRT opened this issue May 13, 2017 · 9 comments
Closed

"." in @INC prose for perl5.26 perldelta #15977

p5pRT opened this issue May 13, 2017 · 9 comments

Comments

@p5pRT
Copy link

p5pRT commented May 13, 2017

Migrated from rt.perl.org#131304 (status was 'resolved')

Searchable as RT131304$

@p5pRT
Copy link
Author

p5pRT commented May 13, 2017

From @kentfredric

Created by @kentfredric

As per​: http​://www.nntp.perl.org/group/perl.perl5.porters/2017/03/msg243763.html

There is a desperate need for documentation in 5.26 final release about '.'
in @​INC.

This bug is just so we can add an urgent blocker so it doesn't slip further.

Attached is the last combination of suggestions to the list, ( which I haven't even read yet )
composed from davem's initial contribution, with suggestions by myself and sawyer,
integrated together by jekeenan.

It does need to have more eyes on before its final though.

Perl Info

Flags:
    category=docs
    severity=high

Site configuration information for perl 5.25.12:

Configured by kent at Fri Apr 21 08:39:37 NZST 2017.

Summary of my perl5 (revision 5 version 25 subversion 12) configuration:
   
  Platform:
    osname=linux
    osvers=4.10.4-gentoo
    archname=x86_64-linux
    uname='linux katipo2 4.10.4-gentoo #47 smp preempt wed mar 22 18:50:38 nzdt 2017 x86_64 intel(r) core(tm) i5-2410m cpu @ 2.30ghz genuineintel gnulinux '
    config_args='-de -Dprefix=/home/kent/perl5/perlbrew/perls/5.25.12-nossp-sip13-nopmc-nodot -Ddefault_inc_excludes_dot -Doptimize= -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native -Dman1dir=none -Dman3dir=none -Dusedevel -Accflags= -fstack-protector-strong -fno-stack-protector -DPERL_HASH_FUNC_SIPHASH13 -DPERL_DISABLE_PMC -Aldflags= -fstack-protector-strong -fno-stack-protector -Aeval:scriptdir=/home/kent/perl5/perlbrew/perls/5.25.12-nossp-sip13-nopmc-nodot/bin'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fstack-protector-strong -fno-stack-protector -DPERL_HASH_FUNC_SIPHASH13 -DPERL_DISABLE_PMC -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    optimize=' -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native'
    cppflags='-fstack-protector-strong -fno-stack-protector -DPERL_HASH_FUNC_SIPHASH13 -DPERL_DISABLE_PMC -fwrapv -fno-strict-aliasing -pipe'
    ccversion=''
    gccversion='5.4.0'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags ='	-fstack-protector-strong -fno-stack-protector -L/usr/local/lib'
    libpth=/usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/include-fixed /usr/lib /usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lpthread -lnsl -lndbm -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.24.so
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.24'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared	-fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native -L/usr/local/lib'



@INC for perl 5.25.12:
    /home/kent/perl5/perlbrew/perls/5.25.12-nossp-sip13-nopmc-nodot/lib/site_perl/5.25.12/x86_64-linux
    /home/kent/perl5/perlbrew/perls/5.25.12-nossp-sip13-nopmc-nodot/lib/site_perl/5.25.12
    /home/kent/perl5/perlbrew/perls/5.25.12-nossp-sip13-nopmc-nodot/lib/5.25.12/x86_64-linux
    /home/kent/perl5/perlbrew/perls/5.25.12-nossp-sip13-nopmc-nodot/lib/5.25.12


Environment for perl 5.25.12:
    HOME=/home/kent
    LANG=en_NZ.UTF8
    LANGUAGE (unset)
    LC_CTYPE=en_NZ.UTF8
    LC_TIME=en_NZ.UTF8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/kent/.rvm/gems/ruby-2.1.2/bin:/home/kent/.rvm/gems/ruby-2.1.2@global/bin:/home/kent/.rvm/rubies/ruby-2.1.2/bin:/home/kent/perl5/perlbrew/bin:/home/kent/perl5/perlbrew/perls/5.25.12-nossp-sip13-nopmc-nodot/bin:/home/kent/.perl6/2013.04/bin:/home/kent/.gem/ruby/1.8/bin/:/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/5.4.0:/usr/x86_64-pc-linux-gnu/x86_64-pc-linux-gnux32/gcc-bin/5.4.0:/usr/x86_64-pc-linux-gnu/arm-unknown-linux-gnueabi/gcc-bin/5.4.0:/usr/lib/llvm/4/bin:/usr/games/bin:/home/kent/.rvm/bin:/home/kent/.rvm/bin
    PERLBREW_BASHRC_VERSION=0.74
    PERLBREW_HOME=/home/kent/.perlbrew
    PERLBREW_MANPATH=/home/kent/perl5/perlbrew/perls/5.25.12-nossp-sip13-nopmc-nodot/man
    PERLBREW_PATH=/home/kent/perl5/perlbrew/bin:/home/kent/perl5/perlbrew/perls/5.25.12-nossp-sip13-nopmc-nodot/bin
    PERLBREW_PERL=5.25.12-nossp-sip13-nopmc-nodot
    PERLBREW_ROOT=/home/kent/perl5/perlbrew
    PERLBREW_VERSION=0.74
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented May 13, 2017

From @kentfredric

dm.perldelta.2.pod

@p5pRT
Copy link
Author

p5pRT commented May 17, 2017

From @jmdh

I reviewed the attached prose and made some editorial tweaks. Attached is both a new file and a diff from the previous one. Overall I found the text very clear and I recommend that they be incorporated into perldelta ASAP. I also include the results of applying this text to blead's perldelta.

@p5pRT
Copy link
Author

p5pRT commented May 17, 2017

From @jmdh

0001-Add-more-extensive-information-about-INC-changes-int.patch
From bd8ba1b0df4992758830744ae209b4d590b816d2 Mon Sep 17 00:00:00 2001
From: Dominic Hargreaves <dom@earth.li>
Date: Wed, 17 May 2017 15:17:42 +0100
Subject: [PATCH] Add more extensive information about @INC changes into
 perldelta (RT#131304)

This mainly takes existing text written by Dave Mitchell, Kent Fredric,
Sawyer X and James E Keenan.
---
 pod/perldelta.pod | 215 +++++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 198 insertions(+), 17 deletions(-)

diff --git a/pod/perldelta.pod b/pod/perldelta.pod
index ed13834..505a22a 100644
--- a/pod/perldelta.pod
+++ b/pod/perldelta.pod
@@ -9,6 +9,24 @@ perldelta - what is new for perl v5.26.0
 This document describes the differences between the 5.24.0 release and the
 5.26.0 release.
 
+=head1 Notice
+
+This release includes two updates with widespread effects:
+
+=over 4
+
+=item * C<.> no longer in C<@INC>
+
+The current modules, and for the execution of scripts. See the section
+L<<Removal of the current directory C<.> from @INC>> for the full details.
+
+=item * C<do> may now warn
+
+C<do> now gives a mandatory warning when it fails to load a file which it
+would have loaded had C<.> been in C<@INC>.
+
+=back
+
 =head1 Core Enhancements
 
 =head2 New regular expression modifier C</xx>
@@ -62,19 +80,6 @@ For example:
 
 prints "Hello there\n" with no leading whitespace.
 
-=head2 '.' and @INC
-
-Perl now provides a way to build perl without C<.> in @INC by default. If you
-want this feature, you can build with -Ddefault_inc_excludes_dot
-
-Because the testing / make process for perl modules do not function well with
-C<.> missing from @INC, Perl now supports the environment variable
-PERL_USE_UNSAFE_INC=1 which makes Perl behave as it previously did, returning
-C<.> to @INC in all child processes.
-
-WARNING: C<PERL_USE_UNSAFE_INC> has been provided during the perl 5.25
-development cycle and is not guaranteed to function in perl 5.26.
-
 =head2 create a safer utf8_hop() called utf8_hop_safe()
 
 Unlike utf8_hop(), utf8_hop_safe() won't navigate before the beginning or after
@@ -192,10 +197,178 @@ has been deprecated to do so since Perl v5.22.
 
 =head1 Security
 
-=head2 Remove current dir (C<.>) from C<@INC>
+=head2 Removal of the current directory (C<.>) from C<@INC>
+
+The perl binary includes a default set of paths in C<@INC>. Historically
+it has also included the current directory (C<.>) as the final entry,
+unless run with taint mode enabled (C<perl -T>). While convenient, this has
+security implications: for example, where a script attempts to load an
+optional module when its current directory is untrusted (such as F</tmp>),
+it could load and execute code from under that directory.
 
-For security reasons, C<@INC> no longer contains the default directory
-(C<.>).
+Starting with v5.26.0, C<.> is always removed by default, not just under
+tainting. This has major implications for installing modules and executing
+scripts.
+
+The following new features have been added to help ameliorate these
+issues.
+
+=over
+
+=item * C<Configure -Udefault_inc_excludes_dot>
+
+There is a new C<Configure> option, C<default_inc_excludes_dot> (enabled
+by default) which builds a perl executable without C<.>; unsetting this
+option using C<-U> reverts perl to the old behaviour. This may fix your
+path issues but will reintroduce all the security concerns, so don't
+build a perl executable like this unless you're I<really> confident that
+such issues are not a concern in your environment.
+
+=item * C<$PERL_USE_UNSAFE_INC>
+
+There is a new environment variable recognised by the perl interpreter.
+If this variable has the value C<1> when the perl interpreter starts up,
+then C<.> will be automatically appended to C<@INC> (except under tainting).
+
+This allows you restore the old perl interpreter behaviour on a
+case-by-case basis. But note that this intended to be a temporary crutch,
+and this feature will likely be removed in some future perl version.
+It is currently set by the C<cpan> utility and C<Test::Harness> to
+ease installation of CPAN modules which have not been updated handle the
+lack of dot. Once again, don't use this unless you are sure that this
+will not reintroduce any security concerns.
+
+=item * A new mandatory warning issued by C<do>.
+
+While it is well-known that C<use> and C<require> use C<@INC> to search
+for the file to load, many people don't realise that C<do "file"> also
+searches C<@INC> if the file is a relative path. With the removal of C<.>,
+a simple C<do "file.pl"> will fail to read in and execute C<file.pl> from
+the current directory. Since this is commonly expected behaviour, a new
+mandatory warning is now issued whenever C<do> fails to load a file which
+it otherwise would have found if dot had been in C<@INC>.
+
+=back
+
+Here are some things script and module authors may need to do to make
+their software work in the new regime.
+
+=over
+
+=item * Script authors
+
+If the issue is within your own code (rather than within included
+modules), then you have two main options. Firstly, if you are confident
+that your script will only be run within a trusted directory (under which
+you expect to find trusted files and modules), then add C<.> back into the
+path; e.g.:
+
+    BEGIN {
+        my $dir = "/some/trusted/directory";
+        chdir $dir or die "Can't chdir to $dir: $!\n";
+        # safe now
+        push @INC, '.';
+    }
+    use "Foo::Bar"; # may load /some/trusted/directory/Foo/Bar.pm
+    do "config.pl"; # may load /some/trusted/directory/config.pl
+
+On the other hand, if your script is intended to be run from within
+untrusted directories (such as F</tmp>), then your script suddenly failing
+to load files may be indicative of a security issue. You most likely want
+to replace any relative paths with full paths; for example,
+
+    do ".foo_config.pl"
+
+might become
+
+    do "$ENV{HOME}/.foo_config.pl"
+
+If you are absolutely certain that you want your script to load and
+execute a file from the current directory, then use a C<./> prefix; for
+example:
+
+    do "./.foo_config.pl"
+
+=item * Installing and using CPAN modules
+
+If you install a CPAN module using an automatic tool like C<cpan>, then
+this tool will itself set the C<PERL_USE_UNSAFE_INC> environment variable
+while building and testing the module, which may be sufficient to install
+a distribution which hasn't been updated to be dot-aware.  If you want to
+install such a module manually, then you'll need to replace the
+traditional invocation:
+
+    perl Makefile.PL && make && make test && make install
+
+with something like
+
+    (export PERL_USE_UNSAFE_INC=1; \
+     perl Makefile.PL && make && make test && make install)
+
+Note that this only helps build and install an unfixed module. It's
+possible for the tests to pass (since they were run under
+C<PERL_USE_UNSAFE_INC=1>), but for the module itself to fail to perform
+correctly in production. In this case you may have to temporarily modify
+your script until such time as fixed version of the module is released.
+For example:
+
+    use Foo::Bar;
+    {
+        local @INC = (@INC, '.');
+        # assuming read_config() needs '.' in @INC
+        $config = Foo::Bar->read_config();
+    }
+
+This is only rarely expected to be necessary. Again, if doing this,
+assess the resultant risks first.
+
+=item * Module Authors
+
+If you maintain a CPAN distribution, it may need updating to run in
+a dotless environment. Although C<cpan> and other such tools will
+currently set the C<PERL_USE_UNSAFE_INC> during module build, this is
+temporary workaround for the set of modules which rely on C<.> being in
+C<@INC> for installation and testing, and this may mask deeper issues. It
+could result in a module which passes tests and installs, but which
+fails at run time.
+
+During build, test and install, it will normally be the case that any perl
+processes will be executing directly within the root directory of the
+untarred distribution, or a known subdirectory of that, such as F<t/>. It
+may well be that F<Makefile.PL> or F<t/foo.t> will attempt to include
+local modules and configuration files using their direct relative
+filenames, which will now fail.
+
+However, as described above, automatic tools like F<cpan> will (for now)
+set the C<PERL_USE_UNSAFE_INC> environment variable, which introduces
+dot during build.
+
+This makes it likely that your existing build and test code will work, but
+this may mask issues with your code which only manifest when used after
+install. It is prudent to try and run your build process with that
+variable explicitly disabled:
+    (export PERL_USE_UNSAFE_INC=0; \
+     perl Makefile.PL && make && make test && make install)
+
+This is more likely to show up any potential problems with your module's
+build process, or even with the module itself. Fixing such issues will
+ensure both that your module can again be installed manually, and that
+it will still build once the C<PERL_USE_UNSAFE_INC> crutch goes away.
+
+When fixing issues in tests due to the removal of dot from C<@INC>,
+reinsertion of dot into C<@INC> should be performed with caution, for this
+too may suppress real errors in your runtime code. You are encouraged
+wherever possible to apply the aforementioned approaches with explicit
+absolute/relative paths, or relocate your needed files into a subdirectory
+and insert that subdirectory into C<@INC> instead.
+
+If your runtime code has problems under the dotless C<@INC>, then the comments
+above on how to fix for script authors will mostly apply here too. Bear in
+mind though that it is considered bad form for a module to globally add dot to
+C<@INC>, since it introduces both a security risk and hides issues of
+accidentally requiring dot in C<@INC>, as explained above.
+
+=back
 
 =head2 "Escaped" colons and relative paths in PATH
 
@@ -1355,6 +1528,14 @@ L<C<${^ENCODING}> is no longer supported. Its use will be fatal in Perl 5.28|per
 (D deprecated) The special variable C<${^ENCODING}>, formerly used to implement
 the C<encoding> pragma, is no longer supported as of Perl 5.26.0.
 
+=item *
+
+Since C<.> is now removed from C<@INC> by default, C<do> will now trigger
+a warning recommending to fix the C<do> statement:
+
+L<do "%s" failed, '.' is no longer in @INC|perldiag/do "%s" failed, '.' is no
+longer in @INC; did you mean do ".\/%s"?>
+
 =back
 
 =head2 Changes to Existing Diagnostics
@@ -1634,7 +1815,7 @@ tests for perlbug. [perl #128020]
 
 =item *
 
-C<DEFAULT_INC_EXCLUDES_DOT> has been turned on as default.
+C<Configure -Ddefault_inc_excludes_dot> has been turned on by default.
 
 =item *
 
-- 
2.1.4

@p5pRT
Copy link
Author

p5pRT commented May 17, 2017

From @jmdh

dm.perldelta.2.dom.pod

@p5pRT
Copy link
Author

p5pRT commented May 17, 2017

From @jmdh

dm.perldelta.2.pod.diff
--- dm.perldelta.2.pod	2017-05-17 14:47:44.595991556 +0100
+++ /tmp/dm.perldelta.2.pod	2017-05-17 15:04:12.751312441 +0100
@@ -1,6 +1,6 @@
 =head1 Notice
 
-This release includes two important updates:
+This release includes two updates with widespread effects:
 
 =over 4
 
@@ -27,7 +27,7 @@
 
 The perl binary includes a default set of paths in C<@INC>. Historically
 it has also included the current directory (C<.>) as the final entry,
-unless run under tainting (C<perl -T>). While convenient, this has
+unless run with taint mode enabled (C<perl -T>). While convenient, this has
 security implications: for example, where a script attempts to load an
 optional module when its current directory is untrusted (such as F</tmp>),
 it could load and execute code from under that directory.
@@ -61,7 +61,8 @@
 and this feature will likely be removed in some future perl version.
 It is currently set by the C<cpan> utility and C<Test::Harness> to
 ease installation of CPAN modules which have not been updated handle the
-lack of dot.
+lack of dot. Once again, don't use this unless you are sure that this
+will not reintroduce any security concerns.
 
 =item * A new mandatory warning issued by C<do>.
 
@@ -144,13 +145,18 @@
         $config = Foo::Bar->read_config();
     }
 
+This is only rarely expected to be necessary. Again, if doing this,
+assess the resultant risks first.
+
 =item * Module Authors
 
 If you maintain a CPAN distribution, it may need updating to run in
 a dotless environment. Although C<cpan> and other such tools will
 currently set the C<PERL_USE_UNSAFE_INC> during module build, this is
-temporary workaround that may mask deeper issues, and could result in a
-module which passes tests and installs, but which fails at run time.
+temporary workaround for the set of modules which rely on C<.> being in
+C<@INC> for installation and testing, and this may mask deeper issues. It
+could result in a module which passes tests and installs, but which
+fails at run time.
 
 During build, test and install, it will normally be the case that any perl
 processes will be executing directly within the root directory of the
@@ -159,14 +165,14 @@
 local modules and configuration files using their direct relative
 filenames, which will now fail.
 
-However, as a temporary convenience for your users, automatic tools like 
-F<cpan> will (for now) set the C<PERL_USE_UNSAFE_INC> environment variable,
-which introduces dot during build.
+However, as described above, automatic tools like F<cpan> will (for now)
+set the C<PERL_USE_UNSAFE_INC> environment variable, which introduces
+dot during build.
 
 This makes it likely that your existing build and test code will work, but
-this may just mask issues with your code which will now only manifest
-themselves when used after install. It is prudent to try and run your
-build process with that variable explicitly disabled:
+this may mask issues with your code which only manifest when used after
+install. It is prudent to try and run your build process with that
+variable explicitly disabled:
 
     (export PERL_USE_UNSAFE_INC=0; \
      perl Makefile.PL && make && make test && make install)
@@ -176,9 +182,9 @@
 ensure both that your module can again be installed manually, and that
 it will still build once the C<PERL_USE_UNSAFE_INC> crutch goes away.
 
-When fixing issues in tests due to the removal of dot from C<@INC>, trivial
+When fixing issues in tests due to the removal of dot from C<@INC>,
 reinsertion of dot into C<@INC> should be performed with caution, for this
-too may suppress real errors in your runtime code, and you are encouraged
+too may suppress real errors in your runtime code. You are encouraged
 wherever possible to apply the aforementioned approaches with explicit
 absolute/relative paths, or relocate your needed files into a subdirectory
 and insert that subdirectory into C<@INC> instead.
@@ -219,7 +225,7 @@
 
 =item *
 
-C<Configure -Ddefault_inc_excludes_dot> has been turned on as default.
+C<Configure -Ddefault_inc_excludes_dot> has been turned on by default.
 
 =back
 

@p5pRT
Copy link
Author

p5pRT commented May 17, 2017

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented May 17, 2017

From @avar

On Wed, 17 May 2017 07​:22​:52 -0700, dom wrote​:

I reviewed the attached prose and made some editorial tweaks. Attached
is both a new file and a diff from the previous one. Overall I found
the text very clear and I recommend that they be incorporated into
perldelta ASAP. I also include the results of applying this text to
blead's perldelta.

This conflicted with my recent changes to the perldelta, but the patch looks very good aside from that. I did my best to rebase it & just pushed it to blead after a quick poll for objections on #p5p​: https://perl5.git.perl.org/perl.git/commitdiff/89853d76c827c07a8cd0cb38bd1727e96afaadef

As noted in the commit message I resolved the conflict with yours of removing the existing (buggy, see my earlier commit) prose by keeping my fixes, but with the resulting caveat noted in the commit message.

That needs fixing up, but it's better to do that on top of blead than have various changes floating around in ML threads or bugs like these.

@p5pRT p5pRT closed this as completed May 17, 2017
@p5pRT
Copy link
Author

p5pRT commented May 17, 2017

@avar - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant