Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5.30.0 - Unable to set supplementary group IDs #17031

Closed
p5pRT opened this issue Jun 3, 2019 · 40 comments
Closed

5.30.0 - Unable to set supplementary group IDs #17031

p5pRT opened this issue Jun 3, 2019 · 40 comments
Milestone

Comments

@p5pRT
Copy link

p5pRT commented Jun 3, 2019

Migrated from rt.perl.org#134169 (status was 'open')

Searchable as RT134169$

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2019

From manuel-perl@mausz.at

Created by manuel-perl@mausz.at

In perl 5.30 it's impossible to set supplementary group IDs as the group
id parsing fails.

This is due to the fact that Perl_grok_atoUV has been changed so endptr
constraints the input. In the current code the endptr simply does not
get reset.

As this might leave processes with extended privileges I've marked this
as critical.

See #21

Perl Info

Flags:
    category=core
    severity=critical

Site configuration information for perl 5.30.0:

Configured by Gentoo at Sun Jun  2 21:08:39 CEST 2019.

Summary of my perl5 (revision 5 version 30 subversion 0) configuration:

  Platform:
    osname=linux
    osvers=4.19.46-gentoo
    archname=x86_64-linux-thread-multi
    uname='linux root.udmedia.de 4.19.46-gentoo #1 smp mon may 27
13:08:13 cest 2019 x86_64 intel(r) xeon(r) cpu e5-2620 v4 @ 2.10ghz
genuineintel gnulinux '
    config_args='-des -Dinstallprefix=/usr -Dinstallusrbinperl=n
-Ui_xlocale -Ui_ndbm -Ui_gdbm -Di_db -Dusethreads -DDEBUGGING=-g
-Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Dnoextensions=ODBM_File
GDBM_File NDBM_File -Duseshrplib -Darchname=x86_64-linux-thread
-Dcc=x86_64-pc-linux-gnu-gcc -Doptimize=-march=nocona -O2 -pipe -g
-Dldflags=-Wl,-O1 -Wl,--as-needed -Dprefix=/usr -Dsiteprefix=/usr/local
-Dvendorprefix=/usr -Dscriptdir=/usr/bin
-Dprivlib=/usr/lib64/perl5/5.30.0
-Darchlib=/usr/lib64/perl5/5.30.0/x86_64-linux-thread-multi
-Dsitelib=/usr/local/lib64/perl5/5.30.0
-Dsitearch=/usr/local/lib64/perl5/5.30.0/x86_64-linux-thread-multi
-Dvendorlib=/usr/lib64/perl5/vendor_perl/5.30.0
-Dvendorarch=/usr/lib64/perl5/vendor_perl/5.30.0/x86_64-linux-thread-multi
-Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3
-Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3
-Dvendorman1dir=/usr/share/man/man1 -Dvendorman3dir=/usr/share/man/man3
-Dman1ext=1 -Dman3ext=3pm -Dlibperl=libperl.so.5.30.0
-Dlocincpth=/usr/include  -Dglibpth=/lib64 /usr/lib64  -Duselargefiles
-Dd_semctl_semun -Dcf_by=Gentoo -Dmyhostname=localhost
-Dperladmin=root@localhost -Ud_csh -Dsh=/bin/sh -Dtargetsh=/bin/sh
-Uusenm -Ui_xlocale -Ui_ndbm -Ui_gdbm -Di_db -Dusethreads -DDEBUGGING=-g
-Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Dnoextensions=ODBM_File
GDBM_File NDBM_File'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='x86_64-pc-linux-gnu-gcc'
    ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing
-pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    optimize='-march=nocona -O2 -pipe -g'
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe'
    ccversion=''
    gccversion='8.3.0'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='x86_64-pc-linux-gnu-gcc'
    ldflags ='-Wl,-O1 -Wl,--as-needed'
    libpth=/usr/local/lib64 /lib64 /usr/lib64 /usr/local/lib
/usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/include-fixed /usr/lib
/lib/../lib64 /usr/lib/../lib64 /lib
    libs=-ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-ldl -lm -lcrypt -lutil -lpthread -lc
    libc=libc-2.29.so
    so=so
    useshrplib=true
    libperl=libperl.so.5.30.0
    gnulibc_version='2.29'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -march=nocona -O2 -pipe -g -Wl,-O1 -Wl,--as-needed'

Locally applied patches:
    gentoo/hints_hpux - Fix hpux hints
    gentoo/aix_soname - aix gcc detection and shared library soname support
    gentoo/EUMM-RUNPATH - https://bugs.gentoo.org/105054
cpan/ExtUtils-MakeMaker: drop $PORTAGE_TMPDIR from LD_RUN_PATH
    gentoo/config_over - Remove -rpath and append LDFLAGS to lddlflags
    gentoo/opensolaris_headers - Add headers for opensolaris
    gentoo/patchlevel - List packaged patches for perl-5.30.0(#1) in
patchlevel.h
    gentoo/cleanup-paths - Cleanup PATH and shrpenv
    gentoo/enc2xs - Tweak enc2xs to follow symlinks and ignore missing
@INC directories.
    gentoo/darwin-cc-ld - https://bugs.gentoo.org/297751 darwin: Use $CC
to link
    gentoo/cpan_definstalldirs - Provide a sensible INSTALLDIRS default
for modules installed from CPAN.
    gentoo/interix - Fix interix hints
    gentoo/create_libperl_soname - https://bugs.gentoo.org/286840 Set
libperl soname
    gentoo/mod_paths - Add /etc/perl to @INC
    gentoo/EUMM_perllocalpod - cpan/ExtUtils-MakeMaker: remove targets
that generate perllocal.pod
    gentoo/drop_fstack_protector - https://bugs.gentoo.org/348557 Don't
force -fstack-protector on everyone
    gentoo/usr_local - Configure: Don't include sources in /usr/local/
for compiling perl
    gentoo/D-SHA-CFLAGS - https://bugs.gentoo.org/506818 Do not set
custom CFLAGS in cpan/Digest-SHA
    gentoo/io_socket_ip_tests - cpan/IO-Socket-IP: Disable network tests
    gentoo/tests - Fix EUMM podlocal tests
    gentoo/no-nsl-cl.patch -
    gentoo/no_porting_tests - Disable porting tests which create fun
false-failures all over travis
    gentoo/pathtools_enoent - Disable PathTools tests which fails under
sandboxing
    debian/cpan-missing-site-dirs - Fix CPAN::FirstTime defaults with
nonexisting site dirs if a parent is writable
    debian/makemaker-pasthru - Pass LD settings through to subdirectories
    fixes/memoize_storable_nstore - [rt.cpan.org #77790]
Memoize::Storable: respect 'nstore' option not respected
    fixes/podman-pipe - Better errors for man pages from standard input
    fixes/respect_umask - Respect umask during installation
    fixes/net_smtp_docs - [rt.cpan.org #36038] Document the Net::SMTP
'Port' option
    fixes/document_makemaker_ccflags - [rt.cpan.org #68613] Document
that CCFLAGS should include $Config{ccflags}
    fixes/parallel-manisort.patch - Fix parallel building


@INC for perl 5.30.0:
    /etc/perl
    /usr/local/lib64/perl5/5.30.0/x86_64-linux-thread-multi
    /usr/local/lib64/perl5/5.30.0
    /usr/lib64/perl5/vendor_perl/5.30.0/x86_64-linux-thread-multi
    /usr/lib64/perl5/vendor_perl/5.30.0
    /usr/lib64/perl5/5.30.0/x86_64-linux-thread-multi
    /usr/lib64/perl5/5.30.0


Environment for perl 5.30.0:
    LANG=en_US.UTF-8
    LANGUAGE=en_US
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=~/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2019

From manuel-perl@mausz.at

supgroups.patch
From d11fa967a3d74bc5530225a05f31f4065dfcec2f Mon Sep 17 00:00:00 2001
From: Manuel Mausz <manuel@mausz.at>
Date: Sun, 2 Jun 2019 19:02:37 +0000
Subject: [PATCH] mg.c reset endptr after use

Perl_grok_atoUV has been changed so endptr constraints the input. Thus we need to reset the endptr after every use.
---
 mg.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/mg.c b/mg.c
index f4783fb68ae..3fe72c89e16 100644
--- a/mg.c
+++ b/mg.c
@@ -3178,7 +3178,8 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
 	{
 	    const char *p = SvPV_const(sv, len);
             Groups_t *gary = NULL;
-            const char* endptr = p + len;
+            const char* p_end = p + len;
+            const char* endptr = p_end;
             UV uv;
 #ifdef _SC_NGROUPS_MAX
            int maxgrp = sysconf(_SC_NGROUPS_MAX);
@@ -3201,6 +3202,7 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
                 if (endptr == NULL)
                     break;
                 p = endptr;
+                endptr = p_end;
                 while (isSPACE(*p))
                     ++p;
                 if (!*p)

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2019

From @jkeenan

On Mon, 03 Jun 2019 12​:52​:50 GMT, manuel-perl@​mausz.at wrote​:

This is a bug report for perl from manuel-perl@​mausz.at,
generated with the help of perlbug 1.41 running under perl 5.30.0.

-----------------------------------------------------------------
[Please describe your issue here]

In perl 5.30 it's impossible to set supplementary group IDs as the group
id parsing fails.

This is due to the fact that Perl_grok_atoUV has been changed so endptr
constraints the input. In the current code the endptr simply does not
get reset.

As this might leave processes with extended privileges I've marked this
as critical.

See #21

Could you provide a brief example of how, previously, you would have expected to be able to set these supplementary grou IDs?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2019

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2019

From @khwilliamson

On 6/3/19 2​:46 PM, James E Keenan via RT wrote​:

On Mon, 03 Jun 2019 12​:52​:50 GMT, manuel-perl@​mausz.at wrote​:

This is a bug report for perl from manuel-perl@​mausz.at,
generated with the help of perlbug 1.41 running under perl 5.30.0.

-----------------------------------------------------------------
[Please describe your issue here]

In perl 5.30 it's impossible to set supplementary group IDs as the group
id parsing fails.

This is due to the fact that Perl_grok_atoUV has been changed so endptr
constraints the input. In the current code the endptr simply does not
get reset.

As this might leave processes with extended privileges I've marked this
as critical.

See #21

Could you provide a brief example of how, previously, you would have expected to be able to set these supplementary grou IDs?

Thank you very much.

Could you submit a patch against blead? We also need a test.

The github repository is only a mirror, and is supposed to not be able
to accept pull requests, but that can't be turned off, unfortunately.

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2019

From @Leont

On Mon, Jun 3, 2019 at 10​:46 PM James E Keenan via RT
<perlbug-followup@​perl.org> wrote​:

Could you provide a brief example of how, previously, you would have expected to be able to set these supplementary grou IDs?

Supplementary GIDs can be set using $), as documented in perlvar.

Leon

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2019

From manuel@mausz.at

I've now replied two times and neither message got attached to this ticket. In case they do pop up some time later please ignore them :-)

On Mon, 03 Jun 2019 13​:46​:11 -0700, jkeenan wrote​:

Could you provide a brief example of how, previously, you would have
expected to be able to set these supplementary grou IDs?

A quick example is​:

my $supgid = getgrnam("users");
print "old groups=", $), $/;
$) = "$) $supgid";
print "new groups=", $), $/;

This should add the "users" group to your current supplementary groups. On Linux you need CAP_SETGID capability (or root) for this to work.

On Mon, 03 Jun 2019 14​:40​:28 -0700, public@​khwilliamson.com wrote​:

Could you submit a patch against blead? We also need a test.

The github repository is only a mirror, and is supposed to not be able
to accept pull requests, but that can't be turned off, unfortunately.

Attached is both my original patch against HEAD and a test I quickly
pulled together. However I have no idea if all supported perl platforms
will correctly skip respectively test the supplementary groups.
I chose nobody as additional group as it should be available on all unix
like platforms.

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2019

From manuel@mausz.at

0001-perl-134169-mg.c-reset-endptr-after-use.patch
From 33608edc013d25ef0683f3ef37ca60d34210045a Mon Sep 17 00:00:00 2001
From: Manuel Mausz <manuel@mausz.at>
Date: Tue, 4 Jun 2019 00:29:09 +0200
Subject: [PATCH 1/2] (perl #134169) mg.c reset endptr after use

Perl_grok_atoUV has been changed so endptr constraints the input. Thus we need to reset the endptr after every use.
---
 mg.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/mg.c b/mg.c
index f4783fb68a..3fe72c89e1 100644
--- a/mg.c
+++ b/mg.c
@@ -3178,7 +3178,8 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
 	{
 	    const char *p = SvPV_const(sv, len);
             Groups_t *gary = NULL;
-            const char* endptr = p + len;
+            const char* p_end = p + len;
+            const char* endptr = p_end;
             UV uv;
 #ifdef _SC_NGROUPS_MAX
            int maxgrp = sysconf(_SC_NGROUPS_MAX);
@@ -3201,6 +3202,7 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
                 if (endptr == NULL)
                     break;
                 p = endptr;
+                endptr = p_end;
                 while (isSPACE(*p))
                     ++p;
                 if (!*p)
-- 
2.21.0

@p5pRT
Copy link
Author

p5pRT commented Jun 3, 2019

From manuel@mausz.at

0002-Add-test-for-perl-134169.patch
From 8ae3c50c17cbd25c745438f23d7bef0c8a2496b3 Mon Sep 17 00:00:00 2001
From: Manuel Mausz <manuel@mausz.at>
Date: Tue, 4 Jun 2019 00:29:22 +0200
Subject: [PATCH 2/2] Add test for perl #134169

---
 t/op/groups.t | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/t/op/groups.t b/t/op/groups.t
index e50c50a8c1..258e77333e 100644
--- a/t/op/groups.t
+++ b/t/op/groups.t
@@ -51,7 +51,7 @@ sub Test {
     my %basegroup = basegroups( $pwgid, $pwgnam );
     my @extracted_supplementary_groups = remove_basegroup( \ %basegroup, \ @extracted_groups );
 
-    plan 2;
+    plan 3;
 
 
     # Test: The supplementary groups in $( should match the
@@ -121,6 +121,17 @@ sub Test {
     $gid_count->{0} //= 0;
     ok 0 == $pwgid || $gid_count->{0} < 2, "groupstype should be type short, not long";
 
+    SKIP: {
+        # try to add nobody as supplementary group
+	my $root_uid = 0;
+	skip "uid!=0", 1 if $< != $root_uid and $> != $root_uid;
+	my $nobody = getgrnam("nobody")
+	  or skip "Group `nobody' does not exist", 1;
+	$) = "$) $nobody";
+	my $ok = grep { $_ == $nobody } split ' ', $);
+	ok $ok, "Group `nobody' added as supplementary group";
+    }
+
     return;
 }
 
-- 
2.21.0

@p5pRT
Copy link
Author

p5pRT commented Jun 4, 2019

From @Grinnz

For posterity, the less weird way to set supplementary group IDs is directly with setgroups() or usually better initgroups(), which unfortunately are not currently provided by POSIX.pm, but an interface is provided by the CPAN modules Unix​::Groups and Unix​::Groups​::FFI.

@p5pRT
Copy link
Author

p5pRT commented Jun 4, 2019

From manuel@mausz.at

On Mon, 03 Jun 2019 16​:26​:59 -0700, manuel@​mausz.at wrote​:

I've now replied two times and neither message got attached to this
ticket. In case they do pop up some time later please ignore them :-)

Replying to myself as I seem to have missed CCing perl5-porters. You can find both the patch and a test in the RT.

@p5pRT
Copy link
Author

p5pRT commented Jun 13, 2019

From devel@sumpfralle.de

Created by devel@sumpfralle.de

The behaviour of "$)" ($EFFECTIVE_GROUP_ID) has changed between 5.28 and 5.30​:

perl 5.28

  # perl -E 'say $); $)="104 104"; say $); say $! '
  0 0 1 2 3 4 6 10 11 26 27
  104 104

perl 5.30

  # perl -E 'say $); $)="104 104"; say $); say $! '
  0 0 1 2 3 4 6 10 11 26 27
  104 0 1 2 3 4 6 10 11 26 27
  Invalid argument

(example by "bes-internal"; see
munin-monitoring/munin#1202 (comment))

Specifying only a single number does not cause an error.
The man page (perlvar) describes that multiple values (separated by space) are
allowed. Thus the new behaviour of 5.30 seems to be unintended.

Thank you for your time!

Cheers,
Lars

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl 5.30.0:

Configured by Debian at Sat May 25 07:14:26 UTC 2019.

Summary of my perl5 (revision 5 version 30 subversion 0) configuration:
   
  Platform:
    osname=linux
    osvers=4.9.0
    archname=i686-linux-gnu-thread-multi-64int
    uname='linux localhost 4.9.0 #1 smp debian 4.9.0 i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dcc=i686-linux-gnu-gcc -Dcpp=i686-linux-gnu-cpp -Dld=i686-linux-gnu-gcc -Dccflags=-DDEBIAN -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/perl-0IhFOA/perl-5.30.0=. -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl,-z,relro -Dlddlflags=-shared -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=i686-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.30 -Darchlib=/usr/lib/i386-linux-gnu/perl/5.30 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/i386-linux-gnu/perl5/5.30 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.30.0 -Dsitearch=/usr/local/lib/i386-linux-gnu/perl/5.30.0 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs
-Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Ui_xlocale -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -dEs -Duseshrplib -Dlibperl=libperl.so.5.30.0'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='i686-linux-gnu-gcc'
    ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    optimize='-O2 -g'
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion=''
    gccversion='8.3.0'
    gccosandvers=''
    intsize=4
    longsize=4
    ptrsize=4
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=12
    longdblkind=3
    ivtype='long long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=4
    prototype=define
  Linker and Libraries:
    ld='i686-linux-gnu-gcc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/i686-linux-gnu/8/include-fixed /usr/include/i386-linux-gnu /usr/lib /lib/i386-linux-gnu /lib/../lib /usr/lib/i386-linux-gnu /usr/lib/../lib /lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=libc-2.28.so
    so=so
    useshrplib=true
    libperl=libperl.so.5.30
    gnulibc_version='2.28'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -L/usr/local/lib -fstack-protector-strong'

Locally applied patches:
    DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS default for modules installed from CPAN.
    DEBPKG:debian/db_file_ver - https://bugs.debian.org/340047 Remove overly restrictive DB_File version check.
    DEBPKG:debian/doc_info - Replace generic man(1) instructions with Debian-specific information.
    DEBPKG:debian/enc2xs_inc - https://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @INC directories.
    DEBPKG:debian/errno_ver - https://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes.
    DEBPKG:debian/libperl_embed_doc - https://bugs.debian.org/186778 Note that libperl-dev package is required for embedded linking
    DEBPKG:fixes/respect_umask - Respect umask during installation
    DEBPKG:debian/writable_site_dirs - Set umask approproately for site install directories
    DEBPKG:debian/extutils_set_libperl_path - EU:MM: set location of libperl.a under /usr/lib
    DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or perllocal.pod for perl or vendor
    DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets.
    DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor.
    DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy.
    DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to /etc/perl/Net as /usr may not be writable.
    DEBPKG:debian/perlivp - https://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local
    DEBPKG:debian/squelch-locale-warnings - https://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts
    DEBPKG:debian/patchlevel - https://bugs.debian.org/567489 List packaged patches for 5.30.0-1 in patchlevel.h
    DEBPKG:fixes/document_makemaker_ccflags - https://bugs.debian.org/628522 [rt.cpan.org #68613] Document that CCFLAGS should include $Config{ccflags}
    DEBPKG:debian/find_html2text - https://bugs.debian.org/640479 Configure CPAN::Distribution with correct name of html2text
    DEBPKG:debian/perl5db-x-terminal-emulator.patch - https://bugs.debian.org/668490 Invoke x-terminal-emulator rather than xterm in perl5db.pl
    DEBPKG:debian/cpan-missing-site-dirs - https://bugs.debian.org/688842 Fix CPAN::FirstTime defaults with nonexisting site dirs if a parent is writable
    DEBPKG:fixes/memoize_storable_nstore - [rt.cpan.org #77790] https://bugs.debian.org/587650 Memoize::Storable: respect 'nstore' option not respected
    DEBPKG:debian/makemaker-pasthru - https://bugs.debian.org/758471 Pass LD settings through to subdirectories
    DEBPKG:debian/makemaker-manext - https://bugs.debian.org/247370 Make EU::MakeMaker honour MANnEXT settings in generated manpage headers
    DEBPKG:debian/kfreebsd-softupdates - https://bugs.debian.org/796798 Work around Debian Bug#796798
    DEBPKG:fixes/autodie-scope - https://bugs.debian.org/798096 Fix a scoping issue with "no autodie" and the "system" sub
    DEBPKG:fixes/memoize-pod - [rt.cpan.org #89441] Fix POD errors in Memoize
    DEBPKG:debian/hurd-softupdates - https://bugs.debian.org/822735 Fix t/op/stat.t failures on hurd
    DEBPKG:fixes/math_complex_doc_great_circle - https://bugs.debian.org/697567 [rt.cpan.org #114104] Math::Trig: clarify definition of great_circle_midpoint
    DEBPKG:fixes/math_complex_doc_see_also - https://bugs.debian.org/697568 [rt.cpan.org #114105] Math::Trig: add missing SEE ALSO
    DEBPKG:fixes/math_complex_doc_angle_units - https://bugs.debian.org/731505 [rt.cpan.org #114106] Math::Trig: document angle units
    DEBPKG:fixes/cpan_web_link - https://bugs.debian.org/367291 CPAN: Add link to main CPAN web site
    DEBPKG:debian/hppa_op_optimize_workaround - https://bugs.debian.org/838613 Temporarily lower the optimization of op.c on hppa due to gcc-6 problems
    DEBPKG:debian/installman-utf8 - https://bugs.debian.org/840211 Generate man pages with UTF-8 characters
    DEBPKG:fixes/getopt-long-4 - https://bugs.debian.org/864544 [rt.cpan.org #122068] Fix issue #122068.
    DEBPKG:debian/hppa_opmini_optimize_workaround - https://bugs.debian.org/869122 Lower the optimization level of opmini.c on hppa
    DEBPKG:debian/sh4_op_optimize_workaround - https://bugs.debian.org/869373 Also lower the optimization level of op.c and opmini.c on sh4
    DEBPKG:debian/perldoc-pager - https://bugs.debian.org/870340 [rt.cpan.org #120229] Fix perldoc terminal escapes when sensible-pager is less
    DEBPKG:debian/prune_libs - https://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need.
    DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian
    DEBPKG:debian/configure-regen - https://bugs.debian.org/762638 Regenerate Configure et al. after probe unit changes
    DEBPKG:debian/deprecate-with-apt - https://bugs.debian.org/747628 Point users to Debian packages of deprecated core modules
    DEBPKG:debian/disable-stack-check - https://bugs.debian.org/902779 [perl #133327] Disable debugperl stack extension checks for binary compatibility with perl
    DEBPKG:fixes/eumm-usrmerge - https://bugs.debian.org/913637 Avoid mangling /bin non-perl shebangs on merged-/usr systems
    DEBPKG:debian/perlbug-editor - https://bugs.debian.org/922609 Use "editor" as the default perlbug editor, as per Debian policy


@INC for perl 5.30.0:
    /etc/perl
    /usr/local/lib/i386-linux-gnu/perl/5.30.0
    /usr/local/share/perl/5.30.0
    /usr/lib/i386-linux-gnu/perl5/5.30
    /usr/share/perl5
    /usr/lib/i386-linux-gnu/perl/5.30
    /usr/share/perl/5.30
    /usr/local/lib/site_perl
    /usr/lib/i386-linux-gnu/perl-base


Environment for perl 5.30.0:
    HOME=/root
    LANG=
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Jun 13, 2019

From @tonycoz

On Wed, 12 Jun 2019 17​:54​:36 -0700, devel@​sumpfralle.de wrote​:

The behaviour of "$)" ($EFFECTIVE_GROUP_ID) has changed between 5.28
and 5.30​:

perl 5.28

# perl -E 'say $); $)="104 104"; say $); say $! '
0 0 1 2 3 4 6 10 11 26 27
104 104

perl 5.30

# perl -E 'say $); $)="104 104"; say $); say $! '
0 0 1 2 3 4 6 10 11 26 27
104 0 1 2 3 4 6 10 11 26 27
Invalid argument

(example by "bes-internal"; see
munin-monitoring/munin#1202 (comment)
501498851)

Specifying only a single number does not cause an error.
The man page (perlvar) describes that multiple values (separated by
space) are
allowed. Thus the new behaviour of 5.30 seems to be unintended.

The attached seems to fix it for me​:

  # ./perl -e '$) = "0 0 1000"; system "id"'
  uid=0(root) gid=0(root) groups=0(root),1000(tony)

There's no tests for setting $) that I can find and I'm not sure it's practical to add one.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jun 13, 2019

From @tonycoz

0001-perl-134194-fix-parsing-supplemental-groups-from.patch
From 4fe915fb80c78fa98164942ad75f29f44400898d Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 13 Jun 2019 15:14:40 +1000
Subject: (perl #134194) fix parsing supplemental groups from $)

For example, if parsing the second number from:

  "123 456 789"

endptr would be left pointing at the first space, while we try to start
parsing at the "4", which grok_atoUV() would reject.
---
 mg.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/mg.c b/mg.c
index f4783fb68a..db831d8e86 100644
--- a/mg.c
+++ b/mg.c
@@ -3178,7 +3178,8 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
 	{
 	    const char *p = SvPV_const(sv, len);
             Groups_t *gary = NULL;
-            const char* endptr = p + len;
+            const char * const end = p + len;
+            const char* endptr = end;
             UV uv;
 #ifdef _SC_NGROUPS_MAX
            int maxgrp = sysconf(_SC_NGROUPS_MAX);
@@ -3209,6 +3210,7 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
                     Newx(gary, i + 1, Groups_t);
                 else
                     Renew(gary, i + 1, Groups_t);
+                endptr = end;
                 if (grok_atoUV(p, &uv, &endptr))
                     gary[i] = (Groups_t)uv;
                 else {
-- 
2.11.0

@p5pRT
Copy link
Author

p5pRT commented Jun 13, 2019

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jun 14, 2019

From richard.purdie@linuxfoundation.org

Created by richard.purdie@linuxfoundation.org

This is a bug report for perl from richard.purdie@​linuxfoundation.org,
generated with the help of perlbug 1.41 running under perl 5.30.0.

-----------------------------------------------------------------
We found that after upgrading from perl 5.28 to 5.30, the acl "make check"
tests started failing. The problem can be simpified down to​:

#!/usr/bin/env perl
$) = "2 2";
print $!;

Result from perl 5.28 under strace​:

setgroups(1, [2]) = 0
setresgid(-1, 2, -1) = 0

Result from perl 5.30 under strace​:

setgroups(1, [-1]) = -1 EINVAL (Invalid argument)
setresgid(-1, 2, -1) = 0

and we isolated this to this change​:

https://perl5.git.perl.org/perl.git/commitdiff/5d4a52b5c68a11bfc97c2e24806993b84a61eade

The issue is that the new function changes the endptr to the end of the
scanned number and needs to be reset to the end of the string for
each iteration of the loop.

A patch which fixes the issue for us is attached.

[Just for reference/completeness, this is a cross compiled perl built using the
Yocto Project/OpenEmbedded]

Perl Info

Flags:
    category=core
    severity=medium
    Type=Patch
    PatchStatus=HasPatch

Site configuration information for perl 5.30.0:

Summary of my perl5 (revision 5 version 30 subversion 0) configuration:
   
  Platform:
    osname=linux
    osvers=current
    archname=x86_64-linux
    uname=''
    config_args='--prefix=/usr --libdir=/usr/lib --target=x86_64-poky-linux -Duseshrplib -Dsoname=libperl.so.5 -Dvendorprefix=/usr -Darchlibexp=/usr/lib/perl5/5.30.0/x86_64-linux --keeplog --mode=target --target=x86_64-poky-linux --targetarch=x86_64-poky-linux-gnu'
    hint=default
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=undef
    use64bitall=undef
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=undef
    bincompat5005=undef
  Compiler:
    cc='x86_64-poky-linux-gcc  -m64 -march=core2 -mtune=core2 -msse3 -mfpmath=sse -fstack-protector-strong  -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security '
    ccflags =' -O2 -pipe -g -feliminate-unused-debug-types  -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    optimize='-fwrapv -fno-strict-aliasing'
    cppflags=''
    ccversion='9.1.0'
    gccversion='9.1.0'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=0
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='x86_64-poky-linux-gcc  -m64 -march=core2 -mtune=core2 -msse3 -mfpmath=sse -fstack-protector-strong  -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security '
    ldflags ='-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now'
    libpth=/lib /usr/lib /usr/local/lib
    libs=-lm -lcrypt -lpthread -lgdbm -ldb -ldl -lgdbm_compat
    perllibs=-lm -lcrypt -ldl
    libc=
    so=so
    useshrplib=true
    libperl=libperl.so.5.30.0
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC -Wno-unused-function'
    lddlflags='-shared'



@INC for perl 5.30.0:
    /usr/lib/perl5/site_perl/5.30.0/x86_64-linux
    /usr/lib/perl5/site_perl/5.30.0
    /usr/lib/perl5/vendor_perl/5.30.0/x86_64-linux
    /usr/lib/perl5/vendor_perl/5.30.0
    /usr/lib/perl5/5.30.0/x86_64-linux
    /usr/lib/perl5/5.30.0


Environment for perl 5.30.0:
    HOME=/home/root
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
    PERL_BADLANG (unset)
    SHELL=/bin/sh

@p5pRT
Copy link
Author

p5pRT commented Jun 14, 2019

From richard.purdie@linuxfoundation.org

fix-setgroups.patch
Index: perl-5.30.0/mg.c
===================================================================
--- perl-5.30.0.orig/mg.c
+++ perl-5.30.0/mg.c
@@ -3179,6 +3256,7 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
 	    const char *p = SvPV_const(sv, len);
             Groups_t *gary = NULL;
             const char* endptr = p + len;
+            const char* realend = p + len;
             UV uv;
 #ifdef _SC_NGROUPS_MAX
            int maxgrp = sysconf(_SC_NGROUPS_MAX);
@@ -3209,6 +3287,7 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
                     Newx(gary, i + 1, Groups_t);
                 else
                     Renew(gary, i + 1, Groups_t);
+                endptr = realend;
                 if (grok_atoUV(p, &uv, &endptr))
                     gary[i] = (Groups_t)uv;
                 else {

@p5pRT
Copy link
Author

p5pRT commented Jun 14, 2019

From @jkeenan

On Fri, 14 Jun 2019 12​:34​:50 GMT, richard.purdie@​linuxfoundation.org wrote​:

This is a bug report for perl from richard.purdie@​linuxfoundation.org,
generated with the help of perlbug 1.41 running under perl 5.30.0.

-----------------------------------------------------------------
We found that after upgrading from perl 5.28 to 5.30, the acl "make
check"
tests started failing. The problem can be simpified down to​:

#!/usr/bin/env perl
$) = "2 2";
print $!;

Result from perl 5.28 under strace​:

setgroups(1, [2]) = 0
setresgid(-1, 2, -1) = 0

Result from perl 5.30 under strace​:

setgroups(1, [-1]) = -1 EINVAL (Invalid argument)
setresgid(-1, 2, -1) = 0

and we isolated this to this change​:

https://perl5.git.perl.org/perl.git/commitdiff/5d4a52b5c68a11bfc97c2e24806993b84a61eade

The issue is that the new function changes the endptr to the end of
the
scanned number and needs to be reset to the end of the string for
each iteration of the loop.

A patch which fixes the issue for us is attached.

[Just for reference/completeness, this is a cross compiled perl built
using the
Yocto Project/OpenEmbedded]

For your program, I get the following results​:

#####
$ perlbrew use perl-5.30.0
[p5p] $ perl 134195-acl.pl
Operation not permitted[p5p] $ perlbrew use perl-5.28.0
[p5p] perl 134195-acl.pl
Operation not permitted[p5p] $ perlbrew use perl-5.26.2
[p5p] $ perl 134195-acl.pl
Operation not permitted
#####

So I'm unsure how you got your results.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Jun 14, 2019

From @jkeenan

134195-acl.pl

@p5pRT
Copy link
Author

p5pRT commented Jun 14, 2019

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jun 15, 2019

From @hvds

On Fri, 14 Jun 2019 15​:10​:01 -0700, jkeenan wrote​:

Operation not permitted

You need to be root, or otherwise privileged.

As root, I can reproduce it here​:
# /opt/v5.28.1/bin/perl -wle '$) = "2 2"; print $!; print $)'

2 2
# /opt/v5.30.0/bin/perl -wle '$) = "2 2"; print $!; print $)'
Invalid argument
2 0
#

Of course that makes it difficult to spot bugs like this from the testsuite. :(

Hugo

@p5pRT
Copy link
Author

p5pRT commented Jun 16, 2019

From @tonycoz

On Fri, 14 Jun 2019 19​:08​:00 -0700, hv wrote​:

On Fri, 14 Jun 2019 15​:10​:01 -0700, jkeenan wrote​:

Operation not permitted

You need to be root, or otherwise privileged.

As root, I can reproduce it here​:
# /opt/v5.28.1/bin/perl -wle '$) = "2 2"; print $!; print $)'

2 2
# /opt/v5.30.0/bin/perl -wle '$) = "2 2"; print $!; print $)'
Invalid argument
2 0
#

Of course that makes it difficult to spot bugs like this from the
testsuite. :(

I wonder if it's worth splitting out the parsing into a non-API (but still exported) function that we can test in XS​::APItest.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jun 17, 2019

From @tonycoz

On Mon, 03 Jun 2019 16​:26​:59 -0700, manuel@​mausz.at wrote​:

I've now replied two times and neither message got attached to this
ticket. In case they do pop up some time later please ignore them :-)

On Mon, 03 Jun 2019 13​:46​:11 -0700, jkeenan wrote​:

Could you provide a brief example of how, previously, you would have
expected to be able to set these supplementary grou IDs?

A quick example is​:

my $supgid = getgrnam("users");
print "old groups=", $), $/;
$) = "$) $supgid";
print "new groups=", $), $/;

This should add the "users" group to your current supplementary
groups. On Linux you need CAP_SETGID capability (or root) for this to
work.

On Mon, 03 Jun 2019 14​:40​:28 -0700, public@​khwilliamson.com wrote​:

Could you submit a patch against blead? We also need a test.

The github repository is only a mirror, and is supposed to not be
able
to accept pull requests, but that can't be turned off, unfortunately.

Attached is both my original patch against HEAD and a test I quickly
pulled together. However I have no idea if all supported perl
platforms
will correctly skip respectively test the supplementary groups.
I chose nobody as additional group as it should be available on all
unix
like platforms.

Which system did you test this on?

My Debian system doesn't have a *group* called "nobody", though it does have a user.

There is a group called "nogroup" (but that appears to be Debian specific.)

Tony

@p5pRT
Copy link
Author

p5pRT commented Jun 17, 2019

From manuel@mausz.at

On Sun, 16 Jun 2019 22​:59​:41 -0700, tonyc wrote​:

Which system did you test this on?
My Debian system doesn't have a *group* called "nobody", though it
does have a user.

Gentoo. But it looks like nobody is only optional​: http​://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/usernames.html#TBL-OPTUSERS

I'll modify the test to iterate over all groups and use the first that the current process isn't part of.

Btw these bugs are all duplicated of this​:
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=134194
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=134195

@p5pRT
Copy link
Author

p5pRT commented Jun 17, 2019

From manuel@mausz.at

On Mon, 17 Jun 2019 00​:57​:14 -0700, manuel@​mausz.at wrote​:

On Sun, 16 Jun 2019 22​:59​:41 -0700, tonyc wrote​:

Which system did you test this on?
My Debian system doesn't have a *group* called "nobody", though it
does have a user.

I'll modify the test to iterate over all groups and use the first that
the current process isn't part of.

@​tony​: Attached is the modified test case. Please give it a try.

@p5pRT
Copy link
Author

p5pRT commented Jun 17, 2019

From manuel@mausz.at

0002-Add-test-for-perl-134169.patch
From a034a095f078a50376034c7dcb56d03e1404c0ca Mon Sep 17 00:00:00 2001
From: Manuel Mausz <manuel@mausz.at>
Date: Mon, 17 Jun 2019 10:24:03 +0200
Subject: [PATCH 2/2] Add test for perl #134169

---
 t/op/groups.t | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/t/op/groups.t b/t/op/groups.t
index e50c50a8c1..7e064cc21f 100644
--- a/t/op/groups.t
+++ b/t/op/groups.t
@@ -51,7 +51,7 @@ sub Test {
     my %basegroup = basegroups( $pwgid, $pwgnam );
     my @extracted_supplementary_groups = remove_basegroup( \ %basegroup, \ @extracted_groups );
 
-    plan 2;
+    plan 3;
 
 
     # Test: The supplementary groups in $( should match the
@@ -121,6 +121,26 @@ sub Test {
     $gid_count->{0} //= 0;
     ok 0 == $pwgid || $gid_count->{0} < 2, "groupstype should be type short, not long";
 
+    SKIP: {
+        # try to add a group as supplementary group
+        my $root_uid = 0;
+        skip "uid!=0", 1 if $< != $root_uid and $> != $root_uid;
+        my @groups = split ' ', $);
+        my @sup_group;
+        setgrent;
+        while(my @ent = getgrent) {
+            next if grep { $_ == $ent[2] } @groups;
+            @sup_group = @ent;
+            last;
+        }
+        endgrent;
+        skip "No group found we could add as a supplementary group", 1
+            if (!@sup_group);
+        $) = "$) @sup_group[2]";
+        my $ok = grep { $_ == $sup_group[2] } split ' ', $);
+        ok $ok, "Group `$sup_group[0]' added as supplementary group";
+    }
+
     return;
 }
 
-- 
2.21.0

@p5pRT
Copy link
Author

p5pRT commented Jun 19, 2019

From @tonycoz

On Mon, 17 Jun 2019 01​:32​:28 -0700, manuel@​mausz.at wrote​:

On Mon, 17 Jun 2019 00​:57​:14 -0700, manuel@​mausz.at wrote​:

On Sun, 16 Jun 2019 22​:59​:41 -0700, tonyc wrote​:

Which system did you test this on?
My Debian system doesn't have a *group* called "nobody", though it
does have a user.

I'll modify the test to iterate over all groups and use the first that
the current process isn't part of.

@​tony​: Attached is the modified test case. Please give it a try.

Applied the fix as 79e302e, the test as 3121d45 and added you to AUTHORS in f83193f.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jul 11, 2019

From @tonycoz

On Sun, 16 Jun 2019 16​:32​:49 -0700, tonyc wrote​:

On Fri, 14 Jun 2019 19​:08​:00 -0700, hv wrote​:

On Fri, 14 Jun 2019 15​:10​:01 -0700, jkeenan wrote​:

Operation not permitted

You need to be root, or otherwise privileged.

As root, I can reproduce it here​:
# /opt/v5.28.1/bin/perl -wle '$) = "2 2"; print $!; print $)'

2 2
# /opt/v5.30.0/bin/perl -wle '$) = "2 2"; print $!; print $)'
Invalid argument
2 0
#

Of course that makes it difficult to spot bugs like this from the
testsuite. :(

I wonder if it's worth splitting out the parsing into a non-API (but
still exported) function that we can test in XS​::APItest.

Something like the attached.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jul 11, 2019

From @tonycoz

0001-perl-134169-make-the-code-that-splits-testable-witho.patch
From 51b6f6aa2f1efb1559977dc46343582022772a93 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 11 Jul 2019 11:03:47 +1000
Subject: (perl #134169) make the code that splits $) testable without root

---
 embed.fnc                 |  6 ++++
 embed.h                   |  3 ++
 ext/XS-APItest/APItest.pm |  2 +-
 ext/XS-APItest/APItest.xs | 20 +++++++++++
 ext/XS-APItest/t/magic.t  |  8 +++++
 mg.c                      | 88 ++++++++++++++++++++++++++---------------------
 proto.h                   |  5 +++
 7 files changed, 92 insertions(+), 40 deletions(-)

diff --git a/embed.fnc b/embed.fnc
index bfc9dca241..ac665ef3cb 100644
--- a/embed.fnc
+++ b/embed.fnc
@@ -1162,6 +1162,12 @@ ApdD	|U32	|mg_length	|NN SV* sv
 ApdT	|void	|mg_magical	|NN SV* sv
 Apd	|int	|mg_set		|NN SV* sv
 Ap	|I32	|mg_size	|NN SV* sv
+
+: exported only for testing in XS::APItest
+#ifdef HAS_SETGROUPS
+ETp	|int	|split_groups	|int maxgrp|NN Groups_t **groups|NN const char *start|NN const char *end
+#endif
+
 ApT	|void	|mini_mktime	|NN struct tm *ptm
 Axmd	|OP*	|op_lvalue	|NULLOK OP* o|I32 type
 poX	|OP*	|op_lvalue_flags|NULLOK OP* o|I32 type|U32 flags
diff --git a/embed.h b/embed.h
index f72a7eec80..489c9aad78 100644
--- a/embed.h
+++ b/embed.h
@@ -1171,6 +1171,9 @@
 #define dump_regex_sets_structures(a,b,c,d)	S_dump_regex_sets_structures(aTHX_ a,b,c,d)
 #    endif
 #  endif
+#  if defined(HAS_SETGROUPS)
+#define split_groups		Perl_split_groups
+#  endif
 #  if defined(PERL_ANY_COW)
 #define sv_setsv_cow(a,b)	Perl_sv_setsv_cow(aTHX_ a,b)
 #  endif
diff --git a/ext/XS-APItest/APItest.pm b/ext/XS-APItest/APItest.pm
index a4573b9028..ad619ea595 100644
--- a/ext/XS-APItest/APItest.pm
+++ b/ext/XS-APItest/APItest.pm
@@ -5,7 +5,7 @@ use strict;
 use warnings;
 use Carp;
 
-our $VERSION = '1.01';
+our $VERSION = '1.02';
 
 require XSLoader;
 
diff --git a/ext/XS-APItest/APItest.xs b/ext/XS-APItest/APItest.xs
index 904462e4b4..033744a849 100644
--- a/ext/XS-APItest/APItest.xs
+++ b/ext/XS-APItest/APItest.xs
@@ -4475,6 +4475,26 @@ sv_magic(SV *sv, SV *thingy)
 CODE:
     sv_magic(SvRV(sv), NULL, PERL_MAGIC_ext, (const char *)thingy, 0);
 
+void
+split_groups(const char *p, int maxgrp = 10)
+    PREINIT:
+      Groups_t *gary = NULL;
+      int i, count;
+    PPCODE:
+      count = split_groups(maxgrp, &gary, p, p+strlen(p));
+      EXTEND(SP, count);
+      for (i = 0; i < count; ++i) {
+          PUSHs(sv_2mortal(newSVuv(gary[i])));
+      }
+      Safefree(gary);
+
+UV
+INVALID_GID()
+    CODE:
+        RETVAL = ((Gid_t)-1);
+    OUTPUT:
+        RETVAL
+
 UV
 test_get_vtbl()
     PREINIT:
diff --git a/ext/XS-APItest/t/magic.t b/ext/XS-APItest/t/magic.t
index e47cd887cb..44ef0abed4 100644
--- a/ext/XS-APItest/t/magic.t
+++ b/ext/XS-APItest/t/magic.t
@@ -62,4 +62,12 @@ is $@, "", 'PERL_MAGIC_ext is permitted on read-only things';
     is($i, 0, "hash () with set magic");
 }
 
+# test the code used by $) magic
+is_deeply([ split_groups("1 2 3") ], [ 1, 2, 3 ],
+	  "simple");
+is_deeply([ split_groups("10 11 12 13", 2) ], [ 10, 11, 12 ],
+	  "hit maxgrps limit");
+is_deeply([ split_groups("x") ], [ INVALID_GID() ],
+	  "bad input");
+
 done_testing;
diff --git a/mg.c b/mg.c
index f235f0ee5a..e0e9c1572c 100644
--- a/mg.c
+++ b/mg.c
@@ -2755,6 +2755,51 @@ S_set_dollarzero(pTHX_ SV *sv)
 #endif
 }
 
+/* (hv) best guess: maybe we'll need configure probes to do a better job,
+ * but you can override it if you need to.
+ */
+#ifndef INVALID_GID
+#define INVALID_GID ((Gid_t)-1)
+#endif
+
+#ifdef HAS_SETGROUPS
+
+int
+Perl_split_groups(int maxgrp, Groups_t **groups, const char *p, const char *p_end) {
+    int i;
+    const char* endptr = p;
+    UV uv;
+    Groups_t *gary = NULL;
+
+    PERL_ARGS_ASSERT_SPLIT_GROUPS;
+
+    for (i = 0; i <= maxgrp; ++i) {
+        if (endptr == NULL)
+            break;
+        p = endptr;
+        endptr = p_end;
+        while (isSPACE(*p))
+            ++p;
+        if (!*p)
+            break;
+        if (!gary)
+            Newx(gary, i + 1, Groups_t);
+        else
+            Renew(gary, i + 1, Groups_t);
+        if (grok_atoUV(p, &uv, &endptr))
+            gary[i] = (Groups_t)uv;
+        else {
+            gary[i] = INVALID_GID;
+            endptr = NULL;
+        }
+    }
+    *groups = gary;
+
+    return i;
+}
+
+#endif
+
 int
 Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
 {
@@ -3166,12 +3211,6 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
 	}
     case ')':
 	{
-/* (hv) best guess: maybe we'll need configure probes to do a better job,
- * but you can override it if you need to.
- */
-#ifndef INVALID_GID
-#define INVALID_GID ((Gid_t)-1)
-#endif
         /* XXX $) currently silently ignores failures */
 	Gid_t new_egid;
 #ifdef HAS_SETGROUPS
@@ -3179,8 +3218,6 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
 	    const char *p = SvPV_const(sv, len);
             Groups_t *gary = NULL;
             const char* p_end = p + len;
-            const char* endptr = p_end;
-            UV uv;
 #ifdef _SC_NGROUPS_MAX
            int maxgrp = sysconf(_SC_NGROUPS_MAX);
 
@@ -3189,37 +3226,10 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
 #else
            int maxgrp = NGROUPS;
 #endif
-
-            while (isSPACE(*p))
-                ++p;
-            if (grok_atoUV(p, &uv, &endptr))
-                new_egid = (Gid_t)uv;
-            else {
-                new_egid = INVALID_GID;
-                endptr = NULL;
-            }
-            for (i = 0; i < maxgrp; ++i) {
-                if (endptr == NULL)
-                    break;
-                p = endptr;
-                endptr = p_end;
-                while (isSPACE(*p))
-                    ++p;
-                if (!*p)
-                    break;
-                if (!gary)
-                    Newx(gary, i + 1, Groups_t);
-                else
-                    Renew(gary, i + 1, Groups_t);
-                if (grok_atoUV(p, &uv, &endptr))
-                    gary[i] = (Groups_t)uv;
-                else {
-                    gary[i] = INVALID_GID;
-                    endptr = NULL;
-                }
-            }
-            if (i)
-                PERL_UNUSED_RESULT(setgroups(i, gary));
+            i = split_groups(maxgrp, &gary, p, p_end);
+            new_egid = i ? gary[0] : INVALID_GID;
+            if (i > 1)
+                PERL_UNUSED_RESULT(setgroups(i-1, gary+1));
 	    Safefree(gary);
 	}
 #else  /* HAS_SETGROUPS */
diff --git a/proto.h b/proto.h
index a708e142cd..d23340fa25 100644
--- a/proto.h
+++ b/proto.h
@@ -4533,6 +4533,11 @@ PERL_CALLCONV int	Perl_PerlProc_pipe_cloexec(pTHX_ int *pipefd)
 	assert(pipefd)
 
 #endif
+#if defined(HAS_SETGROUPS)
+PERL_CALLCONV int	Perl_split_groups(int maxgrp, Groups_t **groups, const char *start, const char *end);
+#define PERL_ARGS_ASSERT_SPLIT_GROUPS	\
+	assert(groups); assert(start); assert(end)
+#endif
 #if defined(HAS_SIGACTION) && defined(SA_SIGINFO)
 PERL_CALLCONV Signal_t	Perl_csighandler(int sig, siginfo_t *info, void *uap);
 PERL_CALLCONV Signal_t	Perl_sighandler(int sig, siginfo_t *info, void *uap);
-- 
2.11.0

@p5pRT
Copy link
Author

p5pRT commented Oct 8, 2019

From @df7cb

Fwiw, this bug is now killing PostgreSQL on Debian with perl 5.30.0-5 where we use $) to get into the ssl-cert group to read the system's SSL key​:

5.28​: $ sudo perl -e '$) = "118 118 42 108"; system "id"'
uid=0(root) gid=0(root) egid=118(postgres) Gruppen=118(postgres),42(shadow),108(ssl-cert)

5.30​: $ sudo perl -e '$) = "118 118 42 108"; system "id"'
uid=0(root) gid=0(root) egid=118(postgres) Gruppen=118(postgres),0(root)

@p5pRT
Copy link
Author

p5pRT commented Oct 8, 2019

From @atoomic

This is a bug report for perl from atoomic@​cpan.org,
generated with the help of perlbug 1.41 running under perl 5.30.0.


Setting $EGID, for example, is not working in the same way in 5.30.0 vs
5.28.0

v5.28.0> ./perl -w -e 'my $x = \$); $$x = "202 202"; warn $! if $!; print
"$$x\n"; exit( $$x eq "202 202" ? 0 : 42 )'
202 202

v5.30.0> ./perl -w -e 'my $x = \$); $$x = "202 202"; warn $! if $!; print
"$$x\n"; exit( $$x eq "202 202" ? 0 : 42 )'
Invalid argument at -e line 1.
202 0

A git bisect points to the commit '5d4a52b5 grok_atoUV​: allow non-C strings
and document'
which was introduced in v5.29.0

5d4a52b

need some investigation



Flags​:
  category=core
  severity=low


Site configuration information for perl 5.30.0​:

Configured by cPanel at Thu Oct 3 17​:32​:39 CDT 2019.

Summary of my perl5 (revision 5 version 30 subversion 0) configuration​:

  Platform​:
  osname=linux
  osvers=3.10.0-123.20.1.el7.x86_64
  archname=x86_64-linux-64int
  uname='linux rpmbuild-64-centos-7.dev.cpanel.net
3.10.0-123.20.1.el7.x86_64 #1 smp thu jan 29 18​:05​:33 utc 2015 x86_64
x86_64 x86_64 gnulinux '
  config_args='-des -Dusedevel -Darchname=x86_64-linux-64int
-Dcc=/usr/bin/gcc -Dcpp=/usr/bin/cpp -Dusemymalloc=n -DDEBUGGING=none
-Doptimize=-Os -Accflags=-m64 -Dccflags=-DPERL_DISABLE_PMC -fPIC -DPIC
-Duseshrplib -Duselargefiles=yes -Duseposix=true -Dhint=recommended
-Duseperlio=yes -Dprefix=/usr/local/cpanel/3rdparty/perl/530
-Dsiteprefix=/opt/cpanel/perl5/530 -Dsitebin=/opt/cpanel/perl5/530/bin
-Dsitelib=/opt/cpanel/perl5/530/site_lib -Dusevendorprefix=true
-Dvendorbin=/usr/local/cpanel/3rdparty/perl/530/bin
-Dvendorprefix=/usr/local/cpanel/3rdparty/perl/530/lib/perl5
-Dvendorlib=/usr/local/cpanel/3rdparty/perl/530/lib/perl5/cpanel_lib
-Dprivlib=/usr/local/cpanel/3rdparty/perl/530/lib/perl5/5.30.0
-Dman1dir=none -Dman3dir=none
-Dscriptdir=/usr/local/cpanel/3rdparty/perl/530/bin
-Dscriptdirexp=/usr/local/cpanel/3rdparty/perl/530/bin -Dsiteman1dir=none
-Dsiteman3dir=none
-Dinstallman1dir=none -Dversiononly=no -Dinstallusrbinperl=no
-Dcf_by=cPanel -Dmyhostname=localhost -Dperladmin=root@​localhost -Dcf_email=
support@​cpanel.net -DDB_File=true -Ud_dosuid -Uuserelocatableinc -Umad
-Uusethreads -Uusemultiplicity -Uusesocks -Uuselongdouble -Duse64bitint
-Uuse64bitall'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=undef
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='/usr/bin/gcc'
  ccflags ='-DPERL_DISABLE_PMC -fPIC -DPIC -m64 -fwrapv
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
  optimize='-Os'
  cppflags='-DPERL_DISABLE_PMC -fPIC -DPIC -m64 -fwrapv
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.8.2 20140120 (Red Hat 4.8.2-16)'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='/usr/bin/gcc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib
/lib64 /usr/lib64 /usr/local/lib64
  libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
-lgdbm_compat
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.17.so
  so=so
  useshrplib=true
  libperl=libperl.so
  gnulibc_version='2.17'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E
-Wl,-rpath,/usr/local/cpanel/3rdparty/perl/530/lib/perl5/5.30.0/x86_64-linux-64int/CORE'
  cccdlflags='-fPIC'
  lddlflags='-shared -Os -L/usr/local/lib -fstack-protector-strong'

Locally applied patches​:
  cPanel - disable man installs
  cPanel - cPanel INC PATH
  cPanel - Avoid importing symbols unless requested
  cPanel - Disable termcap warning when TERM is unset
  cPanel - Do not warn when close fails because the file handle is
  cPanel - COW Static support
  cPanel - Use dup2 from Cpanel​::POSIX​::Tiny
  cPanel - Disable 14fileno.t tests since Cpanel​::POSIX​::Tiny is
  cPanel - Avoid use vars when our will do in Core Perl
  cPanel - add Perl_DeclareStaticMemory
  cPanel - Disable xs handshake
  cPanel - Provide a way to clear swash invlists for B​::C
  cPanel - Switch several CPAN modules to XSLoader
  cPanel - BC Static shared memory for single malloc
  cPanel - Hard code frequent Config checks so it's not needed
  cPanel - Storable do not load Fcntl
  cPanel - Optimize File​::Find performance for backup metadata
  cPanel - Reduce Scalar​::Utils regex overhead
  cPanel - skip shadow call when euid > 0 on linux
  cPanel - Fix warning from Memoize​::Expire
  cPanel - Remove use vars from Digest​::
  cPanel - Remove launcher regex in Config.pm
  cPanel - BC extra protection in Perl_sv_vcatpvfn_flags
  cPanel - Fix Getopt​::Long version number to be 2.50 not 2.5
  cPanel - Reduce startup size of PL_strtab
  cPanel - add test for CVE-2018-12015 (RT #133250)
  cPanel - CPAN​::Config use manual install as default for root
  cPanel - Avoid a warning when prompting install_help intro
  cPanel - Avoid a warning from cpan when run with warnings


@​INC for perl 5.30.0​:
  /root/.dotfiles/perl-must-have/lib
  /root/perl5/lib/perl5//x86_64-linux-64int
  /root/perl5/lib/perl5/
  /usr/local/cpanel

/usr/local/cpanel/3rdparty/perl/530/lib/perl5/cpanel_lib/x86_64-linux-64int
  /usr/local/cpanel/3rdparty/perl/530/lib/perl5/cpanel_lib
  /usr/local/cpanel/3rdparty/perl/530/lib/perl5/5.30.0/x86_64-linux-64int
  /usr/local/cpanel/3rdparty/perl/530/lib/perl5/5.30.0
  /opt/cpanel/perl5/530/site_lib/x86_64-linux-64int
  /opt/cpanel/perl5/530/site_lib


Environment for perl 5.30.0​:
  HOME=/root
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LC_CTYPE=en_US.UTF-8
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)

PATH=/root/.dotfiles/bin​:/usr/local/cpanel/3rdparty/perl/530/bin​:/usr/local/cpanel/3rdparty/perl/528/bin​:/usr/local/cpanel/3rdparty/perl/526/bin​:/usr/local/cpanel/3rdparty/perl/524/bin​:/usr/local/cpanel/3rdparty/perl/522/bin​:/usr/local/cpanel/3rdparty/perl/514/bin​:/usr/local/cpanel/3rdparty/bin​:/root/.cargo/bin​:/usr/local/cpanel/3rdparty/lib/path-bin​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/root/bin​:/opt/cpanel/composer/bin​:/root/.dotfiles/bin​:/root/perl5/bin
  PERL5LIB=/root/.dotfiles/perl-must-have/lib​::/root/perl5/lib/perl5/
  PERL_BADLANG (unset)
  PERL_CPANM_OPT=--quiet
  SHELL=/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Oct 8, 2019

From nicolas@atoomic.org

The only one-liner you need to reproduce the issue is

perl5.30 -e '$) = "202 202"; print "$)\n"'
202 0

Removing the 's < *eptr' protection in Perl_grok_atoUV restore it to the previous behavior
I believe that the eptr is not set correctly in some calls

at first glance this is a regular assign...

╰─> perl -MO=Concise,-exec -e '$) = "202 202"'
1 <0> enter
2 <;> nextstate(main 1 -e​:1) v​:{
3 <$> const(PV "202 202") s
4 <$> gvsv(*)) s
5 <2> sassign vKS/2
6 <@​> leave[1 ref] vKP/REFC
-e syntax OK

@p5pRT
Copy link
Author

p5pRT commented Oct 8, 2019

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Oct 8, 2019

From nicolas@atoomic.org

euid set is magic, which is handled by Perl_magic_set from mg.c
we can notice multiple calls to grok_atoUV close to 'ew_egid = (Gid_t)uv;'

@p5pRT
Copy link
Author

p5pRT commented Oct 8, 2019

From nicolas@atoomic.org

I got a potential patch
going to add some tests and confirm no other locations need similar update

@p5pRT
Copy link
Author

p5pRT commented Oct 8, 2019

From nicolas@atoomic.org

0001-Fixing-EUID-reduce-privilege.patch
From 90ce72106c5e9c6aa26b90342def922f57a6abb9 Mon Sep 17 00:00:00 2001
From: Nicolas R <atoomic@cpan.org>
Date: Tue, 8 Oct 2019 12:58:06 -0500
Subject: [PATCH] Fixing EUID reduce privilege

---
 mg.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mg.c b/mg.c
index b022d63442..1f77727409 100644
--- a/mg.c
+++ b/mg.c
@@ -3171,6 +3171,7 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
 	    const char *p = SvPV_const(sv, len);
             Groups_t *gary = NULL;
             const char* endptr = p + len;
+            const char* back_endptr = endptr;
             UV uv;
 #ifdef _SC_NGROUPS_MAX
            int maxgrp = sysconf(_SC_NGROUPS_MAX);
@@ -3183,6 +3184,7 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
 
             while (isSPACE(*p))
                 ++p;
+            endptr = back_endptr; /* we know where the end is */
             if (grok_atoUV(p, &uv, &endptr))
                 new_egid = (Gid_t)uv;
             else {
@@ -3201,6 +3203,7 @@ Perl_magic_set(pTHX_ SV *sv, MAGIC *mg)
                     Newx(gary, i + 1, Groups_t);
                 else
                     Renew(gary, i + 1, Groups_t);
+                endptr = back_endptr; /* we know where the end is */
                 if (grok_atoUV(p, &uv, &endptr))
                     gary[i] = (Groups_t)uv;
                 else {
-- 
2.23.0

@p5pRT
Copy link
Author

p5pRT commented Oct 8, 2019

From nicolas@atoomic.org

Here is the candidate fix, would appreciate any review

#23

Note​: the test might need some extra love

@p5pRT
Copy link
Author

p5pRT commented Oct 8, 2019

From nicolas@atoomic.org

this seems to be a duplicate of https://rt-archive.perl.org/perl5/Ticket/Display.html?id=134169
which was fixed with a full rewrite without using the offending Perl_grok_atoUV

@p5pRT
Copy link
Author

p5pRT commented Oct 8, 2019

From nicolas@atoomic.org

note, a less intrusive patch could have been
15dbe1a

On Tue, 08 Oct 2019 04​:54​:01 -0700, cb@​df7cb.de wrote​:

Fwiw, this bug is now killing PostgreSQL on Debian with perl 5.30.0-5
where we use $) to get into the ssl-cert group to read the system's
SSL key​:

5.28​: $ sudo perl -e '$) = "118 118 42 108"; system "id"'
uid=0(root) gid=0(root) egid=118(postgres)
Gruppen=118(postgres),42(shadow),108(ssl-cert)

5.30​: $ sudo perl -e '$) = "118 118 42 108"; system "id"'
uid=0(root) gid=0(root) egid=118(postgres)
Gruppen=118(postgres),0(root)

@atoomic
Copy link
Member

atoomic commented Oct 19, 2019

this is fixed in blead via 79e302e + 3121d45
this is fixed in v5.31.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants