From mgb@yosemite.net

The getXXXent PERL functions (e.g. getgrent) use getXXXent_r (e.g.
getgrent_r) glibc functions where available. Incorrect error handling
results in blowups at end of file. For the purposes of this discussion,
I'll focus on getgrent's use of getgrent_r.

If a long line is encountered in /etc/group, the ERANGE return code is
detected, the buffer size is doubled, and the operation is retried.
However, getgrent never clears the error code. At end of /etc/group,
ENOENT is returned. Upon seeing that the return code (ENOENT) is
non-zero, errno is checked, ERANGE is found, and the buffer is doubled
and the operation retried. The process quickly blows up. A number of
affected applications can be found by googling "panic​: realloc'. The
most serious may be WebMin, which works fine on test boxes and breaks on
production boxes (e.g. our small ISP).

Only the return code from getgrent_r should be examined. The value of
errno is undefined after a call to getgrent_r.

This bug exists in perl-5.8.3-18 and perl-5.8.5-4 but was not present in
perl-5.8.0-88.3. It would be great if it could be fixed for the
forthcoming Fedora Core 3 release.

See earlier discussion on Bugzilla​:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=137199
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=137202

From @nwc10

On Tue, Oct 26, 2004 at 10​:10​:22PM -0000, mgb @​ yosemite. net wrote​:

# New Ticket Created by mgb@​yosemite.net
# Please include the string​: [perl #32154]
# in the subject line of all future correspondence about this issue.
# <URL​: http​://rt.perl.org​:80/rt3/Ticket/Display.html?id=32154 >

The getXXXent PERL functions (e.g. getgrent) use getXXXent_r (e.g.
getgrent_r) glibc functions where available. Incorrect error handling
results in blowups at end of file. For the purposes of this discussion,
I'll focus on getgrent's use of getgrent_r.

If a long line is encountered in /etc/group, the ERANGE return code is
detected, the buffer size is doubled, and the operation is retried.
However, getgrent never clears the error code. At end of /etc/group,
ENOENT is returned. Upon seeing that the return code (ENOENT) is
non-zero, errno is checked, ERANGE is found, and the buffer is doubled
and the operation retried. The process quickly blows up. A number of
affected applications can be found by googling "panic​: realloc'. The
most serious may be WebMin, which works fine on test boxes and breaks on
production boxes (e.g. our small ISP).

Only the return code from getgrent_r should be examined. The value of
errno is undefined after a call to getgrent_r.

I had some private correspondence with Jarkko Hietaniemi, the original
author of this code. Attempting to even use the many variants of reentrant
APIs across different platforms is troublesome to say the least.

The resizing code replaces a previous compile time fixed buffer, which was
causing problems in places with very large files. As far as we're aware the
code logic does work as intended where it was tested (on OS X, I think),
so is the behaviour you describe glibc specific? If so, what is the best
way to amend perl's logic to do the right thing for glibc (and any platform
that behaves in the same way) without forcing all platforms to behave that
way?

If anyone is able and willing to supply a patch to resolve this, please
be aware that the code in reentr.c and reentr.h is generated from reentr.pl,
so ultimately any changes to the C need to be worked back into the
generation script.

This bug exists in perl-5.8.3-18 and perl-5.8.5-4 but was not present in
perl-5.8.0-88.3. It would be great if it could be fixed for the
forthcoming Fedora Core 3 release.

5.8.0 does have this bug because it had a fixed size maximum buffer, which I'm
told caused other problems. (Some systems hit its limit). Hence this
auto-resizing algorithm.

It's up to the maintainer of Fedora Core 3 what he/she wishes to put in the
release. Having chased down the source for what ships in Fedora, it seems
that there are already several patches made to the official CPAN release,
so there is nothing stopping the Fedora maintainer from patching this
issue if it's not patched in time for the next stable release of perl.

See earlier discussion on Bugzilla​:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=137199
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=137202

This notes that it is 30 seconds work to create a test /etc/group.
It fails to note one assumption. I personally do not have root access to
any sutiable linux machine with glibc, so cannot not test this as is, nor
any test proposed fix.

Nicholas Clark

The RT System itself - Status changed from 'new' to 'open'

From @rgs

Nicholas Clark wrote​:

See earlier discussion on Bugzilla​:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=137199
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=137202

This notes that it is 30 seconds work to create a test /etc/group.
It fails to note one assumption. I personally do not have root access to
any sutiable linux machine with glibc, so cannot not test this as is, nor
any test proposed fix.

I've tried to reproduce this with perl 5.8.5 and bleadperl (threaded and
non-threaded), on a linux machine with glibc 2.3.3 (glibc-2.3.3-21mdk to
be precise), and failed.

From @Tux

On Tue 02 Nov 2004 16​:21, Rafael Garcia-Suarez <rgarciasuarez@​mandrakesoft.com> wrote​:

Nicholas Clark wrote​:

See earlier discussion on Bugzilla​:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=137199
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=137202

This notes that it is 30 seconds work to create a test /etc/group.
It fails to note one assumption. I personally do not have root access to
any sutiable linux machine with glibc, so cannot not test this as is, nor
any test proposed fix.

I've tried to reproduce this with perl 5.8.5 and bleadperl (threaded and
non-threaded), on a linux machine with glibc 2.3.3 (glibc-2.3.3-21mdk to
be precise), and failed.

lt09mp# grep ^grp /etc/group
lt09mp# perl -le'for$g(200..299){print join"​:","grp$g​:!",$g,join",",map{join"",map{chr(97+int rand 26)}(1..4)}(1..500)}' >>/etc/group
lt09mp# perl bug.pl
Group​: root
Group​: bin
Group​: daemon
Group​: sys
:
:
Group​: grp297
Group​: grp298
Group​: grp299
Before endgrent
After endgrent
lt09mp# perl -v

This is perl, v5.8.5 built for i686-linux-64int
(with 1 registered patch, see perl -V for more detail)

Copyright 1987-2004, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using `man perl' or `perldoc perl'. If you have access to the
Internet, point your browser at http​://www.perl.com/, the Perl Home Page.

lt09mp# /usr/bin/perl bug.pl
Group​: root
Group​: bin
Group​: daemon
Group​: sys
:
:
Group​: grp296
Group​: grp297
Group​: grp298
Group​: grp299
Out of memory!
lt09mp# /usr/bin/perl -v

This is perl, v5.8.3 built for i586-linux-thread-multi

Copyright 1987-2003, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using `man perl' or `perldoc perl'. If you have access to the
Internet, point your browser at http​://www.perl.com/, the Perl Home Page.

lt09mp#

--
H.Merijn Brand Amsterdam Perl Mongers (http​://amsterdam.pm.org/)
using perl-5.6.1, 5.8.5, & 5.9.x, and 809 on HP-UX 10.20 & 11.00, 11i,
  AIX 4.3, AIX 5.2, SuSE 9.1, and Win2k. http​://www.cmve.net/~merijn/
http​://archives.develooper.com/daily-build@​perl.org/ perl-qa@​perl.org
send smoke reports to​: smokers-reports@​perl.org, QA​: http​://qa.perl.org

From @Tux

On Tue 02 Nov 2004 16​:55, "H.Merijn Brand" <h.m.brand@​hccnet.nl> wrote​:

I've tried to reproduce this with perl 5.8.5 and bleadperl (threaded and
non-threaded), on a linux machine with glibc 2.3.3 (glibc-2.3.3-21mdk to
be precise), and failed.

lt09mp# grep ^grp /etc/group
lt09mp# perl -le'for$g(200..299){print join"​:","grp$g​:!",$g,join",",map{join"",map{chr(97+int rand 26)}(1..4)}(1..500)}' >>/etc/group
lt09mp# perl bug.pl
Group​: root
Group​: bin
Group​: daemon
Group​: sys

Forgot system info

lt09mp# rpm -qa | grep glibc
glibc-profile-2.3.3-98
glibc-2.3.3-98
glibc-i18ndata-2.3.3-98
glibc-info-2.3.3-98
glibc-locale-2.3.3-98
glibc-html-2.3.3-98
glibc-devel-2.3.3-98
lt09mp# cat /proc/version
Linux version 2.6.5-7.111-default (geeko@​buildhost) (gcc version 3.3.3 (SuSE Linux)) #1 Wed Oct 13 15​:45​:13 UTC 2004
lt09mp#

perl-5.8.5-dor-64int is built with gcc-3.4.2

/usr/bin/perl-5.8.3-thread is built with gcc-3.3.3

--
H.Merijn Brand Amsterdam Perl Mongers (http​://amsterdam.pm.org/)
using perl-5.6.1, 5.8.5, & 5.9.x, and 809 on HP-UX 10.20 & 11.00, 11i,
  AIX 4.3, AIX 5.2, SuSE 9.1, and Win2k. http​://www.cmve.net/~merijn/
http​://archives.develooper.com/daily-build@​perl.org/ perl-qa@​perl.org
send smoke reports to​: smokers-reports@​perl.org, QA​: http​://qa.perl.org

From mgb@yosemite.net

H. Merijn Brand via RT reports the testcase did not trigger the bug in
v5.8.5 built for i686-linux-64int. Would it be possible to send me
either reentr.c or reentr.pl to check? I'm on dialup today and really
don't want to download the whole source tarball. I want to make sure
that the bug is fixed and that it does not merely fail at a larger
size. Thanks,

  mgb at yosemite.net

From @nwc10

On Tue, Nov 02, 2004 at 12​:24​:25PM -0800, Mike Bird wrote​:

H. Merijn Brand via RT reports the testcase did not trigger the bug in
v5.8.5 built for i686-linux-64int. Would it be possible to send me
either reentr.c or reentr.pl to check? I'm on dialup today and really
don't want to download the whole source tarball. I want to make sure
that the bug is fixed and that it does not merely fail at a larger
size. Thanks,

I'm not certain that it's fixed. Merijn can't reproduce it on SuSE,
Rafael can't reproduce it on Mandrake, so it might only show up on
Redhat. (Of the Linux platforms tested). If so, this would suggest to me
that the different vendors have differently patched glibcs. Pain.

Nicholas Clark

From @nwc10

On Tue, Nov 02, 2004 at 10​:51​:57PM +0000, Nicholas Clark wrote​:

On Tue, Nov 02, 2004 at 12​:24​:25PM -0800, Mike Bird wrote​:

H. Merijn Brand via RT reports the testcase did not trigger the bug in
v5.8.5 built for i686-linux-64int. Would it be possible to send me
either reentr.c or reentr.pl to check? I'm on dialup today and really

oops. Should have mailed the list to say that I'd sent this direct.

It's not widely know that the files are accessible directly by ftp​:

  ftp​://ftp.linux.activestate.com/pub/staff/gsar/APC/perl-5.8.x/

( ftp​://ftp.linux.activestate.com/pub/staff/gsar/APC/perl-current/ etc )

And I'll plug the repository browser while I'm at it​:

  http​://public.activestate.com/cgi-bin/perlbrowse

Nicholas Clark

From mgb@yosemite.net

On Tue, 2004-11-02 at 14​:56, Nicholas Clark wrote​:

ftp​://ftp.linux.activestate.com/pub/staff/gsar/APC/perl-5.8.x/
ftp​://ftp.linux.activestate.com/pub/staff/gsar/APC/perl-current/

Other than comments, there is no difference between reentr.* at the
5.8.x URL and that in the broken 5.8.3-18 in Fedora Core 2.

perl-current source is organized differently but the relevant code seems
to be the same (reentr.c line 694). It erroneously assumes that errno
is meaningful. The obvious fix is to remove the "|| (errno == ERANGE)"
clause. However, there may be other libraries out there with different
definitions of getXXXent_r.

Note that the getXXXent_r functions can be rather complicated, searching
not just /etc/group but optionally NIS too under contro of
nsswitch.conf. They may execute a large number of system calls so errno
may be clobbered many times (or not at all). Hence the need to only
check the return value.

From @smpeters

[mgb@​yosemite.net - Tue Nov 02 16​:25​:47 2004]​:

On Tue, 2004-11-02 at 14​:56, Nicholas Clark wrote​:

ftp​://ftp.linux.activestate.com/pub/staff/gsar/APC/perl-5.8.x/
ftp​://ftp.linux.activestate.com/pub/staff/gsar/APC/perl-current/

Other than comments, there is no difference between reentr.* at the
5.8.x URL and that in the broken 5.8.3-18 in Fedora Core 2.

perl-current source is organized differently but the relevant code seems
to be the same (reentr.c line 694). It erroneously assumes that errno
is meaningful. The obvious fix is to remove the "|| (errno == ERANGE)"
clause. However, there may be other libraries out there with different
definitions of getXXXent_r.

Note that the getXXXent_r functions can be rather complicated, searching
not just /etc/group but optionally NIS too under contro of
nsswitch.conf. They may execute a large number of system calls so errno
may be clobbered many times (or not at all). Hence the need to only
check the return value.

This problem has been fixed with change 25084.

@smpeters - Status changed from 'open' to 'resolved'