Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

heap-buffer-overflow in S_scan_const (toke.c:4103) #16983

Closed
p5pRT opened this issue Apr 27, 2019 · 12 comments
Closed

heap-buffer-overflow in S_scan_const (toke.c:4103) #16983

p5pRT opened this issue Apr 27, 2019 · 12 comments
Milestone

Comments

@p5pRT
Copy link

p5pRT commented Apr 27, 2019

Migrated from rt.perl.org#134067 (status was 'resolved')

Searchable as RT134067$

@p5pRT
Copy link
Author

p5pRT commented Apr 27, 2019

From @dur-randir

Created by @dur-randir

While fuzzing perl v5.29.10-23-g7c0d7520a3 built with afl and run
under libdislocator, I found the following program (also attached to
this message)

00000000 79 20 6f 5c 78 7b 31 30 30 7d c4 8c ff ff 80 80 |y o\x{100}......|
00000010 2d ff 6f 6f |-.oo|

To trigger heap-buffer-overflow write ASAN diagnostics​:

WRITE of size 1 at 0x603000000c21 thread T0
  #0 0x71a17b in S_scan_const /home/afl/afl-asan/toke.c​:4103​:8
  #1 0x6975e4 in Perl_yylex /home/afl/afl-asan/toke.c​:5096​:10
  #2 0x748e6d in Perl_yyparse /home/afl/afl-asan/perly.c​:340​:34
  #3 0x6102bc in S_parse_body /home/afl/afl-asan/perl.c​:2531​:9
  #4 0x6060d6 in perl_parse /home/afl/afl-asan/perl.c​:1822​:2
  #5 0x5382cd in main /home/afl/afl-asan/perlmain.c​:132​:18
  #6 0x7f1bea68909a in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
  #7 0x43fcc9 in _start (/home/afl/afl-asan/perl+0x43fcc9)

0x603000000c21 is located 0 bytes to the right of 17-byte region
[0x603000000c10,0x603000000c21)
allocated by thread T0 here​:
  #0 0x504aa0 in malloc (/home/afl/afl-asan/perl+0x504aa0)
  #1 0x8cfb30 in Perl_safesysmalloc /home/afl/afl-asan/util.c​:153​:21
  #2 0x9f11fe in Perl_sv_grow /home/afl/afl-asan/sv.c​:1599​:17
  #3 0xa424cc in Perl_newSV /home/afl/afl-asan/sv.c​:5653​:2
  #4 0x70c15c in S_scan_const /home/afl/afl-asan/toke.c​:2905​:14
  #5 0x6975e4 in Perl_yylex /home/afl/afl-asan/toke.c​:5096​:10
  #6 0x748e6d in Perl_yyparse /home/afl/afl-asan/perly.c​:340​:34
  #7 0x6102bc in S_parse_body /home/afl/afl-asan/perl.c​:2531​:9
  #8 0x6060d6 in perl_parse /home/afl/afl-asan/perl.c​:1822​:2
  #9 0x5382cd in main /home/afl/afl-asan/perlmain.c​:132​:18
  #10 0x7f1bea68909a in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x2409a)

This is a regression in blead, bisect points to the following commit,
and I feel this is a different issue from
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=134064

commit 7d6e74d
Author​: Karl Williamson <khw@​cpan.org>
Date​: Sat Apr 6 14​:05​:29 2019 -0600

  toke.c​: Streamline a case

  When we are parsing a constant, and the source and destination differ in
  UTF-8ness, I realized, in single stepping through the code, that it's
  simpler and more efficient to split these into two cases, rather than
  try to do one case with some conditionals in the middle.

Perl Info

Flags:
    category=core
    severity=high

Site configuration information for perl 5.29.9:

Configured by dur-randir at Wed Feb 27 14:51:01 MSK 2019.

Summary of my perl5 (revision 5 version 29 subversion 9) configuration:
  Commit id: c1e47bad34ce1d9c84ed57c9b8978bcbd5a02e98
  Platform:
    osname=darwin
    osvers=13.4.0
    archname=darwin-thread-multi-2level
    uname='darwin isengard.local 13.4.0 darwin kernel version 13.4.0:
mon jan 11 18:17:34 pst 2016; root:xnu-2422.115.15~1release_x86_64
x86_64 '
    config_args='-de -Dusedevel -DDEBUGGING -Dusethreads'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include -DPERL_USE_SAFE_PUTENV'
    optimize='-O3 -g'
    cppflags='-fno-common -DPERL_DARWIN -mmacosx-version-min=10.9
-DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -mmacosx-version-min=10.9 -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib
/usr/lib
    libs=-lpthread -lgdbm -ldbm -ldl -lm -lutil -lc
    perllibs=-lpthread -ldl -lm -lutil -lc
    libc=
    so=dylib
    useshrplib=false
    libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=bundle
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' -mmacosx-version-min=10.9 -bundle -undefined
dynamic_lookup -L/usr/local/lib -fstack-protector'



@INC for perl 5.29.9:
    lib
    /usr/local/lib/perl5/site_perl/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/site_perl/5.29.9
    /usr/local/lib/perl5/5.29.9/darwin-thread-multi-2level
    /usr/local/lib/perl5/5.29.9


Environment for perl 5.29.9:
    DYLD_LIBRARY_PATH (unset)
    HOME=/Users/dur-randir
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/texbin
    PERLBREW_HOME=/Users/dur-randir/.perlbrew
    PERLBREW_MANPATH=/Users/dur-randir/perlbrew/perls/perl-5.22.1/man
    PERLBREW_PATH=/Users/dur-randir/perlbrew/bin:/Users/dur-randir/perlbrew/perls/perl-5.22.1/bin
    PERLBREW_PERL=perl-5.22.1
    PERLBREW_ROOT=/Users/dur-randir/perlbrew
    PERLBREW_SHELLRC_VERSION=0.84
    PERLBREW_VERSION=0.84
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Apr 27, 2019

From @dur-randir

0035

@p5pRT
Copy link
Author

p5pRT commented Apr 29, 2019

From @khwilliamson

On 4/27/19 10​:28 AM, Sergey Aleynikov (via RT) wrote​:

# New Ticket Created by Sergey Aleynikov
# Please include the string​: [perl #134067]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=134067 >

This is a bug report for perl from sergey.aleynikov@​gmail.com,
generated with the help of perlbug 1.41 running under perl 5.29.9.

-----------------------------------------------------------------
[Please describe your issue here]

While fuzzing perl v5.29.10-23-g7c0d7520a3 built with afl and run
under libdislocator, I found the following program (also attached to
this message)

00000000 79 20 6f 5c 78 7b 31 30 30 7d c4 8c ff ff 80 80 |y o\x{100}......|
00000010 2d ff 6f 6f |-.oo|

This is a regression in blead, bisect points to the following commit,
and I feel this is a different issue from
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=134064

commit 7d6e74d
Author​: Karl Williamson <khw@​cpan.org>
Date​: Sat Apr 6 14​:05​:29 2019 -0600

 toke\.c&#8203;: Streamline a case

 When we are parsing a constant\, and the source and destination differ in
 UTF\-8ness\, I realized\, in single stepping through the code\, that it's
 simpler and more efficient to split these into two cases\, rather than
 try to do one case with some conditionals in the middle\.

The attached patches fix this. I think this bug has been around before
the blamed commit, but it exacerbated the problem. When parsing tr///,
you can have character ranges like a-z, \x80-\xbf, etc. The fact that
it was a range is remembered while the upper number is parsed, then
afterwards, the upper number is shifted right in the buffer and the
hyphen is inserted. This is because the hyphen is usually not needed.
If the destination buffer of the parse grows after parsing the hyphen
(in other words, while parsing the upper number), it fails to allocate
space for it, and can overflow.

The final patch stems from my realization that we could check before
outputting the terminating NUL if there is space, and grow at that
point. This should be done as a safeguard against miscalculations
earlier. Doing something like this would have prevented the failure in
this ticket.

@p5pRT
Copy link
Author

p5pRT commented Apr 29, 2019

From @khwilliamson

0004-XXX-test-PATCH-perl-134067-heap-buffer-overflow-in-l.patch
From d8a3c082fd2eb2547ec2ca363c00e5928eaf237d Mon Sep 17 00:00:00 2001
From: Karl Williamson <khw@cpan.org>
Date: Sat, 27 Apr 2019 14:04:58 -0600
Subject: [PATCH 4/5] XXX test: PATCH: [perl #134067] heap buffer overflow in
 lexing

This bug happens under tr///.  In some circumstances, a byte is inserted
in the output that wasn't in the input, and it did not check that there
was space available for this character.  The result could be a write
after the buffer end.

I suspect that this bug has been there all along, and the blamed commit
rearranged things so that it is more likely to happen.
---
 toke.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/toke.c b/toke.c
index 3d81b01bfd..8e1b542738 100644
--- a/toke.c
+++ b/toke.c
@@ -3194,11 +3194,23 @@ S_scan_const(pTHX_ char *start)
 		        && (range_min > 255 || ! convert_unicode)
 #endif
                     ) {
+                        const STRLEN off = d - SvPVX(sv);
+                        const STRLEN extra = 1 + (send - s) + 1;
+                        char *e;
+
                         /* Move the high character one byte to the right; then
                          * insert between it and the range begin, an illegal
                          * byte which serves to indicate this is a range (using
                          * a '-' would be ambiguous). */
-                        char *e = d++;
+
+                        if (off + extra > SvLEN(sv)) {
+                            STRLEN max_ptr_off = max_ptr - SvPVX(sv);
+
+                            d = off + SvGROW(sv, off + extra);
+                            max_ptr = d - off + max_ptr_off;
+                        }
+
+                        e = d++;
                         while (e-- > max_ptr) {
                             *(e + 1) = *e;
                         }
-- 
2.17.1

@p5pRT
Copy link
Author

p5pRT commented Apr 29, 2019

From @khwilliamson

0005-S_scan_const-Make-sure-room-for-NUL-in-dest.patch
From c79feaa42f42b483cc640aaa0c68025a61f2f543 Mon Sep 17 00:00:00 2001
From: Karl Williamson <khw@cpan.org>
Date: Sat, 27 Apr 2019 14:30:02 -0600
Subject: [PATCH 5/5] S_scan_const: Make sure room for NUL in dest

At the end of  constant, we add a trailing NUL.  This makes sure there's
room for it.  But the code earlier was supposed to have already mad
enough space, so its a bug if there isn't enough space.  So on DEBUGGING
builds, we panic, as we've done before.  But otherwise we can continue
on with no harm having been done.
---
 toke.c | 25 ++++++++++++++++++++++---
 1 file changed, 22 insertions(+), 3 deletions(-)

diff --git a/toke.c b/toke.c
index 8e1b542738..08251aaec9 100644
--- a/toke.c
+++ b/toke.c
@@ -4113,12 +4113,31 @@ S_scan_const(pTHX_ char *start)
 	}
     } /* while loop to process each character */
 
+    {
+        const STRLEN off = d - SvPVX(sv);
+
+        /* See if room for the terminating NUL */
+        if (UNLIKELY(off >= SvLEN(sv))) {
+
+#ifndef DEBUGGING
+
+            if (off > SvLEN(sv))
+#endif
+                Perl_croak(aTHX_ "panic: constant overflowed allocated space,"
+                        " %" UVuf " >= %" UVuf, (UV)off, (UV)SvLEN(sv));
+
+            /* Whew!  Here we don't have room for the terminating NUL, but
+             * everything else so far has fit.  It's not too late to grow
+             * to fit the NUL and continue on.  But it is a bug, as the code
+             * above was supposed to have made room for this, so under
+             * DEBUGGING builds, we panic anyway.  */
+            d = off + SvGROW(sv, off + 1);
+        }
+    }
+
     /* terminate the string and set up the sv */
     *d = '\0';
     SvCUR_set(sv, d - SvPVX_const(sv));
-    if (SvCUR(sv) >= SvLEN(sv))
-	Perl_croak(aTHX_ "panic: constant overflowed allocated space, %" UVuf
-		   " >= %" UVuf, (UV)SvCUR(sv), (UV)SvLEN(sv));
 
     SvPOK_on(sv);
     if (d_is_utf8) {
-- 
2.17.1

@p5pRT
Copy link
Author

p5pRT commented Apr 29, 2019

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented May 2, 2019

From @dur-randir

I suggest this to be added to the 5.30 blockers list.

@p5pRT
Copy link
Author

p5pRT commented May 2, 2019

From @xsawyerx

+1

On 5/2/19 1​:01 PM, Sergey Aleynikov via RT wrote​:

I suggest this to be added to the 5.30 blockers list.

---
via perlbug​: queue​: perl5 status​: open
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=134067

@p5pRT
Copy link
Author

p5pRT commented May 3, 2019

From @khwilliamson

Fixed by

commit 3fdfceb
Author​: Karl Williamson <khw@​cpan.org>
Date​: Sat Apr 27 14​:04​:58 2019 -0600

  PATCH​: [perl #134067] heap buffer overflow in lexing
 
  This bug happens under tr///. In some circumstances, a byte is inserted
  in the output that wasn't in the input, and it did not check that there
  was space available for this character. The result could be a write
  after the buffer end.
 
  I suspect that this bug has been there all along, and the blamed commit
  rearranged things so that it is more likely to happen; it depends on
  needing to malloc in just the wrong place.
--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented May 3, 2019

@khwilliamson - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented May 22, 2019

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

@p5pRT
Copy link
Author

p5pRT commented May 22, 2019

@khwilliamson - Status changed from 'pending release' to 'resolved'

@p5pRT p5pRT closed this as completed May 22, 2019
@toddr toddr added this to the 5.30.0 milestone Oct 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants