Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak in Perl_yylex (toke.c:7027) #15829

Closed
p5pRT opened this issue Jan 24, 2017 · 7 comments
Closed

Memory leak in Perl_yylex (toke.c:7027) #15829

p5pRT opened this issue Jan 24, 2017 · 7 comments

Comments

@p5pRT
Copy link

p5pRT commented Jan 24, 2017

Migrated from rt.perl.org#130632 (status was 'resolved')

Searchable as RT130632$

@p5pRT
Copy link
Author

p5pRT commented Jan 24, 2017

From @dur-randir

Created by @dur-randir

While fuzzing perl v5.25.8-216-gfbceb79751 built with afl and run
under libdislocator, I found the following program

s//'x' ^ -"s​:\347"/eeg

to cause a memory leak report under ASAN​:

=================================================================
==27209==ERROR​: LeakSanitizer​: detected memory leaks

Direct leak of 4 byte(s) in 1 object(s) allocated from​:
  #0 0x4ea6c8 in malloc (/home/afl/afl-asan/perl+0x4ea6c8)
  #1 0x8504be in Perl_safesysmalloc /home/afl/afl-asan/util.c​:153​:21
  #2 0x8564ab in Perl_savepvn /home/afl/afl-asan/util.c​:1177​:5
  #3 0x675843 in Perl_yylex /home/afl/afl-asan/toke.c​:7027​:23
  #4 0x6f6072 in Perl_yyparse /home/afl/afl-asan/perly.c​:340​:34
  #5 0xb0c78f in S_doeval_compile /home/afl/afl-asan/pp_ctl.c​:3432​:77
  #6 0xb09a17 in Perl_pp_entereval /home/afl/afl-asan/pp_ctl.c​:4292​:9
  #7 0x84b114 in Perl_runops_debug /home/afl/afl-asan/dump.c​:2406​:23
  #8 0x5f1b05 in S_run_body /home/afl/afl-asan/perl.c​:2528​:2
  #9 0x5f1b05 in perl_run /home/afl/afl-asan/perl.c​:2451
  #10 0x5224d2 in main /home/afl/afl-asan/perlmain.c​:123​:9
  #11 0x7efca17d02b0 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x202b0)

SUMMARY​: AddressSanitizer​: 4 byte(s) leaked in 1 allocation(s).

Leaked scalar comes from the following line

pl_yylval.pval = savepvn(PL_tokenbuf, len+1);

In this case PL_tokenbuf is "Us" and len == 2. Amount of memory leaked
scales with the size of this buffer. For example, the following
eternal loop leaks 20 bytes per iteration, so run it with caution​:

s//'x' ^ -"s666666666666666k​:\347"/eeg while (1)

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.25.9:

Configured by root at Sat Jan 14 02:25:05 MSK 2017.

Summary of my perl5 (revision 5 version 25 subversion 9) configuration:
  Commit id: cbe2fc5001aa59cdc73e04cc35e097a2ecfbeec0
  Platform:
    osname=linux
    osvers=3.16.0-4-amd64
    archname=x86_64-linux
    uname='linux dorothy 3.16.0-4-amd64 #1 smp debian 3.16.36-1+deb8u2
(2016-10-19) x86_64 gnulinux '
    config_args='-des -Dusedevel -DDEBUGGING -Dcc=afl-clang-fast
-Doptimize=-O0 -g -ggdb3'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    bincompat5005=undef
  Compiler:
    cc='afl-clang-fast'
    ccflags ='-DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O0 -g -ggdb3'
    cppflags='-DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible Clang 3.9.1 (tags/RELEASE_391/rc2)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='afl-clang-fast'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/llvm-3.9/bin/../lib/clang/3.9.1/lib
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu
/lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.24.so
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.24'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O0 -g -ggdb3 -L/usr/local/lib -fstack-protector-strong'



@INC for perl 5.25.9:
    lib
    /usr/local/lib/perl5/site_perl/5.25.9/x86_64-linux
    /usr/local/lib/perl5/site_perl/5.25.9
    /usr/local/lib/perl5/5.25.9/x86_64-linux
    /usr/local/lib/perl5/5.25.9


Environment for perl 5.25.9:
    HOME=/home/afl
    LANG=en_US.UTF-8
    LANGUAGE=en_US:en
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/afl/perlbrew/bin:/home/afl/perlbrew/perls/perl-5.22.1/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
    PERLBREW_BASHRC_VERSION=0.78
    PERLBREW_HOME=/home/afl/.perlbrew
    PERLBREW_MANPATH=/home/afl/perlbrew/perls/perl-5.22.1/man
    PERLBREW_PATH=/home/afl/perlbrew/bin:/home/afl/perlbrew/perls/perl-5.22.1/bin
    PERLBREW_PERL=perl-5.22.1
    PERLBREW_ROOT=/home/afl/perlbrew
    PERLBREW_VERSION=0.78
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Feb 7, 2017

From @tonycoz

On Tue, 24 Jan 2017 00​:19​:10 -0800, randir wrote​:

While fuzzing perl v5.25.8-216-gfbceb79751 built with afl and run
under libdislocator, I found the following program

s//'x' ^ -"s​:\347"/eeg

to cause a memory leak report under ASAN​:

=================================================================
==27209==ERROR​: LeakSanitizer​: detected memory leaks

Direct leak of 4 byte(s) in 1 object(s) allocated from​:
#0 0x4ea6c8 in malloc (/home/afl/afl-asan/perl+0x4ea6c8)
#1 0x8504be in Perl_safesysmalloc /home/afl/afl-asan/util.c​:153​:21
#2 0x8564ab in Perl_savepvn /home/afl/afl-asan/util.c​:1177​:5
#3 0x675843 in Perl_yylex /home/afl/afl-asan/toke.c​:7027​:23
#4 0x6f6072 in Perl_yyparse /home/afl/afl-asan/perly.c​:340​:34
#5 0xb0c78f in S_doeval_compile /home/afl/afl-
asan/pp_ctl.c​:3432​:77
#6 0xb09a17 in Perl_pp_entereval /home/afl/afl-
asan/pp_ctl.c​:4292​:9
#7 0x84b114 in Perl_runops_debug /home/afl/afl-asan/dump.c​:2406​:23
#8 0x5f1b05 in S_run_body /home/afl/afl-asan/perl.c​:2528​:2
#9 0x5f1b05 in perl_run /home/afl/afl-asan/perl.c​:2451
#10 0x5224d2 in main /home/afl/afl-asan/perlmain.c​:123​:9
#11 0x7efca17d02b0 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x202b0)

This normally freed by newSTATEOP(), but if the label token happens to have been pushed onto the parser shift-reduce stack and the parser aborts, it leaks.

The attached fixes it for me.

Tony

@p5pRT
Copy link
Author

p5pRT commented Feb 7, 2017

From @tonycoz

0001-perl-130632-free-label-pvs-from-the-parser-stack.patch
From 7cc5dfff56d7dfaac8a7f7cbb6c2ff96f3036eb4 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Tue, 7 Feb 2017 15:08:55 +1100
Subject: (perl #130632) free label pvs from the parser stack

For labels, Perl_yylex() generates a LABEL token, with the label name
stored in the pval slot of the yylval, allocated with savepvn().

In the normal course of parsing this is freed by Perl_newSTATEOP(), but
if parsing is aborted with the label on the parser shift-reduce stack
the memory would leak.

Clean up pval entries on the parse stack when clearing the parser
stack.
---
 perly.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/perly.c b/perly.c
index 1c018bb..43cc955 100644
--- a/perly.c
+++ b/perly.c
@@ -232,6 +232,10 @@ S_clear_yystack(pTHX_  const yy_parser *parser)
 	    YYDPRINTF ((Perl_debug_log, "(freeing op)\n"));
 	    op_free(ps->val.opval);
 	}
+        else if (yy_type_tab[yystos[ps->state]] == toketype_pval
+                 && ps->val.pval) {
+            Safefree(ps->val.pval);
+        }
 	SvREFCNT_dec(ps->compcv);
 	ps--;
     }
-- 
2.1.4

@p5pRT
Copy link
Author

p5pRT commented Feb 7, 2017

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jun 12, 2019

From @tonycoz

On Tue, 24 Jan 2017 00​:19​:10 -0800, randir wrote​:

While fuzzing perl v5.25.8-216-gfbceb79751 built with afl and run
under libdislocator, I found the following program

s//'x' ^ -"s​:\347"/eeg

to cause a memory leak report under ASAN​:

=================================================================
==27209==ERROR​: LeakSanitizer​: detected memory leaks

Direct leak of 4 byte(s) in 1 object(s) allocated from​:
#0 0x4ea6c8 in malloc (/home/afl/afl-asan/perl+0x4ea6c8)
#1 0x8504be in Perl_safesysmalloc /home/afl/afl-asan/util.c​:153​:21
#2 0x8564ab in Perl_savepvn /home/afl/afl-asan/util.c​:1177​:5
#3 0x675843 in Perl_yylex /home/afl/afl-asan/toke.c​:7027​:23
#4 0x6f6072 in Perl_yyparse /home/afl/afl-asan/perly.c​:340​:34
#5 0xb0c78f in S_doeval_compile /home/afl/afl-
asan/pp_ctl.c​:3432​:77
#6 0xb09a17 in Perl_pp_entereval /home/afl/afl-
asan/pp_ctl.c​:4292​:9
#7 0x84b114 in Perl_runops_debug /home/afl/afl-asan/dump.c​:2406​:23
#8 0x5f1b05 in S_run_body /home/afl/afl-asan/perl.c​:2528​:2
#9 0x5f1b05 in perl_run /home/afl/afl-asan/perl.c​:2451
#10 0x5224d2 in main /home/afl/afl-asan/perlmain.c​:123​:9
#11 0x7efca17d02b0 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x202b0)

SUMMARY​: AddressSanitizer​: 4 byte(s) leaked in 1 allocation(s).

Leaked scalar comes from the following line

pl_yylval.pval = savepvn(PL_tokenbuf, len+1);

In this case PL_tokenbuf is "Us" and len == 2. Amount of memory leaked
scales with the size of this buffer. For example, the following
eternal loop leaks 20 bytes per iteration, so run it with caution​:

s//'x' ^ -"s666666666666666k​:\347"/eeg while (1)

This looks like it's fixed, I expect by​:

commit 0171920
Author​: David Mitchell <davem@​iabyn.com>
Date​: Fri Apr 5 12​:49​:50 2019 +0100

  parser​: change LABEL type from pval to opval
 
  The items pushed onto the parser stack can be one of several types​:
  ival, opval, pval etc. The only remaining use of pval is when a "label​:"
  is encountered.
 
  When an error occurs during parsing, ops on the parse stack get
  automatically reaped these days as part of the OP slab mechanism;
  but bare strings (pvals) still leak.
 
  Convert this one remaining pval into an opval, making the toker return
  an OP_CONST with an SV holding the label.
 
  Since newSTATEOP() still expects a raw string for the label, the parser
  just grabs the value returned by the toker and makes a copy of the
  string from it, then immediately frees the OP_CONST and its associated
  SV.
 
  The leak was showing up in ext/XS-APItest/t/stmtasexpr.t, which expects
  to parse a statement where labels are banned.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jun 17, 2019

From @tonycoz

On Tue, 11 Jun 2019 23​:21​:07 -0700, tonyc wrote​:

On Tue, 24 Jan 2017 00​:19​:10 -0800, randir wrote​:

While fuzzing perl v5.25.8-216-gfbceb79751 built with afl and run
under libdislocator, I found the following program

s//'x' ^ -"s​:\347"/eeg

to cause a memory leak report under ASAN​:

=================================================================
==27209==ERROR​: LeakSanitizer​: detected memory leaks

Direct leak of 4 byte(s) in 1 object(s) allocated from​:
#0 0x4ea6c8 in malloc (/home/afl/afl-asan/perl+0x4ea6c8)
#1 0x8504be in Perl_safesysmalloc /home/afl/afl-asan/util.c​:153​:21
#2 0x8564ab in Perl_savepvn /home/afl/afl-asan/util.c​:1177​:5
#3 0x675843 in Perl_yylex /home/afl/afl-asan/toke.c​:7027​:23
#4 0x6f6072 in Perl_yyparse /home/afl/afl-asan/perly.c​:340​:34
#5 0xb0c78f in S_doeval_compile /home/afl/afl-
asan/pp_ctl.c​:3432​:77
#6 0xb09a17 in Perl_pp_entereval /home/afl/afl-
asan/pp_ctl.c​:4292​:9
#7 0x84b114 in Perl_runops_debug /home/afl/afl-asan/dump.c​:2406​:23
#8 0x5f1b05 in S_run_body /home/afl/afl-asan/perl.c​:2528​:2
#9 0x5f1b05 in perl_run /home/afl/afl-asan/perl.c​:2451
#10 0x5224d2 in main /home/afl/afl-asan/perlmain.c​:123​:9
#11 0x7efca17d02b0 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x202b0)

SUMMARY​: AddressSanitizer​: 4 byte(s) leaked in 1 allocation(s).

Leaked scalar comes from the following line

pl_yylval.pval = savepvn(PL_tokenbuf, len+1);

In this case PL_tokenbuf is "Us" and len == 2. Amount of memory leaked
scales with the size of this buffer. For example, the following
eternal loop leaks 20 bytes per iteration, so run it with caution​:

s//'x' ^ -"s666666666666666k​:\347"/eeg while (1)

This looks like it's fixed, I expect by​:

commit 0171920
Author​: David Mitchell <davem@​iabyn.com>
Date​: Fri Apr 5 12​:49​:50 2019 +0100

parser&#8203;: change LABEL type from pval to opval

And so closing.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jun 17, 2019

@tonycoz - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant