From imdb95@gmail.com

Hello,
I found this bug when fuzzing perl5.

**********Compilation and environment**********
root@​instance-2​:~/fuzz_perl# ./perl/perl -v

This is perl 5, version 31, subversion 2 (v5.31.2 (v5.31.1-6-g9649a81))
built for x86_64-linux

Copyright 1987-2019, Larry Wall

Perl may be copied only under the terms of either the Artistic License or
the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http​://www.perl.org/, the Perl Home Page.

root@​instance-2​:~/fuzz_perl# uname -a
Linux instance-2 4.15.0-1036-gcp #38~16.04.1-Ubuntu SMP Tue Jun 25 15​:30​:46
UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

root@​instance-2​:~/fuzz_perl# lsb_release -r
Release​: 16.04

Compilation​:
AFL_USE_ASAN=1 ./Configure -des -Dusedevel -DDEBUGGING -Dcc=afl-clang-fast
-Doptimize=-g && AFL_USE_ASAN=1 make

**********Reproduce**********
root@​instance-2​:~/fuzz_perl# cat test.pl
"" =~ /_(?)|_0|\h@​{$^D=rv}\x{100}()/;
root@​instance-2​:~/fuzz_perl# ./perl/perl test.pl
Assembling pattern from 3 elements
Compiling REx "_(?)|_0|\h\x{100}()"
Starting parse and generation
<_(?)|_0|\h\>...| 1| reg
  | | brnc
  | | piec
  | | atom
exact​:19596​: (op EXACT) OK 1 (len 0) (max 65).
** (19603) offset of node 1 is 0.
** (14746) size of node 1 is 0.
<(?)|_0|\h\x>...| 3| piec
  | | atom
<?)|_0|\h\x{>...| | reg
<|_0|\h\x{10>...| | inst - BRANCH
reginsert(19738)​: (op BRANCH) OK copy 2 -> 3 (max 3).
** (19745) offset of node 3 is 0.
** (19746) size of node 3 is 0.
reginsert(19738)​: (op BRANCH) OK copy 1 -> 2 (max 3).
** (19745) offset of node 2 is 0.
** (19746) size of node 2 is 0.
reginsert(19757)​: (op BRANCH) OK 1 <- 4 (max 3).
** (19764) offset of node 1 is 4.
** (19765) size of node 1 is 1.
** (12111) size of node 1 is 0.
** (12112) offset of node 1 is 0.
<_0|\h\x{100}()>| 4| brnc
reg_node​:19596​: (op BRANCH) OK 4 (len 5) (max 4).
** (19603) offset of node 4 is 5.
** (12376) size of node 4 is 1.
  | 5| piec
  | | atom
exact​:19596​: (op EXACT) OK 5 (len 5) (max 69).
** (19603) offset of node 5 is 5.
** (14746) size of node 5 is 1.
<|\h\x{100}()> | 7| tail~ BRANCH (1) -> BRANCH
<\h\x{100}()> | | brnc
reg_node​:19596​: (op BRANCH) OK 7 (len 8) (max 7).
** (19603) offset of node 7 is 8.
** (12376) size of node 7 is 1.
  | 8| piec
  | | atom
reg_node​:19596​: (op POSIXU) OK 8 (len 9) (max 8).
** (19603) offset of node 8 is 9.
** (13650) size of node 8 is 2.
<\x{100}()> | 9| piec
  | | atom
exact​:19596​: (op EXACT) OK 9 (len 10) (max 73).
** (19603) offset of node 9 is 10.
UTF8 mismatch! Converting to utf8 for resizing and compile
Need to redo parse after upgrade
Freeing REx​: "_(?)|_0|\h\x{100}()"
Starting parse and generation
<_(?)|_0|\h\>...| 1| reg
  | | brnc
  | | piec
  | | atom
exact​:19596​: (op EXACT) OK 1 (len 0) (max 65).
** (19603) offset of node 1 is 0.
** (14746) size of node 1 is 0.
<(?)|_0|\h\x>...| 3| piec
  | | atom
<?)|_0|\h\x{>...| | reg
<|_0|\h\x{10>...| | inst - BRANCH
reginsert(19738)​: (op BRANCH) OK copy 2 -> 3 (max 3).
** (19745) offset of node 3 is 0.
** (19746) size of node 3 is 0.
reginsert(19738)​: (op BRANCH) OK copy 1 -> 2 (max 3).
** (19745) offset of node 2 is 0.
** (19746) size of node 2 is 0.
reginsert(19757)​: (op BRANCH) OK 1 <- 4 (max 3).
** (19764) offset of node 1 is 4.
** (19765) size of node 1 is 1.
** (12111) size of node 1 is 0.
** (12112) offset of node 1 is 0.
<_0|\h\x{100}()>| 4| brnc
reg_node​:19596​: (op BRANCH) OK 4 (len 5) (max 4).
** (19603) offset of node 4 is 5.
** (12376) size of node 4 is 1.
  | 5| piec
  | | atom
exact​:19596​: (op EXACT) OK 5 (len 5) (max 69).
** (19603) offset of node 5 is 5.
** (14746) size of node 5 is 1.
<|\h\x{100}()> | 7| tail~ BRANCH (1) -> BRANCH
<\h\x{100}()> | | brnc
reg_node​:19596​: (op BRANCH) OK 7 (len 8) (max 7).
** (19603) offset of node 7 is 8.
** (12376) size of node 7 is 1.
  | 8| piec
  | | atom
reg_node​:19596​: (op POSIXU) OK 8 (len 9) (max 8).
** (19603) offset of node 8 is 9.
** (13650) size of node 8 is 2.
<\x{100}()> | 9| piec
  | | atom
exact​:19596​: (op EXACT) OK 9 (len 10) (max 73).
** (19603) offset of node 9 is 10.
** (14746) size of node 9 is 6.
<()> | 11| tail~ POSIXU[​:blank​:] (8) -> EXACT_ONLY8
  | | piec
  | | atom
<)> | | reg
reganode​:19596​: (op OPEN) OK 11 (len 18) (max 12).
** (19603) offset of node 11 is 18.
  | | Setting open paren #1 to 11
** (12081) size of node 11 is 1.
** (12082) offset of node 11 is 18.
  | 13| brnc
reg_node​:19596​: (op NOTHING) OK 13 (len 18) (max 13).
** (19603) offset of node 13 is 18.
  | 14| tail~ OPEN1 (11) -> NOTHING
reganode​:19596​: (op CLOSE) OK 14 (len 18) (max 15).
** (19603) offset of node 14 is 18.
  | | Setting close paren #1 to 14
** (12173) offset of node 14 is 19.
** (12174) size of node 14 is 1.
  | 16| lsbr~ tying lastbr NOTHING (13) to ender
CLOSE1 (14) offset 1
  | | tail~ NOTHING (13) -> CLOSE
<> | | tail~ EXACT_ONLY8 <\x{100}> (9) -> OPEN
  | | tail~ BRANCH (4) -> BRANCH
reg_node​:19596​: (op END) OK 16 (len 19) (max 16).
** (19603) offset of node 16 is 20.
  | | Setting close paren #0 (END) to 16
  | 17| lsbr~ tying lastbr BRANCH (7) to ender END (16)
offset 9
  | | tail~ BRANCH (7) -> END
  | | tsdy~ EXACT <_> (2) -> EXACT
  | | ~ attach to END (16) offset to 14
  | | tsdy~ EXACT <_0> (5) -> EXACT
  | | ~ attach to END (16) offset to 11
  | | tsdy~ POSIXU[​:blank​:] (8) -> END
  | | ~ EXACT_ONLY8 <\x{100}> (9) -> END
  | | ~ OPEN1 (11) -> END
  | | ~ NOTHING (13) -> END
  | | ~ CLOSE1 (14) -> END
  | | ~ attach to END (16) offset to 2
Required size 16 nodes
Got 33 bytes for offset annotations.
Offsets​: [16]
  4​:5[1] 5​:5[1] 7​:8[1] 8​:9[2] 9​:10[6] 11​:18[1] 13​:18[0] 14​:19[1]
16​:20[0]
Starting post parse optimization

Multi Top Level
RExC_seen​: REG_TOP_LEVEL_BRANCHES_SEEN
study_chunk stopparen=-1 recursed_count=1 depth=0 recursed_depth=0
scan=60c000001aac last=60c000001aec
Peep​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 0
Peep> 1​: BRANCH (4) [ SCF_DO_STCLASS_AND SCF_DO_STCLASS
SCF_WHILEM_VISITED_POS ]
Branch> 1​: BRANCH (4) [ SCF_DO_STCLASS_AND SCF_DO_STCLASS
SCF_WHILEM_VISITED_POS ]
  study_chunk stopparen=-1 recursed_count=2 depth=1 recursed_depth=0
scan=60c000001ab0 last=60c000001ab8
  Peep​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 0
  Peep> 2​: EXACT <_> (16) [ SCF_DO_STCLASS_AND SCF_DO_STCLASS
SCF_WHILEM_VISITED_POS ]
  join> 2​: EXACT <_> (16)
  pre-fin​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 0
  post-fin​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 0
Branch> 4​: BRANCH (7) [ SCF_DO_STCLASS_AND SCF_DO_STCLASS
SCF_WHILEM_VISITED_POS ]
  study_chunk stopparen=-1 recursed_count=3 depth=1 recursed_depth=0
scan=60c000001abc last=60c000001ac4
  Peep​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 0
  Peep> 5​: EXACT <_0> (16) [ SCF_DO_STCLASS_AND SCF_DO_STCLASS
SCF_WHILEM_VISITED_POS ]
  join> 5​: EXACT <_0> (16)
  pre-fin​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 0
  post-fin​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 0
Branch> 7​: BRANCH (16) [ SCF_DO_STCLASS_AND SCF_DO_STCLASS
SCF_WHILEM_VISITED_POS ]
  study_chunk stopparen=-1 recursed_count=4 depth=1 recursed_depth=0
scan=60c000001ac8 last=60c000001ae8
  Peep​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 0
  Peep> 8​: POSIXU[​:blank​:] (9) [ SCF_DO_STCLASS_AND SCF_DO_STCLASS
SCF_WHILEM_VISITED_POS ]
  Peep​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 0
  Peep> 9​: EXACT_ONLY8 <\x{100}> (11) [ SCF_WHILEM_VISITED_POS ]
  join> 9​: EXACT_ONLY8 <\x{100}> (11)
  Peep​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 0
  Peep> 11​: OPEN1 (13) [ SCF_WHILEM_VISITED_POS ]
  Peep​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 0
  Peep> 14​: CLOSE1 (16) [ SCF_WHILEM_VISITED_POS ]
  pre-fin​: Pos​:0/0 Flags​: 0x0 Whilem_c​: 0 Lcp​: 1
  post-fin​: Pos​:0/0 Flags​: 0x80 [SF_HAS_PAR ] Whilem_c​: 0 Lcp​: 1
  Looking for TRIE'able sequences. Tail node is 16​:END
  - 1​:BRANCH (1) -> 2​:EXACT <_> => 16​:END
(First==-1,Last==-1,Cur==1,tt==END,ntt==EXACT,nntt==END)
  - 4​:BRANCH (4) -> 5​:EXACT <_0> => 16​:END
(First==1,Last==-1,Cur==4,tt==EXACT,ntt==EXACT,nntt==END)
  - 7​:BRANCH (7) -> 8​:POSIXU[​:blank​:] => 9​:EXACT_ONLY8 <\x{100}>
(First==1,Last==4,Cur==7,tt==EXACT,ntt==END,nntt==EXACT)
  make_trie start==1, first==1, last==7, tail==16 depth=1
  TRIE(NATIVE)​: W​:2 C​:3 Uq​:2 Min​:1 Max​:2
  Compiling trie using table compiler
  Char : _ 0
  State+---------
  1 : 2 . ( 1)
  2 : . 3 ( 1) W 1
  3 : . . ( 0) W 2
  Alloc​: 9 Orig​: 7 elements, Final​:2. Savings of %71.43
  Statecount​:4 Lasttrans​:3
  MJD offset​:0 MJD length​:0
  Prefix State​: 1 Ofs​:0 Char='_'
  New Start State=2 Class​:
[=================================================================
==31902==ERROR​: AddressSanitizer​: heap-buffer-overflow on address
0x6020000054df at pc 0x000000448a35 bp 0x7ffd2a31e530 sp 0x7ffd2a31dcd8
READ of size 16 at 0x6020000054df thread T0
  #0 0x448a34 in __interceptor_strlen
/scratch/llvm/clang-4/xenial/final/llvm.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc​:284​:5
  #1 0xa4837d in Perl_sv_catpv_flags /root/fuzz_perl/perl/sv.c​:5609​:5
  #2 0xa4837d in Perl_sv_vcatpvfn_flags /root/fuzz_perl/perl/sv.c​:11931
  #3 0xa2ab53 in Perl_sv_vsetpvfn /root/fuzz_perl/perl/sv.c​:10987​:5
  #4 0xa2ab53 in Perl_vnewSVpvf /root/fuzz_perl/perl/sv.c​:9592
  #5 0xd58d1b in PerlIO_vprintf /root/fuzz_perl/perl/perlio.c​:5006​:10
  #6 0x7247cd in Perl_re_printf /root/fuzz_perl/perl/regcomp.c​:1041​:14
  #7 0x7dee0f in S_make_trie /root/fuzz_perl/perl/regcomp.c​:3515​:29
  #8 0x77c3a8 in S_study_chunk /root/fuzz_perl/perl/regcomp.c​:4968​:41
  #9 0x7350bf in Perl_re_op_compile /root/fuzz_perl/perl/regcomp.c​:8185​:11
  #10 0xb09f34 in Perl_pp_regcomp /root/fuzz_perl/perl/pp_ctl.c​:108​:14
  #11 0x88087c in Perl_runops_debug /root/fuzz_perl/perl/dump.c​:2537​:23
  #12 0x5e95f2 in S_run_body /root/fuzz_perl/perl/perl.c
  #13 0x5e95f2 in perl_run /root/fuzz_perl/perl/perl.c​:2639
  #14 0x50a398 in main /root/fuzz_perl/perl/perlmain.c​:127​:9
  #15 0x7f63855a682f in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
  #16 0x4368c8 in _start (/root/fuzz_perl/perl/perl+0x4368c8)

0x6020000054df is located 0 bytes to the right of 15-byte region
[0x6020000054d0,0x6020000054df)
allocated by thread T0 here​:
  #0 0x4dd5cc in malloc
/scratch/llvm/clang-4/xenial/final/llvm.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc​:66​:3
  #1 0x88660e in Perl_safesysmalloc /root/fuzz_perl/perl/util.c​:155​:21

SUMMARY​: AddressSanitizer​: heap-buffer-overflow
/scratch/llvm/clang-4/xenial/final/llvm.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc​:284​:5
in __interceptor_strlen
Shadow bytes around the buggy address​:
  0x0c047fff8a40​: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x0c047fff8a50​: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x0c047fff8a60​: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x0c047fff8a70​: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x0c047fff8a80​: fa fa fd fd fa fa fd fd fa fa 04 fa fa fa fd fd
=>0x0c047fff8a90​: fa fa fd fd fa fa 00 07 fa fa 00[07]fa fa fd fd
  0x0c047fff8aa0​: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x0c047fff8ab0​: fa fa 00 02 fa fa 00 02 fa fa fd fd fa fa fd fd
  0x0c047fff8ac0​: fa fa 00 02 fa fa fd fd fa fa fd fd fa fa fd fd
  0x0c047fff8ad0​: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x0c047fff8ae0​: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
Shadow byte legend (one shadow byte represents 8 application bytes)​:
  Addressable​: 00
  Partially addressable​: 01 02 03 04 05 06 07
  Heap left redzone​: fa
  Freed heap region​: fd
  Stack left redzone​: f1
  Stack mid redzone​: f2
  Stack right redzone​: f3
  Stack after return​: f5
  Stack use after scope​: f8
  Global redzone​: f9
  Global init order​: f6
  Poisoned by user​: f7
  Container overflow​: fc
  Array cookie​: ac
  Intra object redzone​: bb
  ASan internal​: fe
  Left alloca redzone​: ca
  Right alloca redzone​: cb
==31902==ABORTING

Please confirm the bug.

Thanks,
Manh Nguyen

From @iabyn

On Mon, Jul 08, 2019 at 03​:45​:56AM -0700, Nguyen Duc Manh wrote​:

# cat test.pl
"" =~ /_(?)|_0|\h@​{$^D=rv}\x{100}()/;

./perl/perl test.pl
....
New Start State=2 Class​:
[=================================================================
==31902==ERROR​: AddressSanitizer​: heap-buffer-overflow on address
0x6020000054df at pc 0x000000448a35 bp 0x7ffd2a31e530 sp 0x7ffd2a31dcd8
READ of size 16 at 0x6020000054df thread T0
#0 0x448a34 in __interceptor_strlen
/scratch/llvm/clang-4/xenial/final/llvm.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc​:284​:5
#1 0xa4837d in Perl_sv_catpv_flags /root/fuzz_perl/perl/sv.c​:5609​:5
#2 0xa4837d in Perl_sv_vcatpvfn_flags /root/fuzz_perl/perl/sv.c​:11931
#3 0xa2ab53 in Perl_sv_vsetpvfn /root/fuzz_perl/perl/sv.c​:10987​:5
#4 0xa2ab53 in Perl_vnewSVpvf /root/fuzz_perl/perl/sv.c​:9592
#5 0xd58d1b in PerlIO_vprintf /root/fuzz_perl/perl/perlio.c​:5006​:10
#6 0x7247cd in Perl_re_printf /root/fuzz_perl/perl/regcomp.c​:1041​:14
#7 0x7dee0f in S_make_trie /root/fuzz_perl/perl/regcomp.c​:3515​:29

I've fixed this with the commit below. It's not a security issue, as the
buffer overrun only occurs when the regex engine is printing debugging
info at high levels of debugging.

commit 1d84a25
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Tue Aug 6 14​:36​:45 2019 +0100
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Tue Aug 6 14​:40​:52 2019 +0100

  include a trailing \0 in SVs holding trie info
 
  RT #13427
 
  TRIE_STORE_REVCHAR() was creating SvPV()s with no trailing '\0'. This
  doesn't really matter given the specialised use these are put to, but
  it upset valgrind et al when perl was run with -Drv which printf("%s")'s
  the contents of the string.

--
I thought I was wrong once, but I was mistaken.

The RT System itself - Status changed from 'new' to 'open'

@iabyn - Status changed from 'open' to 'pending release'