From @rurban

This is a bug report for perl from rurban@​cpanel.net,
generated with the help of perlbug 1.39 running under perl 5.17.6.

Moose triggers a List​::MoreUtils​::all refcnt error in POP_MULTICALL

gdb --args /usr/local/bin/perl5.17.6d-nt-asan@​6b54ddc5 -Mblib t/metaclasses/metarole_w_metaclass_pm.t
b __asan_report_error
r
==12013== ERROR​: AddressSanitizer heap-use-after-free on address
0x7ffff45178c0 at pc 0x7ffff2c3ecb7 bp 0x7fffffff7750 sp 0x7fffffff7748
READ of size 4 at 0x7ffff45178c0 thread T0
  #0 0x7ffff2c3ecb7
  #(/usr/local/lib/perl5/site_perl/5.17.6/x86_64-linux-debug-asan@​6b54ddc5/auto/List/MoreUtils/MoreUtils.so+0x1ccb7)
  #1 0x7ffff6ffeeec
  #(/usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@​6b54ddc5/CORE/libperl.so+0xbf9eec)
  #2 0x7ffff6cb16a1
  #(/usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@​6b54ddc5/CORE/libperl.so+0x8ac6a1)
  #3 0x7ffff6672305
  #(/usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@​6b54ddc5/CORE/libperl.so+0x26d305)
  #4 0x7ffff666dd85
  #(/usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@​6b54ddc5/CORE/libperl.so+0x268d85)
  #5 0x407020 (/usr/local/bin/perl5.17.6d-nt-asan@​6b54ddc5+0x407020)
  #6 0x7ffff55c4ead (/lib/x86_64-linux-gnu/libc-2.13.so+0x1eead)
0x7ffff45178c0 is located 64 bytes inside of 1920-byte region
[0x7ffff4517880,0x7ffff4518000)
freed by thread T0 here​:
previously allocated by thread T0 here​:
==12013== ABORTING
Stats​: 29M malloced (55M for red zones) by 176798 calls
Stats​: 2M realloced by 19857 calls
Stats​: 13M freed by 122268 calls
Stats​: 0M really freed by 0 calls
Stats​: 108M (27660 full pages) mmaped in 27 calls
  mmaps by size class​: 8​:163830; 9​:8191; 10​:8190; 11​:4094; 12​:2048;
13​:2048; 14​:512; 15​:256; 16​:64; 17​:32;
  mallocs by size class​: 8​:161326; 9​:3816; 10​:4283; 11​:3281; 12​:1621;
13​:1883; 14​:443; 15​:140; 16​:3; 17​:2;
  frees by size class​: 8​:114900; 9​:2287; 10​:2282; 11​:1626; 12​:357; 13​:530;
14​:227; 15​:57; 16​:2;
  rfrees by size class​:
Stats​: malloc large​: 2 small slow​: 679
Shadow byte and word​:
  0x1ffffe8a2f18​: fd
  0x1ffffe8a2f18​: fd fd fd fd fd fd fd fd
More shadow bytes​:
  0x1ffffe8a2ef8​: fa fa fa fa fa fa fa fa
  0x1ffffe8a2f00​: fa fa fa fa fa fa fa fa
  0x1ffffe8a2f08​: fa fa fa fa fa fa fa fa
  0x1ffffe8a2f10​: fd fd fd fd fd fd fd fd
=>0x1ffffe8a2f18​: fd fd fd fd fd fd fd fd
  0x1ffffe8a2f20​: fd fd fd fd fd fd fd fd
  0x1ffffe8a2f28​: fd fd fd fd fd fd fd fd
  0x1ffffe8a2f30​: fd fd fd fd fd fd fd fd
  0x1ffffe8a2f38​: fd fd fd fd fd fd fd fd
[Inferior 1 (process 12013) exited with code 01]

(gdb) l
270 GvSV(PL_defgv) = args[i];
271 MULTICALL;
272 if (!SvTRUE(*PL_stack_sp)) {
273 POP_MULTICALL;
274 XSRETURN_NO;
275 }
276 }
=> 277 POP_MULTICALL;
278 XSRETURN_YES;
279 }

Maybe there's a FREETMPS missing in POP_MULTICALL

Flags​:
  category=library
  severity=medium

This perlbug was built using Perl 5.17.3 - Mon Jul 30 16​:28​:27 CDT 2012
It is being executed now by Perl 5.17.6 - Fri Oct 26 14​:23​:20 CDT 2012.

Site configuration information for perl 5.17.6​:

Configured by rurban at Fri Oct 26 14​:23​:20 CDT 2012.

Summary of my perl5 (revision 5 version 17 subversion 6) configuration​:
  Commit id​: 0db252d9bf45de9a19d214e875f71fa3f0597ce5
  Platform​:
  osname=linux, osvers=3.2.0-2-amd64, archname=x86_64-linux-debug-asan@​6b54ddc5
  uname='linux reini 3.2.0-2-amd64 #1 smp mon may 21 17​:45​:41 utc 2012 x86_64 gnulinux '
  config_args='-de -Dusedevel -Uversiononly -Dinstallman1dir=none -Dinstallman3dir=none -Dinstallsiteman1dir=none -Dinstallsiteman3dir=none -DEBUGGING -Doptimize=-g3 -Uuseithreads -D'cc=clang' -D'ld=clang' -A'ccflags=-faddress-sanitizer' -Aldflags=-faddress-sanitizer -Alddlflags='-shared\ -faddress-sanitizer' -Duseshrplib -Dcf_email='rurban@​cpanel.net' -Dperladmin='rurban@​cpanel.net' -Duseshrplib -Accflags=-Wno-unused-value'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='clang', ccflags ='-faddress-sanitizer -Wno-unused-value -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-g3',
  cppflags='-faddress-sanitizer -Wno-unused-value -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include'
  ccversion='', gccversion='4.2.1 Compatible Clang 3.2 (trunk)', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='clang', ldflags =' -faddress-sanitizer -L/usr/local/lib'
  libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
  libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
  libc=, so=so, useshrplib=true, libperl=libperl.so
  gnulibc_version='2.13'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@​6b54ddc5/CORE'
  cccdlflags='-fPIC', lddlflags=' -shared -faddress-sanitizer -L/usr/local/lib '

Locally applied patches​:
 

@​INC for perl 5.17.6​:
  /usr/local/lib/perl5/site_perl/5.17.6/x86_64-linux-debug-asan@​6b54ddc5
  /usr/local/lib/perl5/site_perl/5.17.6
  /usr/local/lib/perl5/5.17.6/x86_64-linux-debug-asan@​6b54ddc5
  /usr/local/lib/perl5/5.17.6
  /usr/local/lib/perl5/site_perl/5.17.5
  /usr/local/lib/perl5/site_perl/5.17.4
  /usr/local/lib/perl5/site_perl/5.17.3
  /usr/local/lib/perl5/site_perl/5.17.2
  /usr/local/lib/perl5/site_perl/5.17.1
  /usr/local/lib/perl5/site_perl/5.17.0
  /usr/local/lib/perl5/site_perl/5.17
  /usr/local/lib/perl5/site_perl/5.16.1
  /usr/local/lib/perl5/site_perl/5.16.0
  /usr/local/lib/perl5/site_perl/5.15.9
  /usr/local/lib/perl5/site_perl/5.15.8
  /usr/local/lib/perl5/site_perl/5.15.7
  /usr/local/lib/perl5/site_perl/5.15.6
  /usr/local/lib/perl5/site_perl/5.15.5
  /usr/local/lib/perl5/site_perl/5.15.4
  /usr/local/lib/perl5/site_perl/5.14.3
  /usr/local/lib/perl5/site_perl/5.14.2
  /usr/local/lib/perl5/site_perl/5.14.1
  /usr/local/lib/perl5/site_perl/5.12.4
  /usr/local/lib/perl5/site_perl/5.10.1
  /usr/local/lib/perl5/site_perl/5.8.9
  /usr/local/lib/perl5/site_perl/5.8.8
  /usr/local/lib/perl5/site_perl/5.8.7
  /usr/local/lib/perl5/site_perl/5.8.6
  /usr/local/lib/perl5/site_perl/5.8.5
  /usr/local/lib/perl5/site_perl/5.8.4
  /usr/local/lib/perl5/site_perl/5.8.3
  /usr/local/lib/perl5/site_perl/5.8.2
  /usr/local/lib/perl5/site_perl/5.8.1
  /usr/local/lib/perl5/site_perl/5.6.2
  /usr/local/lib/perl5/site_perl
  .

Environment for perl 5.17.6​:
  HOME=/home/rurban
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/home/rurban/bin​:/usr/local/bin​:/usr/bin​:/bin​:/usr/local/games​:/usr/games
  PERL_BADLANG (unset)
  SHELL=/bin/bash

From @rurban

The cpan ticket with some more analysis is at​:
  https://rt.cpan.org/Ticket/Display.html?id=77874

On 11/05/2012 11​:24 AM, perlbug-followup@​perl.org wrote​:

Greetings,

This message has been automatically generated in response to the
creation of a perl bug report regarding​:
"Moose fails in List​::MoreUtils​::all use-after-free".

There is no need to reply to this message right now. Your ticket has been
assigned an ID of [perl #115602].

You can view your ticket at
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=115602

Within the next 24 to 72 hours, your message will be posted to the Perl 5 Porters mailing list. Please be patient!

Please include the string​:

\[perl \#115602\]

in the subject line of all future correspondence about this issue. To do so,
you may reply to this message (please delete unnecessary quotes and text.)

Thank you,
perlbug-followup@​perl.org

-------------------------------------------------------------------------
Received​: (qmail 28525 invoked by uid 225); 5 Nov 2012 17​:24​:22 -0000
Received​: (qmail 28521 invoked by alias); 5 Nov 2012 17​:24​:22 -0000
Received​: from mx1.cpanel.net (HELO mx1.cpanel.net) (208.74.121.68) by la.mx.develooper.com (qpsmtpd/0.28) with ESMTP; Mon, 05 Nov 2012 09​:24​:18 -0800
Received​: from ng1.cptxoffice.net ([208.74.121.102]​:12033 helo=reini) by mx1.cpanel.net with esmtps (TLSv1​:DHE-RSA-AES128-SHA​:128) (Exim 4.80) (envelope-from <rurban@​cpanel.net>) id 1TVQP9-000469-AE for perlbug@​perl.org; Mon, 05 Nov 2012 11​:24​:07 -0600
Received​: from rurban by reini with local (Exim 4.80) (envelope-from <rurban@​cpanel.net>) id 1TVQP6-0003u5-RM for perlbug@​perl.org; Mon, 05 Nov 2012 11​:24​:05 -0600
From rurban@​cpanel.net Mon Nov 05 17​:24​:22 2012
Delivered-To​: rt-perl5@​rt.perl.org
Delivered-To​: perlbug@​perl.org
Subject​: Moose fails in List​::MoreUtils​::all use-after-free
X-Spam-Status​: No, hits=-8.2 required=8.0 tests=BAYES_00,PERLBUG_CONF,RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS,T_RP_MATCHES_RCVD
Return-Path​: <rurban@​cpanel.net>
X-Spam-Check-BY​: la.mx.develooper.com
Date​: Mon, 05 Nov 2012 11​:24​:04 -0600
X-Virus-Checked​: Checked
X-Get-Message-Sender-Via​: mx1.cpanel.net​: acl_c_relayhosts_text_entry​: -unknown-@​cpanel.net|cpanel.net
Reply-To​: rurban@​cpanel.net
Message-ID​: <5.17.6_12470_1352135295@​reini>
To​: perlbug@​perl.org
X-Antiabuse​: This header was added to track abuse, please include it with any abuse report
X-Antiabuse​: Primary Hostname - mx1.cpanel.net
X-Antiabuse​: Original Domain - perl.org
X-Antiabuse​: Originator/Caller UID/GID - [47 12] / [47 12]
X-Antiabuse​: Sender Address Domain - cpanel.net
From​: rurban@​cpanel.net
X-RT-Original-Encoding​: ascii
content-type​: text/plain; charset="utf-8"

--
Reini

Working towards a true Modern Perl.
Slim, functional, unbloated, compile-time optimizable

The RT System itself - Status changed from 'new' to 'open'

From @jkeenan

On Mon Nov 05 09​:24​:34 2012, rurban@​cpanel.net wrote​:

This is a bug report for perl from rurban@​cpanel.net,
generated with the help of perlbug 1.39 running under perl 5.17.6.

-----------------------------------------------------------------
Moose triggers a List​::MoreUtils​::all refcnt error in POP_MULTICALL

Neither Moose nor List​::MoreUtils is part of the Perl 5 core distribution.

This bug report would be better filed at https://rt.cpan.org/Dist/Display.html?Queue=List-
MoreUtils.

From @rurban

On Mon, Nov 5, 2012 at 12​:46 PM, James E Keenan via RT
<perlbug-followup@​perl.org> wrote​:

On Mon Nov 05 09​:24​:34 2012, rurban@​cpanel.net wrote​:

This is a bug report for perl from rurban@​cpanel.net,
generated with the help of perlbug 1.39 running under perl 5.17.6.

-----------------------------------------------------------------
Moose triggers a List​::MoreUtils​::all refcnt error in POP_MULTICALL

Neither Moose nor List​::MoreUtils is part of the Perl 5 core distribution.

This bug report would be better filed at https://rt.cpan.org/Dist/Display.html?Queue=List-MoreUtils.

Sure, but I tried this already in June, List-MoreUtils seems to be
unmaintained,
and due to the nature of the problem Moose and p5p are better targets.
Even I fail to properly understand the MULTICALL failure.

The original cpan ticket with some more analysis is at​:
  https://rt.cpan.org/Ticket/Display.html?id=77874
--
Reini Urban
http​://cpanel.net/ http​://www.perl-compiler.org/

From @iabyn

On Mon, Nov 05, 2012 at 09​:24​:35AM -0800, rurban@​cpanel.net wrote​:

0x7ffff45178c0 is located 64 bytes inside of 1920-byte region
[0x7ffff4517880,0x7ffff4518000)
freed by thread T0 here​:

270 GvSV(PL_defgv) = args[i];
271 MULTICALL;
272 if (!SvTRUE(*PL_stack_sp)) {
273 POP_MULTICALL;
274 XSRETURN_NO;
275 }
276 }
=> 277 POP_MULTICALL;
278 XSRETURN_YES;

#define dMULTICALL \
  SV **newsp; /* set by POPBLOCK */ \
  PERL_CONTEXT *cx; \
  ...

#define PUSH_MULTICALL_WITHDEPTH(the_cv, depth) \
  ...
  PUSHBLOCK(cx, CXt_SUB|CXp_MULTICALL, PL_stack_sp); \
  ...

#define POP_MULTICALL \
  STMT_START { \
  if (! ((CvDEPTH(multicall_cv) = cx->blk_sub.olddepth)) ) { \
  LEAVESUB(multicall_cv); \
  } \
  ...

Looks like MULTICALL expects cx to continue pointing to the current
context frame, which isn't true if the ctx stack gets extended and
ralloced in the meantime.

The issue can be reproduced with just core modules​:

  use List​::Util qw(first);
  sub rec {
  my $n = shift;
  rec($n-1) if $n;
  }
  @​b = first { rec(1000); 1 } qw(1 2 3);

$ valgrind ./perl -Ilib /tmp/p
==4313== Memcheck, a memory error detector
==4313== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==4313== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==4313== Command​: ./perl -Ilib /tmp/p
==4313==
==4313== Invalid read of size 4
==4313== at 0xB477214​: XS_List__Util_first (ListUtil.xs​:303)
==4313== by 0x5BF01C​: Perl_pp_entersub (pp_hot.c​:2770)
==4313== by 0x54AB40​: Perl_runops_debug (dump.c​:2146)
==4313== by 0x45B350​: S_run_body (perl.c​:2392)
==4313== by 0x45A429​: perl_run (perl.c​:2308)
==4313== by 0x41A8BC​: main (perlmain.c​:114)
==4313== Address 0x4cfe9f8 is 88 bytes inside a block of size 1,944 free'd
==4313== at 0x4A08A0E​: realloc (vg_replace_malloc.c​:662)
==4313== by 0x54B928​: Perl_safesysrealloc (util.c​:194)
==4313== by 0x65E574​: Perl_cxinc (scope.c​:80)
==4313== by 0x5BE1C1​: Perl_pp_entersub (pp_hot.c​:2681)
==4313== by 0x54AB40​: Perl_runops_debug (dump.c​:2146)
==4313== by 0xB476B09​: XS_List__Util_first (ListUtil.xs​:301)
==4313== by 0x5BF01C​: Perl_pp_entersub (pp_hot.c​:2770)
==4313== by 0x54AB40​: Perl_runops_debug (dump.c​:2146)
==4313== by 0x45B350​: S_run_body (perl.c​:2392)
==4313== by 0x45A429​: perl_run (perl.c​:2308)
==4313== by 0x41A8BC​: main (perlmain.c​:114)

Looks like the fix is to store the context offset rather than a pointer.
I'll do this sometime soon.

--
I've often wanted to drown my troubles, but I can't get my wife to go
swimming.

From @iabyn

On Tue, Nov 06, 2012 at 02​:39​:40PM +0000, Dave Mitchell wrote​:

Looks like MULTICALL expects cx to continue pointing to the current
context frame, which isn't true if the ctx stack gets extended and
ralloced in the meantime.

Now fixed by

commit 3d26b81
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Sun Nov 11 00​:01​:21 2012 +0000
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Sun Nov 11 00​:01​:21 2012 +0000

  make MULTICALL safe across cxstack reallocs
 
  [perl #115602]
  MUTLICALL sets a local var, cx, to point to the current context stack
  frame. When a function is called, the context stack might be realloc()ed,
  in which case cx would point to freed memory.

M cop.h
M ext/XS-APItest/t/multicall.t

--
The optimist believes that he lives in the best of all possible worlds.
As does the pessimist.

@iabyn - Status changed from 'open' to 'resolved'