Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory corruption with heavy module loading in threads #9008

Closed
p5pRT opened this issue Aug 30, 2007 · 20 comments
Closed

Memory corruption with heavy module loading in threads #9008

p5pRT opened this issue Aug 30, 2007 · 20 comments

Comments

@p5pRT
Copy link

p5pRT commented Aug 30, 2007

Migrated from rt.perl.org#45053 (status was 'resolved')

Searchable as RT45053$

@p5pRT
Copy link
Author

p5pRT commented Aug 30, 2007

From @jdhedden

This is a bug report for perl from jdhedden@​cpan.org,
generated with the help of perlbug 1.36 running under perl 5.9.5.

There seems to be a memory corruption problem that manifests
itself when several threads try to load a whole bunch of
modules at the same time.

The attached script readily evokes this bug. It uses
LWP​::Simple and threads to load web pages (the Google home
page). When run without arguments, it crashes more than 80%
of the time. The cause revolves around what happens when
LWP​::Simple's get() command is executed.

Before the get() call, the following modules are loaded (as
shown via Data​::Dumper on %INC)​:

'Carp.pm' => '/usr/lib/perl5/5.9.5/Carp.pm',
'Config.pm' => '/usr/lib/perl5/5.9.5/cygwin/Config.pm',
'Data/Dumper.pm' => '/usr/lib/perl5/5.9.5/cygwin/Data/Dumper.pm'
'Exporter.pm' => '/usr/lib/perl5/5.9.5/Exporter.pm',
'Exporter/Heavy.pm' => '/usr/lib/perl5/5.9.5/Exporter/Heavy.pm',
'HTTP/Status.pm' => '/usr/lib/perl5/5.9.5/HTTP/Status.pm',
'LWP/Simple.pm' => '/usr/lib/perl5/5.9.5/LWP/Simple.pm',
'XSLoader.pm' => '/usr/lib/perl5/5.9.5/cygwin/XSLoader.pm',
'bytes.pm' => '/usr/lib/perl5/5.9.5/bytes.pm',
'overload.pm' => '/usr/lib/perl5/5.9.5/overload.pm',
'strict.pm' => '/usr/lib/perl5/5.9.5/strict.pm',
'threads.pm' => '/usr/lib/perl5/5.9.5/cygwin/threads.pm',
'vars.pm' => '/usr/lib/perl5/5.9.5/vars.pm',
'warnings.pm' => '/usr/lib/perl5/5.9.5/warnings.pm',
'warnings/register.pm' => '/usr/lib/perl5/5.9.5/warnings/register.pm',

After the get() call, %INC contains​:

'/usr/lib/perl5/5.9.5/cygwin/auto/Compress/Raw/Zlib/autosplit.ix' =>
'/usr/lib/perl5/5.9.5/cygwin/auto/Compress/Raw/Zlib/autosplit.ix',
'/usr/lib/perl5/5.9.5/cygwin/auto/Compress/Zlib/autosplit.ix' =>
'/usr/lib/perl5/5.9.5/cygwin/auto/Compress/Zlib/autosplit.ix',
'AutoLoader.pm' => '/usr/lib/perl5/5.9.5/AutoLoader.pm',
'Carp.pm' => '/usr/lib/perl5/5.9.5/Carp.pm',
'Compress/Raw/Zlib.pm' => '/usr/lib/perl5/5.9.5/cygwin/Compress/Raw/Zlib.pm',
'Compress/Zlib.pm' => '/usr/lib/perl5/5.9.5/cygwin/Compress/Zlib.pm',
'Config.pm' => '/usr/lib/perl5/5.9.5/cygwin/Config.pm',
'Data/Dumper.pm' => '/usr/lib/perl5/5.9.5/cygwin/Data/Dumper.pm',
'Errno.pm' => '/usr/lib/perl5/5.9.5/cygwin/Errno.pm',
'Exporter.pm' => '/usr/lib/perl5/5.9.5/Exporter.pm',
'Exporter/Heavy.pm' => '/usr/lib/perl5/5.9.5/Exporter/Heavy.pm',
'Fcntl.pm' => '/usr/lib/perl5/5.9.5/cygwin/Fcntl.pm',
'File/Glob.pm' => '/usr/lib/perl5/5.9.5/cygwin/File/Glob.pm',
'File/GlobMapper.pm' => '/usr/lib/perl5/5.9.5/cygwin/File/GlobMapper.pm',
'File/Spec.pm' => '/usr/lib/perl5/5.9.5/File/Spec.pm',
'File/Spec/Cygwin.pm' => '/usr/lib/perl5/5.9.5/File/Spec/Cygwin.pm',
'File/Spec/Unix.pm' => '/usr/lib/perl5/5.9.5/File/Spec/Unix.pm',
'HTML/Entities.pm' => '/usr/lib/perl5/5.9.5/cygwin/HTML/Entities.pm',
'HTML/HeadParser.pm' => '/usr/lib/perl5/5.9.5/cygwin/HTML/HeadParser.pm',
'HTML/Parser.pm' => '/usr/lib/perl5/5.9.5/cygwin/HTML/Parser.pm',
'HTTP/Date.pm' => '/usr/lib/perl5/5.9.5/HTTP/Date.pm',
'HTTP/Headers.pm' => '/usr/lib/perl5/5.9.5/HTTP/Headers.pm',
'HTTP/Message.pm' => '/usr/lib/perl5/5.9.5/HTTP/Message.pm',
'HTTP/Request.pm' => '/usr/lib/perl5/5.9.5/HTTP/Request.pm',
'HTTP/Response.pm' => '/usr/lib/perl5/5.9.5/HTTP/Response.pm',
'HTTP/Status.pm' => '/usr/lib/perl5/5.9.5/HTTP/Status.pm',
'IO.pm' => '/usr/lib/perl5/5.9.5/cygwin/IO.pm',
'IO/Compress/Adapter/Deflate.pm' =>
'/usr/lib/perl5/5.9.5/cygwin/IO/Compress/Adapter/Deflate.pm',
'IO/Compress/Base.pm' => '/usr/lib/perl5/5.9.5/cygwin/IO/Compress/Base.pm',
'IO/Compress/Base/Common.pm' =>
'/usr/lib/perl5/5.9.5/cygwin/IO/Compress/Base/Common.pm',
'IO/Compress/Gzip.pm' => '/usr/lib/perl5/5.9.5/cygwin/IO/Compress/Gzip.pm',
'IO/Compress/Gzip/Constants.pm' =>
'/usr/lib/perl5/5.9.5/cygwin/IO/Compress/Gzip/Constants.pm',
'IO/Compress/RawDeflate.pm' =>
'/usr/lib/perl5/5.9.5/cygwin/IO/Compress/RawDeflate.pm',
'IO/Compress/Zlib/Extra.pm' =>
'/usr/lib/perl5/5.9.5/cygwin/IO/Compress/Zlib/Extra.pm',
'IO/File.pm' => '/usr/lib/perl5/5.9.5/cygwin/IO/File.pm',
'IO/Handle.pm' => '/usr/lib/perl5/5.9.5/cygwin/IO/Handle.pm',
'IO/Seekable.pm' => '/usr/lib/perl5/5.9.5/cygwin/IO/Seekable.pm',
'IO/Select.pm' => '/usr/lib/perl5/5.9.5/cygwin/IO/Select.pm',
'IO/Socket.pm' => '/usr/lib/perl5/5.9.5/cygwin/IO/Socket.pm',
'IO/Socket/INET.pm' => '/usr/lib/perl5/5.9.5/cygwin/IO/Socket/INET.pm',
'IO/Socket/UNIX.pm' => '/usr/lib/perl5/5.9.5/cygwin/IO/Socket/UNIX.pm',
'IO/Uncompress/Adapter/Inflate.pm' =>
'/usr/lib/perl5/5.9.5/cygwin/IO/Uncompress/Adapter/Inflate.pm',
'IO/Uncompress/Base.pm' => '/usr/lib/perl5/5.9.5/cygwin/IO/Uncompress/Base.pm',
'IO/Uncompress/Gunzip.pm' =>
'/usr/lib/perl5/5.9.5/cygwin/IO/Uncompress/Gunzip.pm',
'IO/Uncompress/RawInflate.pm' =>
'/usr/lib/perl5/5.9.5/cygwin/IO/Uncompress/RawInflate.pm',
'LWP.pm' => '/usr/lib/perl5/5.9.5/LWP.pm',
'LWP/Debug.pm' => '/usr/lib/perl5/5.9.5/LWP/Debug.pm',
'LWP/MemberMixin.pm' => '/usr/lib/perl5/5.9.5/LWP/MemberMixin.pm',
'LWP/Protocol.pm' => '/usr/lib/perl5/5.9.5/LWP/Protocol.pm',
'LWP/Protocol/http.pm' => '/usr/lib/perl5/5.9.5/LWP/Protocol/http.pm',
'LWP/Simple.pm' => '/usr/lib/perl5/5.9.5/LWP/Simple.pm',
'LWP/UserAgent.pm' => '/usr/lib/perl5/5.9.5/LWP/UserAgent.pm',
'List/Util.pm' => '/usr/lib/perl5/5.9.5/cygwin/List/Util.pm',
'Net/HTTP.pm' => '/usr/lib/perl5/5.9.5/Net/HTTP.pm',
'Net/HTTP/Methods.pm' => '/usr/lib/perl5/5.9.5/Net/HTTP/Methods.pm',
'Scalar/Util.pm' => '/usr/lib/perl5/5.9.5/cygwin/Scalar/Util.pm',
'SelectSaver.pm' => '/usr/lib/perl5/5.9.5/SelectSaver.pm',
'Socket.pm' => '/usr/lib/perl5/5.9.5/cygwin/Socket.pm',
'Symbol.pm' => '/usr/lib/perl5/5.9.5/Symbol.pm',
'Time/Local.pm' => '/usr/lib/perl5/5.9.5/Time/Local.pm',
'URI.pm' => '/usr/lib/perl5/5.9.5/URI.pm',
'URI/Escape.pm' => '/usr/lib/perl5/5.9.5/URI/Escape.pm'
'URI/_generic.pm' => '/usr/lib/perl5/5.9.5/URI/_generic.pm',
'URI/_query.pm' => '/usr/lib/perl5/5.9.5/URI/_query.pm',
'URI/_server.pm' => '/usr/lib/perl5/5.9.5/URI/_server.pm',
'URI/http.pm' => '/usr/lib/perl5/5.9.5/URI/http.pm',
'XSLoader.pm' => '/usr/lib/perl5/5.9.5/cygwin/XSLoader.pm',
'bytes.pm' => '/usr/lib/perl5/5.9.5/bytes.pm',
'constant.pm' => '/usr/lib/perl5/5.9.5/constant.pm',
'integer.pm' => '/usr/lib/perl5/5.9.5/integer.pm',
'overload.pm' => '/usr/lib/perl5/5.9.5/overload.pm',
'strict.pm' => '/usr/lib/perl5/5.9.5/strict.pm',
'threads.pm' => '/usr/lib/perl5/5.9.5/cygwin/threads.pm',
'utf8.pm' => '/usr/lib/perl5/5.9.5/utf8.pm',
'vars.pm' => '/usr/lib/perl5/5.9.5/vars.pm',
'warnings.pm' => '/usr/lib/perl5/5.9.5/warnings.pm',
'warnings/register.pm' => '/usr/lib/perl5/5.9.5/warnings/register.pm',

Yowza! So when the attached script is run without args, all
of this module loading is done inside the threads. With 20
threads doing this at the same time, something gets fouled
up in memory and Perl crashes.

When the script is run with an argument, a get() call is
made in the main thread before any threads are launched. As
a consequence, the threads don't have to load anything, and
the script runs without trouble. (I don't get any failures
in this case.)

As I stated above, the script fails for me more than 80% of
the time under blead (when loading is done in the threads).
When I tried it under 5.8.8, it only failed about 10% of the
time. Hopefully, someone can figure out where the
corruption is occurring and can fix it before 5.10.


Flags​:
  category=core
  severity=high


Site configuration information for perl 5.9.5​:

Configured by Jerry at Wed Aug 29 14​:29​:01 EDT 2007.

Summary of my perl5 (revision 5 version 9 subversion 5 patch 31761)
configuration​:
  Platform​:
  osname=cygwin, osvers=1.5.24(0.15642), archname=cygwin-thread-multi-64int
  uname='cygwin_nt-5.1 pn100-02-2-054p 1.5.24(0.15642) 2007-01-31
10​:57 i686 cygwin '
  config_args='-de -Dusedevel -Dversiononly=no -Dinstallusrbinperl
-DPERL_OLD_COPY_ON_WRITE -Duse64bitint -Dusethreads -Uusemymalloc
-Dnoextensions=attrs IPC/SysV Sys/Syslog DB_File NDBM_File ODBM_File
SDBM_File Devel/DProf Devel/Peek re Text/Soundex Math/BigInt/FastCalc
Time/Piece -A define​:optimize=-O3 -pipe -funit-at-a-time
-mtune=pentium4m -march=pentium4 -mfpmath=sse -mieee-fp -mmmx -msse
-msse2 -A append​:ccflags= -DPERL_TRACK_MEMPOOL -DNO_MATHOMS'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=undef, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='gcc', ccflags ='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__
-DPERL_TRACK_MEMPOOL -DNO_MATHOMS -fno-strict-aliasing -pipe',
  optimize='-O3 -pipe -funit-at-a-time -mtune=pentium4m
-march=pentium4 -mfpmath=sse -mieee-fp -mmmx -msse -msse2',
  cppflags='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__
-DPERL_TRACK_MEMPOOL -DNO_MATHOMS -fno-strict-aliasing -pipe'
  ccversion='', gccversion='3.4.4 (cygming special, gdc 0.12, using
dmd 0.125)', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
  ivtype='long long', ivsize=8, nvtype='double', nvsize=8,
Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='g++', ldflags =' -Wl,--enable-auto-import
-Wl,--export-all-symbols -Wl,--stack,8388608
-Wl,--enable-auto-image-base -Wl,--enable-auto-import -s
-L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib /lib
  libs=-lgdbm -ldl -lcrypt -lgdbm_compat
  perllibs=-ldl -lcrypt -lgdbm_compat
  libc=/usr/lib/libc.a, so=dll, useshrplib=true, libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' -s'
  cccdlflags=' ', lddlflags=' --shared -Wl,--enable-auto-import
-Wl,--export-all-symbols -Wl,--stack,8388608
-Wl,--enable-auto-image-base -Wl,--enable-auto-import -s
-L/usr/local/lib'

Locally applied patches​:
  DEVEL


@​INC for perl 5.9.5​:
  /usr/lib/perl5/5.9.5/cygwin
  /usr/lib/perl5/5.9.5
  .


Environment for perl 5.9.5​:
  CYGWIN=ntsec
  HOME=/home/jhedden
  LANG=C
  LANGUAGE=C
  LC_ALL=C
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/home/jhedden/bin​:/usr/local/src/perl/bin​:/usr/local/bin​:/usr/bin​:/bin​:/usr/X11R6/bin​:/c/Program
Files/WiX​:/c/djgpp/bin​:/c/Program
Files/apache-ant-1.7.0/bin​:/c/Program
Files/nant-0.85/bin​:/c/j2sdk1.4.2_14/bin​:/c/dev-cpp/bin/​:/c/WINDOWS/system32​:/c/WINDOWS​:/c/WINDOWS/system32/WBEM​:/c/blp/API/dde​:/c/blp/API​:/c/oracle/ora92/bin​:/c/Program
Files/Oracle/jre/1.3.1/bin​:/c/Program
Files/Oracle/jre/1.1.8/bin​:/c/Program
Files/Hummingbird/Connectivity/7.10/Accessories/​:.
  PERLIO=perlio
  PERL_BADLANG (unset)
  SHELL (unset)

@p5pRT
Copy link
Author

p5pRT commented Aug 30, 2007

From @jdhedden

fetch_bug.pl

@p5pRT
Copy link
Author

p5pRT commented Sep 5, 2007

From @jdhedden

Jerry D. Hedden wrote​:

There seems to be a memory corruption problem that manifests
itself when several threads try to load a whole bunch of
modules at the same time.

I would consider this bug a showstopper. While the supplied
script uses threads to evoke the bug, it seems to me that
the memory corruption problem could occur even with
non-threaded scripts. (Or am I just being paranoid?)

@p5pRT
Copy link
Author

p5pRT commented Sep 5, 2007

From @schwern

Jerry D. Hedden wrote​:

Jerry D. Hedden wrote​:

There seems to be a memory corruption problem that manifests
itself when several threads try to load a whole bunch of
modules at the same time.

I would consider this bug a showstopper. While the supplied
script uses threads to evoke the bug, it seems to me that
the memory corruption problem could occur even with
non-threaded scripts. (Or am I just being paranoid?)

As 5.8.8 cores, too, I would rate this as "no worse than 5.8.8" and thus not a
stopper.

BTW The fireworks with bleadperl sure are pretty...

$ bleadperl ~/Downloads/fetch_bug.pl
bleadperl(5440,0x190de00) malloc​: *** Deallocation of a pointer not malloced​:
0x4f26c0; This could be a double free(), or free() called with the middle of
an allocated block; Try setting environment variable MallocHelp to see tools
to help debug
bleadperl(5440,0x190de00) malloc​: *** Deallocation of a pointer not malloced​:
0x46a050; This could be a double free(), or free() called with the middle of
an allocated block; Try setting environment variable MallocHelp to see tools
to help debug
bleadperl(5440,0x190de00) malloc​: *** Deallocation of a pointer not malloced​:
0x46a050; This could be a double free(), or free() called with the middle of
an allocated block; Try setting environment variable MallocHelp to see tools
to help debug
bleadperl(5440,0x190de00) malloc​: *** Deallocation of a pointer not malloced​:
0x46a050; This could be a double free(), or free() called with the middle of
an allocated block; Try setting environment variable MallocHelp to see tools
to help debug
bleadperl(5440,0x190de00) malloc​: *** Deallocation of a pointer not malloced​:
0x1196150; This could be a double free(), or free() called with the middle of
an allocated block; Try setting environment variable MallocHelp to see tools
to help debug
bleadperl(5440,0x190de00) malloc​: *** Deallocation of a pointer not malloced​:
0x1196150; This could be a double free(), or free() called with the middle of
an allocated block; Try setting environment variable MallocHelp to see tools
to help debug
bleadperl(5440,0x18c4800) malloc​: *** error for object 0x1197870​: double free
bleadperl(5440,0x18c4800) malloc​: *** set a breakpoint in szone_error to debug
bleadperl(5440,0x18c4800) malloc​: *** error for object 0x1197be0​: double free
bleadperl(5440,0x18c4800) malloc​: *** set a breakpoint in szone_error to debug
bleadperl(5440,0x190de00) malloc​: *** error for object 0x11962d0​: double free
bleadperl(5440,0x190de00) malloc​: *** set a breakpoint in szone_error to debug
bleadperl(5440,0x18c4800) malloc​: *** Deallocation of a pointer not malloced​:
0x1196150; This could be a double free(), or free() called with the middle of
an allocated block; Try setting environment variable MallocHelp to see tools
to help debug
bleadperl(5440,0x18c4800) malloc​: *** Deallocation of a pointer not malloced​:
0x1196150; This could be a double free(), or free() called with the middle of
an allocated block; Try setting environment variable MallocHelp to see tools
to help debug
Unbalanced string table refcount​: (1) for "SO_CHAMELEON" during global
destruction.
Unbalanced string table refcount​: (1) for "AF_GOSIP" during global destruction.
Unbalanced string table refcount​: (1) for "SO_ATTACH_FILTER" during global
destruction.
Unbalanced string table refcount​: (1) for "AF_OSINET" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_WIRE" during global destruction.
Unbalanced string table refcount​: (1) for "PF_CTF" during global destruction.
Unbalanced string table refcount​: (1) for "SCM_CONNECT" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_URG" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_MCAST" during global destruction.
Unbalanced string table refcount​: (1) for "SO_XOPEN" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_CTLFLAGS" during global
destruction.
Unbalanced string table refcount​: (1) for "PF_USER" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_FIN" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_SYN" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_CTLIGNORE" during global
destruction.
Unbalanced string table refcount​: (1) for "SO_PROTOCOL" during global destruction.
Unbalanced string table refcount​: (1) for "SO_DONTLINGER" during global
destruction.
Unbalanced string table refcount​: (1) for "UIO_MAXIOV" during global destruction.
Unbalanced string table refcount​: (1) for "TCP_STDURG" during global destruction.
Unbalanced string table refcount​: (1) for "SO_PEERCRED" during global destruction.
Unbalanced string table refcount​: (1) for "AF_WAN" during global destruction.
Unbalanced string table refcount​: (1) for "SO_FAMILY" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_ETAG" during global destruction.
Unbalanced string table refcount​: (1) for "PF_X25" during global destruction.
Unbalanced string table refcount​: (1) for "SCM_CREDENTIALS" during global
destruction.
Unbalanced string table refcount​: (1) for "PF_AAL" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_BCAST" during global destruction.
Unbalanced string table refcount​: (1) for "AF_X25" during global destruction.
Unbalanced string table refcount​: (1) for "SO_STATE" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_RST" during global destruction.
Unbalanced string table refcount​: (1) for "AF_802" during global destruction.
Unbalanced string table refcount​: (1) for "AF_LAST" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_MAXIOVLEN" during global
destruction.
Unbalanced string table refcount​: (1) for "MSG_BTAG" during global destruction.
Unbalanced string table refcount​: (1) for "AF_USER" during global destruction.
Unbalanced string table refcount​: (1) for "PF_GOSIP" during global destruction.
Unbalanced string table refcount​: (1) for "SO_PROTOTYPE" during global
destruction.
Unbalanced string table refcount​: (1) for "SO_SECURITY_ENCRYPTION_NETWORK"
during global destruction.
Unbalanced string table refcount​: (1) for "PF_NIT" during global destruction.
Unbalanced string table refcount​: (1) for "SO_DETACH_FILTER" during global
destruction.
Unbalanced string table refcount​: (1) for "SO_XSE" during global destruction.
Unbalanced string table refcount​: (1) for "PF_LAST" during global destruction.
Unbalanced string table refcount​: (1) for "SO_BACKLOG" during global destruction.
Unbalanced string table refcount​: (1) for "AF_NIT" during global destruction.
Unbalanced string table refcount​: (1) for "SO_SECURITY_ENCRYPTION_TRANSPORT"
during global destruction.
Unbalanced string table refcount​: (1) for "AF_KEY" during global destruction.
Unbalanced string table refcount​: (1) for "AF_AAL" during global destruction.
Unbalanced string table refcount​: (1) for "SO_SECURITY_AUTHENTICATION" during
global destruction.
Unbalanced string table refcount​: (1) for "AF_CTF" during global destruction.
Unbalanced string table refcount​: (1) for "AF_NBS" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_ERRQUEUE" during global
destruction.
Unbalanced string table refcount​: (1) for "TCP_MAXRT" during global destruction.
Unbalanced string table refcount​: (1) for "SO_PASSCRED" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_PROXY" during global destruction.
Unbalanced string table refcount​: (1) for "PF_802" during global destruction.
Unbalanced string table refcount​: (1) for "MSG_NOSIGNAL" during global
destruction.
Unbalanced string table refcount​: (1) for "PF_NBS" during global destruction.
Unbalanced string table refcount​: (1) for "PF_OSINET" during global destruction.
Unbalanced string table refcount​: (1) for "SO_DGRAM_ERRIND" during global
destruction.
Unbalanced string table refcount​: (1) for "PF_WAN" during global destruction.
Unbalanced string table refcount​: (1) for "SO_PASSIFNAME" during global
destruction.
bleadperl(5440,0x187b600) malloc​: *** Deallocation of a pointer not malloced​:
0x4320d0; This could be a double free(), or free() called with the middle of
an allocated block; Try setting environment variable MallocHelp to see tools
to help debug
Attempt to free non-existent shared string 'SO_BACKLOG', Perl interpreter​:
0x18c4c00 during global destruction.
Attempt to free non-existent shared string 'SO_DETACH_FILTER', Perl
interpreter​: 0x18c4c00 during global destruction.
Attempt to free non-existent shared string 'PF_NBS', Perl interpreter​:
0x18c4c00 during global destruction.
Attempt to free non-existent shared string 'PF_WAN', Perl interpreter​:
0x18c4c00 during global destruction.
Attempt to free non-existent shared string 'PF_AAL', Perl interpreter​:
0x18c4c00 during global destruction.
Attempt to free non-existent shared string 'SO_PASSIFNAME', Perl interpreter​:
0x18c4c00 during global destruction.
Bus error (core dumped)

--
The interface should be as clean as newly fallen snow and its behavior
as explicit as Japanese eel porn.

@p5pRT
Copy link
Author

p5pRT commented Sep 5, 2007

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Sep 5, 2007

From @jdhedden

Jerry D. Hedden wrote​:

There seems to be a memory corruption problem that manifests
itself when several threads try to load a whole bunch of
modules at the same time.

I would consider this bug a showstopper. While the supplied
script uses threads to evoke the bug, it seems to me that
the memory corruption problem could occur even with
non-threaded scripts. (Or am I just being paranoid?)

Michael G Schwern wrote​:

As 5.8.8 cores, too, I would rate this as "no worse than
5.8.8" and thus not a stopper.

It may be that the problem is not new to 5.9, but the ease
with which it can be evoked is very disturbing. The test
script is a simple LWP​::Simple get() in a thread. If that's
unstable, there will be a lot of bug reports when 5.10 goes
out.

@p5pRT
Copy link
Author

p5pRT commented Sep 5, 2007

From @iabyn

On Wed, Sep 05, 2007 at 02​:06​:25PM -0400, Jerry D. Hedden wrote​:

Jerry D. Hedden wrote​:

There seems to be a memory corruption problem that manifests
itself when several threads try to load a whole bunch of
modules at the same time.

I would consider this bug a showstopper. While the supplied
script uses threads to evoke the bug, it seems to me that
the memory corruption problem could occur even with
non-threaded scripts. (Or am I just being paranoid?)

Michael G Schwern wrote​:

As 5.8.8 cores, too, I would rate this as "no worse than
5.8.8" and thus not a stopper.

It may be that the problem is not new to 5.9, but the ease
with which it can be evoked is very disturbing. The test
script is a simple LWP​::Simple get() in a thread. If that's
unstable, there will be a lot of bug reports when 5.10 goes

Actually I've reduced the test case to​:

  use threads;

  threads->create(\&thr_fetch) for 1..20;
  warn "waiting for threads to finish\n";
  $_->join for threads->list();

  sub thr_fetch
  {
  require dummy;
  }

where dummy.pm just contains the single line "1;"

This gives a "panic​: free from wrong pool".

I hope to track down this problem when I get a bit of free time.

--
"Do not dabble in paradox, Edward, it puts you in danger of fortuitous wit."
  -- Lady Croom, "Arcadia"

@p5pRT
Copy link
Author

p5pRT commented Sep 12, 2007

From @jdhedden

Jerry D. Hedden wrote​:

There seems to be a memory corruption problem that manifests
itself when several threads try to load a whole bunch of
modules at the same time.

Dave Mitchell wrote​:

Actually I've reduced the test case to​:

use threads;

threads\->create\(\\&thr\_fetch\) for 1\.\.20;
warn "waiting for threads to finish\\n";
$\_\->join for threads\->list\(\);

sub thr\_fetch
\{
    require dummy;
\}

where dummy.pm just contains the single line "1;"

This gives a "panic​: free from wrong pool".

I've reduced it yet again​:

  use threads;

  threads->create(sub { eval '1' }) for 1..20;
  print("Waiting for threads to finish\n");
  $_->join() foreach (threads->list());

So the problems seems to occur when doing string evals.

I hope to track down this problem when I get a bit of free time.

Sure hope you can get to it before 5.10 goes out.

@p5pRT
Copy link
Author

p5pRT commented Sep 13, 2007

From @jbenjore

On 9/12/07, Jerry D. Hedden <jdhedden@​cpan.org> wrote​:

Jerry D. Hedden wrote​:

There seems to be a memory corruption problem that manifests
itself when several threads try to load a whole bunch of
modules at the same time.

Dave Mitchell wrote​:

Actually I've reduced the test case to​:

use threads;

threads\->create\(\\&thr\_fetch\) for 1\.\.20;
warn "waiting for threads to finish\\n";
$\_\->join for threads\->list\(\);

sub thr\_fetch
\{
    require dummy;
\}

where dummy.pm just contains the single line "1;"

This gives a "panic​: free from wrong pool".

I've reduced it yet again​:

use threads;

threads\->create\(sub \{ eval '1' \}\) for 1\.\.20;
print\("Waiting for threads to finish\\n"\);
$\_\->join\(\) foreach \(threads\->list\(\)\);

So the problems seems to occur when doing string evals.

I hope to track down this problem when I get a bit of free time.

Sure hope you can get to it before 5.10 goes out.

What, simultaneous compilation worked in the past?! Which versions?

Josh

@p5pRT
Copy link
Author

p5pRT commented Sep 13, 2007

From @jdhedden

Jerry D. Hedden wrote​:

There seems to be a memory corruption problem that manifests
itself when several threads try to load a whole bunch of
modules at the same time.

use threads;

threads\->create\(sub \{ eval '1' \}\) for 1\.\.20;
print\("Waiting for threads to finish\\n"\);
$\_\->join\(\) foreach \(threads\->list\(\)\);

So the problems seems to occur when doing string evals.

Joshua ben Jore wrote​:

What, simultaneous compilation worked in the past?! Which
versions?

The above works on 5.8.7 with threads 1.65. It fails with
'Free to wrong pool' errors in 5.8.8 and blead (again with
threads 1.65).

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2007

From @iabyn

On Wed, Sep 12, 2007 at 10​:53​:36AM -0400, Jerry D. Hedden wrote​:

I've reduced it yet again​:

use threads;

threads\->create\(sub \{ eval '1' \}\) for 1\.\.20;
print\("Waiting for threads to finish\\n"\);
$\_\->join\(\) foreach \(threads\->list\(\)\);

Turns out there are at least two bugs. The one above has been fixed by
change #31864. However, the code in the bug report still fails. I have
reduced that to​:

  use threads;

  sub f {
  require IO;
  }

  my @​t;
  push @​t, threads->create(\&f) for 1..80;
  $_->join for @​t;

which fails about 1 in 10 times. It might be associated with creating const
subs via XS. I'll look further when I get time.

--
"You're so sadly neglected, and often ignored.
A poor second to Belgium, When going abroad."
  -- Monty Python, "Finland"

@p5pRT
Copy link
Author

p5pRT commented Sep 29, 2007

From @jdhedden

Jerry D. Hedden wrote​:

I've reduced it yet again​:

use threads;

threads\->create\(sub \{ eval '1' \}\) for 1\.\.20;
print\("Waiting for threads to finish\\n"\);
$\_\->join\(\) foreach \(threads\->list\(\)\);

Dave Mitchell wrote​:

Turns out there are at least two bugs. The one above has been fixed by
change #31864. However, the code in the bug report still fails. I have
reduced that to​:

use threads;

sub f \{
    require IO;
\}

my @&#8203;t;
push @&#8203;t\, threads\->create\(\\&f\) for 1\.\.80;
$\_\->join for @&#8203;t;

which fails about 1 in 10 times. It might be associated with creating const
subs via XS. I'll look further when I get time.

Any chance of tackling this before the code freeze, Dave?

@p5pRT
Copy link
Author

p5pRT commented Oct 10, 2007

From @iabyn

On Fri, Sep 14, 2007 at 07​:31​:22PM +0100, Dave Mitchell wrote​:

On Wed, Sep 12, 2007 at 10​:53​:36AM -0400, Jerry D. Hedden wrote​:

I've reduced it yet again​:

use threads;

threads\->create\(sub \{ eval '1' \}\) for 1\.\.20;
print\("Waiting for threads to finish\\n"\);
$\_\->join\(\) foreach \(threads\->list\(\)\);

Turns out there are at least two bugs. The one above has been fixed by
change #31864. However, the code in the bug report still fails. I have
reduced that to​:

use threads;

sub f \{
require IO;
\}

my @&#8203;t;
push @&#8203;t\, threads\->create\(\\&f\) for 1\.\.80;
$\_\->join for @&#8203;t;

which fails about 1 in 10 times. It might be associated with creating const
subs via XS. I'll look further when I get time.

Now fixed (hopefully) by #32091.

--
"Emacs isn't a bad OS once you get used to it.
It just lacks a decent editor."

Change 32091 by davem@​davem-pigeon on 2007/10/10 15​:03​:16

  newCONTSUB() wasn't thread-safe ([perl #45053])

Affected files ...

... //depot/perl/ext/threads/t/problems.t#18 edit
... //depot/perl/op.c#958 edit

Differences ...

==== //depot/perl/ext/threads/t/problems.t#18 (text) ====

@​@​ -29,9 +29,9 @​@​

  $| = 1;
  if ($] == 5.008) {
- print("1..11\n"); ### Number of tests that will be run ###
+ print("1..12\n"); ### Number of tests that will be run ###
  } else {
- print("1..15\n"); ### Number of tests that will be run ###
+ print("1..16\n"); ### Number of tests that will be run ###
  }
};

@​@​ -178,4 +178,20 @​@​
$child = threads->create(sub { return (scalar(keys(%h))); })->join;
is($child, 1, "keys correct in child with restricted hash");

+
+# [perl #45053] Memory corruption with heavy module loading in threads
+#
+# run-time usage of newCONSTSUB (as done by the IO boot code) wasn't
+# thread-safe - got occasional coredumps or malloc corruption
+
+{
+ my @​t;
+ push @​t, threads->create( sub { require IO }) for 1..100;
+ $_->join for @​t;
+ print("ok $test - [perl #45053]\n");
+ $test++;
+}
+
+
+
# EOF

==== //depot/perl/op.c#958 (text) ====

@​@​ -5696,6 +5696,13 @​@​

  ENTER;

+ if (IN_PERL_RUNTIME) {
+ /* at runtime, it's not safe to manipulate PL_curcop​: it may be
+ * an op shared between threads. Use a non-shared COP for our
+ * dirty work */
+ SAVEVPTR(PL_curcop);
+ PL_curcop = &PL_compiling;
+ }
  SAVECOPLINE(PL_curcop);
  CopLINE_set(PL_curcop, PL_parser ? PL_parser->copline : NOLINE);

@p5pRT
Copy link
Author

p5pRT commented Oct 10, 2007

From @jdhedden

Jerry D. Hedden wrote​:

I've reduced it yet again​:

use threads;

threads\->create\(sub \{ eval '1' \}\) for 1\.\.20;
print\("Waiting for threads to finish\\n"\);
$\_\->join\(\) foreach \(threads\->list\(\)\);

Dave Mitchell wrote​:

Turns out there are at least two bugs. The one above has been fixed by
change #31864. However, the code in the bug report still fails. I have
reduced that to​:

use threads;

sub f \{
  require IO;
\}

my @&#8203;t;
push @&#8203;t\, threads\->create\(\\&f\) for 1\.\.80;
$\_\->join for @&#8203;t;

which fails about 1 in 10 times. It might be associated with creating const
subs via XS. I'll look further when I get time.

Dave Mitchell wrote​:

Now fixed (hopefully) by #32091.

Excellent.

Affected files ...

... //depot/perl/ext/threads/t/problems.t#18 edit
... //depot/perl/op.c#958 edit

I think the test should go in t/op/threads.t because it is a change to
core code, and not to the threads module code. I'll submit a patch
for this.

@p5pRT
Copy link
Author

p5pRT commented Feb 3, 2008

From @jdhedden

Change 31864 was supposed to fix threads crashing bug 45053​:

  http​://www.mail-archive.com/perl5-changes@​perl.org/msg18477.html
  http​://rt.perl.org/rt3/Public/Bug/Display.html?id=45053

However, some residual crashing remains.

The attached script is the test added by 31864 to
t/op/threads.t. It doesn't fail all the time, but you will
get some segfaults if you run it in a loop​:

  for (( ii=0; $ii < 100; ii++ )); do echo $ii; ./crash.pl; done

Occassionally, it outputs​:

  Bad free() ignored (PERL_CORE) during global destruction.

(I cannot reproduce these failures in the latest 5.8.x.)

@p5pRT
Copy link
Author

p5pRT commented Feb 3, 2008

From @jdhedden

crash.pl

@p5pRT
Copy link
Author

p5pRT commented Feb 22, 2008

From @jdhedden

Change 31864 was supposed to fix threads crashing bug 45053​:

http&#8203;://www\.mail\-archive\.com/perl5\-changes@&#8203;perl\.org/msg18477\.html
http&#8203;://rt\.perl\.org/rt3/Public/Bug/Display\.html?id=45053

However, some residual crashing remains.

The attached script is the test added by 31864 to
t/op/threads.t. It doesn't fail all the time, but you will
get some segfaults if you run it in a loop​:

for \(\( ii=0; $ii \< 100; ii\+\+ \)\); do echo $ii; \./crash\.pl; done

Occassionally, it outputs​:

Bad free\(\) ignored \(PERL\_CORE\) during global destruction\.

(I cannot reproduce these failures in the latest 5.8.x.)

This bug occurs on my home PC that has 4GB of RAM, but not on my work
PC that only has 1GB of RAM. I'd like to make a wild-ass suggestion
that perhaps this corruption is caused by an errant signed/unsigned
bug.

Can anyone else even reproduce it?

@p5pRT
Copy link
Author

p5pRT commented Feb 23, 2008

From nospam-abuse@bloodgate.com

On Saturday 23 February 2008 00​:26​:12 Jerry D. Hedden wrote​:

Change 31864 was supposed to fix threads crashing bug 45053​:

http​://www.mail-archive.com/perl5-changes@​perl.org/msg18477.html
http​://rt.perl.org/rt3/Public/Bug/Display.html?id=45053

However, some residual crashing remains.

The attached script is the test added by 31864 to
t/op/threads.t. It doesn't fail all the time, but you will
get some segfaults if you run it in a loop​:

for \(\( ii=0; $ii \< 100; ii\+\+ \)\); do echo $ii; \./crash\.pl; done

Occassionally, it outputs​:

Bad free\(\) ignored \(PERL\_CORE\) during global destruction\.

(I cannot reproduce these failures in the latest 5.8.x.)

This bug occurs on my home PC that has 4GB of RAM, but not on my work
PC that only has 1GB of RAM. I'd like to make a wild-ass suggestion
that perhaps this corruption is caused by an errant signed/unsigned
bug.

Can anyone else even reproduce it?

It doesn't crash here, 64Bit Ubuntu with 4Gbyte of memory​:

Just for testing I replaced the "100" with "10000" and it took quite a
while (20 seconds?) for my system to run out of memory. Since I don't
have swap, things ground to a halt and after a while the script was
killed, but nothing else happened.

Btw, using "2000", my system takes

  # time perl crash.pl

  real 0m30.602s
  user 0m23.793s
  sys 0m7.872s

this seems an aweful long time for creating/joining/destroying 2000
threads.

############################################################
te@​te​:~$ cat crash.pl
#!/usr/bin/perl

use strict;
use warnings;
use threads;

{
  local $SIG{__WARN__} = sub {}; # Ignore any thread creation
failure warnings
  my @​t;
  for (1..100) {
  my $thr = threads->create( sub { require IO });
  last if !defined($thr); # Probably ran out of memory
  push(@​t, $thr);
  }
  $_->join for @​t;
}
exit(0); # Done

# EOF
te@​te​:~$ perl crash.pl
te@​te​:~$ perl -v

This is perl, v5.8.8 built for x86_64-linux-gnu-thread-multi

Copyright 1987-2006, Larry Wall

Perl may be copied only under the terms of either the Artistic License
or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to
the
Internet, point your browser at http​://www.perl.org/, the Perl Home
Page.

te@​te​:~$ cat /proc/meminfo
MemTotal​: 4046056 kB
MemFree​: 3403416 kB
Buffers​: 19324 kB
Cached​: 329688 kB
SwapCached​: 0 kB
Active​: 360664 kB
Inactive​: 208540 kB
SwapTotal​: 0 kB
SwapFree​: 0 kB
Dirty​: 836 kB
Writeback​: 0 kB
AnonPages​: 220244 kB
Mapped​: 84036 kB
Slab​: 34484 kB
SReclaimable​: 17796 kB
SUnreclaim​: 16688 kB
PageTables​: 12200 kB
NFS_Unstable​: 0 kB
Bounce​: 0 kB
CommitLimit​: 2023028 kB
Committed_AS​: 586200 kB
VmallocTotal​: 34359738367 kB
VmallocUsed​: 35184 kB
VmallocChunk​: 34359703035 kB
#########################################################

best wishes,

Tels

--
Signed on Sat Feb 23 10​:15​:05 2008 with key 0x93B84C15.
View my photo gallery​: http​://bloodgate.com/photos
PGP key on http​://bloodgate.com/tels.asc or per email.

This email violates U.S. patent #5,546,528 and EU patent
EP0689133​:

  ________________________________________
  | Header |
Body | Attachements | |
  |--------+
+----------------------|
  | |

  ~ ~

@p5pRT
Copy link
Author

p5pRT commented Aug 3, 2008

From p5p@spam.wizbit.be

On Sun Feb 03 08​:57​:50 2008, jdhedden@​cpan.org wrote​:

Change 31864 was supposed to fix threads crashing bug 45053​:

http&#8203;://www\.mail\-archive\.com/perl5\-changes@&#8203;perl\.org/msg18477\.html
http&#8203;://rt\.perl\.org/rt3/Public/Bug/Display\.html?id=45053

However, some residual crashing remains.

The attached script is the test added by 31864 to
t/op/threads.t. It doesn't fail all the time, but you will
get some segfaults if you run it in a loop​:

for \(\( ii=0; $ii \< 100; ii\+\+ \)\); do echo $ii; \./crash\.pl; done

Occassionally, it outputs​:

Bad free\(\) ignored \(PERL\_CORE\) during global destruction\.

(I cannot reproduce these failures in the latest 5.8.x.)

I believe this was also fixed with change 32091.

http​://public.activestate.com/cgi-bin/perlbrowse/p/32091
Change 32091 by davem@​davem-pigeon on 2007/10/10 15​:03​:16

  newCONTSUB() wasn't thread-safe ([perl #45053])

perl-p-5.9.5\@​32090/perl test_45053_c.pl
*** glibc detected *** perl-p-5.9.5@​32090/perl​: double free or
corruption (fasttop)​: 0x902c2cb0 **

perl-p-5.9.5\@​32091/perl test_45053_c.pl
(no output)

(Also ran the script in a 1..100 loop - no output there either)

@p5pRT
Copy link
Author

p5pRT commented Aug 3, 2008

p5p@spam.wizbit.be - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant