Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

die inside sort comparitor causes SV error #15243

Closed
p5pRT opened this issue Mar 21, 2016 · 16 comments
Closed

die inside sort comparitor causes SV error #15243

p5pRT opened this issue Mar 21, 2016 · 16 comments

Comments

@p5pRT
Copy link

p5pRT commented Mar 21, 2016

Migrated from rt.perl.org#127759 (status was 'resolved')

Searchable as RT127759$

@p5pRT
Copy link
Author

p5pRT commented Mar 21, 2016

From @jimav

This is a bug report for perl from jim.avera@​gmail.com,
generated with the help of perlbug 1.40 running under perl 5.20.2.


If die is called inside a sort comparitor function, errors like the
following (sometimes) occur.

  Attempt to free unreferenced scalar​: SV 0x12c7458, Perl interpreter​: 0x12a2010 at (location of die)

#!/usr/bin/perl
use warnings; use strict; use 5.010;

my @​data = ( ["A"], ["B"], ["C"] );

@​data =
  sort {
  my $aval = $a->[0];
  my $bval = $b->[0];

  die "Die in sort comparator" if $aval eq "C";

  return ($aval cmp $bval);
  }
  @​data;

# Die in sort comparator at /tmp/test.pl line 11.
# Attempt to free unreferenced scalar​: SV 0x12c7458, Perl interpreter​: 0x12a2010 at /tmp/test.pl line 11.
# Attempt to free unreferenced scalar​: SV 0x12c7450, Perl interpreter​: 0x12a2010 at /tmp/test.pl line 11.
#



Flags​:
  category=core
  severity=medium


Site configuration information for perl 5.20.2​:

Configured by Debian Project at Tue Mar 1 16​:51​:39 UTC 2016.

Summary of my perl5 (revision 5 version 20 subversion 2) configuration​:
 
  Platform​:
  osname=linux, osvers=3.13.0-79-generic, archname=x86_64-linux-gnu-thread-multi
  uname='linux lgw01-20 3.13.0-79-generic #123-ubuntu smp fri feb 19 14​:27​:58 utc 2016 x86_64 x86_64 x86_64 gnulinux '
  config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl,-Bsymbolic-functions -Wl,-z,relro -Dlddlflags=-shared -Wl,-Bsymbolic-functions -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.20 -Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.20 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.20 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.20.2 -Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.20.2 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.20.2 -des'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2 -g',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include'
  ccversion='', gccversion='5.2.1 20151010', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
  perllibs=-ldl -lm -lpthread -lc -lcrypt
  libc=libc-2.21.so, so=so, useshrplib=true, libperl=libperl.so.5.20
  gnulibc_version='2.21'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector'

Locally applied patches​:
  DEBPKG​:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS default for modules installed from CPAN.
  DEBPKG​:debian/db_file_ver - http​://bugs.debian.org/340047 Remove overly restrictive DB_File version check.
  DEBPKG​:debian/doc_info - Replace generic man(1) instructions with Debian-specific information.
  DEBPKG​:debian/enc2xs_inc - http​://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @​INC directories.
  DEBPKG​:debian/errno_ver - http​://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes.
  DEBPKG​:debian/libperl_embed_doc - http​://bugs.debian.org/186778 Note that libperl-dev package is required for embedded linking
  DEBPKG​:fixes/respect_umask - Respect umask during installation
  DEBPKG​:debian/writable_site_dirs - Set umask approproately for site install directories
  DEBPKG​:debian/extutils_set_libperl_path - EU​:MM​: set location of libperl.a under /usr/lib
  DEBPKG​:debian/no_packlist_perllocal - Don't install .packlist or perllocal.pod for perl or vendor
  DEBPKG​:debian/prefix_changes - Fiddle with *PREFIX and variables written to the makefile
  DEBPKG​:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets.
  DEBPKG​:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor.
  DEBPKG​:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy.
  DEBPKG​:debian/libnet_config_path - Set location of libnet.cfg to /etc/perl/Net as /usr may not be writable.
  DEBPKG​:debian/mod_paths - Tweak @​INC ordering for Debian
  DEBPKG​:debian/module_build_man_extensions - http​://bugs.debian.org/479460 Adjust Module​::Build manual page extensions for the Debian Perl policy
  DEBPKG​:debian/prune_libs - http​://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need.
  DEBPKG​:fixes/net_smtp_docs - [rt.cpan.org #36038] http​://bugs.debian.org/100195 Document the Net​::SMTP 'Port' option
  DEBPKG​:debian/perlivp - http​://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local
  DEBPKG​:debian/deprecate-with-apt - http​://bugs.debian.org/747628 Point users to Debian packages of deprecated core modules
  DEBPKG​:debian/squelch-locale-warnings - http​://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts
  DEBPKG​:debian/skip-upstream-git-tests - Skip tests specific to the upstream Git repository
  DEBPKG​:debian/patchlevel - http​://bugs.debian.org/567489 List packaged patches for 5.20.2-6ubuntu0.2 in patchlevel.h
  DEBPKG​:debian/skip-kfreebsd-crash - http​://bugs.debian.org/628493 [perl #96272] Skip a crashing test case in t/op/threads.t on GNU/kFreeBSD
  DEBPKG​:fixes/document_makemaker_ccflags - http​://bugs.debian.org/628522 [rt.cpan.org #68613] Document that CCFLAGS should include $Config{ccflags}
  DEBPKG​:debian/find_html2text - http​://bugs.debian.org/640479 Configure CPAN​::Distribution with correct name of html2text
  DEBPKG​:debian/perl5db-x-terminal-emulator.patch - http​://bugs.debian.org/668490 Invoke x-terminal-emulator rather than xterm in perl5db.pl
  DEBPKG​:debian/cpan-missing-site-dirs - http​://bugs.debian.org/688842 Fix CPAN​::FirstTime defaults with nonexisting site dirs if a parent is writable
  DEBPKG​:fixes/memoize_storable_nstore - [rt.cpan.org #77790] http​://bugs.debian.org/587650 Memoize​::Storable​: respect 'nstore' option not respected
  DEBPKG​:debian/regen-skip - Skip a regeneration check in unrelated git repositories
  DEBPKG​:fixes/regcomp-mips-optim - [perl #122817] http​://bugs.debian.org/754054 Downgrade the optimization of regcomp.c on mips and mipsel due to a gcc-4.9 bug
  DEBPKG​:debian/makemaker-pasthru - http​://bugs.debian.org/758471 Pass LD settings through to subdirectories
  DEBPKG​:fixes/perldoc-less-R - [rt.cpan.org #98636] http​://bugs.debian.org/758689 Tell the 'less' pager to allow terminal escape sequences
  DEBPKG​:fixes/pod_man_reproducible_date - http​://bugs.debian.org/759405 Support POD_MAN_DATE in Pod​::Man for the left-hand footer
  DEBPKG​:fixes/io_uncompress_gunzip_inmemory - http​://bugs.debian.org/747363 [rt.cpan.org #95494] Fix gunzip to in-memory file handle
  DEBPKG​:fixes/socket_test_recv_fix - http​://bugs.debian.org/758718 [perl #122657] Compare recv return value to peername in socket test
  DEBPKG​:fixes/hurd_socket_recv_todo - http​://bugs.debian.org/758718 [perl #122657] TODO checking the result of recv() on hurd
  DEBPKG​:fixes/regexp-performance - [0fa70a0] http​://bugs.debian.org/777556 [perl #123743] simpify and speed up /.*.../ handling
  DEBPKG​:fixes/failed_require_diagnostics - http​://bugs.debian.org/781120 [perl #123270] Report inaccesible file on failed require
  DEBPKG​:fixes/array-cloning - http​://bugs.debian.org/779357 [perl #124127] [902d169] fix cloning arrays with unused elements
  DEBPKG​:fixes/perldb-threads - http​://bugs.debian.org/779357 [perl #124127] [41ef2c6] lib/perl5db.pl​: Restore noop lock prototype
  DEBPKG​:debian/locale-robustness - http​://bugs.debian.org/782068 [perl #124310] Make t/run/locale.t survive missing locales masked by LC_ALL
  DEBPKG​:fixes/gcc5-errno - http​://bugs.debian.org/778060 [perl #123784] [816b056] Fix Errno.pm generation for gcc-5.0
  DEBPKG​:fixes/h2ph-hex-constants - http​://bugs.debian.org/778060 [perl #123784] [3bea78d] h2ph​: correct handling of hex constants for the preamble
  DEBPKG​:fixes/h2ph-test-inc - http​://bugs.debian.org/778060 [perl #123784] [3359391] lib/h2ph.t to test generated t/_h2ph_pre.ph instead of the system one
  DEBPKG​:fixes/podman-utc - http​://bugs.debian.org/780259 Make the embedded date from Pod​::Man reproducible
  DEBPKG​:fixes/podman-utc-docs - http​://bugs.debian.org/780259 Documentation and test suite updates for UTC fix
  DEBPKG​:fixes/podman-empty-date - http​://bugs.debian.org/780259 Support an empty POD_MAN_DATE environment variable
  DEBPKG​:fixes/podman-pipe - http​://bugs.debian.org/777405 Better errors for man pages from standard input
  DEBPKG​:debian/pod2man-customized - Update porting/customized.dat for pod2man modifications
  DEBPKG​:fixes/CVE-2015-8607_file_spec_taint_fix - ensure File​::Spec​::canonpath() preserves taint
  DEBPKG​:fixes/CVE-2016-2381.patch - [PATCH 1/2] remove duplicate environment variables from environ


@​INC for perl 5.20.2​:
  /home/jima/perl5/lib/perl5/x86_64-linux-gnu-thread-multi
  /home/jima/perl5/lib/perl5
  /home/jima/lib/perl
  /etc/perl
  /usr/local/lib/x86_64-linux-gnu/perl/5.20.2
  /usr/local/share/perl/5.20.2
  /usr/lib/x86_64-linux-gnu/perl5/5.20
  /usr/share/perl5
  /usr/lib/x86_64-linux-gnu/perl/5.20
  /usr/share/perl/5.20
  /usr/local/lib/site_perl
  .


Environment for perl 5.20.2​:
  HOME=/home/jima
  LANG=en_US.UTF-8
  LANGUAGE=en_US
  LD_LIBRARY_PATH=/home/jima/local/lib
  LOGDIR (unset)
  PATH=/home/jima/perl5/bin​:/home/jima/bin​:/home/jima/local/bin​:/home/jima/jima_tools/x86_64/bin​:/home/jima/jima_tools/bin​:/opt/Adobe/Reader9/bin​:/usr/bin​:/bin​:/usr/sbin​:/sbin​:/usr/bin/X11​:/usr/local/sbin​:/usr/games​:/usr/local/games​:/usr/lib/jvm/java-8-oracle/bin​:/usr/lib/jvm/java-8-oracle/db/bin​:/usr/lib/jvm/java-8-oracle/jre/bin​:.
  PERL5LIB=/home/jima/perl5/lib/perl5​:/home/jima/lib/perl
  PERL_BADLANG (unset)
  PERL_LOCAL_LIB_ROOT=/home/jima/perl5
  PERL_MB_OPT=--install_base "/home/jima/perl5"
  PERL_MM_OPT=INSTALL_BASE=/home/jima/perl5
  SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2016

From @dcollinsn

Confirmed in blead.

** VALGRIND **

Die in sort comparator at 127759.pl line 11.
==25188== Invalid read of size 4
==25188== at 0x5A0B2E​: S_SvREFCNT_dec (inline.h​:162)
==25188== by 0x5A3078​: Perl_av_clear (av.c​:492)
==25188== by 0x656495​: Perl_leave_scope (scope.c​:1108)
==25188== by 0x6641BA​: Perl_dounwind (pp_ctl.c​:1527)
==25188== by 0x46EA22​: S_my_exit_jump (perl.c​:5186)
==25188== by 0x46E892​: Perl_my_failure_exit (perl.c​:5173)
==25188== by 0x66534E​: Perl_die_unwind (pp_ctl.c​:1722)
==25188== by 0x560B43​: Perl_croak_sv (util.c​:1758)
==25188== by 0x560A2C​: Perl_die_sv (util.c​:1668)
==25188== by 0x684534​: Perl_pp_die (pp_sys.c​:523)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== by 0x73D402​: S_sortcv (pp_sort.c​:1783)
==25188== Address 0x6030a40 is 0 bytes after a block of size 32 alloc'd
==25188== at 0x4C28C0F​: malloc (vg_replace_malloc.c​:299)
==25188== by 0x55BDE7​: Perl_safesysmalloc (util.c​:153)
==25188== by 0x5A12AE​: Perl_av_extend_guts (av.c​:186)
==25188== by 0x5A0EBB​: Perl_av_extend (av.c​:80)
==25188== by 0x5AEA7E​: Perl_pp_aassign (pp_hot.c​:1375)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== by 0x46282A​: S_run_body (perl.c​:2517)
==25188== by 0x461E55​: perl_run (perl.c​:2440)
==25188== by 0x41EEDD​: main (perlmain.c​:116)
==25188==
==25188== Invalid read of size 4
==25188== at 0x5F8349​: Perl_sv_free2 (sv.c​:6964)
==25188== by 0x5A0B62​: S_SvREFCNT_dec (inline.h​:166)
==25188== by 0x5A3078​: Perl_av_clear (av.c​:492)
==25188== by 0x656495​: Perl_leave_scope (scope.c​:1108)
==25188== by 0x6641BA​: Perl_dounwind (pp_ctl.c​:1527)
==25188== by 0x46EA22​: S_my_exit_jump (perl.c​:5186)
==25188== by 0x46E892​: Perl_my_failure_exit (perl.c​:5173)
==25188== by 0x66534E​: Perl_die_unwind (pp_ctl.c​:1722)
==25188== by 0x560B43​: Perl_croak_sv (util.c​:1758)
==25188== by 0x560A2C​: Perl_die_sv (util.c​:1668)
==25188== by 0x684534​: Perl_pp_die (pp_sys.c​:523)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== Address 0x6030a44 is 4 bytes after a block of size 32 alloc'd
==25188== at 0x4C28C0F​: malloc (vg_replace_malloc.c​:299)
==25188== by 0x55BDE7​: Perl_safesysmalloc (util.c​:153)
==25188== by 0x5A12AE​: Perl_av_extend_guts (av.c​:186)
==25188== by 0x5A0EBB​: Perl_av_extend (av.c​:80)
==25188== by 0x5AEA7E​: Perl_pp_aassign (pp_hot.c​:1375)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== by 0x46282A​: S_run_body (perl.c​:2517)
==25188== by 0x461E55​: perl_run (perl.c​:2440)
==25188== by 0x41EEDD​: main (perlmain.c​:116)
==25188==
==25188== Invalid read of size 4
==25188== at 0x5F8364​: Perl_sv_free2 (sv.c​:6970)
==25188== by 0x5A0B62​: S_SvREFCNT_dec (inline.h​:166)
==25188== by 0x5A3078​: Perl_av_clear (av.c​:492)
==25188== by 0x656495​: Perl_leave_scope (scope.c​:1108)
==25188== by 0x6641BA​: Perl_dounwind (pp_ctl.c​:1527)
==25188== by 0x46EA22​: S_my_exit_jump (perl.c​:5186)
==25188== by 0x46E892​: Perl_my_failure_exit (perl.c​:5173)
==25188== by 0x66534E​: Perl_die_unwind (pp_ctl.c​:1722)
==25188== by 0x560B43​: Perl_croak_sv (util.c​:1758)
==25188== by 0x560A2C​: Perl_die_sv (util.c​:1668)
==25188== by 0x684534​: Perl_pp_die (pp_sys.c​:523)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== Address 0x6030a44 is 4 bytes after a block of size 32 alloc'd
==25188== at 0x4C28C0F​: malloc (vg_replace_malloc.c​:299)
==25188== by 0x55BDE7​: Perl_safesysmalloc (util.c​:153)
==25188== by 0x5A12AE​: Perl_av_extend_guts (av.c​:186)
==25188== by 0x5A0EBB​: Perl_av_extend (av.c​:80)
==25188== by 0x5AEA7E​: Perl_pp_aassign (pp_hot.c​:1375)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== by 0x46282A​: S_run_body (perl.c​:2517)
==25188== by 0x461E55​: perl_run (perl.c​:2440)
==25188== by 0x41EEDD​: main (perlmain.c​:116)
==25188==
Attempt to free unreferenced scalar​: SV 0x6030a38 at 127759.pl line 8.
Attempt to free unreferenced scalar​: SV 0x6030a30 at 127759.pl line 8.
==25188==
==25188== HEAP SUMMARY​:
==25188== in use at exit​: 326,160 bytes in 1,190 blocks
==25188== total heap usage​: 4,007 allocs, 2,817 frees, 675,586 bytes allocated
==25188==
==25188== LEAK SUMMARY​:
==25188== definitely lost​: 0 bytes in 0 blocks
==25188== indirectly lost​: 0 bytes in 0 blocks
==25188== possibly lost​: 121,696 bytes in 119 blocks
==25188== still reachable​: 204,464 bytes in 1,071 blocks
==25188== of which reachable via heuristic​:
==25188== newarray : 1,024 bytes in 32 blocks
==25188== suppressed​: 0 bytes in 0 blocks
==25188== Rerun with --leak-check=full to see details of leaked memory
==25188==
==25188== For counts of detected and suppressed errors, rerun with​: -v
==25188== ERROR SUMMARY​: 3 errors from 3 contexts (suppressed​: 0 from 0)

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2016

From [Unknown Contact. See original ticket]

Confirmed in blead.

** VALGRIND **

Die in sort comparator at 127759.pl line 11.
==25188== Invalid read of size 4
==25188== at 0x5A0B2E​: S_SvREFCNT_dec (inline.h​:162)
==25188== by 0x5A3078​: Perl_av_clear (av.c​:492)
==25188== by 0x656495​: Perl_leave_scope (scope.c​:1108)
==25188== by 0x6641BA​: Perl_dounwind (pp_ctl.c​:1527)
==25188== by 0x46EA22​: S_my_exit_jump (perl.c​:5186)
==25188== by 0x46E892​: Perl_my_failure_exit (perl.c​:5173)
==25188== by 0x66534E​: Perl_die_unwind (pp_ctl.c​:1722)
==25188== by 0x560B43​: Perl_croak_sv (util.c​:1758)
==25188== by 0x560A2C​: Perl_die_sv (util.c​:1668)
==25188== by 0x684534​: Perl_pp_die (pp_sys.c​:523)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== by 0x73D402​: S_sortcv (pp_sort.c​:1783)
==25188== Address 0x6030a40 is 0 bytes after a block of size 32 alloc'd
==25188== at 0x4C28C0F​: malloc (vg_replace_malloc.c​:299)
==25188== by 0x55BDE7​: Perl_safesysmalloc (util.c​:153)
==25188== by 0x5A12AE​: Perl_av_extend_guts (av.c​:186)
==25188== by 0x5A0EBB​: Perl_av_extend (av.c​:80)
==25188== by 0x5AEA7E​: Perl_pp_aassign (pp_hot.c​:1375)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== by 0x46282A​: S_run_body (perl.c​:2517)
==25188== by 0x461E55​: perl_run (perl.c​:2440)
==25188== by 0x41EEDD​: main (perlmain.c​:116)
==25188==
==25188== Invalid read of size 4
==25188== at 0x5F8349​: Perl_sv_free2 (sv.c​:6964)
==25188== by 0x5A0B62​: S_SvREFCNT_dec (inline.h​:166)
==25188== by 0x5A3078​: Perl_av_clear (av.c​:492)
==25188== by 0x656495​: Perl_leave_scope (scope.c​:1108)
==25188== by 0x6641BA​: Perl_dounwind (pp_ctl.c​:1527)
==25188== by 0x46EA22​: S_my_exit_jump (perl.c​:5186)
==25188== by 0x46E892​: Perl_my_failure_exit (perl.c​:5173)
==25188== by 0x66534E​: Perl_die_unwind (pp_ctl.c​:1722)
==25188== by 0x560B43​: Perl_croak_sv (util.c​:1758)
==25188== by 0x560A2C​: Perl_die_sv (util.c​:1668)
==25188== by 0x684534​: Perl_pp_die (pp_sys.c​:523)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== Address 0x6030a44 is 4 bytes after a block of size 32 alloc'd
==25188== at 0x4C28C0F​: malloc (vg_replace_malloc.c​:299)
==25188== by 0x55BDE7​: Perl_safesysmalloc (util.c​:153)
==25188== by 0x5A12AE​: Perl_av_extend_guts (av.c​:186)
==25188== by 0x5A0EBB​: Perl_av_extend (av.c​:80)
==25188== by 0x5AEA7E​: Perl_pp_aassign (pp_hot.c​:1375)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== by 0x46282A​: S_run_body (perl.c​:2517)
==25188== by 0x461E55​: perl_run (perl.c​:2440)
==25188== by 0x41EEDD​: main (perlmain.c​:116)
==25188==
==25188== Invalid read of size 4
==25188== at 0x5F8364​: Perl_sv_free2 (sv.c​:6970)
==25188== by 0x5A0B62​: S_SvREFCNT_dec (inline.h​:166)
==25188== by 0x5A3078​: Perl_av_clear (av.c​:492)
==25188== by 0x656495​: Perl_leave_scope (scope.c​:1108)
==25188== by 0x6641BA​: Perl_dounwind (pp_ctl.c​:1527)
==25188== by 0x46EA22​: S_my_exit_jump (perl.c​:5186)
==25188== by 0x46E892​: Perl_my_failure_exit (perl.c​:5173)
==25188== by 0x66534E​: Perl_die_unwind (pp_ctl.c​:1722)
==25188== by 0x560B43​: Perl_croak_sv (util.c​:1758)
==25188== by 0x560A2C​: Perl_die_sv (util.c​:1668)
==25188== by 0x684534​: Perl_pp_die (pp_sys.c​:523)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== Address 0x6030a44 is 4 bytes after a block of size 32 alloc'd
==25188== at 0x4C28C0F​: malloc (vg_replace_malloc.c​:299)
==25188== by 0x55BDE7​: Perl_safesysmalloc (util.c​:153)
==25188== by 0x5A12AE​: Perl_av_extend_guts (av.c​:186)
==25188== by 0x5A0EBB​: Perl_av_extend (av.c​:80)
==25188== by 0x5AEA7E​: Perl_pp_aassign (pp_hot.c​:1375)
==25188== by 0x55A6BC​: Perl_runops_debug (dump.c​:2239)
==25188== by 0x46282A​: S_run_body (perl.c​:2517)
==25188== by 0x461E55​: perl_run (perl.c​:2440)
==25188== by 0x41EEDD​: main (perlmain.c​:116)
==25188==
Attempt to free unreferenced scalar​: SV 0x6030a38 at 127759.pl line 8.
Attempt to free unreferenced scalar​: SV 0x6030a30 at 127759.pl line 8.
==25188==
==25188== HEAP SUMMARY​:
==25188== in use at exit​: 326,160 bytes in 1,190 blocks
==25188== total heap usage​: 4,007 allocs, 2,817 frees, 675,586 bytes allocated
==25188==
==25188== LEAK SUMMARY​:
==25188== definitely lost​: 0 bytes in 0 blocks
==25188== indirectly lost​: 0 bytes in 0 blocks
==25188== possibly lost​: 121,696 bytes in 119 blocks
==25188== still reachable​: 204,464 bytes in 1,071 blocks
==25188== of which reachable via heuristic​:
==25188== newarray : 1,024 bytes in 32 blocks
==25188== suppressed​: 0 bytes in 0 blocks
==25188== Rerun with --leak-check=full to see details of leaked memory
==25188==
==25188== For counts of detected and suppressed errors, rerun with​: -v
==25188== ERROR SUMMARY​: 3 errors from 3 contexts (suppressed​: 0 from 0)

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2016

From @iabyn

On Mon, Mar 21, 2016 at 03​:20​:31PM -0700, via RT wrote​:

# New Ticket Created by
# Please include the string​: [perl #127759]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=127759 >

This is a bug report for perl from jim.avera@​gmail.com,
generated with the help of perlbug 1.40 running under perl 5.20.2.

-----------------------------------------------------------------
If die is called inside a sort comparitor function, errors like the
following (sometimes) occur.

Attempt to free unreferenced scalar​: SV 0x12c7458, Perl interpreter​: 0x12a2010 at (location of die)

#!/usr/bin/perl
use warnings; use strict; use 5.010;

my @​data = ( ["A"], ["B"], ["C"] );

@​data =
sort {
my $aval = $a->[0];
my $bval = $b->[0];

die "Die in sort comparator" if $aval eq "C"; 

return \($aval cmp $bval\);

}
@​data;

# Die in sort comparator at /tmp/test.pl line 11.
# Attempt to free unreferenced scalar​: SV 0x12c7458, Perl interpreter​: 0x12a2010 at /tmp/test.pl line 11.
# Attempt to free unreferenced scalar​: SV 0x12c7450, Perl interpreter​: 0x12a2010 at /tmp/test.pl line 11.
#

This is an interaction between sort's 'in place' optimisation, and perl's
implementation of the mergesort algorithm.

In something like

  @​a = sort {...} @​a,
 
where the same array is on both sides, the assign is optimised away and
sort() just sorts the array of SV pointers in @​a's AvARRAY() directly.
But it turns out that (AFAIKT) S_mergesortsv() temporarily stores random
pointers in sort slots (rather than only pointers to the SVs being
sorted). If the compare sub dies in mid-sort, random rubbish gets left in
the AvARRAY array, which then causes crashes when the array falls out of
scope.

The merge sort code is a bit impenetrable, so I'm unsure as yet of the
best way to fix this.

--
A major Starfleet emergency breaks out near the Enterprise, but
fortunately some other ships in the area are able to deal with it to
everyone's satisfaction.
  -- Things That Never Happen in "Star Trek" #13

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2016

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2016

From zefram@fysh.org

Dave Mitchell wrote​:

     If the compare sub dies in mid\-sort\, random rubbish gets left in

the AvARRAY array, which then causes crashes when the array falls out of
scope.

Even if random rubbish were not there, this implies another bad
interaction between in-place sorting and comparator code that can die.
If the comparator code dies with the AvARRAY well-formed, it will contain
a partially-sorted version of the array content, but the semantics of the
Perl statement as written do not allow for the array to ever have that
content. As written, all the sorting is done before any mutation of the
array. So while the comparator code is running, and after the exception
is thrown if it dies, the array should have its original content. So if
it is possible for the comparator code either to examine the array being
sorted or to die then it is not correct to apply this optimisation.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2016

From @cpansprout

On Tue Jun 07 12​:48​:51 2016, zefram@​fysh.org wrote​:

Dave Mitchell wrote​:

     If the compare sub dies in mid\-sort\, random rubbish gets left in

the AvARRAY array, which then causes crashes when the array falls out of
scope.

Even if random rubbish were not there, this implies another bad
interaction between in-place sorting and comparator code that can die.
If the comparator code dies with the AvARRAY well-formed, it will contain
a partially-sorted version of the array content, but the semantics of the
Perl statement as written do not allow for the array to ever have that
content. As written, all the sorting is done before any mutation of the
array. So while the comparator code is running, and after the exception
is thrown if it dies, the array should have its original content. So if
it is possible for the comparator code either to examine the array being
sorted or to die then it is not correct to apply this optimisation.

Is it really a win if we disable an optimisation that helps more often than it hinders? Currently, the array is made read-only to prevent it from being written too. That change was made in response to a bug report, in which some code *accidentally* overwrote the array, due to a programming error.

I do like making things correct, but not at the cost of slowing down correct code that would not suffer from the bugs fixed (though I am not always consistent​:-). Is a compromise possible?

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2016

From zefram@fysh.org

Father Chrysostomos via RT wrote​:

Is it really a win if we disable an optimisation that helps more often
than it hinders?

The dilemma is cause for us to lament how unsuited to optimisation
Perl is.

Some of the win from in-place optimisation, I suspect most of it, comes
not from avoiding copying of the AvARRAY but from not creating new element
scalars. (That's subject to [perl #128340] which I've just reported.)
I suspect that we could get both the correct semantics and most of the
performance win by retaining the special-case code for the high-level
in-place case and just making it slightly less low-level in-place by
performing one copy of the whole AvARRAY. In the fairly-common cases
where the comparator code is detectably innocuous, at least if it's just {
$a cmp $b } or { $a <=> $b }, we can still skip that copy.

If that doesn't satisfactorily win, then we get to agonise again about the
true nature of winning. I'm not going to approve of it giving the wrong
answer, but it wouldn't be the first time that the consensus has been that
optimisation of the common case outweighs correctness of an uncommon case.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2016

From @mauke

Am 07.06.2016 um 22​:16 schrieb Zefram​:

In the fairly-common cases
where the comparator code is detectably innocuous, at least if it's just {
$a cmp $b } or { $a <=> $b }, we can still skip that copy.

If one of the array elements is an object that uses overloading but
doesn't implement <=> (or implements it as a method that dies), then you
still lose.

"detectably innocuous" may not exist in Perl.

--
Lukas Mai <plokinom@​gmail.com>

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2016

From gm@qwurx.de

From the keyboard of Zefram [07.06.16,21​:16]​:

Father Chrysostomos via RT wrote​:

Is it really a win if we disable an optimisation that helps more often
than it hinders?

The dilemma is cause for us to lament how unsuited to optimisation
Perl is.

Some of the win from in-place optimisation, I suspect most of it, comes
not from avoiding copying of the AvARRAY but from not creating new element
scalars. (That's subject to [perl #128340] which I've just reported.)
I suspect that we could get both the correct semantics and most of the
performance win by retaining the special-case code for the high-level
in-place case and just making it slightly less low-level in-place by
performing one copy of the whole AvARRAY. In the fairly-common cases
where the comparator code is detectably innocuous, at least if it's just {
$a cmp $b } or { $a <=> $b }, we can still skip that copy.

Even for the trivial case of { $a <=> $b }, the compiler would have to
check for warnings enabled in the current frame and a __WARN__ handler
(not numeric in comparison) installed which could die for multiple reasons.

So, the optimizer would have to check the complete code path which could
possible affect even the trivial cases.

If that doesn't satisfactorily win, then we get to agonise again about the
true nature of winning. I'm not going to approve of it giving the wrong
answer, but it wouldn't be the first time that the consensus has been that
optimisation of the common case outweighs correctness of an uncommon case.

-zefram

0--gg-

--
_($_=" "x(1<<5)."?\n".q·/)Oo. G°\ /
  /\_¯/(q /
---------------------------- \__(m.====·.(_("always off the crowd"))."·
");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}

@p5pRT
Copy link
Author

p5pRT commented Jun 8, 2016

From @ikegami

On Tue, Jun 7, 2016 at 4​:16 PM, Zefram <zefram@​fysh.org> wrote​:

In the fairly-common cases
where the comparator code is detectably innocuous, at least if it's just {
$a cmp $b } or { $a <=> $b }, we can still skip that copy.

Those aren't necessarily innocuous because of magic scalars and overloaded
operators.

@p5pRT
Copy link
Author

p5pRT commented Jun 8, 2016

From @cpansprout

On Wed Jun 08 11​:06​:18 2016, ikegami@​adaelis.com wrote​:

On Tue, Jun 7, 2016 at 4​:16 PM, Zefram <zefram@​fysh.org> wrote​:

In the fairly-common cases
where the comparator code is detectably innocuous, at least if it's just {
$a cmp $b } or { $a <=> $b }, we can still skip that copy.

Those aren't necessarily innocuous because of magic scalars and overloaded
operators.

Note that pp_sort already looks for magic and overloading at the outset and follows different code paths accordingly.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Aug 10, 2016

From @iabyn

On Wed, Jun 08, 2016 at 01​:13​:10PM -0700, Father Chrysostomos via RT wrote​:

On Wed Jun 08 11​:06​:18 2016, ikegami@​adaelis.com wrote​:

On Tue, Jun 7, 2016 at 4​:16 PM, Zefram <zefram@​fysh.org> wrote​:

In the fairly-common cases
where the comparator code is detectably innocuous, at least if it's just {
$a cmp $b } or { $a <=> $b }, we can still skip that copy.

Those aren't necessarily innocuous because of magic scalars and overloaded
operators.

Note that pp_sort already looks for magic and overloading at the outset and follows different code paths accordingly.

Now (mostly) fixed by v5.25.3-188-g84721d6.

--
My get-up-and-go just got up and went.

@p5pRT
Copy link
Author

p5pRT commented Aug 11, 2016

@iabyn - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented May 30, 2017

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.26.0, this and 210 other issues have been
resolved.

Perl 5.26.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.26.0

If you find that the problem persists, feel free to reopen this ticket.

@p5pRT
Copy link
Author

p5pRT commented May 30, 2017

@khwilliamson - Status changed from 'pending release' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant