Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PATCH] Bring back test.deparse for smoking #13567

Closed
p5pRT opened this issue Jan 31, 2014 · 9 comments
Closed

[PATCH] Bring back test.deparse for smoking #13567

p5pRT opened this issue Jan 31, 2014 · 9 comments
Labels
distro-Linux hasPatch type-core

Comments

@p5pRT
Copy link

p5pRT commented Jan 31, 2014

Migrated from rt.perl.org#121126 (status was 'resolved')

Searchable as RT121126$
@p5pRT
Copy link
Author

p5pRT commented Jan 31, 2014

From @wolfsage

Created by wolfsage@gmail.com

Added back in test.deparse test target.

It now runs all tests with t/TEST, and supports​:

  * Skipping known bad tests entirely without running them
  * Running tests that are expected to fail, and "skipping" them if they do
  Or, if they pass, throwing an error to let us know the test now works

This should allow us to smoke it and make sure things don't get any worse,
while allowing us to fix things as we have time.

Perl Info 

Flags:
    category=core
    severity=low

Site configuration information for perl 5.14.2:

Configured by Debian Project at Mon Mar 18 19:16:26 UTC 2013.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:

  Platform:
    osname=linux, osvers=2.6.42-37-generic,
archname=x86_64-linux-gnu-thread-multi
    uname='linux batsu 2.6.42-37-generic #58-ubuntu smp thu jan 24
15:28:10 utc 2013 x86_64 x86_64 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.14 -Darchlib=/usr/lib/perl/5.14
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.14.2
-Dsitearch=/usr/local/lib/perl/5.14.2 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1
-Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh
-Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -DDEBUGGING=-g -Doptimize=-O2
-Duseshrplib -Dlibperl=libperl.so.5.14.2 -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.6.3', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=, so=so, useshrplib=true, libperl=libperl.so.5.14.2
    gnulibc_version='2.15'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib
-fstack-protector'

Locally applied patches:



@INC for perl 5.14.2:
    /etc/perl
    /usr/local/lib/perl/5.14.2
    /usr/local/share/perl/5.14.2
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.14
    /usr/share/perl/5.14
    /usr/local/lib/site_perl
    .


Environment for perl 5.14.2:
    HOME=/home/mhorsfall
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/mhorsfall/perl5/perlbrew/bin:/home/mhorsfall/bin:/home/mhorsfall/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
    PERLBREW_BASHRC_VERSION=0.66
    PERLBREW_HOME=/home/mhorsfall/.perlbrew
    PERLBREW_ROOT=/home/mhorsfall/perl5/perlbrew
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Jan 31, 2014

From @wolfsage

0001-Bring-back-test.deparse-make-target-and-make-it-smar.patch 
From 59ba3e1820ed5a546c5ac642f7c7a066381cca1d Mon Sep 17 00:00:00 2001
From: Matthew Horsfall <WolfSage@gmail.com>
Date: Thu, 30 Jan 2014 22:31:52 -0500
Subject: [PATCH] Bring back test.deparse make target, and make it smarter.

t/TEST now allows skipping / ignoring test failures under test.deparse.

 * Known bad tests can be skipped entirely (not loaded/run)
 * Known failing tests can be run and checked for a change in state.

   - If they fail as expected, they are skipped
   - If they pass, the test is forced to fail so the report at the end is
     useful.
---
 Makefile.SH      |   12 +-
 pod/perlhack.pod |    6 +
 t/TEST           |  587 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
 t/cmd/mod.t      |    3 +-
 4 files changed, 600 insertions(+), 8 deletions(-)

diff --git a/Makefile.SH b/Makefile.SH
index b122526..2f4ff53 100755
--- a/Makefile.SH
+++ b/Makefile.SH
@@ -1392,7 +1392,8 @@ depend: makedepend
 
 .PHONY: test check test_prep test_prep_nodll test_prep_pre \
 	test_prep_reonly test_tty test-tty test_notty test-notty \
-	test_harness test_harness_notty minitest test-reonly _test
+	test_harness test_harness_notty minitest test-reonly _test \
+	test.deparse test_notty.deparse
 
 _test:
 	echo >&2 The _test target is deprecated. Please upgrade your smoker
@@ -1497,6 +1498,15 @@ test_tty test-tty: test_prep
 test_notty test-notty: test_prep
 	$(RUN_TESTS) no-tty
 
+# Targets for Deparse testing.
+test.deparse:  test_prep
+	@echo "DOING IT"
+	TEST_ARGS=-deparse $(RUN_TESTS) choose
+	@echo "DONE"
+
+test_notty.deparse: test_prep
+	TEST_ARGS=-deparse $(RUN_TESTS) no-tty
+
 !NO!SUBS!
 
 case "${osname}${osvers}" in
diff --git a/pod/perlhack.pod b/pod/perlhack.pod
index dc3f396..c6479af 100644
--- a/pod/perlhack.pod
+++ b/pod/perlhack.pod
@@ -793,6 +793,12 @@ systems.
 This runs some basic sanity tests on the source tree and helps catch
 basic errors before you submit a patch.
 
+=item * test.deparse test_notty.deparse
+
+Run all the tests through L<B::Deparse>. Not all tests will succeed.
+
+(Not available on Win32)
+
 =item * minitest
 
 Run F<miniperl> on F<t/base>, F<t/comp>, F<t/cmd>, F<t/run>, F<t/io>,
diff --git a/t/TEST b/t/TEST
index 96eb6a4..315ccca 100755
--- a/t/TEST
+++ b/t/TEST
@@ -14,6 +14,11 @@
 # In which case, we need to stop t/TEST actually running tests, as all
 # t/harness needs are its subroutines.
 
+# If we're doing deparse tests, ignore failures for these
+my $deparse_failures;
+
+# And skip even running these
+my $deparse_skips;
 
 # directories with special sets of test switches
 my %dir_to_switch =
@@ -126,6 +131,7 @@ our $show_elapsed_time = $ENV{HARNESS_TIMER} || 0;
 	if ($1 =~ /^deparse(,.+)?$/) {
 	    $::deparse = 1;
 	    $::deparse_opts = $1;
+	    _process_deparse_config();
 	}
     }
     @ARGV = @argv;
@@ -536,12 +542,7 @@ EOT
 	    next;
 	}
 	if ($type eq 'deparse') {
-	    if ($test eq "comp/redef.t") {
-		# Redefinition happens at compile time
-		next;
-	    }
-	    elsif ($test =~ m{lib/Switch/t/}) {
-		# B::Deparse doesn't support source filtering
+	    if ($test =~ $deparse_skips) {
 		next;
 	    }
 	}
@@ -722,6 +723,19 @@ EOT
 	    $failure = "FAILED--non-zero wait status: $?";
 	}
 
+	# Deparse? Should it have passed or failed?
+	if ($type eq 'deparse' && $test =~ $deparse_failures) {
+	    if (!$failure) {
+		# Wait, it didn't fail? Great news! Tell someone!
+		$failure = "FAILED--all tests passed but test should have failed";
+
+	    } else {
+		# Bah, still failing. Mask it.
+		print "${te}skipped\n";
+		$tested_files = $tested_files - 1;
+		next;
+	    }
+	}		
 	if (defined $failure) {
 	    print "${te}$failure\n";
 	    $::bad_files = $::bad_files + 1;
@@ -874,4 +888,565 @@ sub gather_conf_platform_info {
 	);
 }
 
+# Generate regexps of known bad filenames / skips from __DATA__ below
+my $in;
+
+sub _process_deparse_config {
+    my @deparse_failures;
+    my @deparse_skips;
+
+    while(<DATA>) {
+        if (/__DEPARSE_FAILURES__/) {
+            $in = \@deparse_failures; next;
+        } elsif (/__DEPARSE_SKIPS__/) {
+            $in = \@deparse_skips; next;
+        } elsif ($in && /^__/) {
+            last; # Skip some new section
+        } elsif (!$in) {
+            next;
+	}
+
+        s/#.*$//; # Kill comments
+        s/\s+$//; # And trailing whitespace
+
+        next unless $_;
+
+        push @$in, $_;
+    }
+
+    for my $f (@deparse_failures, @deparse_skips) {
+        if ($f =~ m|/$|) { # Dir? Skip everything below it
+            $f = qr/\Q$f\E.*/;
+        } else {
+            $f = qr/\Q$f\E/;
+        }
+    }
+
+    $deparse_failures = join('|', @deparse_failures);
+    $deparse_failures = qr/^(?:$deparse_failures)$/;
+
+    $deparse_skips = join('|', @deparse_skips);
+    $deparse_skips = qr/^(?:$deparse_skips)$/;
+}
+
 # ex: set ts=8 sts=4 sw=4 noet:
+
+__DATA__
+
+__DEPARSE_FAILURES__
+
+../cpan/AutoLoader/t/01AutoLoader.t
+../cpan/CGI/t/utf8.t
+../cpan/CPAN-Meta/t/converter.t
+../cpan/Digest-MD5/t/utf8.t
+../cpan/Encode/t/Encode.t
+../cpan/Encode/t/Unicode.t
+../cpan/Encode/t/at-cn.t
+../cpan/Encode/t/at-tw.t
+../cpan/Encode/t/enc_data.t
+../cpan/Encode/t/enc_eucjp.t
+../cpan/Encode/t/enc_module.t
+../cpan/Encode/t/enc_utf8.t
+../cpan/Encode/t/encoding.t
+../cpan/Encode/t/fallback.t
+../cpan/Encode/t/from_to.t
+../cpan/Encode/t/guess.t
+../cpan/Encode/t/jis7-fallback.t
+../cpan/Encode/t/jperl.t
+../cpan/Encode/t/mime-header.t
+../cpan/ExtUtils-MakeMaker/t/arch_check.t
+../cpan/ExtUtils-MakeMaker/t/min_perl_version.t
+../cpan/ExtUtils-MakeMaker/t/xs.t
+../cpan/File-Path/t/taint.t
+../cpan/File-Temp/t/object.t
+../cpan/HTTP-Tiny/t/001_api.t
+../cpan/IO-Compress/t/050interop-gzip.t
+../cpan/IO-Compress/t/cz-08encoding.t
+../cpan/JSON-PP/t/001_utf8.t
+../cpan/JSON-PP/t/109_encode.t
+../cpan/MIME-Base64/t/unicode.t
+../cpan/Math-Complex/t/Trig.t
+../cpan/Module-Build/t/manifypods_with_utf8.t
+../cpan/Module-Metadata/t/encoding.t
+../cpan/Pod-Perldoc/t/man/_get_columns.t
+../cpan/Pod-Simple/t/stree.t
+../cpan/Scalar-List-Utils/t/dualvar.t
+../cpan/Scalar-List-Utils/t/first.t
+../cpan/Scalar-List-Utils/t/reduce.t
+../cpan/Scalar-List-Utils/t/weak.t
+../cpan/Socket/t/sockaddr.t
+../cpan/Term-ANSIColor/t/taint.t
+../cpan/Test-Harness/t/parser-config.t
+../cpan/Test-Harness/t/parser-subclass.t
+../cpan/Test-Harness/t/proverun.t
+../cpan/Test-Simple/t/Builder/carp.t
+../cpan/Test-Simple/t/fail-more.t
+../cpan/Test-Simple/t/is_deeply_fail.t
+../cpan/Test-Simple/t/new_ok.t
+../cpan/Test-Simple/t/plan.t
+../cpan/Test-Simple/t/plan_bad.t
+../cpan/Test-Simple/t/skip.t
+../cpan/Test-Simple/t/subtest/line_numbers.t
+../cpan/Test-Simple/t/subtest/predicate.t
+../cpan/Test-Simple/t/todo.t
+../cpan/Text-Tabs/t/Tabs-ElCid.t
+../cpan/Text-Tabs/t/Wrap-JLB.t
+../cpan/Unicode-Collate/t/altern.t
+../cpan/Unicode-Collate/t/backwds.t
+../cpan/Unicode-Collate/t/cjk_b5.t
+../cpan/Unicode-Collate/t/cjk_gb.t
+../cpan/Unicode-Collate/t/cjk_ja.t
+../cpan/Unicode-Collate/t/cjk_ko.t
+../cpan/Unicode-Collate/t/cjk_py.t
+../cpan/Unicode-Collate/t/cjk_st.t
+../cpan/Unicode-Collate/t/cjk_zy.t
+../cpan/Unicode-Collate/t/cjkrange.t
+../cpan/Unicode-Collate/t/compatui.t
+../cpan/Unicode-Collate/t/contract.t
+../cpan/Unicode-Collate/t/default.t
+../cpan/Unicode-Collate/t/hangul.t
+../cpan/Unicode-Collate/t/ident.t
+../cpan/Unicode-Collate/t/iglevel2.t
+../cpan/Unicode-Collate/t/ignor.t
+../cpan/Unicode-Collate/t/illegal.t
+../cpan/Unicode-Collate/t/index.t
+../cpan/Unicode-Collate/t/loc_af.t
+../cpan/Unicode-Collate/t/loc_ar.t
+../cpan/Unicode-Collate/t/loc_as.t
+../cpan/Unicode-Collate/t/loc_az.t
+../cpan/Unicode-Collate/t/loc_be.t
+../cpan/Unicode-Collate/t/loc_bg.t
+../cpan/Unicode-Collate/t/loc_bn.t
+../cpan/Unicode-Collate/t/loc_bs.t
+../cpan/Unicode-Collate/t/loc_bscy.t
+../cpan/Unicode-Collate/t/loc_ca.t
+../cpan/Unicode-Collate/t/loc_cjk.t
+../cpan/Unicode-Collate/t/loc_cjkc.t
+../cpan/Unicode-Collate/t/loc_cs.t
+../cpan/Unicode-Collate/t/loc_cyrl.t
+../cpan/Unicode-Collate/t/loc_da.t
+../cpan/Unicode-Collate/t/loc_de.t
+../cpan/Unicode-Collate/t/loc_deph.t
+../cpan/Unicode-Collate/t/loc_ee.t
+../cpan/Unicode-Collate/t/loc_eo.t
+../cpan/Unicode-Collate/t/loc_es.t
+../cpan/Unicode-Collate/t/loc_estr.t
+../cpan/Unicode-Collate/t/loc_et.t
+../cpan/Unicode-Collate/t/loc_fa.t
+../cpan/Unicode-Collate/t/loc_fi.t
+../cpan/Unicode-Collate/t/loc_fil.t
+../cpan/Unicode-Collate/t/loc_fiph.t
+../cpan/Unicode-Collate/t/loc_fo.t
+../cpan/Unicode-Collate/t/loc_fr.t
+../cpan/Unicode-Collate/t/loc_gu.t
+../cpan/Unicode-Collate/t/loc_ha.t
+../cpan/Unicode-Collate/t/loc_haw.t
+../cpan/Unicode-Collate/t/loc_hi.t
+../cpan/Unicode-Collate/t/loc_hr.t
+../cpan/Unicode-Collate/t/loc_hu.t
+../cpan/Unicode-Collate/t/loc_hy.t
+../cpan/Unicode-Collate/t/loc_ig.t
+../cpan/Unicode-Collate/t/loc_is.t
+../cpan/Unicode-Collate/t/loc_ja.t
+../cpan/Unicode-Collate/t/loc_jait.t
+../cpan/Unicode-Collate/t/loc_japr.t
+../cpan/Unicode-Collate/t/loc_kk.t
+../cpan/Unicode-Collate/t/loc_kl.t
+../cpan/Unicode-Collate/t/loc_kn.t
+../cpan/Unicode-Collate/t/loc_ko.t
+../cpan/Unicode-Collate/t/loc_kok.t
+../cpan/Unicode-Collate/t/loc_ln.t
+../cpan/Unicode-Collate/t/loc_lt.t
+../cpan/Unicode-Collate/t/loc_lv.t
+../cpan/Unicode-Collate/t/loc_mk.t
+../cpan/Unicode-Collate/t/loc_ml.t
+../cpan/Unicode-Collate/t/loc_mr.t
+../cpan/Unicode-Collate/t/loc_mt.t
+../cpan/Unicode-Collate/t/loc_nb.t
+../cpan/Unicode-Collate/t/loc_nn.t
+../cpan/Unicode-Collate/t/loc_nso.t
+../cpan/Unicode-Collate/t/loc_or.t
+../cpan/Unicode-Collate/t/loc_pa.t
+../cpan/Unicode-Collate/t/loc_pl.t
+../cpan/Unicode-Collate/t/loc_ro.t
+../cpan/Unicode-Collate/t/loc_ru.t
+../cpan/Unicode-Collate/t/loc_sa.t
+../cpan/Unicode-Collate/t/loc_se.t
+../cpan/Unicode-Collate/t/loc_si.t
+../cpan/Unicode-Collate/t/loc_sidt.t
+../cpan/Unicode-Collate/t/loc_sk.t
+../cpan/Unicode-Collate/t/loc_sl.t
+../cpan/Unicode-Collate/t/loc_sq.t
+../cpan/Unicode-Collate/t/loc_sr.t
+../cpan/Unicode-Collate/t/loc_srla.t
+../cpan/Unicode-Collate/t/loc_sv.t
+../cpan/Unicode-Collate/t/loc_svrf.t
+../cpan/Unicode-Collate/t/loc_ta.t
+../cpan/Unicode-Collate/t/loc_te.t
+../cpan/Unicode-Collate/t/loc_test.t
+../cpan/Unicode-Collate/t/loc_th.t
+../cpan/Unicode-Collate/t/loc_tn.t
+../cpan/Unicode-Collate/t/loc_to.t
+../cpan/Unicode-Collate/t/loc_tr.t
+../cpan/Unicode-Collate/t/loc_uk.t
+../cpan/Unicode-Collate/t/loc_ur.t
+../cpan/Unicode-Collate/t/loc_vi.t
+../cpan/Unicode-Collate/t/loc_wae.t
+../cpan/Unicode-Collate/t/loc_wo.t
+../cpan/Unicode-Collate/t/loc_yo.t
+../cpan/Unicode-Collate/t/loc_zh.t
+../cpan/Unicode-Collate/t/loc_zhb5.t
+../cpan/Unicode-Collate/t/loc_zhgb.t
+../cpan/Unicode-Collate/t/loc_zhpy.t
+../cpan/Unicode-Collate/t/loc_zhst.t
+../cpan/Unicode-Collate/t/loc_zhzy.t
+../cpan/Unicode-Collate/t/nonchar.t
+../cpan/Unicode-Collate/t/normal.t
+../cpan/Unicode-Collate/t/notable.t
+../cpan/Unicode-Collate/t/overcjk0.t
+../cpan/Unicode-Collate/t/overcjk1.t
+../cpan/Unicode-Collate/t/override.t
+../cpan/Unicode-Collate/t/rearrang.t
+../cpan/Unicode-Collate/t/rewrite.t
+../cpan/Unicode-Collate/t/test.t
+../cpan/Unicode-Collate/t/trailwt.t
+../cpan/Unicode-Collate/t/variable.t
+../cpan/Unicode-Collate/t/view.t
+../cpan/Unicode-Normalize/t/fcdc.t
+../cpan/Unicode-Normalize/t/form.t
+../cpan/Unicode-Normalize/t/func.t
+../cpan/Unicode-Normalize/t/norm.t
+../cpan/Unicode-Normalize/t/partial1.t
+../cpan/Unicode-Normalize/t/partial2.t
+../cpan/Unicode-Normalize/t/proto.t
+../cpan/Unicode-Normalize/t/split.t
+../cpan/Unicode-Normalize/t/test.t
+../cpan/autodie/t/00-load.t
+../cpan/autodie/t/autodie.t
+../cpan/autodie/t/blog_hints.t
+../cpan/autodie/t/caller.t
+../cpan/autodie/t/chmod.t
+../cpan/autodie/t/chown.t
+../cpan/autodie/t/context.t
+../cpan/autodie/t/context_lexical.t
+../cpan/autodie/t/crickey.t
+../cpan/autodie/t/dbmopen.t
+../cpan/autodie/t/eval_error.t
+../cpan/autodie/t/exception_class.t
+../cpan/autodie/t/exceptions.t
+../cpan/autodie/t/exec.t
+../cpan/autodie/t/filehandles.t
+../cpan/autodie/t/format-clobber.t
+../cpan/autodie/t/hints.t
+../cpan/autodie/t/hints_insist.t
+../cpan/autodie/t/hints_pod_examples.t
+../cpan/autodie/t/hints_provider_does.t
+../cpan/autodie/t/hints_provider_easy_does_it.t
+../cpan/autodie/t/hints_provider_isa.t
+../cpan/autodie/t/kill.t
+../cpan/autodie/t/lethal.t
+../cpan/autodie/t/open.t
+../cpan/autodie/t/recv.t
+../cpan/autodie/t/repeat.t
+../cpan/autodie/t/scope_leak.t
+../cpan/autodie/t/sysopen.t
+../cpan/autodie/t/user-context.t
+../cpan/autodie/t/usersub.t
+../cpan/autodie/t/utf8_open.t
+../cpan/autodie/t/utime.t
+../cpan/autodie/t/version_tag.t
+../cpan/encoding-warnings/t/4-lexical.t
+../cpan/podlators/t/basic.t
+../cpan/version/t/09_list_util.t
+../dist/Attribute-Handlers/t/constants.t
+../dist/Attribute-Handlers/t/data_convert.t
+../dist/Attribute-Handlers/t/linerep.t
+../dist/Attribute-Handlers/t/multi.t
+../dist/Carp/t/Carp.t
+../dist/Carp/t/arg_regexp.t
+../dist/Carp/t/arg_string.t
+../dist/Data-Dumper/t/dumper.t
+../dist/Data-Dumper/t/perl-74170.t
+../dist/Data-Dumper/t/quotekeys.t
+../dist/Exporter/t/Exporter.t
+../dist/ExtUtils-Install/t/Installapi2.t
+../dist/ExtUtils-Install/t/Packlist.t
+../dist/ExtUtils-Install/t/can_write_dir.t
+../dist/ExtUtils-Manifest/t/Manifest.t
+../dist/Filter-Simple/t/data.t
+../dist/I18N-LangTags/t/50_super.t
+../dist/IO/t/io_file_export.t
+../dist/IO/t/io_multihomed.t
+../dist/IO/t/io_sel.t
+../dist/IO/t/io_sock.t
+../dist/IO/t/io_udp.t
+../dist/IO/t/io_utf8.t
+../dist/Locale-Maketext/t/01_about_verbose.t
+../dist/Locale-Maketext/t/10_make.t
+../dist/Locale-Maketext/t/20_get.t
+../dist/Locale-Maketext/t/30_eval_dollar_at.t
+../dist/Locale-Maketext/t/40_super.t
+../dist/Locale-Maketext/t/50_super.t
+../dist/Locale-Maketext/t/60_super.t
+../dist/Locale-Maketext/t/70_fail_auto.t
+../dist/Locale-Maketext/t/90_utf8.t
+../dist/Locale-Maketext/t/91_backslash.t
+../dist/Math-BigInt/t/const_mbf.t
+../dist/Math-BigInt/t/constant.t
+../dist/PathTools/t/cwd.t
+../dist/Storable/t/blessed.t
+../dist/Storable/t/croak.t
+../dist/Storable/t/downgrade.t
+../dist/Storable/t/malice.t
+../dist/Storable/t/utf8.t
+../dist/Term-ReadLine/t/ReadLine.t
+../dist/Thread-Queue/t/08_nothreads.t
+../dist/Tie-File/t/29_downcopy.t
+../dist/Tie-File/t/42_offset.t
+../dist/bignum/t/big_e_pi.t
+../dist/bignum/t/bigexp.t
+../dist/bignum/t/bigint.t
+../dist/bignum/t/bignum.t
+../dist/bignum/t/bigrat.t
+../dist/bignum/t/bii_e_pi.t
+../dist/bignum/t/bir_e_pi.t
+../dist/bignum/t/in_effect.t
+../dist/bignum/t/option_a.t
+../dist/bignum/t/option_l.t
+../dist/bignum/t/option_p.t
+../dist/bignum/t/overrides.t
+../dist/bignum/t/ratopt_a.t
+../dist/bignum/t/scope_f.t
+../dist/bignum/t/scope_i.t
+../dist/bignum/t/scope_r.t
+../dist/constant/t/constant.t
+../dist/threads/t/err.t
+../dist/threads/t/exit.t
+../dist/threads/t/kill2.t
+../dist/threads/t/libc.t
+../dist/threads/t/thread.t
+../ext/B/t/b.t
+../ext/B/t/optree_constants.t
+../ext/B/t/optree_samples.t
+../ext/B/t/pragma.t
+../ext/B/t/xref.t
+../ext/Devel-Peek/t/Peek.t
+../ext/File-Glob/t/basic.t
+../ext/File-Glob/t/taint.t
+../ext/Hash-Util-FieldHash/t/02_function.t
+../ext/Hash-Util-FieldHash/t/11_hashassign.t
+../ext/Hash-Util/t/Util.t
+../ext/IPC-Open3/t/IPC-Open2.t
+../ext/IPC-Open3/t/IPC-Open3.t
+../ext/Opcode/t/Opcode.t
+../ext/POSIX/t/termios.t
+../ext/PerlIO-encoding/t/encoding.t
+../ext/PerlIO-encoding/t/fallback.t
+../ext/PerlIO-scalar/t/scalar.t
+../ext/PerlIO-via/t/via.t
+../ext/XS-APItest/t/autoload.t
+../ext/XS-APItest/t/blockhooks-csc.t
+../ext/XS-APItest/t/blockhooks.t
+../ext/XS-APItest/t/call_checker.t
+../ext/XS-APItest/t/caller.t
+../ext/XS-APItest/t/cleanup.t
+../ext/XS-APItest/t/cophh.t
+../ext/XS-APItest/t/fetch_pad_names.t
+../ext/XS-APItest/t/lexsub.t
+../ext/XS-APItest/t/multicall.t
+../ext/XS-APItest/t/overload.t
+../ext/XS-APItest/t/svpeek.t
+../ext/XS-APItest/t/svpv.t
+../ext/XS-APItest/t/underscore_length.t
+../ext/XS-APItest/t/xsub_h.t
+../ext/arybase/t/aeach.t
+../ext/arybase/t/aelem.t
+../ext/arybase/t/aslice.t
+../ext/arybase/t/av2arylen.t
+../ext/arybase/t/lslice.t
+../ext/arybase/t/scope.t
+../ext/arybase/t/splice.t
+../ext/re/t/reflags.t
+../lib/B/Deparse.t
+../lib/DB.t
+../lib/DBM_Filter/t/01error.t
+../lib/DBM_Filter/t/02core.t
+../lib/DBM_Filter/t/compress.t
+../lib/DBM_Filter/t/encode.t
+../lib/DBM_Filter/t/int32.t
+../lib/DBM_Filter/t/null.t
+../lib/DBM_Filter/t/utf8.t
+../lib/English.t
+../lib/File/Basename.t
+../lib/Getopt/Std.t
+../lib/Unicode/UCD.t
+../lib/bytes.t
+../lib/charnames.t
+../lib/feature/unicode_strings.t
+../lib/less.t
+../lib/locale.t
+../lib/overload.t
+../lib/overloading.t
+../lib/utf8.t
+base/lex.t
+comp/final_line_num.t
+comp/fold.t
+comp/form_scope.t
+comp/hints.t
+comp/opsubs.t
+comp/parser.t
+comp/proto.t
+comp/require.t
+io/inplace.t
+io/utf8.t
+lib/deprecate.t
+lib/mypragma.t
+mro/basic.t
+mro/basic_utf8.t
+mro/dbic_c3.t
+mro/dbic_c3_utf8.t
+mro/dbic_dfs.t
+mro/dbic_dfs_utf8.t
+mro/inconsistent_c3.t
+mro/inconsistent_c3_utf8.t
+mro/isarev.t
+mro/isarev_utf8.t
+mro/method_caching.t
+mro/method_caching_utf8.t
+mro/next_edgecases.t
+mro/next_edgecases_utf8.t
+mro/next_goto.t
+mro/next_goto_utf8.t
+mro/package_aliases_utf8.t
+mro/pkg_gen.t
+mro/pkg_gen_utf8.t
+mro/recursion_c3.t
+mro/recursion_c3_utf8.t
+mro/recursion_dfs.t
+mro/recursion_dfs_utf8.t
+op/array.t
+op/array_base.t
+op/attrhand.t
+op/attrs.t
+op/bop.t
+op/caller.t
+op/chdir.t
+op/chop.t
+op/chr.t
+op/closure.t
+op/concat2.t
+op/coreamp.t
+op/crypt.t
+op/die.t
+op/do.t
+op/each.t
+op/eval.t
+op/evalbytes.t
+op/exec.t
+op/filetest.t
+op/goto.t
+op/hash-rt85026.t
+op/hashassign.t
+op/index.t
+op/join.t
+op/kvaslice.t
+op/kvhslice.t
+op/lc.t
+op/leaky-magic.t
+op/length.t
+op/lexsub.t
+op/local.t
+op/magic.t
+op/method.t
+op/my.t
+op/mydef.t
+op/not.t
+op/ord.t
+op/overload_integer.t
+op/override.t
+op/pack.t
+op/pos.t
+op/postfixderef.t
+op/push.t
+op/qr.t
+op/quotemeta.t
+op/range.t
+op/readline.t
+op/recurse.t
+op/ref.t
+op/sort.t
+op/split.t
+op/sprintf2.t
+op/srand.t
+op/state.t
+op/sub.t
+op/sub_lval.t
+op/substr.t
+op/switch.t
+op/symbolcache.t
+op/taint.t
+op/tiehandle.t
+op/tr.t
+op/utf8cache.t
+op/utf8magic.t
+op/utfhash.t
+op/vec.t
+op/ver.t
+op/warn.t
+op/write.t
+opbasic/cmp.t
+opbasic/concat.t
+porting/diag.t
+porting/globvar.t
+porting/podcheck.t
+re/fold_grind.t
+re/overload.t
+re/pat.t
+re/pat_advanced.t
+re/pat_re_eval.t
+re/pat_rt_report.t
+re/reg_eval_scope.t
+re/reg_fold.t
+re/reg_mesg.t
+re/reg_pmod.t
+re/reg_posixcc.t
+re/regex_sets.t
+re/regexp_unicode_prop.t
+re/rxcode.t
+re/subst.t
+run/switchC.t
+run/switchI.t
+run/switchd-78586.t
+run/switches.t
+uni/attrs.t
+uni/bless.t
+uni/chomp.t
+uni/chr.t
+uni/class.t
+uni/eval.t
+uni/greek.t
+uni/gv.t
+uni/labels.t
+uni/latin2.t
+uni/lex_utf8.t
+uni/method.t
+uni/overload.t
+uni/package.t
+uni/parser.t
+uni/readline.t
+uni/select.t
+uni/sprintf.t
+uni/stash.t
+uni/tie.t
+uni/tr_7jis.t
+uni/tr_eucjp.t
+uni/tr_sjis.t
+uni/tr_utf8.t
+uni/universal.t
+uni/write.t
+
+__DEPARSE_SKIPS__
+
+op/smartkve.t   # Gobbles up all memory...
+comp/redef.t    # Redefinition happens at compile time
+lib/Switch/t/   # B::Deparse doesn't support source filtering
diff --git a/t/cmd/mod.t b/t/cmd/mod.t
index 07617f5..d3048e7 100644
--- a/t/cmd/mod.t
+++ b/t/cmd/mod.t
@@ -32,10 +32,11 @@ if (join(' ',@y) eq '0 2 4 6 8 10 12 14 16 18 20') {
 	print "not ok 7 @y\n";
 }
 
+# Well this is fragile...
 open(foo,'./TEST') || open(foo,'TEST') || open(foo,'t/TEST');
 $x = 0;
 $x++ while <foo>;
-print $x > 50 && $x < 1000 ? "ok 8\n" : "not ok 8\n";
+print $x > 50 && $x < 2000 ? "ok 8\n" : "not ok 8\n";
 
 $x = -0.5;
 print "not " if scalar($x) < 0 and $x >= 0;
-- 
1.7.9.5

@p5pRT
Copy link
Author

p5pRT commented Feb 2, 2014

From @tonycoz

On Thu Jan 30 19​:40​:56 2014, alh wrote​:

Added back in test.deparse test target.

It now runs all tests with t/TEST, and supports​:

* Skipping known bad tests entirely without running them
* Running tests that are expected to fail, and "skipping" them if they
do
Or, if they pass, throwing an error to let us know the test now
works

This should allow us to smoke it and make sure things don't get any
worse,
while allowing us to fix things as we have time.

I'd like to see a response from Nicholas on whether this goes back in or not.

His argument still holds though - without the added makefile targets you can still run a deparse test with​:

  TEST_ARGS=-deparse make test

As to the patch itself, I'd prefer that the known issues list go into a separate file, as they do with t/porting/podcheck.t.

This could simplify a future implementation of regenerating that list, just as porting/podcheck.t allows.

Tony

@p5pRT
Copy link
Author

p5pRT commented Feb 2, 2014

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Feb 19, 2014

From @wolfsage

On Sun, Feb 2, 2014 at 5​:44 PM, Tony Cook via RT
<perlbug-followup@​perl.org> wrote​:

On Thu Jan 30 19​:40​:56 2014, alh wrote​:

Added back in test.deparse test target.

It now runs all tests with t/TEST, and supports​:

* Skipping known bad tests entirely without running them
* Running tests that are expected to fail, and "skipping" them if they
do
Or, if they pass, throwing an error to let us know the test now
works

This should allow us to smoke it and make sure things don't get any
worse,
while allowing us to fix things as we have time.

I'd like to see a response from Nicholas on whether this goes back in or not.

His argument still holds though - without the added makefile targets you can still run a deparse test with​:

TEST_ARGS=-deparse make test

As to the patch itself, I'd prefer that the known issues list go into a separate file, as they do with t/porting/podcheck.t.

Agreed.

I'm wondering how to implement this though. I think the test should
normally be skipped, and then there should be a makefile target (or
just ability to run the single .t file is fine...), but the .t file
needs to run t/TEST recursively.

So something like​:

  TEST_DEPARSE=1 ./TEST /path/to/deparse.t

And deparse.t needs to run ./TEST with the proper arguments to test
everything, capture all output, and report on it?

Also, this is one of my items for the Perl QA Hackathon if I have the
time to get to it.

-- Matthew Horsfall (alh)

@p5pRT
Copy link
Author

p5pRT commented Mar 13, 2014

From @wolfsage

Alright, here's a new attempt which breaks out the skips into
Porting/deparse-skips.txt

I'm not attached to that location, so if we'd prefer it under
t/deparse-skips.txt (or t/deparse-skips.data) I'd be fine with that.

-- Matthew Horsfall (alh)

@p5pRT
Copy link
Author

p5pRT commented Mar 13, 2014

From @wolfsage

0001-Bring-back-test.deparse-make-target-and-make-it-smar.patch 
From e3a9fcbf4eb7f67029a3a1097a2140e14082da2c Mon Sep 17 00:00:00 2001
From: Matthew Horsfall <WolfSage@gmail.com>
Date: Thu, 30 Jan 2014 22:31:52 -0500
Subject: [PATCH] Bring back test.deparse make target, and make it smarter.

t/TEST now allows skipping / ignoring test failures under test.deparse.

 * Known bad tests can be skipped entirely (not loaded/run)
 * Known failing tests can be run and checked for a change in state.

   - If they fail as expected, they are skipped
   - If they pass, the test is forced to fail so the report at the end is
     useful.
---
 MANIFEST                  |    1 +
 Makefile.SH               |   12 +-
 Porting/README.pod        |    4 +
 Porting/deparse-skips.txt |  548 +++++++++++++++++++++++++++++++++++++++++++++
 pod/perlhack.pod          |    6 +
 t/TEST                    |   73 +++++-
 t/cmd/mod.t               |    3 +-
 7 files changed, 639 insertions(+), 8 deletions(-)
 create mode 100644 Porting/deparse-skips.txt

diff --git a/MANIFEST b/MANIFEST
index 331606c..d8207d6 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -4727,6 +4727,7 @@ Porting/corecpan.pl		Reports outdated dual-lived modules
 Porting/corelist-diff		Tool to produce corelist diffs
 Porting/corelist-perldelta.pl	Generates data perldelta from Module::CoreList
 Porting/corelist.pl		Generates data for Module::CoreList
+Porting/deparse-skips.txt	List of test files to ignore/skip from test.deparse Makefile target.
 Porting/epigraphs.pod		the release epigraphs used over the years
 Porting/exec-bit.txt		List of files that get +x in release tarball
 Porting/exercise_makedef.pl	Brute force testing for makedef.pl
diff --git a/Makefile.SH b/Makefile.SH
index 6e9df1a..6d9a7eb 100755
--- a/Makefile.SH
+++ b/Makefile.SH
@@ -1392,7 +1392,8 @@ depend: makedepend
 
 .PHONY: test check test_prep test_prep_nodll test_prep_pre \
 	test_prep_reonly test_tty test-tty test_notty test-notty \
-	test_harness test_harness_notty minitest test-reonly _test
+	test_harness test_harness_notty minitest test-reonly _test \
+	test.deparse test_notty.deparse
 
 _test:
 	echo >&2 The _test target is deprecated. Please upgrade your smoker
@@ -1497,6 +1498,15 @@ test_tty test-tty: test_prep
 test_notty test-notty: test_prep
 	$(RUN_TESTS) no-tty
 
+# Targets for Deparse testing.
+test.deparse:  test_prep
+	@echo "DOING IT"
+	TEST_ARGS=-deparse $(RUN_TESTS) choose
+	@echo "DONE"
+
+test_notty.deparse: test_prep
+	TEST_ARGS=-deparse $(RUN_TESTS) no-tty
+
 !NO!SUBS!
 
 case "${osname}${osvers}" in
diff --git a/Porting/README.pod b/Porting/README.pod
index 2bd612d..8758bf1 100644
--- a/Porting/README.pod
+++ b/Porting/README.pod
@@ -124,6 +124,10 @@ F<perldelta*> files.
 
 Generates info for Module::CoreList from this perl tree.
 
+=head2 F<deparse-skips.txt>
+
+List of test files to ignore/skip from test.deparse Makefile target.
+
 =head2 F<epigraphs.pod>
 
 List of Perl release epigraphs.
diff --git a/Porting/deparse-skips.txt b/Porting/deparse-skips.txt
new file mode 100644
index 0000000..d6126fa
--- /dev/null
+++ b/Porting/deparse-skips.txt
@@ -0,0 +1,548 @@
+# List of test files to ignore/skip from test.deparse Makefile target.
+#
+# This is used by t/TEST
+# 
+# Those under DEPARSE_FAILURES are ones that we need to fix or
+# we expect to fail. These tests will still be run and checked to
+# see if they fail as expected.
+#
+# Those under DEPARSE_SKIPS we don't want to even attempt running
+# because they might do bad things to our system (memory exhaustion,
+# etc), so they will be completely skipped from the test run.
+#
+# Paths are relative from t/
+#
+# Paths that end with '/' are directories to skip entirely
+#
+# Format:
+#   __DEPARSE_FAILURES__
+#   ../path/to/test1
+#   ../path/to/test2
+#   ../dir/to/skip/
+#
+#   __DEPARSE_SKIPS__
+#   ../path/to/test3
+#   ../path/to/test4
+#
+#   # This is a comment
+#   ../path/to/test5 # More comments at end of line
+#
+#   # Blank lines okay
+
+__DEPARSE_FAILURES__
+
+../cpan/AutoLoader/t/01AutoLoader.t
+../cpan/CGI/t/utf8.t
+../cpan/CPAN-Meta/t/converter.t
+../cpan/Digest-MD5/t/utf8.t
+../cpan/Encode/t/Encode.t
+../cpan/Encode/t/Unicode.t
+../cpan/Encode/t/at-cn.t
+../cpan/Encode/t/at-tw.t
+../cpan/Encode/t/enc_data.t
+../cpan/Encode/t/enc_eucjp.t
+../cpan/Encode/t/enc_module.t
+../cpan/Encode/t/enc_utf8.t
+../cpan/Encode/t/encoding.t
+../cpan/Encode/t/fallback.t
+../cpan/Encode/t/from_to.t
+../cpan/Encode/t/guess.t
+../cpan/Encode/t/jis7-fallback.t
+../cpan/Encode/t/jperl.t
+../cpan/Encode/t/mime-header.t
+../cpan/ExtUtils-MakeMaker/t/arch_check.t
+../cpan/ExtUtils-MakeMaker/t/min_perl_version.t
+../cpan/ExtUtils-MakeMaker/t/xs.t
+../cpan/File-Path/t/taint.t
+../cpan/File-Temp/t/object.t
+../cpan/HTTP-Tiny/t/001_api.t
+../cpan/IO-Compress/t/050interop-gzip.t
+../cpan/IO-Compress/t/cz-08encoding.t
+../cpan/JSON-PP/t/001_utf8.t
+../cpan/JSON-PP/t/109_encode.t
+../cpan/MIME-Base64/t/unicode.t
+../cpan/Math-Complex/t/Trig.t
+../cpan/Module-Build/t/manifypods_with_utf8.t
+../cpan/Module-Metadata/t/encoding.t
+../cpan/Pod-Perldoc/t/man/_get_columns.t
+../cpan/Pod-Simple/t/stree.t
+../cpan/Scalar-List-Utils/t/dualvar.t
+../cpan/Scalar-List-Utils/t/first.t
+../cpan/Scalar-List-Utils/t/reduce.t
+../cpan/Scalar-List-Utils/t/weak.t
+../cpan/Socket/t/sockaddr.t
+../cpan/Term-ANSIColor/t/taint.t
+../cpan/Test-Harness/t/parser-config.t
+../cpan/Test-Harness/t/parser-subclass.t
+../cpan/Test-Harness/t/proverun.t
+../cpan/Test-Simple/t/Builder/carp.t
+../cpan/Test-Simple/t/fail-more.t
+../cpan/Test-Simple/t/is_deeply_fail.t
+../cpan/Test-Simple/t/new_ok.t
+../cpan/Test-Simple/t/plan.t
+../cpan/Test-Simple/t/plan_bad.t
+../cpan/Test-Simple/t/skip.t
+../cpan/Test-Simple/t/subtest/line_numbers.t
+../cpan/Test-Simple/t/subtest/predicate.t
+../cpan/Test-Simple/t/todo.t
+../cpan/Text-Tabs/t/Tabs-ElCid.t
+../cpan/Text-Tabs/t/Wrap-JLB.t
+../cpan/Unicode-Collate/t/altern.t
+../cpan/Unicode-Collate/t/backwds.t
+../cpan/Unicode-Collate/t/cjk_b5.t
+../cpan/Unicode-Collate/t/cjk_gb.t
+../cpan/Unicode-Collate/t/cjk_ja.t
+../cpan/Unicode-Collate/t/cjk_ko.t
+../cpan/Unicode-Collate/t/cjk_py.t
+../cpan/Unicode-Collate/t/cjk_st.t
+../cpan/Unicode-Collate/t/cjk_zy.t
+../cpan/Unicode-Collate/t/cjkrange.t
+../cpan/Unicode-Collate/t/compatui.t
+../cpan/Unicode-Collate/t/contract.t
+../cpan/Unicode-Collate/t/default.t
+../cpan/Unicode-Collate/t/hangul.t
+../cpan/Unicode-Collate/t/ident.t
+../cpan/Unicode-Collate/t/iglevel2.t
+../cpan/Unicode-Collate/t/ignor.t
+../cpan/Unicode-Collate/t/illegal.t
+../cpan/Unicode-Collate/t/index.t
+../cpan/Unicode-Collate/t/loc_af.t
+../cpan/Unicode-Collate/t/loc_ar.t
+../cpan/Unicode-Collate/t/loc_as.t
+../cpan/Unicode-Collate/t/loc_az.t
+../cpan/Unicode-Collate/t/loc_be.t
+../cpan/Unicode-Collate/t/loc_bg.t
+../cpan/Unicode-Collate/t/loc_bn.t
+../cpan/Unicode-Collate/t/loc_bs.t
+../cpan/Unicode-Collate/t/loc_bscy.t
+../cpan/Unicode-Collate/t/loc_ca.t
+../cpan/Unicode-Collate/t/loc_cjk.t
+../cpan/Unicode-Collate/t/loc_cjkc.t
+../cpan/Unicode-Collate/t/loc_cs.t
+../cpan/Unicode-Collate/t/loc_cyrl.t
+../cpan/Unicode-Collate/t/loc_da.t
+../cpan/Unicode-Collate/t/loc_de.t
+../cpan/Unicode-Collate/t/loc_deph.t
+../cpan/Unicode-Collate/t/loc_ee.t
+../cpan/Unicode-Collate/t/loc_eo.t
+../cpan/Unicode-Collate/t/loc_es.t
+../cpan/Unicode-Collate/t/loc_estr.t
+../cpan/Unicode-Collate/t/loc_et.t
+../cpan/Unicode-Collate/t/loc_fa.t
+../cpan/Unicode-Collate/t/loc_fi.t
+../cpan/Unicode-Collate/t/loc_fil.t
+../cpan/Unicode-Collate/t/loc_fiph.t
+../cpan/Unicode-Collate/t/loc_fo.t
+../cpan/Unicode-Collate/t/loc_fr.t
+../cpan/Unicode-Collate/t/loc_gu.t
+../cpan/Unicode-Collate/t/loc_ha.t
+../cpan/Unicode-Collate/t/loc_haw.t
+../cpan/Unicode-Collate/t/loc_hi.t
+../cpan/Unicode-Collate/t/loc_hr.t
+../cpan/Unicode-Collate/t/loc_hu.t
+../cpan/Unicode-Collate/t/loc_hy.t
+../cpan/Unicode-Collate/t/loc_ig.t
+../cpan/Unicode-Collate/t/loc_is.t
+../cpan/Unicode-Collate/t/loc_ja.t
+../cpan/Unicode-Collate/t/loc_jait.t
+../cpan/Unicode-Collate/t/loc_japr.t
+../cpan/Unicode-Collate/t/loc_kk.t
+../cpan/Unicode-Collate/t/loc_kl.t
+../cpan/Unicode-Collate/t/loc_kn.t
+../cpan/Unicode-Collate/t/loc_ko.t
+../cpan/Unicode-Collate/t/loc_kok.t
+../cpan/Unicode-Collate/t/loc_ln.t
+../cpan/Unicode-Collate/t/loc_lt.t
+../cpan/Unicode-Collate/t/loc_lv.t
+../cpan/Unicode-Collate/t/loc_mk.t
+../cpan/Unicode-Collate/t/loc_ml.t
+../cpan/Unicode-Collate/t/loc_mr.t
+../cpan/Unicode-Collate/t/loc_mt.t
+../cpan/Unicode-Collate/t/loc_nb.t
+../cpan/Unicode-Collate/t/loc_nn.t
+../cpan/Unicode-Collate/t/loc_nso.t
+../cpan/Unicode-Collate/t/loc_or.t
+../cpan/Unicode-Collate/t/loc_pa.t
+../cpan/Unicode-Collate/t/loc_pl.t
+../cpan/Unicode-Collate/t/loc_ro.t
+../cpan/Unicode-Collate/t/loc_ru.t
+../cpan/Unicode-Collate/t/loc_sa.t
+../cpan/Unicode-Collate/t/loc_se.t
+../cpan/Unicode-Collate/t/loc_si.t
+../cpan/Unicode-Collate/t/loc_sidt.t
+../cpan/Unicode-Collate/t/loc_sk.t
+../cpan/Unicode-Collate/t/loc_sl.t
+../cpan/Unicode-Collate/t/loc_sq.t
+../cpan/Unicode-Collate/t/loc_sr.t
+../cpan/Unicode-Collate/t/loc_srla.t
+../cpan/Unicode-Collate/t/loc_sv.t
+../cpan/Unicode-Collate/t/loc_svrf.t
+../cpan/Unicode-Collate/t/loc_ta.t
+../cpan/Unicode-Collate/t/loc_te.t
+../cpan/Unicode-Collate/t/loc_test.t
+../cpan/Unicode-Collate/t/loc_th.t
+../cpan/Unicode-Collate/t/loc_tn.t
+../cpan/Unicode-Collate/t/loc_to.t
+../cpan/Unicode-Collate/t/loc_tr.t
+../cpan/Unicode-Collate/t/loc_uk.t
+../cpan/Unicode-Collate/t/loc_ur.t
+../cpan/Unicode-Collate/t/loc_vi.t
+../cpan/Unicode-Collate/t/loc_wae.t
+../cpan/Unicode-Collate/t/loc_wo.t
+../cpan/Unicode-Collate/t/loc_yo.t
+../cpan/Unicode-Collate/t/loc_zh.t
+../cpan/Unicode-Collate/t/loc_zhb5.t
+../cpan/Unicode-Collate/t/loc_zhgb.t
+../cpan/Unicode-Collate/t/loc_zhpy.t
+../cpan/Unicode-Collate/t/loc_zhst.t
+../cpan/Unicode-Collate/t/loc_zhzy.t
+../cpan/Unicode-Collate/t/nonchar.t
+../cpan/Unicode-Collate/t/normal.t
+../cpan/Unicode-Collate/t/notable.t
+../cpan/Unicode-Collate/t/overcjk0.t
+../cpan/Unicode-Collate/t/overcjk1.t
+../cpan/Unicode-Collate/t/override.t
+../cpan/Unicode-Collate/t/rearrang.t
+../cpan/Unicode-Collate/t/rewrite.t
+../cpan/Unicode-Collate/t/test.t
+../cpan/Unicode-Collate/t/trailwt.t
+../cpan/Unicode-Collate/t/variable.t
+../cpan/Unicode-Collate/t/view.t
+../cpan/Unicode-Normalize/t/fcdc.t
+../cpan/Unicode-Normalize/t/form.t
+../cpan/Unicode-Normalize/t/func.t
+../cpan/Unicode-Normalize/t/norm.t
+../cpan/Unicode-Normalize/t/partial1.t
+../cpan/Unicode-Normalize/t/partial2.t
+../cpan/Unicode-Normalize/t/proto.t
+../cpan/Unicode-Normalize/t/split.t
+../cpan/Unicode-Normalize/t/test.t
+../cpan/autodie/t/00-load.t
+../cpan/autodie/t/autodie.t
+../cpan/autodie/t/blog_hints.t
+../cpan/autodie/t/caller.t
+../cpan/autodie/t/chmod.t
+../cpan/autodie/t/chown.t
+../cpan/autodie/t/context.t
+../cpan/autodie/t/context_lexical.t
+../cpan/autodie/t/crickey.t
+../cpan/autodie/t/dbmopen.t
+../cpan/autodie/t/eval_error.t
+../cpan/autodie/t/exception_class.t
+../cpan/autodie/t/exceptions.t
+../cpan/autodie/t/exec.t
+../cpan/autodie/t/filehandles.t
+../cpan/autodie/t/format-clobber.t
+../cpan/autodie/t/hints.t
+../cpan/autodie/t/hints_insist.t
+../cpan/autodie/t/hints_pod_examples.t
+../cpan/autodie/t/hints_provider_does.t
+../cpan/autodie/t/hints_provider_easy_does_it.t
+../cpan/autodie/t/hints_provider_isa.t
+../cpan/autodie/t/kill.t
+../cpan/autodie/t/lethal.t
+../cpan/autodie/t/open.t
+../cpan/autodie/t/recv.t
+../cpan/autodie/t/repeat.t
+../cpan/autodie/t/scope_leak.t
+../cpan/autodie/t/sysopen.t
+../cpan/autodie/t/user-context.t
+../cpan/autodie/t/usersub.t
+../cpan/autodie/t/utf8_open.t
+../cpan/autodie/t/utime.t
+../cpan/autodie/t/version_tag.t
+../cpan/encoding-warnings/t/4-lexical.t
+../cpan/podlators/t/basic.t
+../cpan/version/t/09_list_util.t
+../dist/Attribute-Handlers/t/constants.t
+../dist/Attribute-Handlers/t/data_convert.t
+../dist/Attribute-Handlers/t/linerep.t
+../dist/Attribute-Handlers/t/multi.t
+../dist/Carp/t/Carp.t
+../dist/Carp/t/arg_regexp.t
+../dist/Carp/t/arg_string.t
+../dist/Data-Dumper/t/dumper.t
+../dist/Data-Dumper/t/perl-74170.t
+../dist/Data-Dumper/t/quotekeys.t
+../dist/Exporter/t/Exporter.t
+../dist/ExtUtils-Install/t/Installapi2.t
+../dist/ExtUtils-Install/t/Packlist.t
+../dist/ExtUtils-Install/t/can_write_dir.t
+../dist/ExtUtils-Manifest/t/Manifest.t
+../dist/Filter-Simple/t/data.t
+../dist/I18N-LangTags/t/50_super.t
+../dist/IO/t/io_file_export.t
+../dist/IO/t/io_multihomed.t
+../dist/IO/t/io_sel.t
+../dist/IO/t/io_sock.t
+../dist/IO/t/io_udp.t
+../dist/IO/t/io_utf8.t
+../dist/Locale-Maketext/t/01_about_verbose.t
+../dist/Locale-Maketext/t/10_make.t
+../dist/Locale-Maketext/t/20_get.t
+../dist/Locale-Maketext/t/30_eval_dollar_at.t
+../dist/Locale-Maketext/t/40_super.t
+../dist/Locale-Maketext/t/50_super.t
+../dist/Locale-Maketext/t/60_super.t
+../dist/Locale-Maketext/t/70_fail_auto.t
+../dist/Locale-Maketext/t/90_utf8.t
+../dist/Locale-Maketext/t/91_backslash.t
+../dist/Math-BigInt/t/const_mbf.t
+../dist/Math-BigInt/t/constant.t
+../dist/PathTools/t/cwd.t
+../dist/Storable/t/blessed.t
+../dist/Storable/t/croak.t
+../dist/Storable/t/downgrade.t
+../dist/Storable/t/malice.t
+../dist/Storable/t/utf8.t
+../dist/Term-ReadLine/t/ReadLine.t
+../dist/Thread-Queue/t/08_nothreads.t
+../dist/Tie-File/t/29_downcopy.t
+../dist/Tie-File/t/42_offset.t
+../dist/bignum/t/big_e_pi.t
+../dist/bignum/t/bigexp.t
+../dist/bignum/t/bigint.t
+../dist/bignum/t/bignum.t
+../dist/bignum/t/bigrat.t
+../dist/bignum/t/bii_e_pi.t
+../dist/bignum/t/bir_e_pi.t
+../dist/bignum/t/in_effect.t
+../dist/bignum/t/option_a.t
+../dist/bignum/t/option_l.t
+../dist/bignum/t/option_p.t
+../dist/bignum/t/overrides.t
+../dist/bignum/t/ratopt_a.t
+../dist/bignum/t/scope_f.t
+../dist/bignum/t/scope_i.t
+../dist/bignum/t/scope_r.t
+../dist/constant/t/constant.t
+../dist/threads/t/err.t
+../dist/threads/t/exit.t
+../dist/threads/t/kill2.t
+../dist/threads/t/libc.t
+../dist/threads/t/thread.t
+../ext/B/t/b.t
+../ext/B/t/optree_constants.t
+../ext/B/t/optree_samples.t
+../ext/B/t/pragma.t
+../ext/B/t/xref.t
+../ext/Devel-Peek/t/Peek.t
+../ext/File-Glob/t/basic.t
+../ext/File-Glob/t/taint.t
+../ext/Hash-Util-FieldHash/t/02_function.t
+../ext/Hash-Util-FieldHash/t/11_hashassign.t
+../ext/Hash-Util/t/Util.t
+../ext/IPC-Open3/t/IPC-Open2.t
+../ext/IPC-Open3/t/IPC-Open3.t
+../ext/Opcode/t/Opcode.t
+../ext/POSIX/t/termios.t
+../ext/PerlIO-encoding/t/encoding.t
+../ext/PerlIO-encoding/t/fallback.t
+../ext/PerlIO-scalar/t/scalar.t
+../ext/PerlIO-via/t/via.t
+../ext/XS-APItest/t/autoload.t
+../ext/XS-APItest/t/blockhooks-csc.t
+../ext/XS-APItest/t/blockhooks.t
+../ext/XS-APItest/t/call_checker.t
+../ext/XS-APItest/t/caller.t
+../ext/XS-APItest/t/cleanup.t
+../ext/XS-APItest/t/cophh.t
+../ext/XS-APItest/t/fetch_pad_names.t
+../ext/XS-APItest/t/lexsub.t
+../ext/XS-APItest/t/multicall.t
+../ext/XS-APItest/t/overload.t
+../ext/XS-APItest/t/svpeek.t
+../ext/XS-APItest/t/svpv.t
+../ext/XS-APItest/t/underscore_length.t
+../ext/XS-APItest/t/xsub_h.t
+../ext/arybase/t/aeach.t
+../ext/arybase/t/aelem.t
+../ext/arybase/t/aslice.t
+../ext/arybase/t/av2arylen.t
+../ext/arybase/t/lslice.t
+../ext/arybase/t/scope.t
+../ext/arybase/t/splice.t
+../ext/re/t/reflags.t
+../lib/B/Deparse.t
+../lib/DB.t
+../lib/DBM_Filter/t/01error.t
+../lib/DBM_Filter/t/02core.t
+../lib/DBM_Filter/t/compress.t
+../lib/DBM_Filter/t/encode.t
+../lib/DBM_Filter/t/int32.t
+../lib/DBM_Filter/t/null.t
+../lib/DBM_Filter/t/utf8.t
+../lib/English.t
+../lib/File/Basename.t
+../lib/Getopt/Std.t
+../lib/Unicode/UCD.t
+../lib/bytes.t
+../lib/charnames.t
+../lib/feature/unicode_strings.t
+../lib/less.t
+../lib/locale.t
+../lib/overload.t
+../lib/overloading.t
+../lib/utf8.t
+base/lex.t
+comp/final_line_num.t
+comp/fold.t
+comp/form_scope.t
+comp/hints.t
+comp/opsubs.t
+comp/parser.t
+comp/proto.t
+comp/require.t
+io/inplace.t
+io/utf8.t
+lib/deprecate.t
+lib/mypragma.t
+mro/basic.t
+mro/basic_utf8.t
+mro/dbic_c3.t
+mro/dbic_c3_utf8.t
+mro/dbic_dfs.t
+mro/dbic_dfs_utf8.t
+mro/inconsistent_c3.t
+mro/inconsistent_c3_utf8.t
+mro/isarev.t
+mro/isarev_utf8.t
+mro/method_caching.t
+mro/method_caching_utf8.t
+mro/next_edgecases.t
+mro/next_edgecases_utf8.t
+mro/next_goto.t
+mro/next_goto_utf8.t
+mro/package_aliases_utf8.t
+mro/pkg_gen.t
+mro/pkg_gen_utf8.t
+mro/recursion_c3.t
+mro/recursion_c3_utf8.t
+mro/recursion_dfs.t
+mro/recursion_dfs_utf8.t
+op/array.t
+op/array_base.t
+op/attrhand.t
+op/attrs.t
+op/bop.t
+op/caller.t
+op/chdir.t
+op/chop.t
+op/chr.t
+op/closure.t
+op/concat2.t
+op/coreamp.t
+op/crypt.t
+op/die.t
+op/do.t
+op/each.t
+op/eval.t
+op/evalbytes.t
+op/exec.t
+op/filetest.t
+op/goto.t
+op/hash-rt85026.t
+op/hashassign.t
+op/index.t
+op/join.t
+op/kvaslice.t
+op/kvhslice.t
+op/lc.t
+op/leaky-magic.t
+op/length.t
+op/lexsub.t
+op/local.t
+op/magic.t
+op/method.t
+op/my.t
+op/mydef.t
+op/not.t
+op/ord.t
+op/overload_integer.t
+op/override.t
+op/pack.t
+op/pos.t
+op/postfixderef.t
+op/push.t
+op/qr.t
+op/quotemeta.t
+op/range.t
+op/readline.t
+op/recurse.t
+op/ref.t
+op/sort.t
+op/split.t
+op/sprintf2.t
+op/srand.t
+op/state.t
+op/sub.t
+op/sub_lval.t
+op/substr.t
+op/switch.t
+op/symbolcache.t
+op/taint.t
+op/tiehandle.t
+op/tr.t
+op/utf8cache.t
+op/utf8magic.t
+op/utfhash.t
+op/vec.t
+op/ver.t
+op/warn.t
+op/write.t
+opbasic/cmp.t
+opbasic/concat.t
+porting/diag.t
+porting/globvar.t
+porting/podcheck.t
+re/fold_grind.t
+re/overload.t
+re/pat.t
+re/pat_advanced.t
+re/pat_re_eval.t
+re/pat_rt_report.t
+re/reg_eval_scope.t
+re/reg_fold.t
+re/reg_mesg.t
+re/reg_pmod.t
+re/reg_posixcc.t
+re/regex_sets.t
+re/regexp_unicode_prop.t
+re/rxcode.t
+re/subst.t
+run/switchC.t
+run/switchI.t
+run/switchd-78586.t
+run/switches.t
+uni/attrs.t
+uni/bless.t
+uni/chomp.t
+uni/chr.t
+uni/class.t
+uni/eval.t
+uni/greek.t
+uni/gv.t
+uni/labels.t
+uni/latin2.t
+uni/lex_utf8.t
+uni/method.t
+uni/overload.t
+uni/package.t
+uni/parser.t
+uni/readline.t
+uni/select.t
+uni/sprintf.t
+uni/stash.t
+uni/tie.t
+uni/tr_7jis.t
+uni/tr_eucjp.t
+uni/tr_sjis.t
+uni/tr_utf8.t
+uni/universal.t
+uni/write.t
+
+__DEPARSE_SKIPS__
+
+op/smartkve.t   # Gobbles up all memory...
+comp/redef.t    # Redefinition happens at compile time
+lib/Switch/t/   # B::Deparse doesn't support source filtering
diff --git a/pod/perlhack.pod b/pod/perlhack.pod
index dc3f396..c6479af 100644
--- a/pod/perlhack.pod
+++ b/pod/perlhack.pod
@@ -793,6 +793,12 @@ systems.
 This runs some basic sanity tests on the source tree and helps catch
 basic errors before you submit a patch.
 
+=item * test.deparse test_notty.deparse
+
+Run all the tests through L<B::Deparse>. Not all tests will succeed.
+
+(Not available on Win32)
+
 =item * minitest
 
 Run F<miniperl> on F<t/base>, F<t/comp>, F<t/cmd>, F<t/run>, F<t/io>,
diff --git a/t/TEST b/t/TEST
index 96eb6a4..cfee24b 100755
--- a/t/TEST
+++ b/t/TEST
@@ -14,6 +14,11 @@
 # In which case, we need to stop t/TEST actually running tests, as all
 # t/harness needs are its subroutines.
 
+# If we're doing deparse tests, ignore failures for these
+my $deparse_failures;
+
+# And skip even running these
+my $deparse_skips;
 
 # directories with special sets of test switches
 my %dir_to_switch =
@@ -126,6 +131,7 @@ our $show_elapsed_time = $ENV{HARNESS_TIMER} || 0;
 	if ($1 =~ /^deparse(,.+)?$/) {
 	    $::deparse = 1;
 	    $::deparse_opts = $1;
+	    _process_deparse_config();
 	}
     }
     @ARGV = @argv;
@@ -536,12 +542,7 @@ EOT
 	    next;
 	}
 	if ($type eq 'deparse') {
-	    if ($test eq "comp/redef.t") {
-		# Redefinition happens at compile time
-		next;
-	    }
-	    elsif ($test =~ m{lib/Switch/t/}) {
-		# B::Deparse doesn't support source filtering
+	    if ($test =~ $deparse_skips) {
 		next;
 	    }
 	}
@@ -722,6 +723,19 @@ EOT
 	    $failure = "FAILED--non-zero wait status: $?";
 	}
 
+	# Deparse? Should it have passed or failed?
+	if ($type eq 'deparse' && $test =~ $deparse_failures) {
+	    if (!$failure) {
+		# Wait, it didn't fail? Great news! Tell someone!
+		$failure = "FAILED--all tests passed but test should have failed";
+
+	    } else {
+		# Bah, still failing. Mask it.
+		print "${te}skipped\n";
+		$tested_files = $tested_files - 1;
+		next;
+	    }
+	}		
 	if (defined $failure) {
 	    print "${te}$failure\n";
 	    $::bad_files = $::bad_files + 1;
@@ -874,4 +888,51 @@ sub gather_conf_platform_info {
 	);
 }
 
+# Generate regexps of known bad filenames / skips from Porting/deparse-skips.txt
+my $in;
+
+sub _process_deparse_config {
+    my @deparse_failures;
+    my @deparse_skips;
+
+    my $f = '../Porting/deparse-skips.txt';
+
+    my $skips;
+    if (!open($skips, '<', $f)) {
+        warn "Failed to find $f: $!\n";
+        return;
+    }
+
+    while(<$skips>) {
+        if (/__DEPARSE_FAILURES__/) {
+            $in = \@deparse_failures; next;
+        } elsif (/__DEPARSE_SKIPS__/) {
+            $in = \@deparse_skips; next;
+        } elsif (!$in) {
+            next;
+	}
+
+        s/#.*$//; # Kill comments
+        s/\s+$//; # And trailing whitespace
+
+        next unless $_;
+
+        push @$in, $_;
+    }
+
+    for my $f (@deparse_failures, @deparse_skips) {
+        if ($f =~ m|/$|) { # Dir? Skip everything below it
+            $f = qr/\Q$f\E.*/;
+        } else {
+            $f = qr/\Q$f\E/;
+        }
+    }
+
+    $deparse_failures = join('|', @deparse_failures);
+    $deparse_failures = qr/^(?:$deparse_failures)$/;
+
+    $deparse_skips = join('|', @deparse_skips);
+    $deparse_skips = qr/^(?:$deparse_skips)$/;
+}
+
 # ex: set ts=8 sts=4 sw=4 noet:
diff --git a/t/cmd/mod.t b/t/cmd/mod.t
index 07617f5..d3048e7 100644
--- a/t/cmd/mod.t
+++ b/t/cmd/mod.t
@@ -32,10 +32,11 @@ if (join(' ',@y) eq '0 2 4 6 8 10 12 14 16 18 20') {
 	print "not ok 7 @y\n";
 }
 
+# Well this is fragile...
 open(foo,'./TEST') || open(foo,'TEST') || open(foo,'t/TEST');
 $x = 0;
 $x++ while <foo>;
-print $x > 50 && $x < 1000 ? "ok 8\n" : "not ok 8\n";
+print $x > 50 && $x < 2000 ? "ok 8\n" : "not ok 8\n";
 
 $x = -0.5;
 print "not " if scalar($x) < 0 and $x >= 0;
-- 
1.7.9.5

@p5pRT
Copy link
Author

p5pRT commented Jun 13, 2014

From @wolfsage

On Thu Mar 13 04​:40​:29 2014, alh wrote​:

Alright, here's a new attempt which breaks out the skips into
Porting/deparse-skips.txt

I'm not attached to that location, so if we'd prefer it under
t/deparse-skips.txt (or t/deparse-skips.data) I'd be fine with that.

Applied as 2722144, without the Makefile support.

Resolving.

-- Matthew Horsfall (alh)

@p5pRT p5pRT closed this as completed Jun 13, 2014
@p5pRT
Copy link
Author

p5pRT commented Jun 13, 2014

@wolfsage - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
distro-Linux hasPatch type-core
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@p5pRT