Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter::Simple executable_no_comments filters don't work (fatal error) #17122

Closed
p5pRT opened this issue Aug 11, 2019 · 6 comments
Closed

Filter::Simple executable_no_comments filters don't work (fatal error) #17122

p5pRT opened this issue Aug 11, 2019 · 6 comments

Comments

@p5pRT
Copy link

p5pRT commented Aug 11, 2019

Migrated from rt.perl.org#134349 (status was 'open')

Searchable as RT134349$

@p5pRT
Copy link
Author

p5pRT commented Aug 11, 2019

From Jim@balter.name

Created by jim@balter.name

Filter​::Simple version 0.94 (the latest) does not work with
'executable_no_comments' filters, producing

  Can't use an undefined value as a subroutine reference at
/usr/share/perl5/5.26/Filter/Simple.pm line 132.

because there's no such type in %selector_for and it doesn't match /^code/
(line 120). I suspect that it would work if that line were changed to

  if ($type =~ /^(?​:code|executable)/) {

However, the documentation says that only code and code_no_comments filters
do the blanking conditioned by that code. Since I can get away with using
an 'executable' filter, I haven't delved further.

Also​:

In my attempts to write a filter, I discovered that errors and warnings
issued from a filter sub are never seen, instead getting a "no quotelike
operator at offset ..." message. It turns out that the fix is to localize
$@​. While I did this in my sub, it's a common problem and it would be
helpful for Filter​::Simple to do the localization around the invocation of
the filter sub.

This has perhaps been fixed in 5.28.0 ...
https://rt.perl.org/Public/Bug/Display.html?id=38920

but a lot of people are running earlier versions and Filter​::Simple could
help.

Perl Info

Flags:
    category=library
    severity=medium
    module=Filter::Simple

Site configuration information for perl 5.26.2:

Configured by ASSI at Mon Apr 16 19:45:26 CEST 2018.

Summary of my perl5 (revision 5 version 26 subversion 2) configuration:

  Platform:
    osname=cygwin
    osvers=2.10.0(0.32553)
    archname=x86_64-cygwin-threads-multi
    uname='cygwin_nt-6.3 cygwin 2.10.0(0.32553) 2018-02-02 15:16 x86_64
cygwin '
    config_args='-des -Dprefix=/usr -Dmksymlinks
-Darchname=x86_64-cygwin-threads -Dlibperl=cygperl5_26.dll -Dcc=gcc
-Dld=g++ -Accflags=-ggdb -O2 -pipe -Wall -Werror=format-security
-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4
-fdebug-prefix-map=/mnt/share/maint/perl.x86_64/build=/usr/src/debug/perl-5.26.2-1
-fdebug-prefix-map=/mnt/share/maint/perl.x86_64/src/perl-5.26.2=/usr/src/debug/perl-5.26.2-1
-fwrapv'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='gcc'
    ccflags ='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -ggdb
-O2 -pipe -Wall -Werror=format-security -D_FORTIFY_SOURCE=2
-fstack-protector-strong --param=ssp-buffer-size=4
-fdebug-prefix-map=/mnt/share/maint/perl.x86_64/build=/usr/src/debug/perl-5.26.2-1
-fdebug-prefix-map=/mnt/share/maint/perl.x86_64/src/perl-5.26.2=/usr/src/debug/perl-5.26.2-1
-fwrapv -fno-strict-aliasing'
    optimize='-O3'
    cppflags='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -ggdb
-O2 -pipe -Wall -Werror=format-security -D_FORTIFY_SOURCE=2
-fstack-protector-strong --param=ssp-buffer-size=4
-fdebug-prefix-map=/mnt/share/maint/perl.x86_64/build=/usr/src/debug/perl-5.26.2-1
-fdebug-prefix-map=/mnt/share/maint/perl.x86_64/src/perl-5.26.2=/usr/src/debug/perl-5.26.2-1
-fwrapv -fno-strict-aliasing'
    ccversion=''
    gccversion='6.4.0'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='g++'
    ldflags =' -Wl,--enable-auto-import -Wl,--export-all-symbols
-Wl,--enable-auto-image-base -fstack-protector-strong'
    libpth=/usr/lib
    libs=-lpthread -lgdbm -ldb -ldl -lcrypt -lgdbm_compat
    perllibs=-lpthread -ldl -lcrypt
    libc=/usr/lib/libcygwin.a
    so=dll
    useshrplib=true
    libperl=cygperl5_26.dll
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=dll
    d_dlsymun=undef
    ccdlflags=' '
    cccdlflags=' '
    lddlflags=' --shared  -Wl,--enable-auto-import -Wl,--export-all-symbols
-Wl,--enable-auto-image-base -fstack-protector-strong'

Locally applied patches:
    Cygwin: README
    Cygwin: use auto-image-base instead of fixed DLL base address
    Cygwin: modify hints
    Cygwin: Configure correct libsearch
    Cygwin: Configure correct libpth
    Cygwin: Win32 correct UTF8 handling
    Perl: File-Path-2.14 (fixes CVE2017-6512)


@INC for perl 5.26.2:
   [censored]


Environment for perl 5.26.2:
   [censored]

@p5pRT
Copy link
Author

p5pRT commented Aug 11, 2019

From @jkeenan

On Sun, 11 Aug 2019 03​:55​:53 GMT, Jim@​balter.name wrote​:

This is a bug report for perl from jim@​balter.name,
generated with the help of perlbug 1.40 running under perl 5.26.2.

-----------------------------------------------------------------
[Please describe your issue here]

Filter​::Simple version 0.94 (the latest) does not work with
'executable_no_comments' filters, producing

Can't use an undefined value as a subroutine reference at
/usr/share/perl5/5.26/Filter/Simple.pm line 132.

because there's no such type in %selector_for and it doesn't match
/^code/
(line 120). I suspect that it would work if that line were changed to

if ($type =~ /^(?​:code|executable)/) {

However, the documentation says that only code and code_no_comments
filters
do the blanking conditioned by that code. Since I can get away with
using
an 'executable' filter, I haven't delved further.

Also​:

In my attempts to write a filter, I discovered that errors and
warnings
issued from a filter sub are never seen, instead getting a "no
quotelike
operator at offset ..." message. It turns out that the fix is to
localize
$@​. While I did this in my sub, it's a common problem and it would be
helpful for Filter​::Simple to do the localization around the
invocation of
the filter sub.

This has perhaps been fixed in 5.28.0 ...
https://rt.perl.org/Public/Bug/Display.html?id=38920

but a lot of people are running earlier versions and Filter​::Simple
could
help.

Filter-Simple was upgraded to version 0.95 within the Perl 5 core distribution in September 2017. However, it appears that no new CPAN release was made; CPAN is stuck at version 0.94.

Bcc-ing some people who might be able to help with a new CPAN release.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Aug 11, 2019

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Aug 12, 2019

From Jim@balter.name

Thanks for the response. Looking at 0.95 in the perl 5.30 dist, there are
no substantive changes so this is still broken.
And of course core distributions don't help those of us stuck on earlier
versions of perl for any of numerous reasons. In my case because I'm using
cygwin, which is at 5.26.2, released on April 14, 2018 (with Filter​::Simple
0.94).

-- Jim

On Sun, Aug 11, 2019 at 11​:41 AM James E Keenan via RT <
perlbug-followup@​perl.org> wrote​:

On Sun, 11 Aug 2019 03​:55​:53 GMT, Jim@​balter.name wrote​:

This is a bug report for perl from jim@​balter.name,
generated with the help of perlbug 1.40 running under perl 5.26.2.

-----------------------------------------------------------------
[Please describe your issue here]

Filter​::Simple version 0.94 (the latest) does not work with
'executable_no_comments' filters, producing

Can't use an undefined value as a subroutine reference at
/usr/share/perl5/5.26/Filter/Simple.pm line 132.

because there's no such type in %selector_for and it doesn't match
/^code/
(line 120). I suspect that it would work if that line were changed to

if ($type =~ /^(?​:code|executable)/) {

However, the documentation says that only code and code_no_comments
filters
do the blanking conditioned by that code. Since I can get away with
using
an 'executable' filter, I haven't delved further.

Also​:

In my attempts to write a filter, I discovered that errors and
warnings
issued from a filter sub are never seen, instead getting a "no
quotelike
operator at offset ..." message. It turns out that the fix is to
localize
$@​. While I did this in my sub, it's a common problem and it would be
helpful for Filter​::Simple to do the localization around the
invocation of
the filter sub.

This has perhaps been fixed in 5.28.0 ...
https://rt.perl.org/Public/Bug/Display.html?id=38920

but a lot of people are running earlier versions and Filter​::Simple
could
help.

Filter-Simple was upgraded to version 0.95 within the Perl 5 core
distribution in September 2017. However, it appears that no new CPAN
release was made; CPAN is stuck at version 0.94.

Bcc-ing some people who might be able to help with a new CPAN release.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Oct 7, 2019

From @tonycoz

On Sat, 10 Aug 2019 20​:55​:53 -0700, Jim@​balter.name wrote​:

Filter​::Simple version 0.94 (the latest) does not work with
'executable_no_comments' filters, producing

Can't use an undefined value as a subroutine reference at
/usr/share/perl5/5.26/Filter/Simple.pm line 132.

because there's no such type in %selector_for and it doesn't match
/^code/
(line 120). I suspect that it would work if that line were changed to

if ($type =~ /^(?​:code|executable)/) {

However, the documentation says that only code and code_no_comments
filters
do the blanking conditioned by that code. Since I can get away with
using
an 'executable' filter, I haven't delved further.

The attached fixes this I think.

Filter​::Simple could use a lot more tests.

Also​:

In my attempts to write a filter, I discovered that errors and
warnings
issued from a filter sub are never seen, instead getting a "no
quotelike
operator at offset ..." message. It turns out that the fix is to
localize
$@​. While I did this in my sub, it's a common problem and it would be
helpful for Filter​::Simple to do the localization around the
invocation of
the filter sub.

This has perhaps been fixed in 5.28.0 ...
https://rt.perl.org/Public/Bug/Display.html?id=38920

but a lot of people are running earlier versions and Filter​::Simple
could
help.

I couldn't reproduce this. I modfied ExeNoComments.pm from the attached patch​:

package Filter​::Simple​::ExeNoComments;

use Filter​::Simple;

FILTER_ONLY
  executable_no_comments => sub {
  /muddles/ and warn "Muddles are suspect!";
  $_ =~ /shromplex/ and die "We wants no shromplexes!";
  s/ABC/TEST/g;
  };

1;

and tried running this code​:

use lib 't/lib';
use Filter​::Simple​::ExeNoComments;

my $z = "muddles";
my $x = "shromplex";

The result​:

$ ../../perl -I../.. -MTestInit=U2T t/filter_exe_fail.pl
Muddles are suspect! at t/lib/Filter/Simple/ExeNoComments.pm line 7.
We wants no shromplexes! at t/lib/Filter/Simple/ExeNoComments.pm line 8.

Tony

@p5pRT
Copy link
Author

p5pRT commented Oct 7, 2019

From @tonycoz

0001-perl-134349-fix-and-sanity-check-for-executable_no_c.patch
From d79c729fe8a3fb291f85c4917078a61b49a76f2d Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 29 Aug 2019 11:24:35 +1000
Subject: (perl #134349) fix and sanity check for executable_no_comments

---
 MANIFEST                                              |  1 +
 dist/Filter-Simple/lib/Filter/Simple.pm               |  3 ++-
 dist/Filter-Simple/t/filter_only.t                    | 19 ++++++++++++++++++-
 .../t/lib/Filter/Simple/ExeNoComments.pm              | 12 ++++++++++++
 4 files changed, 33 insertions(+), 2 deletions(-)
 create mode 100644 dist/Filter-Simple/t/lib/Filter/Simple/ExeNoComments.pm

diff --git a/MANIFEST b/MANIFEST
index 7bf62d8479..9abc29258f 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -3563,6 +3563,7 @@ dist/Filter-Simple/t/filter.t					See if Filter::Simple works
 dist/Filter-Simple/t/filter_only.t				See if Filter::Simple works
 dist/Filter-Simple/t/import.t					See if Filter::Simple works
 dist/Filter-Simple/t/lib/Filter/Simple/CodeNoComments.pm	Helper file for Filter::Simple tests
+dist/Filter-Simple/t/lib/Filter/Simple/ExeNoComments.pm		Helper file for Filter::Simple tests
 dist/Filter-Simple/t/lib/Filter/Simple/ExportTest.pm		Helper file for Filter::Simple tests
 dist/Filter-Simple/t/lib/Filter/Simple/FilterOnlyTest.pm	Helper file for Filter::Simple tests
 dist/Filter-Simple/t/lib/Filter/Simple/FilterTest.pm		Helper file for Filter::Simple tests
diff --git a/dist/Filter-Simple/lib/Filter/Simple.pm b/dist/Filter-Simple/lib/Filter/Simple.pm
index 1dcf3c80bf..924c2aecbd 100644
--- a/dist/Filter-Simple/lib/Filter/Simple.pm
+++ b/dist/Filter-Simple/lib/Filter/Simple.pm
@@ -2,7 +2,7 @@ package Filter::Simple;
 
 use Text::Balanced ':ALL';
 
-our $VERSION = '0.95';
+our $VERSION = '0.96';
 
 use Filter::Util::Call;
 use Carp;
@@ -70,6 +70,7 @@ my %extractor_for = (
 my %selector_for = (
     all   => sub { my ($t)=@_; sub{ $_=$$_; $t->(@_); $_} },
     executable=> sub { my ($t)=@_; sub{ref() ? $_=$$_ : $t->(@_); $_} }, 
+    executable_no_comments=> sub { my ($t)=@_; sub{ref() ? $_=$$_ : $t->(@_); $_} },
     quotelike => sub { my ($t)=@_; sub{ref() && do{$_=$$_; $t->(@_)}; $_} },
     regex     => sub { my ($t)=@_;
                sub{ref() or return $_;
diff --git a/dist/Filter-Simple/t/filter_only.t b/dist/Filter-Simple/t/filter_only.t
index 57f108677e..cd86707efc 100644
--- a/dist/Filter-Simple/t/filter_only.t
+++ b/dist/Filter-Simple/t/filter_only.t
@@ -4,7 +4,7 @@ BEGIN {
 
 use Filter::Simple::FilterOnlyTest qr/not ok/ => "ok", 
                                    "bad" => "ok", fail => "die";
-print "1..9\n";
+print "1..11\n";
 
 sub fail { print "ok ", $_[0], "\n" }
 sub ok { print "ok ", $_[0], "\n" }
@@ -41,3 +41,20 @@ print "ok 8\n";
 
 print "not " unless "bad" =~ /bad/;
 print "ok 9\n";
+
+use Filter::Simple::ExeNoComments;
+
+=for us
+
+shromplex
+
+=cut
+
+# shromplex
+
+# test the difference from code*
+my $x = "ABC";
+
+print $x eq "TEST" ? "" : "not ", "ok 10 # check strings processed\n";
+
+print "ok 11 # executable_no_comments\n";
diff --git a/dist/Filter-Simple/t/lib/Filter/Simple/ExeNoComments.pm b/dist/Filter-Simple/t/lib/Filter/Simple/ExeNoComments.pm
new file mode 100644
index 0000000000..db2c7bf4f3
--- /dev/null
+++ b/dist/Filter-Simple/t/lib/Filter/Simple/ExeNoComments.pm
@@ -0,0 +1,12 @@
+package Filter::Simple::ExeNoComments;
+
+use Filter::Simple;
+
+FILTER_ONLY
+  executable_no_comments => sub {
+      print STDERR "code <$_>\n";
+            $_ =~ /shromplex/ and die "We wants no shromplexes!";
+            s/ABC/TEST/g;
+	};
+
+1;
-- 
2.11.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant