From ambrus@math.bme.hu

Created by ambrus@math.bme.hu

This is a strange problem with Filter​::Util​::Call. Here is a perl script
that installs a source filter that doesn't transform the code at all, then
follows a simple hello world using that filter.

use Filter​::Util​::Call;
BEGIN{ filter_add(sub{
  my $s = filter_read;
  my $t = $_;
  $_ = $t; # <---
  $s;
}); }
print "hello, world\n";

The above code works as expected​: it prints hello world.

However, if I modify it by deleting the line with the arrow, I get this​:

use Filter​::Util​::Call;
BEGIN{ filter_add(sub{
  my $s = filter_read;
  my $t = $_;
  $s;
}); }
print "hello, world\n";

This code does not work. It exits with 0 exit code without printing
anything. I can't see why they should be different, the line I deleted
shouldn't change the way the filter transforms the code.

I could reproduce this bug on both perl 5.10.0 linux-x86 and perl 5.8.8
linux-x86, both compiled from vanilla source.

I've first posted the bug at "http​://www.perlmonks.com/?node_id=664944" --
see the replies there.

Thanks and keep up the good work,

Ambrus

Flags:
    category=library
    severity=medium

Site configuration information for perl v5.8.8:

Configured by Debian Project at Sun Nov  4 15:45:42 UTC 2007.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.22-3-amd64, archname=x86_64-linux-gnu-thread-multi
    uname='linux deneb 2.6.22-3-amd64 #1 smp thu oct 11 15:23:23 utc
2007 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.8.8
-Dsitearch=/usr/local/lib/perl/5.8.8 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm
-Duseshrplib -Dlibperl=libperl.so.5.8.8 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define
usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=define uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
-DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN
-fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.1.2 20061115 (prerelease) (Debian
4.1.1-21)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.3.6.so, so=so, useshrplib=true, libperl=libperl.so.5.8.8
    gnulibc_version='2.3.6'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:



@INC for perl v5.8.8:
    /etc/perl
    /usr/local/lib/perl/5.8.8
    /usr/local/share/perl/5.8.8
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.8
    /usr/share/perl/5.8
    /usr/local/lib/site_perl
    .


Environment for perl v5.8.8:
    HOME=/home/student/ambrus
    LANG=C
    LANGUAGE (unset)
    LC_CTYPE=hu_HU
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/student/ambrus/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/bash

From @steve-m-hay

It is because the anonymous filter sub is supposed to return the
filtered source via $_, which will be empty unless you set it. (The
filter sub has a local copy of $_ which is always empty initially.)

The return value of the filter sub is simply a success/failure indicator
(< 0 for error, 0 for EOF, > 0 for OK).

This is all documented in the Filter​::Util​::Call manpage​:

http​://search.cpan.org/~pmqs/Filter-1.34/Call/Call.pm#filter()_and_anonymous_sub

@steve-m-hay - Status changed from 'new' to 'rejected'

From ambrus@math.bme.hu

(note​: this is a dupe of a message I sent in email to
"perlbug-followup\x40perl.org"​: as that mail doesn't seem to appear in
the web interface rt.perl.org I think I'd messed up something and repost
it using the web interface. sorry if you get this in dupe.)

On Thu Jan 31 08​:33​:12 2008, shay wrote​:

It is because the anonymous filter sub is supposed to return the
filtered source via $_, which will be empty unless you set it. (The
filter sub has a local copy of $_ which is always empty initially.)

The return value of the filter sub is simply a success/failure
indicator
(< 0 for error, 0 for EOF, > 0 for OK).

This is all documented in the Filter​::Util​::Call manpage​:

http​://search.cpan.org/~pmqs/Filter-
1.34/Call/Call.pm#filter()_and_anonymous_sub

Dear shay,

I'm sorry but I don't think your argument stands, and must insist on
my bug report. Please look at the source again.

1 use Filter​::Util​::Call;
2 BEGIN{ filter_add(sub{
3 my $s = filter_read;
4 my $t = $_;
5 $_ = $t; # <---
6 $s;
7 }); }
8 print "hello, world\n";

The variable $_ is filled with the original source by the filter_read
function, just like it's documented. If you remove both line 4 and
line 5, the code works and acts like a trivial source filter. Line 3
reads an original source line to $_ and saves the status to $s, and
line 6 returns this status from the filter sub, just like all the
simple examples in the Filter​::Util​::Call manpage. Thus, on exit from
the filter sub, $_ contains the original source line unchanged so this
should act as a source filter. Indeed, if you run this code you see
that it prints hello world because line 8 is not changed by the
filter.

The first snippet of the code in my bug report (with both line 4 and
line 5 included) doesn't seem much different from the above​: $_ gets
set to the source line by line 3, and line 5 copies the same value it
already have to $_. Indeed, this snippet appears to work just

The second code snippet in the bug report differs from this only in
that line 5 is omitted. Without that line, $_ should still contain
the source line that line 3 put there, and line 4 should not change
this in any way. Thus, I would think that this snippet should work
all the same. However, if I run it with perl 5.10.0 or 5.8.8, it
turns out that it doesn't print anything.

Finally, please note that BrowserUK has a suspicion on what causes
this bug at "http​://www.perlmonks.com/?node_id=664998"; and tye says
he believes it might be a reference count problem. I don't understand
the XS code so I can't say anything about these.

Thanks for your patience,

Ambrus

ambrus@math.bme.hu - Status changed from 'rejected' to 'new'

From ambrus@math.bme.hu

On ticket #50430, shay comments at Thu Jan 31 08​:33​:12 2008​:

It is because the anonymous filter sub is supposed to return the
filtered source via $_, which will be empty unless you set it. (The
filter sub has a local copy of $_ which is always empty initially.)

The return value of the filter sub is simply a success/failure indicator
(< 0 for error, 0 for EOF, > 0 for OK).
This is all documented in the Filter​::Util​::Call manpage​:
http​://search.cpan.org/~pmqs/Filter-1.34/Call/Call.pm#filter()_and_anonymous_sub

Dear shay,

I'm sorry but I don't think your argument stands, and must insist on
my bug report. Please look at the source again.

1 use Filter​::Util​::Call;
2 BEGIN{ filter_add(sub{
3 my $s = filter_read;
4 my $t = $_;
5 $_ = $t; # <---
6 $s;
7 }); }
8 print "hello, world\n";

The variable $_ is filled with the original source by the filter_read
function, just like it's documented. If you remove both line 4 and
line 5, the code works and acts like a trivial source filter. Line 3
reads an original source line to $_ and saves the status to $s, and
line 6 returns this status from the filter sub, just like all the
simple examples in the Filter​::Util​::Call manpage. Thus, on exit from
the filter sub, $_ contains the original source line unchanged so this
should act as a source filter. Indeed, if you run this code you see
that it prints hello world because line 8 is not changed by the
filter.

The first snippet of the code in my bug report (with both line 4 and
line 5 included) doesn't seem much different from the above​: $_ gets
set to the source line by line 3, and line 5 copies the same value it
already have to $_. Indeed, this snippet appears to work just

The second code snippet in the bug report differs from this only in
that line 5 is omitted. Without that line, $_ should still contain
the source line that line 3 put there, and line 4 should not change
this in any way. Thus, I would think that this snippet should work
all the same. However, if I run it with perl 5.10.0 or 5.8.8, it
turns out that it doesn't print anything.

Finally, please note that BrowserUK has a suspicion on what causes
this bug at "http​://www.perlmonks.com/?node_id=664998"; and tye says
he believes it might be a reference count problem. I don't understand
the XS code so I can't say anything about these.

Thanks for your patience,

Ambrus

From @steve-m-hay

Zsbán Ambrus wrote​:

On ticket #50430, shay comments at Thu Jan 31 08​:33​:12 2008​:

It is because the anonymous filter sub is supposed to return the
filtered source via $_, which will be empty unless you set it. (The
filter sub has a local copy of $_ which is always empty initially.)

The return value of the filter sub is simply a success/failure
indicator (< 0 for error, 0 for EOF, > 0 for OK).
This is all documented in the Filter​::Util​::Call manpage​:
http​://search.cpan.org/~pmqs/Filter-1.34/Call/Call.pm#filter()_and_anonymous_sub

Dear shay,

I'm sorry but I don't think your argument stands, and must insist on
my bug report. Please look at the source again.

1 use Filter​::Util​::Call;
2 BEGIN{ filter_add(sub{
3 my $s = filter_read;
4 my $t = $_;
5 $_ = $t; # <---
6 $s;
7 }); }
8 print "hello, world\n";

The variable $_ is filled with the original source by the filter_read
function, just like it's documented.

You're quite correct. My apologies for being too hasty before. I'd missed the fact that filter_read sets $_ for you, despite it also being clearly documented.

Finally, please note that BrowserUK has a suspicion on what causes
this bug at "http​://www.perlmonks.com/?node_id=664998"; and tye says
he believes it might be a reference count problem. I don't understand
the XS code so I can't say anything about these.

I had a quick look and I wonder if it is to do with the "local" $_ that is created prior to the perl_call_sv() being made mortal. The debugger shows DEFSV is cleared by the time perl_call_sv() returns, so perhaps a FREETMPS has happened somewhere? (Can ENTER+SAVETMPS / FREETMPS+LEAVE blocks be nested? I've always been a little uneasy with this stuff, and am currently rather rusty too.)

Anyway, the attached patch moves the mortalization of the local $_ until just before the FREETMPS, well after the perl_call_sv(), and fixes the bug without breaking any tests.

Would somebody with more knowledge of these matters care to comment?

From @steve-m-hay

filter.pat

The RT System itself - Status changed from 'new' to 'open'

From @steve-m-hay

My previous patch is now applied to bleadperl in change 33341.

@steve-m-hay - Status changed from 'open' to 'resolved'