Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tied filehandles not working with XS-based I/O #14888

Closed
p5pRT opened this issue Sep 3, 2015 · 5 comments
Closed

Tied filehandles not working with XS-based I/O #14888

p5pRT opened this issue Sep 3, 2015 · 5 comments

Comments

@p5pRT
Copy link

p5pRT commented Sep 3, 2015

Migrated from rt.perl.org#125981 (status was 'resolved')

Searchable as RT125981$

@p5pRT
Copy link
Author

p5pRT commented Sep 3, 2015

From jdv@base2bio.com

Created by jdv@base2bio.com

Tied filehandles don't appear to be working as expected in combination
with modules using XS-based I/O (e.g. PerlIO_read). For example,
passing a tied filehandle to the core Digest​::MD5​::addfile() croaks
with 'No filehandle passed'. If the filehandle is first opened on one
file before being tied and opened on another file, the XS-based class
will read from the first file opened and not pass read() calls through
to the tied class' READ method. Plugging in a pure-Perl module instead
works as expected.

Discussed at https://www.perlmonks.org/?node_id=1140411

A MWE example follows. The same behavior is observed using other
Tie​::Handle​:: classes from CPAN and other XS-based modules which take
filehandles as input.

## BEGIN CODE

#!/usr/bin/perl

package TieTest;
use Symbol;

sub TIEHANDLE { return bless gensym(), shift; }
sub OPEN { open $_[0], $_[1], $_[2]; }
sub READ { warn "...tied read\n"; read $_[0], $_[1], $_[2],
$_[3]; } sub SEEK { seek $_[0], $_[1], $_[2]; }

package main;

use strict;
use warnings;
use Digest​::MD5;
use Digest​::Perl​::MD5;

my ($testfile_0, $testfile_1) = @​ARGV;

open FH_PLAIN, '<', $testfile_0;

#open FH_TIED, '<', $testfile_1; # try uncommenting this
tie(*FH_TIED, 'TieTest');
open FH_TIED, '<', $testfile_0;

my %handles = (PLAIN => \*FH_PLAIN, TIED => \*FH_TIED);
for my $title (qw/PLAIN TIED/) {

  my $fh = $handles{$title};
  for my $class (qw/Digest​::Perl​::MD5 Digest​::MD5/) {

  my $sum = eval {$class->new->addfile($fh)-&gt;hexdigest} || $@​;
  printf "%-7s%-19s%-32s\n", $title, $class, $sum;
  seek $fh, 0, 0;
 
  }

}

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl 5.20.2:

Configured by Debian Project at Sun May  3 16:16:25 UTC 2015.

Summary of my perl5 (revision 5 version 20 subversion 2) configuration:
   
  Platform:
    osname=linux, osvers=3.2.0-4-amd64,
archname=x86_64-linux-gnu-thread-multi uname='linux x86-csail-01
3.2.0-4-amd64 #1 smp debian 3.2.68-1+deb7u1 x86_64 gnulinux '
config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat
-Werror=format-security -Dldflags= -Wl,-z,relro -Dlddlflags=-shared
-Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu
-Dprefix=/usr -Dprivlib=/usr/share/perl/5.20
-Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.20 -Dvendorprefix=/usr
-Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.20
-Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.20.2
-Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.20.2
-Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3
-Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3
-Duse64bitint -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio
-Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -Doptimize=-O2
-Duseshrplib -Dlibperl=libperl.so.5.20.2 -des' hint=recommended,
useposix=true, d_sigaction=define useithreads=define,
usemultiplicity=define use64bitint=define, use64bitall=define,
uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv
-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64', optimize='-O2 -g', cppflags='-D_REENTRANT
-D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe
-I/usr/local/include' ccversion='', gccversion='4.9.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8 alignbytes=8, prototype=define Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
perllibs=-ldl -lm -lpthread -lc -lcrypt libc=libc-2.19.so, so=so,
useshrplib=true, libperl=libperl.so.5.20 gnulibc_version='2.19' Dynamic
Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef,
ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared
-L/usr/local/lib -fstack-protector'

Locally applied patches:
    DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS
default for modules installed from CPAN. DEBPKG:debian/db_file_ver -
http://bugs.debian.org/340047 Remove overly restrictive DB_File version
check. DEBPKG:debian/doc_info - Replace generic man(1) instructions
with Debian-specific information. DEBPKG:debian/enc2xs_inc -
http://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and
ignore missing @INC directories. DEBPKG:debian/errno_ver -
http://bugs.debian.org/343351 Remove Errno version check due to upgrade
problems with long-running processes. DEBPKG:debian/libperl_embed_doc -
http://bugs.debian.org/186778 Note that libperl-dev package is required
for embedded linking DEBPKG:fixes/respect_umask - Respect umask during
installation DEBPKG:debian/writable_site_dirs - Set umask approproately
for site install directories DEBPKG:debian/extutils_set_libperl_path -
EU:MM: set location of libperl.a under /usr/lib
DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or
perllocal.pod for perl or vendor DEBPKG:debian/prefix_changes - Fiddle
with *PREFIX and variables written to the makefile
DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the
binary targets. DEBPKG:debian/instmodsh_doc - Debian policy doesn't
install .packlist files for core or vendor. DEBPKG:debian/ld_run_path -
Remove standard libs from LD_RUN_PATH as per Debian policy.
DEBPKG:debian/libnet_config_path - Set location of libnet.cfg
to /etc/perl/Net as /usr may not be writable. DEBPKG:debian/mod_paths -
Tweak @INC ordering for Debian
DEBPKG:debian/module_build_man_extensions -
http://bugs.debian.org/479460 Adjust Module::Build manual page
extensions for the Debian Perl policy DEBPKG:debian/prune_libs -
http://bugs.debian.org/128355 Prune the list of libraries wanted to
what we actually need. DEBPKG:fixes/net_smtp_docs - [rt.cpan.org
#36038] http://bugs.debian.org/100195 Document the Net::SMTP 'Port'
option DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make
perlivp skip include directories in /usr/local
DEBPKG:debian/deprecate-with-apt - http://bugs.debian.org/747628 Point
users to Debian packages of deprecated core modules
DEBPKG:debian/squelch-locale-warnings - http://bugs.debian.org/508764
Squelch locale warnings in Debian package maintainer scripts
DEBPKG:debian/skip-upstream-git-tests - Skip tests specific to the
upstream Git repository DEBPKG:debian/patchlevel -
http://bugs.debian.org/567489 List packaged patches for 5.20.2-3+deb8u1
in patchlevel.h DEBPKG:debian/skip-kfreebsd-crash -
http://bugs.debian.org/628493 [perl #96272] Skip a crashing test case
in t/op/threads.t on GNU/kFreeBSD
DEBPKG:fixes/document_makemaker_ccflags - http://bugs.debian.org/628522
[rt.cpan.org #68613] Document that CCFLAGS should include
$Config{ccflags} DEBPKG:debian/find_html2text -
http://bugs.debian.org/640479 Configure CPAN::Distribution with correct
name of html2text DEBPKG:debian/perl5db-x-terminal-emulator.patch -
http://bugs.debian.org/668490 Invoke x-terminal-emulator rather than
xterm in perl5db.pl DEBPKG:debian/cpan-missing-site-dirs -
http://bugs.debian.org/688842 Fix CPAN::FirstTime defaults with
nonexisting site dirs if a parent is writable
DEBPKG:fixes/memoize_storable_nstore - [rt.cpan.org #77790]
http://bugs.debian.org/587650 Memoize::Storable: respect 'nstore'
option not respected DEBPKG:debian/regen-skip - Skip a regeneration
check in unrelated git repositories DEBPKG:fixes/regcomp-mips-optim -
[perl #122817] http://bugs.debian.org/754054 Downgrade the optimization
of regcomp.c on mips and mipsel due to a gcc-4.9 bug
DEBPKG:debian/makemaker-pasthru - http://bugs.debian.org/758471 Pass LD
settings through to subdirectories DEBPKG:fixes/perldoc-less-R -
[rt.cpan.org #98636] http://bugs.debian.org/758689 Tell the 'less'
pager to allow terminal escape sequences
DEBPKG:fixes/pod_man_reproducible_date - http://bugs.debian.org/759405
Support POD_MAN_DATE in Pod::Man for the left-hand footer
DEBPKG:fixes/io_uncompress_gunzip_inmemory -
http://bugs.debian.org/747363 [rt.cpan.org #95494] Fix gunzip to
in-memory file handle DEBPKG:fixes/socket_test_recv_fix -
http://bugs.debian.org/758718 [perl #122657] Compare recv return value
to peername in socket test DEBPKG:fixes/hurd_socket_recv_todo -
http://bugs.debian.org/758718 [perl #122657] TODO checking the result
of recv() on hurd DEBPKG:fixes/regexp-performance - [0fa70a0]
http://bugs.debian.org/777556 [perl #123743] simpify and speed
up /.*.../ handling DEBPKG:fixes/failed_require_diagnostics -
http://bugs.debian.org/781120 [perl #123270] Report inaccesible file on
failed require DEBPKG:fixes/array-cloning -
http://bugs.debian.org/779357 [perl #124127] [902d169] fix cloning
arrays with unused elements DEBPKG:fixes/perldb-threads -
http://bugs.debian.org/779357 [perl #124127] [41ef2c6] lib/perl5db.pl:
Restore noop lock prototype


@INC for perl 5.20.2:
    /home/jeremy/perl5/lib/perl5/x86_64-linux-gnu-thread-multi
    /home/jeremy/perl5/lib/perl5
    /etc/perl
    /usr/local/lib/x86_64-linux-gnu/perl/5.20.2
    /usr/local/share/perl/5.20.2
    /usr/lib/x86_64-linux-gnu/perl5/5.20
    /usr/share/perl5
    /usr/lib/x86_64-linux-gnu/perl/5.20
    /usr/share/perl/5.20
    /usr/local/lib/site_perl
    .


Environment for perl 5.20.2:
    HOME=/home/jeremy
    LANG=en_US.utf8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/jeremy/.local/bin:/home/jeremy/.gem/ruby/1.9.1/bin:/home/jeremy/perl5/bin:/opt/bin
    PERL5LIB=/home/jeremy/perl5/lib/perl5
    PERL_BADLANG (unset)
    PERL_MB_OPT=--install_base "/home/jeremy/perl5"
    PERL_MM_OPT=INSTALL_BASE=/home/jeremy/perl5
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2015

From @tonycoz

On Thu Sep 03 15​:50​:47 2015, jdv@​base2bio.com wrote​:

Tied filehandles don't appear to be working as expected in combination
with modules using XS-based I/O (e.g. PerlIO_read). For example,
passing a tied filehandle to the core Digest​::MD5​::addfile() croaks
with 'No filehandle passed'. If the filehandle is first opened on one
file before being tied and opened on another file, the XS-based class
will read from the first file opened and not pass read() calls through
to the tied class' READ method. Plugging in a pure-Perl module instead
works as expected.

Discussed at https://www.perlmonks.org/?node_id=1140411

A MWE example follows. The same behavior is observed using other
Tie​::Handle​:: classes from CPAN and other XS-based modules which take
filehandles as input.

This isn't a bug.

PerlIO is the layer below the I/O you see at the perl level, by the time you have a PerlIO * value, the tied information (stored in the GV) isn't available.

If you do want your interface visible at the PerlIO level you'll need to implement it as a PerlIO layer, possibly using PerlIO​::via.

Unfortunately there's no simple XS interface for performing I/O on a GV that honors tie() - I've done it by switching between a callback that uses PerlIO vs a callback that uses call_sv() to call back to perl subs that print/read/seek as needed. Which won't help you for Digest​::MD5.

Tony

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Oct 21, 2015

From @tonycoz

On Sun Sep 13 18​:50​:34 2015, tonyc wrote​:

This isn't a bug.

Closing.

Tony

@p5pRT p5pRT closed this as completed Oct 21, 2015
@p5pRT
Copy link
Author

p5pRT commented Oct 21, 2015

@tonycoz - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant