From muir@arubanetworks.com

Created by muir@arubanetworks.com

This is a bug report for perl from muir@​arubanetworks.com,
generated with the help of perlbug 1.39 running under perl 5.10.1.

-----------------------------------------------------------------

binmode() can clobber $@​ under certain circumstances. This reproduces it​:

$@​ = 'foo';

open(my $fh, '>/tmp/foo');
binmode($fh, '​:encoding(UTF-8)');

warn $@​;

Note that setting the binmode to '​:raw' does not clobber $@​.

If this turns out to not be a bug for some reason, the perldoc for binmode
should mention this behavior.

In case you are curious, the first symptom I saw was die() being unusually
silent (but still exiting the program). We had an object with a DESTROY
method hitting binmode in this way, which of course clobbered the exception
after it was thrown but before it was output to STDERR.

Flags:
    category=core
    severity=low

Site configuration information for perl 5.10.1:

Configured by Red Hat, Inc. at Tue Jun 29 15:55:54 PDT 2010.

Summary of my perl5 (revision 5 version 10 subversion 1) configuration:
   
  Platform:
    osname=linux, osvers=2.6.18-194.3.1.el5, archname=x86_64-linux-thread-multi
    uname='linux kilo.corp.airwave.com 2.6.18-194.3.1.el5 #1 smp thu may 13 13:08:30 edt 2010 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=-O2 -g -m64 -mtune=generic -Dversion=5.10.1 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dman1dir=/opt/airwave/share/man/man1 -Dman3dir=/opt/airwave/share/man/man3 -Dcf_by=Red Hat, Inc. -Dinstallprefix=/opt/airwave -Dprefix=/opt/airwave -Dlibpth=/usr/lib64 /lib64 /opt/airwave/lib64 /usr/lib /lib /opt/airwave/lib -Dprivlib=/opt/airwave/lib/perl5/5.10.1 -Dsitelib=/opt/airwave/local/lib/perl5/site_perl/5.10.1 -Dvendorlib=/opt/airwave/lib/perl5/vendor_perl/5.10.1 -Darchlib=/opt/airwave/lib64/perl5/5.10.1/x86_64-linux-thread-multi -Dsitearch=/opt/airwave/local/lib64/perl5/site_perl/5.10.1/x86_64-linux-thread-multi -Dvendorarch=/opt/airwave/lib64/perl5/vendor_perl/5.10.1/x86_64-linux-thread-multi -Dotherlibdirs=/opt/airwave/local/lib/perl5/site_perl/5.10.0:/opt/airwave/lib/perl5/vendor_perl/5.10.0 -Darchname=x86_64-linux-thread-multi -Dvendorprefix=/opt/airwave -Dsiteprefix=/opt/airwave/local -Duseshrplib -Dusethreads -Duseithreads -D
 uselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/opt/airwave/bin'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g -m64 -mtune=generic',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.1.2 20080704 (Red Hat 4.1.2-48)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -fstack-protector'
    libpth=/usr/lib64 /lib64 /opt/airwave/lib64 /usr/lib /lib /opt/airwave/lib
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.5.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.5'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/opt/airwave/lib64/perl5/5.10.1/x86_64-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -m64 -mtune=generic -fstack-protector'

Locally applied patches:
    


@INC for perl 5.10.1:
    /root/svn/mercury/lib/perl
    /root/svn/devtoys/rally/lib
    /root/svn/devtoys/bug_emails
    /root/svn/devtoys/farm
    /opt/airwave/lib64/perl5/5.10.1/x86_64-linux-thread-multi
    /opt/airwave/lib/perl5/5.10.1
    /opt/airwave/local/lib64/perl5/site_perl/5.10.1/x86_64-linux-thread-multi
    /opt/airwave/local/lib/perl5/site_perl/5.10.1
    /opt/airwave/local/lib/perl5/site_perl/5.10.0
    /opt/airwave/local/lib/perl5/site_perl
    /opt/airwave/lib64/perl5/vendor_perl/5.10.1/x86_64-linux-thread-multi
    /opt/airwave/lib/perl5/vendor_perl/5.10.1
    /opt/airwave/lib/perl5/vendor_perl/5.10.0
    /opt/airwave/lib/perl5/vendor_perl
    /opt/airwave/local/lib/perl5/site_perl/5.10.0
    /opt/airwave/lib/perl5/vendor_perl/5.10.0
    .


Environment for perl 5.10.1:
    HOME=/root
    LANG=en_US
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/opt/airwave/local/bin:/opt/airwave/bin:/usr/local/sbin:/root/svn/mercury/bin:/usr/local/airwave/bin:/usr/java/jre/bin:/usr/java/jdk/bin:/var/airwave/support:/opt/condor/sbin:/opt/condor/bin:/opt/flex/bin:/opt/ant/bin:/usr/sbin:/sbin:/usr/local/bin::/usr/kerberos/sbin:/usr/kerberos/bin:/bin:/usr/bin:/root/bin
    PERL5LIB=/root/svn/mercury/lib/perl:/root/svn/devtoys/rally/lib:/root/svn/devtoys/bug_emails:/root/svn/devtoys/farm
    PERL_BADLANG (unset)
    SHELL=/bin/bash

From @xdg

On Fri, Nov 5, 2010 at 5​:59 PM, muir@​arubanetworks.com
<perlbug-followup@​perl.org> wrote​:

binmode() can clobber $@​ under certain circumstances. This reproduces it​:

Added t/io/layers.t TODO test as commit 54f3160

The RT System itself - Status changed from 'new' to 'open'

From zefram@fysh.org

muir@​arubanetworks.com wrote​:

binmode() can clobber $@​ under certain circumstances.

Any place that you call into arbitrarily complex code can clobber $@​,
due to internal use of eval. PerlIO layers are arbitrarily complex code,
so it's not surprising that binmode would do this. It's as if you'd
called a function from a library module. We don't generally treat this
as a bug, but the specific problem you ran into is a different matter​:

In case you are curious, the first symptom I saw was die() being unusually
silent (but still exiting the program). We had an object with a DESTROY
method hitting binmode in this way, which of course clobbered the exception
after it was thrown but before it was output to STDERR.

In Perl 5.13 we have made a change so that a DESTROY method clobbering $@​
won't interfere with flight of an exception. This problem won't occur
in this form on Perl 5.14 when it is released.

In any case, for a DESTROY method to clobber $@​ or other status variables
is generally a bug in that method, because it intereferes with the flow
of code that doesn't know the method is being called. Same goes for
signal handlers, and this is in contrast to the situation of normal
library functions/methods where the dynamically surrounding code knows
it's calling an arbitrarily complex function. If a DESTROY method (or
signal handler) calls arbitrarily complex code, it must mediate between
these two sets of expectations by ensuring that the status variables
don't get clobbered, which it can easily do by

  local($., $@​, $!, $^E, $?);

This will avoid your problem on the Perl that you are using, and avoid
other forms of the problem on all Perl versions. There has been some
talk of this possibly happening automatically around DESTROY methods,
but nothing has been resolved there.

-zefram

From @druud62

On 2010-11-05 22​:59, muir @​ arubanetworks . com wrote​:

binmode() can clobber $@​ under certain circumstances. This reproduces it​:

$@​ = 'foo';

open(my $fh, '>/tmp/foo');
binmode($fh, '​:encoding(UTF-8)');

warn $@​;

Note that setting the binmode to '​:raw' does not clobber $@​.

If this turns out to not be a bug for some reason, the perldoc for binmode
should mention this behavior.

In case you are curious, the first symptom I saw was die() being unusually
silent (but still exiting the program). We had an object with a DESTROY
method hitting binmode in this way, which of course clobbered the exception
after it was thrown but before it was output to STDERR.

No bug. A quality of global variables is, well ... that they are global.

Decent Perl pattern​:

  my $result;

  eval {
  $result = myfunc();
  1; # success
  }
  or do {
  my $eval_error = $@​ || "unknown";
  ...;
  die $eval_error;
  };

To preserve $@​ you can localize it in your DESTROY.

See also Try​::Tiny and friends.

--
Ruud

From @xdg

On Sat, Nov 6, 2010 at 8​:03 AM, Zefram <zefram@​fysh.org> wrote​:

muir@​arubanetworks.com wrote​:

binmode() can clobber $@​ under certain circumstances.

Any place that you call into arbitrarily complex code can clobber $@​,
due to internal use of eval.  PerlIO layers are arbitrarily complex code,
so it's not surprising that binmode would do this.  It's as if you'd
called a function from a library module.  We don't generally treat this
as a bug, but the specific problem you ran into is a different matter​:

That's true, but it's not immediately obvious to a non-expert how
arbitrarily complex binmode() can be. Why not localize $@​ for the
duration of the binmode call?

(Though I agree with other comments that it's best practice to
localize $@​ and inspect it or copy it immediately after an eval)

-- David

From @cpansprout

Fixed in 523a494.

@cpansprout - Status changed from 'open' to 'resolved'

From @jandubois

On Sat, 06 Nov 2010, David Golden wrote​:

On Sat, Nov 6, 2010 at 8​:03 AM, Zefram <zefram@​fysh.org> wrote​:
That's true, but it's not immediately obvious to a non-expert how
arbitrarily complex binmode() can be. Why not localize $@​ for the
duration of the binmode call?

It is also not obvious to non-experts how arbitrarily complex even the
most simple Perl expressions can be due to tying and overloading.

I would like to register _some_ protest against adding gratuitous cruft
to the Perl internals to deal with non-bugs (I don't feel strongly about
this individual case, but the cumulative effect of many of these
special-purpose hacks over time is what I'm concerned about).

Cheers,
-Jan

From @xdg

On Sat, Nov 6, 2010 at 10​:29 PM, Jan Dubois <jand@​activestate.com> wrote​:

On Sat, 06 Nov 2010, David Golden wrote​:

On Sat, Nov 6, 2010 at 8​:03 AM, Zefram <zefram@​fysh.org> wrote​:
That's true, but it's not immediately obvious to a non-expert how
arbitrarily complex binmode() can be.  Why not localize $@​ for the
duration of the binmode call?

It is also not obvious to non-experts how arbitrarily complex even the
most simple Perl expressions can be due to tying and overloading.

Yes, but tying and overloading is usually done by the user, whereas a
non-expert doesn't realize exactly how many modules are loaded and
methods fired by perl to invoke binmode with an encoding. Most
built-in functions are not acting like arbitrarily complex function
calls.

I would like to register _some_ protest against adding gratuitous cruft
to the Perl internals to deal with non-bugs (I don't feel strongly about
this individual case, but the cumulative effect of many of these
special-purpose hacks over time is what I'm concerned about).

I understand the sentiment, entirely. I still hope someday for
something better than $@​ for managing exceptions.

-- David