Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFE: file handle duping in the Bourne shell tradition #13146

Closed
p5pRT opened this issue Aug 2, 2013 · 13 comments
Closed

RFE: file handle duping in the Bourne shell tradition #13146

p5pRT opened this issue Aug 2, 2013 · 13 comments

Comments

@p5pRT
Copy link

p5pRT commented Aug 2, 2013

Migrated from rt.perl.org#119127 (status was 'rejected')

Searchable as RT119127$

@p5pRT
Copy link
Author

p5pRT commented Aug 2, 2013

From perl-diddler@tlinx.org

Created by perl-diddler@tlinx.org

I noted the 'open' manpage mentions using >& in the bourne
shell tradition. In the same vein, using 2>&1, is a common way
to dup file descriptors, and even though the perl manpage
says if the expression on the right looks like a number, it will
treat it as a 'fd', it seems to neglect the idea that a simple
number on either side, in standard shell, refers to a fd.

Could the open call be made to recognize integer file descriptors
in the bourne shell tradition. Otherwise, we really should lose the
misleading, though perhaps technically accurate verbiage on the
manpage. I'd perfer the former though as the perl open call
in generally will handle it if you use such syntax with an
executed command -- Just not standalone to redirect file handles
prior to execution​:

perl -e'
open ("2>&1") || die "dup of 2->1 gave $!";
'
dup of 2->1 gave No such file or directory at -e line 2.
(or)
perl -e'open (" exec 2>&1") || die "dup of 2->1 gave $!"'
dup of 2->1 gave No such file or directory at -e line 1.
----
I.e. if one wanted to perform redirections before doing
something else and one wanted to use bourne shell syntax,
which perl had as one of its model utils, it doesn't work.

Perl Info

Flags:
    category=core
    severity=wishlist

This perlbug was built using Perl 5.16.2 - Fri Feb 15 01:17:37 UTC 2013
It is being executed now by  Perl 5.16.2 - Fri Feb 15 01:12:05 UTC 2013.

Site configuration information for perl 5.16.2:

Configured by abuild at Fri Feb 15 01:12:05 UTC 2013.

Summary of my perl5 (revision 5 version 16 subversion 2) configuration:
   
  Platform:
    osname=linux, osvers=3.4.6-2.10-default, archname=x86_64-linux-thread-multi
    uname='linux build34 3.4.6-2.10-default #1 smp thu jul 26 09:36:26 utc 2012 (641c197) x86_64 x86_64 x86_64 gnulinux '
    config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Dd_dbm_open -Duseshrplib=true -Doptimize=-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe -Accflags=-DPERL_USE_SAFE_PUTENV -Dotherlibdirs=/usr/lib/perl5/site_perl'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector'
    ccversion='', gccversion='4.7.2 20130108 [gcc-4_7-branch revision 195012]', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib64 -fstack-protector'
    libpth=/lib64 /usr/lib64 /usr/local/lib64
    libs=-lm -ldl -lcrypt -lpthread
    perllibs=-lm -ldl -lcrypt -lpthread
    libc=/lib64/libc-2.17.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.17'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.16.2/x86_64-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64 -fstack-protector'

Locally applied patches:
    


@INC for perl 5.16.2:
    /home/law/bin/lib
    /usr/lib/perl5/site_perl/5.16.2/x86_64-linux-thread-multi
    /usr/lib/perl5/site_perl/5.16.2
    /usr/lib/perl5/vendor_perl/5.16.2/x86_64-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.16.2
    /usr/lib/perl5/5.16.2/x86_64-linux-thread-multi
    /usr/lib/perl5/5.16.2
    /usr/lib/perl5/site_perl/5.16.2/x86_64-linux-thread-multi
    /usr/lib/perl5/site_perl/5.16.2
    /usr/lib/perl5/site_perl
    .


Environment for perl 5.16.2:
    HOME=/home/law
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LC_COLLATE=C
    LC_CTYPE=en_US.UTF-8
    LD_LIBRARY_PATH=/usr/lib64/mpi/gcc/openmpi/lib64
    LOGDIR (unset)
    PATH=/home/law/bin/lib:/sbin:/usr/local/sbin:/usr/lib64/mpi/gcc/openmpi/bin:/home/law/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/kde3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:.:/usr/lib/qt3/bin:/opt/dell/srvadmin/bin:/usr/sbin:/etc/local/func_lib:/home/law/lib
    PERL5OPT=-Mutf8 -CSA -I/home/law/bin/lib
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Aug 2, 2013

From @ikegami

Why would you want to redirect a descriptor for which you don't have a
handle?

For descriptors for which you do have a handle, you can do the following​:

$ perl -E'warn "foo\n";' >/dev/null
foo

$ perl -E'open(STDERR, ">&=", 1) or die $!; warn "foo\n";' >/dev/null

$

@p5pRT
Copy link
Author

p5pRT commented Aug 2, 2013

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Aug 2, 2013

From perl-diddler@tlinx.org

On Fri Aug 02 06​:20​:32 2013, ikegami@​adaelis.com wrote​:

Why would you want to redirect a descriptor for which you don't have a
handle?

For descriptors for which you do have a handle, you can do the following​:

$ perl -E'open(STDERR, ">&=", 1) or die $!; warn "foo\n";' >/dev/null


I know there are alternate ways that are more complex to do what I ask,
but the emphasis here was on the shell redirections people have gotten
used to in the current replacement for bourne, ksh or bash. Certainly,
you can't argue that 2>&1, is easier to use/type/remember than
"STDERR, ">&=, 1)" <- which doesn't look at all like shell -- which was
one of larry's selling points about perl -- at the time it combined
shell, awk, sed, grep, tr, etc... all into 1 cohesive prog -- did it
faster and was compatible. Now? Not so compatible, and not so easy.
No question that it is doable 12 different ways but if I want to dup a
FD to '3', or run a pipeline​:
($pipe, "ls 3>&2 2>/dev/null|sort 2>&3 |") # ignore ls errs but not sort

or ($errs, "2>", \$errvar) && ..runcmd... hoping to pick up stderr
in a variable easily with the write to var...

Sure, there's likely more complex ways to do it, but having the manpage
remind me of the simplicity of doing it in shell, then not allowing me
to use the same syntax, leaves me feeling resentful. Why'd the page
remind me of how easy it is when its so much more complex in perl when
perl was supposed to be as easy as shell? Somehow perl hasn't tracked
and included some of those basic features.

Note, in shell if you really want input from file "2", you need to
make it NOT a simple number, like "./2" or such...

@p5pRT
Copy link
Author

p5pRT commented Aug 2, 2013

From @rjbs

* Linda Walsh <perlbug-followup@​perl.org> [2013-08-02T07​:29​:13]

Could the open call be made to recognize integer file descriptors
in the bourne shell tradition.

I don't think this will happen.

Otherwise, we really should lose the misleading, though perhaps technically
accurate verbiage on the manpage.

What about if, instead of removing the bit about duping, we add a bit reminding
the user that this only trucks in filenames, not descriptor numbers.

--
rjbs

@p5pRT
Copy link
Author

p5pRT commented Aug 3, 2013

From @ikegami

On Fri, Aug 2, 2013 at 3​:53 PM, Linda Walsh via RT <
perlbug-followup@​perl.org> wrote​:

On Fri Aug 02 06​:20​:32 2013, ikegami@​adaelis.com wrote​:

Why would you want to redirect a descriptor for which you don't have a
handle?

For descriptors for which you do have a handle, you can do the following​:

$ perl -E'open(STDERR, ">&=", 1) or die $!; warn "foo\n";' >/dev/null

---
I know there are alternate ways that are more complex to do what I ask,

Maybe, but why would you mentioned those instead of the simpler one that
was mentioned?

but the emphasis here was on the shell redirections people have gotten
used to in the current replacement for bourne, ksh or bash. Certainly,
you can't argue that 2>&1, is easier to use/type/remember than
"STDERR, ">&=, 1)"

I've already said 2>&1 doesn't make much sense. You're actually suggesting

open(fileno($HANDLE).">&1")

which can currently be written as

open(STDERR, ">&1")

in addition to the more consistent

open(STDERR, ">&", 1)

@p5pRT
Copy link
Author

p5pRT commented Aug 3, 2013

From perl-diddler@tlinx.org

On Fri Aug 02 15​:33​:02 2013, perl.p5p@​rjbs.manxome.org wrote​:

* Linda Walsh <perlbug-followup@​perl.org> [2013-08-02T07​:29​:13]

Could the open call be made to recognize integer file descriptors
in the bourne shell tradition.

I don't think this will happen.


Now that's depressing. I was just thinking about perl's history and
where it came from (was thinking about this before, but more so since
this). So many of the utils perl came from evolved, yet the features
that were imported from those basic utils didn't evolve with those
utils. Yet, initially, Larry delayed the release of perl until he could
get a full-compatibility script out for one of those utils. To hope to
see perl evolve with (and beyond) it's roots seems unlikely.

Otherwise, we really should lose the misleading, though perhaps
technically
accurate verbiage on the manpage.

What about if, instead of removing the bit about duping, we add a bit
reminding
the user that this only trucks in filenames, not descriptor numbers.


Um... you mean right in the section where it says the EXPR after >& is
  "interpreted as the name of a filehandle (or file descriptor if
numeric) to be duped (as dup(2) and opened." or later "...if you use
<&=X, where "X" is a file descriptor number or a file handle, then Perl
will do..."

There are at least a few spots that refer to it working with file
descriptor numbers. That it only does so halfway on special occasions,
is not a gleaming jewel in Perl's feature set -- if are able to see how
that looks to people not so into perl's quirks...

@p5pRT
Copy link
Author

p5pRT commented Aug 3, 2013

From perl-diddler@tlinx.org

On Fri Aug 02 20​:03​:06 2013, ikegami@​adaelis.com wrote​:

I've already said 2>&1 doesn't make much sense. You're actually
suggesting

open(fileno($HANDLE).">&1")

which can currently be written as

open(STDERR, ">&1")

I can dup 2 to three with 3>&2, what the perl syntax for that?
Of course there's the >& and |& that redirect or pipe both STDERR+OUT,
that's even shorter.

The point is perl takes fd's on the right -- why not on the left?

If I did open(or maybe exec?) (undef, "2>&1") perl would redirect
STDERR to STDOUT without me having to spell it out for it.

Ideally, I could do something like (undef, "2>&", \$var) and get
STDERR to a var (which I guess you would do with
open(STDERR, ">&", \$var) ?

It's really boiling down to conciseness, balance, and form... having it
be asymmetric, unnecessarily, just makes it that much harder to do
things in perl. A main measure of the strength of a language is is its
conciseness and how much it can allow the programmer to express/create
their ideas with minimal effort/output.

@p5pRT
Copy link
Author

p5pRT commented Aug 3, 2013

From @rjbs

* Linda Walsh via RT <perlbug-followup@​perl.org> [2013-08-02T23​:13​:33]

On Fri Aug 02 15​:33​:02 2013, perl.p5p@​rjbs.manxome.org wrote​:

* Linda Walsh <perlbug-followup@​perl.org> [2013-08-02T07​:29​:13]

Could the open call be made to recognize integer file descriptors
in the bourne shell tradition.

I don't think this will happen.
---

Now that's depressing.
[...]
To hope to see perl evolve with (and beyond) it's roots seems unlikely.

Perl not making one particular change to the behavior of the already very
complicated "open" operation is not the same as Perl not growing better at all.

What about if, instead of removing the bit about duping, we add a bit
reminding the user that this only trucks in filenames, not descriptor
numbers.

Um... you mean right in the section where it says the EXPR after >& is
"interpreted as the name of a filehandle (or file descriptor if
numeric) to be duped (as dup(2) and opened." or later "...if you use
<&=X, where "X" is a file descriptor number or a file handle, then Perl
will do..."

I mean that we can clarify the way it does work by adding a clear piece of
text, rather than by omitting reference to the syntax that inspired what is
available.

--
rjbs

@p5pRT
Copy link
Author

p5pRT commented Aug 4, 2013

From @Leont

On Fri, Aug 2, 2013 at 1​:29 PM, Linda Walsh <perlbug-followup@​perl.org> wrote​:

I noted the 'open' manpage mentions using >& in the bourne
shell tradition. In the same vein, using 2>&1, is a common way
to dup file descriptors, and even though the perl manpage
says if the expression on the right looks like a number, it will
treat it as a 'fd', it seems to neglect the idea that a simple
number on either side, in standard shell, refers to a fd.

Could the open call be made to recognize integer file descriptors
in the bourne shell tradition. Otherwise, we really should lose the
misleading, though perhaps technically accurate verbiage on the
manpage. I'd perfer the former though as the perl open call
in generally will handle it if you use such syntax with an
executed command -- Just not standalone to redirect file handles
prior to execution​:

use POSIX 'dup2'
dup2(1, 2);

Leon

@p5pRT
Copy link
Author

p5pRT commented Aug 4, 2013

From perl-diddler@tlinx.org

On Fri Aug 02 20​:35​:51 2013, perl.p5p@​rjbs.manxome.org wrote​:

Perl not making one particular change to the behavior of the already
very complicated "open" operation is not the same as Perl not growing
better at all.
---- Well there are various measures of better as I'm sure you will
agree.

In this case, it was increasing the regularity of the syntax to approach
more the usage of the shell it claims tradition of as well, perhaps,
updating to some of the more current syntaxes as people using bash might
find it easier to use perl on a drop-in basis.

I think that was a major selling point, back when perl was young, is
that people could do what they were used to in ways they were used to,
have it integrated and work faster, but that's my perception. Perl's
declining popularity/usage bothers me, as fewer things are made to work
with it.

I mean that we can clarify the way it does work by adding a clear
piece of text, rather than by omitting reference to the syntax that
inspired what is available.


Well, it's just confusing since in some places it says shell is
called (in exec) to interpret chars, so those redirects work there,
and open implements a partial set, but not the whole set. Seemed to
me that having open be able to recognize when it should act more
like exec and offer up shell syntax, would be an improvement -- one
less special case to remember.

I tried open(my fd, "cmd 2>&1 |") to get both std err and out on
the same output handle. ... Nada...yet it is supported in the
system call? Having them support the same set of shell features
that they derived syntax from just seemed less "intuitively"
confusing (rather than legalistically explaining why the user's
intuition is wrong in the man page....

Obviously fixing the manpage makes perl legalistically correct, but does
it make it any easier to use?

I noticed that no one answer my more complex Q's save 2->3, dup2to1 FH,
cmd, restore, & read result.​: How to just as easily do
"3>&2 2>&1 |cmd| 2>&3 |" It would just be adding more
the same shell syntax people are used to....that it might
be hard...yeah, maybe, I can't argue that. I did say
this was a wish list item.

then there was my Q about how well the read/write to scalars integrated
into this scheme ... Can I redirect stderr & out at the same time into
a var? cf bash​: varname=$(cmd >&) or older way varname=$(cmd 2>&1 )...
The perl methods are less clear, IMO...

I asked for an easier way to include modules in the same file and be
able to use "use module name" without having it go out on disk. First
I was given the arcane ${INC{'Module/foo.pm"}=<path>}, then
representational form(Module​::foo.pm) when I simplifiied to what I asked
for "use mem;" to use the in-memory copy..

Before that I asked for why sprint and printf had to have different
proto types for common cases .. and was given answers that there might
be obscure programs that relied on that (exmamples?)... none-forthcoming
I wrote P which does that.

Before that-- have UTF8 default to reading local like most other desktop
utils....or have it be symetric -- if it read something as
binary-octets, then write the file in binary octets instead of issuing
a warning and dying -- certainly no one could be relying on the dying
behavior....?? Another lead ballon...*sigh*.

All the while in having these ideas, made to feel like a pariah and
eventually made one. That's why yet another simplification down the
tubes is depressing....

How about this one -- not require sigils where they are unnecessary or
could be figured out by context (where unclear -- required), but "my
($a, $b, $c); a=b+c;" or "my (@​ar, $b, $c); ar=(b,c);" (obviously only
possible if user doesn't use same varname for different types... if so,
the sigil would still be required to disambiguate)...I never botherd to
submit that knowing how it would be received...

anyway... this one -- wishlist that open supported fd's on both sides of
the ops as in a system or exec command where they are allowed (I think)
would still be nice no matter how documented not to work that way.

@p5pRT
Copy link
Author

p5pRT commented Dec 15, 2017

From zefram@fysh.org

The requested feature wouldn't make sense. open() consistently uses
its first argument as a file handle, and its design is focussed on the
needs of file handles. If you want to operate on raw file descriptors,
there's the POSIX module. This ticket should be closed.

-zefram

@p5pRT p5pRT closed this as completed Dec 15, 2017
@p5pRT
Copy link
Author

p5pRT commented Dec 15, 2017

@iabyn - Status changed from 'open' to 'rejected'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant