File::Copy::syscopy doesn't preserve attrs as intended on Linux #12712

p5pRT · 2013-01-19T02:55:37Z

Migrated from rt.perl.org#116451 (status was 'open')

Searchable as RT116451$

p5pRT · 2013-01-19T02:55:37Z

From perl-diddler@tlinx.org

Created by perl-diddler@tlinx.org

File::Copy has a 'syscopy' function that is designed to "preserve
OS-specific attributes and file structure". It then goes on to state
that on Unix systems this is equivalent to the simply copy routine which
doesn't preserve OS-specific attributes.

However, the linux cp command has had a --preserve option for a couple
of years now -- that is intended to preserve all of the OS-specific
attributes and file-structure. (mode, ownership, timestamps, and, if
possible, additional attributes, context, links, xattr, acl's (says all).

Given the design intent of syscopy (over a normal copy), it seems negligent,
not to provide the intended functionality of the call where possible
(e.g. linux), as it appears this functionality is implemented on other
platforms (Mac, Windows...)

If the above verbiage about 'unix' is not meant to apply to linux and
this routine really does call cp --preserve, could the documentation
be updated? ;-)

Thanks!

Perl Info


Flags:
    category=library
    severity=medium
    module=File::Copy

This perlbug was built using Perl 5.16.1 - Mon Sep 10 13:51:31 UTC 2012
It is being executed now by  Perl 5.16.1 - Mon Sep 10 13:45:59 UTC 2012.

Site configuration information for perl 5.16.1:

Configured by abuild at Mon Sep 10 13:45:59 UTC 2012.

Summary of my perl5 (revision 5 version 16 subversion 1) configuration:
   
  Platform:
    osname=linux, osvers=3.4.6-2.10-default, archname=x86_64-linux-thread-multi
    uname='linux build35 3.4.6-2.10-default #1 smp thu jul 26 09:36:26 utc 2012 (641c197) x86_64 x86_64 x86_64 gnulinux '
    config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Dd_dbm_open -Duseshrplib=true -Doptimize=-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe -Accflags=-DPERL_USE_SAFE_PUTENV -Dotherlibdirs=/usr/lib/perl5/site_perl'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector'
    ccversion='', gccversion='4.7.1 20120723 [gcc-4_7-branch revision 189773]', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib64 -fstack-protector'
    libpth=/lib64 /usr/lib64 /usr/local/lib64
    libs=-lm -ldl -lcrypt -lpthread
    perllibs=-lm -ldl -lcrypt -lpthread
    libc=/lib64/libc-2.16.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.16'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.16.1/x86_64-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64 -fstack-protector'

Locally applied patches:
    


@INC for perl 5.16.1:
    /home/law/bin/lib
    /usr/lib/perl5/site_perl/5.16.1/x86_64-linux-thread-multi
    /usr/lib/perl5/site_perl/5.16.1
    /usr/lib/perl5/vendor_perl/5.16.1/x86_64-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.16.1
    /usr/lib/perl5/5.16.1/x86_64-linux-thread-multi
    /usr/lib/perl5/5.16.1
    /usr/lib/perl5/site_perl/5.16.1/x86_64-linux-thread-multi
    /usr/lib/perl5/site_perl/5.16.1
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.16.0/x86_64-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.16.0
    /usr/lib/perl5/vendor_perl
    /usr/lib/perl5/site_perl
    .


Environment for perl 5.16.1:
    HOME=/home/law
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LC_COLLATE=C
    LC_CTYPE=en_US.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=.:/sbin:/usr/local/sbin:/home/law/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/kde3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/opt/dell/srvadmin/bin:/usr/sbin:/etc/local/func_lib:/home/law/lib:/home/law/bin/lib
    PERL5OPT=-CSA -I/home/law/bin/lib
    PERL_BADLANG (unset)
    SHELL=/bin/bash

p5pRT · 2013-02-17T01:37:17Z

From @rjbs

Thanks, either the documentation or the implementation needs to change. Maybe the
documentation for now and the implementation later.

Thanks for the report.

--
rjbs

p5pRT · 2013-02-17T01:37:17Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2013-09-06T02:06:15Z

From @rjbs

I have re-visited this ticket and had a look at the documentation for syscopy, which reads like
this:

File::Copy also provides the "syscopy" routine, which copies the
file specified in the first parameter to the file specified in the
second parameter, preserving OS-specific attributes and file
structure. For Unix systems, this is equivalent to the simple
"copy" routine, which doesn't preserve OS-specific attributes. For
VMS systems, this calls the "rmscopy" routine (see below). For
OS/2 systems, this calls the "syscopy" XSUB directly. For Win32
systems, this calls "Win32::CopyFile".

I don't think I have a change to suggest that is worth making. The "For Unix systems" bit is so
close to the previous statement that I'm not sure any clarification is needed. We could change
the "Unix systems" to "systems other than OS/2, VMS, and Win32."

I worry that considering changing the behavior of syscopy, in the future, is going to make it less
and less reliable, rather than suddenly doing what people want.

Abigail has, if I recall correctly, been recommending just executing `cp -p` for years. I think
this may be the best advice for people seeking the behavior they know on their platform.

--
rjbs

p5pRT · 2013-09-06T02:41:51Z

From perl-diddler@tlinx.org

On Thu Sep 05 19:06:15 2013, rjbs wrote:

I have re-visited this ticket and had a look at the documentation for
syscopy, which reads like
this:
       File&#8203;::Copy also provides the "syscopy" routine\, which
copies the
file specified in the first parameter to the file specified
in the
second parameter, preserving OS-specific attributes and
file
structure. For Unix systems, this is equivalent to the
simple
"copy" routine, which doesn't preserve OS-specific
attributes.
For
Win32
systems, this calls "Win32::CopyFile" [which does preserve
attributes].

I don't think I have a change to suggest that is worth making. The
"For Unix systems" bit is so
close to the previous statement that I'm not sure any clarification is
needed.
====
The difference being -- that on Mac & Win32, the file attribute are
preserved (include ACL's and extended data and times).

If you even just wanted to preserve the time, all the platforms except
unix do this. But ACL's, Extended attr's and ownership are just as
important on Unix as on Win/Mac.

As for changing Unix to be consistent with Mac & Win -- that would seem
to make the call more reliable. As it is now, the behavior is not
reliable between OS's. So it can't be used, reliably, in a program to
get the same effect across OS's.

But that seemed to be the whole point of File::Copy -- to provide an
architecture-agnostic method to "clone" or make an exact copy of a file.

If it does something different for each platform -- especially in not
preserving permissions/access lists and data-forks/alternate data
streams, then yes -- might as well use cp -a (-p only preserves
ownership and permission bits) not ACL's or extended attr's.

I think the point abagail might be making is that this module is
basically broken for Unix -- but works for Mac and Win in making full
copies of the file....

We could change

the "Unix systems" to "systems other than OS/2, VMS, and Win32."

I worry that considering changing the behavior of syscopy, in the
future, is going to make it less
and less reliable, rather than suddenly doing what people want.

Abigail has, if I recall correctly, been recommending just executing
`cp -p` for years. I think
this may be the best advice for people seeking the behavior they know
on their platform.

p5pRT · 2013-09-06T03:01:44Z

From @rjbs

* Linda Walsh via RT <perlbug-followup@perl.org> [2013-09-05T22:41:51]

The difference being -- that on Mac & Win32, the file attribute are
preserved (include ACL's and extended data and times).

That's odd, how does Mac gets this different behavior? There is no special
casing for it in the code (or documentation) that I can see.

[type type type]

Actually, this does not appear to be true:

/etc$ uname -a
Darwin walrus.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May 1 17:57:12
PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
/etc$ /bin/ls -l -@ ntp.conf
-rw-r--r--@ 1 root wheel 22 Oct 22 2010 ntp.conf
com.apple.TextEncoding 15
/etc$ perl -MFile::Copy -E 'File::Copy::syscopy("ntp.conf", "$ENV{HOME}/ntp.conf")'
/etc$ /bin/ls -l -@ ~/ntp.conf
-rw-r--r-- 1 rjbs rjbs 22 Sep 5 22:51 /Users/rjbs/ntp.conf

If you even just wanted to preserve the time, all the platforms except
unix do this. But ACL's, Extended attr's and ownership are just as
important on Unix as on Win/Mac.

I just don't think this is true. From what I can tell, there are three
platforms that do anything other than what copy does: VMS, OS/2, and Win32. Of
all the rest of the platforms, most of which fall under the umbrella of "Unix"
(see perlport), syscopy is literally just copy:

# &syscopy is an XSUB under OS/2
unless (defined &syscopy) {
if ($^O eq 'VMS') {
*syscopy = \&rmscopy;
} elsif ($^O eq 'MSWin32' && defined &DynaLoader::boot_DynaLoader) {
# Win32::CopyFile() fill only work if we can load Win32.xs
*syscopy = sub {
return 0 unless @_ == 2;
return Win32::CopyFile(@_, 1);
};
} else {
$Syscopy_is_copy = 1;
*syscopy = \©
}
}

As for changing Unix to be consistent with Mac & Win -- that would seem
to make the call more reliable. As it is now, the behavior is not
reliable between OS's. So it can't be used, reliably, in a program to
get the same effect across OS's.

I feel somewhat uncomfortable with equating the kind of "everything-preserving
copy" that you get on the three platforms that provide it. Does rmscopy really
do the same kind of thing that `cp -p` does? Based on my limited understanding
of VMS, I think the answer is no.

So by making syscopy start to act like `cp -p`, I don't think we're getting
better cross-platform portability. And then there's the question of whether we
can really guarantee proper cross-platform portability of attribute-preserving
copy across all unixes! Maybe it's trivial, but to quote a wookiee, "I've got
a bad feeling about this."

--
rjbs

p5pRT added Severity Medium distro-Linux type-library labels Oct 19, 2019

xenu removed the affects-5.16 label Nov 19, 2021

xenu removed the Severity Medium label Dec 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

File::Copy::syscopy doesn't preserve attrs as intended on Linux #12712

File::Copy::syscopy doesn't preserve attrs as intended on Linux #12712

p5pRT commented Jan 19, 2013

p5pRT commented Jan 19, 2013

p5pRT commented Feb 17, 2013

p5pRT commented Feb 17, 2013

p5pRT commented Sep 6, 2013

p5pRT commented Sep 6, 2013

p5pRT commented Sep 6, 2013

File::Copy::syscopy doesn't preserve attrs as intended on Linux #12712

File::Copy::syscopy doesn't preserve attrs as intended on Linux #12712

Comments

p5pRT commented Jan 19, 2013

p5pRT commented Jan 19, 2013

From perl-diddler@tlinx.org

Created by perl-diddler@tlinx.org

p5pRT commented Feb 17, 2013

From @rjbs

p5pRT commented Feb 17, 2013

p5pRT commented Sep 6, 2013

From @rjbs

p5pRT commented Sep 6, 2013

From perl-diddler@tlinx.org

p5pRT commented Sep 6, 2013

From @rjbs