File::Copy::mv fails to replicate behavior of Unix mv #8008

p5pRT · 2005-07-11T17:30:53Z

Migrated from rt.perl.org#36502 (status was 'resolved')

Searchable as RT36502$

p5pRT · 2005-07-11T17:30:53Z

From kynn@panix.com

File::Copy::mv does not preserve the file's mtime across NFS-mounted
filesystems. This differs from the behavior of the Unix mv(1)
command, and is not documented in the man page for File::Copy.

In the following example, /d00 is an NFS-mounted filesystem, while
/tmp is in /dev/sda5. First the behavior for File::Copy::mv:

% rm -f /tmp/foo /d00/home/kynn/bar
% touch -t 200001010000 /tmp/foo
% ls -l /tmp/foo
-rw-rw-r-- 1 kynn kynn 0 Jan 1 2000 /tmp/foo
% perl -MFile::Copy -we 'File::Copy::mv "/tmp/foo", "/d00/home/kynn/bar"'
% ls -l /d00/home/kynn/bar
-rw-rw-r-- 1 kynn kynn 0 Jul 11 13:11 /d00/home/kynn/bar

Now, for comparison, the behavior of mv(1):

% rm -f /tmp/foo /d00/home/kynn/bar
% touch -t 200001010000 /tmp/foo
% ls -l /tmp/foo
-rw-rw-r-- 1 kynn kynn 0 Jan 1 2000 /tmp/foo
% /bin/mv /tmp/foo /d00/home/kynn/bar
% ls -l /d00/home/kynn/bar
-rw-rw-r-- 1 kynn kynn 0 Jan 1 2000 /d00/home/kynn/bar

Perl Info


Flags:
    category=core
    severity=medium

Site configuration information for perl v5.8.6:

Configured by kynn at Tue Apr 12 12:24:11 EDT 2005.

Summary of my perl5 (revision 5 version 8 subversion 6) configuration:
  Platform:
    osname=linux, osvers=2.4.18-686-smp, archname=i686-linux-ld
    uname='linux luna 2.4.18-686-smp #1 smp sun apr 14 12:07:19 est 2002 i686 unknown '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=define
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='2.95.4 20011002 (Debian prerelease)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='long double', nvsize=12, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldbm -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=/lib/libc-2.2.5.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.2.5'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.6:
    /home/kynn/local/lib/perl/my
    /home/kynn/local/lib/perl/5.8.4
    /home/kynn/local/lib/perl/5.6.1
    /home/kynn/local/lib/perl5/5.8.6/i686-linux-ld
    /home/kynn/local/lib/perl5/5.8.6
    /home/kynn/local/lib/perl5/i686-linux-ld
    /home/kynn/local/lib/perl5
    /home/kynn/local/lib/perl5/5.8.6/i686-linux-ld
    /home/kynn/local/lib/perl5/5.8.6
    /home/kynn/local/lib/perl5/site_perl/5.8.6/i686-linux-ld
    /home/kynn/local/lib/perl5/site_perl/5.8.6
    /home/kynn/local/lib/perl5/site_perl
    .


Environment for perl v5.8.6:
    HOME=/home/kynn
    LANG=C
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/home/kynn/local/lib:/opt/java/jdk/lib/i386
    LOGDIR (unset)
    PATH=.:/home/kynn/local/bin:/home/kynn/local/usr/bin:/home/kynn/local/local/bin:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/home/kynn/java/javacc/bin:/opt/java/jdk/bin:/usr/pbs/bin:/home/kynn/opt/ant/bin
    PERL5LIB=/home/kynn/local/lib/perl/my:/home/kynn/local/lib/perl/5.8.4:/home/kynn/local/lib/perl/5.6.1:/home/kynn/local/lib/perl5
    PERL5_CPANPLUS_CONFIG=/home/kynn/.cpanplus/config
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

p5pRT · 2005-07-12T00:35:49Z

From @schwern

On Mon, Jul 11, 2005 at 10:30:54AM -0700, kynn jones wrote:

File::Copy::mv does not preserve the file's mtime across NFS-mounted
filesystems. This differs from the behavior of the Unix mv(1)
command, and is not documented in the man page for File::Copy.

You're right. And the problem affects any cross-partition move. mv()
falls back to copy() if it cannot rename, as in the case of moving across
partitions, but it does not adjust the mtimes to match.

The attached patch should fix this by setting the atime and mtime of the
copied file. Tests will be forthcoming after I overhaul the File::Copy
tests.

--
Michael G Schwern schwern@pobox.com http://www.pobox.com/~schwern
Ahh email, my old friend. Do you know that revenge is a dish that is best
served cold? And it is very cold on the Internet!

p5pRT · 2005-07-12T00:35:49Z

From @schwern

FC.patch

--- lib/File/Copy.pm	2005/07/12 00:28:38	1.1
+++ lib/File/Copy.pm	2005/07/12 00:31:50
@@ -205,7 +205,17 @@
                 $tosz2 == $fromsz;                         # it's all there
 
     ($tosz1,$tomt1) = (stat($to))[7,9];  # just in case rename did something
-    return 1 if copy($from,$to) && unlink($from);
+
+    {
+        local $@;
+        eval {
+            copy($from,$to) or die;
+            my($atime, $mtime) = (stat($from))[8,9];
+            utime($atime, $mtime, $to);
+            unlink($from)   or die;
+        };
+        return 1 unless $@;
+    }
     ($sts,$ossts) = ($! + 0, $^E + 0);
 
     ($tosz2,$tomt2) = ((stat($to))[7,9],0,0) if defined $tomt1;

p5pRT · 2005-07-12T00:35:55Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2005-07-12T02:16:02Z

From @schwern

Attached is a test patch to test File::Copy while copying/moving across
partitions. It simulates this by overriding rename() so that it always
fails. All tests are done twice, once with a working rename, once without.

It also adds a test for this bug, it checks to make sure the destination
of a move() retains the source's mtime. The test is written in such a
way that it should continue to work on systems which do not support
mtime or utime().

p5pRT · 2005-07-12T02:16:02Z

From @schwern

mtime.patch

--- lib/File/Copy.t	2005/07/12 01:45:49	1.2
+++ lib/File/Copy.t	2005/07/12 02:13:21
@@ -9,12 +9,23 @@
 
 my $TB = Test::More->builder;
 
-plan tests => 46;
+plan tests => 48;
+
+# We're going to override rename() later on but Perl has to see an override
+# at compile time to honor it.
+BEGIN { *CORE::GLOBAL::rename = sub { CORE::rename($_[0], $_[1]) }; }
+
 
 use File::Copy;
 use Config;
 
-for (1..2) {
+for my $cross_partition_test (0..1) {
+  {
+    # Simulate a cross-partition copy/move by forcing rename to
+    # fail.
+    no warnings 'redefine';
+    *CORE::GLOBAL::rename = sub { 0 } if $cross_partition_test;
+  }
 
   # First we create a file
   open(F, ">file-$$") or die;
@@ -72,12 +83,26 @@
   ok !move("file-$$", "copy-$$"), "move on missing file";
   ok -e "copy-$$",                '  target still there';
 
+  # Doesn't really matter what time it is as long as its not now.
+  my $time = 1000000000;
+  utime( $time, $time, "copy-$$" );
+
+  # Recheck the mtime rather than rely on utime in case we're on a
+  # system where utime doesn't work or there's no mtime at all.
+  # The destination file will reflect the same difficulties.
+  my $mtime = (stat("copy-$$"))[9];
+
   ok move "copy-$$", "file-$$", 'move';
   ok -e "file-$$",              '  destination exists';
   ok !-e "copy-$$",              '  source does not';
   open(R, "file-$$") or die; $foo = <R>; close(R);
   is $foo, "ok\n";
 
+  my $dest_mtime = (stat("file-$$"))[9];
+  is $dest_mtime, $mtime,
+    "mtime preserved by copy()". 
+    ($cross_partition_test ? " while testing cross-partition" : "");
+
   copy "file-$$", "lib";
   open(R, "lib/file-$$") or die; $foo = <R>; close(R);
   is $foo, "ok\n";
@@ -130,7 +155,6 @@
     unlink "hardlink-$$";
     unlink "file-$$";
   }
-
 }

p5pRT · 2005-07-12T11:32:00Z

From @steve-m-hay

Michael G Schwern wrote:

On Mon, Jul 11, 2005 at 10:30:54AM -0700, kynn jones wrote:

File::Copy::mv does not preserve the file's mtime across NFS-mounted
filesystems. This differs from the behavior of the Unix mv(1)
command, and is not documented in the man page for File::Copy.

You're right. And the problem affects any cross-partition move. mv()
falls back to copy() if it cannot rename, as in the case of moving across
partitions, but it does not adjust the mtimes to match.

The attached patch should fix this by setting the atime and mtime of the
copied file.

Thanks. Applied as change 25120.

Radan Computational Ltd.

The information contained in this message and any files transmitted with it are confidential and intended for the addressee(s) only. If you have received this message in error or there are any problems, please notify the sender immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Radan Computational Ltd. The recipient(s) of this message should check it and any attached files for viruses: Radan Computational will accept no liability for any damage caused by any virus transmitted by this email.

p5pRT · 2005-07-12T12:09:07Z

From @steve-m-hay

Michael G Schwern via RT wrote:

Attached is a test patch to test File::Copy while copying/moving across
partitions. It simulates this by overriding rename() so that it always
fails. All tests are done twice, once with a working rename, once without.

Thanks - applied as change 25122.

Radan Computational Ltd.

The information contained in this message and any files transmitted with it are confidential and intended for the addressee(s) only. If you have received this message in error or there are any problems, please notify the sender immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Radan Computational Ltd. The recipient(s) of this message should check it and any attached files for viruses: Radan Computational will accept no liability for any damage caused by any virus transmitted by this email.

p5pRT · 2005-07-12T12:10:27Z

From @steve-m-hay

Now fixed in bleadperl, so should be fixed in 5.8.8, otherwise in 5.10.

p5pRT · 2005-07-12T12:10:29Z

@steve-m-hay - Status changed from 'open' to 'resolved'

p5pRT closed this as completed Jul 12, 2005

p5pRT added Severity Medium distro-Linux type-core labels Oct 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

File::Copy::mv fails to replicate behavior of Unix mv #8008

File::Copy::mv fails to replicate behavior of Unix mv #8008

p5pRT commented Jul 11, 2005

p5pRT commented Jul 11, 2005

p5pRT commented Jul 12, 2005

p5pRT commented Jul 12, 2005

p5pRT commented Jul 12, 2005

p5pRT commented Jul 12, 2005

p5pRT commented Jul 12, 2005

p5pRT commented Jul 12, 2005

p5pRT commented Jul 12, 2005

p5pRT commented Jul 12, 2005

p5pRT commented Jul 12, 2005

File::Copy::mv fails to replicate behavior of Unix mv #8008

File::Copy::mv fails to replicate behavior of Unix mv #8008

Comments

p5pRT commented Jul 11, 2005

p5pRT commented Jul 11, 2005

From kynn@panix.com

p5pRT commented Jul 12, 2005

From @schwern

p5pRT commented Jul 12, 2005

From @schwern

p5pRT commented Jul 12, 2005

p5pRT commented Jul 12, 2005

From @schwern

p5pRT commented Jul 12, 2005

From @schwern

p5pRT commented Jul 12, 2005

From @steve-m-hay

p5pRT commented Jul 12, 2005

From @steve-m-hay

p5pRT commented Jul 12, 2005

From @steve-m-hay

p5pRT commented Jul 12, 2005