Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blead File::Copy fails on netbsd 6.1.5 #16480

Open
p5pRT opened this issue Mar 26, 2018 · 10 comments
Open

blead File::Copy fails on netbsd 6.1.5 #16480

p5pRT opened this issue Mar 26, 2018 · 10 comments

Comments

@p5pRT
Copy link

p5pRT commented Mar 26, 2018

Migrated from rt.perl.org#133030 (status was 'open')

Searchable as RT133030$

@p5pRT
Copy link
Author

p5pRT commented Mar 26, 2018

From @khwilliamson

It is failing with

utimensat unimplemented in this platform at ../lib/File/Copy.t line 111.
# Looks like your test exited with 2 just after 15.

See http​://perl5.test-smoke.org/logfile/64220

@p5pRT
Copy link
Author

p5pRT commented Apr 16, 2018

From @tonycoz

On Mon, 26 Mar 2018 13​:19​:48 -0700, public@​khwilliamson.com wrote​:

It is failing with

utimensat unimplemented in this platform at ../lib/File/Copy.t line 111.
# Looks like your test exited with 2 just after 15.

See http​://perl5.test-smoke.org/logfile/64220

The problem is Time​::HiRes exports its utime() replacement if only one of two functions that depends on is available.

So if you import utime() successfully, the underlying function you need might not be available.

The first of the attached patches makes utime() available only if both are defined. I think this is suitable for 5.28, and prevents the test failure. It however means File​::Copy will no longer preserve sub-second modification times when utime() by name is unavailable.

The second patch adds a fallback to utimes() if that's available (typically on BSDs, which is the case covered by this ticket.)

This doesn't try to work around OS X's special behaviour.

The second patch might not be suitable for 5.28.

Unrelated to this particular bug - there's also Win32 functions to accessing and updating file times to sub-second precision, which I might add at some point.

Tony

@p5pRT
Copy link
Author

p5pRT commented Apr 16, 2018

From @tonycoz

0001-perl-133030-make-utime-available-only-if-we-have-bot.patch
From e151966bc2237a3a65d597f520e2675f84ee64c2 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 12 Apr 2018 21:11:12 +1000
Subject: (perl #133030) make utime() available only if we have both fd and
 name setting

---
 dist/Time-HiRes/Makefile.PL | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/dist/Time-HiRes/Makefile.PL b/dist/Time-HiRes/Makefile.PL
index e0f7dd9a2f..daca5b4f60 100644
--- a/dist/Time-HiRes/Makefile.PL
+++ b/dist/Time-HiRes/Makefile.PL
@@ -731,7 +731,7 @@ EOD
 	print "NOT found.\n";
     }
 
-    my $has_hires_utime = ($has_futimens || $has_utimensat);
+    my $has_hires_utime = ($has_futimens && $has_utimensat);
     if ($has_hires_utime) {
 	$DEFINE .= ' -DTIME_HIRES_UTIME';
         print "You seem to have subsecond timestamp setting.\n";
-- 
2.11.0

@p5pRT
Copy link
Author

p5pRT commented Apr 16, 2018

From @tonycoz

0002-perl-133030-probe-for-and-use-utimes-if-utimensat-is.patch
From c9cdee00bbe4a2d06f7322f191488df530e4e0f1 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Thu, 12 Apr 2018 00:19:18 +1000
Subject: (perl #133030) probe for and use utimes() if utimensat() isn't
 available

Older BSDs implement utimes() which can set times down to the microsecond.
---
 dist/Time-HiRes/HiRes.xs    | 18 ++++++++--
 dist/Time-HiRes/Makefile.PL | 88 ++++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 95 insertions(+), 11 deletions(-)

diff --git a/dist/Time-HiRes/HiRes.xs b/dist/Time-HiRes/HiRes.xs
index b9eaa17cde..e7be236fca 100644
--- a/dist/Time-HiRes/HiRes.xs
+++ b/dist/Time-HiRes/HiRes.xs
@@ -1470,7 +1470,7 @@ PROTOTYPE: $$@
                         }
 		}
 		else {
-#ifdef HAS_UTIMENSAT
+#if defined(HAS_UTIMENSAT)
 	          if (UTIMENSAT_AVAILABLE) {
                     STRLEN len;
                     char * name = SvPV(file, len);
@@ -1481,8 +1481,20 @@ PROTOTYPE: $$@
                   } else {
                     croak("utimensat unimplemented in this platform");
                   }
-#else  /* HAS_UTIMENSAT */
-	          croak("utimensat unimplemented in this platform");
+#elif defined(HAS_UTIMES)
+		  struct timeval tv[2];
+                  STRLEN len;
+                  char * name = SvPV(file, len);
+		  tv[0].tv_sec  = utbuf[0].tv_sec;
+		  tv[0].tv_usec = utbuf[0].tv_nsec / 1000;
+		  tv[1].tv_sec  = utbuf[1].tv_sec;
+		  tv[1].tv_usec = utbuf[1].tv_nsec / 1000;
+                  if (IS_SAFE_PATHNAME(name, len, "utime") &&
+                      utimes(name, tv) == 0) {
+                      tot++;
+                  }
+#else  /* HAS_UTIMES */
+	          croak("Neither utimensat nor utimes unimplemented in this platform");
 #endif /* HAS_UTIMENSAT */
 		}
 	} /* while items */
diff --git a/dist/Time-HiRes/Makefile.PL b/dist/Time-HiRes/Makefile.PL
index daca5b4f60..7206869318 100644
--- a/dist/Time-HiRes/Makefile.PL
+++ b/dist/Time-HiRes/Makefile.PL
@@ -403,6 +403,67 @@ int main(int argc, char** argv)
 EOM
 }
 
+sub _stat_code {
+  my ($which) = $DEFINE =~ /-DTIME_HIRES_STAT=(\d+)/
+    or return;
+
+  return @{
+    [
+     [],
+     [ qw/st_atimespec.tv_nsec st_mtimespec.tv_nsec st_ctimespec.tv_nsec/, 1 ],
+     [ qw/st_atimensec st_mtimensec st_ctimensec/, 1 ],
+     [ qw/st_atime_n st_mtime_n st_ctime_n/, 1 ],
+     [ qw/st_atim.tv_nsec st_mtim.tv_nsec st_ctim.tv_nsec/, 1 ],
+     [ qw/st_uatime st_umtime st_uctime/, 1000 ],
+    ]->[$which]};
+}
+
+sub has_utimes {
+    my ($natime, $nmtime, $nctime, $scale) = _stat_code()
+      or return 0;
+    return 1 if
+    try_compile_and_link(<<EOM, run => 1);
+#include <sys/time.h>
+#include <fcntl.h>
+#include <stdlib.h>
+#include <stdio.h>
+int main(int argc, char** argv)
+{
+    int ret1, ret2;
+    struct timeval ts1[2], ts2[2];
+    struct stat st1, st2;
+
+    if (stat("Changes", &st1) < 0) 
+        return 1;
+
+    ts1[0].tv_sec = st1.st_atime-60;
+    ts1[0].tv_usec = st1.$natime ? 1 : 2;
+    ts1[1].tv_sec = st1.st_mtime-60;
+    ts1[1].tv_usec = st1.$nmtime ? 1 : 2;
+
+    ret1 = utimes("Changes", ts1);
+
+    if (stat("Changes", &st2) < 0)
+        return 1;
+
+    /* the file system might not support sub-second timestamps.
+       it might not support accuracy to a second either, so
+       only check that mtime has changed. */
+    if (st1.st_mtime == st2.st_mtime)
+        return 1;
+
+    /* (hopefully) restore the origina m/atimes */
+    ts2[0].tv_sec = st1.st_atime;
+    ts2[0].tv_usec = st1.$natime / $scale;
+    ts2[1].tv_sec = st1.st_mtime;
+    ts2[1].tv_usec = st1.$nmtime / $scale;
+    utimes("Changes", ts2);
+
+    exit(0);
+}
+EOM
+}
+
 sub has_clockid_t{
     return 1 if
     try_compile_and_link(<<EOM);
@@ -731,14 +792,6 @@ EOD
 	print "NOT found.\n";
     }
 
-    my $has_hires_utime = ($has_futimens && $has_utimensat);
-    if ($has_hires_utime) {
-	$DEFINE .= ' -DTIME_HIRES_UTIME';
-        print "You seem to have subsecond timestamp setting.\n";
-    } else {
-        print "You do NOT seem to have subsecond timestamp setting.\n";
-    }
-
     print "Looking for stat() subsecond timestamps...\n";
 
     print "Trying struct stat st_atimespec.tv_nsec...";
@@ -854,6 +907,25 @@ EOM
         DEFINE('TIME_HIRES_STAT', 5);
     }
 
+    my $has_utimes;
+    unless ($has_utimensat) {
+        print "Looking for the fallback utimes()... ";
+        if (has_utimes()) {
+	    ++$has_utimes;
+	    $DEFINE .= ' -DHAS_UTIMES';
+        }
+	print $has_utimes ? "found.\n" : "NOT found\n";
+    }
+
+    # we need both of these to fully replicate perl's utime
+    my $has_hires_utime = ($has_futimens && ($has_utimensat || $has_utimes));
+    if ($has_hires_utime) {
+	$DEFINE .= ' -DTIME_HIRES_UTIME';
+        print "You seem to have subsecond timestamp setting.\n";
+    } else {
+        print "You do NOT seem to have subsecond timestamp setting.\n";
+    }
+
     my $has_hires_stat = ($DEFINE =~ /-DTIME_HIRES_STAT=(\d+)/) ? $1 : 0;
     if ($has_hires_stat) {
         print "You seem to have subsecond timestamp reading.\n";
-- 
2.11.0

@p5pRT
Copy link
Author

p5pRT commented Apr 16, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Apr 25, 2018

From @iabyn

On Sun, Apr 15, 2018 at 05​:55​:05PM -0700, Tony Cook via RT wrote​:

The problem is Time​::HiRes exports its utime() replacement if only one of two functions that depends on is available.

So if you import utime() successfully, the underlying function you need might not be available.

The first of the attached patches makes utime() available only if both are defined. I think this is suitable for 5.28, and prevents the test failure. It however means File​::Copy will no longer preserve sub-second modification times when utime() by name is unavailable.

+1 for applying this for 5.28.

The second patch adds a fallback to utimes() if that's available (typically on BSDs, which is the case covered by this ticket.)

This doesn't try to work around OS X's special behaviour.

The second patch might not be suitable for 5.28.

I think it can wait till 5.29

--
Nothing ventured, nothing lost.

@p5pRT
Copy link
Author

p5pRT commented Apr 26, 2018

From @tonycoz

On Wed, 25 Apr 2018 03​:42​:35 -0700, davem wrote​:

On Sun, Apr 15, 2018 at 05​:55​:05PM -0700, Tony Cook via RT wrote​:

The problem is Time​::HiRes exports its utime() replacement if only
one of two functions that depends on is available.

So if you import utime() successfully, the underlying function you
need might not be available.

The first of the attached patches makes utime() available only if
both are defined. I think this is suitable for 5.28, and prevents
the test failure. It however means File​::Copy will no longer
preserve sub-second modification times when utime() by name is
unavailable.

+1 for applying this for 5.28.

Done as 5dbe8f0.

Tony

@p5pRT
Copy link
Author

p5pRT commented Apr 27, 2018

From @xsawyerx

On 04/26/2018 08​:04 AM, Tony Cook via RT wrote​:

On Wed, 25 Apr 2018 03​:42​:35 -0700, davem wrote​:

On Sun, Apr 15, 2018 at 05​:55​:05PM -0700, Tony Cook via RT wrote​:

The problem is Time​::HiRes exports its utime() replacement if only
one of two functions that depends on is available.

So if you import utime() successfully, the underlying function you
need might not be available.

The first of the attached patches makes utime() available only if
both are defined. I think this is suitable for 5.28, and prevents
the test failure. It however means File​::Copy will no longer
preserve sub-second modification times when utime() by name is
unavailable.
+1 for applying this for 5.28.
Done as 5dbe8f0.

+1

@p5pRT
Copy link
Author

p5pRT commented Jan 1, 2019

From @tonycoz

On Sun, 15 Apr 2018 17​:55​:04 -0700, tonyc wrote​:

This doesn't try to work around OS X's special behaviour.

I tried adding sub-second times for OS X 10.13, but apparently it doesn't support it.

For a file stored on a Linux system, accessible via CIFS (Samba)​:

Directly on Linux​:

$ perl -MTime​::HiRes=stat -le 'print +(stat shift)[9]' ~/test.txt
1546381854.41244

On cygwin​:

$ perl -MTime​::HiRes=stat -le 'print +(stat shift)[9]' /cygdrive/n/test.txt
1546381854.41244

On Darwin​:

pallas​:t tony$ ./perl -I../lib -MTime​::Hires=stat -le 'print +(stat shift)[9]' /Volumes/tony/test.txt
1546381854

The local filesystem (HFS+) doesn't support sub-second timestamps.

Unrelated to this particular bug - there's also Win32 functions to
accessing and updating file times to sub-second precision, which I
might add at some point.

This still something to do.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jan 2, 2019

From @craigberry

On Tue, Jan 1, 2019 at 4​:37 PM Tony Cook via RT
<perlbug-followup@​perl.org> wrote​:

On Sun, 15 Apr 2018 17​:55​:04 -0700, tonyc wrote​:

This doesn't try to work around OS X's special behaviour.

I tried adding sub-second times for OS X 10.13, but apparently it doesn't support it.

I did a quick​:

$ cc -E /usr/include/sys/stat.h > stat_expanded.h

on 10.13.6 and the time components of the stat struct are of type
struct timespec rather than time_t, so I'm not sure why exactly
sub-second times would be missing.

On Darwin​:

pallas​:t tony$ ./perl -I../lib -MTime​::Hires=stat -le 'print +(stat shift)[9]' /Volumes/tony/test.txt
1546381854

The local filesystem (HFS+) doesn't support sub-second timestamps.

The native filesystem for the last couple of releases is APFS, not
HFS+. It seems unlikely a filesystem designed post-ZFS and post-btrfs
with inspiration from both of them would be missing sub-second times.
Building Time​::HiRes on my system says​:

You seem to have subsecond timestamp setting.
Looking for stat() subsecond timestamps...
Trying struct stat st_atimespec.tv_nsec...found.
Trying struct stat st_atimensec...NOT found.
Trying struct stat st_atime_n...NOT found.
Trying struct stat st_atim.tv_nsec...NOT found.
Trying struct stat st_uatime...NOT found.
You seem to have subsecond timestamp reading.
(Your struct stat has them, but the filesystems must help.)

But I get the same thing you do even on an APFS volume. So there's
something else going on. I haven't had time to dig into what.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants