Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Last mod time from stat() can be wrong on Windows NT/2000/XP #6080

Closed
p5pRT opened this issue Nov 19, 2002 · 13 comments
Closed

Last mod time from stat() can be wrong on Windows NT/2000/XP #6080

p5pRT opened this issue Nov 19, 2002 · 13 comments

Comments

@p5pRT
Copy link

p5pRT commented Nov 19, 2002

Migrated from rt.perl.org#18513 (status was 'stalled')

Searchable as RT18513$

@p5pRT
Copy link
Author

p5pRT commented Nov 19, 2002

From @steve-m-hay

This is a bug report for perl from Steve.Hay@​uk.radan.com,
generated with the help of perlbug 1.34 running under perl v5.8.0.

When running on Windows NT/2000/XP Perl's built-in stat() function can
return the wrong last modification time of a file.

The value returned should be expressed as seconds since the Epoch, and
should therefore be independent of the system's time zone and daylight
savings time (DST) settings.

However, for a file created during a period when DST is in effect the
value actually returned when the system is no longer in a DST period
depends on whether or not the system is configured to automatically
make adjustments for DST.

Example​:

  Running on Windows XP Professional, Service Pack 1
  "Regional and Language Options" set to English (United Kingdom)

  Open the "Date and Time Properties" applet from Control Panel

  Go to the "Time Zone" tab

  Select "(GMT) Greenwich Mean Time​: Dublin, Edinburgh, Lisbon, London"
  and select the option to "Automatically adjust clock for daylight
  saving changes"

  Go to the "Date & Time" tab

  Enter the values "1 June 2002" and "14​:00​:00" (when DST is in effect)

  Click "OK" to save these settings and dismiss the applet

  Open a Comment Prompt and type the following commands​:

  cd \Temp
  echo Hello > test.txt

  Now bring the "Date and Time Properties" applet back up, change the
  values to "1 November 2002" and "14​:00​:00" (when DST is no longer in
  effect) and click "OK" to save it

  Type the following command in the Command Prompt​:

  perl -e "print +(stat 'test.txt')[9]"

  This outputs a number like​:

  1022932805

  depending on how soon you created the file 'test.txt' with the "echo"
  command above after having set the time

  Finally, bring the "Date and Time Properties" applet back up,
  deselect the option to "Automatically adjust clock for daylight saving
  changes" and click "OK" to save it

  Repeat the above "perl" command in the Command Prompt, and a different
  number is output, in this case​:

  1022936405

The two values are different by 3600 seconds, i.e. the amount that a DST
adjustment was made for. The last modification time of the file thus
cannot be determined since there is no way (?) to programmatically
determine whether or not the "Automatically adjust ..." setting is
selected or not.

I believe that this output is wrong. The value returned by stat() should
be the same regardless of whether the "Automatically adjust ..." setting
is selected or not.

One final note​: stat() behaves as I would expect, i.e. always returning
the same value, on Windows 95/98/ME. It is only the "NT" type systems
(NT4/2000/XP) that have the problem.


Flags​:
  category=core
  severity=medium


Site configuration information for perl v5.8.0​:

Configured by radan at Thu Sep 5 09​:10​:23 2002.

Summary of my perl5 (revision 5 version 8 subversion 0) configuration​:
  Platform​:
  osname=MSWin32, osvers=4.0, archname=MSWin32-x86-perlio
  uname=''
  config_args='undef'
  hint=recommended, useposix=true, d_sigaction=undef
  usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
  useperlio=define d_sfio=undef uselargefiles=undef usesocks=undef
  use64bitint=undef use64bitall=undef uselongdouble=undef
  usemymalloc=y, bincompat5005=undef
  Compiler​:
  cc='cl', ccflags ='-nologo -Gf -W3 -MD -DNDEBUG -O1 -DWIN32
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_PERLIO
-DPERL_MSVCRT_READFIX',
  optimize='-MD -DNDEBUG -O1',
  cppflags='-DWIN32'
  ccversion='', gccversion='', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=4
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='link', ldflags ='-nologo -nodefaultlib -release
-libpath​:"C​:\perl5\lib\CORE" -machine​:x86'
  libpth=D​:\PROGRA1\MICROS1\VC98\lib "D​:\openssl\lib"
  libs=libeay32.lib oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib
version.lib odbc32.lib odbccp32.lib msvcrt.lib
  perllibs=libeay32.lib oldnames.lib kernel32.lib user32.lib
gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib
version.lib odbc32.lib odbccp32.lib msvcrt.lib
  libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
  gnulibc_version='undef'
  Dynamic Linking​:
  dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
  cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release
-libpath​:"C​:\perl5\lib\CORE" -machine​:x86'

Locally applied patches​:
 


@​INC for perl v5.8.0​:
  C​:/perl5/lib
  C​:/perl5/site/lib
  .


Environment for perl v5.8.0​:
  HOME (unset)
  LANG (unset)
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
 
PATH=C​:\WINDOWS\system32;C​:\WINDOWS;C​:\WINDOWS\System32\Wbem;C​:\perl5\bin
  PERL5LIB (unset)
  PERL_BADLANG (unset)
  SHELL (unset)

@p5pRT
Copy link
Author

p5pRT commented Dec 12, 2002

From @steve-m-hay

My original bugreport specified

  category=core
  severity=medium

but the RT system seems to have logged it as

  category=unknown
  severity=low

instead. Is this a bug in perlbug?

@p5pRT
Copy link
Author

p5pRT commented Apr 1, 2003

From @eserte

Maybe some Windows expert may take a look at

  http​://www.codeproject.com/datetime/dstbugs.asp

and implement a fix based on the suggestions on this side. Otherwise
there should at least be a comment in perlport.pod.

Regards,
  Slaven

@p5pRT
Copy link
Author

p5pRT commented May 19, 2003

From @steve-m-hay

Regarding the above bug report​:

Is this something that is likely to be fixed in the Perl core any time soon?

Slaven Rezic found a website with an excellent description of the
problem, and a general solution to it, at the URL​:
http​://www.codeproject.com/datetime/dstbugs.asp

I would love to see this done in the Perl core, but I'd be way out of my
depth attempting it myself.

I could, however, produce a Win32​::* module based on it quite easily.
If there is no prospect of a core fix then I'd like to go ahead with
the module.

Any comments?

Steve

@p5pRT
Copy link
Author

p5pRT commented May 19, 2003

From @nwc10

On Mon, May 19, 2003 at 04​:44​:20PM +0100, Steve Hay wrote​:

Regarding the above bug report​:

Is this something that is likely to be fixed in the Perl core any time soon?

Not obviously. People who understand the core usually already have too
much on their plate, so unless someone is being bitten by this problem,
or considers it sufficiently serious to drop an existing pet project in
favour of it, it's unlikely.

Slaven Rezic found a website with an excellent description of the
problem, and a general solution to it, at the URL​:
http​://www.codeproject.com/datetime/dstbugs.asp

I would love to see this done in the Perl core, but I'd be way out of my
depth attempting it myself.

I could, however, produce a Win32​::* module based on it quite easily.
If there is no prospect of a core fix then I'd like to go ahead with
the module.

Any comments?

I don't use Win32, so I have no direct concern about how this is fix.

I think that the best thing to do is to write a module to implement the
fix (no idea about name), even if the module is superceded with a
core patch. Firstly because it's easier to experiment with behaviour
in perl rather than in C (even for people who are familiar with the
core) and secondly because the existing Perl installations that suffer
from this MS design bug would benefit from a module immediately.

From an open-source cat herding perspective, if you're able to contribute
a working perl module to implement a fix, then it's more likely that someone
will help you out by converting your perl algorithm into C in the core.
Offering patches has this strange effect of galvanising other people into
action.

Nicholas Clark

@p5pRT
Copy link
Author

p5pRT commented May 20, 2003

From @steve-m-hay

Nicholas Clark wrote​:

On Mon, May 19, 2003 at 04​:44​:20PM +0100, Steve Hay wrote​:

Regarding the above bug report​:

Is this something that is likely to be fixed in the Perl core any time soon?

Not obviously.
[snip]

Slaven Rezic found a website with an excellent description of the
problem, and a general solution to it, at the URL​:
http​://www.codeproject.com/datetime/dstbugs.asp
[snip]
I could, however, produce a Win32​::* module based on it quite easily.
If there is no prospect of a core fix then I'd like to go ahead with
the module.

Any comments?

I don't use Win32, so I have no direct concern about how this is fix.

I think that the best thing to do is to write a module to implement the
fix (no idea about name), even if the module is superceded with a
core patch.
[snip]

OK, I'll start putting together a Perl module.

Does anyone have any thoughts on a suitable name for it? How about
Win32​::UTCFileTime?

I was thinking of having the module provide overrides for the built-in
stat() and lstat() functions. By default, it would export the overrides
into CLORE​::GLOBAL. Explicitly asking for one/both functions would
disable the CORE​::GLOBAL override and only export the requested
function(s) to the caller​:

  # Override stat() and lstat() globally​:
  use Win32​::UTCFileTime;

  # Override stat() locally; don't override lstat().
  use Win32​::UTCFileTime qw(stat);

Why would anyone want to override stat() but not lstat()? Maybe​:

  # Override stat() and lstat() locally​:
  use Win32​::UTCFileTime qw(​:local);

makes more sense.

Any thoughts on these ideas, or any other ideas?

Steve

@p5pRT
Copy link
Author

p5pRT commented May 20, 2003

From @rgs

Steve Hay <steve.hay@​uk.radan.com> wrote​:

I was thinking of having the module provide overrides for the built-in
stat() and lstat() functions. By default, it would export the overrides
into CLORE​::GLOBAL.

As a general rule, I don't think that's a sensible default.
If people really mean to do require a global override, I'd
prefer an explicit '​:globally' tag (à la File​::Glob).

@p5pRT
Copy link
Author

p5pRT commented Jun 5, 2003

From @steve-m-hay

The uploaded file

  Win32-UTCFileTime-1.00.tar.gz

has entered CPAN as

file​: $CPAN/authors/id/S/SH/SHAY/Win32-UTCFileTime-1.00.tar.gz
size​: 28762 bytes
  md5​: 5ad0ba315e7fb2babf513b856b71834a

This module addresses this bug report.

It provides replacement stat(), lstat() and utime() functions that
correctly handle UTC file times (last access time, last modification
time and creation time). Perl's built-in functions of the same name
inherit some curious behaviour from the Microsoft C library which causes
the times to vary by one hour across DST season changes on NTFS filesystems.

Hopefully the workaround in this module will be incorporated into the
Perl core one day, but in the mean time this module should plug the gap.

Steve

@p5pRT
Copy link
Author

p5pRT commented Nov 16, 2007

From @smpeters

On Thu Jun 05 02​:10​:02 2003, shay wrote​:

The uploaded file

Win32-UTCFileTime-1.00.tar.gz

has entered CPAN as

file​: $CPAN/authors/id/S/SH/SHAY/Win32-UTCFileTime-1.00.tar.gz
size​: 28762 bytes
md5​: 5ad0ba315e7fb2babf513b856b71834a

This module addresses this bug report.

It provides replacement stat(), lstat() and utime() functions that
correctly handle UTC file times (last access time, last modification
time and creation time). Perl's built-in functions of the same name
inherit some curious behaviour from the Microsoft C library which causes
the times to vary by one hour across DST season changes on NTFS
filesystems.

Hopefully the workaround in this module will be incorporated into the
Perl core one day, but in the mean time this module should plug the gap.

Steve

Steve,

Do you know if there has been a fix in the MS core libraries since this
bug report?

Steve Peters

@p5pRT
Copy link
Author

p5pRT commented Nov 16, 2007

From @steve-m-hay

Steve Peters via RT wrote​:

On Thu Jun 05 02​:10​:02 2003, shay wrote​:

The uploaded file

Win32-UTCFileTime-1.00.tar.gz

has entered CPAN as

file​: $CPAN/authors/id/S/SH/SHAY/Win32-UTCFileTime-1.00.tar.gz
size​: 28762 bytes md5​: 5ad0ba315e7fb2babf513b856b71834a

This module addresses this bug report.

It provides replacement stat(), lstat() and utime() functions that
correctly handle UTC file times (last access time, last modification
time and creation time). Perl's built-in functions of the same name
inherit some curious behaviour from the Microsoft C library which
causes the times to vary by one hour across DST season changes on
NTFS filesystems.

Hopefully the workaround in this module will be incorporated into the
Perl core one day, but in the mean time this module should plug the
gap.

Steve

Steve,

Do you know if there has been a fix in the MS core libraries since
this bug report?

Steve Peters

I have a perl built with VC++ 2005 (VC8) and it still exhibits the same
behaviour reported in this bug when run on Windows Vista.

That's what I'd expect, really. When I created the Win32-UTCFileTime
module to workaround this bug, I found the suggestion that this is all
intended behaviour on Microsoft's part, i.e. they don't regard it as a
bug, so it's not surprising that it isn't fixed.

@p5pRT
Copy link
Author

p5pRT commented Nov 16, 2007

From @nwc10

On Fri, Nov 16, 2007 at 02​:52​:25PM -0000, Steve Hay wrote​:

Steve Peters via RT wrote​:

On Thu Jun 05 02​:10​:02 2003, shay wrote​:

It provides replacement stat(), lstat() and utime() functions that
correctly handle UTC file times (last access time, last modification
time and creation time). Perl's built-in functions of the same name
inherit some curious behaviour from the Microsoft C library which
causes the times to vary by one hour across DST season changes on
NTFS filesystems.

That's what I'd expect, really. When I created the Win32-UTCFileTime
module to workaround this bug, I found the suggestion that this is all
intended behaviour on Microsoft's part, i.e. they don't regard it as a
bug, so it's not surprising that it isn't fixed.

Sigh. There are various things I could write about Microsoft for this one,
but most would cause either the police to arrest me for attempting to incite
homicide, or Kermit's lawyers to issue a write for libel.

DST is a whim of politicians. That should be a good enough reason to avoid it.
Look closer, and you see that if you have DST, then for one hour, wallclock
times repeat. Which 1​:30am on that particular Sunday was it?

And then timezones themselves. If I have a laptop, and it goes on a trip
with me to another continent or country or state or county just a Native
American Reservation...

Nicholas Clark

PS Should Perl 6 declare sanity on this from release 6.0.0 ?

@p5pRT
Copy link
Author

p5pRT commented Jul 5, 2016

@dcollinsn - Status changed from 'open' to 'stalled'

@tonycoz tonycoz self-assigned this Oct 11, 2020
genio added a commit to genio/perl5 that referenced this issue Oct 15, 2020
As discussed on the mailing list here:
https://www.nntp.perl.org/group/perl.perl5.porters/2020/10/msg258453.html

This just removes the declaration that we support the very old versions
of Windows that have long since been EOLed.

For reference of problems related to maintaining the EOLed versions:
Perl#4145
Perl#6080
Perl#7410
Perl#8502
Perl#9025
Perl#12431
Perl#14687
tonycoz added a commit that referenced this issue Oct 15, 2020
This fixes at least two problems:

- unlike UCRT, the MSVCRT used for gcc builds has a bug converting
  a FILETIME in an unlike current DST state, returning a time
  offset by an hour.  Fixes GH #6080

- the MSVCRT apparently uses FindFirstFile() to fetch file
  information, but this doesn't follow symlinks(), so stat()
  ends up returning information about the symlink(), not the
  underlying file.  This isn't an issue with the UCRT which
  opens the file as this implementation does.

Currently this code calculates the time_t for st_*time, and the
other way for utime() using a simple multiplication and offset
between time_t and FILETIME values, but this may be incorrect
if leap seconds are enabled.

This code also requires Vista or later.

Some of this is based on code by Tomasz Konojacki (xenu).
tonycoz added a commit that referenced this issue Nov 4, 2020
This fixes at least two problems:

- unlike UCRT, the MSVCRT used for gcc builds has a bug converting
  a FILETIME in an unlike current DST state, returning a time
  offset by an hour.  Fixes GH #6080

- the MSVCRT apparently uses FindFirstFile() to fetch file
  information, but this doesn't follow symlinks(), so stat()
  ends up returning information about the symlink(), not the
  underlying file.  This isn't an issue with the UCRT which
  opens the file as this implementation does.

Currently this code calculates the time_t for st_*time, and the
other way for utime() using a simple multiplication and offset
between time_t and FILETIME values, but this may be incorrect
if leap seconds are enabled.

This code also requires Vista or later.

Some of this is based on code by Tomasz Konojacki (xenu).
tonycoz added a commit that referenced this issue Nov 9, 2020
This fixes at least two problems:

- unlike UCRT, the MSVCRT used for gcc builds has a bug converting
  a FILETIME in an unlike current DST state, returning a time
  offset by an hour.  Fixes GH #6080

- the MSVCRT apparently uses FindFirstFile() to fetch file
  information, but this doesn't follow symlinks(), so stat()
  ends up returning information about the symlink(), not the
  underlying file.  This isn't an issue with the UCRT which
  opens the file as this implementation does.

Currently this code calculates the time_t for st_*time, and the
other way for utime() using a simple multiplication and offset
between time_t and FILETIME values, but this may be incorrect
if leap seconds are enabled.

This code also requires Vista or later.

Some of this is based on code by Tomasz Konojacki (xenu).
tonycoz added a commit that referenced this issue Nov 11, 2020
This fixes at least two problems:

- unlike UCRT, the MSVCRT used for gcc builds has a bug converting
  a FILETIME in an unlike current DST state, returning a time
  offset by an hour.  Fixes GH #6080

- the MSVCRT apparently uses FindFirstFile() to fetch file
  information, but this doesn't follow symlinks(), so stat()
  ends up returning information about the symlink(), not the
  underlying file.  This isn't an issue with the UCRT which
  opens the file as this implementation does.

Currently this code calculates the time_t for st_*time, and the
other way for utime() using a simple multiplication and offset
between time_t and FILETIME values, but this may be incorrect
if leap seconds are enabled.

This code also requires Vista or later.

Some of this is based on code by Tomasz Konojacki (xenu).
tonycoz added a commit that referenced this issue Nov 23, 2020
This fixes at least two problems:

- unlike UCRT, the MSVCRT used for gcc builds has a bug converting
  a FILETIME in an unlike current DST state, returning a time
  offset by an hour.  Fixes GH #6080

- the MSVCRT apparently uses FindFirstFile() to fetch file
  information, but this doesn't follow symlinks(), so stat()
  ends up returning information about the symlink(), not the
  underlying file.  This isn't an issue with the UCRT which
  opens the file as this implementation does.

Currently this code calculates the time_t for st_*time, and the
other way for utime() using a simple multiplication and offset
between time_t and FILETIME values, but this may be incorrect
if leap seconds are enabled.

This code also requires Vista or later.

Some of this is based on code by Tomasz Konojacki (xenu).
tonycoz added a commit that referenced this issue Dec 1, 2020
This fixes at least two problems:

- unlike UCRT, the MSVCRT used for gcc builds has a bug converting
  a FILETIME in an unlike current DST state, returning a time
  offset by an hour.  Fixes GH #6080

- the MSVCRT apparently uses FindFirstFile() to fetch file
  information, but this doesn't follow symlinks(), so stat()
  ends up returning information about the symlink(), not the
  underlying file.  This isn't an issue with the UCRT which
  opens the file as this implementation does.

Currently this code calculates the time_t for st_*time, and the
other way for utime() using a simple multiplication and offset
between time_t and FILETIME values, but this may be incorrect
if leap seconds are enabled.

This code also requires Vista or later.

Some of this is based on code by Tomasz Konojacki (xenu).
@tonycoz
Copy link
Contributor

tonycoz commented Dec 1, 2020

This is (finally) fixed by b277e76.

@tonycoz tonycoz closed this as completed Dec 1, 2020
steve-m-hay added a commit to steve-m-hay/Win32-UTCFileTime that referenced this issue Dec 4, 2020
(This is thanks to perl core commit b277e767f3, which fixes CPAN RT#18513:
See Perl/perl5#6080.)

Also note that this module wasn't quite made redundant by VC14.0 because
utime() still didn't work correctly on directories.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants