Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

localtime corruption #7907

Closed
p5pRT opened this issue May 12, 2005 · 35 comments
Closed

localtime corruption #7907

p5pRT opened this issue May 12, 2005 · 35 comments

Comments

@p5pRT
Copy link

p5pRT commented May 12, 2005

Migrated from rt.perl.org#35420 (status was 'resolved')

Searchable as RT35420$

@p5pRT
Copy link
Author

p5pRT commented May 12, 2005

From dpfeiffer@amadeus.net

This is a bug report for perl from occitan@​esperanto.org,
generated with the help of perlbug 1.35 running under perl v5.8.6.


localtime returns rubbish outside of -~0/2..~0/2. The first quarter of
possible arguments all return 1901, while the last quarter wraps around
after
2038, back to 1970. This should die or, better, be extended.



Flags​:
  category=core
  severity=low


This perlbug was built using Perl v5.8.6 - Sat Mar 19 17​:33​:54 UTC 2005
It is being executed now by Perl v5.8.6 - Sat Mar 19 17​:29​:44 UTC 2005.

Site configuration information for perl v5.8.6​:

Configured by abuild at Sat Mar 19 17​:29​:44 UTC 2005.

Summary of my perl5 (revision 5 version 8 subversion 6) configuration​:
  Platform​:
  osname=linux, osvers=2.6.9, archname=i586-linux-thread-multi
  uname='linux salieri 2.6.9 #1 smp fri jan 14 15​:41​:33 utc 2005 i686
athlon i386 gnulinux '
  config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr
-Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm
-Duseshrplib=true -Doptimize=-O2 -march=i586 -mcpu=i686 -fmessage-length=0
-Wall -g -Wall -pipe'
  hint=recommended, useposix=true, d_sigaction=define
  usethreads=define use5005threads=undef useithreads=define
usemultiplicity=define
  useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
  use64bitint=undef use64bitall=undef uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
-DDEBUGGING -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
  optimize='-O2 -march=i586 -mcpu=i686 -fmessage-length=0 -Wall -g -Wall
-pipe',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING
-fno-strict-aliasing -pipe'
  ccversion='', gccversion='3.3.5 20050117 (prerelease) (SUSE Linux)',
gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
  alignbytes=4, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =''
  libpth=/lib /usr/lib /usr/local/lib
  libs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
  libc=, so=so, useshrplib=true, libperl=libperl.so
  gnulibc_version='2.3.4'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E
-Wl,-rpath,/usr/lib/perl5/5.8.6/i586-linux-thread-multi/CORE'
  cccdlflags='-fPIC', lddlflags='-shared'

Locally applied patches​:


@​INC for perl v5.8.6​:
  /usr/lib/perl5/5.8.6/i586-linux-thread-multi
  /usr/lib/perl5/5.8.6
  /usr/lib/perl5/site_perl/5.8.6/i586-linux-thread-multi
  /usr/lib/perl5/site_perl/5.8.6
  /usr/lib/perl5/site_perl
  /usr/lib/perl5/vendor_perl/5.8.6/i586-linux-thread-multi
  /usr/lib/perl5/vendor_perl/5.8.6
  /usr/lib/perl5/vendor_perl
  .


Environment for perl v5.8.6​:
  HOME=/home/pfeiffer
  LANG=de_DE.UTF-8
  LANGUAGE (unset)
  LC_COLLATE=POSIX
  LD_LIBRARY_PATH=​:/usr/local/lib
  LOGDIR (unset)

PATH=/home/pfeiffer/bin​:/usr/local/bin​:/bin​:/usr/bin​::/sbin​:/usr/sbin​:/usr/games​:/opt/kde3/bin​:/opt/gnome/bin​:/usr/bin/X11​:/opt/OpenOffice.org1.1.3/program​:/opt/MKS/IntegrityClient/bin​:/usr/lib/java/jre/bin
  PERL_BADLANG (unset)
  SHELL=/bin/ksh

--
lerne/learn Esperanto online​: http​://lernu.net/

@p5pRT
Copy link
Author

p5pRT commented May 12, 2005

From @rgs

Daniel Pfeiffer (via RT) wrote​:

localtime returns rubbish outside of -~0/2..~0/2. The first quarter of
possible arguments all return 1901, while the last quarter wraps around
after
2038, back to 1970. This should die or, better, be extended.

Perl can't do better than the OS here.

@p5pRT
Copy link
Author

p5pRT commented May 12, 2005

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented May 12, 2005

@rgs - Status changed from 'open' to 'rejected'

@p5pRT
Copy link
Author

p5pRT commented May 12, 2005

From perl5-porters@ton.iguana.be

In article <20050512140047.3d09ea_c@​grubert.mandrakeso_t.com>,
  Rafael Garcia-Suarez <rgarciasuarez@​mandriva.com> writes​:

Daniel Pfeiffer (via RT) wrote​:

localtime returns rubbish outside of -~0/2..~0/2. The first quarter of
possible arguments all return 1901, while the last quarter wraps around
after
2038, back to 1970. This should die or, better, be extended.

Perl can't do better than the OS here.

Actually, I suppose we could. We don't *have* to use the libc
localtime/gmtime/strftime and could implement our own, though it might
need some per OS code, mainly to find the timezone database and to
decide if the clock is TAI or nominal (without leap) seconds.

@p5pRT
Copy link
Author

p5pRT commented May 12, 2005

From @JohnPeacock

Ton Hospel wrote​:

Actually, I suppose we could. We don't *have* to use the libc
localtime/gmtime/strftime and could implement our own, though it might
need some per OS code, mainly to find the timezone database and to
decide if the clock is TAI or nominal (without leap) seconds.

libtai is public domain​:

  http​://cr.yp.to/libtai.html

the problem being that it requires a 64bit int which Win32 makes
somewhat annoying to use (depending on which compiler you use).

John

--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748

@p5pRT
Copy link
Author

p5pRT commented May 13, 2005

From dpfeiffer@amadeus.net

Hallo Rafael,

Inconsistencies of this kind are a terrible trap! For one thing the doc
states that this is not locale-dependent but a builtin. That sounds like
a lie from your response. For another, configure should detect the valid
range of arguments, and do something (at least die) about the others.

mit freundlichen Grü�en / best regards
Daniel

--
lerne/learn Esperanto online​: http​://lernu.net/

From​: "Rafael Garcia-Suarez via RT" <perlbug-followup@​perl.org> on
12.05.2005 12​:00 GMT
Please respond to perlbug-followup@​perl.org

To​:
dpfeiffer@​amadeus.net

cc​:

Subject​:
Re​: [perl #35420] localtime corruption

Daniel Pfeiffer (via RT) wrote​:

localtime returns rubbish outside of -~0/2..~0/2. The first quarter of
possible arguments all return 1901, while the last quarter wraps around
after
2038, back to 1970. This should die or, better, be extended.

Perl can't do better than the OS here.

@p5pRT
Copy link
Author

p5pRT commented May 13, 2005

From @JohnPeacock

Daniel Pfeiffer wrote​:

Inconsistencies of this kind are a terrible trap! For one thing the doc
states that this is not locale-dependent but a builtin. That sounds like
a lie from your response.

It's not a lie; virtually all modern operating systems use a 32-bit time value
internally, see this site (among others)​:

  http​://www.2038bug.com/

for why this is a problem. Perl could work around it by using a different time
value (e.g. 64bit TAI), but it would require custom code.

The range of the time_t values _is_ documented several places, like here in
Time​::Local​:

  Please note, however, that the range of dates that can be actually be
  handled depends on the size of an integer (time_t) on a given platform.
  Currently, this is 32 bits for most systems, yielding an approximate
  range from Dec 1901 to Jan 2038.

HTH

John

--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747

@p5pRT
Copy link
Author

p5pRT commented May 13, 2005

From @schwern

On Fri, May 13, 2005 at 07​:38​:12AM -0400, John Peacock wrote​:

It's not a lie; virtually all modern operating systems use a 32-bit time
value internally, see this site (among others)​:

http&#8203;://www\.2038bug\.com/

for why this is a problem. Perl could work around it by using a different
time value (e.g. 64bit TAI), but it would require custom code.

A) This assumes new Perl programmers already have knowledge of C or similar
language as well as the details of their operating system. This is no
longer true.

B) There's nothing that says because C made this mistake that Perl should,
too. In fact, everything would suggest Perl would correct this arbitrary
limitation in the same way as we've corrected other common C limitations
such as fixed size strings, arrays and numbers which transparently morph
from integers to floats to doubles...

The range of the time_t values _is_ documented several places, like here in
Time​::Local​:

Time​::Local is the *only* place its documented (there's an offhand mention in
perlfaq4). The localtime docs don't even mention that its based on time_t.
A module is not the appropriate place to document the limitations of
localtime(). perlfunc, perlport and perltrap are.

Daniel is right, it is a trap and it is poorly documented. What should be
done, from minimum work to most solution...

1) Document that there is a system dependent limitation perlfunc and perlport
and to only trust localtime(), and any date handling code derived from
localtime(), to work from 1970 to 2038. It may work for more but that
cannot be trusted.

2) Add a probe to Configure to check what the date range of localtime() is
(ie. feed it -2**31-1, -2**31, -2**31+1, -1, 0, 1, 2**31-1 and 2**31... and
so on for 64 bit time_t) and warn when the user goes outside that range.

3) Have Perl use its own 64 bit time library.

--
Michael G Schwern schwern@​pobox.com http​://www.pobox.com/~schwern
Ahh email, my old friend. Do you know that revenge is a dish that is best
served cold? And it is very cold on the Internet!

@p5pRT
Copy link
Author

p5pRT commented May 13, 2005

From @JohnPeacock

Michael G Schwern wrote​:

for why this is a problem. Perl could work around it by using a different
time value (e.g. 64bit TAI), but it would require custom code.

A) This assumes new Perl programmers already have knowledge of C or similar
language as well as the details of their operating system. This is no
longer true.

Anyone feeding random numbers into localtime shouldn't be considered a
"new Perl programmer" anymore.

B) There's nothing that says because C made this mistake that Perl should,
too. In fact, everything would suggest Perl would correct this arbitrary
limitation in the same way as we've corrected other common C limitations
such as fixed size strings, arrays and numbers which transparently morph
from integers to floats to doubles...

Yeah, in Perl 6. That's down the hall. ;-)

1) Document that there is a system dependent limitation perlfunc and perlport
and to only trust localtime(), and any date handling code derived from
localtime(), to work from 1970 to 2038. It may work for more but that
cannot be trusted.

This gets my vote as the only realistic solution for Perl5. Although we
/could/ produce a portable 64bit time library, it isn't that simple to
handle the conversion from TAI (or whatever) to human dates (see the
DateTime project for more details than you could possibly want to know[1]).

John

1. I like this page​:

  http​://developer.apple.com/technotes/tn/tn1049.html

which correctly points out that the Mac never had a Y2K problem without
mentioning the fact that the Mac epoch (12​:00​:00 a.m., January 1, 1904)
is a floating epoch (i.e. not tied to GMT), so it is completely
worthless for performing date calculations that cross DST, for example.

--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748

@p5pRT
Copy link
Author

p5pRT commented May 13, 2005

From @schwern

On Fri, May 13, 2005 at 04​:40​:56PM -0400, John Peacock wrote​:

A) This assumes new Perl programmers already have knowledge of C or similar
language as well as the details of their operating system. This is no
longer true.

Anyone feeding random numbers into localtime shouldn't be considered a
"new Perl programmer" anymore.

Let's be civil and sane here and not a bunch of old Unix farts. This is
not about stupid users, its about not expecting new Perl users to know
details of Unix esoteria that Perl is supposed to (and usually does) hide.

Here's a simple scenario where a user is surprised by localtime's behavior
that doesn't involve them bashing at the keyboard with a club.

You have dates stored as unix time with math done in unix time. One of
those dates is Jan 2, 1970. You subtract 3 days (60 * 60 * 24 * 3) from
that. Suddenly localtime() is returning weird dates with no indication as
to why. To add additional confusion, on some systems localtime(-1) will work.
On some it will not.

B) There's nothing that says because C made this mistake that Perl should,
too. In fact, everything would suggest Perl would correct this arbitrary
limitation in the same way as we've corrected other common C limitations
such as fixed size strings, arrays and numbers which transparently morph
from integers to floats to doubles...

Yeah, in Perl 6. That's down the hall. ;-)

Parrot is likely to have to do the same thing Perl 5 would​: have its own
internal datetime handling library. Perhaps we should take a peek at what
the Parrot folks are doing.

1) Document that there is a system dependent limitation perlfunc and
perlport
and to only trust localtime(), and any date handling code derived from
localtime(), to work from 1970 to 2038. It may work for more but that
cannot be trusted.

This gets my vote as the only realistic solution for Perl5. Although we
/could/ produce a portable 64bit time library, it isn't that simple to
handle the conversion from TAI (or whatever) to human dates (see the
DateTime project for more details than you could possibly want to know[1]).

Agreeing that its a good idea is orthoganal to its implementation difficulty.
The nice thing about Open Source is some Random Genius can come along and
surprise us with an implementation of something we consider to be very
difficult. From what I can scrape out of the above, you'd be ok if someone
dropped in a patch for native Perl date handling, however it happens to
work.

What about #2​: the warning that you've overflowed the native localtime()?

1. I like this page​:

http&#8203;://developer\.apple\.com/technotes/tn/tn1049\.html

which correctly points out that the Mac never had a Y2K problem without
mentioning the fact that the Mac epoch (12​:00​:00 a.m., January 1, 1904)
is a floating epoch (i.e. not tied to GMT), so it is completely
worthless for performing date calculations that cross DST, for example.

I don't think I understand how this is relevant or where Y2K comes in. I
probably just don't understand the TAI issue.

--
Michael G Schwern schwern@​pobox.com http​://www.pobox.com/~schwern
'All anyone gets in a mirror is themselves,' she said. 'But what you
gets in a good gumbo is everything.'
  -- "Witches Abroad" by Terry Prachett

@p5pRT
Copy link
Author

p5pRT commented May 14, 2005

From @JohnPeacock

Michael G Schwern wrote​:

Let's be civil and sane here and not a bunch of old Unix farts. This is
not about stupid users, its about not expecting new Perl users to know
details of Unix esoteria that Perl is supposed to (and usually does) hide.

I'm not trying to be. Daniel reported that he was checking both very small and
very large numbers. What you describe below is much more realistic of "real
world Perl" than trying values likely to cross boundaries. I think Raphael's
comment that this was as good as the operating system can manage is fully
justifiable. Date math is very hard to do correctly (see below).

It's also not specific to Unix; even though Windows stores it's time in a 64bit
integer (as 100's of nanoseconds for some reason), the native C library
localtime() implementation basically suffers from the same 32bit rollover​:

  http​://www.codeproject.com/datetime/time64.asp

Here's a simple scenario where a user is surprised by localtime's behavior
that doesn't involve them bashing at the keyboard with a club.

You have dates stored as unix time with math done in unix time. One of
those dates is Jan 2, 1970. You subtract 3 days (60 * 60 * 24 * 3) from
that. Suddenly localtime() is returning weird dates with no indication as
to why. To add additional confusion, on some systems localtime(-1) will work.
On some it will not.

And how is Perl going to deal with that? Do we go with the Mac's unsigned 32bit
time starting at Jan 1, 1904? Do we use TAI64, which covers a few hundred
billions of years of real time with 1 second precision​:

  http​://cr.yp.to/proto/utctai.html

Or do we standardize on UTC, which is civil time based on TAI plus leap seconds.
And do we include the Olsen database so that proper handling of timezones over
both geographic space and historical periods (for those benighted O/S's which
don't install it)? Do we *correctly* handle leap second calculations (which
means occasionally updating the database)? Most especially, do we break ranks
with POSIX (which apparently demands that 2100 be a leap year, even though it
isn't).

Agreeing that its a good idea is orthoganal to its implementation difficulty.
The nice thing about Open Source is some Random Genius can come along and
surprise us with an implementation of something we consider to be very
difficult. From what I can scrape out of the above, you'd be ok if someone
dropped in a patch for native Perl date handling, however it happens to
work.

The problem with a custom date library is that it is actually very hard to
translate number of seconds into a human readable date outside of a very narrow
window (oddly enough corresponding to after the Unix epoch of Jan 1, 1970). And
even then, if you are trying to compare times from different locations, you
can't do it without either being very sloppy or using the Olsen database.

I won't try and replicate the information in the very good resources listed at​:

  http​://datetime.perl.org/resources.html

but the basic matter is that until the TAI/UTC standardization in 1970, there
was a very wide variety of ways that different countries (and even, within the
US, different states) dealt with timezones and time in general.

The reason I mentioned the Mac epoch is that there were still a number of
countries who didn't switch to the Gregorian Calendar until the early 20th
century (after 1904). Knowing the number of seconds from a given baseline
doesn't help you render that into month/day/year unless you know *where* you are
at the time (and frequently *when* you are as well). This is what the Olsen
database attempts to handle and what the DateTime project has been struggling
with (see the link above).

What about #2​: the warning that you've overflowed the native localtime()?

If you want to write Configure probes, go right ahead. You should probably
perform something more like a binary search, though, rather than just leaping to
the ends of the range, since some libraries might work well close to 1970 and
not work well outside it. I think a warning is the best we can do given the
enormity of the problem.

John

--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747

@p5pRT
Copy link
Author

p5pRT commented May 14, 2005

From @tamias

On Fri, May 13, 2005 at 12​:02​:22PM +0200, Daniel Pfeiffer wrote​:

Hallo Rafael,

Inconsistencies of this kind are a terrible trap! For one thing the doc
states that this is not locale-dependent but a builtin. That sounds like
a lie from your response.

I think you've misunderstood that part of the documentation.

  In scalar context, "localtime()" returns the ctime(3) value​:

  $now_string = localtime; # e.g., "Thu Oct 13 04​:54​:34 1994"

  This scalar value is not locale dependent, see perllocale, but
  instead a Perl builtin.

Regardless of your locale, the *formatting* of the scalar value will always
be the same.

The conversion from an integer time to a time struct, however, is dependent
on the operating system.

Ronald

@p5pRT
Copy link
Author

p5pRT commented May 15, 2005

From @demerphq

On 5/13/05, Ronald J Kimball <rjk@​tamias.net> wrote​:

On Fri, May 13, 2005 at 12​:02​:22PM +0200, Daniel Pfeiffer wrote​:

Hallo Rafael,

Inconsistencies of this kind are a terrible trap! For one thing the doc
states that this is not locale-dependent but a builtin. That sounds like
a lie from your response.

I think you've misunderstood that part of the documentation.

In scalar context\, "localtime\(\)" returns the ctime\(3\) value&#8203;:

    $now\_string = localtime;  \# e\.g\.\, "Thu Oct 13 04&#8203;:54&#8203;:34 1994"

This scalar value is not locale dependent\, see perllocale\, but
instead a Perl builtin\.

Regardless of your locale, the *formatting* of the scalar value will always
be the same.

And is probably a good example of a trap that Perl6/Parrot should avoid.

Having built ins return language specific date formats is a crappy system.

Localtime should have returned an ISO compliant string like
"YYYY-MM-DD HH​::MM​::SS" which would have been language agnostic,
sortable and much easier to parse. Ie, what POSIX​::strftime "%Y-%m-%d
%H​:%M​:%S" produces.

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented May 15, 2005

From @schwern

On Fri, May 13, 2005 at 09​:40​:33PM -0400, John Peacock wrote​:

It's also not specific to Unix; even though Windows stores it's time in a
64bit integer (as 100's of nanoseconds for some reason), the native C
library localtime() implementation basically suffers from the same 32bit
rollover​:

http&#8203;://www\.codeproject\.com/datetime/time64\.asp

I understand you're trying to show this is a hard problem but just because
everyone else gets it wrong doesn't give us license to do so.

And how is Perl going to deal with that?
<whole lot of informative text about the problem snipped>

Don't know and it doesn't matter for the purposes of this conversation. Its
free for us to put this on a todo list and see what happens. That is my
point. AFTER we decide its a feature we want THEN you can argue about how
its going to be implemented. All I'm looking for right now is "yes, it would
be good if localtime() worked with as wide a range as possible".

What about #2​: the warning that you've overflowed the native localtime()?

If you want to write Configure probes, go right ahead. You should probably
perform something more like a binary search, though, rather than just
leaping to the ends of the range, since some libraries might work well
close to 1970 and not work well outside it. I think a warning is the best
we can do given the enormity of the problem.

Binary search should be efficient enough. Something like 64 comparisons
worst case with a 64bit time_t.

Except...*shudder*...metaconfig and shell programming. I'll take a crack
at it. It will be ugly.

--
Michael G Schwern schwern@​pobox.com http​://www.pobox.com/~schwern
Don't try the paranormal until you know what's normal.
  -- "Lords and Ladies" by Terry Prachett

@p5pRT
Copy link
Author

p5pRT commented May 16, 2005

From @Tux

On Sun, 15 May 2005 14​:50​:28 -0700, Michael G Schwern <schwern@​pobox.com>
wrote​:

If you want to write Configure probes, go right ahead. You should
probably perform something more like a binary search, though, rather than
just leaping to the ends of the range, since some libraries might work
well close to 1970 and not work well outside it. I think a warning is
the best we can do given the enormity of the problem.

Binary search should be efficient enough. Something like 64 comparisons
worst case with a 64bit time_t.

Except...*shudder*...metaconfig and shell programming. I'll take a crack
at it. It will be ugly.

If it's sensible, I'll clean it up and backport to the real metaunits :)

For now, don't you care about *my* problems. Just do what suits you best and
post it here or in a private mail to me (please note my changed mail address)

--
H.Merijn Brand Amsterdam Perl Mongers (http​://amsterdam.pm.org/)
using Perl 5.6.2, 5.8.0, 5.8.5, & 5.9.2 on HP-UX 10.20, 11.00 & 11.11,
AIX 4.3 & 5.2, SuSE 9.2 & 9.3, and Cygwin. http​://www.cmve.net/~merijn
Smoking perl​: http​://www.test-smoke.org, perl QA​: http​://qa.perl.org
reports to​: smokers-reports@​perl.org, perl-qa@​perl.org

@p5pRT
Copy link
Author

p5pRT commented May 16, 2005

From @schwern

[schwern - Fri May 13 13​:31​:36 2005]​:
1) Document that there is a system dependent limitation perlfunc and
perlport
and to only trust localtime(), and any date handling code derived from
localtime(), to work from 1970 to 2038. It may work for more but that
cannot be trusted.

The attached patch implements this suggestion by mentioning the problem
in perlport and referencing it in the localtime and gmtime entries in
perlfunc. Though I'm not crazy about the wording and details it is
something.

@p5pRT
Copy link
Author

p5pRT commented May 16, 2005

From @schwern

localtime1.patch
--- ./pod/perlfunc.pod	2005/05/16 22:03:37	1.1
+++ ./pod/perlfunc.pod	2005/05/16 22:04:29
@@ -2185,6 +2185,8 @@
 instead a Perl builtin.  To get somewhat similar but locale dependent date
 strings, see the example in L</localtime>.
 
+See L<perlport/gmtime> for portability concerns.
+
 =item goto LABEL
 
 =item goto EXPR
@@ -2581,6 +2583,8 @@
 
 Note that the C<%a> and C<%b>, the short forms of the day of the week
 and the month of the year, may not necessarily be three characters wide.
+
+See L<perlport/localtime> for portability concerns.
 
 =item lock THING
 
--- ./pod/perlport.pod	2005/05/16 21:59:12	1.1
+++ ./pod/perlport.pod	2005/05/16 22:02:57
@@ -1783,6 +1783,10 @@
 This operator is implemented via the File::Glob extension on most
 platforms.  See L<File::Glob> for portability information.
 
+=item gmtime EXPR
+
+Same portability caveats as L<localtime>.
+
 =item ioctl FILEHANDLE,FUNCTION,SCALAR
 
 Not implemented. (VMS)
@@ -1815,6 +1819,13 @@
 
 Hard links are implemented on Win32 (Windows NT and Windows 2000)
 under NTFS only.
+
+=item localtime EXPR
+
+Because Perl currently relies on the native standard C localtime()
+function, it is only safe to use times between 0 and (2**31)-1.  Times
+outside this range may result in unexpected behavior depending on your
+operating system's implementation of localtime().
 
 =item lstat FILEHANDLE
 

@p5pRT
Copy link
Author

p5pRT commented May 16, 2005

@schwern - Status changed from 'rejected' to 'open'

@p5pRT
Copy link
Author

p5pRT commented May 16, 2005

From @schwern

[schwern - Fri May 13 13​:31​:36 2005]​:
2) Add a probe to Configure to check what the date range of
localtime() is
(ie. feed it -2**31-1, -2**31, -2**31+1, -1, 0, 1, 2**31-1 and
2**31... and
so on for 64 bit time_t) and warn when the user goes outside that
range.

Attached is an implementation of this in Perl in the hopes that someone
will translate it into metaconfig. I use gmtime() instead of
localtime() to avoid any time zone issues. I use the array return value
to avoid any possible future locale issues. I could not figure out how
to safely use something similar to binary search to get a more accurate
range.

The expected values for a 64bit clean gmtime() need to be filled in as I
don't have such a machine.

@p5pRT
Copy link
Author

p5pRT commented May 16, 2005

From @schwern

find_localtime_limit.plx

@p5pRT
Copy link
Author

p5pRT commented May 17, 2005

From @schwern

I posted up a few followups and patches to this bug via the RT web interface
but I haven't seen them make it to the list. Is RT web -> p5p forwarding
working?

--
Michael G Schwern schwern@​pobox.com http​://www.pobox.com/~schwern
ROCKS FALL! EVERYONE DIES!
  http​://www.somethingpositive.net/sp05032002.shtml

@p5pRT
Copy link
Author

p5pRT commented May 17, 2005

From @rspier

I posted up a few followups and patches to this bug via the RT web interface
but I haven't seen them make it to the list. Is RT web -> p5p forwarding
working?

There is no automatic forwarding from the web interface to the list.
If you want something entered in the web interface to make it to the
list you must add the list to the CC line for that message. (Do not
add the list as a CC for the ticket.)

This is how it has always been. Mail loops are bad.

-R

@p5pRT
Copy link
Author

p5pRT commented May 17, 2005

From @schwern

On Tue, May 17, 2005 at 10​:48​:46AM -0700, Robert Spier wrote​:

I posted up a few followups and patches to this bug via the RT web interface
but I haven't seen them make it to the list. Is RT web -> p5p forwarding
working?

There is no automatic forwarding from the web interface to the list.
If you want something entered in the web interface to make it to the
list you must add the list to the CC line for that message. (Do not
add the list as a CC for the ticket.)

This is how it has always been. Mail loops are bad.

This kinda sucks, a lot of people probably don't realize this and don't
want to have to monitor individual tickets on the web interface to get all
the replies.

Is there a way to break the mail loop perhaps using message-ids?

--
Michael G Schwern schwern@​pobox.com http​://www.pobox.com/~schwern
ROCKS FALL! EVERYONE DIES!
  http​://www.somethingpositive.net/sp05032002.shtml

@p5pRT
Copy link
Author

p5pRT commented May 17, 2005

From @schwern

I posted two patches via RT which can be found here.
http​://rt.perl.org/rt3/Ticket/Display.html?id=35420

One documents the date range limitations of localtime and gmtime.

The other is a proof-of-concept implementation to scan for the limitations
of the system's localtime written in Perl.

--
Michael G Schwern schwern@​pobox.com http​://www.pobox.com/~schwern
You are wicked and wrong to have broken inside and peeked at the
implementation and then relied upon it.
  -- tchrist in <31832.969261130@​chthon>

@p5pRT
Copy link
Author

p5pRT commented May 19, 2005

From @nwc10

On Fri, May 13, 2005 at 09​:40​:33PM -0400, John Peacock wrote​:

calculations (which means occasionally updating the database)? Most
especially, do we break ranks with POSIX (which apparently demands that
2100 be a leap year, even though it isn't).

I would assume here that if POSIX turns out currently to be stating this and
doesn't change, then Linux (or glibc, whichever is important here) will treat
2100 as not-a-leap-year, POSIX-be-damned, and POSIX will end up following.

This isn't saying that it's good. Merely that it's big enough to be a
de-facto standard, and pedantic enough to want to get this sort of thing
right.

On the real issue, I think that documenting the trap and adding a TODO and
checking p5p for patches is the way to go.

It worked for trie optimisations. :-)

Nicholas Clark

@p5pRT
Copy link
Author

p5pRT commented May 19, 2005

From @gisle

Nicholas Clark <nick@​ccl4.org> writes​:

On Fri, May 13, 2005 at 09​:40​:33PM -0400, John Peacock wrote​:

calculations (which means occasionally updating the database)? Most
especially, do we break ranks with POSIX (which apparently demands that
2100 be a leap year, even though it isn't).

I would assume here that if POSIX turns out currently to be stating this and
doesn't change, then Linux (or glibc, whichever is important here) will treat
2100 as not-a-leap-year, POSIX-be-damned, and POSIX will end up following.

This bug in the specs was fixed in POSIX​:2001. See
http​://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_04_14
for the current definition of POSIX time.

--Gisle

@p5pRT
Copy link
Author

p5pRT commented May 19, 2005

From @JohnPeacock

Gisle Aas wrote​:

This bug in the specs was fixed in POSIX​:2001. See
http​://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap04.html#tag_04_14
for the current definition of POSIX time.

Excellent! Based on the history (IEEE Std 1003.1 was originally published in
1988 and became ISO/IEC 9945-1 in 1990), then it will be approximately 2010
before they get around to defining the behavior of negative offsets from the
epoch. ;-)

John

--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747

@p5pRT
Copy link
Author

p5pRT commented May 19, 2005

From @JohnPeacock

Nicholas Clark wrote​:

I would assume here that if POSIX turns out currently to be stating this and
doesn't change, then Linux (or glibc, whichever is important here) will treat
2100 as not-a-leap-year, POSIX-be-damned, and POSIX will end up following.

Well, the reality of the thing is that current 32-bit implementations of
localtime die in 2038 anyways, so the fact that they don't also handle 2100
correctly is kind of beside the point.

On the real issue, I think that documenting the trap and adding a TODO and
checking p5p for patches is the way to go.

I agree. It actually occurred to me that if we only wanted to fix localtime (to
support negative offsets), we could (with some caveats). Doing it right,
however, involves bringing the DateTime suite of modules into the core, IMNSHO.

John

--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747

@p5pRT
Copy link
Author

p5pRT commented May 20, 2005

From dpfeiffer@amadeus.net

John Peacock wrote​:

Daniel reported that he was checking both very small and
very large numbers. What you describe below is much more realistic of
"real
world Perl" than trying values likely to cross boundaries.

I had a rather realistic application​: A colleague claimed that most bad
things indeed happen on friday 13th, because statistically that happens to
be the most frequent date. He challenged me to hack up a Perl check for
that. After a quick test that negatives work (on GNU/Linux), I did that
in 5 minutes by iterating over all values in 86400 second steps and
incrementing weekday-monthday hash values.

To my surprise one value came excessively frequently, those were the
smallest numbers which all mapped to 1901. This led me to control the
distribution of localtime values. When I reduced the range, I easily
falsified that absurd friday 13th statement. All days, except the
29th-31st, are equally frequent (+-1, probably due to the arbitrary limits
of the checked interval).

mit freundlichen Grüßen / best regards
Daniel

--
lerne/learn Esperanto online​: http​://lernu.net/

@p5pRT
Copy link
Author

p5pRT commented May 22, 2005

From wolfgang.laun@alcatel.at

Daniel Pfeiffer wrote​:

A colleague claimed that most bad things indeed happen on
friday 13th, because statistically that happens to be the
most frequent date.

There is indeed a connection between Friday and 13. It is,
however, frequently stated imprecisely, so that checks may be
doomed by that, or by inaccurate programming.

This, then, is the correct statement​: The 13th day of a month
is more often a Friday than any other day of the week.

To check, you have to analyze a full Gregorian Period of 400aG
or 146097d (146097 % 7 == 0), containing 4800 days numbered 13.

The maximum for Friday is 688, which is only 0.33% more than
the average (685 5/7).

Cheers
Wolfgang

@p5pRT
Copy link
Author

p5pRT commented May 26, 2005

From @schwern

My patch to the localtime documentation appears to have been lost in the
noise. Here it is again. It documents in perlport the portability issues
of time ranges localtime() can handle.

--
Michael G Schwern schwern@​pobox.com http​://www.pobox.com/~schwern
Just call me 'Moron Sugar'.
  http​://www.somethingpositive.net/sp05182002.shtml

@p5pRT
Copy link
Author

p5pRT commented May 26, 2005

From @schwern

localtime.patch
--- pod/perlfunc.pod	2005/05/26 20:38:07	1.1
+++ pod/perlfunc.pod	2005/05/26 20:38:17
@@ -2186,6 +2186,8 @@
 instead a Perl builtin.  To get somewhat similar but locale dependent date
 strings, see the example in L</localtime>.
 
+See L<perlport/gmtime> for portability concerns.
+
 =item goto LABEL
 
 =item goto EXPR
@@ -2582,6 +2584,8 @@
 
 Note that the C<%a> and C<%b>, the short forms of the day of the week
 and the month of the year, may not necessarily be three characters wide.
+
+See L<perlport/localtime> for portability concerns.
 
 =item lock THING
 
--- pod/perlport.pod	2005/05/26 20:38:13	1.1
+++ pod/perlport.pod	2005/05/26 20:38:34
@@ -1773,6 +1773,10 @@
 This operator is implemented via the File::Glob extension on most
 platforms.  See L<File::Glob> for portability information.
 
+=item gmtime
+
+Same portability caveats as L<localtime>.
+
 =item ioctl FILEHANDLE,FUNCTION,SCALAR
 
 Not implemented. (VMS)
@@ -1805,6 +1809,13 @@
 
 Hard links are implemented on Win32 (Windows NT and Windows 2000)
 under NTFS only.
+
+=item localtime
+
+Because Perl currently relies on the native standard C localtime()
+function, it is only safe to use times between 0 and (2**31)-1.  Times
+outside this range may result in unexpected behavior depending on your
+operating system's implementation of localtime().
 
 =item lstat
 

@p5pRT
Copy link
Author

p5pRT commented May 27, 2005

From @rgs

Michael G Schwern wrote​:

My patch to the localtime documentation appears to have been lost in the
noise. Here it is again. It documents in perlport the portability issues
of time ranges localtime() can handle.

Thanks, applied as #24593.

@p5pRT p5pRT closed this as completed May 27, 2005
@p5pRT
Copy link
Author

p5pRT commented May 27, 2005

@rgs - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant