Y2038 vs struct cache_time/time_t

Y2038 vs struct cache_time/time_t

[Johannes Schindelin]

Team,

today, in quite an entertaining thread on Twitter
(https://twitter.com/jxxf/status/1219009308438024200) I read about yet
another account how the Year 2038 problem already bites people. And costs
real amounts of money.

And after I stopped shaking my head in disbelief, I had a quick look, and
it seems that we're safe at least until February 7th, 2106. That's not
great, but I plan on not being around at that date anymore, so there. That
date is when the unsigned 32-bit Unix epoch will roll over and play
dead^W^Wwreak havoc (iff the human species manages to actually turn around
and reverse the climate catastrophe it caused, and that's a big iff):
https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs#Year_2106

Concretely, it looks as if we store our own timestamps on disk (in the
index file) as uint32_t:

	/*
	 * The "cache_time" is just the low 32 bits of the
	 * time. It doesn't matter if it overflows - we only
	 * check it for equality in the 32 bits we save.
	 */
	struct cache_time {
		uint32_t sec;
		uint32_t nsec;
	};

The comment seems to indicate that we are still safe even if 2106 comes
around, but I am not _quite_ that sure, as I expect us to have "greater
than" checks, not only equality checks.

But wait, we're still not quite safe. If I remember correctly, 32-bit
Linux still uses _signed_ 32-bit integers as `time_t`, so when we render
dates, for example, and use system-provided functions, on 32-bit Linux we
will at least show the wrong dates starting 2038.

This got me thinking, and I put on my QA hat. Kids, try this at home:

	$ git log --until=1.january.1960

	$ git log --since=1.january.2200

Git does not really do what you expected, eh?

Maybe we want to do something about that, and while at it also fix the
overflow problems, probably requiring a new index format?

Ciao,
Johannes

Re: Y2038 vs struct cache_time/time_t

[Michal Suchánek]

On Mon, Jan 20, 2020 at 08:38:51PM +0100, Johannes Schindelin wrote:
> Team,
> 
> today, in quite an entertaining thread on Twitter
> (https://twitter.com/jxxf/status/1219009308438024200) I read about yet
> another account how the Year 2038 problem already bites people. And costs
> real amounts of money.
> 
> And after I stopped shaking my head in disbelief, I had a quick look, and
> it seems that we're safe at least until February 7th, 2106. That's not
> great, but I plan on not being around at that date anymore, so there. That
> date is when the unsigned 32-bit Unix epoch will roll over and play
> dead^W^Wwreak havoc (iff the human species manages to actually turn around
> and reverse the climate catastrophe it caused, and that's a big iff):
> https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs#Year_2106
> 
> Concretely, it looks as if we store our own timestamps on disk (in the
> index file) as uint32_t:
> 
> 	/*
> 	 * The "cache_time" is just the low 32 bits of the
> 	 * time. It doesn't matter if it overflows - we only
> 	 * check it for equality in the 32 bits we save.
> 	 */
> 	struct cache_time {
> 		uint32_t sec;
> 		uint32_t nsec;
> 	};
> 
> The comment seems to indicate that we are still safe even if 2106 comes
> around, but I am not _quite_ that sure, as I expect us to have "greater
> than" checks, not only equality checks.
> 
> But wait, we're still not quite safe. If I remember correctly, 32-bit
> Linux still uses _signed_ 32-bit integers as `time_t`, so when we render
> dates, for example, and use system-provided functions, on 32-bit Linux we
> will at least show the wrong dates starting 2038.
> 
> This got me thinking, and I put on my QA hat. Kids, try this at home:
> 
> 	$ git log --until=1.january.1960
> 
> 	$ git log --since=1.january.2200
> 
> Git does not really do what you expected, eh?
> 
> Maybe we want to do something about that, and while at it also fix the
> overflow problems, probably requiring a new index format?

Which means we can split off the timestamps to a separate file, too ;-)

Thanks

Michal

RE: Y2038 vs struct cache_time/time_t

[Randall S. Becker]

On January 19, 2038 (no really January 20, 2020 2:39 PM), Johannes
Schindelin wrote:
> today, in quite an entertaining thread on Twitter
> (https://twitter.com/jxxf/status/1219009308438024200) I read about yet
> another account how the Year 2038 problem already bites people. And costs
> real amounts of money.
> 
> And after I stopped shaking my head in disbelief, I had a quick look, and
it
> seems that we're safe at least until February 7th, 2106. That's not great,
but I
> plan on not being around at that date anymore, so there. That date is when
> the unsigned 32-bit Unix epoch will roll over and play dead^W^Wwreak
> havoc (iff the human species manages to actually turn around and reverse
> the climate catastrophe it caused, and that's a big iff):
> https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs#Year_21
> 06
> 
> Concretely, it looks as if we store our own timestamps on disk (in the
index
> file) as uint32_t:
> 
> 	/*
> 	 * The "cache_time" is just the low 32 bits of the
> 	 * time. It doesn't matter if it overflows - we only
> 	 * check it for equality in the 32 bits we save.
> 	 */
> 	struct cache_time {
> 		uint32_t sec;
> 		uint32_t nsec;
> 	};
> 
> The comment seems to indicate that we are still safe even if 2106 comes
> around, but I am not _quite_ that sure, as I expect us to have "greater
than"
> checks, not only equality checks.
> 
> But wait, we're still not quite safe. If I remember correctly, 32-bit
Linux still
> uses _signed_ 32-bit integers as `time_t`, so when we render dates, for
> example, and use system-provided functions, on 32-bit Linux we will at
least
> show the wrong dates starting 2038.
> 
> This got me thinking, and I put on my QA hat. Kids, try this at home:
> 
> 	$ git log --until=1.january.1960
> 
> 	$ git log --since=1.january.2200
> 
> Git does not really do what you expected, eh?
> 
> Maybe we want to do something about that, and while at it also fix the
> overflow problems, probably requiring a new index format?

The preferred way of fixing this is traditionally - for those of us who have
been through it (4-ish times), to convert to time64_t where available (big
legacy machines, like z/OS and NonStop), or in gcc, time_t is 64 bit on 64
bit systems. It has been 64 bit on Windows since VS 2005. I have a
relatively some relatively old Linux distros on 64 bit processors that also
have time_t set as 64 bit in gcc. Those seem to be the standard approaches.
To cover it, I suggest we move to a gittime_t which is always 64 bit (or 128
bit if you don't want to be resurrected after the sun turns into a red giant
or later when we are left with evaporating black holes), no matter what the
platform, and build the selection of what gittime_t is (time_t or time64_t)
into our config and/or compat.h. That way, hopefully, people will rebuild
their git before 2038 or before someone decides to stick a fake date into a
Github repo just to mess with us.

Cheers,
Randall

Re: Y2038 vs struct cache_time/time_t

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2429 bytes --]

Hi Michal,

On Mon, 20 Jan 2020, Michal Suchánek wrote:

> On Mon, Jan 20, 2020 at 08:38:51PM +0100, Johannes Schindelin wrote:
> > Team,
> >
> > today, in quite an entertaining thread on Twitter
> > (https://twitter.com/jxxf/status/1219009308438024200) I read about yet
> > another account how the Year 2038 problem already bites people. And costs
> > real amounts of money.
> >
> > And after I stopped shaking my head in disbelief, I had a quick look, and
> > it seems that we're safe at least until February 7th, 2106. That's not
> > great, but I plan on not being around at that date anymore, so there. That
> > date is when the unsigned 32-bit Unix epoch will roll over and play
> > dead^W^Wwreak havoc (iff the human species manages to actually turn around
> > and reverse the climate catastrophe it caused, and that's a big iff):
> > https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs#Year_2106
> >
> > Concretely, it looks as if we store our own timestamps on disk (in the
> > index file) as uint32_t:
> >
> > 	/*
> > 	 * The "cache_time" is just the low 32 bits of the
> > 	 * time. It doesn't matter if it overflows - we only
> > 	 * check it for equality in the 32 bits we save.
> > 	 */
> > 	struct cache_time {
> > 		uint32_t sec;
> > 		uint32_t nsec;
> > 	};
> >
> > The comment seems to indicate that we are still safe even if 2106 comes
> > around, but I am not _quite_ that sure, as I expect us to have "greater
> > than" checks, not only equality checks.
> >
> > But wait, we're still not quite safe. If I remember correctly, 32-bit
> > Linux still uses _signed_ 32-bit integers as `time_t`, so when we render
> > dates, for example, and use system-provided functions, on 32-bit Linux we
> > will at least show the wrong dates starting 2038.
> >
> > This got me thinking, and I put on my QA hat. Kids, try this at home:
> >
> > 	$ git log --until=1.january.1960
> >
> > 	$ git log --since=1.january.2200
> >
> > Git does not really do what you expected, eh?
> >
> > Maybe we want to do something about that, and while at it also fix the
> > overflow problems, probably requiring a new index format?
>
> Which means we can split off the timestamps to a separate file, too ;-)

Sure. We could also jump from a cliff at the same time. Just because you
can do something does not mean that it is a good idea to actually do it.

Ciao,
Johannes

RE: Y2038 vs struct cache_time/time_t

[Johannes Schindelin]

Hi Randall,

On Mon, 20 Jan 2020, Randall S. Becker wrote:

> On January 19, 2038 (no really January 20, 2020 2:39 PM), Johannes
> Schindelin wrote:
> > today, in quite an entertaining thread on Twitter
> > (https://twitter.com/jxxf/status/1219009308438024200) I read about yet
> > another account how the Year 2038 problem already bites people. And costs
> > real amounts of money.
> >
> > And after I stopped shaking my head in disbelief, I had a quick look, and
> it
> > seems that we're safe at least until February 7th, 2106. That's not great,
> but I
> > plan on not being around at that date anymore, so there. That date is when
> > the unsigned 32-bit Unix epoch will roll over and play dead^W^Wwreak
> > havoc (iff the human species manages to actually turn around and reverse
> > the climate catastrophe it caused, and that's a big iff):
> > https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs#Year_21
> > 06
> >
> > Concretely, it looks as if we store our own timestamps on disk (in the
> index
> > file) as uint32_t:
> >
> > 	/*
> > 	 * The "cache_time" is just the low 32 bits of the
> > 	 * time. It doesn't matter if it overflows - we only
> > 	 * check it for equality in the 32 bits we save.
> > 	 */
> > 	struct cache_time {
> > 		uint32_t sec;
> > 		uint32_t nsec;
> > 	};
> >
> > The comment seems to indicate that we are still safe even if 2106 comes
> > around, but I am not _quite_ that sure, as I expect us to have "greater
> than"
> > checks, not only equality checks.
> >
> > But wait, we're still not quite safe. If I remember correctly, 32-bit
> Linux still
> > uses _signed_ 32-bit integers as `time_t`, so when we render dates, for
> > example, and use system-provided functions, on 32-bit Linux we will at
> least
> > show the wrong dates starting 2038.
> >
> > This got me thinking, and I put on my QA hat. Kids, try this at home:
> >
> > 	$ git log --until=1.january.1960
> >
> > 	$ git log --since=1.january.2200
> >
> > Git does not really do what you expected, eh?
> >
> > Maybe we want to do something about that, and while at it also fix the
> > overflow problems, probably requiring a new index format?
>
> The preferred way of fixing this is traditionally - for those of us who have
> been through it (4-ish times), to convert to time64_t where available (big
> legacy machines, like z/OS and NonStop), or in gcc, time_t is 64 bit on 64
> bit systems. It has been 64 bit on Windows since VS 2005. I have a
> relatively some relatively old Linux distros on 64 bit processors that also
> have time_t set as 64 bit in gcc. Those seem to be the standard approaches.
> To cover it, I suggest we move to a gittime_t which is always 64 bit (or 128
> bit if you don't want to be resurrected after the sun turns into a red giant
> or later when we are left with evaporating black holes), no matter what the
> platform, and build the selection of what gittime_t is (time_t or time64_t)
> into our config and/or compat.h. That way, hopefully, people will rebuild
> their git before 2038 or before someone decides to stick a fake date into a
> Github repo just to mess with us.

I like it. If I had time to tackle this, I would definitely go for
`git_time64_t`.

Ciao,
Dscho