Skip to contentSkip to conversation

timekeeping: Limit system time to prevent 32-bit time_t overflow

PATCH In review Miroslav Lichvar 3 participants 11 messages Oct 7, 2015 – Oct 9, 2015

Subsystems (3)
lore
Share & tools
lore

Miroslav Lichvar

author reviewer
#0 lore raw

PATCH 1/1 timekeeping: Limit system time to prevent 32-bit time_t overflow

On systems with 32-bit time_t there are quite a few problems that applications may have with time overflowing in year 2038. Beside getting to an unexpected state by not checking integer operations with time_t variables, some system calls have unexpected behavior, e.g. the system time can't be set back to the current time (negative value), timers with the ABSTIME flag can't be set (negative value) or they expire immediately (current time is always larger).

It would be unrealistic to expect all applications to be able to handle all these problems. Year 2038 is still many years away, but this can be a problem even now. The clock can get close to the overflow accidentally or maliciously, e.g. when it is synchronized over network by NTP or PTP.

This patch sets a maximum value of the system time to prevent the system time from getting too close to the overflow. The time can't be set to a larger value. When the maximum is reached in normal time accumulation, the clock will be stepped back by one week.

A new kernel sysctl time_max is added to select the maximum time. It can be set to 0 for no limit, 1 for one week before 32-bit time_t overflow, and 2 for one week before ktime_t overflow. The default value is 1 with 32-bit time_t and 2 with 64-bit time_t. This can be changed later to be always 2 when 64-bit versions of system calls working with time_t are available.

Cc John Stultz <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Prarit Bhargava <[email protected]>
Cc Richard Cochran <[email protected]>
Cc Arnd Bergmann <[email protected]>

---

v2: - optimized code in accumulate_nsecs_to_secs() a bit
- improved log message and increased its level to warning

 Documentation/sysctl/kernel.txt | 19 ++++++++++++
 include/linux/timekeeping.h     |  5 ++++
 include/uapi/linux/sysctl.h     |  1 +
 kernel/sysctl.c                 |  9 ++++++
 kernel/sysctl_binary.c          |  1 +
 kernel/time/timekeeping.c       | 64 +++++++++++++++++++++++++++++++++++++----
 6 files changed, 93 insertions(+), 6 deletions(-)
Documentation/sysctl/kernel.txt +19 -0
index 6fccb69..9b2bbdd 100644
@@ -83,6 +83,7 @@ show up in /proc/sys/kernel:
 - sysctl_writes_strict
 - tainted
 - threads-max
+- time_max
 - unknown_nmi_panic
 - watchdog
 - watchdog_thresh
@@ -893,6 +894,24 @@ available RAM pages threads-max is reduced accordingly.
 
 ==============================================================
 
+time_max:
+
+Select the maximum allowed value of the system time. The system clock cannot be
+set to a larger value and when it reaches the maximum on its own, it will be
+stepped back by one week.
+
+0: No limit.
+
+1: One week before 32-bit time_t overflows, i.e. Jan 12 03:14:07 UTC 2038.
+   This is currently the default value with 32-bit time_t, but it may change
+   when 64-bit versions of system calls working with time_t are available.
+
+2: One week before time in the internal kernel representation (ktime_t)
+   overflows, i.e. Apr 4 23:47:16 UTC 2262. This is the default value with
+   64-bit time_t.
+
+==============================================================
+
 unknown_nmi_panic:
 
 The value in this file affects behavior of handling NMI. When the
include/linux/timekeeping.h +5 -0
index ba0ae09..f25df65 100644
@@ -5,6 +5,11 @@
 
 void timekeeping_init(void);
 extern int timekeeping_suspended;
+extern int sysctl_time_max;
+
+struct ctl_table;
+extern int sysctl_time_max_handler(struct ctl_table *table, int write,
+		void __user *buffer, size_t *length, loff_t *ppos);
 
 /*
  * Get and set timeofday
include/uapi/linux/sysctl.h +1 -0
index 0956373..8fd2aab 100644
@@ -154,6 +154,7 @@ enum
 	KERN_NMI_WATCHDOG=75, /* int: enable/disable nmi watchdog */
 	KERN_PANIC_ON_NMI=76, /* int: whether we will panic on an unrecovered */
 	KERN_PANIC_ON_WARN=77, /* int: call panic() in WARN() functions */
+	KERN_TIMEMAX=78,	/* int: select maximum allowed system time */
 };
 
 
kernel/sysctl.c +9 -0
index e69201d..c2cfb33 100644
@@ -1130,6 +1130,15 @@ static struct ctl_table kern_table[] = {
 		.extra1		= &zero,
 		.extra2		= &one,
 	},
+	{
+		.procname	= "time_max",
+		.data		= &sysctl_time_max,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= sysctl_time_max_handler,
+		.extra1		= &zero,
+		.extra2		= &two,
+	},
 #if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ_COMMON)
 	{
 		.procname	= "timer_migration",
kernel/sysctl_binary.c +1 -0
index 7e7746a..66c0946 100644
@@ -138,6 +138,7 @@ static const struct bin_table bin_kern_table[] = {
 	{ CTL_INT,	KERN_MAX_LOCK_DEPTH,		"max_lock_depth" },
 	{ CTL_INT,	KERN_PANIC_ON_NMI,		"panic_on_unrecovered_nmi" },
 	{ CTL_INT,	KERN_PANIC_ON_WARN,		"panic_on_warn" },
+	{ CTL_INT,	KERN_TIMEMAX,			"time_max" },
 	{}
 };
 
kernel/time/timekeeping.c +58 -6

107 lines not on this page · raw message

Signature · 1 line
2.1.0
#1 lore raw
Quoted 18 lines · On Wednesday 07 October 2015 15:22:17 Miroslav Lichvar wrote:

On systems with 32-bit time_t there are quite a few problems that applications may have with time overflowing in year 2038. Beside getting to an unexpected state by not checking integer operations with time_t variables, some system calls have unexpected behavior, e.g. the system time can't be set back to the current time (negative value), timers with the ABSTIME flag can't be set (negative value) or they expire immediately (current time is always larger).

It would be unrealistic to expect all applications to be able to handle all these problems. Year 2038 is still many years away, but this can be a problem even now. The clock can get close to the overflow accidentally or maliciously, e.g. when it is synchronized over network by NTP or PTP.

This patch sets a maximum value of the system time to prevent the system time from getting too close to the overflow. The time can't be set to a larger value. When the maximum is reached in normal time accumulation, the clock will be stepped back by one week.

Quoted 6 lines

A new kernel sysctl time_max is added to select the maximum time. It can be set to 0 for no limit, 1 for one week before 32-bit time_t overflow, and 2 for one week before ktime_t overflow. The default value is 1 with 32-bit time_t and 2 with 64-bit time_t. This can be changed later to be always 2 when 64-bit versions of system calls working with time_t are available.

I can't see whether this is really a good idea: moving the time backwards will break all sorts of drivers that (incorrectly) expect the real time clock to have monotonic behavior, and quite often, file timestamps are expected to be in the past in user space. A common example is 'make', which goes nuts when it sees files in the future.

So for all I can tell, your patch only replaces one set of problems that happens at the time of the overflow with a different set of problems.

Arnd

author reviewer
#2 lore raw
Quoted 9 lines · Arnd Bergmann
Quoted 4 lines · On Wednesday 07 October 2015 15:22:17 Miroslav Lichvar wrote:

This patch sets a maximum value of the system time to prevent the system time from getting too close to the overflow. The time can't be set to a larger value. When the maximum is reached in normal time accumulation, the clock will be stepped back by one week.

I can't see whether this is really a good idea: moving the time backwards will break all sorts of drivers that (incorrectly) expect the real time clock to have monotonic behavior, and quite often, file timestamps

Well, do these drivers break when the clock is stepped back by ntpd? Maybe it would be better to fix them.

Quoted 2 lines

are expected to be in the past in user space. A common example is 'make', which goes nuts when it sees files in the future.

Without the limit added by this patch make will go nuts just one week later when the 32-bit time_t overflows to Dec 13 1901 and the files will appear as 136 years in the future. How is that better?

Quoted 2 lines

So for all I can tell, your patch only replaces one set of problems that happens at the time of the overflow with a different set of problems.

I think it converts a set of problems that are difficult to handle and most people don't even know about to a single well known problem (clock being stepped back).

I'd suggest to run "date -s @$[2**31 - 20]" on a 32-bit system and try working in that.

Signature · 1 line
Miroslav Lichvar
#3 lore raw
Quoted 7 lines · On Wednesday 07 October 2015 16:23:44 Miroslav Lichvar wrote:
Quoted 6 lines · Arnd Bergmann
Quoted 4 lines · On Wednesday 07 October 2015 15:22:17 Miroslav Lichvar wrote:

This patch sets a maximum value of the system time to prevent the system time from getting too close to the overflow. The time can't be set to a larger value. When the maximum is reached in normal time accumulation, the clock will be stepped back by one week.

Quoted 5 lines
Quoted 3 lines

I can't see whether this is really a good idea: moving the time backwards will break all sorts of drivers that (incorrectly) expect the real time clock to have monotonic behavior, and quite often, file timestamps

Well, do these drivers break when the clock is stepped back by ntpd?

Yes.

Quoted 1 line

Maybe it would be better to fix them.

We are in the process of doing that: All drivers that currently use do_gettimeofday() or get_seconds() are being audited and converted to one of ktime_get(), ktime_get_real(), ktime_get_ts64(), ktime_get_real_ts64(), ktime_get_seconds() or ktime_get_real_seconds().

The 'real' versions should only be used when the driver wants to know the wallclock but is ok with time going backwards.

Quoted 6 lines
Quoted 2 lines

are expected to be in the past in user space. A common example is 'make', which goes nuts when it sees files in the future.

Without the limit added by this patch make will go nuts just one week later when the 32-bit time_t overflows to Dec 13 1901 and the files will appear as 136 years in the future. How is that better?

Not better or worse at all, that was my point. The time is still wrong either way, whether you step back by a week or 136 years.

Arnd

author reviewer
#4 lore raw
Quoted 7 lines · Arnd Bergmann
Quoted 3 lines · On Wednesday 07 October 2015 16:23:44 Miroslav Lichvar wrote:

Without the limit added by this patch make will go nuts just one week later when the 32-bit time_t overflows to Dec 13 1901 and the files will appear as 136 years in the future. How is that better?

Not better or worse at all, that was my point. The time is still wrong either way, whether you step back by a week or 136 years.

The difference is that with the one-week step the kernel and userspace still agree on the current time and it is always valid from the kernel point of view, absolute timers can be set, etc.

Signature · 1 line
Miroslav Lichvar
#5 lore raw
Quoted 6 lines · On Thursday 08 October 2015 08:23:44 Miroslav Lichvar wrote:
Quoted 5 lines · Arnd Bergmann
Quoted 3 lines · On Wednesday 07 October 2015 16:23:44 Miroslav Lichvar wrote:

Without the limit added by this patch make will go nuts just one week later when the 32-bit time_t overflows to Dec 13 1901 and the files will appear as 136 years in the future. How is that better?

Quoted 6 lines
Quoted 2 lines

Not better or worse at all, that was my point. The time is still wrong either way, whether you step back by a week or 136 years.

The difference is that with the one-week step the kernel and userspace still agree on the current time and it is always valid from the kernel point of view, absolute timers can be set, etc.

Ok, I can see that as an improvement, but it still seems to give a false sense of safety, and I feel we really should not have any code rely on this behavior.

Arnd

author reviewer
#6 lore raw
Quoted 8 lines · Arnd Bergmann
Quoted 3 lines · On Thursday 08 October 2015 08:23:44 Miroslav Lichvar wrote:

The difference is that with the one-week step the kernel and userspace still agree on the current time and it is always valid from the kernel point of view, absolute timers can be set, etc.

Ok, I can see that as an improvement, but it still seems to give a false sense of safety, and I feel we really should not have any code rely on this behavior.

Applications are not allowed to rely on system time being sane? To me the current behavior looks like the kernel is throwing the applications off a cliff, while it's the only thing that can fly :).

Signature · 1 line
Miroslav Lichvar
#7 lore raw
Quoted 6 lines · Miroslav Lichvar
Quoted 5 lines · Arnd Bergmann
Quoted 3 lines · On Thursday 08 October 2015 08:23:44 Miroslav Lichvar wrote:

The difference is that with the one-week step the kernel and userspace still agree on the current time and it is always valid from the kernel point of view, absolute timers can be set, etc.

Quoted 7 lines
Quoted 3 lines

Ok, I can see that as an improvement, but it still seems to give a false sense of safety, and I feel we really should not have any code rely on this behavior.

Applications are not allowed to rely on system time being sane? To me the current behavior looks like the kernel is throwing the applications off a cliff, while it's the only thing that can fly :).

As Arnd said, you are creating a wrong sense of safety. They fall off the cliff with your changes as well. The fall is just different. Think about timeouts, user space overflows of time_t etc.

We need to fix all of it, no matter what.

Thanks,

tglx

author reviewer
#8 lore raw
Quoted 8 lines · Thomas Gleixner
Quoted 3 lines · Miroslav Lichvar

Applications are not allowed to rely on system time being sane? To me the current behavior looks like the kernel is throwing the applications off a cliff, while it's the only thing that can fly :).

As Arnd said, you are creating a wrong sense of safety. They fall off the cliff with your changes as well. The fall is just different. Think about timeouts, user space overflows of time_t etc.

I think vast majority of them won't fall. It doesn't prevent all problems, but at least the userspace agrees with kernel on what the current time it is, that looks to me like the most difficult one to fix in applications.

Quoted 1 line

We need to fix all of it, no matter what.

Yeah, that would be nice, but I don't think it's realistic.

Do you feel the same about preventing the time from reaching KTIME_MAX?

Signature · 1 line
Miroslav Lichvar
#9 lore raw
Quoted 15 lines · Miroslav Lichvar
Quoted 8 lines · Thomas Gleixner
Quoted 3 lines · Miroslav Lichvar

Applications are not allowed to rely on system time being sane? To me the current behavior looks like the kernel is throwing the applications off a cliff, while it's the only thing that can fly :).

As Arnd said, you are creating a wrong sense of safety. They fall off the cliff with your changes as well. The fall is just different. Think about timeouts, user space overflows of time_t etc.

I think vast majority of them won't fall. It doesn't prevent all problems, but at least the userspace agrees with kernel on what the current time it is, that looks to me like the most difficult one to fix in applications.

Quoted 6 lines
Quoted 1 line

We need to fix all of it, no matter what.

Yeah, that would be nice, but I don't think it's realistic.

Do you feel the same about preventing the time from reaching KTIME_MAX?

That's going to happen in ~500 years from now.

Thanks,

tglx

author reviewer
#10 lore raw
Quoted 5 lines · Thomas Gleixner
Quoted 2 lines · Miroslav Lichvar

Do you feel the same about preventing the time from reaching KTIME_MAX?

That's going to happen in ~500 years from now.

At any time if you include accidents and attacks on services with the CAP_SYS_TIME capability (e.g. NTP/PTP implementations). Even with 32-bit time_t you are not safe, the time can be set forward by any amount with multiple adjtimex(ADJ_SETOFFSET) calls.

Signature · 1 line
Miroslav Lichvar