FAILED: patch "[PATCH] clocksource: Make negative motion detection more robust" failed to apply to 6.12-stable tree

PATCH [email protected] 3 participants 10 messages Dec 12, 2024 – Dec 13, 2024

Subsystems (6)

Stable branch Kselftest Library code RISC-V +2 more subsystems

lore

Share & tools

lore

[email protected]

author maintainer

2024-12-12 13:03 UTC #0 lore raw

The patch below does not apply to the 6.12-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.

To reproduce the conflict and resubmit, you may use the following commands:

git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-6.12.y
git checkout FETCH_HEAD
git cherry-pick -x 76031d9536a076bf023bedbdb1b4317fc801dd67
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<[email protected]>' --in-reply-to '2024121203-griminess-blah-4e97@gregkh' --subject-prefix 'PATCH 6.12.y' HEAD^..

Possible dependencies:

thanks,

greg k-h

------------------ original commit in Linus's tree ------------------

From 76031d9536a076bf023bedbdb1b4317fc801dd67 Mon Sep 17 00:00:00 2001
From: Thomas Gleixner <[email protected]>
Date: Tue, 3 Dec 2024 11:16:30 +0100
Subject: [PATCH] clocksource: Make negative motion detection more robust

Guenter reported boot stalls on a emulated ARM 32-bit platform, which has a 24-bit wide clocksource.

It turns out that the calculated maximal idle time, which limits idle sleeps to prevent clocksource wrap arounds, is close to the point where the negative motion detection triggers.

max_idle_ns: 597268854 ns
negative motion tripping point: 671088640 ns

If the idle wakeup is delayed beyond that point, the clocksource advances far enough to trigger the negative motion detection. This prevents the clock to advance and in the worst case the system stalls completely if the consecutive sleeps based on the stale clock are delayed as well.

Cure this by calculating a more robust cut-off value for negative motion, which covers 87.5% of the actual clocksource counter width. Compare the delta against this value to catch negative motion. This is specifically for clock sources with a small counter width as their wrap around time is close to the half counter width. For clock sources with wide counters this is not a problem because the maximum idle time is far from the half counter width due to the math overflow protection constraints.

For the case at hand this results in a tripping point of 1174405120ns.

Note, that this cannot prevent issues when the delay exceeds the 87.5% margin, but that's not different from the previous unchecked version which allowed arbitrary time jumps.

Systems with small counter width are prone to invalid results, but this problem is unlikely to be seen on real hardware. If such a system completely stalls for more than half a second, then there are other more urgent problems than the counter wrapping around.

Fixes c163e40af9b2 ("timekeeping: Always check for negative motion")

Reported-by Guenter Roeck <[email protected]>

Signed-off-by Thomas Gleixner <[email protected]>

Tested-by Guenter Roeck <[email protected]>

Link https://lore.kernel.org/all/8734j5ul4x.ffs@tglx open here

Closes https://lore.kernel.org/all/[email protected] open here

include/linux/clocksource.h +2 -0

index ef1b16da6ad5..65b7c41471c3 100644
@@ -49,6 +49,7 @@ struct module;
  * @archdata:		Optional arch-specific data
  * @max_cycles:		Maximum safe cycle value which won't overflow on
  *			multiplication
+ * @max_raw_delta:	Maximum safe delta value for negative motion detection
  * @name:		Pointer to clocksource name
  * @list:		List head for registration (internal)
  * @freq_khz:		Clocksource frequency in khz.

@@ -109,6 +110,7 @@ struct clocksource {
 	struct arch_clocksource_data archdata;
 #endif
 	u64			max_cycles;
+	u64			max_raw_delta;
 	const char		*name;
 	struct list_head	list;
 	u32			freq_khz;

kernel/time/clocksource.c +10 -1

index aab6472853fa..7304d7cf47f2 100644
@@ -24,7 +24,7 @@ static void clocksource_enqueue(struct clocksource *cs);
 
 static noinline u64 cycles_to_nsec_safe(struct clocksource *cs, u64 start, u64 end)
 {
-	u64 delta = clocksource_delta(end, start, cs->mask);
+	u64 delta = clocksource_delta(end, start, cs->mask, cs->max_raw_delta);
 
 	if (likely(delta < cs->max_cycles))
 		return clocksource_cyc2ns(delta, cs->mult, cs->shift);

@@ -993,6 +993,15 @@ static inline void clocksource_update_max_deferment(struct clocksource *cs)
 	cs->max_idle_ns = clocks_calc_max_nsecs(cs->mult, cs->shift,
 						cs->maxadj, cs->mask,
 						&cs->max_cycles);
+
+	/*
+	 * Threshold for detecting negative motion in clocksource_delta().
+	 *
+	 * Allow for 0.875 of the counter width so that overly long idle
+	 * sleeps, which go slightly over mask/2, do not trigger the
+	 * negative motion detection.
+	 */
+	cs->max_raw_delta = (cs->mask >> 1) + (cs->mask >> 2) + (cs->mask >> 3);
 }
 
 static struct clocksource *clocksource_find_best(bool oneshot, bool skipcur)

kernel/time/timekeeping.c +4 -2

index 0ca85ff4fbb4..3d128825d343 100644
@@ -755,7 +755,8 @@ static void timekeeping_forward_now(struct timekeeper *tk)
 	u64 cycle_now, delta;
 
 	cycle_now = tk_clock_read(&tk->tkr_mono);
-	delta = clocksource_delta(cycle_now, tk->tkr_mono.cycle_last, tk->tkr_mono.mask);
+	delta = clocksource_delta(cycle_now, tk->tkr_mono.cycle_last, tk->tkr_mono.mask,
+				  tk->tkr_mono.clock->max_raw_delta);
 	tk->tkr_mono.cycle_last = cycle_now;
 	tk->tkr_raw.cycle_last  = cycle_now;

@@ -2230,7 +2231,8 @@ static bool timekeeping_advance(enum timekeeping_adv_mode mode)
 		return false;
 
 	offset = clocksource_delta(tk_clock_read(&tk->tkr_mono),
-				   tk->tkr_mono.cycle_last, tk->tkr_mono.mask);
+				   tk->tkr_mono.cycle_last, tk->tkr_mono.mask,
+				   tk->tkr_mono.clock->max_raw_delta);
 
 	/* Check if there's really nothing to do */
 	if (offset < real_tk->cycle_interval && mode == TK_ADV_TICK)

kernel/time/timekeeping_internal.h +4 -4

index 63e600e943a7..8c9079108ffb 100644
@@ -30,15 +30,15 @@ static inline void timekeeping_inc_mg_floor_swaps(void)
 
 #endif
 
-static inline u64 clocksource_delta(u64 now, u64 last, u64 mask)
+static inline u64 clocksource_delta(u64 now, u64 last, u64 mask, u64 max_delta)
 {
 	u64 ret = (now - last) & mask;
 
 	/*
-	 * Prevent time going backwards by checking the MSB of mask in
-	 * the result. If set, return 0.
+	 * Prevent time going backwards by checking the result against
+	 * @max_delta. If greater, return 0.
 	 */
-	return ret & ~(mask >> 1) ? 0 : ret;
+	return ret > max_delta ? 0 : ret;
 }
 
 /* Semi public for serialization of non timekeeper VDSO updates. */

Thomas Gleixner

2024-12-12 13:58 UTC #1 lore raw

Quoted 7 lines · On Thu, Dec 12 2024 at 14:03, [email protected] wrote:

The patch below does not apply to the 6.12-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.
To reproduce the conflict and resubmit, you may use the following commands:

Quoted 9 lines

git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-6.12.y
git checkout FETCH_HEAD
git cherry-pick -x 76031d9536a076bf023bedbdb1b4317fc801dd67
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<[email protected]>' --in-reply-to '2024121203-griminess-blah-4e97@gregkh' --subject-prefix 'PATCH 6.12.y' HEAD^..
Possible dependencies:

There clearly is a dependency:

Quoted 4 lines

From 76031d9536a076bf023bedbdb1b4317fc801dd67 Mon Sep 17 00:00:00 2001
From: Thomas Gleixner <[email protected]>
Date: Tue, 3 Dec 2024 11:16:30 +0100
Subject: [PATCH] clocksource: Make negative motion detection more robust

<snip>

Quoted 1 line

Fixes c163e40af9b2 ("timekeeping: Always check for negative motion")

This was merged in the 6.13 merge window into Linus tree and not backported to 6.12.y according to my clone of the stable tree.

AI went sideways?

But I don't think these two commits are necessarily stable material, though I don't have a strong opinion on it. If c163e40af9b2 is backported, then it has it's own large dependency chain on pre 6.10 kernels...

Thanks,

tglx

Greg KH

author maintainer

2024-12-12 14:17 UTC #2 lore raw

Quoted 25 lines · Thomas Gleixner

Quoted 7 lines · On Thu, Dec 12 2024 at 14:03, [email protected] wrote:
The patch below does not apply to the 6.12-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.
To reproduce the conflict and resubmit, you may use the following commands:
Quoted 9 lines
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-6.12.y
git checkout FETCH_HEAD
git cherry-pick -x 76031d9536a076bf023bedbdb1b4317fc801dd67
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<[email protected]>' --in-reply-to '2024121203-griminess-blah-4e97@gregkh' --subject-prefix 'PATCH 6.12.y' HEAD^..
Possible dependencies:
There clearly is a dependency:
Quoted 4 lines
From 76031d9536a076bf023bedbdb1b4317fc801dd67 Mon Sep 17 00:00:00 2001
From: Thomas Gleixner <[email protected]>
Date: Tue, 3 Dec 2024 11:16:30 +0100
Subject: [PATCH] clocksource: Make negative motion detection more robust

Quoted 8 lines

<snip>
Quoted 1 line
Fixes c163e40af9b2 ("timekeeping: Always check for negative motion")
This was merged in the 6.13 merge window into Linus tree and not backported to 6.12.y according to my clone of the stable tree.
AI went sideways?

Nope, that commit is now in all of the stable queues, which is why I added this backport. Or attempted to.

Quoted 4 lines

But I don't think these two commits are necessarily stable material, though I don't have a strong opinion on it. If c163e40af9b2 is backported, then it has it's own large dependency chain on pre 6.10 kernels...

It's in the queues for some reason, let me figure out why...

Greg KH

author maintainer

2024-12-12 14:18 UTC #3 lore raw

Quoted 38 lines · Greg KH

Quoted 25 lines · Thomas Gleixner
Quoted 7 lines · On Thu, Dec 12 2024 at 14:03, [email protected] wrote:
The patch below does not apply to the 6.12-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.
To reproduce the conflict and resubmit, you may use the following commands:
Quoted 9 lines
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-6.12.y
git checkout FETCH_HEAD
git cherry-pick -x 76031d9536a076bf023bedbdb1b4317fc801dd67
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<[email protected]>' --in-reply-to '2024121203-griminess-blah-4e97@gregkh' --subject-prefix 'PATCH 6.12.y' HEAD^..
Possible dependencies:
There clearly is a dependency:
Quoted 4 lines
From 76031d9536a076bf023bedbdb1b4317fc801dd67 Mon Sep 17 00:00:00 2001
From: Thomas Gleixner <[email protected]>
Date: Tue, 3 Dec 2024 11:16:30 +0100
Subject: [PATCH] clocksource: Make negative motion detection more robust
Quoted 8 lines
<snip>
Quoted 1 line
Fixes c163e40af9b2 ("timekeeping: Always check for negative motion")
This was merged in the 6.13 merge window into Linus tree and not backported to 6.12.y according to my clone of the stable tree.
AI went sideways?
Nope, that commit is now in all of the stable queues, which is why I added this backport. Or attempted to.

Quoted 6 lines

Quoted 4 lines
But I don't think these two commits are necessarily stable material, though I don't have a strong opinion on it. If c163e40af9b2 is backported, then it has it's own large dependency chain on pre 6.10 kernels...
It's in the queues for some reason, let me figure out why...

Ah, it was an AUTOSEL thing, I'll go drop it from all queues except 6.12.y for now, thanks.

But, for 6.12.y, we want this fixup too, right?

greg k-h

Thomas Gleixner

2024-12-12 14:32 UTC #4 lore raw

Quoted 6 lines · Greg KH

Quoted 5 lines · Greg KH
Quoted 4 lines
But I don't think these two commits are necessarily stable material, though I don't have a strong opinion on it. If c163e40af9b2 is backported, then it has it's own large dependency chain on pre 6.10 kernels...

Quoted 6 lines

Quoted 1 line
It's in the queues for some reason, let me figure out why...
Ah, it was an AUTOSEL thing, I'll go drop it from all queues except 6.12.y for now, thanks.
But, for 6.12.y, we want this fixup too, right?

If you have c163e40af9b2 pulled back into 6.12.y, then yes. I don't know why this actually rejects. I just did

git-cherry-pick c163e40af9b2
git-cherry-pick 51f109e92935

on top of v6.12.4 and that just worked fine.

Thanks,

tglx

Greg KH

author maintainer

2024-12-12 14:35 UTC #5 lore raw

Quoted 14 lines · Thomas Gleixner

Quoted 6 lines · Greg KH
Quoted 5 lines · Greg KH
Quoted 4 lines
But I don't think these two commits are necessarily stable material, though I don't have a strong opinion on it. If c163e40af9b2 is backported, then it has it's own large dependency chain on pre 6.10 kernels...
Quoted 6 lines
Quoted 1 line
It's in the queues for some reason, let me figure out why...
Ah, it was an AUTOSEL thing, I'll go drop it from all queues except 6.12.y for now, thanks.
But, for 6.12.y, we want this fixup too, right?

Quoted 7 lines

If you have c163e40af9b2 pulled back into 6.12.y, then yes. I don't know why this actually rejects. I just did
git-cherry-pick c163e40af9b2
git-cherry-pick 51f109e92935
on top of v6.12.4 and that just worked fine.

The build breaks :(

Greg KH

author maintainer

2024-12-12 14:37 UTC #6 lore raw

Quoted 18 lines · Greg KH

Quoted 17 lines · Thomas Gleixner
Quoted 6 lines · Greg KH
Quoted 5 lines · Greg KH
Quoted 4 lines
But I don't think these two commits are necessarily stable material, though I don't have a strong opinion on it. If c163e40af9b2 is backported, then it has it's own large dependency chain on pre 6.10 kernels...
Quoted 6 lines
Quoted 1 line
It's in the queues for some reason, let me figure out why...
Ah, it was an AUTOSEL thing, I'll go drop it from all queues except 6.12.y for now, thanks.
But, for 6.12.y, we want this fixup too, right?
If you have c163e40af9b2 pulled back into 6.12.y, then yes. I don't know why this actually rejects. I just did

Quoted 6 lines

Quoted 4 lines
git-cherry-pick c163e40af9b2
git-cherry-pick 51f109e92935
on top of v6.12.4 and that just worked fine.
The build breaks :(

To be specific:

kernel/time/timekeeping.c: In function ‘timekeeping_debug_get_ns’:
kernel/time/timekeeping.c:263:17: error: too few arguments to function ‘clocksource_delta’

  263 |         delta = clocksource_delta(now, last, mask);
      |                 ^~~~~~~~~~~~~~~~~

In file included from kernel/time/timekeeping.c:30:

Thomas Gleixner

2024-12-12 15:02 UTC #7 lore raw

Quoted 26 lines · Greg KH

Quoted 18 lines · Greg KH
Quoted 17 lines · Thomas Gleixner
Quoted 6 lines · Greg KH
Quoted 5 lines · Greg KH
Quoted 4 lines
But I don't think these two commits are necessarily stable material, though I don't have a strong opinion on it. If c163e40af9b2 is backported, then it has it's own large dependency chain on pre 6.10 kernels...
Quoted 6 lines
Quoted 1 line
It's in the queues for some reason, let me figure out why...
Ah, it was an AUTOSEL thing, I'll go drop it from all queues except 6.12.y for now, thanks.
But, for 6.12.y, we want this fixup too, right?
If you have c163e40af9b2 pulled back into 6.12.y, then yes. I don't know why this actually rejects. I just did
Quoted 6 lines
Quoted 4 lines
git-cherry-pick c163e40af9b2
git-cherry-pick 51f109e92935
on top of v6.12.4 and that just worked fine.
The build breaks :(

Quoted 7 lines

To be specific:
kernel/time/timekeeping.c: In function ‘timekeeping_debug_get_ns’:
kernel/time/timekeeping.c:263:17: error: too few arguments to function ‘clocksource_delta’
  263 |         delta = clocksource_delta(now, last, mask);
      |                 ^~~~~~~~~~~~~~~~~
In file included from kernel/time/timekeeping.c:30:

Ah. You also need:

d44d26987bb3 ("timekeeping: Remove CONFIG_DEBUG_TIMEKEEPING")

which in turn does not apply cleanly and needs the backport below. *shrug*

Thanks,

tglx
---

arch/riscv/configs/defconfig +0 -1

index 2341393cfac1..26c01b9e3434 100644
@@ -301,7 +301,6 @@ CONFIG_DEBUG_MEMORY_INIT=y
 CONFIG_DEBUG_PER_CPU_MAPS=y
 CONFIG_SOFTLOCKUP_DETECTOR=y
 CONFIG_WQ_WATCHDOG=y
-CONFIG_DEBUG_TIMEKEEPING=y
 CONFIG_DEBUG_RT_MUTEXES=y
 CONFIG_DEBUG_SPINLOCK=y
 CONFIG_DEBUG_MUTEXES=y

include/linux/timekeeper_internal.h +0 ~~-15~~

index 902c20ef495a..715e0919972e 100644
@@ -68,9 +68,6 @@ struct tk_read_base {
  *			shifted nano seconds.
  * @ntp_error_shift:	Shift conversion between clock shifted nano seconds and
  *			ntp shifted nano seconds.
- * @last_warning:	Warning ratelimiter (DEBUG_TIMEKEEPING)
- * @underflow_seen:	Underflow warning flag (DEBUG_TIMEKEEPING)
- * @overflow_seen:	Overflow warning flag (DEBUG_TIMEKEEPING)
  *
  * Note: For timespec(64) based interfaces wall_to_monotonic is what
  * we need to add to xtime (or xtime corrected for sub jiffy times)

@@ -124,18 +121,6 @@ struct timekeeper {
 	u32			ntp_err_mult;
 	/* Flag used to avoid updating NTP twice with same second */
 	u32			skip_second_overflow;
-#ifdef CONFIG_DEBUG_TIMEKEEPING
-	long			last_warning;
-	/*
-	 * These simple flag variables are managed
-	 * without locks, which is racy, but they are
-	 * ok since we don't really care about being
-	 * super precise about how many events were
-	 * seen, just that a problem was observed.
-	 */
-	int			underflow_seen;
-	int			overflow_seen;
-#endif
 };
 
 #ifdef CONFIG_GENERIC_TIME_VSYSCALL

kernel/time/timekeeping.c +3 ~~-105~~

142 lines not on this page · raw message

lib/Kconfig.debug +0 ~~-13~~

index 7312ae7c3cc5..3f9c238bb58e 100644
@@ -1328,19 +1328,6 @@ config SCHEDSTATS
 
 endmenu
 
-config DEBUG_TIMEKEEPING
-	bool "Enable extra timekeeping sanity checking"
-	help
-	  This option will enable additional timekeeping sanity checks
-	  which may be helpful when diagnosing issues where timekeeping
-	  problems are suspected.
-
-	  This may include checks in the timekeeping hotpaths, so this
-	  option may have a (very small) performance impact to some
-	  workloads.
-
-	  If unsure, say N.
-
 config DEBUG_PREEMPT
 	bool "Debug preemptible kernel"
 	depends on DEBUG_KERNEL && PREEMPTION && TRACE_IRQFLAGS_SUPPORT

tools/testing/selftests/wireguard/qemu/debug.config +0 -1

index 9d172210e2c6..139fd9aa8b12 100644
@@ -31,7 +31,6 @@ CONFIG_SCHED_DEBUG=y
 CONFIG_SCHED_INFO=y
 CONFIG_SCHEDSTATS=y
 CONFIG_SCHED_STACK_END_CHECK=y
-CONFIG_DEBUG_TIMEKEEPING=y
 CONFIG_DEBUG_PREEMPT=y
 CONFIG_DEBUG_RT_MUTEXES=y
 CONFIG_DEBUG_SPINLOCK=y

Sasha Levin

maintainer

2024-12-12 17:57 UTC #8 lore raw

[ Sasha's backport helper bot ]

Hi,

Found matching upstream commit: c25ca0c2e42c77e0241411d374d44c41e253b3f5

WARNING: Author mismatch between patch and found commit:
Backport author: Thomas Gleixner <[email protected]> Commit author: Linus Torvalds <[email protected]>

Status in newer kernel trees:
6.12.y | Not found

Note: The patch differs from the upstream commit:
---

1:  76031d9536a07 < -:  ------------- clocksource: Make negative motion detection more robust
-:  ------------- > 1:  58afb84e26cfa FAILED: patch "[PATCH] clocksource: Make negative motion detection more robust" failed to apply to 6.12-stable tree

---

Results of testing on various branches:

| Branch                    | Patch Apply | Build Test |
|---------------------------|-------------|------------|
| stable/linux-6.12.y       |  Success    |  Success   |

Greg KH

author maintainer

2024-12-13 11:33 UTC #9 lore raw

In reply to Thomas Gleixner

Quoted 35 lines · Thomas Gleixner

Quoted 26 lines · Greg KH
Quoted 18 lines · Greg KH
Quoted 17 lines · Thomas Gleixner
Quoted 6 lines · Greg KH
Quoted 5 lines · Greg KH
Quoted 4 lines
But I don't think these two commits are necessarily stable material, though I don't have a strong opinion on it. If c163e40af9b2 is backported, then it has it's own large dependency chain on pre 6.10 kernels...
Quoted 6 lines
Quoted 1 line
It's in the queues for some reason, let me figure out why...
Ah, it was an AUTOSEL thing, I'll go drop it from all queues except 6.12.y for now, thanks.
But, for 6.12.y, we want this fixup too, right?
If you have c163e40af9b2 pulled back into 6.12.y, then yes. I don't know why this actually rejects. I just did
Quoted 6 lines
Quoted 4 lines
git-cherry-pick c163e40af9b2
git-cherry-pick 51f109e92935
on top of v6.12.4 and that just worked fine.
The build breaks :(
Quoted 7 lines
To be specific:
kernel/time/timekeeping.c: In function ‘timekeeping_debug_get_ns’:
kernel/time/timekeeping.c:263:17: error: too few arguments to function ‘clocksource_delta’
  263 |         delta = clocksource_delta(now, last, mask);
      |                 ^~~~~~~~~~~~~~~~~
In file included from kernel/time/timekeeping.c:30:

Quoted 6 lines

Ah. You also need:
d44d26987bb3 ("timekeeping: Remove CONFIG_DEBUG_TIMEKEEPING")
which in turn does not apply cleanly and needs the backport below. *shrug*

Wonderful, thanks for this, that worked I'll go push out a -rc2 with this soon.

greg k-h