Skip to contentSkip to conversation

Patch "kernel/watchdog: Prevent false positives with turbo modes" has been added to the 4.9-stable tree

PATCH [email protected] 1 participant 2 messages Aug 20, 2017 – Aug 21, 2017

Subsystems (3)
lore
Share & tools
lore

[email protected]

author maintainer Applied
#0 lore raw

This is a note to let you know that I've just added the patch titled

kernel/watchdog: Prevent false positives with turbo modes

to the 4.9-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
kernel-watchdog-prevent-false-positives-with-turbo-modes.patch
and it can be found in the queue-4.9 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 1 line

From 7edaeb6841dfb27e362288ab8466ebdc4972e867 Mon Sep 17 00:00:00 2001

From: Thomas Gleixner <[email protected]>
Date: Tue, 15 Aug 2017 09:50:13 +0200
Subject: kernel/watchdog: Prevent false positives with turbo modes

From: Thomas Gleixner <[email protected]>

commit 7edaeb6841dfb27e362288ab8466ebdc4972e867 upstream.

The hardlockup detector on x86 uses a performance counter based on unhalted CPU cycles and a periodic hrtimer. The hrtimer period is about 2/5 of the performance counter period, so the hrtimer should fire 2-3 times before the performance counter NMI fires. The NMI code checks whether the hrtimer fired since the last invocation. If not, it assumess a hard lockup.

The calculation of those periods is based on the nominal CPU frequency. Turbo modes increase the CPU clock frequency and therefore shorten the period of the perf/NMI watchdog. With extreme Turbo-modes (3x nominal frequency) the perf/NMI period is shorter than the hrtimer period which leads to false positives.

A simple fix would be to shorten the hrtimer period, but that comes with the side effect of more frequent hrtimer and softlockup thread wakeups, which is not desired.

Implement a low pass filter, which checks the perf/NMI period against kernel time. If the perf/NMI fires before 4/5 of the watchdog period has elapsed then the event is ignored and postponed to the next perf/NMI.

That solves the problem and avoids the overhead of shorter hrtimer periods and more frequent softlockup thread wakeups.

---
 arch/x86/Kconfig      |    1 
 include/linux/nmi.h   |    8 ++++++
 kernel/watchdog.c     |    1 
 kernel/watchdog_hld.c |   59 ++++++++++++++++++++++++++++++++++++++++++++++++++
 lib/Kconfig.debug     |    7 +++++
 5 files changed, 76 insertions(+)
arch/x86/Kconfig +1 -0
@@ -74,6 +74,7 @@ config X86
 	select GENERIC_STRNCPY_FROM_USER
 	select GENERIC_STRNLEN_USER
 	select GENERIC_TIME_VSYSCALL
+	select HARDLOCKUP_CHECK_TIMESTAMP	if X86_64
 	select HAVE_ACPI_APEI			if ACPI
 	select HAVE_ACPI_APEI_NMI		if ACPI
 	select HAVE_ALIGNED_STRUCT_PAGE		if SLUB
include/linux/nmi.h +8 -0
@@ -118,6 +118,14 @@ extern int sysctl_hardlockup_all_cpu_bac
 #define sysctl_softlockup_all_cpu_backtrace 0
 #define sysctl_hardlockup_all_cpu_backtrace 0
 #endif
+
+#if defined(CONFIG_HARDLOCKUP_CHECK_TIMESTAMP) && \
+    defined(CONFIG_HARDLOCKUP_DETECTOR)
+void watchdog_update_hrtimer_threshold(u64 period);
+#else
+static inline void watchdog_update_hrtimer_threshold(u64 period) { }
+#endif
+
 extern bool is_hardlockup(void);
 struct ctl_table;
 extern int proc_watchdog(struct ctl_table *, int ,
kernel/watchdog.c +1 -0
@@ -158,6 +158,7 @@ static void set_sample_period(void)
 	 * hardlockup detector generates a warning
 	 */
 	sample_period = get_softlockup_thresh() * ((u64)NSEC_PER_SEC / 5);
+	watchdog_update_hrtimer_threshold(sample_period);
 }
 
 /* Commands for resetting the watchdog */
kernel/watchdog_hld.c +59 -0

71 lines not on this page · raw message

lib/Kconfig.debug +7 -0
@@ -325,6 +325,13 @@ config SECTION_MISMATCH_WARN_ONLY
 	  If unsure, say Y.
 
 #
+# Enables a timestamp based low pass filter to compensate for perf based
+# hard lockup detection which runs too fast due to turbo modes.
+#
+config HARDLOCKUP_CHECK_TIMESTAMP
+	bool
+
+#
 # Select this config option from the architecture Kconfig, if it
 # is preferred to always offer frame pointers as a config
 # option on the architecture (regardless of KERNEL_DEBUG):

Patches currently in stable-queue which might be from [email protected] are

queue-4.9/x86-asm-64-clear-ac-on-nmi-entries.patch queue-4.9/genirq-ipi-fixup-checks-against-nr_cpu_ids.patch queue-4.9/genirq-restore-trigger-settings-in-irq_modify_status.patch queue-4.9/kernel-watchdog-prevent-false-positives-with-turbo-modes.patch queue-4.9/mm-revert-x86_64-and-arm64-elf_et_dyn_base-base-changes.patch

author maintainer
#1 lore raw
Quoted 11 lines · On Sun, Aug 20, 2017 at 12:04:41PM -0700, [email protected] wrote:

This is a note to let you know that I've just added the patch titled

kernel/watchdog: Prevent false positives with turbo modes

to the 4.9-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
kernel-watchdog-prevent-false-positives-with-turbo-modes.patch
and it can be found in the queue-4.9 subdirectory.

Nope, this broke the build so I'm dropping it there. Can someone provide a working version?

thanks,

greg k-h