Skip to contentSkip to conversation

watchdog: Allow nmi watchdog to use "ref-cycles" event

PATCH In review Song Liu 4 participants 9 messages May 18, 2023 – Jun 8, 2023 Documentation

lore
Share & tools
lore

Song Liu

author
#0 lore raw

PATCH v4 1/1 watchdog: Allow nmi watchdog to use "ref-cycles" event

NMI watchdog permanently consumes one hardware counters per CPU on the system. For systems that use many hardware counters, this causes more aggressive time multiplexing of perf events.

OTOH, some CPUs (mostly Intel) support "ref-cycles" event, which is rarely used. Add kernel cmdline arg nmi_watchdog=ref-cycles to configure the watchdog to use "ref-cycles" event instead of "cycles".

Cc Andrew Morton <[email protected]>
Cc Peter Zijlstra <[email protected]>

---
Changes in v4:
Fix compile error for !CONFIG_HARDLOCKUP_DETECTOR_PERF. (kernel test bot)

Changes in v3:

Pivot the design to use kernel arg nmi_watchdog=ref-cycles (Peter)

---
 Documentation/admin-guide/kernel-parameters.txt | 5 +++--
 include/linux/nmi.h                             | 2 ++
 kernel/watchdog.c                               | 2 ++
 kernel/watchdog_hld.c                           | 9 +++++++++
 4 files changed, 16 insertions(+), 2 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +3 -2
index 9e5bab29685f..d378e23dad7c 100644
@@ -3593,10 +3593,12 @@
 			Format: [state][,regs][,debounce][,die]
 
 	nmi_watchdog=	[KNL,BUGS=X86] Debugging features for SMP kernels
-			Format: [panic,][nopanic,][num]
+			Format: [panic,][nopanic,][ref-cycles][num]
 			Valid num: 0 or 1
 			0 - turn hardlockup detector in nmi_watchdog off
 			1 - turn hardlockup detector in nmi_watchdog on
+			ref-cycles - configure the watchdog with perf event
+			             "ref-cycles" instead of "cycles"
 			When panic is specified, panic when an NMI watchdog
 			timeout occurs (or 'nopanic' to not panic on an NMI
 			watchdog, if CONFIG_BOOTPARAM_HARDLOCKUP_PANIC is set)
@@ -7097,4 +7099,3 @@
 				memory, and other data can't be written using
 				xmon commands.
 			off	xmon is disabled.
-
include/linux/nmi.h +2 -0
index 048c0b9aa623..edfd1bcce0f6 100644
@@ -102,12 +102,14 @@ extern void hardlockup_detector_perf_disable(void);
 extern void hardlockup_detector_perf_enable(void);
 extern void hardlockup_detector_perf_cleanup(void);
 extern int hardlockup_detector_perf_init(void);
+extern void hardlockup_config_perf_event(const char *str);
 #else
 static inline void hardlockup_detector_perf_stop(void) { }
 static inline void hardlockup_detector_perf_restart(void) { }
 static inline void hardlockup_detector_perf_disable(void) { }
 static inline void hardlockup_detector_perf_enable(void) { }
 static inline void hardlockup_detector_perf_cleanup(void) { }
+static inline void hardlockup_config_perf_event(const char *str) { }
 # if !defined(CONFIG_HAVE_NMI_WATCHDOG)
 static inline int hardlockup_detector_perf_init(void) { return -ENODEV; }
 static inline void arch_touch_nmi_watchdog(void) {}
kernel/watchdog.c +2 -0
index 8e61f21e7e33..fed4f0be8e1a 100644
@@ -81,6 +81,8 @@ static int __init hardlockup_panic_setup(char *str)
 		nmi_watchdog_user_enabled = 0;
 	else if (!strncmp(str, "1", 1))
 		nmi_watchdog_user_enabled = 1;
+	else if (!strncmp(str, "ref-cycles", 10))
+		hardlockup_config_perf_event(str);
 	return 1;
 }
 __setup("nmi_watchdog=", hardlockup_panic_setup);
kernel/watchdog_hld.c +9 -0
index 247bf0b1582c..4deca58ba6ed 100644
@@ -294,3 +294,12 @@ int __init hardlockup_detector_perf_init(void)
 	}
 	return ret;
 }
+
+/**
+ * hardlockup_config_perf_event - Overwrite config of wd_hw_attr
+ */
+void __init hardlockup_config_perf_event(const char *str)
+{
+	if (!strncmp(str, "ref-cycles", 10))
+		wd_hw_attr.config = PERF_COUNT_HW_REF_CPU_CYCLES;
+}
Signature · 1 line
2.34.1
#1 lore raw
Quoted 7 lines · Song Liu

NMI watchdog permanently consumes one hardware counters per CPU on the system. For systems that use many hardware counters, this causes more aggressive time multiplexing of perf events.

OTOH, some CPUs (mostly Intel) support "ref-cycles" event, which is rarely used. Add kernel cmdline arg nmi_watchdog=ref-cycles to configure the watchdog to use "ref-cycles" event instead of "cycles".

Maybe list some example how this new option will used?

Quoted 66 lines
Cc Andrew Morton <[email protected]>
Cc Peter Zijlstra <[email protected]>

---
Changes in v4:
Fix compile error for !CONFIG_HARDLOCKUP_DETECTOR_PERF. (kernel test bot)

Changes in v3:

Pivot the design to use kernel arg nmi_watchdog=ref-cycles (Peter)

---
  Documentation/admin-guide/kernel-parameters.txt | 5 +++--
  include/linux/nmi.h                             | 2 ++
  kernel/watchdog.c                               | 2 ++
  kernel/watchdog_hld.c                           | 9 +++++++++
  4 files changed, 16 insertions(+), 2 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +3 -2
index 9e5bab29685f..d378e23dad7c 100644
@@ -3593,10 +3593,12 @@
  			Format: [state][,regs][,debounce][,die]
 
  	nmi_watchdog=	[KNL,BUGS=X86] Debugging features for SMP kernels
-			Format: [panic,][nopanic,][num]
+			Format: [panic,][nopanic,][ref-cycles][num]
  			Valid num: 0 or 1
  			0 - turn hardlockup detector in nmi_watchdog off
  			1 - turn hardlockup detector in nmi_watchdog on
+			ref-cycles - configure the watchdog with perf event
+			             "ref-cycles" instead of "cycles"
  			When panic is specified, panic when an NMI watchdog
  			timeout occurs (or 'nopanic' to not panic on an NMI
  			watchdog, if CONFIG_BOOTPARAM_HARDLOCKUP_PANIC is set)
@@ -7097,4 +7099,3 @@
  				memory, and other data can't be written using
  				xmon commands.
  			off	xmon is disabled.
-
include/linux/nmi.h +2 -0
index 048c0b9aa623..edfd1bcce0f6 100644
@@ -102,12 +102,14 @@ extern void hardlockup_detector_perf_disable(void);
  extern void hardlockup_detector_perf_enable(void);
  extern void hardlockup_detector_perf_cleanup(void);
  extern int hardlockup_detector_perf_init(void);
+extern void hardlockup_config_perf_event(const char *str);
  #else
  static inline void hardlockup_detector_perf_stop(void) { }
  static inline void hardlockup_detector_perf_restart(void) { }
  static inline void hardlockup_detector_perf_disable(void) { }
  static inline void hardlockup_detector_perf_enable(void) { }
  static inline void hardlockup_detector_perf_cleanup(void) { }
+static inline void hardlockup_config_perf_event(const char *str) { }
  # if !defined(CONFIG_HAVE_NMI_WATCHDOG)
  static inline int hardlockup_detector_perf_init(void) { return -ENODEV; }
  static inline void arch_touch_nmi_watchdog(void) {}
kernel/watchdog.c +0 -0
index 8e61f21e7e33..fed4f0be8e1a 100644

kernel/watchdog.c static int __init hardlockup_panic_setup(char *str) · line 84

Quoted 4 lines
kernel/watchdog.c +1 -0
@@ -81,6 +81,8 @@ static int __init hardlockup_panic_setup(char *str)
  		nmi_watchdog_user_enabled = 0;
  	else if (!strncmp(str, "1", 1))
  		nmi_watchdog_user_enabled = 1;
+	else if (!strncmp(str, "ref-cycles", 10))

str vs. 'ref-cycles' is tested here.

Quoted 9 lines
kernel/watchdog.c +1 -0
+		hardlockup_config_perf_event(str);
  	return 1;
  }
  __setup("nmi_watchdog=", hardlockup_panic_setup);
kernel/watchdog_hld.c +0 -0
index 247bf0b1582c..4deca58ba6ed 100644

kernel/watchdog_hld.c int __init hardlockup_detector_perf_init(void) · line 303

Quoted 10 lines
kernel/watchdog_hld.c +7 -0
@@ -294,3 +294,12 @@ int __init hardlockup_detector_perf_init(void)
  	}
  	return ret;
  }
+
+/**
+ * hardlockup_config_perf_event - Overwrite config of wd_hw_attr
+ */
+void __init hardlockup_config_perf_event(const char *str)
+{
+	if (!strncmp(str, "ref-cycles", 10))

It is unnecessarily tested again here.

Quoted 2 lines
kernel/watchdog_hld.c +2 -0
+		wd_hw_attr.config = PERF_COUNT_HW_REF_CPU_CYCLES;
+}
#2 lore raw
Quoted 7 lines · Yonghong Song

On 5/17/23 5:25 PM, Song Liu wrote:

Quoted 8 lines
Quoted 6 lines

NMI watchdog permanently consumes one hardware counters per CPU on the system. For systems that use many hardware counters, this causes more aggressive time multiplexing of perf events.
OTOH, some CPUs (mostly Intel) support "ref-cycles" event, which is rarely used. Add kernel cmdline arg nmi_watchdog=ref-cycles to configure the watchdog to use "ref-cycles" event instead of "cycles".

Maybe list some example how this new option will used?

In most case, all we need is to add "nmi_watchdog=refcycles" to kernel args.

Quoted 81 lines
Quoted 61 lines
Cc Andrew Morton <[email protected]>
Cc Peter Zijlstra <[email protected]>

---
Changes in v4:
Fix compile error for !CONFIG_HARDLOCKUP_DETECTOR_PERF. (kernel test bot) Changes in v3:
Pivot the design to use kernel arg nmi_watchdog=ref-cycles (Peter)

---
 Documentation/admin-guide/kernel-parameters.txt | 5 +++--
 include/linux/nmi.h                             | 2 ++
 kernel/watchdog.c                               | 2 ++
 kernel/watchdog_hld.c                           | 9 +++++++++
 4 files changed, 16 insertions(+), 2 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +3 -2
index 9e5bab29685f..d378e23dad7c 100644
@@ -3593,10 +3593,12 @@
  Format: [state][,regs][,debounce][,die]
    nmi_watchdog= [KNL,BUGS=X86] Debugging features for SMP kernels
- Format: [panic,][nopanic,][num]
+ Format: [panic,][nopanic,][ref-cycles][num]
  Valid num: 0 or 1
  0 - turn hardlockup detector in nmi_watchdog off
  1 - turn hardlockup detector in nmi_watchdog on
+ ref-cycles - configure the watchdog with perf event
+              "ref-cycles" instead of "cycles"
  When panic is specified, panic when an NMI watchdog
  timeout occurs (or 'nopanic' to not panic on an NMI
  watchdog, if CONFIG_BOOTPARAM_HARDLOCKUP_PANIC is set)
@@ -7097,4 +7099,3 @@
  memory, and other data can't be written using
  xmon commands.
  off xmon is disabled.
-
include/linux/nmi.h +2 -0
index 048c0b9aa623..edfd1bcce0f6 100644
@@ -102,12 +102,14 @@ extern void hardlockup_detector_perf_disable(void);
 extern void hardlockup_detector_perf_enable(void);
 extern void hardlockup_detector_perf_cleanup(void);
 extern int hardlockup_detector_perf_init(void);
+extern void hardlockup_config_perf_event(const char *str);
 #else
 static inline void hardlockup_detector_perf_stop(void) { }
 static inline void hardlockup_detector_perf_restart(void) { }
 static inline void hardlockup_detector_perf_disable(void) { }
 static inline void hardlockup_detector_perf_enable(void) { }
 static inline void hardlockup_detector_perf_cleanup(void) { }
+static inline void hardlockup_config_perf_event(const char *str) { }
 # if !defined(CONFIG_HAVE_NMI_WATCHDOG)
 static inline int hardlockup_detector_perf_init(void) { return -ENODEV; }
 static inline void arch_touch_nmi_watchdog(void) {}
kernel/watchdog.c +0 -0
index 8e61f21e7e33..fed4f0be8e1a 100644

kernel/watchdog.c static int __init hardlockup_panic_setup(char *str) · line 84

Quoted 4 lines
kernel/watchdog.c +1 -0
@@ -81,6 +81,8 @@ static int __init hardlockup_panic_setup(char *str)
  nmi_watchdog_user_enabled = 0;
  else if (!strncmp(str, "1", 1))
  nmi_watchdog_user_enabled = 1;
+ else if (!strncmp(str, "ref-cycles", 10))

str vs. 'ref-cycles' is tested here.

Quoted 13 lines
kernel/watchdog.c +1 -0
+ hardlockup_config_perf_event(str);
  return 1;
 }
 __setup("nmi_watchdog=", hardlockup_panic_setup);
kernel/watchdog_hld.c +1 -0
index 247bf0b1582c..4deca58ba6ed 100644
@@ -294,3 +294,12 @@ int __init hardlockup_detector_perf_init(void)
  }
  return ret;
 }
+
Quoted 8 lines

kernel/watchdog_hld.c int __init hardlockup_detector_perf_init(void) · line 303

Quoted 6 lines
kernel/watchdog_hld.c +6 -0
+/**
+ * hardlockup_config_perf_event - Overwrite config of wd_hw_attr
+ */
+void __init hardlockup_config_perf_event(const char *str)
+{
+ if (!strncmp(str, "ref-cycles", 10))

It is unnecessarily tested again here.

In that case, we can probably rename the function as hardlockup_use_ref_cycles(). I am fine with either way.

Thanks,
Song

#3 lore raw
In reply to Song Liu

Hi Andrew and Peter,

Does this version look good do you?

Thanks,
Song

Quoted 103 lines · Song Liu

NMI watchdog permanently consumes one hardware counters per CPU on the system. For systems that use many hardware counters, this causes more aggressive time multiplexing of perf events.

OTOH, some CPUs (mostly Intel) support "ref-cycles" event, which is rarely used. Add kernel cmdline arg nmi_watchdog=ref-cycles to configure the watchdog to use "ref-cycles" event instead of "cycles".

Cc Andrew Morton <[email protected]>
Cc Peter Zijlstra <[email protected]>

---
Changes in v4:
Fix compile error for !CONFIG_HARDLOCKUP_DETECTOR_PERF. (kernel test bot)

Changes in v3:

Pivot the design to use kernel arg nmi_watchdog=ref-cycles (Peter)
---

Documentation/admin-guide/kernel-parameters.txt | 5 +++--
include/linux/nmi.h                             | 2 ++
kernel/watchdog.c                               | 2 ++
kernel/watchdog_hld.c                           | 9 +++++++++
4 files changed, 16 insertions(+), 2 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +0 -0
index 9e5bab29685f..d378e23dad7c 100644
@@ -3593,10 +3593,12 @@

Format: [state][,regs][,debounce][,die]

nmi_watchdog= [KNL,BUGS=X86] Debugging features for SMP kernels
- Format: [panic,][nopanic,][num]
+ Format: [panic,][nopanic,][ref-cycles][num]
Valid num: 0 or 1
0 - turn hardlockup detector in nmi_watchdog off
1 - turn hardlockup detector in nmi_watchdog on
+ ref-cycles - configure the watchdog with perf event
+ "ref-cycles" instead of "cycles"
When panic is specified, panic when an NMI watchdog timeout occurs (or 'nopanic' to not panic on an NMI watchdog, if CONFIG_BOOTPARAM_HARDLOCKUP_PANIC is set)

Documentation/admin-guide/kernel-parameters.txt +0 -0
@@ -7097,4 +7099,3 @@

memory, and other data can't be written using
xmon commands.
off xmon is disabled.

Documentation/admin-guide/kernel-parameters.txt +0 -1
-
include/linux/nmi.h +2 -0
index 048c0b9aa623..edfd1bcce0f6 100644
@@ -102,12 +102,14 @@ extern void hardlockup_detector_perf_disable(void);
 extern void hardlockup_detector_perf_enable(void);
 extern void hardlockup_detector_perf_cleanup(void);
 extern int hardlockup_detector_perf_init(void);
+extern void hardlockup_config_perf_event(const char *str);
 #else
 static inline void hardlockup_detector_perf_stop(void) { }
 static inline void hardlockup_detector_perf_restart(void) { }
 static inline void hardlockup_detector_perf_disable(void) { }
 static inline void hardlockup_detector_perf_enable(void) { }
 static inline void hardlockup_detector_perf_cleanup(void) { }
+static inline void hardlockup_config_perf_event(const char *str) { }
 # if !defined(CONFIG_HAVE_NMI_WATCHDOG)
 static inline int hardlockup_detector_perf_init(void) { return -ENODEV; }
 static inline void arch_touch_nmi_watchdog(void) {}
kernel/watchdog.c +2 -0
index 8e61f21e7e33..fed4f0be8e1a 100644
@@ -81,6 +81,8 @@ static int __init hardlockup_panic_setup(char *str)
 nmi_watchdog_user_enabled = 0;
 else if (!strncmp(str, "1", 1))
 nmi_watchdog_user_enabled = 1;
+ else if (!strncmp(str, "ref-cycles", 10))
+ hardlockup_config_perf_event(str);
 return 1;
 }
 __setup("nmi_watchdog=", hardlockup_panic_setup);
kernel/watchdog_hld.c +9 -0
index 247bf0b1582c..4deca58ba6ed 100644
@@ -294,3 +294,12 @@ int __init hardlockup_detector_perf_init(void)
 }
 return ret;
 }
+
+/**
+ * hardlockup_config_perf_event - Overwrite config of wd_hw_attr
+ */
+void __init hardlockup_config_perf_event(const char *str)
+{
+ if (!strncmp(str, "ref-cycles", 10))
+ wd_hw_attr.config = PERF_COUNT_HW_REF_CPU_CYCLES;
+}
Signature · 1 line
2.34.1
#4 lore raw

Hi Andrew and Peter,

Quoted 22 lines · Song Liu

Hi Andrew and Peter,

Does this version look good do you?

Thanks,
Song

Quoted 13 lines · Song Liu

NMI watchdog permanently consumes one hardware counters per CPU on the system. For systems that use many hardware counters, this causes more aggressive time multiplexing of perf events.

OTOH, some CPUs (mostly Intel) support "ref-cycles" event, which is rarely used. Add kernel cmdline arg nmi_watchdog=ref-cycles to configure the watchdog to use "ref-cycles" event instead of "cycles".

Cc Andrew Morton <[email protected]>
Cc Peter Zijlstra <[email protected]>

Could you please share your comments on this patch?

Thanks,
Song

Quoted 91 lines
Quoted 90 lines

---
Changes in v4:
Fix compile error for !CONFIG_HARDLOCKUP_DETECTOR_PERF. (kernel test bot)

Changes in v3:

Pivot the design to use kernel arg nmi_watchdog=ref-cycles (Peter)
---

Documentation/admin-guide/kernel-parameters.txt | 5 +++--
include/linux/nmi.h                             | 2 ++
kernel/watchdog.c                               | 2 ++
kernel/watchdog_hld.c                           | 9 +++++++++
4 files changed, 16 insertions(+), 2 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +0 -0
index 9e5bab29685f..d378e23dad7c 100644
@@ -3593,10 +3593,12 @@

Format: [state][,regs][,debounce][,die]

nmi_watchdog= [KNL,BUGS=X86] Debugging features for SMP kernels
- Format: [panic,][nopanic,][num]
+ Format: [panic,][nopanic,][ref-cycles][num]
Valid num: 0 or 1
0 - turn hardlockup detector in nmi_watchdog off
1 - turn hardlockup detector in nmi_watchdog on
+ ref-cycles - configure the watchdog with perf event
+ "ref-cycles" instead of "cycles"
When panic is specified, panic when an NMI watchdog timeout occurs (or 'nopanic' to not panic on an NMI watchdog, if CONFIG_BOOTPARAM_HARDLOCKUP_PANIC is set)

Documentation/admin-guide/kernel-parameters.txt +0 -0
@@ -7097,4 +7099,3 @@

memory, and other data can't be written using
xmon commands.
off xmon is disabled.

Documentation/admin-guide/kernel-parameters.txt +0 -1
-
include/linux/nmi.h +2 -0
index 048c0b9aa623..edfd1bcce0f6 100644
@@ -102,12 +102,14 @@ extern void hardlockup_detector_perf_disable(void);
 extern void hardlockup_detector_perf_enable(void);
 extern void hardlockup_detector_perf_cleanup(void);
 extern int hardlockup_detector_perf_init(void);
+extern void hardlockup_config_perf_event(const char *str);
 #else
 static inline void hardlockup_detector_perf_stop(void) { }
 static inline void hardlockup_detector_perf_restart(void) { }
 static inline void hardlockup_detector_perf_disable(void) { }
 static inline void hardlockup_detector_perf_enable(void) { }
 static inline void hardlockup_detector_perf_cleanup(void) { }
+static inline void hardlockup_config_perf_event(const char *str) { }
 # if !defined(CONFIG_HAVE_NMI_WATCHDOG)
 static inline int hardlockup_detector_perf_init(void) { return -ENODEV; }
 static inline void arch_touch_nmi_watchdog(void) {}
kernel/watchdog.c +2 -0
index 8e61f21e7e33..fed4f0be8e1a 100644
@@ -81,6 +81,8 @@ static int __init hardlockup_panic_setup(char *str)
 nmi_watchdog_user_enabled = 0;
 else if (!strncmp(str, "1", 1))
 nmi_watchdog_user_enabled = 1;
+ else if (!strncmp(str, "ref-cycles", 10))
+ hardlockup_config_perf_event(str);
 return 1;
 }
 __setup("nmi_watchdog=", hardlockup_panic_setup);
kernel/watchdog_hld.c +9 -0
index 247bf0b1582c..4deca58ba6ed 100644
@@ -294,3 +294,12 @@ int __init hardlockup_detector_perf_init(void)
 }
 return ret;
 }
+
+/**
+ * hardlockup_config_perf_event - Overwrite config of wd_hw_attr
+ */
+void __init hardlockup_config_perf_event(const char *str)
+{
+ if (!strncmp(str, "ref-cycles", 10))
+ wd_hw_attr.config = PERF_COUNT_HW_REF_CPU_CYCLES;
+}
Signature · 1 line
2.34.1
#5 lore raw

Hi Andrew and Peter,

Friendly ping... Any comment on this one?

Thanks,
Song

Quoted 32 lines · Song Liu

Hi Andrew and Peter,

Quoted 22 lines · Song Liu

Hi Andrew and Peter,

Does this version look good do you?

Thanks,
Song

Quoted 13 lines · Song Liu

NMI watchdog permanently consumes one hardware counters per CPU on the system. For systems that use many hardware counters, this causes more aggressive time multiplexing of perf events.

OTOH, some CPUs (mostly Intel) support "ref-cycles" event, which is rarely used. Add kernel cmdline arg nmi_watchdog=ref-cycles to configure the watchdog to use "ref-cycles" event instead of "cycles".

Cc Andrew Morton <[email protected]>
Cc Peter Zijlstra <[email protected]>

Could you please share your comments on this patch?

Thanks,
Song

#6 lore raw
In reply to Song Liu
Quoted 34 lines · Song Liu

NMI watchdog permanently consumes one hardware counters per CPU on the system. For systems that use many hardware counters, this causes more aggressive time multiplexing of perf events.

OTOH, some CPUs (mostly Intel) support "ref-cycles" event, which is rarely used. Add kernel cmdline arg nmi_watchdog=ref-cycles to configure the watchdog to use "ref-cycles" event instead of "cycles".

Cc Andrew Morton <[email protected]>
Cc Peter Zijlstra <[email protected]>

---
Changes in v4:
Fix compile error for !CONFIG_HARDLOCKUP_DETECTOR_PERF. (kernel test bot)

Changes in v3:

Pivot the design to use kernel arg nmi_watchdog=ref-cycles (Peter)

---
 Documentation/admin-guide/kernel-parameters.txt | 5 +++--
 include/linux/nmi.h                             | 2 ++
 kernel/watchdog.c                               | 2 ++
 kernel/watchdog_hld.c                           | 9 +++++++++
 4 files changed, 16 insertions(+), 2 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +0 -0
index 9e5bab29685f..d378e23dad7c 100644
@@ -3593,10 +3593,12 @@
 			Format: [state][,regs][,debounce][,die]
 
 	nmi_watchdog=	[KNL,BUGS=X86] Debugging features for SMP kernels

Documentation/admin-guide/kernel-parameters.txt line 3604

Quoted 10 lines
Documentation/admin-guide/kernel-parameters.txt +3 -1
-			Format: [panic,][nopanic,][num]
+			Format: [panic,][nopanic,][ref-cycles][num]
 			Valid num: 0 or 1
 			0 - turn hardlockup detector in nmi_watchdog off
 			1 - turn hardlockup detector in nmi_watchdog on
+			ref-cycles - configure the watchdog with perf event
+			             "ref-cycles" instead of "cycles"
 			When panic is specified, panic when an NMI watchdog
 			timeout occurs (or 'nopanic' to not panic on an NMI
 			watchdog, if CONFIG_BOOTPARAM_HARDLOCKUP_PANIC is set)

I still hate the whole ref-cycles thing, at the very least powerpc also has HAVE_HARDLOCKUP_DETECTOR_PERF and they don't have ref-cycles, but perhaps them wants to use a different event when the moon is just so...

What again was wrong with the option of specifying a raw event value and falling back to cpu-cycles if that fails?

#7 lore raw
Quoted 54 lines · Peter Zijlstra
Quoted 33 lines · Song Liu

NMI watchdog permanently consumes one hardware counters per CPU on the system. For systems that use many hardware counters, this causes more aggressive time multiplexing of perf events.

OTOH, some CPUs (mostly Intel) support "ref-cycles" event, which is rarely used. Add kernel cmdline arg nmi_watchdog=ref-cycles to configure the watchdog to use "ref-cycles" event instead of "cycles".

Cc Andrew Morton <[email protected]>
Cc Peter Zijlstra <[email protected]>

---
Changes in v4:
Fix compile error for !CONFIG_HARDLOCKUP_DETECTOR_PERF. (kernel test bot)

Changes in v3:

Pivot the design to use kernel arg nmi_watchdog=ref-cycles (Peter)
---

Documentation/admin-guide/kernel-parameters.txt | 5 +++--
include/linux/nmi.h                             | 2 ++
kernel/watchdog.c                               | 2 ++
kernel/watchdog_hld.c                           | 9 +++++++++
4 files changed, 16 insertions(+), 2 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +0 -0
index 9e5bab29685f..d378e23dad7c 100644
@@ -3593,10 +3593,12 @@

Format: [state][,regs][,debounce][,die]

Quoted 11 lines

nmi_watchdog= [KNL,BUGS=X86] Debugging features for SMP kernels
- Format: [panic,][nopanic,][num]
+ Format: [panic,][nopanic,][ref-cycles][num]
Valid num: 0 or 1
0 - turn hardlockup detector in nmi_watchdog off
1 - turn hardlockup detector in nmi_watchdog on
+ ref-cycles - configure the watchdog with perf event
+ "ref-cycles" instead of "cycles"
When panic is specified, panic when an NMI watchdog timeout occurs (or 'nopanic' to not panic on an NMI watchdog, if CONFIG_BOOTPARAM_HARDLOCKUP_PANIC is set)

I still hate the whole ref-cycles thing, at the very least powerpc also has HAVE_HARDLOCKUP_DETECTOR_PERF and they don't have ref-cycles, but perhaps them wants to use a different event when the moon is just so...

Quoted 2 lines

What again was wrong with the option of specifying a raw event value and falling back to cpu-cycles if that fails?

The same raw event number may mean different events on different hardware. So it is more likely to make mistakes in configurations. For example, r300 means ref-cycles on Intel CPUs, but it also means something else on AMD CPUs. I need to be very careful which hosts to run with nmi_watchdog=r300, as it may cause surprises. OTOH, nmi_watchdog=ref-cycles won't have this issue. Of course, this won't work for powerpc.

Does this make sense?

Thanks,
Song

#8 lore raw

Hi Peter,

Quoted 1 line · Song Liu

[...]

Quoted 19 lines
Quoted 12 lines
Quoted 11 lines

nmi_watchdog= [KNL,BUGS=X86] Debugging features for SMP kernels
- Format: [panic,][nopanic,][num]
+ Format: [panic,][nopanic,][ref-cycles][num]
Valid num: 0 or 1
0 - turn hardlockup detector in nmi_watchdog off
1 - turn hardlockup detector in nmi_watchdog on
+ ref-cycles - configure the watchdog with perf event
+ "ref-cycles" instead of "cycles"
When panic is specified, panic when an NMI watchdog timeout occurs (or 'nopanic' to not panic on an NMI watchdog, if CONFIG_BOOTPARAM_HARDLOCKUP_PANIC is set)

Quoted 6 lines

I still hate the whole ref-cycles thing, at the very least powerpc also has HAVE_HARDLOCKUP_DETECTOR_PERF and they don't have ref-cycles, but perhaps them wants to use a different event when the moon is just so...

What again was wrong with the option of specifying a raw event value and falling back to cpu-cycles if that fails?

Quoted 6 lines

The same raw event number may mean different events on different hardware. So it is more likely to make mistakes in configurations. For example, r300 means ref-cycles on Intel CPUs, but it also means something else on AMD CPUs. I need to be very careful which hosts to run with nmi_watchdog=r300, as it may cause surprises. OTOH, nmi_watchdog=ref-cycles won't have this issue. Of course, this won't work for powerpc.

Does this make sense? Do we have other ideas to configure this?

Thanks,
Song