Skip to contentSkip to conversation

hung_task_timeout: configurable default

PATCH In review Jeff Mahoney 3 participants 5 messages Apr 27, 2011 – May 4, 2011

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 hung_task_timeout: configurable default

This patch allows the default value for sysctl_hung_task_timeout_secs to be set at build time. The feature carries virtually no overhead, so it makes sense to keep it enabled. On heavily loaded systems, though, it can end up triggering stack traces when there is no bug other than the system being underprovisioned. We use this patch to keep the hung task facility available but disabled at boot-time.

The default of 120 seconds is preserved. As a note, commit e162b39a may have accidentally reverted commit fb822db4, which raised the default from 120 seconds to 480 seconds.

---
 kernel/hung_task.c |    3 ++-
 lib/Kconfig.debug  |   14 ++++++++++++++
 2 files changed, 16 insertions(+), 1 deletion(-)
kernel/hung_task.c +2 -1
@@ -33,7 +33,8 @@ unsigned long __read_mostly sysctl_hung_
 /*
  * Zero means infinite timeout - no checking done:
  */
-unsigned long __read_mostly sysctl_hung_task_timeout_secs = 120;
+unsigned long __read_mostly sysctl_hung_task_timeout_secs =
+					CONFIG_DEFAULT_HUNG_TASK_TIMEOUT;
 
 unsigned long __read_mostly sysctl_hung_task_warnings = 10;
 
lib/Kconfig.debug +14 -0
@@ -214,6 +214,20 @@ config DETECT_HUNG_TASK
 	  enabled then all held locks will also be reported. This
 	  feature has negligible overhead.
 
+config DEFAULT_HUNG_TASK_TIMEOUT
+	int "Default timeout for hung task detection (in seconds)"
+	depends on DETECT_HUNG_TASK
+	default 120
+	help
+	  This option controls the default timeout (in seconds) used
+	  to determine when a task has become non-responsive and should
+	  be considered hung.
+
+	  It can be adjusted at runtime via the kernel.hung_task_timeout
+	  sysctl or by writing a value to /proc/sys/kernel/hung_task_timeout.
+
+	  A timeout of 0 disables the check.  The default is 120 seconds.
+
 config BOOTPARAM_HUNG_TASK_PANIC
 	bool "Panic (Reboot) On Hung Tasks"
 	depends on DETECT_HUNG_TASK
Signature · 2 lines
Jeff Mahoney
SUSE Labs
Acked-by on v1 1/1
#1 lore raw
Quoted 7 lines · Jeff Mahoney

This patch allows the default value for sysctl_hung_task_timeout_secs to be set at build time. The feature carries virtually no overhead, so it makes sense to keep it enabled. On heavily loaded systems, though, it can end up triggering stack traces when there is no bug other than the system being underprovisioned. We use this patch to keep the hung task facility available but disabled at boot-time.

Clever.

Quoted 5 lines

The default of 120 seconds is preserved. As a note, commit e162b39a may have accidentally reverted commit fb822db4, which raised the default from 120 seconds to 480 seconds.

Quoted 43 lines
---
 kernel/hung_task.c |    3 ++-
 lib/Kconfig.debug  |   14 ++++++++++++++
 2 files changed, 16 insertions(+), 1 deletion(-)
kernel/hung_task.c +2 -1
@@ -33,7 +33,8 @@ unsigned long __read_mostly sysctl_hung_
 /*
  * Zero means infinite timeout - no checking done:
  */
-unsigned long __read_mostly sysctl_hung_task_timeout_secs = 120;
+unsigned long __read_mostly sysctl_hung_task_timeout_secs =
+					CONFIG_DEFAULT_HUNG_TASK_TIMEOUT;
 
 unsigned long __read_mostly sysctl_hung_task_warnings = 10;
 
lib/Kconfig.debug +14 -0
@@ -214,6 +214,20 @@ config DETECT_HUNG_TASK
 	  enabled then all held locks will also be reported. This
 	  feature has negligible overhead.
 
+config DEFAULT_HUNG_TASK_TIMEOUT
+	int "Default timeout for hung task detection (in seconds)"
+	depends on DETECT_HUNG_TASK
+	default 120
+	help
+	  This option controls the default timeout (in seconds) used
+	  to determine when a task has become non-responsive and should
+	  be considered hung.
+
+	  It can be adjusted at runtime via the kernel.hung_task_timeout
+	  sysctl or by writing a value to /proc/sys/kernel/hung_task_timeout.
+
+	  A timeout of 0 disables the check.  The default is 120 seconds.
+
 config BOOTPARAM_HUNG_TASK_PANIC
 	bool "Panic (Reboot) On Hung Tasks"
 	depends on DETECT_HUNG_TASK
Signature · 2 lines
Jeff Mahoney
SUSE Labs
#2 lore raw

[tip:core/locking] watchdog, hung_task_timeout: Add Kconfig configurable default

In reply to Jeff Mahoney
Commit-ID:  e11feaa1192a079ba8e88a12121e9b12d55d4239
Gitweb:     http://git.kernel.org/tip/e11feaa1192a079ba8e88a12121e9b12d55d4239
Author:     Jeff Mahoney <[email protected]>
AuthorDate: Wed, 27 Apr 2011 14:27:24 -0400
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 28 Apr 2011 09:13:17 +0200

watchdog, hung_task_timeout: Add Kconfig configurable default

This patch allows the default value for sysctl_hung_task_timeout_secs to be set at build time. The feature carries virtually no overhead, so it makes sense to keep it enabled. On heavily loaded systems, though, it can end up triggering stack traces when there is no bug other than the system being underprovisioned. We use this patch to keep the hung task facility available but disabled at boot-time.

The default of 120 seconds is preserved. As a note, commit e162b39a may have accidentally reverted commit fb822db4, which raised the default from 120 seconds to 480 seconds.

---
 kernel/hung_task.c |    2 +-
 lib/Kconfig.debug  |   15 +++++++++++++++
 2 files changed, 16 insertions(+), 1 deletions(-)
kernel/hung_task.c +1 -1
index 53ead17..ea64012 100644
@@ -33,7 +33,7 @@ unsigned long __read_mostly sysctl_hung_task_check_count = PID_MAX_LIMIT;
 /*
  * Zero means infinite timeout - no checking done:
  */
-unsigned long __read_mostly sysctl_hung_task_timeout_secs = 120;
+unsigned long __read_mostly sysctl_hung_task_timeout_secs = CONFIG_DEFAULT_HUNG_TASK_TIMEOUT;
 
 unsigned long __read_mostly sysctl_hung_task_warnings = 10;
 
lib/Kconfig.debug +15 -0
index c768bcd..debbb05 100644
@@ -238,6 +238,21 @@ config DETECT_HUNG_TASK
 	  enabled then all held locks will also be reported. This
 	  feature has negligible overhead.
 
+config DEFAULT_HUNG_TASK_TIMEOUT
+	int "Default timeout for hung task detection (in seconds)"
+	depends on DETECT_HUNG_TASK
+	default 120
+	help
+	  This option controls the default timeout (in seconds) used
+	  to determine when a task has become non-responsive and should
+	  be considered hung.
+
+	  It can be adjusted at runtime via the kernel.hung_task_timeout
+	  sysctl or by writing a value to /proc/sys/kernel/hung_task_timeout.
+
+	  A timeout of 0 disables the check.  The default is two minutes.
+	  Keeping the default should be fine in most cases.
+
 config BOOTPARAM_HUNG_TASK_PANIC
 	bool "Panic (Reboot) On Hung Tasks"
 	depends on DETECT_HUNG_TASK
maintainer
#3 lore raw
In reply to Jeff Mahoney
Quoted 10 lines · On Wed, 27 Apr 2011 14:27:24 -0400 Jeff Mahoney <[email protected]> wrote:

This patch allows the default value for sysctl_hung_task_timeout_secs to be set at build time. The feature carries virtually no overhead, so it makes sense to keep it enabled. On heavily loaded systems, though, it can end up triggering stack traces when there is no bug other than the system being underprovisioned. We use this patch to keep the hung task facility available but disabled at boot-time.

The default of 120 seconds is preserved. As a note, commit e162b39a may have accidentally reverted commit fb822db4, which raised the default from 120 seconds to 480 seconds.

The changelog forgot to tell us why the patch's author considers the patch to be needed. This happens quite a lot.

kernel/hung_task.c unsigned long __read_mostly sysctl_hung_ · line 38

Quoted 8 lines
kernel/hung_task.c +2 -1
@@ -33,7 +33,8 @@ unsigned long __read_mostly sysctl_hung_
 /*
  * Zero means infinite timeout - no checking done:
  */
-unsigned long __read_mostly sysctl_hung_task_timeout_secs = 120;
+unsigned long __read_mostly sysctl_hung_task_timeout_secs =
+					CONFIG_DEFAULT_HUNG_TASK_TIMEOUT;
 

For the life of me I can't understand why you distro guys need to keep patching the kernel when you could just add a line to your initscripts.

I'm suspecting that lameness is involved.

#4 lore raw
Quoted 29 lines · Andrew Morton
Quoted 10 lines · On Wed, 27 Apr 2011 14:27:24 -0400 Jeff Mahoney <[email protected]> wrote:

This patch allows the default value for sysctl_hung_task_timeout_secs to be set at build time. The feature carries virtually no overhead, so it makes sense to keep it enabled. On heavily loaded systems, though, it can end up triggering stack traces when there is no bug other than the system being underprovisioned. We use this patch to keep the hung task facility available but disabled at boot-time.

The default of 120 seconds is preserved. As a note, commit e162b39a may have accidentally reverted commit fb822db4, which raised the default from 120 seconds to 480 seconds.

The changelog forgot to tell us why the patch's author considers the patch to be needed. This happens quite a lot.

kernel/hung_task.c unsigned long __read_mostly sysctl_hung_ · line 38

Quoted 8 lines
kernel/hung_task.c +2 -1
@@ -33,7 +33,8 @@ unsigned long __read_mostly sysctl_hung_
 /*
 * Zero means infinite timeout - no checking done:
 */
-unsigned long __read_mostly sysctl_hung_task_timeout_secs = 120;
+unsigned long __read_mostly sysctl_hung_task_timeout_secs =
+                    CONFIG_DEFAULT_HUNG_TASK_TIMEOUT;
 

For the life of me I can't understand why you distro guys need to keep patching the kernel when you could just add a line to your initscripts.

Quoted 1 line

I'm suspecting that lameness is involved.

Good point, and one we actually made ourselves internally after starting to submit these in the last round when we were tweaking more sensitive knobs.

Skip this one and I'll submit a patch fixing the 480s -> 120s regression.

-Jeff

--
Jeff Mahoney
(mobile)