Skip to contentSkip to conversation

add a counter for writers spinning on a rwlock

RFC PATCH In review Frederic Weisbecker 5 participants 31 messages Jan 25, 2009 – Feb 5, 2009

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 2/2 add a counter for writers spinning on a rwlock

This patch adds a counter for writers that enter a rwlock slow path. For example it can be useful for slow background tasks which perform some jobs on the tasklist, such as the hung_task detector (kernel/hung_task.c).

It adds a inc/dec pair on the slow path and 4 bytes for each rwlocks, so the overhead is not null.

Only x86 is supported for now, writers_spinning_lock() will return 0 on other archs (which is perhaps not a good idea).

Comments?

---
 arch/Kconfig                          |    4 ++++
 arch/x86/Kconfig                      |    1 +
 arch/x86/include/asm/spinlock.h       |    5 +++++
 arch/x86/include/asm/spinlock_types.h |    1 +
 arch/x86/lib/rwlock_64.S              |   10 +++++++---
 include/linux/spinlock.h              |    7 +++++++
 6 files changed, 25 insertions(+), 3 deletions(-)
arch/Kconfig +4 -0
index 550dab2..86c22e0 100644
@@ -106,3 +106,7 @@ config HAVE_CLK
 	  The <linux/clk.h> calls support software clock gating and
 	  thus are a key power management tool on many systems.
 
+config HAVE_NB_WRITERS_SPINNING_LOCK
+	bool
+
+
arch/x86/Kconfig +1 -0
index 967e114..0aeae17 100644
@@ -40,6 +40,7 @@ config X86
 	select HAVE_GENERIC_DMA_COHERENT if X86_32
 	select HAVE_EFFICIENT_UNALIGNED_ACCESS
 	select USER_STACKTRACE_SUPPORT
+	select HAVE_NB_WRITERS_SPINNING_LOCK
 
 config ARCH_DEFCONFIG
 	string
arch/x86/include/asm/spinlock.h +5 -0
index 139b424..d9ee21b 100644
@@ -283,6 +283,11 @@ static inline int __raw_write_trylock(raw_rwlock_t *lock)
 	return 0;
 }
 
+static inline int __raw_writers_spinning_lock(raw_rwlock_t *lock)
+{
+	return lock->spinning_writers;
+}
+
 static inline void __raw_read_unlock(raw_rwlock_t *rw)
 {
 	asm volatile(LOCK_PREFIX "incl %0" :"+m" (rw->lock) : : "memory");
arch/x86/include/asm/spinlock_types.h +1 -0
index 845f81c..163e6de 100644
@@ -13,6 +13,7 @@ typedef struct raw_spinlock {
 
 typedef struct {
 	unsigned int lock;
+	unsigned int spinning_writers;
 } raw_rwlock_t;
 
 #define __RAW_RW_LOCK_UNLOCKED		{ RW_LOCK_BIAS }
arch/x86/lib/rwlock_64.S +7 -3
index 05ea55f..9589b74 100644
@@ -9,14 +9,18 @@
 ENTRY(__write_lock_failed)
 	CFI_STARTPROC
 	LOCK_PREFIX
+	incl 4(%rdi) /* add ourself as a spinning writer */
+1:	LOCK_PREFIX
 	addl $RW_LOCK_BIAS,(%rdi)
-1:	rep
+2:	rep
 	nop
 	cmpl $RW_LOCK_BIAS,(%rdi)
-	jne 1b
+	jne 2b
 	LOCK_PREFIX
 	subl $RW_LOCK_BIAS,(%rdi)
-	jnz  __write_lock_failed
+	jnz  1b
+	LOCK_PREFIX
+	decl 4(%rdi)
 	ret
 	CFI_ENDPROC
 END(__write_lock_failed)
include/linux/spinlock.h +7 -0
index e0c0fcc..8ce22af 100644
@@ -121,6 +121,13 @@ do {								\
 
 #define spin_is_locked(lock)	__raw_spin_is_locked(&(lock)->raw_lock)
 
+#ifdef CONFIG_HAVE_NB_WRITERS_SPINNING_LOCK
+#define writers_spinning_lock(rwlock)				\
+	__raw_writers_spinning_lock(&(rwlock)->raw_lock)
+#else
+#define writers_spinning_lock(rwlock)	0
+#endif
+
 #ifdef CONFIG_GENERIC_LOCKBREAK
 #define spin_is_contended(lock) ((lock)->break_lock)
 #else
Signature · 1 line
1.6.1
#1 lore raw
Quoted 5 lines · Frederic Weisbecker

This patch adds a counter for writers that enter a rwlock slow path. For example it can be useful for slow background tasks which perform some jobs on the tasklist, such as the hung_task detector (kernel/hung_task.c).

Quoted 7 lines

It adds a inc/dec pair on the slow path and 4 bytes for each rwlocks, so the overhead is not null.

Only x86 is supported for now, writers_spinning_lock() will return 0 on other archs (which is perhaps not a good idea).

Comments?

hm, it increases the rwlock data type:

arch/x86/include/asm/spinlock_types.h typedef struct raw_spinlock { · line 17

Quoted 10 lines
arch/x86/include/asm/spinlock_types.h +1 -0
index 845f81c..163e6de 100644
@@ -13,6 +13,7 @@ typedef struct raw_spinlock {
 
 typedef struct {
 	unsigned int lock;
+	unsigned int spinning_writers;
 } raw_rwlock_t;

that's generally not done lightly. Performance figures for a relevant workload are obligatory in this case - proving that it's worth the size bloat.

Ingo

maintainer
#2 lore raw
In reply to Frederic Weisbecker
Quoted 11 lines · Frederic Weisbecker

This patch adds a counter for writers that enter a rwlock slow path. For example it can be useful for slow background tasks which perform some jobs on the tasklist, such as the hung_task detector (kernel/hung_task.c).

It adds a inc/dec pair on the slow path and 4 bytes for each rwlocks, so the overhead is not null.

Only x86 is supported for now, writers_spinning_lock() will return 0 on other archs (which is perhaps not a good idea).

Comments?

_why_ ?

#3 lore raw

2009/1/26 Peter Zijlstra <[email protected]>:

Quoted 8 lines · Frederic Weisbecker
Quoted 7 lines

This patch adds a counter for writers that enter a rwlock slow path. For example it can be useful for slow background tasks which perform some jobs on the tasklist, such as the hung_task detector (kernel/hung_task.c).

It adds a inc/dec pair on the slow path and 4 bytes for each rwlocks, so the overhead is not null.

Quoted 6 lines
Quoted 4 lines

Only x86 is supported for now, writers_spinning_lock() will return 0 on other archs (which is perhaps not a good idea).

Comments?

_why_ ?

The hung task detector runs a periodic loop through the task_list, and currently it doesn't run
over an arbitrary threshold of tasks to not hold the task_list lock for too long.

So we thought about a way to detect if there are some writers waiting for the lock, anf if so, release
the lock, schedule and retry.

I'm not sure this is something easy to measure, since this is more about small latency gains (if there are).

maintainer
#4 lore raw
Quoted 16 lines · Frédéric Weisbecker

2009/1/26 Peter Zijlstra <[email protected]>:

Quoted 8 lines · Frederic Weisbecker
Quoted 7 lines

This patch adds a counter for writers that enter a rwlock slow path. For example it can be useful for slow background tasks which perform some jobs on the tasklist, such as the hung_task detector (kernel/hung_task.c).

It adds a inc/dec pair on the slow path and 4 bytes for each rwlocks, so the overhead is not null.

Quoted 6 lines
Quoted 4 lines

Only x86 is supported for now, writers_spinning_lock() will return 0 on other archs (which is perhaps not a good idea).

Comments?

_why_ ?

Quoted 8 lines

The hung task detector runs a periodic loop through the task_list, and currently it doesn't run
over an arbitrary threshold of tasks to not hold the task_list lock for too long.

So we thought about a way to detect if there are some writers waiting for the lock, anf if so, release
the lock, schedule and retry.

Ah, if it can do that, then it can also use RCU, no? Only users who really have to hold off new tasks need the read-task_lock. The rest can use RCU.

#5 lore raw

2009/1/26 Peter Zijlstra <[email protected]>:

Quoted 22 lines · Frédéric Weisbecker
Quoted 21 lines

2009/1/26 Peter Zijlstra <[email protected]>:

Quoted 8 lines · Frederic Weisbecker
Quoted 7 lines

This patch adds a counter for writers that enter a rwlock slow path. For example it can be useful for slow background tasks which perform some jobs on the tasklist, such as the hung_task detector (kernel/hung_task.c).

It adds a inc/dec pair on the slow path and 4 bytes for each rwlocks, so the overhead is not null.

Quoted 6 lines
Quoted 4 lines

Only x86 is supported for now, writers_spinning_lock() will return 0 on other archs (which is perhaps not a good idea).

Comments?

_why_ ?

The hung task detector runs a periodic loop through the task_list, and currently it doesn't run
over an arbitrary threshold of tasks to not hold the task_list lock for too long.

Quoted 7 lines
Quoted 3 lines

So we thought about a way to detect if there are some writers waiting for the lock, anf if so, release
the lock, schedule and retry.

Ah, if it can do that, then it can also use RCU, no? Only users who really have to hold off new tasks need the read-task_lock. The rest can use RCU.

Really?
That sounds a good news. Mandeep, what do you think?

#6 lore raw

Hi Frédéric,

Unfortunately, this can't be done for hung_task. It writes to the task_struct here:

static void check_hung_task(struct task_struct *t, unsigned long now,
unsigned long timeout)

{
        unsigned long switch_count = t->nvcsw + t->nivcsw;

        if (t->flags & PF_FROZEN)
                return;

        if (switch_count != t->last_switch_count || !t->last_switch_timestamp) {
                t->last_switch_count = switch_count;
                t->last_switch_timestamp = now;
                return;
        }

It is able to get away with using only a read_lock because no one else reads or writes to these fields.

Regards,
Mandeep

Quoted 35 lines · Frédéric Weisbecker

2009/1/26 Peter Zijlstra <[email protected]>:

Quoted 22 lines · Frédéric Weisbecker
Quoted 21 lines

2009/1/26 Peter Zijlstra <[email protected]>:

Quoted 8 lines · Frederic Weisbecker
Quoted 7 lines

This patch adds a counter for writers that enter a rwlock slow path. For example it can be useful for slow background tasks which perform some jobs on the tasklist, such as the hung_task detector (kernel/hung_task.c).

It adds a inc/dec pair on the slow path and 4 bytes for each rwlocks, so the overhead is not null.

Quoted 6 lines
Quoted 4 lines

Only x86 is supported for now, writers_spinning_lock() will return 0 on other archs (which is perhaps not a good idea).

Comments?

_why_ ?

The hung task detector runs a periodic loop through the task_list, and currently it doesn't run
over an arbitrary threshold of tasks to not hold the task_list lock for too long.

Quoted 7 lines
Quoted 3 lines

So we thought about a way to detect if there are some writers waiting for the lock, anf if so, release
the lock, schedule and retry.

Ah, if it can do that, then it can also use RCU, no? Only users who really have to hold off new tasks need the read-task_lock. The rest can use RCU.

Really?
That sounds a good news. Mandeep, what do you think?

maintainer
#7 lore raw
Quoted 2 lines · Mandeep Baines

Unfortunately, this can't be done for hung_task. It writes to the task_struct here:

Don't top post!

Quoted 14 lines

static void check_hung_task(struct task_struct *t, unsigned long now,
unsigned long timeout)

{
        unsigned long switch_count = t->nvcsw + t->nivcsw;

        if (t->flags & PF_FROZEN)
                return;

        if (switch_count != t->last_switch_count || !t->last_switch_timestamp) {
                t->last_switch_count = switch_count;
                t->last_switch_timestamp = now;
                return;
        }
Quoted 2 lines

It is able to get away with using only a read_lock because no one else reads or writes to these fields.

How would RCU be different here?

#8 lore raw

PATCH v4 1/1 softlockup: remove hung_task_check_count

Quoted 21 lines · Peter Zijlstra
Quoted 2 lines · Mandeep Baines

Unfortunately, this can't be done for hung_task. It writes to the task_struct here:

Don't top post!

Quoted 14 lines

static void check_hung_task(struct task_struct *t, unsigned long now,
unsigned long timeout)

{
        unsigned long switch_count = t->nvcsw + t->nivcsw;

        if (t->flags & PF_FROZEN)
                return;

        if (switch_count != t->last_switch_count || !t->last_switch_timestamp) {
                t->last_switch_count = switch_count;
                t->last_switch_timestamp = now;
                return;
        }
Quoted 5 lines
Quoted 2 lines

It is able to get away with using only a read_lock because no one else reads or writes to these fields.

How would RCU be different here?

My bad, RCU wouldn't be any different. I misunderstood how RCU works. Just spent the morning reading the LWN 3-part series on RCU and I think I'm able to grok it now;)

Below is a patch to hung_task which removes the hung_task_check_count and converts the read_locks to RCU.

Thanks Frédéric and Peter!

---
To avoid holding the tasklist lock too long, hung_task_check_count was used as an upper bound on the number of tasks that are checked by hung_task. This patch removes the hung_task_check_count sysctl.

Instead of checking a limited number of tasks, all tasks are checked. To avoid holding the CPU for too long, need_resched() is checked often. To avoid blocking out writers, the read_lock has been converted to an rcu_read_lock().

It is safe convert to an rcu_read_lock() because the tasks and thread_group lists are both protected by list_*_rcu() operations. The worst that can happen is that hung_task will update last_switch_timestamp field of a DEAD task.

The design was proposed by Frédéric Weisbecker. Peter Zijlstra suggested the use of RCU.

---
 include/linux/sched.h |    1 -
 kernel/hung_task.c    |   12 +++---------
 kernel/sysctl.c       |    9 ---------
 3 files changed, 3 insertions(+), 19 deletions(-)
include/linux/sched.h +0 -1
index f2f94d5..278121c 100644
@@ -315,7 +315,6 @@ static inline void touch_all_softlockup_watchdogs(void)
 
 #ifdef CONFIG_DETECT_HUNG_TASK
 extern unsigned int  sysctl_hung_task_panic;
-extern unsigned long sysctl_hung_task_check_count;
 extern unsigned long sysctl_hung_task_timeout_secs;
 extern unsigned long sysctl_hung_task_warnings;
 extern int proc_dohung_task_timeout_secs(struct ctl_table *table, int write,
kernel/hung_task.c +3 -9
index ba8ccd4..7d67350 100644
@@ -17,11 +17,6 @@
 #include <linux/sysctl.h>
 
 /*
- * Have a reasonable limit on the number of tasks checked:
- */
-unsigned long __read_mostly sysctl_hung_task_check_count = 1024;
-
-/*
  * Zero means infinite timeout - no checking done:
  */
 unsigned long __read_mostly sysctl_hung_task_timeout_secs = 120;
@@ -116,7 +111,6 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
  */
 static void check_hung_uninterruptible_tasks(unsigned long timeout)
 {
-	int max_count = sysctl_hung_task_check_count;
 	unsigned long now = get_timestamp();
 	struct task_struct *g, *t;
 
@@ -127,16 +121,16 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	if (test_taint(TAINT_DIE) || did_panic)
 		return;
 
-	read_lock(&tasklist_lock);
+	rcu_read_lock();
 	do_each_thread(g, t) {
-		if (!--max_count)
+		if (need_resched())
 			goto unlock;
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);
 	} while_each_thread(g, t);
  unlock:
-	read_unlock(&tasklist_lock);
+	rcu_read_unlock();
 }
 
 static void update_poll_jiffies(void)
kernel/sysctl.c +0 -9
index 2481ed3..16526a2 100644
@@ -820,15 +820,6 @@ static struct ctl_table kern_table[] = {
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
-		.procname	= "hung_task_check_count",
-		.data		= &sysctl_hung_task_check_count,
-		.maxlen		= sizeof(unsigned long),
-		.mode		= 0644,
-		.proc_handler	= &proc_doulongvec_minmax,
-		.strategy	= &sysctl_intvec,
-	},
-	{
-		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "hung_task_timeout_secs",
 		.data		= &sysctl_hung_task_timeout_secs,
 		.maxlen		= sizeof(unsigned long),
Signature · 1 line
1.5.4.5
#9 lore raw

Re: [PATCH v4] softlockup: remove hung_task_check_count

Quoted 141 lines · Mandeep Singh Baines
Quoted 21 lines · Peter Zijlstra
Quoted 2 lines · Mandeep Baines

Unfortunately, this can't be done for hung_task. It writes to the task_struct here:

Don't top post!

Quoted 14 lines

static void check_hung_task(struct task_struct *t, unsigned long now,
unsigned long timeout)

{
        unsigned long switch_count = t->nvcsw + t->nivcsw;

        if (t->flags & PF_FROZEN)
                return;

        if (switch_count != t->last_switch_count || !t->last_switch_timestamp) {
                t->last_switch_count = switch_count;
                t->last_switch_timestamp = now;
                return;
        }
Quoted 5 lines
Quoted 2 lines

It is able to get away with using only a read_lock because no one else reads or writes to these fields.

How would RCU be different here?

My bad, RCU wouldn't be any different. I misunderstood how RCU works. Just spent the morning reading the LWN 3-part series on RCU and I think I'm able to grok it now;)

Below is a patch to hung_task which removes the hung_task_check_count and converts the read_locks to RCU.

Thanks Frédéric and Peter!

---
To avoid holding the tasklist lock too long, hung_task_check_count was used as an upper bound on the number of tasks that are checked by hung_task. This patch removes the hung_task_check_count sysctl.

Instead of checking a limited number of tasks, all tasks are checked. To avoid holding the CPU for too long, need_resched() is checked often. To avoid blocking out writers, the read_lock has been converted to an rcu_read_lock().

It is safe convert to an rcu_read_lock() because the tasks and thread_group lists are both protected by list_*_rcu() operations. The worst that can happen is that hung_task will update last_switch_timestamp field of a DEAD task.

The design was proposed by Frédéric Weisbecker. Peter Zijlstra suggested the use of RCU.

---
 include/linux/sched.h |    1 -
 kernel/hung_task.c    |   12 +++---------
 kernel/sysctl.c       |    9 ---------
 3 files changed, 3 insertions(+), 19 deletions(-)
include/linux/sched.h +0 -1
index f2f94d5..278121c 100644
@@ -315,7 +315,6 @@ static inline void touch_all_softlockup_watchdogs(void)
 
 #ifdef CONFIG_DETECT_HUNG_TASK
 extern unsigned int  sysctl_hung_task_panic;
-extern unsigned long sysctl_hung_task_check_count;
 extern unsigned long sysctl_hung_task_timeout_secs;
 extern unsigned long sysctl_hung_task_warnings;
 extern int proc_dohung_task_timeout_secs(struct ctl_table *table, int write,
kernel/hung_task.c +3 -9
index ba8ccd4..7d67350 100644
@@ -17,11 +17,6 @@
 #include <linux/sysctl.h>
 
 /*
- * Have a reasonable limit on the number of tasks checked:
- */
-unsigned long __read_mostly sysctl_hung_task_check_count = 1024;
-
-/*
  * Zero means infinite timeout - no checking done:
  */
 unsigned long __read_mostly sysctl_hung_task_timeout_secs = 120;
@@ -116,7 +111,6 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
  */
 static void check_hung_uninterruptible_tasks(unsigned long timeout)
 {
-	int max_count = sysctl_hung_task_check_count;
 	unsigned long now = get_timestamp();
 	struct task_struct *g, *t;
 
@@ -127,16 +121,16 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	if (test_taint(TAINT_DIE) || did_panic)
 		return;
 
-	read_lock(&tasklist_lock);
+	rcu_read_lock();
 	do_each_thread(g, t) {
-		if (!--max_count)
+		if (need_resched())
 			goto unlock;
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);
 	} while_each_thread(g, t);
  unlock:
-	read_unlock(&tasklist_lock);
+	rcu_read_unlock();
 }
 
 static void update_poll_jiffies(void)
kernel/sysctl.c +0 -9
index 2481ed3..16526a2 100644
@@ -820,15 +820,6 @@ static struct ctl_table kern_table[] = {
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
-		.procname	= "hung_task_check_count",
-		.data		= &sysctl_hung_task_check_count,
-		.maxlen		= sizeof(unsigned long),
-		.mode		= 0644,
-		.proc_handler	= &proc_doulongvec_minmax,
-		.strategy	= &sysctl_intvec,
-	},
-	{
-		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "hung_task_timeout_secs",
 		.data		= &sysctl_hung_task_timeout_secs,
 		.maxlen		= sizeof(unsigned long),
Signature · 1 line
1.5.4.5

That looks good :-)

#10 lore raw

Re: [PATCH v4] softlockup: remove hung_task_check_count

In reply to Mandeep Singh Baines
Quoted 2 lines · Mandeep Singh Baines

The design was proposed by Frédéric Weisbecker. Peter Zijlstra suggested the use of RCU.

ok, this looks _much_ cleaner.

One question:

kernel/hung_task.c static void check_hung_uninterruptible_tasks(unsigned long timeout) · line 127

Quoted 6 lines
kernel/hung_task.c +2 -2
-	read_lock(&tasklist_lock);
+	rcu_read_lock();
 	do_each_thread(g, t) {
-		if (!--max_count)
+		if (need_resched())
 			goto unlock;

Isnt it dangerous to skip a check just because we got marked for reschedule? Since it runs so rarely it could by accident be preempted and we'd not get any checking done for a long time.

Ingo

#11 lore raw

Re: [PATCH v4] softlockup: remove hung_task_check_count

Quoted 10 lines · Ingo Molnar
Quoted 2 lines · Mandeep Singh Baines

The design was proposed by Frédéric Weisbecker. Peter Zijlstra suggested the use of RCU.

ok, this looks _much_ cleaner.

One question:

Quoted 11 lines

kernel/hung_task.c static void check_hung_uninterruptible_tasks(unsigned long timeout) · line 127

Quoted 6 lines
kernel/hung_task.c +2 -2
-	read_lock(&tasklist_lock);
+	rcu_read_lock();
 	do_each_thread(g, t) {
-		if (!--max_count)
+		if (need_resched())
 			goto unlock;

Isnt it dangerous to skip a check just because we got marked for reschedule? Since it runs so rarely it could by accident be preempted and we'd not get any checking done for a long time.

Yeah, the checking could be deferred indefinitely. So you could have a system where tasks are hung but it takes a really long time to detect this and finally panic the system. Not so good for high-availability.

What if a check_count sysctl was added? But instead of being a max_check_count, it would be a min_check_count. This would guarantee that a minimum amount of checking is done.

Alternatively, the code for trying to continue the iteration after a reschedule could be re-inserted. That code is a little tricky and potentially fragile since continuation of a tasklist iteration is not really supported by the sched.h APIs. But it is does have the nice properties of getting through the entire list almost all the time and still playing nice with the scheduler.

Regards,
Mandeep

maintainer
#12 lore raw

Re: [PATCH v4] softlockup: remove hung_task_check_count

Quoted 23 lines · Mandeep Singh Baines
Quoted 10 lines · Ingo Molnar
Quoted 2 lines · Mandeep Singh Baines

The design was proposed by Frédéric Weisbecker. Peter Zijlstra suggested the use of RCU.

ok, this looks _much_ cleaner.

One question:

Quoted 11 lines

kernel/hung_task.c static void check_hung_uninterruptible_tasks(unsigned long timeout) · line 127

Quoted 6 lines
kernel/hung_task.c +2 -2
-	read_lock(&tasklist_lock);
+	rcu_read_lock();
 	do_each_thread(g, t) {
-		if (!--max_count)
+		if (need_resched())
 			goto unlock;

Isnt it dangerous to skip a check just because we got marked for reschedule? Since it runs so rarely it could by accident be preempted and we'd not get any checking done for a long time.

Quoted 3 lines

Yeah, the checking could be deferred indefinitely. So you could have a system where tasks are hung but it takes a really long time to detect this and finally panic the system. Not so good for high-availability.

Why break out at all? Are you that worried about khungtaskd introducing latencies? Is using preemptible RCU an option for you?

#13 lore raw

Re: [PATCH v4] softlockup: remove hung_task_check_count

Quoted 24 lines · Peter Zijlstra
Quoted 23 lines · Mandeep Singh Baines
Quoted 21 lines · Ingo Molnar
Quoted 2 lines · Mandeep Singh Baines

The design was proposed by Frédéric Weisbecker. Peter Zijlstra suggested the use of RCU.

ok, this looks _much_ cleaner.

One question:

kernel/hung_task.c static void check_hung_uninterruptible_tasks(unsigned long timeout) · line 127

Quoted 6 lines
kernel/hung_task.c +2 -2
-	read_lock(&tasklist_lock);
+	rcu_read_lock();
 	do_each_thread(g, t) {
-		if (!--max_count)
+		if (need_resched())
 			goto unlock;

Isnt it dangerous to skip a check just because we got marked for reschedule? Since it runs so rarely it could by accident be preempted and we'd not get any checking done for a long time.

Quoted 6 lines
Quoted 3 lines

Yeah, the checking could be deferred indefinitely. So you could have a system where tasks are hung but it takes a really long time to detect this and finally panic the system. Not so good for high-availability.

Why break out at all? Are you that worried about khungtaskd introducing latencies?

Yes, I was worried about disabling preemption for an unbounded amount of time.

Quoted 2 lines

Is using preemptible RCU an option for you?

I had not even considered that. To be honest, I had not even heard of it till now. So I spent another morning at LWN grokking preemptible RCU;)

I think it can work. I'm a little worried about the OOM risk. It could take a really long time to iterate over the task list. A lot of pending kfree()s could build up in that time.

I'm still unclear as to whether khungtaskd would get priority boosted or not. Are only tasks that explicitly block (by sleeping on a lock) boostable or would a task which got pre-empted inside the critical section also be boostable? My worry here is khungtaskd would run at high priority for an indefinite amount of time making it more overhead. Is there an OOMing-soon booster?

But I think this can all be mitigated by adding back the max_count sysctl. But preemptible RCU would allow increasing the default to a much higher value or we may even be able to make the default "check all tasks".

Running out of time today but I'll send out a patch tomorrow.

Thanks Peter!

Regards,
Mandeep

#14 lore raw

Re: [PATCH v4] softlockup: remove hung_task_check_count

Quoted 11 lines · Mandeep Singh Baines
Quoted 3 lines · Peter Zijlstra

Why break out at all? Are you that worried about khungtaskd introducing latencies?

Yes, I was worried about disabling preemption for an unbounded amount of time.

Quoted 2 lines

Is using preemptible RCU an option for you?

Quoted 7 lines

I had not even considered that. To be honest, I had not even heard of it till now. So I spent another morning at LWN grokking preemptible RCU;)

I think it can work. I'm a little worried about the OOM risk. It could take a really long time to iterate over the task list. A lot of pending kfree()s could build up in that time.

I misunderstood preemptible RCU. I assumed it was a new API but its not. So I don't think preemptible RCU is an option since it would force a dependency on CONFIG_PREEMPT_RCU.

I'm going to break up this patch in two. One patch for converting to rcu. A second patch which will support checking all tasks. To support checking all tasks I reverted back to a design similar to Frédéric original proposal.

I'll send the patches out right after this email.

[PATCH 1/2] softlockup: convert read_lock in hung_task to rcu_read_lock [PATCH 2/2] softlockup: check all tasks in hung_task

Regards,
Mandeep

#15 lore raw

PATCH 1/2 softlockup: convert read_lock in hung_task to rcu_read_lock

In reply to Mandeep Singh Baines

Peter Zijlstra suggested the use of RCU.

---
 kernel/hung_task.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
kernel/hung_task.c +2 -2
index ba8ccd4..a841db3 100644
@@ -127,7 +127,7 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	if (test_taint(TAINT_DIE) || did_panic)
 		return;
 
-	read_lock(&tasklist_lock);
+	rcu_read_lock();
 	do_each_thread(g, t) {
 		if (!--max_count)
 			goto unlock;
@@ -136,7 +136,7 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 			check_hung_task(t, now, timeout);
 	} while_each_thread(g, t);
  unlock:
-	read_unlock(&tasklist_lock);
+	rcu_read_unlock();
 }
 
 static void update_poll_jiffies(void)
Signature · 1 line
1.5.4.5
#16 lore raw

PATCH 2/2 softlockup: check all tasks in hung_task

In reply to Mandeep Singh Baines

Instead of checking only hung_task_check_count tasks, all tasks are checked. hung_task_check_count is still used to put an upper bound on the critical section. Every hung_task_check_count checks, the critical section is refreshed. Keeping the critical section small minimizes time preemption is disabled and keeps rcu grace periods small.

To prevent following a stale pointer, get_task_struct is called on g and t. To verify that g and t have not been unhashed while outside the critical section, the task states are checked.

The design was proposed by Frédéric Weisbecker.

Quoted 15 lines · Frédéric Weisbecker

Instead of having this arbitrary limit of tasks, why not just lurk the need_resched() and then schedule if it needs too.

I know that sounds a bit racy, because you will have to release the tasklist_lock and
a lot of things can happen in the task list until you become resched. But you can do a get_task_struct() on g and t before your thread is going to sleep and then put them
when it is awaken.
Perhaps some tasks will disappear or be appended in the list before g and t, but that doesn't really matter:
if they disappear, they didn't lockup, and if they were appended, they are not enough cold to be analyzed :-)

Quoted 4 lines

This way you can drop the arbitrary limit of task number given by the user....

Frederic.

---
 kernel/hung_task.c |   28 ++++++++++++++++++++++++++--
 1 files changed, 26 insertions(+), 2 deletions(-)
kernel/hung_task.c +26 -2
index a841db3..1c8c9f9 100644
@@ -109,6 +109,25 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
 		panic("hung_task: blocked tasks");
 }
 
+ /*
+  * To avoid extending the RCU grace period for an unbounded amount of time,
+  * periodically exit the critical section and enter a new one.
+  *
+  * For preemptible RCU it is sufficient to call rcu_read_unlock in order
+  * exit the grace period. For classic RCU, a reschedule is required.
+  */
+static void check_hung_rcu_refresh(struct task_struct *g, struct task_struct *t)
+{
+	get_task_struct(g);
+	get_task_struct(t);
+	rcu_read_unlock();
+	if (need_resched())
+		schedule();
+	rcu_read_lock();
+	put_task_struct(t);
+	put_task_struct(g);
+}
+
 /*
  * Check whether a TASK_UNINTERRUPTIBLE does not get woken up for
  * a really long time (120 seconds). If that happens, print out
@@ -129,8 +148,13 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 
 	rcu_read_lock();
 	do_each_thread(g, t) {
-		if (!--max_count)
-			goto unlock;
+		if (sysctl_hung_task_check_count && !(max_count--)) {
+			max_count = sysctl_hung_task_check_count;
+			check_hung_rcu_refresh(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;
+		}
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);
Signature · 1 line
1.5.4.5
maintainer
#17 lore raw

Re: [PATCH 2/2] softlockup: check all tasks in hung_task

Quoted 50 lines · Mandeep Singh Baines

Instead of checking only hung_task_check_count tasks, all tasks are checked. hung_task_check_count is still used to put an upper bound on the critical section. Every hung_task_check_count checks, the critical section is refreshed. Keeping the critical section small minimizes time preemption is disabled and keeps rcu grace periods small.

To prevent following a stale pointer, get_task_struct is called on g and t. To verify that g and t have not been unhashed while outside the critical section, the task states are checked.

The design was proposed by Frédéric Weisbecker.

Quoted 15 lines · Frédéric Weisbecker

Instead of having this arbitrary limit of tasks, why not just lurk the need_resched() and then schedule if it needs too.

I know that sounds a bit racy, because you will have to release the tasklist_lock and
a lot of things can happen in the task list until you become resched. But you can do a get_task_struct() on g and t before your thread is going to sleep and then put them
when it is awaken.
Perhaps some tasks will disappear or be appended in the list before g and t, but that doesn't really matter:
if they disappear, they didn't lockup, and if they were appended, they are not enough cold to be analyzed :-)

Quoted 4 lines

This way you can drop the arbitrary limit of task number given by the user....

Frederic.

---
 kernel/hung_task.c |   28 ++++++++++++++++++++++++++--
 1 files changed, 26 insertions(+), 2 deletions(-)
kernel/hung_task.c +4 -0
index a841db3..1c8c9f9 100644
@@ -109,6 +109,25 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
 		panic("hung_task: blocked tasks");
 }
 
+ /*
+  * To avoid extending the RCU grace period for an unbounded amount of time,
+  * periodically exit the critical section and enter a new one.
+  *

kernel/hung_task.c static void check_hung_task(struct task_struct *t, unsigned long now, · line 125

Quoted 10 lines
kernel/hung_task.c +10 -0
+  * For preemptible RCU it is sufficient to call rcu_read_unlock in order
+  * exit the grace period. For classic RCU, a reschedule is required.
+  */
+static void check_hung_rcu_refresh(struct task_struct *g, struct task_struct *t)
+{
+	get_task_struct(g);
+	get_task_struct(t);
+	rcu_read_unlock();
+	if (need_resched())
+		schedule();

won't a simple cond_resched(), do?

Quoted 11 lines
kernel/hung_task.c +5 -0
+	rcu_read_lock();
+	put_task_struct(t);
+	put_task_struct(g);
+}
+
 /*
  * Check whether a TASK_UNINTERRUPTIBLE does not get woken up for
  * a really long time (120 seconds). If that happens, print out
@@ -129,8 +148,13 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 
 	rcu_read_lock();

kernel/hung_task.c static void check_hung_uninterruptible_tasks(unsigned long timeout) · line 157

Quoted 10 lines
kernel/hung_task.c +7 -2
 	do_each_thread(g, t) {
-		if (!--max_count)
-			goto unlock;
+		if (sysctl_hung_task_check_count && !(max_count--)) {
+			max_count = sysctl_hung_task_check_count;
+			check_hung_rcu_refresh(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;
+		}

Its all a bit ugly, but I suppose there's no way around that.

Quoted 3 lines
kernel/hung_task.c +0 -0
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);
#18 lore raw

PATCH v2 2/2 softlockup: check all tasks in hung_task

On Sat, Jan 31, 2009 at 11:22 AM, Peter Zijlstra <[email protected]> wro= te:

Quoted 1 line · Mandeep Singh Baines

<snip>

Quoted 1 line
Quoted 1 line

+static void check_hung_rcu_refresh(struct task_struct *g, struct task_s=

truct *t)

Quoted 9 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+{
+     get_task_struct(g);
+     get_task_struct(t);
+     rcu_read_unlock();
+     if (need_resched())
+             schedule();

won't a simple cond_resched(), do?

Yes, it will. Thanks Peter!

Quoted 9 lines

Review static void check_hung_uninterruptible_tasks(unsign=

Quoted 9 lines
(quoted diff) +5 -0
+     rcu_read_lock();
+     put_task_struct(t);
+     put_task_struct(g);
+}
+
 /*
  * Check whether a TASK_UNINTERRUPTIBLE does not get woken up for
  * a really long time (120 seconds). If that happens, print out
@@ -129,8 +148,13 @@ static void check_hung_uninterruptible_tasks(unsign=

ed long timeout)

Quoted 10 lines

Review static void check_hung_uninterruptible_tasks(unsign= · line 154

Quoted 10 lines
(quoted diff) +5 -2
      rcu_read_lock();
      do_each_thread(g, t) {
-             if (!--max_count)
-                     goto unlock;
+             if (sysctl_hung_task_check_count && !(max_count--)) {
+                     max_count =3D sysctl_hung_task_check_count;
+                     check_hung_rcu_refresh(g, t);
+                     /* Exit if t or g was unhashed during refresh. */
+                     if (t->state =3D=3D TASK_DEAD || g->state =3D=3D T=

ASK_DEAD)

Quoted 4 lines

Review static void check_hung_uninterruptible_tasks(unsign= · line 156

Quoted 2 lines
(quoted diff) +2 -0
+                             goto unlock;
+             }

Its all a bit ugly, but I suppose there's no way around that.

Yeah, need to guarantee that next of t and g isn't pointing to a freed task_struct. If the t or g has not been unlinked, then its guaranteed that the next fields are valid.

If the state is != TASK_DEAD then its guaranteed that the task hasn't been unlinked. I guess that's the shakiest assumption in the chain. If the code were ever changed to set the state to TASK_DEAD after unlinking ...

<snip>

Here's v2 of this patch. The only thing that's changed is the cond_resched().

---
Instead of checking only hung_task_check_count tasks, all tasks are checked. hung_task_check_count is still used to put an upper bound on the critical section. Every hung_task_check_count checks, the critical section is refreshed. Keeping the critical section small minimizes time preemption is disabled and keeps rcu grace periods small.

To prevent following a stale pointer, get_task_struct is called on g and t. To verify that g and t have not been unhashed while outside the critical section, the task states are checked.

The design was proposed by Frédéric Weisbecker.

Quoted 15 lines · Frédéric Weisbecker

Instead of having this arbitrary limit of tasks, why not just lurk the need_resched() and then schedule if it needs too.

I know that sounds a bit racy, because you will have to release the tasklist_lock and
a lot of things can happen in the task list until you become resched. But you can do a get_task_struct() on g and t before your thread is going to sleep and then put them
when it is awaken.
Perhaps some tasks will disappear or be appended in the list before g and t, but that doesn't really matter:
if they disappear, they didn't lockup, and if they were appended, they are not enough cold to be analyzed :-)

Quoted 4 lines

This way you can drop the arbitrary limit of task number given by the user....

Frederic.

---
 kernel/hung_task.c |   27 +++++++++++++++++++++++++--
 1 files changed, 25 insertions(+), 2 deletions(-)
kernel/hung_task.c +25 -2
index a841db3..464c96e 100644
@@ -109,6 +109,24 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
 		panic("hung_task: blocked tasks");
 }
 
+ /*
+  * To avoid extending the RCU grace period for an unbounded amount of time,
+  * periodically exit the critical section and enter a new one.
+  *
+  * For preemptible RCU it is sufficient to call rcu_read_unlock in order
+  * exit the grace period. For classic RCU, a reschedule is required.
+  */
+static void check_hung_rcu_refresh(struct task_struct *g, struct task_struct *t)
+{
+	get_task_struct(g);
+	get_task_struct(t);
+	rcu_read_unlock();
+	cond_resched();
+	rcu_read_lock();
+	put_task_struct(t);
+	put_task_struct(g);
+}
+
 /*
  * Check whether a TASK_UNINTERRUPTIBLE does not get woken up for
  * a really long time (120 seconds). If that happens, print out
@@ -129,8 +147,13 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 
 	rcu_read_lock();
 	do_each_thread(g, t) {
-		if (!--max_count)
-			goto unlock;
+		if (sysctl_hung_task_check_count && !(max_count--)) {
+			max_count = sysctl_hung_task_check_count;
+			check_hung_rcu_refresh(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;
+		}
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);
Signature · 1 line
1.5.4.5
#19 lore raw

Re: [PATCH 2/2 v2] softlockup: check all tasks in hung_task

kernel/hung_task.c static void check_hung_uninterruptible_tasks(unsigned long timeout) · line 151

Quoted 2 lines · Mandeep Singh Baines
kernel/hung_task.c +2 -0
+		if (sysctl_hung_task_check_count && !(max_count--)) {
+			max_count = sysctl_hung_task_check_count;

just a minor nit, why not:

	!--max_count

?

That way we can lose the parenthesis and we'll also not overcount and wont let max_count go down to -1.

Ingo

#20 lore raw

PATCH v3 2/2 softlockup: check all tasks in hung_task

Quoted 8 lines · Ingo Molnar

Review

Quoted 2 lines · Mandeep Singh Baines
(quoted diff) +2 -0
+		if (sysctl_hung_task_check_count && !(max_count--)) {
+			max_count = sysctl_hung_task_check_count;

just a minor nit, why not:

Quoted 8 lines
	!--max_count

?

That way we can lose the parenthesis and we'll also not overcount and wont let max_count go down to -1.

Ingo

Good question. Yeah, there's really nothing about this patch that requires changing the !--max_count test.

Changed the condition back to a simple !--max_count test.

---
Instead of checking only hung_task_check_count tasks, all tasks are checked. hung_task_check_count is still used to put an upper bound on the critical section. Every hung_task_check_count checks, the critical section is refreshed. Keeping the critical section small minimizes time preemption is disabled and keeps rcu grace periods small.

To prevent following a stale pointer, get_task_struct is called on g and t. To verify that g and t have not been unhashed while outside the critical section, the task states are checked.

The design was proposed by Frédéric Weisbecker.

Quoted 15 lines · Frédéric Weisbecker

Instead of having this arbitrary limit of tasks, why not just lurk the need_resched() and then schedule if it needs too.

I know that sounds a bit racy, because you will have to release the tasklist_lock and
a lot of things can happen in the task list until you become resched. But you can do a get_task_struct() on g and t before your thread is going to sleep and then put them
when it is awaken.
Perhaps some tasks will disappear or be appended in the list before g and t, but that doesn't really matter:
if they disappear, they didn't lockup, and if they were appended, they are not enough cold to be analyzed :-)

Quoted 4 lines

This way you can drop the arbitrary limit of task number given by the user....

Frederic.

---
 kernel/hung_task.c |   27 +++++++++++++++++++++++++--
 1 files changed, 25 insertions(+), 2 deletions(-)
kernel/hung_task.c +25 -2
index a841db3..f47eea4 100644
@@ -109,6 +109,24 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
 		panic("hung_task: blocked tasks");
 }
 
+ /*
+  * To avoid extending the RCU grace period for an unbounded amount of time,
+  * periodically exit the critical section and enter a new one.
+  *
+  * For preemptible RCU it is sufficient to call rcu_read_unlock in order
+  * exit the grace period. For classic RCU, a reschedule is required.
+  */
+static void check_hung_rcu_refresh(struct task_struct *g, struct task_struct *t)
+{
+	get_task_struct(g);
+	get_task_struct(t);
+	rcu_read_unlock();
+	cond_resched();
+	rcu_read_lock();
+	put_task_struct(t);
+	put_task_struct(g);
+}
+
 /*
  * Check whether a TASK_UNINTERRUPTIBLE does not get woken up for
  * a really long time (120 seconds). If that happens, print out
@@ -129,8 +147,13 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 
 	rcu_read_lock();
 	do_each_thread(g, t) {
-		if (!--max_count)
-			goto unlock;
+		if (!--max_count) {
+			max_count = sysctl_hung_task_check_count;
+			check_hung_rcu_refresh(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;
+		}
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);
Signature · 1 line
1.5.4.5
#21 lore raw

Re: [PATCH 2/2 v3] softlockup: check all tasks in hung_task

Quoted 1 line · Mandeep Singh Baines

+static void check_hung_rcu_refresh(struct task_struct *g, struct task_struct *t)

please rename this to rcu_lock_break().

kernel/hung_task.c static void check_hung_uninterruptible_tasks(unsigned long timeout) · line 155

Quoted 9 lines
kernel/hung_task.c +6 -2
 	do_each_thread(g, t) {
-		if (!--max_count)
-			goto unlock;
+		if (!--max_count) {
+			max_count = sysctl_hung_task_check_count;
+			check_hung_rcu_refresh(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;

Thinking about it some more, i think a slightly different approach (that has the same end effect):

- Add a "static const int check_count_batching = 1024;" variable that adds some natural batching - and initialize max_count to that value. There's little point to make that batching configurable.

- Leave sysctl_hung_task_check_count present but change its default to something really large like MAX_PID.

Thanks,

Ingo

#22 lore raw

PATCH v4 2/2 softlockup: check all tasks in hung_task

Quoted 7 lines · Ingo Molnar
Quoted 1 line · Mandeep Singh Baines

+static void check_hung_rcu_refresh(struct task_struct *g, struct task_struct *t)

please rename this to rcu_lock_break().

Fixed.

Quoted 13 lines

Review

Quoted 9 lines
(quoted diff) +6 -2
 	do_each_thread(g, t) {
-		if (!--max_count)
-			goto unlock;
+		if (!--max_count) {
+			max_count = sysctl_hung_task_check_count;
+			check_hung_rcu_refresh(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;

Thinking about it some more, i think a slightly different approach (that has the same end effect):

Quoted 4 lines

- Add a "static const int check_count_batching = 1024;" variable that adds some natural batching - and initialize max_count to that value. There's little point to make that batching configurable.

Fixed.

The batch_count controls the preemptibility of hung_task. While it might not make sense to expose the value to user-space, we may want to use a different value for the PREEMPT config (not sure what the specific values should be):

#if defined(CONFIG_PREEMPT) && !defined(CONFIG_PREEMPT_RCU) static const int check_count_batching = 256;
#else
static const int check_count_batching = 2048; #endif

Quoted 2 lines

- Leave sysctl_hung_task_check_count present but change its default to something really large like MAX_PID.

Fixed.

Alternatively, the user could renice khungtaskd in order to control the share of CPU used.

---
Changed the default value of hung_task_check_count to PID_MAX_LIMIT. hung_task_batch_count added to put an upper bound on the critical section. Every hung_task_batch_count checks, the rcu lock is broken. Keeping the critical section small minimizes time preemption is disabled and keeps rcu grace periods small.

To prevent following a stale pointer, get_task_struct is called on g and t. To verify that g and t have not been unhashed while outside the critical section, the task states are checked.

The design was proposed by Frédéric Weisbecker.

Quoted 15 lines · Frédéric Weisbecker

Instead of having this arbitrary limit of tasks, why not just lurk the need_resched() and then schedule if it needs too.

I know that sounds a bit racy, because you will have to release the tasklist_lock and
a lot of things can happen in the task list until you become resched. But you can do a get_task_struct() on g and t before your thread is going to sleep and then put them
when it is awaken.
Perhaps some tasks will disappear or be appended in the list before g and t, but that doesn't really matter:
if they disappear, they didn't lockup, and if they were appended, they are not enough cold to be analyzed :-)

Quoted 4 lines

This way you can drop the arbitrary limit of task number given by the user....

Frederic.

---
 kernel/hung_task.c |   39 +++++++++++++++++++++++++++++++++++++--
 1 files changed, 37 insertions(+), 2 deletions(-)
kernel/hung_task.c +37 -2
index a841db3..34b678c 100644
@@ -17,9 +17,18 @@
 #include <linux/sysctl.h>
 
 /*
- * Have a reasonable limit on the number of tasks checked:
+ * The number of tasks checked:
  */
-unsigned long __read_mostly sysctl_hung_task_check_count = 1024;
+unsigned long __read_mostly sysctl_hung_task_check_count = PID_MAX_LIMIT;
+
+/*
+ * Limit number of tasks checked in a batch.
+ *
+ * This value controls the preemptibility of khungtaskd since preemption
+ * is disabled during the critical section. It also controls the size of
+ * the RCU grace period. So it needs to be upper-bound.
+ */
+static const int hung_task_batching = 1024;
 
 /*
  * Zero means infinite timeout - no checking done:
@@ -109,6 +118,24 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
 		panic("hung_task: blocked tasks");
 }
 
+ /*
+  * To avoid extending the RCU grace period for an unbounded amount of time,
+  * periodically exit the critical section and enter a new one.
+  *
+  * For preemptible RCU it is sufficient to call rcu_read_unlock in order
+  * exit the grace period. For classic RCU, a reschedule is required.
+  */
+static void rcu_lock_break(struct task_struct *g, struct task_struct *t)
+{
+	get_task_struct(g);
+	get_task_struct(t);
+	rcu_read_unlock();
+	cond_resched();
+	rcu_read_lock();
+	put_task_struct(t);
+	put_task_struct(g);
+}
+
 /*
  * Check whether a TASK_UNINTERRUPTIBLE does not get woken up for
  * a really long time (120 seconds). If that happens, print out
@@ -116,6 +143,7 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
  */
 static void check_hung_uninterruptible_tasks(unsigned long timeout)
 {
+	int batch_count = hung_task_batching;
 	int max_count = sysctl_hung_task_check_count;
 	unsigned long now = get_timestamp();
 	struct task_struct *g, *t;
@@ -131,6 +159,13 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	do_each_thread(g, t) {
 		if (!--max_count)
 			goto unlock;
+		if (!--batch_count) {
+			batch_count = hung_task_batching;
+			rcu_lock_break(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;
+		}
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);
Signature · 1 line
1.5.4.5
Applied
#23 lore raw

Re: [PATCH 2/2 v4] softlockup: check all tasks in hung_task

Quoted 10 lines · Mandeep Singh Baines

The batch_count controls the preemptibility of hung_task. While it might not make sense to expose the value to user-space, we may want to use a different value for the PREEMPT config (not sure what the specific values should be):

#if defined(CONFIG_PREEMPT) && !defined(CONFIG_PREEMPT_RCU) static const int check_count_batching = 256;
#else
static const int check_count_batching = 2048; #endif

That really is not worth it - and such #ifdefs always look ugly and split testing as well.

I've applied your patch to tip:core/softlockup, thanks Mandeep!

Ingo

------------->

Quoted 1 line

From 9952001914a7230ab88212a1e2d0e80208bdb907 Mon Sep 17 00:00:00 2001

From: Mandeep Singh Baines <[email protected]>
Date: Wed, 4 Feb 2009 20:35:48 -0800
Subject: [PATCH] softlockup: check all tasks in hung_task
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

Impact: extend the scope of hung-task checks

Changed the default value of hung_task_check_count to PID_MAX_LIMIT. hung_task_batch_count added to put an upper bound on the critical section. Every hung_task_batch_count checks, the rcu lock is never held for a too long time.

Keeping the critical section small minimizes time preemption is disabled and keeps rcu grace periods small.

To prevent following a stale pointer, get_task_struct is called on g and t. To verify that g and t have not been unhashed while outside the critical section, the task states are checked.

The design was proposed by Frédéric Weisbecker.

---
 kernel/hung_task.c |   39 +++++++++++++++++++++++++++++++++++++--
 1 files changed, 37 insertions(+), 2 deletions(-)
kernel/hung_task.c +37 -2
index ba8ccd4..d32d293 100644
@@ -17,9 +17,18 @@
 #include <linux/sysctl.h>
 
 /*
- * Have a reasonable limit on the number of tasks checked:
+ * The number of tasks checked:
  */
-unsigned long __read_mostly sysctl_hung_task_check_count = 1024;
+unsigned long __read_mostly sysctl_hung_task_check_count = PID_MAX_LIMIT;
+
+/*
+ * Limit number of tasks checked in a batch.
+ *
+ * This value controls the preemptibility of khungtaskd since preemption
+ * is disabled during the critical section. It also controls the size of
+ * the RCU grace period. So it needs to be upper-bound.
+ */
+static const int hung_task_batching = 1024;
 
 /*
  * Zero means infinite timeout - no checking done:
@@ -109,6 +118,24 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
 		panic("hung_task: blocked tasks");
 }
 
+ /*
+  * To avoid extending the RCU grace period for an unbounded amount of time,
+  * periodically exit the critical section and enter a new one.
+  *
+  * For preemptible RCU it is sufficient to call rcu_read_unlock in order
+  * exit the grace period. For classic RCU, a reschedule is required.
+  */
+static void rcu_lock_break(struct task_struct *g, struct task_struct *t)
+{
+	get_task_struct(g);
+	get_task_struct(t);
+	rcu_read_unlock();
+	cond_resched();
+	rcu_read_lock();
+	put_task_struct(t);
+	put_task_struct(g);
+}
+
 /*
  * Check whether a TASK_UNINTERRUPTIBLE does not get woken up for
  * a really long time (120 seconds). If that happens, print out
@@ -116,6 +143,7 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
  */
 static void check_hung_uninterruptible_tasks(unsigned long timeout)
 {
+	int batch_count = hung_task_batching;
 	int max_count = sysctl_hung_task_check_count;
 	unsigned long now = get_timestamp();
 	struct task_struct *g, *t;
@@ -131,6 +159,13 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	do_each_thread(g, t) {
 		if (!--max_count)
 			goto unlock;
+		if (!--batch_count) {
+			batch_count = hung_task_batching;
+			rcu_lock_break(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;
+		}
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);
maintainer
#24 lore raw

Re: [PATCH 2/2 v4] softlockup: check all tasks in hung_task

Quoted 5 lines · On Thu, 5 Feb 2009 15:34:53 +0100 Ingo Molnar <[email protected]> wrote:

Subject: [PATCH] softlockup: check all tasks in hung_task

Impact: extend the scope of hung-task checks

A nanonit:

Quoted 1 line

+static const int hung_task_batching = 1024;

static const definitions look pretty but they're a bit misleading.

Quoted 10 lines

static void check_hung_uninterruptible_tasks(unsigned long timeout) {

kernel/hung_task.c +1 -0
+	int batch_count = hung_task_batching;
 	int max_count = sysctl_hung_task_check_count;
 	unsigned long now = get_timestamp();
 	struct task_struct *g, *t;
@@ -131,6 +159,13 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	do_each_thread(g, t) {
 		if (!--max_count)
 			goto unlock;

kernel/hung_task.c static void check_hung_uninterruptible_tasks(unsigned long timeout) · line 171

Quoted 10 lines
kernel/hung_task.c +7 -0
+		if (!--batch_count) {
+			batch_count = hung_task_batching;
+			rcu_lock_break(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;
+		}
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);

The reader of this area of the code will expect that hung_task_batching is a variable. It _looks_ like the value of that variable can be altered at any time by some other thread. It _looks_ like this code will explode if someone has accidentally set hung_task_batching to zero, etc.

But none of that is actually true, because hung_task_batching is, surprisingly, a compile-time constant.

All this misleadingness would be fixed if it were called HUNG_TASK_BATCHING. But then it wouldn't be pretty.

#25 lore raw

Re: [PATCH 2/2 v4] softlockup: check all tasks in hung_task

Quoted 9 lines · Andrew Morton
Quoted 5 lines · On Thu, 5 Feb 2009 15:34:53 +0100 Ingo Molnar <[email protected]> wrote:

Subject: [PATCH] softlockup: check all tasks in hung_task

Impact: extend the scope of hung-task checks

A nanonit:

agreed.

Quoted 30 lines
Quoted 1 line

+static const int hung_task_batching = 1024;

static const definitions look pretty but they're a bit misleading.

Quoted 10 lines

static void check_hung_uninterruptible_tasks(unsigned long timeout) {

(quoted diff) +1 -0
+	int batch_count = hung_task_batching;
 	int max_count = sysctl_hung_task_check_count;
 	unsigned long now = get_timestamp();
 	struct task_struct *g, *t;
@@ -131,6 +159,13 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	do_each_thread(g, t) {
 		if (!--max_count)
 			goto unlock;

Review static void check_hung_uninterruptible_tasks(unsigned long timeout) · line 171

Quoted 10 lines
(quoted diff) +7 -0
+		if (!--batch_count) {
+			batch_count = hung_task_batching;
+			rcu_lock_break(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;
+		}
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);

The reader of this area of the code will expect that hung_task_batching is a variable. It _looks_ like the value of that variable can be altered at any time by some other thread. It _looks_ like this code will explode if someone has accidentally set hung_task_batching to zero, etc.

Quoted 5 lines

But none of that is actually true, because hung_task_batching is, surprisingly, a compile-time constant.

All this misleadingness would be fixed if it were called HUNG_TASK_BATCHING. But then it wouldn't be pretty.

i keep running into this paradox myself too. Explicit const C types are the perfect replacements for defines, but they create confusion by making it look like a variable.

I tend to agree with you that avoiding the confusion is more important than having a type - it's not like we are about to have any type related troubles here. So i amended the commit in the way below - does that look good to you?

Ingo

---------------->

Quoted 1 line

From 9d03ba30018a546d20d4aa8bba58978492c82520 Mon Sep 17 00:00:00 2001

From: Mandeep Singh Baines <[email protected]>
Date: Wed, 4 Feb 2009 20:35:48 -0800
Subject: [PATCH] softlockup: check all tasks in hung_task
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

Impact: extend the scope of hung-task checks

Changed the default value of hung_task_check_count to PID_MAX_LIMIT. hung_task_batch_count added to put an upper bound on the critical section. Every hung_task_batch_count checks, the rcu lock is never held for a too long time.

Keeping the critical section small minimizes time preemption is disabled and keeps rcu grace periods small.

To prevent following a stale pointer, get_task_struct is called on g and t. To verify that g and t have not been unhashed while outside the critical section, the task states are checked.

The design was proposed by Frédéric Weisbecker.

---
 kernel/hung_task.c |   39 +++++++++++++++++++++++++++++++++++++--
 1 files changed, 37 insertions(+), 2 deletions(-)
kernel/hung_task.c +37 -2
index ba8ccd4..3c6190b 100644
@@ -17,9 +17,18 @@
 #include <linux/sysctl.h>
 
 /*
- * Have a reasonable limit on the number of tasks checked:
+ * The number of tasks checked:
  */
-unsigned long __read_mostly sysctl_hung_task_check_count = 1024;
+unsigned long __read_mostly sysctl_hung_task_check_count = PID_MAX_LIMIT;
+
+/*
+ * Limit number of tasks checked in a batch.
+ *
+ * This value controls the preemptibility of khungtaskd since preemption
+ * is disabled during the critical section. It also controls the size of
+ * the RCU grace period. So it needs to be upper-bound.
+ */
+#define HUNG_TASK_BATCHING 1024;
 
 /*
  * Zero means infinite timeout - no checking done:
@@ -110,6 +119,24 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
 }
 
 /*
+ * To avoid extending the RCU grace period for an unbounded amount of time,
+ * periodically exit the critical section and enter a new one.
+ *
+ * For preemptible RCU it is sufficient to call rcu_read_unlock in order
+ * exit the grace period. For classic RCU, a reschedule is required.
+ */
+static void rcu_lock_break(struct task_struct *g, struct task_struct *t)
+{
+	get_task_struct(g);
+	get_task_struct(t);
+	rcu_read_unlock();
+	cond_resched();
+	rcu_read_lock();
+	put_task_struct(t);
+	put_task_struct(g);
+}
+
+/*
  * Check whether a TASK_UNINTERRUPTIBLE does not get woken up for
  * a really long time (120 seconds). If that happens, print out
  * a warning.
@@ -117,6 +144,7 @@ static void check_hung_task(struct task_struct *t, unsigned long now,
 static void check_hung_uninterruptible_tasks(unsigned long timeout)
 {
 	int max_count = sysctl_hung_task_check_count;
+	int batch_count = HUNG_TASK_BATCHING;
 	unsigned long now = get_timestamp();
 	struct task_struct *g, *t;
 
@@ -131,6 +159,13 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	do_each_thread(g, t) {
 		if (!--max_count)
 			goto unlock;
+		if (!--batch_count) {
+			batch_count = HUNG_TASK_BATCHING;
+			rcu_lock_break(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;
+		}
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);
maintainer
#26 lore raw

Re: [PATCH 2/2 v4] softlockup: check all tasks in hung_task

Quoted 3 lines · On Thu, 5 Feb 2009 19:07:49 +0100 Ingo Molnar <[email protected]> wrote:

So i amended the commit in the way below - does that look good to you?

yup.

Quoted 1 line

+#define HUNG_TASK_BATCHING 1024;

s/;//

#27 lore raw

Re: [PATCH 2/2 v4] softlockup: check all tasks in hung_task

Quoted 7 lines · Andrew Morton
Quoted 3 lines · On Thu, 5 Feb 2009 19:07:49 +0100 Ingo Molnar <[email protected]> wrote:

So i amended the commit in the way below - does that look good to you?

yup.

added your ack - thanks!

Quoted 3 lines
Quoted 1 line

+#define HUNG_TASK_BATCHING 1024;

s/;//

fixed.

Ingo

#28 lore raw

Re: [PATCH 2/2 v4] softlockup: check all tasks in hung_task

In reply to Andrew Morton
Quoted 31 lines · Andrew Morton
Quoted 5 lines · On Thu, 5 Feb 2009 15:34:53 +0100 Ingo Molnar <[email protected]> wrote:

Subject: [PATCH] softlockup: check all tasks in hung_task

Impact: extend the scope of hung-task checks

A nanonit:

Quoted 1 line

+static const int hung_task_batching = 1024;

static const definitions look pretty but they're a bit misleading.

Quoted 17 lines

static void check_hung_uninterruptible_tasks(unsigned long timeout) {

kernel/hung_task.c +8 -0
+	int batch_count = hung_task_batching;
 	int max_count = sysctl_hung_task_check_count;
 	unsigned long now = get_timestamp();
 	struct task_struct *g, *t;
@@ -131,6 +159,13 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	do_each_thread(g, t) {
 		if (!--max_count)
 			goto unlock;
+		if (!--batch_count) {
+			batch_count = hung_task_batching;
+			rcu_lock_break(g, t);
+			/* Exit if t or g was unhashed during refresh. */
+			if (t->state == TASK_DEAD || g->state == TASK_DEAD)
+				goto unlock;
+		}
Quoted 9 lines

kernel/hung_task.c static void check_hung_uninterruptible_tasks(unsigned long timeout) · line 171

Quoted 3 lines
kernel/hung_task.c +0 -0
 		/* use "==" to skip the TASK_KILLABLE tasks waiting on NFS */
 		if (t->state == TASK_UNINTERRUPTIBLE)
 			check_hung_task(t, now, timeout);

The reader of this area of the code will expect that hung_task_batching is a variable. It _looks_ like the value of that variable can be altered at any time by some other thread. It _looks_ like this code will explode if someone has accidentally set hung_task_batching to zero, etc.

The code would not break if hung_task_batching was exported out as a sysctl.

If hung_task_batching is set to zero at any time, the behavior will be to process all tasks in one batch. This seems like a reasonable behavior for the zero case. It is also consistent with the behavior of sysctl_hung_task_check_count.

Maybe a comment should be added by the declaration of both variables explaining the zero behavior?

Quoted 6 lines

But none of that is actually true, because hung_task_batching is, surprisingly, a compile-time constant.

All this misleadingness would be fixed if it were called HUNG_TASK_BATCHING. But then it wouldn't be pretty.

#29 lore raw

PATCH 1/1 softlockup: convert read_lock in hung_task to rcu_read_lock

In reply to Ingo Molnar
Quoted 3 lines · Ingo Molnar

I've applied your patch to tip:core/softlockup, thanks Mandeep!

Oops, I put the conversion to rcu_read_lock in a seperate patch. I understand now the convention is to use a patch series only when the patches can be applied independently.

---
Since the tasklist is protected by rcu list operations, it is safe to convert the read_lock()s to rcu_read_lock().

---
 kernel/hung_task.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
kernel/hung_task.c +2 -2
index d32d293..34b678c 100644
@@ -155,7 +155,7 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	if (test_taint(TAINT_DIE) || did_panic)
 		return;
 
-	read_lock(&tasklist_lock);
+	rcu_read_lock();
 	do_each_thread(g, t) {
 		if (!--max_count)
 			goto unlock;
@@ -171,7 +171,7 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 			check_hung_task(t, now, timeout);
 	} while_each_thread(g, t);
  unlock:
-	read_unlock(&tasklist_lock);
+	rcu_read_unlock();
 }
 
 static void update_poll_jiffies(void)
Signature · 1 line
1.5.4.5
Applied
#30 lore raw

Re: [PATCH] softlockup: convert read_lock in hung_task to rcu_read_lock

Quoted 8 lines · Mandeep Singh Baines
Quoted 3 lines · Ingo Molnar

I've applied your patch to tip:core/softlockup, thanks Mandeep!

Oops, I put the conversion to rcu_read_lock in a seperate patch. I understand now the convention is to use a patch series only when the patches can be applied independently.

What you did is in fact the preferred approach: keep independent stuff independent and make patches as gradual and split-up as possible. (as long as the interim state is a working kernel too)

Quoted 9 lines

---
Since the tasklist is protected by rcu list operations, it is safe to convert the read_lock()s to rcu_read_lock().

---
 kernel/hung_task.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

Applied to tip:core/softlockup, thanks!

Ingo