Skip to contentSkip to conversation

fix lock inversion in lockless sys_times()

PATCH In review [email protected] 3 participants 6 messages Sep 12, 2014 – Sep 19, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover fix lock inversion in lockless sys_times()

The sig->stats_lock nests inside the tasklist_lock and the sighand->siglock in __exit_signal and wait_task_zombie.

However, both of those locks can be taken from irq context, which means we need to use the interrupt safe variant of read_seqbegin_or_lock. This blocks interrupts when the "lock" branch is taken (seq is odd), preventing the lock inversion.

On the first (lockless) pass through the loop, irqs are not blocked.

This fixes the lockdep complaints that Stanislaw reported.

#1 lore raw

PATCH 1/2 seqlock: add irqsave variant of read_seqbegin_or_lock

From: Rik van Riel <[email protected]>

There are cases where read_seqbegin_or_lock needs to block irqs, because the seqlock in question nests inside a lock that is also be taken from irq context.

Add read_seqbegin_or_lock_irqsave and done_seqretry_irqrestore, which are almost identical to read_seqbegin_or_lock and done_seqretry.

---
 include/linux/seqlock.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
include/linux/seqlock.h +17 -0
index cc35963..4364cd3 100644
@@ -456,4 +456,21 @@ read_sequnlock_excl_irqrestore(seqlock_t *sl, unsigned long flags)
 	spin_unlock_irqrestore(&sl->lock, flags);
 }
 
+static inline unsigned long read_seqbegin_or_lock_irqsave(seqlock_t *lock,
+							  int *seq)
+{
+	unsigned long flags = 0;
+	if (!(*seq & 1))	/* Even */
+		*seq = read_seqbegin(lock);
+	else			/* Odd */
+		read_seqlock_excl_irqsave(lock, flags);
+	return flags;
+}
+
+static inline void done_seqretry_irqrestore(seqlock_t *lock, int seq,
+					    unsigned long flags)
+{
+	if (seq & 1)
+		read_sequnlock_excl_irqrestore(lock, flags);
+}
 #endif /* __LINUX_SEQLOCK_H */
Signature · 1 line
1.9.3
#2 lore raw

[tip:sched/core] seqlock: Add irqsave variant of read_seqbegin_or_lock()

seqlock: Add irqsave variant of read_seqbegin_or_lock()
Commit-ID:  ef8ac06359ddf95431cf6bb04ad2b36fff562328
Gitweb:     http://git.kernel.org/tip/ef8ac06359ddf95431cf6bb04ad2b36fff562328
Author:     Rik van Riel <[email protected]>
AuthorDate: Fri, 12 Sep 2014 09:12:14 -0400
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 19 Sep 2014 12:35:16 +0200

seqlock: Add irqsave variant of read_seqbegin_or_lock()

There are cases where read_seqbegin_or_lock() needs to block irqs, because the seqlock in question nests inside a lock that is also be taken from irq context.

Add read_seqbegin_or_lock_irqsave() and done_seqretry_irqrestore(), which are almost identical to read_seqbegin_or_lock() and done_seqretry().

Cc Al Viro <[email protected]>
Cc John Stultz <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc Stephen Boyd <[email protected]>
Cc Trond Myklebust <[email protected]>
[ Improved the readability of the code a bit. ]
---
 include/linux/seqlock.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
include/linux/seqlock.h +19 -0
index cc35963..f5df8f6 100644
@@ -456,4 +456,23 @@ read_sequnlock_excl_irqrestore(seqlock_t *sl, unsigned long flags)
 	spin_unlock_irqrestore(&sl->lock, flags);
 }
 
+static inline unsigned long
+read_seqbegin_or_lock_irqsave(seqlock_t *lock, int *seq)
+{
+	unsigned long flags = 0;
+
+	if (!(*seq & 1))	/* Even */
+		*seq = read_seqbegin(lock);
+	else			/* Odd */
+		read_seqlock_excl_irqsave(lock, flags);
+
+	return flags;
+}
+
+static inline void
+done_seqretry_irqrestore(seqlock_t *lock, int seq, unsigned long flags)
+{
+	if (seq & 1)
+		read_sequnlock_excl_irqrestore(lock, flags);
+}
 #endif /* __LINUX_SEQLOCK_H */
#3 lore raw

PATCH 2/2 sched,time: fix lock inversion in thread_group_cputime

In reply to [email protected]

From: Rik van Riel <[email protected]>

The sig->stats_lock nests inside the tasklist_lock and the sighand->siglock in __exit_signal and wait_task_zombie.

However, both of those locks can be taken from irq context, which means we need to use the interrupt safe variant of read_seqbegin_or_lock. This blocks interrupts when the "lock" branch is taken (seq is odd), preventing the lock inversion.

On the first (lockless) pass through the loop, irqs are not blocked.

---
 kernel/sched/cputime.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/sched/cputime.c +3 -2
index 2b57031..64492df 100644
@@ -289,13 +289,14 @@ void thread_group_cputime(struct task_struct *tsk, struct task_cputime *times)
 	cputime_t utime, stime;
 	struct task_struct *t;
 	unsigned int seq, nextseq;
+	unsigned long flags;
 
 	rcu_read_lock();
 	/* Attempt a lockless read on the first round. */
 	nextseq = 0;
 	do {
 		seq = nextseq;
-		read_seqbegin_or_lock(&sig->stats_lock, &seq);
+		flags = read_seqbegin_or_lock_irqsave(&sig->stats_lock, &seq);
 		times->utime = sig->utime;
 		times->stime = sig->stime;
 		times->sum_exec_runtime = sig->sum_sched_runtime;
@@ -309,7 +310,7 @@ void thread_group_cputime(struct task_struct *tsk, struct task_cputime *times)
 		/* If lockless access failed, take the lock. */
 		nextseq = 1;
 	} while (need_seqretry(&sig->stats_lock, seq));
-	done_seqretry(&sig->stats_lock, seq);
+	done_seqretry_irqrestore(&sig->stats_lock, seq, flags);
 	rcu_read_unlock();
 }
 
Signature · 1 line
1.9.3
#4 lore raw

[tip:sched/core] sched, time: Fix lock inversion in thread_group_cputime()

sched, time: Fix lock inversion in thread_group_cputime()
Commit-ID:  9c368b5b6eccce1cbd7f68142106b3b4ddb1c5b5
Gitweb:     http://git.kernel.org/tip/9c368b5b6eccce1cbd7f68142106b3b4ddb1c5b5
Author:     Rik van Riel <[email protected]>
AuthorDate: Fri, 12 Sep 2014 09:12:15 -0400
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 19 Sep 2014 12:35:17 +0200

sched, time: Fix lock inversion in thread_group_cputime()

The sig->stats_lock nests inside the tasklist_lock and the sighand->siglock in __exit_signal and wait_task_zombie.

However, both of those locks can be taken from irq context, which means we need to use the interrupt safe variant of read_seqbegin_or_lock. This blocks interrupts when the "lock" branch is taken (seq is odd), preventing the lock inversion.

On the first (lockless) pass through the loop, irqs are not blocked.

---
 kernel/sched/cputime.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/sched/cputime.c +3 -2
index 2b57031..64492df 100644
@@ -289,13 +289,14 @@ void thread_group_cputime(struct task_struct *tsk, struct task_cputime *times)
 	cputime_t utime, stime;
 	struct task_struct *t;
 	unsigned int seq, nextseq;
+	unsigned long flags;
 
 	rcu_read_lock();
 	/* Attempt a lockless read on the first round. */
 	nextseq = 0;
 	do {
 		seq = nextseq;
-		read_seqbegin_or_lock(&sig->stats_lock, &seq);
+		flags = read_seqbegin_or_lock_irqsave(&sig->stats_lock, &seq);
 		times->utime = sig->utime;
 		times->stime = sig->stime;
 		times->sum_exec_runtime = sig->sum_sched_runtime;
@@ -309,7 +310,7 @@ void thread_group_cputime(struct task_struct *tsk, struct task_cputime *times)
 		/* If lockless access failed, take the lock. */
 		nextseq = 1;
 	} while (need_seqretry(&sig->stats_lock, seq));
-	done_seqretry(&sig->stats_lock, seq);
+	done_seqretry_irqrestore(&sig->stats_lock, seq, flags);
 	rcu_read_unlock();
 }
 
maintainer
#5 lore raw
In reply to [email protected]
Quoted 8 lines · On Fri, Sep 12, 2014 at 09:12:13AM -0400, [email protected] wrote:

The sig->stats_lock nests inside the tasklist_lock and the sighand->siglock in __exit_signal and wait_task_zombie.

However, both of those locks can be taken from irq context, which means we need to use the interrupt safe variant of read_seqbegin_or_lock. This blocks interrupts when the "lock" branch is taken (seq is odd), preventing the lock inversion.

Quoted 5 lines

On the first (lockless) pass through the loop, irqs are not blocked.

This fixes the lockdep complaints that Stanislaw reported.

Thanks Rik