Skip to contentSkip to conversation

sched: Fix NULL user_cpus_ptr check in dup_user_cpus_ptr()

PATCH Superseded Waiman Long 2 participants 3 messages Nov 22, 2022 – Nov 23, 2022 Scheduler

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sched: Fix NULL user_cpus_ptr check in dup_user_cpus_ptr()

In general, a non-null user_cpus_ptr will remain set until the task dies. A possible exception to this is the fact that do_set_cpus_allowed() will clear a non-null user_cpus_ptr. To allow this possible racing condition, we need to check for NULL user_cpus_ptr under the pi_lock before duping the user mask.

Fixes 851a723e45d1 ("sched: Always clear user_cpus_ptr in do_set_cpus_allowed()")
---
 kernel/sched/core.c | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)
kernel/sched/core.c +21 -2
index 8df51b08bb38..f447a6285ea2 100644
@@ -2625,7 +2625,14 @@ int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src,
 		      int node)
 {
 	unsigned long flags;
+	cpumask_t *user_mask = NULL;
 
+	/*
+	 * If there is a concurrent sched_setaffinity(), we may miss the
+	 * newly updated user_cpus_ptr. However, a non-NULL user_cpus_ptr
+	 * is relatively unlikely and it is not worth the extra overhead
+	 * of taking the pi_lock on every fork/clone.
+	 */
 	if (!src->user_cpus_ptr)
 		return 0;
 
@@ -2633,10 +2640,22 @@ int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src,
 	if (!dst->user_cpus_ptr)
 		return -ENOMEM;
 
-	/* Use pi_lock to protect content of user_cpus_ptr */
+	/*
+	 * Use pi_lock to protect content of user_cpus_ptr
+	 *
+	 * Though unlikely, user_cpus_ptr can be reset to NULL by a concurrent
+	 * do_set_cpus_allowed().
+	 */
 	raw_spin_lock_irqsave(&src->pi_lock, flags);
-	cpumask_copy(dst->user_cpus_ptr, src->user_cpus_ptr);
+	if (src->user_cpus_ptr)
+		cpumask_copy(dst->user_cpus_ptr, src->user_cpus_ptr);
+	else
+		swap(dst->user_cpus_ptr, user_mask);
 	raw_spin_unlock_irqrestore(&src->pi_lock, flags);
+
+	if (unlikely(user_mask))
+		kfree(user_mask);
+
 	return 0;
 }
 
Signature · 1 line
2.31.1
maintainer
#1 lore raw
Quoted 17 lines · Waiman Long

In general, a non-null user_cpus_ptr will remain set until the task dies. A possible exception to this is the fact that do_set_cpus_allowed() will clear a non-null user_cpus_ptr. To allow this possible racing condition, we need to check for NULL user_cpus_ptr under the pi_lock before duping the user mask.

Fixes 851a723e45d1 ("sched: Always clear user_cpus_ptr in do_set_cpus_allowed()")
---
 kernel/sched/core.c | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)
kernel/sched/core.c +0 -0
index 8df51b08bb38..f447a6285ea2 100644

kernel/sched/core.c int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, · line 2628

Quoted 4 lines
kernel/sched/core.c +1 -0
@@ -2625,7 +2625,14 @@ int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src,
 		      int node)
 {
 	unsigned long flags;
+	cpumask_t *user_mask = NULL;

The inverse xmas tree is sad :-(

kernel/sched/core.c int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, · line 2635

Quoted 7 lines
kernel/sched/core.c +6 -0
+	/*
+	 * If there is a concurrent sched_setaffinity(), we may miss the
+	 * newly updated user_cpus_ptr. However, a non-NULL user_cpus_ptr
+	 * is relatively unlikely and it is not worth the extra overhead
+	 * of taking the pi_lock on every fork/clone.
+	 */

I think the correct argument is saying the thing is racy and loosing the race is a valid situation. IOW, this is the same as if the concurrent sched_setaffinity() happens after fork().

Quoted 9 lines
kernel/sched/core.c +1 -1
 	if (!src->user_cpus_ptr)
 		return 0;
 
@@ -2633,10 +2640,22 @@ int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src,
 	if (!dst->user_cpus_ptr)
 		return -ENOMEM;
 
-	/* Use pi_lock to protect content of user_cpus_ptr */
+	/*

kernel/sched/core.c int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, · line 2653

Quoted 10 lines
kernel/sched/core.c +9 -0
+	 * Use pi_lock to protect content of user_cpus_ptr
+	 *
+	 * Though unlikely, user_cpus_ptr can be reset to NULL by a concurrent
+	 * do_set_cpus_allowed().
+	 */
 	raw_spin_lock_irqsave(&src->pi_lock, flags);
+	if (src->user_cpus_ptr)
+		cpumask_copy(dst->user_cpus_ptr, src->user_cpus_ptr);
+	else
+		swap(dst->user_cpus_ptr, user_mask);

Uhhhh, did you mean to write:

	if (src->user_cpus_ptr) {
		cpumask_copy(user_mask, src->user_cpus_ptr);
		swap(dst->user_cpus_ptr, user_mask);
	}

?

Quoted 11 lines
kernel/sched/core.c +4 -0
 	raw_spin_unlock_irqrestore(&src->pi_lock, flags);
+
+	if (unlikely(user_mask))
+		kfree(user_mask);
+
 	return 0;
 }
 
Signature · 1 line
2.31.1
#2 lore raw
Quoted 13 lines · Peter Zijlstra
Quoted 12 lines · Waiman Long

In general, a non-null user_cpus_ptr will remain set until the task dies. A possible exception to this is the fact that do_set_cpus_allowed() will clear a non-null user_cpus_ptr. To allow this possible racing condition, we need to check for NULL user_cpus_ptr under the pi_lock before duping the user mask.

Fixes 851a723e45d1 ("sched: Always clear user_cpus_ptr in do_set_cpus_allowed()")
---
  kernel/sched/core.c | 23 +++++++++++++++++++++--
  1 file changed, 21 insertions(+), 2 deletions(-)
Quoted 10 lines

kernel/sched/core.c int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, · line 2628

Quoted 9 lines
kernel/sched/core.c +1 -0
index 8df51b08bb38..f447a6285ea2 100644
@@ -2625,7 +2625,14 @@ int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src,
  		      int node)
  {
  	unsigned long flags;
+	cpumask_t *user_mask = NULL;

The inverse xmas tree is sad :-(

Right. The inverse xmas tree rule. Will fix that.

Quoted 11 lines

kernel/sched/core.c int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, · line 2635

Quoted 8 lines
kernel/sched/core.c +6 -0
+	/*
+	 * If there is a concurrent sched_setaffinity(), we may miss the
+	 * newly updated user_cpus_ptr. However, a non-NULL user_cpus_ptr
+	 * is relatively unlikely and it is not worth the extra overhead
+	 * of taking the pi_lock on every fork/clone.
+	 */

I think the correct argument is saying the thing is racy and loosing the race is a valid situation. IOW, this is the same as if the concurrent sched_setaffinity() happens after fork().

Good point, will update the comment.

Quoted 21 lines
Quoted 9 lines
kernel/sched/core.c +1 -1
  	if (!src->user_cpus_ptr)
  		return 0;
 
@@ -2633,10 +2640,22 @@ int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src,
  	if (!dst->user_cpus_ptr)
  		return -ENOMEM;
 
-	/* Use pi_lock to protect content of user_cpus_ptr */
+	/*

kernel/sched/core.c int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src, · line 2653

Quoted 10 lines
kernel/sched/core.c +9 -0
+	 * Use pi_lock to protect content of user_cpus_ptr
+	 *
+	 * Though unlikely, user_cpus_ptr can be reset to NULL by a concurrent
+	 * do_set_cpus_allowed().
+	 */
  	raw_spin_lock_irqsave(&src->pi_lock, flags);
+	if (src->user_cpus_ptr)
+		cpumask_copy(dst->user_cpus_ptr, src->user_cpus_ptr);
+	else
+		swap(dst->user_cpus_ptr, user_mask);

Uhhhh, did you mean to write:

Quoted 6 lines
	if (src->user_cpus_ptr) {
		cpumask_copy(user_mask, src->user_cpus_ptr);
		swap(dst->user_cpus_ptr, user_mask);
	}

?

Not really. The point is that dst->user_cpus_ptr has been allocated. If src->user_cpus_ptr turns out to be NULL, we need to clear dst->user_cpus_ptr which is what the swap() does and then free that memory after unlock. Will add a comment to make this point clear.

Cheers,
Longman