Skip to contentSkip to conversation

signal: Avoid preempt_disable() in ptrace_stop() on PREEMPT_RT

PATCH Superseded Sebastian Andrzej Siewior 3 participants 9 messages Jun 6, 2023

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover signal: Avoid preempt_disable() in ptrace_stop() on PREEMPT_RT

this mini series updates the comment to properly explain why the preempt-disable section has been added and then disables this on PREEMPT_RT explaining why it can not be done.

This is a repost of

	https://lore.kernel.org/[email protected]

Sebastian

#1 lore raw

PATCH 1/2 signal: Add proper comment about the preempt-disable in ptrace_stop().

Commit 53da1d9456fe7 ("fix ptrace slowness") added a preempt-disable section between read_unlock() and the following schedule() invocation without explaining why it is needed.

Replace the comment with an explanation why this is needed. Clarify that it is needed for correctness but for performance reasons.

---
 kernel/signal.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
kernel/signal.c +14 -3
index 2547fa73bde51..da017a5461163 100644
@@ -2313,10 +2313,21 @@ static int ptrace_stop(int exit_code, int why, unsigned long message,
 		do_notify_parent_cldstop(current, false, why);
 
 	/*
-	 * Don't want to allow preemption here, because
-	 * sys_ptrace() needs this task to be inactive.
+	 * The previous do_notify_parent_cldstop() invocation woke ptracer.
+	 * One a PREEMPTION kernel this can result in preemption requirement
+	 * which will be fulfilled after read_unlock() and the ptracer will be
+	 * put on the CPU.
+	 * The ptracer is in wait_task_inactive(, __TASK_TRACED) waiting for
+	 * this task wait in schedule(). If this task gets preempted then it
+	 * remains enqueued on the runqueue. The ptracer will observe this and
+	 * then sleep for a delay of one HZ tick. In the meantime this task
+	 * gets scheduled, enters schedule() and will wait for the ptracer.
 	 *
-	 * XXX: implement read_unlock_no_resched().
+	 * This preemption point is not bad from correctness point of view but
+	 * extends the runtime by one HZ tick time due to the ptracer's sleep.
+	 * The preempt-disable section ensures that there will be no preemption
+	 * between unlock and schedule() and so improving the performance since
+	 * the ptracer has no reason to sleep.
 	 */
 	preempt_disable();
 	read_unlock(&tasklist_lock);
Signature · 1 line
2.40.1
Acked-by on v1 1/2
#2 lore raw

Re: [PATCH REPOST 1/2] signal: Add proper comment about the preempt-disable in ptrace_stop().

Quoted 9 lines · Sebastian Andrzej Siewior

Commit 53da1d9456fe7 ("fix ptrace slowness") added a preempt-disable section between read_unlock() and the following schedule() invocation without explaining why it is needed.

Replace the comment with an explanation why this is needed. Clarify that it is needed for correctness but for performance reasons.

#3 lore raw

PATCH 2/2 signal: Don't disable preemption in ptrace_stop() on PREEMPT_RT.

In reply to Sebastian Andrzej Siewior

On PREEMPT_RT keeping preemption disabled during the invocation of cgroup_enter_frozen() is a problem because the function acquires css_set_lock which is a sleeping lock on PREEMPT_RT and must not be acquired with disabled preemption.
The preempt-disabled section is only for performance optimisation reasons and can be avoided.

Extend the comment and don't disable preemption before scheduling on PREEMPT_RT.

---
 kernel/signal.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
kernel/signal.c +7 -2
index da017a5461163..9e07b3075c72e 100644
@@ -2328,11 +2328,16 @@ static int ptrace_stop(int exit_code, int why, unsigned long message,
 	 * The preempt-disable section ensures that there will be no preemption
 	 * between unlock and schedule() and so improving the performance since
 	 * the ptracer has no reason to sleep.
+	 *
+	 * This optimisation is not doable on PREEMPT_RT due to the spinlock_t
+	 * within the preempt-disable section.
 	 */
-	preempt_disable();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		preempt_disable();
 	read_unlock(&tasklist_lock);
 	cgroup_enter_frozen();
-	preempt_enable_no_resched();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		preempt_enable_no_resched();
 	schedule();
 	cgroup_leave_frozen(true);
 
Signature · 1 line
2.40.1
#4 lore raw

Re: [PATCH REPOST 2/2] signal: Don't disable preemption in ptrace_stop() on PREEMPT_RT.

The patch LGTM, but I am a bit confused by the changelog/comments, I guess I missed something...

Quoted 4 lines · Sebastian Andrzej Siewior

kernel/signal.c static int ptrace_stop(int exit_code, int why, unsigned long message, · line 2336

Quoted 10 lines
kernel/signal.c +5 -1
@@ -2328,11 +2328,16 @@ static int ptrace_stop(int exit_code, int why, unsigned long message,
 	 * The preempt-disable section ensures that there will be no preemption
 	 * between unlock and schedule() and so improving the performance since
 	 * the ptracer has no reason to sleep.
+	 *
+	 * This optimisation is not doable on PREEMPT_RT due to the spinlock_t
+	 * within the preempt-disable section.
 	 */
-	preempt_disable();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		preempt_disable();

Not only we the problems with cgroup_enter_frozen(), afaics (please correct me) this optimisation doesn't work on RT anyway?

IIUC, read_lock() on RT disables migration but not preemption, so it is simply too late to do preempt_disable() before unlock/schedule. The tracer can preempt the tracee right after do_notify_parent_cldstop().

Oleg.

#5 lore raw

Re: [PATCH REPOST 2/2] signal: Don't disable preemption in ptrace_stop() on PREEMPT_RT.

Quoted 22 lines · Oleg Nesterov

The patch LGTM, but I am a bit confused by the changelog/comments, I guess I missed something...

Quoted 4 lines · Sebastian Andrzej Siewior

kernel/signal.c static int ptrace_stop(int exit_code, int why, unsigned long message, · line 2336

Quoted 10 lines
kernel/signal.c +5 -1
@@ -2328,11 +2328,16 @@ static int ptrace_stop(int exit_code, int why, unsigned long message,
 	 * The preempt-disable section ensures that there will be no preemption
 	 * between unlock and schedule() and so improving the performance since
 	 * the ptracer has no reason to sleep.
+	 *
+	 * This optimisation is not doable on PREEMPT_RT due to the spinlock_t
+	 * within the preempt-disable section.
 	 */
-	preempt_disable();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		preempt_disable();

Not only we the problems with cgroup_enter_frozen(), afaics (please correct me) this optimisation doesn't work on RT anyway?

Quoted 3 lines

IIUC, read_lock() on RT disables migration but not preemption, so it is simply too late to do preempt_disable() before unlock/schedule. The tracer can preempt the tracee right after do_notify_parent_cldstop().

Correct -- but I think you can disable preemption over what is effectivly rwsem_up_read(), but you can't over the effective rtmutex_lock() that cgroup_enter_frozen() will then attempt.

(iow, unlock() doesn't tend to sleep, while lock() does)

But you're correct to point out that the whole preempt_disable() thing is entirely pointless due to the whole task_lock region being preemptible before it.

#6 lore raw

Re: [PATCH REPOST 2/2] signal: Don't disable preemption in ptrace_stop() on PREEMPT_RT.

Quoted 32 lines · Peter Zijlstra
Quoted 22 lines · Oleg Nesterov

The patch LGTM, but I am a bit confused by the changelog/comments, I guess I missed something...

Quoted 4 lines · Sebastian Andrzej Siewior

kernel/signal.c static int ptrace_stop(int exit_code, int why, unsigned long message, · line 2336

Quoted 10 lines
kernel/signal.c +5 -1
@@ -2328,11 +2328,16 @@ static int ptrace_stop(int exit_code, int why, unsigned long message,
 	 * The preempt-disable section ensures that there will be no preemption
 	 * between unlock and schedule() and so improving the performance since
 	 * the ptracer has no reason to sleep.
+	 *
+	 * This optimisation is not doable on PREEMPT_RT due to the spinlock_t
+	 * within the preempt-disable section.
 	 */
-	preempt_disable();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		preempt_disable();

Not only we the problems with cgroup_enter_frozen(), afaics (please correct me) this optimisation doesn't work on RT anyway?

Quoted 3 lines

IIUC, read_lock() on RT disables migration but not preemption, so it is simply too late to do preempt_disable() before unlock/schedule. The tracer can preempt the tracee right after do_notify_parent_cldstop().

Correct -- but I think you can disable preemption over what is effectivly rwsem_up_read(), but you can't over the effective rtmutex_lock() that cgroup_enter_frozen() will then attempt.

Quoted 5 lines

(iow, unlock() doesn't tend to sleep, while lock() does)

But you're correct to point out that the whole preempt_disable() thing is entirely pointless due to the whole task_lock region being preemptible before it.

Thanks Peter.

So I think the comment should be updated. Otherwise it looks as if it makes sense to try to move cgroup_enter_frozen() up before preempt_disable().

Oleg.

#7 lore raw

PATCH v2 2/2 signal: Don't disable preemption in ptrace_stop() on PREEMPT_RT.

On PREEMPT_RT keeping preemption disabled during the invocation of cgroup_enter_frozen() is a problem because the function acquires css_set_lock which is a sleeping lock on PREEMPT_RT and must not be acquired with disabled preemption.
The preempt-disabled section is only for performance optimisation reasons and can be avoided.

Extend the comment and don't disable preemption before scheduling on PREEMPT_RT.

---

Is this better?

v1…v2:
- Extend the comment to note that preemption isn't disabled due to the lock to make it obvious that the optimisation isn't just harmful but also pointless.

 kernel/signal.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
kernel/signal.c +11 -2
index da017a5461163..dcb0b1fbcb3a8 100644
@@ -2328,11 +2328,20 @@ static int ptrace_stop(int exit_code, int why, unsigned long message,
 	 * The preempt-disable section ensures that there will be no preemption
 	 * between unlock and schedule() and so improving the performance since
 	 * the ptracer has no reason to sleep.
+	 *
+	 * On PREEMPT_RT locking tasklist_lock does not disable preemption.
+	 * Therefore the task can be preempted (after
+	 * do_notify_parent_cldstop()) before unlocking tasklist_lock so there
+	 * is no benefit in doing this. The optimisation is harmful on
+	 * PEEMPT_RT because the spinlock_t (in cgroup_enter_frozen()) must not
+	 * be acquired with disabled preemption.
 	 */
-	preempt_disable();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		preempt_disable();
 	read_unlock(&tasklist_lock);
 	cgroup_enter_frozen();
-	preempt_enable_no_resched();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		preempt_enable_no_resched();
 	schedule();
 	cgroup_leave_frozen(true);
 
Signature · 1 line
2.40.1
Acked-by on v2 2/2
#8 lore raw

Re: [PATCH 2/2 v2] signal: Don't disable preemption in ptrace_stop() on PREEMPT_RT.

Quoted 5 lines · Sebastian Andrzej Siewior

v1…v2:
- Extend the comment to note that preemption isn't disabled due to the lock to make it obvious that the optimisation isn't just harmful but also pointless.

Thanks,