x86, ptrace, bts: stop bts tracing early in do_exit
Subsystems (2)
PATCH 3/14 x86, ptrace, bts: stop bts tracing early in do_exit
Bts tracing had been stopped in __ptrace_unlink() which is called with interrupts disabled and tasklist_lock write-held.
Release the bts tracer early in do_exit() when interrupts are enabled and tasklist_lock is not held.
Resolve races between exit and detach by atomically exchanging the bts tracer pointer with NULL.
---
arch/x86/include/asm/ptrace.h +3 -1
@@ -235,12 +235,14 @@ extern int do_get_thread_area(struct tasextern int do_set_thread_area (struct task_struct *p,int idx,struct user_desc __user *info,int can_allocate); -extern void x86_ptrace_untrace (struct task_struct *); +extern void x86_ptrace_untrace (struct task_struct *child);extern void x86_ptrace_fork (struct task_struct *child,unsigned long clone_flags); +extern void x86_ptrace_report_exit (long exit_code);#define arch_ptrace_untrace (tsk) x86_ptrace_untrace(tsk)#define arch_ptrace_fork (child, flags) x86_ptrace_fork(child, flags) +#define arch_ptrace_report_exit (code) x86_ptrace_report_exit(code)#endif /* __KERNEL__ */
arch/x86/kernel/ptrace.c +133 -19
187 lines not on this page · raw message
---------------------------------------------------------------------
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen Germany
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
Registergericht: Muenchen HRB 47456 Ust.-IdNr.
VAT Registration No.: DE129385895
Citibank Frankfurt (BLZ 502 109 00) 600119052
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Oleg Nesterov
maintainerarch/x86/kernel/ptrace.c static void ptrace_bts_fork(struct task_ · line 915
Quoted 9 lines · Markus Metzger
arch/x86/kernel/ptrace.c +8
-0+static void ptrace_bts_exit (void ) +{ +if (unlikely (!list_empty (¤t->ptraced))) +ptrace_bts_exit_tracer (); + +if (unlikely (current->bts)) +ptrace_bts_exit_tracee (); +}
Could you explain why do we need ptrace_bts_exit_tracee() ?
If current is traced, the tracer should do ptrace_bts_release() eventually, no?
And if we really need to do ptrace_bts_exit_tracee(), then "if (unlikely(current->bts))" check is racy. The tracer can do PTRACE_BTS_CONFIG right after the check.
Oleg.
Quoted 16 lines
-----Original Message-----
From: Oleg Nesterov [mailto:[email protected]]
Sent: Friday, March 27, 2009 3:35 PMarch/x86/kernel/ptrace.c static void ptrace_bts_fork(struct task_ · line 915
Quoted 8 lines
arch/x86/kernel/ptrace.c +8
-0+static void ptrace_bts_exit (void ) +{ +if (unlikely (!list_empty (¤t->ptraced))) +ptrace_bts_exit_tracer (); + +if (unlikely (current->bts)) +ptrace_bts_exit_tracee (); +}Could you explain why do we need
ptrace_bts_exit_tracee()?
Quoted 2 lines
If current is traced, the tracer should do
ptrace_bts_release()eventually, no?
If current is traced and exits, it may be reaped by another thread that is not the tracer (that's actually your example you made in an earlier thread to describe the race between a normal detach and an exiting tracee).
The ptrace_unlink() call to detach the tracer is executed with irq's disabled. I need irq's enabled (see the other discussion, to wait for the traced task).
Therefore, I have the tracee disable bts tracing itself when it exits.
Quoted 3 lines
And if we really need to do
ptrace_bts_exit_tracee(), then "if (unlikely(current->bts))" check is racy. The tracer can doPTRACE_BTS_CONFIGright after the check.
The ptrace system call to do this would require the tracee to be stopped.
regards,
markus.
---------------------------------------------------------------------
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen Germany
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
Registergericht: Muenchen HRB 47456 Ust.-IdNr.
VAT Registration No.: DE129385895
Citibank Frankfurt (BLZ 502 109 00) 600119052
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Oleg Nesterov
maintainerQuoted 19 lines · Markus T
Quoted 16 lines
-----Original Message-----
From: Oleg Nesterov [mailto:[email protected]]
Sent: Friday, March 27, 2009 3:35 PMarch/x86/kernel/ptrace.c static void ptrace_bts_fork(struct task_ · line 915
Quoted 8 lines
arch/x86/kernel/ptrace.c +8
-0+static void ptrace_bts_exit (void ) +{ +if (unlikely (!list_empty (¤t->ptraced))) +ptrace_bts_exit_tracer (); + +if (unlikely (current->bts)) +ptrace_bts_exit_tracee (); +}Could you explain why do we need
ptrace_bts_exit_tracee()?Quoted 2 lines
If current is traced, the tracer should do
ptrace_bts_release()eventually, no?
Quoted 6 lines
If current is traced and exits, it may be reaped by another thread that is not the tracer (that's actually your example you made in an earlier thread to describe the race between a normal detach and an exiting tracee).
The
ptrace_unlink()call to detach the tracer is executed with irq's disabled. I need irq's enabled (see the other discussion, to wait for the traced task).
OK,
Quoted 8 lines
Therefore, I have the tracee disable bts tracing itself when it exits.
Quoted 3 lines
And if we really need to do
ptrace_bts_exit_tracee(), then "if (unlikely(current->bts))" check is racy. The tracer can doPTRACE_BTS_CONFIGright after the check.The ptrace system call to do this would require the tracee to be stopped.
Yes, but this doesn't matter.
The tracer starts ptrace(PTRACE_BTS_CONFIG) and stops the tracee. But when the tracer calls ptrace_bts_config() the tracee can be already killed, and it can exit and bypass ptrace_bts_exit_tracee().
Oleg.
Quoted 36 lines · Oleg Nesterov
Quoted 19 lines · Markus T
Quoted 16 lines
-----Original Message-----
From: Oleg Nesterov [mailto:[email protected]]
Sent: Friday, March 27, 2009 3:35 PMarch/x86/kernel/ptrace.c static void ptrace_bts_fork(struct task_ · line 915
Quoted 8 lines
arch/x86/kernel/ptrace.c +8
-0+static void ptrace_bts_exit (void ) +{ +if (unlikely (!list_empty (¤t->ptraced))) +ptrace_bts_exit_tracer (); + +if (unlikely (current->bts)) +ptrace_bts_exit_tracee (); +}Could you explain why do we need
ptrace_bts_exit_tracee()?Quoted 2 lines
If current is traced, the tracer should do
ptrace_bts_release()eventually, no?Quoted 6 lines
If current is traced and exits, it may be reaped by another thread that is not the tracer (that's actually your example you made in an earlier thread to describe the race between a normal detach and an exiting tracee).
The
ptrace_unlink()call to detach the tracer is executed with irq's disabled. I need irq's enabled (see the other discussion, to wait for the traced task).OK,
Quoted 7 lines
Therefore, I have the tracee disable bts tracing itself when it exits.
Quoted 3 lines
And if we really need to do
ptrace_bts_exit_tracee(), then "if (unlikely(current->bts))" check is racy. The tracer can doPTRACE_BTS_CONFIGright after the check.
Quoted 7 lines
Quoted 1 line
The ptrace system call to do this would require the tracee to be stopped.
Yes, but this doesn't matter.
The tracer starts
ptrace(PTRACE_BTS_CONFIG)and stops the tracee. But when the tracer callsptrace_bts_config()the tracee can be already killed, and it can exit and bypassptrace_bts_exit_tracee().
Thanks for your review! This is really helpful.
This exit race is a nasty thing.
I think I need to either prevent a task from running while I change the tracing configuration, or have the tracee execute all the setup/cleanup code.
The former looks more attractive to me. I would expect that there is some infrastructure for that. I asked for pointers in a previous email, already.
thanks and regards,
markus.
Quoted 36 lines · Oleg Nesterov
Quoted 19 lines · Markus T
Quoted 16 lines
-----Original Message-----
From: Oleg Nesterov [mailto:[email protected]]
Sent: Friday, March 27, 2009 3:35 PMarch/x86/kernel/ptrace.c static void ptrace_bts_fork(struct task_ · line 915
Quoted 8 lines
arch/x86/kernel/ptrace.c +8
-0+static void ptrace_bts_exit (void ) +{ +if (unlikely (!list_empty (¤t->ptraced))) +ptrace_bts_exit_tracer (); + +if (unlikely (current->bts)) +ptrace_bts_exit_tracee (); +}Could you explain why do we need
ptrace_bts_exit_tracee()?Quoted 2 lines
If current is traced, the tracer should do
ptrace_bts_release()eventually, no?Quoted 6 lines
If current is traced and exits, it may be reaped by another thread that is not the tracer (that's actually your example you made in an earlier thread to describe the race between a normal detach and an exiting tracee).
The
ptrace_unlink()call to detach the tracer is executed with irq's disabled. I need irq's enabled (see the other discussion, to wait for the traced task).OK,
Quoted 7 lines
Therefore, I have the tracee disable bts tracing itself when it exits.
Quoted 3 lines
And if we really need to do
ptrace_bts_exit_tracee(), then "if (unlikely(current->bts))" check is racy. The tracer can doPTRACE_BTS_CONFIGright after the check.
Quoted 7 lines
Quoted 1 line
The ptrace system call to do this would require the tracee to be stopped.
Yes, but this doesn't matter.
The tracer starts
ptrace(PTRACE_BTS_CONFIG)and stops the tracee. But when the tracer callsptrace_bts_config()the tracee can be already killed, and it can exit and bypassptrace_bts_exit_tracee().
Could something like that work?
ds_release_bts(struct bts_tracer *tracer)
{
struct task_struct *task =
tracer->ds.context->task;
do {
set_task_state(task, TASK_UNINTERRUPTIBLE);
while (!wait_task_inactive(task, TASK_UNINTERRUPTIBLE));
ds_suspend_bts(tracer);
ds_free_bts(tracer);
wake_up_process(task);
}I guess it would not work in general, since task could already sleep on some event and be woken up after the do loop.
I was thinking it might work for the exit race, since we don't sleep during exit, but it might have started sleeping before the SIGKILL.
Isn't this a general problem for ptrace?
Ptrace uses a similar pattern in ptrace_check_attach().
It stops the traced task, but that task might wake up immediately after that check. It might be scheduled in any time during a ptrace operation.
In that case, must I always assume I'm operating on a running task?
regards,
markus.
Oleg Nesterov
maintainerQuoted 8 lines · Markus Metzger
ds_release_bts(struct bts_tracer *tracer){ struct task_struct *task = tracer->ds.context->task; do { set_task_state(task, TASK_UNINTERRUPTIBLE);
Oh, this is not right,
Quoted 1 line
while (!wait_task_inactive(task, TASK_UNINTERRUPTIBLE));
and can't help to wait_task_inactive(). Again, unless this task does a blocking call wait_task_inactive() can't force this task to be deactivated. Even we set TASK_UNINTERRUPTIBLE.
Quoted 4 lines
ds_suspend_bts(tracer); ds_free_bts(tracer); wake_up_process(task);
makes it TASK_RUNNING. This can't restores the proper ->state if it was not running (say, TASK_STOPPED).
Quoted 4 lines
}
I guess it would not work in general, since task could already sleep on some event and be woken up after the do loop.
This too,
Quoted 2 lines
I was thinking it might work for the exit race, since we don't sleep during exit,
We do. In the context of bts problems, we don't care if the task sleeps after _free_tracee(). But this task can have tracees too, it can in turn sleep in wait_task_inactive().
Quoted 5 lines
Isn't this a general problem for ptrace?
Ptrace uses a similar pattern in
ptrace_check_attach().
It stops the traced task, but that task might wake up immediately after that check. It might be scheduled in any time during a ptrace operation.
Yes. ptrace() can assume the tracee is TASK_TRACED, or it is dying/dead.
If you need to make sure it is still traced, you can re-check ->state under ->siglock. Until you drop this lock, the tracee (if still traced) can't escape from ptrace_stop/do_signal_stop, even if scheduled.
Oleg.
Quoted 11 lines
-----Original Message-----
From: Oleg Nesterov [mailto:[email protected]]
Sent: Monday, March 30, 2009 3:22 AM
To: Markus MetzgerQuoted 5 lines
ds_release_bts(struct bts_tracer *tracer){ struct task_struct *task = tracer->ds.context->task;
Quoted 4 lines
Quoted 2 lines
do { set_task_state(task, TASK_UNINTERRUPTIBLE);Oh, this is not right,
Agreed.
I wonder if it is ever safe to change another task's state. There's a lot of code that sets the task state seemingly unprotected - but always for current.
Quoted 11 lines
Quoted 5 lines
Isn't this a general problem for ptrace?
Ptrace uses a similar pattern in
ptrace_check_attach().
It stops the traced task, but that task might wake up immediately after that check. It might be scheduled in any time during a ptrace operation.Yes.
ptrace()can assume the tracee isTASK_TRACED, or it is dying/dead.If you need to make sure it is still traced, you can re-check ->state under ->siglock. Until you drop this lock, the tracee (if still traced) can't escape from ptrace_stop/do_signal_stop, even if scheduled.
I can't hold the siglock for the entire ptrace operation, can I?
Instead, I put all bts stuff into a context struct and I use use-counts to make sure the tracer is not released while it is used by a ptrace operation.
regards,
markus.
---------------------------------------------------------------------
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen Germany
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
Registergericht: Muenchen HRB 47456 Ust.-IdNr.
VAT Registration No.: DE129385895
Citibank Frankfurt (BLZ 502 109 00) 600119052
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
ds_release_bts(struct bts_tracer *tracer){ struct task_struct *task = tracer->ds.context->task; do { set_task_state(task, TASK_UNINTERRUPTIBLE);
Quoted 7 lines
Quoted 1 line
Oh, this is not right,
Agreed.
I wonder if it is ever safe to change another task's state. There's a lot of code that sets the task state seemingly unprotected - but always for current.
It should be safe when you can guarantee that the other task is stopped (as in ptrace) and you protect against yourself (only a single external writer)
-Andi