Ptrace documentation, draft #1

Hi Tejun,

Since I am not helping you much in kernel ptrace hacking, I can at least help you with the docs.

Ptrace discussions repeatedly display a higher than average amount of misunderstanding and confusion. New ptrace users and even people who already worked with it are repeatedly confused by details which are not documented anywhere and knowledge about which exists mostly in the brains of strace/gdb/other_such_tools developers.

This document is meant as a brain dump of this knowledge. It assumes that the reader has basic understanding what ptrace is.

Tejun, Oleg, Jan, please comment. If some paragraphs are unclear, post your version of the whole paragraph(s).

I am especially interested in adding info on ptrace quirks which I forgot or don't know yet - Jan, you must know quite a bit of those from your gdb work.

Tejun, Oleg - "2. Linux kernel implementation" section is for you to fill with a brain dump of kernel side of the story.

======================================================================
======================================================================
======================================================================

Ptrace API (ab)uses standard Unix parent/child signaling over waitpid. An unfortunate effect of it is that resulting API is complex and has subtle quirks. This document aims to describe these quirks.

It is split into two parts. First part focuses exclusively on userspace-visible API and behavior. Second section describes kernel internals of ptrace.

1. Userspace API.

(Note to editors: in this section, do not use kernel concepts and terms which are not observable through userspace API and user-visible behavior. Use section 2 for that.)

Debugged processes (tracees) first need to be attached to the debugging process (tracer). Attachment and subsequent commands are per-thread: in multi-threaded process, every thread can be individually attached to a (potentially different) tracer, or left not attached and thus not debugged. Therefore, "tracee" always means "(one) thread", never "a (possibly multi-threaded) process". Ptrace commands are always sent to a specific tracee using ptrace(PTRACE_foo, pid, ...), where pid is a TID of the corresponding Linux thread.

After attachment, each tracee can be in two states: running or stopped.

There are many kinds of states when tracee is stopped, and in ptrace discussions they are often conflated. Therefore, it is important to use precise terms.

In this document, any stopped state in which tracee is ready to accept ptrace commands from the tracer is called ptrace-stop. Ptrace-stops can be further subdivided into signal-delivery-stop, group-stop, syscall-stop and so on. They are described in detail later.

1.x Death under ptrace.

When a (possibly multi-threaded) process receives a killing signal (a signal set to SIG_DFL and whose default action is to kill the process), all threads exit. Tracees report their death to the tracer(s). This is not a ptrace-stop (because tracer can't query tracee status such as register contents, cannot restart tracee etc) but the notification about this event is delivered through waitpid API similarly to ptrace-stop.

Note that killing signal will first cause signal-delivery-stop (on one tracee only), and only after it is injected by tracer (or after it was dispatched to a thread which isn't traced), death from signal will happen on ALL tracees within multi-threaded process.

SIGKILL operates similarly, with exceptions. No signal-delivery-stop is generated for SIGKILL and therefore tracer can't suppress it. SIGKILL kills even within syscalls (syscall-exit-stop is not generated prior to death by SIGKILL). The net effect is that SIGKILL always kills the process (all its threads), even if some threads of the process are ptraced.

Tracer can kill a tracee with ptrace(PTRACE_KILL, pid, 0, 0).

??? Does it kill only the tracee or the whole process? What is exit
status? Will tracer see the death? Other tracers?

When tracee executes exit syscall, it reports its death to its tracer. Other threads are not affected.

When any thread executes exit_group syscall, every tracee reports its death to its tracer.

??? Is it true that *every* thread reports death?
IIRC it isn't... one does (leader? what if there is no leader?) and the rest simply disappear.

Tracer cannot assume that ptrace-stopped tracee exists. There are many scenarios when tracee may die while stopped (such as SIGKILL). Therefore, tracer must always be prepared to handle ESRCH error on any ptrace operation. Unfortunately, the same error is returned if tracee exists but is not ptrace-stopped (for commands which require stopped tracee). Tracer needs to keep track of stopped/running state, and interpret ESRCH as "tracee died unexpectedly" only if it knows that tracee has been observed to enter ptrace-stop.

??? is there a recommended usage of waitpid(WNOHANG) to check whether
tracee is dead or alive?

1.x Stopped states.

When running tracee enters ptrace-stop, it notifies its tracer using waitpid API. Tracer should use waitpid family of syscalls to wait for tracee to stop. Most of this document assumes that tracer waits with:

	pid = waitpid(pid_or_minus_1, &status, __WALL);

Ptrace-stopped tracees are reported as returns with pid > 0 and WIFSTOPPED(status) == true.

??? any pitfalls with WNOHANG (I remember that there are bugs in this
area)? effects of WSTOPPED, WEXITED, WCONTINUED bits? Are they ok? waitid usage? WNOWAIT?

1.x.x Signal-delivery-stop

When (possibly multi-threaded) process receives any signal except SIGKILL, kernel selects a thread which handles the signal (if signal is generated with tgkill, thread selection is done by user). If selected thread is traced, it enters signal-delivery-stop. By this point, signal is not yet delivered to the process, and can be suppressed by tracer. If tracer doesn't suppress the signal, it passes signal to tracee in the next ptrace request. This is called "signal injection" and will be described later. Note that if signal is blocked, signal-delivery-stop doesn't happen until signal is unblocked, with the usual exception that SIGSTOP can't be blocked.

Signal-delivery-stop is observed by tracer as waitpid returning with WIFSTOPPED(status) == true, WSTOPSIG(status) == signal. If WSTOPSIG(status) == SIGTRAP, this may be a different kind of ptrace-stop - see syscall-stop section below for details. If WSTOPSIG(status) == stopping signal, this may be a group-stop - see below.

Kernel delivers an extra SIGTRAP to tracee after execve syscall returns. This is an ordinary signal (similar to one generated by kill -TRAP), not a special kind of ptrace-stop. If PTRACE_O_TRACEEXEC option is in effect, a PTRACE_EVENT_EXEC-stop is generated instead.

??? can this SIGTRAP be distinguished from "real" user-generated SIGTRAP
by looking at its siginfo?

Usually, tracer (for example, strace) would not want to show this extra post-execve SIGTRAP signal to the user, and would suppress its delivery to the tracee (if SIGTRAP is set to SIG_DFL, it is a killing signal).

1.x.x Signal injection and suppression.

After signal-delivery-stop is observed by tracer, PTRACE_GETSIGINFO can be used to retrieve corresponding siginfo_t structure. PTRACE_SETSIGINFO may be used to modify it.

Tracer should restart tracee with
ptrace(PTRACE_rest, pid, 0, sig)
call, where PTRACE_rest is one of the restarting ptrace ops. If sig is 0, then signal is not delivered and has no effect. Otherwise, signal sig is delivered. This operation is called "signal injection", to distinguish it from signal delivery which causes signal-delivery-stop.

Note that sig value may be different from WSTOPSIG(status) value - tracer can cause a different signal to be injected. If before restarting PTRACE_SETSIGINFO command was used to alter injected signal's siginfo_t, si_signo field and sig parameter in restarting command must match.

??? Are syscalls interrupted by signals which are suppressed by tracer?
If yes, document it here

Note that restarting ptrace commands issued in ptrace-stops other than signal-delivery-stop do NOT inject a signal, even if sig is nonzero. No error is reported either. This is a cause of confusion among ptrace users. One typical scenario is that tracer observes group-stop, mistakes it for signal-delivery-stop, restarts tracee with ptrace(PTRACE_rest, pid, 0, stopsig) with the intention of injecting stopsig, but stopsig gets ignored and tracee continues to run.

??? TODO: there *are* some other ptrace-stops which can take sig.
Document them.

SIGCONT signal has a side effect of waking up (all threads of) group-stopped process. This side effect happens before signal-delivery-stop. Tracer can't suppress this side-effect (it can only suppress signal injection, which only causes SIGCONT handler to not be executed in the tracee, if such handler is installed). In fact, waking up from group-stop may be followed by signal-delivery-stop for signal(s) *other than* SIGCONT, if they were pending when SIGCONT was delivered. IOW: SIGCONT may be not the first signal observed by the tracee after it was sent.

Stopping signals cause (all threads of) process to enter group-stop. This side effect happens after signal injection, and therefore can be suppressed by tracer.

1.x.x Group-stop

When a (possibly multi-threaded) process receives a stopping signal, all threads stop. If some threads are traced, they enter a group-stop. Note that stopping signal will first cause signal-delivery-stop (on one tracee only), and only after it is injected by tracer (or after it was dispatched to a thread which isn't thraced), group-stop will be initiated on ALL tracees within multi-threaded process. As usual, every tracee reports its group-stop to corresponding tracer.

Group-stop is observed by tracer as waitpid returning with the following result: WIFSTOPPED(status) is true, WSTOPSIG(status) is the delivered signal. The same result is returned by some other classes of ptrace-stops, therefore the recommended practice is to perform
ptrace(PTRACE_GETSIGINFO, pid, 0, &siginfo)
call. The call can be avoided if signal number is not SIGSTOP, SIGTSTP, SIGTTIN or SIGTTOU - only these four signals are stopping signals. If tracer sees something else, it can't be group-stop. Otherwise, tracer needs to call PTRACE_GETSIGINFO. If PTRACE_GETSIGINFO fails, then it is a group-stop. If it succeeds, it's a signal-delivery-stop.

As of kernel 2.6.38, after tracer sees tracee ptrace-stop and until it restarts or kills it, tracee will not run, and will not send notifications (except SIGKILL death) to tracer, even if tracer enters into another waitpid call.

Currently, it causes problem with transparent handling of stopping
signals: if tracer restarts tracee after group-stop, SIGSTOP is effectively ignored: tracee doesn't remain stopped, it runs. If tracer doesn't restart tracee before entering into next waitpid, future SIGCONT will not be reported to the tracer. Which would make SIGCONT to have no effect.

??? ...at least no effect on this tracee - how will other threads
be affected?

1.x.x Syscall-stops

If tracee was restarted by PTRACE_SYSCALL, tracee enters syscall-enter-stop just prior to entering any syscall. If tracer restarts it with PTRACE_SYSCALL, tracee enters syscall-exit-stop when syscall is finished, or if it is interrupted by a signal. (That is, signal-delivery-stop never happens between syscall-enter-stop and syscall-exit-stop, it happens after syscall-exit-stop). Other possibilities are that tracee may exit (if it entered exit or exit_group syscall), be killed by SIGKILL, or die when other threads' actions terminate all threads of the process (such as execve syscall).

??? how such death-because-of-other-thread is reported?

Syscall-enter-stop and syscall-exit-stop are observed by tracer as waitpid returning with the following result: WIFSTOPPED(status) is true, WSTOPSIG(status) == SIGTRAP. If PTRACE_O_TRACESYSGOOD option was set by tracer, then WSTOPSIG(status) == (SIGTRAP | 0x80). Otherwise, it is impossible to distinguish them from signal-delivery-stop with SIGTRAP.

Syscall-enter-stop and syscall-exit-stop are indistinguishable by tracer. Tracer needs to keep track of the sequence of ptrace-stops in order to not misinterpret syscall-enter-stop as syscall-exit-stop or vice versa. The rule is that syscall-enter-stop is always followed by syscall-exit-stop or tracee's death - no other kinds of ptrace-stop can occur in between.

??? What will happen if trace uses *NOT* PTRACE_SYSCALL to restart
tracee after syscall-enter-stop?

??? what PTRACE_GETSIGINFO returns on syscall stops?

1.x.x SINGLESTEP, SYSEMU, SYSEMU_SINGLESTEP

??? document PTRACE_SINGLESTEP, PTRACE_SYSEMU, PTRACE_SYSEMU_SINGLESTEP

1.x.x PTRACE_EVENT-stops

If tracer sets TRACE_O_TRACEfoo options, tracee will enter ptrace-stops called PTRACE_EVENT-stops.

PTRACE_EVENT-stops are observed by tracer as waitpid returning with the following result: WIFSTOPPED(status) is true, WSTOPSIG(status) == SIGTRAP. Additional bit is set in a higher byte of status word: value ((status >> 8) & 0xffff) will be (SIGTRAP | PTRACE_EVENT_foo << 8).

PTRACE_EVENT_VFORK - stop after return from vfork/clone+CLONE_VFORK ??? continuing it will make it wait for child to exit/exec, right? PTRACE_EVENT_FORK - stop after return from fork/clone+SIGCHLD PTRACE_EVENT_CLONE - stop after return from clone PTRACE_EVENT_VFORK_DONE - stop after vfork child unblocks this tracee

	For all four: stop occurs in parent, not in new thread;
	PTRACE_GETEVENTMSG can be used to retrieve new thread's tid.

PTRACE_EVENT_EXEC - stop after return from exec PTRACE_EVENT_EXIT - stop before exit
PTRACE_GETEVENTMSG returns exit status.
Registers can be examined (unlike when "real" exit happens).
??? needs to be PTRACE_CONTed to finish exit, or not?

??? what PTRACE_GETSIGINFO returns on PTRACE_EVENT-stops?

1.x Informational and restarting ptrace commands.

Most ptrace commands (all except ATTACH, TRACEME, KILL, SETOPTIONS) require tracee to be in ptrace-stop, otherwise they fail with ESRCH.

When tracee is in ptrace-stop, tracer can read and write data to tracee using informational commands. They leave tracee in ptrace-stopped state.

longv = ptrace(PTRACE_PEEKTEXT/PTRACE_PEEKDATA/PTRACE_PEEKUSER, pid, addr, 0);
ptrace(PTRACE_POKETEXT/PTRACE_POKEDATA/PTRACE_POKEUSER, pid,

addr, long_val);
	ptrace(PTRACE_GETREGS/PTRACE_GETFPREGS, pid, 0, &struct);
	ptrace(PTRACE_SETREGS/PTRACE_SETFPREGS, pid, 0, &struct);
	ptrace(PTRACE_GETSIGINFO, pid, 0, &siginfo);
	ptrace(PTRACE_SETSIGINFO, pid, 0, &siginfo);
	ptrace(PTRACE_GETEVENTMSG, pid, 0, &long_var);

Note that some errors are not reported. For example, setting siginfo may have no effect in some ptrace-stops, yet the call will succeed (return 0 and not set errno).

Another group of ptrace commands makes ptrace-stopped tracee run. They all have the form:

	ptrace(PTRACE_cmd, pid, 0, sig);

where cmd is CONT, DETACH, SYSCALL, SINGLESTEP, SYSEMU, SYSEMU_SINGLESTEP. If tracee is in signal-delivery-stop, sig is the signal to be injected. Otherwise, sig is ignored.

1.x Attaching and detaching

A thread can be attached to using ptrace(PTRACE_ATTACH, pid, 0, 0) call. This also sends SIGSTOP to this thread. If tracer wants this SIGSTOP to have no effect, it needs to suppress it. Note that if other signals are concurrently sent to this thread during attach, tracer may see tracee enter signal-delivery-stop with other signal(s) first! The usual practice is to reinject these signals until SIGSTOP is seen, then suppress SIGSTOP injection. The design bug here is that attach and concurrent SIGSTOP race and SIGSTOP may be lost.

??? Is there a bug/misfeature that attaching interrupts some syscalls,
such as nanosleep? Document it. (I guess even suppressed SIGSTOP causes syscall to return prematurely).

ptrace(PTRACE_TRACEME, 0, 0, 0) request turns current thread into a tracee. It continues to run (doesn't enter ptrace-stop). A common practice is follow ptrace(PTRACE_TRACEME) with raise(SIGSTOP) and allow parent (which is our tracer now) to observe our signal-delivery-stop.

If PTRACE_O_TRACE[V]FORK or PTRACE_O_TRACECLONE options are in effect, then children created by (vfork or clone(CLONE_VFORK)), (fork or clone(SIGCHLD)) and (other kinds of clone) respectively are automatically attached to the same tracer which traced their parent. SIGSTOP is delivered to them, causing them to enter signal-delivery-stop as they exit syscall which created them.

Detaching of tracee is performed by ptrace(PTRACE_DETACH, pid, 0, sig). PTRACE_DETACH is a restarting operation, therefore it requires tracee to be in ptrace-stop. If tracee is in signal-delivery-stop, signal can be injected. Othervice, sig parameter is ignored.

If tracee is running when tracer wants to detach it, the usual solution is to send SIGSTOP (using tgkill, to mkae sure it goes to the correct thread), wait for tracee to stop in signal-delivery-stop for SIGSTOP and then detach it (suppressing SIGSTOP injection). Design bug is that this can race with concurrent SIGSTOPs. Another complication is that tracee may enter other ptrace-stops and needs to be restarted and waited for again, until SIGSTOP is seen. Yet another complication is to be sure that tracee is not already group-stopped, because no signal delivery happens while it is - not even SIGSTOP.

??? is above accurate?

If tracer dies, all tracees are automatically detached.

??? are they restarted if they were in some ptrace-stop?
Even those which were in group-stop?
Is signal injected if they were in signal-delivery-stop?

1.x Real parent

Ptrace API (ab)uses standard Unix parent/child signaling over waitpid. This used to cause real parent of the process to stop receiving several kinds of waitpid notifications when child process is traced by some other process.

Many of these bugs have been fixed, but as of 2.6.38 several still exist.

As of 2.6.38, the following is believed to work correctly:

- exit/death by signal is reported both to tracer and to real parent. If they are the same process, the report is sent only once.

- ??? add more docs

Following bugs still exist:

- group-stop notifications are sent to tracer, but not to real parent.

- ??? add more known bugs here

2. Linux kernel implementation

TODO

Hello,

Oh, I love you. :-)

I'm a bit backed up at the moment but will get to this later this week.

Thank you very much.

tejun

Denys, thanks for doing this.

Note: currently a killed PT_TRACE_EXIT tracee can stop and report PTRACE_EVENT_EXIT before it actually exits. I'd say this is wrong and should be fixed.

Another problem: the tracee can silently "disappear" during exec, if it was the group leader and exec is called by its sub-thread. Unfortunately, this is not easy to fix. The new leader inherits the same pid. In fact, the old thread can "disappear", exactly because it changes its pid.

IOW. If the old leader was traced - it disappears. If the new leader is traced, it continues to be traced but it changes its pid, so it is visible as the old leader to the tracer.

Oh, no. This is more or less equivalent to PTRACE_CONT(SIGKILL) except PTRACE_KILL doesn't return the error if the tracee is not stopped.

I'd say: do not use PTRACE_KILL, never. If the tracer wants to kill the tracee - kill or tkill should be used.

Yes, if you mean do_wait() as above.

I don't think so. It can be dying but not dead yet, so WNOHANG | WEXITED will fail.

Afaics no. Well, except .si_pid shows that the signal was sent by the tracing process to itself.

I'd say it is better to assume nobody sends SIGTRAP to the tracee. Even if the tracer could filter out the "real" signals, SIGTRAP doesn't queue.

Please reiterate, can't understand.

Yes. Except syscall entry/exit. But in this case SET_SIGINFO doesn't work to add more confusion ;)

Well, this is not exactly true. Initially the tracee sleeps in TASK_STOPPED and thus it can be woken by SIGCONT. But the first ptrace request changes turns this state into TASK_TRACED.

This was already changed by the pending patches.

This is true. But, just in case, please note that PTRACE_EVENT_EXEC or PTRACE_EVENT_{FORK,CLONE,etc} can be reported in between.

again, can't understand the question.

Almost... at least on x86 rax = -ENOSYS in syscall-enter-stop.

Nothing special. syscall-exit-stop won't be reported.

The same info as any other ptrace_notify(). Except si_code which you already described: SIGTRAP, or SIGTRAP | 0x80 if PT_TRACESYSGOOD.

Yes.

The contents is always the same,

	si_signo = SIGTRAP;
	si_pid	= tid_of_the_tracee;
	si_uid	= uid;

But si_code = (event << 8) | SIGTRAP and depends on reported event.

SETOPTIONS needs the stopped tracee as well.

There is another special case. If the tracee single-stepps into the signal handler, it reports SIGTRAP as if it recieved this SIGNAL. But ptrace(PTRACE, ..., sig) doesn't inject after that.

Yes, this is the "normal" signal and thus it can interrupt syscalls.

yes,

well, the current code is buggy, but it tries to leave it stopped.

Yes, The tracee resumes and handles the previously reported signal.

First to the tracer. Once it does do_wait(), we notify the real parent. And of course, the real parent is not notified about the exiting threads.

There is additional complication with the group-leader. If it is traced and exits, do_wait(WEXITED) doesn't work (until all threads exit) for the tracer. Should be changed, I think.

Oleg.

Hello,

I haven't read the whole thing yet but wanna comment on some.

If you do GETSIGINFO and look at si->si_code, user generated signals can't have non-zero value there (assuming SETSIGINFO doesn't allow uploading __SI_TRAP siginfo), so, if si->si_code contains SIGTRAP | PTRACE_EVENT_* << 8, I think it reliably identifies a specific trap. We probably want to add more PTRACE_EVENT_* so that each trap site can be identified uniquely. Note that the si_code is also returned as exit_code on wait(2) and combined with CLD_TRAPPED ptrace traps should be reliably identifiable.

Yeah, I hate these signal injections. Maybe we can remove these if SEIZED? Tracer might as well just do tkill(). I don't really see the point of the signal injection feature.

We can add PTRACE_EVENT_* values if SEIZED.

Ditto as PTRACE_SYSCALL. I think resuming from signal delivery trap should be the only place where @data means signo.

Please note that this one really isn't an injection. It's just given a chance to either suppress or alter a signal which is being delivered. It's not creating a new one.

Thank you.

tejun

Hmm. The can? sys_kill() sets si_code = 0, but tkill() or queueinfo() can pass any si_code < 0. Also, the kernel can generate the signal with si_code > 0.

But in this case (without PT_TRACE_EXEC) the tracee simply sends SIGTRAP to itself. It will be reported later like a normal signal.

Welcome to the club!

Heh... I am not sure.

Hmm. What do you mean? tracehook_report_syscall_exit() can skip ptrace_report_syscall() if step, but this is another story. And in this case the tracee sends the real signal to itself, unlike tracehook_signal_handler() which does ptrace_notify().

Agreed, it would be more clean/simple/understandable.

Yes, agreed, we do not create the new one.

Oleg.

Hello,

Yeah, sure, not non-zero then, but it's still distinguishible, no?

Ah, okay, I missed the context. I thought we were talking about GETSIGINFO after traps not the extra SIGTRAP after execve. Sorry about that. :-)

Yeah, I'm not sure either. I think the most reasonable solution would be leaving them alone but strongly discourage their use.

Please disregard. Still was fixated on GETSIGINFO.

Thanks.

tejun

Hmm. Now it is me who is missing context. And probably I misunderstood the original question:

Kernel delivers an extra SIGTRAP to tracee after execve syscall returns. This is an ordinary signal
...
??? can this SIGTRAP be distinguished from "real" user-generated SIGTRAP
by looking at its siginfo?

I meant, we can't distinguish this SIGTRAP. But if PTRACE_O_TRACEEXEC is set then the tracee reports PTRACE_EVENT_EXEC, this is probably distinguishible. IIUC, si_code > 0 if the tracee reports an event.

Oleg.

I think ptrace API never promised that it will be possible to "create new signal" by passing non-zero data - only that the already pending signal can be passed, altered or suppressed. If there *are* cases where "create new signal" currently works, it's a quirk.

I changed doc to reflect this. If we tighten it up so that we can state that all such attempts will be ignored, we can update docs.

vda

Yes, I assumed this is normal.
Or do you mean that *killed* tracee (that is, by signal) also stops there?

This should be so much fun for ptrace users :)
I am documenting this.

Regardless. We need to tell users what to expect after they do PTRACE_KILL. Should they expect that every tracee report WIFSIGNALED(status) and WTERMSIG(status) = SIGKILL? Can they get stale, buffered waitpid results before they get this last one?

And will PTRACE_EVENT_EXIT happen for *every* tracee (which has it configured)?

What about si_code? Is it set to SI_KERNEL for this signal? AFAIR userspace can't create signals with such si_code.

Yes, I understand that the race with real SIGTRAPs is not fixable. I mostly look for a way for tracer to say "aha, this is that pesky SIGTRAP from execve, ignore it". One way is to set PTRACE_O_TRACEEXEC. Is GETSIGINFO another?

Let's say tracee is in nanosleep. Then some signal arrives, but tracer decides to ignore it. In tracer:

waitpid: WIFSTOPPED, WSTOPSIG = some_sig <===
ptrace(PTRACE_CONT, pid, 0, 0) ===>

will this interrupt nanosleep in tracee?

I rewrote it like this:

Note that restarting ptrace commands issued in ptrace-stops other than signal-delivery-stop is not guaranteed to inject a signal, even if sig is nonzero.
No error is reported, nonzero sig may simply be ignored. Ptrace users should not try to "create new signal" this way: use tgkill(2) instead.

This is an extremely subtle point, and is not really a part of API "as designed":
how tracer can know that it's a group-stop *without* performing GETSIGINFO? And after it performed GETSIGINFO, it "destroyed" TASK_STOPPED... so knowing that it *was* TASK_STOPPED, which *could have been* woken up by SIGCONT does absolutely no good for the poor tracer, it's too late!

It looks insane.

I propose to not document it, as you guys plan to fix this thing for good.

Aha, so PTRACE_EVENT-stops happen "within" the syscall? Meaning, between syscall-enter-stop and syscall-exit-stop?

Aha! I sort-of expected SI_KERNEL there...

This is part of missing doc about PTRACE_SINGLESTEP.

implies PTRACE_SYSCALL behavior: "report syscall-stops".

Updated this part.

vda

Yes.

Once again, PTRACE_KILL == ptrace(PTRACE_CONT, SIGKILL), except it doesn't return the error if the tracee is not stopped.

OTOH, it does the unconditional wakeup, but I don't think we should document this bug.

Oh. This depends on /dev/random. Most probably the exiting tracee dequeues the (implicit) SIGKILL and report PTRACE_EVENT_EXIT. Oh, unless arch_ptrace_stop_needed() is true. But it can exit on its own or deque another fatal signal, then it won't stop because of fatal_signal_pending().

In short: this should be fixed. We already discussed this a bit (many times ;), first of all we should define the correct behaviour. If you ask me, personally I think PTRACE_EVENT_EXIT should be always reported unless the task was explicitly killed by SIGKILL. But this is not clear.

No, SI_USER.

Yes.

How? We simply send SIGTRAP as if it was sent by kill(), the tracee will dequeue this signal and report later.

note that the tracee is already interrupted here, sys_nanosleep() returns ERESTART_RESTARTBLOCK.

Yes and no. Once again, the tracee already returned from sys_nanosleep, but it will restart this syscall (actually, it will do sys_restart_syscall) and continue to sleep.

I think this was never designed ;)

Agreed.

Yes.

Hmm. Why do you think so?

Oleg.

Thanks, noted.

Oleg, this doesn't explain the resulting behavior in terms understandable to mere mortals. *What will happen* when user does ptrace(PTRACE_KILL)?

Yes, it's obvious that the tracee gets SIGKILLed, but will it report WIFSIGNALED or not? Userspace folks won't be 100.00% sure if we won't be exact about it.

They may think "hmm... maybe this PTRACE_KILL thing is so powerful it makes it unnecessary to waitpid for the nuked process?", which actualy isn't such a stupid hupothesis - if tracer itself PTRACE_KILL's tracee, it doesn't want to know about it anymore, so why should it waitpid for it?

Documented with "KNOWN BUG:" tag.

This is stupid. This signal is sent by kernel. Why is it flagged as "from user"? Maybe we should change it?

(BTW, where is it generated in the kernel source? I found PTRACE_EVENT_EXEC generation, but failed to find "old-school SIGTRAP" generation code...)

Documented as such.

I am totally unfamiliar with PTRACE_SINGLESTEP.

Thanks! Expect draft #3 soon.

vda

No, it is not necessarily killed. Once again, it is equivalent to PTRACE_CONT except it returns 0 (but does nothing) if the tracee is not stopped.

"does nothing" is not 100% true, it does wake_up_process() but this shouldn't be documented, this should be fixed.

Well, may be... I dunno actually. Hmm. On a second thought you are probably right. ptrace_report_syscall() does send_sig() too, but it sets SI_KERNEL. I am starting to think, at least they should use the same si_code.

tracehook_report_exec()->send_sig(SIGTRAP).

Oleg.

In fact ptrace_resume()->wake_up_process() is obviously wrong anyway, I think the patch below makes sense even for 2.6.40.

But it is much worse in PTRACE_KILL case. Just for example,

	int main(void)
	{
		int child, status;

		child = fork();
		if (!child) {
			int ret;

			assert(ptrace(PTRACE_TRACEME, 0,0,0) == 0);

			ret = pause();
			printf("pause: %d %m\n", ret);

			return 0x23;
		}

		sleep(1);
		assert(ptrace(PTRACE_KILL, child, 0,0) == 0);

		assert(child == wait(&status));
		printf("wait: %x\n", status);

		return 0;
	}

leaks -ERESTARTNOHAND. Yes, we should probably fix sys_pause() as well, it should check signal_pending(). But we shouldn't allow to wake up the tracee in unknown state/path.

Can't understand why this wasn't fixed before... I always knew this looks wrong, but I never sent the patch. Probably because I never understood the original reason for wake_up_process...

Oleg.

@@ -706,7 +706,7 @@ static int ptrace_resume(struct task_str
 	}
 
 	child->exit_code = data;
-	wake_up_process(child);
+	wake_up_state(child, TASK_TRACED);
 
 	return 0;
 }

Send me a signed-off version with a somewhat cleaned-up commit message, and we can make that happen.

Or a pull request ..

Linus

Ah, I sent the patch a minute ago.

Whatever is more convenient for you. If everyone agree with this change, I'll add it to ptrace branch and send [GIT PULL] tomorrow.

Oleg.

OK, nobody replied, I assume there are no objections...

Please pull from

git://git.kernel.org/pub/scm/linux/kernel/git/oleg/misc.git for-2.6.40

Oleg Nesterov (2):
ptrace: ptrace_resume() shouldn't wake up !TASK_TRACED thread
signal: sys_pause() should check signal_pending()

 kernel/ptrace.c |    2 +-
 kernel/signal.c |    6 ++++--
 2 files changed, 5 insertions(+), 3 deletions(-)

index 7a81fc0..2df1157 100644
@@ -562,7 +562,7 @@ static int ptrace_resume(struct task_struct *child, long request,
 	}
 
 	child->exit_code = data;
-	wake_up_process(child);
+	wake_up_state(child, __TASK_TRACED);
 
 	return 0;
 }

index ad5e818..86c32b8 100644
@@ -3023,8 +3023,10 @@ SYSCALL_DEFINE2(signal, int, sig, __sighandler_t, handler)
 
 SYSCALL_DEFINE0(pause)
 {
-	current->state = TASK_INTERRUPTIBLE;
-	schedule();
+	while (!signal_pending(current)) {
+		current->state = TASK_INTERRUPTIBLE;
+		schedule();
+	}
 	return -ERESTARTNOHAND;
 }
 

So, I am going to add these 2 simple fixes to ptrace branch if they pass the review. Imho 1/2 makes sense for 2.6.40.

Oleg.

It is not clear why ptrace_resume() does wake_up_process(). Unless the caller is PTRACE_KILL the tracee should be TASK_TRACED so we can use wake_up_state(__TASK_TRACED). If sys_ptrace() races with SIGKILL we do not need the extra and potentionally spurious wakeup.

If the caller is PTRACE_KILL, wake_up_process() is even more wrong. The tracee can sleep in any state in any place, and if we have a buggy code which doesn't handle a spurious wakeup correctly PTRACE_KILL can be used to exploit it. For example:

	int main(void)
	{
		int child, status;

		child = fork();
		if (!child) {
			int ret;

			assert(ptrace(PTRACE_TRACEME, 0,0,0) == 0);

			ret = pause();
			printf("pause: %d %m\n", ret);

			return 0x23;
		}

		sleep(1);
		assert(ptrace(PTRACE_KILL, child, 0,0) == 0);

		assert(child == wait(&status));
		printf("wait: %x\n", status);

		return 0;
	}

prints "pause: -1 Unknown error 514", -ERESTARTNOHAND leaks to the userland. In this case sys_pause() is buggy as well and should be fixed.

I do not know what was the original rationality behind PTRACE_KILL. The man page is simply wrong and afaics it was always wrong. Imho it should be deprecated, or may be it should do send_sig(SIGKILL) as Denys suggests, but in any case I do not think that the current behaviour was intentional.

Note: there is another problem, ptrace_resume() changes ->exit_code and this can race with SIGKILL too. Eventually we should change to not use ->exit_code.

---

 kernel/ptrace.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

@@ -561,7 +561,7 @@ static int ptrace_resume(struct task_str
 	}
 
 	child->exit_code = data;
-	wake_up_process(child);
+	wake_up_state(child, __TASK_TRACED);
 
 	return 0;
 }

ERESTART* is always wrong without TIF_SIGPENDING. Teach sys_pause() to handle the spurious wakeup correctly.

---

 kernel/signal.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

@@ -3023,8 +3023,10 @@ SYSCALL_DEFINE2(signal, int, sig, __sigh
 
 SYSCALL_DEFINE0(pause)
 {
-	current->state = TASK_INTERRUPTIBLE;
-	schedule();
+	while (!signal_pending(current)) {
+		current->state = TASK_INTERRUPTIBLE;
+		schedule();
+	}
 	return -ERESTARTNOHAND;
 }