x86 NMI-safe INT3 and Page Fault (v5)

include/linux/hardirq.h extern void irq_enter(void); · line 176

Quoted 7 lines · On Thu, 17 Apr 2008 16:14:10 -0400 Mathieu Desnoyers…

include/linux/hardirq.h +7 -0

+#define nmi_enter()					\
+	do {						\
+		lockdep_off();				\
+		BUG_ON(hardnmi_count());		\
+		add_preempt_count(HARDNMI_OFFSET);	\
+		__irq_enter();				\
+	} while (0)

<did it _have_ to be a macro?>

Doing BUG() inside an NMI should be OK most of the time. But the BUG-handling code does want to know if we're in interrupt context - at least for the "fatal exception in interrupt" stuff, and probably other things.

But afacit the failure to include HARDNMI_MASK in

#define irq_count() (preempt_count() & (HARDIRQ_MASK | SOFTIRQ_MASK))

will prevent that.

So.

Should we or should we not make in_interrupt() return true in NMI? "should", I expect.

If not, we'd need to do something else to communicate the current processing state down to the BUG-handling code.

Quoted 6 lines

Should we or should we not make in_interrupt() return true in NMI? "should", I expect.

If not, we'd need to do something else to communicate the current processing state down to the BUG-handling code.

You bring an interesting question. In practice, since this BUG_ON could only happen if we have an NMI nested over another NMI or an nmi which fails to decrement its HARDNMI_MASK. Given that the HARDIRQ_MASK is incremented right after the HARDNMI_MASK increment (the reverse is also true), really bad things (TM) must have happened for the BUG_ON to be triggered outside of the __irq_enter()/__irq_exit() scope of the NMI below the buggy one.

But since this code is there to extract as much information as possible when things go wrong, I would say it's safer to, at least, add HARDNMI_MASK to irq_count().

Instead, though, I think we could add :

if (in_nmi())
panic("Fatal exception in non-maskable interrupt");

to die().

That would be clearer. I just added it to x86_32, but can't find where x86_64 reports the "fatal exception in interrupt" and friends message. Any idea ?

By dealing with this case specifically, I think we don't really have to add HARDNMI_MASK to irq_count(), considering it's normally an HARDIRQ too.

It allows placing immediate values (and therefore optimized trace_marks) in NMI code

Quoted 3 lines · Mathieu Desnoyers

It allows placing immediate values (and therefore optimized trace_marks) in NMI code

Only if all your trace_mark infrastructure is lock less.

-Andi

Quoted 7 lines · * Andi Kleen ([email protected]) wrote:

Quoted 2 lines · Mathieu Desnoyers

It allows placing immediate values (and therefore optimized trace_marks) in NMI code

Only if all your trace_mark infrastructure is lock less.

-Andi

It uses RCU-style updates and has been designed to be lockless from the ground up.

Quoted 9 lines · Mathieu Desnoyers

Quoted 7 lines · * Andi Kleen ([email protected]) wrote:

Quoted 2 lines · Mathieu Desnoyers

It allows placing immediate values (and therefore optimized trace_marks) in NMI code

Only if all your trace_mark infrastructure is lock less.

-Andi

Quoted 2 lines

It uses RCU-style updates and has been designed to be lockless from the ground up.

Wrong. If it causes vmalloc faults it is not lockless.

-Andi

It uses RCU-style updates and has been designed to be lockless from the ground up.

Quoted 3 lines · Mathieu Desnoyers

It uses RCU-style updates and has been designed to be lockless from the ground up.

RCU is not necessarily NMI safe. In most cases RCU needs writer locks which you cannot do with NMIs.

-Andi

(hopefully finally CCing LKML) :)

Implements an alternative iret with popf and return so trap and exception handlers can return to the NMI handler without issuing iret. iret would cause NMIs to be reenabled prematurely. x86_32 uses popf and far return. x86_64 has to copy the return instruction pointer to the top of the previous stack, issue a popf, loads the previous esp and issue a near return (ret).

Quoted 7 lines · Mathieu Desnoyers

(hopefully finally CCing LKML) :)

Implements an alternative iret with popf and return so trap and exception handlers can return to the NMI handler without issuing iret. iret would cause NMIs to be reenabled prematurely. x86_32 uses popf and far return. x86_64 has to copy the return instruction pointer to the top of the previous stack, issue a popf, loads the previous esp and issue a near return (ret).

sounds expensive. Does it slow down normal loads?

Quoted 11 lines · Pavel Machek

Quoted 10 lines · Mathieu Desnoyers

(hopefully finally CCing LKML) :)

Implements an alternative iret with popf and return so trap and exception handlers can return to the NMI handler without issuing iret. iret would cause
NMIs to be reenabled prematurely. x86_32 uses popf and far return. x86_64 has to
copy the return instruction pointer to the top of the previous stack, issue a
popf, loads the previous esp and issue a near return (ret).

Quoted 1 line

sounds expensive. Does it slow down normal loads?

It should *only* be used to return from NMI, #MC or INT3 (breakpoint), which should never happen in normal operation, and even then only when interrupting another NMI or #MC handler.

-hpa

Quoted 11 lines · Pavel Machek

Quoted 10 lines · Mathieu Desnoyers

(hopefully finally CCing LKML) :)

Implements an alternative iret with popf and return so trap and exception handlers can return to the NMI handler without issuing iret. iret would cause
NMIs to be reenabled prematurely. x86_32 uses popf and far return. x86_64 has to
copy the return instruction pointer to the top of the previous stack, issue a
popf, loads the previous esp and issue a near return (ret).

Quoted 1 line

sounds expensive. Does it slow down normal loads?

It should *only* be used to return from NMI, #MC or INT3 (breakpoint), which should never happen in normal operation, and even then only when interrupting another NMI or #MC handler.

-hpa

Quoted 18 lines · * H. Peter Anvin ([email protected]) wrote:

Quoted 11 lines · Pavel Machek

Quoted 10 lines · Mathieu Desnoyers

(hopefully finally CCing LKML) :)

Implements an alternative iret with popf and return so trap and exception handlers can return to the NMI handler without issuing iret. iret would cause
NMIs to be reenabled prematurely. x86_32 uses popf and far return. x86_64 has to
copy the return instruction pointer to the top of the previous stack, issue a
popf, loads the previous esp and issue a near return (ret).

Quoted 1 line

sounds expensive. Does it slow down normal loads?

It should *only* be used to return from NMI, #MC or INT3 (breakpoint), which should never happen in normal operation, and even then only when interrupting another NMI or #MC handler.

Quoted 2 lines

-hpa

Just to be clear : the added cost on normal interrupt return is to add a supplementary test of the thread flags already loaded in registers and

a conditional branch. This is used to detect if we are nested over an NMI handler. I doubt anyone ever notice an impact caused by this added test/branch.

Mathieu

Signature · 3 lines

Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68

Just to be clear : the added cost on normal interrupt return is to add a supplementary test of the thread flags already loaded in registers and a conditional branch. This is used to detect if we are nested over an NMI handler. I doubt anyone ever notice an impact caused by this added test/branch.

Quoted 5 lines · Mathieu Desnoyers

Just to be clear : the added cost on normal interrupt return is to add a supplementary test of the thread flags already loaded in registers and a conditional branch. This is used to detect if we are nested over an NMI handler. I doubt anyone ever notice an impact caused by this added test/branch.

Why the **** would you do this except in the handful of places where you actually *could* be nested over an NMI handler (basically #MC, #DB and INT3)?

-hpa

Quoted 5 lines · Mathieu Desnoyers

Just to be clear : the added cost on normal interrupt return is to add a supplementary test of the thread flags already loaded in registers and a conditional branch. This is used to detect if we are nested over an NMI handler. I doubt anyone ever notice an impact caused by this added test/branch.

Why the **** would you do this except in the handful of places where you actually *could* be nested over an NMI handler (basically #MC, #DB and INT3)?

-hpa

Quoted 7 lines · * H. Peter Anvin ([email protected]) wrote:

Quoted 5 lines · Mathieu Desnoyers

Just to be clear : the added cost on normal interrupt return is to add a supplementary test of the thread flags already loaded in registers and a conditional branch. This is used to detect if we are nested over an NMI handler. I doubt anyone ever notice an impact caused by this added test/branch.

Quoted 3 lines

Why the **** would you do this except in the handful of places where you actually *could* be nested over an NMI handler (basically #MC, #DB and INT3)?

There is also the page fault case. I think putting this test in ret_from_exception would be both safe (it is executed for any exception return) and fast (exceptions are rare).

Quoted 4 lines

There is also the page fault case. I think putting this test in ret_from_exception would be both safe (it is executed for any exception return) and fast (exceptions are rare).

Eh? I thought that page fault is one of the hottest paths in kernel (along with syscall and packet receive/send)...
Pavel

Quoted 6 lines · Pavel Machek

Quoted 3 lines

There is also the page fault case. I think putting this test in ret_from_exception would be both safe (it is executed for any exception return) and fast (exceptions are rare).

Eh? I thought that page fault is one of the hottest paths in kernel (along with syscall and packet receive/send)...
Pavel

Yeah, and the concept of handling page faults inside an NMI handler is pure fantasy.

-hpa

Quoted 4 lines · * H. Peter Anvin ([email protected]) wrote:

Quoted 3 lines · Pavel Machek

Quoted 3 lines

There is also the page fault case. I think putting this test in ret_from_exception would be both safe (it is executed for any exception return) and fast (exceptions are rare).

Quoted 3 lines

Quoted 3 lines

Eh? I thought that page fault is one of the hottest paths in kernel (along with syscall and packet receive/send)...
Pavel

On x86_64, we can pinpoint only the page faults returning to the kernel, which are rare and only caused by vmalloc accesses. Ideally we could do the same on x86_32.

Quoted 8 lines · Mathieu Desnoyers

Quoted 7 lines · * H. Peter Anvin ([email protected]) wrote:

Quoted 6 lines · Pavel Machek

Quoted 3 lines

There is also the page fault case. I think putting this test in ret_from_exception would be both safe (it is executed for any exception return) and fast (exceptions are rare).

Eh? I thought that page fault is one of the hottest paths in kernel (along with syscall and packet receive/send)...
Pavel

Quoted 3 lines

On x86_64, we can pinpoint only the page faults returning to the kernel, which are rare and only caused by vmalloc accesses. Ideally we could do the same on x86_32.

Pinpoint, how? Ultimately you need a runtime test, and you better be showing that people are going to die unless before you add a cycle to the page fault path. I'm only slightly exaggerating that.

-hpa