Skip to contentSkip to conversation

x86: Barf when faults happen in NMI

RFC PATCH In review Frederic Weisbecker 3 participants 5 messages Sep 27, 2010 – Sep 28, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 x86: Barf when faults happen in NMI

In x86, faults exit by executing the iret instruction, which then reenables NMIs if we faulted in NMI context. Then if a fault happens in NMI, another NMI can nest after the fault exits.

But we don't yet support nested NMIs because we have only one NMI stack. To prevent that, trigger a bug when a fault happens in NMI context.

Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc Peter Zijlstra <[email protected]>

---

I first thought about putting it in the vmalloc fault path only. But then I saw more occasions for the kernel to fault (kmemcheck or so), and so I thought it should be better put in the all in one path. But I suspect you won't like that conditional in the big x86 fault path.

 arch/x86/mm/fault.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/mm/fault.c +2 -0
index 4c4508e..80c997e 100644
@@ -955,6 +955,8 @@ do_page_fault(struct pt_regs *regs, unsigned long error_code)
 	int write;
 	int fault;
 
+	BUG_ON(in_nmi());
+
 	tsk = current;
 	mm = tsk->mm;
 
Signature · 1 line
1.6.2.3
#1 lore raw
Quoted 31 lines · * Frederic Weisbecker ([email protected]) wrote:

In x86, faults exit by executing the iret instruction, which then reenables NMIs if we faulted in NMI context. Then if a fault happens in NMI, another NMI can nest after the fault exits.

But we don't yet support nested NMIs because we have only one NMI stack. To prevent that, trigger a bug when a fault happens in NMI context.

Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc Peter Zijlstra <[email protected]>

---

I first thought about putting it in the vmalloc fault path only. But then I saw more occasions for the kernel to fault (kmemcheck or so), and so I thought it should be better put in the all in one path. But I suspect you won't like that conditional in the big x86 fault path.

 arch/x86/mm/fault.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/mm/fault.c +0 -0
index 4c4508e..80c997e 100644

arch/x86/mm/fault.c do_page_fault(struct pt_regs *regs, unsigned long error_code) · line 958

Quoted 4 lines
arch/x86/mm/fault.c +1 -0
@@ -955,6 +955,8 @@ do_page_fault(struct pt_regs *regs, unsigned long error_code)
 	int write;
 	int fault;
 
+	BUG_ON(in_nmi());

Alternative idea: we could put the test at the beginning of the NMI handler, so if a NMI handler nests over a processor already "in_nmi", then we bug. I agree that this will trigger less easily than bugging in the fault handler (because we need to hit the actual nmi-coming-in-because-iret-reenabled-them-too-early scenario, but it's far less intrusive.

Thoughts ?

Mathieu

Quoted 7 lines
arch/x86/mm/fault.c +1 -0
+
 	tsk = current;
 	mm = tsk->mm;
 
Signature · 1 line
1.6.2.3
Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
#2 lore raw
Quoted 43 lines · Mathieu Desnoyers
Quoted 31 lines · * Frederic Weisbecker ([email protected]) wrote:

In x86, faults exit by executing the iret instruction, which then reenables NMIs if we faulted in NMI context. Then if a fault happens in NMI, another NMI can nest after the fault exits.

But we don't yet support nested NMIs because we have only one NMI stack. To prevent that, trigger a bug when a fault happens in NMI context.

Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc Peter Zijlstra <[email protected]>

---

I first thought about putting it in the vmalloc fault path only. But then I saw more occasions for the kernel to fault (kmemcheck or so), and so I thought it should be better put in the all in one path. But I suspect you won't like that conditional in the big x86 fault path.

 arch/x86/mm/fault.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/mm/fault.c +0 -0
index 4c4508e..80c997e 100644

arch/x86/mm/fault.c do_page_fault(struct pt_regs *regs, unsigned long error_code) · line 958

Quoted 4 lines
arch/x86/mm/fault.c +1 -0
@@ -955,6 +955,8 @@ do_page_fault(struct pt_regs *regs, unsigned long error_code)
 	int write;
 	int fault;
 
+	BUG_ON(in_nmi());

Alternative idea: we could put the test at the beginning of the NMI handler, so if a NMI handler nests over a processor already "in_nmi", then we bug. I agree that this will trigger less easily than bugging in the fault handler (because we need to hit the actual nmi-coming-in-because-iret-reenabled-them-too-early scenario, but it's far less intrusive.

Quoted 1 line

Thoughts ?

In fact we have that already in nmi_enter(). Now as you said that alone is probably too light to find the reason of a nested NMI or to prevent it.

#3 lore raw
In reply to Frederic Weisbecker
Quoted 30 lines · Frederic Weisbecker

In x86, faults exit by executing the iret instruction, which then reenables NMIs if we faulted in NMI context. Then if a fault happens in NMI, another NMI can nest after the fault exits.

But we don't yet support nested NMIs because we have only one NMI stack. To prevent that, trigger a bug when a fault happens in NMI context.

Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc Peter Zijlstra <[email protected]>

---

I first thought about putting it in the vmalloc fault path only. But then I saw more occasions for the kernel to fault (kmemcheck or so), and so I thought it should be better put in the all in one path. But I suspect you won't like that conditional in the big x86 fault path.

 arch/x86/mm/fault.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/mm/fault.c +0 -0
index 4c4508e..80c997e 100644

arch/x86/mm/fault.c do_page_fault(struct pt_regs *regs, unsigned long error_code) · line 959

Quoted 5 lines
arch/x86/mm/fault.c +2 -0
@@ -955,6 +955,8 @@ do_page_fault(struct pt_regs *regs, unsigned long error_code)
 	int write;
 	int fault;
 
+	BUG_ON(in_nmi());
+

Hm, this is a fastpath, so it would be nice to put this into the vmalloc and kmemcheck paths (even though it's less clean that way).

Also, a WARN_ON_ONCE() would probably help more in getting bug reports out of such systems, than a BUG_ON() crashed kernel.

Thanks,

Ingo

#4 lore raw
Quoted 42 lines · Ingo Molnar
Quoted 30 lines · Frederic Weisbecker

In x86, faults exit by executing the iret instruction, which then reenables NMIs if we faulted in NMI context. Then if a fault happens in NMI, another NMI can nest after the fault exits.

But we don't yet support nested NMIs because we have only one NMI stack. To prevent that, trigger a bug when a fault happens in NMI context.

Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc Peter Zijlstra <[email protected]>

---

I first thought about putting it in the vmalloc fault path only. But then I saw more occasions for the kernel to fault (kmemcheck or so), and so I thought it should be better put in the all in one path. But I suspect you won't like that conditional in the big x86 fault path.

 arch/x86/mm/fault.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/mm/fault.c +0 -0
index 4c4508e..80c997e 100644

arch/x86/mm/fault.c do_page_fault(struct pt_regs *regs, unsigned long error_code) · line 959

Quoted 5 lines
arch/x86/mm/fault.c +2 -0
@@ -955,6 +955,8 @@ do_page_fault(struct pt_regs *regs, unsigned long error_code)
 	int write;
 	int fault;
 
+	BUG_ON(in_nmi());
+

Hm, this is a fastpath, so it would be nice to put this into the vmalloc and kmemcheck paths (even though it's less clean that way).

Quoted 6 lines

Also, a WARN_ON_ONCE() would probably help more in getting bug reports out of such systems, than a BUG_ON() crashed kernel.

Thanks,

Ingo

Ok, will resend then.

Thanks.