Skip to contentSkip to conversation

x86: fix page fault tracing when KVM guest support enabled

PATCH In review Dave Hansen 4 participants 5 messages May 16, 2014

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86: fix page fault tracing when KVM guest support enabled

From: Dave Hansen <[email protected]>

I noticed on some of my systems that page fault tracing doesn't work:

cd /sys/kernel/debug/tracing
echo 1 > events/exceptions/enable
cat trace;
# nothing shows up

I eventually traced it down to CONFIG_KVM_GUEST. At least in a KVM VM, enabling that option breaks page fault tracing, and disabling fixes it. I tried on some old kernels and this does not appear to be a regression: it never worked.

There are two page-fault entry functions today. One when tracing is on and another when it is off. The KVM code calls do_page_fault() directly instead of calling the traced version:

Quoted 11 lines

dotraplinkage void __kprobes do_async_page_fault(struct pt_regs *regs, unsigned long error_code)

{
        enum ctx_state prev_state;

        switch (kvm_read_and_reset_pf_reason()) {
        default:
                do_page_fault(regs, error_code);
                break;
        case KVM_PV_REASON_PAGE_NOT_PRESENT:

I'm also having problems with the page fault tracing on bare metal (same symptom of no trace output). I'm unsure if it's related.

Steven had an alternative to this which has zero overhead when tracing is off where this includes the standard noops even when tracing is disabled. I'm unconvinced that the extra complexity of his apporach:

	http://lkml.kernel.org/r/[email protected]

is worth it, expecially considering that the KVM code is already making page fault entry slower here. This solution is dirt-simple.

Gleb, please apply.

Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Gleb Natapov <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Steven Rostedt <[email protected]>

---

 b/arch/x86/include/asm/traps.h |    5 +++++
 b/arch/x86/kernel/kvm.c        |    2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)
arch/x86/include/asm/traps.h renamed from arch/x86/include/asm/traps.h~muck-with-kvm-guest-code +5 -0
diff -puN arch/x86/include/asm/traps.h~muck-with-kvm-guest-code arch/x86/include/asm/traps.h
@@ -74,6 +74,11 @@ dotraplinkage void do_general_protection
 dotraplinkage void do_page_fault(struct pt_regs *, unsigned long);
 #ifdef CONFIG_TRACING
 dotraplinkage void trace_do_page_fault(struct pt_regs *, unsigned long);
+#else
+static inline void trace_do_page_fault(struct pt_regs *regs, unsigned long error)
+{
+	do_page_fault(regs, error);
+}
 #endif
 dotraplinkage void do_spurious_interrupt_bug(struct pt_regs *, long);
 dotraplinkage void do_coprocessor_error(struct pt_regs *, long);
arch/x86/kernel/kvm.c renamed from arch/x86/kernel/kvm.c~muck-with-kvm-guest-code +1 -1
diff -puN arch/x86/kernel/kvm.c~muck-with-kvm-guest-code arch/x86/kernel/kvm.c
@@ -259,7 +259,7 @@ do_async_page_fault(struct pt_regs *regs
 
 	switch (kvm_read_and_reset_pf_reason()) {
 	default:
-		do_page_fault(regs, error_code);
+		trace_do_page_fault(regs, error_code);
 		break;
 	case KVM_PV_REASON_PAGE_NOT_PRESENT:
 		/* page is swapped out by the host. */

_

Acked-by on v1 1/1
#1 lore raw
Quoted 11 lines · On Fri, 16 May 2014 12:45:15 -0700 Dave Hansen <[email protected]> wrote:

Steven had an alternative to this which has zero overhead when tracing is off where this includes the standard noops even when tracing is disabled. I'm unconvinced that the extra complexity of his apporach:

	http://lkml.kernel.org/r/[email protected]

is worth it, expecially considering that the KVM code is already making page fault entry slower here. This solution is dirt-simple.

I just threw it out there as a suggestion. I don't care either way.

You probably need an acked-by from hpa or one of the other x86 maintainers as it touches the generic traps.h header.

-- Steve

Quoted 46 lines

Gleb, please apply.

Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Gleb Natapov <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Steven Rostedt <[email protected]>

---

 b/arch/x86/include/asm/traps.h |    5 +++++
 b/arch/x86/kernel/kvm.c        |    2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff -puN arch/x86/include/asm/traps.h~muck-with-kvm-guest-code arch/x86/include/asm/traps.h

arch/x86/include/asm/traps.h renamed from arch/x86/include/asm/traps.h~muck-with-kvm-guest-code +5 -0
@@ -74,6 +74,11 @@ dotraplinkage void do_general_protection
 dotraplinkage void do_page_fault(struct pt_regs *, unsigned long);
 #ifdef CONFIG_TRACING
 dotraplinkage void trace_do_page_fault(struct pt_regs *, unsigned long);
+#else
+static inline void trace_do_page_fault(struct pt_regs *regs, unsigned long error)
+{
+	do_page_fault(regs, error);
+}
 #endif
 dotraplinkage void do_spurious_interrupt_bug(struct pt_regs *, long);
 dotraplinkage void do_coprocessor_error(struct pt_regs *, long);

diff -puN arch/x86/kernel/kvm.c~muck-with-kvm-guest-code arch/x86/kernel/kvm.c

arch/x86/kernel/kvm.c renamed from arch/x86/kernel/kvm.c~muck-with-kvm-guest-code +1 -1
@@ -259,7 +259,7 @@ do_async_page_fault(struct pt_regs *regs
 
 	switch (kvm_read_and_reset_pf_reason()) {
 	default:
-		do_page_fault(regs, error_code);
+		trace_do_page_fault(regs, error_code);
 		break;
 	case KVM_PV_REASON_PAGE_NOT_PRESENT:
 		/* page is swapped out by the host. */

_

reviewer Acked-by on v1 1/1
#2 lore raw
In reply to Dave Hansen
Quoted 48 lines · Dave Hansen

From: Dave Hansen <[email protected]>

I noticed on some of my systems that page fault tracing doesn't work:

cd /sys/kernel/debug/tracing
echo 1 > events/exceptions/enable
cat trace;
# nothing shows up

I eventually traced it down to CONFIG_KVM_GUEST. At least in a KVM VM, enabling that option breaks page fault tracing, and disabling fixes it. I tried on some old kernels and this does not appear to be a regression: it never worked.

There are two page-fault entry functions today. One when tracing is on and another when it is off. The KVM code calls do_page_fault() directly instead of calling the traced version:

Quoted 11 lines

dotraplinkage void __kprobes do_async_page_fault(struct pt_regs *regs, unsigned long error_code)

{
        enum ctx_state prev_state;

        switch (kvm_read_and_reset_pf_reason()) {
        default:
                do_page_fault(regs, error_code);
                break;
        case KVM_PV_REASON_PAGE_NOT_PRESENT:

I'm also having problems with the page fault tracing on bare metal (same symptom of no trace output). I'm unsure if it's related.

Steven had an alternative to this which has zero overhead when tracing is off where this includes the standard noops even when tracing is disabled. I'm unconvinced that the extra complexity of his apporach:

	http://lkml.kernel.org/r/[email protected]

is worth it, expecially considering that the KVM code is already making page fault entry slower here. This solution is dirt-simple.

Gleb, please apply.

Quoted 9 lines
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Gleb Natapov <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Steven Rostedt <[email protected]>

If Gleb and Paolo are okay with it, I am.

-hpa

maintainer
#3 lore raw

Il 16/05/2014 22:53, H. Peter Anvin ha scritto:

Quoted 59 lines · Dave Hansen
Quoted 48 lines

From: Dave Hansen <[email protected]>

I noticed on some of my systems that page fault tracing doesn't work:

cd /sys/kernel/debug/tracing
echo 1 > events/exceptions/enable
cat trace;
# nothing shows up

I eventually traced it down to CONFIG_KVM_GUEST. At least in a KVM VM, enabling that option breaks page fault tracing, and disabling fixes it. I tried on some old kernels and this does not appear to be a regression: it never worked.

There are two page-fault entry functions today. One when tracing is on and another when it is off. The KVM code calls do_page_fault() directly instead of calling the traced version:

Quoted 11 lines

dotraplinkage void __kprobes do_async_page_fault(struct pt_regs *regs, unsigned long error_code)

{
        enum ctx_state prev_state;

        switch (kvm_read_and_reset_pf_reason()) {
        default:
                do_page_fault(regs, error_code);
                break;
        case KVM_PV_REASON_PAGE_NOT_PRESENT:

I'm also having problems with the page fault tracing on bare metal (same symptom of no trace output). I'm unsure if it's related.

Steven had an alternative to this which has zero overhead when tracing is off where this includes the standard noops even when tracing is disabled. I'm unconvinced that the extra complexity of his apporach:

	http://lkml.kernel.org/r/[email protected]

is worth it, expecially considering that the KVM code is already making page fault entry slower here. This solution is dirt-simple.

Gleb, please apply.

Quoted 9 lines
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Gleb Natapov <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Paolo Bonzini <[email protected]>
Cc Steven Rostedt <[email protected]>
Quoted 3 lines

If Gleb and Paolo are okay with it, I am.

Yes, of course. Dave, ok to only have it in 3.16?

Paolo

#4 lore raw
Quoted 1 line · Paolo Bonzini

Yes, of course. Dave, ok to only have it in 3.16?

Sure, it's been broken for a long time, so it's no hurry to get fixed.