kvm: x86: Add cr3 to struct kvm_debug_exit_arch

PATCH In review Jim Mattson 4 participants 6 messages Oct 24, 2019 – Nov 5, 2019

Subsystems (3)

x86 KVM KVM x86

lore

Share & tools

lore

Jim Mattson

author

2019-10-24 19:54 UTC #0 lore raw

PATCH v2 1/1 kvm: x86: Add cr3 to struct kvm_debug_exit_arch

From: Ken Hofsass <[email protected]>

A userspace agent can use cr3 to quickly determine whether a KVM_EXIT_DEBUG is associated with a guest process of interest.

KVM_CAP_DEBUG_EVENT_PDBR indicates support for the extension.

Signed-off-by Ken Hofsass <[email protected]>

Signed-off-by Jim Mattson <[email protected]>

Cc Peter Shier <[email protected]>

---
v1 -> v2: Changed KVM_CAP_DEBUG_EVENT_PG_BASE_ADDR to KVM_CAP_DEBUG_EVENT_PDBR
Set debug.arch.cr3 in kvm_vcpu_do_singlestep and
kvm_vcpu_check_breakpoint
Added svm support

 arch/x86/include/uapi/asm/kvm.h | 1 +
 arch/x86/kvm/svm.c              | 3 +++
 arch/x86/kvm/vmx/vmx.c          | 2 ++
 arch/x86/kvm/x86.c              | 3 +++
 include/uapi/linux/kvm.h        | 1 +
 5 files changed, 10 insertions(+)

arch/x86/include/uapi/asm/kvm.h +1 -0

index 503d3f42da167..cea355c7ee8e7 100644
@@ -254,6 +254,7 @@ struct kvm_debug_exit_arch {
 	__u64 pc;
 	__u64 dr6;
 	__u64 dr7;
+	__u64 cr3; /* Depends on KVM_CAP_DEBUG_EVENT_PDBR */
 };
 
 #define KVM_GUESTDBG_USE_SW_BP		0x00010000

arch/x86/kvm/svm.c +3 -0

index f8ecb6df51066..1a774d2c78eef 100644
@@ -2738,6 +2738,7 @@ static int db_interception(struct vcpu_svm *svm)
 		kvm_run->exit_reason = KVM_EXIT_DEBUG;
 		kvm_run->debug.arch.pc =
 			svm->vmcb->save.cs.base + svm->vmcb->save.rip;
+		kvm_run->debug.arch.cr3 = kvm_read_cr3(vcpu);
 		kvm_run->debug.arch.exception = DB_VECTOR;
 		return 0;
 	}

@@ -2748,9 +2749,11 @@ static int db_interception(struct vcpu_svm *svm)
 static int bp_interception(struct vcpu_svm *svm)
 {
 	struct kvm_run *kvm_run = svm->vcpu.run;
+	struct kvm_vcpu *vcpu = &svm->vcpu;
 
 	kvm_run->exit_reason = KVM_EXIT_DEBUG;
 	kvm_run->debug.arch.pc = svm->vmcb->save.cs.base + svm->vmcb->save.rip;
+	kvm_run->debug.arch.cr3 = kvm_read_cr3(vcpu);
 	kvm_run->debug.arch.exception = BP_VECTOR;
 	return 0;
 }

arch/x86/kvm/vmx/vmx.c +2 -0

index e7970a2e8eae9..736284d293c4a 100644
@@ -4690,6 +4690,7 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)
 		kvm_run->exit_reason = KVM_EXIT_DEBUG;
 		rip = kvm_rip_read(vcpu);
 		kvm_run->debug.arch.pc = vmcs_readl(GUEST_CS_BASE) + rip;
+		kvm_run->debug.arch.cr3 = kvm_read_cr3(vcpu);
 		kvm_run->debug.arch.exception = ex_no;
 		break;
 	default:

@@ -4909,6 +4910,7 @@ static int handle_dr(struct kvm_vcpu *vcpu)
 			vcpu->run->debug.arch.dr6 = vcpu->arch.dr6;
 			vcpu->run->debug.arch.dr7 = dr7;
 			vcpu->run->debug.arch.pc = kvm_get_linear_rip(vcpu);
+			vcpu->run->debug.arch.cr3 = kvm_read_cr3(vcpu);
 			vcpu->run->debug.arch.exception = DB_VECTOR;
 			vcpu->run->exit_reason = KVM_EXIT_DEBUG;
 			return 0;

arch/x86/kvm/x86.c +3 -0

index 661e2bf385266..2fd18b55462a9 100644
@@ -3222,6 +3222,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_GET_MSR_FEATURES:
 	case KVM_CAP_MSR_PLATFORM_INFO:
 	case KVM_CAP_EXCEPTION_PAYLOAD:
+	case KVM_CAP_DEBUG_EVENT_PDBR:
 		r = 1;
 		break;
 	case KVM_CAP_SYNC_REGS:

@@ -6490,6 +6491,7 @@ static int kvm_vcpu_do_singlestep(struct kvm_vcpu *vcpu)
 	if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) {
 		kvm_run->debug.arch.dr6 = DR6_BS | DR6_FIXED_1 | DR6_RTM;
 		kvm_run->debug.arch.pc = vcpu->arch.singlestep_rip;
+		kvm_run->debug.arch.cr3 = kvm_read_cr3(vcpu);
 		kvm_run->debug.arch.exception = DB_VECTOR;
 		kvm_run->exit_reason = KVM_EXIT_DEBUG;
 		return 0;

@@ -6534,6 +6536,7 @@ static bool kvm_vcpu_check_breakpoint(struct kvm_vcpu *vcpu, int *r)
 		if (dr6 != 0) {
 			kvm_run->debug.arch.dr6 = dr6 | DR6_FIXED_1 | DR6_RTM;
 			kvm_run->debug.arch.pc = eip;
+			kvm_run->debug.arch.cr3 = kvm_read_cr3(vcpu);
 			kvm_run->debug.arch.exception = DB_VECTOR;
 			kvm_run->exit_reason = KVM_EXIT_DEBUG;
 			*r = 0;

include/uapi/linux/kvm.h +1 -0

index 52641d8ca9e83..cde4b28338482 100644
@@ -1000,6 +1000,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_PMU_EVENT_FILTER 173
 #define KVM_CAP_ARM_IRQ_LINE_LAYOUT_2 174
 #define KVM_CAP_HYPERV_DIRECT_TLBFLUSH 175
+#define KVM_CAP_DEBUG_EVENT_PDBR 176
 
 #ifdef KVM_CAP_IRQ_ROUTING

Signature · 1 line

2.24.0.rc0.303.g954a862665-goog

Sean Christopherson

2019-10-24 20:21 UTC #1 lore raw

Quoted 6 lines · Jim Mattson

From: Ken Hofsass <[email protected]>
A userspace agent can use cr3 to quickly determine whether a KVM_EXIT_DEBUG is associated with a guest process of interest.
KVM_CAP_DEBUG_EVENT_PDBR indicates support for the extension.

Isn't PDBR x86-specific terminology? If we're going to use something that is x86-specific then just call it GUEST_CR3. If there's a chance that this is useful on other architectures then it'd probably be better to go with Linux's own terminology, e.g. KVM_CAP_DEBUG_EVENT_GUEST_PGD.

Quoted 8 lines

Signed-off-by Ken Hofsass <[email protected]>
Signed-off-by Jim Mattson <[email protected]>
Cc Peter Shier <[email protected]>
---
v1 -> v2: Changed KVM_CAP_DEBUG_EVENT_PG_BASE_ADDR to KVM_CAP_DEBUG_EVENT_PDBR
Set debug.arch.cr3 in kvm_vcpu_do_singlestep and
kvm_vcpu_check_breakpoint
Added svm support

Heh, I wonder what the record is for longest time between legitimate versions of a single patch.

Paolo Bonzini

maintainer

2019-10-24 22:18 UTC #2 lore raw

In reply to Jim Mattson

Quoted 7 lines · Jim Mattson

From: Ken Hofsass <[email protected]>
A userspace agent can use cr3 to quickly determine whether a KVM_EXIT_DEBUG is associated with a guest process of interest.
KVM_CAP_DEBUG_EVENT_PDBR indicates support for the extension.

Quoted 8 lines

Signed-off-by Ken Hofsass <[email protected]>
Signed-off-by Jim Mattson <[email protected]>
Cc Peter Shier <[email protected]>
---
v1 -> v2: Changed KVM_CAP_DEBUG_EVENT_PG_BASE_ADDR to KVM_CAP_DEBUG_EVENT_PDBR
Set debug.arch.cr3 in kvm_vcpu_do_singlestep and
kvm_vcpu_check_breakpoint
Added svm support

Perhaps you have already considered using KVM_CAP_SYNC_REGS instead, since Google contributed it in the first place, but anyway... would it be enough for userspace to request KVM_SYNC_X86_SREGS when it enables breakpoints or singlestep?

Thanks,

Paolo

Ken Hofsass

2019-10-25 17:09 UTC #3 lore raw

Quoted 17 lines · Paolo Bonzini

Quoted 7 lines · Jim Mattson
From: Ken Hofsass <[email protected]>
A userspace agent can use cr3 to quickly determine whether a KVM_EXIT_DEBUG is associated with a guest process of interest.
KVM_CAP_DEBUG_EVENT_PDBR indicates support for the extension.
Quoted 8 lines
Signed-off-by Ken Hofsass <[email protected]>
Signed-off-by Jim Mattson <[email protected]>
Cc Peter Shier <[email protected]>
---
v1 -> v2: Changed KVM_CAP_DEBUG_EVENT_PG_BASE_ADDR to KVM_CAP_DEBUG_EVENT_PDBR
Set debug.arch.cr3 in kvm_vcpu_do_singlestep and
kvm_vcpu_check_breakpoint
Added svm support

Quoted 4 lines

Perhaps you have already considered using KVM_CAP_SYNC_REGS instead, since Google contributed it in the first place, but anyway... would it be enough for userspace to request KVM_SYNC_X86_SREGS when it enables breakpoints or singlestep?

Hi Paolo, from a functional perspective, using KVM_SYNC_X86_SREGS is totally reasonable. But it currently introduces a non-trivial amount of overhead because it affects all exits.

This change is a targeted optimization for use in instrumentation scenarios. Specifically where debug breakpoint exits are a small percentage of total exits and only a small percentage of the debug exits are from processes of interest.

thanks,
Ken

Jim Mattson

author

2019-11-05 19:59 UTC #4 lore raw

Quoted 28 lines · Ken Hofsass

Quoted 17 lines · Paolo Bonzini
Quoted 7 lines · Jim Mattson
From: Ken Hofsass <[email protected]>
A userspace agent can use cr3 to quickly determine whether a KVM_EXIT_DEBUG is associated with a guest process of interest.
KVM_CAP_DEBUG_EVENT_PDBR indicates support for the extension.
Quoted 8 lines
Signed-off-by Ken Hofsass <[email protected]>
Signed-off-by Jim Mattson <[email protected]>
Cc Peter Shier <[email protected]>
---
v1 -> v2: Changed KVM_CAP_DEBUG_EVENT_PG_BASE_ADDR to KVM_CAP_DEBUG_EVENT_PDBR
Set debug.arch.cr3 in kvm_vcpu_do_singlestep and
kvm_vcpu_check_breakpoint
Added svm support
Quoted 4 lines
Perhaps you have already considered using KVM_CAP_SYNC_REGS instead, since Google contributed it in the first place, but anyway... would it be enough for userspace to request KVM_SYNC_X86_SREGS when it enables breakpoints or singlestep?
Hi Paolo, from a functional perspective, using KVM_SYNC_X86_SREGS is totally reasonable. But it currently introduces a non-trivial amount of overhead because it affects all exits.

Quoted 7 lines

This change is a targeted optimization for use in instrumentation scenarios. Specifically where debug breakpoint exits are a small percentage of total exits and only a small percentage of the debug exits are from processes of interest.
thanks,
Ken

Another possibility would be to add flags to KVM_SET_GET_DEBUG that request a SYNC_REGS on a breakpoint or single step.

Paolo Bonzini

maintainer

2019-11-05 22:23 UTC #5 lore raw

In reply to Ken Hofsass

Quoted 23 lines · Ken Hofsass

Quoted 17 lines · Paolo Bonzini
Quoted 7 lines · Jim Mattson
From: Ken Hofsass <[email protected]>
A userspace agent can use cr3 to quickly determine whether a KVM_EXIT_DEBUG is associated with a guest process of interest.
KVM_CAP_DEBUG_EVENT_PDBR indicates support for the extension.
Quoted 8 lines
Signed-off-by Ken Hofsass <[email protected]>
Signed-off-by Jim Mattson <[email protected]>
Cc Peter Shier <[email protected]>
---
v1 -> v2: Changed KVM_CAP_DEBUG_EVENT_PG_BASE_ADDR to KVM_CAP_DEBUG_EVENT_PDBR
Set debug.arch.cr3 in kvm_vcpu_do_singlestep and
kvm_vcpu_check_breakpoint
Added svm support
Quoted 4 lines
Perhaps you have already considered using KVM_CAP_SYNC_REGS instead, since Google contributed it in the first place, but anyway... would it be enough for userspace to request KVM_SYNC_X86_SREGS when it enables breakpoints or singlestep?

Quoted 8 lines

Hi Paolo, from a functional perspective, using KVM_SYNC_X86_SREGS is totally reasonable. But it currently introduces a non-trivial amount of overhead because it affects all exits.
This change is a targeted optimization for use in instrumentation scenarios. Specifically where debug breakpoint exits are a small percentage of total exits and only a small percentage of the debug exits are from processes of interest.

Sorry for the delayed response, mostly due to KVM Forum.

Is this due to Google's non-upstream userspace emulation patches? With a properly configured guest and in-kernel emulation, userspace exits should be so few as to make KVM_CAP_SYNC_REGS's overhead low.

But in any case, what about adding a new KVM_SYNC_X86_CR3 that is a subset of SREGS?

Thanks,

Paolo