Skip to contentSkip to conversation

KVM: Enhancements and fixes around guest debugging

PATCH In review Jan Kiszka 2 participants 10 messages Feb 23, 2010 – Feb 24, 2010

Subsystems (3)
lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

v2 cover KVM: Enhancements and fixes around guest debugging

Changes in v2:
- Retreat TF emulation patch, waiting for emulator rework
- Spilt out svm_queue_exception moving
- Factor out kvm_is_linear_rip ("Saved linear RIP == current RIP?")
- Fixed breakage in INT3 emulation code and addressed review comments

Patch 5 targets also 2.6.33., patch 1 should additionally be considered for stable 2.6.32.

You may pull these patches from

git://git.kiszka.org/linux-kvm gdb

Jan Kiszka (7):
KVM: VMX: Update instruction length on intercepted BP
KVM: SVM: Move svm_queue_exception
KVM: x86: Add kvm_is_linear_rip
KVM: SVM: Emulate nRIP feature when reinjecting INT3
KVM: x86: Add KVM_CAP_X86_ROBUST_SINGLESTEP
KVM: x86: Drop RF manipulation for guest single-stepping
KVM: x86: Preserve injected TF across emulation

 arch/x86/include/asm/kvm_host.h |    4 ++-
 arch/x86/kvm/svm.c              |   72 ++++++++++++++++++++++++++++-----------
 arch/x86/kvm/vmx.c              |   13 +++++++
 arch/x86/kvm/x86.c              |   30 ++++++++++------
 include/linux/kvm.h             |    1 +
 5 files changed, 87 insertions(+), 33 deletions(-)
author
#1 lore raw

PATCH v2 1/7 KVM: VMX: Update instruction length on intercepted BP

We intercept #BP while in guest debugging mode. As VM exits due to intercepted exceptions do not necessarily come with valid idt_vectoring, we have to update event_exit_inst_len explicitly in such cases. At least in the absence of migration, this ensures that re-injections of #BP will find and use the correct instruction length.

---
 arch/x86/kvm/vmx.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)
arch/x86/kvm/vmx.c +13 -0
index ce5ec41..d772476 100644
@@ -2775,6 +2775,12 @@ static int handle_rmode_exception(struct kvm_vcpu *vcpu,
 		kvm_queue_exception(vcpu, vec);
 		return 1;
 	case BP_VECTOR:
+		/*
+		 * Update instruction length as we may reinject the exception
+		 * from user space while in guest debugging mode.
+		 */
+		to_vmx(vcpu)->vcpu.arch.event_exit_inst_len =
+			vmcs_read32(VM_EXIT_INSTRUCTION_LEN);
 		if (vcpu->guest_debug & KVM_GUESTDBG_USE_SW_BP)
 			return 0;
 		/* fall through */
@@ -2897,6 +2903,13 @@ static int handle_exception(struct kvm_vcpu *vcpu)
 		kvm_run->debug.arch.dr7 = vmcs_readl(GUEST_DR7);
 		/* fall through */
 	case BP_VECTOR:
+		/*
+		 * Update instruction length as we may reinject #BP from
+		 * user space while in guest debugging mode. Reading it for
+		 * #DB as well causes no harm, it is not used in that case.
+		 */
+		vmx->vcpu.arch.event_exit_inst_len =
+			vmcs_read32(VM_EXIT_INSTRUCTION_LEN);
 		kvm_run->exit_reason = KVM_EXIT_DEBUG;
 		kvm_run->debug.arch.pc = vmcs_readl(GUEST_CS_BASE) + rip;
 		kvm_run->debug.arch.exception = ex_no;
Signature · 1 line
1.6.0.2
author
#2 lore raw

PATCH v2 2/7 KVM: SVM: Move svm_queue_exception

In reply to Jan Kiszka

Move svm_queue_exception past skip_emulated_instruction to allow calling it later on.

---
 arch/x86/kvm/svm.c |   34 +++++++++++++++++-----------------
 1 files changed, 17 insertions(+), 17 deletions(-)
arch/x86/kvm/svm.c +17 -17
index 1d76899..2fceac8 100644
@@ -235,23 +235,6 @@ static void svm_set_efer(struct kvm_vcpu *vcpu, u64 efer)
 	vcpu->arch.efer = efer;
 }
 
-static void svm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr,
-				bool has_error_code, u32 error_code)
-{
-	struct vcpu_svm *svm = to_svm(vcpu);
-
-	/* If we are within a nested VM we'd better #VMEXIT and let the
-	   guest handle the exception */
-	if (nested_svm_check_exception(svm, nr, has_error_code, error_code))
-		return;
-
-	svm->vmcb->control.event_inj = nr
-		| SVM_EVTINJ_VALID
-		| (has_error_code ? SVM_EVTINJ_VALID_ERR : 0)
-		| SVM_EVTINJ_TYPE_EXEPT;
-	svm->vmcb->control.event_inj_err = error_code;
-}
-
 static int is_external_interrupt(u32 info)
 {
 	info &= SVM_EVTINJ_TYPE_MASK | SVM_EVTINJ_VALID;
@@ -297,6 +280,23 @@ static void skip_emulated_instruction(struct kvm_vcpu *vcpu)
 	svm_set_interrupt_shadow(vcpu, 0);
 }
 
+static void svm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr,
+				bool has_error_code, u32 error_code)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+
+	/* If we are within a nested VM we'd better #VMEXIT and let the
+	   guest handle the exception */
+	if (nested_svm_check_exception(svm, nr, has_error_code, error_code))
+		return;
+
+	svm->vmcb->control.event_inj = nr
+		| SVM_EVTINJ_VALID
+		| (has_error_code ? SVM_EVTINJ_VALID_ERR : 0)
+		| SVM_EVTINJ_TYPE_EXEPT;
+	svm->vmcb->control.event_inj_err = error_code;
+}
+
 static int has_svm(void)
 {
 	const char *msg;
Signature · 1 line
1.6.0.2
author
#3 lore raw

PATCH v2 3/7 KVM: x86: Add kvm_is_linear_rip

In reply to Jan Kiszka

Based on Gleb's suggestion: Add a helper kvm_is_linear_rip that matches a given linear RIP against the current one. Use this for guest single-stepping, more users will follow.

---
 arch/x86/include/asm/kvm_host.h |    4 +++-
 arch/x86/kvm/x86.c              |   21 +++++++++++++--------
 2 files changed, 16 insertions(+), 9 deletions(-)
arch/x86/include/asm/kvm_host.h +3 -1
index d46e791..502fff1 100644
@@ -362,8 +362,8 @@ struct kvm_vcpu_arch {
 	u64 *mce_banks;
 
 	/* used for guest single stepping over the given code position */
-	u16 singlestep_cs;
 	unsigned long singlestep_rip;
+
 	/* fields used by HYPER-V emulation */
 	u64 hv_vapic;
 };
@@ -820,4 +820,6 @@ int kvm_cpu_get_interrupt(struct kvm_vcpu *v);
 void kvm_define_shared_msr(unsigned index, u32 msr);
 void kvm_set_shared_msr(unsigned index, u64 val, u64 mask);
 
+bool kvm_is_linear_rip(struct kvm_vcpu *vcpu, unsigned long linear_rip);
+
 #endif /* _ASM_X86_KVM_HOST_H */
arch/x86/kvm/x86.c +13 -8
index e25a522..1a97c43 100644
@@ -5390,11 +5390,9 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
 		vcpu->arch.switch_db_regs = (vcpu->arch.dr7 & DR7_BP_EN_MASK);
 	}
 
-	if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) {
-		vcpu->arch.singlestep_cs =
-			get_segment_selector(vcpu, VCPU_SREG_CS);
-		vcpu->arch.singlestep_rip = kvm_rip_read(vcpu);
-	}
+	if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP)
+		vcpu->arch.singlestep_rip = kvm_rip_read(vcpu) +
+			get_segment_base(vcpu, VCPU_SREG_CS);
 
 	/*
 	 * Trigger an rflags update that will inject or remove the trace
@@ -5885,6 +5883,15 @@ int kvm_arch_interrupt_allowed(struct kvm_vcpu *vcpu)
 	return kvm_x86_ops->interrupt_allowed(vcpu);
 }
 
+bool kvm_is_linear_rip(struct kvm_vcpu *vcpu, unsigned long linear_rip)
+{
+	unsigned long current_rip = kvm_rip_read(vcpu) +
+		get_segment_base(vcpu, VCPU_SREG_CS);
+
+	return current_rip == linear_rip;
+}
+EXPORT_SYMBOL_GPL(kvm_is_linear_rip);
+
 unsigned long kvm_get_rflags(struct kvm_vcpu *vcpu)
 {
 	unsigned long rflags;
@@ -5899,9 +5906,7 @@ EXPORT_SYMBOL_GPL(kvm_get_rflags);
 void kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)
 {
 	if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP &&
-	    vcpu->arch.singlestep_cs ==
-			get_segment_selector(vcpu, VCPU_SREG_CS) &&
-	    vcpu->arch.singlestep_rip == kvm_rip_read(vcpu))
+	    kvm_is_linear_rip(vcpu, vcpu->arch.singlestep_rip))
 		rflags |= X86_EFLAGS_TF | X86_EFLAGS_RF;
 	kvm_x86_ops->set_rflags(vcpu, rflags);
 }
Signature · 1 line
1.6.0.2
author
#4 lore raw

PATCH v2 4/7 KVM: SVM: Emulate nRIP feature when reinjecting INT3

In reply to Jan Kiszka

When in guest debugging mode, we have to reinject those #BP software exceptions that are caused by guest-injected INT3. As older AMD processors do not support the required nRIP VMCB field, try to emulate it by moving RIP past the instruction on exception injection. Fix it up again in case the injection failed and we were able to catch this. This does not work for unintercepted faults, but it is better than doing nothing.

---
 arch/x86/kvm/svm.c |   38 +++++++++++++++++++++++++++++++++++---
 1 files changed, 35 insertions(+), 3 deletions(-)
arch/x86/kvm/svm.c +35 -3
index 2fceac8..d11ff46 100644
@@ -46,6 +46,7 @@ MODULE_LICENSE("GPL");
 #define SVM_FEATURE_NPT  (1 << 0)
 #define SVM_FEATURE_LBRV (1 << 1)
 #define SVM_FEATURE_SVML (1 << 2)
+#define SVM_FEATURE_NRIP (1 << 3)
 #define SVM_FEATURE_PAUSE_FILTER (1 << 10)
 
 #define NESTED_EXIT_HOST	0	/* Exit handled on host level */
@@ -109,6 +110,9 @@ struct vcpu_svm {
 	struct nested_state nested;
 
 	bool nmi_singlestep;
+
+	unsigned int3_injected;
+	unsigned long int3_rip;
 };
 
 /* enable NPT for AMD64 and X86 with PAE */
@@ -290,6 +294,22 @@ static void svm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr,
 	if (nested_svm_check_exception(svm, nr, has_error_code, error_code))
 		return;
 
+	if (nr == BP_VECTOR && !svm_has(SVM_FEATURE_NRIP)) {
+		unsigned long rip, old_rip = kvm_rip_read(&svm->vcpu);
+
+		/*
+		 * For guest debugging where we have to reinject #BP if some
+		 * INT3 is guest-owned:
+		 * Emulate nRIP by moving RIP forward. Will fail if injection
+		 * raises a fault that is not intercepted. Still better than
+		 * failing in all cases.
+		 */
+		skip_emulated_instruction(&svm->vcpu);
+		rip = kvm_rip_read(&svm->vcpu);
+		svm->int3_rip = rip + svm->vmcb->save.cs.base;
+		svm->int3_injected = rip - old_rip;
+	}
+
 	svm->vmcb->control.event_inj = nr
 		| SVM_EVTINJ_VALID
 		| (has_error_code ? SVM_EVTINJ_VALID_ERR : 0)
@@ -2695,6 +2715,9 @@ static void svm_complete_interrupts(struct vcpu_svm *svm)
 	u8 vector;
 	int type;
 	u32 exitintinfo = svm->vmcb->control.exit_int_info;
+	unsigned int3_injected = svm->int3_injected;
+
+	svm->int3_injected = 0;
 
 	if (svm->vcpu.arch.hflags & HF_IRET_MASK)
 		svm->vcpu.arch.hflags &= ~(HF_NMI_MASK | HF_IRET_MASK);
@@ -2714,12 +2737,21 @@ static void svm_complete_interrupts(struct vcpu_svm *svm)
 		svm->vcpu.arch.nmi_injected = true;
 		break;
 	case SVM_EXITINTINFO_TYPE_EXEPT:
-		/* In case of software exception do not reinject an exception
-		   vector, but re-execute and instruction instead */
 		if (is_nested(svm))
 			break;
-		if (kvm_exception_is_soft(vector))
+		/*
+		 * In case of software exceptions, do not reinject the vector,
+		 * but re-execute the instruction instead. Rewind RIP first
+		 * if we emulated INT3 before.
+		 */
+		if (kvm_exception_is_soft(vector)) {
+			if (vector == BP_VECTOR && int3_injected &&
+			    kvm_is_linear_rip(&svm->vcpu, svm->int3_rip))
+				kvm_rip_write(&svm->vcpu,
+					      kvm_rip_read(&svm->vcpu) -
+					      int3_injected);
 			break;
+		}
 		if (exitintinfo & SVM_EXITINTINFO_VALID_ERR) {
 			u32 err = svm->vmcb->control.exit_int_info_err;
 			kvm_queue_exception_e(&svm->vcpu, vector, err);
Signature · 1 line
1.6.0.2
author
#5 lore raw

PATCH v2 5/7 KVM: x86: Add KVM_CAP_X86_ROBUST_SINGLESTEP

In reply to Jan Kiszka

This marks the guest single-step API improvement of 94fe45da and 91586a3b with a capability flag to allow reliable detection by user space.

---
 arch/x86/kvm/x86.c  |    1 +
 include/linux/kvm.h |    1 +
 2 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/kvm/x86.c +1 -0
index 1a97c43..7abc3c0 100644
@@ -1571,6 +1571,7 @@ int kvm_dev_ioctl_check_extension(long ext)
 	case KVM_CAP_HYPERV_SPIN:
 	case KVM_CAP_PCI_SEGMENT:
 	case KVM_CAP_DEBUGREGS:
+	case KVM_CAP_X86_ROBUST_SINGLESTEP:
 		r = 1;
 		break;
 	case KVM_CAP_COALESCED_MMIO:
include/linux/kvm.h +1 -0
index d25912e..ce28767 100644
@@ -506,6 +506,7 @@ struct kvm_ioeventfd {
 #ifdef __KVM_HAVE_DEBUGREGS
 #define KVM_CAP_DEBUGREGS 50
 #endif
+#define KVM_CAP_X86_ROBUST_SINGLESTEP 51
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
Signature · 1 line
1.6.0.2
author
#6 lore raw

PATCH v2 6/7 KVM: x86: Drop RF manipulation for guest single-stepping

In reply to Jan Kiszka

RF is not required for injecting TF as the latter will trigger only after an instruction execution anyway. So do not touch RF when arming or disarming guest single-step mode.

---
 arch/x86/kvm/x86.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +2 -2
index 7abc3c0..8181595 100644
@@ -5899,7 +5899,7 @@ unsigned long kvm_get_rflags(struct kvm_vcpu *vcpu)
 
 	rflags = kvm_x86_ops->get_rflags(vcpu);
 	if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP)
-		rflags &= ~(unsigned long)(X86_EFLAGS_TF | X86_EFLAGS_RF);
+		rflags &= ~X86_EFLAGS_TF;
 	return rflags;
 }
 EXPORT_SYMBOL_GPL(kvm_get_rflags);
@@ -5908,7 +5908,7 @@ void kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)
 {
 	if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP &&
 	    kvm_is_linear_rip(vcpu, vcpu->arch.singlestep_rip))
-		rflags |= X86_EFLAGS_TF | X86_EFLAGS_RF;
+		rflags |= X86_EFLAGS_TF;
 	kvm_x86_ops->set_rflags(vcpu, rflags);
 }
 EXPORT_SYMBOL_GPL(kvm_set_rflags);
Signature · 1 line
1.6.0.2
author
#7 lore raw

PATCH v2 7/7 KVM: x86: Preserve injected TF across emulation

In reply to Jan Kiszka

Call directly into the vendor services for getting/setting rflags in emulate_instruction to ensure injected TF survives the emulation.

---
 arch/x86/kvm/x86.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +2 -2
index 8181595..7b436c8 100644
@@ -3468,7 +3468,7 @@ int emulate_instruction(struct kvm_vcpu *vcpu,
 		kvm_x86_ops->get_cs_db_l_bits(vcpu, &cs_db, &cs_l);
 
 		vcpu->arch.emulate_ctxt.vcpu = vcpu;
-		vcpu->arch.emulate_ctxt.eflags = kvm_get_rflags(vcpu);
+		vcpu->arch.emulate_ctxt.eflags = kvm_x86_ops->get_rflags(vcpu);
 		vcpu->arch.emulate_ctxt.mode =
 			(!is_protmode(vcpu)) ? X86EMUL_MODE_REAL :
 			(vcpu->arch.emulate_ctxt.eflags & X86_EFLAGS_VM)
@@ -3547,7 +3547,7 @@ int emulate_instruction(struct kvm_vcpu *vcpu,
 		return EMULATE_DO_MMIO;
 	}
 
-	kvm_set_rflags(vcpu, vcpu->arch.emulate_ctxt.eflags);
+	kvm_x86_ops->set_rflags(vcpu, vcpu->arch.emulate_ctxt.eflags);
 
 	if (vcpu->mmio_is_write) {
 		vcpu->mmio_needed = 0;
Signature · 1 line
1.6.0.2
#8 lore raw
In reply to Jan Kiszka
Quoted 9 lines · Jan Kiszka

Changes in v2:
- Retreat TF emulation patch, waiting for emulator rework
- Spilt out svm_queue_exception moving
- Factor out kvm_is_linear_rip ("Saved linear RIP == current RIP?")
- Fixed breakage in INT3 emulation code and addressed review comments

Patch 5 targets also 2.6.33., patch 1 should additionally be considered for stable 2.6.32.

All applied, thanks.

Signature · 1 line
error compiling committee.c: too many arguments to function
#9 lore raw
Quoted 7 lines · Avi Kivity
Quoted 6 lines · Jan Kiszka

Changes in v2:
- Retreat TF emulation patch, waiting for emulator rework
- Spilt out svm_queue_exception moving
- Factor out kvm_is_linear_rip ("Saved linear RIP == current RIP?")
- Fixed breakage in INT3 emulation code and addressed review comments

Quoted 5 lines
Quoted 2 lines

Patch 5 targets also 2.6.33., patch 1 should additionally be considered for stable 2.6.32.

All applied, thanks.

Oh, and 1 queued for .34 and .33, 5 queued for .33.

Signature · 1 line
error compiling committee.c: too many arguments to function