Skip to contentSkip to conversation

KVM: my debug patch queue

PATCH Superseded Maxim Levitsky 3 participants 16 messages Apr 1, 2021 – Apr 27, 2021

Subsystems (9)
Documentation x86 KVM arm64 +5 more subsystems

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover KVM: my debug patch queue

Hi!

I would like to publish two debug features which were needed for other stuff I work on.

One is the reworked lx-symbols script which now actually works on at least gdb 9.1 (gdb 9.2 was reported to fail to load the debug symbols from the kernel for some reason, not related to this patch) and upstream qemu.

The other feature is the ability to trap all guest exceptions (on SVM for now) and see them in kvmtrace prior to potential merge to double/triple fault.

This can be very useful and I already had to manually patch KVM a few times for this.
I will, once time permits, implement this feature on Intel as well.

V2:

* Some more refactoring and workarounds for lx-symbols script

* added KVM_GUESTDBG_BLOCKEVENTS flag to enable 'block interrupts on single step' together with KVM_CAP_SET_GUEST_DEBUG2 capability to indicate which guest debug flags are supported.

This is a replacement for unconditional block of interrupts on single step that was done in previous version of this patch set. Patches to qemu to use that feature will be sent soon.

* Reworked the the 'intercept all exceptions for debug' feature according to the review feedback:

- renamed the parameter that enables the feature and moved it to common kvm module.
(only SVM part is currently implemented though)

- disable the feature for SEV guests as was suggested during the review
- made the vmexit table const again, as was suggested in the review as well.

Best regards,
Maxim Levitsky

Maxim Levitsky (9):
scripts/gdb: rework lx-symbols gdb script
KVM: introduce KVM_CAP_SET_GUEST_DEBUG2
KVM: x86: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: aarch64: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: s390x: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: x86: implement KVM_GUESTDBG_BLOCKEVENTS
KVM: SVM: split svm_handle_invalid_exit
KVM: x86: add force_intercept_exceptions_mask
KVM: SVM: implement force_intercept_exceptions_mask

 Documentation/virt/kvm/api.rst    |   4 +
 arch/arm64/include/asm/kvm_host.h |   4 +
 arch/arm64/kvm/arm.c              |   2 +
 arch/arm64/kvm/guest.c            |   5 -
 arch/s390/include/asm/kvm_host.h  |   4 +
 arch/s390/kvm/kvm-s390.c          |   3 +
 arch/x86/include/asm/kvm_host.h   |  12 ++
 arch/x86/include/uapi/asm/kvm.h   |   1 +
 arch/x86/kvm/svm/svm.c            |  87 +++++++++++--
 arch/x86/kvm/svm/svm.h            |   6 +-
 arch/x86/kvm/x86.c                |  14 ++-
 arch/x86/kvm/x86.h                |   2 +
 include/uapi/linux/kvm.h          |   1 +
 kernel/module.c                   |   8 +-
 scripts/gdb/linux/symbols.py      | 203 ++++++++++++++++++++----------
 15 files changed, 272 insertions(+), 84 deletions(-)
Signature · 8 lines
2.26.2



_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#1 lore raw

PATCH v2 1/9 scripts/gdb: rework lx-symbols gdb script

Fix several issues that are present in lx-symbols script:

* Track module unloads by placing another software breakpoint at 'free_module'
(force uninline this symbol just in case), and use remove-symbol-file gdb command to unload the symobls of the module that is unloading.

That gives the gdb a chance to mark all software breakpoints from this module as pending again.
Also remove the module from the 'known' module list once it is unloaded.

* Since we now track module unload, we don't need to reload all symbols anymore when 'known' module loaded again (that can't happen anymore).
This allows reloading a module in the debugged kernel to finish much faster, while lx-symbols tracks module loads and unloads.

* Disable/enable all gdb breakpoints on both module load and unload breakpoint hits, and not only in 'load_all_symbols' as was done before. (load_all_symbols is no longer called on breakpoint hit) That allows gdb to avoid getting confused about the state of the (now two) internal breakpoints we place.
Otherwise it will leave them in the kernel code segment, when continuing which triggers a guest kernel panic as soon as it skips over the 'int3' instruction and executes the garbage tail of the optcode on which the breakpoint was placed.

* Block SIGINT while symbols are reloading as this seems to crash gdb. (new in V2)

* Add a basic check that kernel is already loaded into the guest memory to avoid confusing errors.
(new in V2)

---
 kernel/module.c              |   8 +-
 scripts/gdb/linux/symbols.py | 203 +++++++++++++++++++++++------------
 2 files changed, 143 insertions(+), 68 deletions(-)
kernel/module.c +6 -2
index 30479355ab85..ea81fc06ea1f 100644
@@ -901,8 +901,12 @@ int module_refcount(struct module *mod)
 }
 EXPORT_SYMBOL(module_refcount);
 
-/* This exists whether we can unload or not */
-static void free_module(struct module *mod);
+/* This exists whether we can unload or not
+ * Keep it uninlined to provide a reliable breakpoint target,
+ * e.g. for the gdb helper command 'lx-symbols'.
+ */
+
+static noinline void free_module(struct module *mod);
 
 SYSCALL_DEFINE2(delete_module, const char __user *, name_user,
 		unsigned int, flags)
scripts/gdb/linux/symbols.py +137 -66

259 lines not on this page · raw message

Signature · 7 lines
2.26.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#2 lore raw

PATCH v2 2/9 KVM: introduce KVM_CAP_SET_GUEST_DEBUG2

In reply to Maxim Levitsky

This capability will allow the user to know which KVM_GUESTDBG_* bits are supported.

---
 Documentation/virt/kvm/api.rst | 3 +++
 include/uapi/linux/kvm.h       | 1 +
 2 files changed, 4 insertions(+)
Documentation/virt/kvm/api.rst +3 -0
index 38e327d4b479..9778b2434c03 100644
@@ -3357,6 +3357,9 @@ indicating the number of supported registers.
 For ppc, the KVM_CAP_PPC_GUEST_DEBUG_SSTEP capability indicates whether
 the single-step debug event (KVM_GUESTDBG_SINGLESTEP) is supported.
 
+Also when supported, KVM_CAP_SET_GUEST_DEBUG2 capability indicates the
+supported KVM_GUESTDBG_* bits in the control field.
+
 When debug events exit the main run loop with the reason
 KVM_EXIT_DEBUG with the kvm_debug_exit_arch part of the kvm_run
 structure containing architecture specific debug information.
include/uapi/linux/kvm.h +1 -0
index f6afee209620..727010788eff 100644
@@ -1078,6 +1078,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_DIRTY_LOG_RING 192
 #define KVM_CAP_X86_BUS_LOCK_EXIT 193
 #define KVM_CAP_PPC_DAWR1 194
+#define KVM_CAP_SET_GUEST_DEBUG2 195
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
Signature · 7 lines
2.26.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#3 lore raw

PATCH v2 3/9 KVM: x86: implement KVM_CAP_SET_GUEST_DEBUG2

In reply to Maxim Levitsky

Store the supported bits into KVM_GUESTDBG_VALID_MASK macro, similar to how arm does this.

---
 arch/x86/include/asm/kvm_host.h | 9 +++++++++
 arch/x86/kvm/x86.c              | 2 ++
 2 files changed, 11 insertions(+)
arch/x86/include/asm/kvm_host.h +9 -0
index a52f973bdff6..cc7c82a449d5 100644
@@ -221,6 +221,15 @@ enum x86_intercept_stage;
 #define DR7_FIXED_1	0x00000400
 #define DR7_VOLATILE	0xffff2bff
 
+#define KVM_GUESTDBG_VALID_MASK \
+	(KVM_GUESTDBG_ENABLE | \
+	KVM_GUESTDBG_SINGLESTEP | \
+	KVM_GUESTDBG_USE_HW_BP | \
+	KVM_GUESTDBG_USE_SW_BP | \
+	KVM_GUESTDBG_INJECT_BP | \
+	KVM_GUESTDBG_INJECT_DB)
+
+
 #define PFERR_PRESENT_BIT 0
 #define PFERR_WRITE_BIT 1
 #define PFERR_USER_BIT 2
arch/x86/kvm/x86.c +2 -0
index a9d95f90a048..956e8e0bd6af 100644
@@ -3798,6 +3798,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_ENFORCE_PV_FEATURE_CPUID:
 		r = 1;
 		break;
+	case KVM_CAP_SET_GUEST_DEBUG2:
+		return KVM_GUESTDBG_VALID_MASK;
 #ifdef CONFIG_KVM_XEN
 	case KVM_CAP_XEN_HVM:
 		r = KVM_XEN_HVM_CONFIG_HYPERCALL_MSR |
Signature · 7 lines
2.26.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#4 lore raw

PATCH v2 4/9 KVM: aarch64: implement KVM_CAP_SET_GUEST_DEBUG2

In reply to Maxim Levitsky

Move KVM_GUESTDBG_VALID_MASK to kvm_host.h and use it to return the value of this capability. Compile tested only.

---
 arch/arm64/include/asm/kvm_host.h | 4 ++++
 arch/arm64/kvm/arm.c              | 2 ++
 arch/arm64/kvm/guest.c            | 5 -----
 3 files changed, 6 insertions(+), 5 deletions(-)
arch/arm64/include/asm/kvm_host.h +4 -0
index 3d10e6527f7d..613421454ab6 100644
@@ -401,6 +401,10 @@ struct kvm_vcpu_arch {
 #define KVM_ARM64_PENDING_EXCEPTION	(1 << 8) /* Exception pending */
 #define KVM_ARM64_EXCEPT_MASK		(7 << 9) /* Target EL/MODE */
 
+#define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \
+				 KVM_GUESTDBG_USE_SW_BP | \
+				 KVM_GUESTDBG_USE_HW | \
+				 KVM_GUESTDBG_SINGLESTEP)
 /*
  * When KVM_ARM64_PENDING_EXCEPTION is set, KVM_ARM64_EXCEPT_MASK can
  * take the following values:
arch/arm64/kvm/arm.c +2 -0
index 7f06ba76698d..e575eff76e97 100644
@@ -208,6 +208,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_VCPU_ATTRIBUTES:
 		r = 1;
 		break;
+	case KVM_CAP_SET_GUEST_DEBUG2:
+		return KVM_GUESTDBG_VALID_MASK;
 	case KVM_CAP_ARM_SET_DEVICE_ADDR:
 		r = 1;
 		break;
arch/arm64/kvm/guest.c +0 -5
index 9bbd30e62799..6cb39ee74acd 100644
@@ -888,11 +888,6 @@ int kvm_arch_vcpu_ioctl_translate(struct kvm_vcpu *vcpu,
 	return -EINVAL;
 }
 
-#define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE |    \
-			    KVM_GUESTDBG_USE_SW_BP | \
-			    KVM_GUESTDBG_USE_HW | \
-			    KVM_GUESTDBG_SINGLESTEP)
-
 /**
  * kvm_arch_vcpu_ioctl_set_guest_debug - set up guest debugging
  * @kvm:	pointer to the KVM struct
Signature · 7 lines
2.26.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#5 lore raw

PATCH v2 5/9 KVM: s390x: implement KVM_CAP_SET_GUEST_DEBUG2

In reply to Maxim Levitsky

Define KVM_GUESTDBG_VALID_MASK and use it to implement this capabiity. Compile tested only.

---
 arch/s390/include/asm/kvm_host.h | 4 ++++
 arch/s390/kvm/kvm-s390.c         | 3 +++
 2 files changed, 7 insertions(+)
arch/s390/include/asm/kvm_host.h +4 -0
index 6bcfc5614bbc..a3902b57b825 100644
@@ -700,6 +700,10 @@ struct kvm_hw_bp_info_arch {
 #define guestdbg_exit_pending(vcpu) (guestdbg_enabled(vcpu) && \
 		(vcpu->guest_debug & KVM_GUESTDBG_EXIT_PENDING))
 
+#define KVM_GUESTDBG_VALID_MASK \
+		(KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_SINGLESTEP |\
+		KVM_GUESTDBG_USE_HW_BP | KVM_GUESTDBG_EXIT_PENDING)
+
 struct kvm_guestdbg_info_arch {
 	unsigned long cr0;
 	unsigned long cr9;
arch/s390/kvm/kvm-s390.c +3 -0
index 2f09e9d7dc95..2049fc8c222a 100644
@@ -544,6 +544,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_S390_DIAG318:
 		r = 1;
 		break;
+	case KVM_CAP_SET_GUEST_DEBUG2:
+		r = KVM_GUESTDBG_VALID_MASK;
+		break;
 	case KVM_CAP_S390_HPAGE_1M:
 		r = 0;
 		if (hpage && !kvm_is_ucontrol(kvm))
Signature · 7 lines
2.26.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#6 lore raw

PATCH v2 6/9 KVM: x86: implement KVM_GUESTDBG_BLOCKEVENTS

In reply to Maxim Levitsky

KVM_GUESTDBG_BLOCKEVENTS is a guest debug feature that will allow KVM to block all interrupts while running. It is mostly intended to be used together with single stepping, to make it more robust, and has the following benefits:

* Resuming from a breakpoint is much more reliable:
When resuming execution from a breakpoint, with interrupts enabled, more often than not, KVM would inject an interrupt and make the CPU jump immediately to the interrupt handler and eventually return to the breakpoint, only to trigger it again.

From the gdb's user point of view it looks like the CPU has never executed a single instruction and in some cases that can even prevent forward progress, for example, when the breakpoint is placed by an automated script (e.g lx-symbols), which does something in response to the breakpoint and then continues the guest automatically.
If the script execution takes enough time for another interrupt to arrive, the guest will be stuck on the same breakpoint forever.

* Normal single stepping is much more predictable, since it won't land the debugger into an interrupt handler.

* Chances of RFLAGS.TF being leaked to the guest are reduced:

KVM sets that flag behind the guest's back to single step it, but if the single step lands the vCPU into an interrupt/exception handler the RFLAGS.TF will be leaked to the guest in the form of being pushed to the stack.
This doesn't completely eliminate this problem as exceptions can still happen, but at least this eliminates the common case.

---
 Documentation/virt/kvm/api.rst  | 1 +
 arch/x86/include/asm/kvm_host.h | 3 ++-
 arch/x86/include/uapi/asm/kvm.h | 1 +
 arch/x86/kvm/x86.c              | 4 ++++
 4 files changed, 8 insertions(+), 1 deletion(-)
Documentation/virt/kvm/api.rst +1 -0
index 9778b2434c03..a4f2dc84741f 100644
@@ -3338,6 +3338,7 @@ flags which can include the following:
   - KVM_GUESTDBG_INJECT_DB:     inject DB type exception [x86]
   - KVM_GUESTDBG_INJECT_BP:     inject BP type exception [x86]
   - KVM_GUESTDBG_EXIT_PENDING:  trigger an immediate guest exit [s390]
+  - KVM_GUESTDBG_BLOCKIRQ:      avoid injecting interrupts/NMI/SMI [x86]
 
 For example KVM_GUESTDBG_USE_SW_BP indicates that software breakpoints
 are enabled in memory so we need to ensure breakpoint exceptions are
arch/x86/include/asm/kvm_host.h +2 -1
index cc7c82a449d5..8c529ae9dbbe 100644
@@ -227,7 +227,8 @@ enum x86_intercept_stage;
 	KVM_GUESTDBG_USE_HW_BP | \
 	KVM_GUESTDBG_USE_SW_BP | \
 	KVM_GUESTDBG_INJECT_BP | \
-	KVM_GUESTDBG_INJECT_DB)
+	KVM_GUESTDBG_INJECT_DB | \
+	KVM_GUESTDBG_BLOCKIRQ)
 
 
 #define PFERR_PRESENT_BIT 0
arch/x86/include/uapi/asm/kvm.h +1 -0
index 5a3022c8af82..b0f9945067f7 100644
@@ -282,6 +282,7 @@ struct kvm_debug_exit_arch {
 #define KVM_GUESTDBG_USE_HW_BP		0x00020000
 #define KVM_GUESTDBG_INJECT_DB		0x00040000
 #define KVM_GUESTDBG_INJECT_BP		0x00080000
+#define KVM_GUESTDBG_BLOCKIRQ		0x00100000
 
 /* for KVM_SET_GUEST_DEBUG */
 struct kvm_guest_debug_arch {
arch/x86/kvm/x86.c +4 -0
index 956e8e0bd6af..3627ce8fe5bb 100644
@@ -8460,6 +8460,10 @@ static void inject_pending_event(struct kvm_vcpu *vcpu, bool *req_immediate_exit
 		can_inject = false;
 	}
 
+	/* Don't inject interrupts if the user asked to avoid doing so */
+	if (vcpu->guest_debug & KVM_GUESTDBG_BLOCKIRQ)
+		return;
+
 	/*
 	 * Finally, inject interrupt events.  If an event cannot be injected
 	 * due to architectural conditions (e.g. IF=0) a window-open exit
Signature · 7 lines
2.26.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
maintainer
#7 lore raw

Re: [PATCH v2 6/9] KVM: x86: implement KVM_GUESTDBG_BLOCKEVENTS

Quoted 33 lines · Maxim Levitsky

KVM_GUESTDBG_BLOCKEVENTS is a guest debug feature that will allow KVM to block all interrupts while running. It is mostly intended to be used together with single stepping, to make it more robust, and has the following benefits:

* Resuming from a breakpoint is much more reliable:
When resuming execution from a breakpoint, with interrupts enabled, more often than not, KVM would inject an interrupt and make the CPU jump immediately to the interrupt handler and eventually return to the breakpoint, only to trigger it again.

From the gdb's user point of view it looks like the CPU has never executed a single instruction and in some cases that can even prevent forward progress, for example, when the breakpoint is placed by an automated script (e.g lx-symbols), which does something in response to the breakpoint and then continues the guest automatically.
If the script execution takes enough time for another interrupt to arrive, the guest will be stuck on the same breakpoint forever.

* Normal single stepping is much more predictable, since it won't land the debugger into an interrupt handler.

* Chances of RFLAGS.TF being leaked to the guest are reduced:

KVM sets that flag behind the guest's back to single step it, but if the single step lands the vCPU into an interrupt/exception handler the RFLAGS.TF will be leaked to the guest in the form of being pushed to the stack.
This doesn't completely eliminate this problem as exceptions can still happen, but at least this eliminates the common case.

Quoted 1 line

The patch uses BLOCKIRQ instead of BLOCKEVENTS.

Paolo

Quoted 61 lines
---
  Documentation/virt/kvm/api.rst  | 1 +
  arch/x86/include/asm/kvm_host.h | 3 ++-
  arch/x86/include/uapi/asm/kvm.h | 1 +
  arch/x86/kvm/x86.c              | 4 ++++
  4 files changed, 8 insertions(+), 1 deletion(-)
Documentation/virt/kvm/api.rst +1 -0
index 9778b2434c03..a4f2dc84741f 100644
@@ -3338,6 +3338,7 @@ flags which can include the following:
    - KVM_GUESTDBG_INJECT_DB:     inject DB type exception [x86]
    - KVM_GUESTDBG_INJECT_BP:     inject BP type exception [x86]
    - KVM_GUESTDBG_EXIT_PENDING:  trigger an immediate guest exit [s390]
+  - KVM_GUESTDBG_BLOCKIRQ:      avoid injecting interrupts/NMI/SMI [x86]
 
  For example KVM_GUESTDBG_USE_SW_BP indicates that software breakpoints
  are enabled in memory so we need to ensure breakpoint exceptions are
arch/x86/include/asm/kvm_host.h +2 -1
index cc7c82a449d5..8c529ae9dbbe 100644
@@ -227,7 +227,8 @@ enum x86_intercept_stage;
  	KVM_GUESTDBG_USE_HW_BP | \
  	KVM_GUESTDBG_USE_SW_BP | \
  	KVM_GUESTDBG_INJECT_BP | \
-	KVM_GUESTDBG_INJECT_DB)
+	KVM_GUESTDBG_INJECT_DB | \
+	KVM_GUESTDBG_BLOCKIRQ)
 
 
  #define PFERR_PRESENT_BIT 0
arch/x86/include/uapi/asm/kvm.h +1 -0
index 5a3022c8af82..b0f9945067f7 100644
@@ -282,6 +282,7 @@ struct kvm_debug_exit_arch {
  #define KVM_GUESTDBG_USE_HW_BP		0x00020000
  #define KVM_GUESTDBG_INJECT_DB		0x00040000
  #define KVM_GUESTDBG_INJECT_BP		0x00080000
+#define KVM_GUESTDBG_BLOCKIRQ		0x00100000
 
  /* for KVM_SET_GUEST_DEBUG */
  struct kvm_guest_debug_arch {
arch/x86/kvm/x86.c +4 -0
index 956e8e0bd6af..3627ce8fe5bb 100644
@@ -8460,6 +8460,10 @@ static void inject_pending_event(struct kvm_vcpu *vcpu, bool *req_immediate_exit
  		can_inject = false;
  	}
 
+	/* Don't inject interrupts if the user asked to avoid doing so */
+	if (vcpu->guest_debug & KVM_GUESTDBG_BLOCKIRQ)
+		return;
+
  	/*
  	 * Finally, inject interrupt events.  If an event cannot be injected
  	 * due to architectural conditions (e.g. IF=0) a window-open exit
#8 lore raw

PATCH v2 7/9 KVM: SVM: split svm_handle_invalid_exit

In reply to Maxim Levitsky

Split the check for having a vmexit handler to svm_check_exit_valid, and make svm_handle_invalid_exit only handle a vmexit that is already not valid.

---
 arch/x86/kvm/svm/svm.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/svm.c +9 -8
index 271196400495..2aa951bc470c 100644
@@ -3220,12 +3220,14 @@ static void dump_vmcb(struct kvm_vcpu *vcpu)
 	       "excp_to:", save->last_excp_to);
 }
 
-static int svm_handle_invalid_exit(struct kvm_vcpu *vcpu, u64 exit_code)
+static bool svm_check_exit_valid(struct kvm_vcpu *vcpu, u64 exit_code)
 {
-	if (exit_code < ARRAY_SIZE(svm_exit_handlers) &&
-	    svm_exit_handlers[exit_code])
-		return 0;
+	return (exit_code < ARRAY_SIZE(svm_exit_handlers) &&
+		svm_exit_handlers[exit_code]);
+}
 
+static int svm_handle_invalid_exit(struct kvm_vcpu *vcpu, u64 exit_code)
+{
 	vcpu_unimpl(vcpu, "svm: unexpected exit reason 0x%llx\n", exit_code);
 	dump_vmcb(vcpu);
 	vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
@@ -3233,14 +3235,13 @@ static int svm_handle_invalid_exit(struct kvm_vcpu *vcpu, u64 exit_code)
 	vcpu->run->internal.ndata = 2;
 	vcpu->run->internal.data[0] = exit_code;
 	vcpu->run->internal.data[1] = vcpu->arch.last_vmentry_cpu;
-
-	return -EINVAL;
+	return 0;
 }
 
 int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)
 {
-	if (svm_handle_invalid_exit(vcpu, exit_code))
-		return 0;
+	if (!svm_check_exit_valid(vcpu, exit_code))
+		return svm_handle_invalid_exit(vcpu, exit_code);
 
 #ifdef CONFIG_RETPOLINE
 	if (exit_code == SVM_EXIT_MSR)
Signature · 7 lines
2.26.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#9 lore raw

PATCH v2 8/9 KVM: x86: add force_intercept_exceptions_mask

In reply to Maxim Levitsky

This parameter will be used by VMX and SVM code to force interception of a set of exceptions, given by a bitmask for guest debug and/or kvm debug.

This option is not intended for production.

This is based on an idea first shown here:
https://patchwork.kernel.org/project/kvm/patch/[email protected]/

---
 arch/x86/kvm/x86.c | 3 +++
 arch/x86/kvm/x86.h | 2 ++
 2 files changed, 5 insertions(+)
arch/x86/kvm/x86.c +3 -0
index 3627ce8fe5bb..1a51031d64d8 100644
@@ -176,6 +176,9 @@ module_param(force_emulation_prefix, bool, S_IRUGO);
 int __read_mostly pi_inject_timer = -1;
 module_param(pi_inject_timer, bint, S_IRUGO | S_IWUSR);
 
+uint force_intercept_exceptions_mask;
+module_param(force_intercept_exceptions_mask, uint, S_IRUGO | S_IWUSR);
+EXPORT_SYMBOL_GPL(force_intercept_exceptions_mask);
 /*
  * Restoring the host value for MSRs that are only consumed when running in
  * usermode, e.g. SYSCALL MSRs and TSC_AUX, can be deferred until the CPU
arch/x86/kvm/x86.h +2 -0
index daccf20fbcd5..644480711ff7 100644
@@ -311,6 +311,8 @@ extern struct static_key kvm_no_apic_vcpu;
 
 extern bool report_ignored_msrs;
 
+extern uint force_intercept_exceptions_mask;
+
 static inline u64 nsec_to_cycles(struct kvm_vcpu *vcpu, u64 nsec)
 {
 	return pvclock_scale_delta(nsec, vcpu->arch.virtual_tsc_mult,
Signature · 7 lines
2.26.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
maintainer
#10 lore raw

Re: [PATCH v2 8/9] KVM: x86: add force_intercept_exceptions_mask

Quoted 21 lines · Maxim Levitsky

This parameter will be used by VMX and SVM code to force interception of a set of exceptions, given by a bitmask for guest debug and/or kvm debug.

This option is not intended for production.

This is based on an idea first shown here:
https://patchwork.kernel.org/project/kvm/patch/[email protected]/

---
 arch/x86/kvm/x86.c | 3 +++
 arch/x86/kvm/x86.h | 2 ++
 2 files changed, 5 insertions(+)
arch/x86/kvm/x86.c +0 -0
index 3627ce8fe5bb..1a51031d64d8 100644

arch/x86/kvm/x86.c module_param(force_emulation_prefix, bool, S_IRUGO); · line 181

Quoted 6 lines
arch/x86/kvm/x86.c +3 -0
@@ -176,6 +176,9 @@ module_param(force_emulation_prefix, bool, S_IRUGO);
 int __read_mostly pi_inject_timer = -1;
 module_param(pi_inject_timer, bint, S_IRUGO | S_IWUSR);
 
+uint force_intercept_exceptions_mask;
+module_param(force_intercept_exceptions_mask, uint, S_IRUGO | S_IWUSR);
+EXPORT_SYMBOL_GPL(force_intercept_exceptions_mask);

That's nice.

I could use some text explaning the usage though, i.e. that thing takes a bitfield of exception vectors, so that I don't have to look at the code each time. :-)

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#11 lore raw

PATCH v2 9/9 KVM: SVM: implement force_intercept_exceptions_mask

In reply to Maxim Levitsky

Currently #TS interception is only done once. Also exception interception is not enabled for SEV guests.

---
 arch/x86/include/asm/kvm_host.h |  2 +
 arch/x86/kvm/svm/svm.c          | 70 +++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.h          |  6 ++-
 arch/x86/kvm/x86.c              |  5 ++-
 4 files changed, 80 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
index 8c529ae9dbbe..d15ae64a2c4e 100644
@@ -1574,6 +1574,8 @@ int kvm_emulate_rdpmc(struct kvm_vcpu *vcpu);
 void kvm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr);
 void kvm_queue_exception_e(struct kvm_vcpu *vcpu, unsigned nr, u32 error_code);
 void kvm_queue_exception_p(struct kvm_vcpu *vcpu, unsigned nr, unsigned long payload);
+void kvm_queue_exception_e_p(struct kvm_vcpu *vcpu, unsigned nr,
+			     u32 error_code, unsigned long payload);
 void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned nr);
 void kvm_requeue_exception_e(struct kvm_vcpu *vcpu, unsigned nr, u32 error_code);
 void kvm_inject_page_fault(struct kvm_vcpu *vcpu, struct x86_exception *fault);
arch/x86/kvm/svm/svm.c +70 -0

106 lines not on this page · raw message

arch/x86/kvm/svm/svm.h +5 -1
index 8e276c4fb33d..79d0aea87753 100644
@@ -183,6 +183,7 @@ struct vcpu_svm {
 	bool ghcb_sa_free;
 
 	bool guest_state_loaded;
+	u32 force_intercept_exceptions_mask;
 };
 
 struct svm_cpu_data {
@@ -333,8 +334,11 @@ static inline void clr_exception_intercept(struct vcpu_svm *svm, u32 bit)
 	struct vmcb *vmcb = svm->vmcb01.ptr;
 
 	WARN_ON_ONCE(bit >= 32);
-	vmcb_clr_intercept(&vmcb->control, INTERCEPT_EXCEPTION_OFFSET + bit);
 
+	if ((1 << bit) & svm->force_intercept_exceptions_mask)
+		return;
+
+	vmcb_clr_intercept(&vmcb->control, INTERCEPT_EXCEPTION_OFFSET + bit);
 	recalc_intercepts(svm);
 }
 
arch/x86/kvm/x86.c +3 -2
index 1a51031d64d8..ae57816fe6d9 100644
@@ -630,12 +630,13 @@ void kvm_queue_exception_p(struct kvm_vcpu *vcpu, unsigned nr,
 }
 EXPORT_SYMBOL_GPL(kvm_queue_exception_p);
 
-static void kvm_queue_exception_e_p(struct kvm_vcpu *vcpu, unsigned nr,
-				    u32 error_code, unsigned long payload)
+void kvm_queue_exception_e_p(struct kvm_vcpu *vcpu, unsigned nr,
+			     u32 error_code, unsigned long payload)
 {
 	kvm_multiple_exception(vcpu, nr, true, error_code,
 			       true, payload, false);
 }
+EXPORT_SYMBOL_GPL(kvm_queue_exception_e_p);
 
 int kvm_complete_insn_gp(struct kvm_vcpu *vcpu, int err)
 {
Signature · 7 lines
2.26.2


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
maintainer
#12 lore raw
In reply to Maxim Levitsky
Quoted 8 lines · Maxim Levitsky

Hi!

I would like to publish two debug features which were needed for other stuff I work on.

One is the reworked lx-symbols script which now actually works on at least gdb 9.1 (gdb 9.2 was reported to fail to load the debug symbols from the kernel for some reason, not related to this patch) and upstream qemu.

Queued patches 2-5 for now. 6 is okay but it needs a selftest. (e.g. using KVM_VCPU_SET_EVENTS) and the correct name for the constant.

Paolo

Quoted 60 lines

The other feature is the ability to trap all guest exceptions (on SVM for now) and see them in kvmtrace prior to potential merge to double/triple fault.

This can be very useful and I already had to manually patch KVM a few times for this.
I will, once time permits, implement this feature on Intel as well.

V2:

* Some more refactoring and workarounds for lx-symbols script

* added KVM_GUESTDBG_BLOCKEVENTS flag to enable 'block interrupts on single step' together with KVM_CAP_SET_GUEST_DEBUG2 capability to indicate which guest debug flags are supported.

This is a replacement for unconditional block of interrupts on single step that was done in previous version of this patch set. Patches to qemu to use that feature will be sent soon.

* Reworked the the 'intercept all exceptions for debug' feature according to the review feedback:

- renamed the parameter that enables the feature and moved it to common kvm module.
(only SVM part is currently implemented though)

- disable the feature for SEV guests as was suggested during the review
- made the vmexit table const again, as was suggested in the review as well.

Best regards,
Maxim Levitsky

Maxim Levitsky (9):
scripts/gdb: rework lx-symbols gdb script
KVM: introduce KVM_CAP_SET_GUEST_DEBUG2
KVM: x86: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: aarch64: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: s390x: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: x86: implement KVM_GUESTDBG_BLOCKEVENTS
KVM: SVM: split svm_handle_invalid_exit
KVM: x86: add force_intercept_exceptions_mask
KVM: SVM: implement force_intercept_exceptions_mask

  Documentation/virt/kvm/api.rst    |   4 +
  arch/arm64/include/asm/kvm_host.h |   4 +
  arch/arm64/kvm/arm.c              |   2 +
  arch/arm64/kvm/guest.c            |   5 -
  arch/s390/include/asm/kvm_host.h  |   4 +
  arch/s390/kvm/kvm-s390.c          |   3 +
  arch/x86/include/asm/kvm_host.h   |  12 ++
  arch/x86/include/uapi/asm/kvm.h   |   1 +
  arch/x86/kvm/svm/svm.c            |  87 +++++++++++--
  arch/x86/kvm/svm/svm.h            |   6 +-
  arch/x86/kvm/x86.c                |  14 ++-
  arch/x86/kvm/x86.h                |   2 +
  include/uapi/linux/kvm.h          |   1 +
  kernel/module.c                   |   8 +-
  scripts/gdb/linux/symbols.py      | 203 ++++++++++++++++++++----------
  15 files changed, 272 insertions(+), 84 deletions(-)
#13 lore raw
Quoted 6 lines · Paolo Bonzini
Quoted 5 lines · Maxim Levitsky

Hi!

I would like to publish two debug features which were needed for other stuff I work on.

Quoted 6 lines
Quoted 3 lines

One is the reworked lx-symbols script which now actually works on at least gdb 9.1 (gdb 9.2 was reported to fail to load the debug symbols from the kernel for some reason, not related to this patch) and upstream qemu.

Queued patches 2-5 for now. 6 is okay but it needs a selftest. (e.g. using KVM_VCPU_SET_EVENTS) and the correct name for the constant.

Thanks!
I will do this very soon.

Best regards,
Maxim Levitsky

Quoted 63 lines

Paolo

Quoted 60 lines

The other feature is the ability to trap all guest exceptions (on SVM for now) and see them in kvmtrace prior to potential merge to double/triple fault.

This can be very useful and I already had to manually patch KVM a few times for this.
I will, once time permits, implement this feature on Intel as well.

V2:

* Some more refactoring and workarounds for lx-symbols script

* added KVM_GUESTDBG_BLOCKEVENTS flag to enable 'block interrupts on single step' together with KVM_CAP_SET_GUEST_DEBUG2 capability to indicate which guest debug flags are supported.

This is a replacement for unconditional block of interrupts on single step that was done in previous version of this patch set. Patches to qemu to use that feature will be sent soon.

* Reworked the the 'intercept all exceptions for debug' feature according to the review feedback:

- renamed the parameter that enables the feature and moved it to common kvm module.
(only SVM part is currently implemented though)

- disable the feature for SEV guests as was suggested during the review
- made the vmexit table const again, as was suggested in the review as well.

Best regards,
Maxim Levitsky

Maxim Levitsky (9):
scripts/gdb: rework lx-symbols gdb script
KVM: introduce KVM_CAP_SET_GUEST_DEBUG2
KVM: x86: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: aarch64: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: s390x: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: x86: implement KVM_GUESTDBG_BLOCKEVENTS
KVM: SVM: split svm_handle_invalid_exit
KVM: x86: add force_intercept_exceptions_mask
KVM: SVM: implement force_intercept_exceptions_mask

  Documentation/virt/kvm/api.rst    |   4 +
  arch/arm64/include/asm/kvm_host.h |   4 +
  arch/arm64/kvm/arm.c              |   2 +
  arch/arm64/kvm/guest.c            |   5 -
  arch/s390/include/asm/kvm_host.h  |   4 +
  arch/s390/kvm/kvm-s390.c          |   3 +
  arch/x86/include/asm/kvm_host.h   |  12 ++
  arch/x86/include/uapi/asm/kvm.h   |   1 +
  arch/x86/kvm/svm/svm.c            |  87 +++++++++++--
  arch/x86/kvm/svm/svm.h            |   6 +-
  arch/x86/kvm/x86.c                |  14 ++-
  arch/x86/kvm/x86.h                |   2 +
  include/uapi/linux/kvm.h          |   1 +
  kernel/module.c                   |   8 +-
  scripts/gdb/linux/symbols.py      | 203 ++++++++++++++++++++----------
  15 files changed, 272 insertions(+), 84 deletions(-)
#14 lore raw
In reply to Paolo Bonzini
Quoted 6 lines · Paolo Bonzini
Quoted 5 lines · Maxim Levitsky

Hi!

I would like to publish two debug features which were needed for other stuff I work on.

Quoted 6 lines
Quoted 3 lines

One is the reworked lx-symbols script which now actually works on at least gdb 9.1 (gdb 9.2 was reported to fail to load the debug symbols from the kernel for some reason, not related to this patch) and upstream qemu.

Queued patches 2-5 for now. 6 is okay but it needs a selftest. (e.g. using KVM_VCPU_SET_EVENTS) and the correct name for the constant.

Do you mean to add a kvm-unit-test or to add a test to kernel's kvm unit tests for this?

Best regards,
Maxim Levitsky

Quoted 63 lines

Paolo

Quoted 60 lines

The other feature is the ability to trap all guest exceptions (on SVM for now) and see them in kvmtrace prior to potential merge to double/triple fault.

This can be very useful and I already had to manually patch KVM a few times for this.
I will, once time permits, implement this feature on Intel as well.

V2:

* Some more refactoring and workarounds for lx-symbols script

* added KVM_GUESTDBG_BLOCKEVENTS flag to enable 'block interrupts on single step' together with KVM_CAP_SET_GUEST_DEBUG2 capability to indicate which guest debug flags are supported.

This is a replacement for unconditional block of interrupts on single step that was done in previous version of this patch set. Patches to qemu to use that feature will be sent soon.

* Reworked the the 'intercept all exceptions for debug' feature according to the review feedback:

- renamed the parameter that enables the feature and moved it to common kvm module.
(only SVM part is currently implemented though)

- disable the feature for SEV guests as was suggested during the review
- made the vmexit table const again, as was suggested in the review as well.

Best regards,
Maxim Levitsky

Maxim Levitsky (9):
scripts/gdb: rework lx-symbols gdb script
KVM: introduce KVM_CAP_SET_GUEST_DEBUG2
KVM: x86: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: aarch64: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: s390x: implement KVM_CAP_SET_GUEST_DEBUG2
KVM: x86: implement KVM_GUESTDBG_BLOCKEVENTS
KVM: SVM: split svm_handle_invalid_exit
KVM: x86: add force_intercept_exceptions_mask
KVM: SVM: implement force_intercept_exceptions_mask

  Documentation/virt/kvm/api.rst    |   4 +
  arch/arm64/include/asm/kvm_host.h |   4 +
  arch/arm64/kvm/arm.c              |   2 +
  arch/arm64/kvm/guest.c            |   5 -
  arch/s390/include/asm/kvm_host.h  |   4 +
  arch/s390/kvm/kvm-s390.c          |   3 +
  arch/x86/include/asm/kvm_host.h   |  12 ++
  arch/x86/include/uapi/asm/kvm.h   |   1 +
  arch/x86/kvm/svm/svm.c            |  87 +++++++++++--
  arch/x86/kvm/svm/svm.h            |   6 +-
  arch/x86/kvm/x86.c                |  14 ++-
  arch/x86/kvm/x86.h                |   2 +
  include/uapi/linux/kvm.h          |   1 +
  kernel/module.c                   |   8 +-
  scripts/gdb/linux/symbols.py      | 203 ++++++++++++++++++++----------
  15 files changed, 272 insertions(+), 84 deletions(-)
maintainer
#15 lore raw
Quoted 7 lines · Maxim Levitsky
Quoted 6 lines · Paolo Bonzini
Quoted 5 lines · Maxim Levitsky

Hi!

I would like to publish two debug features which were needed for other stuff I work on.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

One is the reworked lx-symbols script which now actually works on at least gdb 9.1 (gdb 9.2 was reported to fail to load the debug symbols from the kernel for some reason, not related to this patch) and upstream qemu.

Queued patches 2-5 for now. 6 is okay but it needs a selftest. (e.g. using KVM_VCPU_SET_EVENTS) and the correct name for the constant.

Do you mean to add a kvm-unit-test or to add a test to kernel's kvm unit tests for this?

selftests is used for those in the kernel. "Unit test" can mean both, but usually it means kvm-unit-tests because they're easier to write.

Paolo