KVM: x86: Hyper-V VSM (Virtual Trust Levels) on VM planes
Subsystems (8)
cover KVM: x86: Hyper-V VSM (Virtual Trust Levels) on VM planes
From: David Woodhouse <[email protected]>
Windows uses Virtualization Based Security for Credential Guard and friends, and enables it by default in current releases. A guest with VBS enabled runs its secure kernel in VTL1, a higher privilege level which the hypervisor has to keep isolated from the normal kernel in VTL0. KVM has no concept of that today, so Windows guests either lose VBS or emulate it with nested virtualization, which is slower and loses features.
There have been completely in-kernel implementations of VBS, which did unnatural things all within a single kvm_vcpu (four APICs, etc.)
This is an attempt to build it more cleanly, more in userspace, based on KVM planes (https://lwn.net/Articles/1087590/).
The behavioural reference was the in-kernel VSM implementation in the Nitro Hypervisor, complete with its latest compatibility fixes since it was last posted — largely turned into a bunch of kvm unit tests. The result boots Windows Server 2016 through 2025 with Credential Guard active, on both Intel and AMD hosts, with VTL1 protecting its memory from VTL0. With hibernate/resume tested, since that part has always been extra fun.
For now I've structured the userspace part as a separable library which can be pulled into QEMU or other VMMs. Partly because QEMU doesn't want it when AI did the typing (but *might* tolerate it as an external library), and partly because I think it might make sense to have a more permissively licensed standard implementation for how KVM-based VMMs implement VSM, and have some input from the Microsoft side and some improved expectation that it'll *keep* working with Windows and not keep breaking for every new release. But that's a future conversation. For now I'm just preempting the question, "why does it look like that?".
• https://git.infradead.org/?p=users/dwmw2/linux.git;a=shortlog;h=refs/heads/vsm
• https://git.infradead.org/?p=users/dwmw2/qemu.git;a=shortlog;h=refs/heads/vsm
• https://git.infradead.org/?p=users/dwmw2/kvm-unit-tests.git;a=shortlog;h=refs/heads/vsm
Pulled into my base tree on v7.1-rc7 (not reposted here):
• KVM planes itself (Jörg's latest from June)
https://lore.kernel.org/kvm/[email protected]/
https://github.com/joergroedel/linux/tree/kvm-planes-v7.1
• Paolo's memory protection attributes v3
https://lore.kernel.org/kvm/[email protected]/
• Melody Wang's SEV-SNP #HV doorbell / Restricted Injection series
https://lore.kernel.org/all/[email protected]/
• A few fixes to the above ride just above them in the git tree, each citing the report it addresses; not part of this posting
Generic fixes and features in KVM before we get to VSM itself:
• Userspace opt-in (KVM_CAP_PLANES) and plane run-state ioctls; MP state, guest-debug, KVM_TRANSLATE, SETUP_MCE and a filtered KVM_ENABLE_CAP on plane vCPU fds; unwind of a failed KVM_CREATE_PLANE
• Plane mechanics: per-plane request fan-out, kicking the twin that is actually running, XCR0 and TSC matching shared across twins, runtime CPUID per plane, APICv inhibited while planes are enabled (for now).
• Per-plane memory attributes and overlays, and instruction state on the memory-fault exit, for the VMM to build intercept messages from
• Fixes found along the way: KVM_TRANSLATE reporting a real write-access walk, SVM keeping TPR[3:0] across exits without a guest CR8 write
Finally we get to the actual Hyper-V VSM bits:
• Hypercall forwarding to userspace (KVM_CAP_HYPERV_FORWARD_HCALL), with completion or injected #UD, for the VSM hypercalls KVM doesn't handle itself
• HvCallVtlCall/HvCallVtlReturn switching between planes in the kernel; forced VTL entry gated on interrupt deliverability and re-armed when a plane suspends with IRR set
• Per-plane Hyper-V state: GUEST_OS_ID/HYPERCALL MSRs, the reference TSC page, SynIC and event routing, TLB flushes
• VTL register intercepts, including the descriptor-table load traps Credential Guard requires
• The suspended-VP state (from Nikolas Wipper's series), refusal of SendIpi to a higher VTL, XMM fast hypercall output
• Selftests for both halves; a hyperv_vsm suite for kvm-unit-tests covering VTL call/return, protections, intercepts and eventing follows separately
The four WIP-marked patches need finishing and splitting; they are included because the series does not function without them. This is very much an RFC.
Held back in the git tree for a follow-up posting:
• KVM_TRANSLATE2 and the page-walker flags/status work (also Nikolas's), and the HvCallTranslateVirtualAddress handling built on it
• TLB flush inhibit (TlbLocked)
Not done at all yet: VINA, and migration of mid-VTL-switch state (the forced-entry and captured-register state has no GET/SET exposure). The interaction with SEV-SNP VMPLs, the other consumer of planes, is untested from this side.
Mushahid also has a version which boots Windows 11 on arm64, which we'll reconcile when some of the dust settles.
I think this will take a while to reconcile and merge, even after the base dependencies (planes, memattrs, etc.) are in. I'd like to start settling the basic functionality so we can start preparing to merge it in little pieces, if we can reach a general consensus.
David Woodhouse (44):
KVM: x86/mmu: Move ACC_* masks from spte.h to mmu.h
KVM: x86: Let userspace opt in to multiple planes with KVM_CAP_PLANES
KVM: selftests: Add x86 planes lifecycle test
KVM: x86: hyper-v: Add KVM_CAP_HYPERV_FORWARD_HCALL
KVM: selftests: Add Hyper-V hypercall forwarding test
KVM: Add KVM_GET/SET_PLANE_RUN_STATE vCPU ioctls
KVM: selftests: Cover plane run-state ioctls in planes_test
KVM: x86: hyper-v: Let userspace fault a forwarded hypercall
KVM: x86: Allow MP state ioctls on plane vCPU fds
KVM: x86: Filter KVM_ENABLE_CAP on plane vCPU fds
KVM: x86: initialise Hyper-V state on every plane vCPU
KVM: implement per-plane memory attributes
KVM: x86: hyper-v: bank GUEST_OS_ID and HYPERCALL MSRs per plane
KVM: x86: hyper-v: bank the reference TSC page per plane
KVM: x86: allow KVM_SET_GUEST_DEBUG on plane vCPU fds
KVM: x86: hyper-v: flush the calling plane's TLB on flush hypercalls
KVM: x86/mmu: free obsolete roots of all planes
KVM: x86: hyperv: don't arm PV EOI for a higher plane's VP assist page
KVM: x86: allow KVM_TRANSLATE on plane vCPU fds
KVM: x86: hyper-v: handle VTL call/return in the kernel
KVM: x86: load each plane twin while updating it after KVM_SET_CPUID
WIP: userspace VTL interrupt entry and register intercepts
WIP: KVM: x86: deliver events to suspended higher planes
WIP: KVM: per-plane memory overlays, richer protection-fault exits
WIP: KVM: x86: VSM event-delivery parity fixes from Windows boot
triage
KVM: planes: fan out per-plane requests to twin vCPUs
KVM: x86: inhibit APICv when VM planes are enabled
KVM: x86: hyperv: divert LLDT and LTR into register intercepts
KVM: planes: Unwind published state on KVM_CREATE_PLANE failure
KVM: planes: Kick the running twin's pCPU, not the request target's
KVM: planes: Keep current_vcpu in sync in vcpu_load()
KVM: x86: Report a real write-access walk from KVM_TRANSLATE
KVM: planes: Share XCR0 across plane twins
KVM: x86: hyperv: Refuse a synthetic IPI to a higher VTL
KVM: x86: Gate forced VTL entry on interrupt deliverability
KVM: selftests: Test a plane suspending with an undelivered interrupt
KVM: x86: hyperv: Advertise XMM fast hypercall output
KVM: SVM: Carry instruction state on reflected plane-protection faults
KVM: SVM: Trap descriptor-table loads for VTL register intercepts
KVM: SVM: Keep TPR[3:0] across exits without a guest CR8 write
KVM: selftests: Add Hyper-V reference TSC page memory attribute test
KVM: SVM: Hedge the v_tpr_entry comment
KVM: Check every plane's protections before caching a gfn->hva
KVM: planes: Annotate the lockless twin-publication protocol
Mushahid Hussain (9):
KVM: x86: Leave TSC matching to the plane 0 vCPUs
KVM: x86: hyperv: Track the reference TSC page per plane
KVM: x86: hyperv: Update the reference TSC page past protections
KVM: x86: hyperv: Publish an added reference TSC page at once
KVM: x86: hyperv: Harden the per-plane reference TSC page updates
KVM: x86: Derive runtime CPUID bits from the executing plane
KVM: selftests: Test hugepage sizing over mixed protection attributes
KVM: x86: hyperv: Do not skip a moved vCPU when completing a hypercall
KVM: x86: Allow KVM_X86_SETUP_MCE on plane vCPUs
Nikolas Wipper (1):
KVM: x86: Implement Hyper-V's vCPU suspended state
Paolo Bonzini (1):
KVM: selftests: Add VM plane infrastructure and tests
Documentation/virt/kvm/api.rst | 241 ++++- arch/x86/hyperv/hv_apic.c | 3 +- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 140 ++- arch/x86/kvm/cpuid.c | 139 ++- arch/x86/kvm/cpuid.h | 6 +- arch/x86/kvm/emulate.c | 21 + arch/x86/kvm/hyperv.c | 995 ++++++++++++++++-- arch/x86/kvm/hyperv.h | 37 +- arch/x86/kvm/kvm_emulate.h | 10 + arch/x86/kvm/lapic.c | 85 +- arch/x86/kvm/mmu.h | 14 + arch/x86/kvm/mmu/mmu.c | 131 ++- arch/x86/kvm/mmu/paging_tmpl.h | 7 +- arch/x86/kvm/mmu/spte.h | 14 - arch/x86/kvm/mmu/tdp_mmu.c | 23 +- arch/x86/kvm/svm/nested.c | 16 + arch/x86/kvm/svm/svm.c | 143 ++- arch/x86/kvm/svm/svm.h | 16 +- arch/x86/kvm/vmx/main.c | 13 +- arch/x86/kvm/vmx/nested.c | 1 + arch/x86/kvm/vmx/vmx.c | 118 ++- arch/x86/kvm/vmx/x86_ops.h | 1 + arch/x86/kvm/x86.c | 668 +++++++++++- include/hyperv/hvgdk_mini.h | 5 +- include/hyperv/hvhdk_mini.h | 3 +- include/linux/kvm_host.h | 149 ++- include/uapi/linux/kvm.h | 63 +- tools/include/uapi/linux/kvm.h | 54 +- tools/testing/selftests/kvm/Makefile.kvm | 7 + .../testing/selftests/kvm/include/kvm_util.h | 74 ++ .../selftests/kvm/include/x86/processor.h | 1 + tools/testing/selftests/kvm/lib/kvm_util.c | 85 +- .../testing/selftests/kvm/lib/x86/processor.c | 15 + .../testing/selftests/kvm/memory_attributes.c | 6 +- tools/testing/selftests/kvm/plane_test.c | 137 +++ .../selftests/kvm/x86/hyperv_forward_hcall.c | 144 +++ .../selftests/kvm/x86/hyperv_tsc_page_attrs.c | 129 +++ .../selftests/kvm/x86/memattr_hugepage_test.c | 132 +++ .../selftests/kvm/x86/plane_x86_test.c | 325 ++++++ tools/testing/selftests/kvm/x86/planes_test.c | 142 +++ .../selftests/kvm/x86/planes_vtl_entry_test.c | 188 ++++ virt/kvm/kvm_main.c | 699 +++++++++++- virt/kvm/pfncache.c | 3 +- 44 files changed, 4956 insertions(+), 248 deletions(-) create mode 100644 tools/testing/selftests/kvm/plane_test.c create mode 100644 tools/testing/selftests/kvm/x86/hyperv_forward_hcall.c create mode 100644 tools/testing/selftests/kvm/x86/hyperv_tsc_page_attrs.c create mode 100644 tools/testing/selftests/kvm/x86/memattr_hugepage_test.c create mode 100644 tools/testing/selftests/kvm/x86/plane_x86_test.c create mode 100644 tools/testing/selftests/kvm/x86/planes_test.c create mode 100644 tools/testing/selftests/kvm/x86/planes_vtl_entry_test.c
Signature · 1 line
2.50.1
RFC PATCH 1/55 KVM: x86/mmu: Move ACC_* masks from spte.h to mmu.h
From: David Woodhouse <[email protected]>
The memory-attributes backport uses the ACC_* masks in mmu_internal.h, which cannot include spte.h because spte.h includes *it*. Anything pulling in mmu_internal.h without spte.h (debugfs.c, page_track.c) now fails to build.
Move the ACC_* definitions to mmu.h, next to the PT_WRITABLE_MASK and PT_USER_MASK definitions they are built from. Both headers already include it.
--- arch/x86/kvm/mmu.h | 14 ++++++++++++++ arch/x86/kvm/mmu/spte.h | 14 -------------- 2 files changed, 14 insertions(+), 14 deletions(-)
arch/x86/kvm/mmu.h +14 -0
@@ -32,6 +32,20 @@ extern bool __read_mostly enable_mmio_caching;#define PT_DIR_PAT_SHIFT 12#define PT_DIR_PAT_MASK (1ULL << PT_DIR_PAT_SHIFT) +#define ACC_EXEC_MASK 1 +#define ACC_WRITE_MASK PT_WRITABLE_MASK +#define ACC_USER_MASK PT_USER_MASK +#define ACC_ALL (ACC_EXEC_MASK | ACC_WRITE_MASK | ACC_USER_MASK) + +/* +* Read permission for a gfn's memory protection attributes. Not part of +* ACC_ALL and never set in a PTE's access mask: PTE formats cannot express +* no-read, and x86 rejects protection attributes that forbid only read +* (kvm_arch_mem_attributes_supported_prot()), so a no-read gfn always +* takes a memory fault exit before any SPTE is created. +*/ +#define ACC_READ_MASK BIT(3) +#define PT64_ROOT_5LEVEL 5#define PT64_ROOT_4LEVEL 4#define PT32_ROOT_LEVEL 2
arch/x86/kvm/mmu/spte.h +0 -14
@@ -45,20 +45,6 @@ static_assert(SPTE_TDP_AD_ENABLED == 0);#define SPTE_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_mask \ | shadow_x_mask | shadow_nx_mask | shadow_me_mask) -#define ACC_EXEC_MASK 1 -#define ACC_WRITE_MASK PT_WRITABLE_MASK -#define ACC_USER_MASK PT_USER_MASK -#define ACC_ALL (ACC_EXEC_MASK | ACC_WRITE_MASK | ACC_USER_MASK) - -/* -* Read permission for a gfn's memory protection attributes. Not part of -* ACC_ALL and never set in a PTE's access mask: PTE formats cannot express -* no-read, and x86 rejects protection attributes that forbid only read -* (kvm_arch_mem_attributes_supported_prot()), so a no-read gfn always -* takes a memory fault exit before any SPTE is created. -*/ -#define ACC_READ_MASK BIT(3) -/* The mask for the R/X bits in EPT PTEs */ #define SPTE_EPT_READABLE_MASK 0x1ull#define SPTE_EPT_EXECUTABLE_MASK 0x4ull
Signature · 1 line
2.50.1
RFC PATCH 2/55 KVM: x86: Let userspace opt in to multiple planes with KVM_CAP_PLANES
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
A VMM implementing a software privilege model such as Hyper-V VSM needs more than one plane, but kvm_arch_max_planes() returns the vendor ceiling — 1 everywhere except SEV-SNP, where VMPLs are guest-architectural — so KVM_CREATE_PLANE rejects every id above zero.
Make KVM_CAP_PLANES a VM enable capability taking the number of planes userspace is going to manage, up to KVM_MAX_PLANES. It must be set before any vCPU or extra plane exists, so nothing is sized against the old limit, and cannot be changed once set. Non-default VM types are
rejected: SEV-SNP keeps deriving its count from the supported VMPLs, and protected-state VM types need their own audit first. A VM that does not enable the capability keeps reporting a single plane.
--- Documentation/virt/kvm/api.rst | 27 +++++++++++++++++++++ arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 42 ++++++++++++++++++++++++++++++++- 3 files changed, 69 insertions(+), 1 deletion(-)
Documentation/virt/kvm/api.rst +27 -0
@@ -6598,6 +6598,12 @@ KVM_S390_KEYOP_SSKE Creates a new *plane*, i.e. a separate privilege level for the virtual machine. +The range of valid plane ids is [0, N) where N is the value returned by +KVM_CHECK_EXTENSION(KVM_CAP_PLANES) on the VM file descriptor. Plane 0 is +the VM itself; for VM types where the plane count is not guest-architectural, +ids above 0 only become valid after userspace enables KVM_CAP_PLANES with +the number of planes it is going to manage (see section 7.47). + Each plane has a numeric id that is used when communicating with KVM. While KVM is currently agnostic to whether low ids are more or less privileged, it is expected that this will not always be the case in the future. For example KVM
@@ -8959,6 +8965,27 @@ helpful if user space wants to emulate instructions which are not This capability can be enabled dynamically even if VCPUs were already created and are running. +7.47 KVM_CAP_PLANES +------------------- + +:Architectures: x86 +:Target: VM +:Parameters: args[0] contains the number of planes to make available + +Opts the VM in to using multiple planes, raising the limit on plane ids +that KVM_CREATE_PLANE accepts to args[0] (between 1 and the KVM_MAX_PLANES +value implied by KVM_CHECK_EXTENSION(KVM_CAP_PLANES) on the system fd). + +Planes other than plane 0 only exist when userspace manages them, so by +default a VM reports and accepts a single plane. VM types whose plane +count is guest-architectural (such as SEV-SNP, where it reflects the +supported VMPLs) report that count without this capability, and enabling +it on them fails with -EINVAL. + +This capability must be enabled before any vCPU or extra plane has been +created, and the chosen count cannot be changed afterwards; otherwise +-EEXIST is returned. + 7.49 KVM_CAP_HCALL_FAULT_EXIT ------------------------------------
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1442,6 +1442,7 @@ struct kvm_arch {unsigned int indirect_shadow_pages;u8 mmu_valid_gen;u8 vm_type; +unsigned int user_max_planes;bool has_private_mem;bool has_protected_state;bool has_protected_eoi;
arch/x86/kvm/x86.c +41 -1
58 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 3/55 KVM: selftests: Add VM plane infrastructure and tests
In reply to David WoodhouseFrom: Paolo Bonzini <[email protected]>
Add plane and plane-vCPU wrappers to the selftest library and two tests:plane_test (plane and plane vCPU creation, sibling rules, register state isolation) and plane_x86_test (per-plane FPU, MSRs and LAPIC; MSI delivery to a plane). Planes require a split IRQ chip, so the tests create their VM with vm_create_barebones_irqchip(true).
The code is from the planes RFC's selftest work, already adapted to the KVM_CREATE_VCPU on a plane fd shape of the posted series.
[dwmw2: ported to this stack's uAPI: KVM_CAP_PLANES is a VM-level
opt-in taking the plane count and must precede vCPU creation, so the tests enable it with 2 planes right after VM creation. The main() probe uses a split-irqchip VM, because kvm_arch_max_planes() reports 1 without one, and skips when the opt-in is refused. Two subtest groups of the later planes series are not carried: the foreign-thread EBUSY test (this kernel does not restrict vCPU ioctls to the vCPU's own thread) and the userspace plane-switch group (this kernel drives plane switches through the Hyper-V VSM hypercalls). kvm_force_split_irqchip already exists here; all seven remaining subtests pass on this kernel.]
(re-done by hand from Mushahid's port)
--- tools/testing/selftests/kvm/Makefile.kvm | 2 + .../testing/selftests/kvm/include/kvm_util.h | 73 ++++ .../selftests/kvm/include/x86/processor.h | 1 + tools/testing/selftests/kvm/lib/kvm_util.c | 74 +++- .../testing/selftests/kvm/lib/x86/processor.c | 15 + tools/testing/selftests/kvm/plane_test.c | 137 ++++++++ .../selftests/kvm/x86/plane_x86_test.c | 325 ++++++++++++++++++ 7 files changed, 622 insertions(+), 5 deletions(-) create mode 100644 tools/testing/selftests/kvm/plane_test.c create mode 100644 tools/testing/selftests/kvm/x86/plane_x86_test.c
tools/testing/selftests/kvm/Makefile.kvm +2 -0
@@ -66,6 +66,7 @@ TEST_GEN_PROGS_COMMON += kvm_page_table_test TEST_GEN_PROGS_COMMON += set_memory_region_test TEST_GEN_PROGS_COMMON += memslot_modification_stress_test TEST_GEN_PROGS_COMMON += memslot_perf_test +TEST_GEN_PROGS_COMMON += plane_test # Compiled test targets TEST_GEN_PROGS_x86 = $(TEST_GEN_PROGS_COMMON)
@@ -102,6 +103,7 @@ TEST_GEN_PROGS_x86 += x86/nested_tsc_adjust_test TEST_GEN_PROGS_x86 += x86/nested_tsc_scaling_test TEST_GEN_PROGS_x86 += x86/nested_vmsave_vmload_test TEST_GEN_PROGS_x86 += x86/platform_info_test +TEST_GEN_PROGS_x86 += x86/plane_x86_test TEST_GEN_PROGS_x86 += x86/pmu_counters_test TEST_GEN_PROGS_x86 += x86/pmu_event_filter_test TEST_GEN_PROGS_x86 += x86/private_mem_conversions_test
tools/testing/selftests/kvm/include/kvm_util.h +73 -0
119 lines not on this page · raw message
tools/testing/selftests/kvm/include/x86/processor.h +1 -0
@@ -1136,6 +1136,7 @@ static inline void vcpu_clear_cpuid_feature(struct kvm_vcpu *vcpu,u64 vcpu_get_msr (struct kvm_vcpu *vcpu,u64 msr_index);int _vcpu_set_msr (struct kvm_vcpu *vcpu,u64 msr_index,u64 msr_value); +int _plane_vcpu_set_msr (struct kvm_plane_vcpu *plane_vcpu,u64 msr_index,u64 msr_value);/* * Assert on an MSR access(es) and pretty print the MSR name when possible.
tools/testing/selftests/kvm/lib/kvm_util.c +69 -5
110 lines not on this page · raw message
tools/testing/selftests/kvm/lib/x86/processor.c +15 -0
@@ -1092,6 +1092,21 @@ int _vcpu_set_msr(struct kvm_vcpu *vcpu, u64 msr_index, u64 msr_value)return __vcpu_ioctl (vcpu,KVM_SET_MSRS , &buffer.header); } +int _plane_vcpu_set_msr (struct kvm_plane_vcpu *plane_vcpu,u64 msr_index,u64 msr_value) +{ +struct { +struct kvm_msrs header; +struct kvm_msr_entry entry; + } buffer = {}; + +memset (&buffer,0 ,sizeof (buffer)); + buffer.header.nmsrs =1 ; + buffer.entry.index = msr_index; + buffer.entry.data = msr_value; + +return __plane_vcpu_ioctl (plane_vcpu,KVM_SET_MSRS , &buffer.header); +} +void vcpu_args_set (struct kvm_vcpu *vcpu,unsigned int num, ...) {va_list ap;
tools/testing/selftests/kvm/plane_test.c new file +137 -0
137 lines not on this page · raw message
tools/testing/selftests/kvm/x86/plane_x86_test.c new file +325 -0
325 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 4/55 KVM: selftests: Add x86 planes lifecycle test
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
Cover KVM_CAP_PLANES argument validation and ordering, plane creation bounds, plane vCPU creation, and per-plane isolation of vCPU register state.
The tools copy of linux/kvm.h is synced to pick up the VM planes definitions the test builds against.
--- tools/include/uapi/linux/kvm.h | 40 +++++- tools/testing/selftests/kvm/Makefile.kvm | 1 + tools/testing/selftests/kvm/x86/planes_test.c | 120 ++++++++++++++++++ 3 files changed, 156 insertions(+), 5 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86/planes_test.c
tools/include/uapi/linux/kvm.h +35 -5
@@ -11,6 +11,7 @@#include <linux/const.h> #include <linux/types.h> #include <linux/compiler.h> +#include <linux/stddef.h> #include <linux/ioctl.h> #include <asm/kvm.h>
@@ -145,6 +146,13 @@ struct kvm_exit_snp_req_certs {__u64 ret; }; +struct kvm_plane_event_exit { +#define KVM_PLANE_EVENT_CREATE_VCPU 1 +__u32 cause; +__u32 plane; +__u64 extra[8 ]; +}; +#define KVM_S390_GET_SKEYS_NONE 1#define KVM_S390_SKEYS_MAX 1048576
@@ -192,6 +200,7 @@ struct kvm_exit_snp_req_certs {#define KVM_EXIT_ARM_SEA 41#define KVM_EXIT_ARM_LDST64B 42#define KVM_EXIT_SNP_REQ_CERTS 43 +#define KVM_EXIT_PLANE_EVENT 44/* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */
@@ -456,6 +465,9 @@ struct kvm_run { } notify;/* KVM_EXIT_MEMORY_FAULT */ struct { +#define KVM_MEMORY_EXIT_FLAG_READ (1ULL << 0) +#define KVM_MEMORY_EXIT_FLAG_WRITE (1ULL << 1) +#define KVM_MEMORY_EXIT_FLAG_EXEC (1ULL << 2)#define KVM_MEMORY_EXIT_FLAG_PRIVATE (1ULL << 3)__u64 flags;__u64 gpa;
@@ -496,6 +508,8 @@ struct kvm_run { } arm_sea;/* KVM_EXIT_SNP_REQ_CERTS */ struct kvm_exit_snp_req_certs snp_req_certs; +/* KVM_EXIT_PLANE_EVENT */ +struct kvm_plane_event_exit plane_event;/* Fix the size of the union. */ char padding[256 ]; };
@@ -542,7 +556,7 @@ struct kvm_coalesced_mmio {struct kvm_coalesced_mmio_ring {__u32 first, last; -struct kvm_coalesced_mmio coalesced_mmio[]; + __DECLARE_FLEX_ARRAY(struct kvm_coalesced_mmio, coalesced_mmio); };#define KVM_COALESCED_MMIO_MAX \
@@ -592,7 +606,7 @@ struct kvm_clear_dirty_log {/* for KVM_SET_SIGNAL_MASK */ struct kvm_signal_mask {__u32 len; -__u8 sigset[]; + __DECLARE_FLEX_ARRAY(__u8, sigset); };/* for KVM_TPR_ACCESS_REPORTING */
@@ -703,6 +717,11 @@ struct kvm_enable_cap {#define KVM_VM_TYPE_ARM_IPA_SIZE_MASK 0xffULL#define KVM_VM_TYPE_ARM_IPA_SIZE (x) \ ((x) & KVM_VM_TYPE_ARM_IPA_SIZE_MASK) + +#define KVM_VM_TYPE_ARM_PROTECTED (1UL << 31) +#define KVM_VM_TYPE_ARM_MASK (KVM_VM_TYPE_ARM_IPA_SIZE_MASK | \ + KVM_VM_TYPE_ARM_PROTECTED) +/* * ioctls for /dev/kvm fds: */
@@ -724,6 +743,11 @@ struct kvm_enable_cap {#define KVM_GET_EMULATED_CPUID _IOWR(KVMIO, 0x09, struct kvm_cpuid2)#define KVM_GET_MSR_FEATURE_INDEX_LIST _IOWR(KVMIO, 0x0a, struct kvm_msr_list) +/* +* Maximum number of supported planes +*/ +#define KVM_MAX_PLANES 16 +/* * Extension capability list. */
@@ -989,6 +1013,8 @@ struct kvm_enable_cap {#define KVM_CAP_ARM_SEA_TO_USER 245#define KVM_CAP_S390_USER_OPEREXEC 246#define KVM_CAP_S390_KEYOP 247 +#define KVM_CAP_S390_VSIE_ESAMODE 248 +#define KVM_CAP_PLANES 249struct kvm_irq_routing_irqchip {__u32 irqchip;
@@ -1051,7 +1077,7 @@ struct kvm_irq_routing_entry {struct kvm_irq_routing {__u32 nr;__u32 flags; -struct kvm_irq_routing_entry entries[]; + __DECLARE_FLEX_ARRAY(struct kvm_irq_routing_entry, entries); };#define KVM_IRQFD_FLAG_DEASSIGN (1 << 0)
@@ -1142,7 +1168,7 @@ struct kvm_dirty_tlb {struct kvm_reg_list {__u64 n;/* number of regs */ -__u64 reg[]; + __DECLARE_FLEX_ARRAY(__u64, reg); };struct kvm_one_reg {
@@ -1224,6 +1250,8 @@ enum kvm_device_type {#define KVM_DEV_TYPE_LOONGARCH_EIOINTC KVM_DEV_TYPE_LOONGARCH_EIOINTC KVM_DEV_TYPE_LOONGARCH_PCHPIC,#define KVM_DEV_TYPE_LOONGARCH_PCHPIC KVM_DEV_TYPE_LOONGARCH_PCHPIC + KVM_DEV_TYPE_LOONGARCH_DMSINTC, +#define KVM_DEV_TYPE_LOONGARCH_DMSINTC KVM_DEV_TYPE_LOONGARCH_DMSINTC KVM_DEV_TYPE_ARM_VGIC_V5,#define KVM_DEV_TYPE_ARM_VGIC_V5 KVM_DEV_TYPE_ARM_VGIC_V5
@@ -1340,6 +1368,8 @@ struct kvm_s390_keyop {#define KVM_GET_DEVICE_ATTR _IOW(KVMIO, 0xe2, struct kvm_device_attr)#define KVM_HAS_DEVICE_ATTR _IOW(KVMIO, 0xe3, struct kvm_device_attr) +#define KVM_CREATE_PLANE _IO(KVMIO, 0xe4) +/* * ioctls for vcpu fds */
@@ -1610,7 +1640,7 @@ struct kvm_stats_desc {#ifdef __KERNEL__char name[KVM_STATS_NAME_SIZE ];#else -char name[]; +__DECLARE_FLEX_ARRAY (char, name);#endif };
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -104,6 +104,7 @@ TEST_GEN_PROGS_x86 += x86/nested_tsc_scaling_test TEST_GEN_PROGS_x86 += x86/nested_vmsave_vmload_test TEST_GEN_PROGS_x86 += x86/platform_info_test TEST_GEN_PROGS_x86 += x86/plane_x86_test +TEST_GEN_PROGS_x86 += x86/planes_test TEST_GEN_PROGS_x86 += x86/pmu_counters_test TEST_GEN_PROGS_x86 += x86/pmu_event_filter_test TEST_GEN_PROGS_x86 += x86/private_mem_conversions_test
tools/testing/selftests/kvm/x86/planes_test.c new file +120 -0
120 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 5/55 KVM: x86: hyper-v: Add KVM_CAP_HYPERV_FORWARD_HCALL
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
A VMM implementing Hyper-V hypercalls that KVM does not know about, such as the VSM set, has no way to receive them: kvm_hv_hypercall() fails unknown codes with HV_STATUS_INVALID_HYPERCALL_CODE, and only syndbg and slow-form extended hypercalls ever exit to userspace.
Add a VM capability which routes any hypercall KVM would have failed as unknown out through the existing KVM_EXIT_HYPERV_HCALL exit, fast forms included. The exit already carries the control value and both parameters (RDX and R8 for fast calls), and XMM state is reachable through the ordinary vCPU ioctls, so no new exit format is needed. A future in-kernel accelerator for a specific hypercall can still claim its code behind its own capability, with userspace handling as the fallback.
--- Documentation/virt/kvm/api.rst | 26 ++++++++++++++++++++++++++ arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/hyperv.c | 5 ++++- arch/x86/kvm/x86.c | 10 ++++++++++ include/uapi/linux/kvm.h | 1 + 5 files changed, 42 insertions(+), 1 deletion(-)
Documentation/virt/kvm/api.rst +26 -0
@@ -8986,6 +8986,32 @@ This capability must be enabled before any vCPU or extra plane has been created, and the chosen count cannot be changed afterwards; otherwise -EEXIST is returned. +7.48 KVM_CAP_HYPERV_FORWARD_HCALL +--------------------------------- + +:Architectures: x86 +:Target: VM +:Parameters: args[0] is 1 to enable forwarding, 0 to disable it + +When enabled, a Hyper-V hypercall that KVM would otherwise have failed +with HV_STATUS_INVALID_HYPERCALL_CODE instead exits to userspace with +KVM_EXIT_HYPERV_HCALL, and extended hypercalls in fast form exit the +same way instead of failing with HV_STATUS_INVALID_PARAMETER. This +lets a VMM implement hypercalls KVM does not know, such as the Hyper-V +VSM set. The synthetic-debugger hypercalls (HvCallResetDebugSession, +HvCallPostDebugData, HvCallRetrieveDebugData) are an exception: while +the synthetic debugger is not enabled they still fail in KVM with +HV_STATUS_INVALID_HYPERCALL_CODE and are not forwarded. + +The exit carries the control value and the input and output parameters +(for fast hypercalls, RDX and R8 from a 64-bit caller, or the EBX:ECX +and EDI:ESI pairs from a 32-bit caller). Register argument and +return state of fast hypercalls, such as the XMM registers, is read and +written with the ordinary vCPU state ioctls before resuming the vCPU +with ``hyperv.u.hcall.result`` set. Hypercalls KVM implements itself, +and the ordering of CPUID-enforcement checks against hypercall +dispatch, are unaffected. + 7.49 KVM_CAP_HCALL_FAULT_EXIT ------------------------------------
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1204,6 +1204,7 @@ struct kvm_hv {struct mutex hv_lock;u64 hv_guest_os_id;u64 hv_hypercall; +bool hv_forward_hcalls;u64 hv_tsc_page;enum hv_tsc_page_status hv_tsc_page_status;
arch/x86/kvm/hyperv.c +4 -1
@@ -2777,12 +2777,15 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)return kvm_hv_hypercall_userspace_exit (vcpu, &hc); } case HV_EXT_CALL_QUERY_CAPABILITIES ... HV_EXT_CALL_MAX: -if (unlikely (hc.fast)) { +if (unlikely (hc.fast) && + !to_kvm_hv (vcpu->kvm)->hv_forward_hcalls) { ret =HV_STATUS_INVALID_PARAMETER ;break ; }return kvm_hv_hypercall_userspace_exit (vcpu, &hc);default : +if (to_kvm_hv (vcpu->kvm)->hv_forward_hcalls) +return kvm_hv_hypercall_userspace_exit (vcpu, &hc); ret =HV_STATUS_INVALID_HYPERCALL_CODE ;break ; }
arch/x86/kvm/x86.c +10 -0
@@ -4883,6 +4883,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_HYPERV_CPUID :case KVM_CAP_HYPERV_ENFORCE_CPUID :case KVM_CAP_SYS_HYPERV_CPUID : +case KVM_CAP_HYPERV_FORWARD_HCALL :#endif case KVM_CAP_HCALL_FAULT_EXIT :case KVM_CAP_PCI_SEGMENT :
@@ -6886,6 +6887,15 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,mutex_unlock (&kvm->lock);break ; } +#ifdef CONFIG_KVM_HYPERV +case KVM_CAP_HYPERV_FORWARD_HCALL : + r = -EINVAL ; +if (cap->args[0 ] & ~1ULL ) +break ; +to_kvm_hv (kvm)->hv_forward_hcalls = !!cap->args[0 ]; + r =0 ; +break ; +#endif case KVM_CAP_X2APIC_API : r = -EINVAL ;if (cap->args[0 ] & ~KVM_X2APIC_API_VALID_FLAGS )
include/uapi/linux/kvm.h +1 -0
@@ -1015,6 +1015,7 @@ struct kvm_enable_cap {#define KVM_CAP_S390_KEYOP 247#define KVM_CAP_S390_VSIE_ESAMODE 248#define KVM_CAP_PLANES 249 +#define KVM_CAP_HYPERV_FORWARD_HCALL 250#define KVM_CAP_HCALL_FAULT_EXIT 252struct kvm_irq_routing_irqchip {
Signature · 1 line
2.50.1
RFC PATCH 6/55 KVM: selftests: Add Hyper-V hypercall forwarding test
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
Drive an unimplemented hypercall code through the three interesting
shapes: forwarding disabled (the guest sees HV_STATUS_INVALID_HYPERCALL_CODE from KVM itself), a slow-form call (the exit carries the control value and GPAs, and the userspace-chosen result reaches the guest), and a fast-form call (RDX and R8 in the exit parameters, XMM0 visible through KVM_GET_FPU).
The tools copy of linux/kvm.h picks up the KVM_CAP_HYPERV_FORWARD_HCALL capability number for the test.
--- tools/include/uapi/linux/kvm.h | 1 + tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/x86/hyperv_forward_hcall.c | 144 ++++++++++++++++++ 3 files changed, 146 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/hyperv_forward_hcall.c
tools/include/uapi/linux/kvm.h +1 -0
@@ -1015,6 +1015,7 @@ struct kvm_enable_cap {#define KVM_CAP_S390_KEYOP 247#define KVM_CAP_S390_VSIE_ESAMODE 248#define KVM_CAP_PLANES 249 +#define KVM_CAP_HYPERV_FORWARD_HCALL 250struct kvm_irq_routing_irqchip {__u32 irqchip;
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -85,6 +85,7 @@ TEST_GEN_PROGS_x86 += x86/hyperv_cpuid TEST_GEN_PROGS_x86 += x86/hyperv_evmcs TEST_GEN_PROGS_x86 += x86/hyperv_extended_hypercalls TEST_GEN_PROGS_x86 += x86/hyperv_features +TEST_GEN_PROGS_x86 += x86/hyperv_forward_hcall TEST_GEN_PROGS_x86 += x86/hyperv_ipi TEST_GEN_PROGS_x86 += x86/hyperv_svm_test TEST_GEN_PROGS_x86 += x86/hyperv_tlb_flush
tools/testing/selftests/kvm/x86/hyperv_forward_hcall.c new file +144 -0
144 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 7/55 KVM: Add KVM_GET/SET_PLANE_RUN_STATE vCPU ioctls
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
A VMM doing VSM in userspace has to switch the executing plane when the guest asks for a privilege transition, but a vCPU's plane run state is kernel-internal — set only by the SEV-SNP VMPL switch, plane-0 event delivery, and vCPU creation. A VTL call forwarded to userspace is a dead end.
Add ioctls to read and write the run state as stopped/runnable. KVM_RUN already executes the lowest-numbered runnable plane, so a userspace plane switch is just: mark the target plane's vCPU runnable, mark the current one stopped, re-enter KVM_RUN.
A plane 0 stopped this way *stays* stopped. Forcing it runnable on a pending event fits the SEV-SNP model, but under a userspace privilege model plane 0 would preempt the higher VTL on any interrupt and resume mid-hypercall with whatever completion value the shared kvm_run last held; the events stay pending until the VMM switches back. In-kernel transitions never set the stopped-by-userspace flag, so SNP behaviour is unchanged.
The ioctls return -EINVAL unless kvm_arch_max_planes() reports plane support, the same condition KVM_CREATE_PLANE enforces: only x86 consumes KVM_REQ_PLANE_RESCHED, and on the other architectures a stray request bit would abort every subsequent entry attempt, leaving KVM_RUN spinning at 100% with no way for the VMM to recover the vCPU.
--- Documentation/virt/kvm/api.rst | 26 ++++++++++++++++++++ arch/x86/kvm/x86.c | 3 ++- include/linux/kvm_host.h | 6 +++++ include/uapi/linux/kvm.h | 4 ++++ virt/kvm/kvm_main.c | 44 ++++++++++++++++++++++++++++++++++ 5 files changed, 82 insertions(+), 1 deletion(-)
Documentation/virt/kvm/api.rst +26 -0
@@ -6611,6 +6611,32 @@ in the future may use the plane id when planes are supported by hardware (as is the case for VMPLs in AMD), or if KVM supports accelerated plane switch operations (as might be the case for Hyper-V VTLs). +4.146 KVM_GET_PLANE_RUN_STATE / KVM_SET_PLANE_RUN_STATE +-------------------------------------------------------- + +:Capability: KVM_CAP_PLANES +:Architectures: x86 +:Type: vcpu ioctl +:Parameters: pointer to a __u32 run state +:Returns: 0 on success, <0 on error + +Reads or sets the per-plane run state of a vCPU: +KVM_PLANE_RUN_STATE_STOPPED (0) or KVM_PLANE_RUN_STATE_RUNNABLE (1). + +When a vCPU thread calls KVM_RUN, KVM executes the lowest-numbered plane +whose vCPU is runnable, falling back to plane 0. A VMM implementing a +software privilege model (such as Hyper-V VSM) switches planes by +marking the target plane's vCPU runnable and the current one stopped +before re-entering KVM_RUN. Pending events for plane 0 make it runnable +again on their own only as long as userspace has not explicitly stopped +it with this ioctl; a VMM-stopped plane 0 keeps its events pending until +the VMM makes it runnable, so a software privilege model can keep a +higher plane running undisturbed. The ioctl is accepted on any plane's +vCPU fd. + +The run state is distinct from KVM_GET/SET_MP_STATE, which describes +the architectural state of a vCPU within its plane. + 5. The kvm_run structure ========================
arch/x86/kvm/x86.c +2 -1
@@ -11508,7 +11508,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) } } -if (kvm_check_plane0_events (vcpu)) { +if (kvm_check_plane0_events (vcpu) && + !vcpu->common->vcpus[0 ]->plane_user_stopped) {kvm_vcpu_set_plane_runnable (vcpu->common->vcpus[0 ]);kvm_make_request (KVM_REQ_EVENT , vcpu);
include/linux/kvm_host.h +6 -0
@@ -413,6 +413,12 @@ struct kvm_vcpu {u64 plane_requests;enum kvm_vcpu_state plane_state; +/* +* Set when userspace stopped this plane with +* KVM_SET_PLANE_RUN_STATE: pending plane-0 events then no longer +* make plane 0 runnable on their own, the VMM owns the switch. +*/ +bool plane_user_stopped;/* S390 only */ bool valid_wakeup;
include/uapi/linux/kvm.h +4 -0
@@ -1371,6 +1371,10 @@ struct kvm_s390_keyop {#define KVM_HAS_DEVICE_ATTR _IOW(KVMIO, 0xe3, struct kvm_device_attr)#define KVM_CREATE_PLANE _IO(KVMIO, 0xe4) +#define KVM_GET_PLANE_RUN_STATE _IOR(KVMIO, 0xe5, __u32) +#define KVM_SET_PLANE_RUN_STATE _IOW(KVMIO, 0xe6, __u32) +#define KVM_PLANE_RUN_STATE_STOPPED 0 +#define KVM_PLANE_RUN_STATE_RUNNABLE 1/* * ioctls for vcpu fds
virt/kvm/kvm_main.c +44 -0
56 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 8/55 KVM: selftests: Cover plane run-state ioctls in planes_test
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
Assert the initial stopped state of a new plane-1 vCPU, a runnable and back-to-stopped round trip, and rejection of out-of-range states.
The tools copy of linux/kvm.h picks up the plane run-state ioctl definitions.
--- tools/include/uapi/linux/kvm.h | 4 ++++ tools/testing/selftests/kvm/x86/planes_test.c | 22 +++++++++++++++++++ 2 files changed, 26 insertions(+)
tools/include/uapi/linux/kvm.h +4 -0
@@ -1370,6 +1370,10 @@ struct kvm_s390_keyop {#define KVM_HAS_DEVICE_ATTR _IOW(KVMIO, 0xe3, struct kvm_device_attr)#define KVM_CREATE_PLANE _IO(KVMIO, 0xe4) +#define KVM_GET_PLANE_RUN_STATE _IOR(KVMIO, 0xe5, __u32) +#define KVM_SET_PLANE_RUN_STATE _IOW(KVMIO, 0xe6, __u32) +#define KVM_PLANE_RUN_STATE_STOPPED 0 +#define KVM_PLANE_RUN_STATE_RUNNABLE 1/* * ioctls for vcpu fds
tools/testing/selftests/kvm/x86/planes_test.c +22 -0
@@ -89,6 +89,28 @@ static void test_plane_lifecycle(void)TEST_ASSERT (vcpu1_fd >=0 ,"KVM_CREATE_VCPU on plane 1 failed: errno %d" , errno); +/* Plane 1 starts stopped; run state is readable and writable. */ + { +u32 state =0xff ; + +kvm_ioctl (vcpu1_fd,KVM_GET_PLANE_RUN_STATE , &state); +TEST_ASSERT_EQ (state,KVM_PLANE_RUN_STATE_STOPPED ); + + state =KVM_PLANE_RUN_STATE_RUNNABLE ; +kvm_ioctl (vcpu1_fd,KVM_SET_PLANE_RUN_STATE , &state); + state =0xff ; +kvm_ioctl (vcpu1_fd,KVM_GET_PLANE_RUN_STATE , &state); +TEST_ASSERT_EQ (state,KVM_PLANE_RUN_STATE_RUNNABLE ); + + state =KVM_PLANE_RUN_STATE_STOPPED ; +kvm_ioctl (vcpu1_fd,KVM_SET_PLANE_RUN_STATE , &state); + + state =2 ; +TEST_ASSERT_EQ (__kvm_ioctl (vcpu1_fd,KVM_SET_PLANE_RUN_STATE , + &state),-1 ); +TEST_ASSERT_EQ (errno,EINVAL ); + } +/* Register state is per plane: write plane 1, plane 0 unaffected. */ kvm_ioctl (vcpu1_fd,KVM_GET_REGS , ®s1); regs1.rbx =0x5157ca11 ;/* arbitrary marker */
Signature · 1 line
2.50.1
RFC PATCH 9/55 KVM: x86: hyper-v: Let userspace fault a forwarded hypercall
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The architectural failure mode for an invalid VTL transition is #UD at the VMCALL, not a hypercall status — a status would advance RIP and complete the call. Define a result flag for the forwarded-hypercall completion which queues #UD at the unadvanced RIP and ignores the rest of the result.
--- arch/x86/kvm/hyperv.c | 14 +++++++++++++- include/uapi/linux/kvm.h | 8 ++++++++ tools/include/uapi/linux/kvm.h | 9 +++++++++ 3 files changed, 30 insertions(+), 1 deletion(-)
arch/x86/kvm/hyperv.c +13 -1
@@ -2473,7 +2473,19 @@ static int kvm_hv_hypercall_complete(struct kvm_vcpu *vcpu, u64 result)static int kvm_hv_hypercall_complete_userspace (struct kvm_vcpu *vcpu) { -return kvm_hv_hypercall_complete (vcpu, vcpu->run->hyperv.u.hcall.result); +u64 result = vcpu->run->hyperv.u.hcall.result; + +/* +* A hypercall the VMM judged malformed beyond returning a status +* (such as a VTL call that is invalid in the current state) faults +* at the instruction instead of completing. +*/ +if (result &KVM_HV_HCALL_RESULT_RAISE_UD ) { +kvm_queue_exception (vcpu,UD_VECTOR ); +return 1 ; + } + +return kvm_hv_hypercall_complete (vcpu, result); }static int kvm_hvcall_signal_event (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc)
include/uapi/linux/kvm.h +8 -0
@@ -1016,6 +1016,14 @@ struct kvm_enable_cap {#define KVM_CAP_S390_VSIE_ESAMODE 248#define KVM_CAP_PLANES 249#define KVM_CAP_HYPERV_FORWARD_HCALL 250 + +/* +* Userspace completion of a forwarded Hyper-V hypercall (KVM_EXIT_HYPERV_HCALL +* under KVM_CAP_HYPERV_FORWARD_HCALL): setting this bit in the result makes +* KVM raise #UD at the faulting instruction instead of completing the +* hypercall; all other result bits are then ignored. +*/ +#define KVM_HV_HCALL_RESULT_RAISE_UD (1ULL << 63)#define KVM_CAP_HCALL_FAULT_EXIT 252struct kvm_irq_routing_irqchip {
tools/include/uapi/linux/kvm.h +9 -0
@@ -1017,6 +1017,15 @@ struct kvm_enable_cap {#define KVM_CAP_PLANES 249#define KVM_CAP_HYPERV_FORWARD_HCALL 250 +/* +* Userspace completion of a forwarded Hyper-V hypercall (KVM_EXIT_HYPERV_HCALL +* under KVM_CAP_HYPERV_FORWARD_HCALL): setting this bit in the result makes +* KVM raise #UD at the faulting instruction instead of completing the +* hypercall; all other result bits are then ignored. +*/ +#define KVM_HV_HCALL_RESULT_RAISE_UD (1ULL << 63) +#define KVM_CAP_HCALL_FAULT_EXIT 252 +struct kvm_irq_routing_irqchip {__u32 irqchip;__u32 pin;
Signature · 1 line
2.50.1
RFC PATCH 10/55 KVM: x86: Allow MP state ioctls on plane vCPU fds
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
A new plane vCPU for an AP starts, like any non-boot x86 vCPU, in the uninitialised wait-for-INIT state, and nothing ever sends INIT/SIPI to the higher plane of an AP. The VMM placing an initial VTL context on it has to mark it runnable itself.
--- arch/x86/kvm/x86.c | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/kvm/x86.c +2 -0
@@ -6278,6 +6278,8 @@ bool kvm_arch_is_vcpu_plane_ioctl(unsigned ioctl)case KVM_SET_ONE_REG :case KVM_GET_SREGS2 :case KVM_SET_SREGS2 : +case KVM_GET_MP_STATE : +case KVM_SET_MP_STATE :case KVM_GET_VCPU_EVENTS :case KVM_SET_VCPU_EVENTS :case KVM_GET_XCRS :
Signature · 1 line
2.50.1
RFC PATCH 11/55 KVM: x86: Filter KVM_ENABLE_CAP on plane vCPU fds
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The plane-fd ioctl allowlist admits KVM_ENABLE_CAP wholesale, so a plane twin's fd reaches every vCPU cap handler: enlightened VMCS, direct TLB flush, register intercepts. The nested paths behind the first two have never been audited for plane_level > 0, the register intercept only ever diverts plane 0 (hv_divert_dt_write() skips higher planes, so arming it on a twin buys native-emulation overhead and no diversion), and any future vCPU cap would become plane-enabled by default without anyone deciding that.
Accept only the audited caps on a plane fd: the per-plane SynIC pair and the two CPUID-enforcement toggles, which mutate state the planes genuinely bank per twin. Everything else fails with -EINVAL, matching how non-allowlisted ioctls already behave on plane fds.
The exposure is widened by our per-plane Hyper-V state: the VTL code assumes per-vCPU capability state is uniform across a vCPU's twins unless a cap is explicitly plane-aware.
Found by a local Sashiko review:
### M17 — whole KVM_ENABLE_CAP surface admitted on plane fds kvm_arch_is_vcpu_plane_ioctl() (x86.c) lists KVM_ENABLE_CAP with no per-cap filter in kvm_vcpu_ioctl_enable_cap(), so plane fds reach enable_evmcs, enable_l2_tlb_flush, ENFORCE_CPUID, REG_INTERCEPT etc. No demonstrated bug (all mutate per-vCPU state), but the vendor nested paths are unaudited for plane_level>0 and any future vCPU cap becomes plane-enabled implicitly.
Fix shape: per-cap allowlist for plane vCPUs, or a comment recording the audit.
--- arch/x86/kvm/x86.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
arch/x86/kvm/x86.c +20 -0
@@ -6088,6 +6088,26 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,if (cap->flags)return -EINVAL ; +/* +* KVM_ENABLE_CAP is accepted on plane vCPU fds, but only the caps +* below have been audited for plane_level > 0. The nested paths +* behind eVMCS and direct TLB flush have not, and the register +* intercept only ever diverts plane 0 (hv_divert_dt_write() skips +* higher planes); refuse those rather than implicitly extending +* every current and future vCPU cap to planes. +*/ +if (vcpu->plane_level) { +switch (cap->cap) { +case KVM_CAP_HYPERV_SYNIC : +case KVM_CAP_HYPERV_SYNIC2 : +case KVM_CAP_HYPERV_ENFORCE_CPUID : +case KVM_CAP_ENFORCE_PV_FEATURE_CPUID : +break ; +default : +return -EINVAL ; + } + } +switch (cap->cap) {#ifdef CONFIG_KVM_HYPERV case KVM_CAP_HYPERV_SYNIC2 :
Signature · 1 line
2.50.1
RFC PATCH 12/55 KVM: x86: initialise Hyper-V state on every plane vCPU
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
When userspace sets a Hyper-V CPUID, kvm_set_cpuid() walks the vCPU's planes to allocate the per-plane kvm_vcpu_hv, but passes the plane-0 vCPU to kvm_hv_vcpu_init() on every iteration. Non-zero plane vCPUs are left without hv_vcpu, which trips the WARN_ON_ONCE(vcpu->arch.hyperv_enabled) in kvm_hv_set_cpuid() and leaves their vp_index and SynIC state uninitialised until a lazy init path happens to run. Pass the plane vCPU being iterated.
A twin created after KVM_SET_CPUID never passes through kvm_set_cpuid() at all: it shares the common CPUID, and kvm_arch_vcpu_create() syncs it through kvm_vcpu_after_set_cpuid(), so that walk cannot cover it and the twin hits the same WARN with an empty cpuid_cache. Allocate the context in kvm_vcpu_after_set_cpuid() when the CPUID has the Hyper-V signature and the vCPU doesn't have one yet. The allocation is idempotent, so the kvm_set_cpuid() path is unaffected; if it fails, treat Hyper-V as disabled for the vCPU exactly as a failing kvm_set_cpuid() would have.
Also admit KVM_ENABLE_CAP on plane vCPU fds, so userspace can enable per-vCPU capabilities such as KVM_CAP_HYPERV_SYNIC2 on a plane vCPU before it first runs.
--- arch/x86/kvm/cpuid.c | 18 +++++++++++++++--- arch/x86/kvm/cpuid.h | 1 + arch/x86/kvm/x86.c | 20 +++++++++++++++++++- 3 files changed, 35 insertions(+), 4 deletions(-)
arch/x86/kvm/cpuid.c +15 -3
@@ -324,7 +324,7 @@ static void kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu) vcpu->arch.ia32_xss, true); } -static bool kvm_cpuid_has_hyperv (struct kvm_vcpu *vcpu) +bool kvm_cpuid_has_hyperv (struct kvm_vcpu *vcpu) {#ifdef CONFIG_KVM_HYPERV struct kvm_cpuid_entry2 *entry;
@@ -380,6 +380,7 @@ void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)struct kvm_cpuid_entry2 *best;struct kvm_cpuid_entry2 *entry;bool allow_gbpages; +bool hyperv_enabled;int i;memset (common->arch.cpu_caps,0 ,sizeof (common->arch.cpu_caps));
@@ -456,7 +457,18 @@ void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)__cr4_reserved_bits (guest_cpu_cap_has, vcpu);#undef __kvm_cpu_cap_has -kvm_hv_set_cpuid (vcpu,kvm_cpuid_has_hyperv (vcpu)); + hyperv_enabled =kvm_cpuid_has_hyperv (vcpu); + +/* +* A plane twin created after KVM_SET_CPUID shares the common CPUID +* without passing through kvm_set_cpuid(), which is what allocates +* the Hyper-V context for the planes that exist at set time; +* allocate it here for a late-created twin. +*/ +if (hyperv_enabled &&kvm_hv_vcpu_init (vcpu)) + hyperv_enabled = false; + +kvm_hv_set_cpuid (vcpu, hyperv_enabled);/* Invoke the vendor callback only after the above state is updated. */ kvm_x86_call (vcpu_after_set_cpuid)(vcpu);
@@ -565,7 +577,7 @@ static int kvm_set_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid_entry2 *e2,#ifdef CONFIG_KVM_HYPERV if (kvm_cpuid_has_hyperv (vcpu)) {vcpu_for_each_plane (common, i, v) { - r =kvm_hv_vcpu_init (vcpu); + r =kvm_hv_vcpu_init (v);if (r) goto err; }
arch/x86/kvm/cpuid.h +1 -0
@@ -19,6 +19,7 @@ static inline void kvm_finalize_cpu_caps(void) }void kvm_vcpu_after_set_cpuid (struct kvm_vcpu *vcpu); +bool kvm_cpuid_has_hyperv (struct kvm_vcpu *vcpu);struct kvm_cpuid_entry2 *kvm_find_cpuid_entry2 (struct kvm_cpuid_entry2 *entries,int nent,u32 function,u64 index);/*
arch/x86/kvm/x86.c +19 -1
@@ -6286,6 +6286,7 @@ static int kvm_get_reg_list(struct kvm_vcpu *vcpu,bool kvm_arch_is_vcpu_plane_ioctl (unsigned ioctl) {switch (ioctl) { +case KVM_ENABLE_CAP :case KVM_GET_DEBUGREGS :case KVM_SET_DEBUGREGS :case KVM_GET_LAPIC :
@@ -13135,9 +13136,24 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) vcpu->arch.hv_root_tdp =INVALID_PAGE ;#endif +/* +* A plane twin created after KVM_SET_CPUID shares CPUID that may +* already advertise Hyper-V. Allocate the Hyper-V context now so +* a failure fails the creation; left to kvm_vcpu_after_set_cpuid() +* below, an -ENOMEM silently downgrades just this twin to no +* Hyper-V (its MSRs fail) while the shared CPUID still advertises +* the Hyper-V signature, and nothing short of another full +* KVM_SET_CPUID repairs it. +*/ +if (kvm_cpuid_has_hyperv (vcpu)) { + r =kvm_hv_vcpu_init (vcpu); +if (r) + goto free_guest_fpu; + } + r =kvm_x86_call (vcpu_create)(vcpu);if (r) - goto free_guest_fpu; + goto free_hv;kvm_xen_init_vcpu (vcpu);vcpu_load (vcpu);
@@ -13148,6 +13164,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)vcpu_put (vcpu);return 0 ; +free_hv: +kvm_hv_vcpu_uninit (vcpu); free_guest_fpu:fpu_free_guest_fpstate (&vcpu->arch.guest_fpu); free_emulate_ctxt:
Signature · 1 line
2.50.1
RFC PATCH 13/55 KVM: implement per-plane memory attributes
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
Memory attributes lived solely on struct kvm and the page role carried no plane identity, so every plane shared one attribute array and one set of TDP roots. A VTL1 guest protecting VTL0 pages therefore revoked its *own* access too: the Windows secure kernel's first HvCallModifyVtlProtectionMask made its own hypercall page non-executable, and the next VTL transition died on an unhandled protection fault.
Give each non-zero plane its own attribute xarray (plane 0 keeps kvm->mem_attrs, which also carries partition-wide attributes such as KVM_MEMORY_ATTRIBUTE_PRIVATE), accept KVM_SET_MEMORY_ATTRIBUTES on plane fds restricted to the NR/NW/NX protection bits, and add the plane to kvm_mmu_page_role so each plane builds its own roots with its own protections applied at fault time.
Guest accesses are checked against the attributes of the plane doing the accessing, not plane 0's: the vCPU-context hva lookups, kvm_vcpu_map(), the guest page read/write helpers (which gain a kvm_plane argument; the VM-level variants keep checking plane 0), and the software page walker's PTE reads. Without that, a higher plane is refused access to pages it has itself protected against plane 0 — the secure kernel's VP assist page registration and the page walks through its own protected page tables both fail exactly there.
Hugepage mixed-attribute tracking still follows plane 0 only; faults on other planes clamp to 4KiB whenever the plane carries any attribute within the maximal mapping range.
--- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/mmu/mmu.c | 78 +++++++++++---- arch/x86/kvm/mmu/paging_tmpl.h | 4 +- include/linux/kvm_host.h | 52 ++++++++++ virt/kvm/kvm_main.c | 172 +++++++++++++++++++++++++------- 5 files changed, 251 insertions(+), 57 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -1
@@ -366,7 +366,7 @@ union kvm_mmu_page_role { unsigned guest_mode:1; unsigned passthrough:1; unsigned is_mirror:1; - unsigned :4; + unsigned plane:4;/* * This is left at the top of the word so that
arch/x86/kvm/mmu/mmu.c +59 -19
133 lines not on this page · raw message
arch/x86/kvm/mmu/paging_tmpl.h +2 -2
@@ -403,13 +403,13 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,if (unlikely (kvm_is_error_hva (host_addr))) goto error; -if (!kvm_mem_attributes_may_read_gfn (vcpu->kvm,gpa_to_gfn (real_gpa))) { +if (!kvm_vcpu_mem_attributes_may_read_gfn (vcpu,gpa_to_gfn (real_gpa))) { walker->memory_attributes_fault = true; walker->gfn =gpa_to_gfn (real_gpa); goto error; } -if (!kvm_mem_attributes_may_write_gfn (vcpu->kvm,gpa_to_gfn (real_gpa))) +if (!kvm_vcpu_mem_attributes_may_write_gfn (vcpu,gpa_to_gfn (real_gpa))) walker->pte_writable[walker->level -1 ] = false; ptep_user = (pt_element_t __user *)((void *)host_addr + offset);
include/linux/kvm_host.h +52 -0
82 lines not on this page · raw message
virt/kvm/kvm_main.c +137 -35
376 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 14/55 KVM: x86: hyper-v: bank GUEST_OS_ID and HYPERCALL MSRs per plane
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The TLFS makes HV_X64_MSR_GUEST_OS_ID and HV_X64_MSR_HYPERCALL separate instances per VTL; KVM kept a single partition-wide copy, so the Windows secure kernel enabling its own hypercall page from VTL1 clobbered VTL0's.
Index both MSRs by the vCPU's plane. The hypercall page contents are still written into guest RAM on enable, now once per plane's page.
--- arch/x86/include/asm/kvm_host.h | 9 +++++++-- arch/x86/kvm/hyperv.c | 19 ++++++++++--------- arch/x86/kvm/hyperv.h | 3 ++- 3 files changed, 19 insertions(+), 12 deletions(-)
arch/x86/include/asm/kvm_host.h +7 -2
@@ -1202,8 +1202,13 @@ enum hv_tsc_page_status {/* Hyper-V emulation context */ struct kvm_hv {struct mutex hv_lock; -u64 hv_guest_os_id; -u64 hv_hypercall; +/* +* GUEST_OS_ID and the hypercall page are per-VTL instances in the +* TLFS; bank them per plane so a higher VTL enabling its own +* hypercall page does not clobber the lower VTL's. +*/ +u64 hv_guest_os_id[KVM_MAX_PLANES ]; +u64 hv_hypercall[KVM_MAX_PLANES ];bool hv_forward_hcalls;u64 hv_tsc_page;enum hv_tsc_page_status hv_tsc_page_status;
arch/x86/kvm/hyperv.c +10 -9
@@ -1342,7 +1342,7 @@ static void __kvm_hv_xsaves_xsavec_maybe_warn(struct kvm_vcpu *vcpu)if (hv->xsaves_xsavec_checked)return ; -if ((hv->hv_guest_os_id &KVM_HV_WIN2016_GUEST_ID_MASK ) != +if ((hv->hv_guest_os_id[0 ] &KVM_HV_WIN2016_GUEST_ID_MASK ) !=KVM_HV_WIN2016_GUEST_ID )return ;
@@ -1384,10 +1384,11 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data,switch (msr) {case HV_X64_MSR_GUEST_OS_ID : - hv->hv_guest_os_id = data; + hv->hv_guest_os_id[vcpu->plane_level] = data;/* setting guest os id to zero disables hypercall page */ -if (!hv->hv_guest_os_id) - hv->hv_hypercall &= ~HV_X64_MSR_HYPERCALL_ENABLE ; +if (!hv->hv_guest_os_id[vcpu->plane_level]) + hv->hv_hypercall[vcpu->plane_level] &= + ~HV_X64_MSR_HYPERCALL_ENABLE ;break ;case HV_X64_MSR_HYPERCALL : {u8 instructions[9 ];
@@ -1395,10 +1396,10 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data,u64 addr;/* if guest os id is not set hypercall should remain disabled */ -if (!hv->hv_guest_os_id) +if (!hv->hv_guest_os_id[vcpu->plane_level])break ;if (!(data &HV_X64_MSR_HYPERCALL_ENABLE )) { - hv->hv_hypercall = data; + hv->hv_hypercall[vcpu->plane_level] = data;break ; }
@@ -1427,7 +1428,7 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data, addr = data &HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_MASK ;if (kvm_vcpu_write_guest (vcpu, addr, instructions, i))return 1 ; - hv->hv_hypercall = data; + hv->hv_hypercall[vcpu->plane_level] = data;break ; }case HV_X64_MSR_REFERENCE_TSC :
@@ -1640,10 +1641,10 @@ static int kvm_hv_get_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata,switch (msr) {case HV_X64_MSR_GUEST_OS_ID : - data = hv->hv_guest_os_id; + data = hv->hv_guest_os_id[vcpu->plane_level];break ;case HV_X64_MSR_HYPERCALL : - data = hv->hv_hypercall; + data = hv->hv_hypercall[vcpu->plane_level];break ;case HV_X64_MSR_TIME_REF_COUNT : data =get_time_ref_counter (kvm);
arch/x86/kvm/hyperv.h +2 -1
@@ -97,7 +97,8 @@ int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata, bool host)static inline bool kvm_hv_hypercall_enabled (struct kvm_vcpu *vcpu) { -return vcpu->arch.hyperv_enabled &&to_kvm_hv (vcpu->kvm)->hv_guest_os_id; +return vcpu->arch.hyperv_enabled && +to_kvm_hv (vcpu->kvm)->hv_guest_os_id[vcpu->plane_level]; }int kvm_hv_hypercall (struct kvm_vcpu *vcpu);
Signature · 1 line
2.50.1
RFC PATCH 15/55 KVM: x86: hyper-v: bank the reference TSC page per plane
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
HV_X64_MSR_REFERENCE_TSC is a per-VTL instance like GUEST_OS_ID and HYPERCALL. With a single partition-wide copy, the secure kernel enabling its own reference TSC page from VTL1 invalidated VTL0's, and the normal kernel fell back to a TIME_REF_COUNT MSR read — a VM exit — on every clock read.
Keep the masterclock-maintained page bound to plane 0, and give non-zero planes a banked MSR whose page gets a one-shot copy of the current parameters; those stay valid while the masterclock is stable, which is enough for VTL1's bounded execution windows.
The copy is made only when hv->tsc_ref is host-computed. With HV_X64_MSR_TSC_EMULATION_CONTROL set (reenlightenment, restored across migration), kvm_hv_setup_tsc_page() repopulates hv->tsc_ref from the plane-0 page in guest RAM, which the lowest VTL can write; handing a copy of that to a higher plane with hypervisor authority would let VTL0 forge the sequence/scale/offset the secure kernel computes time from. In that case write the plane's page with a zero sequence, making it fall back to the TIME_REF_COUNT MSR. The reference VSM host keeps fully separate per-VTL pages and never crosses VTLs here.
The copy uses the same zero-sequence/body/sequence protocol as the plane-0 path, so a concurrent reader never sees torn parameters, and the MSR write fails when the guest write faults, as the HV_X64_MSR_HYPERCALL case does.
--- Documentation/virt/kvm/api.rst | 7 ++++ arch/x86/include/asm/kvm_host.h | 8 +++++ arch/x86/kvm/hyperv.c | 58 ++++++++++++++++++++++++++++++++- 3 files changed, 72 insertions(+), 1 deletion(-)
Documentation/virt/kvm/api.rst +7 -0
@@ -6611,6 +6611,13 @@ in the future may use the plane id when planes are supported by hardware (as is the case for VMPLs in AMD), or if KVM supports accelerated plane switch operations (as might be the case for Hyper-V VTLs). +On x86, the Hyper-V partition-wide MSRs HV_X64_MSR_GUEST_OS_ID, +HV_X64_MSR_HYPERCALL and HV_X64_MSR_REFERENCE_TSC are banked per plane, +matching their per-VTL instances in the TLFS: an MSR access reads or +writes the instance belonging to the plane of the vCPU fd that performs +it. A migrating VMM must therefore save and restore these MSRs through +a vCPU fd of every plane, not just plane 0. + 4.146 KVM_GET_PLANE_RUN_STATE / KVM_SET_PLANE_RUN_STATE --------------------------------------------------------
arch/x86/include/asm/kvm_host.h +8 -0
@@ -1211,6 +1211,8 @@ struct kvm_hv {u64 hv_hypercall[KVM_MAX_PLANES ];bool hv_forward_hcalls;u64 hv_tsc_page; +/* Banked REFERENCE_TSC MSR values for non-zero planes (VTLs). */ +u64 hv_tsc_page_plane[KVM_MAX_PLANES ];enum hv_tsc_page_status hv_tsc_page_status;/* Hyper-v based guest crash (NT kernel bugcheck) parameters */
@@ -1218,6 +1220,12 @@ struct kvm_hv {u64 hv_crash_ctl;struct ms_hyperv_tsc_page tsc_ref; +/* +* tsc_ref was read back from the plane-0 page (guest TSC emulation +* after migration) rather than computed by the host; such values +* are plane-0-writable and must never seed another plane's page. +*/ +bool tsc_ref_from_guest;struct idr conn_to_evt;
arch/x86/kvm/hyperv.c +57 -1
82 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 16/55 KVM: x86: allow KVM_SET_GUEST_DEBUG on plane vCPU fds
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
Debugging a non-zero plane needs single-step and breakpoint control on the plane vCPU. The debug exits surface through the plane-0 KVM_RUN loop like any other plane exit.
--- Documentation/virt/kvm/api.rst | 6 +++--- arch/x86/kvm/x86.c | 1 + 2 files changed, 4 insertions(+), 3 deletions(-)
Documentation/virt/kvm/api.rst +3 -3
@@ -1536,7 +1536,7 @@ for vm-wide capabilities. :Capability: KVM_CAP_MP_STATE :Architectures: x86, s390, arm64, riscv, loongarch -:Type: vcpu ioctl +:Type: vcpu ioctl (all planes) :Parameters: struct kvm_mp_state (out) :Returns: 0 on success; -1 on error
@@ -1617,7 +1617,7 @@ whether the vcpu is runnable. :Capability: KVM_CAP_MP_STATE :Architectures: x86, s390, arm64, riscv, loongarch -:Type: vcpu ioctl +:Type: vcpu ioctl (all planes) :Parameters: struct kvm_mp_state (in) :Returns: 0 on success; -1 on error
@@ -3746,7 +3746,7 @@ handled. :Capability: KVM_CAP_SET_GUEST_DEBUG :Architectures: x86, s390, ppc, arm64 -:Type: vcpu ioctl +:Type: vcpu ioctl (all planes) :Parameters: struct kvm_guest_debug (in) :Returns: 0 on success; -1 on error
arch/x86/kvm/x86.c +1 -0
@@ -6287,6 +6287,7 @@ bool kvm_arch_is_vcpu_plane_ioctl(unsigned ioctl) {switch (ioctl) {case KVM_ENABLE_CAP : +case KVM_SET_GUEST_DEBUG :case KVM_GET_DEBUGREGS :case KVM_SET_DEBUGREGS :case KVM_GET_LAPIC :
Signature · 1 line
2.50.1
RFC PATCH 17/55 KVM: x86: hyper-v: flush the calling plane's TLB on flush hypercalls
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The Hyper-V TLB flush hypercalls resolve their target vCPUs through the plane-0 array, so a flush issued from a non-zero plane flushed only the plane-0 vCPUs and the caller's own plane kept stale translations. The Windows secure kernel repopulated a PFN-database page table entry, flushed, then took a fatal page fault on the very address it had just mapped, because VTL1's TLB still held the old read-only zero-page translation.
Resolve each flush target to the vCPU of the calling plane (falling back to plane 0 where no such plane vCPU exists — it then has no TLB to flush either) and raise the flush request on that vCPU directly.
--- arch/x86/kvm/hyperv.c | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-)
arch/x86/kvm/hyperv.c +32 -2
@@ -2122,6 +2122,23 @@ static int kvm_hv_hypercall_check_params(struct kvm_vcpu *vcpu,return 0 ; } +/* +* A TLB flush hypercall flushes the caller's VTL view of the target VPs: +* resolve the target to the vCPU of the caller's plane, which has its own +* roots and TLB tag. Fall back to plane 0 when the plane vCPU does not +* exist (it then has no TLB to flush either). +*/ +static struct kvm_vcpu *hv_flush_plane_vcpu (struct kvm_vcpu *caller, +struct kvm_vcpu *v) +{ +struct kvm_vcpu *pv; + +if (!caller->plane_level) +return v; + pv = v->common->vcpus[caller->plane_level]; +return pv ?: v; +} +static s64 kvm_hv_flush_tlb (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) {struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu);
@@ -2257,12 +2274,18 @@ static s64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)*/ if (all_cpus && !is_guest_mode (vcpu)) {kvm_for_each_vcpu (i, v, kvm) { + v =hv_flush_plane_vcpu (vcpu, v); tlb_flush_fifo =kvm_hv_get_tlb_flush_fifo (v, false);hv_tlb_flush_enqueue (v, tlb_flush_fifo, tlb_flush_entries, hc->rep_cnt); +if (vcpu->plane_level) { +kvm_make_request (KVM_REQ_HV_TLB_FLUSH , v); +kvm_vcpu_kick (v); + } } -kvm_make_all_cpus_request (kvm,KVM_REQ_HV_TLB_FLUSH ); +if (!vcpu->plane_level) +kvm_make_all_cpus_request (kvm,KVM_REQ_HV_TLB_FLUSH ); }else if (!is_guest_mode (vcpu)) {sparse_set_to_vcpu_mask (kvm, sparse_banks, valid_bank_mask, vcpu_mask);
@@ -2270,12 +2293,19 @@ static s64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) v =kvm_get_vcpu (kvm, i);if (!v)continue ; + v =hv_flush_plane_vcpu (vcpu, v); tlb_flush_fifo =kvm_hv_get_tlb_flush_fifo (v, false);hv_tlb_flush_enqueue (v, tlb_flush_fifo, tlb_flush_entries, hc->rep_cnt); +if (vcpu->plane_level) { +kvm_make_request (KVM_REQ_HV_TLB_FLUSH , v); +kvm_vcpu_kick (v); + } } -kvm_make_vcpus_request_mask (kvm,KVM_REQ_HV_TLB_FLUSH , vcpu_mask); +if (!vcpu->plane_level) +kvm_make_vcpus_request_mask (kvm,KVM_REQ_HV_TLB_FLUSH , + vcpu_mask); }else {struct kvm_vcpu_hv *hv_v;
Signature · 1 line
2.50.1
RFC PATCH 18/55 KVM: x86/mmu: free obsolete roots of all planes
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
KVM_REQ_MMU_FREE_OBSOLETE_ROOTS lives in the request set shared by all planes of a vCPU. A fast zap (e.g. memslot deletion) invalidates *every* plane's TDP roots and raises the request once per vCPU, but whichever plane enters the guest next consumes the shared bit and frees only its own obsolete roots. The other planes keep an obsolete root with a valid-looking root.hpa, kvm_mmu_reload() does nothing for them, and every page fault bails out in is_page_fault_stale() withRET_PF_RETRY: the vCPU livelocks taking the same EPT violation forever. Observed with a Windows VSM guest spinning at ~18k EPT violations/s after a memslot update during boot, the root obsolete (inv=1 obs=1) and the request already consumed (req=0).
Service every plane twin in kvm_mmu_free_obsolete_roots(). The inactive twins cannot be running (all planes of a vCPU share one thread), so their MMUs are stable.
--- arch/x86/kvm/mmu/mmu.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)
arch/x86/kvm/mmu/mmu.c +17 -2
@@ -6234,8 +6234,23 @@ static void __kvm_mmu_free_obsolete_roots(struct kvm *kvm, struct kvm_mmu *mmu)void kvm_mmu_free_obsolete_roots (struct kvm_vcpu *vcpu) { -__kvm_mmu_free_obsolete_roots (vcpu->kvm, &vcpu->arch.root_mmu); -__kvm_mmu_free_obsolete_roots (vcpu->kvm, &vcpu->arch.guest_mmu); +struct kvm_vcpu *v; +int i; + +/* +* KVM_REQ_MMU_FREE_OBSOLETE_ROOTS is tracked in the request set +* shared by all of the vCPU's planes, so the plane that consumes +* the request is not necessarily the plane whose root is obsolete. +* Service every plane: the inactive twins cannot be running (all +* planes of a vCPU share one thread), so their MMUs are stable. +*/ +for (i =0 ; i <KVM_MAX_PLANES ; i++) { + v = vcpu->common->vcpus[i]; +if (!v) +continue ; +__kvm_mmu_free_obsolete_roots (v->kvm, &v->arch.root_mmu); +__kvm_mmu_free_obsolete_roots (v->kvm, &v->arch.guest_mmu); + } }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_mmu_free_obsolete_roots);
Signature · 1 line
2.50.1
RFC PATCH 19/55 KVM: x86: hyperv: don't arm PV EOI for a higher plane's VP assist page
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
Registering the VP assist page arms the PV EOI / APIC assist optimisation through a gfn_to_hva cache, and that cache refuses ranges carrying RW memory protections in plane 0's view. The Windows secure kernel places its VTL1 assist page in memory it has protected against VTL0, so the MSR write failed and VTL1 had no assist page at all: no VTL entry reasons, no VTL-return register control, no boot.
Register the page without arming PV EOI when the vCPU is in a plane above 0. KVM then never sets the no-EOI-required bit for that plane, so the guest always performs a real EOI; only the fast path is lost.
--- arch/x86/kvm/hyperv.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-)
arch/x86/kvm/hyperv.c +20 -4
@@ -1609,7 +1609,8 @@ static int kvm_hv_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host)if (!(data &HV_X64_MSR_VP_ASSIST_PAGE_ENABLE )) { hv_vcpu->hv_vapic = data; -if (kvm_lapic_set_pv_eoi (vcpu,0 ,0 )) +if (!vcpu->plane_level && +kvm_lapic_set_pv_eoi (vcpu,0 ,0 ))return 1 ;break ; }
@@ -1627,9 +1628,24 @@ static int kvm_hv_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool host)return 1 ; hv_vcpu->hv_vapic = data;kvm_vcpu_mark_page_dirty (vcpu, gfn); -if (kvm_lapic_set_pv_eoi (vcpu, -gfn_to_gpa (gfn) |KVM_MSR_ENABLED , -sizeof (struct hv_vp_assist_page ))) + +/* +* The gfn_to_hva cache behind the PV EOI / APIC assist +* optimization validates against plane 0's memory +* attributes, and a higher plane's VP assist page may +* live on a page the plane protected against plane 0 +* (the Windows secure kernel's does). Don't arm PV EOI +* there: pv_eoi.msr_val stays clear, which keeps +* kvm_hv_assist_page_enabled() false for the plane, so +* every assist-page consumer behind it (PV EOI, nested +* enlightened VMCS, the SVM VP-assist read) is disabled +* and KVM never touches the page's contents. The guest +* always performs a real EOI. +*/ +if (!vcpu->plane_level && +kvm_lapic_set_pv_eoi (vcpu, +gfn_to_gpa (gfn) |KVM_MSR_ENABLED , +sizeof (struct hv_vp_assist_page )))return 1 ;break ; }
Signature · 1 line
2.50.1
RFC PATCH 20/55 KVM: x86: allow KVM_TRANSLATE on plane vCPU fds
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The VMM needs to translate guest virtual addresses in the context of a specific plane to implement HvCallTranslateVirtualAddress, which a higher VTL uses to resolve a lower VTL's buffer addresses. KVM_TRANSLATE already works on any vCPU; permit it on plane fds too.
--- arch/x86/kvm/x86.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kvm/x86.c +1 -0
@@ -6310,6 +6310,7 @@ bool kvm_arch_is_vcpu_plane_ioctl(unsigned ioctl)case KVM_GET_XSAVE2 :case KVM_SET_XSAVE :case KVM_GET_REG_LIST : +case KVM_TRANSLATE :return true;default :return false;
Signature · 1 line
2.50.1
RFC PATCH 21/55 KVM: x86: hyper-v: handle VTL call/return in the kernel
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
Every VTL transition exits to userspace, which performs the switch with a dozen cross-thread vCPU ioctls. A Windows guest boots VSM with hundreds of thousands of transitions, so boot takes *minutes* of wall-clock time in switch overhead alone.
Handle HvCallVtlCall and HvCallVtlReturn in kvm_hv_hypercall() when userspace opts in with KVM_CAP_HYPERV_VTL_SWITCH. The switch copies the VTL-shared state directly between the plane vCPUs (GPRs except RSP, CR2), maintains the entered VTL's assist-page control area (entry reason, caller's RAX/RCX), and restores RAX/RCX on a non-fast VTL return. The suspended plane keeps its deferred hypercall completion pending, RIP still at the vmcall, exactly as userspace-driven switching leaves it.
The shared guest FPU is carried across in the plane run loop, once the outgoing plane's FPU has been saved, by copying the legacy FXSAVE area between the twins. The source's FP and SSE XSTATE_BV bits are propagated set-or-clear: when those components are in init state, XSAVES skipped them and the vzalloc'ed legacy area is zeroes, and blindly marking them valid would load MXCSR=0 (every SIMD exception unmasked) and FCW=0 into the other VTL as architectural state.
Transitions the kernel cannot handle (no twin plane vCPU) still go to userspace, with the vmcall retired and the completion dropped before the exit. A completion left armed across the suspension fires on whatever entry next resumes the plane — after a forced higher-VTL entry that is an arbitrary interrupted instruction, which must not have RAX overwritten and RIP skipped. The reference hypervisor likewise retires the instruction before switching, which also makes the suspended RIP a higher VTL observes through GetVpRegisters point past the vmcall. An hv_vtl_switch_limit module parameter caps how many transitions the kernel handles, with the rest forwarded; that is the bisection knob for behavioural divergence between the two switch paths.
Also report the originating plane of every forwarded hypercall in a previously-padding field of struct kvm_hyperv_exit, so userspace can tell which VTL issued a hypercall without tracking every switch itself.
--- Documentation/virt/kvm/api.rst | 59 +++++++++- arch/x86/include/asm/kvm_host.h | 14 +++ arch/x86/kvm/hyperv.c | 193 ++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 69 ++++++++++++ include/hyperv/hvgdk_mini.h | 2 + include/linux/kvm_host.h | 21 ++++ include/uapi/linux/kvm.h | 3 +- virt/kvm/kvm_main.c | 1 + 8 files changed, 360 insertions(+), 2 deletions(-)
Documentation/virt/kvm/api.rst +58 -1
@@ -7126,8 +7126,10 @@ EOI was received. #define KVM_EXIT_HYPERV_SYNIC 1 #define KVM_EXIT_HYPERV_HCALL 2 #define KVM_EXIT_HYPERV_SYNDBG 3 + #define KVM_EXIT_HYPERV_VTL_INTERRUPT 4 + #define KVM_EXIT_HYPERV_REG_INTERCEPT 5 __u32 type; - __u32 pad1; + __u32 plane; union { struct { __u32 msr;
@@ -7141,6 +7143,15 @@ EOI was received. __u64 result; __u64 params[2]; } hcall; + struct { + __u32 reg; + __u8 insn_len; + __u8 pad[3]; + __u64 base; + __u32 limit; + __u32 pad2; + __u64 rip; + } reg_intercept; struct { __u32 msr; __u32 pad2;
@@ -7158,6 +7169,11 @@ EOI was received. Indicates that the VCPU exits into userspace to process some tasks related to Hyper-V emulation. +'plane' identifies the plane the exit concerns: the plane of the vCPU +that triggered the exit, or for KVM_EXIT_HYPERV_VTL_INTERRUPT the +higher plane to be entered. On kernels without plane support the +field was padding and reads as zero. + Valid values for 'type' are: - KVM_EXIT_HYPERV_SYNIC -- synchronously notify user-space about
@@ -7172,6 +7188,25 @@ Hyper-V Synthetic debugger state change. Notification is used to either update the pending_page location or to send a control command (send the buffer located in send_page or recv a buffer to recv_page). + - KVM_EXIT_HYPERV_VTL_INTERRUPT -- the plane named by 'plane' has a + +pending event that preempts the current, lower plane (TLFS: higher VTL +interrupts preempt lower VTLs). The VMM is expected to perform a VTL +entry into that plane. Delivered only while in-kernel VTL switching +(KVM_CAP_HYPERV_VTL_SWITCH, see section 7.50) is disabled, and +single-shot: if the VMM declines the entry, the event waits in the +target plane for a cooperative entry rather than re-triggering the +exit. + + - KVM_EXIT_HYPERV_REG_INTERCEPT -- a descriptor-table write that a + +higher plane intercepted (KVM_CAP_HYPERV_REG_INTERCEPT) has been +diverted to userspace instead of being applied. 'reg_intercept' +carries the HV_X64_REGISTER_* id of the written register, the +descriptor-table base and limit the guest tried to load, and the RIP +and length of the writing instruction. The vCPU's RIP has been +rewound to that instruction; the higher plane owns its completion. + :: /* KVM_EXIT_ARM_NISV / KVM_EXIT_ARM_LDST64B */
@@ -9064,6 +9099,28 @@ For Hyper-V, unknown hypercalls are passed to userspace without checking their parameter pages. The fault GPA is in the physical address space of the VM managed by userspace, after nested GPA translation. +7.50 KVM_CAP_HYPERV_VTL_SWITCH +------------------------------ + +:Architectures: x86 +:Target: VM +:Parameters: args[0] is 1 to enable in-kernel VTL switches, 0 to disable them + +When enabled, KVM handles the non-fast forms of the HvCallVtlCall and +HvCallVtlReturn hypercalls itself where it can: the calling plane's +vCPU is suspended mid-hypercall and its twin in the target plane +resumed, with the VTL-shared register state (GPRs except RSP, CR2, FPU) +carried across and the entered VTL's assist page updated as the TLFS +specifies. A transition KVM cannot handle (a fast-form call, no twin +vCPU in the target plane, a target that cannot be entered) falls back +to the KVM_CAP_HYPERV_FORWARD_HCALL path, so a VMM enabling this +capability must still implement the userspace switch and should enable +forwarding too. + +While this capability is disabled, a pending event for a higher plane +instead surfaces as a KVM_EXIT_HYPERV_VTL_INTERRUPT exit (see section +5) telling the VMM to enter that plane. + 8. Other capabilities. ======================
arch/x86/include/asm/kvm_host.h +14 -0
@@ -731,6 +731,14 @@ struct kvm_vcpu_hv {u64 vm_id;u32 vp_id; } nested; + +/* +* RAX/RCX of the lower VTL captured at VTL entry, restored on a +* non-fast VTL return when this plane has no VP assist page to +* carry them in its VTL control area. +*/ +u64 vtl_entry_rax; +u64 vtl_entry_rcx; };struct kvm_hypervisor_cpuid {
@@ -1202,6 +1210,12 @@ enum hv_tsc_page_status {/* Hyper-V emulation context */ struct kvm_hv {struct mutex hv_lock; +bool hv_inkernel_vtl_switch; +/* +* In-kernel VTL switches performed by this VM, counted against the +* hv_vtl_switch_limit debug knob; saturates instead of wrapping. +*/ +atomic_t vtl_switch_count;/* * GUEST_OS_ID and the hypercall page are per-VTL instances in the * TLFS; bank them per plane so a higher VTL enabling its own
arch/x86/kvm/hyperv.c +193 -0
211 lines not on this page · raw message
arch/x86/kvm/x86.c +69 -0
99 lines not on this page · raw message
include/hyperv/hvgdk_mini.h +2 -0
@@ -482,6 +482,8 @@ union hv_vp_assist_msr_contents { /* HV_REGISTER_VP_ASSIST_PAGE */#define HVCALL_ASSERT_VIRTUAL_INTERRUPT 0x0094#define HVCALL_CREATE_PORT 0x0095#define HVCALL_CONNECT_PORT 0x0096 +#define HVCALL_VTL_CALL 0x0011 +#define HVCALL_VTL_RETURN 0x0012#define HVCALL_START_VP 0x0099#define HVCALL_GET_VP_INDEX_FROM_APIC_ID 0x009a#define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af
include/linux/kvm_host.h +21 -0
@@ -338,6 +338,16 @@ struct kvm_vcpu_common {/* Currently active VCPU */ struct kvm_vcpu *current_vcpu; +#ifdef CONFIG_KVM_HYPERV +/* +* An in-kernel VTL switch moved execution away from this plane +* (-1: none pending); the run loop copies the (VTL-shared) guest +* FPU state from it when it selects the new plane, after the old +* one's FPU has been saved. +*/ +int vtl_fpu_from; +#endif +struct kvm_vcpu *vcpus[KVM_MAX_PLANES ];/* Locks */
@@ -458,6 +468,17 @@ struct kvm_vcpu {unsigned plane_level; }; +/* +* Cancel (or initialise away) a pending in-kernel VTL switch FPU carry. +* A no-op on configurations without the Hyper-V VTL switch. +*/ +static inline void kvm_vcpu_cancel_vtl_fpu_carry (struct kvm_vcpu_common *common) +{ +#ifdef CONFIG_KVM_HYPERV + common->vtl_fpu_from =-1 ; +#endif +} +void kvm_vcpu_set_plane_runnable (struct kvm_vcpu *vcpu);void kvm_vcpu_set_plane_stopped (struct kvm_vcpu *vcpu);struct kvm_vcpu *kvm_vcpu_select_plane (struct kvm_vcpu *vcpu);
include/uapi/linux/kvm.h +2 -1
@@ -100,7 +100,7 @@ struct kvm_hyperv_exit {#define KVM_EXIT_HYPERV_HCALL 2#define KVM_EXIT_HYPERV_SYNDBG 3__u32 type; -__u32 pad1; +__u32 plane; union { struct { __u32 msr;
@@ -1025,6 +1025,7 @@ struct kvm_enable_cap {*/ #define KVM_HV_HCALL_RESULT_RAISE_UD (1ULL << 63)#define KVM_CAP_HCALL_FAULT_EXIT 252 +#define KVM_CAP_HYPERV_VTL_SWITCH 253struct kvm_irq_routing_irqchip {__u32 irqchip;
virt/kvm/kvm_main.c +1 -0
@@ -474,6 +474,7 @@ static int kvm_vcpu_init_common(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned goto out_drop_counter; } common->run =page_address (page); +kvm_vcpu_cancel_vtl_fpu_carry (common);mutex_init (&common->mutex);
Signature · 1 line
2.50.1
RFC PATCH 22/55 KVM: x86: load each plane twin while updating it after KVM_SET_CPUID
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
kvm_vcpu_ioctl_set_cpuid() runs kvm_vcpu_after_set_cpuid() on every plane twin, but only the calling vCPU is loaded. The vendor callback writes VMCS/VMCB controls, and on VMX those writes go to whatever VMCS is loaded on the current pCPU: the caller's, not the twin's. The twin's controls shadow is updated anyway, so the stale values in its real VMCS can never be corrected later; a twin created before KVM_SET_CPUID2 keeps its empty-CPUID controls (RDSEED/RDRAND exiting among them), and the guest's higher VTL takes a spurious #UD on its first RDSEED.
Swap the twin in as the loaded vCPU around its update. Plane ioctls all share the common vCPU mutex, so the twin cannot be loaded elsewhere concurrently.
--- arch/x86/kvm/cpuid.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
arch/x86/kvm/cpuid.c +13 -0
@@ -592,7 +592,20 @@ static int kvm_set_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid_entry2 *e2,#ifdef CONFIG_KVM_XEN v->arch.xen.cpuid =kvm_get_hypervisor_cpuid (vcpu,XEN_SIGNATURE );#endif +/* +* The ioctl loaded the calling vCPU; updating a twin's +* VMCS/VMCB controls requires that twin to be the loaded +* one, so swap it in around the update. +*/ +if (v != vcpu) { +vcpu_put (vcpu); +vcpu_load (v); + }kvm_vcpu_after_set_cpuid (v); +if (v != vcpu) { +vcpu_put (v); +vcpu_load (vcpu); + } } success:
Signature · 1 line
2.50.1
RFC PATCH 23/55 WIP: userspace VTL interrupt entry and register intercepts
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
Windows VSM boot stalls and PatchGuard-induced triple faults traced to missing host behaviours, all confirmed against the reference VSM host:
- A pending interrupt for a suspended higher VTL must preempt the lower VTL. In userspace-switch mode, report it as a new KVM_EXIT_HYPERV_VTL_INTERRUPT exit — from the halted/block path too, since a halted lower VTL never reaches vcpu_enter_guest() — so userspace enters the higher VTL itself and its view of the active VTL stays in sync. kvm_higher_plane_with_event() tests raw pending-ness (kvm_apic_has_interrupt), because a suspended plane carries a stale apicv_active that makes kvm_cpu_has_injectable_intr() report false.
- The in-kernel forced entry, like the in-kernel VTL switch, writes the entered VTL's control area through the *entered* plane's context, since the assist page may be protected against the lower VTL; on failure a VTL call forwards to userspace and a forced entry stays pending. The refusal conditions live in kvm_hv_vtl_entry_blocked(), which also gates kvm_higher_plane_with_event() in in-kernel mode: an armed event on a plane that cannot be entered (never run, stopped by userspace, parked on a non-hypercall completion owning the shared run union) must not keep waking a halted lower VTL that has no way to consume it. A plane which has never entered the guest holds reset state and userspace owns the HvCallEnableVpVtl tracking (the reference host fails such calls with HV_STATUS_INVALID_VP_STATE), so hand it the switch and leave the event pending. The resume value is guest-derived in every switch flavour, so KVM_HV_HCALL_RESULT_RAISE_UD is masked out of it, and a target with no completion armed receives it directly in RAX, as the TLFS register-restore contract on VTL return requires.
- RAX/RCX of the lower VTL are captured at in-kernel VTL entry and restored on an assist-less, non-fast return; vtl_entry_regs_valid tracks whether the capture is live. A return finding it clear (the entry was performed by userspace, or predates a migration — the capture has no GET/SET exposure) is forwarded to userspace, which owns restoring the lower VTL's registers.
- plane_user_stopped is a pure userspace-policy latch: only KVM_SET_PLANE_RUN_STATE writes it, and no runnable plane can carry it (it is cleared at the single RUNNABLE choke point). In-kernel suspensions get their own plane_kernel_stopped flag, reported and restorable as KVM_PLANE_RUN_STATE_KERNEL_STOPPED, with the same plane-0 event gating. The in-kernel VTL switch and the forced entry refuse a user-stopped target (forwarding the switch to userspace or leaving the event pending), and the kvm_vcpu_select_plane() fallback refuses a user-stopped plane 0: KVM_RUN fails with -EINVAL, surfacing the VMM's ordering bug instead of resuming a plane the VMM parked mid-hypercall.
- The secure kernel arms HvRegisterCrInterceptControl descriptor-table write intercepts for VTL0, then PatchGuard runs its protected patching window under an intercepted lidt. The write must *not* be applied — the window would execute with a genuinely null IDT and the first event triple-faults the guest. Add a per-vCPU KVM_CAP_HYPERV_REG_INTERCEPT that arms the control word, trap the descriptor-table instructions (SECONDARY_EXEC_DESC, dropped again when the intercepts disarm), and divert armed writes into a KVM_EXIT_HYPERV_REG_INTERCEPT exit with RIP rewound to the intercepted instruction: the higher VTL owns completion, and Windows' patch-protection probe matches its reflected #GP(0) against the probe RIP.
- HvCallSendSyntheticClusterIpi(Ex) honours the input's in_vtl field (both the memory and fast forms carry it after the vector), delivering to the resolved plane's twins; reserved bits 7:5 are rejected with HV_STATUS_INVALID_HYPERCALL_INPUT before anything is delivered, as on the reference host.
- hv_inkernel_vtl_switch is written under kvm->lock but read locklessly from vCPU context; the accesses carry WRITE_ONCE/READ_ONCE so each individual check reads a coherent value. A VMM that flips the cap while vCPUs run owns the consequences of catching a transition in flight.
Also includes debug instrumentation, to be stripped before posting.
--- Documentation/virt/kvm/api.rst | 31 ++-- arch/x86/hyperv/hv_apic.c | 3 +- arch/x86/include/asm/kvm_host.h | 21 ++- arch/x86/kvm/hyperv.c | 257 +++++++++++++++++++++++++++++--- arch/x86/kvm/hyperv.h | 3 + arch/x86/kvm/vmx/vmx.c | 23 ++- arch/x86/kvm/x86.c | 210 +++++++++++++++++++++++++- include/hyperv/hvgdk_mini.h | 3 +- include/hyperv/hvhdk_mini.h | 3 +- include/linux/kvm_host.h | 12 ++ include/uapi/linux/kvm.h | 13 ++ virt/kvm/kvm_main.c | 35 ++++- 12 files changed, 571 insertions(+), 43 deletions(-)
Documentation/virt/kvm/api.rst +22 -9
@@ -6628,19 +6628,32 @@ a vCPU fd of every plane, not just plane 0. :Returns: 0 on success, <0 on error Reads or sets the per-plane run state of a vCPU: -KVM_PLANE_RUN_STATE_STOPPED (0) or KVM_PLANE_RUN_STATE_RUNNABLE (1). +KVM_PLANE_RUN_STATE_STOPPED (0), KVM_PLANE_RUN_STATE_RUNNABLE (1) or +KVM_PLANE_RUN_STATE_KERNEL_STOPPED (2). When a vCPU thread calls KVM_RUN, KVM executes the lowest-numbered plane -whose vCPU is runnable, falling back to plane 0. A VMM implementing a -software privilege model (such as Hyper-V VSM) switches planes by -marking the target plane's vCPU runnable and the current one stopped -before re-entering KVM_RUN. Pending events for plane 0 make it runnable -again on their own only as long as userspace has not explicitly stopped -it with this ioctl; a VMM-stopped plane 0 keeps its events pending until -the VMM makes it runnable, so a software privilege model can keep a -higher plane running undisturbed. The ioctl is accepted on any plane's +whose vCPU is runnable, falling back to plane 0. If no plane is +runnable and userspace has stopped plane 0, KVM_RUN fails with -EINVAL +instead of overriding the stop. A VMM implementing a software privilege +model (such as Hyper-V VSM) switches planes by marking the target +plane's vCPU runnable and the current one stopped before re-entering +KVM_RUN. Pending events for plane 0 make it runnable again on their own +only as long as userspace has not explicitly stopped it with this ioctl; +a VMM-stopped plane 0 keeps its events pending until the VMM makes it +runnable, so a software privilege model can keep a higher plane running +undisturbed. In-kernel plane transitions never override a userspace +stop: a guest-initiated VTL switch targeting a stopped plane is +forwarded to userspace instead. The ioctl is accepted on any plane's vCPU fd. +KVM_PLANE_RUN_STATE_KERNEL_STOPPED is reported for a plane that KVM +itself suspended (such as the lower plane of an in-kernel VTL switch); +such a plane becomes runnable again on the matching in-kernel transition +without userspace involvement. Setting it stops the plane without +recording a userspace stop, so restoring previously read run states +round-trips a kernel suspension without converting it into userspace +stop policy. + The run state is distinct from KVM_GET/SET_MP_STATE, which describes the architectural state of a vCPU within its plane.
arch/x86/hyperv/hv_apic.c +2 -1
@@ -127,7 +127,8 @@ static bool __send_ipi_mask_ex(const struct cpumask *mask, int vector, goto ipi_mask_ex_done; ipi_arg->vector = vector; - ipi_arg->reserved =0 ; + ipi_arg->in_vtl.as_uint8 =0 ; +memset (ipi_arg->reserved,0 ,sizeof (ipi_arg->reserved)); ipi_arg->vp_set.valid_bank_mask =0 ;/*
arch/x86/include/asm/kvm_host.h +20 -1
@@ -732,13 +732,32 @@ struct kvm_vcpu_hv {u32 vp_id; } nested; +/* +* HvRegisterCrInterceptControl value the higher VTL armed for the +* lower VTL (userspace VSM): descriptor-table write bits divert +* the emulated write into a KVM_EXIT_HYPERV_REG_INTERCEPT exit. +*/ +u64 reg_intercept_control; +struct { +u32 reg; +u8 insn_len; +bool pending; +u64 base; +u32 limit; +u64 rip; + } reg_intercept; +/* * RAX/RCX of the lower VTL captured at VTL entry, restored on a * non-fast VTL return when this plane has no VP assist page to -* carry them in its VTL control area. +* carry them in its VTL control area. Only valid between an +* in-kernel VTL entry and the matching return: a plane entered by +* userspace, or restored from migration, has no captured values +* and its assist-less return is forwarded to userspace. */ u64 vtl_entry_rax;u64 vtl_entry_rcx; +bool vtl_entry_regs_valid; };struct kvm_hypervisor_cpuid {
arch/x86/kvm/hyperv.c +238 -19
371 lines not on this page · raw message
arch/x86/kvm/hyperv.h +3 -0
@@ -102,6 +102,9 @@ static inline bool kvm_hv_hypercall_enabled(struct kvm_vcpu *vcpu) }int kvm_hv_hypercall (struct kvm_vcpu *vcpu); +bool kvm_hv_vtl_interrupt_entry (struct kvm_vcpu *vcpu, +struct kvm_vcpu *target); +bool kvm_hv_vtl_entry_blocked (struct kvm_vcpu *target);void kvm_hv_irq_routing_update (struct kvm *kvm); int kvm_hv_synic_set_irq(struct kvm_kernel_irq_routing_entry *e, struct kvm *kvm,
arch/x86/kvm/vmx/vmx.c +22 -1
@@ -5640,7 +5640,14 @@ static int handle_desc(struct kvm_vcpu *vcpu)*/ BUILD_BUG_ON (KVM_POSSIBLE_CR4_GUEST_BITS &X86_CR4_UMIP ); -WARN_ON_ONCE (!kvm_is_cr4_bit_set (vcpu,X86_CR4_UMIP )); +/* +* Descriptor-table exits are also armed when a higher VTL registers +* DT intercepts (reg_intercept_control); only an exit with neither +* UMIP emulation nor an armed intercept indicates a stale +* SECONDARY_EXEC_DESC control. +*/ +WARN_ON_ONCE (!kvm_is_cr4_bit_set (vcpu,X86_CR4_UMIP ) && + !(to_hv_vcpu (vcpu) &&to_hv_vcpu (vcpu)->reg_intercept_control));return kvm_emulate_instruction (vcpu,0 ); }
@@ -8044,6 +8051,20 @@ void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)/* Refresh #PF interception to account for MAXPHYADDR changes. */ vmx_update_exception_bitmap (vcpu); + +/* +* Trap descriptor-table instructions when a higher VTL armed +* register intercepts on them, and stop trapping when the +* intercepts are disarmed. Preserve the control when UMIP +* emulation needs it (CR4.UMIP owns it then). +*/ +if (vmcs_config.cpu_based_2nd_exec_ctrl &SECONDARY_EXEC_DESC ) { +if (to_hv_vcpu (vcpu) &&to_hv_vcpu (vcpu)->reg_intercept_control) +secondary_exec_controls_setbit (vmx,SECONDARY_EXEC_DESC ); +else if (!(vmx_umip_emulated () && +kvm_is_cr4_bit_set (vcpu,X86_CR4_UMIP ))) +secondary_exec_controls_clearbit (vmx,SECONDARY_EXEC_DESC ); + } }static __init u64 vmx_get_perf_capabilities (void )
arch/x86/kvm/x86.c +206 -4
283 lines not on this page · raw message
include/hyperv/hvgdk_mini.h +2 -1
@@ -1346,7 +1346,8 @@ struct hv_input_set_vp_registers {/* HvCallSendSyntheticClusterIpi hypercall */ struct hv_send_ipi {/* HV_INPUT_SEND_SYNTHETIC_CLUSTER_IPI */ u32 vector; -u32 reserved; +union hv_input_vtl in_vtl; +u8 reserved[3 ];u64 cpu_mask; } __packed;
include/hyperv/hvhdk_mini.h +2 -1
@@ -464,7 +464,8 @@ struct hv_input_unmap_device_interrupt {struct hv_send_ipi_ex {/* HV_INPUT_SEND_SYNTHETIC_CLUSTER_IPI_EX */ u32 vector; -u32 reserved; +union hv_input_vtl in_vtl; +u8 reserved[3 ];struct hv_vpset vp_set; } __packed;
include/linux/kvm_host.h +12 -0
@@ -427,8 +427,20 @@ struct kvm_vcpu {* Set when userspace stopped this plane with * KVM_SET_PLANE_RUN_STATE: pending plane-0 events then no longer * make plane 0 runnable on their own, the VMM owns the switch. +* Only the ioctl sets it; in-kernel plane transitions never +* override it, and kvm_vcpu_set_plane_runnable() clears it so a +* runnable plane cannot carry a stale latch. */ bool plane_user_stopped; +/* +* Set when an in-kernel VTL switch suspends this plane: plane-0 +* events are gated exactly as for a userspace stop, but +* KVM_GET_PLANE_RUN_STATE reports the state as +* KVM_PLANE_RUN_STATE_KERNEL_STOPPED so that a save/restore +* round trip does not convert a transient kernel suspension +* into userspace stop policy. +*/ +bool plane_kernel_stopped;/* S390 only */ bool valid_wakeup;
include/uapi/linux/kvm.h +13 -0
@@ -99,6 +99,8 @@ struct kvm_hyperv_exit {#define KVM_EXIT_HYPERV_SYNIC 1#define KVM_EXIT_HYPERV_HCALL 2#define KVM_EXIT_HYPERV_SYNDBG 3 +#define KVM_EXIT_HYPERV_VTL_INTERRUPT 4 +#define KVM_EXIT_HYPERV_REG_INTERCEPT 5__u32 type;__u32 plane;union {
@@ -114,6 +116,15 @@ struct kvm_hyperv_exit {__u64 result;__u64 params[2 ]; } hcall; +struct { +__u32 reg; +__u8 insn_len; +__u8 pad[3 ]; +__u64 base; +__u32 limit; +__u32 pad2; +__u64 rip; + } reg_intercept;struct {__u32 msr;__u32 pad2;
@@ -1026,6 +1037,7 @@ struct kvm_enable_cap {#define KVM_HV_HCALL_RESULT_RAISE_UD (1ULL << 63)#define KVM_CAP_HCALL_FAULT_EXIT 252#define KVM_CAP_HYPERV_VTL_SWITCH 253 +#define KVM_CAP_HYPERV_REG_INTERCEPT 254struct kvm_irq_routing_irqchip {__u32 irqchip;
@@ -1384,6 +1396,7 @@ struct kvm_s390_keyop {#define KVM_SET_PLANE_RUN_STATE _IOW(KVMIO, 0xe6, __u32)#define KVM_PLANE_RUN_STATE_STOPPED 0#define KVM_PLANE_RUN_STATE_RUNNABLE 1 +#define KVM_PLANE_RUN_STATE_KERNEL_STOPPED 2/* * ioctls for vcpu fds
virt/kvm/kvm_main.c +29 -6
@@ -5023,9 +5023,14 @@ static long kvm_vcpu_ioctl(struct file *filp,break ; }case KVM_GET_PLANE_RUN_STATE : { -u32 state = vcpu->plane_state ==RUNNABLE ? -KVM_PLANE_RUN_STATE_RUNNABLE : -KVM_PLANE_RUN_STATE_STOPPED ; +u32 state; + +if (vcpu->plane_state ==RUNNABLE ) + state =KVM_PLANE_RUN_STATE_RUNNABLE ; +else if (vcpu->plane_user_stopped) + state =KVM_PLANE_RUN_STATE_STOPPED ; +else + state =KVM_PLANE_RUN_STATE_KERNEL_STOPPED ; r = -EINVAL ;if (kvm_arch_max_planes (vcpu->kvm) <=1 )
@@ -5052,13 +5057,15 @@ static long kvm_vcpu_ioctl(struct file *filp,if (copy_from_user (&state, argp,sizeof (state))) goto out; r = -EINVAL ; -if (state >KVM_PLANE_RUN_STATE_RUNNABLE ) +if (state >KVM_PLANE_RUN_STATE_KERNEL_STOPPED ) goto out;if (state ==KVM_PLANE_RUN_STATE_RUNNABLE ) { - vcpu->plane_user_stopped = false;kvm_vcpu_set_plane_runnable (vcpu); }else { - vcpu->plane_user_stopped = true; + vcpu->plane_user_stopped = + state ==KVM_PLANE_RUN_STATE_STOPPED ; + vcpu->plane_kernel_stopped = + state ==KVM_PLANE_RUN_STATE_KERNEL_STOPPED ;kvm_vcpu_set_plane_stopped (vcpu); } r =0 ;
@@ -5319,6 +5326,13 @@ static struct file_operations kvm_plane_fops = {void kvm_vcpu_set_plane_runnable (struct kvm_vcpu *vcpu) { +/* +* A runnable plane must not carry a stale stop latch: the latches +* gate the plane-0 event wakeup, and KVM_GET_PLANE_RUN_STATE could +* not report the inconsistency. +*/ + vcpu->plane_user_stopped = false; + vcpu->plane_kernel_stopped = false; vcpu->plane_state =RUNNABLE ; vcpu->common->plane_switch = true;kvm_make_request (KVM_REQ_PLANE_RESCHED , vcpu);
@@ -5349,7 +5363,16 @@ struct kvm_vcpu *kvm_vcpu_select_plane(struct kvm_vcpu *vcpu) }if (ret ==NULL ) { +/* +* Never force a userspace-stopped plane 0 to run: it may be +* suspended mid-hypercall, and the VMM owns restarting it. +* KVM_RUN fails instead, surfacing the VMM's ordering bug. +*/ ret = common->vcpus[0 ]; +if (ret->plane_user_stopped) +return NULL ; + + ret->plane_kernel_stopped = false; ret->plane_state =RUNNABLE ; }
Signature · 1 line
2.50.1
Re: [RFC PATCH 23/55] WIP: userspace VTL interrupt entry and register intercepts
Hi David,
I hit a Windows VBS boot failure involving hv_divert_dt_write() when OVMF was executing in SMM.
The helper was diverting firmware descriptor-table writes into the VTL register-intercept path. The failure trace showed a reflected #GP being delivered in an SMM context with IDT base and limit both zero, followed by a triple fault and guest reset.
Excluding SMM from this helper lets those firmware descriptor-table updates proceed normally, without changing interception for ordinary VTL0 execution:
arch/x86/kvm/x86.c +1 -1
@@ -5884,7 +5884,7 @@ static bool hv_divert_dt_write(struct x86_emulate_ctxt *ctxt, u32 hv_reg,struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu);if (!hv_vcpu || vcpu->plane_level ||is_guest_mode (vcpu) || - !(hv_vcpu->reg_intercept_control & ctl_bit)) +is_smm (vcpu) || !(hv_vcpu->reg_intercept_control & ctl_bit))return false; hv_vcpu->reg_intercept.reg = hv_reg;
The context above includes the is_guest_mode() check added later in the series by patch 50.
With this and the accompanying integration/QEMU fixes, msinfo32 reports VBS running. This was an integrated v7.3-based tree, not a test of patch 23 in isolation.
Thanks,
Sriram
RFC PATCH 24/55 WIP: KVM: x86: deliver events to suspended higher planes
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
A suspended plane's events must preempt the lower plane running on the same vCPU, but nothing told the running twin:
- An interrupt, local-APIC timer or Hyper-V stimer landing on a stopped twin only kicked the twin itself — a no-op while it is parked. Kick the lower twins too, with KVM_REQ_UNBLOCK so a sibling already past its event checks aborts the in-flight vmentry.
- A parked plane's lapic timer rode the VMX preemption timer, which only counts in guest mode, so it never fired while the plane was suspended. Hand it to the hrtimer when a non-zero plane's KVM_RUN ends.
- KVM_REQ_HV_STIMER was a shared request, so the running plane consumed the twin's request without processing the twin's stimers. Make it per-plane.
- kvm_higher_plane_with_event() now also reports a pending-but-not-injected apic timer.
- Suppress KVM_EXIT_HYPERV_SYNIC for non-zero planes: userspace's SynIC device models plane 0 only, and treated a twin's SIMP move as a plane-0 overlay relocation, carrying stale overlay contents to the new GPA.
--- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/hyperv.c | 27 +++++++++++++++++++ arch/x86/kvm/lapic.c | 46 ++++++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 19 ++++++++++++-- 4 files changed, 90 insertions(+), 4 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -1
@@ -112,7 +112,7 @@#define KVM_REQ_IOAPIC_EOI_EXIT KVM_ARCH_PLANE_REQ(19)#define KVM_REQ_HV_RESET KVM_ARCH_REQ(20)#define KVM_REQ_HV_EXIT KVM_ARCH_REQ(21) -#define KVM_REQ_HV_STIMER KVM_ARCH_REQ(22) +#define KVM_REQ_HV_STIMER KVM_ARCH_PLANE_REQ(22)#define KVM_REQ_LOAD_EOI_EXITMAP KVM_ARCH_REQ(23)#define KVM_REQ_GET_NESTED_STATE_PAGES KVM_ARCH_REQ(24)#define KVM_REQ_APICV_UPDATE \
arch/x86/kvm/hyperv.c +27 -0
@@ -247,6 +247,15 @@ static void synic_exit(struct kvm_vcpu_hv_synic *synic, u32 msr)struct kvm_vcpu *vcpu =hv_synic_to_vcpu (synic);struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); +/* +* Userspace's SynIC device models plane 0 only; a higher plane's +* exit would be mistaken for a plane-0 page move and relocate the +* overlay the VMM keeps for SIMP/SIEFP. Higher-plane SynICs are +* handled fully in-kernel. +*/ +if (vcpu->plane_level) +return ; + hv_vcpu->exit.type =KVM_EXIT_HYPERV_SYNIC ; hv_vcpu->exit.u.synic.msr = msr; hv_vcpu->exit.u.synic.control = synic->control;
@@ -598,6 +607,24 @@ static void stimer_mark_pending(struct kvm_vcpu_hv_stimer *stimer,kvm_make_request (KVM_REQ_HV_STIMER , vcpu);if (vcpu_kick)kvm_vcpu_kick (vcpu); + +/* +* A pending stimer on a suspended higher plane is only processed +* once that plane runs; kick the lower twins so the running one +* re-evaluates pending higher-plane events and forces entry. +*/ +if (vcpu_kick && vcpu->plane_level) { +struct kvm_vcpu *twin; +int p; + +for (p =0 ; p < vcpu->plane_level; p++) { + twin = vcpu->common->vcpus[p]; +if (twin) { +kvm_make_request (KVM_REQ_UNBLOCK , twin); +kvm_vcpu_kick (twin); + } + } + } }static void stimer_cleanup (struct kvm_vcpu_hv_stimer *stimer)
arch/x86/kvm/lapic.c +45 -1
64 lines not on this page · raw message
arch/x86/kvm/x86.c +17 -2
@@ -11226,7 +11226,9 @@ static struct kvm_vcpu *kvm_higher_plane_with_event(struct kvm_vcpu *vcpu)*/ if (kvm_cpu_has_extint (higher) ||kvm_apic_has_interrupt (higher) !=-1 || -kvm_test_request (KVM_REQ_HV_STIMER , higher)) +kvm_test_request (KVM_REQ_HV_STIMER , higher) || + (higher->arch.apic && +atomic_read (&higher->arch.apic->lapic_timer.pending)))return higher; }return NULL ;
@@ -11336,8 +11338,12 @@ void __kvm_vcpu_update_apicv(struct kvm_vcpu *vcpu)down_read (&vcpu->kvm->arch.apicv_update_lock);preempt_disable (); -/* Do not activate APICV when APIC is disabled */ +/* +* Do not activate APICV when APIC is disabled, nor on higher-plane +* twins (see kvm_create_lapic). +*/ activate =kvm_vcpu_apicv_activated (vcpu) && + !vcpu->plane_level && (kvm_get_apic_mode (vcpu) !=LAPIC_MODE_DISABLED );if (apic->apicv_active == activate)
@@ -12566,6 +12572,15 @@ static int __kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)post_kvm_run_save (vcpu);kvm_vcpu_srcu_read_unlock (vcpu); +/* +* A non-zero plane may stay suspended indefinitely after this run +* ends, and its hv timer only counts in VMX non-root mode. Hand +* the lapic timer to the hrtimer so it keeps firing while the +* plane is parked; firing is what forces re-entry into the plane. +*/ +if (vcpu->plane_level &&kvm_lapic_hv_timer_in_use (vcpu)) +kvm_lapic_switch_to_sw_timer (vcpu); +kvm_sigset_deactivate (vcpu);vcpu_put (vcpu);return r;
Signature · 1 line
2.50.1
RFC PATCH 25/55 WIP: KVM: per-plane memory overlays, richer protection-fault exits
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The pieces the VSM contract tests need:
- KVM_SET_PLANE_MEMORY_OVERLAY maps one page of userspace memory over a GPA for a single plane. TLFS overlay pages (SynIC SIMP in particular) show each VTL a private page while the other VTLs keep seeing the guest RAM underneath; per-plane EPT roots make a per-plane pfn override in the fault path sufficient. Mappings for a plane with overlays stay 4KiB so a neighbouring fault's huge page cannot span an overlaid gfn, and kvm_tdp_mmu_recover_huge_pages() skips nonzero-plane roots so a later promotion cannot widen the plane's attribute or overlay clamps back out (planes keep no lpage tracking by design; their fault path rebuilds each mapping at the largest level the plane allows). The overlay update runs under the kvm_mmu_invalidate_begin()/end() protocol with the gfn registered, as the mmu notifiers do, so a concurrent lockless fault cannot install an SPTE for the old backing after the zap.
- SynIC message delivery reads and writes the overlay through the same lookup, and fails with -EFAULT when a higher plane's SIMP page has no overlay installed yet, rather than falling through to kvm_vcpu_write_guest_page() and corrupting underlay RAM that plane 0 owns; the sender retries once the overlay appears. A non-zero plane's SynIC exits carry the plane instead of being suppressed, so the VMM can keep its plane-0 device state separate.
- A plane-protection fault reflected as KVM_EXIT_MEMORY_FAULT now carries the guest-linear address and the faulting instruction's length and bytes (via VM_EXIT_INSTRUCTION_LEN and a guest-virt read), which the TLFS GPA-intercept message requires. LLDT joins LGDT/LIDT in the descriptor-write divert, reporting the written selector.
--- Documentation/virt/kvm/api.rst | 62 ++++++++ arch/x86/kvm/hyperv.c | 82 +++++++--- arch/x86/kvm/mmu/mmu.c | 29 +++- arch/x86/kvm/mmu/tdp_mmu.c | 23 ++- arch/x86/kvm/vmx/vmx.c | 51 ++++++- arch/x86/kvm/x86.c | 14 ++ include/linux/kvm_host.h | 17 +++ include/uapi/linux/kvm.h | 17 +++ virt/kvm/kvm_main.c | 271 +++++++++++++++++++++++++++++++++ 9 files changed, 541 insertions(+), 25 deletions(-)
Documentation/virt/kvm/api.rst +62 -0
@@ -6657,6 +6657,38 @@ stop policy. The run state is distinct from KVM_GET/SET_MP_STATE, which describes the architectural state of a vCPU within its plane. +4.149 KVM_SET_PLANE_MEMORY_OVERLAY +---------------------------------- + +:Capability: KVM_CAP_PLANE_MEMORY_OVERLAYS +:Architectures: x86 +:Type: vm ioctl +:Parameters: struct kvm_plane_memory_overlay (in) +:Returns: 0 on success, <0 on error + +:: + + struct kvm_plane_memory_overlay { + __u32 plane; + __u32 flags; + __u64 gfn; + __u64 uaddr; /* page-aligned; 0 removes the overlay */ + }; + +Installs, replaces or removes a memory overlay for a single guest page +frame on one plane. While an overlay is installed, translations of +'gfn' for vCPUs of 'plane' resolve to the (pinned) page at 'uaddr' in +the calling process instead of to the memslot backing; other planes are +unaffected. Hyper-V VSM uses this for per-VTL overlay pages such as +the SynIC message and event-flags pages. + +'flags' must be zero and 'uaddr' must be page-aligned. At most +KVM_CHECK_EXTENSION(KVM_CAP_PLANE_MEMORY_OVERLAYS) distinct gfns can +carry an overlay per plane at any one time; further installs fail with +-ENOSPC until an overlay is removed. An overlay cannot be installed +over a private or guest_memfd-only gfn (-EINVAL). Passing 'uaddr' == 0 +removes the overlay for 'gfn', if any. + 5. The kvm_run structure ========================
@@ -7387,9 +7419,15 @@ spec refer, https://github.com/riscv/riscv-sbi-doc. #define KVM_MEMORY_EXIT_FLAG_WRITE (1ULL << 1) #define KVM_MEMORY_EXIT_FLAG_EXEC (1ULL << 2) #define KVM_MEMORY_EXIT_FLAG_PRIVATE (1ULL << 3) + #define KVM_MEMORY_EXIT_FLAG_GVA_VALID (1ULL << 4) __u64 flags; __u64 gpa; __u64 size; + /* set on plane-protection faults when available */ + __u64 gva; + __u32 insn_len; + __u8 insn_bytes[16]; + __u32 pad; } memory_fault; KVM_EXIT_MEMORY_FAULT indicates the vCPU has encountered a memory fault that
@@ -7402,6 +7440,14 @@ describes properties of the faulting access that are likely pertinent: - KVM_MEMORY_EXIT_FLAG_PRIVATE - When set, indicates the memory fault occurred on a private memory access. When clear, indicates the fault occurred on a shared access. + - KVM_MEMORY_EXIT_FLAG_GVA_VALID - When set, 'gva' holds the faulting guest + virtual address. + +The 'gva', 'insn_len' and 'insn_bytes' fields exist only on kernels +which advertise KVM_CAP_PLANE_MEMORY_OVERLAYS (see section 8.48). On +plane memory-protection faults, 'insn_len' and 'insn_bytes' carry a +best-effort copy of the faulting instruction when KVM could fetch it; +an 'insn_len' of 0 means no instruction bytes are available. Note! KVM_EXIT_MEMORY_FAULT is unique among all KVM exit reasons in that it accompanies a return code of '-1', not '0'! errno will always be set to EFAULT
@@ -9676,6 +9722,22 @@ take care to differentiate between these cases. The presence of this capability indicates that the nested KVM guest can start in ESA mode. +8.48 KVM_CAP_PLANE_MEMORY_OVERLAYS +---------------------------------- + +:Architectures: x86 + +A positive return from KVM_CHECK_EXTENSION(KVM_CAP_PLANE_MEMORY_OVERLAYS) +announces two things: + + - The KVM_SET_PLANE_MEMORY_OVERLAY vm ioctl (section 4.149). The + returned value is the number of gfns which can carry an overlay + simultaneously on each plane. + + - The extended KVM_EXIT_MEMORY_FAULT layout: 'memory_fault' carries + the additional 'gva', 'insn_len' and 'insn_bytes' fields, with + KVM_MEMORY_EXIT_FLAG_GVA_VALID qualifying 'gva' (see section 5). + 9. Known KVM API problems =========================
arch/x86/kvm/hyperv.c +60 -22
118 lines not on this page · raw message
arch/x86/kvm/mmu/mmu.c +28 -1
@@ -4620,7 +4620,13 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu,if (fault->slot->flags &KVM_MEM_READONLY ) fault->host_access &= ~ACC_WRITE_MASK ; - fault->max_level =kvm_max_level_for_order (max_order); +/* +* Only ever narrow max_level: the overlay check in +* __kvm_mmu_faultin_pfn() may already have clamped it to 4KiB, and +* a huge gmem mapping must not span an overlaid gfn. +*/ + fault->max_level =min (fault->max_level, +kvm_max_level_for_order (max_order));return RET_PF_CONTINUE ; }
@@ -4682,8 +4688,29 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) {unsigned int foll = fault->write ?FOLL_WRITE :0 ; +struct page *overlay;bool writable; +/* +* A plane overlay page (VSM SynIC SIMP/SIEFP etc.) replaces the +* memslot translation for this plane only. The page is pinned +* for the overlay's lifetime, so no refcounting here. +*/ +/* +* Keep this plane's mappings at 4KiB while it has overlays: a huge +* mapping established by a neighbouring gfn's fault would span the +* overlaid gfn and bypass the override below. +*/ +if (READ_ONCE (vcpu->plane->nr_overlays)) { + fault->max_level =PG_LEVEL_4K ; + + overlay =kvm_plane_overlay_page (vcpu->plane, fault->gfn); +if (overlay) { + fault->pfn =page_to_pfn (overlay); +return RET_PF_CONTINUE ; + } + } +if (fault->is_private ||kvm_memslot_is_gmem_only (fault->slot))return kvm_mmu_faultin_pfn_gmem (vcpu, fault);
arch/x86/kvm/mmu/tdp_mmu.c +22 -1
@@ -1871,8 +1871,29 @@ void kvm_tdp_mmu_recover_huge_pages(struct kvm *kvm,struct kvm_mmu_page *root;lockdep_assert_held_read (&kvm->mmu_lock); -for_each_valid_tdp_mmu_root_yield_safe (kvm, root, slot->as_id) +for_each_valid_tdp_mmu_root_yield_safe (kvm, root, slot->as_id) { +/* +* kvm_mmu_max_mapping_level() consults only plane-0 state +* (lpage_info and plane-0 private attributes). Nonzero +* planes carry per-plane memory attributes and memory +* overlays whose fault-time 4KiB clamps a huge SPTE would +* undo, extending one page's protections or overlay +* backing over gfns the plane maps differently. Leave +* those roots alone; their fault path rebuilds mappings at +* the largest level the plane allows. +* +* Plane 0 can carry overlays too (kvm->planes[0] always +* exists and KVM_SET_PLANE_MEMORY_OVERLAY accepts plane 0), +* and the same reasoning applies: a recovered huge SPTE +* would either span an overlaid gfn with the memslot +* backing, or huge-align the pinned overlay page's pfn and +* map ~2MiB of unrelated host memory around it. +*/ +if (root->role.plane || +READ_ONCE (kvm->planes[root->role.plane]->nr_overlays)) +continue ;recover_huge_pages_range (kvm, root, slot); + } }/*
arch/x86/kvm/vmx/vmx.c +50 -1
69 lines not on this page · raw message
arch/x86/kvm/x86.c +14 -0
@@ -8858,8 +8858,10 @@ static bool hv_divert_dt_write(struct x86_emulate_ctxt *ctxt, u32 hv_reg,#define HV_CR_INTERCEPT_GDTR_WRITE BIT_ULL(15)#define HV_CR_INTERCEPT_IDTR_WRITE BIT_ULL(16) +#define HV_CR_INTERCEPT_LDTR_WRITE BIT_ULL(17)#define HV_X64_REGISTER_IDTR_ID 0x00070000#define HV_X64_REGISTER_GDTR_ID 0x00070001 +#define HV_X64_REGISTER_LDTR_ID 0x00060006static void emulator_set_gdt (struct x86_emulate_ctxt *ctxt,struct desc_ptr *dt) {
@@ -8947,6 +8949,18 @@ static void emulator_set_segment(struct x86_emulate_ctxt *ctxt, u16 selector, var.unusable = !var.present; var.padding =0 ; +if (seg ==VCPU_SREG_LDTR ) { +struct desc_ptr sel_as_dt = { + .address = selector, + .size =0 , + }; + +/* The intercept message carries the written selector. */ +if (hv_divert_dt_write (ctxt,HV_X64_REGISTER_LDTR_ID , + &sel_as_dt,HV_CR_INTERCEPT_LDTR_WRITE )) +return ; + } +kvm_set_segment (vcpu, &var, seg);return ; }
include/linux/kvm_host.h +17 -0
@@ -927,9 +927,26 @@ struct kvm_plane {struct xarray mem_attrs;#endif +/* +* VSM overlay pages: per-plane gfn -> pinned user page overriding +* the memslot translation for this plane only. Written under +* kvm->slots_lock; readers tolerate a transient stale view, the +* post-update zap makes the EPT coherent. +*/ +#define KVM_PLANE_MAX_OVERLAYS 64 +struct kvm_plane_overlay { +gfn_t gfn; +unsigned long uaddr; +struct page *page; + } overlays[KVM_PLANE_MAX_OVERLAYS ]; +int nr_overlays; +struct kvm_arch_plane arch; }; +struct page *kvm_plane_overlay_page (struct kvm_plane *plane,gfn_t gfn); +unsigned long kvm_plane_overlay_uaddr (struct kvm_plane *plane,gfn_t gfn); +struct kvm {#ifdef KVM_HAVE_MMU_RWLOCK rwlock_t mmu_lock;
include/uapi/linux/kvm.h +17 -0
@@ -480,9 +480,15 @@ struct kvm_run {#define KVM_MEMORY_EXIT_FLAG_WRITE (1ULL << 1)#define KVM_MEMORY_EXIT_FLAG_EXEC (1ULL << 2)#define KVM_MEMORY_EXIT_FLAG_PRIVATE (1ULL << 3) +#define KVM_MEMORY_EXIT_FLAG_GVA_VALID (1ULL << 4)__u64 flags;__u64 gpa;__u64 size; +/* set on plane-protection faults when available */ +__u64 gva; +__u32 insn_len; +__u8 insn_bytes[16 ]; +__u32 pad; } memory_fault;/* KVM_EXIT_TDX */ struct {
@@ -1038,6 +1044,7 @@ struct kvm_enable_cap {#define KVM_CAP_HCALL_FAULT_EXIT 252#define KVM_CAP_HYPERV_VTL_SWITCH 253#define KVM_CAP_HYPERV_REG_INTERCEPT 254 +#define KVM_CAP_PLANE_MEMORY_OVERLAYS 257struct kvm_irq_routing_irqchip {__u32 irqchip;
@@ -1394,6 +1401,16 @@ struct kvm_s390_keyop {#define KVM_CREATE_PLANE _IO(KVMIO, 0xe4)#define KVM_GET_PLANE_RUN_STATE _IOR(KVMIO, 0xe5, __u32)#define KVM_SET_PLANE_RUN_STATE _IOW(KVMIO, 0xe6, __u32) + +/* Map one page of userspace memory over a GPA for a single plane (VSM +* overlay pages: SynIC SIMP/SIEFP, hypercall page, VP assist page). */ +struct kvm_plane_memory_overlay { +__u32 plane; +__u32 flags; +__u64 gfn; +__u64 uaddr;/* page-aligned; 0 removes the overlay */ +}; +#define KVM_SET_PLANE_MEMORY_OVERLAY _IOW(KVMIO, 0xe7, struct kvm_plane_memory_overlay)#define KVM_PLANE_RUN_STATE_STOPPED 0#define KVM_PLANE_RUN_STATE_RUNNABLE 1#define KVM_PLANE_RUN_STATE_KERNEL_STOPPED 2
virt/kvm/kvm_main.c +271 -0
295 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 26/55 WIP: KVM: x86: VSM event-delivery parity fixes from Windows boot triage
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
All derived from the reference host's behaviour, and verified against its VSM unit-test suite (still green) and Windows Server 2025 VBS boot:
- Forced entry into a higher plane is edge-triggered, as on the reference host, which attempts one forced entry per posted interrupt. Delivery of a fixed vector or NMI to a suspended higher plane sets hv_apic_event_pending; the evaluation triggers on that marker and the consumable requests (stimer, NMI, APIC timer), not on a scan of the plane's IRR — a vector still latched after a refused entry waits for the target's next cooperative entry or next posted event instead of retrying on every run of the lower plane. The marker is cleared only by a run that actually reaches event evaluation; a run aborted by the VTL-exit or KVM_REQ_PLANE_RESCHED bail-outs leaves it set, since delivery never re-arms it. A vector masked by the higher VTL's TPR still forces exactly one entry.
- One exception to single-shot: a plane that suspends (VtlReturn or the run-state ioctl) with a vector still latched in its IRR re-arms its marker, via kvm_arch_vcpu_plane_stopped() from kvm_vcpu_set_plane_stopped(). Evaluating does not imply delivering:
Windows hibernation posts vector 0xE0 to a twin mid-run with IF clear, the run consumes the marker without injecting, and the suspended twin's vector is stranded — VTL0 freezes all processors, spins on the secure kernel and wedges the VM at 100% host CPU. A re-armed marker cannot storm: the deliverability gate still applies, and a plane that reached suspension has run before, so userspace can enter it again.
- NMIs aimed at a higher plane fold back to plane 0 for delivery after the VTL returns; the higher plane only sees the forced entry. A real NMI inside VTL1 bugchecks the secure kernel (0x18B).
- VTL1 exempted from the reserved-bits check on Hyper-V vAPIC MSR
writes: the secure kernel writes HV_X64_MSR_EOI with stale upper bits, and the reference host truncates for VTL1.
- KVM_REQ_HV_TLB_FLUSH made per-plane: a shared request could be consumed by the running twin, leaving the targeted plane's TLB stale.
- Lower-twin kicks on the stimer retry (vcpu_kick=false) paths.
--- arch/x86/include/asm/kvm_host.h | 12 ++++- arch/x86/kvm/hyperv.c | 3 +- arch/x86/kvm/lapic.c | 39 ++++++++++++++- arch/x86/kvm/x86.c | 89 +++++++++++++++++++++++++++++---- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 5 ++ 6 files changed, 135 insertions(+), 14 deletions(-)
arch/x86/include/asm/kvm_host.h +11 -1
@@ -127,7 +127,8 @@#define KVM_REQ_MMU_FREE_OBSOLETE_ROOTS \ KVM_ARCH_REQ_FLAGS(31, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)#define KVM_REQ_HV_TLB_FLUSH \ - KVM_ARCH_REQ_FLAGS(32, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) + KVM_ARCH_REQ_FLAGS(32, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP | \ + KVM_REQUEST_PER_PLANE)#define KVM_REQ_UPDATE_PROTECTED_GUEST_STATE \ KVM_ARCH_REQ_FLAGS(34, KVM_REQUEST_WAIT | KVM_REQUEST_PER_PLANE)
@@ -854,6 +855,15 @@ int kvm_arch_vcpu_common_init(struct kvm_vcpu_common *common);void kvm_arch_vcpu_common_destroy (struct kvm_vcpu_common *common);struct kvm_vcpu_arch { +/* +* Hyper-V VSM: an APIC event (fixed vector or NMI) was delivered to +* this plane while it was suspended. Set at delivery, cleared when +* the plane next runs; a lower plane's run loop uses it to force a +* VTL entry even when the vector is currently masked by this +* plane's TPR (matching the reference host's apic_pending_vtls). +*/ +bool hv_apic_event_pending; +/* * rip and regs accesses must go through * kvm_{register,rip}_{read,write} functions.
arch/x86/kvm/hyperv.c +2 -1
@@ -605,8 +605,9 @@ static void stimer_mark_pending(struct kvm_vcpu_hv_stimer *stimer,* A pending stimer on a suspended higher plane is only processed * once that plane runs; kick the lower twins so the running one * re-evaluates pending higher-plane events and forces entry. +* This applies to retry paths too (vcpu_kick == false). */ -if (vcpu_kick && vcpu->plane_level) { +if (vcpu->plane_level) {struct kvm_vcpu *twin;int p;
arch/x86/kvm/lapic.c +37 -2
@@ -1448,6 +1448,7 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode,struct kvm_vcpu *twin;int p; +WRITE_ONCE (vcpu->arch.hv_apic_event_pending, true);for (p =0 ; p < vcpu->plane_level; p++) { twin = vcpu->common->vcpus[p];if (twin) {
@@ -1474,8 +1475,31 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode,case APIC_DM_NMI : result =1 ; -kvm_inject_nmi (vcpu); -kvm_vcpu_kick (vcpu); +/* +* An NMI aimed at a higher plane forces entry into that +* plane (reason INTERRUPT, below) but the NMI itself is +* folded back to plane 0: the reference host never delivers +* an NMI inside a higher VTL, it re-queues it to VTL0's +* context for delivery after the VTL returns. +*/ +if (vcpu->plane_level) { +struct kvm_vcpu *twin; +int p; + +WRITE_ONCE (vcpu->arch.hv_apic_event_pending, true); +if (vcpu->common->vcpus[0 ]) +kvm_inject_nmi (vcpu->common->vcpus[0 ]); +for (p =0 ; p < vcpu->plane_level; p++) { + twin = vcpu->common->vcpus[p]; +if (twin) { +kvm_make_request (KVM_REQ_UNBLOCK , twin); +kvm_vcpu_kick (twin); + } + } + }else { +kvm_inject_nmi (vcpu); +kvm_vcpu_kick (vcpu); + }break ;case APIC_DM_INIT :
@@ -3535,6 +3559,17 @@ int kvm_hv_vapic_msr_write(struct kvm_vcpu *vcpu, u32 reg, u64 data)if (!lapic_in_kernel (vcpu))return 1 ; +/* +* Bits 63:32 are reserved in non-ICR registers, but the Windows +* secure kernel leaves stale data in the high bits when writing +* the Hyper-V vAPIC MSRs from a higher VTL. The reference host +* exempts those writes and truncates; a #GP here is a VTL1 +* bugcheck. The exemption covers only this PV path: a native +* x2APIC WRMSR takes the architectural #GP whatever the plane. +*/ +if (vcpu->plane_level && reg !=APIC_ICR ) + data = (u32 )data; +return kvm_lapic_msr_write (vcpu->arch.apic, reg, data); }
arch/x86/kvm/x86.c +79 -10
115 lines not on this page · raw message
include/linux/kvm_host.h +1 -0
@@ -493,6 +493,7 @@ static inline void kvm_vcpu_cancel_vtl_fpu_carry(struct kvm_vcpu_common *common)void kvm_vcpu_set_plane_runnable (struct kvm_vcpu *vcpu);void kvm_vcpu_set_plane_stopped (struct kvm_vcpu *vcpu); +void kvm_arch_vcpu_plane_stopped (struct kvm_vcpu *vcpu);struct kvm_vcpu *kvm_vcpu_select_plane (struct kvm_vcpu *vcpu);static inline bool kvm_vcpu_wants_to_run (struct kvm_vcpu *vcpu)
virt/kvm/kvm_main.c +5 -0
@@ -5347,9 +5347,14 @@ void kvm_vcpu_set_plane_runnable(struct kvm_vcpu *vcpu) }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_vcpu_set_plane_runnable); +void __weak kvm_arch_vcpu_plane_stopped (struct kvm_vcpu *vcpu) +{ +} +void kvm_vcpu_set_plane_stopped (struct kvm_vcpu *vcpu) { vcpu->plane_state =STOPPED ; +kvm_arch_vcpu_plane_stopped (vcpu);kvm_make_request (KVM_REQ_PLANE_RESCHED , vcpu); }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_vcpu_set_plane_stopped);
Signature · 1 line
2.50.1
RFC PATCH 27/55 KVM: planes: fan out per-plane requests to twin vCPUs
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
A request raised on one plane's vCPU for a VM-wide reason — TLB flush after a host-side zap, APIC page reload, obsolete-roots release — is consumed by whichever twin runs first; the planes series does not yet fan requests out to twins, so the other planes keep stale translations. The plane-1 twin of a Windows VSM guest then runs on a zapped root and the guest bugchecks.
Mark those three requests per-plane and make kvm_make_vcpu_request() replicate any KVM_REQUEST_PER_PLANE request to every twin of the target vCPU. Mode, pCPU and wait live in the shared common, so the existing kick logic already reaches the twins.
Move the Hyper-V TLB-flush hypercalls onto the same fan-out. The reference hypervisor's TLB is VTL-blind: both VTLs share one hardware translation context, a flush from either VTL covers both, and the hypercall does not return until the targets have left guest mode. The secure kernel relies on all of that when it edits mappings the normal kernel also uses — with per-plane TLB tags and the previous unwaited kvm_make_request() plus kick for non-zero-plane callers, VTL0 read stale PTEs and Windows bugchecked 0x1A (MEMORY_MANAGEMENT, 0x6102) minutes into an SMP boot. Enqueue the flush entries for every twin of each target VP and dispatch KVM_REQ_HV_TLB_FLUSH through the waiting fan-out for callers in either VTL.
--- arch/x86/include/asm/kvm_host.h | 6 ++- arch/x86/kvm/hyperv.c | 65 ++++++++++++++++----------------- arch/x86/kvm/mmu/mmu.c | 13 ++++--- include/linux/kvm_host.h | 3 +- virt/kvm/kvm_main.c | 22 ++++++++++- 5 files changed, 66 insertions(+), 43 deletions(-)
arch/x86/include/asm/kvm_host.h +4 -2
@@ -107,7 +107,8 @@ KVM_ARCH_REQ_FLAGS(15, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)#define KVM_REQ_GLOBAL_CLOCK_UPDATE KVM_ARCH_REQ(16)#define KVM_REQ_APIC_PAGE_RELOAD \ - KVM_ARCH_REQ_FLAGS(17, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) + KVM_ARCH_REQ_FLAGS(17, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP | \ + KVM_REQUEST_PER_PLANE)#define KVM_REQ_HV_CRASH KVM_ARCH_REQ(18)#define KVM_REQ_IOAPIC_EOI_EXIT KVM_ARCH_PLANE_REQ(19)#define KVM_REQ_HV_RESET KVM_ARCH_REQ(20)
@@ -125,7 +126,8 @@#define KVM_REQ_UPDATE_CPU_DIRTY_LOGGING \ KVM_ARCH_REQ_FLAGS(30, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)#define KVM_REQ_MMU_FREE_OBSOLETE_ROOTS \ - KVM_ARCH_REQ_FLAGS(31, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) + KVM_ARCH_REQ_FLAGS(31, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP | \ + KVM_REQUEST_PER_PLANE)#define KVM_REQ_HV_TLB_FLUSH \ KVM_ARCH_REQ_FLAGS(32, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP | \ KVM_REQUEST_PER_PLANE)
arch/x86/kvm/hyperv.c +31 -34
89 lines not on this page · raw message
arch/x86/kvm/mmu/mmu.c +8 -5
@@ -6265,11 +6265,14 @@ void kvm_mmu_free_obsolete_roots(struct kvm_vcpu *vcpu)int i;/* -* KVM_REQ_MMU_FREE_OBSOLETE_ROOTS is tracked in the request set -* shared by all of the vCPU's planes, so the plane that consumes -* the request is not necessarily the plane whose root is obsolete. -* Service every plane: the inactive twins cannot be running (all -* planes of a vCPU share one thread), so their MMUs are stable. +* KVM_REQ_MMU_FREE_OBSOLETE_ROOTS fans out to every plane twin, +* but each twin only consumes its own copy when it next runs, and +* a plane that userspace never re-enters would pin its obsolete +* roots indefinitely. Service every plane from whichever twin +* consumes its request first: the inactive twins cannot be +* running (all planes of a vCPU share one thread), so their MMUs +* are stable, and a twin's own later request finds nothing left +* to free. */ for (i =0 ; i <KVM_MAX_PLANES ; i++) { v = vcpu->common->vcpus[i];
include/linux/kvm_host.h +2 -1
@@ -164,7 +164,8 @@ static inline bool kvm_is_error_gpa(gpa_t gpa)* Architecture-independent vcpu->requests bit members * Bits 3-7 are reserved for more arch-independent bits. */ -#define KVM_REQ_TLB_FLUSH (0 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) +#define KVM_REQ_TLB_FLUSH \ + (0 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP | KVM_REQUEST_PER_PLANE)#define KVM_REQ_VM_DEAD (1 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)#define KVM_REQ_UNBLOCK 2#define KVM_REQ_DIRTY_RING_SOFT_FULL 3
virt/kvm/kvm_main.c +21 -1
@@ -218,9 +218,29 @@ static void kvm_make_vcpu_request(struct kvm_vcpu *vcpu, unsigned int req, {int cpu; -if (likely (!(req &KVM_REQUEST_NO_ACTION ))) +if (likely (!(req &KVM_REQUEST_NO_ACTION ))) {__kvm_make_request (req, vcpu); +/* +* A per-plane request raised on "the vCPU" (TLB flush after +* a host-side zap, obsolete-roots reload, ...) concerns +* every plane: fan it out to the twins, or the plane that +* runs next consumes a bit the others still need. Mode, +* pCPU and wait are shared with the common, so the kick +* logic below covers the twins too. +*/ +if (req &KVM_REQUEST_PER_PLANE ) { +struct kvm_vcpu *twin; +int p; + +for (p =0 ; p <KVM_MAX_PLANES ; p++) { + twin = vcpu->common->vcpus[p]; +if (twin && twin != vcpu) +__kvm_make_request (req, twin); + } + } + } +if (!(req &KVM_REQUEST_NO_WAKEUP ) &&kvm_vcpu_wake_up (vcpu))return ;
Signature · 1 line
2.50.1
RFC PATCH 28/55 KVM: x86: inhibit APICv when VM planes are enabled
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
With APICv live on plane-0 vCPUs while their plane-1 twins use software injection, the twins trip the APICv coherency checks: stale vcpu-vs-VM activation state at vcpu_enter_guest(), and timer interrupt injection against a vCPU that is not the running one. Cross-plane event delivery (forced VTL entry, twin kicks, NMI folding) depends on software-injection bookkeeping that hardware acceleration bypasses.
Add APICV_INHIBIT_REASON_PLANES, set it when a plane above 0 is created, and include it in the VMX and AVIC required-inhibit masks; __kvm_set_or_clear_apicv_inhibit() silently ignores a reason the vendor mask does not carry. The reference VSM hypervisor likewise force-disables APICv when VTLs are enabled.
--- arch/x86/include/asm/kvm_host.h | 10 +++++++++- arch/x86/kvm/svm/svm.h | 3 ++- arch/x86/kvm/vmx/main.c | 3 ++- arch/x86/kvm/x86.c | 10 ++++++++++ 4 files changed, 23 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +9 -1
@@ -1436,6 +1436,13 @@ enum kvm_apicv_inhibit {*/ APICV_INHIBIT_REASON_PHYSICAL_ID_TOO_BIG , +/* +* APICv is inhibited when the VM has multiple planes (Hyper-V VTLs). +* Cross-plane event delivery relies on software injection; hardware +* acceleration on one twin would bypass it. +*/ +APICV_INHIBIT_REASON_PLANES , + NR_APICV_INHIBIT_REASONS, };
@@ -1455,7 +1462,8 @@ enum kvm_apicv_inhibit { __APICV_INHIBIT_REASON(PIT_REINJ), \ __APICV_INHIBIT_REASON(SEV), \ __APICV_INHIBIT_REASON(LOGICAL_ID_ALIASED), \ - __APICV_INHIBIT_REASON(PHYSICAL_ID_TOO_BIG) + __APICV_INHIBIT_REASON(PHYSICAL_ID_TOO_BIG), \ + __APICV_INHIBIT_REASON(PLANES) struct kvm_possible_nx_huge_pages { /*
arch/x86/kvm/svm/svm.h +2 -1
@@ -925,7 +925,8 @@ extern struct kvm_x86_nested_ops svm_nested_ops; BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) | \ BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED) | \ BIT(APICV_INHIBIT_REASON_LOGICAL_ID_ALIASED) | \ - BIT(APICV_INHIBIT_REASON_PHYSICAL_ID_TOO_BIG) \ + BIT(APICV_INHIBIT_REASON_PHYSICAL_ID_TOO_BIG) | \ + BIT(APICV_INHIBIT_REASON_PLANES) \ )bool __init avic_hardware_setup (void );
arch/x86/kvm/vmx/main.c +2 -1
@@ -876,7 +876,8 @@ static int vt_gmem_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, BIT(APICV_INHIBIT_REASON_BLOCKIRQ) | \ BIT(APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED) | \ BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) | \ - BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED)) + BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED) | \ + BIT(APICV_INHIBIT_REASON_PLANES))struct kvm_x86_ops vt_x86_ops__initdata = { .name =KBUILD_MODNAME ,
arch/x86/kvm/x86.c +10 -0
@@ -13979,6 +13979,16 @@ int kvm_arch_plane_init(struct kvm *kvm, struct kvm_plane *plane, {mutex_init (&plane->arch.apic_map_lock); +/* +* Cross-plane event delivery (forced VTL entry, twin kicks, NMI +* folding) relies on software injection via KVM_REQ_EVENT; hardware +* APIC virtualization on one twin would bypass the bookkeeping the +* other twin's plane depends on. Mirrors the reference VSM +* hypervisor, which force-disables APICv when VTLs are enabled. +*/ +if (plane_level >0 ) +kvm_set_apicv_inhibit (kvm,APICV_INHIBIT_REASON_PLANES ); +return 0 ; }
Signature · 1 line
2.50.1
RFC PATCH 29/55 KVM: x86: hyperv: divert LLDT and LTR into register intercepts
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
When a higher VTL arms the ldtr_write or tr_write bits of HvRegisterCrInterceptControl, hv_divert_dt_write() covers GDTR and IDTR writes, but an LTR in the lower VTL retires natively, and the LDTR divert sat in emulator_set_segment(), which task-switch emulation also reaches via __load_segment_descriptor(). A protected-mode task switch with the intercept armed — reachable by a malicious lower VTL, even though 64-bit Windows takes no task switches — had its LDTR load silently dropped, and the higher VTL received a bogus mid-task-switch intercept carrying the wrong RIP and instruction length. The TLFS intercepts cover the LLDT and LTR instructions only.
Add a divert_seg_write(ctxt, selector, seg) emulator op and call it from em_lldt() and em_ltr() before load_segment_descriptor(). Diverting from the instruction handlers, before any architectural effect, keeps hardware task switches out of the intercept; for LTR it also matters that __load_segment_descriptor() marks the TSS descriptor busy in the GDT first, and a busy descriptor would #GP the LTR when it re-executes after a cooperative VTL return. emulator_set_segment() no longer diverts anything.
No vmx change is needed: SECONDARY_EXEC_DESC already exits on LTR (handle_desc covers EXIT_REASON_LDTR_TR) whenever any reg_intercept_control bit is set.
--- arch/x86/kvm/emulate.c | 21 +++++++++++++++++++++ arch/x86/kvm/kvm_emulate.h | 10 ++++++++++ arch/x86/kvm/x86.c | 33 +++++++++++++++++++++------------ 3 files changed, 52 insertions(+), 12 deletions(-)
arch/x86/kvm/emulate.c +21 -0
@@ -3387,6 +3387,16 @@ static int em_lldt(struct x86_emulate_ctxt *ctxt)/* Disable writeback. */ ctxt->dst.type =OP_NONE ; + +/* +* A higher VTL's LDTR-write intercept suspends the LLDT before any +* architectural effect. Diverting here rather than in +* emulator_set_segment() keeps task switches, which load LDTR +* through set_segment as well, out of the intercept. +*/ +if (ctxt->ops->divert_seg_write (ctxt, sel,VCPU_SREG_LDTR )) +return X86EMUL_CONTINUE ; +return load_segment_descriptor (ctxt, sel,VCPU_SREG_LDTR ); }
@@ -3401,6 +3411,17 @@ static int em_ltr(struct x86_emulate_ctxt *ctxt)/* Disable writeback. */ ctxt->dst.type =OP_NONE ; + +/* +* A higher VTL's TR-write intercept suspends the LTR before any +* architectural effect. The divert must precede +* load_segment_descriptor(): that path marks the TSS descriptor +* busy in the GDT, and a busy descriptor would #GP the re-executed +* LTR after a cooperative VTL return. +*/ +if (ctxt->ops->divert_seg_write (ctxt, sel,VCPU_SREG_TR )) +return X86EMUL_CONTINUE ; +return load_segment_descriptor (ctxt, sel,VCPU_SREG_TR ); }
arch/x86/kvm/kvm_emulate.h +10 -0
@@ -202,6 +202,16 @@ struct x86_emulate_ops { struct desc_struct *desc, u32 *base3, int seg);void (*set_segment)(struct x86_emulate_ctxt *ctxt,u16 selector,struct desc_struct *desc,u32 base3,int seg); +/* +* Divert an LTR or LLDT into a Hyper-V VSM register-intercept exit +* when the higher VTL armed one. Returns true when the write was +* diverted: the instruction must then have no architectural effect, +* so this is checked before descriptor validation (and, for LTR, +* the busy-bit write). Only the instructions divert; task-switch +* loads of LDTR/TR are not covered by the TLFS intercepts. +*/ +bool (*divert_seg_write)(struct x86_emulate_ctxt *ctxt,u16 selector, +int seg);unsigned long (*get_cached_segment_base)(struct x86_emulate_ctxt *ctxt,int seg);void (*get_gdt)(struct x86_emulate_ctxt *ctxt,struct desc_ptr *dt);
arch/x86/kvm/x86.c +21 -12
@@ -8859,9 +8859,11 @@ static bool hv_divert_dt_write(struct x86_emulate_ctxt *ctxt, u32 hv_reg,#define HV_CR_INTERCEPT_GDTR_WRITE BIT_ULL(15)#define HV_CR_INTERCEPT_IDTR_WRITE BIT_ULL(16)#define HV_CR_INTERCEPT_LDTR_WRITE BIT_ULL(17) +#define HV_CR_INTERCEPT_TR_WRITE BIT_ULL(18)#define HV_X64_REGISTER_IDTR_ID 0x00070000#define HV_X64_REGISTER_GDTR_ID 0x00070001#define HV_X64_REGISTER_LDTR_ID 0x00060006 +#define HV_X64_REGISTER_TR_ID 0x00060007static void emulator_set_gdt (struct x86_emulate_ctxt *ctxt,struct desc_ptr *dt) {
@@ -8949,22 +8951,28 @@ static void emulator_set_segment(struct x86_emulate_ctxt *ctxt, u16 selector, var.unusable = !var.present; var.padding =0 ; -if (seg ==VCPU_SREG_LDTR ) { -struct desc_ptr sel_as_dt = { - .address = selector, - .size =0 , - }; - -/* The intercept message carries the written selector. */ -if (hv_divert_dt_write (ctxt,HV_X64_REGISTER_LDTR_ID , - &sel_as_dt,HV_CR_INTERCEPT_LDTR_WRITE )) -return ; - } -kvm_set_segment (vcpu, &var, seg);return ; } +static bool emulator_divert_seg_write (struct x86_emulate_ctxt *ctxt, +u16 selector,int seg) +{ +struct desc_ptr sel_as_dt = { + .address = selector, + .size =0 , + }; + +/* The intercept message carries the written selector. */ +if (seg ==VCPU_SREG_LDTR ) +return hv_divert_dt_write (ctxt,HV_X64_REGISTER_LDTR_ID , + &sel_as_dt, +HV_CR_INTERCEPT_LDTR_WRITE ); + +return hv_divert_dt_write (ctxt,HV_X64_REGISTER_TR_ID , &sel_as_dt, +HV_CR_INTERCEPT_TR_WRITE ); +} +static int emulator_get_msr_with_filter (struct x86_emulate_ctxt *ctxt,u32 msr_index,u64 *pdata) {
@@ -9167,6 +9175,7 @@ static const struct x86_emulate_ops emulate_ops = { .pio_out_emulated = emulator_pio_out_emulated, .get_segment = emulator_get_segment, .set_segment = emulator_set_segment, + .divert_seg_write = emulator_divert_seg_write, .get_cached_segment_base = emulator_get_cached_segment_base, .get_gdt = emulator_get_gdt, .get_idt = emulator_get_idt,
Signature · 1 line
2.50.1
RFC PATCH 30/55 KVM: planes: Unwind published state on KVM_CREATE_PLANE failure
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
kvm_create_plane() publishes the new plane in kvm->planes[] and takes the APICV_INHIBIT_REASON_PLANES inhibit before kvm_vm_ioctl_create_plane() can still fail at anon_inode_getfile(), and the error path freed the plane without unpublishing it: a VMM retrying plane setup after such a failure wrote overlays through the freed plane via KVM_SET_PLANE_MEMORY_OVERLAY, and VM teardown destroyed the plane a second time. The VM also kept APICv inhibited with no nonzero plane left to justify it.
Clear the kvm->planes[] slot before freeing, and drop the APICv inhibit in kvm_arch_plane_destroy() when the last nonzero plane of a live VM goes away (at VM teardown plane 0 is already gone, which skips the update). Make the overlay ioctl read kvm->planes[] under kvm->lock, which the create path holds across publish and unpublish, so a plane pointer it observes stays valid until VM teardown; publish with smp_store_release() for the remaining lock-free readers.
Found by Sashiko review.
--- arch/x86/kvm/x86.c | 32 ++++++++++++++++++++++++++++++++ virt/kvm/kvm_main.c | 25 ++++++++++++++++++++++--- 2 files changed, 54 insertions(+), 3 deletions(-)
arch/x86/kvm/x86.c +32 -0
@@ -14003,7 +14003,39 @@ int kvm_arch_plane_init(struct kvm *kvm, struct kvm_plane *plane,void kvm_arch_plane_destroy (struct kvm_plane *plane) { +struct kvm *kvm = plane->kvm; +int i; +kvfree (rcu_dereference_check (plane->arch.apic_map,1 )); + +if (plane->level ==0 ) +return ; + +/* +* Drop the APICv inhibit taken in kvm_arch_plane_init() when the +* last nonzero plane goes away. Only the KVM_CREATE_PLANE error +* path destroys a nonzero plane while the VM is live; at VM +* teardown plane 0 is destroyed first and kvm->planes[0] is +* already NULL, which skips the update (the inhibit machinery +* needs plane 0 and SRCU, both gone by then). +*/ +if (!kvm->planes[0 ]) +return ; + +/* +* Reaching here means a nonzero plane died while the VM is live, +* which only the KVM_CREATE_PLANE error path does - under +* kvm->lock. A future kvm_destroy_planes() reordered to tear +* nonzero planes down before plane 0 would get here locklessly; +* catch that instead of silently poking freed APICv state. +*/ +lockdep_assert_held (&kvm->lock); + +for (i =1 ; i <KVM_MAX_PLANES ; i++) +if (kvm->planes[i] && kvm->planes[i] != plane) +return ; + +kvm_clear_apicv_inhibit (kvm,APICV_INHIBIT_REASON_PLANES ); }int kvm_arch_init_vm (struct kvm *kvm,unsigned long type)
virt/kvm/kvm_main.c +22 -3
@@ -1259,7 +1259,12 @@ static struct kvm_plane *kvm_create_plane(struct kvm *kvm, unsigned plane_level)if (kvm_arch_plane_init (kvm, plane, plane_level)) goto out_free_plane; - kvm->planes[plane_level] = plane; +/* +* Pairs with the lock-free readers of kvm->planes[] (MSI delivery, +* SEV VMPL targeting): the plane's fields must be visible before +* the pointer is. +*/ +smp_store_release (&kvm->planes[plane_level], plane);return plane;
@@ -5915,8 +5920,15 @@ static int kvm_vm_ioctl_create_plane(struct kvm *kvm, unsigned id)return fd; put_kvm: -kvm_put_kvm (kvm); +/* +* kvm_create_plane() published the plane; unpublish it before +* freeing (kvm->lock is held throughout, so readers who take it +* never observe the dead pointer) and so that VM teardown does +* not destroy it a second time. +*/ + kvm->planes[id] =NULL ;kvm_destroy_one_plane (plane); +kvm_put_kvm (kvm); put_fd:put_unused_fd (fd);return r;
@@ -6037,7 +6049,14 @@ static int kvm_vm_ioctl_set_plane_overlay(struct kvm *kvm,return -EINVAL ;if (ov->uaddr & ~PAGE_MASK )return -EINVAL ; - plane = kvm->planes[ov->plane]; +/* +* Read under kvm->lock: KVM_CREATE_PLANE publishes and, on its +* error path, unpublishes-and-frees the plane under that lock, so +* a pointer observed here is either NULL or a plane that lives +* until VM teardown. +*/ +scoped_guard (mutex, &kvm->lock) + plane = kvm->planes[ov->plane];if (!plane)return -EINVAL ;
Signature · 1 line
2.50.1
RFC PATCH 31/55 KVM: planes: Kick the running twin's pCPU, not the request target's
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The remote kick paths (kvm_make_vcpus_request_mask(), kvm_make_all_cpus_request(), __kvm_vcpu_kick()) read READ_ONCE(vcpu->cpu) of the plane-0 vCPU. But vcpu->cpu is per plane vCPU — kvm_arch_vcpu_load() refreshes only the loaded twin's copy — while vcpu->mode is shared in kvm_vcpu_common. When the vCPU thread has migrated pCPUs while a non-zero plane was loaded, the shared mode reads IN_GUEST_MODE but the IPI goes to plane 0's stale pCPU: a flush hypercall returns HV_STATUS_SUCCESS while the twin keeps running on stale translations, breaking the TLFS guarantee that the flush covers the whole VP.
Read the pCPU from common->current_vcpu, whose cpu field every kvm_arch_vcpu_load() refreshes, in both kick paths. Also read the twin array with READ_ONCE in the per-plane request fan-out and correct the comment there: mode and wait are shared with the common, the pCPU is not.
Found by Sashiko review.
--- virt/kvm/kvm_main.c | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-)
virt/kvm/kvm_main.c +19 -6
@@ -225,16 +225,16 @@ static void kvm_make_vcpu_request(struct kvm_vcpu *vcpu, unsigned int req,* A per-plane request raised on "the vCPU" (TLB flush after * a host-side zap, obsolete-roots reload, ...) concerns * every plane: fan it out to the twins, or the plane that -* runs next consumes a bit the others still need. Mode, -* pCPU and wait are shared with the common, so the kick -* logic below covers the twins too. +* runs next consumes a bit the others still need. Mode and +* wait live in the common and the kick below targets the +* running twin's pCPU, so it covers the twins too. */ if (req &KVM_REQUEST_PER_PLANE ) {struct kvm_vcpu *twin;int p;for (p =0 ; p <KVM_MAX_PLANES ; p++) { - twin = vcpu->common->vcpus[p]; + twin =READ_ONCE (vcpu->common->vcpus[p]);if (twin && twin != vcpu)__kvm_make_request (req, twin); }
@@ -255,7 +255,14 @@ static void kvm_make_vcpu_request(struct kvm_vcpu *vcpu, unsigned int req,* finalized. See kvm_vcpu_kick() for more details on handling requests. */ if (kvm_request_needs_ipi (vcpu, req)) { - cpu =READ_ONCE (vcpu->cpu); +/* +* vcpu->cpu is per plane vCPU and kvm_arch_vcpu_load() +* refreshes only the loaded twin's, so this plane's copy +* can point at a pCPU the thread has long left while a +* twin runs elsewhere. The mode just checked is shared; +* the IPI must go where the running twin actually is. +*/ + cpu =READ_ONCE (READ_ONCE (vcpu->common->current_vcpu)->cpu);if (cpu !=-1 && cpu != current_cpu)__cpumask_set_cpu (cpu, tmp); }
@@ -4334,7 +4341,13 @@ void __kvm_vcpu_kick(struct kvm_vcpu *vcpu, bool wait)* vCPU also requires it to leave IN_GUEST_MODE. */ if (kvm_arch_vcpu_should_kick (vcpu)) { - cpu =READ_ONCE (vcpu->cpu); +/* +* Target the running twin's pCPU: this plane's vcpu->cpu +* goes stale whenever the thread migrates while another +* plane is loaded, and the IN_GUEST_MODE just observed is +* the (shared) mode of whichever twin runs. +*/ + cpu =READ_ONCE (READ_ONCE (vcpu->common->current_vcpu)->cpu);if (cpu != me && (unsigned int )cpu < nr_cpu_ids &&cpu_online (cpu)) {/* * Use a reschedule IPI to kick the vCPU if the caller
Signature · 1 line
2.50.1
RFC PATCH 32/55 KVM: planes: Keep current_vcpu in sync in vcpu_load()
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The preempt notifiers save and restore common->current_vcpu (kvm_sched_out()/kvm_sched_in()), but vcpu_load() does not update it. kvm_vcpu_ioctl_set_cpuid() swaps the loaded vCPU to each twin around kvm_vcpu_after_set_cpuid(), a window that is preemptible and sleeps (kvm_pmu_refresh, mmu_lock): a reschedule there arch-put the already-put caller a second time, never saved the twin, and reloaded the caller's VMCS, so the vendor callback wrote the twin's controls into the caller's VMCS and the trailing vcpu_put() unloaded a non-resident vCPU, corrupting the per-pCPU loaded_vmcs tracking. Any KVM_SET_CPUID2 on a multi-plane VM reaches this; ioctls taken on a plane-N fd while current_vcpu pointed at another plane had the same exposure, as do the kick paths that now read current_vcpu->cpu.
Set common->current_vcpu in vcpu_load(), making every load path (run loop, vcpu ioctls, the cpuid twin swap) agree with what the preempt notifiers will operate on. kvm_vcpu_select_plane() keeps its own assignment; both stores use WRITE_ONCE to pair with the lock-free readers in the kick paths.
Found by Sashiko review.
--- virt/kvm/kvm_main.c | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-)
virt/kvm/kvm_main.c +30 -1
@@ -165,6 +165,16 @@ void vcpu_load(struct kvm_vcpu *vcpu) {int cpu =get_cpu (); +/* +* The preempt notifiers operate on common->current_vcpu, so it +* must name the plane vCPU being loaded here: an ioctl taken on +* another plane's fd than the one that ran last would otherwise +* have a preemption arch-put the stale twin (whose state is not +* resident) and arch-load it back over the state this load makes +* resident. The kick paths also read it to find the running +* twin's pCPU. +*/ +WRITE_ONCE (vcpu->common->current_vcpu, vcpu);__this_cpu_write (kvm_running_vcpu, vcpu->common);preempt_notifier_register (&vcpu->common->preempt_notifier);kvm_arch_vcpu_load (vcpu, cpu);
@@ -4719,7 +4729,26 @@ static int kvm_plane_ioctl_create_vcpu(struct kvm_plane *plane, unsigned long idkvm_vcpu_init (vcpu, kvm, id); +/* +* A non-zero plane shares 'common' with a plane-0 twin which may +* already be running. kvm_arch_vcpu_create() calls vcpu_load(), +* which publishes the half-created vCPU through +* common->current_vcpu and registers the shared preempt notifier, +* so it must not run concurrently with a twin's KVM_RUN: hold the +* shared vCPU mutex across it. Drop it before kvm->lock is taken +* below, to respect the kvm->lock -> vcpu mutex ordering. +*/ +if (plane->level >0 ) { + r =mutex_lock_killable (kvm_vcpu_mutex (vcpu)); +if (r) + goto vcpu_free_common; + } + r =kvm_arch_vcpu_create (vcpu); + +if (plane->level >0 ) +kvm_vcpu_unlock (vcpu); +if (r) goto vcpu_free_common;
@@ -5427,7 +5456,7 @@ struct kvm_vcpu *kvm_vcpu_select_plane(struct kvm_vcpu *vcpu) ret->plane_state =RUNNABLE ; } - common->current_vcpu = ret; +WRITE_ONCE (common->current_vcpu, ret); common->plane_switch = false;
Signature · 1 line
2.50.1
RFC PATCH 33/55 KVM: x86: Report a real write-access walk from KVM_TRANSLATE
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
KVM_TRANSLATE hardcodes writeable = 1, so userspace answering HvCallTranslateVirtualAddress from this ioctl cannot honour the hypercall's write-validation flag: validating write access against a read-only PTE succeeds instead of reporting the address unmapped, and the Windows secure kernel's page verification trusts a mapping it cannot write.
Walk the address a second time with PFERR_WRITE_MASK at supervisor level and report the outcome, matching the reference hypervisor's validation walk.
--- arch/x86/kvm/x86.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +10 -1
@@ -13305,10 +13305,19 @@ int kvm_arch_vcpu_ioctl_translate(struct kvm_vcpu *vcpu, idx =srcu_read_lock (&vcpu->kvm->srcu); gpa =kvm_mmu_gva_to_gpa_system (vcpu, vaddr,NULL ); +/* +* Report write access from a supervisor-level walk instead of the +* hardcoded 1: Hyper-V's TranslateVirtualAddress write validation +* is answered from this ioctl, and the reference hypervisor walks +* with PFERR_WRITE_MASK and no user bit. +*/ + tr->writeable = vcpu->arch.walk_mmu->gva_to_gpa (vcpu, + vcpu->arch.walk_mmu, + vaddr,PFERR_WRITE_MASK , +NULL ) !=INVALID_GPA ;srcu_read_unlock (&vcpu->kvm->srcu, idx); tr->physical_address = gpa; tr->valid = gpa !=INVALID_GPA ; - tr->writeable =1 ; tr->usermode =0 ;vcpu_put (vcpu);
Signature · 1 line
2.50.1
RFC PATCH 34/55 KVM: planes: Share XCR0 across plane twins
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
XCR0/XFEM is shared between VTLs: the TLFS banks no vector state and the reference host keeps a single XCR0 (its store_vtl/load_vtl carry no XSAVE state). The planes series shares the FPU payload between twins but keeps arch.xcr0 per-vCPU, so an XSETBV in one VTL was invisible in the other: VTL1 read a stale XFEM, and state enabled by VTL0 (AVX, for the Windows secure kernel) silently vanished across a VTL switch.
Propagate the validated value to every plane twin in __kvm_set_xcr(), covering both guest XSETBV and the KVM_SET_XCRS ioctl path. The CPUID dynamic-bits dirty marker lives in the shared common state, so the existing single set covers the twins.
--- arch/x86/kvm/x86.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
arch/x86/kvm/x86.c +30 -0
@@ -1361,6 +1361,28 @@ int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr) vcpu->arch.xcr0 = xcr0; +/* +* XCR0/XFEM is shared across VTLs (TLFS 15.4; the reference host +* banks no vector state): propagate the new value to every plane +* twin so the other VTLs observe it. This covers both guest +* XSETBV and the KVM_SET_XCRS ioctl path. The CPUID dirty marker +* below lives in the shared common state, so one set covers the +* twins too. +* +* Skip the fan-out until this vCPU is published in +* common->vcpus[]: the architectural reset of a twin still being +* created must not clobber a live sibling's XCR0 (the new twin +* inherits the shared value instead, in kvm_arch_vcpu_create()). +*/ +if (vcpu->common && + vcpu->common->vcpus[vcpu->plane_level] == vcpu) { +struct kvm_vcpu *twin; +int i; + +vcpu_for_each_plane (vcpu->common, i, twin) + twin->arch.xcr0 = xcr0; + } +if ((xcr0 ^ old_xcr0) &XFEATURE_MASK_EXTEND )cpuid_set_dirty (vcpu);return 0 ;
@@ -13549,6 +13571,14 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)kvm_vcpu_after_set_cpuid (vcpu);kvm_set_tsc_khz (vcpu, vcpu->kvm->arch.default_tsc_khz);kvm_vcpu_reset (vcpu, false); +/* +* XCR0 is shared across planes. A twin created after its plane-0 +* sibling has run (Hyper-V VSM enables VTL1 at runtime) inherits +* the current shared value; the architectural reset value only +* stands when no sibling exists yet. +*/ +if (vcpu->plane_level && vcpu->common->vcpus[0 ]) + vcpu->arch.xcr0 = vcpu->common->vcpus[0 ]->arch.xcr0;kvm_init_mmu (vcpu);vcpu_put (vcpu);return 0 ;
Signature · 1 line
2.50.1
RFC PATCH 35/55 KVM: x86: Leave TSC matching to the plane 0 vCPUs
In reply to David WoodhouseFrom: Mushahid Hussain <[email protected]>
KVM enables the master clock when every vCPU has a matching TSC, judged by nr_vcpus_matched_tsc + 1 == online_vcpus. online_vcpus counts a vCPU once, but each of its plane vCPUs synchronizes its TSC at creation and counts as another matching vCPU. With one higher plane the count can never equal the number of vCPUs again, the master clock stays off and kvmclock loses PVCLOCK_TSC_STABLE_BIT. Among other things the Hyper-V reference TSC page then breaks: KVM writes sequence 0 to it, finds the clock unstable and leaves it that way.
The planes of a vCPU share its TSC. Give a plane vCPU the current generation and leave the matching count and generation changes to its plane 0 sibling.
--- arch/x86/kvm/x86.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
arch/x86/kvm/x86.c +25 -0
@@ -2858,6 +2858,31 @@ static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc,if (vcpu->arch.guest_tsc_protected)return ; +/* +* The planes of a vCPU share its TSC and online_vcpus counts the +* vCPU once, so a plane vCPU neither counts as a matching vCPU nor +* starts a generation; its plane 0 sibling speaks for it. Nor may +* it perturb the global matching window (last_tsc_ and +* user_set_tsc): KVM_SET_MSRS is accepted on plane fds, and a TSC +* write on a plane fd +* moving the window would corrupt the next plane-0 write's match +* decision. Latch the generation from the plane-0 sibling, which +* exists by construction for a plane twin, rather than from the +* VM-global cur_tsc_* that may belong to a different vCPU's +* generation entirely. +*/ +if (vcpu->plane_level) { +struct kvm_vcpu *sibling = vcpu->common->vcpus[0 ]; + + vcpu->arch.last_guest_tsc = tsc; +kvm_vcpu_write_tsc_offset (vcpu, offset); + + vcpu->arch.this_tsc_generation = sibling->arch.this_tsc_generation; + vcpu->arch.this_tsc_nsec = sibling->arch.this_tsc_nsec; + vcpu->arch.this_tsc_write = sibling->arch.this_tsc_write; +return ; + } +if (user_set_tsc) vcpu->kvm->arch.user_set_tsc = true;
Signature · 1 line
2.50.1
RFC PATCH 36/55 KVM: x86: hyperv: Track the reference TSC page per plane
In reply to David WoodhouseFrom: Mushahid Hussain <[email protected]>
HV_X64_MSR_REFERENCE_TSC is private to each Virtual Trust Level: the secure VTL maps its own page. The reference time behind the pages is one clock per partition, so every page must show the same scale, offset and sequence.
KVM keeps a single page address per VM. A write from another plane replaces it, the old page stops being updated and the new page starts a sequence of its own. Keep the MSR value per plane and write the shared page contents to every enabled page. The lowest enabled plane owns the sequence number; disabling one plane's page keeps the others updated.
[dwmw2: our tree already banks the MSR per plane (fbdffa08a242) with a
one-shot copy for higher planes; replace that with the maintained pages, keeping the bank in kvm_hv rather than kvm_arch_plane. ]
--- arch/x86/include/asm/kvm_host.h | 6 +- arch/x86/kvm/hyperv.c | 118 +++++++++++++++----------------- 2 files changed, 58 insertions(+), 66 deletions(-)
arch/x86/include/asm/kvm_host.h +4 -2
@@ -1255,8 +1255,10 @@ struct kvm_hv {u64 hv_guest_os_id[KVM_MAX_PLANES ];u64 hv_hypercall[KVM_MAX_PLANES ];bool hv_forward_hcalls; -u64 hv_tsc_page; -/* Banked REFERENCE_TSC MSR values for non-zero planes (VTLs). */ +/* +* HV_X64_MSR_REFERENCE_TSC is private to each VTL, the reference time +* it exposes is not: every enabled page gets the same contents. +*/ u64 hv_tsc_page_plane[KVM_MAX_PLANES ];enum hv_tsc_page_status hv_tsc_page_status;
arch/x86/kvm/hyperv.c +54 -64
169 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 37/55 KVM: x86: hyperv: Update the reference TSC page past protections
In reply to David WoodhouseFrom: Mushahid Hussain <[email protected]>
The reference TSC page is a page the hypervisor maintains for the guest. A higher Virtual Trust Level keeps its own page, and it protects the memory that holds that page against the lower levels. KVM writes the page through kvm_write_guest(), which refuses a write to protected memory. The refusal aborts the update of every page: KVM has written sequence zero to all of them and leaves them that way for the rest of the guest's life, so the guest falls back to the time reference count register for every read of time.
Memory protections restrict the guest. They do not restrict the hypervisor's update of a page the hypervisor owns. Add kvm_write_guest_unprotected() for such a page and use it for the reference TSC page.
[dwmw2: the original message called the function kvm_write_guest_hv();
it is kvm_write_guest_unprotected() in the code, say so. ]
--- arch/x86/kvm/hyperv.c | 2 +- include/linux/kvm_host.h | 2 ++ virt/kvm/kvm_main.c | 27 +++++++++++++++++++++++++++ 3 files changed, 30 insertions(+), 1 deletion(-)
arch/x86/kvm/hyperv.c +1 -1
@@ -1260,7 +1260,7 @@ static int hv_write_tsc_pages(struct kvm *kvm, size_t len)if (!hv_tsc_page_gfn (hv, i, &gfn))continue ; -if (kvm_write_guest (kvm,gfn_to_gpa (gfn), &hv->tsc_ref, len)) +if (kvm_write_guest_unprotected (kvm,gfn_to_gpa (gfn), &hv->tsc_ref, len))return -EFAULT ; }
include/linux/kvm_host.h +2 -0
@@ -1551,6 +1551,8 @@ int kvm_write_guest_page(struct kvm *kvm, gfn_t gfn, const void *data, int offset, int len);int kvm_write_guest (struct kvm *kvm,gpa_t gpa,const void *data,unsigned long len); +int kvm_write_guest_unprotected (struct kvm *kvm,gpa_t gpa,const void *data, +unsigned long len);int kvm_write_guest_cached (struct kvm *kvm,struct gfn_to_hva_cache *ghc,void *data,unsigned long len); int kvm_write_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc,
virt/kvm/kvm_main.c +27 -0
@@ -3832,6 +3832,33 @@ int kvm_write_guest(struct kvm *kvm, gpa_t gpa, const void *data, }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_write_guest); +/* Write a page KVM maintains for the guest; plane protections do not apply. */ +int kvm_write_guest_unprotected (struct kvm *kvm,gpa_t gpa,const void *data, +unsigned long len) +{ +gfn_t gfn = gpa >>PAGE_SHIFT ; +int seg; +int offset =offset_in_page (gpa); + +while ((seg =next_segment (len, offset)) !=0 ) { +struct kvm_memory_slot *slot =gfn_to_memslot (kvm, gfn); +unsigned long addr; + + addr =gfn_to_hva_many (slot, gfn,NULL ); +if (kvm_is_error_hva (addr)) +return -EFAULT ; +if (__copy_to_user ((void __user *)addr + offset, data, seg)) +return -EFAULT ; +mark_page_dirty_in_slot (kvm, slot, gfn); + offset =0 ; + len -= seg; + data += seg; + ++gfn; + } +return 0 ; +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_write_guest_unprotected); +int kvm_vcpu_write_guest (struct kvm_vcpu *vcpu,gpa_t gpa,const void *data,unsigned long len) {
Signature · 1 line
2.50.1
RFC PATCH 38/55 KVM: x86: hyperv: Publish an added reference TSC page at once
In reply to David WoodhouseFrom: Mushahid Hussain <[email protected]>
When a plane enables its reference TSC page while another plane's page is valid, KVM marks the partition's parameters changed and requests a master clock update. The update writes sequence zero to every enabled page, computes the parameters again, and writes them to every page. The parameters have not changed, so the valid pages lose their validity for nothing, and a guest that reads the time in that window falls back to the time reference count register.
Write the current parameters to the page the plane enables, the struct first and the sequence last, and leave the other pages and the status as they are. Only the first page of the partition, or a change of the parameters, takes the full update.
[dwmw2: keep the tsc_ref_from_guest guard from fbdffa08a242: parameters
read back from the plane-0 page never seed a higher plane's page; it gets a zero sequence instead, as the one-shot copy used to do. ]
--- arch/x86/kvm/hyperv.c | 44 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+)
arch/x86/kvm/hyperv.c +44 -0
@@ -1235,6 +1235,39 @@ static bool hv_tsc_page_gfn(struct kvm_hv *hv, int plane, u64 *gfn)return true; } +/* Publish the current parameters to one plane's page: struct first, then sequence. */ +static int hv_publish_tsc_page (struct kvm *kvm,unsigned int plane,u64 gfn) +{ +struct kvm_hv *hv =to_kvm_hv (kvm); +struct ms_hyperv_tsc_page page = hv->tsc_ref; +u32 tsc_seq = page.tsc_sequence; + +/* +* Parameters read back from the plane-0 page under guest TSC +* emulation are plane-0-writable; handing them to a higher plane +* would let the lowest VTL forge that plane's clock. Publish a +* zero sequence there instead: the plane falls back to the +* TIME_REF_COUNT MSR. +*/ +if (plane && hv->tsc_ref_from_guest) + tsc_seq =0 ; + + page.tsc_sequence =0 ; +if (kvm_write_guest_unprotected (kvm,gfn_to_gpa (gfn), &page,sizeof (page))) +return -EFAULT ; + +if (!tsc_seq) +return 0 ; + +/* The struct is complete before the sequence makes it valid. */ +smp_wmb (); + page.tsc_sequence = tsc_seq; +if (kvm_write_guest_unprotected (kvm,gfn_to_gpa (gfn), &page, +sizeof (page.tsc_sequence))) +return -EFAULT ; +return 0 ; +} +/* The lowest plane with an enabled page; it owns the sequence number. */ static bool hv_first_tsc_page_gfn (struct kvm *kvm,u64 *gfn) {
@@ -1542,6 +1575,17 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data,case HV_X64_MSR_REFERENCE_TSC : {u64 gfn; +/* The parameters are valid already: publish them to this page only. */ +if (!host && hv->hv_tsc_page_status ==HV_TSC_PAGE_SET && + (data &HV_X64_MSR_TSC_REFERENCE_ENABLE )) { +/* The caller holds hv_lock. */ + hv->hv_tsc_page_plane[vcpu->plane_level] = data; + gfn = data >>HV_X64_MSR_TSC_REFERENCE_ADDRESS_SHIFT ; +if (hv_publish_tsc_page (kvm, vcpu->plane_level, gfn)) + hv->hv_tsc_page_status =HV_TSC_PAGE_BROKEN ; +break ; + } + hv->hv_tsc_page_plane[vcpu->plane_level] = data;/* Another plane's page may still be enabled: keep it updated. */ if (hv_first_tsc_page_gfn (kvm, &gfn)) {
Signature · 1 line
2.50.1
RFC PATCH 39/55 KVM: x86: hyperv: Harden the per-plane reference TSC page updates
In reply to David WoodhouseFrom: Mushahid Hussain <[email protected]>
The per-plane reference TSC pages were updated through kvm_write_guest_unprotected(), which bypasses the memory attribute checks entirely: a plane writing HV_X64_MSR_REFERENCE_TSC had KVM write the first 24 bytes of an arbitrary gfn of its choosing even where that plane's own attributes forbid it, which is precisely the protection a higher VTL counts on. The sequence stream was also seeded from the lowest enabled plane's page — guest-writable memory, normally VTL0's — so VTL0 could stage S-1 on its own page (S being the sequence live on VTL1's page), force a rewrite with an MSR write and have a VTL1 reader which sampled S accept a torn body: guest-steered time in a higher trust level. And a failed update marked the whole partition HV_TSC_PAGE_BROKEN, leaving every page at sequence zero for the rest of the guest's life although the previous parameters still described a valid clock.
Access each plane's page as the plane that owns it, with kvm_plane_write_guest() and kvm_plane_read_guest() checking the given plane's attributes (not exported to vendor modules: their only callers live in kvm.ko, and the helpers bypass the usual "as the vCPU's own plane" accessors). Author the sequence host-side (hv_tsc_sequence: monotonic, never read back from a page), invalidate a page with a discrete zero-sequence write before touching its body, invalidate synchronously on guest enables, make a guest MSR write under TSC emulation take the recompute path, and invalidate any enabled page whose content does not match the adopted parameters.
A refused access or a failed parameter computation now degrades only the affected plane: its page is tracked in a per-plane hv_tsc_page_broken bitmap and skipped until the plane writes its MSR again or the next clock update retries, while the other planes' pages are still maintained. The pages are left invalid rather than republishing the previous parameters: that fallback (TIME_REF_COUNT answering from kvmclock) is always correct, and the previous scale/offset would advertise a clock that had just stopped being TSC-based. Nothing marks the partition broken any more.
[dwmw2: folded in the plane-aware accessor conversion, the host-side
sequence authorship and invalidation protocol, the per-plane broken tracking replacing the partition-wide HV_TSC_PAGE_BROKEN restore logic, and the removal of the helper exports; the original keep-the- previous-values restore was dropped because parameters read back from the plane-0 page (tsc_ref_from_guest) written to a higher plane's page with hypervisor authority would let VTL0 forge its clock.]
--- arch/x86/include/asm/kvm_host.h | 14 ++- arch/x86/kvm/hyperv.c | 216 ++++++++++++++++++++++++-------- include/linux/kvm_host.h | 6 +- virt/kvm/kvm_main.c | 48 +++++-- 4 files changed, 218 insertions(+), 66 deletions(-)
arch/x86/include/asm/kvm_host.h +12 -2
@@ -1233,8 +1233,6 @@ enum hv_tsc_page_status {HV_TSC_PAGE_HOST_CHANGED ,/* TSC page was properly set up and is currently active */ HV_TSC_PAGE_SET, -/* TSC page was set up with an inaccessible GPA */ - HV_TSC_PAGE_BROKEN, };#ifdef CONFIG_KVM_HYPERV
@@ -1260,6 +1258,18 @@ struct kvm_hv {* it exposes is not: every enabled page gets the same contents. */ u64 hv_tsc_page_plane[KVM_MAX_PLANES ]; +/* +* Planes whose reference TSC page could not be written, typically +* because the plane's own memory attributes forbid the write. Such +* a page is left alone until the plane writes its MSR again. +*/ +DECLARE_BITMAP (hv_tsc_page_broken,KVM_MAX_PLANES ); +/* +* The last sequence number published to the planes' pages. Host- +* authored and monotonic; never seeded from the (guest-writable) +* pages themselves. +*/ +u32 hv_tsc_sequence;enum hv_tsc_page_status hv_tsc_page_status;/* Hyper-v based guest crash (NT kernel bugcheck) parameters */
arch/x86/kvm/hyperv.c +166 -50
318 lines not on this page · raw message
include/linux/kvm_host.h +4 -2
@@ -1551,8 +1551,10 @@ int kvm_write_guest_page(struct kvm *kvm, gfn_t gfn, const void *data, int offset, int len);int kvm_write_guest (struct kvm *kvm,gpa_t gpa,const void *data,unsigned long len); -int kvm_write_guest_unprotected (struct kvm *kvm,gpa_t gpa,const void *data, -unsigned long len); +int kvm_plane_write_guest (struct kvm_plane *plane,gpa_t gpa,const void *data, +unsigned long len); +int kvm_plane_read_guest (struct kvm_plane *plane,gpa_t gpa,void *data, +unsigned long len);int kvm_write_guest_cached (struct kvm *kvm,struct gfn_to_hva_cache *ghc,void *data,unsigned long len); int kvm_write_guest_offset_cached(struct kvm *kvm, struct gfn_to_hva_cache *ghc,
virt/kvm/kvm_main.c +36 -12
67 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 40/55 KVM: x86: Implement Hyper-V's vCPU suspended state
In reply to David WoodhouseFrom: Nikolas Wipper <[email protected]>
Introduce a suspension state for Hyper-V enlightened vCPUs. Microsoft's "Hypervisor Top Level Functional Specification" (TLFS) introduces this state as a "vCPU that is stopped on a instruction guest boundary, either explicitly or implicitly due to an intercept". The only documented explicit suspension is caused in response to a TLB Flush hypercall, which will use the state switching API in subsequent patches.
Each Hyper-V vCPU now has a 'suspended' boolean field, which is checked before entering the guest. When set, it forces the vCPU to block. Once in that state, the vCPU ignores any events. The vCPU is unsuspended by clearing 'suspend' and issuing a request to force the vCPU out of sleep.
Suspensions are issued as a mechanism to halt execution until state change is observed on a remote vCPU. Hyper-V vCPUs track this with 'waiting_on', which holds the 'vcpu_id' of the remote vCPU that forced the vCPU to enter the suspended state. It's the remote vCPU's responsibility to wake up the suspended vCPUs when ready. 'waiting_on' ensures the remote vCPU can selectively unsuspend vCPUs that blocked on its behalf while leaving other suspended vCPUs undisturbed. One vCPU can only be suspended due to a single remote vCPU, but different vCPUs can be suspended on behalf of different or the same remote vCPU(s). The guest is responsible for avoiding circular dependencies between suspended vCPUs.
Callers of the suspend API are responsible for ensuring that suspend and unsuspend aren't called in parallel while targeting the same pair of vCPUs. Otherwise kvm_hv_vcpu_{un}suspend_tlb_flush() ensure 'waiting_on' and 'suspended' are updated and accessed in the correct order. This, for example, avoids races where the unsuspended vCPU re-suspends before kvm_hv_vcpu_unsuspend_tlb_flush() is done updating 'waiting_on'.
(cherry picked from the posted series, re-done by hand) [hmushi: struct kvm_vcpu_hv now lives in arch/x86/kvm/hyperv.h;
bitmap_zero(vcpu_mask) added (the original used an uninitialised stack bitmap)]
(re-done by hand from the reference VSM implementation) [dwmw2: struct kvm_vcpu_hv is in arch/x86/include/asm/kvm_host.h on
this branch. Unsuspend walks the suspended waiters' own plane and kicks each one directly: kvm_make_vcpus_request_mask() resolves indices against plane 0, which is the wrong twin for a waiter in a higher plane.]
--- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/hyperv.c | 28 ++++++++++++++++++++++++++++ arch/x86/kvm/hyperv.h | 15 +++++++++++++++ arch/x86/kvm/x86.c | 2 +- 4 files changed, 47 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +3 -0
@@ -761,6 +761,9 @@ struct kvm_vcpu_hv {u64 vtl_entry_rax;u64 vtl_entry_rcx;bool vtl_entry_regs_valid; + +bool suspended; +int waiting_on; };struct kvm_hypervisor_cpuid {
arch/x86/kvm/hyperv.c +28 -0
@@ -1040,6 +1040,7 @@ int kvm_hv_vcpu_init(struct kvm_vcpu *vcpu) vcpu->arch.hyperv = hv_vcpu; hv_vcpu->vcpu = vcpu; + hv_vcpu->waiting_on =-1 ;synic_init (&hv_vcpu->synic);
@@ -3764,3 +3765,30 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,return 0 ; } + +void kvm_hv_vcpu_suspend_tlb_flush (struct kvm_vcpu *vcpu,int vcpu_id) +{ +/* waiting_on's store should happen before suspended's */ +WRITE_ONCE (vcpu->arch.hyperv->waiting_on, vcpu_id); +WRITE_ONCE (vcpu->arch.hyperv->suspended, true); +} + +void kvm_hv_vcpu_unsuspend_tlb_flush (struct kvm_vcpu *vcpu) +{ +struct kvm_vcpu_hv *vcpu_hv; +struct kvm_vcpu *v; +unsigned long i; + +plane_for_each_vcpu (i, v, vcpu->plane) { + vcpu_hv =to_hv_vcpu (v); + +if (vcpu_hv &&kvm_hv_vcpu_suspended (v) && +READ_ONCE (vcpu_hv->waiting_on) == vcpu->vcpu_id) { +/* waiting_on's store should happen before suspended's */ +WRITE_ONCE (vcpu_hv->waiting_on,-1 ); +WRITE_ONCE (vcpu_hv->suspended, false); +kvm_make_request (KVM_REQ_EVENT , v); +kvm_vcpu_kick (v); + } + } +}
arch/x86/kvm/hyperv.h +15 -0
@@ -228,6 +228,15 @@ static inline bool guest_hv_cpuid_has_l2_tlb_flush(struct kvm_vcpu *vcpu) (hv_vcpu->cpuid_cache.nested_eax &HV_X64_NESTED_DIRECT_FLUSH ); } +void kvm_hv_vcpu_suspend_tlb_flush (struct kvm_vcpu *vcpu,int vcpu_id); +void kvm_hv_vcpu_unsuspend_tlb_flush (struct kvm_vcpu *vcpu); + +static inline bool kvm_hv_vcpu_suspended (struct kvm_vcpu *vcpu) +{ +return vcpu->arch.hyperv_enabled && +READ_ONCE (vcpu->arch.hyperv->suspended); +} +static inline bool kvm_hv_is_tlb_flush_hcall (struct kvm_vcpu *vcpu) {struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu);
@@ -309,6 +318,12 @@ static inline bool kvm_hv_has_stimer_pending(struct kvm_vcpu *vcpu) {return false; } +static inline void kvm_hv_vcpu_suspend_tlb_flush (struct kvm_vcpu *vcpu,int vcpu_id) {} +static inline void kvm_hv_vcpu_unsuspend_tlb_flush (struct kvm_vcpu *vcpu) {} +static inline bool kvm_hv_vcpu_suspended (struct kvm_vcpu *vcpu) +{ +return false; +}static inline int kvm_hv_verify_vp_assist (struct kvm_vcpu *vcpu) {return 0 ;
arch/x86/kvm/x86.c +1 -1
@@ -12182,7 +12182,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)static bool kvm_vcpu_running (struct kvm_vcpu *vcpu) {return (vcpu->arch.mp_state ==KVM_MP_STATE_RUNNABLE && - !vcpu->arch.apf.halted); + !vcpu->arch.apf.halted && !kvm_hv_vcpu_suspended (vcpu)); }bool kvm_vcpu_has_events (struct kvm_vcpu *vcpu)
Signature · 1 line
2.50.1
RFC PATCH 41/55 KVM: x86: hyperv: Refuse a synthetic IPI to a higher VTL
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
HvCallSendSyntheticClusterIpi(Ex) honours the target VTL in its input but does not check it against the caller's VTL. A guest running in a lower VTL can therefore assert any fixed interrupt in a higher VTL on any VP, which crosses the boundary that VTLs exist to enforce.
Refuse a target VTL above the caller's with HV_STATUS_ACCESS_DENIED. A VTL may still interrupt its own level and lower ones. The old upper-bound check on the target plane goes away: a target at or below the caller's level is below KVM_MAX_PLANES by construction.
Matches the equivalent fix in the reference VSM implementation. The production Nitro Hypervisor permits the injection; the conformance suite probes for either behaviour.
--- arch/x86/kvm/hyperv.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-)
arch/x86/kvm/hyperv.c +13 -5
@@ -2710,13 +2710,21 @@ static int kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)if ((vector <HV_IPI_LOW_VECTOR ) || (vector >HV_IPI_HIGH_VECTOR ))return HV_STATUS_INVALID_HYPERCALL_INPUT ; -/* TLFS: in_vtl bit 4 selects an explicit target VTL. */ -if (in_vtl &0x10 ) +/* +* TLFS: in_vtl bit 4 selects an explicit target VTL. A VTL may +* interrupt its own level and lower ones, never a higher one +* (TLFS 15.9, "VTL Interrupt Management": each VTL sends and +* receives interrupts without interference from a less secure +* VTL). The reference host omits this check; the suite's +* SendIpi test probes for both behaviours. +*/ +if (in_vtl &0x10 ) { target_plane = in_vtl &0xf ; -else +if (target_plane > vcpu->plane_level) +return HV_STATUS_ACCESS_DENIED ; + }else { target_plane = vcpu->plane_level; -if (target_plane >=KVM_MAX_PLANES ) -return HV_STATUS_INVALID_HYPERCALL_INPUT ; + }if (all_cpus)kvm_hv_send_ipi_to_many (kvm, vector,NULL ,0 , target_plane);
Signature · 1 line
2.50.1
RFC PATCH 42/55 KVM: x86: Gate forced VTL entry on interrupt deliverability
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
A fixed vector posted to a higher plane forces a VTL entry even when the higher VTL's TPR masks it, so the secure kernel is entered with reason INTERRUPT and nothing to deliver. The TLFS (15.9, "VTL Interrupt Management") holds such an interrupt pending without
switching: only a deliverable interrupt preempts the lower VTL.
Gate the forced-entry evaluation on the higher plane's APIC reporting a deliverable vector under its PPR. A masked vector keeps its posted marker instead of consuming the single entry attempt, so the entry fires if a later VTL round trip leaves the vector deliverable; the higher plane still consumes its own marker when it runs. Each marker is consumed only by its own approved clause, in both the userspace exit and the in-kernel refusal paths: an exit triggered by extint, a stimer or the LAPIC timer while the posted vector is still masked leaves the APIC marker armed.
NMIs move to a marker of their own, hv_nmi_event_pending, and keep forcing entry unconditionally: the NMI is folded back to plane 0 at acceptance, so the higher plane's APIC carries nothing the PPR gate could see. The marker is armed wherever NMI state is posted to a non-zero plane (kvm_inject_nmi() and the vCPU-events ioctl, which is allowlisted on plane fds), and the level-triggered NMI state (KVM_REQ_NMI, nmi_queued, nmi_pending) does not feed the gate: a posting earns exactly one wakeup, like the APIC event path, instead of keeping the gate permanently true for a parked higher plane.
The in-kernel switch consumes the markers on a refused entry attempt too. The only refusal left on that path is a failing VTL-control write (blocked planes are filtered in kvm_higher_plane_with_event()), which stays failed until an overlay or memory-attribute change; the event remains latched in the plane's own pending state for its next entry, rather than repeating the failing guest write on every exit of the lower plane and keeping the vCPU from ever blocking.
Matches the deliverability gating of the reference VSM implementation (its kvm_plane_has_interrupt); the production Nitro Hypervisor remains PPR-blind, and the conformance suite's TPR-masked test probes for either behaviour.
--- arch/x86/include/asm/kvm_host.h | 21 +++++-- arch/x86/kvm/lapic.c | 2 +- arch/x86/kvm/x86.c | 104 ++++++++++++++++++++++++-------- 3 files changed, 96 insertions(+), 31 deletions(-)
arch/x86/include/asm/kvm_host.h +16 -5
@@ -861,14 +861,25 @@ void kvm_arch_vcpu_common_destroy(struct kvm_vcpu_common *common);struct kvm_vcpu_arch {/* -* Hyper-V VSM: an APIC event (fixed vector or NMI) was delivered to -* this plane while it was suspended. Set at delivery, cleared when -* the plane next runs; a lower plane's run loop uses it to force a -* VTL entry even when the vector is currently masked by this -* plane's TPR (matching the reference host's apic_pending_vtls). +* Hyper-V VSM: a fixed APIC vector was delivered to this plane +* while it was suspended. Set at delivery, cleared when the plane +* next runs; a lower plane's run loop uses it to force a VTL entry +* once the vector is deliverable under this plane's PPR (TLFS 15.9: +* an interrupt the higher VTL's priority masks is held pending and +* the VTL does not switch). While masked, the marker is held, so +* the entry fires when a VTL round trip leaves the vector +* deliverable. */ bool hv_apic_event_pending; +/* +* Hyper-V VSM: an NMI was aimed at this (higher) plane. Forces a +* VTL entry unconditionally: the NMI itself is folded back to +* plane 0 (reference host behaviour), so the higher plane's APIC +* carries no trace of it and the PPR gate above cannot apply. +*/ +bool hv_nmi_event_pending; +/* * rip and regs accesses must go through * kvm_{register,rip}_{read,write} functions.
arch/x86/kvm/lapic.c +1 -1
@@ -1486,7 +1486,7 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode,struct kvm_vcpu *twin;int p; -WRITE_ONCE (vcpu->arch.hv_apic_event_pending, true); +WRITE_ONCE (vcpu->arch.hv_nmi_event_pending, true);if (vcpu->common->vcpus[0 ])kvm_inject_nmi (vcpu->common->vcpus[0 ]);for (p =0 ; p < vcpu->plane_level; p++) {
arch/x86/kvm/x86.c +79 -25
148 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 43/55 KVM: x86: Derive runtime CPUID bits from the executing plane
In reply to David WoodhouseFrom: Mushahid Hussain <[email protected]>
Some CPUID bits follow registers of the vCPU: CPUID.1:ECX.OSXSAVE follows CR4.OSXSAVE, CPUID.7:ECX.OSPKE follows CR4.PKE, CPUID.1:EDX.APIC follows the APIC enable, CPUID.1:ECX.MWAIT follows MISC_ENABLE, and the XSAVE area sizes in leaf 0xD follow XCR0 and XSS. KVM keeps the CPUID table of a vCPU in the state its planes share, and refreshes those bits into the table from the state of whichever vCPU runs when the table is dirty. The planes of a vCPU have their own control registers, so after one plane sets CR4.OSXSAVE, CPUID on every plane of that vCPU reports OSXSAVE until a plane with it clear refreshes the table, and the other way round. The architecture requires CPUID to describe the processor that executes it, and for a plane that is the plane's own registers.
A Windows guest with Virtual Secure Mode shows the leak. Its secure kernel runs on a plane whose CR4.OSXSAVE is clear while the normal kernel on the other plane has set it. The secure kernel executes the usual probe, CPUID.1 followed by a test of OSXSAVE, reads the bit set and executes XGETBV, which raises #UD because its own CR4 does not permit it.
Compute the bits that follow vCPU registers when CPUID is emulated, from the registers of the vCPU that executes it, and no longer refresh the shared table on that path. The table keeps the view of plane 0 for KVM_GET_CPUID2 and KVM_SET_CPUID2, which act on the plane 0 vCPU, and their refresh stays as it is.
(re-done by hand from the reference VSM implementation) [dwmw2: dropped the plane selftest hunk; this branch has no
plane_x86_test.c. The per-VTL OSXSAVE contract is pinned by the VSM conformance suite instead.]
--- Documentation/virt/kvm/api.rst | 7 +++ arch/x86/kvm/cpuid.c | 108 +++++++++++++++++++++++---------- arch/x86/kvm/cpuid.h | 5 +- 3 files changed, 87 insertions(+), 33 deletions(-)
Documentation/virt/kvm/api.rst +7 -0
@@ -370,6 +370,13 @@ cpu's hardware control block. VCPUs for non-zero planes can only be created if the VCPU the same index has already been created for plane zero. +The planes of a vCPU share its CPUID model, set through the plane 0 fd with +KVM_SET_CPUID2. The CPUID bits that follow a vCPU's own registers (OSXSAVE +from CR4.OSXSAVE, OSPKE from CR4.PKE, the APIC enable, MWAIT from +MISC_ENABLE, and the XSAVE area sizes from XCR0 and XSS) reflect the plane +that executes CPUID, since every plane has these registers of its own. +KVM_GET_CPUID2 returns them as plane 0 sees them. + 4.8 KVM_GET_DIRTY_LOG ---------------------
arch/x86/kvm/cpuid.c +77 -31
156 lines not on this page · raw message
arch/x86/kvm/cpuid.h +3 -2
@@ -273,8 +273,9 @@ static __always_inline bool guest_cpu_cap_has(struct kvm_vcpu *vcpu,struct kvm_vcpu_common *common = vcpu->common;/* -* Except for MWAIT, querying dynamic feature bits is disallowed, so -* that KVM can defer runtime updates until the next CPUID emulation. +* Except for MWAIT, querying dynamic feature bits is disallowed: the +* guest gets them from the state of the plane that executes CPUID, and +* the table copy is refreshed only for the CPUID ioctls. */ BUILD_BUG_ON(x86_feature == X86_FEATURE_APIC || x86_feature == X86_FEATURE_OSXSAVE ||
Signature · 1 line
2.50.1
RFC PATCH 44/55 KVM: selftests: Test hugepage sizing over mixed protection attributes
In reply to David WoodhouseFrom: Mushahid Hussain <[email protected]>
Back a 2 MiB-aligned region with a transparent hugepage and give it alternating NX and NW protection attributes on its 4 KiB pages. The guest reads an NX page, which maps the region, then writes the NW page next to it: the write must exit with KVM_EXIT_MEMORY_FAULT and leave the page unchanged, instead of going through a 2 MiB leaf that inherited the NX page's permissions. Clearing the attribute lets the write through.
This pins the hugepage side of the VSM memory protection boundary: a single 4 KiB protection set by a higher VTL must not be widened away by a neighbouring gfn's fault installing a hugepage whose access was computed from the neighbour alone. The host refuses that hugepage via the KVM_LPAGE_MIXED_FLAG tracking which kvm_arch_post_set_memory_attributes() maintains over the whole attribute word, NR/NW/NX included.
[dwmw2: re-done by hand from Mushahid's original, adapted from the
KVM_SET_PLANE_MEMORY_ATTRIBUTES uAPI to the per-plane memory attributes design: plane 0 attributes are set through KVM_SET_MEMORY_ATTRIBUTES with the NR/NW/NX protection bits, no plane runtime is needed, and the fault exit carries no plane field]
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/x86/memattr_hugepage_test.c | 132 ++++++++++++++++++ 2 files changed, 133 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/memattr_hugepage_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -92,6 +92,7 @@ TEST_GEN_PROGS_x86 += x86/hyperv_tlb_flush TEST_GEN_PROGS_x86 += x86/kvm_clock_test TEST_GEN_PROGS_x86 += x86/kvm_pv_test TEST_GEN_PROGS_x86 += x86/kvm_buslock_test +TEST_GEN_PROGS_x86 += x86/memattr_hugepage_test TEST_GEN_PROGS_x86 += x86/monitor_mwait_test TEST_GEN_PROGS_x86 += x86/msrs_test TEST_GEN_PROGS_x86 += x86/nested_close_kvm_test
tools/testing/selftests/kvm/x86/memattr_hugepage_test.c new file +132 -0
132 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 45/55 KVM: selftests: Test a plane suspending with an undelivered interrupt
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
Pin the forced-VTL-entry re-arm: a plane-1 twin self-IPIs with IF clear, runs on (consuming its single-shot hv_apic_event_pending marker), and is then suspended by KVM_SET_PLANE_RUN_STATE exactly like a userspace VTL switch. Running plane 0 must report the suspended twin's deliverable interrupt as a KVM_EXIT_HYPERV_VTL_INTERRUPT exit, and the re-entered twin must receive the vector once it sets IF. On a kernel without the re-arm, plane 0 reaches its guest code and the twin's interrupt is stranded in its IRR, which is the Windows VSM hibernate deadlock in miniature.
The default selftest VM owns a fully in-kernel irqchip, which planes refuse; add a kvm_force_split_irqchip override to the library (as in the public planes tree) so a planes test can use the stock VM setup.
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../testing/selftests/kvm/include/kvm_util.h | 1 + tools/testing/selftests/kvm/lib/kvm_util.c | 11 + .../selftests/kvm/x86/planes_vtl_entry_test.c | 188 ++++++++++++++++++ 4 files changed, 201 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/planes_vtl_entry_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -107,6 +107,7 @@ TEST_GEN_PROGS_x86 += x86/nested_vmsave_vmload_test TEST_GEN_PROGS_x86 += x86/platform_info_test TEST_GEN_PROGS_x86 += x86/plane_x86_test TEST_GEN_PROGS_x86 += x86/planes_test +TEST_GEN_PROGS_x86 += x86/planes_vtl_entry_test TEST_GEN_PROGS_x86 += x86/pmu_counters_test TEST_GEN_PROGS_x86 += x86/pmu_event_filter_test TEST_GEN_PROGS_x86 += x86/private_mem_conversions_test
tools/testing/selftests/kvm/include/kvm_util.h +1 -0
@@ -713,6 +713,7 @@ static inline bool is_smt_on(void)return false; } +extern bool kvm_force_split_irqchip;void vm_create_irqchip (struct kvm_vm *vm);/*
tools/testing/selftests/kvm/lib/kvm_util.c +11 -0
@@ -1696,11 +1696,22 @@ void *addr_gpa2alias(struct kvm_vm *vm, gpa_t gpa)return (void *) ((uintptr_t ) region->host_alias + offset); } +/* +* Tests that need a split IRQ chip (x86 only), e.g. for VM planes, set this +* around VM creation to override the fully in-kernel default. +*/ +bool kvm_force_split_irqchip; +/* Create an interrupt controller chip for the specified VM. */ void vm_create_irqchip (struct kvm_vm *vm) {int r; +if (kvm_force_split_irqchip &&kvm_has_cap (KVM_CAP_SPLIT_IRQCHIP )) { +vm_enable_split_irqchip (vm); +return ; + } +/* * Allocate a fully in-kernel IRQ chip by default, but fall back to a * split model (x86 only) if that fails (KVM x86 allows compiling out
tools/testing/selftests/kvm/x86/planes_vtl_entry_test.c new file +188 -0
188 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 46/55 KVM: x86: hyperv: Do not skip a moved vCPU when completing a hypercall
In reply to David WoodhouseFrom: Mushahid Hussain <[email protected]>
A Hyper-V hypercall that exits to userspace is completed on the next KVM_RUN of the vCPU, and the completion skips the VMCALL. With planes, the lower plane is not run again until the secure kernel returns, and userspace applies the registers the secure kernel sets for it in between. On a resume from hibernation the secure kernel sets VTL0's RIP to the hibernate continuation. The deferred completion then skips an instruction on top of that RIP.
The two vendors fail differently. SVM writes the stale next_rip of the old VMCALL exit, so the VP resumes at the hypercall page's RET with the new RSP, pops a zero and faults at RIP 0: Windows Server 2025 with Credential Guard bugchecks 0xD1 (0, 0xff, 0, 0) on every resume on AMD. VMX adds the VMCALL length to the new RIP and lands three bytes into PopSaveHiberContextWrapper+0x35, where the bytes happen to decode as a harmless instruction before resynchronising, so Intel passes by luck.
Record the RIP of the hypercall when exiting to userspace and complete without skipping when the vCPU is no longer there. Applies to both the VTL switch completion and the generic userspace completion. The generic KVM_EXIT_HYPERCALL and Xen completions have the same shape but userspace does not move the vCPU there; on arm64 the HVC is skipped at exit time, so the planes series is unaffected there.
Measured on an m6a.metal (AMD) planes bench, WS2025 CG hibernate:
SK SETs VTL0 RIP = wrapper+0x35, RSP = cont; first plane-0 exits after the return are at hypercall page+3, then RIP 0; KiBugCheckData 0xD1.
[dwmw2: this stack retires a forwarded VtlCall/VtlReturn eagerly at
exit time with no pending completion, so that path is unexposed; the exposures are the FORWARD_HCALL completion and the plane an in-kernel VTL switch suspends at its vmcall, whose RIP the higher VTL rewrites via HvCallSetVpRegisters. Both resume through kvm_hv_hypercall_complete_userspace(), so one moved-check there covers both; the RIP is recorded in kvm_hv_hypercall_userspace_exit() and kvm_hv_vtl_switch(). The check yields to the TLB-flush suspension retry (a suspended vCPU re-executes the instruction, so there is no skip to suppress).]
(re-done by hand)
[dwmw2: folded in the RAISE_UD completion-contract tightening: the
KVM_CAP_HYPERV_FORWARD_HCALL extension check returns 2 to advertise that the kernel honours KVM_HV_HCALL_RESULT_RAISE_UD, the bit is honoured only while forwarding is enabled (the legacy syndbg and slow-form completions treat bit 63 as part of the result) and only after the suspended and moved guards, and an in-kernel-armed completion is tagged with hcall_resume_pending so it re-executes the hypercall instead of consuming stale run-union bytes left by an unrelated exit.]
--- Documentation/virt/kvm/api.rst | 6 +++ arch/x86/include/asm/kvm_host.h | 17 ++++++++ arch/x86/kvm/hyperv.c | 77 ++++++++++++++++++++++++++++++--- arch/x86/kvm/x86.c | 11 ++++- include/uapi/linux/kvm.h | 4 +- 5 files changed, 106 insertions(+), 9 deletions(-)
Documentation/virt/kvm/api.rst +6 -0
@@ -9146,6 +9146,12 @@ with ``hyperv.u.hcall.result`` set. Hypercalls KVM implements itself, and the ordering of CPUID-enforcement checks against hypercall dispatch, are unaffected. +When KVM_CHECK_EXTENSION reports a value of 2 or more, a result with +KVM_HV_HCALL_RESULT_RAISE_UD set makes KVM raise #UD at the hypercall +instruction instead of completing it (honoured only while forwarding +is enabled). A VMM must not set that bit on kernels reporting 1, +where it lands in the guest's RAX as part of the result. + 7.49 KVM_CAP_HCALL_FAULT_EXIT ------------------------------------
arch/x86/include/asm/kvm_host.h +17 -0
@@ -762,6 +762,23 @@ struct kvm_vcpu_hv {u64 vtl_entry_rcx;bool vtl_entry_regs_valid; +/* +* Linear RIP of the hypercall instruction whose deferred +* completion (userspace exit or in-kernel VTL switch suspension) +* is pending; linear so that a CS.base change while the vCPU is +* parked cannot alias the recorded offset. +*/ +u64 hcall_exit_rip; + +/* +* Set when an in-kernel VTL switch armed the deferred completion: +* the resume value only lands in run->hyperv.u.hcall.result when +* the opposite-direction switch provides it. While set, the run +* union holds unrelated bytes and the completion must retry the +* hypercall instead of consuming them. +*/ +bool hcall_resume_pending; +bool suspended;int waiting_on; };
arch/x86/kvm/hyperv.c +70 -7
125 lines not on this page · raw message
arch/x86/kvm/x86.c +10 -1
@@ -4937,7 +4937,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_HYPERV_CPUID :case KVM_CAP_HYPERV_ENFORCE_CPUID :case KVM_CAP_SYS_HYPERV_CPUID : -case KVM_CAP_HYPERV_FORWARD_HCALL :case KVM_CAP_HYPERV_VTL_SWITCH :case KVM_CAP_HYPERV_REG_INTERCEPT :#endif
@@ -4985,6 +4984,16 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_ONE_REG : r =1 ;break ; +#ifdef CONFIG_KVM_HYPERV +case KVM_CAP_HYPERV_FORWARD_HCALL : +/* +* 2: KVM_HV_HCALL_RESULT_RAISE_UD is honoured on hypercall +* completion. A VMM must not set the bit on kernels +* reporting 1, where it lands in guest RAX instead. +*/ + r =2 ; +break ; +#endif case KVM_CAP_PRE_FAULT_MEMORY : r = tdp_enabled;break ;
include/uapi/linux/kvm.h +3 -1
@@ -1038,7 +1038,9 @@ struct kvm_enable_cap {* Userspace completion of a forwarded Hyper-V hypercall (KVM_EXIT_HYPERV_HCALL * under KVM_CAP_HYPERV_FORWARD_HCALL): setting this bit in the result makes * KVM raise #UD at the faulting instruction instead of completing the -* hypercall; all other result bits are then ignored. +* hypercall; all other result bits are then ignored. Only honoured while +* forwarding is enabled, and only on kernels where +* KVM_CHECK_EXTENSION(KVM_CAP_HYPERV_FORWARD_HCALL) returns >= 2. */ #define KVM_HV_HCALL_RESULT_RAISE_UD (1ULL << 63)#define KVM_CAP_HCALL_FAULT_EXIT 252
Signature · 1 line
2.50.1
RFC PATCH 47/55 KVM: x86: Allow KVM_X86_SETUP_MCE on plane vCPUs
In reply to David WoodhouseFrom: Mushahid Hussain <[email protected]>
mcg_cap is per vCPU. kvm_arch_vcpu_create() gives every vCPU the KVM_MAX_MCE_BANKS default, plane twins included, but KVM_X86_SETUP_MCE was only accepted on the plane-0 fd: userspace programs plane 0 and a higher plane keeps running with the default, so the MCG view diverges across planes. Hyper-V defines the MCG state as shared across Virtual Trust Levels, so userspace needs to program the same value on every
plane: accept the ioctl on plane vCPUs.
[dwmw2: ours gates plane vCPU fd ioctls in kvm_arch_is_vcpu_plane_ioctl()
in the same shape; same one-line addition.]
(re-done by hand)
--- Documentation/virt/kvm/api.rst | 2 +- arch/x86/kvm/x86.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
Documentation/virt/kvm/api.rst +1 -1
@@ -4594,7 +4594,7 @@ capabilities will have the corresponding bits set. :Capability: KVM_CAP_MCE :Architectures: x86 -:Type: vcpu ioctl +:Type: vcpu ioctl (all planes) :Parameters: u64 mcg_cap (in) :Returns: 0 on success, -EFAULT if u64 mcg_cap cannot be read,
arch/x86/kvm/x86.c +1 -0
@@ -6400,6 +6400,7 @@ bool kvm_arch_is_vcpu_plane_ioctl(unsigned ioctl)case KVM_SET_XSAVE :case KVM_GET_REG_LIST :case KVM_TRANSLATE : +case KVM_X86_SETUP_MCE :return true;default :return false;
Signature · 1 line
2.50.1
RFC PATCH 48/55 KVM: x86: hyperv: Advertise XMM fast hypercall output
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The Hyper-V VSM register hypercalls return data in XMM registers when the guest uses their fast forms, and the userspace handler answering them writes those registers through the vCPU FPU state. Report HV_X64_HYPERCALL_XMM_OUTPUT_AVAILABLE in KVM_GET_SUPPORTED_HV_CPUID so the VMM can expose it; the reference hypervisor advertises it, and the Windows secure kernel only uses the fast forms when it is present.
--- arch/x86/kvm/hyperv.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kvm/hyperv.c +1 -0
@@ -3751,6 +3751,7 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, ent->ebx |=HV_ENABLE_EXTENDED_HYPERCALLS ; ent->edx |=HV_X64_HYPERCALL_XMM_INPUT_AVAILABLE ; + ent->edx |=HV_X64_HYPERCALL_XMM_OUTPUT_AVAILABLE ; ent->edx |=HV_FEATURE_FREQUENCY_MSRS_AVAILABLE ; ent->edx |=HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE ;
Signature · 1 line
2.50.1
RFC PATCH 49/55 KVM: SVM: Carry instruction state on reflected plane-protection faults
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
A plane-protection fault reflected to userspace becomes a TLFS GPA intercept message, which carries the faulting instruction's length, its bytes (for data accesses), and the fault-adjusted RFLAGS. The VMX exit handler harvests all of that from the VMCS, but npf_interception() reflected the bare fault: on AMD the higher VTL received intercept messages with instruction length 0, no instruction bytes and RFLAGS.RF clear, failing the hyperv_vsm KUT intercept-message tests ("Seen an expected read/write intercept message").
Decode assists stash the raw fetch window (up to 15 bytes, not the decoded length) in the VMCB for data accesses; run it through the instruction decoder and report the decoded length and that many bytes. Set RFLAGS.RF before exiting: fault delivery gives the restarted instruction RF=1, and VMX saves that fault-adjusted value in guest RFLAGS on fault-type exits while SVM saves RF as it was during execution.
An execute fault stores no bytes, and the length of the branch that fetched the faulting page is not architecturally available on SVM (VMX hardware reports it), so execute intercepts carry length 0.
The validity of the harvested state gets its own flag, KVM_MEMORY_EXIT_FLAG_INSN_VALID, set by the VMX and SVM harvests only when they populated the fields: VM_EXIT_INSTRUCTION_LEN is undefined for EPT violations, all-zero insn_bytes is itself a valid x86 encoding, and kvm_prepare_memory_fault_exit() now clears gva/insn_len/insn_bytes so no other memory-fault exit carries stale values. The flag bit (and PLANE_PROT before it) is enumerated by the plane memory overlay capability, which also covers the extended memory_fault layout. The memory_attributes selftest asserted strict equality on the exit flags; it now masks the informational flags out and compares only the access that faulted.
--- arch/x86/kvm/mmu/mmu.c | 2 + arch/x86/kvm/mmu/paging_tmpl.h | 3 + arch/x86/kvm/svm/svm.c | 66 +++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 13 ++-- include/linux/kvm_host.h | 12 ++++ include/uapi/linux/kvm.h | 17 ++++- .../testing/selftests/kvm/memory_attributes.c | 6 +- 7 files changed, 112 insertions(+), 7 deletions(-)
arch/x86/kvm/mmu/mmu.c +2 -0
@@ -4787,6 +4787,8 @@ static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu,if (kvm_faultin_memory_protections (vcpu, fault)) {kvm_mmu_prepare_memory_fault_exit (vcpu, fault); +if (kvm_arch_max_planes (kvm) >1 ) + vcpu->run->memory_fault.flags |=KVM_MEMORY_EXIT_FLAG_PLANE_PROT ;return -EFAULT ; }
arch/x86/kvm/mmu/paging_tmpl.h +3 -0
@@ -795,6 +795,9 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *faultif (walker.memory_attributes_fault) {kvm_prepare_memory_fault_exit (vcpu, walker.gfn <<PAGE_SHIFT ,PAGE_SIZE , false, false, false); +if (kvm_arch_max_planes (vcpu->kvm) >1 ) + vcpu->run->memory_fault.flags |= +KVM_MEMORY_EXIT_FLAG_PLANE_PROT ;return -EFAULT ; }
arch/x86/kvm/svm/svm.c +66 -0
72 lines not on this page · raw message
arch/x86/kvm/vmx/vmx.c +8 -5
@@ -5987,9 +5987,7 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu)*/ if (ret == -EFAULT && vcpu->run->exit_reason ==KVM_EXIT_MEMORY_FAULT && - (vcpu->run->memory_fault.flags & (KVM_MEMORY_EXIT_FLAG_READ | -KVM_MEMORY_EXIT_FLAG_WRITE | -KVM_MEMORY_EXIT_FLAG_EXEC ))) { + (vcpu->run->memory_fault.flags &KVM_MEMORY_EXIT_FLAG_PLANE_PROT )) {unsigned int len;if (exit_qualification &EPT_VIOLATION_GVA_IS_VALID ) {
@@ -6007,7 +6005,6 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu) len =vmcs_read32 (VM_EXIT_INSTRUCTION_LEN ); len = min_t(unsigned int, len,sizeof (vcpu->run->memory_fault.insn_bytes)); - vcpu->run->memory_fault.insn_len = len;if (!(exit_qualification &EPT_VIOLATION_ACC_INSTR )) {struct x86_exception e;
@@ -6019,10 +6016,16 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu)*/ if (kvm_read_guest_virt (vcpu,kvm_get_linear_rip (vcpu), vcpu->run->memory_fault.insn_bytes, - len, &e)) + len, &e)) {memset (vcpu->run->memory_fault.insn_bytes,0 ,sizeof (vcpu->run->memory_fault.insn_bytes)); + len =0 ; + } } + vcpu->run->memory_fault.insn_len = len; +if (len) + vcpu->run->memory_fault.flags |= +KVM_MEMORY_EXIT_FLAG_INSN_VALID ; }return ret; }
include/linux/kvm_host.h +12 -0
@@ -2755,6 +2755,18 @@ static inline void kvm_prepare_memory_fault_exit(struct kvm_vcpu *vcpu, vcpu->run->memory_fault.gpa = gpa; vcpu->run->memory_fault.size = size; +/* +* Clear the extended fields: only the x86 plane-protection harvest +* fills them in (with the matching validity flags), and a leftover +* from an earlier exit would otherwise masquerade as this fault's +* instruction. All-zero insn_bytes is a valid x86 encoding, so +* zeroing alone is not a distinguishable "not provided" marker. +*/ + vcpu->run->memory_fault.gva =0 ; + vcpu->run->memory_fault.insn_len =0 ; +memset (vcpu->run->memory_fault.insn_bytes,0 , +sizeof (vcpu->run->memory_fault.insn_bytes)); + vcpu->run->memory_fault.flags =0 ;if (is_write)
include/uapi/linux/kvm.h +16 -1
@@ -481,10 +481,25 @@ struct kvm_run {#define KVM_MEMORY_EXIT_FLAG_EXEC (1ULL << 2)#define KVM_MEMORY_EXIT_FLAG_PRIVATE (1ULL << 3)#define KVM_MEMORY_EXIT_FLAG_GVA_VALID (1ULL << 4) +/* The fault was caused by the vCPU's plane's memory protections. */ +#define KVM_MEMORY_EXIT_FLAG_PLANE_PROT (1ULL << 5) +/* +* insn_len/insn_bytes are valid. For an EXEC fault the faulting +* instruction itself cannot be read: insn_bytes stays zero, and +* insn_len is the length of the instruction which performed the +* fetch (reported by VMX hardware; not available on SVM). +*/ +#define KVM_MEMORY_EXIT_FLAG_INSN_VALID (1ULL << 6)__u64 flags;__u64 gpa;__u64 size; -/* set on plane-protection faults when available */ +/* +* gva/insn_len/insn_bytes are set on plane-protection +* faults when available, and are zero on every other +* memory-fault exit. They are enumerated, along with +* the PLANE_PROT and INSN_VALID flags, by the plane +* memory overlay capability. +*/ __u64 gva;__u32 insn_len;__u8 insn_bytes[16 ];
tools/testing/selftests/kvm/memory_attributes.c +5 -1
@@ -198,7 +198,11 @@ static void test_page_restricted(struct kvm_vcpu *vcpu, int op,TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_MEMORY_FAULT );TEST_ASSERT_EQ (vcpu->run->memory_fault.gpa, fault_paddr);if (fault_reason) -TEST_ASSERT_EQ (vcpu->run->memory_fault.flags, fault_reason); +TEST_ASSERT_EQ (vcpu->run->memory_fault.flags & + ~(KVM_MEMORY_EXIT_FLAG_GVA_VALID | +KVM_MEMORY_EXIT_FLAG_PLANE_PROT | +KVM_MEMORY_EXIT_FLAG_INSN_VALID ), + fault_reason);TEST_ASSERT_EQ (vcpu->run->memory_fault.size, vm->page_size); }
Signature · 1 line
2.50.1
Re: [RFC PATCH 49/55] KVM: SVM: Carry instruction state on reflected plane-protection faults
Hi David,
The tools copy of the KVM UAPI header also needs the final memory-fault flags and comments introduced here. I synchronized it as below.
The last hunk is a related header synchronization from patch 09: the RAISE_UD completion bit requires a forwarding capability version of at least 2, and is only honored while forwarding is enabled.
tools/include/uapi/linux/kvm.h +19 -2
@@ -481,10 +481,25 @@ struct kvm_run {#define KVM_MEMORY_EXIT_FLAG_EXEC (1ULL << 2)#define KVM_MEMORY_EXIT_FLAG_PRIVATE (1ULL << 3)#define KVM_MEMORY_EXIT_FLAG_GVA_VALID (1ULL << 4) +/* The fault was caused by the vCPU's plane's memory protections. */ +#define KVM_MEMORY_EXIT_FLAG_PLANE_PROT (1ULL << 5) +/* +* insn_len/insn_bytes are valid. For an EXEC fault the faulting +* instruction itself cannot be read: insn_bytes stays zero, and +* insn_len is the length of the instruction which performed the +* fetch (reported by VMX hardware; not available on SVM). +*/ +#define KVM_MEMORY_EXIT_FLAG_INSN_VALID (1ULL << 6)__u64 flags;__u64 gpa;__u64 size; -/* set on plane-protection faults when available */ +/* +* gva/insn_len/insn_bytes are set on plane-protection +* faults when available, and are zero on every other +* memory-fault exit. They are enumerated, along with +* the PLANE_PROT and INSN_VALID flags, by the plane +* memory overlay capability. +*/ __u64 gva;__u32 insn_len;__u8 insn_bytes[16 ];
@@ -1043,7 +1058,9 @@ struct kvm_enable_cap {* Userspace completion of a forwarded Hyper-V hypercall (KVM_EXIT_HYPERV_HCALL * under KVM_CAP_HYPERV_FORWARD_HCALL): setting this bit in the result makes * KVM raise #UD at the faulting instruction instead of completing the -* hypercall; all other result bits are then ignored. +* hypercall; all other result bits are then ignored. Only honoured while +* forwarding is enabled, and only on kernels where +* KVM_CHECK_EXTENSION(KVM_CAP_HYPERV_FORWARD_HCALL) returns >= 2. */ #define KVM_HV_HCALL_RESULT_RAISE_UD (1ULL << 63)#define KVM_CAP_HYPERV_VTL_SWITCH 256
Thanks,
Sriram
Re: [RFC PATCH 49/55] KVM: SVM: Carry instruction state on reflected plane-protection faults
In reply to David WoodhouseHi David,
One build adjustment when integrating this onto my v7.3-based tree:static_cpu_has() is no longer available to this caller. I changed the decode-assist test to cpu_feature_enabled(), matching the surrounding SVM code:
arch/x86/kvm/svm/svm.c +1 -1
@@ -2094,7 +2094,7 @@ static int npf_interception(struct kvm_vcpu *vcpu) (vcpu->run->memory_fault.flags & KVM_MEMORY_EXIT_FLAG_PLANE_PROT)) { unsigned int len = 0; - if (static_cpu_has(X86_FEATURE_DECODEASSISTS) && + if (cpu_feature_enabled(X86_FEATURE_DECODEASSISTS) && svm->vmcb->control.insn_len && x86_decode_emulated_instruction(vcpu, 0, svm->vmcb->control.insn_bytes,
This is a newer-base integration adjustment, rather than a claim that the original v7.1-rc7-based posting fails to build.
Thanks,
Sriram
RFC PATCH 50/55 KVM: SVM: Trap descriptor-table loads for VTL register intercepts
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
A higher VTL arms register intercepts on IDTR/GDTR/LDTR/TR writes (KVM_CAP_HYPERV_REG_INTERCEPT), and the emulator diverts an intercepted load into a KVM_EXIT_HYPERV_REG_INTERCEPT exit instead of applying it (hv_divert_dt_write()). Only VMX armed the hardware exits (SECONDARY_EXEC_DESC), so on AMD a VTL0 lidt/lgdt/lldt/ltr executed unintercepted and the higher VTL never received its register intercept
messages: some twenty hyperv_vsm KUT register-intercept tests fail, and a VTL1 that relies on the intercepts for kernel patch protection silently loses them.
Arm the four VMCB descriptor-table load intercepts whenever reg_intercept_control is non-zero, mirroring the all-or-nothing VMX control, and route the exits into kvm_emulate_instruction(): a load the divert check declines is applied by the emulator as before. Store intercepts stay untrapped; the control has no bits for them.
The (dis)arming gets a dedicated kvm_x86_ops.update_hv_reg_intercepts() op touching only the L1 controls, instead of riding on hooks with lifecycles of their own: vmx_set_cr4() owned SECONDARY_EXEC_DESC for UMIP emulation and a VTL0 set-then-clear of CR4.UMIP disarmed the very lidt/lgdt trap hv_divert_dt_write() exists to catch; enabling the cap through vcpu_after_set_cpuid() after an exit from L2 rewrote vmcs02 state that the next nested VM-exit threw away. On VMX the op defers while vmcs02 is loaded and nested_vmx_vmexit() refreshes vmcs01 after switching back; on SVM it arms vmcb01, and nested_vmcb02_recalc_intercepts() strips the LOAD_* bits from vmcb02 unless vmcb12 wants them, matching the VMX DESC strip. The CR4 path preserves the control while the intercepts are armed, and hv_divert_dt_write() bails in guest mode so an L2 descriptor exit which still reaches the emulator (UMIP emulation, for example) applies the load normally.
Arming is refused with -EINVAL on SEV guests, failing KVM_ENABLE_CAP before anything is armed (disarming still succeeds): the diverted load needs full emulation, which encrypted guests cannot have. On SEV-ES desc_interception() hits svm_check_emulate_instruction() returning X86EMUL_RETRY_INSTR, resuming at the same RIP with the intercept still armed, so one descriptor-table load spins a vCPU (and a host CPU) forever; on plain SEV the emulator has no instruction bytes and injects #UD on an architecturally valid lgdt.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 7 ++++ arch/x86/kvm/hyperv.h | 16 ++++++++ arch/x86/kvm/svm/nested.c | 16 ++++++++ arch/x86/kvm/svm/svm.c | 66 ++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/main.c | 10 +++++ arch/x86/kvm/vmx/nested.c | 1 + arch/x86/kvm/vmx/vmx.c | 65 ++++++++++++++++++++++------- arch/x86/kvm/vmx/x86_ops.h | 1 + arch/x86/kvm/x86.c | 19 ++++++--- 10 files changed, 183 insertions(+), 19 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -140,6 +140,7 @@ KVM_X86_OP(get_feature_msr) KVM_X86_OP(check_emulate_instruction) KVM_X86_OP(apic_init_signal_blocked) KVM_X86_OP_OPTIONAL(enable_l2_tlb_flush) +KVM_X86_OP(update_hv_reg_intercepts) KVM_X86_OP_OPTIONAL(migrate_timers) KVM_X86_OP(recalc_intercepts) KVM_X86_OP(complete_emulated_msr)
arch/x86/include/asm/kvm_host.h +7 -0
@@ -2109,6 +2109,13 @@ struct kvm_x86_ops {bool (*apic_init_signal_blocked)(struct kvm_vcpu *vcpu);int (*enable_l2_tlb_flush)(struct kvm_vcpu *vcpu); +/* +* Bring the L1 (vmcs01/vmcb01) descriptor-table intercepts in line +* with hv_vcpu->reg_intercept_control. Never touches L2 controls; +* safe to call at any point, including with a vCPU in guest mode. +*/ +int (*update_hv_reg_intercepts)(struct kvm_vcpu *vcpu); +void (*migrate_timers)(struct kvm_vcpu *vcpu);void (*recalc_intercepts)(struct kvm_vcpu *vcpu);int (*complete_emulated_msr)(struct kvm_vcpu *vcpu,int err);
arch/x86/kvm/hyperv.h +16 -0
@@ -237,6 +237,18 @@ static inline bool kvm_hv_vcpu_suspended(struct kvm_vcpu *vcpu)READ_ONCE (vcpu->arch.hyperv->suspended); } +/* +* True when a higher VTL armed descriptor-table write intercepts +* (HvRegisterCrInterceptControl) on this vCPU, i.e. when the vendor code +* must trap the L1 guest's descriptor-table loads for hv_divert_dt_write(). +*/ +static inline bool kvm_hv_dt_intercepts_armed (struct kvm_vcpu *vcpu) +{ +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); + +return hv_vcpu && hv_vcpu->reg_intercept_control; +} +static inline bool kvm_hv_is_tlb_flush_hcall (struct kvm_vcpu *vcpu) {struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu);
@@ -328,6 +340,10 @@ static inline int kvm_hv_verify_vp_assist(struct kvm_vcpu *vcpu) {return 0 ; } +static inline bool kvm_hv_dt_intercepts_armed (struct kvm_vcpu *vcpu) +{ +return false; +}static inline u32 kvm_hv_get_vpindex (struct kvm_vcpu *vcpu) {return vcpu->vcpu_idx;
arch/x86/kvm/svm/nested.c +16 -0
@@ -156,6 +156,22 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)for (i =0 ; i <MAX_INTERCEPT ; i++) vmcb02->control.intercepts[i] |= vmcb12_ctrl->intercepts[i]; +/* +* KVM arms the descriptor-table load intercepts in vmcb01 to divert +* L1's loads into Hyper-V register-intercept exits (see +* hv_divert_dt_write()); they do not apply to L2. Drop them from +* vmcb02 unless L1 wants the intercept itself, matching the +* SECONDARY_EXEC_DESC strip in VMX's prepare_vmcs02(). +*/ +if (!vmcb12_is_intercept (vmcb12_ctrl,INTERCEPT_LOAD_IDTR )) +vmcb_clr_intercept (&vmcb02->control,INTERCEPT_LOAD_IDTR ); +if (!vmcb12_is_intercept (vmcb12_ctrl,INTERCEPT_LOAD_GDTR )) +vmcb_clr_intercept (&vmcb02->control,INTERCEPT_LOAD_GDTR ); +if (!vmcb12_is_intercept (vmcb12_ctrl,INTERCEPT_LOAD_LDTR )) +vmcb_clr_intercept (&vmcb02->control,INTERCEPT_LOAD_LDTR ); +if (!vmcb12_is_intercept (vmcb12_ctrl,INTERCEPT_LOAD_TR )) +vmcb_clr_intercept (&vmcb02->control,INTERCEPT_LOAD_TR ); +/* If SMI is not intercepted, ignore guest SMI intercept as well */ if (!intercept_smi)vmcb_clr_intercept (&vmcb02->control,INTERCEPT_SMI );
arch/x86/kvm/svm/svm.c +66 -0
96 lines not on this page · raw message
arch/x86/kvm/vmx/main.c +10 -0
@@ -207,6 +207,15 @@ static void vt_recalc_intercepts(struct kvm_vcpu *vcpu)vmx_recalc_intercepts (vcpu); } +static int vt_update_hv_reg_intercepts (struct kvm_vcpu *vcpu) +{ +/* TDX doesn't allow descriptor-table loads to be intercepted. */ +if (is_td_vcpu (vcpu)) +return -EINVAL ; + +return vmx_update_hv_reg_intercepts (vcpu); +} +static int vt_complete_emulated_msr (struct kvm_vcpu *vcpu,int err) {if (is_td_vcpu (vcpu))
@@ -1018,6 +1027,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .check_emulate_instruction = vt_op(check_emulate_instruction), .apic_init_signal_blocked = vt_op(apic_init_signal_blocked), + .update_hv_reg_intercepts = vt_op(update_hv_reg_intercepts), .migrate_timers = vmx_migrate_timers, .recalc_intercepts = vt_op(recalc_intercepts),
arch/x86/kvm/vmx/nested.c +1 -0
@@ -5142,6 +5142,7 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,* Update any VMCS fields that might have changed while vmcs02 was the * active VMCS. The tracking is per-vCPU, not per-VMCS. */ +vmx_update_hv_reg_intercepts (vcpu);vmcs_write32 (VM_EXIT_MSR_STORE_COUNT , vmx->msr_autostore.nr);vmcs_write32 (VM_EXIT_MSR_LOAD_COUNT , vmx->msr_autoload.host.nr);vmcs_write32 (VM_ENTRY_MSR_LOAD_COUNT , vmx->msr_autoload.guest.nr);
arch/x86/kvm/vmx/vmx.c +51 -14
90 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h +1 -0
@@ -53,6 +53,7 @@ void vmx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,void vmx_vcpu_after_set_cpuid (struct kvm_vcpu *vcpu);bool vmx_has_emulated_msr (struct kvm *kvm,u32 index);void vmx_recalc_intercepts (struct kvm_vcpu *vcpu); +int vmx_update_hv_reg_intercepts (struct kvm_vcpu *vcpu);void vmx_prepare_switch_to_guest (struct kvm_vcpu *vcpu);void vmx_update_exception_bitmap (struct kvm_vcpu *vcpu);int vmx_get_feature_msr (u32 msr,u64 *data);
arch/x86/kvm/x86.c +14 -5
@@ -6222,9 +6222,16 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,return r; hv_vcpu =to_hv_vcpu (vcpu); hv_vcpu->reg_intercept_control = cap->args[0 ]; -/* Refresh the descriptor-table exiting controls. */ -kvm_x86_call (vcpu_after_set_cpuid)(vcpu); -return 0 ; +/* +* Refresh the descriptor-table exiting controls. The vendor +* op touches only the L1 controls, so it is safe here even +* when the vCPU last exited from a nested guest and +* vcpu_load() restored vmcs02/vmcb02. +*/ + r =kvm_x86_call (update_hv_reg_intercepts)(vcpu); +if (r) + hv_vcpu->reg_intercept_control =0 ; +return r; }case KVM_CAP_HYPERV_DIRECT_TLBFLUSH :if (!kvm_x86_ops.enable_l2_tlb_flush)
@@ -8900,7 +8907,9 @@ static void emulator_get_idt(struct x86_emulate_ctxt *ctxt, struct desc_ptr *dt)/* * Divert a descriptor-table write into a register-intercept exit when the * higher VTL armed one: the write is not applied, and userspace delivers -* the TLFS register-intercept message to the higher VTL. +* the TLFS register-intercept message to the higher VTL. The intercept +* contract covers only the L1 guest; a descriptor load emulated on behalf +* of a nested (L2) guest, however its exit arrived, is applied normally. */ static bool hv_divert_dt_write (struct x86_emulate_ctxt *ctxt,u32 hv_reg,const struct desc_ptr *dt,u64 ctl_bit)
@@ -8909,7 +8918,7 @@ static bool hv_divert_dt_write(struct x86_emulate_ctxt *ctxt, u32 hv_reg,struct kvm_vcpu *vcpu =emul_to_vcpu (ctxt);struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); -if (!hv_vcpu || vcpu->plane_level || +if (!hv_vcpu || vcpu->plane_level ||is_guest_mode (vcpu) || !(hv_vcpu->reg_intercept_control & ctl_bit))return false;
Signature · 1 line
2.50.1
RFC PATCH 51/55 KVM: SVM: Keep TPR[3:0] across exits without a guest CR8 write
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
V_TPR holds only TPR[7:4], and sync_cr8_to_lapic() pushed it back into the APIC after every VMRUN. A guest that sets TASKPRI through the APIC page or MSR loses the low nibble at the next exit: VTL1 in the hyperv_vsm KUT writes 0xff and reads back 0xf0 ("VTL1 TASKPRI write sticks"), and interrupts in the sub-class the guest meant to mask are let through. VMX keeps all eight TPR bits in the virtual-APIC page and does not have this problem.
Hardware changes V_TPR only on a guest CR8 write. Record the value sync_lapic_to_cr8() loads on entry and push V_TPR back into the APIC only when it differs, i.e. only when the guest actually wrote CR8. A CR8 write whose value matches the loaded TPR[7:4] is indistinguishable from no write and now preserves TPR[3:0], where architecturally it would clear it; the previous behaviour cleared TPR[3:0] on every exit regardless.
--- arch/x86/kvm/svm/svm.c | 11 ++++++++++- arch/x86/kvm/svm/svm.h | 9 +++++++++ 2 files changed, 19 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/svm.c +10 -1
@@ -4320,7 +4320,15 @@ static inline void sync_cr8_to_lapic(struct kvm_vcpu *vcpu)if (!svm_is_intercept (svm,INTERCEPT_CR8_WRITE )) {int cr8 = svm->vmcb->control.int_ctl &V_TPR_MASK ; -kvm_set_cr8 (vcpu, cr8); + +/* +* Hardware changes V_TPR only on a guest CR8 write. Pushing +* an unchanged value into the APIC would zero TPR[3:0] (CR8 +* maps TPR[7:4] alone), wiping the low nibble of a TASKPRI +* value the guest set through the APIC page or MSR. +*/ +if (cr8 != svm->v_tpr_entry) +kvm_set_cr8 (vcpu, cr8); } }
@@ -4335,6 +4343,7 @@ static inline void sync_lapic_to_cr8(struct kvm_vcpu *vcpu) cr8 =kvm_get_cr8 (vcpu); svm->vmcb->control.int_ctl &= ~V_TPR_MASK ; svm->vmcb->control.int_ctl |= cr8 &V_TPR_MASK ; + svm->v_tpr_entry = cr8 &V_TPR_MASK ; } static void svm_complete_soft_interrupt(struct kvm_vcpu *vcpu, u8 vector,
arch/x86/kvm/svm/svm.h +9 -0
@@ -316,6 +316,15 @@ struct vcpu_svm {void *msrpm; +/* +* V_TPR value written by sync_lapic_to_cr8() on the last entry. +* sync_cr8_to_lapic() pushes V_TPR back into the APIC only when it +* differs, i.e. when the guest wrote CR8 in guest mode: V_TPR holds +* TPR[7:4] only, so an unconditional back-sync would zero TPR[3:0] +* after every exit, losing the low nibble of an APIC TASKPRI write. +*/ +u8 v_tpr_entry; +ulong nmi_iret_rip;struct svm_nested_state nested;
Signature · 1 line
2.50.1
RFC PATCH 52/55 KVM: selftests: Add Hyper-V reference TSC page memory attribute test
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The reference TSC page is written by the host, not by the guest, so the guest-side memory attribute tests never cover it. Add a test that a page at an unprotected gfn is maintained as usual, that pointing HV_X64_MSR_REFERENCE_TSC at a gfn carrying KVM_MEMORY_ATTRIBUTE_NW leaves the page contents untouched — on the MSR write itself and on a subsequent KVM_SET_CLOCK-forced update — and that re-pointing the MSR at a writable page recovers.
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/x86/hyperv_tsc_page_attrs.c | 129 ++++++++++++++++++ 2 files changed, 130 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/hyperv_tsc_page_attrs.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -89,6 +89,7 @@ TEST_GEN_PROGS_x86 += x86/hyperv_forward_hcall TEST_GEN_PROGS_x86 += x86/hyperv_ipi TEST_GEN_PROGS_x86 += x86/hyperv_svm_test TEST_GEN_PROGS_x86 += x86/hyperv_tlb_flush +TEST_GEN_PROGS_x86 += x86/hyperv_tsc_page_attrs TEST_GEN_PROGS_x86 += x86/kvm_clock_test TEST_GEN_PROGS_x86 += x86/kvm_pv_test TEST_GEN_PROGS_x86 += x86/kvm_buslock_test
tools/testing/selftests/kvm/x86/hyperv_tsc_page_attrs.c new file +129 -0
129 lines not on this page · raw message
Signature · 1 line
2.50.1
RFC PATCH 53/55 KVM: SVM: Hedge the v_tpr_entry comment
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The comment claimed V_TPR "differs, i.e. when the guest wrote CR8":
differing implies a write, but a write does not imply differing. A same-value CR8 write is invisible to the back-sync, so TPR[3:0] survives where the architectural write would clear it — a documented tradeoff of commit f34c595cb8a5 ("KVM: SVM: Keep TPR[3:0] across exits without a guest CR8 write"), which deserves stating at the field rather than only in that commit's message. No functional change.
--- arch/x86/kvm/svm/svm.h | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/svm.h +7 -3
@@ -319,9 +319,13 @@ struct vcpu_svm {/* * V_TPR value written by sync_lapic_to_cr8() on the last entry. * sync_cr8_to_lapic() pushes V_TPR back into the APIC only when it -* differs, i.e. when the guest wrote CR8 in guest mode: V_TPR holds -* TPR[7:4] only, so an unconditional back-sync would zero TPR[3:0] -* after every exit, losing the low nibble of an APIC TASKPRI write. +* differs: V_TPR holds TPR[7:4] only, so an unconditional back-sync +* would zero TPR[3:0] after every exit, losing the low nibble of an +* APIC TASKPRI write. A differing V_TPR implies a guest CR8 write, +* but not the converse: a CR8 write of the same value is +* indistinguishable from no write, so TPR[3:0] survives it where +* the architectural write (and the intercepted path through +* kvm_lapic_set_tpr()) would clear it. */ u8 v_tpr_entry;
Signature · 1 line
2.50.1
RFC PATCH 54/55 KVM: Check every plane's protections before caching a gfn->hva
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
The gfn_to_hva_cache init and the pfncache refresh refuse to cache a range carrying NR/NW attributes, but consulted only plane 0's array. Cached accesses made in vCPU context for a non-zero plane vCPU (steal time, PV EOI, pvclock) therefore enforced plane 0's protections only: a plane-N guest could point a PV MSR at a gfn its own plane marked NW and kvm_write_guest_offset_cached() wrote through it, while the uncached paths correctly use the vCPU's own plane.
A cached translation has no plane context at use time, so refuse the cache when ANY plane protects the range. The attribute generation that invalidates these caches is already partition-wide, so updates on any plane force a re-check. The plane array walk follows the existing lock-free kvm->planes[] reader pattern (acquire paired with the publishing release).
--- include/linux/kvm_host.h | 8 ++++++++ virt/kvm/kvm_main.c | 31 ++++++++++++++++++++++++++++++- virt/kvm/pfncache.c | 3 ++- 3 files changed, 40 insertions(+), 2 deletions(-)
include/linux/kvm_host.h +8 -0
@@ -2846,6 +2846,8 @@ bool kvm_plane_range_has_any_attrs(struct kvm_plane *plane, gfn_t start, gfn_t end);bool kvm_range_has_any_memory_attributes (struct kvm *kvm,gfn_t start,gfn_t end,unsigned long mask); +bool kvm_any_plane_range_has_rw_memory_protections (struct kvm *kvm, +gfn_t start,gfn_t end);bool kvm_arch_pre_set_memory_attributes (struct kvm *kvm,struct kvm_gfn_range *range); bool kvm_arch_post_set_memory_attributes(struct kvm *kvm,
@@ -2889,6 +2891,12 @@ static inline bool kvm_range_has_any_memory_attributes(struct kvm *kvm, {return false; } +static inline bool kvm_any_plane_range_has_rw_memory_protections (struct kvm *kvm, +gfn_t start, +gfn_t end) +{ +return false; +}static inline u64 kvm_mem_attributes_generation (struct kvm *kvm) {return 0 ;
virt/kvm/kvm_main.c +30 -1
@@ -2779,6 +2779,34 @@ static bool kvm_plane_range_has_memory_attributes(struct kvm_plane *plane,return false; } +/* +* Returns true if any plane's attributes deny read or write anywhere in +* [@start, @end). Cached gfn->hva translations carry no plane context +* at use time, so a cache must be refused when ANY plane protects the +* range, not just plane 0. +*/ +bool kvm_any_plane_range_has_rw_memory_protections (struct kvm *kvm, +gfn_t start,gfn_t end) +{ +unsigned long mask =KVM_MEMORY_ATTRIBUTE_NR |KVM_MEMORY_ATTRIBUTE_NW ; +int i; + +if (kvm_range_has_rw_memory_protections (kvm, start, end)) +return true; + +for (i =1 ; i <KVM_MAX_PLANES ; i++) { +/* Pairs with the release in kvm_create_plane(). */ +struct kvm_plane *plane =smp_load_acquire (&kvm->planes[i]); + +if (plane && +kvm_plane_range_has_memory_attributes (plane, start, end, + mask)) +return true; + } + +return false; +} +static bool kvm_range_memory_attributes_need_sync (struct kvm_plane *plane,gfn_t start,gfn_t end,unsigned long attributes)
@@ -3939,7 +3967,8 @@ static int __kvm_gfn_to_hva_cache_init(struct kvm *kvm,* check has to happen after storing the generation number. */ ghc->attrs_generation =kvm_mem_attributes_generation (kvm); -if (kvm_range_has_rw_memory_protections (kvm, start_gfn, end_gfn +1 )) { +if (kvm_any_plane_range_has_rw_memory_protections (kvm, start_gfn, + end_gfn +1 )) { ghc->hva =KVM_HVA_ERR_BAD ;return -EFAULT ; }
virt/kvm/pfncache.c +2 -1
@@ -304,7 +304,8 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned l* generation number. */ gpc->attrs_generation =kvm_mem_attributes_generation (kvm); -if (kvm_range_has_rw_memory_protections (kvm, gfn, gfn +1 )) { +if (kvm_any_plane_range_has_rw_memory_protections (kvm, gfn, + gfn +1 )) { gpc->uhva =KVM_HVA_ERR_BAD ; ret = -EFAULT ; goto out;
Signature · 1 line
2.50.1
RFC PATCH 55/55 KVM: planes: Annotate the lockless twin-publication protocol
In reply to David WoodhouseFrom: David Woodhouse <[email protected]>
Plane twins are published with a plain store to common->vcpus[] plus an smp_wmb() that pairs with online_vcpus, not with twin initialisation, while the lockless readers grown across the series (the twin-kick paths, the Hyper-V TLB-flush and stimer twin lookups, the SEND_IPI target lookup, vcpu_for_each_plane) read the slots with a mixture of READ_ONCE and bare loads. x86-TSO hides the lot, but the protocol is unannotated generic code: KCSAN reports it, and nothing stops a reordered publication from exposing a half-initialised twin to a kick.
Publish the slot with smp_store_release(), make every lockless reader use READ_ONCE (vcpu_for_each_plane included), and document the pairing at the iterator. Mutex-serialised lookups (the in-kernel VTL switch runs under the shared vCPU mutex that twin creation also takes) stay plain. Also document KVM_REQUEST_PER_PLANE's two scopes at its
definition: the fan-out to twins happens in kvm_make_vcpu_request(), while a direct kvm_make_request() marks only the twin it was given - a distinction a future direct-raise caller would otherwise trip over.
No functional change on x86.
--- arch/x86/kvm/hyperv.c | 6 +++--- include/linux/kvm_host.h | 13 ++++++++++++- virt/kvm/kvm_main.c | 8 +++++++- 3 files changed, 22 insertions(+), 5 deletions(-)
arch/x86/kvm/hyperv.c +3 -3
@@ -612,7 +612,7 @@ static void stimer_mark_pending(struct kvm_vcpu_hv_stimer *stimer,int p;for (p =0 ; p < vcpu->plane_level; p++) { - twin = vcpu->common->vcpus[p]; + twin =READ_ONCE (vcpu->common->vcpus[p]);if (twin) {kvm_make_request (KVM_REQ_UNBLOCK , twin);kvm_vcpu_kick (twin);
@@ -2370,7 +2370,7 @@ static void hv_tlb_flush_enqueue_all_planes(struct kvm_vcpu *v,int p;for (p =0 ; p <KVM_MAX_PLANES ; p++) { - twin = v->common->vcpus[p]; + twin =READ_ONCE (v->common->vcpus[p]);if (!twin || !to_hv_vcpu (twin))continue ; fifo =kvm_hv_get_tlb_flush_fifo (twin, false);
@@ -2601,7 +2601,7 @@ static void kvm_hv_send_ipi_to_many(struct kvm *kvm, u32 vector,* VTL): deliver into that plane's APIC. */ if (target_plane) { - target = vcpu->common->vcpus[target_plane]; + target =READ_ONCE (vcpu->common->vcpus[target_plane]);if (!target)continue ; }
include/linux/kvm_host.h +12 -1
@@ -159,6 +159,12 @@ static inline bool kvm_is_error_gpa(gpa_t gpa)#define KVM_REQUEST_NO_WAKEUP BIT(8)#define KVM_REQUEST_WAIT BIT(9)#define KVM_REQUEST_NO_ACTION BIT(10) +/* +* A KVM_REQUEST_PER_PLANE request has two scopes: raised through +* kvm_make_vcpu_request() (kicks, mass requests) it fans out to every +* plane twin of the vCPU, while a direct kvm_make_request() marks only +* the twin it was given. A direct caller must pick the twin it means. +*/ #define KVM_REQUEST_PER_PLANE BIT(11)/* * Architecture-independent vcpu->requests bit members
@@ -400,9 +406,14 @@ struct kvm_vcpu_common {struct kvm_vcpu_arch_common arch; }; +/* +* The slot read pairs with the smp_store_release() publication in +* kvm_vcpu_finish_common() for iterators running outside the locks that +* serialise twin creation. +*/ #define vcpu_for_each_plane (common, i, v) \ for ((i) = 0; (i) < KVM_MAX_PLANES; ++(i)) \ - if (((v) = common->vcpus[(i)]) != NULL) + if (((v) = READ_ONCE(common->vcpus[(i)])) != NULL)/* Tracked per plane-VCPU - used for deciding which plane-vcpu to run */ enum kvm_vcpu_state {
virt/kvm/kvm_main.c +7 -1
@@ -563,7 +563,13 @@ static int kvm_vcpu_init_common(struct kvm_vcpu *vcpu, struct kvm *kvm, unsignedstatic void kvm_vcpu_finish_common (struct kvm_vcpu *vcpu) {WARN_ON (vcpu->common->vcpus[vcpu->plane_level] !=NULL ); - vcpu->common->vcpus[vcpu->plane_level] = vcpu; +/* +* Publish with release semantics so the lockless twin readers +* (the kick paths, vcpu_for_each_plane, the hyperv twin lookups) +* observe a fully initialised vCPU through their READ_ONCE of the +* slot; the plain store relied on x86-TSO. +*/ +smp_store_release (&vcpu->common->vcpus[vcpu->plane_level], vcpu);smp_wmb ();if (vcpu->plane_level ==0 ) {/*
Signature · 1 line
2.50.1
Hi David,
Thanks for posting this. I've been trying the series and the QEMU VSM implementation in my local integration trees. With the fixes below, Windows now reports "Virtualization-based security: Running" in msinfo32.
This was on a v7.3-based kernel tree with existing local VM-planes/LVBS changes, rather than an otherwise unmodified copy of your v7.1-rc7-based tree. I don't want to imply that all of the integration changes were defects in the posted series.
The main runtime issue I found was descriptor-table interception while OVMF was executing in SMM. hv_divert_dt_write() was diverting firmware descriptor-table writes through the VTL register-intercept path. The failure trace showed a reflected #GP delivered in SMM with IDT base and limit both zero, followed by a triple fault/reset. Adding an is_smm() exclusion to the helper fixed that boot failure. I'll reply to patch 23 with the change.
On the QEMU side, I also changed vsm_glue_reset() to make plane 0 runnable when stopping plane 1. Otherwise a reset following a VTL switch can leave the normal-world vCPU userspace-stopped. I'll send that change separately in this thread, since the QEMU glue isn't part of the kernel posting.
Two build/header items from the newer kernel integration:
- In patch 49, I used cpu_feature_enabled(X86_FEATURE_DECODEASSISTS) instead of static_cpu_has(), matching the current SVM code.
- I synchronized the tools UAPI header with the PLANE_PROT/INSN_VALID flags and their field-validity comments, and the capability-version requirement for KVM_HV_HCALL_RESULT_RAISE_UD.
I also encountered an issue in the underlying planes dependency:
"kvm: Make KVM_SIGNAL_MSI per plane" guards the VM-fd and plane-fd ioctl dispatch with CONFIG_HAVE_KVM_MSI. Those guards need to use __KVM_HAVE_MSI; otherwise KVM_SIGNAL_MSI is compiled out and returns ENOTTY. The existing test_plane_msi case exposed this in my tree. This isn't introduced by one of the 55 patches here.
Finally, my older LVBS tree had raised the default plane count to two. I reverted that local change to respect the KVM_CAP_PLANES opt-in contract and added early opt-in for my LVBS userspace path. Your QEMU VSM path already has its own opt-in, so I am not reporting that as a missing VSM change.
My runtime result here is the msinfo32 VBS-running status; I am not claiming separate Credential Guard, migration, or hibernate/resume validation.
Thanks,
Sriram
Quoted 5 lines · Sriram Nambakam
Hi David,
Thanks for posting this. I've been trying the series and the QEMU VSM implementation in my local integration trees. With the fixes below, Windows now reports "Virtualization-based security: Running" in msinfo32.
Great, thanks for testing!
Quoted 5 lines
This was on a v7.3-based kernel tree with existing local VM-planes/LVBS changes, rather than an otherwise unmodified copy of your v7.1-rc7-based tree. I don't want to imply that all of the integration changes were defects in the posted series.
Quoted 7 lines
The main runtime issue I found was descriptor-table interception while OVMF was executing in SMM.
hv_divert_dt_write()was diverting firmware descriptor-table writes through the VTL register-intercept path. The failure trace showed a reflected #GP delivered in SMM with IDT base and limit both zero, followed by a triple fault/reset. Adding anis_smm()exclusion to the helper fixed that boot failure. I'll reply to patch 23 with the change.
Thanks. I'll add OVMF with Secure Boot (hence SMM) to the test matrix, and pull in your fixes. Could I trouble you for a Signed-off-by: for each of those please?
Quoted 5 lines
On the QEMU side, I also changed
vsm_glue_reset()to make plane 0 runnable when stopping plane 1. Otherwise a reset following a VTL switch can leave the normal-world vCPU userspace-stopped. I'll send that change separately in this thread, since the QEMU glue isn't part of the kernel posting.
Quoted 7 lines
Two build/header items from the newer kernel integration:
- In patch 49, I used
cpu_feature_enabled(X86_FEATURE_DECODEASSISTS)instead ofstatic_cpu_has(), matching the current SVM code.
- I synchronized the tools UAPI header with the PLANE_PROT/INSN_VALID flags and their field-validity comments, and the capability-version requirement forKVM_HV_HCALL_RESULT_RAISE_UD.
I'm playing with the HV_HCALL stuff to make the capability into a bitmask, like the Xen one, which says which calls are handled in- kernel. That means the basic support can be submitted first, before anything else.
Quoted 6 lines
I also encountered an issue in the underlying planes dependency:
"kvm: MakeKVM_SIGNAL_MSIper plane" guards the VM-fd and plane-fd ioctl dispatch withCONFIG_HAVE_KVM_MSI. Those guards need to use__KVM_HAVE_MSI; otherwiseKVM_SIGNAL_MSIis compiled out and returns ENOTTY. The existingtest_plane_msicase exposed this in my tree. This isn't introduced by one of the 55 patches here.
Yeah, I have a few more fixes for the underlying Planes series, which I'll try to split out for Jörg too.
Quoted 5 lines
Finally, my older LVBS tree had raised the default plane count to two. I reverted that local change to respect the
KVM_CAP_PLANESopt-in contract and added early opt-in for my LVBS userspace path. Your QEMU VSM path already has its own opt-in, so I am not reporting that as a missing VSM change.
There are going to be a lot of intersecting details like that as the various trees come together and the dust settles. I'm trying to get things in some semblance of a mergeable order and form a plan for what could be posted immediately (the basic hv hypercall exit to userspace), what could follow on as soon as Planes/memattrs land, and then what needs more thought.
I'd like to get the *minimum* kernel functionality landed, and then follow on with "optimisations" like the in-kernel VTL switching and extra features like VINA etc.
Hi David,
One fix in the associated QEMU VSM glue: vsm_glue_reset() stops plane 1 but also needs to make plane 0 runnable.
After a VTL switch, the normal-world vCPU can still be marked userspace-stopped. Stopping only the higher plane during reset leaves that state behind, rather than restoring the VTL0 execution path for the fresh boot.
The functional change from my local QEMU commit is:
target/i386/kvm/hv-vsm-qemu.c +1 -0
@@ -974,6 +974,7 @@ static void vsm_glue_reset(void *opaque)for (uint32_t vp =0 ; vp < vsm_glue.nr_vps; vp++) {if (vsm_glue.plane1_vcpu_fd[vp] >=0 ) {vsm_vcpu_set_run_state (&vsm_glue,1 , vp, false); +vsm_vcpu_set_run_state (&vsm_glue,0 , vp, true); } }qemu_mutex_unlock (&vsm_glue.lock);
I also added temporary exception-delivery diagnostics to print the SMM state, CS selector/base, CR0, and IDT base/limit. Those exposed the SMM/#GP failure described in my reply to kernel patch 23, but aren't needed for the reset fix above.
Thanks,
Sriram