Skip to contentSkip to conversation

Enable FRED with KVM VMX

PATCH In review Xin Li (Intel) 13 participants 131 messages Oct 26, 2025 – Sep 10

Subsystems (7)
Documentation x86 KVM Kselftest +3 more subsystems
lore
Share & tools
lore

Xin Li (Intel)

author maintainer
#0 lore raw

v9 cover Enable FRED with KVM VMX

This patch set enables the Intel flexible return and event delivery (FRED) architecture with KVM VMX to allow guests to utilize FRED.

The FRED architecture defines simple new transitions that change privilege level (ring transitions). The FRED architecture was designed with the following goals:

1) Improve overall performance and response time by replacing event delivery through the interrupt descriptor table (IDT event delivery) and event return by the IRET instruction with lower latency transitions.

2) Improve software robustness by ensuring that event delivery establishes the full supervisor context and that event return establishes the full user context.

The new transitions defined by the FRED architecture are FRED event delivery and, for returning from events, two FRED return instructions. FRED event delivery can effect a transition from ring 3 to ring 0, but it is used also to deliver events incident to ring 0. One FRED instruction (ERETU) effects a return from ring 0 to ring 3, while the other (ERETS) returns while remaining in ring 0. Collectively, FRED event delivery and the FRED return instructions are FRED transitions.

Intel VMX architecture is extended to run FRED guests, and the major changes are:

1) New VMCS fields for FRED context management, which includes two new event data VMCS fields, eight new guest FRED context VMCS fields and eight new host FRED context VMCS fields.

2) VMX nested-exception support for proper virtualization of stack levels introduced with FRED architecture.

Search for the latest FRED spec in most search engines with this search pattern:

site:intel.com FRED (flexible return and event delivery) specification

Although FRED and CET supervisor shadow stacks are independent CPU features, FRED unconditionally includes FRED shadow stack pointer MSRs IA32_FRED_SSP[0123], and IA32_FRED_SSP0 is just an alias of the CET MSR IA32_PL0_SSP. IOW, the state management of MSR IA32_PL0_SSP becomes an overlap area, and Sean requested that FRED virtualization to land after CET virtualization [1].

With CET virtualization now merged in v6.18, the path is clear to submit the FRED virtualization patch series :).

Changes in v9:
* Rebased to the latest kvm-x86/next branch, tag kvm-x86-next-2025.10.20-2.
* Guard FRED state save/restore with guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) in patch 19 (syzbot & Chao).
* Use array indexing for exception stack access, eliminating the need for the ESTACKS_MEMBERS() macro in struct cea_exception_stacks, and then exported __this_cpu_ist_top_va() in a subsequent patch (Dave Hansen).
* Rewrote some of the change logs.

Following is the link to v8 of this patch set:
https://lore.kernel.org/lkml/[email protected]/

[1]: https://lore.kernel.org/kvm/[email protected]/

Xin Li (18):
KVM: VMX: Enable support for secondary VM exit controls
KVM: VMX: Initialize VM entry/exit FRED controls in vmcs_config
KVM: VMX: Disable FRED if FRED consistency checks fail
KVM: VMX: Initialize VMCS FRED fields
KVM: VMX: Set FRED MSR intercepts
KVM: VMX: Save/restore guest FRED RSP0
KVM: VMX: Add support for saving and restoring FRED MSRs
KVM: x86: Add a helper to detect if FRED is enabled for a vCPU
KVM: VMX: Virtualize FRED event_data
KVM: VMX: Virtualize FRED nested exception tracking
KVM: x86: Mark CR4.FRED as not reserved
KVM: VMX: Dump FRED context in dump_vmcs()
KVM: x86: Advertise support for FRED
KVM: nVMX: Enable support for secondary VM exit controls
KVM: nVMX: Handle FRED VMCS fields in nested VMX context
KVM: nVMX: Validate FRED-related VMCS fields
KVM: nVMX: Guard SHADOW_FIELD_R[OW] macros with VMX feature checks
KVM: nVMX: Enable VMX FRED controls

Xin Li (Intel) (4):
x86/cea: Prefix event stack names with ESTACK_ x86/cea: Use array indexing to simplify exception stack access x86/cea: Export __this_cpu_ist_top_va() to KVM
KVM: x86: Save/restore the nested flag of an exception

 Documentation/virt/kvm/api.rst        |  21 +-
 arch/x86/coco/sev/noinstr.c           |   4 +-
 arch/x86/coco/sev/vc-handle.c         |   2 +-
 arch/x86/include/asm/cpu_entry_area.h |  70 +++---
 arch/x86/include/asm/kvm_host.h       |  13 +-
 arch/x86/include/asm/msr-index.h      |   1 +
 arch/x86/include/asm/vmx.h            |  48 +++-
 arch/x86/include/uapi/asm/kvm.h       |   4 +-
 arch/x86/kernel/cpu/common.c          |  10 +-
 arch/x86/kernel/dumpstack_64.c        |  18 +-
 arch/x86/kernel/fred.c                |   6 +-
 arch/x86/kernel/traps.c               |   2 +-
 arch/x86/kvm/cpuid.c                  |   1 +
 arch/x86/kvm/kvm_cache_regs.h         |  15 ++
 arch/x86/kvm/svm/svm.c                |   2 +-
 arch/x86/kvm/vmx/capabilities.h       |  25 +-
 arch/x86/kvm/vmx/nested.c             | 343 +++++++++++++++++++++++---
 arch/x86/kvm/vmx/nested.h             |  22 ++
 arch/x86/kvm/vmx/vmcs.h               |   1 +
 arch/x86/kvm/vmx/vmcs12.c             |  19 ++
 arch/x86/kvm/vmx/vmcs12.h             |  40 ++-
 arch/x86/kvm/vmx/vmcs_shadow_fields.h |  37 ++-
 arch/x86/kvm/vmx/vmx.c                | 247 +++++++++++++++++--
 arch/x86/kvm/vmx/vmx.h                |  54 +++-
 arch/x86/kvm/x86.c                    | 131 +++++++++-
 arch/x86/kvm/x86.h                    |   8 +-
 arch/x86/mm/cpu_entry_area.c          |  39 ++-
 arch/x86/mm/fault.c                   |   2 +-
 include/uapi/linux/kvm.h              |   1 +
 29 files changed, 1038 insertions(+), 148 deletions(-)

base-commit: 4cc167c50eb19d44ac7e204938724e685e3d8057

Signature · 1 line
2.51.0
author maintainer
#1 lore raw

PATCH v9 1/22 KVM: VMX: Enable support for secondary VM exit controls

From: Xin Li <[email protected]>

Introduce infrastructure to support secondary VM exit controls.

Always load the controls when supported by hardware, though all control bits remain clear in this patch.

---

Change in v5:
* Add TB from Xuelian Guo.

Changes in v4:
* Fix clearing VM_EXIT_ACTIVATE_SECONDARY_CONTROLS (Chao Gao).
* Check VM exit/entry consistency based on the new macro from Sean Christopherson.

Change in v3:
* Do FRED controls consistency checks in the VM exit/entry consistency check framework (Sean Christopherson).

Change in v2:
* Always load the secondary VM exit controls (Sean Christopherson).

---
 arch/x86/include/asm/msr-index.h |  1 +
 arch/x86/include/asm/vmx.h       |  3 +++
 arch/x86/kvm/vmx/capabilities.h  |  9 ++++++++-
 arch/x86/kvm/vmx/vmcs.h          |  1 +
 arch/x86/kvm/vmx/vmx.c           | 29 +++++++++++++++++++++++++++--
 arch/x86/kvm/vmx/vmx.h           |  7 ++++++-
 6 files changed, 46 insertions(+), 4 deletions(-)
arch/x86/include/asm/msr-index.h +1 -0
index 9e1720d73244..baf5e1648418 100644
@@ -1225,6 +1225,7 @@
 #define MSR_IA32_VMX_TRUE_ENTRY_CTLS     0x00000490
 #define MSR_IA32_VMX_VMFUNC             0x00000491
 #define MSR_IA32_VMX_PROCBASED_CTLS3	0x00000492
+#define MSR_IA32_VMX_EXIT_CTLS2		0x00000493
 
 /* Resctrl MSRs: */
 /* - Intel: */
arch/x86/include/asm/vmx.h +3 -0
index c85c50019523..1f60c04d11fb 100644
@@ -107,6 +107,7 @@
 #define VM_EXIT_PT_CONCEAL_PIP			0x01000000
 #define VM_EXIT_CLEAR_IA32_RTIT_CTL		0x02000000
 #define VM_EXIT_LOAD_CET_STATE                  0x10000000
+#define VM_EXIT_ACTIVATE_SECONDARY_CONTROLS	0x80000000
 
 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR	0x00036dff
 
@@ -262,6 +263,8 @@ enum vmcs_field {
 	SHARED_EPT_POINTER		= 0x0000203C,
 	PID_POINTER_TABLE		= 0x00002042,
 	PID_POINTER_TABLE_HIGH		= 0x00002043,
+	SECONDARY_VM_EXIT_CONTROLS	= 0x00002044,
+	SECONDARY_VM_EXIT_CONTROLS_HIGH	= 0x00002045,
 	GUEST_PHYSICAL_ADDRESS          = 0x00002400,
 	GUEST_PHYSICAL_ADDRESS_HIGH     = 0x00002401,
 	VMCS_LINK_POINTER               = 0x00002800,
arch/x86/kvm/vmx/capabilities.h +8 -1
index 02aadb9d730e..6bd67c40ca3b 100644
@@ -55,8 +55,9 @@ struct vmcs_config {
 	u32 cpu_based_exec_ctrl;
 	u32 cpu_based_2nd_exec_ctrl;
 	u64 cpu_based_3rd_exec_ctrl;
-	u32 vmexit_ctrl;
 	u32 vmentry_ctrl;
+	u32 vmexit_ctrl;
+	u64 vmexit_2nd_ctrl;
 	u64 misc;
 	struct nested_vmx_msrs nested;
 };
@@ -141,6 +142,12 @@ static inline bool cpu_has_tertiary_exec_ctrls(void)
 		CPU_BASED_ACTIVATE_TERTIARY_CONTROLS;
 }
 
+static inline bool cpu_has_secondary_vmexit_ctrls(void)
+{
+	return vmcs_config.vmexit_ctrl &
+		VM_EXIT_ACTIVATE_SECONDARY_CONTROLS;
+}
+
 static inline bool cpu_has_vmx_virtualize_apic_accesses(void)
 {
 	return vmcs_config.cpu_based_2nd_exec_ctrl &
arch/x86/kvm/vmx/vmcs.h +1 -0
index b25625314658..ae152a9d1963 100644
@@ -47,6 +47,7 @@ struct vmcs_host_state {
 struct vmcs_controls_shadow {
 	u32 vm_entry;
 	u32 vm_exit;
+	u64 secondary_vm_exit;
 	u32 pin;
 	u32 exec;
 	u32 secondary_exec;
arch/x86/kvm/vmx/vmx.c +27 -2
index 1021d3b65ea0..8de841c9c905 100644
@@ -2595,8 +2595,9 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 	u32 _cpu_based_exec_control = 0;
 	u32 _cpu_based_2nd_exec_control = 0;
 	u64 _cpu_based_3rd_exec_control = 0;
-	u32 _vmexit_control = 0;
 	u32 _vmentry_control = 0;
+	u32 _vmexit_control = 0;
+	u64 _vmexit2_control = 0;
 	u64 basic_msr;
 	u64 misc_msr;
 
@@ -2617,6 +2618,12 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 		{ VM_ENTRY_LOAD_CET_STATE,		VM_EXIT_LOAD_CET_STATE },
 	};
 
+	struct {
+		u32 entry_control;
+		u64 exit_control;
+	} const vmcs_entry_exit2_pairs[] = {
+	};
+
 	memset(vmcs_conf, 0, sizeof(*vmcs_conf));
 
 	if (adjust_vmx_controls(KVM_REQUIRED_VMX_CPU_BASED_VM_EXEC_CONTROL,
@@ -2703,10 +2710,19 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 				&_vmentry_control))
 		return -EIO;
 
+	if (_vmexit_control & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS)
+		_vmexit2_control =
+			adjust_vmx_controls64(KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS,
+					      MSR_IA32_VMX_EXIT_CTLS2);
+
 	if (vmx_check_entry_exit_pairs(vmcs_entry_exit_pairs,
 				       _vmentry_control, _vmexit_control))
 		return -EIO;
 
+	if (vmx_check_entry_exit_pairs(vmcs_entry_exit2_pairs,
+				       _vmentry_control, _vmexit2_control))
+		return -EIO;
+
 	/*
 	 * Some cpus support VM_{ENTRY,EXIT}_IA32_PERF_GLOBAL_CTRL but they
 	 * can't be used due to an errata where VM Exit may incorrectly clear
@@ -2755,8 +2771,9 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 	vmcs_conf->cpu_based_exec_ctrl = _cpu_based_exec_control;
 	vmcs_conf->cpu_based_2nd_exec_ctrl = _cpu_based_2nd_exec_control;
 	vmcs_conf->cpu_based_3rd_exec_ctrl = _cpu_based_3rd_exec_control;
-	vmcs_conf->vmexit_ctrl         = _vmexit_control;
 	vmcs_conf->vmentry_ctrl        = _vmentry_control;
+	vmcs_conf->vmexit_ctrl         = _vmexit_control;
+	vmcs_conf->vmexit_2nd_ctrl     = _vmexit2_control;
 	vmcs_conf->misc	= misc_msr;
 
 #if IS_ENABLED(CONFIG_HYPERV)
@@ -4429,6 +4446,11 @@ static u32 vmx_get_initial_vmexit_ctrl(void)
 		~(VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL | VM_EXIT_LOAD_IA32_EFER);
 }
 
+static u64 vmx_secondary_vmexit_ctrl(void)
+{
+	return vmcs_config.vmexit_2nd_ctrl;
+}
+
 void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
@@ -4771,6 +4793,9 @@ static void init_vmcs(struct vcpu_vmx *vmx)
 
 	vm_exit_controls_set(vmx, vmx_get_initial_vmexit_ctrl());
 
+	if (cpu_has_secondary_vmexit_ctrls())
+		secondary_vm_exit_controls_set(vmx, vmx_secondary_vmexit_ctrl());
+
 	/* 22.2.1, 20.8.1 */
 	vm_entry_controls_set(vmx, vmx_get_initial_vmentry_ctrl());
 
arch/x86/kvm/vmx/vmx.h +6 -1
index 6cb04a6afeef..349d96e68f96 100644
@@ -510,7 +510,11 @@ static inline u8 vmx_get_rvi(void)
 	       VM_EXIT_CLEAR_BNDCFGS |					\
 	       VM_EXIT_PT_CONCEAL_PIP |					\
 	       VM_EXIT_CLEAR_IA32_RTIT_CTL |				\
-	       VM_EXIT_LOAD_CET_STATE)
+	       VM_EXIT_LOAD_CET_STATE |					\
+	       VM_EXIT_ACTIVATE_SECONDARY_CONTROLS)
+
+#define KVM_REQUIRED_VMX_SECONDARY_VM_EXIT_CONTROLS (0)
+#define KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS (0)
 
 #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL			\
 	(PIN_BASED_EXT_INTR_MASK |					\
@@ -623,6 +627,7 @@ static __always_inline void lname##_controls_changebit(struct vcpu_vmx *vmx, u##
 }
 BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS, 32)
 BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS, 32)
+BUILD_CONTROLS_SHADOW(secondary_vm_exit, SECONDARY_VM_EXIT_CONTROLS, 64)
 BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL, 32)
 BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL, 32)
 BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL, 32)
Signature · 1 line
2.51.0
author maintainer
#2 lore raw

PATCH v9 2/22 KVM: VMX: Initialize VM entry/exit FRED controls in vmcs_config

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Setup VM entry/exit FRED controls in the global vmcs_config for proper FRED VMCS fields management:
1) load guest FRED state upon VM entry.
2) save guest FRED state during VM exit.
3) load host FRED state during VM exit.

Also add FRED control consistency checks to the existing VM entry/exit consistency check framework.

---

Change in v5:
* Remove the pair VM_ENTRY_LOAD_IA32_FRED/VM_EXIT_ACTIVATE_SECONDARY_CONTROLS, since the secondary VM exit controls are unconditionally enabled anyway, and there are features other than FRED needing it (Chao Gao).
* Add TB from Xuelian Guo.

Change in v4:
* Do VM exit/entry consistency checks using the new macro from Sean Christopherson.

Changes in v3:
* Add FRED control consistency checks to the existing VM entry/exit consistency check framework (Sean Christopherson).
* Just do the unnecessary FRED state load/store on every VM entry/exit (Sean Christopherson).

---
 arch/x86/include/asm/vmx.h | 4 ++++
 arch/x86/kvm/vmx/vmx.c     | 2 ++
 arch/x86/kvm/vmx/vmx.h     | 7 +++++--
 3 files changed, 11 insertions(+), 2 deletions(-)
arch/x86/include/asm/vmx.h +4 -0
index 1f60c04d11fb..dd79d027ea70 100644
@@ -109,6 +109,9 @@
 #define VM_EXIT_LOAD_CET_STATE                  0x10000000
 #define VM_EXIT_ACTIVATE_SECONDARY_CONTROLS	0x80000000
 
+#define SECONDARY_VM_EXIT_SAVE_IA32_FRED	BIT_ULL(0)
+#define SECONDARY_VM_EXIT_LOAD_IA32_FRED	BIT_ULL(1)
+
 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR	0x00036dff
 
 #define VM_ENTRY_LOAD_DEBUG_CONTROLS            0x00000004
@@ -122,6 +125,7 @@
 #define VM_ENTRY_PT_CONCEAL_PIP			0x00020000
 #define VM_ENTRY_LOAD_IA32_RTIT_CTL		0x00040000
 #define VM_ENTRY_LOAD_CET_STATE                 0x00100000
+#define VM_ENTRY_LOAD_IA32_FRED			0x00800000
 
 #define VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR	0x000011ff
 
arch/x86/kvm/vmx/vmx.c +2 -0
index 8de841c9c905..be48ba2d70e1 100644
@@ -2622,6 +2622,8 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 		u32 entry_control;
 		u64 exit_control;
 	} const vmcs_entry_exit2_pairs[] = {
+		{ VM_ENTRY_LOAD_IA32_FRED,
+			SECONDARY_VM_EXIT_SAVE_IA32_FRED | SECONDARY_VM_EXIT_LOAD_IA32_FRED },
 	};
 
 	memset(vmcs_conf, 0, sizeof(*vmcs_conf));
arch/x86/kvm/vmx/vmx.h +5 -2
index 349d96e68f96..645b0343e88c 100644
@@ -487,7 +487,8 @@ static inline u8 vmx_get_rvi(void)
 	 VM_ENTRY_LOAD_BNDCFGS |					\
 	 VM_ENTRY_PT_CONCEAL_PIP |					\
 	 VM_ENTRY_LOAD_IA32_RTIT_CTL |					\
-	 VM_ENTRY_LOAD_CET_STATE)
+	 VM_ENTRY_LOAD_CET_STATE |					\
+	 VM_ENTRY_LOAD_IA32_FRED)
 
 #define __KVM_REQUIRED_VMX_VM_EXIT_CONTROLS				\
 	(VM_EXIT_SAVE_DEBUG_CONTROLS |					\
@@ -514,7 +515,9 @@ static inline u8 vmx_get_rvi(void)
 	       VM_EXIT_ACTIVATE_SECONDARY_CONTROLS)
 
 #define KVM_REQUIRED_VMX_SECONDARY_VM_EXIT_CONTROLS (0)
-#define KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS (0)
+#define KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS			\
+	     (SECONDARY_VM_EXIT_SAVE_IA32_FRED |			\
+	      SECONDARY_VM_EXIT_LOAD_IA32_FRED)
 
 #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL			\
 	(PIN_BASED_EXT_INTR_MASK |					\
Signature · 1 line
2.51.0
Reviewed-by on v9 2/22
#3 lore raw

Re: [PATCH v9 02/22] KVM: VMX: Initialize VM entry/exit FRED controls in vmcs_config

Quoted 8 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Setup VM entry/exit FRED controls in the global vmcs_config for proper FRED VMCS fields management:
1) load guest FRED state upon VM entry.
2) save guest FRED state during VM exit.
3) load host FRED state during VM exit.

Nit:
I think it's worth noting that IA32_FRED_RSP0 and IA32_FRED_SSP0 are treated differently. The change log might need more context on which MSRs are atomically switched in order to describe that though.

Quoted 2 lines

Also add FRED control consistency checks to the existing VM entry/exit consistency check framework.

Quoted 89 lines

---

Change in v5:
* Remove the pair VM_ENTRY_LOAD_IA32_FRED/VM_EXIT_ACTIVATE_SECONDARY_CONTROLS, since the secondary VM exit controls are unconditionally enabled anyway, and there are features other than FRED needing it (Chao Gao).
* Add TB from Xuelian Guo.

Change in v4:
* Do VM exit/entry consistency checks using the new macro from Sean Christopherson.

Changes in v3:
* Add FRED control consistency checks to the existing VM entry/exit consistency check framework (Sean Christopherson).
* Just do the unnecessary FRED state load/store on every VM entry/exit (Sean Christopherson).

---
 arch/x86/include/asm/vmx.h | 4 ++++
 arch/x86/kvm/vmx/vmx.c     | 2 ++
 arch/x86/kvm/vmx/vmx.h     | 7 +++++--
 3 files changed, 11 insertions(+), 2 deletions(-)
arch/x86/include/asm/vmx.h +4 -0
index 1f60c04d11fb..dd79d027ea70 100644
@@ -109,6 +109,9 @@
 #define VM_EXIT_LOAD_CET_STATE                  0x10000000
 #define VM_EXIT_ACTIVATE_SECONDARY_CONTROLS	0x80000000
 
+#define SECONDARY_VM_EXIT_SAVE_IA32_FRED	BIT_ULL(0)
+#define SECONDARY_VM_EXIT_LOAD_IA32_FRED	BIT_ULL(1)
+
 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR	0x00036dff
 
 #define VM_ENTRY_LOAD_DEBUG_CONTROLS            0x00000004
@@ -122,6 +125,7 @@
 #define VM_ENTRY_PT_CONCEAL_PIP			0x00020000
 #define VM_ENTRY_LOAD_IA32_RTIT_CTL		0x00040000
 #define VM_ENTRY_LOAD_CET_STATE                 0x00100000
+#define VM_ENTRY_LOAD_IA32_FRED			0x00800000
 
 #define VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR	0x000011ff
 
arch/x86/kvm/vmx/vmx.c +2 -0
index 8de841c9c905..be48ba2d70e1 100644
@@ -2622,6 +2622,8 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf,
 		u32 entry_control;
 		u64 exit_control;
 	} const vmcs_entry_exit2_pairs[] = {
+		{ VM_ENTRY_LOAD_IA32_FRED,
+			SECONDARY_VM_EXIT_SAVE_IA32_FRED | SECONDARY_VM_EXIT_LOAD_IA32_FRED },
 	};
 
 	memset(vmcs_conf, 0, sizeof(*vmcs_conf));
arch/x86/kvm/vmx/vmx.h +5 -2
index 349d96e68f96..645b0343e88c 100644
@@ -487,7 +487,8 @@ static inline u8 vmx_get_rvi(void)
 	 VM_ENTRY_LOAD_BNDCFGS |					\
 	 VM_ENTRY_PT_CONCEAL_PIP |					\
 	 VM_ENTRY_LOAD_IA32_RTIT_CTL |					\
-	 VM_ENTRY_LOAD_CET_STATE)
+	 VM_ENTRY_LOAD_CET_STATE |					\
+	 VM_ENTRY_LOAD_IA32_FRED)
 
 #define __KVM_REQUIRED_VMX_VM_EXIT_CONTROLS				\
 	(VM_EXIT_SAVE_DEBUG_CONTROLS |					\
@@ -514,7 +515,9 @@ static inline u8 vmx_get_rvi(void)
 	       VM_EXIT_ACTIVATE_SECONDARY_CONTROLS)
 
 #define KVM_REQUIRED_VMX_SECONDARY_VM_EXIT_CONTROLS (0)
-#define KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS (0)
+#define KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS			\
+	     (SECONDARY_VM_EXIT_SAVE_IA32_FRED |			\
+	      SECONDARY_VM_EXIT_LOAD_IA32_FRED)
 
 #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL			\
 	(PIN_BASED_EXT_INTR_MASK |					\
author maintainer
#4 lore raw

Re: [PATCH v9 02/22] KVM: VMX: Initialize VM entry/exit FRED controls in vmcs_config

Quoted 12 lines · Binbin Wu
Quoted 8 lines

Setup VM entry/exit FRED controls in the global vmcs_config for proper FRED VMCS fields management:
1) load guest FRED state upon VM entry.
2) save guest FRED state during VM exit.
3) load host FRED state during VM exit.

Quoted 4 lines

Nit:
I think it's worth noting that IA32_FRED_RSP0 and IA32_FRED_SSP0 are treated differently. The change log might need more context on which MSRs are atomically switched in order to describe that though.

I deliberately omitted this point, as I felt that mentioning special cases would be a distraction within a patch intended for the common framework.

In the overall patch series, this particular patch is like initially claiming FRED simplifies virtualization state management, only to introduce complexities in subsequent patches.

author maintainer
#5 lore raw

PATCH v9 3/22 KVM: VMX: Disable FRED if FRED consistency checks fail

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Do not virtualize FRED if FRED consistency checks fail.

Either on broken hardware, or when run KVM on top of another hypervisor before the underlying hypervisor implements nested FRED correctly.

---

Changes in v5:
* Drop the cpu_feature_enabled() in cpu_has_vmx_fred() (Sean).
* Add TB from Xuelian Guo.

Change in v4:
* Call out the reason why not check FRED VM-exit controls in cpu_has_vmx_fred() (Chao Gao).

---
 arch/x86/kvm/vmx/capabilities.h | 10 ++++++++++
 arch/x86/kvm/vmx/vmx.c          |  3 +++
 2 files changed, 13 insertions(+)
arch/x86/kvm/vmx/capabilities.h +10 -0
index 6bd67c40ca3b..651507627ef3 100644
@@ -405,6 +405,16 @@ static inline bool vmx_pebs_supported(void)
 	return boot_cpu_has(X86_FEATURE_PEBS) && kvm_pmu_cap.pebs_ept;
 }
 
+static inline bool cpu_has_vmx_fred(void)
+{
+	/*
+	 * setup_vmcs_config() guarantees FRED VM-entry/exit controls
+	 * are either all set or none.  So, no need to check FRED VM-exit
+	 * controls.
+	 */
+	return (vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_IA32_FRED);
+}
+
 static inline bool cpu_has_notify_vmexit(void)
 {
 	return vmcs_config.cpu_based_2nd_exec_ctrl &
arch/x86/kvm/vmx/vmx.c +3 -0
index be48ba2d70e1..fcfa99160018 100644
@@ -8020,6 +8020,9 @@ static __init void vmx_set_cpu_caps(void)
 		kvm_cpu_cap_check_and_set(X86_FEATURE_DTES64);
 	}
 
+	if (!cpu_has_vmx_fred())
+		kvm_cpu_cap_clear(X86_FEATURE_FRED);
+
 	if (!enable_pmu)
 		kvm_cpu_cap_clear(X86_FEATURE_PDCM);
 	kvm_caps.supported_perf_cap = vmx_get_perf_capabilities();
Signature · 1 line
2.51.0
#6 lore raw

Re: [PATCH v9 03/22] KVM: VMX: Disable FRED if FRED consistency checks fail

Quoted 5 lines · Xin Li (Intel)
arch/x86/kvm/vmx/capabilities.h +0 -0
index 6bd67c40ca3b..651507627ef3 100644

arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pebs_supported(void) · line 414

Quoted 10 lines
arch/x86/kvm/vmx/capabilities.h +7 -0
@@ -405,6 +405,16 @@ static inline bool vmx_pebs_supported(void)
 	return boot_cpu_has(X86_FEATURE_PEBS) && kvm_pmu_cap.pebs_ept;
 }
 
+static inline bool cpu_has_vmx_fred(void)
+{
+	/*
+	 * setup_vmcs_config() guarantees FRED VM-entry/exit controls
+	 * are either all set or none.  So, no need to check FRED VM-exit
+	 * controls.
+	 */

Eh, omit the comment, there are at least six other control pairs that use this logic.

arch/x86/kvm/vmx/capabilities.h static inline bool vmx_pebs_supported(void) · line 415

Quoted 1 line
arch/x86/kvm/vmx/capabilities.h +1 -0
+	return (vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_IA32_FRED);

Unnecessary parentheses.

Quoted 22 lines
arch/x86/kvm/vmx/capabilities.h +2 -0
+}
+
 static inline bool cpu_has_notify_vmexit(void)
 {
 	return vmcs_config.cpu_based_2nd_exec_ctrl &
arch/x86/kvm/vmx/vmx.c +3 -0
index be48ba2d70e1..fcfa99160018 100644
@@ -8020,6 +8020,9 @@ static __init void vmx_set_cpu_caps(void)
 		kvm_cpu_cap_check_and_set(X86_FEATURE_DTES64);
 	}
 
+	if (!cpu_has_vmx_fred())
+		kvm_cpu_cap_clear(X86_FEATURE_FRED);
+
 	if (!enable_pmu)
 		kvm_cpu_cap_clear(X86_FEATURE_PDCM);
 	kvm_caps.supported_perf_cap = vmx_get_perf_capabilities();
Signature · 1 line
2.51.0
author maintainer
#7 lore raw

PATCH v9 4/22 x86/cea: Prefix event stack names with ESTACK_

In reply to Xin Li (Intel)

Add the ESTACK_ prefix to event stack names to improve clarity and readability. Without the prefix, names like DF, NMI, and DB are too brief and potentially ambiguous.

This renaming also prepares for converting __this_cpu_ist_top_va from a macro into a function that accepts an enum exception_stack_ordering argument, without requiring changes to existing callsites.

---

Changes in v7:
* Move rename code to this patch (Dave Hansen).
* Fix a vertical alignment (Dave Hansen).

---
 arch/x86/coco/sev/noinstr.c           |  4 ++--
 arch/x86/coco/sev/vc-handle.c         |  2 +-
 arch/x86/include/asm/cpu_entry_area.h | 26 +++++++++++++-------------
 arch/x86/kernel/cpu/common.c          | 10 +++++-----
 arch/x86/kernel/dumpstack_64.c        | 14 +++++++-------
 arch/x86/kernel/fred.c                |  6 +++---
 arch/x86/kernel/traps.c               |  2 +-
 arch/x86/mm/cpu_entry_area.c          | 12 ++++++------
 arch/x86/mm/fault.c                   |  2 +-
 9 files changed, 39 insertions(+), 39 deletions(-)
arch/x86/coco/sev/noinstr.c +2 -2
index b527eafb6312..c3985c9b232c 100644
@@ -30,7 +30,7 @@ static __always_inline bool on_vc_stack(struct pt_regs *regs)
 	if (ip_within_syscall_gap(regs))
 		return false;
 
-	return ((sp >= __this_cpu_ist_bottom_va(VC)) && (sp < __this_cpu_ist_top_va(VC)));
+	return ((sp >= __this_cpu_ist_bottom_va(ESTACK_VC)) && (sp < __this_cpu_ist_top_va(ESTACK_VC)));
 }
 
 /*
@@ -82,7 +82,7 @@ void noinstr __sev_es_ist_exit(void)
 	/* Read IST entry */
 	ist = __this_cpu_read(cpu_tss_rw.x86_tss.ist[IST_INDEX_VC]);
 
-	if (WARN_ON(ist == __this_cpu_ist_top_va(VC)))
+	if (WARN_ON(ist == __this_cpu_ist_top_va(ESTACK_VC)))
 		return;
 
 	/* Read back old IST entry and write it to the TSS */
arch/x86/coco/sev/vc-handle.c +1 -1
index 7fc136a35334..1d3f086ae4c3 100644
@@ -871,7 +871,7 @@ static enum es_result vc_handle_exitcode(struct es_em_ctxt *ctxt,
 
 static __always_inline bool is_vc2_stack(unsigned long sp)
 {
-	return (sp >= __this_cpu_ist_bottom_va(VC2) && sp < __this_cpu_ist_top_va(VC2));
+	return (sp >= __this_cpu_ist_bottom_va(ESTACK_VC2) && sp < __this_cpu_ist_top_va(ESTACK_VC2));
 }
 
 static __always_inline bool vc_from_invalid_context(struct pt_regs *regs)
arch/x86/include/asm/cpu_entry_area.h +13 -13
index 462fc34f1317..d0f884c28178 100644
@@ -18,19 +18,19 @@
 
 /* Macro to enforce the same ordering and stack sizes */
 #define ESTACKS_MEMBERS(guardsize, optional_stack_size)		\
-	char	DF_stack_guard[guardsize];			\
-	char	DF_stack[EXCEPTION_STKSZ];			\
-	char	NMI_stack_guard[guardsize];			\
-	char	NMI_stack[EXCEPTION_STKSZ];			\
-	char	DB_stack_guard[guardsize];			\
-	char	DB_stack[EXCEPTION_STKSZ];			\
-	char	MCE_stack_guard[guardsize];			\
-	char	MCE_stack[EXCEPTION_STKSZ];			\
-	char	VC_stack_guard[guardsize];			\
-	char	VC_stack[optional_stack_size];			\
-	char	VC2_stack_guard[guardsize];			\
-	char	VC2_stack[optional_stack_size];			\
-	char	IST_top_guard[guardsize];			\
+	char	ESTACK_DF_stack_guard[guardsize];		\
+	char	ESTACK_DF_stack[EXCEPTION_STKSZ];		\
+	char	ESTACK_NMI_stack_guard[guardsize];		\
+	char	ESTACK_NMI_stack[EXCEPTION_STKSZ];		\
+	char	ESTACK_DB_stack_guard[guardsize];		\
+	char	ESTACK_DB_stack[EXCEPTION_STKSZ];		\
+	char	ESTACK_MCE_stack_guard[guardsize];		\
+	char	ESTACK_MCE_stack[EXCEPTION_STKSZ];		\
+	char	ESTACK_VC_stack_guard[guardsize];		\
+	char	ESTACK_VC_stack[optional_stack_size];		\
+	char	ESTACK_VC2_stack_guard[guardsize];		\
+	char	ESTACK_VC2_stack[optional_stack_size];		\
+	char	ESTACK_IST_top_guard[guardsize];		\
 
 /* The exception stacks' physical storage. No guard pages required */
 struct exception_stacks {
arch/x86/kernel/cpu/common.c +5 -5
index c7d3512914ca..5f78b8f63d8d 100644
@@ -2332,12 +2332,12 @@ static inline void setup_getcpu(int cpu)
 static inline void tss_setup_ist(struct tss_struct *tss)
 {
 	/* Set up the per-CPU TSS IST stacks */
-	tss->x86_tss.ist[IST_INDEX_DF] = __this_cpu_ist_top_va(DF);
-	tss->x86_tss.ist[IST_INDEX_NMI] = __this_cpu_ist_top_va(NMI);
-	tss->x86_tss.ist[IST_INDEX_DB] = __this_cpu_ist_top_va(DB);
-	tss->x86_tss.ist[IST_INDEX_MCE] = __this_cpu_ist_top_va(MCE);
+	tss->x86_tss.ist[IST_INDEX_DF]	= __this_cpu_ist_top_va(ESTACK_DF);
+	tss->x86_tss.ist[IST_INDEX_NMI]	= __this_cpu_ist_top_va(ESTACK_NMI);
+	tss->x86_tss.ist[IST_INDEX_DB]	= __this_cpu_ist_top_va(ESTACK_DB);
+	tss->x86_tss.ist[IST_INDEX_MCE]	= __this_cpu_ist_top_va(ESTACK_MCE);
 	/* Only mapped when SEV-ES is active */
-	tss->x86_tss.ist[IST_INDEX_VC] = __this_cpu_ist_top_va(VC);
+	tss->x86_tss.ist[IST_INDEX_VC]	= __this_cpu_ist_top_va(ESTACK_VC);
 }
 #else /* CONFIG_X86_64 */
 static inline void tss_setup_ist(struct tss_struct *tss) { }
arch/x86/kernel/dumpstack_64.c +7 -7
index 6c5defd6569a..40f51e278171 100644
@@ -73,7 +73,7 @@ struct estack_pages {
 	 PFN_DOWN(CEA_ESTACK_OFFS(st) + CEA_ESTACK_SIZE(st) - 1)] = {	\
 		.offs	= CEA_ESTACK_OFFS(st),				\
 		.size	= CEA_ESTACK_SIZE(st),				\
-		.type	= STACK_TYPE_EXCEPTION + ESTACK_ ##st, }
+		.type	= STACK_TYPE_EXCEPTION + st, }
 
 /*
  * Array of exception stack page descriptors. If the stack is larger than
@@ -83,12 +83,12 @@ struct estack_pages {
  */
 static const
 struct estack_pages estack_pages[CEA_ESTACK_PAGES] ____cacheline_aligned = {
-	EPAGERANGE(DF),
-	EPAGERANGE(NMI),
-	EPAGERANGE(DB),
-	EPAGERANGE(MCE),
-	EPAGERANGE(VC),
-	EPAGERANGE(VC2),
+	EPAGERANGE(ESTACK_DF),
+	EPAGERANGE(ESTACK_NMI),
+	EPAGERANGE(ESTACK_DB),
+	EPAGERANGE(ESTACK_MCE),
+	EPAGERANGE(ESTACK_VC),
+	EPAGERANGE(ESTACK_VC2),
 };
 
 static __always_inline bool in_exception_stack(unsigned long *stack, struct stack_info *info)
arch/x86/kernel/fred.c +3 -3
index 816187da3a47..06d944a3d051 100644
@@ -87,7 +87,7 @@ void cpu_init_fred_rsps(void)
 	       FRED_STKLVL(X86_TRAP_DF,  FRED_DF_STACK_LEVEL));
 
 	/* The FRED equivalents to IST stacks... */
-	wrmsrq(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
-	wrmsrq(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
-	wrmsrq(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));
+	wrmsrq(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
+	wrmsrq(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
+	wrmsrq(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));
 }
arch/x86/kernel/traps.c +1 -1
index 6b22611e69cc..47b7b7495114 100644
@@ -954,7 +954,7 @@ asmlinkage __visible noinstr struct pt_regs *vc_switch_off_ist(struct pt_regs *r
 
 	if (!get_stack_info_noinstr(stack, current, &info) || info.type == STACK_TYPE_ENTRY ||
 	    info.type > STACK_TYPE_EXCEPTION_LAST)
-		sp = __this_cpu_ist_top_va(VC2);
+		sp = __this_cpu_ist_top_va(ESTACK_VC2);
 
 sync:
 	/*
arch/x86/mm/cpu_entry_area.c +6 -6
index 575f863f3c75..9fa371af8abc 100644
@@ -151,15 +151,15 @@ static void __init percpu_setup_exception_stacks(unsigned int cpu)
 	 * by guard pages so each stack must be mapped separately. DB2 is
 	 * not mapped; it just exists to catch triple nesting of #DB.
 	 */
-	cea_map_stack(DF);
-	cea_map_stack(NMI);
-	cea_map_stack(DB);
-	cea_map_stack(MCE);
+	cea_map_stack(ESTACK_DF);
+	cea_map_stack(ESTACK_NMI);
+	cea_map_stack(ESTACK_DB);
+	cea_map_stack(ESTACK_MCE);
 
 	if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) {
 		if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
-			cea_map_stack(VC);
-			cea_map_stack(VC2);
+			cea_map_stack(ESTACK_VC);
+			cea_map_stack(ESTACK_VC2);
 		}
 	}
 }
arch/x86/mm/fault.c +1 -1
index 998bd807fc7b..1804eb86cc14 100644
@@ -671,7 +671,7 @@ page_fault_oops(struct pt_regs *regs, unsigned long error_code,
 		 * and then double-fault, though, because we're likely to
 		 * break the console driver and lose most of the stack dump.
 		 */
-		call_on_stack(__this_cpu_ist_top_va(DF) - sizeof(void*),
+		call_on_stack(__this_cpu_ist_top_va(ESTACK_DF) - sizeof(void*),
 			      handle_stack_overflow,
 			      ASM_CALL_ARG3,
 			      , [arg1] "r" (regs), [arg2] "r" (address), [arg3] "r" (&info));
Signature · 1 line
2.51.0
author maintainer
#8 lore raw

PATCH v9 5/22 x86/cea: Use array indexing to simplify exception stack access

In reply to Xin Li (Intel)

Refactor struct cea_exception_stacks to leverage array indexing for exception stack access, improving code clarity and eliminating the need for the ESTACKS_MEMBERS() macro.

Convert __this_cpu_ist_{bottom,top}_va() from macros to functions, allowing removal of the now-obsolete CEA_ESTACK_BOT and CEA_ESTACK_TOP macros.

Also drop CEA_ESTACK_SIZE, which just duplicated EXCEPTION_STKSZ.

---

Change in v9:
* Refactor first and then export in a separate patch (Dave Hansen).

Change in v7:
* Access cea_exception_stacks using array indexing (Dave Hansen).
* Use BUILD_BUG_ON(ESTACK_DF != 0) to ensure the starting index is 0 (Dave Hansen).
* Remove Suggested-bys (Dave Hansen).
* Move rename code in a separate patch (Dave Hansen).

Change in v5:
* Export accessor instead of data (Christoph Hellwig).
* Add TB from Xuelian Guo.

Change in v4:
* Rewrite the change log and add comments to the export (Dave Hansen).

---
 arch/x86/include/asm/cpu_entry_area.h | 52 ++++++++++++---------------
 arch/x86/kernel/dumpstack_64.c        |  4 +--
 arch/x86/mm/cpu_entry_area.c          | 21 ++++++++++-
 3 files changed, 44 insertions(+), 33 deletions(-)
arch/x86/include/asm/cpu_entry_area.h +22 -30
index d0f884c28178..509e52fc3a0f 100644
@@ -16,6 +16,19 @@
 #define VC_EXCEPTION_STKSZ	0
 #endif
 
+/*
+ * The exception stack ordering in [cea_]exception_stacks
+ */
+enum exception_stack_ordering {
+	ESTACK_DF,
+	ESTACK_NMI,
+	ESTACK_DB,
+	ESTACK_MCE,
+	ESTACK_VC,
+	ESTACK_VC2,
+	N_EXCEPTION_STACKS
+};
+
 /* Macro to enforce the same ordering and stack sizes */
 #define ESTACKS_MEMBERS(guardsize, optional_stack_size)		\
 	char	ESTACK_DF_stack_guard[guardsize];		\
@@ -39,37 +52,22 @@ struct exception_stacks {
 
 /* The effective cpu entry area mapping with guard pages. */
 struct cea_exception_stacks {
-	ESTACKS_MEMBERS(PAGE_SIZE, EXCEPTION_STKSZ)
-};
-
-/*
- * The exception stack ordering in [cea_]exception_stacks
- */
-enum exception_stack_ordering {
-	ESTACK_DF,
-	ESTACK_NMI,
-	ESTACK_DB,
-	ESTACK_MCE,
-	ESTACK_VC,
-	ESTACK_VC2,
-	N_EXCEPTION_STACKS
+	struct {
+		char stack_guard[PAGE_SIZE];
+		char stack[EXCEPTION_STKSZ];
+	} event_stacks[N_EXCEPTION_STACKS];
+	char IST_top_guard[PAGE_SIZE];
 };
 
-#define CEA_ESTACK_SIZE(st)					\
-	sizeof(((struct cea_exception_stacks *)0)->st## _stack)
-
-#define CEA_ESTACK_BOT(ceastp, st)				\
-	((unsigned long)&(ceastp)->st## _stack)
-
-#define CEA_ESTACK_TOP(ceastp, st)				\
-	(CEA_ESTACK_BOT(ceastp, st) + CEA_ESTACK_SIZE(st))
-
 #define CEA_ESTACK_OFFS(st)					\
-	offsetof(struct cea_exception_stacks, st## _stack)
+	offsetof(struct cea_exception_stacks, event_stacks[st].stack)
 
 #define CEA_ESTACK_PAGES					\
 	(sizeof(struct cea_exception_stacks) / PAGE_SIZE)
 
+extern unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack);
+extern unsigned long __this_cpu_ist_bottom_va(enum exception_stack_ordering stack);
+
 #endif
 
 #ifdef CONFIG_X86_32
@@ -144,10 +142,4 @@ static __always_inline struct entry_stack *cpu_entry_stack(int cpu)
 	return &get_cpu_entry_area(cpu)->entry_stack_page.stack;
 }
 
-#define __this_cpu_ist_top_va(name)					\
-	CEA_ESTACK_TOP(__this_cpu_read(cea_exception_stacks), name)
-
-#define __this_cpu_ist_bottom_va(name)					\
-	CEA_ESTACK_BOT(__this_cpu_read(cea_exception_stacks), name)
-
 #endif
arch/x86/kernel/dumpstack_64.c +2 -2
index 40f51e278171..93b10b264e53 100644
@@ -70,9 +70,9 @@ struct estack_pages {
 
 #define EPAGERANGE(st)							\
 	[PFN_DOWN(CEA_ESTACK_OFFS(st)) ...				\
-	 PFN_DOWN(CEA_ESTACK_OFFS(st) + CEA_ESTACK_SIZE(st) - 1)] = {	\
+	 PFN_DOWN(CEA_ESTACK_OFFS(st) + EXCEPTION_STKSZ - 1)] = {	\
 		.offs	= CEA_ESTACK_OFFS(st),				\
-		.size	= CEA_ESTACK_SIZE(st),				\
+		.size	= EXCEPTION_STKSZ,				\
 		.type	= STACK_TYPE_EXCEPTION + st, }
 
 /*
arch/x86/mm/cpu_entry_area.c +20 -1
index 9fa371af8abc..b3d90f9cfbb1 100644
@@ -18,6 +18,25 @@ static DEFINE_PER_CPU_PAGE_ALIGNED(struct entry_stack_page, entry_stack_storage)
 static DEFINE_PER_CPU_PAGE_ALIGNED(struct exception_stacks, exception_stacks);
 DEFINE_PER_CPU(struct cea_exception_stacks*, cea_exception_stacks);
 
+/*
+ * Typically invoked by entry code, so must be noinstr.
+ */
+noinstr unsigned long __this_cpu_ist_bottom_va(enum exception_stack_ordering stack)
+{
+	struct cea_exception_stacks *s;
+
+	BUILD_BUG_ON(ESTACK_DF != 0);
+
+	s = __this_cpu_read(cea_exception_stacks);
+
+	return (unsigned long)&s->event_stacks[stack].stack;
+}
+
+noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack)
+{
+	return __this_cpu_ist_bottom_va(stack) + EXCEPTION_STKSZ;
+}
+
 static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, _cea_offset);
 
 static __always_inline unsigned int cea_offset(unsigned int cpu)
@@ -132,7 +151,7 @@ static void __init percpu_setup_debug_store(unsigned int cpu)
 
 #define cea_map_stack(name) do {					\
 	npages = sizeof(estacks->name## _stack) / PAGE_SIZE;		\
-	cea_map_percpu_pages(cea->estacks.name## _stack,		\
+	cea_map_percpu_pages(cea->estacks.event_stacks[name].stack,	\
 			estacks->name## _stack, npages, PAGE_KERNEL);	\
 	} while (0)
 
Signature · 1 line
2.51.0
Acked-by on v9 5/22
#9 lore raw

Re: [PATCH v9 05/22] x86/cea: Use array indexing to simplify exception stack access

Quoted 8 lines · Xin Li (Intel)

Refactor struct cea_exception_stacks to leverage array indexing for exception stack access, improving code clarity and eliminating the need for the ESTACKS_MEMBERS() macro.

Convert __this_cpu_ist_{bottom,top}_va() from macros to functions, allowing removal of the now-obsolete CEA_ESTACK_BOT and CEA_ESTACK_TOP macros.

Quoted 7 lines

Also drop CEA_ESTACK_SIZE, which just duplicated EXCEPTION_STKSZ.

---

Change in v9:
* Refactor first and then export in a separate patch (Dave Hansen).

Thanks for the changes. This also removes the extra union{} that was in the last version for padding.

author maintainer
#10 lore raw

Re: [PATCH v9 05/22] x86/cea: Use array indexing to simplify exception stack access

Quoted 15 lines · Dave Hansen
Quoted 10 lines · Xin Li (Intel)

Refactor struct cea_exception_stacks to leverage array indexing for exception stack access, improving code clarity and eliminating the need for the ESTACKS_MEMBERS() macro.

Convert __this_cpu_ist_{bottom,top}_va() from macros to functions, allowing removal of the now-obsolete CEA_ESTACK_BOT and CEA_ESTACK_TOP macros.

Also drop CEA_ESTACK_SIZE, which just duplicated EXCEPTION_STKSZ.

Quoted 8 lines
Quoted 5 lines

---

Change in v9:
* Refactor first and then export in a separate patch (Dave Hansen).

Thanks for the changes. This also removes the extra union{} that was in the last version for padding.

I would say you foresaw it because you suggested to use array indexing:

https://lore.kernel.org/lkml/[email protected]/

Thanks a lot for making it much cleaner.
Xin

maintainer
#11 lore raw

Re: [PATCH v9 05/22] x86/cea: Use array indexing to simplify exception stack access

In reply to Xin Li (Intel)

arch/x86/mm/cpu_entry_area.c static DEFINE_PER_CPU_PAGE_ALIGNED(struct entry_stack_page, entry_stack_storage) · line 36

Quoted 2 lines · Xin Li (Intel)
arch/x86/mm/cpu_entry_area.c +2 -0
+noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack)
+{

You better check this @stack is within the array bounds of event_stacks[].

Quoted 2 lines
arch/x86/mm/cpu_entry_area.c +2 -0
+	return __this_cpu_ist_bottom_va(stack) + EXCEPTION_STKSZ;
+}
Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#12 lore raw

PATCH v9 6/22 x86/cea: Export __this_cpu_ist_top_va() to KVM

In reply to Xin Li (Intel)

Export __this_cpu_ist_top_va() to allow KVM to retrieve the per-CPU exception stack top.

FRED introduced new fields in the host-state area of the VMCS for stack levels 1->3 (HOST_IA32_FRED_RSP[123]), each respectively corresponding to per-CPU exception stacks for #DB, NMI and #DF. KVM must populate these fields each time a vCPU is loaded onto a CPU.

---
 arch/x86/mm/cpu_entry_area.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/x86/mm/cpu_entry_area.c +6 -0
index b3d90f9cfbb1..e507621d5c20 100644
@@ -19,6 +19,11 @@ static DEFINE_PER_CPU_PAGE_ALIGNED(struct exception_stacks, exception_stacks);
 DEFINE_PER_CPU(struct cea_exception_stacks*, cea_exception_stacks);
 
 /*
+ * FRED introduced new fields in the host-state area of the VMCS for
+ * stack levels 1->3 (HOST_IA32_FRED_RSP[123]), each respectively
+ * corresponding to per CPU stacks for #DB, NMI and #DF.  KVM must
+ * populate these each time a vCPU is loaded onto a CPU.
+ *
  * Typically invoked by entry code, so must be noinstr.
  */
 noinstr unsigned long __this_cpu_ist_bottom_va(enum exception_stack_ordering stack)
@@ -36,6 +41,7 @@ noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack)
 {
 	return __this_cpu_ist_bottom_va(stack) + EXCEPTION_STKSZ;
 }
+EXPORT_SYMBOL_FOR_MODULES(__this_cpu_ist_top_va, "kvm-intel");
 
 static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, _cea_offset);
 
Signature · 1 line
2.51.0
Acked-by on v9 6/22
#13 lore raw

Re: [PATCH v9 06/22] x86/cea: Export __this_cpu_ist_top_va() to KVM

Quoted 7 lines · Xin Li (Intel)

Export __this_cpu_ist_top_va() to allow KVM to retrieve the per-CPU exception stack top.

FRED introduced new fields in the host-state area of the VMCS for stack levels 1->3 (HOST_IA32_FRED_RSP[123]), each respectively corresponding to per-CPU exception stacks for #DB, NMI and #DF. KVM must populate these fields each time a vCPU is loaded onto a CPU.

maintainer
#14 lore raw

Re: [PATCH v9 06/22] x86/cea: Export __this_cpu_ist_top_va() to KVM

In reply to Xin Li (Intel)

arch/x86/mm/cpu_entry_area.c noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack) · line 44

Quoted 5 lines · Xin Li (Intel)
arch/x86/mm/cpu_entry_area.c +1 -0
@@ -36,6 +41,7 @@ noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack)
 {
 	return __this_cpu_ist_bottom_va(stack) + EXCEPTION_STKSZ;
 }
+EXPORT_SYMBOL_FOR_MODULES(__this_cpu_ist_top_va, "kvm-intel");

Why is this function name still kept with the "__" prefix but it is being exported at the same time?

It looks to me like we're exporting the wrong thing as the "__" kinda says it is an internal helper.

Just drop the prefix and call it something more sensible please. The caller couldn't care less about "ist_top_va".

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#15 lore raw

Re: [PATCH v9 06/22] x86/cea: Export __this_cpu_ist_top_va() to KVM

Quoted 14 lines · Borislav Petkov

arch/x86/mm/cpu_entry_area.c noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack) · line 44

Quoted 5 lines · Xin Li (Intel)
arch/x86/mm/cpu_entry_area.c +1 -0
@@ -36,6 +41,7 @@ noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack)
 {
 return __this_cpu_ist_bottom_va(stack) + EXCEPTION_STKSZ;
 }
+EXPORT_SYMBOL_FOR_MODULES(__this_cpu_ist_top_va, "kvm-intel");

Why is this function name still kept with the "__" prefix but it is being exported at the same time?

Quoted 5 lines

It looks to me like we're exporting the wrong thing as the "__" kinda says it is an internal helper.

Just drop the prefix and call it something more sensible please. The caller couldn't care less about "ist_top_va".

Right, the “__” prefix no longer makes sense.

What is the right order of rename and refactor?

I think we usually do renames in the first patch and then refactor in a following patch.

maintainer
#16 lore raw

Re: [PATCH v9 06/22] x86/cea: Export __this_cpu_ist_top_va() to KVM

Quoted 1 line · Xin Li

What is the right order of rename and refactor?

You can rename it when you do the export. The name should be more sensible, as said.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#17 lore raw

Re: [PATCH v9 06/22] x86/cea: Export __this_cpu_ist_top_va() to KVM

In reply to Borislav Petkov
Quoted 14 lines · Borislav Petkov

Review noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack) · line 44

Quoted 5 lines · Xin Li (Intel)
(quoted diff) +1 -0
@@ -36,6 +41,7 @@ noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack)
 {
 return __this_cpu_ist_bottom_va(stack) + EXCEPTION_STKSZ;
 }
+EXPORT_SYMBOL_FOR_MODULES(__this_cpu_ist_top_va, "kvm-intel");

Why is this function name still kept with the "__" prefix but it is being exported at the same time?

Quoted 5 lines

It looks to me like we're exporting the wrong thing as the "__" kinda says it is an internal helper.

Just drop the prefix and call it something more sensible please. The caller couldn't care less about "ist_top_va".

Sean suggested to replace direct, raw use of __this_cpu_ist_top_va() with a self-explanatory this_cpu_fred_rsp() helper for better readability.

https://lore.kernel.org/lkml/[email protected]/

So __this_cpu_ist_top_va() no longer needs to be exported, thus no need to do the renaming.

The new patch is below:

commit 7f0d77e48751bd2f3f65f93eaf466257382a25b9

Author: Xin Li <[email protected]>
Date:   Fri Oct 24 11:52:32 2025 -0700

x86/fred: Export this_cpu_fred_rsp() for KVM usage

Introduce and export this_cpu_fred_rsp() to provide KVM with a self- explanatory interface for retrieving per-CPU FRED regular stacks for stack levels 1->3.

FRED introduced new fields in the VMCS host-state area for stack levels 1–>3 (HOST_IA32_FRED_RSP[123]), which correspond to the per-CPU FRED regular stacks for stack levels 1->3. KVM must populate these fields each time a vCPU is loaded onto a CPU to ensure a complete valid FRED event delivery context immediately after any VM-Exits.

---

Change in v10:
* Replace direct, raw use of __this_cpu_ist_top_va() with a self- explanatory this_cpu_fred_rsp() helper for better readability (Sean).

arch/x86/include/asm/fred.h +10 -0
index 2bb65677c079..7eea65bfc838 100644
@@ -35,6 +35,13 @@
 
 #ifndef __ASSEMBLER__
 
+enum fred_stack_level {
+	FRED_STACK_LEVEL_0,
+	FRED_STACK_LEVEL_1,
+	FRED_STACK_LEVEL_2,
+	FRED_STACK_LEVEL_3
+};
+
 #ifdef CONFIG_X86_FRED
 #include <linux/kernel.h>
 #include <linux/sched/task_stack.h>
@@ -105,6 +112,8 @@ static __always_inline void fred_update_rsp0(void)
 		__this_cpu_write(fred_rsp0, rsp0);
 	}
 }
+
+unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl);
 #else /* CONFIG_X86_FRED */
 static __always_inline unsigned long fred_event_data(struct pt_regs *regs) { return 0; }
 static inline void cpu_init_fred_exceptions(void) { }
@@ -113,6 +122,7 @@ static inline void fred_complete_exception_setup(void) { }
 static inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) { }
 static inline void fred_sync_rsp0(unsigned long rsp0) { }
 static inline void fred_update_rsp0(void) { }
+static unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl) { return 0; }
 #endif /* CONFIG_X86_FRED */
 #endif /* !__ASSEMBLER__ */
 
arch/x86/kernel/fred.c +20 -3
index 433c4a6f1773..363c53701012 100644
@@ -72,6 +72,23 @@ void cpu_init_fred_exceptions(void)
 	setup_clear_cpu_cap(X86_FEATURE_SYSCALL32);
 }
 
+unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl)
+{
+	switch (lvl) {
+	case FRED_STACK_LEVEL_0:
+		return __this_cpu_read(fred_rsp0);
+	case FRED_STACK_LEVEL_1:
+		return __this_cpu_ist_top_va(ESTACK_DB);
+	case FRED_STACK_LEVEL_2:
+		return __this_cpu_ist_top_va(ESTACK_NMI);
+	case FRED_STACK_LEVEL_3:
+		return __this_cpu_ist_top_va(ESTACK_DF);
+	default:
+		BUG();
+	}
+}
+EXPORT_SYMBOL_FOR_MODULES(this_cpu_fred_rsp, "kvm-intel");
+
 /* Must be called after setup_cpu_entry_areas() */
 void cpu_init_fred_rsps(void)
 {
@@ -87,7 +104,7 @@ void cpu_init_fred_rsps(void)
 	       FRED_STKLVL(X86_TRAP_DF,  FRED_DF_STACK_LEVEL));
 
 	/* The FRED equivalents to IST stacks... */
-	wrmsrq(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
-	wrmsrq(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
-	wrmsrq(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));
+	wrmsrq(MSR_IA32_FRED_RSP1, this_cpu_fred_rsp(FRED_STACK_LEVEL_1));
+	wrmsrq(MSR_IA32_FRED_RSP2, this_cpu_fred_rsp(FRED_STACK_LEVEL_2));
+	wrmsrq(MSR_IA32_FRED_RSP3, this_cpu_fred_rsp(FRED_STACK_LEVEL_3));
 }
#18 lore raw

Re: [PATCH v9 06/22] x86/cea: Export __this_cpu_ist_top_va() to KVM

Quoted 6 lines · Xin Li
Quoted 1 line · Borislav Petkov
arch/x86/include/asm/fred.h +0 -0
index 2bb65677c079..7eea65bfc838 100644

arch/x86/include/asm/fred.h line 42

Quoted 8 lines
arch/x86/include/asm/fred.h +5 -0
@@ -35,6 +35,13 @@
 
 #ifndef __ASSEMBLER__
 
+enum fred_stack_level {
+	FRED_STACK_LEVEL_0,
+	FRED_STACK_LEVEL_1,
+	FRED_STACK_LEVEL_2,
+	FRED_STACK_LEVEL_3

Why bother with a layer of indirection and more enums? Just pivot on the MSR index.

Quoted 34 lines
arch/x86/include/asm/fred.h +3 -0
 #ifdef CONFIG_X86_FRED
 #include <linux/kernel.h>
 #include <linux/sched/task_stack.h>
@@ -105,6 +112,8 @@ static __always_inline void fred_update_rsp0(void)
 		__this_cpu_write(fred_rsp0, rsp0);
 	}
 }
+
+unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl);
 #else /* CONFIG_X86_FRED */
 static __always_inline unsigned long fred_event_data(struct pt_regs *regs) { return 0; }
 static inline void cpu_init_fred_exceptions(void) { }
@@ -113,6 +122,7 @@ static inline void fred_complete_exception_setup(void) { }
 static inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) { }
 static inline void fred_sync_rsp0(unsigned long rsp0) { }
 static inline void fred_update_rsp0(void) { }
+static unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl) { return 0; }
 #endif /* CONFIG_X86_FRED */
 #endif /* !__ASSEMBLER__ */
 
arch/x86/kernel/fred.c +6 -0
index 433c4a6f1773..363c53701012 100644
@@ -72,6 +72,23 @@ void cpu_init_fred_exceptions(void)
 	setup_clear_cpu_cap(X86_FEATURE_SYSCALL32);
 }
 
+unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl)
+{
+	switch (lvl) {
+	case FRED_STACK_LEVEL_0:
+		return __this_cpu_read(fred_rsp0);
+	case FRED_STACK_LEVEL_1:

arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 90

Quoted 10 lines
arch/x86/kernel/fred.c +10 -0
+		return __this_cpu_ist_top_va(ESTACK_DB);
+	case FRED_STACK_LEVEL_2:
+		return __this_cpu_ist_top_va(ESTACK_NMI);
+	case FRED_STACK_LEVEL_3:
+		return __this_cpu_ist_top_va(ESTACK_DF);
+	default:
+		BUG();
+	}
+}
+EXPORT_SYMBOL_FOR_MODULES(this_cpu_fred_rsp, "kvm-intel");

Meh, just do EXPORT_SYMBOL_FOR_KVM so that there's no export when KVM_X86=y|n. And it's possible AMD may need to grab the MSRs too.

Quoted 15 lines
arch/x86/kernel/fred.c +4 -3
+
 /* Must be called after setup_cpu_entry_areas() */
 void cpu_init_fred_rsps(void)
 {
@@ -87,7 +104,7 @@ void cpu_init_fred_rsps(void)
 	       FRED_STKLVL(X86_TRAP_DF,  FRED_DF_STACK_LEVEL));
 
 	/* The FRED equivalents to IST stacks... */
-	wrmsrq(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
-	wrmsrq(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
-	wrmsrq(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));
+	wrmsrq(MSR_IA32_FRED_RSP1, this_cpu_fred_rsp(FRED_STACK_LEVEL_1));
+	wrmsrq(MSR_IA32_FRED_RSP2, this_cpu_fred_rsp(FRED_STACK_LEVEL_2));
+	wrmsrq(MSR_IA32_FRED_RSP3, this_cpu_fred_rsp(FRED_STACK_LEVEL_3));
 }
author maintainer
#19 lore raw

Re: [PATCH v9 06/22] x86/cea: Export __this_cpu_ist_top_va() to KVM

Quoted 14 lines · Sean Christopherson
Quoted 9 lines · Xin Li
Quoted 1 line · Borislav Petkov
arch/x86/include/asm/fred.h +0 -0
index 2bb65677c079..7eea65bfc838 100644
@@ -35,6 +35,13 @@
 
 #ifndef __ASSEMBLER__
 
Quoted 8 lines

arch/x86/include/asm/fred.h line 42

Quoted 5 lines
arch/x86/include/asm/fred.h +5 -0
+enum fred_stack_level {
+ FRED_STACK_LEVEL_0,
+ FRED_STACK_LEVEL_1,
+ FRED_STACK_LEVEL_2,
+ FRED_STACK_LEVEL_3

Why bother with a layer of indirection and more enums? Just pivot on the MSR index.

It’s a bit weird to pass a u32 MSR index to this_cpu_fred_rsp().

We will want to rename it to this_cpu_fred_xsp() when FRED shadow stack is supported.

Quoted 40 lines
Quoted 40 lines
arch/x86/include/asm/fred.h +3 -0
 #ifdef CONFIG_X86_FRED
 #include <linux/kernel.h>
 #include <linux/sched/task_stack.h>
@@ -105,6 +112,8 @@ static __always_inline void fred_update_rsp0(void)
 __this_cpu_write(fred_rsp0, rsp0);
 }
 }
+
+unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl);
 #else /* CONFIG_X86_FRED */
 static __always_inline unsigned long fred_event_data(struct pt_regs *regs) { return 0; }
 static inline void cpu_init_fred_exceptions(void) { }
@@ -113,6 +122,7 @@ static inline void fred_complete_exception_setup(void) { }
 static inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) { }
 static inline void fred_sync_rsp0(unsigned long rsp0) { }
 static inline void fred_update_rsp0(void) { }
+static unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl) { return 0; }
 #endif /* CONFIG_X86_FRED */
 #endif /* !__ASSEMBLER__ */
 
arch/x86/kernel/fred.c +11 -0
index 433c4a6f1773..363c53701012 100644
@@ -72,6 +72,23 @@ void cpu_init_fred_exceptions(void)
 setup_clear_cpu_cap(X86_FEATURE_SYSCALL32);
 }
 
+unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl)
+{
+ switch (lvl) {
+ case FRED_STACK_LEVEL_0:
+ return __this_cpu_read(fred_rsp0);
+ case FRED_STACK_LEVEL_1:
+ return __this_cpu_ist_top_va(ESTACK_DB);
+ case FRED_STACK_LEVEL_2:
+ return __this_cpu_ist_top_va(ESTACK_NMI);
+ case FRED_STACK_LEVEL_3:
+ return __this_cpu_ist_top_va(ESTACK_DF);
Quoted 8 lines

arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 90

Quoted 5 lines
arch/x86/kernel/fred.c +5 -0
+ default:
+ BUG();
+ }
+}
+EXPORT_SYMBOL_FOR_MODULES(this_cpu_fred_rsp, "kvm-intel");

Meh, just do EXPORT_SYMBOL_FOR_KVM so that there's no export when KVM_X86=y|n. And it's possible AMD may need to grab the MSRs too.

I thought about it and checked AMD SVM FRED patches, but wasn't able to find host RSPs get set explicitly during vCPU migration.

But anyway, I will make the change.

Quoted 16 lines
Quoted 16 lines
arch/x86/kernel/fred.c +4 -3
+
 /* Must be called after setup_cpu_entry_areas() */
 void cpu_init_fred_rsps(void)
 {
@@ -87,7 +104,7 @@ void cpu_init_fred_rsps(void)
       FRED_STKLVL(X86_TRAP_DF,  FRED_DF_STACK_LEVEL));
 
 /* The FRED equivalents to IST stacks... */
- wrmsrq(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
- wrmsrq(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
- wrmsrq(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));
+ wrmsrq(MSR_IA32_FRED_RSP1, this_cpu_fred_rsp(FRED_STACK_LEVEL_1));
+ wrmsrq(MSR_IA32_FRED_RSP2, this_cpu_fred_rsp(FRED_STACK_LEVEL_2));
+ wrmsrq(MSR_IA32_FRED_RSP3, this_cpu_fred_rsp(FRED_STACK_LEVEL_3));
 }
author maintainer
#20 lore raw

PATCH v9 7/22 KVM: VMX: Initialize VMCS FRED fields

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Initialize host VMCS FRED fields with host FRED MSRs' value and guest VMCS FRED fields to 0.

FRED CPU state is managed in 9 new FRED MSRs:

        IA32_FRED_CONFIG,
        IA32_FRED_STKLVLS,
        IA32_FRED_RSP0,
        IA32_FRED_RSP1,
        IA32_FRED_RSP2,
        IA32_FRED_RSP3,
        IA32_FRED_SSP1,
        IA32_FRED_SSP2,
        IA32_FRED_SSP3,
as well as a few existing CPU registers and MSRs:
        CR4.FRED,
        IA32_STAR,
        IA32_KERNEL_GS_BASE,
        IA32_PL0_SSP (also known as IA32_FRED_SSP0).

CR4, IA32_KERNEL_GS_BASE and IA32_STAR are already well managed. Except IA32_FRED_RSP0 and IA32_FRED_SSP0, all other FRED CPU state MSRs have corresponding VMCS fields in both the host-state and guest-state areas. So KVM just needs to initialize them, and with proper VM entry/exit FRED controls, a FRED CPU will keep tracking host and guest FRED CPU state in VMCS automatically.

---

Change in v5:
* Add TB from Xuelian Guo.

Change in v4:
* Initialize host SSP[1-3] to 0s in vmx_set_constant_host_state() because Linux doesn't support kernel shadow stacks (Chao Gao).

Change in v3:
* Use structure kvm_host_values to keep host fred config & stack levels (Sean Christopherson).

Changes in v2:
* Use kvm_cpu_cap_has() instead of cpu_feature_enabled() to decouple KVM's capability to virtualize a feature and host's enabling of a feature (Chao Gao).
* Move guest FRED state init into __vmx_vcpu_reset() (Chao Gao).

---
 arch/x86/include/asm/vmx.h | 32 ++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/vmx.c     | 36 ++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/x86.h         |  3 +++
 3 files changed, 71 insertions(+)
arch/x86/include/asm/vmx.h +32 -0
index dd79d027ea70..6f8b8947c60c 100644
@@ -293,12 +293,44 @@ enum vmcs_field {
 	GUEST_BNDCFGS_HIGH              = 0x00002813,
 	GUEST_IA32_RTIT_CTL		= 0x00002814,
 	GUEST_IA32_RTIT_CTL_HIGH	= 0x00002815,
+	GUEST_IA32_FRED_CONFIG		= 0x0000281a,
+	GUEST_IA32_FRED_CONFIG_HIGH	= 0x0000281b,
+	GUEST_IA32_FRED_RSP1		= 0x0000281c,
+	GUEST_IA32_FRED_RSP1_HIGH	= 0x0000281d,
+	GUEST_IA32_FRED_RSP2		= 0x0000281e,
+	GUEST_IA32_FRED_RSP2_HIGH	= 0x0000281f,
+	GUEST_IA32_FRED_RSP3		= 0x00002820,
+	GUEST_IA32_FRED_RSP3_HIGH	= 0x00002821,
+	GUEST_IA32_FRED_STKLVLS		= 0x00002822,
+	GUEST_IA32_FRED_STKLVLS_HIGH	= 0x00002823,
+	GUEST_IA32_FRED_SSP1		= 0x00002824,
+	GUEST_IA32_FRED_SSP1_HIGH	= 0x00002825,
+	GUEST_IA32_FRED_SSP2		= 0x00002826,
+	GUEST_IA32_FRED_SSP2_HIGH	= 0x00002827,
+	GUEST_IA32_FRED_SSP3		= 0x00002828,
+	GUEST_IA32_FRED_SSP3_HIGH	= 0x00002829,
 	HOST_IA32_PAT			= 0x00002c00,
 	HOST_IA32_PAT_HIGH		= 0x00002c01,
 	HOST_IA32_EFER			= 0x00002c02,
 	HOST_IA32_EFER_HIGH		= 0x00002c03,
 	HOST_IA32_PERF_GLOBAL_CTRL	= 0x00002c04,
 	HOST_IA32_PERF_GLOBAL_CTRL_HIGH	= 0x00002c05,
+	HOST_IA32_FRED_CONFIG		= 0x00002c08,
+	HOST_IA32_FRED_CONFIG_HIGH	= 0x00002c09,
+	HOST_IA32_FRED_RSP1		= 0x00002c0a,
+	HOST_IA32_FRED_RSP1_HIGH	= 0x00002c0b,
+	HOST_IA32_FRED_RSP2		= 0x00002c0c,
+	HOST_IA32_FRED_RSP2_HIGH	= 0x00002c0d,
+	HOST_IA32_FRED_RSP3		= 0x00002c0e,
+	HOST_IA32_FRED_RSP3_HIGH	= 0x00002c0f,
+	HOST_IA32_FRED_STKLVLS		= 0x00002c10,
+	HOST_IA32_FRED_STKLVLS_HIGH	= 0x00002c11,
+	HOST_IA32_FRED_SSP1		= 0x00002c12,
+	HOST_IA32_FRED_SSP1_HIGH	= 0x00002c13,
+	HOST_IA32_FRED_SSP2		= 0x00002c14,
+	HOST_IA32_FRED_SSP2_HIGH	= 0x00002c15,
+	HOST_IA32_FRED_SSP3		= 0x00002c16,
+	HOST_IA32_FRED_SSP3_HIGH	= 0x00002c17,
 	PIN_BASED_VM_EXEC_CONTROL       = 0x00004000,
 	CPU_BASED_VM_EXEC_CONTROL       = 0x00004002,
 	EXCEPTION_BITMAP                = 0x00004004,
arch/x86/kvm/vmx/vmx.c +36 -0
index fcfa99160018..c8b5359123bf 100644
@@ -1459,6 +1459,15 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu)
 				    (unsigned long)(cpu_entry_stack(cpu) + 1));
 		}
 
+		/* Per-CPU FRED MSRs */
+		if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+#ifdef CONFIG_X86_64
+			vmcs_write64(HOST_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
+			vmcs_write64(HOST_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
+			vmcs_write64(HOST_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));
+#endif
+		}
+
 		vmx->loaded_vmcs->cpu = cpu;
 	}
 }
@@ -4330,6 +4339,17 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx)
 	 */
 	vmcs_write16(HOST_DS_SELECTOR, 0);
 	vmcs_write16(HOST_ES_SELECTOR, 0);
+
+	if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+		/* FRED CONFIG and STKLVLS are the same on all CPUs */
+		vmcs_write64(HOST_IA32_FRED_CONFIG, kvm_host.fred_config);
+		vmcs_write64(HOST_IA32_FRED_STKLVLS, kvm_host.fred_stklvls);
+
+		/* Linux doesn't support kernel shadow stacks, thus SSPs are 0s */
+		vmcs_write64(HOST_IA32_FRED_SSP1, 0);
+		vmcs_write64(HOST_IA32_FRED_SSP2, 0);
+		vmcs_write64(HOST_IA32_FRED_SSP3, 0);
+	}
 #else
 	vmcs_write16(HOST_DS_SELECTOR, __KERNEL_DS);  /* 22.2.4 */
 	vmcs_write16(HOST_ES_SELECTOR, __KERNEL_DS);  /* 22.2.4 */
@@ -4841,6 +4861,17 @@ static void init_vmcs(struct vcpu_vmx *vmx)
 	}
 
 	vmx_setup_uret_msrs(vmx);
+
+	if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+		vmcs_write64(GUEST_IA32_FRED_CONFIG, 0);
+		vmcs_write64(GUEST_IA32_FRED_RSP1, 0);
+		vmcs_write64(GUEST_IA32_FRED_RSP2, 0);
+		vmcs_write64(GUEST_IA32_FRED_RSP3, 0);
+		vmcs_write64(GUEST_IA32_FRED_STKLVLS, 0);
+		vmcs_write64(GUEST_IA32_FRED_SSP1, 0);
+		vmcs_write64(GUEST_IA32_FRED_SSP2, 0);
+		vmcs_write64(GUEST_IA32_FRED_SSP3, 0);
+	}
 }
 
 static void __vmx_vcpu_reset(struct kvm_vcpu *vcpu)
@@ -8717,6 +8748,11 @@ __init int vmx_hardware_setup(void)
 
 	kvm_caps.inapplicable_quirks &= ~KVM_X86_QUIRK_IGNORE_GUEST_PAT;
 
+	if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+		rdmsrl(MSR_IA32_FRED_CONFIG, kvm_host.fred_config);
+		rdmsrl(MSR_IA32_FRED_STKLVLS, kvm_host.fred_stklvls);
+	}
+
 	return r;
 }
 
arch/x86/kvm/x86.h +3 -0
index f3dc77f006f9..0c1fbf75442b 100644
@@ -52,6 +52,9 @@ struct kvm_host_values {
 	u64 xss;
 	u64 s_cet;
 	u64 arch_capabilities;
+
+	u64 fred_config;
+	u64 fred_stklvls;
 };
 
 void kvm_spurious_fault(void);
Signature · 1 line
2.51.0
Reviewed-by on v9 7/22
#21 lore raw

Re: [PATCH v9 07/22] KVM: VMX: Initialize VMCS FRED fields

Quoted 28 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Initialize host VMCS FRED fields with host FRED MSRs' value and guest VMCS FRED fields to 0.

FRED CPU state is managed in 9 new FRED MSRs:

       IA32_FRED_CONFIG,
       IA32_FRED_STKLVLS,
       IA32_FRED_RSP0,
       IA32_FRED_RSP1,
       IA32_FRED_RSP2,
       IA32_FRED_RSP3,
       IA32_FRED_SSP1,
       IA32_FRED_SSP2,
       IA32_FRED_SSP3,
as well as a few existing CPU registers and MSRs:
       CR4.FRED,
       IA32_STAR,
       IA32_KERNEL_GS_BASE,
       IA32_PL0_SSP (also known as IA32_FRED_SSP0).

CR4, IA32_KERNEL_GS_BASE and IA32_STAR are already well managed. Except IA32_FRED_RSP0 and IA32_FRED_SSP0, all other FRED CPU state MSRs have corresponding VMCS fields in both the host-state and guest-state areas. So KVM just needs to initialize them, and with proper VM entry/exit FRED controls, a FRED CPU will keep tracking host and guest FRED CPU state in VMCS automatically.

Quoted 4 lines

one nit below,

arch/x86/kvm/vmx/vmx.c __init int vmx_hardware_setup(void) · line 8753

Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +3 -0
@@ -8717,6 +8748,11 @@ __init int vmx_hardware_setup(void)
 
 	kvm_caps.inapplicable_quirks &= ~KVM_X86_QUIRK_IGNORE_GUEST_PAT;
 
+	if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+		rdmsrl(MSR_IA32_FRED_CONFIG, kvm_host.fred_config);
+		rdmsrl(MSR_IA32_FRED_STKLVLS, kvm_host.fred_stklvls);

s/rdmsrl/rdmsrq

Quoted 22 lines
arch/x86/kvm/vmx/vmx.c +2 -0
+	}
+
 	return r;
 }
 
arch/x86/kvm/x86.h +3 -0
index f3dc77f006f9..0c1fbf75442b 100644
@@ -52,6 +52,9 @@ struct kvm_host_values {
 	u64 xss;
 	u64 s_cet;
 	u64 arch_capabilities;
+
+	u64 fred_config;
+	u64 fred_stklvls;
 };
 
 void kvm_spurious_fault(void);
Signature · 1 line
2.51.0
Reviewed-by on v9 7/22
#22 lore raw

Re: [PATCH v9 07/22] KVM: VMX: Initialize VMCS FRED fields

In reply to Xin Li (Intel)
Quoted 21 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Initialize host VMCS FRED fields with host FRED MSRs' value and guest VMCS FRED fields to 0.

FRED CPU state is managed in 9 new FRED MSRs:

        IA32_FRED_CONFIG,
        IA32_FRED_STKLVLS,
        IA32_FRED_RSP0,
        IA32_FRED_RSP1,
        IA32_FRED_RSP2,
        IA32_FRED_RSP3,
        IA32_FRED_SSP1,
        IA32_FRED_SSP2,
        IA32_FRED_SSP3,
as well as a few existing CPU registers and MSRs:
        CR4.FRED,
        IA32_STAR,
        IA32_KERNEL_GS_BASE,
        IA32_PL0_SSP (also known as IA32_FRED_SSP0).
Quoted 7 lines

CR4, IA32_KERNEL_GS_BASE and IA32_STAR are already well managed. Except IA32_FRED_RSP0 and IA32_FRED_SSP0, all other FRED CPU state MSRs have corresponding VMCS fields in both the host-state and guest-state areas. So KVM just needs to initialize them, and with proper VM entry/exit FRED controls, a FRED CPU will keep tracking host and guest FRED CPU state in VMCS automatically.

One nit below.

[...]

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu) · line 1464

Quoted 11 lines
arch/x86/kvm/vmx/vmx.c +3 -0
index fcfa99160018..c8b5359123bf 100644
@@ -1459,6 +1459,15 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu)
 				    (unsigned long)(cpu_entry_stack(cpu) + 1));
 		}
 
+		/* Per-CPU FRED MSRs */
+		if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+#ifdef CONFIG_X86_64

Nit:

Is this needed?

FRED is initialized by X86_64_F(), if CONFIG_X86_64 is not enabled, this path is not reachable.
There should be no compilation issue without #ifdef CONFIG_X86_64 / #endif.

There are several similar patterns in this patch, using #ifdef CONFIG_X86_64 / #endif or not seems not consistent. E.g. __vmx_vcpu_reset() and init_vmcs() doesn't check the config, but here does.

Quoted 71 lines
arch/x86/kvm/vmx/vmx.c +33 -0
+			vmcs_write64(HOST_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
+			vmcs_write64(HOST_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
+			vmcs_write64(HOST_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));
+#endif
+		}
+
 		vmx->loaded_vmcs->cpu = cpu;
 	}
 }
@@ -4330,6 +4339,17 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx)
 	 */
 	vmcs_write16(HOST_DS_SELECTOR, 0);
 	vmcs_write16(HOST_ES_SELECTOR, 0);
+
+	if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+		/* FRED CONFIG and STKLVLS are the same on all CPUs */
+		vmcs_write64(HOST_IA32_FRED_CONFIG, kvm_host.fred_config);
+		vmcs_write64(HOST_IA32_FRED_STKLVLS, kvm_host.fred_stklvls);
+
+		/* Linux doesn't support kernel shadow stacks, thus SSPs are 0s */
+		vmcs_write64(HOST_IA32_FRED_SSP1, 0);
+		vmcs_write64(HOST_IA32_FRED_SSP2, 0);
+		vmcs_write64(HOST_IA32_FRED_SSP3, 0);
+	}
 #else
 	vmcs_write16(HOST_DS_SELECTOR, __KERNEL_DS);  /* 22.2.4 */
 	vmcs_write16(HOST_ES_SELECTOR, __KERNEL_DS);  /* 22.2.4 */
@@ -4841,6 +4861,17 @@ static void init_vmcs(struct vcpu_vmx *vmx)
 	}
 
 	vmx_setup_uret_msrs(vmx);
+
+	if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+		vmcs_write64(GUEST_IA32_FRED_CONFIG, 0);
+		vmcs_write64(GUEST_IA32_FRED_RSP1, 0);
+		vmcs_write64(GUEST_IA32_FRED_RSP2, 0);
+		vmcs_write64(GUEST_IA32_FRED_RSP3, 0);
+		vmcs_write64(GUEST_IA32_FRED_STKLVLS, 0);
+		vmcs_write64(GUEST_IA32_FRED_SSP1, 0);
+		vmcs_write64(GUEST_IA32_FRED_SSP2, 0);
+		vmcs_write64(GUEST_IA32_FRED_SSP3, 0);
+	}
 }
 
 static void __vmx_vcpu_reset(struct kvm_vcpu *vcpu)
@@ -8717,6 +8748,11 @@ __init int vmx_hardware_setup(void)
 
 	kvm_caps.inapplicable_quirks &= ~KVM_X86_QUIRK_IGNORE_GUEST_PAT;
 
+	if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+		rdmsrl(MSR_IA32_FRED_CONFIG, kvm_host.fred_config);
+		rdmsrl(MSR_IA32_FRED_STKLVLS, kvm_host.fred_stklvls);
+	}
+
 	return r;
 }
 
arch/x86/kvm/x86.h +3 -0
index f3dc77f006f9..0c1fbf75442b 100644
@@ -52,6 +52,9 @@ struct kvm_host_values {
 	u64 xss;
 	u64 s_cet;
 	u64 arch_capabilities;
+
+	u64 fred_config;
+	u64 fred_stklvls;
 };
 
 void kvm_spurious_fault(void);
author maintainer
#23 lore raw

Re: [PATCH v9 07/22] KVM: VMX: Initialize VMCS FRED fields

Quoted 10 lines · Binbin Wu
Quoted 2 lines

+#ifdef CONFIG_X86_64

Nit:

Is this needed?

Quoted 7 lines

FRED is initialized by X86_64_F(), if CONFIG_X86_64 is not enabled, this path is not reachable.
There should be no compilation issue without #ifdef CONFIG_X86_64 / #endif.

There are several similar patterns in this patch, using #ifdef CONFIG_X86_64 / #endif or not seems not consistent. E.g. __vmx_vcpu_reset() and init_vmcs() doesn't check the config, but here does.

I tried removing all such #ifdef, and it turned out that I had to keep this per the last round of build checks.

Anyway, I will do another build check on x86_32.

author maintainer
#24 lore raw

Re: [PATCH v9 07/22] KVM: VMX: Initialize VMCS FRED fields

Quoted 23 lines · Xin Li
Quoted 11 lines · Binbin Wu
Quoted 2 lines

+#ifdef CONFIG_X86_64

Nit:

Is this needed?

Quoted 7 lines

FRED is initialized by X86_64_F(), if CONFIG_X86_64 is not enabled, this path is not reachable.
There should be no compilation issue without #ifdef CONFIG_X86_64 / #endif.

There are several similar patterns in this patch, using #ifdef CONFIG_X86_64 / #endif or not seems not consistent. E.g. __vmx_vcpu_reset() and init_vmcs() doesn't check the config, but here does.

Quoted 5 lines

I tried removing all such #ifdef, and it turned out that I had to keep this per the last round of build checks.

Anyway, I will do another build check on x86_32.

The trouble comes from __this_cpu_ist_top_va():

arch/x86/kvm/vmx/vmx.c: In function ‘vmx_vcpu_load_vmcs’:
arch/x86/kvm/vmx/vmx.c:1608:59: error: implicit declaration of function ‘__this_cpu_ist_top_va’ [-Werror=implicit-function-declaration]

 1608 |                         vmcs_write64(HOST_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
      |                                                           ^~~~~~~~~~~~~~~~~~~~~

arch/x86/kvm/vmx/vmx.c:1608:81: error: ‘ESTACK_DB’ undeclared (first use in this function)
1608 | vmcs_write64(HOST_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
| ^~~~~~~~~
arch/x86/kvm/vmx/vmx.c:1608:81: note: each undeclared identifier is reported only once for each function it appears in
CC [M] crypto/md4.o
CC lib/crypto/sha512.o
arch/x86/kvm/vmx/vmx.c:1609:81: error: ‘ESTACK_NMI’ undeclared (first use in this function)
1609 | vmcs_write64(HOST_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
| ^~~~~~~~~~
arch/x86/kvm/vmx/vmx.c:1610:81: error: ‘ESTACK_DF’ undeclared (first use in this function)
1610 | vmcs_write64(HOST_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));
| ^~~~~~~~~

#25 lore raw

Re: [PATCH v9 07/22] KVM: VMX: Initialize VMCS FRED fields

Quoted 23 lines · Xin Li
Quoted 23 lines · Xin Li
Quoted 18 lines · Binbin Wu
Quoted 2 lines

+#ifdef CONFIG_X86_64

Nit:

Is this needed?

FRED is initialized by X86_64_F(), if CONFIG_X86_64 is not enabled, this path is not reachable.
There should be no compilation issue without #ifdef CONFIG_X86_64 / #endif.

There are several similar patterns in this patch, using #ifdef CONFIG_X86_64 / #endif or not seems not consistent. E.g. __vmx_vcpu_reset() and init_vmcs() doesn't check the config, but here does.

Quoted 8 lines
Quoted 5 lines

I tried removing all such #ifdef, and it turned out that I had to keep this per the last round of build checks.

Anyway, I will do another build check on x86_32.

The trouble comes from __this_cpu_ist_top_va():

Oh, right!
Sorry for the noise.

Quoted 18 lines

arch/x86/kvm/vmx/vmx.c: In function ‘vmx_vcpu_load_vmcs’:
arch/x86/kvm/vmx/vmx.c:1608:59: error: implicit declaration of function ‘__this_cpu_ist_top_va’ [-Werror=implicit-function-declaration]

 1608 |                         vmcs_write64(HOST_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
      |                                                           ^~~~~~~~~~~~~~~~~~~~~

arch/x86/kvm/vmx/vmx.c:1608:81: error: ‘ESTACK_DB’ undeclared (first use in this function)
1608 | vmcs_write64(HOST_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
| ^~~~~~~~~
arch/x86/kvm/vmx/vmx.c:1608:81: note: each undeclared identifier is reported only once for each function it appears in
CC [M] crypto/md4.o
CC lib/crypto/sha512.o
arch/x86/kvm/vmx/vmx.c:1609:81: error: ‘ESTACK_NMI’ undeclared (first use in this function)
1609 | vmcs_write64(HOST_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
| ^~~~~~~~~~
arch/x86/kvm/vmx/vmx.c:1610:81: error: ‘ESTACK_DF’ undeclared (first use in this function)
1610 | vmcs_write64(HOST_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));
| ^~~~~~~~~

author maintainer
#26 lore raw

Re: [PATCH v9 07/22] KVM: VMX: Initialize VMCS FRED fields

Quoted 5 lines
Quoted 2 lines

The trouble comes from __this_cpu_ist_top_va():

Oh, right!
Sorry for the noise.

It’s absolutely NOT noise, because we don’t like #ifdef in C file.

With the following additional patch, we can remove this #ifdef CONFIG_X86_64:

arch/x86/include/asm/cpu_entry_area.h +14 -7
index 509e52fc3a0f..c35ef2cb9b8a 100644
@@ -8,14 +8,6 @@
 #include <asm/intel_ds.h>
 #include <asm/pgtable_areas.h>
  -#ifdef CONFIG_X86_64
-
-#ifdef CONFIG_AMD_MEM_ENCRYPT
-#define VC_EXCEPTION_STKSZ     EXCEPTION_STKSZ
-#else
-#define VC_EXCEPTION_STKSZ     0
-#endif
-
 /*
  * The exception stack ordering in [cea_]exception_stacks
  */
@@ -26,9 +18,19 @@ enum exception_stack_ordering {
        ESTACK_MCE,
        ESTACK_VC,
        ESTACK_VC2,
+#ifdef CONFIG_X86_64
        N_EXCEPTION_STACKS
+#endif
 };
  +#ifdef CONFIG_X86_64
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+#define VC_EXCEPTION_STKSZ     EXCEPTION_STKSZ
+#else
+#define VC_EXCEPTION_STKSZ     0
+#endif
+
 /* Macro to enforce the same ordering and stack sizes */
 #define ESTACKS_MEMBERS(guardsize, optional_stack_size)                \
        char    ESTACK_DF_stack_guard[guardsize];               \
@@ -75,6 +77,11 @@ struct doublefault_stack {
        unsigned long stack[(PAGE_SIZE - sizeof(struct x86_hw_tss)) / sizeof(unsigned long)];
        struct x86_hw_tss tss;
 } __aligned(PAGE_SIZE);
+
+static inline unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack)
+{
+       return 0;
+}
 #endif
   /*
 

Another simpler way to define __this_cpu_ist_top_va() in a VMX header, but I don’t think it’s the right way.

#27 lore raw

Re: [PATCH v9 07/22] KVM: VMX: Initialize VMCS FRED fields

In reply to Binbin Wu
Quoted 10 lines · Binbin Wu

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu) · line 1462

Quoted 9 lines · Xin Li (Intel)
arch/x86/kvm/vmx/vmx.c +1 -0
index fcfa99160018..c8b5359123bf 100644
@@ -1459,6 +1459,15 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu)
 				    (unsigned long)(cpu_entry_stack(cpu) + 1));
 		}
 
+		/* Per-CPU FRED MSRs */

Meh, this is pretty self-explanatory code.

Quoted 11 lines

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu) · line 1464

Quoted 2 lines
arch/x86/kvm/vmx/vmx.c +2 -0
+		if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+#ifdef CONFIG_X86_64

Nit:

Is this needed?

FRED is initialized by X86_64_F(), if CONFIG_X86_64 is not enabled, this path is not reachable.
There should be no compilation issue without #ifdef CONFIG_X86_64 / #endif.

Quoted 7 lines

There are several similar patterns in this patch, using #ifdef CONFIG_X86_64 / #endif or not seems not consistent. E.g. __vmx_vcpu_reset() and init_vmcs() doesn't check the config, but here does.

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu) · line 1467

Quoted 3 lines
arch/x86/kvm/vmx/vmx.c +3 -0
+			vmcs_write64(HOST_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
+			vmcs_write64(HOST_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
+			vmcs_write64(HOST_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));

IMO, this is flawed for other reasons. KVM shouldn't be relying on kernel implementation details with respect to what FRED stack handles what event.

The simplest approach would be to read the actual MSR. _If_ using a per-CPU read provides meaningful performance benefits over RDMSR (or RDMSR w/ immediate? I don't see an API for that...), then have the kernel provide a dedicated accessor.

Then the accessor can be a non-inlined functions, and this code can be e.g.:

	if (IS_ENABLED(CONFIG_X86_64) && kvm_cpu_cap_has(X86_FEATURE_FRED)) {
		vmcs_write64(HOST_IA32_FRED_RSP1, fred_rsp(MSR_IA32_FRED_RSP1));
		vmcs_write64(HOST_IA32_FRED_RSP2, fred_rsp(MSR_IA32_FRED_RSP2));
		vmcs_write64(HOST_IA32_FRED_RSP3, fred_rsp(MSR_IA32_FRED_RSP2));
	}

where fred_rsp() is _declared_ unconditionally, but implemented only for 64-bit. That way the compiler will be happy, and the actual usage will be dropped before linking via dead-code elimination.

Actually, we can probably do one better?

	if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) {
author maintainer
#28 lore raw

Re: [PATCH v9 07/22] KVM: VMX: Initialize VMCS FRED fields

Quoted 14 lines · Sean Christopherson
Quoted 9 lines · Binbin Wu
Quoted 8 lines · Xin Li (Intel)
arch/x86/kvm/vmx/vmx.c +0 -0
index fcfa99160018..c8b5359123bf 100644
@@ -1459,6 +1459,15 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu)
   (unsigned long)(cpu_entry_stack(cpu) + 1));
 }
 
Quoted 3 lines
Quoted 1 line

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu) · line 1462

Quoted 1 line
arch/x86/kvm/vmx/vmx.c +1 -0
+ /* Per-CPU FRED MSRs */

Meh, this is pretty self-explanatory code.

I want to remind that this is “Per-CPU”.

On the other side, FRED_CONFIG and FRED_STKLVLS are typically the same on all CPUs, and they don’t need to be updated during vCPU migration.

But anyway vCPU migration only cares per-CPU MSRs, so this is redundant.

It often bothers me whether to explain the code a bit more or not, with a short brief comment or a lengthy one.

Quoted 20 lines
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu) · line 1464

Quoted 3 lines
arch/x86/kvm/vmx/vmx.c +2 -0
+ if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+#ifdef CONFIG_X86_64

Nit:

Is this needed?

Quoted 11 lines

FRED is initialized by X86_64_F(), if CONFIG_X86_64 is not enabled, this path is not reachable.
There should be no compilation issue without #ifdef CONFIG_X86_64 / #endif.

There are several similar patterns in this patch, using #ifdef CONFIG_X86_64 / #endif or not seems not consistent. E.g. __vmx_vcpu_reset() and init_vmcs() doesn't check the config, but here does.

Quoted 3 lines
arch/x86/kvm/vmx/vmx.c +3 -0
+ vmcs_write64(HOST_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
+ vmcs_write64(HOST_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
+ vmcs_write64(HOST_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));
Quoted 6 lines

IMO, this is flawed for other reasons. KVM shouldn't be relying on kernel implementation details with respect to what FRED stack handles what event.

The simplest approach would be to read the actual MSR. _If_ using a per-CPU read provides meaningful performance benefits over RDMSR (or RDMSR w/ immediate? I don't see an API for that...), then have the kernel provide a dedicated accessor.

I think you asked for it:

https://lore.kernel.org/kvm/[email protected]/

I assume fetching through per-CPU cache is fast, but I might have misunderstood your suggestion.

Quoted 9 lines

Then the accessor can be a non-inlined functions, and this code can be e.g.:

if (IS_ENABLED(CONFIG_X86_64) && kvm_cpu_cap_has(X86_FEATURE_FRED)) { vmcs_write64(HOST_IA32_FRED_RSP1, fred_rsp(MSR_IA32_FRED_RSP1)); vmcs_write64(HOST_IA32_FRED_RSP2, fred_rsp(MSR_IA32_FRED_RSP2)); vmcs_write64(HOST_IA32_FRED_RSP3, fred_rsp(MSR_IA32_FRED_RSP2)); }

Quoted 3 lines

where fred_rsp() is _declared_ unconditionally, but implemented only for 64-bit. That way the compiler will be happy, and the actual usage will be dropped before linking via dead-code elimination.

If KVM can’t rely on kernel side implementation details, fred_rsp() has to read directly from the corresponding MSRs, right?

Quoted 4 lines

Actually, we can probably do one better?

if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) {

I think KVM now forces X86_FEATURE_FRED=y, no?

#29 lore raw

Re: [PATCH v9 07/22] KVM: VMX: Initialize VMCS FRED fields

Quoted 20 lines · Xin Li
Quoted 14 lines · Sean Christopherson
Quoted 9 lines · Binbin Wu
Quoted 8 lines · Xin Li (Intel)
arch/x86/kvm/vmx/vmx.c +0 -0
index fcfa99160018..c8b5359123bf 100644
@@ -1459,6 +1459,15 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu)
   (unsigned long)(cpu_entry_stack(cpu) + 1));
 }
 
Quoted 3 lines
Quoted 1 line

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu) · line 1462

Quoted 1 line
arch/x86/kvm/vmx/vmx.c +1 -0
+ /* Per-CPU FRED MSRs */

Meh, this is pretty self-explanatory code.

I want to remind that this is “Per-CPU”.

Quoted 7 lines

On the other side, FRED_CONFIG and FRED_STKLVLS are typically the same on all CPUs, and they don’t need to be updated during vCPU migration.

But anyway vCPU migration only cares per-CPU MSRs, so this is redundant.

It often bothers me whether to explain the code a bit more or not, with a short brief comment or a lengthy one.

Oh, I'm all for comments, But I generally dislike terse comments like the above, as they're only useful for readers that *already* know many of the gory details, and for everyone else, it's largely just noise.

E.g. in this case, what the subtlety of what you were trying to convey with "Per-CPU" was completely lost on me. It would have been a wee bit more helpful in earlier versions that used RDMSR, but even then it required the reader to make large leaps of intuition to understand the full context.

And the terse comment is also "wrong" in the sense that it lies by omission, because this isn't *all* of the per-CPU MSRs.

E.g.

/*
* Update the FRED RSP MSRs values that are restored on VM-Exit, as
* Linux uses dedicated per-CPU stacks for things like #DBs and NMIs.
* Note, RSP0 is _only_ used to load RSP on user=>kernel transitions,
* i.e. doesn't need to be loaded on VM-Exit and so doesn't have a VMCS
* field. Note #2, the SSP MSRs will likely be per-CPU as well, but
* Linux doesn't yet support supervisor shadow stacks.
*/

Quoted 22 lines
Quoted 22 lines
Quoted 7 lines

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu) · line 1464

Quoted 2 lines
arch/x86/kvm/vmx/vmx.c +2 -0
+ if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+#ifdef CONFIG_X86_64

Nit:

Is this needed?

Quoted 11 lines

FRED is initialized by X86_64_F(), if CONFIG_X86_64 is not enabled, this path is not reachable.
There should be no compilation issue without #ifdef CONFIG_X86_64 / #endif.

There are several similar patterns in this patch, using #ifdef CONFIG_X86_64 / #endif or not seems not consistent. E.g. __vmx_vcpu_reset() and init_vmcs() doesn't check the config, but here does.

Quoted 3 lines
arch/x86/kvm/vmx/vmx.c +3 -0
+ vmcs_write64(HOST_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
+ vmcs_write64(HOST_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
+ vmcs_write64(HOST_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));

IMO, this is flawed for other reasons. KVM shouldn't be relying on kernel implementation details with respect to what FRED stack handles what event.

Quoted 7 lines
Quoted 3 lines

The simplest approach would be to read the actual MSR. _If_ using a per-CPU read provides meaningful performance benefits over RDMSR (or RDMSR w/ immediate? I don't see an API for that...), then have the kernel provide a dedicated accessor.

I think you asked for it:

https://lore.kernel.org/kvm/[email protected]/

Gah, so I did. FWIW, comments like that aren't intended to be "you must do it this way", i.e. it's ok to push back (if there's justification to do so), but I can totally see how it came across that way.

Quoted 2 lines

I assume fetching through per-CPU cache is fast, but I might have misunderstood your suggestion.

Oh, per-CPU cache is likely faster than RDMSR (though it would be nice to verify). The part I specifically don't like is referencing DB, NMI, and DF stacks.

Quoted 6 lines
Quoted 6 lines

Then the accessor can be a non-inlined functions, and this code can be e.g.:

if (IS_ENABLED(CONFIG_X86_64) && kvm_cpu_cap_has(X86_FEATURE_FRED)) { vmcs_write64(HOST_IA32_FRED_RSP1, fred_rsp(MSR_IA32_FRED_RSP1)); vmcs_write64(HOST_IA32_FRED_RSP2, fred_rsp(MSR_IA32_FRED_RSP2)); vmcs_write64(HOST_IA32_FRED_RSP3, fred_rsp(MSR_IA32_FRED_RSP2));

                                                                ^
                                                                |
                                                                3 (guess who failed at copy+paste)
Quoted 8 lines
Quoted 5 lines

}

where fred_rsp() is _declared_ unconditionally, but implemented only for 64-bit. That way the compiler will be happy, and the actual usage will be dropped before linking via dead-code elimination.

If KVM can’t rely on kernel side implementation details, fred_rsp() has to read directly from the corresponding MSRs, right?

No. If the kernel provides an API specifically for getting RSP values, then KVM isn't rely on implementation details, KVM is relying on an (exported?) API, and it's the _kernel's_ responsibility to ensure that API is updated as needed.

Yeah, yeah, KVM is also part of the kernel, but it's a _lot_ easier to forget to update code when the implementation details change when some of the usage is "far" away.

Quoted 5 lines
Quoted 3 lines

Actually, we can probably do one better?

if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) {

I think KVM now forces X86_FEATURE_FRED=y, no?

32-bit doesn't :-D

But even for 64-bit, X86_FEATURE_FRED should be cleared when FRED isn't supported by hardware, in which case cpu_feature_enabled() still patches out the text.

author maintainer
#30 lore raw

Re: [PATCH v9 07/22] KVM: VMX: Initialize VMCS FRED fields

Quoted 29 lines
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

+ /* Per-CPU FRED MSRs */

Meh, this is pretty self-explanatory code.

I want to remind that this is “Per-CPU”.

Quoted 7 lines

On the other side, FRED_CONFIG and FRED_STKLVLS are typically the same on all CPUs, and they don’t need to be updated during vCPU migration.

But anyway vCPU migration only cares per-CPU MSRs, so this is redundant.

It often bothers me whether to explain the code a bit more or not, with a short brief comment or a lengthy one.

Oh, I'm all for comments, But I generally dislike terse comments like the above, as they're only useful for readers that *already* know many of the gory details, and for everyone else, it's largely just noise.

E.g. in this case, what the subtlety of what you were trying to convey with "Per-CPU" was completely lost on me. It would have been a wee bit more helpful in earlier versions that used RDMSR, but even then it required the reader to make large leaps of intuition to understand the full context.

And the terse comment is also "wrong" in the sense that it lies by omission, because this isn't *all* of the per-CPU MSRs.

E.g.

Quoted 8 lines

/*
* Update the FRED RSP MSRs values that are restored on VM-Exit, as
* Linux uses dedicated per-CPU stacks for things like #DBs and NMIs.
* Note, RSP0 is _only_ used to load RSP on user=>kernel transitions,
* i.e. doesn't need to be loaded on VM-Exit and so doesn't have a VMCS
* field. Note #2, the SSP MSRs will likely be per-CPU as well, but
* Linux doesn't yet support supervisor shadow stacks.
*/

You’re right, terse comment could be bad.

Quoted 59 lines
Quoted 23 lines
Quoted 23 lines
Quoted 8 lines
Quoted 3 lines

+ if (kvm_cpu_cap_has(X86_FEATURE_FRED)) {
+#ifdef CONFIG_X86_64

Nit:

Is this needed?

Quoted 11 lines

FRED is initialized by X86_64_F(), if CONFIG_X86_64 is not enabled, this path is not reachable.
There should be no compilation issue without #ifdef CONFIG_X86_64 / #endif.

There are several similar patterns in this patch, using #ifdef CONFIG_X86_64 / #endif or not seems not consistent. E.g. __vmx_vcpu_reset() and init_vmcs() doesn't check the config, but here does.

Quoted 3 lines

+ vmcs_write64(HOST_IA32_FRED_RSP1, __this_cpu_ist_top_va(ESTACK_DB));
+ vmcs_write64(HOST_IA32_FRED_RSP2, __this_cpu_ist_top_va(ESTACK_NMI));
+ vmcs_write64(HOST_IA32_FRED_RSP3, __this_cpu_ist_top_va(ESTACK_DF));

IMO, this is flawed for other reasons. KVM shouldn't be relying on kernel implementation details with respect to what FRED stack handles what event.

Quoted 7 lines
Quoted 3 lines

The simplest approach would be to read the actual MSR. _If_ using a per-CPU read provides meaningful performance benefits over RDMSR (or RDMSR w/ immediate? I don't see an API for that...), then have the kernel provide a dedicated accessor.

I think you asked for it:

https://lore.kernel.org/kvm/[email protected]/

Gah, so I did. FWIW, comments like that aren't intended to be "you must do it this way", i.e. it's ok to push back (if there's justification to do so), but I can totally see how it came across that way.

Quoted 2 lines

I assume fetching through per-CPU cache is fast, but I might have misunderstood your suggestion.

Oh, per-CPU cache is likely faster than RDMSR (though it would be nice to verify). The part I specifically don't like is referencing DB, NMI, and DF stacks.

Quoted 6 lines
Quoted 6 lines

Then the accessor can be a non-inlined functions, and this code can be e.g.:

if (IS_ENABLED(CONFIG_X86_64) && kvm_cpu_cap_has(X86_FEATURE_FRED)) { vmcs_write64(HOST_IA32_FRED_RSP1, fred_rsp(MSR_IA32_FRED_RSP1)); vmcs_write64(HOST_IA32_FRED_RSP2, fred_rsp(MSR_IA32_FRED_RSP2)); vmcs_write64(HOST_IA32_FRED_RSP3, fred_rsp(MSR_IA32_FRED_RSP2));

                                                               ^
                                                               |
                                                               3 (guess who failed at copy+paste)
Quoted 8 lines
Quoted 5 lines

}

where fred_rsp() is _declared_ unconditionally, but implemented only for 64-bit. That way the compiler will be happy, and the actual usage will be dropped before linking via dead-code elimination.

If KVM can’t rely on kernel side implementation details, fred_rsp() has to read directly from the corresponding MSRs, right?

Quoted 7 lines

No. If the kernel provides an API specifically for getting RSP values, then KVM isn't rely on implementation details, KVM is relying on an (exported?) API, and it's the _kernel's_ responsibility to ensure that API is updated as needed.

Yeah, yeah, KVM is also part of the kernel, but it's a _lot_ easier to forget to update code when the implementation details change when some of the usage is "far" away.

Okay, I think it’s clearer to me what bothers you.

A self-explanatory API is the right thing, and should be a standard practice.

Quoted 11 lines
Quoted 6 lines
Quoted 4 lines

Actually, we can probably do one better?

if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) {

I think KVM now forces X86_FEATURE_FRED=y, no?

32-bit doesn't :-D

But even for 64-bit, X86_FEATURE_FRED should be cleared when FRED isn't supported by hardware, in which case cpu_feature_enabled() still patches out the text.

Right, it can be patched out thus has “zero” impact on existing hardwares.

(Just that the FRED IRQ handling code is not as it’s used by KVM.)

author maintainer
#31 lore raw

PATCH v9 8/22 KVM: VMX: Set FRED MSR intercepts

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

On a userspace MSR filter change, set FRED MSR intercepts.

The eight FRED MSRs, MSR_IA32_FRED_RSP[123], MSR_IA32_FRED_STKLVLS, MSR_IA32_FRED_SSP[123] and MSR_IA32_FRED_CONFIG, are all safe to passthrough, because each has a corresponding host and guest field in VMCS.

Both MSR_IA32_FRED_RSP0 and MSR_IA32_FRED_SSP0 (aka MSR_IA32_PL0_SSP) are dedicated for userspace event delivery, IOW they are NOT used in any kernel event delivery and the execution of ERETS. Thus KVM can run safely with guest values in the two MSRs. As a result, save and restore of their guest values are deferred until vCPU context switch, Host MSR_IA32_FRED_RSP0 is restored upon returning to userspace, and Host MSR_IA32_PL0_SSP is managed with XRSTORS/XSAVES.

Note, FRED SSP MSRs, including MSR_IA32_PL0_SSP, are available on any processor that enumerates FRED. On processors that support FRED but not CET, FRED transitions do not use these MSRs, but they remain accessible via MSR instructions such as RDMSR and WRMSR.

Intercept MSR_IA32_PL0_SSP when CET shadow stack is not supported, regardless of FRED support. This ensures the guest value remains fully virtual and does not modify the hardware FRED SSP0 MSR.

This behavior is consistent with the current setup in vmx_recalc_msr_intercepts(), so no change is needed to the interception logic for MSR_IA32_PL0_SSP.

---

Changes in v7:
* Rewrite the changelog and comment, majorly for MSR_IA32_PL0_SSP.

Changes in v5:
* Skip execution of vmx_set_intercept_for_fred_msr() if FRED is not available or enabled (Sean).
* Use 'intercept' as the variable name to indicate whether MSR interception should be enabled (Sean).
* Add TB from Xuelian Guo.

---
 arch/x86/kvm/vmx/vmx.c | 47 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
arch/x86/kvm/vmx/vmx.c +47 -0
index c8b5359123bf..ef9765779884 100644
@@ -4146,6 +4146,51 @@ void pt_update_intercept_for_msr(struct kvm_vcpu *vcpu)
 	}
 }
 
+static void vmx_set_intercept_for_fred_msr(struct kvm_vcpu *vcpu)
+{
+	bool intercept = !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED);
+
+	if (!kvm_cpu_cap_has(X86_FEATURE_FRED))
+		return;
+
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP1, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP2, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP3, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_STKLVLS, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP1, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP2, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP3, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_CONFIG, MSR_TYPE_RW, intercept);
+
+	/*
+	 * MSR_IA32_FRED_RSP0 and MSR_IA32_PL0_SSP (aka MSR_IA32_FRED_SSP0) are
+	 * designed for event delivery while executing in userspace.  Since KVM
+	 * operates entirely in kernel mode (CPL is always 0 after any VM exit),
+	 * it can safely retain and operate with guest-defined values for these
+	 * MSRs.
+	 *
+	 * As a result, interception of MSR_IA32_FRED_RSP0 and MSR_IA32_PL0_SSP
+	 * is unnecessary.
+	 *
+	 * Note: Saving and restoring MSR_IA32_PL0_SSP is part of CET supervisor
+	 * context management.  However, FRED SSP MSRs, including MSR_IA32_PL0_SSP,
+	 * are available on any processor that enumerates FRED.
+	 *
+	 * On processors that support FRED but not CET, FRED transitions do not
+	 * use these MSRs, but they remain accessible via MSR instructions such
+	 * as RDMSR and WRMSR.
+	 *
+	 * Intercept MSR_IA32_PL0_SSP when CET shadow stack is not supported,
+	 * regardless of FRED support.  This ensures the guest value remains
+	 * fully virtual and does not modify the hardware FRED SSP0 MSR.
+	 *
+	 * This behavior is consistent with the current setup in
+	 * vmx_recalc_msr_intercepts(), so no change is needed to the interception
+	 * logic for MSR_IA32_PL0_SSP.
+	 */
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP0, MSR_TYPE_RW, intercept);
+}
+
 static void vmx_recalc_msr_intercepts(struct kvm_vcpu *vcpu)
 {
 	bool intercept;
@@ -4212,6 +4257,8 @@ static void vmx_recalc_msr_intercepts(struct kvm_vcpu *vcpu)
 		vmx_set_intercept_for_msr(vcpu, MSR_IA32_S_CET, MSR_TYPE_RW, intercept);
 	}
 
+	vmx_set_intercept_for_fred_msr(vcpu);
+
 	/*
 	 * x2APIC and LBR MSR intercepts are modified on-demand and cannot be
 	 * filtered by userspace.
Signature · 1 line
2.51.0
#32 lore raw

Re: [PATCH v9 08/22] KVM: VMX: Set FRED MSR intercepts

Quoted 8 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

On a userspace MSR filter change, set FRED MSR intercepts.

The eight FRED MSRs, MSR_IA32_FRED_RSP[123], MSR_IA32_FRED_STKLVLS, MSR_IA32_FRED_SSP[123] and MSR_IA32_FRED_CONFIG, are all safe to passthrough, because each has a corresponding host and guest field in VMCS.

Sean prefers to pass through MSRs only when there is a reason to do that rather than just because it is free. My thinking is that RSPs and SSPs are per-task and are context-switched frequently, so we need to pass through them. But I am not sure if there is a reason for STKLVLS and CONFIG.

[*] https://lore.kernel.org/all/[email protected]/

Quoted 64 lines

Both MSR_IA32_FRED_RSP0 and MSR_IA32_FRED_SSP0 (aka MSR_IA32_PL0_SSP) are dedicated for userspace event delivery, IOW they are NOT used in any kernel event delivery and the execution of ERETS. Thus KVM can run safely with guest values in the two MSRs. As a result, save and restore of their guest values are deferred until vCPU context switch, Host MSR_IA32_FRED_RSP0 is restored upon returning to userspace, and Host MSR_IA32_PL0_SSP is managed with XRSTORS/XSAVES.

Note, FRED SSP MSRs, including MSR_IA32_PL0_SSP, are available on any processor that enumerates FRED. On processors that support FRED but not CET, FRED transitions do not use these MSRs, but they remain accessible via MSR instructions such as RDMSR and WRMSR.

Intercept MSR_IA32_PL0_SSP when CET shadow stack is not supported, regardless of FRED support. This ensures the guest value remains fully virtual and does not modify the hardware FRED SSP0 MSR.

This behavior is consistent with the current setup in vmx_recalc_msr_intercepts(), so no change is needed to the interception logic for MSR_IA32_PL0_SSP.

---

Changes in v7:
* Rewrite the changelog and comment, majorly for MSR_IA32_PL0_SSP.

Changes in v5:
* Skip execution of vmx_set_intercept_for_fred_msr() if FRED is not available or enabled (Sean).
* Use 'intercept' as the variable name to indicate whether MSR interception should be enabled (Sean).
* Add TB from Xuelian Guo.
---
arch/x86/kvm/vmx/vmx.c | 47 ++++++++++++++++++++++++++++++++++++++++++

1 file changed, 47 insertions(+)
arch/x86/kvm/vmx/vmx.c +15 -0
index c8b5359123bf..ef9765779884 100644
@@ -4146,6 +4146,51 @@ void pt_update_intercept_for_msr(struct kvm_vcpu *vcpu)
 	}
 }
 
+static void vmx_set_intercept_for_fred_msr(struct kvm_vcpu *vcpu)
+{
+	bool intercept = !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED);
+
+	if (!kvm_cpu_cap_has(X86_FEATURE_FRED))
+		return;
+
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP1, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP2, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP3, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_STKLVLS, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP1, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP2, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP3, MSR_TYPE_RW, intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_CONFIG, MSR_TYPE_RW, intercept);

arch/x86/kvm/vmx/vmx.c void pt_update_intercept_for_msr(struct kvm_vcpu *vcpu) · line 4173

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+
+	/*
+	 * MSR_IA32_FRED_RSP0 and MSR_IA32_PL0_SSP (aka MSR_IA32_FRED_SSP0) are
+	 * designed for event delivery while executing in userspace.  Since KVM
+	 * operates entirely in kernel mode (CPL is always 0 after any VM exit),
+	 * it can safely retain and operate with guest-defined values for these
+	 * MSRs.
+	 *
+	 * As a result, interception of MSR_IA32_FRED_RSP0 and MSR_IA32_PL0_SSP
+	 * is unnecessary.

I think it would be slightly better to document why MSRs need to be passed through rather than just why it is safe to pass through.

arch/x86/kvm/vmx/vmx.c void pt_update_intercept_for_msr(struct kvm_vcpu *vcpu) · line 4185

Quoted 12 lines
arch/x86/kvm/vmx/vmx.c +12 -0
+	 *
+	 * Note: Saving and restoring MSR_IA32_PL0_SSP is part of CET supervisor
+	 * context management.  However, FRED SSP MSRs, including MSR_IA32_PL0_SSP,
+	 * are available on any processor that enumerates FRED.
+	 *
+	 * On processors that support FRED but not CET, FRED transitions do not
+	 * use these MSRs, but they remain accessible via MSR instructions such
+	 * as RDMSR and WRMSR.
+	 *
+	 * Intercept MSR_IA32_PL0_SSP when CET shadow stack is not supported,
+	 * regardless of FRED support.  This ensures the guest value remains
+	 * fully virtual and does not modify the hardware FRED SSP0 MSR.

Modifying the hardware MSR itself isn't a problem. The problem is that the MSR isn't supposed to be accessed frequently in the guest if CET isn't supported and will never be accessed via XSAVES. So, there is no good reason to pass through it. And passing through the MSR means KVM needs to context-switch it along with vcpu load/put, i.e., more code and complexity.

Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +8 -0
+	 *
+	 * This behavior is consistent with the current setup in
+	 * vmx_recalc_msr_intercepts(), so no change is needed to the interception
+	 * logic for MSR_IA32_PL0_SSP.
+	 */
+	vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP0, MSR_TYPE_RW, intercept);
+}
+
#33 lore raw

Re: [PATCH v9 08/22] KVM: VMX: Set FRED MSR intercepts

Quoted 10 lines · Chao Gao
Quoted 8 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

On a userspace MSR filter change, set FRED MSR intercepts.

The eight FRED MSRs, MSR_IA32_FRED_RSP[123], MSR_IA32_FRED_STKLVLS, MSR_IA32_FRED_SSP[123] and MSR_IA32_FRED_CONFIG, are all safe to passthrough, because each has a corresponding host and guest field in VMCS.

Quoted 4 lines

Sean prefers to pass through MSRs only when there is a reason to do that rather than just because it is free. My thinking is that RSPs and SSPs are per-task and are context-switched frequently, so we need to pass through them. But I am not sure if there is a reason for STKLVLS and CONFIG.

There are VMCS fields, at which point intercepting and emulating is probably more work than just letting the guest access directly. :-/

Ah, and there needs to be VMCS fields because presumably everything needs to be switch atomically, e.g. an NMI that arrives shortly after VM-Exit presumbably consumes STKLVLS and CONFIG.

author maintainer
#34 lore raw

Re: [PATCH v9 08/22] KVM: VMX: Set FRED MSR intercepts

Quoted 20 lines · Sean Christopherson
Quoted 10 lines · Chao Gao
Quoted 8 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

On a userspace MSR filter change, set FRED MSR intercepts.

The eight FRED MSRs, MSR_IA32_FRED_RSP[123], MSR_IA32_FRED_STKLVLS, MSR_IA32_FRED_SSP[123] and MSR_IA32_FRED_CONFIG, are all safe to passthrough, because each has a corresponding host and guest field in VMCS.

Quoted 4 lines

Sean prefers to pass through MSRs only when there is a reason to do that rather than just because it is free. My thinking is that RSPs and SSPs are per-task and are context-switched frequently, so we need to pass through them. But I am not sure if there is a reason for STKLVLS and CONFIG.

Quoted 6 lines

There are VMCS fields, at which point intercepting and emulating is probably more work than just letting the guest access directly. :-/

Ah, and there needs to be VMCS fields because presumably everything needs to be switch atomically, e.g. an NMI that arrives shortly after VM-Exit presumbably consumes STKLVLS and CONFIG.

Yes, first it is a correctness issue and I assumed it’s obvious (like we need a valid IDT/GDT immediately after VM exit).

Anyway, Peter had given a much better write-up and I switched to it.

#35 lore raw

Re: [PATCH v9 08/22] KVM: VMX: Set FRED MSR intercepts

In reply to Sean Christopherson
Quoted 11 lines · Sean Christopherson
Quoted 10 lines · Chao Gao
Quoted 8 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

On a userspace MSR filter change, set FRED MSR intercepts.

The eight FRED MSRs, MSR_IA32_FRED_RSP[123], MSR_IA32_FRED_STKLVLS, MSR_IA32_FRED_SSP[123] and MSR_IA32_FRED_CONFIG, are all safe to passthrough, because each has a corresponding host and guest field in VMCS.

Quoted 7 lines
Quoted 4 lines

Sean prefers to pass through MSRs only when there is a reason to do that rather than just because it is free. My thinking is that RSPs and SSPs are per-task and are context-switched frequently, so we need to pass through them. But I am not sure if there is a reason for STKLVLS and CONFIG.

There are VMCS fields, at which point intercepting and emulating is probably more work than just letting the guest access directly. :-/

Just drop the MSR intercepting code and everything should work, right? KVM needs to handle userspace writes anyway. so, there is no "more work" to me.

Quoted 4 lines

Ah, and there needs to be VMCS fields because presumably everything needs to be switch atomically, e.g. an NMI that arrives shortly after VM-Exit presumbably consumes STKLVLS and CONFIG.

I assume CET's MSR_IA32_INT_SSP_TAB is in the same situation: it has a VMCS field and needs to be switched atomically.

Either way, it's up to you.

#36 lore raw

Re: [PATCH v9 08/22] KVM: VMX: Set FRED MSR intercepts

Quoted 17 lines · Chao Gao
Quoted 16 lines · Sean Christopherson
Quoted 14 lines · Chao Gao
Quoted 8 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

On a userspace MSR filter change, set FRED MSR intercepts.

The eight FRED MSRs, MSR_IA32_FRED_RSP[123], MSR_IA32_FRED_STKLVLS, MSR_IA32_FRED_SSP[123] and MSR_IA32_FRED_CONFIG, are all safe to passthrough, because each has a corresponding host and guest field in VMCS.

Sean prefers to pass through MSRs only when there is a reason to do that rather than just because it is free. My thinking is that RSPs and SSPs are per-task and are context-switched frequently, so we need to pass through them. But I am not sure if there is a reason for STKLVLS and CONFIG.

Quoted 5 lines
Quoted 2 lines

There are VMCS fields, at which point intercepting and emulating is probably more work than just letting the guest access directly. :-/

Just drop the MSR intercepting code and everything should work, right? KVM needs to handle userspace writes anyway. so, there is no "more work" to me.

True. I was thinking KVM would need to marshall the value to/from the hardware MSR, but that's obviously not necessary :-) (and also would be comically wrong).

After working through the various implications, I think it makes to adjust the "rule" to be "if necessary for performance OR it's _completely_ free (minus the interception toggling)" (and in both cases, obviously disabling interception needs to be functionally safe/correct too). Because I think we'll end up with confusing code if we limit disable interception only for performance reasons.

E.g. I can't imagine MSR_IA32_S_CET will get modified post-boot, so by the performance-only rule, KVM should always intercept S_CET. But MSR_IA32_U_CET can be read/written much more frequency, and so should be passed through. And then we'd end up intercept S_CET but not U_CET, which _looks_ wrong.

The FRED MSRs fall into the same boat. Intercepting only STKLVLS and CONFIG is likely a-ok from a performance perspective, but once this is all merged and folks that weren't part of this discussion come along, readers will likely be wondering why STKLVLS and CONFIG are "missing".

All in all, unless someone has an functional or performance argument against disabling interception, I think it makes sense to disabling interception for all FRED MSRs that are context switched by hardware.

#37 lore raw

Re: [PATCH v9 08/22] KVM: VMX: Set FRED MSR intercepts

In reply to Xin Li (Intel)
Quoted 7 lines · Xin Li (Intel)

Both MSR_IA32_FRED_RSP0 and MSR_IA32_FRED_SSP0 (aka MSR_IA32_PL0_SSP) are dedicated for userspace event delivery, IOW they are NOT used in any kernel event delivery and the execution of ERETS. Thus KVM can run safely with guest values in the two MSRs. As a result, save and restore of their guest values are deferred until vCPU context switch, Host MSR_IA32_FRED_RSP0 is restored upon returning to userspace, and Host MSR_IA32_PL0_SSP is managed with XRSTORS/XSAVES.

Is it worth making MSR_IA32_FRED_RSP0 special versus MSR_IA32_FRED_RSP[123]?

Is it needed because MSR_IA32_FRED_RSP0 is rewritten all the time as CPUs switch between threads? But MSR_IA32_FRED_RSP[123] are not frequently written?

I'd like to hear more about the motivation.

maintainer
#38 lore raw

Re: [PATCH v9 08/22] KVM: VMX: Set FRED MSR intercepts

Quoted 9 lines · Dave Hansen
Quoted 7 lines · Xin Li (Intel)

Both MSR_IA32_FRED_RSP0 and MSR_IA32_FRED_SSP0 (aka MSR_IA32_PL0_SSP) are dedicated for userspace event delivery, IOW they are NOT used in any kernel event delivery and the execution of ERETS. Thus KVM can run safely with guest values in the two MSRs. As a result, save and restore of their guest values are deferred until vCPU context switch, Host MSR_IA32_FRED_RSP0 is restored upon returning to userspace, and Host MSR_IA32_PL0_SSP is managed with XRSTORS/XSAVES.

Quoted 7 lines

Is it worth making MSR_IA32_FRED_RSP0 special versus MSR_IA32_FRED_RSP[123]?

Is it needed because MSR_IA32_FRED_RSP0 is rewritten all the time as CPUs switch between threads? But MSR_IA32_FRED_RSP[123] are not frequently written?

I'd like to hear more about the motivation.

Because RSP[123] (and SSP[123]) are used by the kernel itself they are context-switched by VTx automatically. This is necessary in order to preserve the FRED architectural invariant that there should NEVER be a "gap" during which it is unsafe to take an exception.

[RS]SP0 are not used while in kernel mode (since the only time we switch *onto* the level 0 kernel stack is when entering from user space, logically "CSL -1"), so during FRED architectural discussions it was agreed that it was better to leave its management to kernel code, especially as KVM often does not need to cross back into user space after each VMEXIT.

A nice side effect, but just that -- a side effect -- is that we don't need to actually modify [RS]SP0 in the context-switch or task setup code.

The invariant that needs to be maintained is that IF the cached rsp0 value is equal to the initial stack pointer for the running task, THEN the MSR MUST match the cached value. A corollary of that is that if we modify either the MSR or the cached value from an event that may have interrupted the kernel we MUST make sure that this invariant cannot be inadvertently broken.

Setting the cached value to an invalid value (e.g. NULL/0) should work; there shouldn't be an actual need to read the MSR unless I'm mistaken -- but I have been working on other code today and so my cache for this specific code is not 100% up to date.

-hpa

author maintainer
#39 lore raw

PATCH v9 9/22 KVM: VMX: Save/restore guest FRED RSP0

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Save guest FRED RSP0 in vmx_prepare_switch_to_host() and restore it in vmx_prepare_switch_to_guest() because MSR_IA32_FRED_RSP0 is passed through to the guest, thus is volatile/unknown.

Note, host FRED RSP0 is restored in arch_exit_to_user_mode_prepare(), regardless of whether it is modified in KVM.

---

Changes in v5:
* Remove the cpu_feature_enabled() check when set/get guest MSR_IA32_FRED_RSP0, as guest_cpu_cap_has() should suffice (Sean).
* Add a comment when synchronizing current MSR_IA32_FRED_RSP0 MSR to the kernel's local cache, because its handling is different from the MSR_KERNEL_GS_BASE handling (Sean).
* Add TB from Xuelian Guo.

Changes in v3:
* KVM only needs to save/restore guest FRED RSP0 now as host FRED RSP0 is restored in arch_exit_to_user_mode_prepare() (Sean Christopherson).

Changes in v2:
* Don't use guest_cpuid_has() in vmx_prepare_switch_to_{host,guest}(), which are called from IRQ-disabled context (Chao Gao).
* Reset msr_guest_fred_rsp0 in __vmx_vcpu_reset() (Chao Gao).

---
 arch/x86/kvm/vmx/vmx.c | 13 +++++++++++++
 arch/x86/kvm/vmx/vmx.h |  1 +
 2 files changed, 14 insertions(+)
arch/x86/kvm/vmx/vmx.c +13 -0
index ef9765779884..c1fb3745247c 100644
@@ -1292,6 +1292,9 @@ void vmx_prepare_switch_to_guest(struct kvm_vcpu *vcpu)
 	}
 
 	wrmsrq(MSR_KERNEL_GS_BASE, vmx->msr_guest_kernel_gs_base);
+
+	if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))
+		wrmsrns(MSR_IA32_FRED_RSP0, vmx->msr_guest_fred_rsp0);
 #else
 	savesegment(fs, fs_sel);
 	savesegment(gs, gs_sel);
@@ -1336,6 +1339,16 @@ static void vmx_prepare_switch_to_host(struct vcpu_vmx *vmx)
 	invalidate_tss_limit();
 #ifdef CONFIG_X86_64
 	wrmsrq(MSR_KERNEL_GS_BASE, vmx->vt.msr_host_kernel_gs_base);
+
+	if (guest_cpu_cap_has(&vmx->vcpu, X86_FEATURE_FRED)) {
+		vmx->msr_guest_fred_rsp0 = read_msr(MSR_IA32_FRED_RSP0);
+		/*
+		 * Synchronize the current value in hardware to the kernel's
+		 * local cache.  The desired host RSP0 will be set when the
+		 * CPU exits to userspace (RSP0 is a per-task value).
+		 */
+		fred_sync_rsp0(vmx->msr_guest_fred_rsp0);
+	}
 #endif
 	load_fixmap_gdt(raw_smp_processor_id());
 	vmx->vt.guest_state_loaded = false;
arch/x86/kvm/vmx/vmx.h +1 -0
index 645b0343e88c..48a5ab12cccf 100644
@@ -227,6 +227,7 @@ struct vcpu_vmx {
 	bool                  guest_uret_msrs_loaded;
 #ifdef CONFIG_X86_64
 	u64		      msr_guest_kernel_gs_base;
+	u64		      msr_guest_fred_rsp0;
 #endif
 
 	u64		      spec_ctrl;
Signature · 1 line
2.51.0
Reviewed-by on v9 9/22
#40 lore raw

Re: [PATCH v9 09/22] KVM: VMX: Save/restore guest FRED RSP0

Quoted 6 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Save guest FRED RSP0 in vmx_prepare_switch_to_host() and restore it in vmx_prepare_switch_to_guest() because MSR_IA32_FRED_RSP0 is passed through to the guest, thus is volatile/unknown.

Quoted 7 lines

Note, host FRED RSP0 is restored in arch_exit_to_user_mode_prepare(), regardless of whether it is modified in KVM.

Reviewed-by on v9 9/22
#41 lore raw

Re: [PATCH v9 09/22] KVM: VMX: Save/restore guest FRED RSP0

In reply to Xin Li (Intel)
Quoted 6 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Save guest FRED RSP0 in vmx_prepare_switch_to_host() and restore it in vmx_prepare_switch_to_guest() because MSR_IA32_FRED_RSP0 is passed through to the guest, thus is volatile/unknown.

Quoted 7 lines

Note, host FRED RSP0 is restored in arch_exit_to_user_mode_prepare(), regardless of whether it is modified in KVM.

Quoted 66 lines

---

Changes in v5:
* Remove the cpu_feature_enabled() check when set/get guest MSR_IA32_FRED_RSP0, as guest_cpu_cap_has() should suffice (Sean).
* Add a comment when synchronizing current MSR_IA32_FRED_RSP0 MSR to the kernel's local cache, because its handling is different from the MSR_KERNEL_GS_BASE handling (Sean).
* Add TB from Xuelian Guo.

Changes in v3:
* KVM only needs to save/restore guest FRED RSP0 now as host FRED RSP0 is restored in arch_exit_to_user_mode_prepare() (Sean Christopherson).

Changes in v2:
* Don't use guest_cpuid_has() in vmx_prepare_switch_to_{host,guest}(), which are called from IRQ-disabled context (Chao Gao).
* Reset msr_guest_fred_rsp0 in __vmx_vcpu_reset() (Chao Gao).

---
 arch/x86/kvm/vmx/vmx.c | 13 +++++++++++++
 arch/x86/kvm/vmx/vmx.h |  1 +
 2 files changed, 14 insertions(+)
arch/x86/kvm/vmx/vmx.c +13 -0
index ef9765779884..c1fb3745247c 100644
@@ -1292,6 +1292,9 @@ void vmx_prepare_switch_to_guest(struct kvm_vcpu *vcpu)
 	}
 
 	wrmsrq(MSR_KERNEL_GS_BASE, vmx->msr_guest_kernel_gs_base);
+
+	if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))
+		wrmsrns(MSR_IA32_FRED_RSP0, vmx->msr_guest_fred_rsp0);
 #else
 	savesegment(fs, fs_sel);
 	savesegment(gs, gs_sel);
@@ -1336,6 +1339,16 @@ static void vmx_prepare_switch_to_host(struct vcpu_vmx *vmx)
 	invalidate_tss_limit();
 #ifdef CONFIG_X86_64
 	wrmsrq(MSR_KERNEL_GS_BASE, vmx->vt.msr_host_kernel_gs_base);
+
+	if (guest_cpu_cap_has(&vmx->vcpu, X86_FEATURE_FRED)) {
+		vmx->msr_guest_fred_rsp0 = read_msr(MSR_IA32_FRED_RSP0);
+		/*
+		 * Synchronize the current value in hardware to the kernel's
+		 * local cache.  The desired host RSP0 will be set when the
+		 * CPU exits to userspace (RSP0 is a per-task value).
+		 */
+		fred_sync_rsp0(vmx->msr_guest_fred_rsp0);
+	}
 #endif
 	load_fixmap_gdt(raw_smp_processor_id());
 	vmx->vt.guest_state_loaded = false;
arch/x86/kvm/vmx/vmx.h +1 -0
index 645b0343e88c..48a5ab12cccf 100644
@@ -227,6 +227,7 @@ struct vcpu_vmx {
 	bool                  guest_uret_msrs_loaded;
 #ifdef CONFIG_X86_64
 	u64		      msr_guest_kernel_gs_base;
+	u64		      msr_guest_fred_rsp0;
 #endif
 
 	u64		      spec_ctrl;
author maintainer
#42 lore raw

PATCH v9 10/22 KVM: VMX: Add support for saving and restoring FRED MSRs

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Introduce support for handling FRED MSR access requests, enabling both host and guest to read and write FRED MSRs, which is essential for VM save/restore and live migration, and allows userspace tools such as QEMU to access the relevant MSRs.

Specially, intercept accesses to the FRED SSP0 MSR (IA32_PL0_SSP), which remains accessible when FRED is enumerated even if CET is not. This ensures the guest value is fully virtual and does not alter the hardware FRED SSP0 MSR.

---

Change in v7:
* Intercept accesses to FRED SSP0, i.e., IA32_PL0_SSP, which remains accessible when FRED but !CET (Sean).

Change in v6:
* Return KVM_MSR_RET_UNSUPPORTED instead of 1 when FRED is not available (Chao Gao)
* Handle MSR_IA32_PL0_SSP when FRED is enumerated but CET not.

Change in v5:
* Use the newly added guest MSR read/write helpers (Sean).
* Check the size of fred_msr_vmcs_fields[] using static_assert() (Sean).
* Rewrite setting FRED MSRs to make it much easier to read (Sean).
* Add TB from Xuelian Guo.

Changes since v2:
* Add a helper to convert FRED MSR index to VMCS field encoding to make the code more compact (Chao Gao).
* Get rid of the "host_initiated" check because userspace has to set CPUID before MSRs (Chao Gao & Sean Christopherson).
* Address a few cleanup comments (Sean Christopherson).

Changes since v1:
* Use kvm_cpu_cap_has() instead of cpu_feature_enabled() (Chao Gao).
* Fail host requested FRED MSRs access if KVM cannot virtualize FRED (Chao Gao).
* Handle the case FRED MSRs are valid but KVM cannot virtualize FRED (Chao Gao).
* Add sanity checks when writing to FRED MSRs.

---
 arch/x86/include/asm/kvm_host.h |  5 ++
 arch/x86/kvm/vmx/vmx.c          | 45 +++++++++++++++++
 arch/x86/kvm/x86.c              | 85 +++++++++++++++++++++++++++++++--
 3 files changed, 132 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +5 -0
index 48598d017d6f..43a18e265289 100644
@@ -1092,6 +1092,11 @@ struct kvm_vcpu_arch {
 #if IS_ENABLED(CONFIG_HYPERV)
 	hpa_t hv_root_tdp;
 #endif
+	/*
+	 * Stores the FRED SSP0 MSR when CET is not supported, prompting KVM
+	 * to intercept its accesses.
+	 */
+	u64 fred_ssp0_fallback;
 };
 
 struct kvm_lpage_info {
arch/x86/kvm/vmx/vmx.c +45 -0

69 lines not on this page · raw message

arch/x86/kvm/x86.c +82 -3

133 lines not on this page · raw message

Signature · 1 line
2.51.0
Reviewed-by on v9 10/22
#43 lore raw

Re: [PATCH v9 10/22] KVM: VMX: Add support for saving and restoring FRED MSRs

arch/x86/kvm/x86.c int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 4378

Quoted 6 lines
arch/x86/kvm/x86.c +2 -0
@@ -4316,6 +4374,12 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 #endif
 	case MSR_IA32_U_CET:
 	case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:
+		if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) {
+			WARN_ON_ONCE(msr != MSR_IA32_FRED_SSP0);

This will be triggered if the guest only supports IBT and tries to write U_CET here.

arch/x86/kvm/x86.c int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 4737

Quoted 13 lines
arch/x86/kvm/x86.c +6 -0
+			vcpu->arch.fred_ssp0_fallback = data;
+			break;
+		}
+
 		kvm_set_xstate_msr(vcpu, msr_info);
 		break;
 	default:
@@ -4669,6 +4733,12 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 #endif
 	case MSR_IA32_U_CET:
 	case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:
+		if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) {
+			WARN_ON_ONCE(msr_info->index != MSR_IA32_FRED_SSP0);

ditto.

With this fixed,

author maintainer
#44 lore raw

Re: [PATCH v9 10/22] KVM: VMX: Add support for saving and restoring FRED MSRs

Quoted 9 lines
Quoted 7 lines
(quoted diff) +0 -0
@@ -4316,6 +4374,12 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 #endif

case MSR_IA32_U_CET:
case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:
+ if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) {
+ WARN_ON_ONCE(msr != MSR_IA32_FRED_SSP0);

This will be triggered if the guest only supports IBT and tries to write U_CET here.

You’re right, my stupid.

author maintainer
#45 lore raw

PATCH v9 11/22 KVM: x86: Add a helper to detect if FRED is enabled for a vCPU

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

[ Sean: removed the "kvm_" prefix from the function name ]

---

Change in v5:
* Add TB from Xuelian Guo.

---
 arch/x86/kvm/kvm_cache_regs.h | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
arch/x86/kvm/kvm_cache_regs.h +15 -0
index 8ddb01191d6f..3c8dbb77d7d4 100644
@@ -205,6 +205,21 @@ static __always_inline bool kvm_is_cr4_bit_set(struct kvm_vcpu *vcpu,
 	return !!kvm_read_cr4_bits(vcpu, cr4_bit);
 }
 
+/*
+ * It's enough to check just CR4.FRED (X86_CR4_FRED) to tell if
+ * a vCPU is running with FRED enabled, because:
+ * 1) CR4.FRED can be set to 1 only _after_ IA32_EFER.LMA = 1.
+ * 2) To leave IA-32e mode, CR4.FRED must be cleared first.
+ */
+static inline bool is_fred_enabled(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_X86_64
+	return kvm_is_cr4_bit_set(vcpu, X86_CR4_FRED);
+#else
+	return false;
+#endif
+}
+
 static inline ulong kvm_read_cr3(struct kvm_vcpu *vcpu)
 {
 	if (!kvm_register_is_available(vcpu, VCPU_EXREG_CR3))
Signature · 1 line
2.51.0
Reviewed-by on v9 11/22
#46 lore raw

Re: [PATCH v9 11/22] KVM: x86: Add a helper to detect if FRED is enabled for a vCPU

Quoted 8 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

[ Sean: removed the "kvm_" prefix from the function name ]
#47 lore raw

Re: [PATCH v9 11/22] KVM: x86: Add a helper to detect if FRED is enabled for a vCPU

In reply to Xin Li (Intel)
Quoted 2 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Not sure if it's OK with empty change log even though the patch is simple and the title has already described it.

Quoted 40 lines
[ Sean: removed the "kvm_" prefix from the function name ]

---

Change in v5:
* Add TB from Xuelian Guo.

---
 arch/x86/kvm/kvm_cache_regs.h | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
arch/x86/kvm/kvm_cache_regs.h +15 -0
index 8ddb01191d6f..3c8dbb77d7d4 100644
@@ -205,6 +205,21 @@ static __always_inline bool kvm_is_cr4_bit_set(struct kvm_vcpu *vcpu,
 	return !!kvm_read_cr4_bits(vcpu, cr4_bit);
 }
 
+/*
+ * It's enough to check just CR4.FRED (X86_CR4_FRED) to tell if
+ * a vCPU is running with FRED enabled, because:
+ * 1) CR4.FRED can be set to 1 only _after_ IA32_EFER.LMA = 1.
+ * 2) To leave IA-32e mode, CR4.FRED must be cleared first.
+ */
+static inline bool is_fred_enabled(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_X86_64
+	return kvm_is_cr4_bit_set(vcpu, X86_CR4_FRED);
+#else
+	return false;
+#endif
+}
+
 static inline ulong kvm_read_cr3(struct kvm_vcpu *vcpu)
 {
 	if (!kvm_register_is_available(vcpu, VCPU_EXREG_CR3))
author maintainer
#48 lore raw

Re: [PATCH v9 11/22] KVM: x86: Add a helper to detect if FRED is enabled for a vCPU

Quoted 6 lines · Binbin Wu

Not sure if it's OK with empty change log even though the patch is simple and the title has already described it.

IIRC, Sean changed it this way ;)

maintainer Tested-by Shan Kang
#49 lore raw

Re: [PATCH v9 11/22] KVM: x86: Add a helper to detect if FRED is enabled for a vCPU

Quoted 8 lines · Xin Li
Quoted 6 lines · Binbin Wu

Not sure if it's OK with empty change log even though the patch is simple and the title has already described it.

IIRC, Sean changed it this way ;)

I doubt that. Ha! Found it. From: https://lore.kernel.org/all/[email protected]
as an attachment:

[-- Attachment #4: 0013-KVM-x86-Add-a-helper-to-detect-if-FRED-is-enabled-fo.patch --] [-- Type: text/x-diff, Size: 1418 bytes --]

From f38dcc04e334cda572289f05f4be7702bebfc96a Mon Sep 17 00:00:00 2001
From: Xin Li <[email protected]>
Date: Wed, 7 Feb 2024 09:26:31 -0800
Subject: [PATCH 13/28] KVM: x86: Add a helper to detect if FRED is enabled for
a vCPU

Add is_fred_enabled() to detect if FRED is enabled on a vCPU.

---
 arch/x86/kvm/kvm_cache_regs.h | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
arch/x86/kvm/kvm_cache_regs.h +15 -0
index 75eae9c4998a..fe5546efd388 100644
@@ -187,6 +187,21 @@ static __always_inline bool kvm_is_cr4_bit_set(struct kvm_vcpu *vcpu,
 	return !!kvm_read_cr4_bits(vcpu, cr4_bit);
 }
 
+/*
+ * It's enough to check just CR4.FRED (X86_CR4_FRED) to tell if
+ * a vCPU is running with FRED enabled, because:
+ * 1) CR4.FRED can be set to 1 only _after_ IA32_EFER.LMA = 1.
+ * 2) To leave IA-32e mode, CR4.FRED must be cleared first.
+ */
+static inline bool is_fred_enabled(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_X86_64
+	return kvm_is_cr4_bit_set(vcpu, X86_CR4_FRED);
+#else
+	return false;
+#endif
+}
+
 static inline ulong kvm_read_cr3(struct kvm_vcpu *vcpu)
 {
 	if (!kvm_register_is_available(vcpu, VCPU_EXREG_CR3))
Signature · 1 line
2.45.2.627.g7a2c4fd464-goog
author maintainer
#51 lore raw

PATCH v9 12/22 KVM: VMX: Virtualize FRED event_data

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Set injected-event data when injecting a #PF, #DB, or #NM caused by extended feature disable using FRED event delivery, and save original-event data for being used as injected-event data.

Unlike IDT using some extra CPU register as part of an event context, e.g., %cr2 for #PF, FRED saves a complete event context in its stack frame, e.g., FRED saves the faulting linear address of a #PF into the event data field defined in its stack frame.

Thus a new VMX control field called injected-event data is added to provide the event data that will be pushed into a FRED stack frame for VM entries that inject an event using FRED event delivery. In addition, a new VM exit information field called original-event data is added to store the event data that would have saved into a FRED stack frame for VM exits that occur during FRED event delivery. After such a VM exit is handled to allow the original-event to be delivered, the data in the original-event data VMCS field needs to be set into the injected-event data VMCS field for the injection of the original event.

[ Sean: reworked event data injection for nested ]

---

Change in v5:
* Add TB from Xuelian Guo.

Change in v3:
* Rework event data injection for nested (Chao Gao & Sean Christopherson).

Changes in v2:
* Document event data should be equal to CR2/DR6/IA32_XFD_ERR instead of using WARN_ON() (Chao Gao).
* Zero event data if a #NM was not caused by extended feature disable (Chao Gao).

---
 arch/x86/include/asm/kvm_host.h |  3 ++-
 arch/x86/include/asm/vmx.h      |  4 ++++
 arch/x86/kvm/svm/svm.c          |  2 +-
 arch/x86/kvm/vmx/vmx.c          | 22 ++++++++++++++++++----
 arch/x86/kvm/x86.c              | 16 +++++++++++++++-
 5 files changed, 40 insertions(+), 7 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -1
index 43a18e265289..550a8716a227 100644
@@ -760,6 +760,7 @@ struct kvm_queued_exception {
 	u32 error_code;
 	unsigned long payload;
 	bool has_payload;
+	u64 event_data;
 };
 
 /*
@@ -2230,7 +2231,7 @@ void kvm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr);
 void kvm_queue_exception_e(struct kvm_vcpu *vcpu, unsigned nr, u32 error_code);
 void kvm_queue_exception_p(struct kvm_vcpu *vcpu, unsigned nr, unsigned long payload);
 void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr,
-			   bool has_error_code, u32 error_code);
+			   bool has_error_code, u32 error_code, u64 event_data);
 void kvm_inject_page_fault(struct kvm_vcpu *vcpu, struct x86_exception *fault);
 void kvm_inject_emulated_page_fault(struct kvm_vcpu *vcpu,
 				    struct x86_exception *fault);
arch/x86/include/asm/vmx.h +4 -0
index 6f8b8947c60c..539af190ad3e 100644
@@ -269,8 +269,12 @@ enum vmcs_field {
 	PID_POINTER_TABLE_HIGH		= 0x00002043,
 	SECONDARY_VM_EXIT_CONTROLS	= 0x00002044,
 	SECONDARY_VM_EXIT_CONTROLS_HIGH	= 0x00002045,
+	INJECTED_EVENT_DATA		= 0x00002052,
+	INJECTED_EVENT_DATA_HIGH	= 0x00002053,
 	GUEST_PHYSICAL_ADDRESS          = 0x00002400,
 	GUEST_PHYSICAL_ADDRESS_HIGH     = 0x00002401,
+	ORIGINAL_EVENT_DATA		= 0x00002404,
+	ORIGINAL_EVENT_DATA_HIGH	= 0x00002405,
 	VMCS_LINK_POINTER               = 0x00002800,
 	VMCS_LINK_POINTER_HIGH          = 0x00002801,
 	GUEST_IA32_DEBUGCTL             = 0x00002802,
arch/x86/kvm/svm/svm.c +1 -1
index f14709a511aa..2f20c68fcfb3 100644
@@ -4104,7 +4104,7 @@ static void svm_complete_interrupts(struct kvm_vcpu *vcpu)
 
 		kvm_requeue_exception(vcpu, vector,
 				      exitintinfo & SVM_EXITINTINFO_VALID_ERR,
-				      error_code);
+				      error_code, 0);
 		break;
 	}
 	case SVM_EXITINTINFO_TYPE_INTR:
arch/x86/kvm/vmx/vmx.c +18 -4
index 4a74c9f64f90..0b5d04c863a8 100644
@@ -1860,6 +1860,9 @@ void vmx_inject_exception(struct kvm_vcpu *vcpu)
 
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr_info);
 
+	if (is_fred_enabled(vcpu))
+		vmcs_write64(INJECTED_EVENT_DATA, ex->event_data);
+
 	vmx_clear_hlt(vcpu);
 }
 
@@ -7299,7 +7302,8 @@ static void vmx_recover_nmi_blocking(struct vcpu_vmx *vmx)
 static void __vmx_complete_interrupts(struct kvm_vcpu *vcpu,
 				      u32 idt_vectoring_info,
 				      int instr_len_field,
-				      int error_code_field)
+				      int error_code_field,
+				      int event_data_field)
 {
 	u8 vector;
 	int type;
@@ -7334,13 +7338,17 @@ static void __vmx_complete_interrupts(struct kvm_vcpu *vcpu,
 		fallthrough;
 	case INTR_TYPE_HARD_EXCEPTION: {
 		u32 error_code = 0;
+		u64 event_data = 0;
 
 		if (idt_vectoring_info & VECTORING_INFO_DELIVER_CODE_MASK)
 			error_code = vmcs_read32(error_code_field);
+		if (is_fred_enabled(vcpu))
+			event_data = vmcs_read64(event_data_field);
 
 		kvm_requeue_exception(vcpu, vector,
 				      idt_vectoring_info & VECTORING_INFO_DELIVER_CODE_MASK,
-				      error_code);
+				      error_code,
+				      event_data);
 		break;
 	}
 	case INTR_TYPE_SOFT_INTR:
@@ -7358,7 +7366,8 @@ static void vmx_complete_interrupts(struct vcpu_vmx *vmx)
 {
 	__vmx_complete_interrupts(&vmx->vcpu, vmx->idt_vectoring_info,
 				  VM_EXIT_INSTRUCTION_LEN,
-				  IDT_VECTORING_ERROR_CODE);
+				  IDT_VECTORING_ERROR_CODE,
+				  ORIGINAL_EVENT_DATA);
 }
 
 void vmx_cancel_injection(struct kvm_vcpu *vcpu)
@@ -7366,7 +7375,8 @@ void vmx_cancel_injection(struct kvm_vcpu *vcpu)
 	__vmx_complete_interrupts(vcpu,
 				  vmcs_read32(VM_ENTRY_INTR_INFO_FIELD),
 				  VM_ENTRY_INSTRUCTION_LEN,
-				  VM_ENTRY_EXCEPTION_ERROR_CODE);
+				  VM_ENTRY_EXCEPTION_ERROR_CODE,
+				  INJECTED_EVENT_DATA);
 
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, 0);
 }
@@ -7520,6 +7530,10 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu,
 
 	vmx_disable_fb_clear(vmx);
 
+	/*
+	 * Note, even though FRED delivers the faulting linear address via the
+	 * event data field on the stack, CR2 is still updated.
+	 */
 	if (vcpu->arch.cr2 != native_read_cr2())
 		native_write_cr2(vcpu->arch.cr2);
 
arch/x86/kvm/x86.c +15 -1
index 3d612803f5f2..10f1663d51d7 100644
@@ -815,9 +815,22 @@ void kvm_deliver_exception_payload(struct kvm_vcpu *vcpu,
 		 * breakpoint), it is reserved and must be zero in DR6.
 		 */
 		vcpu->arch.dr6 &= ~BIT(12);
+
+		/*
+		 * FRED #DB event data matches DR6, but follows the polarity of
+		 * VMX's pending debug exceptions, not DR6.
+		 */
+		ex->event_data = ex->payload & ~BIT(12);
+		break;
+	case NM_VECTOR:
+		ex->event_data = ex->payload;
 		break;
 	case PF_VECTOR:
 		vcpu->arch.cr2 = ex->payload;
+		ex->event_data = ex->payload;
+		break;
+	default:
+		ex->event_data = 0;
 		break;
 	}
 
@@ -925,7 +938,7 @@ static void kvm_queue_exception_e_p(struct kvm_vcpu *vcpu, unsigned nr,
 }
 
 void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr,
-			   bool has_error_code, u32 error_code)
+			   bool has_error_code, u32 error_code, u64 event_data)
 {
 
 	/*
@@ -950,6 +963,7 @@ void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr,
 	vcpu->arch.exception.error_code = error_code;
 	vcpu->arch.exception.has_payload = false;
 	vcpu->arch.exception.payload = 0;
+	vcpu->arch.exception.event_data = event_data;
 }
 EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_requeue_exception);
 
Signature · 1 line
2.51.0
#52 lore raw

Re: [PATCH v9 12/22] KVM: VMX: Virtualize FRED event_data

arch/x86/kvm/vmx/vmx.c void vmx_inject_exception(struct kvm_vcpu *vcpu) · line 1864

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +2 -0
index 4a74c9f64f90..0b5d04c863a8 100644
@@ -1860,6 +1860,9 @@ void vmx_inject_exception(struct kvm_vcpu *vcpu)
 
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr_info);
 
+	if (is_fred_enabled(vcpu))
+		vmcs_write64(INJECTED_EVENT_DATA, ex->event_data);

I think event_data should be reset to 0 in kvm_clear_exception_queue(). Otherwise, ex->event_data may be stale here, i.e., the event_data from the previous event may be injected along with the next event.

<snip>

arch/x86/kvm/vmx/vmx.c void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr, · line 966

Quoted 11 lines
arch/x86/kvm/vmx/vmx.c +1 -0
+
 	vmx_clear_hlt(vcpu);
 }
 
	/*
arch/x86/kvm/vmx/vmx.c +1 -0
@@ -950,6 +963,7 @@ void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr,
 	vcpu->arch.exception.error_code = error_code;
 	vcpu->arch.exception.has_payload = false;
 	vcpu->arch.exception.payload = 0;
+	vcpu->arch.exception.event_data = event_data;

If userspace saves guest events (via kvm_vcpu_ioctl_x86_get_vcpu_events()) right after an event is requeued, event_data will be lost (as that uAPI only saves the payload and KVM doesn't convert the event_data back to a payload there). So this event will be delivered with incorrect event_data if the event is restored on another system after migration.

Quoted 6 lines
arch/x86/kvm/vmx/vmx.c +0 -0
 }
 EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_requeue_exception);
 
Signature · 1 line
2.51.0
author maintainer
#53 lore raw

Re: [PATCH v9 12/22] KVM: VMX: Virtualize FRED event_data

Quoted 9 lines · Chao Gao
Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +0 -0
index 4a74c9f64f90..0b5d04c863a8 100644
@@ -1860,6 +1860,9 @@ void vmx_inject_exception(struct kvm_vcpu *vcpu)
 
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c void vmx_inject_exception(struct kvm_vcpu *vcpu) · line 1864

Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +2 -0
 vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr_info);
 
+ if (is_fred_enabled(vcpu))
+ vmcs_write64(INJECTED_EVENT_DATA, ex->event_data);

I think event_data should be reset to 0 in kvm_clear_exception_queue(). Otherwise, ex->event_data may be stale here, i.e., the event_data from the previous event may be injected along with the next event.

It’s no harm to reset it, although it shouldn’t be stale when an event that uses event data is being injected (otherwise it’s a bug).

Quoted 6 lines
Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +1 -0
+
 vmx_clear_hlt(vcpu);
 }
 
Quoted 12 lines

arch/x86/kvm/vmx/vmx.c void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr, · line 966

Quoted 6 lines

/*

arch/x86/kvm/vmx/vmx.c +1 -0
@@ -950,6 +963,7 @@ void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr,
 vcpu->arch.exception.error_code = error_code;
 vcpu->arch.exception.has_payload = false;
 vcpu->arch.exception.payload = 0;
+ vcpu->arch.exception.event_data = event_data;

If userspace saves guest events (via kvm_vcpu_ioctl_x86_get_vcpu_events()) right after an event is requeued, event_data will be lost (as that uAPI only saves the payload and KVM doesn't convert the event_data back to a payload there). So this event will be delivered with incorrect event_data if the event is restored on another system after migration.

Nice catch!

Just to confirm, you are referring to requeueing an original event via vmx_complete_interrupts(), right?

Regardless of whether FRED or IDT is in use, the event payload is delivered into the appropriate guest state and then invalidated in kvm_deliver_exception_payload():

1) CR2 for #PF

2) DR6 for #DB

3) guest_fpu.xfd_err for #NM (in handle_nm_fault_irqoff())

We should be able to recover the FRED event data from there.

Alternatively, we could drop the original event and allow the hardware to regenerate it upon resuming the guest. However, this breaks #DB delivery, as debug exceptions sometimes are triggered post-instruction.

Sean, does it make sense to recover the FRED event data from guest CPU state?

maintainer
#54 lore raw

Re: [PATCH v9 12/22] KVM: VMX: Virtualize FRED event_data

Quoted 56 lines · Xin Li
Quoted 9 lines · Chao Gao
Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +0 -0
index 4a74c9f64f90..0b5d04c863a8 100644
@@ -1860,6 +1860,9 @@ void vmx_inject_exception(struct kvm_vcpu *vcpu)
 
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c void vmx_inject_exception(struct kvm_vcpu *vcpu) · line 1864

Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +2 -0
 vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr_info);
 
+ if (is_fred_enabled(vcpu))
+ vmcs_write64(INJECTED_EVENT_DATA, ex->event_data);

I think event_data should be reset to 0 in kvm_clear_exception_queue(). Otherwise, ex->event_data may be stale here, i.e., the event_data from the previous event may be injected along with the next event.

It’s no harm to reset it, although it shouldn’t be stale when an event that uses event data is being injected (otherwise it’s a bug).

Quoted 6 lines
Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +1 -0
+
 vmx_clear_hlt(vcpu);
 }
 
Quoted 12 lines

arch/x86/kvm/vmx/vmx.c void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr, · line 966

Quoted 6 lines

/*

arch/x86/kvm/vmx/vmx.c +1 -0
@@ -950,6 +963,7 @@ void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr,
 vcpu->arch.exception.error_code = error_code;
 vcpu->arch.exception.has_payload = false;
 vcpu->arch.exception.payload = 0;
+ vcpu->arch.exception.event_data = event_data;

If userspace saves guest events (via kvm_vcpu_ioctl_x86_get_vcpu_events()) right after an event is requeued, event_data will be lost (as that uAPI only saves the payload and KVM doesn't convert the event_data back to a payload there). So this event will be delivered with incorrect event_data if the event is restored on another system after migration.

Nice catch!

Just to confirm, you are referring to requeueing an original event via vmx_complete_interrupts(), right?

Regardless of whether FRED or IDT is in use, the event payload is delivered into the appropriate guest state and then invalidated in kvm_deliver_exception_payload():

1) CR2 for #PF

2) DR6 for #DB

3) guest_fpu.xfd_err for #NM (in handle_nm_fault_irqoff())

Quoted 11 lines

We should be able to recover the FRED event data from there.

Alternatively, we could drop the original event and allow the hardware to regenerate it upon resuming the guest. However, this breaks #DB delivery, as debug exceptions sometimes are triggered post-instruction.

Sean, does it make sense to recover the FRED event data from guest CPU state?

I think some bits in DR6 are "sticky", and so unless the guest has explicitly cleared DR6 the event data isn't necessarily derivable from DR6. However, the FRED event data for #DB is directly based on the data already reported by VTx (for exactly the same reason – knowing what the *currently taken* trap represents.)

author maintainer
#55 lore raw

Re: [PATCH v9 12/22] KVM: VMX: Virtualize FRED event_data

Quoted 19 lines · H. Peter Anvin
Quoted 16 lines

Just to confirm, you are referring to requeueing an original event via vmx_complete_interrupts(), right?

Regardless of whether FRED or IDT is in use, the event payload is delivered into the appropriate guest state and then invalidated in kvm_deliver_exception_payload():

1) CR2 for #PF

2) DR6 for #DB

3) guest_fpu.xfd_err for #NM (in handle_nm_fault_irqoff())

We should be able to recover the FRED event data from there.

Quoted 8 lines
Quoted 6 lines

Alternatively, we could drop the original event and allow the hardware to regenerate it upon resuming the guest. However, this breaks #DB delivery, as debug exceptions sometimes are triggered post-instruction.

Sean, does it make sense to recover the FRED event data from guest CPU state?

I think some bits in DR6 are "sticky", and so unless the guest has explicitly cleared DR6 the event data isn't necessarily derivable from DR6. However, the FRED event data for #DB is directly based on the data already reported by VTx (for exactly the same reason – knowing what the *currently taken* trap represents.)

Yeah, it's important to keep in mind that DR6 bits are 'sticky'.

Regarding vmx_complete_interrupts(), when a VM migration occurs immediately following a VM exit with a valid original event saved in the VMCS, we can safely assume the guest DR6 state remains consistent with the original event data because there is no chance for guest OS to modify DR6.

#56 lore raw

Re: [PATCH v9 12/22] KVM: VMX: Virtualize FRED event_data

Quoted 15 lines · Xin Li
Quoted 15 lines · H. Peter Anvin
Quoted 14 lines

Just to confirm, you are referring to requeueing an original event via vmx_complete_interrupts(), right?

Regardless of whether FRED or IDT is in use, the event payload is delivered into the appropriate guest state and then invalidated in kvm_deliver_exception_payload():

1) CR2 for #PF

2) DR6 for #DB

3) guest_fpu.xfd_err for #NM (in handle_nm_fault_irqoff())

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

We should be able to recover the FRED event data from there.

Alternatively, we could drop the original event and allow the hardware to regenerate it upon resuming the guest. However, this breaks #DB delivery, as debug exceptions sometimes are triggered post-instruction.

Sean, does it make sense to recover the FRED event data from guest CPU state?

No? As Peter points out, the payload is tied to the exception and shouldn't change.

Quoted 6 lines
Quoted 5 lines

I think some bits in DR6 are "sticky", and so unless the guest has explicitly cleared DR6 the event data isn't necessarily derivable from DR6. However, the FRED event data for #DB is directly based on the data already reported by VTx (for exactly the same reason – knowing what the *currently taken* trap represents.)

Quoted 6 lines

Yeah, it's important to keep in mind that DR6 bits are 'sticky'.

Regarding vmx_complete_interrupts(), when a VM migration occurs immediately following a VM exit with a valid original event saved in the VMCS, we can safely assume the guest DR6 state remains consistent with the original event data because there is no chance for guest OS to modify DR6.

There's a different problem though. If there's a re-injected exception at the time of save/restore, the destination vCPU won't see a valid payload and thus won't set the appropriate FRED VMCS fields.

We _could_ extend KVM's uAPI to save/restore event_data, but ugh. Rather than add event_data, what if we reuse payload, and then simply skip updating register state on re-injection? E.g.

arch/x86/kvm/vmx/vmx.c +3 -0
index 967b58a8ab9d..b79d545d69c7 100644
@@ -1941,6 +1941,9 @@ void vmx_inject_exception(struct kvm_vcpu *vcpu)
        u32 intr_info = ex->vector | INTR_INFO_VALID_MASK;
        struct vcpu_vmx *vmx = to_vmx(vcpu);
 
+       if (ex->has_payload)
+               <do fred>;
+
        kvm_deliver_exception_payload(vcpu, ex);
 
        if (ex->has_error_code) {
arch/x86/kvm/x86.c +4 -0
index db3f393192d9..485eec337203 100644
@@ -773,6 +773,9 @@ void kvm_deliver_exception_payload(struct kvm_vcpu *vcpu,
        if (!ex->has_payload)
                return;
 
+       if (ex->injected)
+               goto clear_payload;
+
        switch (ex->vector) {
        case DB_VECTOR:
                /*
@@ -814,6 +817,7 @@ void kvm_deliver_exception_payload(struct kvm_vcpu *vcpu,
                break;
        }
 
+clear_payload:
        ex->has_payload = false;
        ex->payload = 0;
 }
author maintainer
#57 lore raw

PATCH v9 13/22 KVM: VMX: Virtualize FRED nested exception tracking

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Set the VMX nested exception bit in VM-entry interruption information field when injecting a nested exception using FRED event delivery to ensure:
1) A nested exception is injected on a correct stack level.
2) The nested bit defined in FRED stack frame is set.

The event stack level used by FRED event delivery depends on whether the event was a nested exception encountered during delivery of an earlier event, because a nested exception is "regarded" as happening on ring 0. E.g., when #PF is configured to use stack level 1 in IA32_FRED_STKLVLS MSR:
- nested #PF will be delivered on the stack pointed by IA32_FRED_RSP1 MSR when encountered in ring 3 and ring 0.
- normal #PF will be delivered on the stack pointed by IA32_FRED_RSP0 MSR when encountered in ring 3.

The VMX nested-exception support ensures a correct event stack level is chosen when a VM entry injects a nested exception.

[ Sean: reworked kvm_requeue_exception() to simply the code changes ]

---

Change in v5:
* Add TB from Xuelian Guo.

Change in v4:
* Move the check is_fred_enable() from kvm_multiple_exception() to vmx_inject_exception() thus avoid bleeding FRED details into kvm_multiple_exception() (Chao Gao).

Change in v3:
* Rework kvm_requeue_exception() to simply the code changes (Sean Christopherson).

Change in v2:
* Set the nested flag when there is an original interrupt (Chao Gao).

---
 arch/x86/include/asm/kvm_host.h |  4 +++-
 arch/x86/include/asm/vmx.h      |  5 ++++-
 arch/x86/kvm/svm/svm.c          |  2 +-
 arch/x86/kvm/vmx/vmx.c          |  6 +++++-
 arch/x86/kvm/x86.c              | 13 ++++++++++++-
 arch/x86/kvm/x86.h              |  1 +
 6 files changed, 26 insertions(+), 5 deletions(-)
arch/x86/include/asm/kvm_host.h +3 -1
index 550a8716a227..3b6dadf368eb 100644
@@ -760,6 +760,7 @@ struct kvm_queued_exception {
 	u32 error_code;
 	unsigned long payload;
 	bool has_payload;
+	bool nested;
 	u64 event_data;
 };
 
@@ -2231,7 +2232,8 @@ void kvm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr);
 void kvm_queue_exception_e(struct kvm_vcpu *vcpu, unsigned nr, u32 error_code);
 void kvm_queue_exception_p(struct kvm_vcpu *vcpu, unsigned nr, unsigned long payload);
 void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr,
-			   bool has_error_code, u32 error_code, u64 event_data);
+			   bool has_error_code, u32 error_code, bool nested,
+			   u64 event_data);
 void kvm_inject_page_fault(struct kvm_vcpu *vcpu, struct x86_exception *fault);
 void kvm_inject_emulated_page_fault(struct kvm_vcpu *vcpu,
 				    struct x86_exception *fault);
arch/x86/include/asm/vmx.h +4 -1
index 539af190ad3e..7b34a9357b28 100644
@@ -140,6 +140,7 @@
 #define VMX_BASIC_INOUT				BIT_ULL(54)
 #define VMX_BASIC_TRUE_CTLS			BIT_ULL(55)
 #define VMX_BASIC_NO_HW_ERROR_CODE_CC		BIT_ULL(56)
+#define VMX_BASIC_NESTED_EXCEPTION		BIT_ULL(58)
 
 static inline u32 vmx_basic_vmcs_revision_id(u64 vmx_basic)
 {
@@ -442,13 +443,15 @@ enum vmcs_field {
 #define INTR_INFO_INTR_TYPE_MASK        0x700           /* 10:8 */
 #define INTR_INFO_DELIVER_CODE_MASK     0x800           /* 11 */
 #define INTR_INFO_UNBLOCK_NMI		0x1000		/* 12 */
+#define INTR_INFO_NESTED_EXCEPTION_MASK	0x2000		/* 13 */
 #define INTR_INFO_VALID_MASK            0x80000000      /* 31 */
-#define INTR_INFO_RESVD_BITS_MASK       0x7ffff000
+#define INTR_INFO_RESVD_BITS_MASK       0x7fffd000
 
 #define VECTORING_INFO_VECTOR_MASK           	INTR_INFO_VECTOR_MASK
 #define VECTORING_INFO_TYPE_MASK        	INTR_INFO_INTR_TYPE_MASK
 #define VECTORING_INFO_DELIVER_CODE_MASK    	INTR_INFO_DELIVER_CODE_MASK
 #define VECTORING_INFO_VALID_MASK       	INTR_INFO_VALID_MASK
+#define VECTORING_INFO_NESTED_EXCEPTION_MASK	INTR_INFO_NESTED_EXCEPTION_MASK
 
 #define INTR_TYPE_EXT_INTR		(EVENT_TYPE_EXTINT << 8)	/* external interrupt */
 #define INTR_TYPE_RESERVED		(EVENT_TYPE_RESERVED << 8)	/* reserved */
arch/x86/kvm/svm/svm.c +1 -1
index 2f20c68fcfb3..e3702ca2f633 100644
@@ -4104,7 +4104,7 @@ static void svm_complete_interrupts(struct kvm_vcpu *vcpu)
 
 		kvm_requeue_exception(vcpu, vector,
 				      exitintinfo & SVM_EXITINTINFO_VALID_ERR,
-				      error_code, 0);
+				      error_code, false, 0);
 		break;
 	}
 	case SVM_EXITINTINFO_TYPE_INTR:
arch/x86/kvm/vmx/vmx.c +5 -1
index 0b5d04c863a8..34e057f65513 100644
@@ -1855,8 +1855,11 @@ void vmx_inject_exception(struct kvm_vcpu *vcpu)
 		vmcs_write32(VM_ENTRY_INSTRUCTION_LEN,
 			     vmx->vcpu.arch.event_exit_inst_len);
 		intr_info |= INTR_TYPE_SOFT_EXCEPTION;
-	} else
+	} else {
 		intr_info |= INTR_TYPE_HARD_EXCEPTION;
+		if (ex->nested && is_fred_enabled(vcpu))
+			intr_info |= INTR_INFO_NESTED_EXCEPTION_MASK;
+	}
 
 	vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr_info);
 
@@ -7348,6 +7351,7 @@ static void __vmx_complete_interrupts(struct kvm_vcpu *vcpu,
 		kvm_requeue_exception(vcpu, vector,
 				      idt_vectoring_info & VECTORING_INFO_DELIVER_CODE_MASK,
 				      error_code,
+				      idt_vectoring_info & VECTORING_INFO_NESTED_EXCEPTION_MASK,
 				      event_data);
 		break;
 	}
arch/x86/kvm/x86.c +12 -1
index 10f1663d51d7..554442c07f27 100644
@@ -879,6 +879,10 @@ static void kvm_multiple_exception(struct kvm_vcpu *vcpu, unsigned int nr,
 		vcpu->arch.exception.pending = true;
 		vcpu->arch.exception.injected = false;
 
+		vcpu->arch.exception.nested = vcpu->arch.exception.nested ||
+					      vcpu->arch.nmi_injected ||
+					      vcpu->arch.interrupt.injected;
+
 		vcpu->arch.exception.has_error_code = has_error;
 		vcpu->arch.exception.vector = nr;
 		vcpu->arch.exception.error_code = error_code;
@@ -908,8 +912,13 @@ static void kvm_multiple_exception(struct kvm_vcpu *vcpu, unsigned int nr,
 		vcpu->arch.exception.injected = false;
 		vcpu->arch.exception.pending = false;
 
+		/* #DF is NOT a nested event, per its definition. */
+		vcpu->arch.exception.nested = false;
+
 		kvm_queue_exception_e(vcpu, DF_VECTOR, 0);
 	} else {
+		vcpu->arch.exception.nested = true;
+
 		/* replace previous exception with a new one in a hope
 		   that instruction re-execution will regenerate lost
 		   exception */
@@ -938,7 +947,8 @@ static void kvm_queue_exception_e_p(struct kvm_vcpu *vcpu, unsigned nr,
 }
 
 void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr,
-			   bool has_error_code, u32 error_code, u64 event_data)
+			   bool has_error_code, u32 error_code, bool nested,
+			   u64 event_data)
 {
 
 	/*
@@ -963,6 +973,7 @@ void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr,
 	vcpu->arch.exception.error_code = error_code;
 	vcpu->arch.exception.has_payload = false;
 	vcpu->arch.exception.payload = 0;
+	vcpu->arch.exception.nested = nested;
 	vcpu->arch.exception.event_data = event_data;
 }
 EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_requeue_exception);
arch/x86/kvm/x86.h +1 -0
index 0c1fbf75442b..4f5d12d7136e 100644
@@ -198,6 +198,7 @@ static inline void kvm_clear_exception_queue(struct kvm_vcpu *vcpu)
 {
 	vcpu->arch.exception.pending = false;
 	vcpu->arch.exception.injected = false;
+	vcpu->arch.exception.nested = false;
 	vcpu->arch.exception_vmexit.pending = false;
 }
 
Signature · 1 line
2.51.0
Reviewed-by on v9 13/22
#58 lore raw

Re: [PATCH v9 13/22] KVM: VMX: Virtualize FRED nested exception tracking

Quoted 21 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Set the VMX nested exception bit in VM-entry interruption information field when injecting a nested exception using FRED event delivery to ensure:
1) A nested exception is injected on a correct stack level.
2) The nested bit defined in FRED stack frame is set.

The event stack level used by FRED event delivery depends on whether the event was a nested exception encountered during delivery of an earlier event, because a nested exception is "regarded" as happening on ring 0. E.g., when #PF is configured to use stack level 1 in IA32_FRED_STKLVLS MSR:
- nested #PF will be delivered on the stack pointed by IA32_FRED_RSP1 MSR when encountered in ring 3 and ring 0.
- normal #PF will be delivered on the stack pointed by IA32_FRED_RSP0 MSR when encountered in ring 3.

The VMX nested-exception support ensures a correct event stack level is chosen when a VM entry injects a nested exception.

Quoted 6 lines
[ Sean: reworked kvm_requeue_exception() to simply the code changes ]
#59 lore raw

Re: [PATCH v9 13/22] KVM: VMX: Virtualize FRED nested exception tracking

In reply to Xin Li (Intel)
Quoted 21 lines · Xin Li (Intel)
 arch/x86/include/asm/kvm_host.h |  4 +++-
 arch/x86/include/asm/vmx.h      |  5 ++++-
 arch/x86/kvm/svm/svm.c          |  2 +-
 arch/x86/kvm/vmx/vmx.c          |  6 +++++-
 arch/x86/kvm/x86.c              | 13 ++++++++++++-
 arch/x86/kvm/x86.h              |  1 +
 6 files changed, 26 insertions(+), 5 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index 550a8716a227..3b6dadf368eb 100644
@@ -760,6 +760,7 @@ struct kvm_queued_exception {
 	u32 error_code;
 	unsigned long payload;
 	bool has_payload;
+	bool nested;
 	u64 event_data;
 };
 

arch/x86/include/asm/kvm_host.h void kvm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr); · line 2235

Quoted 5 lines
arch/x86/include/asm/kvm_host.h +1 -1
@@ -2231,7 +2232,8 @@ void kvm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr);
 void kvm_queue_exception_e(struct kvm_vcpu *vcpu, unsigned nr, u32 error_code);
 void kvm_queue_exception_p(struct kvm_vcpu *vcpu, unsigned nr, unsigned long payload);
 void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr,
-			   bool has_error_code, u32 error_code, u64 event_data);
+			   bool has_error_code, u32 error_code, bool nested,

I think we should pick a different name, as both VMX and SVM declare "nested" as a global boolean. I.e. this creates some nasty variable shadowing.

Maybe is_nested?

author maintainer
#60 lore raw

Re: [PATCH v9 13/22] KVM: VMX: Virtualize FRED nested exception tracking

Quoted 14 lines · Sean Christopherson

arch/x86/include/asm/kvm_host.h void kvm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr); · line 2235

Quoted 7 lines
arch/x86/include/asm/kvm_host.h +1 -1
@@ -2231,7 +2232,8 @@ void kvm_queue_exception(struct kvm_vcpu *vcpu, unsigned nr);
 void kvm_queue_exception_e(struct kvm_vcpu *vcpu, unsigned nr, u32 error_code);
 void kvm_queue_exception_p(struct kvm_vcpu *vcpu, unsigned nr, unsigned long payload);
 void kvm_requeue_exception(struct kvm_vcpu *vcpu, unsigned int nr,
-    bool has_error_code, u32 error_code, u64 event_data);
+    bool has_error_code, u32 error_code, bool nested,

I think we should pick a different name, as both VMX and SVM declare "nested" as a global boolean. I.e. this creates some nasty variable shadowing.

Quoted 1 line

Maybe is_nested?

is_nested looks good to me.

I thought about is_nested_exp, however the function names already contain “exception”, no point to duplicate it.

author maintainer
#61 lore raw

PATCH v9 14/22 KVM: x86: Save/restore the nested flag of an exception

In reply to Xin Li (Intel)

Save/restore the nested flag of an exception during VM save/restore and live migration to ensure a correct event stack level is chosen when a nested exception is injected through FRED event delivery.

---

Change in v8:
* Update KVM_CAP_EXCEPTION_NESTED_FLAG, as the number in v7 is used by another new cap.

Change in v5:
* Add TB from Xuelian Guo.

Change in v4:
* Add live migration support for exception nested flag (Chao Gao).

---
 Documentation/virt/kvm/api.rst  | 21 ++++++++++++++++++++-
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/include/uapi/asm/kvm.h |  4 +++-
 arch/x86/kvm/x86.c              | 19 ++++++++++++++++++-
 include/uapi/linux/kvm.h        |  1 +
 5 files changed, 43 insertions(+), 3 deletions(-)
Documentation/virt/kvm/api.rst +20 -1
index 06d79e2cf7bf..4e9678adf661 100644
@@ -1184,6 +1184,10 @@ The following bits are defined in the flags field:
   fields contain a valid state. This bit will be set whenever
   KVM_CAP_EXCEPTION_PAYLOAD is enabled.
 
+- KVM_VCPUEVENT_VALID_NESTED_FLAG may be set to inform that the
+  exception is a nested exception. This bit will be set whenever
+  KVM_CAP_EXCEPTION_NESTED_FLAG is enabled.
+
 - KVM_VCPUEVENT_VALID_TRIPLE_FAULT may be set to signal that the
   triple_fault_pending field contains a valid state. This bit will
   be set whenever KVM_CAP_X86_TRIPLE_FAULT_EVENT is enabled.
@@ -1286,6 +1290,10 @@ can be set in the flags field to signal that the
 exception_has_payload, exception_payload, and exception.pending fields
 contain a valid state and shall be written into the VCPU.
 
+If KVM_CAP_EXCEPTION_NESTED_FLAG is enabled, KVM_VCPUEVENT_VALID_NESTED_FLAG
+can be set in the flags field to inform that the exception is a nested
+exception and exception_is_nested shall be written into the VCPU.
+
 If KVM_CAP_X86_TRIPLE_FAULT_EVENT is enabled, KVM_VCPUEVENT_VALID_TRIPLE_FAULT
 can be set in flags field to signal that the triple_fault field contains
 a valid state and shall be written into the VCPU.
@@ -8692,7 +8700,7 @@ given VM.
 When this capability is enabled, KVM resets the VCPU when setting
 MP_STATE_INIT_RECEIVED through IOCTL.  The original MP_STATE is preserved.
 
-7.43 KVM_CAP_ARM_CACHEABLE_PFNMAP_SUPPORTED
+7.44 KVM_CAP_ARM_CACHEABLE_PFNMAP_SUPPORTED
 -------------------------------------------
 
 :Architectures: arm64
@@ -8703,6 +8711,17 @@ This capability indicate to the userspace whether a PFNMAP memory region
 can be safely mapped as cacheable. This relies on the presence of
 force write back (FWB) feature support on the hardware.
 
+7.45 KVM_CAP_EXCEPTION_NESTED_FLAG
+----------------------------------
+
+:Architectures: x86
+:Parameters: args[0] whether feature should be enabled or not
+
+With this capability enabled, an exception is save/restored with the
+additional information of whether it was nested or not. FRED event
+delivery uses this information to ensure a correct event stack level
+is chosen when a VM entry injects a nested exception.
+
 8. Other capabilities.
 ======================
 
arch/x86/include/asm/kvm_host.h +1 -0
index 3b6dadf368eb..5fff22d837aa 100644
@@ -1491,6 +1491,7 @@ struct kvm_arch {
 	bool has_mapped_host_mmio;
 	bool guest_can_read_msr_platform_info;
 	bool exception_payload_enabled;
+	bool exception_nested_flag_enabled;
 
 	bool triple_fault_event;
 
arch/x86/include/uapi/asm/kvm.h +3 -1
index d420c9c066d4..fbeeea236fc2 100644
@@ -331,6 +331,7 @@ struct kvm_reinject_control {
 #define KVM_VCPUEVENT_VALID_SMM		0x00000008
 #define KVM_VCPUEVENT_VALID_PAYLOAD	0x00000010
 #define KVM_VCPUEVENT_VALID_TRIPLE_FAULT	0x00000020
+#define KVM_VCPUEVENT_VALID_NESTED_FLAG	0x00000040
 
 /* Interrupt shadow states */
 #define KVM_X86_SHADOW_INT_MOV_SS	0x01
@@ -368,7 +369,8 @@ struct kvm_vcpu_events {
 	struct {
 		__u8 pending;
 	} triple_fault;
-	__u8 reserved[26];
+	__u8 reserved[25];
+	__u8 exception_is_nested;
 	__u8 exception_has_payload;
 	__u64 exception_payload;
 };
arch/x86/kvm/x86.c +18 -1
index 554442c07f27..6762f5564341 100644
@@ -4968,6 +4968,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_GET_MSR_FEATURES:
 	case KVM_CAP_MSR_PLATFORM_INFO:
 	case KVM_CAP_EXCEPTION_PAYLOAD:
+	case KVM_CAP_EXCEPTION_NESTED_FLAG:
 	case KVM_CAP_X86_TRIPLE_FAULT_EVENT:
 	case KVM_CAP_SET_GUEST_DEBUG:
 	case KVM_CAP_LAST_CPU:
@@ -5713,6 +5714,7 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu *vcpu,
 	events->exception.error_code = ex->error_code;
 	events->exception_has_payload = ex->has_payload;
 	events->exception_payload = ex->payload;
+	events->exception_is_nested = ex->nested;
 
 	events->interrupt.injected =
 		vcpu->arch.interrupt.injected && !vcpu->arch.interrupt.soft;
@@ -5738,6 +5740,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu *vcpu,
 			 | KVM_VCPUEVENT_VALID_SMM);
 	if (vcpu->kvm->arch.exception_payload_enabled)
 		events->flags |= KVM_VCPUEVENT_VALID_PAYLOAD;
+	if (vcpu->kvm->arch.exception_nested_flag_enabled)
+		events->flags |= KVM_VCPUEVENT_VALID_NESTED_FLAG;
 	if (vcpu->kvm->arch.triple_fault_event) {
 		events->triple_fault.pending = kvm_test_request(KVM_REQ_TRIPLE_FAULT, vcpu);
 		events->flags |= KVM_VCPUEVENT_VALID_TRIPLE_FAULT;
@@ -5752,7 +5756,8 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu,
 			      | KVM_VCPUEVENT_VALID_SHADOW
 			      | KVM_VCPUEVENT_VALID_SMM
 			      | KVM_VCPUEVENT_VALID_PAYLOAD
-			      | KVM_VCPUEVENT_VALID_TRIPLE_FAULT))
+			      | KVM_VCPUEVENT_VALID_TRIPLE_FAULT
+			      | KVM_VCPUEVENT_VALID_NESTED_FLAG))
 		return -EINVAL;
 
 	if (events->flags & KVM_VCPUEVENT_VALID_PAYLOAD) {
@@ -5767,6 +5772,13 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu,
 		events->exception_has_payload = 0;
 	}
 
+	if (events->flags & KVM_VCPUEVENT_VALID_NESTED_FLAG) {
+		if (!vcpu->kvm->arch.exception_nested_flag_enabled)
+			return -EINVAL;
+	} else {
+		events->exception_is_nested = 0;
+	}
+
 	if ((events->exception.injected || events->exception.pending) &&
 	    (events->exception.nr > 31 || events->exception.nr == NMI_VECTOR))
 		return -EINVAL;
@@ -5792,6 +5804,7 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu,
 	vcpu->arch.exception.error_code = events->exception.error_code;
 	vcpu->arch.exception.has_payload = events->exception_has_payload;
 	vcpu->arch.exception.payload = events->exception_payload;
+	vcpu->arch.exception.nested = events->exception_is_nested;
 
 	vcpu->arch.interrupt.injected = events->interrupt.injected;
 	vcpu->arch.interrupt.nr = events->interrupt.nr;
@@ -6912,6 +6925,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		kvm->arch.exception_payload_enabled = cap->args[0];
 		r = 0;
 		break;
+	case KVM_CAP_EXCEPTION_NESTED_FLAG:
+		kvm->arch.exception_nested_flag_enabled = cap->args[0];
+		r = 0;
+		break;
 	case KVM_CAP_X86_TRIPLE_FAULT_EVENT:
 		kvm->arch.triple_fault_event = cap->args[0];
 		r = 0;
include/uapi/linux/kvm.h +1 -0
index 52f6000ab020..ec3cc37b9373 100644
@@ -963,6 +963,7 @@ struct kvm_enable_cap {
 #define KVM_CAP_RISCV_MP_STATE_RESET 242
 #define KVM_CAP_ARM_CACHEABLE_PFNMAP_SUPPORTED 243
 #define KVM_CAP_GUEST_MEMFD_FLAGS 244
+#define KVM_CAP_EXCEPTION_NESTED_FLAG 245
 
 struct kvm_irq_routing_irqchip {
 	__u32 irqchip;
Signature · 1 line
2.51.0
Reviewed-by on v9 14/22
#62 lore raw

Re: [PATCH v9 14/22] KVM: x86: Save/restore the nested flag of an exception

Quoted 6 lines · Xin Li (Intel)

Save/restore the nested flag of an exception during VM save/restore and live migration to ensure a correct event stack level is chosen when a nested exception is injected through FRED event delivery.

author maintainer
#63 lore raw

PATCH v9 15/22 KVM: x86: Mark CR4.FRED as not reserved

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

The CR4.FRED bit, i.e., CR4[32], is no longer a reserved bit when guest cpu cap has FRED, i.e.,
1) All of FRED KVM support is in place.
2) Guest enumerates FRED.

Otherwise it is still a reserved bit.

---

Change in v5:
* Add TB from Xuelian Guo.

Change in v4:
* Rebase on top of "guest_cpu_cap".

Change in v3:
* Don't allow CR4.FRED=1 before all of FRED KVM support is in place (Sean Christopherson).

---
 arch/x86/include/asm/kvm_host.h | 2 +-
 arch/x86/kvm/x86.h              | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -1
index 5fff22d837aa..558f260a1afd 100644
@@ -142,7 +142,7 @@
 			  | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \
 			  | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \
 			  | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP \
-			  | X86_CR4_LAM_SUP | X86_CR4_CET))
+			  | X86_CR4_LAM_SUP | X86_CR4_CET | X86_CR4_FRED))
 
 #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR)
 
arch/x86/kvm/x86.h +2 -0
index 4f5d12d7136e..e9c6f304b02e 100644
@@ -687,6 +687,8 @@ static inline bool __kvm_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
 	if (!__cpu_has(__c, X86_FEATURE_SHSTK) &&       \
 	    !__cpu_has(__c, X86_FEATURE_IBT))           \
 		__reserved_bits |= X86_CR4_CET;         \
+	if (!__cpu_has(__c, X86_FEATURE_FRED))          \
+		__reserved_bits |= X86_CR4_FRED;        \
 	__reserved_bits;                                \
 })
 
Signature · 1 line
2.51.0
Reviewed-by on v9 15/22
#64 lore raw

Re: [PATCH v9 15/22] KVM: x86: Mark CR4.FRED as not reserved

Quoted 7 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

The CR4.FRED bit, i.e., CR4[32], is no longer a reserved bit when guest cpu cap has FRED, i.e.,
1) All of FRED KVM support is in place.
2) Guest enumerates FRED.

Quoted 6 lines

Otherwise it is still a reserved bit.

I am not sure about two things regarding CR4.FRED and emulator code:

1. Should kvm_set_cr4() reject setting CR4.FRED when the vCPU isn't in long mode? The concern is that emulator code may call kvm_set_cr4(). This could cause VM-entry failure if CR4.FRED is set in other modes.

2. mk_cr_64() drops the high 32 bits of the new CR4 value. So, CR4.FRED is always dropped. This may need an update.

This patch itself looks good, so:

Quoted 45 lines

---

Change in v5:
* Add TB from Xuelian Guo.

Change in v4:
* Rebase on top of "guest_cpu_cap".

Change in v3:
* Don't allow CR4.FRED=1 before all of FRED KVM support is in place (Sean Christopherson).
---
arch/x86/include/asm/kvm_host.h | 2 +-
arch/x86/kvm/x86.h | 2 ++

2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -1
index 5fff22d837aa..558f260a1afd 100644
@@ -142,7 +142,7 @@
 			  | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \
 			  | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \
 			  | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP \
-			  | X86_CR4_LAM_SUP | X86_CR4_CET))
+			  | X86_CR4_LAM_SUP | X86_CR4_CET | X86_CR4_FRED))
 
 #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR)
 
arch/x86/kvm/x86.h +2 -0
index 4f5d12d7136e..e9c6f304b02e 100644
@@ -687,6 +687,8 @@ static inline bool __kvm_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
 	if (!__cpu_has(__c, X86_FEATURE_SHSTK) &&       \
 	    !__cpu_has(__c, X86_FEATURE_IBT))           \
 		__reserved_bits |= X86_CR4_CET;         \
+	if (!__cpu_has(__c, X86_FEATURE_FRED))          \
+		__reserved_bits |= X86_CR4_FRED;        \
 	__reserved_bits;                                \
 })
 
Signature · 1 line
2.51.0
#65 lore raw

Re: [PATCH v9 15/22] KVM: x86: Mark CR4.FRED as not reserved

Quoted 15 lines · Chao Gao
Quoted 7 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

The CR4.FRED bit, i.e., CR4[32], is no longer a reserved bit when guest cpu cap has FRED, i.e.,
1) All of FRED KVM support is in place.
2) Guest enumerates FRED.

Quoted 6 lines

Otherwise it is still a reserved bit.

Quoted 5 lines

I am not sure about two things regarding CR4.FRED and emulator code:

1. Should kvm_set_cr4() reject setting CR4.FRED when the vCPU isn't in long mode? The concern is that emulator code may call kvm_set_cr4(). This could cause VM-entry failure if CR4.FRED is set in other modes.

This has nothing to do with the emulator, KVM will intercept and emulate all CR4 writes that toggle CR4.FRED. KVM also needs to enforce leaving 64-bit mode with CR4.FRED=1.

Quoted 2 lines

2. mk_cr_64() drops the high 32 bits of the new CR4 value. So, CR4.FRED is always dropped. This may need an update.

Ugh, I didn't realize FRED broke into bits 63:32. Yeah, that needs to be updated, and _that_ one is unique to the emulator.

Unless Chao and I can't read code and are missing magic, KVM's virtualization of FRED is quite lacking.

More importantly, I don't see *any* tests. At a bare minimum, KVM's msrs_test needs to be updated too get coverage for userspace vs. guest accesses, save/restore needs to be covered (maybe nothing additional required?), and there need to be negative tests for things like leaving 64-bit mode with FRED=1. We can probably get enough confidence in the "happy" paths just by running VMs, but even then I would ideally like to see tests for edge cases that are relatively rare when just running a VM.

I'm straight up not going to look at new versions if there aren't tests. Like CET before it, both Intel and AMD are pushing FRED and want to get it merged, yet no one is providing tests. That's not going to fly this time, as I don't have the bandwidth to help write the number of testcases FRED warrants.

author maintainer
#66 lore raw

Re: [PATCH v9 15/22] KVM: x86: Mark CR4.FRED as not reserved

Quoted 18 lines
Quoted 3 lines

2. mk_cr_64() drops the high 32 bits of the new CR4 value. So, CR4.FRED is always dropped. This may need an update.

Ugh, I didn't realize FRED broke into bits 63:32. Yeah, that needs to be updated, and _that_ one is unique to the emulator.

Unless Chao and I can't read code and are missing magic, KVM's virtualization of FRED is quite lacking.

More importantly, I don't see *any* tests. At a bare minimum, KVM's msrs_test needs to be updated too get coverage for userspace vs. guest accesses, save/restore needs to be covered (maybe nothing additional required?), and there need to be negative tests for things like leaving 64-bit mode with FRED=1. We can probably get enough confidence in the "happy" paths just by running VMs, but even then I would ideally like to see tests for edge cases that are relatively rare when just running a VM.

Quoted 4 lines

I'm straight up not going to look at new versions if there aren't tests. Like CET before it, both Intel and AMD are pushing FRED and want to get it merged, yet no one is providing tests. That's not going to fly this time, as I don't have the bandwidth to help write the number of testcases FRED warrants.

I must admit the issues Chao raised were a clear oversight on my part.

I wrote some basic functionality unit tests in kernel selftests, which were included in v1 and v2.

Later I started to create FRED tests in kvm-unit-tests and extended one nested test case to CET:

https://lore.kernel.org/kvm/[email protected]/

I planned to send out these new kvm unit tests (not just FRED tests) after KVM FRED patch series gets merged.

#67 lore raw

Re: [PATCH v9 15/22] KVM: x86: Mark CR4.FRED as not reserved

Quoted 28 lines · Xin Li
Quoted 18 lines
Quoted 3 lines

2. mk_cr_64() drops the high 32 bits of the new CR4 value. So, CR4.FRED is always dropped. This may need an update.

Ugh, I didn't realize FRED broke into bits 63:32. Yeah, that needs to be updated, and _that_ one is unique to the emulator.

Unless Chao and I can't read code and are missing magic, KVM's virtualization of FRED is quite lacking.

More importantly, I don't see *any* tests. At a bare minimum, KVM's msrs_test needs to be updated too get coverage for userspace vs. guest accesses, save/restore needs to be covered (maybe nothing additional required?), and there need to be negative tests for things like leaving 64-bit mode with FRED=1. We can probably get enough confidence in the "happy" paths just by running VMs, but even then I would ideally like to see tests for edge cases that are relatively rare when just running a VM.

Quoted 4 lines

I'm straight up not going to look at new versions if there aren't tests. Like CET before it, both Intel and AMD are pushing FRED and want to get it merged, yet no one is providing tests. That's not going to fly this time, as I don't have the bandwidth to help write the number of testcases FRED warrants.

I must admit the issues Chao raised were a clear oversight on my part.

I wrote some basic functionality unit tests in kernel selftests, which were included in v1 and v2.

Quoted 7 lines

Later I started to create FRED tests in kvm-unit-tests and extended one nested test case to CET:

https://lore.kernel.org/kvm/[email protected]/

I planned to send out these new kvm unit tests (not just FRED tests) after KVM FRED patch series gets merged.

Definitely send them before. Even if no one outside of Intel can run them (I forget when FRED hardware is coming), people like me can at least see what you're testing, which makes it more likely that we'll spot bugs. E.g. in this case, the lack of negative tests for CR4.FRED is a big red flag.

author maintainer
#68 lore raw

Re: [PATCH v9 15/22] KVM: x86: Mark CR4.FRED as not reserved

Quoted 8 lines
Quoted 3 lines

I planned to send out these new kvm unit tests (not just FRED tests) after KVM FRED patch series gets merged.

Definitely send them before. Even if no one outside of Intel can run them (I forget when FRED hardware is coming), people like me can at least see what you're testing, which makes it more likely that we'll spot bugs. E.g. in this case, the lack of negative tests for CR4.FRED is a big red flag.

No excuse, sadly mk_cr_64() completely escaped my notice.

And I didn’t even think about adding a CR4 test for FRED.

And as you expected, the happy path doesn’t expose it at all.

author maintainer
#69 lore raw

Re: [PATCH v9 15/22] KVM: x86: Mark CR4.FRED as not reserved

Quoted 8 lines · Xin Li
Quoted 5 lines
Quoted 4 lines

I planned to send out these new kvm unit tests (not just FRED tests) after KVM FRED patch series gets merged.

Quoted 6 lines
Quoted 4 lines

Definitely send them before. Even if no one outside of Intel can run them (I forget when FRED hardware is coming), people like me can at least see what you're testing, which makes it more likely that we'll spot bugs. E.g. in this case, the lack of negative tests for CR4.FRED is a big red flag.

No excuse, sadly mk_cr_64() completely escaped my notice.

BTW, mk_cr_64() looks a bad name to me, it’s more like mk_cr_32().

Quoted 7 lines

And I didn’t even think about adding a CR4 test for FRED.

And as you expected, the happy path doesn’t expose it at all.

#70 lore raw

Re: [PATCH v9 15/22] KVM: x86: Mark CR4.FRED as not reserved

In reply to Sean Christopherson
Quoted 16 lines · Sean Christopherson
Quoted 15 lines · Chao Gao
Quoted 13 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

The CR4.FRED bit, i.e., CR4[32], is no longer a reserved bit when guest cpu cap has FRED, i.e.,
1) All of FRED KVM support is in place.
2) Guest enumerates FRED.

Otherwise it is still a reserved bit.

Quoted 8 lines
Quoted 5 lines

I am not sure about two things regarding CR4.FRED and emulator code:

1. Should kvm_set_cr4() reject setting CR4.FRED when the vCPU isn't in long mode? The concern is that emulator code may call kvm_set_cr4(). This could cause VM-entry failure if CR4.FRED is set in other modes.

This has nothing to do with the emulator, KVM will intercept and emulate all CR4 writes that toggle CR4.FRED.

I think I misanalyzed this.

For normal (non-emulator) code paths, kvm_register_read() drops the high-32 bits which ensures CR4.FRED won't be set outside 64-bit mode.

Looking at the emulator code again, the Op3264 flag and fetch_register_operand() will also prevent CR4.FRED from being set outside 64-bit mode.

The only scenario where CR4.FRED could be set in non-64-bit mode is through userspace VMM's CR4 writes. But, KVM currently allows userspace to write invalid values anyway, so this isn't a problem.

Quoted 2 lines

KVM also needs to enforce leaving 64-bit mode
with CR4.FRED=1.

Yes.

author maintainer
#71 lore raw

PATCH v9 16/22 KVM: VMX: Dump FRED context in dump_vmcs()

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Add FRED related VMCS fields to dump_vmcs() to dump FRED context.

---

Changes in v5:
* Read guest FRED RSP0 with vmx_read_guest_fred_rsp0() (Sean).
* Add TB from Xuelian Guo.

Change in v3:
* Use (vmentry_ctrl & VM_ENTRY_LOAD_IA32_FRED) instead of is_fred_enabled() (Chao Gao).

Changes in v2:
* Use kvm_cpu_cap_has() instead of cpu_feature_enabled() (Chao Gao).
* Dump guest FRED states only if guest has FRED enabled (Nikolay Borisov).

---
 arch/x86/kvm/vmx/vmx.c | 43 +++++++++++++++++++++++++++++++++++-------
 1 file changed, 36 insertions(+), 7 deletions(-)
arch/x86/kvm/vmx/vmx.c +36 -7
index 34e057f65513..04442f869abb 100644
@@ -1398,6 +1398,9 @@ static void vmx_write_guest_fred_rsp0(struct vcpu_vmx *vmx, u64 data)
 	vmx_write_guest_host_msr(vmx, MSR_IA32_FRED_RSP0, data,
 				 &vmx->msr_guest_fred_rsp0);
 }
+#else
+/* Make sure it builds on 32-bit */
+static u64 vmx_read_guest_fred_rsp0(struct vcpu_vmx *vmx) { return 0; }
 #endif
 
 static void grow_ple_window(struct kvm_vcpu *vcpu)
@@ -6460,7 +6463,7 @@ void dump_vmcs(struct kvm_vcpu *vcpu)
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
 	u32 vmentry_ctl, vmexit_ctl;
 	u32 cpu_based_exec_ctrl, pin_based_exec_ctrl, secondary_exec_control;
-	u64 tertiary_exec_control;
+	u64 tertiary_exec_control, secondary_vmexit_ctl;
 	unsigned long cr4;
 	int efer_slot;
 
@@ -6471,6 +6474,8 @@ void dump_vmcs(struct kvm_vcpu *vcpu)
 
 	vmentry_ctl = vmcs_read32(VM_ENTRY_CONTROLS);
 	vmexit_ctl = vmcs_read32(VM_EXIT_CONTROLS);
+	secondary_vmexit_ctl = cpu_has_secondary_vmexit_ctrls() ?
+			       vmcs_read64(SECONDARY_VM_EXIT_CONTROLS) : 0;
 	cpu_based_exec_ctrl = vmcs_read32(CPU_BASED_VM_EXEC_CONTROL);
 	pin_based_exec_ctrl = vmcs_read32(PIN_BASED_VM_EXEC_CONTROL);
 	cr4 = vmcs_readl(GUEST_CR4);
@@ -6517,6 +6522,16 @@ void dump_vmcs(struct kvm_vcpu *vcpu)
 	vmx_dump_sel("LDTR:", GUEST_LDTR_SELECTOR);
 	vmx_dump_dtsel("IDTR:", GUEST_IDTR_LIMIT);
 	vmx_dump_sel("TR:  ", GUEST_TR_SELECTOR);
+	if (vmentry_ctl & VM_ENTRY_LOAD_IA32_FRED)
+		pr_err("FRED guest: config=0x%016llx, stack_levels=0x%016llx\n"
+		       "RSP0=0x%016llx, RSP1=0x%016llx\n"
+		       "RSP2=0x%016llx, RSP3=0x%016llx\n",
+		       vmcs_read64(GUEST_IA32_FRED_CONFIG),
+		       vmcs_read64(GUEST_IA32_FRED_STKLVLS),
+		       vmx_read_guest_fred_rsp0(vmx),
+		       vmcs_read64(GUEST_IA32_FRED_RSP1),
+		       vmcs_read64(GUEST_IA32_FRED_RSP2),
+		       vmcs_read64(GUEST_IA32_FRED_RSP3));
 	efer_slot = vmx_find_loadstore_msr_slot(&vmx->msr_autoload.guest, MSR_EFER);
 	if (vmentry_ctl & VM_ENTRY_LOAD_IA32_EFER)
 		pr_err("EFER= 0x%016llx\n", vmcs_read64(GUEST_IA32_EFER));
@@ -6568,6 +6583,16 @@ void dump_vmcs(struct kvm_vcpu *vcpu)
 	       vmcs_readl(HOST_TR_BASE));
 	pr_err("GDTBase=%016lx IDTBase=%016lx\n",
 	       vmcs_readl(HOST_GDTR_BASE), vmcs_readl(HOST_IDTR_BASE));
+	if (vmexit_ctl & SECONDARY_VM_EXIT_LOAD_IA32_FRED)
+		pr_err("FRED host: config=0x%016llx, stack_levels=0x%016llx\n"
+		       "RSP0=0x%016lx, RSP1=0x%016llx\n"
+		       "RSP2=0x%016llx, RSP3=0x%016llx\n",
+		       vmcs_read64(HOST_IA32_FRED_CONFIG),
+		       vmcs_read64(HOST_IA32_FRED_STKLVLS),
+		       (unsigned long)task_stack_page(current) + THREAD_SIZE,
+		       vmcs_read64(HOST_IA32_FRED_RSP1),
+		       vmcs_read64(HOST_IA32_FRED_RSP2),
+		       vmcs_read64(HOST_IA32_FRED_RSP3));
 	pr_err("CR0=%016lx CR3=%016lx CR4=%016lx\n",
 	       vmcs_readl(HOST_CR0), vmcs_readl(HOST_CR3),
 	       vmcs_readl(HOST_CR4));
@@ -6593,25 +6618,29 @@ void dump_vmcs(struct kvm_vcpu *vcpu)
 	pr_err("*** Control State ***\n");
 	pr_err("CPUBased=0x%08x SecondaryExec=0x%08x TertiaryExec=0x%016llx\n",
 	       cpu_based_exec_ctrl, secondary_exec_control, tertiary_exec_control);
-	pr_err("PinBased=0x%08x EntryControls=%08x ExitControls=%08x\n",
-	       pin_based_exec_ctrl, vmentry_ctl, vmexit_ctl);
+	pr_err("PinBased=0x%08x EntryControls=0x%08x\n",
+	       pin_based_exec_ctrl, vmentry_ctl);
+	pr_err("ExitControls=0x%08x SecondaryExitControls=0x%016llx\n",
+	       vmexit_ctl, secondary_vmexit_ctl);
 	pr_err("ExceptionBitmap=%08x PFECmask=%08x PFECmatch=%08x\n",
 	       vmcs_read32(EXCEPTION_BITMAP),
 	       vmcs_read32(PAGE_FAULT_ERROR_CODE_MASK),
 	       vmcs_read32(PAGE_FAULT_ERROR_CODE_MATCH));
-	pr_err("VMEntry: intr_info=%08x errcode=%08x ilen=%08x\n",
+	pr_err("VMEntry: intr_info=%08x errcode=%08x ilen=%08x event_data=%016llx\n",
 	       vmcs_read32(VM_ENTRY_INTR_INFO_FIELD),
 	       vmcs_read32(VM_ENTRY_EXCEPTION_ERROR_CODE),
-	       vmcs_read32(VM_ENTRY_INSTRUCTION_LEN));
+	       vmcs_read32(VM_ENTRY_INSTRUCTION_LEN),
+	       kvm_cpu_cap_has(X86_FEATURE_FRED) ? vmcs_read64(INJECTED_EVENT_DATA) : 0);
 	pr_err("VMExit: intr_info=%08x errcode=%08x ilen=%08x\n",
 	       vmcs_read32(VM_EXIT_INTR_INFO),
 	       vmcs_read32(VM_EXIT_INTR_ERROR_CODE),
 	       vmcs_read32(VM_EXIT_INSTRUCTION_LEN));
 	pr_err("        reason=%08x qualification=%016lx\n",
 	       vmcs_read32(VM_EXIT_REASON), vmcs_readl(EXIT_QUALIFICATION));
-	pr_err("IDTVectoring: info=%08x errcode=%08x\n",
+	pr_err("IDTVectoring: info=%08x errcode=%08x event_data=%016llx\n",
 	       vmcs_read32(IDT_VECTORING_INFO_FIELD),
-	       vmcs_read32(IDT_VECTORING_ERROR_CODE));
+	       vmcs_read32(IDT_VECTORING_ERROR_CODE),
+	       kvm_cpu_cap_has(X86_FEATURE_FRED) ? vmcs_read64(ORIGINAL_EVENT_DATA) : 0);
 	pr_err("TSC Offset = 0x%016llx\n", vmcs_read64(TSC_OFFSET));
 	if (secondary_exec_control & SECONDARY_EXEC_TSC_SCALING)
 		pr_err("TSC Multiplier = 0x%016llx\n",
Signature · 1 line
2.51.0
#72 lore raw

Re: [PATCH v9 16/22] KVM: VMX: Dump FRED context in dump_vmcs()

Quoted 3 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Add FRED related VMCS fields to dump_vmcs() to dump FRED context.

Why are SSPx not dumped?

author maintainer
#73 lore raw

Re: [PATCH v9 16/22] KVM: VMX: Dump FRED context in dump_vmcs()

Quoted 4 lines
Quoted 2 lines

Add FRED related VMCS fields to dump_vmcs() to dump FRED context.

Why are SSPx not dumped?

Good eye!

It needs extra logic to extract FRED_SSP0, and I’m a bit lazy to do it now ;)

author maintainer
#74 lore raw

PATCH v9 17/22 KVM: x86: Advertise support for FRED

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Advertise support for FRED to userspace after changes required to enable FRED in a KVM guest are in place.

---

Change in v5:
* Don't advertise FRED/LKGS together, LKGS can be advertised as an independent feature (Sean).
* Add TB from Xuelian Guo.

---
 arch/x86/kvm/cpuid.c | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/cpuid.c +1 -0
index d563a948318b..0bf97b8a3216 100644
@@ -1014,6 +1014,7 @@ void kvm_set_cpu_caps(void)
 		F(FSRS),
 		F(FSRC),
 		F(WRMSRNS),
+		X86_64_F(FRED),
 		X86_64_F(LKGS),
 		F(AMX_FP16),
 		F(AVX_IFMA),
Signature · 1 line
2.51.0
Reviewed-by on v9 17/22
#75 lore raw

Re: [PATCH v9 17/22] KVM: x86: Advertise support for FRED

Quoted 4 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Advertise support for FRED to userspace after changes required to enable FRED in a KVM guest are in place.

I'm not sure if AMD CPUs support FRED, but just in case, we can clear FRED i.e., kvm_cpu_cap_clear(X86_FEATURE_FRED) in svm_set_cpu_caps().

With this fixed:

Quoted 30 lines

---

Change in v5:
* Don't advertise FRED/LKGS together, LKGS can be advertised as an independent feature (Sean).
* Add TB from Xuelian Guo.
---
arch/x86/kvm/cpuid.c | 1 +

1 file changed, 1 insertion(+)
arch/x86/kvm/cpuid.c +1 -0
index d563a948318b..0bf97b8a3216 100644
@@ -1014,6 +1014,7 @@ void kvm_set_cpu_caps(void)
 		F(FSRS),
 		F(FSRC),
 		F(WRMSRNS),
+		X86_64_F(FRED),
 		X86_64_F(LKGS),
 		F(AMX_FP16),
 		F(AVX_IFMA),
Signature · 1 line
2.51.0
#77 lore raw

Re: [PATCH v9 17/22] KVM: x86: Advertise support for FRED

Quoted 4 lines · Xin Li
Quoted 4 lines · Chao Gao
Quoted 8 lines
Quoted 2 lines

I'm not sure if AMD CPUs support FRED, but just in case, we can clear FRED i.e., kvm_cpu_cap_clear(X86_FEATURE_FRED) in svm_set_cpu_caps().

AMD will support FRED, with ISA level compatibility:

https://www.amd.com/en/blogs/2025/amd-and-intel-celebrate-first-anniversary-of-x86-ecosys.html

Thus we don’t need to clear the bit.

In this case, we need to clear FRED for AMD.

The concern is that before AMD's FRED KVM support is implemented, FRED will be exposed to userspace on AMD FRED-capable hardware. This may cause issues.

author maintainer
#78 lore raw

Re: [PATCH v9 17/22] KVM: x86: Advertise support for FRED

Quoted 14 lines · Chao Gao
Quoted 9 lines · Xin Li
Quoted 6 lines · Chao Gao

I'm not sure if AMD CPUs support FRED, but just in case, we can clear FRED i.e., kvm_cpu_cap_clear(X86_FEATURE_FRED) in svm_set_cpu_caps().

AMD will support FRED, with ISA level compatibility:

Quoted 8 lines
Quoted 3 lines

https://www.amd.com/en/blogs/2025/amd-and-intel-celebrate-first-anniversary-of-x86-ecosys.html

Thus we don’t need to clear the bit.

In this case, we need to clear FRED for AMD.

The concern is that before AMD's FRED KVM support is implemented, FRED will be exposed to userspace on AMD FRED-capable hardware. This may cause issues.

Hmm, I think it’s Qemu does that.

We have 2 filters, one in Qemu and one in KVM, only both are set a feature is enabled.

What I have missed?

#79 lore raw

Re: [PATCH v9 17/22] KVM: x86: Advertise support for FRED

Quoted 20 lines · Xin Li
Quoted 20 lines · Chao Gao
Quoted 4 lines · Xin Li
Quoted 4 lines · Chao Gao
Quoted 8 lines
Quoted 2 lines

I'm not sure if AMD CPUs support FRED, but just in case, we can clear FRED i.e., kvm_cpu_cap_clear(X86_FEATURE_FRED) in svm_set_cpu_caps().

AMD will support FRED, with ISA level compatibility:

https://www.amd.com/en/blogs/2025/amd-and-intel-celebrate-first-anniversary-of-x86-ecosys.html

Thus we don’t need to clear the bit.

In this case, we need to clear FRED for AMD.

Quoted 8 lines
Quoted 2 lines

The concern is that before AMD's FRED KVM support is implemented, FRED will be exposed to userspace on AMD FRED-capable hardware. This may cause issues.

Hmm, I think it’s Qemu does that.

We have 2 filters, one in Qemu and one in KVM, only both are set a feature is enabled.

What I have missed?

If a newer QEMU (with AMD's FRED support patch) + an older KVM (without AMD's FRED support, but KVM advertises it), it may cause issues.

I guess it's no safety issue for host though, AMD should also require some control bit(s) to be set to allow guests to use the feature.

I agree with Chao that it should be cleared for AMD before AMD's FRED KVM support is implemented.

#80 lore raw

Re: [PATCH v9 17/22] KVM: x86: Advertise support for FRED

Quoted 19 lines · Binbin Wu
Quoted 18 lines · Xin Li
Quoted 18 lines · Chao Gao
Quoted 4 lines · Xin Li
Quoted 4 lines · Chao Gao
Quoted 8 lines
Quoted 2 lines

I'm not sure if AMD CPUs support FRED, but just in case, we can clear FRED i.e., kvm_cpu_cap_clear(X86_FEATURE_FRED) in svm_set_cpu_caps().

AMD will support FRED, with ISA level compatibility:

https://www.amd.com/en/blogs/2025/amd-and-intel-celebrate-first-anniversary-of-x86-ecosys.html

Thus we don’t need to clear the bit.

In this case, we need to clear FRED for AMD.

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

The concern is that before AMD's FRED KVM support is implemented, FRED will be exposed to userspace on AMD FRED-capable hardware. This may cause issues.

Hmm, I think it’s Qemu does that.

We have 2 filters, one in Qemu and one in KVM, only both are set a feature is enabled.

What I have missed?

The userspace VMM, e.g. QEMU, is completely irrelevant. KVM must not advertise support for features it doesn't actually implement, and more importantly must not internally treat such features as supported.

Quoted 2 lines

If a newer QEMU (with AMD's FRED support patch) + an older KVM (without AMD's FRED support, but KVM advertises it), it may cause issues.

Yep.

Quoted 1 line

I guess it's no safety issue for host though,

Maybe. Without fully analyzing the SVM implementation for FRED and its interaction with KVM, I don't think we can confidently say that incorrectly treating FRED as supported is benign for the host. It's a moot point, I just want to emphasize how it important it is that KVM doesn't over-report features.

author maintainer
#81 lore raw

Re: [PATCH v9 17/22] KVM: x86: Advertise support for FRED

Quoted 9 lines · Sean Christopherson
Quoted 3 lines
Quoted 3 lines

What I have missed?

The userspace VMM, e.g. QEMU, is completely irrelevant. KVM must not advertise support for features it doesn't actually implement, and more importantly must not internally treat such features as supported.

Right, FRED virtualization is not supported on AMD and obviously SVM will need to add FRED virtualization in their own fashion later.

Quoted 4 lines
Quoted 3 lines

If a newer QEMU (with AMD's FRED support patch) + an older KVM (without AMD's FRED support, but KVM advertises it), it may cause issues.

Quoted 8 lines

Yep.

Quoted 1 line

I guess it's no safety issue for host though,

Maybe. Without fully analyzing the SVM implementation for FRED and its interaction with KVM, I don't think we can confidently say that incorrectly treating FRED as supported is benign for the host. It's a moot point, I just want to emphasize how it important it is that KVM doesn't over-report features.

I will remove it on AMD.

author maintainer
#82 lore raw

Re: [PATCH v9 17/22] KVM: x86: Advertise support for FRED

In reply to Binbin Wu
Quoted 12 lines
Quoted 11 lines
Quoted 5 lines

In this case, we need to clear FRED for AMD.

The concern is that before AMD's FRED KVM support is implemented, FRED will be exposed to userspace on AMD FRED-capable hardware. This may cause issues.

Hmm, I think it’s Qemu does that.

We have 2 filters, one in Qemu and one in KVM, only both are set a feature is enabled.

What I have missed?

Quoted 8 lines

If a newer QEMU (with AMD's FRED support patch) + an older KVM (without AMD's FRED support, but KVM advertises it), it may cause issues.

I guess it's no safety issue for host though, AMD should also require some control bit(s) to be set to allow guests to use the feature.

I agree with Chao that it should be cleared for AMD before AMD's FRED KVM support is implemented.

Damn, I somehow thought the host won’t report FRED on AMD...

author maintainer
#83 lore raw

PATCH v9 18/22 KVM: nVMX: Enable support for secondary VM exit controls

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Add support for secondary VM exit controls in nested VMX to facilitate future FRED integration.

---

Changes in v8:
* Relocate secondary_vm_exit_controls to the last u64 padding field.
* Remove the change to Documentation/virt/kvm/x86/nested-vmx.rst.

Changes in v5:
* Allow writing MSR_IA32_VMX_EXIT_CTLS2 (Sean).
* Add TB from Xuelian Guo.

Change in v3:
* Read secondary VM exit controls from vmcs_conf insteasd of the hardware MSR MSR_IA32_VMX_EXIT_CTLS2 to avoid advertising features to L1 that KVM itself doesn't support, e.g. because the expected entry+exit pairs aren't supported. (Sean Christopherson)

---
 arch/x86/kvm/vmx/capabilities.h |  1 +
 arch/x86/kvm/vmx/nested.c       | 26 +++++++++++++++++++++++++-
 arch/x86/kvm/vmx/vmcs12.c       |  1 +
 arch/x86/kvm/vmx/vmcs12.h       |  3 ++-
 arch/x86/kvm/x86.h              |  2 +-
 5 files changed, 30 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/capabilities.h +1 -0
index 651507627ef3..f390f9f883c3 100644
@@ -34,6 +34,7 @@ struct nested_vmx_msrs {
 	u32 pinbased_ctls_high;
 	u32 exit_ctls_low;
 	u32 exit_ctls_high;
+	u64 secondary_exit_ctls;
 	u32 entry_ctls_low;
 	u32 entry_ctls_high;
 	u32 misc_low;
arch/x86/kvm/vmx/nested.c +25 -1
index b0cd745518b4..cbb682424a5b 100644
@@ -1534,6 +1534,11 @@ int vmx_set_vmx_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data)
 			return -EINVAL;
 		vmx->nested.msrs.vmfunc_controls = data;
 		return 0;
+	case MSR_IA32_VMX_EXIT_CTLS2:
+		if (data & ~vmcs_config.nested.secondary_exit_ctls)
+			return -EINVAL;
+		vmx->nested.msrs.secondary_exit_ctls = data;
+		return 0;
 	default:
 		/*
 		 * The rest of the VMX capability MSRs do not support restore.
@@ -1573,6 +1578,9 @@ int vmx_get_vmx_msr(struct nested_vmx_msrs *msrs, u32 msr_index, u64 *pdata)
 		if (msr_index == MSR_IA32_VMX_EXIT_CTLS)
 			*pdata |= VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR;
 		break;
+	case MSR_IA32_VMX_EXIT_CTLS2:
+		*pdata = msrs->secondary_exit_ctls;
+		break;
 	case MSR_IA32_VMX_TRUE_ENTRY_CTLS:
 	case MSR_IA32_VMX_ENTRY_CTLS:
 		*pdata = vmx_control_msr(
@@ -2514,6 +2522,11 @@ static void prepare_vmcs02_early(struct vcpu_vmx *vmx, struct loaded_vmcs *vmcs0
 		exec_control &= ~VM_EXIT_LOAD_IA32_EFER;
 	vm_exit_controls_set(vmx, exec_control);
 
+	if (exec_control & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) {
+		exec_control = __secondary_vm_exit_controls_get(vmcs01);
+		secondary_vm_exit_controls_set(vmx, exec_control);
+	}
+
 	/*
 	 * Interrupt/Exception Fields
 	 */
@@ -7128,7 +7141,8 @@ static void nested_vmx_setup_exit_ctls(struct vmcs_config *vmcs_conf,
 		VM_EXIT_HOST_ADDR_SPACE_SIZE |
 #endif
 		VM_EXIT_LOAD_IA32_PAT | VM_EXIT_SAVE_IA32_PAT |
-		VM_EXIT_CLEAR_BNDCFGS | VM_EXIT_LOAD_CET_STATE;
+		VM_EXIT_CLEAR_BNDCFGS | VM_EXIT_LOAD_CET_STATE |
+		VM_EXIT_ACTIVATE_SECONDARY_CONTROLS;
 	msrs->exit_ctls_high |=
 		VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR |
 		VM_EXIT_LOAD_IA32_EFER | VM_EXIT_SAVE_IA32_EFER |
@@ -7141,6 +7155,16 @@ static void nested_vmx_setup_exit_ctls(struct vmcs_config *vmcs_conf,
 
 	/* We support free control of debug control saving. */
 	msrs->exit_ctls_low &= ~VM_EXIT_SAVE_DEBUG_CONTROLS;
+
+	if (msrs->exit_ctls_high & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) {
+		msrs->secondary_exit_ctls = vmcs_conf->vmexit_2nd_ctrl;
+		/*
+		 * As the secondary VM exit control is always loaded, do not
+		 * advertise any feature in it to nVMX until its nVMX support
+		 * is ready.
+		 */
+		msrs->secondary_exit_ctls &= 0;
+	}
 }
 
 static void nested_vmx_setup_entry_ctls(struct vmcs_config *vmcs_conf,
arch/x86/kvm/vmx/vmcs12.c +1 -0
index 4233b5ca9461..3b01175f392a 100644
@@ -66,6 +66,7 @@ const unsigned short vmcs12_field_offsets[] = {
 	FIELD64(HOST_IA32_PAT, host_ia32_pat),
 	FIELD64(HOST_IA32_EFER, host_ia32_efer),
 	FIELD64(HOST_IA32_PERF_GLOBAL_CTRL, host_ia32_perf_global_ctrl),
+	FIELD64(SECONDARY_VM_EXIT_CONTROLS, secondary_vm_exit_controls),
 	FIELD(PIN_BASED_VM_EXEC_CONTROL, pin_based_vm_exec_control),
 	FIELD(CPU_BASED_VM_EXEC_CONTROL, cpu_based_vm_exec_control),
 	FIELD(EXCEPTION_BITMAP, exception_bitmap),
arch/x86/kvm/vmx/vmcs12.h +2 -1
index 4ad6b16525b9..fa5306dc0311 100644
@@ -71,7 +71,7 @@ struct __packed vmcs12 {
 	u64 pml_address;
 	u64 encls_exiting_bitmap;
 	u64 tsc_multiplier;
-	u64 padding64[1]; /* room for future expansion */
+	u64 secondary_vm_exit_controls;
 	/*
 	 * To allow migration of L1 (complete with its L2 guests) between
 	 * machines of different natural widths (32 or 64 bit), we cannot have
@@ -261,6 +261,7 @@ static inline void vmx_check_vmcs12_offsets(void)
 	CHECK_OFFSET(pml_address, 312);
 	CHECK_OFFSET(encls_exiting_bitmap, 320);
 	CHECK_OFFSET(tsc_multiplier, 328);
+	CHECK_OFFSET(secondary_vm_exit_controls, 336);
 	CHECK_OFFSET(cr0_guest_host_mask, 344);
 	CHECK_OFFSET(cr4_guest_host_mask, 352);
 	CHECK_OFFSET(cr0_read_shadow, 360);
arch/x86/kvm/x86.h +1 -1
index e9c6f304b02e..1576f192a647 100644
@@ -95,7 +95,7 @@ do {											\
  * associated feature that KVM supports for nested virtualization.
  */
 #define KVM_FIRST_EMULATED_VMX_MSR	MSR_IA32_VMX_BASIC
-#define KVM_LAST_EMULATED_VMX_MSR	MSR_IA32_VMX_VMFUNC
+#define KVM_LAST_EMULATED_VMX_MSR	MSR_IA32_VMX_EXIT_CTLS2
 
 #define KVM_DEFAULT_PLE_GAP		128
 #define KVM_VMX_DEFAULT_PLE_WINDOW	4096
Signature · 1 line
2.51.0
#84 lore raw

Re: [PATCH v9 18/22] KVM: nVMX: Enable support for secondary VM exit controls

Quoted 61 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Add support for secondary VM exit controls in nested VMX to facilitate future FRED integration.

---

Changes in v8:
* Relocate secondary_vm_exit_controls to the last u64 padding field.
* Remove the change to Documentation/virt/kvm/x86/nested-vmx.rst.

Changes in v5:
* Allow writing MSR_IA32_VMX_EXIT_CTLS2 (Sean).
* Add TB from Xuelian Guo.

Change in v3:
* Read secondary VM exit controls from vmcs_conf insteasd of the hardware MSR MSR_IA32_VMX_EXIT_CTLS2 to avoid advertising features to L1 that KVM itself doesn't support, e.g. because the expected entry+exit pairs aren't supported. (Sean Christopherson)
---

arch/x86/kvm/vmx/capabilities.h |  1 +
arch/x86/kvm/vmx/nested.c       | 26 +++++++++++++++++++++++++-
arch/x86/kvm/vmx/vmcs12.c       |  1 +
arch/x86/kvm/vmx/vmcs12.h       |  3 ++-
arch/x86/kvm/x86.h              |  2 +-
5 files changed, 30 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/capabilities.h +1 -0
index 651507627ef3..f390f9f883c3 100644
@@ -34,6 +34,7 @@ struct nested_vmx_msrs {
 	u32 pinbased_ctls_high;
 	u32 exit_ctls_low;
 	u32 exit_ctls_high;
+	u64 secondary_exit_ctls;
 	u32 entry_ctls_low;
 	u32 entry_ctls_high;
 	u32 misc_low;
arch/x86/kvm/vmx/nested.c +5 -0
index b0cd745518b4..cbb682424a5b 100644
@@ -1534,6 +1534,11 @@ int vmx_set_vmx_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data)
 			return -EINVAL;
 		vmx->nested.msrs.vmfunc_controls = data;
 		return 0;
+	case MSR_IA32_VMX_EXIT_CTLS2:
+		if (data & ~vmcs_config.nested.secondary_exit_ctls)
+			return -EINVAL;
+		vmx->nested.msrs.secondary_exit_ctls = data;
+		return 0;
 	default:
 		/*
 		 * The rest of the VMX capability MSRs do not support restore.

arch/x86/kvm/vmx/nested.c int vmx_get_vmx_msr(struct nested_vmx_msrs *msrs, u32 msr_index, u64 *pdata) · line 1582

Quoted 5 lines
arch/x86/kvm/vmx/nested.c +2 -0
@@ -1573,6 +1578,9 @@ int vmx_get_vmx_msr(struct nested_vmx_msrs *msrs, u32 msr_index, u64 *pdata)
 		if (msr_index == MSR_IA32_VMX_EXIT_CTLS)
 			*pdata |= VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR;
 		break;
+	case MSR_IA32_VMX_EXIT_CTLS2:
+		*pdata = msrs->secondary_exit_ctls;

MSR_IA32_VMX_EXIT_CTLS2 should be added to emulated_msrs_all[] for live migration.

#85 lore raw

Re: [PATCH v9 19/26] KVM: nVMX: Enable support for secondary VM exit controls

In reply to Xin Li (Intel)

arch/x86/kvm/vmx/nested.c int vmx_get_vmx_msr(struct nested_vmx_msrs *msrs, u32 msr_index, u64 *pdata) · line 1583

Quoted 3 lines
arch/x86/kvm/vmx/nested.c +3 -0
+	case MSR_IA32_VMX_EXIT_CTLS2:
+		*pdata = msrs->secondary_exit_ctls;
+		break;

vmx_get_vmx_msr() returns msrs->secondary_exit_ctls for MSR_IA32_VMX_EXIT_CTLS2 unconditionally. But IA32_VMX_EXIT_CTLS2 is only valid if VM-exit controls advertise VM_EXIT_ACTIVATE_SECONDARY_CONTROLS (bit 63 of IA32_VMX_{TRUE_}EXIT_CTLS). On a vCPU where that bit isn't exposed to L1, nested_vmx_setup_exit_ctls() never populates secondary_exit_ctls, yet an L1 RDMSR of 0x493 still succeeds and returns 0 instead of #GP'ing as it would on bare metal.

The write side already validates against vmcs_config.nested.secondary_exit_ctls; the read side should likewise gate on the control being advertised:

	case MSR_IA32_VMX_EXIT_CTLS2:
+		if (!(msrs->exit_ctls_high & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS))
+			return 1;
		*pdata = msrs->secondary_exit_ctls;
		break;

Thanks,
Emily

#86 lore raw

Re: [PATCH v9 19/26] KVM: nVMX: Enable support for secondary VM exit controls

Quoted 9 lines

The write side already validates against

vmcs_config.nested.secondary_exit_ctls; the read side should likewise gate

on the control being advertised:

You are right, the read can be gated on the control being advertised. Looking at the rest of the read function, it doesn't seem to have any other equivalent check. I think there might be others that have similar behavior.

But, I don't see any harm in adding the below check to match the bare metal behavior for the new code. I'll add it to v10 unless someone objects.

Quoted 11 lines
	case MSR_IA32_VMX_EXIT_CTLS2:
(quoted diff) +2 -0
+		if (!(msrs->exit_ctls_high & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS))
 
+			return 1;
		*pdata = msrs->secondary_exit_ctls;

		break;
#87 lore raw

Re: [PATCH v9 19/26] KVM: nVMX: Enable support for secondary VM exit controls

Quoted 4 lines · Sohil Mehta
Quoted 4 lines

The write side already validates against

vmcs_config.nested.secondary_exit_ctls; the read side should likewise gate

Quoted 8 lines
Quoted 3 lines

on the control being advertised:

You are right, the read can be gated on the control being advertised. Looking at the rest of the read function, it doesn't seem to have any other equivalent check. I think there might be others that have similar behavior.

Yes. Secondary controls, tertiary controls, VMFUNC, EPT/VPID, etc.

Quoted 2 lines

But, I don't see any harm in adding the below check to match the bare metal behavior for the new code. I'll add it to v10 unless someone objects.

Normally I want MSR accesses to have the same fault semantics for userspace and guest accesses, but for the VMX MSRs, I think we should let userspace read at all times since they're feature MSRs. E.g. I don't want to end up in a state where userspace can't read an MSR because it restored/set MSRs in the "wrong" order.

We could plumb in @host_initiated to vmx_get_vmx_msr(), but I think I would rather add the check in vmx_get_msr(). E.g. shoot for something like:

x86/kvm/vmx/vmx.c +4 -0
index 504630f0eb40..a99cebfe50e0 100644
@@ -2200,6 +2200,10 @@ int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
        case KVM_FIRST_EMULATED_VMX_MSR ... KVM_LAST_EMULATED_VMX_MSR:
                if (!guest_cpu_cap_has(vcpu, X86_FEATURE_VMX))
                        return 1;
+               if (!msr_info->host_initiated &&
+                   !guest_cpu_has_vmx_msr(msr_info->index))
+                       return 1;
+
                if (vmx_get_vmx_msr(&vmx->nested.msrs, msr_info->index,
                                    &msr_info->data))
                        return 1;

That'll require yet another switch(), but reading these MSRs should never be a hot path.

Quoted 11 lines
Quoted 10 lines
	case MSR_IA32_VMX_EXIT_CTLS2:
(quoted diff) +2 -0
+		if (!(msrs->exit_ctls_high & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS))
 
+			return 1;
		*pdata = msrs->secondary_exit_ctls;

		break;
#88 lore raw

Re: [PATCH v9 19/26] KVM: nVMX: Enable support for secondary VM exit controls

Quoted 5 lines · Sean Christopherson

Normally I want MSR accesses to have the same fault semantics for userspace and guest accesses, but for the VMX MSRs, I think we should let userspace read at all times since they're feature MSRs. E.g. I don't want to end up in a state where userspace can't read an MSR because it restored/set MSRs in the "wrong" order.

Thanks Sean for the clarification.

Quoted 3 lines

We could plumb in @host_initiated to vmx_get_vmx_msr(), but I think I would rather add the check in vmx_get_msr(). E.g. shoot for something like:

Happy to not be the bottleneck here since this is FRED independent. Emily, do you want to pursue this given that you reported the original issue?

If not, I can probably take a look after I send the FRED patches out. I am sure Sashiko will report other such pre-existing stuff based on the FRED patchset :)

Quoted 20 lines
x86/kvm/vmx/vmx.c +4 -0
index 504630f0eb40..a99cebfe50e0 100644
@@ -2200,6 +2200,10 @@ int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
        case KVM_FIRST_EMULATED_VMX_MSR ... KVM_LAST_EMULATED_VMX_MSR:
                if (!guest_cpu_cap_has(vcpu, X86_FEATURE_VMX))
                        return 1;
+               if (!msr_info->host_initiated &&
+                   !guest_cpu_has_vmx_msr(msr_info->index))
+                       return 1;
+
                if (vmx_get_vmx_msr(&vmx->nested.msrs, msr_info->index,
                                    &msr_info->data))
                        return 1;

That'll require yet another switch(), but reading these MSRs should never be a hot path.

#89 lore raw

Re: [PATCH v9 19/26] KVM: nVMX: Enable support for secondary VM exit controls

Quoted 3 lines

Happy to not be the bottleneck here since this is FRED independent. Emily, do you want to pursue this given that you reported the original issue?

Yes I would like to take this, just busy this week so will probably only find time in a week or two :)

author maintainer
#90 lore raw

PATCH v9 19/22 KVM: nVMX: Handle FRED VMCS fields in nested VMX context

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Extend nested VMX context management to include FRED-related VMCS fields, enabling proper handling of FRED state during nested virtualization.

Because KVM always sets SECONDARY_VM_EXIT_SAVE_IA32_FRED, FRED MSRs are always saved to vmcs02. However an L1 VMM may choose to clear this bit, i.e., not to save FRED MSRs to vmcs12. This is not a problem when the L1 VMM sets SECONDARY_VM_EXIT_LOAD_IA32_FRED, as KVM then immediately loads host FRED MSRs of vmcs12 to guest FRED MSRs of vmcs01. However if the L1 VMM clears SECONDARY_VM_EXIT_LOAD_IA32_FRED, KVM should retain FRED MSRs to run the L1 VMM.

To propagate guest FRED MSRs from vmcs02 to vmcs01, save them in sync_vmcs02_to_vmcs12() regardless of whether SECONDARY_VM_EXIT_SAVE_IA32_FRED is set in vmcs12. Then, use the saved values to set guest FRED MSRs in vmcs01 within load_vmcs12_host_state() when !nested_cpu_load_host_fred_state().

---

Changes in v9:
* Rebase to kvm-x86/next.
* Guard FRED state save/restore with guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) (syzbot & Chao).

Changes in v8:
* Make the newly added FRED fields 64-bit aligned in vmcs12 (Isaku).
* Remove the change to Documentation/virt/kvm/x86/nested-vmx.rst.

Change in v6:
* Handle FRED MSR pre-vmenter save/restore (Chao Gao).
* Save FRED MSRs of vmcs02 at VM-Exit even an L1 VMM clears SECONDARY_VM_EXIT_SAVE_IA32_FRED.
* Save FRED MSRs in sync_vmcs02_to_vmcs12() instead of its rare version.

Change in v5:
* Add TB from Xuelian Guo.

Changes in v4:
* Advertise VMX nested exception as if the CPU supports it (Chao Gao).
* Split FRED state management controls (Chao Gao).

Changes in v3:
* Add and use nested_cpu_has_fred(vmcs12) because vmcs02 should be set from vmcs12 if and only if the field is enabled in L1's VMX config (Sean Christopherson).
* Fix coding style issues (Sean Christopherson).

Changes in v2:
* Remove hyperv TLFS related changes (Jeremi Piotrowski).
* Use kvm_cpu_cap_has() instead of cpu_feature_enabled() (Chao Gao).

---
 arch/x86/kvm/vmx/capabilities.h       |   5 ++
 arch/x86/kvm/vmx/nested.c             | 118 +++++++++++++++++++++++++-
 arch/x86/kvm/vmx/nested.h             |  22 +++++
 arch/x86/kvm/vmx/vmcs12.c             |  18 ++++
 arch/x86/kvm/vmx/vmcs12.h             |  37 ++++++++
 arch/x86/kvm/vmx/vmcs_shadow_fields.h |   4 +
 arch/x86/kvm/vmx/vmx.h                |  41 +++++++++
 7 files changed, 243 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/capabilities.h +5 -0
index f390f9f883c3..5eba2530ffb4 100644
@@ -80,6 +80,11 @@ static inline bool cpu_has_vmx_basic_no_hw_errcode_cc(void)
 	return	vmcs_config.basic & VMX_BASIC_NO_HW_ERROR_CODE_CC;
 }
 
+static inline bool cpu_has_vmx_nested_exception(void)
+{
+	return vmcs_config.basic & VMX_BASIC_NESTED_EXCEPTION;
+}
+
 static inline bool cpu_has_virtual_nmis(void)
 {
 	return vmcs_config.pin_based_exec_ctrl & PIN_BASED_VIRTUAL_NMIS &&
arch/x86/kvm/vmx/nested.c +116 -2

197 lines not on this page · raw message

arch/x86/kvm/vmx/nested.h +22 -0
index 983484d42ebf..a99d3d83d58e 100644
@@ -249,6 +249,11 @@ static inline bool nested_cpu_has_save_preemption_timer(struct vmcs12 *vmcs12)
 	    VM_EXIT_SAVE_VMX_PREEMPTION_TIMER;
 }
 
+static inline bool nested_cpu_has_secondary_vm_exit_controls(struct vmcs12 *vmcs12)
+{
+	return vmcs12->vm_exit_controls & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS;
+}
+
 static inline bool nested_exit_on_nmi(struct kvm_vcpu *vcpu)
 {
 	return nested_cpu_has_nmi_exiting(get_vmcs12(vcpu));
@@ -269,6 +274,23 @@ static inline bool nested_cpu_has_encls_exit(struct vmcs12 *vmcs12)
 	return nested_cpu_has2(vmcs12, SECONDARY_EXEC_ENCLS_EXITING);
 }
 
+static inline bool nested_cpu_load_guest_fred_state(struct vmcs12 *vmcs12)
+{
+	return vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_FRED;
+}
+
+static inline bool nested_cpu_save_guest_fred_state(struct vmcs12 *vmcs12)
+{
+	return nested_cpu_has_secondary_vm_exit_controls(vmcs12) &&
+	       vmcs12->secondary_vm_exit_controls & SECONDARY_VM_EXIT_SAVE_IA32_FRED;
+}
+
+static inline bool nested_cpu_load_host_fred_state(struct vmcs12 *vmcs12)
+{
+	return nested_cpu_has_secondary_vm_exit_controls(vmcs12) &&
+	       vmcs12->secondary_vm_exit_controls & SECONDARY_VM_EXIT_LOAD_IA32_FRED;
+}
+
 /*
  * if fixed0[i] == 1: val[i] must be 1
  * if fixed1[i] == 0: val[i] must be 0
arch/x86/kvm/vmx/vmcs12.c +18 -0
index 3b01175f392a..9691e709061f 100644
@@ -67,6 +67,24 @@ const unsigned short vmcs12_field_offsets[] = {
 	FIELD64(HOST_IA32_EFER, host_ia32_efer),
 	FIELD64(HOST_IA32_PERF_GLOBAL_CTRL, host_ia32_perf_global_ctrl),
 	FIELD64(SECONDARY_VM_EXIT_CONTROLS, secondary_vm_exit_controls),
+	FIELD64(INJECTED_EVENT_DATA, injected_event_data),
+	FIELD64(ORIGINAL_EVENT_DATA, original_event_data),
+	FIELD64(GUEST_IA32_FRED_CONFIG, guest_ia32_fred_config),
+	FIELD64(GUEST_IA32_FRED_RSP1, guest_ia32_fred_rsp1),
+	FIELD64(GUEST_IA32_FRED_RSP2, guest_ia32_fred_rsp2),
+	FIELD64(GUEST_IA32_FRED_RSP3, guest_ia32_fred_rsp3),
+	FIELD64(GUEST_IA32_FRED_STKLVLS, guest_ia32_fred_stklvls),
+	FIELD64(GUEST_IA32_FRED_SSP1, guest_ia32_fred_ssp1),
+	FIELD64(GUEST_IA32_FRED_SSP2, guest_ia32_fred_ssp2),
+	FIELD64(GUEST_IA32_FRED_SSP3, guest_ia32_fred_ssp3),
+	FIELD64(HOST_IA32_FRED_CONFIG, host_ia32_fred_config),
+	FIELD64(HOST_IA32_FRED_RSP1, host_ia32_fred_rsp1),
+	FIELD64(HOST_IA32_FRED_RSP2, host_ia32_fred_rsp2),
+	FIELD64(HOST_IA32_FRED_RSP3, host_ia32_fred_rsp3),
+	FIELD64(HOST_IA32_FRED_STKLVLS, host_ia32_fred_stklvls),
+	FIELD64(HOST_IA32_FRED_SSP1, host_ia32_fred_ssp1),
+	FIELD64(HOST_IA32_FRED_SSP2, host_ia32_fred_ssp2),
+	FIELD64(HOST_IA32_FRED_SSP3, host_ia32_fred_ssp3),
 	FIELD(PIN_BASED_VM_EXEC_CONTROL, pin_based_vm_exec_control),
 	FIELD(CPU_BASED_VM_EXEC_CONTROL, cpu_based_vm_exec_control),
 	FIELD(EXCEPTION_BITMAP, exception_bitmap),
arch/x86/kvm/vmx/vmcs12.h +37 -0
index fa5306dc0311..051016a3afba 100644
@@ -191,6 +191,25 @@ struct __packed vmcs12 {
 	u16 host_gs_selector;
 	u16 host_tr_selector;
 	u16 guest_pml_index;
+	u16 padding16[1]; /* align to 64-bit boundary */
+	u64 guest_ia32_fred_config;
+	u64 guest_ia32_fred_rsp1;
+	u64 guest_ia32_fred_rsp2;
+	u64 guest_ia32_fred_rsp3;
+	u64 guest_ia32_fred_stklvls;
+	u64 guest_ia32_fred_ssp1;
+	u64 guest_ia32_fred_ssp2;
+	u64 guest_ia32_fred_ssp3;
+	u64 host_ia32_fred_config;
+	u64 host_ia32_fred_rsp1;
+	u64 host_ia32_fred_rsp2;
+	u64 host_ia32_fred_rsp3;
+	u64 host_ia32_fred_stklvls;
+	u64 host_ia32_fred_ssp1;
+	u64 host_ia32_fred_ssp2;
+	u64 host_ia32_fred_ssp3;
+	u64 injected_event_data;
+	u64 original_event_data;
 };
 
 /*
@@ -373,6 +392,24 @@ static inline void vmx_check_vmcs12_offsets(void)
 	CHECK_OFFSET(host_gs_selector, 992);
 	CHECK_OFFSET(host_tr_selector, 994);
 	CHECK_OFFSET(guest_pml_index, 996);
+	CHECK_OFFSET(guest_ia32_fred_config, 1000);
+	CHECK_OFFSET(guest_ia32_fred_rsp1, 1008);
+	CHECK_OFFSET(guest_ia32_fred_rsp2, 1016);
+	CHECK_OFFSET(guest_ia32_fred_rsp3, 1024);
+	CHECK_OFFSET(guest_ia32_fred_stklvls, 1032);
+	CHECK_OFFSET(guest_ia32_fred_ssp1, 1040);
+	CHECK_OFFSET(guest_ia32_fred_ssp2, 1048);
+	CHECK_OFFSET(guest_ia32_fred_ssp3, 1056);
+	CHECK_OFFSET(host_ia32_fred_config, 1064);
+	CHECK_OFFSET(host_ia32_fred_rsp1, 1072);
+	CHECK_OFFSET(host_ia32_fred_rsp2, 1080);
+	CHECK_OFFSET(host_ia32_fred_rsp3, 1088);
+	CHECK_OFFSET(host_ia32_fred_stklvls, 1096);
+	CHECK_OFFSET(host_ia32_fred_ssp1, 1104);
+	CHECK_OFFSET(host_ia32_fred_ssp2, 1112);
+	CHECK_OFFSET(host_ia32_fred_ssp3, 1120);
+	CHECK_OFFSET(injected_event_data, 1128);
+	CHECK_OFFSET(original_event_data, 1136);
 }
 
 extern const unsigned short vmcs12_field_offsets[];
arch/x86/kvm/vmx/vmcs_shadow_fields.h +4 -0
index cad128d1657b..da338327c2b3 100644
@@ -74,6 +74,10 @@ SHADOW_FIELD_RW(HOST_GS_BASE, host_gs_base)
 /* 64-bit */
 SHADOW_FIELD_RO(GUEST_PHYSICAL_ADDRESS, guest_physical_address)
 SHADOW_FIELD_RO(GUEST_PHYSICAL_ADDRESS_HIGH, guest_physical_address)
+SHADOW_FIELD_RO(ORIGINAL_EVENT_DATA, original_event_data)
+SHADOW_FIELD_RO(ORIGINAL_EVENT_DATA_HIGH, original_event_data)
+SHADOW_FIELD_RW(INJECTED_EVENT_DATA, injected_event_data)
+SHADOW_FIELD_RW(INJECTED_EVENT_DATA_HIGH, injected_event_data)
 
 #undef SHADOW_FIELD_RO
 #undef SHADOW_FIELD_RW
arch/x86/kvm/vmx/vmx.h +41 -0

53 lines not on this page · raw message

Signature · 1 line
2.51.0
#91 lore raw

Re: [PATCH v9 19/22] KVM: nVMX: Handle FRED VMCS fields in nested VMX context

Quoted 21 lines
arch/x86/kvm/vmx/capabilities.h +5 -0
index f390f9f883c3..5eba2530ffb4 100644
@@ -80,6 +80,11 @@ static inline bool cpu_has_vmx_basic_no_hw_errcode_cc(void)
 	return	vmcs_config.basic & VMX_BASIC_NO_HW_ERROR_CODE_CC;
 }
 
+static inline bool cpu_has_vmx_nested_exception(void)
+{
+	return vmcs_config.basic & VMX_BASIC_NESTED_EXCEPTION;
+}
+
 static inline bool cpu_has_virtual_nmis(void)
 {
 	return vmcs_config.pin_based_exec_ctrl & PIN_BASED_VIRTUAL_NMIS &&
arch/x86/kvm/vmx/nested.c +0 -0
index cbb682424a5b..63cdfffba58b 100644

arch/x86/kvm/vmx/nested.c static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, · line 713

Quoted 6 lines
arch/x86/kvm/vmx/nested.c +3 -0
@@ -708,6 +708,9 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu,
 
 	nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0,
 					 MSR_KERNEL_GS_BASE, MSR_TYPE_RW);
+
+	nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0,
+					 MSR_IA32_FRED_RSP0, MSR_TYPE_RW);

Why is only this specific MSR handled? What about other FRED MSRs?

Quoted 41 lines
arch/x86/kvm/vmx/nested.c +17 -2
 #endif
 	nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0,
 					 MSR_IA32_SPEC_CTRL, MSR_TYPE_RW);
@@ -1294,9 +1297,11 @@ static int vmx_restore_vmx_basic(struct vcpu_vmx *vmx, u64 data)
 	const u64 feature_bits = VMX_BASIC_DUAL_MONITOR_TREATMENT |
 				 VMX_BASIC_INOUT |
 				 VMX_BASIC_TRUE_CTLS |
-				 VMX_BASIC_NO_HW_ERROR_CODE_CC;
+				 VMX_BASIC_NO_HW_ERROR_CODE_CC |
+				 VMX_BASIC_NESTED_EXCEPTION;
 
-	const u64 reserved_bits = GENMASK_ULL(63, 57) |
+	const u64 reserved_bits = GENMASK_ULL(63, 59) |
+				  BIT_ULL(57) |
 				  GENMASK_ULL(47, 45) |
 				  BIT_ULL(31);
 
@@ -2539,6 +2544,8 @@ static void prepare_vmcs02_early(struct vcpu_vmx *vmx, struct loaded_vmcs *vmcs0
 			     vmcs12->vm_entry_instruction_len);
 		vmcs_write32(GUEST_INTERRUPTIBILITY_INFO,
 			     vmcs12->guest_interruptibility_info);
+		if (cpu_has_vmx_fred())
+			vmcs_write64(INJECTED_EVENT_DATA, vmcs12->injected_event_data);
 		vmx->loaded_vmcs->nmi_known_unmasked =
 			!(vmcs12->guest_interruptibility_info & GUEST_INTR_STATE_NMI);
 	} else {
@@ -2693,6 +2700,18 @@ static void prepare_vmcs02_rare(struct vcpu_vmx *vmx, struct vmcs12 *vmcs12)
 				     vmcs12->guest_ssp, vmcs12->guest_ssp_tbl);
 
 	set_cr4_guest_host_mask(vmx);
+
+	if (guest_cpu_cap_has(&vmx->vcpu, X86_FEATURE_FRED) &&
+	    nested_cpu_load_guest_fred_state(vmcs12)) {
+		vmcs_write64(GUEST_IA32_FRED_CONFIG, vmcs12->guest_ia32_fred_config);
+		vmcs_write64(GUEST_IA32_FRED_RSP1, vmcs12->guest_ia32_fred_rsp1);
+		vmcs_write64(GUEST_IA32_FRED_RSP2, vmcs12->guest_ia32_fred_rsp2);
+		vmcs_write64(GUEST_IA32_FRED_RSP3, vmcs12->guest_ia32_fred_rsp3);
+		vmcs_write64(GUEST_IA32_FRED_STKLVLS, vmcs12->guest_ia32_fred_stklvls);
+		vmcs_write64(GUEST_IA32_FRED_SSP1, vmcs12->guest_ia32_fred_ssp1);
+		vmcs_write64(GUEST_IA32_FRED_SSP2, vmcs12->guest_ia32_fred_ssp2);
+		vmcs_write64(GUEST_IA32_FRED_SSP3, vmcs12->guest_ia32_fred_ssp3);

...

Quoted 8 lines
arch/x86/kvm/vmx/nested.c +1 -0
+	}

}

/*

arch/x86/kvm/vmx/nested.c +0 -0
@@ -2759,6 +2778,18 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,
 		vmcs_write64(GUEST_IA32_PAT, vcpu->arch.pat);
 	}
 

arch/x86/kvm/vmx/nested.c static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, · line 2790

Quoted 10 lines
arch/x86/kvm/vmx/nested.c +10 -0
+	if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) &&
+	    (!vmx->nested.nested_run_pending || !nested_cpu_load_guest_fred_state(vmcs12))) {
+		vmcs_write64(GUEST_IA32_FRED_CONFIG, vmx->nested.pre_vmenter_fred_config);
+		vmcs_write64(GUEST_IA32_FRED_RSP1, vmx->nested.pre_vmenter_fred_rsp1);
+		vmcs_write64(GUEST_IA32_FRED_RSP2, vmx->nested.pre_vmenter_fred_rsp2);
+		vmcs_write64(GUEST_IA32_FRED_RSP3, vmx->nested.pre_vmenter_fred_rsp3);
+		vmcs_write64(GUEST_IA32_FRED_STKLVLS, vmx->nested.pre_vmenter_fred_stklvls);
+		vmcs_write64(GUEST_IA32_FRED_SSP1, vmx->nested.pre_vmenter_fred_ssp1);
+		vmcs_write64(GUEST_IA32_FRED_SSP2, vmx->nested.pre_vmenter_fred_ssp2);
+		vmcs_write64(GUEST_IA32_FRED_SSP3, vmx->nested.pre_vmenter_fred_ssp3);

Would it be clearer to add two helpers to read/write FRED VMCS fields? e.g., (compile test only)

arch/x86/kvm/vmx/nested.c +0 -0
index c8edbe9c7e00..b709f4cdcba3 100644
@@ -2614,6 +2614,30 @@ static void vmcs_write_cet_state(struct kvm_vcpu *vcpu, u64 s_cet,
	}

}

+static void vmcs_read_fred_msrs(struct fred_msrs *msrs)
+{
+ msrs->fred_config = vmcs_read64(GUEST_IA32_FRED_CONFIG);
+ msrs->fred_rsp1 = vmcs_read64(GUEST_IA32_FRED_RSP1);
+ msrs->fred_rsp2 = vmcs_read64(GUEST_IA32_FRED_RSP2);
+ msrs->fred_rsp3 = vmcs_read64(GUEST_IA32_FRED_RSP3);
+ msrs->fred_stklvls = vmcs_read64(GUEST_IA32_FRED_STKLVLS);
+ msrs->fred_ssp1 = vmcs_read64(GUEST_IA32_FRED_SSP1);
+ msrs->fred_ssp2 = vmcs_read64(GUEST_IA32_FRED_SSP2);
+ msrs->fred_ssp3 = vmcs_read64(GUEST_IA32_FRED_SSP3);
+}
+
+static void vmcs_write_fred_msrs(struct fred_msrs *msrs)
+{
+ vmcs_write64(GUEST_IA32_FRED_CONFIG, msrs->fred_config);
+ vmcs_write64(GUEST_IA32_FRED_RSP1, msrs->fred_rsp1);
+ vmcs_write64(GUEST_IA32_FRED_RSP2, msrs->fred_rsp2);
+ vmcs_write64(GUEST_IA32_FRED_RSP3, msrs->fred_rsp3);
+ vmcs_write64(GUEST_IA32_FRED_STKLVLS, msrs->fred_stklvls);
+ vmcs_write64(GUEST_IA32_FRED_SSP1, msrs->fred_ssp1);
+ vmcs_write64(GUEST_IA32_FRED_SSP2, msrs->fred_ssp2);
+ vmcs_write64(GUEST_IA32_FRED_SSP3, msrs->fred_ssp3);
+}
+
static void prepare_vmcs02_rare(struct vcpu_vmx *vmx, struct vmcs12 *vmcs12)

 {
	struct hv_enlightened_vmcs *hv_evmcs = nested_vmx_evmcs(vmx);

@@ -2736,16 +2760,8 @@ static void prepare_vmcs02_rare(struct vcpu_vmx *vmx, struct vmcs12 *vmcs12)

	set_cr4_guest_host_mask(vmx);

- if (nested_cpu_load_guest_fred_state(vmcs12)) {
- vmcs_write64(GUEST_IA32_FRED_CONFIG, vmcs12->guest_ia32_fred_config);
- vmcs_write64(GUEST_IA32_FRED_RSP1, vmcs12->guest_ia32_fred_rsp1);
- vmcs_write64(GUEST_IA32_FRED_RSP2, vmcs12->guest_ia32_fred_rsp2);
- vmcs_write64(GUEST_IA32_FRED_RSP3, vmcs12->guest_ia32_fred_rsp3);
- vmcs_write64(GUEST_IA32_FRED_STKLVLS, vmcs12->guest_ia32_fred_stklvls);
- vmcs_write64(GUEST_IA32_FRED_SSP1, vmcs12->guest_ia32_fred_ssp1);
- vmcs_write64(GUEST_IA32_FRED_SSP2, vmcs12->guest_ia32_fred_ssp2);
- vmcs_write64(GUEST_IA32_FRED_SSP3, vmcs12->guest_ia32_fred_ssp3);
- }
+ if (nested_cpu_load_guest_fred_state(vmcs12))
+ vmcs_write_fred_msrs((struct fred_msrs *)&vmcs12->guest_ia32_fred_config);
}

/*
@@ -2813,16 +2829,8 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,

	}
 
	if (!vmx->nested.nested_run_pending ||

- !nested_cpu_load_guest_fred_state(vmcs12)) {
- vmcs_write64(GUEST_IA32_FRED_CONFIG, vmx->nested.pre_vmenter_fred_config);
- vmcs_write64(GUEST_IA32_FRED_RSP1, vmx->nested.pre_vmenter_fred_rsp1);
- vmcs_write64(GUEST_IA32_FRED_RSP2, vmx->nested.pre_vmenter_fred_rsp2);
- vmcs_write64(GUEST_IA32_FRED_RSP3, vmx->nested.pre_vmenter_fred_rsp3);
- vmcs_write64(GUEST_IA32_FRED_STKLVLS, vmx->nested.pre_vmenter_fred_stklvls);
- vmcs_write64(GUEST_IA32_FRED_SSP1, vmx->nested.pre_vmenter_fred_ssp1);
- vmcs_write64(GUEST_IA32_FRED_SSP2, vmx->nested.pre_vmenter_fred_ssp2);
- vmcs_write64(GUEST_IA32_FRED_SSP3, vmx->nested.pre_vmenter_fred_ssp3);
- }
+ !nested_cpu_load_guest_fred_state(vmcs12))
+ vmcs_write_fred_msrs(&vmx->nested.fred_msrs_pre_vmenter);

	vcpu->arch.tsc_offset = kvm_calc_nested_tsc_offset(
			vcpu->arch.l1_tsc_offset,
@@ -3830,16 +3838,8 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,
				    &vmx->nested.pre_vmenter_ssp_tbl);
 
	if (!vmx->nested.nested_run_pending ||

- !nested_cpu_load_guest_fred_state(vmcs12)) {
- vmx->nested.pre_vmenter_fred_config = vmcs_read64(GUEST_IA32_FRED_CONFIG);
- vmx->nested.pre_vmenter_fred_rsp1 = vmcs_read64(GUEST_IA32_FRED_RSP1);
- vmx->nested.pre_vmenter_fred_rsp2 = vmcs_read64(GUEST_IA32_FRED_RSP2);
- vmx->nested.pre_vmenter_fred_rsp3 = vmcs_read64(GUEST_IA32_FRED_RSP3);
- vmx->nested.pre_vmenter_fred_stklvls = vmcs_read64(GUEST_IA32_FRED_STKLVLS);
- vmx->nested.pre_vmenter_fred_ssp1 = vmcs_read64(GUEST_IA32_FRED_SSP1);
- vmx->nested.pre_vmenter_fred_ssp2 = vmcs_read64(GUEST_IA32_FRED_SSP2);
- vmx->nested.pre_vmenter_fred_ssp3 = vmcs_read64(GUEST_IA32_FRED_SSP3);
- }
+ !nested_cpu_load_guest_fred_state(vmcs12))
+ vmcs_read_fred_msrs(&vmx->nested.fred_msrs_pre_vmenter);

	/*
	 * Overwrite vmcs01.GUEST_CR3 with L1's CR3 if EPT is disabled *and*
@@ -4938,25 +4938,10 @@ static void sync_vmcs02_to_vmcs12(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12)
			    &vmcs12->guest_ssp,
			    &vmcs12->guest_ssp_tbl);

- vmx->nested.fred_msr_at_vmexit.fred_config = vmcs_read64(GUEST_IA32_FRED_CONFIG);
- vmx->nested.fred_msr_at_vmexit.fred_rsp1 = vmcs_read64(GUEST_IA32_FRED_RSP1);
- vmx->nested.fred_msr_at_vmexit.fred_rsp2 = vmcs_read64(GUEST_IA32_FRED_RSP2);
- vmx->nested.fred_msr_at_vmexit.fred_rsp3 = vmcs_read64(GUEST_IA32_FRED_RSP3);
- vmx->nested.fred_msr_at_vmexit.fred_stklvls = vmcs_read64(GUEST_IA32_FRED_STKLVLS);
- vmx->nested.fred_msr_at_vmexit.fred_ssp1 = vmcs_read64(GUEST_IA32_FRED_SSP1);
- vmx->nested.fred_msr_at_vmexit.fred_ssp2 = vmcs_read64(GUEST_IA32_FRED_SSP2);
- vmx->nested.fred_msr_at_vmexit.fred_ssp3 = vmcs_read64(GUEST_IA32_FRED_SSP3);
+ vmcs_read_fred_msrs(&vmx->nested.fred_msrs_at_vmexit);

- if (nested_cpu_save_guest_fred_state(vmcs12)) {
- vmcs12->guest_ia32_fred_config = vmx->nested.fred_msr_at_vmexit.fred_config;
- vmcs12->guest_ia32_fred_rsp1 = vmx->nested.fred_msr_at_vmexit.fred_rsp1;
- vmcs12->guest_ia32_fred_rsp2 = vmx->nested.fred_msr_at_vmexit.fred_rsp2;
- vmcs12->guest_ia32_fred_rsp3 = vmx->nested.fred_msr_at_vmexit.fred_rsp3;
- vmcs12->guest_ia32_fred_stklvls = vmx->nested.fred_msr_at_vmexit.fred_stklvls;
- vmcs12->guest_ia32_fred_ssp1 = vmx->nested.fred_msr_at_vmexit.fred_ssp1;
- vmcs12->guest_ia32_fred_ssp2 = vmx->nested.fred_msr_at_vmexit.fred_ssp2;
- vmcs12->guest_ia32_fred_ssp3 = vmx->nested.fred_msr_at_vmexit.fred_ssp3;
- }
+ if (nested_cpu_save_guest_fred_state(vmcs12))
+ memcpy(&vmcs12->guest_ia32_fred_config, &vmx->nested.fred_msrs_at_vmexit, sizeof(struct fred_msrs));
}

/*
@@ -5119,25 +5104,10 @@ static void load_vmcs12_host_state(struct kvm_vcpu *vcpu,

		WARN_ON_ONCE(__kvm_emulate_msr_write(vcpu, MSR_CORE_PERF_GLOBAL_CTRL,
						     vmcs12->host_ia32_perf_global_ctrl));

- if (nested_cpu_load_host_fred_state(vmcs12)) {
- vmcs_write64(GUEST_IA32_FRED_CONFIG, vmcs12->host_ia32_fred_config);
- vmcs_write64(GUEST_IA32_FRED_RSP1, vmcs12->host_ia32_fred_rsp1);
- vmcs_write64(GUEST_IA32_FRED_RSP2, vmcs12->host_ia32_fred_rsp2);
- vmcs_write64(GUEST_IA32_FRED_RSP3, vmcs12->host_ia32_fred_rsp3);
- vmcs_write64(GUEST_IA32_FRED_STKLVLS, vmcs12->host_ia32_fred_stklvls);
- vmcs_write64(GUEST_IA32_FRED_SSP1, vmcs12->host_ia32_fred_ssp1);
- vmcs_write64(GUEST_IA32_FRED_SSP2, vmcs12->host_ia32_fred_ssp2);
- vmcs_write64(GUEST_IA32_FRED_SSP3, vmcs12->host_ia32_fred_ssp3);
- } else {
- vmcs_write64(GUEST_IA32_FRED_CONFIG, vmx->nested.fred_msr_at_vmexit.fred_config);
- vmcs_write64(GUEST_IA32_FRED_RSP1, vmx->nested.fred_msr_at_vmexit.fred_rsp1);
- vmcs_write64(GUEST_IA32_FRED_RSP2, vmx->nested.fred_msr_at_vmexit.fred_rsp2);
- vmcs_write64(GUEST_IA32_FRED_RSP3, vmx->nested.fred_msr_at_vmexit.fred_rsp3);
- vmcs_write64(GUEST_IA32_FRED_STKLVLS, vmx->nested.fred_msr_at_vmexit.fred_stklvls);
- vmcs_write64(GUEST_IA32_FRED_SSP1, vmx->nested.fred_msr_at_vmexit.fred_ssp1);
- vmcs_write64(GUEST_IA32_FRED_SSP2, vmx->nested.fred_msr_at_vmexit.fred_ssp2);
- vmcs_write64(GUEST_IA32_FRED_SSP3, vmx->nested.fred_msr_at_vmexit.fred_ssp3);
- }
+ if (nested_cpu_load_host_fred_state(vmcs12))
+ vmcs_write_fred_msrs((struct fred_msrs *)vmcs12->host_ia32_fred_config);
+ else
+ vmcs_write_fred_msrs(&vmx->nested.fred_msrs_at_vmexit);

	/* Set L1 segment info according to Intel SDM
	    27.5.2 Loading Host Segment and Descriptor-Table Registers */
arch/x86/kvm/vmx/vmx.h +1 -1
index 36dcc888e5c6..c1c32e8ae068 100644
@@ -87,7 +87,7 @@ struct pt_desc {
  * running the L1 VMM if SECONDARY_VM_EXIT_LOAD_IA32_FRED is cleared in
  * vmcs12.
  */
-struct fred_msr_at_vmexit {
+struct fred_msrs {
	u64 fred_config;
	u64 fred_rsp1;
	u64 fred_rsp2;
@@ -215,16 +215,8 @@ struct nested_vmx {
	u64 pre_vmenter_s_cet;
	u64 pre_vmenter_ssp;
	u64 pre_vmenter_ssp_tbl;

- u64 pre_vmenter_fred_config;
- u64 pre_vmenter_fred_rsp1;
- u64 pre_vmenter_fred_rsp2;
- u64 pre_vmenter_fred_rsp3;
- u64 pre_vmenter_fred_stklvls;
- u64 pre_vmenter_fred_ssp1;
- u64 pre_vmenter_fred_ssp2;
- u64 pre_vmenter_fred_ssp3;
-
- struct fred_msr_at_vmexit fred_msr_at_vmexit;
+
+ struct fred_msrs fred_msrs_at_vmexit, fred_msrs_pre_vmenter;

	/* to migrate it to L1 if L2 writes to L1's CR8 directly */
	int l1_tpr_threshold;
author maintainer
#92 lore raw

Re: [PATCH v9 19/22] KVM: nVMX: Handle FRED VMCS fields in nested VMX context

Quoted 9 lines · Chao Gao

Review

Quoted 4 lines
(quoted diff) +3 -0
+
+ nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0,
+  MSR_IA32_FRED_RSP0, MSR_TYPE_RW);

Why is only this specific MSR handled? What about other FRED MSRs?

Peter just gave a good explanation:

https://lore.kernel.org/lkml/[email protected]/

#93 lore raw

Re: [PATCH v9 19/22] KVM: nVMX: Handle FRED VMCS fields in nested VMX context

Quoted 7 lines · Xin Li
Quoted 4 lines · Chao Gao
Quoted 2 lines

+ nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0,
+ MSR_IA32_FRED_RSP0, MSR_TYPE_RW);

Why is only this specific MSR handled? What about other FRED MSRs?

Peter just gave a good explanation:

https://lore.kernel.org/lkml/[email protected]/

Yup! It would be great to get some of that content into the changelogs and comments.

author maintainer
#94 lore raw

Re: [PATCH v9 19/22] KVM: nVMX: Handle FRED VMCS fields in nested VMX context

Quoted 7 lines · Dave Hansen
Quoted 3 lines · Xin Li
Quoted 3 lines · Chao Gao
Quoted 2 lines

+ nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0,
+ MSR_IA32_FRED_RSP0, MSR_TYPE_RW);

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Why is only this specific MSR handled? What about other FRED MSRs?

Peter just gave a good explanation:

https://lore.kernel.org/lkml/[email protected]/

Yup! It would be great to get some of that content into the changelogs and comments.

Sure, I will do it with Peter :)

And we already have a document file: Documentation/arch/x86/x86_64/fred.rst.

I think the comment is more for native, right?

#95 lore raw

Re: [PATCH v9 19/22] KVM: nVMX: Handle FRED VMCS fields in nested VMX context

In reply to Xin Li
Quoted 8 lines · Xin Li
Quoted 8 lines · Chao Gao
Quoted 4 lines
(quoted diff) +3 -0
+
+ nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0,
+  MSR_IA32_FRED_RSP0, MSR_TYPE_RW);
Quoted 5 lines
Quoted 1 line

Why is only this specific MSR handled? What about other FRED MSRs?

Peter just gave a good explanation:

https://lore.kernel.org/lkml/[email protected]/

Do you need to set up vmcs02's MSR bitmap for other FRED MSRs?

Other FRED MSRs are passed through to L1 guests in patch 8. Is there any concern about passing through them to L2 guests?

author maintainer
#96 lore raw

Re: [PATCH v9 19/22] KVM: nVMX: Handle FRED VMCS fields in nested VMX context

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines

+ nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0,
+ MSR_IA32_FRED_RSP0, MSR_TYPE_RW);

Why is only this specific MSR handled? What about other FRED MSRs?

Quoted 9 lines
Quoted 3 lines

Peter just gave a good explanation:

https://lore.kernel.org/lkml/[email protected]/

Do you need to set up vmcs02's MSR bitmap for other FRED MSRs?

Other FRED MSRs are passed through to L1 guests in patch 8. Is there any concern about passing through them to L2 guests?

Hmm, this caught me. Yes, I need to do the same to other FRED MSRs.

#97 lore raw

Re: [PATCH v9 19/22] KVM: nVMX: Handle FRED VMCS fields in nested VMX context

In reply to Xin Li (Intel)

arch/x86/kvm/vmx/vmcs_shadow_fields.h SHADOW_FIELD_RW(HOST_GS_BASE, host_gs_base) · line 80

Quoted 12 lines · Xin Li (Intel)
arch/x86/kvm/vmx/vmcs_shadow_fields.h +4 -0
index cad128d1657b..da338327c2b3 100644
@@ -74,6 +74,10 @@ SHADOW_FIELD_RW(HOST_GS_BASE, host_gs_base)
 /* 64-bit */
 SHADOW_FIELD_RO(GUEST_PHYSICAL_ADDRESS, guest_physical_address)
 SHADOW_FIELD_RO(GUEST_PHYSICAL_ADDRESS_HIGH, guest_physical_address)
+SHADOW_FIELD_RO(ORIGINAL_EVENT_DATA, original_event_data)
+SHADOW_FIELD_RO(ORIGINAL_EVENT_DATA_HIGH, original_event_data)
+SHADOW_FIELD_RW(INJECTED_EVENT_DATA, injected_event_data)
+SHADOW_FIELD_RW(INJECTED_EVENT_DATA_HIGH, injected_event_data)

Please add shadow fields in a separate patch, with sufficient explain to justify why KVM needs to enable VMCS shadowing for the fields (it's purely a performance optimazation).

author maintainer
#98 lore raw

PATCH v9 20/22 KVM: nVMX: Validate FRED-related VMCS fields

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Extend nested VMX field validation to include FRED-specific VMCS fields, mirroring hardware behavior.

This enables support for nested FRED by ensuring control and guest/host state fields are properly checked.

---

Change in v5:
* Add TB from Xuelian Guo.

---
 arch/x86/kvm/vmx/nested.c | 117 +++++++++++++++++++++++++++++++++-----
 1 file changed, 104 insertions(+), 13 deletions(-)
arch/x86/kvm/vmx/nested.c +104 -13
index 63cdfffba58b..8682709d8759 100644
@@ -3030,6 +3030,8 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 					  struct vmcs12 *vmcs12)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	bool fred_enabled = (vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) &&
+			    (vmcs12->guest_cr4 & X86_CR4_FRED);
 
 	if (CC(!vmx_control_verify(vmcs12->vm_entry_controls,
 				    vmx->nested.msrs.entry_ctls_low,
@@ -3047,22 +3049,11 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 		u8 vector = intr_info & INTR_INFO_VECTOR_MASK;
 		u32 intr_type = intr_info & INTR_INFO_INTR_TYPE_MASK;
 		bool has_error_code = intr_info & INTR_INFO_DELIVER_CODE_MASK;
+		bool has_nested_exception = vmx->nested.msrs.basic & VMX_BASIC_NESTED_EXCEPTION;
 		bool urg = nested_cpu_has2(vmcs12,
 					   SECONDARY_EXEC_UNRESTRICTED_GUEST);
 		bool prot_mode = !urg || vmcs12->guest_cr0 & X86_CR0_PE;
 
-		/* VM-entry interruption-info field: interruption type */
-		if (CC(intr_type == INTR_TYPE_RESERVED) ||
-		    CC(intr_type == INTR_TYPE_OTHER_EVENT &&
-		       !nested_cpu_supports_monitor_trap_flag(vcpu)))
-			return -EINVAL;
-
-		/* VM-entry interruption-info field: vector */
-		if (CC(intr_type == INTR_TYPE_NMI_INTR && vector != NMI_VECTOR) ||
-		    CC(intr_type == INTR_TYPE_HARD_EXCEPTION && vector > 31) ||
-		    CC(intr_type == INTR_TYPE_OTHER_EVENT && vector != 0))
-			return -EINVAL;
-
 		/*
 		 * Cannot deliver error code in real mode or if the interrupt
 		 * type is not hardware exception. For other cases, do the
@@ -3086,8 +3077,28 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 		if (CC(intr_info & INTR_INFO_RESVD_BITS_MASK))
 			return -EINVAL;
 
-		/* VM-entry instruction length */
+		/*
+		 * When the CPU enumerates VMX nested-exception support, bit 13
+		 * (set to indicate a nested exception) of the intr info field
+		 * may have value 1.  Otherwise bit 13 is reserved.
+		 */
+		if (CC(!(has_nested_exception && intr_type == INTR_TYPE_HARD_EXCEPTION) &&
+		       intr_info & INTR_INFO_NESTED_EXCEPTION_MASK))
+			return -EINVAL;
+
 		switch (intr_type) {
+		case INTR_TYPE_EXT_INTR:
+			break;
+		case INTR_TYPE_RESERVED:
+			return -EINVAL;
+		case INTR_TYPE_NMI_INTR:
+			if (CC(vector != NMI_VECTOR))
+				return -EINVAL;
+			break;
+		case INTR_TYPE_HARD_EXCEPTION:
+			if (CC(vector > 31))
+				return -EINVAL;
+			break;
 		case INTR_TYPE_SOFT_EXCEPTION:
 		case INTR_TYPE_SOFT_INTR:
 		case INTR_TYPE_PRIV_SW_EXCEPTION:
@@ -3095,6 +3106,24 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 			    CC(vmcs12->vm_entry_instruction_len == 0 &&
 			    CC(!nested_cpu_has_zero_length_injection(vcpu))))
 				return -EINVAL;
+			break;
+		case INTR_TYPE_OTHER_EVENT:
+			switch (vector) {
+			case 0:
+				if (CC(!nested_cpu_supports_monitor_trap_flag(vcpu)))
+					return -EINVAL;
+				break;
+			case 1:
+			case 2:
+				if (CC(!fred_enabled))
+					return -EINVAL;
+				if (CC(vmcs12->vm_entry_instruction_len > X86_MAX_INSTRUCTION_LENGTH))
+					return -EINVAL;
+				break;
+			default:
+				return -EINVAL;
+			}
+			break;
 		}
 	}
 
@@ -3213,9 +3242,29 @@ static int nested_vmx_check_host_state(struct kvm_vcpu *vcpu,
 	if (ia32e) {
 		if (CC(!(vmcs12->host_cr4 & X86_CR4_PAE)))
 			return -EINVAL;
+		if (vmcs12->vm_exit_controls & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS &&
+		    vmcs12->secondary_vm_exit_controls & SECONDARY_VM_EXIT_LOAD_IA32_FRED) {
+			if (CC(vmcs12->host_ia32_fred_config &
+			       (BIT_ULL(11) | GENMASK_ULL(5, 4) | BIT_ULL(2))) ||
+			    CC(vmcs12->host_ia32_fred_rsp1 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->host_ia32_fred_rsp2 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->host_ia32_fred_rsp3 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->host_ia32_fred_ssp1 & GENMASK_ULL(2, 0)) ||
+			    CC(vmcs12->host_ia32_fred_ssp2 & GENMASK_ULL(2, 0)) ||
+			    CC(vmcs12->host_ia32_fred_ssp3 & GENMASK_ULL(2, 0)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_config & PAGE_MASK, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp1, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp2, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp3, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp1, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp2, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp3, vcpu)))
+				return -EINVAL;
+		}
 	} else {
 		if (CC(vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) ||
 		    CC(vmcs12->host_cr4 & X86_CR4_PCIDE) ||
+		    CC(vmcs12->host_cr4 & X86_CR4_FRED) ||
 		    CC((vmcs12->host_rip) >> 32))
 			return -EINVAL;
 	}
@@ -3384,6 +3433,48 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu *vcpu,
 	     CC((vmcs12->guest_bndcfgs & MSR_IA32_BNDCFGS_RSVD))))
 		return -EINVAL;
 
+	if (ia32e) {
+		if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_FRED) {
+			if (CC(vmcs12->guest_ia32_fred_config &
+			       (BIT_ULL(11) | GENMASK_ULL(5, 4) | BIT_ULL(2))) ||
+			    CC(vmcs12->guest_ia32_fred_rsp1 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->guest_ia32_fred_rsp2 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->guest_ia32_fred_rsp3 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->guest_ia32_fred_ssp1 & GENMASK_ULL(2, 0)) ||
+			    CC(vmcs12->guest_ia32_fred_ssp2 & GENMASK_ULL(2, 0)) ||
+			    CC(vmcs12->guest_ia32_fred_ssp3 & GENMASK_ULL(2, 0)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_config & PAGE_MASK, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp1, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp2, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp3, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp1, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp2, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp3, vcpu)))
+				return -EINVAL;
+		}
+		if (vmcs12->guest_cr4 & X86_CR4_FRED) {
+			unsigned int ss_dpl = VMX_AR_DPL(vmcs12->guest_ss_ar_bytes);
+			switch (ss_dpl) {
+			case 0:
+				if (CC(!(vmcs12->guest_cs_ar_bytes & VMX_AR_L_MASK)))
+					return -EINVAL;
+				break;
+			case 1:
+			case 2:
+				return -EINVAL;
+			case 3:
+				if (CC(vmcs12->guest_rflags & X86_EFLAGS_IOPL))
+					return -EINVAL;
+				if (CC(vmcs12->guest_interruptibility_info & GUEST_INTR_STATE_STI))
+					return -EINVAL;
+				break;
+			}
+		}
+	} else {
+		if (CC(vmcs12->guest_cr4 & X86_CR4_FRED))
+			return -EINVAL;
+	}
+
 	if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_CET_STATE) {
 		if (nested_vmx_check_cet_state_common(vcpu, vmcs12->guest_s_cet,
 						      vmcs12->guest_ssp,
Signature · 1 line
2.51.0
Reviewed-by on v9 20/22
#99 lore raw

Re: [PATCH v9 20/22] KVM: nVMX: Validate FRED-related VMCS fields

Quoted 5 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Extend nested VMX field validation to include FRED-specific VMCS fields, mirroring hardware behavior.

Quoted 7 lines

This enables support for nested FRED by ensuring control and guest/host state fields are properly checked.

There are some minor issues below that may need to be fixed.

Quoted 22 lines

---

Change in v5:
* Add TB from Xuelian Guo.
---
arch/x86/kvm/vmx/nested.c | 117 +++++++++++++++++++++++++++++++++-----

1 file changed, 104 insertions(+), 13 deletions(-)
arch/x86/kvm/vmx/nested.c +2 -0
index 63cdfffba58b..8682709d8759 100644
@@ -3030,6 +3030,8 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 					  struct vmcs12 *vmcs12)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	bool fred_enabled = (vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) &&
+			    (vmcs12->guest_cr4 & X86_CR4_FRED);
 
 	if (CC(!vmx_control_verify(vmcs12->vm_entry_controls,
 				    vmx->nested.msrs.entry_ctls_low,

arch/x86/kvm/vmx/nested.c static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3052

Quoted 4 lines
arch/x86/kvm/vmx/nested.c +1 -0
@@ -3047,22 +3049,11 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 		u8 vector = intr_info & INTR_INFO_VECTOR_MASK;
 		u32 intr_type = intr_info & INTR_INFO_INTR_TYPE_MASK;
 		bool has_error_code = intr_info & INTR_INFO_DELIVER_CODE_MASK;
+		bool has_nested_exception = vmx->nested.msrs.basic & VMX_BASIC_NESTED_EXCEPTION;

has_error_code reflects whether the to-be-injected event has an error code. Using has_nested_exception for CPU capabilities here is a bit confusing.

Quoted 26 lines
arch/x86/kvm/vmx/nested.c +2 -13
 		bool urg = nested_cpu_has2(vmcs12,
 					   SECONDARY_EXEC_UNRESTRICTED_GUEST);
 		bool prot_mode = !urg || vmcs12->guest_cr0 & X86_CR0_PE;
 
-		/* VM-entry interruption-info field: interruption type */
-		if (CC(intr_type == INTR_TYPE_RESERVED) ||
-		    CC(intr_type == INTR_TYPE_OTHER_EVENT &&
-		       !nested_cpu_supports_monitor_trap_flag(vcpu)))
-			return -EINVAL;
-
-		/* VM-entry interruption-info field: vector */
-		if (CC(intr_type == INTR_TYPE_NMI_INTR && vector != NMI_VECTOR) ||
-		    CC(intr_type == INTR_TYPE_HARD_EXCEPTION && vector > 31) ||
-		    CC(intr_type == INTR_TYPE_OTHER_EVENT && vector != 0))
-			return -EINVAL;
-
 		/*
 		 * Cannot deliver error code in real mode or if the interrupt
 		 * type is not hardware exception. For other cases, do the
@@ -3086,8 +3077,28 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 		if (CC(intr_info & INTR_INFO_RESVD_BITS_MASK))
 			return -EINVAL;
 
-		/* VM-entry instruction length */
+		/*
+		 * When the CPU enumerates VMX nested-exception support, bit 13

arch/x86/kvm/vmx/nested.c static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3091

Quoted 10 lines
arch/x86/kvm/vmx/nested.c +9 -0
+		 * (set to indicate a nested exception) of the intr info field
+		 * may have value 1.  Otherwise bit 13 is reserved.
+		 */
+		if (CC(!(has_nested_exception && intr_type == INTR_TYPE_HARD_EXCEPTION) &&
+		       intr_info & INTR_INFO_NESTED_EXCEPTION_MASK))
+			return -EINVAL;
+
 		switch (intr_type) {
+		case INTR_TYPE_EXT_INTR:
+			break;

This can be dropped, as the "default" case will handle it.

arch/x86/kvm/vmx/nested.c static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3093

Quoted 2 lines
arch/x86/kvm/vmx/nested.c +2 -0
+		case INTR_TYPE_RESERVED:
+			return -EINVAL;

I think we need to add a CC() statement to make it easier to correlate a VM-entry failure with a specific consistency check.

Quoted 12 lines
arch/x86/kvm/vmx/nested.c +8 -0
+		case INTR_TYPE_NMI_INTR:
+			if (CC(vector != NMI_VECTOR))
+				return -EINVAL;
+			break;
+		case INTR_TYPE_HARD_EXCEPTION:
+			if (CC(vector > 31))
+				return -EINVAL;
+			break;
 		case INTR_TYPE_SOFT_EXCEPTION:
 		case INTR_TYPE_SOFT_INTR:
 		case INTR_TYPE_PRIV_SW_EXCEPTION:

arch/x86/kvm/vmx/nested.c static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3114

Quoted 9 lines
arch/x86/kvm/vmx/nested.c +6 -0
@@ -3095,6 +3106,24 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 			    CC(vmcs12->vm_entry_instruction_len == 0 &&
 			    CC(!nested_cpu_has_zero_length_injection(vcpu))))
 				return -EINVAL;
+			break;
+		case INTR_TYPE_OTHER_EVENT:
+			switch (vector) {
+			case 0:
+				if (CC(!nested_cpu_supports_monitor_trap_flag(vcpu)))
+					return -EINVAL;

Does this nested_cpu_supports_monitor_trap_flag() check apply to case 1/2?

arch/x86/kvm/vmx/nested.c static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3124

Quoted 10 lines
arch/x86/kvm/vmx/nested.c +10 -0
+				break;
+			case 1:
+			case 2:
+				if (CC(!fred_enabled))
+					return -EINVAL;
+				if (CC(vmcs12->vm_entry_instruction_len > X86_MAX_INSTRUCTION_LENGTH))
+					return -EINVAL;
+				break;
+			default:
+				return -EINVAL;

Again, I think -EINVAL should be accompanied by a CC() statement.

Quoted 58 lines
arch/x86/kvm/vmx/nested.c +41 -0
+			}
+			break;
 		}
 	}
 
@@ -3213,9 +3242,29 @@ static int nested_vmx_check_host_state(struct kvm_vcpu *vcpu,
 	if (ia32e) {
 		if (CC(!(vmcs12->host_cr4 & X86_CR4_PAE)))
 			return -EINVAL;
+		if (vmcs12->vm_exit_controls & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS &&
+		    vmcs12->secondary_vm_exit_controls & SECONDARY_VM_EXIT_LOAD_IA32_FRED) {
+			if (CC(vmcs12->host_ia32_fred_config &
+			       (BIT_ULL(11) | GENMASK_ULL(5, 4) | BIT_ULL(2))) ||
+			    CC(vmcs12->host_ia32_fred_rsp1 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->host_ia32_fred_rsp2 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->host_ia32_fred_rsp3 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->host_ia32_fred_ssp1 & GENMASK_ULL(2, 0)) ||
+			    CC(vmcs12->host_ia32_fred_ssp2 & GENMASK_ULL(2, 0)) ||
+			    CC(vmcs12->host_ia32_fred_ssp3 & GENMASK_ULL(2, 0)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_config & PAGE_MASK, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp1, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp2, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp3, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp1, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp2, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp3, vcpu)))
+				return -EINVAL;
+		}
 	} else {
 		if (CC(vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) ||
 		    CC(vmcs12->host_cr4 & X86_CR4_PCIDE) ||
+		    CC(vmcs12->host_cr4 & X86_CR4_FRED) ||
 		    CC((vmcs12->host_rip) >> 32))
 			return -EINVAL;
 	}
@@ -3384,6 +3433,48 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu *vcpu,
 	     CC((vmcs12->guest_bndcfgs & MSR_IA32_BNDCFGS_RSVD))))
 		return -EINVAL;
 
+	if (ia32e) {
+		if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_FRED) {
+			if (CC(vmcs12->guest_ia32_fred_config &
+			       (BIT_ULL(11) | GENMASK_ULL(5, 4) | BIT_ULL(2))) ||
+			    CC(vmcs12->guest_ia32_fred_rsp1 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->guest_ia32_fred_rsp2 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->guest_ia32_fred_rsp3 & GENMASK_ULL(5, 0)) ||
+			    CC(vmcs12->guest_ia32_fred_ssp1 & GENMASK_ULL(2, 0)) ||
+			    CC(vmcs12->guest_ia32_fred_ssp2 & GENMASK_ULL(2, 0)) ||
+			    CC(vmcs12->guest_ia32_fred_ssp3 & GENMASK_ULL(2, 0)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_config & PAGE_MASK, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp1, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp2, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp3, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp1, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp2, vcpu)) ||
+			    CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp3, vcpu)))
+				return -EINVAL;
+		}

arch/x86/kvm/vmx/nested.c static int nested_vmx_check_guest_state(struct kvm_vcpu *vcpu, · line 3464

Quoted 10 lines
arch/x86/kvm/vmx/nested.c +10 -0
+		if (vmcs12->guest_cr4 & X86_CR4_FRED) {
+			unsigned int ss_dpl = VMX_AR_DPL(vmcs12->guest_ss_ar_bytes);
+			switch (ss_dpl) {
+			case 0:
+				if (CC(!(vmcs12->guest_cs_ar_bytes & VMX_AR_L_MASK)))
+					return -EINVAL;
+				break;
+			case 1:
+			case 2:
+				return -EINVAL;

Ditto.

Quoted 20 lines
arch/x86/kvm/vmx/nested.c +13 -0
+			case 3:
+				if (CC(vmcs12->guest_rflags & X86_EFLAGS_IOPL))
+					return -EINVAL;
+				if (CC(vmcs12->guest_interruptibility_info & GUEST_INTR_STATE_STI))
+					return -EINVAL;
+				break;
+			}
+		}
+	} else {
+		if (CC(vmcs12->guest_cr4 & X86_CR4_FRED))
+			return -EINVAL;
+	}
+
 	if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_CET_STATE) {
 		if (nested_vmx_check_cet_state_common(vcpu, vmcs12->guest_s_cet,
 						      vmcs12->guest_ssp,
Signature · 1 line
2.51.0
author maintainer
#100 lore raw

Re: [PATCH v9 20/22] KVM: nVMX: Validate FRED-related VMCS fields

Quoted 8 lines

arch/x86/kvm/vmx/nested.c static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3052

Quoted 5 lines
arch/x86/kvm/vmx/nested.c +1 -0
@@ -3047,22 +3049,11 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 u8 vector = intr_info & INTR_INFO_VECTOR_MASK;
 u32 intr_type = intr_info & INTR_INFO_INTR_TYPE_MASK;
 bool has_error_code = intr_info & INTR_INFO_DELIVER_CODE_MASK;
+ bool has_nested_exception = vmx->nested.msrs.basic & VMX_BASIC_NESTED_EXCEPTION;

has_error_code reflects whether the to-be-injected event has an error code. Using has_nested_exception for CPU capabilities here is a bit confusing.

Looks better to just remove has_error_code.

Quoted 24 lines
Quoted 24 lines
arch/x86/kvm/vmx/nested.c +0 -12
 bool urg = nested_cpu_has2(vmcs12,
   SECONDARY_EXEC_UNRESTRICTED_GUEST);
 bool prot_mode = !urg || vmcs12->guest_cr0 & X86_CR0_PE;
 
- /* VM-entry interruption-info field: interruption type */
- if (CC(intr_type == INTR_TYPE_RESERVED) ||
-     CC(intr_type == INTR_TYPE_OTHER_EVENT &&
-        !nested_cpu_supports_monitor_trap_flag(vcpu)))
- return -EINVAL;
-
- /* VM-entry interruption-info field: vector */
- if (CC(intr_type == INTR_TYPE_NMI_INTR && vector != NMI_VECTOR) ||
-     CC(intr_type == INTR_TYPE_HARD_EXCEPTION && vector > 31) ||
-     CC(intr_type == INTR_TYPE_OTHER_EVENT && vector != 0))
- return -EINVAL;
-
 /*
 * Cannot deliver error code in real mode or if the interrupt
 * type is not hardware exception. For other cases, do the
@@ -3086,8 +3077,28 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 if (CC(intr_info & INTR_INFO_RESVD_BITS_MASK))
 return -EINVAL;
 
Quoted 15 lines

arch/x86/kvm/vmx/nested.c static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3091

Quoted 13 lines
arch/x86/kvm/vmx/nested.c +11 -1
- /* VM-entry instruction length */
+ /*
+  * When the CPU enumerates VMX nested-exception support, bit 13
+  * (set to indicate a nested exception) of the intr info field
+  * may have value 1.  Otherwise bit 13 is reserved.
+  */
+ if (CC(!(has_nested_exception && intr_type == INTR_TYPE_HARD_EXCEPTION) &&
+        intr_info & INTR_INFO_NESTED_EXCEPTION_MASK))
+ return -EINVAL;
+
 switch (intr_type) {
+ case INTR_TYPE_EXT_INTR:
+ break;

This can be dropped, as the "default" case will handle it.

We don’t have a default case, as all 8 cases are listed (INTR_INFO_INTR_TYPE_MASK is 0x700).

Quoted 6 lines

Review static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3093

Quoted 3 lines
(quoted diff) +2 -0
+ case INTR_TYPE_RESERVED:
+ return -EINVAL;

I think we need to add a CC() statement to make it easier to correlate a VM-entry failure with a specific consistency check.

What do you want me to put in CC()?

CC(intr_type == INTR_TYPE_RESERVED)?

Quoted 16 lines
Quoted 16 lines
arch/x86/kvm/vmx/nested.c +8 -0
+ case INTR_TYPE_NMI_INTR:
+ if (CC(vector != NMI_VECTOR))
+ return -EINVAL;
+ break;
+ case INTR_TYPE_HARD_EXCEPTION:
+ if (CC(vector > 31))
+ return -EINVAL;
+ break;

case INTR_TYPE_SOFT_EXCEPTION:
case INTR_TYPE_SOFT_INTR:
case INTR_TYPE_PRIV_SW_EXCEPTION:

arch/x86/kvm/vmx/nested.c +0 -0
@@ -3095,6 +3106,24 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
    CC(vmcs12->vm_entry_instruction_len == 0 &&
    CC(!nested_cpu_has_zero_length_injection(vcpu))))
 return -EINVAL;
Quoted 8 lines

arch/x86/kvm/vmx/nested.c static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3114

Quoted 6 lines
arch/x86/kvm/vmx/nested.c +6 -0
+ break;
+ case INTR_TYPE_OTHER_EVENT:
+ switch (vector) {
+ case 0:
+ if (CC(!nested_cpu_supports_monitor_trap_flag(vcpu)))
+ return -EINVAL;

Does this nested_cpu_supports_monitor_trap_flag() check apply to case 1/2?

I did check the spec when writing the code but I will doublecheck.

Quoted 103 lines

arch/x86/kvm/vmx/nested.c static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3124

Quoted 11 lines
arch/x86/kvm/vmx/nested.c +10 -0
+ break;
+ case 1:
+ case 2:
+ if (CC(!fred_enabled))
+ return -EINVAL;
+ if (CC(vmcs12->vm_entry_instruction_len > X86_MAX_INSTRUCTION_LENGTH))
+ return -EINVAL;
+ break;
+ default:
+ return -EINVAL;

Again, I think -EINVAL should be accompanied by a CC() statement.

Quoted 58 lines
arch/x86/kvm/vmx/nested.c +21 -0
+ }
+ break;
 }
 }
 
@@ -3213,9 +3242,29 @@ static int nested_vmx_check_host_state(struct kvm_vcpu *vcpu,
 if (ia32e) {
 if (CC(!(vmcs12->host_cr4 & X86_CR4_PAE)))
 return -EINVAL;
+ if (vmcs12->vm_exit_controls & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS &&
+     vmcs12->secondary_vm_exit_controls & SECONDARY_VM_EXIT_LOAD_IA32_FRED) {
+ if (CC(vmcs12->host_ia32_fred_config &
+        (BIT_ULL(11) | GENMASK_ULL(5, 4) | BIT_ULL(2))) ||
+     CC(vmcs12->host_ia32_fred_rsp1 & GENMASK_ULL(5, 0)) ||
+     CC(vmcs12->host_ia32_fred_rsp2 & GENMASK_ULL(5, 0)) ||
+     CC(vmcs12->host_ia32_fred_rsp3 & GENMASK_ULL(5, 0)) ||
+     CC(vmcs12->host_ia32_fred_ssp1 & GENMASK_ULL(2, 0)) ||
+     CC(vmcs12->host_ia32_fred_ssp2 & GENMASK_ULL(2, 0)) ||
+     CC(vmcs12->host_ia32_fred_ssp3 & GENMASK_ULL(2, 0)) ||
+     CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_config & PAGE_MASK, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp1, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp2, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp3, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp1, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp2, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp3, vcpu)))
+ return -EINVAL;
+ }
 } else {

if (CC(vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) ||

    CC(vmcs12->host_cr4 & X86_CR4_PCIDE) ||
+     CC(vmcs12->host_cr4 & X86_CR4_FRED) ||
    CC((vmcs12->host_rip) >> 32))

return -EINVAL;
}

arch/x86/kvm/vmx/nested.c +19 -0
@@ -3384,6 +3433,48 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu *vcpu,
     CC((vmcs12->guest_bndcfgs & MSR_IA32_BNDCFGS_RSVD))))
 return -EINVAL;
 
+ if (ia32e) {
+ if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_FRED) {
+ if (CC(vmcs12->guest_ia32_fred_config &
+        (BIT_ULL(11) | GENMASK_ULL(5, 4) | BIT_ULL(2))) ||
+     CC(vmcs12->guest_ia32_fred_rsp1 & GENMASK_ULL(5, 0)) ||
+     CC(vmcs12->guest_ia32_fred_rsp2 & GENMASK_ULL(5, 0)) ||
+     CC(vmcs12->guest_ia32_fred_rsp3 & GENMASK_ULL(5, 0)) ||
+     CC(vmcs12->guest_ia32_fred_ssp1 & GENMASK_ULL(2, 0)) ||
+     CC(vmcs12->guest_ia32_fred_ssp2 & GENMASK_ULL(2, 0)) ||
+     CC(vmcs12->guest_ia32_fred_ssp3 & GENMASK_ULL(2, 0)) ||
+     CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_config & PAGE_MASK, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp1, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp2, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp3, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp1, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp2, vcpu)) ||
+     CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp3, vcpu)))
+ return -EINVAL;
+ }

arch/x86/kvm/vmx/nested.c static int nested_vmx_check_guest_state(struct kvm_vcpu *vcpu, · line 3464

Quoted 10 lines
arch/x86/kvm/vmx/nested.c +10 -0
+ if (vmcs12->guest_cr4 & X86_CR4_FRED) {
+ unsigned int ss_dpl = VMX_AR_DPL(vmcs12->guest_ss_ar_bytes);
+ switch (ss_dpl) {
+ case 0:
+ if (CC(!(vmcs12->guest_cs_ar_bytes & VMX_AR_L_MASK)))
+ return -EINVAL;
+ break;
+ case 1:
+ case 2:
+ return -EINVAL;

Ditto.

Quoted 18 lines
arch/x86/kvm/vmx/nested.c +13 -0
+ case 3:
+ if (CC(vmcs12->guest_rflags & X86_EFLAGS_IOPL))
+ return -EINVAL;
+ if (CC(vmcs12->guest_interruptibility_info & GUEST_INTR_STATE_STI))
+ return -EINVAL;
+ break;
+ }
+ }
+ } else {
+ if (CC(vmcs12->guest_cr4 & X86_CR4_FRED))
+ return -EINVAL;
+ }
+
 if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_CET_STATE) {
 if (nested_vmx_check_cet_state_common(vcpu, vmcs12->guest_s_cet,
      vmcs12->guest_ssp,
Signature · 1 line
2.51.0
#101 lore raw

Re: [PATCH v9 20/22] KVM: nVMX: Validate FRED-related VMCS fields

Quoted 57 lines · Xin Li
Quoted 8 lines

Review static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3052

Quoted 5 lines
(quoted diff) +1 -0
@@ -3047,22 +3049,11 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 u8 vector = intr_info & INTR_INFO_VECTOR_MASK;
 u32 intr_type = intr_info & INTR_INFO_INTR_TYPE_MASK;
 bool has_error_code = intr_info & INTR_INFO_DELIVER_CODE_MASK;
+ bool has_nested_exception = vmx->nested.msrs.basic & VMX_BASIC_NESTED_EXCEPTION;

has_error_code reflects whether the to-be-injected event has an error code. Using has_nested_exception for CPU capabilities here is a bit confusing.

Looks better to just remove has_error_code.

Quoted 24 lines
Quoted 24 lines
(quoted diff) +0 -12
 bool urg = nested_cpu_has2(vmcs12,
   SECONDARY_EXEC_UNRESTRICTED_GUEST);
 bool prot_mode = !urg || vmcs12->guest_cr0 & X86_CR0_PE;
 
- /* VM-entry interruption-info field: interruption type */
- if (CC(intr_type == INTR_TYPE_RESERVED) ||
-     CC(intr_type == INTR_TYPE_OTHER_EVENT &&
-        !nested_cpu_supports_monitor_trap_flag(vcpu)))
- return -EINVAL;
-
- /* VM-entry interruption-info field: vector */
- if (CC(intr_type == INTR_TYPE_NMI_INTR && vector != NMI_VECTOR) ||
-     CC(intr_type == INTR_TYPE_HARD_EXCEPTION && vector > 31) ||
-     CC(intr_type == INTR_TYPE_OTHER_EVENT && vector != 0))
- return -EINVAL;
-
 /*
 * Cannot deliver error code in real mode or if the interrupt
 * type is not hardware exception. For other cases, do the
@@ -3086,8 +3077,28 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
 if (CC(intr_info & INTR_INFO_RESVD_BITS_MASK))
 return -EINVAL;
 
Quoted 15 lines

Review static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu, · line 3091

Quoted 13 lines
(quoted diff) +11 -1
- /* VM-entry instruction length */
+ /*
+  * When the CPU enumerates VMX nested-exception support, bit 13
+  * (set to indicate a nested exception) of the intr info field
+  * may have value 1.  Otherwise bit 13 is reserved.
+  */
+ if (CC(!(has_nested_exception && intr_type == INTR_TYPE_HARD_EXCEPTION) &&
+        intr_info & INTR_INFO_NESTED_EXCEPTION_MASK))
+ return -EINVAL;
+
 switch (intr_type) {
+ case INTR_TYPE_EXT_INTR:
+ break;

This can be dropped, as the "default" case will handle it.

We don’t have a default case, as all 8 cases are listed (INTR_INFO_INTR_TYPE_MASK is 0x700).

Quoted 4 lines
Quoted 3 lines
(quoted diff) +2 -0
+ case INTR_TYPE_RESERVED:
+ return -EINVAL;
Quoted 6 lines
Quoted 2 lines

I think we need to add a CC() statement to make it easier to correlate a VM-entry failure with a specific consistency check.

What do you want me to put in CC()?

CC(intr_type == INTR_TYPE_RESERVED)?

how about this incremental change?

I prefer to make has_error_code and has_nested_exception consistent, and add a CC() statement before all "return -EINVAL" statements for debugging.

t a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 8682709d8759..f13df70405d9 100644

arch/x86/kvm/vmx/nested.c +0 -0
@@ -3049,7 +3049,7 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
		u8 vector = intr_info & INTR_INFO_VECTOR_MASK;
		u32 intr_type = intr_info & INTR_INFO_INTR_TYPE_MASK;
		bool has_error_code = intr_info & INTR_INFO_DELIVER_CODE_MASK;
-		bool has_nested_exception = vmx->nested.msrs.basic & VMX_BASIC_NESTED_EXCEPTION;
+		bool has_nested_exception = intr_info & INTR_INFO_NESTED_EXCEPTION_MASK;
		bool urg = nested_cpu_has2(vmcs12,
					   SECONDARY_EXEC_UNRESTRICTED_GUEST);
		bool prot_mode = !urg || vmcs12->guest_cr0 & X86_CR0_PE;
@@ -3077,20 +3077,10 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
		if (CC(intr_info & INTR_INFO_RESVD_BITS_MASK))
			return -EINVAL;

- /*
- * When the CPU enumerates VMX nested-exception support, bit 13
- * (set to indicate a nested exception) of the intr info field
- * may have value 1. Otherwise bit 13 is reserved.
- */
- if (CC(!(has_nested_exception && intr_type == INTR_TYPE_HARD_EXCEPTION) &&
- intr_info & INTR_INFO_NESTED_EXCEPTION_MASK))
+ if (CC(intr_type == INTR_TYPE_RESERVED))

			return -EINVAL;
 
		switch (intr_type) {
-		case INTR_TYPE_EXT_INTR:
-			break;
-		case INTR_TYPE_RESERVED:
-			return -EINVAL;
		case INTR_TYPE_NMI_INTR:
			if (CC(vector != NMI_VECTOR))
				return -EINVAL;
@@ -3098,6 +3088,13 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
		case INTR_TYPE_HARD_EXCEPTION:
			if (CC(vector > 31))
				return -EINVAL;

+ /*
+ * When the CPU enumerates VMX nested-exception support, bit 13
+ * (set to indicate a nested exception) of the intr info field
+ * may have value 1. Otherwise bit 13 is reserved.
+ */
+ if (CC(has_nested_exception && !(vmx->nested.msrs.basic & VMX_BASIC_NESTED_EXCEPTION)))

+				return -EINVAL;
			break;
		case INTR_TYPE_SOFT_EXCEPTION:
		case INTR_TYPE_SOFT_INTR:
@@ -3108,6 +3105,9 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
				return -EINVAL;
			break;
		case INTR_TYPE_OTHER_EVENT:
+			if (CC(vector > 3))
+				return -EINVAL;
+
			switch (vector) {
			case 0:
				if (CC(!nested_cpu_supports_monitor_trap_flag(vcpu)))
@@ -3121,7 +3121,7 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu *vcpu,
					return -EINVAL;
				break;
			default:
-				return -EINVAL;
+				break;
			}
			break;
		}
@@ -3454,14 +3454,15 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu *vcpu,
		}
		if (vmcs12->guest_cr4 & X86_CR4_FRED) {
			unsigned int ss_dpl = VMX_AR_DPL(vmcs12->guest_ss_ar_bytes);

+
+ if (CC(ss_dpl == 1 || ss_dpl == 2))
+ return -EINVAL;
+

			switch (ss_dpl) {
			case 0:
				if (CC(!(vmcs12->guest_cs_ar_bytes & VMX_AR_L_MASK)))
					return -EINVAL;
				break;
-			case 1:
-			case 2:
-				return -EINVAL;
			case 3:
				if (CC(vmcs12->guest_rflags & X86_EFLAGS_IOPL))
					return -EINVAL;
author maintainer
#102 lore raw

PATCH v9 21/22 KVM: nVMX: Guard SHADOW_FIELD_R[OW] macros with VMX feature checks

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Add VMX feature checks to the SHADOW_FIELD_R[OW] macros to prevent access to VMCS fields that may be unsupported on some CPUs.

Functions like copy_shadow_to_vmcs12() and copy_vmcs12_to_shadow() access VMCS fields that may not exist on certain hardware, such as INJECTED_EVENT_DATA. To avoid VMREAD/VMWRITE warnings, skip syncing fields tied to unsupported VMX features.

---

Change in v5:
* Add TB from Xuelian Guo.

Change since v2:
* Add __SHADOW_FIELD_R[OW] for better readability or maintability (Sean).

---
 arch/x86/kvm/vmx/nested.c             | 79 +++++++++++++++++++--------
 arch/x86/kvm/vmx/vmcs_shadow_fields.h | 41 +++++++++-----
 2 files changed, 83 insertions(+), 37 deletions(-)
arch/x86/kvm/vmx/nested.c +55 -24
index 8682709d8759..37ab8250dd31 100644
@@ -55,14 +55,14 @@ struct shadow_vmcs_field {
 	u16	offset;
 };
 static struct shadow_vmcs_field shadow_read_only_fields[] = {
-#define SHADOW_FIELD_RO(x, y) { x, offsetof(struct vmcs12, y) },
+#define __SHADOW_FIELD_RO(x, y, c) { x, offsetof(struct vmcs12, y) },
 #include "vmcs_shadow_fields.h"
 };
 static int max_shadow_read_only_fields =
 	ARRAY_SIZE(shadow_read_only_fields);
 
 static struct shadow_vmcs_field shadow_read_write_fields[] = {
-#define SHADOW_FIELD_RW(x, y) { x, offsetof(struct vmcs12, y) },
+#define __SHADOW_FIELD_RW(x, y, c) { x, offsetof(struct vmcs12, y) },
 #include "vmcs_shadow_fields.h"
 };
 static int max_shadow_read_write_fields =
@@ -85,6 +85,17 @@ static void init_vmcs_shadow_fields(void)
 			pr_err("Missing field from shadow_read_only_field %x\n",
 			       field + 1);
 
+		switch (field) {
+#define __SHADOW_FIELD_RO(x, y, c)		\
+		case x:				\
+			if (!(c))		\
+				continue;	\
+			break;
+#include "vmcs_shadow_fields.h"
+		default:
+			break;
+		}
+
 		clear_bit(field, vmx_vmread_bitmap);
 		if (field & 1)
 #ifdef CONFIG_X86_64
@@ -110,24 +121,13 @@ static void init_vmcs_shadow_fields(void)
 			  field <= GUEST_TR_AR_BYTES,
 			  "Update vmcs12_write_any() to drop reserved bits from AR_BYTES");
 
-		/*
-		 * PML and the preemption timer can be emulated, but the
-		 * processor cannot vmwrite to fields that don't exist
-		 * on bare metal.
-		 */
 		switch (field) {
-		case GUEST_PML_INDEX:
-			if (!cpu_has_vmx_pml())
-				continue;
-			break;
-		case VMX_PREEMPTION_TIMER_VALUE:
-			if (!cpu_has_vmx_preemption_timer())
-				continue;
-			break;
-		case GUEST_INTR_STATUS:
-			if (!cpu_has_vmx_apicv())
-				continue;
+#define __SHADOW_FIELD_RW(x, y, c)		\
+		case x:				\
+			if (!(c))		\
+				continue;	\
 			break;
+#include "vmcs_shadow_fields.h"
 		default:
 			break;
 		}
@@ -1636,8 +1636,8 @@ int vmx_get_vmx_msr(struct nested_vmx_msrs *msrs, u32 msr_index, u64 *pdata)
 /*
  * Copy the writable VMCS shadow fields back to the VMCS12, in case they have
  * been modified by the L1 guest.  Note, "writable" in this context means
- * "writable by the guest", i.e. tagged SHADOW_FIELD_RW; the set of
- * fields tagged SHADOW_FIELD_RO may or may not align with the "read-only"
+ * "writable by the guest", i.e. tagged __SHADOW_FIELD_RW; the set of
+ * fields tagged __SHADOW_FIELD_RO may or may not align with the "read-only"
  * VM-exit information fields (which are actually writable if the vCPU is
  * configured to support "VMWRITE to any supported field in the VMCS").
  */
@@ -1658,6 +1658,18 @@ static void copy_shadow_to_vmcs12(struct vcpu_vmx *vmx)
 
 	for (i = 0; i < max_shadow_read_write_fields; i++) {
 		field = shadow_read_write_fields[i];
+
+		switch (field.encoding) {
+#define __SHADOW_FIELD_RW(x, y, c)		\
+		case x:				\
+			if (!(c))		\
+				continue;	\
+			break;
+#include "vmcs_shadow_fields.h"
+		default:
+			break;
+		}
+
 		val = __vmcs_readl(field.encoding);
 		vmcs12_write_any(vmcs12, field.encoding, field.offset, val);
 	}
@@ -1692,6 +1704,23 @@ static void copy_vmcs12_to_shadow(struct vcpu_vmx *vmx)
 	for (q = 0; q < ARRAY_SIZE(fields); q++) {
 		for (i = 0; i < max_fields[q]; i++) {
 			field = fields[q][i];
+
+			switch (field.encoding) {
+#define __SHADOW_FIELD_RO(x, y, c)			\
+			case x:				\
+				if (!(c))		\
+					continue;	\
+				break;
+#define __SHADOW_FIELD_RW(x, y, c)			\
+			case x:				\
+				if (!(c))		\
+					continue;	\
+				break;
+#include "vmcs_shadow_fields.h"
+			default:
+				break;
+			}
+
 			val = vmcs12_read_any(vmcs12, field.encoding,
 					      field.offset);
 			__vmcs_writel(field.encoding, val);
@@ -5951,9 +5980,10 @@ static int handle_vmread(struct kvm_vcpu *vcpu)
 static bool is_shadow_field_rw(unsigned long field)
 {
 	switch (field) {
-#define SHADOW_FIELD_RW(x, y) case x:
+#define __SHADOW_FIELD_RW(x, y, c)	\
+	case x:				\
+		return c;
 #include "vmcs_shadow_fields.h"
-		return true;
 	default:
 		break;
 	}
@@ -5963,9 +5993,10 @@ static bool is_shadow_field_rw(unsigned long field)
 static bool is_shadow_field_ro(unsigned long field)
 {
 	switch (field) {
-#define SHADOW_FIELD_RO(x, y) case x:
+#define __SHADOW_FIELD_RO(x, y, c)	\
+	case x:				\
+		return c;
 #include "vmcs_shadow_fields.h"
-		return true;
 	default:
 		break;
 	}
arch/x86/kvm/vmx/vmcs_shadow_fields.h +28 -13

76 lines not on this page · raw message

Signature · 1 line
2.51.0
Reviewed-by on v9 21/22
#103 lore raw

Re: [PATCH v9 21/22] KVM: nVMX: Guard SHADOW_FIELD_R[OW] macros with VMX feature checks

Quoted 10 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Add VMX feature checks to the SHADOW_FIELD_R[OW] macros to prevent access to VMCS fields that may be unsupported on some CPUs.

Functions like copy_shadow_to_vmcs12() and copy_vmcs12_to_shadow() access VMCS fields that may not exist on certain hardware, such as INJECTED_EVENT_DATA. To avoid VMREAD/VMWRITE warnings, skip syncing fields tied to unsupported VMX features.

Quoted 4 lines
#104 lore raw

Re: [PATCH v9 21/22] KVM: nVMX: Guard SHADOW_FIELD_R[OW] macros with VMX feature checks

In reply to Xin Li (Intel)

+Yosry

Quoted 14 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Add VMX feature checks to the SHADOW_FIELD_R[OW] macros to prevent access to VMCS fields that may be unsupported on some CPUs.

Functions like copy_shadow_to_vmcs12() and copy_vmcs12_to_shadow() access VMCS fields that may not exist on certain hardware, such as INJECTED_EVENT_DATA. To avoid VMREAD/VMWRITE warnings, skip syncing fields tied to unsupported VMX features.

Quoted 7 lines

---

Change in v5:
* Add TB from Xuelian Guo.

Change since v2:
* Add __SHADOW_FIELD_R[OW] for better readability or maintability (Sean).

Coming back to this with fresh eyes, handling fields that conditionally exist _only_ for VMCS shadowing is somewhat ridiculous. For PML and the VMX preemption timer, the special case handling makes sense because the fields are emulated by KVM irrespective of hardware suport. But for fields that KVM doesn't emulate in software, e.g. GUEST_INTR_STATUS and the FRED fields, allowing accesses through emulated VMREAD/VMWRITE and then filtering out VMCS shadowing accesses is just us being stubborn.

I still 100% think that not restricting based on the virtual CPU model defined by userspace is the way to go[*], because that'd require an absurd amount of effort, complexity, and memory to solve a problem no one actually cares about. But updating KVM's array of vmcs12 fields once during kvm-intel.ko load isn't difficult, and would make KVM suck a little less when running on old hardware.

E.g. running the test_vmwrite_vmread KUT subtest on CPUs without TSC scaling still fails with the wonderful:

FAIL: VMX_VMCS_ENUM.MAX_INDEX expected: 19, actual: 17

due to QEMU (sanely) setting the max index to 17 (VMX preemption timer) when the virtual CPU model doesn't support TSC scaling.

And looking forward, we're going to have the same mess with FRED due QEMU (again, sanely) basing its

    if (f[FEAT_7_1_EAX] & CPUID_7_1_EAX_FRED) {
        /* FRED injected-event data (0x2052).  */
        kvm_msr_entry_add(cpu, MSR_IA32_VMX_VMCS_ENUM, 0x52);
    } else if (f[FEAT_VMX_EXIT_CTLS] &
               VMX_VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) {
        /* Secondary VM-exit controls (0x2044).  */
        kvm_msr_entry_add(cpu, MSR_IA32_VMX_VMCS_ENUM, 0x44);
    } else if (f[FEAT_VMX_SECONDARY_CTLS] & VMX_SECONDARY_EXEC_TSC_SCALING) {
        /* TSC multiplier (0x2032).  */
        kvm_msr_entry_add(cpu, MSR_IA32_VMX_VMCS_ENUM, 0x32);
    } else {
        /* Preemption timer (0x482E).  */
        kvm_msr_entry_add(cpu, MSR_IA32_VMX_VMCS_ENUM, 0x2E);
    }

KVM will still have virtualization holes, e.g. if userspace hides TSC scaling when running on hardware+KVM that supports TSC scaling, but as above I don't think that's a problem worth solving.

I'll post a patch (just need to test on bare metal) to sanitize vmcs12 fields, at which point FRED nVMX support shouldn't have to do anything special beyond noting the depending, i.e. it should only take a few lines of code.

[*] https://lore.kernel.org/all/[email protected]

author maintainer
#105 lore raw

PATCH v9 22/22 KVM: nVMX: Enable VMX FRED controls

In reply to Xin Li (Intel)

From: Xin Li <[email protected]>

Permit use of VMX FRED controls in nested VMX now that support for nested FRED is implemented.

---

Change in v5:
* Add TB from Xuelian Guo.

---
 arch/x86/kvm/vmx/nested.c | 5 +++--
 arch/x86/kvm/vmx/vmx.c    | 1 +
 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/nested.c +3 -2
index 37ab8250dd31..655257b34d15 100644
@@ -7397,7 +7397,8 @@ static void nested_vmx_setup_exit_ctls(struct vmcs_config *vmcs_conf,
 		 * advertise any feature in it to nVMX until its nVMX support
 		 * is ready.
 		 */
-		msrs->secondary_exit_ctls &= 0;
+		msrs->secondary_exit_ctls &= SECONDARY_VM_EXIT_SAVE_IA32_FRED |
+					     SECONDARY_VM_EXIT_LOAD_IA32_FRED;
 	}
 }
 
@@ -7413,7 +7414,7 @@ static void nested_vmx_setup_entry_ctls(struct vmcs_config *vmcs_conf,
 		VM_ENTRY_IA32E_MODE |
 #endif
 		VM_ENTRY_LOAD_IA32_PAT | VM_ENTRY_LOAD_BNDCFGS |
-		VM_ENTRY_LOAD_CET_STATE;
+		VM_ENTRY_LOAD_CET_STATE | VM_ENTRY_LOAD_IA32_FRED;
 	msrs->entry_ctls_high |=
 		(VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR | VM_ENTRY_LOAD_IA32_EFER |
 		 VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL);
arch/x86/kvm/vmx/vmx.c +1 -0
index 04442f869abb..8f3805a71a97 100644
@@ -7994,6 +7994,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)
 
 	entry = kvm_find_cpuid_entry_index(vcpu, 0x7, 1);
 	cr4_fixed1_update(X86_CR4_LAM_SUP,    eax, feature_bit(LAM));
+	cr4_fixed1_update(X86_CR4_FRED,       eax, feature_bit(FRED));
 
 #undef cr4_fixed1_update
 }
Signature · 1 line
2.51.0
Reviewed-by on v9 22/22
#106 lore raw

Re: [PATCH v9 22/22] KVM: nVMX: Enable VMX FRED controls

Quoted 9 lines · Xin Li (Intel)

From: Xin Li <[email protected]>

Permit use of VMX FRED controls in nested VMX now that support for nested FRED is implemented.

Quoted 14 lines

---

Change in v5:
* Add TB from Xuelian Guo.
---
arch/x86/kvm/vmx/nested.c | 5 +++--
arch/x86/kvm/vmx/vmx.c | 1 +

2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/nested.c +0 -0
index 37ab8250dd31..655257b34d15 100644

arch/x86/kvm/vmx/nested.c static void nested_vmx_setup_exit_ctls(struct vmcs_config *vmcs_conf, · line 7399

Quoted 3 lines
arch/x86/kvm/vmx/nested.c +0 -0
@@ -7397,7 +7397,8 @@ static void nested_vmx_setup_exit_ctls(struct vmcs_config *vmcs_conf,
 		 * advertise any feature in it to nVMX until its nVMX support
 		 * is ready.
 		 */

Shouldn't this comment be removed? I suppose it was a note to reviewers explaining why it's hard-coded to 0. Since some features have been added, the comment can be dropped.

Quoted 30 lines
arch/x86/kvm/vmx/nested.c +3 -2
-		msrs->secondary_exit_ctls &= 0;
+		msrs->secondary_exit_ctls &= SECONDARY_VM_EXIT_SAVE_IA32_FRED |
+					     SECONDARY_VM_EXIT_LOAD_IA32_FRED;
 	}
 }
 
@@ -7413,7 +7414,7 @@ static void nested_vmx_setup_entry_ctls(struct vmcs_config *vmcs_conf,
 		VM_ENTRY_IA32E_MODE |
 #endif
 		VM_ENTRY_LOAD_IA32_PAT | VM_ENTRY_LOAD_BNDCFGS |
-		VM_ENTRY_LOAD_CET_STATE;
+		VM_ENTRY_LOAD_CET_STATE | VM_ENTRY_LOAD_IA32_FRED;
 	msrs->entry_ctls_high |=
 		(VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR | VM_ENTRY_LOAD_IA32_EFER |
 		 VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL);
arch/x86/kvm/vmx/vmx.c +1 -0
index 04442f869abb..8f3805a71a97 100644
@@ -7994,6 +7994,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu *vcpu)
 
 	entry = kvm_find_cpuid_entry_index(vcpu, 0x7, 1);
 	cr4_fixed1_update(X86_CR4_LAM_SUP,    eax, feature_bit(LAM));
+	cr4_fixed1_update(X86_CR4_FRED,       eax, feature_bit(FRED));
 
 #undef cr4_fixed1_update
 }
Signature · 1 line
2.51.0
author maintainer
#107 lore raw
In reply to Xin Li (Intel)
Quoted 51 lines · Xin Li (Intel)

This patch set enables the Intel flexible return and event delivery (FRED) architecture with KVM VMX to allow guests to utilize FRED.

The FRED architecture defines simple new transitions that change privilege level (ring transitions). The FRED architecture was designed with the following goals:

1) Improve overall performance and response time by replacing event delivery through the interrupt descriptor table (IDT event delivery) and event return by the IRET instruction with lower latency transitions.

2) Improve software robustness by ensuring that event delivery establishes the full supervisor context and that event return establishes the full user context.

The new transitions defined by the FRED architecture are FRED event delivery and, for returning from events, two FRED return instructions. FRED event delivery can effect a transition from ring 3 to ring 0, but it is used also to deliver events incident to ring 0. One FRED instruction (ERETU) effects a return from ring 0 to ring 3, while the other (ERETS) returns while remaining in ring 0. Collectively, FRED event delivery and the FRED return instructions are FRED transitions.

Intel VMX architecture is extended to run FRED guests, and the major changes are:

1) New VMCS fields for FRED context management, which includes two new event data VMCS fields, eight new guest FRED context VMCS fields and eight new host FRED context VMCS fields.

2) VMX nested-exception support for proper virtualization of stack levels introduced with FRED architecture.

Search for the latest FRED spec in most search engines with this search pattern:

site:intel.com FRED (flexible return and event delivery) specification

Although FRED and CET supervisor shadow stacks are independent CPU features, FRED unconditionally includes FRED shadow stack pointer MSRs IA32_FRED_SSP[0123], and IA32_FRED_SSP0 is just an alias of the CET MSR IA32_PL0_SSP. IOW, the state management of MSR IA32_PL0_SSP becomes an overlap area, and Sean requested that FRED virtualization to land after CET virtualization [1].

Quoted 2 lines

With CET virtualization now merged in v6.18, the path is clear to submit the FRED virtualization patch series :).

Sean, what is the plan for the FRED patch series?

A good news is that we have got acks on all 3 common x86 patches.

Thanks!
Xin

Quoted 9 lines

Changes in v9:
* Rebased to the latest kvm-x86/next branch, tag kvm-x86-next-2025.10.20-2.
* Guard FRED state save/restore with guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) in patch 19 (syzbot & Chao).
* Use array indexing for exception stack access, eliminating the need for the ESTACKS_MEMBERS() macro in struct cea_exception_stacks, and then exported __this_cpu_ist_top_va() in a subsequent patch (Dave Hansen).
* Rewrote some of the change logs.

#108 lore raw
Quoted 9 lines · Xin Li
Quoted 9 lines · Xin Li (Intel)

Although FRED and CET supervisor shadow stacks are independent CPU features, FRED unconditionally includes FRED shadow stack pointer MSRs IA32_FRED_SSP[0123], and IA32_FRED_SSP0 is just an alias of the CET MSR IA32_PL0_SSP. IOW, the state management of MSR IA32_PL0_SSP becomes an overlap area, and Sean requested that FRED virtualization to land after CET virtualization [1].

Quoted 4 lines
Quoted 2 lines

With CET virtualization now merged in v6.18, the path is clear to submit the FRED virtualization patch series :).

Sean, what is the plan for the FRED patch series?

Review and merge it asap. Unfortunately, "asap" isn't all that soon, mostly because I've been bogged down with non-upstream stuff. I'm hoping to dive in next week (but take that with a grain of salt; I've more or less said that exact thing to someone else for other patches for three weeks running).

#109 lore raw
In reply to Xin Li (Intel)
Quoted 20 lines · Xin Li (Intel)

Xin Li (18):
KVM: VMX: Enable support for secondary VM exit controls
KVM: VMX: Initialize VM entry/exit FRED controls in vmcs_config
KVM: VMX: Disable FRED if FRED consistency checks fail
KVM: VMX: Initialize VMCS FRED fields
KVM: VMX: Set FRED MSR intercepts
KVM: VMX: Save/restore guest FRED RSP0
KVM: VMX: Add support for saving and restoring FRED MSRs
KVM: x86: Add a helper to detect if FRED is enabled for a vCPU
KVM: VMX: Virtualize FRED event_data
KVM: VMX: Virtualize FRED nested exception tracking
KVM: x86: Mark CR4.FRED as not reserved
KVM: VMX: Dump FRED context in dump_vmcs()
KVM: x86: Advertise support for FRED
KVM: nVMX: Enable support for secondary VM exit controls
KVM: nVMX: Handle FRED VMCS fields in nested VMX context
KVM: nVMX: Validate FRED-related VMCS fields
KVM: nVMX: Guard SHADOW_FIELD_R[OW] macros with VMX feature checks
KVM: nVMX: Enable VMX FRED controls

Quoted 1 line

Xin Li (Intel) (4):

I'm guessing the two different "names" isn't intended?

Quoted 4 lines

x86/cea: Prefix event stack names with ESTACK_ x86/cea: Use array indexing to simplify exception stack access x86/cea: Export __this_cpu_ist_top_va() to KVM
KVM: x86: Save/restore the nested flag of an exception

author maintainer
#110 lore raw
Quoted 6 lines · Sean Christopherson
Quoted 2 lines

Xin Li (Intel) (4):

I'm guessing the two different "names" isn't intended?

Not at all :’(

I've switched from using my Intel email to [email protected] due to recipient threshold limits. Earlier versions of the patch series were sent to LKML with the Intel address, while all new patches use [email protected].

#111 lore raw
In reply to Xin Li (Intel)
Quoted 2 lines · Xin Li (Intel)

This patch set enables the Intel flexible return and event delivery (FRED) architecture with KVM VMX to allow guests to utilize FRED.

This could do with having a *lot* of KVM selftests.

Here's one to get you started (untested as I haven't found suitable hardware to test it on).

From bd465aabebcb124e09a26fe9f4c861354febabe4 Mon Sep 17 00:00:00 2001
From: David Woodhouse <[email protected]>
Date: Thu, 23 Apr 2026 15:20:11 +0100
Subject: [PATCH] KVM: selftests: Add FRED event type classification test

Test that software-generated events deliver the correct FRED event type in the FRED stack frame.

In FRED mode, the CPU pushes event metadata including the event type into the FRED stack frame. Different instruction types should produce different event types:
- ICEBP (INT1): EVENT_TYPE_PRIV_SWEVT (5)
- INT3: EVENT_TYPE_SWEXC (6)
- INT n: EVENT_TYPE_SWINT (4)

The test sets up a guest in FRED mode by enabling CR4.FRED and configuring the FRED MSRs, with a hand-written FRED entry point that extracts the event type from the FRED SS field. It exercises each software event type and verifies the classification is correct.

Requires KVM guest FRED support (the FRED KVM VMX series) and hardware FRED support. Skips if either is unavailable.

---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../selftests/kvm/x86/int1_fred_test.c        | 225 ++++++++++++++++++
 2 files changed, 226 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86/int1_fred_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index 148d427ff24b..62e76617f82f 100644
@@ -74,6 +74,7 @@ TEST_GEN_PROGS_x86 += x86/feature_msrs_test
 TEST_GEN_PROGS_x86 += x86/exit_on_emulation_failure_test
 TEST_GEN_PROGS_x86 += x86/fastops_test
 TEST_GEN_PROGS_x86 += x86/fix_hypercall_test
+TEST_GEN_PROGS_x86 += x86/int1_fred_test
 TEST_GEN_PROGS_x86 += x86/hwcr_msr_test
 TEST_GEN_PROGS_x86 += x86/hyperv_clock
 TEST_GEN_PROGS_x86 += x86/hyperv_cpuid
tools/testing/selftests/kvm/x86/int1_fred_test.c new file +225 -0

225 lines not on this page · raw message

Signature · 1 line
2.43.0
author maintainer
#112 lore raw
Quoted 9 lines · David Woodhouse
Quoted 2 lines · Xin Li (Intel)

This patch set enables the Intel flexible return and event delivery (FRED) architecture with KVM VMX to allow guests to utilize FRED.

This could do with having a *lot* of KVM selftests.

Hi David,

Thanks, I will integrate your tests into this FRED test:
https://lore.kernel.org/kvm/[email protected]/

Quoted 3 lines

Here's one to get you started (untested as I haven't found suitable hardware to test it on).

Same here for me now :(

Quoted 280 lines

From bd465aabebcb124e09a26fe9f4c861354febabe4 Mon Sep 17 00:00:00 2001
From: David Woodhouse <[email protected]>
Date: Thu, 23 Apr 2026 15:20:11 +0100
Subject: [PATCH] KVM: selftests: Add FRED event type classification test

Test that software-generated events deliver the correct FRED event type in the FRED stack frame.

In FRED mode, the CPU pushes event metadata including the event type into the FRED stack frame. Different instruction types should produce different event types:
- ICEBP (INT1): EVENT_TYPE_PRIV_SWEVT (5)
- INT3: EVENT_TYPE_SWEXC (6)
- INT n: EVENT_TYPE_SWINT (4)

The test sets up a guest in FRED mode by enabling CR4.FRED and configuring the FRED MSRs, with a hand-written FRED entry point that extracts the event type from the FRED SS field. It exercises each software event type and verifies the classification is correct.

Requires KVM guest FRED support (the FRED KVM VMX series) and hardware FRED support. Skips if either is unavailable.

---
tools/testing/selftests/kvm/Makefile.kvm | 1 +
.../selftests/kvm/x86/int1_fred_test.c | 225 ++++++++++++++++++

2 files changed, 226 insertions(+)

create mode 100644 tools/testing/selftests/kvm/x86/int1_fred_test.c

tools/testing/selftests/kvm/Makefile.kvm +0 -0
index 148d427ff24b..62e76617f82f 100644
@@ -74,6 +74,7 @@ TEST_GEN_PROGS_x86 += x86/feature_msrs_test

TEST_GEN_PROGS_x86 += x86/exit_on_emulation_failure_test TEST_GEN_PROGS_x86 += x86/fastops_test TEST_GEN_PROGS_x86 += x86/fix_hypercall_test +TEST_GEN_PROGS_x86 += x86/int1_fred_test TEST_GEN_PROGS_x86 += x86/hwcr_msr_test TEST_GEN_PROGS_x86 += x86/hyperv_clock TEST_GEN_PROGS_x86 += x86/hyperv_cpuid

tools/testing/selftests/kvm/x86/int1_fred_test.c new file +225 -0
new file mode 100644
index 000000000000..7ffb54b56047
@@ -0,0 +1,225 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Test FRED event type classification for software-generated events.
+ *
+ * In FRED mode, the CPU pushes event metadata including the event type
+ * into the FRED stack frame SS field. This test verifies the correct
+ * event type for:
+ *   - ICEBP (INT1, opcode 0xF1): EVENT_TYPE_PRIV_SWEXC (5)
+ *   - INT3 (opcode 0xCC):        EVENT_TYPE_SWEXC (6)
+ *   - INT n (opcode 0xCD nn):    EVENT_TYPE_SWINT (4)
+ *
+ * The FRED stack page is evicted via MADV_DONTNEED before each test
+ * to also exercise EPT/NPT fault handling during FRED frame push.
+ *
+ * This test requires KVM guest FRED support and hardware FRED support.
+ * It will skip if either is unavailable.
+ */
+#include <stdint.h>
+#include <string.h>
+#include <sys/mman.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+
+/* FRED: CPUID.(EAX=7,ECX=1):EAX bit 17 */
+#define X86_FEATURE_FRED KVM_X86_CPU_FEATURE(7, 1, EAX, 17)
+
+/* FRED MSRs */
+#define MSR_IA32_FRED_RSP0 0x1cc
+#define MSR_IA32_FRED_STKLVLS 0x1d0
+#define MSR_IA32_FRED_CONFIG 0x1d4
+
+/* FRED event types (from arch/x86/include/asm/trapnr.h) */
+#define EVENT_TYPE_SWINT 4 /* INT n */
+#define EVENT_TYPE_PRIV_SWEXC 5 /* INT1 (ICEBP) */
+#define EVENT_TYPE_SWEXC 6 /* INTO, INT3 */
+
+/* FRED SS field layout */
+#define FRED_SS_TYPE_SHIFT 48
+#define FRED_SS_TYPE_MASK (0xfULL << FRED_SS_TYPE_SHIFT)
+#define FRED_SS_VECTOR_SHIFT 32
+#define FRED_SS_VECTOR_MASK (0xffULL << FRED_SS_VECTOR_SHIFT)
+
+/* CR4.FRED (bit 32) */
+#define X86_CR4_FRED (1ULL << 32)
+
+/* Guest memory layout */
+#define FRED_STACK_GPA 0xc0000000ULL
+#define FRED_STACK_SIZE PAGE_SIZE
+
+/*
+ * FRED stack frame as pushed by the CPU on event delivery.
+ * Layout from low address (RSP) to high address:
+ *
+ *   RSP+0x00: error code
+ *   RSP+0x08: RIP (return address)
+ *   RSP+0x10: CS (with auxiliary info in upper bits)
+ *   RSP+0x18: RFLAGS
+ *   RSP+0x20: RSP (original)
+ *   RSP+0x28: SS (with event type/vector in upper bits)
+ */
+struct fred_stack_frame {
+ uint64_t error_code;
+ uint64_t rip;
+ uint64_t cs;
+ uint64_t rflags;
+ uint64_t rsp;
+ uint64_t ss;
+};
+
+/* Shared result area */
+static volatile uint64_t fred_ss_value;
+static volatile uint64_t fred_saved_rip;
+static volatile bool fred_handler_called;
+
+static void __used fred_handler(struct fred_stack_frame *frame)
+{
+ fred_ss_value = frame->ss;
+ fred_saved_rip = frame->rip;
+ fred_handler_called = true;
+}
+
+/*
+ * FRED entry points. MSR_IA32_FRED_CONFIG points to the page-aligned
+ * base. Ring 3 events enter at base+0, ring 0 events at base+0x100.
+ * Since ICEBP executes in ring 0, the CPU enters at fred_entrypoint
+ * + 256 = fred_entrypoint_kernel.
+ */
+extern void fred_entrypoint(void);
+
+asm(
+ ".pushsection .text\n"
+ ".global fred_entrypoint\n"
+ ".balign 4096\n"
+"fred_entrypoint:\n"
+ /* Ring 3 entry — unused, no userspace in this test */
+ "ud2\n"
+ /* Pad to +256 for ring 0 entry */
+ ".org fred_entrypoint + 256, 0xcc\n"
+"fred_entrypoint_kernel:\n"
+ "movq %rsp, %rdi\n"
+ "call fred_handler\n"
+ ".byte 0xf2, 0x0f, 0x01, 0xca\n" /* ERETS */
+ ".popsection\n"
+);
+
+static void check_fred_event(uint64_t expected_rip, uint8_t expected_vector,
+     uint8_t expected_type, const char *name)
+{
+ uint64_t event_type, event_vector;
+
+ GUEST_ASSERT(fred_handler_called);
+ GUEST_ASSERT_EQ(fred_saved_rip, expected_rip);
+
+ event_type = (fred_ss_value & FRED_SS_TYPE_MASK) >> FRED_SS_TYPE_SHIFT;
+ event_vector = (fred_ss_value & FRED_SS_VECTOR_MASK) >> FRED_SS_VECTOR_SHIFT;
+
+ GUEST_ASSERT_EQ(event_vector, expected_vector);
+ GUEST_ASSERT_EQ(event_type, expected_type);
+}
+
+static void guest_code(void)
+{
+ uint64_t expected_rip;
+
+ /*
+ * Enable FRED:
+ *  1. Set MSR_IA32_FRED_CONFIG to our entry point
+ *  2. Set MSR_IA32_FRED_RSP0 to our FRED stack
+ *  3. Set MSR_IA32_FRED_STKLVLS (all events use stack level 0)
+ *  4. Set CR4.FRED
+ */
+ wrmsr(MSR_IA32_FRED_CONFIG, (uint64_t)fred_entrypoint);
+ wrmsr(MSR_IA32_FRED_RSP0, FRED_STACK_GPA + FRED_STACK_SIZE);
+ wrmsr(MSR_IA32_FRED_STKLVLS, 0);
+
+ {
+ uint64_t cr4;
+ asm volatile("mov %%cr4, %0" : "=r"(cr4));
+ cr4 |= X86_CR4_FRED;
+ asm volatile("mov %0, %%cr4" :: "r"(cr4));
+ }
+
+ /* Test 1: ICEBP (INT1) — should be EVENT_TYPE_PRIV_SWEXC (5) */
+ fred_handler_called = false;
+ asm volatile("lea 1f(%%rip), %0\n\t"
+     ".byte 0xf1\n\t"
+     "1:" : "=r"(expected_rip) :: "memory");
+ check_fred_event(expected_rip, DB_VECTOR, EVENT_TYPE_PRIV_SWEXC,
+ "ICEBP");
+ GUEST_SYNC(0);
+
+ /* Test 2: INT3 — should be EVENT_TYPE_SWEXC (6) */
+ fred_handler_called = false;
+ asm volatile("lea 1f(%%rip), %0\n\t"
+     "int3\n\t"
+     "1:" : "=r"(expected_rip) :: "memory");
+ check_fred_event(expected_rip, BP_VECTOR, EVENT_TYPE_SWEXC, "INT3");
+ GUEST_SYNC(1);
+
+ /* Test 3: INT $0x20 — should be EVENT_TYPE_SWINT (4) */
+ fred_handler_called = false;
+ asm volatile("lea 1f(%%rip), %0\n\t"
+     "int $0x20\n\t"
+     "1:" : "=r"(expected_rip) :: "memory");
+ check_fred_event(expected_rip, 0x20, EVENT_TYPE_SWINT, "INT $0x20");
+ GUEST_SYNC(2);
+}
+
+static const char * const test_names[] = {
+ [0] = "ICEBP (EVENT_TYPE_PRIV_SWEXC)",
+ [1] = "INT3 (EVENT_TYPE_SWEXC)",
+ [2] = "INT $0x20 (EVENT_TYPE_SWINT)",
+};
+
+int main(int argc, char *argv[])
+{
+ struct kvm_vcpu *vcpu;
+ struct kvm_vm *vm;
+ struct ucall uc;
+ unsigned int nr_pages;
+ void *fred_stack_hva;
+ int test;
+
+ TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_FRED));
+
+ pr_info("Testing FRED event types with EPT fault on stack\n");
+
+ vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+
+ /* Map the FRED stack page */
+ nr_pages = vm_calc_num_guest_pages(VM_MODE_DEFAULT, FRED_STACK_SIZE);
+ vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS,
+    FRED_STACK_GPA, 1, nr_pages, 0);
+ virt_map(vm, FRED_STACK_GPA, FRED_STACK_GPA, nr_pages);
+ fred_stack_hva = addr_gpa2hva(vm, FRED_STACK_GPA);
+
+ for (test = 0; test < ARRAY_SIZE(test_names); test++) {
+ /*
+ * Evict the FRED stack page so the CPU takes an EPT/NPT
+ * violation when pushing the FRED frame. KVM must handle
+ * the fault, map the page, and reinject the event.
+ */
+ madvise(fred_stack_hva, FRED_STACK_SIZE, MADV_DONTNEED);
+
+ vcpu_run(vcpu);
+ TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO);
+
+ switch (get_ucall(vcpu, &uc)) {
+ case UCALL_ABORT:
+ REPORT_GUEST_ASSERT(uc);
+ break;
+ case UCALL_SYNC:
+ TEST_ASSERT_EQ(uc.args[1], test);
+ pr_info("  %s: PASSED\n", test_names[test]);
+ break;
+ default:
+ TEST_FAIL("Unexpected ucall");
+ }
+ }
+
+ kvm_vm_free(vm);
+ return 0;
+}
Signature · 1 line
2.43.0
#113 lore raw

Hello!

Quoted 5 lines · Xin Li
Quoted 3 lines · David Woodhouse

Here's one to get you started (untested as I haven't found suitable hardware to test it on).

Same here for me now :(

I ran David's selftest on a PTL laptop and ran into a couple of issues.

Quoted 6 lines
Quoted 6 lines

From bd465aabebcb124e09a26fe9f4c861354febabe4 Mon Sep 17 00:00:00 2001
From: David Woodhouse <[email protected]>
Date: Thu, 23 Apr 2026 15:20:11 +0100
Subject: [PATCH] KVM: selftests: Add FRED event type classification test

Quoted 6 lines

tools/testing/selftests/kvm/x86/int1_fred_test.c line 82

Quoted 6 lines
tools/testing/selftests/kvm/x86/int1_fred_test.c new file +6 -0
+static void __used fred_handler(struct fred_stack_frame *frame)
+{
+ fred_ss_value = frame->ss;
+ fred_saved_rip = frame->rip;
+ fred_handler_called = true;
+}

fred_handler() has problems getting linked:

/usr/bin/ld: /home/maciej/linux/tools/testing/selftests/kvm/x86/int1_fred_test.o: in function `fred_entrypoint_kernel':
int1_fred_test.c:(.text+0x104): undefined reference to `fred_handler'
collect2: error: ld returned 1 exit status

I guess the .pushsection below makes it a different translation unit? Because getting rid of the static keyword takes care of the problem for me.

Quoted 25 lines
Quoted 25 lines
tools/testing/selftests/kvm/x86/int1_fred_test.c new file +25 -0
+
+/*
+ * FRED entry points. MSR_IA32_FRED_CONFIG points to the page-aligned
+ * base. Ring 3 events enter at base+0, ring 0 events at base+0x100.
+ * Since ICEBP executes in ring 0, the CPU enters at fred_entrypoint
+ * + 256 = fred_entrypoint_kernel.
+ */
+extern void fred_entrypoint(void);
+
+asm(
+ ".pushsection .text\n"
+ ".global fred_entrypoint\n"
+ ".balign 4096\n"
+"fred_entrypoint:\n"
+ /* Ring 3 entry — unused, no userspace in this test */
+ "ud2\n"
+ /* Pad to +256 for ring 0 entry */
+ ".org fred_entrypoint + 256, 0xcc\n"
+"fred_entrypoint_kernel:\n"
+ "movq %rsp, %rdi\n"
+ "call fred_handler\n"
+ ".byte 0xf2, 0x0f, 0x01, 0xca\n" /* ERETS */
+ ".popsection\n"
+);
+

...

Quoted 9 lines

tools/testing/selftests/kvm/x86/int1_fred_test.c line 152

Quoted 9 lines
tools/testing/selftests/kvm/x86/int1_fred_test.c new file +9 -0
+
+ /* Test 1: ICEBP (INT1) — should be EVENT_TYPE_PRIV_SWEXC (5) */
+ fred_handler_called = false;
+ asm volatile("lea 1f(%%rip), %0\n\t"
+     ".byte 0xf1\n\t"
+     "1:" : "=r"(expected_rip) :: "memory");
+ check_fred_event(expected_rip, DB_VECTOR, EVENT_TYPE_PRIV_SWEXC,
+ "ICEBP");
+ GUEST_SYNC(0);

The above event type test seems to fail and return 0x3 instead of 0x5:

Random seed: 0x6b8b4567
Testing FRED event types with EPT fault on stack ==== Test Assertion Failure ====
x86/int1_fred_test.c:120: event_type == expected_type pid=16646 tid=16646 errno=4 - Interrupted system call

     1  0x0000000000413349: assert_on_unhandled_exception at processor.c:659
     2  0x0000000000407d36: _vcpu_run at kvm_util.c:1703
     3   (inlined by) vcpu_run at kvm_util.c:1714
     4  0x0000000000403104: main at int1_fred_test.c:207
     5  0x00007ff8d4c2a1c9: ?? ??:0
     6  0x00007ff8d4c2a28a: ?? ??:0
     7  0x0000000000403314: _start at ??:?

0x3 != 0x5 (event_type != expected_type)

after a little digging I think the issue could be this in arch/x86/kvm/x86.h:

	static inline bool kvm_exception_is_soft(unsigned int nr)
	{
		return (nr == BP_VECTOR) || (nr == OF_VECTOR);
	}

Since ICEBP(INT1) results in a DB_VECTOR it's not take into account and the check fails. Then in vmx_inject_exception() INTR_TYPE_HARD_EXCEPTION is picked which is 0x3 when decoded.

I think you'd need to add another check in vmx_inject_exception() to handle that DB_VECTOR too. Simply changing the event type if the vector is of DB_VECTOR type fixes that problem but then the selftest fails in other places (assert fred_handler_called and saved rip vs expected_rip). I didn't yet have the time to figure out what could be wrong there, maybe you would have more of an idea :)

Signature · 2 lines
Kind regards
Maciej Wieczór-Retman
#114 lore raw
Quoted 23 lines · Maciej Wieczor-Retman

Hello!

Quoted 4 lines · Xin Li
Quoted 3 lines · David Woodhouse

Here's one to get you started (untested as I haven't found suitable hardware to test it on).

Same here for me now :(

I ran David's selftest on a PTL laptop and ran into a couple of issues.

Quoted 11 lines

tools/testing/selftests/kvm/x86/int1_fred_test.c line 82

Quoted 11 lines

From bd465aabebcb124e09a26fe9f4c861354febabe4 Mon Sep 17 00:00:00 2001
From: David Woodhouse <[email protected]>
Date: Thu, 23 Apr 2026 15:20:11 +0100
Subject: [PATCH] KVM: selftests: Add FRED event type classification test

tools/testing/selftests/kvm/x86/int1_fred_test.c new file +6 -0
+static void __used fred_handler(struct fred_stack_frame *frame)
+{
+ fred_ss_value = frame->ss;
+ fred_saved_rip = frame->rip;
+ fred_handler_called = true;
+}

fred_handler() has problems getting linked:

Quoted 6 lines

/usr/bin/ld: /home/maciej/linux/tools/testing/selftests/kvm/x86/int1_fred_test.o: in function `fred_entrypoint_kernel':
int1_fred_test.c:(.text+0x104): undefined reference to `fred_handler'
collect2: error: ld returned 1 exit status

I guess the .pushsection below makes it a different translation unit? Because getting rid of the static keyword takes care of the problem for me.

The problem is, being static, fred_handler() is eligible to be optimised away, because the compiler can't see that the asm() refers to it.

Dropping static is the right fix to make.

GCC 15 can now do references out of global asm() to identify the symbols they use, but it's going to be years before this capability is safe to use generally.

Quoted 54 lines
Quoted 26 lines
Quoted 26 lines
tools/testing/selftests/kvm/x86/int1_fred_test.c new file +25 -0
+
+/*
+ * FRED entry points. MSR_IA32_FRED_CONFIG points to the page-aligned
+ * base. Ring 3 events enter at base+0, ring 0 events at base+0x100.
+ * Since ICEBP executes in ring 0, the CPU enters at fred_entrypoint
+ * + 256 = fred_entrypoint_kernel.
+ */
+extern void fred_entrypoint(void);
+
+asm(
+ ".pushsection .text\n"
+ ".global fred_entrypoint\n"
+ ".balign 4096\n"
+"fred_entrypoint:\n"
+ /* Ring 3 entry — unused, no userspace in this test */
+ "ud2\n"
+ /* Pad to +256 for ring 0 entry */
+ ".org fred_entrypoint + 256, 0xcc\n"
+"fred_entrypoint_kernel:\n"
+ "movq %rsp, %rdi\n"
+ "call fred_handler\n"
+ ".byte 0xf2, 0x0f, 0x01, 0xca\n" /* ERETS */
+ ".popsection\n"
+);
+

...

Quoted 9 lines

tools/testing/selftests/kvm/x86/int1_fred_test.c line 152

Quoted 9 lines
tools/testing/selftests/kvm/x86/int1_fred_test.c new file +9 -0
+
+ /* Test 1: ICEBP (INT1) — should be EVENT_TYPE_PRIV_SWEXC (5) */
+ fred_handler_called = false;
+ asm volatile("lea 1f(%%rip), %0\n\t"
+     ".byte 0xf1\n\t"
+     "1:" : "=r"(expected_rip) :: "memory");
+ check_fred_event(expected_rip, DB_VECTOR, EVENT_TYPE_PRIV_SWEXC,
+ "ICEBP");
+ GUEST_SYNC(0);

The above event type test seems to fail and return 0x3 instead of 0x5:

Random seed: 0x6b8b4567
Testing FRED event types with EPT fault on stack ==== Test Assertion Failure ====
x86/int1_fred_test.c:120: event_type == expected_type pid=16646 tid=16646 errno=4 - Interrupted system call

     1  0x0000000000413349: assert_on_unhandled_exception at processor.c:659
     2  0x0000000000407d36: _vcpu_run at kvm_util.c:1703
     3   (inlined by) vcpu_run at kvm_util.c:1714
     4  0x0000000000403104: main at int1_fred_test.c:207
     5  0x00007ff8d4c2a1c9: ?? ??:0
     6  0x00007ff8d4c2a28a: ?? ??:0
     7  0x0000000000403314: _start at ??:?

0x3 != 0x5 (event_type != expected_type)

after a little digging I think the issue could be this in arch/x86/kvm/x86.h:

Quoted 8 lines
	static inline bool kvm_exception_is_soft(unsigned int nr)
	{
		return (nr == BP_VECTOR) || (nr == OF_VECTOR);
	}

Since ICEBP(INT1) results in a DB_VECTOR it's not take into account and the check fails. Then in vmx_inject_exception() INTR_TYPE_HARD_EXCEPTION is picked which is 0x3 when decoded.

That's a real bug then.

Quoted 5 lines

I think you'd need to add another check in vmx_inject_exception() to handle that DB_VECTOR too. Simply changing the event type if the vector is of DB_VECTOR type fixes that problem but then the selftest fails in other places (assert fred_handler_called and saved rip vs expected_rip). I didn't yet have the time to figure out what could be wrong there, maybe you would have more of an idea :)

#DB is intercepted to mitigate CVE-2015-8104 (systemwide DoS).  But, to start with, check that the test passes when #DB is not intercepted. That's the basecase for architectural behaviour.

When #DB is intercepted, the type in EXIT_INTR_INFO needs preserving and forwarding into ENTRY_INTR_INFO, because that is what distinguishes an ICEBP #DB from other #DBs.  There's no way of recovering this detail after the fact.

On the injection side, some #DB's are traps and some are faults.  ICEBP will have a fault-like VMExit but need trap semantics, so like other soft interrupts, need INSN_LEN adding to %rip.  But, type=3 #DBs need to leave %rip unchanged.

~Andrew

maintainer
#115 lore raw
Quoted 120 lines · Andrew Cooper
Quoted 23 lines · Maciej Wieczor-Retman

Hello!

Quoted 4 lines · Xin Li
Quoted 3 lines · David Woodhouse

Here's one to get you started (untested as I haven't found suitable hardware to test it on).

Same here for me now :(

I ran David's selftest on a PTL laptop and ran into a couple of issues.

Quoted 11 lines

tools/testing/selftests/kvm/x86/int1_fred_test.c line 82

Quoted 11 lines

From bd465aabebcb124e09a26fe9f4c861354febabe4 Mon Sep 17 00:00:00 2001
From: David Woodhouse <[email protected]>
Date: Thu, 23 Apr 2026 15:20:11 +0100
Subject: [PATCH] KVM: selftests: Add FRED event type classification test

tools/testing/selftests/kvm/x86/int1_fred_test.c new file +6 -0
+static void __used fred_handler(struct fred_stack_frame *frame)
+{
+ fred_ss_value = frame->ss;
+ fred_saved_rip = frame->rip;
+ fred_handler_called = true;
+}

fred_handler() has problems getting linked:

Quoted 6 lines

/usr/bin/ld: /home/maciej/linux/tools/testing/selftests/kvm/x86/int1_fred_test.o: in function `fred_entrypoint_kernel':
int1_fred_test.c:(.text+0x104): undefined reference to `fred_handler'
collect2: error: ld returned 1 exit status

I guess the .pushsection below makes it a different translation unit? Because getting rid of the static keyword takes care of the problem for me.

The problem is, being static, fred_handler() is eligible to be optimised away, because the compiler can't see that the asm() refers to it.

Dropping static is the right fix to make.

GCC 15 can now do references out of global asm() to identify the symbols they use, but it's going to be years before this capability is safe to use generally.

Quoted 54 lines
Quoted 26 lines
Quoted 26 lines
tools/testing/selftests/kvm/x86/int1_fred_test.c new file +25 -0
+
+/*
+ * FRED entry points. MSR_IA32_FRED_CONFIG points to the page-aligned
+ * base. Ring 3 events enter at base+0, ring 0 events at base+0x100.
+ * Since ICEBP executes in ring 0, the CPU enters at fred_entrypoint
+ * + 256 = fred_entrypoint_kernel.
+ */
+extern void fred_entrypoint(void);
+
+asm(
+ ".pushsection .text\n"
+ ".global fred_entrypoint\n"
+ ".balign 4096\n"
+"fred_entrypoint:\n"
+ /* Ring 3 entry — unused, no userspace in this test */
+ "ud2\n"
+ /* Pad to +256 for ring 0 entry */
+ ".org fred_entrypoint + 256, 0xcc\n"
+"fred_entrypoint_kernel:\n"
+ "movq %rsp, %rdi\n"
+ "call fred_handler\n"
+ ".byte 0xf2, 0x0f, 0x01, 0xca\n" /* ERETS */
+ ".popsection\n"
+);
+

...

Quoted 9 lines

tools/testing/selftests/kvm/x86/int1_fred_test.c line 152

Quoted 9 lines
tools/testing/selftests/kvm/x86/int1_fred_test.c new file +9 -0
+
+ /* Test 1: ICEBP (INT1) — should be EVENT_TYPE_PRIV_SWEXC (5) */
+ fred_handler_called = false;
+ asm volatile("lea 1f(%%rip), %0\n\t"
+     ".byte 0xf1\n\t"
+     "1:" : "=r"(expected_rip) :: "memory");
+ check_fred_event(expected_rip, DB_VECTOR, EVENT_TYPE_PRIV_SWEXC,
+ "ICEBP");
+ GUEST_SYNC(0);

The above event type test seems to fail and return 0x3 instead of 0x5:

Random seed: 0x6b8b4567
Testing FRED event types with EPT fault on stack ==== Test Assertion Failure ====
x86/int1_fred_test.c:120: event_type == expected_type pid=16646 tid=16646 errno=4 - Interrupted system call

     1  0x0000000000413349: assert_on_unhandled_exception at processor.c:659
     2  0x0000000000407d36: _vcpu_run at kvm_util.c:1703
     3   (inlined by) vcpu_run at kvm_util.c:1714
     4  0x0000000000403104: main at int1_fred_test.c:207
     5  0x00007ff8d4c2a1c9: ?? ??:0
     6  0x00007ff8d4c2a28a: ?? ??:0
     7  0x0000000000403314: _start at ??:?

0x3 != 0x5 (event_type != expected_type)

after a little digging I think the issue could be this in arch/x86/kvm/x86.h:

Quoted 8 lines
	static inline bool kvm_exception_is_soft(unsigned int nr)
	{
		return (nr == BP_VECTOR) || (nr == OF_VECTOR);
	}

Since ICEBP(INT1) results in a DB_VECTOR it's not take into account and the check fails. Then in vmx_inject_exception() INTR_TYPE_HARD_EXCEPTION is picked which is 0x3 when decoded.

That's a real bug then.

Quoted 5 lines

I think you'd need to add another check in vmx_inject_exception() to handle that DB_VECTOR too. Simply changing the event type if the vector is of DB_VECTOR type fixes that problem but then the selftest fails in other places (assert fred_handler_called and saved rip vs expected_rip). I didn't yet have the time to figure out what could be wrong there, maybe you would have more of an idea :)

#DB is intercepted to mitigate CVE-2015-8104 (systemwide DoS).  But, to start with, check that the test passes when #DB is not intercepted. That's the basecase for architectural behaviour.

When #DB is intercepted, the type in EXIT_INTR_INFO needs preserving and forwarding into ENTRY_INTR_INFO, because that is what distinguishes an ICEBP #DB from other #DBs.  There's no way of recovering this detail after the fact.

Quoted 6 lines

On the injection side, some #DB's are traps and some are faults.  ICEBP will have a fault-like VMExit but need trap semantics, so like other soft interrupts, need INSN_LEN adding to %rip.  But, type=3 #DBs need to leave %rip unchanged.

~Andrew

Also, a function that is accessed only from assembly should have the __visible annotation.

#116 lore raw
In reply to Andrew Cooper
Quoted 56 lines · Andrew Cooper
Quoted 38 lines · Maciej Wieczor-Retman

Hello!

Quoted 4 lines · Xin Li
Quoted 3 lines · David Woodhouse

Here's one to get you started (untested as I haven't found suitable hardware to test it on).

Same here for me now :(

I ran David's selftest on a PTL laptop and ran into a couple of issues.

...

Quoted 9 lines

tools/testing/selftests/kvm/x86/int1_fred_test.c line 152

Quoted 9 lines
tools/testing/selftests/kvm/x86/int1_fred_test.c new file +9 -0
+
+ /* Test 1: ICEBP (INT1) — should be EVENT_TYPE_PRIV_SWEXC (5) */
+ fred_handler_called = false;
+ asm volatile("lea 1f(%%rip), %0\n\t"
+     ".byte 0xf1\n\t"
+     "1:" : "=r"(expected_rip) :: "memory");
+ check_fred_event(expected_rip, DB_VECTOR, EVENT_TYPE_PRIV_SWEXC,
+ "ICEBP");
+ GUEST_SYNC(0);

The above event type test seems to fail and return 0x3 instead of 0x5:

Random seed: 0x6b8b4567
Testing FRED event types with EPT fault on stack ==== Test Assertion Failure ====
x86/int1_fred_test.c:120: event_type == expected_type pid=16646 tid=16646 errno=4 - Interrupted system call

     1  0x0000000000413349: assert_on_unhandled_exception at processor.c:659
     2  0x0000000000407d36: _vcpu_run at kvm_util.c:1703
     3   (inlined by) vcpu_run at kvm_util.c:1714
     4  0x0000000000403104: main at int1_fred_test.c:207
     5  0x00007ff8d4c2a1c9: ?? ??:0
     6  0x00007ff8d4c2a28a: ?? ??:0
     7  0x0000000000403314: _start at ??:?

0x3 != 0x5 (event_type != expected_type)

after a little digging I think the issue could be this in arch/x86/kvm/x86.h:

Quoted 8 lines
	static inline bool kvm_exception_is_soft(unsigned int nr)
	{
		return (nr == BP_VECTOR) || (nr == OF_VECTOR);
	}

Since ICEBP(INT1) results in a DB_VECTOR it's not take into account and the check fails. Then in vmx_inject_exception() INTR_TYPE_HARD_EXCEPTION is picked which is 0x3 when decoded.

That's a real bug then.

Quoted 5 lines

I think you'd need to add another check in vmx_inject_exception() to handle that DB_VECTOR too. Simply changing the event type if the vector is of DB_VECTOR type fixes that problem but then the selftest fails in other places (assert fred_handler_called and saved rip vs expected_rip). I didn't yet have the time to figure out what could be wrong there, maybe you would have more of an idea :)

Quoted 3 lines

#DB is intercepted to mitigate CVE-2015-8104 (systemwide DoS).  But, to start with, check that the test passes when #DB is not intercepted. That's the basecase for architectural behaviour.

I take it you mean dropping the ICEBP selftest test case and just checking INT3 and INT $0x20? In that case the other two tests pass after a minor change - namely in guest_code() the expected_rip needs to be volatile as well. Otherwise there is a RIP mismatch.

Or did you mean I should check something else?

Quoted 11 lines

When #DB is intercepted, the type in EXIT_INTR_INFO needs preserving and forwarding into ENTRY_INTR_INFO, because that is what distinguishes an ICEBP #DB from other #DBs.  There's no way of recovering this detail after the fact.

On the injection side, some #DB's are traps and some are faults.  ICEBP will have a fault-like VMExit but need trap semantics, so like other soft interrupts, need INSN_LEN adding to %rip.  But, type=3 #DBs need to leave %rip unchanged.

~Andrew

Signature · 2 lines
Kind regards
Maciej Wieczór-Retman
#117 lore raw
Quoted 11 lines · Maciej Wieczor-Retman
Quoted 10 lines · Andrew Cooper
Quoted 5 lines · Maciej Wieczor-Retman

I think you'd need to add another check in vmx_inject_exception() to handle that DB_VECTOR too. Simply changing the event type if the vector is of DB_VECTOR type fixes that problem but then the selftest fails in other places (assert fred_handler_called and saved rip vs expected_rip). I didn't yet have the time to figure out what could be wrong there, maybe you would have more of an idea :)

#DB is intercepted to mitigate CVE-2015-8104 (systemwide DoS).  But, to start with, check that the test passes when #DB is not intercepted. That's the basecase for architectural behaviour.

Quoted 6 lines

I take it you mean dropping the ICEBP selftest test case and just checking INT3 and INT $0x20? In that case the other two tests pass after a minor change - namely in guest_code() the expected_rip needs to be volatile as well. Otherwise there is a RIP mismatch.

Or did you mean I should check something else?

The selftest is correct AIUI.  You should be able to prove this by disabling #DB interception, and observing the test to pass.

Then, there's a logic bug to fix to cause the test to pass even when #DB interception is active (which is necessary due to CVE-2015-8104).

~Andrew

#118 lore raw
Quoted 17 lines · Andrew Cooper
Quoted 16 lines · Maciej Wieczor-Retman
Quoted 10 lines · Andrew Cooper
Quoted 5 lines · Maciej Wieczor-Retman

I think you'd need to add another check in vmx_inject_exception() to handle that DB_VECTOR too. Simply changing the event type if the vector is of DB_VECTOR type fixes that problem but then the selftest fails in other places (assert fred_handler_called and saved rip vs expected_rip). I didn't yet have the time to figure out what could be wrong there, maybe you would have more of an idea :)

#DB is intercepted to mitigate CVE-2015-8104 (systemwide DoS).  But, to start with, check that the test passes when #DB is not intercepted. That's the basecase for architectural behaviour.

I take it you mean dropping the ICEBP selftest test case and just checking INT3 and INT $0x20? In that case the other two tests pass after a minor change - namely in guest_code() the expected_rip needs to be volatile as well. Otherwise there is a RIP mismatch.

Quoted 4 lines
Quoted 1 line

Or did you mean I should check something else?

The selftest is correct AIUI.  You should be able to prove this by disabling #DB interception, and observing the test to pass.

Okay, I commented out '(1u << DB_VECTOR)' from vmx_update_exception_bitmap(), and booting with that and applying the two changes to the selftest that I mentioned [1] it passes as you said.

Quoted 2 lines

Then, there's a logic bug to fix to cause the test to pass even when #DB interception is active (which is necessary due to CVE-2015-8104).

I'll dig around the #DB thing, maybe I'll figure out what's wrong there.

Quoted 2 lines

~Andrew

[1] Changes being making fred_handler() non-static and making expected_rip volatile.

Signature · 2 lines
Kind regards
Maciej Wieczór-Retman
#119 lore raw
In reply to Maciej Wieczor-Retman
Quoted 5 lines · Maciej Wieczor-Retman

I take it you mean dropping the ICEBP selftest test case and just checking INT3 and INT $0x20? In that case the other two tests pass after a minor change - namely in guest_code() the expected_rip needs to be volatile as well. Otherwise there is a RIP mismatch.

I don't understand the part about making expected_rip volatile. Are the asm constraints there not correct? If not I'd rather *fix* them than use 'volatile' to paper over it. I can't see the issue though.

Can you show the generated asm both with and without it?

#120 lore raw
Quoted 7 lines · David Woodhouse
Quoted 5 lines · Maciej Wieczor-Retman

I take it you mean dropping the ICEBP selftest test case and just checking INT3 and INT $0x20? In that case the other two tests pass after a minor change - namely in guest_code() the expected_rip needs to be volatile as well. Otherwise there is a RIP mismatch.

Quoted 5 lines

I don't understand the part about making expected_rip volatile. Are the asm constraints there not correct? If not I'd rather *fix* them than use 'volatile' to paper over it. I can't see the issue though.

Can you show the generated asm both with and without it?

---------- not volatile ---------------- | -------------- volatile -----------------
#APP #APP
# 156 "x86/int1_fred_test.c" 1 # 156 "x86/int1_fred_test.c" 1

	lea 1f(%rip), %rdi		      |		lea 1f(%rip), %rax
	int3						int3
	1:						1:
# 0 "" 2					# 0 "" 2
.LVL42:					      <
	.loc 3 159 2 view .LVU146	      <
#NO_APP						#NO_APP
					      >		movq	%rax, 8(%rsp)
					      >		.loc 3 159 2 view .LVU146
					      >		movq	8(%rsp), %rdi
	movl	$6, %edx				movl	$6, %edx
	movl	$3, %esi				movl	$3, %esi
	call	check_fred_event.isra.0			call	check_fred_event.isra.0

I think that when the FRED event happens it doesn't save RDI and overwrites it before going into check_fred_event. In the volatile case it is saved to the stack before check_fred_event() (.loc 159 is the check_fred_event() call).

The below does work, it generates pretty much the same assembly as the volatile expected_rip. Do you like it? I'm not sure if it can be simplified better?

	asm volatile("lea 1f(%%rip), %%rax\n\t"
		     "movq %%rax, %0\n\t"
		     "int3\n\t"
		     "1:" : "=m"(expected_rip) :: "memory", "rax");
Signature · 2 lines
Kind regards
Maciej Wieczór-Retman
#121 lore raw
Quoted 31 lines · Maciej Wieczor-Retman
Quoted 7 lines · David Woodhouse
Quoted 5 lines · Maciej Wieczor-Retman

I take it you mean dropping the ICEBP selftest test case and just checking INT3 and INT $0x20? In that case the other two tests pass after a minor change - namely in guest_code() the expected_rip needs to be volatile as well. Otherwise there is a RIP mismatch.

Quoted 5 lines

I don't understand the part about making expected_rip volatile. Are the asm constraints there not correct? If not I'd rather *fix* them than use 'volatile' to paper over it. I can't see the issue though.

Can you show the generated asm both with and without it?

---------- not volatile ---------------- | -------------- volatile -----------------
#APP #APP
# 156 "x86/int1_fred_test.c" 1 # 156 "x86/int1_fred_test.c" 1

	lea 1f(%rip), %rdi		      |		lea 1f(%rip), %rax
	int3						int3
	1:						1:
# 0 "" 2					# 0 "" 2
.LVL42:					      <
	.loc 3 159 2 view .LVU146	      <
#NO_APP						#NO_APP
					      >		movq	%rax, 8(%rsp)
					      >		.loc 3 159 2 view .LVU146
					      >		movq	8(%rsp), %rdi
	movl	$6, %edx				movl	$6, %edx
	movl	$3, %esi				movl	$3, %esi
	call	check_fred_event.isra.0			call	check_fred_event.isra.0
Quoted 3 lines

I think that when the FRED event happens it doesn't save RDI and overwrites it before going into check_fred_event. In the volatile case it is saved to the stack before check_fred_event() (.loc 159 is the check_fred_event() call).

That looks OK to me. When it calls check_fred_event(), expected_rip is the first argument and thus lives in %rdi. On the right hand side where it's volatile, it gets explicitly loaded again from 8(%rsp) for the call to check_fred_event().

On the left hand side, the compiler doesn't mess with it at all; just chooses %rdi as the register to use for the output %0 of the inline assembly. So it's loaded *directly* into %rdi by our 'lea' and it goes straight from there to the check_fred_event() function as it should.

Your version turns it into a memory operand and forces it to get written out to the stack, after which the compiler has to load it again, so yes it'll look a lot more like your right hand side.

But I don't see anything actually wrong with the original.

#122 lore raw
Quoted 46 lines · David Woodhouse
Quoted 31 lines · Maciej Wieczor-Retman
Quoted 7 lines · David Woodhouse
Quoted 5 lines · Maciej Wieczor-Retman

I take it you mean dropping the ICEBP selftest test case and just checking INT3 and INT $0x20? In that case the other two tests pass after a minor change - namely in guest_code() the expected_rip needs to be volatile as well. Otherwise there is a RIP mismatch.

Quoted 5 lines

I don't understand the part about making expected_rip volatile. Are the asm constraints there not correct? If not I'd rather *fix* them than use 'volatile' to paper over it. I can't see the issue though.

Can you show the generated asm both with and without it?

---------- not volatile ---------------- | -------------- volatile -----------------
#APP #APP
# 156 "x86/int1_fred_test.c" 1 # 156 "x86/int1_fred_test.c" 1

	lea 1f(%rip), %rdi		      |		lea 1f(%rip), %rax
	int3						int3
	1:						1:
# 0 "" 2					# 0 "" 2
.LVL42:					      <
	.loc 3 159 2 view .LVU146	      <
#NO_APP						#NO_APP
					      >		movq	%rax, 8(%rsp)
					      >		.loc 3 159 2 view .LVU146
					      >		movq	8(%rsp), %rdi
	movl	$6, %edx				movl	$6, %edx
	movl	$3, %esi				movl	$3, %esi
	call	check_fred_event.isra.0			call	check_fred_event.isra.0
Quoted 3 lines

I think that when the FRED event happens it doesn't save RDI and overwrites it before going into check_fred_event. In the volatile case it is saved to the stack before check_fred_event() (.loc 159 is the check_fred_event() call).

That looks OK to me. When it calls check_fred_event(), expected_rip is the first argument and thus lives in %rdi. On the right hand side where it's volatile, it gets explicitly loaded again from 8(%rsp) for the call to check_fred_event().

On the left hand side, the compiler doesn't mess with it at all; just chooses %rdi as the register to use for the output %0 of the inline assembly. So it's loaded *directly* into %rdi by our 'lea' and it goes straight from there to the check_fred_event() function as it should.

Quoted 5 lines

Your version turns it into a memory operand and forces it to get written out to the stack, after which the compiler has to load it again, so yes it'll look a lot more like your right hand side.

But I don't see anything actually wrong with the original.

My theory is that after 'int3' call the FRED event is handled elsewhere and %rdi is not preserved. So the original version of the assembly looks okay but I was thinking that int3 has side effects.

Below is the test output of the RIPs not matching when running the original, does that help in any way?

Random seed: 0x6b8b4567
Testing FRED event types with EPT fault on stack ==== Test Assertion Failure ====
x86/int1_fred_test.c:114: fred_saved_rip == expected_rip pid=193114 tid=193114 errno=4 - Interrupted system call

     1  0x0000000000413319: assert_on_unhandled_exception at processor.c:659
     2  0x0000000000407d06: _vcpu_run at kvm_util.c:1703
     3   (inlined by) vcpu_run at kvm_util.c:1714
     4  0x0000000000403104: main at int1_fred_test.c:209
     5  0x00007f90b3c2a1c9: ?? ??:0
     6  0x00007f90b3c2a28a: ?? ??:0
     7  0x0000000000403314: _start at ??:?

0x40446b != 0xabaf80 (fred_saved_rip != expected_rip)

Signature · 2 lines
Kind regards
Maciej Wieczór-Retman
#123 lore raw
Quoted 7 lines · Maciej Wieczor-Retman

My theory is that after 'int3' call the FRED event is handled elsewhere and %rdi is not preserved. So the original version of the assembly looks okay but I was thinking that int3 has side effects.

Below is the test output of the RIPs not matching when running the original, does that help in any way?

Ah right, of course!

+"fred_entrypoint_kernel:\n"
+ "movq %rsp, %rdi\n"
+ "call fred_handler\n"
+ ".byte 0xf2, 0x0f, 0x01, 0xca\n" /* ERETS */
+ ".popsection\n"

That needs to save the %rdi it's scribbling on, plus all the callee- clobbered registers that fred_handler() might use.

#124 lore raw
In reply to Maciej Wieczor-Retman
Quoted 4 lines · Maciej Wieczor-Retman

My theory is that after 'int3' call the FRED event is handled elsewhere and %rdi is not preserved. So the original version of the assembly looks okay but I was thinking that int3 has side effects.

Please could you try the version at
https://git.infradead.org/?p=users/dwmw2/linux.git;a=commitdiff;h=fred
or this incremental patch:

tools/testing/selftests/kvm/x86/int1_fred_test.c +35 -2
index 7ffb54b56047..b91600abda62 100644
@@ -74,7 +74,7 @@ static volatile uint64_t fred_ss_value;
 static volatile uint64_t fred_saved_rip;
 static volatile bool fred_handler_called;
 
-static void __used fred_handler(struct fred_stack_frame *frame)
+static __attribute__((used)) void fred_handler(struct fred_stack_frame *frame)
 {
 	fred_ss_value = frame->ss;
 	fred_saved_rip = frame->rip;
@@ -99,8 +99,41 @@ asm(
 	/* Pad to +256 for ring 0 entry */
 	".org fred_entrypoint + 256, 0xcc\n"
 "fred_entrypoint_kernel:\n"
-	"movq %rsp, %rdi\n"
+	/* Save all GPRs — exception must be transparent to interrupted code */
+	"pushq %rax\n"
+	"pushq %rcx\n"
+	"pushq %rdx\n"
+	"pushq %rbx\n"
+	"pushq %rbp\n"
+	"pushq %rsi\n"
+	"pushq %rdi\n"
+	"pushq %r8\n"
+	"pushq %r9\n"
+	"pushq %r10\n"
+	"pushq %r11\n"
+	"pushq %r12\n"
+	"pushq %r13\n"
+	"pushq %r14\n"
+	"pushq %r15\n"
+	/* Pass pointer to FRED stack frame (above our saved regs) */
+	"leaq 120(%rsp), %rdi\n"
 	"call fred_handler\n"
+	/* Restore all GPRs */
+	"popq %r15\n"
+	"popq %r14\n"
+	"popq %r13\n"
+	"popq %r12\n"
+	"popq %r11\n"
+	"popq %r10\n"
+	"popq %r9\n"
+	"popq %r8\n"
+	"popq %rdi\n"
+	"popq %rsi\n"
+	"popq %rbp\n"
+	"popq %rbx\n"
+	"popq %rdx\n"
+	"popq %rcx\n"
+	"popq %rax\n"
 	".byte 0xf2, 0x0f, 0x01, 0xca\n"	/* ERETS */
 	".popsection\n"
 );
#125 lore raw
Quoted 9 lines · David Woodhouse
Quoted 4 lines · Maciej Wieczor-Retman

My theory is that after 'int3' call the FRED event is handled elsewhere and %rdi is not preserved. So the original version of the assembly looks okay but I was thinking that int3 has side effects.

Please could you try the version at
https://git.infradead.org/?p=users/dwmw2/linux.git;a=commitdiff;h=fred
or this incremental patch:

...

Quoted 1 line

Just tested it and now it works fine :)
(aside from the ICEBP thing of course but that's on the kernel side)

Signature · 2 lines
Kind regards
Maciej Wieczór-Retman
#126 lore raw
Quoted 2 lines · Maciej Wieczor-Retman

Just tested it and now it works fine :)

Great, thanks. Including the __attribute__((used)) part?

Quoted 1 line

(aside from the ICEBP thing of course but that's on the kernel side)

Well yes, that was kind of the point in generating the selftest.

I don't have access to hardware right now, but I can do test driven development by proxy... :)

#127 lore raw
Quoted 5 lines · David Woodhouse
Quoted 2 lines · Maciej Wieczor-Retman

Just tested it and now it works fine :)

Great, thanks. Including the __attribute__((used)) part?

Yeah, I didn't have to modify anything aside from commenting out the ICEBP part for the test to pass.

Quoted 6 lines
Quoted 1 line

(aside from the ICEBP thing of course but that's on the kernel side)

Well yes, that was kind of the point in generating the selftest.

I don't have access to hardware right now, but I can do test driven development by proxy... :)

Anytime :b

Signature · 2 lines
Kind regards
Maciej Wieczór-Retman
#128 lore raw
In reply to Xin Li (Intel)
Quoted 62 lines · Xin Li (Intel)

This patch set enables the Intel flexible return and event delivery (FRED) architecture with KVM VMX to allow guests to utilize FRED.

The FRED architecture defines simple new transitions that change privilege level (ring transitions). The FRED architecture was designed with the following goals:

1) Improve overall performance and response time by replacing event delivery through the interrupt descriptor table (IDT event delivery) and event return by the IRET instruction with lower latency transitions.

2) Improve software robustness by ensuring that event delivery establishes the full supervisor context and that event return establishes the full user context.

The new transitions defined by the FRED architecture are FRED event delivery and, for returning from events, two FRED return instructions. FRED event delivery can effect a transition from ring 3 to ring 0, but it is used also to deliver events incident to ring 0. One FRED instruction (ERETU) effects a return from ring 0 to ring 3, while the other (ERETS) returns while remaining in ring 0. Collectively, FRED event delivery and the FRED return instructions are FRED transitions.

Intel VMX architecture is extended to run FRED guests, and the major changes are:

1) New VMCS fields for FRED context management, which includes two new event data VMCS fields, eight new guest FRED context VMCS fields and eight new host FRED context VMCS fields.

2) VMX nested-exception support for proper virtualization of stack levels introduced with FRED architecture.

Search for the latest FRED spec in most search engines with this search pattern:

site:intel.com FRED (flexible return and event delivery) specification

Although FRED and CET supervisor shadow stacks are independent CPU features, FRED unconditionally includes FRED shadow stack pointer MSRs IA32_FRED_SSP[0123], and IA32_FRED_SSP0 is just an alias of the CET MSR IA32_PL0_SSP. IOW, the state management of MSR IA32_PL0_SSP becomes an overlap area, and Sean requested that FRED virtualization to land after CET virtualization [1].

With CET virtualization now merged in v6.18, the path is clear to submit the FRED virtualization patch series :).

Changes in v9:
* Rebased to the latest kvm-x86/next branch, tag kvm-x86-next-2025.10.20-2.
* Guard FRED state save/restore with guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) in patch 19 (syzbot & Chao).
* Use array indexing for exception stack access, eliminating the need for the ESTACKS_MEMBERS() macro in struct cea_exception_stacks, and then exported __this_cpu_ist_top_va() in a subsequent patch (Dave Hansen).
* Rewrote some of the change logs.

Quoted 5 lines

Following is the link to v8 of this patch set:
https://lore.kernel.org/lkml/[email protected]/

[1]: https://lore.kernel.org/kvm/[email protected]/

Hi Xin,

Any rough idea when v10 might go out? I'm about to post v3 of the SVM side and would rather base it on your latest than on v9, so happy to wait if it's close.

If it's likely to be a while, a question for the maintainers: would it be reasonable for me to carry the common vFRED parts of this series as prerequisites in my posting, with authorship and tags intact? Some would need splitting out of the VMX patches, so only if Xin is fine with it.

Thanks,
Shivansh

#129 lore raw

Hi Shivansh,

Quoted 4 lines · Shivansh Dhiman

Any rough idea when v10 might go out? I'm about to post v3 of the SVM side and would rather base it on your latest than on v9, so happy to wait if it's close.

I am planning to send the v10 patchset soon after the merge window closes. Recently, I received a set of updated patches from Xin (offlist). I am in the process of resolving some minor issues based on internal testing and AI reviews.

Barring any last minute hiccups, I am hoping to post the series around the 7.3-rc1/rc2 timeframe. If you want an early look, the work-in-progress patches are located at:
https://github.com/sohilmeh/linux-dev/tree/fred-kvm-next.

Sohil

#130 lore raw
Quoted 16 lines · Sohil Mehta

[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Hi Shivansh,

Quoted 4 lines · Shivansh Dhiman

Any rough idea when v10 might go out? I'm about to post v3 of the SVM side and would rather base it on your latest than on v9, so happy to wait if it's close.

I am planning to send the v10 patchset soon after the merge window closes. Recently, I received a set of updated patches from Xin (offlist). I am in the process of resolving some minor issues based on internal testing and AI reviews.

Quoted 4 lines

Barring any last minute hiccups, I am hoping to post the series around the 7.3-rc1/rc2 timeframe. If you want an early look, the work-in-progress patches are located at:
https://github.com/sohilmeh/linux-dev/tree/fred-kvm-next.

Thanks Sohil for the update and the tree. The timeline works for me. I'll hold v3 and post the SVM side rebased on top of v10 after you post.

Cheers,
Shivansh

Quoted 3 lines

Sohil