Add support for the Idle HLT intercept feature
cover Add support for the Idle HLT intercept feature
The upcoming new Idle HLT Intercept feature allows for the HLT instruction execution by a vCPU to be intercepted by the hypervisor only if there are no pending V_INTR and V_NMI events for the vCPU. When the vCPU is expected to service the pending V_INTR and V_NMI events, the Idle HLT intercept won’t trigger. The feature allows the hypervisor to determine if the vCPU is actually idle and reduces wasteful VMEXITs.
Presence of the Idle HLT Intercept feature is indicated via CPUID function Fn8000_000A_EDX[30].
Document for the Idle HLT intercept feature will be available in the next version of "AMD64 Architecture Programmer’s Manual".
Testing Done:
Added a selftest to test the Idle HLT intercept functionality. Tested SEV and SEV-ES guest for the Idle HLT intercept functionality.
Manali Shukla (5):
x86/cpufeatures: Add CPUID feature bit for Idle HLT intercept
KVM: SVM: Add Idle HLT intercept support
tools: Add KVM exit reason for the Idle HLT
selftests: Add an interface to read the data of named vcpu stat
selftests: KVM: SVM: Add Idle HLT intercept test
arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/svm.h | 1 + arch/x86/include/uapi/asm/svm.h | 2 + arch/x86/kvm/svm/svm.c | 11 +- tools/arch/x86/include/uapi/asm/svm.h | 2 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/kvm_util_base.h | 11 ++ tools/testing/selftests/kvm/lib/kvm_util.c | 41 ++++++ .../selftests/kvm/x86_64/svm_idlehlt_test.c | 119 ++++++++++++++++++ 9 files changed, 186 insertions(+), 3 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c
base-commit: fdd58834d132046149699b88a27a0db26829f4fb
Signature · 1 line
2.34.1
PATCH 1/5 x86/cpufeatures: Add CPUID feature bit for Idle HLT intercept
From: Manali Shukla <[email protected]>
The Idle HLT Intercept feature allows for the HLT instruction execution by a vCPU to be intercepted by the hypervisor only if there are no pending events (V_INTR and V_NMI) for the vCPU. When the vCPU is expected to service the pending events (V_INTR and V_NMI), the Idle HLT intercept won’t trigger. The feature allows the hypervisor to determine if the vCPU is idle and reduces wasteful VMEXITs.
Presence of the Idle HLT intercept feature for guests is indicated via CPUID function 0x8000000A_EDX[30].
--- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -379,6 +379,7 @@#define X86_FEATURE_V_SPEC_CTRL (15*32+20)/* Virtual SPEC_CTRL */ #define X86_FEATURE_VNMI (15*32+25)/* Virtual NMI */ #define X86_FEATURE_SVME_ADDR_CHK (15*32+28)/* "" SVME addr check */ +#define X86_FEATURE_IDLE_HLT (15*32+30)/* "" IDLE HLT intercept */ /* Intel-defined CPU features, CPUID level 0x00000007:0 (ECX), word 16 */ #define X86_FEATURE_AVX512VBMI (16*32+ 1)/* AVX512 Vector Bit Manipulation instructions*/
Signature · 1 line
2.34.1
PATCH 2/5 KVM: SVM: Add Idle HLT intercept support
In reply to Manali ShuklaFrom: Manali Shukla <[email protected]>
Execution of the HLT instruction by a vCPU can be intercepted by the hypervisor by setting the HLT-Intercept Bit in VMCB, thus resulting in a VMEXIT. It can be possible that soon after the VMEXIT, hypervisor observes that there are pending V_INTR and V_NMI events for the vCPU and causes it to perform a VMRUN to service those events. In that case, the VMEXIT is wasteful.
The Idle HLT intercept feature allows for the HLT instruction execution by a vCPU to be intercepted by the hypervisor only if there are no pending V_INTR and V_NMI events for the vCPU. The Idle HLT intercept will not be triggerred, when the vCPU is expected to have pending events (V_INR and V_NMI).
The feature allows the hypervisor to determine whether the vCPU is actually idle and reduces wasteful VMEXITs.
--- arch/x86/include/asm/svm.h | 1 + arch/x86/include/uapi/asm/svm.h | 2 ++ arch/x86/kvm/svm/svm.c | 11 ++++++++--- 3 files changed, 11 insertions(+), 3 deletions(-)
arch/x86/include/asm/svm.h +1 -0
@@ -116,6 +116,7 @@ enum {INTERCEPT_INVPCID ,INTERCEPT_MCOMMIT ,INTERCEPT_TLBSYNC , + INTERCEPT_IDLE_HLT = 166, };
arch/x86/include/uapi/asm/svm.h +2 -0
@@ -95,6 +95,7 @@#define SVM_EXIT_CR14_WRITE_TRAP 0x09e#define SVM_EXIT_CR15_WRITE_TRAP 0x09f#define SVM_EXIT_INVPCID 0x0a2 +#define SVM_EXIT_IDLE_HLT 0x0a6#define SVM_EXIT_NPF 0x400#define SVM_EXIT_AVIC_INCOMPLETE_IPI 0x401#define SVM_EXIT_AVIC_UNACCELERATED_ACCESS 0x402
@@ -223,6 +224,7 @@ { SVM_EXIT_CR4_WRITE_TRAP, "write_cr4_trap" }, \ { SVM_EXIT_CR8_WRITE_TRAP, "write_cr8_trap" }, \ { SVM_EXIT_INVPCID, "invpcid" }, \ + { SVM_EXIT_IDLE_HLT, "idle-halt" }, \ { SVM_EXIT_NPF, "npf" }, \ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \
arch/x86/kvm/svm/svm.c +8 -3
@@ -1289,8 +1289,12 @@ static void init_vmcb(struct kvm_vcpu *vcpu)svm_set_intercept (svm,INTERCEPT_MWAIT ); } -if (!kvm_hlt_in_guest (vcpu->kvm)) -svm_set_intercept (svm,INTERCEPT_HLT ); +if (!kvm_hlt_in_guest (vcpu->kvm)) { +if (cpu_feature_enabled (X86_FEATURE_IDLE_HLT )) +svm_set_intercept (svm,INTERCEPT_IDLE_HLT ); +else +svm_set_intercept (svm,INTERCEPT_HLT ); + } control->iopm_base_pa =__sme_set (iopm_base); control->msrpm_base_pa =__sme_set (__pa (svm->msrpm));
@@ -3302,6 +3306,7 @@ static int (*const svm_exit_handlers[])(struct kvm_vcpu *vcpu) = { [SVM_EXIT_CR4_WRITE_TRAP] = cr_trap, [SVM_EXIT_CR8_WRITE_TRAP] = cr_trap, [SVM_EXIT_INVPCID] = invpcid_interception, + [SVM_EXIT_IDLE_HLT] = kvm_emulate_halt, [SVM_EXIT_NPF] = npf_interception, [SVM_EXIT_RSM] = rsm_interception, [SVM_EXIT_AVIC_INCOMPLETE_IPI] = avic_incomplete_ipi_interception,
@@ -3462,7 +3467,7 @@ int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)return interrupt_window_interception (vcpu); else if (exit_code == SVM_EXIT_INTR) return intr_interception(vcpu); - else if (exit_code == SVM_EXIT_HLT) + else if (exit_code == SVM_EXIT_HLT || exit_code == SVM_EXIT_IDLE_HLT) return kvm_emulate_halt(vcpu); else if (exit_code == SVM_EXIT_NPF) return npf_interception(vcpu);
Signature · 1 line
2.34.1
PATCH 3/5 tools: Add KVM exit reason for the Idle HLT
In reply to Manali ShuklaFrom: Manali Shukla <[email protected]>
The Idle HLT intercept feature allows for the HLT instruction execution by a vCPU to be intercepted by hypervisor only if there are no pending V_INR and V_NMI events for the vCPU. The Idle HLT intercept will not be triggerred when vCPU is expected to service pending events (V_INTR and V_NMI).
The new SVM_EXIT_IDLE_HLT is introduced as part of the Idle HLT intercept feature. Add it to SVM_EXIT_REASONS, so that the SVM_EXIT_IDLE_HLT type of VMEXIT is recognized by tools like perf etc.
--- tools/arch/x86/include/uapi/asm/svm.h | 2 ++ 1 file changed, 2 insertions(+)
tools/arch/x86/include/uapi/asm/svm.h +2 -0
@@ -95,6 +95,7 @@#define SVM_EXIT_CR14_WRITE_TRAP 0x09e#define SVM_EXIT_CR15_WRITE_TRAP 0x09f#define SVM_EXIT_INVPCID 0x0a2 +#define SVM_EXIT_IDLE_HLT 0x0a6#define SVM_EXIT_NPF 0x400#define SVM_EXIT_AVIC_INCOMPLETE_IPI 0x401#define SVM_EXIT_AVIC_UNACCELERATED_ACCESS 0x402
@@ -224,6 +225,7 @@ { SVM_EXIT_CR8_WRITE_TRAP, "write_cr8_trap" }, \ { SVM_EXIT_INVPCID, "invpcid" }, \ { SVM_EXIT_NPF, "npf" }, \ + { SVM_EXIT_IDLE_HLT, "idle-halt" }, \ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \ { SVM_EXIT_VMGEXIT, "vmgexit" }, \
Signature · 1 line
2.34.1
Sean Christopherson
maintainerRe: [PATCH v1 3/5] tools: Add KVM exit reason for the Idle HLT
Quoted 8 lines · Manali Shukla
From: Manali Shukla <[email protected]>
The Idle HLT intercept feature allows for the HLT instruction execution by a vCPU to be intercepted by hypervisor only if there are no pending
V_INRandV_NMIevents for the vCPU. The Idle HLT intercept will not be triggerred when vCPU is expected to service pending events (V_INTRandV_NMI).
Quoted 7 lines
The new
SVM_EXIT_IDLE_HLTis introduced as part of the Idle HLT intercept feature. Add it toSVM_EXIT_REASONS, so that theSVM_EXIT_IDLE_HLTtype of VMEXIT is recognized by tools like perf etc.Signed-off-by Manali Shukla <[email protected]>--- tools/arch/x86/include/uapi/asm/svm.h | 2 ++
Please drop the tools/ uapi headers update. Nothing KVM-related in tools/ actually relies on the headers being copied into tools/, e.g. KVM selftests pulls KVM's headers from the .../usr/include/ directory that's populated by make headers_install.
Perf's tooling is what actually "needs" the headers to be copied into tools/; let the tools/perf maintainers deal with the headache of keeping everything up-to-date.
Re: [PATCH v1 3/5] tools: Add KVM exit reason for the Idle HLT
Hi Sean,
Thank you for reviewing my patches
Quoted 17 lines · Sean Christopherson
Quoted 8 lines · Manali Shukla
From: Manali Shukla <[email protected]>
The Idle HLT intercept feature allows for the HLT instruction execution by a vCPU to be intercepted by hypervisor only if there are no pending
V_INRandV_NMIevents for the vCPU. The Idle HLT intercept will not be triggerred when vCPU is expected to service pending events (V_INTRandV_NMI).Quoted 7 lines
The new
SVM_EXIT_IDLE_HLTis introduced as part of the Idle HLT intercept feature. Add it toSVM_EXIT_REASONS, so that theSVM_EXIT_IDLE_HLTtype of VMEXIT is recognized by tools like perf etc.Signed-off-by Manali Shukla <[email protected]>--- tools/arch/x86/include/uapi/asm/svm.h | 2 ++
Quoted 7 lines
Please drop the
tools/uapi headers update. Nothing KVM-related intools/actually relies on the headers being copied intotools/, e.g. KVM selftests pulls KVM's headers from the .../usr/include/ directory that's populated bymake headers_install.Perf's tooling is what actually "needs" the headers to be copied into
tools/; let thetools/perfmaintainers deal with the headache of keeping everything up-to-date.
Sure I will drop this patch in V2.
-Manali
PATCH 4/5 selftests: Add an interface to read the data of named vcpu stat
In reply to Manali ShuklaFrom: Manali Shukla <[email protected]>
The interface is used to read the data values of a specified vcpu stat from the currenly available binary stats interface.
--- .../selftests/kvm/include/kvm_util_base.h | 11 +++++ tools/testing/selftests/kvm/lib/kvm_util.c | 41 +++++++++++++++++++ 2 files changed, 52 insertions(+)
tools/testing/selftests/kvm/include/kvm_util_base.h +11 -0
@@ -512,6 +512,17 @@ void read_stat_data(int stats_fd, struct kvm_stats_header *header, struct kvm_stats_desc *desc, uint64_t *data, size_t max_elements); +void __vcpu_get_stat (struct kvm_vcpu *vcpu,const char *stat_name,uint64_t *data, +size_t max_elements); + +static inline uint64_t vcpu_get_stat (struct kvm_vcpu *vcpu,const char *stat_name) +{ +uint64_t data; + +__vcpu_get_stat (vcpu, stat_name, &data,1 ); +return data; +} +void __vm_get_stat (struct kvm_vm *vm,const char *stat_name,uint64_t *data,size_t max_elements);
tools/testing/selftests/kvm/lib/kvm_util.c +41 -0
47 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH 5/5 selftests: KVM: SVM: Add Idle HLT intercept test
In reply to Manali ShuklaFrom: Manali Shukla <[email protected]>
The Execution of the HLT instruction results in a VMEXIT. Hypervisor observes pending V_INTR and V_NMI events just after the VMEXIT generated by the HLT for the vCPU and causes VM entry to service the pending events. The Idle HLT intercept feature avoids the wasteful VMEXIT during the halt if there are pending V_INTR and V_NMI events for the vCPU.
The selftest for the Idle HLT intercept instruments the above-mentioned scenario.
--- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/x86_64/svm_idlehlt_test.c | 118 ++++++++++++++++++ 2 files changed, 119 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c
tools/testing/selftests/kvm/Makefile +1 -0
@@ -89,6 +89,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/smaller_maxphyaddr_emulation_test TEST_GEN_PROGS_x86_64 += x86_64/smm_test TEST_GEN_PROGS_x86_64 += x86_64/state_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_preemption_timer_test +TEST_GEN_PROGS_x86_64 += x86_64/svm_idlehlt_test TEST_GEN_PROGS_x86_64 += x86_64/svm_vmcall_test TEST_GEN_PROGS_x86_64 += x86_64/svm_int_ctl_test TEST_GEN_PROGS_x86_64 += x86_64/svm_nested_shutdown_test
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +118 -0
118 lines not on this page · raw message
Signature · 1 line
2.34.1
Sean Christopherson
maintainerRe: [PATCH v1 5/5] selftests: KVM: SVM: Add Idle HLT intercept test
Quoted 24 lines · Manali Shukla
From: Manali Shukla <[email protected]>
The Execution of the HLT instruction results in a VMEXIT. Hypervisor observes pending
V_INTRandV_NMIevents just after the VMEXIT generated by the HLT for the vCPU and causes VM entry to service the pending events. The Idle HLT intercept feature avoids the wasteful VMEXIT during the halt if there are pendingV_INTRandV_NMIevents for the vCPU.The selftest for the Idle HLT intercept instruments the above-mentioned scenario.
Signed-off-by Manali Shukla <[email protected]>--- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/x86_64/svm_idlehlt_test.c | 118 ++++++++++++++++++ 2 files changed, 119 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.ctools/testing/selftests/kvm/Makefile +0
-0
tools/testing/selftests/kvm/Makefile TEST_GEN_PROGS_x86_64 += x86_64/smaller_maxphyaddr_emulation_test · line 92
Quoted 4 lines
tools/testing/selftests/kvm/Makefile +1
-0@@ -89,6 +89,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/smaller_maxphyaddr_emulation_test TEST_GEN_PROGS_x86_64 += x86_64/smm_test TEST_GEN_PROGS_x86_64 += x86_64/state_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_preemption_timer_test +TEST_GEN_PROGS_x86_64 += x86_64/svm_idlehlt_test
Uber nit, maybe svm_idle_hlt_test? I find "idlehlt" hard to parse.
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 4
Quoted 13 lines
tools/testing/selftests/kvm/Makefile +0
-0TEST_GEN_PROGS_x86_64 += x86_64/svm_vmcall_test TEST_GEN_PROGS_x86_64 += x86_64/svm_int_ctl_test TEST_GEN_PROGS_x86_64 += x86_64/svm_nested_shutdown_testtools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +4
-0@@ -0,0 +1,118 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* svm_idlehalt_test +*
Please omit this, file comments that state the name of the test inevitably become stale (see above).
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 7
Quoted 3 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +3
-0+* Copyright (C) 2024 Advanced Micro Devices, Inc. +* +* For licencing details see kernel-base/COPYING
This seems gratuitous, doesn't the SPDX stuff take care this?
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 19
Quoted 12 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +12
-0+* +* Author: +* Manali Shukla <[email protected]> +*/ +#include "kvm_util.h" +#include "svm_util.h" +#include "processor.h" +#include "test_util.h" +#include "apic.h" + +#define VINTR_VECTOR 0x30 +#define NUM_ITERATIONS 100000
What's the runtime? If it's less than a second, then whatever, but if it's at all longer than that, then I'd prefer to use a lower default and make this user- configurable.
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 23
Quoted 3 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +3
-0+/* +* Incremented in the VINTR handler. Provides evidence to the sender that the +* VINR is arrived at the destination.
Evidence is useless if there's no detective looking for it. Yeah, it gets printed out in the end, but in reality, no one is going to look at that.
Rather than read this from the host, just make it a non-volatile bool and assert that it set after every
Quoted 8 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +8
-0+*/ +static volatile uint64_t vintr_rcvd; + +void verify_apic_base_addr (void ) +{ +uint64_t msr =rdmsr (MSR_IA32_APICBASE ); +uint64_t base =GET_APIC_BASE (msr); +
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 41
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +10
-0+GUEST_ASSERT (base ==APIC_DEFAULT_GPA ); +} + +/* +* The halting guest code instruments the scenario where there is a V_INTR pending +* event available while hlt instruction is executed. The HLT VM Exit doesn't +* occur in above-mentioned scenario if the Idle HLT intercept feature is enabled. +*/ + +static void halter_guest_code (void )
Just "guest_code()". Yeah, it's a weird generic name, but at this point it's so ubiquitous that it's analogous to main(), i.e. readers know that guest_code() is the entry point. And deviating from that suggests that there is a second vCPU running _other_ guest code (otherwise, why differentiate?), which isn't the case.
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 46
Quoted 5 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +5
-0+{ +uint32_t icr_val; +int i; + +verify_apic_base_addr ();
Why? The test will fail if the APIC is borked, this is just unnecessary noise that distracts readers.
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 53
Quoted 7 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +7
-0+xapic_enable (); + + icr_val = (APIC_DEST_SELF |APIC_INT_ASSERT |VINTR_VECTOR ); + +for (i =0 ; i <NUM_ITERATIONS ; i++) { +xapic_write_reg (APIC_ICR , icr_val); + asmvolatile ("sti; hlt; cli" );
Please add safe_halt() and cli() helpers in processor.h. And then do:
cli(); xapic_write_reg(APIC_ICR, icr_val); safe_halt();
to guarantee that interrupts are disabled when the IPI is sent. And as above,
GUEST_ASSERT(READ_ONCE(irq_received)); WRITE_ONCE(irq_received, false);
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 61
Quoted 8 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +8
-0+ } +GUEST_DONE (); +} + +static void guest_vintr_handler (struct ex_regs *regs) +{ + vintr_rcvd++; +xapic_write_reg (APIC_EOI ,0x30 );
EOI is typically written with '0', not the vector, because the legacy EOI register clears the highest ISR vector, not whatever is specified. IIRC, one of the Intel or AMD specs even says to use '0'.
AMD's Specific EOI register does allow targeting a specific vector, but that's not what's being used here.
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 72
Quoted 11 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +11
-0+} + +int main (int argc,char *argv[]) +{ +struct kvm_vm *vm; +struct kvm_vcpu *vcpu; +struct ucall uc; +uint64_t halt_exits, vintr_exits; +uint64_t *pvintr_rcvd; + +TEST_REQUIRE (kvm_cpu_has (X86_FEATURE_SVM ));
No, this test doesn't require SVM, which is KVM advertising *nested* SVM. This test does require idle-hlt support though...
Quoted 4 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +4
-0+/* Check the extension for binary stats */ +TEST_REQUIRE (kvm_has_cap (KVM_CAP_BINARY_STATS_FD )); + + vm =vm_create_with_one_vcpu (&vcpu, halter_guest_code);
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 87
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +10
-0+ +vm_init_descriptor_tables (vm); +vcpu_init_descriptor_tables (vcpu); +vm_install_exception_handler (vm,VINTR_VECTOR , guest_vintr_handler); +virt_pg_map (vm,APIC_DEFAULT_GPA ,APIC_DEFAULT_GPA ); + +vcpu_run (vcpu); +TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_IO ); + + halt_exits =vcpu_get_stat (vcpu,"halt_exits" );
Is there really no way to get binary stats without having to pass in strings?
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 94
Quoted 7 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +7
-0+ vintr_exits =vcpu_get_stat (vcpu,"irq_window_exits" ); + pvintr_rcvd = (uint64_t *)addr_gva2hva (vm, (uint64_t )&vintr_rcvd); + +switch (get_ucall (vcpu, &uc)) { +case UCALL_ABORT : +REPORT_GUEST_ASSERT (uc); +/* NOT REACHED */
Eh, just put a "break;" here and drop the comment.
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 96
Quoted 2 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +2
-0+case UCALL_DONE : + goto done;
break;
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 102
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +6
-0+default : +TEST_FAIL ("Unknown ucall 0x%lx." , uc.cmd); + } + +done: + TEST_ASSERT(halt_exits == 0,
So in all honesty, this isn't a very interesting test. It's more of a CPU test than a KVM test. I do think it's worth adding, because why not.
But I'd also like to see a testcase for KVM_X86_DISABLE_EXITS_HLT. It would be a generic test, i.e. not specific to idle-hlt since there is no dependency and the test shouldn't care. I _think_ it would be fairly straightforward: create a VM without an in-kernel APIC (so that HLT exits to userspace), disable HLT interception, send a signal from a different task after some delay, and execute HLT in the guest. Then verify the vCPU exited because of -EINTR and not HLT.
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 107
Quoted 5 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +5
-0+"Test Failed:\n" +"Guest executed VINTR followed by halts: %d times\n" +"The guest exited due to halt: %ld times and number\n" +"of vintr exits: %ld and vintr got re-injected: %ld times\n" , + NUM_ITERATIONS, halt_exits, vintr_exits, *pvintr_rcvd);
I appreciate the effort to provide more info, but this is way too noisy. If anything, print gory details in a pr_debug() *before* the assert (see below), and then simply do:
TEST_ASSERT_EQ(halt_exits, 0);
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 114
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +6
-0+fprintf (stderr, +"Test Successful:\n" +"Guest executed VINTR followed by halts: %d times\n" +"The guest exited due to halt: %ld times and number\n" +"of vintr exits: %ld and vintr got re-injected: %ld times\n" , +NUM_ITERATIONS , halt_exits, vintr_exits, *pvintr_rcvd);
And this should be pr_debug(), because no human is going to look at this except when the test isn't working correctly.
Quoted 7 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +4
-0+ +kvm_vm_free (vm); +return 0 ; +}Signature · 2 lines
2.34.1 /pvintr_rcvd
Sean Christopherson
maintainerRe: [PATCH v1 5/5] selftests: KVM: SVM: Add Idle HLT intercept test
Quoted 10 lines · Sean Christopherson
On Thu, Mar 07, 2024, Manali Shukla wrote: From: Manali Shukla <[email protected]>
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 53
Quoted 7 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +7
-0+xapic_enable (); + + icr_val = (APIC_DEST_SELF |APIC_INT_ASSERT |VINTR_VECTOR ); + +for (i =0 ; i <NUM_ITERATIONS ; i++) { +xapic_write_reg (APIC_ICR , icr_val); + asmvolatile ("sti; hlt; cli" );Please add
safe_halt()andcli()helpers inprocessor.h. And then do:
Doh, saw something shiny and forgot to finish my though. For safe_halt(), copy the thing verbatim from KVM-Unit-Tests, as not everyone is familiar with the sti=>hlt trick.
/*
* Execute HLT in an STI interrupt shadow to ensure that a pending IRQ that's
* intended to be a wake event arrives *after* HLT is executed. Modern CPUs,
* except for a few oddballs that KVM is unlikely to run on, block IRQs for one
* instruction after STI, *if* RFLAGS.IF=0 before STI. Note, Intel CPUs may
* block other events beyond regular IRQs, e.g. may block NMIs and SMIs too. */
static inline void safe_halt(void)
{
asm volatile("sti; hlt");
}Re: [PATCH v1 5/5] selftests: KVM: SVM: Add Idle HLT intercept test
In reply to Sean ChristophersonHi Sean,
Thank you for reviewing my patches.
Quoted 20 lines · Sean Christopherson
Quoted 19 lines · Manali Shukla
From: Manali Shukla <[email protected]>
The Execution of the HLT instruction results in a VMEXIT. Hypervisor observes pending
V_INTRandV_NMIevents just after the VMEXIT generated by the HLT for the vCPU and causes VM entry to service the pending events. The Idle HLT intercept feature avoids the wasteful VMEXIT during the halt if there are pendingV_INTRandV_NMIevents for the vCPU.The selftest for the Idle HLT intercept instruments the above-mentioned scenario.
Signed-off-by Manali Shukla <[email protected]>--- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/x86_64/svm_idlehlt_test.c | 118 ++++++++++++++++++ 2 files changed, 119 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c
Quoted 11 lines
tools/testing/selftests/kvm/Makefile TEST_GEN_PROGS_x86_64 += x86_64/smaller_maxphyaddr_emulation_test · line 92
Quoted 9 lines
tools/testing/selftests/kvm/Makefile +1
-0@@ -89,6 +89,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/smaller_maxphyaddr_emulation_test TEST_GEN_PROGS_x86_64 += x86_64/smm_test TEST_GEN_PROGS_x86_64 += x86_64/state_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_preemption_timer_test +TEST_GEN_PROGS_x86_64 += x86_64/svm_idlehlt_testUber nit, maybe
svm_idle_hlt_test? I find "idlehlt" hard to parse.
Sure I will change it to svm_idle_hlt_test.
Quoted 17 lines
Quoted 10 lines
tools/testing/selftests/kvm/Makefile +0
-0TEST_GEN_PROGS_x86_64 += x86_64/svm_vmcall_test TEST_GEN_PROGS_x86_64 += x86_64/svm_int_ctl_test TEST_GEN_PROGS_x86_64 += x86_64/svm_nested_shutdown_testtools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +0
-0tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 4
Quoted 4 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +4
-0@@ -0,0 +1,118 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* svm_idlehalt_test +* Please omit this, file comments that state the name of the test inevitably become stale (see above).
Sure. I will remove it.
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 7
Quoted 4 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +3
-0+* Copyright (C) 2024 Advanced Micro Devices, Inc. +* +* For licencing details see kernel-base/COPYING This seems gratuitous, doesn't the SPDX stuff take care this?
Agreed, I will remove this.
Quoted 17 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 19
Quoted 13 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +12
-0+* +* Author: +* Manali Shukla <[email protected]> +*/ +#include "kvm_util.h" +#include "svm_util.h" +#include "processor.h" +#include "test_util.h" +#include "apic.h" + +#define VINTR_VECTOR 0x30 +#define NUM_ITERATIONS 100000What's the runtime? If it's less than a second, then whatever, but if it's at all longer than that, then I'd prefer to use a lower default and make this user- configurable.
It takes ~34s to run this test.
Quoted 11 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 23
Quoted 4 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +3
-0+/* +* Incremented in the VINTR handler. Provides evidence to the sender that the +* VINR is arrived at the destination. Evidence is useless if there's no detective looking for it. Yeah, it gets printed out in the end, but in reality, no one is going to look at that.
Rather than read this from the host, just make it a non-volatile bool and assert that it set after every
Sure. I can do that.
Quoted 15 lines
Quoted 15 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +15
-0+*/ +static volatile uint64_t vintr_rcvd; + +void verify_apic_base_addr (void ) +{ +uint64_t msr =rdmsr (MSR_IA32_APICBASE ); +uint64_t base =GET_APIC_BASE (msr); + +GUEST_ASSERT (base ==APIC_DEFAULT_GPA ); +} + +/* +* The halting guest code instruments the scenario where there is a V_INTR pending +* event available while hlt instruction is executed. The HLT VM Exit doesn't +* occur in above-mentioned scenario if the Idle HLT intercept feature is enabled.
Quoted 9 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 41
Quoted 3 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +3
-0+*/ + +static void halter_guest_code (void )Just "
guest_code()". Yeah, it's a weird generic name, but at this point it's so ubiquitous that it's analogous tomain(), i.e. readers know thatguest_code()is the entry point. And deviating from that suggests that there is a second vCPU running _other_ guest code (otherwise, why differentiate?), which isn't the case.
Sure.
Quoted 21 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 46
Quoted 5 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +5
-0+{ +uint32_t icr_val; +int i; + +verify_apic_base_addr ();Why? The test will fail if the APIC is borked, this is just unnecessary noise that distracts readers.
Sure. I will remove it in V2.
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 53
Quoted 7 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +7
-0+xapic_enable (); + + icr_val = (APIC_DEST_SELF |APIC_INT_ASSERT |VINTR_VECTOR ); + +for (i =0 ; i <NUM_ITERATIONS ; i++) { +xapic_write_reg (APIC_ICR , icr_val); + asmvolatile ("sti; hlt; cli" );Please add
safe_halt()andcli()helpers inprocessor.h. And then do:
Quoted 9 lines
cli(); xapic_write_reg(APIC_ICR, icr_val); safe_halt();to guarantee that interrupts are disabled when the IPI is sent. And as above,
GUEST_ASSERT(READ_ONCE(irq_received)); WRITE_ONCE(irq_received, false);
Sure.
Quoted 13 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 61
Quoted 8 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +8
-0+ } +GUEST_DONE (); +} + +static void guest_vintr_handler (struct ex_regs *regs) +{ + vintr_rcvd++; +xapic_write_reg (APIC_EOI ,0x30 );EOI is typically written with '0', not the vector, because the legacy EOI register clears the highest ISR vector, not whatever is specified. IIRC, one of the Intel or AMD specs even says to use '0'.
Quoted 2 lines
AMD's Specific EOI register does allow targeting a specific vector, but that's not what's being used here.
Sure.
Quoted 15 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 72
Quoted 12 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +11
-0+} + +int main (int argc,char *argv[]) +{ +struct kvm_vm *vm; +struct kvm_vcpu *vcpu; +struct ucall uc; +uint64_t halt_exits, vintr_exits; +uint64_t *pvintr_rcvd; + +TEST_REQUIRE (kvm_cpu_has (X86_FEATURE_SVM ));No, this test doesn't require SVM, which is KVM advertising *nested* SVM. This test does require idle-hlt support though...
Sure. I will add it in V2.
Quoted 17 lines
Quoted 5 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +4
-0+/* Check the extension for binary stats */ +TEST_REQUIRE (kvm_has_cap (KVM_CAP_BINARY_STATS_FD )); + + vm =vm_create_with_one_vcpu (&vcpu, halter_guest_code);tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 87
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +10
-0+ +vm_init_descriptor_tables (vm); +vcpu_init_descriptor_tables (vcpu); +vm_install_exception_handler (vm,VINTR_VECTOR , guest_vintr_handler); +virt_pg_map (vm,APIC_DEFAULT_GPA ,APIC_DEFAULT_GPA ); + +vcpu_run (vcpu); +TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_IO ); + + halt_exits =vcpu_get_stat (vcpu,"halt_exits" );Is there really no way to get binary stats without having to pass in strings?
I could see one of the test case is passing the strings to get binary stats of vm. That is why I used strings to get binary stats of vcpu. I will try to find another way to get the binary stats.
Quoted 11 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 94
Quoted 8 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +7
-0+ vintr_exits =vcpu_get_stat (vcpu,"irq_window_exits" ); + pvintr_rcvd = (uint64_t *)addr_gva2hva (vm, (uint64_t )&vintr_rcvd); + +switch (get_ucall (vcpu, &uc)) { +case UCALL_ABORT : +REPORT_GUEST_ASSERT (uc); +/* NOT REACHED */ Eh, just put a "break;" here and drop the comment.
Sure.
Quoted 15 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 96
Quoted 2 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +2
-0+case UCALL_DONE : + goto done;break;
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 102
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +6
-0+default : +TEST_FAIL ("Unknown ucall 0x%lx." , uc.cmd); + } + +done: + TEST_ASSERT(halt_exits == 0,So in all honesty, this isn't a very interesting test. It's more of a CPU test than a KVM test. I do think it's worth adding, because why not.
Quoted 6 lines
But I'd also like to see a testcase for
KVM_X86_DISABLE_EXITS_HLT. It would be a generic test, i.e. not specific to idle-hlt since there is no dependency and the test shouldn't care. I _think_ it would be fairly straightforward: create a VM without an in-kernel APIC (so that HLT exits to userspace), disable HLT interception, send a signal from a different task after some delay, and execute HLT in the guest. Then verify the vCPU exited because of-EINTRand not HLT.
I will create this test.
Quoted 11 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 107
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +5
-0+"Test Failed:\n" +"Guest executed VINTR followed by halts: %d times\n" +"The guest exited due to halt: %ld times and number\n" +"of vintr exits: %ld and vintr got re-injected: %ld times\n" , + NUM_ITERATIONS, halt_exits, vintr_exits, *pvintr_rcvd);I appreciate the effort to provide more info, but this is way too noisy. If anything, print gory details in a
pr_debug()*before* the assert (see below), and then simply do:
Quoted 2 lines
TEST_ASSERT_EQ(halt_exits, 0);
Sure.
Quoted 9 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 114
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +6
-0+fprintf (stderr, +"Test Successful:\n" +"Guest executed VINTR followed by halts: %d times\n" +"The guest exited due to halt: %ld times and number\n" +"of vintr exits: %ld and vintr got re-injected: %ld times\n" , +NUM_ITERATIONS , halt_exits, vintr_exits, *pvintr_rcvd);And this should be
pr_debug(), because no human is going to look at this except when the test isn't working correctly.
I will change it to pr_debug() in V2.
Quoted 9 lines
Quoted 8 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +4
-0+ +kvm_vm_free (vm); +return 0 ; +}Signature · 2 lines
2.34.1 /pvintr_rcvd
Sean Christopherson
maintainerRe: [PATCH v1 5/5] selftests: KVM: SVM: Add Idle HLT intercept test
Quoted 8 lines · Manali Shukla
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 19
Quoted 2 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +2
-0+#define VINTR_VECTOR 0x30 +#define NUM_ITERATIONS 100000What's the runtime? If it's less than a second, then whatever, but if it's at all longer than that, then I'd prefer to use a lower default and make this user- configurable.
It takes ~34s to run this test.
LOL, yeah, no. That's 33+ seconds of wasted time. From a *KVM* perspective, this is quite binary: either KVM intercepts HLT or it doesn't. Any timing bugs would be purely CPU bugs.
Please adjust this to have the default runtime <1 second. If you feel strongly about the need for a long-running test, feel free to add a command line option to control the number of iterations.
Re: [PATCH v1 5/5] selftests: KVM: SVM: Add Idle HLT intercept test
Quoted 8 lines · Sean Christopherson
Quoted 7 lines · Manali Shukla
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c line 19
Quoted 2 lines
tools/testing/selftests/kvm/x86_64/svm_idlehlt_test.c new file +2
-0+#define VINTR_VECTOR 0x30 +#define NUM_ITERATIONS 100000What's the runtime? If it's less than a second, then whatever, but if it's at all longer than that, then I'd prefer to use a lower default and make this user- configurable.
Quoted 7 lines
Quoted 1 line
It takes ~34s to run this test.
LOL, yeah, no. That's 33+ seconds of wasted time. From a *KVM* perspective, this
is quite binary: either KVM intercepts HLT or it doesn't. Any timing bugs would
be purely CPU bugs.>> Please adjust this to have the default runtime <1 second. If you feel strongly about the need for a long-running test, feel free to add a command line option
to control the number of iterations.
Ack.