Add support for the Idle HLT intercept feature
v5 cover Add support for the Idle HLT intercept feature
The upcoming new Idle HLT Intercept feature allows for the HLT instruction execution by a vCPU to be intercepted by the hypervisor only if there are no pending V_INTR and V_NMI events for the vCPU. When the vCPU is expected to service the pending V_INTR and V_NMI events, the Idle HLT intercept won’t trigger. The feature allows the hypervisor to determine if the vCPU is actually idle and reduces wasteful VMEXITs.
The Idle HLT intercept feature is used for enlightened guests who wish to securely handle the events. When an enlightened guest does a HLT while an interrupt is pending, hypervisor will not have a way to figure out whether the guest needs to be re-entered or not. The Idle HLT intercept feature allows the HLT execution only if there are no pending V_INTR and V_NMI events.
Presence of the Idle HLT Intercept feature is indicated via CPUID function Fn8000_000A_EDX[30].
Document for the Idle HLT intercept feature is available at [1].
This series is based on kvm-x86/next (13e98294d7ce) + [2] + [3].
Testing Done:
- Tested the functionality for the Idle HLT intercept feature using selftest ipi_hlt_test.
- Tested on normal, SEV, SEV-ES, SEV-SNP guest for the Idle HLT intercept functionality.
- Tested the Idle HLT intercept functionality on nested guest.
v4 -> v5
- Incorporated Sean's review comments on nested Idle HLT intercept support.
- Make svm_idle_hlt_test independent of the Idle HLT to run on all hardware.
v3 -> v4
- Drop the patches to add vcpu_get_stat() into a new series [2].
- Added nested Idle HLT intercept support.
v2 -> v3
- Incorporated Andrew's suggestion to structure vcpu_stat_types in a way that each architecture can share the generic types and also provide its own.
v1 -> v2
- Done changes in svm_idle_hlt_test based on the review comments from Sean.
- Added an enum based approach to get binary stats in vcpu_get_stat() which doesn't use string to get stat data based on the comments from Sean.
- Added safe_halt() and cli() helpers based on the comments from Sean.
[1]: AMD64 Architecture Programmer's Manual Pub. 24593, April 2024,
Vol 2, 15.9 Instruction Intercepts (Table 15-7: IDLE_HLT).
https://bugzilla.kernel.org/attachment.cgi?id=306250[2]: https://lore.kernel.org/kvm/[email protected]/T/#u
[3]: https://lore.kernel.org/kvm/[email protected]/T/#u
---
V4: https://lore.kernel.org/kvm/[email protected]/
V3: https://lore.kernel.org/kvm/[email protected]/T/
V2: https://lore.kernel.org/kvm/[email protected]/
V1: https://lore.kernel.org/kvm/[email protected]/
Manali Shukla (3):
x86/cpufeatures: Add CPUID feature bit for Idle HLT intercept
KVM: SVM: Add Idle HLT intercept support
KVM: selftests: Add self IPI HLT test
arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/svm.h | 1 + arch/x86/include/uapi/asm/svm.h | 2 + arch/x86/kvm/svm/svm.c | 13 ++- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/include/x86/processor.h | 1 + tools/testing/selftests/kvm/ipi_hlt_test.c | 85 +++++++++++++++++++ 7 files changed, 101 insertions(+), 3 deletions(-) create mode 100644 tools/testing/selftests/kvm/ipi_hlt_test.c
base-commit: 13e98294d7cec978e31138d16824f50556a62d17
prerequisite-patch-id: cb345fc0d814a351df2b5788b76eee0eef9de549
prerequisite-patch-id: 71806f400cffe09f47d6231cb072cbdbd540de1b
prerequisite-patch-id: 9ea0412aab7ecd8555fcee3e9609dbfe8456d47b
prerequisite-patch-id: 3504df50cdd33958456f2e56139d76867273525c
prerequisite-patch-id: 674e56729a56cc487cb85be1a64ef561eb7bac8a
prerequisite-patch-id: 48e87354f9d6e6bd121ca32ab73cd0d7f1dce74f
prerequisite-patch-id: 74daffd7677992995f37e5a5cb784b8d4357e342
prerequisite-patch-id: 509018dc2fc1657debc641544e86f5a92d04bc1a
prerequisite-patch-id: 4a50c6a4dc3b3c8c8c640a86072faafb7bae4384
Signature · 1 line
2.34.1
PATCH v5 1/3 x86/cpufeatures: Add CPUID feature bit for Idle HLT intercept
From: Manali Shukla <[email protected]>
The Idle HLT Intercept feature allows for the HLT instruction execution by a vCPU to be intercepted by the hypervisor only if there are no pending events (V_INTR and V_NMI) for the vCPU. When the vCPU is expected to service the pending events (V_INTR and V_NMI), the Idle HLT intercept won’t trigger. The feature allows the hypervisor to determine if the vCPU is idle and reduces wasteful VMEXITs.
In addition to the aforementioned use case, the Idle HLT intercept feature is also used for enlightened guests who aim to securely manage events without the hypervisor’s awareness. If a HLT occurs while a virtual event is pending and the hypervisor is unaware of this pending event (as could be the case with enlightened guests), the absence of the Idle HLT intercept feature could result in a vCPU being suspended indefinitely.
Presence of Idle HLT intercept feature for guests is indicated via CPUID function 0x8000000A_EDX[30].
--- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -386,6 +386,7 @@#define X86_FEATURE_V_SPEC_CTRL (15*32+20)/* "v_spec_ctrl" Virtual SPEC_CTRL */ #define X86_FEATURE_VNMI (15*32+25)/* "vnmi" Virtual NMI */ #define X86_FEATURE_SVME_ADDR_CHK (15*32+28)/* SVME addr check */ +#define X86_FEATURE_IDLE_HLT (15*32+30)/* IDLE HLT intercept */ /* Intel-defined CPU features, CPUID level 0x00000007:0 (ECX), word 16 */ #define X86_FEATURE_AVX512VBMI (16*32+ 1)/* "avx512vbmi" AVX512 Vector Bit Manipulation instructions*/
base-commit: 13e98294d7cec978e31138d16824f50556a62d17
prerequisite-patch-id: cb345fc0d814a351df2b5788b76eee0eef9de549
prerequisite-patch-id: 71806f400cffe09f47d6231cb072cbdbd540de1b
prerequisite-patch-id: 9ea0412aab7ecd8555fcee3e9609dbfe8456d47b
prerequisite-patch-id: 3504df50cdd33958456f2e56139d76867273525c
prerequisite-patch-id: 674e56729a56cc487cb85be1a64ef561eb7bac8a
prerequisite-patch-id: 48e87354f9d6e6bd121ca32ab73cd0d7f1dce74f
prerequisite-patch-id: 74daffd7677992995f37e5a5cb784b8d4357e342
prerequisite-patch-id: 509018dc2fc1657debc641544e86f5a92d04bc1a
prerequisite-patch-id: 4a50c6a4dc3b3c8c8c640a86072faafb7bae4384
Signature · 1 line
2.34.1
PATCH v5 2/3 KVM: SVM: Add Idle HLT intercept support
In reply to Manali ShuklaFrom: Manali Shukla <[email protected]>
The hypervisor can intercept the HLT instruction by setting the HLT-Intercept Bit in VMCB, causing a VMEXIT. This can be wasteful if there are pending V_INTR and V_NMI events, as the hypervisor must then initiate a VMRUN to handle them.
If the HLT-Intercept Bit is cleared and the vCPU executes HLT while there are pending V_INTR and V_NMI events, the hypervisor won’t detect them, potentially causing indefinite suspension of the vCPU. This poses a problem for enlightened guests who wish to securely handle the events.
For Secure AVIC scenarios, if a guest does a HLT while an interrupt is pending (in IRR), the hypervisor does not have a way to figure out whether the guest needs to be re-entered, as it cannot read the guest backing page. The Idle HLT intercept feature allows the hypervisor to intercept HLT execution only if there are no pending V_INTR and V_NMI events.
There are two use cases for the Idle HLT intercept feature:
- Secure VMs that wish to handle pending events securely without exiting to the hypervisor on HLT (Secure AVIC).
- Optimization for all the VMs to avoid a wasteful VMEXIT during HLT when there are pending events.
On discovering the Idle HLT Intercept, the KVM hypervisor, Sets the Idle HLT Intercept bit (bit (6), offset 0x14h) in the VMCB. When the Idle HLT Intercept bit is set, HLT Intercept bit (bit (0), offset 0xFh) should be cleared.
Before entering the HLT state, the HLT instruction performs checks in following order:
- The HLT intercept check, if set, it unconditionally triggers SVM_EXIT_HLT (0x78).
- The Idle HLT intercept check, if set and there are no pending V_INTR or V_NMI events, triggers SVM_EXIT_IDLE_HLT (0xA6).
Details about the Idle HLT intercept feature can be found in AMD APM [1].
[1]: AMD64 Architecture Programmer's Manual Pub. 24593, April
2024, Vol 2, 15.9 Instruction Intercepts (Table 15-7: IDLE_HLT).
https://bugzilla.kernel.org/attachment.cgi?id=306250--- arch/x86/include/asm/svm.h | 1 + arch/x86/include/uapi/asm/svm.h | 2 ++ arch/x86/kvm/svm/svm.c | 13 ++++++++++--- 3 files changed, 13 insertions(+), 3 deletions(-)
arch/x86/include/asm/svm.h +1 -0
@@ -116,6 +116,7 @@ enum {INTERCEPT_INVPCID ,INTERCEPT_MCOMMIT ,INTERCEPT_TLBSYNC , + INTERCEPT_IDLE_HLT = 166, };
arch/x86/include/uapi/asm/svm.h +2 -0
@@ -95,6 +95,7 @@#define SVM_EXIT_CR14_WRITE_TRAP 0x09e#define SVM_EXIT_CR15_WRITE_TRAP 0x09f#define SVM_EXIT_INVPCID 0x0a2 +#define SVM_EXIT_IDLE_HLT 0x0a6#define SVM_EXIT_NPF 0x400#define SVM_EXIT_AVIC_INCOMPLETE_IPI 0x401#define SVM_EXIT_AVIC_UNACCELERATED_ACCESS 0x402
@@ -224,6 +225,7 @@ { SVM_EXIT_CR4_WRITE_TRAP, "write_cr4_trap" }, \ { SVM_EXIT_CR8_WRITE_TRAP, "write_cr8_trap" }, \ { SVM_EXIT_INVPCID, "invpcid" }, \ + { SVM_EXIT_IDLE_HLT, "idle-halt" }, \ { SVM_EXIT_NPF, "npf" }, \ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \
arch/x86/kvm/svm/svm.c +10 -3
@@ -1296,8 +1296,12 @@ static void init_vmcb(struct kvm_vcpu *vcpu)svm_set_intercept (svm,INTERCEPT_MWAIT ); } -if (!kvm_hlt_in_guest (vcpu->kvm)) -svm_set_intercept (svm,INTERCEPT_HLT ); +if (!kvm_hlt_in_guest (vcpu->kvm)) { +if (cpu_feature_enabled (X86_FEATURE_IDLE_HLT )) +svm_set_intercept (svm,INTERCEPT_IDLE_HLT ); +else +svm_set_intercept (svm,INTERCEPT_HLT ); + } control->iopm_base_pa = iopm_base; control->msrpm_base_pa =__sme_set (__pa (svm->msrpm));
@@ -3341,6 +3345,7 @@ static int (*const svm_exit_handlers[])(struct kvm_vcpu *vcpu) = { [SVM_EXIT_CR4_WRITE_TRAP] = cr_trap, [SVM_EXIT_CR8_WRITE_TRAP] = cr_trap, [SVM_EXIT_INVPCID] = invpcid_interception, + [SVM_EXIT_IDLE_HLT] = kvm_emulate_halt, [SVM_EXIT_NPF] = npf_interception, [SVM_EXIT_RSM] = rsm_interception, [SVM_EXIT_AVIC_INCOMPLETE_IPI] = avic_incomplete_ipi_interception,
@@ -3503,7 +3508,7 @@ int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)return interrupt_window_interception (vcpu); else if (exit_code == SVM_EXIT_INTR) return intr_interception(vcpu); - else if (exit_code == SVM_EXIT_HLT) + else if (exit_code == SVM_EXIT_HLT || exit_code == SVM_EXIT_IDLE_HLT) return kvm_emulate_halt(vcpu); else if (exit_code == SVM_EXIT_NPF) return npf_interception(vcpu);
@@ -5224,6 +5229,8 @@ static __init void svm_set_cpu_caps(void)if (vnmi)kvm_cpu_cap_set (X86_FEATURE_VNMI ); +kvm_cpu_cap_check_and_set (X86_FEATURE_IDLE_HLT ); +/* Nested VM can receive #VMEXIT instead of triggering #GP */ kvm_cpu_cap_set (X86_FEATURE_SVME_ADDR_CHK ); }
Signature · 1 line
2.34.1
Re: [PATCH v5 2/3] KVM: SVM: Add Idle HLT intercept support
Quoted 38 lines · Manali Shukla
From: Manali Shukla <[email protected]>
The hypervisor can intercept the HLT instruction by setting the HLT-Intercept Bit in VMCB, causing a VMEXIT. This can be wasteful if there are pending
V_INTRandV_NMIevents, as the hypervisor must then initiate a VMRUN to handle them.If the HLT-Intercept Bit is cleared and the vCPU executes HLT while there are pending
V_INTRandV_NMIevents, the hypervisor won’t detect them, potentially causing indefinite suspension of the vCPU. This poses a problem for enlightened guests who wish to securely handle the events.For Secure AVIC scenarios, if a guest does a HLT while an interrupt is pending (in IRR), the hypervisor does not have a way to figure out whether the guest needs to be re-entered, as it cannot read the guest backing page. The Idle HLT intercept feature allows the hypervisor to intercept HLT execution only if there are no pending
V_INTRandV_NMIevents.There are two use cases for the Idle HLT intercept feature:
- Secure VMs that wish to handle pending events securely without exiting to the hypervisor on HLT (Secure AVIC).
- Optimization for all the VMs to avoid a wasteful VMEXIT during HLT when there are pending events.On discovering the Idle HLT Intercept, the KVM hypervisor, Sets the Idle HLT Intercept bit (bit (6), offset 0x14h) in the VMCB. When the Idle HLT Intercept bit is set, HLT Intercept bit (bit (0), offset 0xFh) should be cleared.
Before entering the HLT state, the HLT instruction performs checks in following order:
- The HLT intercept check, if set, it unconditionally triggersSVM_EXIT_HLT(0x78).
- The Idle HLT intercept check, if set and there are no pendingV_INTRorV_NMIevents, triggersSVM_EXIT_IDLE_HLT(0xA6).
Quoted 7 lines
Details about the Idle HLT intercept feature can be found in AMD APM [1].
[1]: AMD64 Architecture Programmer's Manual Pub. 24593, April
2024, Vol 2, 15.9 Instruction Intercepts (Table 15-7: IDLE_HLT). https://bugzilla.kernel.org/attachment.cgi?id=306250Signed-off-by Manali Shukla <[email protected]>
LGTM
Quoted 85 lines
--- arch/x86/include/asm/svm.h | 1 + arch/x86/include/uapi/asm/svm.h | 2 ++ arch/x86/kvm/svm/svm.c | 13 ++++++++++--- 3 files changed, 13 insertions(+), 3 deletions(-)arch/x86/include/asm/svm.h +1
-0@@ -116,6 +116,7 @@ enum {INTERCEPT_INVPCID ,INTERCEPT_MCOMMIT ,INTERCEPT_TLBSYNC , + INTERCEPT_IDLE_HLT = 166, };arch/x86/include/uapi/asm/svm.h +2
-0@@ -95,6 +95,7 @@#define SVM_EXIT_CR14_WRITE_TRAP 0x09e#define SVM_EXIT_CR15_WRITE_TRAP 0x09f#define SVM_EXIT_INVPCID 0x0a2 +#define SVM_EXIT_IDLE_HLT 0x0a6#define SVM_EXIT_NPF 0x400#define SVM_EXIT_AVIC_INCOMPLETE_IPI 0x401#define SVM_EXIT_AVIC_UNACCELERATED_ACCESS 0x402@@ -224,6 +225,7 @@ { SVM_EXIT_CR4_WRITE_TRAP, "write_cr4_trap" }, \ { SVM_EXIT_CR8_WRITE_TRAP, "write_cr8_trap" }, \ { SVM_EXIT_INVPCID, "invpcid" }, \ + { SVM_EXIT_IDLE_HLT, "idle-halt" }, \ { SVM_EXIT_NPF, "npf" }, \ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \arch/x86/kvm/svm/svm.c +10
-3@@ -1296,8 +1296,12 @@ static void init_vmcb(struct kvm_vcpu *vcpu)svm_set_intercept (svm,INTERCEPT_MWAIT ); } -if (!kvm_hlt_in_guest (vcpu->kvm)) -svm_set_intercept (svm,INTERCEPT_HLT ); +if (!kvm_hlt_in_guest (vcpu->kvm)) { +if (cpu_feature_enabled (X86_FEATURE_IDLE_HLT )) +svm_set_intercept (svm,INTERCEPT_IDLE_HLT ); +else +svm_set_intercept (svm,INTERCEPT_HLT ); + } control->iopm_base_pa = iopm_base; control->msrpm_base_pa =__sme_set (__pa (svm->msrpm));@@ -3341,6 +3345,7 @@ static int (*const svm_exit_handlers[])(struct kvm_vcpu *vcpu) = { [SVM_EXIT_CR4_WRITE_TRAP] = cr_trap, [SVM_EXIT_CR8_WRITE_TRAP] = cr_trap, [SVM_EXIT_INVPCID] = invpcid_interception, + [SVM_EXIT_IDLE_HLT] = kvm_emulate_halt, [SVM_EXIT_NPF] = npf_interception, [SVM_EXIT_RSM] = rsm_interception, [SVM_EXIT_AVIC_INCOMPLETE_IPI] = avic_incomplete_ipi_interception,@@ -3503,7 +3508,7 @@ int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)return interrupt_window_interception (vcpu); else if (exit_code == SVM_EXIT_INTR) return intr_interception(vcpu); - else if (exit_code == SVM_EXIT_HLT) + else if (exit_code == SVM_EXIT_HLT || exit_code == SVM_EXIT_IDLE_HLT) return kvm_emulate_halt(vcpu); else if (exit_code == SVM_EXIT_NPF) return npf_interception(vcpu);@@ -5224,6 +5229,8 @@ static __init void svm_set_cpu_caps(void)if (vnmi)kvm_cpu_cap_set (X86_FEATURE_VNMI ); +kvm_cpu_cap_check_and_set (X86_FEATURE_IDLE_HLT ); +/* Nested VM can receive #VMEXIT instead of triggering #GP */ kvm_cpu_cap_set (X86_FEATURE_SVME_ADDR_CHK ); }Signature · 1 line
2.34.1
PATCH v5 3/3 KVM: selftests: Add self IPI HLT test
In reply to Manali ShuklaFrom: Manali Shukla <[email protected]>
The IPI HLT test simulates a scenario where a pending event is present while the HLT instruction is executed.
Evaluates the idle HLT intercept feature of the AMD architecture, if available. If the feature is not present, this selftest can be extended in the future to include cross-vCPU IPI testing.
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/include/x86/processor.h | 1 + tools/testing/selftests/kvm/ipi_hlt_test.c | 83 +++++++++++++++++++ 3 files changed, 85 insertions(+) create mode 100644 tools/testing/selftests/kvm/ipi_hlt_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -135,6 +135,7 @@ TEST_GEN_PROGS_x86 += steal_time TEST_GEN_PROGS_x86 += kvm_binary_stats_test TEST_GEN_PROGS_x86 += system_counter_offset_test TEST_GEN_PROGS_x86 += pre_fault_memory_test +TEST_GEN_PROGS_x86 += ipi_hlt_test # Compiled outputs used by test targets TEST_GEN_PROGS_EXTENDED_x86 += x86/nx_huge_pages_test
tools/testing/selftests/kvm/include/x86/processor.h +1 -0
@@ -197,6 +197,7 @@ struct kvm_x86_cpu_feature {#define X86_FEATURE_PAUSEFILTER KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 10)#define X86_FEATURE_PFTHRESHOLD KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 12)#define X86_FEATURE_VGIF KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 16) +#define X86_FEATURE_IDLE_HLT KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 30)#define X86_FEATURE_SEV KVM_X86_CPU_FEATURE(0x8000001F, 0, EAX, 1)#define X86_FEATURE_SEV_ES KVM_X86_CPU_FEATURE(0x8000001F, 0, EAX, 3)
tools/testing/selftests/kvm/ipi_hlt_test.c new file +83 -0
83 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v5 3/3] KVM: selftests: Add self IPI HLT test
Quoted 6 lines · Manali Shukla
From: Manali Shukla <[email protected]>
The IPI HLT test simulates a scenario where a pending event is present while the HLT instruction is executed.
Evaluates the idle HLT intercept feature of the AMD architecture, if
Nit: Start as "The test evaluates idle HLT intercept feature ..." ?
Quoted 1 line
available. If the feature is not present, this selftest can be extended
If idle halt intercept feature is not present the test exercises halt-exits/guest-entry for pending interrupts. So, maybe the test in its current form is still useful if idle halt intercept feature is not present. Is that understanding wrong? Cross-vCPU IPI or performance testing of halt exits are future extensions to this test.
Quoted 54 lines
in the future to include cross-vCPU IPI testing.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Manali Shukla <[email protected]>--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/include/x86/processor.h | 1 + tools/testing/selftests/kvm/ipi_hlt_test.c | 83 +++++++++++++++++++ 3 files changed, 85 insertions(+) create mode 100644 tools/testing/selftests/kvm/ipi_hlt_test.ctools/testing/selftests/kvm/Makefile.kvm +1
-0@@ -135,6 +135,7 @@ TEST_GEN_PROGS_x86 += steal_time TEST_GEN_PROGS_x86 += kvm_binary_stats_test TEST_GEN_PROGS_x86 += system_counter_offset_test TEST_GEN_PROGS_x86 += pre_fault_memory_test +TEST_GEN_PROGS_x86 += ipi_hlt_test # Compiled outputs used by test targets TEST_GEN_PROGS_EXTENDED_x86 += x86/nx_huge_pages_testtools/testing/selftests/kvm/include/x86/processor.h +1
-0@@ -197,6 +197,7 @@ struct kvm_x86_cpu_feature {#define X86_FEATURE_PAUSEFILTER KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 10)#define X86_FEATURE_PFTHRESHOLD KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 12)#define X86_FEATURE_VGIF KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 16) +#define X86_FEATURE_IDLE_HLT KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 30)#define X86_FEATURE_SEV KVM_X86_CPU_FEATURE(0x8000001F, 0, EAX, 1)#define X86_FEATURE_SEV_ES KVM_X86_CPU_FEATURE(0x8000001F, 0, EAX, 3)tools/testing/selftests/kvm/ipi_hlt_test.c new file +13
-0@@ -0,0 +1,83 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (C) 2024 Advanced Micro Devices, Inc. +* +*/ +#include <kvm_util.h> +#include <processor.h> +#include <test_util.h> +#include "apic.h" + +#define INTR_VECTOR 0x30 +#define NUM_ITERATIONS 1000 +
tools/testing/selftests/kvm/ipi_hlt_test.c line 23
Quoted 10 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +10
-0+static bool irq_received; + +/* +* The guest code instruments the scenario where there is a V_INTR pending +* event available while hlt instruction is executed. +*/ + +static void guest_code (void ) +{ +uint32_t icr_val;
uint64_t ?
Quoted 7 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +7
-0+int i; + +x2apic_enable (); + + icr_val = (APIC_DEST_SELF |APIC_INT_ASSERT |INTR_VECTOR ); + +for (i =0 ; i <NUM_ITERATIONS ; i++) {
tools/testing/selftests/kvm/ipi_hlt_test.c line 40
Quoted 10 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +10
-0+cli (); +x2apic_write_reg (APIC_ICR , icr_val); +safe_halt (); +GUEST_ASSERT (READ_ONCE (irq_received)); +WRITE_ONCE (irq_received, false); + } +GUEST_DONE (); +} + +static void guest_vintr_handler (struct ex_regs *regs)
s/guest_vintr_handler/guest_intr_handler/ ?
tools/testing/selftests/kvm/ipi_hlt_test.c line 51
Quoted 11 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +11
-0+{ +WRITE_ONCE (irq_received, true); +x2apic_write_reg (APIC_EOI ,0x00 ); +} + +int main (int argc,char *argv[]) +{ +struct kvm_vm *vm; +struct kvm_vcpu *vcpu; +struct ucall uc; +uint64_t halt_exits;
Extra space
tools/testing/selftests/kvm/ipi_hlt_test.c line 58
Quoted 7 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +7
-0+ +TEST_REQUIRE (kvm_has_cap (KVM_CAP_BINARY_STATS_FD )); + + vm =vm_create_with_one_vcpu (&vcpu, guest_code); + +vm_install_exception_handler (vm,INTR_VECTOR , guest_vintr_handler); +virt_pg_map (vm,APIC_DEFAULT_GPA ,APIC_DEFAULT_GPA );
virt_pg_map() is not required for x2apic. So, remove?
tools/testing/selftests/kvm/ipi_hlt_test.c line 71
Quoted 13 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +13
-0+ +vcpu_run (vcpu); +TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_IO ); + + halt_exits =vcpu_get_stat (vcpu, halt_exits); + +switch (get_ucall (vcpu, &uc)) { +case UCALL_ABORT : +REPORT_GUEST_ASSERT (uc); +/* NOT REACHED */ +case UCALL_DONE : +break ; +
Nit: empty line not required?
- Neeraj
Quoted 12 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +12
-0+default : +TEST_FAIL ("Unknown ucall 0x%lx." , uc.cmd); + } + +if (kvm_cpu_has (X86_FEATURE_IDLE_HLT )) +TEST_ASSERT_EQ (halt_exits,0 ); +else +TEST_ASSERT_EQ (halt_exits,NUM_ITERATIONS ); + +kvm_vm_free (vm); +return 0 ; +}
Re: [PATCH v5 3/3] KVM: selftests: Add self IPI HLT test
Hi Neeraj,
Thanks for reviewing my patches.
Quoted 11 lines · Neeraj Upadhyay
Quoted 6 lines · Manali Shukla
From: Manali Shukla <[email protected]>
The IPI HLT test simulates a scenario where a pending event is present while the HLT instruction is executed.
Evaluates the idle HLT intercept feature of the AMD architecture, if
Nit: Start as "The test evaluates idle HLT intercept feature ..." ?
Sure.
Quoted 8 lines
Quoted 2 lines
available. If the feature is not present, this selftest can be extended
If idle halt intercept feature is not present the test exercises
halt-exits/guest-entry for pending interrupts. So, maybe the test in
its current form is still useful if idle halt intercept feature is
not present. Is that understanding wrong? Cross-vCPU IPI or performance> testing of halt exits are future extensions to this test.
Test in its current form is useful when idle halt intercept feature is not present. I will reword the commit message.
Quoted 58 lines
Quoted 58 lines
in the future to include cross-vCPU IPI testing.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Manali Shukla <[email protected]>--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/include/x86/processor.h | 1 + tools/testing/selftests/kvm/ipi_hlt_test.c | 83 +++++++++++++++++++ 3 files changed, 85 insertions(+) create mode 100644 tools/testing/selftests/kvm/ipi_hlt_test.ctools/testing/selftests/kvm/Makefile.kvm +1
-0@@ -135,6 +135,7 @@ TEST_GEN_PROGS_x86 += steal_time TEST_GEN_PROGS_x86 += kvm_binary_stats_test TEST_GEN_PROGS_x86 += system_counter_offset_test TEST_GEN_PROGS_x86 += pre_fault_memory_test +TEST_GEN_PROGS_x86 += ipi_hlt_test # Compiled outputs used by test targets TEST_GEN_PROGS_EXTENDED_x86 += x86/nx_huge_pages_testtools/testing/selftests/kvm/include/x86/processor.h +1
-0@@ -197,6 +197,7 @@ struct kvm_x86_cpu_feature {#define X86_FEATURE_PAUSEFILTER KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 10)#define X86_FEATURE_PFTHRESHOLD KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 12)#define X86_FEATURE_VGIF KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 16) +#define X86_FEATURE_IDLE_HLT KVM_X86_CPU_FEATURE(0x8000000A, 0, EDX, 30)#define X86_FEATURE_SEV KVM_X86_CPU_FEATURE(0x8000001F, 0, EAX, 1)#define X86_FEATURE_SEV_ES KVM_X86_CPU_FEATURE(0x8000001F, 0, EAX, 3)tools/testing/selftests/kvm/ipi_hlt_test.c new file +17
-0@@ -0,0 +1,83 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (C) 2024 Advanced Micro Devices, Inc. +* +*/ +#include <kvm_util.h> +#include <processor.h> +#include <test_util.h> +#include "apic.h" + +#define INTR_VECTOR 0x30 +#define NUM_ITERATIONS 1000 + +static bool irq_received; + +/* +* The guest code instruments the scenario where there is a V_INTR pending
Quoted 8 lines
tools/testing/selftests/kvm/ipi_hlt_test.c line 23
Quoted 6 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +6
-0+* event available while hlt instruction is executed. +*/ + +static void guest_code (void ) +{ +uint32_t icr_val;
uint64_t?
Sure.
Quoted 12 lines
Quoted 12 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +11
-0+int i; + +x2apic_enable (); + + icr_val = (APIC_DEST_SELF |APIC_INT_ASSERT |INTR_VECTOR ); + +for (i =0 ; i <NUM_ITERATIONS ; i++) { +cli (); +x2apic_write_reg (APIC_ICR , icr_val); +safe_halt (); +GUEST_ASSERT (READ_ONCE (irq_received));
Quoted 9 lines
tools/testing/selftests/kvm/ipi_hlt_test.c line 40
Quoted 6 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +6
-0+WRITE_ONCE (irq_received, false); + } +GUEST_DONE (); +} + +static void guest_vintr_handler (struct ex_regs *regs)s/guest_vintr_handler/guest_intr_handler/ ?
Sure.
Quoted 14 lines
tools/testing/selftests/kvm/ipi_hlt_test.c line 51
Quoted 12 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +11
-0+{ +WRITE_ONCE (irq_received, true); +x2apic_write_reg (APIC_EOI ,0x00 ); +} + +int main (int argc,char *argv[]) +{ +struct kvm_vm *vm; +struct kvm_vcpu *vcpu; +struct ucall uc; +uint64_t halt_exits;Extra space
Sure.
Quoted 11 lines
tools/testing/selftests/kvm/ipi_hlt_test.c line 58
Quoted 8 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +7
-0+ +TEST_REQUIRE (kvm_has_cap (KVM_CAP_BINARY_STATS_FD )); + + vm =vm_create_with_one_vcpu (&vcpu, guest_code); + +vm_install_exception_handler (vm,INTR_VECTOR , guest_vintr_handler); +virt_pg_map (vm,APIC_DEFAULT_GPA ,APIC_DEFAULT_GPA );
virt_pg_map()is not required for x2apic. So, remove?
I missed to remove it. I will remove it.
Quoted 7 lines
Quoted 7 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +7
-0+ +vcpu_run (vcpu); +TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_IO ); + + halt_exits =vcpu_get_stat (vcpu, halt_exits); + +switch (get_ucall (vcpu, &uc)) {
Quoted 8 lines
tools/testing/selftests/kvm/ipi_hlt_test.c line 71
Quoted 6 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +6
-0+case UCALL_ABORT : +REPORT_GUEST_ASSERT (uc); +/* NOT REACHED */ +case UCALL_DONE : +break ; +Nit: empty line not required?
Sure.
Quoted 4 lines
- Neeraj
Quoted 13 lines
tools/testing/selftests/kvm/ipi_hlt_test.c line 83
Quoted 12 lines
tools/testing/selftests/kvm/ipi_hlt_test.c new file +12
-0+default : +TEST_FAIL ("Unknown ucall 0x%lx." , uc.cmd); + } + +if (kvm_cpu_has (X86_FEATURE_IDLE_HLT )) +TEST_ASSERT_EQ (halt_exits,0 ); +else +TEST_ASSERT_EQ (halt_exits,NUM_ITERATIONS ); + +kvm_vm_free (vm); +return 0 ; +}
-Manali
Quoted 79 lines · Manali Shukla
The upcoming new Idle HLT Intercept feature allows for the HLT instruction execution by a vCPU to be intercepted by the hypervisor only if there are no pending
V_INTRandV_NMIevents for the vCPU. When the vCPU is expected to service the pendingV_INTRandV_NMIevents, the Idle HLT intercept won’t trigger. The feature allows the hypervisor to determine if the vCPU is actually idle and reduces wasteful VMEXITs.The Idle HLT intercept feature is used for enlightened guests who wish to securely handle the events. When an enlightened guest does a HLT while an interrupt is pending, hypervisor will not have a way to figure out whether the guest needs to be re-entered or not. The Idle HLT intercept feature allows the HLT execution only if there are no pending
V_INTRandV_NMIevents.Presence of the Idle HLT Intercept feature is indicated via CPUID function Fn8000_000A_EDX[30].
Document for the Idle HLT intercept feature is available at [1].
This series is based on kvm-x86/next (
13e98294d7ce) + [2] + [3].Testing Done:
- Tested the functionality for the Idle HLT intercept feature using selftestipi_hlt_test.
- Tested on normal, SEV, SEV-ES, SEV-SNP guest for the Idle HLT intercept functionality.
- Tested the Idle HLT intercept functionality on nested guest.v4 -> v5
- Incorporated Sean's review comments on nested Idle HLT intercept support.
- Makesvm_idle_hlt_testindependent of the Idle HLT to run on all hardware.v3 -> v4
- Drop the patches to addvcpu_get_stat()into a new series [2].
- Added nested Idle HLT intercept support.v2 -> v3
- Incorporated Andrew's suggestion to structurevcpu_stat_typesin a way that each architecture can share the generic types and also provide its own.v1 -> v2
- Done changes insvm_idle_hlt_testbased on the review comments from Sean.
- Added an enum based approach to get binary stats invcpu_get_stat()which doesn't use string to get stat data based on the comments from Sean.
- Addedsafe_halt()andcli()helpers based on the comments from Sean.[1]: AMD64 Architecture Programmer's Manual Pub. 24593, April 2024,
Vol 2, 15.9 Instruction Intercepts (Table 15-7: IDLE_HLT). https://bugzilla.kernel.org/attachment.cgi?id=306250[2]: https://lore.kernel.org/kvm/[email protected]/T/#u
[3]: https://lore.kernel.org/kvm/[email protected]/T/#u
---
V4: https://lore.kernel.org/kvm/[email protected]/
V3: https://lore.kernel.org/kvm/[email protected]/T/
V2: https://lore.kernel.org/kvm/[email protected]/
V1: https://lore.kernel.org/kvm/[email protected]/Manali Shukla (3):
x86/cpufeatures: Add CPUID feature bit for Idle HLT intercept
KVM: SVM: Add Idle HLT intercept support
KVM: selftests: Add self IPI HLT testarch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/svm.h | 1 + arch/x86/include/uapi/asm/svm.h | 2 + arch/x86/kvm/svm/svm.c | 13 ++- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/include/x86/processor.h | 1 + tools/testing/selftests/kvm/ipi_hlt_test.c | 85 +++++++++++++++++++ 7 files changed, 101 insertions(+), 3 deletions(-) create mode 100644 tools/testing/selftests/kvm/ipi_hlt_test.c
Quoted 10 lines
base-commit:
13e98294d7cec978e31138d16824f50556a62d17
prerequisite-patch-id:cb345fc0d814a351df2b5788b76eee0eef9de549
prerequisite-patch-id:71806f400cffe09f47d6231cb072cbdbd540de1b
prerequisite-patch-id:9ea0412aab7ecd8555fcee3e9609dbfe8456d47b
prerequisite-patch-id:3504df50cdd33958456f2e56139d76867273525c
prerequisite-patch-id:674e56729a56cc487cb85be1a64ef561eb7bac8a
prerequisite-patch-id:48e87354f9d6e6bd121ca32ab73cd0d7f1dce74f
prerequisite-patch-id:74daffd7677992995f37e5a5cb784b8d4357e342
prerequisite-patch-id:509018dc2fc1657debc641544e86f5a92d04bc1a
prerequisite-patch-id:4a50c6a4dc3b3c8c8c640a86072faafb7bae4384
A gentle reminder for the review.
-Manali
Quoted 92 lines · Manali Shukla
Quoted 79 lines · Manali Shukla
The upcoming new Idle HLT Intercept feature allows for the HLT instruction execution by a vCPU to be intercepted by the hypervisor only if there are no pending
V_INTRandV_NMIevents for the vCPU. When the vCPU is expected to service the pendingV_INTRandV_NMIevents, the Idle HLT intercept won’t trigger. The feature allows the hypervisor to determine if the vCPU is actually idle and reduces wasteful VMEXITs.The Idle HLT intercept feature is used for enlightened guests who wish to securely handle the events. When an enlightened guest does a HLT while an interrupt is pending, hypervisor will not have a way to figure out whether the guest needs to be re-entered or not. The Idle HLT intercept feature allows the HLT execution only if there are no pending
V_INTRandV_NMIevents.Presence of the Idle HLT Intercept feature is indicated via CPUID function Fn8000_000A_EDX[30].
Document for the Idle HLT intercept feature is available at [1].
This series is based on kvm-x86/next (
13e98294d7ce) + [2] + [3].Testing Done:
- Tested the functionality for the Idle HLT intercept feature using selftestipi_hlt_test.
- Tested on normal, SEV, SEV-ES, SEV-SNP guest for the Idle HLT intercept functionality.
- Tested the Idle HLT intercept functionality on nested guest.v4 -> v5
- Incorporated Sean's review comments on nested Idle HLT intercept support.
- Makesvm_idle_hlt_testindependent of the Idle HLT to run on all hardware.v3 -> v4
- Drop the patches to addvcpu_get_stat()into a new series [2].
- Added nested Idle HLT intercept support.v2 -> v3
- Incorporated Andrew's suggestion to structurevcpu_stat_typesin a way that each architecture can share the generic types and also provide its own.v1 -> v2
- Done changes insvm_idle_hlt_testbased on the review comments from Sean.
- Added an enum based approach to get binary stats invcpu_get_stat()which doesn't use string to get stat data based on the comments from Sean.
- Addedsafe_halt()andcli()helpers based on the comments from Sean.[1]: AMD64 Architecture Programmer's Manual Pub. 24593, April 2024,
Vol 2, 15.9 Instruction Intercepts (Table 15-7: IDLE_HLT). https://bugzilla.kernel.org/attachment.cgi?id=306250[2]: https://lore.kernel.org/kvm/[email protected]/T/#u
[3]: https://lore.kernel.org/kvm/[email protected]/T/#u
---
V4: https://lore.kernel.org/kvm/[email protected]/
V3: https://lore.kernel.org/kvm/[email protected]/T/
V2: https://lore.kernel.org/kvm/[email protected]/
V1: https://lore.kernel.org/kvm/[email protected]/Manali Shukla (3):
x86/cpufeatures: Add CPUID feature bit for Idle HLT intercept
KVM: SVM: Add Idle HLT intercept support
KVM: selftests: Add self IPI HLT testarch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/svm.h | 1 + arch/x86/include/uapi/asm/svm.h | 2 + arch/x86/kvm/svm/svm.c | 13 ++- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/include/x86/processor.h | 1 + tools/testing/selftests/kvm/ipi_hlt_test.c | 85 +++++++++++++++++++ 7 files changed, 101 insertions(+), 3 deletions(-) create mode 100644 tools/testing/selftests/kvm/ipi_hlt_test.cQuoted 10 lines
base-commit:
13e98294d7cec978e31138d16824f50556a62d17
prerequisite-patch-id:cb345fc0d814a351df2b5788b76eee0eef9de549
prerequisite-patch-id:71806f400cffe09f47d6231cb072cbdbd540de1b
prerequisite-patch-id:9ea0412aab7ecd8555fcee3e9609dbfe8456d47b
prerequisite-patch-id:3504df50cdd33958456f2e56139d76867273525c
prerequisite-patch-id:674e56729a56cc487cb85be1a64ef561eb7bac8a
prerequisite-patch-id:48e87354f9d6e6bd121ca32ab73cd0d7f1dce74f
prerequisite-patch-id:74daffd7677992995f37e5a5cb784b8d4357e342
prerequisite-patch-id:509018dc2fc1657debc641544e86f5a92d04bc1a
prerequisite-patch-id:4a50c6a4dc3b3c8c8c640a86072faafb7bae4384
Quoted 3 lines
A gentle reminder for the review.
-Manali
A gentle reminder for the review.
-Manali