x86/apic: PV IPI robustness and subtest selection
Subsystems (2)
cover x86/apic: PV IPI robustness and subtest selection
This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.
1. Robustness: Skip the PV IPI test if KVM PV IPI features are not supported or if not running under KVM, preventing false failures and potential #UDs on bare metal or non-KVM hypervisors.
2. Code Reuse: Hoist the command-line test selection helper from x86/vmexit.c into the common library (lib/argv.c) as argv_test_wanted().
3. Subtest Selection: Refactor x86/apic.c to use this library helper, allowing users to run a subset of APIC tests (e.g., via QEMU's -append option).
4. Refactoring: Extract the KVM PV IPI hypercall inline assembly into a shared pv_ipi() helper.
5. New Test: Add test_pv_ipi_sparse() to verify PV IPI delivery to a vCPU with APIC ID 255. APIC ID 255 is a key boundary condition (the xAPIC/x2APIC transition point, and the MSB of the 64-bit cluster bitmap used by KVM's PV IPI hypercall). A sparse topology is used to verify KVM's handling of sparse APIC ID maps.
Jim Mattson (5):
x86/apic: Skip PV IPI test if feature is not supported
lib: Hoist test selection logic to library for code reuse x86/apic: Support subtest selection via command line x86/apic: Introduce pv_ipi()
x86/apic: Add a test for a PV IPI boundary condition
lib/argv.c | 14 +++++ lib/argv.h | 3 + lib/x86/processor.h | 1 + x86/apic.c | 133 ++++++++++++++++++++++++++++++++++---------- x86/unittests.cfg | 12 +++- x86/vmexit.c | 15 +---- 6 files changed, 135 insertions(+), 43 deletions(-)
base-commit: afcc98a30a84aaefc1f0d6783ffffec16c736717
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
PATCH 1/5 x86/apic: Skip PV IPI test if feature is not supported
Check for KVM_FEATURE_PV_SEND_IPI before running the PV IPI test to avoid false failures or potential #UD exceptions in environments where KVM PV features are disabled, not supported by the host, or when running on non-KVM hypervisors or bare metal.
Define the KVM_FEATURE_PV_SEND_IPI macro (bit 11 of EAX in KVM CPUID leaf 0x40000001) in processor.h to facilitate this check.
Additionally, update the [xapic] and [x2apic] configurations in unittests.cfg to advertise PV IPIs (+kvm-pv-ipi) in the guest CPUID, ensuring the PV IPI tests are not skipped in standard test runs.
--- lib/x86/processor.h | 1 + x86/apic.c | 5 +++++ x86/unittests.cfg | 4 ++-- 3 files changed, 8 insertions(+), 2 deletions(-)
lib/x86/processor.h +1 -0
@@ -354,6 +354,7 @@ struct x86_cpu_feature {*/ #define KVM_FEATURE_CLOCKSOURCE2 X86_CPU_FEATURE(0x40000001, 0, EAX, 3)#define KVM_FEATURE_ASYNC_PF X86_CPU_FEATURE(0x40000001, 0, EAX, 4) +#define KVM_FEATURE_PV_SEND_IPI X86_CPU_FEATURE(0x40000001, 0, EAX, 11)#define KVM_FEATURE_ASYNC_PF_INT X86_CPU_FEATURE(0x40000001, 0, EAX, 14)/*
x86/apic.c +5 -0
@@ -642,6 +642,11 @@ static void test_pv_ipi(void)int ret;unsigned long a0 =0xFFFFFFFF , a1 =0 , a2 =0xFFFFFFFF , a3 =0x0 ; +if (!this_cpu_has_kvm () || !this_cpu_has (KVM_FEATURE_PV_SEND_IPI )) { +report_skip ("PV IPIs are not supported" ); +return ; + } +if (!test_device_enabled ())return ;
x86/unittests.cfg +2 -2
@@ -10,7 +10,7 @@ [x2apic] file = apic.flat smp = 2 -qemu_params = -cpu qemu64,+x2apic,+tsc-deadline +qemu_params = -cpu qemu64,+x2apic,+tsc-deadline,+kvm-pv-ipi arch = x86_64 timeout = 30 groups = apic
@@ -19,7 +19,7 @@ groups = apic [xapic] file = apic.flat smp = 2 -qemu_params = -cpu qemu64,-x2apic,+tsc-deadline +qemu_params = -cpu qemu64,-x2apic,+tsc-deadline,+kvm-pv-ipi arch = x86_64 timeout = 60 groups = apic
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
PATCH 2/5 lib: Hoist test selection logic to library for code reuse
In reply to Jim MattsonHoist the 'test_wanted()' helper from x86/vmexit.c into the common library as 'argv_test_wanted()' to facilitate code reuse.
This eliminates duplicate code and provides a generic, architecture-independent helper that any test suite can leverage to support running a subset of tests specified via command-line arguments.
Refactor x86/vmexit.c to use the new library helper.
--- lib/argv.c | 14 ++++++++++++++ lib/argv.h | 3 +++ x86/vmexit.c | 15 ++------------- 3 files changed, 19 insertions(+), 13 deletions(-)
lib/argv.c +14 -0
@@ -146,3 +146,17 @@ void setup_env(char *env, int size) *env++ ='\0' ; } } + +bool argv_test_wanted (const char *name,char *wanted[],int nwanted) +{ +int i; + +if (!nwanted) +return true; + +for (i =0 ; i < nwanted; ++i) +if (strcmp (wanted[i], name) ==0 ) +return true; + +return false; +}
lib/argv.h +3 -0
@@ -8,10 +8,13 @@#ifndef _ARGV_H_#define _ARGV_H_ +#include <stdbool.h> +extern void __setup_args (void );extern void setup_args (const char *args);extern void setup_args_progname (const char *args);extern void setup_env (char *env,int size);extern void add_setup_arg (const char *arg); +extern bool argv_test_wanted (const char *name,char *wanted[],int nwanted);#endif
x86/vmexit.c +2 -13
@@ -1,4 +1,5 @@#include "libcflat.h" +#include "argv.h" #include "acpi.h" #include "smp.h" #include "vmalloc.h"
@@ -594,19 +595,7 @@ static void enable_nx(void *junk)wrmsr (MSR_EFER ,rdmsr (MSR_EFER ) |EFER_NX_MASK ); } -static bool test_wanted (struct test *test,char *wanted[],int nwanted) -{ -int i; - -if (!nwanted) -return true; - -for (i =0 ; i < nwanted; ++i) -if (strcmp (wanted[i], test->name) ==0 ) -return true; -return false; -}int main (int ac,char **av) {
@@ -636,7 +625,7 @@ int main(int ac, char **av) }for (i =0 ; i <ARRAY_SIZE (tests); ++i) -if (test_wanted (&tests[i], av +1 , ac -1 )) +if (argv_test_wanted (tests[i].name, av +1 , ac -1 ))while (do_test (&tests[i])) {}return 0 ;
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
PATCH 3/5 x86/apic: Support subtest selection via command line
In reply to Jim MattsonRefactor the APIC test suite to support running a subset of tests specified via command line arguments (e.g., passed via QEMU's -append option). This matches the test selection pattern already established in x86/vmexit.c.
Define 'struct apic_test' to associate a name with each test function and leverage the new 'argv_test_wanted()' library helper to filter the execution loop.
--- x86/apic.c | 57 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 30 insertions(+), 27 deletions(-)
x86/apic.c +30 -27
@@ -1,4 +1,5 @@#include "libcflat.h" +#include "argv.h" #include "apic.h" #include "vm.h" #include "smp.h"
@@ -905,43 +906,42 @@ static void test_aliased_xapic_physical_ipi(void)report (!f,"IPI to aliased xAPIC physical IDs" ); } -typedef void (*apic_test_fn )(void ); +struct apic_test { +const char *name; +void (*fn)(void ); +}; -int main (void ) + + +int main (int argc,char **argv) {bool is_x2apic =is_x2apic_enabled ();u32 spiv =apic_read (APIC_SPIV );int i; -const apic_test_fn tests[] = { - test_lapic_existence, - - test_apic_disable, - test_enable_x2apic, - - test_self_ipi_xapic, - test_self_ipi_x2apic, - test_physical_broadcast, - test_logical_ipi_xapic, - - test_pv_ipi, - - test_sti_nmi, - test_multiple_nmi, - test_pending_nmi, - - test_apic_timer_one_shot, - test_apic_change_mode, - test_tsc_deadline_timer, - +const struct apic_test tests[] = { + {"lapic_existence" , test_lapic_existence }, + {"apic_disable" , test_apic_disable }, + {"enable_x2apic" , test_enable_x2apic }, + {"self_ipi_xapic" , test_self_ipi_xapic }, + {"self_ipi_x2apic" , test_self_ipi_x2apic }, + {"physical_broadcast" , test_physical_broadcast }, + {"logical_ipi_xapic" , test_logical_ipi_xapic }, + {"pv_ipi" , test_pv_ipi }, + {"sti_nmi" , test_sti_nmi }, + {"multiple_nmi" , test_multiple_nmi }, + {"pending_nmi" , test_pending_nmi }, + {"apic_timer_one_shot" , test_apic_timer_one_shot }, + {"apic_change_mode" , test_apic_change_mode }, + {"tsc_deadline_timer" , test_tsc_deadline_timer },/* * KVM may disable APICv if the APIC ID and/or APIC_BASE is * modified, keep these tests at the end so that the test as a * whole provides coverage for APICv (when it's enabled). */ - test_apic_id, - test_apicbase, - test_aliased_xapic_physical_ipi, + {"apic_id" , test_apic_id }, + {"apicbase" , test_apicbase }, + {"aliased_xapic_physical_ipi" , test_aliased_xapic_physical_ipi }, }; assert_msg(is_apic_hw_enabled() && is_apic_sw_enabled(),
@@ -953,7 +953,10 @@ int main(void)sti ();for (i =0 ; i <ARRAY_SIZE (tests); i++) { - tests[i](); +if (!argv_test_wanted (tests[i].name, argv +1 , argc -1 )) +continue ; + + tests[i].fn ();if (is_x2apic)enable_x2apic ();
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
PATCH 4/5 x86/apic: Introduce pv_ipi()
In reply to Jim MattsonExtract the inline assembly hypercall from test_pv_ipi() into a new helper function, pv_ipi(), to facilitate code sharing.
--- x86/apic.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-)
x86/apic.c +13 -4
@@ -638,6 +638,18 @@ static void test_apic_change_mode(void)#define KVM_HC_SEND_IPI 10 +static int pv_ipi (unsigned long a0,unsigned long a1,unsigned long a2,unsigned long a3) +{ +int ret; + +if (is_intel ()) + asmvolatile ("vmcall" :"=a" (ret) :"a" (KVM_HC_SEND_IPI ),"b" (a0),"c" (a1),"d" (a2),"S" (a3)); +else + asmvolatile ("vmmcall" :"=a" (ret) :"a" (KVM_HC_SEND_IPI ),"b" (a0),"c" (a1),"d" (a2),"S" (a3)); + +return ret; +} +static void test_pv_ipi (void ) {int ret;
@@ -651,10 +663,7 @@ static void test_pv_ipi(void)if (!test_device_enabled ())return ; -if (is_intel ()) - asmvolatile ("vmcall" :"=a" (ret) :"a" (KVM_HC_SEND_IPI ),"b" (a0),"c" (a1),"d" (a2),"S" (a3)); -else - asmvolatile ("vmmcall" :"=a" (ret) :"a" (KVM_HC_SEND_IPI ),"b" (a0),"c" (a1),"d" (a2),"S" (a3)); + ret =pv_ipi (a0, a1, a2, a3);report (!ret,"PV IPIs testing" ); }
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
PATCH 5/5 x86/apic: Add a test for a PV IPI boundary condition
In reply to Jim MattsonAdd a new test case, test_pv_ipi_sparse(), to verify that PV IPIs are correctly delivered to vCPUs with APIC ID 255.
APIC ID 255 represents a critical boundary condition for PV IPIs:
1. It is the transition point where 0xff ceases to be a broadcast ID (as in xAPIC) and becomes a valid unicast ID (in x2APIC).
2. KVM's PV IPI mechanism groups vCPUs into 64-bit clusters. APIC ID 255 maps to the 64th bit (bit 63) of the cluster bitmap. This tests potential off-by-one or fencepost errors at the MSB boundary of the bitmap processing loop.
Additionally, this introduces a sparse topology test configuration (smp=1, maxcpus=256, and a hotplugged CPU at APIC ID 255) to ensure KVM's sparse APIC ID map handling is robust and does not suffer from out-of-bounds accesses when resolving target vCPUs.
--- x86/apic.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++ x86/unittests.cfg | 8 +++++++ 2 files changed, 68 insertions(+)
x86/apic.c +60 -0
@@ -915,6 +915,65 @@ static void test_aliased_xapic_physical_ipi(void)report (!f,"IPI to aliased xAPIC physical IDs" ); } +static atomic_t pv_ipi_received[256 ]; + +static void handle_pv_ipi (isr_regs_t *regs) +{ +atomic_inc (&pv_ipi_received[smp_id ()]); +apic_write (APIC_EOI ,0 ); +} + +/* +* Verify that PV IPIs are correctly delivered to a vCPU with APIC ID 255. +*/ +static void test_pv_ipi_sparse (void ) +{ +const u8 vector =0xf2 ; +int ret; + +if (!this_cpu_has_kvm () || !this_cpu_has (KVM_FEATURE_PV_SEND_IPI )) { +report_skip ("PV IPIs are not supported" ); +return ; + } + +if (!test_device_enabled ()) { +report_skip ("Test device not enabled" ); +return ; + } + +if (cpu_count () !=2 ) { +report_skip ("Requires 2 CPUs" ); +return ; + } + +/* 255 is a broadcast ID in xAPIC, so we must use x2APIC */ +if (!is_x2apic_enabled ()) { +report_skip ("x2APIC is not enabled" ); +return ; + } + +if (id_map[1 ] !=255 ) { +report_skip ("vCPU 1 does not have APIC ID 255" ); +return ; + } + +handle_irq (vector, handle_pv_ipi); + +/* Send PV IPI to 255 (vCPU 1) */ + ret =pv_ipi (1 ,0 ,255 ,APIC_DM_FIXED | vector); + +/* Wait for IPI to be received on vCPU 1 */ +u64 start =rdtsc (); +while (rdtsc () - start <1000000000 && + !atomic_read (&pv_ipi_received[255 ])) { +pause (); + } + +report (ret ==1 &&atomic_read (&pv_ipi_received[255 ]) ==1 && +atomic_read (&pv_ipi_received[0 ]) ==0 , +"PV IPI to sparse APIC ID 255" ); +} +struct apic_test {const char *name;void (*fn)(void );
@@ -937,6 +996,7 @@ int main(int argc, char **argv) { "physical_broadcast", test_physical_broadcast }, { "logical_ipi_xapic", test_logical_ipi_xapic }, { "pv_ipi", test_pv_ipi }, + { "pv_ipi_sparse", test_pv_ipi_sparse }, { "sti_nmi", test_sti_nmi }, { "multiple_nmi", test_multiple_nmi }, { "pending_nmi", test_pending_nmi },
x86/unittests.cfg +8 -0
@@ -535,3 +535,11 @@ qemu_params = -enable-kvm -m 2048 -cpu host file = lam.flat arch = x86_64 qemu_params = -enable-kvm -cpu max + +[apic_sparse] +file = apic.flat +smp = 1,maxcpus=256 +qemu_params = -M q35 -cpu qemu64,+x2apic,+tsc-deadline,+kvm-pv-ipi -device qemu64-x86_64-cpu,apic-id=255 -append 'pv_ipi_sparse' +arch = x86_64 +timeout = 30 +groups = apic
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
Quoted 4 lines · Jim Mattson
This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.
Ping.
Quoted 8 lines · Jim Mattson
Quoted 4 lines · Jim Mattson
This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.
Ping.
Adding Thomas Huth.
Quoted 4 lines · Jim Mattson
Quoted 3 lines · Jim Mattson
Quoted 1 line · Jim Mattson
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.
Ping.
Adding Thomas Huth.
x86 is handled by Paolo, so I hope he will finally respond.
Thomas
Paolo,
Is this a silent NAK?
Quoted 19 lines · Thomas Huth
Quoted 4 lines · Jim Mattson
Quoted 3 lines · Jim Mattson
Quoted 1 line · Jim Mattson
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.
Ping.
Adding Thomas Huth.
x86 is handled by Paolo, so I hope he will finally respond.
Thomas
Quoted 21 lines · Jim Mattson
This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.
1. Robustness: Skip the PV IPI test if KVM PV IPI features are not supported or if not running under KVM, preventing false failures and potential #UDs on bare metal or non-KVM hypervisors.
2. Code Reuse: Hoist the command-line test selection helper fromx86/vmexit.cinto the common library (lib/argv.c) asargv_test_wanted().
3. Subtest Selection: Refactorx86/apic.cto use this library helper, allowing users to run a subset of APIC tests (e.g., via QEMU's -append option).
4. Refactoring: Extract the KVM PV IPI hypercall inline assembly into a sharedpv_ipi()helper.
5. New Test: Addtest_pv_ipi_sparse()to verify PV IPI delivery to a vCPU with APIC ID 255. APIC ID 255 is a key boundary condition (the xAPIC/x2APIC transition point, and the MSB of the 64-bit cluster bitmap used by KVM's PV IPI hypercall). A sparse topology is used to verify KVM's handling of sparse APIC ID maps.
Quoted 1 line
[...]
Applied to kvm-x86 next, thanks!
[1/5] x86/apic: Skip PV IPI test if feature is not supported
https://github.com/kvm-x86/kvm-unit-tests/commit/6d3b20008057 [2/5] lib: Hoist test selection logic to library for code reuse https://github.com/kvm-x86/kvm-unit-tests/commit/8ba01c764a8e [3/5] x86/apic: Support subtest selection via command line https://github.com/kvm-x86/kvm-unit-tests/commit/6389f4189d27 [4/5] x86/apic: Introduce pv_ipi() https://github.com/kvm-x86/kvm-unit-tests/commit/160394cff670 [5/5] x86/apic: Add a test for a PV IPI boundary condition https://github.com/kvm-x86/kvm-unit-tests/commit/1a50e4fc12cc