Skip to contentSkip to conversation

x86/apic: PV IPI robustness and subtest selection

PATCH Applied Jim Mattson 3 participants 11 messages Jun 19 – Sep 25

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover x86/apic: PV IPI robustness and subtest selection

This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.

1. Robustness: Skip the PV IPI test if KVM PV IPI features are not supported or if not running under KVM, preventing false failures and potential #UDs on bare metal or non-KVM hypervisors.
2. Code Reuse: Hoist the command-line test selection helper from x86/vmexit.c into the common library (lib/argv.c) as argv_test_wanted().
3. Subtest Selection: Refactor x86/apic.c to use this library helper, allowing users to run a subset of APIC tests (e.g., via QEMU's -append option).
4. Refactoring: Extract the KVM PV IPI hypercall inline assembly into a shared pv_ipi() helper.
5. New Test: Add test_pv_ipi_sparse() to verify PV IPI delivery to a vCPU with APIC ID 255. APIC ID 255 is a key boundary condition (the xAPIC/x2APIC transition point, and the MSB of the 64-bit cluster bitmap used by KVM's PV IPI hypercall). A sparse topology is used to verify KVM's handling of sparse APIC ID maps.

Jim Mattson (5):
x86/apic: Skip PV IPI test if feature is not supported
lib: Hoist test selection logic to library for code reuse x86/apic: Support subtest selection via command line x86/apic: Introduce pv_ipi()
x86/apic: Add a test for a PV IPI boundary condition

 lib/argv.c          |  14 +++++
 lib/argv.h          |   3 +
 lib/x86/processor.h |   1 +
 x86/apic.c          | 133 ++++++++++++++++++++++++++++++++++----------
 x86/unittests.cfg   |  12 +++-
 x86/vmexit.c        |  15 +----
 6 files changed, 135 insertions(+), 43 deletions(-)

base-commit: afcc98a30a84aaefc1f0d6783ffffec16c736717

Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#1 lore raw

PATCH 1/5 x86/apic: Skip PV IPI test if feature is not supported

Check for KVM_FEATURE_PV_SEND_IPI before running the PV IPI test to avoid false failures or potential #UD exceptions in environments where KVM PV features are disabled, not supported by the host, or when running on non-KVM hypervisors or bare metal.

Define the KVM_FEATURE_PV_SEND_IPI macro (bit 11 of EAX in KVM CPUID leaf 0x40000001) in processor.h to facilitate this check.

Additionally, update the [xapic] and [x2apic] configurations in unittests.cfg to advertise PV IPIs (+kvm-pv-ipi) in the guest CPUID, ensuring the PV IPI tests are not skipped in standard test runs.

Fixes de8d3fcc5b63 ("x86: apic: pv ipis out-of-bounds access testing")
Assisted-by Gemini:gemini-3.1-pro
---
 lib/x86/processor.h | 1 +
 x86/apic.c          | 5 +++++
 x86/unittests.cfg   | 4 ++--
 3 files changed, 8 insertions(+), 2 deletions(-)
lib/x86/processor.h +1 -0
index b2726d939a47..c2bc0a3d9ef5 100644
@@ -354,6 +354,7 @@ struct x86_cpu_feature {
  */
 #define KVM_FEATURE_CLOCKSOURCE2	X86_CPU_FEATURE(0x40000001, 0, EAX, 3)
 #define KVM_FEATURE_ASYNC_PF		X86_CPU_FEATURE(0x40000001, 0, EAX, 4)
+#define KVM_FEATURE_PV_SEND_IPI		X86_CPU_FEATURE(0x40000001, 0, EAX, 11)
 #define KVM_FEATURE_ASYNC_PF_INT	X86_CPU_FEATURE(0x40000001, 0, EAX, 14)
 
 /*
x86/apic.c +5 -0
index 1c37cb295209..fee1b7893185 100644
@@ -642,6 +642,11 @@ static void test_pv_ipi(void)
 	int ret;
 	unsigned long a0 = 0xFFFFFFFF, a1 = 0, a2 = 0xFFFFFFFF, a3 = 0x0;
 
+	if (!this_cpu_has_kvm() || !this_cpu_has(KVM_FEATURE_PV_SEND_IPI)) {
+		report_skip("PV IPIs are not supported");
+		return;
+	}
+
 	if (!test_device_enabled())
 		return;
 
x86/unittests.cfg +2 -2
index b9cbcc596c3f..0c433ac29e5f 100644
@@ -10,7 +10,7 @@
 [x2apic]
 file = apic.flat
 smp = 2
-qemu_params = -cpu qemu64,+x2apic,+tsc-deadline
+qemu_params = -cpu qemu64,+x2apic,+tsc-deadline,+kvm-pv-ipi
 arch = x86_64
 timeout = 30
 groups = apic
@@ -19,7 +19,7 @@ groups = apic
 [xapic]
 file = apic.flat
 smp = 2
-qemu_params = -cpu qemu64,-x2apic,+tsc-deadline
+qemu_params = -cpu qemu64,-x2apic,+tsc-deadline,+kvm-pv-ipi
 arch = x86_64
 timeout = 60
 groups = apic
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#2 lore raw

PATCH 2/5 lib: Hoist test selection logic to library for code reuse

In reply to Jim Mattson

Hoist the 'test_wanted()' helper from x86/vmexit.c into the common library as 'argv_test_wanted()' to facilitate code reuse.

This eliminates duplicate code and provides a generic, architecture-independent helper that any test suite can leverage to support running a subset of tests specified via command-line arguments.

Refactor x86/vmexit.c to use the new library helper.

Assisted-by Gemini:gemini-3.1-pro
---
 lib/argv.c   | 14 ++++++++++++++
 lib/argv.h   |  3 +++
 x86/vmexit.c | 15 ++-------------
 3 files changed, 19 insertions(+), 13 deletions(-)
lib/argv.c +14 -0
index fa5ff9ae9d28..b290c1f33903 100644
@@ -146,3 +146,17 @@ void setup_env(char *env, int size)
 		*env++ = '\0';
 	}
 }
+
+bool argv_test_wanted(const char *name, char *wanted[], int nwanted)
+{
+	int i;
+
+	if (!nwanted)
+		return true;
+
+	for (i = 0; i < nwanted; ++i)
+		if (strcmp(wanted[i], name) == 0)
+			return true;
+
+	return false;
+}
lib/argv.h +3 -0
index 0fa7772549da..bb539b0dcb44 100644
@@ -8,10 +8,13 @@
 #ifndef _ARGV_H_
 #define _ARGV_H_
 
+#include <stdbool.h>
+
 extern void __setup_args(void);
 extern void setup_args(const char *args);
 extern void setup_args_progname(const char *args);
 extern void setup_env(char *env, int size);
 extern void add_setup_arg(const char *arg);
+extern bool argv_test_wanted(const char *name, char *wanted[], int nwanted);
 
 #endif
x86/vmexit.c +2 -13
index 1b675c00f930..82f13a2faa5f 100644
@@ -1,4 +1,5 @@
 #include "libcflat.h"
+#include "argv.h"
 #include "acpi.h"
 #include "smp.h"
 #include "vmalloc.h"
@@ -594,19 +595,7 @@ static void enable_nx(void *junk)
 		wrmsr(MSR_EFER, rdmsr(MSR_EFER) | EFER_NX_MASK);
 }
 
-static bool test_wanted(struct test *test, char *wanted[], int nwanted)
-{
-	int i;
-
-	if (!nwanted)
-		return true;
-
-	for (i = 0; i < nwanted; ++i)
-		if (strcmp(wanted[i], test->name) == 0)
-			return true;
 
-	return false;
-}
 
 int main(int ac, char **av)
 {
@@ -636,7 +625,7 @@ int main(int ac, char **av)
 	}
 
 	for (i = 0; i < ARRAY_SIZE(tests); ++i)
-		if (test_wanted(&tests[i], av + 1, ac - 1))
+		if (argv_test_wanted(tests[i].name, av + 1, ac - 1))
 			while (do_test(&tests[i])) {}
 
 	return 0;
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#3 lore raw

PATCH 3/5 x86/apic: Support subtest selection via command line

In reply to Jim Mattson

Refactor the APIC test suite to support running a subset of tests specified via command line arguments (e.g., passed via QEMU's -append option). This matches the test selection pattern already established in x86/vmexit.c.

Define 'struct apic_test' to associate a name with each test function and leverage the new 'argv_test_wanted()' library helper to filter the execution loop.

Assisted-by Gemini:gemini-3.1-pro
---
 x86/apic.c | 57 ++++++++++++++++++++++++++++--------------------------
 1 file changed, 30 insertions(+), 27 deletions(-)
x86/apic.c +30 -27
index fee1b7893185..65198ecfb3c1 100644
@@ -1,4 +1,5 @@
 #include "libcflat.h"
+#include "argv.h"
 #include "apic.h"
 #include "vm.h"
 #include "smp.h"
@@ -905,43 +906,42 @@ static void test_aliased_xapic_physical_ipi(void)
 	report(!f, "IPI to aliased xAPIC physical IDs");
 }
 
-typedef void (*apic_test_fn)(void);
+struct apic_test {
+	const char *name;
+	void (*fn)(void);
+};
 
-int main(void)
+
+
+int main(int argc, char **argv)
 {
 	bool is_x2apic = is_x2apic_enabled();
 	u32 spiv = apic_read(APIC_SPIV);
 	int i;
 
-	const apic_test_fn tests[] = {
-		test_lapic_existence,
-
-		test_apic_disable,
-		test_enable_x2apic,
-
-		test_self_ipi_xapic,
-		test_self_ipi_x2apic,
-		test_physical_broadcast,
-		test_logical_ipi_xapic,
-
-		test_pv_ipi,
-
-		test_sti_nmi,
-		test_multiple_nmi,
-		test_pending_nmi,
-
-		test_apic_timer_one_shot,
-		test_apic_change_mode,
-		test_tsc_deadline_timer,
-
+	const struct apic_test tests[] = {
+		{ "lapic_existence", test_lapic_existence },
+		{ "apic_disable", test_apic_disable },
+		{ "enable_x2apic", test_enable_x2apic },
+		{ "self_ipi_xapic", test_self_ipi_xapic },
+		{ "self_ipi_x2apic", test_self_ipi_x2apic },
+		{ "physical_broadcast", test_physical_broadcast },
+		{ "logical_ipi_xapic", test_logical_ipi_xapic },
+		{ "pv_ipi", test_pv_ipi },
+		{ "sti_nmi", test_sti_nmi },
+		{ "multiple_nmi", test_multiple_nmi },
+		{ "pending_nmi", test_pending_nmi },
+		{ "apic_timer_one_shot", test_apic_timer_one_shot },
+		{ "apic_change_mode", test_apic_change_mode },
+		{ "tsc_deadline_timer", test_tsc_deadline_timer },
 		/*
 		 * KVM may disable APICv if the APIC ID and/or APIC_BASE is
 		 * modified, keep these tests at the end so that the test as a
 		 * whole provides coverage for APICv (when it's enabled).
 		 */
-		test_apic_id,
-		test_apicbase,
-		test_aliased_xapic_physical_ipi,
+		{ "apic_id", test_apic_id },
+		{ "apicbase", test_apicbase },
+		{ "aliased_xapic_physical_ipi", test_aliased_xapic_physical_ipi },
 	};
 
 	assert_msg(is_apic_hw_enabled() && is_apic_sw_enabled(),
@@ -953,7 +953,10 @@ int main(void)
 	sti();
 
 	for (i = 0; i < ARRAY_SIZE(tests); i++) {
-		tests[i]();
+		if (!argv_test_wanted(tests[i].name, argv + 1, argc - 1))
+			continue;
+
+		tests[i].fn();
 
 		if (is_x2apic)
 			enable_x2apic();
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#4 lore raw

PATCH 4/5 x86/apic: Introduce pv_ipi()

In reply to Jim Mattson

Extract the inline assembly hypercall from test_pv_ipi() into a new helper function, pv_ipi(), to facilitate code sharing.

Assisted-by Gemini:gemini-3.1-pro
---
 x86/apic.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
x86/apic.c +13 -4
index 65198ecfb3c1..eea0c8d8e0f9 100644
@@ -638,6 +638,18 @@ static void test_apic_change_mode(void)
 
 #define KVM_HC_SEND_IPI 10
 
+static int pv_ipi(unsigned long a0, unsigned long a1, unsigned long a2, unsigned long a3)
+{
+	int ret;
+
+	if (is_intel())
+		asm volatile("vmcall"  : "=a"(ret) :"a"(KVM_HC_SEND_IPI), "b"(a0), "c"(a1), "d"(a2), "S"(a3));
+	else
+		asm volatile("vmmcall" : "=a"(ret) :"a"(KVM_HC_SEND_IPI), "b"(a0), "c"(a1), "d"(a2), "S"(a3));
+
+	return ret;
+}
+
 static void test_pv_ipi(void)
 {
 	int ret;
@@ -651,10 +663,7 @@ static void test_pv_ipi(void)
 	if (!test_device_enabled())
 		return;
 
-	if (is_intel())
-		asm volatile("vmcall"  : "=a"(ret) :"a"(KVM_HC_SEND_IPI), "b"(a0), "c"(a1), "d"(a2), "S"(a3));
-	else
-		asm volatile("vmmcall" : "=a"(ret) :"a"(KVM_HC_SEND_IPI), "b"(a0), "c"(a1), "d"(a2), "S"(a3));
+	ret = pv_ipi(a0, a1, a2, a3);
 	report(!ret, "PV IPIs testing");
 }
 
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#5 lore raw

PATCH 5/5 x86/apic: Add a test for a PV IPI boundary condition

In reply to Jim Mattson

Add a new test case, test_pv_ipi_sparse(), to verify that PV IPIs are correctly delivered to vCPUs with APIC ID 255.

APIC ID 255 represents a critical boundary condition for PV IPIs:

1. It is the transition point where 0xff ceases to be a broadcast ID (as in xAPIC) and becomes a valid unicast ID (in x2APIC).
2. KVM's PV IPI mechanism groups vCPUs into 64-bit clusters. APIC ID 255 maps to the 64th bit (bit 63) of the cluster bitmap. This tests potential off-by-one or fencepost errors at the MSB boundary of the bitmap processing loop.

Additionally, this introduces a sparse topology test configuration (smp=1, maxcpus=256, and a hotplugged CPU at APIC ID 255) to ensure KVM's sparse APIC ID map handling is robust and does not suffer from out-of-bounds accesses when resolving target vCPUs.

Assisted-by Gemini:gemini-3.1-pro
---
 x86/apic.c        | 60 +++++++++++++++++++++++++++++++++++++++++++++++
 x86/unittests.cfg |  8 +++++++
 2 files changed, 68 insertions(+)
x86/apic.c +60 -0
index eea0c8d8e0f9..2b220c9e3909 100644
@@ -915,6 +915,65 @@ static void test_aliased_xapic_physical_ipi(void)
 	report(!f, "IPI to aliased xAPIC physical IDs");
 }
 
+static atomic_t pv_ipi_received[256];
+
+static void handle_pv_ipi(isr_regs_t *regs)
+{
+	atomic_inc(&pv_ipi_received[smp_id()]);
+	apic_write(APIC_EOI, 0);
+}
+
+/*
+ * Verify that PV IPIs are correctly delivered to a vCPU with APIC ID 255.
+ */
+static void test_pv_ipi_sparse(void)
+{
+	const u8 vector = 0xf2;
+	int ret;
+
+	if (!this_cpu_has_kvm() || !this_cpu_has(KVM_FEATURE_PV_SEND_IPI)) {
+		report_skip("PV IPIs are not supported");
+		return;
+	}
+
+	if (!test_device_enabled()) {
+		report_skip("Test device not enabled");
+		return;
+	}
+
+	if (cpu_count() != 2) {
+		report_skip("Requires 2 CPUs");
+		return;
+	}
+
+	/* 255 is a broadcast ID in xAPIC, so we must use x2APIC */
+	if (!is_x2apic_enabled()) {
+		report_skip("x2APIC is not enabled");
+		return;
+	}
+
+	if (id_map[1] != 255) {
+		report_skip("vCPU 1 does not have APIC ID 255");
+		return;
+	}
+
+	handle_irq(vector, handle_pv_ipi);
+
+	/* Send PV IPI to 255 (vCPU 1) */
+	ret = pv_ipi(1, 0, 255, APIC_DM_FIXED | vector);
+
+	/* Wait for IPI to be received on vCPU 1 */
+	u64 start = rdtsc();
+	while (rdtsc() - start < 1000000000 &&
+	       !atomic_read(&pv_ipi_received[255])) {
+		pause();
+	}
+
+	report(ret == 1 && atomic_read(&pv_ipi_received[255]) == 1 &&
+	       atomic_read(&pv_ipi_received[0]) == 0,
+	       "PV IPI to sparse APIC ID 255");
+}
+
 struct apic_test {
 	const char *name;
 	void (*fn)(void);
@@ -937,6 +996,7 @@ int main(int argc, char **argv)
 		{ "physical_broadcast", test_physical_broadcast },
 		{ "logical_ipi_xapic", test_logical_ipi_xapic },
 		{ "pv_ipi", test_pv_ipi },
+		{ "pv_ipi_sparse", test_pv_ipi_sparse },
 		{ "sti_nmi", test_sti_nmi },
 		{ "multiple_nmi", test_multiple_nmi },
 		{ "pending_nmi", test_pending_nmi },
x86/unittests.cfg +8 -0
index 0c433ac29e5f..0bdeb4b1826f 100644
@@ -535,3 +535,11 @@ qemu_params = -enable-kvm -m 2048 -cpu host
 file = lam.flat
 arch = x86_64
 qemu_params = -enable-kvm -cpu max
+
+[apic_sparse]
+file = apic.flat
+smp = 1,maxcpus=256
+qemu_params = -M q35 -cpu qemu64,+x2apic,+tsc-deadline,+kvm-pv-ipi -device qemu64-x86_64-cpu,apic-id=255 -append 'pv_ipi_sparse'
+arch = x86_64
+timeout = 30
+groups = apic
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#6 lore raw
In reply to Jim Mattson
Quoted 4 lines · Jim Mattson

This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.

Ping.

#7 lore raw
Quoted 8 lines · Jim Mattson
Quoted 4 lines · Jim Mattson

This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.

Ping.

Adding Thomas Huth.

#8 lore raw
Quoted 4 lines · Jim Mattson
Quoted 3 lines · Jim Mattson
Quoted 1 line · Jim Mattson
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.

Ping.

Adding Thomas Huth.

x86 is handled by Paolo, so I hope he will finally respond.

Thomas

#9 lore raw

Paolo,

Is this a silent NAK?

Quoted 19 lines · Thomas Huth
Quoted 4 lines · Jim Mattson
Quoted 3 lines · Jim Mattson
Quoted 1 line · Jim Mattson
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.

Ping.

Adding Thomas Huth.

x86 is handled by Paolo, so I hope he will finally respond.

Thomas

reviewer Applied
#10 lore raw
In reply to Jim Mattson
Quoted 21 lines · Jim Mattson

This series improves the robustness of the APIC test suite, introduces generic subtest selection, and adds a new test case for a PV IPI boundary condition.

1. Robustness: Skip the PV IPI test if KVM PV IPI features are not supported or if not running under KVM, preventing false failures and potential #UDs on bare metal or non-KVM hypervisors.
2. Code Reuse: Hoist the command-line test selection helper from x86/vmexit.c into the common library (lib/argv.c) as argv_test_wanted().
3. Subtest Selection: Refactor x86/apic.c to use this library helper, allowing users to run a subset of APIC tests (e.g., via QEMU's -append option).
4. Refactoring: Extract the KVM PV IPI hypercall inline assembly into a shared pv_ipi() helper.
5. New Test: Add test_pv_ipi_sparse() to verify PV IPI delivery to a vCPU with APIC ID 255. APIC ID 255 is a key boundary condition (the xAPIC/x2APIC transition point, and the MSB of the 64-bit cluster bitmap used by KVM's PV IPI hypercall). A sparse topology is used to verify KVM's handling of sparse APIC ID maps.

Quoted 1 line

[...]

Applied to kvm-x86 next, thanks!

[1/5] x86/apic: Skip PV IPI test if feature is not supported

      https://github.com/kvm-x86/kvm-unit-tests/commit/6d3b20008057
[2/5] lib: Hoist test selection logic to library for code reuse
      https://github.com/kvm-x86/kvm-unit-tests/commit/8ba01c764a8e
[3/5] x86/apic: Support subtest selection via command line
      https://github.com/kvm-x86/kvm-unit-tests/commit/6389f4189d27
[4/5] x86/apic: Introduce pv_ipi()
      https://github.com/kvm-x86/kvm-unit-tests/commit/160394cff670
[5/5] x86/apic: Add a test for a PV IPI boundary condition
      https://github.com/kvm-x86/kvm-unit-tests/commit/1a50e4fc12cc

--
https://github.com/kvm-x86/kvm-unit-tests/tree/next