Skip to contentSkip to conversation

KVM: selftests: Add eventfd+VFIO IRQ test

PATCH Superseded Sean Christopherson 3 participants 33 messages Jun 13 – Jun 18

Subsystems (4)

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

v7 cover KVM: selftests: Add eventfd+VFIO IRQ test

David and Josh's series to add a selftest for verifying interrupt delivery via eventfd (via KVM_IRQFD), and also from a real device, wired up via VFIO.

I originally wanted to get this into 7.2, but that's not going to happen. But I hope to get this applied early in the 7.3 cycle so that additional features and whatnot can be developed on top without too much pain (hopefully).

Gory details in the patches, and in the v5 cover letter.

v7:
- Seed kvm_rng during kvm_selftest_init(). [Sashiko]
- Seed libc's RNG during kvm_selftest_init(), so that the seed (from random()) that's printed and passed to kvm_rng isn't the same every time.
- Init irq_cpu to -1 in all paths. [Sashiko]

v6:
- Massage most changelogs.
- Fix SoB ordering issues.
- Clean up KVM_SET_GSI_ROUTING helper.
- Remove misleading "IRQ injection" and "emulated eventfd" terminology.
- Add GUEST_RECEIVED_INTERRUPT() to simplifiy the core loop.
- Use cpu_relax() in tight loops while waiting for interrupts.
- Print as much information as possible in the actual assert, instead of printing to stdout separately.
- Make a best guess as to the right VFIO vs. IOMMUFD mode instead of assuming IOMMUFD, and give the user the option to overide said guess.
- Simplify open_proc_irq_smp_affinity_list() + write_proc_irq_smp_affinity_list() into proc_irq_set_smp_affinity().
- Drop print_proc_irq_file() and kvm_print_vcpu_affinity() (for now) to avoid potential issues on systems with high CPU counts.
- Drop the blocking/HLT testing as it was at best broken.
- Use -e for "empty", not -c for "clear", when completely tearing down GSI routing, because routing can be "cleared" without completely emptying the routing information.
- Use the main task's CPU affinity as the available_cpus set.
- Allow overcommiting vCPUs:pCPUs.
- Set the target vCPU's affinity instead of batching when vCPU0 is targeted.
- Add support for 256+ vCPUs with x2APIC.
- Restrict xAPIC mode to 255 vCPUs.
- Restrict the test to KVM selftest's max supported vCPUs.

v5:
- https://lore.kernel.org/all/[email protected]
- Rename get_proc_vfio_irq_number() to vfio_msix_to_host_irq()
- Rename open_proc_irq_affinity() and write_proc_irq_affinity() to include "_smp_affinity_list"
- Print /proc/irq/<irq>/smp_affinity and effective_affinity on timeout failures
- Convert IRQ type from 'int' to 'unsigned int' across helpers and the test
- Fix compiler warnings for uninitialized variables in irq_test.c
- Remove rate-limiting on affinity changes

v4: https://lore.kernel.org/kvm/[email protected]

David Matlack (11):
KVM: selftests: Build and link selftests/vfio/lib into KVM selftests
KVM: selftests: Add macros to read/write+sync to/from guest memory
KVM: selftests: Add an irqfd send+receive (and later IRQ bypass) test
KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ
bypass test
KVM: selftests: Add VFIO device support to eventfd IRQ test
KVM: selftests: Verify interrupts are received when IRQ affinity
changes in IRQ test
KVM: selftests: Add option to set empty routing between IRQs in
eventfd IRQ test
KVM: selftests: Make number of IRQs configurable in IRQ test
KVM: selftests: Verify vCPU migration during IRQ delivery in IRQ test
KVM: selftests: Make number of vCPUs configurable in IRQ test
KVM: selftests: Add xAPIC support in eventfd IRQ test

Josh Hilke (6):
KVM: selftests: Rename guest_rng to kvm_rng
KVM: selftests: Add helper to generate random u64 in range [min,max]
KVM: selftests: Add a helper to set proc IRQ affinity for IRQ test
KVM: selftests: Add kvm_gettid() wrapper and convert users
KVM: selftests: Add kvm_sched_getaffinity() wrapper and convert users
KVM: selftests: Add a utility to pin a task to a random CPU, given a
CPU set

Sean Christopherson (3):
KVM: selftests: Initialize the default/global pRNG during
kvm_selftest_init()
KVM: selftests: Seed libc's RNG before using it to generate a seed for
KVM's pRNG
KVM: selftests: Verify non-postable IRQ remapping in IRQ test

 tools/testing/selftests/kvm/Makefile.kvm      |   7 +-
 tools/testing/selftests/kvm/arch_timer.c      |   2 +-
 .../kvm/arm64/arch_timer_edge_cases.c         |   2 +-
 .../selftests/kvm/demand_paging_test.c        |   2 +-
 .../selftests/kvm/dirty_log_perf_test.c       |   4 +-
 tools/testing/selftests/kvm/dirty_log_test.c  |  11 +-
 .../selftests/kvm/include/kvm_syscalls.h      |   7 +
 .../testing/selftests/kvm/include/kvm_util.h  |  12 +
 .../testing/selftests/kvm/include/proc_util.h |  11 +
 .../testing/selftests/kvm/include/test_util.h |  25 +-
 .../selftests/kvm/include/x86/kvm_util_arch.h |   4 +-
 tools/testing/selftests/kvm/irq_test.c        | 350 ++++++++++++++++++
 tools/testing/selftests/kvm/lib/assert.c      |   8 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    |  53 ++-
 tools/testing/selftests/kvm/lib/memstress.c   |   8 +-
 tools/testing/selftests/kvm/lib/proc_util.c   |  54 +++
 tools/testing/selftests/kvm/lib/test_util.c   |  27 +-
 tools/testing/selftests/kvm/mmu_stress_test.c |  15 +-
 tools/testing/selftests/kvm/rseq_test.c       |   6 +-
 tools/testing/selftests/kvm/steal_time.c      |  22 +-
 .../testing/selftests/kvm/x86/sev_dbg_test.c  |   2 +-
 21 files changed, 548 insertions(+), 84 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/include/proc_util.h
 create mode 100644 tools/testing/selftests/kvm/irq_test.c
 create mode 100644 tools/testing/selftests/kvm/lib/proc_util.c

base-commit: c1f7303302927f9cbf4efedf70f0512cde168c65

Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#1 lore raw

PATCH v7 1/20 KVM: selftests: Build and link selftests/vfio/lib into KVM selftests

From: David Matlack <[email protected]>

Include libvfio.mk into the KVM selftests Makefile and link it into all KVM selftests by adding it to LIBKVM_OBJS.

This lays the groundwork for future changes to utilize VFIO devices to verify IRQ bypass in KVM selftests.

Note that KVM selftests build their own copy of selftests/vfio/lib and the resulting object files are placed in $(OUTPUT)/lib. This allows the KVM and VFIO selftests to apply different CFLAGS when building without conflicting with each other.

---
 tools/testing/selftests/kvm/Makefile.kvm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/Makefile.kvm +4 -1
index 9462919d2660..59ec232b1e93 100644
@@ -258,6 +258,7 @@ OVERRIDE_TARGETS = 1
 # which causes the environment variable to override the makefile).
 include ../lib.mk
 include ../cgroup/lib/libcgroup.mk
+include ../vfio/lib/libvfio.mk
 
 INSTALL_HDR_PATH = $(top_srcdir)/usr
 LINUX_HDR_PATH = $(INSTALL_HDR_PATH)/include/
@@ -312,7 +313,9 @@ LIBKVM_S := $(filter %.S,$(LIBKVM))
 LIBKVM_C_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_C))
 LIBKVM_S_OBJ := $(patsubst %.S, $(OUTPUT)/%.o, $(LIBKVM_S))
 LIBKVM_STRING_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_STRING))
-LIBKVM_OBJS = $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ) $(LIBKVM_STRING_OBJ) $(LIBCGROUP_O)
+LIBKVM_OBJS = $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ) $(LIBKVM_STRING_OBJ)
+LIBKVM_OBJS += $(LIBCGROUP_O)
+LIBKVM_OBJS += $(LIBVFIO_O)
 SPLIT_TEST_GEN_PROGS := $(patsubst %, $(OUTPUT)/%, $(SPLIT_TESTS))
 SPLIT_TEST_GEN_OBJ := $(patsubst %, $(OUTPUT)/$(ARCH)/%.o, $(SPLIT_TESTS))
 
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#2 lore raw

PATCH v7 2/20 KVM: selftests: Add macros to read/write+sync to/from guest memory

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Add SYNC_FROM_GUEST_AND_READ(vm, variable), to read a variable value from the guest. Add WRITE_AND_SYNC_TO_GUEST(vm, variable, value) to write a value to a guest variable. These macros improve the readability of code which reads and writes data between host and guest in tests.

Use the new macro in existing tests that do back-to-back write+sync.

No functional changes are intended.

---
 tools/testing/selftests/kvm/dirty_log_test.c  |  9 +++-----
 .../testing/selftests/kvm/include/kvm_util.h  | 10 +++++++++
 tools/testing/selftests/kvm/mmu_stress_test.c |  9 +++-----
 tools/testing/selftests/kvm/steal_time.c      | 22 +++++++------------
 4 files changed, 24 insertions(+), 26 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +3 -6
index 74ca096bf976..087e94a8a81a 100644
@@ -708,8 +708,7 @@ static void run_test(enum vm_guest_mode mode, void *arg)
 
 		sync_global_to_guest(vm, iteration);
 
-		WRITE_ONCE(nr_writes, 0);
-		sync_global_to_guest(vm, nr_writes);
+		WRITE_AND_SYNC_TO_GUEST(vm, nr_writes, 0);
 
 		dirty_ring_prev_iteration_last_page = dirty_ring_last_page;
 		WRITE_ONCE(dirty_ring_vcpu_ring_full, false);
@@ -775,16 +774,14 @@ static void run_test(enum vm_guest_mode mode, void *arg)
 		 * writing memory during verification, pages that this thread
 		 * sees as clean may be written with this iteration's value.
 		 */
-		WRITE_ONCE(vcpu_stop, true);
-		sync_global_to_guest(vm, vcpu_stop);
+		WRITE_AND_SYNC_TO_GUEST(vm, vcpu_stop, true);
 		sem_wait(&sem_vcpu_stop);
 
 		/*
 		 * Clear vcpu_stop after the vCPU thread has acknowledge the
 		 * stop request and is waiting, i.e. is definitely not running!
 		 */
-		WRITE_ONCE(vcpu_stop, false);
-		sync_global_to_guest(vm, vcpu_stop);
+		WRITE_AND_SYNC_TO_GUEST(vm, vcpu_stop, false);
 
 		/*
 		 * Sync the number of writes performed before verification, the
tools/testing/selftests/kvm/include/kvm_util.h +10 -0
index 04a910164a29..c1f588154398 100644
@@ -1138,6 +1138,16 @@ vm_adjust_num_guest_pages(enum vm_guest_mode mode, unsigned int num_guest_pages)
 	memcpy(&(g), _p, sizeof(g));				\
 })
 
+#define SYNC_FROM_GUEST_AND_READ(_vm, _variable) ({		\
+	sync_global_from_guest(_vm, _variable);			\
+	READ_ONCE(_variable);					\
+})
+
+#define WRITE_AND_SYNC_TO_GUEST(_vm, _variable, _value) do {	\
+	WRITE_ONCE(_variable, _value);				\
+	sync_global_to_guest(_vm, _variable);			\
+} while (0)
+
 /*
  * Write a global value, but only in the VM's (guest's) domain.  Primarily used
  * for "globals" that hold per-VM values (VMs always duplicate code and global
tools/testing/selftests/kvm/mmu_stress_test.c +3 -6
index 54d281419d31..473ef4c0ea9f 100644
@@ -155,10 +155,8 @@ static void *vcpu_worker(void *data)
 		    "Expected EFAULT on write to RO memory, got r = %d, errno = %d", r, errno);
 
 	atomic_inc(&nr_ro_faults);
-	if (atomic_read(&nr_ro_faults) == nr_vcpus) {
-		WRITE_ONCE(all_vcpus_hit_ro_fault, true);
-		sync_global_to_guest(vm, all_vcpus_hit_ro_fault);
-	}
+	if (atomic_read(&nr_ro_faults) == nr_vcpus)
+		WRITE_AND_SYNC_TO_GUEST(vm, all_vcpus_hit_ro_fault, true);
 
 #if defined(__x86_64__) || defined(__aarch64__)
 	/*
@@ -383,8 +381,7 @@ int main(int argc, char *argv[])
 	rendezvous_with_vcpus(&time_run2, "run 2");
 
 	mprotect(mem, slot_size, PROT_READ);
-	mprotect_ro_done = true;
-	sync_global_to_guest(vm, mprotect_ro_done);
+	WRITE_AND_SYNC_TO_GUEST(vm, mprotect_ro_done, true);
 
 	rendezvous_with_vcpus(&time_ro, "mprotect RO");
 	mprotect(mem, slot_size, PROT_READ | PROT_WRITE);
tools/testing/selftests/kvm/steal_time.c +8 -14
index 76fcdd1fd3cb..2de87549fcc0 100644
@@ -70,8 +70,8 @@ static bool is_steal_time_supported(struct kvm_vcpu *vcpu)
 static void steal_time_init(struct kvm_vcpu *vcpu, u32 i)
 {
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	sync_global_to_guest(vcpu->vm, st_gva[i]);
+	WRITE_AND_SYNC_TO_GUEST(vcpu->vm, st_gva[i],
+				(void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
 
 	vcpu_set_msr(vcpu, MSR_KVM_STEAL_TIME, (ulong)st_gva[i] | KVM_MSR_ENABLED);
 }
@@ -187,8 +187,7 @@ static void steal_time_init(struct kvm_vcpu *vcpu, u32 i)
 	};
 
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	sync_global_to_guest(vm, st_gva[i]);
+	WRITE_AND_SYNC_TO_GUEST(vm, st_gva[i], (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
 
 	st_ipa = (ulong)st_gva[i];
 	vcpu_ioctl(vcpu, KVM_SET_DEVICE_ATTR, &dev);
@@ -310,10 +309,8 @@ static bool is_steal_time_supported(struct kvm_vcpu *vcpu)
 static void steal_time_init(struct kvm_vcpu *vcpu, u32 i)
 {
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	st_gpa[i] = addr_gva2gpa(vcpu->vm, (gva_t)st_gva[i]);
-	sync_global_to_guest(vcpu->vm, st_gva[i]);
-	sync_global_to_guest(vcpu->vm, st_gpa[i]);
+	WRITE_AND_SYNC_TO_GUEST(vcpu->vm, st_gva[i], (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
+	WRITE_AND_SYNC_TO_GUEST(vcpu->vm, st_gpa[i], addr_gva2gpa(vcpu->vm, (gva_t)st_gva[i]));
 }
 
 static void steal_time_dump(struct kvm_vm *vm, u32 vcpu_idx)
@@ -442,8 +439,7 @@ static void steal_time_init(struct kvm_vcpu *vcpu, u32 i)
 	};
 
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	sync_global_to_guest(vm, st_gva[i]);
+	WRITE_AND_SYNC_TO_GUEST(vm, st_gva[i], (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
 
 	err = __vcpu_ioctl(vcpu, KVM_HAS_DEVICE_ATTR, &attr);
 	TEST_ASSERT(err == 0, "No PV stealtime Feature");
@@ -549,8 +545,7 @@ int main(int ac, char **av)
 
 		/* Second VCPU run, expect guest stolen time to be <= run_delay */
 		run_vcpu(vcpus[i]);
-		sync_global_from_guest(vm, guest_stolen_time[i]);
-		stolen_time = guest_stolen_time[i];
+		stolen_time = SYNC_FROM_GUEST_AND_READ(vm, guest_stolen_time[i]);
 		run_delay = get_run_delay();
 		TEST_ASSERT(stolen_time <= run_delay,
 			    "Expected stolen time <= %ld, got %ld",
@@ -570,8 +565,7 @@ int main(int ac, char **av)
 
 		/* Run VCPU again to confirm stolen time is consistent with run_delay */
 		run_vcpu(vcpus[i]);
-		sync_global_from_guest(vm, guest_stolen_time[i]);
-		stolen_time = guest_stolen_time[i] - stolen_time;
+		stolen_time = SYNC_FROM_GUEST_AND_READ(vm, guest_stolen_time[i]) - stolen_time;
 		TEST_ASSERT(stolen_time >= run_delay,
 			    "Expected stolen time >= %ld, got %ld",
 			    run_delay, stolen_time);
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#3 lore raw

PATCH v7 3/20 KVM: selftests: Rename guest_rng to kvm_rng

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Rename functions prefixed with 'guest_random_' to 'kvm_random_' and the global random state variable 'guest_rng' to 'kvm_rng', as the pRNG isn't strictly limited to guest code. This will allow using the pRNG in host code without creating confusing/misleading function calls.

No functional changes are intended.

---
 .../selftests/kvm/dirty_log_perf_test.c       |  4 ++--
 tools/testing/selftests/kvm/dirty_log_test.c  |  2 +-
 .../testing/selftests/kvm/include/test_util.h | 22 +++++++++----------
 .../selftests/kvm/include/x86/kvm_util_arch.h |  4 ++--
 tools/testing/selftests/kvm/lib/kvm_util.c    | 20 ++++++++---------
 tools/testing/selftests/kvm/lib/memstress.c   |  8 +++----
 tools/testing/selftests/kvm/lib/test_util.c   |  6 ++---
 .../testing/selftests/kvm/x86/sev_dbg_test.c  |  2 +-
 8 files changed, 34 insertions(+), 34 deletions(-)
tools/testing/selftests/kvm/dirty_log_perf_test.c +2 -2
index ef779fa91827..7c5abe1ae9e0 100644
@@ -311,7 +311,7 @@ int main(int argc, char *argv[])
 	int opt;
 
 	/* Override the seed to be deterministic by default. */
-	guest_random_seed = 1;
+	kvm_random_seed = 1;
 
 	dirty_log_manual_caps =
 		kvm_check_cap(KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2);
@@ -357,7 +357,7 @@ int main(int argc, char *argv[])
 			p.phys_offset = strtoull(optarg, NULL, 0);
 			break;
 		case 'r':
-			guest_random_seed = atoi_positive("Random seed", optarg);
+			kvm_random_seed = atoi_positive("Random seed", optarg);
 			break;
 		case 's':
 			p.backing_src = parse_backing_src_type(optarg);
tools/testing/selftests/kvm/dirty_log_test.c +1 -1
index 087e94a8a81a..e8419d7da1ea 100644
@@ -121,7 +121,7 @@ static void guest_code(void)
 	while (true) {
 		while (!READ_ONCE(vcpu_stop)) {
 			addr = guest_test_virt_mem;
-			addr += (guest_random_u64(&guest_rng) % guest_num_pages)
+			addr += (kvm_random_u64(&kvm_rng) % guest_num_pages)
 				* guest_page_size;
 			addr = align_down(addr, host_page_size);
 
tools/testing/selftests/kvm/include/test_util.h +11 -11
index a56271c237ae..44c0104d60ac 100644
@@ -108,30 +108,30 @@ struct timespec timespec_sub(struct timespec ts1, struct timespec ts2);
 struct timespec timespec_elapsed(struct timespec start);
 struct timespec timespec_div(struct timespec ts, int divisor);
 
-struct guest_random_state {
+struct kvm_random_state {
 	u32 seed;
 };
 
-extern u32 guest_random_seed;
-extern struct guest_random_state guest_rng;
+extern u32 kvm_random_seed;
+extern struct kvm_random_state kvm_rng;
 
-struct guest_random_state new_guest_random_state(u32 seed);
-u32 guest_random_u32(struct guest_random_state *state);
+struct kvm_random_state new_kvm_random_state(u32 seed);
+u32 kvm_random_u32(struct kvm_random_state *state);
 
-static inline bool __guest_random_bool(struct guest_random_state *state,
+static inline bool __kvm_random_bool(struct kvm_random_state *state,
 				       u8 percent)
 {
-	return (guest_random_u32(state) % 100) < percent;
+	return (kvm_random_u32(state) % 100) < percent;
 }
 
-static inline bool guest_random_bool(struct guest_random_state *state)
+static inline bool kvm_random_bool(struct kvm_random_state *state)
 {
-	return __guest_random_bool(state, 50);
+	return __kvm_random_bool(state, 50);
 }
 
-static inline u64 guest_random_u64(struct guest_random_state *state)
+static inline u64 kvm_random_u64(struct kvm_random_state *state)
 {
-	return ((u64)guest_random_u32(state) << 32) | guest_random_u32(state);
+	return ((u64)kvm_random_u32(state) << 32) | kvm_random_u32(state);
 }
 
 enum vm_mem_backing_src_type {
tools/testing/selftests/kvm/include/x86/kvm_util_arch.h +2 -2
index c33ab6e04171..6904dbda79f9 100644
@@ -55,9 +55,9 @@ static inline bool __vm_arch_has_protected_memory(struct kvm_vm_arch *arch)
 do {											\
 	const typeof(mem) val = (__val);						\
 											\
-	if (!is_forced_emulation_enabled || guest_random_bool(&guest_rng)) {		\
+	if (!is_forced_emulation_enabled || kvm_random_bool(&kvm_rng)) {		\
 		(mem) = val;								\
-	} else if (guest_random_bool(&guest_rng)) {					\
+	} else if (kvm_random_bool(&kvm_rng)) {					\
 		__asm__ __volatile__(KVM_FEP "mov %1, %0"				\
 				     : "+m" (mem)					\
 				     : "r" (val) : "memory");				\
tools/testing/selftests/kvm/lib/kvm_util.c +10 -10
index 195f3fdae1e3..875030c22d07 100644
@@ -20,9 +20,9 @@
 
 #define KVM_UTIL_MIN_PFN	2
 
-u32 guest_random_seed;
-struct guest_random_state guest_rng;
-static u32 last_guest_seed;
+u32 kvm_random_seed;
+struct kvm_random_state kvm_rng;
+static u32 last_kvm_seed;
 
 static size_t vcpu_mmap_sz(void);
 
@@ -515,12 +515,12 @@ struct kvm_vm *__vm_create(struct vm_shape shape, u32 nr_runnable_vcpus,
 	slot0 = memslot2region(vm, 0);
 	ucall_init(vm, slot0->region.guest_phys_addr + slot0->region.memory_size);
 
-	if (guest_random_seed != last_guest_seed) {
-		pr_info("Random seed: 0x%x\n", guest_random_seed);
-		last_guest_seed = guest_random_seed;
+	if (kvm_random_seed != last_kvm_seed) {
+		pr_info("Random seed: 0x%x\n", kvm_random_seed);
+		last_kvm_seed = kvm_random_seed;
 	}
-	guest_rng = new_guest_random_state(guest_random_seed);
-	sync_global_to_guest(vm, guest_rng);
+	kvm_rng = new_kvm_random_state(kvm_random_seed);
+	sync_global_to_guest(vm, kvm_rng);
 
 	kvm_arch_vm_post_create(vm, nr_runnable_vcpus);
 
@@ -2279,8 +2279,8 @@ void __attribute((constructor)) kvm_selftest_init(void)
 	sigaction(SIGILL, &sig_sa, NULL);
 	sigaction(SIGFPE, &sig_sa, NULL);
 
-	guest_random_seed = last_guest_seed = random();
-	pr_info("Random seed: 0x%x\n", guest_random_seed);
+	kvm_random_seed = last_kvm_seed = random();
+	pr_info("Random seed: 0x%x\n", kvm_random_seed);
 
 	kvm_selftest_arch_init();
 }
tools/testing/selftests/kvm/lib/memstress.c +4 -4
index 6dcd15910a06..3599b75d97c9 100644
@@ -48,14 +48,14 @@ void memstress_guest_code(u32 vcpu_idx)
 {
 	struct memstress_args *args = &memstress_args;
 	struct memstress_vcpu_args *vcpu_args = &args->vcpu_args[vcpu_idx];
-	struct guest_random_state rand_state;
+	struct kvm_random_state rand_state;
 	gva_t gva;
 	u64 pages;
 	u64 addr;
 	u64 page;
 	int i;
 
-	rand_state = new_guest_random_state(guest_random_seed + vcpu_idx);
+	rand_state = new_kvm_random_state(kvm_random_seed + vcpu_idx);
 
 	gva = vcpu_args->gva;
 	pages = vcpu_args->pages;
@@ -69,13 +69,13 @@ void memstress_guest_code(u32 vcpu_idx)
 
 		for (i = 0; i < pages; i++) {
 			if (args->random_access)
-				page = guest_random_u32(&rand_state) % pages;
+				page = kvm_random_u32(&rand_state) % pages;
 			else
 				page = i;
 
 			addr = gva + (page * args->guest_page_size);
 
-			if (__guest_random_bool(&rand_state, args->write_percent))
+			if (__kvm_random_bool(&rand_state, args->write_percent))
 				*(u64 *)addr = 0x0123456789ABCDEF;
 			else
 				READ_ONCE(*(u64 *)addr);
tools/testing/selftests/kvm/lib/test_util.c +3 -3
index bab1bd2b775b..e98ca7ef439c 100644
@@ -30,13 +30,13 @@ void __attribute__((used)) expect_sigbus_handler(int signum)
  * Park-Miller LCG using standard constants.
  */
 
-struct guest_random_state new_guest_random_state(u32 seed)
+struct kvm_random_state new_kvm_random_state(u32 seed)
 {
-	struct guest_random_state s = {.seed = seed};
+	struct kvm_random_state s = {.seed = seed};
 	return s;
 }
 
-u32 guest_random_u32(struct guest_random_state *state)
+u32 kvm_random_u32(struct kvm_random_state *state)
 {
 	state->seed = (u64)state->seed * 48271 % ((u32)(1 << 31) - 1);
 	return state->seed;
tools/testing/selftests/kvm/x86/sev_dbg_test.c +1 -1
index a9d8e4c059f9..eaa8201b937d 100644
@@ -34,7 +34,7 @@ static void validate_buffers(void)
 
 static void ____test_sev_dbg(struct kvm_vm *vm, int i, int j, int nr_bytes)
 {
-	u8 pattern = guest_random_u32(&guest_rng);
+	u8 pattern = kvm_random_u32(&kvm_rng);
 
 	if (i + nr_bytes > BUFFER_SIZE || j + nr_bytes > BUFFER_SIZE)
 		return;
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#4 lore raw

PATCH v7 4/20 KVM: selftests: Initialize the default/global pRNG during kvm_selftest_init()

In reply to Sean Christopherson

Initialize the default kvm_rng during selftest initialization so that the pRNG can be used by tests before creating a VM. As pointed out by Sashiko, failure to actually initialize the generate makes it decidedly not random.

---
 tools/testing/selftests/kvm/lib/kvm_util.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
tools/testing/selftests/kvm/lib/kvm_util.c +11 -7
index 875030c22d07..1016865d3f7a 100644
@@ -24,6 +24,13 @@ u32 kvm_random_seed;
 struct kvm_random_state kvm_rng;
 static u32 last_kvm_seed;
 
+static void kvm_seed_rng(u32 seed)
+{
+	kvm_random_seed = last_kvm_seed = seed;
+	pr_info("Random seed: 0x%x\n", kvm_random_seed);
+	kvm_rng = new_kvm_random_state(kvm_random_seed);
+}
+
 static size_t vcpu_mmap_sz(void);
 
 int __open_path_or_exit(const char *path, int flags, const char *enoent_help)
@@ -515,11 +522,9 @@ struct kvm_vm *__vm_create(struct vm_shape shape, u32 nr_runnable_vcpus,
 	slot0 = memslot2region(vm, 0);
 	ucall_init(vm, slot0->region.guest_phys_addr + slot0->region.memory_size);
 
-	if (kvm_random_seed != last_kvm_seed) {
-		pr_info("Random seed: 0x%x\n", kvm_random_seed);
-		last_kvm_seed = kvm_random_seed;
-	}
-	kvm_rng = new_kvm_random_state(kvm_random_seed);
+	if (kvm_random_seed != last_kvm_seed)
+		kvm_seed_rng(kvm_random_seed);
+
 	sync_global_to_guest(vm, kvm_rng);
 
 	kvm_arch_vm_post_create(vm, nr_runnable_vcpus);
@@ -2279,8 +2284,7 @@ void __attribute((constructor)) kvm_selftest_init(void)
 	sigaction(SIGILL, &sig_sa, NULL);
 	sigaction(SIGFPE, &sig_sa, NULL);
 
-	kvm_random_seed = last_kvm_seed = random();
-	pr_info("Random seed: 0x%x\n", kvm_random_seed);
+	kvm_seed_rng(random());
 
 	kvm_selftest_arch_init();
 }
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#5 lore raw

PATCH v7 5/20 KVM: selftests: Seed libc's RNG before using it to generate a seed for KVM's pRNG

In reply to Sean Christopherson

Seed the RNG used by random() using the de facto standard method of srand(time(0)), so that a different seed is actually used in each test run. E.g. without seeding the RNG, literally every test on x86 will use 0x6b8b4567 to seed the KVM RNG.

---
 tools/testing/selftests/kvm/lib/kvm_util.c | 1 +
 1 file changed, 1 insertion(+)
tools/testing/selftests/kvm/lib/kvm_util.c +1 -0
index 1016865d3f7a..2e08d9fcefc7 100644
@@ -2284,6 +2284,7 @@ void __attribute((constructor)) kvm_selftest_init(void)
 	sigaction(SIGILL, &sig_sa, NULL);
 	sigaction(SIGFPE, &sig_sa, NULL);
 
+	srandom(time(0));
 	kvm_seed_rng(random());
 
 	kvm_selftest_arch_init();
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
#6 lore raw

Re: [PATCH v7 05/20] KVM: selftests: Seed libc's RNG before using it to generate a seed for KVM's pRNG

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Missing `<time.h>` inclusion causes an implicit function declaration of `time()`. --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Missing <time.h> inclusion causes an implicit function declaration of time(). --

commit 3749f42be6b5df3b7d863a819171920d1f5980cc
Author: Sean Christopherson <[email protected]>

KVM: selftests: Seed libc's RNG before using it to generate a seed for KVM's pRNG

This commit updates the KVM selftests initialization to seed the RNG used by random() with the current time. This ensures a different seed is used in each test run instead of using the same static seed.

tools/testing/selftests/kvm/lib/kvm_util.c void __attribute((constructor)) kvm_selftest_init(void) · line 2287

Quoted 9 lines
tools/testing/selftests/kvm/lib/kvm_util.c +1 -0
index 1016865d3f7a2..2e08d9fcefc77 100644
@@ -2284,6 +2284,7 @@ void __attribute((constructor)) kvm_selftest_init(void)
 	sigaction(SIGILL, &sig_sa, NULL);
 	sigaction(SIGFPE, &sig_sa, NULL);
 
+	srandom(time(0));

[Severity: Low]
Does this file need to explicitly include <time.h>?

While it might compile cleanly on standard glibc environments due to transitive inclusions, using time() without explicitly including <time.h> can cause an implicit function declaration warning or error on stricter build environments like musl.

Quoted 3 lines
tools/testing/selftests/kvm/lib/kvm_util.c +0 -0
 	kvm_seed_rng(random());
 
 	kvm_selftest_arch_init();
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
author maintainer
#7 lore raw

PATCH v7 6/20 KVM: selftests: Add helper to generate random u64 in range [min,max]

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Introduce kvm_random_u64_in_range(state, min, max). This function returns a random u64 in the inclusive range of [min, max] using a struct kvm_random_state.

---
 .../testing/selftests/kvm/include/test_util.h  |  3 +++
 tools/testing/selftests/kvm/lib/test_util.c    | 18 ++++++++++++++++++
 2 files changed, 21 insertions(+)
tools/testing/selftests/kvm/include/test_util.h +3 -0
index 44c0104d60ac..d64c8a228207 100644
@@ -134,6 +134,9 @@ static inline u64 kvm_random_u64(struct kvm_random_state *state)
 	return ((u64)kvm_random_u32(state) << 32) | kvm_random_u32(state);
 }
 
+u64 kvm_random_u64_in_range(struct kvm_random_state *state, u64 min,
+			    u64 max);
+
 enum vm_mem_backing_src_type {
 	VM_MEM_SRC_ANONYMOUS,
 	VM_MEM_SRC_ANONYMOUS_THP,
tools/testing/selftests/kvm/lib/test_util.c +18 -0
index e98ca7ef439c..e208a57f190c 100644
@@ -42,6 +42,24 @@ u32 kvm_random_u32(struct kvm_random_state *state)
 	return state->seed;
 }
 
+/* Returns a random u64 in the inclusive range [min, max] */
+u64 kvm_random_u64_in_range(struct kvm_random_state *state, u64 min,
+			    u64 max)
+{
+	u64 value;
+	u64 range;
+
+	TEST_ASSERT(min <= max, "PEBKAC, min = 0x%lx, max = 0x%lx", min, max);
+
+	value = kvm_random_u64(state);
+
+	range = max - min;
+	if (range == ULLONG_MAX)
+		return value;
+
+	return min + (value % (range + 1));
+}
+
 /*
  * Parses "[0-9]+[kmgt]?".
  */
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
#8 lore raw

Re: [PATCH v7 06/20] KVM: selftests: Add helper to generate random u64 in range [min,max]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The generic PRNG helper `kvm_random_u64_in_range()` is fundamentally incapable of generating…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The generic PRNG helper kvm_random_u64_in_range() is fundamentally incapable of generating values in large portions of the requested range due to the structural limitations of the underlying generator. --

commit 9b7095f015b8aad9cb0b24b2b051b56cd3917145
Author: Josh Hilke <[email protected]>

KVM: selftests: Add helper to generate random u64 in range [min,max]

Introduce kvm_random_u64_in_range(state, min, max) to return a random u64 in the inclusive range of [min, max] using a struct kvm_random_state.

Quoted 12 lines
tools/testing/selftests/kvm/lib/test_util.c +4 -0
index e98ca7ef439c4..e208a57f190cf 100644
@@ -42,6 +42,24 @@ u32 kvm_random_u32(struct kvm_random_state *state)
 	return state->seed;
 }
 
+/* Returns a random u64 in the inclusive range [min, max] */
+u64 kvm_random_u64_in_range(struct kvm_random_state *state, u64 min,
+			    u64 max)
+{

tools/testing/selftests/kvm/lib/test_util.c u32 kvm_random_u32(struct kvm_random_state *state) · line 58

Quoted 10 lines
tools/testing/selftests/kvm/lib/test_util.c +10 -0
+	u64 value;
+	u64 range;
+
+	TEST_ASSERT(min <= max, "PEBKAC, min = 0x%lx, max = 0x%lx", min, max);
+
+	value = kvm_random_u64(state);
+
+	range = max - min;
+	if (range == ULLONG_MAX)
+		return value;

[Severity: Medium]
Can this regression cause tests to silently miss important edge cases?

Since kvm_random_u32() uses a linear congruential generator modulo 0x7FFFFFFF, its 31st bit is permanently 0, and it never returns 0.

Because kvm_random_u64() concatenates two such values:

tools/testing/selftests/kvm/include/test_util.h:kvm_random_u64() {
	return ((u64)kvm_random_u32(state) << 32) | kvm_random_u32(state);
}

bits 31 and 63 of the resulting 64-bit number are permanently 0, and the lower 32 bits are never 0.

When this is called with a large range like min = 0 and max = ULLONG_MAX, it directly returns this 64-bit value. Thus, it can never generate numbers above 0x7FFFFFFE7FFFFFFE, any numbers with bit 31 or 63 set, or any numbers below 2^32.

Does this give a false illusion of full mathematical coverage for randomized inputs?

Quoted 3 lines
tools/testing/selftests/kvm/lib/test_util.c +3 -0
+
+	return min + (value % (range + 1));
+}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
author maintainer
#9 lore raw

PATCH v7 7/20 KVM: selftests: Add an irqfd send+receive (and later IRQ bypass) test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Add a new test, irq_test to verify that KVM correctly delivers interrupts to a running vCPU, when triggered via an eventfd bound to a KVM GSI using KVM's irqfd mechanism.

This test is intentionally simple, for now. Support for sending interrupts via VFIO devices, for IRQ bypass, and for other features will be added in the near future.

Add the test in common code, even though it currently will only build and run on x86, as the concept and the bulk of the host-side code isn't specific to x86.

---
 tools/testing/selftests/kvm/Makefile.kvm |   1 +
 tools/testing/selftests/kvm/irq_test.c   | 170 +++++++++++++++++++++++
 2 files changed, 171 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/irq_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index 59ec232b1e93..de5f6e91203b 100644
@@ -157,6 +157,7 @@ TEST_GEN_PROGS_x86 += coalesced_io_test
 TEST_GEN_PROGS_x86 += dirty_log_perf_test
 TEST_GEN_PROGS_x86 += guest_memfd_test
 TEST_GEN_PROGS_x86 += hardware_disable_test
+TEST_GEN_PROGS_x86 += irq_test
 TEST_GEN_PROGS_x86 += mmu_stress_test
 TEST_GEN_PROGS_x86 += rseq_test
 TEST_GEN_PROGS_x86 += steal_time
tools/testing/selftests/kvm/irq_test.c new file +170 -0

170 lines not on this page · raw message

Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#10 lore raw

PATCH v7 8/20 KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ bypass test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Introduce proc_util.c and proc_util.h to house utility functions for interacting with the proc filesystem.

Add vfio_msix_to_host_irq(), which parses /proc/interrupts, to get the host Linux IRQ for a given VFIO device BDF and MSI-X vector.

This helper will be used by the eventfd IRQ test to print the host IRQ number when triggering IRQs via VFIO device, e.g. to aid in debugging if the test fails.

---
 tools/testing/selftests/kvm/Makefile.kvm      |  1 +
 .../testing/selftests/kvm/include/proc_util.h |  9 +++++
 tools/testing/selftests/kvm/lib/proc_util.c   | 40 +++++++++++++++++++
 3 files changed, 50 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/include/proc_util.h
 create mode 100644 tools/testing/selftests/kvm/lib/proc_util.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index de5f6e91203b..c112cedd3a2a 100644
@@ -11,6 +11,7 @@ LIBKVM += lib/kvm_util.c
 LIBKVM += lib/lru_gen_util.c
 LIBKVM += lib/memstress.c
 LIBKVM += lib/guest_sprintf.c
+LIBKVM += lib/proc_util.c
 LIBKVM += lib/rbtree.c
 LIBKVM += lib/sparsebit.c
 LIBKVM += lib/test_util.c
tools/testing/selftests/kvm/include/proc_util.h new file +9 -0
new file mode 100644
index 000000000000..704839b6d7af
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef SELFTEST_KVM_PROC_UTIL_H
+#define SELFTEST_KVM_PROC_UTIL_H
+
+#include <stdint.h>
+
+unsigned int vfio_msix_to_host_irq(const char *vfio_device_bdf, int msix);
+
+#endif /* SELFTEST_KVM_PROC_UTIL_H */
tools/testing/selftests/kvm/lib/proc_util.c new file +40 -0
new file mode 100644
index 000000000000..84d30f055a0a
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "kvm_util.h"
+#include "test_util.h"
+#include "proc_util.h"
+
+static FILE *open_proc_interrupts(void)
+{
+	FILE *fp;
+
+	fp = fopen("/proc/interrupts", "r");
+	TEST_ASSERT(fp, "fopen(/proc/interrupts) failed");
+
+	return fp;
+}
+
+unsigned int vfio_msix_to_host_irq(const char *device_bdf, int msix)
+{
+	char search_string[64];
+	char line[4096];
+	int irq = -1;
+	FILE *fp;
+
+	fp = open_proc_interrupts();
+
+	snprintf(search_string, sizeof(search_string), "vfio-msix[%d]", msix);
+
+	while (fgets(line, sizeof(line), fp)) {
+		if (strstr(line, device_bdf) && strstr(line, search_string)) {
+			TEST_ASSERT_EQ(1, sscanf(line, "%d:", &irq));
+			break;
+		}
+	}
+
+	fclose(fp);
+
+	TEST_ASSERT(irq != -1, "Failed to locate IRQ for %s %s", device_bdf,
+		    search_string);
+	return (unsigned int)irq;
+}
+
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
#11 lore raw

Re: [PATCH v7 08/20] KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ bypass test

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Reading /proc/interrupts with a fixed 4096-byte buffer truncates lines on large machines…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Reading /proc/interrupts with a fixed 4096-byte buffer truncates lines on large machines, leading to incorrect IRQ numbers or test crashes. --

commit 399b5fbbf0a019bffe136ff08d75cf31f2c2655f
Author: David Matlack <[email protected]>

KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ bypass test

This commit introduces proc_util.c and vfio_msix_to_host_irq() to parse /proc/interrupts and retrieve the host Linux IRQ for a given VFIO device BDF and MSI-X vector. The helper is intended to be used by the eventfd IRQ test.

[ ... ]

Quoted 5 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +0 -0
new file mode 100644
index 0000000000000..84d30f055a0a9

[ ... ]

Quoted 4 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +4 -0
+unsigned int vfio_msix_to_host_irq(const char *device_bdf, int msix)
+{
+	char search_string[64];
+	char line[4096];

tools/testing/selftests/kvm/lib/proc_util.c line 29

Quoted 10 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +10 -0
+	int irq = -1;
+	FILE *fp;
+
+	fp = open_proc_interrupts();
+
+	snprintf(search_string, sizeof(search_string), "vfio-msix[%d]", msix);
+
+	while (fgets(line, sizeof(line), fp)) {
+		if (strstr(line, device_bdf) && strstr(line, search_string)) {
+			TEST_ASSERT_EQ(1, sscanf(line, "%d:", &irq));

[Severity: Medium]
Does this 4096-byte buffer accommodate lines from /proc/interrupts on hosts with a large number of CPUs?

If a host has enough CPUs to push a single line beyond 4096 bytes, fgets() will split the line across multiple reads.

If the target device strings happen to fall into a subsequent chunk, could sscanf() mistakenly parse a CPU interrupt count as the host IRQ, or fail entirely and trigger the assert?

Quoted 10 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +10 -0
+			break;
+		}
+	}
+
+	fclose(fp);
+
+	TEST_ASSERT(irq != -1, "Failed to locate IRQ for %s %s", device_bdf,
+		    search_string);
+	return (unsigned int)irq;
+}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
author maintainer
#12 lore raw

PATCH v7 9/20 KVM: selftests: Add VFIO device support to eventfd IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-d' argument that takes a BDF (in the format segment:bus:device.function) of an interrupt-capable PCI(e) device bound to VFIO, and use said device to trigger interrupts instead of always synthesizing interrupts via direct writes to the eventfd.

Using a VFIO device to trigger interrupts validates the end-to-end delivery of IRQs for "real" devices, and when supported by hardware (and KVM), also validates interrupt delivery via IRQ bypass, i.e. via device posted IRQs.

Now that IOMMUFD is a thing, auto-probe IOMMUFD vs. "legacy" VFIO by temporarily opening /dev/iommufd, and skip the test if neither IOMMUFD nor legacy VFIO is available. Add a '-t' option to the user override the probe logic, e.g. in case IOMMUFD is available but the system is configured for legacy usage.

Note, the device must have a VFIO selftest driver in order to work with the test. A helper script to list supported devices will hopefully be available in the near future at tools/testing/selftests/vfio/scripts/list_supported_devices.sh[1].

Example:

$ ./tools/testing/selftests/kvm/irq_test -d 0000:06:0a.1
---
 tools/testing/selftests/kvm/irq_test.c | 86 ++++++++++++++++++++++++--
 1 file changed, 80 insertions(+), 6 deletions(-)
tools/testing/selftests/kvm/irq_test.c +80 -6
index 9f8895b89821..6888be54ee4a 100644
@@ -3,7 +3,10 @@
 #include "test_util.h"
 #include "apic.h"
 #include "processor.h"
+#include "proc_util.h"
 
+#include <libvfio.h>
+#include <linux/sizes.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <unistd.h>
@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)
 	return NULL;
 }
 
+static int vfio_setup_msi(struct vfio_pci_device *device)
+{
+	const int flags = MAP_SHARED | MAP_ANONYMOUS;
+	const int prot = PROT_READ | PROT_WRITE;
+	struct dma_region *region;
+
+	/* A driver is required to generate an MSI. */
+	TEST_REQUIRE(device->driver.ops);
+
+	/* Set up a DMA-able region for the driver to use. */
+	region = &device->driver.region;
+	region->iova = 0;
+	region->size = SZ_2M;
+	region->vaddr = kvm_mmap(region->size, prot, flags, -1);
+	TEST_ASSERT(region->vaddr != MAP_FAILED, "mmap() failed\n");
+	iommu_map(device->iommu, region);
+
+	vfio_pci_driver_init(device);
+	return device->driver.msi;
+}
+
+static void trigger_interrupt(struct vfio_pci_device *device, int eventfd)
+{
+	if (device)
+		vfio_pci_driver_send_msi(device);
+	else
+		eventfd_write(eventfd, 1);
+}
+
+
 static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 			  u8 vector)
 {
@@ -74,11 +107,29 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 	vm_ioctl(vm, KVM_SET_GSI_ROUTING, &routing.header);
 }
 
+static const char *probe_iommu_type(void)
+{
+	int io_fd;
+
+	io_fd = open("/dev/iommu", O_RDONLY);
+	if (io_fd >= 0) {
+		close(io_fd);
+		return MODE_IOMMUFD;
+	}
+
+	io_fd = __open_path_or_exit("/dev/vfio", O_RDONLY,
+				    "Is VFIO (or IOMMUFD) loaded and enabled?");
+	close(io_fd);
+	return MODE_VFIO_TYPE1_IOMMU;
+}
+
 static void help(const char *name)
 {
-	printf("Usage: %s [-h]\n", name);
+	printf("Usage: %s [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
+	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
+	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
 	exit(KSFT_FAIL);
 }
@@ -100,14 +151,25 @@ int main(int argc, char **argv)
 	u32 gsi = kvm_random_u64_in_range(&kvm_rng, 24, KVM_MAX_IRQ_ROUTES - 1);
 	u8 vector = kvm_random_u64_in_range(&kvm_rng, 32, UINT8_MAX);
 
-	struct kvm_vcpu *vcpus[KVM_MAX_VCPUS];
 	pthread_t vcpu_threads[KVM_MAX_VCPUS];
+	struct kvm_vcpu *vcpus[KVM_MAX_VCPUS];
+	struct vfio_pci_device *device = NULL;
 	int nr_irqs = 1000, nr_vcpus = 1;
-	int i, j, c, eventfd;
+	const char *device_bdf = NULL;
+	const char *iommu_type = NULL;
+	int i, j, c, msix, eventfd;
+	struct iommu *iommu;
 	struct kvm_vm *vm;
+	int irq;
 
-	while ((c = getopt(argc, argv, "h")) != -1) {
+	while ((c = getopt(argc, argv, "d:ht:")) != -1) {
 		switch (c) {
+		case 'd':
+			device_bdf = optarg;
+			break;
+		case 't':
+			iommu_type = optarg;
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
@@ -119,7 +181,19 @@ int main(int argc, char **argv)
 	vm = vm_create_with_vcpus(nr_vcpus, guest_code, vcpus);
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
 
-	eventfd = kvm_new_eventfd();
+	if (device_bdf) {
+		if (!iommu_type)
+			iommu_type = probe_iommu_type();
+		iommu = iommu_init(iommu_type);
+		device = vfio_pci_device_init(device_bdf, iommu);
+		msix = vfio_setup_msi(device);
+		irq = vfio_msix_to_host_irq(device_bdf, msix);
+		eventfd = device->msi_eventfds[msix];
+		printf("Using device %s MSI-X[%d] (IRQ-%u)\n", device_bdf, msix,
+		       irq);
+	} else {
+		eventfd = kvm_new_eventfd();
+	}
 
 	pr_info("Injecting interrupts for GSI %d (guest vector 0x%x) %d times\n",
 		gsi, vector, nr_irqs);
@@ -147,7 +221,7 @@ int main(int argc, char **argv)
 				    "IRQ flag for vCPU %d not clear prior to test",
 				    vcpus[j]->id);
 
-		eventfd_write(eventfd, 1);
+		trigger_interrupt(device, eventfd);
 
 		clock_gettime(CLOCK_MONOTONIC, &start);
 		while (!GUEST_RECEIVED_IRQ(vcpu) &&
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
#13 lore raw

Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] probe_iommu_type() incorrectly probes the /dev/vfio directory instead of the /dev/vfio/vfio…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] probe_iommu_type() incorrectly probes the /dev/vfio directory instead of the /dev/vfio/vfio legacy container character device, bypassing the framework's intended skip behavior for unprivileged users.
- [Medium] vfio_setup_msi() hardcodes the device's DMA region IOVA to 0, which can cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0.
--

commit 2bdd8160743775c8d18e4004e72e7ff0898fbd70
Author: David Matlack <[email protected]>

KVM: selftests: Add VFIO device support to eventfd IRQ test

This commit extends the eventfd IRQ test to support triggering interrupts using a VFIO device instead of manually signaling the eventfd. It also probes for IOMMUFD vs legacy VFIO availability.

Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0 -0
index 9f8895b898214..6888be54ee4a9 100644

[ ... ]

Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2 -0
@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)
 	return NULL;
 }
 
+static int vfio_setup_msi(struct vfio_pci_device *device)
+{

tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72

Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10 -0
+	const int flags = MAP_SHARED | MAP_ANONYMOUS;
+	const int prot = PROT_READ | PROT_WRITE;
+	struct dma_region *region;
+
+	/* A driver is required to generate an MSI. */
+	TEST_REQUIRE(device->driver.ops);
+
+	/* Set up a DMA-able region for the driver to use. */
+	region = &device->driver.region;
+	region->iova = 0;

[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?

If IOVA 0 is unsupported, it seems like iommu_map() will fail and trigger a test assertion crash instead of running.

Quoted 8 lines
tools/testing/selftests/kvm/irq_test.c +8 -0
+	region->size = SZ_2M;
+	region->vaddr = kvm_mmap(region->size, prot, flags, -1);
+	TEST_ASSERT(region->vaddr != MAP_FAILED, "mmap() failed\n");
+	iommu_map(device->iommu, region);
+
+	vfio_pci_driver_init(device);
+	return device->driver.msi;
+}

[ ... ]

Quoted 5 lines
tools/testing/selftests/kvm/irq_test.c +1 -0
@@ -74,11 +107,29 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 	vm_ioctl(vm, KVM_SET_GSI_ROUTING, &routing.header);
 }
 
+static const char *probe_iommu_type(void)

tools/testing/selftests/kvm/irq_test.c static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu, · line 120

Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10 -0
+{
+	int io_fd;
+
+	io_fd = open("/dev/iommu", O_RDONLY);
+	if (io_fd >= 0) {
+		close(io_fd);
+		return MODE_IOMMUFD;
+	}
+
+	io_fd = __open_path_or_exit("/dev/vfio", O_RDONLY,

[Severity: Medium]
Because /dev/vfio is a directory, will this open() call succeed for any user with read and execute permissions to the directory, bypassing the intended skip behavior?

Should this probe the /dev/vfio/vfio legacy container character device instead so the test can skip gracefully for unprivileged users rather than crashing later in iommu_init() due to lack of device permissions?

Quoted 5 lines
tools/testing/selftests/kvm/irq_test.c +5 -0
+				    "Is VFIO (or IOMMUFD) loaded and enabled?");
+	close(io_fd);
+	return MODE_VFIO_TYPE1_IOMMU;
+}
+
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
#14 lore raw

Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test

Quoted 43 lines · On Fri, Jun 12, 2026 at 5:35 PM <[email protected]> wrote:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] probe_iommu_type() incorrectly probes the /dev/vfio directory instead of the /dev/vfio/vfio legacy container character device, bypassing the framework's intended skip behavior for unprivileged users.
- [Medium] vfio_setup_msi() hardcodes the device's DMA region IOVA to 0, which can cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0.
--

commit 2bdd8160743775c8d18e4004e72e7ff0898fbd70
Author: David Matlack <[email protected]>

KVM: selftests: Add VFIO device support to eventfd IRQ test

This commit extends the eventfd IRQ test to support triggering interrupts using a VFIO device instead of manually signaling the eventfd. It also probes for IOMMUFD vs legacy VFIO availability.

Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0 -0
index 9f8895b898214..6888be54ee4a9 100644

[ ... ]

Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2 -0
@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)
      return NULL;
 }
 
+static int vfio_setup_msi(struct vfio_pci_device *device)
+{

tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72

Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10 -0
+     const int flags = MAP_SHARED | MAP_ANONYMOUS;
+     const int prot = PROT_READ | PROT_WRITE;
+     struct dma_region *region;
+
+     /* A driver is required to generate an MSI. */
+     TEST_REQUIRE(device->driver.ops);
+
+     /* Set up a DMA-able region for the driver to use. */
+     region = &device->driver.region;
+     region->iova = 0;

[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?

Quoted 2 lines

If IOVA 0 is unsupported, it seems like iommu_map() will fail and trigger a test assertion crash instead of running.

This is probably fine for now given the test is only for x86_64. But if you are going to send a v8 you can use the VFIO selftests iova_allocator to fix this, instead of hard-coding.

Quoted 31 lines
Quoted 9 lines
tools/testing/selftests/kvm/irq_test.c +8 -0
+     region->size = SZ_2M;
+     region->vaddr = kvm_mmap(region->size, prot, flags, -1);
+     TEST_ASSERT(region->vaddr != MAP_FAILED, "mmap() failed\n");
+     iommu_map(device->iommu, region);
+
+     vfio_pci_driver_init(device);
+     return device->driver.msi;
+}

[ ... ]

Quoted 5 lines
tools/testing/selftests/kvm/irq_test.c +1 -0
@@ -74,11 +107,29 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
      vm_ioctl(vm, KVM_SET_GSI_ROUTING, &routing.header);
 }
 
+static const char *probe_iommu_type(void)

tools/testing/selftests/kvm/irq_test.c static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu, · line 120

Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10 -0
+{
+     int io_fd;
+
+     io_fd = open("/dev/iommu", O_RDONLY);
+     if (io_fd >= 0) {
+             close(io_fd);
+             return MODE_IOMMUFD;
+     }
+
+     io_fd = __open_path_or_exit("/dev/vfio", O_RDONLY,

[Severity: Medium]
Because /dev/vfio is a directory, will this open() call succeed for any user with read and execute permissions to the directory, bypassing the intended skip behavior?

Quoted 3 lines

Should this probe the /dev/vfio/vfio legacy container character device instead so the test can skip gracefully for unprivileged users rather than crashing later in iommu_init() due to lack of device permissions?

I agree with Sashiko here.

Quoted 10 lines

tools/testing/selftests/kvm/irq_test.c static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu, · line 125

Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +5 -0
+                                 "Is VFIO (or IOMMUFD) loaded and enabled?");
+     close(io_fd);
+     return MODE_VFIO_TYPE1_IOMMU;
+}
+

--
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9

author maintainer
#15 lore raw

Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test

Quoted 27 lines · David Matlack
Quoted 26 lines · On Fri, Jun 12, 2026 at 5:35 PM <[email protected]> wrote:
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0 -0
index 9f8895b898214..6888be54ee4a9 100644

[ ... ]

Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2 -0
@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)
      return NULL;
 }
 
+static int vfio_setup_msi(struct vfio_pci_device *device)
+{

tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72

Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10 -0
+     const int flags = MAP_SHARED | MAP_ANONYMOUS;
+     const int prot = PROT_READ | PROT_WRITE;
+     struct dma_region *region;
+
+     /* A driver is required to generate an MSI. */
+     TEST_REQUIRE(device->driver.ops);
+
+     /* Set up a DMA-able region for the driver to use. */
+     region = &device->driver.region;
+     region->iova = 0;

[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?

Quoted 6 lines
Quoted 2 lines

If IOVA 0 is unsupported, it seems like iommu_map() will fail and trigger a test assertion crash instead of running.

This is probably fine for now given the test is only for x86_64. But if you are going to send a v8 you can use the VFIO selftests iova_allocator to fix this, instead of hard-coding.

I'm confused, why is this test setting up a DMA-able region in the first place?

#16 lore raw

Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test

Quoted 29 lines · Sean Christopherson
Quoted 27 lines · David Matlack
Quoted 26 lines · On Fri, Jun 12, 2026 at 5:35 PM <[email protected]> wrote:
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0 -0
index 9f8895b898214..6888be54ee4a9 100644

[ ... ]

Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2 -0
@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)
      return NULL;
 }
 
+static int vfio_setup_msi(struct vfio_pci_device *device)
+{

tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72

Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10 -0
+     const int flags = MAP_SHARED | MAP_ANONYMOUS;
+     const int prot = PROT_READ | PROT_WRITE;
+     struct dma_region *region;
+
+     /* A driver is required to generate an MSI. */
+     TEST_REQUIRE(device->driver.ops);
+
+     /* Set up a DMA-able region for the driver to use. */
+     region = &device->driver.region;
+     region->iova = 0;

[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

If IOVA 0 is unsupported, it seems like iommu_map() will fail and trigger a test assertion crash instead of running.

This is probably fine for now given the test is only for x86_64. But if you are going to send a v8 you can use the VFIO selftests iova_allocator to fix this, instead of hard-coding.

I'm confused, why is this test setting up a DMA-able region in the first place?

There might be some devices with which you can trigger an MSI with pure MMIO. But most devices you need to at least set up some descriptors in memory that describe what action you want the device to take and then use MMIO to poke the device to read the desriptor and perform the action. Reading those descriptors is DMA.

For Intel DSA specifically, send_msi() works by telling the device to perform an 8-byte memcpy and send an interrupt when it's done. The driver region contains the descriptor for that memcpy, as well as the 8-byte source/destination buffers the device should use.

author maintainer
#17 lore raw

Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test

Quoted 45 lines · David Matlack
Quoted 29 lines · Sean Christopherson
Quoted 27 lines · David Matlack
Quoted 26 lines · On Fri, Jun 12, 2026 at 5:35 PM <[email protected]> wrote:
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0 -0
index 9f8895b898214..6888be54ee4a9 100644

[ ... ]

Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2 -0
@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)
      return NULL;
 }
 
+static int vfio_setup_msi(struct vfio_pci_device *device)
+{

tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72

Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10 -0
+     const int flags = MAP_SHARED | MAP_ANONYMOUS;
+     const int prot = PROT_READ | PROT_WRITE;
+     struct dma_region *region;
+
+     /* A driver is required to generate an MSI. */
+     TEST_REQUIRE(device->driver.ops);
+
+     /* Set up a DMA-able region for the driver to use. */
+     region = &device->driver.region;
+     region->iova = 0;

[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

If IOVA 0 is unsupported, it seems like iommu_map() will fail and trigger a test assertion crash instead of running.

This is probably fine for now given the test is only for x86_64. But if you are going to send a v8 you can use the VFIO selftests iova_allocator to fix this, instead of hard-coding.

I'm confused, why is this test setting up a DMA-able region in the first place?

There might be some devices with which you can trigger an MSI with pure MMIO. But most devices you need to at least set up some descriptors in memory that describe what action you want the device to take and then use MMIO to poke the device to read the desriptor and perform the action. Reading those descriptors is DMA.

Quoted 4 lines

For Intel DSA specifically, send_msi() works by telling the device to perform an 8-byte memcpy and send an interrupt when it's done. The driver region contains the descriptor for that memcpy, as well as the 8-byte source/destination buffers the device should use.

Ah, gotcha. For a comment, how's this look?

tools/testing/selftests/kvm/irq_test.c +6 -1
index 6888be54ee4a..9a495fdb52f5 100644
@@ -67,7 +67,12 @@ static int vfio_setup_msi(struct vfio_pci_device *device)
        /* A driver is required to generate an MSI. */
        TEST_REQUIRE(device->driver.ops);
 
-       /* Set up a DMA-able region for the driver to use. */
+       /*
+        * Set up a DMA-able region for the driver to use.   Very few devices
+        * provide a way to arbitrarily send interrupts (MSIs), e.g. by writing
+        * an MMIO register.  Instead, most devices send MSIs when an action is
+        * completed, and practically all actions involve DMA of some form.
+        */
        region = &device->driver.region;
        region->iova = 0;
        region->size = SZ_2M;
#18 lore raw

Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test

Quoted 47 lines · Sean Christopherson
Quoted 45 lines · David Matlack
Quoted 29 lines · Sean Christopherson
Quoted 27 lines · David Matlack
Quoted 26 lines · On Fri, Jun 12, 2026 at 5:35 PM <[email protected]> wrote:
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0 -0
index 9f8895b898214..6888be54ee4a9 100644

[ ... ]

Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2 -0
@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)
      return NULL;
 }
 
+static int vfio_setup_msi(struct vfio_pci_device *device)
+{

tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72

Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10 -0
+     const int flags = MAP_SHARED | MAP_ANONYMOUS;
+     const int prot = PROT_READ | PROT_WRITE;
+     struct dma_region *region;
+
+     /* A driver is required to generate an MSI. */
+     TEST_REQUIRE(device->driver.ops);
+
+     /* Set up a DMA-able region for the driver to use. */
+     region = &device->driver.region;
+     region->iova = 0;

[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

If IOVA 0 is unsupported, it seems like iommu_map() will fail and trigger a test assertion crash instead of running.

This is probably fine for now given the test is only for x86_64. But if you are going to send a v8 you can use the VFIO selftests iova_allocator to fix this, instead of hard-coding.

I'm confused, why is this test setting up a DMA-able region in the first place?

There might be some devices with which you can trigger an MSI with pure MMIO. But most devices you need to at least set up some descriptors in memory that describe what action you want the device to take and then use MMIO to poke the device to read the desriptor and perform the action. Reading those descriptors is DMA.

Quoted 6 lines
Quoted 4 lines

For Intel DSA specifically, send_msi() works by telling the device to perform an 8-byte memcpy and send an interrupt when it's done. The driver region contains the descriptor for that memcpy, as well as the 8-byte source/destination buffers the device should use.

Ah, gotcha. For a comment, how's this look?

That looks great!

Quoted 19 lines
tools/testing/selftests/kvm/irq_test.c +6 -1
index 6888be54ee4a..9a495fdb52f5 100644
@@ -67,7 +67,12 @@ static int vfio_setup_msi(struct vfio_pci_device *device)
        /* A driver is required to generate an MSI. */
        TEST_REQUIRE(device->driver.ops);
 
-       /* Set up a DMA-able region for the driver to use. */
+       /*
+        * Set up a DMA-able region for the driver to use.   Very few devices
+        * provide a way to arbitrarily send interrupts (MSIs), e.g. by writing
+        * an MMIO register.  Instead, most devices send MSIs when an action is
+        * completed, and practically all actions involve DMA of some form.
+        */
        region = &device->driver.region;
        region->iova = 0;
        region->size = SZ_2M;
#19 lore raw

Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test

In reply to Sean Christopherson
Quoted 22 lines · Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-d' argument that takes a BDF (in the format segment:bus:device.function) of an interrupt-capable PCI(e) device bound to VFIO, and use said device to trigger interrupts instead of always synthesizing interrupts via direct writes to the eventfd.

Using a VFIO device to trigger interrupts validates the end-to-end delivery of IRQs for "real" devices, and when supported by hardware (and KVM), also validates interrupt delivery via IRQ bypass, i.e. via device posted IRQs.

Now that IOMMUFD is a thing, auto-probe IOMMUFD vs. "legacy" VFIO by temporarily opening /dev/iommufd, and skip the test if neither IOMMUFD nor legacy VFIO is available. Add a '-t' option to the user override the probe logic, e.g. in case IOMMUFD is available but the system is configured for legacy usage.

Note, the device must have a VFIO selftest driver in order to work with the test. A helper script to list supported devices will hopefully be available in the near future at tools/testing/selftests/vfio/scripts/list_supported_devices.sh[1].

tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 68

Quoted 8 lines
tools/testing/selftests/kvm/irq_test.c +8 -0
+static int vfio_setup_msi(struct vfio_pci_device *device)
+{
+	const int flags = MAP_SHARED | MAP_ANONYMOUS;
+	const int prot = PROT_READ | PROT_WRITE;
+	struct dma_region *region;
+
+	/* A driver is required to generate an MSI. */
+	TEST_REQUIRE(device->driver.ops);

This series is probably going to race with merging another VFIO selftests patch to make send_msi() optional [1]. Can you add a check for that here?

  TEST_REQUIRE(device->driver.ops);
  TEST_REQUIRE(device->driver.ops->send_msi);

[1] https://lore.kernel.org/kvm/[email protected]/

author maintainer
#20 lore raw

Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test

Quoted 36 lines · David Matlack
Quoted 22 lines · Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-d' argument that takes a BDF (in the format segment:bus:device.function) of an interrupt-capable PCI(e) device bound to VFIO, and use said device to trigger interrupts instead of always synthesizing interrupts via direct writes to the eventfd.

Using a VFIO device to trigger interrupts validates the end-to-end delivery of IRQs for "real" devices, and when supported by hardware (and KVM), also validates interrupt delivery via IRQ bypass, i.e. via device posted IRQs.

Now that IOMMUFD is a thing, auto-probe IOMMUFD vs. "legacy" VFIO by temporarily opening /dev/iommufd, and skip the test if neither IOMMUFD nor legacy VFIO is available. Add a '-t' option to the user override the probe logic, e.g. in case IOMMUFD is available but the system is configured for legacy usage.

Note, the device must have a VFIO selftest driver in order to work with the test. A helper script to list supported devices will hopefully be available in the near future at tools/testing/selftests/vfio/scripts/list_supported_devices.sh[1].

tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 68

Quoted 8 lines
tools/testing/selftests/kvm/irq_test.c +8 -0
+static int vfio_setup_msi(struct vfio_pci_device *device)
+{
+	const int flags = MAP_SHARED | MAP_ANONYMOUS;
+	const int prot = PROT_READ | PROT_WRITE;
+	struct dma_region *region;
+
+	/* A driver is required to generate an MSI. */
+	TEST_REQUIRE(device->driver.ops);

This series is probably going to race with merging another VFIO selftests patch to make send_msi() optional [1]. Can you add a check for that here?

Quoted 2 lines
  TEST_REQUIRE(device->driver.ops);
  TEST_REQUIRE(device->driver.ops->send_msi);

Ya, will do.

author maintainer
#21 lore raw

PATCH v7 10/20 KVM: selftests: Add a helper to set proc IRQ affinity for IRQ test

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Add a utility, proc_irq_set_smp_affinity(), to set the CPU affinity of a Linux host IRQ via the proc filesystem. Use smp_affinity_list instead of smp_affinity to avoid having to convert the single CPU to a bitmask.

The helper will be used by the eventfd IRQ test to verify delivery of IRQs when the affinity is randomized/modified.

[sean: make the utility self-contained, drop "list", massage changelog]
---
 tools/testing/selftests/kvm/include/proc_util.h |  2 ++
 tools/testing/selftests/kvm/lib/proc_util.c     | 14 ++++++++++++++
 2 files changed, 16 insertions(+)
tools/testing/selftests/kvm/include/proc_util.h +2 -0
index 704839b6d7af..d1ddc967d11d 100644
@@ -6,4 +6,6 @@
 
 unsigned int vfio_msix_to_host_irq(const char *vfio_device_bdf, int msix);
 
+void proc_irq_set_smp_affinity(unsigned int irq, int cpu);
+
 #endif /* SELFTEST_KVM_PROC_UTIL_H */
tools/testing/selftests/kvm/lib/proc_util.c +14 -0
index 84d30f055a0a..3960b3841d63 100644
@@ -38,3 +38,17 @@ unsigned int vfio_msix_to_host_irq(const char *device_bdf, int msix)
 	return (unsigned int)irq;
 }
 
+void proc_irq_set_smp_affinity(unsigned int irq, int cpu)
+{
+	char path[PATH_MAX];
+	int r, fd;
+
+	snprintf(path, sizeof(path), "/proc/irq/%u/smp_affinity_list", irq);
+	fd = open(path, O_RDWR);
+	TEST_ASSERT(fd >= 0, "Failed to open %s", path);
+
+	r = dprintf(fd, "%d\n", cpu);
+	TEST_ASSERT(r > 0, "Failed to affinitize IRQ-%u to CPU %d", irq, cpu);
+
+	kvm_close(fd);
+}
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#22 lore raw

PATCH v7 11/20 KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extent the eventfd IRQ test with a '-a' flag to randomly affinitize the device's host IRQ to different physical CPUs throughout the test. This stresses the kernel's ability to maintain correct interrupt routing and delivery even as the underlying hardware IRQ affinity is changed dynamically via /proc/<irq>/smp_affinity{,_list}.

---
 tools/testing/selftests/kvm/irq_test.c | 27 +++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/irq_test.c +22 -5
index 6888be54ee4a..c01aa313f719 100644
@@ -12,10 +12,12 @@
 #include <unistd.h>
 #include <pthread.h>
 #include <sys/eventfd.h>
+#include <sys/sysinfo.h>
 
 static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;
 static bool guest_ready_for_irqs[KVM_MAX_VCPUS];
 static bool guest_received_irq[KVM_MAX_VCPUS];
+static bool irq_affinity;
 static bool done;
 
 #define GUEST_RECEIVED_IRQ(__vcpu)	\
@@ -125,9 +127,10 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
+	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
@@ -160,10 +163,13 @@ int main(int argc, char **argv)
 	int i, j, c, msix, eventfd;
 	struct iommu *iommu;
 	struct kvm_vm *vm;
-	int irq;
+	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "d:ht:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ht:")) != -1) {
 		switch (c) {
+		case 'a':
+			irq_affinity = true;
+			break;
 		case 'd':
 			device_bdf = optarg;
 			break;
@@ -192,7 +198,11 @@ int main(int argc, char **argv)
 		printf("Using device %s MSI-X[%d] (IRQ-%u)\n", device_bdf, msix,
 		       irq);
 	} else {
+		TEST_ASSERT(!irq_affinity,
+			    "Setting IRQ affinity (-a) requires a backing device (-d)");
+
 		eventfd = kvm_new_eventfd();
+		irq = -1;
 	}
 
 	pr_info("Injecting interrupts for GSI %d (guest vector 0x%x) %d times\n",
@@ -210,12 +220,19 @@ int main(int argc, char **argv)
 			continue;
 	}
 
+	irq_cpu = -1;
+
 	for (i = 0; i < nr_irqs; i++) {
 		struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];
 		struct timespec start;
 
 		kvm_route_msi(vm, gsi, vcpu, vector);
 
+		if (irq_affinity) {
+			irq_cpu = kvm_random_u64(&kvm_rng) % get_nprocs();
+			proc_irq_set_smp_affinity(irq, irq_cpu);
+		}
+
 		for (j = 0; j < nr_vcpus; j++)
 			TEST_ASSERT(!GUEST_RECEIVED_IRQ(vcpus[j]),
 				    "IRQ flag for vCPU %d not clear prior to test",
@@ -229,8 +246,8 @@ int main(int argc, char **argv)
 			cpu_relax();
 
 		TEST_ASSERT(GUEST_RECEIVED_IRQ(vcpu),
-			    "vCPU %d timed out waiting for IRQ (vector 0x%x) from GSI %d\n",
-			    vcpu->id, vector, gsi);
+			    "vCPU %d timed out waiting for IRQ (vector 0x%x) from GSI %d (via CPU %d)\n",
+			    vcpu->id, vector, gsi, irq_cpu);
 
 		WRITE_AND_SYNC_TO_GUEST(vm, guest_received_irq[vcpu->id], false);
 	}
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
#23 lore raw

Re: [PATCH v7 11/20] KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The test calculates the target host CPU for IRQ affinity using a modulo of `get_nprocs()`…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The test calculates the target host CPU for IRQ affinity using a modulo of get_nprocs(), incorrectly assuming that CPU IDs are contiguous and 0-indexed without gaps. This causes spurious test failures on sparse topologies or systems with offline CPUs. --

commit f0fe406c9bb6f2cc39673ee344b3e53be280d025
Author: David Matlack <[email protected]>

KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test

This commit extends the eventfd IRQ test with a '-a' flag to randomly change the device's host IRQ affinity to different physical CPUs during the test. This ensures the kernel maintains correct interrupt routing when hardware IRQ affinity changes dynamically.

Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0 -0
index 6888be54ee4a9..c01aa313f7198 100644

[ ... ]

Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0 -0
@@ -210,12 +220,19 @@ int main(int argc, char **argv)
 			continue;
 	}
 

tools/testing/selftests/kvm/irq_test.c int main(int argc, char **argv) · line 232

Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +4 -0
+	irq_cpu = -1;
+
 	for (i = 0; i < nr_irqs; i++) {
 		struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];
 		struct timespec start;
 
 		kvm_route_msi(vm, gsi, vcpu, vector);
 
+		if (irq_affinity) {
+			irq_cpu = kvm_random_u64(&kvm_rng) % get_nprocs();

[Severity: Medium]
Does this code assume that CPU IDs are contiguous and 0-indexed?

If get_nprocs() returns the number of currently available processors, could this modulo calculation yield an invalid or offline CPU ID on systems with sparse CPU topologies or offline CPUs?

This regression might cause the subsequent call to proc_irq_set_smp_affinity() to fail with -EINVAL and spuriously abort the test.

Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +3 -0
+			proc_irq_set_smp_affinity(irq, irq_cpu);
+		}
+
 		for (j = 0; j < nr_vcpus; j++)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
author maintainer
#24 lore raw

PATCH v7 12/20 KVM: selftests: Add option to set empty routing between IRQs in eventfd IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with an '-e' flag to set empty GSI routing between interrupts. Clobbering the GSI routing table verifies that KVM correctly handles CPUx => NULL => CPUy transitions, not just CPUx => CPUy transitions, and verifies that KVM can "rebuild" an entire routing setup.

[sean: '-e' for "empty" instead of '-c' for "clear", massage changelog]
---
 tools/testing/selftests/kvm/irq_test.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +18 -2
index c01aa313f719..2356912e2272 100644
@@ -109,6 +109,13 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 	vm_ioctl(vm, KVM_SET_GSI_ROUTING, &routing.header);
 }
 
+static void kvm_set_empty_gsi_routing(struct kvm_vm *vm)
+{
+	struct kvm_irq_routing routing = {};
+
+	vm_ioctl(vm, KVM_SET_GSI_ROUTING, &routing);
+}
+
 static const char *probe_iommu_type(void)
 {
 	int io_fd;
@@ -127,11 +134,12 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
+	printf("-e	Set empty GSI routing in-between some interrupts\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
 	exit(KSFT_FAIL);
@@ -158,6 +166,7 @@ int main(int argc, char **argv)
 	struct kvm_vcpu *vcpus[KVM_MAX_VCPUS];
 	struct vfio_pci_device *device = NULL;
 	int nr_irqs = 1000, nr_vcpus = 1;
+	bool set_empty_routing = false;
 	const char *device_bdf = NULL;
 	const char *iommu_type = NULL;
 	int i, j, c, msix, eventfd;
@@ -165,7 +174,7 @@ int main(int argc, char **argv)
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:ht:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:eht:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -173,6 +182,9 @@ int main(int argc, char **argv)
 		case 'd':
 			device_bdf = optarg;
 			break;
+		case 'e':
+			set_empty_routing = true;
+			break;
 		case 't':
 			iommu_type = optarg;
 			break;
@@ -223,9 +235,13 @@ int main(int argc, char **argv)
 	irq_cpu = -1;
 
 	for (i = 0; i < nr_irqs; i++) {
+		const bool do_set_empty_routing = set_empty_routing && (i & BIT(3));
 		struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];
 		struct timespec start;
 
+		if (do_set_empty_routing)
+			kvm_set_empty_gsi_routing(vm);
+
 		kvm_route_msi(vm, gsi, vcpu, vector);
 
 		if (irq_affinity) {
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#25 lore raw

PATCH v7 13/20 KVM: selftests: Make number of IRQs configurable in IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-i' flag to let the user specify the the number of IRQs to generate (instead of hardcoding the test to always generate 1000 interrupts).

---
 tools/testing/selftests/kvm/irq_test.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +6 -2
index 2356912e2272..2cfb6c24e8d6 100644
@@ -134,12 +134,13 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
 	printf("-e	Set empty GSI routing in-between some interrupts\n");
+	printf("-i	The number of IRQs to generate during the test\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
 	exit(KSFT_FAIL);
@@ -174,7 +175,7 @@ int main(int argc, char **argv)
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:eht:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ehi:t:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -185,6 +186,9 @@ int main(int argc, char **argv)
 		case 'e':
 			set_empty_routing = true;
 			break;
+		case 'i':
+			nr_irqs = atoi_positive("Number of IRQs", optarg);
+			break;
 		case 't':
 			iommu_type = optarg;
 			break;
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#26 lore raw

PATCH v7 14/20 KVM: selftests: Verify non-postable IRQ remapping in IRQ test

In reply to Sean Christopherson

Extend the eventfd IRQ test with an '-n' flag to route a subset of device interrupts as NMIs (Non-Maskable Interrupts) into the guest using an alternating pattern of 4 NMIs followed by 4 regular interrupts.

While this adds coverage for NMI injection, the primary goal is to validate KVM's handling of non-postable interrupt delivery (AMD and Intel IOMMUs only support posting fixed IRQs targeting a single vCPU). KVM has historically bungled handling transitions between posted and remapped modes. Use NMIs to stress the transitions, because they are a reliable, architectural way to force these code paths.

[sean: add GUEST_RECEIVED_INTERRUPT(), massage changelog]
---
 tools/testing/selftests/kvm/irq_test.c | 48 ++++++++++++++++++++------
 1 file changed, 37 insertions(+), 11 deletions(-)
tools/testing/selftests/kvm/irq_test.c +37 -11
index 2cfb6c24e8d6..d2d861119854 100644
@@ -17,11 +17,17 @@
 static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;
 static bool guest_ready_for_irqs[KVM_MAX_VCPUS];
 static bool guest_received_irq[KVM_MAX_VCPUS];
+static bool guest_received_nmi[KVM_MAX_VCPUS];
 static bool irq_affinity;
 static bool done;
 
 #define GUEST_RECEIVED_IRQ(__vcpu)	\
 	SYNC_FROM_GUEST_AND_READ((__vcpu)->vm, guest_received_irq[(__vcpu)->id])
+#define GUEST_RECEIVED_NMI(__vcpu)	\
+	SYNC_FROM_GUEST_AND_READ((__vcpu)->vm, guest_received_nmi[(__vcpu)->id])
+
+#define GUEST_RECEIVED_INTERRUPT(__vcpu, __nmi)	\
+	((__nmi) ? GUEST_RECEIVED_NMI(__vcpu) : GUEST_RECEIVED_IRQ(__vcpu))
 
 static u32 guest_get_vcpu_id(void)
 {
@@ -35,6 +41,11 @@ static void guest_irq_handler(struct ex_regs *regs)
 	x2apic_write_reg(APIC_EOI, 0);
 }
 
+static void guest_nmi_handler(struct ex_regs *regs)
+{
+	WRITE_ONCE(guest_received_nmi[guest_get_vcpu_id()], true);
+}
+
 static void guest_code(void)
 {
 	x2apic_enable();
@@ -91,7 +102,7 @@ static void trigger_interrupt(struct vfio_pci_device *device, int eventfd)
 
 
 static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
-			  u8 vector)
+			  u8 vector, bool use_nmi)
 {
 	struct {
 		struct kvm_irq_routing header;
@@ -102,7 +113,7 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 			.gsi = gsi,
 			.type = KVM_IRQ_ROUTING_MSI,
 			.u.msi.address_lo = 0xFEE00000 | (vcpu->id << 12),
-			.u.msi.data = vector,
+			.u.msi.data = use_nmi ? NMI_VECTOR | (4 << 8) : vector,
 		},
 	};
 
@@ -134,13 +145,14 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-n] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
 	printf("-e	Set empty GSI routing in-between some interrupts\n");
 	printf("-i	The number of IRQs to generate during the test\n");
+	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
 	exit(KSFT_FAIL);
@@ -171,11 +183,12 @@ int main(int argc, char **argv)
 	const char *device_bdf = NULL;
 	const char *iommu_type = NULL;
 	int i, j, c, msix, eventfd;
+	bool use_nmi = false;
 	struct iommu *iommu;
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:ehi:t:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ehi:nt:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -189,6 +202,9 @@ int main(int argc, char **argv)
 		case 'i':
 			nr_irqs = atoi_positive("Number of IRQs", optarg);
 			break;
+		case 'n':
+			use_nmi = true;
+			break;
 		case 't':
 			iommu_type = optarg;
 			break;
@@ -202,6 +218,7 @@ int main(int argc, char **argv)
 
 	vm = vm_create_with_vcpus(nr_vcpus, guest_code, vcpus);
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
+	vm_install_exception_handler(vm, NMI_VECTOR, guest_nmi_handler);
 
 	if (device_bdf) {
 		if (!iommu_type)
@@ -240,36 +257,45 @@ int main(int argc, char **argv)
 
 	for (i = 0; i < nr_irqs; i++) {
 		const bool do_set_empty_routing = set_empty_routing && (i & BIT(3));
+		const bool do_use_nmi = use_nmi && (i & BIT(2));
 		struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];
 		struct timespec start;
 
 		if (do_set_empty_routing)
 			kvm_set_empty_gsi_routing(vm);
 
-		kvm_route_msi(vm, gsi, vcpu, vector);
+		kvm_route_msi(vm, gsi, vcpu, vector, do_use_nmi);
 
 		if (irq_affinity) {
 			irq_cpu = kvm_random_u64(&kvm_rng) % get_nprocs();
 			proc_irq_set_smp_affinity(irq, irq_cpu);
 		}
 
-		for (j = 0; j < nr_vcpus; j++)
+		for (j = 0; j < nr_vcpus; j++) {
 			TEST_ASSERT(!GUEST_RECEIVED_IRQ(vcpus[j]),
 				    "IRQ flag for vCPU %d not clear prior to test",
 				    vcpus[j]->id);
+			TEST_ASSERT(!GUEST_RECEIVED_NMI(vcpus[j]),
+				    "NMI flag for vCPU %d not clear prior to test",
+				    vcpus[j]->id);
+		}
 
 		trigger_interrupt(device, eventfd);
 
 		clock_gettime(CLOCK_MONOTONIC, &start);
-		while (!GUEST_RECEIVED_IRQ(vcpu) &&
+		while (!GUEST_RECEIVED_INTERRUPT(vcpu, do_use_nmi) &&
 		       timespec_to_ns(timespec_elapsed(start)) <= timeout_ns)
 			cpu_relax();
 
-		TEST_ASSERT(GUEST_RECEIVED_IRQ(vcpu),
-			    "vCPU %d timed out waiting for IRQ (vector 0x%x) from GSI %d (via CPU %d)\n",
-			    vcpu->id, vector, gsi, irq_cpu);
+		TEST_ASSERT(GUEST_RECEIVED_INTERRUPT(vcpu, do_use_nmi),
+			    "vCPU %d timed out waiting for %s (vector 0x%x) from GSI %d (via CPU %d)\n",
+			    vcpu->id, do_use_nmi ? "NMI" : "IRQ",
+			    do_use_nmi ? NMI_VECTOR : vector, gsi, irq_cpu);
 
-		WRITE_AND_SYNC_TO_GUEST(vm, guest_received_irq[vcpu->id], false);
+		if (do_use_nmi)
+			WRITE_AND_SYNC_TO_GUEST(vm, guest_received_nmi[vcpu->id], false);
+		else
+			WRITE_AND_SYNC_TO_GUEST(vm, guest_received_irq[vcpu->id], false);
 	}
 
 	WRITE_AND_SYNC_TO_GUEST(vm, done, true);
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#27 lore raw

PATCH v7 15/20 KVM: selftests: Add kvm_gettid() wrapper and convert users

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Add a KVM wrapper for the gettid() syscall so that tests don't have to open code the syscall() themselves. Unfortunately, not all flavors of libc that KVM selftests support provide gettid(). Convert all existing users of the syscall to the new wrapper.

Note, per the gettid() manpage[1], "This call is always successful", i.e. prefixing kvm_ to the syscall name is aligned with the goal of providing syscall wrappers that guarantee success.

No functional changes intended.

---
 tools/testing/selftests/kvm/demand_paging_test.c   | 2 +-
 tools/testing/selftests/kvm/include/kvm_syscalls.h | 5 +++++
 tools/testing/selftests/kvm/lib/assert.c           | 8 ++------
 tools/testing/selftests/kvm/lib/test_util.c        | 3 ++-
 tools/testing/selftests/kvm/rseq_test.c            | 2 +-
 5 files changed, 11 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/demand_paging_test.c +1 -1
index 302c4923d093..f8b3d0b68830 100644
@@ -57,7 +57,7 @@ static void vcpu_worker(struct memstress_vcpu_args *vcpu_args)
 static int handle_uffd_page_request(int uffd_mode, int uffd,
 		struct uffd_msg *msg)
 {
-	pid_t tid = syscall(__NR_gettid);
+	pid_t tid = kvm_gettid();
 	u64 addr = msg->arg.pagefault.address;
 	struct timespec start;
 	struct timespec ts_diff;
tools/testing/selftests/kvm/include/kvm_syscalls.h +5 -0
index 6cb3bed29b81..dc4fb97aef8d 100644
@@ -83,6 +83,11 @@ static inline int kvm_dup(int fd)
 	return new_fd;
 }
 
+static inline pid_t kvm_gettid(void)
+{
+	return syscall(__NR_gettid);
+}
+
 __KVM_SYSCALL_DEFINE(munmap, 2, void *, mem, size_t, size);
 __KVM_SYSCALL_DEFINE(close, 1, int, fd);
 __KVM_SYSCALL_DEFINE(fallocate, 4, int, fd, int, mode, loff_t, offset, loff_t, len);
tools/testing/selftests/kvm/lib/assert.c +2 -6
index 8be0d09ecf0f..1d72dcdfce3b 100644
@@ -10,6 +10,7 @@
 #include <sys/syscall.h>
 
 #include "kselftest.h"
+#include "kvm_syscalls.h"
 
 #ifdef __GLIBC__
 #include <execinfo.h>
@@ -64,11 +65,6 @@ static void test_dump_stack(void)
 static void test_dump_stack(void) {}
 #endif
 
-static pid_t _gettid(void)
-{
-	return syscall(SYS_gettid);
-}
-
 void __attribute__((noinline))
 test_assert(bool exp, const char *exp_str,
 	const char *file, unsigned int line, const char *fmt, ...)
@@ -81,7 +77,7 @@ test_assert(bool exp, const char *exp_str,
 		fprintf(stderr, "==== Test Assertion Failure ====\n"
 			"  %s:%u: %s\n"
 			"  pid=%d tid=%d errno=%d - %s\n",
-			file, line, exp_str, getpid(), _gettid(),
+			file, line, exp_str, getpid(), kvm_gettid(),
 			errno, strerror(errno));
 		test_dump_stack();
 		if (fmt) {
tools/testing/selftests/kvm/lib/test_util.c +2 -1
index e208a57f190c..6b00ab11f3c0 100644
@@ -17,6 +17,7 @@
 #include "linux/kernel.h"
 
 #include "test_util.h"
+#include "kvm_syscalls.h"
 
 sigjmp_buf expect_sigbus_jmpbuf;
 
@@ -395,7 +396,7 @@ long get_run_delay(void)
 	long val[2];
 	FILE *fp;
 
-	sprintf(path, "/proc/%ld/schedstat", syscall(SYS_gettid));
+	sprintf(path, "/proc/%ld/schedstat", (long)kvm_gettid());
 	fp = fopen(path, "r");
 	/* Return MIN_RUN_DELAY_NS upon failure just to be safe */
 	if (fscanf(fp, "%ld %ld ", &val[0], &val[1]) < 2)
tools/testing/selftests/kvm/rseq_test.c +1 -1
index f80ad6b47d16..6510fbfd64f1 100644
@@ -244,7 +244,7 @@ int main(int argc, char *argv[])
 	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
 
 	pthread_create(&migration_thread, NULL, migration_worker,
-		       (void *)(unsigned long)syscall(SYS_gettid));
+		       (void *)(unsigned long)kvm_gettid());
 
 	if (latency >= 0) {
 		/*
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#28 lore raw

PATCH v7 16/20 KVM: selftests: Add kvm_sched_getaffinity() wrapper and convert users

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Add and use a KVM wrapper for the sched_getaffinity() syscall so that selftests don't need to manually assert that the syscall succeeded.

Note, some tests didn't actually assert success, but they all obviously rely on the syscall to succeed.

---
 tools/testing/selftests/kvm/arch_timer.c                  | 2 +-
 tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c | 2 +-
 tools/testing/selftests/kvm/include/kvm_syscalls.h        | 2 ++
 tools/testing/selftests/kvm/lib/kvm_util.c                | 5 ++---
 tools/testing/selftests/kvm/mmu_stress_test.c             | 6 +-----
 tools/testing/selftests/kvm/rseq_test.c                   | 4 +---
 6 files changed, 8 insertions(+), 13 deletions(-)
tools/testing/selftests/kvm/arch_timer.c +1 -1
index 90c475a61b22..f8b02597897b 100644
@@ -85,7 +85,7 @@ static u32 test_get_pcpu(void)
 	cpu_set_t online_cpuset;
 
 	nproc_conf = get_nprocs_conf();
-	sched_getaffinity(0, sizeof(cpu_set_t), &online_cpuset);
+	kvm_sched_getaffinity(0, sizeof(cpu_set_t), &online_cpuset);
 
 	/* Randomly find an available pCPU to place a vCPU on */
 	do {
tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c +1 -1
index f7625eb711d6..d9c9377a6325 100644
@@ -1039,7 +1039,7 @@ int main(int argc, char *argv[])
 	if (!parse_args(argc, argv))
 		exit(KSFT_SKIP);
 
-	sched_getaffinity(0, sizeof(default_cpuset), &default_cpuset);
+	kvm_sched_getaffinity(0, sizeof(default_cpuset), &default_cpuset);
 	set_counter_defaults();
 
 	if (test_args.test_virtual) {
tools/testing/selftests/kvm/include/kvm_syscalls.h +2 -0
index dc4fb97aef8d..5dae6143ddb0 100644
@@ -12,6 +12,7 @@
 #include <sys/mman.h>
 #include <sys/syscall.h>
 
+#include <sched.h>
 #include <test_util.h>
 
 #define MAP_ARGS0(m,...)
@@ -93,6 +94,7 @@ __KVM_SYSCALL_DEFINE(close, 1, int, fd);
 __KVM_SYSCALL_DEFINE(fallocate, 4, int, fd, int, mode, loff_t, offset, loff_t, len);
 __KVM_SYSCALL_DEFINE(ftruncate, 2, unsigned int, fd, off_t, length);
 __KVM_SYSCALL_DEFINE(madvise, 3, void *, addr, size_t, length, int, advice);
+__KVM_SYSCALL_DEFINE(sched_getaffinity, 3, pid_t, pid, size_t, cpusetsize, cpu_set_t *, mask);
 
 #define kvm_free_fd(fd)		\
 do {				\
tools/testing/selftests/kvm/lib/kvm_util.c +2 -3
index 2e08d9fcefc7..9bf28b7d9d7a 100644
@@ -673,13 +673,12 @@ void kvm_parse_vcpu_pinning(const char *pcpus_string, u32 vcpu_to_pcpu[],
 	cpu_set_t allowed_mask;
 	char *cpu, *cpu_list;
 	char delim[2] = ",";
-	int i, r;
+	int i;
 
 	cpu_list = strdup(pcpus_string);
 	TEST_ASSERT(cpu_list, "strdup() allocation failed.");
 
-	r = sched_getaffinity(0, sizeof(allowed_mask), &allowed_mask);
-	TEST_ASSERT(!r, "sched_getaffinity() failed");
+	kvm_sched_getaffinity(0, sizeof(allowed_mask), &allowed_mask);
 
 	cpu = strtok(cpu_list, delim);
 
tools/testing/selftests/kvm/mmu_stress_test.c +1 -5
index 473ef4c0ea9f..3d5f33a63b2b 100644
@@ -255,11 +255,7 @@ static void rendezvous_with_vcpus(struct timespec *time, const char *name)
 static void calc_default_nr_vcpus(void)
 {
 	cpu_set_t possible_mask;
-	int r;
-
-	r = sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
-	TEST_ASSERT(!r, "sched_getaffinity failed, errno = %d (%s)",
-		    errno, strerror(errno));
+	kvm_sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
 
 	nr_vcpus = CPU_COUNT(&possible_mask);
 	TEST_ASSERT(nr_vcpus > 0, "Uh, no CPUs?");
tools/testing/selftests/kvm/rseq_test.c +1 -3
index 6510fbfd64f1..557e393c223b 100644
@@ -226,9 +226,7 @@ int main(int argc, char *argv[])
 		}
 	}
 
-	r = sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
-	TEST_ASSERT(!r, "sched_getaffinity failed, errno = %d (%s)", errno,
-		    strerror(errno));
+	kvm_sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
 
 	calc_min_max_cpu();
 
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#29 lore raw

PATCH v7 17/20 KVM: selftests: Add a utility to pin a task to a random CPU, given a CPU set

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Add a helper function, pin_task_to_random_cpu(), to pin a task to a random CPU from a given cpu_set_t.

This helper will be used eventfd IRQ test to migrate vCPUs to random pCPUs, to stress host-side interrupt routing and delivery.

---
 .../testing/selftests/kvm/include/kvm_util.h  |  2 ++
 tools/testing/selftests/kvm/lib/kvm_util.c    | 21 +++++++++++++++++++
 2 files changed, 23 insertions(+)
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
index c1f588154398..b39e713c30a4 100644
@@ -1094,6 +1094,8 @@ static inline void pin_task_to_cpu(pthread_t task, int cpu)
 	TEST_ASSERT(!r, "Failed to set thread affinity to pCPU '%u'", cpu);
 }
 
+void pin_task_to_random_cpu(pthread_t task, cpu_set_t *possible_cpus);
+
 static inline int pin_task_to_any_cpu(pthread_t task)
 {
 	int cpu = sched_getcpu();
tools/testing/selftests/kvm/lib/kvm_util.c +21 -0
index 9bf28b7d9d7a..2eee1e46f9e9 100644
@@ -667,6 +667,27 @@ void kvm_print_vcpu_pinning_help(void)
 	       "     (default: no pinning)\n", name, name);
 }
 
+void pin_task_to_random_cpu(pthread_t task, cpu_set_t *possible_cpus)
+{
+	int target_idx;
+	int nr_cpus;
+	int cpu;
+
+	nr_cpus = CPU_COUNT(possible_cpus);
+	TEST_ASSERT(nr_cpus > 0, "No CPUs available in possible_cpus");
+
+	target_idx = kvm_random_u64(&kvm_rng) % nr_cpus;
+
+	for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+		if (CPU_ISSET(cpu, possible_cpus) && target_idx-- == 0) {
+			pin_task_to_cpu(task, cpu);
+			return;
+		}
+	}
+
+	TEST_FAIL("Failed to find random CPU in possible_cpus");
+}
+
 void kvm_parse_vcpu_pinning(const char *pcpus_string, u32 vcpu_to_pcpu[],
 			    int nr_vcpus)
 {
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#30 lore raw

PATCH v7 18/20 KVM: selftests: Verify vCPU migration during IRQ delivery in IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-m' flag to have the test migrate the target vCPU to a random physical CPU before triggering its interrupt, e.g. to validate KVM's ability to update device posted IRQ routing.

[sean: pin one vCPU at a time to simplify things, use main()'s affinity]
---
 tools/testing/selftests/kvm/irq_test.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +14 -2
index d2d861119854..5049cb69cd95 100644
@@ -145,13 +145,14 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-n] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
 	printf("-e	Set empty GSI routing in-between some interrupts\n");
 	printf("-i	The number of IRQs to generate during the test\n");
+	printf("-m	Pin target vCPU to random physical CPU before triggering interrupt\n");
 	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
@@ -183,12 +184,14 @@ int main(int argc, char **argv)
 	const char *device_bdf = NULL;
 	const char *iommu_type = NULL;
 	int i, j, c, msix, eventfd;
+	bool migrate_vcpus = false;
+	cpu_set_t available_cpus;
 	bool use_nmi = false;
 	struct iommu *iommu;
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:ehi:nt:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ehi:mnt:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -202,6 +205,9 @@ int main(int argc, char **argv)
 		case 'i':
 			nr_irqs = atoi_positive("Number of IRQs", optarg);
 			break;
+		case 'm':
+			migrate_vcpus = true;
+			break;
 		case 'n':
 			use_nmi = true;
 			break;
@@ -243,6 +249,9 @@ int main(int argc, char **argv)
 
 	kvm_assign_irqfd(vm, gsi, eventfd);
 
+	if (migrate_vcpus)
+		kvm_sched_getaffinity(0, sizeof(available_cpus), &available_cpus);
+
 	for (i = 0; i < nr_vcpus; i++)
 		pthread_create(&vcpu_threads[i], NULL, vcpu_thread_main, vcpus[i]);
 
@@ -271,6 +280,9 @@ int main(int argc, char **argv)
 			proc_irq_set_smp_affinity(irq, irq_cpu);
 		}
 
+		if (migrate_vcpus)
+			pin_task_to_random_cpu(vcpu_threads[i % nr_vcpus], &available_cpus);
+
 		for (j = 0; j < nr_vcpus; j++) {
 			TEST_ASSERT(!GUEST_RECEIVED_IRQ(vcpus[j]),
 				    "IRQ flag for vCPU %d not clear prior to test",
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#31 lore raw

PATCH v7 19/20 KVM: selftests: Make number of vCPUs configurable in IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-v' flag to allow the user to configure the number of vCPUs to create and run (versus only ever using a single vCPU).

Update the routing logic to play nice with 32 bit IDs, enable x2APIC format in KVM (to enable 32-bit ID routing), and disable KVM's x2APIC broadcast quirk so that targeting vCPU 255 doesn't blast the interrupt to all vCPUs when in x2APIC mode.

---
 tools/testing/selftests/kvm/irq_test.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/irq_test.c +13 -3
index 5049cb69cd95..dfdb9f8e398c 100644
@@ -112,7 +112,8 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 		.entry = {
 			.gsi = gsi,
 			.type = KVM_IRQ_ROUTING_MSI,
-			.u.msi.address_lo = 0xFEE00000 | (vcpu->id << 12),
+			.u.msi.address_lo = 0xFEE00000 | (vcpu->id & GENMASK(7, 0)) << 12,
+			.u.msi.address_hi = vcpu->id & GENMASK(31, 8),
 			.u.msi.data = use_nmi ? NMI_VECTOR | (4 << 8) : vector,
 		},
 	};
@@ -145,7 +146,7 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type] [-v nr_vcpus]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
@@ -155,6 +156,7 @@ static void help(const char *name)
 	printf("-m	Pin target vCPU to random physical CPU before triggering interrupt\n");
 	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
+	printf("-v	Number of vCPUS to run\n");
 	printf("\n");
 	exit(KSFT_FAIL);
 }
@@ -191,7 +193,7 @@ int main(int argc, char **argv)
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:ehi:mnt:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ehi:mnt:v:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -214,6 +216,11 @@ int main(int argc, char **argv)
 		case 't':
 			iommu_type = optarg;
 			break;
+		case 'v':
+			nr_vcpus = atoi_positive("Number of vCPUS", optarg);
+			TEST_ASSERT(nr_vcpus <= KVM_MAX_VCPUS,
+				    "KVM selftests support at most %u vCPUs", KVM_MAX_VCPUS);
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
@@ -223,6 +230,9 @@ int main(int argc, char **argv)
 	TEST_REQUIRE(kvm_arch_has_default_irqchip());
 
 	vm = vm_create_with_vcpus(nr_vcpus, guest_code, vcpus);
+	vm_enable_cap(vm, KVM_CAP_X2APIC_API, KVM_X2APIC_API_USE_32BIT_IDS |
+					      KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK);
+
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
 	vm_install_exception_handler(vm, NMI_VECTOR, guest_nmi_handler);
 
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
author maintainer
#32 lore raw

PATCH v7 20/20 KVM: selftests: Add xAPIC support in eventfd IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-x' flag to let the user run the test in xAPIC mode instead of the default x2APIC mode. When using xAPIC mode, sanity check user input to ensure the test is being run with at most 255 vCPUs, as xAPIC can only address IDs 0-254 (255, i.e. 0xff, broadcasts to all CPUs).

[sean: add sanity check on number of vCPUs]
---
 tools/testing/selftests/kvm/irq_test.c | 31 +++++++++++++++++++++-----
 1 file changed, 26 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/irq_test.c +26 -5
index dfdb9f8e398c..a17e274c9b4a 100644
@@ -18,6 +18,7 @@ static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;
 static bool guest_ready_for_irqs[KVM_MAX_VCPUS];
 static bool guest_received_irq[KVM_MAX_VCPUS];
 static bool guest_received_nmi[KVM_MAX_VCPUS];
+static bool x2apic = true;
 static bool irq_affinity;
 static bool done;
 
@@ -31,14 +32,20 @@ static bool done;
 
 static u32 guest_get_vcpu_id(void)
 {
-	return x2apic_read_reg(APIC_ID);
+	if (x2apic)
+		return x2apic_read_reg(APIC_ID);
+	else
+		return xapic_read_reg(APIC_ID) >> 24;
 }
 
 static void guest_irq_handler(struct ex_regs *regs)
 {
 	WRITE_ONCE(guest_received_irq[guest_get_vcpu_id()], true);
 
-	x2apic_write_reg(APIC_EOI, 0);
+	if (x2apic)
+		x2apic_write_reg(APIC_EOI, 0);
+	else
+		xapic_write_reg(APIC_EOI, 0);
 }
 
 static void guest_nmi_handler(struct ex_regs *regs)
@@ -48,7 +55,10 @@ static void guest_nmi_handler(struct ex_regs *regs)
 
 static void guest_code(void)
 {
-	x2apic_enable();
+	if (x2apic)
+		x2apic_enable();
+	else
+		xapic_enable();
 
 	sti_nop();
 
@@ -146,7 +156,7 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type] [-v nr_vcpus]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type] [-v nr_vcpus] [-x]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
@@ -157,6 +167,7 @@ static void help(const char *name)
 	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("-v	Number of vCPUS to run\n");
+	printf("-x	Use xAPIC mode instead of x2APIC mode in the guest\n");
 	printf("\n");
 	exit(KSFT_FAIL);
 }
@@ -193,7 +204,7 @@ int main(int argc, char **argv)
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:ehi:mnt:v:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ehi:mnt:v:x")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -221,6 +232,9 @@ int main(int argc, char **argv)
 			TEST_ASSERT(nr_vcpus <= KVM_MAX_VCPUS,
 				    "KVM selftests support at most %u vCPUs", KVM_MAX_VCPUS);
 			break;
+		case 'x':
+			x2apic = false;
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
@@ -236,6 +250,11 @@ int main(int argc, char **argv)
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
 	vm_install_exception_handler(vm, NMI_VECTOR, guest_nmi_handler);
 
+	if (!x2apic) {
+		TEST_ASSERT(nr_vcpus < 256, "xAPIC can only target IDs [0-254] (255 vCPUs)");
+		virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA);
+	}
+
 	if (device_bdf) {
 		if (!iommu_type)
 			iommu_type = probe_iommu_type();
@@ -259,6 +278,8 @@ int main(int argc, char **argv)
 
 	kvm_assign_irqfd(vm, gsi, eventfd);
 
+	sync_global_to_guest(vm, x2apic);
+
 	if (migrate_vcpus)
 		kvm_sched_getaffinity(0, sizeof(available_cpus), &available_cpus);
 
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog