Skip to contentSkip to conversation

KVM: selftests: Add eventfd+VFIO IRQ test

PATCH In review Sean Christopherson 2 participants 29 messages Jun 26 – Jul 14

Subsystems (4)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

v8 cover KVM: selftests: Add eventfd+VFIO IRQ test

David and Josh's series to add a selftest for verifying interrupt delivery via eventfd (via KVM_IRQFD), and also from a real device, wired up via VFIO.

Gory details in the patches, and in the v5 cover letter.

v8:
- Explicitly include <time.h>. [Sashiko]
- Use iova_allocator instea of hardcoding the IOVA. [Sashiko, David]
- Add a comment explaining why DMA needs to be allocated. [David]
- Probe /dev/vfio/vfio, not /dev/vfio. [Sashiko, David]

v7:
- https://lore.kernel.org/all/[email protected]
- Seed kvm_rng during kvm_selftest_init(). [Sashiko]
- Seed libc's RNG during kvm_selftest_init(), so that the seed (from random()) that's printed and passed to kvm_rng isn't the same every time.
- Init irq_cpu to -1 in all paths. [Sashiko]
- Require the driver to provide .send_msi(). [David]

v6:
- Massage most changelogs.
- Fix SoB ordering issues.
- Clean up KVM_SET_GSI_ROUTING helper.
- Remove misleading "IRQ injection" and "emulated eventfd" terminology.
- Add GUEST_RECEIVED_INTERRUPT() to simplifiy the core loop.
- Use cpu_relax() in tight loops while waiting for interrupts.
- Print as much information as possible in the actual assert, instead of printing to stdout separately.
- Make a best guess as to the right VFIO vs. IOMMUFD mode instead of assuming IOMMUFD, and give the user the option to overide said guess.
- Simplify open_proc_irq_smp_affinity_list() + write_proc_irq_smp_affinity_list() into proc_irq_set_smp_affinity().
- Drop print_proc_irq_file() and kvm_print_vcpu_affinity() (for now) to avoid potential issues on systems with high CPU counts.
- Drop the blocking/HLT testing as it was at best broken.
- Use -e for "empty", not -c for "clear", when completely tearing down GSI routing, because routing can be "cleared" without completely emptying the routing information.
- Use the main task's CPU affinity as the available_cpus set.
- Allow overcommiting vCPUs:pCPUs.
- Set the target vCPU's affinity instead of batching when vCPU0 is targeted.
- Add support for 256+ vCPUs with x2APIC.
- Restrict xAPIC mode to 255 vCPUs.
- Restrict the test to KVM selftest's max supported vCPUs.

v5:
- https://lore.kernel.org/all/[email protected]
- Rename get_proc_vfio_irq_number() to vfio_msix_to_host_irq()
- Rename open_proc_irq_affinity() and write_proc_irq_affinity() to include "_smp_affinity_list"
- Print /proc/irq/<irq>/smp_affinity and effective_affinity on timeout failures
- Convert IRQ type from 'int' to 'unsigned int' across helpers and the test
- Fix compiler warnings for uninitialized variables in irq_test.c
- Remove rate-limiting on affinity changes

v4: https://lore.kernel.org/kvm/[email protected]

David Matlack (11):
KVM: selftests: Build and link selftests/vfio/lib into KVM selftests
KVM: selftests: Add macros to read/write+sync to/from guest memory
KVM: selftests: Add an irqfd send+receive (and later IRQ bypass) test
KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ
bypass test
KVM: selftests: Add VFIO device support to eventfd IRQ test
KVM: selftests: Verify interrupts are received when IRQ affinity
changes in IRQ test
KVM: selftests: Add option to set empty routing between IRQs in
eventfd IRQ test
KVM: selftests: Make number of IRQs configurable in IRQ test
KVM: selftests: Verify vCPU migration during IRQ delivery in IRQ test
KVM: selftests: Make number of vCPUs configurable in IRQ test
KVM: selftests: Add xAPIC support in eventfd IRQ test

Josh Hilke (6):
KVM: selftests: Rename guest_rng to kvm_rng
KVM: selftests: Add helper to generate random u64 in range [min,max]
KVM: selftests: Add a helper to set proc IRQ affinity for IRQ test
KVM: selftests: Add kvm_gettid() wrapper and convert users
KVM: selftests: Add kvm_sched_getaffinity() wrapper and convert users
KVM: selftests: Add a utility to pin a task to a random CPU, given a
CPU set

Sean Christopherson (3):
KVM: selftests: Initialize the default/global pRNG during
kvm_selftest_init()
KVM: selftests: Seed libc's RNG before using it to generate a seed for
KVM's pRNG
KVM: selftests: Verify non-postable IRQ remapping in IRQ test

 tools/testing/selftests/kvm/Makefile.kvm      |   7 +-
 tools/testing/selftests/kvm/arch_timer.c      |   2 +-
 .../kvm/arm64/arch_timer_edge_cases.c         |   2 +-
 .../selftests/kvm/demand_paging_test.c        |   2 +-
 .../selftests/kvm/dirty_log_perf_test.c       |   4 +-
 tools/testing/selftests/kvm/dirty_log_test.c  |  11 +-
 .../selftests/kvm/include/kvm_syscalls.h      |   7 +
 .../testing/selftests/kvm/include/kvm_util.h  |  12 +
 .../testing/selftests/kvm/include/proc_util.h |  11 +
 .../testing/selftests/kvm/include/test_util.h |  25 +-
 .../selftests/kvm/include/x86/kvm_util_arch.h |   4 +-
 tools/testing/selftests/kvm/irq_test.c        | 362 ++++++++++++++++++
 tools/testing/selftests/kvm/lib/assert.c      |   8 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    |  54 ++-
 tools/testing/selftests/kvm/lib/memstress.c   |   8 +-
 tools/testing/selftests/kvm/lib/proc_util.c   |  54 +++
 tools/testing/selftests/kvm/lib/test_util.c   |  27 +-
 tools/testing/selftests/kvm/mmu_stress_test.c |  15 +-
 tools/testing/selftests/kvm/rseq_test.c       |   6 +-
 tools/testing/selftests/kvm/steal_time.c      |  22 +-
 .../testing/selftests/kvm/x86/sev_dbg_test.c  |   2 +-
 21 files changed, 561 insertions(+), 84 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/include/proc_util.h
 create mode 100644 tools/testing/selftests/kvm/irq_test.c
 create mode 100644 tools/testing/selftests/kvm/lib/proc_util.c

base-commit: a204badd8432f93b7e862e7dac6db0fe3d65f370

Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#1 lore raw

PATCH v8 1/20 KVM: selftests: Build and link selftests/vfio/lib into KVM selftests

From: David Matlack <[email protected]>

Include libvfio.mk into the KVM selftests Makefile and link it into all KVM selftests by adding it to LIBKVM_OBJS.

This lays the groundwork for future changes to utilize VFIO devices to verify IRQ bypass in KVM selftests.

Note that KVM selftests build their own copy of selftests/vfio/lib and the resulting object files are placed in $(OUTPUT)/lib. This allows the KVM and VFIO selftests to apply different CFLAGS when building without conflicting with each other.

---
 tools/testing/selftests/kvm/Makefile.kvm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/Makefile.kvm +4 -1
index 4ace12606e93..ae0dfd67dd31 100644
@@ -259,6 +259,7 @@ OVERRIDE_TARGETS = 1
 # which causes the environment variable to override the makefile).
 include ../lib.mk
 include ../cgroup/lib/libcgroup.mk
+include ../vfio/lib/libvfio.mk
 
 INSTALL_HDR_PATH = $(top_srcdir)/usr
 LINUX_HDR_PATH = $(INSTALL_HDR_PATH)/include/
@@ -313,7 +314,9 @@ LIBKVM_S := $(filter %.S,$(LIBKVM))
 LIBKVM_C_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_C))
 LIBKVM_S_OBJ := $(patsubst %.S, $(OUTPUT)/%.o, $(LIBKVM_S))
 LIBKVM_STRING_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_STRING))
-LIBKVM_OBJS = $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ) $(LIBKVM_STRING_OBJ) $(LIBCGROUP_O)
+LIBKVM_OBJS = $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ) $(LIBKVM_STRING_OBJ)
+LIBKVM_OBJS += $(LIBCGROUP_O)
+LIBKVM_OBJS += $(LIBVFIO_O)
 SPLIT_TEST_GEN_PROGS := $(patsubst %, $(OUTPUT)/%, $(SPLIT_TESTS))
 SPLIT_TEST_GEN_OBJ := $(patsubst %, $(OUTPUT)/$(ARCH)/%.o, $(SPLIT_TESTS))
 
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#2 lore raw

Re: [PATCH v8 01/20] KVM: selftests: Build and link selftests/vfio/lib into KVM selftests

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] The use of `dirname` with multiple arguments in `libvfio.mk` is a GNU-ism that breaks the build on…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The use of dirname with multiple arguments in libvfio.mk is a GNU-ism that breaks the build on POSIX/BSD systems. --

commit 070d3095b74eed8d9c8207b3141eb0bb2449cb39
Author: David Matlack <[email protected]>

KVM: selftests: Build and link selftests/vfio/lib into KVM selftests

This commit includes libvfio.mk into the KVM selftests Makefile and links it into all KVM selftests by adding it to LIBKVM_OBJS. This lays the groundwork for future changes to utilize VFIO devices to verify IRQ bypass in KVM selftests.

tools/testing/selftests/kvm/Makefile.kvm OVERRIDE_TARGETS = 1 · line 262

Quoted 9 lines
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index 4ace12606e937..ae0dfd67dd31e 100644
@@ -259,6 +259,7 @@ OVERRIDE_TARGETS = 1
 # which causes the environment variable to override the makefile).
 include ../lib.mk
 include ../cgroup/lib/libcgroup.mk
+include ../vfio/lib/libvfio.mk

[Severity: Low]
Does including libvfio.mk here break the build for KVM selftests on environments with a non-GNU dirname?

Looking at tools/testing/selftests/vfio/lib/libvfio.mk, it contains:

    LIBVFIO_O_DIRS := $(shell dirname $(LIBVFIO_O) | uniq)

Passing multiple arguments to dirname is a GNU extension. On POSIX or BSD systems, this command fails and returns an empty string. The subsequent mkdir command then fails to create the required output directories, which causes the compiler to fail with a "No such file or directory" error when attempting to write the libvfio object files.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author maintainer
#3 lore raw

PATCH v8 2/20 KVM: selftests: Add macros to read/write+sync to/from guest memory

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Add SYNC_FROM_GUEST_AND_READ(vm, variable), to read a variable value from the guest. Add WRITE_AND_SYNC_TO_GUEST(vm, variable, value) to write a value to a guest variable. These macros improve the readability of code which reads and writes data between host and guest in tests.

Use the new macro in existing tests that do back-to-back write+sync.

No functional changes are intended.

---
 tools/testing/selftests/kvm/dirty_log_test.c  |  9 +++-----
 .../testing/selftests/kvm/include/kvm_util.h  | 10 +++++++++
 tools/testing/selftests/kvm/mmu_stress_test.c |  9 +++-----
 tools/testing/selftests/kvm/steal_time.c      | 22 +++++++------------
 4 files changed, 24 insertions(+), 26 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +3 -6
index 74ca096bf976..087e94a8a81a 100644
@@ -708,8 +708,7 @@ static void run_test(enum vm_guest_mode mode, void *arg)
 
 		sync_global_to_guest(vm, iteration);
 
-		WRITE_ONCE(nr_writes, 0);
-		sync_global_to_guest(vm, nr_writes);
+		WRITE_AND_SYNC_TO_GUEST(vm, nr_writes, 0);
 
 		dirty_ring_prev_iteration_last_page = dirty_ring_last_page;
 		WRITE_ONCE(dirty_ring_vcpu_ring_full, false);
@@ -775,16 +774,14 @@ static void run_test(enum vm_guest_mode mode, void *arg)
 		 * writing memory during verification, pages that this thread
 		 * sees as clean may be written with this iteration's value.
 		 */
-		WRITE_ONCE(vcpu_stop, true);
-		sync_global_to_guest(vm, vcpu_stop);
+		WRITE_AND_SYNC_TO_GUEST(vm, vcpu_stop, true);
 		sem_wait(&sem_vcpu_stop);
 
 		/*
 		 * Clear vcpu_stop after the vCPU thread has acknowledge the
 		 * stop request and is waiting, i.e. is definitely not running!
 		 */
-		WRITE_ONCE(vcpu_stop, false);
-		sync_global_to_guest(vm, vcpu_stop);
+		WRITE_AND_SYNC_TO_GUEST(vm, vcpu_stop, false);
 
 		/*
 		 * Sync the number of writes performed before verification, the
tools/testing/selftests/kvm/include/kvm_util.h +10 -0
index 04a910164a29..c1f588154398 100644
@@ -1138,6 +1138,16 @@ vm_adjust_num_guest_pages(enum vm_guest_mode mode, unsigned int num_guest_pages)
 	memcpy(&(g), _p, sizeof(g));				\
 })
 
+#define SYNC_FROM_GUEST_AND_READ(_vm, _variable) ({		\
+	sync_global_from_guest(_vm, _variable);			\
+	READ_ONCE(_variable);					\
+})
+
+#define WRITE_AND_SYNC_TO_GUEST(_vm, _variable, _value) do {	\
+	WRITE_ONCE(_variable, _value);				\
+	sync_global_to_guest(_vm, _variable);			\
+} while (0)
+
 /*
  * Write a global value, but only in the VM's (guest's) domain.  Primarily used
  * for "globals" that hold per-VM values (VMs always duplicate code and global
tools/testing/selftests/kvm/mmu_stress_test.c +3 -6
index 54d281419d31..473ef4c0ea9f 100644
@@ -155,10 +155,8 @@ static void *vcpu_worker(void *data)
 		    "Expected EFAULT on write to RO memory, got r = %d, errno = %d", r, errno);
 
 	atomic_inc(&nr_ro_faults);
-	if (atomic_read(&nr_ro_faults) == nr_vcpus) {
-		WRITE_ONCE(all_vcpus_hit_ro_fault, true);
-		sync_global_to_guest(vm, all_vcpus_hit_ro_fault);
-	}
+	if (atomic_read(&nr_ro_faults) == nr_vcpus)
+		WRITE_AND_SYNC_TO_GUEST(vm, all_vcpus_hit_ro_fault, true);
 
 #if defined(__x86_64__) || defined(__aarch64__)
 	/*
@@ -383,8 +381,7 @@ int main(int argc, char *argv[])
 	rendezvous_with_vcpus(&time_run2, "run 2");
 
 	mprotect(mem, slot_size, PROT_READ);
-	mprotect_ro_done = true;
-	sync_global_to_guest(vm, mprotect_ro_done);
+	WRITE_AND_SYNC_TO_GUEST(vm, mprotect_ro_done, true);
 
 	rendezvous_with_vcpus(&time_ro, "mprotect RO");
 	mprotect(mem, slot_size, PROT_READ | PROT_WRITE);
tools/testing/selftests/kvm/steal_time.c +8 -14
index 76fcdd1fd3cb..2de87549fcc0 100644
@@ -70,8 +70,8 @@ static bool is_steal_time_supported(struct kvm_vcpu *vcpu)
 static void steal_time_init(struct kvm_vcpu *vcpu, u32 i)
 {
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	sync_global_to_guest(vcpu->vm, st_gva[i]);
+	WRITE_AND_SYNC_TO_GUEST(vcpu->vm, st_gva[i],
+				(void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
 
 	vcpu_set_msr(vcpu, MSR_KVM_STEAL_TIME, (ulong)st_gva[i] | KVM_MSR_ENABLED);
 }
@@ -187,8 +187,7 @@ static void steal_time_init(struct kvm_vcpu *vcpu, u32 i)
 	};
 
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	sync_global_to_guest(vm, st_gva[i]);
+	WRITE_AND_SYNC_TO_GUEST(vm, st_gva[i], (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
 
 	st_ipa = (ulong)st_gva[i];
 	vcpu_ioctl(vcpu, KVM_SET_DEVICE_ATTR, &dev);
@@ -310,10 +309,8 @@ static bool is_steal_time_supported(struct kvm_vcpu *vcpu)
 static void steal_time_init(struct kvm_vcpu *vcpu, u32 i)
 {
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	st_gpa[i] = addr_gva2gpa(vcpu->vm, (gva_t)st_gva[i]);
-	sync_global_to_guest(vcpu->vm, st_gva[i]);
-	sync_global_to_guest(vcpu->vm, st_gpa[i]);
+	WRITE_AND_SYNC_TO_GUEST(vcpu->vm, st_gva[i], (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
+	WRITE_AND_SYNC_TO_GUEST(vcpu->vm, st_gpa[i], addr_gva2gpa(vcpu->vm, (gva_t)st_gva[i]));
 }
 
 static void steal_time_dump(struct kvm_vm *vm, u32 vcpu_idx)
@@ -442,8 +439,7 @@ static void steal_time_init(struct kvm_vcpu *vcpu, u32 i)
 	};
 
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	sync_global_to_guest(vm, st_gva[i]);
+	WRITE_AND_SYNC_TO_GUEST(vm, st_gva[i], (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
 
 	err = __vcpu_ioctl(vcpu, KVM_HAS_DEVICE_ATTR, &attr);
 	TEST_ASSERT(err == 0, "No PV stealtime Feature");
@@ -549,8 +545,7 @@ int main(int ac, char **av)
 
 		/* Second VCPU run, expect guest stolen time to be <= run_delay */
 		run_vcpu(vcpus[i]);
-		sync_global_from_guest(vm, guest_stolen_time[i]);
-		stolen_time = guest_stolen_time[i];
+		stolen_time = SYNC_FROM_GUEST_AND_READ(vm, guest_stolen_time[i]);
 		run_delay = get_run_delay();
 		TEST_ASSERT(stolen_time <= run_delay,
 			    "Expected stolen time <= %ld, got %ld",
@@ -570,8 +565,7 @@ int main(int ac, char **av)
 
 		/* Run VCPU again to confirm stolen time is consistent with run_delay */
 		run_vcpu(vcpus[i]);
-		sync_global_from_guest(vm, guest_stolen_time[i]);
-		stolen_time = guest_stolen_time[i] - stolen_time;
+		stolen_time = SYNC_FROM_GUEST_AND_READ(vm, guest_stolen_time[i]) - stolen_time;
 		TEST_ASSERT(stolen_time >= run_delay,
 			    "Expected stolen time >= %ld, got %ld",
 			    run_delay, stolen_time);
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#4 lore raw

PATCH v8 3/20 KVM: selftests: Rename guest_rng to kvm_rng

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Rename functions prefixed with 'guest_random_' to 'kvm_random_' and the global random state variable 'guest_rng' to 'kvm_rng', as the pRNG isn't strictly limited to guest code. This will allow using the pRNG in host code without creating confusing/misleading function calls.

No functional changes are intended.

---
 .../selftests/kvm/dirty_log_perf_test.c       |  4 ++--
 tools/testing/selftests/kvm/dirty_log_test.c  |  2 +-
 .../testing/selftests/kvm/include/test_util.h | 22 +++++++++----------
 .../selftests/kvm/include/x86/kvm_util_arch.h |  4 ++--
 tools/testing/selftests/kvm/lib/kvm_util.c    | 20 ++++++++---------
 tools/testing/selftests/kvm/lib/memstress.c   |  8 +++----
 tools/testing/selftests/kvm/lib/test_util.c   |  6 ++---
 .../testing/selftests/kvm/x86/sev_dbg_test.c  |  2 +-
 8 files changed, 34 insertions(+), 34 deletions(-)
tools/testing/selftests/kvm/dirty_log_perf_test.c +2 -2
index ef779fa91827..7c5abe1ae9e0 100644
@@ -311,7 +311,7 @@ int main(int argc, char *argv[])
 	int opt;
 
 	/* Override the seed to be deterministic by default. */
-	guest_random_seed = 1;
+	kvm_random_seed = 1;
 
 	dirty_log_manual_caps =
 		kvm_check_cap(KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2);
@@ -357,7 +357,7 @@ int main(int argc, char *argv[])
 			p.phys_offset = strtoull(optarg, NULL, 0);
 			break;
 		case 'r':
-			guest_random_seed = atoi_positive("Random seed", optarg);
+			kvm_random_seed = atoi_positive("Random seed", optarg);
 			break;
 		case 's':
 			p.backing_src = parse_backing_src_type(optarg);
tools/testing/selftests/kvm/dirty_log_test.c +1 -1
index 087e94a8a81a..e8419d7da1ea 100644
@@ -121,7 +121,7 @@ static void guest_code(void)
 	while (true) {
 		while (!READ_ONCE(vcpu_stop)) {
 			addr = guest_test_virt_mem;
-			addr += (guest_random_u64(&guest_rng) % guest_num_pages)
+			addr += (kvm_random_u64(&kvm_rng) % guest_num_pages)
 				* guest_page_size;
 			addr = align_down(addr, host_page_size);
 
tools/testing/selftests/kvm/include/test_util.h +11 -11
index a56271c237ae..44c0104d60ac 100644
@@ -108,30 +108,30 @@ struct timespec timespec_sub(struct timespec ts1, struct timespec ts2);
 struct timespec timespec_elapsed(struct timespec start);
 struct timespec timespec_div(struct timespec ts, int divisor);
 
-struct guest_random_state {
+struct kvm_random_state {
 	u32 seed;
 };
 
-extern u32 guest_random_seed;
-extern struct guest_random_state guest_rng;
+extern u32 kvm_random_seed;
+extern struct kvm_random_state kvm_rng;
 
-struct guest_random_state new_guest_random_state(u32 seed);
-u32 guest_random_u32(struct guest_random_state *state);
+struct kvm_random_state new_kvm_random_state(u32 seed);
+u32 kvm_random_u32(struct kvm_random_state *state);
 
-static inline bool __guest_random_bool(struct guest_random_state *state,
+static inline bool __kvm_random_bool(struct kvm_random_state *state,
 				       u8 percent)
 {
-	return (guest_random_u32(state) % 100) < percent;
+	return (kvm_random_u32(state) % 100) < percent;
 }
 
-static inline bool guest_random_bool(struct guest_random_state *state)
+static inline bool kvm_random_bool(struct kvm_random_state *state)
 {
-	return __guest_random_bool(state, 50);
+	return __kvm_random_bool(state, 50);
 }
 
-static inline u64 guest_random_u64(struct guest_random_state *state)
+static inline u64 kvm_random_u64(struct kvm_random_state *state)
 {
-	return ((u64)guest_random_u32(state) << 32) | guest_random_u32(state);
+	return ((u64)kvm_random_u32(state) << 32) | kvm_random_u32(state);
 }
 
 enum vm_mem_backing_src_type {
tools/testing/selftests/kvm/include/x86/kvm_util_arch.h +2 -2
index c33ab6e04171..6904dbda79f9 100644
@@ -55,9 +55,9 @@ static inline bool __vm_arch_has_protected_memory(struct kvm_vm_arch *arch)
 do {											\
 	const typeof(mem) val = (__val);						\
 											\
-	if (!is_forced_emulation_enabled || guest_random_bool(&guest_rng)) {		\
+	if (!is_forced_emulation_enabled || kvm_random_bool(&kvm_rng)) {		\
 		(mem) = val;								\
-	} else if (guest_random_bool(&guest_rng)) {					\
+	} else if (kvm_random_bool(&kvm_rng)) {					\
 		__asm__ __volatile__(KVM_FEP "mov %1, %0"				\
 				     : "+m" (mem)					\
 				     : "r" (val) : "memory");				\
tools/testing/selftests/kvm/lib/kvm_util.c +10 -10
index 195f3fdae1e3..875030c22d07 100644
@@ -20,9 +20,9 @@
 
 #define KVM_UTIL_MIN_PFN	2
 
-u32 guest_random_seed;
-struct guest_random_state guest_rng;
-static u32 last_guest_seed;
+u32 kvm_random_seed;
+struct kvm_random_state kvm_rng;
+static u32 last_kvm_seed;
 
 static size_t vcpu_mmap_sz(void);
 
@@ -515,12 +515,12 @@ struct kvm_vm *__vm_create(struct vm_shape shape, u32 nr_runnable_vcpus,
 	slot0 = memslot2region(vm, 0);
 	ucall_init(vm, slot0->region.guest_phys_addr + slot0->region.memory_size);
 
-	if (guest_random_seed != last_guest_seed) {
-		pr_info("Random seed: 0x%x\n", guest_random_seed);
-		last_guest_seed = guest_random_seed;
+	if (kvm_random_seed != last_kvm_seed) {
+		pr_info("Random seed: 0x%x\n", kvm_random_seed);
+		last_kvm_seed = kvm_random_seed;
 	}
-	guest_rng = new_guest_random_state(guest_random_seed);
-	sync_global_to_guest(vm, guest_rng);
+	kvm_rng = new_kvm_random_state(kvm_random_seed);
+	sync_global_to_guest(vm, kvm_rng);
 
 	kvm_arch_vm_post_create(vm, nr_runnable_vcpus);
 
@@ -2279,8 +2279,8 @@ void __attribute((constructor)) kvm_selftest_init(void)
 	sigaction(SIGILL, &sig_sa, NULL);
 	sigaction(SIGFPE, &sig_sa, NULL);
 
-	guest_random_seed = last_guest_seed = random();
-	pr_info("Random seed: 0x%x\n", guest_random_seed);
+	kvm_random_seed = last_kvm_seed = random();
+	pr_info("Random seed: 0x%x\n", kvm_random_seed);
 
 	kvm_selftest_arch_init();
 }
tools/testing/selftests/kvm/lib/memstress.c +4 -4
index 6dcd15910a06..3599b75d97c9 100644
@@ -48,14 +48,14 @@ void memstress_guest_code(u32 vcpu_idx)
 {
 	struct memstress_args *args = &memstress_args;
 	struct memstress_vcpu_args *vcpu_args = &args->vcpu_args[vcpu_idx];
-	struct guest_random_state rand_state;
+	struct kvm_random_state rand_state;
 	gva_t gva;
 	u64 pages;
 	u64 addr;
 	u64 page;
 	int i;
 
-	rand_state = new_guest_random_state(guest_random_seed + vcpu_idx);
+	rand_state = new_kvm_random_state(kvm_random_seed + vcpu_idx);
 
 	gva = vcpu_args->gva;
 	pages = vcpu_args->pages;
@@ -69,13 +69,13 @@ void memstress_guest_code(u32 vcpu_idx)
 
 		for (i = 0; i < pages; i++) {
 			if (args->random_access)
-				page = guest_random_u32(&rand_state) % pages;
+				page = kvm_random_u32(&rand_state) % pages;
 			else
 				page = i;
 
 			addr = gva + (page * args->guest_page_size);
 
-			if (__guest_random_bool(&rand_state, args->write_percent))
+			if (__kvm_random_bool(&rand_state, args->write_percent))
 				*(u64 *)addr = 0x0123456789ABCDEF;
 			else
 				READ_ONCE(*(u64 *)addr);
tools/testing/selftests/kvm/lib/test_util.c +3 -3
index bab1bd2b775b..e98ca7ef439c 100644
@@ -30,13 +30,13 @@ void __attribute__((used)) expect_sigbus_handler(int signum)
  * Park-Miller LCG using standard constants.
  */
 
-struct guest_random_state new_guest_random_state(u32 seed)
+struct kvm_random_state new_kvm_random_state(u32 seed)
 {
-	struct guest_random_state s = {.seed = seed};
+	struct kvm_random_state s = {.seed = seed};
 	return s;
 }
 
-u32 guest_random_u32(struct guest_random_state *state)
+u32 kvm_random_u32(struct kvm_random_state *state)
 {
 	state->seed = (u64)state->seed * 48271 % ((u32)(1 << 31) - 1);
 	return state->seed;
tools/testing/selftests/kvm/x86/sev_dbg_test.c +1 -1
index a9d8e4c059f9..eaa8201b937d 100644
@@ -34,7 +34,7 @@ static void validate_buffers(void)
 
 static void ____test_sev_dbg(struct kvm_vm *vm, int i, int j, int nr_bytes)
 {
-	u8 pattern = guest_random_u32(&guest_rng);
+	u8 pattern = kvm_random_u32(&kvm_rng);
 
 	if (i + nr_bytes > BUFFER_SIZE || j + nr_bytes > BUFFER_SIZE)
 		return;
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#5 lore raw

PATCH v8 4/20 KVM: selftests: Initialize the default/global pRNG during kvm_selftest_init()

In reply to Sean Christopherson

Initialize the default kvm_rng during selftest initialization so that the pRNG can be used by tests before creating a VM. As pointed out by Sashiko, failure to actually initialize the generate makes it decidedly not random.

---
 tools/testing/selftests/kvm/lib/kvm_util.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
tools/testing/selftests/kvm/lib/kvm_util.c +11 -7
index 875030c22d07..1016865d3f7a 100644
@@ -24,6 +24,13 @@ u32 kvm_random_seed;
 struct kvm_random_state kvm_rng;
 static u32 last_kvm_seed;
 
+static void kvm_seed_rng(u32 seed)
+{
+	kvm_random_seed = last_kvm_seed = seed;
+	pr_info("Random seed: 0x%x\n", kvm_random_seed);
+	kvm_rng = new_kvm_random_state(kvm_random_seed);
+}
+
 static size_t vcpu_mmap_sz(void);
 
 int __open_path_or_exit(const char *path, int flags, const char *enoent_help)
@@ -515,11 +522,9 @@ struct kvm_vm *__vm_create(struct vm_shape shape, u32 nr_runnable_vcpus,
 	slot0 = memslot2region(vm, 0);
 	ucall_init(vm, slot0->region.guest_phys_addr + slot0->region.memory_size);
 
-	if (kvm_random_seed != last_kvm_seed) {
-		pr_info("Random seed: 0x%x\n", kvm_random_seed);
-		last_kvm_seed = kvm_random_seed;
-	}
-	kvm_rng = new_kvm_random_state(kvm_random_seed);
+	if (kvm_random_seed != last_kvm_seed)
+		kvm_seed_rng(kvm_random_seed);
+
 	sync_global_to_guest(vm, kvm_rng);
 
 	kvm_arch_vm_post_create(vm, nr_runnable_vcpus);
@@ -2279,8 +2284,7 @@ void __attribute((constructor)) kvm_selftest_init(void)
 	sigaction(SIGILL, &sig_sa, NULL);
 	sigaction(SIGFPE, &sig_sa, NULL);
 
-	kvm_random_seed = last_kvm_seed = random();
-	pr_info("Random seed: 0x%x\n", kvm_random_seed);
+	kvm_seed_rng(random());
 
 	kvm_selftest_arch_init();
 }
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#6 lore raw

Re: [PATCH v8 04/20] KVM: selftests: Initialize the default/global pRNG during kvm_selftest_init()

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Global PRNG state cannot be reseeded for pre-VM usage, breaking reproducibility in tests like…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Global PRNG state cannot be reseeded for pre-VM usage, breaking reproducibility in tests like irq_test. --

commit 62d9f92bdfe9d424d969de02ee382ed391548f81
Author: Sean Christopherson <[email protected]>

KVM: selftests: Initialize the default/global pRNG during kvm_selftest_init()

This commit initializes the default kvm_rng during the selftest constructor so tests can use the pRNG before creating a VM. It also extracts the seeding logic into a new kvm_seed_rng() helper.

Quoted 5 lines
tools/testing/selftests/kvm/lib/kvm_util.c +0 -0
index 875030c22d07b..1016865d3f7a2 100644

tools/testing/selftests/kvm/lib/kvm_util.c u32 kvm_random_seed; · line 33

Quoted 10 lines
tools/testing/selftests/kvm/lib/kvm_util.c +7 -0
@@ -24,6 +24,13 @@ u32 kvm_random_seed;
 struct kvm_random_state kvm_rng;
 static u32 last_kvm_seed;
 
+static void kvm_seed_rng(u32 seed)
+{
+	kvm_random_seed = last_kvm_seed = seed;
+	pr_info("Random seed: 0x%x\n", kvm_random_seed);
+	kvm_rng = new_kvm_random_state(kvm_random_seed);
+}
+

[Severity: Medium]
Does making this function static prevent tests from reseeding the PRNG early?

If a test draws from kvm_rng before __vm_create() is called (for example, generating gsi and vector at the start of main() in irq_test), the test will use the initial seed from the constructor.

Since kvm_seed_rng() is static and only automatically reseeded during __vm_create(), how can developers override the seed to reproduce CI failures for these pre-VM random variables?

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
author maintainer
#7 lore raw

Re: [PATCH v8 04/20] KVM: selftests: Initialize the default/global pRNG during kvm_selftest_init()

Quoted 20 lines · On Fri, Jun 26, 2026, [email protected] wrote:
Quoted 5 lines
tools/testing/selftests/kvm/lib/kvm_util.c +0 -0
index 875030c22d07b..1016865d3f7a2 100644

tools/testing/selftests/kvm/lib/kvm_util.c u32 kvm_random_seed; · line 33

Quoted 10 lines
tools/testing/selftests/kvm/lib/kvm_util.c +7 -0
@@ -24,6 +24,13 @@ u32 kvm_random_seed;
 struct kvm_random_state kvm_rng;
 static u32 last_kvm_seed;
 
+static void kvm_seed_rng(u32 seed)
+{
+	kvm_random_seed = last_kvm_seed = seed;
+	pr_info("Random seed: 0x%x\n", kvm_random_seed);
+	kvm_rng = new_kvm_random_state(kvm_random_seed);
+}
+

[Severity: Medium]
Does making this function static prevent tests from reseeding the PRNG early?

Quoted 7 lines

If a test draws from kvm_rng before __vm_create() is called (for example, generating gsi and vector at the start of main() in irq_test), the test will use the initial seed from the constructor.

Since kvm_seed_rng() is static and only automatically reseeded during __vm_create(), how can developers override the seed to reproduce CI failures for these pre-VM random variables?

By hacking selftests. Yes, it's ugly, but AFAIK we haven't yet had a failure that's actually been dependent on the RNG seed, and plumbing in a way to set the seed for every test would be painful.

author maintainer
#8 lore raw

PATCH v8 5/20 KVM: selftests: Seed libc's RNG before using it to generate a seed for KVM's pRNG

In reply to Sean Christopherson

Seed the RNG used by random() using the de facto standard method of srand(time(0)), so that a different seed is actually used in each test run. E.g. without seeding the RNG, literally every test on x86 will use 0x6b8b4567 to seed the KVM RNG.

---
 tools/testing/selftests/kvm/lib/kvm_util.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/kvm/lib/kvm_util.c +2 -0
index 1016865d3f7a..277166ab1aa9 100644
@@ -15,6 +15,7 @@
 #include <sys/resource.h>
 #include <sys/types.h>
 #include <sys/stat.h>
+#include <time.h>
 #include <unistd.h>
 #include <linux/kernel.h>
 
@@ -2284,6 +2285,7 @@ void __attribute((constructor)) kvm_selftest_init(void)
 	sigaction(SIGILL, &sig_sa, NULL);
 	sigaction(SIGFPE, &sig_sa, NULL);
 
+	srandom(time(0));
 	kvm_seed_rng(random());
 
 	kvm_selftest_arch_init();
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#9 lore raw

Re: [PATCH v8 05/20] KVM: selftests: Seed libc's RNG before using it to generate a seed for KVM's pRNG

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The KVM pseudo-random number generator gets trapped generating only 0s if `random()` returns 0…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The KVM pseudo-random number generator gets trapped generating only 0s if random() returns 0 or 2147483647. --

commit cb357af51bf4702a2c21b353556fb23a80cc5efe
Author: Sean Christopherson <[email protected]>

KVM: selftests: Seed libc's RNG before using it to generate a seed for KVM's pRNG

This commit initializes the libc RNG with the current time before using it to generate the initial seed for KVM's selftest pseudo-random number generator.

Quoted 4 lines
tools/testing/selftests/kvm/lib/kvm_util.c +0 -0
index 1016865d3f7a2..277166ab1aa99 100644

[ ... ]

tools/testing/selftests/kvm/lib/kvm_util.c void __attribute((constructor)) kvm_selftest_init(void) · line 2289

Quoted 6 lines
tools/testing/selftests/kvm/lib/kvm_util.c +1 -0
@@ -2284,6 +2285,7 @@ void __attribute((constructor)) kvm_selftest_init(void)
 	sigaction(SIGILL, &sig_sa, NULL);
 	sigaction(SIGFPE, &sig_sa, NULL);
 
+	srandom(time(0));
 	kvm_seed_rng(random());

[Severity: Medium]
Can this cause the KVM PRNG to get stuck returning only 0s, resulting in intermittent test failures?

If random() happens to return 0 or 2147483647 (the modulus), passing it to kvm_seed_rng() will cause the Lehmer LCG calculation in kvm_random_u32() to permanently evaluate to 0:

tools/testing/selftests/kvm/lib/test_util.c:kvm_random_u32() {
    ...
    state->seed = (u64)state->seed * 48271 % ((u32)(1 << 31) - 1);
    return state->seed;
}

When state->seed becomes 0, all subsequent PRNG calls will endlessly return 0. Does it make sense to sanitize the result of random() here to avoid these pathological seeds?

Quoted 2 lines
tools/testing/selftests/kvm/lib/kvm_util.c +0 -0
 	kvm_selftest_arch_init();
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
author maintainer
#10 lore raw

PATCH v8 6/20 KVM: selftests: Add helper to generate random u64 in range [min,max]

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Introduce kvm_random_u64_in_range(state, min, max). This function returns a random u64 in the inclusive range of [min, max] using a struct kvm_random_state.

---
 .../testing/selftests/kvm/include/test_util.h  |  3 +++
 tools/testing/selftests/kvm/lib/test_util.c    | 18 ++++++++++++++++++
 2 files changed, 21 insertions(+)
tools/testing/selftests/kvm/include/test_util.h +3 -0
index 44c0104d60ac..d64c8a228207 100644
@@ -134,6 +134,9 @@ static inline u64 kvm_random_u64(struct kvm_random_state *state)
 	return ((u64)kvm_random_u32(state) << 32) | kvm_random_u32(state);
 }
 
+u64 kvm_random_u64_in_range(struct kvm_random_state *state, u64 min,
+			    u64 max);
+
 enum vm_mem_backing_src_type {
 	VM_MEM_SRC_ANONYMOUS,
 	VM_MEM_SRC_ANONYMOUS_THP,
tools/testing/selftests/kvm/lib/test_util.c +18 -0
index e98ca7ef439c..e208a57f190c 100644
@@ -42,6 +42,24 @@ u32 kvm_random_u32(struct kvm_random_state *state)
 	return state->seed;
 }
 
+/* Returns a random u64 in the inclusive range [min, max] */
+u64 kvm_random_u64_in_range(struct kvm_random_state *state, u64 min,
+			    u64 max)
+{
+	u64 value;
+	u64 range;
+
+	TEST_ASSERT(min <= max, "PEBKAC, min = 0x%lx, max = 0x%lx", min, max);
+
+	value = kvm_random_u64(state);
+
+	range = max - min;
+	if (range == ULLONG_MAX)
+		return value;
+
+	return min + (value % (range + 1));
+}
+
 /*
  * Parses "[0-9]+[kmgt]?".
  */
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#11 lore raw

Re: [PATCH v8 06/20] KVM: selftests: Add helper to generate random u64 in range [min,max]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] kvm_random_u64() never sets bits 31 and 63, causing kvm_random_u64_in_range() to have severe…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] kvm_random_u64() never sets bits 31 and 63, causing kvm_random_u64_in_range() to have severe blind spots. --

commit 5a3dcffb3cb35a622072ea5194b7875f774cb3a7
Author: Josh Hilke <[email protected]>

KVM: selftests: Add helper to generate random u64 in range [min,max]

Introduce kvm_random_u64_in_range(state, min, max) to return a random u64 within an inclusive range.

Quoted 4 lines
tools/testing/selftests/kvm/lib/test_util.c +0 -0
index e98ca7ef439c4..e208a57f190cf 100644

[ ... ]

Quoted 4 lines
tools/testing/selftests/kvm/lib/test_util.c +0 -0
@@ -42,6 +42,24 @@ u32 kvm_random_u32(struct kvm_random_state *state)
 	return state->seed;
 }
 

tools/testing/selftests/kvm/lib/test_util.c u32 kvm_random_u32(struct kvm_random_state *state) · line 54

Quoted 10 lines
tools/testing/selftests/kvm/lib/test_util.c +10 -0
+/* Returns a random u64 in the inclusive range [min, max] */
+u64 kvm_random_u64_in_range(struct kvm_random_state *state, u64 min,
+			    u64 max)
+{
+	u64 value;
+	u64 range;
+
+	TEST_ASSERT(min <= max, "PEBKAC, min = 0x%lx, max = 0x%lx", min, max);
+
+	value = kvm_random_u64(state);

[Severity: Medium]
Will this result in severe blind spots within the generated range?

Looking at how the random value is constructed:

tools/testing/selftests/kvm/include/test_util.h:kvm_random_u64() {
    return ((u64)kvm_random_u32(state) << 32) | kvm_random_u32(state);
}

And considering kvm_random_u32() limits its return value with a modulo of ((u32)(1 << 31) - 1), it appears bits 31 and 63 will always remain zero.

Does this mean the subsequent modulo operation in this function:

tools/testing/selftests/kvm/lib/test_util.c u32 kvm_random_u32(struct kvm_random_state *state) · line 61

Quoted 7 lines
tools/testing/selftests/kvm/lib/test_util.c +7 -0
+
+	range = max - min;
+	if (range == ULLONG_MAX)
+		return value;
+
+	return min + (value % (range + 1));
+}

will fail to cover large portions of requested ranges? For example, if a test requests a 4GB range, could this only ever return values in the lower 2GB of that range?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
author maintainer
#12 lore raw

PATCH v8 7/20 KVM: selftests: Add an irqfd send+receive (and later IRQ bypass) test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Add a new test, irq_test to verify that KVM correctly delivers interrupts to a running vCPU, when triggered via an eventfd bound to a KVM GSI using KVM's irqfd mechanism.

This test is intentionally simple, for now. Support for sending interrupts via VFIO devices, for IRQ bypass, and for other features will be added in the near future.

Add the test in common code, even though it currently will only build and run on x86, as the concept and the bulk of the host-side code isn't specific to x86.

---
 tools/testing/selftests/kvm/Makefile.kvm |   1 +
 tools/testing/selftests/kvm/irq_test.c   | 170 +++++++++++++++++++++++
 2 files changed, 171 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/irq_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index ae0dfd67dd31..a24875fada03 100644
@@ -157,6 +157,7 @@ TEST_GEN_PROGS_x86 += coalesced_io_test
 TEST_GEN_PROGS_x86 += dirty_log_perf_test
 TEST_GEN_PROGS_x86 += guest_memfd_test
 TEST_GEN_PROGS_x86 += hardware_disable_test
+TEST_GEN_PROGS_x86 += irq_test
 TEST_GEN_PROGS_x86 += mmu_stress_test
 TEST_GEN_PROGS_x86 += rseq_test
 TEST_GEN_PROGS_x86 += steal_time
tools/testing/selftests/kvm/irq_test.c new file +170 -0

170 lines not on this page · raw message

Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#13 lore raw

PATCH v8 8/20 KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ bypass test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Introduce proc_util.c and proc_util.h to house utility functions for interacting with the proc filesystem.

Add vfio_msix_to_host_irq(), which parses /proc/interrupts, to get the host Linux IRQ for a given VFIO device BDF and MSI-X vector.

This helper will be used by the eventfd IRQ test to print the host IRQ number when triggering IRQs via VFIO device, e.g. to aid in debugging if the test fails.

---
 tools/testing/selftests/kvm/Makefile.kvm      |  1 +
 .../testing/selftests/kvm/include/proc_util.h |  9 +++++
 tools/testing/selftests/kvm/lib/proc_util.c   | 40 +++++++++++++++++++
 3 files changed, 50 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/include/proc_util.h
 create mode 100644 tools/testing/selftests/kvm/lib/proc_util.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index a24875fada03..88c6c8046dde 100644
@@ -11,6 +11,7 @@ LIBKVM += lib/kvm_util.c
 LIBKVM += lib/lru_gen_util.c
 LIBKVM += lib/memstress.c
 LIBKVM += lib/guest_sprintf.c
+LIBKVM += lib/proc_util.c
 LIBKVM += lib/rbtree.c
 LIBKVM += lib/sparsebit.c
 LIBKVM += lib/test_util.c
tools/testing/selftests/kvm/include/proc_util.h new file +9 -0
new file mode 100644
index 000000000000..704839b6d7af
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef SELFTEST_KVM_PROC_UTIL_H
+#define SELFTEST_KVM_PROC_UTIL_H
+
+#include <stdint.h>
+
+unsigned int vfio_msix_to_host_irq(const char *vfio_device_bdf, int msix);
+
+#endif /* SELFTEST_KVM_PROC_UTIL_H */
tools/testing/selftests/kvm/lib/proc_util.c new file +40 -0
new file mode 100644
index 000000000000..84d30f055a0a
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "kvm_util.h"
+#include "test_util.h"
+#include "proc_util.h"
+
+static FILE *open_proc_interrupts(void)
+{
+	FILE *fp;
+
+	fp = fopen("/proc/interrupts", "r");
+	TEST_ASSERT(fp, "fopen(/proc/interrupts) failed");
+
+	return fp;
+}
+
+unsigned int vfio_msix_to_host_irq(const char *device_bdf, int msix)
+{
+	char search_string[64];
+	char line[4096];
+	int irq = -1;
+	FILE *fp;
+
+	fp = open_proc_interrupts();
+
+	snprintf(search_string, sizeof(search_string), "vfio-msix[%d]", msix);
+
+	while (fgets(line, sizeof(line), fp)) {
+		if (strstr(line, device_bdf) && strstr(line, search_string)) {
+			TEST_ASSERT_EQ(1, sscanf(line, "%d:", &irq));
+			break;
+		}
+	}
+
+	fclose(fp);
+
+	TEST_ASSERT(irq != -1, "Failed to locate IRQ for %s %s", device_bdf,
+		    search_string);
+	return (unsigned int)irq;
+}
+
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#14 lore raw

Re: [PATCH v8 08/20] KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ bypass test

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The 4096-byte buffer used to read `/proc/interrupts` is too small for systems with many CPUs…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The 4096-byte buffer used to read /proc/interrupts is too small for systems with many CPUs, causing line truncation and leading to either deterministic test crashes or silent parsing of incorrect IRQ numbers. --

commit 87caa5d203166722732c2c1909ec869049e95fe7
Author: David Matlack <[email protected]>

KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ bypass test

This commit introduces proc_util.c and adds vfio_msix_to_host_irq() to parse /proc/interrupts and get the host Linux IRQ for a given VFIO device BDF and MSI-X vector.

Quoted 5 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +0 -0
new file mode 100644
index 0000000000000..84d30f055a0a9

[ ... ]

Quoted 4 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +4 -0
+unsigned int vfio_msix_to_host_irq(const char *device_bdf, int msix)
+{
+	char search_string[64];
+	char line[4096];

tools/testing/selftests/kvm/lib/proc_util.c line 29

Quoted 10 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +10 -0
+	int irq = -1;
+	FILE *fp;
+
+	fp = open_proc_interrupts();
+
+	snprintf(search_string, sizeof(search_string), "vfio-msix[%d]", msix);
+
+	while (fgets(line, sizeof(line), fp)) {
+		if (strstr(line, device_bdf) && strstr(line, search_string)) {
+			TEST_ASSERT_EQ(1, sscanf(line, "%d:", &irq));

[Severity: Medium]
Will this 4096-byte buffer truncate lines on systems with many CPUs?

When /proc/interrupts is read on a host with a large number of CPUs (e.g., >360 CPUs), the per-CPU interrupt counts can cause the line to exceed the fixed 4096-byte limit, causing fgets() to split the line into multiple chunks.

If the final chunk contains the device strings (device_bdf and search_string) but not the original IRQ number at the start of the line, sscanf() could evaluate incorrectly.

If the trailing chunk begins with a CPU count, sscanf(line, "%d:", &irq) would extract the count, ignore the missing colon, return 1, and silently assign the wrong IRQ.

If the chunk starts with non-numeric text, sscanf() returns 0, failing TEST_ASSERT_EQ() and aborting the test.

Does this regression require a larger fixed buffer or dynamic allocation to handle systems with higher CPU counts?

Quoted 10 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +10 -0
+			break;
+		}
+	}
+
+	fclose(fp);
+
+	TEST_ASSERT(irq != -1, "Failed to locate IRQ for %s %s", device_bdf,
+		    search_string);
+	return (unsigned int)irq;
+}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
author maintainer
#15 lore raw

PATCH v8 9/20 KVM: selftests: Add VFIO device support to eventfd IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-d' argument that takes a BDF (in the format segment:bus:device.function) of an interrupt-capable PCI(e) device bound to VFIO, and use said device to trigger interrupts instead of always synthesizing interrupts via direct writes to the eventfd.

Using a VFIO device to trigger interrupts validates the end-to-end delivery of IRQs for "real" devices, and when supported by hardware (and KVM), also validates interrupt delivery via IRQ bypass, i.e. via device posted IRQs.

Now that IOMMUFD is a thing, auto-probe IOMMUFD vs. "legacy" VFIO by temporarily opening /dev/iommufd, and skip the test if neither IOMMUFD nor legacy VFIO is available. Add a '-t' option to the user override the probe logic, e.g. in case IOMMUFD is available but the system is configured for legacy usage.

Note, the device must have a VFIO selftest driver in order to work with the test. A helper script to list supported devices will hopefully be available in the near future at tools/testing/selftests/vfio/scripts/list_supported_devices.sh[1].

Example:

$ ./tools/testing/selftests/kvm/irq_test -d 0000:06:0a.1
---
 tools/testing/selftests/kvm/irq_test.c | 98 ++++++++++++++++++++++++--
 1 file changed, 92 insertions(+), 6 deletions(-)
tools/testing/selftests/kvm/irq_test.c +92 -6
index 9f8895b89821..70b2c9cac279 100644
@@ -3,7 +3,10 @@
 #include "test_util.h"
 #include "apic.h"
 #include "processor.h"
+#include "proc_util.h"
 
+#include <libvfio.h>
+#include <linux/sizes.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <unistd.h>
@@ -55,6 +58,48 @@ static void *vcpu_thread_main(void *arg)
 	return NULL;
 }
 
+static int vfio_setup_msi(struct vfio_pci_device *device)
+{
+	const int flags = MAP_SHARED | MAP_ANONYMOUS;
+	const int prot = PROT_READ | PROT_WRITE;
+	struct iova_allocator *allocator;
+	struct dma_region *region;
+
+	/* Sanity check that the device+driver can actually send MSIs. */
+	TEST_REQUIRE(device->driver.ops);
+	TEST_REQUIRE(device->driver.ops->send_msi);
+
+	/*
+	 * Set up a DMA-able region for the driver to use.   Very few devices
+	 * provide a way to arbitrarily send interrupts (MSIs), e.g. by writing
+	 * an MMIO register.  Instead, most devices send MSIs when an action is
+	 * completed, and practically all actions involve DMA of some form.
+	 */
+	allocator = iova_allocator_init(device->iommu);
+
+	region = &device->driver.region;
+	region->size = SZ_2M;
+	region->iova = iova_allocator_alloc(allocator, region->size);
+	region->vaddr = kvm_mmap(region->size, prot, flags, -1);
+	TEST_ASSERT(region->vaddr != MAP_FAILED, "mmap() failed\n");
+	iommu_map(device->iommu, region);
+
+	iova_allocator_cleanup(allocator);
+
+	vfio_pci_driver_init(device);
+
+	return device->driver.msi;
+}
+
+static void trigger_interrupt(struct vfio_pci_device *device, int eventfd)
+{
+	if (device)
+		vfio_pci_driver_send_msi(device);
+	else
+		eventfd_write(eventfd, 1);
+}
+
+
 static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 			  u8 vector)
 {
@@ -74,11 +119,29 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 	vm_ioctl(vm, KVM_SET_GSI_ROUTING, &routing.header);
 }
 
+static const char *probe_iommu_type(void)
+{
+	int io_fd;
+
+	io_fd = open("/dev/iommu", O_RDONLY);
+	if (io_fd >= 0) {
+		close(io_fd);
+		return MODE_IOMMUFD;
+	}
+
+	io_fd = __open_path_or_exit("/dev/vfio/vfio", O_RDONLY,
+				    "Is VFIO (or IOMMUFD) loaded and enabled?");
+	close(io_fd);
+	return MODE_VFIO_TYPE1_IOMMU;
+}
+
 static void help(const char *name)
 {
-	printf("Usage: %s [-h]\n", name);
+	printf("Usage: %s [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
+	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
+	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
 	exit(KSFT_FAIL);
 }
@@ -100,14 +163,25 @@ int main(int argc, char **argv)
 	u32 gsi = kvm_random_u64_in_range(&kvm_rng, 24, KVM_MAX_IRQ_ROUTES - 1);
 	u8 vector = kvm_random_u64_in_range(&kvm_rng, 32, UINT8_MAX);
 
-	struct kvm_vcpu *vcpus[KVM_MAX_VCPUS];
 	pthread_t vcpu_threads[KVM_MAX_VCPUS];
+	struct kvm_vcpu *vcpus[KVM_MAX_VCPUS];
+	struct vfio_pci_device *device = NULL;
 	int nr_irqs = 1000, nr_vcpus = 1;
-	int i, j, c, eventfd;
+	const char *device_bdf = NULL;
+	const char *iommu_type = NULL;
+	int i, j, c, msix, eventfd;
+	struct iommu *iommu;
 	struct kvm_vm *vm;
+	int irq;
 
-	while ((c = getopt(argc, argv, "h")) != -1) {
+	while ((c = getopt(argc, argv, "d:ht:")) != -1) {
 		switch (c) {
+		case 'd':
+			device_bdf = optarg;
+			break;
+		case 't':
+			iommu_type = optarg;
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
@@ -119,7 +193,19 @@ int main(int argc, char **argv)
 	vm = vm_create_with_vcpus(nr_vcpus, guest_code, vcpus);
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
 
-	eventfd = kvm_new_eventfd();
+	if (device_bdf) {
+		if (!iommu_type)
+			iommu_type = probe_iommu_type();
+		iommu = iommu_init(iommu_type);
+		device = vfio_pci_device_init(device_bdf, iommu);
+		msix = vfio_setup_msi(device);
+		irq = vfio_msix_to_host_irq(device_bdf, msix);
+		eventfd = device->msi_eventfds[msix];
+		printf("Using device %s MSI-X[%d] (IRQ-%u)\n", device_bdf, msix,
+		       irq);
+	} else {
+		eventfd = kvm_new_eventfd();
+	}
 
 	pr_info("Injecting interrupts for GSI %d (guest vector 0x%x) %d times\n",
 		gsi, vector, nr_irqs);
@@ -147,7 +233,7 @@ int main(int argc, char **argv)
 				    "IRQ flag for vCPU %d not clear prior to test",
 				    vcpus[j]->id);
 
-		eventfd_write(eventfd, 1);
+		trigger_interrupt(device, eventfd);
 
 		clock_gettime(CLOCK_MONOTONIC, &start);
 		while (!GUEST_RECEIVED_IRQ(vcpu) &&
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#16 lore raw

PATCH v8 10/20 KVM: selftests: Add a helper to set proc IRQ affinity for IRQ test

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Add a utility, proc_irq_set_smp_affinity(), to set the CPU affinity of a Linux host IRQ via the proc filesystem. Use smp_affinity_list instead of smp_affinity to avoid having to convert the single CPU to a bitmask.

The helper will be used by the eventfd IRQ test to verify delivery of IRQs when the affinity is randomized/modified.

[sean: make the utility self-contained, drop "list", massage changelog]
---
 tools/testing/selftests/kvm/include/proc_util.h |  2 ++
 tools/testing/selftests/kvm/lib/proc_util.c     | 14 ++++++++++++++
 2 files changed, 16 insertions(+)
tools/testing/selftests/kvm/include/proc_util.h +2 -0
index 704839b6d7af..d1ddc967d11d 100644
@@ -6,4 +6,6 @@
 
 unsigned int vfio_msix_to_host_irq(const char *vfio_device_bdf, int msix);
 
+void proc_irq_set_smp_affinity(unsigned int irq, int cpu);
+
 #endif /* SELFTEST_KVM_PROC_UTIL_H */
tools/testing/selftests/kvm/lib/proc_util.c +14 -0
index 84d30f055a0a..3960b3841d63 100644
@@ -38,3 +38,17 @@ unsigned int vfio_msix_to_host_irq(const char *device_bdf, int msix)
 	return (unsigned int)irq;
 }
 
+void proc_irq_set_smp_affinity(unsigned int irq, int cpu)
+{
+	char path[PATH_MAX];
+	int r, fd;
+
+	snprintf(path, sizeof(path), "/proc/irq/%u/smp_affinity_list", irq);
+	fd = open(path, O_RDWR);
+	TEST_ASSERT(fd >= 0, "Failed to open %s", path);
+
+	r = dprintf(fd, "%d\n", cpu);
+	TEST_ASSERT(r > 0, "Failed to affinitize IRQ-%u to CPU %d", irq, cpu);
+
+	kvm_close(fd);
+}
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#17 lore raw

PATCH v8 11/20 KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extent the eventfd IRQ test with a '-a' flag to randomly affinitize the device's host IRQ to different physical CPUs throughout the test. This stresses the kernel's ability to maintain correct interrupt routing and delivery even as the underlying hardware IRQ affinity is changed dynamically via /proc/<irq>/smp_affinity{,_list}.

---
 tools/testing/selftests/kvm/irq_test.c | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/irq_test.c +21 -5
index 70b2c9cac279..fd386e3e9ac3 100644
@@ -12,10 +12,12 @@
 #include <unistd.h>
 #include <pthread.h>
 #include <sys/eventfd.h>
+#include <sys/sysinfo.h>
 
 static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;
 static bool guest_ready_for_irqs[KVM_MAX_VCPUS];
 static bool guest_received_irq[KVM_MAX_VCPUS];
+static bool irq_affinity;
 static bool done;
 
 #define GUEST_RECEIVED_IRQ(__vcpu)	\
@@ -137,9 +139,10 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
+	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
@@ -172,10 +175,13 @@ int main(int argc, char **argv)
 	int i, j, c, msix, eventfd;
 	struct iommu *iommu;
 	struct kvm_vm *vm;
-	int irq;
+	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "d:ht:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ht:")) != -1) {
 		switch (c) {
+		case 'a':
+			irq_affinity = true;
+			break;
 		case 'd':
 			device_bdf = optarg;
 			break;
@@ -204,7 +210,12 @@ int main(int argc, char **argv)
 		printf("Using device %s MSI-X[%d] (IRQ-%u)\n", device_bdf, msix,
 		       irq);
 	} else {
+		TEST_ASSERT(!irq_affinity,
+			    "Setting IRQ affinity (-a) requires a backing device (-d)");
+
 		eventfd = kvm_new_eventfd();
+		irq = -1;
+		irq_cpu = -1;
 	}
 
 	pr_info("Injecting interrupts for GSI %d (guest vector 0x%x) %d times\n",
@@ -228,6 +239,11 @@ int main(int argc, char **argv)
 
 		kvm_route_msi(vm, gsi, vcpu, vector);
 
+		if (irq_affinity) {
+			irq_cpu = kvm_random_u64(&kvm_rng) % get_nprocs();
+			proc_irq_set_smp_affinity(irq, irq_cpu);
+		}
+
 		for (j = 0; j < nr_vcpus; j++)
 			TEST_ASSERT(!GUEST_RECEIVED_IRQ(vcpus[j]),
 				    "IRQ flag for vCPU %d not clear prior to test",
@@ -241,8 +257,8 @@ int main(int argc, char **argv)
 			cpu_relax();
 
 		TEST_ASSERT(GUEST_RECEIVED_IRQ(vcpu),
-			    "vCPU %d timed out waiting for IRQ (vector 0x%x) from GSI %d\n",
-			    vcpu->id, vector, gsi);
+			    "vCPU %d timed out waiting for IRQ (vector 0x%x) from GSI %d (via CPU %d)\n",
+			    vcpu->id, vector, gsi, irq_cpu);
 
 		WRITE_AND_SYNC_TO_GUEST(vm, guest_received_irq[vcpu->id], false);
 	}
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
#18 lore raw

Re: [PATCH v8 11/20] KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The IRQ affinity logic incorrectly assumes that CPU IDs are contiguous and zero-indexed up to…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The IRQ affinity logic incorrectly assumes that CPU IDs are contiguous and zero-indexed up to get_nprocs() - 1, which will cause spurious test crashes on systems with sparse CPU topologies or offline CPUs. --

commit 907b9a0cf2f427a532d2facb5d6393850a4b6083
Author: David Matlack <[email protected]>

KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test

This commit extends the KVM eventfd IRQ selftest with a '-a' flag. This flag randomly affinitizes the device's host IRQ to different physical CPUs to stress KVM's interrupt routing logic while affinity changes dynamically.

Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0 -0
index 70b2c9cac2790..fd386e3e9ac3e 100644

[ ... ]

tools/testing/selftests/kvm/irq_test.c int main(int argc, char **argv) · line 244

Quoted 7 lines
tools/testing/selftests/kvm/irq_test.c +3 -0
@@ -228,6 +239,11 @@ int main(int argc, char **argv)
 
 		kvm_route_msi(vm, gsi, vcpu, vector);
 
+		if (irq_affinity) {
+			irq_cpu = kvm_random_u64(&kvm_rng) % get_nprocs();
+			proc_irq_set_smp_affinity(irq, irq_cpu);

[Severity: Medium]
Will this cause spurious test crashes on systems with non-contiguous CPU IDs or offline CPUs?

Since get_nprocs() returns the count of currently online processors, using it with a modulo operation in main() assumes that the online CPU IDs are contiguous and start from 0. On a system where CPUs are offline or the numbering is sparse (for example, if only CPUs 0 and 2 are online, get_nprocs() returns 2, and the modulo yields 0 or 1), this could select an offline or non-existent CPU ID.

Attempting to write an invalid CPU ID to /proc/irq/<irq>/smp_affinity_list via proc_irq_set_smp_affinity() would then fail with -EINVAL and erroneously crash the test.

Quoted 3 lines
tools/testing/selftests/kvm/irq_test.c +2 -0
+		}
+
 		for (j = 0; j < nr_vcpus; j++)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
author maintainer
#19 lore raw

PATCH v8 12/20 KVM: selftests: Add option to set empty routing between IRQs in eventfd IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with an '-e' flag to set empty GSI routing between interrupts. Clobbering the GSI routing table verifies that KVM correctly handles CPUx => NULL => CPUy transitions, not just CPUx => CPUy transitions, and verifies that KVM can "rebuild" an entire routing setup.

[sean: '-e' for "empty" instead of '-c' for "clear", massage changelog]
---
 tools/testing/selftests/kvm/irq_test.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +18 -2
index fd386e3e9ac3..fd03ec940362 100644
@@ -121,6 +121,13 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 	vm_ioctl(vm, KVM_SET_GSI_ROUTING, &routing.header);
 }
 
+static void kvm_set_empty_gsi_routing(struct kvm_vm *vm)
+{
+	struct kvm_irq_routing routing = {};
+
+	vm_ioctl(vm, KVM_SET_GSI_ROUTING, &routing);
+}
+
 static const char *probe_iommu_type(void)
 {
 	int io_fd;
@@ -139,11 +146,12 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
+	printf("-e	Set empty GSI routing in-between some interrupts\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
 	exit(KSFT_FAIL);
@@ -170,6 +178,7 @@ int main(int argc, char **argv)
 	struct kvm_vcpu *vcpus[KVM_MAX_VCPUS];
 	struct vfio_pci_device *device = NULL;
 	int nr_irqs = 1000, nr_vcpus = 1;
+	bool set_empty_routing = false;
 	const char *device_bdf = NULL;
 	const char *iommu_type = NULL;
 	int i, j, c, msix, eventfd;
@@ -177,7 +186,7 @@ int main(int argc, char **argv)
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:ht:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:eht:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -185,6 +194,9 @@ int main(int argc, char **argv)
 		case 'd':
 			device_bdf = optarg;
 			break;
+		case 'e':
+			set_empty_routing = true;
+			break;
 		case 't':
 			iommu_type = optarg;
 			break;
@@ -234,9 +246,13 @@ int main(int argc, char **argv)
 	}
 
 	for (i = 0; i < nr_irqs; i++) {
+		const bool do_set_empty_routing = set_empty_routing && (i & BIT(3));
 		struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];
 		struct timespec start;
 
+		if (do_set_empty_routing)
+			kvm_set_empty_gsi_routing(vm);
+
 		kvm_route_msi(vm, gsi, vcpu, vector);
 
 		if (irq_affinity) {
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#20 lore raw

PATCH v8 13/20 KVM: selftests: Make number of IRQs configurable in IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-i' flag to let the user specify the the number of IRQs to generate (instead of hardcoding the test to always generate 1000 interrupts).

---
 tools/testing/selftests/kvm/irq_test.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +6 -2
index fd03ec940362..bb20bb81e930 100644
@@ -146,12 +146,13 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
 	printf("-e	Set empty GSI routing in-between some interrupts\n");
+	printf("-i	The number of IRQs to generate during the test\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
 	exit(KSFT_FAIL);
@@ -186,7 +187,7 @@ int main(int argc, char **argv)
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:eht:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ehi:t:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -197,6 +198,9 @@ int main(int argc, char **argv)
 		case 'e':
 			set_empty_routing = true;
 			break;
+		case 'i':
+			nr_irqs = atoi_positive("Number of IRQs", optarg);
+			break;
 		case 't':
 			iommu_type = optarg;
 			break;
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#21 lore raw

PATCH v8 14/20 KVM: selftests: Verify non-postable IRQ remapping in IRQ test

In reply to Sean Christopherson

Extend the eventfd IRQ test with an '-n' flag to route a subset of device interrupts as NMIs (Non-Maskable Interrupts) into the guest using an alternating pattern of 4 NMIs followed by 4 regular interrupts.

While this adds coverage for NMI injection, the primary goal is to validate KVM's handling of non-postable interrupt delivery (AMD and Intel IOMMUs only support posting fixed IRQs targeting a single vCPU). KVM has historically bungled handling transitions between posted and remapped modes. Use NMIs to stress the transitions, because they are a reliable, architectural way to force these code paths.

[sean: add GUEST_RECEIVED_INTERRUPT(), massage changelog]
---
 tools/testing/selftests/kvm/irq_test.c | 48 ++++++++++++++++++++------
 1 file changed, 37 insertions(+), 11 deletions(-)
tools/testing/selftests/kvm/irq_test.c +37 -11
index bb20bb81e930..48520c6dea69 100644
@@ -17,11 +17,17 @@
 static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;
 static bool guest_ready_for_irqs[KVM_MAX_VCPUS];
 static bool guest_received_irq[KVM_MAX_VCPUS];
+static bool guest_received_nmi[KVM_MAX_VCPUS];
 static bool irq_affinity;
 static bool done;
 
 #define GUEST_RECEIVED_IRQ(__vcpu)	\
 	SYNC_FROM_GUEST_AND_READ((__vcpu)->vm, guest_received_irq[(__vcpu)->id])
+#define GUEST_RECEIVED_NMI(__vcpu)	\
+	SYNC_FROM_GUEST_AND_READ((__vcpu)->vm, guest_received_nmi[(__vcpu)->id])
+
+#define GUEST_RECEIVED_INTERRUPT(__vcpu, __nmi)	\
+	((__nmi) ? GUEST_RECEIVED_NMI(__vcpu) : GUEST_RECEIVED_IRQ(__vcpu))
 
 static u32 guest_get_vcpu_id(void)
 {
@@ -35,6 +41,11 @@ static void guest_irq_handler(struct ex_regs *regs)
 	x2apic_write_reg(APIC_EOI, 0);
 }
 
+static void guest_nmi_handler(struct ex_regs *regs)
+{
+	WRITE_ONCE(guest_received_nmi[guest_get_vcpu_id()], true);
+}
+
 static void guest_code(void)
 {
 	x2apic_enable();
@@ -103,7 +114,7 @@ static void trigger_interrupt(struct vfio_pci_device *device, int eventfd)
 
 
 static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
-			  u8 vector)
+			  u8 vector, bool use_nmi)
 {
 	struct {
 		struct kvm_irq_routing header;
@@ -114,7 +125,7 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 			.gsi = gsi,
 			.type = KVM_IRQ_ROUTING_MSI,
 			.u.msi.address_lo = 0xFEE00000 | (vcpu->id << 12),
-			.u.msi.data = vector,
+			.u.msi.data = use_nmi ? NMI_VECTOR | (4 << 8) : vector,
 		},
 	};
 
@@ -146,13 +157,14 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-n] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
 	printf("-e	Set empty GSI routing in-between some interrupts\n");
 	printf("-i	The number of IRQs to generate during the test\n");
+	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
 	exit(KSFT_FAIL);
@@ -183,11 +195,12 @@ int main(int argc, char **argv)
 	const char *device_bdf = NULL;
 	const char *iommu_type = NULL;
 	int i, j, c, msix, eventfd;
+	bool use_nmi = false;
 	struct iommu *iommu;
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:ehi:t:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ehi:nt:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -201,6 +214,9 @@ int main(int argc, char **argv)
 		case 'i':
 			nr_irqs = atoi_positive("Number of IRQs", optarg);
 			break;
+		case 'n':
+			use_nmi = true;
+			break;
 		case 't':
 			iommu_type = optarg;
 			break;
@@ -214,6 +230,7 @@ int main(int argc, char **argv)
 
 	vm = vm_create_with_vcpus(nr_vcpus, guest_code, vcpus);
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
+	vm_install_exception_handler(vm, NMI_VECTOR, guest_nmi_handler);
 
 	if (device_bdf) {
 		if (!iommu_type)
@@ -251,36 +268,45 @@ int main(int argc, char **argv)
 
 	for (i = 0; i < nr_irqs; i++) {
 		const bool do_set_empty_routing = set_empty_routing && (i & BIT(3));
+		const bool do_use_nmi = use_nmi && (i & BIT(2));
 		struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];
 		struct timespec start;
 
 		if (do_set_empty_routing)
 			kvm_set_empty_gsi_routing(vm);
 
-		kvm_route_msi(vm, gsi, vcpu, vector);
+		kvm_route_msi(vm, gsi, vcpu, vector, do_use_nmi);
 
 		if (irq_affinity) {
 			irq_cpu = kvm_random_u64(&kvm_rng) % get_nprocs();
 			proc_irq_set_smp_affinity(irq, irq_cpu);
 		}
 
-		for (j = 0; j < nr_vcpus; j++)
+		for (j = 0; j < nr_vcpus; j++) {
 			TEST_ASSERT(!GUEST_RECEIVED_IRQ(vcpus[j]),
 				    "IRQ flag for vCPU %d not clear prior to test",
 				    vcpus[j]->id);
+			TEST_ASSERT(!GUEST_RECEIVED_NMI(vcpus[j]),
+				    "NMI flag for vCPU %d not clear prior to test",
+				    vcpus[j]->id);
+		}
 
 		trigger_interrupt(device, eventfd);
 
 		clock_gettime(CLOCK_MONOTONIC, &start);
-		while (!GUEST_RECEIVED_IRQ(vcpu) &&
+		while (!GUEST_RECEIVED_INTERRUPT(vcpu, do_use_nmi) &&
 		       timespec_to_ns(timespec_elapsed(start)) <= timeout_ns)
 			cpu_relax();
 
-		TEST_ASSERT(GUEST_RECEIVED_IRQ(vcpu),
-			    "vCPU %d timed out waiting for IRQ (vector 0x%x) from GSI %d (via CPU %d)\n",
-			    vcpu->id, vector, gsi, irq_cpu);
+		TEST_ASSERT(GUEST_RECEIVED_INTERRUPT(vcpu, do_use_nmi),
+			    "vCPU %d timed out waiting for %s (vector 0x%x) from GSI %d (via CPU %d)\n",
+			    vcpu->id, do_use_nmi ? "NMI" : "IRQ",
+			    do_use_nmi ? NMI_VECTOR : vector, gsi, irq_cpu);
 
-		WRITE_AND_SYNC_TO_GUEST(vm, guest_received_irq[vcpu->id], false);
+		if (do_use_nmi)
+			WRITE_AND_SYNC_TO_GUEST(vm, guest_received_nmi[vcpu->id], false);
+		else
+			WRITE_AND_SYNC_TO_GUEST(vm, guest_received_irq[vcpu->id], false);
 	}
 
 	WRITE_AND_SYNC_TO_GUEST(vm, done, true);
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#22 lore raw

PATCH v8 15/20 KVM: selftests: Add kvm_gettid() wrapper and convert users

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Add a KVM wrapper for the gettid() syscall so that tests don't have to open code the syscall() themselves. Unfortunately, not all flavors of libc that KVM selftests support provide gettid(). Convert all existing users of the syscall to the new wrapper.

Note, per the gettid() manpage[1], "This call is always successful", i.e. prefixing kvm_ to the syscall name is aligned with the goal of providing syscall wrappers that guarantee success.

No functional changes intended.

---
 tools/testing/selftests/kvm/demand_paging_test.c   | 2 +-
 tools/testing/selftests/kvm/include/kvm_syscalls.h | 5 +++++
 tools/testing/selftests/kvm/lib/assert.c           | 8 ++------
 tools/testing/selftests/kvm/lib/test_util.c        | 3 ++-
 tools/testing/selftests/kvm/rseq_test.c            | 2 +-
 5 files changed, 11 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/demand_paging_test.c +1 -1
index 302c4923d093..f8b3d0b68830 100644
@@ -57,7 +57,7 @@ static void vcpu_worker(struct memstress_vcpu_args *vcpu_args)
 static int handle_uffd_page_request(int uffd_mode, int uffd,
 		struct uffd_msg *msg)
 {
-	pid_t tid = syscall(__NR_gettid);
+	pid_t tid = kvm_gettid();
 	u64 addr = msg->arg.pagefault.address;
 	struct timespec start;
 	struct timespec ts_diff;
tools/testing/selftests/kvm/include/kvm_syscalls.h +5 -0
index 6cb3bed29b81..dc4fb97aef8d 100644
@@ -83,6 +83,11 @@ static inline int kvm_dup(int fd)
 	return new_fd;
 }
 
+static inline pid_t kvm_gettid(void)
+{
+	return syscall(__NR_gettid);
+}
+
 __KVM_SYSCALL_DEFINE(munmap, 2, void *, mem, size_t, size);
 __KVM_SYSCALL_DEFINE(close, 1, int, fd);
 __KVM_SYSCALL_DEFINE(fallocate, 4, int, fd, int, mode, loff_t, offset, loff_t, len);
tools/testing/selftests/kvm/lib/assert.c +2 -6
index 8be0d09ecf0f..1d72dcdfce3b 100644
@@ -10,6 +10,7 @@
 #include <sys/syscall.h>
 
 #include "kselftest.h"
+#include "kvm_syscalls.h"
 
 #ifdef __GLIBC__
 #include <execinfo.h>
@@ -64,11 +65,6 @@ static void test_dump_stack(void)
 static void test_dump_stack(void) {}
 #endif
 
-static pid_t _gettid(void)
-{
-	return syscall(SYS_gettid);
-}
-
 void __attribute__((noinline))
 test_assert(bool exp, const char *exp_str,
 	const char *file, unsigned int line, const char *fmt, ...)
@@ -81,7 +77,7 @@ test_assert(bool exp, const char *exp_str,
 		fprintf(stderr, "==== Test Assertion Failure ====\n"
 			"  %s:%u: %s\n"
 			"  pid=%d tid=%d errno=%d - %s\n",
-			file, line, exp_str, getpid(), _gettid(),
+			file, line, exp_str, getpid(), kvm_gettid(),
 			errno, strerror(errno));
 		test_dump_stack();
 		if (fmt) {
tools/testing/selftests/kvm/lib/test_util.c +2 -1
index e208a57f190c..6b00ab11f3c0 100644
@@ -17,6 +17,7 @@
 #include "linux/kernel.h"
 
 #include "test_util.h"
+#include "kvm_syscalls.h"
 
 sigjmp_buf expect_sigbus_jmpbuf;
 
@@ -395,7 +396,7 @@ long get_run_delay(void)
 	long val[2];
 	FILE *fp;
 
-	sprintf(path, "/proc/%ld/schedstat", syscall(SYS_gettid));
+	sprintf(path, "/proc/%ld/schedstat", (long)kvm_gettid());
 	fp = fopen(path, "r");
 	/* Return MIN_RUN_DELAY_NS upon failure just to be safe */
 	if (fscanf(fp, "%ld %ld ", &val[0], &val[1]) < 2)
tools/testing/selftests/kvm/rseq_test.c +1 -1
index f80ad6b47d16..6510fbfd64f1 100644
@@ -244,7 +244,7 @@ int main(int argc, char *argv[])
 	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
 
 	pthread_create(&migration_thread, NULL, migration_worker,
-		       (void *)(unsigned long)syscall(SYS_gettid));
+		       (void *)(unsigned long)kvm_gettid());
 
 	if (latency >= 0) {
 		/*
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#23 lore raw

PATCH v8 16/20 KVM: selftests: Add kvm_sched_getaffinity() wrapper and convert users

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Add and use a KVM wrapper for the sched_getaffinity() syscall so that selftests don't need to manually assert that the syscall succeeded.

Note, some tests didn't actually assert success, but they all obviously rely on the syscall to succeed.

---
 tools/testing/selftests/kvm/arch_timer.c                  | 2 +-
 tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c | 2 +-
 tools/testing/selftests/kvm/include/kvm_syscalls.h        | 2 ++
 tools/testing/selftests/kvm/lib/kvm_util.c                | 5 ++---
 tools/testing/selftests/kvm/mmu_stress_test.c             | 6 +-----
 tools/testing/selftests/kvm/rseq_test.c                   | 4 +---
 6 files changed, 8 insertions(+), 13 deletions(-)
tools/testing/selftests/kvm/arch_timer.c +1 -1
index 90c475a61b22..f8b02597897b 100644
@@ -85,7 +85,7 @@ static u32 test_get_pcpu(void)
 	cpu_set_t online_cpuset;
 
 	nproc_conf = get_nprocs_conf();
-	sched_getaffinity(0, sizeof(cpu_set_t), &online_cpuset);
+	kvm_sched_getaffinity(0, sizeof(cpu_set_t), &online_cpuset);
 
 	/* Randomly find an available pCPU to place a vCPU on */
 	do {
tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c +1 -1
index f7625eb711d6..d9c9377a6325 100644
@@ -1039,7 +1039,7 @@ int main(int argc, char *argv[])
 	if (!parse_args(argc, argv))
 		exit(KSFT_SKIP);
 
-	sched_getaffinity(0, sizeof(default_cpuset), &default_cpuset);
+	kvm_sched_getaffinity(0, sizeof(default_cpuset), &default_cpuset);
 	set_counter_defaults();
 
 	if (test_args.test_virtual) {
tools/testing/selftests/kvm/include/kvm_syscalls.h +2 -0
index dc4fb97aef8d..5dae6143ddb0 100644
@@ -12,6 +12,7 @@
 #include <sys/mman.h>
 #include <sys/syscall.h>
 
+#include <sched.h>
 #include <test_util.h>
 
 #define MAP_ARGS0(m,...)
@@ -93,6 +94,7 @@ __KVM_SYSCALL_DEFINE(close, 1, int, fd);
 __KVM_SYSCALL_DEFINE(fallocate, 4, int, fd, int, mode, loff_t, offset, loff_t, len);
 __KVM_SYSCALL_DEFINE(ftruncate, 2, unsigned int, fd, off_t, length);
 __KVM_SYSCALL_DEFINE(madvise, 3, void *, addr, size_t, length, int, advice);
+__KVM_SYSCALL_DEFINE(sched_getaffinity, 3, pid_t, pid, size_t, cpusetsize, cpu_set_t *, mask);
 
 #define kvm_free_fd(fd)		\
 do {				\
tools/testing/selftests/kvm/lib/kvm_util.c +2 -3
index 277166ab1aa9..6a0ee033fdef 100644
@@ -674,13 +674,12 @@ void kvm_parse_vcpu_pinning(const char *pcpus_string, u32 vcpu_to_pcpu[],
 	cpu_set_t allowed_mask;
 	char *cpu, *cpu_list;
 	char delim[2] = ",";
-	int i, r;
+	int i;
 
 	cpu_list = strdup(pcpus_string);
 	TEST_ASSERT(cpu_list, "strdup() allocation failed.");
 
-	r = sched_getaffinity(0, sizeof(allowed_mask), &allowed_mask);
-	TEST_ASSERT(!r, "sched_getaffinity() failed");
+	kvm_sched_getaffinity(0, sizeof(allowed_mask), &allowed_mask);
 
 	cpu = strtok(cpu_list, delim);
 
tools/testing/selftests/kvm/mmu_stress_test.c +1 -5
index 473ef4c0ea9f..3d5f33a63b2b 100644
@@ -255,11 +255,7 @@ static void rendezvous_with_vcpus(struct timespec *time, const char *name)
 static void calc_default_nr_vcpus(void)
 {
 	cpu_set_t possible_mask;
-	int r;
-
-	r = sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
-	TEST_ASSERT(!r, "sched_getaffinity failed, errno = %d (%s)",
-		    errno, strerror(errno));
+	kvm_sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
 
 	nr_vcpus = CPU_COUNT(&possible_mask);
 	TEST_ASSERT(nr_vcpus > 0, "Uh, no CPUs?");
tools/testing/selftests/kvm/rseq_test.c +1 -3
index 6510fbfd64f1..557e393c223b 100644
@@ -226,9 +226,7 @@ int main(int argc, char *argv[])
 		}
 	}
 
-	r = sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
-	TEST_ASSERT(!r, "sched_getaffinity failed, errno = %d (%s)", errno,
-		    strerror(errno));
+	kvm_sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
 
 	calc_min_max_cpu();
 
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#24 lore raw

PATCH v8 17/20 KVM: selftests: Add a utility to pin a task to a random CPU, given a CPU set

In reply to Sean Christopherson

From: Josh Hilke <[email protected]>

Add a helper function, pin_task_to_random_cpu(), to pin a task to a random CPU from a given cpu_set_t.

This helper will be used eventfd IRQ test to migrate vCPUs to random pCPUs, to stress host-side interrupt routing and delivery.

---
 .../testing/selftests/kvm/include/kvm_util.h  |  2 ++
 tools/testing/selftests/kvm/lib/kvm_util.c    | 21 +++++++++++++++++++
 2 files changed, 23 insertions(+)
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
index c1f588154398..b39e713c30a4 100644
@@ -1094,6 +1094,8 @@ static inline void pin_task_to_cpu(pthread_t task, int cpu)
 	TEST_ASSERT(!r, "Failed to set thread affinity to pCPU '%u'", cpu);
 }
 
+void pin_task_to_random_cpu(pthread_t task, cpu_set_t *possible_cpus);
+
 static inline int pin_task_to_any_cpu(pthread_t task)
 {
 	int cpu = sched_getcpu();
tools/testing/selftests/kvm/lib/kvm_util.c +21 -0
index 6a0ee033fdef..3794575d2ca0 100644
@@ -668,6 +668,27 @@ void kvm_print_vcpu_pinning_help(void)
 	       "     (default: no pinning)\n", name, name);
 }
 
+void pin_task_to_random_cpu(pthread_t task, cpu_set_t *possible_cpus)
+{
+	int target_idx;
+	int nr_cpus;
+	int cpu;
+
+	nr_cpus = CPU_COUNT(possible_cpus);
+	TEST_ASSERT(nr_cpus > 0, "No CPUs available in possible_cpus");
+
+	target_idx = kvm_random_u64(&kvm_rng) % nr_cpus;
+
+	for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+		if (CPU_ISSET(cpu, possible_cpus) && target_idx-- == 0) {
+			pin_task_to_cpu(task, cpu);
+			return;
+		}
+	}
+
+	TEST_FAIL("Failed to find random CPU in possible_cpus");
+}
+
 void kvm_parse_vcpu_pinning(const char *pcpus_string, u32 vcpu_to_pcpu[],
 			    int nr_vcpus)
 {
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#25 lore raw

PATCH v8 18/20 KVM: selftests: Verify vCPU migration during IRQ delivery in IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-m' flag to have the test migrate the target vCPU to a random physical CPU before triggering its interrupt, e.g. to validate KVM's ability to update device posted IRQ routing.

[sean: pin one vCPU at a time to simplify things, use main()'s affinity]
---
 tools/testing/selftests/kvm/irq_test.c | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/irq_test.c +16 -3
index 48520c6dea69..c0ff6e60b167 100644
@@ -157,13 +157,14 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-n] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n");
 	printf("-e	Set empty GSI routing in-between some interrupts\n");
 	printf("-i	The number of IRQs to generate during the test\n");
+	printf("-m	Pin target vCPU to random physical CPU before triggering interrupt\n");
 	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("\n");
@@ -195,12 +196,14 @@ int main(int argc, char **argv)
 	const char *device_bdf = NULL;
 	const char *iommu_type = NULL;
 	int i, j, c, msix, eventfd;
+	bool migrate_vcpus = false;
+	cpu_set_t available_cpus;
 	bool use_nmi = false;
 	struct iommu *iommu;
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:ehi:nt:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ehi:mnt:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -214,6 +217,9 @@ int main(int argc, char **argv)
 		case 'i':
 			nr_irqs = atoi_positive("Number of IRQs", optarg);
 			break;
+		case 'm':
+			migrate_vcpus = true;
+			break;
 		case 'n':
 			use_nmi = true;
 			break;
@@ -248,7 +254,6 @@ int main(int argc, char **argv)
 
 		eventfd = kvm_new_eventfd();
 		irq = -1;
-		irq_cpu = -1;
 	}
 
 	pr_info("Injecting interrupts for GSI %d (guest vector 0x%x) %d times\n",
@@ -256,6 +261,9 @@ int main(int argc, char **argv)
 
 	kvm_assign_irqfd(vm, gsi, eventfd);
 
+	if (migrate_vcpus)
+		kvm_sched_getaffinity(0, sizeof(available_cpus), &available_cpus);
+
 	for (i = 0; i < nr_vcpus; i++)
 		pthread_create(&vcpu_threads[i], NULL, vcpu_thread_main, vcpus[i]);
 
@@ -266,6 +274,8 @@ int main(int argc, char **argv)
 			continue;
 	}
 
+	irq_cpu = -1;
+
 	for (i = 0; i < nr_irqs; i++) {
 		const bool do_set_empty_routing = set_empty_routing && (i & BIT(3));
 		const bool do_use_nmi = use_nmi && (i & BIT(2));
@@ -282,6 +292,9 @@ int main(int argc, char **argv)
 			proc_irq_set_smp_affinity(irq, irq_cpu);
 		}
 
+		if (migrate_vcpus)
+			pin_task_to_random_cpu(vcpu_threads[i % nr_vcpus], &available_cpus);
+
 		for (j = 0; j < nr_vcpus; j++) {
 			TEST_ASSERT(!GUEST_RECEIVED_IRQ(vcpus[j]),
 				    "IRQ flag for vCPU %d not clear prior to test",
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#26 lore raw

PATCH v8 19/20 KVM: selftests: Make number of vCPUs configurable in IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-v' flag to allow the user to configure the number of vCPUs to create and run (versus only ever using a single vCPU).

Update the routing logic to play nice with 32 bit IDs, enable x2APIC format in KVM (to enable 32-bit ID routing), and disable KVM's x2APIC broadcast quirk so that targeting vCPU 255 doesn't blast the interrupt to all vCPUs when in x2APIC mode.

---
 tools/testing/selftests/kvm/irq_test.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/irq_test.c +13 -3
index c0ff6e60b167..d2c745c54960 100644
@@ -124,7 +124,8 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 		.entry = {
 			.gsi = gsi,
 			.type = KVM_IRQ_ROUTING_MSI,
-			.u.msi.address_lo = 0xFEE00000 | (vcpu->id << 12),
+			.u.msi.address_lo = 0xFEE00000 | (vcpu->id & GENMASK(7, 0)) << 12,
+			.u.msi.address_hi = vcpu->id & GENMASK(31, 8),
 			.u.msi.data = use_nmi ? NMI_VECTOR | (4 << 8) : vector,
 		},
 	};
@@ -157,7 +158,7 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type] [-v nr_vcpus]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
@@ -167,6 +168,7 @@ static void help(const char *name)
 	printf("-m	Pin target vCPU to random physical CPU before triggering interrupt\n");
 	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
+	printf("-v	Number of vCPUS to run\n");
 	printf("\n");
 	exit(KSFT_FAIL);
 }
@@ -203,7 +205,7 @@ int main(int argc, char **argv)
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:ehi:mnt:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ehi:mnt:v:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -226,6 +228,11 @@ int main(int argc, char **argv)
 		case 't':
 			iommu_type = optarg;
 			break;
+		case 'v':
+			nr_vcpus = atoi_positive("Number of vCPUS", optarg);
+			TEST_ASSERT(nr_vcpus <= KVM_MAX_VCPUS,
+				    "KVM selftests support at most %u vCPUs", KVM_MAX_VCPUS);
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
@@ -235,6 +242,9 @@ int main(int argc, char **argv)
 	TEST_REQUIRE(kvm_arch_has_default_irqchip());
 
 	vm = vm_create_with_vcpus(nr_vcpus, guest_code, vcpus);
+	vm_enable_cap(vm, KVM_CAP_X2APIC_API, KVM_X2APIC_API_USE_32BIT_IDS |
+					      KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK);
+
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
 	vm_install_exception_handler(vm, NMI_VECTOR, guest_nmi_handler);
 
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#27 lore raw

PATCH v8 20/20 KVM: selftests: Add xAPIC support in eventfd IRQ test

In reply to Sean Christopherson

From: David Matlack <[email protected]>

Extend the eventfd IRQ test with a '-x' flag to let the user run the test in xAPIC mode instead of the default x2APIC mode. When using xAPIC mode, sanity check user input to ensure the test is being run with at most 255 vCPUs, as xAPIC can only address IDs 0-254 (255, i.e. 0xff, broadcasts to all CPUs).

[sean: add sanity check on number of vCPUs]
---
 tools/testing/selftests/kvm/irq_test.c | 31 +++++++++++++++++++++-----
 1 file changed, 26 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/irq_test.c +26 -5
index d2c745c54960..240f6f0fdbe4 100644
@@ -18,6 +18,7 @@ static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;
 static bool guest_ready_for_irqs[KVM_MAX_VCPUS];
 static bool guest_received_irq[KVM_MAX_VCPUS];
 static bool guest_received_nmi[KVM_MAX_VCPUS];
+static bool x2apic = true;
 static bool irq_affinity;
 static bool done;
 
@@ -31,14 +32,20 @@ static bool done;
 
 static u32 guest_get_vcpu_id(void)
 {
-	return x2apic_read_reg(APIC_ID);
+	if (x2apic)
+		return x2apic_read_reg(APIC_ID);
+	else
+		return xapic_read_reg(APIC_ID) >> 24;
 }
 
 static void guest_irq_handler(struct ex_regs *regs)
 {
 	WRITE_ONCE(guest_received_irq[guest_get_vcpu_id()], true);
 
-	x2apic_write_reg(APIC_EOI, 0);
+	if (x2apic)
+		x2apic_write_reg(APIC_EOI, 0);
+	else
+		xapic_write_reg(APIC_EOI, 0);
 }
 
 static void guest_nmi_handler(struct ex_regs *regs)
@@ -48,7 +55,10 @@ static void guest_nmi_handler(struct ex_regs *regs)
 
 static void guest_code(void)
 {
-	x2apic_enable();
+	if (x2apic)
+		x2apic_enable();
+	else
+		xapic_enable();
 
 	sti_nop();
 
@@ -158,7 +168,7 @@ static const char *probe_iommu_type(void)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type] [-v nr_vcpus]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type] [-v nr_vcpus] [-x]\n", name);
 	printf("\n");
 	printf("Tests KVM interrupt routing and delivery via irqfd.\n");
 	printf("-a	Affine the device's host IRQ to a random physical CPU\n");
@@ -169,6 +179,7 @@ static void help(const char *name)
 	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
 	printf("-t	Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n");
 	printf("-v	Number of vCPUS to run\n");
+	printf("-x	Use xAPIC mode instead of x2APIC mode in the guest\n");
 	printf("\n");
 	exit(KSFT_FAIL);
 }
@@ -205,7 +216,7 @@ int main(int argc, char **argv)
 	struct kvm_vm *vm;
 	int irq, irq_cpu;
 
-	while ((c = getopt(argc, argv, "ad:ehi:mnt:v:")) != -1) {
+	while ((c = getopt(argc, argv, "ad:ehi:mnt:v:x")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -233,6 +244,9 @@ int main(int argc, char **argv)
 			TEST_ASSERT(nr_vcpus <= KVM_MAX_VCPUS,
 				    "KVM selftests support at most %u vCPUs", KVM_MAX_VCPUS);
 			break;
+		case 'x':
+			x2apic = false;
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
@@ -248,6 +262,11 @@ int main(int argc, char **argv)
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
 	vm_install_exception_handler(vm, NMI_VECTOR, guest_nmi_handler);
 
+	if (!x2apic) {
+		TEST_ASSERT(nr_vcpus < 256, "xAPIC can only target IDs [0-254] (255 vCPUs)");
+		virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA);
+	}
+
 	if (device_bdf) {
 		if (!iommu_type)
 			iommu_type = probe_iommu_type();
@@ -271,6 +290,8 @@ int main(int argc, char **argv)
 
 	kvm_assign_irqfd(vm, gsi, eventfd);
 
+	sync_global_to_guest(vm, x2apic);
+
 	if (migrate_vcpus)
 		kvm_sched_getaffinity(0, sizeof(available_cpus), &available_cpus);
 
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
author maintainer
#28 lore raw
In reply to Sean Christopherson
Quoted 5 lines · Sean Christopherson

David and Josh's series to add a selftest for verifying interrupt delivery via eventfd (via KVM_IRQFD), and also from a real device, wired up via VFIO.

Gory details in the patches, and in the v5 cover letter.

Quoted 7 lines

v8:
- Explicitly include <time.h>. [Sashiko]
- Use iova_allocator instea of hardcoding the IOVA. [Sashiko, David]
- Add a comment explaining why DMA needs to be allocated. [David]
- Probe /dev/vfio/vfio, not /dev/vfio. [Sashiko, David]

[...]

Applied to kvm-x86 irq_test, thanks!

[01/20] KVM: selftests: Build and link selftests/vfio/lib into KVM selftests

        https://github.com/kvm-x86/linux/commit/a262fc49e0aa
[02/20] KVM: selftests: Add macros to read/write+sync to/from guest memory
        https://github.com/kvm-x86/linux/commit/ac050f2adcdf
[03/20] KVM: selftests: Rename guest_rng to kvm_rng
        https://github.com/kvm-x86/linux/commit/e0bd29eddf7e
[04/20] KVM: selftests: Initialize the default/global pRNG during kvm_selftest_init()
        https://github.com/kvm-x86/linux/commit/bae85cd758fe
[05/20] KVM: selftests: Seed libc's RNG before using it to generate a seed for KVM's pRNG
        https://github.com/kvm-x86/linux/commit/5506feb24282
[06/20] KVM: selftests: Add helper to generate random u64 in range [min,max]
        https://github.com/kvm-x86/linux/commit/c772591d279e
[07/20] KVM: selftests: Add an irqfd send+receive (and later IRQ bypass) test
        https://github.com/kvm-x86/linux/commit/3ceab37ac973
[08/20] KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ bypass test
        https://github.com/kvm-x86/linux/commit/362cc0016249
[09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test
        https://github.com/kvm-x86/linux/commit/3902e8b02dce
[10/20] KVM: selftests: Add a helper to set proc IRQ affinity for IRQ test
        https://github.com/kvm-x86/linux/commit/380b0e5e0e30
[11/20] KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test
        https://github.com/kvm-x86/linux/commit/197b0cd9ac66
[12/20] KVM: selftests: Add option to set empty routing between IRQs in eventfd IRQ test
        https://github.com/kvm-x86/linux/commit/ec333e3ace25
[13/20] KVM: selftests: Make number of IRQs configurable in IRQ test
        https://github.com/kvm-x86/linux/commit/38950af64a1c
[14/20] KVM: selftests: Verify non-postable IRQ remapping in IRQ test
        https://github.com/kvm-x86/linux/commit/25e268b76e77
[15/20] KVM: selftests: Add kvm_gettid() wrapper and convert users
        https://github.com/kvm-x86/linux/commit/4512b2776631
[16/20] KVM: selftests: Add kvm_sched_getaffinity() wrapper and convert users
        https://github.com/kvm-x86/linux/commit/76703ca0cbab
[17/20] KVM: selftests: Add a utility to pin a task to a random CPU, given a CPU set
        https://github.com/kvm-x86/linux/commit/26fd1fb4737a
[18/20] KVM: selftests: Verify vCPU migration during IRQ delivery in IRQ test
        https://github.com/kvm-x86/linux/commit/6b9470baea6c
[19/20] KVM: selftests: Make number of vCPUs configurable in IRQ test
        https://github.com/kvm-x86/linux/commit/0315713a1938
[20/20] KVM: selftests: Add xAPIC support in eventfd IRQ test
        https://github.com/kvm-x86/linux/commit/2708ecca4dbd

--
https://github.com/kvm-x86/linux/tree/next