Skip to contentSkip to conversation

KVM: selftests: Link with VFIO selftests lib

PATCH Superseded Josh Hilke 2 participants 25 messages May 30 – Jun 1

Subsystems (4)

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Josh Hilke

author
#0 lore raw

v4 cover KVM: selftests: Link with VFIO selftests lib

This is v4 of a series which introduces tools/testing/selftests/kvm/irq_test.c in KVM selftests. This test exercises the delivery of interrupts (both emulated via eventfd and real from a VFIO device) to guest vCPUs. Beyond basic injection, the series adds coverage for several complex scenarios, including:
- Dynamic updates to KVM's GSI routing table while interrupts are active.
- Waking up halted vCPUs via interrupts.
- Stressing interrupt delivery during random host IRQ affinity changes.
- Stressing interrupt delivery during random vCPU thread migration across physical CPUs.
- Testing non-postable interrupt remapping (using NMIs to force transitions).
- Supporting both xAPIC and x2APIC modes in the guest.

The series also links the VFIO selftests library into KVM selftests to enable testing of VFIO-KVM interactions.

The test can optionally use a PCI device bound to vfio-pci to test physical device interrupts. If using a device, it can be invoked by passing the BDF to the VFIO selftests setup script, and then running the test with the device BDF passed via the -d option:

$ ./tools/testing/selftests/vfio/scripts/setup.sh 0000:6a:01.0
$ tools/testing/selftests/kvm/irq_test -d 0000:6a:01.0

This test only supports x86. Testing physical device interrupts (-d argument) requires a device with a supported VFIO selftest driver. Currently supported devices include:
- Intel DSA (Data Streaming Accelerator), 8086:0b25
- Intel IOAT (I/O Acceleration Technology), 8086:2021

The test can be run with following command-line arguments allow for broad coverage of the interrupt delivery path:
-a: Random IRQ Affinity. Randomly affinitizes the device IRQ to different host
CPUs to verify stable delivery during interrupt steering changes.
-b: Block vCPUs. Causes vCPUs to HLT instead of spinning, verifying that
Posted-interrupt wakeup (PIW) correctly kicks blocked vCPUs.
-c: Clear GSI Routes. Periodically destroys/recreates KVM's GSI routing table to
verify handling of dynamic IRQ updates.
-d: Device MSI Triggers. Uses the physical device to trigger MSIs instead of
eventfd emulation (requires a supported device driver).
-i: IRQ Count. Sets the number of interrupts to generate (default 1000). -m: vCPU Migration. Migrates vCPUs to random physical CPUs to verify that
posted interrupts follow the vCPU across host cores.
-n: NMI Delivery. Routes interrupts as NMIs into the guest to verify the
VFIO-NMI delivery path.
-v: vCPU Count. Distributes interrupts across multiple vCPUs via round-robin
routing.
-x: xAPIC Mode. Forces legacy xAPIC mode to verify compatibility.

---
Changelog

v3 -> v4
- Add context to short logs specifying if the change relates specifically to the IRQ test (Sean)
- Format code correctly (Sean)
- Fix compilation errors for gettid() wrapper (Sean)
- Add macros for GUEST_RECEIVED_NMI and GUEST_RECEIVED_IRQ (Sean)
- Split helper functions into separate patches (Sean)

David Matlack (12):
KVM: selftests: Build and link selftests/vfio/lib into KVM selftests
KVM: selftests: Add /proc/interrupts parsing helpers for IRQ test
KVM: selftests: Add guest read/write macros
KVM: selftests: Add IRQ injection test
KVM: selftests: Verify IRQ bypass works in IRQ test
KVM: selftests: Verify interrupts are received when IRQ affinity
changes in IRQ test
KVM: selftests: Verify IRQs wake up halted vCPUs in IRQ test
KVM: selftests: Verify interrupts are received after modifying IRQ
routes in IRQ test
KVM: selftests: Make number of IRQs configurable in IRQ test
KVM: selftests: Verify non-postable IRQ remapping in IRQ test
KVM: selftests: Make number of vCPUs configurable in IRQ test
KVM: selftests: Add xAPIC support in IRQ test

Josh Hilke (7):
KVM: selftests: Rename guest_rng to kvm_rng
KVM: selftests: Add helper to generate random u64 in range [min,max]
KVM: selftests: Add kvm_gettid() wrapper and convert users
KVM: selftests: Add kvm_sched_getaffinity() wrapper and convert users
KVM: selftests: Add pin_task_to_random_cpu() helper function
KVM: selftests: Verify vCPU migration during IRQ delivery in IRQ test
KVM: selftests: Print vCPU affinity on timeout during IRQ test

 tools/testing/selftests/kvm/Makefile.kvm      |   8 +-
 tools/testing/selftests/kvm/arch_timer.c      |   2 +-
 .../kvm/arm64/arch_timer_edge_cases.c         |   2 +-
 .../selftests/kvm/demand_paging_test.c        |   2 +-
 .../selftests/kvm/dirty_log_perf_test.c       |   4 +-
 tools/testing/selftests/kvm/dirty_log_test.c  |   8 +-
 .../selftests/kvm/include/kvm_syscalls.h      |   6 +
 .../testing/selftests/kvm/include/kvm_util.h  |  13 +
 .../testing/selftests/kvm/include/proc_util.h |  28 ++
 .../testing/selftests/kvm/include/test_util.h |  25 +-
 .../selftests/kvm/include/x86/kvm_util_arch.h |   4 +-
 tools/testing/selftests/kvm/irq_test.c        | 370 ++++++++++++++++++
 tools/testing/selftests/kvm/lib/assert.c      |   8 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    |  95 ++++-
 tools/testing/selftests/kvm/lib/memstress.c   |   8 +-
 tools/testing/selftests/kvm/lib/proc_util.c   |  62 +++
 tools/testing/selftests/kvm/lib/test_util.c   |  27 +-
 tools/testing/selftests/kvm/mmu_stress_test.c |  15 +-
 tools/testing/selftests/kvm/rseq_test.c       |   6 +-
 tools/testing/selftests/kvm/steal_time.c      |  22 +-
 20 files changed, 635 insertions(+), 80 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/include/proc_util.h
 create mode 100644 tools/testing/selftests/kvm/irq_test.c
 create mode 100644 tools/testing/selftests/kvm/lib/proc_util.c
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#1 lore raw

PATCH v4 1/19 KVM: selftests: Build and link selftests/vfio/lib into KVM selftests

From: David Matlack <[email protected]>

Include libvfio.mk into the KVM selftests Makefile and link it into all KVM selftests by adding it to LIBKVM_OBJS.

This lays the groundwork for future changes to utilize VFIO devices to verify IRQ bypass in KVM selftests.

Note that KVM selftests build their own copy of sefltests/vfio/lib and the resulting object files are placed in $(OUTPUT)/lib. This allows the KVM and VFIO selftests to apply different CFLAGS when building without conflicting with each other.

---
 tools/testing/selftests/kvm/Makefile.kvm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/Makefile.kvm +4 -1
index 6471fa214a9f..b3e223f45575 100644
@@ -252,6 +252,7 @@ OVERRIDE_TARGETS = 1
 # which causes the environment variable to override the makefile).
 include ../lib.mk
 include ../cgroup/lib/libcgroup.mk
+include ../vfio/lib/libvfio.mk
 
 INSTALL_HDR_PATH = $(top_srcdir)/usr
 LINUX_HDR_PATH = $(INSTALL_HDR_PATH)/include/
@@ -306,7 +307,9 @@ LIBKVM_S := $(filter %.S,$(LIBKVM))
 LIBKVM_C_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_C))
 LIBKVM_S_OBJ := $(patsubst %.S, $(OUTPUT)/%.o, $(LIBKVM_S))
 LIBKVM_STRING_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_STRING))
-LIBKVM_OBJS = $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ) $(LIBKVM_STRING_OBJ) $(LIBCGROUP_O)
+LIBKVM_OBJS = $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ) $(LIBKVM_STRING_OBJ)
+LIBKVM_OBJS += $(LIBCGROUP_O)
+LIBKVM_OBJS += $(LIBVFIO_O)
 SPLIT_TEST_GEN_PROGS := $(patsubst %, $(OUTPUT)/%, $(SPLIT_TESTS))
 SPLIT_TEST_GEN_OBJ := $(patsubst %, $(OUTPUT)/$(ARCH)/%.o, $(SPLIT_TESTS))
 
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#2 lore raw

PATCH v4 2/19 KVM: selftests: Add /proc/interrupts parsing helpers for IRQ test

In reply to Josh Hilke

From: David Matlack <[email protected]>

Introduce utility functions to parse /proc/interrupts and extract the host IRQ number for a given VFIO device and MSI vector.

These helpers lay the groundwork for subsequent patches that will creat a KVM IRQ test and use real VFIO devices to trigger hardware interrupts. Specifically, they are needed to allow the test to log the exact IRQ number being used for better debugging.

---
 tools/testing/selftests/kvm/Makefile.kvm      |  1 +
 .../testing/selftests/kvm/include/proc_util.h |  9 ++++
 tools/testing/selftests/kvm/lib/proc_util.c   | 42 +++++++++++++++++++
 3 files changed, 52 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/include/proc_util.h
 create mode 100644 tools/testing/selftests/kvm/lib/proc_util.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
index b3e223f45575..d944b81cad7d 100644
@@ -11,6 +11,7 @@ LIBKVM += lib/kvm_util.c
 LIBKVM += lib/lru_gen_util.c
 LIBKVM += lib/memstress.c
 LIBKVM += lib/guest_sprintf.c
+LIBKVM += lib/proc_util.c
 LIBKVM += lib/rbtree.c
 LIBKVM += lib/sparsebit.c
 LIBKVM += lib/test_util.c
tools/testing/selftests/kvm/include/proc_util.h new file +9 -0
new file mode 100644
index 000000000000..7c465e8584e2
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef SELFTEST_KVM_PROC_UTIL_H
+#define SELFTEST_KVM_PROC_UTIL_H
+
+#include <stdint.h>
+
+int get_proc_vfio_irq_number(const char *vfio_device_bdf, int msi);
+
+#endif /* SELFTEST_KVM_PROC_UTIL_H */
tools/testing/selftests/kvm/lib/proc_util.c new file +42 -0

42 lines not on this page · raw message

Signature · 1 line
2.54.0.929.g9b7fa37559-goog
#3 lore raw

Re: [PATCH v4 02/19] KVM: selftests: Add /proc/interrupts parsing helpers for IRQ test

tools/testing/selftests/kvm/include/proc_util.h line 7

Quoted 11 lines · Josh Hilke

index 000000000000..7c465e8584e2

tools/testing/selftests/kvm/include/proc_util.h new file +7 -0
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef SELFTEST_KVM_PROC_UTIL_H
+#define SELFTEST_KVM_PROC_UTIL_H
+
+#include <stdint.h>
+
+int get_proc_vfio_irq_number(const char *vfio_device_bdf, int msi);

I feel like the "proc" is confusing and unintesting. How the utility gets the information is an implementation detail that I don't think the caller particular cares about.

"irq_number" is also vague. There are myriad IRQ "numbers" in play. "virtual IRQ", as they're sometimes described by the IRQ subsystem, is unfortunately terrible terminology for KVM+VFIO. host_irq and linux_irq are the two names I can think of that are fairly specific without too much potential for confusion. I'd probably vote for host_irq?

I also think the function needs to have msix in the name, because within VFIO, there are multiple flavors of interrupts that can be wired up, and this _only_ works for MSI-X. And The param should be @msix.

I also think it might make sense to return an "unsigned int", to communicate that it guarantees a valid IRQ was found, but I don't care terribly either way.

Maybe something like this?

unsigned int vfio_msix_to_host_irq(const char *vfio_device_bdf, int msix)

or

unsigned int vfio_get_msix_host_irq(const char *vfio_device_bdf, int msix)

or

unsigned int vfio_get_host_irq_for_msix(const char *vfio_device_bdf, int msix)

I'd probably vote for the first one, vfio_msix_to_host_irq(), as it's more obviously "just" a translation helper, e.g. isn't allocating an IRQ or anything.

author
#4 lore raw

PATCH v4 3/19 KVM: selftests: Add guest read/write macros

In reply to Josh Hilke

From: David Matlack <[email protected]>

Add SYNC_FROM_GUEST_AND_READ(vm, variable), to read a variable value from the guest. Add WRITE_AND_SYNC_TO_GUEST(vm, variable) to write a value to a guest variable. These macros improve the readability of code which reads and writes data between host and guest in tests.

Also update existing guest reads/writes to use these macros in the following tests:
- tools/testing/selftests/kvm/dirty_log_test.c
- tools/testing/selftests/kvm/mmu_stress_test.c
- tools/testing/selftests/kvm/steal_time.c

No functional changes are intended.

---
 tools/testing/selftests/kvm/dirty_log_test.c  |  6 ++---
 .../testing/selftests/kvm/include/kvm_util.h  | 10 +++++++++
 tools/testing/selftests/kvm/mmu_stress_test.c |  9 +++-----
 tools/testing/selftests/kvm/steal_time.c      | 22 +++++++------------
 4 files changed, 23 insertions(+), 24 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +2 -4
index d58a641b0e6a..66c3d9554379 100644
@@ -764,16 +764,14 @@ static void run_test(enum vm_guest_mode mode, void *arg)
 		 * writing memory during verification, pages that this thread
 		 * sees as clean may be written with this iteration's value.
 		 */
-		WRITE_ONCE(vcpu_stop, true);
-		sync_global_to_guest(vm, vcpu_stop);
+		WRITE_AND_SYNC_TO_GUEST(vm, vcpu_stop, true);
 		sem_wait(&sem_vcpu_stop);
 
 		/*
 		 * Clear vcpu_stop after the vCPU thread has acknowledge the
 		 * stop request and is waiting, i.e. is definitely not running!
 		 */
-		WRITE_ONCE(vcpu_stop, false);
-		sync_global_to_guest(vm, vcpu_stop);
+		WRITE_AND_SYNC_TO_GUEST(vm, vcpu_stop, false);
 
 		/*
 		 * Sync the number of writes performed before verification, the
tools/testing/selftests/kvm/include/kvm_util.h +10 -0
index 8b39cb919f4f..499e695f1fc4 100644
@@ -1144,6 +1144,16 @@ vm_adjust_num_guest_pages(enum vm_guest_mode mode, unsigned int num_guest_pages)
 	memcpy(&(g), _p, sizeof(g));				\
 })
 
+#define SYNC_FROM_GUEST_AND_READ(_vm, _variable) ({		\
+	sync_global_from_guest(_vm, _variable);			\
+	READ_ONCE(_variable);					\
+})
+
+#define WRITE_AND_SYNC_TO_GUEST(_vm, _variable, _value) do {	\
+	WRITE_ONCE(_variable, _value);				\
+	sync_global_to_guest(_vm, _variable);			\
+} while (0)
+
 /*
  * Write a global value, but only in the VM's (guest's) domain.  Primarily used
  * for "globals" that hold per-VM values (VMs always duplicate code and global
tools/testing/selftests/kvm/mmu_stress_test.c +3 -6
index 51c070556f3e..a9fac4bc02c2 100644
@@ -155,10 +155,8 @@ static void *vcpu_worker(void *data)
 		    "Expected EFAULT on write to RO memory, got r = %d, errno = %d", r, errno);
 
 	atomic_inc(&nr_ro_faults);
-	if (atomic_read(&nr_ro_faults) == nr_vcpus) {
-		WRITE_ONCE(all_vcpus_hit_ro_fault, true);
-		sync_global_to_guest(vm, all_vcpus_hit_ro_fault);
-	}
+	if (atomic_read(&nr_ro_faults) == nr_vcpus)
+		WRITE_AND_SYNC_TO_GUEST(vm, all_vcpus_hit_ro_fault, true);
 
 #if defined(__x86_64__) || defined(__aarch64__)
 	/*
@@ -383,8 +381,7 @@ int main(int argc, char *argv[])
 	rendezvous_with_vcpus(&time_run2, "run 2");
 
 	mprotect(mem, slot_size, PROT_READ);
-	mprotect_ro_done = true;
-	sync_global_to_guest(vm, mprotect_ro_done);
+	WRITE_AND_SYNC_TO_GUEST(vm, mprotect_ro_done, true);
 
 	rendezvous_with_vcpus(&time_ro, "mprotect RO");
 	mprotect(mem, slot_size, PROT_READ | PROT_WRITE);
tools/testing/selftests/kvm/steal_time.c +8 -14
index 7be8adfe5dd3..a866927581fa 100644
@@ -72,8 +72,8 @@ static void steal_time_init(struct kvm_vcpu *vcpu, uint32_t i)
 	int ret;
 
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	sync_global_to_guest(vcpu->vm, st_gva[i]);
+	WRITE_AND_SYNC_TO_GUEST(vcpu->vm, st_gva[i],
+				(void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
 
 	ret = _vcpu_set_msr(vcpu, MSR_KVM_STEAL_TIME,
 			    (ulong)st_gva[i] | KVM_STEAL_RESERVED_MASK);
@@ -181,8 +181,7 @@ static void steal_time_init(struct kvm_vcpu *vcpu, uint32_t i)
 	vcpu_ioctl(vcpu, KVM_HAS_DEVICE_ATTR, &dev);
 
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	sync_global_to_guest(vm, st_gva[i]);
+	WRITE_AND_SYNC_TO_GUEST(vm, st_gva[i], (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
 
 	st_ipa = (ulong)st_gva[i] | 1;
 	ret = __vcpu_ioctl(vcpu, KVM_SET_DEVICE_ATTR, &dev);
@@ -279,10 +278,8 @@ static bool is_steal_time_supported(struct kvm_vcpu *vcpu)
 static void steal_time_init(struct kvm_vcpu *vcpu, uint32_t i)
 {
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	st_gpa[i] = addr_gva2gpa(vcpu->vm, (vm_vaddr_t)st_gva[i]);
-	sync_global_to_guest(vcpu->vm, st_gva[i]);
-	sync_global_to_guest(vcpu->vm, st_gpa[i]);
+	WRITE_AND_SYNC_TO_GUEST(vcpu->vm, st_gva[i], (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
+	WRITE_AND_SYNC_TO_GUEST(vcpu->vm, st_gpa[i], addr_gva2gpa(vcpu->vm, (vm_vaddr_t)st_gva[i]));
 }
 
 static void steal_time_dump(struct kvm_vm *vm, uint32_t vcpu_idx)
@@ -376,8 +373,7 @@ static void steal_time_init(struct kvm_vcpu *vcpu, uint32_t i)
 	};
 
 	/* ST_GPA_BASE is identity mapped */
-	st_gva[i] = (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE);
-	sync_global_to_guest(vm, st_gva[i]);
+	WRITE_AND_SYNC_TO_GUEST(vm, st_gva[i], (void *)(ST_GPA_BASE + i * STEAL_TIME_SIZE));
 
 	err = __vcpu_ioctl(vcpu, KVM_HAS_DEVICE_ATTR, &attr);
 	TEST_ASSERT(err == 0, "No PV stealtime Feature");
@@ -476,8 +472,7 @@ int main(int ac, char **av)
 
 		/* Second VCPU run, expect guest stolen time to be <= run_delay */
 		run_vcpu(vcpus[i]);
-		sync_global_from_guest(vm, guest_stolen_time[i]);
-		stolen_time = guest_stolen_time[i];
+		stolen_time = SYNC_FROM_GUEST_AND_READ(vm, guest_stolen_time[i]);
 		run_delay = get_run_delay();
 		TEST_ASSERT(stolen_time <= run_delay,
 			    "Expected stolen time <= %ld, got %ld",
@@ -497,8 +492,7 @@ int main(int ac, char **av)
 
 		/* Run VCPU again to confirm stolen time is consistent with run_delay */
 		run_vcpu(vcpus[i]);
-		sync_global_from_guest(vm, guest_stolen_time[i]);
-		stolen_time = guest_stolen_time[i] - stolen_time;
+		stolen_time = SYNC_FROM_GUEST_AND_READ(vm, guest_stolen_time[i]) - stolen_time;
 		TEST_ASSERT(stolen_time >= run_delay,
 			    "Expected stolen time >= %ld, got %ld",
 			    run_delay, stolen_time);
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#5 lore raw

PATCH v4 4/19 KVM: selftests: Rename guest_rng to kvm_rng

In reply to Josh Hilke

Rename functions prefixed with 'guest_random_' to 'kvm_random_' and the global random state variable 'guest_rng' to 'kvm_rng'. This expands their usage and reflects that the random number generator can be used by both the host and the guest.

No functional changes are intended.

---
 .../selftests/kvm/dirty_log_perf_test.c       |  4 ++--
 tools/testing/selftests/kvm/dirty_log_test.c  |  2 +-
 .../testing/selftests/kvm/include/test_util.h | 22 +++++++++----------
 .../selftests/kvm/include/x86/kvm_util_arch.h |  4 ++--
 tools/testing/selftests/kvm/lib/kvm_util.c    | 20 ++++++++---------
 tools/testing/selftests/kvm/lib/memstress.c   |  8 +++----
 tools/testing/selftests/kvm/lib/test_util.c   |  6 ++---
 7 files changed, 33 insertions(+), 33 deletions(-)
tools/testing/selftests/kvm/dirty_log_perf_test.c +2 -2
index 0a1ea1d1e2d8..606abd858a12 100644
@@ -311,7 +311,7 @@ int main(int argc, char *argv[])
 	int opt;
 
 	/* Override the seed to be deterministic by default. */
-	guest_random_seed = 1;
+	kvm_random_seed = 1;
 
 	dirty_log_manual_caps =
 		kvm_check_cap(KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2);
@@ -357,7 +357,7 @@ int main(int argc, char *argv[])
 			p.phys_offset = strtoull(optarg, NULL, 0);
 			break;
 		case 'r':
-			guest_random_seed = atoi_positive("Random seed", optarg);
+			kvm_random_seed = atoi_positive("Random seed", optarg);
 			break;
 		case 's':
 			p.backing_src = parse_backing_src_type(optarg);
tools/testing/selftests/kvm/dirty_log_test.c +1 -1
index 66c3d9554379..cddf56400e6c 100644
@@ -121,7 +121,7 @@ static void guest_code(void)
 	while (true) {
 		while (!READ_ONCE(vcpu_stop)) {
 			addr = guest_test_virt_mem;
-			addr += (guest_random_u64(&guest_rng) % guest_num_pages)
+			addr += (kvm_random_u64(&kvm_rng) % guest_num_pages)
 				* guest_page_size;
 			addr = align_down(addr, host_page_size);
 
tools/testing/selftests/kvm/include/test_util.h +11 -11
index b4872ba8ed12..ae39c4293b9a 100644
@@ -106,30 +106,30 @@ struct timespec timespec_sub(struct timespec ts1, struct timespec ts2);
 struct timespec timespec_elapsed(struct timespec start);
 struct timespec timespec_div(struct timespec ts, int divisor);
 
-struct guest_random_state {
+struct kvm_random_state {
 	uint32_t seed;
 };
 
-extern uint32_t guest_random_seed;
-extern struct guest_random_state guest_rng;
+extern uint32_t kvm_random_seed;
+extern struct kvm_random_state kvm_rng;
 
-struct guest_random_state new_guest_random_state(uint32_t seed);
-uint32_t guest_random_u32(struct guest_random_state *state);
+struct kvm_random_state new_kvm_random_state(uint32_t seed);
+uint32_t kvm_random_u32(struct kvm_random_state *state);
 
-static inline bool __guest_random_bool(struct guest_random_state *state,
+static inline bool __kvm_random_bool(struct kvm_random_state *state,
 				       uint8_t percent)
 {
-	return (guest_random_u32(state) % 100) < percent;
+	return (kvm_random_u32(state) % 100) < percent;
 }
 
-static inline bool guest_random_bool(struct guest_random_state *state)
+static inline bool kvm_random_bool(struct kvm_random_state *state)
 {
-	return __guest_random_bool(state, 50);
+	return __kvm_random_bool(state, 50);
 }
 
-static inline uint64_t guest_random_u64(struct guest_random_state *state)
+static inline uint64_t kvm_random_u64(struct kvm_random_state *state)
 {
-	return ((uint64_t)guest_random_u32(state) << 32) | guest_random_u32(state);
+	return ((uint64_t)kvm_random_u32(state) << 32) | kvm_random_u32(state);
 }
 
 enum vm_mem_backing_src_type {
tools/testing/selftests/kvm/include/x86/kvm_util_arch.h +2 -2
index be35d26bb320..63a12ef9008e 100644
@@ -55,9 +55,9 @@ static inline bool __vm_arch_has_protected_memory(struct kvm_vm_arch *arch)
 do {											\
 	const typeof(mem) val = (__val);						\
 											\
-	if (!is_forced_emulation_enabled || guest_random_bool(&guest_rng)) {		\
+	if (!is_forced_emulation_enabled || kvm_random_bool(&kvm_rng)) {		\
 		(mem) = val;								\
-	} else if (guest_random_bool(&guest_rng)) {					\
+	} else if (kvm_random_bool(&kvm_rng)) {					\
 		__asm__ __volatile__(KVM_FEP "mov %1, %0"				\
 				     : "+m" (mem)					\
 				     : "r" (val) : "memory");				\
tools/testing/selftests/kvm/lib/kvm_util.c +10 -10
index 1959bf556e88..b247b2015b2a 100644
@@ -20,9 +20,9 @@
 
 #define KVM_UTIL_MIN_PFN	2
 
-uint32_t guest_random_seed;
-struct guest_random_state guest_rng;
-static uint32_t last_guest_seed;
+uint32_t kvm_random_seed;
+struct kvm_random_state kvm_rng;
+static uint32_t last_kvm_seed;
 
 static size_t vcpu_mmap_sz(void);
 
@@ -515,12 +515,12 @@ struct kvm_vm *__vm_create(struct vm_shape shape, uint32_t nr_runnable_vcpus,
 	slot0 = memslot2region(vm, 0);
 	ucall_init(vm, slot0->region.guest_phys_addr + slot0->region.memory_size);
 
-	if (guest_random_seed != last_guest_seed) {
-		pr_info("Random seed: 0x%x\n", guest_random_seed);
-		last_guest_seed = guest_random_seed;
+	if (kvm_random_seed != last_kvm_seed) {
+		pr_info("Random seed: 0x%x\n", kvm_random_seed);
+		last_kvm_seed = kvm_random_seed;
 	}
-	guest_rng = new_guest_random_state(guest_random_seed);
-	sync_global_to_guest(vm, guest_rng);
+	kvm_rng = new_kvm_random_state(kvm_random_seed);
+	sync_global_to_guest(vm, kvm_rng);
 
 	kvm_arch_vm_post_create(vm, nr_runnable_vcpus);
 
@@ -2358,8 +2358,8 @@ void __attribute((constructor)) kvm_selftest_init(void)
 	sigaction(SIGILL, &sig_sa, NULL);
 	sigaction(SIGFPE, &sig_sa, NULL);
 
-	guest_random_seed = last_guest_seed = random();
-	pr_info("Random seed: 0x%x\n", guest_random_seed);
+	kvm_random_seed = last_kvm_seed = random();
+	pr_info("Random seed: 0x%x\n", kvm_random_seed);
 
 	kvm_selftest_arch_init();
 }
tools/testing/selftests/kvm/lib/memstress.c +4 -4
index 557c0a0a5658..c5df9c99102a 100644
@@ -48,14 +48,14 @@ void memstress_guest_code(uint32_t vcpu_idx)
 {
 	struct memstress_args *args = &memstress_args;
 	struct memstress_vcpu_args *vcpu_args = &args->vcpu_args[vcpu_idx];
-	struct guest_random_state rand_state;
+	struct kvm_random_state rand_state;
 	uint64_t gva;
 	uint64_t pages;
 	uint64_t addr;
 	uint64_t page;
 	int i;
 
-	rand_state = new_guest_random_state(guest_random_seed + vcpu_idx);
+	rand_state = new_kvm_random_state(kvm_random_seed + vcpu_idx);
 
 	gva = vcpu_args->gva;
 	pages = vcpu_args->pages;
@@ -69,13 +69,13 @@ void memstress_guest_code(uint32_t vcpu_idx)
 
 		for (i = 0; i < pages; i++) {
 			if (args->random_access)
-				page = guest_random_u32(&rand_state) % pages;
+				page = kvm_random_u32(&rand_state) % pages;
 			else
 				page = i;
 
 			addr = gva + (page * args->guest_page_size);
 
-			if (__guest_random_bool(&rand_state, args->write_percent))
+			if (__kvm_random_bool(&rand_state, args->write_percent))
 				*(uint64_t *)addr = 0x0123456789ABCDEF;
 			else
 				READ_ONCE(*(uint64_t *)addr);
tools/testing/selftests/kvm/lib/test_util.c +3 -3
index 8a1848586a85..e342d9fb4771 100644
@@ -30,13 +30,13 @@ void __attribute__((used)) expect_sigbus_handler(int signum)
  * Park-Miller LCG using standard constants.
  */
 
-struct guest_random_state new_guest_random_state(uint32_t seed)
+struct kvm_random_state new_kvm_random_state(uint32_t seed)
 {
-	struct guest_random_state s = {.seed = seed};
+	struct kvm_random_state s = {.seed = seed};
 	return s;
 }
 
-uint32_t guest_random_u32(struct guest_random_state *state)
+uint32_t kvm_random_u32(struct kvm_random_state *state)
 {
 	state->seed = (uint64_t)state->seed * 48271 % ((uint32_t)(1 << 31) - 1);
 	return state->seed;
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#6 lore raw

PATCH v4 5/19 KVM: selftests: Add helper to generate random u64 in range [min,max]

In reply to Josh Hilke

Introduce kvm_random_u64_in_range(state, min, max). This function returns a random u64 in the inclusive range of [min, max] using a struct kvm_random_state.

---
 .../testing/selftests/kvm/include/test_util.h  |  3 +++
 tools/testing/selftests/kvm/lib/test_util.c    | 18 ++++++++++++++++++
 2 files changed, 21 insertions(+)
tools/testing/selftests/kvm/include/test_util.h +3 -0
index ae39c4293b9a..d601227dbdba 100644
@@ -132,6 +132,9 @@ static inline uint64_t kvm_random_u64(struct kvm_random_state *state)
 	return ((uint64_t)kvm_random_u32(state) << 32) | kvm_random_u32(state);
 }
 
+uint64_t kvm_random_u64_in_range(struct kvm_random_state *state, uint64_t min,
+				 uint64_t max);
+
 enum vm_mem_backing_src_type {
 	VM_MEM_SRC_ANONYMOUS,
 	VM_MEM_SRC_ANONYMOUS_THP,
tools/testing/selftests/kvm/lib/test_util.c +18 -0
index e342d9fb4771..dc47ed9736e9 100644
@@ -42,6 +42,24 @@ uint32_t kvm_random_u32(struct kvm_random_state *state)
 	return state->seed;
 }
 
+/* Returns a random u64 in the inclusive range [min, max] */
+uint64_t kvm_random_u64_in_range(struct kvm_random_state *state, uint64_t min,
+				 uint64_t max)
+{
+	uint64_t value;
+	uint64_t range;
+
+	TEST_ASSERT(min <= max, "PEBKAC, min = 0x%lx, max = 0x%lx", min, max);
+
+	value = kvm_random_u64(state);
+
+	range = max - min;
+	if (range == ULLONG_MAX)
+		return value;
+
+	return min + (value % (range + 1));
+}
+
 /*
  * Parses "[0-9]+[kmgt]?".
  */
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#7 lore raw

PATCH v4 6/19 KVM: selftests: Add IRQ injection test

In reply to Josh Hilke

From: David Matlack <[email protected]>

Add a new test, irq_test.c, which verifies that KVM correctly injects interrupts into a running guest when triggered via an eventfd bound to a GSI using the irqfd mechanism.

This test is intentionally simple. Support for IRQ bypass, receiving interrupts from real devices, and other features will be added to this test in future changes.

---
 tools/testing/selftests/kvm/Makefile.kvm |   2 +
 tools/testing/selftests/kvm/irq_test.c   | 174 +++++++++++++++++++++++
 2 files changed, 176 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/irq_test.c
tools/testing/selftests/kvm/Makefile.kvm +2 -0
index d944b81cad7d..9e5a00699dd0 100644
@@ -149,6 +149,7 @@ TEST_GEN_PROGS_x86 += coalesced_io_test
 TEST_GEN_PROGS_x86 += dirty_log_perf_test
 TEST_GEN_PROGS_x86 += guest_memfd_test
 TEST_GEN_PROGS_x86 += hardware_disable_test
+TEST_GEN_PROGS_x86 += irq_test
 TEST_GEN_PROGS_x86 += memslot_modification_stress_test
 TEST_GEN_PROGS_x86 += memslot_perf_test
 TEST_GEN_PROGS_x86 += mmu_stress_test
@@ -157,6 +158,7 @@ TEST_GEN_PROGS_x86 += steal_time
 TEST_GEN_PROGS_x86 += system_counter_offset_test
 TEST_GEN_PROGS_x86 += pre_fault_memory_test
 
+
 # Compiled outputs used by test targets
 TEST_GEN_PROGS_EXTENDED_x86 += x86/nx_huge_pages_test
 
tools/testing/selftests/kvm/irq_test.c new file +174 -0

174 lines not on this page · raw message

Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#8 lore raw

PATCH v4 7/19 KVM: selftests: Verify IRQ bypass works in IRQ test

In reply to Josh Hilke

From: David Matlack <[email protected]>

Trigger interrupts from a VFIO device instead of emulating interrupts using KVM eventfds. This verifies that guests receive interrupts via IRQ bypass.

Add a '-d' argument to tools/testing/selftests/kvm/irq_test which takes the segment:bus:device.function number of a PCI device bound to VFIO which will trigger interrupts. The device must have a VFIO selftest driver in order to work with the test.

Example:

$ ./tools/testing/selftests/kvm/irq_test -d 0000:06:0a.1
---
 tools/testing/selftests/kvm/irq_test.c | 61 +++++++++++++++++++++++---
 1 file changed, 55 insertions(+), 6 deletions(-)
tools/testing/selftests/kvm/irq_test.c +55 -6
index d3fee51cf451..616eb38c05b8 100644
@@ -1,14 +1,17 @@
 // SPDX-License-Identifier: GPL-2.0
 #include "kvm_util.h"
 #include "test_util.h"
+#include <linux/sizes.h>
 #include "apic.h"
 #include "processor.h"
+#include "proc_util.h"
 
 #include <stdio.h>
 #include <stdlib.h>
 #include <unistd.h>
 #include <pthread.h>
 #include <sys/eventfd.h>
+#include <libvfio.h>
 
 static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;
 static bool guest_ready_for_irqs[KVM_MAX_VCPUS];
@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)
 	return NULL;
 }
 
+static int vfio_setup_msi(struct vfio_pci_device *device)
+{
+	const int flags = MAP_SHARED | MAP_ANONYMOUS;
+	const int prot = PROT_READ | PROT_WRITE;
+	struct dma_region *region;
+
+	/* A driver is required to generate an MSI. */
+	TEST_REQUIRE(device->driver.ops);
+
+	/* Set up a DMA-able region for the driver to use. */
+	region = &device->driver.region;
+	region->iova = 0;
+	region->size = SZ_2M;
+	region->vaddr = kvm_mmap(region->size, prot, flags, -1);
+	TEST_ASSERT(region->vaddr != MAP_FAILED, "mmap() failed\n");
+	iommu_map(device->iommu, region);
+
+	vfio_pci_driver_init(device);
+	return device->driver.msi;
+}
+
+static void trigger_interrupt(struct vfio_pci_device *device, int eventfd)
+{
+	if (device)
+		vfio_pci_driver_send_msi(device);
+	else
+		eventfd_write(eventfd, 1);
+}
+
+
 static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 			  u8 vector)
 {
@@ -76,9 +109,10 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-h]\n", name);
+	printf("Usage: %s [-d <segment:bus:device.function>] [-h]\n", name);
 	printf("\n");
 	printf("Tests KVM IRQ injection via irqfd using an emulated eventfd.\n");
+	printf("-d	Use a VFIO device to send MSI-X interrupts instead of using an emulated eventfd\n");
 	printf("\n");
 	exit(KSFT_FAIL);
 }
@@ -102,12 +136,18 @@ int main(int argc, char **argv)
 
 	struct kvm_vcpu *vcpus[KVM_MAX_VCPUS];
 	pthread_t vcpu_threads[KVM_MAX_VCPUS];
+	struct vfio_pci_device *device = NULL;
 	int nr_irqs = 1000, nr_vcpus = 1;
-	int i, j, c, eventfd;
+	int i, j, c, msi, irq, eventfd;
+	const char *device_bdf = NULL;
+	struct iommu *iommu;
 	struct kvm_vm *vm;
 
-	while ((c = getopt(argc, argv, "h")) != -1) {
+	while ((c = getopt(argc, argv, "d:h")) != -1) {
 		switch (c) {
+		case 'd':
+			device_bdf = optarg;
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
@@ -119,7 +159,17 @@ int main(int argc, char **argv)
 	vm = vm_create_with_vcpus(nr_vcpus, guest_code, vcpus);
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
 
-	eventfd = kvm_new_eventfd();
+	if (device_bdf) {
+		iommu = iommu_init(default_iommu_mode);
+		device = vfio_pci_device_init(device_bdf, iommu);
+		msi = vfio_setup_msi(device);
+		irq = get_proc_vfio_irq_number(device_bdf, msi);
+		eventfd = device->msi_eventfds[msi];
+		printf("Using device %s MSI-X[%d] (IRQ-%d)\n", device_bdf, msi,
+		       irq);
+	} else {
+		eventfd = kvm_new_eventfd();
+	}
 
 	printf("Injecting interrupts for GSI %d (Vector 0x%x) %d times\n",
 	       gsi, vector, nr_irqs);
@@ -147,8 +197,7 @@ int main(int argc, char **argv)
 				    "IRQ flag for vCPU %d not clear prior to test",
 				    vcpus[j]->id);
 
-		/* Trigger interrupt */
-		eventfd_write(eventfd, 1);
+		trigger_interrupt(device, eventfd);
 
 		clock_gettime(CLOCK_MONOTONIC, &start);
 		for (;;) {
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#9 lore raw

PATCH v4 8/19 KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test

In reply to Josh Hilke

From: David Matlack <[email protected]>

Add the '-a' flag to tools/testing/selftests/kvm/irq_test.c to randomly affinitize the device's host IRQ to different physical CPUs throughout the test. This stresses the kernel's ability to maintain correct interrupt routing and delivery even as the underlying hardware IRQ affinity is changed dynamically via /proc/irq/.

---
 .../testing/selftests/kvm/include/proc_util.h | 19 +++++++++++
 tools/testing/selftests/kvm/irq_test.c        | 34 ++++++++++++++++---
 tools/testing/selftests/kvm/lib/proc_util.c   | 20 +++++++++++
 3 files changed, 69 insertions(+), 4 deletions(-)
tools/testing/selftests/kvm/include/proc_util.h +19 -0
index 7c465e8584e2..2c8086158f68 100644
@@ -6,4 +6,23 @@
 
 int get_proc_vfio_irq_number(const char *vfio_device_bdf, int msi);
 
+/*
+ * open_proc_irq_affinity - Open the smp_affinity_list file for a given IRQ
+ * @irq: The IRQ number
+ *
+ * Opens /proc/irq/<irq>/smp_affinity_list for writing and returns the FILE
+ * pointer.
+ */
+FILE *open_proc_irq_affinity(int irq);
+
+/*
+ * write_proc_irq_affinity - Write a CPU number to the smp_affinity_list file
+ * @fp: The FILE pointer for the smp_affinity_list file
+ * @irq: The IRQ number (for error reporting)
+ * @irq_cpu: The CPU number to write
+ *
+ * Writes the given CPU number to the provided FILE pointer.
+ */
+void write_proc_irq_affinity(FILE *fp, int irq, int irq_cpu);
+
 #endif /* SELFTEST_KVM_PROC_UTIL_H */
tools/testing/selftests/kvm/irq_test.c +30 -4
index 616eb38c05b8..e76784e0a054 100644
@@ -11,11 +11,13 @@
 #include <unistd.h>
 #include <pthread.h>
 #include <sys/eventfd.h>
+#include <sys/sysinfo.h>
 #include <libvfio.h>
 
 static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;
 static bool guest_ready_for_irqs[KVM_MAX_VCPUS];
 static bool guest_received_irq[KVM_MAX_VCPUS];
+static bool irq_affinity;
 static bool done;
 
 #define GUEST_RECEIVED_IRQ(__vcpu)	\
@@ -109,9 +111,10 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-d <segment:bus:device.function>] [-h]\n", name);
+	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-h]\n", name);
 	printf("\n");
 	printf("Tests KVM IRQ injection via irqfd using an emulated eventfd.\n");
+	printf("-a	Randomly affinitize the device's host IRQ to different physical CPUs throughout the test\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of using an emulated eventfd\n");
 	printf("\n");
 	exit(KSFT_FAIL);
@@ -134,17 +137,21 @@ int main(int argc, char **argv)
 	u32 gsi = kvm_random_u64_in_range(&kvm_rng, 24, KVM_MAX_IRQ_ROUTES - 1);
 	u8 vector = kvm_random_u64_in_range(&kvm_rng, 32, UINT8_MAX);
 
+	int i, j, c, msi, irq, eventfd, irq_cpu;
 	struct kvm_vcpu *vcpus[KVM_MAX_VCPUS];
 	pthread_t vcpu_threads[KVM_MAX_VCPUS];
 	struct vfio_pci_device *device = NULL;
 	int nr_irqs = 1000, nr_vcpus = 1;
-	int i, j, c, msi, irq, eventfd;
 	const char *device_bdf = NULL;
+	FILE *irq_affinity_fp = NULL;
 	struct iommu *iommu;
 	struct kvm_vm *vm;
 
-	while ((c = getopt(argc, argv, "d:h")) != -1) {
+	while ((c = getopt(argc, argv, "ad:h")) != -1) {
 		switch (c) {
+		case 'a':
+			irq_affinity = true;
+			break;
 		case 'd':
 			device_bdf = optarg;
 			break;
@@ -171,6 +178,11 @@ int main(int argc, char **argv)
 		eventfd = kvm_new_eventfd();
 	}
 
+	if (irq_affinity) {
+		TEST_ASSERT(device_bdf, "-a requires -d");
+		irq_affinity_fp = open_proc_irq_affinity(irq);
+	}
+
 	printf("Injecting interrupts for GSI %d (Vector 0x%x) %d times\n",
 	       gsi, vector, nr_irqs);
 
@@ -192,6 +204,11 @@ int main(int argc, char **argv)
 
 		kvm_route_msi(vm, gsi, vcpu, vector);
 
+		if (irq_affinity && vcpu->id == 0) {
+			irq_cpu = kvm_random_u64(&kvm_rng) % get_nprocs();
+			write_proc_irq_affinity(irq_affinity_fp, irq, irq_cpu);
+		}
+
 		for (j = 0; j < nr_vcpus; j++)
 			TEST_ASSERT(!GUEST_RECEIVED_IRQ(vcpus[j]),
 				    "IRQ flag for vCPU %d not clear prior to test",
@@ -204,9 +221,15 @@ int main(int argc, char **argv)
 			if (GUEST_RECEIVED_IRQ(vcpu))
 				break;
 
-			if (timespec_to_ns(timespec_elapsed(start)) > timeout_ns)
+			if (timespec_to_ns(timespec_elapsed(start)) > timeout_ns) {
+				printf("Timeout waiting for interrupt!\n");
+				printf("  vCPU: %d\n", vcpu->id);
+				if (irq_affinity)
+					printf("  irq_cpu: %d\n", irq_cpu);
+
 				TEST_FAIL("vCPU %d timed out waiting for IRQ from GSI %d (Vector 0x%x) !\n",
 					  vcpu->id, gsi, vector);
+			}
 		}
 
 		WRITE_AND_SYNC_TO_GUEST(vm, guest_received_irq[vcpu->id], false);
@@ -217,6 +240,9 @@ int main(int argc, char **argv)
 	for (i = 0; i < nr_vcpus; i++)
 		pthread_join(vcpu_threads[i], NULL);
 
+	if (irq_affinity)
+		fclose(irq_affinity_fp);
+
 	printf("Test passed!\n");
 
 	return 0;
tools/testing/selftests/kvm/lib/proc_util.c +20 -0
index ad1c54a81869..e5a4ee262f5e 100644
@@ -40,3 +40,23 @@ int get_proc_vfio_irq_number(const char *device_bdf, int msi)
 	return irq;
 }
 
+FILE *open_proc_irq_affinity(int irq)
+{
+	char path[PATH_MAX];
+	FILE *fp;
+
+	snprintf(path, sizeof(path), "/proc/irq/%d/smp_affinity_list", irq);
+	fp = fopen(path, "w");
+	TEST_ASSERT(fp, "fopen(%s) failed", path);
+
+	return fp;
+}
+
+void write_proc_irq_affinity(FILE *fp, int irq, int irq_cpu)
+{
+	int ret;
+
+	ret = fprintf(fp, "%d\n", irq_cpu);
+	TEST_ASSERT(ret > 0, "Failed to affinitize IRQ-%d to CPU %d", irq, irq_cpu);
+	fflush(fp);
+}
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
#10 lore raw

Re: [PATCH v4 08/19] KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test

Review int main(int argc, char **argv) · line 140

Quoted 5 lines · Josh Hilke
(quoted diff) +1 -0
@@ -134,17 +137,21 @@ int main(int argc, char **argv)
 	u32 gsi = kvm_random_u64_in_range(&kvm_rng, 24, KVM_MAX_IRQ_ROUTES - 1);
 	u8 vector = kvm_random_u64_in_range(&kvm_rng, 32, UINT8_MAX);
 
+	int i, j, c, msi, irq, eventfd, irq_cpu;

This fails to build with -Werror:

irq_test.c: In function ‘main’:
irq_test.c:183:35: error: ‘irq’ may be used uninitialized [-Werror=maybe-uninitialized]

  183 |                 irq_affinity_fp = open_proc_irq_affinity(irq);
      |                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~

irq_test.c:140:27: note: ‘irq’ was declared here
140 | int i, j, c, msi, irq, eventfd, irq_cpu;
| ^~~
irq_test.c:228:41: error: ‘irq_cpu’ may be used uninitialized [-Werror=maybe-uninitialized]

  228 |                                         printf("  irq_cpu: %d\n", irq_cpu);
      |                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

irq_test.c:140:41: note: ‘irq_cpu’ was declared here
140 | int i, j, c, msi, irq, eventfd, irq_cpu;
| ^~~~~~~

The first one (of three distinct uses) is easy enough to solve:

tools/testing/selftests/kvm/irq_test.c +6 -5
index 29f54a435c8e..23c82ddee6b9 100644
@@ -251,15 +251,16 @@ int main(int argc, char **argv)
                eventfd = device->msi_eventfds[msi];
                printf("Using device %s MSI-X[%d] (IRQ-%d)\n", device_bdf, msi,
                       irq);
+
+               if (irq_affinity)
+                       irq_affinity_fp = open_proc_irq_affinity(irq);
        } else {
+               TEST_ASSERT(!irq_affinity,
+                           "Setting IRQ affinity (-a) requires a backing device (-d)");
+
                eventfd = kvm_new_eventfd();
        }
 
-       if (irq_affinity) {
-               TEST_ASSERT(device_bdf, "-a requires -d");
-               irq_affinity_fp = open_proc_irq_affinity(irq);
-       }
-
        printf("Injecting interrupts for GSI %d (Vector 0x%x) %d times\n",
               gsi, vector, nr_irqs);
 

And the second can be addressed by checking irq_affinity_fp instead of irq_affinity:

@@ -298,7 +299,7 @@ int main(int argc, char **argv)

                kvm_route_msi(vm, gsi, vcpu, vector, do_use_nmi);

- if (irq_affinity && vcpu->id == 0) {

+               if (irq_affinity_fp && vcpu->id == 0) {
                        irq_cpu = kvm_random_u64(&kvm_rng) % get_nprocs();
                        write_proc_irq_affinity(irq_affinity_fp, irq, irq_cpu);
                }

but I don't think that's quite correct. Keying off vCPU0 for task migration makes sense, because the test round-robins IRQs to each vCPU, so it's logical to redo task/vCPU affinity after every round.

But IRQs are being spread over _all_ host CPUs, and there is no round-robin behavior, it's completely random. So rather than rate-limit based on number of vCPUs, I think it makes more sense to hardcode the changes in affinity, e.g.

		if (irq_affinity_fp && !(i % 10)) {
			irq_cpu = kvm_random_u64(&kvm_rng) % get_nprocs();
			write_proc_irq_affinity(irq_affinity_fp, irq, irq_cpu);
		}

Sadly, even that doesn't address the third "used uninitialized" issue:

irq_test.c:334:41: error: ‘irq_cpu’ may be used uninitialized [-Werror=maybe-uninitialized]

  334 |                                         printf("  irq_cpu: %d\n", irq_cpu);
      |                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

because apparently gcc isn't smart enough to understand that "!(i % 10)" (or even "!i" is guaranteed to run on the first iteration. However, that's false positive probably a blessing in disguise, because it highlights that printing only what userspace has _requested_ could be very misleading, given that the whole point of the offending printf() is to spit out information when the IRQ got lost.

Writing smp_affinity_list (or smp_affinity) does NOT guarantee the new affinity is realized in hardware. E.g. the kernel typically "lazily" migrates IRQs, and only actually changes the affinity when an IRQ actually arrives. To see what CPU an IRQ is actually wired up to, you need to read /proc/<irq>/effective_affinity{,list}.

As it pertains to the "used uninitialized" flaw, I think the test should spit out three things:

1. irq_cpu, i.e. what the test thought it set the affinity to
2. /proc/<irq>/smp_affinity, i.e. what the kernel has for the allowed list
3. /proc/<irq>/effective_affinity, i.e. what the actual, current affinity is

And then _if_ you want to ratelimit changes in affinity, fix the "used uninitialized" issue by explicitly initializing irq_cpu, with a comment calling out that the compiler is being stupid.

@@ -287,6 +288,12 @@ int main(int argc, char **argv)

                }
        }

+ /*

+        * Suppress a false positive maybe-uninitialized compiler warning due
+        * to ratelimiting changes in IRQ affinity.
+        */
+       irq_cpu = -1;
+
        for (i = 0; i < nr_irqs; i++) {
                const bool do_clear_routes = clear_routes && (i & BIT(3));
                const bool do_use_nmi = use_nmi && (i & BIT(2));

@@ -298,7 +305,7 @@ int main(int argc, char **argv)

                kvm_route_msi(vm, gsi, vcpu, vector, do_use_nmi);

- if (irq_affinity && vcpu->id == 0) {

+               if (irq_affinity_fp && !(i % 10)) {
                        irq_cpu = kvm_random_u64(&kvm_rng) % get_nprocs();
                        write_proc_irq_affinity(irq_affinity_fp, irq, irq_cpu);
                }
@@ -329,7 +336,7 @@ int main(int argc, char **argv)
                                printf("Timeout waiting for interrupt!\n");
                                printf("  vCPU: %d\n", vcpu->id);
                                printf("  is interrupt NMI: %s\n", do_use_nmi ? "true" : "false");
-                               if (irq_affinity)
+                               if (irq_affinity_fp)
                                        printf("  irq_cpu: %d\n", irq_cpu);
                                if (migrate_vcpus)
                                        kvm_print_vcpu_affinity(vcpu, vcpu_tids[vcpu->id]);

But honestly, unless it impacts the runtime, I don't see much value in ratelimiting changes in IRQ affinity. Putting that much stress on the IRQ subsystem is actually probably interesting.

And no matter what, /proc/<irq>/effective_affinity needs to be printed on failure.

author
#11 lore raw

PATCH v4 9/19 KVM: selftests: Verify IRQs wake up halted vCPUs in IRQ test

In reply to Josh Hilke

From: David Matlack <[email protected]>

Add the '-b' flag to tools/testing/selftests/kvm/irq_test.c to verify that interrupts successfully wake up vCPUs that are in a halted state.

By default, the guest code spins using cpu_relax() while waiting for interrupts. With the '-b' flag, the guest executes the HLT instruction instead. At the end of the test, the host explicitly routes and triggers a final interrupt for each vCPU. If the interrupt fails to wake the halted vCPU, the guest will never exit its loop, and the host's call to pthread_join() will hang, causing the test to time out and fail.

---
 tools/testing/selftests/kvm/irq_test.c | 32 ++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/irq_test.c +27 -5
index e76784e0a054..562b9ab187a4 100644
@@ -18,6 +18,7 @@ static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;
 static bool guest_ready_for_irqs[KVM_MAX_VCPUS];
 static bool guest_received_irq[KVM_MAX_VCPUS];
 static bool irq_affinity;
+static bool block_vcpus;
 static bool done;
 
 #define GUEST_RECEIVED_IRQ(__vcpu)	\
@@ -43,8 +44,12 @@ static void guest_code(void)
 
 	WRITE_ONCE(guest_ready_for_irqs[guest_get_vcpu_id()], true);
 
-	while (!READ_ONCE(done))
-		cpu_relax();
+	while (!READ_ONCE(done)) {
+		if (block_vcpus)
+			hlt();
+		else
+			cpu_relax();
+	}
 
 	GUEST_DONE();
 }
@@ -111,10 +116,11 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-d <segment:bus:device.function>] [-h]\n", name);
+	printf("Usage: %s [-a] [-b] [-d <segment:bus:device.function>] [-h]\n", name);
 	printf("\n");
 	printf("Tests KVM IRQ injection via irqfd using an emulated eventfd.\n");
 	printf("-a	Randomly affinitize the device's host IRQ to different physical CPUs throughout the test\n");
+	printf("-b	Block vCPUs (e.g. HLT) instead of spinning in guest-mode\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of using an emulated eventfd\n");
 	printf("\n");
 	exit(KSFT_FAIL);
@@ -147,11 +153,14 @@ int main(int argc, char **argv)
 	struct iommu *iommu;
 	struct kvm_vm *vm;
 
-	while ((c = getopt(argc, argv, "ad:h")) != -1) {
+	while ((c = getopt(argc, argv, "abd:h")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
 			break;
+		case 'b':
+			block_vcpus = true;
+			break;
 		case 'd':
 			device_bdf = optarg;
 			break;
@@ -188,6 +197,8 @@ int main(int argc, char **argv)
 
 	kvm_assign_irqfd(vm, gsi, eventfd);
 
+	sync_global_to_guest(vm, block_vcpus);
+
 	for (i = 0; i < nr_vcpus; i++)
 		pthread_create(&vcpu_threads[i], NULL, vcpu_thread_main, vcpus[i]);
 
@@ -237,8 +248,19 @@ int main(int argc, char **argv)
 
 	WRITE_AND_SYNC_TO_GUEST(vm, done, true);
 
-	for (i = 0; i < nr_vcpus; i++)
+	for (i = 0; i < nr_vcpus; i++) {
+		/*
+		 * Verify that sending an interrupt to a halted vCPU wakes it
+		 * up. If the vCPU does not wake up, the call to pthread_join(),
+		 * below, will hang.
+		 */
+		if (block_vcpus) {
+			kvm_route_msi(vm, gsi, vcpus[i], vector);
+			trigger_interrupt(device, eventfd);
+		}
+
 		pthread_join(vcpu_threads[i], NULL);
+	}
 
 	if (irq_affinity)
 		fclose(irq_affinity_fp);
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#12 lore raw

PATCH v4 10/19 KVM: selftests: Verify interrupts are received after modifying IRQ routes in IRQ test

In reply to Josh Hilke

From: David Matlack <[email protected]>

Add the '-c' flag to tools/testing/selftests/kvm/irq_test.c to destroy and recreate KVM's GSI routing table between interrupts. This ensures that KVM correctly handles dynamic updates to the interrupt routing table while interrupts are actively being signaled by assigned devices.

---
 tools/testing/selftests/kvm/irq_test.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +18 -2
index 562b9ab187a4..72b9106216ed 100644
@@ -114,13 +114,21 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 	vm_ioctl(vm, KVM_SET_GSI_ROUTING, routes);
 }
 
+static void kvm_clear_gsi_routes(struct kvm_vm *vm)
+{
+	struct kvm_irq_routing routes = {};
+
+	vm_ioctl(vm, KVM_SET_GSI_ROUTING, &routes);
+}
+
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-b] [-d <segment:bus:device.function>] [-h]\n", name);
+	printf("Usage: %s [-a] [-b] [-c] [-d <segment:bus:device.function>] [-h]\n", name);
 	printf("\n");
 	printf("Tests KVM IRQ injection via irqfd using an emulated eventfd.\n");
 	printf("-a	Randomly affinitize the device's host IRQ to different physical CPUs throughout the test\n");
 	printf("-b	Block vCPUs (e.g. HLT) instead of spinning in guest-mode\n");
+	printf("-c	Destroy and recreate KVM's GSI routing table in between some interrupts\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of using an emulated eventfd\n");
 	printf("\n");
 	exit(KSFT_FAIL);
@@ -150,10 +158,11 @@ int main(int argc, char **argv)
 	int nr_irqs = 1000, nr_vcpus = 1;
 	const char *device_bdf = NULL;
 	FILE *irq_affinity_fp = NULL;
+	bool clear_routes = false;
 	struct iommu *iommu;
 	struct kvm_vm *vm;
 
-	while ((c = getopt(argc, argv, "abd:h")) != -1) {
+	while ((c = getopt(argc, argv, "abcd:h")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -164,6 +173,9 @@ int main(int argc, char **argv)
 		case 'd':
 			device_bdf = optarg;
 			break;
+		case 'c':
+			clear_routes = true;
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
@@ -210,9 +222,13 @@ int main(int argc, char **argv)
 	}
 
 	for (i = 0; i < nr_irqs; i++) {
+		const bool do_clear_routes = clear_routes && (i & BIT(3));
 		struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];
 		struct timespec start;
 
+		if (do_clear_routes)
+			kvm_clear_gsi_routes(vm);
+
 		kvm_route_msi(vm, gsi, vcpu, vector);
 
 		if (irq_affinity && vcpu->id == 0) {
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#13 lore raw

PATCH v4 11/19 KVM: selftests: Make number of IRQs configurable in IRQ test

In reply to Josh Hilke

From: David Matlack <[email protected]>

Add a '-i' flag to tools/testing/selftests/kvm/irq_test.c to allow users to specify the number of IRQs generated during a test run.

---
 tools/testing/selftests/kvm/irq_test.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +6 -2
index 72b9106216ed..e79b66c050ff 100644
@@ -123,13 +123,14 @@ static void kvm_clear_gsi_routes(struct kvm_vm *vm)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-b] [-c] [-d <segment:bus:device.function>] [-h]\n", name);
+	printf("Usage: %s [-a] [-b] [-c] [-d <segment:bus:device.function>] [-h] [-i nr_irqs]\n", name);
 	printf("\n");
 	printf("Tests KVM IRQ injection via irqfd using an emulated eventfd.\n");
 	printf("-a	Randomly affinitize the device's host IRQ to different physical CPUs throughout the test\n");
 	printf("-b	Block vCPUs (e.g. HLT) instead of spinning in guest-mode\n");
 	printf("-c	Destroy and recreate KVM's GSI routing table in between some interrupts\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of using an emulated eventfd\n");
+	printf("-i	The number of IRQs to generate during the test\n");
 	printf("\n");
 	exit(KSFT_FAIL);
 }
@@ -162,7 +163,7 @@ int main(int argc, char **argv)
 	struct iommu *iommu;
 	struct kvm_vm *vm;
 
-	while ((c = getopt(argc, argv, "abcd:h")) != -1) {
+	while ((c = getopt(argc, argv, "abcd:hi:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -176,6 +177,9 @@ int main(int argc, char **argv)
 		case 'c':
 			clear_routes = true;
 			break;
+		case 'i':
+			nr_irqs = atoi_positive("Number of IRQs", optarg);
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#14 lore raw

PATCH v4 12/19 KVM: selftests: Verify non-postable IRQ remapping in IRQ test

In reply to Josh Hilke

From: David Matlack <[email protected]>

Add the -n flag to tools/testing/selftests/kvm/irq_test.c to route a portion of device interrupts as NMIs (Non-Maskable Interrupts) into the guest using an alternating pattern of 4 NMIs followed by 4 regular interrupts.

While this adds coverage for NMI injection, the primary goal is to validate KVM's handling of non-postable interrupt delivery. Specifically, the transitions between posted and remapped modes. NMIs are used for this purpose because they are a reliable, architectural way to force these code paths.

---
 tools/testing/selftests/kvm/irq_test.c | 45 ++++++++++++++++++++------
 1 file changed, 35 insertions(+), 10 deletions(-)
tools/testing/selftests/kvm/irq_test.c +35 -10
index e79b66c050ff..79b0691c9889 100644
@@ -17,12 +17,15 @@
 static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;
 static bool guest_ready_for_irqs[KVM_MAX_VCPUS];
 static bool guest_received_irq[KVM_MAX_VCPUS];
+static bool guest_received_nmi[KVM_MAX_VCPUS];
 static bool irq_affinity;
 static bool block_vcpus;
 static bool done;
 
 #define GUEST_RECEIVED_IRQ(__vcpu)	\
 	SYNC_FROM_GUEST_AND_READ((__vcpu)->vm, guest_received_irq[(__vcpu)->id])
+#define GUEST_RECEIVED_NMI(__vcpu)	\
+	SYNC_FROM_GUEST_AND_READ((__vcpu)->vm, guest_received_nmi[(__vcpu)->id])
 
 static u32 guest_get_vcpu_id(void)
 {
@@ -36,6 +39,11 @@ static void guest_irq_handler(struct ex_regs *regs)
 	x2apic_write_reg(APIC_EOI, 0);
 }
 
+static void guest_nmi_handler(struct ex_regs *regs)
+{
+	WRITE_ONCE(guest_received_nmi[guest_get_vcpu_id()], true);
+}
+
 static void guest_code(void)
 {
 	x2apic_enable();
@@ -96,7 +104,7 @@ static void trigger_interrupt(struct vfio_pci_device *device, int eventfd)
 
 
 static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
-			  u8 vector)
+			  u8 vector, bool use_nmi)
 {
 	struct {
 		struct kvm_irq_routing head;
@@ -109,7 +117,7 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,
 	routes->entries[0].gsi = gsi;
 	routes->entries[0].type = KVM_IRQ_ROUTING_MSI;
 	routes->entries[0].u.msi.address_lo = 0xFEE00000 | (vcpu->id << 12);
-	routes->entries[0].u.msi.data = vector;
+	routes->entries[0].u.msi.data = use_nmi ? NMI_VECTOR | (4 << 8) : vector;
 
 	vm_ioctl(vm, KVM_SET_GSI_ROUTING, routes);
 }
@@ -123,7 +131,7 @@ static void kvm_clear_gsi_routes(struct kvm_vm *vm)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-b] [-c] [-d <segment:bus:device.function>] [-h] [-i nr_irqs]\n", name);
+	printf("Usage: %s [-a] [-b] [-c] [-d <segment:bus:device.function>] [-h] [-i nr_irqs] [-n]\n", name);
 	printf("\n");
 	printf("Tests KVM IRQ injection via irqfd using an emulated eventfd.\n");
 	printf("-a	Randomly affinitize the device's host IRQ to different physical CPUs throughout the test\n");
@@ -131,6 +139,7 @@ static void help(const char *name)
 	printf("-c	Destroy and recreate KVM's GSI routing table in between some interrupts\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of using an emulated eventfd\n");
 	printf("-i	The number of IRQs to generate during the test\n");
+	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
 	printf("\n");
 	exit(KSFT_FAIL);
 }
@@ -152,6 +161,7 @@ int main(int argc, char **argv)
 	u32 gsi = kvm_random_u64_in_range(&kvm_rng, 24, KVM_MAX_IRQ_ROUTES - 1);
 	u8 vector = kvm_random_u64_in_range(&kvm_rng, 32, UINT8_MAX);
 
+	bool clear_routes = false, use_nmi = false;
 	int i, j, c, msi, irq, eventfd, irq_cpu;
 	struct kvm_vcpu *vcpus[KVM_MAX_VCPUS];
 	pthread_t vcpu_threads[KVM_MAX_VCPUS];
@@ -159,11 +169,10 @@ int main(int argc, char **argv)
 	int nr_irqs = 1000, nr_vcpus = 1;
 	const char *device_bdf = NULL;
 	FILE *irq_affinity_fp = NULL;
-	bool clear_routes = false;
 	struct iommu *iommu;
 	struct kvm_vm *vm;
 
-	while ((c = getopt(argc, argv, "abcd:hi:")) != -1) {
+	while ((c = getopt(argc, argv, "abcd:hi:n")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -180,6 +189,9 @@ int main(int argc, char **argv)
 		case 'i':
 			nr_irqs = atoi_positive("Number of IRQs", optarg);
 			break;
+		case 'n':
+			use_nmi = true;
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
@@ -190,6 +202,7 @@ int main(int argc, char **argv)
 
 	vm = vm_create_with_vcpus(nr_vcpus, guest_code, vcpus);
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
+	vm_install_exception_handler(vm, NMI_VECTOR, guest_nmi_handler);
 
 	if (device_bdf) {
 		iommu = iommu_init(default_iommu_mode);
@@ -227,34 +240,43 @@ int main(int argc, char **argv)
 
 	for (i = 0; i < nr_irqs; i++) {
 		const bool do_clear_routes = clear_routes && (i & BIT(3));
+		const bool do_use_nmi = use_nmi && (i & BIT(2));
 		struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];
 		struct timespec start;
 
 		if (do_clear_routes)
 			kvm_clear_gsi_routes(vm);
 
-		kvm_route_msi(vm, gsi, vcpu, vector);
+		kvm_route_msi(vm, gsi, vcpu, vector, do_use_nmi);
 
 		if (irq_affinity && vcpu->id == 0) {
 			irq_cpu = kvm_random_u64(&kvm_rng) % get_nprocs();
 			write_proc_irq_affinity(irq_affinity_fp, irq, irq_cpu);
 		}
 
-		for (j = 0; j < nr_vcpus; j++)
+		for (j = 0; j < nr_vcpus; j++) {
 			TEST_ASSERT(!GUEST_RECEIVED_IRQ(vcpus[j]),
 				    "IRQ flag for vCPU %d not clear prior to test",
 				    vcpus[j]->id);
+			TEST_ASSERT(!GUEST_RECEIVED_NMI(vcpus[j]),
+				    "NMI flag for vCPU %d not clear prior to test",
+				    vcpus[j]->id);
+		}
 
 		trigger_interrupt(device, eventfd);
 
 		clock_gettime(CLOCK_MONOTONIC, &start);
 		for (;;) {
-			if (GUEST_RECEIVED_IRQ(vcpu))
+			if (!do_use_nmi && GUEST_RECEIVED_IRQ(vcpu))
+				break;
+
+			if (do_use_nmi && GUEST_RECEIVED_NMI(vcpu))
 				break;
 
 			if (timespec_to_ns(timespec_elapsed(start)) > timeout_ns) {
 				printf("Timeout waiting for interrupt!\n");
 				printf("  vCPU: %d\n", vcpu->id);
+				printf("  is interrupt NMI: %s\n", do_use_nmi ? "true" : "false");
 				if (irq_affinity)
 					printf("  irq_cpu: %d\n", irq_cpu);
 
@@ -263,7 +285,10 @@ int main(int argc, char **argv)
 			}
 		}
 
-		WRITE_AND_SYNC_TO_GUEST(vm, guest_received_irq[vcpu->id], false);
+		if (do_use_nmi)
+			WRITE_AND_SYNC_TO_GUEST(vm, guest_received_nmi[vcpu->id], false);
+		else
+			WRITE_AND_SYNC_TO_GUEST(vm, guest_received_irq[vcpu->id], false);
 	}
 
 	WRITE_AND_SYNC_TO_GUEST(vm, done, true);
@@ -275,7 +300,7 @@ int main(int argc, char **argv)
 		 * below, will hang.
 		 */
 		if (block_vcpus) {
-			kvm_route_msi(vm, gsi, vcpus[i], vector);
+			kvm_route_msi(vm, gsi, vcpus[i], vector, false);
 			trigger_interrupt(device, eventfd);
 		}
 
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#15 lore raw

PATCH v4 13/19 KVM: selftests: Add kvm_gettid() wrapper and convert users

In reply to Josh Hilke

Add a KVM wrapper for the gettid() syscall to avoid compilation errors in versions of Glibc that don't contain gettid(). Convert existing users of the syscall to use this new wrapper.

This helper will be used in a future change in the KVM IRQ test to migrate vCPU threads.

No functional changes intended.

---
 tools/testing/selftests/kvm/demand_paging_test.c   | 2 +-
 tools/testing/selftests/kvm/include/kvm_syscalls.h | 5 +++++
 tools/testing/selftests/kvm/lib/assert.c           | 8 ++------
 tools/testing/selftests/kvm/lib/test_util.c        | 3 ++-
 tools/testing/selftests/kvm/rseq_test.c            | 2 +-
 5 files changed, 11 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/demand_paging_test.c +1 -1
index 0202b78f8680..4ebb2a9c3ba1 100644
@@ -57,7 +57,7 @@ static void vcpu_worker(struct memstress_vcpu_args *vcpu_args)
 static int handle_uffd_page_request(int uffd_mode, int uffd,
 		struct uffd_msg *msg)
 {
-	pid_t tid = syscall(__NR_gettid);
+	pid_t tid = kvm_gettid();
 	uint64_t addr = msg->arg.pagefault.address;
 	struct timespec start;
 	struct timespec ts_diff;
tools/testing/selftests/kvm/include/kvm_syscalls.h +5 -0
index d4e613162bba..522ffff0462c 100644
@@ -73,6 +73,11 @@ static inline int kvm_dup(int fd)
 	return new_fd;
 }
 
+static inline pid_t kvm_gettid(void)
+{
+	return syscall(__NR_gettid);
+}
+
 __KVM_SYSCALL_DEFINE(munmap, 2, void *, mem, size_t, size);
 __KVM_SYSCALL_DEFINE(close, 1, int, fd);
 __KVM_SYSCALL_DEFINE(fallocate, 4, int, fd, int, mode, loff_t, offset, loff_t, len);
tools/testing/selftests/kvm/lib/assert.c +2 -6
index b49690658c60..7914046d334d 100644
@@ -10,6 +10,7 @@
 #include <sys/syscall.h>
 
 #include "kselftest.h"
+#include "kvm_syscalls.h"
 
 /* Dumps the current stack trace to stderr. */
 static void __attribute__((noinline)) test_dump_stack(void);
@@ -58,11 +59,6 @@ static void test_dump_stack(void)
 #pragma GCC diagnostic pop
 }
 
-static pid_t _gettid(void)
-{
-	return syscall(SYS_gettid);
-}
-
 void __attribute__((noinline))
 test_assert(bool exp, const char *exp_str,
 	const char *file, unsigned int line, const char *fmt, ...)
@@ -75,7 +71,7 @@ test_assert(bool exp, const char *exp_str,
 		fprintf(stderr, "==== Test Assertion Failure ====\n"
 			"  %s:%u: %s\n"
 			"  pid=%d tid=%d errno=%d - %s\n",
-			file, line, exp_str, getpid(), _gettid(),
+			file, line, exp_str, getpid(), kvm_gettid(),
 			errno, strerror(errno));
 		test_dump_stack();
 		if (fmt) {
tools/testing/selftests/kvm/lib/test_util.c +2 -1
index dc47ed9736e9..2006e46926d3 100644
@@ -17,6 +17,7 @@
 #include "linux/kernel.h"
 
 #include "test_util.h"
+#include "kvm_syscalls.h"
 
 sigjmp_buf expect_sigbus_jmpbuf;
 
@@ -395,7 +396,7 @@ long get_run_delay(void)
 	long val[2];
 	FILE *fp;
 
-	sprintf(path, "/proc/%ld/schedstat", syscall(SYS_gettid));
+	sprintf(path, "/proc/%ld/schedstat", (long)kvm_gettid());
 	fp = fopen(path, "r");
 	/* Return MIN_RUN_DELAY_NS upon failure just to be safe */
 	if (fscanf(fp, "%ld %ld ", &val[0], &val[1]) < 2)
tools/testing/selftests/kvm/rseq_test.c +1 -1
index f80ad6b47d16..6510fbfd64f1 100644
@@ -244,7 +244,7 @@ int main(int argc, char *argv[])
 	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
 
 	pthread_create(&migration_thread, NULL, migration_worker,
-		       (void *)(unsigned long)syscall(SYS_gettid));
+		       (void *)(unsigned long)kvm_gettid());
 
 	if (latency >= 0) {
 		/*
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#16 lore raw

PATCH v4 14/19 KVM: selftests: Add kvm_sched_getaffinity() wrapper and convert users

In reply to Josh Hilke

Add a KVM wrapper for the sched_getaffinity syscall named kvm_sched_getaffinity(). Convert existing users of sched_getaffinity to use the new wrapper.

This helper will be used in a future change in the KVM IRQ test to migrate vCPU threads.

No functional changes intended.

---
 tools/testing/selftests/kvm/arch_timer.c                  | 2 +-
 tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c | 2 +-
 tools/testing/selftests/kvm/include/kvm_syscalls.h        | 1 +
 tools/testing/selftests/kvm/lib/kvm_util.c                | 5 ++---
 tools/testing/selftests/kvm/mmu_stress_test.c             | 6 +-----
 tools/testing/selftests/kvm/rseq_test.c                   | 4 +---
 6 files changed, 7 insertions(+), 13 deletions(-)
tools/testing/selftests/kvm/arch_timer.c +1 -1
index cf8fb67104f1..e4289800f432 100644
@@ -85,7 +85,7 @@ static uint32_t test_get_pcpu(void)
 	cpu_set_t online_cpuset;
 
 	nproc_conf = get_nprocs_conf();
-	sched_getaffinity(0, sizeof(cpu_set_t), &online_cpuset);
+	kvm_sched_getaffinity(0, sizeof(cpu_set_t), &online_cpuset);
 
 	/* Randomly find an available pCPU to place a vCPU on */
 	do {
tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c +1 -1
index 993c9e38e729..c7fd601566c1 100644
@@ -1040,7 +1040,7 @@ int main(int argc, char *argv[])
 	if (!parse_args(argc, argv))
 		exit(KSFT_SKIP);
 
-	sched_getaffinity(0, sizeof(default_cpuset), &default_cpuset);
+	kvm_sched_getaffinity(0, sizeof(default_cpuset), &default_cpuset);
 	set_counter_defaults();
 
 	if (test_args.test_virtual) {
tools/testing/selftests/kvm/include/kvm_syscalls.h +1 -0
index 522ffff0462c..2e362fda50af 100644
@@ -82,5 +82,6 @@ __KVM_SYSCALL_DEFINE(munmap, 2, void *, mem, size_t, size);
 __KVM_SYSCALL_DEFINE(close, 1, int, fd);
 __KVM_SYSCALL_DEFINE(fallocate, 4, int, fd, int, mode, loff_t, offset, loff_t, len);
 __KVM_SYSCALL_DEFINE(ftruncate, 2, unsigned int, fd, off_t, length);
+__KVM_SYSCALL_DEFINE(sched_getaffinity, 3, pid_t, pid, size_t, cpusetsize, cpu_set_t *, mask);
 
 #endif /* SELFTEST_KVM_SYSCALLS_H */
tools/testing/selftests/kvm/lib/kvm_util.c +2 -3
index b247b2015b2a..ebd04cf1847b 100644
@@ -668,13 +668,12 @@ void kvm_parse_vcpu_pinning(const char *pcpus_string, uint32_t vcpu_to_pcpu[],
 	cpu_set_t allowed_mask;
 	char *cpu, *cpu_list;
 	char delim[2] = ",";
-	int i, r;
+	int i;
 
 	cpu_list = strdup(pcpus_string);
 	TEST_ASSERT(cpu_list, "strdup() allocation failed.");
 
-	r = sched_getaffinity(0, sizeof(allowed_mask), &allowed_mask);
-	TEST_ASSERT(!r, "sched_getaffinity() failed");
+	kvm_sched_getaffinity(0, sizeof(allowed_mask), &allowed_mask);
 
 	cpu = strtok(cpu_list, delim);
 
tools/testing/selftests/kvm/mmu_stress_test.c +1 -5
index a9fac4bc02c2..98546edbf11b 100644
@@ -255,11 +255,7 @@ static void rendezvous_with_vcpus(struct timespec *time, const char *name)
 static void calc_default_nr_vcpus(void)
 {
 	cpu_set_t possible_mask;
-	int r;
-
-	r = sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
-	TEST_ASSERT(!r, "sched_getaffinity failed, errno = %d (%s)",
-		    errno, strerror(errno));
+	kvm_sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
 
 	nr_vcpus = CPU_COUNT(&possible_mask);
 	TEST_ASSERT(nr_vcpus > 0, "Uh, no CPUs?");
tools/testing/selftests/kvm/rseq_test.c +1 -3
index 6510fbfd64f1..557e393c223b 100644
@@ -226,9 +226,7 @@ int main(int argc, char *argv[])
 		}
 	}
 
-	r = sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
-	TEST_ASSERT(!r, "sched_getaffinity failed, errno = %d (%s)", errno,
-		    strerror(errno));
+	kvm_sched_getaffinity(0, sizeof(possible_mask), &possible_mask);
 
 	calc_min_max_cpu();
 
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
#17 lore raw

Re: [PATCH v4 14/19] KVM: selftests: Add kvm_sched_getaffinity() wrapper and convert users

tools/testing/selftests/kvm/include/kvm_syscalls.h __KVM_SYSCALL_DEFINE(munmap, 2, void *, mem, size_t, size); · line 85

Quoted 9 lines · Josh Hilke
tools/testing/selftests/kvm/include/kvm_syscalls.h +1 -0
index 522ffff0462c..2e362fda50af 100644
@@ -82,5 +82,6 @@ __KVM_SYSCALL_DEFINE(munmap, 2, void *, mem, size_t, size);
 __KVM_SYSCALL_DEFINE(close, 1, int, fd);
 __KVM_SYSCALL_DEFINE(fallocate, 4, int, fd, int, mode, loff_t, offset, loff_t, len);
 __KVM_SYSCALL_DEFINE(ftruncate, 2, unsigned int, fd, off_t, length);
+__KVM_SYSCALL_DEFINE(sched_getaffinity, 3, pid_t, pid, size_t, cpusetsize, cpu_set_t *, mask);

This doesn't compile, needs:

testing/selftests/kvm/include/kvm_syscalls.h +2 -0
index 3e4cc91d5f87..d796a7228d9d 100644
@@ -2,6 +2,8 @@
 #ifndef SELFTEST_KVM_SYSCALLS_H
 #define SELFTEST_KVM_SYSCALLS_H
 
+#include <sched.h>
+
 #include <sys/syscall.h>
 
 #define MAP_ARGS0(m,...)
author
#18 lore raw

PATCH v4 15/19 KVM: selftests: Add pin_task_to_random_cpu() helper function

In reply to Josh Hilke

Add a helper function pin_task_to_random_cpu() to randomly select a CPU from a given cpu_set_t and pin the specified task to it.

This helper will be used in a future change in the KVM IRQ test to migrate vCPU threads.

---
 .../testing/selftests/kvm/include/kvm_util.h  |  2 ++
 tools/testing/selftests/kvm/lib/kvm_util.c    | 21 +++++++++++++++++++
 2 files changed, 23 insertions(+)
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
index 499e695f1fc4..f725ab0ed4f9 100644
@@ -1096,6 +1096,8 @@ static inline void pin_task_to_cpu(pthread_t task, int cpu)
 	TEST_ASSERT(!r, "Failed to set thread affinity to pCPU '%u'", cpu);
 }
 
+void pin_task_to_random_cpu(pthread_t task, cpu_set_t *possible_cpus);
+
 static inline int pin_task_to_any_cpu(pthread_t task)
 {
 	int cpu = sched_getcpu();
tools/testing/selftests/kvm/lib/kvm_util.c +21 -0
index ebd04cf1847b..29cd649a54d5 100644
@@ -662,6 +662,27 @@ void kvm_print_vcpu_pinning_help(void)
 	       "     (default: no pinning)\n", name, name);
 }
 
+void pin_task_to_random_cpu(pthread_t task, cpu_set_t *possible_cpus)
+{
+	int target_idx;
+	int nr_cpus;
+	int cpu;
+
+	nr_cpus = CPU_COUNT(possible_cpus);
+	TEST_ASSERT(nr_cpus > 0, "No CPUs available in possible_cpus");
+
+	target_idx = kvm_random_u64(&kvm_rng) % nr_cpus;
+
+	for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+		if (CPU_ISSET(cpu, possible_cpus) && target_idx-- == 0) {
+			pin_task_to_cpu(task, cpu);
+			return;
+		}
+	}
+
+	TEST_FAIL("Failed to find random CPU in possible_cpus");
+}
+
 void kvm_parse_vcpu_pinning(const char *pcpus_string, uint32_t vcpu_to_pcpu[],
 			    int nr_vcpus)
 {
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#19 lore raw

PATCH v4 16/19 KVM: selftests: Verify vCPU migration during IRQ delivery in IRQ test

In reply to Josh Hilke

Add the '-m' flag to tools/testing/selftests/kvm/irq_test.c to migrate vCPU threads across random physical CPUs during the test. This validates KVM's ability to handle vCPUs changing physical CPUs while interrupts are actively being injected.

---
 tools/testing/selftests/kvm/irq_test.c | 34 ++++++++++++++++++++++++--
 1 file changed, 32 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +32 -2
index 79b0691c9889..f653317022e3 100644
@@ -18,6 +18,8 @@ static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;
 static bool guest_ready_for_irqs[KVM_MAX_VCPUS];
 static bool guest_received_irq[KVM_MAX_VCPUS];
 static bool guest_received_nmi[KVM_MAX_VCPUS];
+static pid_t vcpu_tids[KVM_MAX_VCPUS];
+static bool migrate_vcpus;
 static bool irq_affinity;
 static bool block_vcpus;
 static bool done;
@@ -67,12 +69,23 @@ static void *vcpu_thread_main(void *arg)
 	struct kvm_vcpu *vcpu = arg;
 	struct ucall uc;
 
+	WRITE_ONCE(vcpu_tids[vcpu->id], kvm_gettid());
+
 	vcpu_run(vcpu);
 	TEST_ASSERT_EQ(UCALL_DONE, get_ucall(vcpu, &uc));
 
 	return NULL;
 }
 
+static void migrate_vcpu_threads(int nr_vcpus, pthread_t *vcpu_threads,
+				 cpu_set_t *available_cpus)
+{
+	int i;
+
+	for (i = 0; i < nr_vcpus; i++)
+		pin_task_to_random_cpu(vcpu_threads[i], available_cpus);
+}
+
 static int vfio_setup_msi(struct vfio_pci_device *device)
 {
 	const int flags = MAP_SHARED | MAP_ANONYMOUS;
@@ -131,7 +144,7 @@ static void kvm_clear_gsi_routes(struct kvm_vm *vm)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-b] [-c] [-d <segment:bus:device.function>] [-h] [-i nr_irqs] [-n]\n", name);
+	printf("Usage: %s [-a] [-b] [-c] [-d <segment:bus:device.function>] [-h] [-i nr_irqs] [-m] [-n]\n", name);
 	printf("\n");
 	printf("Tests KVM IRQ injection via irqfd using an emulated eventfd.\n");
 	printf("-a	Randomly affinitize the device's host IRQ to different physical CPUs throughout the test\n");
@@ -139,6 +152,7 @@ static void help(const char *name)
 	printf("-c	Destroy and recreate KVM's GSI routing table in between some interrupts\n");
 	printf("-d	Use a VFIO device to send MSI-X interrupts instead of using an emulated eventfd\n");
 	printf("-i	The number of IRQs to generate during the test\n");
+	printf("-m	Pin vCPU threads to random physical CPUs throughout the test\n");
 	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
 	printf("\n");
 	exit(KSFT_FAIL);
@@ -171,8 +185,9 @@ int main(int argc, char **argv)
 	FILE *irq_affinity_fp = NULL;
 	struct iommu *iommu;
 	struct kvm_vm *vm;
+	cpu_set_t available_cpus;
 
-	while ((c = getopt(argc, argv, "abcd:hi:n")) != -1) {
+	while ((c = getopt(argc, argv, "abcd:hi:mn")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -189,6 +204,9 @@ int main(int argc, char **argv)
 		case 'i':
 			nr_irqs = atoi_positive("Number of IRQs", optarg);
 			break;
+		case 'm':
+			migrate_vcpus = true;
+			break;
 		case 'n':
 			use_nmi = true;
 			break;
@@ -238,6 +256,15 @@ int main(int argc, char **argv)
 			continue;
 	}
 
+	if (migrate_vcpus) {
+		kvm_sched_getaffinity(vcpu_tids[0], sizeof(available_cpus), &available_cpus);
+
+		if (nr_vcpus > CPU_COUNT(&available_cpus)) {
+			printf("There are more vCPUs than pCPUs; refusing to migrate.\n");
+			migrate_vcpus = false;
+		}
+	}
+
 	for (i = 0; i < nr_irqs; i++) {
 		const bool do_clear_routes = clear_routes && (i & BIT(3));
 		const bool do_use_nmi = use_nmi && (i & BIT(2));
@@ -254,6 +281,9 @@ int main(int argc, char **argv)
 			write_proc_irq_affinity(irq_affinity_fp, irq, irq_cpu);
 		}
 
+		if (migrate_vcpus && vcpu->id == 0)
+			migrate_vcpu_threads(nr_vcpus, vcpu_threads, &available_cpus);
+
 		for (j = 0; j < nr_vcpus; j++) {
 			TEST_ASSERT(!GUEST_RECEIVED_IRQ(vcpus[j]),
 				    "IRQ flag for vCPU %d not clear prior to test",
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#20 lore raw

PATCH v4 17/19 KVM: selftests: Print vCPU affinity on timeout during IRQ test

In reply to Josh Hilke

Add kvm_print_vcpu_affinity() to lib/kvm_util.c to print the physical CPU affinity of a vCPU thread in a human-readable format.

Use this new helper in tools/testing/selftests/kvm/irq_test.c to print the vCPU's affinity if it times out waiting for an interrupt while vCPU migration (-m) is enabled. This provides valuable context for debugging IRQ delivery timeouts when vCPUs are being actively migrated across physical CPUs.

---
 .../testing/selftests/kvm/include/kvm_util.h  |  1 +
 tools/testing/selftests/kvm/irq_test.c        |  2 +
 tools/testing/selftests/kvm/lib/kvm_util.c    | 49 +++++++++++++++++++
 3 files changed, 52 insertions(+)
tools/testing/selftests/kvm/include/kvm_util.h +1 -0
index f725ab0ed4f9..0ec82088bc43 100644
@@ -1119,6 +1119,7 @@ static inline int pin_self_to_any_cpu(void)
 void kvm_print_vcpu_pinning_help(void);
 void kvm_parse_vcpu_pinning(const char *pcpus_string, uint32_t vcpu_to_pcpu[],
 			    int nr_vcpus);
+void kvm_print_vcpu_affinity(struct kvm_vcpu *vcpu, pid_t tid);
 
 unsigned long vm_compute_max_gfn(struct kvm_vm *vm);
 unsigned int vm_calc_num_guest_pages(enum vm_guest_mode mode, size_t size);
tools/testing/selftests/kvm/irq_test.c +2 -0
index f653317022e3..0db4ea4eb1ef 100644
@@ -309,6 +309,8 @@ int main(int argc, char **argv)
 				printf("  is interrupt NMI: %s\n", do_use_nmi ? "true" : "false");
 				if (irq_affinity)
 					printf("  irq_cpu: %d\n", irq_cpu);
+				if (migrate_vcpus)
+					kvm_print_vcpu_affinity(vcpu, vcpu_tids[vcpu->id]);
 
 				TEST_FAIL("vCPU %d timed out waiting for IRQ from GSI %d (Vector 0x%x) !\n",
 					  vcpu->id, gsi, vector);
tools/testing/selftests/kvm/lib/kvm_util.c +49 -0

61 lines not on this page · raw message

Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#21 lore raw

PATCH v4 18/19 KVM: selftests: Make number of vCPUs configurable in IRQ test

In reply to Josh Hilke

From: David Matlack <[email protected]>

Add a '-v' flag to tools/testing/selftests/kvm/irq_test.c to allow users to configure the number of vCPUs to run in the test.

---
 tools/testing/selftests/kvm/irq_test.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +6 -2
index 0db4ea4eb1ef..967083b8e2ff 100644
@@ -144,7 +144,7 @@ static void kvm_clear_gsi_routes(struct kvm_vm *vm)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-b] [-c] [-d <segment:bus:device.function>] [-h] [-i nr_irqs] [-m] [-n]\n", name);
+	printf("Usage: %s [-a] [-b] [-c] [-d <segment:bus:device.function>] [-h] [-i nr_irqs] [-m] [-n] [-v nr_vcpus]\n", name);
 	printf("\n");
 	printf("Tests KVM IRQ injection via irqfd using an emulated eventfd.\n");
 	printf("-a	Randomly affinitize the device's host IRQ to different physical CPUs throughout the test\n");
@@ -154,6 +154,7 @@ static void help(const char *name)
 	printf("-i	The number of IRQs to generate during the test\n");
 	printf("-m	Pin vCPU threads to random physical CPUs throughout the test\n");
 	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
+	printf("-v	Number of vCPUS to run\n");
 	printf("\n");
 	exit(KSFT_FAIL);
 }
@@ -187,7 +188,7 @@ int main(int argc, char **argv)
 	struct kvm_vm *vm;
 	cpu_set_t available_cpus;
 
-	while ((c = getopt(argc, argv, "abcd:hi:mn")) != -1) {
+	while ((c = getopt(argc, argv, "abcd:hi:mnv:")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -210,6 +211,9 @@ int main(int argc, char **argv)
 		case 'n':
 			use_nmi = true;
 			break;
+		case 'v':
+			nr_vcpus = atoi_positive("Number of vCPUS", optarg);
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#22 lore raw

PATCH v4 19/19 KVM: selftests: Add xAPIC support in IRQ test

In reply to Josh Hilke

From: David Matlack <[email protected]>

Add the '-x' flag to tools/testing/selftests/kvm/irq_test.c to configure the guest to use xAPIC mode instead of the default x2APIC mode. This expands test coverage to ensure that IRQ injection and routing work correctly when the guest uses the older xAPIC interface.

---
 tools/testing/selftests/kvm/irq_test.c | 28 +++++++++++++++++++++-----
 1 file changed, 23 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/irq_test.c +23 -5
index 967083b8e2ff..29f54a435c8e 100644
@@ -20,6 +20,7 @@ static bool guest_received_irq[KVM_MAX_VCPUS];
 static bool guest_received_nmi[KVM_MAX_VCPUS];
 static pid_t vcpu_tids[KVM_MAX_VCPUS];
 static bool migrate_vcpus;
+static bool x2apic = true;
 static bool irq_affinity;
 static bool block_vcpus;
 static bool done;
@@ -31,14 +32,20 @@ static bool done;
 
 static u32 guest_get_vcpu_id(void)
 {
-	return x2apic_read_reg(APIC_ID);
+	if (x2apic)
+		return x2apic_read_reg(APIC_ID);
+	else
+		return xapic_read_reg(APIC_ID) >> 24;
 }
 
 static void guest_irq_handler(struct ex_regs *regs)
 {
 	WRITE_ONCE(guest_received_irq[guest_get_vcpu_id()], true);
 
-	x2apic_write_reg(APIC_EOI, 0);
+	if (x2apic)
+		x2apic_write_reg(APIC_EOI, 0);
+	else
+		xapic_write_reg(APIC_EOI, 0);
 }
 
 static void guest_nmi_handler(struct ex_regs *regs)
@@ -48,7 +55,10 @@ static void guest_nmi_handler(struct ex_regs *regs)
 
 static void guest_code(void)
 {
-	x2apic_enable();
+	if (x2apic)
+		x2apic_enable();
+	else
+		xapic_enable();
 
 	sti_nop();
 
@@ -144,7 +154,7 @@ static void kvm_clear_gsi_routes(struct kvm_vm *vm)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-a] [-b] [-c] [-d <segment:bus:device.function>] [-h] [-i nr_irqs] [-m] [-n] [-v nr_vcpus]\n", name);
+	printf("Usage: %s [-a] [-b] [-c] [-d <segment:bus:device.function>] [-h] [-i nr_irqs] [-m] [-n] [-v nr_vcpus] [-x]\n", name);
 	printf("\n");
 	printf("Tests KVM IRQ injection via irqfd using an emulated eventfd.\n");
 	printf("-a	Randomly affinitize the device's host IRQ to different physical CPUs throughout the test\n");
@@ -155,6 +165,7 @@ static void help(const char *name)
 	printf("-m	Pin vCPU threads to random physical CPUs throughout the test\n");
 	printf("-n	Deliver 50 percent of IRQs as non-maskable interrupts\n");
 	printf("-v	Number of vCPUS to run\n");
+	printf("-x	Use xAPIC mode instead of x2APIC mode in the guest\n");
 	printf("\n");
 	exit(KSFT_FAIL);
 }
@@ -188,7 +199,7 @@ int main(int argc, char **argv)
 	struct kvm_vm *vm;
 	cpu_set_t available_cpus;
 
-	while ((c = getopt(argc, argv, "abcd:hi:mnv:")) != -1) {
+	while ((c = getopt(argc, argv, "abcd:hi:mnv:x")) != -1) {
 		switch (c) {
 		case 'a':
 			irq_affinity = true;
@@ -214,6 +225,9 @@ int main(int argc, char **argv)
 		case 'v':
 			nr_vcpus = atoi_positive("Number of vCPUS", optarg);
 			break;
+		case 'x':
+			x2apic = false;
+			break;
 		case 'h':
 		default:
 			help(argv[0]);
@@ -226,6 +240,9 @@ int main(int argc, char **argv)
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
 	vm_install_exception_handler(vm, NMI_VECTOR, guest_nmi_handler);
 
+	if (!x2apic)
+		virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA);
+
 	if (device_bdf) {
 		iommu = iommu_init(default_iommu_mode);
 		device = vfio_pci_device_init(device_bdf, iommu);
@@ -248,6 +265,7 @@ int main(int argc, char **argv)
 
 	kvm_assign_irqfd(vm, gsi, eventfd);
 
+	sync_global_to_guest(vm, x2apic);
 	sync_global_to_guest(vm, block_vcpus);
 
 	for (i = 0; i < nr_vcpus; i++)
Signature · 1 line
2.54.0.929.g9b7fa37559-goog
author
#23 lore raw
In reply to Josh Hilke
Quoted 10 lines · Josh Hilke

---
Changelog

v3 -> v4
- Add context to short logs specifying if the change relates specifically to the IRQ test (Sean)
- Format code correctly (Sean)
- Fix compilation errors for gettid() wrapper (Sean)
- Add macros for GUEST_RECEIVED_NMI and GUEST_RECEIVED_IRQ (Sean)
- Split helper functions into separate patches (Sean)

link to v3: https://lore.kernel.org/kvm/[email protected]/

#24 lore raw
In reply to Josh Hilke
Quoted 41 lines · Josh Hilke

David Matlack (12):
KVM: selftests: Build and link selftests/vfio/lib into KVM selftests
KVM: selftests: Add /proc/interrupts parsing helpers for IRQ test
KVM: selftests: Add guest read/write macros
KVM: selftests: Add IRQ injection test
KVM: selftests: Verify IRQ bypass works in IRQ test
KVM: selftests: Verify interrupts are received when IRQ affinity
changes in IRQ test
KVM: selftests: Verify IRQs wake up halted vCPUs in IRQ test
KVM: selftests: Verify interrupts are received after modifying IRQ
routes in IRQ test
KVM: selftests: Make number of IRQs configurable in IRQ test
KVM: selftests: Verify non-postable IRQ remapping in IRQ test
KVM: selftests: Make number of vCPUs configurable in IRQ test
KVM: selftests: Add xAPIC support in IRQ test

Josh Hilke (7):
KVM: selftests: Rename guest_rng to kvm_rng
KVM: selftests: Add helper to generate random u64 in range [min,max]
KVM: selftests: Add kvm_gettid() wrapper and convert users
KVM: selftests: Add kvm_sched_getaffinity() wrapper and convert users
KVM: selftests: Add pin_task_to_random_cpu() helper function
KVM: selftests: Verify vCPU migration during IRQ delivery in IRQ test
KVM: selftests: Print vCPU affinity on timeout during IRQ test

 tools/testing/selftests/kvm/Makefile.kvm      |   8 +-
 tools/testing/selftests/kvm/arch_timer.c      |   2 +-
 .../kvm/arm64/arch_timer_edge_cases.c         |   2 +-
 .../selftests/kvm/demand_paging_test.c        |   2 +-
 .../selftests/kvm/dirty_log_perf_test.c       |   4 +-
 tools/testing/selftests/kvm/dirty_log_test.c  |   8 +-
 .../selftests/kvm/include/kvm_syscalls.h      |   6 +
 .../testing/selftests/kvm/include/kvm_util.h  |  13 +
 .../testing/selftests/kvm/include/proc_util.h |  28 ++
 .../testing/selftests/kvm/include/test_util.h |  25 +-
 .../selftests/kvm/include/x86/kvm_util_arch.h |   4 +-
 tools/testing/selftests/kvm/irq_test.c        | 370 ++++++++++++++++++
 tools/testing/selftests/kvm/lib/assert.c      |   8 +-
 tools/testing/selftests/kvm/lib/kvm_util.c    |  95 ++++-
 tools/testing/selftests/kvm/lib/memstress.c   |   8 +-
 tools/testing/selftests/kvm/lib/proc_util.c   |  62 +++
Quoted 10 lines
 tools/testing/selftests/kvm/lib/test_util.c   |  27 +-
 tools/testing/selftests/kvm/mmu_stress_test.c |  15 +-
 tools/testing/selftests/kvm/rseq_test.c       |   6 +-
 tools/testing/selftests/kvm/steal_time.c      |  22 +-
 20 files changed, 635 insertions(+), 80 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/include/proc_util.h
 create mode 100644 tools/testing/selftests/kvm/irq_test.c
 create mode 100644 tools/testing/selftests/kvm/lib/proc_util.c

Please read through Documentation/process/maintainer-kvm-x86.rst. This isn't strictly scoped to KVM x86, but at the same time it's pretty obvious who is going to be applying this series.

Specifically:

Git Base
~~~~~~~~
If you are using git version 2.9.0 or later (Googlers, this is all of you!), use ``git format-patch`` with the ``--base`` flag to automatically include the base tree information in the generated patches.

Note, ``--base=auto`` works as expected if and only if a branch's upstream is set to the base topic branch, e.g. it will do the wrong thing if your upstream is set to your personal repository for backup purposes. An alternative "auto" solution is to derive the names of your development branches based on their KVM x86 topic, and feed that into ``--base``. E.g. ``x86/pmu/my_branch_name``, and then write a small wrapper to extract ``pmu`` from the current branch name to yield ``--base=x/pmu``, where ``x`` is whatever name your repository uses to track the KVM x86 remote.

and to a lesser extent:

Base Tree/Branch
~~~~~~~~~~~~~~~~
Fixes that target the current release, a.k.a. mainline, should be based on ``git://git.kernel.org/pub/scm/virt/kvm/kvm.git master``. Note, fixes do not automatically warrant inclusion in the current release. There is no singular rule, but typically only fixes for bugs that are urgent, critical, and/or were introduced in the current release should target the current release.

Everything else should be based on ``kvm-x86/next``, i.e. there is no need to select a specific topic branch as the base. If there are conflicts and/or dependencies across topic branches, it is the maintainer's job to sort them out.

The only exception to using ``kvm-x86/next`` as the base is if a patch/series is a multi-arch series, i.e. has non-trivial modifications to common KVM code and/or has more than superficial changes to other architectures' code. Multi- arch patch/series should instead be based on a common, stable point in KVM's history, e.g. the release candidate upon which ``kvm-x86 next`` is based. If you're unsure whether a patch/series is truly multi-arch, err on the side of caution and treat it as multi-arch, i.e. use a common base.

Through random guessing, I discovered that this appears to be based on 7.0.
(a) I shouldn't have to guess. (b) that's simply too old of a base, e.g. this doesn't even apply clean only 7.1-rc1.

While there's room for interpretation in the "Base Tree/Branch" section, IMO using any base older than 7.1-rc2 is blatantly wrong. Any of 7.1-rc2+, kvm/next, or kvm-x86/next would be acceptable (though fairly literal reading of the docs would say "don't use kvm-x86/next").

Quoted 2 lines

2.54.0.929.g9b7fa37559-goog