KVM: selftests: Add eventfd+VFIO IRQ test
Sean Christopherson
maintainerv7 cover KVM: selftests: Add eventfd+VFIO IRQ test
David and Josh's series to add a selftest for verifying interrupt delivery via eventfd (via KVM_IRQFD), and also from a real device, wired up via VFIO.
I originally wanted to get this into 7.2, but that's not going to happen. But I hope to get this applied early in the 7.3 cycle so that additional features and whatnot can be developed on top without too much pain (hopefully).
Gory details in the patches, and in the v5 cover letter.
v7:
- Seed kvm_rng during kvm_selftest_init(). [Sashiko]
- Seed libc's RNG during kvm_selftest_init(), so that the seed (from random()) that's printed and passed to kvm_rng isn't the same every time.
- Init irq_cpu to -1 in all paths. [Sashiko]
v6:
- Massage most changelogs.
- Fix SoB ordering issues.
- Clean up KVM_SET_GSI_ROUTING helper.
- Remove misleading "IRQ injection" and "emulated eventfd" terminology.
- Add GUEST_RECEIVED_INTERRUPT() to simplifiy the core loop.
- Use cpu_relax() in tight loops while waiting for interrupts.
- Print as much information as possible in the actual assert, instead of printing to stdout separately.
- Make a best guess as to the right VFIO vs. IOMMUFD mode instead of assuming IOMMUFD, and give the user the option to overide said guess.
- Simplify open_proc_irq_smp_affinity_list() + write_proc_irq_smp_affinity_list() into proc_irq_set_smp_affinity().
- Drop print_proc_irq_file() and kvm_print_vcpu_affinity() (for now) to avoid potential issues on systems with high CPU counts.
- Drop the blocking/HLT testing as it was at best broken.
- Use -e for "empty", not -c for "clear", when completely tearing down GSI routing, because routing can be "cleared" without completely emptying the routing information.
- Use the main task's CPU affinity as the available_cpus set.
- Allow overcommiting vCPUs:pCPUs.
- Set the target vCPU's affinity instead of batching when vCPU0 is targeted.
- Add support for 256+ vCPUs with x2APIC.
- Restrict xAPIC mode to 255 vCPUs.
- Restrict the test to KVM selftest's max supported vCPUs.
v5:
- https://lore.kernel.org/all/[email protected]
- Rename get_proc_vfio_irq_number() to vfio_msix_to_host_irq()
- Rename open_proc_irq_affinity() and write_proc_irq_affinity() to include "_smp_affinity_list"
- Print /proc/irq/<irq>/smp_affinity and effective_affinity on timeout failures
- Convert IRQ type from 'int' to 'unsigned int' across helpers and the test
- Fix compiler warnings for uninitialized variables in irq_test.c
- Remove rate-limiting on affinity changes
v4: https://lore.kernel.org/kvm/[email protected]
David Matlack (11):
KVM: selftests: Build and link selftests/vfio/lib into KVM selftests
KVM: selftests: Add macros to read/write+sync to/from guest memory
KVM: selftests: Add an irqfd send+receive (and later IRQ bypass) test
KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ
bypass test
KVM: selftests: Add VFIO device support to eventfd IRQ test
KVM: selftests: Verify interrupts are received when IRQ affinity
changes in IRQ test
KVM: selftests: Add option to set empty routing between IRQs in
eventfd IRQ test
KVM: selftests: Make number of IRQs configurable in IRQ test
KVM: selftests: Verify vCPU migration during IRQ delivery in IRQ test
KVM: selftests: Make number of vCPUs configurable in IRQ test
KVM: selftests: Add xAPIC support in eventfd IRQ test
Josh Hilke (6):
KVM: selftests: Rename guest_rng to kvm_rng
KVM: selftests: Add helper to generate random u64 in range [min,max]
KVM: selftests: Add a helper to set proc IRQ affinity for IRQ test
KVM: selftests: Add kvm_gettid() wrapper and convert users
KVM: selftests: Add kvm_sched_getaffinity() wrapper and convert users
KVM: selftests: Add a utility to pin a task to a random CPU, given a
CPU set
Sean Christopherson (3):
KVM: selftests: Initialize the default/global pRNG during
kvm_selftest_init()
KVM: selftests: Seed libc's RNG before using it to generate a seed for
KVM's pRNG
KVM: selftests: Verify non-postable IRQ remapping in IRQ test
tools/testing/selftests/kvm/Makefile.kvm | 7 +- tools/testing/selftests/kvm/arch_timer.c | 2 +- .../kvm/arm64/arch_timer_edge_cases.c | 2 +- .../selftests/kvm/demand_paging_test.c | 2 +- .../selftests/kvm/dirty_log_perf_test.c | 4 +- tools/testing/selftests/kvm/dirty_log_test.c | 11 +- .../selftests/kvm/include/kvm_syscalls.h | 7 + .../testing/selftests/kvm/include/kvm_util.h | 12 + .../testing/selftests/kvm/include/proc_util.h | 11 + .../testing/selftests/kvm/include/test_util.h | 25 +- .../selftests/kvm/include/x86/kvm_util_arch.h | 4 +- tools/testing/selftests/kvm/irq_test.c | 350 ++++++++++++++++++ tools/testing/selftests/kvm/lib/assert.c | 8 +- tools/testing/selftests/kvm/lib/kvm_util.c | 53 ++- tools/testing/selftests/kvm/lib/memstress.c | 8 +- tools/testing/selftests/kvm/lib/proc_util.c | 54 +++ tools/testing/selftests/kvm/lib/test_util.c | 27 +- tools/testing/selftests/kvm/mmu_stress_test.c | 15 +- tools/testing/selftests/kvm/rseq_test.c | 6 +- tools/testing/selftests/kvm/steal_time.c | 22 +- .../testing/selftests/kvm/x86/sev_dbg_test.c | 2 +- 21 files changed, 548 insertions(+), 84 deletions(-) create mode 100644 tools/testing/selftests/kvm/include/proc_util.h create mode 100644 tools/testing/selftests/kvm/irq_test.c create mode 100644 tools/testing/selftests/kvm/lib/proc_util.c
base-commit: c1f7303302927f9cbf4efedf70f0512cde168c65
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 1/20 KVM: selftests: Build and link selftests/vfio/lib into KVM selftests
From: David Matlack <[email protected]>
Include libvfio.mk into the KVM selftests Makefile and link it into all KVM selftests by adding it to LIBKVM_OBJS.
This lays the groundwork for future changes to utilize VFIO devices to verify IRQ bypass in KVM selftests.
Note that KVM selftests build their own copy of selftests/vfio/lib and the resulting object files are placed in $(OUTPUT)/lib. This allows the KVM and VFIO selftests to apply different CFLAGS when building without conflicting with each other.
--- tools/testing/selftests/kvm/Makefile.kvm | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/Makefile.kvm +4 -1
@@ -258,6 +258,7 @@ OVERRIDE_TARGETS = 1 # which causes the environment variable to override the makefile). include ../lib.mk include ../cgroup/lib/libcgroup.mk +include ../vfio/lib/libvfio.mk INSTALL_HDR_PATH = $(top_srcdir)/usr LINUX_HDR_PATH = $(INSTALL_HDR_PATH)/include/
@@ -312,7 +313,9 @@ LIBKVM_S := $(filter %.S,$(LIBKVM)) LIBKVM_C_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_C)) LIBKVM_S_OBJ := $(patsubst %.S, $(OUTPUT)/%.o, $(LIBKVM_S)) LIBKVM_STRING_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_STRING)) -LIBKVM_OBJS = $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ) $(LIBKVM_STRING_OBJ) $(LIBCGROUP_O) +LIBKVM_OBJS = $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ) $(LIBKVM_STRING_OBJ) +LIBKVM_OBJS += $(LIBCGROUP_O) +LIBKVM_OBJS += $(LIBVFIO_O) SPLIT_TEST_GEN_PROGS := $(patsubst %, $(OUTPUT)/%, $(SPLIT_TESTS)) SPLIT_TEST_GEN_OBJ := $(patsubst %, $(OUTPUT)/$(ARCH)/%.o, $(SPLIT_TESTS))
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 2/20 KVM: selftests: Add macros to read/write+sync to/from guest memory
In reply to Sean ChristophersonFrom: David Matlack <[email protected]>
Add SYNC_FROM_GUEST_AND_READ(vm, variable), to read a variable value from the guest. Add WRITE_AND_SYNC_TO_GUEST(vm, variable, value) to write a value to a guest variable. These macros improve the readability of code which reads and writes data between host and guest in tests.
Use the new macro in existing tests that do back-to-back write+sync.
No functional changes are intended.
--- tools/testing/selftests/kvm/dirty_log_test.c | 9 +++----- .../testing/selftests/kvm/include/kvm_util.h | 10 +++++++++ tools/testing/selftests/kvm/mmu_stress_test.c | 9 +++----- tools/testing/selftests/kvm/steal_time.c | 22 +++++++------------ 4 files changed, 24 insertions(+), 26 deletions(-)
tools/testing/selftests/kvm/dirty_log_test.c +3 -6
@@ -708,8 +708,7 @@ static void run_test(enum vm_guest_mode mode, void *arg)sync_global_to_guest (vm, iteration); -WRITE_ONCE (nr_writes,0 ); -sync_global_to_guest (vm, nr_writes); +WRITE_AND_SYNC_TO_GUEST (vm, nr_writes,0 ); dirty_ring_prev_iteration_last_page = dirty_ring_last_page;WRITE_ONCE (dirty_ring_vcpu_ring_full, false);
@@ -775,16 +774,14 @@ static void run_test(enum vm_guest_mode mode, void *arg)* writing memory during verification, pages that this thread * sees as clean may be written with this iteration's value. */ -WRITE_ONCE (vcpu_stop, true); -sync_global_to_guest (vm, vcpu_stop); +WRITE_AND_SYNC_TO_GUEST (vm, vcpu_stop, true);sem_wait (&sem_vcpu_stop);/* * Clear vcpu_stop after the vCPU thread has acknowledge the * stop request and is waiting, i.e. is definitely not running! */ -WRITE_ONCE (vcpu_stop, false); -sync_global_to_guest (vm, vcpu_stop); +WRITE_AND_SYNC_TO_GUEST (vm, vcpu_stop, false);/* * Sync the number of writes performed before verification, the
tools/testing/selftests/kvm/include/kvm_util.h +10 -0
@@ -1138,6 +1138,16 @@ vm_adjust_num_guest_pages(enum vm_guest_mode mode, unsigned int num_guest_pages)memcpy (&(g), _p,sizeof (g)); \ }) +#define SYNC_FROM_GUEST_AND_READ (_vm, _variable) ({ \ + sync_global_from_guest(_vm, _variable); \ + READ_ONCE(_variable); \ +}) + +#define WRITE_AND_SYNC_TO_GUEST (_vm, _variable, _value) do { \ + WRITE_ONCE(_variable, _value); \ + sync_global_to_guest(_vm, _variable); \ +} while (0) +/* * Write a global value, but only in the VM's (guest's) domain. Primarily used * for "globals" that hold per-VM values (VMs always duplicate code and global
tools/testing/selftests/kvm/mmu_stress_test.c +3 -6
@@ -155,10 +155,8 @@ static void *vcpu_worker(void *data) "Expected EFAULT on write to RO memory, got r = %d, errno = %d", r, errno);atomic_inc (&nr_ro_faults); -if (atomic_read (&nr_ro_faults) == nr_vcpus) { -WRITE_ONCE (all_vcpus_hit_ro_fault, true); -sync_global_to_guest (vm, all_vcpus_hit_ro_fault); - } +if (atomic_read (&nr_ro_faults) == nr_vcpus) +WRITE_AND_SYNC_TO_GUEST (vm, all_vcpus_hit_ro_fault, true);#if defined(__x86_64__) || defined(__aarch64__)/*
@@ -383,8 +381,7 @@ int main(int argc, char *argv[])rendezvous_with_vcpus (&time_run2,"run 2" );mprotect (mem, slot_size,PROT_READ ); - mprotect_ro_done = true; -sync_global_to_guest (vm, mprotect_ro_done); +WRITE_AND_SYNC_TO_GUEST (vm, mprotect_ro_done, true);rendezvous_with_vcpus (&time_ro,"mprotect RO" );mprotect (mem, slot_size,PROT_READ |PROT_WRITE );
tools/testing/selftests/kvm/steal_time.c +8 -14
@@ -70,8 +70,8 @@ static bool is_steal_time_supported(struct kvm_vcpu *vcpu)static void steal_time_init (struct kvm_vcpu *vcpu,u32 i) {/* ST_GPA_BASE is identity mapped */ - st_gva[i] = (void *)(ST_GPA_BASE + i *STEAL_TIME_SIZE ); -sync_global_to_guest (vcpu->vm, st_gva[i]); +WRITE_AND_SYNC_TO_GUEST (vcpu->vm, st_gva[i], + (void *)(ST_GPA_BASE + i *STEAL_TIME_SIZE ));vcpu_set_msr (vcpu,MSR_KVM_STEAL_TIME , (ulong )st_gva[i] |KVM_MSR_ENABLED ); }
@@ -187,8 +187,7 @@ static void steal_time_init(struct kvm_vcpu *vcpu, u32 i) };/* ST_GPA_BASE is identity mapped */ - st_gva[i] = (void *)(ST_GPA_BASE + i *STEAL_TIME_SIZE ); -sync_global_to_guest (vm, st_gva[i]); +WRITE_AND_SYNC_TO_GUEST (vm, st_gva[i], (void *)(ST_GPA_BASE + i *STEAL_TIME_SIZE )); st_ipa = (ulong )st_gva[i];vcpu_ioctl (vcpu,KVM_SET_DEVICE_ATTR , &dev);
@@ -310,10 +309,8 @@ static bool is_steal_time_supported(struct kvm_vcpu *vcpu)static void steal_time_init (struct kvm_vcpu *vcpu,u32 i) {/* ST_GPA_BASE is identity mapped */ - st_gva[i] = (void *)(ST_GPA_BASE + i *STEAL_TIME_SIZE ); - st_gpa[i] =addr_gva2gpa (vcpu->vm, (gva_t )st_gva[i]); -sync_global_to_guest (vcpu->vm, st_gva[i]); -sync_global_to_guest (vcpu->vm, st_gpa[i]); +WRITE_AND_SYNC_TO_GUEST (vcpu->vm, st_gva[i], (void *)(ST_GPA_BASE + i *STEAL_TIME_SIZE )); +WRITE_AND_SYNC_TO_GUEST (vcpu->vm, st_gpa[i],addr_gva2gpa (vcpu->vm, (gva_t )st_gva[i])); }static void steal_time_dump (struct kvm_vm *vm,u32 vcpu_idx)
@@ -442,8 +439,7 @@ static void steal_time_init(struct kvm_vcpu *vcpu, u32 i) };/* ST_GPA_BASE is identity mapped */ - st_gva[i] = (void *)(ST_GPA_BASE + i *STEAL_TIME_SIZE ); -sync_global_to_guest (vm, st_gva[i]); +WRITE_AND_SYNC_TO_GUEST (vm, st_gva[i], (void *)(ST_GPA_BASE + i *STEAL_TIME_SIZE )); err =__vcpu_ioctl (vcpu,KVM_HAS_DEVICE_ATTR , &attr);TEST_ASSERT (err ==0 ,"No PV stealtime Feature" );
@@ -549,8 +545,7 @@ int main(int ac, char **av)/* Second VCPU run, expect guest stolen time to be <= run_delay */ run_vcpu (vcpus[i]); -sync_global_from_guest (vm, guest_stolen_time[i]); - stolen_time = guest_stolen_time[i]; + stolen_time =SYNC_FROM_GUEST_AND_READ (vm, guest_stolen_time[i]); run_delay =get_run_delay (); TEST_ASSERT(stolen_time <= run_delay, "Expected stolen time <= %ld, got %ld",
@@ -570,8 +565,7 @@ int main(int ac, char **av)/* Run VCPU again to confirm stolen time is consistent with run_delay */ run_vcpu (vcpus[i]); -sync_global_from_guest (vm, guest_stolen_time[i]); - stolen_time = guest_stolen_time[i] - stolen_time; + stolen_time =SYNC_FROM_GUEST_AND_READ (vm, guest_stolen_time[i]) - stolen_time;TEST_ASSERT (stolen_time >= run_delay,"Expected stolen time >= %ld, got %ld" , run_delay, stolen_time);
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 3/20 KVM: selftests: Rename guest_rng to kvm_rng
In reply to Sean ChristophersonFrom: Josh Hilke <[email protected]>
Rename functions prefixed with 'guest_random_' to 'kvm_random_' and the global random state variable 'guest_rng' to 'kvm_rng', as the pRNG isn't strictly limited to guest code. This will allow using the pRNG in host code without creating confusing/misleading function calls.
No functional changes are intended.
--- .../selftests/kvm/dirty_log_perf_test.c | 4 ++-- tools/testing/selftests/kvm/dirty_log_test.c | 2 +- .../testing/selftests/kvm/include/test_util.h | 22 +++++++++---------- .../selftests/kvm/include/x86/kvm_util_arch.h | 4 ++-- tools/testing/selftests/kvm/lib/kvm_util.c | 20 ++++++++--------- tools/testing/selftests/kvm/lib/memstress.c | 8 +++---- tools/testing/selftests/kvm/lib/test_util.c | 6 ++--- .../testing/selftests/kvm/x86/sev_dbg_test.c | 2 +- 8 files changed, 34 insertions(+), 34 deletions(-)
tools/testing/selftests/kvm/dirty_log_perf_test.c +2 -2
@@ -311,7 +311,7 @@ int main(int argc, char *argv[])int opt;/* Override the seed to be deterministic by default. */ - guest_random_seed =1 ; + kvm_random_seed =1 ; dirty_log_manual_caps =kvm_check_cap (KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 );
@@ -357,7 +357,7 @@ int main(int argc, char *argv[]) p.phys_offset =strtoull (optarg,NULL ,0 );break ;case 'r' : - guest_random_seed =atoi_positive ("Random seed" , optarg); + kvm_random_seed =atoi_positive ("Random seed" , optarg);break ;case 's' : p.backing_src =parse_backing_src_type (optarg);
tools/testing/selftests/kvm/dirty_log_test.c +1 -1
@@ -121,7 +121,7 @@ static void guest_code(void)while (true) {while (!READ_ONCE (vcpu_stop)) { addr = guest_test_virt_mem; - addr += (guest_random_u64 (&guest_rng) % guest_num_pages) + addr += (kvm_random_u64 (&kvm_rng) % guest_num_pages) * guest_page_size; addr =align_down (addr, host_page_size);
tools/testing/selftests/kvm/include/test_util.h +11 -11
@@ -108,30 +108,30 @@ struct timespec timespec_sub(struct timespec ts1, struct timespec ts2);struct timespec timespec_elapsed (struct timespec start);struct timespec timespec_div (struct timespec ts,int divisor); -struct guest_random_state { +struct kvm_random_state {u32 seed; }; -extern u32 guest_random_seed; -extern struct guest_random_state guest_rng; +extern u32 kvm_random_seed; +extern struct kvm_random_state kvm_rng; -struct guest_random_state new_guest_random_state (u32 seed); -u32 guest_random_u32 (struct guest_random_state *state); +struct kvm_random_state new_kvm_random_state (u32 seed); +u32 kvm_random_u32 (struct kvm_random_state *state); -static inline bool __guest_random_bool (struct guest_random_state *state, +static inline bool __kvm_random_bool (struct kvm_random_state *state,u8 percent) { -return (guest_random_u32 (state) %100 ) < percent; +return (kvm_random_u32 (state) %100 ) < percent; } -static inline bool guest_random_bool (struct guest_random_state *state) +static inline bool kvm_random_bool (struct kvm_random_state *state) { -return __guest_random_bool (state,50 ); +return __kvm_random_bool (state,50 ); } -static inline u64 guest_random_u64 (struct guest_random_state *state) +static inline u64 kvm_random_u64 (struct kvm_random_state *state) { -return ((u64 )guest_random_u32 (state) <<32 ) |guest_random_u32 (state); +return ((u64 )kvm_random_u32 (state) <<32 ) |kvm_random_u32 (state); }enum vm_mem_backing_src_type {
tools/testing/selftests/kvm/include/x86/kvm_util_arch.h +2 -2
@@ -55,9 +55,9 @@ static inline bool __vm_arch_has_protected_memory(struct kvm_vm_arch *arch) do { \ const typeof(mem) val = (__val); \ \ - if (!is_forced_emulation_enabled || guest_random_bool(&guest_rng)) { \ + if (!is_forced_emulation_enabled || kvm_random_bool(&kvm_rng)) { \ (mem) = val; \ - } else if (guest_random_bool(&guest_rng)) { \ + } else if (kvm_random_bool(&kvm_rng)) { \ __asm__ __volatile__(KVM_FEP "mov %1, %0" \ : "+m" (mem) \ : "r" (val) : "memory"); \
tools/testing/selftests/kvm/lib/kvm_util.c +10 -10
@@ -20,9 +20,9 @@#define KVM_UTIL_MIN_PFN 2 -u32 guest_random_seed; -struct guest_random_state guest_rng; -static u32 last_guest_seed; +u32 kvm_random_seed; +struct kvm_random_state kvm_rng; +static u32 last_kvm_seed;static size_t vcpu_mmap_sz (void );
@@ -515,12 +515,12 @@ struct kvm_vm *__vm_create(struct vm_shape shape, u32 nr_runnable_vcpus, slot0 =memslot2region (vm,0 );ucall_init (vm, slot0->region.guest_phys_addr + slot0->region.memory_size); -if (guest_random_seed != last_guest_seed) { -pr_info ("Random seed: 0x%x\n" , guest_random_seed); - last_guest_seed = guest_random_seed; +if (kvm_random_seed != last_kvm_seed) { +pr_info ("Random seed: 0x%x\n" , kvm_random_seed); + last_kvm_seed = kvm_random_seed; } - guest_rng =new_guest_random_state (guest_random_seed); -sync_global_to_guest (vm, guest_rng); + kvm_rng =new_kvm_random_state (kvm_random_seed); +sync_global_to_guest (vm, kvm_rng);kvm_arch_vm_post_create (vm, nr_runnable_vcpus);
@@ -2279,8 +2279,8 @@ void __attribute((constructor)) kvm_selftest_init(void)sigaction (SIGILL , &sig_sa,NULL );sigaction (SIGFPE , &sig_sa,NULL ); - guest_random_seed = last_guest_seed =random (); -pr_info ("Random seed: 0x%x\n" , guest_random_seed); + kvm_random_seed = last_kvm_seed =random (); +pr_info ("Random seed: 0x%x\n" , kvm_random_seed);kvm_selftest_arch_init (); }
tools/testing/selftests/kvm/lib/memstress.c +4 -4
@@ -48,14 +48,14 @@ void memstress_guest_code(u32 vcpu_idx) {struct memstress_args *args = &memstress_args;struct memstress_vcpu_args *vcpu_args = &args->vcpu_args[vcpu_idx]; -struct guest_random_state rand_state; +struct kvm_random_state rand_state;gva_t gva;u64 pages;u64 addr;u64 page;int i; - rand_state =new_guest_random_state (guest_random_seed + vcpu_idx); + rand_state =new_kvm_random_state (kvm_random_seed + vcpu_idx); gva = vcpu_args->gva; pages = vcpu_args->pages;
@@ -69,13 +69,13 @@ void memstress_guest_code(u32 vcpu_idx)for (i =0 ; i < pages; i++) {if (args->random_access) - page =guest_random_u32 (&rand_state) % pages; + page =kvm_random_u32 (&rand_state) % pages;else page = i; addr = gva + (page * args->guest_page_size); -if (__guest_random_bool (&rand_state, args->write_percent)) +if (__kvm_random_bool (&rand_state, args->write_percent)) *(u64 *)addr =0x0123456789ABCDEF ;else READ_ONCE (*(u64 *)addr);
tools/testing/selftests/kvm/lib/test_util.c +3 -3
@@ -30,13 +30,13 @@ void __attribute__((used)) expect_sigbus_handler(int signum)* Park-Miller LCG using standard constants. */ -struct guest_random_state new_guest_random_state (u32 seed) +struct kvm_random_state new_kvm_random_state (u32 seed) { -struct guest_random_state s = {.seed = seed}; +struct kvm_random_state s = {.seed = seed};return s; } -u32 guest_random_u32 (struct guest_random_state *state) +u32 kvm_random_u32 (struct kvm_random_state *state) { state->seed = (u64 )state->seed *48271 % ((u32)(1 <<31 ) -1 );return state->seed;
tools/testing/selftests/kvm/x86/sev_dbg_test.c +1 -1
@@ -34,7 +34,7 @@ static void validate_buffers(void)static void ____test_sev_dbg (struct kvm_vm *vm,int i,int j,int nr_bytes) { -u8 pattern =guest_random_u32 (&guest_rng); +u8 pattern =kvm_random_u32 (&kvm_rng);if (i + nr_bytes >BUFFER_SIZE || j + nr_bytes >BUFFER_SIZE )return ;
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 4/20 KVM: selftests: Initialize the default/global pRNG during kvm_selftest_init()
In reply to Sean ChristophersonInitialize the default kvm_rng during selftest initialization so that the pRNG can be used by tests before creating a VM. As pointed out by Sashiko, failure to actually initialize the generate makes it decidedly not random.
--- tools/testing/selftests/kvm/lib/kvm_util.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
tools/testing/selftests/kvm/lib/kvm_util.c +11 -7
@@ -24,6 +24,13 @@ u32 kvm_random_seed;struct kvm_random_state kvm_rng;static u32 last_kvm_seed; +static void kvm_seed_rng (u32 seed) +{ + kvm_random_seed = last_kvm_seed = seed; +pr_info ("Random seed: 0x%x\n" , kvm_random_seed); + kvm_rng =new_kvm_random_state (kvm_random_seed); +} +static size_t vcpu_mmap_sz (void );int __open_path_or_exit (const char *path,int flags,const char *enoent_help)
@@ -515,11 +522,9 @@ struct kvm_vm *__vm_create(struct vm_shape shape, u32 nr_runnable_vcpus, slot0 =memslot2region (vm,0 );ucall_init (vm, slot0->region.guest_phys_addr + slot0->region.memory_size); -if (kvm_random_seed != last_kvm_seed) { -pr_info ("Random seed: 0x%x\n" , kvm_random_seed); - last_kvm_seed = kvm_random_seed; - } - kvm_rng =new_kvm_random_state (kvm_random_seed); +if (kvm_random_seed != last_kvm_seed) +kvm_seed_rng (kvm_random_seed); +sync_global_to_guest (vm, kvm_rng);kvm_arch_vm_post_create (vm, nr_runnable_vcpus);
@@ -2279,8 +2284,7 @@ void __attribute((constructor)) kvm_selftest_init(void)sigaction (SIGILL , &sig_sa,NULL );sigaction (SIGFPE , &sig_sa,NULL ); - kvm_random_seed = last_kvm_seed =random (); -pr_info ("Random seed: 0x%x\n" , kvm_random_seed); +kvm_seed_rng (random ());kvm_selftest_arch_init (); }
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 5/20 KVM: selftests: Seed libc's RNG before using it to generate a seed for KVM's pRNG
In reply to Sean ChristophersonSeed the RNG used by random() using the de facto standard method of srand(time(0)), so that a different seed is actually used in each test run. E.g. without seeding the RNG, literally every test on x86 will use 0x6b8b4567 to seed the KVM RNG.
--- tools/testing/selftests/kvm/lib/kvm_util.c | 1 + 1 file changed, 1 insertion(+)
tools/testing/selftests/kvm/lib/kvm_util.c +1 -0
@@ -2284,6 +2284,7 @@ void __attribute((constructor)) kvm_selftest_init(void)sigaction (SIGILL , &sig_sa,NULL );sigaction (SIGFPE , &sig_sa,NULL ); +srandom (time (0 ));kvm_seed_rng (random ());kvm_selftest_arch_init ();
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
[email protected]
AI reviewRe: [PATCH v7 05/20] KVM: selftests: Seed libc's RNG before using it to generate a seed for KVM's pRNG
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Missing `<time.h>` inclusion causes an implicit function declaration of `time()`. --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Missing <time.h> inclusion causes an implicit function declaration of time(). --
commit 3749f42be6b5df3b7d863a819171920d1f5980cc
Author: Sean Christopherson <[email protected]>
KVM: selftests: Seed libc's RNG before using it to generate a seed for KVM's pRNG
This commit updates the KVM selftests initialization to seed the RNG used by random() with the current time. This ensures a different seed is used in each test run instead of using the same static seed.
tools/testing/selftests/kvm/lib/kvm_util.c void __attribute((constructor)) kvm_selftest_init(void) · line 2287
Quoted 9 lines
tools/testing/selftests/kvm/lib/kvm_util.c +1
-0@@ -2284,6 +2284,7 @@ void __attribute((constructor)) kvm_selftest_init(void)sigaction (SIGILL , &sig_sa,NULL );sigaction (SIGFPE , &sig_sa,NULL ); +srandom (time (0 ));
[Severity: Low]
Does this file need to explicitly include <time.h>?
While it might compile cleanly on standard glibc environments due to transitive inclusions, using time() without explicitly including <time.h> can cause an implicit function declaration warning or error on stricter build environments like musl.
Quoted 3 lines
tools/testing/selftests/kvm/lib/kvm_util.c +0
-0kvm_seed_rng (random ());kvm_selftest_arch_init ();
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Sean Christopherson
maintainerPATCH v7 6/20 KVM: selftests: Add helper to generate random u64 in range [min,max]
In reply to Sean ChristophersonFrom: Josh Hilke <[email protected]>
Introduce kvm_random_u64_in_range(state, min, max). This function returns a random u64 in the inclusive range of [min, max] using a struct kvm_random_state.
--- .../testing/selftests/kvm/include/test_util.h | 3 +++ tools/testing/selftests/kvm/lib/test_util.c | 18 ++++++++++++++++++ 2 files changed, 21 insertions(+)
tools/testing/selftests/kvm/include/test_util.h +3 -0
@@ -134,6 +134,9 @@ static inline u64 kvm_random_u64(struct kvm_random_state *state)return ((u64 )kvm_random_u32 (state) <<32 ) |kvm_random_u32 (state); } +u64 kvm_random_u64_in_range (struct kvm_random_state *state,u64 min, +u64 max); +enum vm_mem_backing_src_type {VM_MEM_SRC_ANONYMOUS ,VM_MEM_SRC_ANONYMOUS_THP ,
tools/testing/selftests/kvm/lib/test_util.c +18 -0
@@ -42,6 +42,24 @@ u32 kvm_random_u32(struct kvm_random_state *state)return state->seed; } +/* Returns a random u64 in the inclusive range [min, max] */ +u64 kvm_random_u64_in_range (struct kvm_random_state *state,u64 min, +u64 max) +{ +u64 value; +u64 range; + +TEST_ASSERT (min <= max,"PEBKAC, min = 0x%lx, max = 0x%lx" , min, max); + + value =kvm_random_u64 (state); + + range = max - min; +if (range ==ULLONG_MAX ) +return value; + +return min + (value % (range +1 )); +} +/* * Parses "[0-9]+[kmgt]?". */
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
[email protected]
AI reviewRe: [PATCH v7 06/20] KVM: selftests: Add helper to generate random u64 in range [min,max]
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The generic PRNG helper `kvm_random_u64_in_range()` is fundamentally incapable of generating…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The generic PRNG helper kvm_random_u64_in_range() is fundamentally incapable of generating values in large portions of the requested range due to the structural limitations of the underlying generator. --
commit 9b7095f015b8aad9cb0b24b2b051b56cd3917145
Author: Josh Hilke <[email protected]>
KVM: selftests: Add helper to generate random u64 in range [min,max]
Introduce kvm_random_u64_in_range(state, min, max) to return a random u64 in the inclusive range of [min, max] using a struct kvm_random_state.
Quoted 12 lines
tools/testing/selftests/kvm/lib/test_util.c +4
-0@@ -42,6 +42,24 @@ u32 kvm_random_u32(struct kvm_random_state *state)return state->seed; } +/* Returns a random u64 in the inclusive range [min, max] */ +u64 kvm_random_u64_in_range (struct kvm_random_state *state,u64 min, +u64 max) +{
tools/testing/selftests/kvm/lib/test_util.c u32 kvm_random_u32(struct kvm_random_state *state) · line 58
Quoted 10 lines
tools/testing/selftests/kvm/lib/test_util.c +10
-0+u64 value; +u64 range; + +TEST_ASSERT (min <= max,"PEBKAC, min = 0x%lx, max = 0x%lx" , min, max); + + value =kvm_random_u64 (state); + + range = max - min; +if (range ==ULLONG_MAX ) +return value;
[Severity: Medium]
Can this regression cause tests to silently miss important edge cases?
Since kvm_random_u32() uses a linear congruential generator modulo 0x7FFFFFFF, its 31st bit is permanently 0, and it never returns 0.
Because kvm_random_u64() concatenates two such values:
tools/testing/selftests/kvm/include/test_util.h:kvm_random_u64() {
return ((u64)kvm_random_u32(state) << 32) | kvm_random_u32(state);
}bits 31 and 63 of the resulting 64-bit number are permanently 0, and the lower 32 bits are never 0.
When this is called with a large range like min = 0 and max = ULLONG_MAX, it directly returns this 64-bit value. Thus, it can never generate numbers above 0x7FFFFFFE7FFFFFFE, any numbers with bit 31 or 63 set, or any numbers below 2^32.
Does this give a false illusion of full mathematical coverage for randomized inputs?
Quoted 3 lines
tools/testing/selftests/kvm/lib/test_util.c +3
-0+ +return min + (value % (range +1 )); +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Sean Christopherson
maintainerPATCH v7 7/20 KVM: selftests: Add an irqfd send+receive (and later IRQ bypass) test
In reply to Sean ChristophersonFrom: David Matlack <[email protected]>
Add a new test, irq_test to verify that KVM correctly delivers interrupts to a running vCPU, when triggered via an eventfd bound to a KVM GSI using KVM's irqfd mechanism.
This test is intentionally simple, for now. Support for sending interrupts via VFIO devices, for IRQ bypass, and for other features will be added in the near future.
Add the test in common code, even though it currently will only build and run on x86, as the concept and the bulk of the host-side code isn't specific to x86.
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + tools/testing/selftests/kvm/irq_test.c | 170 +++++++++++++++++++++++ 2 files changed, 171 insertions(+) create mode 100644 tools/testing/selftests/kvm/irq_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -157,6 +157,7 @@ TEST_GEN_PROGS_x86 += coalesced_io_test TEST_GEN_PROGS_x86 += dirty_log_perf_test TEST_GEN_PROGS_x86 += guest_memfd_test TEST_GEN_PROGS_x86 += hardware_disable_test +TEST_GEN_PROGS_x86 += irq_test TEST_GEN_PROGS_x86 += mmu_stress_test TEST_GEN_PROGS_x86 += rseq_test TEST_GEN_PROGS_x86 += steal_time
tools/testing/selftests/kvm/irq_test.c new file +170 -0
170 lines not on this page · raw message
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 8/20 KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ bypass test
In reply to Sean ChristophersonFrom: David Matlack <[email protected]>
Introduce proc_util.c and proc_util.h to house utility functions for interacting with the proc filesystem.
Add vfio_msix_to_host_irq(), which parses /proc/interrupts, to get the host Linux IRQ for a given VFIO device BDF and MSI-X vector.
This helper will be used by the eventfd IRQ test to print the host IRQ number when triggering IRQs via VFIO device, e.g. to aid in debugging if the test fails.
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../testing/selftests/kvm/include/proc_util.h | 9 +++++ tools/testing/selftests/kvm/lib/proc_util.c | 40 +++++++++++++++++++ 3 files changed, 50 insertions(+) create mode 100644 tools/testing/selftests/kvm/include/proc_util.h create mode 100644 tools/testing/selftests/kvm/lib/proc_util.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -11,6 +11,7 @@ LIBKVM += lib/kvm_util.c LIBKVM += lib/lru_gen_util.c LIBKVM += lib/memstress.c LIBKVM += lib/guest_sprintf.c +LIBKVM += lib/proc_util.c LIBKVM += lib/rbtree.c LIBKVM += lib/sparsebit.c LIBKVM += lib/test_util.c
tools/testing/selftests/kvm/include/proc_util.h new file +9 -0
@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef SELFTEST_KVM_PROC_UTIL_H +#define SELFTEST_KVM_PROC_UTIL_H + +#include <stdint.h> + +unsigned int vfio_msix_to_host_irq (const char *vfio_device_bdf,int msix); + +#endif /* SELFTEST_KVM_PROC_UTIL_H */
tools/testing/selftests/kvm/lib/proc_util.c new file +40 -0
@@ -0,0 +1,40 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "kvm_util.h" +#include "test_util.h" +#include "proc_util.h" + +static FILE *open_proc_interrupts (void ) +{ +FILE *fp; + + fp =fopen ("/proc/interrupts" ,"r" ); +TEST_ASSERT (fp,"fopen(/proc/interrupts) failed" ); + +return fp; +} + +unsigned int vfio_msix_to_host_irq (const char *device_bdf,int msix) +{ +char search_string[64 ]; +char line[4096 ]; +int irq =-1 ; +FILE *fp; + + fp =open_proc_interrupts (); + +snprintf (search_string,sizeof (search_string),"vfio-msix[%d]" , msix); + +while (fgets (line,sizeof (line), fp)) { +if (strstr (line, device_bdf) &&strstr (line, search_string)) { +TEST_ASSERT_EQ (1 ,sscanf (line,"%d:" , &irq)); +break ; + } + } + +fclose (fp); + +TEST_ASSERT (irq !=-1 ,"Failed to locate IRQ for %s %s" , device_bdf, + search_string); +return (unsigned int )irq; +} +
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
[email protected]
AI reviewRe: [PATCH v7 08/20] KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ bypass test
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Reading /proc/interrupts with a fixed 4096-byte buffer truncates lines on large machines…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Reading /proc/interrupts with a fixed 4096-byte buffer truncates lines on large machines, leading to incorrect IRQ numbers or test crashes. --
commit 399b5fbbf0a019bffe136ff08d75cf31f2c2655f
Author: David Matlack <[email protected]>
KVM: selftests: Add helper to get host IRQ from device MSI-X for IRQ bypass test
This commit introduces proc_util.c and vfio_msix_to_host_irq() to parse /proc/interrupts and retrieve the host Linux IRQ for a given VFIO device BDF and MSI-X vector. The helper is intended to be used by the eventfd IRQ test.
[ ... ]
Quoted 5 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +0
-0
[ ... ]
Quoted 4 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +4
-0+unsigned int vfio_msix_to_host_irq (const char *device_bdf,int msix) +{ +char search_string[64 ]; +char line[4096 ];
tools/testing/selftests/kvm/lib/proc_util.c line 29
Quoted 10 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +10
-0+int irq =-1 ; +FILE *fp; + + fp =open_proc_interrupts (); + +snprintf (search_string,sizeof (search_string),"vfio-msix[%d]" , msix); + +while (fgets (line,sizeof (line), fp)) { +if (strstr (line, device_bdf) &&strstr (line, search_string)) { +TEST_ASSERT_EQ (1 ,sscanf (line,"%d:" , &irq));
[Severity: Medium]
Does this 4096-byte buffer accommodate lines from /proc/interrupts on hosts with a large number of CPUs?
If a host has enough CPUs to push a single line beyond 4096 bytes, fgets() will split the line across multiple reads.
If the target device strings happen to fall into a subsequent chunk, could sscanf() mistakenly parse a CPU interrupt count as the host IRQ, or fail entirely and trigger the assert?
Quoted 10 lines
tools/testing/selftests/kvm/lib/proc_util.c new file +10
-0+break ; + } + } + +fclose (fp); + +TEST_ASSERT (irq !=-1 ,"Failed to locate IRQ for %s %s" , device_bdf, + search_string); +return (unsigned int )irq; +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Sean Christopherson
maintainerPATCH v7 9/20 KVM: selftests: Add VFIO device support to eventfd IRQ test
In reply to Sean ChristophersonFrom: David Matlack <[email protected]>
Extend the eventfd IRQ test with a '-d' argument that takes a BDF (in the format segment:bus:device.function) of an interrupt-capable PCI(e) device bound to VFIO, and use said device to trigger interrupts instead of always synthesizing interrupts via direct writes to the eventfd.
Using a VFIO device to trigger interrupts validates the end-to-end delivery of IRQs for "real" devices, and when supported by hardware (and KVM), also validates interrupt delivery via IRQ bypass, i.e. via device posted IRQs.
Now that IOMMUFD is a thing, auto-probe IOMMUFD vs. "legacy" VFIO by temporarily opening /dev/iommufd, and skip the test if neither IOMMUFD nor legacy VFIO is available. Add a '-t' option to the user override the probe logic, e.g. in case IOMMUFD is available but the system is configured for legacy usage.
Note, the device must have a VFIO selftest driver in order to work with the test. A helper script to list supported devices will hopefully be available in the near future at tools/testing/selftests/vfio/scripts/list_supported_devices.sh[1].
Example:
$ ./tools/testing/selftests/kvm/irq_test -d 0000:06:0a.1
--- tools/testing/selftests/kvm/irq_test.c | 86 ++++++++++++++++++++++++-- 1 file changed, 80 insertions(+), 6 deletions(-)
tools/testing/selftests/kvm/irq_test.c +80 -6
@@ -3,7 +3,10 @@#include "test_util.h" #include "apic.h" #include "processor.h" +#include "proc_util.h" +#include <libvfio.h> +#include <linux/sizes.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h>
@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)return NULL ; } +static int vfio_setup_msi (struct vfio_pci_device *device) +{ +const int flags =MAP_SHARED |MAP_ANONYMOUS ; +const int prot =PROT_READ |PROT_WRITE ; +struct dma_region *region; + +/* A driver is required to generate an MSI. */ +TEST_REQUIRE (device->driver.ops); + +/* Set up a DMA-able region for the driver to use. */ + region = &device->driver.region; + region->iova =0 ; + region->size =SZ_2M ; + region->vaddr =kvm_mmap (region->size, prot, flags,-1 ); +TEST_ASSERT (region->vaddr !=MAP_FAILED ,"mmap() failed\n" ); +iommu_map (device->iommu, region); + +vfio_pci_driver_init (device); +return device->driver.msi; +} + +static void trigger_interrupt (struct vfio_pci_device *device,int eventfd) +{ +if (device) +vfio_pci_driver_send_msi (device); +else +eventfd_write (eventfd,1 ); +} + +static void kvm_route_msi (struct kvm_vm *vm,u32 gsi,struct kvm_vcpu *vcpu,u8 vector) {
@@ -74,11 +107,29 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,vm_ioctl (vm,KVM_SET_GSI_ROUTING , &routing.header); } +static const char *probe_iommu_type (void ) +{ +int io_fd; + + io_fd =open ("/dev/iommu" ,O_RDONLY ); +if (io_fd >=0 ) { +close (io_fd); +return MODE_IOMMUFD ; + } + + io_fd =__open_path_or_exit ("/dev/vfio" ,O_RDONLY , +"Is VFIO (or IOMMUFD) loaded and enabled?" ); +close (io_fd); +return MODE_VFIO_TYPE1_IOMMU ; +} +static void help (const char *name) { -printf ("Usage: %s [-h]\n" , name); +printf ("Usage: %s [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n" , name);printf ("\n" );printf ("Tests KVM interrupt routing and delivery via irqfd.\n" ); +printf ("-d Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n" ); +printf ("-t Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n" );printf ("\n" );exit (KSFT_FAIL ); }
@@ -100,14 +151,25 @@ int main(int argc, char **argv)u32 gsi =kvm_random_u64_in_range (&kvm_rng,24 ,KVM_MAX_IRQ_ROUTES -1 );u8 vector =kvm_random_u64_in_range (&kvm_rng,32 ,UINT8_MAX ); -struct kvm_vcpu *vcpus[KVM_MAX_VCPUS ];pthread_t vcpu_threads[KVM_MAX_VCPUS ]; +struct kvm_vcpu *vcpus[KVM_MAX_VCPUS ]; +struct vfio_pci_device *device =NULL ;int nr_irqs =1000 , nr_vcpus =1 ; -int i, j, c, eventfd; +const char *device_bdf =NULL ; +const char *iommu_type =NULL ; +int i, j, c, msix, eventfd; +struct iommu *iommu;struct kvm_vm *vm; +int irq; -while ((c =getopt (argc, argv,"h" )) !=-1 ) { +while ((c =getopt (argc, argv,"d:ht:" )) !=-1 ) {switch (c) { +case 'd' : + device_bdf = optarg; +break ; +case 't' : + iommu_type = optarg; +break ;case 'h' :default :help (argv[0 ]);
@@ -119,7 +181,19 @@ int main(int argc, char **argv) vm =vm_create_with_vcpus (nr_vcpus, guest_code, vcpus);vm_install_exception_handler (vm, vector, guest_irq_handler); - eventfd =kvm_new_eventfd (); +if (device_bdf) { +if (!iommu_type) + iommu_type =probe_iommu_type (); + iommu =iommu_init (iommu_type); + device =vfio_pci_device_init (device_bdf, iommu); + msix =vfio_setup_msi (device); + irq =vfio_msix_to_host_irq (device_bdf, msix); + eventfd = device->msi_eventfds[msix]; +printf ("Using device %s MSI-X[%d] (IRQ-%u)\n" , device_bdf, msix, + irq); + }else { + eventfd =kvm_new_eventfd (); + }pr_info ("Injecting interrupts for GSI %d (guest vector 0x%x) %d times\n" , gsi, vector, nr_irqs);
@@ -147,7 +221,7 @@ int main(int argc, char **argv)"IRQ flag for vCPU %d not clear prior to test" , vcpus[j]->id); -eventfd_write (eventfd,1 ); +trigger_interrupt (device, eventfd);clock_gettime (CLOCK_MONOTONIC , &start); while (!GUEST_RECEIVED_IRQ(vcpu) &&
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
[email protected]
AI reviewRe: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] probe_iommu_type() incorrectly probes the /dev/vfio directory instead of the /dev/vfio/vfio…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] probe_iommu_type() incorrectly probes the /dev/vfio directory instead of the /dev/vfio/vfio legacy container character device, bypassing the framework's intended skip behavior for unprivileged users.
- [Medium] vfio_setup_msi() hardcodes the device's DMA region IOVA to 0, which can cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0.
--
commit 2bdd8160743775c8d18e4004e72e7ff0898fbd70
Author: David Matlack <[email protected]>
KVM: selftests: Add VFIO device support to eventfd IRQ test
This commit extends the eventfd IRQ test to support triggering interrupts using a VFIO device instead of manually signaling the eventfd. It also probes for IOMMUFD vs legacy VFIO availability.
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0
-0
[ ... ]
Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2
-0@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)return NULL ; } +static int vfio_setup_msi (struct vfio_pci_device *device) +{
tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72
Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10
-0+const int flags =MAP_SHARED |MAP_ANONYMOUS ; +const int prot =PROT_READ |PROT_WRITE ; +struct dma_region *region; + +/* A driver is required to generate an MSI. */ +TEST_REQUIRE (device->driver.ops); + +/* Set up a DMA-able region for the driver to use. */ + region = &device->driver.region; + region->iova =0 ;
[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?
If IOVA 0 is unsupported, it seems like iommu_map() will fail and trigger a test assertion crash instead of running.
Quoted 8 lines
tools/testing/selftests/kvm/irq_test.c +8
-0+ region->size =SZ_2M ; + region->vaddr =kvm_mmap (region->size, prot, flags,-1 ); +TEST_ASSERT (region->vaddr !=MAP_FAILED ,"mmap() failed\n" ); +iommu_map (device->iommu, region); + +vfio_pci_driver_init (device); +return device->driver.msi; +}
[ ... ]
Quoted 5 lines
tools/testing/selftests/kvm/irq_test.c +1
-0@@ -74,11 +107,29 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,vm_ioctl (vm,KVM_SET_GSI_ROUTING , &routing.header); } +static const char *probe_iommu_type (void )
tools/testing/selftests/kvm/irq_test.c static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu, · line 120
Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10
-0+{ +int io_fd; + + io_fd =open ("/dev/iommu" ,O_RDONLY ); +if (io_fd >=0 ) { +close (io_fd); +return MODE_IOMMUFD ; + } + + io_fd = __open_path_or_exit("/dev/vfio", O_RDONLY,
[Severity: Medium]
Because /dev/vfio is a directory, will this open() call succeed for any user with read and execute permissions to the directory, bypassing the intended skip behavior?
Should this probe the /dev/vfio/vfio legacy container character device instead so the test can skip gracefully for unprivileged users rather than crashing later in iommu_init() due to lack of device permissions?
Quoted 5 lines
tools/testing/selftests/kvm/irq_test.c +5
-0+ "Is VFIO (or IOMMUFD) loaded and enabled?"); +close (io_fd); +return MODE_VFIO_TYPE1_IOMMU ; +} +
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test
Quoted 43 lines · On Fri, Jun 12, 2026 at 5:35 PM <[email protected]> wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium]probe_iommu_type()incorrectly probes the/dev/vfiodirectory instead of the/dev/vfio/vfiolegacy container character device, bypassing the framework's intended skip behavior for unprivileged users.
- [Medium]vfio_setup_msi()hardcodes the device's DMA region IOVA to 0, which can cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0.
--commit
2bdd8160743775c8d18e4004e72e7ff0898fbd70
Author: David Matlack <[email protected]>KVM: selftests: Add VFIO device support to eventfd IRQ test
This commit extends the eventfd IRQ test to support triggering interrupts using a VFIO device instead of manually signaling the eventfd. It also probes for IOMMUFD vs legacy VFIO availability.
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0
-0[ ... ]
Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2
-0@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)return NULL ; } +static int vfio_setup_msi (struct vfio_pci_device *device) +{tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72
Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10
-0+const int flags =MAP_SHARED |MAP_ANONYMOUS ; +const int prot =PROT_READ |PROT_WRITE ; +struct dma_region *region; + +/* A driver is required to generate an MSI. */ +TEST_REQUIRE (device->driver.ops); + +/* Set up a DMA-able region for the driver to use. */ + region = &device->driver.region; + region->iova =0 ;[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?
Quoted 2 lines
If IOVA 0 is unsupported, it seems like
iommu_map()will fail and trigger a test assertion crash instead of running.
This is probably fine for now given the test is only for x86_64. But if you are going to send a v8 you can use the VFIO selftests iova_allocator to fix this, instead of hard-coding.
Quoted 31 lines
Quoted 9 lines
tools/testing/selftests/kvm/irq_test.c +8
-0+ region->size =SZ_2M ; + region->vaddr =kvm_mmap (region->size, prot, flags,-1 ); +TEST_ASSERT (region->vaddr !=MAP_FAILED ,"mmap() failed\n" ); +iommu_map (device->iommu, region); + +vfio_pci_driver_init (device); +return device->driver.msi; +}[ ... ]
Quoted 5 lines
tools/testing/selftests/kvm/irq_test.c +1
-0@@ -74,11 +107,29 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,vm_ioctl (vm,KVM_SET_GSI_ROUTING , &routing.header); } +static const char *probe_iommu_type (void )tools/testing/selftests/kvm/irq_test.c static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu, · line 120
Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10
-0+{ +int io_fd; + + io_fd =open ("/dev/iommu" ,O_RDONLY ); +if (io_fd >=0 ) { +close (io_fd); +return MODE_IOMMUFD ; + } + + io_fd = __open_path_or_exit("/dev/vfio", O_RDONLY,[Severity: Medium]
Because/dev/vfiois a directory, will thisopen()call succeed for any user with read and execute permissions to the directory, bypassing the intended skip behavior?
Quoted 3 lines
Should this probe the
/dev/vfio/vfiolegacy container character device instead so the test can skip gracefully for unprivileged users rather than crashing later iniommu_init()due to lack of device permissions?
I agree with Sashiko here.
Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu, · line 125
Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +5
-0+ "Is VFIO (or IOMMUFD) loaded and enabled?"); +close (io_fd); +return MODE_VFIO_TYPE1_IOMMU ; +} +--
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Sean Christopherson
maintainerRe: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test
Quoted 27 lines · David Matlack
Quoted 26 lines · On Fri, Jun 12, 2026 at 5:35 PM <[email protected]> wrote:
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0
-0[ ... ]
Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2
-0@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)return NULL ; } +static int vfio_setup_msi (struct vfio_pci_device *device) +{tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72
Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10
-0+const int flags =MAP_SHARED |MAP_ANONYMOUS ; +const int prot =PROT_READ |PROT_WRITE ; +struct dma_region *region; + +/* A driver is required to generate an MSI. */ +TEST_REQUIRE (device->driver.ops); + +/* Set up a DMA-able region for the driver to use. */ + region = &device->driver.region; + region->iova =0 ;[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?
Quoted 6 lines
Quoted 2 lines
If IOVA 0 is unsupported, it seems like
iommu_map()will fail and trigger a test assertion crash instead of running.This is probably fine for now given the test is only for
x86_64. But if you are going to send a v8 you can use the VFIO selftestsiova_allocatorto fix this, instead of hard-coding.
I'm confused, why is this test setting up a DMA-able region in the first place?
Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test
Quoted 29 lines · Sean Christopherson
Quoted 27 lines · David Matlack
Quoted 26 lines · On Fri, Jun 12, 2026 at 5:35 PM <[email protected]> wrote:
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0
-0[ ... ]
Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2
-0@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)return NULL ; } +static int vfio_setup_msi (struct vfio_pci_device *device) +{tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72
Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10
-0+const int flags =MAP_SHARED |MAP_ANONYMOUS ; +const int prot =PROT_READ |PROT_WRITE ; +struct dma_region *region; + +/* A driver is required to generate an MSI. */ +TEST_REQUIRE (device->driver.ops); + +/* Set up a DMA-able region for the driver to use. */ + region = &device->driver.region; + region->iova =0 ;[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
If IOVA 0 is unsupported, it seems like
iommu_map()will fail and trigger a test assertion crash instead of running.This is probably fine for now given the test is only for
x86_64. But if you are going to send a v8 you can use the VFIO selftestsiova_allocatorto fix this, instead of hard-coding.I'm confused, why is this test setting up a DMA-able region in the first place?
There might be some devices with which you can trigger an MSI with pure MMIO. But most devices you need to at least set up some descriptors in memory that describe what action you want the device to take and then use MMIO to poke the device to read the desriptor and perform the action. Reading those descriptors is DMA.
For Intel DSA specifically, send_msi() works by telling the device to perform an 8-byte memcpy and send an interrupt when it's done. The driver region contains the descriptor for that memcpy, as well as the 8-byte source/destination buffers the device should use.
Sean Christopherson
maintainerRe: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test
Quoted 45 lines · David Matlack
Quoted 29 lines · Sean Christopherson
Quoted 27 lines · David Matlack
Quoted 26 lines · On Fri, Jun 12, 2026 at 5:35 PM <[email protected]> wrote:
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0
-0[ ... ]
Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2
-0@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)return NULL ; } +static int vfio_setup_msi (struct vfio_pci_device *device) +{tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72
Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10
-0+const int flags =MAP_SHARED |MAP_ANONYMOUS ; +const int prot =PROT_READ |PROT_WRITE ; +struct dma_region *region; + +/* A driver is required to generate an MSI. */ +TEST_REQUIRE (device->driver.ops); + +/* Set up a DMA-able region for the driver to use. */ + region = &device->driver.region; + region->iova =0 ;[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
If IOVA 0 is unsupported, it seems like
iommu_map()will fail and trigger a test assertion crash instead of running.This is probably fine for now given the test is only for
x86_64. But if you are going to send a v8 you can use the VFIO selftestsiova_allocatorto fix this, instead of hard-coding.I'm confused, why is this test setting up a DMA-able region in the first place?
There might be some devices with which you can trigger an MSI with pure MMIO. But most devices you need to at least set up some descriptors in memory that describe what action you want the device to take and then use MMIO to poke the device to read the desriptor and perform the action. Reading those descriptors is DMA.
Quoted 4 lines
For Intel DSA specifically,
send_msi()works by telling the device to perform an 8-byte memcpy and send an interrupt when it's done. The driver region contains the descriptor for that memcpy, as well as the 8-byte source/destination buffers the device should use.
Ah, gotcha. For a comment, how's this look?
tools/testing/selftests/kvm/irq_test.c +6 -1
@@ -67,7 +67,12 @@ static int vfio_setup_msi(struct vfio_pci_device *device)/* A driver is required to generate an MSI. */ TEST_REQUIRE (device->driver.ops); -/* Set up a DMA-able region for the driver to use. */ +/* +* Set up a DMA-able region for the driver to use. Very few devices +* provide a way to arbitrarily send interrupts (MSIs), e.g. by writing +* an MMIO register. Instead, most devices send MSIs when an action is +* completed, and practically all actions involve DMA of some form. +*/ region = &device->driver.region; region->iova =0 ; region->size =SZ_2M ;
Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test
Quoted 47 lines · Sean Christopherson
Quoted 45 lines · David Matlack
Quoted 29 lines · Sean Christopherson
Quoted 27 lines · David Matlack
Quoted 26 lines · On Fri, Jun 12, 2026 at 5:35 PM <[email protected]> wrote:
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0
-0[ ... ]
Quoted 6 lines
tools/testing/selftests/kvm/irq_test.c +2
-0@@ -55,6 +58,36 @@ static void *vcpu_thread_main(void *arg)return NULL ; } +static int vfio_setup_msi (struct vfio_pci_device *device) +{tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 72
Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +10
-0+const int flags =MAP_SHARED |MAP_ANONYMOUS ; +const int prot =PROT_READ |PROT_WRITE ; +struct dma_region *region; + +/* A driver is required to generate an MSI. */ +TEST_REQUIRE (device->driver.ops); + +/* Set up a DMA-able region for the driver to use. */ + region = &device->driver.region; + region->iova =0 ;[Severity: Medium]
Does hardcoding the DMA region's IOVA to 0 cause IOMMU mapping failures on platforms that reserve or restrict IOVA 0?Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
If IOVA 0 is unsupported, it seems like
iommu_map()will fail and trigger a test assertion crash instead of running.This is probably fine for now given the test is only for
x86_64. But if you are going to send a v8 you can use the VFIO selftestsiova_allocatorto fix this, instead of hard-coding.I'm confused, why is this test setting up a DMA-able region in the first place?
There might be some devices with which you can trigger an MSI with pure MMIO. But most devices you need to at least set up some descriptors in memory that describe what action you want the device to take and then use MMIO to poke the device to read the desriptor and perform the action. Reading those descriptors is DMA.
Quoted 6 lines
Quoted 4 lines
For Intel DSA specifically,
send_msi()works by telling the device to perform an 8-byte memcpy and send an interrupt when it's done. The driver region contains the descriptor for that memcpy, as well as the 8-byte source/destination buffers the device should use.Ah, gotcha. For a comment, how's this look?
That looks great!
Quoted 19 lines
tools/testing/selftests/kvm/irq_test.c +6
-1@@ -67,7 +67,12 @@ static int vfio_setup_msi(struct vfio_pci_device *device)/* A driver is required to generate an MSI. */ TEST_REQUIRE (device->driver.ops); -/* Set up a DMA-able region for the driver to use. */ +/* +* Set up a DMA-able region for the driver to use. Very few devices +* provide a way to arbitrarily send interrupts (MSIs), e.g. by writing +* an MMIO register. Instead, most devices send MSIs when an action is +* completed, and practically all actions involve DMA of some form. +*/ region = &device->driver.region; region->iova =0 ; region->size =SZ_2M ;
Re: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test
In reply to Sean ChristophersonQuoted 22 lines · Sean Christopherson
From: David Matlack <[email protected]>
Extend the eventfd IRQ test with a '-d' argument that takes a BDF (in the format segment:bus:device.function) of an interrupt-capable PCI(e) device bound to VFIO, and use said device to trigger interrupts instead of always synthesizing interrupts via direct writes to the eventfd.
Using a VFIO device to trigger interrupts validates the end-to-end delivery of IRQs for "real" devices, and when supported by hardware (and KVM), also validates interrupt delivery via IRQ bypass, i.e. via device posted IRQs.
Now that IOMMUFD is a thing, auto-probe IOMMUFD vs. "legacy" VFIO by temporarily opening
/dev/iommufd, and skip the test if neither IOMMUFD nor legacy VFIO is available. Add a '-t' option to the user override the probe logic, e.g. in case IOMMUFD is available but the system is configured for legacy usage.Note, the device must have a VFIO selftest driver in order to work with the test. A helper script to list supported devices will hopefully be available in the near future at tools/testing/selftests/vfio/scripts/list_supported_devices.sh[1].
tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 68
Quoted 8 lines
tools/testing/selftests/kvm/irq_test.c +8
-0+static int vfio_setup_msi (struct vfio_pci_device *device) +{ +const int flags =MAP_SHARED |MAP_ANONYMOUS ; +const int prot =PROT_READ |PROT_WRITE ; +struct dma_region *region; + +/* A driver is required to generate an MSI. */ +TEST_REQUIRE (device->driver.ops);
This series is probably going to race with merging another VFIO selftests patch to make send_msi() optional [1]. Can you add a check for that here?
TEST_REQUIRE(device->driver.ops); TEST_REQUIRE(device->driver.ops->send_msi);
Sean Christopherson
maintainerRe: [PATCH v7 09/20] KVM: selftests: Add VFIO device support to eventfd IRQ test
Quoted 36 lines · David Matlack
Quoted 22 lines · Sean Christopherson
From: David Matlack <[email protected]>
Extend the eventfd IRQ test with a '-d' argument that takes a BDF (in the format segment:bus:device.function) of an interrupt-capable PCI(e) device bound to VFIO, and use said device to trigger interrupts instead of always synthesizing interrupts via direct writes to the eventfd.
Using a VFIO device to trigger interrupts validates the end-to-end delivery of IRQs for "real" devices, and when supported by hardware (and KVM), also validates interrupt delivery via IRQ bypass, i.e. via device posted IRQs.
Now that IOMMUFD is a thing, auto-probe IOMMUFD vs. "legacy" VFIO by temporarily opening
/dev/iommufd, and skip the test if neither IOMMUFD nor legacy VFIO is available. Add a '-t' option to the user override the probe logic, e.g. in case IOMMUFD is available but the system is configured for legacy usage.Note, the device must have a VFIO selftest driver in order to work with the test. A helper script to list supported devices will hopefully be available in the near future at tools/testing/selftests/vfio/scripts/list_supported_devices.sh[1].
tools/testing/selftests/kvm/irq_test.c static void *vcpu_thread_main(void *arg) · line 68
Quoted 8 lines
tools/testing/selftests/kvm/irq_test.c +8
-0+static int vfio_setup_msi (struct vfio_pci_device *device) +{ +const int flags =MAP_SHARED |MAP_ANONYMOUS ; +const int prot =PROT_READ |PROT_WRITE ; +struct dma_region *region; + +/* A driver is required to generate an MSI. */ +TEST_REQUIRE (device->driver.ops);This series is probably going to race with merging another VFIO selftests patch to make
send_msi()optional [1]. Can you add a check for that here?
Quoted 2 lines
TEST_REQUIRE(device->driver.ops); TEST_REQUIRE(device->driver.ops->send_msi);
Ya, will do.
Sean Christopherson
maintainerPATCH v7 10/20 KVM: selftests: Add a helper to set proc IRQ affinity for IRQ test
In reply to Sean ChristophersonFrom: Josh Hilke <[email protected]>
Add a utility, proc_irq_set_smp_affinity(), to set the CPU affinity of a Linux host IRQ via the proc filesystem. Use smp_affinity_list instead of smp_affinity to avoid having to convert the single CPU to a bitmask.
The helper will be used by the eventfd IRQ test to verify delivery of IRQs when the affinity is randomized/modified.
--- tools/testing/selftests/kvm/include/proc_util.h | 2 ++ tools/testing/selftests/kvm/lib/proc_util.c | 14 ++++++++++++++ 2 files changed, 16 insertions(+)
tools/testing/selftests/kvm/include/proc_util.h +2 -0
@@ -6,4 +6,6 @@unsigned int vfio_msix_to_host_irq (const char *vfio_device_bdf,int msix); +void proc_irq_set_smp_affinity (unsigned int irq,int cpu); +#endif /* SELFTEST_KVM_PROC_UTIL_H */
tools/testing/selftests/kvm/lib/proc_util.c +14 -0
@@ -38,3 +38,17 @@ unsigned int vfio_msix_to_host_irq(const char *device_bdf, int msix)return (unsigned int )irq; } +void proc_irq_set_smp_affinity (unsigned int irq,int cpu) +{ +char path[PATH_MAX ]; +int r, fd; + +snprintf (path,sizeof (path),"/proc/irq/%u/smp_affinity_list" , irq); + fd =open (path,O_RDWR ); +TEST_ASSERT (fd >=0 ,"Failed to open %s" , path); + + r =dprintf (fd,"%d\n" , cpu); +TEST_ASSERT (r >0 ,"Failed to affinitize IRQ-%u to CPU %d" , irq, cpu); + +kvm_close (fd); +}
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 11/20 KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test
In reply to Sean ChristophersonFrom: David Matlack <[email protected]>
Extent the eventfd IRQ test with a '-a' flag to randomly affinitize the device's host IRQ to different physical CPUs throughout the test. This stresses the kernel's ability to maintain correct interrupt routing and delivery even as the underlying hardware IRQ affinity is changed dynamically via /proc/<irq>/smp_affinity{,_list}.
--- tools/testing/selftests/kvm/irq_test.c | 27 +++++++++++++++++++++----- 1 file changed, 22 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/irq_test.c +22 -5
@@ -12,10 +12,12 @@#include <unistd.h> #include <pthread.h> #include <sys/eventfd.h> +#include <sys/sysinfo.h> static u64 timeout_ns =2ULL *1000 *1000 *1000 ;static bool guest_ready_for_irqs[KVM_MAX_VCPUS ];static bool guest_received_irq[KVM_MAX_VCPUS ]; +static bool irq_affinity;static bool done;#define GUEST_RECEIVED_IRQ (__vcpu) \
@@ -125,9 +127,10 @@ static const char *probe_iommu_type(void)static void help (const char *name) { -printf ("Usage: %s [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n" , name); +printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n" , name);printf ("\n" );printf ("Tests KVM interrupt routing and delivery via irqfd.\n" ); +printf ("-a Affine the device's host IRQ to a random physical CPU\n" );printf ("-d Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n" );printf ("-t Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n" );printf ("\n" );
@@ -160,10 +163,13 @@ int main(int argc, char **argv)int i, j, c, msix, eventfd;struct iommu *iommu;struct kvm_vm *vm; -int irq; +int irq, irq_cpu; -while ((c =getopt (argc, argv,"d:ht:" )) !=-1 ) { +while ((c =getopt (argc, argv,"ad:ht:" )) !=-1 ) {switch (c) { +case 'a' : + irq_affinity = true; +break ;case 'd' : device_bdf = optarg;break ;
@@ -192,7 +198,11 @@ int main(int argc, char **argv)printf ("Using device %s MSI-X[%d] (IRQ-%u)\n" , device_bdf, msix, irq); }else { +TEST_ASSERT (!irq_affinity, +"Setting IRQ affinity (-a) requires a backing device (-d)" ); + eventfd =kvm_new_eventfd (); + irq =-1 ; } pr_info("Injecting interrupts for GSI %d (guest vector 0x%x) %d times\n",
@@ -210,12 +220,19 @@ int main(int argc, char **argv)continue ; } + irq_cpu =-1 ; +for (i =0 ; i < nr_irqs; i++) {struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];struct timespec start;kvm_route_msi (vm, gsi, vcpu, vector); +if (irq_affinity) { + irq_cpu =kvm_random_u64 (&kvm_rng) %get_nprocs (); +proc_irq_set_smp_affinity (irq, irq_cpu); + } +for (j =0 ; j < nr_vcpus; j++) TEST_ASSERT(!GUEST_RECEIVED_IRQ(vcpus[j]), "IRQ flag for vCPU %d not clear prior to test",
@@ -229,8 +246,8 @@ int main(int argc, char **argv)cpu_relax ();TEST_ASSERT (GUEST_RECEIVED_IRQ (vcpu), -"vCPU %d timed out waiting for IRQ (vector 0x%x) from GSI %d\n" , - vcpu->id, vector, gsi); +"vCPU %d timed out waiting for IRQ (vector 0x%x) from GSI %d (via CPU %d)\n" , + vcpu->id, vector, gsi, irq_cpu);WRITE_AND_SYNC_TO_GUEST (vm, guest_received_irq[vcpu->id], false); }
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
[email protected]
AI reviewRe: [PATCH v7 11/20] KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The test calculates the target host CPU for IRQ affinity using a modulo of `get_nprocs()`…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The test calculates the target host CPU for IRQ affinity using a modulo of get_nprocs(), incorrectly assuming that CPU IDs are contiguous and 0-indexed without gaps. This causes spurious test failures on sparse topologies or systems with offline CPUs. --
commit f0fe406c9bb6f2cc39673ee344b3e53be280d025
Author: David Matlack <[email protected]>
KVM: selftests: Verify interrupts are received when IRQ affinity changes in IRQ test
This commit extends the eventfd IRQ test with a '-a' flag to randomly change the device's host IRQ affinity to different physical CPUs during the test. This ensures the kernel maintains correct interrupt routing when hardware IRQ affinity changes dynamically.
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0
-0
[ ... ]
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +0
-0@@ -210,12 +220,19 @@ int main(int argc, char **argv)continue ; }
tools/testing/selftests/kvm/irq_test.c int main(int argc, char **argv) · line 232
Quoted 10 lines
tools/testing/selftests/kvm/irq_test.c +4
-0+ irq_cpu =-1 ; +for (i =0 ; i < nr_irqs; i++) {struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];struct timespec start;kvm_route_msi (vm, gsi, vcpu, vector); +if (irq_affinity) { + irq_cpu =kvm_random_u64 (&kvm_rng) %get_nprocs ();
[Severity: Medium]
Does this code assume that CPU IDs are contiguous and 0-indexed?
If get_nprocs() returns the number of currently available processors, could this modulo calculation yield an invalid or offline CPU ID on systems with sparse CPU topologies or offline CPUs?
This regression might cause the subsequent call to proc_irq_set_smp_affinity() to fail with -EINVAL and spuriously abort the test.
Quoted 4 lines
tools/testing/selftests/kvm/irq_test.c +3
-0+proc_irq_set_smp_affinity (irq, irq_cpu); + } +for (j =0 ; j < nr_vcpus; j++)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Sean Christopherson
maintainerPATCH v7 12/20 KVM: selftests: Add option to set empty routing between IRQs in eventfd IRQ test
In reply to Sean ChristophersonFrom: David Matlack <[email protected]>
Extend the eventfd IRQ test with an '-e' flag to set empty GSI routing between interrupts. Clobbering the GSI routing table verifies that KVM correctly handles CPUx => NULL => CPUy transitions, not just CPUx => CPUy transitions, and verifies that KVM can "rebuild" an entire routing setup.
--- tools/testing/selftests/kvm/irq_test.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +18 -2
@@ -109,6 +109,13 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu,vm_ioctl (vm,KVM_SET_GSI_ROUTING , &routing.header); } +static void kvm_set_empty_gsi_routing (struct kvm_vm *vm) +{ +struct kvm_irq_routing routing = {}; + +vm_ioctl (vm,KVM_SET_GSI_ROUTING , &routing); +} +static const char *probe_iommu_type (void ) {int io_fd;
@@ -127,11 +134,12 @@ static const char *probe_iommu_type(void)static void help (const char *name) { -printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-h] [-t iommu_type]\n" , name); +printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-t iommu_type]\n" , name);printf ("\n" );printf ("Tests KVM interrupt routing and delivery via irqfd.\n" );printf ("-a Affine the device's host IRQ to a random physical CPU\n" );printf ("-d Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n" ); +printf ("-e Set empty GSI routing in-between some interrupts\n" );printf ("-t Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n" );printf ("\n" );exit (KSFT_FAIL );
@@ -158,6 +166,7 @@ int main(int argc, char **argv)struct kvm_vcpu *vcpus[KVM_MAX_VCPUS ];struct vfio_pci_device *device =NULL ;int nr_irqs =1000 , nr_vcpus =1 ; +bool set_empty_routing = false;const char *device_bdf =NULL ;const char *iommu_type =NULL ;int i, j, c, msix, eventfd;
@@ -165,7 +174,7 @@ int main(int argc, char **argv)struct kvm_vm *vm;int irq, irq_cpu; -while ((c =getopt (argc, argv,"ad:ht:" )) !=-1 ) { +while ((c =getopt (argc, argv,"ad:eht:" )) !=-1 ) {switch (c) {case 'a' : irq_affinity = true;
@@ -173,6 +182,9 @@ int main(int argc, char **argv)case 'd' : device_bdf = optarg;break ; +case 'e' : + set_empty_routing = true; +break ;case 't' : iommu_type = optarg;break ;
@@ -223,9 +235,13 @@ int main(int argc, char **argv) irq_cpu =-1 ;for (i =0 ; i < nr_irqs; i++) { +const bool do_set_empty_routing = set_empty_routing && (i &BIT (3 ));struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];struct timespec start; +if (do_set_empty_routing) +kvm_set_empty_gsi_routing (vm); +kvm_route_msi (vm, gsi, vcpu, vector);if (irq_affinity) {
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 13/20 KVM: selftests: Make number of IRQs configurable in IRQ test
In reply to Sean ChristophersonFrom: David Matlack <[email protected]>
Extend the eventfd IRQ test with a '-i' flag to let the user specify the the number of IRQs to generate (instead of hardcoding the test to always generate 1000 interrupts).
--- tools/testing/selftests/kvm/irq_test.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +6 -2
@@ -134,12 +134,13 @@ static const char *probe_iommu_type(void)static void help (const char *name) { -printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-t iommu_type]\n" , name); +printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-t iommu_type]\n" , name);printf ("\n" );printf ("Tests KVM interrupt routing and delivery via irqfd.\n" );printf ("-a Affine the device's host IRQ to a random physical CPU\n" );printf ("-d Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n" );printf ("-e Set empty GSI routing in-between some interrupts\n" ); +printf ("-i The number of IRQs to generate during the test\n" );printf ("-t Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n" );printf ("\n" );exit (KSFT_FAIL );
@@ -174,7 +175,7 @@ int main(int argc, char **argv)struct kvm_vm *vm;int irq, irq_cpu; -while ((c =getopt (argc, argv,"ad:eht:" )) !=-1 ) { +while ((c =getopt (argc, argv,"ad:ehi:t:" )) !=-1 ) {switch (c) {case 'a' : irq_affinity = true;
@@ -185,6 +186,9 @@ int main(int argc, char **argv)case 'e' : set_empty_routing = true;break ; +case 'i' : + nr_irqs =atoi_positive ("Number of IRQs" , optarg); +break ;case 't' : iommu_type = optarg;break ;
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 14/20 KVM: selftests: Verify non-postable IRQ remapping in IRQ test
In reply to Sean ChristophersonExtend the eventfd IRQ test with an '-n' flag to route a subset of device interrupts as NMIs (Non-Maskable Interrupts) into the guest using an alternating pattern of 4 NMIs followed by 4 regular interrupts.
While this adds coverage for NMI injection, the primary goal is to validate KVM's handling of non-postable interrupt delivery (AMD and Intel IOMMUs only support posting fixed IRQs targeting a single vCPU). KVM has historically bungled handling transitions between posted and remapped modes. Use NMIs to stress the transitions, because they are a reliable, architectural way to force these code paths.
--- tools/testing/selftests/kvm/irq_test.c | 48 ++++++++++++++++++++------ 1 file changed, 37 insertions(+), 11 deletions(-)
tools/testing/selftests/kvm/irq_test.c +37 -11
@@ -17,11 +17,17 @@static u64 timeout_ns =2ULL *1000 *1000 *1000 ;static bool guest_ready_for_irqs[KVM_MAX_VCPUS ];static bool guest_received_irq[KVM_MAX_VCPUS ]; +static bool guest_received_nmi[KVM_MAX_VCPUS ];static bool irq_affinity;static bool done;#define GUEST_RECEIVED_IRQ (__vcpu) \ SYNC_FROM_GUEST_AND_READ((__vcpu)->vm, guest_received_irq[(__vcpu)->id]) +#define GUEST_RECEIVED_NMI (__vcpu) \ + SYNC_FROM_GUEST_AND_READ((__vcpu)->vm, guest_received_nmi[(__vcpu)->id]) + +#define GUEST_RECEIVED_INTERRUPT (__vcpu, __nmi) \ + ((__nmi) ? GUEST_RECEIVED_NMI(__vcpu) : GUEST_RECEIVED_IRQ(__vcpu))static u32 guest_get_vcpu_id (void ) {
@@ -35,6 +41,11 @@ static void guest_irq_handler(struct ex_regs *regs)x2apic_write_reg (APIC_EOI ,0 ); } +static void guest_nmi_handler (struct ex_regs *regs) +{ +WRITE_ONCE (guest_received_nmi[guest_get_vcpu_id ()], true); +} +static void guest_code (void ) {x2apic_enable ();
@@ -91,7 +102,7 @@ static void trigger_interrupt(struct vfio_pci_device *device, int eventfd)static void kvm_route_msi (struct kvm_vm *vm,u32 gsi,struct kvm_vcpu *vcpu, -u8 vector) +u8 vector,bool use_nmi) {struct {struct kvm_irq_routing header;
@@ -102,7 +113,7 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu, .gsi = gsi, .type = KVM_IRQ_ROUTING_MSI, .u.msi.address_lo = 0xFEE00000 | (vcpu->id << 12), - .u.msi.data = vector, + .u.msi.data = use_nmi ? NMI_VECTOR | (4 << 8) : vector, }, };
@@ -134,13 +145,14 @@ static const char *probe_iommu_type(void)static void help (const char *name) { -printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-t iommu_type]\n" , name); +printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-n] [-t iommu_type]\n" , name);printf ("\n" );printf ("Tests KVM interrupt routing and delivery via irqfd.\n" );printf ("-a Affine the device's host IRQ to a random physical CPU\n" );printf ("-d Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n" );printf ("-e Set empty GSI routing in-between some interrupts\n" );printf ("-i The number of IRQs to generate during the test\n" ); +printf ("-n Deliver 50 percent of IRQs as non-maskable interrupts\n" );printf ("-t Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n" );printf ("\n" );exit (KSFT_FAIL );
@@ -171,11 +183,12 @@ int main(int argc, char **argv)const char *device_bdf =NULL ;const char *iommu_type =NULL ;int i, j, c, msix, eventfd; +bool use_nmi = false;struct iommu *iommu;struct kvm_vm *vm;int irq, irq_cpu; -while ((c =getopt (argc, argv,"ad:ehi:t:" )) !=-1 ) { +while ((c =getopt (argc, argv,"ad:ehi:nt:" )) !=-1 ) {switch (c) {case 'a' : irq_affinity = true;
@@ -189,6 +202,9 @@ int main(int argc, char **argv)case 'i' : nr_irqs =atoi_positive ("Number of IRQs" , optarg);break ; +case 'n' : + use_nmi = true; +break ;case 't' : iommu_type = optarg;break ;
@@ -202,6 +218,7 @@ int main(int argc, char **argv) vm =vm_create_with_vcpus (nr_vcpus, guest_code, vcpus);vm_install_exception_handler (vm, vector, guest_irq_handler); +vm_install_exception_handler (vm,NMI_VECTOR , guest_nmi_handler);if (device_bdf) {if (!iommu_type)
@@ -240,36 +257,45 @@ int main(int argc, char **argv)for (i =0 ; i < nr_irqs; i++) {const bool do_set_empty_routing = set_empty_routing && (i &BIT (3 )); +const bool do_use_nmi = use_nmi && (i &BIT (2 ));struct kvm_vcpu *vcpu = vcpus[i % nr_vcpus];struct timespec start;if (do_set_empty_routing)kvm_set_empty_gsi_routing (vm); -kvm_route_msi (vm, gsi, vcpu, vector); +kvm_route_msi (vm, gsi, vcpu, vector, do_use_nmi);if (irq_affinity) { irq_cpu =kvm_random_u64 (&kvm_rng) %get_nprocs ();proc_irq_set_smp_affinity (irq, irq_cpu); } -for (j =0 ; j < nr_vcpus; j++) +for (j =0 ; j < nr_vcpus; j++) {TEST_ASSERT (!GUEST_RECEIVED_IRQ (vcpus[j]),"IRQ flag for vCPU %d not clear prior to test" , vcpus[j]->id); +TEST_ASSERT (!GUEST_RECEIVED_NMI (vcpus[j]), +"NMI flag for vCPU %d not clear prior to test" , + vcpus[j]->id); + }trigger_interrupt (device, eventfd);clock_gettime (CLOCK_MONOTONIC , &start); -while (!GUEST_RECEIVED_IRQ (vcpu) && +while (!GUEST_RECEIVED_INTERRUPT (vcpu, do_use_nmi) &×pec_to_ns (timespec_elapsed (start)) <= timeout_ns)cpu_relax (); -TEST_ASSERT (GUEST_RECEIVED_IRQ (vcpu), -"vCPU %d timed out waiting for IRQ (vector 0x%x) from GSI %d (via CPU %d)\n" , - vcpu->id, vector, gsi, irq_cpu); +TEST_ASSERT (GUEST_RECEIVED_INTERRUPT (vcpu, do_use_nmi), +"vCPU %d timed out waiting for %s (vector 0x%x) from GSI %d (via CPU %d)\n" , + vcpu->id, do_use_nmi ?"NMI" :"IRQ" , + do_use_nmi ?NMI_VECTOR : vector, gsi, irq_cpu); -WRITE_AND_SYNC_TO_GUEST (vm, guest_received_irq[vcpu->id], false); +if (do_use_nmi) +WRITE_AND_SYNC_TO_GUEST (vm, guest_received_nmi[vcpu->id], false); +else +WRITE_AND_SYNC_TO_GUEST (vm, guest_received_irq[vcpu->id], false); }WRITE_AND_SYNC_TO_GUEST (vm, done, true);
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 15/20 KVM: selftests: Add kvm_gettid() wrapper and convert users
In reply to Sean ChristophersonFrom: Josh Hilke <[email protected]>
Add a KVM wrapper for the gettid() syscall so that tests don't have to open code the syscall() themselves. Unfortunately, not all flavors of libc that KVM selftests support provide gettid(). Convert all existing users of the syscall to the new wrapper.
Note, per the gettid() manpage[1], "This call is always successful", i.e. prefixing kvm_ to the syscall name is aligned with the goal of providing syscall wrappers that guarantee success.
No functional changes intended.
--- tools/testing/selftests/kvm/demand_paging_test.c | 2 +- tools/testing/selftests/kvm/include/kvm_syscalls.h | 5 +++++ tools/testing/selftests/kvm/lib/assert.c | 8 ++------ tools/testing/selftests/kvm/lib/test_util.c | 3 ++- tools/testing/selftests/kvm/rseq_test.c | 2 +- 5 files changed, 11 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/demand_paging_test.c +1 -1
@@ -57,7 +57,7 @@ static void vcpu_worker(struct memstress_vcpu_args *vcpu_args)static int handle_uffd_page_request (int uffd_mode,int uffd,struct uffd_msg *msg) { -pid_t tid =syscall (__NR_gettid); +pid_t tid =kvm_gettid ();u64 addr = msg->arg.pagefault.address;struct timespec start;struct timespec ts_diff;
tools/testing/selftests/kvm/include/kvm_syscalls.h +5 -0
@@ -83,6 +83,11 @@ static inline int kvm_dup(int fd)return new_fd; } +static inline pid_t kvm_gettid (void ) +{ +return syscall (__NR_gettid); +} + __KVM_SYSCALL_DEFINE(munmap, 2, void *, mem, size_t, size);__KVM_SYSCALL_DEFINE (close,1 , int, fd);__KVM_SYSCALL_DEFINE (fallocate,4 , int, fd, int, mode, loff_t, offset, loff_t, len);
tools/testing/selftests/kvm/lib/assert.c +2 -6
@@ -10,6 +10,7 @@#include <sys/syscall.h> #include "kselftest.h" +#include "kvm_syscalls.h" #ifdef __GLIBC__#include <execinfo.h>
@@ -64,11 +65,6 @@ static void test_dump_stack(void)static void test_dump_stack (void ) {}#endif -static pid_t _gettid (void ) -{ -return syscall (SYS_gettid); -} -void __attribute__((noinline))test_assert (bool exp,const char *exp_str,const char *file,unsigned int line,const char *fmt, ...)
@@ -81,7 +77,7 @@ test_assert(bool exp, const char *exp_str,fprintf (stderr,"==== Test Assertion Failure ====\n" " %s:%u: %s\n" " pid=%d tid=%d errno=%d - %s\n" , - file, line, exp_str,getpid (),_gettid (), + file, line, exp_str,getpid (),kvm_gettid (), errno,strerror (errno));test_dump_stack ();if (fmt) {
tools/testing/selftests/kvm/lib/test_util.c +2 -1
@@ -17,6 +17,7 @@#include "linux/kernel.h" #include "test_util.h" +#include "kvm_syscalls.h" sigjmp_buf expect_sigbus_jmpbuf;
@@ -395,7 +396,7 @@ long get_run_delay(void)long val[2 ];FILE *fp; -sprintf (path,"/proc/%ld/schedstat" ,syscall (SYS_gettid)); +sprintf (path,"/proc/%ld/schedstat" , (long )kvm_gettid ()); fp =fopen (path,"r" );/* Return MIN_RUN_DELAY_NS upon failure just to be safe */ if (fscanf (fp,"%ld %ld " , &val[0 ], &val[1 ]) <2 )
tools/testing/selftests/kvm/rseq_test.c +1 -1
@@ -244,7 +244,7 @@ int main(int argc, char *argv[]) vm =vm_create_with_one_vcpu (&vcpu, guest_code);pthread_create (&migration_thread,NULL , migration_worker, - (void *)(unsigned long )syscall (SYS_gettid)); + (void *)(unsigned long )kvm_gettid ());if (latency >=0 ) {/*
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 16/20 KVM: selftests: Add kvm_sched_getaffinity() wrapper and convert users
In reply to Sean ChristophersonFrom: Josh Hilke <[email protected]>
Add and use a KVM wrapper for the sched_getaffinity() syscall so that selftests don't need to manually assert that the syscall succeeded.
Note, some tests didn't actually assert success, but they all obviously rely on the syscall to succeed.
--- tools/testing/selftests/kvm/arch_timer.c | 2 +- tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c | 2 +- tools/testing/selftests/kvm/include/kvm_syscalls.h | 2 ++ tools/testing/selftests/kvm/lib/kvm_util.c | 5 ++--- tools/testing/selftests/kvm/mmu_stress_test.c | 6 +----- tools/testing/selftests/kvm/rseq_test.c | 4 +--- 6 files changed, 8 insertions(+), 13 deletions(-)
tools/testing/selftests/kvm/arch_timer.c +1 -1
@@ -85,7 +85,7 @@ static u32 test_get_pcpu(void)cpu_set_t online_cpuset; nproc_conf =get_nprocs_conf (); -sched_getaffinity (0 ,sizeof (cpu_set_t), &online_cpuset); +kvm_sched_getaffinity (0 ,sizeof (cpu_set_t), &online_cpuset);/* Randomly find an available pCPU to place a vCPU on */ do {
tools/testing/selftests/kvm/arm64/arch_timer_edge_cases.c +1 -1
@@ -1039,7 +1039,7 @@ int main(int argc, char *argv[])if (!parse_args (argc, argv))exit (KSFT_SKIP ); -sched_getaffinity (0 ,sizeof (default_cpuset), &default_cpuset); +kvm_sched_getaffinity (0 ,sizeof (default_cpuset), &default_cpuset);set_counter_defaults ();if (test_args.test_virtual) {
tools/testing/selftests/kvm/include/kvm_syscalls.h +2 -0
@@ -12,6 +12,7 @@#include <sys/mman.h> #include <sys/syscall.h> +#include <sched.h> #include <test_util.h> #define MAP_ARGS0 (m,...)
@@ -93,6 +94,7 @@ __KVM_SYSCALL_DEFINE(close, 1, int, fd);__KVM_SYSCALL_DEFINE (fallocate,4 , int, fd, int, mode, loff_t, offset, loff_t, len); __KVM_SYSCALL_DEFINE(ftruncate, 2, unsigned int, fd, off_t, length); __KVM_SYSCALL_DEFINE(madvise, 3, void *, addr, size_t, length, int, advice); +__KVM_SYSCALL_DEFINE(sched_getaffinity, 3, pid_t, pid, size_t, cpusetsize, cpu_set_t *, mask);#define kvm_free_fd (fd) \ do { \
tools/testing/selftests/kvm/lib/kvm_util.c +2 -3
@@ -673,13 +673,12 @@ void kvm_parse_vcpu_pinning(const char *pcpus_string, u32 vcpu_to_pcpu[],cpu_set_t allowed_mask;char *cpu, *cpu_list;char delim[2 ] ="," ; -int i, r; +int i; cpu_list =strdup (pcpus_string);TEST_ASSERT (cpu_list,"strdup() allocation failed." ); - r =sched_getaffinity (0 ,sizeof (allowed_mask), &allowed_mask); -TEST_ASSERT (!r,"sched_getaffinity() failed" ); +kvm_sched_getaffinity (0 ,sizeof (allowed_mask), &allowed_mask); cpu =strtok (cpu_list, delim);
tools/testing/selftests/kvm/mmu_stress_test.c +1 -5
@@ -255,11 +255,7 @@ static void rendezvous_with_vcpus(struct timespec *time, const char *name)static void calc_default_nr_vcpus (void ) {cpu_set_t possible_mask; -int r; - - r =sched_getaffinity (0 ,sizeof (possible_mask), &possible_mask); -TEST_ASSERT (!r,"sched_getaffinity failed, errno = %d (%s)" , - errno,strerror (errno)); +kvm_sched_getaffinity (0 ,sizeof (possible_mask), &possible_mask); nr_vcpus =CPU_COUNT (&possible_mask);TEST_ASSERT (nr_vcpus >0 ,"Uh, no CPUs?" );
tools/testing/selftests/kvm/rseq_test.c +1 -3
@@ -226,9 +226,7 @@ int main(int argc, char *argv[]) } } - r =sched_getaffinity (0 ,sizeof (possible_mask), &possible_mask); -TEST_ASSERT (!r,"sched_getaffinity failed, errno = %d (%s)" , errno, -strerror (errno)); +kvm_sched_getaffinity (0 ,sizeof (possible_mask), &possible_mask);calc_min_max_cpu ();
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 17/20 KVM: selftests: Add a utility to pin a task to a random CPU, given a CPU set
In reply to Sean ChristophersonFrom: Josh Hilke <[email protected]>
Add a helper function, pin_task_to_random_cpu(), to pin a task to a random CPU from a given cpu_set_t.
This helper will be used eventfd IRQ test to migrate vCPUs to random pCPUs, to stress host-side interrupt routing and delivery.
--- .../testing/selftests/kvm/include/kvm_util.h | 2 ++ tools/testing/selftests/kvm/lib/kvm_util.c | 21 +++++++++++++++++++ 2 files changed, 23 insertions(+)
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
@@ -1094,6 +1094,8 @@ static inline void pin_task_to_cpu(pthread_t task, int cpu)TEST_ASSERT (!r,"Failed to set thread affinity to pCPU '%u'" , cpu); } +void pin_task_to_random_cpu (pthread_t task,cpu_set_t *possible_cpus); +static inline int pin_task_to_any_cpu (pthread_t task) {int cpu =sched_getcpu ();
tools/testing/selftests/kvm/lib/kvm_util.c +21 -0
@@ -667,6 +667,27 @@ void kvm_print_vcpu_pinning_help(void) " (default: no pinning)\n", name, name); } +void pin_task_to_random_cpu (pthread_t task,cpu_set_t *possible_cpus) +{ +int target_idx; +int nr_cpus; +int cpu; + + nr_cpus =CPU_COUNT (possible_cpus); +TEST_ASSERT (nr_cpus >0 ,"No CPUs available in possible_cpus" ); + + target_idx =kvm_random_u64 (&kvm_rng) % nr_cpus; + +for (cpu =0 ; cpu <CPU_SETSIZE ; cpu++) { +if (CPU_ISSET (cpu, possible_cpus) && target_idx-- ==0 ) { +pin_task_to_cpu (task, cpu); +return ; + } + } + +TEST_FAIL ("Failed to find random CPU in possible_cpus" ); +} +void kvm_parse_vcpu_pinning (const char *pcpus_string,u32 vcpu_to_pcpu[],int nr_vcpus) {
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 18/20 KVM: selftests: Verify vCPU migration during IRQ delivery in IRQ test
In reply to Sean ChristophersonFrom: David Matlack <[email protected]>
Extend the eventfd IRQ test with a '-m' flag to have the test migrate the target vCPU to a random physical CPU before triggering its interrupt, e.g. to validate KVM's ability to update device posted IRQ routing.
--- tools/testing/selftests/kvm/irq_test.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/irq_test.c +14 -2
@@ -145,13 +145,14 @@ static const char *probe_iommu_type(void)static void help (const char *name) { -printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-n] [-t iommu_type]\n" , name); +printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type]\n" , name);printf ("\n" );printf ("Tests KVM interrupt routing and delivery via irqfd.\n" );printf ("-a Affine the device's host IRQ to a random physical CPU\n" );printf ("-d Use a VFIO device to send MSI-X interrupts instead of manually signaling the eventfd\n" );printf ("-e Set empty GSI routing in-between some interrupts\n" );printf ("-i The number of IRQs to generate during the test\n" ); +printf ("-m Pin target vCPU to random physical CPU before triggering interrupt\n" );printf ("-n Deliver 50 percent of IRQs as non-maskable interrupts\n" );printf ("-t Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n" );printf ("\n" );
@@ -183,12 +184,14 @@ int main(int argc, char **argv)const char *device_bdf =NULL ;const char *iommu_type =NULL ;int i, j, c, msix, eventfd; +bool migrate_vcpus = false; +cpu_set_t available_cpus;bool use_nmi = false;struct iommu *iommu;struct kvm_vm *vm;int irq, irq_cpu; -while ((c =getopt (argc, argv,"ad:ehi:nt:" )) !=-1 ) { +while ((c =getopt (argc, argv,"ad:ehi:mnt:" )) !=-1 ) {switch (c) {case 'a' : irq_affinity = true;
@@ -202,6 +205,9 @@ int main(int argc, char **argv)case 'i' : nr_irqs =atoi_positive ("Number of IRQs" , optarg);break ; +case 'm' : + migrate_vcpus = true; +break ;case 'n' : use_nmi = true;break ;
@@ -243,6 +249,9 @@ int main(int argc, char **argv)kvm_assign_irqfd (vm, gsi, eventfd); +if (migrate_vcpus) +kvm_sched_getaffinity (0 ,sizeof (available_cpus), &available_cpus); +for (i =0 ; i < nr_vcpus; i++)pthread_create (&vcpu_threads[i],NULL , vcpu_thread_main, vcpus[i]);
@@ -271,6 +280,9 @@ int main(int argc, char **argv)proc_irq_set_smp_affinity (irq, irq_cpu); } +if (migrate_vcpus) +pin_task_to_random_cpu (vcpu_threads[i % nr_vcpus], &available_cpus); +for (j =0 ; j < nr_vcpus; j++) { TEST_ASSERT(!GUEST_RECEIVED_IRQ(vcpus[j]), "IRQ flag for vCPU %d not clear prior to test",
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 19/20 KVM: selftests: Make number of vCPUs configurable in IRQ test
In reply to Sean ChristophersonFrom: David Matlack <[email protected]>
Extend the eventfd IRQ test with a '-v' flag to allow the user to configure the number of vCPUs to create and run (versus only ever using a single vCPU).
Update the routing logic to play nice with 32 bit IDs, enable x2APIC format in KVM (to enable 32-bit ID routing), and disable KVM's x2APIC broadcast quirk so that targeting vCPU 255 doesn't blast the interrupt to all vCPUs when in x2APIC mode.
--- tools/testing/selftests/kvm/irq_test.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/irq_test.c +13 -3
@@ -112,7 +112,8 @@ static void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu, .entry = { .gsi = gsi, .type = KVM_IRQ_ROUTING_MSI, - .u.msi.address_lo = 0xFEE00000 | (vcpu->id << 12), + .u.msi.address_lo = 0xFEE00000 | (vcpu->id & GENMASK(7, 0)) << 12, + .u.msi.address_hi = vcpu->id & GENMASK(31, 8), .u.msi.data = use_nmi ? NMI_VECTOR | (4 << 8) : vector, }, };
@@ -145,7 +146,7 @@ static const char *probe_iommu_type(void)static void help (const char *name) { -printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type]\n" , name); +printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type] [-v nr_vcpus]\n" , name);printf ("\n" );printf ("Tests KVM interrupt routing and delivery via irqfd.\n" );printf ("-a Affine the device's host IRQ to a random physical CPU\n" );
@@ -155,6 +156,7 @@ static void help(const char *name)printf ("-m Pin target vCPU to random physical CPU before triggering interrupt\n" );printf ("-n Deliver 50 percent of IRQs as non-maskable interrupts\n" );printf ("-t Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n" ); +printf ("-v Number of vCPUS to run\n" );printf ("\n" );exit (KSFT_FAIL ); }
@@ -191,7 +193,7 @@ int main(int argc, char **argv)struct kvm_vm *vm;int irq, irq_cpu; -while ((c =getopt (argc, argv,"ad:ehi:mnt:" )) !=-1 ) { +while ((c =getopt (argc, argv,"ad:ehi:mnt:v:" )) !=-1 ) {switch (c) {case 'a' : irq_affinity = true;
@@ -214,6 +216,11 @@ int main(int argc, char **argv)case 't' : iommu_type = optarg;break ; +case 'v' : + nr_vcpus =atoi_positive ("Number of vCPUS" , optarg); +TEST_ASSERT (nr_vcpus <=KVM_MAX_VCPUS , +"KVM selftests support at most %u vCPUs" ,KVM_MAX_VCPUS ); +break ;case 'h' :default :help (argv[0 ]);
@@ -223,6 +230,9 @@ int main(int argc, char **argv)TEST_REQUIRE (kvm_arch_has_default_irqchip ()); vm =vm_create_with_vcpus (nr_vcpus, guest_code, vcpus); +vm_enable_cap (vm,KVM_CAP_X2APIC_API ,KVM_X2APIC_API_USE_32BIT_IDS | +KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK ); +vm_install_exception_handler (vm, vector, guest_irq_handler);vm_install_exception_handler (vm,NMI_VECTOR , guest_nmi_handler);
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog
Sean Christopherson
maintainerPATCH v7 20/20 KVM: selftests: Add xAPIC support in eventfd IRQ test
In reply to Sean ChristophersonFrom: David Matlack <[email protected]>
Extend the eventfd IRQ test with a '-x' flag to let the user run the test in xAPIC mode instead of the default x2APIC mode. When using xAPIC mode, sanity check user input to ensure the test is being run with at most 255 vCPUs, as xAPIC can only address IDs 0-254 (255, i.e. 0xff, broadcasts to all CPUs).
--- tools/testing/selftests/kvm/irq_test.c | 31 +++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/irq_test.c +26 -5
@@ -18,6 +18,7 @@ static u64 timeout_ns = 2ULL * 1000 * 1000 * 1000;static bool guest_ready_for_irqs[KVM_MAX_VCPUS ];static bool guest_received_irq[KVM_MAX_VCPUS ];static bool guest_received_nmi[KVM_MAX_VCPUS ]; +static bool x2apic = true;static bool irq_affinity;static bool done;
@@ -31,14 +32,20 @@ static bool done;static u32 guest_get_vcpu_id (void ) { -return x2apic_read_reg (APIC_ID ); +if (x2apic) +return x2apic_read_reg (APIC_ID ); +else +return xapic_read_reg (APIC_ID ) >>24 ; }static void guest_irq_handler (struct ex_regs *regs) {WRITE_ONCE (guest_received_irq[guest_get_vcpu_id ()], true); -x2apic_write_reg (APIC_EOI ,0 ); +if (x2apic) +x2apic_write_reg (APIC_EOI ,0 ); +else +xapic_write_reg (APIC_EOI ,0 ); }static void guest_nmi_handler (struct ex_regs *regs)
@@ -48,7 +55,10 @@ static void guest_nmi_handler(struct ex_regs *regs)static void guest_code (void ) { -x2apic_enable (); +if (x2apic) +x2apic_enable (); +else +xapic_enable ();sti_nop ();
@@ -146,7 +156,7 @@ static const char *probe_iommu_type(void)static void help (const char *name) { -printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type] [-v nr_vcpus]\n" , name); +printf ("Usage: %s [-a] [-d <segment:bus:device.function>] [-e] [-h] [-i nr_irqs] [-m] [-n] [-t iommu_type] [-v nr_vcpus] [-x]\n" , name);printf ("\n" );printf ("Tests KVM interrupt routing and delivery via irqfd.\n" );printf ("-a Affine the device's host IRQ to a random physical CPU\n" );
@@ -157,6 +167,7 @@ static void help(const char *name)printf ("-n Deliver 50 percent of IRQs as non-maskable interrupts\n" );printf ("-t Override the IOMMU type to use (vfio_type1_iommu or iommufd)\n" );printf ("-v Number of vCPUS to run\n" ); +printf ("-x Use xAPIC mode instead of x2APIC mode in the guest\n" );printf ("\n" );exit (KSFT_FAIL ); }
@@ -193,7 +204,7 @@ int main(int argc, char **argv)struct kvm_vm *vm;int irq, irq_cpu; -while ((c =getopt (argc, argv,"ad:ehi:mnt:v:" )) !=-1 ) { +while ((c =getopt (argc, argv,"ad:ehi:mnt:v:x" )) !=-1 ) {switch (c) {case 'a' : irq_affinity = true;
@@ -221,6 +232,9 @@ int main(int argc, char **argv)TEST_ASSERT (nr_vcpus <=KVM_MAX_VCPUS ,"KVM selftests support at most %u vCPUs" ,KVM_MAX_VCPUS );break ; +case 'x' : + x2apic = false; +break ;case 'h' :default :help (argv[0 ]);
@@ -236,6 +250,11 @@ int main(int argc, char **argv)vm_install_exception_handler (vm, vector, guest_irq_handler);vm_install_exception_handler (vm,NMI_VECTOR , guest_nmi_handler); +if (!x2apic) { +TEST_ASSERT (nr_vcpus <256 ,"xAPIC can only target IDs [0-254] (255 vCPUs)" ); +virt_pg_map (vm,APIC_DEFAULT_GPA ,APIC_DEFAULT_GPA ); + } +if (device_bdf) {if (!iommu_type) iommu_type =probe_iommu_type ();
@@ -259,6 +278,8 @@ int main(int argc, char **argv)kvm_assign_irqfd (vm, gsi, eventfd); +sync_global_to_guest (vm, x2apic); +if (migrate_vcpus)kvm_sched_getaffinity (0 ,sizeof (available_cpus), &available_cpus);
Signature · 1 line
2.54.0.1136.gdb2ca164c4-goog