KVM: Add idempotent controls for migrating system counter state
Subsystems (7)
v5 cover KVM: Add idempotent controls for migrating system counter state
KVM's current means of saving/restoring system counters is plagued with temporal issues. At least on ARM64 and x86, we migrate the guest's system counter by-value through the respective guest system register values (cntvct_el0, ia32_tsc). Restoring system counters by-value is brittle as the state is not idempotent: the host system counter is still oscillating between the attempted save and restore. Furthermore, VMMs may wish to transparently live migrate guest VMs, meaning that they include the elapsed time due to live migration blackout in the guest system counter view. The VMM thread could be preempted for any number of reasons (scheduler, L0 hypervisor under nested) between the time that it calculates the desired guest counter value and when KVM actually sets this counter state.
Despite the value-based interface that we present to userspace, KVM actually has idempotent guest controls by way of system counter offsets. We can avoid all of the issues associated with a value-based interface by abstracting these offset controls in new ioctls. This series introduces new vCPU device attributes to provide userspace access to the vCPU's system counter offset.
Patch 1 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK ioctls to provide userspace with a (host_tsc, realtime) instant. This is essential for a VMM to perform precise migration of the guest's system counters.
Patches 2-3 add support for x86 by shoehorning the new controls into the pre-existing synchronization heuristics.
Patches 4-5 implement a test for the new additions to KVM_{GET,SET}_CLOCK.
Patch 6 fixes some assertions in the kvm device attribute helpers.
Patches 7-8 implement at test for the tsc offset attribute introduced in patch 3.
Patch 9 adds a device attribute for the arm64 virtual counter-timer offset.
Patch 10 extends the test from patch 8 to cover the arm64 virtual counter-timer offset.
Patch 11 adds a device attribute for the arm64 physical counter-timer offset. Currently, this is implemented as a synthetic register, forcing the guest to trap to the host and emulating the offset in the fast exit path. Later down the line we will have hardware with FEAT_ECV, which allows the hypervisor to perform physical counter-timer offsetting in hardware (CNTPOFF_EL2).
Patch 12 extends the test from patch 8 to cover the arm64 physical counter-timer offset.
Patch 13 introduces a benchmark to measure the overhead of emulation in patch 11.
This series was tested on both an Ampere Mt. Jade and Haswell systems.
Physical counter benchmark
--------------------------
The following data was collected by running 10000 iterations of the benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S machine with 2 80-core Ampere Altra SoCs. Measurements were collected for both VHE and nVHE operation using the kvm-arm.mode= command-line parameter.
nVHE
----
+--------------------+--------+---------+ | Metric | Native | Trapped | +--------------------+--------+---------+ | Average | 54ns | 148ns | | Standard Deviation | 124ns | 122ns | | 95th Percentile | 258ns | 348ns | +--------------------+--------+---------+
VHE
---
+--------------------+--------+---------+ | Metric | Native | Trapped | +--------------------+--------+---------+ | Average | 53ns | 152ns | | Standard Deviation | 92ns | 94ns | | 95th Percentile | 204ns | 307ns | +--------------------+--------+---------+
This series applies cleanly to kvm/queue at the following commit:
8ad5e63649ff ("KVM: Don't take mmu_lock for range invalidation unless necessary")
v1 -> v2:
- Reimplemented as vCPU device attributes instead of a distinct ioctl.
- Added the (realtime, host_tsc) instant support to KVM_{GET,SET}_CLOCK
- Changed the arm64 implementation to broadcast counter offset values to all vCPUs in a guest. This upholds the architectural expectations of a consistent counter-timer across CPUs.
- Fixed a bug with traps in VHE mode. We now configure traps on every transition into a guest to handle differing VMs (trapped, emulated).
v2 -> v3:
- Added documentation for additions to KVM_{GET,SET}_CLOCK
- Added documentation for all new vCPU attributes
- Added documentation for suggested algorithm to migrate a guest's
TSC(s)
- Bug fixes throughout series
- Rename KVM_CLOCK_REAL_TIME -> KVM_CLOCK_REALTIME
v3 -> v4:
- Added patch to address incorrect device helper assertions (Drew)
- Carried Drew's r-b tags where appropriate
- x86 selftest cleanup
- Removed stale kvm_timer_init_vhe() function
- Removed unnecessary GUEST_DONE() from selftests
v4 -> v5:
- Fix typo in TSC migration algorithm
- Carry more of Drew's r-b tags
- clean up run loop logic in counter emulation benchmark (missed from Drew's comments on v3)
v1: https://lore.kernel.org/kvm/[email protected]/
v2: https://lore.kernel.org/r/[email protected]
v3: https://lore.kernel.org/r/[email protected]
v4: https://lore.kernel.org/r/[email protected]
Oliver Upton (13):
KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
KVM: x86: Refactor tsc synchronization code
KVM: x86: Expose TSC offset controls to userspace
tools: arch: x86: pull in pvclock headers
selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
selftests: KVM: Fix kvm device helper ioctl assertions
selftests: KVM: Add helpers for vCPU device attributes
selftests: KVM: Introduce system counter offset test
KVM: arm64: Allow userspace to configure a vCPU's virtual offset
selftests: KVM: Add support for aarch64 to system_counter_offset_test
KVM: arm64: Provide userspace access to the physical counter offset
selftests: KVM: Test physical counter offsetting
selftests: KVM: Add counter emulation benchmark
Documentation/virt/kvm/api.rst | 42 +- Documentation/virt/kvm/devices/vcpu.rst | 101 +++++ Documentation/virt/kvm/locking.rst | 11 + arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/asm/kvm_hyp.h | 2 - arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 2 + arch/arm64/kvm/arch_timer.c | 140 ++++-- arch/arm64/kvm/arm.c | 4 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 23 + arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++ arch/arm64/kvm/hyp/nvhe/switch.c | 2 - arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +- arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 ++ arch/x86/include/asm/kvm_host.h | 4 + arch/x86/include/uapi/asm/kvm.h | 4 + arch/x86/kvm/x86.c | 422 ++++++++++++++---- include/kvm/arm_arch_timer.h | 2 - include/uapi/linux/kvm.h | 7 +- tools/arch/x86/include/asm/pvclock-abi.h | 48 ++ tools/arch/x86/include/asm/pvclock.h | 103 +++++ tools/testing/selftests/kvm/.gitignore | 3 + tools/testing/selftests/kvm/Makefile | 4 + .../kvm/aarch64/counter_emulation_benchmark.c | 207 +++++++++ .../selftests/kvm/include/aarch64/processor.h | 24 + .../testing/selftests/kvm/include/kvm_util.h | 11 + tools/testing/selftests/kvm/lib/kvm_util.c | 44 +- .../kvm/system_counter_offset_test.c | 205 +++++++++ .../selftests/kvm/x86_64/kvm_clock_test.c | 204 +++++++++ 29 files changed, 1537 insertions(+), 158 deletions(-) create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h create mode 100644 tools/arch/x86/include/asm/pvclock-abi.h create mode 100644 tools/arch/x86/include/asm/pvclock.h create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
Signature · 1 line
2.32.0.432.gabb21c7263-goog
PATCH v5 1/13 KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
Handling the migration of TSCs correctly is difficult, in part because Linux does not provide userspace with the ability to retrieve a (TSC, realtime) clock pair for a single instant in time. In lieu of a more convenient facility, KVM can report similar information in the kvm_clock structure.
Provide userspace with a host TSC & realtime pair iff the realtime clock is based on the TSC. If userspace provides KVM_SET_CLOCK with a valid realtime value, advance the KVM clock by the amount of elapsed time. Do not step the KVM clock backwards, though, as it is a monotonic oscillator.
--- Documentation/virt/kvm/api.rst | 42 +++++++-- arch/x86/include/asm/kvm_host.h | 3 + arch/x86/kvm/x86.c | 149 ++++++++++++++++++++------------ include/uapi/linux/kvm.h | 7 +- 4 files changed, 137 insertions(+), 64 deletions(-)
Documentation/virt/kvm/api.rst +33 -9
@@ -993,20 +993,34 @@ such as migration. When KVM_CAP_ADJUST_CLOCK is passed to KVM_CHECK_EXTENSION, it returns the set of bits that KVM can return in struct kvm_clock_data's flag member. -The only flag defined now is KVM_CLOCK_TSC_STABLE. If set, the returned -value is the exact kvmclock value seen by all VCPUs at the instant -when KVM_GET_CLOCK was called. If clear, the returned value is simply -CLOCK_MONOTONIC plus a constant offset; the offset can be modified -with KVM_SET_CLOCK. KVM will try to make all VCPUs follow this clock, -but the exact value read by each VCPU could differ, because the host -TSC is not stable. +FLAGS: + +KVM_CLOCK_TSC_STABLE. If set, the returned value is the exact kvmclock +value seen by all VCPUs at the instant when KVM_GET_CLOCK was called. +If clear, the returned value is simply CLOCK_MONOTONIC plus a constant +offset; the offset can be modified with KVM_SET_CLOCK. KVM will try +to make all VCPUs follow this clock, but the exact value read by each +VCPU could differ, because the host TSC is not stable. + +KVM_CLOCK_REALTIME. If set, the `realtime` field in the kvm_clock_data +structure is populated with the value of the host's real time +clocksource at the instant when KVM_GET_CLOCK was called. If clear, +the `realtime` field does not contain a value. + +KVM_CLOCK_HOST_TSC. If set, the `host_tsc` field in the kvm_clock_data +structure is populated with the value of the host's timestamp counter (TSC) +at the instant when KVM_GET_CLOCK was called. If clear, the `host_tsc` field +does not contain a value. :: struct kvm_clock_data { __u64 clock; /* kvmclock current value */ __u32 flags; - __u32 pad[9]; + __u32 pad0; + __u64 realtime; + __u64 host_tsc; + __u32 pad[4]; };
@@ -1023,12 +1037,22 @@ Sets the current timestamp of kvmclock to the value specified in its parameter. In conjunction with KVM_GET_CLOCK, it is used to ensure monotonicity on scenarios such as migration. +FLAGS: + +KVM_CLOCK_REALTIME. If set, KVM will compare the value of the `realtime` field +with the value of the host's real time clocksource at the instant when +KVM_SET_CLOCK was called. The difference in elapsed time is added to the final +kvmclock value that will be provided to guests. + :: struct kvm_clock_data { __u64 clock; /* kvmclock current value */ __u32 flags; - __u32 pad[9]; + __u32 pad0; + __u64 realtime; + __u64 host_tsc; + __u32 pad[4]; };
arch/x86/include/asm/kvm_host.h +3 -0
@@ -1932,4 +1932,7 @@ int kvm_cpu_dirty_log_size(void);int alloc_all_memslots_rmaps (struct kvm *kvm); +#define KVM_CLOCK_VALID_FLAGS \ + (KVM_CLOCK_TSC_STABLE | KVM_CLOCK_REALTIME | KVM_CLOCK_HOST_TSC) +#endif /* _ASM_X86_KVM_HOST_H */
arch/x86/kvm/x86.c +95 -54
200 lines not on this page · raw message
include/uapi/linux/kvm.h +6 -1
@@ -1223,11 +1223,16 @@ struct kvm_irqfd {/* Do not use 1, KVM_CHECK_EXTENSION returned it before we had flags. */ #define KVM_CLOCK_TSC_STABLE 2 +#define KVM_CLOCK_REALTIME (1 << 2) +#define KVM_CLOCK_HOST_TSC (1 << 3)struct kvm_clock_data {__u64 clock;__u32 flags; -__u32 pad[9 ]; +__u32 pad0; +__u64 realtime; +__u64 host_tsc; +__u32 pad[4 ]; };/* For KVM_CAP_SW_TLB */
Signature · 1 line
2.32.0.432.gabb21c7263-goog
Sean Christopherson
maintainerRe: [PATCH v5 01/13] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
Quoted 41 lines · Oliver Upton
arch/x86/kvm/x86.c +13
-3@@ -2782,17 +2782,24 @@ static void kvm_gen_update_masterclock(struct kvm *kvm)#endif } -u64 get_kvmclock_ns (struct kvm *kvm) +/* +* Returns true if realtime and TSC values were written back to the caller. +* Returns false if a clock triplet cannot be obtained, such as if the host's +* realtime clock is not based on the TSC. +*/ +static bool get_kvmclock_and_realtime (struct kvm *kvm,u64 *kvmclock_ns, +u64 *realtime_ns,u64 *tsc) {struct kvm_arch *ka = &kvm->arch;struct pvclock_vcpu_time_info hv_clock;unsigned long flags; -u64 ret; +bool ret = false;spin_lock_irqsave (&ka->pvclock_gtod_sync_lock, flags);if (!ka->use_master_clock) {spin_unlock_irqrestore (&ka->pvclock_gtod_sync_lock, flags); -return get_kvmclock_base_ns () + ka->kvmclock_offset; + *kvmclock_ns =get_kvmclock_base_ns () + ka->kvmclock_offset; +return false; } hv_clock.tsc_timestamp = ka->master_cycle_now;@@ -2803,18 +2810,36 @@ u64 get_kvmclock_ns(struct kvm *kvm)get_cpu ();if (__this_cpu_read (cpu_tsc_khz)) { +struct timespec64 ts; +u64 tsc_val; + kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL, &hv_clock.tsc_shift,
arch/x86/kvm/x86.c u64 get_kvmclock_ns(struct kvm *kvm) · line 2826
Quoted 10 lines
arch/x86/kvm/x86.c +8
-1&hv_clock.tsc_to_system_mul); - ret =__pvclock_read_cycles (&hv_clock,rdtsc ()); + +if (kvm_get_walltime_and_clockread (&ts, &tsc_val)) { + *realtime_ns = ts.tv_nsec +NSEC_PER_SEC * ts.tv_sec; + *tsc = tsc_val; + ret = true; + } + + *kvmclock_ns =__pvclock_read_cycles (&hv_clock, tsc_val);
This is buggy, as tsc_val is not initialized if kvm_get_walltime_and_clockread() returns false. This also straight up fails to compile on 32-bit kernels because kvm_get_walltime_and_clockread() is wrapped with CONFIG_X86_64.
A gross way to resolve both issues would be (see below regarding 'data'):
if (__this_cpu_read(cpu_tsc_khz)) {
#ifdef CONFIG_X86_64
struct timespec64 ts;
if (kvm_get_walltime_and_clockread(&ts, &data->host_tsc)) {
data->realtime = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec;
data->flags |= KVM_CLOCK_REALTIME | KVM_CLOCK_HOST_TSC;
} else
#endif
data->host_tsc = rdtsc();
kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
&hv_clock.tsc_shift,
&hv_clock.tsc_to_system_mul);
data->clock = __pvclock_read_cycles(&hv_clock, data->host_tsc);
} else {
data->clock = get_kvmclock_base_ns() + ka->kvmclock_offset;
}arch/x86/kvm/x86.c u64 get_kvmclock_ns(struct kvm *kvm) · line 2827
Quoted 1 line
arch/x86/kvm/x86.c +0
-0}else
Not your code, but this needs braces.
Quoted 7 lines
arch/x86/kvm/x86.c +1
-1- ret =get_kvmclock_base_ns () + ka->kvmclock_offset; + *kvmclock_ns =get_kvmclock_base_ns () + ka->kvmclock_offset;put_cpu ();return ret; }
...
Quoted 7 lines
arch/x86/kvm/x86.c +3
-0@@ -5820,6 +5845,68 @@ int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state) }#endif /* CONFIG_HAVE_KVM_PM_NOTIFIER */ +static int kvm_vm_ioctl_get_clock (struct kvm *kvm, +void __user *argp) +{
arch/x86/kvm/x86.c int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state) · line 5860
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+struct kvm_clock_data data; + +memset (&data,0 ,sizeof (data)); + +if (get_kvmclock_and_realtime (kvm, &data.clock, &data.realtime, + &data.host_tsc)) + data.flags |=KVM_CLOCK_REALTIME |KVM_CLOCK_HOST_TSC ; + +if (kvm->arch.use_master_clock) + data.flags |=KVM_CLOCK_TSC_STABLE ;
I know nothing about the actual behavior, but this appears to have a potential race (though it came from the existing code). get_kvmclock_and_realtime() checks arch.use_master_clock under pvclock_gtod_sync_lock, but this does not. Even if that's functionally ok, it's a needless complication.
Fixing that weirdness would also provide an opportunity to clean up the API, e.g. the boolean return is not exactly straightforward. The simplest approach would be to take "struct kvm_clock_data *data" in get_kvmclock_and_realtime() instead of multiple params. Then that helper can set the flags accordingly, thus avoiding the question of whether or not there's a race and eliminating the boolean return. E.g.
arch/x86/kvm/x86.c +27 -31
@@ -2782,25 +2782,20 @@ static void kvm_gen_update_masterclock(struct kvm *kvm)#endif } -/* -* Returns true if realtime and TSC values were written back to the caller. -* Returns false if a clock triplet cannot be obtained, such as if the host's -* realtime clock is not based on the TSC. -*/ -static bool get_kvmclock_and_realtime (struct kvm *kvm,u64 *kvmclock_ns, -u64 *realtime_ns,u64 *tsc) +static void get_kvmclock_and_realtime (struct kvm *kvm, +struct kvm_clock_data *data) {struct kvm_arch *ka = &kvm->arch;struct pvclock_vcpu_time_info hv_clock;unsigned long flags; -bool ret = false;spin_lock_irqsave (&ka->pvclock_gtod_sync_lock, flags);if (!ka->use_master_clock) {spin_unlock_irqrestore (&ka->pvclock_gtod_sync_lock, flags); - *kvmclock_ns =get_kvmclock_base_ns () + ka->kvmclock_offset; -return false; + data->clock =get_kvmclock_base_ns () + ka->kvmclock_offset; +return ; } + data->flags |=KVM_CLOCK_TSC_STABLE ; hv_clock.tsc_timestamp = ka->master_cycle_now; hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;
@@ -2810,34 +2805,40 @@ static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,get_cpu ();if (__this_cpu_read (cpu_tsc_khz)) { +#ifdef CONFIG_X86_64 struct timespec64 ts; -u64 tsc_val; + +if (kvm_get_walltime_and_clockread (&ts, &data->host_tsc)) { + data->realtime = ts.tv_nsec +NSEC_PER_SEC * ts.tv_sec; + data->flags |=KVM_CLOCK_REALTIME |KVM_CLOCK_HOST_TSC ; + } else +#endif + data->host_tsc =rdtsc ();kvm_get_time_scale (NSEC_PER_SEC ,__this_cpu_read (cpu_tsc_khz) *1000LL , &hv_clock.tsc_shift, &hv_clock.tsc_to_system_mul); -if (kvm_get_walltime_and_clockread (&ts, &tsc_val)) { - *realtime_ns = ts.tv_nsec +NSEC_PER_SEC * ts.tv_sec; - *tsc = tsc_val; - ret = true; - } - - *kvmclock_ns =__pvclock_read_cycles (&hv_clock, tsc_val); - }else - *kvmclock_ns =get_kvmclock_base_ns () + ka->kvmclock_offset; + data->clock =__pvclock_read_cycles (&hv_clock, data->host_tsc); + }else { + data->clock =get_kvmclock_base_ns () + ka->kvmclock_offset; + }put_cpu (); - -return ret; }u64 get_kvmclock_ns (struct kvm *kvm) { -u64 kvmclock_ns, realtime_ns, tsc; +struct kvm_clock_data data; -get_kvmclock_and_realtime (kvm, &kvmclock_ns, &realtime_ns, &tsc); -return kvmclock_ns; +/* +* Zero flags as it's accessed RMW, leave everything else uninitialized +* as clock is always written and no other fields are consumed. +*/ + data.flags =0 ; + +get_kvmclock_and_realtime (kvm, &data); +return data.clock; } static void kvm_setup_pvclock_page(struct kvm_vcpu *v,
@@ -5852,12 +5853,7 @@ static int kvm_vm_ioctl_get_clock(struct kvm *kvm,memset (&data,0 ,sizeof (data)); -if (get_kvmclock_and_realtime (kvm, &data.clock, &data.realtime, - &data.host_tsc)) - data.flags |=KVM_CLOCK_REALTIME |KVM_CLOCK_HOST_TSC ; - -if (kvm->arch.use_master_clock) - data.flags |=KVM_CLOCK_TSC_STABLE ; +get_kvmclock_and_realtime (kvm, &data);if (copy_to_user (argp, &data,sizeof (data)))return -EFAULT ;
Quoted 9 lines
arch/x86/kvm/x86.c +9
-0+ +if (copy_to_user (argp, &data,sizeof (data))) +return -EFAULT ; + +return 0 ; +} + +static int kvm_vm_ioctl_set_clock (struct kvm *kvm, +void __user *argp)
arch/x86/kvm/x86.c int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state) · line 5879
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+{ +struct kvm_arch *ka = &kvm->arch; +struct kvm_clock_data data; +u64 now_raw_ns; + +if (copy_from_user (&data, argp,sizeof (data))) +return -EFAULT ; + +if (data.flags & ~KVM_CLOCK_REALTIME ) +return -EINVAL ;
Huh, this an odd ABI, the output of KVM_GET_CLOCK isn't legal input to KVM_SET_CLOCK. The existing code has the same behavior, so apparently it's ok, just odd.
Quoted 34 lines
arch/x86/kvm/x86.c +30
-0+ +/* +* TODO: userspace has to take care of races with VCPU_RUN, so +* kvm_gen_update_masterclock() can be cut down to locked +* pvclock_update_vm_gtod_copy(). +*/ +kvm_gen_update_masterclock (kvm); + +spin_lock_irq (&ka->pvclock_gtod_sync_lock); +if (data.flags &KVM_CLOCK_REALTIME ) { +u64 now_real_ns =ktime_get_real_ns (); + +/* +* Avoid stepping the kvmclock backwards. +*/ +if (now_real_ns > data.realtime) + data.clock += now_real_ns - data.realtime; + } + +if (ka->use_master_clock) + now_raw_ns = ka->master_kernel_ns; +else + now_raw_ns =get_kvmclock_base_ns (); + ka->kvmclock_offset = data.clock - now_raw_ns; +spin_unlock_irq (&ka->pvclock_gtod_sync_lock); + +kvm_make_all_cpus_request (kvm,KVM_REQ_CLOCK_UPDATE ); +return 0 ; +} +long kvm_arch_vm_ioctl (struct file *filp,unsigned int ioctl,unsigned long arg) {
arch/x86/kvm/x86.c long kvm_arch_vm_ioctl(struct file *filp, · line 6153
Quoted 3 lines
arch/x86/kvm/x86.c +0
-0@@ -6064,57 +6151,11 @@ long kvm_arch_vm_ioctl(struct file *filp, }#endif case KVM_SET_CLOCK : {
The curly braces can be dropped, both here and in KVM_GET_CLOCK.
Quoted 2 lines
arch/x86/kvm/x86.c +0
-0}case KVM_GET_CLOCK : {
...
Quoted 1 line
}
Re: [PATCH v5 01/13] KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
Quoted 49 lines · Sean Christopherson
Quoted 47 lines · Oliver Upton
arch/x86/kvm/x86.c +17
-4@@ -2782,17 +2782,24 @@ static void kvm_gen_update_masterclock(struct kvm *kvm)#endif } -u64 get_kvmclock_ns (struct kvm *kvm) +/* +* Returns true if realtime and TSC values were written back to the caller. +* Returns false if a clock triplet cannot be obtained, such as if the host's +* realtime clock is not based on the TSC. +*/ +static bool get_kvmclock_and_realtime (struct kvm *kvm,u64 *kvmclock_ns, +u64 *realtime_ns,u64 *tsc) {struct kvm_arch *ka = &kvm->arch;struct pvclock_vcpu_time_info hv_clock;unsigned long flags; -u64 ret; +bool ret = false;spin_lock_irqsave (&ka->pvclock_gtod_sync_lock, flags);if (!ka->use_master_clock) {spin_unlock_irqrestore (&ka->pvclock_gtod_sync_lock, flags); -return get_kvmclock_base_ns () + ka->kvmclock_offset; + *kvmclock_ns =get_kvmclock_base_ns () + ka->kvmclock_offset; +return false; } hv_clock.tsc_timestamp = ka->master_cycle_now;@@ -2803,18 +2810,36 @@ u64 get_kvmclock_ns(struct kvm *kvm)get_cpu ();if (__this_cpu_read (cpu_tsc_khz)) { +struct timespec64 ts; +u64 tsc_val; +kvm_get_time_scale (NSEC_PER_SEC ,__this_cpu_read (cpu_tsc_khz) *1000LL , &hv_clock.tsc_shift, &hv_clock.tsc_to_system_mul); - ret =__pvclock_read_cycles (&hv_clock,rdtsc ()); + +if (kvm_get_walltime_and_clockread (&ts, &tsc_val)) { + *realtime_ns = ts.tv_nsec +NSEC_PER_SEC * ts.tv_sec; + *tsc = tsc_val;
Quoted 8 lines
arch/x86/kvm/x86.c static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,
Quoted 4 lines
arch/x86/kvm/x86.c +4
-0+ ret = true; + } + + *kvmclock_ns =__pvclock_read_cycles (&hv_clock, tsc_val);This is buggy, as
tsc_valis not initialized ifkvm_get_walltime_and_clockread()returns false. This also straight up fails to compile on 32-bit kernels becausekvm_get_walltime_and_clockread()is wrapped withCONFIG_X86_64.
Pssh... works on my machine ;-)
Quoted 14 lines
A gross way to resolve both issues would be (see below regarding 'data'):
if (__this_cpu_read(cpu_tsc_khz)) { #ifdef CONFIG_X86_64 struct timespec64 ts; if (kvm_get_walltime_and_clockread(&ts, &data->host_tsc)) { data->realtime = ts.tv_nsec + NSEC_PER_SEC * ts.tv_sec; data->flags |= KVM_CLOCK_REALTIME | KVM_CLOCK_HOST_TSC; } else #endif data->host_tsc = rdtsc();
Quoted 9 lines
kvm_get_time_scale(NSEC_PER_SEC, __this_cpu_read(cpu_tsc_khz) * 1000LL,
&hv_clock.tsc_shift,
&hv_clock.tsc_to_system_mul);
data->clock = __pvclock_read_cycles(&hv_clock, data->host_tsc);
} else {
data->clock = get_kvmclock_base_ns() + ka->kvmclock_offset;
}I'll mull on this if there's any cleaner way to fix it, but thanks for the suggested fix! I missed the x86_64 ifdefs first time around.
Quoted 3 lines
arch/x86/kvm/x86.c u64 get_kvmclock_ns(struct kvm *kvm) · line 2827
Quoted 1 line
arch/x86/kvm/x86.c +0
-0}else Not your code, but this needs braces.
Ack.
Quoted 34 lines
Quoted 8 lines
arch/x86/kvm/x86.c +1
-1- ret =get_kvmclock_base_ns () + ka->kvmclock_offset; + *kvmclock_ns =get_kvmclock_base_ns () + ka->kvmclock_offset;put_cpu ();return ret; }...
Quoted 7 lines
arch/x86/kvm/x86.c +3
-0@@ -5820,6 +5845,68 @@ int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state) }#endif /* CONFIG_HAVE_KVM_PM_NOTIFIER */ +static int kvm_vm_ioctl_get_clock (struct kvm *kvm, +void __user *argp) +{arch/x86/kvm/x86.c int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state) · line 5860
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+struct kvm_clock_data data; + +memset (&data,0 ,sizeof (data)); + +if (get_kvmclock_and_realtime (kvm, &data.clock, &data.realtime, + &data.host_tsc)) + data.flags |=KVM_CLOCK_REALTIME |KVM_CLOCK_HOST_TSC ; + +if (kvm->arch.use_master_clock) + data.flags |=KVM_CLOCK_TSC_STABLE ;I know nothing about the actual behavior, but this appears to have a potential race (though it came from the existing code).
get_kvmclock_and_realtime()checks arch.use_master_clock underpvclock_gtod_sync_lock, but this does not. Even if that's functionally ok, it's a needless complication.
Quoted 6 lines
Fixing that weirdness would also provide an opportunity to clean up the API, e.g. the boolean return is not exactly straightforward. The simplest approach would be to take "struct
kvm_clock_data*data" inget_kvmclock_and_realtime()instead of multiple params. Then that helper can set the flags accordingly, thus avoiding the question of whether or not there's a race and eliminating the boolean return. E.g.
Yeah, agreed. I think it may be best to separate fixing the mess from the new API here.
Quoted 121 lines
arch/x86/kvm/x86.c +27
-31@@ -2782,25 +2782,20 @@ static void kvm_gen_update_masterclock(struct kvm *kvm)#endif } -/* -* Returns true if realtime and TSC values were written back to the caller. -* Returns false if a clock triplet cannot be obtained, such as if the host's -* realtime clock is not based on the TSC. -*/ -static bool get_kvmclock_and_realtime (struct kvm *kvm,u64 *kvmclock_ns, -u64 *realtime_ns,u64 *tsc) +static void get_kvmclock_and_realtime (struct kvm *kvm, +struct kvm_clock_data *data) {struct kvm_arch *ka = &kvm->arch;struct pvclock_vcpu_time_info hv_clock;unsigned long flags; -bool ret = false;spin_lock_irqsave (&ka->pvclock_gtod_sync_lock, flags);if (!ka->use_master_clock) {spin_unlock_irqrestore (&ka->pvclock_gtod_sync_lock, flags); - *kvmclock_ns =get_kvmclock_base_ns () + ka->kvmclock_offset; -return false; + data->clock =get_kvmclock_base_ns () + ka->kvmclock_offset; +return ; } + data->flags |=KVM_CLOCK_TSC_STABLE ; hv_clock.tsc_timestamp = ka->master_cycle_now; hv_clock.system_time = ka->master_kernel_ns + ka->kvmclock_offset;@@ -2810,34 +2805,40 @@ static bool get_kvmclock_and_realtime(struct kvm *kvm, u64 *kvmclock_ns,get_cpu ();if (__this_cpu_read (cpu_tsc_khz)) { +#ifdef CONFIG_X86_64 struct timespec64 ts; -u64 tsc_val; + +if (kvm_get_walltime_and_clockread (&ts, &data->host_tsc)) { + data->realtime = ts.tv_nsec +NSEC_PER_SEC * ts.tv_sec; + data->flags |=KVM_CLOCK_REALTIME |KVM_CLOCK_HOST_TSC ; + } else +#endif + data->host_tsc =rdtsc ();kvm_get_time_scale (NSEC_PER_SEC ,__this_cpu_read (cpu_tsc_khz) *1000LL , &hv_clock.tsc_shift, &hv_clock.tsc_to_system_mul); -if (kvm_get_walltime_and_clockread (&ts, &tsc_val)) { - *realtime_ns = ts.tv_nsec +NSEC_PER_SEC * ts.tv_sec; - *tsc = tsc_val; - ret = true; - } - - *kvmclock_ns =__pvclock_read_cycles (&hv_clock, tsc_val); - }else - *kvmclock_ns =get_kvmclock_base_ns () + ka->kvmclock_offset; + data->clock =__pvclock_read_cycles (&hv_clock, data->host_tsc); + }else { + data->clock =get_kvmclock_base_ns () + ka->kvmclock_offset; + }put_cpu (); - -return ret; }u64 get_kvmclock_ns (struct kvm *kvm) { -u64 kvmclock_ns, realtime_ns, tsc; +struct kvm_clock_data data; -get_kvmclock_and_realtime (kvm, &kvmclock_ns, &realtime_ns, &tsc); -return kvmclock_ns; +/* +* Zero flags as it's accessed RMW, leave everything else uninitialized +* as clock is always written and no other fields are consumed. +*/ + data.flags =0 ; + +get_kvmclock_and_realtime (kvm, &data); +return data.clock; } static void kvm_setup_pvclock_page(struct kvm_vcpu *v,@@ -5852,12 +5853,7 @@ static int kvm_vm_ioctl_get_clock(struct kvm *kvm,memset (&data,0 ,sizeof (data)); -if (get_kvmclock_and_realtime (kvm, &data.clock, &data.realtime, - &data.host_tsc)) - data.flags |=KVM_CLOCK_REALTIME |KVM_CLOCK_HOST_TSC ; - -if (kvm->arch.use_master_clock) - data.flags |=KVM_CLOCK_TSC_STABLE ; +get_kvmclock_and_realtime (kvm, &data);if (copy_to_user (argp, &data,sizeof (data)))return -EFAULT ;Quoted 15 lines
arch/x86/kvm/x86.c +14
-0+ +if (copy_to_user (argp, &data,sizeof (data))) +return -EFAULT ; + +return 0 ; +} + +static int kvm_vm_ioctl_set_clock (struct kvm *kvm, +void __user *argp) +{ +struct kvm_arch *ka = &kvm->arch; +struct kvm_clock_data data; +u64 now_raw_ns; +
Quoted 8 lines
arch/x86/kvm/x86.c int kvm_arch_pm_notifier(struct kvm *kvm, unsigned long state) · line 5879
Quoted 5 lines
arch/x86/kvm/x86.c +5
-0+if (copy_from_user (&data, argp,sizeof (data))) +return -EFAULT ; + +if (data.flags & ~KVM_CLOCK_REALTIME ) +return -EINVAL ;Huh, this an odd ABI, the output of
KVM_GET_CLOCKisn't legal input toKVM_SET_CLOCK. The existing code has the same behavior, so apparently it's ok, just odd.
Quite the head scratcher to me as well /shrug
Quoted 31 lines
Quoted 31 lines
arch/x86/kvm/x86.c +30
-0+ +/* +* TODO: userspace has to take care of races with VCPU_RUN, so +* kvm_gen_update_masterclock() can be cut down to locked +* pvclock_update_vm_gtod_copy(). +*/ +kvm_gen_update_masterclock (kvm); + +spin_lock_irq (&ka->pvclock_gtod_sync_lock); +if (data.flags &KVM_CLOCK_REALTIME ) { +u64 now_real_ns =ktime_get_real_ns (); + +/* +* Avoid stepping the kvmclock backwards. +*/ +if (now_real_ns > data.realtime) + data.clock += now_real_ns - data.realtime; + } + +if (ka->use_master_clock) + now_raw_ns = ka->master_kernel_ns; +else + now_raw_ns =get_kvmclock_base_ns (); + ka->kvmclock_offset = data.clock - now_raw_ns; +spin_unlock_irq (&ka->pvclock_gtod_sync_lock); + +kvm_make_all_cpus_request (kvm,KVM_REQ_CLOCK_UPDATE ); +return 0 ; +} + long kvm_arch_vm_ioctl(struct file *filp,
Quoted 8 lines
arch/x86/kvm/x86.c long kvm_arch_vm_ioctl(struct file *filp, · line 6153
Quoted 6 lines
arch/x86/kvm/x86.c +0
-0unsigned int ioctl, unsigned long arg) {@@ -6064,57 +6151,11 @@ long kvm_arch_vm_ioctl(struct file *filp, }#endif case KVM_SET_CLOCK : {The curly braces can be dropped, both here and in
KVM_GET_CLOCK.
Ack.
Quoted 6 lines
Quoted 2 lines
arch/x86/kvm/x86.c +0
-0}case KVM_GET_CLOCK : {...
Quoted 1 line
}
As always, thanks for the careful review Sean!
--
Best,
Oliver
PATCH v5 2/13 KVM: x86: Refactor tsc synchronization code
In reply to Oliver UptonRefactor kvm_synchronize_tsc to make a new function that allows callers to specify TSC parameters (offset, value, nanoseconds, etc.) explicitly for the sake of participating in TSC synchronization.
This changes the locking semantics around TSC writes. Writes to the TSC will now take the pvclock gtod lock while holding the tsc write lock, whereas before these locks were disjoint.
--- Documentation/virt/kvm/locking.rst | 11 +++ arch/x86/kvm/x86.c | 106 +++++++++++++++++------------ 2 files changed, 74 insertions(+), 43 deletions(-)
Documentation/virt/kvm/locking.rst +11 -0
@@ -36,6 +36,9 @@ On x86: holding kvm->arch.mmu_lock (typically with ``read_lock``, otherwise there's no need to take kvm->arch.tdp_mmu_pages_lock at all). +- kvm->arch.tsc_write_lock is taken outside + kvm->arch.pvclock_gtod_sync_lock + Everything else is a leaf: no other lock is taken inside the critical sections.
@@ -222,6 +225,14 @@ time it will be set using the Dirty tracking mechanism described above. :Comment: 'raw' because hardware enabling/disabling must be atomic /wrt migration. +:Name: kvm_arch::pvclock_gtod_sync_lock +:Type: raw_spinlock_t +:Arch: x86 +:Protects: kvm_arch::{cur_tsc_generation,cur_tsc_nsec,cur_tsc_write, + cur_tsc_offset,nr_vcpus_matched_tsc} +:Comment: 'raw' because updating the kvm master clock must not be + preempted. + :Name: kvm_arch::tsc_write_lock :Type: raw_spinlock :Arch: x86
arch/x86/kvm/x86.c +63 -43
127 lines not on this page · raw message
Signature · 1 line
2.32.0.432.gabb21c7263-goog
Sean Christopherson
maintainerRe: [PATCH v5 02/13] KVM: x86: Refactor tsc synchronization code
Quoted 5 lines · Oliver Upton
Refactor
kvm_synchronize_tscto make a new function that allows callers to specify TSC parameters (offset, value, nanoseconds, etc.) explicitly for the sake of participating in TSC synchronization.This changes the locking semantics around TSC writes.
"refactor" and "changes the locking semantics" are somewhat contradictory. The correct way to do this is to first change the locking semantics, then extract the helper you want. That makes review and archaeology easier, and isolates the locking change in case it isn't so safe after all.
Quoted 5 lines
Writes to the TSC will now take the pvclock gtod lock while holding the tsc write lock, whereas before these locks were disjoint.
Reviewed-by David Matlack <[email protected]>Signed-off-by Oliver Upton <[email protected]>
Quoted 7 lines
---
+/*
+ * Infers attempts to synchronize the guest's tsc from host writes. Sets the
+ * offset for the vcpu and tracks the TSC matching generation that the vcpu
+ * participates in.
+ *
+ * Must holdkvm->arch.tsc_write_lockto call this function.
Drop this blurb, lockdep assertions exist for a reason :-)
arch/x86/kvm/x86.c static inline bool kvm_check_tsc_unstable(void) · line 2457
Quoted 6 lines
+ */
arch/x86/kvm/x86.c +5
-0+static void __kvm_synchronize_tsc (struct kvm_vcpu *vcpu,u64 offset,u64 tsc, +u64 ns,bool matched) +{ +struct kvm *kvm = vcpu->kvm; +bool already_matched;
Eww, not your code, but "matched" and "already_matched" are not helpful names, e.g. they don't provide a clue as to _what_ matched, and thus don't explain why there are two separate variables. And I would expect an "already" variant to come in from the caller, not the other way 'round.
matched => freq_matched
already_matched => gen_matched
Quoted 15 lines
arch/x86/kvm/x86.c +15
-0+unsigned long flags; + +lockdep_assert_held (&kvm->arch.tsc_write_lock); + + already_matched = + (vcpu->arch.this_tsc_generation == kvm->arch.cur_tsc_generation); + +/* +* We track the most recent recorded KHZ, write and time to +* allow the matching interval to be extended at each write. +*/ + kvm->arch.last_tsc_nsec = ns; + kvm->arch.last_tsc_write = tsc; + kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz; +
arch/x86/kvm/x86.c static inline bool kvm_check_tsc_unstable(void) · line 2482
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+ vcpu->arch.last_guest_tsc = tsc; + +/* Keep track of which generation this VCPU has synchronized to */ + vcpu->arch.this_tsc_generation = kvm->arch.cur_tsc_generation; + vcpu->arch.this_tsc_nsec = kvm->arch.cur_tsc_nsec; + vcpu->arch.this_tsc_write = kvm->arch.cur_tsc_write; + +kvm_vcpu_write_tsc_offset (vcpu, offset); + +spin_lock_irqsave (&kvm->arch.pvclock_gtod_sync_lock, flags);
I believe this can be spin_lock(), since AFAICT the caller _must_ disable IRQs when taking tsc_write_lock, i.e. we know IRQs are disabled at this point.
Quoted 5 lines
arch/x86/kvm/x86.c +5
-0+if (!matched) { +/* +* We split periods of matched TSC writes into generations. +* For each generation, we track the original measured +* nanosecond time, offset, and write, so if TSCs are in
arch/x86/kvm/x86.c static inline bool kvm_check_tsc_unstable(void) · line 2497
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+* sync, we can match exact offset, and if not, we can match +* exact software computation in compute_guest_tsc() +* +* These values are tracked in kvm->arch.cur_xxx variables. +*/ + kvm->arch.nr_vcpus_matched_tsc =0 ; + kvm->arch.cur_tsc_generation++; + kvm->arch.cur_tsc_nsec = ns; + kvm->arch.cur_tsc_write = tsc; + kvm->arch.cur_tsc_offset = offset;
IMO, adjusting kvm->arch.cur_tsc_* belongs outside of pvclock_gtod_sync_lock. Based on the existing code, it is protected by tsc_write_lock. I don't care about the extra work while holding pvclock_gtod_sync_lock, but it's very confusing to see code that reads variables outside of a lock, then take a lock and write those same variables without first rechecking.
arch/x86/kvm/x86.c static inline bool kvm_check_tsc_unstable(void) · line 2498
Quoted 1 line
arch/x86/kvm/x86.c +1
-0+ matched = false;
What's the point of clearing "matched"? It's already false...
Quoted 8 lines
arch/x86/kvm/x86.c +8
-0+ }else if (!already_matched) { + kvm->arch.nr_vcpus_matched_tsc++; + } + +kvm_track_tsc_matching (vcpu); +spin_unlock_irqrestore (&kvm->arch.pvclock_gtod_sync_lock, flags); +} +
Re: [PATCH v5 02/13] KVM: x86: Refactor tsc synchronization code
Quoted 6 lines · Sean Christopherson
Quoted 4 lines · Oliver Upton
Refactor
kvm_synchronize_tscto make a new function that allows callers to specify TSC parameters (offset, value, nanoseconds, etc.) explicitly for the sake of participating in TSC synchronization.
Quoted 6 lines
Quoted 1 line
This changes the locking semantics around TSC writes.
"refactor" and "changes the locking semantics" are somewhat contradictory. The correct way to do this is to first change the locking semantics, then extract the helper you want. That makes review and archaeology easier, and isolates the locking change in case it isn't so safe after all.
Indeed, it was mere laziness doing so :)
Quoted 13 lines
Quoted 5 lines
Writes to the TSC will now take the pvclock gtod lock while holding the tsc write lock, whereas before these locks were disjoint.
Reviewed-by David Matlack <[email protected]>Signed-off-by Oliver Upton <[email protected]>Quoted 7 lines
---
+/*
+ * Infers attempts to synchronize the guest's tsc from host writes. Sets the
+ * offset for the vcpu and tracks the TSC matching generation that the vcpu
+ * participates in.
+ *
+ * Must holdkvm->arch.tsc_write_lockto call this function.
Quoted 2 lines
Drop this blurb, lockdep assertions exist for a reason :-)
Ack.
Quoted 12 lines
arch/x86/kvm/x86.c static inline bool kvm_check_tsc_unstable(void) · line 2457
Quoted 6 lines
+ */
arch/x86/kvm/x86.c +5
-0+static void __kvm_synchronize_tsc (struct kvm_vcpu *vcpu,u64 offset,u64 tsc, +u64 ns,bool matched) +{ +struct kvm *kvm = vcpu->kvm; +bool already_matched;Eww, not your code, but "matched" and "
already_matched" are not helpful names, e.g. they don't provide a clue as to _what_ matched, and thus don't explain why there are two separate variables. And I would expect an "already" variant to come in from the caller, not the other way 'round.
Quoted 2 lines
matched =>
freq_matched
already_matched=>gen_matched
Yeah, everything this series touches is a bit messy. I greedily avoided the pile of cleanups that are needed, but alas...
Quoted 4 lines
Quoted 1 line
+
spin_lock_irqsave(&kvm->arch.pvclock_gtod_sync_lock, flags);I believe this can be
spin_lock(), since AFAICT the caller _must_ disable IRQs when takingtsc_write_lock, i.e. we know IRQs are disabled at this point.
Definitely.
Quoted 23 lines
Quoted 6 lines
arch/x86/kvm/x86.c +5
-0+if (!matched) { +/* +* We split periods of matched TSC writes into generations. +* For each generation, we track the original measured +* nanosecond time, offset, and write, so if TSCs are in arch/x86/kvm/x86.c static inline bool kvm_check_tsc_unstable(void) · line 2497
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+* sync, we can match exact offset, and if not, we can match +* exact software computation in compute_guest_tsc() +* +* These values are tracked in kvm->arch.cur_xxx variables. +*/ + kvm->arch.nr_vcpus_matched_tsc =0 ; + kvm->arch.cur_tsc_generation++; + kvm->arch.cur_tsc_nsec = ns; + kvm->arch.cur_tsc_write = tsc; + kvm->arch.cur_tsc_offset = offset;IMO, adjusting kvm->arch.cur_tsc_* belongs outside of
pvclock_gtod_sync_lock. Based on the existing code, it is protected bytsc_write_lock. I don't care about the extra work while holdingpvclock_gtod_sync_lock, but it's very confusing to see code that reads variables outside of a lock, then take a lock and write those same variables without first rechecking.
Quoted 3 lines
Quoted 1 line
+ matched = false;
What's the point of clearing "matched"? It's already false...
None, besides just yanking the old chunk of code :)
Quoted 9 lines
arch/x86/kvm/x86.c static inline bool kvm_check_tsc_unstable(void) · line 2506
Quoted 9 lines
arch/x86/kvm/x86.c +8
-0+ }else if (!already_matched) { + kvm->arch.nr_vcpus_matched_tsc++; + } + +kvm_track_tsc_matching (vcpu); +spin_unlock_irqrestore (&kvm->arch.pvclock_gtod_sync_lock, flags); +} +
--
Thanks,
Oliver
PATCH v5 3/13 KVM: x86: Expose TSC offset controls to userspace
In reply to Oliver UptonTo date, VMM-directed TSC synchronization and migration has been a bit messy. KVM has some baked-in heuristics around TSC writes to infer if the VMM is attempting to synchronize. This is problematic, as it depends on host userspace writing to the guest's TSC within 1 second of the last write.
A much cleaner approach to configuring the guest's views of the TSC is to simply migrate the TSC offset for every vCPU. Offsets are idempotent, and thus not subject to change depending on when the VMM actually reads/writes values from/to KVM. The VMM can then read the TSC once with KVM_GET_CLOCK to capture a (realtime, host_tsc) pair at the instant when the guest is paused.
--- Documentation/virt/kvm/devices/vcpu.rst | 57 ++++++++ arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 4 + arch/x86/kvm/x86.c | 167 ++++++++++++++++++++++++ 4 files changed, 229 insertions(+)
Documentation/virt/kvm/devices/vcpu.rst +57 -0
@@ -161,3 +161,60 @@ Specifies the base address of the stolen time structure for this VCPU. The base address must be 64 byte aligned and exist within a valid guest memory region. See Documentation/virt/kvm/arm/pvtime.rst for more information including the layout of the stolen time structure. + +4. GROUP: KVM_VCPU_TSC_CTRL +=========================== + +:Architectures: x86 + +4.1 ATTRIBUTE: KVM_VCPU_TSC_OFFSET + +:Parameters: 64-bit unsigned TSC offset + +Returns: + + ======= ====================================== + -EFAULT Error reading/writing the provided + parameter address. + -ENXIO Attribute not supported + ======= ====================================== + +Specifies the guest's TSC offset relative to the host's TSC. The guest's +TSC is then derived by the following equation: + + guest_tsc = host_tsc + KVM_VCPU_TSC_OFFSET + +This attribute is useful for the precise migration of a guest's TSC. The +following describes a possible algorithm to use for the migration of a +guest's TSC: + +From the source VMM process: + +1. Invoke the KVM_GET_CLOCK ioctl to record the host TSC (t_0), + kvmclock nanoseconds (k_0), and realtime nanoseconds (r_0). + +2. Read the KVM_VCPU_TSC_OFFSET attribute for every vCPU to record the + guest TSC offset (off_n). + +3. Invoke the KVM_GET_TSC_KHZ ioctl to record the frequency of the + guest's TSC (freq). + +From the destination VMM process: + +4. Invoke the KVM_SET_CLOCK ioctl, providing the kvmclock nanoseconds + (k_0) and realtime nanoseconds (r_0) in their respective fields. + Ensure that the KVM_CLOCK_REALTIME flag is set in the provided + structure. KVM will advance the VM's kvmclock to account for elapsed + time since recording the clock values. + +5. Invoke the KVM_GET_CLOCK ioctl to record the host TSC (t_1) and + kvmclock nanoseconds (k_1). + +6. Adjust the guest TSC offsets for every vCPU to account for (1) time + elapsed since recording state and (2) difference in TSCs between the + source and destination machine: + + new_off_n = t_0 + off_n + (k_1 - k_0) * freq - t_1 + +7. Write the KVM_VCPU_TSC_OFFSET attribute for every vCPU with the + respective value derived in the previous step.
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1070,6 +1070,7 @@ struct kvm_arch {u64 last_tsc_nsec;u64 last_tsc_write;u32 last_tsc_khz; +u64 last_tsc_offset;u64 cur_tsc_nsec;u64 cur_tsc_write;u64 cur_tsc_offset;
arch/x86/include/uapi/asm/kvm.h +4 -0
@@ -503,4 +503,8 @@ struct kvm_pmu_event_filter {#define KVM_PMU_EVENT_ALLOW 0#define KVM_PMU_EVENT_DENY 1 +/* for KVM_{GET,SET,HAS}_DEVICE_ATTR */ +#define KVM_VCPU_TSC_CTRL 0/* control group for the timestamp counter (TSC) */ +#define KVM_VCPU_TSC_OFFSET 0/* attribute for the TSC offset */ +#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/x86.c +167 -0
191 lines not on this page · raw message
Signature · 1 line
2.32.0.432.gabb21c7263-goog
Sean Christopherson
maintainerRe: [PATCH v5 03/13] KVM: x86: Expose TSC offset controls to userspace
Quoted 35 lines · Oliver Upton
To date, VMM-directed TSC synchronization and migration has been a bit messy. KVM has some baked-in heuristics around TSC writes to infer if the VMM is attempting to synchronize. This is problematic, as it depends on host userspace writing to the guest's TSC within 1 second of the last write.
A much cleaner approach to configuring the guest's views of the TSC is to simply migrate the TSC offset for every vCPU. Offsets are idempotent, and thus not subject to change depending on when the VMM actually reads/writes values from/to KVM. The VMM can then read the TSC once with
KVM_GET_CLOCKto capture a (realtime,host_tsc) pair at the instant when the guest is paused.Cc David Matlack <[email protected]>Signed-off-by Oliver Upton <[email protected]>--- Documentation/virt/kvm/devices/vcpu.rst | 57 ++++++++ arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 4 + arch/x86/kvm/x86.c | 167 ++++++++++++++++++++++++ 4 files changed, 229 insertions(+)Documentation/virt/kvm/devices/vcpu.rst +5
-0@@ -161,3 +161,60 @@ Specifies the base address of the stolen time structure for this VCPU. The base address must be 64 byte aligned and exist within a valid guest memory region. See Documentation/virt/kvm/arm/pvtime.rst for more information including the layout of the stolen time structure. + +4. GROUP: KVM_VCPU_TSC_CTRL +=========================== + +:Architectures: x86
Documentation/virt/kvm/devices/vcpu.rst Specifies the base address of the stolen time structure for this VCPU. The · line 178
Quoted 10 lines
Documentation/virt/kvm/devices/vcpu.rst +10
-0+ +4.1 ATTRIBUTE: KVM_VCPU_TSC_OFFSET + +:Parameters: 64-bit unsigned TSC offset + +Returns: + + ======= ====================================== + -EFAULT Error reading/writing the provided + parameter address.
There's a rogue space in here (git show highlights it).
Quoted 72 lines
Documentation/virt/kvm/devices/vcpu.rst +42
-0+ -ENXIO Attribute not supported + ======= ====================================== + +Specifies the guest's TSC offset relative to the host's TSC. The guest's +TSC is then derived by the following equation: + + guest_tsc = host_tsc + KVM_VCPU_TSC_OFFSET + +This attribute is useful for the precise migration of a guest's TSC. The +following describes a possible algorithm to use for the migration of a +guest's TSC: + +From the source VMM process: + +1. Invoke the KVM_GET_CLOCK ioctl to record the host TSC (t_0), + kvmclock nanoseconds (k_0), and realtime nanoseconds (r_0). + +2. Read the KVM_VCPU_TSC_OFFSET attribute for every vCPU to record the + guest TSC offset (off_n). + +3. Invoke the KVM_GET_TSC_KHZ ioctl to record the frequency of the + guest's TSC (freq). + +From the destination VMM process: + +4. Invoke the KVM_SET_CLOCK ioctl, providing the kvmclock nanoseconds + (k_0) and realtime nanoseconds (r_0) in their respective fields. + Ensure that the KVM_CLOCK_REALTIME flag is set in the provided + structure. KVM will advance the VM's kvmclock to account for elapsed + time since recording the clock values. + +5. Invoke the KVM_GET_CLOCK ioctl to record the host TSC (t_1) and + kvmclock nanoseconds (k_1). + +6. Adjust the guest TSC offsets for every vCPU to account for (1) time + elapsed since recording state and (2) difference in TSCs between the + source and destination machine: + + new_off_n = t_0 + off_n + (k_1 - k_0) * freq - t_1 + +7. Write the KVM_VCPU_TSC_OFFSET attribute for every vCPU with the + respective value derived in the previous step.arch/x86/include/asm/kvm_host.h +1
-0@@ -1070,6 +1070,7 @@ struct kvm_arch {u64 last_tsc_nsec;u64 last_tsc_write;u32 last_tsc_khz; +u64 last_tsc_offset;u64 cur_tsc_nsec;u64 cur_tsc_write;u64 cur_tsc_offset;arch/x86/include/uapi/asm/kvm.h +4
-0@@ -503,4 +503,8 @@ struct kvm_pmu_event_filter {#define KVM_PMU_EVENT_ALLOW 0#define KVM_PMU_EVENT_DENY 1 +/* for KVM_{GET,SET,HAS}_DEVICE_ATTR */ +#define KVM_VCPU_TSC_CTRL 0/* control group for the timestamp counter (TSC) */ +#define KVM_VCPU_TSC_OFFSET 0/* attribute for the TSC offset */ +#endif /* _ASM_X86_KVM_H */ arch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 l1_offset) · line 2419
Quoted 7 lines
arch/x86/kvm/x86.c +4
-0@@ -2413,6 +2413,11 @@ static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 l1_offset)static_call (kvm_x86_write_tsc_offset)(vcpu, vcpu->arch.tsc_offset); } +static u64 kvm_vcpu_read_tsc_offset (struct kvm_vcpu *vcpu) +{ +return vcpu->arch.l1_tsc_offset; +}
This is not a helpful, uh, helper. Based on the name, I would expect it to read the TSC_OFFSET in the context of L1 vs. L2. Assuming you really want L1's offset, just read vcpu->arch.l1_tsc_offset directly. That will obviate the "need" for a local offset (see below).
Quoted 36 lines
arch/x86/kvm/x86.c +22
-0+static void kvm_vcpu_write_tsc_multiplier (struct kvm_vcpu *vcpu,u64 l1_multiplier) { vcpu->arch.l1_tsc_scaling_ratio = l1_multiplier;@@ -2469,6 +2474,7 @@ static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc, kvm->arch.last_tsc_nsec = ns; kvm->arch.last_tsc_write = tsc; kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz; + kvm->arch.last_tsc_offset = offset; vcpu->arch.last_guest_tsc = tsc;@@ -4928,6 +4934,137 @@ static int kvm_set_guest_paused(struct kvm_vcpu *vcpu)return 0 ; } +static int kvm_arch_tsc_has_attr (struct kvm_vcpu *vcpu, +struct kvm_device_attr *attr) +{ +int r; + +switch (attr->attr) { +case KVM_VCPU_TSC_OFFSET : + r =0 ; +break ; +default : + r = -ENXIO ; + } + +return r; +} + +static int kvm_arch_tsc_get_attr (struct kvm_vcpu *vcpu, +struct kvm_device_attr *attr) +{ +void __user *uaddr = (void __user *)attr->addr;
arch/x86/kvm/x86.c static int kvm_set_guest_paused(struct kvm_vcpu *vcpu) · line 4966
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+int r; + +switch (attr->attr) { +case KVM_VCPU_TSC_OFFSET : { +u64 offset; + + offset =kvm_vcpu_read_tsc_offset (vcpu); + r = -EFAULT ; +if (copy_to_user (uaddr, &offset,sizeof (offset))) +break ;
Use put_user(), then this all becomes short and sweet without curly braces:
case KVM_VCPU_TSC_OFFSET: r = -EFAULT; if (put_user(vcpu->arch.l1_tsc_offset, uaddr)) break; r = 0; break;
Quoted 16 lines
arch/x86/kvm/x86.c +16
-0+ + r =0 ; +break ; + } +default : + r = -ENXIO ; + } + +return r; +} + +static int kvm_arch_tsc_set_attr (struct kvm_vcpu *vcpu, +struct kvm_device_attr *attr) +{ +void __user *uaddr = (void __user *)attr->addr; +struct kvm *kvm = vcpu->kvm;
arch/x86/kvm/x86.c static int kvm_set_guest_paused(struct kvm_vcpu *vcpu) · line 4992
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+int r; + +switch (attr->attr) { +case KVM_VCPU_TSC_OFFSET : { +u64 offset, tsc, ns; +unsigned long flags; +bool matched; + + r = -EFAULT ; +if (copy_from_user (&offset, uaddr,sizeof (offset)))
This can be get_user().
Quoted 23 lines
arch/x86/kvm/x86.c +23
-0+break ; + +raw_spin_lock_irqsave (&kvm->arch.tsc_write_lock, flags); + + matched = (vcpu->arch.virtual_tsc_khz && + kvm->arch.last_tsc_khz == vcpu->arch.virtual_tsc_khz && + kvm->arch.last_tsc_offset == offset); + + tsc =kvm_scale_tsc (vcpu,rdtsc (), vcpu->arch.l1_tsc_scaling_ratio) + offset; + ns =get_kvmclock_base_ns (); + +__kvm_synchronize_tsc (vcpu, offset, tsc, ns, matched); +raw_spin_unlock_irqrestore (&kvm->arch.tsc_write_lock, flags); + + r =0 ; +break ; + } +default : + r = -ENXIO ; + } + +return r; +}
...
Quoted 26 lines
static int kvm_vcpu_ioctl_enable_cap(struct
kvm_vcpu*vcpu,arch/x86/kvm/x86.c +19
-0struct kvm_enable_cap *cap) {@@ -5382,6 +5519,36 @@ long kvm_arch_vcpu_ioctl(struct file *filp, r =__set_sregs2 (vcpu, u.sregs2);break ; } +case KVM_HAS_DEVICE_ATTR : { +struct kvm_device_attr attr; + + r = -EFAULT ; +if (copy_from_user (&attr, argp,sizeof (attr))) + goto out; + + r =kvm_vcpu_ioctl_has_device_attr (vcpu, &attr); +break ; + } +case KVM_GET_DEVICE_ATTR : { +struct kvm_device_attr attr; + + r = -EFAULT ; +if (copy_from_user (&attr, argp,sizeof (attr))) + goto out; + + r =kvm_vcpu_ioctl_get_device_attr (vcpu, &attr); +break ;
arch/x86/kvm/x86.c long kvm_arch_vcpu_ioctl(struct file *filp, · line 5550
Quoted 10 lines
arch/x86/kvm/x86.c +10
-0+ } +case KVM_SET_DEVICE_ATTR : { +struct kvm_device_attr attr; + + r = -EFAULT ; +if (copy_from_user (&attr, argp,sizeof (attr))) + goto out; + + r =kvm_vcpu_ioctl_set_device_attr (vcpu, &attr); +break ;
That's a lot of copy paste code, and I omitted a big chunk, too. What about something like:
static int kvm_vcpu_ioctl_do_device_attr(struct kvm_vcpu *vcpu,
unsigned int ioctl, void __user *argp)
{
struct kvm_device_attr attr;
int r;
if (copy_from_user(&attr, argp, sizeof(attr)))
return -EFAULT;
if (attr->group != KVM_VCPU_TSC_CTRL)
return -ENXIO;
switch (ioctl) {
case KVM_HAS_DEVICE_ATTR:
r = kvm_arch_tsc_has_attr(vcpu, attr);
break;
case KVM_GET_DEVICE_ATTR:
r = kvm_arch_tsc_get_attr(vcpu, attr);
break;
case KVM_SET_DEVICE_ATTR:
r = kvm_arch_tsc_set_attr(vcpu, attr);
break;
default:
BUG();
}
return r;
}and then:
case KVM_GET_DEVICE_ATTR: case KVM_HAS_DEVICE_ATTR: case KVM_SET_DEVICE_ATTR: r = kvm_vcpu_ioctl_do_device_attr(vcpu, ioctl, argp); break;
or if we want to plan for the future a bit more:
static int kvm_vcpu_ioctl_do_device_attr(struct kvm_vcpu *vcpu,
unsigned int ioctl, void __user *argp)
{
struct kvm_device_attr attr;
int r;
if (copy_from_user(&attr, argp, sizeof(attr)))
return -EFAULT;
r = -ENXIO;
switch (ioctl) {
case KVM_HAS_DEVICE_ATTR:
if (attr->group != KVM_VCPU_TSC_CTRL)
r = kvm_arch_tsc_has_attr(vcpu, attr);
break;
case KVM_GET_DEVICE_ATTR:
if (attr->group != KVM_VCPU_TSC_CTRL)
r = kvm_arch_tsc_get_attr(vcpu, attr);
break;
case KVM_SET_DEVICE_ATTR:
if (attr->group != KVM_VCPU_TSC_CTRL)
r = kvm_arch_tsc_set_attr(vcpu, attr);
break;
}
return r;
}Quoted 7 lines
arch/x86/kvm/x86.c +1
-0+ }default : r = -EINVAL ; }Signature · 1 line
2.32.0.432.gabb21c7263-goog
PATCH v5 4/13 tools: arch: x86: pull in pvclock headers
In reply to Oliver UptonCopy over approximately clean versions of the pvclock headers into tools. Reconcile headers/symbols missing in tools that are unneeded.
--- tools/arch/x86/include/asm/pvclock-abi.h | 48 +++++++++++ tools/arch/x86/include/asm/pvclock.h | 103 +++++++++++++++++++++++ 2 files changed, 151 insertions(+) create mode 100644 tools/arch/x86/include/asm/pvclock-abi.h create mode 100644 tools/arch/x86/include/asm/pvclock.h
tools/arch/x86/include/asm/pvclock-abi.h new file +48 -0
@@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_PVCLOCK_ABI_H +#define _ASM_X86_PVCLOCK_ABI_H +#ifndef __ASSEMBLY__ + +/* +* These structs MUST NOT be changed. +* They are the ABI between hypervisor and guest OS. +* Both Xen and KVM are using this. +* +* pvclock_vcpu_time_info holds the system time and the tsc timestamp +* of the last update. So the guest can use the tsc delta to get a +* more precise system time. There is one per virtual cpu. +* +* pvclock_wall_clock references the point in time when the system +* time was zero (usually boot time), thus the guest calculates the +* current wall clock by adding the system time. +* +* Protocol for the "version" fields is: hypervisor raises it (making +* it uneven) before it starts updating the fields and raises it again +* (making it even) when it is done. Thus the guest can make sure the +* time values it got are consistent by checking the version before +* and after reading them. +*/ + +struct pvclock_vcpu_time_info { +u32 version; +u32 pad0; +u64 tsc_timestamp; +u64 system_time; +u32 tsc_to_system_mul; +s8 tsc_shift; +u8 flags; +u8 pad[2 ]; +} __attribute__((__packed__ ));/* 32 bytes */ + +struct pvclock_wall_clock { +u32 version; +u32 sec; +u32 nsec; +} __attribute__((__packed__ )); + +#define PVCLOCK_TSC_STABLE_BIT (1 << 0) +#define PVCLOCK_GUEST_STOPPED (1 << 1) +/* PVCLOCK_COUNTS_FROM_ZERO broke ABI and can't be used anymore. */ +#define PVCLOCK_COUNTS_FROM_ZERO (1 << 2) +#endif /* __ASSEMBLY__ */ +#endif /* _ASM_X86_PVCLOCK_ABI_H */
tools/arch/x86/include/asm/pvclock.h new file +103 -0
103 lines not on this page · raw message
Signature · 1 line
2.32.0.432.gabb21c7263-goog
PATCH v5 5/13 selftests: KVM: Add test for KVM_{GET,SET}_CLOCK
In reply to Oliver UptonAdd a selftest for the new KVM clock UAPI that was introduced. Ensure that the KVM clock is consistent between userspace and the guest, and that the difference in realtime will only ever cause the KVM clock to advance forward.
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../testing/selftests/kvm/include/kvm_util.h | 2 + .../selftests/kvm/x86_64/kvm_clock_test.c | 204 ++++++++++++++++++ 4 files changed, 208 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/kvm_clock_test.c
tools/testing/selftests/kvm/.gitignore +1 -0
@@ -11,6 +11,7 @@ /x86_64/emulator_error_test /x86_64/get_cpuid_test /x86_64/get_msr_index_features +/x86_64/kvm_clock_test /x86_64/kvm_pv_test /x86_64/hyperv_clock /x86_64/hyperv_cpuid
tools/testing/selftests/kvm/Makefile +1 -0
@@ -46,6 +46,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/get_cpuid_test TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features +TEST_GEN_PROGS_x86_64 += x86_64/kvm_clock_test TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test TEST_GEN_PROGS_x86_64 += x86_64/mmio_warning_test TEST_GEN_PROGS_x86_64 += x86_64/mmu_role_test
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
@@ -19,6 +19,8 @@#define KVM_DEV_PATH "/dev/kvm"#define KVM_MAX_VCPUS 512 +#define NSEC_PER_SEC 1000000000L +/* * Callers of kvm_util only have an incomplete/opaque description of the * structure kvm_util is using to maintain the state of a VM.
tools/testing/selftests/kvm/x86_64/kvm_clock_test.c new file +204 -0
204 lines not on this page · raw message
Signature · 1 line
2.32.0.432.gabb21c7263-goog
PATCH v5 6/13 selftests: KVM: Fix kvm device helper ioctl assertions
In reply to Oliver UptonThe KVM_CREATE_DEVICE and KVM_{GET,SET}_DEVICE_ATTR ioctls are defined to return a value of zero on success. As such, tighten the assertions in the helper functions to only pass if the return code is zero.
--- tools/testing/selftests/kvm/lib/kvm_util.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tools/testing/selftests/kvm/lib/kvm_util.c +3 -3
@@ -1984,7 +1984,7 @@ int kvm_device_check_attr(int dev_fd, uint32_t group, uint64_t attr) {int ret =_kvm_device_check_attr (dev_fd, group, attr); -TEST_ASSERT (ret >=0 ,"KVM_HAS_DEVICE_ATTR failed, rc: %i errno: %i" , ret, errno); +TEST_ASSERT (!ret,"KVM_HAS_DEVICE_ATTR failed, rc: %i errno: %i" , ret, errno);return ret; }
@@ -2008,7 +2008,7 @@ int kvm_create_device(struct kvm_vm *vm, uint64_t type, bool test) ret =_kvm_create_device (vm, type, test, &fd);if (!test) { -TEST_ASSERT (ret >=0 , +TEST_ASSERT (!ret,"KVM_CREATE_DEVICE IOCTL failed, rc: %i errno: %i" , ret, errno);return fd; }
@@ -2036,7 +2036,7 @@ int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr, {int ret =_kvm_device_access (dev_fd, group, attr, val, write); -TEST_ASSERT (ret >=0 ,"KVM_SET|GET_DEVICE_ATTR IOCTL failed, rc: %i errno: %i" , ret, errno); +TEST_ASSERT (!ret,"KVM_SET|GET_DEVICE_ATTR IOCTL failed, rc: %i errno: %i" , ret, errno);return ret; }
Signature · 1 line
2.32.0.432.gabb21c7263-goog
PATCH v5 7/13 selftests: KVM: Add helpers for vCPU device attributes
In reply to Oliver UptonvCPU file descriptors are abstracted away from test code in KVM selftests, meaning that tests cannot directly access a vCPU's device attributes. Add helpers that tests can use to get at vCPU device attributes.
--- .../testing/selftests/kvm/include/kvm_util.h | 9 +++++ tools/testing/selftests/kvm/lib/kvm_util.c | 38 +++++++++++++++++++ 2 files changed, 47 insertions(+)
tools/testing/selftests/kvm/include/kvm_util.h +9 -0
@@ -240,6 +240,15 @@ int _kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,int kvm_device_access (int dev_fd,uint32_t group,uint64_t attr,void *val,bool write); +int _vcpu_has_device_attr (struct kvm_vm *vm,uint32_t vcpuid,uint32_t group, +uint64_t attr); +int vcpu_has_device_attr (struct kvm_vm *vm,uint32_t vcpuid,uint32_t group, +uint64_t attr); +int _vcpu_access_device_attr (struct kvm_vm *vm,uint32_t vcpuid,uint32_t group, +uint64_t attr,void *val,bool write); +int vcpu_access_device_attr (struct kvm_vm *vm,uint32_t vcpuid,uint32_t group, +uint64_t attr,void *val,bool write); +const char *exit_reason_str (unsigned int exit_reason);void virt_pgd_alloc (struct kvm_vm *vm);
tools/testing/selftests/kvm/lib/kvm_util.c +38 -0
@@ -2040,6 +2040,44 @@ int kvm_device_access(int dev_fd, uint32_t group, uint64_t attr,return ret; } +int _vcpu_has_device_attr (struct kvm_vm *vm,uint32_t vcpuid,uint32_t group, +uint64_t attr) +{ +struct vcpu *vcpu =vcpu_find (vm, vcpuid); + +TEST_ASSERT (vcpu,"nonexistent vcpu id: %d" , vcpuid); + +return _kvm_device_check_attr (vcpu->fd, group, attr); +} + +int vcpu_has_device_attr (struct kvm_vm *vm,uint32_t vcpuid,uint32_t group, +uint64_t attr) +{ +int ret =_vcpu_has_device_attr (vm, vcpuid, group, attr); + +TEST_ASSERT (!ret,"KVM_HAS_DEVICE_ATTR IOCTL failed, rc: %i errno: %i" , ret, errno); +return ret; +} + +int _vcpu_access_device_attr (struct kvm_vm *vm,uint32_t vcpuid,uint32_t group, +uint64_t attr,void *val,bool write) +{ +struct vcpu *vcpu =vcpu_find (vm, vcpuid); + +TEST_ASSERT (vcpu,"nonexistent vcpu id: %d" , vcpuid); + +return _kvm_device_access (vcpu->fd, group, attr, val, write); +} + +int vcpu_access_device_attr (struct kvm_vm *vm,uint32_t vcpuid,uint32_t group, +uint64_t attr,void *val,bool write) +{ +int ret =_vcpu_access_device_attr (vm, vcpuid, group, attr, val, write); + +TEST_ASSERT (!ret,"KVM_SET|GET_DEVICE_ATTR IOCTL failed, rc: %i errno: %i" , ret, errno); +return ret; +} +/* * VM Dump *
Signature · 1 line
2.32.0.432.gabb21c7263-goog
PATCH v5 8/13 selftests: KVM: Introduce system counter offset test
In reply to Oliver UptonIntroduce a KVM selftest to verify that userspace manipulation of the TSC (via the new vCPU attribute) results in the correct behavior within the guest.
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../kvm/system_counter_offset_test.c | 132 ++++++++++++++++++ 3 files changed, 134 insertions(+) create mode 100644 tools/testing/selftests/kvm/system_counter_offset_test.c
tools/testing/selftests/kvm/.gitignore +1 -0
@@ -52,3 +52,4 @@ /set_memory_region_test /steal_time /kvm_binary_stats_test +/system_counter_offset_test
tools/testing/selftests/kvm/Makefile +1 -0
@@ -85,6 +85,7 @@ TEST_GEN_PROGS_x86_64 += memslot_perf_test TEST_GEN_PROGS_x86_64 += set_memory_region_test TEST_GEN_PROGS_x86_64 += steal_time TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test +TEST_GEN_PROGS_x86_64 += system_counter_offset_test TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
tools/testing/selftests/kvm/system_counter_offset_test.c new file +132 -0
132 lines not on this page · raw message
Signature · 1 line
2.32.0.432.gabb21c7263-goog
PATCH v5 9/13 KVM: arm64: Allow userspace to configure a vCPU's virtual offset
In reply to Oliver UptonAdd a new vCPU attribute that allows userspace to directly manipulate the virtual counter-timer offset. Exposing such an interface allows for the precise migration of guest virtual counter-timers, as it is an indepotent interface.
Uphold the existing behavior of writes to CNTVOFF_EL2 for this new interface, wherein a write to a single vCPU is broadcasted to all vCPUs within a VM.
--- Documentation/virt/kvm/devices/vcpu.rst | 22 ++++++++ arch/arm64/include/uapi/asm/kvm.h | 1 + arch/arm64/kvm/arch_timer.c | 68 ++++++++++++++++++++++++- 3 files changed, 89 insertions(+), 2 deletions(-)
Documentation/virt/kvm/devices/vcpu.rst +22 -0
@@ -139,6 +139,28 @@ configured values on other VCPUs. Userspace should configure the interrupt numbers on at least one VCPU after creating all VCPUs and before running any VCPUs. +2.2. ATTRIBUTE: KVM_ARM_VCPU_TIMER_OFFSET_VTIMER +------------------------------------------------ + +:Parameters: Pointer to a 64-bit unsigned counter-timer offset. + +Returns: + + ======= ====================================== + -EFAULT Error reading/writing the provided + parameter address + -ENXIO Attribute not supported + ======= ====================================== + +Specifies the guest's virtual counter-timer offset from the host's +virtual counter. The guest's virtual counter is then derived by +the following equation: + + guest_cntvct = host_cntvct - KVM_ARM_VCPU_TIMER_OFFSET_VTIMER + +KVM does not allow the use of varying offset values for different vCPUs; +the last written offset value will be broadcasted to all vCPUs in a VM. + 3. GROUP: KVM_ARM_VCPU_PVTIME_CTRL ==================================
arch/arm64/include/uapi/asm/kvm.h +1 -0
@@ -365,6 +365,7 @@ struct kvm_arm_copy_mte_tags {#define KVM_ARM_VCPU_TIMER_CTRL 1#define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0#define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1 +#define KVM_ARM_VCPU_TIMER_OFFSET_VTIMER 2#define KVM_ARM_VCPU_PVTIME_CTRL 2#define KVM_ARM_VCPU_PVTIME_IPA 0
arch/arm64/kvm/arch_timer.c +66 -2
91 lines not on this page · raw message
Signature · 1 line
2.32.0.432.gabb21c7263-goog
Marc Zyngier
maintainerRe: [PATCH v5 09/13] KVM: arm64: Allow userspace to configure a vCPU's virtual offset
Quoted 34 lines · Oliver Upton
Add a new vCPU attribute that allows userspace to directly manipulate the virtual counter-timer offset. Exposing such an interface allows for the precise migration of guest virtual counter-timers, as it is an indepotent interface.
Uphold the existing behavior of writes to
CNTVOFF_EL2for this new interface, wherein a write to a single vCPU is broadcasted to all vCPUs within a VM.Reviewed-by Andrew Jones <[email protected]>Signed-off-by Oliver Upton <[email protected]>--- Documentation/virt/kvm/devices/vcpu.rst | 22 ++++++++ arch/arm64/include/uapi/asm/kvm.h | 1 + arch/arm64/kvm/arch_timer.c | 68 ++++++++++++++++++++++++- 3 files changed, 89 insertions(+), 2 deletions(-)Documentation/virt/kvm/devices/vcpu.rst +8
-0@@ -139,6 +139,28 @@ configured values on other VCPUs. Userspace should configure the interrupt numbers on at least one VCPU after creating all VCPUs and before running any VCPUs. +2.2. ATTRIBUTE: KVM_ARM_VCPU_TIMER_OFFSET_VTIMER +------------------------------------------------ + +:Parameters: Pointer to a 64-bit unsigned counter-timer offset. + +Returns: + + ======= ======================================
Documentation/virt/kvm/devices/vcpu.rst configured values on other VCPUs. Userspace should configure the interrupt · line 159
Quoted 10 lines
Documentation/virt/kvm/devices/vcpu.rst +10
-0+ -EFAULT Error reading/writing the provided + parameter address + -ENXIO Attribute not supported + ======= ====================================== + +Specifies the guest's virtual counter-timer offset from the host's +virtual counter. The guest's virtual counter is then derived by +the following equation: + + guest_cntvct = host_cntvct - KVM_ARM_VCPU_TIMER_OFFSET_VTIMER
I still have a problem with this, specially as you later introduce a physical timer offset. My gut feeling is that the virtual offset should be relative to the physical counter *of the guest*, and not that of the host. The physical offset should be the only one that is relative to the host. Anything else should be deriving from it.
If you don't set the ptimer offset, then the two definitions are strictly identical. It will also match the definition of a userspace-visible CNTVOFF_EL2 with NV, which is strictly relative to the guest view of the physical counter.
Documentation/virt/kvm/devices/vcpu.rst configured values on other VCPUs. Userspace should configure the interrupt · line 163
Quoted 4 lines
Documentation/virt/kvm/devices/vcpu.rst +4
-0+ +KVM does not allow the use of varying offset values for different vCPUs; +the last written offset value will be broadcasted to all vCPUs in a VM. +
Please document the effects of this attribute w.r.t. writing CNTVCT_EL0 from userspace.
Quoted 34 lines
Documentation/virt/kvm/devices/vcpu.rst +0
-03. GROUP: KVM_ARM_VCPU_PVTIME_CTRL ==================================arch/arm64/include/uapi/asm/kvm.h +1
-0@@ -365,6 +365,7 @@ struct kvm_arm_copy_mte_tags {#define KVM_ARM_VCPU_TIMER_CTRL 1#define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0#define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1 +#define KVM_ARM_VCPU_TIMER_OFFSET_VTIMER 2#define KVM_ARM_VCPU_PVTIME_CTRL 2#define KVM_ARM_VCPU_PVTIME_IPA 0arch/arm64/kvm/arch_timer.c +2
-2@@ -1305,7 +1305,7 @@ static void set_timer_irqs(struct kvm *kvm, int vtimer_irq, int ptimer_irq) } } -int kvm_arm_timer_set_attr (struct kvm_vcpu *vcpu,struct kvm_device_attr *attr) +int kvm_arm_timer_set_attr_irq (struct kvm_vcpu *vcpu,struct kvm_device_attr *attr) {int __user *uaddr = (int __user *)(long )attr->addr;struct arch_timer_context *vtimer =vcpu_vtimer (vcpu);@@ -1338,7 +1338,39 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)return 0 ; } -int kvm_arm_timer_get_attr (struct kvm_vcpu *vcpu,struct kvm_device_attr *attr) +int kvm_arm_timer_set_attr_offset (struct kvm_vcpu *vcpu,struct kvm_device_attr *attr)
arch/arm64/kvm/arch_timer.c int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) · line 1351
Quoted 10 lines
arch/arm64/kvm/arch_timer.c +10
-0+{ +u64 __user *uaddr = (u64 __user *)(long )attr->addr; +u64 offset; + +if (get_user (offset, uaddr)) +return -EFAULT ; + +switch (attr->attr) { +case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER : +update_vtimer_cntvoff (vcpu, offset);
Probably not a good idea if the timer is already enabled on any of the CPUs (we probably already have that problem, so let's fix it once and for all).
Quoted 31 lines
arch/arm64/kvm/arch_timer.c +24
-0+break ; +default : +return -ENXIO ; + } + +return 0 ; +} + +int kvm_arm_timer_set_attr (struct kvm_vcpu *vcpu,struct kvm_device_attr *attr) +{ +switch (attr->attr) { +case KVM_ARM_VCPU_TIMER_IRQ_VTIMER : +case KVM_ARM_VCPU_TIMER_IRQ_PTIMER : +return kvm_arm_timer_set_attr_irq (vcpu, attr); +case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER : +return kvm_arm_timer_set_attr_offset (vcpu, attr); + } + +return -ENXIO ; +} + +int kvm_arm_timer_get_attr_irq (struct kvm_vcpu *vcpu,struct kvm_device_attr *attr) {int __user *uaddr = (int __user *)(long )attr->addr;struct arch_timer_context *timer;@@ -1359,11 +1391,43 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)return put_user (irq, uaddr); } +int kvm_arm_timer_get_attr_offset (struct kvm_vcpu *vcpu,struct kvm_device_attr *attr) +{
arch/arm64/kvm/arch_timer.c int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) · line 1405
Quoted 10 lines
arch/arm64/kvm/arch_timer.c +10
-0+u64 __user *uaddr = (u64 __user *)(long )attr->addr; +struct arch_timer_context *timer; +u64 offset; + +switch (attr->attr) { +case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER : + timer =vcpu_vtimer (vcpu); +break ; +default : +return -ENXIO ;
What is the rational for retrieving this offset the first place? I don't dislike the symmetry, but we already have an architectural way of getting it (read the counter registers).
Quoted 18 lines
arch/arm64/kvm/arch_timer.c +18
-0+ } + + offset =timer_get_offset (timer); +return put_user (offset, uaddr); +} + +int kvm_arm_timer_get_attr (struct kvm_vcpu *vcpu,struct kvm_device_attr *attr) +{ +switch (attr->attr) { +case KVM_ARM_VCPU_TIMER_IRQ_VTIMER : +case KVM_ARM_VCPU_TIMER_IRQ_PTIMER : +return kvm_arm_timer_get_attr_irq (vcpu, attr); +case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER : +return kvm_arm_timer_get_attr_offset (vcpu, attr); + } + +return -ENXIO ; +}
arch/arm64/kvm/arch_timer.c int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) · line 1433
Quoted 10 lines
arch/arm64/kvm/arch_timer.c +2
-0+int kvm_arm_timer_has_attr (struct kvm_vcpu *vcpu,struct kvm_device_attr *attr) {switch (attr->attr) {case KVM_ARM_VCPU_TIMER_IRQ_VTIMER :case KVM_ARM_VCPU_TIMER_IRQ_PTIMER : +case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER :return 0 ; }
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v5 09/13] KVM: arm64: Allow userspace to configure a vCPU's virtual offset
Quoted 54 lines · Marc Zyngier
Quoted 34 lines · Oliver Upton
Add a new vCPU attribute that allows userspace to directly manipulate the virtual counter-timer offset. Exposing such an interface allows for the precise migration of guest virtual counter-timers, as it is an indepotent interface.
Uphold the existing behavior of writes to
CNTVOFF_EL2for this new interface, wherein a write to a single vCPU is broadcasted to all vCPUs within a VM.Reviewed-by Andrew Jones <[email protected]>Signed-off-by Oliver Upton <[email protected]>--- Documentation/virt/kvm/devices/vcpu.rst | 22 ++++++++ arch/arm64/include/uapi/asm/kvm.h | 1 + arch/arm64/kvm/arch_timer.c | 68 ++++++++++++++++++++++++- 3 files changed, 89 insertions(+), 2 deletions(-)Documentation/virt/kvm/devices/vcpu.rst +8
-0@@ -139,6 +139,28 @@ configured values on other VCPUs. Userspace should configure the interrupt numbers on at least one VCPU after creating all VCPUs and before running any VCPUs. +2.2. ATTRIBUTE: KVM_ARM_VCPU_TIMER_OFFSET_VTIMER +------------------------------------------------ + +:Parameters: Pointer to a 64-bit unsigned counter-timer offset. + +Returns: + + ======= ======================================Documentation/virt/kvm/devices/vcpu.rst configured values on other VCPUs. Userspace should configure the interrupt · line 159
Quoted 10 lines
Documentation/virt/kvm/devices/vcpu.rst +10
-0+ -EFAULT Error reading/writing the provided + parameter address + -ENXIO Attribute not supported + ======= ====================================== + +Specifies the guest's virtual counter-timer offset from the host's +virtual counter. The guest's virtual counter is then derived by +the following equation: + + guest_cntvct = host_cntvct - KVM_ARM_VCPU_TIMER_OFFSET_VTIMERI still have a problem with this, specially as you later introduce a physical timer offset. My gut feeling is that the virtual offset should be relative to the physical counter *of the guest*, and not that of the host. The physical offset should be the only one that is relative to the host. Anything else should be deriving from it.
Quoted 4 lines
If you don't set the ptimer offset, then the two definitions are strictly identical. It will also match the definition of a userspace-visible
CNTVOFF_EL2with NV, which is strictly relative to the guest view of the physical counter.
Yeah, this sounds good to me. I very much like the idea of maintaining exactly one offset from the host to the guest. So long as users are fine with paying the cost of an emulated physical counter-timer on non-ECV hosts. That said, a non-NV guest shouldn't be using the physical counter in the first place..
Quoted 9 lines
Documentation/virt/kvm/devices/vcpu.rst configured values on other VCPUs. Userspace should configure the interrupt · line 163
Quoted 5 lines
Documentation/virt/kvm/devices/vcpu.rst +4
-0+ +KVM does not allow the use of varying offset values for different vCPUs; +the last written offset value will be broadcasted to all vCPUs in a VM. +Please document the effects of this attribute w.r.t. writing
CNTVCT_EL0from userspace.
Good idea.
Quoted 8 lines
Quoted 8 lines
Documentation/virt/kvm/devices/vcpu.rst +7
-1-int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) +int kvm_arm_timer_set_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) +{ + u64 __user *uaddr = (u64 __user *)(long)attr->addr; + u64 offset; + + if (get_user(offset, uaddr)) + return -EFAULT;
Quoted 8 lines
Documentation/virt/kvm/devices/vcpu.rst
Quoted 4 lines
Documentation/virt/kvm/devices/vcpu.rst +4
-0+ + switch (attr->attr) { + case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER: + update_vtimer_cntvoff(vcpu, offset);Probably not a good idea if the timer is already enabled on any of the CPUs (we probably already have that problem, so let's fix it once and for all).
hmm... would this cause any issues to enforce ordering on an existing UAPI? If I understand the suggestion correctly, we will refuse to write the counter offset for a VM with an active timer.
If that is the case, then when we migrate a guest the VMM would have to be very deliberate about the order in which it restores registers, no?
Quoted 8 lines
Quoted 8 lines
Documentation/virt/kvm/devices/vcpu.rst +8
-0+int kvm_arm_timer_get_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) +{ + u64 __user *uaddr = (u64 __user *)(long)attr->addr; + struct arch_timer_context *timer; + u64 offset; + + switch (attr->attr) { + case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER:
Quoted 8 lines
Documentation/virt/kvm/devices/vcpu.rst
Quoted 4 lines
Documentation/virt/kvm/devices/vcpu.rst +4
-0+ timer = vcpu_vtimer(vcpu); + break; + default: + return -ENXIO;What is the rational for retrieving this offset the first place? I don't dislike the symmetry, but we already have an architectural way of getting it (read the counter registers).
I don't believe this is necessary any more.
The reason that I had exposed the virtual counter offset as a device attribute was to separate VMM and guest manipulation of the virtual counter. A VMM migrating an EL2 guest would likely want to adjust the vtimer according to the difference in virtual counters between two hosts without changing any guest-visible sysregs. However, if we go with your suggestion above, the hypervisor would only ever need to poke a physical offset attribute to make transparent changes to *both* counters.
So, I suppose this is what I'm proposing: treat VMM writes to CNTVOFF_EL2 the same as guest writes. For CNTPOFF_EL2, we do a special dance; guest writes to CNTPOFF_EL2 will be visible in the register _and_ change the value KVM writes to CNTPOFF_EL2 in hardware. Host writes to a physical offset device attribute will cause KVM to change the hardware value of CNTPOFF_EL2, but not update the guest-visible register value. This way, a guest can be transparently migrated between hosts with different counters.
Quoted 36 lines
Quoted 19 lines
Documentation/virt/kvm/devices/vcpu.rst +17
-0+ } + + offset = timer_get_offset(timer); + return put_user(offset, uaddr); +} + +int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) +{ + switch (attr->attr) { + case KVM_ARM_VCPU_TIMER_IRQ_VTIMER: + case KVM_ARM_VCPU_TIMER_IRQ_PTIMER: + return kvm_arm_timer_get_attr_irq(vcpu, attr); + case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER: + return kvm_arm_timer_get_attr_offset(vcpu, attr); + } + + return -ENXIO;Documentation/virt/kvm/devices/vcpu.rst
Quoted 10 lines
Documentation/virt/kvm/devices/vcpu.rst +3
-0+} + int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) { switch (attr->attr) { case KVM_ARM_VCPU_TIMER_IRQ_VTIMER: case KVM_ARM_VCPU_TIMER_IRQ_PTIMER: + case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER: return 0; }Thanks,
M.--
Without deviation from the norm, progress is not possible.
PATCH v5 10/13 selftests: KVM: Add support for aarch64 to system_counter_offset_test
In reply to Oliver UptonKVM/arm64 now allows userspace to adjust the guest virtual counter-timer via a vCPU device attribute. Test that changes to the virtual counter-timer offset result in the correct view being presented to the guest.
--- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/aarch64/processor.h | 12 +++++ .../kvm/system_counter_offset_test.c | 54 ++++++++++++++++++- 3 files changed, 66 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/Makefile +1 -0
@@ -98,6 +98,7 @@ TEST_GEN_PROGS_aarch64 += kvm_page_table_test TEST_GEN_PROGS_aarch64 += set_memory_region_test TEST_GEN_PROGS_aarch64 += steal_time TEST_GEN_PROGS_aarch64 += kvm_binary_stats_test +TEST_GEN_PROGS_aarch64 += system_counter_offset_test TEST_GEN_PROGS_s390x = s390x/memop TEST_GEN_PROGS_s390x += s390x/resets
tools/testing/selftests/kvm/include/aarch64/processor.h +12 -0
@@ -129,4 +129,16 @@ void vm_install_sync_handler(struct kvm_vm *vm,#define isb () asm volatile("isb" : : : "memory") +static inline uint64_t read_cntvct_ordered (void ) +{ +uint64_t r; + + __asm__ __volatile__("isb\n\t" +"mrs %0, cntvct_el0\n\t" +"isb\n\t" + :"=r" (r)); + +return r; +} +#endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/system_counter_offset_test.c +53 -1
60 lines not on this page · raw message
Signature · 1 line
2.32.0.432.gabb21c7263-goog
PATCH v5 11/13 KVM: arm64: Provide userspace access to the physical counter offset
In reply to Oliver UptonPresently, KVM provides no facilities for correctly migrating a guest that depends on the physical counter-timer. While most guests (barring NV, of course) should not depend on the physical counter-timer, an operator may still wish to provide a consistent view of the physical counter-timer across migrations.
Provide userspace with a new vCPU attribute to modify the guest physical counter-timer offset. Since the base architecture doesn't provide a physical counter-timer offset register, emulate the correct behavior by trapping accesses to the physical counter-timer whenever the offset value is non-zero.
Uphold the same behavior as CNTVOFF_EL2 and broadcast the physical offset to all vCPUs whenever written. This guarantees that the counter-timer we provide the guest remains architectural, wherein all views of the counter-timer are consistent across vCPUs. Reconfigure timer traps for VHE on every guest entry, as different VMs will now have different traps enabled. Enable physical counter traps for nVHE whenever the offset is nonzero (we already trap physical timer registers in nVHE).
FEAT_ECV provides a guest physical counter-timer offset register (CNTPOFF_EL2), but ECV-enabled hardware is nonexistent at the time of writing so support for it was elided for the sake of the author :)
--- Documentation/virt/kvm/devices/vcpu.rst | 22 +++++++ arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/asm/kvm_hyp.h | 2 - arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 1 + arch/arm64/kvm/arch_timer.c | 72 ++++++++++++++--------- arch/arm64/kvm/arm.c | 4 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 23 ++++++++ arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 2 - arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +++---- arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 +++++++++ include/kvm/arm_arch_timer.h | 2 - 13 files changed, 158 insertions(+), 46 deletions(-) create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h
Documentation/virt/kvm/devices/vcpu.rst +22 -0
@@ -161,6 +161,28 @@ the following equation: KVM does not allow the use of varying offset values for different vCPUs; the last written offset value will be broadcasted to all vCPUs in a VM. +2.3. ATTRIBUTE: KVM_ARM_VCPU_TIMER_OFFSET_PTIMER +------------------------------------------------ + +:Parameters: Pointer to a 64-bit unsigned counter-timer offset. + +Returns: + + ======= ====================================== + -EFAULT Error reading/writing the provided + parameter address + -ENXIO Attribute not supported + ======= ====================================== + +Specifies the guest's physical counter-timer offset from the host's +virtual counter. The guest's physical counter is then derived by +the following equation: + + guest_cntpct = host_cntvct - KVM_ARM_VCPU_TIMER_OFFSET_PTIMER + +KVM does not allow the use of varying offset values for different vCPUs; +the last written offset value will be broadcasted to all vCPUs in a VM. + 3. GROUP: KVM_ARM_VCPU_PVTIME_CTRL ==================================
arch/arm64/include/asm/kvm_host.h +1 -0
@@ -204,6 +204,7 @@ enum vcpu_sysreg {SP_EL1 ,SPSR_EL1 , +CNTPOFF_EL2 ,CNTVOFF_EL2 ,CNTV_CVAL_EL0 , CNTV_CTL_EL0,
arch/arm64/include/asm/kvm_hyp.h +0 -2
@@ -65,10 +65,8 @@ void __vgic_v3_save_aprs(struct vgic_v3_cpu_if *cpu_if);void __vgic_v3_restore_aprs (struct vgic_v3_cpu_if *cpu_if);int __vgic_v3_perform_cpuif_access (struct kvm_vcpu *vcpu); -#ifdef __KVM_NVHE_HYPERVISOR__void __timer_enable_traps (struct kvm_vcpu *vcpu);void __timer_disable_traps (struct kvm_vcpu *vcpu); -#endif #ifdef __KVM_NVHE_HYPERVISOR__void __sysreg_save_state_nvhe (struct kvm_cpu_context *ctxt);
arch/arm64/include/asm/sysreg.h +1 -0
@@ -505,6 +505,7 @@#define SYS_AMEVCNTR0_MEM_STALL SYS_AMEVCNTR0_EL0(3)#define SYS_CNTFRQ_EL0 sys_reg(3, 3, 14, 0, 0) +#define SYS_CNTPCT_EL0 sys_reg(3, 3, 14, 0, 1)#define SYS_CNTP_TVAL_EL0 sys_reg(3, 3, 14, 2, 0)#define SYS_CNTP_CTL_EL0 sys_reg(3, 3, 14, 2, 1)
arch/arm64/include/uapi/asm/kvm.h +1 -0
@@ -366,6 +366,7 @@ struct kvm_arm_copy_mte_tags {#define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0#define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1#define KVM_ARM_VCPU_TIMER_OFFSET_VTIMER 2 +#define KVM_ARM_VCPU_TIMER_OFFSET_PTIMER 3#define KVM_ARM_VCPU_PVTIME_CTRL 2#define KVM_ARM_VCPU_PVTIME_IPA 0
arch/arm64/kvm/arch_timer.c +45 -27
149 lines not on this page · raw message
arch/arm64/kvm/arm.c +1 -3
@@ -1558,9 +1558,7 @@ static void cpu_hyp_reinit(void)cpu_hyp_reset (); -if (is_kernel_in_hyp_mode ()) -kvm_timer_init_vhe (); -else +if (!is_kernel_in_hyp_mode ())cpu_init_hyp_mode ();cpu_set_hyp_vector ();
arch/arm64/kvm/hyp/include/hyp/switch.h +23 -0
@@ -8,6 +8,7 @@#define __ARM64_KVM_HYP_SWITCH_H__#include <hyp/adjust_pc.h> +#include <hyp/timer-sr.h> #include <linux/arm-smccc.h> #include <linux/kvm_host.h>
@@ -113,6 +114,8 @@ static inline void ___activate_traps(struct kvm_vcpu *vcpu)if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN ) && (hcr &HCR_VSE ))write_sysreg_s (vcpu->arch.vsesr_el2,SYS_VSESR_EL2 ); + +__timer_enable_traps (vcpu); }static inline void ___deactivate_traps (struct kvm_vcpu *vcpu)
@@ -127,6 +130,8 @@ static inline void ___deactivate_traps(struct kvm_vcpu *vcpu) vcpu->arch.hcr_el2 &= ~HCR_VSE ; vcpu->arch.hcr_el2 |=read_sysreg (hcr_el2) &HCR_VSE ; } + +__timer_disable_traps (vcpu); }static inline bool __translate_far_to_hpfar (u64 far,u64 *hpfar)
@@ -405,6 +410,21 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu)return true; } +static inline bool __hyp_handle_counter (struct kvm_vcpu *vcpu) +{ +u32 sysreg =esr_sys64_to_sysreg (kvm_vcpu_get_esr (vcpu)); +int rt =kvm_vcpu_sys_get_rt (vcpu); +u64 rv; + +if (sysreg !=SYS_CNTPCT_EL0 ) +return false; + + rv =__timer_read_cntpct (vcpu); +vcpu_set_reg (vcpu, rt, rv); +__kvm_skip_instr (vcpu); +return true; +} +/* * Return true when we were able to fixup the guest exit and should return to * the guest, false when we should restore the host state and return to the
@@ -439,6 +459,9 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)if (*exit_code !=ARM_EXCEPTION_TRAP ) goto exit; +if (__hyp_handle_counter (vcpu)) + goto guest; +if (cpus_have_final_cap (ARM64_WORKAROUND_CAVIUM_TX2_219_TVM ) &&kvm_vcpu_trap_get_class (vcpu) == ESR_ELx_EC_SYS64 &&handle_tx2_tvm (vcpu))
arch/arm64/kvm/hyp/include/hyp/timer-sr.h new file +26 -0
@@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* Copyright (C) 2021 Google LLC +* Author: Oliver Upton <[email protected]> +*/ + +#ifndef __ARM64_KVM_HYP_TIMER_SR_H__ +#define __ARM64_KVM_HYP_TIMER_SR_H__ + +#include <linux/compiler.h> +#include <linux/kvm_host.h> + +#include <asm/kvm_asm.h> +#include <asm/kvm_hyp.h> + +static inline bool __timer_physical_emulation_required (struct kvm_vcpu *vcpu) +{ +return __vcpu_sys_reg (vcpu,CNTPOFF_EL2 ); +} + +static inline u64 __timer_read_cntpct (struct kvm_vcpu *vcpu) +{ +return read_sysreg (cntpct_el0) -__vcpu_sys_reg (vcpu,CNTPOFF_EL2 ); +} + +#endif /* __ARM64_KVM_HYP_TIMER_SR_H__ */
arch/arm64/kvm/hyp/nvhe/switch.c +0 -2
@@ -217,7 +217,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)__activate_traps (vcpu);__hyp_vgic_restore_state (vcpu); -__timer_enable_traps (vcpu);__debug_switch_to_guest (vcpu);
@@ -230,7 +229,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)__sysreg_save_state_nvhe (guest_ctxt);__sysreg32_save_state (vcpu); -__timer_disable_traps (vcpu);__hyp_vgic_save_state (vcpu);__deactivate_traps (vcpu);
arch/arm64/kvm/hyp/nvhe/timer-sr.c +11 -10
@@ -9,16 +9,13 @@#include <linux/kvm_host.h> #include <asm/kvm_hyp.h> +#include <hyp/timer-sr.h> void __kvm_timer_set_cntvoff (u64 cntvoff) {write_sysreg (cntvoff, cntvoff_el2); } -/* -* Should only be called on non-VHE systems. -* VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). -*/ void __timer_disable_traps (struct kvm_vcpu *vcpu) {u64 val;
@@ -29,20 +26,24 @@ void __timer_disable_traps(struct kvm_vcpu *vcpu)write_sysreg (val, cnthctl_el2); } -/* -* Should only be called on non-VHE systems. -* VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). -*/ void __timer_enable_traps (struct kvm_vcpu *vcpu) {u64 val;/* * Disallow physical timer access for the guest -* Physical counter access is allowed */ val =read_sysreg (cnthctl_el2); val &= ~CNTHCTL_EL1PCEN ; - val |=CNTHCTL_EL1PCTEN ; + +/* +* Disallow physical counter access for the guest if offsetting is +* requested. +*/ +if (__timer_physical_emulation_required (vcpu)) + val &= ~CNTHCTL_EL1PCTEN ; +else + val |=CNTHCTL_EL1PCTEN ; +write_sysreg (val, cnthctl_el2); }
arch/arm64/kvm/hyp/vhe/timer-sr.c +27 -0
@@ -4,9 +4,36 @@* Author: Marc Zyngier <[email protected]> */ +#include <asm/kvm_host.h> #include <asm/kvm_hyp.h> +#include <hyp/timer-sr.h> void __kvm_timer_set_cntvoff (u64 cntvoff) {write_sysreg (cntvoff, cntvoff_el2); } + +void __timer_enable_traps (struct kvm_vcpu *vcpu) +{ +/* When HCR_EL2.E2H == 1, EL1PCEN nad EL1PCTEN are shifted by 10 */ +u32 cnthctl_shift =10 ; +u64 val, mask; + + mask =CNTHCTL_EL1PCEN << cnthctl_shift; + mask |=CNTHCTL_EL1PCTEN << cnthctl_shift; + + val =read_sysreg (cnthctl_el2); + +/* +* VHE systems allow the guest direct access to the EL1 physical +* timer/counter if offsetting isn't requested. +*/ +if (__timer_physical_emulation_required (vcpu)) + val &= ~mask; +else + val |= mask; + +write_sysreg (val, cnthctl_el2); +} + +void __timer_disable_traps (struct kvm_vcpu *vcpu) {}
include/kvm/arm_arch_timer.h +0 -2
@@ -83,8 +83,6 @@ u64 kvm_phys_timer_read(void);void kvm_timer_vcpu_load (struct kvm_vcpu *vcpu);void kvm_timer_vcpu_put (struct kvm_vcpu *vcpu); -void kvm_timer_init_vhe (void ); -bool kvm_arch_timer_get_input_level (int vintid);#define vcpu_timer (v) (&(v)->arch.timer_cpu)
Signature · 1 line
2.32.0.432.gabb21c7263-goog
Marc Zyngier
maintainerRe: [PATCH v5 11/13] KVM: arm64: Provide userspace access to the physical counter offset
Quoted 22 lines · Oliver Upton
Presently, KVM provides no facilities for correctly migrating a guest that depends on the physical counter-timer. While most guests (barring NV, of course) should not depend on the physical counter-timer, an operator may still wish to provide a consistent view of the physical counter-timer across migrations.
Provide userspace with a new vCPU attribute to modify the guest physical counter-timer offset. Since the base architecture doesn't provide a physical counter-timer offset register, emulate the correct behavior by trapping accesses to the physical counter-timer whenever the offset value is non-zero.
Uphold the same behavior as
CNTVOFF_EL2and broadcast the physical offset to all vCPUs whenever written. This guarantees that the counter-timer we provide the guest remains architectural, wherein all views of the counter-timer are consistent across vCPUs. Reconfigure timer traps for VHE on every guest entry, as different VMs will now have different traps enabled. Enable physical counter traps for nVHE whenever the offset is nonzero (we already trap physical timer registers in nVHE).
Quoted 3 lines
FEAT_ECVprovides a guest physical counter-timer offset register (CNTPOFF_EL2), but ECV-enabled hardware is nonexistent at the time of writing so support for it was elided for the sake of the author :)
You seem to have done most the work for that, and there are SW models out there that implement ECV. If anything, the ECV support should come first, and its emulation only as a poor man's workaround.
It is also good to show silicon vendors that they suck at providing useful features, and pointing them to the code that would use these features *is* an incentive.
Quoted 145 lines
Reviewed-by Andrew Jones <[email protected]>Signed-off-by Oliver Upton <[email protected]>--- Documentation/virt/kvm/devices/vcpu.rst | 22 +++++++ arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/asm/kvm_hyp.h | 2 - arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 1 + arch/arm64/kvm/arch_timer.c | 72 ++++++++++++++--------- arch/arm64/kvm/arm.c | 4 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 23 ++++++++ arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 2 - arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +++---- arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 +++++++++ include/kvm/arm_arch_timer.h | 2 - 13 files changed, 158 insertions(+), 46 deletions(-) create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.hDocumentation/virt/kvm/devices/vcpu.rst +22
-0@@ -161,6 +161,28 @@ the following equation: KVM does not allow the use of varying offset values for different vCPUs; the last written offset value will be broadcasted to all vCPUs in a VM. +2.3. ATTRIBUTE: KVM_ARM_VCPU_TIMER_OFFSET_PTIMER +------------------------------------------------ + +:Parameters: Pointer to a 64-bit unsigned counter-timer offset. + +Returns: + + ======= ====================================== + -EFAULT Error reading/writing the provided + parameter address + -ENXIO Attribute not supported + ======= ====================================== + +Specifies the guest's physical counter-timer offset from the host's +virtual counter. The guest's physical counter is then derived by +the following equation: + + guest_cntpct = host_cntvct - KVM_ARM_VCPU_TIMER_OFFSET_PTIMER + +KVM does not allow the use of varying offset values for different vCPUs; +the last written offset value will be broadcasted to all vCPUs in a VM. + 3. GROUP: KVM_ARM_VCPU_PVTIME_CTRL ==================================arch/arm64/include/asm/kvm_host.h +1
-0@@ -204,6 +204,7 @@ enum vcpu_sysreg {SP_EL1 ,SPSR_EL1 , +CNTPOFF_EL2 ,CNTVOFF_EL2 ,CNTV_CVAL_EL0 , CNTV_CTL_EL0,arch/arm64/include/asm/kvm_hyp.h +0
-2@@ -65,10 +65,8 @@ void __vgic_v3_save_aprs(struct vgic_v3_cpu_if *cpu_if);void __vgic_v3_restore_aprs (struct vgic_v3_cpu_if *cpu_if);int __vgic_v3_perform_cpuif_access (struct kvm_vcpu *vcpu); -#ifdef __KVM_NVHE_HYPERVISOR__void __timer_enable_traps (struct kvm_vcpu *vcpu);void __timer_disable_traps (struct kvm_vcpu *vcpu); -#endif #ifdef __KVM_NVHE_HYPERVISOR__void __sysreg_save_state_nvhe (struct kvm_cpu_context *ctxt);arch/arm64/include/asm/sysreg.h +1
-0@@ -505,6 +505,7 @@#define SYS_AMEVCNTR0_MEM_STALL SYS_AMEVCNTR0_EL0(3)#define SYS_CNTFRQ_EL0 sys_reg(3, 3, 14, 0, 0) +#define SYS_CNTPCT_EL0 sys_reg(3, 3, 14, 0, 1)#define SYS_CNTP_TVAL_EL0 sys_reg(3, 3, 14, 2, 0)#define SYS_CNTP_CTL_EL0 sys_reg(3, 3, 14, 2, 1)arch/arm64/include/uapi/asm/kvm.h +1
-0@@ -366,6 +366,7 @@ struct kvm_arm_copy_mte_tags {#define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0#define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1#define KVM_ARM_VCPU_TIMER_OFFSET_VTIMER 2 +#define KVM_ARM_VCPU_TIMER_OFFSET_PTIMER 3#define KVM_ARM_VCPU_PVTIME_CTRL 2#define KVM_ARM_VCPU_PVTIME_IPA 0arch/arm64/kvm/arch_timer.c +18
-0@@ -89,7 +89,10 @@ static u64 timer_get_offset(struct arch_timer_context *ctxt)switch (arch_timer_ctx_index (ctxt)) {case TIMER_VTIMER :return __vcpu_sys_reg (vcpu,CNTVOFF_EL2 ); +case TIMER_PTIMER : +return __vcpu_sys_reg (vcpu,CNTPOFF_EL2 );default : +WARN_ONCE (1 ,"unrecognized timer index %ld" ,arch_timer_ctx_index (ctxt));return 0 ; } }@@ -134,6 +137,9 @@ static void timer_set_offset(struct arch_timer_context *ctxt, u64 offset)case TIMER_VTIMER :__vcpu_sys_reg (vcpu,CNTVOFF_EL2 ) = offset;break ; +case TIMER_PTIMER : +__vcpu_sys_reg (vcpu,CNTPOFF_EL2 ) = offset; +break ;default :WARN (offset,"timer %ld\n" ,arch_timer_ctx_index (ctxt)); }@@ -144,9 +150,24 @@ u64 kvm_phys_timer_read(void)return timecounter->cc->read (timecounter->cc); } +static bool timer_emulation_required (struct arch_timer_context *ctx) +{ +enum kvm_arch_timers timer =arch_timer_ctx_index (ctx); + +switch (timer) { +case TIMER_VTIMER : +return false; +case TIMER_PTIMER : +return timer_get_offset (ctx); +default : +WARN_ONCE (1 ,"unrecognized timer index %ld\n" , timer); +return false;
arch/arm64/kvm/arch_timer.c u64 kvm_phys_timer_read(void) · line 173
Quoted 10 lines
arch/arm64/kvm/arch_timer.c +4
-1+ } +} +static void get_timer_map (struct kvm_vcpu *vcpu,struct timer_map *map) { -if (has_vhe ()) { +if (has_vhe () && !timer_emulation_required (vcpu_ptimer (vcpu))) { map->direct_vtimer =vcpu_vtimer (vcpu); map->direct_ptimer =vcpu_ptimer (vcpu); map->emul_ptimer =NULL ;
I really dislike the fallback to !vhe here. I'd rather you special-case the emulated ptimer in the VHE case:
if (has_vhe()) {
map->direct_vtimer = vcpu_vtimer(vcpu);
if (!timer_get_offset(vcpu_ptimer(vcpu)))
map->direct_ptimer = vcpu_ptimer(vcpu);
map->emul_ptimer = NULL;
} else {
map->direct_ptimer = NULL;
map->emul_ptimer = vcpu_ptimer(vcpu);
}
} else {and you can drop the timer_emulation_required() helper which doesn't serve any purpose.
Quoted 159 lines
arch/arm64/kvm/arch_timer.c +23
-26@@ -747,8 +768,9 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)return 0 ; } -/* Make the updates of cntvoff for all vtimer contexts atomic */ -static void update_vtimer_cntvoff (struct kvm_vcpu *vcpu,u64 cntvoff) +/* Make the updates of offset for all timer contexts atomic */ +static void update_timer_offset (struct kvm_vcpu *vcpu, +enum kvm_arch_timers timer,u64 offset) {int i;struct kvm *kvm = vcpu->kvm;@@ -756,16 +778,26 @@ static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)mutex_lock (&kvm->lock);kvm_for_each_vcpu (i, tmp, kvm) -timer_set_offset (vcpu_vtimer (tmp), cntvoff); +timer_set_offset (vcpu_get_timer (tmp, timer), offset);/* * When called from the vcpu create path, the CPU being created is not * included in the loop above, so we just set it here as well. */ -timer_set_offset (vcpu_vtimer (vcpu), cntvoff); +timer_set_offset (vcpu_get_timer (vcpu, timer), offset);mutex_unlock (&kvm->lock); } +static void update_vtimer_cntvoff (struct kvm_vcpu *vcpu,u64 cntvoff) +{ +update_timer_offset (vcpu,TIMER_VTIMER , cntvoff); +} + +static void update_ptimer_cntpoff (struct kvm_vcpu *vcpu,u64 cntpoff) +{ +update_timer_offset (vcpu,TIMER_PTIMER , cntpoff); +} +void kvm_timer_vcpu_init (struct kvm_vcpu *vcpu) {struct arch_timer_cpu *timer =vcpu_timer (vcpu);@@ -1272,28 +1304,6 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)return 0 ; } -/* -* On VHE system, we only need to configure the EL2 timer trap register once, -* not for every world switch. -* The host kernel runs at EL2 with HCR_EL2.TGE == 1, -* and this makes those bits have no effect for the host kernel execution. -*/ -void kvm_timer_init_vhe (void ) -{ -/* When HCR_EL2.E2H ==1, EL1PCEN and EL1PCTEN are shifted by 10 */ -u32 cnthctl_shift =10 ; -u64 val; - -/* -* VHE systems allow the guest direct access to the EL1 physical -* timer/counter. -*/ - val =read_sysreg (cnthctl_el2); - val |= (CNTHCTL_EL1PCEN << cnthctl_shift); - val |= (CNTHCTL_EL1PCTEN << cnthctl_shift); -write_sysreg (val, cnthctl_el2); -} -static void set_timer_irqs (struct kvm *kvm,int vtimer_irq,int ptimer_irq) {struct kvm_vcpu *vcpu;@@ -1350,6 +1360,9 @@ int kvm_arm_timer_set_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attrcase KVM_ARM_VCPU_TIMER_OFFSET_VTIMER :update_vtimer_cntvoff (vcpu, offset);break ; +case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER : +update_ptimer_cntpoff (vcpu, offset); +break ;default :return -ENXIO ; }@@ -1364,6 +1377,7 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)case KVM_ARM_VCPU_TIMER_IRQ_PTIMER :return kvm_arm_timer_set_attr_irq (vcpu, attr);case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER : +case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER :return kvm_arm_timer_set_attr_offset (vcpu, attr); }@@ -1400,6 +1414,8 @@ int kvm_arm_timer_get_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attrswitch (attr->attr) {case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER : timer =vcpu_vtimer (vcpu); +case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER : + timer =vcpu_ptimer (vcpu);break ;default :return -ENXIO ;@@ -1416,6 +1432,7 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)case KVM_ARM_VCPU_TIMER_IRQ_PTIMER :return kvm_arm_timer_get_attr_irq (vcpu, attr);case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER : +case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER :return kvm_arm_timer_get_attr_offset (vcpu, attr); }@@ -1428,6 +1445,7 @@ int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)case KVM_ARM_VCPU_TIMER_IRQ_VTIMER :case KVM_ARM_VCPU_TIMER_IRQ_PTIMER :case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER : +case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER :return 0 ; }arch/arm64/kvm/arm.c +1
-3@@ -1558,9 +1558,7 @@ static void cpu_hyp_reinit(void)cpu_hyp_reset (); -if (is_kernel_in_hyp_mode ()) -kvm_timer_init_vhe (); -else +if (!is_kernel_in_hyp_mode ())cpu_init_hyp_mode ();cpu_set_hyp_vector ();arch/arm64/kvm/hyp/include/hyp/switch.h +5
-0@@ -8,6 +8,7 @@#define __ARM64_KVM_HYP_SWITCH_H__#include <hyp/adjust_pc.h> +#include <hyp/timer-sr.h> #include <linux/arm-smccc.h> #include <linux/kvm_host.h> @@ -113,6 +114,8 @@ static inline void ___activate_traps(struct kvm_vcpu *vcpu)if (cpus_have_final_cap (ARM64_HAS_RAS_EXTN ) && (hcr &HCR_VSE ))write_sysreg_s (vcpu->arch.vsesr_el2,SYS_VSESR_EL2 ); + +__timer_enable_traps (vcpu); }static inline void ___deactivate_traps (struct kvm_vcpu *vcpu)@@ -127,6 +130,8 @@ static inline void ___deactivate_traps(struct kvm_vcpu *vcpu) vcpu->arch.hcr_el2 &= ~HCR_VSE ; vcpu->arch.hcr_el2 |=read_sysreg (hcr_el2) &HCR_VSE ; } + +__timer_disable_traps (vcpu); }static inline bool __translate_far_to_hpfar (u64 far,u64 *hpfar)
arch/arm64/kvm/hyp/include/hyp/switch.h static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu) · line 419
Quoted 10 lines
arch/arm64/kvm/hyp/include/hyp/switch.h +7
-0@@ -405,6 +410,21 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu)return true; } +static inline bool __hyp_handle_counter (struct kvm_vcpu *vcpu) +{ +u32 sysreg =esr_sys64_to_sysreg (kvm_vcpu_get_esr (vcpu)); +int rt =kvm_vcpu_sys_get_rt (vcpu); +u64 rv; + +if (sysreg !=SYS_CNTPCT_EL0 )
That's feels really optimistic. You don't check for the exception class, and pray that the bits will align? ;-)
Quoted 43 lines
arch/arm64/kvm/hyp/include/hyp/switch.h +11
-0+return false; + + rv =__timer_read_cntpct (vcpu); +vcpu_set_reg (vcpu, rt, rv); +__kvm_skip_instr (vcpu); +return true; +} +/* * Return true when we were able to fixup the guest exit and should return to * the guest, false when we should restore the host state and return to the @@ -439,6 +459,9 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)if (*exit_code !=ARM_EXCEPTION_TRAP ) goto exit; +if (__hyp_handle_counter (vcpu)) + goto guest; +if (cpus_have_final_cap (ARM64_WORKAROUND_CAVIUM_TX2_219_TVM ) &&kvm_vcpu_trap_get_class (vcpu) == ESR_ELx_EC_SYS64 &&handle_tx2_tvm (vcpu))arch/arm64/kvm/hyp/include/hyp/timer-sr.h new file +16
-0@@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* Copyright (C) 2021 Google LLC +* Author: Oliver Upton <[email protected]> +*/ + +#ifndef __ARM64_KVM_HYP_TIMER_SR_H__ +#define __ARM64_KVM_HYP_TIMER_SR_H__ + +#include <linux/compiler.h> +#include <linux/kvm_host.h> + +#include <asm/kvm_asm.h> +#include <asm/kvm_hyp.h> + +static inline bool __timer_physical_emulation_required (struct kvm_vcpu *vcpu)
arch/arm64/kvm/hyp/include/hyp/timer-sr.h line 26
Quoted 10 lines
arch/arm64/kvm/hyp/include/hyp/timer-sr.h new file +10
-0+{ +return __vcpu_sys_reg (vcpu,CNTPOFF_EL2 ); +} + +static inline u64 __timer_read_cntpct (struct kvm_vcpu *vcpu) +{ +return read_sysreg (cntpct_el0) -__vcpu_sys_reg (vcpu,CNTPOFF_EL2 ); +} + +#endif /* __ARM64_KVM_HYP_TIMER_SR_H__ */
Please don't add more small includes like this. It makes things hard to read and hides bugs: the counter read can (and will) be speculated, so you definitely need an ISB before the access. Please also look at __arch_counter_get_cntpct() and arch_counter_enforce_ordering().
Quoted 43 lines
arch/arm64/kvm/hyp/nvhe/switch.c +0
-2@@ -217,7 +217,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)__activate_traps (vcpu);__hyp_vgic_restore_state (vcpu); -__timer_enable_traps (vcpu);__debug_switch_to_guest (vcpu);@@ -230,7 +229,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)__sysreg_save_state_nvhe (guest_ctxt);__sysreg32_save_state (vcpu); -__timer_disable_traps (vcpu);__hyp_vgic_save_state (vcpu);__deactivate_traps (vcpu);arch/arm64/kvm/hyp/nvhe/timer-sr.c +1
-4@@ -9,16 +9,13 @@#include <linux/kvm_host.h> #include <asm/kvm_hyp.h> +#include <hyp/timer-sr.h> void __kvm_timer_set_cntvoff (u64 cntvoff) {write_sysreg (cntvoff, cntvoff_el2); } -/* -* Should only be called on non-VHE systems. -* VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). -*/ void __timer_disable_traps (struct kvm_vcpu *vcpu) {u64 val;
arch/arm64/kvm/hyp/nvhe/timer-sr.c void __timer_disable_traps(struct kvm_vcpu *vcpu) · line 35
Quoted 7 lines
arch/arm64/kvm/hyp/nvhe/timer-sr.c +0
-4@@ -29,20 +26,24 @@ void __timer_disable_traps(struct kvm_vcpu *vcpu)write_sysreg (val, cnthctl_el2); } -/* -* Should only be called on non-VHE systems. -* VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). -*/
This comment still applies. this function *is* nVHE specific.
Quoted 51 lines
arch/arm64/kvm/hyp/nvhe/timer-sr.c +10
-2void __timer_enable_traps (struct kvm_vcpu *vcpu) {u64 val;/* * Disallow physical timer access for the guest -* Physical counter access is allowed */ val =read_sysreg (cnthctl_el2); val &= ~CNTHCTL_EL1PCEN ; - val |=CNTHCTL_EL1PCTEN ; + +/* +* Disallow physical counter access for the guest if offsetting is +* requested. +*/ +if (__timer_physical_emulation_required (vcpu)) + val &= ~CNTHCTL_EL1PCTEN ; +else + val |=CNTHCTL_EL1PCTEN ; +write_sysreg (val, cnthctl_el2); }arch/arm64/kvm/hyp/vhe/timer-sr.c +14
-0@@ -4,9 +4,36 @@* Author: Marc Zyngier <[email protected]> */ +#include <asm/kvm_host.h> #include <asm/kvm_hyp.h> +#include <hyp/timer-sr.h> void __kvm_timer_set_cntvoff (u64 cntvoff) {write_sysreg (cntvoff, cntvoff_el2); } + +void __timer_enable_traps (struct kvm_vcpu *vcpu) +{ +/* When HCR_EL2.E2H == 1, EL1PCEN nad EL1PCTEN are shifted by 10 */ +u32 cnthctl_shift =10 ; +u64 val, mask; + + mask =CNTHCTL_EL1PCEN << cnthctl_shift; + mask |=CNTHCTL_EL1PCTEN << cnthctl_shift; + + val =read_sysreg (cnthctl_el2); +
arch/arm64/kvm/hyp/vhe/timer-sr.c line 36
Quoted 10 lines
arch/arm64/kvm/hyp/vhe/timer-sr.c +10
-0+/* +* VHE systems allow the guest direct access to the EL1 physical +* timer/counter if offsetting isn't requested. +*/ +if (__timer_physical_emulation_required (vcpu)) + val &= ~mask; +else + val |= mask; + +write_sysreg (val, cnthctl_el2);
You now pay the price of reprogramming CNTHCTL_EL2 on every entry for VHE, and that's not right. On VHE, it should be enough to perform the programming on vcpu_load() only.
Quoted 8 lines
arch/arm64/kvm/hyp/vhe/timer-sr.c +3
-0+} + +void __timer_disable_traps (struct kvm_vcpu *vcpu) {}include/kvm/arm_arch_timer.h +0
-0
include/kvm/arm_arch_timer.h u64 kvm_phys_timer_read(void); · line 88
Quoted 8 lines
include/kvm/arm_arch_timer.h +0
-2@@ -83,8 +83,6 @@ u64 kvm_phys_timer_read(void);void kvm_timer_vcpu_load (struct kvm_vcpu *vcpu);void kvm_timer_vcpu_put (struct kvm_vcpu *vcpu); -void kvm_timer_init_vhe (void ); -bool kvm_arch_timer_get_input_level (int vintid);#define vcpu_timer (v) (&(v)->arch.timer_cpu)
Overall, this patch needs a bit of splitting up (userspace interface, HV rework), re-optimise VHE, and it *definitely* wants the ECV support for the physical timer.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v5 11/13] KVM: arm64: Provide userspace access to the physical counter offset
Quoted 11 lines · Marc Zyngier
Quoted 3 lines
FEAT_ECVprovides a guest physical counter-timer offset register (CNTPOFF_EL2), but ECV-enabled hardware is nonexistent at the time of writing so support for it was elided for the sake of the author :)You seem to have done most the work for that, and there are SW models out there that implement ECV. If anything, the ECV support should come first, and its emulation only as a poor man's workaround.
It is also good to show silicon vendors that they suck at providing useful features, and pointing them to the code that would use these features *is* an incentive.
Lol, then so be it. I'll add ECV support to this series. What can I test with, though? I don't recall QEMU supporting ECV last I checked..
Quoted 13 lines
I really dislike the fallback to !vhe here. I'd rather you special-case the emulated ptimer in the VHE case:
if (has_vhe()) { map->direct_vtimer = vcpu_vtimer(vcpu); if (!timer_get_offset(vcpu_ptimer(vcpu))) map->direct_ptimer = vcpu_ptimer(vcpu); map->emul_ptimer = NULL; } else { map->direct_ptimer = NULL; map->emul_ptimer = vcpu_ptimer(vcpu); } } else {
Ack.
Quoted 2 lines
and you can drop the
timer_emulation_required()helper which doesn't serve any purpose.
Especially if I add ECV to this series, I'd prefer to carry it than open-code the check for CNTPOFF && !ECV in get_timer_map(). Do you concur? I can tighten it to ptimer_emulation_required() and avoid taking an arch_timer context if you prefer.
Quoted 10 lines
arch/arm64/kvm/hyp/include/hyp/switch.h static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu) · line 419
Quoted 7 lines
arch/arm64/kvm/hyp/include/hyp/switch.h +7
-0+static inline bool __hyp_handle_counter (struct kvm_vcpu *vcpu) +{ +u32 sysreg =esr_sys64_to_sysreg (kvm_vcpu_get_esr (vcpu)); +int rt =kvm_vcpu_sys_get_rt (vcpu); +u64 rv; + +if (sysreg !=SYS_CNTPCT_EL0 )That's feels really optimistic. You don't check for the exception class, and pray that the bits will align? ;-)
Doh! Missed the EC check.
Quoted 2 lines
Please don't add more small includes like this. It makes things hard to read and hides bugs
Ack.
Quoted 3 lines
the counter read can (and will) be speculated,
so you definitely need an ISB before the access. Please also look at__arch_counter_get_cntpct()andarch_counter_enforce_ordering().
Wouldn't it be up to the guest to decide if it wants the counter to be speculated or not? I debated this a bit myself in the implementation, as we would trap both ordered and un-ordered reads. Well, no way to detect it :)
Quoted 8 lines
Quoted 5 lines
-/*
- * Should only be called on non-VHE systems.
- * VHE systems use EL2 timers and configure EL1 timers inkvm_timer_init_vhe().
- */This comment still applies. this function *is* nVHE specific.
Ack.
Quoted 5 lines
You now pay the price of reprogramming
CNTHCTL_EL2on every entry for VHE, and that's not right. On VHE, it should be enough to perform the programming onvcpu_load()only.
True. I'll rework the programming in the next series.
Quoted 4 lines
Overall, this patch needs a bit of splitting up (userspace interface, HV rework), re-optimise VHE, and it *definitely* wants the ECV support for the physical timer.
Sure thing, thanks for the review Marc!
--
Best,
Oliver
Marc Zyngier
maintainerRe: [PATCH v5 11/13] KVM: arm64: Provide userspace access to the physical counter offset
Quoted 10 lines · Oliver Upton
Quoted 8 lines · Marc Zyngier
Quoted 3 lines
FEAT_ECVprovides a guest physical counter-timer offset register (CNTPOFF_EL2), but ECV-enabled hardware is nonexistent at the time of writing so support for it was elided for the sake of the author :)You seem to have done most the work for that, and there are SW models out there that implement ECV. If anything, the ECV support should come first, and its emulation only as a poor man's workaround.
Quoted 6 lines
Quoted 3 lines
It is also good to show silicon vendors that they suck at providing useful features, and pointing them to the code that would use these features *is* an incentive.
Lol, then so be it. I'll add ECV support to this series. What can I test with, though? I don't recall QEMU supporting ECV last I checked..
You want the ARM FVP model, or maybe even the Foundation model. It has support all the way to ARMv8.7 apparently. I personally use the FVP, get in touch offline and I'll help you with the setup.
In general, I tend to trust the ARM models a lot more than QEMU for the quality of the emulation. You can tune it in some bizarre way (the cache modelling is terrifying), and it will definitely do all kind of crazy reordering and speculation.
Quoted 17 lines
Quoted 4 lines
I really dislike the fallback to !vhe here. I'd rather you special-case the emulated ptimer in the VHE case:
Quoted 10 lines
if (has_vhe()) { map->direct_vtimer = vcpu_vtimer(vcpu); if (!timer_get_offset(vcpu_ptimer(vcpu))) map->direct_ptimer = vcpu_ptimer(vcpu); map->emul_ptimer = NULL; } else { map->direct_ptimer = NULL; map->emul_ptimer = vcpu_ptimer(vcpu); } } else {Ack.
Quoted 7 lines
Quoted 2 lines
and you can drop the
timer_emulation_required()helper which doesn't serve any purpose.Especially if I add ECV to this series, I'd prefer to carry it than open-code the check for CNTPOFF && !ECV in
get_timer_map(). Do you concur? I can tighten it toptimer_emulation_required()and avoid taking anarch_timercontext if you prefer.
Sure, whatever you prefer. I'm not set on one way or another, but in the above case, it was clearly easier to get rid of the helper.
Quoted 8 lines
Quoted 3 lines
the counter read can (and will) be speculated,
so you definitely need an ISB before the access. Please also look at__arch_counter_get_cntpct()andarch_counter_enforce_ordering().Wouldn't it be up to the guest to decide if it wants the counter to be speculated or not? I debated this a bit myself in the implementation, as we would trap both ordered and un-ordered reads. Well, no way to detect it :)
There has been a trap, so that already was a context synchronisation event. but it would be pretty common for the counter to be speculated way early, when entering EL2. That's not the end of the world, but that's not an accurate emulation. You'd want it to be as close as possible to the reentry point into the guest.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v5 11/13] KVM: arm64: Provide userspace access to the physical counter offset
Quoted 8 lines · Marc Zyngier
You want the ARM FVP model, or maybe even the Foundation model. It has support all the way to ARMv8.7 apparently. I personally use the FVP, get in touch offline and I'll help you with the setup.
In general, I tend to trust the ARM models a lot more than QEMU for the quality of the emulation. You can tune it in some bizarre way (the cache modelling is terrifying), and it will definitely do all kind of crazy reordering and speculation.
Awesome, thanks. I'll give this a try.
Quoted 21 lines
Quoted 21 lines
Quoted 5 lines
I really dislike the fallback to !vhe here. I'd rather you special-case the emulated ptimer in the VHE case:
Quoted 10 lines
if (has_vhe()) { map->direct_vtimer = vcpu_vtimer(vcpu); if (!timer_get_offset(vcpu_ptimer(vcpu))) map->direct_ptimer = vcpu_ptimer(vcpu); map->emul_ptimer = NULL; } else { map->direct_ptimer = NULL; map->emul_ptimer = vcpu_ptimer(vcpu); } } else {Ack.
Quoted 2 lines
and you can drop the
timer_emulation_required()helper which doesn't serve any purpose.
Quoted 7 lines
Quoted 4 lines
Especially if I add ECV to this series, I'd prefer to carry it than open-code the check for CNTPOFF && !ECV in
get_timer_map(). Do you concur? I can tighten it toptimer_emulation_required()and avoid taking anarch_timercontext if you prefer.Sure, whatever you prefer. I'm not set on one way or another, but in the above case, it was clearly easier to get rid of the helper.
Agreed, it made sense to prune before.
Quoted 5 lines
There has been a trap, so that already was a context synchronisation event. but it would be pretty common for the counter to be speculated way early, when entering EL2. That's not the end of the world, but that's not an accurate emulation. You'd want it to be as close as possible to the reentry point into the guest.
Cool, I'll add the barrier to kick the can closer to guest entry.
--
Thanks,
Oliver
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v5 11/13] KVM: arm64: Provide userspace access to the physical counter offset
Hi Marc,
Quoted 6 lines · Oliver Upton
Quoted 4 lines · Marc Zyngier
You want the ARM FVP model, or maybe even the Foundation model. It has support all the way to ARMv8.7 apparently. I personally use the FVP, get in touch offline and I'll help you with the setup.
Quoted 7 lines
Quoted 4 lines
In general, I tend to trust the ARM models a lot more than QEMU for the quality of the emulation. You can tune it in some bizarre way (the cache modelling is terrifying), and it will definitely do all kind of crazy reordering and speculation.
Awesome, thanks. I'll give this a try.
I have another spin of this series ready to kick out the door that implements ECV support but ran into some issues testing it... Seems that the ARM Foundation model only implements ECV=0x01, when we need ECV=0x02 for CNTPOFF_EL2 to be valid. Any thoughts, or shall I just send out the series and stare at it long enough to make sure the ECV parts look right ;-)
--
Thanks,
Oliver
PATCH v5 12/13 selftests: KVM: Test physical counter offsetting
In reply to Oliver UptonTest that userpace adjustment of the guest physical counter-timer results in the correct view of within the guest.
--- .../selftests/kvm/include/aarch64/processor.h | 12 ++++++++ .../kvm/system_counter_offset_test.c | 29 ++++++++++++++++--- 2 files changed, 37 insertions(+), 4 deletions(-)
tools/testing/selftests/kvm/include/aarch64/processor.h +12 -0
@@ -141,4 +141,16 @@ static inline uint64_t read_cntvct_ordered(void) return r; } +static inline uint64_t read_cntpct_ordered (void ) +{ +uint64_t r; + + __asm__ __volatile__("isb\n\t" +"mrs %0, cntpct_el0\n\t" +"isb\n\t" + :"=r" (r)); + +return r; +} +#endif /* SELFTEST_KVM_PROCESSOR_H */
tools/testing/selftests/kvm/system_counter_offset_test.c +25 -4
@@ -57,6 +57,7 @@ static uint64_t host_read_guest_system_counter(struct test_case *test)enum arch_counter {VIRTUAL , +PHYSICAL , };struct test_case {
@@ -68,23 +69,41 @@ static struct test_case test_cases[] = { { .counter = VIRTUAL, .offset = 0 }, { .counter = VIRTUAL, .offset = 180 * NSEC_PER_SEC }, { .counter = VIRTUAL, .offset = -180 * NSEC_PER_SEC }, + { .counter = PHYSICAL, .offset = 0 }, + { .counter = PHYSICAL, .offset = 180 * NSEC_PER_SEC }, + { .counter = PHYSICAL, .offset = -180 * NSEC_PER_SEC }, };static void check_preconditions (struct kvm_vm *vm) {if (!_vcpu_has_device_attr (vm,VCPU_ID ,KVM_ARM_VCPU_TIMER_CTRL , -KVM_ARM_VCPU_TIMER_OFFSET_VTIMER )) +KVM_ARM_VCPU_TIMER_OFFSET_VTIMER ) && + !_vcpu_has_device_attr (vm,VCPU_ID ,KVM_ARM_VCPU_TIMER_CTRL , +KVM_ARM_VCPU_TIMER_OFFSET_PTIMER ))return ; -print_skip ("KVM_ARM_VCPU_TIMER_OFFSET_VTIMER not supported; skipping test" ); +print_skip ("KVM_ARM_VCPU_TIMER_OFFSET_{VTIMER,PTIMER} not supported; skipping test" );exit (KSFT_SKIP ); }static void setup_system_counter (struct kvm_vm *vm,struct test_case *test) { +u64 attr =0 ; + +switch (test->counter) { +case VIRTUAL : + attr =KVM_ARM_VCPU_TIMER_OFFSET_VTIMER ; +break ; +case PHYSICAL : + attr =KVM_ARM_VCPU_TIMER_OFFSET_PTIMER ; +break ; +default : +TEST_ASSERT (false,"unrecognized counter index %u" , + test->counter); + } +vcpu_access_device_attr (vm,VCPU_ID ,KVM_ARM_VCPU_TIMER_CTRL , -KVM_ARM_VCPU_TIMER_OFFSET_VTIMER , &test->offset, - true); + attr, &test->offset, true); }static uint64_t guest_read_system_counter (struct test_case *test)
@@ -92,6 +111,8 @@ static uint64_t guest_read_system_counter(struct test_case *test)switch (test->counter) {case VIRTUAL :return read_cntvct_ordered (); +case PHYSICAL : +return read_cntpct_ordered ();default :GUEST_ASSERT (0 ); }
Signature · 1 line
2.32.0.432.gabb21c7263-goog
PATCH v5 13/13 selftests: KVM: Add counter emulation benchmark
In reply to Oliver UptonAdd a test case for counter emulation on arm64. A side effect of how KVM handles physical counter offsetting on non-ECV systems is that the virtual counter will always hit hardware and the physical could be emulated. Force emulation by writing a nonzero offset to the physical counter and compare the elapsed cycles to a direct read of the hardware register.
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../kvm/aarch64/counter_emulation_benchmark.c | 207 ++++++++++++++++++ 3 files changed, 209 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
tools/testing/selftests/kvm/.gitignore +1 -0
@@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only +/aarch64/counter_emulation_benchmark /aarch64/debug-exceptions /aarch64/get-reg-list /aarch64/vgic_init
tools/testing/selftests/kvm/Makefile +1 -0
@@ -87,6 +87,7 @@ TEST_GEN_PROGS_x86_64 += steal_time TEST_GEN_PROGS_x86_64 += kvm_binary_stats_test TEST_GEN_PROGS_x86_64 += system_counter_offset_test +TEST_GEN_PROGS_aarch64 += aarch64/counter_emulation_benchmark TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c new file +207 -0
207 lines not on this page · raw message
Signature · 1 line
2.32.0.432.gabb21c7263-goog
Re: [PATCH v5 13/13] selftests: KVM: Add counter emulation benchmark
Quoted 10 lines · Oliver Upton
Add a test case for counter emulation on arm64. A side effect of how KVM handles physical counter offsetting on non-ECV systems is that the virtual counter will always hit hardware and the physical could be emulated. Force emulation by writing a nonzero offset to the physical counter and compare the elapsed cycles to a direct read of the hardware register.
Cc Andrew Jones <[email protected]>Reviewed-by Ricardo Koller <[email protected]>Signed-off-by Oliver Upton <[email protected]>
Quoted 7 lines
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../kvm/aarch64/counter_emulation_benchmark.c | 207 ++++++++++++++++++ 3 files changed, 209 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c