KVM: Add idempotent controls for migrating system counter state
Subsystems (7)
cover KVM: Add idempotent controls for migrating system counter state
KVM's current means of saving/restoring system counters is plagued with temporal issues. At least on ARM64 and x86, we migrate the guest's system counter by-value through the respective guest system register values (cntvct_el0, ia32_tsc). Restoring system counters by-value is brittle as the state is not idempotent: the host system counter is still oscillating between the attempted save and restore. Furthermore, VMMs may wish to transparently live migrate guest VMs, meaning that they include the elapsed time due to live migration blackout in the guest system counter view. The VMM thread could be preempted for any number of reasons (scheduler, L0 hypervisor under nested) between the time that it calculates the desired guest counter value and when KVM actually sets this counter state.
Despite the value-based interface that we present to userspace, KVM actually has idempotent guest controls by way of system counter offsets. We can avoid all of the issues associated with a value-based interface by abstracting these offset controls in new ioctls. This series introduces KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls, meant to provide userspace with idempotent controls of the guest system counter.
Patch 1 defines the ioctls, and was separated from the two provided implementations for the sake of review. If it is more intuitive, this patch can be squashed into the implementation commit.
Patch 2 realizes initial support for ARM64, migrating only the state associated with the guest's virtual counter-timer. Patch 3 introduces a KVM selftest to assert that userspace manipulation via the aforementioned ioctls produces the expected system counter values within the guest.
Patch 4 extends upon the ARM64 implementation by adding support for physical counter-timer offsetting. This is currently backed by a trap-and-emulate implementation, but can also be virtualized in hardware that fully implements ARMv8.6-ECV. ECV support has been elided from this series out of convenience for the author :) Patch 5 adds some test cases to the newly-minted kvm selftest to validate expectations of physical counter-timer emulation.
Patch 6 introduces yet another KVM selftest for aarch64, intended to measure the effects of physical counter-timer emulation. Data for this test can be found below, but basically there is some tradeoff of overhead for the sake of correctness, but it isn't too bad.
Patches 7-8 add support for the ioctls to x86 by shoehorning the controls into the pre-existing synchronization heuristics. Patch 7 provides necessary helper methods for the implementation to play nice with those heuristics, and patch 8 actually implements the ioctls.
Patch 9 adds x86 test cases to the system counter KVM selftest. Lastly, patch 10 documents the ioctls for both x86 and arm64.
All patches apply cleanly to kvm/next at the following commit:
a4345a7cecfb ("Merge tag 'kvmarm-fixes-5.13-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD")
Physical counter benchmark
--------------------------
The following data was collected by running 10000 iterations of the benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S machine with 2 80-core Ampere Altra SoCs. Measurements were collected for both VHE and nVHE operation using the kvm-arm.mode= command-line parameter.
nVHE
----
+--------------------+--------+---------+ | Metric | Native | Trapped | +--------------------+--------+---------+ | Average | 54ns | 148ns | | Standard Deviation | 124ns | 122ns | | 95th Percentile | 258ns | 348ns | +--------------------+--------+---------+
VHE
---
+--------------------+--------+---------+ | Metric | Native | Trapped | +--------------------+--------+---------+ | Average | 53ns | 152ns | | Standard Deviation | 92ns | 94ns | | 95th Percentile | 204ns | 307ns | +--------------------+--------+---------+
Oliver Upton (10):
KVM: Introduce KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls
KVM: arm64: Implement initial support for KVM_CAP_SYSTEM_COUNTER_STATE
selftests: KVM: Introduce system_counter_state_test
KVM: arm64: Add userspace control of the guest's physical counter
selftests: KVM: Add test cases for physical counter offsetting
selftests: KVM: Add counter emulation benchmark
KVM: x86: Refactor tsc synchronization code
KVM: x86: Implement KVM_CAP_SYSTEM_COUNTER_STATE
selftests: KVM: Add support for x86 to system_counter_state_test
Documentation: KVM: Document KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls
Documentation/virt/kvm/api.rst | 98 +++++++ Documentation/virt/kvm/locking.rst | 11 + arch/arm64/include/asm/kvm_host.h | 6 + arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 17 ++ arch/arm64/kvm/arch_timer.c | 84 +++++- arch/arm64/kvm/arm.c | 25 ++ arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +++ arch/arm64/kvm/hyp/nvhe/timer-sr.c | 16 +- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 8 + arch/x86/kvm/x86.c | 176 +++++++++--- include/uapi/linux/kvm.h | 5 + tools/testing/selftests/kvm/.gitignore | 2 + tools/testing/selftests/kvm/Makefile | 3 + .../kvm/aarch64/counter_emulation_benchmark.c | 209 ++++++++++++++ .../selftests/kvm/include/aarch64/processor.h | 24 ++ .../selftests/kvm/system_counter_state_test.c | 256 ++++++++++++++++++ 18 files changed, 926 insertions(+), 47 deletions(-) create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c create mode 100644 tools/testing/selftests/kvm/system_counter_state_test.c
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
PATCH 1/10 KVM: Introduce KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls
Correctly migrating a guest's view of the system counter is challenging. To date, KVM has utilized value-based interfaces (e.g. guest value for IA32_TSC msr on x86, CNTVCT_EL0 on arm64) for migrating guest counter state. Restoring counters by value is problematic, especially for operators who intend to have the guest's counters account for elapsed time during live migration. Reason being, the guest counters _still_ increment even after calculating an appropriate value to restore a guest's counter on the target machine.
Furthermore, maintaining the phase relationship between vCPU counters is impossible with a value-based approach. The only hope for maintaining the phase relationship of counters is to restore them by offset.
Introduce a new pair of vcpu ioctls, KVM_GET_SYSTEM_COUNTER_STATE and KVM_SET_SYSTEM_COUNTER_STATE, that aim to do just that. Each implementing architecture will define its own counter state structure, allowing for flexibility with ISAs that may have multiple counters (like arm64).
--- include/uapi/linux/kvm.h | 5 +++++ 1 file changed, 5 insertions(+)
include/uapi/linux/kvm.h +5 -0
@@ -1082,6 +1082,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_SGX_ATTRIBUTE 196#define KVM_CAP_VM_COPY_ENC_CONTEXT_FROM 197#define KVM_CAP_PTP_KVM 198 +#define KVM_CAP_SYSTEM_COUNTER_STATE 199#ifdef KVM_CAP_IRQ_ROUTING
@@ -1645,6 +1646,10 @@ struct kvm_xen_vcpu_attr {#define KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_DATA 0x4#define KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_ADJUST 0x5 +/* Available with KVM_CAP_SYSTEM_COUNTER_STATE */ +#define KVM_GET_SYSTEM_COUNTER_STATE _IOWR(KVMIO, 0xcc, struct kvm_system_counter_state) +#define KVM_SET_SYSTEM_COUNTER_STATE _IOW(KVMIO, 0xcd, struct kvm_system_counter_state) +/* Secure Encrypted Virtualization command */ enum sev_cmd_id {/* Guest initialization commands */
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
PATCH 2/10 KVM: arm64: Implement initial support for KVM_CAP_SYSTEM_COUNTER_STATE
In reply to Oliver UptonARMv8 provides for a virtual counter-timer offset that is added to guest views of the virtual counter-timer (CNTVOFF_EL2). To date, KVM has not provided userspace with any perception of this, and instead affords a value-based scheme of migrating the virtual counter-timer by directly reading/writing the guest's CNTVCT_EL0. This is problematic because counters continue to elapse while the register is being written, meaning it is possible for drift to sneak in to the guest's time scale. This is exacerbated by the fact that KVM will calculate an appropriate CNTVOFF_EL2 every time the register is written, which will be broadcast to all virtual CPUs. The only possible way to avoid causing guest time to drift is to restore counter-timers by offset.
Implement initial support for KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls to migrate the value of CNTVOFF_EL2. These ioctls yield precise control of the virtual counter-timers to userspace, allowing it to define its own heuristics for managing vCPU offsets.
--- arch/arm64/include/asm/kvm_host.h | 5 +++++ arch/arm64/include/uapi/asm/kvm.h | 10 ++++++++++ arch/arm64/kvm/arch_timer.c | 22 ++++++++++++++++++++++ arch/arm64/kvm/arm.c | 25 +++++++++++++++++++++++++ 4 files changed, 62 insertions(+)
arch/arm64/include/asm/kvm_host.h +5 -0
@@ -781,4 +781,9 @@ void __init kvm_hyp_reserve(void);static inline void kvm_hyp_reserve (void ) { }#endif +int kvm_arm_vcpu_get_system_counter_state (struct kvm_vcpu *vcpu, +struct kvm_system_counter_state *state); +int kvm_arm_vcpu_set_system_counter_state (struct kvm_vcpu *vcpu, +struct kvm_system_counter_state *state); +#endif /* __ARM64_KVM_HOST_H__ */
arch/arm64/include/uapi/asm/kvm.h +10 -0
@@ -184,6 +184,16 @@ struct kvm_vcpu_events {__u32 reserved[12 ]; }; +/* for KVM_{GET,SET}_SYSTEM_COUNTER_STATE */ +struct kvm_system_counter_state { +/* indicates what fields are valid in the structure */ +__u32 flags; +__u32 pad; +/* guest counter-timer offset, relative to host cntpct_el0 */ +__u64 cntvoff; +__u64 rsvd[7 ]; +}; +/* If you need to interpret the index values, here is the key: */ #define KVM_REG_ARM_COPROC_MASK 0x000000000FFF0000#define KVM_REG_ARM_COPROC_SHIFT 16
arch/arm64/kvm/arch_timer.c +22 -0
@@ -1259,3 +1259,25 @@ int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)return -ENXIO ; } + +int kvm_arm_vcpu_get_system_counter_state (struct kvm_vcpu *vcpu, +struct kvm_system_counter_state *state) +{ +if (state->flags) +return -EINVAL ; + + state->cntvoff =timer_get_offset (vcpu_vtimer (vcpu)); + +return 0 ; +} + +int kvm_arm_vcpu_set_system_counter_state (struct kvm_vcpu *vcpu, +struct kvm_system_counter_state *state) +{ +if (state->flags) +return -EINVAL ; + +timer_set_offset (vcpu_vtimer (vcpu), state->cntvoff); + +return 0 ; +}
arch/arm64/kvm/arm.c +25 -0
@@ -207,6 +207,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_SET_GUEST_DEBUG :case KVM_CAP_VCPU_ATTRIBUTES :case KVM_CAP_PTP_KVM : +case KVM_CAP_SYSTEM_COUNTER_STATE : r =1 ;break ;case KVM_CAP_SET_GUEST_DEBUG2 :
@@ -1273,6 +1274,30 @@ long kvm_arch_vcpu_ioctl(struct file *filp,return kvm_arm_vcpu_finalize (vcpu, what); } +case KVM_GET_SYSTEM_COUNTER_STATE : { +struct kvm_system_counter_state state; + +if (copy_from_user (&state, argp,sizeof (state))) +return -EFAULT ; + + r =kvm_arm_vcpu_get_system_counter_state (vcpu, &state); +if (r) +break ; + +if (copy_to_user (argp, &state,sizeof (state))) +return -EFAULT ; + +break ; + } +case KVM_SET_SYSTEM_COUNTER_STATE : { +struct kvm_system_counter_state state; + +if (copy_from_user (&state, argp,sizeof (state))) +return -EFAULT ; + + r =kvm_arm_vcpu_set_system_counter_state (vcpu, &state); +break ; + }default : r = -EINVAL ; }
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
Re: [PATCH 02/10] KVM: arm64: Implement initial support for KVM_CAP_SYSTEM_COUNTER_STATE
Quoted 81 lines · Oliver Upton
ARMv8 provides for a virtual counter-timer offset that is added to guest views of the virtual counter-timer (
CNTVOFF_EL2). To date, KVM has not provided userspace with any perception of this, and instead affords a value-based scheme of migrating the virtual counter-timer by directly reading/writing the guest'sCNTVCT_EL0. This is problematic because counters continue to elapse while the register is being written, meaning it is possible for drift to sneak in to the guest's time scale. This is exacerbated by the fact that KVM will calculate an appropriateCNTVOFF_EL2every time the register is written, which will be broadcast to all virtual CPUs. The only possible way to avoid causing guest time to drift is to restore counter-timers by offset.Implement initial support for KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls to migrate the value of
CNTVOFF_EL2. These ioctls yield precise control of the virtual counter-timers to userspace, allowing it to define its own heuristics for managing vCPU offsets.Reviewed-by Jim Mattson <[email protected]>Reviewed-by Jing Zhang <[email protected]>Reviewed-by Peter Shier <[email protected]>Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/include/asm/kvm_host.h | 5 +++++ arch/arm64/include/uapi/asm/kvm.h | 10 ++++++++++ arch/arm64/kvm/arch_timer.c | 22 ++++++++++++++++++++++ arch/arm64/kvm/arm.c | 25 +++++++++++++++++++++++++ 4 files changed, 62 insertions(+)arch/arm64/include/asm/kvm_host.h +5
-0@@ -781,4 +781,9 @@ void __init kvm_hyp_reserve(void);static inline void kvm_hyp_reserve (void ) { }#endif +int kvm_arm_vcpu_get_system_counter_state (struct kvm_vcpu *vcpu, +struct kvm_system_counter_state *state); +int kvm_arm_vcpu_set_system_counter_state (struct kvm_vcpu *vcpu, +struct kvm_system_counter_state *state); +#endif /* __ARM64_KVM_HOST_H__ */ arch/arm64/include/uapi/asm/kvm.h +10
-0@@ -184,6 +184,16 @@ struct kvm_vcpu_events {__u32 reserved[12 ]; }; +/* for KVM_{GET,SET}_SYSTEM_COUNTER_STATE */ +struct kvm_system_counter_state { +/* indicates what fields are valid in the structure */ +__u32 flags; +__u32 pad; +/* guest counter-timer offset, relative to host cntpct_el0 */ +__u64 cntvoff; +__u64 rsvd[7 ]; +}; +/* If you need to interpret the index values, here is the key: */ #define KVM_REG_ARM_COPROC_MASK 0x000000000FFF0000#define KVM_REG_ARM_COPROC_SHIFT 16arch/arm64/kvm/arch_timer.c +9
-0@@ -1259,3 +1259,25 @@ int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)return -ENXIO ; } + +int kvm_arm_vcpu_get_system_counter_state (struct kvm_vcpu *vcpu, +struct kvm_system_counter_state *state) +{ +if (state->flags) +return -EINVAL ; + + state->cntvoff =timer_get_offset (vcpu_vtimer (vcpu)); +
arch/arm64/kvm/arch_timer.c int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) · line 1280
Quoted 10 lines
arch/arm64/kvm/arch_timer.c +10
-0+return 0 ; +} + +int kvm_arm_vcpu_set_system_counter_state (struct kvm_vcpu *vcpu, +struct kvm_system_counter_state *state) +{ +if (state->flags) +return -EINVAL ; + +timer_set_offset (vcpu_vtimer (vcpu), state->cntvoff);
Adding some discussion that Ricardo and I had regarding this portion of the patch:
Ricardo asks if it would make more sense to have the KVM_SET_SYSTEM_COUNTER_STATE ioctl broadcast the counter offset to all vCPUs, like we do for the value-based SET_REG() implementation. To me, the broadcasting was more necessary for the value-based interface as it is difficult/impossible to synchronize by value, but now some of the onus to do the right thing might be on the VMM. No strong opinions either way, so open to suggestions here.
--
Thanks,
Oliver
Quoted 49 lines
arch/arm64/kvm/arch_timer.c +3
-0+ +return 0 ; +}arch/arm64/kvm/arm.c +25
-1@@ -207,6 +207,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_SET_GUEST_DEBUG :case KVM_CAP_VCPU_ATTRIBUTES :case KVM_CAP_PTP_KVM : +case KVM_CAP_SYSTEM_COUNTER_STATE : r =1 ;break ;case KVM_CAP_SET_GUEST_DEBUG2 :@@ -1273,6 +1274,30 @@ long kvm_arch_vcpu_ioctl(struct file *filp,return kvm_arm_vcpu_finalize (vcpu, what); } +case KVM_GET_SYSTEM_COUNTER_STATE : { +struct kvm_system_counter_state state; + +if (copy_from_user (&state, argp,sizeof (state))) +return -EFAULT ; + + r =kvm_arm_vcpu_get_system_counter_state (vcpu, &state); +if (r) +break ; + +if (copy_to_user (argp, &state,sizeof (state))) +return -EFAULT ; + +break ; + } +case KVM_SET_SYSTEM_COUNTER_STATE : { +struct kvm_system_counter_state state; + +if (copy_from_user (&state, argp,sizeof (state))) +return -EFAULT ; + + r =kvm_arm_vcpu_set_system_counter_state (vcpu, &state); +break ; + }default : r = -EINVAL ; } --2.32.0.rc1.229.g3e70b5a671-goog
Marc Zyngier
maintainerRe: [PATCH 02/10] KVM: arm64: Implement initial support for KVM_CAP_SYSTEM_COUNTER_STATE
In reply to Oliver UptonHi Oliver,
Please Cc the KVM/arm64 reviewers (now added). Also, please consider subscribing to the kvmarm mailing list so that I don't have to manually approve your posts ;-).
Quoted 12 lines · Oliver Upton
ARMv8 provides for a virtual counter-timer offset that is added to guest views of the virtual counter-timer (
CNTVOFF_EL2). To date, KVM has not provided userspace with any perception of this, and instead affords a value-based scheme of migrating the virtual counter-timer by directly reading/writing the guest'sCNTVCT_EL0. This is problematic because counters continue to elapse while the register is being written, meaning it is possible for drift to sneak in to the guest's time scale. This is exacerbated by the fact that KVM will calculate an appropriateCNTVOFF_EL2every time the register is written, which will be broadcast to all virtual CPUs. The only possible way to avoid causing guest time to drift is to restore counter-timers by offset.
Well, the current method has one huge advantage: time can never go backward from the guest PoV if you restore what you have saved. Yes, time can elapse, but you don't even need to migrate to observe that.
Quoted 5 lines
Implement initial support for KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls to migrate the value of
CNTVOFF_EL2. These ioctls yield precise control of the virtual counter-timers to userspace, allowing it to define its own heuristics for managing vCPU offsets.
I'm not really in favour of inventing a completely new API, for multiple reasons:
- CNTVOFF is an EL2 concept. I'd rather not expose it as such as it becomes really confusing with NV (which does expose its own CNTVOFF via the ONE_REG interface)
- You seem to allow each vcpu to get its own offset. I don't think that's right. The architecture defines that all PEs have the same view of the counters, and an EL1 guest should be given that illusion.
- by having a parallel save/restore interface, you make it harder to reason about what happens with concurrent calls to both interfaces
- the userspace API is already horribly bloated, and I'm not overly keen on adding more if we can avoid it.
I'd rather you extend the current ONE_REG interface and make it modal, either allowing the restore of an absolute value or an offset for CNTVCT_EL0. This would also keep a consistent behaviour when restoring vcpus. The same logic would apply to the physical offset.
As for how to make it modal, we have plenty of bits left in the ONE_REG encoding. Pick one, and make that a "relative" attribute. This will result in some minor surgery in the get/set code paths, but at least no entirely new mechanism.
One question though: how do you plan to reliably compute the offset? As far as I can see, it is subject to the same issues you described above (while the guest is being restored, time flies), and you have the added risk of exposing a counter going backward from a guest perspective.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH 02/10] KVM: arm64: Implement initial support for KVM_CAP_SYSTEM_COUNTER_STATE
Quoted 6 lines · Marc Zyngier
Hi Oliver,
Please Cc the KVM/arm64 reviewers (now added). Also, please consider subscribing to the kvmarm mailing list so that I don't have to manually approve your posts ;-).
/facepalm
Thought I had done this already. Re-requested to join kvmarm@. Seems that gmail politely decided the mailing list was spam, so no confirmation email came through.
Quoted 25 lines
Quoted 12 lines · Oliver Upton
ARMv8 provides for a virtual counter-timer offset that is added to guest views of the virtual counter-timer (
CNTVOFF_EL2). To date, KVM has not provided userspace with any perception of this, and instead affords a value-based scheme of migrating the virtual counter-timer by directly reading/writing the guest'sCNTVCT_EL0. This is problematic because counters continue to elapse while the register is being written, meaning it is possible for drift to sneak in to the guest's time scale. This is exacerbated by the fact that KVM will calculate an appropriateCNTVOFF_EL2every time the register is written, which will be broadcast to all virtual CPUs. The only possible way to avoid causing guest time to drift is to restore counter-timers by offset.Well, the current method has one huge advantage: time can never go backward from the guest PoV if you restore what you have saved. Yes, time can elapse, but you don't even need to migrate to observe that.
Quoted 5 lines
Implement initial support for KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls to migrate the value of
CNTVOFF_EL2. These ioctls yield precise control of the virtual counter-timers to userspace, allowing it to define its own heuristics for managing vCPU offsets.
Quoted 6 lines
I'm not really in favour of inventing a completely new API, for multiple reasons:
- CNTVOFF is an EL2 concept. I'd rather not expose it as such as it becomes really confusing with NV (which does expose its own CNTVOFF via the
ONE_REGinterface)
Very true. At least on x86, there's a fair bit of plumbing to handle the KVM-owned L0 offset reg and the guest-owned L1 offset reg.
Quoted 4 lines
- You seem to allow each vcpu to get its own offset. I don't think that's right. The architecture defines that all PEs have the same view of the counters, and an EL1 guest should be given that illusion.
Agreed. I would have preferred a VM-wide ioctl to do this, but since x86 explicitly allows for drifted TSCs that can't be the case in a generic ioctl. I can do the same broadcasting as we do in the case of a VMM write to CNTVCT_EL0.
Quoted 5 lines
- by having a parallel save/restore interface, you make it harder to reason about what happens with concurrent calls to both interfaces
- the userspace API is already horribly bloated, and I'm not overly keen on adding more if we can avoid it.
Pssh. My ioctl numbers aren't _too_ close to the limit ;-)
Quoted 10 lines
I'd rather you extend the current
ONE_REGinterface and make it modal, either allowing the restore of an absolute value or an offset forCNTVCT_EL0. This would also keep a consistent behaviour when restoring vcpus. The same logic would apply to the physical offset.As for how to make it modal, we have plenty of bits left in the
ONE_REGencoding. Pick one, and make that a "relative" attribute. This will result in some minor surgery in the get/set code paths, but at least no entirely new mechanism.
Yeah, it'd be good to do it w/o adding new plumbing. The only reason I'd considered it is because x86 might necessitate it. Not wanting to apply bad convention to other arches, but keeping at least a somewhat consistent UAPI would be nice.
Quoted 5 lines
One question though: how do you plan to reliably compute the offset? As far as I can see, it is subject to the same issues you described above (while the guest is being restored, time flies), and you have the added risk of exposing a counter going backward from a guest perspective.
Indeed, we do have the risk of time going backwards, but I'd say that the VMM shares in the responsibility to provide a consistent view of the counter too.
Here's how I envisioned it working:
Record the time, cycles, and offset (T0, C0, Off0) when saving the counter state. Record time and cycles (T1, C1) again when trying to restore counter state. Compute the new offset:
Off1 = Off0 - (T1-T0) * CNTFRQ - (C0 - C1).
The primary concern here is idempotence. Once Off1 is calculated, it doesn't matter how much time elapses between the calculation and the call into KVM, it will always produce the intended result. If instead we restore the counters by-value (whilst trying to account for elapsed time), my impression is that we'd do the following: Record time and guest counter (T0, G0) when saving counter state. Record time again when trying to restore counter state.
In userspace, compute the time elapsed and fold it into the guest counter (G1):
G1 = G0 + (T1-T0) * CNTFRQ
And then in the kernel:
CNTVOFF = G1 - CNTPCT
Any number of things can happen in between the kernel and userspace portions of this operation, causing some drift of the VM's counter. Fundamentally I suppose the issue we have is that we sample the host counter twice (T1, G1), when really we'd want to only do so once.
So, open to any suggestions where we avoid the issue of causing the guest counter to drift, offsets only seemed to be the easiest thing given that they ought to be constant for the lifetime of a VM on a host and is the backing state used by hardware.
--
Thanks,
Oliver
Quoted 7 lines
Thanks,
M.--
Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Paolo Bonzini
maintainerRe: [PATCH 02/10] KVM: arm64: Implement initial support for KVM_CAP_SYSTEM_COUNTER_STATE
Quoted 9 lines · Oliver Upton
Quoted 4 lines
- You seem to allow each vcpu to get its own offset. I don't think that's right. The architecture defines that all PEs have the same view of the counters, and an EL1 guest should be given that illusion.
Agreed. I would have preferred a VM-wide ioctl to do this, but since x86 explicitly allows for drifted TSCs that can't be the case in a generic ioctl. I can do the same broadcasting as we do in the case of a VMM write to
CNTVCT_EL0.
If you use VM-wide GET/SET_DEVICE_ATTR, please make it retrieve the host CLOCK_REALTIME and counter at the same time.
Paolo
Paolo Bonzini
maintainerRe: [PATCH 02/10] KVM: arm64: Implement initial support for KVM_CAP_SYSTEM_COUNTER_STATE
In reply to Marc ZyngierQuoted 19 lines · Marc Zyngier
Quoted 4 lines
Implement initial support for KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls to migrate the value of
CNTVOFF_EL2. These ioctls yield precise control of the virtual counter-timers to userspace, allowing it to define its own heuristics for managing vCPU offsets.I'm not really in favour of inventing a completely new API, for multiple reasons:
- CNTVOFF is an EL2 concept. I'd rather not expose it as such as it becomes really confusing with NV (which does expose its own CNTVOFF via the
ONE_REGinterface)- You seem to allow each vcpu to get its own offset. I don't think that's right. The architecture defines that all PEs have the same view of the counters, and an EL1 guest should be given that illusion.
- by having a parallel save/restore interface, you make it harder to reason about what happens with concurrent calls to both interfaces
Quoted 7 lines
- the userspace API is already horribly bloated, and I'm not overly keen on adding more if we can avoid it.
I'd rather you extend the current
ONE_REGinterface and make it modal, either allowing the restore of an absolute value or an offset forCNTVCT_EL0. This would also keep a consistent behaviour when restoring vcpus. The same logic would apply to the physical offset.
What about using KVM_GET/SET_DEVICE_ATTR? It makes sense that the guest value for nested virt goes through ONE_REG, while the host value goes through DEVICE_ATTR.
Paolo
PATCH 3/10 selftests: KVM: Introduce system_counter_state_test
In reply to Oliver UptonTest that the KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls correctly configure the guest's view of the virtual counter-timer (CNTVCT_EL0).
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/aarch64/processor.h | 24 +++ .../selftests/kvm/system_counter_state_test.c | 199 ++++++++++++++++++ 4 files changed, 225 insertions(+) create mode 100644 tools/testing/selftests/kvm/system_counter_state_test.c
tools/testing/selftests/kvm/.gitignore +1 -0
@@ -34,6 +34,7 @@ /x86_64/xen_vmcall_test /x86_64/xss_msr_test /x86_64/vmx_pmu_msrs_test +/system_counter_state_test /demand_paging_test /dirty_log_test /dirty_log_perf_test
tools/testing/selftests/kvm/Makefile +1 -0
@@ -80,6 +80,7 @@ TEST_GEN_PROGS_x86_64 += steal_time TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list-sve TEST_GEN_PROGS_aarch64 += aarch64/vgic_init +TEST_GEN_PROGS_aarch64 += system_counter_state_test TEST_GEN_PROGS_aarch64 += demand_paging_test TEST_GEN_PROGS_aarch64 += dirty_log_test TEST_GEN_PROGS_aarch64 += dirty_log_perf_test
tools/testing/selftests/kvm/include/aarch64/processor.h +24 -0
@@ -52,6 +52,30 @@ static inline void set_reg(struct kvm_vm *vm, uint32_t vcpuid, uint64_t id, uintvcpu_ioctl (vm, vcpuid,KVM_SET_ONE_REG , ®); } +static inline uint64_t read_cntpct_ordered (void ) +{ +uint64_t r; + + asmvolatile ("isb\n\t" +"mrs %0, cntpct_el0\n\t" +"isb\n\t" + :"=r" (r)); + +return r; +} + +static inline uint64_t read_cntvct_ordered (void ) +{ +uint64_t r; + + asmvolatile ("isb\n\t" +"mrs %0, cntvct_el0\n\t" +"isb\n\t" + :"=r" (r)); + +return r; +} +void aarch64_vcpu_setup (struct kvm_vm *vm,int vcpuid,struct kvm_vcpu_init *init);void aarch64_vcpu_add_default (struct kvm_vm *vm,uint32_t vcpuid,struct kvm_vcpu_init *init,void *guest_code);
tools/testing/selftests/kvm/system_counter_state_test.c new file +199 -0
199 lines not on this page · raw message
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
PATCH 4/10 KVM: arm64: Add userspace control of the guest's physical counter
In reply to Oliver UptonARMv8.6 adds an extension to the architecture providing hypervisors with more extensive controls of the guest's counters. A particularly interesting control is CNTPOFF_EL2, a fixed offset subtracted from the physical counter value to derive the guest's value. VMMs that live migrate their guests may be particularly interested in this feature in order to provide a consistent view of the physical counter across live migrations.
In the interim, KVM can emulate this behavior by simply enabling traps on CNTPCT_EL0 and subtracting an offset.
Add a new field to kvm_system_counter_state allowing a VMM to request an offset to the physical counter. If this offset is nonzero, enable traps on CNTPCT_EL0. Emulate guest reads to the register in the fast path to keep counter reads reasonably performant, avoiding a full exit from the guest.
--- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 9 +++- arch/arm64/kvm/arch_timer.c | 66 +++++++++++++++++++++++-- arch/arm64/kvm/hyp/include/hyp/switch.h | 31 ++++++++++++ arch/arm64/kvm/hyp/nvhe/timer-sr.c | 16 ++++-- 6 files changed, 117 insertions(+), 7 deletions(-)
arch/arm64/include/asm/kvm_host.h +1 -0
@@ -200,6 +200,7 @@ enum vcpu_sysreg {SP_EL1 ,SPSR_EL1 , +CNTPOFF_EL2 ,CNTVOFF_EL2 ,CNTV_CVAL_EL0 , CNTV_CTL_EL0,
arch/arm64/include/asm/sysreg.h +1 -0
@@ -505,6 +505,7 @@#define SYS_AMEVCNTR0_MEM_STALL SYS_AMEVCNTR0_EL0(3)#define SYS_CNTFRQ_EL0 sys_reg(3, 3, 14, 0, 0) +#define SYS_CNTPCT_EL0 sys_reg(3, 3, 14, 0, 1)#define SYS_CNTP_TVAL_EL0 sys_reg(3, 3, 14, 2, 0)#define SYS_CNTP_CTL_EL0 sys_reg(3, 3, 14, 2, 1)
arch/arm64/include/uapi/asm/kvm.h +8 -1
@@ -184,6 +184,8 @@ struct kvm_vcpu_events {__u32 reserved[12 ]; }; +#define KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET (1ul << 0) +/* for KVM_{GET,SET}_SYSTEM_COUNTER_STATE */ struct kvm_system_counter_state {/* indicates what fields are valid in the structure */
@@ -191,7 +193,12 @@ struct kvm_system_counter_state {__u32 pad;/* guest counter-timer offset, relative to host cntpct_el0 */ __u64 cntvoff; -__u64 rsvd[7 ]; +/* +* Guest physical counter-timer offset, relative to host cntpct_el0. +* Valid when KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET is set. +*/ +__u64 cntpoff; +__u64 rsvd[6 ]; };/* If you need to interpret the index values, here is the key: */
arch/arm64/kvm/arch_timer.c +63 -3
124 lines not on this page · raw message
arch/arm64/kvm/hyp/include/hyp/switch.h +31 -0
@@ -287,6 +287,30 @@ static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)return true; } +static inline u64 __hyp_read_cntpct (struct kvm_vcpu *vcpu) +{ +return read_sysreg (cntpct_el0) -__vcpu_sys_reg (vcpu,CNTPOFF_EL2 ); +} + +static inline bool __hyp_handle_counter (struct kvm_vcpu *vcpu) +{ +u32 sysreg =esr_sys64_to_sysreg (kvm_vcpu_get_esr (vcpu)); +int rt =kvm_vcpu_sys_get_rt (vcpu); +u64 rv; + +switch (sysreg) { +case SYS_CNTPCT_EL0 : + rv =__hyp_read_cntpct (vcpu); +break ; +default : +return false; + } + +vcpu_set_reg (vcpu, rt, rv); +__kvm_skip_instr (vcpu); +return true; +} +static inline bool handle_tx2_tvm (struct kvm_vcpu *vcpu) {u32 sysreg =esr_sys64_to_sysreg (kvm_vcpu_get_esr (vcpu));
@@ -439,6 +463,13 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)if (*exit_code !=ARM_EXCEPTION_TRAP ) goto exit; +/* +* We trap acesses to the physical counter value register (CNTPCT_EL0) +* if userspace has requested a physical counter offset. +*/ +if (__hyp_handle_counter (vcpu)) + goto guest; +if (cpus_have_final_cap (ARM64_WORKAROUND_CAVIUM_TX2_219_TVM ) &&kvm_vcpu_trap_get_class (vcpu) == ESR_ELx_EC_SYS64 &&handle_tx2_tvm (vcpu))
arch/arm64/kvm/hyp/nvhe/timer-sr.c +13 -3
@@ -35,14 +35,24 @@ void __timer_disable_traps(struct kvm_vcpu *vcpu)*/ void __timer_enable_traps (struct kvm_vcpu *vcpu) { -u64 val; +u64 val, cntpoff; + + cntpoff =__vcpu_sys_reg (vcpu,CNTPOFF_EL2 );/* * Disallow physical timer access for the guest -* Physical counter access is allowed */ val =read_sysreg (cnthctl_el2); val &= ~CNTHCTL_EL1PCEN ; - val |=CNTHCTL_EL1PCTEN ; + +/* +* Disallow physical counter access for the guest if offsetting is +* requested. +*/ +if (cntpoff) + val &= ~CNTHCTL_EL1PCTEN ; +else + val |=CNTHCTL_EL1PCTEN ; +write_sysreg (val, cnthctl_el2); }
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
Re: [PATCH 04/10] KVM: arm64: Add userspace control of the guest's physical counter
Quoted 221 lines · Oliver Upton
ARMv8.6 adds an extension to the architecture providing hypervisors with more extensive controls of the guest's counters. A particularly interesting control is
CNTPOFF_EL2, a fixed offset subtracted from the physical counter value to derive the guest's value. VMMs that live migrate their guests may be particularly interested in this feature in order to provide a consistent view of the physical counter across live migrations.In the interim, KVM can emulate this behavior by simply enabling traps on
CNTPCT_EL0and subtracting an offset.Add a new field to
kvm_system_counter_stateallowing a VMM to request an offset to the physical counter. If this offset is nonzero, enable traps onCNTPCT_EL0. Emulate guest reads to the register in the fast path to keep counter reads reasonably performant, avoiding a full exit from the guest.Reviewed-by Peter Shier <[email protected]>Reviewed-by Ricardo Koller <[email protected]>Signed-off-by Oliver Upton <[email protected]>--- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 9 +++- arch/arm64/kvm/arch_timer.c | 66 +++++++++++++++++++++++-- arch/arm64/kvm/hyp/include/hyp/switch.h | 31 ++++++++++++ arch/arm64/kvm/hyp/nvhe/timer-sr.c | 16 ++++-- 6 files changed, 117 insertions(+), 7 deletions(-)arch/arm64/include/asm/kvm_host.h +1
-0@@ -200,6 +200,7 @@ enum vcpu_sysreg {SP_EL1 ,SPSR_EL1 , +CNTPOFF_EL2 ,CNTVOFF_EL2 ,CNTV_CVAL_EL0 , CNTV_CTL_EL0,arch/arm64/include/asm/sysreg.h +1
-0@@ -505,6 +505,7 @@#define SYS_AMEVCNTR0_MEM_STALL SYS_AMEVCNTR0_EL0(3)#define SYS_CNTFRQ_EL0 sys_reg(3, 3, 14, 0, 0) +#define SYS_CNTPCT_EL0 sys_reg(3, 3, 14, 0, 1)#define SYS_CNTP_TVAL_EL0 sys_reg(3, 3, 14, 2, 0)#define SYS_CNTP_CTL_EL0 sys_reg(3, 3, 14, 2, 1)arch/arm64/include/uapi/asm/kvm.h +8
-1@@ -184,6 +184,8 @@ struct kvm_vcpu_events {__u32 reserved[12 ]; }; +#define KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET (1ul << 0) +/* for KVM_{GET,SET}_SYSTEM_COUNTER_STATE */ struct kvm_system_counter_state {/* indicates what fields are valid in the structure */ @@ -191,7 +193,12 @@ struct kvm_system_counter_state {__u32 pad;/* guest counter-timer offset, relative to host cntpct_el0 */ __u64 cntvoff; -__u64 rsvd[7 ]; +/* +* Guest physical counter-timer offset, relative to host cntpct_el0. +* Valid when KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET is set. +*/ +__u64 cntpoff; +__u64 rsvd[6 ]; };/* If you need to interpret the index values, here is the key: */ arch/arm64/kvm/arch_timer.c +63
-3@@ -50,6 +50,7 @@ static void kvm_arm_timer_write(struct kvm_vcpu *vcpu,static u64 kvm_arm_timer_read (struct kvm_vcpu *vcpu,struct arch_timer_context *timer,enum kvm_arch_timer_regs treg); +static bool kvm_timer_emulation_required (struct arch_timer_context *ctx);u32 timer_get_ctl (struct arch_timer_context *ctxt) {@@ -86,6 +87,8 @@ static u64 timer_get_offset(struct arch_timer_context *ctxt)struct kvm_vcpu *vcpu = ctxt->vcpu;switch (arch_timer_ctx_index (ctxt)) { +case TIMER_PTIMER : +return __vcpu_sys_reg (vcpu,CNTPOFF_EL2 );case TIMER_VTIMER :return __vcpu_sys_reg (vcpu,CNTVOFF_EL2 );default :@@ -130,6 +133,9 @@ static void timer_set_offset(struct arch_timer_context *ctxt, u64 offset)struct kvm_vcpu *vcpu = ctxt->vcpu;switch (arch_timer_ctx_index (ctxt)) { +case TIMER_PTIMER : +__vcpu_sys_reg (vcpu,CNTPOFF_EL2 ) = offset; +break ;case TIMER_VTIMER :__vcpu_sys_reg (vcpu,CNTVOFF_EL2 ) = offset;break ;@@ -145,7 +151,7 @@ u64 kvm_phys_timer_read(void)static void get_timer_map (struct kvm_vcpu *vcpu,struct timer_map *map) { -if (has_vhe ()) { +if (has_vhe () && !kvm_timer_emulation_required (vcpu_ptimer (vcpu))) { map->direct_vtimer =vcpu_vtimer (vcpu); map->direct_ptimer =vcpu_ptimer (vcpu); map->emul_ptimer =NULL ;@@ -746,6 +752,30 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)return 0 ; } +bool kvm_timer_emulation_required (struct arch_timer_context *ctx) +{ +int idx =arch_timer_ctx_index (ctx); + +switch (idx) { +/* +* hardware doesn't support offsetting of the physical counter/timer. +* If offsetting is requested, enable emulation of the physical +* counter/timer. +*/ +case TIMER_PTIMER : +return timer_get_offset (ctx); +/* +* Conversely, hardware does support offsetting of the virtual +* counter/timer. +*/ +case TIMER_VTIMER : +return false; +default : +WARN_ON (1 ); +return false; + } +} +/* Make the updates of cntvoff for all vtimer contexts atomic */ static void update_vtimer_cntvoff (struct kvm_vcpu *vcpu,u64 cntvoff) {@@ -1184,6 +1214,24 @@ void kvm_timer_init_vhe(void)write_sysreg (val, cnthctl_el2); } +static void kvm_timer_update_traps_vhe (struct kvm_vcpu *vcpu) +{ +u32 cnthctl_shift =10 ; +u64 val; + +if (!kvm_timer_emulation_required (vcpu_ptimer (vcpu))) +return ; + +/* +* We must trap accesses to the physical counter/timer to emulate the +* nonzero offset. +*/ + val =read_sysreg (cnthctl_el2); + val &= ~(CNTHCTL_EL1PCEN << cnthctl_shift); + val &= ~(CNTHCTL_EL1PCTEN << cnthctl_shift); +write_sysreg (val, cnthctl_el2); +} +static void set_timer_irqs (struct kvm *kvm,int vtimer_irq,int ptimer_irq) {struct kvm_vcpu *vcpu;@@ -1260,24 +1308,36 @@ int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)return -ENXIO ; } +#define KVM_SYSTEM_COUNTER_STATE_VALID_FLAG_BITS \ + (KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET) +int kvm_arm_vcpu_get_system_counter_state (struct kvm_vcpu *vcpu,struct kvm_system_counter_state *state) { -if (state->flags) +if (state->flags & ~KVM_SYSTEM_COUNTER_STATE_VALID_FLAG_BITS )return -EINVAL ; state->cntvoff =timer_get_offset (vcpu_vtimer (vcpu)); +if (state->flags &KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET ) + state->cntpoff =timer_get_offset (vcpu_ptimer (vcpu)); +return 0 ; }int kvm_arm_vcpu_set_system_counter_state (struct kvm_vcpu *vcpu,struct kvm_system_counter_state *state) { -if (state->flags) +if (state->flags & ~KVM_SYSTEM_COUNTER_STATE_VALID_FLAG_BITS )return -EINVAL ;timer_set_offset (vcpu_vtimer (vcpu), state->cntvoff); +if (state->flags &KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET ) +timer_set_offset (vcpu_ptimer (vcpu), state->cntpoff); + +if (has_vhe ()) +kvm_timer_update_traps_vhe (vcpu); +return 0 ; }arch/arm64/kvm/hyp/include/hyp/switch.h +0
-0
arch/arm64/kvm/hyp/include/hyp/switch.h static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu) · line 292
Quoted 6 lines
arch/arm64/kvm/hyp/include/hyp/switch.h +3
-0@@ -287,6 +287,30 @@ static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)return true; } +static inline u64 __hyp_read_cntpct (struct kvm_vcpu *vcpu) +{ +return read_sysreg (cntpct_el0) -__vcpu_sys_reg (vcpu,CNTPOFF_EL2 );
Question for those with much more experience on the ARM side: is there any decent way to infer the counter bit-width to properly emulate here? This code is problematic when migrating a narrower guest (i.e. 56-bit counter) to a wider host (say an 8.6 implementation with a 64-bit counter). Otherwise, it would seem that userspace needs to explicitly request a counter width.
--
Thanks,
Oliver
Quoted 73 lines
arch/arm64/kvm/hyp/include/hyp/switch.h +28
-0+} + +static inline bool __hyp_handle_counter (struct kvm_vcpu *vcpu) +{ +u32 sysreg =esr_sys64_to_sysreg (kvm_vcpu_get_esr (vcpu)); +int rt =kvm_vcpu_sys_get_rt (vcpu); +u64 rv; + +switch (sysreg) { +case SYS_CNTPCT_EL0 : + rv =__hyp_read_cntpct (vcpu); +break ; +default : +return false; + } + +vcpu_set_reg (vcpu, rt, rv); +__kvm_skip_instr (vcpu); +return true; +} +static inline bool handle_tx2_tvm (struct kvm_vcpu *vcpu) {u32 sysreg =esr_sys64_to_sysreg (kvm_vcpu_get_esr (vcpu));@@ -439,6 +463,13 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)if (*exit_code !=ARM_EXCEPTION_TRAP ) goto exit; +/* +* We trap acesses to the physical counter value register (CNTPCT_EL0) +* if userspace has requested a physical counter offset. +*/ +if (__hyp_handle_counter (vcpu)) + goto guest; +if (cpus_have_final_cap (ARM64_WORKAROUND_CAVIUM_TX2_219_TVM ) &&kvm_vcpu_trap_get_class (vcpu) == ESR_ELx_EC_SYS64 &&handle_tx2_tvm (vcpu))arch/arm64/kvm/hyp/nvhe/timer-sr.c +13
-4@@ -35,14 +35,24 @@ void __timer_disable_traps(struct kvm_vcpu *vcpu)*/ void __timer_enable_traps (struct kvm_vcpu *vcpu) { -u64 val; +u64 val, cntpoff; + + cntpoff =__vcpu_sys_reg (vcpu,CNTPOFF_EL2 );/* * Disallow physical timer access for the guest -* Physical counter access is allowed */ val =read_sysreg (cnthctl_el2); val &= ~CNTHCTL_EL1PCEN ; - val |=CNTHCTL_EL1PCTEN ; + +/* +* Disallow physical counter access for the guest if offsetting is +* requested. +*/ +if (cntpoff) + val &= ~CNTHCTL_EL1PCTEN ; +else + val |=CNTHCTL_EL1PCTEN ; +write_sysreg (val, cnthctl_el2); } --2.32.0.rc1.229.g3e70b5a671-goog
PATCH 5/10 selftests: KVM: Add test cases for physical counter offsetting
In reply to Oliver UptonAdd support for physical counter offsetting to counter_state_test. Assert that guest reads of the physical counter are within the expected bounds.
--- .../selftests/kvm/system_counter_state_test.c | 27 ++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
tools/testing/selftests/kvm/system_counter_state_test.c +26 -1
@@ -62,11 +62,34 @@ static struct system_counter_state_test test_cases[] = { .cntvoff = -1 } }, + { + .counter = PHYSICAL, + .state = { + .flags = KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET, + .cntpoff = 0 + } + }, + { + .counter = PHYSICAL, + .state = { + .flags = KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET, + .cntpoff = 1000000 + } + }, + { + .counter = PHYSICAL, + .state = { + .flags = KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET, + .cntpoff = -1 + } + }, };static void pr_test (struct system_counter_state_test *test) { -pr_info ("counter: %s, cntvoff: %lld\n" ,counter_name (test->counter), test->state.cntvoff); +pr_info ("counter: %s, cntvoff: %lld, cntpoff: %lld\n" , +counter_name (test->counter), test->state.cntvoff, + test->state.cntpoff); } static struct kvm_system_counter_state *
@@ -103,6 +126,8 @@ static uint64_t host_read_guest_counter(struct system_counter_state_test *test) r =read_cntvct_ordered ();switch (test->counter) { +case PHYSICAL : + r -= test->state.cntpoff;case VIRTUAL : r -= test->state.cntvoff;break ;
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
PATCH 6/10 selftests: KVM: Add counter emulation benchmark
In reply to Oliver UptonAdd a test case for counter emulation on arm64. A side effect of how KVM handles physical counter offsetting on non-ECV systems is that the virtual counter will always hit hardware and the physical could be emulated. Force emulation by writing a nonzero offset to the physical counter and compare the elapsed cycles to a direct read of the hardware register.
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../kvm/aarch64/counter_emulation_benchmark.c | 209 ++++++++++++++++++ 3 files changed, 211 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c
tools/testing/selftests/kvm/.gitignore +1 -0
@@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only +/aarch64/counter_emulation_benchmark /aarch64/get-reg-list /aarch64/get-reg-list-sve /aarch64/vgic_init
tools/testing/selftests/kvm/Makefile +1 -0
@@ -77,6 +77,7 @@ TEST_GEN_PROGS_x86_64 += memslot_modification_stress_test TEST_GEN_PROGS_x86_64 += set_memory_region_test TEST_GEN_PROGS_x86_64 += steal_time +TEST_GEN_PROGS_aarch64 += aarch64/counter_emulation_benchmark TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list-sve TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c new file +209 -0
209 lines not on this page · raw message
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
PATCH 7/10 KVM: x86: Refactor tsc synchronization code
In reply to Oliver UptonRefactor kvm_synchronize_tsc to make a new function that allows callers to specify TSC parameters (offset, value, nanoseconds, etc.) explicitly for the sake of participating in TSC synchronization.
This changes the locking semantics around TSC writes. Writes to the TSC will now take the pvclock gtod lock while holding the tsc write lock, whereas before these locks were disjoint.
--- Documentation/virt/kvm/locking.rst | 11 +++ arch/x86/kvm/x86.c | 106 +++++++++++++++++------------ 2 files changed, 74 insertions(+), 43 deletions(-)
Documentation/virt/kvm/locking.rst +11 -0
@@ -25,6 +25,9 @@ On x86: holding kvm->arch.mmu_lock (typically with ``read_lock``, otherwise there's no need to take kvm->arch.tdp_mmu_pages_lock at all). +- kvm->arch.tsc_write_lock is taken outside + kvm->arch.pvclock_gtod_sync_lock + Everything else is a leaf: no other lock is taken inside the critical sections.
@@ -211,6 +214,14 @@ time it will be set using the Dirty tracking mechanism described above. :Comment: 'raw' because hardware enabling/disabling must be atomic /wrt migration. +:Name: kvm_arch::pvclock_gtod_sync_lock +:Type: raw_spinlock_t +:Arch: x86 +:Protects: kvm_arch::{cur_tsc_generation,cur_tsc_nsec,cur_tsc_write, + cur_tsc_offset,nr_vcpus_matched_tsc} +:Comment: 'raw' because updating the kvm master clock must not be + preempted. + :Name: kvm_arch::tsc_write_lock :Type: raw_spinlock :Arch: x86
arch/x86/kvm/x86.c +63 -43
127 lines not on this page · raw message
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
PATCH 8/10 KVM: x86: Implement KVM_CAP_SYSTEM_COUNTER_STATE
In reply to Oliver UptonTo date, VMM-directed TSC synchronization and migration has been messy. KVM has some baked-in heuristics around TSC writes to infer if the VMM is attempting to synchronize. This is problematic, as it depends on the host writing to the guest's TSC within 1 second of the last write.
A much cleaner approach to configuring the guest's views of the TSC is to simply migrate the TSC offset for every vCPU. Offsets are idempotent, and thus are not subject to change depending on when the VMM actually reads the values from KVM. The VMM can then read the TSC once to capture the instant at which the guest's TSCs are paused.
Implement the KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls and advertise KVM_CAP_SYSTEM_COUNTER_STATE to userspace.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 8 ++++ arch/x86/kvm/x86.c | 70 +++++++++++++++++++++++++++++++++ 3 files changed, 79 insertions(+)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1018,6 +1018,7 @@ struct kvm_arch {u64 last_tsc_nsec;u64 last_tsc_write;u32 last_tsc_khz; +u64 last_tsc_offset;u64 cur_tsc_nsec;u64 cur_tsc_write;u64 cur_tsc_offset;
arch/x86/include/uapi/asm/kvm.h +8 -0
@@ -490,4 +490,12 @@ struct kvm_pmu_event_filter {#define KVM_PMU_EVENT_ALLOW 0#define KVM_PMU_EVENT_DENY 1 +/* for KVM_CAP_SYSTEM_COUNTER_STATE */ +struct kvm_system_counter_state { +__u32 flags; +__u32 pad; +__u64 tsc_offset; +__u64 rsvd[6 ]; +}; +#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/x86.c +70 -0
100 lines not on this page · raw message
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
PATCH 9/10 selftests: KVM: Add support for x86 to system_counter_state_test
In reply to Oliver UptonTest that userspace manipulation of the tsc offset (via KVM_{GET,SET}_SYSTEM_COUNTER_STATE) results in the expected guest view of the TSC.
--- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/system_counter_state_test.c | 32 +++++++++++++++++++ 2 files changed, 33 insertions(+)
tools/testing/selftests/kvm/Makefile +1 -0
@@ -76,6 +76,7 @@ TEST_GEN_PROGS_x86_64 += kvm_page_table_test TEST_GEN_PROGS_x86_64 += memslot_modification_stress_test TEST_GEN_PROGS_x86_64 += set_memory_region_test TEST_GEN_PROGS_x86_64 += steal_time +TEST_GEN_PROGS_x86_64 += system_counter_state_test TEST_GEN_PROGS_aarch64 += aarch64/counter_emulation_benchmark TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
tools/testing/selftests/kvm/system_counter_state_test.c +32 -0
@@ -138,6 +138,38 @@ static uint64_t host_read_guest_counter(struct system_counter_state_test *test) return r; } +#elif __x86_64__ + +/* no additional information required beyond the counter state. */ +#define system_counter_state_test kvm_system_counter_state + +static struct system_counter_state_test test_cases[] = { + { .tsc_offset =0 }, + { .tsc_offset =1000000 }, + { .tsc_offset =-1 }, +}; + +static void pr_test (struct system_counter_state_test *test) +{ +printf ("tsc_offset: %lld\n" , test->tsc_offset); +} + +static struct kvm_system_counter_state * +get_system_counter_state (struct system_counter_state_test *test) +{ +return test; +} + +static uint64_t guest_read_counter (struct system_counter_state_test *test) +{ +return rdtsc (); +} + +static uint64_t host_read_guest_counter (struct system_counter_state_test *test) +{ +return rdtsc () + test->tsc_offset; +} +#else #error test not implemented for architecture being built!#endif
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
PATCH 10/10 Documentation: KVM: Document KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls
In reply to Oliver Upton--- Documentation/virt/kvm/api.rst | 98 ++++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+)
Documentation/virt/kvm/api.rst +98 -0
@@ -5034,6 +5034,104 @@ see KVM_XEN_VCPU_SET_ATTR above. The KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_ADJUST type may not be used with the KVM_XEN_VCPU_GET_ATTR ioctl. +4.130 KVM_GET_SYSTEM_COUNTER_STATE +--------------------------- + +:Capability: KVM_CAP_SYSTEM_COUNTER_STATE +:Architectures: arm64, x86 +:Type: vcpu ioctl +:Parameters: struct kvm_system_counter_state +:Returns: 0 on success, < 0 on error + +Allows the vCPU counter state to be read. Each architecture defines +its own kvm_system_counter_state structure depending on the backing hardware +controls used for the guest's counter. + +ARM64 + +:: + + struct kvm_system_counter_state { + /* indicates what fields are valid in the structure */ + __u32 flags; + +Enumerates what fields are valid in the kvm_system_counter_state structure. +Userspace should set this field to indicate what fields it wants the kernel +to populate. + +:: + + __u32 pad; + /* + * Guest physical counter-timer offset, relative to host cntpct_el0. + * Valid when KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET is set. + */ + __u64 cntvoff; + +Offset for the guest virtual counter-timer, as it relates to the host's +physical counter-timer (CNTPCT_EL0). This field is populated when the +KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET bit is set in the flags field. + +:: + + /* guest physical counter-timer offset, relative to host cntpct_el0 */ + __u64 cntpoff; + +Offset for the guest physical counter-timer, as it relates to the host's +physical counter-timer (CNTPCT_EL0). + +:: + + __u64 rsvd[5]; + }; + +x86 + +:: + + struct kvm_system_counter_state { + __u32 flags; + +Enumerates what fields are valid in the kvm_system_counter_state structure. +Currently, the structure has not been extended, so there are no valid flag +bits. This field should then be set to zero. + +:: + + __u32 pad; + __u64 tsc_offset; + +Offset for the guest TSC, as it relates to the host's TSC. + +:: + + __u64 rsvd[6]; + }; + +4.131 KVM_SET_SYSTEM_COUNTER_STATE +--------------------------- + +:Capability: KVM_CAP_SYSTEM_COUNTER_STATE +:Architectures: arm64, x86 +:Type: vcpu ioctl +:Parameters: struct kvm_system_counter_state +:Returns: 0 on success, < 0 on error. + +Allows the vCPU counter state to be written. For more details on the +structure, see KVM_GET_SYSTEM_COUNTER_STATE above. + +ARM64 + +VMMs should either use this ioctl *OR* directly write to the vCPU's +CNTVCT_EL0 register. Mixing both methods of restoring counter state +can cause drift between virtual CPUs. + +x86 + +VMMs should either use this ioctl *OR* directly write to the vCPU's +IA32_TSC register. Mixing both methods of restoring TSC state can +cause drift between virtual CPUs. + 5. The kvm_run structure ========================
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
Paolo Bonzini
maintainerQuoted 13 lines · Oliver Upton
KVM's current means of saving/restoring system counters is plagued with temporal issues. At least on ARM64 and x86, we migrate the guest's system counter by-value through the respective guest system register values (
cntvct_el0,ia32_tsc). Restoring system counters by-value is brittle as the state is not idempotent: the host system counter is still oscillating between the attempted save and restore. Furthermore, VMMs may wish to transparently live migrate guest VMs, meaning that they include the elapsed time due to live migration blackout in the guest system counter view. The VMM thread could be preempted for any number of reasons (scheduler, L0 hypervisor under nested) between the time that it calculates the desired guest counter value and when KVM actually sets this counter state.
Quoted 6 lines
Despite the value-based interface that we present to userspace, KVM actually has idempotent guest controls by way of system counter offsets. We can avoid all of the issues associated with a value-based interface by abstracting these offset controls in new ioctls. This series introduces KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls, meant to provide userspace with idempotent controls of the guest system counter.
Hi Oliver,
I wonder how this compares to the idea of initializing the TSC via a synchronized (nanoseconds, TSC) pair. (https://lore.kernel.org/r/[email protected]), and whether it makes sense to apply that idea to ARM as well. If so, it certainly is a good idea to use the same capability and ioctl, even though the details of the struct would be architecture-dependent.
In your patches there isn't much architecture dependency in struct kvm_system_counter_state. However, Maxim's also added an MSR_IA32_TSC_ADJUST value to the struct, thus ensuring that the host could write not just an arbitrary TSC value, but also tie it to an arbitrary MSR_IA32_TSC_ADJUST value. Specifying both in the same ioctl simplifies the userspace API.
Paolo
Quoted 100 lines
Patch 1 defines the ioctls, and was separated from the two provided implementations for the sake of review. If it is more intuitive, this patch can be squashed into the implementation commit.
Patch 2 realizes initial support for ARM64, migrating only the state associated with the guest's virtual counter-timer. Patch 3 introduces a KVM selftest to assert that userspace manipulation via the aforementioned ioctls produces the expected system counter values within the guest.
Patch 4 extends upon the ARM64 implementation by adding support for physical counter-timer offsetting. This is currently backed by a trap-and-emulate implementation, but can also be virtualized in hardware that fully implements ARMv8.6-ECV. ECV support has been elided from this series out of convenience for the author :) Patch 5 adds some test cases to the newly-minted kvm selftest to validate expectations of physical counter-timer emulation.
Patch 6 introduces yet another KVM selftest for aarch64, intended to measure the effects of physical counter-timer emulation. Data for this test can be found below, but basically there is some tradeoff of overhead for the sake of correctness, but it isn't too bad.
Patches 7-8 add support for the ioctls to x86 by shoehorning the controls into the pre-existing synchronization heuristics. Patch 7 provides necessary helper methods for the implementation to play nice with those heuristics, and patch 8 actually implements the ioctls.
Patch 9 adds x86 test cases to the system counter KVM selftest. Lastly, patch 10 documents the ioctls for both x86 and arm64.
All patches apply cleanly to kvm/next at the following commit:
a4345a7cecfb("Merge tag 'kvmarm-fixes-5.13-1' ofgit://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarminto HEAD")Physical counter benchmark
--------------------------The following data was collected by running 10000 iterations of the benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S machine with 2 80-core Ampere Altra SoCs. Measurements were collected for both VHE and nVHE operation using the
kvm-arm.mode=command-line parameter.nVHE
----+--------------------+--------+---------+ | Metric | Native | Trapped | +--------------------+--------+---------+ | Average | 54ns | 148ns | | Standard Deviation | 124ns | 122ns | | 95th Percentile | 258ns | 348ns | +--------------------+--------+---------+VHE
---+--------------------+--------+---------+ | Metric | Native | Trapped | +--------------------+--------+---------+ | Average | 53ns | 152ns | | Standard Deviation | 92ns | 94ns | | 95th Percentile | 204ns | 307ns | +--------------------+--------+---------+Oliver Upton (10):
KVM: Introduce KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls
KVM: arm64: Implement initial support forKVM_CAP_SYSTEM_COUNTER_STATE
selftests: KVM: Introducesystem_counter_state_test
KVM: arm64: Add userspace control of the guest's physical counter
selftests: KVM: Add test cases for physical counter offsetting
selftests: KVM: Add counter emulation benchmark
KVM: x86: Refactor tsc synchronization code
KVM: x86: ImplementKVM_CAP_SYSTEM_COUNTER_STATE
selftests: KVM: Add support for x86 tosystem_counter_state_test
Documentation: KVM: Document KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctlsDocumentation/virt/kvm/api.rst | 98 +++++++ Documentation/virt/kvm/locking.rst | 11 + arch/arm64/include/asm/kvm_host.h | 6 + arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 17 ++ arch/arm64/kvm/arch_timer.c | 84 +++++- arch/arm64/kvm/arm.c | 25 ++ arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +++ arch/arm64/kvm/hyp/nvhe/timer-sr.c | 16 +- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 8 + arch/x86/kvm/x86.c | 176 +++++++++--- include/uapi/linux/kvm.h | 5 + tools/testing/selftests/kvm/.gitignore | 2 + tools/testing/selftests/kvm/Makefile | 3 + .../kvm/aarch64/counter_emulation_benchmark.c | 209 ++++++++++++++ .../selftests/kvm/include/aarch64/processor.h | 24 ++ .../selftests/kvm/system_counter_state_test.c | 256 ++++++++++++++++++ 18 files changed, 926 insertions(+), 47 deletions(-)create mode 100644
tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.ccreate mode 100644tools/testing/selftests/kvm/system_counter_state_test.c
Quoted 22 lines · Paolo Bonzini
Quoted 13 lines · Oliver Upton
KVM's current means of saving/restoring system counters is plagued with temporal issues. At least on ARM64 and x86, we migrate the guest's system counter by-value through the respective guest system register values (
cntvct_el0,ia32_tsc). Restoring system counters by-value is brittle as the state is not idempotent: the host system counter is still oscillating between the attempted save and restore. Furthermore, VMMs may wish to transparently live migrate guest VMs, meaning that they include the elapsed time due to live migration blackout in the guest system counter view. The VMM thread could be preempted for any number of reasons (scheduler, L0 hypervisor under nested) between the time that it calculates the desired guest counter value and when KVM actually sets this counter state.Quoted 6 lines
Despite the value-based interface that we present to userspace, KVM actually has idempotent guest controls by way of system counter offsets. We can avoid all of the issues associated with a value-based interface by abstracting these offset controls in new ioctls. This series introduces KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls, meant to provide userspace with idempotent controls of the guest system counter.
Quoted 8 lines
Hi Oliver,
I wonder how this compares to the idea of initializing the TSC via a synchronized (nanoseconds, TSC) pair.
(https://lore.kernel.org/r/[email protected]),
and whether it makes sense to apply that idea to ARM as well. If so, it certainly is a good idea to use the same capability and ioctl, even though the details of the struct would be architecture-dependent.
Hey Paolo,
Yeah, great question, I had actually alluded to this on [02/10] in talking to Marc about this.
Really the issue we want to avoid is sampling the host counter twice, which at least based on the existing means of counter migration is impossible as the VMM must account for elapsed time. Maxim's patches appear to address that very issue as well.
Perhaps this will clarify the motivation for my approach: what if the kernel wasn't the authoritative source for wall time in a system? Furthermore, VMMs may wish to define their own heuristics for counter migration (e.g. we only allow the counter to 'jump' by X seconds during migration blackout). If a VMM tried to assert its whims on the TSC state before handing it down to the kernel, we would inadvertently be sampling the host counter twice again. And, anything can happen between the time we assert elapsed time is within SLO and KVM computing the TSC offset (scheduling, L0 hypervisor preemption).
So, Maxim's changes would address my concerns in the general case, but maybe not as much in edge cases where an operator may make decisions about how much time can elapse while the guest hasn't had CPU time.
--
Thanks,
Oliver
Quoted 7 lines
In your patches there isn't much architecture dependency in struct
kvm_system_counter_state. However, Maxim's also added anMSR_IA32_TSC_ADJUSTvalue to the struct, thus ensuring that the host could write not just an arbitrary TSC value, but also tie it to an arbitraryMSR_IA32_TSC_ADJUSTvalue. Specifying both in the same ioctl simplifies the userspace API.
Quoted 103 lines
Paolo
Quoted 100 lines
Patch 1 defines the ioctls, and was separated from the two provided implementations for the sake of review. If it is more intuitive, this patch can be squashed into the implementation commit.
Patch 2 realizes initial support for ARM64, migrating only the state associated with the guest's virtual counter-timer. Patch 3 introduces a KVM selftest to assert that userspace manipulation via the aforementioned ioctls produces the expected system counter values within the guest.
Patch 4 extends upon the ARM64 implementation by adding support for physical counter-timer offsetting. This is currently backed by a trap-and-emulate implementation, but can also be virtualized in hardware that fully implements ARMv8.6-ECV. ECV support has been elided from this series out of convenience for the author :) Patch 5 adds some test cases to the newly-minted kvm selftest to validate expectations of physical counter-timer emulation.
Patch 6 introduces yet another KVM selftest for aarch64, intended to measure the effects of physical counter-timer emulation. Data for this test can be found below, but basically there is some tradeoff of overhead for the sake of correctness, but it isn't too bad.
Patches 7-8 add support for the ioctls to x86 by shoehorning the controls into the pre-existing synchronization heuristics. Patch 7 provides necessary helper methods for the implementation to play nice with those heuristics, and patch 8 actually implements the ioctls.
Patch 9 adds x86 test cases to the system counter KVM selftest. Lastly, patch 10 documents the ioctls for both x86 and arm64.
All patches apply cleanly to kvm/next at the following commit:
a4345a7cecfb("Merge tag 'kvmarm-fixes-5.13-1' ofgit://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarminto HEAD")Physical counter benchmark
--------------------------The following data was collected by running 10000 iterations of the benchmark test from Patch 6 on an Ampere Mt. Jade reference server, A 2S machine with 2 80-core Ampere Altra SoCs. Measurements were collected for both VHE and nVHE operation using the
kvm-arm.mode=command-line parameter.nVHE
----+--------------------+--------+---------+ | Metric | Native | Trapped | +--------------------+--------+---------+ | Average | 54ns | 148ns | | Standard Deviation | 124ns | 122ns | | 95th Percentile | 258ns | 348ns | +--------------------+--------+---------+VHE
---+--------------------+--------+---------+ | Metric | Native | Trapped | +--------------------+--------+---------+ | Average | 53ns | 152ns | | Standard Deviation | 92ns | 94ns | | 95th Percentile | 204ns | 307ns | +--------------------+--------+---------+Oliver Upton (10):
KVM: Introduce KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls
KVM: arm64: Implement initial support forKVM_CAP_SYSTEM_COUNTER_STATE
selftests: KVM: Introducesystem_counter_state_test
KVM: arm64: Add userspace control of the guest's physical counter
selftests: KVM: Add test cases for physical counter offsetting
selftests: KVM: Add counter emulation benchmark
KVM: x86: Refactor tsc synchronization code
KVM: x86: ImplementKVM_CAP_SYSTEM_COUNTER_STATE
selftests: KVM: Add support for x86 tosystem_counter_state_test
Documentation: KVM: Document KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctlsDocumentation/virt/kvm/api.rst | 98 +++++++ Documentation/virt/kvm/locking.rst | 11 + arch/arm64/include/asm/kvm_host.h | 6 + arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 17 ++ arch/arm64/kvm/arch_timer.c | 84 +++++- arch/arm64/kvm/arm.c | 25 ++ arch/arm64/kvm/hyp/include/hyp/switch.h | 31 +++ arch/arm64/kvm/hyp/nvhe/timer-sr.c | 16 +- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 8 + arch/x86/kvm/x86.c | 176 +++++++++--- include/uapi/linux/kvm.h | 5 + tools/testing/selftests/kvm/.gitignore | 2 + tools/testing/selftests/kvm/Makefile | 3 + .../kvm/aarch64/counter_emulation_benchmark.c | 209 ++++++++++++++ .../selftests/kvm/include/aarch64/processor.h | 24 ++ .../selftests/kvm/system_counter_state_test.c | 256 ++++++++++++++++++ 18 files changed, 926 insertions(+), 47 deletions(-)create mode 100644
tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.ccreate mode 100644tools/testing/selftests/kvm/system_counter_state_test.c
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Paolo Bonzini
maintainerQuoted 10 lines · Oliver Upton
Perhaps this will clarify the motivation for my approach: what if the kernel wasn't the authoritative source for wall time in a system? Furthermore, VMMs may wish to define their own heuristics for counter migration (e.g. we only allow the counter to 'jump' by X seconds during migration blackout). If a VMM tried to assert its whims on the TSC state before handing it down to the kernel, we would inadvertently be sampling the host counter twice again. And, anything can happen between the time we assert elapsed time is within SLO and KVM computing the TSC offset (scheduling, L0 hypervisor preemption).
Quoted 3 lines
So, Maxim's changes would address my concerns in the general case, but maybe not as much in edge cases where an operator may make decisions about how much time can elapse while the guest hasn't had CPU time.
I think I understand. We still need a way to get a consistent (host_TSC, nanosecond) pair on the source, the TSC offset is not enough.
This is arguably not a KVM issue, but we're still the one having to
provide a solution, so we would need a slightly more complicated interface.
1) In the kernel:
* KVM_GET_CLOCK should also return kvmclock_ns - realtime_ns and host_TSC. It should set two flags in struct kvm_clock_data saying that the respective fields are valid.
* KVM_SET_CLOCK checks the flag for kvmclock_ns - realtime_ns. If set, it looks at the kvmclock_ns - realtime_ns field and disregards the kvmclock_ns field.
2) On the source, userspace will:
* per-VM: invoke KVM_GET_CLOCK. Migrate kvmclock_ns - realtime_ns and kvmclock_ns. Stash host_TSC for subsequent use.
* per-vCPU: retrieve guest_TSC - host_TSC with your new ioctl. Sum it to the stashed host_TSC value; migrate the resulting value (a guest TSC).
3) On the destination:
* per-VM: Pass the migrated kvmclock_ns - realtime_ns to KVM_SET_CLOCK.
Use KVM_GET_CLOCK to get a consistent pair of kvmclock_ns ("newNS"
below) and host TSC ("newHostTSC"). Stash them for subsequent use, together with the migrated kvmclock_ns value ("sourceNS") that you haven't used yet.
* per-vCPU: using the data of the previous step, and the sourceGuestTSC in the migration stream, compute sourceGuestTSC + (newNS - sourceNS) * freq - newHostTSC. This is the TSC offset to be passed to your new ioctl.
Your approach still needs to use the "quirky" approach to host-initiated MSR_IA32_TSC_ADJUST writes, which write the MSR without affecting the VMCS offset. This is just a documentation issue.
Does this make sense?
Paolo
Quoted 12 lines · Paolo Bonzini
Quoted 10 lines · Oliver Upton
Perhaps this will clarify the motivation for my approach: what if the kernel wasn't the authoritative source for wall time in a system? Furthermore, VMMs may wish to define their own heuristics for counter migration (e.g. we only allow the counter to 'jump' by X seconds during migration blackout). If a VMM tried to assert its whims on the TSC state before handing it down to the kernel, we would inadvertently be sampling the host counter twice again. And, anything can happen between the time we assert elapsed time is within SLO and KVM computing the TSC offset (scheduling, L0 hypervisor preemption).
Quoted 8 lines
Quoted 3 lines
So, Maxim's changes would address my concerns in the general case, but maybe not as much in edge cases where an operator may make decisions about how much time can elapse while the guest hasn't had CPU time.
I think I understand. We still need a way to get a consistent (
host_TSC, nanosecond) pair on the source, the TSC offset is not enough.
This is arguably not a KVM issue, but we're still the one having to
provide a solution, so we would need a slightly more complicated interface.
Ah, right, good luck doing that without some help from the kernel. +1 to this _not_ being a KVM issue, but anyways...
Quoted 9 lines
1) In the kernel:
*
KVM_GET_CLOCKshould also returnkvmclock_ns-realtime_nsandhost_TSC. It should set two flags in structkvm_clock_datasaying that the respective fields are valid.*
KVM_SET_CLOCKchecks the flag forkvmclock_ns-realtime_ns. If set, it looks at thekvmclock_ns-realtime_nsfield and disregards thekvmclock_nsfield.
Yeah, these additions all make sense to me.
Quoted 17 lines
2) On the source, userspace will:
* per-VM: invoke
KVM_GET_CLOCK. Migratekvmclock_ns-realtime_nsandkvmclock_ns. Stashhost_TSCfor subsequent use.* per-vCPU: retrieve
guest_TSC-host_TSCwith your new ioctl. Sum it to the stashedhost_TSCvalue; migrate the resulting value (a guest TSC).3) On the destination:
* per-VM: Pass the migrated
kvmclock_ns-realtime_nstoKVM_SET_CLOCK. UseKVM_GET_CLOCKto get a consistent pair ofkvmclock_ns("newNS"
below) and host TSC ("newHostTSC"). Stash them for subsequent use, together with the migratedkvmclock_nsvalue ("sourceNS") that you haven't used yet.
Quoted 7 lines
* per-vCPU: using the data of the previous step, and the sourceGuestTSC in the migration stream, compute sourceGuestTSC + (newNS - sourceNS) * freq - newHostTSC. This is the TSC offset to be passed to your new ioctl.
Your approach still needs to use the "quirky" approach to host-initiated
MSR_IA32_TSC_ADJUSTwrites, which write the MSR without affecting the VMCS offset. This is just a documentation issue.
I think I follow what you're saying. To confirm:
My suggested ioctl for the vCPU will still exist, and it will still affect the VMCS tsc offset, right? However, we need to do one of the following:
- Stash the guest's MSR_IA32_TSC_ADJUST value in the kvm_system_counter_state structure. During KVM_SET_SYSTEM_COUNTER_STATE, check to see if the field is valid. If so, treat it as a dumb value (i.e. show it to the guest but don't fold it into the offset).
- Inform userspace that it must still migrate MSR_IA32_TSC_ADJUST, and continue to our quirky behavior around host-initiated writes of the MSR.
This is why Maxim's spin migrated a value for IA32_TSC_ADJUST, right? Doing so ensures we don't have any guest-observable consequences due to our migration of TSC state. To me, adding the guest IA32_TSC_ADJUST MSR into the new counter state structure is probably best. No strong opinions in either direction on this point, though :)
--
Thanks,
Oliver
Quoted 5 lines
Does this make sense?
Paolo
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Paolo Bonzini
maintainerQuoted 8 lines · Oliver Upton
Quoted 3 lines
Your approach still needs to use the "quirky" approach to host-initiated
MSR_IA32_TSC_ADJUSTwrites, which write the MSR without affecting the VMCS offset. This is just a documentation issue.My suggested ioctl for the vCPU will still exist, and it will still affect the VMCS tsc offset, right? However, we need to do one of the following:
Quoted 5 lines
- Stash the guest's
MSR_IA32_TSC_ADJUSTvalue in thekvm_system_counter_statestructure. DuringKVM_SET_SYSTEM_COUNTER_STATE, check to see if the field is valid. If so, treat it as a dumb value (i.e. show it to the guest but don't fold it into the offset).
Yes, it's already folded into the guestTSC-hostTSC offset that the caller provides.
Quoted 5 lines
- Inform userspace that it must still migrate
MSR_IA32_TSC_ADJUST, and continue to our quirky behavior around host-initiated writes of the MSR.This is why Maxim's spin migrated a value for
IA32_TSC_ADJUST, right?
Yes, so that he could then remove (optionally) the quirk for host-initiated writes to the TSC and TSC_ADJUST MSRs.
Quoted 4 lines
Doing so ensures we don't have any guest-observable consequences due to our migration of TSC state. To me, adding the guest
IA32_TSC_ADJUSTMSR into the new counter state structure is probably best. No strong opinions in either direction on this point, though:)
Either is good for me, since documentation will be very important either way. This is a complex API to use due to the possibility of skewed TSCs.
Just one thing, please don't introduce a new ioctl and use KVM_GET_DEVICE_ATTR/KVM_SET_DEVICE_ATTR/KVM_HAS_DEVICE_ATTR.
Christian, based on what Oliver mentions here, it's probably useful for s390 to have functionality to get/set kvm->arch.epoch and kvm->arch.epdx in addition to the absolute TOD values that you are migrating now.
Paolo
Quoted 29 lines
Quoted 29 lines
1) In the kernel:
*
KVM_GET_CLOCKshould also returnkvmclock_ns-realtime_nsandhost_TSC. It should set two flags in structkvm_clock_datasaying that the respective fields are valid.*
KVM_SET_CLOCKchecks the flag forkvmclock_ns-realtime_ns. If set, it looks at thekvmclock_ns-realtime_nsfield and disregards thekvmclock_nsfield.2) On the source, userspace will:
* per-VM: invoke
KVM_GET_CLOCK. Migratekvmclock_ns-realtime_nsandkvmclock_ns. Stashhost_TSCfor subsequent use.* per-vCPU: retrieve
guest_TSC-host_TSCwith your new ioctl. Sum it to the stashedhost_TSCvalue; migrate the resulting value (a guest TSC).3) On the destination:
* per-VM: Pass the migrated
kvmclock_ns-realtime_nstoKVM_SET_CLOCK. UseKVM_GET_CLOCKto get a consistent pair ofkvmclock_ns("newNS"
below) and host TSC ("newHostTSC"). Stash them for subsequent use, together with the migratedkvmclock_nsvalue ("sourceNS") that you haven't used yet.* per-vCPU: using the data of the previous step, and the sourceGuestTSC in the migration stream, compute sourceGuestTSC + (newNS - sourceNS) * freq - newHostTSC. This is the TSC offset to be passed to your new ioctl.
Quoted 30 lines · Paolo Bonzini
Quoted 8 lines · Oliver Upton
Quoted 3 lines
Your approach still needs to use the "quirky" approach to host-initiated
MSR_IA32_TSC_ADJUSTwrites, which write the MSR without affecting the VMCS offset. This is just a documentation issue.My suggested ioctl for the vCPU will still exist, and it will still affect the VMCS tsc offset, right? However, we need to do one of the following:
Quoted 5 lines
- Stash the guest's
MSR_IA32_TSC_ADJUSTvalue in thekvm_system_counter_statestructure. DuringKVM_SET_SYSTEM_COUNTER_STATE, check to see if the field is valid. If so, treat it as a dumb value (i.e. show it to the guest but don't fold it into the offset).Yes, it's already folded into the guestTSC-hostTSC offset that the caller provides.
Quoted 5 lines
- Inform userspace that it must still migrate
MSR_IA32_TSC_ADJUST, and continue to our quirky behavior around host-initiated writes of the MSR.This is why Maxim's spin migrated a value for
IA32_TSC_ADJUST, right?Yes, so that he could then remove (optionally) the quirk for host-initiated writes to the TSC and
TSC_ADJUSTMSRs.Quoted 4 lines
Doing so ensures we don't have any guest-observable consequences due to our migration of TSC state. To me, adding the guest
IA32_TSC_ADJUSTMSR into the new counter state structure is probably best. No strong opinions in either direction on this point, though:)
Quoted 5 lines
Either is good for me, since documentation will be very important either way. This is a complex API to use due to the possibility of skewed TSCs.
Just one thing, please don't introduce a new ioctl and use KVM_GET_DEVICE_ATTR/KVM_SET_DEVICE_ATTR/KVM_HAS_DEVICE_ATTR.
Christian, based on what Oliver mentions here, it's probably useful for s390 to have functionality to get/set
kvm->arch.epochandkvm->arch.epdxin addition to the absolute TOD values that you are migrating now.
Yes, a scheme where we migrate the offsets (assuming that the hosts are synced) would be often better. If the hosts are not synced, things will be harder. I will have a look at this series, Thanks for the pointer.