Skip to content

KVM: x86: Add idempotent controls for migrating system counter state

Oliver Upton kvm kvmarm linux-arm-kernel 8 versions – Applied

Status from this message: Queued, thanks.

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v8 Latest7 patches 32 repliesApplied
  2. v76 patches 14 repliesSuperseded
  3. v621 patches 29 repliesSuperseded
Earlier versions · 5
VersionPostedPatchesRepliesStatus
v513 patches13 repliesSuperseded
v413 patches5 repliesSuperseded
v312 patches5 repliesSuperseded
v212 patches7 repliesSuperseded
v110 patches12 repliesSuperseded

Changes in v8 from the cover letter

- Rebased to 5.15-rc1
- Picked up Paolo's version of the series, which includes locking changes
- Make KVM advertise KVM_CAP_VCPU_ATTRIBUTES

Patches in v8 Applied

#Subject and filesSizeRepliesApplied
1/7kvm: x86: abstract locking around pvclock_update_vm_gtod_copyarch/x86/include/asm/kvm_host.h arch/x86/kvm/x86.c+29 -340 replies
2/7KVM: x86: extract KVM_GET_CLOCK/KVM_SET_CLOCK to separate functionsarch/x86/kvm/x86.c+52 -470 replies
3/7KVM: x86: Fix potential race in KVM_GET_CLOCKarch/x86/kvm/x86.c+23 -131 reply
4/7KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCKDocumentation/virt/kvm/api.rst arch/x86/include/asm/kvm_host.h arch/x86/kvm/x86.c +1 more+70 -1817 replies
5/7kvm: x86: protect masterclock with a seqcountarch/x86/include/asm/kvm_host.h arch/x86/kvm/x86.c+49 -413 replies
6/7KVM: x86: Refactor tsc synchronization codearch/x86/kvm/x86.c+58 -420 replies
7/7KVM: x86: Expose TSC offset controls to userspaceDocumentation/virt/kvm/devices/vcpu.rst arch/x86/include/asm/kvm_host.h arch/x86/include/uapi/asm/kvm.h +1 more+172 -010 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v8 yet.

Read the v8 thread

Cover letter [PATCH v8 0/7] KVM: x86: Add idempotent controls for migrating system counter state

KVM's current means of saving/restoring system counters is plagued with temporal issues. On x86, we migrate the guest's system counter by-value through the respective guest's IA32_TSC value. Restoring system counters by-value is brittle as the state is not idempotent: the host system counter is still oscillating between the attempted save and restore. Furthermore, VMMs may wish to transparently live migrate guest VMs, meaning that they include the elapsed time due to live migration blackout in the guest system counter view. The VMM thread could be preempted for any number of reasons (scheduler, L0 hypervisor under nested) between the time that it calculates the desired guest counter value and when KVM actually sets this counter state.

Despite the value-based interface that we present to userspace, KVM actually has idempotent guest controls by way of the TSC offset. We can avoid all of the issues associated with a value-based interface by abstracting these offset controls in a new device attribute. This series introduces new vCPU device attributes to provide userspace access to the vCPU's system counter offset.

Patches 1-2 are Paolo's refactorings around locking and the KVM_{GET,SET}_CLOCK ioctls.

Patch 3 cures a race where use_master_clock is read outside of the pvclock lock in the KVM_GET_CLOCK ioctl.

Patch 4 adopts Paolo's suggestion, augmenting the KVM_{GET,SET}_CLOCK ioctls to provide userspace with a (host_tsc, realtime) instant. This is essential for a VMM to perform precise migration of the guest's system counters.

Patch 5 does away with the pvclock spin lock in favor of a sequence lock based on the tsc_write_lock. The original patch is from Paolo, I touched it up a bit to fix a deadlock and some unused variables that caused -Werror to scream.

Patch 6 extracts the TSC synchronization tracking code in a way that it can be used for both offset-based and value-based TSC synchronization schemes.

Finally, patch 7 implements a vCPU device attribute which allows VMMs to get at the TSC offset of a vCPU.

This series was tested with the new KVM selftests for the KVM clock and system counter offset controls on Haswell hardware. Kernel was built with CONFIG_LOCKDEP given the new locking changes/lockdep assertions here.

Note that these tests are mailed as a separate series due to the dependencies in both x86 and arm64.

Applies cleanly to 5.15-rc1

v8: http://lore.kernel.org/r/[email protected]

v7 -> v8:
- Rebased to 5.15-rc1
- Picked up Paolo's version of the series, which includes locking changes
- Make KVM advertise KVM_CAP_VCPU_ATTRIBUTES

Oliver Upton (4):
KVM: x86: Fix potential race in KVM_GET_CLOCK
KVM: x86: Report host tsc and realtime values in KVM_GET_CLOCK
KVM: x86: Refactor tsc synchronization code
KVM: x86: Expose TSC offset controls to userspace

Paolo Bonzini (3):
kvm: x86: abstract locking around pvclock_update_vm_gtod_copy
KVM: x86: extract KVM_GET_CLOCK/KVM_SET_CLOCK to separate functions
kvm: x86: protect masterclock with a seqcount

 Documentation/virt/kvm/api.rst          |  42 ++-
 Documentation/virt/kvm/devices/vcpu.rst |  57 +++
 arch/x86/include/asm/kvm_host.h         |  12 +-
 arch/x86/include/uapi/asm/kvm.h         |   4 +
 arch/x86/kvm/x86.c                      | 458 ++++++++++++++++--------
 include/uapi/linux/kvm.h                |   7 +-
 6 files changed, 419 insertions(+), 161 deletions(-)
Signature · 6 lines
2.33.0.309.g3052b89438-goog

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm