KVM: x86: Include host suspended time in steal time
Subsystems (5)
v7 cover KVM: x86: Include host suspended time in steal time
This series makes it so that the time that the host is suspended is included in guests' steal time.
When the host resumes from a suspend, the guest thinks any task that was running during the suspend ran for a long time, even though the effective run time was much shorter, which can end up having negative effects with scheduling.
To mitigate this issue, include the time that the host was suspended in steal time, if the guest requests it, which lets the guest subtract the duration from the tasks' runtime. Add new ABI to make this behavior opt-in per-guest.
In addition, make the guest TSC behavior consistent whether the host TSC went backwards or not.
v7:
- Fix build.
- Make advancing TSC dependent on X86_64.
v6: https://lore.kernel.org/kvm/[email protected]/
- Use true/false for bools.
- Indentation.
- Remove superfluous flag.
- Use atomic operations for accumulating suspend duration.
- Reuse generic vcpu block/kick infrastructure instead of rolling our own.
- Add ABI to make the behavior opt-in per-guest.
- Add command line parameter to make guest use this.
- Reword commit messages in imperative mood.
v5: https://lore.kernel.org/kvm/[email protected]/
- Fix grammar mistakes in commit message.
v4: https://lore.kernel.org/kvm/[email protected]/
- Advance guest TSC on suspends where host TSC goes backwards.
- Block vCPUs from running until resume notifier.
- Move suspend duration accounting out of machine-independent kvm to x86.
- Merge code and documentation patches.
- Reworded documentation.
v3: https://lore.kernel.org/kvm/[email protected]/
- Use PM notifier instead of syscore ops (kvm_suspend()/kvm_resume()), because the latter doesn't get called on shallow suspend.
- Don't call function under UACCESS.
- Whitespace.
v2: https://lore.kernel.org/kvm/[email protected]/
- Accumulate suspend time at machine-independent kvm layer and track per-VCPU instead of per-VM.
- Document changes.
v1: https://lore.kernel.org/kvm/[email protected]/
Suleiman Souhlal (3):
KVM: x86: Advance guest TSC after deep suspend.
KVM: x86: Include host suspended duration in steal time
KVM: x86: Add "suspendsteal" cmdline to request host to add suspend
duration in steal time
.../admin-guide/kernel-parameters.txt | 5 + Documentation/virt/kvm/x86/cpuid.rst | 4 + Documentation/virt/kvm/x86/msr.rst | 14 +++ arch/x86/include/asm/kvm_host.h | 6 + arch/x86/include/uapi/asm/kvm_para.h | 2 + arch/x86/kernel/kvm.c | 15 +++ arch/x86/kvm/cpuid.c | 4 +- arch/x86/kvm/x86.c | 112 +++++++++++++++++- 8 files changed, 156 insertions(+), 6 deletions(-)
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
PATCH v7 1/3 KVM: x86: Advance guest TSC after deep suspend.
Try to advance guest TSC to current time after suspend when the host TSCs went backwards.
This makes the behavior consistent between suspends where host TSC resets and suspends where it doesn't, such as suspend-to-idle, where in the former case if the host TSC resets, the guests' would previously be "frozen" due to KVM's backwards TSC prevention, while in the latter case they would advance.
--- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/x86.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+)
arch/x86/include/asm/kvm_host.h +3 -0
@@ -1414,6 +1414,9 @@ struct kvm_arch {u64 cur_tsc_offset;u64 cur_tsc_generation;int nr_vcpus_matched_tsc; +#ifdef CONFIG_X86_64 +bool host_was_suspended; +#endif u32 default_tsc_khz;bool user_set_tsc;
arch/x86/kvm/x86.c +32 -0
@@ -5035,7 +5035,36 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)/* Apply any externally detected TSC adjustments (due to suspend) */ if (unlikely (vcpu->arch.tsc_offset_adjustment)) { +#ifdef CONFIG_X86_64 +unsigned long flags; +struct kvm *kvm; +bool advance; +u64 kernel_ns, l1_tsc, offset, tsc_now; + + kvm = vcpu->kvm; + advance =kvm_get_time_and_clockread (&kernel_ns, &tsc_now); +raw_spin_lock_irqsave (&kvm->arch.tsc_write_lock, flags); +/* +* Advance the guest's TSC to current time instead of only +* preventing it from going backwards, while making sure +* all the vCPUs use the same offset. +*/ +if (kvm->arch.host_was_suspended && advance) { + l1_tsc =nsec_to_cycles (vcpu, + kvm->arch.kvmclock_offset + kernel_ns); + offset =kvm_compute_l1_tsc_offset (vcpu, l1_tsc); + kvm->arch.cur_tsc_offset = offset; +kvm_vcpu_write_tsc_offset (vcpu, offset); + }else if (advance) { +kvm_vcpu_write_tsc_offset (vcpu, kvm->arch.cur_tsc_offset); + }else { +adjust_tsc_offset_host (vcpu, vcpu->arch.tsc_offset_adjustment); + } + kvm->arch.host_was_suspended = false; +raw_spin_unlock_irqrestore (&kvm->arch.tsc_write_lock, flags); +#else adjust_tsc_offset_host (vcpu, vcpu->arch.tsc_offset_adjustment); +#endif /* CONFIG_X86_64 */ vcpu->arch.tsc_offset_adjustment =0 ;kvm_make_request (KVM_REQ_CLOCK_UPDATE , vcpu); }
@@ -12729,6 +12758,9 @@ int kvm_arch_enable_virtualization_cpu(void)kvm_make_request (KVM_REQ_MASTERCLOCK_UPDATE , vcpu); } +#ifdef CONFIG_X86_64 + kvm->arch.host_was_suspended = true; +#endif /* * We have to disable TSC offset matching.. if you were * booting a VM while issuing an S4 host suspend....
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
Re: [PATCH v7 1/3] KVM: x86: Advance guest TSC after deep suspend.
Quoted 11 lines · Suleiman Souhlal
Try to advance guest TSC to current time after suspend when the host TSCs went backwards.
This makes the behavior consistent between suspends where host TSC resets and suspends where it doesn't, such as suspend-to-idle, where in the former case if the host TSC resets, the guests' would previously be "frozen" due to KVM's backwards TSC prevention, while in the latter case they would advance.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Suleiman Souhlal <[email protected]>
Tested again with comparing date before and after suspend-to-RAM:
echo deep >/sys/power/mem_sleep
echo $(date '+%s' -d '+3 minutes') >/sys/class/rtc/rtc0/wakealarm echo mem >/sys/power/state
Without the patch, the guest's date is slower (~3 mins) than the host's after resuming.
Quoted 24 lines
arch/x86/kvm/x86.c +20
-0@@ -5035,7 +5035,36 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)/* Apply any externally detected TSC adjustments (due to suspend) */ if (unlikely (vcpu->arch.tsc_offset_adjustment)) { +#ifdef CONFIG_X86_64 +unsigned long flags; +struct kvm *kvm; +bool advance; +u64 kernel_ns, l1_tsc, offset, tsc_now; + + kvm = vcpu->kvm; + advance =kvm_get_time_and_clockread (&kernel_ns, &tsc_now); +raw_spin_lock_irqsave (&kvm->arch.tsc_write_lock, flags); +/* +* Advance the guest's TSC to current time instead of only +* preventing it from going backwards, while making sure +* all the vCPUs use the same offset. +*/ +if (kvm->arch.host_was_suspended && advance) { + l1_tsc =nsec_to_cycles (vcpu, + kvm->arch.kvmclock_offset + kernel_ns); + offset =kvm_compute_l1_tsc_offset (vcpu, l1_tsc); + kvm->arch.cur_tsc_offset = offset; +kvm_vcpu_write_tsc_offset (vcpu, offset);
arch/x86/kvm/x86.c void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 5067
Quoted 10 lines
arch/x86/kvm/x86.c +9
-0+ } else if (advance) { +kvm_vcpu_write_tsc_offset (vcpu, kvm->arch.cur_tsc_offset); + }else { +adjust_tsc_offset_host (vcpu, vcpu->arch.tsc_offset_adjustment); + } + kvm->arch.host_was_suspended = false; +raw_spin_unlock_irqrestore (&kvm->arch.tsc_write_lock, flags); +#else adjust_tsc_offset_host (vcpu, vcpu->arch.tsc_offset_adjustment); +#endif /* CONFIG_X86_64 */
Wondering if it needs to acquire the tsc_write_lock, given that:
- The original code adjust_tsc_offset_host() doesn't acquire. Note:
adjust_tsc_offset_host() eventually calls kvm_vcpu_write_tsc_offset() too.
- Documentation/virt/kvm/locking.rst [1].
[1] https://elixir.bootlin.com/linux/v6.15/source/Documentation/virt/kvm/locking.rst#L264
Re: [PATCH v7 1/3] KVM: x86: Advance guest TSC after deep suspend.
Quoted 19 lines · Tzung-Bi Shih
Quoted 11 lines · Suleiman Souhlal
Try to advance guest TSC to current time after suspend when the host TSCs went backwards.
This makes the behavior consistent between suspends where host TSC resets and suspends where it doesn't, such as suspend-to-idle, where in the former case if the host TSC resets, the guests' would previously be "frozen" due to KVM's backwards TSC prevention, while in the latter case they would advance.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Suleiman Souhlal <[email protected]>Tested again with comparing
datebefore and after suspend-to-RAM:
echo deep >/sys/power/mem_sleep
echo $(date '+%s' -d '+3 minutes') >/sys/class/rtc/rtc0/wakealarm echo mem >/sys/power/state
Quoted 4 lines
Without the patch, the guest's
dateis slower (~3 mins) than the host's after resuming.Tested-by Tzung-Bi Shih <[email protected]>
Thanks for testing!
Quoted 41 lines
Quoted 25 lines
arch/x86/kvm/x86.c +20
-0@@ -5035,7 +5035,36 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)/* Apply any externally detected TSC adjustments (due to suspend) */ if (unlikely (vcpu->arch.tsc_offset_adjustment)) { +#ifdef CONFIG_X86_64 +unsigned long flags; +struct kvm *kvm; +bool advance; +u64 kernel_ns, l1_tsc, offset, tsc_now; + + kvm = vcpu->kvm; + advance =kvm_get_time_and_clockread (&kernel_ns, &tsc_now); +raw_spin_lock_irqsave (&kvm->arch.tsc_write_lock, flags); +/* +* Advance the guest's TSC to current time instead of only +* preventing it from going backwards, while making sure +* all the vCPUs use the same offset. +*/ +if (kvm->arch.host_was_suspended && advance) { + l1_tsc =nsec_to_cycles (vcpu, + kvm->arch.kvmclock_offset + kernel_ns); + offset =kvm_compute_l1_tsc_offset (vcpu, l1_tsc); + kvm->arch.cur_tsc_offset = offset; +kvm_vcpu_write_tsc_offset (vcpu, offset);arch/x86/kvm/x86.c void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 5067
Quoted 10 lines
arch/x86/kvm/x86.c +9
-0+ } else if (advance) { +kvm_vcpu_write_tsc_offset (vcpu, kvm->arch.cur_tsc_offset); + }else { +adjust_tsc_offset_host (vcpu, vcpu->arch.tsc_offset_adjustment); + } + kvm->arch.host_was_suspended = false; +raw_spin_unlock_irqrestore (&kvm->arch.tsc_write_lock, flags); +#else adjust_tsc_offset_host (vcpu, vcpu->arch.tsc_offset_adjustment); +#endif /* CONFIG_X86_64 */ Wondering if it needs to acquire the
tsc_write_lock, given that:
- The original codeadjust_tsc_offset_host()doesn't acquire. Note:
adjust_tsc_offset_host()eventually callskvm_vcpu_write_tsc_offset()too.
-Documentation/virt/kvm/locking.rst[1].
Quoted 1 line
[1] https://elixir.bootlin.com/linux/v6.15/source/Documentation/virt/kvm/locking.rst#L264
This is an excellent question.
I used a lock here to make sure that only one VCPU computes the offset and that all the others reuse it.
It might be doable with atomic operations, but using a lock seemed simpler to me.
I don't think it has to be tsc_write_lock specifically, but reusing it for this purpose seemed appropriate to me.
Thanks,
-- Suleiman
Re: [PATCH v7 1/3] KVM: x86: Advance guest TSC after deep suspend.
In reply to Suleiman SouhlalQuoted 59 lines · Suleiman Souhlal
Try to advance guest TSC to current time after suspend when the host TSCs went backwards.
This makes the behavior consistent between suspends where host TSC resets and suspends where it doesn't, such as suspend-to-idle, where in the former case if the host TSC resets, the guests' would previously be "frozen" due to KVM's backwards TSC prevention, while in the latter case they would advance.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Suleiman Souhlal <[email protected]>--- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/x86.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+)arch/x86/include/asm/kvm_host.h +3
-0@@ -1414,6 +1414,9 @@ struct kvm_arch {u64 cur_tsc_offset;u64 cur_tsc_generation;int nr_vcpus_matched_tsc; +#ifdef CONFIG_X86_64 +bool host_was_suspended; +#endif u32 default_tsc_khz;bool user_set_tsc;arch/x86/kvm/x86.c +20
-0@@ -5035,7 +5035,36 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)/* Apply any externally detected TSC adjustments (due to suspend) */ if (unlikely (vcpu->arch.tsc_offset_adjustment)) { +#ifdef CONFIG_X86_64 +unsigned long flags; +struct kvm *kvm; +bool advance; +u64 kernel_ns, l1_tsc, offset, tsc_now; + + kvm = vcpu->kvm; + advance =kvm_get_time_and_clockread (&kernel_ns, &tsc_now); +raw_spin_lock_irqsave (&kvm->arch.tsc_write_lock, flags); +/* +* Advance the guest's TSC to current time instead of only +* preventing it from going backwards, while making sure +* all the vCPUs use the same offset. +*/ +if (kvm->arch.host_was_suspended && advance) { + l1_tsc =nsec_to_cycles (vcpu, + kvm->arch.kvmclock_offset + kernel_ns); + offset =kvm_compute_l1_tsc_offset (vcpu, l1_tsc); + kvm->arch.cur_tsc_offset = offset; +kvm_vcpu_write_tsc_offset (vcpu, offset);
arch/x86/kvm/x86.c void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 5067
Quoted 10 lines
arch/x86/kvm/x86.c +9
-0+ } else if (advance) { +kvm_vcpu_write_tsc_offset (vcpu, kvm->arch.cur_tsc_offset); + }else { +adjust_tsc_offset_host (vcpu, vcpu->arch.tsc_offset_adjustment); + } + kvm->arch.host_was_suspended = false; +raw_spin_unlock_irqrestore (&kvm->arch.tsc_write_lock, flags); +#else adjust_tsc_offset_host (vcpu, vcpu->arch.tsc_offset_adjustment); +#endif /* CONFIG_X86_64 */
Just style wise, it seems like renaming adjust_tsc_offset_host() to __adjust_tsc_offset_host(), and then moving the ifdefed logic into a new adjust_tsc_offset_host() implementation might be cleaner? Then you could have:
#ifdef COFNIG_X86_64
static inline void adjust_tsc_offset_host(...)
{
/* added logic above */
}
#else
static inline void adjust_tsc_offset_host(...)
{
__adjust_tsc_offset_host(...);
}#endif
arch/x86/kvm/x86.c int kvm_arch_enable_virtualization_cpu(void) · line 12763
Quoted 10 lines
arch/x86/kvm/x86.c +3
-0vcpu->arch.tsc_offset_adjustment =0 ;kvm_make_request (KVM_REQ_CLOCK_UPDATE , vcpu); }@@ -12729,6 +12758,9 @@ int kvm_arch_enable_virtualization_cpu(void)kvm_make_request (KVM_REQ_MASTERCLOCK_UPDATE , vcpu); } +#ifdef CONFIG_X86_64 + kvm->arch.host_was_suspended = true; +#endif
Similarly I'd wrap this in a:
#ifdef CONFIG_x86_64
static inline void kvm_set_host_was_suspended(*kvm)
{
kvm->arch.host_was_suspended = true;
}#else
static inline void kvm_set_host_was_suspended(*kvm) {
}
#endif
then call kvm_set_host_was_suspended(kvm) unconditionally in the logic above.
thanks
-john
Re: [PATCH v7 1/3] KVM: x86: Advance guest TSC after deep suspend.
Quoted 107 lines · John Stultz
Quoted 59 lines · Suleiman Souhlal
Try to advance guest TSC to current time after suspend when the host TSCs went backwards.
This makes the behavior consistent between suspends where host TSC resets and suspends where it doesn't, such as suspend-to-idle, where in the former case if the host TSC resets, the guests' would previously be "frozen" due to KVM's backwards TSC prevention, while in the latter case they would advance.
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Suleiman Souhlal <[email protected]>--- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/x86.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+)arch/x86/include/asm/kvm_host.h +3
-0@@ -1414,6 +1414,9 @@ struct kvm_arch {u64 cur_tsc_offset;u64 cur_tsc_generation;int nr_vcpus_matched_tsc; +#ifdef CONFIG_X86_64 +bool host_was_suspended; +#endif u32 default_tsc_khz;bool user_set_tsc;arch/x86/kvm/x86.c +20
-0@@ -5035,7 +5035,36 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)/* Apply any externally detected TSC adjustments (due to suspend) */ if (unlikely (vcpu->arch.tsc_offset_adjustment)) { +#ifdef CONFIG_X86_64 +unsigned long flags; +struct kvm *kvm; +bool advance; +u64 kernel_ns, l1_tsc, offset, tsc_now; + + kvm = vcpu->kvm; + advance =kvm_get_time_and_clockread (&kernel_ns, &tsc_now); +raw_spin_lock_irqsave (&kvm->arch.tsc_write_lock, flags); +/* +* Advance the guest's TSC to current time instead of only +* preventing it from going backwards, while making sure +* all the vCPUs use the same offset. +*/ +if (kvm->arch.host_was_suspended && advance) { + l1_tsc =nsec_to_cycles (vcpu, + kvm->arch.kvmclock_offset + kernel_ns); + offset =kvm_compute_l1_tsc_offset (vcpu, l1_tsc); + kvm->arch.cur_tsc_offset = offset; +kvm_vcpu_write_tsc_offset (vcpu, offset);arch/x86/kvm/x86.c void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 5067
Quoted 10 lines
arch/x86/kvm/x86.c +9
-0+ } else if (advance) { +kvm_vcpu_write_tsc_offset (vcpu, kvm->arch.cur_tsc_offset); + }else { +adjust_tsc_offset_host (vcpu, vcpu->arch.tsc_offset_adjustment); + } + kvm->arch.host_was_suspended = false; +raw_spin_unlock_irqrestore (&kvm->arch.tsc_write_lock, flags); +#else adjust_tsc_offset_host (vcpu, vcpu->arch.tsc_offset_adjustment); +#endif /* CONFIG_X86_64 */ Just style wise, it seems like renaming
adjust_tsc_offset_host()to__adjust_tsc_offset_host(), and then moving the ifdefed logic into a newadjust_tsc_offset_host()implementation might be cleaner? Then you could have:#ifdef
COFNIG_X86_64
static inline voidadjust_tsc_offset_host(...)
{
/* added logic above */
}
#else
static inline voidadjust_tsc_offset_host(...){ __adjust_tsc_offset_host(...); }#endif
arch/x86/kvm/x86.c int kvm_arch_enable_virtualization_cpu(void) · line 12763
Quoted 10 lines
arch/x86/kvm/x86.c +3
-0vcpu->arch.tsc_offset_adjustment =0 ;kvm_make_request (KVM_REQ_CLOCK_UPDATE , vcpu); }@@ -12729,6 +12758,9 @@ int kvm_arch_enable_virtualization_cpu(void)kvm_make_request (KVM_REQ_MASTERCLOCK_UPDATE , vcpu); } +#ifdef CONFIG_X86_64 + kvm->arch.host_was_suspended = true; +#endif Similarly I'd wrap this in a:
#ifdef
CONFIG_x86_64
static inline voidkvm_set_host_was_suspended(*kvm){ kvm->arch.host_was_suspended = true; }
Quoted 7 lines
#else
static inline voidkvm_set_host_was_suspended(*kvm){
}
#endifthen call
kvm_set_host_was_suspended(kvm)unconditionally in the logic above.
Thanks for the good suggestions. I'll incorporate them into v8.
-- Suleiman
PATCH v7 2/3 KVM: x86: Include host suspended duration in steal time
In reply to Suleiman SouhlalIntroduce MSR_KVM_SUSPEND_STEAL which controls whether or not a guest wants the duration of host suspend to be included in steal time.
This lets guests subtract the duration during which the host was suspended from the runtime of tasks that were running over the suspend, in order to prevent cases where host suspend causes long runtimes in guest tasks, even though their effective runtime was much shorter.
--- Documentation/virt/kvm/x86/cpuid.rst | 4 ++ Documentation/virt/kvm/x86/msr.rst | 14 +++++ arch/x86/include/asm/kvm_host.h | 3 ++ arch/x86/include/uapi/asm/kvm_para.h | 2 + arch/x86/kvm/cpuid.c | 4 +- arch/x86/kvm/x86.c | 80 ++++++++++++++++++++++++++-- 6 files changed, 101 insertions(+), 6 deletions(-)
Documentation/virt/kvm/x86/cpuid.rst +4 -0
@@ -103,6 +103,10 @@ KVM_FEATURE_HC_MAP_GPA_RANGE 16 guest checks this feature bit bef KVM_FEATURE_MIGRATION_CONTROL 17 guest checks this feature bit before using MSR_KVM_MIGRATION_CONTROL +KVM_FEATURE_SUSPEND_STEAL 18 guest checks this feature bit + before using + MSR_KVM_SUSPEND_STEAL. + KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24 host will warn if no guest-side per-cpu warps are expected in kvmclock
Documentation/virt/kvm/x86/msr.rst +14 -0
@@ -296,6 +296,12 @@ data: the amount of time in which this vCPU did not run, in nanoseconds. Time during which the vcpu is idle, will not be reported as steal time. + If the guest set the enable bit in MSR_KVM_SUSPEND_STEAL, + steal time includes the duration during which the host is + suspended. The case where the host suspends during a VM + migration might not be accounted if VCPUs aren't entered + post-resume. A workaround would be for the VMM to ensure that + the guest is entered with KVM_RUN after resuming from suspend. preempted: indicate the vCPU who owns this struct is running or
@@ -388,3 +394,11 @@ data: guest is communicating page encryption status to the host using the ``KVM_HC_MAP_GPA_RANGE`` hypercall, it can set bit 0 in this MSR to allow live migration of the guest. + +MSR_KVM_SUSPEND_STEAL: + 0x4b564d09 + +data: + This MSR is available if KVM_FEATURE_SUSPEND_STEAL is present in + CPUID. Bit 0 controls whether the host should include the duration it + has been suspended in steal time (1), or not (0).
arch/x86/include/asm/kvm_host.h +3 -0
@@ -932,6 +932,8 @@ struct kvm_vcpu_arch {u8 preempted;u64 msr_val;u64 last_steal; +u64 suspend_ts; +atomic64_t suspend_ns;struct gfn_to_hva_cache cache; } st;
@@ -1028,6 +1030,7 @@ struct kvm_vcpu_arch { } pv_eoi;u64 msr_kvm_poll_control; +u64 msr_kvm_suspend_steal;/* pv related host specific info */ struct {
arch/x86/include/uapi/asm/kvm_para.h +2 -0
@@ -36,6 +36,7 @@#define KVM_FEATURE_MSI_EXT_DEST_ID 15#define KVM_FEATURE_HC_MAP_GPA_RANGE 16#define KVM_FEATURE_MIGRATION_CONTROL 17 +#define KVM_FEATURE_SUSPEND_STEAL 18#define KVM_HINTS_REALTIME 0
@@ -58,6 +59,7 @@#define MSR_KVM_ASYNC_PF_INT 0x4b564d06#define MSR_KVM_ASYNC_PF_ACK 0x4b564d07#define MSR_KVM_MIGRATION_CONTROL 0x4b564d08 +#define MSR_KVM_SUSPEND_STEAL 0x4b564d09struct kvm_steal_time {__u64 steal;
arch/x86/kvm/cpuid.c +3 -1
@@ -1614,8 +1614,10 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) (1 <<KVM_FEATURE_PV_SCHED_YIELD ) | (1 <<KVM_FEATURE_ASYNC_PF_INT ); -if (sched_info_on ()) +if (sched_info_on ()) { entry->eax |= (1 <<KVM_FEATURE_STEAL_TIME ); + entry->eax |= (1 <<KVM_FEATURE_SUSPEND_STEAL ); + } entry->ebx =0 ; entry->ecx =0 ;
arch/x86/kvm/x86.c +75 -5
118 lines not on this page · raw message
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
PATCH v7 3/3 KVM: x86: Add "suspendsteal" cmdline to request host to add suspend duration in steal time
In reply to Suleiman SouhlalIntroduce a new command line parameter, "suspendsteal", enabling the guest to use MSR_KVM_SUSPEND_STEAL, which tells the host that it would like host suspend duration to be included in steal time.
--- Documentation/admin-guide/kernel-parameters.txt | 5 +++++ arch/x86/kernel/kvm.c | 15 +++++++++++++++ 2 files changed, 20 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +5 -0
@@ -7074,6 +7074,11 @@ improve throughput, but will also increase the amount of memory reserved for use by the client. + suspendsteal + [X86,PV_OPS] + Enable requesting the host to include the duration the + host was suspended in steal time. Disabled by default. + suspend.pm_test_delay= [SUSPEND] Sets the number of seconds to remain in a suspend test
arch/x86/kernel/kvm.c +15 -0
@@ -320,6 +320,18 @@ static void __init paravirt_ops_setup(void)#endif } +static bool suspend_steal; + +static int __init suspendsteal_setup (char *s) +{ +if (kvm_para_has_feature (KVM_FEATURE_SUSPEND_STEAL )) + suspend_steal = true; + +return 0 ; +} + +early_param ("suspendsteal" , suspendsteal_setup); +static void kvm_register_steal_time (void ) {int cpu =smp_processor_id ();
@@ -331,6 +343,9 @@ static void kvm_register_steal_time(void)wrmsrq (MSR_KVM_STEAL_TIME , (slow_virt_to_phys (st) |KVM_MSR_ENABLED ));pr_debug ("stealtime: cpu %d, msr %llx\n" , cpu, (unsigned long long )slow_virt_to_phys (st)); + +if (suspend_steal) +wrmsrl (MSR_KVM_SUSPEND_STEAL ,KVM_MSR_ENABLED ); } static DEFINE_PER_CPU_DECRYPTED(unsigned long, kvm_apic_eoi) = KVM_PV_EOI_DISABLED;
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog