Skip to contentSkip to conversation

kvm: x86: Disable KVM_HC_CLOCK_PAIRING if tsc is in always catchup mode

PATCH Superseded Anton Romanov 4 participants 7 messages Feb 15, 2022 – Feb 17, 2022

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 kvm: x86: Disable KVM_HC_CLOCK_PAIRING if tsc is in always catchup mode

If vcpu has tsc_always_catchup set each request updates pvclock data. KVM_HC_CLOCK_PAIRING consumers such as ptp_kvm_x86 rely on tsc read on host's side and do hypercall inside pvclock_read_retry loop leading to infinite loop in such situation.

v2:
Added warn

---
 arch/x86/kvm/x86.c | 9 +++++++++
 1 file changed, 9 insertions(+)
arch/x86/kvm/x86.c +9 -0
index 7131d735b1ef..aaafb46a6048 100644
@@ -8945,6 +8945,15 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr,
 	if (!kvm_get_walltime_and_clockread(&ts, &cycle))
 		return -KVM_EOPNOTSUPP;
 
+	/*
+	 * When tsc is in permanent catchup mode guests won't be able to use
+	 * pvclock_read_retry loop to get consistent view of pvclock
+	 */
+	if (vcpu->arch.tsc_always_catchup) {
+		pr_warn_ratelimited("KVM_HC_CLOCK_PAIRING not supported if vcpu is in tsc catchup mode\n");
+		return -KVM_EOPNOTSUPP;
+	}
+
 	clock_pairing.sec = ts.tv_sec;
 	clock_pairing.nsec = ts.tv_nsec;
 	clock_pairing.tsc = kvm_read_l1_tsc(vcpu, cycle);
Signature · 1 line
2.35.1.265.g69c8d7142f-goog
#1 lore raw
Quoted 18 lines · Anton Romanov

If vcpu has tsc_always_catchup set each request updates pvclock data. KVM_HC_CLOCK_PAIRING consumers such as ptp_kvm_x86 rely on tsc read on host's side and do hypercall inside pvclock_read_retry loop leading to infinite loop in such situation.

v2:
Added warn

---
 arch/x86/kvm/x86.c | 9 +++++++++
 1 file changed, 9 insertions(+)
arch/x86/kvm/x86.c +0 -0
index 7131d735b1ef..aaafb46a6048 100644

arch/x86/kvm/x86.c static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, · line 8954

Quoted 10 lines
arch/x86/kvm/x86.c +7 -0
@@ -8945,6 +8945,15 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr,
 	if (!kvm_get_walltime_and_clockread(&ts, &cycle))
 		return -KVM_EOPNOTSUPP;
 
+	/*
+	 * When tsc is in permanent catchup mode guests won't be able to use
+	 * pvclock_read_retry loop to get consistent view of pvclock
+	 */
+	if (vcpu->arch.tsc_always_catchup) {
+		pr_warn_ratelimited("KVM_HC_CLOCK_PAIRING not supported if vcpu is in tsc catchup mode\n");
+		return -KVM_EOPNOTSUPP;

I'm not sure this warn is a good idea. It is guest triggerable and 'tsc_always_catchup' is not a bug, it is a perfectly valid situation in the configuration when TSC scaling is unavailable. Even ratelimited, it's not nice when guests can pollute host's logs.

Also, EOPNOTSUPP makes it sound like the hypercall is unsupported, I'd suggest changing this to KVM_EFAULT.

Quoted 5 lines
arch/x86/kvm/x86.c +2 -0
+	}
+
 	clock_pairing.sec = ts.tv_sec;
 	clock_pairing.nsec = ts.tv_nsec;
 	clock_pairing.tsc = kvm_read_l1_tsc(vcpu, cycle);
Signature · 1 line
Vitaly
#2 lore raw
Quoted 9 lines · Vitaly Kuznetsov
Quoted 7 lines · Anton Romanov

If vcpu has tsc_always_catchup set each request updates pvclock data. KVM_HC_CLOCK_PAIRING consumers such as ptp_kvm_x86 rely on tsc read on host's side and do hypercall inside pvclock_read_retry loop leading to infinite loop in such situation.

v2:
Added warn

Versioning info goes in the "ignored" section, not the changelog.

Quoted 2 lines
Quoted 2 lines

---

This part is ignored by git. Versioning info, and/or any commentary that doesn't belong in the changelog, for a patch goes here.

Quoted 11 lines
Quoted 11 lines
 arch/x86/kvm/x86.c | 9 +++++++++
 1 file changed, 9 insertions(+)
arch/x86/kvm/x86.c +0 -0
index 7131d735b1ef..aaafb46a6048 100644
@@ -8945,6 +8945,15 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr,
 	if (!kvm_get_walltime_and_clockread(&ts, &cycle))
 		return -KVM_EOPNOTSUPP;
 
Quoted 12 lines

arch/x86/kvm/x86.c static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, · line 8954

Quoted 7 lines
arch/x86/kvm/x86.c +7 -0
+	/*
+	 * When tsc is in permanent catchup mode guests won't be able to use
+	 * pvclock_read_retry loop to get consistent view of pvclock
+	 */
+	if (vcpu->arch.tsc_always_catchup) {
+		pr_warn_ratelimited("KVM_HC_CLOCK_PAIRING not supported if vcpu is in tsc catchup mode\n");
+		return -KVM_EOPNOTSUPP;

I'm not sure this warn is a good idea. It is guest triggerable and 'tsc_always_catchup' is not a bug, it is a perfectly valid situation in the configuration when TSC scaling is unavailable. Even ratelimited, it's not nice when guests can pollute host's logs.

Agreed. And if we want to alert the user/admin, it'd probably be better do so on tsc_always_catchup first being set. Doubt it's worth it though, assuming my other patch to prevent KVM from setting tsc_always_catchup=true on non-ancient hardware without userspace interaction gets merged.

Quoted 2 lines

Also, EOPNOTSUPP makes it sound like the hypercall is unsupported, I'd suggest changing this to KVM_EFAULT.

Eh, it's consistent with the above check though, where KVM returns KVM_EOPNOTSUPP due to the vclock mode being incompatible. This is more or less the same, it's just a different "mode". KVM_EFAULT suggests that the guest did something wrong and/or that the guest can remedy the problem in someway, e.g. by providing a different address. This issue is purely in the host's domain.

#3 lore raw
Quoted 1 line · Sean Christopherson

On Wed, Feb 16, 2022, Vitaly Kuznetsov wrote:

...

Quoted 9 lines
Quoted 3 lines

Also, EOPNOTSUPP makes it sound like the hypercall is unsupported, I'd suggest changing this to KVM_EFAULT.

Eh, it's consistent with the above check though, where KVM returns KVM_EOPNOTSUPP due to the vclock mode being incompatible. This is more or less the same, it's just a different "mode". KVM_EFAULT suggests that the guest did something wrong and/or that the guest can remedy the problem in someway, e.g. by providing a different address. This issue is purely in the host's domain.

Ack, Paolo's already made the change back to KVM_EOPNOTSUPP upon commit (but I still mildly dislike using 'EOPNOTSUPP' for a temporary condition on the host).

Signature · 1 line
Vitaly
maintainer
#4 lore raw
Quoted 15 lines · Vitaly Kuznetsov
Quoted 1 line · Sean Christopherson

On Wed, Feb 16, 2022, Vitaly Kuznetsov wrote:

...

Quoted 9 lines
Quoted 3 lines

Also, EOPNOTSUPP makes it sound like the hypercall is unsupported, I'd suggest changing this to KVM_EFAULT.

Eh, it's consistent with the above check though, where KVM returns KVM_EOPNOTSUPP due to the vclock mode being incompatible. This is more or less the same, it's just a different "mode". KVM_EFAULT suggests that the guest did something wrong and/or that the guest can remedy the problem in someway, e.g. by providing a different address. This issue is purely in the host's domain.

Quoted 3 lines

Ack, Paolo's already made the change back to KVM_EOPNOTSUPP upon commit (but I still mildly dislike using 'EOPNOTSUPP' for a temporary condition on the host).

It's not temporary, always-catchup is set on KVM_SET_TSC_KHZ and never goes away.

Paolo

#5 lore raw
Quoted 1 line · Paolo Bonzini

On 2/17/22 17:09, Vitaly Kuznetsov wrote:

...

Quoted 5 lines
Quoted 2 lines

(but I still mildly dislike using 'EOPNOTSUPP' for a temporary condition on the host).

It's not temporary, always-catchup is set on KVM_SET_TSC_KHZ and never goes away.

Even if the guest is migrated (back) to the host which supports TSC scaling?

Signature · 1 line
Vitaly
maintainer
#6 lore raw
Quoted 4 lines · Vitaly Kuznetsov
Quoted 1 line · Paolo Bonzini

On 2/17/22 17:09, Vitaly Kuznetsov wrote:

...

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

(but I still mildly dislike using 'EOPNOTSUPP' for a temporary condition on the host).

It's not temporary, always-catchup is set on KVM_SET_TSC_KHZ and never goes away.

Even if the guest is migrated (back) to the host which supports TSC scaling?

Ah right, in that case no. But this hypercall does not have a CPUID bit, so the supported status can change on migration anyway.

Paolo