Skip to contentSkip to conversation

Improve KVM_SET_TSC_KHZ handling for CoCo VMs

PATCH Superseded Kai Huang 5 participants 17 messages Jul 9, 2025 – Jul 13, 2025

Subsystems (4)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Kai Huang

author
#0 lore raw

cover Improve KVM_SET_TSC_KHZ handling for CoCo VMs

This series follows Sean's suggestions [1][2] to:

- Reject vCPU scope KVM_SET_TSC_KHZ ioctl for TSC protected vCPU
- Reject VM scope KVM_SET_TSC_KHZ ioctl when vCPUs have been created

.. in the discussion of SEV-SNP Secure TSC support series.

This series has been sanity tested with TDX guests using today's Qemu:

- With this series Qemu can still run TDX guests successfully.
- With some hack to the Qemu, both VM and vCPU scope KVM_SET_TSC_KHZ ioctls failed as expected.

Currently only TDX guests are TSC protected. The SEV-SNP Secure TSC support will enable protected TSC too and can also benefit from this series.

[1]: https://lore.kernel.org/kvm/[email protected]/
[2]: https://lore.kernel.org/kvm/[email protected]/

Kai Huang (2):
KVM: x86: Reject KVM_SET_TSC_KHZ vCPU ioctl for TSC protected guest
KVM: x86: Reject KVM_SET_TSC_KHZ VM ioctl when vCPU has been created

 arch/x86/kvm/x86.c | 8 ++++++++
 1 file changed, 8 insertions(+)

base-commit: 6c7ecd725e503bf2ca69ff52c6cc48bb650b1f11

Signature · 1 line
2.50.0
author
#1 lore raw

PATCH 1/2 KVM: x86: Reject KVM_SET_TSC_KHZ vCPU ioctl for TSC protected guest

Reject KVM_SET_TSC_KHZ vCPU ioctl if guest's TSC is protected and not changeable by KVM.

For such TSC protected guests, e.g. TDX guests, typically the TSC is configured once at VM level before any vCPU are created and remains unchanged during VM's lifetime. KVM provides the KVM_SET_TSC_KHZ VM scope ioctl to allow the userspace VMM to configure the TSC of such VM. After that the userspace VMM is not supposed to call the KVM_SET_TSC_KHZ vCPU scope ioctl anymore when creating the vCPU.

The de facto userspace VMM Qemu does this for TDX guests. The upcoming SEV-SNP guests with Secure TSC should follow.

Note this could be a break of ABI. But for now only TDX guests are TSC protected and only Qemu supports TDX, thus in practice this should not break any existing userspace.

---
 arch/x86/kvm/x86.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kvm/x86.c +4 -0
index 2806f7104295..699ca5e74bba 100644
@@ -6186,6 +6186,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 		u32 user_tsc_khz;
 
 		r = -EINVAL;
+
+		if (vcpu->arch.guest_tsc_protected)
+			goto out;
+
 		user_tsc_khz = (u32)arg;
 
 		if (kvm_caps.has_tsc_control &&
Signature · 1 line
2.50.0
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] KVM: x86: Reject KVM_SET_TSC_KHZ vCPU ioctl for TSC protected guest

Quoted 13 lines · Kai Huang

Reject KVM_SET_TSC_KHZ vCPU ioctl if guest's TSC is protected and not changeable by KVM.

For such TSC protected guests, e.g. TDX guests, typically the TSC is configured once at VM level before any vCPU are created and remains unchanged during VM's lifetime. KVM provides the KVM_SET_TSC_KHZ VM scope ioctl to allow the userspace VMM to configure the TSC of such VM. After that the userspace VMM is not supposed to call the KVM_SET_TSC_KHZ vCPU scope ioctl anymore when creating the vCPU.

The de facto userspace VMM Qemu does this for TDX guests. The upcoming SEV-SNP guests with Secure TSC should follow.

Quoted 6 lines

Note this could be a break of ABI. But for now only TDX guests are TSC protected and only Qemu supports TDX, thus in practice this should not break any existing userspace.

arch/x86/kvm/x86.c long kvm_arch_vcpu_ioctl(struct file *filp, · line 6188

Quoted 12 lines
---
  arch/x86/kvm/x86.c | 4 ++++
  1 file changed, 4 insertions(+)
arch/x86/kvm/x86.c +0 -0
index 2806f7104295..699ca5e74bba 100644
@@ -6186,6 +6186,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
  		u32 user_tsc_khz;
 
  		r = -EINVAL;

I wondered if -EPERM is more appropriate. But I search the KVM files and find the KVM convention is to return -EINVAL.

Quoted 6 lines
arch/x86/kvm/x86.c +3 -0
+		if (vcpu->arch.guest_tsc_protected)
+			goto out;
+
  		user_tsc_khz = (u32)arg;
 
  		if (kvm_caps.has_tsc_control &&
#3 lore raw

Re: [PATCH 1/2] KVM: x86: Reject KVM_SET_TSC_KHZ vCPU ioctl for TSC protected guest

In reply to Kai Huang
Quoted 13 lines · Kai Huang

Reject KVM_SET_TSC_KHZ vCPU ioctl if guest's TSC is protected and not changeable by KVM.

For such TSC protected guests, e.g. TDX guests, typically the TSC is configured once at VM level before any vCPU are created and remains unchanged during VM's lifetime. KVM provides the KVM_SET_TSC_KHZ VM scope ioctl to allow the userspace VMM to configure the TSC of such VM. After that the userspace VMM is not supposed to call the KVM_SET_TSC_KHZ vCPU scope ioctl anymore when creating the vCPU.

The de facto userspace VMM Qemu does this for TDX guests. The upcoming SEV-SNP guests with Secure TSC should follow.

Quoted 6 lines

Note this could be a break of ABI. But for now only TDX guests are TSC protected and only Qemu supports TDX, thus in practice this should not break any existing userspace.

Need to add this in Documentation/virt/kvm/api.rst as well, saying that for TDX and SecureTSC enabled SNP guests, KVM_SET_TSC_KHZ vCPU ioctl is not valid.

Quoted 9 lines
---
 arch/x86/kvm/x86.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kvm/x86.c +0 -0
index 2806f7104295..699ca5e74bba 100644

arch/x86/kvm/x86.c long kvm_arch_vcpu_ioctl(struct file *filp, · line 6195

Quoted 10 lines
arch/x86/kvm/x86.c +4 -0
@@ -6186,6 +6186,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 		u32 user_tsc_khz;
 
 		r = -EINVAL;
+
+		if (vcpu->arch.guest_tsc_protected)
+			goto out;
+
 		user_tsc_khz = (u32)arg;
 
 		if (kvm_caps.has_tsc_control &&

Regards
Nikunj

author
#4 lore raw

Re: [PATCH 1/2] KVM: x86: Reject KVM_SET_TSC_KHZ vCPU ioctl for TSC protected guest

Quoted 19 lines · Nikunj A. Dadhania
Quoted 17 lines · Kai Huang

Reject KVM_SET_TSC_KHZ vCPU ioctl if guest's TSC is protected and not changeable by KVM.

For such TSC protected guests, e.g. TDX guests, typically the TSC is configured once at VM level before any vCPU are created and remains unchanged during VM's lifetime. KVM provides the KVM_SET_TSC_KHZ VM scope ioctl to allow the userspace VMM to configure the TSC of such VM. After that the userspace VMM is not supposed to call the KVM_SET_TSC_KHZ vCPU scope ioctl anymore when creating the vCPU.

The de facto userspace VMM Qemu does this for TDX guests. The upcoming SEV-SNP guests with Secure TSC should follow.

Note this could be a break of ABI. But for now only TDX guests are TSC protected and only Qemu supports TDX, thus in practice this should not break any existing userspace.

Quoted 8 lines
Quoted 2 lines

Need to add this in Documentation/virt/kvm/api.rst as well, saying that for TDX and SecureTSC enabled SNP guests, KVM_SET_TSC_KHZ vCPU ioctl is not valid.

Good point. Thanks for bringing it up.

I will add below to the doc unless someone has comments?

I'll probably split the doc diff into two parts and merge each to the respective code change patch, since the change to the doc contains change to both vm ioctl and vcpu ioctl.

Btw, I think I'll not mention Secure TSC enabled SEV-SNP guests for now because it is not in upstream yet. But I tried to make the text in a way that could be easily extended to cover Secure TSC guests.

(quoted diff) +0 -0

b/Documentation/virt/kvm/api.rst
index 43ed57e048a8..ad61bcba3791 100644

Documentation/virt/kvm/api.rst +7 -1
@@ -2006,7 +2006,13 @@ frequency is KHz.
 
 If the KVM_CAP_VM_TSC_CONTROL capability is advertised, this can also
 be used as a vm ioctl to set the initial tsc frequency of subsequently
-created vCPUs.
+created vCPUs.  It must be called before any vCPU is created.
+
+For TSC protected CoCo VMs where TSC is configured once at VM scope and
+remains unchanged during VM's lifetime, the VM ioctl should be used to
+configure the TSC and the vCPU ioctl fails.
+
+Example of such CoCo VMs: TDX guests.
 
 
 
#5 lore raw

Re: [PATCH 1/2] KVM: x86: Reject KVM_SET_TSC_KHZ vCPU ioctl for TSC protected guest

Quoted 33 lines · Kai
Quoted 19 lines · Nikunj A. Dadhania
Quoted 17 lines · Kai Huang

Reject KVM_SET_TSC_KHZ vCPU ioctl if guest's TSC is protected and not changeable by KVM.

For such TSC protected guests, e.g. TDX guests, typically the TSC is configured once at VM level before any vCPU are created and remains unchanged during VM's lifetime. KVM provides the KVM_SET_TSC_KHZ VM scope ioctl to allow the userspace VMM to configure the TSC of such VM. After that the userspace VMM is not supposed to call the KVM_SET_TSC_KHZ vCPU scope ioctl anymore when creating the vCPU.

The de facto userspace VMM Qemu does this for TDX guests. The upcoming SEV-SNP guests with Secure TSC should follow.

Note this could be a break of ABI. But for now only TDX guests are TSC protected and only Qemu supports TDX, thus in practice this should not break any existing userspace.

Quoted 8 lines
Quoted 2 lines

Need to add this in Documentation/virt/kvm/api.rst as well, saying that for TDX and SecureTSC enabled SNP guests, KVM_SET_TSC_KHZ vCPU ioctl is not valid.

Good point. Thanks for bringing it up.

I will add below to the doc unless someone has comments?

Quoted 7 lines

I'll probably split the doc diff into two parts and merge each to the respective code change patch, since the change to the doc contains change to both vm ioctl and vcpu ioctl.

Btw, I think I'll not mention Secure TSC enabled SEV-SNP guests for now because it is not in upstream yet. But I tried to make the text in a way that could be easily extended to cover Secure TSC guests.

Sure, I can add that later.

Documentation/virt/kvm/api.rst frequency is KHz. · line 2009

Quoted 12 lines
Documentation/virt/kvm/api.rst +1 -1
index 43ed57e048a8..ad61bcba3791 100644
@@ -2006,7 +2006,13 @@ frequency is KHz.
 
 If the KVM_CAP_VM_TSC_CONTROL capability is advertised, this can also
 be used as a vm ioctl to set the initial tsc frequency of subsequently
-created vCPUs.
+created vCPUs.  It must be called before any vCPU is created.

s/It/The VM Scope ioctl/

Quoted 3 lines
Documentation/virt/kvm/api.rst +2 -0
+
+For TSC protected CoCo VMs where TSC is configured once at VM scope

and

s/CoCo/Confidential Computing (CoCo)/
s/TSC is/TSC frequency is/

Documentation/virt/kvm/api.rst frequency is KHz. · line 2013

Quoted 2 lines
Documentation/virt/kvm/api.rst +2 -0
+remains unchanged during VM's lifetime, the VM ioctl should be used to
+configure the TSC and the vCPU ioctl fails.

s/TSC/TSC frequency/

s/vcpu ioctl fails/vcpu ioctl is not supported/

Documentation/virt/kvm/api.rst frequency is KHz. · line 2015

Quoted 3 lines
Documentation/virt/kvm/api.rst +3 -0
+
+
+Example of such CoCo VMs: TDX guests.

Regards
Nikunj

author
#6 lore raw

Re: [PATCH 1/2] KVM: x86: Reject KVM_SET_TSC_KHZ vCPU ioctl for TSC protected guest

Quoted 53 lines · Nikunj A Dadhania
Quoted 33 lines · Kai
Quoted 19 lines · Nikunj A. Dadhania
Quoted 17 lines · Kai Huang

Reject KVM_SET_TSC_KHZ vCPU ioctl if guest's TSC is protected and not changeable by KVM.

For such TSC protected guests, e.g. TDX guests, typically the TSC is configured once at VM level before any vCPU are created and remains unchanged during VM's lifetime. KVM provides the KVM_SET_TSC_KHZ VM scope ioctl to allow the userspace VMM to configure the TSC of such VM. After that the userspace VMM is not supposed to call the KVM_SET_TSC_KHZ vCPU scope ioctl anymore when creating the vCPU.

The de facto userspace VMM Qemu does this for TDX guests. The upcoming SEV-SNP guests with Secure TSC should follow.

Note this could be a break of ABI. But for now only TDX guests are TSC protected and only Qemu supports TDX, thus in practice this should not break any existing userspace.

Quoted 8 lines
Quoted 2 lines

Need to add this in Documentation/virt/kvm/api.rst as well, saying that for TDX and SecureTSC enabled SNP guests, KVM_SET_TSC_KHZ vCPU ioctl is not valid.

Good point. Thanks for bringing it up.

I will add below to the doc unless someone has comments?

Quoted 7 lines

I'll probably split the doc diff into two parts and merge each to the respective code change patch, since the change to the doc contains change to both vm ioctl and vcpu ioctl.

Btw, I think I'll not mention Secure TSC enabled SEV-SNP guests for now because it is not in upstream yet. But I tried to make the text in a way that could be easily extended to cover Secure TSC guests.

Sure, I can add that later.

Quoted 8 lines
Documentation/virt/kvm/api.rst +0 -0
index 43ed57e048a8..ad61bcba3791 100644
@@ -2006,7 +2006,13 @@ frequency is KHz.
 
Quoted 6 lines

Documentation/virt/kvm/api.rst frequency is KHz. · line 2009

Quoted 4 lines
Documentation/virt/kvm/api.rst +1 -1
 If the KVM_CAP_VM_TSC_CONTROL capability is advertised, this can also
 be used as a vm ioctl to set the initial tsc frequency of subsequently
-created vCPUs.
+created vCPUs.  It must be called before any vCPU is created.

s/It/The VM Scope ioctl/

OK. I'll use "The vm ioctl", though, to make it consistent with the previous sentence.

Quoted 7 lines
Quoted 4 lines
Documentation/virt/kvm/api.rst +2 -0
+
+For TSC protected CoCo VMs where TSC is configured once at VM scope

and

s/CoCo/Confidential Computing (CoCo)/
s/TSC is/TSC frequency is/

OK.

Quoted 7 lines

Documentation/virt/kvm/api.rst frequency is KHz. · line 2013

Quoted 3 lines
Documentation/virt/kvm/api.rst +2 -0
+remains unchanged during VM's lifetime, the VM ioctl should be used to
+configure the TSC and the vCPU ioctl fails.

s/TSC/TSC frequency/

s/vcpu ioctl fails/vcpu ioctl is not supported/

OK.

Quoted 7 lines

Documentation/virt/kvm/api.rst frequency is KHz. · line 2015

Quoted 4 lines
Documentation/virt/kvm/api.rst +3 -0
+
+
+Example of such CoCo VMs: TDX guests.

Regards
Nikunj

author
#7 lore raw

PATCH 2/2 KVM: x86: Reject KVM_SET_TSC_KHZ VM ioctl when vCPU has been created

In reply to Kai Huang

Reject the KVM_SET_TSC_KHZ VM ioctl when there's vCPU has already been created.

The VM scope KVM_SET_TSC_KHZ ioctl is used to set up the default TSC frequency that all subsequent created vCPUs use. It is only intended to be called before any vCPU is created. Allowing it to be called after that only results in confusion but nothing good.

Note this is an ABI change. But currently in Qemu (the de facto userspace VMM) only TDX uses this VM ioctl, and it is only called once before creating any vCPU, therefore the risk of breaking userspace is pretty low.

---
 arch/x86/kvm/x86.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kvm/x86.c +4 -0
index 699ca5e74bba..e5e55d549468 100644
@@ -7194,6 +7194,10 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
 		u32 user_tsc_khz;
 
 		r = -EINVAL;
+
+		if (kvm->created_vcpus)
+			goto out;
+
 		user_tsc_khz = (u32)arg;
 
 		if (kvm_caps.has_tsc_control &&
Signature · 1 line
2.50.0
Reviewed-by on v1 2/2
#8 lore raw

Re: [PATCH 2/2] KVM: x86: Reject KVM_SET_TSC_KHZ VM ioctl when vCPU has been created

Quoted 8 lines · Kai Huang

Reject the KVM_SET_TSC_KHZ VM ioctl when there's vCPU has already been created.

The VM scope KVM_SET_TSC_KHZ ioctl is used to set up the default TSC frequency that all subsequent created vCPUs use. It is only intended to be called before any vCPU is created. Allowing it to be called after that only results in confusion but nothing good.

Quoted 7 lines

Note this is an ABI change. But currently in Qemu (the de facto userspace VMM) only TDX uses this VM ioctl, and it is only called once before creating any vCPU, therefore the risk of breaking userspace is pretty low.

Quoted 19 lines
---
  arch/x86/kvm/x86.c | 4 ++++
  1 file changed, 4 insertions(+)
arch/x86/kvm/x86.c +4 -0
index 699ca5e74bba..e5e55d549468 100644
@@ -7194,6 +7194,10 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
  		u32 user_tsc_khz;
 
  		r = -EINVAL;
+
+		if (kvm->created_vcpus)
+			goto out;
+
  		user_tsc_khz = (u32)arg;
 
  		if (kvm_caps.has_tsc_control &&
#9 lore raw

Re: [PATCH 2/2] KVM: x86: Reject KVM_SET_TSC_KHZ VM ioctl when vCPU has been created

In reply to Kai Huang
Quoted 24 lines · Kai Huang

Reject the KVM_SET_TSC_KHZ VM ioctl when there's vCPU has already been created.

The VM scope KVM_SET_TSC_KHZ ioctl is used to set up the default TSC frequency that all subsequent created vCPUs use. It is only intended to be called before any vCPU is created. Allowing it to be called after that only results in confusion but nothing good.

Note this is an ABI change. But currently in Qemu (the de facto userspace VMM) only TDX uses this VM ioctl, and it is only called once before creating any vCPU, therefore the risk of breaking userspace is pretty low.

---
arch/x86/kvm/x86.c | 4 ++++

1 file changed, 4 insertions(+)
arch/x86/kvm/x86.c +0 -0
index 699ca5e74bba..e5e55d549468 100644

arch/x86/kvm/x86.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 7200

Quoted 7 lines
arch/x86/kvm/x86.c +4 -0
@@ -7194,6 +7194,10 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
 		u32 user_tsc_khz;
 
 		r = -EINVAL;
+
+		if (kvm->created_vcpus)
+			goto out;
+

shouldn't kvm->lock be held?

Quoted 7 lines
arch/x86/kvm/x86.c +0 -0
 		user_tsc_khz = (u32)arg;
 
 		if (kvm_caps.has_tsc_control &&
Signature · 1 line
2.50.0
#10 lore raw

Re: [PATCH 2/2] KVM: x86: Reject KVM_SET_TSC_KHZ VM ioctl when vCPU has been created

Quoted 27 lines · Chao Gao
Quoted 26 lines · Kai Huang

Reject the KVM_SET_TSC_KHZ VM ioctl when there's vCPU has already been created.

The VM scope KVM_SET_TSC_KHZ ioctl is used to set up the default TSC frequency that all subsequent created vCPUs use. It is only intended to be called before any vCPU is created. Allowing it to be called after that only results in confusion but nothing good.

Note this is an ABI change. But currently in Qemu (the de facto userspace VMM) only TDX uses this VM ioctl, and it is only called once before creating any vCPU, therefore the risk of breaking userspace is pretty low.

---
arch/x86/kvm/x86.c | 4 ++++

1 file changed, 4 insertions(+)
arch/x86/kvm/x86.c +0 -0
index 699ca5e74bba..e5e55d549468 100644
@@ -7194,6 +7194,10 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
 		u32 user_tsc_khz;
 
Quoted 7 lines

arch/x86/kvm/x86.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 7200

Quoted 5 lines
arch/x86/kvm/x86.c +4 -0
 		r = -EINVAL;
+
+		if (kvm->created_vcpus)
+			goto out;
+

shouldn't kvm->lock be held?

Yep.

author
#11 lore raw

Re: [PATCH 2/2] KVM: x86: Reject KVM_SET_TSC_KHZ VM ioctl when vCPU has been created

Quoted 36 lines · Sean Christopherson
Quoted 27 lines · Chao Gao
Quoted 26 lines · Kai Huang

Reject the KVM_SET_TSC_KHZ VM ioctl when there's vCPU has already been created.

The VM scope KVM_SET_TSC_KHZ ioctl is used to set up the default TSC frequency that all subsequent created vCPUs use. It is only intended to be called before any vCPU is created. Allowing it to be called after that only results in confusion but nothing good.

Note this is an ABI change. But currently in Qemu (the de facto userspace VMM) only TDX uses this VM ioctl, and it is only called once before creating any vCPU, therefore the risk of breaking userspace is pretty low.

---
arch/x86/kvm/x86.c | 4 ++++

1 file changed, 4 insertions(+)
arch/x86/kvm/x86.c +0 -0
index 699ca5e74bba..e5e55d549468 100644
@@ -7194,6 +7194,10 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
 		u32 user_tsc_khz;
 
Quoted 7 lines

arch/x86/kvm/x86.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 7200

Quoted 5 lines
arch/x86/kvm/x86.c +4 -0
 		r = -EINVAL;
+
+		if (kvm->created_vcpus)
+			goto out;
+

shouldn't kvm->lock be held?

Quoted 1 line

Yep.

My bad. I'll fixup and send out v2 soon, together with the doc update.

author
#12 lore raw

Re: [PATCH 2/2] KVM: x86: Reject KVM_SET_TSC_KHZ VM ioctl when vCPU has been created

Quoted 37 lines · Kai
Quoted 36 lines · Sean Christopherson
Quoted 34 lines · Chao Gao
Quoted 24 lines · Kai Huang

Reject the KVM_SET_TSC_KHZ VM ioctl when there's vCPU has already been created.

The VM scope KVM_SET_TSC_KHZ ioctl is used to set up the default TSC frequency that all subsequent created vCPUs use. It is only intended to be called before any vCPU is created. Allowing it to be called after that only results in confusion but nothing good.

Note this is an ABI change. But currently in Qemu (the de facto userspace VMM) only TDX uses this VM ioctl, and it is only called once before creating any vCPU, therefore the risk of breaking userspace is pretty low.

---
arch/x86/kvm/x86.c | 4 ++++

1 file changed, 4 insertions(+)
arch/x86/kvm/x86.c +0 -0
index 699ca5e74bba..e5e55d549468 100644

arch/x86/kvm/x86.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 7200

Quoted 7 lines
arch/x86/kvm/x86.c +4 -0
@@ -7194,6 +7194,10 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
 		u32 user_tsc_khz;
 
 		r = -EINVAL;
+
+		if (kvm->created_vcpus)
+			goto out;
+

shouldn't kvm->lock be held?

Quoted 3 lines
Quoted 1 line

Yep.

My bad. I'll fixup and send out v2 soon, together with the doc update.

Sorry for multiple emails, but I ended up with below.

AFAICT the actual updating of kvm->arch.default_tsc_khz needs to be in the kvm->lock mutex too. Please let me know if you found any issue?

(quoted diff) +0 -0

b/Documentation/virt/kvm/api.rst
index 43ed57e048a8..86ea1e2b2737 100644

Documentation/virt/kvm/api.rst +1 -1
@@ -2006,7 +2006,7 @@ frequency is KHz.
 
 If the KVM_CAP_VM_TSC_CONTROL capability is advertised, this can also
 be used as a vm ioctl to set the initial tsc frequency of subsequently
-created vCPUs.
+created vCPUs.  It must be called before any vCPU is created.
 
 4.56 KVM_GET_TSC_KHZ
 --------------------
arch/x86/kvm/x86.c +0 -0
index 2806f7104295..4051c0cacb92 100644
@@ -7199,9 +7199,12 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned

int ioctl, unsigned long arg)

                if (user_tsc_khz == 0)
                        user_tsc_khz = tsc_khz;

- WRITE_ONCE(kvm->arch.default_tsc_khz, user_tsc_khz);
- r = 0;
-
+ mutex_lock(&kvm->lock);
+ if (!kvm->created_vcpus) {
+ WRITE_ONCE(kvm->arch.default_tsc_khz,
user_tsc_khz);
+ r = 0;
+ }

+               mutex_unlock(&kvm->lock);
                goto out;
        }
        case KVM_GET_TSC_KHZ: {
#13 lore raw

Re: [PATCH 2/2] KVM: x86: Reject KVM_SET_TSC_KHZ VM ioctl when vCPU has been created

Quoted 2 lines

AFAICT the actual updating of kvm->arch.default_tsc_khz needs to be in the kvm->lock mutex too.

Yep.

Documentation/virt/kvm/api.rst frequency is KHz. · line 2009

Quoted 13 lines

Please let me know if you found any issue?

Documentation/virt/kvm/api.rst +0 -0
index 43ed57e048a8..86ea1e2b2737 100644
@@ -2006,7 +2006,7 @@ frequency is KHz.
 

If the KVM_CAP_VM_TSC_CONTROL capability is advertised, this can also be used as a vm ioctl to set the initial tsc frequency of subsequently

Documentation/virt/kvm/api.rst +1 -1
-created vCPUs.
+created vCPUs.  It must be called before any vCPU is created.

^^ remove one space here.

"must be" sounds like a mandatory action, but IIUC the vm ioctl is optional for non-CC VMs. I'm not sure if this is just a problem of my interpretation.

To make the API documentation super clear, how about:

If the KVM_CAP_VM_TSC_CONTROL capability is advertised, this can also be used as a vm ioctl to set the initial tsc frequency of vCPUs before any vCPU is created. Attempting to call this vm ioctl after vCPU creation will return an EINVAL error.

Quoted 12 lines

4.56 KVM_GET_TSC_KHZ
--------------------

arch/x86/kvm/x86.c +0 -0
index 2806f7104295..4051c0cacb92 100644
@@ -7199,9 +7199,12 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned
 int ioctl, unsigned long arg)
               if (user_tsc_khz == 0)
                       user_tsc_khz = tsc_khz;
 

arch/x86/kvm/x86.c int kvm_arch_vm_ioctl(struct file *filp, unsigned · line 7209

Quoted 10 lines
arch/x86/kvm/x86.c +6 -3
-               WRITE_ONCE(kvm->arch.default_tsc_khz, user_tsc_khz);
-               r = 0;
-
+               mutex_lock(&kvm->lock);
+               if (!kvm->created_vcpus) {
+                       WRITE_ONCE(kvm->arch.default_tsc_khz,
 user_tsc_khz);
+                       r = 0;
+               }
+               mutex_unlock(&kvm->lock);

LGTM.

Quoted 3 lines
arch/x86/kvm/x86.c +0 -0
               goto out;
       }
       case KVM_GET_TSC_KHZ: {
author
#14 lore raw

Re: [PATCH 2/2] KVM: x86: Reject KVM_SET_TSC_KHZ VM ioctl when vCPU has been created

Quoted 14 lines · Chao Gao
Quoted 2 lines

AFAICT the actual updating of kvm->arch.default_tsc_khz needs to be in the kvm->lock mutex too.

Yep.

Quoted 9 lines

Please let me know if you found any issue?

Documentation/virt/kvm/api.rst +0 -0
index 43ed57e048a8..86ea1e2b2737 100644
@@ -2006,7 +2006,7 @@ frequency is KHz.
 
Quoted 6 lines

Documentation/virt/kvm/api.rst frequency is KHz. · line 2009

Quoted 4 lines

If the KVM_CAP_VM_TSC_CONTROL capability is advertised, this can also be used as a vm ioctl to set the initial tsc frequency of subsequently

Documentation/virt/kvm/api.rst +1 -1
-created vCPUs.
+created vCPUs.  It must be called before any vCPU is created.

^^ remove one space here.

OK.

Quoted 3 lines

"must be" sounds like a mandatory action, but IIUC the vm ioctl is optional for non-CC VMs. I'm not sure if this is just a problem of my interpretation.

The context of that paragraph has "can also be used ...", so to me it's implied that "if it is called", i.e., it's implied that it is optional for non-CC VMs.

Quoted 7 lines

To make the API documentation super clear, how about:

If the KVM_CAP_VM_TSC_CONTROL capability is advertised, this can also be used as a vm ioctl to set the initial tsc frequency of vCPUs before any vCPU is created. Attempting to call this vm ioctl after vCPU creation will return an EINVAL error.

I am not sure we need to mention -EINVAL. This IOCTL is already returning -EINVAL for other errors (when invalid user_tsc_khz is supplied).

IMHO: Userspace should just care about whether success or not. Being explicit is good, but sometimes it's better to have some room here.

But I'll let Sean/Paolo to decide.

Quoted 29 lines
Quoted 13 lines

4.56 KVM_GET_TSC_KHZ
--------------------

arch/x86/kvm/x86.c +0 -0
index 2806f7104295..4051c0cacb92 100644
@@ -7199,9 +7199,12 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned
 int ioctl, unsigned long arg)
                if (user_tsc_khz == 0)
                        user_tsc_khz = tsc_khz;
 

arch/x86/kvm/x86.c int kvm_arch_vm_ioctl(struct file *filp, unsigned · line 7209

Quoted 10 lines
arch/x86/kvm/x86.c +6 -3
-               WRITE_ONCE(kvm->arch.default_tsc_khz, user_tsc_khz);
-               r = 0;
-
+               mutex_lock(&kvm->lock);
+               if (!kvm->created_vcpus) {
+                       WRITE_ONCE(kvm->arch.default_tsc_khz,
 user_tsc_khz);
+                       r = 0;
+               }
+               mutex_unlock(&kvm->lock);

LGTM.

Quoted 3 lines
arch/x86/kvm/x86.c +0 -0
                goto out;

       }
       case KVM_GET_TSC_KHZ: {

Reviewed-by on v1 2/2
#15 lore raw

Re: [PATCH 2/2] KVM: x86: Reject KVM_SET_TSC_KHZ VM ioctl when vCPU has been created

In reply to Kai Huang
Quoted 2 lines · Kai Huang

Reject the KVM_SET_TSC_KHZ VM ioctl when there's vCPU has already been created.

Probably the below is clear:

Reject KVM_SET_TSC_KHZ VM ioctl when vCPUs have been created

Quoted 6 lines

The VM scope KVM_SET_TSC_KHZ ioctl is used to set up the default TSC frequency that all subsequent created vCPUs use. It is only intended to be called before any vCPU is created. Allowing it to be called after that only results in confusion but nothing good.

Quoted 7 lines

Note this is an ABI change. But currently in Qemu (the de facto userspace VMM) only TDX uses this VM ioctl, and it is only called once before creating any vCPU, therefore the risk of breaking userspace is pretty low.

Quoted 19 lines
---
 arch/x86/kvm/x86.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kvm/x86.c +4 -0
index 699ca5e74bba..e5e55d549468 100644
@@ -7194,6 +7194,10 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
 		u32 user_tsc_khz;
 
 		r = -EINVAL;
+
+		if (kvm->created_vcpus)
+			goto out;
+
 		user_tsc_khz = (u32)arg;
 
 		if (kvm_caps.has_tsc_control &&
author
#16 lore raw

Re: [PATCH 2/2] KVM: x86: Reject KVM_SET_TSC_KHZ VM ioctl when vCPU has been created

Quoted 8 lines · Nikunj A. Dadhania
Quoted 2 lines · Kai Huang

Reject the KVM_SET_TSC_KHZ VM ioctl when there's vCPU has already been created.

Probably the below is clear:

Reject KVM_SET_TSC_KHZ VM ioctl when vCPUs have been created

Will do. Thanks.

Quoted 12 lines
Quoted 12 lines

The VM scope KVM_SET_TSC_KHZ ioctl is used to set up the default TSC frequency that all subsequent created vCPUs use. It is only intended to be called before any vCPU is created. Allowing it to be called after that only results in confusion but nothing good.

Note this is an ABI change. But currently in Qemu (the de facto userspace VMM) only TDX uses this VM ioctl, and it is only called once before creating any vCPU, therefore the risk of breaking userspace is pretty low.

Quoted 4 lines
Quoted 2 lines

Thanks. I'll keep you guys RB anyway after adding the mutex. Let me know if you are not OK.