Skip to contentSkip to conversation

KVM: x86: Enhance vendor module error messages

PATCH Applied Sean Christopherson 3 participants 7 messages Oct 18, 2021 – Oct 22, 2021

Subsystems (3)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

cover KVM: x86: Enhance vendor module error messages

Paul Menzel encountered the bad userspace behavior of spamming KVM module loading on all CPUs, which was mitigated by commit ef935c25fd64 ("kvm: x86:
Limit the number of "kvm: disabled by bios" messages"), except this time userspace is extra "clever" and spams both kvm_intel and kvm_amd. Because the "already loaded the other module" message isn't ratelimited, the bogus module load managed to spam the kernel log.

Patch 1 addresses another suggestion from Paul by incorporating the vendor module name into the error messages.

Patch 2 addresses the original report by prioritizing the hardware/bios support messages over the "already loaded" error. In additional to reducing spam, doing so also ensures consistent messaging if a vendor module isn't supported regardless of what other modules may be loaded.

[*] https://lkml.kernel.org/r/[email protected]

Sean Christopherson (2):
KVM: x86: Add vendor name to kvm_x86_ops, use it for error messages
KVM: x86: Defer "already loaded" check until after basic support
checks

 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/svm/svm.c          |  2 ++
 arch/x86/kvm/vmx/vmx.c          |  2 ++
 arch/x86/kvm/x86.c              | 17 +++++++++--------
 4 files changed, 15 insertions(+), 8 deletions(-)
Signature · 1 line
2.33.0.1079.g6e70778dc9-goog
author maintainer
#1 lore raw

PATCH 1/2 KVM: x86: Add vendor name to kvm_x86_ops, use it for error messages

Paul pointed out the error messages when KVM fails to load are unhelpful in understanding exactly what went wrong if userspace probes the "wrong" module.

Add a mandatory kvm_x86_ops field to track vendor module names, kvm_intel and kvm_amd, and use the name for relevant error message when KVM fails to load so that the user knows which module failed to load.

Opportunistically tweak the "disabled by bios" error message to clarify that _support_ was disabled, not that the module itself was magically disabled by BIOS.

---
 arch/x86/include/asm/kvm_host.h | 2 ++
 arch/x86/kvm/svm/svm.c          | 2 ++
 arch/x86/kvm/vmx/vmx.c          | 2 ++
 arch/x86/kvm/x86.c              | 8 +++++---
 4 files changed, 11 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
index 80f4b8a9233c..b05bfcc72042 100644
@@ -1302,6 +1302,8 @@ static inline u16 kvm_lapic_irq_dest_mode(bool dest_mode_logical)
 }
 
 struct kvm_x86_ops {
+	const char *name;
+
 	int (*hardware_enable)(void);
 	void (*hardware_disable)(void);
 	void (*hardware_unsetup)(void);
arch/x86/kvm/svm/svm.c +2 -0
index 89077160d463..cee4915d2ce3 100644
@@ -4580,6 +4580,8 @@ static int svm_vm_init(struct kvm *kvm)
 }
 
 static struct kvm_x86_ops svm_x86_ops __initdata = {
+	.name = "kvm_amd",
+
 	.hardware_unsetup = svm_hardware_teardown,
 	.hardware_enable = svm_hardware_enable,
 	.hardware_disable = svm_hardware_disable,
arch/x86/kvm/vmx/vmx.c +2 -0
index 1c8b2b6e7ed9..c147438eaafc 100644
@@ -7568,6 +7568,8 @@ static bool vmx_check_apicv_inhibit_reasons(ulong bit)
 }
 
 static struct kvm_x86_ops vmx_x86_ops __initdata = {
+	.name = "kvm_intel",
+
 	.hardware_unsetup = hardware_unsetup,
 
 	.hardware_enable = hardware_enable,
arch/x86/kvm/x86.c +5 -3
index c59b63c56af9..e966e9cdd805 100644
@@ -8539,18 +8539,20 @@ int kvm_arch_init(void *opaque)
 	int r;
 
 	if (kvm_x86_ops.hardware_enable) {
-		printk(KERN_ERR "kvm: already loaded the other module\n");
+		pr_err("kvm: already loaded vendor module '%s'\n", kvm_x86_ops.name);
 		r = -EEXIST;
 		goto out;
 	}
 
 	if (!ops->cpu_has_kvm_support()) {
-		pr_err_ratelimited("kvm: no hardware support\n");
+		pr_err_ratelimited("kvm: no hardware support for '%s'\n",
+				   ops->runtime_ops->name);
 		r = -EOPNOTSUPP;
 		goto out;
 	}
 	if (ops->disabled_by_bios()) {
-		pr_err_ratelimited("kvm: disabled by bios\n");
+		pr_err_ratelimited("kvm: support for '%s' disabled by bios\n",
+				   ops->runtime_ops->name);
 		r = -EOPNOTSUPP;
 		goto out;
 	}
Signature · 1 line
2.33.0.1079.g6e70778dc9-goog
#2 lore raw

Re: [PATCH 1/2] KVM: x86: Add vendor name to kvm_x86_ops, use it for error messages

Quoted 12 lines · Sean Christopherson

Paul pointed out the error messages when KVM fails to load are unhelpful in understanding exactly what went wrong if userspace probes the "wrong" module.

Add a mandatory kvm_x86_ops field to track vendor module names, kvm_intel and kvm_amd, and use the name for relevant error message when KVM fails to load so that the user knows which module failed to load.

Opportunistically tweak the "disabled by bios" error message to clarify that _support_ was disabled, not that the module itself was magically disabled by BIOS.

...

Quoted 63 lines
---
 arch/x86/include/asm/kvm_host.h | 2 ++
 arch/x86/kvm/svm/svm.c          | 2 ++
 arch/x86/kvm/vmx/vmx.c          | 2 ++
 arch/x86/kvm/x86.c              | 8 +++++---
 4 files changed, 11 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
index 80f4b8a9233c..b05bfcc72042 100644
@@ -1302,6 +1302,8 @@ static inline u16 kvm_lapic_irq_dest_mode(bool dest_mode_logical)
 }
 
 struct kvm_x86_ops {
+	const char *name;
+
 	int (*hardware_enable)(void);
 	void (*hardware_disable)(void);
 	void (*hardware_unsetup)(void);
arch/x86/kvm/svm/svm.c +2 -0
index 89077160d463..cee4915d2ce3 100644
@@ -4580,6 +4580,8 @@ static int svm_vm_init(struct kvm *kvm)
 }
 
 static struct kvm_x86_ops svm_x86_ops __initdata = {
+	.name = "kvm_amd",
+
 	.hardware_unsetup = svm_hardware_teardown,
 	.hardware_enable = svm_hardware_enable,
 	.hardware_disable = svm_hardware_disable,
arch/x86/kvm/vmx/vmx.c +2 -0
index 1c8b2b6e7ed9..c147438eaafc 100644
@@ -7568,6 +7568,8 @@ static bool vmx_check_apicv_inhibit_reasons(ulong bit)
 }
 
 static struct kvm_x86_ops vmx_x86_ops __initdata = {
+	.name = "kvm_intel",
+
 	.hardware_unsetup = hardware_unsetup,
 
 	.hardware_enable = hardware_enable,
arch/x86/kvm/x86.c +1 -1
index c59b63c56af9..e966e9cdd805 100644
@@ -8539,18 +8539,20 @@ int kvm_arch_init(void *opaque)
 	int r;
 
 	if (kvm_x86_ops.hardware_enable) {
-		printk(KERN_ERR "kvm: already loaded the other module\n");
+		pr_err("kvm: already loaded vendor module '%s'\n", kvm_x86_ops.name);
 		r = -EEXIST;
 		goto out;
 	}
 
 	if (!ops->cpu_has_kvm_support()) {

arch/x86/kvm/x86.c int kvm_arch_init(void *opaque) · line 8555

Quoted 10 lines
arch/x86/kvm/x86.c +4 -2
-		pr_err_ratelimited("kvm: no hardware support\n");
+		pr_err_ratelimited("kvm: no hardware support for '%s'\n",
+				   ops->runtime_ops->name);
 		r = -EOPNOTSUPP;
 		goto out;
 	}
 	if (ops->disabled_by_bios()) {
-		pr_err_ratelimited("kvm: disabled by bios\n");
+		pr_err_ratelimited("kvm: support for '%s' disabled by bios\n",
+				   ops->runtime_ops->name);

I'd suggest we change this to

		pr_err_ratelimited("kvm: %s: virtualization disabled in BIOS\n",
				   ops->runtime_ops->name);

or something like that as generally, it makes little sense to search for 'KVM' in BIOS settings. You need too look for either 'Virtualization' or VT-x/AMD-v.

Quoted 3 lines
arch/x86/kvm/x86.c +0 -0
 		r = -EOPNOTSUPP;
 		goto out;
 	}
Signature · 1 line
Vitaly
author maintainer
#3 lore raw

Re: [PATCH 1/2] KVM: x86: Add vendor name to kvm_x86_ops, use it for error messages

Quoted 8 lines · Vitaly Kuznetsov

arch/x86/kvm/x86.c int kvm_arch_init(void *opaque) · line 8555

Quoted 4 lines
arch/x86/kvm/x86.c +2 -1
 	if (ops->disabled_by_bios()) {
-		pr_err_ratelimited("kvm: disabled by bios\n");
+		pr_err_ratelimited("kvm: support for '%s' disabled by bios\n",
+				   ops->runtime_ops->name);

I'd suggest we change this to

Quoted 6 lines
		pr_err_ratelimited("kvm: %s: virtualization disabled in BIOS\n",
				   ops->runtime_ops->name);

or something like that as generally, it makes little sense to search for 'KVM' in BIOS settings. You need too look for either 'Virtualization' or VT-x/AMD-v.

I'd prefer to avoid VT-x/AMD-v so as not to speculate on the module being loaded or the underlying hardware, e.g. I've no idea what Hygon, Zhaoxin, etc... use for "code" names.

What about

		pr_err_ratelimited("kvm: virtualization support for '%s' disabled by BIOS\n",
				   ops->runtime_ops->name);

to add the virtualization flavor but still make it clear that error is coming from the base kvm module.

#4 lore raw

Re: [PATCH 1/2] KVM: x86: Add vendor name to kvm_x86_ops, use it for error messages

Quoted 20 lines · Sean Christopherson
Quoted 8 lines · Vitaly Kuznetsov

arch/x86/kvm/x86.c int kvm_arch_init(void *opaque) · line 8555

Quoted 4 lines
arch/x86/kvm/x86.c +2 -1
 	if (ops->disabled_by_bios()) {
-		pr_err_ratelimited("kvm: disabled by bios\n");
+		pr_err_ratelimited("kvm: support for '%s' disabled by bios\n",
+				   ops->runtime_ops->name);

I'd suggest we change this to

Quoted 6 lines
		pr_err_ratelimited("kvm: %s: virtualization disabled in BIOS\n",
				   ops->runtime_ops->name);

or something like that as generally, it makes little sense to search for 'KVM' in BIOS settings. You need too look for either 'Virtualization' or VT-x/AMD-v.

I'd prefer to avoid VT-x/AMD-v so as not to speculate on the module being loaded or the underlying hardware, e.g. I've no idea what Hygon, Zhaoxin, etc... use for "code" names.

Quoted 7 lines

What about

		pr_err_ratelimited("kvm: virtualization support for '%s' disabled by BIOS\n",
				   ops->runtime_ops->name);

to add the virtualization flavor but still make it clear that error is coming from the base kvm module.

Works for me, thanks! I just want to make sure people know what to do when they see the message.

Signature · 1 line
Vitaly
author maintainer
#5 lore raw

PATCH 2/2 KVM: x86: Defer "already loaded" check until after basic support checks

In reply to Sean Christopherson

Move KVM's check for a vendor module already being loaded below the basic functionality checks so that attempting to load an unsupported module provides the same result regardless of whether or not a supported vendor module is already loaded.

Intentionally keep the err non-ratelimited; if userspace is probing two different modules for the same vendor on all CPUs, it deserves the spam.

---
 arch/x86/kvm/x86.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
arch/x86/kvm/x86.c +5 -6
index e966e9cdd805..f67da77be267 100644
@@ -8538,12 +8538,6 @@ int kvm_arch_init(void *opaque)
 	struct kvm_x86_init_ops *ops = opaque;
 	int r;
 
-	if (kvm_x86_ops.hardware_enable) {
-		pr_err("kvm: already loaded vendor module '%s'\n", kvm_x86_ops.name);
-		r = -EEXIST;
-		goto out;
-	}
-
 	if (!ops->cpu_has_kvm_support()) {
 		pr_err_ratelimited("kvm: no hardware support for '%s'\n",
 				   ops->runtime_ops->name);
@@ -8556,6 +8550,11 @@ int kvm_arch_init(void *opaque)
 		r = -EOPNOTSUPP;
 		goto out;
 	}
+	if (kvm_x86_ops.hardware_enable) {
+		pr_err("kvm: already loaded vendor module '%s'\n", kvm_x86_ops.name);
+		r = -EEXIST;
+		goto out;
+	}
 
 	/*
 	 * KVM explicitly assumes that the guest has an FPU and
Signature · 1 line
2.33.0.1079.g6e70778dc9-goog
maintainer Applied
#6 lore raw
In reply to Sean Christopherson
Quoted 22 lines · Sean Christopherson

Paul Menzel encountered the bad userspace behavior of spamming KVM module loading on all CPUs, which was mitigated by commit ef935c25fd64 ("kvm: x86:
Limit the number of "kvm: disabled by bios" messages"), except this time userspace is extra "clever" and spams both kvm_intel and kvm_amd. Because the "already loaded the other module" message isn't ratelimited, the bogus module load managed to spam the kernel log.

Patch 1 addresses another suggestion from Paul by incorporating the vendor module name into the error messages.

Patch 2 addresses the original report by prioritizing the hardware/bios support messages over the "already loaded" error. In additional to reducing spam, doing so also ensures consistent messaging if a vendor module isn't supported regardless of what other modules may be loaded.

[*] https://lkml.kernel.org/r/[email protected]

Sean Christopherson (2):
KVM: x86: Add vendor name to kvm_x86_ops, use it for error messages
KVM: x86: Defer "already loaded" check until after basic support
checks

Quoted 6 lines
  arch/x86/include/asm/kvm_host.h |  2 ++
  arch/x86/kvm/svm/svm.c          |  2 ++
  arch/x86/kvm/vmx/vmx.c          |  2 ++
  arch/x86/kvm/x86.c              | 17 +++++++++--------
  4 files changed, 15 insertions(+), 8 deletions(-)

Queued, thanks. But I kept the ratelimiting; given the recent "clashes" with the chief penguin, I am also ratelimiting the snark in the commit messages.

Paolo