KVM: x86: Enhance vendor module error messages
Sean Christopherson
maintainercover KVM: x86: Enhance vendor module error messages
Paul Menzel encountered the bad userspace behavior of spamming KVM module loading on all CPUs, which was mitigated by commit ef935c25fd64 ("kvm: x86:
Limit the number of "kvm: disabled by bios" messages"), except this time userspace is extra "clever" and spams both kvm_intel and kvm_amd. Because the "already loaded the other module" message isn't ratelimited, the bogus module load managed to spam the kernel log.
Patch 1 addresses another suggestion from Paul by incorporating the vendor module name into the error messages.
Patch 2 addresses the original report by prioritizing the hardware/bios support messages over the "already loaded" error. In additional to reducing spam, doing so also ensures consistent messaging if a vendor module isn't supported regardless of what other modules may be loaded.
[*] https://lkml.kernel.org/r/[email protected]
Sean Christopherson (2):
KVM: x86: Add vendor name to kvm_x86_ops, use it for error messages
KVM: x86: Defer "already loaded" check until after basic support
checks
arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 17 +++++++++-------- 4 files changed, 15 insertions(+), 8 deletions(-)
Signature · 1 line
2.33.0.1079.g6e70778dc9-goog
Sean Christopherson
maintainerPATCH 1/2 KVM: x86: Add vendor name to kvm_x86_ops, use it for error messages
Paul pointed out the error messages when KVM fails to load are unhelpful in understanding exactly what went wrong if userspace probes the "wrong" module.
Add a mandatory kvm_x86_ops field to track vendor module names, kvm_intel and kvm_amd, and use the name for relevant error message when KVM fails to load so that the user knows which module failed to load.
Opportunistically tweak the "disabled by bios" error message to clarify that _support_ was disabled, not that the module itself was magically disabled by BIOS.
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 8 +++++--- 4 files changed, 11 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1302,6 +1302,8 @@ static inline u16 kvm_lapic_irq_dest_mode(bool dest_mode_logical) }struct kvm_x86_ops { +const char *name; +int (*hardware_enable)(void );void (*hardware_disable)(void );void (*hardware_unsetup)(void );
arch/x86/kvm/svm/svm.c +2 -0
@@ -4580,6 +4580,8 @@ static int svm_vm_init(struct kvm *kvm) }static struct kvm_x86_ops svm_x86_ops__initdata = { + .name ="kvm_amd" , + .hardware_unsetup = svm_hardware_teardown, .hardware_enable = svm_hardware_enable, .hardware_disable = svm_hardware_disable,
arch/x86/kvm/vmx/vmx.c +2 -0
@@ -7568,6 +7568,8 @@ static bool vmx_check_apicv_inhibit_reasons(ulong bit) }static struct kvm_x86_ops vmx_x86_ops__initdata = { + .name ="kvm_intel" , + .hardware_unsetup = hardware_unsetup, .hardware_enable = hardware_enable,
arch/x86/kvm/x86.c +5 -3
@@ -8539,18 +8539,20 @@ int kvm_arch_init(void *opaque)int r;if (kvm_x86_ops.hardware_enable) { -printk (KERN_ERR "kvm: already loaded the other module\n" ); +pr_err ("kvm: already loaded vendor module '%s'\n" , kvm_x86_ops.name); r = -EEXIST ; goto out; }if (!ops->cpu_has_kvm_support ()) { -pr_err_ratelimited ("kvm: no hardware support\n" ); +pr_err_ratelimited ("kvm: no hardware support for '%s'\n" , + ops->runtime_ops->name); r = -EOPNOTSUPP ; goto out; }if (ops->disabled_by_bios ()) { -pr_err_ratelimited ("kvm: disabled by bios\n" ); +pr_err_ratelimited ("kvm: support for '%s' disabled by bios\n" , + ops->runtime_ops->name); r = -EOPNOTSUPP ; goto out; }
Signature · 1 line
2.33.0.1079.g6e70778dc9-goog
Re: [PATCH 1/2] KVM: x86: Add vendor name to kvm_x86_ops, use it for error messages
Quoted 12 lines · Sean Christopherson
Paul pointed out the error messages when KVM fails to load are unhelpful in understanding exactly what went wrong if userspace probes the "wrong" module.
Add a mandatory
kvm_x86_opsfield to track vendor module names,kvm_intelandkvm_amd, and use the name for relevant error message when KVM fails to load so that the user knows which module failed to load.Opportunistically tweak the "disabled by bios" error message to clarify that _support_ was disabled, not that the module itself was magically disabled by BIOS.
...
Quoted 63 lines
Suggested-by Paul Menzel <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 8 +++++--- 4 files changed, 11 insertions(+), 3 deletions(-)arch/x86/include/asm/kvm_host.h +2
-0@@ -1302,6 +1302,8 @@ static inline u16 kvm_lapic_irq_dest_mode(bool dest_mode_logical) }struct kvm_x86_ops { +const char *name; +int (*hardware_enable)(void );void (*hardware_disable)(void );void (*hardware_unsetup)(void );arch/x86/kvm/svm/svm.c +2
-0@@ -4580,6 +4580,8 @@ static int svm_vm_init(struct kvm *kvm) }static struct kvm_x86_ops svm_x86_ops__initdata = { + .name ="kvm_amd" , + .hardware_unsetup = svm_hardware_teardown, .hardware_enable = svm_hardware_enable, .hardware_disable = svm_hardware_disable,arch/x86/kvm/vmx/vmx.c +2
-0@@ -7568,6 +7568,8 @@ static bool vmx_check_apicv_inhibit_reasons(ulong bit) }static struct kvm_x86_ops vmx_x86_ops__initdata = { + .name ="kvm_intel" , + .hardware_unsetup = hardware_unsetup, .hardware_enable = hardware_enable,arch/x86/kvm/x86.c +1
-1@@ -8539,18 +8539,20 @@ int kvm_arch_init(void *opaque)int r;if (kvm_x86_ops.hardware_enable) { -printk (KERN_ERR "kvm: already loaded the other module\n" ); +pr_err ("kvm: already loaded vendor module '%s'\n" , kvm_x86_ops.name); r = -EEXIST ; goto out; }if (!ops->cpu_has_kvm_support ()) {
arch/x86/kvm/x86.c int kvm_arch_init(void *opaque) · line 8555
Quoted 10 lines
arch/x86/kvm/x86.c +4
-2-pr_err_ratelimited ("kvm: no hardware support\n" ); +pr_err_ratelimited ("kvm: no hardware support for '%s'\n" , + ops->runtime_ops->name); r = -EOPNOTSUPP ; goto out; }if (ops->disabled_by_bios ()) { -pr_err_ratelimited ("kvm: disabled by bios\n" ); +pr_err_ratelimited ("kvm: support for '%s' disabled by bios\n" , + ops->runtime_ops->name);
I'd suggest we change this to
pr_err_ratelimited("kvm: %s: virtualization disabled in BIOS\n",
ops->runtime_ops->name);or something like that as generally, it makes little sense to search for 'KVM' in BIOS settings. You need too look for either 'Virtualization' or VT-x/AMD-v.
Quoted 3 lines
arch/x86/kvm/x86.c +0
-0r = -EOPNOTSUPP ; goto out; }
Signature · 1 line
Vitaly
Sean Christopherson
maintainerRe: [PATCH 1/2] KVM: x86: Add vendor name to kvm_x86_ops, use it for error messages
Quoted 8 lines · Vitaly Kuznetsov
arch/x86/kvm/x86.c int kvm_arch_init(void *opaque) · line 8555
Quoted 4 lines
arch/x86/kvm/x86.c +2
-1if (ops->disabled_by_bios ()) { -pr_err_ratelimited ("kvm: disabled by bios\n" ); +pr_err_ratelimited ("kvm: support for '%s' disabled by bios\n" , + ops->runtime_ops->name);I'd suggest we change this to
Quoted 6 lines
pr_err_ratelimited("kvm: %s: virtualization disabled in BIOS\n", ops->runtime_ops->name);or something like that as generally, it makes little sense to search for 'KVM' in BIOS settings. You need too look for either 'Virtualization' or VT-x/AMD-v.
I'd prefer to avoid VT-x/AMD-v so as not to speculate on the module being loaded or the underlying hardware, e.g. I've no idea what Hygon, Zhaoxin, etc... use for "code" names.
What about
pr_err_ratelimited("kvm: virtualization support for '%s' disabled by BIOS\n",
ops->runtime_ops->name);to add the virtualization flavor but still make it clear that error is coming from the base kvm module.
Re: [PATCH 1/2] KVM: x86: Add vendor name to kvm_x86_ops, use it for error messages
Quoted 20 lines · Sean Christopherson
Quoted 8 lines · Vitaly Kuznetsov
arch/x86/kvm/x86.c int kvm_arch_init(void *opaque) · line 8555
Quoted 4 lines
arch/x86/kvm/x86.c +2
-1if (ops->disabled_by_bios ()) { -pr_err_ratelimited ("kvm: disabled by bios\n" ); +pr_err_ratelimited ("kvm: support for '%s' disabled by bios\n" , + ops->runtime_ops->name);I'd suggest we change this to
Quoted 6 lines
pr_err_ratelimited("kvm: %s: virtualization disabled in BIOS\n", ops->runtime_ops->name);or something like that as generally, it makes little sense to search for 'KVM' in BIOS settings. You need too look for either 'Virtualization' or VT-x/AMD-v.
I'd prefer to avoid VT-x/AMD-v so as not to speculate on the module being loaded or the underlying hardware, e.g. I've no idea what Hygon, Zhaoxin, etc... use for "code" names.
Quoted 7 lines
What about
pr_err_ratelimited("kvm: virtualization support for '%s' disabled by BIOS\n", ops->runtime_ops->name);to add the virtualization flavor but still make it clear that error is coming from the base kvm module.
Works for me, thanks! I just want to make sure people know what to do when they see the message.
Signature · 1 line
Vitaly
Sean Christopherson
maintainerPATCH 2/2 KVM: x86: Defer "already loaded" check until after basic support checks
In reply to Sean ChristophersonMove KVM's check for a vendor module already being loaded below the basic functionality checks so that attempting to load an unsupported module provides the same result regardless of whether or not a supported vendor module is already loaded.
Intentionally keep the err non-ratelimited; if userspace is probing two different modules for the same vendor on all CPUs, it deserves the spam.
--- arch/x86/kvm/x86.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)
arch/x86/kvm/x86.c +5 -6
@@ -8538,12 +8538,6 @@ int kvm_arch_init(void *opaque)struct kvm_x86_init_ops *ops = opaque;int r; -if (kvm_x86_ops.hardware_enable) { -pr_err ("kvm: already loaded vendor module '%s'\n" , kvm_x86_ops.name); - r = -EEXIST ; - goto out; - } -if (!ops->cpu_has_kvm_support ()) {pr_err_ratelimited ("kvm: no hardware support for '%s'\n" , ops->runtime_ops->name);
@@ -8556,6 +8550,11 @@ int kvm_arch_init(void *opaque) r = -EOPNOTSUPP ; goto out; } +if (kvm_x86_ops.hardware_enable) { +pr_err ("kvm: already loaded vendor module '%s'\n" , kvm_x86_ops.name); + r = -EEXIST ; + goto out; + }/* * KVM explicitly assumes that the guest has an FPU and
Signature · 1 line
2.33.0.1079.g6e70778dc9-goog
Quoted 22 lines · Sean Christopherson
Paul Menzel encountered the bad userspace behavior of spamming KVM module loading on all CPUs, which was mitigated by commit
ef935c25fd64("kvm: x86:
Limit the number of "kvm: disabled by bios" messages"), except this time userspace is extra "clever" and spams bothkvm_intelandkvm_amd. Because the "already loaded the other module" message isn't ratelimited, the bogus module load managed to spam the kernel log.Patch 1 addresses another suggestion from Paul by incorporating the vendor module name into the error messages.
Patch 2 addresses the original report by prioritizing the hardware/bios support messages over the "already loaded" error. In additional to reducing spam, doing so also ensures consistent messaging if a vendor module isn't supported regardless of what other modules may be loaded.
[*] https://lkml.kernel.org/r/[email protected]
Sean Christopherson (2):
KVM: x86: Add vendor name tokvm_x86_ops, use it for error messages
KVM: x86: Defer "already loaded" check until after basic support
checks
Quoted 6 lines
arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 17 +++++++++-------- 4 files changed, 15 insertions(+), 8 deletions(-)
Queued, thanks. But I kept the ratelimiting; given the recent "clashes" with the chief penguin, I am also ratelimiting the snark in the commit messages.
Paolo