Fix diagnostics and capability probes for protected VMs
Fuad Tabba
reviewercover Fix diagnostics and capability probes for protected VMs
Hi folks,
These are the kvmtool changes that go with the pKVM series confining a protected VM's vCPU state to EL2 [1].
That series stops the host reading a protected vCPU's registers once it has run, and stops it setting the PVTIME attribute. kvmtool dies on both. Its arm64 register-dump helpers call die() when KVM_GET_ONE_REG fails, which turns a diagnostic into a VMM abort. kvm_cpu__setup_pvtime() runs into the new -EPERM and fails vCPU init.
The PVTIME failure is a probe on the wrong fd. kvmtool asks the global /dev/kvm fd whether steal-time is available, and that fd advertises it for a protected VM, which cannot use it.
The last patch fixes the same wrong-fd probe for the counter offset, where "lkvm run --protected --counter-offset" already fails on Linux 7.3-rc1, without the kernel series.
The same conversion went in for the vCPU feature probes in July [2].
Patches 2 and 3 are not specific to protected VMs. They fix the exit-reason table on the same panic path as patch 1. The lookup ran off the end of it, and the table had stopped naming anything KVM added after KVM_EXIT_INTERNAL_ERROR, which on arm64 includes KVM_EXIT_SYSTEM_EVENT and KVM_EXIT_ARM_NISV.
Patch 1's path needs a protected VM that is already running: "lkvm debug -d" against one kills the VMM without this series, and prints the errno with it.
Based on kvmtool master (f67bc0b).
Cheers,
/fuad
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
Fuad Tabba (5):
arm64: Do not abort on register-dump failures
kvm: Bound-check the exit-reason string lookup
kvm: Name every exit reason the UAPI header defines
arm64: Query steal-time support on the VM fd
arm64: Query counter-offset support on the VM fd
arm64/kvm-cpu.c | 36 ++++++++++++++++++++++++------------ arm64/kvm.c | 2 +- arm64/pvtime.c | 4 ++-- builtin-run.c | 2 +- include/kvm/kvm.h | 2 +- kvm.c | 37 ++++++++++++++++++++++++++++++++++--- 6 files changed, 63 insertions(+), 20 deletions(-)
base-commit: f67bc0bdae9433a9cfd05e65ea2c1bb6102566d9
Signature · 1 line
2.39.5
Fuad Tabba
reviewerPATCH 1/5 arm64: Do not abort on register-dump failures
kvm_cpu__show_registers() and kvm_cpu__show_code() read the vCPU's core registers with KVM_GET_ONE_REG, and die() if the ioctl fails. Both are diagnostics. They run from the KVM_EXIT_DEBUG case in kvm_cpu__start(), from the "lkvm debug -d" dump path in handle_sigusr1(), and from the panic dump in kvm_cpu_thread().
Once a protected vCPU has run, its registers belong to the guest and KVM_GET_ONE_REG returns -EPERM. Dying on that turns a diagnostic into a VMM abort.
Report that the register state is unavailable, with the errno, and return instead of dying.
--- arm64/kvm-cpu.c | 36 ++++++++++++++++++++++++------------ 1 file changed, 24 insertions(+), 12 deletions(-)
arm64/kvm-cpu.c +24 -12
@@ -476,15 +476,19 @@ void kvm_cpu__show_code(struct kvm_cpu *vcpu)dprintf (debug_fd,"\n*pc:\n" ); reg.id =ARM64_CORE_REG (regs.pc); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (show_code @ PC)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pc): %s" ,strerror (errno)); +return ; + }kvm__dump_mem (vcpu->kvm, data,32 , debug_fd);dprintf (debug_fd,"\n*lr:\n" ); reg.id =ARM64_CORE_REG (regs.regs[30 ]); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (show_code @ LR)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (lr): %s" ,strerror (errno)); +return ; + }kvm__dump_mem (vcpu->kvm, data,32 , debug_fd); }
@@ -499,23 +503,31 @@ void kvm_cpu__show_registers(struct kvm_cpu *vcpu)dprintf (debug_fd,"\n Registers:\n" ); reg.id =ARM64_CORE_REG (regs.pc); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (pc)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pc): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," PC: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (regs.pstate); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (pstate)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pstate): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," PSTATE: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (sp_el1); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (sp_el1)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (sp_el1): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," SP_EL1: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (regs.regs[30 ]); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (lr)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (lr): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," LR: 0x%lx\n" , data); }
Signature · 1 line
2.39.5
Re: [PATCH kvmtool 1/5] arm64: Do not abort on register-dump failures
Quoted 9 lines · Fuad Tabba
kvm_cpu__show_registers()andkvm_cpu__show_code()read the vCPU's core registers withKVM_GET_ONE_REG, anddie()if the ioctl fails. Both are diagnostics. They run from theKVM_EXIT_DEBUGcase inkvm_cpu__start(), from the "lkvm debug -d" dump path inhandle_sigusr1(), and from the panic dump inkvm_cpu_thread().Once a protected vCPU has run, its registers belong to the guest and
KVM_GET_ONE_REGreturns-EPERM. Dying on that turns a diagnostic into a VMM abort.
Marc once suggested that these could always succeed with junk values for protected/Realm VMs. Do you think that is an option for pKVM ?
If we go with this approach :
Quoted 78 lines
Report that the register state is unavailable, with the errno, and return instead of dying.
Signed-off-by Fuad Tabba <[email protected]>--- arm64/kvm-cpu.c | 36 ++++++++++++++++++++++++------------ 1 file changed, 24 insertions(+), 12 deletions(-)arm64/kvm-cpu.c +8
-4@@ -476,15 +476,19 @@ void kvm_cpu__show_code(struct kvm_cpu *vcpu)dprintf (debug_fd,"\n*pc:\n" ); reg.id =ARM64_CORE_REG (regs.pc); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (show_code @ PC)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pc): %s" ,strerror (errno)); +return ; + }kvm__dump_mem (vcpu->kvm, data,32 , debug_fd);dprintf (debug_fd,"\n*lr:\n" ); reg.id =ARM64_CORE_REG (regs.regs[30 ]); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (show_code @ LR)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (lr): %s" ,strerror (errno)); +return ; + }kvm__dump_mem (vcpu->kvm, data,32 , debug_fd);}arm64/kvm-cpu.c +16
-8@@ -499,23 +503,31 @@ void kvm_cpu__show_registers(struct kvm_cpu *vcpu)dprintf (debug_fd,"\n Registers:\n" ); reg.id =ARM64_CORE_REG (regs.pc); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (pc)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pc): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," PC: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (regs.pstate); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (pstate)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pstate): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," PSTATE: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (sp_el1); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (sp_el1)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (sp_el1): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," SP_EL1: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (regs.regs[30 ]); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (lr)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (lr): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," LR: 0x%lx\n" , data); }
Fuad Tabba
reviewerRe: [PATCH kvmtool 1/5] arm64: Do not abort on register-dump failures
Hi Suzuki,
Quoted 8 lines · Suzuki K Poulose
Quoted 6 lines · Fuad Tabba
kvm_cpu__show_registers()andkvm_cpu__show_code()read the vCPU's core registers withKVM_GET_ONE_REG, anddie()if the ioctl fails. Both are diagnostics. They run from theKVM_EXIT_DEBUGcase inkvm_cpu__start(), from the "lkvm debug -d" dump path inhandle_sigusr1(), and from the panic dump inkvm_cpu_thread().
Quoted 6 lines
Quoted 3 lines
Once a protected vCPU has run, its registers belong to the guest and
KVM_GET_ONE_REGreturns-EPERM. Dying on that turns a diagnostic into a VMM abort.Marc once suggested that these could always succeed with junk values for protected/Realm VMs. Do you think that is an option for pKVM ?
It's possible, the host copy is still there, but I'd rather not. Once the vCPU has run that copy is a mix rather than junk: the boot state the VMM wrote, plus what the exit handlers copy out (the PSTATE mode, x0 for an MMIO write, x0 to x2 for a forwarded PSCI call). A dump would show a live PSTATE beside the PC from boot, and the VMM has no way to distinguish them. With the error it prints that the state is unavailable, which is what this patch does.
It's also what the tree does for protected state elsewhere: s390 returns -EINVAL from KVM_GET/SET_ONE_REG on a protected VM, and x86 does the same from the register ioctls for the SEV-ES VM type, SNP and TDX. Legacy SEV-ES is the one place that succeeds silently, kept for backwards compatibility, and the commit that added the errors (517987e3fb19) describes that as a problem. Sean's rationale for TDX:
KVM can't provide sane data, so it's userspace's job not to ask for it [1].
Quoted 4 lines
If we go with this approach :
Reviewed-by Suzuki K Poulose <[email protected]>
Thanks, I'll add the tag. :)
Cheers,
/fuad
Quoted 81 lines
Quoted 80 lines
Report that the register state is unavailable, with the errno, and return instead of dying.
Signed-off-by Fuad Tabba <[email protected]>--- arm64/kvm-cpu.c | 36 ++++++++++++++++++++++++------------ 1 file changed, 24 insertions(+), 12 deletions(-)arm64/kvm-cpu.c +8
-4@@ -476,15 +476,19 @@ void kvm_cpu__show_code(struct kvm_cpu *vcpu)dprintf (debug_fd,"\n*pc:\n" ); reg.id =ARM64_CORE_REG (regs.pc); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (show_code @ PC)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pc): %s" ,strerror (errno)); +return ; + }kvm__dump_mem (vcpu->kvm, data,32 , debug_fd);dprintf (debug_fd,"\n*lr:\n" ); reg.id =ARM64_CORE_REG (regs.regs[30 ]); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (show_code @ LR)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (lr): %s" ,strerror (errno)); +return ; + }kvm__dump_mem (vcpu->kvm, data,32 , debug_fd);}arm64/kvm-cpu.c +16
-8@@ -499,23 +503,31 @@ void kvm_cpu__show_registers(struct kvm_cpu *vcpu)dprintf (debug_fd,"\n Registers:\n" ); reg.id =ARM64_CORE_REG (regs.pc); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (pc)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pc): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," PC: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (regs.pstate); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (pstate)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pstate): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," PSTATE: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (sp_el1); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (sp_el1)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (sp_el1): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," SP_EL1: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (regs.regs[30 ]); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (lr)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (lr): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," LR: 0x%lx\n" , data); }
Fuad Tabba
reviewerRe: [PATCH kvmtool 1/5] arm64: Do not abort on register-dump failures
Quoted 37 lines · Fuad Tabba
Hi Suzuki,
Quoted 8 lines · Suzuki K Poulose
Quoted 6 lines · Fuad Tabba
kvm_cpu__show_registers()andkvm_cpu__show_code()read the vCPU's core registers withKVM_GET_ONE_REG, anddie()if the ioctl fails. Both are diagnostics. They run from theKVM_EXIT_DEBUGcase inkvm_cpu__start(), from the "lkvm debug -d" dump path inhandle_sigusr1(), and from the panic dump inkvm_cpu_thread().Quoted 6 lines
Quoted 3 lines
Once a protected vCPU has run, its registers belong to the guest and
KVM_GET_ONE_REGreturns-EPERM. Dying on that turns a diagnostic into a VMM abort.Marc once suggested that these could always succeed with junk values for protected/Realm VMs. Do you think that is an option for pKVM ?
It's possible, the host copy is still there, but I'd rather not. Once the vCPU has run that copy is a mix rather than junk: the boot state the VMM wrote, plus what the exit handlers copy out (the PSTATE mode, x0 for an MMIO write, x0 to x2 for a forwarded PSCI call). A dump would show a live PSTATE beside the PC from boot, and the VMM has no way to distinguish them. With the error it prints that the state is unavailable, which is what this patch does.
It's also what the tree does for protected state elsewhere: s390 returns
-EINVALfrom KVM_GET/SET_ONE_REG on a protected VM, and x86 does the same from the register ioctls for the SEV-ES VM type, SNP and TDX. Legacy SEV-ES is the one place that succeeds silently, kept for backwards compatibility, and the commit that added the errors (517987e3fb19) describes that as a problem. Sean's rationale for TDX:
KVM can't provide sane data, so it's userspace's job not to ask for it [1].Quoted 1 line
Quoted 5 lines
Quoted 3 lines
If we go with this approach :
Reviewed-by Suzuki K Poulose <[email protected]>Thanks, I'll add the tag. :)
Unless I misunderstood what "this" refers to exactly, please shout if that's the case, or continue the discussion :)
/fuad
Quoted 87 lines
Cheers,
/fuadQuoted 81 lines
Quoted 80 lines
Report that the register state is unavailable, with the errno, and return instead of dying.
Signed-off-by Fuad Tabba <[email protected]>--- arm64/kvm-cpu.c | 36 ++++++++++++++++++++++++------------ 1 file changed, 24 insertions(+), 12 deletions(-)arm64/kvm-cpu.c +8
-4@@ -476,15 +476,19 @@ void kvm_cpu__show_code(struct kvm_cpu *vcpu)dprintf (debug_fd,"\n*pc:\n" ); reg.id =ARM64_CORE_REG (regs.pc); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (show_code @ PC)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pc): %s" ,strerror (errno)); +return ; + }kvm__dump_mem (vcpu->kvm, data,32 , debug_fd);dprintf (debug_fd,"\n*lr:\n" ); reg.id =ARM64_CORE_REG (regs.regs[30 ]); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (show_code @ LR)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (lr): %s" ,strerror (errno)); +return ; + }kvm__dump_mem (vcpu->kvm, data,32 , debug_fd);}arm64/kvm-cpu.c +16
-8@@ -499,23 +503,31 @@ void kvm_cpu__show_registers(struct kvm_cpu *vcpu)dprintf (debug_fd,"\n Registers:\n" ); reg.id =ARM64_CORE_REG (regs.pc); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (pc)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pc): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," PC: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (regs.pstate); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (pstate)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (pstate): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," PSTATE: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (sp_el1); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (sp_el1)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (sp_el1): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," SP_EL1: 0x%lx\n" , data); reg.id =ARM64_CORE_REG (regs.regs[30 ]); -if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) -die ("KVM_GET_ONE_REG failed (lr)" ); +if (ioctl (vcpu->vcpu_fd,KVM_GET_ONE_REG , ®) <0 ) { +pr_err ("register state unavailable (lr): %s" ,strerror (errno)); +return ; + }dprintf (debug_fd," LR: 0x%lx\n" , data); }
Suzuki K Poulose
reviewerRe: [PATCH kvmtool 1/5] arm64: Do not abort on register-dump failures
Quoted 28 lines · Fuad Tabba
Quoted 27 lines · Fuad Tabba
Hi Suzuki,
Quoted 8 lines · Suzuki K Poulose
Quoted 6 lines · Fuad Tabba
kvm_cpu__show_registers()andkvm_cpu__show_code()read the vCPU's core registers withKVM_GET_ONE_REG, anddie()if the ioctl fails. Both are diagnostics. They run from theKVM_EXIT_DEBUGcase inkvm_cpu__start(), from the "lkvm debug -d" dump path inhandle_sigusr1(), and from the panic dump inkvm_cpu_thread().Quoted 6 lines
Quoted 3 lines
Once a protected vCPU has run, its registers belong to the guest and
KVM_GET_ONE_REGreturns-EPERM. Dying on that turns a diagnostic into a VMM abort.Marc once suggested that these could always succeed with junk values for protected/Realm VMs. Do you think that is an option for pKVM ?
It's possible, the host copy is still there, but I'd rather not. Once the vCPU has run that copy is a mix rather than junk: the boot state the VMM wrote, plus what the exit handlers copy out (the PSTATE mode, x0 for an MMIO write, x0 to x2 for a forwarded PSCI call). A dump would show a live PSTATE beside the PC from boot, and the VMM has no way to distinguish them. With the error it prints that the state is unavailable, which is what this patch does.
Quoted 9 lines
Quoted 9 lines
It's also what the tree does for protected state elsewhere: s390 returns
-EINVALfrom KVM_GET/SET_ONE_REG on a protected VM, and x86 does the same from the register ioctls for the SEV-ES VM type, SNP and TDX. Legacy SEV-ES is the one place that succeeds silently, kept for backwards compatibility, and the commit that added the errors (517987e3fb19) describes that as a problem. Sean's rationale for TDX:
KVM can't provide sane data, so it's userspace's job not to ask for it [1].
Thanks, I am aware of this. I will leave the decision to Marc. For the record, for CCA we use GET_ONE_REG/SET_ONE_REG to configure the SVE vector length, PMU Counter and the Debug HW BPRs and watch points.
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
If we go with this approach :
Reviewed-by Suzuki K Poulose <[email protected]>Thanks, I'll add the tag. :)
Unless I misunderstood what "this" refers to exactly, please shout if that's the case, or continue the discussion :)
You got it right. I meant if we are happy to go with approach in the patch.
Fuad Tabba
reviewerPATCH 2/5 kvm: Bound-check the exit-reason string lookup
In reply to Fuad Tabbakvm_cpu_thread()'s panic path uses the kernel's exit_reason to index kvm_exit_reasons[] without checking it against the array's size. The table does not cover every exit reason KVM can return, so a reason past its last entry reads out of bounds. KVM_EXIT_SYSTEM_EVENT and KVM_EXIT_ARM_NISV are both past it.
Move the table behind kvm__exit_reason_str(), which checks the index first. The array now has a single caller inside kvm.c, so make it static const.
--- builtin-run.c | 2 +- include/kvm/kvm.h | 2 +- kvm.c | 10 +++++++++- 3 files changed, 11 insertions(+), 3 deletions(-)
builtin-run.c +1 -1
@@ -298,7 +298,7 @@ static void *kvm_cpu_thread(void *arg) panic_kvm:pr_err ("KVM exit reason: %u (\"%s\")" , current_kvm_cpu->kvm_run->exit_reason, - kvm_exit_reasons[current_kvm_cpu->kvm_run->exit_reason]); +kvm__exit_reason_str (current_kvm_cpu->kvm_run->exit_reason));if (current_kvm_cpu->kvm_run->exit_reason ==KVM_EXIT_UNKNOWN ) { pr_err("KVM exit code: %llu",
include/kvm/kvm.h +1 -1
@@ -255,7 +255,7 @@ int kvm__for_each_mem_bank(struct kvm *kvm, enum kvm_mem_type type,*/ void kvm__dump_mem (struct kvm *kvm,unsigned long addr,unsigned long size,int debug_fd); -extern const char *kvm_exit_reasons[]; +const char *kvm__exit_reason_str (__u32 exit_reason);static inline bool host_ptr_in_ram (struct kvm *kvm,void *p) {
kvm.c +9 -1
@@ -33,7 +33,7 @@#define DEFINE_KVM_EXIT_REASON (reason) [reason] = #reason -const char *kvm_exit_reasons[] = { +static const char *const kvm_exit_reasons[] = {DEFINE_KVM_EXIT_REASON (KVM_EXIT_UNKNOWN ),DEFINE_KVM_EXIT_REASON (KVM_EXIT_EXCEPTION ),DEFINE_KVM_EXIT_REASON (KVM_EXIT_IO ),
@@ -57,6 +57,14 @@ const char *kvm_exit_reasons[] = {#endif }; +const char *kvm__exit_reason_str (__u32 exit_reason) +{ +if (exit_reason >=ARRAY_SIZE (kvm_exit_reasons)) +return "UNKNOWN" ; + +return kvm_exit_reasons[exit_reason]; +} +static int pause_event;static DEFINE_MUTEX (pause_lock);static struct kvm_cpu *pause_req_cpu;
Signature · 1 line
2.39.5
Re: [PATCH kvmtool 2/5] kvm: Bound-check the exit-reason string lookup
Quoted 11 lines · Fuad Tabba
kvm_cpu_thread()'s panic path uses the kernel's
exit_reasonto index kvm_exit_reasons[] without checking it against the array's size. The table does not cover every exit reason KVM can return, so a reason past its last entry reads out of bounds.KVM_EXIT_SYSTEM_EVENTandKVM_EXIT_ARM_NISVare both past it.Move the table behind
kvm__exit_reason_str(), which checks the index first. The array now has a single caller insidekvm.c, so make it static const.Signed-off-by Fuad Tabba <[email protected]>
Quoted 60 lines
--- builtin-run.c | 2 +- include/kvm/kvm.h | 2 +- kvm.c | 10 +++++++++- 3 files changed, 11 insertions(+), 3 deletions(-)builtin-run.c +1
-1@@ -298,7 +298,7 @@ static void *kvm_cpu_thread(void *arg) panic_kvm:pr_err ("KVM exit reason: %u (\"%s\")" , current_kvm_cpu->kvm_run->exit_reason, - kvm_exit_reasons[current_kvm_cpu->kvm_run->exit_reason]); +kvm__exit_reason_str (current_kvm_cpu->kvm_run->exit_reason));if (current_kvm_cpu->kvm_run->exit_reason ==KVM_EXIT_UNKNOWN ) { pr_err("KVM exit code: %llu",include/kvm/kvm.h +1
-1@@ -255,7 +255,7 @@ int kvm__for_each_mem_bank(struct kvm *kvm, enum kvm_mem_type type,*/ void kvm__dump_mem (struct kvm *kvm,unsigned long addr,unsigned long size,int debug_fd); -extern const char *kvm_exit_reasons[]; +const char *kvm__exit_reason_str (__u32 exit_reason);static inline bool host_ptr_in_ram (struct kvm *kvm,void *p) {kvm.c +9
-1@@ -33,7 +33,7 @@#define DEFINE_KVM_EXIT_REASON (reason) [reason] = #reason -const char *kvm_exit_reasons[] = { +static const char *const kvm_exit_reasons[] = {DEFINE_KVM_EXIT_REASON (KVM_EXIT_UNKNOWN ),DEFINE_KVM_EXIT_REASON (KVM_EXIT_EXCEPTION ),DEFINE_KVM_EXIT_REASON (KVM_EXIT_IO ),@@ -57,6 +57,14 @@ const char *kvm_exit_reasons[] = {#endif }; +const char *kvm__exit_reason_str (__u32 exit_reason) +{ +if (exit_reason >=ARRAY_SIZE (kvm_exit_reasons)) +return "UNKNOWN" ; + +return kvm_exit_reasons[exit_reason]; +} +static int pause_event;static DEFINE_MUTEX (pause_lock);static struct kvm_cpu *pause_req_cpu;
Fuad Tabba
reviewerPATCH 3/5 kvm: Name every exit reason the UAPI header defines
In reply to Fuad Tabbakvm_exit_reasons[] stops at KVM_EXIT_INTERNAL_ERROR, so the panic path prints "UNKNOWN" for every reason KVM has gained since. On arm64 that covers KVM_EXIT_SYSTEM_EVENT and KVM_EXIT_ARM_NISV, and the newer KVM_EXIT_MEMORY_FAULT, KVM_EXIT_ARM_SEA and KVM_EXIT_ARM_LDST64B.
Name the rest of the reasons the synced uapi header defines, up to KVM_EXIT_SNP_REQ_CERTS. KVM_EXIT_PAPR_HCALL also loses its CONFIG_PPC64 guard, which the build never defines, so that entry was missing from every configuration. The neighbouring PPC reasons are unconditional already.
--- kvm.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-)
kvm.c +25 -2
@@ -52,9 +52,32 @@ static const char * const kvm_exit_reasons[] = {DEFINE_KVM_EXIT_REASON (KVM_EXIT_DCR ),DEFINE_KVM_EXIT_REASON (KVM_EXIT_NMI ),DEFINE_KVM_EXIT_REASON (KVM_EXIT_INTERNAL_ERROR ), -#ifdef CONFIG_PPC64 +DEFINE_KVM_EXIT_REASON (KVM_EXIT_OSI ),DEFINE_KVM_EXIT_REASON (KVM_EXIT_PAPR_HCALL ), -#endif +DEFINE_KVM_EXIT_REASON (KVM_EXIT_S390_UCONTROL ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_WATCHDOG ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_S390_TSCH ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_EPR ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_SYSTEM_EVENT ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_S390_STSI ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_IOAPIC_EOI ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_HYPERV ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_ARM_NISV ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_X86_RDMSR ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_X86_WRMSR ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_DIRTY_RING_FULL ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_AP_RESET_HOLD ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_X86_BUS_LOCK ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_XEN ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_RISCV_SBI ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_RISCV_CSR ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_NOTIFY ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_LOONGARCH_IOCSR ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_MEMORY_FAULT ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_TDX ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_ARM_SEA ), +DEFINE_KVM_EXIT_REASON (KVM_EXIT_ARM_LDST64B ), + DEFINE_KVM_EXIT_REASON(KVM_EXIT_SNP_REQ_CERTS), };const char *kvm__exit_reason_str (__u32 exit_reason)
Signature · 1 line
2.39.5
Re: [PATCH kvmtool 3/5] kvm: Name every exit reason the UAPI header defines
Quoted 5 lines · Fuad Tabba
kvm_exit_reasons[] stops at
KVM_EXIT_INTERNAL_ERROR, so the panic path prints "UNKNOWN" for every reason KVM has gained since. On arm64 that coversKVM_EXIT_SYSTEM_EVENTandKVM_EXIT_ARM_NISV, and the newerKVM_EXIT_MEMORY_FAULT,KVM_EXIT_ARM_SEAandKVM_EXIT_ARM_LDST64B.
Quoted 7 lines
Name the rest of the reasons the synced uapi header defines, up to
KVM_EXIT_SNP_REQ_CERTS.KVM_EXIT_PAPR_HCALLalso loses itsCONFIG_PPC64guard, which the build never defines, so that entry was missing from every configuration. The neighbouring PPC reasons are unconditional already.Signed-off-by Fuad Tabba <[email protected]>
Fuad Tabba
reviewerPATCH 4/5 arm64: Query steal-time support on the VM fd
In reply to Fuad Tabbakvm_cpu__setup_pvtime() probes KVM_CAP_STEAL_TIME with kvm__supports_extension(), which issues KVM_CHECK_EXTENSION on the global /dev/kvm fd. That reports the host's raw capabilities, unaware of any per-VM restrictions.
pKVM does not offer steal-time to a protected VM, and reflects that on the VM fd alone. kvmtool goes on to KVM_HAS_DEVICE_ATTR, the kernel refuses the PVTIME attribute with -EPERM, and kvm_cpu__arch_init() dies:
Fatal: Unable to initialise vcpu
Query it on the VM fd via kvm__supports_vm_extension(), as commit 84464ba0246b ("arm64: Query per-VM capabilities when selecting vCPU features") did for the vCPU feature probes. kvmtool then takes the no_pvtime path it already has when steal-time is unsupported.
--- arm64/pvtime.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arm64/pvtime.c +2 -2
@@ -58,8 +58,8 @@ int kvm_cpu__setup_pvtime(struct kvm_cpu *vcpu)if (kvm_cfg->no_pvtime)return 0 ; - has_stolen_time =kvm__supports_extension (vcpu->kvm, -KVM_CAP_STEAL_TIME ); + has_stolen_time =kvm__supports_vm_extension (vcpu->kvm, +KVM_CAP_STEAL_TIME );if (!has_stolen_time) { kvm_cfg->no_pvtime = true;return 0 ;
Signature · 1 line
2.39.5
Re: [PATCH kvmtool 4/5] arm64: Query steal-time support on the VM fd
Quoted 9 lines · Fuad Tabba
kvm_cpu__setup_pvtime()probesKVM_CAP_STEAL_TIMEwithkvm__supports_extension(), which issuesKVM_CHECK_EXTENSIONon the global/dev/kvmfd. That reports the host's raw capabilities, unaware of any per-VM restrictions.pKVM does not offer steal-time to a protected VM, and reflects that on the VM fd alone. kvmtool goes on to
KVM_HAS_DEVICE_ATTR, the kernel refuses the PVTIME attribute with-EPERM, andkvm_cpu__arch_init()dies:
Quoted 8 lines
Fatal: Unable to initialise vcpu
Query it on the VM fd via
kvm__supports_vm_extension(), as commit84464ba0246b("arm64: Query per-VM capabilities when selecting vCPU features") did for the vCPU feature probes. kvmtool then takes theno_pvtimepath it already has when steal-time is unsupported.Signed-off-by Fuad Tabba <[email protected]>
Quoted 19 lines
--- arm64/pvtime.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arm64/pvtime.c +2
-2@@ -58,8 +58,8 @@ int kvm_cpu__setup_pvtime(struct kvm_cpu *vcpu)if (kvm_cfg->no_pvtime)return 0 ; - has_stolen_time =kvm__supports_extension (vcpu->kvm, -KVM_CAP_STEAL_TIME ); + has_stolen_time =kvm__supports_vm_extension (vcpu->kvm, +KVM_CAP_STEAL_TIME );if (!has_stolen_time) { kvm_cfg->no_pvtime = true;return 0 ;
Fuad Tabba
reviewerPATCH 5/5 arm64: Query counter-offset support on the VM fd
In reply to Fuad Tabbakvm__arch_set_counter_offset() probes KVM_CAP_COUNTER_OFFSET with kvm__supports_extension(), which issues KVM_CHECK_EXTENSION on the global /dev/kvm fd. That reports the host's raw capabilities, unaware of any per-VM restrictions.
pKVM does not offer the counter offset to a protected VM, and reflects that on the VM fd alone. kvmtool's own check passes, and the KVM_ARM_SET_COUNTER_OFFSET that follows is refused by commit b12b3b04f6ba ("KVM: arm64: Check whether a VM IOCTL is allowed in pKVM"):
KVM_ARM_SET_COUNTER_OFFSET: Invalid argument
Query it on the VM fd via kvm__supports_vm_extension(), so --counter-offset on a protected VM fails with kvmtool's own message:
Fatal: No support for global counter offset
--- arm64/kvm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arm64/kvm.c +1 -1
@@ -155,7 +155,7 @@ static void kvm__arch_set_counter_offset(struct kvm *kvm)if (!kvm->cfg.arch.counter_offset)return ; -if (!kvm__supports_extension (kvm,KVM_CAP_COUNTER_OFFSET )) +if (!kvm__supports_vm_extension (kvm,KVM_CAP_COUNTER_OFFSET ))die ("No support for global counter offset" );if (ioctl (kvm->vm_fd,KVM_ARM_SET_COUNTER_OFFSET , &offset))
Signature · 1 line
2.39.5
Re: [PATCH kvmtool 5/5] arm64: Query counter-offset support on the VM fd
Quoted 11 lines · Fuad Tabba
kvm__arch_set_counter_offset()probesKVM_CAP_COUNTER_OFFSETwithkvm__supports_extension(), which issuesKVM_CHECK_EXTENSIONon the global/dev/kvmfd. That reports the host's raw capabilities, unaware of any per-VM restrictions.pKVM does not offer the counter offset to a protected VM, and reflects that on the VM fd alone. kvmtool's own check passes, and the
KVM_ARM_SET_COUNTER_OFFSETthat follows is refused by commitb12b3b04f6ba("KVM: arm64: Check whether a VM IOCTL is allowed in pKVM"):
Quoted 8 lines
KVM_ARM_SET_COUNTER_OFFSET: Invalid argumentQuery it on the VM fd via
kvm__supports_vm_extension(), so --counter-offset on a protected VM fails with kvmtool's own message:Fatal: No support for global counter offset
Signed-off-by Fuad Tabba <[email protected]>
Quoted 17 lines
--- arm64/kvm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arm64/kvm.c +1
-1@@ -155,7 +155,7 @@ static void kvm__arch_set_counter_offset(struct kvm *kvm)if (!kvm->cfg.arch.counter_offset)return ; -if (!kvm__supports_extension (kvm,KVM_CAP_COUNTER_OFFSET )) +if (!kvm__supports_vm_extension (kvm,KVM_CAP_COUNTER_OFFSET ))die ("No support for global counter offset" );if (ioctl (kvm->vm_fd,KVM_ARM_SET_COUNTER_OFFSET , &offset))
Fuad Tabba
reviewerHi Will,
Gentle ping on this series. Marc has merged the pKVM core series into kvmarm/next, including the KVM_GET_ONE_REG -EPERM that patch 1 handles, and all five patches have Suzuki's Reviewed-by.
Cheers,
/fuad
Quoted 3 lines · Fuad Tabba
Gentle ping on this series. Marc has merged the pKVM core series into kvmarm/next, including the
KVM_GET_ONE_REG-EPERMthat patch 1 handles, and all five patches have Suzuki's Reviewed-by.
I usually wait for the patches to land in Linus' tree before I update kvmtool (although I may be fishing when that happens).
Will
Fuad Tabba
reviewerQuoted 10 lines · Will Deacon
Quoted 3 lines · Fuad Tabba
Gentle ping on this series. Marc has merged the pKVM core series into kvmarm/next, including the
KVM_GET_ONE_REG-EPERMthat patch 1 handles, and all five patches have Suzuki's Reviewed-by.I usually wait for the patches to land in Linus' tree before I update kvmtool (although I may be fishing when that happens).
(although I may be fishing when that happens).
Took me a while to understand that part :D
No rush, and enjoy the fishing.
/fuad
Quoted 3 lines
Will