RISC-V: KVM: add KVM_CAP_RISCV_USERSPACE_SBI
Subsystems (4)
PATCH v4 1/1 RISC-V: KVM: add KVM_CAP_RISCV_USERSPACE_SBI
The new capability allows userspace to implement SBI extensions that KVM does not handle. This allows userspace to implement any SBI ecall as userspace already has the ability to disable acceleration of selected SBI extensions.
The base extension is made controllable as well, but only with the new capability, because it was previously handled specially for some reason. *** The related compatibility TODO in the code needs addressing. ***
This is a VM capability, because userspace will most likely want to have the same behavior for all VCPUs. We can easily make it both a VCPU and a VM capability if there is demand in the future.
---
v4:
* forward base extension as well
* change the id to 242, because 241 is already taken in linux-next
* QEMU example: https://github.com/radimkrcmar/qemu/tree/mp_state_reset
v3: new
--- Documentation/virt/kvm/api.rst | 11 +++++++++++ arch/riscv/include/asm/kvm_host.h | 3 +++ arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu_sbi.c | 17 ++++++++++++++--- arch/riscv/kvm/vm.c | 5 +++++ include/uapi/linux/kvm.h | 1 + 6 files changed, 35 insertions(+), 3 deletions(-)
Documentation/virt/kvm/api.rst +11 -0
@@ -8507,6 +8507,17 @@ given VM. When this capability is enabled, KVM resets the VCPU when setting MP_STATE_INIT_RECEIVED through IOCTL. The original MP_STATE is preserved. +7.44 KVM_CAP_RISCV_USERSPACE_SBI +-------------------------------- + +:Architectures: riscv +:Type: VM +:Parameters: None +:Returns: 0 on success, -EINVAL if arg[0] is not zero + +When this capability is enabled, KVM forwards ecalls from disabled or unknown +SBI extensions to userspace. + 8. Other capabilities. ======================
arch/riscv/include/asm/kvm_host.h +3 -0
@@ -122,6 +122,9 @@ struct kvm_arch {/* KVM_CAP_RISCV_MP_STATE_RESET */ bool mp_state_reset; + +/* KVM_CAP_RISCV_USERSPACE_SBI */ +bool userspace_sbi; };struct kvm_cpu_trap {
arch/riscv/include/uapi/asm/kvm.h +1 -0
@@ -204,6 +204,7 @@ enum KVM_RISCV_SBI_EXT_ID {KVM_RISCV_SBI_EXT_DBCN ,KVM_RISCV_SBI_EXT_STA ,KVM_RISCV_SBI_EXT_SUSP , +KVM_RISCV_SBI_EXT_BASE , KVM_RISCV_SBI_EXT_MAX, };
arch/riscv/kvm/vcpu_sbi.c +14 -3
@@ -39,7 +39,7 @@ static const struct kvm_riscv_sbi_extension_entry sbi_ext[] = { .ext_ptr = &vcpu_sbi_ext_v01, }, { - .ext_idx = KVM_RISCV_SBI_EXT_MAX, /* Can't be disabled */ + .ext_idx = KVM_RISCV_SBI_EXT_BASE, .ext_ptr = &vcpu_sbi_ext_base, }, {
@@ -217,6 +217,11 @@ static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu,if (!sext || scontext->ext_status[sext->ext_idx] ==KVM_RISCV_SBI_EXT_STATUS_UNAVAILABLE )return -ENOENT ; +// TODO: probably remove, the extension originally couldn't be +// disabled, but it doesn't seem necessary +if (!vcpu->kvm->arch.userspace_sbi && sext->ext_id ==KVM_RISCV_SBI_EXT_BASE ) +return -ENOENT ; + scontext->ext_status[sext->ext_idx] = (reg_val) ?KVM_RISCV_SBI_EXT_STATUS_ENABLED :KVM_RISCV_SBI_EXT_STATUS_DISABLED ;
@@ -471,8 +476,14 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)#endif ret = sbi_ext->handler (vcpu, run, &sbi_ret); }else { -/* Return error for unsupported SBI calls */ - cp->a0 =SBI_ERR_NOT_SUPPORTED ; +if (vcpu->kvm->arch.userspace_sbi) { + next_sepc = false; + ret =0 ; +kvm_riscv_vcpu_sbi_forward (vcpu, run); + }else { +/* Return error for unsupported SBI calls */ + cp->a0 =SBI_ERR_NOT_SUPPORTED ; + } goto ecall_done; }
arch/riscv/kvm/vm.c +5 -0
@@ -217,6 +217,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)return -EINVAL ; kvm->arch.mp_state_reset = true;return 0 ; +case KVM_CAP_RISCV_USERSPACE_SBI : +if (cap->flags) +return -EINVAL ; + kvm->arch.userspace_sbi = true; +return 0 ;default :return -EINVAL ; }
include/uapi/linux/kvm.h +1 -0
@@ -931,6 +931,7 @@ struct kvm_enable_cap {#define KVM_CAP_X86_GUEST_MODE 238#define KVM_CAP_ARM_WRITABLE_IMP_ID_REGS 239#define KVM_CAP_RISCV_MP_STATE_RESET 240 +#define KVM_CAP_RISCV_USERSPACE_SBI 242struct kvm_irq_routing_irqchip {__u32 irqchip;
Signature · 1 line
2.49.0
Quoted 90 lines · Radim Krčmář
The new capability allows userspace to implement SBI extensions that KVM does not handle. This allows userspace to implement any SBI ecall as userspace already has the ability to disable acceleration of selected SBI extensions.
The base extension is made controllable as well, but only with the new capability, because it was previously handled specially for some reason. *** The related compatibility TODO in the code needs addressing. ***This is a VM capability, because userspace will most likely want to have the same behavior for all VCPUs. We can easily make it both a VCPU and a VM capability if there is demand in the future.
Signed-off-by Radim Krčmář <[email protected]>---
v4:
* forward base extension as well
* change the id to 242, because 241 is already taken in linux-next
* QEMU example: https://github.com/radimkrcmar/qemu/tree/mp_state_reset
v3: new--- Documentation/virt/kvm/api.rst | 11 +++++++++++ arch/riscv/include/asm/kvm_host.h | 3 +++ arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu_sbi.c | 17 ++++++++++++++--- arch/riscv/kvm/vm.c | 5 +++++ include/uapi/linux/kvm.h | 1 + 6 files changed, 35 insertions(+), 3 deletions(-)Documentation/virt/kvm/api.rst +11
-0@@ -8507,6 +8507,17 @@ given VM. When this capability is enabled, KVM resets the VCPU when setting MP_STATE_INIT_RECEIVED through IOCTL. The original MP_STATE is preserved. +7.44 KVM_CAP_RISCV_USERSPACE_SBI +-------------------------------- + +:Architectures: riscv +:Type: VM +:Parameters: None +:Returns: 0 on success, -EINVAL if arg[0] is not zero + +When this capability is enabled, KVM forwards ecalls from disabled or unknown +SBI extensions to userspace. + 8. Other capabilities. ======================arch/riscv/include/asm/kvm_host.h +3
-0@@ -122,6 +122,9 @@ struct kvm_arch {/* KVM_CAP_RISCV_MP_STATE_RESET */ bool mp_state_reset; + +/* KVM_CAP_RISCV_USERSPACE_SBI */ +bool userspace_sbi; };struct kvm_cpu_trap {arch/riscv/include/uapi/asm/kvm.h +1
-0@@ -204,6 +204,7 @@ enum KVM_RISCV_SBI_EXT_ID {KVM_RISCV_SBI_EXT_DBCN ,KVM_RISCV_SBI_EXT_STA ,KVM_RISCV_SBI_EXT_SUSP , +KVM_RISCV_SBI_EXT_BASE , KVM_RISCV_SBI_EXT_MAX, };arch/riscv/kvm/vcpu_sbi.c +1
-1@@ -39,7 +39,7 @@ static const struct kvm_riscv_sbi_extension_entry sbi_ext[] = { .ext_ptr = &vcpu_sbi_ext_v01, }, { - .ext_idx = KVM_RISCV_SBI_EXT_MAX, /* Can't be disabled */ + .ext_idx = KVM_RISCV_SBI_EXT_BASE, .ext_ptr = &vcpu_sbi_ext_base, }, {
arch/riscv/kvm/vcpu_sbi.c static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu, · line 224
Quoted 8 lines
arch/riscv/kvm/vcpu_sbi.c +5
-0@@ -217,6 +217,11 @@ static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu,if (!sext || scontext->ext_status[sext->ext_idx] ==KVM_RISCV_SBI_EXT_STATUS_UNAVAILABLE )return -ENOENT ; +// TODO: probably remove, the extension originally couldn't be +// disabled, but it doesn't seem necessary +if (!vcpu->kvm->arch.userspace_sbi && sext->ext_id ==KVM_RISCV_SBI_EXT_BASE ) +return -ENOENT ; +
I agree that we don't need to babysit userspace and it's even conceivable to have guests that don't need SBI. KVM should only need checks in its UAPI to protect itself from userspace and to enforce proper use of the API. It's not KVM's place to ensure userspace doesn't violate the SBI spec or create broken guests (userspace is the boss, even if it's a boss that doesn't make sense)
So, I vote we drop the check.
Quoted 51 lines
arch/riscv/kvm/vcpu_sbi.c +8
-2scontext->ext_status[sext->ext_idx] = (reg_val) ?KVM_RISCV_SBI_EXT_STATUS_ENABLED :KVM_RISCV_SBI_EXT_STATUS_DISABLED ;@@ -471,8 +476,14 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)#endif ret = sbi_ext->handler (vcpu, run, &sbi_ret); }else { -/* Return error for unsupported SBI calls */ - cp->a0 =SBI_ERR_NOT_SUPPORTED ; +if (vcpu->kvm->arch.userspace_sbi) { + next_sepc = false; + ret =0 ; +kvm_riscv_vcpu_sbi_forward (vcpu, run); + }else { +/* Return error for unsupported SBI calls */ + cp->a0 =SBI_ERR_NOT_SUPPORTED ; + } goto ecall_done; }arch/riscv/kvm/vm.c +5
-0@@ -217,6 +217,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)return -EINVAL ; kvm->arch.mp_state_reset = true;return 0 ; +case KVM_CAP_RISCV_USERSPACE_SBI : +if (cap->flags) +return -EINVAL ; + kvm->arch.userspace_sbi = true; +return 0 ;default :return -EINVAL ; }include/uapi/linux/kvm.h +1
-0@@ -931,6 +931,7 @@ struct kvm_enable_cap {#define KVM_CAP_X86_GUEST_MODE 238#define KVM_CAP_ARM_WRITABLE_IMP_ID_REGS 239#define KVM_CAP_RISCV_MP_STATE_RESET 240 +#define KVM_CAP_RISCV_USERSPACE_SBI 242struct kvm_irq_routing_irqchip {__u32 irqchip;Signature · 1 line
2.49.0
Otherwise,
Anup Patel
maintainerQuoted 164 lines · Andrew Jones
Quoted 90 lines · Radim Krčmář
The new capability allows userspace to implement SBI extensions that KVM does not handle. This allows userspace to implement any SBI ecall as userspace already has the ability to disable acceleration of selected SBI extensions.
The base extension is made controllable as well, but only with the new capability, because it was previously handled specially for some reason. *** The related compatibility TODO in the code needs addressing. ***This is a VM capability, because userspace will most likely want to have the same behavior for all VCPUs. We can easily make it both a VCPU and a VM capability if there is demand in the future.
Signed-off-by Radim Krčmář <[email protected]>---
v4:
* forward base extension as well
* change the id to 242, because 241 is already taken in linux-next
* QEMU example: https://github.com/radimkrcmar/qemu/tree/mp_state_reset
v3: new--- Documentation/virt/kvm/api.rst | 11 +++++++++++ arch/riscv/include/asm/kvm_host.h | 3 +++ arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu_sbi.c | 17 ++++++++++++++--- arch/riscv/kvm/vm.c | 5 +++++ include/uapi/linux/kvm.h | 1 + 6 files changed, 35 insertions(+), 3 deletions(-)Documentation/virt/kvm/api.rst +11
-0@@ -8507,6 +8507,17 @@ given VM. When this capability is enabled, KVM resets the VCPU when setting MP_STATE_INIT_RECEIVED through IOCTL. The original MP_STATE is preserved. +7.44 KVM_CAP_RISCV_USERSPACE_SBI +-------------------------------- + +:Architectures: riscv +:Type: VM +:Parameters: None +:Returns: 0 on success, -EINVAL if arg[0] is not zero + +When this capability is enabled, KVM forwards ecalls from disabled or unknown +SBI extensions to userspace. + 8. Other capabilities. ======================arch/riscv/include/asm/kvm_host.h +3
-0@@ -122,6 +122,9 @@ struct kvm_arch {/* KVM_CAP_RISCV_MP_STATE_RESET */ bool mp_state_reset; + +/* KVM_CAP_RISCV_USERSPACE_SBI */ +bool userspace_sbi; };struct kvm_cpu_trap {arch/riscv/include/uapi/asm/kvm.h +1
-0@@ -204,6 +204,7 @@ enum KVM_RISCV_SBI_EXT_ID {KVM_RISCV_SBI_EXT_DBCN ,KVM_RISCV_SBI_EXT_STA ,KVM_RISCV_SBI_EXT_SUSP , +KVM_RISCV_SBI_EXT_BASE , KVM_RISCV_SBI_EXT_MAX, };arch/riscv/kvm/vcpu_sbi.c +1
-1@@ -39,7 +39,7 @@ static const struct kvm_riscv_sbi_extension_entry sbi_ext[] = { .ext_ptr = &vcpu_sbi_ext_v01, }, { - .ext_idx = KVM_RISCV_SBI_EXT_MAX, /* Can't be disabled */ + .ext_idx = KVM_RISCV_SBI_EXT_BASE, .ext_ptr = &vcpu_sbi_ext_base, }, {arch/riscv/kvm/vcpu_sbi.c static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu, · line 224
Quoted 8 lines
arch/riscv/kvm/vcpu_sbi.c +5
-0@@ -217,6 +217,11 @@ static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu,if (!sext || scontext->ext_status[sext->ext_idx] ==KVM_RISCV_SBI_EXT_STATUS_UNAVAILABLE )return -ENOENT ; +// TODO: probably remove, the extension originally couldn't be +// disabled, but it doesn't seem necessary +if (!vcpu->kvm->arch.userspace_sbi && sext->ext_id ==KVM_RISCV_SBI_EXT_BASE ) +return -ENOENT ; +I agree that we don't need to babysit userspace and it's even conceivable to have guests that don't need SBI. KVM should only need checks in its UAPI to protect itself from userspace and to enforce proper use of the API. It's not KVM's place to ensure userspace doesn't violate the SBI spec or create broken guests (userspace is the boss, even if it's a boss that doesn't make sense)
So, I vote we drop the check.
Quoted 49 lines
arch/riscv/kvm/vcpu_sbi.c +8
-2scontext->ext_status[sext->ext_idx] = (reg_val) ?KVM_RISCV_SBI_EXT_STATUS_ENABLED :KVM_RISCV_SBI_EXT_STATUS_DISABLED ;@@ -471,8 +476,14 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)#endif ret = sbi_ext->handler (vcpu, run, &sbi_ret); }else { -/* Return error for unsupported SBI calls */ - cp->a0 =SBI_ERR_NOT_SUPPORTED ; +if (vcpu->kvm->arch.userspace_sbi) { + next_sepc = false; + ret =0 ; +kvm_riscv_vcpu_sbi_forward (vcpu, run); + }else { +/* Return error for unsupported SBI calls */ + cp->a0 =SBI_ERR_NOT_SUPPORTED ; + } goto ecall_done; }arch/riscv/kvm/vm.c +5
-0@@ -217,6 +217,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)return -EINVAL ; kvm->arch.mp_state_reset = true;return 0 ; +case KVM_CAP_RISCV_USERSPACE_SBI : +if (cap->flags) +return -EINVAL ; + kvm->arch.userspace_sbi = true; +return 0 ;default :return -EINVAL ; }include/uapi/linux/kvm.h +1
-1@@ -931,6 +931,7 @@ struct kvm_enable_cap {#define KVM_CAP_X86_GUEST_MODE 238#define KVM_CAP_ARM_WRITABLE_IMP_ID_REGS 239#define KVM_CAP_RISCV_MP_STATE_RESET 240 +#define KVM_CAP_RISCV_USERSPACE_SBI 242struct kvm_irq_routing_irqchip {__u32 irqchip; --Quoted 2 lines
2.49.0
Otherwise,
Quoted 1 line
Reviewed-by Andrew Jones <[email protected]>
We are not going ahead with this approach for the reasons mentioned in v3 series [1].
Regards,
Anup
[1] https://patchwork.ozlabs.org/project/kvm-riscv/cover/[email protected]/
Quoted 165 lines · Anup Patel
Quoted 164 lines · Andrew Jones
Quoted 90 lines · Radim Krčmář
The new capability allows userspace to implement SBI extensions that KVM does not handle. This allows userspace to implement any SBI ecall as userspace already has the ability to disable acceleration of selected SBI extensions.
The base extension is made controllable as well, but only with the new capability, because it was previously handled specially for some reason. *** The related compatibility TODO in the code needs addressing. ***This is a VM capability, because userspace will most likely want to have the same behavior for all VCPUs. We can easily make it both a VCPU and a VM capability if there is demand in the future.
Signed-off-by Radim Krčmář <[email protected]>---
v4:
* forward base extension as well
* change the id to 242, because 241 is already taken in linux-next
* QEMU example: https://github.com/radimkrcmar/qemu/tree/mp_state_reset
v3: new--- Documentation/virt/kvm/api.rst | 11 +++++++++++ arch/riscv/include/asm/kvm_host.h | 3 +++ arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu_sbi.c | 17 ++++++++++++++--- arch/riscv/kvm/vm.c | 5 +++++ include/uapi/linux/kvm.h | 1 + 6 files changed, 35 insertions(+), 3 deletions(-)Documentation/virt/kvm/api.rst +11
-0@@ -8507,6 +8507,17 @@ given VM. When this capability is enabled, KVM resets the VCPU when setting MP_STATE_INIT_RECEIVED through IOCTL. The original MP_STATE is preserved. +7.44 KVM_CAP_RISCV_USERSPACE_SBI +-------------------------------- + +:Architectures: riscv +:Type: VM +:Parameters: None +:Returns: 0 on success, -EINVAL if arg[0] is not zero + +When this capability is enabled, KVM forwards ecalls from disabled or unknown +SBI extensions to userspace. + 8. Other capabilities. ======================arch/riscv/include/asm/kvm_host.h +3
-0@@ -122,6 +122,9 @@ struct kvm_arch {/* KVM_CAP_RISCV_MP_STATE_RESET */ bool mp_state_reset; + +/* KVM_CAP_RISCV_USERSPACE_SBI */ +bool userspace_sbi; };struct kvm_cpu_trap {arch/riscv/include/uapi/asm/kvm.h +1
-0@@ -204,6 +204,7 @@ enum KVM_RISCV_SBI_EXT_ID {KVM_RISCV_SBI_EXT_DBCN ,KVM_RISCV_SBI_EXT_STA ,KVM_RISCV_SBI_EXT_SUSP , +KVM_RISCV_SBI_EXT_BASE , KVM_RISCV_SBI_EXT_MAX, };arch/riscv/kvm/vcpu_sbi.c +1
-1@@ -39,7 +39,7 @@ static const struct kvm_riscv_sbi_extension_entry sbi_ext[] = { .ext_ptr = &vcpu_sbi_ext_v01, }, { - .ext_idx = KVM_RISCV_SBI_EXT_MAX, /* Can't be disabled */ + .ext_idx = KVM_RISCV_SBI_EXT_BASE, .ext_ptr = &vcpu_sbi_ext_base, }, {arch/riscv/kvm/vcpu_sbi.c static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu, · line 224
Quoted 8 lines
arch/riscv/kvm/vcpu_sbi.c +5
-0@@ -217,6 +217,11 @@ static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu,if (!sext || scontext->ext_status[sext->ext_idx] ==KVM_RISCV_SBI_EXT_STATUS_UNAVAILABLE )return -ENOENT ; +// TODO: probably remove, the extension originally couldn't be +// disabled, but it doesn't seem necessary +if (!vcpu->kvm->arch.userspace_sbi && sext->ext_id ==KVM_RISCV_SBI_EXT_BASE ) +return -ENOENT ; +I agree that we don't need to babysit userspace and it's even conceivable to have guests that don't need SBI. KVM should only need checks in its UAPI to protect itself from userspace and to enforce proper use of the API. It's not KVM's place to ensure userspace doesn't violate the SBI spec or create broken guests (userspace is the boss, even if it's a boss that doesn't make sense)
So, I vote we drop the check.
Quoted 49 lines
arch/riscv/kvm/vcpu_sbi.c +8
-2scontext->ext_status[sext->ext_idx] = (reg_val) ?KVM_RISCV_SBI_EXT_STATUS_ENABLED :KVM_RISCV_SBI_EXT_STATUS_DISABLED ;@@ -471,8 +476,14 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)#endif ret = sbi_ext->handler (vcpu, run, &sbi_ret); }else { -/* Return error for unsupported SBI calls */ - cp->a0 =SBI_ERR_NOT_SUPPORTED ; +if (vcpu->kvm->arch.userspace_sbi) { + next_sepc = false; + ret =0 ; +kvm_riscv_vcpu_sbi_forward (vcpu, run); + }else { +/* Return error for unsupported SBI calls */ + cp->a0 =SBI_ERR_NOT_SUPPORTED ; + } goto ecall_done; }arch/riscv/kvm/vm.c +5
-0@@ -217,6 +217,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)return -EINVAL ; kvm->arch.mp_state_reset = true;return 0 ; +case KVM_CAP_RISCV_USERSPACE_SBI : +if (cap->flags) +return -EINVAL ; + kvm->arch.userspace_sbi = true; +return 0 ;default :return -EINVAL ; }include/uapi/linux/kvm.h +1
-1@@ -931,6 +931,7 @@ struct kvm_enable_cap {#define KVM_CAP_X86_GUEST_MODE 238#define KVM_CAP_ARM_WRITABLE_IMP_ID_REGS 239#define KVM_CAP_RISCV_MP_STATE_RESET 240 +#define KVM_CAP_RISCV_USERSPACE_SBI 242struct kvm_irq_routing_irqchip {__u32 irqchip; --Quoted 2 lines
2.49.0
Otherwise,
Quoted 4 lines
Quoted 1 line
Reviewed-by Andrew Jones <[email protected]>We are not going ahead with this approach for the reasons mentioned in v3 series [1].
IIUC, the main concern in that thread is that userspace won't know what to do with some of the exits it gets or that it'll try to take control of extensions that it can't emulate. I feel like not exiting to userspace in those cases is trying to second guess it, i.e. KVM is trying to enforce a policy on userspace. But, KVM shouldn't be doing that, as userspace should be the policy maker. If userspace uses this capability to opt into getting all the SBI exits (which it doesn't want KVM to handle), then it should be allowed to get them -- and, if userspace doesn't know what it's doing, then it can keep all the pieces.
Thanks,
drew
Quoted 5 lines
Regards,
Anup[1] https://patchwork.ozlabs.org/project/kvm-riscv/cover/[email protected]/
Anup Patel
maintainerQuoted 172 lines · Andrew Jones
Quoted 165 lines · Anup Patel
Quoted 164 lines · Andrew Jones
Quoted 90 lines · Radim Krčmář
The new capability allows userspace to implement SBI extensions that KVM does not handle. This allows userspace to implement any SBI ecall as userspace already has the ability to disable acceleration of selected SBI extensions.
The base extension is made controllable as well, but only with the new capability, because it was previously handled specially for some reason. *** The related compatibility TODO in the code needs addressing. ***This is a VM capability, because userspace will most likely want to have the same behavior for all VCPUs. We can easily make it both a VCPU and a VM capability if there is demand in the future.
Signed-off-by Radim Krčmář <[email protected]>---
v4:
* forward base extension as well
* change the id to 242, because 241 is already taken in linux-next
* QEMU example: https://github.com/radimkrcmar/qemu/tree/mp_state_reset
v3: new--- Documentation/virt/kvm/api.rst | 11 +++++++++++ arch/riscv/include/asm/kvm_host.h | 3 +++ arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu_sbi.c | 17 ++++++++++++++--- arch/riscv/kvm/vm.c | 5 +++++ include/uapi/linux/kvm.h | 1 + 6 files changed, 35 insertions(+), 3 deletions(-)Documentation/virt/kvm/api.rst +11
-0@@ -8507,6 +8507,17 @@ given VM. When this capability is enabled, KVM resets the VCPU when setting MP_STATE_INIT_RECEIVED through IOCTL. The original MP_STATE is preserved. +7.44 KVM_CAP_RISCV_USERSPACE_SBI +-------------------------------- + +:Architectures: riscv +:Type: VM +:Parameters: None +:Returns: 0 on success, -EINVAL if arg[0] is not zero + +When this capability is enabled, KVM forwards ecalls from disabled or unknown +SBI extensions to userspace. + 8. Other capabilities. ======================arch/riscv/include/asm/kvm_host.h +3
-0@@ -122,6 +122,9 @@ struct kvm_arch {/* KVM_CAP_RISCV_MP_STATE_RESET */ bool mp_state_reset; + +/* KVM_CAP_RISCV_USERSPACE_SBI */ +bool userspace_sbi; };struct kvm_cpu_trap {arch/riscv/include/uapi/asm/kvm.h +1
-0@@ -204,6 +204,7 @@ enum KVM_RISCV_SBI_EXT_ID {KVM_RISCV_SBI_EXT_DBCN ,KVM_RISCV_SBI_EXT_STA ,KVM_RISCV_SBI_EXT_SUSP , +KVM_RISCV_SBI_EXT_BASE , KVM_RISCV_SBI_EXT_MAX, };arch/riscv/kvm/vcpu_sbi.c +1
-1@@ -39,7 +39,7 @@ static const struct kvm_riscv_sbi_extension_entry sbi_ext[] = { .ext_ptr = &vcpu_sbi_ext_v01, }, { - .ext_idx = KVM_RISCV_SBI_EXT_MAX, /* Can't be disabled */ + .ext_idx = KVM_RISCV_SBI_EXT_BASE, .ext_ptr = &vcpu_sbi_ext_base, }, {arch/riscv/kvm/vcpu_sbi.c static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu, · line 224
Quoted 8 lines
arch/riscv/kvm/vcpu_sbi.c +5
-0@@ -217,6 +217,11 @@ static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu,if (!sext || scontext->ext_status[sext->ext_idx] ==KVM_RISCV_SBI_EXT_STATUS_UNAVAILABLE )return -ENOENT ; +// TODO: probably remove, the extension originally couldn't be +// disabled, but it doesn't seem necessary +if (!vcpu->kvm->arch.userspace_sbi && sext->ext_id ==KVM_RISCV_SBI_EXT_BASE ) +return -ENOENT ; +I agree that we don't need to babysit userspace and it's even conceivable to have guests that don't need SBI. KVM should only need checks in its UAPI to protect itself from userspace and to enforce proper use of the API. It's not KVM's place to ensure userspace doesn't violate the SBI spec or create broken guests (userspace is the boss, even if it's a boss that doesn't make sense)
So, I vote we drop the check.
Quoted 49 lines
arch/riscv/kvm/vcpu_sbi.c +8
-2scontext->ext_status[sext->ext_idx] = (reg_val) ?KVM_RISCV_SBI_EXT_STATUS_ENABLED :KVM_RISCV_SBI_EXT_STATUS_DISABLED ;@@ -471,8 +476,14 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)#endif ret = sbi_ext->handler (vcpu, run, &sbi_ret); }else { -/* Return error for unsupported SBI calls */ - cp->a0 =SBI_ERR_NOT_SUPPORTED ; +if (vcpu->kvm->arch.userspace_sbi) { + next_sepc = false; + ret =0 ; +kvm_riscv_vcpu_sbi_forward (vcpu, run); + }else { +/* Return error for unsupported SBI calls */ + cp->a0 =SBI_ERR_NOT_SUPPORTED ; + } goto ecall_done; }arch/riscv/kvm/vm.c +5
-0@@ -217,6 +217,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)return -EINVAL ; kvm->arch.mp_state_reset = true;return 0 ; +case KVM_CAP_RISCV_USERSPACE_SBI : +if (cap->flags) +return -EINVAL ; + kvm->arch.userspace_sbi = true; +return 0 ;default :return -EINVAL ; }include/uapi/linux/kvm.h +1
-1@@ -931,6 +931,7 @@ struct kvm_enable_cap {#define KVM_CAP_X86_GUEST_MODE 238#define KVM_CAP_ARM_WRITABLE_IMP_ID_REGS 239#define KVM_CAP_RISCV_MP_STATE_RESET 240 +#define KVM_CAP_RISCV_USERSPACE_SBI 242struct kvm_irq_routing_irqchip {__u32 irqchip; --Quoted 2 lines
2.49.0
Otherwise,
Quoted 4 lines
Quoted 1 line
Reviewed-by Andrew Jones <[email protected]>We are not going ahead with this approach for the reasons mentioned in v3 series [1].
Quoted 9 lines
IIUC, the main concern in that thread is that userspace won't know what to do with some of the exits it gets or that it'll try to take control of extensions that it can't emulate. I feel like not exiting to userspace in those cases is trying to second guess it, i.e. KVM is trying to enforce a policy on userspace. But, KVM shouldn't be doing that, as userspace should be the policy maker. If userspace uses this capability to opt into getting all the SBI exits (which it doesn't want KVM to handle), then it should be allowed to get them -- and, if userspace doesn't know what it's doing, then it can keep all the pieces.
The userspace already has a mechanism to opt-in for select SBI exits which it can implement such as SBI DBCN and SBI SUSP. With SBI v3.0, userspace will be able implement SBI MPXY but SBI SSE, FWFT, and DBTR will be in kernel space due to reasons mentioned in the v3 series.
There is no point in forwarding all SBI exits to userspace when userspace has no mechanism to implement many critical SBI extensions.
Regards,
Anup
Quoted 183 lines · Anup Patel
Quoted 172 lines · Andrew Jones
Quoted 165 lines · Anup Patel
Quoted 164 lines · Andrew Jones
Quoted 90 lines · Radim Krčmář
The new capability allows userspace to implement SBI extensions that KVM does not handle. This allows userspace to implement any SBI ecall as userspace already has the ability to disable acceleration of selected SBI extensions.
The base extension is made controllable as well, but only with the new capability, because it was previously handled specially for some reason. *** The related compatibility TODO in the code needs addressing. ***This is a VM capability, because userspace will most likely want to have the same behavior for all VCPUs. We can easily make it both a VCPU and a VM capability if there is demand in the future.
Signed-off-by Radim Krčmář <[email protected]>---
v4:
* forward base extension as well
* change the id to 242, because 241 is already taken in linux-next
* QEMU example: https://github.com/radimkrcmar/qemu/tree/mp_state_reset
v3: new--- Documentation/virt/kvm/api.rst | 11 +++++++++++ arch/riscv/include/asm/kvm_host.h | 3 +++ arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/vcpu_sbi.c | 17 ++++++++++++++--- arch/riscv/kvm/vm.c | 5 +++++ include/uapi/linux/kvm.h | 1 + 6 files changed, 35 insertions(+), 3 deletions(-)Documentation/virt/kvm/api.rst +11
-0@@ -8507,6 +8507,17 @@ given VM. When this capability is enabled, KVM resets the VCPU when setting MP_STATE_INIT_RECEIVED through IOCTL. The original MP_STATE is preserved. +7.44 KVM_CAP_RISCV_USERSPACE_SBI +-------------------------------- + +:Architectures: riscv +:Type: VM +:Parameters: None +:Returns: 0 on success, -EINVAL if arg[0] is not zero + +When this capability is enabled, KVM forwards ecalls from disabled or unknown +SBI extensions to userspace. + 8. Other capabilities. ======================arch/riscv/include/asm/kvm_host.h +3
-0@@ -122,6 +122,9 @@ struct kvm_arch {/* KVM_CAP_RISCV_MP_STATE_RESET */ bool mp_state_reset; + +/* KVM_CAP_RISCV_USERSPACE_SBI */ +bool userspace_sbi; };struct kvm_cpu_trap {arch/riscv/include/uapi/asm/kvm.h +1
-0@@ -204,6 +204,7 @@ enum KVM_RISCV_SBI_EXT_ID {KVM_RISCV_SBI_EXT_DBCN ,KVM_RISCV_SBI_EXT_STA ,KVM_RISCV_SBI_EXT_SUSP , +KVM_RISCV_SBI_EXT_BASE , KVM_RISCV_SBI_EXT_MAX, };arch/riscv/kvm/vcpu_sbi.c +1
-1@@ -39,7 +39,7 @@ static const struct kvm_riscv_sbi_extension_entry sbi_ext[] = { .ext_ptr = &vcpu_sbi_ext_v01, }, { - .ext_idx = KVM_RISCV_SBI_EXT_MAX, /* Can't be disabled */ + .ext_idx = KVM_RISCV_SBI_EXT_BASE, .ext_ptr = &vcpu_sbi_ext_base, }, {arch/riscv/kvm/vcpu_sbi.c static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu, · line 224
Quoted 8 lines
arch/riscv/kvm/vcpu_sbi.c +5
-0@@ -217,6 +217,11 @@ static int riscv_vcpu_set_sbi_ext_single(struct kvm_vcpu *vcpu,if (!sext || scontext->ext_status[sext->ext_idx] ==KVM_RISCV_SBI_EXT_STATUS_UNAVAILABLE )return -ENOENT ; +// TODO: probably remove, the extension originally couldn't be +// disabled, but it doesn't seem necessary +if (!vcpu->kvm->arch.userspace_sbi && sext->ext_id ==KVM_RISCV_SBI_EXT_BASE ) +return -ENOENT ; +I agree that we don't need to babysit userspace and it's even conceivable to have guests that don't need SBI. KVM should only need checks in its UAPI to protect itself from userspace and to enforce proper use of the API. It's not KVM's place to ensure userspace doesn't violate the SBI spec or create broken guests (userspace is the boss, even if it's a boss that doesn't make sense)
So, I vote we drop the check.
Quoted 49 lines
arch/riscv/kvm/vcpu_sbi.c +8
-2scontext->ext_status[sext->ext_idx] = (reg_val) ?KVM_RISCV_SBI_EXT_STATUS_ENABLED :KVM_RISCV_SBI_EXT_STATUS_DISABLED ;@@ -471,8 +476,14 @@ int kvm_riscv_vcpu_sbi_ecall(struct kvm_vcpu *vcpu, struct kvm_run *run)#endif ret = sbi_ext->handler (vcpu, run, &sbi_ret); }else { -/* Return error for unsupported SBI calls */ - cp->a0 =SBI_ERR_NOT_SUPPORTED ; +if (vcpu->kvm->arch.userspace_sbi) { + next_sepc = false; + ret =0 ; +kvm_riscv_vcpu_sbi_forward (vcpu, run); + }else { +/* Return error for unsupported SBI calls */ + cp->a0 =SBI_ERR_NOT_SUPPORTED ; + } goto ecall_done; }arch/riscv/kvm/vm.c +5
-0@@ -217,6 +217,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)return -EINVAL ; kvm->arch.mp_state_reset = true;return 0 ; +case KVM_CAP_RISCV_USERSPACE_SBI : +if (cap->flags) +return -EINVAL ; + kvm->arch.userspace_sbi = true; +return 0 ;default :return -EINVAL ; }include/uapi/linux/kvm.h +1
-1@@ -931,6 +931,7 @@ struct kvm_enable_cap {#define KVM_CAP_X86_GUEST_MODE 238#define KVM_CAP_ARM_WRITABLE_IMP_ID_REGS 239#define KVM_CAP_RISCV_MP_STATE_RESET 240 +#define KVM_CAP_RISCV_USERSPACE_SBI 242struct kvm_irq_routing_irqchip {__u32 irqchip; --Quoted 2 lines
2.49.0
Otherwise,
Quoted 4 lines
Quoted 1 line
Reviewed-by Andrew Jones <[email protected]>We are not going ahead with this approach for the reasons mentioned in v3 series [1].
Quoted 9 lines
IIUC, the main concern in that thread is that userspace won't know what to do with some of the exits it gets or that it'll try to take control of extensions that it can't emulate. I feel like not exiting to userspace in those cases is trying to second guess it, i.e. KVM is trying to enforce a policy on userspace. But, KVM shouldn't be doing that, as userspace should be the policy maker. If userspace uses this capability to opt into getting all the SBI exits (which it doesn't want KVM to handle), then it should be allowed to get them -- and, if userspace doesn't know what it's doing, then it can keep all the pieces.
Quoted 7 lines
The userspace already has a mechanism to opt-in for select SBI exits which it can implement such as SBI DBCN and SBI SUSP. With SBI v3.0, userspace will be able implement SBI MPXY but SBI SSE, FWFT, and DBTR will be in kernel space due to reasons mentioned in the v3 series.
There is no point in forwarding all SBI exits to userspace when userspace has no mechanism to implement many critical SBI extensions.
Userspace can implement all truly disabled extensions, it just returns not-supported. So, while I may be contriving this example a bit, enabling userspace to log attempts to use disabled extensions would be one reason to forward them.
Thanks,
drew