KVM: PPC: Handle CPU compatibility mode for nested guests
Subsystems (4)
v3 cover KVM: PPC: Handle CPU compatibility mode for nested guests
On POWER systems, newer processor generations can operate in compatibility modes corresponding to earlier generations (e.g., a Power11 system running in Power10 compatibility mode). In such cases, the effective CPU level exposed to guests differs from the physical processor generation.
This creates a problem for nested virtualization. When booting a nested KVM guest (L2) inside a host KVM guest (L1) running in a compatibility mode, userspace (e.g., QEMU) may derive the CPU model from the raw hardware PVR and attempt to configure the nested guest accordingly. However, the L1 partition is constrained by the compatibility level negotiated with the hypervisor (L0), and requests exceeding that level are rejected, leading to guest boot failures such as:
KVM-NESTEDv2: couldn't set guest wide elements
This series addresses the issue in two steps:
1. Detect and reject invalid compatibility requests early in KVM to avoid late failures.
2. Provide a mechanism for userspace to query the effective CPU compatibility modes supported by the host, so it can select an appropriate CPU model for nested guests.
To achieve this, the series introduces a new KVM capability and ioctl (KVM_CAP_PPC_COMPAT_CAPS / KVM_PPC_GET_COMPAT_CAPS) that expose the compatibility modes supported by the host.
Why a new UAPI?
---------------
While cpu-version is available in /proc/device-tree/cpus/<cpu#>/cpu-version on both L1 booted on PowerNV and PowerVM LPARs, the UAPI approach is preferable for several reasons:
1. pHYP (L0) capabilities: On PowerVM, we need to rely on capabilities negotiated with pHYP in KVM, not just device tree properties. The cpu-version property depicts the current compat mode but doesn't point to what all compat modes are supported for the nested guest.
2. procfs dependency: Not all systems run with procfs enabled (CONFIG_PROC_FS is optional). Minimal configurations like buildroot might disable it, but KVM ioctl works regardless since it accesses kernel data structures directly.
3. Kernel validation: The kernel validates and normalizes the compatibility information. Patch 1 adds validation logic that rejects invalid compatibility requests early, ensuring userspace gets validated, consistent data.
4. Abstraction & stability: /proc/device-tree is an implementation detail. The UAPI provides a stable interface that won't break if the underlying mechanism changes.
5. Semantic clarity: KVM_PPC_GET_COMPAT_CAPS clearly expresses what compatibility modes can be used for KVM guests, vs. parsing device tree which requires understanding the semantic meaning of cpu-version.
The implementation supports both:
- PowerVM (nested API v2), where compatibility information is obtained via the H_GUEST_GET_CAPABILITIES hypercall.
- PowerNV (nested API v1), where compatibility is derived from the device tree ("cpu-version") representing the effective processor compatibility level.
This allows userspace (e.g., QEMU) to select a CPU model consistent with the host compatibility mode, avoiding mismatches and enabling successful nested guest boot.
Changes in v3:
- Added "Why a new UAPI?" section to cover letter addressing questions about the need for a new UAPI vs. using existing mechanisms like /proc/device-tree
- Fixed initialization of 'r' in KVM_PPC_GET_COMPAT_CAPS ioctl handler from 0 to -ENOTTY for proper error handling when the operation is not supported
- Added Vaibhav's "Suggested-by" tags
- Have retained Anushree's "Tested-by" tags as no major code changes
- Fixed documentation build warning reported by kernel test robot and added "Reported-by" and "Closes" tags to patch 5
Changes in v2:
- Squashed patches 2 and 3 from v1 (capability introduction and ioctl wiring) into a single patch for better logical grouping
- Changed kvm_ppc_compat_caps.flags from __u32 to __u64 for consistency and future extensibility
- Addressed other review comments
- Improved commit messages with clearer explanations of the changes
Patch summary:
[1/5] Validate arch_compat against host compatibility mode [2/5] Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl [3/5] Implement capability retrieval for PowerVM (API v2) [4/5] Add PowerNV support (API v1)
[5/5] Document the new ioctl
Tested on:
- Power11 pSeries LPAR in Power10 compatibility mode (nested API v2)
- Power10 PowerNV system (and QEMU TCG PowerNV 11) with nested virtualization (API v1) with various combinations of KVM L1/L2 guests in various supported compatibility modes.
With this series, nested guests boot successfully in configurations where they previously failed due to compatibility mismatches.
Related QEMU series:
A corresponding QEMU series adds support for querying and using these compatibility capabilities when configuring nested KVM guests:
https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/linuxppc-dev/[email protected]/
v1: https://lore.kernel.org/linuxppc-dev/[email protected]/
Amit Machhiwal (5):
KVM: PPC: Book3S HV: Validate arch_compat against host compatibility
mode
KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM
on PowerVM
KVM: PPC: Book3S HV: Add support for compat CPU capabilities for KVM
on PowerNV
KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl
Documentation/virt/kvm/api.rst | 35 ++++++++++++++++ arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/include/uapi/asm/kvm.h | 6 +++ arch/powerpc/kvm/book3s_hv.c | 63 +++++++++++++++++++++++++++++ arch/powerpc/kvm/powerpc.c | 21 ++++++++++ include/uapi/linux/kvm.h | 4 ++ 6 files changed, 130 insertions(+)
base-commit: 1d5dcaa3bd65f2e8c9baa14a393d3a2dc5db7524
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH v3 1/5 KVM: PPC: Book3S HV: Validate arch_compat against host compatibility mode
On IBM POWER systems, newer processor generations can operate in compatibility modes corresponding to earlier generations. This becomes relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Currently, when running a nested KVM guest (L2) inside a Power11 pSeries logical partition (L1) booted in Power10 compatibility mode, the guest fails to boot while setting 'arch_compat'. This happens because the CPU class is derived from the hardware PVR (via mfspr()), which reflects the physical processor generation (Power11), rather than the effective compatibility mode (Power10).
As a result, userspace may request a Power11 arch_compat for the L2 guest. However, the L1 partition, running in Power10 compatibility, has only negotiated support up to Power10 with the Power Hypervisor (L0). When H_SET_STATE is invoked with a Power11 Logical PVR, the hypervisor rejects the request, leading to a late guest boot failure:
KVM-NESTEDv2: couldn't set guest wide elements
[..KVM reg dump..]
This situation should be detected earlier. Rejecting unsupported 'arch_compat' values in 'kvmppc_set_arch_compat()' avoids issuing an invalid H_SET_STATE hcall and provides a clearer failure mode.
Add a check to reject Power11 'arch_compat' requests when the host is running in Power10 compatibility mode, returning -EINVAL early instead of deferring the failure to the hypervisor.
--- arch/powerpc/kvm/book3s_hv.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
arch/powerpc/kvm/book3s_hv.c +12 -0
@@ -446,7 +446,19 @@ static int kvmppc_set_arch_compat(struct kvm_vcpu *vcpu, u32 arch_compat) guest_pcr_bit =PCR_ARCH_300 ;break ;case PVR_ARCH_31 : + guest_pcr_bit =PCR_ARCH_31 ; +break ;case PVR_ARCH_31_P11 : +/* +* Need to check this for ISA 3.1, as Power10 and +* Power11 share the same PCR. For any subsequent ISA +* versions, this will be taken care of by the guest vs +* host PCR comparison below. +*/ +if ((PVR_ARCH_31 & cur_cpu_spec->pvr_mask) == + cur_cpu_spec->pvr_value) { +return -EINVAL ; + } guest_pcr_bit =PCR_ARCH_31 ;break ;default :
Signature · 1 line
2.50.1 (Apple Git-155)
Ritesh Harjani (IBM)
reviewerRe: [PATCH v3 1/5] KVM: PPC: Book3S HV: Validate arch_compat against host compatibility mode
Quoted 12 lines · Amit Machhiwal
On IBM POWER systems, newer processor generations can operate in compatibility modes corresponding to earlier generations. This becomes relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Currently, when running a nested KVM guest (L2) inside a Power11 pSeries logical partition (L1) booted in Power10 compatibility mode, the guest fails to boot while setting '
arch_compat'. This happens because the CPU class is derived from the hardware PVR (viamfspr()), which reflects the physical processor generation (Power11), rather than the effective compatibility mode (Power10).
Quoted 4 lines
As a result, userspace may request a Power11
arch_compatfor the L2 guest. However, the L1 partition, running in Power10 compatibility, has only negotiated support up to Power10 with the Power Hypervisor (L0). WhenH_SET_STATEis invoked with a Power11 Logical PVR, the hypervisor
s/H_SET_STATE/H_GUEST_SET_STATE
Quoted 5 lines
rejects the request, leading to a late guest boot failure:
KVM-NESTEDv2: couldn't set guest wide elements
[..KVM reg dump..]
I think irrespective of the other UAPI changes, we should still get this fixed - so that we don't see a late KVM guest boot failure msgs.
So, in this review, I would like to mainly look at fixing this issue first and would request if we can defer the UAPI changes as a separate patch series please.
Quoted 3 lines
This situation should be detected earlier. Rejecting unsupported '
arch_compat' values in 'kvmppc_set_arch_compat()' avoids issuing an invalidH_SET_STATEhcall and provides a clearer failure mode.
s/H_SET_STATE/H_GUEST_SET_STATE
Quoted 23 lines
Add a check to reject Power11 '
arch_compat' requests when the host is running in Power10 compatibility mode, returning-EINVALearly instead of deferring the failure to the hypervisor.Suggested-by Vaibhav Jain <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- arch/powerpc/kvm/book3s_hv.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)arch/powerpc/kvm/book3s_hv.c +2
-0@@ -446,7 +446,19 @@ static int kvmppc_set_arch_compat(struct kvm_vcpu *vcpu, u32 arch_compat) guest_pcr_bit =PCR_ARCH_300 ;break ;case PVR_ARCH_31 : + guest_pcr_bit =PCR_ARCH_31 ; +break ;case PVR_ARCH_31_P11 :
arch/powerpc/kvm/book3s_hv.c static int kvmppc_set_arch_compat(struct kvm_vcpu *vcpu, u32 arch_compat) · line 461
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +10
-0+/* +* Need to check this for ISA 3.1, as Power10 and +* Power11 share the same PCR. For any subsequent ISA +* versions, this will be taken care of by the guest vs +* host PCR comparison below. +*/ +if ((PVR_ARCH_31 & cur_cpu_spec->pvr_mask) == + cur_cpu_spec->pvr_value) { +return -EINVAL ; + }
Instead of the complicated check can we simply do this?
if (!cpu_has_feature(CPU_FTR_P11_PVR)) return -EINVAL;
which means that if the Qemu is trying to set the arch_compat with P11 PVR (arch_compat) and if the host cpu FTR doesn't support P11 PVR, then simply return -EINVAL
-ritesh
Re: [PATCH v3 1/5] KVM: PPC: Book3S HV: Validate arch_compat against host compatibility mode
Hi Ritesh,
Thanks for taking a look at this patch. Please find my response inline below:
Quoted 14 lines · Ritesh Harjani
Quoted 12 lines · Amit Machhiwal
On IBM POWER systems, newer processor generations can operate in compatibility modes corresponding to earlier generations. This becomes relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Currently, when running a nested KVM guest (L2) inside a Power11 pSeries logical partition (L1) booted in Power10 compatibility mode, the guest fails to boot while setting '
arch_compat'. This happens because the CPU class is derived from the hardware PVR (viamfspr()), which reflects the physical processor generation (Power11), rather than the effective compatibility mode (Power10).
Quoted 6 lines
Quoted 4 lines
As a result, userspace may request a Power11
arch_compatfor the L2 guest. However, the L1 partition, running in Power10 compatibility, has only negotiated support up to Power10 with the Power Hypervisor (L0). WhenH_SET_STATEis invoked with a Power11 Logical PVR, the hypervisors/H_SET_STATE/H_GUEST_SET_STATE
Good catch! I'll fix this in the next version.
Quoted 7 lines
Quoted 6 lines
rejects the request, leading to a late guest boot failure:
KVM-NESTEDv2: couldn't set guest wide elements
[..KVM reg dump..]
Quoted 6 lines
I think irrespective of the other UAPI changes, we should still get this fixed - so that we don't see a late KVM guest boot failure msgs.
So, in this review, I would like to mainly look at fixing this issue first and would request if we can defer the UAPI changes as a separate patch series please.
This patch only enables the guest boot to bail out early instead of going upto making an H_GUEST_SET_STATE hcall with a non supported compatibility mode. In addition to that, it does not fix the real problem where guest fails to boot on Power11 LPAR booted in a Power10 compatibility mode.
I understand your point and to make the segregation explicitly clear, I shall update the cover letter to mention that patch 1 only takes care of failing earlier as soon as an invalid compatibility mode is detected and Patch 2-5 introduce a new uAPI for evaluating the right compatibility mode.
The actual L2 guest boot fix with L1 booted in a compatibility mode is done via the next 4 patches which enables correct compatibility mode detection using the newly introduced uAPI. So, we would still want to prioritize the whole series instead of just this one patch.
Quoted 7 lines
Quoted 5 lines
This situation should be detected earlier. Rejecting unsupported '
arch_compat' values in 'kvmppc_set_arch_compat()' avoids issuing an invalidH_SET_STATEhcall and provides a clearer failure mode.s/H_SET_STATE/H_GUEST_SET_STATE
Will rectify in the next version.
Quoted 36 lines
Quoted 24 lines
Add a check to reject Power11 '
arch_compat' requests when the host is running in Power10 compatibility mode, returning-EINVALearly instead of deferring the failure to the hypervisor.Suggested-by Vaibhav Jain <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- arch/powerpc/kvm/book3s_hv.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)arch/powerpc/kvm/book3s_hv.c +2
-0@@ -446,7 +446,19 @@ static int kvmppc_set_arch_compat(struct kvm_vcpu *vcpu, u32 arch_compat) guest_pcr_bit =PCR_ARCH_300 ;break ;case PVR_ARCH_31 : + guest_pcr_bit =PCR_ARCH_31 ; +break ;case PVR_ARCH_31_P11 :arch/powerpc/kvm/book3s_hv.c static int kvmppc_set_arch_compat(struct kvm_vcpu *vcpu, u32 arch_compat) · line 461
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +10
-0+/* +* Need to check this for ISA 3.1, as Power10 and +* Power11 share the same PCR. For any subsequent ISA +* versions, this will be taken care of by the guest vs +* host PCR comparison below. +*/ +if ((PVR_ARCH_31 & cur_cpu_spec->pvr_mask) == + cur_cpu_spec->pvr_value) { +return -EINVAL ; + }Instead of the complicated check can we simply do this?
Quoted 2 lines
if (!cpu_has_feature(CPU_FTR_P11_PVR)) return -EINVAL;
Sure, I can base this check on CPU features.
Thanks,
Amit
Quoted 7 lines
which means that if the Qemu is trying to set the
arch_compatwith P11 PVR (arch_compat) and if the host cpu FTR doesn't support P11 PVR, then simply return-EINVAL-ritesh
Ritesh Harjani (IBM)
reviewerRe: [PATCH v3 1/5] KVM: PPC: Book3S HV: Validate arch_compat against host compatibility mode
Quoted 3 lines · Amit Machhiwal
So, we would still want to prioritize the whole series instead of just this one patch.
Patch-1 could go as a bug fix even in 7.1-rc6 (or maybe with 7.2 bug fixes). - Maddy?
So, you may want to add a fixes tag and maybe even cc stable if you are seeing this issue from older kernels maybe when nestedv2 got introduced? However the new UAPI discussion might still require more discussion with the community and I don't think it is ready for 7.2 yet ;)
-ritesh
Re: [PATCH v3 1/5] KVM: PPC: Book3S HV: Validate arch_compat against host compatibility mode
Hi Ritesh, thanks for looking into this patch. My responses to your review comments inline below.
Quoted 11 lines · Ritesh Harjani (IBM)
Quoted 3 lines · Amit Machhiwal
So, we would still want to prioritize the whole series instead of just this one patch.
Patch-1 could go as a bug fix even in 7.1-rc6 (or maybe with 7.2 bug fixes). - Maddy?
So, you may want to add a fixes tag and maybe even cc stable if you are seeing this issue from older kernels maybe when nestedv2 got introduced?
This isnt a 'bug fix' per-se but rather strengthening of compat mode checks so that any non compatible PVR being used by the VMM can be caught early. The hypervisor anyway ultimately prevents non-compatible PVRs from being used by the VMM. So there isnt a bug thats being fixed in this patch.
The rest of the patch series builds on top of this patch to advertise the available compatible PVRs to the VMM so that it can further preemptively prevent users from forcibly using a non-compatible PVR.
Hence IMHO, this patch can be marked for stable tree and potential candidate for 7.2 merge window. But dont see applicability of a 'fixes' tag to this patch
Quoted 2 lines
However the new UAPI discussion might still require more discussion with the community and I don't think it is ready for 7.2 yet ;)
Somewhat agree with the above
Quoted 3 lines
-ritesh
Signature · 2 lines
Cheers ~ Vaibhav
Madhavan Srinivasan
maintainerRe: [PATCH v3 1/5] KVM: PPC: Book3S HV: Validate arch_compat against host compatibility mode
Quoted 21 lines · Vaibhav Jain
Hi Ritesh, thanks for looking into this patch. My responses to your review comments inline below.
Quoted 10 lines · Ritesh Harjani (IBM)
Quoted 3 lines · Amit Machhiwal
So, we would still want to prioritize the whole series instead of just this one patch.
Patch-1 could go as a bug fix even in 7.1-rc6 (or maybe with 7.2 bug fixes). - Maddy?
So, you may want to add a fixes tag and maybe even cc stable if you are seeing this issue from older kernels maybe when nestedv2 got introduced?
This isnt a 'bug fix' per-se but rather strengthening of compat mode checks so that any non compatible PVR being used by the VMM can be caught early. The hypervisor anyway ultimately prevents non-compatible PVRs from being used by the VMM. So there isnt a bug thats being fixed in this patch.
Quoted 7 lines
The rest of the patch series builds on top of this patch to advertise the available compatible PVRs to the VMM so that it can further preemptively prevent users from forcibly using a non-compatible PVR.
Hence IMHO, this patch can be marked for stable tree and potential candidate for 7.2 merge window. But dont see applicability of a 'fixes' tag to this patch
amit, can you just post this alone as a separate patch, so that we could pull it for 7.2 merge?
Quoted 6 lines
Quoted 2 lines
However the new UAPI discussion might still require more discussion with the community and I don't think it is ready for 7.2 yet ;)
Somewhat agree with the above
Quoted 2 lines
-ritesh
Re: [PATCH v3 1/5] KVM: PPC: Book3S HV: Validate arch_compat against host compatibility mode
Quoted 27 lines · Madhavan Srinivasan
Quoted 25 lines · Vaibhav Jain
Hi Ritesh, thanks for looking into this patch. My responses to your review comments inline below.
Quoted 10 lines · Ritesh Harjani (IBM)
Quoted 3 lines · Amit Machhiwal
So, we would still want to prioritize the whole series instead of just this one patch.
Patch-1 could go as a bug fix even in 7.1-rc6 (or maybe with 7.2 bug fixes). - Maddy?
So, you may want to add a fixes tag and maybe even cc stable if you are seeing this issue from older kernels maybe when nestedv2 got introduced?
This isnt a 'bug fix' per-se but rather strengthening of compat mode checks so that any non compatible PVR being used by the VMM can be caught early. The hypervisor anyway ultimately prevents non-compatible PVRs from being used by the VMM. So there isnt a bug thats being fixed in this patch.
The rest of the patch series builds on top of this patch to advertise the available compatible PVRs to the VMM so that it can further preemptively prevent users from forcibly using a non-compatible PVR.
Quoted 6 lines
Quoted 3 lines
Hence IMHO, this patch can be marked for stable tree and potential candidate for 7.2 merge window. But dont see applicability of a 'fixes' tag to this patch
amit, can you just post this alone as a separate patch, so that we could pull it for 7.2 merge?
FWIW, b4 am -P1 <mbox> should fetch this patch alone (and not the entire series), See b4 am --help for more options to select a subset of patches.
regards,
Harsh>
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
However the new UAPI discussion might still require more discussion with the community and I don't think it is ready for 7.2 yet ;)
Somewhat agree with the above
Quoted 2 lines
-ritesh
Ritesh Harjani (IBM)
reviewerRe: [PATCH v3 1/5] KVM: PPC: Book3S HV: Validate arch_compat against host compatibility mode
Quoted 7 lines · Harsh Prateek Bora
Quoted 3 lines
amit, can you just post this alone as a separate patch, so that we could pull it for 7.2 merge?
FWIW, b4 am -P1 <mbox> should fetch this patch alone (and not the entire series), See b4 am --help for more options to select a subset of patches.
I agree, however as an FYI in this case -
I had few review comments on PATCH-1 here [1] - which along with the commit msg changes, also had a code change involved, so IMO, it's still a good idea if Amit can test and send an updated patch separately for this - to be pulled in for 7.2.
[1]: https://lore.kernel.org/linuxppc-dev/[email protected]/
Replying to Vaibhav comment here so that we can reach to the conclusion at one place.
Quoted 3 lines
Hence IMHO, this patch can be marked for stable tree and potential candidate for 7.2 merge window. But dont see applicability of a 'fixes' tag to this patch
I agree, we need not use a fixes tag then. So, we shall mark this with v6.10 tag then.
(I calculated this based on when Power11 was added:
git tag --contains c2ed087ed35ca | grep -E "^v" |head -1
v6.10
)
-ritesh
Re: [PATCH v3 1/5] KVM: PPC: Book3S HV: Validate arch_compat against host compatibility mode
Quoted 29 lines · Ritesh Harjani (IBM)
Quoted 7 lines · Harsh Prateek Bora
Quoted 3 lines
amit, can you just post this alone as a separate patch, so that we could pull it for 7.2 merge?
FWIW, b4 am -P1 <mbox> should fetch this patch alone (and not the entire series), See b4 am --help for more options to select a subset of patches.
I agree, however as an FYI in this case -
I had few review comments on PATCH-1 here [1] - which along with the commit msg changes, also had a code change involved, so IMO, it's still a good idea if Amit can test and send an updated patch separately for this - to be pulled in for 7.2.[1]: https://lore.kernel.org/linuxppc-dev/[email protected]/
Replying to Vaibhav comment here so that we can reach to the conclusion at one place.
Quoted 3 lines
Hence IMHO, this patch can be marked for stable tree and potential candidate for 7.2 merge window. But dont see applicability of a 'fixes' tag to this patch
I agree, we need not use a fixes tag then. So, we shall mark this with v6.10 tag then.
Quoted 6 lines
Cc [email protected] # v6.10+(I calculated this based on when Power11 was added:
git tag --containsc2ed087ed35ca| grep -E "^v" |head -1
v6.10
)
Thanks for help with this, Ritesh!
Quoted 3 lines
-ritesh
Re: [PATCH v3 1/5] KVM: PPC: Book3S HV: Validate arch_compat against host compatibility mode
In reply to Ritesh Harjani (IBM)Hi Maddy, Ritesh,
I have made the suggested changes and posted the separate patch here:
https://lore.kernel.org/all/[email protected]/
<..snip..>
Quoted 8 lines
Quoted 3 lines
Hence IMHO, this patch can be marked for stable tree and potential candidate for 7.2 merge window. But dont see applicability of a 'fixes' tag to this patch
I agree, we need not use a fixes tag then. So, we shall mark this with v6.10 tag then.
Cc [email protected] # v6.10+
Please note that I have marked the patch for stable v6.13+ as the KVM support for Power11 was added via 96e266e3bcd6 ("KVM: PPC: Book3S HV:
Add Power11 capability support for Nested PAPR guests"). Also, this commit had introduced CPU_FTR_P11_PVR on which the compat PVR check in the patch is based on.
Thanks,
Amit
PATCH v3 2/5 KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
In reply to Amit MachhiwalIntroduce a new capability and ioctl to expose CPU compatibility modes supported by the host processor for nested guests.
On IBM POWER systems, newer processor generations (N) can operate in compatibility modes corresponding to earlier generations, like (N-1) and (N-2). This is particularly relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Introduce KVM_CAP_PPC_COMPAT_CAPS capability and the corresponding KVM_PPC_GET_COMPAT_CAPS vm ioctl. The ioctl returns a bitmap describing the compatibility modes supported by the host in respective bit numbers, allowing userspace (e.g., QEMU) to select an appropriate compatibility level when configuring nested KVM guests.
The ioctl handling is added in kvm_arch_vm_ioctl() and retrieves host CPU compatibility capabilities via a PowerPC-specific backend implementation when available. If the capability is not supported, the ioctl returns success with no capabilities set, allowing userspace to fall back gracefully.
--- arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/include/uapi/asm/kvm.h | 6 ++++++ arch/powerpc/kvm/powerpc.c | 21 +++++++++++++++++++++ include/uapi/linux/kvm.h | 4 ++++ 4 files changed, 32 insertions(+)
arch/powerpc/include/asm/kvm_ppc.h +1 -0
@@ -319,6 +319,7 @@ struct kvmppc_ops {bool (*hash_v3_possible)(void );int (*create_vm_debugfs)(struct kvm *kvm);int (*create_vcpu_debugfs)(struct kvm_vcpu *vcpu,struct dentry *debugfs_dentry); +int (*get_compat_cpu_ver)(struct kvm_ppc_compat_caps *host_caps); };extern struct kvmppc_ops *kvmppc_hv_ops;
arch/powerpc/include/uapi/asm/kvm.h +6 -0
@@ -437,6 +437,12 @@ struct kvm_ppc_cpu_char {__u64 behaviour_mask;/* valid bits in behaviour */ }; +/* For KVM_PPC_GET_COMPAT_CAPS */ +struct kvm_ppc_compat_caps { +__u64 flags;/* Reserved for future use */ +__u64 compat_capabilities;/* Capabilities supported by the host */ +}; +/* * Values for character and character_mask. * These are identical to the values used by H_GET_CPU_CHARACTERISTICS.
arch/powerpc/kvm/powerpc.c +21 -0
@@ -697,6 +697,13 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) } }break ; +#if defined(CONFIG_KVM_BOOK3S_HV_POSSIBLE ) +case KVM_CAP_PPC_COMPAT_CAPS : + r =0 ; +if (kvmhv_on_pseries ()) + r =1 ; +break ; +#endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */ default : r =0 ;break ;
@@ -2463,6 +2470,20 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = kvm->arch.kvm_ops->svm_off (kvm);break ; } +case KVM_PPC_GET_COMPAT_CAPS : { +struct kvm_ppc_compat_caps host_caps; + + r = -ENOTTY ; +memset (&host_caps,0 ,sizeof (host_caps)); +if (!kvm->arch.kvm_ops->get_compat_cpu_ver) + goto out; + + r = kvm->arch.kvm_ops->get_compat_cpu_ver (&host_caps); +if (!r &©_to_user (argp, &host_caps, +sizeof (host_caps))) + r = -EFAULT ; +break ; + }default : {struct kvm *kvm = filp->private_data; r = kvm->arch.kvm_ops->arch_vm_ioctl (filp, ioctl, arg);
include/uapi/linux/kvm.h +4 -0
@@ -996,6 +996,7 @@ struct kvm_enable_cap {#define KVM_CAP_S390_USER_OPEREXEC 246#define KVM_CAP_S390_KEYOP 247#define KVM_CAP_S390_VSIE_ESAMODE 248 +#define KVM_CAP_PPC_COMPAT_CAPS 249struct kvm_irq_routing_irqchip {__u32 irqchip;
@@ -1349,6 +1350,9 @@ struct kvm_s390_keyop {#define KVM_GET_DEVICE_ATTR _IOW(KVMIO, 0xe2, struct kvm_device_attr)#define KVM_HAS_DEVICE_ATTR _IOW(KVMIO, 0xe3, struct kvm_device_attr) +/* Available with KVM_CAP_PPC_COMPAT_CAPS */ +#define KVM_PPC_GET_COMPAT_CAPS _IOR(KVMIO, 0xe4, struct kvm_ppc_compat_caps) +/* * ioctls for vcpu fds */
Signature · 1 line
2.50.1 (Apple Git-155)
Re: [PATCH v3 2/5] KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
Hi Amit,
Thanks for this patch. Few review comments below:
Quoted 48 lines · Amit Machhiwal
Introduce a new capability and ioctl to expose CPU compatibility modes supported by the host processor for nested guests.
On IBM POWER systems, newer processor generations (N) can operate in compatibility modes corresponding to earlier generations, like (N-1) and (N-2). This is particularly relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Introduce
KVM_CAP_PPC_COMPAT_CAPScapability and the correspondingKVM_PPC_GET_COMPAT_CAPSvm ioctl. The ioctl returns a bitmap describing the compatibility modes supported by the host in respective bit numbers, allowing userspace (e.g., QEMU) to select an appropriate compatibility level when configuring nested KVM guests.The ioctl handling is added in
kvm_arch_vm_ioctl()and retrieves host CPU compatibility capabilities via a PowerPC-specific backend implementation when available. If the capability is not supported, the ioctl returns success with no capabilities set, allowing userspace to fall back gracefully.Suggested-by Vaibhav Jain <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/include/uapi/asm/kvm.h | 6 ++++++ arch/powerpc/kvm/powerpc.c | 21 +++++++++++++++++++++ include/uapi/linux/kvm.h | 4 ++++ 4 files changed, 32 insertions(+)arch/powerpc/include/asm/kvm_ppc.h +1
-0@@ -319,6 +319,7 @@ struct kvmppc_ops {bool (*hash_v3_possible)(void );int (*create_vm_debugfs)(struct kvm *kvm);int (*create_vcpu_debugfs)(struct kvm_vcpu *vcpu,struct dentry *debugfs_dentry); +int (*get_compat_cpu_ver)(struct kvm_ppc_compat_caps *host_caps); };extern struct kvmppc_ops *kvmppc_hv_ops;arch/powerpc/include/uapi/asm/kvm.h +0
-0
arch/powerpc/include/uapi/asm/kvm.h struct kvm_ppc_cpu_char { · line 442
Quoted 6 lines
arch/powerpc/include/uapi/asm/kvm.h +3
-0@@ -437,6 +437,12 @@ struct kvm_ppc_cpu_char {__u64 behaviour_mask;/* valid bits in behaviour */ }; +/* For KVM_PPC_GET_COMPAT_CAPS */ +struct kvm_ppc_compat_caps { +__u64 flags;/* Reserved for future use */
Please introduce a size field also for the UAPI so that in this structure can evolve in future without breaking kernel ABI.
Quoted 29 lines
arch/powerpc/include/uapi/asm/kvm.h +3
-0+__u64 compat_capabilities;/* Capabilities supported by the host */ +}; +/* * Values for character and character_mask. * These are identical to the values used by H_GET_CPU_CHARACTERISTICS. arch/powerpc/kvm/powerpc.c +8
-0@@ -697,6 +697,13 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) } }break ; +#if defined(CONFIG_KVM_BOOK3S_HV_POSSIBLE ) +case KVM_CAP_PPC_COMPAT_CAPS : + r =0 ; +if (kvmhv_on_pseries ()) + r =1 ; +break ; +#endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */ default : r =0 ;break ;@@ -2463,6 +2470,20 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = kvm->arch.kvm_ops->svm_off (kvm);break ; } +case KVM_PPC_GET_COMPAT_CAPS : {
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2483
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +10
-0+struct kvm_ppc_compat_caps host_caps; + + r = -ENOTTY ; +memset (&host_caps,0 ,sizeof (host_caps)); +if (!kvm->arch.kvm_ops->get_compat_cpu_ver) + goto out; + + r = kvm->arch.kvm_ops->get_compat_cpu_ver (&host_caps); +if (!r &©_to_user (argp, &host_caps, +sizeof (host_caps)))
As mentioned above please introduce a size field in the structure thats being copied to the userspace and use the size field to copy the apporiate structure to the userspace. Otherwise a future kernel may unintentionally overwrite unintended userspace memory if it happens to be using a larger structure size then what VMM knows about.
Quoted 19 lines
arch/powerpc/kvm/powerpc.c +3
-0+ r = -EFAULT ; +break ; + }default : {struct kvm *kvm = filp->private_data; r = kvm->arch.kvm_ops->arch_vm_ioctl (filp, ioctl, arg);include/uapi/linux/kvm.h +1
-0@@ -996,6 +996,7 @@ struct kvm_enable_cap {#define KVM_CAP_S390_USER_OPEREXEC 246#define KVM_CAP_S390_KEYOP 247#define KVM_CAP_S390_VSIE_ESAMODE 248 +#define KVM_CAP_PPC_COMPAT_CAPS 249struct kvm_irq_routing_irqchip {__u32 irqchip;
include/uapi/linux/kvm.h struct kvm_s390_keyop { · line 1355
Quoted 6 lines
include/uapi/linux/kvm.h +2
-0@@ -1349,6 +1350,9 @@ struct kvm_s390_keyop {#define KVM_GET_DEVICE_ATTR _IOW(KVMIO, 0xe2, struct kvm_device_attr)#define KVM_HAS_DEVICE_ATTR _IOW(KVMIO, 0xe3, struct kvm_device_attr) +/* Available with KVM_CAP_PPC_COMPAT_CAPS */ +#define KVM_PPC_GET_COMPAT_CAPS _IOR(KVMIO, 0xe4, struct kvm_ppc_compat_caps)
Minor: you may want to align the name of the newly introduced kvmppc_ops to KVM CAP you are introducing here.
Quoted 8 lines
include/uapi/linux/kvm.h +1
-0+/* * ioctls for vcpu fds */
Signature · 1 line
2.50.1 (Apple Git-155)
Signature · 2 lines
Cheers ~ Vaibhav
Re: [PATCH v3 2/5] KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
Hi Vaibhav,
Thanks for reviewing the patches. Please find my response inline.
Quoted 57 lines · Vaibhav Jain
Hi Amit,
Thanks for this patch. Few review comments below:
Quoted 51 lines · Amit Machhiwal
Introduce a new capability and ioctl to expose CPU compatibility modes supported by the host processor for nested guests.
On IBM POWER systems, newer processor generations (N) can operate in compatibility modes corresponding to earlier generations, like (N-1) and (N-2). This is particularly relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Introduce
KVM_CAP_PPC_COMPAT_CAPScapability and the correspondingKVM_PPC_GET_COMPAT_CAPSvm ioctl. The ioctl returns a bitmap describing the compatibility modes supported by the host in respective bit numbers, allowing userspace (e.g., QEMU) to select an appropriate compatibility level when configuring nested KVM guests.The ioctl handling is added in
kvm_arch_vm_ioctl()and retrieves host CPU compatibility capabilities via a PowerPC-specific backend implementation when available. If the capability is not supported, the ioctl returns success with no capabilities set, allowing userspace to fall back gracefully.Suggested-by Vaibhav Jain <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/include/uapi/asm/kvm.h | 6 ++++++ arch/powerpc/kvm/powerpc.c | 21 +++++++++++++++++++++ include/uapi/linux/kvm.h | 4 ++++ 4 files changed, 32 insertions(+)arch/powerpc/include/asm/kvm_ppc.h +1
-0@@ -319,6 +319,7 @@ struct kvmppc_ops {bool (*hash_v3_possible)(void );int (*create_vm_debugfs)(struct kvm *kvm);int (*create_vcpu_debugfs)(struct kvm_vcpu *vcpu,struct dentry *debugfs_dentry); +int (*get_compat_cpu_ver)(struct kvm_ppc_compat_caps *host_caps); };extern struct kvmppc_ops *kvmppc_hv_ops;arch/powerpc/include/uapi/asm/kvm.h +0
-0@@ -437,6 +437,12 @@ struct kvm_ppc_cpu_char {__u64 behaviour_mask;/* valid bits in behaviour */ };
Quoted 5 lines
arch/powerpc/include/uapi/asm/kvm.h struct kvm_ppc_cpu_char { · line 442
Quoted 3 lines
arch/powerpc/include/uapi/asm/kvm.h +3
-0+/* For KVM_PPC_GET_COMPAT_CAPS */ +struct kvm_ppc_compat_caps { +__u64 flags;/* Reserved for future use */ Please introduce a size field also for the UAPI so that in this structure can evolve in future without breaking kernel ABI.
Sure, adding a size field for validations and future extensibility makes sense. I'll add it in the next version.
Quoted 37 lines
Quoted 37 lines
arch/powerpc/include/uapi/asm/kvm.h +3
-0+__u64 compat_capabilities;/* Capabilities supported by the host */ +}; +/* * Values for character and character_mask. * These are identical to the values used by H_GET_CPU_CHARACTERISTICS. arch/powerpc/kvm/powerpc.c +15
-0@@ -697,6 +697,13 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) } }break ; +#if defined(CONFIG_KVM_BOOK3S_HV_POSSIBLE ) +case KVM_CAP_PPC_COMPAT_CAPS : + r =0 ; +if (kvmhv_on_pseries ()) + r =1 ; +break ; +#endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */ default : r =0 ;break ;@@ -2463,6 +2470,20 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = kvm->arch.kvm_ops->svm_off (kvm);break ; } +case KVM_PPC_GET_COMPAT_CAPS : { +struct kvm_ppc_compat_caps host_caps; + + r = -ENOTTY ; +memset (&host_caps,0 ,sizeof (host_caps)); +if (!kvm->arch.kvm_ops->get_compat_cpu_ver) + goto out; +
Quoted 8 lines
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2483
Quoted 3 lines
arch/powerpc/kvm/powerpc.c +3
-0+ r = kvm->arch.kvm_ops->get_compat_cpu_ver (&host_caps); +if (!r &©_to_user (argp, &host_caps, +sizeof (host_caps)))As mentioned above please introduce a size field in the structure thats being copied to the userspace and use the size field to copy the apporiate structure to the userspace. Otherwise a future kernel may unintentionally overwrite unintended userspace memory if it happens to be using a larger structure size then what VMM knows about.
Sure, I'll add a validation around the structure size.
Quoted 23 lines
Quoted 23 lines
arch/powerpc/kvm/powerpc.c +3
-0+ r = -EFAULT ; +break ; + }default : {struct kvm *kvm = filp->private_data; r = kvm->arch.kvm_ops->arch_vm_ioctl (filp, ioctl, arg);include/uapi/linux/kvm.h +1
-0@@ -996,6 +996,7 @@ struct kvm_enable_cap {#define KVM_CAP_S390_USER_OPEREXEC 246#define KVM_CAP_S390_KEYOP 247#define KVM_CAP_S390_VSIE_ESAMODE 248 +#define KVM_CAP_PPC_COMPAT_CAPS 249struct kvm_irq_routing_irqchip {__u32 irqchip;@@ -1349,6 +1350,9 @@ struct kvm_s390_keyop {#define KVM_GET_DEVICE_ATTR _IOW(KVMIO, 0xe2, struct kvm_device_attr)#define KVM_HAS_DEVICE_ATTR _IOW(KVMIO, 0xe3, struct kvm_device_attr)
Quoted 5 lines
include/uapi/linux/kvm.h struct kvm_s390_keyop { · line 1355
Quoted 3 lines
include/uapi/linux/kvm.h +2
-0+/* Available with KVM_CAP_PPC_COMPAT_CAPS */ +#define KVM_PPC_GET_COMPAT_CAPS _IOR(KVMIO, 0xe4, struct kvm_ppc_compat_caps)Minor: you may want to align the name of the newly introduced
kvmppc_opsto KVM CAP you are introducing here.
Will do.
Thanks,
Amit
Quoted 13 lines
Quoted 9 lines
include/uapi/linux/kvm.h +1
-0+/* * ioctls for vcpu fds */
Signature · 1 line
2.50.1 (Apple Git-155)Signature · 2 lines
Cheers ~ Vaibhav
PATCH v3 3/5 KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM on PowerVM
In reply to Amit MachhiwalOn POWER systems, the host CPU may run in a compatibility mode (e.g., a Power11 processor operating in Power10 compatibility mode). In such cases, the effective CPU level exposed to guests differs from the physical processor generation.
When running nested KVM guests, QEMU derives the host CPU type using mfpvr(), which reflects the physical processor version. This can result in a mismatch between the CPU model selected by QEMU and the compatibility mode enforced by the host, leading to guest boot failures.
For example, booting a nested guest on a Power11 LPAR configured in Power10 compatibility mode fails with:
KVM-NESTEDv2: couldn't set guest wide elements
[..KVM reg dump..]
This occurs because QEMU selects a CPU model corresponding to the physical processor (via mfpvr()), while the host operates in a lower compatibility mode. As a result, KVM rejects the requested compatibility level during guest initialization.
Add support for retrieving host CPU compatibility capabilities for nested guests on PowerVM (PAPR nested API v2). The hypervisor provides the effective compatibility levels via the H_GUEST_GET_CAPABILITIES hcall, which reflects the processor modes negotiated between the Power hypervisor (L0) and the host partition (L1).
On pseries systems, obtain the capability bitmap using plpar_guest_get_capabilities() and return it via struct kvm_ppc_compat_caps. This information is then exposed to userspace through the KVM_PPC_GET_COMPAT_CAPS ioctl.
Hook the implementation into the Book3S HV kvmppc_ops so that it can be invoked by the generic KVM ioctl handling code.
--- arch/powerpc/kvm/book3s_hv.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
arch/powerpc/kvm/book3s_hv.c +16 -0
@@ -6522,6 +6522,21 @@ static bool kvmppc_hash_v3_possible(void)return true; } + +static int kvmppc_get_compat_cpu_caps (struct kvm_ppc_compat_caps *host_caps) +{ +unsigned long capabilities =0 ; +long rc = -EINVAL ; + +if (kvmhv_on_pseries ()) { +if (kvmhv_is_nestedv2 ()) + rc =plpar_guest_get_capabilities (0 , &capabilities); + host_caps->compat_capabilities = capabilities; + } + +return rc; +} +static struct kvmppc_ops kvm_ops_hv = { .get_sregs = kvm_arch_vcpu_ioctl_get_sregs_hv, .set_sregs = kvm_arch_vcpu_ioctl_set_sregs_hv,
@@ -6564,6 +6579,7 @@ static struct kvmppc_ops kvm_ops_hv = { .hash_v3_possible = kvmppc_hash_v3_possible, .create_vcpu_debugfs = kvmppc_arch_create_vcpu_debugfs_hv, .create_vm_debugfs = kvmppc_arch_create_vm_debugfs_hv, + .get_compat_cpu_ver = kvmppc_get_compat_cpu_caps, };static int kvm_init_subcore_bitmap (void )
Signature · 1 line
2.50.1 (Apple Git-155)
Re: [PATCH v3 3/5] KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM on PowerVM
Hi Amit,
Thanks for the patch. My review comments inline below:
Quoted 50 lines · Amit Machhiwal
On POWER systems, the host CPU may run in a compatibility mode (e.g., a Power11 processor operating in Power10 compatibility mode). In such cases, the effective CPU level exposed to guests differs from the physical processor generation.
When running nested KVM guests, QEMU derives the host CPU type using
mfpvr(), which reflects the physical processor version. This can result in a mismatch between the CPU model selected by QEMU and the compatibility mode enforced by the host, leading to guest boot failures.For example, booting a nested guest on a Power11 LPAR configured in Power10 compatibility mode fails with:
KVM-NESTEDv2: couldn't set guest wide elements
[..KVM reg dump..]This occurs because QEMU selects a CPU model corresponding to the physical processor (via
mfpvr()), while the host operates in a lower compatibility mode. As a result, KVM rejects the requested compatibility level during guest initialization.Add support for retrieving host CPU compatibility capabilities for nested guests on PowerVM (PAPR nested API v2). The hypervisor provides the effective compatibility levels via the
H_GUEST_GET_CAPABILITIEShcall, which reflects the processor modes negotiated between the Power hypervisor (L0) and the host partition (L1).On pseries systems, obtain the capability bitmap using
plpar_guest_get_capabilities()and return it via structkvm_ppc_compat_caps. This information is then exposed to userspace through theKVM_PPC_GET_COMPAT_CAPSioctl.Hook the implementation into the Book3S HV
kvmppc_opsso that it can be invoked by the generic KVM ioctl handling code.Suggested-by Vaibhav Jain <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- arch/powerpc/kvm/book3s_hv.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)arch/powerpc/kvm/book3s_hv.c +0
-0@@ -6522,6 +6522,21 @@ static bool kvmppc_hash_v3_possible(void)return true; }
arch/powerpc/kvm/book3s_hv.c static bool kvmppc_hash_v3_possible(void) · line 6534
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +9
-0+ +static int kvmppc_get_compat_cpu_caps (struct kvm_ppc_compat_caps *host_caps) +{ +unsigned long capabilities =0 ; +long rc = -EINVAL ; + +if (kvmhv_on_pseries ()) { +if (kvmhv_is_nestedv2 ()) + rc =plpar_guest_get_capabilities (0 , &capabilities);
since this value will trikle back to userspace please apply a mask on the hcall return value so that any reserved and non-PVR related bits doesnt leak back to userspace.
Quoted 21 lines
arch/powerpc/kvm/book3s_hv.c +6
-0+ host_caps->compat_capabilities = capabilities; + } + +return rc; +} +static struct kvmppc_ops kvm_ops_hv = { .get_sregs = kvm_arch_vcpu_ioctl_get_sregs_hv,arch/powerpc/kvm/book3s_hv.c +1
-0.set_sregs = kvm_arch_vcpu_ioctl_set_sregs_hv,@@ -6564,6 +6579,7 @@ static struct kvmppc_ops kvm_ops_hv = { .hash_v3_possible = kvmppc_hash_v3_possible, .create_vcpu_debugfs = kvmppc_arch_create_vcpu_debugfs_hv, .create_vm_debugfs = kvmppc_arch_create_vm_debugfs_hv, + .get_compat_cpu_ver = kvmppc_get_compat_cpu_caps, };static int kvm_init_subcore_bitmap (void )Signature · 1 line
2.50.1 (Apple Git-155)
Signature · 2 lines
Cheers ~ Vaibhav
Re: [PATCH v3 3/5] KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM on PowerVM
Hi Vaibhav,
Thanks for taking a look at this patch. My response is inline.
Quoted 56 lines · Vaibhav Jain
Hi Amit,
Thanks for the patch. My review comments inline below:
Quoted 50 lines · Amit Machhiwal
On POWER systems, the host CPU may run in a compatibility mode (e.g., a Power11 processor operating in Power10 compatibility mode). In such cases, the effective CPU level exposed to guests differs from the physical processor generation.
When running nested KVM guests, QEMU derives the host CPU type using
mfpvr(), which reflects the physical processor version. This can result in a mismatch between the CPU model selected by QEMU and the compatibility mode enforced by the host, leading to guest boot failures.For example, booting a nested guest on a Power11 LPAR configured in Power10 compatibility mode fails with:
KVM-NESTEDv2: couldn't set guest wide elements
[..KVM reg dump..]This occurs because QEMU selects a CPU model corresponding to the physical processor (via
mfpvr()), while the host operates in a lower compatibility mode. As a result, KVM rejects the requested compatibility level during guest initialization.Add support for retrieving host CPU compatibility capabilities for nested guests on PowerVM (PAPR nested API v2). The hypervisor provides the effective compatibility levels via the
H_GUEST_GET_CAPABILITIEShcall, which reflects the processor modes negotiated between the Power hypervisor (L0) and the host partition (L1).On pseries systems, obtain the capability bitmap using
plpar_guest_get_capabilities()and return it via structkvm_ppc_compat_caps. This information is then exposed to userspace through theKVM_PPC_GET_COMPAT_CAPSioctl.Hook the implementation into the Book3S HV
kvmppc_opsso that it can be invoked by the generic KVM ioctl handling code.Suggested-by Vaibhav Jain <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- arch/powerpc/kvm/book3s_hv.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)arch/powerpc/kvm/book3s_hv.c +0
-0@@ -6522,6 +6522,21 @@ static bool kvmppc_hash_v3_possible(void)return true; }
Quoted 14 lines
arch/powerpc/kvm/book3s_hv.c static bool kvmppc_hash_v3_possible(void)
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +9
-0+ +static int kvmppc_get_compat_cpu_caps (struct kvm_ppc_compat_caps *host_caps) +{ +unsigned long capabilities =0 ; +long rc = -EINVAL ; + +if (kvmhv_on_pseries ()) { +if (kvmhv_is_nestedv2 ()) + rc =plpar_guest_get_capabilities (0 , &capabilities);since this value will trikle back to userspace please apply a mask on the hcall return value so that any reserved and non-PVR related bits doesnt leak back to userspace.
Though currently we only supply the bits corresponding to supported processor versions, it makes sense to mask out unrelated bits so that they don't unnecesarily passed on to the userspace. I'll make the changes in v4.
Thanks,
Amit
Quoted 26 lines
Quoted 22 lines
arch/powerpc/kvm/book3s_hv.c +6
-0+ host_caps->compat_capabilities = capabilities; + } + +return rc; +} +static struct kvmppc_ops kvm_ops_hv = { .get_sregs = kvm_arch_vcpu_ioctl_get_sregs_hv,arch/powerpc/kvm/book3s_hv.c +1
-0.set_sregs = kvm_arch_vcpu_ioctl_set_sregs_hv,@@ -6564,6 +6579,7 @@ static struct kvmppc_ops kvm_ops_hv = { .hash_v3_possible = kvmppc_hash_v3_possible, .create_vcpu_debugfs = kvmppc_arch_create_vcpu_debugfs_hv, .create_vm_debugfs = kvmppc_arch_create_vm_debugfs_hv, + .get_compat_cpu_ver = kvmppc_get_compat_cpu_caps, };static int kvm_init_subcore_bitmap (void )Signature · 1 line
2.50.1 (Apple Git-155)Signature · 2 lines
Cheers ~ Vaibhav
PATCH v3 4/5 KVM: PPC: Book3S HV: Add support for compat CPU capabilities for KVM on PowerNV
In reply to Amit MachhiwalCurrently, when booting a compatibility-mode KVM guest (L1) on a PowerNV hypervisor (L0), the guest runs with the expected processor compatibility level. However, when booting a nested KVM guest (L2) inside the L1, QEMU derives the CPU model from the raw host PVR and attempts to run the nested guest at that level, instead of honoring the compatibility mode of the L1.
Extend host CPU compatibility capability reporting to support nested virtualization on PowerNV systems (PAPR nested API v1).
For nested API v2 (PowerVM), compatibility capabilities are obtained from the hypervisor via the H_GUEST_GET_CAPABILITIES hcall. This information is not available on PowerNV systems.
For nested API v1, derive the compatibility capabilities from the L1 guest by reading the "cpu-version" property from the device tree, which reflects the effective (logical) processor compatibility level. Map this value to the corresponding compatibility capability bitmap.
Introduce a helper to translate CPU version values into compatibility capability bits and integrate it into kvmppc_get_compat_cpu_caps().
This allows userspace to query host CPU compatibility modes on both PowerVM and PowerNV platforms via the KVM_PPC_GET_COMPAT_CAPS ioctl.
--- arch/powerpc/kvm/book3s_hv.c | 37 +++++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-)
arch/powerpc/kvm/book3s_hv.c +36 -1
@@ -6522,15 +6522,50 @@ static bool kvmppc_hash_v3_possible(void)return true; } +static int kvmppc_map_compat_capabilities (const __be32 cpu_version, +unsigned long *capabilities) +{ +switch (cpu_version) { +case PVR_ARCH_31_P11 : + *capabilities |=H_GUEST_CAP_POWER11 ; +break ; +case PVR_ARCH_31 : + *capabilities |=H_GUEST_CAP_POWER10 ; +break ; +case PVR_ARCH_300 : + *capabilities |=H_GUEST_CAP_POWER9 ; +break ; +default : +return -EINVAL ; + } + +return 0 ; +}static int kvmppc_get_compat_cpu_caps (struct kvm_ppc_compat_caps *host_caps) { +struct device_node *np;unsigned long capabilities =0 ; +const __be32 *prop =NULL ;long rc = -EINVAL ; +u32 cpu_version;if (kvmhv_on_pseries ()) { -if (kvmhv_is_nestedv2 ()) +if (kvmhv_is_nestedv2 ()) { rc =plpar_guest_get_capabilities (0 , &capabilities); + }else { +for_each_node_by_type (np,"cpu" ) { + prop =of_get_property (np,"cpu-version" ,NULL ); +if (prop) { + cpu_version =be32_to_cpup (prop); +break ; + } + } +if (!prop) +return -EINVAL ; + rc =kvmppc_map_compat_capabilities (cpu_version, + &capabilities); + } host_caps->compat_capabilities = capabilities; }
Signature · 1 line
2.50.1 (Apple Git-155)
Re: [PATCH v3 4/5] KVM: PPC: Book3S HV: Add support for compat CPU capabilities for KVM on PowerNV
Hi Amit,
Thanks for the patch. My review comments inline:
Quoted 63 lines · Amit Machhiwal
Currently, when booting a compatibility-mode KVM guest (L1) on a PowerNV hypervisor (L0), the guest runs with the expected processor compatibility level. However, when booting a nested KVM guest (L2) inside the L1, QEMU derives the CPU model from the raw host PVR and attempts to run the nested guest at that level, instead of honoring the compatibility mode of the L1.
Extend host CPU compatibility capability reporting to support nested virtualization on PowerNV systems (PAPR nested API v1).
For nested API v2 (PowerVM), compatibility capabilities are obtained from the hypervisor via the
H_GUEST_GET_CAPABILITIEShcall. This information is not available on PowerNV systems.For nested API v1, derive the compatibility capabilities from the L1 guest by reading the "cpu-version" property from the device tree, which reflects the effective (logical) processor compatibility level. Map this value to the corresponding compatibility capability bitmap.
Introduce a helper to translate CPU version values into compatibility capability bits and integrate it into
kvmppc_get_compat_cpu_caps().This allows userspace to query host CPU compatibility modes on both PowerVM and PowerNV platforms via the
KVM_PPC_GET_COMPAT_CAPSioctl.Suggested-by Vaibhav Jain <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- arch/powerpc/kvm/book3s_hv.c | 37 +++++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-)arch/powerpc/kvm/book3s_hv.c +20
-0@@ -6522,15 +6522,50 @@ static bool kvmppc_hash_v3_possible(void)return true; } +static int kvmppc_map_compat_capabilities (const __be32 cpu_version, +unsigned long *capabilities) +{ +switch (cpu_version) { +case PVR_ARCH_31_P11 : + *capabilities |=H_GUEST_CAP_POWER11 ; +break ; +case PVR_ARCH_31 : + *capabilities |=H_GUEST_CAP_POWER10 ; +break ; +case PVR_ARCH_300 : + *capabilities |=H_GUEST_CAP_POWER9 ; +break ; +default : +return -EINVAL ; + } + +return 0 ; +}static int kvmppc_get_compat_cpu_caps (struct kvm_ppc_compat_caps *host_caps) { +struct device_node *np;
arch/powerpc/kvm/book3s_hv.c static bool kvmppc_hash_v3_possible(void) · line 6556
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +3
-1unsigned long capabilities =0 ; +const __be32 *prop =NULL ;long rc = -EINVAL ; +u32 cpu_version;if (kvmhv_on_pseries ()) { -if (kvmhv_is_nestedv2 ()) +if (kvmhv_is_nestedv2 ()) { rc =plpar_guest_get_capabilities (0 , &capabilities);
Need to mask capabilities as mentioned in the review comments for previous patch. I would suggest creating a helper that performs the hcall and applies the mask which can then be used at plpar_guest_get_capabilities() call sites.
Quoted 19 lines
arch/powerpc/kvm/book3s_hv.c +13
-0+ }else { +for_each_node_by_type (np,"cpu" ) { + prop =of_get_property (np,"cpu-version" ,NULL ); +if (prop) { + cpu_version =be32_to_cpup (prop); +break ; + } + } +if (!prop) +return -EINVAL ; + rc =kvmppc_map_compat_capabilities (cpu_version, + &capabilities); + } host_caps->compat_capabilities = capabilities; }Signature · 2 lines
-- 2.50.1 (Apple Git-155)
Signature · 2 lines
Cheers ~ Vaibhav
Re: [PATCH v3 4/5] KVM: PPC: Book3S HV: Add support for compat CPU capabilities for KVM on PowerNV
Quoted 74 lines · Vaibhav Jain
Hi Amit,
Thanks for the patch. My review comments inline:
Quoted 68 lines · Amit Machhiwal
Currently, when booting a compatibility-mode KVM guest (L1) on a PowerNV hypervisor (L0), the guest runs with the expected processor compatibility level. However, when booting a nested KVM guest (L2) inside the L1, QEMU derives the CPU model from the raw host PVR and attempts to run the nested guest at that level, instead of honoring the compatibility mode of the L1.
Extend host CPU compatibility capability reporting to support nested virtualization on PowerNV systems (PAPR nested API v1).
For nested API v2 (PowerVM), compatibility capabilities are obtained from the hypervisor via the
H_GUEST_GET_CAPABILITIEShcall. This information is not available on PowerNV systems.For nested API v1, derive the compatibility capabilities from the L1 guest by reading the "cpu-version" property from the device tree, which reflects the effective (logical) processor compatibility level. Map this value to the corresponding compatibility capability bitmap.
Introduce a helper to translate CPU version values into compatibility capability bits and integrate it into
kvmppc_get_compat_cpu_caps().This allows userspace to query host CPU compatibility modes on both PowerVM and PowerNV platforms via the
KVM_PPC_GET_COMPAT_CAPSioctl.Suggested-by Vaibhav Jain <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- arch/powerpc/kvm/book3s_hv.c | 37 +++++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-)arch/powerpc/kvm/book3s_hv.c +22
-0@@ -6522,15 +6522,50 @@ static bool kvmppc_hash_v3_possible(void)return true; } +static int kvmppc_map_compat_capabilities (const __be32 cpu_version, +unsigned long *capabilities) +{ +switch (cpu_version) { +case PVR_ARCH_31_P11 : + *capabilities |=H_GUEST_CAP_POWER11 ; +break ; +case PVR_ARCH_31 : + *capabilities |=H_GUEST_CAP_POWER10 ; +break ; +case PVR_ARCH_300 : + *capabilities |=H_GUEST_CAP_POWER9 ; +break ; +default : +return -EINVAL ; + } + +return 0 ; +}static int kvmppc_get_compat_cpu_caps (struct kvm_ppc_compat_caps *host_caps) { +struct device_node *np;unsigned long capabilities =0 ; +const __be32 *prop =NULL ;long rc = -EINVAL ; +u32 cpu_version;
Quoted 9 lines
arch/powerpc/kvm/book3s_hv.c static bool kvmppc_hash_v3_possible(void)
Quoted 5 lines
arch/powerpc/kvm/book3s_hv.c +1
-1if (kvmhv_on_pseries ()) { -if (kvmhv_is_nestedv2 ()) +if (kvmhv_is_nestedv2 ()) { rc =plpar_guest_get_capabilities (0 , &capabilities);Need to mask capabilities as mentioned in the review comments for previous patch. I would suggest creating a helper that performs the hcall and applies the mask which can then be used at
plpar_guest_get_capabilities()call sites.
Sure, will do.
Thanks,
Amit
Quoted 24 lines
Quoted 20 lines
arch/powerpc/kvm/book3s_hv.c +13
-0+ }else { +for_each_node_by_type (np,"cpu" ) { + prop =of_get_property (np,"cpu-version" ,NULL ); +if (prop) { + cpu_version =be32_to_cpup (prop); +break ; + } + } +if (!prop) +return -EINVAL ; + rc =kvmppc_map_compat_capabilities (cpu_version, + &capabilities); + } host_caps->compat_capabilities = capabilities; }Signature · 2 lines
-- 2.50.1 (Apple Git-155)Signature · 2 lines
Cheers ~ Vaibhav
PATCH v3 5/5 KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl
In reply to Amit MachhiwalAdd documentation for the KVM_PPC_GET_COMPAT_CAPS ioctl to the KVM API documentation.
The ioctl exposes host processor compatibility modes supported for nested KVM guests on PowerPC systems.
--- Documentation/virt/kvm/api.rst | 35 ++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
Documentation/virt/kvm/api.rst +35 -0
@@ -6555,6 +6555,41 @@ KVM_S390_KEYOP_SSKE .. _kvm_run: +4.145 KVM_PPC_GET_COMPAT_CAPS +----------------------------- +:Capability: KVM_CAP_PPC_COMPAT_CAPS +:Architectures: powerpc +:Type: vm ioctl +:Parameters: struct kvm_ppc_compat_caps (out) +:Returns: + 0 on successful completion, + -EFAULT if ``struct kvm_ppc_compat_caps`` cannot be written + +IBM POWER system server-based processors provide a compatibility mode feature +where an Nth generation processor can operate in modes consistent with earlier +generations such as (N-1) and (N-2). + +This ioctl provides userspace with information about the CPU compatibility modes +supported by the current host processor for booting the nested KVM guests on +PowerNV (KVM nested APIv1) and PowerVM (KVM nested APIv2) platforms. + +:: + + struct kvm_ppc_compat_caps { + __u64 flags; /* Reserved for future use */ + __u64 compat_capabilities; /* Capabilities supported by the host */ + }; + +The ``compat_capabilities`` bit field describes the processor compatibility +modes supported by the host. For example, the following bits indicate support +for specific processor modes. + +:: + + H_GUEST_CAP_POWER9 (bit 1): KVM guests can run in Power9 processor mode + H_GUEST_CAP_POWER10 (bit 2): KVM guests can run in Power10 processor mode + H_GUEST_CAP_POWER11 (bit 3): KVM guests can run in Power11 processor mode + 5. The kvm_run structure ========================
Signature · 1 line
2.50.1 (Apple Git-155)
[email protected]
AI reviewRe: [PATCH v3 5/5] KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
New issues:
- [Medium] Macros referenced in the user-facing API documentation are missing from UAPI headers.
- [Medium] The documentation uses ambiguous and non-standard bit numbering (MSB-0), likely causing userspace to check the wrong bits.
- [Low] The new documentation section breaks the existing .. _kvm_run: reStructuredText anchor target.
- [Low] The documented return values omit multiple valid error codes natively returned by the ioctl (-EINVAL, -ENOTTY).
Pre-existing issues:
- [High] kvmppc_map_compat_capabilities under-reports supported compatibility modes on APIv1 platforms. --
commit 72ed781761ff2f69a81eff7405946f0c54c590e0
Author: Amit Machhiwal <[email protected]>
KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl
This commit adds documentation for the KVM_PPC_GET_COMPAT_CAPS ioctl to the KVM API documentation. It describes the compatibility modes supported for nested KVM guests on PowerPC systems.
Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6559
Quoted 10 lines
Documentation/virt/kvm/api.rst +2
-0@@ -6555,6 +6555,41 @@ KVM_S390_KEYOP_SSKE .. _kvm_run: +4.145 KVM_PPC_GET_COMPAT_CAPS +-----------------------------
[Severity: Low]
Does inserting the new section here break the rST anchor target?
By placing the new section directly after the .. _kvm_run: anchor, cross references to the kvm_run anchor throughout the documentation will jump to this new 4.145 section instead of section 5.
Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6566
Quoted 7 lines
Documentation/virt/kvm/api.rst +7
-0+:Capability: KVM_CAP_PPC_COMPAT_CAPS +:Architectures: powerpc +:Type: vm ioctl +:Parameters: struct kvm_ppc_compat_caps (out) +:Returns: + 0 on successful completion, + -EFAULT if ``struct kvm_ppc_compat_caps`` cannot be written
[Severity: Low]
Should this documentation also include -EINVAL and -ENOTTY?
The kernel implementation explicitly returns -ENOTTY if the underlying architecture callback is unsupported, and returns -EINVAL if capabilities cannot be mapped or the CPU version cannot be parsed.
Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6574
Quoted 8 lines
Documentation/virt/kvm/api.rst +8
-0+ +IBM POWER system server-based processors provide a compatibility mode feature +where an Nth generation processor can operate in modes consistent with earlier +generations such as (N-1) and (N-2). + +This ioctl provides userspace with information about the CPU compatibility modes +supported by the current host processor for booting the nested KVM guests on +PowerNV (KVM nested APIv1) and PowerVM (KVM nested APIv2) platforms.
[Severity: High]
This isn't a bug introduced by this patch, but does the APIv1 implementation of kvmppc_map_compat_capabilities under-report supported compatibility modes?
The underlying implementation for APIv1 nested KVM uses a switch statement with breaks that only sets a single bit matching the host's native generation.
Because backward compatibility modes are missed, QEMU might erroneously conclude the host does not support older guests.
Quoted 7 lines
Documentation/virt/kvm/api.rst +7
-0+ +:: + + struct kvm_ppc_compat_caps { + __u64 flags; /* Reserved for future use */ + __u64 compat_capabilities; /* Capabilities supported by the host */ + };
Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6591
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+ +The ``compat_capabilities`` bit field describes the processor compatibility +modes supported by the host. For example, the following bits indicate support +for specific processor modes. + +:: + + H_GUEST_CAP_POWER9 (bit 1): KVM guests can run in Power9 processor mode + H_GUEST_CAP_POWER10 (bit 2): KVM guests can run in Power10 processor mode + H_GUEST_CAP_POWER11 (bit 3): KVM guests can run in Power11 processor mode
[Severity: Medium]
Are these macros exported to include/uapi/linux/kvm.h?
The documentation directs userspace to interpret the bitmask using macros like H_GUEST_CAP_POWER9. However, these are defined exclusively in the kernel-internal header arch/powerpc/include/asm/hvcall.h. This forces userspace to hardcode internal kernel values.
[Severity: Medium]
Will the bit numbering here mislead userspace developers?
The documentation specifies capabilities as (bit 1), (bit 2), etc., which implies standard LSB-0 numbering. However, the KVM implementation uses IBM's MSB-0 bit numbering, where H_GUEST_CAP_POWER9 is defined as (1UL << (63 - 1)).
Quoted 3 lines
Documentation/virt/kvm/api.rst +0
-05. The kvm_run structure ========================
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Re: [PATCH v3 5/5] KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl
Quoted 36 lines · On 2026/05/22 05:56 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
New issues:
- [Medium] Macros referenced in the user-facing API documentation are missing from UAPI headers.
- [Medium] The documentation uses ambiguous and non-standard bit numbering (MSB-0), likely causing userspace to check the wrong bits.
- [Low] The new documentation section breaks the existing.. _kvm_run:reStructuredText anchor target.
- [Low] The documented return values omit multiple valid error codes natively returned by the ioctl (-EINVAL,-ENOTTY).Pre-existing issues:
- [High]kvmppc_map_compat_capabilitiesunder-reports supported compatibility modes on APIv1 platforms. --commit
72ed781761ff2f69a81eff7405946f0c54c590e0
Author: Amit Machhiwal <[email protected]>
KVM: PPC: DocumentKVM_PPC_GET_COMPAT_CAPSioctlThis commit adds documentation for the
KVM_PPC_GET_COMPAT_CAPSioctl to the KVM API documentation. It describes the compatibility modes supported for nested KVM guests on PowerPC systems.Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6559
Quoted 10 lines
Documentation/virt/kvm/api.rst +2
-0@@ -6555,6 +6555,41 @@ KVM_S390_KEYOP_SSKE .. _kvm_run: +4.145 KVM_PPC_GET_COMPAT_CAPS +-----------------------------[Severity: Low]
Does inserting the new section here break the rST anchor target?
Quoted 3 lines
By placing the new section directly after the ..
_kvm_run: anchor, cross references to thekvm_runanchor throughout the documentation will jump to this new 4.145 section instead of section 5.
Yes, it got overlooked. I'll fix this is next version.
Quoted 11 lines
Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6566
Quoted 8 lines
Documentation/virt/kvm/api.rst +7
-0+:Capability: KVM_CAP_PPC_COMPAT_CAPS +:Architectures: powerpc +:Type: vm ioctl +:Parameters: struct kvm_ppc_compat_caps (out) +:Returns: + 0 on successful completion, + -EFAULT if ``struct kvm_ppc_compat_caps`` cannot be written[Severity: Low]
Should this documentation also include-EINVALand-ENOTTY?
Yes, I missed to include -ENOTTY from the change in my previous version. Looked more closely and we do return -EINVAL from one path. I'll document that too in the next version.
Quoted 17 lines
The kernel implementation explicitly returns
-ENOTTYif the underlying architecture callback is unsupported, and returns-EINVALif capabilities cannot be mapped or the CPU version cannot be parsed.Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6574
Quoted 8 lines
Documentation/virt/kvm/api.rst +8
-0+ +IBM POWER system server-based processors provide a compatibility mode feature +where an Nth generation processor can operate in modes consistent with earlier +generations such as (N-1) and (N-2). + +This ioctl provides userspace with information about the CPU compatibility modes +supported by the current host processor for booting the nested KVM guests on +PowerNV (KVM nested APIv1) and PowerVM (KVM nested APIv2) platforms.[Severity: High]
This isn't a bug introduced by this patch, but does the APIv1 implementation ofkvmppc_map_compat_capabilitiesunder-report supported compatibility modes?
Quoted 5 lines
The underlying implementation for APIv1 nested KVM uses a switch statement with breaks that only sets a single bit matching the host's native generation.
Because backward compatibility modes are missed, QEMU might erroneously conclude the host does not support older guests.
No, its currenlty intentionally done for APIv1. I had responded to a similar query here:
https://lore.kernel.org/all/[email protected]/
Quoted 22 lines
Quoted 8 lines
Documentation/virt/kvm/api.rst +7
-0+ +:: + + struct kvm_ppc_compat_caps { + __u64 flags; /* Reserved for future use */ + __u64 compat_capabilities; /* Capabilities supported by the host */ + };Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6591
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+ +The ``compat_capabilities`` bit field describes the processor compatibility +modes supported by the host. For example, the following bits indicate support +for specific processor modes. + +:: + + H_GUEST_CAP_POWER9 (bit 1): KVM guests can run in Power9 processor mode + H_GUEST_CAP_POWER10 (bit 2): KVM guests can run in Power10 processor mode + H_GUEST_CAP_POWER11 (bit 3): KVM guests can run in Power11 processor mode[Severity: Medium]
Are these macros exported toinclude/uapi/linux/kvm.h?
Quoted 4 lines
The documentation directs userspace to interpret the bitmask using macros like
H_GUEST_CAP_POWER9. However, these are defined exclusively in the kernel-internal headerarch/powerpc/include/asm/hvcall.h. This forces userspace to hardcode internal kernel values.
These macro names (H_GUEST_CAP_*) are for documentation clarity only. Userspace should use the bit numbers directly as these macros are not exported to UAPI headers.
Quoted 7 lines
[Severity: Medium]
Will the bit numbering here mislead userspace developers?The documentation specifies capabilities as (bit 1), (bit 2), etc., which implies standard LSB-0 numbering. However, the KVM implementation uses IBM's MSB-0 bit numbering, where
H_GUEST_CAP_POWER9is defined as (1UL << (63 - 1)).
I'll add a note clarifying that IBM PowerPC uses MSB-0 bit numbering (bit 0 = most significant), so bit 1 corresponds to (1UL << (63 -1 )).
Thanks,
Amit
Quoted 6 lines
Quoted 3 lines
Documentation/virt/kvm/api.rst +0
-05. The kvm_run structure ========================Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5