KVM: PPC: Expose CPU compatibility modes for nested guests
Subsystems (4)
v7 cover KVM: PPC: Expose CPU compatibility modes for nested guests
On POWER systems, newer processor generations can operate in compatibility modes corresponding to earlier generations (e.g., a Power11 system running in Power10 compatibility mode). In such cases, the effective CPU level exposed to guests differs from the physical processor generation.
This creates a problem for nested virtualization. When booting a nested KVM guest (L2) inside a host KVM guest (L1) running in a compatibility mode, userspace (e.g., QEMU) may derive the CPU model from the raw hardware PVR and attempt to configure the nested guest accordingly. However, the L1 partition is constrained by the compatibility level negotiated with the hypervisor (L0), and requests exceeding that level are rejected, leading to guest boot failures such as:
KVM-NESTEDv2: couldn't set guest wide elements
This series provides a mechanism for userspace to query the effective CPU compatibility modes supported by the host, so it can select an appropriate CPU model for nested guests.
To achieve this, the series introduces a new KVM capability and ioctl (KVM_CAP_PPC_COMPAT_CAPS / KVM_PPC_GET_COMPAT_CAPS) that expose the compatibility modes supported by the host.
Why a new UAPI?
===============
While cpu-version is available in /proc/device-tree/cpus/<cpu#>/cpu-version on both L1 booted on PowerNV and PowerVM LPARs, the UAPI approach is preferable for several reasons:
1. pHYP (L0) capabilities: On PowerVM, we need to rely on capabilities negotiated with pHYP in KVM, not just device tree properties. The cpu-version property depicts the current compat mode but doesn't point to what all compat modes are supported for the nested guest.
2. procfs dependency: Not all systems run with procfs enabled (CONFIG_PROC_FS is optional). Minimal configurations like buildroot might disable it, but KVM ioctl works regardless since it accesses kernel data structures directly.
3. Kernel validation: The kernel validates and normalizes the compatibility information, ensuring userspace gets validated, consistent data.
4. Abstraction & stability: /proc/device-tree is an implementation detail. The UAPI provides a stable interface that won't break if the underlying mechanism changes.
5. Semantic clarity: KVM_PPC_GET_COMPAT_CAPS clearly expresses what compatibility modes can be used for KVM guests, vs. parsing device tree which requires understanding the semantic meaning of cpu-version.
The implementation supports both:
- KVM on PowerVM (nested API v2), where compatibility information is served from the cached nested_capabilities value, originally obtained via the H_GUEST_GET_CAPABILITIES hypercall at module init.
- KVM on PowerNV (nested API v1), where compatibility is derived from the device tree ("cpu-version") representing the effective processor compatibility level.
This allows userspace (e.g., QEMU) to select a CPU model consistent with the host compatibility mode, avoiding mismatches and enabling successful nested guest boot.
Note: This series is built on top of patch [1] which must be applied first. Patch [1] ensures arch_compat is validated against the host compatibility mode before this series adds the capability query mechanism. Commit e4de1b9cb3b5 ("powerpc/dt_cpu_ftrs: Set CPU_FTR_P11_PVR for Power11 and later processors") which was also a prerequisite has been merged upstream.
Changes in v7:
- KVM_CAP_PPC_COMPAT_CAPS: add hv_enabled guard to align the capability check with ioctl availability; a PR KVM guest on pseries now correctly returns 0 for the capability [Sashiko]
Patch summary:
[1/4] Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl [2/4] Implement capability retrieval for KVM on PowerVM (API v2) [3/4] Add KVM on PowerNV support (API v1)
[4/4] Document the new ioctl
Testing (with QEMU v5 patches and on top of patch [1]):
KVM APIv1 Testing
=================
On P10 PowerNV machine (L0)
---------------------------
- P10 L1 KVM guest -> works
- P10 nested L2 KVM guest -> works
- P9 compat nested L2 KVM guest -> works
- P9 compat L1 KVM guest -> works
- P9 nested L2 KVM guest -> works
On Powernv11 TCG Guest (L0)
---------------------------
- P11 PowerNV TCG L0 guest -> works
- P11 L1 KVM guest -> works
- P11 L2 KVM guest -> works
- P10 compat L1 KVM guest -> works
- P10 L2 KVM guest -> works
- P9 compat L1 KVM guest -> works
- P9 L2 KVM guest -> works
KVM APIv2 Testing
=================
On P11 PowerVM LPAR (L1)
------------------------
- P11 L2 KVM guest -> works
- P10 compat L2 KVM guest -> works
- P9 compat L2 KVM guest fails to boot as expected
- Without QEMU patches but Linux patches
- P11 L2 KVM guest -> works
- P10 compat L2 KVM guest -> works
- P9 compat L2 KVM guest fails to boot as expected
- Without Linux patches but QEMU patches
- P11 L2 KVM guest -> works
- P10 compat L2 KVM guest -> works
On P11 LPAR in P10 compat (L1)
------------------------------
- P10 (host compat) L2 KVM guest -> works
- Without QEMU patch but Linux patches
- P10 guest fails to boot as expected (error: kvm run failed Invalid argument)
- Without Linux patch but QEMU patches
- P10 guest fails to boot as expected (KVM: unknown exit, hardware reason ffffffffffffffea)
On P10 PowerVM LPAR (L1)
------------------------
- P10 L2 KVM guest -> works
- P9 compat L2 KVM guest fails to boot as expected
TCG pSeries Guest
=================
- P11 (default) pSeries guest boots fine
ABI Extensibility Testing (struct size 32, extra member)
=========================================================
- Newer struct on QEMU, older kernel -> works (kernel returns -E2BIG, QEMU retries with correct size)
- New struct on Linux kernel, older QEMU -> works (kernel zero-pads trailing fields, QEMU gets correct data)
With this series, nested guests boot successfully in configurations where they previously failed due to compatibility mismatches.
Related QEMU series:
====================
QEMU v5 series:
https://lore.kernel.org/all/[email protected]/
Previous QEMU versions:
v4: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/
Previous versions:
==================
v6: https://lore.kernel.org/linuxppc-dev/[email protected]/
v5: https://lore.kernel.org/linuxppc-dev/[email protected]/
v4: https://lore.kernel.org/linuxppc-dev/[email protected]/
v3: https://lore.kernel.org/linuxppc-dev/[email protected]/
v2: https://lore.kernel.org/linuxppc-dev/[email protected]/
v1: https://lore.kernel.org/linuxppc-dev/[email protected]/
References:
===========
[1] https://lore.kernel.org/all/[email protected]/
Amit Machhiwal (4):
KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM
on PowerVM
KVM: PPC: Book3S HV: Add support for compat CPU capabilities for KVM
on PowerNV
KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl
Documentation/virt/kvm/api.rst | 79 +++++++++++++++++++++++++++++ arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/include/uapi/asm/kvm.h | 18 +++++++ arch/powerpc/kvm/book3s_hv.c | 56 ++++++++++++++++++++ arch/powerpc/kvm/powerpc.c | 71 ++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 3 ++ 6 files changed, 228 insertions(+)
base-commit: fcaeecb8b0cd44f77d03b28de0671258d4db18f8
prerequisite-patch-id: 7755786f0e4f415e47065ff1972765008727fe10
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH v7 1/4 KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
Introduce a new capability and ioctl to expose CPU compatibility modes supported by the host processor for nested guests.
On IBM POWER systems, newer processor generations (N) can operate in compatibility modes corresponding to earlier generations, like (N-1) and (N-2). This is particularly relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Introduce KVM_CAP_PPC_COMPAT_CAPS capability and the corresponding KVM_PPC_GET_COMPAT_CAPS vm ioctl. The ioctl returns a bitmap describing the compatibility modes supported by the host in respective bit numbers, allowing userspace (e.g., QEMU) to select an appropriate compatibility level when configuring nested KVM guests.
The ioctl handling is added in kvm_arch_vm_ioctl() and retrieves host CPU compatibility capabilities via a PowerPC-specific backend implementation when available.
The struct kvm_ppc_compat_caps places the 'size' field first so it can be read alone via get_user() before copy_struct_from_user() is called, avoiding pointer arithmetic to locate the size field.
The ioctl is defined using _IO so the ioctl number remains stable even if the struct grows in future versions. It uses copy_struct_from_user() and copy_struct_to_user() to provide forward- and backward-compatible
extensibility: older userspace passing a smaller struct to a newer kernel gets zero-padded trailing fields, while newer userspace passing a larger struct to an older kernel (usize > ksize) gets sizeof(struct kvm_ppc_compat_caps) written back to host_caps.size so it can retry with the older kernel-supported size, after which the kernel returns -E2BIG.
KVM_PPC_COMPAT_CAPS_SIZE_VER0 is defined as a frozen integer constant
(24) marking the size of the initial struct version, used as the minimum floor for size field validation, similar to other versioned struct interfaces in the kernel.
The 'flags' field is reserved for future use. The kernel rejects any call where flags is non-zero with -EINVAL, preventing garbage values from being baked into ABI permanently.
The ioctl returns appropriate error codes: EINVAL for an invalid size or non-zero reserved fields, E2BIG if new userspace provides a larger struct than the kernel knows about (with ksize written back into host_caps.size for the retry), EFAULT for failed copy operations, and ENOTTY if the backend doesn't implement get_compat_caps.
---
Changes in this version:
- KVM_CAP_PPC_COMPAT_CAPS: add hv_enabled guard to align the capability check with ioctl availability; a PR KVM VM on pseries now correctly returns 0 for the capability [Sashiko]
arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/include/uapi/asm/kvm.h | 8 ++++ arch/powerpc/kvm/powerpc.c | 71 +++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 3 ++ 4 files changed, 83 insertions(+)
arch/powerpc/include/asm/kvm_ppc.h +1 -0
@@ -319,6 +319,7 @@ struct kvmppc_ops {bool (*hash_v3_possible)(void );int (*create_vm_debugfs)(struct kvm *kvm);int (*create_vcpu_debugfs)(struct kvm_vcpu *vcpu,struct dentry *debugfs_dentry); +int (*get_compat_caps)(struct kvm_ppc_compat_caps *host_caps); };extern struct kvmppc_ops *kvmppc_hv_ops;
arch/powerpc/include/uapi/asm/kvm.h +8 -0
@@ -437,6 +437,14 @@ struct kvm_ppc_cpu_char {__u64 behaviour_mask;/* valid bits in behaviour */ }; +/* For KVM_PPC_GET_COMPAT_CAPS */ +struct kvm_ppc_compat_caps { +__u64 size;/* Size of this structure */ +__u64 flags;/* Reserved for future use */ +__u64 compat_capabilities;/* Capabilities supported by the host */ +}; +#define KVM_PPC_COMPAT_CAPS_SIZE_VER0 24/* sizeof first published struct */ +/* * Values for character and character_mask. * These are identical to the values used by H_GET_CPU_CHARACTERISTICS.
arch/powerpc/kvm/powerpc.c +71 -0
83 lines not on this page · raw message
include/uapi/linux/kvm.h +3 -0
@@ -997,6 +997,7 @@ struct kvm_enable_cap {#define KVM_CAP_S390_KEYOP 247#define KVM_CAP_S390_VSIE_ESAMODE 248#define KVM_CAP_S390_HPAGE_2G 249 +#define KVM_CAP_PPC_COMPAT_CAPS 250struct kvm_irq_routing_irqchip {__u32 irqchip;
@@ -1341,6 +1342,8 @@ struct kvm_s390_keyop {/* Available with KVM_CAP_COUNTER_OFFSET */ #define KVM_ARM_SET_COUNTER_OFFSET _IOW(KVMIO, 0xb5, struct kvm_arm_counter_offset)#define KVM_ARM_GET_REG_WRITABLE_MASKS _IOR(KVMIO, 0xb6, struct reg_mask_range) +/* Available with KVM_CAP_PPC_COMPAT_CAPS */ +#define KVM_PPC_GET_COMPAT_CAPS _IO(KVMIO, 0xb8)/* ioctl for vm fd */ #define KVM_CREATE_DEVICE _IOWR(KVMIO, 0xe0, struct kvm_create_device)
Signature · 1 line
2.50.1 (Apple Git-155)
[email protected]
AI reviewRe: [PATCH v7 1/4] KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [High] KVM_CAP_PPC_COMPAT_CAPS capability is advertised but the ioctl backend is completely…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] KVM_CAP_PPC_COMPAT_CAPS capability is advertised but the ioctl backend is completely unimplemented, causing unconditional failure.
- [High] The KVM_PPC_GET_COMPAT_CAPS ioctl overwrites the userspace buffer's size field with a value larger than the allocated buffer, creating an out-of-bounds read risk in userspace.
- [Medium] The manual validation block explicitly rejecting usize > sizeof(host_caps) defeats the standard forward-compatibility guarantee of copy_struct_from_user(). --
--- Patch [1]: [PATCH v7 1/4] KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl --- commit 8c51e17a26cbffd727576c50f93a34eea2340547
Author: Amit Machhiwal <[email protected]>
KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
Introduce a new capability and ioctl to expose CPU compatibility modes supported by the host processor for nested guests.
Quoted 4 lines
arch/powerpc/kvm/powerpc.c +0
-0
[ ... ]
Quoted 1 line
arch/powerpc/kvm/powerpc.c +0
-0@@ -2463,6 +2470,70 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
[ ... ]
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2513
Quoted 13 lines
arch/powerpc/kvm/powerpc.c +13
-0+/* +* New userspace with a larger struct called an older kernel. +* Write back ksize in host_caps.size so userspace knows which +* older struct to retry with, then fail with -E2BIG. +*/ +if (usize >sizeof (host_caps)) { + host_caps.size =sizeof (host_caps); + r = -EFAULT ; +if (put_user (host_caps.size, (__u64 __user *)argp)) + goto out; + r = -E2BIG ; + goto out; + }
[Severity: Medium]
Does this manual size check defeat the standard forward-compatibility guarantee of copy_struct_from_user()?
Normally, copy_struct_from_user() verifies that any trailing bytes provided by a newer userspace are zero. By short-circuiting here and unconditionally returning -E2BIG, aren't we rejecting perfectly valid, zero-initialized structs and deviating from standard kernel extensible UAPI patterns (like clone3 or openat2)?
[ ... ]
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2536
Quoted 7 lines
arch/powerpc/kvm/powerpc.c +7
-0+ r = -ENOTTY ; +if (!kvm->arch.kvm_ops->get_compat_caps) + goto out; + + r = kvm->arch.kvm_ops->get_compat_caps (&host_caps); +if (r) + goto out;
[Severity: High]
Will this ioctl always fail with -ENOTTY?
The capability KVM_CAP_PPC_COMPAT_CAPS is advertised as supported to userspace (returning 1 on pseries hosts in kvm_vm_ioctl_check_extension()), but the get_compat_caps backend function pointer is never assigned anywhere in this patch.
If userspace attempts to use this ioctl based on the capability check, won't it unconditionally hit this missing backend check and fail, causing regressions in userspace startup flows?
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2542
Quoted 6 lines
arch/powerpc/kvm/powerpc.c +6
-0+ + host_caps.size =sizeof (host_caps); + r =copy_struct_to_user (argp, usize, &host_caps, +sizeof (host_caps),NULL ); +break ; + }
[Severity: High]
Is it safe to overwrite the userspace size field with the kernel's internal struct size?
In a forward-compatibility scenario where an older userspace passes a smaller struct to a newer kernel, usize will be smaller than sizeof(host_caps). By setting host_caps.size = sizeof(host_caps) and copying it back, the userspace struct will receive a larger size value than it actually allocated.
If userspace trusts this size field for bounds checking or iteration, could this regression lead to an out-of-bounds memory read in userspace? Shouldn't the kernel preserve the originally requested usize?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ritesh Harjani (IBM)
reviewerRe: [PATCH v7 1/4] KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
In reply to Amit MachhiwalQuoted 100 lines · Amit Machhiwal
Introduce a new capability and ioctl to expose CPU compatibility modes supported by the host processor for nested guests.
On IBM POWER systems, newer processor generations (N) can operate in compatibility modes corresponding to earlier generations, like (N-1) and (N-2). This is particularly relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Introduce
KVM_CAP_PPC_COMPAT_CAPScapability and the correspondingKVM_PPC_GET_COMPAT_CAPSvm ioctl. The ioctl returns a bitmap describing the compatibility modes supported by the host in respective bit numbers, allowing userspace (e.g., QEMU) to select an appropriate compatibility level when configuring nested KVM guests.The ioctl handling is added in
kvm_arch_vm_ioctl()and retrieves host CPU compatibility capabilities via a PowerPC-specific backend implementation when available.The struct
kvm_ppc_compat_capsplaces the 'size' field first so it can be read alone viaget_user()beforecopy_struct_from_user()is called, avoiding pointer arithmetic to locate the size field.The ioctl is defined using
_IOso the ioctl number remains stable even if the struct grows in future versions. It usescopy_struct_from_user()andcopy_struct_to_user()to provide forward- and backward-compatible
extensibility: older userspace passing a smaller struct to a newer kernel gets zero-padded trailing fields, while newer userspace passing a larger struct to an older kernel (usize > ksize) getssizeof(struct kvm_ppc_compat_caps)written back to host_caps.size so it can retry with the older kernel-supported size, after which the kernel returns-E2BIG.
KVM_PPC_COMPAT_CAPS_SIZE_VER0is defined as a frozen integer constant
(24) marking the size of the initial struct version, used as the minimum floor for size field validation, similar to other versioned struct interfaces in the kernel.The 'flags' field is reserved for future use. The kernel rejects any call where flags is non-zero with
-EINVAL, preventing garbage values from being baked into ABI permanently.The ioctl returns appropriate error codes: EINVAL for an invalid size or non-zero reserved fields, E2BIG if new userspace provides a larger struct than the kernel knows about (with ksize written back into host_caps.size for the retry), EFAULT for failed copy operations, and ENOTTY if the backend doesn't implement
get_compat_caps.Suggested-by Vaibhav Jain <[email protected]>Tested-by Gautam Menghani <[email protected]>Reviewed-by Gautam Menghani <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>---
Changes in this version:
-KVM_CAP_PPC_COMPAT_CAPS: addhv_enabledguard to align the capability check with ioctl availability; a PR KVM VM on pseries now correctly returns 0 for the capability [Sashiko]arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/include/uapi/asm/kvm.h | 8 ++++ arch/powerpc/kvm/powerpc.c | 71 +++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 3 ++ 4 files changed, 83 insertions(+)arch/powerpc/include/asm/kvm_ppc.h +1
-0@@ -319,6 +319,7 @@ struct kvmppc_ops {bool (*hash_v3_possible)(void );int (*create_vm_debugfs)(struct kvm *kvm);int (*create_vcpu_debugfs)(struct kvm_vcpu *vcpu,struct dentry *debugfs_dentry); +int (*get_compat_caps)(struct kvm_ppc_compat_caps *host_caps); };extern struct kvmppc_ops *kvmppc_hv_ops;arch/powerpc/include/uapi/asm/kvm.h +8
-0@@ -437,6 +437,14 @@ struct kvm_ppc_cpu_char {__u64 behaviour_mask;/* valid bits in behaviour */ }; +/* For KVM_PPC_GET_COMPAT_CAPS */ +struct kvm_ppc_compat_caps { +__u64 size;/* Size of this structure */ +__u64 flags;/* Reserved for future use */ +__u64 compat_capabilities;/* Capabilities supported by the host */ +}; +#define KVM_PPC_COMPAT_CAPS_SIZE_VER0 24/* sizeof first published struct */ +/* * Values for character and character_mask. * These are identical to the values used by H_GET_CPU_CHARACTERISTICS. arch/powerpc/kvm/powerpc.c +0
-0
arch/powerpc/kvm/powerpc.c int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · line 709
Quoted 7 lines
arch/powerpc/kvm/powerpc.c +4
-0@@ -703,6 +703,13 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) } }break ; +#if defined(CONFIG_KVM_BOOK3S_HV_POSSIBLE ) +case KVM_CAP_PPC_COMPAT_CAPS : + r =0 ; +if (hv_enabled &&kvmhv_on_pseries ())
I think sashiko is just complaining in the 1st patch because we have not yet wired up the kvmppc_hv_ops->get_compat_caps() yet in patch-1. I think it is expecting..
if (hv_enabled && kvmhv_on_pseries() && kvmppc_hv_ops->get_compat_caps)
But either way is fine.
Quoted 35 lines
arch/powerpc/kvm/powerpc.c +28
-0+ r =1 ; +break ; +#endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */ default : r =0 ;break ;@@ -2469,6 +2476,70 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = kvm->arch.kvm_ops->svm_off (kvm);break ; } +case KVM_PPC_GET_COMPAT_CAPS : { +struct kvm_ppc_compat_caps host_caps = {}; +u64 usize; + +/* +* Read the size field first to drive copy_struct_from_user. +* size must be the first field of the struct. +*/ + r = -EFAULT ; +if (get_user (usize, (__u64 __user *)argp)) + goto out; + +/* +* Enforce a minimum: reject buffers smaller than the initial +* struct version (VER0). This allows old userspace compiled +* against the original struct to still work on a newer kernel +* that has grown the struct with appended fields. +*/ + r = -EINVAL ; +if (usize <KVM_PPC_COMPAT_CAPS_SIZE_VER0 ) + goto out; + +/* +* New userspace with a larger struct called an older kernel. +* Write back ksize in host_caps.size so userspace knows which
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2513
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +10
-0+* older struct to retry with, then fail with -E2BIG. +*/ +if (usize >sizeof (host_caps)) { + host_caps.size =sizeof (host_caps); + r = -EFAULT ; +if (put_user (host_caps.size, (__u64 __user *)argp)) + goto out; + r = -E2BIG ; + goto out; + }
You anyways mentioned copy_struct_from_user() is taking care of both forward and backward compat. Then what is the point of this check? shouldn't we get rid of this complete if logic? I don't see a point of this if we are anyway using copy_struct_from_user().
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2518
Quoted 5 lines
arch/powerpc/kvm/powerpc.c +5
-0+ +/* +* copy_struct_from_user() handles forward/backward compat: +* usize == ksize: verbatim copy +* usize < ksize: zero-pad trailing (old userspace, new kernel)
+ * usize > ksize: succeed iff the trailing bytes userspace
+ * sent are zero, else -E2BIG
shouldn't we add usize > ksize case details, like ^^^, which copy_struct_from_user() handles since we are already adding the details of other 2 cases.
</btw snip from copy_struct_from_user kdoc>
* There are three cases to consider:
* * If @usize == @ksize, then it's copied verbatim.
* * If @usize < @ksize, then the userspace has passed an old struct to a
* newer kernel. The rest of the trailing bytes in @dst (@ksize - @usize)
* are to be zero-filled.
* * If @usize > @ksize, then the userspace has passed a new struct to an
* older kernel. The trailing bytes unknown to the kernel (@usize - @ksize)
* are checked to ensure they are zeroed, otherwise -E2BIG is returned.
*
* Returns (in all cases, some data may have been copied):
* * -E2BIG: (@usize > @ksize) and there are non-zero trailing bytes in @src.
* * -EFAULT: access to userspace failed.
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +10
-0+*/ + r =copy_struct_from_user (&host_caps,sizeof (host_caps), + argp, usize); +if (r) + goto out; + +/* Reserved fields must be zero */ + r = -EINVAL ; +if (host_caps.flags) + goto out;
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2538
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +10
-0+ + r = -ENOTTY ; +if (!kvm->arch.kvm_ops->get_compat_caps) + goto out; + + r = kvm->arch.kvm_ops->get_compat_caps (&host_caps); +if (r) + goto out; + + host_caps.size =sizeof (host_caps);
shouldn't this be, since we don't want to be reporting a larger size to the user?
host_caps.size = min_t(u64, usize, sizeof(host_caps));
Quoted 30 lines
arch/powerpc/kvm/powerpc.c +4
-0+ r =copy_struct_to_user (argp, usize, &host_caps, +sizeof (host_caps),NULL ); +break ; + }default : {struct kvm *kvm = filp->private_data; r = kvm->arch.kvm_ops->arch_vm_ioctl (filp, ioctl, arg);include/uapi/linux/kvm.h +3
-0@@ -997,6 +997,7 @@ struct kvm_enable_cap {#define KVM_CAP_S390_KEYOP 247#define KVM_CAP_S390_VSIE_ESAMODE 248#define KVM_CAP_S390_HPAGE_2G 249 +#define KVM_CAP_PPC_COMPAT_CAPS 250struct kvm_irq_routing_irqchip {__u32 irqchip;@@ -1341,6 +1342,8 @@ struct kvm_s390_keyop {/* Available with KVM_CAP_COUNTER_OFFSET */ #define KVM_ARM_SET_COUNTER_OFFSET _IOW(KVMIO, 0xb5, struct kvm_arm_counter_offset)#define KVM_ARM_GET_REG_WRITABLE_MASKS _IOR(KVMIO, 0xb6, struct reg_mask_range) +/* Available with KVM_CAP_PPC_COMPAT_CAPS */ +#define KVM_PPC_GET_COMPAT_CAPS _IO(KVMIO, 0xb8)/* ioctl for vm fd */ #define KVM_CREATE_DEVICE _IOWR(KVMIO, 0xe0, struct kvm_create_device)Signature · 1 line
2.50.1 (Apple Git-155)
Re: [PATCH v7 1/4] KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
Hi Ritesh,
Thanks for reviewing this patch. Please find my response inline below.
Quoted 161 lines · Ritesh Harjani
Quoted 100 lines · Amit Machhiwal
Introduce a new capability and ioctl to expose CPU compatibility modes supported by the host processor for nested guests.
On IBM POWER systems, newer processor generations (N) can operate in compatibility modes corresponding to earlier generations, like (N-1) and (N-2). This is particularly relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Introduce
KVM_CAP_PPC_COMPAT_CAPScapability and the correspondingKVM_PPC_GET_COMPAT_CAPSvm ioctl. The ioctl returns a bitmap describing the compatibility modes supported by the host in respective bit numbers, allowing userspace (e.g., QEMU) to select an appropriate compatibility level when configuring nested KVM guests.The ioctl handling is added in
kvm_arch_vm_ioctl()and retrieves host CPU compatibility capabilities via a PowerPC-specific backend implementation when available.The struct
kvm_ppc_compat_capsplaces the 'size' field first so it can be read alone viaget_user()beforecopy_struct_from_user()is called, avoiding pointer arithmetic to locate the size field.The ioctl is defined using
_IOso the ioctl number remains stable even if the struct grows in future versions. It usescopy_struct_from_user()andcopy_struct_to_user()to provide forward- and backward-compatible
extensibility: older userspace passing a smaller struct to a newer kernel gets zero-padded trailing fields, while newer userspace passing a larger struct to an older kernel (usize > ksize) getssizeof(struct kvm_ppc_compat_caps)written back to host_caps.size so it can retry with the older kernel-supported size, after which the kernel returns-E2BIG.
KVM_PPC_COMPAT_CAPS_SIZE_VER0is defined as a frozen integer constant
(24) marking the size of the initial struct version, used as the minimum floor for size field validation, similar to other versioned struct interfaces in the kernel.The 'flags' field is reserved for future use. The kernel rejects any call where flags is non-zero with
-EINVAL, preventing garbage values from being baked into ABI permanently.The ioctl returns appropriate error codes: EINVAL for an invalid size or non-zero reserved fields, E2BIG if new userspace provides a larger struct than the kernel knows about (with ksize written back into host_caps.size for the retry), EFAULT for failed copy operations, and ENOTTY if the backend doesn't implement
get_compat_caps.Suggested-by Vaibhav Jain <[email protected]>Tested-by Gautam Menghani <[email protected]>Reviewed-by Gautam Menghani <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>---
Changes in this version:
-KVM_CAP_PPC_COMPAT_CAPS: addhv_enabledguard to align the capability check with ioctl availability; a PR KVM VM on pseries now correctly returns 0 for the capability [Sashiko]arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/include/uapi/asm/kvm.h | 8 ++++ arch/powerpc/kvm/powerpc.c | 71 +++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 3 ++ 4 files changed, 83 insertions(+)arch/powerpc/include/asm/kvm_ppc.h +1
-0@@ -319,6 +319,7 @@ struct kvmppc_ops {bool (*hash_v3_possible)(void );int (*create_vm_debugfs)(struct kvm *kvm);int (*create_vcpu_debugfs)(struct kvm_vcpu *vcpu,struct dentry *debugfs_dentry); +int (*get_compat_caps)(struct kvm_ppc_compat_caps *host_caps); };extern struct kvmppc_ops *kvmppc_hv_ops;arch/powerpc/include/uapi/asm/kvm.h +8
-0@@ -437,6 +437,14 @@ struct kvm_ppc_cpu_char {__u64 behaviour_mask;/* valid bits in behaviour */ }; +/* For KVM_PPC_GET_COMPAT_CAPS */ +struct kvm_ppc_compat_caps { +__u64 size;/* Size of this structure */ +__u64 flags;/* Reserved for future use */ +__u64 compat_capabilities;/* Capabilities supported by the host */ +}; +#define KVM_PPC_COMPAT_CAPS_SIZE_VER0 24/* sizeof first published struct */ +/* * Values for character and character_mask. * These are identical to the values used by H_GET_CPU_CHARACTERISTICS. arch/powerpc/kvm/powerpc.c +0
-0arch/powerpc/kvm/powerpc.c int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · line 709
Quoted 7 lines
arch/powerpc/kvm/powerpc.c +4
-0@@ -703,6 +703,13 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) } }break ; +#if defined(CONFIG_KVM_BOOK3S_HV_POSSIBLE ) +case KVM_CAP_PPC_COMPAT_CAPS : + r =0 ; +if (hv_enabled &&kvmhv_on_pseries ())I think sashiko is just complaining in the 1st patch because we have not yet wired up the
kvmppc_hv_ops->get_compat_caps()yet in patch-1. I think it is expecting..if (
hv_enabled&&kvmhv_on_pseries()&&kvmppc_hv_ops->get_compat_caps)But either way is fine.
Quoted 42 lines
arch/powerpc/kvm/powerpc.c +35
-0+ r =1 ; +break ; +#endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */ default : r =0 ;break ;@@ -2469,6 +2476,70 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = kvm->arch.kvm_ops->svm_off (kvm);break ; } +case KVM_PPC_GET_COMPAT_CAPS : { +struct kvm_ppc_compat_caps host_caps = {}; +u64 usize; + +/* +* Read the size field first to drive copy_struct_from_user. +* size must be the first field of the struct. +*/ + r = -EFAULT ; +if (get_user (usize, (__u64 __user *)argp)) + goto out; + +/* +* Enforce a minimum: reject buffers smaller than the initial +* struct version (VER0). This allows old userspace compiled +* against the original struct to still work on a newer kernel +* that has grown the struct with appended fields. +*/ + r = -EINVAL ; +if (usize <KVM_PPC_COMPAT_CAPS_SIZE_VER0 ) + goto out; + +/* +* New userspace with a larger struct called an older kernel. +* Write back ksize in host_caps.size so userspace knows which +* older struct to retry with, then fail with -E2BIG. +*/ +if (usize >sizeof (host_caps)) { + host_caps.size =sizeof (host_caps); + r = -EFAULT ; +if (put_user (host_caps.size, (__u64 __user *)argp)) + goto out;
Quoted 8 lines
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2513
Quoted 3 lines
arch/powerpc/kvm/powerpc.c +3
-0+ r = -E2BIG ; + goto out; + }You anyways mentioned
copy_struct_from_user()is taking care of both forward and backward compat. Then what is the point of this check? shouldn't we get rid of this complete if logic? I don't see a point of this if we are anyway usingcopy_struct_from_user().
The pre-check is intentional and serves a purpose that copy_struct_from_user() alone cannot provide: explicit kernel struct size discovery.
copy_struct_from_user() returns -E2BIG when usize > ksize and trailing bytes are non-zero — but it gives userspace no way to know what ksize to retry with. It also silently succeeds when trailing bytes are zero, which means a new userspace on an old kernel would never learn the kernel's struct size at all.
This design is different by intent: when new userspace passes a larger struct, we always return -E2BIG and write back sizeof(host_caps) into the size field so userspace learns the exact kernel-supported size and can retry with it. This explicit negotiation is verified in the ABI extensibility testing in the cover letter ("Newer struct on QEMU, older kernel -> works"). The explicit -E2BIG + ksize writeback makes the version negotiation unambiguous.
Quoted 10 lines
Quoted 6 lines
arch/powerpc/kvm/powerpc.c +5
-0+ +/* +* copy_struct_from_user() handles forward/backward compat: +* usize == ksize: verbatim copy +* usize < ksize: zero-pad trailing (old userspace, new kernel) (quoted diff) +2
-0+ * usize > ksize: succeed iff the trailing bytes userspace + * sent are zero, else -E2BIG
Quoted 3 lines
shouldn't we add usize > ksize case details, like ^^^, which
copy_struct_from_user()handles since we are already adding the details of other 2 cases.
The comment is intentionally limited to two cases. By the time we reach copy_struct_from_user(), the usize > sizeof(host_caps) case has already been caught by the pre-check above and returned -E2BIG. So copy_struct_from_user() can only ever see usize == ksize or usize < ksize at this point — those are the only two cases worth documenting here.
Adding the usize > ksize case would be misleading since that code path is unreachable for this call site.
Quoted 41 lines
</btw snip from
copy_struct_from_userkdoc>* There are three cases to consider:
* * If @usize == @ksize, then it's copied verbatim.
* * If @usize < @ksize, then the userspace has passed an old struct to a
* newer kernel. The rest of the trailing bytes in @dst (@ksize - @usize)
* are to be zero-filled.
* * If @usize > @ksize, then the userspace has passed a new struct to an
* older kernel. The trailing bytes unknown to the kernel (@usize - @ksize)
* are checked to ensure they are zeroed, otherwise-E2BIGis returned.
*
* Returns (in all cases, some data may have been copied):
* *-E2BIG: (@usize > @ksize) and there are non-zero trailing bytes in @src.
* *-EFAULT: access to userspace failed.Quoted 10 lines
arch/powerpc/kvm/powerpc.c +10
-0+*/ + r =copy_struct_from_user (&host_caps,sizeof (host_caps), + argp, usize); +if (r) + goto out; + +/* Reserved fields must be zero */ + r = -EINVAL ; +if (host_caps.flags) + goto out;arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2538
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +10
-0+ + r = -ENOTTY ; +if (!kvm->arch.kvm_ops->get_compat_caps) + goto out; + + r = kvm->arch.kvm_ops->get_compat_caps (&host_caps); +if (r) + goto out; + + host_caps.size =sizeof (host_caps);shouldn't this be, since we don't want to be reporting a larger size to the user?
Quoted 1 line
host_caps.size = min_t(u64, usize, sizeof(host_caps));
At this point in the code, usize has already been bounds-checked to [KVM_PPC_COMPAT_CAPS_SIZE_VER0, sizeof(host_caps)], so min_t(u64, usize, sizeof(host_caps)) is always equal to usize here — the min_t never actually clamps anything.
Thanks,
Amit
Ritesh Harjani (IBM)
reviewerRe: [PATCH v7 1/4] KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
Quoted 185 lines · Amit Machhiwal
Hi Ritesh,
Thanks for reviewing this patch. Please find my response inline below.
Quoted 161 lines · Ritesh Harjani
Quoted 100 lines · Amit Machhiwal
Introduce a new capability and ioctl to expose CPU compatibility modes supported by the host processor for nested guests.
On IBM POWER systems, newer processor generations (N) can operate in compatibility modes corresponding to earlier generations, like (N-1) and (N-2). This is particularly relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Introduce
KVM_CAP_PPC_COMPAT_CAPScapability and the correspondingKVM_PPC_GET_COMPAT_CAPSvm ioctl. The ioctl returns a bitmap describing the compatibility modes supported by the host in respective bit numbers, allowing userspace (e.g., QEMU) to select an appropriate compatibility level when configuring nested KVM guests.The ioctl handling is added in
kvm_arch_vm_ioctl()and retrieves host CPU compatibility capabilities via a PowerPC-specific backend implementation when available.The struct
kvm_ppc_compat_capsplaces the 'size' field first so it can be read alone viaget_user()beforecopy_struct_from_user()is called, avoiding pointer arithmetic to locate the size field.The ioctl is defined using
_IOso the ioctl number remains stable even if the struct grows in future versions. It usescopy_struct_from_user()andcopy_struct_to_user()to provide forward- and backward-compatible
extensibility: older userspace passing a smaller struct to a newer kernel gets zero-padded trailing fields, while newer userspace passing a larger struct to an older kernel (usize > ksize) getssizeof(struct kvm_ppc_compat_caps)written back to host_caps.size so it can retry with the older kernel-supported size, after which the kernel returns-E2BIG.
KVM_PPC_COMPAT_CAPS_SIZE_VER0is defined as a frozen integer constant
(24) marking the size of the initial struct version, used as the minimum floor for size field validation, similar to other versioned struct interfaces in the kernel.The 'flags' field is reserved for future use. The kernel rejects any call where flags is non-zero with
-EINVAL, preventing garbage values from being baked into ABI permanently.The ioctl returns appropriate error codes: EINVAL for an invalid size or non-zero reserved fields, E2BIG if new userspace provides a larger struct than the kernel knows about (with ksize written back into host_caps.size for the retry), EFAULT for failed copy operations, and ENOTTY if the backend doesn't implement
get_compat_caps.Suggested-by Vaibhav Jain <[email protected]>Tested-by Gautam Menghani <[email protected]>Reviewed-by Gautam Menghani <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>---
Changes in this version:
-KVM_CAP_PPC_COMPAT_CAPS: addhv_enabledguard to align the capability check with ioctl availability; a PR KVM VM on pseries now correctly returns 0 for the capability [Sashiko]arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/include/uapi/asm/kvm.h | 8 ++++ arch/powerpc/kvm/powerpc.c | 71 +++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 3 ++ 4 files changed, 83 insertions(+)arch/powerpc/include/asm/kvm_ppc.h +1
-0@@ -319,6 +319,7 @@ struct kvmppc_ops {bool (*hash_v3_possible)(void );int (*create_vm_debugfs)(struct kvm *kvm);int (*create_vcpu_debugfs)(struct kvm_vcpu *vcpu,struct dentry *debugfs_dentry); +int (*get_compat_caps)(struct kvm_ppc_compat_caps *host_caps); };extern struct kvmppc_ops *kvmppc_hv_ops;arch/powerpc/include/uapi/asm/kvm.h +8
-0@@ -437,6 +437,14 @@ struct kvm_ppc_cpu_char {__u64 behaviour_mask;/* valid bits in behaviour */ }; +/* For KVM_PPC_GET_COMPAT_CAPS */ +struct kvm_ppc_compat_caps { +__u64 size;/* Size of this structure */ +__u64 flags;/* Reserved for future use */ +__u64 compat_capabilities;/* Capabilities supported by the host */ +}; +#define KVM_PPC_COMPAT_CAPS_SIZE_VER0 24/* sizeof first published struct */ +/* * Values for character and character_mask. * These are identical to the values used by H_GET_CPU_CHARACTERISTICS. arch/powerpc/kvm/powerpc.c +0
-0arch/powerpc/kvm/powerpc.c int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · line 709
Quoted 7 lines
arch/powerpc/kvm/powerpc.c +4
-0@@ -703,6 +703,13 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) } }break ; +#if defined(CONFIG_KVM_BOOK3S_HV_POSSIBLE ) +case KVM_CAP_PPC_COMPAT_CAPS : + r =0 ; +if (hv_enabled &&kvmhv_on_pseries ())I think sashiko is just complaining in the 1st patch because we have not yet wired up the
kvmppc_hv_ops->get_compat_caps()yet in patch-1. I think it is expecting..if (
hv_enabled&&kvmhv_on_pseries()&&kvmppc_hv_ops->get_compat_caps)But either way is fine.
Quoted 42 lines
arch/powerpc/kvm/powerpc.c +35
-0+ r =1 ; +break ; +#endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */ default : r =0 ;break ;@@ -2469,6 +2476,70 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = kvm->arch.kvm_ops->svm_off (kvm);break ; } +case KVM_PPC_GET_COMPAT_CAPS : { +struct kvm_ppc_compat_caps host_caps = {}; +u64 usize; + +/* +* Read the size field first to drive copy_struct_from_user. +* size must be the first field of the struct. +*/ + r = -EFAULT ; +if (get_user (usize, (__u64 __user *)argp)) + goto out; + +/* +* Enforce a minimum: reject buffers smaller than the initial +* struct version (VER0). This allows old userspace compiled +* against the original struct to still work on a newer kernel +* that has grown the struct with appended fields. +*/ + r = -EINVAL ; +if (usize <KVM_PPC_COMPAT_CAPS_SIZE_VER0 ) + goto out; + +/* +* New userspace with a larger struct called an older kernel. +* Write back ksize in host_caps.size so userspace knows which +* older struct to retry with, then fail with -E2BIG. +*/ +if (usize >sizeof (host_caps)) { + host_caps.size =sizeof (host_caps); + r = -EFAULT ; +if (put_user (host_caps.size, (__u64 __user *)argp)) + goto out;Quoted 8 lines
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2513
Quoted 3 lines
arch/powerpc/kvm/powerpc.c +3
-0+ r = -E2BIG ; + goto out; + }You anyways mentioned
copy_struct_from_user()is taking care of both forward and backward compat. Then what is the point of this check? shouldn't we get rid of this complete if logic? I don't see a point of this if we are anyway usingcopy_struct_from_user().The pre-check is intentional and serves a purpose that
copy_struct_from_user()alone cannot provide: explicit kernel struct size discovery.
copy_struct_from_user()returns-E2BIGwhen usize > ksize and trailing bytes are non-zero — but it gives userspace no way to know what ksize to retry with. It also silently succeeds when trailing bytes are zero, which means a new userspace on an old kernel would never learn the kernel's struct size at all.
Quoted 8 lines
This design is different by intent: when new userspace passes a larger struct, we always return
-E2BIGand write backsizeof(host_caps)into the size field so userspace learns the exact kernel-supported size and can retry with it. This explicit negotiation is verified in the ABI extensibility testing in the cover letter ("Newer struct on QEMU, older kernel -> works"). The explicit-E2BIG+ ksize writeback makes the version negotiation unambiguous.
yup, my bad, should have seen the comment in the code above.
But isn't this a bad design? Tomorrow, say we have a v2, a larger version of this struct and if your newer Qemu (with larger struct size) is running on an older kernel with a smaller struct size, then you will fail it because usize > ksize even though the userspace has zero filled the rest of the trailing bytes...
if (usize > sizeof(host_caps))
Instead wouldn't it be better if we do something like this?
r = copy_struct_from_user(&host_caps, sizeof(host_caps),
argp, usize);
if (r) {
if (r == -E2BIG)
put_user(sizeof(host_caps), &argp->size);
goto out;
}So, we only fail if the userspace has passed a newer struct and the trailing bytes, which are unknown to this older kernel, are zeroed, otherwise we return -E2BIG. We also write the ksize back, so the user knows what ksize it supports.
This would avoid an unncessary round trip and will simplify the userspace design, isn't it?
Quoted 74 lines
Quoted 10 lines
Quoted 6 lines
arch/powerpc/kvm/powerpc.c +5
-0+ +/* +* copy_struct_from_user() handles forward/backward compat: +* usize == ksize: verbatim copy +* usize < ksize: zero-pad trailing (old userspace, new kernel) (quoted diff) +2
-0+ * usize > ksize: succeed iff the trailing bytes userspace + * sent are zero, else -E2BIGQuoted 3 lines
shouldn't we add usize > ksize case details, like ^^^, which
copy_struct_from_user()handles since we are already adding the details of other 2 cases.The comment is intentionally limited to two cases. By the time we reach
copy_struct_from_user(), the usize >sizeof(host_caps)case has already been caught by the pre-check above and returned-E2BIG. Socopy_struct_from_user()can only ever see usize == ksize or usize < ksize at this point — those are the only two cases worth documenting here.Adding the usize > ksize case would be misleading since that code path is unreachable for this call site.
Quoted 41 lines
</btw snip from
copy_struct_from_userkdoc>* There are three cases to consider:
* * If @usize == @ksize, then it's copied verbatim.
* * If @usize < @ksize, then the userspace has passed an old struct to a
* newer kernel. The rest of the trailing bytes in @dst (@ksize - @usize)
* are to be zero-filled.
* * If @usize > @ksize, then the userspace has passed a new struct to an
* older kernel. The trailing bytes unknown to the kernel (@usize - @ksize)
* are checked to ensure they are zeroed, otherwise-E2BIGis returned.
*
* Returns (in all cases, some data may have been copied):
* *-E2BIG: (@usize > @ksize) and there are non-zero trailing bytes in @src.
* *-EFAULT: access to userspace failed.Quoted 10 lines
arch/powerpc/kvm/powerpc.c +10
-0+*/ + r =copy_struct_from_user (&host_caps,sizeof (host_caps), + argp, usize); +if (r) + goto out; + +/* Reserved fields must be zero */ + r = -EINVAL ; +if (host_caps.flags) + goto out;arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2538
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +10
-0+ + r = -ENOTTY ; +if (!kvm->arch.kvm_ops->get_compat_caps) + goto out; + + r = kvm->arch.kvm_ops->get_compat_caps (&host_caps); +if (r) + goto out; + + host_caps.size =sizeof (host_caps);shouldn't this be, since we don't want to be reporting a larger size to the user?
Quoted 1 line
host_caps.size = min_t(u64, usize, sizeof(host_caps));At this point in the code, usize has already been bounds-checked to [
KVM_PPC_COMPAT_CAPS_SIZE_VER0,sizeof(host_caps)], somin_t(u64, usize, sizeof(host_caps))is always equal to usize here — themin_tnever actually clamps anything.Thanks,
Amit
Ritesh Harjani (IBM)
reviewerRe: [PATCH v7 1/4] KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
Quoted 215 lines · Ritesh Harjani (IBM)
Quoted 185 lines · Amit Machhiwal
Hi Ritesh,
Thanks for reviewing this patch. Please find my response inline below.
Quoted 161 lines · Ritesh Harjani
Quoted 100 lines · Amit Machhiwal
Introduce a new capability and ioctl to expose CPU compatibility modes supported by the host processor for nested guests.
On IBM POWER systems, newer processor generations (N) can operate in compatibility modes corresponding to earlier generations, like (N-1) and (N-2). This is particularly relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Introduce
KVM_CAP_PPC_COMPAT_CAPScapability and the correspondingKVM_PPC_GET_COMPAT_CAPSvm ioctl. The ioctl returns a bitmap describing the compatibility modes supported by the host in respective bit numbers, allowing userspace (e.g., QEMU) to select an appropriate compatibility level when configuring nested KVM guests.The ioctl handling is added in
kvm_arch_vm_ioctl()and retrieves host CPU compatibility capabilities via a PowerPC-specific backend implementation when available.The struct
kvm_ppc_compat_capsplaces the 'size' field first so it can be read alone viaget_user()beforecopy_struct_from_user()is called, avoiding pointer arithmetic to locate the size field.The ioctl is defined using
_IOso the ioctl number remains stable even if the struct grows in future versions. It usescopy_struct_from_user()andcopy_struct_to_user()to provide forward- and backward-compatible
extensibility: older userspace passing a smaller struct to a newer kernel gets zero-padded trailing fields, while newer userspace passing a larger struct to an older kernel (usize > ksize) getssizeof(struct kvm_ppc_compat_caps)written back to host_caps.size so it can retry with the older kernel-supported size, after which the kernel returns-E2BIG.
KVM_PPC_COMPAT_CAPS_SIZE_VER0is defined as a frozen integer constant
(24) marking the size of the initial struct version, used as the minimum floor for size field validation, similar to other versioned struct interfaces in the kernel.The 'flags' field is reserved for future use. The kernel rejects any call where flags is non-zero with
-EINVAL, preventing garbage values from being baked into ABI permanently.The ioctl returns appropriate error codes: EINVAL for an invalid size or non-zero reserved fields, E2BIG if new userspace provides a larger struct than the kernel knows about (with ksize written back into host_caps.size for the retry), EFAULT for failed copy operations, and ENOTTY if the backend doesn't implement
get_compat_caps.Suggested-by Vaibhav Jain <[email protected]>Tested-by Gautam Menghani <[email protected]>Reviewed-by Gautam Menghani <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>---
Changes in this version:
-KVM_CAP_PPC_COMPAT_CAPS: addhv_enabledguard to align the capability check with ioctl availability; a PR KVM VM on pseries now correctly returns 0 for the capability [Sashiko]arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/include/uapi/asm/kvm.h | 8 ++++ arch/powerpc/kvm/powerpc.c | 71 +++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 3 ++ 4 files changed, 83 insertions(+)arch/powerpc/include/asm/kvm_ppc.h +1
-0@@ -319,6 +319,7 @@ struct kvmppc_ops {bool (*hash_v3_possible)(void );int (*create_vm_debugfs)(struct kvm *kvm);int (*create_vcpu_debugfs)(struct kvm_vcpu *vcpu,struct dentry *debugfs_dentry); +int (*get_compat_caps)(struct kvm_ppc_compat_caps *host_caps); };extern struct kvmppc_ops *kvmppc_hv_ops;arch/powerpc/include/uapi/asm/kvm.h +8
-0@@ -437,6 +437,14 @@ struct kvm_ppc_cpu_char {__u64 behaviour_mask;/* valid bits in behaviour */ }; +/* For KVM_PPC_GET_COMPAT_CAPS */ +struct kvm_ppc_compat_caps { +__u64 size;/* Size of this structure */ +__u64 flags;/* Reserved for future use */ +__u64 compat_capabilities;/* Capabilities supported by the host */ +}; +#define KVM_PPC_COMPAT_CAPS_SIZE_VER0 24/* sizeof first published struct */ +/* * Values for character and character_mask. * These are identical to the values used by H_GET_CPU_CHARACTERISTICS. arch/powerpc/kvm/powerpc.c +0
-0arch/powerpc/kvm/powerpc.c int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · line 709
Quoted 7 lines
arch/powerpc/kvm/powerpc.c +4
-0@@ -703,6 +703,13 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) } }break ; +#if defined(CONFIG_KVM_BOOK3S_HV_POSSIBLE ) +case KVM_CAP_PPC_COMPAT_CAPS : + r =0 ; +if (hv_enabled &&kvmhv_on_pseries ())I think sashiko is just complaining in the 1st patch because we have not yet wired up the
kvmppc_hv_ops->get_compat_caps()yet in patch-1. I think it is expecting..if (
hv_enabled&&kvmhv_on_pseries()&&kvmppc_hv_ops->get_compat_caps)But either way is fine.
Quoted 42 lines
arch/powerpc/kvm/powerpc.c +35
-0+ r =1 ; +break ; +#endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */ default : r =0 ;break ;@@ -2469,6 +2476,70 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = kvm->arch.kvm_ops->svm_off (kvm);break ; } +case KVM_PPC_GET_COMPAT_CAPS : { +struct kvm_ppc_compat_caps host_caps = {}; +u64 usize; + +/* +* Read the size field first to drive copy_struct_from_user. +* size must be the first field of the struct. +*/ + r = -EFAULT ; +if (get_user (usize, (__u64 __user *)argp)) + goto out; + +/* +* Enforce a minimum: reject buffers smaller than the initial +* struct version (VER0). This allows old userspace compiled +* against the original struct to still work on a newer kernel +* that has grown the struct with appended fields. +*/ + r = -EINVAL ; +if (usize <KVM_PPC_COMPAT_CAPS_SIZE_VER0 ) + goto out; + +/* +* New userspace with a larger struct called an older kernel. +* Write back ksize in host_caps.size so userspace knows which +* older struct to retry with, then fail with -E2BIG. +*/ +if (usize >sizeof (host_caps)) { + host_caps.size =sizeof (host_caps); + r = -EFAULT ; +if (put_user (host_caps.size, (__u64 __user *)argp)) + goto out;Quoted 8 lines
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2513
Quoted 3 lines
arch/powerpc/kvm/powerpc.c +3
-0+ r = -E2BIG ; + goto out; + }You anyways mentioned
copy_struct_from_user()is taking care of both forward and backward compat. Then what is the point of this check? shouldn't we get rid of this complete if logic? I don't see a point of this if we are anyway usingcopy_struct_from_user().The pre-check is intentional and serves a purpose that
copy_struct_from_user()alone cannot provide: explicit kernel struct size discovery.
copy_struct_from_user()returns-E2BIGwhen usize > ksize and trailing bytes are non-zero — but it gives userspace no way to know what ksize to retry with. It also silently succeeds when trailing bytes are zero, which means a new userspace on an old kernel would never learn the kernel's struct size at all.Quoted 8 lines
This design is different by intent: when new userspace passes a larger struct, we always return
-E2BIGand write backsizeof(host_caps)into the size field so userspace learns the exact kernel-supported size and can retry with it. This explicit negotiation is verified in the ABI extensibility testing in the cover letter ("Newer struct on QEMU, older kernel -> works"). The explicit-E2BIG+ ksize writeback makes the version negotiation unambiguous.yup, my bad, should have seen the comment in the code above.
But isn't this a bad design? Tomorrow, say we have a v2, a larger version of this struct and if your newer Qemu (with larger struct size) is running on an older kernel with a smaller struct size, then you will fail it because usize > ksize even though the userspace has zero filled the rest of the trailing bytes...
if (usize >sizeof(host_caps))Instead wouldn't it be better if we do something like this?
r = copy_struct_from_user(&host_caps, sizeof(host_caps), argp, usize); if (r) { if (r == -E2BIG) put_user(sizeof(host_caps), &argp->size); goto out; }
Quoted 3 lines
So, we only fail if the userspace has passed a newer struct and the trailing bytes, which are unknown to this older kernel, are zeroed, otherwise we return
-E2BIG.
Sorry made some typos in above paragraph, what I meant is -
So we only fail with -E2BIG if the userspace has passed a newer struct and the trailing bytes, which are unknown to this older kernel, are non-zero. if they're all zero, the call succeeds, since those bytes are safe to ignore. We also write the ksize back on the failure path, so the user knows what ksize this kernel supports.
Re: [PATCH v7 1/4] KVM: PPC: Introduce KVM_CAP_PPC_COMPAT_CAPS and wire up ioctl
In reply to Ritesh Harjani (IBM)Quoted 196 lines · Ritesh Harjani
Quoted 185 lines · Amit Machhiwal
Hi Ritesh,
Thanks for reviewing this patch. Please find my response inline below.
Quoted 161 lines · Ritesh Harjani
Quoted 100 lines · Amit Machhiwal
Introduce a new capability and ioctl to expose CPU compatibility modes supported by the host processor for nested guests.
On IBM POWER systems, newer processor generations (N) can operate in compatibility modes corresponding to earlier generations, like (N-1) and (N-2). This is particularly relevant for nested virtualization, where nested KVM guests may need to run with a specific processor compatibility level.
Introduce
KVM_CAP_PPC_COMPAT_CAPScapability and the correspondingKVM_PPC_GET_COMPAT_CAPSvm ioctl. The ioctl returns a bitmap describing the compatibility modes supported by the host in respective bit numbers, allowing userspace (e.g., QEMU) to select an appropriate compatibility level when configuring nested KVM guests.The ioctl handling is added in
kvm_arch_vm_ioctl()and retrieves host CPU compatibility capabilities via a PowerPC-specific backend implementation when available.The struct
kvm_ppc_compat_capsplaces the 'size' field first so it can be read alone viaget_user()beforecopy_struct_from_user()is called, avoiding pointer arithmetic to locate the size field.The ioctl is defined using
_IOso the ioctl number remains stable even if the struct grows in future versions. It usescopy_struct_from_user()andcopy_struct_to_user()to provide forward- and backward-compatible
extensibility: older userspace passing a smaller struct to a newer kernel gets zero-padded trailing fields, while newer userspace passing a larger struct to an older kernel (usize > ksize) getssizeof(struct kvm_ppc_compat_caps)written back to host_caps.size so it can retry with the older kernel-supported size, after which the kernel returns-E2BIG.
KVM_PPC_COMPAT_CAPS_SIZE_VER0is defined as a frozen integer constant
(24) marking the size of the initial struct version, used as the minimum floor for size field validation, similar to other versioned struct interfaces in the kernel.The 'flags' field is reserved for future use. The kernel rejects any call where flags is non-zero with
-EINVAL, preventing garbage values from being baked into ABI permanently.The ioctl returns appropriate error codes: EINVAL for an invalid size or non-zero reserved fields, E2BIG if new userspace provides a larger struct than the kernel knows about (with ksize written back into host_caps.size for the retry), EFAULT for failed copy operations, and ENOTTY if the backend doesn't implement
get_compat_caps.Suggested-by Vaibhav Jain <[email protected]>Tested-by Gautam Menghani <[email protected]>Reviewed-by Gautam Menghani <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>---
Changes in this version:
-KVM_CAP_PPC_COMPAT_CAPS: addhv_enabledguard to align the capability check with ioctl availability; a PR KVM VM on pseries now correctly returns 0 for the capability [Sashiko]arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/include/uapi/asm/kvm.h | 8 ++++ arch/powerpc/kvm/powerpc.c | 71 +++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 3 ++ 4 files changed, 83 insertions(+)arch/powerpc/include/asm/kvm_ppc.h +1
-0@@ -319,6 +319,7 @@ struct kvmppc_ops {bool (*hash_v3_possible)(void );int (*create_vm_debugfs)(struct kvm *kvm);int (*create_vcpu_debugfs)(struct kvm_vcpu *vcpu,struct dentry *debugfs_dentry); +int (*get_compat_caps)(struct kvm_ppc_compat_caps *host_caps); };extern struct kvmppc_ops *kvmppc_hv_ops;arch/powerpc/include/uapi/asm/kvm.h +8
-0@@ -437,6 +437,14 @@ struct kvm_ppc_cpu_char {__u64 behaviour_mask;/* valid bits in behaviour */ }; +/* For KVM_PPC_GET_COMPAT_CAPS */ +struct kvm_ppc_compat_caps { +__u64 size;/* Size of this structure */ +__u64 flags;/* Reserved for future use */ +__u64 compat_capabilities;/* Capabilities supported by the host */ +}; +#define KVM_PPC_COMPAT_CAPS_SIZE_VER0 24/* sizeof first published struct */ +/* * Values for character and character_mask. * These are identical to the values used by H_GET_CPU_CHARACTERISTICS. arch/powerpc/kvm/powerpc.c +0
-0arch/powerpc/kvm/powerpc.c int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · line 709
Quoted 7 lines
arch/powerpc/kvm/powerpc.c +4
-0@@ -703,6 +703,13 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) } }break ; +#if defined(CONFIG_KVM_BOOK3S_HV_POSSIBLE ) +case KVM_CAP_PPC_COMPAT_CAPS : + r =0 ; +if (hv_enabled &&kvmhv_on_pseries ())I think sashiko is just complaining in the 1st patch because we have not yet wired up the
kvmppc_hv_ops->get_compat_caps()yet in patch-1. I think it is expecting..if (
hv_enabled&&kvmhv_on_pseries()&&kvmppc_hv_ops->get_compat_caps)But either way is fine.
Quoted 42 lines
arch/powerpc/kvm/powerpc.c +35
-0+ r =1 ; +break ; +#endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */ default : r =0 ;break ;@@ -2469,6 +2476,70 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = kvm->arch.kvm_ops->svm_off (kvm);break ; } +case KVM_PPC_GET_COMPAT_CAPS : { +struct kvm_ppc_compat_caps host_caps = {}; +u64 usize; + +/* +* Read the size field first to drive copy_struct_from_user. +* size must be the first field of the struct. +*/ + r = -EFAULT ; +if (get_user (usize, (__u64 __user *)argp)) + goto out; + +/* +* Enforce a minimum: reject buffers smaller than the initial +* struct version (VER0). This allows old userspace compiled +* against the original struct to still work on a newer kernel +* that has grown the struct with appended fields. +*/ + r = -EINVAL ; +if (usize <KVM_PPC_COMPAT_CAPS_SIZE_VER0 ) + goto out; + +/* +* New userspace with a larger struct called an older kernel. +* Write back ksize in host_caps.size so userspace knows which +* older struct to retry with, then fail with -E2BIG. +*/ +if (usize >sizeof (host_caps)) { + host_caps.size =sizeof (host_caps); + r = -EFAULT ; +if (put_user (host_caps.size, (__u64 __user *)argp)) + goto out;Quoted 8 lines
arch/powerpc/kvm/powerpc.c int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) · line 2513
Quoted 3 lines
arch/powerpc/kvm/powerpc.c +3
-0+ r = -E2BIG ; + goto out; + }You anyways mentioned
copy_struct_from_user()is taking care of both forward and backward compat. Then what is the point of this check? shouldn't we get rid of this complete if logic? I don't see a point of this if we are anyway usingcopy_struct_from_user().The pre-check is intentional and serves a purpose that
copy_struct_from_user()alone cannot provide: explicit kernel struct size discovery.
copy_struct_from_user()returns-E2BIGwhen usize > ksize and trailing bytes are non-zero — but it gives userspace no way to know what ksize to retry with. It also silently succeeds when trailing bytes are zero, which means a new userspace on an old kernel would never learn the kernel's struct size at all.Quoted 8 lines
This design is different by intent: when new userspace passes a larger struct, we always return
-E2BIGand write backsizeof(host_caps)into the size field so userspace learns the exact kernel-supported size and can retry with it. This explicit negotiation is verified in the ABI extensibility testing in the cover letter ("Newer struct on QEMU, older kernel -> works"). The explicit-E2BIG+ ksize writeback makes the version negotiation unambiguous.
Quoted 1 line
yup, my bad, should have seen the comment in the code above.
No worries...
Quoted 6 lines
But isn't this a bad design? Tomorrow, say we have a v2, a larger version of this struct and if your newer Qemu (with larger struct size) is running on an older kernel with a smaller struct size, then you will fail it because usize > ksize even though the userspace has zero filled the rest of the trailing bytes...
You're right and thanks for pointing this out. A newer userspace that zero-initialises the new fields (e.g. struct caps = {}; caps.size = sizeof(caps);) is a valid forward-compat call — zero means "use default for unknown fields", which is exactly the contract copy_struct_from_user() is designed to honour. Forcing a round-trip in that case is wrong.
I'll drop the manual usize > sizeof(host_caps) pre-check and let copy_struct_from_user() own the usize > ksize path entirely.
One small correction to your suggested snippet though: put_user() return value is ignored there. put_user() returns 0 on success or -EFAULT on failure, so if the write-back fails we'd silently return -E2BIG instead of -EFAULT. The fix is straightforward:
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index e64b3cfadd3a..ed48f069fb76 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -2498,29 +2498,28 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
if (usize < KVM_PPC_COMPAT_CAPS_SIZE_VER0)
goto out;- /*
- * New userspace with a larger struct called an older kernel.
- * Write back ksize in host_caps.size so userspace knows which
- * older struct to retry with, then fail with -E2BIG.
- */
- if (usize > sizeof(host_caps)) {
- host_caps.size = sizeof(host_caps);
- r = -EFAULT;
- if (put_user(host_caps.size, (__u64 __user *)argp))
- goto out;
- r = -E2BIG;
- goto out;
- }
-
/*
* copy_struct_from_user() handles forward/backward compat:
* usize == ksize: verbatim copy
* usize < ksize: zero-pad trailing (old userspace, new kernel)
+ * usize > ksize: succeed iff trailing bytes are zero, else -E2BIG
*/
r = copy_struct_from_user(&host_caps, sizeof(host_caps),
argp, usize);
- if (r)
+ if (r) {
+ /*
+ * New userspace with a larger struct called an older
+ * kernel. Write back ksize in host_caps.size so
+ * userspace knows which older struct to retry with,
+ * then fail with -E2BIG.
+ */
+ if (r == -E2BIG)
+ if (put_user((__u64)sizeof(host_caps),
+ (__u64 __user *)argp))
+ r = -EFAULT;
goto out;+ }
/* Reserved fields must be zero */
r = -EINVAL;This preserves -EFAULT priority if put_user() fails, consistent with how the get_user() guard earlier in the same handler works.
Will post v8 with this fix.
Thanks,
Amit
Quoted 21 lines
if (usize >
sizeof(host_caps))Instead wouldn't it be better if we do something like this?
r = copy_struct_from_user(&host_caps, sizeof(host_caps), argp, usize); if (r) { if (r == -E2BIG) put_user(sizeof(host_caps), &argp->size); goto out; }So, we only fail if the userspace has passed a newer struct and the trailing bytes, which are unknown to this older kernel, are zeroed, otherwise we return
-E2BIG. We also write the ksize back, so the user knows what ksize it supports.This would avoid an unncessary round trip and will simplify the userspace design, isn't it?
Quoted 1 line
Quoted 1 line
<snip>
PATCH v7 2/4 KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM on PowerVM
In reply to Amit MachhiwalOn POWER systems, the host CPU may run in a compatibility mode (e.g., a Power11 processor operating in Power10 compatibility mode). In such cases, the effective CPU level exposed to guests differs from the physical processor generation.
When running nested KVM guests, QEMU derives the host CPU type using mfpvr(), which reflects the physical processor version. This can result in a mismatch between the CPU model selected by QEMU and the compatibility mode enforced by the host, leading to guest boot failures.
For example, booting a nested guest on a Power11 LPAR configured in Power10 compatibility mode fails with:
KVM-NESTEDv2: couldn't set guest wide elements
[..KVM reg dump..]
This occurs because QEMU selects a CPU model corresponding to the physical processor (via mfpvr()), while the host operates in a lower compatibility mode. As a result, KVM rejects the requested compatibility level during guest initialization.
On pseries nestedv2 systems, add support for retrieving host CPU compatibility capabilities for nested guests on PowerVM. The capability bitmap reflects the processor modes negotiated between the Power hypervisor (L0) and the host partition (L1) via the H_GUEST_GET_CAPABILITIES hcall, but is retrieved from the cached nested_capabilities value populated during module initialization, avoiding repeated hypervisor calls. A WARN_ON_ONCE() flags the unexpected case where nested_capabilities is zero on a nestedv2 system. The implementation defines KVM-specific capability constants (KVM_PPC_COMPAT_CAP_POWER9/10/11), masks unsupported bits, and exposes the result through the KVM_PPC_GET_COMPAT_CAPS ioctl.
Hook the implementation into the Book3S HV kvmppc_ops so that it can be invoked by the generic KVM ioctl handling code.
--- arch/powerpc/include/uapi/asm/kvm.h | 10 ++++++++++ arch/powerpc/kvm/book3s_hv.c | 20 ++++++++++++++++++++ 2 files changed, 30 insertions(+)
arch/powerpc/include/uapi/asm/kvm.h +10 -0
@@ -445,6 +445,16 @@ struct kvm_ppc_compat_caps { };#define KVM_PPC_COMPAT_CAPS_SIZE_VER0 24/* sizeof first published struct */ +/* +* Capability bits for compat_capabilities field in kvm_ppc_compat_caps. +* These bits indicate which processor compatibility modes are supported. +*/ +#define KVM_PPC_COMPAT_CAP_POWER9 (1ULL << 62) +#define KVM_PPC_COMPAT_CAP_POWER10 (1ULL << 61) +#define KVM_PPC_COMPAT_CAP_POWER11 (1ULL << 60) +#define KVM_PPC_COMPAT_BITMASK (KVM_PPC_COMPAT_CAP_POWER9 | \ + KVM_PPC_COMPAT_CAP_POWER10 | \ + KVM_PPC_COMPAT_CAP_POWER11)/* * Values for character and character_mask. * These are identical to the values used by H_GET_CPU_CHARACTERISTICS.
arch/powerpc/kvm/book3s_hv.c +20 -0
@@ -6523,6 +6523,25 @@ static bool kvmppc_hash_v3_possible(void)return true; } + +static int kvmppc_get_compat_caps (struct kvm_ppc_compat_caps *host_caps) +{ +unsigned long capabilities =0 ; +long rc = -EINVAL ; + +if (kvmhv_on_pseries ()) { +if (kvmhv_is_nestedv2 ()) { +WARN_ON_ONCE (!nested_capabilities); + capabilities = nested_capabilities; + rc =0 ; + } + } + + host_caps->compat_capabilities = capabilities &KVM_PPC_COMPAT_BITMASK ; + +return rc; +} +static struct kvmppc_ops kvm_ops_hv = { .get_sregs = kvm_arch_vcpu_ioctl_get_sregs_hv, .set_sregs = kvm_arch_vcpu_ioctl_set_sregs_hv,
@@ -6565,6 +6584,7 @@ static struct kvmppc_ops kvm_ops_hv = { .hash_v3_possible = kvmppc_hash_v3_possible, .create_vcpu_debugfs = kvmppc_arch_create_vcpu_debugfs_hv, .create_vm_debugfs = kvmppc_arch_create_vm_debugfs_hv, + .get_compat_caps = kvmppc_get_compat_caps, };static int kvm_init_subcore_bitmap (void )
Signature · 1 line
2.50.1 (Apple Git-155)
[email protected]
AI reviewRe: [PATCH v7 2/4] KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM on PowerVM
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] KVM_CAP_PPC_COMPAT_CAPS incorrectly advertises support on non-nestedv2 pseries hosts, but…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] KVM_CAP_PPC_COMPAT_CAPS incorrectly advertises support on non-nestedv2 pseries hosts, but KVM_PPC_GET_COMPAT_CAPS unconditionally fails with -EINVAL. --
--- Patch [2]: [PATCH v7 2/4] KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM on PowerVM --- commit 7fc253c4f6dbfb301a4c2d7a89c141048079da6c
Author: Amit Machhiwal <[email protected]>
KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM on PowerVM
On POWER systems, the host CPU may run in a compatibility mode (e.g., a Power11 processor operating in Power10 compatibility mode). In such cases, the effective CPU level exposed to guests differs from the physical processor generation. This patch exposes host CPU compatibility mode capabilities to userspace for nested KVM guests via a new ioctl.
Quoted 16 lines
arch/powerpc/kvm/book3s_hv.c +8
-0@@ -6510,6 +6510,25 @@ static bool kvmppc_hash_v3_possible(void)return true; } + +static int kvmppc_get_compat_caps (struct kvm_ppc_compat_caps *host_caps) +{ +unsigned long capabilities =0 ; +long rc = -EINVAL ; + +if (kvmhv_on_pseries ()) { +if (kvmhv_is_nestedv2 ()) {
arch/powerpc/kvm/book3s_hv.c static bool kvmppc_hash_v3_possible(void) · line 6530
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +10
-0+WARN_ON_ONCE (!nested_capabilities); + capabilities = nested_capabilities; + rc =0 ; + } + } + + host_caps->compat_capabilities = capabilities &KVM_PPC_COMPAT_BITMASK ; + +return rc; +}
[Severity: High]
Does this code break guest initialization on non-nestedv2 pseries hosts?
In arch/powerpc/kvm/powerpc.c, kvm_vm_ioctl_check_extension() advertises KVM_CAP_PPC_COMPAT_CAPS support on all pseries hosts:
case KVM_CAP_PPC_COMPAT_CAPS:
r = 0;
if (hv_enabled && kvmhv_on_pseries())
r = 1;
break;But here in kvmppc_get_compat_caps(), if kvmhv_on_pseries() is true and kvmhv_is_nestedv2() is false, rc remains -EINVAL.
When userspace (like QEMU) sees the capability is supported, it will call the KVM_PPC_GET_COMPAT_CAPS ioctl. Receiving an unexpected -EINVAL here causes QEMU to fail VM initialization, breaking guest boots on standard pseries hosts.
Should the check extension logic be updated to also require kvmhv_is_nestedv2(), or should this function handle non-nestedv2 pseries hosts without returning an error?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Re: [PATCH v7 2/4] KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM on PowerVM
In reply to Amit MachhiwalQuoted 41 lines · Amit Machhiwal
On POWER systems, the host CPU may run in a compatibility mode (e.g., a Power11 processor operating in Power10 compatibility mode). In such cases, the effective CPU level exposed to guests differs from the physical processor generation.
When running nested KVM guests, QEMU derives the host CPU type using
mfpvr(), which reflects the physical processor version. This can result in a mismatch between the CPU model selected by QEMU and the compatibility mode enforced by the host, leading to guest boot failures.For example, booting a nested guest on a Power11 LPAR configured in Power10 compatibility mode fails with:
KVM-NESTEDv2: couldn't set guest wide elements
[..KVM reg dump..]This occurs because QEMU selects a CPU model corresponding to the physical processor (via
mfpvr()), while the host operates in a lower compatibility mode. As a result, KVM rejects the requested compatibility level during guest initialization.On pseries nestedv2 systems, add support for retrieving host CPU compatibility capabilities for nested guests on PowerVM. The capability bitmap reflects the processor modes negotiated between the Power hypervisor (L0) and the host partition (L1) via the
H_GUEST_GET_CAPABILITIEShcall, but is retrieved from the cachednested_capabilitiesvalue populated during module initialization, avoiding repeated hypervisor calls. AWARN_ON_ONCE()flags the unexpected case wherenested_capabilitiesis zero on a nestedv2 system. The implementation defines KVM-specific capability constants (KVM_PPC_COMPAT_CAP_POWER9/10/11), masks unsupported bits, and exposes the result through theKVM_PPC_GET_COMPAT_CAPSioctl.Hook the implementation into the Book3S HV
kvmppc_opsso that it can be invoked by the generic KVM ioctl handling code.Suggested-by Vaibhav Jain <[email protected]>Tested-by Gautam Menghani <[email protected]>Reviewed-by Gautam Menghani <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>
Quoted 4 lines
--- arch/powerpc/include/uapi/asm/kvm.h | 10 ++++++++++ arch/powerpc/kvm/book3s_hv.c | 20 ++++++++++++++++++++ 2 files changed, 30 insertions(+)
Looks good to me. Please feel free to add:
Re: [PATCH v7 2/4] KVM: PPC: Book3S HV: Implement compat CPU capability retrieval for KVM on PowerVM
Quoted 48 lines · Ritesh Harjani
Quoted 41 lines · Amit Machhiwal
On POWER systems, the host CPU may run in a compatibility mode (e.g., a Power11 processor operating in Power10 compatibility mode). In such cases, the effective CPU level exposed to guests differs from the physical processor generation.
When running nested KVM guests, QEMU derives the host CPU type using
mfpvr(), which reflects the physical processor version. This can result in a mismatch between the CPU model selected by QEMU and the compatibility mode enforced by the host, leading to guest boot failures.For example, booting a nested guest on a Power11 LPAR configured in Power10 compatibility mode fails with:
KVM-NESTEDv2: couldn't set guest wide elements
[..KVM reg dump..]This occurs because QEMU selects a CPU model corresponding to the physical processor (via
mfpvr()), while the host operates in a lower compatibility mode. As a result, KVM rejects the requested compatibility level during guest initialization.On pseries nestedv2 systems, add support for retrieving host CPU compatibility capabilities for nested guests on PowerVM. The capability bitmap reflects the processor modes negotiated between the Power hypervisor (L0) and the host partition (L1) via the
H_GUEST_GET_CAPABILITIEShcall, but is retrieved from the cachednested_capabilitiesvalue populated during module initialization, avoiding repeated hypervisor calls. AWARN_ON_ONCE()flags the unexpected case wherenested_capabilitiesis zero on a nestedv2 system. The implementation defines KVM-specific capability constants (KVM_PPC_COMPAT_CAP_POWER9/10/11), masks unsupported bits, and exposes the result through theKVM_PPC_GET_COMPAT_CAPSioctl.Hook the implementation into the Book3S HV
kvmppc_opsso that it can be invoked by the generic KVM ioctl handling code.Suggested-by Vaibhav Jain <[email protected]>Tested-by Gautam Menghani <[email protected]>Reviewed-by Gautam Menghani <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>Quoted 4 lines
--- arch/powerpc/include/uapi/asm/kvm.h | 10 ++++++++++ arch/powerpc/kvm/book3s_hv.c | 20 ++++++++++++++++++++ 2 files changed, 30 insertions(+)
Quoted 3 lines
Looks good to me. Please feel free to add:
Reviewed-by Ritesh Harjani (IBM) <[email protected]>
Thanks for the review, Ritesh!
~Amit
PATCH v7 3/4 KVM: PPC: Book3S HV: Add support for compat CPU capabilities for KVM on PowerNV
In reply to Amit MachhiwalCurrently, when booting a compatibility-mode KVM guest (L1) on a PowerNV hypervisor (L0), the guest runs with the expected processor compatibility level. However, when booting a nested KVM guest (L2) inside the L1, QEMU derives the CPU model from the raw host PVR and attempts to run the nested guest at that level, instead of honoring the compatibility mode of the L1.
Extend host CPU compatibility capability reporting to support nested virtualization on PowerNV systems (PAPR nested API v1).
For nested API v2 (PowerVM), compatibility capabilities are served from the cached nested_capabilities value (populated at module init via kvmhv_nested_init() using the H_GUEST_GET_CAPABILITIES hcall). This information is not available on PowerNV systems.
For nested API v1, derive the compatibility capabilities from the L1 guest by reading the "cpu-version" property from the device tree, which reflects the effective (logical) processor compatibility level. Map this value to the corresponding compatibility capability bitmap using KVM-specific constants.
The mapping is cumulative: a system running at a given compatibility level is assumed to also support older generations down the supported chain. Note that unlike KVM on PowerVM (nested API v2), KVM on PowerNV currently does not strictly enforce older generation compatibility modes for nested guests - the reported capabilities reflect what the host CPU can present, not what the hypervisor independently validates.
Introduce a helper kvmppc_map_compat_capabilities() to translate CPU version values into KVM_PPC_COMPAT_CAP bits using a fallthrough switch, and integrate it into kvmppc_get_compat_caps(). The implementation applies masking to ensure only supported processor modes are exposed.
This allows userspace to query host CPU compatibility modes on both KVM on PowerVM and on PowerNV platforms via the KVM_PPC_GET_COMPAT_CAPS ioctl.
--- arch/powerpc/kvm/book3s_hv.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
arch/powerpc/kvm/book3s_hv.c +36 -0
@@ -6523,20 +6523,56 @@ static bool kvmppc_hash_v3_possible(void)return true; } +static int kvmppc_map_compat_capabilities (u32 cpu_version, +unsigned long *capabilities) +{ +switch (cpu_version) { +case PVR_ARCH_31_P11 : + *capabilities |=KVM_PPC_COMPAT_CAP_POWER11 ; + fallthrough; +case PVR_ARCH_31 : + *capabilities |=KVM_PPC_COMPAT_CAP_POWER10 ; + fallthrough; +case PVR_ARCH_300 : + *capabilities |=KVM_PPC_COMPAT_CAP_POWER9 ; +break ; +default : +return -EINVAL ; + } + +return 0 ; +}static int kvmppc_get_compat_caps (struct kvm_ppc_compat_caps *host_caps) { +struct device_node *np;unsigned long capabilities =0 ;long rc = -EINVAL ; +u32 cpu_version =0 ;if (kvmhv_on_pseries ()) {if (kvmhv_is_nestedv2 ()) {WARN_ON_ONCE (!nested_capabilities); capabilities = nested_capabilities; rc =0 ; + }else { +for_each_node_by_type (np,"cpu" ) { +if (!of_property_read_u32 (np,"cpu-version" , + &cpu_version)) { +of_node_put (np); +break ; + } + } +if (!cpu_version) +return -EINVAL ; + rc =kvmppc_map_compat_capabilities (cpu_version, + &capabilities); } } +if (rc <0 ) +return rc; + host_caps->compat_capabilities = capabilities &KVM_PPC_COMPAT_BITMASK ;return rc;
Signature · 1 line
2.50.1 (Apple Git-155)
Ritesh Harjani (IBM)
reviewerRe: [PATCH v7 3/4] KVM: PPC: Book3S HV: Add support for compat CPU capabilities for KVM on PowerNV
Quoted 84 lines · Amit Machhiwal
Currently, when booting a compatibility-mode KVM guest (L1) on a PowerNV hypervisor (L0), the guest runs with the expected processor compatibility level. However, when booting a nested KVM guest (L2) inside the L1, QEMU derives the CPU model from the raw host PVR and attempts to run the nested guest at that level, instead of honoring the compatibility mode of the L1.
Extend host CPU compatibility capability reporting to support nested virtualization on PowerNV systems (PAPR nested API v1).
For nested API v2 (PowerVM), compatibility capabilities are served from the cached
nested_capabilitiesvalue (populated at module init viakvmhv_nested_init()using theH_GUEST_GET_CAPABILITIEShcall). This information is not available on PowerNV systems.For nested API v1, derive the compatibility capabilities from the L1 guest by reading the "cpu-version" property from the device tree, which reflects the effective (logical) processor compatibility level. Map this value to the corresponding compatibility capability bitmap using KVM-specific constants.
The mapping is cumulative: a system running at a given compatibility level is assumed to also support older generations down the supported chain. Note that unlike KVM on PowerVM (nested API v2), KVM on PowerNV currently does not strictly enforce older generation compatibility modes for nested guests - the reported capabilities reflect what the host CPU can present, not what the hypervisor independently validates.
Introduce a helper
kvmppc_map_compat_capabilities()to translate CPU version values intoKVM_PPC_COMPAT_CAPbits using a fallthrough switch, and integrate it intokvmppc_get_compat_caps(). The implementation applies masking to ensure only supported processor modes are exposed.This allows userspace to query host CPU compatibility modes on both KVM on PowerVM and on PowerNV platforms via the
KVM_PPC_GET_COMPAT_CAPSioctl.Suggested-by Vaibhav Jain <[email protected]>Tested-by Gautam Menghani <[email protected]>Reviewed-by Gautam Menghani <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- arch/powerpc/kvm/book3s_hv.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)arch/powerpc/kvm/book3s_hv.c +21
-0@@ -6523,20 +6523,56 @@ static bool kvmppc_hash_v3_possible(void)return true; } +static int kvmppc_map_compat_capabilities (u32 cpu_version, +unsigned long *capabilities) +{ +switch (cpu_version) { +case PVR_ARCH_31_P11 : + *capabilities |=KVM_PPC_COMPAT_CAP_POWER11 ; + fallthrough; +case PVR_ARCH_31 : + *capabilities |=KVM_PPC_COMPAT_CAP_POWER10 ; + fallthrough; +case PVR_ARCH_300 : + *capabilities |=KVM_PPC_COMPAT_CAP_POWER9 ; +break ; +default : +return -EINVAL ; + } + +return 0 ; +}static int kvmppc_get_compat_caps (struct kvm_ppc_compat_caps *host_caps) { +struct device_node *np;unsigned long capabilities =0 ;long rc = -EINVAL ; +u32 cpu_version =0 ;if (kvmhv_on_pseries ()) {if (kvmhv_is_nestedv2 ()) {WARN_ON_ONCE (!nested_capabilities);
arch/powerpc/kvm/book3s_hv.c static bool kvmppc_hash_v3_possible(void) · line 6565
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +8
-0capabilities = nested_capabilities; rc =0 ; + }else { +for_each_node_by_type (np,"cpu" ) { +if (!of_property_read_u32 (np,"cpu-version" , + &cpu_version)) { +of_node_put (np); +break ; + } + }
What happens when we don't have "cpu-version" DT property?
Check this commit5a61ef74f269f2 ("powerpc/64s: Support new device tree binding for discovering CPU features")
And also why do we need to parse the DT properties again? Shouldn't we check something like this?
if (cpu_has_feature(CPU_FTR_P11_PVR)) capabilities |= KVM_PPC_COMPAT_CAP_POWER11; if (cpu_has_feature(CPU_FTR_ARCH_31)) capabilities |= KVM_PPC_COMPAT_CAP_POWER10; if (cpu_has_feature(CPU_FTR_ARCH_300)) capabilities |= KVM_PPC_COMPAT_CAP_POWER9;
-ritesh
Quoted 15 lines
arch/powerpc/kvm/book3s_hv.c +7
-0+if (!cpu_version) +return -EINVAL ; + rc =kvmppc_map_compat_capabilities (cpu_version, + &capabilities); } } +if (rc <0 ) +return rc; + host_caps->compat_capabilities = capabilities &KVM_PPC_COMPAT_BITMASK ;return rc;Signature · 1 line
2.50.1 (Apple Git-155)
Re: [PATCH v7 3/4] KVM: PPC: Book3S HV: Add support for compat CPU capabilities for KVM on PowerNV
Quoted 1 line · Ritesh Harjani
Amit Machhiwal <[email protected]> writes:
<snip>
Quoted 29 lines
Quoted 11 lines
static int
kvmppc_get_compat_caps(struct kvm_ppc_compat_caps *host_caps){arch/powerpc/kvm/book3s_hv.c +2
-0+struct device_node *np;unsigned long capabilities =0 ;long rc = -EINVAL ; +u32 cpu_version =0 ;if (kvmhv_on_pseries ()) {if (kvmhv_is_nestedv2 ()) {WARN_ON_ONCE (!nested_capabilities);arch/powerpc/kvm/book3s_hv.c static bool kvmppc_hash_v3_possible(void) · line 6565
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +8
-0capabilities = nested_capabilities; rc =0 ; + }else { +for_each_node_by_type (np,"cpu" ) { +if (!of_property_read_u32 (np,"cpu-version" , + &cpu_version)) { +of_node_put (np); +break ; + } + }What happens when we don't have "cpu-version" DT property?
Check this commit5a61ef74f269f2("powerpc/64s: Support new device tree binding for discovering CPU features")And also why do we need to parse the DT properties again? Shouldn't we check something like this?
Quoted 7 lines
if (cpu_has_feature(CPU_FTR_P11_PVR)) capabilities |= KVM_PPC_COMPAT_CAP_POWER11; if (cpu_has_feature(CPU_FTR_ARCH_31)) capabilities |= KVM_PPC_COMPAT_CAP_POWER10; if (cpu_has_feature(CPU_FTR_ARCH_300)) capabilities |= KVM_PPC_COMPAT_CAP_POWER9;
Thanks for the suggestion, Ritesh! After closer analysis, cpu_has_feature() would give the same result on pseries, but I believe that using 'cpu-version' directly is both more correct and more explicit for this context.
Here's why:
1. 'ibm,powerpc-cpu-features' / dt-cpu-ftrs is baremetal (powernv/OPAL) only. SLOF firmware used by pseries guests never provides that node. On pseries, dt_cpu_ftrs_in_use() is always false, and CPU feature bits including CPU_FTR_ARCH_300, CPU_FTR_ARCH_31, CPU_FTR_P11_PVR are set by identify_cpu() from the cpu-version DT property at prom.c:423:
if (!dt_cpu_ftrs_in_use()) {
prop = of_get_flat_dt_prop(node, "cpu-version", NULL);
if (prop && (be32_to_cpup(prop) & 0xff000000) == 0x0f000000) {
identify_cpu(0, be32_to_cpup(prop)); So cpu_has_feature() on pseries is just an indirect readback of what identify_cpu() already derived from 'cpu-version' — the source of truth is still cpu-version.
2. Commit e4de1b9cb3b5 ("powerpc/dt_cpu_ftrs: Set CPU_FTR_P11_PVR for Power11 and later processors") explicitly documents this split:
"This issue does not affect pseries guests, where SLOF firmware
does not provide this node, causing the kernel to fall back to the traditional cputable path (identify_cpu) which correctly sets CPU_FTR_P11_PVR during PVR-based CPU identification."
That fix was needed on powernv only — and our code is in the kvmhv_on_pseries() branch, so the dt-cpu-ftrs path is never taken.
3. 'cpu-version' is the PAPR-defined compat level indicator — it is what PHYP and QEMU explicitly set to communicate the negotiated compat mode. Reading it directly is semantically correct: we are reporting the compat level the hypervisor advertised, not a kernel-internal feature bit derived from it.
I also responded to a similar concern from Sashiko covering points 1 and 3 here:
https://lore.kernel.org/all/[email protected]/
So I'd prefer to keep the direct cpu-version lookup.
Thanks,
Amit
Re: [PATCH v7 3/4] KVM: PPC: Book3S HV: Add support for compat CPU capabilities for KVM on PowerNV
Quoted 42 lines · Amit Machhiwal
Quoted 1 line · Ritesh Harjani
Amit Machhiwal <[email protected]> writes:
<snip>
Quoted 29 lines
Quoted 11 lines
static int
kvmppc_get_compat_caps(struct kvm_ppc_compat_caps *host_caps){arch/powerpc/kvm/book3s_hv.c +2
-0+struct device_node *np;unsigned long capabilities =0 ;long rc = -EINVAL ; +u32 cpu_version =0 ;if (kvmhv_on_pseries ()) {if (kvmhv_is_nestedv2 ()) {WARN_ON_ONCE (!nested_capabilities);arch/powerpc/kvm/book3s_hv.c static bool kvmppc_hash_v3_possible(void) · line 6565
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +8
-0capabilities = nested_capabilities; rc =0 ; + }else { +for_each_node_by_type (np,"cpu" ) { +if (!of_property_read_u32 (np,"cpu-version" , + &cpu_version)) { +of_node_put (np); +break ; + } + }What happens when we don't have "cpu-version" DT property?
Check this commit5a61ef74f269f2("powerpc/64s: Support new device tree binding for discovering CPU features")And also why do we need to parse the DT properties again? Shouldn't we check something like this?
Quoted 7 lines
if (cpu_has_feature(CPU_FTR_P11_PVR)) capabilities |= KVM_PPC_COMPAT_CAP_POWER11; if (cpu_has_feature(CPU_FTR_ARCH_31)) capabilities |= KVM_PPC_COMPAT_CAP_POWER10; if (cpu_has_feature(CPU_FTR_ARCH_300)) capabilities |= KVM_PPC_COMPAT_CAP_POWER9;
Quoted 7 lines
Thanks for the suggestion, Ritesh! After closer analysis,
cpu_has_feature()would give the same result on pseries, but I believe that using 'cpu-version' directly is both more correct and more explicit for this context.Here's why:
1. 'ibm,powerpc-cpu-features' / dt-cpu-ftrs is baremetal (powernv/OPAL) only.
Arghh. That make sense. Thanks for checking. The design with "cpu-version" sounds good in that case.
Please feel free to add:
PATCH v7 4/4 KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl
In reply to Amit MachhiwalAdd documentation for the KVM_PPC_GET_COMPAT_CAPS ioctl to the KVM API documentation.
The ioctl exposes host processor compatibility modes supported for nested KVM guests on PowerPC systems. The documentation covers error code descriptions including E2BIG for forward compatibility, the extensible size-based versioning contract using KVM_PPC_COMPAT_CAPS_SIZE_VER0, the rationale for rejecting non-zero reserved fields to prevent ABI ambiguity, bit numbering clarification for IBM MSB-0 convention, and KVM-specific capability bit constants.
--- Documentation/virt/kvm/api.rst | 79 ++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+)
Documentation/virt/kvm/api.rst +79 -0
@@ -6566,6 +6566,85 @@ KVM_S390_KEYOP_SSKE Sets the storage key for the guest address ``guest_addr`` to the key specified in ``key``, returning the previous value in ``key``. +4.145 KVM_PPC_GET_COMPAT_CAPS +----------------------------- +:Capability: KVM_CAP_PPC_COMPAT_CAPS +:Architectures: powerpc +:Type: vm ioctl +:Parameters: struct kvm_ppc_compat_caps (in/out) +:Returns: 0 on success, negative value on failure + +Errors include: + + ======== ============================================================ + EFAULT if ``struct kvm_ppc_compat_caps`` cannot be read from or + written to userspace + EINVAL if the ``size`` field is smaller than + ``KVM_PPC_COMPAT_CAPS_SIZE_VER0``, if the ``flags`` field + is non-zero, or if the backend fails to retrieve or map + CPU compatibility capabilities + E2BIG if ``size`` is larger than the kernel's struct size + (new userspace on old kernel); the kernel writes back its + own struct size into the ``size`` field so userspace can + retry with the correct size + ENOTTY if the backend does not implement the ``get_compat_caps`` + operation (e.g., on non-HV KVM implementations where the + required KVM operations are not available) + ======== ============================================================ + +IBM POWER system server-based processors provide a compatibility mode feature +where an Nth generation processor can operate in modes consistent with earlier +generations such as (N-1) and (N-2). + +This ioctl provides userspace with information about the CPU compatibility modes +supported by the current host processor for booting the nested KVM guests on +KVM on PowerNV (nested API v1) and KVM on PowerVM (nested API v2) platforms. + +:: + + struct kvm_ppc_compat_caps { + __u64 size; /* Size of this structure */ + __u64 flags; /* Reserved for future use, must be 0 */ + __u64 compat_capabilities; /* Capabilities supported by the host */ + }; + +Before calling this ioctl, userspace must set the ``size`` field to +``sizeof(struct kvm_ppc_compat_caps)`` and zero the ``flags`` field. +The kernel rejects non-zero ``flags`` with ``-EINVAL`` to prevent +uninitialized stack values from being silently accepted, keeping the +field available for future use without ABI ambiguity. + +The ioctl uses ``copy_struct_from_user()`` and ``copy_struct_to_user()`` +to support extensible versioning: if userspace passes a struct smaller +than the current kernel version (``size >= KVM_PPC_COMPAT_CAPS_SIZE_VER0``), +the kernel zero-pads unknown trailing fields. If userspace passes a larger +struct (``size > sizeof(struct kvm_ppc_compat_caps)``), the kernel writes +back its own struct size into the ``size`` field and returns ``-E2BIG``, +allowing userspace to discover the kernel's struct size and retry. +``KVM_PPC_COMPAT_CAPS_SIZE_VER0`` (24) is a frozen constant marking the +size of the initial struct version. + +The ``compat_capabilities`` bit field describes the processor compatibility +modes supported by the host. The following bits indicate support for specific +processor modes (using IBM's MSB-0 convention where bit 0 is the most +significant bit): + +- ``KVM_PPC_COMPAT_CAP_POWER9`` (bit 1) -- KVM guests can run in Power9 processor mode +- ``KVM_PPC_COMPAT_CAP_POWER10`` (bit 2) -- KVM guests can run in Power10 processor mode +- ``KVM_PPC_COMPAT_CAP_POWER11`` (bit 3) -- KVM guests can run in Power11 processor mode + +.. note:: + + The bit numbering above uses IBM's MSB-0 convention (bit 0 is the most + significant bit). In the actual implementation, these are defined as: + + - ``KVM_PPC_COMPAT_CAP_POWER9`` = ``(1ULL << 62)`` + - ``KVM_PPC_COMPAT_CAP_POWER10`` = ``(1ULL << 61)`` + - ``KVM_PPC_COMPAT_CAP_POWER11`` = ``(1ULL << 60)`` + + Userspace should use the defined constants from ``<linux/kvm.h>`` rather + than hardcoding bit positions. + .. _kvm_run: 5. The kvm_run structure
Signature · 1 line
2.50.1 (Apple Git-155)
[email protected]
AI reviewRe: [PATCH v7 4/4] KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The KVM_PPC_GET_COMPAT_CAPS ioctl overwrites the user-provided size field with the kernel's struct…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The KVM_PPC_GET_COMPAT_CAPS ioctl overwrites the user-provided size field with the kernel's struct size on a successful call, which creates a buffer overflow vulnerability in user space if the struct is reused. --
--- Patch [4]: [PATCH v7 4/4] KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl --- commit eb0bad8f47d2ffecff727c5b9aabfa2d83d1f6de
Author: Amit Machhiwal <[email protected]>
KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl
This commit adds documentation for the KVM_PPC_GET_COMPAT_CAPS ioctl, which exposes host processor compatibility modes for nested KVM guests on PowerPC systems. The documentation describes the structure, error codes, and the extensible versioning mechanism using the size field.
Quoted 5 lines
Documentation/virt/kvm/api.rst +0
-0@@ -6566,6 +6566,85 @@ KVM_S390_KEYOP_SSKE
[ ... ]
Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6625
Quoted 9 lines
Documentation/virt/kvm/api.rst +9
-0+The ioctl uses ``copy_struct_from_user()`` and ``copy_struct_to_user()`` +to support extensible versioning: if userspace passes a struct smaller +than the current kernel version (``size >= KVM_PPC_COMPAT_CAPS_SIZE_VER0``), +the kernel zero-pads unknown trailing fields. If userspace passes a larger +struct (``size > sizeof(struct kvm_ppc_compat_caps)``), the kernel writes +back its own struct size into the ``size`` field and returns ``-E2BIG``, +allowing userspace to discover the kernel's struct size and retry. +``KVM_PPC_COMPAT_CAPS_SIZE_VER0`` (24) is a frozen constant marking the +size of the initial struct version.
[Severity: High]
Does the implementation of this extensible versioning in kvm_arch_vm_ioctl() introduce a potential buffer overflow when userspace passes a smaller struct?
Looking at the code in arch/powerpc/kvm/powerpc.c:kvm_arch_vm_ioctl():
host_caps.size = sizeof(host_caps); r = copy_struct_to_user(argp, usize, &host_caps, sizeof(host_caps), NULL);
If a userspace process compiled with an older, smaller version of the struct (e.g., 24 bytes) calls the ioctl on a newer kernel where the struct has grown to 32 bytes, copy_struct_to_user() will copy up to the user-provided usize (24 bytes).
Because the kernel explicitly executes host_caps.size = sizeof(host_caps) before copying, the first 8 bytes of the user's buffer are overwritten with the kernel's larger size (32).
If userspace reuses this struct for another ioctl call, it would unknowingly pass usize = 32. The kernel would trust this size and write 32 bytes to the 24-byte user buffer, causing an out-of-bounds write in userspace.
Should the kernel only write back its struct size when returning -E2BIG, rather than on the successful path when the user's struct is smaller?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Ritesh Harjani (IBM)
reviewerRe: [PATCH v7 4/4] KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl
In reply to Amit MachhiwalQuoted 41 lines · Amit Machhiwal
Add documentation for the
KVM_PPC_GET_COMPAT_CAPSioctl to the KVM API documentation.The ioctl exposes host processor compatibility modes supported for nested KVM guests on PowerPC systems. The documentation covers error code descriptions including E2BIG for forward compatibility, the extensible size-based versioning contract using
KVM_PPC_COMPAT_CAPS_SIZE_VER0, the rationale for rejecting non-zero reserved fields to prevent ABI ambiguity, bit numbering clarification for IBM MSB-0 convention, and KVM-specific capability bit constants.Tested-by Gautam Menghani <[email protected]>Reviewed-by Gautam Menghani <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- Documentation/virt/kvm/api.rst | 79 ++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+)Documentation/virt/kvm/api.rst +14
-0@@ -6566,6 +6566,85 @@ KVM_S390_KEYOP_SSKE Sets the storage key for the guest address ``guest_addr`` to the key specified in ``key``, returning the previous value in ``key``. +4.145 KVM_PPC_GET_COMPAT_CAPS +----------------------------- +:Capability: KVM_CAP_PPC_COMPAT_CAPS +:Architectures: powerpc +:Type: vm ioctl +:Parameters: struct kvm_ppc_compat_caps (in/out) +:Returns: 0 on success, negative value on failure + +Errors include: + + ======== ============================================================ + EFAULT if ``struct kvm_ppc_compat_caps`` cannot be read from or + written to userspace + EINVAL if the ``size`` field is smaller than
Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6592
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+ ``KVM_PPC_COMPAT_CAPS_SIZE_VER0``, if the ``flags`` field + is non-zero, or if the backend fails to retrieve or map + CPU compatibility capabilities + E2BIG if ``size`` is larger than the kernel's struct size + (new userspace on old kernel); the kernel writes back its + own struct size into the ``size`` field so userspace can + retry with the correct size + ENOTTY if the backend does not implement the ``get_compat_caps`` + operation (e.g., on non-HV KVM implementations where the + required KVM operations are not available)
Amit, this may not be true anymore right after your changes in v7? Can we please update the documentation accordingly as well.
Quoted 18 lines
Documentation/virt/kvm/api.rst +18
-0+ ======== ============================================================ + +IBM POWER system server-based processors provide a compatibility mode feature +where an Nth generation processor can operate in modes consistent with earlier +generations such as (N-1) and (N-2). + +This ioctl provides userspace with information about the CPU compatibility modes +supported by the current host processor for booting the nested KVM guests on +KVM on PowerNV (nested API v1) and KVM on PowerVM (nested API v2) platforms. + +:: + + struct kvm_ppc_compat_caps { + __u64 size; /* Size of this structure */ + __u64 flags; /* Reserved for future use, must be 0 */ + __u64 compat_capabilities; /* Capabilities supported by the host */ + }; +
Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6620
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+Before calling this ioctl, userspace must set the ``size`` field to +``sizeof(struct kvm_ppc_compat_caps)`` and zero the ``flags`` field. +The kernel rejects non-zero ``flags`` with ``-EINVAL`` to prevent +uninitialized stack values from being silently accepted, keeping the +field available for future use without ABI ambiguity. + +The ioctl uses ``copy_struct_from_user()`` and ``copy_struct_to_user()`` +to support extensible versioning: if userspace passes a struct smaller +than the current kernel version (``size >= KVM_PPC_COMPAT_CAPS_SIZE_VER0``), +the kernel zero-pads unknown trailing fields. If userspace passes a larger
So I already requested that we should fix this. We cannot write more bytes than requested by the user, since that memory may not be allocated for this struct in userspace.
On checking Sashiko comments in reply to this patch - I think that is also complaining of the same thing that it could cause buffer overflow.
Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6625
Quoted 5 lines
Documentation/virt/kvm/api.rst +5
-0+struct (``size > sizeof(struct kvm_ppc_compat_caps)``), the kernel writes +back its own struct size into the ``size`` field and returns ``-E2BIG``, +allowing userspace to discover the kernel's struct size and retry. +``KVM_PPC_COMPAT_CAPS_SIZE_VER0`` (24) is a frozen constant marking the +size of the initial struct version.
Once we update the comments in patch-1 - I think we should correct this documentation too accordingly. We should just simply use copy_to|from_user_struct() style for doing this.
Quoted 27 lines
Documentation/virt/kvm/api.rst +22
-0+ +The ``compat_capabilities`` bit field describes the processor compatibility +modes supported by the host. The following bits indicate support for specific +processor modes (using IBM's MSB-0 convention where bit 0 is the most +significant bit): + +- ``KVM_PPC_COMPAT_CAP_POWER9`` (bit 1) -- KVM guests can run in Power9 processor mode +- ``KVM_PPC_COMPAT_CAP_POWER10`` (bit 2) -- KVM guests can run in Power10 processor mode +- ``KVM_PPC_COMPAT_CAP_POWER11`` (bit 3) -- KVM guests can run in Power11 processor mode + +.. note:: + + The bit numbering above uses IBM's MSB-0 convention (bit 0 is the most + significant bit). In the actual implementation, these are defined as: + + - ``KVM_PPC_COMPAT_CAP_POWER9`` = ``(1ULL << 62)`` + - ``KVM_PPC_COMPAT_CAP_POWER10`` = ``(1ULL << 61)`` + - ``KVM_PPC_COMPAT_CAP_POWER11`` = ``(1ULL << 60)`` + + Userspace should use the defined constants from ``<linux/kvm.h>`` rather + than hardcoding bit positions. + .. _kvm_run: 5. The kvm_run structureSignature · 1 line
2.50.1 (Apple Git-155)
Re: [PATCH v7 4/4] KVM: PPC: Document KVM_PPC_GET_COMPAT_CAPS ioctl
Quoted 48 lines · Ritesh Harjani
Quoted 46 lines · Amit Machhiwal
Add documentation for the
KVM_PPC_GET_COMPAT_CAPSioctl to the KVM API documentation.The ioctl exposes host processor compatibility modes supported for nested KVM guests on PowerPC systems. The documentation covers error code descriptions including E2BIG for forward compatibility, the extensible size-based versioning contract using
KVM_PPC_COMPAT_CAPS_SIZE_VER0, the rationale for rejecting non-zero reserved fields to prevent ABI ambiguity, bit numbering clarification for IBM MSB-0 convention, and KVM-specific capability bit constants.Tested-by Gautam Menghani <[email protected]>Reviewed-by Gautam Menghani <[email protected]>Tested-by Anushree Mathur <[email protected]>Signed-off-by Amit Machhiwal <[email protected]>--- Documentation/virt/kvm/api.rst | 79 ++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+)Documentation/virt/kvm/api.rst +19
-0@@ -6566,6 +6566,85 @@ KVM_S390_KEYOP_SSKE Sets the storage key for the guest address ``guest_addr`` to the key specified in ``key``, returning the previous value in ``key``. +4.145 KVM_PPC_GET_COMPAT_CAPS +----------------------------- +:Capability: KVM_CAP_PPC_COMPAT_CAPS +:Architectures: powerpc +:Type: vm ioctl +:Parameters: struct kvm_ppc_compat_caps (in/out) +:Returns: 0 on success, negative value on failure + +Errors include: + + ======== ============================================================ + EFAULT if ``struct kvm_ppc_compat_caps`` cannot be read from or + written to userspace + EINVAL if the ``size`` field is smaller than + ``KVM_PPC_COMPAT_CAPS_SIZE_VER0``, if the ``flags`` field + is non-zero, or if the backend fails to retrieve or map + CPU compatibility capabilities + E2BIG if ``size`` is larger than the kernel's struct size + (new userspace on old kernel); the kernel writes back its
Quoted 8 lines
Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6592
Quoted 5 lines
Documentation/virt/kvm/api.rst +5
-0+ own struct size into the ``size`` field so userspace can + retry with the correct size + ENOTTY if the backend does not implement the ``get_compat_caps`` + operation (e.g., on non-HV KVM implementations where the + required KVM operations are not available)Amit, this may not be true anymore right after your changes in v7? Can we please update the documentation accordingly as well.
Agreed. After dropping the manual pre-check in patch-1 and delegating to copy_struct_from_user(), -E2BIG is no longer unconditional when usize > ksize — it only fires if the unknown trailing bytes are non-zero. Will update the E2BIG entry to:
E2BIG if ``size`` is larger than the kernel's struct size and
the unknown trailing bytes are non-zero (new userspace on old kernel with non-default fields set); the kernel writes back its own struct size into the ``size`` field so userspace can retry with the correct size
Quoted 34 lines
Quoted 19 lines
Documentation/virt/kvm/api.rst +18
-0+ ======== ============================================================ + +IBM POWER system server-based processors provide a compatibility mode feature +where an Nth generation processor can operate in modes consistent with earlier +generations such as (N-1) and (N-2). + +This ioctl provides userspace with information about the CPU compatibility modes +supported by the current host processor for booting the nested KVM guests on +KVM on PowerNV (nested API v1) and KVM on PowerVM (nested API v2) platforms. + +:: + + struct kvm_ppc_compat_caps { + __u64 size; /* Size of this structure */ + __u64 flags; /* Reserved for future use, must be 0 */ + __u64 compat_capabilities; /* Capabilities supported by the host */ + }; +Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6620
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+Before calling this ioctl, userspace must set the ``size`` field to +``sizeof(struct kvm_ppc_compat_caps)`` and zero the ``flags`` field. +The kernel rejects non-zero ``flags`` with ``-EINVAL`` to prevent +uninitialized stack values from being silently accepted, keeping the +field available for future use without ABI ambiguity. + +The ioctl uses ``copy_struct_from_user()`` and ``copy_struct_to_user()`` +to support extensible versioning: if userspace passes a struct smaller +than the current kernel version (``size >= KVM_PPC_COMPAT_CAPS_SIZE_VER0``), +the kernel zero-pads unknown trailing fields. If userspace passes a largerSo I already requested that we should fix this. We cannot write more bytes than requested by the user, since that memory may not be allocated for this struct in userspace.
Quoted 2 lines
On checking Sashiko comments in reply to this patch - I think that is also complaining of the same thing that it could cause buffer overflow.
copy_struct_to_user() itself is safe — it caps its write to min(ksize, usize) bytes so it never writes past the user's buffer. However, the problem is in the value written back in the size field: if usize < sizeof(host_caps) (old userspace, new kernel), we'd write size = sizeof(host_caps) into the first 8 bytes of the user's smaller buffer. If userspace then reuses the struct naively, it would pass usize = sizeof(host_caps) against its smaller allocation, which would cause an actual overflow on the next call.
The fix is to write back usize instead:
host_caps.size = usize;
r = copy_struct_to_user(argp, usize, &host_caps,
sizeof(host_caps), NULL);This tells userspace "I filled exactly as many bytes as you gave me", which is the correct contract for copy_struct_to_user().
Will update both the code in patch-1 and the versioning paragraph in the documentation accordingly in v8.
Thanks,
Amit
Quoted 39 lines
Documentation/virt/kvm/api.rst KVM_S390_KEYOP_SSKE · line 6625
Quoted 7 lines
Documentation/virt/kvm/api.rst +5
-0+struct (``size > sizeof(struct kvm_ppc_compat_caps)``), the kernel writes +back its own struct size into the ``size`` field and returns ``-E2BIG``, +allowing userspace to discover the kernel's struct size and retry. +``KVM_PPC_COMPAT_CAPS_SIZE_VER0`` (24) is a frozen constant marking the +size of the initial struct version.Once we update the comments in patch-1 - I think we should correct this documentation too accordingly. We should just simply use copy_to|from_user_struct() style for doing this.
Quoted 27 lines
Documentation/virt/kvm/api.rst +22
-0+ +The ``compat_capabilities`` bit field describes the processor compatibility +modes supported by the host. The following bits indicate support for specific +processor modes (using IBM's MSB-0 convention where bit 0 is the most +significant bit): + +- ``KVM_PPC_COMPAT_CAP_POWER9`` (bit 1) -- KVM guests can run in Power9 processor mode +- ``KVM_PPC_COMPAT_CAP_POWER10`` (bit 2) -- KVM guests can run in Power10 processor mode +- ``KVM_PPC_COMPAT_CAP_POWER11`` (bit 3) -- KVM guests can run in Power11 processor mode + +.. note:: + + The bit numbering above uses IBM's MSB-0 convention (bit 0 is the most + significant bit). In the actual implementation, these are defined as: + + - ``KVM_PPC_COMPAT_CAP_POWER9`` = ``(1ULL << 62)`` + - ``KVM_PPC_COMPAT_CAP_POWER10`` = ``(1ULL << 61)`` + - ``KVM_PPC_COMPAT_CAP_POWER11`` = ``(1ULL << 60)`` + + Userspace should use the defined constants from ``<linux/kvm.h>`` rather + than hardcoding bit positions. + .. _kvm_run: 5. The kvm_run structureSignature · 1 line
2.50.1 (Apple Git-155)