Skip to contentSkip to conversation

ppc/kvm: Handle CPU compatibility mode correctly for nested guests

PATCH Superseded Amit Machhiwal 3 participants 7 messages Aug 4 – Aug 5 KVM

v5 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v5 cover ppc/kvm: Handle CPU compatibility mode correctly for nested guests

On POWER systems, newer processor generations can operate in compatibility modes corresponding to earlier generations (e.g., a Power11 system running in Power10 compatibility mode). In such cases, the effective CPU level exposed to guests differs from the physical processor generation.

This creates issues for nested virtualization. When booting a nested KVM guest, QEMU may derive the CPU model from the raw hardware PVR and attempt to configure the guest accordingly. However, the host is constrained by the compatibility level negotiated with the hypervisor, and requests exceeding that level are rejected by KVM, leading to guest boot failures such as:

KVM-NESTEDv2: couldn't set guest wide elements

This series addresses the issue by preventing fallback to raw mode when the host itself is booted in a compatibility mode, and by querying the effective CPU compatibility modes supported by the host via KVM. The kernel interface uses copy_struct_from/to_user() for forward and backward ABI compatibility. With these changes, QEMU ensures that nested guests are configured with CPU models consistent with the host compatibility mode, allowing them to boot correctly.

Patch summary:
[1/3] [DO_NOT_MERGE] linux-headers: Add uapi header changes [2/3] target/ppc/kvm: Add support for querying host compatibility mode [3/3] target/ppc/kvm: Use host compatibility mode for nested guests

Changes in v5:
- Patch 1: Updated KVM_PPC_GET_COMPAT_CAPS ioctl number from 0xe4 to

    0xb8 to match the corresponding Linux kernel v6 series change; the
    0xe0-0xe3 range is reserved for KVM_CREATE_DEVICE fd ioctls

Testing (with kernel v6 patches):

KVM APIv1 Testing
=================
On P10 PowerNV machine (L0)
---------------------------
- P10 L1 KVM guest -> works
- P10 nested L2 KVM guest -> works
- P9 compat nested L2 KVM guest -> works
- P9 compat L1 KVM guest -> works
- P9 nested L2 KVM guest -> works

On Powernv11 TCG Guest (L0)
---------------------------
- P11 PowerNV TCG L0 guest -> works
- P11 L1 KVM guest -> works
- P11 L2 KVM guest -> works
- P10 compat L1 KVM guest -> works
- P10 L2 KVM guest -> works
- P9 compat L1 KVM guest -> works
- P9 L2 KVM guest -> works

KVM APIv2 Testing
=================
On P11 PowerVM LPAR (L1)
------------------------
- P11 L2 KVM guest -> works
- P10 compat L2 KVM guest -> works
- P9 compat L2 KVM guest fails to boot as expected
- Without QEMU patches but Linux patches
- P11 L2 KVM guest -> works
- P10 compat L2 KVM guest -> works
- P9 compat L2 KVM guest fails to boot as expected
- Without Linux patches but QEMU patches
- P11 L2 KVM guest -> works
- P10 compat L2 KVM guest -> works

On P11 LPAR in P10 compat (L1)
------------------------------
- P10 (host compat) L2 KVM guest -> works
- Without QEMU patch but Linux patches
- P10 guest fails to boot as expected (error: kvm run failed Invalid argument)
- Without Linux patch but QEMU patches
- P10 guest fails to boot as expected (KVM: unknown exit, hardware reason ffffffffffffffea)

On P10 PowerVM LPAR (L1)
------------------------
- P10 L2 KVM guest -> works
- P9 compat L2 KVM guest fails to boot as expected

TCG pSeries Guest
=================
- P11 (default) pSeries guest boots fine

ABI Extensibility Testing (struct size 32, extra member)
=========================================================
- Newer struct on QEMU, older kernel -> works (kernel returns -E2BIG, QEMU retries with correct size)
- New struct on Linux kernel, older QEMU -> works (kernel zero-pads trailing fields, QEMU gets correct data)

Note: Patch 1 is marked DO_NOT_MERGE as it contains linux-headers updates that will be synced separately once the corresponding kernel patches are merged.

The corresponding Linux kernel patches (v6) are being posted concurrently.

v4: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/

Previous kernel patch versions:
v6: https://lore.kernel.org/all/[email protected]/
v5: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/

Amit Machhiwal (3):
[DO_NOT_MERGE] linux-headers: Add uapi header changes target/ppc/kvm: Add support for querying host compatibility mode target/ppc/kvm: Use host compatibility mode for nested guests

 hw/ppc/spapr_hcall.c            | 14 ++++++
 linux-headers/asm-powerpc/kvm.h | 19 +++++++
 linux-headers/linux/kvm.h       |  3 ++
 target/ppc/kvm.c                | 87 +++++++++++++++++++++++++++++++++
 target/ppc/kvm_ppc.h            |  7 +++
 5 files changed, 130 insertions(+)

base-commit: b428fe036233cbd15d37e3c027ab6ca4d3661a80

Signature · 1 line
2.50.1 (Apple Git-155)
#1 lore raw

PATCH v5 1/3 linux-headers: Add uapi header changes

This is a temporary patch intended for review and testing purposes only. It syncs the QEMU linux-headers with the kernel v6 changes that introduce the KVM_PPC_GET_COMPAT_CAPS ioctl for querying host CPU compatibility capabilities. The struct kvm_ppc_compat_caps places 'size' as the first field as required by copy_struct_from/to_user() versioning, with KVM_PPC_COMPAT_CAPS_SIZE_VER0 (24) defined as the frozen version-floor constant. The capability number is KVM_CAP_PPC_COMPAT_CAPS (250) and the ioctl is defined as _IO so the ioctl number remains stable if the struct grows in future versions. Capability bit definitions for POWER9, POWER10, and POWER11 compatibility modes are also included.

The actual header sync will be done via scripts/update-linux-headers.sh once the kernel changes [1] are merged upstream.

[1] https://lore.kernel.org/all/[email protected]/

---
Changes in this version:
- Updated KVM_PPC_GET_COMPAT_CAPS ioctl number from 0xe4 to 0xb8 to match the corresponding Linux kernel v6 series change; the 0xe0-0xe3 range is reserved for KVM_CREATE_DEVICE fd ioctls

 linux-headers/asm-powerpc/kvm.h | 19 +++++++++++++++++++
 linux-headers/linux/kvm.h       |  3 +++
 2 files changed, 22 insertions(+)
linux-headers/asm-powerpc/kvm.h +19 -0
index 077c5437f521..2c229fe0562c 100644
@@ -437,6 +437,25 @@ struct kvm_ppc_cpu_char {
 	__u64	behaviour_mask;		/* valid bits in behaviour */
 };
 
+/* For KVM_PPC_GET_COMPAT_CAPS */
+struct kvm_ppc_compat_caps {
+	__u64	size;			/* Size of this structure */
+	__u64	flags;			/* Reserved for future use */
+	__u64	compat_capabilities;	/* Capabilities supported by the host */
+};
+#define KVM_PPC_COMPAT_CAPS_SIZE_VER0	24 /* sizeof first published struct */
+
+/*
+ * Capability bits for compat_capabilities field in kvm_ppc_compat_caps.
+ * These bits indicate which processor compatibility modes are supported.
+ */
+#define KVM_PPC_COMPAT_CAP_POWER9	(1ULL << 62)
+#define KVM_PPC_COMPAT_CAP_POWER10	(1ULL << 61)
+#define KVM_PPC_COMPAT_CAP_POWER11	(1ULL << 60)
+#define KVM_PPC_COMPAT_BITMASK		(KVM_PPC_COMPAT_CAP_POWER9 | \
+					 KVM_PPC_COMPAT_CAP_POWER10 | \
+					 KVM_PPC_COMPAT_CAP_POWER11)
+
 /*
  * Values for character and character_mask.
  * These are identical to the values used by H_GET_CPU_CHARACTERISTICS.
linux-headers/linux/kvm.h +3 -0
index dd52e2a65bfd..aea4ab5c6953 100644
@@ -986,6 +986,7 @@ struct kvm_enable_cap {
 #define KVM_CAP_S390_KEYOP 247
 #define KVM_CAP_S390_VSIE_ESAMODE 248
 #define KVM_CAP_S390_HPAGE_2G 249
+#define KVM_CAP_PPC_COMPAT_CAPS 250
 
 struct kvm_irq_routing_irqchip {
 	__u32 irqchip;
@@ -1330,6 +1331,8 @@ struct kvm_s390_keyop {
 /* Available with KVM_CAP_COUNTER_OFFSET */
 #define KVM_ARM_SET_COUNTER_OFFSET _IOW(KVMIO,  0xb5, struct kvm_arm_counter_offset)
 #define KVM_ARM_GET_REG_WRITABLE_MASKS _IOR(KVMIO,  0xb6, struct reg_mask_range)
+/* Available with KVM_CAP_PPC_COMPAT_CAPS */
+#define KVM_PPC_GET_COMPAT_CAPS	_IO(KVMIO,  0xb8)
 
 /* ioctl for vm fd */
 #define KVM_CREATE_DEVICE	  _IOWR(KVMIO,  0xe0, struct kvm_create_device)
Signature · 1 line
2.50.1 (Apple Git-155)
#2 lore raw

PATCH v5 2/3 target/ppc/kvm: Add support for querying host compatibility mode

In reply to Amit Machhiwal

Add infrastructure to query the host CPU compatibility mode via the KVM_PPC_GET_COMPAT_CAPS ioctl. This allows QEMU to determine if the host is running in a compatibility mode (e.g., a Power11 processor operating in Power10 compatibility mode).

The kvmppc_get_compat_caps() function issues the ioctl and returns the compat_capabilities bitmap. The kvm_ppc_host_compat_pvr() function derives the effective PVR from the bitmap using ctz64() to find the lowest set bit (highest supported compat level in IBM MSB-0 numbering).

The struct kvm_ppc_compat_caps places 'size' first and userspace sets it to sizeof(struct kvm_ppc_compat_caps) before calling the ioctl. The kernel uses copy_struct_from/to_user() to handle forward and backward ABI
compatibility: an older userspace with a smaller struct gets trailing fields zero-padded. When newer userspace passes a larger struct to an older kernel (usize > ksize), the kernel unconditionally returns -E2BIG and writes its own ksize back into host_compat.size. QEMU detects this, validates the returned size against KVM_PPC_COMPAT_CAPS_SIZE_VER0, and retries with that size.

Additionally, cas_check_pvr() in hw/ppc/spapr_hcall.c is updated to prevent fallback to raw mode when the host is running in compatibility mode. This ensures that nested guests cannot exceed the host's compatibility level. The call is guarded with kvm_enabled() since kvm_ppc_host_compat_pvr() invokes kvm_vm_ioctl() which dereferences kvm_state; without the guard, a TCG guest on a CONFIG_KVM=y binary would segfault.

If the capability is not supported or the query fails, the functions return 0, allowing fallback to existing behavior.

---
No changes in this version.

 hw/ppc/spapr_hcall.c | 14 +++++++++
 target/ppc/kvm.c     | 75 ++++++++++++++++++++++++++++++++++++++++++++
 target/ppc/kvm_ppc.h |  7 +++++
 3 files changed, 96 insertions(+)
hw/ppc/spapr_hcall.c +14 -0
index 23bcd788daf6..708902934cff 100644
@@ -1136,6 +1136,7 @@ static uint32_t cas_check_pvr(PowerPCCPU *cpu, uint32_t max_compat,
 {
     bool explicit_match = false; /* Matched the CPU's real PVR */
     uint32_t best_compat = 0;
+    uint32_t compat_host_pvr = 0;
     int i;
 
     /*
@@ -1163,6 +1164,19 @@ static uint32_t cas_check_pvr(PowerPCCPU *cpu, uint32_t max_compat,
         }
     }
 
+    if (explicit_match && kvm_enabled()) {
+        compat_host_pvr = kvm_ppc_host_compat_pvr();
+        /*
+         * If the host is booted in a compatibility mode, do not try booting in
+         * the raw mode as it may allow KVM guests to boot with a higher CPU
+         * version compared to what host was booted with; which should not be
+         * allowed.
+         */
+        if (compat_host_pvr) {
+            explicit_match = false;
+        }
+    }
+
     *raw_mode_supported = explicit_match;
 
     /* Parsing finished */
target/ppc/kvm.c +75 -0

81 lines not on this page · raw message

target/ppc/kvm_ppc.h +7 -0
index 742881231e16..195dbaac5e17 100644
@@ -81,6 +81,8 @@ bool kvmppc_supports_ail_3(void);
 int kvmppc_enable_hwrng(void);
 int kvmppc_put_books_sregs(PowerPCCPU *cpu);
 PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void);
+
+uint32_t kvm_ppc_host_compat_pvr(void);
 void kvmppc_check_papr_resize_hpt(Error **errp);
 int kvmppc_resize_hpt_prepare(PowerPCCPU *cpu, target_ulong flags, int shift);
 int kvmppc_resize_hpt_commit(PowerPCCPU *cpu, target_ulong flags, int shift);
@@ -440,6 +442,11 @@ static inline PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void)
     return NULL;
 }
 
+static inline uint32_t kvm_ppc_host_compat_pvr(void)
+{
+    return 0;
+}
+
 static inline void kvmppc_check_papr_resize_hpt(Error **errp)
 {
 }
Signature · 1 line
2.50.1 (Apple Git-155)
#3 lore raw

PATCH v5 3/3 target/ppc/kvm: Use host compatibility mode for nested guests

In reply to Amit Machhiwal

On POWER systems, the host CPU may run in a compatibility mode (e.g., a Power11 processor operating in Power10 compatibility mode). When running nested KVM guests, QEMU currently derives the host CPU type using mfpvr(), which reflects the physical processor version. This can result in a mismatch between the CPU model used by QEMU and the compatibility mode enforced by the host, leading to guest boot failures such as "KVM-NESTEDv2: couldn't set guest wide elements".

Update kvm_ppc_get_host_cpu_class() to check if the host is running in a compatibility mode using kvm_ppc_host_compat_pvr(). When available, use the compatibility PVR instead of the raw hardware PVR when selecting the CPU model. This ensures that QEMU selects a CPU model consistent with the host compatibility mode, allowing nested guests to boot correctly.

The guard uses #if defined(TARGET_PPC64) to prevent build breakage on ppc32 targets where the POWER9/10/11 PVR constants are not defined.

---
No changes in this version.

 target/ppc/kvm.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
target/ppc/kvm.c +12 -0
index d4c5601a00c4..aa8269186a1a 100644
@@ -2682,6 +2682,18 @@ PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void)
     uint32_t host_pvr = mfpvr();
     PowerPCCPUClass *pvr_pcc;
 
+#if defined(TARGET_PPC64)
+#ifndef CONFIG_KVM
+#error "CONFIG_KVM is not enabled"
+#endif
+    uint32_t compat_host_pvr;
+
+    compat_host_pvr = kvm_ppc_host_compat_pvr();
+    if (compat_host_pvr) {
+        host_pvr = compat_host_pvr;
+    }
+#endif /* TARGET_PPC64 */
+
     pvr_pcc = ppc_cpu_class_by_pvr(host_pvr);
     if (pvr_pcc == NULL) {
         pvr_pcc = ppc_cpu_class_by_pvr_mask(host_pvr);
Signature · 1 line
2.50.1 (Apple Git-155)
#4 lore raw

Re: [PATCH v5 3/3] target/ppc/kvm: Use host compatibility mode for nested guests

Quoted 32 lines · Amit Machhiwal

On POWER systems, the host CPU may run in a compatibility mode (e.g., a Power11 processor operating in Power10 compatibility mode). When running nested KVM guests, QEMU currently derives the host CPU type using mfpvr(), which reflects the physical processor version. This can result in a mismatch between the CPU model used by QEMU and the compatibility mode enforced by the host, leading to guest boot failures such as "KVM-NESTEDv2: couldn't set guest wide elements".

Update kvm_ppc_get_host_cpu_class() to check if the host is running in a compatibility mode using kvm_ppc_host_compat_pvr(). When available, use the compatibility PVR instead of the raw hardware PVR when selecting the CPU model. This ensures that QEMU selects a CPU model consistent with the host compatibility mode, allowing nested guests to boot correctly.

The guard uses #if defined(TARGET_PPC64) to prevent build breakage on ppc32 targets where the POWER9/10/11 PVR constants are not defined.

---
No changes in this version.

target/ppc/kvm.c | 12 ++++++++++++

1 file changed, 12 insertions(+)
target/ppc/kvm.c +0 -0
index d4c5601a00c4..aa8269186a1a 100644

target/ppc/kvm.c PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void) · line 2688

Quoted 7 lines
target/ppc/kvm.c +4 -0
@@ -2682,6 +2682,18 @@ PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void)
    uint32_t host_pvr = mfpvr();
    PowerPCCPUClass *pvr_pcc;
 
+#if defined(TARGET_PPC64)
+#ifndef CONFIG_KVM
+#error "CONFIG_KVM is not enabled"
+#endif

In meson.build:

ppc_system_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'))

so I think this check and #error does not make sense.

Regards,
BALATON Zoltan

Quoted 12 lines
target/ppc/kvm.c +8 -0
+    uint32_t compat_host_pvr;
+
+    compat_host_pvr = kvm_ppc_host_compat_pvr();
+    if (compat_host_pvr) {
+        host_pvr = compat_host_pvr;
+    }
+#endif /* TARGET_PPC64 */
+
    pvr_pcc = ppc_cpu_class_by_pvr(host_pvr);
    if (pvr_pcc == NULL) {
        pvr_pcc = ppc_cpu_class_by_pvr_mask(host_pvr);
#5 lore raw

Re: [PATCH v5 3/3] target/ppc/kvm: Use host compatibility mode for nested guests

Hi BALATON,

Thanks for reviewing this patch. Please find my response inline below.

Quoted 43 lines · BALATON Zoltan
Quoted 32 lines · Amit Machhiwal

On POWER systems, the host CPU may run in a compatibility mode (e.g., a Power11 processor operating in Power10 compatibility mode). When running nested KVM guests, QEMU currently derives the host CPU type using mfpvr(), which reflects the physical processor version. This can result in a mismatch between the CPU model used by QEMU and the compatibility mode enforced by the host, leading to guest boot failures such as "KVM-NESTEDv2: couldn't set guest wide elements".

Update kvm_ppc_get_host_cpu_class() to check if the host is running in a compatibility mode using kvm_ppc_host_compat_pvr(). When available, use the compatibility PVR instead of the raw hardware PVR when selecting the CPU model. This ensures that QEMU selects a CPU model consistent with the host compatibility mode, allowing nested guests to boot correctly.

The guard uses #if defined(TARGET_PPC64) to prevent build breakage on ppc32 targets where the POWER9/10/11 PVR constants are not defined.

---
No changes in this version.

target/ppc/kvm.c | 12 ++++++++++++

1 file changed, 12 insertions(+)
target/ppc/kvm.c +0 -0
index d4c5601a00c4..aa8269186a1a 100644

target/ppc/kvm.c PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void) · line 2688

Quoted 7 lines
target/ppc/kvm.c +4 -0
@@ -2682,6 +2682,18 @@ PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void)
    uint32_t host_pvr = mfpvr();
    PowerPCCPUClass *pvr_pcc;
 
+#if defined(TARGET_PPC64)
+#ifndef CONFIG_KVM
+#error "CONFIG_KVM is not enabled"
+#endif

In meson.build:

Quoted 3 lines

ppc_system_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'))

so I think this check and #error does not make sense.

You're right that kvm.c is always compiled with CONFIG_KVM enabled due to the meson.build constraint, so the #ifndef CONFIG_KVM / #error check can never fire. I added it as an explicit compile-time assertion to document the assumption, but it is redundant given the build system already enforces it. I'll drop it in the next version.

Thanks,
Amit

Quoted 16 lines

Regards,
BALATON Zoltan

Quoted 12 lines
target/ppc/kvm.c +8 -0
+    uint32_t compat_host_pvr;
+
+    compat_host_pvr = kvm_ppc_host_compat_pvr();
+    if (compat_host_pvr) {
+        host_pvr = compat_host_pvr;
+    }
+#endif /* TARGET_PPC64 */
+
    pvr_pcc = ppc_cpu_class_by_pvr(host_pvr);
    if (pvr_pcc == NULL) {
        pvr_pcc = ppc_cpu_class_by_pvr_mask(host_pvr);
Tested-by on the v5 series
#6 lore raw
In reply to Amit Machhiwal
Quoted 124 lines · Amit Machhiwal

On POWER systems, newer processor generations can operate in compatibility modes corresponding to earlier generations (e.g., a Power11 system running in Power10 compatibility mode). In such cases, the effective CPU level exposed to guests differs from the physical processor generation.

This creates issues for nested virtualization. When booting a nested KVM guest, QEMU may derive the CPU model from the raw hardware PVR and attempt to configure the guest accordingly. However, the host is constrained by the compatibility level negotiated with the hypervisor, and requests exceeding that level are rejected by KVM, leading to guest boot failures such as:

KVM-NESTEDv2: couldn't set guest wide elements

This series addresses the issue by preventing fallback to raw mode when the host itself is booted in a compatibility mode, and by querying the effective CPU compatibility modes supported by the host via KVM. The kernel interface uses copy_struct_from/to_user() for forward and backward ABI compatibility. With these changes, QEMU ensures that nested guests are configured with CPU models consistent with the host compatibility mode, allowing them to boot correctly.

Patch summary:
[1/3] [DO_NOT_MERGE] linux-headers: Add uapi header changes [2/3] target/ppc/kvm: Add support for querying host compatibility mode [3/3] target/ppc/kvm: Use host compatibility mode for nested guests

Changes in v5:
- Patch 1: Updated KVM_PPC_GET_COMPAT_CAPS ioctl number from 0xe4 to

     0xb8 to match the corresponding Linux kernel v6 series change; the
     0xe0-0xe3 range is reserved for KVM_CREATE_DEVICE fd ioctls

Testing (with kernel v6 patches):

KVM APIv1 Testing
=================
On P10 PowerNV machine (L0)
---------------------------
- P10 L1 KVM guest -> works
- P10 nested L2 KVM guest -> works
- P9 compat nested L2 KVM guest -> works
- P9 compat L1 KVM guest -> works
- P9 nested L2 KVM guest -> works

On Powernv11 TCG Guest (L0)
---------------------------
- P11 PowerNV TCG L0 guest -> works
- P11 L1 KVM guest -> works
- P11 L2 KVM guest -> works
- P10 compat L1 KVM guest -> works
- P10 L2 KVM guest -> works
- P9 compat L1 KVM guest -> works
- P9 L2 KVM guest -> works

KVM APIv2 Testing
=================
On P11 PowerVM LPAR (L1)
------------------------
- P11 L2 KVM guest -> works
- P10 compat L2 KVM guest -> works
- P9 compat L2 KVM guest fails to boot as expected
- Without QEMU patches but Linux patches
- P11 L2 KVM guest -> works
- P10 compat L2 KVM guest -> works
- P9 compat L2 KVM guest fails to boot as expected
- Without Linux patches but QEMU patches
- P11 L2 KVM guest -> works
- P10 compat L2 KVM guest -> works

On P11 LPAR in P10 compat (L1)
------------------------------
- P10 (host compat) L2 KVM guest -> works
- Without QEMU patch but Linux patches
- P10 guest fails to boot as expected (error: kvm run failed Invalid argument)
- Without Linux patch but QEMU patches
- P10 guest fails to boot as expected (KVM: unknown exit, hardware reason ffffffffffffffea)

On P10 PowerVM LPAR (L1)
------------------------
- P10 L2 KVM guest -> works
- P9 compat L2 KVM guest fails to boot as expected

TCG pSeries Guest
=================
- P11 (default) pSeries guest boots fine

ABI Extensibility Testing (struct size 32, extra member)
=========================================================
- Newer struct on QEMU, older kernel -> works (kernel returns -E2BIG, QEMU retries with correct size)
- New struct on Linux kernel, older QEMU -> works (kernel zero-pads trailing fields, QEMU gets correct data)

Note: Patch 1 is marked DO_NOT_MERGE as it contains linux-headers updates that will be synced separately once the corresponding kernel patches are merged.

The corresponding Linux kernel patches (v6) are being posted concurrently.

v4: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/

Previous kernel patch versions:
v6: https://lore.kernel.org/all/[email protected]/
v5: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/

Amit Machhiwal (3):
[DO_NOT_MERGE] linux-headers: Add uapi header changes target/ppc/kvm: Add support for querying host compatibility mode target/ppc/kvm: Use host compatibility mode for nested guests

  hw/ppc/spapr_hcall.c            | 14 ++++++
  linux-headers/asm-powerpc/kvm.h | 19 +++++++
  linux-headers/linux/kvm.h       |  3 ++
  target/ppc/kvm.c                | 87 +++++++++++++++++++++++++++++++++
  target/ppc/kvm_ppc.h            |  7 +++
  5 files changed, 130 insertions(+)
Quoted 1 line

base-commit: b428fe036233cbd15d37e3c027ab6ca4d3661a80

Hi Amit,
I have tested this patch and it works as expected. Here is my analysis :

I booted a host with Power10 compat mode and tried following scenarios -

lscpu on host :

Architecture:                ppc64le
  Byte Order:                Little Endian
CPU(s):                      8
  On-line CPU(s) list:       0-7
Model name:                  POWER10 (architected), altivec supported

Before applying the patch :

When I am trying to bringup the guest on a compat mode host it was bringing up a Power11 guest and was failing as

[ 1411.578944] [   T2928] KVM-NESTEDv2: couldn't set guest wide elements
[ 1411.578963] [   T2928] vcpu 000000000b9c4155 (0):
[ 1411.578968] [   T2928] pc  = 000000007daf9790  msr = 8000000000103000  trap = ffffffea
[ 1411.578973] [   T2928] r 0 = 8000000000003000  r16 = 0000000000000000
[ 1411.578978] [   T2928] r 1 = 000000007e581e20  r17 = 0000000000000000
[ 1411.578982] [   T2928] r 2 = 000000007db26c00  r18 = 0000000000000000
[ 1411.578985] [   T2928] r 3 = 0000000000000000  r19 = 0000000000000000
[ 1411.578989] [   T2928] r 4 = 0000000002e30c80  r20 = 0000000000000000
[ 1411.578993] [   T2928] r 5 = 000000007df80000  r21 = 0000000000000000
[ 1411.578996] [   T2928] r 6 = 0000000000200000  r22 = 00000000018c5fd6
[ 1411.579000] [   T2928] r 7 = 000000007df80000  r23 = 000000007db21cc0
[ 1411.579003] [   T2928] r 8 = 000000007db6e5d8  r24 = 000000007db66000
[ 1411.579006] [   T2928] r 9 = 000000007e6655d8  r25 = 000000007e665508
[ 1411.579010] [   T2928] r10 = 000000007db6e5d0  r26 = 00000000018c5fd6
[ 1411.579013] [   T2928] r11 = 0000000000003000  r27 = 0000000000000003
[ 1411.579017] [   T2928] r12 = 8000000000000001  r28 = 000000007db6e5e0
[ 1411.579020] [   T2928] r13 = 0000000000000000  r29 = 000000007db224b0
[ 1411.579024] [   T2928] r14 = 0000000000000000  r30 = 000000007daf274c
[ 1411.579028] [   T2928] r15 = 0000000000000000  r31 = 000000007db76000
[ 1411.579033] [   T2928] ctr = 000000007daf1b44  lr  = 000000007daf1b7c
[ 1411.579037] [   T2928] srr0 = 000000007daf9790 srr1 = 8000000000102000
[ 1411.579041] [   T2928] sprg0 = 0000000000000000 sprg1 = 000000000000ff10
[ 1411.579045] [   T2928] sprg2 = 0000000000000000 sprg3 = 0000000000000000
[ 1411.579049] [   T2928] cr = 20000402  xer = 0000000020040000 dsisr = 00000000
[ 1411.579054] [   T2928] dar = 0000000000000000
[ 1411.579057] [   T2928] fault dar = 0000000000000000 dsisr = 00000000
[ 1411.579061] [   T2928] SLB (0 entries):
[ 1411.579064] [   T2928] lpcr = 0040000000020400 sdr1 = 0000000000000000 last_inst = ffffffffffffffff
[ 1411.579069] [   T2928] trap=0xffffffea | pc=0x7daf9790 | msr=0x8000000000103000

After applying this patch along with the kernel built with it's dependent patch (https://lore.kernel.org/all/[email protected]/):

I am able to bringup a guest and it got boot up with Power10 by default:

lscpu on guest -

ltcbonn53-vm2:~ # lscpu
Architecture:                ppc64le
  Byte Order:                Little Endian
CPU(s):                      8
  On-line CPU(s) list:       0-7
Model name:                  POWER10 (architected), altivec supported Model:                     2.0 (pvr 0082 0200)
  Thread(s) per core:        2
  Core(s) per socket:        4
  Socket(s):                 1

Please feel free to add my tested-by:

Thank you,
Anushree Mathur