Skip to contentSkip to conversation

QEMU Support for KVM Planes

RFC PATCH In review Jörg Rödel 2 participants 13 messages Jun 8

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover QEMU Support for KVM Planes

From: Joerg Roedel <[email protected]>

Hi,

here are the required QEMU changes to make use of the KVM Planes interface posted here[1].

The patches are based on QEMU v11.0.0 and can be used to launch an AMD SEV-SNP VM with COCONUT-SVSM + a Linux guest.

To make this work a change to the QEMU command line is required to tell QEMU which plane to target external IRQs to. this is done with the new device-plane property to the machine specification, e.g:

$ qemu-system-x86_64 \
  -enable-kvm \
  -cpu EPYC-v4 \
  -machine q35,confidential-guest-support=sev0,memory-backend=ram1,igvm-cfg=igvm0,kernel-irqchip=split,device-plane=2 \
  -object memory-backend-memfd,id=ram1,size=32G,share=true \
  -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1 \
  -object igvm-cfg,id=igvm0,file=$IGVM_FILE \
  ...

Any feedback appreciated.

-Joerg

[1] https://lore.kernel.org/kvm/[email protected]/T/#t

Joerg Roedel (10):
Update Linux Header for KVM Planes Support accel/kvm: Extend KVMState to carry fds for planes accel/kvm: Extend CPUState to handle Planes
accel: Add nr_planes() op
accel/kvm: Support nr_planes call-back accel/kvm: Handle KVM_PLANE_EVENT_CREATE_CPU event hw/core/machine: Add device-plane property
qdev: Add plane property
MSI: Inject into correct plane
KVM: Set GSI routes for default plane

 accel/accel-system.c                          |  13 +
 accel/kvm/kvm-all.c                           | 247 ++++++++--
 accel/kvm/trace-events                        |   2 +
 dtc                                           |   1 +
 hw/core/machine.c                             |  22 +
 hw/core/qdev.c                                |  26 +
 hw/i386/kvm/apic.c                            |   6 +-
 hw/pci/msi.c                                  |   3 +
 hw/pci/msix.c                                 |   3 +
 include/accel/accel-ops.h                     |   3 +
 include/hw/core/boards.h                      |   3 +
 include/hw/core/cpu.h                         |  17 +-
 include/hw/core/qdev.h                        |   5 +
 include/hw/pci/msi.h                          |   1 +
 include/qemu/accel.h                          |   7 +
 include/standard-headers/drm/drm_fourcc.h     |  28 +-
 include/standard-headers/linux/const.h        |  18 +
 include/standard-headers/linux/ethtool.h      |  28 +-
 .../linux/input-event-codes.h                 |  13 +
 include/standard-headers/linux/pci_regs.h     |  71 ++-
 include/standard-headers/linux/typelimits.h   |   8 +
 include/standard-headers/linux/virtio_ring.h  |   3 +-
 include/standard-headers/linux/virtio_rtc.h   | 237 ++++++++++
 include/standard-headers/linux/vmclock-abi.h  |  20 +
 include/system/kvm.h                          |   7 +
 include/system/kvm_int.h                      |  30 +-
 linux-headers/asm-arm64/kvm.h                 |   1 +
 linux-headers/asm-arm64/unistd_64.h           |   1 +
 linux-headers/asm-generic/unistd.h            |   5 +-
 linux-headers/asm-loongarch/kvm.h             |   5 +
 linux-headers/asm-loongarch/kvm_para.h        |   1 +
 linux-headers/asm-loongarch/unistd_64.h       |   2 +
 linux-headers/asm-mips/unistd_n32.h           |   1 +
 linux-headers/asm-mips/unistd_n64.h           |   1 +
 linux-headers/asm-mips/unistd_o32.h           |   1 +
 linux-headers/asm-powerpc/unistd_32.h         |   1 +
 linux-headers/asm-powerpc/unistd_64.h         |   1 +
 linux-headers/asm-riscv/kvm.h                 |  11 +-
 linux-headers/asm-riscv/ptrace.h              |  37 ++
 linux-headers/asm-riscv/unistd_32.h           |   1 +
 linux-headers/asm-riscv/unistd_64.h           |   1 +
 linux-headers/asm-s390/unistd_32.h            | 446 ------------------
 linux-headers/asm-s390/unistd_64.h            |   1 +
 linux-headers/asm-x86/kvm.h                   |  21 +-
 linux-headers/asm-x86/unistd_32.h             |   1 +
 linux-headers/asm-x86/unistd_64.h             |   1 +
 linux-headers/asm-x86/unistd_x32.h            |   1 +
 linux-headers/linux/const.h                   |  18 +
 linux-headers/linux/iommufd.h                 |  48 ++
 linux-headers/linux/kvm.h                     |  64 ++-
 linux-headers/linux/mshv.h                    |   4 +-
 linux-headers/linux/psp-sev.h                 |   2 +-
 linux-headers/linux/stddef.h                  |   4 +
 linux-headers/linux/vduse.h                   |  85 +++-
 linux-headers/linux/vfio.h                    |  30 +-
 target/arm/kvm.c                              |   2 +-
 target/i386/kvm/kvm.c                         |  57 +++
 tests/unit/test-qdev-global-props.c           |   5 +
 tests/unit/test-qdev.c                        |   5 +
 ui/keycodemapdb                               |   1 +
 60 files changed, 1150 insertions(+), 538 deletions(-)
 create mode 160000 dtc
 create mode 100644 include/standard-headers/linux/typelimits.h
 create mode 100644 include/standard-headers/linux/virtio_rtc.h
 delete mode 100644 linux-headers/asm-s390/unistd_32.h
 create mode 160000 ui/keycodemapdb
Signature · 1 line
2.53.0
#1 lore raw

RFC PATCH 1/10 Update Linux Header for KVM Planes Support

From: Joerg Roedel <[email protected]>

---
 include/standard-headers/drm/drm_fourcc.h     |  28 +-
 include/standard-headers/linux/const.h        |  18 +
 include/standard-headers/linux/ethtool.h      |  28 +-
 .../linux/input-event-codes.h                 |  13 +
 include/standard-headers/linux/pci_regs.h     |  71 ++-
 include/standard-headers/linux/typelimits.h   |   8 +
 include/standard-headers/linux/virtio_ring.h  |   3 +-
 include/standard-headers/linux/virtio_rtc.h   | 237 ++++++++++
 include/standard-headers/linux/vmclock-abi.h  |  20 +
 linux-headers/asm-arm64/kvm.h                 |   1 +
 linux-headers/asm-arm64/unistd_64.h           |   1 +
 linux-headers/asm-generic/unistd.h            |   5 +-
 linux-headers/asm-loongarch/kvm.h             |   5 +
 linux-headers/asm-loongarch/kvm_para.h        |   1 +
 linux-headers/asm-loongarch/unistd_64.h       |   2 +
 linux-headers/asm-mips/unistd_n32.h           |   1 +
 linux-headers/asm-mips/unistd_n64.h           |   1 +
 linux-headers/asm-mips/unistd_o32.h           |   1 +
 linux-headers/asm-powerpc/unistd_32.h         |   1 +
 linux-headers/asm-powerpc/unistd_64.h         |   1 +
 linux-headers/asm-riscv/kvm.h                 |  11 +-
 linux-headers/asm-riscv/ptrace.h              |  37 ++
 linux-headers/asm-riscv/unistd_32.h           |   1 +
 linux-headers/asm-riscv/unistd_64.h           |   1 +
 linux-headers/asm-s390/unistd_32.h            | 446 ------------------
 linux-headers/asm-s390/unistd_64.h            |   1 +
 linux-headers/asm-x86/kvm.h                   |  21 +-
 linux-headers/asm-x86/unistd_32.h             |   1 +
 linux-headers/asm-x86/unistd_64.h             |   1 +
 linux-headers/asm-x86/unistd_x32.h            |   1 +
 linux-headers/linux/const.h                   |  18 +
 linux-headers/linux/iommufd.h                 |  48 ++
 linux-headers/linux/kvm.h                     |  64 ++-
 linux-headers/linux/mshv.h                    |   4 +-
 linux-headers/linux/psp-sev.h                 |   2 +-
 linux-headers/linux/stddef.h                  |   4 +
 linux-headers/linux/vduse.h                   |  85 +++-
 linux-headers/linux/vfio.h                    |  30 +-
 38 files changed, 729 insertions(+), 493 deletions(-)
 create mode 100644 include/standard-headers/linux/typelimits.h
 create mode 100644 include/standard-headers/linux/virtio_rtc.h
 delete mode 100644 linux-headers/asm-s390/unistd_32.h
include/standard-headers/drm/drm_fourcc.h +22 -6
index b39e197cc79f..4bad457cc2d1 100644
@@ -400,8 +400,8 @@ extern "C" {
  * implementation can multiply the values by 2^6=64. For that reason the padding
  * must only contain zeros.
  * index 0 = Y plane, [15:0] z:Y [6:10] little endian
- * index 1 = Cr plane, [15:0] z:Cr [6:10] little endian
- * index 2 = Cb plane, [15:0] z:Cb [6:10] little endian
+ * index 1 = Cb plane, [15:0] z:Cb [6:10] little endian
+ * index 2 = Cr plane, [15:0] z:Cr [6:10] little endian
  */
 #define DRM_FORMAT_S010	fourcc_code('S', '0', '1', '0') /* 2x2 subsampled Cb (1) and Cr (2) planes 10 bits per channel */
 #define DRM_FORMAT_S210	fourcc_code('S', '2', '1', '0') /* 2x1 subsampled Cb (1) and Cr (2) planes 10 bits per channel */
@@ -413,8 +413,8 @@ extern "C" {
  * implementation can multiply the values by 2^4=16. For that reason the padding
  * must only contain zeros.
  * index 0 = Y plane, [15:0] z:Y [4:12] little endian
- * index 1 = Cr plane, [15:0] z:Cr [4:12] little endian
- * index 2 = Cb plane, [15:0] z:Cb [4:12] little endian
+ * index 1 = Cb plane, [15:0] z:Cb [4:12] little endian
+ * index 2 = Cr plane, [15:0] z:Cr [4:12] little endian
  */
 #define DRM_FORMAT_S012	fourcc_code('S', '0', '1', '2') /* 2x2 subsampled Cb (1) and Cr (2) planes 12 bits per channel */
 #define DRM_FORMAT_S212	fourcc_code('S', '2', '1', '2') /* 2x1 subsampled Cb (1) and Cr (2) planes 12 bits per channel */
@@ -423,8 +423,8 @@ extern "C" {
 /*
  * 3 plane YCbCr
  * index 0 = Y plane, [15:0] Y little endian
- * index 1 = Cr plane, [15:0] Cr little endian
- * index 2 = Cb plane, [15:0] Cb little endian
+ * index 1 = Cb plane, [15:0] Cb little endian
+ * index 2 = Cr plane, [15:0] Cr little endian
  */
 #define DRM_FORMAT_S016	fourcc_code('S', '0', '1', '6') /* 2x2 subsampled Cb (1) and Cr (2) planes 16 bits per channel */
 #define DRM_FORMAT_S216	fourcc_code('S', '2', '1', '6') /* 2x1 subsampled Cb (1) and Cr (2) planes 16 bits per channel */
@@ -1421,6 +1421,22 @@ drm_fourcc_canonicalize_nvidia_format_mod(uint64_t modifier)
 #define DRM_FORMAT_MOD_ARM_16X16_BLOCK_U_INTERLEAVED \
 	DRM_FORMAT_MOD_ARM_CODE(DRM_FORMAT_MOD_ARM_TYPE_MISC, 1ULL)
 
+/*
+ * ARM 64k interleaved modifier
+ *
+ * This is used by ARM Mali v10+ GPUs. With this modifier, the plane is divided
+ * into 64k byte 1:1 or 2:1 -sided tiles. The 64k tiles are laid out linearly.
+ * Each 64k tile is divided into blocks of 16x16 texel blocks, which are
+ * themselves laid out linearly within a 64k tile. Then within each 16x16
+ * block, texel blocks are laid out according to U order, similar to
+ * 16X16_BLOCK_U_INTERLEAVED.
+ *
+ * Note that unlike 16X16_BLOCK_U_INTERLEAVED, the layout does not change
+ * depending on whether a format is compressed or not.
+ */
+#define DRM_FORMAT_MOD_ARM_INTERLEAVED_64K \
+	DRM_FORMAT_MOD_ARM_CODE(DRM_FORMAT_MOD_ARM_TYPE_MISC, 2ULL)
+
 /*
  * Allwinner tiled modifier
  *
include/standard-headers/linux/const.h +18 -0
index 95ede2334204..c6a9d0c9835c 100644
@@ -50,4 +50,22 @@
 
 #define __KERNEL_DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
 
+/*
+ * Divide positive or negative dividend by positive or negative divisor
+ * and round to closest integer. Result is undefined for negative
+ * divisors if the dividend variable type is unsigned and for negative
+ * dividends if the divisor variable type is unsigned.
+ */
+#define __KERNEL_DIV_ROUND_CLOSEST(x, divisor)		\
+({							\
+	__typeof__(x) __x = x;				\
+	__typeof__(divisor) __d = divisor;		\
+							\
+	(((__typeof__(x))-1) > 0 ||			\
+	 ((__typeof__(divisor))-1) > 0 ||		\
+	 (((__x) > 0) == ((__d) > 0))) ?		\
+		(((__x) + ((__d) / 2)) / (__d)) :	\
+		(((__x) - ((__d) / 2)) / (__d));	\
+})
+
 #endif /* _LINUX_CONST_H */
include/standard-headers/linux/ethtool.h +22 -6
index d0f7a63f1099..5d82126cd7e8 100644
@@ -17,11 +17,10 @@
 #include "net/eth.h"
 
 #include "standard-headers/linux/const.h"
+#include "standard-headers/linux/typelimits.h"
 #include "standard-headers/linux/types.h"
 #include "standard-headers/linux/if_ether.h"
 
-#include <limits.h> /* for INT_MAX */
-
 /* All structures exposed to userland should be defined such that they
  * have the same layout for 32-bit and 64-bit userland.
  */
@@ -228,7 +227,7 @@ enum tunable_id {
 	ETHTOOL_ID_UNSPEC,
 	ETHTOOL_RX_COPYBREAK,
 	ETHTOOL_TX_COPYBREAK,
-	ETHTOOL_PFC_PREVENTION_TOUT, /* timeout in msecs */
+	ETHTOOL_PFC_PREVENTION_TOUT, /* both pause and pfc, see man ethtool */
 	ETHTOOL_TX_COPYBREAK_BUF_SIZE,
 	/*
 	 * Add your fresh new tunable attribute above and remember to update
@@ -603,6 +602,8 @@ enum ethtool_link_ext_state {
 	ETHTOOL_LINK_EXT_STATE_POWER_BUDGET_EXCEEDED,
 	ETHTOOL_LINK_EXT_STATE_OVERHEAT,
 	ETHTOOL_LINK_EXT_STATE_MODULE,
+	ETHTOOL_LINK_EXT_STATE_OTP_SPEED_VIOLATION,
+	ETHTOOL_LINK_EXT_STATE_BMC_REQUEST_DOWN,
 };
 
 /* More information in addition to ETHTOOL_LINK_EXT_STATE_AUTONEG. */
@@ -1094,13 +1095,20 @@ enum ethtool_module_fw_flash_status {
  * struct ethtool_gstrings - string set for data tagging
  * @cmd: Command number = %ETHTOOL_GSTRINGS
  * @string_set: String set ID; one of &enum ethtool_stringset
- * @len: On return, the number of strings in the string set
+ * @len: Number of strings in the string set
  * @data: Buffer for strings.  Each string is null-padded to a size of
  *	%ETH_GSTRING_LEN.
  *
  * Users must use %ETHTOOL_GSSET_INFO to find the number of strings in
  * the string set.  They must allocate a buffer of the appropriate
  * size immediately following this structure.
+ *
+ * Setting @len on input is optional (though preferred), but must be zeroed
+ * otherwise.
+ * When set, @len will return the requested count if it matches the actual
+ * count; otherwise, it will be zero.
+ * This prevents issues when the number of strings is different than the
+ * userspace allocation.
  */
 struct ethtool_gstrings {
 	uint32_t	cmd;
@@ -1177,13 +1185,20 @@ struct ethtool_test {
 /**
  * struct ethtool_stats - device-specific statistics
  * @cmd: Command number = %ETHTOOL_GSTATS
- * @n_stats: On return, the number of statistics
+ * @n_stats: Number of statistics
  * @data: Array of statistics
  *
  * Users must use %ETHTOOL_GSSET_INFO or %ETHTOOL_GDRVINFO to find the
  * number of statistics that will be returned.  They must allocate a
  * buffer of the appropriate size (8 * number of statistics)
  * immediately following this structure.
+ *
+ * Setting @n_stats on input is optional (though preferred), but must be zeroed
+ * otherwise.
+ * When set, @n_stats will return the requested count if it matches the actual
+ * count; otherwise, it will be zero.
+ * This prevents issues when the number of stats is different than the
+ * userspace allocation.
  */
 struct ethtool_stats {
 	uint32_t	cmd;
@@ -2190,6 +2205,7 @@ enum ethtool_link_mode_bit_indices {
 #define SPEED_40000		40000
 #define SPEED_50000		50000
 #define SPEED_56000		56000
+#define SPEED_80000		80000
 #define SPEED_100000		100000
 #define SPEED_200000		200000
 #define SPEED_400000		400000
@@ -2200,7 +2216,7 @@ enum ethtool_link_mode_bit_indices {
 
 static inline int ethtool_validate_speed(uint32_t speed)
 {
-	return speed <= INT_MAX || speed == (uint32_t)SPEED_UNKNOWN;
+	return speed <= __KERNEL_INT_MAX || speed == (uint32_t)SPEED_UNKNOWN;
 }
 
 /* Duplex, half or full. */
include/standard-headers/linux/input-event-codes.h +13 -0
index ede79c6ae4f5..dd7c986106e3 100644
@@ -643,6 +643,10 @@
 #define KEY_EPRIVACY_SCREEN_ON		0x252
 #define KEY_EPRIVACY_SCREEN_OFF		0x253
 
+#define KEY_ACTION_ON_SELECTION		0x254	/* AL Action on Selection (HUTRR119) */
+#define KEY_CONTEXTUAL_INSERT		0x255	/* AL Contextual Insertion (HUTRR119) */
+#define KEY_CONTEXTUAL_QUERY		0x256	/* AL Contextual Query (HUTRR119) */
+
 #define KEY_KBDINPUTASSIST_PREV		0x260
 #define KEY_KBDINPUTASSIST_NEXT		0x261
 #define KEY_KBDINPUTASSIST_PREVGROUP		0x262
@@ -891,6 +895,7 @@
 
 #define ABS_VOLUME		0x20
 #define ABS_PROFILE		0x21
+#define ABS_SND_PROFILE		0x22
 
 #define ABS_MISC		0x28
 
@@ -1000,4 +1005,12 @@
 #define SND_MAX			0x07
 #define SND_CNT			(SND_MAX+1)
 
+/*
+ * ABS_SND_PROFILE values
+ */
+
+#define SND_PROFILE_SILENT	0x00
+#define SND_PROFILE_VIBRATE	0x01
+#define SND_PROFILE_RING	0x02
+
 #endif
include/standard-headers/linux/pci_regs.h +65 -6

93 lines not on this page · raw message

include/standard-headers/linux/typelimits.h new file +8 -0
new file mode 100644
index 000000000000..8166c639b518
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _UAPI_LINUX_TYPELIMITS_H
+#define _UAPI_LINUX_TYPELIMITS_H
+
+#define __KERNEL_INT_MAX ((int)(~0U >> 1))
+#define __KERNEL_INT_MIN (-__KERNEL_INT_MAX - 1)
+
+#endif /* _UAPI_LINUX_TYPELIMITS_H */
include/standard-headers/linux/virtio_ring.h +1 -2
index 22f6eb8ca710..7baf1968a360 100644
@@ -31,7 +31,6 @@
  * SUCH DAMAGE.
  *
  * Copyright Rusty Russell IBM Corporation 2007. */
-#include <stdint.h>
 #include "standard-headers/linux/types.h"
 #include "standard-headers/linux/virtio_types.h"
 
@@ -200,7 +199,7 @@ static inline void vring_init(struct vring *vr, unsigned int num, void *p,
 	vr->num = num;
 	vr->desc = p;
 	vr->avail = (struct vring_avail *)((char *)p + num * sizeof(struct vring_desc));
-	vr->used = (void *)(((uintptr_t)&vr->avail->ring[num] + sizeof(__virtio16)
+	vr->used = (void *)(((unsigned long)&vr->avail->ring[num] + sizeof(__virtio16)
 		+ align-1) & ~(align - 1));
 }
 
include/standard-headers/linux/virtio_rtc.h new file +237 -0

237 lines not on this page · raw message

include/standard-headers/linux/vmclock-abi.h +20 -0
index 15b0316cb4cd..fe824badc044 100644
@@ -115,6 +115,17 @@ struct vmclock_abi {
 	 * bit again after the update, using the about-to-be-valid fields.
 	 */
 #define VMCLOCK_FLAG_TIME_MONOTONIC		(1 << 7)
+	/*
+	 * If the VM_GEN_COUNTER_PRESENT flag is set, the hypervisor will
+	 * bump the vm_generation_counter field every time the guest is
+	 * loaded from some save state (restored from a snapshot).
+	 */
+#define VMCLOCK_FLAG_VM_GEN_COUNTER_PRESENT     (1 << 8)
+	/*
+	 * If the NOTIFICATION_PRESENT flag is set, the hypervisor will send
+	 * a notification every time it updates seq_count to a new even number.
+	 */
+#define VMCLOCK_FLAG_NOTIFICATION_PRESENT       (1 << 9)
 
 	uint8_t pad[2];
 	uint8_t clock_status;
@@ -177,6 +188,15 @@ struct vmclock_abi {
 	uint64_t time_frac_sec;		/* Units of 1/2^64 of a second */
 	uint64_t time_esterror_nanosec;
 	uint64_t time_maxerror_nanosec;
+
+	/*
+	 * This field changes to another non-repeating value when the guest
+	 * has been loaded from a snapshot. In addition to handling a
+	 * disruption in time (which will also be signalled through the
+	 * disruption_marker field), a guest may wish to discard UUIDs,
+	 * reset network connections, reseed entropy, etc.
+	 */
+	uint64_t vm_generation_counter;
 };
 
 #endif /*  __VMCLOCK_ABI_H__ */
linux-headers/asm-arm64/kvm.h +1 -0
index 46ffbddab54b..6aefe7973814 100644
@@ -416,6 +416,7 @@ enum {
 #define   KVM_DEV_ARM_ITS_RESTORE_TABLES        2
 #define   KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES	3
 #define   KVM_DEV_ARM_ITS_CTRL_RESET		4
+#define   KVM_DEV_ARM_VGIC_USERSPACE_PPIS	5
 
 /* Device Control API on vcpu fd */
 #define KVM_ARM_VCPU_PMU_V3_CTRL	0
linux-headers/asm-arm64/unistd_64.h +1 -0
index 1ef9c408135b..70b3754a4247 100644
@@ -327,6 +327,7 @@
 #define __NR_file_getattr 468
 #define __NR_file_setattr 469
 #define __NR_listns 470
+#define __NR_rseq_slice_yield 471
 
 
 #endif /* _ASM_UNISTD_64_H */
linux-headers/asm-generic/unistd.h +4 -1
index 942370b3f5d2..a627acc8fb5f 100644
@@ -860,8 +860,11 @@ __SYSCALL(__NR_file_setattr, sys_file_setattr)
 #define __NR_listns 470
 __SYSCALL(__NR_listns, sys_listns)
 
+#define __NR_rseq_slice_yield 471
+__SYSCALL(__NR_rseq_slice_yield, sys_rseq_slice_yield)
+
 #undef __NR_syscalls
-#define __NR_syscalls 471
+#define __NR_syscalls 472
 
 /*
  * 32 bit systems traditionally used different
linux-headers/asm-loongarch/kvm.h +5 -0
index de6c3f18e40a..cd0b5c11ca9c 100644
@@ -105,6 +105,7 @@ struct kvm_fpu {
 #define  KVM_LOONGARCH_VM_FEAT_PV_STEALTIME	7
 #define  KVM_LOONGARCH_VM_FEAT_PTW		8
 #define  KVM_LOONGARCH_VM_FEAT_MSGINT		9
+#define  KVM_LOONGARCH_VM_FEAT_PV_PREEMPT	10
 
 /* Device Control API on vcpu fd */
 #define KVM_LOONGARCH_VCPU_CPUCFG	0
@@ -154,4 +155,8 @@ struct kvm_iocsr_entry {
 #define KVM_DEV_LOONGARCH_PCH_PIC_GRP_CTRL	        0x40000006
 #define KVM_DEV_LOONGARCH_PCH_PIC_CTRL_INIT	        0
 
+#define KVM_DEV_LOONGARCH_DMSINTC_GRP_CTRL		0x40000007
+#define KVM_DEV_LOONGARCH_DMSINTC_MSG_ADDR_BASE		0x0
+#define KVM_DEV_LOONGARCH_DMSINTC_MSG_ADDR_SIZE		0x1
+
 #endif /* __UAPI_ASM_LOONGARCH_KVM_H */
linux-headers/asm-loongarch/kvm_para.h +1 -0
index fd7f40713d49..3fd87a096b66 100644
@@ -15,6 +15,7 @@
 #define CPUCFG_KVM_FEATURE		(CPUCFG_KVM_BASE + 4)
 #define  KVM_FEATURE_IPI		1
 #define  KVM_FEATURE_STEAL_TIME		2
+#define  KVM_FEATURE_PREEMPT		3
 /* BIT 24 - 31 are features configurable by user space vmm */
 #define  KVM_FEATURE_VIRT_EXTIOI	24
 #define  KVM_FEATURE_USER_HCALL		25
linux-headers/asm-loongarch/unistd_64.h +2 -0
index aa5daac4ef90..3a29d86e1dee 100644
@@ -300,6 +300,7 @@
 #define __NR_landlock_create_ruleset 444
 #define __NR_landlock_add_rule 445
 #define __NR_landlock_restrict_self 446
+#define __NR_memfd_secret 447
 #define __NR_process_mrelease 448
 #define __NR_futex_waitv 449
 #define __NR_set_mempolicy_home_node 450
@@ -323,6 +324,7 @@
 #define __NR_file_getattr 468
 #define __NR_file_setattr 469
 #define __NR_listns 470
+#define __NR_rseq_slice_yield 471
 
 
 #endif /* _ASM_UNISTD_64_H */
linux-headers/asm-mips/unistd_n32.h +1 -0
index a33d106dca76..5fa1ee0cb465 100644
@@ -399,5 +399,6 @@
 #define __NR_file_getattr (__NR_Linux + 468)
 #define __NR_file_setattr (__NR_Linux + 469)
 #define __NR_listns (__NR_Linux + 470)
+#define __NR_rseq_slice_yield (__NR_Linux + 471)
 
 #endif /* _ASM_UNISTD_N32_H */
linux-headers/asm-mips/unistd_n64.h +1 -0
index 1bc251e4507c..e1f873d83a5d 100644
@@ -375,5 +375,6 @@
 #define __NR_file_getattr (__NR_Linux + 468)
 #define __NR_file_setattr (__NR_Linux + 469)
 #define __NR_listns (__NR_Linux + 470)
+#define __NR_rseq_slice_yield (__NR_Linux + 471)
 
 #endif /* _ASM_UNISTD_N64_H */
linux-headers/asm-mips/unistd_o32.h +1 -0
index c57175d496c0..8207e9ca4f67 100644
@@ -445,5 +445,6 @@
 #define __NR_file_getattr (__NR_Linux + 468)
 #define __NR_file_setattr (__NR_Linux + 469)
 #define __NR_listns (__NR_Linux + 470)
+#define __NR_rseq_slice_yield (__NR_Linux + 471)
 
 #endif /* _ASM_UNISTD_O32_H */
linux-headers/asm-powerpc/unistd_32.h +1 -0
index a3f4aa2fe20f..1f633601201b 100644
@@ -452,6 +452,7 @@
 #define __NR_file_getattr 468
 #define __NR_file_setattr 469
 #define __NR_listns 470
+#define __NR_rseq_slice_yield 471
 
 
 #endif /* _ASM_UNISTD_32_H */
linux-headers/asm-powerpc/unistd_64.h +1 -0
index d4444557f1ce..87439c53c121 100644
@@ -424,6 +424,7 @@
 #define __NR_file_getattr 468
 #define __NR_file_setattr 469
 #define __NR_listns 470
+#define __NR_rseq_slice_yield 471
 
 
 #endif /* _ASM_UNISTD_64_H */
linux-headers/asm-riscv/kvm.h +7 -4
index 54f3ad7ed2e4..504e73305343 100644
@@ -110,6 +110,10 @@ struct kvm_riscv_timer {
 	__u64 state;
 };
 
+/* Possible states for kvm_riscv_timer */
+#define KVM_RISCV_TIMER_STATE_OFF	0
+#define KVM_RISCV_TIMER_STATE_ON	1
+
 /*
  * ISA extension IDs specific to KVM. This is not the same as the host ISA
  * extension IDs as that is internal to the host and should not be exposed
@@ -192,6 +196,9 @@ enum KVM_RISCV_ISA_EXT_ID {
 	KVM_RISCV_ISA_EXT_ZFBFMIN,
 	KVM_RISCV_ISA_EXT_ZVFBFMIN,
 	KVM_RISCV_ISA_EXT_ZVFBFWMA,
+	KVM_RISCV_ISA_EXT_ZCLSD,
+	KVM_RISCV_ISA_EXT_ZILSD,
+	KVM_RISCV_ISA_EXT_ZALASR,
 	KVM_RISCV_ISA_EXT_MAX,
 };
 
@@ -235,10 +242,6 @@ struct kvm_riscv_sbi_fwft {
 	struct kvm_riscv_sbi_fwft_feature pointer_masking;
 };
 
-/* Possible states for kvm_riscv_timer */
-#define KVM_RISCV_TIMER_STATE_OFF	0
-#define KVM_RISCV_TIMER_STATE_ON	1
-
 /* If you need to interpret the index values, here is the key: */
 #define KVM_REG_RISCV_TYPE_MASK		0x00000000FF000000
 #define KVM_REG_RISCV_TYPE_SHIFT	24
linux-headers/asm-riscv/ptrace.h +37 -0
index a3f8211ede44..cf8764299496 100644
@@ -9,6 +9,7 @@
 #ifndef __ASSEMBLER__
 
 #include <linux/types.h>
+#include <linux/const.h>
 
 #define PTRACE_GETFDPIC		33
 
@@ -127,6 +128,42 @@ struct __riscv_v_regset_state {
  */
 #define RISCV_MAX_VLENB (8192)
 
+struct __sc_riscv_cfi_state {
+	unsigned long ss_ptr;   /* shadow stack pointer */
+};
+
+#define PTRACE_CFI_BRANCH_LANDING_PAD_EN_BIT		0
+#define PTRACE_CFI_BRANCH_LANDING_PAD_LOCK_BIT		1
+#define PTRACE_CFI_BRANCH_EXPECTED_LANDING_PAD_BIT	2
+#define PTRACE_CFI_SHADOW_STACK_EN_BIT			3
+#define PTRACE_CFI_SHADOW_STACK_LOCK_BIT		4
+#define PTRACE_CFI_SHADOW_STACK_PTR_BIT			5
+
+#define PTRACE_CFI_BRANCH_LANDING_PAD_EN_STATE		_BITUL(PTRACE_CFI_BRANCH_LANDING_PAD_EN_BIT)
+#define PTRACE_CFI_BRANCH_LANDING_PAD_LOCK_STATE	\
+	_BITUL(PTRACE_CFI_BRANCH_LANDING_PAD_LOCK_BIT)
+#define PTRACE_CFI_BRANCH_EXPECTED_LANDING_PAD_STATE	\
+	_BITUL(PTRACE_CFI_BRANCH_EXPECTED_LANDING_PAD_BIT)
+#define PTRACE_CFI_SHADOW_STACK_EN_STATE		_BITUL(PTRACE_CFI_SHADOW_STACK_EN_BIT)
+#define PTRACE_CFI_SHADOW_STACK_LOCK_STATE		_BITUL(PTRACE_CFI_SHADOW_STACK_LOCK_BIT)
+#define PTRACE_CFI_SHADOW_STACK_PTR_STATE		_BITUL(PTRACE_CFI_SHADOW_STACK_PTR_BIT)
+
+#define PTRACE_CFI_STATE_INVALID_MASK	~(PTRACE_CFI_BRANCH_LANDING_PAD_EN_STATE | \
+					  PTRACE_CFI_BRANCH_LANDING_PAD_LOCK_STATE | \
+					  PTRACE_CFI_BRANCH_EXPECTED_LANDING_PAD_STATE | \
+					  PTRACE_CFI_SHADOW_STACK_EN_STATE | \
+					  PTRACE_CFI_SHADOW_STACK_LOCK_STATE | \
+					  PTRACE_CFI_SHADOW_STACK_PTR_STATE)
+
+struct __cfi_status {
+	__u64 cfi_state;
+};
+
+struct user_cfi_state {
+	struct __cfi_status	cfi_status;
+	__u64 shstk_ptr;
+};
+
 #endif /* __ASSEMBLER__ */
 
 #endif /* _ASM_RISCV_PTRACE_H */
linux-headers/asm-riscv/unistd_32.h +1 -0
index 9f3395624639..828f3c2b9de1 100644
@@ -318,6 +318,7 @@
 #define __NR_file_getattr 468
 #define __NR_file_setattr 469
 #define __NR_listns 470
+#define __NR_rseq_slice_yield 471
 
 
 #endif /* _ASM_UNISTD_32_H */
linux-headers/asm-riscv/unistd_64.h +1 -0
index c2e725891647..8fa59835a333 100644
@@ -328,6 +328,7 @@
 #define __NR_file_getattr 468
 #define __NR_file_setattr 469
 #define __NR_listns 470
+#define __NR_rseq_slice_yield 471
 
 
 #endif /* _ASM_UNISTD_64_H */
linux-headers/asm-s390/unistd_32.h deleted +0 -446

446 lines not on this page · raw message

linux-headers/asm-s390/unistd_64.h +1 -0
index 8d9e579ef50d..01f674c1bcb7 100644
@@ -390,6 +390,7 @@
 #define __NR_file_getattr 468
 #define __NR_file_setattr 469
 #define __NR_listns 470
+#define __NR_rseq_slice_yield 471
 
 
 #endif /* _ASM_UNISTD_64_H */
linux-headers/asm-x86/kvm.h +13 -8
index b804fd25a2b8..01d46e29294f 100644
@@ -197,13 +197,13 @@ struct kvm_msrs {
 	__u32 nmsrs; /* number of msrs in entries */
 	__u32 pad;
 
-	struct kvm_msr_entry entries[];
+	__DECLARE_FLEX_ARRAY(struct kvm_msr_entry, entries);
 };
 
 /* for KVM_GET_MSR_INDEX_LIST */
 struct kvm_msr_list {
 	__u32 nmsrs; /* number of msrs in entries */
-	__u32 indices[];
+	__DECLARE_FLEX_ARRAY(__u32, indices);
 };
 
 /* Maximum size of any access bitmap in bytes */
@@ -243,7 +243,7 @@ struct kvm_cpuid_entry {
 struct kvm_cpuid {
 	__u32 nent;
 	__u32 padding;
-	struct kvm_cpuid_entry entries[];
+	__DECLARE_FLEX_ARRAY(struct kvm_cpuid_entry, entries);
 };
 
 struct kvm_cpuid_entry2 {
@@ -265,7 +265,7 @@ struct kvm_cpuid_entry2 {
 struct kvm_cpuid2 {
 	__u32 nent;
 	__u32 padding;
-	struct kvm_cpuid_entry2 entries[];
+	__DECLARE_FLEX_ARRAY(struct kvm_cpuid_entry2, entries);
 };
 
 /* for KVM_GET_PIT and KVM_SET_PIT */
@@ -396,7 +396,7 @@ struct kvm_xsave {
 	 * the contents of CPUID leaf 0xD on the host.
 	 */
 	__u32 region[1024];
-	__u32 extra[];
+	__DECLARE_FLEX_ARRAY(__u32, extra);
 };
 
 #define KVM_MAX_XCRS	16
@@ -474,6 +474,7 @@ struct kvm_sync_regs {
 #define KVM_X86_QUIRK_SLOT_ZAP_ALL		(1 << 7)
 #define KVM_X86_QUIRK_STUFF_FEATURE_MSRS	(1 << 8)
 #define KVM_X86_QUIRK_IGNORE_GUEST_PAT		(1 << 9)
+#define KVM_X86_QUIRK_VMCS12_ALLOW_FREEZE_IN_SMM (1 << 10)
 
 #define KVM_STATE_NESTED_FORMAT_VMX	0
 #define KVM_STATE_NESTED_FORMAT_SVM	1
@@ -501,6 +502,7 @@ struct kvm_sync_regs {
 #define KVM_X86_GRP_SEV			1
 #  define KVM_X86_SEV_VMSA_FEATURES	0
 #  define KVM_X86_SNP_POLICY_BITS	1
+#  define KVM_X86_SEV_SNP_REQ_CERTS	2
 
 struct kvm_vmx_nested_state_data {
 	__u8 vmcs12[KVM_STATE_NESTED_VMX_VMCS_SIZE];
@@ -562,7 +564,7 @@ struct kvm_pmu_event_filter {
 	__u32 fixed_counter_bitmap;
 	__u32 flags;
 	__u32 pad[4];
-	__u64 events[];
+	__DECLARE_FLEX_ARRAY(__u64, events);
 };
 
 #define KVM_PMU_EVENT_ALLOW 0
@@ -741,6 +743,7 @@ enum sev_cmd_id {
 	KVM_SEV_SNP_LAUNCH_START = 100,
 	KVM_SEV_SNP_LAUNCH_UPDATE,
 	KVM_SEV_SNP_LAUNCH_FINISH,
+	KVM_SEV_SNP_ENABLE_REQ_CERTS,
 
 	KVM_SEV_NR_MAX,
 };
@@ -912,8 +915,10 @@ struct kvm_sev_snp_launch_finish {
 	__u64 pad1[4];
 };
 
-#define KVM_X2APIC_API_USE_32BIT_IDS            (1ULL << 0)
-#define KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK  (1ULL << 1)
+#define KVM_X2APIC_API_USE_32BIT_IDS			_BITULL(0)
+#define KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK		_BITULL(1)
+#define KVM_X2APIC_ENABLE_SUPPRESS_EOI_BROADCAST	_BITULL(2)
+#define KVM_X2APIC_DISABLE_SUPPRESS_EOI_BROADCAST	_BITULL(3)
 
 struct kvm_hyperv_eventfd {
 	__u32 conn_id;
linux-headers/asm-x86/unistd_32.h +1 -0
index 34255aac64f0..e94546882962 100644
@@ -461,6 +461,7 @@
 #define __NR_file_getattr 468
 #define __NR_file_setattr 469
 #define __NR_listns 470
+#define __NR_rseq_slice_yield 471
 
 
 #endif /* _ASM_UNISTD_32_H */
linux-headers/asm-x86/unistd_64.h +1 -0
index 07f242a5fa43..3c49b00ed13c 100644
@@ -385,6 +385,7 @@
 #define __NR_file_getattr 468
 #define __NR_file_setattr 469
 #define __NR_listns 470
+#define __NR_rseq_slice_yield 471
 
 
 #endif /* _ASM_UNISTD_64_H */
linux-headers/asm-x86/unistd_x32.h +1 -0
index 08fc9da2fab5..bd2af9ad088d 100644
@@ -338,6 +338,7 @@
 #define __NR_file_getattr (__X32_SYSCALL_BIT + 468)
 #define __NR_file_setattr (__X32_SYSCALL_BIT + 469)
 #define __NR_listns (__X32_SYSCALL_BIT + 470)
+#define __NR_rseq_slice_yield (__X32_SYSCALL_BIT + 471)
 #define __NR_rt_sigaction (__X32_SYSCALL_BIT + 512)
 #define __NR_rt_sigreturn (__X32_SYSCALL_BIT + 513)
 #define __NR_ioctl (__X32_SYSCALL_BIT + 514)
linux-headers/linux/const.h +18 -0
index 95ede2334204..c6a9d0c9835c 100644
@@ -50,4 +50,22 @@
 
 #define __KERNEL_DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d))
 
+/*
+ * Divide positive or negative dividend by positive or negative divisor
+ * and round to closest integer. Result is undefined for negative
+ * divisors if the dividend variable type is unsigned and for negative
+ * dividends if the divisor variable type is unsigned.
+ */
+#define __KERNEL_DIV_ROUND_CLOSEST(x, divisor)		\
+({							\
+	__typeof__(x) __x = x;				\
+	__typeof__(divisor) __d = divisor;		\
+							\
+	(((__typeof__(x))-1) > 0 ||			\
+	 ((__typeof__(divisor))-1) > 0 ||		\
+	 (((__x) > 0) == ((__d) > 0))) ?		\
+		(((__x) + ((__d) / 2)) / (__d)) :	\
+		(((__x) - ((__d) / 2)) / (__d));	\
+})
+
 #endif /* _LINUX_CONST_H */
linux-headers/linux/iommufd.h +48 -0

99 lines not on this page · raw message

linux-headers/linux/kvm.h +58 -6

169 lines not on this page · raw message

linux-headers/linux/mshv.h +3 -1
index acceeddc1c9f..6c7d3a93162c 100644
@@ -27,6 +27,8 @@ enum {
 	MSHV_PT_BIT_X2APIC,
 	MSHV_PT_BIT_GPA_SUPER_PAGES,
 	MSHV_PT_BIT_CPU_AND_XSAVE_FEATURES,
+	MSHV_PT_BIT_NESTED_VIRTUALIZATION,
+	MSHV_PT_BIT_SMT_ENABLED_GUEST,
 	MSHV_PT_BIT_COUNT,
 };
 
@@ -355,7 +357,7 @@ struct mshv_vtl_sint_post_msg {
 
 struct mshv_vtl_ram_disposition {
 	__u64 start_pfn;
-	__u64 last_pfn;
+	__u64 last_pfn; /* last_pfn is excluded from the range [start_pfn, last_pfn) */
 };
 
 struct mshv_vtl_set_poll_file {
linux-headers/linux/psp-sev.h +1 -1
index 9479928a4ad6..7df50022592a 100644
@@ -277,7 +277,7 @@ struct sev_user_data_snp_wrapped_vlek_hashstick {
  * struct sev_issue_cmd - SEV ioctl parameters
  *
  * @cmd: SEV commands to execute
- * @opaque: pointer to the command structure
+ * @data: pointer to the command structure
  * @error: SEV FW return code on failure
  */
 struct sev_issue_cmd {
linux-headers/linux/stddef.h +4 -0
index 48ee4438e0ef..457498259494 100644
@@ -69,6 +69,10 @@
 #define __counted_by_be(m)
 #endif
 
+#ifndef __counted_by_ptr
+#define __counted_by_ptr(m)
+#endif
+
 #define __kernel_nonstring
 
 #endif /* _LINUX_STDDEF_H */
linux-headers/linux/vduse.h +80 -5

180 lines not on this page · raw message

linux-headers/linux/vfio.h +29 -1
index 720edfee7af6..f3282b8e8650 100644
@@ -141,7 +141,7 @@ struct vfio_info_cap_header {
  *
  * Retrieve information about the group.  Fills in provided
  * struct vfio_group_info.  Caller sets argsz.
- * Return: 0 on succes, -errno on failure.
+ * Return: 0 on success, -errno on failure.
  * Availability: Always
  */
 struct vfio_group_status {
@@ -964,6 +964,10 @@ struct vfio_device_bind_iommufd {
  * hwpt corresponding to the given pt_id.
  *
  * Return: 0 on success, -errno on failure.
+ *
+ * When a device is resetting, -EBUSY will be returned to reject any concurrent
+ * attachment to the resetting device itself or any sibling device in the IOMMU
+ * group having the resetting device.
  */
 struct vfio_device_attach_iommufd_pt {
 	__u32	argsz;
@@ -1262,6 +1266,19 @@ enum vfio_device_mig_state {
  * The initial_bytes field indicates the amount of initial precopy
  * data available from the device. This field should have a non-zero initial
  * value and decrease as migration data is read from the device.
+ * The presence of the VFIO_PRECOPY_INFO_REINIT output flag indicates
+ * that new initial data is present on the stream.
+ * The new initial data may result, for example, from device reconfiguration
+ * during migration that requires additional initialization data.
+ * In that case initial_bytes may report a non-zero value irrespective of
+ * any previously reported values, which progresses towards zero as precopy
+ * data is read from the data stream. dirty_bytes is also reset
+ * to zero and represents the state change of the device relative to the new
+ * initial_bytes.
+ * VFIO_PRECOPY_INFO_REINIT can be reported only after userspace opts in to
+ * VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2. Without this opt-in, the flags field
+ * of struct vfio_precopy_info is reserved for bug-compatibility reasons.
+ *
  * It is recommended to leave PRE_COPY for STOP_COPY only after this field
  * reaches zero. Leaving PRE_COPY earlier might make things slower.
  *
@@ -1297,6 +1314,7 @@ enum vfio_device_mig_state {
 struct vfio_precopy_info {
 	__u32 argsz;
 	__u32 flags;
+#define VFIO_PRECOPY_INFO_REINIT (1 << 0) /* output - new initial data is present */
 	__aligned_u64 initial_bytes;
 	__aligned_u64 dirty_bytes;
 };
@@ -1506,6 +1524,16 @@ struct vfio_device_feature_dma_buf {
 	struct vfio_region_dma_range dma_ranges[] __counted_by(nr_ranges);
 };
 
+/*
+ * Enables the migration precopy_info_v2 behaviour.
+ *
+ * VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2.
+ *
+ * On SET, enables the v2 pre_copy_info behaviour, where the
+ * vfio_precopy_info.flags is a valid output field.
+ */
+#define VFIO_DEVICE_FEATURE_MIG_PRECOPY_INFOv2  12
+
 /* -------- API for Type1 VFIO IOMMU -------- */
 
 /**
Signature · 1 line
2.53.0
#2 lore raw

RFC PATCH 2/10 accel/kvm: Extend KVMState to carry fds for planes

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Extend the vmfd member of KVMState into an array and rename it to plane_fds. The vmfd will be stored at index 0.

---
 accel/kvm/kvm-all.c      | 97 ++++++++++++++++++++++++++++++++--------
 accel/kvm/trace-events   |  1 +
 include/system/kvm.h     |  3 ++
 include/system/kvm_int.h | 22 ++++++++-
 target/arm/kvm.c         |  2 +-
 5 files changed, 104 insertions(+), 21 deletions(-)
accel/kvm/kvm-all.c +78 -19
index 92af42503b1c..1a2f8e0f417c 100644
@@ -770,8 +770,12 @@ void kvm_close(void)
     }
 
     if (kvm_state && kvm_state->fd != -1) {
-        close(kvm_state->vmfd);
-        kvm_state->vmfd = -1;
+        unsigned plane_id = KVM_MAX_PLANES;
+        do {
+            plane_id--;
+            close(kvm_get_plane_fd(kvm_state, plane_id));
+            kvm_set_plane_fd(kvm_state, plane_id, -1);
+        } while (plane_id != 0);
         close(kvm_state->fd);
         kvm_state->fd = -1;
     }
@@ -2774,12 +2778,41 @@ static int kvm_setup_dirty_ring(KVMState *s)
 
     return 0;
 }
+static int kvm_create_plane(KVMState *s, unsigned id)
+{
+    int fd = kvm_vm_ioctl(s, KVM_CREATE_PLANE, id);
+    if (fd >= 0) {
+        kvm_set_plane_fd(s, id, fd);
+    }
+
+    return fd;
+}
+
+int kvm_get_or_create_plane_fd(KVMState *s, unsigned id)
+{
+    int fd = kvm_get_plane_fd(s, id);
+    if (fd >= 0) {
+        return fd;
+    }
+
+    return kvm_create_plane(s, id);
+}
+
+static void kvm_init_plane_fds(KVMState *s)
+{
+    int i;
+
+    for (i = 0; i < KVM_MAX_PLANES; i++) {
+        kvm_set_plane_fd(s, i, -1);
+    }
+}
 
 static int kvm_reset_vmfd(MachineState *ms)
 {
     KVMState *s;
     KVMMemoryListener *kml;
     int ret = 0, type;
+    unsigned plane_id;
     Error *err = NULL;
 
     /*
@@ -2805,9 +2838,14 @@ static int kvm_reset_vmfd(MachineState *ms)
     }
     assert(!err);
 
-    if (s->vmfd >= 0) {
-        close(s->vmfd);
-    }
+    plane_id = KVM_MAX_PLANES;
+    do {
+        plane_id--;
+        if (kvm_get_plane_fd(s, plane_id) >= 0) {
+            close(kvm_get_plane_fd(s, plane_id));
+            kvm_set_plane_fd(s, plane_id, -1);
+        }
+    } while (plane_id != 0);
 
     type = find_kvm_machine_type(ms);
     if (type < 0) {
@@ -2819,7 +2857,7 @@ static int kvm_reset_vmfd(MachineState *ms)
         return ret;
     }
 
-    s->vmfd = ret;
+    kvm_set_vm_fd(s, ret);
 
     /* guest state is now unprotected again */
     kvm_state->guest_state_protected = false;
@@ -2846,7 +2884,7 @@ static int kvm_reset_vmfd(MachineState *ms)
     /*
      * notify everyone that vmfd has changed.
      */
-    vmfd_notifier.vmfd = s->vmfd;
+    vmfd_notifier.vmfd = kvm_vm_fd(s);
     vmfd_notifier.pre = false;
 
     ret = kvm_vmfd_change_notify(&err);
@@ -2913,6 +2951,8 @@ static int kvm_init(AccelState *as, MachineState *ms)
 
     qemu_mutex_init(&kml_slots_lock);
 
+    kvm_init_plane_fds(s);
+
     /*
      * On systems where the kernel can support different base page
      * sizes, host page size may be different from TARGET_PAGE_SIZE,
@@ -2969,7 +3009,7 @@ static int kvm_init(AccelState *as, MachineState *ms)
         goto err;
     }
 
-    s->vmfd = ret;
+    kvm_set_plane_fd(s, 0, ret);
 
     s->nr_as = kvm_vm_check_extension(s, KVM_CAP_MULTI_ADDRESS_SPACE);
     if (s->nr_as <= 1) {
@@ -3109,8 +3149,8 @@ static int kvm_init(AccelState *as, MachineState *ms)
 
 err:
     assert(ret < 0);
-    if (s->vmfd >= 0) {
-        close(s->vmfd);
+    if (kvm_vm_fd(s) >= 0) {
+        close(kvm_vm_fd(s));
     }
     if (s->fd != -1) {
         close(s->fd);
@@ -3646,9 +3686,21 @@ int kvm_ioctl(KVMState *s, unsigned long type, ...)
     return ret;
 }
 
-int kvm_vm_ioctl(KVMState *s, unsigned long type, ...)
+static int __vm_plane_ioctl(KVMState *s, unsigned plane_id, unsigned long type, void *arg)
 {
     int ret;
+
+    accel_ioctl_begin();
+    ret = ioctl(kvm_get_plane_fd(s, plane_id), type, arg);
+    accel_ioctl_end();
+    if (ret == -1) {
+        ret = -errno;
+    }
+    return ret;
+}
+
+int kvm_vm_ioctl(KVMState *s, unsigned long type, ...)
+{
     void *arg;
     va_list ap;
 
@@ -3657,13 +3709,20 @@ int kvm_vm_ioctl(KVMState *s, unsigned long type, ...)
     va_end(ap);
 
     trace_kvm_vm_ioctl(type, arg);
-    accel_ioctl_begin();
-    ret = ioctl(s->vmfd, type, arg);
-    if (ret == -1) {
-        ret = -errno;
-    }
-    accel_ioctl_end();
-    return ret;
+    return __vm_plane_ioctl(s, 0, type, arg);
+}
+
+int kvm_vm_plane_ioctl(KVMState *s, unsigned plane_id, unsigned long type, ...)
+{
+    void *arg;
+    va_list ap;
+
+    va_start(ap, type);
+    arg = va_arg(ap, void *);
+    va_end(ap);
+
+    trace_kvm_vm_plane_ioctl(type, plane_id, arg);
+    return __vm_plane_ioctl(s, plane_id, type, arg);
 }
 
 int kvm_vcpu_ioctl(CPUState *cpu, unsigned long type, ...)
@@ -4266,8 +4325,8 @@ static void kvm_accel_instance_init(Object *obj)
 {
     KVMState *s = KVM_STATE(obj);
 
+    kvm_init_plane_fds(s);
     s->fd = -1;
-    s->vmfd = -1;
     s->kvm_shadow_mem = -1;
     s->kernel_irqchip_allowed = true;
     s->kernel_irqchip_split = ON_OFF_AUTO_AUTO;
accel/kvm/trace-events +1 -0
index 4a8921c632bf..2f3bd9ba7052 100644
@@ -3,6 +3,7 @@
 # kvm-all.c
 kvm_ioctl(unsigned long type, void *arg) "type 0x%lx, arg %p"
 kvm_vm_ioctl(unsigned long type, void *arg) "type 0x%lx, arg %p"
+kvm_vm_plane_ioctl(unsigned long type, unsigned id, void *arg) "type 0x%lx, plane_id %d arg %p"
 kvm_vcpu_ioctl(int cpu_index, unsigned long type, void *arg) "cpu_index %d, type 0x%lx, arg %p"
 kvm_run_exit(int cpu_index, uint32_t reason) "cpu_index %d, reason %d"
 kvm_device_ioctl(int fd, unsigned long type, void *arg) "dev fd %d, type 0x%lx, arg %p"
include/system/kvm.h +3 -0
index 5fa33eddda38..885ed35b061a 100644
@@ -216,6 +216,9 @@ int kvm_on_sigbus(int code, void *addr);
 int kvm_check_extension(KVMState *s, unsigned int extension);
 
 int kvm_vm_ioctl(KVMState *s, unsigned long type, ...);
+int kvm_vm_plane_ioctl(KVMState *s, unsigned plane_id, unsigned long type, ...);
+
+int kvm_get_or_create_plane_fd(KVMState *s, unsigned id);
 
 void kvm_flush_coalesced_mmio_buffer(void);
 
include/system/kvm_int.h +21 -1
index 0876aac938d3..bfac331949f9 100644
@@ -107,7 +107,7 @@ struct KVMState
     /* Max number of KVM slots supported */
     int nr_slots_max;
     int fd;
-    int vmfd;
+    int plane_fds[KVM_MAX_PLANES];
     int coalesced_mmio;
     int coalesced_pio;
     struct kvm_coalesced_mmio_ring *coalesced_mmio_ring;
@@ -170,6 +170,26 @@ struct KVMState
     OnOffAuto honor_guest_pat;
 };
 
+static inline void kvm_set_plane_fd(KVMState *s, unsigned plane, int fd)
+{
+    s->plane_fds[plane] = fd;
+}
+
+static inline int kvm_get_plane_fd(KVMState *s, unsigned plane)
+{
+    return s->plane_fds[plane];
+}
+
+static inline void kvm_set_vm_fd(KVMState *s, int vmfd)
+{
+    kvm_set_plane_fd(s, 0, vmfd);
+}
+
+static inline int kvm_vm_fd(KVMState *s)
+{
+    return kvm_get_plane_fd(s, 0);
+}
+
 void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml,
                                   AddressSpace *as, int as_id, const char *name);
 
target/arm/kvm.c +1 -1
index d4a68874b880..0bc869aa5d92 100644
@@ -134,7 +134,7 @@ bool kvm_arm_create_scratch_host_vcpu(int *fdarray,
         KVMState kvm_state;
 
         kvm_state.fd = kvmfd;
-        kvm_state.vmfd = vmfd;
+        kvm_set_vm_fd(&kvm_state, vmfd);
         kvm_vm_enable_cap(&kvm_state, KVM_CAP_ARM_MTE, 0);
     }
 
Signature · 1 line
2.53.0
#3 lore raw

RFC PATCH 3/10 accel/kvm: Extend CPUState to handle Planes

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Extend the KVM specific part of the CPUState data structure to handle the FDs for multiple planes.

---
 accel/kvm/kvm-all.c      | 121 +++++++++++++++++++++++++++++++--------
 accel/kvm/trace-events   |   1 +
 include/hw/core/cpu.h    |  17 +++++-
 include/system/kvm.h     |   4 ++
 include/system/kvm_int.h |   8 +++
 5 files changed, 126 insertions(+), 25 deletions(-)
accel/kvm/kvm-all.c +98 -23
index 1a2f8e0f417c..7429e2be8ba9 100644
@@ -133,6 +133,7 @@ static NotifierWithReturnList register_vcpufd_changed_notifiers =
 static int map_kvm_run(KVMState *s, CPUState *cpu, Error **errp);
 static int map_kvm_dirty_gfns(KVMState *s, CPUState *cpu, Error **errp);
 static int vcpu_unmap_regions(KVMState *s, CPUState *cpu);
+static void kvm_alloc_vcpu_plane(CPUState *cpu, unsigned plane_id, int kvm_fd);
 
 struct KVMResampleFd {
     int gsi;
@@ -429,10 +430,16 @@ err:
 
 static void kvm_create_vcpu_internal(CPUState *cpu, KVMState *s, int kvm_fd)
 {
-    cpu->kvm_fd = kvm_fd;
+    if (cpu->kvm_plane_state[0] == NULL) {
+        kvm_alloc_vcpu_plane(cpu, 0, kvm_fd);
+    } else {
+        cpu_kvm_plane(cpu, 0)->kvm_fd = kvm_fd;
+    }
+
+    cpu->kvm_plane = 0;
     cpu->kvm_state = s;
     if (!s->guest_state_protected) {
-        cpu->vcpu_dirty = true;
+        cpu_kvm_plane(cpu, 0)->vcpu_dirty = true;
     }
     cpu->dirty_pages = 0;
     cpu->throttle_us_per_full = 0;
@@ -450,8 +457,8 @@ static int kvm_rebind_vcpus(Error **errp)
     CPU_FOREACH(cpu) {
         vcpu_id = kvm_arch_vcpu_id(cpu);
 
-        if (cpu->kvm_fd) {
-            close(cpu->kvm_fd);
+        if (cpu_kvm_plane(cpu, 0)->kvm_fd) {
+            close(cpu_kvm_plane(cpu, 0)->kvm_fd);
         }
 
         ret = kvm_arch_destroy_vcpu(cpu);
@@ -501,8 +508,9 @@ static int kvm_rebind_vcpus(Error **errp)
                              vcpu_id);
         }
 
-        close(cpu->kvm_vcpu_stats_fd);
-        cpu->kvm_vcpu_stats_fd = kvm_vcpu_ioctl(cpu, KVM_GET_STATS_FD, NULL);
+        close(cpu_kvm_plane(cpu, 0)->kvm_vcpu_stats_fd);
+        cpu_kvm_plane(cpu, 0)->kvm_vcpu_stats_fd =
+            kvm_vcpu_ioctl(cpu, KVM_GET_STATS_FD, NULL);
         kvm_init_cpu_signals(cpu);
     }
     trace_kvm_rebind_vcpus();
@@ -519,7 +527,7 @@ static void kvm_park_vcpu(CPUState *cpu)
 
     vcpu = g_malloc0(sizeof(*vcpu));
     vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
-    vcpu->kvm_fd = cpu->kvm_fd;
+    vcpu->kvm_fd = cpu_kvm_plane(cpu, 0)->kvm_fd;
     QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
 }
 
@@ -551,6 +559,34 @@ static void kvm_reset_parked_vcpus(KVMState *s)
     }
 }
 
+static void kvm_alloc_vcpu_plane(CPUState *cpu, unsigned plane_id, int kvm_fd)
+{
+    struct KVMPlane *p = NULL;
+
+    if (cpu->kvm_plane_state[plane_id] != NULL) {
+        return;
+    }
+
+    p = g_malloc0(sizeof(struct KVMPlane));
+    p->kvm_fd = kvm_fd;
+
+    cpu->kvm_plane_state[plane_id] = p;
+}
+
+void kvm_create_vcpu_plane(CPUState *cpu, unsigned plane_id, int kvm_fd)
+{
+    int vcpu_fd = cpu_kvm_plane(cpu, 0)->kvm_fd;
+    int plane_fd = kvm_vm_plane_ioctl(cpu->kvm_state, plane_id, KVM_CREATE_VCPU, vcpu_fd);
+
+    if (plane_fd < 0) {
+        fprintf(stderr, "Failed to create plane vcpu\n");
+        abort();
+    }
+
+    kvm_alloc_vcpu_plane(cpu, plane_id, plane_fd);
+}
+
+
 /**
  * kvm_create_vcpu - Gets a parked KVM vCPU or creates a KVM vCPU
  * @cpu: QOM CPUState object for which KVM vCPU has to be fetched/created.
@@ -676,7 +712,7 @@ static int map_kvm_run(KVMState *s, CPUState *cpu, Error **errp)
     }
 
     cpu->kvm_run = mmap(NULL, mmap_size, PROT_READ | PROT_WRITE, MAP_SHARED,
-                        cpu->kvm_fd, 0);
+                        cpu_kvm_plane(cpu, 0)->kvm_fd, 0);
     if (cpu->kvm_run == MAP_FAILED) {
         ret = -errno;
         error_setg_errno(errp, ret,
@@ -700,7 +736,7 @@ static int map_kvm_dirty_gfns(KVMState *s, CPUState *cpu, Error **errp)
     /* Use MAP_SHARED to share pages with the kernel */
     cpu->kvm_dirty_gfns = mmap(NULL, s->kvm_dirty_ring_bytes,
                                PROT_READ | PROT_WRITE, MAP_SHARED,
-                               cpu->kvm_fd,
+                               cpu_kvm_plane(cpu, 0)->kvm_fd,
                                PAGE_SIZE * KVM_DIRTY_LOG_PAGE_OFFSET);
     if (cpu->kvm_dirty_gfns == MAP_FAILED) {
         ret = -errno;
@@ -747,7 +783,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
                          "kvm_init_vcpu: kvm_arch_init_vcpu failed (%lu)",
                          kvm_arch_vcpu_id(cpu));
     }
-    cpu->kvm_vcpu_stats_fd = kvm_vcpu_ioctl(cpu, KVM_GET_STATS_FD, NULL);
+    cpu_kvm_plane(cpu, 0)->kvm_vcpu_stats_fd = kvm_vcpu_ioctl(cpu, KVM_GET_STATS_FD, NULL);
 
 err:
     return ret;
@@ -762,11 +798,17 @@ void kvm_close(void)
     }
 
     CPU_FOREACH(cpu) {
+        unsigned plane_id = KVM_MAX_PLANES;
         cpu_remove_sync(cpu);
-        close(cpu->kvm_fd);
-        cpu->kvm_fd = -1;
-        close(cpu->kvm_vcpu_stats_fd);
-        cpu->kvm_vcpu_stats_fd = -1;
+        do {
+            struct KVMPlane *plane;
+            plane_id--;
+            plane = cpu_kvm_plane(cpu, plane_id);
+            close(plane->kvm_fd);
+            plane->kvm_fd = -1;
+            close(plane->kvm_vcpu_stats_fd);
+            plane->kvm_vcpu_stats_fd = -1;
+        } while (plane_id != 0);
     }
 
     if (kvm_state && kvm_state->fd != -1) {
@@ -3238,7 +3280,9 @@ void kvm_flush_coalesced_mmio_buffer(void)
 
 static void do_kvm_cpu_synchronize_state(CPUState *cpu, run_on_cpu_data arg)
 {
-    if (!cpu->vcpu_dirty && !kvm_state->guest_state_protected) {
+    KVMPlane *plane = cpu_active_kvm_plane(cpu);
+
+    if (!plane->vcpu_dirty && !kvm_state->guest_state_protected) {
         Error *err = NULL;
         int ret = kvm_arch_get_registers(cpu, &err);
         if (ret) {
@@ -3252,13 +3296,15 @@ static void do_kvm_cpu_synchronize_state(CPUState *cpu, run_on_cpu_data arg)
             vm_stop(RUN_STATE_INTERNAL_ERROR);
         }
 
-        cpu->vcpu_dirty = true;
+        plane->vcpu_dirty = true;
     }
 }
 
 void kvm_cpu_synchronize_state(CPUState *cpu)
 {
-    if (!cpu->vcpu_dirty && !kvm_state->guest_state_protected) {
+    KVMPlane *plane = cpu_active_kvm_plane(cpu);
+
+    if (!plane->vcpu_dirty && !kvm_state->guest_state_protected) {
         run_on_cpu(cpu, do_kvm_cpu_synchronize_state, RUN_ON_CPU_NULL);
     }
 }
@@ -3278,7 +3324,7 @@ static bool kvm_cpu_synchronize_put(CPUState *cpu, KvmPutState state,
         return false;
     }
 
-    cpu->vcpu_dirty = false;
+    cpu_active_kvm_plane(cpu)->vcpu_dirty = false;
 
     return true;
 }
@@ -3320,7 +3366,7 @@ void kvm_cpu_synchronize_post_init(CPUState *cpu)
 
 static void do_kvm_cpu_synchronize_pre_loadvm(CPUState *cpu, run_on_cpu_data arg)
 {
-    cpu->vcpu_dirty = true;
+    cpu_active_kvm_plane(cpu)->vcpu_dirty = true;
 }
 
 void kvm_cpu_synchronize_pre_loadvm(CPUState *cpu)
@@ -3478,6 +3524,7 @@ out_unref:
 
 int kvm_cpu_exec(CPUState *cpu)
 {
+    KVMPlane *plane = cpu_active_kvm_plane(cpu);
     struct kvm_run *run = cpu->kvm_run;
     int ret, run_ret;
 
@@ -3493,7 +3540,7 @@ int kvm_cpu_exec(CPUState *cpu)
     do {
         MemTxAttrs attrs;
 
-        if (cpu->vcpu_dirty) {
+        if (plane->vcpu_dirty) {
             if (!kvm_cpu_synchronize_put(cpu, KVM_PUT_RUNTIME_STATE,
                                          "at runtime")) {
                 ret = -1;
@@ -3725,8 +3772,36 @@ int kvm_vm_plane_ioctl(KVMState *s, unsigned plane_id, unsigned long type, ...)
     return __vm_plane_ioctl(s, plane_id, type, arg);
 }
 
+static inline int __vcpu_plane_ioctl(KVMPlane *plane, unsigned long type, void *arg)
+{
+    return ioctl(plane->kvm_fd, type, arg);
+}
+
+int kvm_vcpu_plane_ioctl(CPUState *cpu, unsigned plane_id, unsigned long type, ...)
+{
+    KVMPlane *plane = cpu_kvm_plane(cpu, plane_id);
+    int ret;
+    void *arg;
+    va_list ap;
+
+    va_start(ap, type);
+    arg = va_arg(ap, void *);
+    va_end(ap);
+
+    trace_kvm_vcpu_plane_ioctl(cpu->cpu_index, plane_id, type, arg);
+    accel_cpu_ioctl_begin(cpu);
+    ret = __vcpu_plane_ioctl(plane, type, arg);
+    accel_cpu_ioctl_end(cpu);
+    if (ret == -1) {
+        ret = -errno;
+    }
+    return ret;
+}
+
 int kvm_vcpu_ioctl(CPUState *cpu, unsigned long type, ...)
 {
+    /* Most VCPU IOCTLs (including KVM_RUN) must happen on the Plane-0 FD */
+    KVMPlane *plane = cpu_kvm_plane(cpu, 0);
     int ret;
     void *arg;
     va_list ap;
@@ -3737,7 +3812,7 @@ int kvm_vcpu_ioctl(CPUState *cpu, unsigned long type, ...)
 
     trace_kvm_vcpu_ioctl(cpu->cpu_index, type, arg);
     accel_cpu_ioctl_begin(cpu);
-    ret = ioctl(cpu->kvm_fd, type, arg);
+    ret = __vcpu_plane_ioctl(plane, type, arg);
     accel_cpu_ioctl_end(cpu);
     if (ret == -1) {
         ret = -errno;
@@ -4731,7 +4806,7 @@ static void query_stats_schema(StatsSchemaList **result, StatsTarget target,
 
 static void query_stats_vcpu(CPUState *cpu, StatsArgs *kvm_stats_args)
 {
-    int stats_fd = cpu->kvm_vcpu_stats_fd;
+    int stats_fd = cpu_active_kvm_plane(cpu)->kvm_vcpu_stats_fd;
     Error *local_err = NULL;
 
     if (stats_fd == -1) {
@@ -4746,7 +4821,7 @@ static void query_stats_vcpu(CPUState *cpu, StatsArgs *kvm_stats_args)
 
 static void query_stats_schema_vcpu(CPUState *cpu, StatsArgs *kvm_stats_args)
 {
-    int stats_fd = cpu->kvm_vcpu_stats_fd;
+    int stats_fd = cpu_active_kvm_plane(cpu)->kvm_vcpu_stats_fd;
     Error *local_err = NULL;
 
     if (stats_fd == -1) {
accel/kvm/trace-events +1 -0
index 2f3bd9ba7052..1ca7be8a4b3b 100644
@@ -5,6 +5,7 @@ kvm_ioctl(unsigned long type, void *arg) "type 0x%lx, arg %p"
 kvm_vm_ioctl(unsigned long type, void *arg) "type 0x%lx, arg %p"
 kvm_vm_plane_ioctl(unsigned long type, unsigned id, void *arg) "type 0x%lx, plane_id %d arg %p"
 kvm_vcpu_ioctl(int cpu_index, unsigned long type, void *arg) "cpu_index %d, type 0x%lx, arg %p"
+kvm_vcpu_plane_ioctl(int cpu_index, unsigned plane_id, unsigned long type, void *arg) "cpu_index %d, plane_id %u type 0x%lx, arg %p"
 kvm_run_exit(int cpu_index, uint32_t reason) "cpu_index %d, reason %d"
 kvm_device_ioctl(int fd, unsigned long type, void *arg) "dev fd %d, type 0x%lx, arg %p"
 kvm_failed_reg_get(uint64_t id, const char *msg) "Warning: Unable to retrieve ONEREG %" PRIu64 " from KVM: %s"
include/hw/core/cpu.h +15 -2
index 04e1f970caf2..4025db67e13b 100644
@@ -36,6 +36,7 @@
 #include "qemu/lockcnt.h"
 #include "qemu/thread.h"
 #include "qom/object.h"
+#include "linux/kvm.h"
 
 typedef int (*WriteCoreDumpFunction)(const void *buf, size_t size,
                                      void *opaque);
@@ -545,13 +546,15 @@ struct CPUState {
     uintptr_t mem_io_pc;
 
     /* Only used in KVM */
-    int kvm_fd;
     struct KVMState *kvm_state;
     struct kvm_run *kvm_run;
     struct kvm_dirty_gfn *kvm_dirty_gfns;
     uint32_t kvm_fetch_index;
     uint64_t dirty_pages;
-    int kvm_vcpu_stats_fd;
+
+    /* KVM plane state */
+    unsigned kvm_plane;                                    /* Current active plane */
+    struct KVMPlane *kvm_plane_state[KVM_MAX_PLANES]; /* Per-Plane state */
 
     /* Use by accel-block: CPU is executing an ioctl() */
     QemuLockCnt in_ioctl_lock;
@@ -596,6 +599,16 @@ struct CPUState {
     CPUNegativeOffsetState neg;
 };
 
+static inline struct KVMPlane *cpu_kvm_plane(CPUState *s, unsigned plane_id)
+{
+	return s->kvm_plane_state[plane_id];
+}
+
+static inline struct KVMPlane *cpu_active_kvm_plane(CPUState *s)
+{
+	return s->kvm_plane_state[s->kvm_plane];
+}
+
 /* Validate placement of CPUNegativeOffsetState. */
 QEMU_BUILD_BUG_ON(offsetof(CPUState, neg) !=
                   sizeof(CPUState) - sizeof(CPUNegativeOffsetState));
include/system/kvm.h +4 -0
index 885ed35b061a..16597333cfa5 100644
@@ -172,10 +172,12 @@ typedef struct KVMCapabilityInfo {
 #define KVM_CAP_INFO(CAP) { "KVM_CAP_" stringify(CAP), KVM_CAP_##CAP }
 #define KVM_CAP_LAST_INFO { NULL, 0 }
 
+struct KVMPlane;
 struct KVMState;
 
 #define TYPE_KVM_ACCEL ACCEL_CLASS_NAME("kvm")
 typedef struct KVMState KVMState;
+typedef struct KVMPlane KVMPlane;
 DECLARE_INSTANCE_CHECKER(KVMState, KVM_STATE,
                          TYPE_KVM_ACCEL)
 
@@ -219,6 +221,7 @@ int kvm_vm_ioctl(KVMState *s, unsigned long type, ...);
 int kvm_vm_plane_ioctl(KVMState *s, unsigned plane_id, unsigned long type, ...);
 
 int kvm_get_or_create_plane_fd(KVMState *s, unsigned id);
+void kvm_create_vcpu_plane(CPUState *cpu, unsigned plane, int kvm_fd);
 
 void kvm_flush_coalesced_mmio_buffer(void);
 
@@ -251,6 +254,7 @@ static inline int kvm_update_guest_debug(CPUState *cpu, unsigned long reinject_t
 
 int kvm_ioctl(KVMState *s, unsigned long type, ...);
 
+int kvm_vcpu_plane_ioctl(CPUState *cpu, unsigned plane_id, unsigned long type, ...);
 int kvm_vcpu_ioctl(CPUState *cpu, unsigned long type, ...);
 
 /**
include/system/kvm_int.h +8 -0
index bfac331949f9..70b381f1ba05 100644
@@ -101,6 +101,14 @@ struct KVMDirtyRingReaper {
     volatile uint64_t reaper_iteration; /* iteration number of reaper thr */
     volatile enum KVMDirtyRingReaperState reaper_state; /* reap thr state */
 };
+
+/* VCPU per-plane state */
+struct KVMPlane {
+    int kvm_fd;
+    int kvm_vcpu_stats_fd;
+    bool vcpu_dirty;
+};
+
 struct KVMState
 {
     AccelState parent_obj;
Signature · 1 line
2.53.0
#4 lore raw

RFC PATCH 4/10 accel: Add nr_planes() op

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Add a new accelerator operation to request the highest supported plane number of a given machine instance.

---
 accel/accel-system.c      | 13 +++++++++++++
 include/accel/accel-ops.h |  3 +++
 include/qemu/accel.h      |  7 +++++++
 3 files changed, 23 insertions(+)
accel/accel-system.c +13 -0
index 150af05bf5bf..968473b8692a 100644
@@ -75,6 +75,19 @@ void accel_pre_resume(MachineState *ms, bool step_pending)
     }
 }
 
+uint8_t accel_nr_planes(MachineState *ms)
+{
+    AccelState *accel = ms->accelerator;
+    AccelClass *acc = ACCEL_GET_CLASS(accel);
+    uint8_t nr_planes = 1;
+
+    if (acc->nr_planes != NULL) {
+        nr_planes = acc->nr_planes(accel, ms);
+    }
+
+    return nr_planes;
+}
+
 /* initialize the arch-independent accel operation interfaces */
 void accel_init_ops_interfaces(AccelClass *ac)
 {
include/accel/accel-ops.h +3 -0
index f46492e3fe15..1d5decb9359b 100644
@@ -36,6 +36,9 @@ struct AccelClass {
     bool (*has_memory)(AccelState *accel, AddressSpace *as,
                        hwaddr start_addr, hwaddr size);
 
+    /* planes related hooks */
+    uint8_t (*nr_planes)(AccelState *as, MachineState *ms);
+
     /* gdbstub related hooks */
     int (*gdbstub_supported_sstep_flags)(AccelState *as);
 
include/qemu/accel.h +7 -0
index d3638c7bfda7..2ecf33e1fa21 100644
@@ -81,4 +81,11 @@ void accel_cpu_common_unrealize(CPUState *cpu);
  */
 int accel_supported_gdbstub_sstep_flags(void);
 
+/**
+ * accel_nr_planes:
+ *
+ * Returns the number of the highest support plane of a given MachineState.
+ */
+uint8_t accel_nr_planes(MachineState *ms);
+
 #endif /* QEMU_ACCEL_H */
Signature · 1 line
2.53.0
#5 lore raw

RFC PATCH 5/10 accel/kvm: Support nr_planes call-back

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

---
 accel/kvm/kvm-all.c | 19 +++++++++++++++++++
 dtc                 |  1 +
 ui/keycodemapdb     |  1 +
 3 files changed, 21 insertions(+)
 create mode 160000 dtc
 create mode 160000 ui/keycodemapdb
accel/kvm/kvm-all.c +19 -0
index 7429e2be8ba9..dbfef63a84b0 100644
@@ -4247,6 +4247,24 @@ static bool kvm_accel_has_memory(AccelState *accel, AddressSpace *as,
     return false;
 }
 
+static uint8_t kvm_nr_planes(AccelState *accel, MachineState *ms)
+{
+    uint8_t nr_planes = 1;
+
+    // Planes are only supported with in-kernel APIC
+    if (kvm_irqchip_in_kernel()) {
+	    int ret;
+        KVMState *kvm = KVM_STATE(accel);
+
+        ret = kvm_vm_ioctl(kvm, KVM_CHECK_EXTENSION, KVM_CAP_PLANES);
+        if (ret > 0) {
+            nr_planes = ret;
+        }
+    }
+
+    return nr_planes;
+}
+
 static void kvm_get_kvm_shadow_mem(Object *obj, Visitor *v,
                                    const char *name, void *opaque,
                                    Error **errp)
@@ -4437,6 +4455,7 @@ static void kvm_accel_class_init(ObjectClass *oc, const void *data)
     ac->init_machine = kvm_init;
     ac->rebuild_guest = kvm_reset_vmfd;
     ac->has_memory = kvm_accel_has_memory;
+    ac->nr_planes = kvm_nr_planes;
     ac->allowed = &kvm_allowed;
     ac->gdbstub_supported_sstep_flags = kvm_gdbstub_sstep_flags;
 
dtc new file +1 -0
new file mode 160000
index 000000000000..b6910bec1161
@@ -0,0 +1 @@
+Subproject commit b6910bec11614980a21e46fbccc35934b671bd81
ui/keycodemapdb new file +1 -0
new file mode 160000
index 000000000000..d21009b1c9f9
@@ -0,0 +1 @@
+Subproject commit d21009b1c9f94b740ea66be8e48a1d8ad8124023
Signature · 1 line
2.53.0
#6 lore raw

RFC PATCH 6/10 accel/kvm: Handle KVM_PLANE_EVENT_CREATE_CPU event

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Implement the plane event handling infrastructure and handle the KVM_PLANE_EVENT_CREATE_CPU event.

---
 accel/kvm/kvm-all.c   | 13 +++++++---
 include/system/kvm.h  |  2 +-
 target/i386/kvm/kvm.c | 57 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 67 insertions(+), 5 deletions(-)
accel/kvm/kvm-all.c +9 -4
index dbfef63a84b0..c5fe6d189e62 100644
@@ -573,11 +573,17 @@ static void kvm_alloc_vcpu_plane(CPUState *cpu, unsigned plane_id, int kvm_fd)
     cpu->kvm_plane_state[plane_id] = p;
 }
 
-void kvm_create_vcpu_plane(CPUState *cpu, unsigned plane_id, int kvm_fd)
+void kvm_create_vcpu_plane(CPUState *cpu, unsigned plane_id)
 {
-    int vcpu_fd = cpu_kvm_plane(cpu, 0)->kvm_fd;
-    int plane_fd = kvm_vm_plane_ioctl(cpu->kvm_state, plane_id, KVM_CREATE_VCPU, vcpu_fd);
+    X86CPU *x86_cpu = X86_CPU(cpu);
+    int plane_fd;
 
+    if (kvm_get_or_create_plane_fd(cpu->kvm_state, plane_id) < 0) {
+        fprintf(stderr, "Failed to create plane %d\n", plane_id);
+        abort();
+    }
+
+    plane_fd = kvm_vm_plane_ioctl(cpu->kvm_state, plane_id, KVM_CREATE_VCPU, x86_cpu->apic_id);
     if (plane_fd < 0) {
         fprintf(stderr, "Failed to create plane vcpu\n");
         abort();
@@ -586,7 +592,6 @@ void kvm_create_vcpu_plane(CPUState *cpu, unsigned plane_id, int kvm_fd)
     kvm_alloc_vcpu_plane(cpu, plane_id, plane_fd);
 }
 
-
 /**
  * kvm_create_vcpu - Gets a parked KVM vCPU or creates a KVM vCPU
  * @cpu: QOM CPUState object for which KVM vCPU has to be fetched/created.
include/system/kvm.h +1 -1
index 16597333cfa5..24a21915366f 100644
@@ -221,7 +221,7 @@ int kvm_vm_ioctl(KVMState *s, unsigned long type, ...);
 int kvm_vm_plane_ioctl(KVMState *s, unsigned plane_id, unsigned long type, ...);
 
 int kvm_get_or_create_plane_fd(KVMState *s, unsigned id);
-void kvm_create_vcpu_plane(CPUState *cpu, unsigned plane, int kvm_fd);
+void kvm_create_vcpu_plane(CPUState *cpu, unsigned plane);
 
 void kvm_flush_coalesced_mmio_buffer(void);
 
target/i386/kvm/kvm.c +57 -0

69 lines not on this page · raw message

Signature · 1 line
2.53.0
#7 lore raw

RFC PATCH 7/10 hw/core/machine: Add device-plane property

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Add a property to the QEMU MachineState to specify the default plane to send device interrupts to.

---
 hw/core/machine.c        | 22 ++++++++++++++++++++++
 include/hw/core/boards.h |  3 +++
 include/hw/core/qdev.h   |  1 +
 3 files changed, 26 insertions(+)
hw/core/machine.c +22 -0
index 0aa77a57e956..62ea86512645 100644
@@ -1218,6 +1218,7 @@ static void machine_initfn(Object *obj)
     ms->kernel_cmdline = g_strdup("");
     ms->ram_size = mc->default_ram_size;
     ms->maxram_size = mc->default_ram_size;
+    ms->device_plane = 0;
 
     if (mc->nvdimm_supported) {
         ms->nvdimms_state = g_new0(NVDIMMState, 1);
@@ -1253,6 +1254,12 @@ static void machine_initfn(Object *obj)
                                    "ACPI Serial Port Console Redirection "
                                    "Table (spcr)");
 
+    /* Default Device Plane */
+    object_property_add_uint8_ptr(obj, "device-plane", &ms->device_plane,
+                                  OBJ_PROP_FLAG_READWRITE);
+    object_property_set_description(obj, "device-plane",
+                                    "Default plane to receive device IRQs");
+
     /* default to mc->default_cpus */
     ms->smp.cpus = mc->default_cpus;
     ms->smp.max_cpus = mc->default_cpus;
@@ -1675,6 +1682,12 @@ void machine_run_board_init(MachineState *machine, const char *mem_path, Error *
                                    "on", false);
     }
 
+    if (machine->device_plane >= accel_nr_planes(machine)) {
+        error_report("Invalid plane specified: %d (highest supported plane: %d)",
+                     machine->device_plane, accel_nr_planes(machine) - 1);
+        exit(EXIT_FAILURE);
+    }
+
     accel_init_interfaces(ACCEL_GET_CLASS(machine->accelerator));
     machine_class->init(machine);
     phase_advance(PHASE_MACHINE_INITIALIZED);
@@ -1761,6 +1774,15 @@ void qdev_machine_creation_done(void)
     register_global_state();
 }
 
+uint8_t qdev_default_plane(void)
+{
+    if (current_machine != NULL) {
+        return current_machine->device_plane;
+    } else {
+        return 0;
+    }
+}
+
 static const TypeInfo machine_info = {
     .name = TYPE_MACHINE,
     .parent = TYPE_OBJECT,
include/hw/core/boards.h +3 -0
index b8dad0a1074d..d2d1336939ed 100644
@@ -447,6 +447,9 @@ struct MachineState {
      * Set to false by default for all regular use.
      */
     bool new_accel_vmfd_on_reset;
+
+    /* Default plane to receive device IRQs */
+    uint8_t device_plane;
 };
 
 /*
include/hw/core/qdev.h +1 -0
index f99a8979ccb1..83ad1d5f1550 100644
@@ -560,6 +560,7 @@ void qdev_simple_device_unplug_cb(HotplugHandler *hotplug_dev,
                                   DeviceState *dev, Error **errp);
 void qdev_machine_creation_done(void);
 bool qdev_machine_modified(void);
+uint8_t qdev_default_plane(void);
 
 /**
  * qdev_add_unplug_blocker: Add an unplug blocker to a device
Signature · 1 line
2.53.0
#8 lore raw

RFC PATCH 8/10 qdev: Add plane property

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Add a property to track the plane into which the qdev needs to inject IRQs.

---
 hw/core/qdev.c                      | 26 ++++++++++++++++++++++++++
 include/hw/core/qdev.h              |  4 ++++
 tests/unit/test-qdev-global-props.c |  5 +++++
 tests/unit/test-qdev.c              |  5 +++++
 4 files changed, 40 insertions(+)
hw/core/qdev.c +26 -0
index e48616b2c6f2..73d18fc0d639 100644
@@ -662,6 +662,28 @@ static bool device_get_hotplugged(Object *obj, Error **errp)
     return dev->hotplugged;
 }
 
+static void device_get_plane(Object *obj, Visitor *v, const char *name,
+                             void *opaque, Error **errp)
+{
+    DeviceState *dev = DEVICE(obj);
+    uint8_t value = dev->plane;
+
+    visit_type_uint8(v, name, &value, errp);
+}
+
+static void device_set_plane(Object *obj, Visitor *v, const char *name,
+                             void *opaque, Error **errp)
+{
+    DeviceState *dev = DEVICE(obj);
+    uint8_t value;
+
+    if (!visit_type_uint8(v, name, &value, errp)) {
+        return;
+    }
+
+    dev->plane = value;
+}
+
 static void device_initfn(Object *obj)
 {
     DeviceState *dev = DEVICE(obj);
@@ -674,6 +696,7 @@ static void device_initfn(Object *obj)
     dev->instance_id_alias = -1;
     dev->realized = false;
     dev->allow_unplug_during_migration = false;
+    dev->plane = qdev_default_plane();
 
     QLIST_INIT(&dev->gpios);
     QLIST_INIT(&dev->clocks);
@@ -796,6 +819,9 @@ static void device_class_init(ObjectClass *class, const void *data)
                                    device_get_hotplugged, NULL);
     object_class_property_add_link(class, "parent_bus", TYPE_BUS,
                                    offsetof(DeviceState, parent_bus), NULL, 0);
+    object_class_property_add(class, "plane", "uint8",
+                              device_get_plane, device_set_plane,
+                              NULL, NULL);
 }
 
 static void do_legacy_reset(Object *obj, ResetType type)
include/hw/core/qdev.h +4 -0
index 83ad1d5f1550..28d2efcbe455 100644
@@ -295,6 +295,10 @@ struct DeviceState {
      * Used to prevent re-entrancy confusing things.
      */
     MemReentrancyGuard mem_reentrancy_guard;
+    /**
+     * @plane: Plane the device is assigned to.
+     */
+    uint8_t plane;
 };
 
 typedef struct DeviceListener DeviceListener;
tests/unit/test-qdev-global-props.c +5 -0
index 8ea362cbb902..2aca5bda22b9 100644
@@ -71,6 +71,11 @@ static const TypeInfo subclass_type = {
     .parent = TYPE_STATIC_PROPS,
 };
 
+uint8_t qdev_default_plane(void)
+{
+    return 0;
+}
+
 /*
  * Initialize a fake machine, being prepared for future tests.
  *
tests/unit/test-qdev.c +5 -0
index 20eae38e03f4..6e3127b41afd 100644
@@ -26,6 +26,11 @@ static const Property my_dev_props[] = {
                      qdev_prop_uint32, uint32_t),
 };
 
+uint8_t qdev_default_plane(void)
+{
+    return 0;
+}
+
 static void my_dev_class_init(ObjectClass *oc, const void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(oc);
Signature · 1 line
2.53.0
#9 lore raw

RFC PATCH 9/10 MSI: Inject into correct plane

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

Inject MSI and MSI-X IRQs into the correct device plane.

---
 accel/kvm/kvm-all.c  | 2 +-
 hw/i386/kvm/apic.c   | 6 +++++-
 hw/pci/msi.c         | 3 +++
 hw/pci/msix.c        | 3 +++
 include/hw/pci/msi.h | 1 +
 5 files changed, 13 insertions(+), 2 deletions(-)
accel/kvm/kvm-all.c +1 -1
index c5fe6d189e62..31d80f7ac48b 100644
@@ -2407,7 +2407,7 @@ int kvm_irqchip_send_msi(KVMState *s, MSIMessage msg)
     msi.flags = 0;
     memset(msi.pad, 0, sizeof(msi.pad));
 
-    return kvm_vm_ioctl(s, KVM_SIGNAL_MSI, &msi);
+    return kvm_vm_plane_ioctl(s, msg.plane_id, KVM_SIGNAL_MSI, &msi);
 }
 
 int kvm_irqchip_add_msi_route(KVMRouteChange *c, int vector, PCIDevice *dev)
hw/i386/kvm/apic.c +5 -1
index 82355f04631a..4dd946d6f26b 100644
@@ -210,7 +210,11 @@ static uint64_t kvm_apic_mem_read(void *opaque, hwaddr addr,
 static void kvm_apic_mem_write(void *opaque, hwaddr addr,
                                uint64_t data, unsigned size)
 {
-    MSIMessage msg = { .address = addr, .data = data };
+    MSIMessage msg = {
+	    .address = addr,
+	    .data = data,
+	    .plane_id = qdev_default_plane(),
+    };
 
     kvm_send_msi(&msg);
 }
hw/pci/msi.c +3 -0
index b9f5b45920b6..d0373131dd06 100644
@@ -142,6 +142,7 @@ static MSIMessage msi_prepare_message(PCIDevice *dev, unsigned int vector)
     uint16_t flags = pci_get_word(dev->config + msi_flags_off(dev));
     bool msi64bit = flags & PCI_MSI_FLAGS_64BIT;
     unsigned int nr_vectors = msi_nr_vectors(flags);
+    DeviceState *dev_state= DEVICE(dev);
     MSIMessage msg;
 
     assert(vector < nr_vectors);
@@ -159,6 +160,8 @@ static MSIMessage msi_prepare_message(PCIDevice *dev, unsigned int vector)
         msg.data |= vector;
     }
 
+    msg.plane_id =  dev_state->plane;
+
     return msg;
 }
 
hw/pci/msix.c +3 -0
index 1b23eaf10079..1773f8eccae8 100644
@@ -37,10 +37,13 @@
 static MSIMessage msix_prepare_message(PCIDevice *dev, unsigned vector)
 {
     uint8_t *table_entry = dev->msix_table + vector * PCI_MSIX_ENTRY_SIZE;
+    DeviceState *dev_state= DEVICE(dev);
     MSIMessage msg;
 
     msg.address = pci_get_quad(table_entry + PCI_MSIX_ENTRY_LOWER_ADDR);
     msg.data = pci_get_long(table_entry + PCI_MSIX_ENTRY_DATA);
+    msg.plane_id = dev_state->plane;
+
     return msg;
 }
 
include/hw/pci/msi.h +1 -0
index abcfd1392521..6bedf97b6f03 100644
@@ -26,6 +26,7 @@
 struct MSIMessage {
     uint64_t address;
     uint32_t data;
+    uint8_t plane_id;
 };
 
 extern bool msi_nonbroken;
Signature · 1 line
2.53.0
#10 lore raw

RFC PATCH 10/10 KVM: Set GSI routes for default plane

In reply to Jörg Rödel

From: Joerg Roedel <[email protected]>

This ensures that all IOAPIC IRQs are routed to the default device plane in the KVM guest.

---
 accel/kvm/kvm-all.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
accel/kvm/kvm-all.c +10 -1
index 31d80f7ac48b..2bd98efaadab 100644
@@ -134,6 +134,7 @@ static int map_kvm_run(KVMState *s, CPUState *cpu, Error **errp);
 static int map_kvm_dirty_gfns(KVMState *s, CPUState *cpu, Error **errp);
 static int vcpu_unmap_regions(KVMState *s, CPUState *cpu);
 static void kvm_alloc_vcpu_plane(CPUState *cpu, unsigned plane_id, int kvm_fd);
+static int kvm_create_plane(KVMState *s, unsigned id);
 
 struct KVMResampleFd {
     int gsi;
@@ -2238,6 +2239,7 @@ void kvm_init_irq_routing(KVMState *s)
 
 void kvm_irqchip_commit_routes(KVMState *s)
 {
+    unsigned plane = qdev_default_plane();
     int ret;
 
     if (kvm_gsi_direct_mapping()) {
@@ -2250,7 +2252,7 @@ void kvm_irqchip_commit_routes(KVMState *s)
 
     s->irq_routes->flags = 0;
     trace_kvm_irqchip_commit_routes();
-    ret = kvm_vm_ioctl(s, KVM_SET_GSI_ROUTING, s->irq_routes);
+    ret = kvm_vm_plane_ioctl(s, plane, KVM_SET_GSI_ROUTING, s->irq_routes);
     assert(ret == 0);
 }
 
@@ -2667,6 +2669,8 @@ static int do_kvm_irqchip_create(KVMState *s)
 
 static void kvm_irqchip_create(KVMState *s)
 {
+    int device_plane = qdev_default_plane();
+
     assert(s->kernel_irqchip_split != ON_OFF_AUTO_AUTO);
 
     if (do_kvm_irqchip_create(s) < 0) {
@@ -2679,6 +2683,11 @@ static void kvm_irqchip_create(KVMState *s)
     kvm_async_interrupts_allowed = true;
     kvm_halt_in_kernel_allowed = true;
 
+    /* Make sure irqchip target plane is known to KVM */
+    if (device_plane != 0) {
+        kvm_create_plane(s, device_plane);
+    }
+
     kvm_init_irq_routing(s);
 
     s->gsimap = g_hash_table_new(g_direct_hash, g_direct_equal);
Signature · 1 line
2.53.0
#11 lore raw
In reply to Jörg Rödel
Quoted 14 lines · Jörg Rödel

From: Joerg Roedel <[email protected]>

Hi,

here are the required QEMU changes to make use of the KVM Planes interface posted here[1].

The patches are based on QEMU v11.0.0 and can be used to launch an AMD SEV-SNP VM with COCONUT-SVSM + a Linux guest.

To make this work a change to the QEMU command line is required to tell QEMU which plane to target external IRQs to. this is done with the new device-plane property to the machine specification, e.g:

Quoted 8 lines
$ qemu-system-x86_64 \
  -enable-kvm \
  -cpu EPYC-v4 \
  -machine q35,confidential-guest-support=sev0,memory-backend=ram1,igvm-cfg=igvm0,kernel-irqchip=split,device-plane=2 \
  -object memory-backend-memfd,id=ram1,size=32G,share=true \
  -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1 \
  -object igvm-cfg,id=igvm0,file=$IGVM_FILE \
  ...

How are device-plane values intended to be chosen by the user ?

Is "2" a value that can/should always be used, or is there a dependency on the igvm file, or somethihng else ?

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|
#12 lore raw

Hi Daniel,

Quoted 4 lines · Daniel P. Berrangé

How are device-plane values intended to be chosen by the user ?

Is "2" a value that can/should always be used, or is there a dependency on the igvm file, or somethihng else ?

Currently it is based on how COCONUT in the IGVM file it built. It will launch the Linux guest at plane 2, so all IRQs must go to plane 2 as well. For the future we are considering passing the device-plane value to the SVSM so it can decide on which plane to run Linux based on that value.

-Joerg