Skip to contentSkip to conversation

Pass zPCI hardware information via VFIO

PATCH In review Matthew Rosato 4 participants 14 messages Oct 7, 2020 – Oct 9, 2020

Subsystems (5)
KVM s390 VFIO driver s390 PCI +1 more subsystems
lore
Share & tools
lore

Matthew Rosato

author maintainer
#0 lore raw

v3 cover Pass zPCI hardware information via VFIO

This patchset provides a means by which hardware information about the underlying PCI device can be passed up to userspace (ie, QEMU) so that this hardware information can be used rather than previously hard-coded assumptions. The VFIO_DEVICE_GET_INFO ioctl is extended to allow capability chains and zPCI devices provide the hardware information via capabilities.

A form of these patches saw some rounds last year but has been back- tabled for a while. The original work for this feature was done by Pierre Morel. I'd like to refresh the discussion on this and get this finished up so that we can move forward with better-supporting additional types of PCI-attached devices.

This feature is toggled via the CONFIG_VFIO_PCI_ZDEV configuration entry.

Changes since v2:
- Added ACKs (thanks!)
- Patch 3+4: Re-write to use VFIO_DEVICE_GET_INFO capabilities rather than a vfio device region.

Matthew Rosato (5):
s390/pci: stash version in the zpci_dev
s390/pci: track whether util_str is valid in the zpci_dev
vfio: Introduce capability definitions for VFIO_DEVICE_GET_INFO vfio-pci/zdev: Add zPCI capabilities to VFIO_DEVICE_GET_INFO
MAINTAINERS: Add entry for s390 vfio-pci

 MAINTAINERS                         |   8 ++
 arch/s390/include/asm/pci.h         |   4 +-
 arch/s390/pci/pci_clp.c             |   2 +
 drivers/vfio/pci/Kconfig            |  13 ++++
 drivers/vfio/pci/Makefile           |   1 +
 drivers/vfio/pci/vfio_pci.c         |  37 ++++++++++
 drivers/vfio/pci/vfio_pci_private.h |  12 +++
 drivers/vfio/pci/vfio_pci_zdev.c    | 143 ++++++++++++++++++++++++++++++++++++
 include/uapi/linux/vfio.h           |  11 +++
 include/uapi/linux/vfio_zdev.h      |  78 ++++++++++++++++++++
 10 files changed, 308 insertions(+), 1 deletion(-)
 create mode 100644 drivers/vfio/pci/vfio_pci_zdev.c
 create mode 100644 include/uapi/linux/vfio_zdev.h
Signature · 1 line
1.8.3.1
author maintainer
#1 lore raw

PATCH v3 1/5 s390/pci: stash version in the zpci_dev

In preparation for passing the info on to vfio-pci devices, stash the supported PCI version for the target device in the zpci_dev.

---
 arch/s390/include/asm/pci.h | 1 +
 arch/s390/pci/pci_clp.c     | 1 +
 2 files changed, 2 insertions(+)
arch/s390/include/asm/pci.h +1 -0
index 99b92c3..882e233 100644
@@ -179,6 +179,7 @@ struct zpci_dev {
 	atomic64_t mapped_pages;
 	atomic64_t unmapped_pages;
 
+	u8		version;
 	enum pci_bus_speed max_bus_speed;
 
 	struct dentry	*debugfs_dev;
arch/s390/pci/pci_clp.c +1 -0
index 7e735f4..48bf316 100644
@@ -102,6 +102,7 @@ static void clp_store_query_pci_fngrp(struct zpci_dev *zdev,
 	zdev->msi_addr = response->msia;
 	zdev->max_msi = response->noi;
 	zdev->fmb_update = response->mui;
+	zdev->version = response->version;
 
 	switch (response->version) {
 	case 1:
Signature · 1 line
1.8.3.1
author maintainer
#2 lore raw

PATCH v3 2/5 s390/pci: track whether util_str is valid in the zpci_dev

In reply to Matthew Rosato

We'll need to keep track of whether or not the byte string in util_str is valid and thus needs to be passed to a vfio-pci passthrough device.

---
 arch/s390/include/asm/pci.h | 3 ++-
 arch/s390/pci/pci_clp.c     | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)
arch/s390/include/asm/pci.h +2 -1
index 882e233..fa1fed4 100644
@@ -132,7 +132,8 @@ struct zpci_dev {
 	u8		rid_available	: 1;
 	u8		has_hp_slot	: 1;
 	u8		is_physfn	: 1;
-	u8		reserved	: 5;
+	u8		util_str_avail	: 1;
+	u8		reserved	: 4;
 	unsigned int	devfn;		/* DEVFN part of the RID*/
 
 	struct mutex lock;
arch/s390/pci/pci_clp.c +1 -0
index 48bf316..322689b 100644
@@ -168,6 +168,7 @@ static int clp_store_query_pci_fn(struct zpci_dev *zdev,
 	if (response->util_str_avail) {
 		memcpy(zdev->util_str, response->util_str,
 		       sizeof(zdev->util_str));
+		zdev->util_str_avail = 1;
 	}
 	zdev->mio_capable = response->mio_addr_avail;
 	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
Signature · 1 line
1.8.3.1
author maintainer
#3 lore raw

PATCH v3 3/5 vfio: Introduce capability definitions for VFIO_DEVICE_GET_INFO

In reply to Matthew Rosato

Allow the VFIO_DEVICE_GET_INFO ioctl to include a capability chain. Add a flag indicating capability chain support, and introduce the definitions for the first set of capabilities which are specified to s390 zPCI devices.

---
 include/uapi/linux/vfio.h      | 11 ++++++
 include/uapi/linux/vfio_zdev.h | 78 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 89 insertions(+)
 create mode 100644 include/uapi/linux/vfio_zdev.h
include/uapi/linux/vfio.h +11 -0
index 9204705..836a25b 100644
@@ -201,8 +201,10 @@ struct vfio_device_info {
 #define VFIO_DEVICE_FLAGS_AMBA  (1 << 3)	/* vfio-amba device */
 #define VFIO_DEVICE_FLAGS_CCW	(1 << 4)	/* vfio-ccw device */
 #define VFIO_DEVICE_FLAGS_AP	(1 << 5)	/* vfio-ap device */
+#define VFIO_DEVICE_FLAGS_CAPS	(1 << 6)	/* Info supports caps */
 	__u32	num_regions;	/* Max region index + 1 */
 	__u32	num_irqs;	/* Max IRQ index + 1 */
+	__u32   cap_offset;	/* Offset within info struct of first cap */
 };
 #define VFIO_DEVICE_GET_INFO		_IO(VFIO_TYPE, VFIO_BASE + 7)
 
@@ -218,6 +220,15 @@ struct vfio_device_info {
 #define VFIO_DEVICE_API_CCW_STRING		"vfio-ccw"
 #define VFIO_DEVICE_API_AP_STRING		"vfio-ap"
 
+/*
+ * The following capabilities are unique to s390 zPCI devices.  Their contents
+ * are further-defined in vfio_zdev.h
+ */
+#define VFIO_DEVICE_INFO_CAP_ZPCI_BASE		1
+#define VFIO_DEVICE_INFO_CAP_ZPCI_GROUP		2
+#define VFIO_DEVICE_INFO_CAP_ZPCI_UTIL		3
+#define VFIO_DEVICE_INFO_CAP_ZPCI_PFIP		4
+
 /**
  * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8,
  *				       struct vfio_region_info)
include/uapi/linux/vfio_zdev.h new file +78 -0

78 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#4 lore raw

Re: [PATCH v3 3/5] vfio: Introduce capability definitions for VFIO_DEVICE_GET_INFO

Quoted 17 lines · On Wed, 7 Oct 2020 14:56:22 -0400 Matthew Rosato <[email protected]>…

Allow the VFIO_DEVICE_GET_INFO ioctl to include a capability chain. Add a flag indicating capability chain support, and introduce the definitions for the first set of capabilities which are specified to s390 zPCI devices.

---
 include/uapi/linux/vfio.h      | 11 ++++++
 include/uapi/linux/vfio_zdev.h | 78 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 89 insertions(+)
 create mode 100644 include/uapi/linux/vfio_zdev.h
include/uapi/linux/vfio.h +0 -0
index 9204705..836a25b 100644

include/uapi/linux/vfio.h struct vfio_device_info { · line 204

Quoted 4 lines
include/uapi/linux/vfio.h +1 -0
@@ -201,8 +201,10 @@ struct vfio_device_info {
 #define VFIO_DEVICE_FLAGS_AMBA  (1 << 3)	/* vfio-amba device */
 #define VFIO_DEVICE_FLAGS_CCW	(1 << 4)	/* vfio-ccw device */
 #define VFIO_DEVICE_FLAGS_AP	(1 << 5)	/* vfio-ap device */
+#define VFIO_DEVICE_FLAGS_CAPS	(1 << 6)	/* Info supports caps */

FYI, I'm going to change this to (1 << 7) because the new fsl-mc bus driver patches are claiming a new device type with (1 << 6) and I don't want the conflict to be magically resolved on merge. Thanks,

Alex

Quoted 106 lines
include/uapi/linux/vfio.h +10 -0
 	__u32	num_regions;	/* Max region index + 1 */
 	__u32	num_irqs;	/* Max IRQ index + 1 */
+	__u32   cap_offset;	/* Offset within info struct of first cap */
 };
 #define VFIO_DEVICE_GET_INFO		_IO(VFIO_TYPE, VFIO_BASE + 7)
 
@@ -218,6 +220,15 @@ struct vfio_device_info {
 #define VFIO_DEVICE_API_CCW_STRING		"vfio-ccw"
 #define VFIO_DEVICE_API_AP_STRING		"vfio-ap"
 
+/*
+ * The following capabilities are unique to s390 zPCI devices.  Their contents
+ * are further-defined in vfio_zdev.h
+ */
+#define VFIO_DEVICE_INFO_CAP_ZPCI_BASE		1
+#define VFIO_DEVICE_INFO_CAP_ZPCI_GROUP		2
+#define VFIO_DEVICE_INFO_CAP_ZPCI_UTIL		3
+#define VFIO_DEVICE_INFO_CAP_ZPCI_PFIP		4
+
 /**
  * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8,
  *				       struct vfio_region_info)
include/uapi/linux/vfio_zdev.h new file +78 -0
new file mode 100644
index 0000000..b430939
@@ -0,0 +1,78 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * VFIO Region definitions for ZPCI devices
+ *
+ * Copyright IBM Corp. 2020
+ *
+ * Author(s): Pierre Morel <[email protected]>
+ *            Matthew Rosato <[email protected]>
+ */
+
+#ifndef _VFIO_ZDEV_H_
+#define _VFIO_ZDEV_H_
+
+#include <linux/types.h>
+#include <linux/vfio.h>
+
+/**
+ * VFIO_DEVICE_INFO_CAP_ZPCI_BASE - Base PCI Function information
+ *
+ * This capability provides a set of descriptive information about the
+ * associated PCI function.
+ */
+struct vfio_device_info_cap_zpci_base {
+	struct vfio_info_cap_header header;
+	__u64 start_dma;	/* Start of available DMA addresses */
+	__u64 end_dma;		/* End of available DMA addresses */
+	__u16 pchid;		/* Physical Channel ID */
+	__u16 vfn;		/* Virtual function number */
+	__u16 fmb_length;	/* Measurement Block Length (in bytes) */
+	__u8 pft;		/* PCI Function Type */
+	__u8 gid;		/* PCI function group ID */
+};
+
+/**
+ * VFIO_DEVICE_INFO_CAP_ZPCI_GROUP - Base PCI Function Group information
+ *
+ * This capability provides a set of descriptive information about the group of
+ * PCI functions that the associated device belongs to.
+ */
+struct vfio_device_info_cap_zpci_group {
+	struct vfio_info_cap_header header;
+	__u64 dasm;		/* DMA Address space mask */
+	__u64 msi_addr;		/* MSI address */
+	__u64 flags;
+#define VFIO_DEVICE_INFO_ZPCI_FLAG_REFRESH 1 /* Program-specified TLB refresh */
+	__u16 mui;		/* Measurement Block Update Interval */
+	__u16 noi;		/* Maximum number of MSIs */
+	__u16 maxstbl;		/* Maximum Store Block Length */
+	__u8 version;		/* Supported PCI Version */
+};
+
+/**
+ * VFIO_DEVICE_INFO_CAP_ZPCI_UTIL - Utility String
+ *
+ * This capability provides the utility string for the associated device, which
+ * is a device identifier string made up of EBCDID characters.  'size' specifies
+ * the length of 'util_str'.
+ */
+struct vfio_device_info_cap_zpci_util {
+	struct vfio_info_cap_header header;
+	__u32 size;
+	__u8 util_str[];
+};
+
+/**
+ * VFIO_DEVICE_INFO_CAP_ZPCI_PFIP - PCI Function Path
+ *
+ * This capability provides the PCI function path string, which is an identifier
+ * that describes the internal hardware path of the device. 'size' specifies
+ * the length of 'pfip'.
+ */
+struct vfio_device_info_cap_zpci_pfip {
+	struct vfio_info_cap_header header;
+	__u32 size;
+	__u8 pfip[];
+};
+
+#endif
Reviewed-by on v3 3/5
#5 lore raw

Re: [PATCH v3 3/5] vfio: Introduce capability definitions for VFIO_DEVICE_GET_INFO

In reply to Matthew Rosato
Quoted 11 lines · On Wed, 7 Oct 2020 14:56:22 -0400 Matthew Rosato <[email protected]>…

Allow the VFIO_DEVICE_GET_INFO ioctl to include a capability chain. Add a flag indicating capability chain support, and introduce the definitions for the first set of capabilities which are specified to s390 zPCI devices.

---
 include/uapi/linux/vfio.h      | 11 ++++++
 include/uapi/linux/vfio_zdev.h | 78 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 89 insertions(+)
 create mode 100644 include/uapi/linux/vfio_zdev.h
author maintainer
#6 lore raw

PATCH v3 4/5 vfio-pci/zdev: Add zPCI capabilities to VFIO_DEVICE_GET_INFO

In reply to Matthew Rosato

Define a new configuration entry VFIO_PCI_ZDEV for VFIO/PCI.

When this s390-only feature is configured we add capabilities to the VFIO_DEVICE_GET_INFO ioctl that describe features of the associated zPCI device and its underlying hardware.

This patch is based on work previously done by Pierre Morel.

---
 drivers/vfio/pci/Kconfig            |  13 ++++
 drivers/vfio/pci/Makefile           |   1 +
 drivers/vfio/pci/vfio_pci.c         |  37 ++++++++++
 drivers/vfio/pci/vfio_pci_private.h |  12 +++
 drivers/vfio/pci/vfio_pci_zdev.c    | 143 ++++++++++++++++++++++++++++++++++++
 5 files changed, 206 insertions(+)
 create mode 100644 drivers/vfio/pci/vfio_pci_zdev.c
drivers/vfio/pci/Kconfig +13 -0
index ac3c1dd..6adf37a 100644
@@ -45,3 +45,16 @@ config VFIO_PCI_NVLINK2
 	depends on VFIO_PCI && PPC_POWERNV
 	help
 	  VFIO PCI support for P9 Witherspoon machine with NVIDIA V100 GPUs
+
+config VFIO_PCI_ZDEV
+	bool "VFIO PCI ZPCI device CLP support"
+	depends on VFIO_PCI && S390
+	default y
+	help
+	  Enabling this option exposes VFIO capabilities containing hardware
+	  configuration for zPCI devices. This enables userspace (e.g. QEMU)
+	  to supply proper configuration values instead of hard-coded defaults
+	  for zPCI devices passed through via VFIO on s390.
+
+	  Say Y here.
+
drivers/vfio/pci/Makefile +1 -0
index f027f8a..781e080 100644
@@ -3,5 +3,6 @@
 vfio-pci-y := vfio_pci.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o
 vfio-pci-$(CONFIG_VFIO_PCI_IGD) += vfio_pci_igd.o
 vfio-pci-$(CONFIG_VFIO_PCI_NVLINK2) += vfio_pci_nvlink2.o
+vfio-pci-$(CONFIG_VFIO_PCI_ZDEV) += vfio_pci_zdev.o
 
 obj-$(CONFIG_VFIO_PCI) += vfio-pci.o
drivers/vfio/pci/vfio_pci.c +37 -0
index 1ab1f5c..208dea5 100644
@@ -807,15 +807,25 @@ static long vfio_pci_ioctl(void *device_data,
 
 	if (cmd == VFIO_DEVICE_GET_INFO) {
 		struct vfio_device_info info;
+		struct vfio_info_cap caps = { .buf = NULL, .size = 0 };
+		unsigned long capsz;
 
 		minsz = offsetofend(struct vfio_device_info, num_irqs);
 
+		/* For backward compatibility, cannot require this */
+		capsz = offsetofend(struct vfio_iommu_type1_info, cap_offset);
+
 		if (copy_from_user(&info, (void __user *)arg, minsz))
 			return -EFAULT;
 
 		if (info.argsz < minsz)
 			return -EINVAL;
 
+		if (info.argsz >= capsz) {
+			minsz = capsz;
+			info.cap_offset = 0;
+		}
+
 		info.flags = VFIO_DEVICE_FLAGS_PCI;
 
 		if (vdev->reset_works)
@@ -824,6 +834,33 @@ static long vfio_pci_ioctl(void *device_data,
 		info.num_regions = VFIO_PCI_NUM_REGIONS + vdev->num_regions;
 		info.num_irqs = VFIO_PCI_NUM_IRQS;
 
+		if (IS_ENABLED(CONFIG_VFIO_PCI_ZDEV)) {
+			int ret = vfio_pci_info_zdev_add_caps(vdev, &caps);
+
+			if (ret && ret != -ENODEV) {
+				pci_warn(vdev->pdev, "Failed to setup zPCI info capabilities\n");
+				return ret;
+			}
+		}
+
+		if (caps.size) {
+			info.flags |= VFIO_DEVICE_FLAGS_CAPS;
+			if (info.argsz < sizeof(info) + caps.size) {
+				info.argsz = sizeof(info) + caps.size;
+			} else {
+				vfio_info_cap_shift(&caps, sizeof(info));
+				if (copy_to_user((void __user *)arg +
+						  sizeof(info), caps.buf,
+						  caps.size)) {
+					kfree(caps.buf);
+					return -EFAULT;
+				}
+				info.cap_offset = sizeof(info);
+			}
+
+			kfree(caps.buf);
+		}
+
 		return copy_to_user((void __user *)arg, &info, minsz) ?
 			-EFAULT : 0;
 
drivers/vfio/pci/vfio_pci_private.h +12 -0
index 61ca8ab..9d28484 100644
@@ -213,4 +213,16 @@ static inline int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
 	return -ENODEV;
 }
 #endif
+
+#ifdef CONFIG_VFIO_PCI_ZDEV
+extern int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev,
+				       struct vfio_info_cap *caps);
+#else
+static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev,
+					      struct vfio_info_cap *caps);
+{
+	return -ENODEV;
+}
+#endif
+
 #endif /* VFIO_PCI_PRIVATE_H */
drivers/vfio/pci/vfio_pci_zdev.c new file +143 -0

143 lines not on this page · raw message

Signature · 1 line
1.8.3.1
#7 lore raw

Re: [PATCH v3 4/5] vfio-pci/zdev: Add zPCI capabilities to VFIO_DEVICE_GET_INFO

Quoted 5 lines · On Wed, 7 Oct 2020 14:56:23 -0400 Matthew Rosato <[email protected]>…
drivers/vfio/pci/vfio_pci_private.h +0 -0
index 61ca8ab..9d28484 100644

drivers/vfio/pci/vfio_pci_private.h static inline int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev) · line 222

Quoted 10 lines
drivers/vfio/pci/vfio_pci_private.h +7 -0
@@ -213,4 +213,16 @@ static inline int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev)
 	return -ENODEV;
 }
 #endif
+
+#ifdef CONFIG_VFIO_PCI_ZDEV
+extern int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev,
+				       struct vfio_info_cap *caps);
+#else
+static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev,
+					      struct vfio_info_cap *caps);

Ooops......................................................................^

Fixed. Thanks,

Alex

#8 lore raw

Re: [PATCH v3 4/5] vfio-pci/zdev: Add zPCI capabilities to VFIO_DEVICE_GET_INFO

In reply to Matthew Rosato
I love your patch! Yet something to improve:

Hi Matthew,

I love your patch! Yet something to improve:

[auto build test ERROR on vfio/next]
[also build test ERROR on s390/features linus/master kvms390/next v5.9-rc8 next-20201007] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Matthew-Rosato/Pass-zPCI-hardware-information-via-VFIO/20201008-034906
base:   https://github.com/awilliam/linux-vfio.git next
config: xtensa-allyesconfig (attached as .config)
compiler: xtensa-linux-gcc (GCC) 9.3.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/c09b4427b95cc448fb0ed2a4b3125643dea2bc7b
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matthew-Rosato/Pass-zPCI-hardware-information-via-VFIO/20201008-034906 git checkout c09b4427b95cc448fb0ed2a4b3125643dea2bc7b
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=xtensa

If you fix the issue, kindly add following tag as appropriate

All error/warnings (new ones prefixed by >>):

In file included from drivers/vfio/pci/vfio_pci.c:31:

>> drivers/vfio/pci/vfio_pci_private.h:223:1: error: expected identifier or '(' before '{' token
     223 | {
         | ^
>> drivers/vfio/pci/vfio_pci_private.h:221:19: warning: 'vfio_pci_info_zdev_add_caps' used but never defined

221 | static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
--
In file included from drivers/vfio/pci/vfio_pci_intrs.c:23:

>> drivers/vfio/pci/vfio_pci_private.h:223:1: error: expected identifier or '(' before '{' token
     223 | {
         | ^

drivers/vfio/pci/vfio_pci_private.h:221:19: warning: 'vfio_pci_info_zdev_add_caps' declared 'static' but never defined [-Wunused-function]
221 | static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~

vim +223 drivers/vfio/pci/vfio_pci_private.h

216
217 #ifdef CONFIG_VFIO_PCI_ZDEV
218 extern int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev,
219 struct vfio_info_cap *caps);
220 #else

 > 221	static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev,
   222						      struct vfio_info_cap *caps);
 > 223	{

224 return -ENODEV;
225 }
226 #endif
227

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#9 lore raw

Re: [PATCH v3 4/5] vfio-pci/zdev: Add zPCI capabilities to VFIO_DEVICE_GET_INFO

In reply to Matthew Rosato
I love your patch! Yet something to improve:

Hi Matthew,

I love your patch! Yet something to improve:

[auto build test ERROR on vfio/next]
[also build test ERROR on s390/features linus/master kvms390/next v5.9-rc8 next-20201007] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Matthew-Rosato/Pass-zPCI-hardware-information-via-VFIO/20201008-034906
base:   https://github.com/awilliam/linux-vfio.git next
config: x86_64-randconfig-a016-20201008 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project 4d1d8ae7100ec3c7e1709addb7b3ec6f9ad0b44f)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/c09b4427b95cc448fb0ed2a4b3125643dea2bc7b
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matthew-Rosato/Pass-zPCI-hardware-information-via-VFIO/20201008-034906 git checkout c09b4427b95cc448fb0ed2a4b3125643dea2bc7b
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=x86_64

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

In file included from drivers/vfio/pci/vfio_pci.c:31:

>> drivers/vfio/pci/vfio_pci_private.h:223:1: error: expected identifier or '('

{
^
1 error generated.

vim +223 drivers/vfio/pci/vfio_pci_private.h

216
217 #ifdef CONFIG_VFIO_PCI_ZDEV
218 extern int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev,
219 struct vfio_info_cap *caps);
220 #else
221 static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev,
222 struct vfio_info_cap *caps);

 > 223	{

224 return -ENODEV;
225 }
226 #endif
227

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

Reviewed-by on v3 4/5
#10 lore raw

Re: [PATCH v3 4/5] vfio-pci/zdev: Add zPCI capabilities to VFIO_DEVICE_GET_INFO

In reply to Matthew Rosato
Quoted 9 lines · On Wed, 7 Oct 2020 14:56:23 -0400 Matthew Rosato <[email protected]>…

Define a new configuration entry VFIO_PCI_ZDEV for VFIO/PCI.

When this s390-only feature is configured we add capabilities to the VFIO_DEVICE_GET_INFO ioctl that describe features of the associated zPCI device and its underlying hardware.

This patch is based on work previously done by Pierre Morel.

Quoted 8 lines
---
 drivers/vfio/pci/Kconfig            |  13 ++++
 drivers/vfio/pci/Makefile           |   1 +
 drivers/vfio/pci/vfio_pci.c         |  37 ++++++++++
 drivers/vfio/pci/vfio_pci_private.h |  12 +++
 drivers/vfio/pci/vfio_pci_zdev.c    | 143 ++++++++++++++++++++++++++++++++++++
 5 files changed, 206 insertions(+)
 create mode 100644 drivers/vfio/pci/vfio_pci_zdev.c

With the compilation fix,

author maintainer
#11 lore raw

PATCH v3 5/5 MAINTAINERS: Add entry for s390 vfio-pci

In reply to Matthew Rosato

Add myself to cover s390-specific items related to vfio-pci.

---
 MAINTAINERS | 8 ++++++++
 1 file changed, 8 insertions(+)
MAINTAINERS +8 -0
index 9a54806..a0e8d14 100644
@@ -15170,6 +15170,14 @@ F:	Documentation/s390/vfio-ccw.rst
 F:	drivers/s390/cio/vfio_ccw*
 F:	include/uapi/linux/vfio_ccw.h
 
+S390 VFIO-PCI DRIVER
+M:	Matthew Rosato <[email protected]>
+L:	[email protected]
+L:	[email protected]
+S:	Supported
+F:	drivers/vfio/pci/vfio_pci_zdev.c
+F:	include/uapi/linux/vfio_zdev.h
+
 S390 ZCRYPT DRIVER
 M:	Harald Freudenberger <[email protected]>
 L:	[email protected]
Signature · 1 line
1.8.3.1
author maintainer
#12 lore raw
In reply to Matthew Rosato
Quoted 12 lines · Matthew Rosato

This patchset provides a means by which hardware information about the underlying PCI device can be passed up to userspace (ie, QEMU) so that this hardware information can be used rather than previously hard-coded assumptions. The VFIO_DEVICE_GET_INFO ioctl is extended to allow capability chains and zPCI devices provide the hardware information via capabilities.

A form of these patches saw some rounds last year but has been back- tabled for a while. The original work for this feature was done by Pierre Morel. I'd like to refresh the discussion on this and get this finished up so that we can move forward with better-supporting additional types of PCI-attached devices.

Quoted 6 lines

This feature is toggled via the CONFIG_VFIO_PCI_ZDEV configuration entry.

Changes since v2:
- Added ACKs (thanks!)
- Patch 3+4: Re-write to use VFIO_DEVICE_GET_INFO capabilities rather than a vfio device region.

Link to latest QEMU patch set:
https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg01948.html

Quoted 22 lines

Matthew Rosato (5):
s390/pci: stash version in the zpci_dev
s390/pci: track whether util_str is valid in the zpci_dev
vfio: Introduce capability definitions for VFIO_DEVICE_GET_INFO vfio-pci/zdev: Add zPCI capabilities to VFIO_DEVICE_GET_INFO
MAINTAINERS: Add entry for s390 vfio-pci

  MAINTAINERS                         |   8 ++
  arch/s390/include/asm/pci.h         |   4 +-
  arch/s390/pci/pci_clp.c             |   2 +
  drivers/vfio/pci/Kconfig            |  13 ++++
  drivers/vfio/pci/Makefile           |   1 +
  drivers/vfio/pci/vfio_pci.c         |  37 ++++++++++
  drivers/vfio/pci/vfio_pci_private.h |  12 +++
  drivers/vfio/pci/vfio_pci_zdev.c    | 143 ++++++++++++++++++++++++++++++++++++
  include/uapi/linux/vfio.h           |  11 +++
  include/uapi/linux/vfio_zdev.h      |  78 ++++++++++++++++++++
  10 files changed, 308 insertions(+), 1 deletion(-)

create mode 100644 drivers/vfio/pci/vfio_pci_zdev.c create mode 100644 include/uapi/linux/vfio_zdev.h

#13 lore raw
In reply to Matthew Rosato
Quoted 12 lines · On Wed, 7 Oct 2020 14:56:19 -0400 Matthew Rosato <[email protected]>…

This patchset provides a means by which hardware information about the underlying PCI device can be passed up to userspace (ie, QEMU) so that this hardware information can be used rather than previously hard-coded assumptions. The VFIO_DEVICE_GET_INFO ioctl is extended to allow capability chains and zPCI devices provide the hardware information via capabilities.

A form of these patches saw some rounds last year but has been back- tabled for a while. The original work for this feature was done by Pierre Morel. I'd like to refresh the discussion on this and get this finished up so that we can move forward with better-supporting additional types of PCI-attached devices.

Quoted 6 lines

This feature is toggled via the CONFIG_VFIO_PCI_ZDEV configuration entry.

Changes since v2:
- Added ACKs (thanks!)
- Patch 3+4: Re-write to use VFIO_DEVICE_GET_INFO capabilities rather than a vfio device region.

Looks good to me, I'll let Connie and others double check and throw in their reviews, but I'll plan to include this for v5.10. Thanks,

Alex

Quoted 21 lines

Matthew Rosato (5):
s390/pci: stash version in the zpci_dev
s390/pci: track whether util_str is valid in the zpci_dev
vfio: Introduce capability definitions for VFIO_DEVICE_GET_INFO vfio-pci/zdev: Add zPCI capabilities to VFIO_DEVICE_GET_INFO
MAINTAINERS: Add entry for s390 vfio-pci

 MAINTAINERS                         |   8 ++
 arch/s390/include/asm/pci.h         |   4 +-
 arch/s390/pci/pci_clp.c             |   2 +
 drivers/vfio/pci/Kconfig            |  13 ++++
 drivers/vfio/pci/Makefile           |   1 +
 drivers/vfio/pci/vfio_pci.c         |  37 ++++++++++
 drivers/vfio/pci/vfio_pci_private.h |  12 +++
 drivers/vfio/pci/vfio_pci_zdev.c    | 143 ++++++++++++++++++++++++++++++++++++
 include/uapi/linux/vfio.h           |  11 +++
 include/uapi/linux/vfio_zdev.h      |  78 ++++++++++++++++++++
 10 files changed, 308 insertions(+), 1 deletion(-)
 create mode 100644 drivers/vfio/pci/vfio_pci_zdev.c
 create mode 100644 include/uapi/linux/vfio_zdev.h