Skip to contentSkip to conversation

Enable PF-VF linking with pdev->no_vf_scan (s390)

PATCH New Niklas Schnelle 1 participant 3 messages May 7, 2020

Subsystems (3)
lore
Share & tools
lore

Niklas Schnelle

author maintainer
#0 lore raw

cover Enable PF-VF linking with pdev->no_vf_scan (s390)

Hello Kernel Hackers,

the following series enables PF-VF linking for architectures using the pdev->no_vf_scan flag (currently just s390). This includes kernel internal linking with pdev->physfn as well as creation of the relevant sysfs links. The former are required for example by libvirt to manage VFs.

The series consists of two patches against the featuress branch of git.kernel.org/pub/scm/linux/kernel/git/s390/linux.git

- PCI/IOV: Introduce pci_iov_sysfs_link() function

This patch factors the sysfs link creation out of pci_iov_add_virtfn() and into a new pci_iov_sysfs_link() function. On its own it also applies cleanly on v5.7-rc4.

- s390/pci: create links between PFs and VFs

This patch makes use of the pci_iov_sysfs_link() function introduced in the first patch to create the sysfs PF-VF links. It exploits recent work for multi-function support on s390 that gives us the real devfnto identify the parent PF of a given VF.
Apart from these s390 specific support contained within arch/s390/ it also removes the fencing off of pci_iov_remove_virtfn() by the pdev->no_vf_scan flag making use of the common code path for clean VF removal. While this is common code s390 is currently the only case where pdev->no_vf_scan is true I can of course split this into a separate patch if requested but wanted to keep this together for the discussion. This patch will go via the s390 tree once the first patch landed.

Best regards and your feedback is welcome,
Niklas Schnelle

Changes since the RFC:
- Addressed commit message feedback (thanks Bjorn Helgaas)
- Addressed style feedback (thanks Bjorn Helgaas)
- Add Acked-by
- Fixed struct field name collision for reserved in second patch. (Was added as style fix).

Niklas Schnelle (2):
PCI/IOV: Introduce pci_iov_sysfs_link() function s390/pci: create links between PFs and VFs

 arch/s390/include/asm/pci.h     |  3 +-
 arch/s390/include/asm/pci_clp.h |  3 +-
 arch/s390/pci/pci_bus.c         | 69 ++++++++++++++++++++++++++++++++-
 arch/s390/pci/pci_clp.c         |  1 +
 drivers/pci/iov.c               | 39 ++++++++++++-------
 include/linux/pci.h             |  8 ++++
 6 files changed, 106 insertions(+), 17 deletions(-)
Signature · 1 line
2.17.1
author maintainer
#1 lore raw

PATCH 1/2 PCI/IOV: Introduce pci_iov_sysfs_link() function

Currently pci_iov_add_virtfn() scans the SR-IOV BARs, adds the VF to the bus and also creates the sysfs links between the newly added VF and its parent PF.

With pdev->no_vf_scan fencing off the entire pci_iov_add_virtfn() call s390 as the sole pdev->no_vf_scan user thus ends up missing these sysfs links which are required for example by QEMU/libvirt.

Instead of duplicating the code refactor pci_iov_add_virtfn() to make sysfs link creation callable separately.

---
 drivers/pci/iov.c   | 36 +++++++++++++++++++++++++-----------
 include/linux/pci.h |  8 ++++++++
 2 files changed, 33 insertions(+), 11 deletions(-)
drivers/pci/iov.c +25 -11
index 4d1f392b05f9..ee6fbe688498 100644
@@ -133,12 +133,35 @@ static void pci_read_vf_config_common(struct pci_dev *virtfn)
 			     &physfn->sriov->subsystem_device);
 }
 
+int pci_iov_sysfs_link(struct pci_dev *dev,
+		struct pci_dev *virtfn, int id)
+{
+	char buf[VIRTFN_ID_LEN];
+	int rc;
+
+	sprintf(buf, "virtfn%u", id);
+	rc = sysfs_create_link(&dev->dev.kobj, &virtfn->dev.kobj, buf);
+	if (rc)
+		goto failed;
+	rc = sysfs_create_link(&virtfn->dev.kobj, &dev->dev.kobj, "physfn");
+	if (rc)
+		goto failed1;
+
+	kobject_uevent(&virtfn->dev.kobj, KOBJ_CHANGE);
+
+	return 0;
+
+failed1:
+	sysfs_remove_link(&dev->dev.kobj, buf);
+failed:
+	return rc;
+}
+
 int pci_iov_add_virtfn(struct pci_dev *dev, int id)
 {
 	int i;
 	int rc = -ENOMEM;
 	u64 size;
-	char buf[VIRTFN_ID_LEN];
 	struct pci_dev *virtfn;
 	struct resource *res;
 	struct pci_sriov *iov = dev->sriov;
@@ -182,23 +205,14 @@ int pci_iov_add_virtfn(struct pci_dev *dev, int id)
 	}
 
 	pci_device_add(virtfn, virtfn->bus);
-
-	sprintf(buf, "virtfn%u", id);
-	rc = sysfs_create_link(&dev->dev.kobj, &virtfn->dev.kobj, buf);
+	rc = pci_iov_sysfs_link(dev, virtfn, id);
 	if (rc)
 		goto failed1;
-	rc = sysfs_create_link(&virtfn->dev.kobj, &dev->dev.kobj, "physfn");
-	if (rc)
-		goto failed2;
-
-	kobject_uevent(&virtfn->dev.kobj, KOBJ_CHANGE);
 
 	pci_bus_add_device(virtfn);
 
 	return 0;
 
-failed2:
-	sysfs_remove_link(&dev->dev.kobj, buf);
 failed1:
 	pci_stop_and_remove_bus_device(virtfn);
 	pci_dev_put(dev);
include/linux/pci.h +8 -0
index 83ce1cdf5676..93a063a7d7f9 100644
@@ -2048,6 +2048,8 @@ int pci_iov_virtfn_devfn(struct pci_dev *dev, int id);
 
 int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn);
 void pci_disable_sriov(struct pci_dev *dev);
+
+int pci_iov_sysfs_link(struct pci_dev *dev, struct pci_dev *virtfn, int id);
 int pci_iov_add_virtfn(struct pci_dev *dev, int id);
 void pci_iov_remove_virtfn(struct pci_dev *dev, int id);
 int pci_num_vf(struct pci_dev *dev);
@@ -2073,6 +2075,12 @@ static inline int pci_iov_virtfn_devfn(struct pci_dev *dev, int id)
 }
 static inline int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn)
 { return -ENODEV; }
+
+static inline int pci_iov_sysfs_link(struct pci_dev *dev,
+				     struct pci_dev *virtfn, int id)
+{
+	return -ENODEV;
+}
 static inline int pci_iov_add_virtfn(struct pci_dev *dev, int id)
 {
 	return -ENOSYS;
Signature · 1 line
2.17.1
author maintainer
#2 lore raw

PATCH 2/2 s390/pci: create links between PFs and VFs

In reply to Niklas Schnelle

On s390 PCI Virtual Functions (VFs) are scanned by firmware and are made available to Linux via the hot-plug interface. As such the common code path of doing the scan directly using the parent Physical Function (PF) is not used and fenced off with the no_vf_scan attribute.

Even if the partition created the VFs itself e.g. using the sriov_numvfs attribute of a PF, the PF/VF links thus need to be established after the fact. To do this when a VF is plugged we scan through all functions on the same zbus and test whether they are the parent PF in which case we establish the necessary links.

With these links established there is now no more need to fence off pci_iov_remove_virtfn() for pdev->no_vf_scan as the common code now works fine.

---
 arch/s390/include/asm/pci.h     |  3 +-
 arch/s390/include/asm/pci_clp.h |  3 +-
 arch/s390/pci/pci_bus.c         | 69 ++++++++++++++++++++++++++++++++-
 arch/s390/pci/pci_clp.c         |  1 +
 drivers/pci/iov.c               |  3 --
 5 files changed, 73 insertions(+), 6 deletions(-)
arch/s390/include/asm/pci.h +2 -1
index c1558cf071b8..99b92c3e46b0 100644
@@ -131,7 +131,8 @@ struct zpci_dev {
 	u8		port;
 	u8		rid_available	: 1;
 	u8		has_hp_slot	: 1;
-	u8		reserved	: 6;
+	u8		is_physfn	: 1;
+	u8		reserved	: 5;
 	unsigned int	devfn;		/* DEVFN part of the RID*/
 
 	struct mutex lock;
arch/s390/include/asm/pci_clp.h +2 -1
index 896ee41e23e3..eb51272dd2cc 100644
@@ -95,7 +95,8 @@ struct clp_rsp_query_pci {
 	u16 vfn;			/* virtual fn number */
 	u16			:  3;
 	u16 rid_avail		:  1;
-	u16			:  2;
+	u16 is_physfn		:  1;
+	u16 reserved1		:  1;
 	u16 mio_addr_avail	:  1;
 	u16 util_str_avail	:  1;	/* utility string available? */
 	u16 pfgid		:  8;	/* pci function group id */
arch/s390/pci/pci_bus.c +68 -1

85 lines not on this page · raw message

arch/s390/pci/pci_clp.c +1 -0
index 9b318824a134..d7bd3c287cf7 100644
@@ -159,6 +159,7 @@ static int clp_store_query_pci_fn(struct zpci_dev *zdev,
 	zdev->uid = response->uid;
 	zdev->fmb_length = sizeof(u32) * response->fmb_len;
 	zdev->rid_available = response->rid_avail;
+	zdev->is_physfn = response->is_physfn;
 	if (!s390_pci_no_rid && zdev->rid_available)
 		zdev->devfn = response->rid & ZPCI_RID_MASK_DEVFN;
 
drivers/pci/iov.c +0 -3
index ee6fbe688498..b37e08c4f9d1 100644
@@ -571,9 +571,6 @@ static void sriov_del_vfs(struct pci_dev *dev)
 	struct pci_sriov *iov = dev->sriov;
 	int i;
 
-	if (dev->no_vf_scan)
-		return;
-
 	for (i = 0; i < iov->num_VFs; i++)
 		pci_iov_remove_virtfn(dev, i);
 }
Signature · 1 line
2.17.1