Skip to contentSkip to conversation

virtio: vdpa: new SolidNET DPU driver

PATCH In review Alvaro Karsz 2 participants 6 messages Jan 10, 2023

Subsystems (3)
lore
Share & tools
lore

Alvaro Karsz

author reviewer
#0 lore raw

v9 cover virtio: vdpa: new SolidNET DPU driver

This series adds a vDPA driver for SolidNET DPU.

Patch 1 adds SolidRun vendor ID to pci_ids.
Patch 2 adds a PCI quirk needed by the DPU.
Patch 3 has the driver source code.

Patch 1 is prerequisite for both patch 2 and patch 3.

v2:
- Semantics fixes in commit log - Patch 1.
- Move the vendor ID to the right place, sorted by vendor ID - Patch 1.
- Update patch subject to be more meaningful and similar to previous quirks - Patch 2.
- Update the commit log to describe better what the patch does - Patch 2.
- Auto detect the BAR used for communication - Patch 3.
- When waiting for the DPU to write a config, wait for 5secs before giving up on the device - Patch 3.
- Return EOPNOTSUPP error code in vDPA set_vq_state callback if the vq state is not the same as the initial one - Patch 3.
- Implement a vDPA reset callback - Patch 3.
- Wait for an ACK when sending a message to the DPU - Patch 3.
- Add endianness comments on 64bit read/write functions - Patch 3.
- Remove the get_iova_range and free vDPA callbacks - Patch 3.
- Usage of managed device functions to ioremap a region - Patch 3.
- Call pci_set_drvdata and pci_set_master before vdpa_register_device - Patch 3.
- Create DMA isolation between the vDPA devices by using the chip SR-IOV feature.
Every vDPA device gets a PCIe VF with its own DMA device - Patch 3.

v3:
- Validate vDPA config length while receiving the DPU's config, not while trying to write the vDPA config to the DPU - Patch 3.
- Request IRQs when vDPA status is set to VIRTIO_CONFIG_S_DRIVER_OK - Patch 3.
- Remove snet_reset_dev() from the PCI remove function for a VF - Patch 3.

v4:
- Get SolidRun vendor ID from pci_ids - Patch 3.

v5:
- Remove "select HWMON" from Kconfig.
Usage of "depends on HWMON || HWMON=n" instead and usage of IS_ENABLED(CONFIG_HWMON) when calling to snet hwmon functions. snet_hwmon.c is compiled only if CONFIG_HWMON is defined - Patch 3.
- Remove the #include <linux/hwmon-sysfs.h> from snet_hwmon.c - Patch 3.
- Remove the unnecessary (long) casting from snet_hwmon_read_reg - Patch 3.
- Remove the "_hwmon" ending from the hwmon name - Patch 3.
- Usage of IS_ERR macro to check if devm_hwmon_device_register_with_info failed - Patch 3.
- Replace schedule() with usleep_range() in the "hot loop" in psnet_detect_bar - Patch 3.
- Remove the logging of memory allocation failures - Patch 3.
- Add parenthesis to arguments in SNET_* macros - Patch 3.
- Prefer sizeof(*variable) instead of sizeof(struct x) when allocating memory - Patch 3.

v6:
- SNET_WRN -> SNET_WARN - Patch 3.

v7:
- Explain the dependency of SNET_VDPA on HWMON in Kconfig - Patch 3.
- Fix snprintf size argument in psnet_open_pf_bar and snet_open_vf_bar - Patch 3.
- Fix kernel warning in snet_vdpa_remove_vf.
Call pci_free_irq_vectors after vdpa_unregister_device, otherwise we'll get "remove_proc_entry: removing non-empty directory 'irq/..', leaking at least '..'" warnings - Patch 3.
- Remove the psnet_create_hwmon function empty definition if HWMON is not enabled, otherwise, we'll get "warning: no previous prototype for 'psnet_create_hwmon'" when compiling with W=1.
This was reported by kernel test robot <[email protected]> - Patch 3.

v8:
- Fix the series versioning.
I updated the versions of every patch separately, which seems to be wrong.

v9:
- Indent PCI id in pci_ids.h with tabs - Patch 1.
- Wrap patch comment log to fill 75 columns - Patch 1 + 2.

Alvaro Karsz (3):
PCI: Add SolidRun vendor ID
PCI: Avoid FLR for SolidRun SNET DPU rev 1
virtio: vdpa: new SolidNET DPU driver.

 MAINTAINERS                        |    4 +
 drivers/pci/quirks.c               |    8 +
 drivers/vdpa/Kconfig               |   18 +
 drivers/vdpa/Makefile              |    1 +
 drivers/vdpa/solidrun/Makefile     |    6 +
 drivers/vdpa/solidrun/snet_hwmon.c |  188 +++++
 drivers/vdpa/solidrun/snet_main.c  | 1110 ++++++++++++++++++++++++++++
 drivers/vdpa/solidrun/snet_vdpa.h  |  194 +++++
 include/linux/pci_ids.h            |    2 +
 9 files changed, 1531 insertions(+)
 create mode 100644 drivers/vdpa/solidrun/Makefile
 create mode 100644 drivers/vdpa/solidrun/snet_hwmon.c
 create mode 100644 drivers/vdpa/solidrun/snet_main.c
 create mode 100644 drivers/vdpa/solidrun/snet_vdpa.h
Signature · 6 lines
2.32.0

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
author reviewer
#1 lore raw

PATCH v9 1/3 PCI: Add SolidRun vendor ID

Add SolidRun vendor ID to pci_ids.h

The vendor ID is used in 2 different source files, the SNET vDPA driver and PCI quirks.

---
 include/linux/pci_ids.h | 2 ++
 1 file changed, 2 insertions(+)
include/linux/pci_ids.h +2 -0
index b362d90eb9b..6716e6371a1 100644
@@ -3092,6 +3092,8 @@
 
 #define PCI_VENDOR_ID_3COM_2		0xa727
 
+#define PCI_VENDOR_ID_SOLIDRUN		0xd063
+
 #define PCI_VENDOR_ID_DIGIUM		0xd161
 #define PCI_DEVICE_ID_DIGIUM_HFC4S	0xb410
 
Signature · 6 lines
2.32.0

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
author reviewer
#2 lore raw

PATCH v9 2/3 PCI: Avoid FLR for SolidRun SNET DPU rev 1

In reply to Alvaro Karsz

This patch fixes a FLR bug on the SNET DPU rev 1 by setting the PCI_DEV_FLAGS_NO_FLR_RESET flag.

As there is a quirk to avoid FLR (quirk_no_flr), I added a new quirk to check the rev ID before calling to quirk_no_flr.

Without this patch, a SNET DPU rev 1 may hang when FLR is applied.

---
 drivers/pci/quirks.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/pci/quirks.c +8 -0
index 285acc4aacc..809d03272c2 100644
@@ -5343,6 +5343,14 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x149c, quirk_no_flr);
 DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1502, quirk_no_flr);
 DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1503, quirk_no_flr);
 
+/* FLR may cause the SolidRun SNET DPU (rev 0x1) to hang */
+static void quirk_no_flr_snet(struct pci_dev *dev)
+{
+	if (dev->revision == 0x1)
+		quirk_no_flr(dev);
+}
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_SOLIDRUN, 0x1000, quirk_no_flr_snet);
+
 static void quirk_no_ext_tags(struct pci_dev *pdev)
 {
 	struct pci_host_bridge *bridge = pci_find_host_bridge(pdev->bus);
Signature · 6 lines
2.32.0

_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
author reviewer
#3 lore raw

PATCH v9 3/3 virtio: vdpa: new SolidNET DPU driver.

In reply to Alvaro Karsz

This commit includes:
1) The driver to manage the controlplane over vDPA bus.
2) A HW monitor device to read health values from the DPU.

---
 MAINTAINERS                        |    4 +
 drivers/vdpa/Kconfig               |   18 +
 drivers/vdpa/Makefile              |    1 +
 drivers/vdpa/solidrun/Makefile     |    6 +
 drivers/vdpa/solidrun/snet_hwmon.c |  188 +++++
 drivers/vdpa/solidrun/snet_main.c  | 1110 ++++++++++++++++++++++++++++
 drivers/vdpa/solidrun/snet_vdpa.h  |  194 +++++
 7 files changed, 1521 insertions(+)
 create mode 100644 drivers/vdpa/solidrun/Makefile
 create mode 100644 drivers/vdpa/solidrun/snet_hwmon.c
 create mode 100644 drivers/vdpa/solidrun/snet_main.c
 create mode 100644 drivers/vdpa/solidrun/snet_vdpa.h
MAINTAINERS +4 -0
index 7f86d02cb42..78e181e5cf5 100644
@@ -22024,6 +22024,10 @@ IFCVF VIRTIO DATA PATH ACCELERATOR
 R:	Zhu Lingshan <[email protected]>
 F:	drivers/vdpa/ifcvf/
 
+SNET DPU VIRTIO DATA PATH ACCELERATOR
+R:	Alvaro Karsz <[email protected]>
+F:	drivers/vdpa/solidrun/
+
 VIRTIO BALLOON
 M:	"Michael S. Tsirkin" <[email protected]>
 M:	David Hildenbrand <[email protected]>
drivers/vdpa/Kconfig +18 -0
index 50f45d03761..7396e92b485 100644
@@ -86,4 +86,22 @@ config ALIBABA_ENI_VDPA
 	  VDPA driver for Alibaba ENI (Elastic Network Interface) which is built upon
 	  virtio 0.9.5 specification.
 
+ config SNET_VDPA
+	tristate "SolidRun's vDPA driver for SolidNET"
+	depends on PCI_MSI && PCI_IOV && (HWMON || HWMON=n)
+
+	# This driver MAY create a HWMON device.
+	# Depending on (HWMON || HWMON=n) ensures that:
+	# If HWMON=n the driver can be compiled either as a module or built-in.
+	# If HWMON=y the driver can be compiled either as a module or built-in.
+	# If HWMON=m the driver is forced to be compiled as a module.
+	# By doing so, IS_ENABLED can be used instead of IS_REACHABLE
+
+	help
+	  vDPA driver for SolidNET DPU.
+	  With this driver, the VirtIO dataplane can be
+	  offloaded to a SolidNET DPU.
+	  This driver includes a HW monitor device that
+	  reads health values from the DPU.
+
 endif # VDPA
drivers/vdpa/Makefile +1 -0
index 15665563a7f..59396ff2a31 100644
@@ -6,3 +6,4 @@ obj-$(CONFIG_IFCVF)    += ifcvf/
 obj-$(CONFIG_MLX5_VDPA) += mlx5/
 obj-$(CONFIG_VP_VDPA)    += virtio_pci/
 obj-$(CONFIG_ALIBABA_ENI_VDPA) += alibaba/
+obj-$(CONFIG_SNET_VDPA) += solidrun/
drivers/vdpa/solidrun/Makefile new file +6 -0
new file mode 100644
index 00000000000..c0aa3415bf7
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_SNET_VDPA) += snet_vdpa.o
+snet_vdpa-$(CONFIG_SNET_VDPA) += snet_main.o
+ifdef CONFIG_HWMON
+snet_vdpa-$(CONFIG_SNET_VDPA) += snet_hwmon.o
+endif
drivers/vdpa/solidrun/snet_hwmon.c new file +188 -0

188 lines not on this page · raw message

drivers/vdpa/solidrun/snet_main.c new file +1110 -0

1,110 lines not on this page · raw message

drivers/vdpa/solidrun/snet_vdpa.h new file +194 -0

194 lines not on this page · raw message

Signature · 7 lines
2.32.0


_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization