Skip to contentSkip to conversation

UIO support for dfl devices

PATCH Superseded Xu Yilun 3 participants 6 messages Feb 1, 2021 – Feb 2, 2021

Subsystems (4)

v10 of 12 Superseded Latest is v12 →

lore
Share & tools
lore

Xu Yilun

author maintainer
#0 lore raw

v10 cover UIO support for dfl devices

This patchset supports some dfl device drivers written in userspace.

In the patchset v1, the "driver_override" interface should be used to bind the DFL UIO driver to DFL devices. But there is concern that the "driver_override" interface is not OK itself.

In v2, we use a new matching algorithem. The "driver_override" interface is abandoned, the DFL UIO driver matches any DFL device which could not be handled by other DFL drivers. So the DFL UIO driver could be used for new DFL devices which are not supported by kernel. The concern is the UIO may not be suitable as a default/generic driver for all dfl features, such as features with multiple interrupts.

In v4, we specify each matching device in the id_table of the UIO driver, just the same as other dfl drivers do. Now the UIO driver supports Ether Group feature. To support more DFL features, their feature ids should be added to the driver's id_table.

Before v9, we create a "uio_pdrv_genirq" platform device using DFL devices' resources. Then we leverage the uio_pdrv_genirq driver for UIO support. It is suggested that we implement a driver in drivers/uio that directly calls UIO framework APIs. So we implement the uio_dfl driver in v9. The driver now only binds the ether group feature, which has no irq. So the irq support is not implemented yet.

Main changes from v1:
- switch to the new matching algorithem. It matches DFL devices which could not be handled by other DFL drivers.
- refacor the code about device resources filling.
- add the documentation.

Main changes from v2:
- split the match ops changes in dfl.c to an independent patch.
- move the declarations needed for dfl-uio-pdev from include/linux/dfl.h to driver/fpga/dfl.h
- some minor fixes.

Main changes from v3:
- switch to specifying each matching device in the driver's id_table.
- refactor the irq handling code.

Main changes from v4:
- refactor the irq handling code.

Main changes from v5:
- fix the res[] zero initialization issue.
- improve the return code for probe().
- some doc improvement.

Main changes from v6:
- use platform_device_register_resndata() for pdev creation.

Main changes from v7:
- some doc fixes.

Main changes from v8:
- switch to add a uio driver in drivers/uio

Main changes from v9:
- add this source file in MAINTAINERS
- improve the Kconfig, add more descriptive Kconfig header, add detailed path for opae uio example in Kconfig.

Xu Yilun (2):
uio: uio_dfl: add userspace i/o driver for DFL bus
Documentation: fpga: dfl: Add description for DFL UIO support

 Documentation/fpga/dfl.rst | 23 ++++++++++++++++
 MAINTAINERS                |  1 +
 drivers/uio/Kconfig        | 17 ++++++++++++
 drivers/uio/Makefile       |  1 +
 drivers/uio/uio_dfl.c      | 66 ++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 108 insertions(+)
 create mode 100644 drivers/uio/uio_dfl.c
Signature · 1 line
2.7.4
author maintainer
#1 lore raw

PATCH v10 1/2 uio: uio_dfl: add userspace i/o driver for DFL bus

This patch supports the DFL drivers be written in userspace. This is realized by exposing the userspace I/O device interfaces.

The driver now only binds the ether group feature, which has no irq. So the irq support is not implemented yet.

---
v9: switch to add a uio driver in drivers/uio
v10: add the source file in MAINTAINERS
more descriptive Kconfig header
add detailed path for opae uio example

---
 MAINTAINERS           |  1 +
 drivers/uio/Kconfig   | 17 +++++++++++++
 drivers/uio/Makefile  |  1 +
 drivers/uio/uio_dfl.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 85 insertions(+)
 create mode 100644 drivers/uio/uio_dfl.c
MAINTAINERS +1 -0
index 147d1d9..4d01a21 100644
@@ -6943,6 +6943,7 @@ S:	Maintained
 F:	Documentation/ABI/testing/sysfs-bus-dfl*
 F:	Documentation/fpga/dfl.rst
 F:	drivers/fpga/dfl*
+F:	drivers/uio/uio_dfl.c
 F:	include/linux/dfl.h
 F:	include/uapi/linux/fpga-dfl.h
 
drivers/uio/Kconfig +17 -0
index 202ee81..5531f3a 100644
@@ -165,4 +165,21 @@ config UIO_HV_GENERIC
 	  to network and storage devices from userspace.
 
 	  If you compile this as a module, it will be called uio_hv_generic.
+
+config UIO_DFL
+	tristate "Generic driver for DFL (Device Feature List) bus"
+	depends on FPGA_DFL
+	help
+	  Generic DFL (Device Feature List) driver for Userspace I/O devices.
+	  It is useful to provide direct access to DFL devices from userspace.
+	  A sample userspace application using this driver is available for
+	  download in a git repository:
+
+	    git clone https://github.com/OPAE/opae-sdk.git
+
+	  It could be found at:
+
+	    opae-sdk/tools/libopaeuio/
+
+	  If you compile this as a module, it will be called uio_dfl.
 endif
drivers/uio/Makefile +1 -0
index c285dd2..f2f416a1 100644
@@ -11,3 +11,4 @@ obj-$(CONFIG_UIO_PRUSS)         += uio_pruss.o
 obj-$(CONFIG_UIO_MF624)         += uio_mf624.o
 obj-$(CONFIG_UIO_FSL_ELBC_GPCM)	+= uio_fsl_elbc_gpcm.o
 obj-$(CONFIG_UIO_HV_GENERIC)	+= uio_hv_generic.o
+obj-$(CONFIG_UIO_DFL)	+= uio_dfl.o
drivers/uio/uio_dfl.c new file +66 -0

66 lines not on this page · raw message

Signature · 1 line
2.7.4
reviewer Reviewed-by on v10 1/2
#2 lore raw

Re: [PATCH v10 1/2] uio: uio_dfl: add userspace i/o driver for DFL bus

Quoted 45 lines · Xu Yilun

This patch supports the DFL drivers be written in userspace. This is realized by exposing the userspace I/O device interfaces.

The driver now only binds the ether group feature, which has no irq. So the irq support is not implemented yet.

---
v9: switch to add a uio driver in drivers/uio
v10: add the source file in MAINTAINERS
more descriptive Kconfig header
add detailed path for opae uio example

---
 MAINTAINERS           |  1 +
 drivers/uio/Kconfig   | 17 +++++++++++++
 drivers/uio/Makefile  |  1 +
 drivers/uio/uio_dfl.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 85 insertions(+)
 create mode 100644 drivers/uio/uio_dfl.c
MAINTAINERS +1 -0
index 147d1d9..4d01a21 100644
@@ -6943,6 +6943,7 @@ S:	Maintained
 F:	Documentation/ABI/testing/sysfs-bus-dfl*
 F:	Documentation/fpga/dfl.rst
 F:	drivers/fpga/dfl*
+F:	drivers/uio/uio_dfl.c
 F:	include/linux/dfl.h
 F:	include/uapi/linux/fpga-dfl.h
 
drivers/uio/Kconfig +5 -0
index 202ee81..5531f3a 100644
@@ -165,4 +165,21 @@ config UIO_HV_GENERIC
 	  to network and storage devices from userspace.
 
 	  If you compile this as a module, it will be called uio_hv_generic.
+
+config UIO_DFL
+	tristate "Generic driver for DFL (Device Feature List) bus"
+	depends on FPGA_DFL
+	help

drivers/uio/Kconfig config UIO_HV_GENERIC · line 182

Quoted 10 lines
drivers/uio/Kconfig +10 -0
+	  Generic DFL (Device Feature List) driver for Userspace I/O devices.
+	  It is useful to provide direct access to DFL devices from userspace.
+	  A sample userspace application using this driver is available for
+	  download in a git repository:
+
+	    git clone https://github.com/OPAE/opae-sdk.git
+
+	  It could be found at:
+
+	    opae-sdk/tools/libopaeuio/

Yes, it is there.  Thanks!

Quoted 84 lines
drivers/uio/Kconfig +2 -0
+
+	  If you compile this as a module, it will be called uio_dfl.
 endif
drivers/uio/Makefile +1 -0
index c285dd2..f2f416a1 100644
@@ -11,3 +11,4 @@ obj-$(CONFIG_UIO_PRUSS)         += uio_pruss.o
 obj-$(CONFIG_UIO_MF624)         += uio_mf624.o
 obj-$(CONFIG_UIO_FSL_ELBC_GPCM)	+= uio_fsl_elbc_gpcm.o
 obj-$(CONFIG_UIO_HV_GENERIC)	+= uio_hv_generic.o
+obj-$(CONFIG_UIO_DFL)	+= uio_dfl.o
drivers/uio/uio_dfl.c new file +66 -0
new file mode 100644
index 0000000..89c0fc7
@@ -0,0 +1,66 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Generic DFL driver for Userspace I/O devicess
+ *
+ * Copyright (C) 2021 Intel Corporation, Inc.
+ */
+#include <linux/dfl.h>
+#include <linux/errno.h>
+#include <linux/module.h>
+#include <linux/uio_driver.h>
+
+#define DRIVER_NAME "uio_dfl"
+
+static int uio_dfl_probe(struct dfl_device *ddev)
+{
+	struct resource *r = &ddev->mmio_res;
+	struct device *dev = &ddev->dev;
+	struct uio_info *uioinfo;
+	struct uio_mem *uiomem;
+	int ret;
+
+	uioinfo = devm_kzalloc(dev, sizeof(struct uio_info), GFP_KERNEL);
+	if (!uioinfo)
+		return -ENOMEM;
+
+	uioinfo->name = DRIVER_NAME;
+	uioinfo->version = "0";
+
+	uiomem = &uioinfo->mem[0];
+	uiomem->memtype = UIO_MEM_PHYS;
+	uiomem->addr = r->start & PAGE_MASK;
+	uiomem->offs = r->start & ~PAGE_MASK;
+	uiomem->size = (uiomem->offs + resource_size(r)
+			+ PAGE_SIZE - 1) & PAGE_MASK;
+	uiomem->name = r->name;
+
+	/* Irq is yet to be supported */
+	uioinfo->irq = UIO_IRQ_NONE;
+
+	ret = devm_uio_register_device(dev, uioinfo);
+	if (ret)
+		dev_err(dev, "unable to register uio device\n");
+
+	return ret;
+}
+
+#define FME_FEATURE_ID_ETH_GROUP	0x10
+
+static const struct dfl_device_id uio_dfl_ids[] = {
+	{ FME_ID, FME_FEATURE_ID_ETH_GROUP },
+	{ }
+};
+MODULE_DEVICE_TABLE(dfl, uio_dfl_ids);
+
+static struct dfl_driver uio_dfl_driver = {
+	.drv = {
+		.name = DRIVER_NAME,
+	},
+	.id_table	= uio_dfl_ids,
+	.probe		= uio_dfl_probe,
+};
+module_dfl_driver(uio_dfl_driver);
+
+MODULE_DESCRIPTION("Generic DFL driver for Userspace I/O devices");
+MODULE_AUTHOR("Intel Corporation");
+MODULE_LICENSE("GPL v2");
author maintainer
#3 lore raw

PATCH v10 2/2 Documentation: fpga: dfl: Add description for DFL UIO support

In reply to Xu Yilun

This patch adds description for UIO support for dfl devices on DFL bus.

---
v2: no doc in v1, add it for v2.
v3: some documentation fixes.
v4: documentation change since the driver matching is changed.
v5: no change.
v6: improve the title of the userspace driver support section.
some word improvement.
v7: rebased to next-20210119
v8: some doc fixes.
v9: some doc change since we switch to the driver in drivers/uio.
v10: no change.

---
 Documentation/fpga/dfl.rst | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
Documentation/fpga/dfl.rst +23 -0
index c41ac76..e35cf87 100644
@@ -7,6 +7,7 @@ Authors:
 - Enno Luebbers <[email protected]>
 - Xiao Guangrong <[email protected]>
 - Wu Hao <[email protected]>
+- Xu Yilun <[email protected]>
 
 The Device Feature List (DFL) FPGA framework (and drivers according to
 this framework) hides the very details of low layer hardwares and provides
@@ -530,6 +531,28 @@ Being able to specify more than one DFL per BAR has been considered, but it
 was determined the use case did not provide value.  Specifying a single DFL
 per BAR simplifies the implementation and allows for extra error checking.
 
+
+Userspace driver support for DFL devices
+========================================
+The purpose of an FPGA is to be reprogrammed with newly developed hardware
+components. New hardware can instantiate a new private feature in the DFL, and
+then present a DFL device in the system. In some cases users may need a
+userspace driver for the DFL device:
+
+* Users may need to run some diagnostic test for their hardware.
+* Users may prototype the kernel driver in user space.
+* Some hardware is designed for specific purposes and does not fit into one of
+  the standard kernel subsystems.
+
+This requires direct access to MMIO space and interrupt handling from
+userspace. The uio_dfl module exposes the UIO device interfaces for this
+purpose.
+
+UIO_DFL should be selected to enable the uio_dfl module driver. To support a
+new DFL feature via UIO direct access, its feature id should be added to the
+driver's id_table.
+
+
 Open discussion
 ===============
 FME driver exports one ioctl (DFL_FPGA_FME_PORT_PR) for partial reconfiguration
Signature · 1 line
2.7.4
#4 lore raw

RE: [PATCH v10 2/2] Documentation: fpga: dfl: Add description for DFL UIO support

Quoted 48 lines

Subject: [PATCH v10 2/2] Documentation: fpga: dfl: Add description for DFL UIO support

This patch adds description for UIO support for dfl devices on DFL bus.

---
v2: no doc in v1, add it for v2.
v3: some documentation fixes.
v4: documentation change since the driver matching is changed.
v5: no change.
v6: improve the title of the userspace driver support section.
some word improvement.
v7: rebased to next-20210119
v8: some doc fixes.
v9: some doc change since we switch to the driver in drivers/uio.
v10: no change.

---
 Documentation/fpga/dfl.rst | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
Documentation/fpga/dfl.rst +1 -0
index c41ac76..e35cf87 100644
@@ -7,6 +7,7 @@ Authors:
 - Enno Luebbers <[email protected]>
 - Xiao Guangrong <[email protected]>
 - Wu Hao <[email protected]>
+- Xu Yilun <[email protected]>
 
 The Device Feature List (DFL) FPGA framework (and drivers according to
 this framework) hides the very details of low layer hardwares and provides
@@ -530,6 +531,28 @@ Being able to specify more than one DFL per BAR has

been considered, but it
was determined the use case did not provide value. Specifying a single DFL per BAR simplifies the implementation and allows for extra error checking.

Documentation/fpga/dfl.rst +4 -0
+
+Userspace driver support for DFL devices
+========================================
+The purpose of an FPGA is to be reprogrammed with newly developed

hardware
+components. New hardware can instantiate a new private feature in the DFL, and

Documentation/fpga/dfl.rst +1 -0
+then present a DFL device in the system. In some cases users may need a

Documentation/fpga/dfl.rst Being able to specify more than one DFL per BAR has been considered, but it · line 549

Quoted 10 lines
Documentation/fpga/dfl.rst +10 -0
+userspace driver for the DFL device:
+
+* Users may need to run some diagnostic test for their hardware.
+* Users may prototype the kernel driver in user space.
+* Some hardware is designed for specific purposes and does not fit into one of
+  the standard kernel subsystems.
+
+This requires direct access to MMIO space and interrupt handling from
+userspace. The uio_dfl module exposes the UIO device interfaces for this
+purpose.

Current uio_dfl doesn't have interrupt handling support, right? I guess we need to make sure no confusion on the description here. other place looks good to me.

Hao

Quoted 12 lines
Documentation/fpga/dfl.rst +6 -0
+
+UIO_DFL should be selected to enable the uio_dfl module driver. To support a
+new DFL feature via UIO direct access, its feature id should be added to the
+driver's id_table.
+
+
 Open discussion
 ===============
 FME driver exports one ioctl (DFL_FPGA_FME_PORT_PR) for partial

reconfiguration
--
2.7.4

author maintainer
#5 lore raw

Re: [PATCH v10 2/2] Documentation: fpga: dfl: Add description for DFL UIO support

Quoted 53 lines · Hao
Quoted 53 lines

Subject: [PATCH v10 2/2] Documentation: fpga: dfl: Add description for DFL UIO support

This patch adds description for UIO support for dfl devices on DFL bus.

---
v2: no doc in v1, add it for v2.
v3: some documentation fixes.
v4: documentation change since the driver matching is changed.
v5: no change.
v6: improve the title of the userspace driver support section.
some word improvement.
v7: rebased to next-20210119
v8: some doc fixes.
v9: some doc change since we switch to the driver in drivers/uio.
v10: no change.

---
 Documentation/fpga/dfl.rst | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
Documentation/fpga/dfl.rst +1 -0
index c41ac76..e35cf87 100644
@@ -7,6 +7,7 @@ Authors:
 - Enno Luebbers <[email protected]>
 - Xiao Guangrong <[email protected]>
 - Wu Hao <[email protected]>
+- Xu Yilun <[email protected]>
 
 The Device Feature List (DFL) FPGA framework (and drivers according to
 this framework) hides the very details of low layer hardwares and provides
@@ -530,6 +531,28 @@ Being able to specify more than one DFL per BAR has

been considered, but it
was determined the use case did not provide value. Specifying a single DFL per BAR simplifies the implementation and allows for extra error checking.

Documentation/fpga/dfl.rst +4 -0
+
+Userspace driver support for DFL devices
+========================================
+The purpose of an FPGA is to be reprogrammed with newly developed

hardware
+components. New hardware can instantiate a new private feature in the DFL, and

Documentation/fpga/dfl.rst +6 -0
+then present a DFL device in the system. In some cases users may need a
+userspace driver for the DFL device:
+
+* Users may need to run some diagnostic test for their hardware.
+* Users may prototype the kernel driver in user space.
+* Some hardware is designed for specific purposes and does not fit into one of
Quoted 8 lines

Documentation/fpga/dfl.rst Being able to specify more than one DFL per BAR has been considered, but it · line 549

Quoted 5 lines
Documentation/fpga/dfl.rst +5 -0
+  the standard kernel subsystems.
+
+This requires direct access to MMIO space and interrupt handling from
+userspace. The uio_dfl module exposes the UIO device interfaces for this
+purpose.

Current uio_dfl doesn't have interrupt handling support, right? I guess we need to make sure no confusion on the description here. other place looks good to me.

Will fix it.

Thanks,
Yilun