Skip to contentSkip to conversation

FPGA DFL updates

PATCH In review Wu Hao 3 participants 18 messages Aug 12, 2019 – Aug 27, 2019

Subsystems (4)
lore
Share & tools
lore

Wu Hao

author
#0 lore raw

v5 cover FPGA DFL updates

Hi Greg,

This is v5 patchset which adds more features to FPGA DFL. Marjor changes against v4 are sysfs related code rework to address comments on v4.

Please help to take a look. Thanks!

Main changes from v4:
- convert code to use dev_groups for sysfs entries (#2, #3, #4, #6, #8).
- clean up for empty init function after remove sysfs add/remove (#1).
- introduce is_visible for sysfs groups (#3, #4, #6, #8).
- remove revision sysfs entries (#4, #6, #8).
- improve naming on shared functions (#5).
- reorganize sysfs entries for port and fme error reporting (#6, #8).

Main changes from v3:
- drop avx512 partail reconfiguration patch for now.
- split dfl_fpga_cdev_config_port to 2 functions *_release/assign_port

    (#1).
  - split __dfl_fpga_cdev_config_port_vf into 2 functions with locking
    added (#2).

- improve description in sysfs doc to avoid misunderstanding (#3).
- switch to boolean in sysfs entry store function (#3).
- remove dev_dbg in init/uinit callback function (#7, #9, #11).
- remove uinit callback which does does nothing (#8, #9)

Main changes from v2:
- update kernel version/date in sysfs doc (patch #4, #5, #8, #10, #11).
- add back Documentation patch (patch #12).

Main changes from v1:
- remove DRV/MODULE_VERSION modifications. (patch #1, #3, #4, #6)
- remove argsz from new ioctls. (patch #2)
- replace sysfs_create/remove_* with device_add/remove_* for sysfs entries.

    (patch #5, #8, #11)

Wu Hao (9):
fpga: dfl: make init callback optional
fpga: dfl: fme: convert platform_driver to use dev_groups
fpga: dfl: afu: convert platform_driver to use dev_groups
fpga: dfl: afu: add userclock sysfs interfaces.
fpga: dfl: afu: expose __afu_port_enable/disable function.
fpga: dfl: afu: add error reporting support.
fpga: dfl: afu: add STP (SignalTap) support
fpga: dfl: fme: add global error reporting support
Documentation: fpga: dfl: add descriptions for virtualization and new
interfaces.

 Documentation/ABI/testing/sysfs-platform-dfl-fme  |  62 ++++
 Documentation/ABI/testing/sysfs-platform-dfl-port |  53 ++++
 Documentation/fpga/dfl.rst                        | 105 +++++++
 drivers/fpga/Makefile                             |   3 +-
 drivers/fpga/dfl-afu-error.c                      | 230 ++++++++++++++
 drivers/fpga/dfl-afu-main.c                       | 230 +++++++++++---
 drivers/fpga/dfl-afu.h                            |   9 +
 drivers/fpga/dfl-fme-error.c                      | 359 ++++++++++++++++++++++
 drivers/fpga/dfl-fme-main.c                       |  42 +--
 drivers/fpga/dfl-fme.h                            |   3 +
 drivers/fpga/dfl.c                                |  10 +-
 drivers/fpga/dfl.h                                |   9 +
 12 files changed, 1041 insertions(+), 74 deletions(-)
 create mode 100644 drivers/fpga/dfl-afu-error.c
 create mode 100644 drivers/fpga/dfl-fme-error.c
Signature · 1 line
1.8.3.1
author
#1 lore raw

PATCH v5 1/9 fpga: dfl: make init callback optional

This patch makes init callback of sub features optional. With this change, people don't need to prepare any empty init callback.

---
 drivers/fpga/dfl.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/fpga/dfl.c +6 -4
index c0512af..96a2b82 100644
@@ -271,11 +271,13 @@ static int dfl_feature_instance_init(struct platform_device *pdev,
 				     struct dfl_feature *feature,
 				     struct dfl_feature_driver *drv)
 {
-	int ret;
+	int ret = 0;
 
-	ret = drv->ops->init(pdev, feature);
-	if (ret)
-		return ret;
+	if (drv->ops->init) {
+		ret = drv->ops->init(pdev, feature);
+		if (ret)
+			return ret;
+	}
 
 	feature->ops = drv->ops;
 
Signature · 1 line
1.8.3.1
maintainer Acked-by on v5 1/9
#2 lore raw

Re: [PATCH v5 1/9] fpga: dfl: make init callback optional

Hi,

Quoted 4 lines · Wu Hao

This patch makes init callback of sub features optional. With this change, people don't need to prepare any empty init callback.

Quoted 15 lines
---
 drivers/fpga/dfl.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/fpga/dfl.c +1 -1
index c0512af..96a2b82 100644
@@ -271,11 +271,13 @@ static int dfl_feature_instance_init(struct platform_device *pdev,
 				     struct dfl_feature *feature,
 				     struct dfl_feature_driver *drv)
 {
-	int ret;
+	int ret = 0;
 

drivers/fpga/dfl.c static int dfl_feature_instance_init(struct platform_device *pdev, · line 282

Quoted 10 lines
drivers/fpga/dfl.c +5 -3
-	ret = drv->ops->init(pdev, feature);
-	if (ret)
-		return ret;
+	if (drv->ops->init) {
+		ret = drv->ops->init(pdev, feature);
+		if (ret)
+			return ret;
+	}
 
 	feature->ops = drv->ops;

You could swap it around maybe like so:

int dfl_feature_instance_init() ...

{
	feature->ops = drv->ops;
	if (drv->ops->init)
		return drv->ops->init(pdev, feature);

	return 0;
}

With the caveat that feature->ops gets always set ...

Your call.

Thanks,
Moritz

author
#3 lore raw

Re: [PATCH v5 1/9] fpga: dfl: make init callback optional

Quoted 46 lines · Moritz Fischer

Hi,

Quoted 4 lines · Wu Hao

This patch makes init callback of sub features optional. With this change, people don't need to prepare any empty init callback.

Quoted 15 lines
---
 drivers/fpga/dfl.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/fpga/dfl.c +1 -1
index c0512af..96a2b82 100644
@@ -271,11 +271,13 @@ static int dfl_feature_instance_init(struct platform_device *pdev,
 				     struct dfl_feature *feature,
 				     struct dfl_feature_driver *drv)
 {
-	int ret;
+	int ret = 0;
 

drivers/fpga/dfl.c static int dfl_feature_instance_init(struct platform_device *pdev, · line 282

Quoted 10 lines
drivers/fpga/dfl.c +5 -3
-	ret = drv->ops->init(pdev, feature);
-	if (ret)
-		return ret;
+	if (drv->ops->init) {
+		ret = drv->ops->init(pdev, feature);
+		if (ret)
+			return ret;
+	}
 
 	feature->ops = drv->ops;

You could swap it around maybe like so:

int dfl_feature_instance_init() ...

{
	feature->ops = drv->ops;
	if (drv->ops->init)
		return drv->ops->init(pdev, feature);

	return 0;
}
Quoted 3 lines

With the caveat that feature->ops gets always set ...

Your call.

Hi Moritz,

Thanks a lot for the review and comments. It does simplify the code, will modify it.

Thanks
Hao

Quoted 3 lines

Thanks,
Moritz

author
#4 lore raw

PATCH v5 2/9 fpga: dfl: fme: convert platform_driver to use dev_groups

In reply to Wu Hao

This patch takes advantage of driver core which helps to create and remove sysfs attribute files, so there is no need to register sysfs entries manually in dfl-fme platform river code.

---
 drivers/fpga/dfl-fme-main.c | 29 ++---------------------------
 1 file changed, 2 insertions(+), 27 deletions(-)
drivers/fpga/dfl-fme-main.c +2 -27
index f033f1c..bf8114d 100644
@@ -129,30 +129,6 @@ static ssize_t socket_id_show(struct device *dev,
 };
 ATTRIBUTE_GROUPS(fme_hdr);
 
-static int fme_hdr_init(struct platform_device *pdev,
-			struct dfl_feature *feature)
-{
-	void __iomem *base = feature->ioaddr;
-	int ret;
-
-	dev_dbg(&pdev->dev, "FME HDR Init.\n");
-	dev_dbg(&pdev->dev, "FME cap %llx.\n",
-		(unsigned long long)readq(base + FME_HDR_CAP));
-
-	ret = device_add_groups(&pdev->dev, fme_hdr_groups);
-	if (ret)
-		return ret;
-
-	return 0;
-}
-
-static void fme_hdr_uinit(struct platform_device *pdev,
-			  struct dfl_feature *feature)
-{
-	dev_dbg(&pdev->dev, "FME HDR UInit.\n");
-	device_remove_groups(&pdev->dev, fme_hdr_groups);
-}
-
 static long fme_hdr_ioctl_release_port(struct dfl_feature_platform_data *pdata,
 				       unsigned long arg)
 {
@@ -199,8 +175,6 @@ static long fme_hdr_ioctl(struct platform_device *pdev,
 };
 
 static const struct dfl_feature_ops fme_hdr_ops = {
-	.init = fme_hdr_init,
-	.uinit = fme_hdr_uinit,
 	.ioctl = fme_hdr_ioctl,
 };
 
@@ -361,7 +335,8 @@ static int fme_remove(struct platform_device *pdev)
 
 static struct platform_driver fme_driver = {
 	.driver	= {
-		.name    = DFL_FPGA_FEATURE_DEV_FME,
+		.name       = DFL_FPGA_FEATURE_DEV_FME,
+		.dev_groups = fme_hdr_groups,
 	},
 	.probe   = fme_probe,
 	.remove  = fme_remove,
Signature · 1 line
1.8.3.1
maintainer Acked-by on v5 2/9
#5 lore raw

Re: [PATCH v5 2/9] fpga: dfl: fme: convert platform_driver to use dev_groups

Hi Hao,

Quoted 3 lines · Wu Hao

This patch takes advantage of driver core which helps to create and remove sysfs attribute files, so there is no need to register sysfs entries manually in dfl-fme platform river code.

Nit: s/river/driver

Quoted 2 lines
Quoted 61 lines
---
 drivers/fpga/dfl-fme-main.c | 29 ++---------------------------
 1 file changed, 2 insertions(+), 27 deletions(-)
drivers/fpga/dfl-fme-main.c +2 -27
index f033f1c..bf8114d 100644
@@ -129,30 +129,6 @@ static ssize_t socket_id_show(struct device *dev,
 };
 ATTRIBUTE_GROUPS(fme_hdr);
 
-static int fme_hdr_init(struct platform_device *pdev,
-			struct dfl_feature *feature)
-{
-	void __iomem *base = feature->ioaddr;
-	int ret;
-
-	dev_dbg(&pdev->dev, "FME HDR Init.\n");
-	dev_dbg(&pdev->dev, "FME cap %llx.\n",
-		(unsigned long long)readq(base + FME_HDR_CAP));
-
-	ret = device_add_groups(&pdev->dev, fme_hdr_groups);
-	if (ret)
-		return ret;
-
-	return 0;
-}
-
-static void fme_hdr_uinit(struct platform_device *pdev,
-			  struct dfl_feature *feature)
-{
-	dev_dbg(&pdev->dev, "FME HDR UInit.\n");
-	device_remove_groups(&pdev->dev, fme_hdr_groups);
-}
-
 static long fme_hdr_ioctl_release_port(struct dfl_feature_platform_data *pdata,
 				       unsigned long arg)
 {
@@ -199,8 +175,6 @@ static long fme_hdr_ioctl(struct platform_device *pdev,
 };
 
 static const struct dfl_feature_ops fme_hdr_ops = {
-	.init = fme_hdr_init,
-	.uinit = fme_hdr_uinit,
 	.ioctl = fme_hdr_ioctl,
 };
 
@@ -361,7 +335,8 @@ static int fme_remove(struct platform_device *pdev)
 
 static struct platform_driver fme_driver = {
 	.driver	= {
-		.name    = DFL_FPGA_FEATURE_DEV_FME,
+		.name       = DFL_FPGA_FEATURE_DEV_FME,
+		.dev_groups = fme_hdr_groups,
 	},
 	.probe   = fme_probe,
 	.remove  = fme_remove,
Signature · 1 line
1.8.3.1

Thanks,
Moritz

author
#6 lore raw

PATCH v5 3/9 fpga: dfl: afu: convert platform_driver to use dev_groups

In reply to Wu Hao

This patch takes advantage of driver core which helps to create and remove sysfs attribute files, so there is no need to register sysfs entries manually in dfl-afu platform river code.

---
 drivers/fpga/dfl-afu-main.c | 69 +++++++++++++++++++++++----------------------
 1 file changed, 36 insertions(+), 33 deletions(-)
drivers/fpga/dfl-afu-main.c +36 -33
index e50c45e..e955149 100644
@@ -282,24 +282,17 @@ static int port_get_id(struct platform_device *pdev)
 	&dev_attr_power_state.attr,
 	NULL,
 };
-ATTRIBUTE_GROUPS(port_hdr);
+
+static const struct attribute_group port_hdr_group = {
+	.attrs = port_hdr_attrs,
+};
 
 static int port_hdr_init(struct platform_device *pdev,
 			 struct dfl_feature *feature)
 {
-	dev_dbg(&pdev->dev, "PORT HDR Init.\n");
-
 	port_reset(pdev);
 
-	return device_add_groups(&pdev->dev, port_hdr_groups);
-}
-
-static void port_hdr_uinit(struct platform_device *pdev,
-			   struct dfl_feature *feature)
-{
-	dev_dbg(&pdev->dev, "PORT HDR UInit.\n");
-
-	device_remove_groups(&pdev->dev, port_hdr_groups);
+	return 0;
 }
 
 static long
@@ -330,7 +323,6 @@ static void port_hdr_uinit(struct platform_device *pdev,
 
 static const struct dfl_feature_ops port_hdr_ops = {
 	.init = port_hdr_init,
-	.uinit = port_hdr_uinit,
 	.ioctl = port_hdr_ioctl,
 };
 
@@ -361,32 +353,37 @@ static void port_hdr_uinit(struct platform_device *pdev,
 	&dev_attr_afu_id.attr,
 	NULL
 };
-ATTRIBUTE_GROUPS(port_afu);
 
-static int port_afu_init(struct platform_device *pdev,
-			 struct dfl_feature *feature)
+static umode_t port_afu_attrs_visible(struct kobject *kobj,
+				      struct attribute *attr, int n)
 {
-	struct resource *res = &pdev->resource[feature->resource_index];
-	int ret;
-
-	dev_dbg(&pdev->dev, "PORT AFU Init.\n");
+	struct device *dev = kobj_to_dev(kobj);
 
-	ret = afu_mmio_region_add(dev_get_platdata(&pdev->dev),
-				  DFL_PORT_REGION_INDEX_AFU, resource_size(res),
-				  res->start, DFL_PORT_REGION_READ |
-				  DFL_PORT_REGION_WRITE | DFL_PORT_REGION_MMAP);
-	if (ret)
-		return ret;
+	/*
+	 * sysfs entries are visible only if related private feature is
+	 * enumerated.
+	 */
+	if (!dfl_get_feature_by_id(dev, PORT_FEATURE_ID_AFU))
+		return 0;
 
-	return device_add_groups(&pdev->dev, port_afu_groups);
+	return attr->mode;
 }
 
-static void port_afu_uinit(struct platform_device *pdev,
-			   struct dfl_feature *feature)
+static const struct attribute_group port_afu_group = {
+	.attrs      = port_afu_attrs,
+	.is_visible = port_afu_attrs_visible,
+};
+
+static int port_afu_init(struct platform_device *pdev,
+			 struct dfl_feature *feature)
 {
-	dev_dbg(&pdev->dev, "PORT AFU UInit.\n");
+	struct resource *res = &pdev->resource[feature->resource_index];
 
-	device_remove_groups(&pdev->dev, port_afu_groups);
+	return afu_mmio_region_add(dev_get_platdata(&pdev->dev),
+				   DFL_PORT_REGION_INDEX_AFU,
+				   resource_size(res), res->start,
+				   DFL_PORT_REGION_MMAP | DFL_PORT_REGION_READ |
+				   DFL_PORT_REGION_WRITE);
 }
 
 static const struct dfl_feature_id port_afu_id_table[] = {
@@ -396,7 +393,6 @@ static void port_afu_uinit(struct platform_device *pdev,
 
 static const struct dfl_feature_ops port_afu_ops = {
 	.init = port_afu_init,
-	.uinit = port_afu_uinit,
 };
 
 static struct dfl_feature_driver port_feature_drvs[] = {
@@ -748,9 +744,16 @@ static int afu_remove(struct platform_device *pdev)
 	return 0;
 }
 
+static const struct attribute_group *afu_dev_groups[] = {
+	&port_hdr_group,
+	&port_afu_group,
+	NULL
+};
+
 static struct platform_driver afu_driver = {
 	.driver	= {
-		.name    = DFL_FPGA_FEATURE_DEV_PORT,
+		.name	    = DFL_FPGA_FEATURE_DEV_PORT,
+		.dev_groups = afu_dev_groups,
 	},
 	.probe   = afu_probe,
 	.remove  = afu_remove,
Signature · 1 line
1.8.3.1
maintainer Acked-by on v5 3/9
#7 lore raw

Re: [PATCH v5 3/9] fpga: dfl: afu: convert platform_driver to use dev_groups

Hi Hao,

Quoted 3 lines · Wu Hao

This patch takes advantage of driver core which helps to create and remove sysfs attribute files, so there is no need to register sysfs entries manually in dfl-afu platform river code.

Same nit: s/river/driver

Quoted 2 lines
Quoted 81 lines
---
 drivers/fpga/dfl-afu-main.c | 69 +++++++++++++++++++++++----------------------
 1 file changed, 36 insertions(+), 33 deletions(-)
drivers/fpga/dfl-afu-main.c +15 -28
index e50c45e..e955149 100644
@@ -282,24 +282,17 @@ static int port_get_id(struct platform_device *pdev)
 	&dev_attr_power_state.attr,
 	NULL,
 };
-ATTRIBUTE_GROUPS(port_hdr);
+
+static const struct attribute_group port_hdr_group = {
+	.attrs = port_hdr_attrs,
+};
 
 static int port_hdr_init(struct platform_device *pdev,
 			 struct dfl_feature *feature)
 {
-	dev_dbg(&pdev->dev, "PORT HDR Init.\n");
-
 	port_reset(pdev);
 
-	return device_add_groups(&pdev->dev, port_hdr_groups);
-}
-
-static void port_hdr_uinit(struct platform_device *pdev,
-			   struct dfl_feature *feature)
-{
-	dev_dbg(&pdev->dev, "PORT HDR UInit.\n");
-
-	device_remove_groups(&pdev->dev, port_hdr_groups);
+	return 0;
 }
 
 static long
@@ -330,7 +323,6 @@ static void port_hdr_uinit(struct platform_device *pdev,
 
 static const struct dfl_feature_ops port_hdr_ops = {
 	.init = port_hdr_init,
-	.uinit = port_hdr_uinit,
 	.ioctl = port_hdr_ioctl,
 };
 
@@ -361,32 +353,37 @@ static void port_hdr_uinit(struct platform_device *pdev,
 	&dev_attr_afu_id.attr,
 	NULL
 };
-ATTRIBUTE_GROUPS(port_afu);
 
-static int port_afu_init(struct platform_device *pdev,
-			 struct dfl_feature *feature)
+static umode_t port_afu_attrs_visible(struct kobject *kobj,
+				      struct attribute *attr, int n)
 {
-	struct resource *res = &pdev->resource[feature->resource_index];
-	int ret;
-
-	dev_dbg(&pdev->dev, "PORT AFU Init.\n");
+	struct device *dev = kobj_to_dev(kobj);
 
-	ret = afu_mmio_region_add(dev_get_platdata(&pdev->dev),
-				  DFL_PORT_REGION_INDEX_AFU, resource_size(res),
-				  res->start, DFL_PORT_REGION_READ |
-				  DFL_PORT_REGION_WRITE | DFL_PORT_REGION_MMAP);
-	if (ret)
-		return ret;
+	/*
+	 * sysfs entries are visible only if related private feature is
+	 * enumerated.
+	 */
+	if (!dfl_get_feature_by_id(dev, PORT_FEATURE_ID_AFU))
+		return 0;
 
-	return device_add_groups(&pdev->dev, port_afu_groups);
+	return attr->mode;
 }
 
-static void port_afu_uinit(struct platform_device *pdev,

drivers/fpga/dfl-afu-main.c static void port_hdr_uinit(struct platform_device *pdev, · line 387

Quoted 10 lines
drivers/fpga/dfl-afu-main.c +7 -2
-			   struct dfl_feature *feature)
+static const struct attribute_group port_afu_group = {
+	.attrs      = port_afu_attrs,
+	.is_visible = port_afu_attrs_visible,
+};
+
+static int port_afu_init(struct platform_device *pdev,
+			 struct dfl_feature *feature)
 {
-	dev_dbg(&pdev->dev, "PORT AFU UInit.\n");

Thanks.

Quoted 40 lines
drivers/fpga/dfl-afu-main.c +14 -3
+	struct resource *res = &pdev->resource[feature->resource_index];
 
-	device_remove_groups(&pdev->dev, port_afu_groups);
+	return afu_mmio_region_add(dev_get_platdata(&pdev->dev),
+				   DFL_PORT_REGION_INDEX_AFU,
+				   resource_size(res), res->start,
+				   DFL_PORT_REGION_MMAP | DFL_PORT_REGION_READ |
+				   DFL_PORT_REGION_WRITE);
 }
 
 static const struct dfl_feature_id port_afu_id_table[] = {
@@ -396,7 +393,6 @@ static void port_afu_uinit(struct platform_device *pdev,
 
 static const struct dfl_feature_ops port_afu_ops = {
 	.init = port_afu_init,
-	.uinit = port_afu_uinit,
 };
 
 static struct dfl_feature_driver port_feature_drvs[] = {
@@ -748,9 +744,16 @@ static int afu_remove(struct platform_device *pdev)
 	return 0;
 }
 
+static const struct attribute_group *afu_dev_groups[] = {
+	&port_hdr_group,
+	&port_afu_group,
+	NULL
+};
+
 static struct platform_driver afu_driver = {
 	.driver	= {
-		.name    = DFL_FPGA_FEATURE_DEV_PORT,
+		.name	    = DFL_FPGA_FEATURE_DEV_PORT,
+		.dev_groups = afu_dev_groups,
 	},
 	.probe   = afu_probe,
 	.remove  = afu_remove,
Signature · 1 line
1.8.3.1

Thanks,
Moritz

author
#8 lore raw

Re: [PATCH v5 3/9] fpga: dfl: afu: convert platform_driver to use dev_groups

Quoted 10 lines · Moritz Fischer

Hi Hao,

Quoted 3 lines · Wu Hao

This patch takes advantage of driver core which helps to create and remove sysfs attribute files, so there is no need to register sysfs entries manually in dfl-afu platform river code.

Same nit: s/river/driver

Quoted 2 lines

Hi Moritz

Thanks a lot for the review. : )

Have you got a chance to look into the other patches in this patchset?

Thanks
Hao

Quoted 135 lines
Quoted 81 lines
---
 drivers/fpga/dfl-afu-main.c | 69 +++++++++++++++++++++++----------------------
 1 file changed, 36 insertions(+), 33 deletions(-)
drivers/fpga/dfl-afu-main.c +15 -28
index e50c45e..e955149 100644
@@ -282,24 +282,17 @@ static int port_get_id(struct platform_device *pdev)
 	&dev_attr_power_state.attr,
 	NULL,
 };
-ATTRIBUTE_GROUPS(port_hdr);
+
+static const struct attribute_group port_hdr_group = {
+	.attrs = port_hdr_attrs,
+};
 
 static int port_hdr_init(struct platform_device *pdev,
 			 struct dfl_feature *feature)
 {
-	dev_dbg(&pdev->dev, "PORT HDR Init.\n");
-
 	port_reset(pdev);
 
-	return device_add_groups(&pdev->dev, port_hdr_groups);
-}
-
-static void port_hdr_uinit(struct platform_device *pdev,
-			   struct dfl_feature *feature)
-{
-	dev_dbg(&pdev->dev, "PORT HDR UInit.\n");
-
-	device_remove_groups(&pdev->dev, port_hdr_groups);
+	return 0;
 }
 
 static long
@@ -330,7 +323,6 @@ static void port_hdr_uinit(struct platform_device *pdev,
 
 static const struct dfl_feature_ops port_hdr_ops = {
 	.init = port_hdr_init,
-	.uinit = port_hdr_uinit,
 	.ioctl = port_hdr_ioctl,
 };
 
@@ -361,32 +353,37 @@ static void port_hdr_uinit(struct platform_device *pdev,
 	&dev_attr_afu_id.attr,
 	NULL
 };
-ATTRIBUTE_GROUPS(port_afu);
 
-static int port_afu_init(struct platform_device *pdev,
-			 struct dfl_feature *feature)
+static umode_t port_afu_attrs_visible(struct kobject *kobj,
+				      struct attribute *attr, int n)
 {
-	struct resource *res = &pdev->resource[feature->resource_index];
-	int ret;
-
-	dev_dbg(&pdev->dev, "PORT AFU Init.\n");
+	struct device *dev = kobj_to_dev(kobj);
 
-	ret = afu_mmio_region_add(dev_get_platdata(&pdev->dev),
-				  DFL_PORT_REGION_INDEX_AFU, resource_size(res),
-				  res->start, DFL_PORT_REGION_READ |
-				  DFL_PORT_REGION_WRITE | DFL_PORT_REGION_MMAP);
-	if (ret)
-		return ret;
+	/*
+	 * sysfs entries are visible only if related private feature is
+	 * enumerated.
+	 */
+	if (!dfl_get_feature_by_id(dev, PORT_FEATURE_ID_AFU))
+		return 0;
 
-	return device_add_groups(&pdev->dev, port_afu_groups);
+	return attr->mode;
 }
 
-static void port_afu_uinit(struct platform_device *pdev,

drivers/fpga/dfl-afu-main.c static void port_hdr_uinit(struct platform_device *pdev, · line 387

Quoted 10 lines
drivers/fpga/dfl-afu-main.c +7 -2
-			   struct dfl_feature *feature)
+static const struct attribute_group port_afu_group = {
+	.attrs      = port_afu_attrs,
+	.is_visible = port_afu_attrs_visible,
+};
+
+static int port_afu_init(struct platform_device *pdev,
+			 struct dfl_feature *feature)
 {
-	dev_dbg(&pdev->dev, "PORT AFU UInit.\n");

Thanks.

Quoted 40 lines
drivers/fpga/dfl-afu-main.c +14 -3
+	struct resource *res = &pdev->resource[feature->resource_index];
 
-	device_remove_groups(&pdev->dev, port_afu_groups);
+	return afu_mmio_region_add(dev_get_platdata(&pdev->dev),
+				   DFL_PORT_REGION_INDEX_AFU,
+				   resource_size(res), res->start,
+				   DFL_PORT_REGION_MMAP | DFL_PORT_REGION_READ |
+				   DFL_PORT_REGION_WRITE);
 }
 
 static const struct dfl_feature_id port_afu_id_table[] = {
@@ -396,7 +393,6 @@ static void port_afu_uinit(struct platform_device *pdev,
 
 static const struct dfl_feature_ops port_afu_ops = {
 	.init = port_afu_init,
-	.uinit = port_afu_uinit,
 };
 
 static struct dfl_feature_driver port_feature_drvs[] = {
@@ -748,9 +744,16 @@ static int afu_remove(struct platform_device *pdev)
 	return 0;
 }
 
+static const struct attribute_group *afu_dev_groups[] = {
+	&port_hdr_group,
+	&port_afu_group,
+	NULL
+};
+
 static struct platform_driver afu_driver = {
 	.driver	= {
-		.name    = DFL_FPGA_FEATURE_DEV_PORT,
+		.name	    = DFL_FPGA_FEATURE_DEV_PORT,
+		.dev_groups = afu_dev_groups,
 	},
 	.probe   = afu_probe,
 	.remove  = afu_remove,
Signature · 1 line
1.8.3.1

Thanks,
Moritz

author
#9 lore raw

PATCH v5 4/9 fpga: dfl: afu: add userclock sysfs interfaces.

In reply to Wu Hao

This patch introduces userclock sysfs interfaces for AFU, user could use these interfaces for clock setting to AFU.

Please note that, this is only working for port header feature with revision 0, for later revisions, userclock setting is moved to a separated private feature, so one revision sysfs interface is exposed to userspace application for this purpose too.

---
v2: rebased, and switched to use device_add/remove_groups for sysfs
v3: update kernel version and date in sysfs doc
v4: rebased.
v5: drop sysfs add/remove in init/uinit callback.
add missed locking for sysfs.
use is_visible to decide if hardware supports userclk or not. remove revision sysfs entry.

---
 Documentation/ABI/testing/sysfs-platform-dfl-port |  28 ++++++
 drivers/fpga/dfl-afu-main.c                       | 111 +++++++++++++++++++++-
 drivers/fpga/dfl.h                                |   9 ++
 3 files changed, 147 insertions(+), 1 deletion(-)
Documentation/ABI/testing/sysfs-platform-dfl-port +28 -0
index 1ab3e6f..c2660e4 100644
@@ -46,3 +46,31 @@ Contact:	Wu Hao <[email protected]>
 Description:	Read-write. Read or set AFU latency tolerance reporting value.
 		Set ltr to 1 if the AFU can tolerate latency >= 40us or set it
 		to 0 if it is latency sensitive.
+
+What:		/sys/bus/platform/devices/dfl-port.0/userclk_freqcmd
+Date:		August 2019
+KernelVersion:	5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Write-only. User writes command to this interface to set
+		userclock to AFU.
+
+What:		/sys/bus/platform/devices/dfl-port.0/userclk_freqsts
+Date:		August 2019
+KernelVersion:	5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-only. Read this file to get the status of issued command
+		to userclck_freqcmd.
+
+What:		/sys/bus/platform/devices/dfl-port.0/userclk_freqcntrcmd
+Date:		August 2019
+KernelVersion:	5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Write-only. User writes command to this interface to set
+		userclock counter.
+
+What:		/sys/bus/platform/devices/dfl-port.0/userclk_freqcntrsts
+Date:		August 2019
+KernelVersion:	5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-only. Read this file to get the status of issued command
+		to userclck_freqcntrcmd.
drivers/fpga/dfl-afu-main.c +110 -1

127 lines not on this page · raw message

drivers/fpga/dfl.h +9 -0
index 856ea4e..9f0e656 100644
@@ -120,6 +120,10 @@
 #define PORT_HDR_CAP		0x30
 #define PORT_HDR_CTRL		0x38
 #define PORT_HDR_STS		0x40
+#define PORT_HDR_USRCLK_CMD0	0x50
+#define PORT_HDR_USRCLK_CMD1	0x58
+#define PORT_HDR_USRCLK_STS0	0x60
+#define PORT_HDR_USRCLK_STS1	0x68
 
 /* Port Capability Register Bitfield */
 #define PORT_CAP_PORT_NUM	GENMASK_ULL(1, 0)	/* ID of this port */
@@ -355,6 +359,11 @@ static inline bool dfl_feature_is_port(void __iomem *base)
 		(FIELD_GET(DFH_ID, v) == DFH_ID_FIU_PORT);
 }
 
+static inline u8 dfl_feature_revision(void __iomem *base)
+{
+	return (u8)FIELD_GET(DFH_REVISION, readq(base + DFH));
+}
+
 /**
  * struct dfl_fpga_enum_info - DFL FPGA enumeration information
  *
Signature · 1 line
1.8.3.1
author
#10 lore raw

PATCH v5 5/9 fpga: dfl: afu: expose __afu_port_enable/disable function.

In reply to Wu Hao

As these two functions are used by other private features within the same driver module but different driver files. e.g. in error reporting private feature, it requires to clear errors when port is in reset.

---
v2: rebased
v5: add afu prefix to __port_enable/disable function to keep
alignment with other func exposed across different afu files.

---
 drivers/fpga/dfl-afu-main.c | 26 +++++++++++++++-----------
 drivers/fpga/dfl-afu.h      |  4 ++++
 2 files changed, 19 insertions(+), 11 deletions(-)
drivers/fpga/dfl-afu-main.c +15 -11
index f0b45f2..449185c 100644
@@ -22,14 +22,17 @@
 #include "dfl-afu.h"
 
 /**
- * port_enable - enable a port
+ * __afu_port_enable - enable a port by clear reset
  * @pdev: port platform device.
  *
  * Enable Port by clear the port soft reset bit, which is set by default.
  * The AFU is unable to respond to any MMIO access while in reset.
- * port_enable function should only be used after port_disable function.
+ * __afu_port_enable function should only be used after __afu_port_disable
+ * function.
+ *
+ * The caller needs to hold lock for protection.
  */
-static void port_enable(struct platform_device *pdev)
+void __afu_port_enable(struct platform_device *pdev)
 {
 	struct dfl_feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
 	void __iomem *base;
@@ -52,13 +55,14 @@ static void port_enable(struct platform_device *pdev)
 #define RST_POLL_TIMEOUT 1000 /* us */
 
 /**
- * port_disable - disable a port
+ * __afu_port_disable - disable a port by hold reset
  * @pdev: port platform device.
  *
- * Disable Port by setting the port soft reset bit, it puts the port into
- * reset.
+ * Disable Port by setting the port soft reset bit, it puts the port into reset.
+ *
+ * The caller needs to hold lock for protection.
  */
-static int port_disable(struct platform_device *pdev)
+int __afu_port_disable(struct platform_device *pdev)
 {
 	struct dfl_feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
 	void __iomem *base;
@@ -104,9 +108,9 @@ static int __port_reset(struct platform_device *pdev)
 {
 	int ret;
 
-	ret = port_disable(pdev);
+	ret = __afu_port_disable(pdev);
 	if (!ret)
-		port_enable(pdev);
+		__afu_port_enable(pdev);
 
 	return ret;
 }
@@ -799,9 +803,9 @@ static int port_enable_set(struct platform_device *pdev, bool enable)
 
 	mutex_lock(&pdata->lock);
 	if (enable)
-		port_enable(pdev);
+		__afu_port_enable(pdev);
 	else
-		ret = port_disable(pdev);
+		ret = __afu_port_disable(pdev);
 	mutex_unlock(&pdata->lock);
 
 	return ret;
drivers/fpga/dfl-afu.h +4 -0
index 0c7630a..83683f2 100644
@@ -79,6 +79,10 @@ struct dfl_afu {
 	struct dfl_feature_platform_data *pdata;
 };
 
+/* hold pdata->lock when call __afu_port_enable/disable */
+void __afu_port_enable(struct platform_device *pdev);
+int __afu_port_disable(struct platform_device *pdev);
+
 void afu_mmio_region_init(struct dfl_feature_platform_data *pdata);
 int afu_mmio_region_add(struct dfl_feature_platform_data *pdata,
 			u32 region_index, u64 region_size, u64 phys, u32 flags);
Signature · 1 line
1.8.3.1
author
#11 lore raw

PATCH v5 6/9 fpga: dfl: afu: add error reporting support.

In reply to Wu Hao

Error reporting is one important private feature, it reports error detected on port and accelerated function unit (AFU). It introduces several sysfs interfaces to allow userspace to check and clear errors detected by hardware.

---
v2: switch to device_add/remove_group for sysfs.
v3: update kernel version and date in sysfs doc
v4: remove dev_dbg in init/uinit callback function.
v5: rework init/uinit function and improve naming.
update sysfs entries:
remove revision sysfs entry.
merge WO "clear" sysfs to RO "errors" to keep alignment with latest changes in fme error reporting support.
expose error related sysfs entries via dev_groups.

---
 Documentation/ABI/testing/sysfs-platform-dfl-port |  25 +++
 drivers/fpga/Makefile                             |   1 +
 drivers/fpga/dfl-afu-error.c                      | 230 ++++++++++++++++++++++
 drivers/fpga/dfl-afu-main.c                       |   5 +
 drivers/fpga/dfl-afu.h                            |   5 +
 5 files changed, 266 insertions(+)
 create mode 100644 drivers/fpga/dfl-afu-error.c
Documentation/ABI/testing/sysfs-platform-dfl-port +25 -0
index c2660e4..6565826 100644
@@ -74,3 +74,28 @@ KernelVersion:	5.4
 Contact:	Wu Hao <[email protected]>
 Description:	Read-only. Read this file to get the status of issued command
 		to userclck_freqcntrcmd.
+
+What:		/sys/bus/platform/devices/dfl-port.0/errors/errors
+Date:		August 2019
+KernelVersion:	5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-Write. Read this file to get errors detected on port and
+		Accelerated Function Unit (AFU). Write error code to this file
+		to clear errors. Write fails with -EINVAL if input parsing
+		fails or input error code doesn't match. Write fails with
+		-EBUSY or -ETIMEDOUT if error can't be cleared as hardware
+		in low power state (-EBUSY) or not respoding (-ETIMEDOUT).
+
+What:		/sys/bus/platform/devices/dfl-port.0/errors/first_error
+Date:		August 2019
+KernelVersion:	5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-only. Read this file to get the first error detected by
+		hardware.
+
+What:		/sys/bus/platform/devices/dfl-port.0/errors/first_malformed_req
+Date:		August 2019
+KernelVersion:	5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-only. Read this file to get the first malformed request
+		captured by hardware.
drivers/fpga/Makefile +1 -0
index 312b937..7255891 100644
@@ -41,6 +41,7 @@ obj-$(CONFIG_FPGA_DFL_AFU)		+= dfl-afu.o
 
 dfl-fme-objs := dfl-fme-main.o dfl-fme-pr.o
 dfl-afu-objs := dfl-afu-main.o dfl-afu-region.o dfl-afu-dma-region.o
+dfl-afu-objs += dfl-afu-error.o
 
 # Drivers for FPGAs which implement DFL
 obj-$(CONFIG_FPGA_DFL_PCI)		+= dfl-pci.o
drivers/fpga/dfl-afu-error.c new file +230 -0

230 lines not on this page · raw message

drivers/fpga/dfl-afu-main.c +5 -0
index 449185c..e11352a 100644
@@ -518,6 +518,10 @@ static int port_afu_init(struct platform_device *pdev,
 		.ops = &port_afu_ops,
 	},
 	{
+		.id_table = port_err_id_table,
+		.ops = &port_err_ops,
+	},
+	{
 		.ops = NULL,
 	}
 };
@@ -860,6 +864,7 @@ static int afu_remove(struct platform_device *pdev)
 static const struct attribute_group *afu_dev_groups[] = {
 	&port_hdr_group,
 	&port_afu_group,
+	&port_err_group,
 	NULL
 };
 
drivers/fpga/dfl-afu.h +5 -0
index 83683f2..576e949 100644
@@ -101,4 +101,9 @@ int afu_dma_map_region(struct dfl_feature_platform_data *pdata,
 struct dfl_afu_dma_region *
 afu_dma_region_find(struct dfl_feature_platform_data *pdata,
 		    u64 iova, u64 size);
+
+extern const struct dfl_feature_ops port_err_ops;
+extern const struct dfl_feature_id port_err_id_table[];
+extern const struct attribute_group port_err_group;
+
 #endif /* __DFL_AFU_H */
Signature · 1 line
1.8.3.1
author
#12 lore raw

PATCH v5 7/9 fpga: dfl: afu: add STP (SignalTap) support

In reply to Wu Hao

STP (SignalTap) is one of the private features under the port for debugging. This patch adds private feature driver support for it to allow userspace applications to mmap related mmio region and provide STP service.

---
v4: remove uinit callback which does nothing.
remove dev_dbg in init callback function.

---
 drivers/fpga/dfl-afu-main.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
drivers/fpga/dfl-afu-main.c +25 -0
index e11352a..e4a34dc 100644
@@ -508,6 +508,27 @@ static int port_afu_init(struct platform_device *pdev,
 	.init = port_afu_init,
 };
 
+static int port_stp_init(struct platform_device *pdev,
+			 struct dfl_feature *feature)
+{
+	struct resource *res = &pdev->resource[feature->resource_index];
+
+	return afu_mmio_region_add(dev_get_platdata(&pdev->dev),
+				   DFL_PORT_REGION_INDEX_STP,
+				   resource_size(res), res->start,
+				   DFL_PORT_REGION_MMAP | DFL_PORT_REGION_READ |
+				   DFL_PORT_REGION_WRITE);
+}
+
+static const struct dfl_feature_id port_stp_id_table[] = {
+	{.id = PORT_FEATURE_ID_STP,},
+	{0,}
+};
+
+static const struct dfl_feature_ops port_stp_ops = {
+	.init = port_stp_init,
+};
+
 static struct dfl_feature_driver port_feature_drvs[] = {
 	{
 		.id_table = port_hdr_id_table,
@@ -522,6 +543,10 @@ static int port_afu_init(struct platform_device *pdev,
 		.ops = &port_err_ops,
 	},
 	{
+		.id_table = port_stp_id_table,
+		.ops = &port_stp_ops,
+	},
+	{
 		.ops = NULL,
 	}
 };
Signature · 1 line
1.8.3.1
author
#13 lore raw

PATCH v5 8/9 fpga: dfl: fme: add global error reporting support

In reply to Wu Hao

This patch adds support for global error reporting for FPGA Management Engine (FME), it introduces sysfs interfaces to report different error detected by the hardware, and allow user to clear errors or inject error for testing purpose.

---
v2: switch to device_add/remove_groups for sysfs.
v3: update kernel version and date in sysfs doc
v4: rebase, remove dev_dbg in init/uinit callback.
v5: reorganize sysfs entries:
remove "fme-errors" sub folder and related sysfs entries to upper level folder.
merge WO "clear" to RO "errors" to keep alignment with error sysfs entries in upper level folder.
remove revision sysfs entry.
add missed locking in sysfs entries.
expose sysfs group with is_visible via dev_groups.

---
 Documentation/ABI/testing/sysfs-platform-dfl-fme |  62 ++++
 drivers/fpga/Makefile                            |   2 +-
 drivers/fpga/dfl-fme-error.c                     | 359 +++++++++++++++++++++++
 drivers/fpga/dfl-fme-main.c                      |  17 +-
 drivers/fpga/dfl-fme.h                           |   3 +
 5 files changed, 440 insertions(+), 3 deletions(-)
 create mode 100644 drivers/fpga/dfl-fme-error.c
Documentation/ABI/testing/sysfs-platform-dfl-fme +62 -0
index 65372aa..72634d3 100644
@@ -44,3 +44,65 @@ Description:	Read-only. It returns socket_id to indicate which socket
 		this FPGA belongs to, only valid for integrated solution.
 		User only needs this information, in case standard numa node
 		can't provide correct information.
+
+What:		/sys/bus/platform/devices/dfl-fme.0/errors/pcie0_errors
+Date:		August 2019
+KernelVersion:  5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-Write. Read this file for errors detected on pcie0 link.
+		Write this file to clear errors logged in pcie0_errors. Write
+		fails with -EINVAL if input parsing fails or input error code
+		doesn't match.
+
+What:		/sys/bus/platform/devices/dfl-fme.0/errors/pcie1_errors
+Date:		August 2019
+KernelVersion:  5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-Write. Read this file for errors detected on pcie1 link.
+		Write this file to clear errors logged in pcie1_errors. Write
+		fails with -EINVAL if input parsing fails or input error code
+		doesn't match.
+
+What:		/sys/bus/platform/devices/dfl-fme.0/errors/nonfatal_errors
+Date:		August 2019
+KernelVersion:  5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-only. It returns non-fatal errors detected.
+
+What:		/sys/bus/platform/devices/dfl-fme.0/errors/catfatal_errors
+Date:		August 2019
+KernelVersion:  5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-only. It returns catastrophic and fatal errors detected.
+
+What:		/sys/bus/platform/devices/dfl-fme.0/errors/inject_errors
+Date:		August 2019
+KernelVersion:  5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-Write. Read this file to check errors injected. Write this
+		file to inject errors for testing purpose. Write fails with
+		-EINVAL if input parsing fails or input inject error code isn't
+		supported.
+
+What:		/sys/bus/platform/devices/dfl-fme.0/errors/fme_errors
+Date:		August 2019
+KernelVersion:  5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-Write. Read this file to get errors detected on FME.
+		Write this file to clear errors logged in fme_errors. Write
+		fials with -EINVAL if input parsing fails or input error code
+		doesn't match.
+
+What:		/sys/bus/platform/devices/dfl-fme.0/errors/first_error
+Date:		August 2019
+KernelVersion:  5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-only. Read this file to get the first error detected by
+		hardware.
+
+What:		/sys/bus/platform/devices/dfl-fme.0/errors/next_error
+Date:		August 2019
+KernelVersion:  5.4
+Contact:	Wu Hao <[email protected]>
+Description:	Read-only. Read this file to get the second error detected by
+		hardware.
drivers/fpga/Makefile +1 -1
index 7255891..4865b74 100644
@@ -39,7 +39,7 @@ obj-$(CONFIG_FPGA_DFL_FME_BRIDGE)	+= dfl-fme-br.o
 obj-$(CONFIG_FPGA_DFL_FME_REGION)	+= dfl-fme-region.o
 obj-$(CONFIG_FPGA_DFL_AFU)		+= dfl-afu.o
 
-dfl-fme-objs := dfl-fme-main.o dfl-fme-pr.o
+dfl-fme-objs := dfl-fme-main.o dfl-fme-pr.o dfl-fme-error.o
 dfl-afu-objs := dfl-afu-main.o dfl-afu-region.o dfl-afu-dma-region.o
 dfl-afu-objs += dfl-afu-error.o
 
drivers/fpga/dfl-fme-error.c new file +359 -0

359 lines not on this page · raw message

drivers/fpga/dfl-fme-main.c +15 -2
index bf8114d..4d78e18 100644
@@ -127,7 +127,10 @@ static ssize_t socket_id_show(struct device *dev,
 	&dev_attr_socket_id.attr,
 	NULL,
 };
-ATTRIBUTE_GROUPS(fme_hdr);
+
+static const struct attribute_group fme_hdr_group = {
+	.attrs = fme_hdr_attrs,
+};
 
 static long fme_hdr_ioctl_release_port(struct dfl_feature_platform_data *pdata,
 				       unsigned long arg)
@@ -188,6 +191,10 @@ static long fme_hdr_ioctl(struct platform_device *pdev,
 		.ops = &fme_pr_mgmt_ops,
 	},
 	{
+		.id_table = fme_global_err_id_table,
+		.ops = &fme_global_err_ops,
+	},
+	{
 		.ops = NULL,
 	},
 };
@@ -333,10 +340,16 @@ static int fme_remove(struct platform_device *pdev)
 	return 0;
 }
 
+static const struct attribute_group *fme_dev_groups[] = {
+	&fme_hdr_group,
+	&fme_global_err_group,
+	NULL
+};
+
 static struct platform_driver fme_driver = {
 	.driver	= {
 		.name       = DFL_FPGA_FEATURE_DEV_FME,
-		.dev_groups = fme_hdr_groups,
+		.dev_groups = fme_dev_groups,
 	},
 	.probe   = fme_probe,
 	.remove  = fme_remove,
drivers/fpga/dfl-fme.h +3 -0
index e4131e8..6685c8e 100644
@@ -35,5 +35,8 @@ struct dfl_fme {
 
 extern const struct dfl_feature_ops fme_pr_mgmt_ops;
 extern const struct dfl_feature_id fme_pr_mgmt_id_table[];
+extern const struct dfl_feature_ops fme_global_err_ops;
+extern const struct dfl_feature_id fme_global_err_id_table[];
+extern const struct attribute_group fme_global_err_group;
 
 #endif /* __DFL_FME_H */
Signature · 1 line
1.8.3.1
author
#14 lore raw

PATCH v5 9/9 Documentation: fpga: dfl: add descriptions for virtualization and new interfaces.

In reply to Wu Hao

This patch adds virtualization support description for DFL based FPGA devices (based on PCIe SRIOV), and introductions to new interfaces added by new dfl private feature drivers.

[[email protected]: Fixed up to make it work with new reStructuredText docs]
---
 Documentation/fpga/dfl.rst | 105 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
Documentation/fpga/dfl.rst +105 -0
index 2f125ab..6fa483f 100644
@@ -87,6 +87,8 @@ The following functions are exposed through ioctls:
 - Get driver API version (DFL_FPGA_GET_API_VERSION)
 - Check for extensions (DFL_FPGA_CHECK_EXTENSION)
 - Program bitstream (DFL_FPGA_FME_PORT_PR)
+- Assign port to PF (DFL_FPGA_FME_PORT_ASSIGN)
+- Release port from PF (DFL_FPGA_FME_PORT_RELEASE)
 
 More functions are exposed through sysfs
 (/sys/class/fpga_region/regionX/dfl-fme.n/):
@@ -102,6 +104,10 @@ More functions are exposed through sysfs
      one FPGA device may have more than one port, this sysfs interface indicates
      how many ports the FPGA device has.
 
+ Global error reporting management (errors/)
+     error reporting sysfs interfaces allow user to read errors detected by the
+     hardware, and clear the logged errors.
+
 
 FIU - PORT
 ==========
@@ -143,6 +149,10 @@ More functions are exposed through sysfs:
  Read Accelerator GUID (afu_id)
      afu_id indicates which PR bitstream is programmed to this AFU.
 
+ Error reporting (errors/)
+     error reporting sysfs interfaces allow user to read port/afu errors
+     detected by the hardware, and clear the logged errors.
+
 
 DFL Framework Overview
 ======================
@@ -218,6 +228,101 @@ the compat_id exposed by the target FPGA region. This check is usually done by
 userspace before calling the reconfiguration IOCTL.
 
 
+FPGA virtualization - PCIe SRIOV
+================================
+This section describes the virtualization support on DFL based FPGA device to
+enable accessing an accelerator from applications running in a virtual machine
+(VM). This section only describes the PCIe based FPGA device with SRIOV support.
+
+Features supported by the particular FPGA device are exposed through Device
+Feature Lists, as illustrated below:
+
+::
+
+    +-------------------------------+  +-------------+
+    |              PF               |  |     VF      |
+    +-------------------------------+  +-------------+
+        ^            ^         ^              ^
+        |            |         |              |
+  +-----|------------|---------|--------------|-------+
+  |     |            |         |              |       |
+  |  +-----+     +-------+ +-------+      +-------+   |
+  |  | FME |     | Port0 | | Port1 |      | Port2 |   |
+  |  +-----+     +-------+ +-------+      +-------+   |
+  |                  ^         ^              ^       |
+  |                  |         |              |       |
+  |              +-------+ +------+       +-------+   |
+  |              |  AFU  | |  AFU |       |  AFU  |   |
+  |              +-------+ +------+       +-------+   |
+  |                                                   |
+  |            DFL based FPGA PCIe Device             |
+  +---------------------------------------------------+
+
+FME is always accessed through the physical function (PF).
+
+Ports (and related AFUs) are accessed via PF by default, but could be exposed
+through virtual function (VF) devices via PCIe SRIOV. Each VF only contains
+1 Port and 1 AFU for isolation. Users could assign individual VFs (accelerators)
+created via PCIe SRIOV interface, to virtual machines.
+
+The driver organization in virtualization case is illustrated below:
+::
+
+    +-------++------++------+             |
+    | FME   || FME  || FME  |             |
+    | FPGA  || FPGA || FPGA |             |
+    |Manager||Bridge||Region|             |
+    +-------++------++------+             |
+    +-----------------------+  +--------+ |             +--------+
+    |          FME          |  |  AFU   | |             |  AFU   |
+    |         Module        |  | Module | |             | Module |
+    +-----------------------+  +--------+ |             +--------+
+          +-----------------------+       |       +-----------------------+
+          | FPGA Container Device |       |       | FPGA Container Device |
+          |  (FPGA Base Region)   |       |       |  (FPGA Base Region)   |
+          +-----------------------+       |       +-----------------------+
+            +------------------+          |         +------------------+
+            | FPGA PCIE Module |          | Virtual | FPGA PCIE Module |
+            +------------------+   Host   | Machine +------------------+
+   -------------------------------------- | ------------------------------
+             +---------------+            |          +---------------+
+             | PCI PF Device |            |          | PCI VF Device |
+             +---------------+            |          +---------------+
+
+FPGA PCIe device driver is always loaded first once a FPGA PCIe PF or VF device
+is detected. It:
+
+* Finishes enumeration on both FPGA PCIe PF and VF device using common
+  interfaces from DFL framework.
+* Supports SRIOV.
+
+The FME device driver plays a management role in this driver architecture, it
+provides ioctls to release Port from PF and assign Port to PF. After release
+a port from PF, then it's safe to expose this port through a VF via PCIe SRIOV
+sysfs interface.
+
+To enable accessing an accelerator from applications running in a VM, the
+respective AFU's port needs to be assigned to a VF using the following steps:
+
+#. The PF owns all AFU ports by default. Any port that needs to be
+   reassigned to a VF must first be released through the
+   DFL_FPGA_FME_PORT_RELEASE ioctl on the FME device.
+
+#. Once N ports are released from PF, then user can use command below
+   to enable SRIOV and VFs. Each VF owns only one Port with AFU.
+
+   ::
+
+      echo N > $PCI_DEVICE_PATH/sriov_numvfs
+
+#. Pass through the VFs to VMs
+
+#. The AFU under VF is accessible from applications in VM (using the
+   same driver inside the VF).
+
+Note that an FME can't be assigned to a VF, thus PR and other management
+functions are only available via the PF.
+
 Device enumeration
 ==================
 This section introduces how applications enumerate the fpga device from
Signature · 1 line
1.8.3.1
author
#15 lore raw
In reply to Wu Hao
Quoted 6 lines · Wu Hao

Hi Greg,

This is v5 patchset which adds more features to FPGA DFL. Marjor changes against v4 are sysfs related code rework to address comments on v4.

Please help to take a look. Thanks!

Hi Greg,

Did you get a chance to take a look at this new version patchset? :)

Thanks
Hao

Quoted 60 lines

Main changes from v4:
- convert code to use dev_groups for sysfs entries (#2, #3, #4, #6, #8).
- clean up for empty init function after remove sysfs add/remove (#1).
- introduce is_visible for sysfs groups (#3, #4, #6, #8).
- remove revision sysfs entries (#4, #6, #8).
- improve naming on shared functions (#5).
- reorganize sysfs entries for port and fme error reporting (#6, #8).

Main changes from v3:
- drop avx512 partail reconfiguration patch for now.
- split dfl_fpga_cdev_config_port to 2 functions *_release/assign_port

    (#1).
  - split __dfl_fpga_cdev_config_port_vf into 2 functions with locking
    added (#2).

- improve description in sysfs doc to avoid misunderstanding (#3).
- switch to boolean in sysfs entry store function (#3).
- remove dev_dbg in init/uinit callback function (#7, #9, #11).
- remove uinit callback which does does nothing (#8, #9)

Main changes from v2:
- update kernel version/date in sysfs doc (patch #4, #5, #8, #10, #11).
- add back Documentation patch (patch #12).

Main changes from v1:
- remove DRV/MODULE_VERSION modifications. (patch #1, #3, #4, #6)
- remove argsz from new ioctls. (patch #2)
- replace sysfs_create/remove_* with device_add/remove_* for sysfs entries.

    (patch #5, #8, #11)

Wu Hao (9):
fpga: dfl: make init callback optional
fpga: dfl: fme: convert platform_driver to use dev_groups
fpga: dfl: afu: convert platform_driver to use dev_groups
fpga: dfl: afu: add userclock sysfs interfaces.
fpga: dfl: afu: expose __afu_port_enable/disable function.
fpga: dfl: afu: add error reporting support.
fpga: dfl: afu: add STP (SignalTap) support
fpga: dfl: fme: add global error reporting support
Documentation: fpga: dfl: add descriptions for virtualization and new
interfaces.

 Documentation/ABI/testing/sysfs-platform-dfl-fme  |  62 ++++
 Documentation/ABI/testing/sysfs-platform-dfl-port |  53 ++++
 Documentation/fpga/dfl.rst                        | 105 +++++++
 drivers/fpga/Makefile                             |   3 +-
 drivers/fpga/dfl-afu-error.c                      | 230 ++++++++++++++
 drivers/fpga/dfl-afu-main.c                       | 230 +++++++++++---
 drivers/fpga/dfl-afu.h                            |   9 +
 drivers/fpga/dfl-fme-error.c                      | 359 ++++++++++++++++++++++
 drivers/fpga/dfl-fme-main.c                       |  42 +--
 drivers/fpga/dfl-fme.h                            |   3 +
 drivers/fpga/dfl.c                                |  10 +-
 drivers/fpga/dfl.h                                |   9 +
 12 files changed, 1041 insertions(+), 74 deletions(-)
 create mode 100644 drivers/fpga/dfl-afu-error.c
 create mode 100644 drivers/fpga/dfl-fme-error.c
Signature · 1 line
1.8.3.1
#16 lore raw
Quoted 11 lines · Wu Hao
Quoted 6 lines · Wu Hao

Hi Greg,

This is v5 patchset which adds more features to FPGA DFL. Marjor changes against v4 are sysfs related code rework to address comments on v4.

Please help to take a look. Thanks!

Hi Greg,

Did you get a chance to take a look at this new version patchset? :)

I'm not the FPGA maintainer, what about the review from the other one first? :)

thanks,

greg k-h

author
#17 lore raw
Quoted 7 lines · Greg KH
Quoted 6 lines · Wu Hao
Quoted 5 lines · Wu Hao

Hi Greg,

This is v5 patchset which adds more features to FPGA DFL. Marjor changes against v4 are sysfs related code rework to address comments on v4.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Please help to take a look. Thanks!

Hi Greg,

Did you get a chance to take a look at this new version patchset? :)

I'm not the FPGA maintainer, what about the review from the other one first? :)

Sure! :)

Hi Moritz

Could you please help review these patches? Thanks! :)

Thanks
Hao

Quoted 4 lines

thanks,

greg k-h