Skip to contentSkip to conversation

fpga: dfl: optional VSEC for start of dfl

PATCH Superseded [email protected] 4 participants 11 messages Nov 24, 2020 – Dec 2, 2020 2 messages not in the archive

Subsystems (3)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover fpga: dfl: optional VSEC for start of dfl

From: Matthew Gerlach <[email protected]>

The start of a Device Feature List (DFL) is currently assumed to be at Bar0/Offset 0 on the PCIe bus by drivers/fpga/dfl-pci.c. This patchset adds support for the start one or more DFLs to be specified in a Vendor-Specific Capability (VSEC) structure in PCIe config space. If no such VSEC structure exists, then the start is assumed to be Bar0/Offset 0 for backward compatibility.

Matthew Gerlach (2):
fpga: dfl: refactor cci_enumerate_feature_devs()
fpga: dfl: look for vendor specific capability

 Documentation/fpga/dfl.rst |  25 ++++++
 drivers/fpga/dfl-pci.c     | 169 +++++++++++++++++++++++++++++--------
 2 files changed, 159 insertions(+), 35 deletions(-)
Signature · 1 line
2.25.2
#1 lore raw

PATCH v3 1/2 fpga: dfl: refactor cci_enumerate_feature_devs()

From: Matthew Gerlach <[email protected]>

In preparation of looking for dfls based on a vendor specific pcie capability, move code that assumes Bar0/offset0 as start of DFL to its own function.

---
v3: no change

v2: remove spurious blank lines
rename find_dfl_in_bar0 to find_dfls_by_default

---
 drivers/fpga/dfl-pci.c | 84 +++++++++++++++++++++++-------------------
 1 file changed, 47 insertions(+), 37 deletions(-)
drivers/fpga/dfl-pci.c +47 -37
index a2203d03c9e2..b27fae045536 100644
@@ -119,49 +119,20 @@ static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec)
 	return table;
 }
 
-/* enumerate feature devices under pci device */
-static int cci_enumerate_feature_devs(struct pci_dev *pcidev)
+static int find_dfls_by_default(struct pci_dev *pcidev,
+				struct dfl_fpga_enum_info *info)
 {
-	struct cci_drvdata *drvdata = pci_get_drvdata(pcidev);
-	int port_num, bar, i, nvec, ret = 0;
-	struct dfl_fpga_enum_info *info;
-	struct dfl_fpga_cdev *cdev;
 	resource_size_t start, len;
+	int port_num, bar, i;
 	void __iomem *base;
-	int *irq_table;
+	int ret = 0;
 	u32 offset;
 	u64 v;
 
-	/* allocate enumeration info via pci_dev */
-	info = dfl_fpga_enum_info_alloc(&pcidev->dev);
-	if (!info)
-		return -ENOMEM;
-
-	/* add irq info for enumeration if the device support irq */
-	nvec = cci_pci_alloc_irq(pcidev);
-	if (nvec < 0) {
-		dev_err(&pcidev->dev, "Fail to alloc irq %d.\n", nvec);
-		ret = nvec;
-		goto enum_info_free_exit;
-	} else if (nvec) {
-		irq_table = cci_pci_create_irq_table(pcidev, nvec);
-		if (!irq_table) {
-			ret = -ENOMEM;
-			goto irq_free_exit;
-		}
-
-		ret = dfl_fpga_enum_info_add_irq(info, nvec, irq_table);
-		kfree(irq_table);
-		if (ret)
-			goto irq_free_exit;
-	}
-
-	/* start to find Device Feature List in Bar 0 */
+	/* start to find Device Feature List from Bar 0 */
 	base = cci_pci_ioremap_bar0(pcidev);
-	if (!base) {
-		ret = -ENOMEM;
-		goto irq_free_exit;
-	}
+	if (!base)
+		return -ENOMEM;
 
 	/*
 	 * PF device has FME and Ports/AFUs, and VF device only has one
@@ -208,12 +179,51 @@ static int cci_enumerate_feature_devs(struct pci_dev *pcidev)
 		dfl_fpga_enum_info_add_dfl(info, start, len);
 	} else {
 		ret = -ENODEV;
-		goto irq_free_exit;
 	}
 
 	/* release I/O mappings for next step enumeration */
 	pcim_iounmap_regions(pcidev, BIT(0));
 
+	return ret;
+}
+
+/* enumerate feature devices under pci device */
+static int cci_enumerate_feature_devs(struct pci_dev *pcidev)
+{
+	struct cci_drvdata *drvdata = pci_get_drvdata(pcidev);
+	struct dfl_fpga_enum_info *info;
+	struct dfl_fpga_cdev *cdev;
+	int nvec, ret = 0;
+	int *irq_table;
+
+	/* allocate enumeration info via pci_dev */
+	info = dfl_fpga_enum_info_alloc(&pcidev->dev);
+	if (!info)
+		return -ENOMEM;
+
+	/* add irq info for enumeration if the device support irq */
+	nvec = cci_pci_alloc_irq(pcidev);
+	if (nvec < 0) {
+		dev_err(&pcidev->dev, "Fail to alloc irq %d.\n", nvec);
+		ret = nvec;
+		goto enum_info_free_exit;
+	} else if (nvec) {
+		irq_table = cci_pci_create_irq_table(pcidev, nvec);
+		if (!irq_table) {
+			ret = -ENOMEM;
+			goto irq_free_exit;
+		}
+
+		ret = dfl_fpga_enum_info_add_irq(info, nvec, irq_table);
+		kfree(irq_table);
+		if (ret)
+			goto irq_free_exit;
+	}
+
+	ret = find_dfls_by_default(pcidev, info);
+	if (ret)
+		goto irq_free_exit;
+
 	/* start enumeration with prepared enumeration information */
 	cdev = dfl_fpga_feature_devs_enumerate(info);
 	if (IS_ERR(cdev)) {
Signature · 1 line
2.25.2

A message that is not in the archive (sent off-list or not yet received): DM6PR11MB3819B8E32400D0662CFAD20685F70@DM6PR11MB3819.namprd11.prod.outlook.com

#3 lore raw

RE: [PATCH v3 1/2] fpga: dfl: refactor cci_enumerate_feature_devs()

Quoted 10 lines · Hao
Quoted 7 lines

Subject: [PATCH v3 1/2] fpga: dfl: refactor cci_enumerate_feature_devs()

From: Matthew Gerlach <[email protected]>

In preparation of looking for dfls based on a vendor specific pcie capability, move code that assumes Bar0/offset0 as start of DFL to its own function.

as the default method to locate the first dfl.

How about the following?

In preparation of looking for dfls based on a vendor specific pci capability, move the code for the default method of finding the first dfl at offset 0 of Bar 0 to its own function.

Quoted 28 lines
Quoted 16 lines

---
v3: no change

v2: remove spurious blank lines
rename find_dfl_in_bar0 to find_dfls_by_default

---
 drivers/fpga/dfl-pci.c | 84 +++++++++++++++++++++++-------------------
 1 file changed, 47 insertions(+), 37 deletions(-)
drivers/fpga/dfl-pci.c +0 -0
index a2203d03c9e2..b27fae045536 100644

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev · line 124

Quoted 8 lines
drivers/fpga/dfl-pci.c +2 -2
@@ -119,49 +119,20 @@ static int *cci_pci_create_irq_table(struct pci_dev
 *pcidev, unsigned int nvec)
 	return table;
 }
 
-/* enumerate feature devices under pci device */
-static int cci_enumerate_feature_devs(struct pci_dev *pcidev)
+static int find_dfls_by_default(struct pci_dev *pcidev,
+				struct dfl_fpga_enum_info *info)

Please add one line comment to describe this is the default method is finding it in bar 0 and offset 0.

Quoted 12 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 128

Quoted 10 lines
drivers/fpga/dfl-pci.c +2 -5
 {
-	struct cci_drvdata *drvdata = pci_get_drvdata(pcidev);
-	int port_num, bar, i, nvec, ret = 0;
-	struct dfl_fpga_enum_info *info;
-	struct dfl_fpga_cdev *cdev;
 	resource_size_t start, len;
+	int port_num, bar, i;
 	void __iomem *base;
-	int *irq_table;
+	int ret = 0;

can be put into one line as previous code.

Ok, I can fix that.

Quoted 106 lines

Other places look good to me.

with above fixings,

Thanks
Hao

Quoted 95 lines
drivers/fpga/dfl-pci.c +3 -29
 	u32 offset;
 	u64 v;
 
-	/* allocate enumeration info via pci_dev */
-	info = dfl_fpga_enum_info_alloc(&pcidev->dev);
-	if (!info)
-		return -ENOMEM;
-
-	/* add irq info for enumeration if the device support irq */
-	nvec = cci_pci_alloc_irq(pcidev);
-	if (nvec < 0) {
-		dev_err(&pcidev->dev, "Fail to alloc irq %d.\n", nvec);
-		ret = nvec;
-		goto enum_info_free_exit;
-	} else if (nvec) {
-		irq_table = cci_pci_create_irq_table(pcidev, nvec);
-		if (!irq_table) {
-			ret = -ENOMEM;
-			goto irq_free_exit;
-		}
-
-		ret = dfl_fpga_enum_info_add_irq(info, nvec, irq_table);
-		kfree(irq_table);
-		if (ret)
-			goto irq_free_exit;
-	}
-
-	/* start to find Device Feature List in Bar 0 */
+	/* start to find Device Feature List from Bar 0 */
 	base = cci_pci_ioremap_bar0(pcidev);
-	if (!base) {
-		ret = -ENOMEM;
-		goto irq_free_exit;
-	}
+	if (!base)
+		return -ENOMEM;
 
 	/*
drivers/fpga/dfl-pci.c +40 -1
 	 * PF device has FME and Ports/AFUs, and VF device only has one
@@ -208,12 +179,51 @@ static int cci_enumerate_feature_devs(struct
 pci_dev *pcidev)
 		dfl_fpga_enum_info_add_dfl(info, start, len);
 	} else {
 		ret = -ENODEV;
-		goto irq_free_exit;
 	}
 
 	/* release I/O mappings for next step enumeration */
 	pcim_iounmap_regions(pcidev, BIT(0));
 
+	return ret;
+}
+
+/* enumerate feature devices under pci device */
+static int cci_enumerate_feature_devs(struct pci_dev *pcidev)
+{
+	struct cci_drvdata *drvdata = pci_get_drvdata(pcidev);
+	struct dfl_fpga_enum_info *info;
+	struct dfl_fpga_cdev *cdev;
+	int nvec, ret = 0;
+	int *irq_table;
+
+	/* allocate enumeration info via pci_dev */
+	info = dfl_fpga_enum_info_alloc(&pcidev->dev);
+	if (!info)
+		return -ENOMEM;
+
+	/* add irq info for enumeration if the device support irq */
+	nvec = cci_pci_alloc_irq(pcidev);
+	if (nvec < 0) {
+		dev_err(&pcidev->dev, "Fail to alloc irq %d.\n", nvec);
+		ret = nvec;
+		goto enum_info_free_exit;
+	} else if (nvec) {
+		irq_table = cci_pci_create_irq_table(pcidev, nvec);
+		if (!irq_table) {
+			ret = -ENOMEM;
+			goto irq_free_exit;
+		}
+
+		ret = dfl_fpga_enum_info_add_irq(info, nvec, irq_table);
+		kfree(irq_table);
+		if (ret)
+			goto irq_free_exit;
+	}
+
+	ret = find_dfls_by_default(pcidev, info);
+	if (ret)
+		goto irq_free_exit;
+
 	/* start enumeration with prepared enumeration information */
 	cdev = dfl_fpga_feature_devs_enumerate(info);

if (IS_ERR(cdev)) {
--
2.25.2

#4 lore raw

PATCH v3 2/2 fpga: dfl: look for vendor specific capability

In reply to [email protected]

From: Matthew Gerlach <[email protected]>

A DFL may not begin at offset 0 of BAR 0. A PCIe vendor specific capability can be used to specify the start of a number of DFLs.

---
v3: Add text and ascii art to documentation.
Ensure not to exceed PCIe config space in loop.

v2: Update documentation for clarity.
Clean up macro names.
Use GENMASK.
Removed spurious blank lines.
Changed some calls from dev_info to dev_dbg.
Specifically check for VSEC not found, -ENODEV.
Ensure correct pci vendor id.
Remove check for page alignment.
Rename find_dfl_in_cfg to find_dfls_by_vsec.
Initialize target memory of pci_read_config_dword to invalid values before use.

---
 Documentation/fpga/dfl.rst | 25 +++++++++++
 drivers/fpga/dfl-pci.c     | 91 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 115 insertions(+), 1 deletion(-)
Documentation/fpga/dfl.rst +25 -0
index 0404fe6ffc74..fa0da884a818 100644
@@ -501,6 +501,31 @@ Developer only needs to provide a sub feature driver with matched feature id.
 FME Partial Reconfiguration Sub Feature driver (see drivers/fpga/dfl-fme-pr.c)
 could be a reference.
 
+Location of DFLs on a PCI Device
+===========================
+There are two ways of locating DFLs on a PCI Device.  The original
+method assumed the start of the first DFL to offset 0 of bar 0.
+If the first node of the DFL is an FME, then further DFLs
+in the port(s) are specified in FME header registers.
+Alternatively, a vendor specific capability structure can be used to
+specify the location of all the DFLs on the device, providing flexibility
+for the type of starting node in the DFL.  Intel has reserved the
+VSEC ID of 0x43 for this purpose.  The vendor specific
+data begins with a 4 byte vendor specific register for the number of DFLs followed 4 byte
+Offset/BIR vendor specific registers for each DFL. Bits 2:0 of Offset/BIR register
+indicates the BAR, and bits 31:3 form the 8 byte aligned offset where bits 2:0 are
+zero.
+
+        +----------------------------+
+        |31     Number of DFLS      0|
+        +----------------------------+
+        |31     Offset     3|2 BIR  0|
+        +----------------------------+
+                      . . .
+        +----------------------------+
+        |31     Offset     3|2 BIR  0|
+        +----------------------------+
+
 
 Open discussion
 ===============
drivers/fpga/dfl-pci.c +90 -1

109 lines not on this page · raw message

Signature · 1 line
2.25.2

A message that is not in the archive (sent off-list or not yet received): DM6PR11MB38191D8C5E27E6E04B8DAA1A85F70@DM6PR11MB3819.namprd11.prod.outlook.com

#6 lore raw

RE: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Quoted 6 lines · Hao
Quoted 1 line

Subject: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Maybe we can change the title a little bit, what about
fpga: dfl-pci: locate DFLs by PCIe vendor specific capability

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

From: Matthew Gerlach <[email protected]>

A DFL may not begin at offset 0 of BAR 0. A PCIe vendor specific capability can be used to specify the start of a number of DFLs.

A PCIe vendor specific extended capability is introduced by Intel to specify the start of a number of DFLs.

Your suggestion is more precise.

Quoted 34 lines
Quoted 34 lines

---
v3: Add text and ascii art to documentation.
Ensure not to exceed PCIe config space in loop.

v2: Update documentation for clarity.
Clean up macro names.
Use GENMASK.
Removed spurious blank lines.
Changed some calls from dev_info to dev_dbg.
Specifically check for VSEC not found, -ENODEV.
Ensure correct pci vendor id.
Remove check for page alignment.
Rename find_dfl_in_cfg to find_dfls_by_vsec.
Initialize target memory of pci_read_config_dword to invalid values before
use.

---
 Documentation/fpga/dfl.rst | 25 +++++++++++
 drivers/fpga/dfl-pci.c     | 91 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 115 insertions(+), 1 deletion(-)
Documentation/fpga/dfl.rst +0 -0
index 0404fe6ffc74..fa0da884a818 100644
@@ -501,6 +501,31 @@ Developer only needs to provide a sub feature

driver with matched feature id.
FME Partial Reconfiguration Sub Feature driver (see drivers/fpga/dfl-fme-
pr.c)
could be a reference.

Quoted 7 lines

Documentation/fpga/dfl.rst Developer only needs to provide a sub feature driver with matched feature id. · line 506

Quoted 3 lines
Documentation/fpga/dfl.rst +3 -0
+Location of DFLs on a PCI Device
+===========================
+There are two ways of locating DFLs on a PCI Device.  The original

I found this new VSEC is only for PCIe device, correct? If so, let's make sure descriptions are accurate. E.g. default method for all devices and a new method for PCIe device.

Yes, the default method can be used with PCI and PCIe device, and the VSEC approach is PCIe, only. Documentation can be made more precise.

Quoted 15 lines
Quoted 9 lines
Documentation/fpga/dfl.rst +8 -0
+method assumed the start of the first DFL to offset 0 of bar 0.
+If the first node of the DFL is an FME, then further DFLs
+in the port(s) are specified in FME header registers.
+Alternatively, a vendor specific capability structure can be used to
+specify the location of all the DFLs on the device, providing flexibility
+for the type of starting node in the DFL.  Intel has reserved the
+VSEC ID of 0x43 for this purpose.  The vendor specific
+data begins with a 4 byte vendor specific register for the number of DFLs
Quoted 3 lines

followed 4 byte
+Offset/BIR vendor specific registers for each DFL. Bits 2:0 of Offset/BIR register

Do we have a defined register name here? or it's named as Offset/BIR register? Sounds a little wired, and I see you defined it as DFLS_RES?

The Offset/BIR terminology is also used in the MSI-X capability structure.

Quoted 10 lines
Quoted 10 lines

+indicates the BAR, and bits 31:3 form the 8 byte aligned offset where bits 2:0 are

Documentation/fpga/dfl.rst +7 -0
+zero.
+
+        +----------------------------+
+        |31     Number of DFLS      0|
+        +----------------------------+
+        |31     Offset     3|2 BIR  0|
+        +----------------------------+
Quoted 8 lines

Documentation/fpga/dfl.rst Developer only needs to provide a sub feature driver with matched feature id. · line 528

Quoted 5 lines
Documentation/fpga/dfl.rst +5 -0
+                      . . .
+        +----------------------------+
+        |31     Offset     3|2 BIR  0|
+        +----------------------------+
+

Maybe it's better to have register name and offset in above table.
BTW: if there is some public link to related spec, that helps too.

I'll consider adding a register name and offset, but I am not sure it adds much information.

Quoted 19 lines
Quoted 9 lines

Open discussion
===============

drivers/fpga/dfl-pci.c +0 -0
index b27fae045536..a58bf4299d6b 100644

drivers/fpga/dfl-pci.c line 33

Quoted 7 lines
drivers/fpga/dfl-pci.c +4 -0
@@ -27,6 +27,14 @@
 #define DRV_VERSION	"0.8"
 #define DRV_NAME	"dfl-pci"
 
+#define PCI_VSEC_ID_INTEL_DFLS 0x43
+
+#define PCI_VNDR_DFLS_CNT 8
+#define PCI_VNDR_DFLS_RES 0x0c

They are both register offset? it's better to use the same style.

Quoted 4 lines

0x8
0xc

Something like this.

Agrreed.

Quoted 25 lines
Quoted 12 lines
drivers/fpga/dfl-pci.c +4 -0
+
+#define PCI_VNDR_DFLS_RES_BAR_MASK GENMASK(2, 0)
+#define PCI_VNDR_DFLS_RES_OFF_MASK GENMASK(31, 3)
+
 struct cci_drvdata {
 	struct dfl_fpga_cdev *cdev;	/* container device */
 };
@@ -119,6 +127,84 @@ static int *cci_pci_create_irq_table(struct pci_dev
 *pcidev, unsigned int nvec)
 	return table;
 }

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev · line 139

Quoted 10 lines
drivers/fpga/dfl-pci.c +8 -0
 
+static int find_dfls_by_vsec(struct pci_dev *pcidev, struct
 dfl_fpga_enum_info *info)
+{
+	u32 bar, offset, vndr_hdr, dfl_cnt, dfl_res;
+	int dfl_res_off, i, voff = 0;
+	resource_size_t start, len;
+
+	if (pcidev->vendor != PCI_VENDOR_ID_INTEL)
+		return -ENODEV;

Check it later then other vendor can add their ones easily?

Quoted 6 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 140

Quoted 4 lines
drivers/fpga/dfl-pci.c +3 -0
+
+	while ((voff = pci_find_next_ext_capability(pcidev, voff,
 PCI_EXT_CAP_ID_VNDR))) {
+		vndr_hdr = 0;

It seems it doesn't need this.

Initializing vndr_hdr = 0 ensures a failed pci_read_config_dword() failure is handled properly. I will remove the call and the debug information anyway.

Quoted 13 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 147

Quoted 10 lines
drivers/fpga/dfl-pci.c +1 -0
+		pci_read_config_dword(pcidev, voff + PCI_VNDR_HEADER,

&vndr_hdr);

drivers/fpga/dfl-pci.c +3 -0
+
+		dev_dbg(&pcidev->dev,
+			"vendor-specific capability id 0x%x, rev 0x%x len

0x%x\n",

drivers/fpga/dfl-pci.c +3 -0
+			PCI_VNDR_HEADER_ID(vndr_hdr),
+			PCI_VNDR_HEADER_REV(vndr_hdr),
+			PCI_VNDR_HEADER_LEN(vndr_hdr));

Suggest remove this debug information.

Quoted 7 lines
Quoted 3 lines
drivers/fpga/dfl-pci.c +2 -0
+
+		if (PCI_VNDR_HEADER_ID(vndr_hdr) ==

PCI_VSEC_ID_INTEL_DFLS)

How about

		if (vendor == intel && header_id == INTEL_DFLS)
			break;

Seems reasonable.

Quoted 10 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 154

Quoted 6 lines
drivers/fpga/dfl-pci.c +5 -0
+			break;
+	}
+
+	if (!voff) {
+		dev_dbg(&pcidev->dev, "%s no VSEC found\n", __func__);

No DFL VSEC found

There could be many different VSECs but no DFL ones.

Agreed.

Quoted 8 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 158

Quoted 5 lines
drivers/fpga/dfl-pci.c +4 -0
+		return -ENODEV;
+	}
+
+	dfl_cnt = 0;

Can be merged into the line which defines dfl_cnt? Or we can just remove this line.

This initialization ensures that a failure to the pci_read_config_dword function below is handled properly. It can be merged into the definition line.

Quoted 9 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 163

Quoted 7 lines
drivers/fpga/dfl-pci.c +1 -0
+	pci_read_config_dword(pcidev, voff + PCI_VNDR_DFLS_CNT,

&dfl_cnt);

drivers/fpga/dfl-pci.c +4 -0
+	dev_dbg(&pcidev->dev, "dfl_cnt %d\n", dfl_cnt);
+	for (i = 0; i < dfl_cnt; i++) {
+		dfl_res_off = voff + PCI_VNDR_DFLS_RES +
+				      (i * sizeof(dfl_res));

Above two line can be put into one line, it's < 80 length.

Agreed.

Quoted 11 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 170

Quoted 9 lines
drivers/fpga/dfl-pci.c +2 -0
+		if (dfl_res_off >= PCI_CFG_SPACE_EXP_SIZE) {
+			dev_err(&pcidev->dev, "%s offset too big for PCIe

config space\n",

drivers/fpga/dfl-pci.c +5 -0
+				__func__);
+			return -EINVAL;
+		}
+
+		dfl_res = GENMASK(31, 0);

do we really need this?

Again, the assignment is ensuring that a failure to the pci_read_config_dword function is handled properly.

Quoted 7 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 173

Quoted 4 lines
drivers/fpga/dfl-pci.c +3 -0
+		pci_read_config_dword(pcidev, dfl_res_off, &dfl_res);
+
+		dev_dbg(&pcidev->dev, "dfl_res 0x%x\n", dfl_res);

Can be read by lspci even without driver, so we may not really need this debug information here.

I suppose the call to dev_dbg can be removed.

Quoted 17 lines
Quoted 13 lines
drivers/fpga/dfl-pci.c +11 -0
+
+		bar = dfl_res & PCI_VNDR_DFLS_RES_BAR_MASK;
+		if (bar >= PCI_STD_NUM_BARS) {
+			dev_err(&pcidev->dev, "%s bad bar number %d\n",
+				__func__, bar);
+			return -EINVAL;
+		}
+
+		len = pci_resource_len(pcidev, bar);
+		if (len == 0) {
+			dev_err(&pcidev->dev, "%s unmapped bar

number %d\n",

drivers/fpga/dfl-pci.c

Quoted 2 lines
drivers/fpga/dfl-pci.c +2 -0
+				__func__, bar);
+			return -EINVAL;

can be covered by below case, as mentioned in previous patch.

Agreed, I forgot to remove it.

Quoted 26 lines
Quoted 12 lines
drivers/fpga/dfl-pci.c +10 -0
+		}
+
+		offset = dfl_res & PCI_VNDR_DFLS_RES_OFF_MASK;
+		if (offset >= len) {
+			dev_err(&pcidev->dev, "%s bad offset %u >= %pa\n",
+				__func__, offset, &len);
+			return -EINVAL;
+		}
+
+		dev_dbg(&pcidev->dev, "%s BAR %d offset 0x%x\n",

__func__, bar, offset);

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 202

Quoted 6 lines
drivers/fpga/dfl-pci.c +6 -0
+
+		len -= offset;
+
+		start = pci_resource_start(pcidev, bar) + offset;
+
+		dfl_fpga_enum_info_add_dfl(info, start, len);

That means everytime, we pass [start, endofbar] region to dfl core for enumeration, if there are multiple DFLs in one bar, then each range ends at the same endofbar, it seems fine as enumeration can be done one by one, but ideally the best case is that this capability can provide end address or size too, right? It is possible that information can be added to the capability as well? then we don't have such limitation.

Quoted 1 line

Hao

I am not sure having more than one DFL in a bar serves any purpose over a single DFL. Regardless, I think the consistency of just having Offset/BIR in the VSEC is better than adding more infomation that has little or no added value.

Quoted 25 lines
Quoted 23 lines
drivers/fpga/dfl-pci.c +9 -2
+	}
+
+	return 0;
+}
+
 static int find_dfls_by_default(struct pci_dev *pcidev,
 				struct dfl_fpga_enum_info *info)
 {
@@ -220,7 +306,10 @@ static int cci_enumerate_feature_devs(struct
 pci_dev *pcidev)
 			goto irq_free_exit;
 	}
 
-	ret = find_dfls_by_default(pcidev, info);
+	ret = find_dfls_by_vsec(pcidev, info);
+	if (ret == -ENODEV)
+		ret = find_dfls_by_default(pcidev, info);
+
 	if (ret)
 		goto irq_free_exit;
 
--

2.25.2

maintainer
#7 lore raw

Re: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Hi Matthew,

Quoted 62 lines · On Mon, Nov 30, 2020 at 04:45:20PM -0800, [email protected] wrote:
Quoted 6 lines · Hao
Quoted 1 line

Subject: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Maybe we can change the title a little bit, what about
fpga: dfl-pci: locate DFLs by PCIe vendor specific capability

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

From: Matthew Gerlach <[email protected]>

A DFL may not begin at offset 0 of BAR 0. A PCIe vendor specific capability can be used to specify the start of a number of DFLs.

A PCIe vendor specific extended capability is introduced by Intel to specify the start of a number of DFLs.

Your suggestion is more precise.

Quoted 34 lines
Quoted 34 lines

---
v3: Add text and ascii art to documentation.
Ensure not to exceed PCIe config space in loop.

v2: Update documentation for clarity.
Clean up macro names.
Use GENMASK.
Removed spurious blank lines.
Changed some calls from dev_info to dev_dbg.
Specifically check for VSEC not found, -ENODEV.
Ensure correct pci vendor id.
Remove check for page alignment.
Rename find_dfl_in_cfg to find_dfls_by_vsec.
Initialize target memory of pci_read_config_dword to invalid values before
use.

---
 Documentation/fpga/dfl.rst | 25 +++++++++++
 drivers/fpga/dfl-pci.c     | 91 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 115 insertions(+), 1 deletion(-)
Documentation/fpga/dfl.rst +0 -0
index 0404fe6ffc74..fa0da884a818 100644
@@ -501,6 +501,31 @@ Developer only needs to provide a sub feature

driver with matched feature id.
FME Partial Reconfiguration Sub Feature driver (see drivers/fpga/dfl-fme-
pr.c)
could be a reference.

Quoted 7 lines

Documentation/fpga/dfl.rst Developer only needs to provide a sub feature driver with matched feature id. · line 506

Quoted 3 lines
Documentation/fpga/dfl.rst +3 -0
+Location of DFLs on a PCI Device
+===========================
+There are two ways of locating DFLs on a PCI Device.  The original

I found this new VSEC is only for PCIe device, correct? If so, let's make sure descriptions are accurate. E.g. default method for all devices and a new method for PCIe device.

Quoted 8 lines

Yes, the default method can be used with PCI and PCIe device, and the VSEC approach is PCIe, only. Documentation can be made more precise.

Quoted 5 lines

Documentation/fpga/dfl.rst Developer only needs to provide a sub feature driver with matched feature id. · line 510

Quoted 5 lines
Documentation/fpga/dfl.rst +4 -0
+method assumed the start of the first DFL to offset 0 of bar 0.
+If the first node of the DFL is an FME, then further DFLs
+in the port(s) are specified in FME header registers.
+Alternatively, a vendor specific capability structure can be used to

Maybe: a vendor specific extended capability (VSEC) ...

Quoted 11 lines
Quoted 10 lines
Quoted 7 lines
Documentation/fpga/dfl.rst +4 -0
+specify the location of all the DFLs on the device, providing flexibility
+for the type of starting node in the DFL.  Intel has reserved the
+VSEC ID of 0x43 for this purpose.  The vendor specific
+data begins with a 4 byte vendor specific register for the number of DFLs

followed 4 byte
+Offset/BIR vendor specific registers for each DFL. Bits 2:0 of Offset/BIR register

Do we have a defined register name here? or it's named as Offset/BIR register? Sounds a little wired, and I see you defined it as DFLS_RES?

Quoted 1 line

The Offset/BIR terminology is also used in the MSI-X capability structure.

Yeah, this intuitively made sense to me having worked with PCIe :)

Quoted 20 lines
Quoted 11 lines
Quoted 11 lines

+indicates the BAR, and bits 31:3 form the 8 byte aligned offset where bits 2:0 are

Documentation/fpga/dfl.rst +7 -0
+zero.
+
+        +----------------------------+
+        |31     Number of DFLS      0|
+        +----------------------------+
+        |31     Offset     3|2 BIR  0|
+        +----------------------------+
Quoted 8 lines

Documentation/fpga/dfl.rst Developer only needs to provide a sub feature driver with matched feature id. · line 528

Quoted 5 lines
Documentation/fpga/dfl.rst +5 -0
+                      . . .
+        +----------------------------+
+        |31     Offset     3|2 BIR  0|
+        +----------------------------+
+

Maybe it's better to have register name and offset in above table.
BTW: if there is some public link to related spec, that helps too.

Quoted 2 lines

I'll consider adding a register name and offset, but I am not sure it adds much information.

I think this is fine together with textual description you have above.

Quoted 190 lines
Quoted 20 lines
Quoted 10 lines

Open discussion
===============

drivers/fpga/dfl-pci.c +0 -0
index b27fae045536..a58bf4299d6b 100644

drivers/fpga/dfl-pci.c line 33

Quoted 7 lines
drivers/fpga/dfl-pci.c +4 -0
@@ -27,6 +27,14 @@
 #define DRV_VERSION	"0.8"
 #define DRV_NAME	"dfl-pci"
 
+#define PCI_VSEC_ID_INTEL_DFLS 0x43
+
+#define PCI_VNDR_DFLS_CNT 8
+#define PCI_VNDR_DFLS_RES 0x0c

They are both register offset? it's better to use the same style.

Quoted 4 lines

0x8
0xc

Something like this.

Agrreed.

Quoted 25 lines
Quoted 12 lines
drivers/fpga/dfl-pci.c +4 -0
+
+#define PCI_VNDR_DFLS_RES_BAR_MASK GENMASK(2, 0)
+#define PCI_VNDR_DFLS_RES_OFF_MASK GENMASK(31, 3)
+
 struct cci_drvdata {
 	struct dfl_fpga_cdev *cdev;	/* container device */
 };
@@ -119,6 +127,84 @@ static int *cci_pci_create_irq_table(struct pci_dev
 *pcidev, unsigned int nvec)
 	return table;
 }

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev · line 139

Quoted 10 lines
drivers/fpga/dfl-pci.c +8 -0
 
+static int find_dfls_by_vsec(struct pci_dev *pcidev, struct
 dfl_fpga_enum_info *info)
+{
+	u32 bar, offset, vndr_hdr, dfl_cnt, dfl_res;
+	int dfl_res_off, i, voff = 0;
+	resource_size_t start, len;
+
+	if (pcidev->vendor != PCI_VENDOR_ID_INTEL)
+		return -ENODEV;

Check it later then other vendor can add their ones easily?

Quoted 6 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 140

Quoted 4 lines
drivers/fpga/dfl-pci.c +3 -0
+
+	while ((voff = pci_find_next_ext_capability(pcidev, voff,
 PCI_EXT_CAP_ID_VNDR))) {
+		vndr_hdr = 0;

It seems it doesn't need this.

Initializing vndr_hdr = 0 ensures a failed pci_read_config_dword() failure is handled properly. I will remove the call and the debug information anyway.

Quoted 13 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 147

Quoted 10 lines
drivers/fpga/dfl-pci.c +1 -0
+		pci_read_config_dword(pcidev, voff + PCI_VNDR_HEADER,

&vndr_hdr);

drivers/fpga/dfl-pci.c +3 -0
+
+		dev_dbg(&pcidev->dev,
+			"vendor-specific capability id 0x%x, rev 0x%x len

0x%x\n",

drivers/fpga/dfl-pci.c +3 -0
+			PCI_VNDR_HEADER_ID(vndr_hdr),
+			PCI_VNDR_HEADER_REV(vndr_hdr),
+			PCI_VNDR_HEADER_LEN(vndr_hdr));

Suggest remove this debug information.

Quoted 7 lines
Quoted 3 lines
drivers/fpga/dfl-pci.c +2 -0
+
+		if (PCI_VNDR_HEADER_ID(vndr_hdr) ==

PCI_VSEC_ID_INTEL_DFLS)

How about

		if (vendor == intel && header_id == INTEL_DFLS)
			break;

Seems reasonable.

Quoted 10 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 154

Quoted 6 lines
drivers/fpga/dfl-pci.c +5 -0
+			break;
+	}
+
+	if (!voff) {
+		dev_dbg(&pcidev->dev, "%s no VSEC found\n", __func__);

No DFL VSEC found

There could be many different VSECs but no DFL ones.

Agreed.

Quoted 8 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 158

Quoted 5 lines
drivers/fpga/dfl-pci.c +4 -0
+		return -ENODEV;
+	}
+
+	dfl_cnt = 0;

Can be merged into the line which defines dfl_cnt? Or we can just remove this line.

This initialization ensures that a failure to the pci_read_config_dword function below is handled properly. It can be merged into the definition line.

Quoted 9 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 163

Quoted 7 lines
drivers/fpga/dfl-pci.c +1 -0
+	pci_read_config_dword(pcidev, voff + PCI_VNDR_DFLS_CNT,

&dfl_cnt);

drivers/fpga/dfl-pci.c +4 -0
+	dev_dbg(&pcidev->dev, "dfl_cnt %d\n", dfl_cnt);
+	for (i = 0; i < dfl_cnt; i++) {
+		dfl_res_off = voff + PCI_VNDR_DFLS_RES +
+				      (i * sizeof(dfl_res));

Above two line can be put into one line, it's < 80 length.

Agreed.

Quoted 11 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 170

Quoted 9 lines
drivers/fpga/dfl-pci.c +2 -0
+		if (dfl_res_off >= PCI_CFG_SPACE_EXP_SIZE) {
+			dev_err(&pcidev->dev, "%s offset too big for PCIe

config space\n",

drivers/fpga/dfl-pci.c +5 -0
+				__func__);
+			return -EINVAL;
+		}
+
+		dfl_res = GENMASK(31, 0);

do we really need this?

Again, the assignment is ensuring that a failure to the pci_read_config_dword function is handled properly.

Quoted 7 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 173

Quoted 4 lines
drivers/fpga/dfl-pci.c +3 -0
+		pci_read_config_dword(pcidev, dfl_res_off, &dfl_res);
+
+		dev_dbg(&pcidev->dev, "dfl_res 0x%x\n", dfl_res);

Can be read by lspci even without driver, so we may not really need this debug information here.

I suppose the call to dev_dbg can be removed.

Quoted 17 lines
Quoted 13 lines
drivers/fpga/dfl-pci.c +11 -0
+
+		bar = dfl_res & PCI_VNDR_DFLS_RES_BAR_MASK;
+		if (bar >= PCI_STD_NUM_BARS) {
+			dev_err(&pcidev->dev, "%s bad bar number %d\n",
+				__func__, bar);
+			return -EINVAL;
+		}
+
+		len = pci_resource_len(pcidev, bar);
+		if (len == 0) {
+			dev_err(&pcidev->dev, "%s unmapped bar

number %d\n",

drivers/fpga/dfl-pci.c

Quoted 2 lines
drivers/fpga/dfl-pci.c +2 -0
+				__func__, bar);
+			return -EINVAL;

can be covered by below case, as mentioned in previous patch.

Agreed, I forgot to remove it.

Quoted 26 lines
Quoted 12 lines
drivers/fpga/dfl-pci.c +10 -0
+		}
+
+		offset = dfl_res & PCI_VNDR_DFLS_RES_OFF_MASK;
+		if (offset >= len) {
+			dev_err(&pcidev->dev, "%s bad offset %u >= %pa\n",
+				__func__, offset, &len);
+			return -EINVAL;
+		}
+
+		dev_dbg(&pcidev->dev, "%s BAR %d offset 0x%x\n",

__func__, bar, offset);

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 202

Quoted 6 lines
drivers/fpga/dfl-pci.c +6 -0
+
+		len -= offset;
+
+		start = pci_resource_start(pcidev, bar) + offset;
+
+		dfl_fpga_enum_info_add_dfl(info, start, len);

That means everytime, we pass [start, endofbar] region to dfl core for enumeration, if there are multiple DFLs in one bar, then each range ends at the same endofbar, it seems fine as enumeration can be done one by one, but ideally the best case is that this capability can provide end address or size too, right? It is possible that information can be added to the capability as well? then we don't have such limitation.

Quoted 6 lines
Quoted 1 line

Hao

I am not sure having more than one DFL in a bar serves any purpose over a single DFL. Regardless, I think the consistency of just having Offset/BIR in the VSEC is better than adding more infomation that has little or no added value.

Agreed. Can't you just link the DFLs in that case?

Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
drivers/fpga/dfl-pci.c +9 -1
+	}
+
+	return 0;
+}
+
 static int find_dfls_by_default(struct pci_dev *pcidev,
 				struct dfl_fpga_enum_info *info)
 {
@@ -220,7 +306,10 @@ static int cci_enumerate_feature_devs(struct
 pci_dev *pcidev)
 			goto irq_free_exit;
 	}
 
-	ret = find_dfls_by_default(pcidev, info);
+	ret = find_dfls_by_vsec(pcidev, info);
+	if (ret == -ENODEV)
+		ret = find_dfls_by_default(pcidev, info);
+
 	if (ret)
 		goto irq_free_exit;
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

--
2.25.2

- Moritz

maintainer
#8 lore raw

Re: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Quoted 26 lines
Quoted 26 lines
Quoted 26 lines
Quoted 12 lines
drivers/fpga/dfl-pci.c +10 -0
+		}
+
+		offset = dfl_res & PCI_VNDR_DFLS_RES_OFF_MASK;
+		if (offset >= len) {
+			dev_err(&pcidev->dev, "%s bad offset %u >= %pa\n",
+				__func__, offset, &len);
+			return -EINVAL;
+		}
+
+		dev_dbg(&pcidev->dev, "%s BAR %d offset 0x%x\n",

__func__, bar, offset);

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 202

Quoted 6 lines
drivers/fpga/dfl-pci.c +6 -0
+
+		len -= offset;
+
+		start = pci_resource_start(pcidev, bar) + offset;
+
+		dfl_fpga_enum_info_add_dfl(info, start, len);

That means everytime, we pass [start, endofbar] region to dfl core for enumeration, if there are multiple DFLs in one bar, then each range ends at the same endofbar, it seems fine as enumeration can be done one by one, but ideally the best case is that this capability can provide end address or size too, right? It is possible that information can be added to the capability as well? then we don't have such limitation.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Hao

I am not sure having more than one DFL in a bar serves any purpose over a single DFL. Regardless, I think the consistency of just having Offset/BIR in the VSEC is better than adding more infomation that has little or no added value.

Agreed. Can't you just link the DFLs in that case?

I didn't see the value of more DFLs in one bar either. So I think we'd better document it.

Thanks,
Yilun

#9 lore raw

RE: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Quoted 31 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec)

Quoted 6 lines
drivers/fpga/dfl-pci.c +5 -0
+		}
+
+		offset = dfl_res & PCI_VNDR_DFLS_RES_OFF_MASK;
+		if (offset >= len) {
+			dev_err(&pcidev->dev, "%s bad

offset %u >= %pa\n",

Quoted 20 lines
Quoted 20 lines
Quoted 20 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 202

Quoted 12 lines
drivers/fpga/dfl-pci.c +5 -0
+				__func__, offset, &len);
+			return -EINVAL;
+		}
+
+		dev_dbg(&pcidev->dev, "%s BAR %d offset 0x%x\n",

__func__, bar, offset);

drivers/fpga/dfl-pci.c +6 -0
+
+		len -= offset;
+
+		start = pci_resource_start(pcidev, bar) + offset;
+
+		dfl_fpga_enum_info_add_dfl(info, start, len);

That means everytime, we pass [start, endofbar] region to dfl core for enumeration, if there are multiple DFLs in one bar, then each range ends at the same endofbar, it seems fine as enumeration can be done one by one, but ideally the best case is that this capability can provide end address or size too, right? It is possible that information can be added to the capability as well? then we don't have such limitation.

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

Hao

I am not sure having more than one DFL in a bar serves any purpose over

a

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

single DFL. Regardless, I think the consistency of just having Offset/BIR in the VSEC is better than adding more infomation that has little or no added value.

Agreed. Can't you just link the DFLs in that case?

I didn't see the value of more DFLs in one bar either. So I think we'd better document it.

Yes, it needs to be documented well, otherwise users may have their own choices, e.g. link 100 queues together by modify DFH registers of the queues one by one, or just have them done together in the VSEC. I am not sure which one is the easier way for logic developer, but at least we need to document what driver can support.

Thanks
Hao

Quoted 3 lines

Thanks,
Yilun

#10 lore raw

RE: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Quoted 40 lines · Hao
Quoted 31 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec)

Quoted 6 lines
drivers/fpga/dfl-pci.c +5 -0
+		}
+
+		offset = dfl_res & PCI_VNDR_DFLS_RES_OFF_MASK;
+		if (offset >= len) {
+			dev_err(&pcidev->dev, "%s bad

offset %u >= %pa\n",

Quoted 20 lines
Quoted 20 lines
Quoted 20 lines

drivers/fpga/dfl-pci.c static int *cci_pci_create_irq_table(struct pci_dev *pcidev, unsigned int nvec) · line 202

Quoted 12 lines
drivers/fpga/dfl-pci.c +5 -0
+				__func__, offset, &len);
+			return -EINVAL;
+		}
+
+		dev_dbg(&pcidev->dev, "%s BAR %d offset 0x%x\n",

__func__, bar, offset);

drivers/fpga/dfl-pci.c +6 -0
+
+		len -= offset;
+
+		start = pci_resource_start(pcidev, bar) + offset;
+
+		dfl_fpga_enum_info_add_dfl(info, start, len);

That means everytime, we pass [start, endofbar] region to dfl core for enumeration, if there are multiple DFLs in one bar, then each range ends at the same endofbar, it seems fine as enumeration can be done one by one, but ideally the best case is that this capability can provide end address or size too, right? It is possible that information can be added to the capability as well? then we don't have such limitation.

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

Hao

I am not sure having more than one DFL in a bar serves any purpose over

a

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

single DFL. Regardless, I think the consistency of just having Offset/BIR in the VSEC is better than adding more infomation that has little or no added value.

Agreed. Can't you just link the DFLs in that case?

I didn't see the value of more DFLs in one bar either. So I think we'd better document it.

Quoted 9 lines

Yes, it needs to be documented well, otherwise users may have their own choices, e.g. link 100 queues together by modify DFH registers of the queues one by one, or just have them done together in the VSEC. I am not sure which one is the easier way for logic developer, but at least we need to document what driver can support.

Thanks
Hao

I will update the documentation to clarify the basis of the rule of one DFL per bar. Since we have a clearly stated rule, we can enforce the rule in the code and perform more error checking.

Quoted 4 lines
Quoted 3 lines

Thanks,
Yilun

#11 lore raw

RE: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

In reply to Moritz Fischer
Quoted 90 lines

Subject: Re: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Hi Matthew,

Quoted 12 lines · On Mon, Nov 30, 2020 at 04:45:20PM -0800, [email protected] wrote:
Quoted 8 lines · Hao
Quoted 1 line

Subject: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Maybe we can change the title a little bit, what about
fpga: dfl-pci: locate DFLs by PCIe vendor specific capability

Quoted 3 lines

From: Matthew Gerlach <[email protected]>

Quoted 26 lines
Quoted 6 lines
Quoted 3 lines

A DFL may not begin at offset 0 of BAR 0. A PCIe vendor specific capability can be used to specify the start of a number of DFLs.

A PCIe vendor specific extended capability is introduced by Intel to specify the start of a number of DFLs.

Your suggestion is more precise.

Quoted 18 lines
Quoted 18 lines

---
v3: Add text and ascii art to documentation.
Ensure not to exceed PCIe config space in loop.

v2: Update documentation for clarity.
Clean up macro names.
Use GENMASK.
Removed spurious blank lines.
Changed some calls from dev_info to dev_dbg. Specifically check for VSEC not found, -ENODEV.
Ensure correct pci vendor id.
Remove check for page alignment.
Rename find_dfl_in_cfg to find_dfls_by_vsec.
Initialize target memory of pci_read_config_dword to invalid values

before

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

use.

---
 Documentation/fpga/dfl.rst | 25 +++++++++++
 drivers/fpga/dfl-pci.c     | 91

+++++++++++++++++++++++++++++++++++++-

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
 2 files changed, 115 insertions(+), 1 deletion(-)
Documentation/fpga/dfl.rst +0 -0
index 0404fe6ffc74..fa0da884a818 100644
@@ -501,6 +501,31 @@ Developer only needs to provide a sub feature

driver with matched feature id.
FME Partial Reconfiguration Sub Feature driver (see drivers/fpga/dfl-

fme-

Quoted 11 lines
Quoted 10 lines

Documentation/fpga/dfl.rst Developer only needs to provide a sub feature driver with matched feature id. · line 506

Quoted 6 lines

pr.c)
could be a reference.

Documentation/fpga/dfl.rst +3 -0
+Location of DFLs on a PCI Device
+===========================
+There are two ways of locating DFLs on a PCI Device.  The original

I found this new VSEC is only for PCIe device, correct? If so, let's make sure descriptions are accurate. E.g. default method for all devices and a new method for PCIe device.

Quoted 1 line

Yes, the default method can be used with PCI and PCIe device, and the

VSEC

Quoted 7 lines

approach is PCIe, only. Documentation can be made more precise.

Quoted 5 lines

Documentation/fpga/dfl.rst Developer only needs to provide a sub feature driver with matched feature id. · line 510

Quoted 5 lines
Documentation/fpga/dfl.rst +4 -0
+method assumed the start of the first DFL to offset 0 of bar 0.
+If the first node of the DFL is an FME, then further DFLs
+in the port(s) are specified in FME header registers.
+Alternatively, a vendor specific capability structure can be used to

Maybe: a vendor specific extended capability (VSEC) ...

Quoted 4 lines
Quoted 4 lines

Documentation/fpga/dfl.rst Developer only needs to provide a sub feature driver with matched feature id.

Quoted 4 lines
Documentation/fpga/dfl.rst +4 -0
+specify the location of all the DFLs on the device, providing flexibility
+for the type of starting node in the DFL.  Intel has reserved the
+VSEC ID of 0x43 for this purpose.  The vendor specific
+data begins with a 4 byte vendor specific register for the number of

DFLs

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

followed 4 byte
+Offset/BIR vendor specific registers for each DFL. Bits 2:0 of Offset/BIR register

Quoted 7 lines
Quoted 1 line
Quoted 1 line

Do we have a defined register name here? or it's named as Offset/BIR

register?

Quoted 3 lines
Quoted 1 line

Sounds a little wired, and I see you defined it as DFLS_RES?

The Offset/BIR terminology is also used in the MSI-X capability structure.

Yeah, this intuitively made sense to me having worked with PCIe :)

I just feel that it's better to use the same register name defined in the code below. So people can find matched information in both code and doc easily. : )

Thanks
Hao

#12 lore raw

RE: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Quoted 92 lines · Hao
Quoted 92 lines

Subject: Re: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Hi Matthew,

Quoted 12 lines · On Mon, Nov 30, 2020 at 04:45:20PM -0800, [email protected] wrote:
Quoted 8 lines · Hao
Quoted 1 line

Subject: [PATCH v3 2/2] fpga: dfl: look for vendor specific capability

Maybe we can change the title a little bit, what about
fpga: dfl-pci: locate DFLs by PCIe vendor specific capability

Quoted 3 lines

From: Matthew Gerlach <[email protected]>

Quoted 26 lines
Quoted 6 lines
Quoted 3 lines

A DFL may not begin at offset 0 of BAR 0. A PCIe vendor specific capability can be used to specify the start of a number of DFLs.

A PCIe vendor specific extended capability is introduced by Intel to specify the start of a number of DFLs.

Your suggestion is more precise.

Quoted 18 lines
Quoted 18 lines

---
v3: Add text and ascii art to documentation.
Ensure not to exceed PCIe config space in loop.

v2: Update documentation for clarity.
Clean up macro names.
Use GENMASK.
Removed spurious blank lines.
Changed some calls from dev_info to dev_dbg. Specifically check for VSEC not found, -ENODEV.
Ensure correct pci vendor id.
Remove check for page alignment.
Rename find_dfl_in_cfg to find_dfls_by_vsec.
Initialize target memory of pci_read_config_dword to invalid values

before

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

use.

---
 Documentation/fpga/dfl.rst | 25 +++++++++++
 drivers/fpga/dfl-pci.c     | 91

+++++++++++++++++++++++++++++++++++++-

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
 2 files changed, 115 insertions(+), 1 deletion(-)
Documentation/fpga/dfl.rst +0 -0
index 0404fe6ffc74..fa0da884a818 100644
@@ -501,6 +501,31 @@ Developer only needs to provide a sub feature

driver with matched feature id.
FME Partial Reconfiguration Sub Feature driver (see drivers/fpga/dfl-

fme-

Quoted 11 lines
Quoted 10 lines

Documentation/fpga/dfl.rst Developer only needs to provide a sub feature driver with matched feature id. · line 506

Quoted 6 lines

pr.c)
could be a reference.

Documentation/fpga/dfl.rst +3 -0
+Location of DFLs on a PCI Device
+===========================
+There are two ways of locating DFLs on a PCI Device.  The original

I found this new VSEC is only for PCIe device, correct? If so, let's make sure descriptions are accurate. E.g. default method for all devices and a new method for PCIe device.

Quoted 1 line

Yes, the default method can be used with PCI and PCIe device, and the

VSEC

Quoted 7 lines

approach is PCIe, only. Documentation can be made more precise.

Quoted 5 lines

Documentation/fpga/dfl.rst Developer only needs to provide a sub feature driver with matched feature id. · line 510

Quoted 5 lines
Documentation/fpga/dfl.rst +4 -0
+method assumed the start of the first DFL to offset 0 of bar 0.
+If the first node of the DFL is an FME, then further DFLs
+in the port(s) are specified in FME header registers.
+Alternatively, a vendor specific capability structure can be used to

Maybe: a vendor specific extended capability (VSEC) ...

Quoted 4 lines
Quoted 4 lines

Documentation/fpga/dfl.rst Developer only needs to provide a sub feature driver with matched feature id.

Quoted 4 lines
Documentation/fpga/dfl.rst +4 -0
+specify the location of all the DFLs on the device, providing flexibility
+for the type of starting node in the DFL.  Intel has reserved the
+VSEC ID of 0x43 for this purpose.  The vendor specific
+data begins with a 4 byte vendor specific register for the number of

DFLs

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

followed 4 byte
+Offset/BIR vendor specific registers for each DFL. Bits 2:0 of Offset/BIR register

Do we have a defined register name here? or it's named as Offset/BIR

register?

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Sounds a little wired, and I see you defined it as DFLS_RES?

The Offset/BIR terminology is also used in the MSI-X capability structure.

Yeah, this intuitively made sense to me having worked with PCIe :)

I just feel that it's better to use the same register name defined in the code below. So people can find matched information in both code and doc easily. : )

I think this makes sense. I can change the name of the variable to bir to match the documentation.

Quoted 4 lines

Thanks
Hao