Skip to contentSkip to conversation

PCI: enable extended tags support for PCIe devices

PATCH Applied Sinan Kaya 2 participants 4 messages Jan 20, 2017 – Feb 9, 2017 PCI

lore
Share & tools
lore

Sinan Kaya

author
#0 lore raw

PATCH v2 1/1 PCI: enable extended tags support for PCIe devices

Each PCIe device can issue up to 32 transactions at a time by default. This limitation is historically coming from PCI. Each transaction is tracked by a tag number on the bus. 2.2.6.2. Transaction Descriptor
– Transaction ID Field section of the PCIe 3.1 specification describes extended tags.

PCI supports 32 outstanding non-posted requests at a given time. This number has been extended to 256 on PCI Express. According to the specification, all PCIe devices are required to support receiving 8-bit Tags (Tag completer). The PCIe-PCI bridges handle the translation of 8-bit tags to 5-bit tags.

However, the generation of 8-bit tags is left optional to a particular HW implementation. The code needs to check HW support before attempting to enable extended tags producer capability.

32 outstanding transactions is not enough for some performance critical applications especially when a lot of small sized frames are transmitted.

Extended tags support increases this number to 256. Devices not supporting extended tags tie-off this field to 0. According to ECN, it is safe to enable this feature for all PCIe devices.

---
 drivers/pci/probe.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
drivers/pci/probe.c +16 -0
index e164b5c..1192475 100644
@@ -1650,12 +1650,28 @@ static void program_hpp_type2(struct pci_dev *dev, struct hpp_type2 *hpp)
 	 */
 }
 
+static void pci_configure_extended_tags(struct pci_dev *pdev)
+{
+	u32 dev_cap;
+	int ret;
+
+	ret = pcie_capability_read_dword(pdev, PCI_EXP_DEVCAP, &dev_cap);
+
+	if (ret)
+		return;
+
+	if (dev_cap & PCI_EXP_DEVCAP_EXT_TAG)
+		pcie_capability_set_word(pdev, PCI_EXP_DEVCTL,
+					 PCI_EXP_DEVCTL_EXT_TAG);
+}
+
 static void pci_configure_device(struct pci_dev *dev)
 {
 	struct hotplug_params hpp;
 	int ret;
 
 	pci_configure_mps(dev);
+	pci_configure_extended_tags(dev);
 
 	memset(&hpp, 0, sizeof(hpp));
 	ret = pci_get_hp_params(dev, &hpp);
Signature · 1 line
1.9.1
author
#1 lore raw
Quoted 6 lines · Sinan Kaya

Each PCIe device can issue up to 32 transactions at a time by default. This limitation is historically coming from PCI. Each transaction is tracked by a tag number on the bus. 2.2.6.2. Transaction Descriptor
– Transaction ID Field section of the PCIe 3.1 specification describes extended tags.

Apologies for the spam, no idea what happened. I need to review my publish scripts.

#2 lore raw
In reply to Sinan Kaya
Quoted 16 lines · Sinan Kaya

Each PCIe device can issue up to 32 transactions at a time by default. This limitation is historically coming from PCI. Each transaction is tracked by a tag number on the bus. 2.2.6.2. Transaction Descriptor
– Transaction ID Field section of the PCIe 3.1 specification describes extended tags.

PCI supports 32 outstanding non-posted requests at a given time. This number has been extended to 256 on PCI Express. According to the specification, all PCIe devices are required to support receiving 8-bit Tags (Tag completer). The PCIe-PCI bridges handle the translation of 8-bit tags to 5-bit tags.

However, the generation of 8-bit tags is left optional to a particular HW implementation. The code needs to check HW support before attempting to enable extended tags producer capability.

Quoted 8 lines

32 outstanding transactions is not enough for some performance critical applications especially when a lot of small sized frames are transmitted.

Extended tags support increases this number to 256. Devices not supporting extended tags tie-off this field to 0. According to ECN, it is safe to enable this feature for all PCIe devices.

Applied to pci/enumeration for v4.11 as follows, thanks!

I added a check for PCIe, so we don't have to rely on pcie_capability_read_dword() doing the right thing for PCI and PCI-X devices. I think pcie_capability_read_dword() would set dev_cap to zero and return zero, which would be safe, but I don't the reader should have to do that much analysis to figure it out.

commit 8de9dcb0d1e0c92c525a826dd69ce61a21264050

Author: Sinan Kaya <[email protected]>
Date:   Fri Jan 20 09:16:51 2017 -0500

PCI: Enable PCIe Extended Tags if supported

Every PCIe device can generate 5-bit transaction Tags, which allow up to 32 concurrent requests. Some devices can generate 8-bit Extended Tags, which allow up to 256 concurrent requests.

Per the ECN mentioned below, all PCIe Receivers are expected to support Extended Tags, so devices are allowed (but not required) to enable them by default.

If a device supports Extended Tags but does not enable them by default, enable them. This allows the device to have up to 256 outstanding transactions at a time, which may improve performance.

drivers/pci/probe.c +18 -0
index aca5b2466adb..3abc94212197 100644
@@ -1661,12 +1661,30 @@ static void program_hpp_type2(struct pci_dev *dev, struct hpp_type2 *hpp)
 	 */
 }
 
+static void pci_configure_extended_tags(struct pci_dev *dev)
+{
+	u32 dev_cap;
+	int ret;
+
+	if (!pci_is_pcie(dev))
+		return;
+
+	ret = pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, &dev_cap);
+	if (ret)
+		return;
+
+	if (dev_cap & PCI_EXP_DEVCAP_EXT_TAG)
+		pcie_capability_set_word(dev, PCI_EXP_DEVCTL,
+					 PCI_EXP_DEVCTL_EXT_TAG);
+}
+
 static void pci_configure_device(struct pci_dev *dev)
 {
 	struct hotplug_params hpp;
 	int ret;
 
 	pci_configure_mps(dev);
+	pci_configure_extended_tags(dev);
 
 	memset(&hpp, 0, sizeof(hpp));
 	ret = pci_get_hp_params(dev, &hpp);
Quoted 40 lines
---
 drivers/pci/probe.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
drivers/pci/probe.c +16 -0
index e164b5c..1192475 100644
@@ -1650,12 +1650,28 @@ static void program_hpp_type2(struct pci_dev *dev, struct hpp_type2 *hpp)
 	 */
 }
 
+static void pci_configure_extended_tags(struct pci_dev *pdev)
+{
+	u32 dev_cap;
+	int ret;
+
+	ret = pcie_capability_read_dword(pdev, PCI_EXP_DEVCAP, &dev_cap);
+
+	if (ret)
+		return;
+
+	if (dev_cap & PCI_EXP_DEVCAP_EXT_TAG)
+		pcie_capability_set_word(pdev, PCI_EXP_DEVCTL,
+					 PCI_EXP_DEVCTL_EXT_TAG);
+}
+
 static void pci_configure_device(struct pci_dev *dev)
 {
 	struct hotplug_params hpp;
 	int ret;
 
 	pci_configure_mps(dev);
+	pci_configure_extended_tags(dev);
 
 	memset(&hpp, 0, sizeof(hpp));
 	ret = pci_get_hp_params(dev, &hpp);
Signature · 1 line
1.9.1
author
#3 lore raw
Quoted 7 lines · Bjorn Helgaas

Applied to pci/enumeration for v4.11 as follows, thanks!

I added a check for PCIe, so we don't have to rely on pcie_capability_read_dword() doing the right thing for PCI and PCI-X devices. I think pcie_capability_read_dword() would set dev_cap to zero and return zero, which would be safe, but I don't the reader should have to do that much analysis to figure it out.

Thanks for cleaning up the commit message and adding the pci_is_pcie check.

Signature · 3 lines
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.