Skip to contentSkip to conversation

PCI: dwc: Add support to configure for ECRC

PATCH Superseded Vidya Sagar 5 participants 16 messages Nov 9, 2020 – Dec 11, 2020

Subsystems (3)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 PCI: dwc: Add support to configure for ECRC

DesignWare core has a TLP digest (TD) override bit in one of the control registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare IP version 4.90a.

---
 drivers/pci/controller/dwc/pcie-designware.c | 50 ++++++++++++++++++--
 drivers/pci/controller/dwc/pcie-designware.h |  1 +
 2 files changed, 47 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-designware.c +46 -4
index c2dea8fc97c8..ebdc37a58e94 100644
@@ -225,6 +225,44 @@ static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg,
 	dw_pcie_writel_atu(pci, offset + reg, val);
 }
 
+static inline u32 dw_pcie_enable_ecrc(u32 val)
+{
+	/*
+	 *     DesignWare core version 4.90A has this strange design issue
+	 * where the 'TD' bit in the Control register-1 of the ATU outbound
+	 * region acts like an override for the ECRC setting i.e. the presence
+	 * of TLP Digest(ECRC) in the outgoing TLPs is solely determined by
+	 * this bit. This is contrary to the PCIe spec which says that the
+	 * enablement of the ECRC is solely determined by the AER registers.
+	 *     Because of this, even when the ECRC is enabled through AER
+	 * registers, the transactions going through ATU won't have TLP Digest
+	 * as there is no way the AER sub-system could program the TD bit which
+	 * is specific to DsignWare core.
+	 *    The best way to handle this scenario is to program the TD bit
+	 * always. It affects only the traffic from root port to downstream
+	 * devices.
+	 * At this point,
+	 *     When ECRC is enabled in AER registers, everything works
+	 * normally
+	 *     When ECRC is NOT enabled in AER registers, then,
+	 * on Root Port:- TLP Digest (DWord size) gets appended to each packet
+	 *                even through it is not required. Since downstream
+	 *                TLPs are mostly for configuration accesses and BAR
+	 *                accesses, they are not in critical path and won't
+	 *                have much negative effect on the performance.
+	 * on End Point:- TLP Digest is received for some/all the packets coming
+	 *                from the root port. TLP Digest is ignored because,
+	 *                as per the PCIe Spec r5.0 v1.0 section 2.2.3 "TLP Digest Rules",
+	 *                when an endpoint receives TLP Digest when its
+	 *                ECRC check functionality is disabled in AER registers,
+	 *                received TLP Digest is just ignored.
+	 * Since there is no issue or error reported either side, best way to
+	 * handle the scenario is to program TD bit by default.
+	 */
+
+	return val | PCIE_ATU_TD;
+}
+
 static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 					     int index, int type,
 					     u64 cpu_addr, u64 pci_addr,
@@ -245,8 +283,10 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 				 lower_32_bits(pci_addr));
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
 				 upper_32_bits(pci_addr));
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1,
-				 type | PCIE_ATU_FUNC_NUM(func_no));
+	val = type | PCIE_ATU_FUNC_NUM(func_no);
+	if (pci->version == 0x490A)
+		val = dw_pcie_enable_ecrc(val);
+	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
 				 PCIE_ATU_ENABLE);
 
@@ -292,8 +332,10 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 			   lower_32_bits(pci_addr));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET,
 			   upper_32_bits(pci_addr));
-	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, type |
-			   PCIE_ATU_FUNC_NUM(func_no));
+	val = type | PCIE_ATU_FUNC_NUM(func_no);
+	if (pci->version == 0x490A)
+		val = dw_pcie_enable_ecrc(val);
+	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
 	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
 
 	/*
drivers/pci/controller/dwc/pcie-designware.h +1 -0
index 9d2f511f13fa..285c0ae364ae 100644
@@ -88,6 +88,7 @@
 #define PCIE_ATU_TYPE_IO		0x2
 #define PCIE_ATU_TYPE_CFG0		0x4
 #define PCIE_ATU_TYPE_CFG1		0x5
+#define PCIE_ATU_TD			BIT(8)
 #define PCIE_ATU_FUNC_NUM(pf)           ((pf) << 20)
 #define PCIE_ATU_CR2			0x908
 #define PCIE_ATU_ENABLE			BIT(31)
Signature · 1 line
2.17.1
Acked-by on v1 1/1
#1 lore raw
Quoted 6 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in one of the control registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare IP version 4.90a.

Modulo typos/formatting comments below,

Thanks for working through this.

Quoted 16 lines
---
 drivers/pci/controller/dwc/pcie-designware.c | 50 ++++++++++++++++++--
 drivers/pci/controller/dwc/pcie-designware.h |  1 +
 2 files changed, 47 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-designware.c +3 -0
index c2dea8fc97c8..ebdc37a58e94 100644
@@ -225,6 +225,44 @@ static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg,
 	dw_pcie_writel_atu(pci, offset + reg, val);
 }
 
+static inline u32 dw_pcie_enable_ecrc(u32 val)
+{
+	/*

drivers/pci/controller/dwc/pcie-designware.c static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg, · line 240

Quoted 10 lines
drivers/pci/controller/dwc/pcie-designware.c +10 -0
+	 *     DesignWare core version 4.90A has this strange design issue
+	 * where the 'TD' bit in the Control register-1 of the ATU outbound
+	 * region acts like an override for the ECRC setting i.e. the presence
+	 * of TLP Digest(ECRC) in the outgoing TLPs is solely determined by
+	 * this bit. This is contrary to the PCIe spec which says that the
+	 * enablement of the ECRC is solely determined by the AER registers.
+	 *     Because of this, even when the ECRC is enabled through AER
+	 * registers, the transactions going through ATU won't have TLP Digest
+	 * as there is no way the AER sub-system could program the TD bit which
+	 * is specific to DsignWare core.

s/DsignWare/DesignWare/

drivers/pci/controller/dwc/pcie-designware.c static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg, · line 243

Quoted 3 lines
drivers/pci/controller/dwc/pcie-designware.c +3 -0
+	 *    The best way to handle this scenario is to program the TD bit
+	 * always. It affects only the traffic from root port to downstream
+	 * devices.

Convention is to separate paragraphs with blank lines, not to indent the first line.

drivers/pci/controller/dwc/pcie-designware.c static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg, · line 255

Quoted 12 lines
drivers/pci/controller/dwc/pcie-designware.c +12 -0
+	 * At this point,
+	 *     When ECRC is enabled in AER registers, everything works
+	 * normally
+	 *     When ECRC is NOT enabled in AER registers, then,
+	 * on Root Port:- TLP Digest (DWord size) gets appended to each packet
+	 *                even through it is not required. Since downstream
+	 *                TLPs are mostly for configuration accesses and BAR
+	 *                accesses, they are not in critical path and won't
+	 *                have much negative effect on the performance.
+	 * on End Point:- TLP Digest is received for some/all the packets coming
+	 *                from the root port. TLP Digest is ignored because,
+	 *                as per the PCIe Spec r5.0 v1.0 section 2.2.3 "TLP Digest Rules",

Wrap to fit in 80 columns.

Quoted 54 lines
drivers/pci/controller/dwc/pcie-designware.c +18 -4
+	 *                when an endpoint receives TLP Digest when its
+	 *                ECRC check functionality is disabled in AER registers,
+	 *                received TLP Digest is just ignored.
+	 * Since there is no issue or error reported either side, best way to
+	 * handle the scenario is to program TD bit by default.
+	 */
+
+	return val | PCIE_ATU_TD;
+}
+
 static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 					     int index, int type,
 					     u64 cpu_addr, u64 pci_addr,
@@ -245,8 +283,10 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 				 lower_32_bits(pci_addr));
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
 				 upper_32_bits(pci_addr));
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1,
-				 type | PCIE_ATU_FUNC_NUM(func_no));
+	val = type | PCIE_ATU_FUNC_NUM(func_no);
+	if (pci->version == 0x490A)
+		val = dw_pcie_enable_ecrc(val);
+	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
 				 PCIE_ATU_ENABLE);
 
@@ -292,8 +332,10 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 			   lower_32_bits(pci_addr));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET,
 			   upper_32_bits(pci_addr));
-	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, type |
-			   PCIE_ATU_FUNC_NUM(func_no));
+	val = type | PCIE_ATU_FUNC_NUM(func_no);
+	if (pci->version == 0x490A)
+		val = dw_pcie_enable_ecrc(val);
+	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
 	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
 
 	/*
drivers/pci/controller/dwc/pcie-designware.h +1 -0
index 9d2f511f13fa..285c0ae364ae 100644
@@ -88,6 +88,7 @@
 #define PCIE_ATU_TYPE_IO		0x2
 #define PCIE_ATU_TYPE_CFG0		0x4
 #define PCIE_ATU_TYPE_CFG1		0x5
+#define PCIE_ATU_TD			BIT(8)
 #define PCIE_ATU_FUNC_NUM(pf)           ((pf) << 20)
 #define PCIE_ATU_CR2			0x908
 #define PCIE_ATU_ENABLE			BIT(31)
Signature · 1 line
2.17.1
#2 lore raw

PATCH v2 1/1 PCI: dwc: Add support to configure for ECRC

In reply to Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in one of the control registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare IP version 4.90a.

---
V2:
* Addressed Bjorn's comments

 drivers/pci/controller/dwc/pcie-designware.c | 52 ++++++++++++++++++--
 drivers/pci/controller/dwc/pcie-designware.h |  1 +
 2 files changed, 49 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-designware.c +48 -4
index c2dea8fc97c8..ec0d13ab6bad 100644
@@ -225,6 +225,46 @@ static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg,
 	dw_pcie_writel_atu(pci, offset + reg, val);
 }
 
+static inline u32 dw_pcie_enable_ecrc(u32 val)
+{
+	/*
+	 * DesignWare core version 4.90A has this strange design issue
+	 * where the 'TD' bit in the Control register-1 of the ATU outbound
+	 * region acts like an override for the ECRC setting i.e. the presence
+	 * of TLP Digest(ECRC) in the outgoing TLPs is solely determined by
+	 * this bit. This is contrary to the PCIe spec which says that the
+	 * enablement of the ECRC is solely determined by the AER registers.
+	 *
+	 * Because of this, even when the ECRC is enabled through AER
+	 * registers, the transactions going through ATU won't have TLP Digest
+	 * as there is no way the AER sub-system could program the TD bit which
+	 * is specific to DesignWare core.
+	 *
+	 * The best way to handle this scenario is to program the TD bit
+	 * always. It affects only the traffic from root port to downstream
+	 * devices.
+	 *
+	 * At this point,
+	 * When ECRC is enabled in AER registers, everything works normally
+	 * When ECRC is NOT enabled in AER registers, then,
+	 * on Root Port:- TLP Digest (DWord size) gets appended to each packet
+	 *                even through it is not required. Since downstream
+	 *                TLPs are mostly for configuration accesses and BAR
+	 *                accesses, they are not in critical path and won't
+	 *                have much negative effect on the performance.
+	 * on End Point:- TLP Digest is received for some/all the packets coming
+	 *                from the root port. TLP Digest is ignored because,
+	 *                as per the PCIe Spec r5.0 v1.0 section 2.2.3
+	 *                "TLP Digest Rules", when an endpoint receives TLP
+	 *                Digest when its ECRC check functionality is disabled
+	 *                in AER registers, received TLP Digest is just ignored.
+	 * Since there is no issue or error reported either side, best way to
+	 * handle the scenario is to program TD bit by default.
+	 */
+
+	return val | PCIE_ATU_TD;
+}
+
 static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 					     int index, int type,
 					     u64 cpu_addr, u64 pci_addr,
@@ -245,8 +285,10 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 				 lower_32_bits(pci_addr));
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
 				 upper_32_bits(pci_addr));
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1,
-				 type | PCIE_ATU_FUNC_NUM(func_no));
+	val = type | PCIE_ATU_FUNC_NUM(func_no);
+	if (pci->version == 0x490A)
+		val = dw_pcie_enable_ecrc(val);
+	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
 				 PCIE_ATU_ENABLE);
 
@@ -292,8 +334,10 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 			   lower_32_bits(pci_addr));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET,
 			   upper_32_bits(pci_addr));
-	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, type |
-			   PCIE_ATU_FUNC_NUM(func_no));
+	val = type | PCIE_ATU_FUNC_NUM(func_no);
+	if (pci->version == 0x490A)
+		val = dw_pcie_enable_ecrc(val);
+	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
 	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
 
 	/*
drivers/pci/controller/dwc/pcie-designware.h +1 -0
index 9d2f511f13fa..285c0ae364ae 100644
@@ -88,6 +88,7 @@
 #define PCIE_ATU_TYPE_IO		0x2
 #define PCIE_ATU_TYPE_CFG0		0x4
 #define PCIE_ATU_TYPE_CFG1		0x5
+#define PCIE_ATU_TD			BIT(8)
 #define PCIE_ATU_FUNC_NUM(pf)           ((pf) << 20)
 #define PCIE_ATU_CR2			0x908
 #define PCIE_ATU_ENABLE			BIT(31)
Signature · 1 line
2.17.1
maintainer
#3 lore raw
Quoted 21 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in one of the control registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare IP version 4.90a.

---
V2:
* Addressed Bjorn's comments

 drivers/pci/controller/dwc/pcie-designware.c | 52 ++++++++++++++++++--
 drivers/pci/controller/dwc/pcie-designware.h |  1 +
 2 files changed, 49 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-designware.c +0 -0
index c2dea8fc97c8..ec0d13ab6bad 100644

drivers/pci/controller/dwc/pcie-designware.c static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg, · line 228

Quoted 4 lines
drivers/pci/controller/dwc/pcie-designware.c +1 -0
@@ -225,6 +225,46 @@ static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg,
 	dw_pcie_writel_atu(pci, offset + reg, val);
 }
 
+static inline u32 dw_pcie_enable_ecrc(u32 val)

What is the reason to use inline here?

Best regards,
Jingoo Han

Quoted 82 lines
drivers/pci/controller/dwc/pcie-designware.c +47 -4
+{
+	/*
+	 * DesignWare core version 4.90A has this strange design issue
+	 * where the 'TD' bit in the Control register-1 of the ATU outbound
+	 * region acts like an override for the ECRC setting i.e. the presence
+	 * of TLP Digest(ECRC) in the outgoing TLPs is solely determined by
+	 * this bit. This is contrary to the PCIe spec which says that the
+	 * enablement of the ECRC is solely determined by the AER registers.
+	 *
+	 * Because of this, even when the ECRC is enabled through AER
+	 * registers, the transactions going through ATU won't have TLP Digest
+	 * as there is no way the AER sub-system could program the TD bit which
+	 * is specific to DesignWare core.
+	 *
+	 * The best way to handle this scenario is to program the TD bit
+	 * always. It affects only the traffic from root port to downstream
+	 * devices.
+	 *
+	 * At this point,
+	 * When ECRC is enabled in AER registers, everything works normally
+	 * When ECRC is NOT enabled in AER registers, then,
+	 * on Root Port:- TLP Digest (DWord size) gets appended to each packet
+	 *                even through it is not required. Since downstream
+	 *                TLPs are mostly for configuration accesses and BAR
+	 *                accesses, they are not in critical path and won't
+	 *                have much negative effect on the performance.
+	 * on End Point:- TLP Digest is received for some/all the packets coming
+	 *                from the root port. TLP Digest is ignored because,
+	 *                as per the PCIe Spec r5.0 v1.0 section 2.2.3
+	 *                "TLP Digest Rules", when an endpoint receives TLP
+	 *                Digest when its ECRC check functionality is disabled
+	 *                in AER registers, received TLP Digest is just ignored.
+	 * Since there is no issue or error reported either side, best way to
+	 * handle the scenario is to program TD bit by default.
+	 */
+
+	return val | PCIE_ATU_TD;
+}
+
 static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 					     int index, int type,
 					     u64 cpu_addr, u64 pci_addr,
@@ -245,8 +285,10 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
 				 lower_32_bits(pci_addr));
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
 				 upper_32_bits(pci_addr));
-	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1,
-				 type | PCIE_ATU_FUNC_NUM(func_no));
+	val = type | PCIE_ATU_FUNC_NUM(func_no);
+	if (pci->version == 0x490A)
+		val = dw_pcie_enable_ecrc(val);
+	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
 	dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
 				 PCIE_ATU_ENABLE);
 
@@ -292,8 +334,10 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
 			   lower_32_bits(pci_addr));
 	dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET,
 			   upper_32_bits(pci_addr));
-	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, type |
-			   PCIE_ATU_FUNC_NUM(func_no));
+	val = type | PCIE_ATU_FUNC_NUM(func_no);
+	if (pci->version == 0x490A)
+		val = dw_pcie_enable_ecrc(val);
+	dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
 	dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
 
 	/*
drivers/pci/controller/dwc/pcie-designware.h +1 -0
index 9d2f511f13fa..285c0ae364ae 100644
@@ -88,6 +88,7 @@
 #define PCIE_ATU_TYPE_IO		0x2
 #define PCIE_ATU_TYPE_CFG0		0x4
 #define PCIE_ATU_TYPE_CFG1		0x5
+#define PCIE_ATU_TD			BIT(8)
 #define PCIE_ATU_FUNC_NUM(pf)           ((pf) << 20)
 #define PCIE_ATU_CR2			0x908
 #define PCIE_ATU_ENABLE			BIT(31)
Signature · 1 line
2.17.1
#4 lore raw
Quoted 28 lines · Jingoo Han

External email: Use caution opening links or attachments

Quoted 24 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in one of the control registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare IP version 4.90a.

---
V2:
* Addressed Bjorn's comments

  drivers/pci/controller/dwc/pcie-designware.c | 52 ++++++++++++++++++--
  drivers/pci/controller/dwc/pcie-designware.h |  1 +
  2 files changed, 49 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-designware.c +0 -0
index c2dea8fc97c8..ec0d13ab6bad 100644
@@ -225,6 +225,46 @@ static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg,
       dw_pcie_writel_atu(pci, offset + reg, val);
  }
 
Quoted 3 lines

drivers/pci/controller/dwc/pcie-designware.c static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg, · line 228

Quoted 1 line
drivers/pci/controller/dwc/pcie-designware.c +1 -0
+static inline u32 dw_pcie_enable_ecrc(u32 val)

What is the reason to use inline here?

Actually, I wanted to move the programming part inside the respective APIs but then I wanted to give some details as well in comments so to avoid duplication, I came up with this function. But, I'm making it inline for better code optimization by compiler.

Quoted 88 lines

Best regards,
Jingoo Han

Quoted 82 lines
drivers/pci/controller/dwc/pcie-designware.c +47 -4
+{
+     /*
+      * DesignWare core version 4.90A has this strange design issue
+      * where the 'TD' bit in the Control register-1 of the ATU outbound
+      * region acts like an override for the ECRC setting i.e. the presence
+      * of TLP Digest(ECRC) in the outgoing TLPs is solely determined by
+      * this bit. This is contrary to the PCIe spec which says that the
+      * enablement of the ECRC is solely determined by the AER registers.
+      *
+      * Because of this, even when the ECRC is enabled through AER
+      * registers, the transactions going through ATU won't have TLP Digest
+      * as there is no way the AER sub-system could program the TD bit which
+      * is specific to DesignWare core.
+      *
+      * The best way to handle this scenario is to program the TD bit
+      * always. It affects only the traffic from root port to downstream
+      * devices.
+      *
+      * At this point,
+      * When ECRC is enabled in AER registers, everything works normally
+      * When ECRC is NOT enabled in AER registers, then,
+      * on Root Port:- TLP Digest (DWord size) gets appended to each packet
+      *                even through it is not required. Since downstream
+      *                TLPs are mostly for configuration accesses and BAR
+      *                accesses, they are not in critical path and won't
+      *                have much negative effect on the performance.
+      * on End Point:- TLP Digest is received for some/all the packets coming
+      *                from the root port. TLP Digest is ignored because,
+      *                as per the PCIe Spec r5.0 v1.0 section 2.2.3
+      *                "TLP Digest Rules", when an endpoint receives TLP
+      *                Digest when its ECRC check functionality is disabled
+      *                in AER registers, received TLP Digest is just ignored.
+      * Since there is no issue or error reported either side, best way to
+      * handle the scenario is to program TD bit by default.
+      */
+
+     return val | PCIE_ATU_TD;
+}
+
  static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
                                            int index, int type,
                                            u64 cpu_addr, u64 pci_addr,
@@ -245,8 +285,10 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no,
                                lower_32_bits(pci_addr));
       dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
                                upper_32_bits(pci_addr));
-     dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1,
-                              type | PCIE_ATU_FUNC_NUM(func_no));
+     val = type | PCIE_ATU_FUNC_NUM(func_no);
+     if (pci->version == 0x490A)
+             val = dw_pcie_enable_ecrc(val);
+     dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
       dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
                                PCIE_ATU_ENABLE);
 
@@ -292,8 +334,10 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no,
                          lower_32_bits(pci_addr));
       dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET,
                          upper_32_bits(pci_addr));
-     dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, type |
-                        PCIE_ATU_FUNC_NUM(func_no));
+     val = type | PCIE_ATU_FUNC_NUM(func_no);
+     if (pci->version == 0x490A)
+             val = dw_pcie_enable_ecrc(val);
+     dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
       dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
 
       /*
drivers/pci/controller/dwc/pcie-designware.h +1 -1
index 9d2f511f13fa..285c0ae364ae 100644
@@ -88,6 +88,7 @@
  #define PCIE_ATU_TYPE_IO             0x2
  #define PCIE_ATU_TYPE_CFG0           0x4
  #define PCIE_ATU_TYPE_CFG1           0x5
+#define PCIE_ATU_TD                  BIT(8)
  #define PCIE_ATU_FUNC_NUM(pf)           ((pf) << 20)
  #define PCIE_ATU_CR2                 0x908
  #define PCIE_ATU_ENABLE                      BIT(31)
--

2.17.1

#5 lore raw
Quoted 31 lines · Vidya Sagar
Quoted 28 lines · Jingoo Han

External email: Use caution opening links or attachments

Quoted 24 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in one of the control registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare IP version 4.90a.

---
V2:
* Addressed Bjorn's comments

  drivers/pci/controller/dwc/pcie-designware.c | 52 ++++++++++++++++++--
  drivers/pci/controller/dwc/pcie-designware.h |  1 +
  2 files changed, 49 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-designware.c +0 -0
index c2dea8fc97c8..ec0d13ab6bad 100644
@@ -225,6 +225,46 @@ static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg,
       dw_pcie_writel_atu(pci, offset + reg, val);
  }
 
Quoted 8 lines
Quoted 3 lines

drivers/pci/controller/dwc/pcie-designware.c static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg, · line 228

Quoted 1 line
drivers/pci/controller/dwc/pcie-designware.c +1 -0
+static inline u32 dw_pcie_enable_ecrc(u32 val)

What is the reason to use inline here?

Actually, I wanted to move the programming part inside the respective APIs but then I wanted to give some details as well in comments so to avoid duplication, I came up with this function. But, I'm making it inline for better code optimization by compiler.

I don't really care either way, but I'd be surprised if the compiler didn't inline this all by itself even without the explicit "inline".

Quoted 38 lines
Quoted 38 lines
Quoted 38 lines
drivers/pci/controller/dwc/pcie-designware.c +38 -0
+{
+     /*
+      * DesignWare core version 4.90A has this strange design issue
+      * where the 'TD' bit in the Control register-1 of the ATU outbound
+      * region acts like an override for the ECRC setting i.e. the presence
+      * of TLP Digest(ECRC) in the outgoing TLPs is solely determined by
+      * this bit. This is contrary to the PCIe spec which says that the
+      * enablement of the ECRC is solely determined by the AER registers.
+      *
+      * Because of this, even when the ECRC is enabled through AER
+      * registers, the transactions going through ATU won't have TLP Digest
+      * as there is no way the AER sub-system could program the TD bit which
+      * is specific to DesignWare core.
+      *
+      * The best way to handle this scenario is to program the TD bit
+      * always. It affects only the traffic from root port to downstream
+      * devices.
+      *
+      * At this point,
+      * When ECRC is enabled in AER registers, everything works normally
+      * When ECRC is NOT enabled in AER registers, then,
+      * on Root Port:- TLP Digest (DWord size) gets appended to each packet
+      *                even through it is not required. Since downstream
+      *                TLPs are mostly for configuration accesses and BAR
+      *                accesses, they are not in critical path and won't
+      *                have much negative effect on the performance.
+      * on End Point:- TLP Digest is received for some/all the packets coming
+      *                from the root port. TLP Digest is ignored because,
+      *                as per the PCIe Spec r5.0 v1.0 section 2.2.3
+      *                "TLP Digest Rules", when an endpoint receives TLP
+      *                Digest when its ECRC check functionality is disabled
+      *                in AER registers, received TLP Digest is just ignored.
+      * Since there is no issue or error reported either side, best way to
+      * handle the scenario is to program TD bit by default.
+      */
+
+     return val | PCIE_ATU_TD;
+}
#6 lore raw
Quoted 39 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 35 lines · Vidya Sagar
Quoted 31 lines · Jingoo Han

External email: Use caution opening links or attachments

Quoted 21 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in one of the control registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare IP version 4.90a.

---
V2:
* Addressed Bjorn's comments

   drivers/pci/controller/dwc/pcie-designware.c | 52 ++++++++++++++++++--
   drivers/pci/controller/dwc/pcie-designware.h |  1 +
   2 files changed, 49 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-designware.c +0 -0
index c2dea8fc97c8..ec0d13ab6bad 100644

drivers/pci/controller/dwc/pcie-designware.c static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg, · line 228

Quoted 4 lines
drivers/pci/controller/dwc/pcie-designware.c +1 -0
@@ -225,6 +225,46 @@ static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg,
        dw_pcie_writel_atu(pci, offset + reg, val);
   }
 
+static inline u32 dw_pcie_enable_ecrc(u32 val)

What is the reason to use inline here?

Quoted 7 lines
Quoted 4 lines

Actually, I wanted to move the programming part inside the respective APIs but then I wanted to give some details as well in comments so to avoid duplication, I came up with this function. But, I'm making it inline for better code optimization by compiler.

I don't really care either way, but I'd be surprised if the compiler didn't inline this all by itself even without the explicit "inline".

I just checked it and you are right that compiler is indeed inlining it without explicitly mentioning 'inline'. I hope it is ok to leave it that way.

Quoted 39 lines
Quoted 39 lines
Quoted 39 lines
Quoted 39 lines
drivers/pci/controller/dwc/pcie-designware.c +38 -0
+{
+     /*
+      * DesignWare core version 4.90A has this strange design issue
+      * where the 'TD' bit in the Control register-1 of the ATU outbound
+      * region acts like an override for the ECRC setting i.e. the presence
+      * of TLP Digest(ECRC) in the outgoing TLPs is solely determined by
+      * this bit. This is contrary to the PCIe spec which says that the
+      * enablement of the ECRC is solely determined by the AER registers.
+      *
+      * Because of this, even when the ECRC is enabled through AER
+      * registers, the transactions going through ATU won't have TLP Digest
+      * as there is no way the AER sub-system could program the TD bit which
+      * is specific to DesignWare core.
+      *
+      * The best way to handle this scenario is to program the TD bit
+      * always. It affects only the traffic from root port to downstream
+      * devices.
+      *
+      * At this point,
+      * When ECRC is enabled in AER registers, everything works normally
+      * When ECRC is NOT enabled in AER registers, then,
+      * on Root Port:- TLP Digest (DWord size) gets appended to each packet
+      *                even through it is not required. Since downstream
+      *                TLPs are mostly for configuration accesses and BAR
+      *                accesses, they are not in critical path and won't
+      *                have much negative effect on the performance.
+      * on End Point:- TLP Digest is received for some/all the packets coming
+      *                from the root port. TLP Digest is ignored because,
+      *                as per the PCIe Spec r5.0 v1.0 section 2.2.3
+      *                "TLP Digest Rules", when an endpoint receives TLP
+      *                Digest when its ECRC check functionality is disabled
+      *                in AER registers, received TLP Digest is just ignored.
+      * Since there is no issue or error reported either side, best way to
+      * handle the scenario is to program TD bit by default.
+      */
+
+     return val | PCIE_ATU_TD;
+}
#7 lore raw

Hi Bjorn,
Please let me know if this patch needs any further modifications

Thanks,
Vidya Sagar

Quoted 114 lines · Vidya Sagar

External email: Use caution opening links or attachments

Quoted 41 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 37 lines · Vidya Sagar
Quoted 33 lines · Jingoo Han

External email: Use caution opening links or attachments

Quoted 22 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in one of the control registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare IP version 4.90a.

---
V2:
* Addressed Bjorn's comments

   drivers/pci/controller/dwc/pcie-designware.c | 52
++++++++++++++++++--
   drivers/pci/controller/dwc/pcie-designware.h |  1 +

   2 files changed, 49 insertions(+), 4 deletions(-)
drivers/pci/controller/dwc/pcie-designware.c +0 -0
index c2dea8fc97c8..ec0d13ab6bad 100644

drivers/pci/controller/dwc/pcie-designware.c static void dw_pcie_writel_ob_unroll(struct · line 229

Quoted 5 lines
drivers/pci/controller/dwc/pcie-designware.c +1 -0
@@ -225,6 +225,46 @@ static void dw_pcie_writel_ob_unroll(struct 
 dw_pcie *pci, u32 index, u32 reg,
         dw_pcie_writel_atu(pci, offset + reg, val);
    }
 
+static inline u32 dw_pcie_enable_ecrc(u32 val)

What is the reason to use inline here?

Quoted 7 lines
Quoted 4 lines

Actually, I wanted to move the programming part inside the respective APIs but then I wanted to give some details as well in comments so to avoid duplication, I came up with this function. But, I'm making it inline for better code optimization by compiler.

I don't really care either way, but I'd be surprised if the compiler didn't inline this all by itself even without the explicit "inline".

I just checked it and you are right that compiler is indeed inlining it without explicitly mentioning 'inline'.
I hope it is ok to leave it that way.

Quoted 58 lines
Quoted 58 lines
Quoted 58 lines
Quoted 58 lines
drivers/pci/controller/dwc/pcie-designware.c +4 -0
+{
+     /*
+      * DesignWare core version 4.90A has this strange design issue
+      * where the 'TD' bit in the Control register-1 of the ATU 

outbound
+      * region acts like an override for the ECRC setting i.e. the presence
+      * of TLP Digest(ECRC) in the outgoing TLPs is solely determined by
+      * this bit. This is contrary to the PCIe spec which says that the
+      * enablement of the ECRC is solely determined by the AER registers.
+      *
+      * Because of this, even when the ECRC is enabled through AER +      * registers, the transactions going through ATU won't have TLP Digest
+      * as there is no way the AER sub-system could program the TD bit which
+      * is specific to DesignWare core.
+      *
+      * The best way to handle this scenario is to program the TD bit +      * always. It affects only the traffic from root port to downstream
+      * devices.
+      *
+      * At this point,
+      * When ECRC is enabled in AER registers, everything works normally
+      * When ECRC is NOT enabled in AER registers, then,
+      * on Root Port:- TLP Digest (DWord size) gets appended to each packet
+      *                even through it is not required. Since downstream
+      *                TLPs are mostly for configuration accesses and BAR
+      *                accesses, they are not in critical path and won't
+      *                have much negative effect on the performance. +      * on End Point:- TLP Digest is received for some/all the packets coming
+      *                from the root port. TLP Digest is ignored because,
+      *                as per the PCIe Spec r5.0 v1.0 section 2.2.3 +      *                "TLP Digest Rules", when an endpoint receives TLP
+      *                Digest when its ECRC check functionality is disabled
+      *                in AER registers, received TLP Digest is just ignored.
+      * Since there is no issue or error reported either side, best way to
+      * handle the scenario is to program TD bit by default. +      */

drivers/pci/controller/dwc/pcie-designware.c +3 -0
+
+     return val | PCIE_ATU_TD;
+}
#8 lore raw
Quoted 2 lines · Vidya Sagar

Hi Bjorn,
Please let me know if this patch needs any further modifications

I'm fine with it, but of course Lorenzo will take care of it.

Quoted 121 lines · Vidya Sagar
Quoted 120 lines

External email: Use caution opening links or attachments

Quoted 46 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 42 lines · Vidya Sagar
Quoted 38 lines · Jingoo Han

External email: Use caution opening links or attachments

Quoted 26 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in
one of the control
registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare
IP version 4.90a.

---
V2:
* Addressed Bjorn's comments

   drivers/pci/controller/dwc/pcie-designware.c | 52
++++++++++++++++++--
   drivers/pci/controller/dwc/pcie-designware.h |  1 +

   2 files changed, 49 insertions(+), 4 deletions(-)

diff --git
a/drivers/pci/controller/dwc/pcie-designware.c
b/drivers/pci/controller/dwc/pcie-designware.c
index c2dea8fc97c8..ec0d13ab6bad 100644

drivers/pci/controller/dwc/pcie-designware.c static void · line 230

Quoted 6 lines
drivers/pci/controller/dwc/pcie-designware.c +1 -0
@@ -225,6 +225,46 @@ static void
 dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index,
 u32 reg,
         dw_pcie_writel_atu(pci, offset + reg, val);
    }
 
+static inline u32 dw_pcie_enable_ecrc(u32 val)

What is the reason to use inline here?

Quoted 8 lines
Quoted 5 lines

Actually, I wanted to move the programming part inside the respective APIs
but then I wanted to give some details as well in comments so to avoid duplication, I came up with this function. But, I'm making it inline for better code optimization by compiler.

I don't really care either way, but I'd be surprised if the compiler didn't inline this all by itself even without the explicit "inline".

I just checked it and you are right that compiler is indeed inlining it without explicitly mentioning 'inline'.
I hope it is ok to leave it that way.

Quoted 58 lines
Quoted 58 lines
Quoted 58 lines
Quoted 58 lines
drivers/pci/controller/dwc/pcie-designware.c +4 -0
+{
+     /*
+      * DesignWare core version 4.90A has this strange design issue
+      * where the 'TD' bit in the Control register-1 of

the ATU outbound
+      * region acts like an override for the ECRC
setting i.e. the presence
+      * of TLP Digest(ECRC) in the outgoing TLPs is
solely determined by
+      * this bit. This is contrary to the PCIe spec
which says that the
+      * enablement of the ECRC is solely determined by
the AER registers.
+      *
+      * Because of this, even when the ECRC is enabled through AER +      * registers, the transactions going through ATU won't have TLP Digest
+      * as there is no way the AER sub-system could program the TD bit which
+      * is specific to DesignWare core.
+      *
+      * The best way to handle this scenario is to program the TD bit +      * always. It affects only the traffic from root
port to downstream
+      * devices.
+      *
+      * At this point,
+      * When ECRC is enabled in AER registers,
everything works normally
+      * When ECRC is NOT enabled in AER registers, then,
+      * on Root Port:- TLP Digest (DWord size) gets appended to each packet
+      *                even through it is not required. Since downstream
+      *                TLPs are mostly for
configuration accesses and BAR
+      *                accesses, they are not in
critical path and won't
+      *                have much negative effect on the performance. +      * on End Point:- TLP Digest is received for
some/all the packets coming
+      *                from the root port. TLP Digest
is ignored because,
+      *                as per the PCIe Spec r5.0 v1.0 section 2.2.3 +      *                "TLP Digest Rules", when an endpoint receives TLP
+      *                Digest when its ECRC check functionality is disabled
+      *                in AER registers, received TLP Digest is just ignored.
+      * Since there is no issue or error reported
either side, best way to
+      * handle the scenario is to program TD bit by default. +      */

drivers/pci/controller/dwc/pcie-designware.c +3 -0
+
+     return val | PCIE_ATU_TD;
+}
#9 lore raw
Quoted 8 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 2 lines · Vidya Sagar

Hi Bjorn,
Please let me know if this patch needs any further modifications

I'm fine with it, but of course Lorenzo will take care of it.

Thanks Bjorn.

Hi Lorenzo,
Please let me know if you have any comments for this patch.

Thanks,
Vidya Sagar

Quoted 122 lines
Quoted 122 lines · Vidya Sagar
Quoted 120 lines

External email: Use caution opening links or attachments

Quoted 46 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 42 lines · Vidya Sagar
Quoted 38 lines · Jingoo Han

External email: Use caution opening links or attachments

Quoted 26 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in
one of the control
registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare
IP version 4.90a.

---
V2:
* Addressed Bjorn's comments

    drivers/pci/controller/dwc/pcie-designware.c | 52
++++++++++++++++++--
    drivers/pci/controller/dwc/pcie-designware.h |  1 +
    2 files changed, 49 insertions(+), 4 deletions(-)

diff --git
a/drivers/pci/controller/dwc/pcie-designware.c
b/drivers/pci/controller/dwc/pcie-designware.c
index c2dea8fc97c8..ec0d13ab6bad 100644

drivers/pci/controller/dwc/pcie-designware.c static void · line 230

Quoted 6 lines
drivers/pci/controller/dwc/pcie-designware.c +1 -0
@@ -225,6 +225,46 @@ static void
 dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index,
 u32 reg,
         dw_pcie_writel_atu(pci, offset + reg, val);
    }
 
+static inline u32 dw_pcie_enable_ecrc(u32 val)

What is the reason to use inline here?

Quoted 8 lines
Quoted 5 lines

Actually, I wanted to move the programming part inside the respective APIs
but then I wanted to give some details as well in comments so to avoid duplication, I came up with this function. But, I'm making it inline for better code optimization by compiler.

I don't really care either way, but I'd be surprised if the compiler didn't inline this all by itself even without the explicit "inline".

I just checked it and you are right that compiler is indeed inlining it without explicitly mentioning 'inline'.
I hope it is ok to leave it that way.

Quoted 58 lines
Quoted 58 lines
Quoted 58 lines
Quoted 58 lines
drivers/pci/controller/dwc/pcie-designware.c +4 -0
+{
+     /*
+      * DesignWare core version 4.90A has this strange design issue
+      * where the 'TD' bit in the Control register-1 of

the ATU outbound
+ * region acts like an override for the ECRC
setting i.e. the presence
+ * of TLP Digest(ECRC) in the outgoing TLPs is
solely determined by
+ * this bit. This is contrary to the PCIe spec
which says that the
+ * enablement of the ECRC is solely determined by
the AER registers.

drivers/pci/controller/dwc/pcie-designware.c +3 -0
+      *
+      * Because of this, even when the ECRC is enabled through AER
+      * registers, the transactions going through ATU

won't have TLP Digest
+ * as there is no way the AER sub-system could
program the TD bit which

drivers/pci/controller/dwc/pcie-designware.c +4 -0
+      * is specific to DesignWare core.
+      *
+      * The best way to handle this scenario is to program the TD bit
+      * always. It affects only the traffic from root

port to downstream

drivers/pci/controller/dwc/pcie-designware.c +4 -0
+      * devices.
+      *
+      * At this point,
+      * When ECRC is enabled in AER registers,

everything works normally

drivers/pci/controller/dwc/pcie-designware.c +2 -0
+      * When ECRC is NOT enabled in AER registers, then,
+      * on Root Port:- TLP Digest (DWord size) gets

appended to each packet
+ * even through it is not required.
Since downstream
+ * TLPs are mostly for
configuration accesses and BAR
+ * accesses, they are not in
critical path and won't

drivers/pci/controller/dwc/pcie-designware.c +2 -0
+      *                have much negative effect on the performance.
+      * on End Point:- TLP Digest is received for

some/all the packets coming
+ * from the root port. TLP Digest
is ignored because,

drivers/pci/controller/dwc/pcie-designware.c +2 -0
+      *                as per the PCIe Spec r5.0 v1.0 section 2.2.3
+      *                "TLP Digest Rules", when an

endpoint receives TLP
+ * Digest when its ECRC check
functionality is disabled
+ * in AER registers, received TLP
Digest is just ignored.
+ * Since there is no issue or error reported
either side, best way to

drivers/pci/controller/dwc/pcie-designware.c +5 -0
+      * handle the scenario is to program TD bit by default.
+      */
+
+     return val | PCIE_ATU_TD;
+}
#10 lore raw

Hi Lorenzo,
Apologies to bug you, but wondering if you have any further comments on this patch that I need to take care of?

Thanks,
Vidya Sagar

Quoted 145 lines · Vidya Sagar
Quoted 8 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 2 lines · Vidya Sagar

Hi Bjorn,
Please let me know if this patch needs any further modifications

I'm fine with it, but of course Lorenzo will take care of it.

Thanks Bjorn.

Hi Lorenzo,
Please let me know if you have any comments for this patch.

Thanks,
Vidya Sagar

Quoted 126 lines
Quoted 126 lines · Vidya Sagar
Quoted 124 lines

External email: Use caution opening links or attachments

Quoted 46 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 42 lines · Vidya Sagar
Quoted 38 lines · Jingoo Han

External email: Use caution opening links or attachments

Quoted 26 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in
one of the control
registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare
IP version 4.90a.

---
V2:
* Addressed Bjorn's comments

    drivers/pci/controller/dwc/pcie-designware.c | 52
++++++++++++++++++--
    drivers/pci/controller/dwc/pcie-designware.h |  1 +

    2 files changed, 49 insertions(+), 4 deletions(-)

diff --git
a/drivers/pci/controller/dwc/pcie-designware.c
b/drivers/pci/controller/dwc/pcie-designware.c
index c2dea8fc97c8..ec0d13ab6bad 100644

drivers/pci/controller/dwc/pcie-designware.c static void · line 230

Quoted 6 lines
drivers/pci/controller/dwc/pcie-designware.c +1 -0
@@ -225,6 +225,46 @@ static void
 dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index,
 u32 reg,
          dw_pcie_writel_atu(pci, offset + reg, val);
     }
 
+static inline u32 dw_pcie_enable_ecrc(u32 val)

What is the reason to use inline here?

Quoted 8 lines
Quoted 5 lines

Actually, I wanted to move the programming part inside the respective APIs
but then I wanted to give some details as well in comments so to avoid duplication, I came up with this function. But, I'm making it inline for better code optimization by compiler.

I don't really care either way, but I'd be surprised if the compiler didn't inline this all by itself even without the explicit "inline".

I just checked it and you are right that compiler is indeed inlining it without explicitly mentioning 'inline'.
I hope it is ok to leave it that way.

Quoted 62 lines
Quoted 62 lines
Quoted 62 lines
Quoted 62 lines
drivers/pci/controller/dwc/pcie-designware.c +3 -0
+{
+     /*
+      * DesignWare core version 4.90A has this strange design 

issue
+      * where the 'TD' bit in the Control register-1 of the ATU outbound
+      * region acts like an override for the ECRC setting i.e. the presence
+      * of TLP Digest(ECRC) in the outgoing TLPs is solely determined by
+      * this bit. This is contrary to the PCIe spec which says that the
+      * enablement of the ECRC is solely determined by the AER registers.
+      *
+      * Because of this, even when the ECRC is enabled through AER +      * registers, the transactions going through ATU won't have TLP Digest
+      * as there is no way the AER sub-system could program the TD bit which
+      * is specific to DesignWare core.
+      *
+      * The best way to handle this scenario is to program the TD bit
+      * always. It affects only the traffic from root port to downstream
+      * devices.
+      *
+      * At this point,
+      * When ECRC is enabled in AER registers, everything works normally
+      * When ECRC is NOT enabled in AER registers, then, +      * on Root Port:- TLP Digest (DWord size) gets appended to each packet
+      *                even through it is not required. Since downstream
+      *                TLPs are mostly for configuration accesses and BAR
+      *                accesses, they are not in critical path and won't
+      *                have much negative effect on the performance.
+      * on End Point:- TLP Digest is received for some/all the packets coming
+      *                from the root port. TLP Digest is ignored because,
+      *                as per the PCIe Spec r5.0 v1.0 section 2.2.3
+      *                "TLP Digest Rules", when an endpoint receives TLP
+      *                Digest when its ECRC check functionality is disabled
+      *                in AER registers, received TLP Digest is just ignored.
+      * Since there is no issue or error reported either side, best way to
+      * handle the scenario is to program TD bit by default. +      */

drivers/pci/controller/dwc/pcie-designware.c +3 -0
+
+     return val | PCIE_ATU_TD;
+}
reviewer
#11 lore raw
Quoted 4 lines · Vidya Sagar

Hi Lorenzo,
Apologies to bug you, but wondering if you have any further comments on this patch that I need to take care of?

You can check the status of your patches in Patchwork:

https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/

If it's in 'New' state and delegated to Lorenzo or Bjorn, it's in their queue. You can shorten the queue by reviewing stuff in front of you. :)

Rob

Quoted 153 lines

Thanks,
Vidya Sagar

Quoted 148 lines · Vidya Sagar
Quoted 8 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 2 lines · Vidya Sagar

Hi Bjorn,
Please let me know if this patch needs any further modifications

I'm fine with it, but of course Lorenzo will take care of it.

Thanks Bjorn.

Hi Lorenzo,
Please let me know if you have any comments for this patch.

Thanks,
Vidya Sagar

Quoted 129 lines
Quoted 129 lines · Vidya Sagar
Quoted 127 lines

External email: Use caution opening links or attachments

Quoted 55 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 43 lines · Vidya Sagar
Quoted 36 lines · Jingoo Han

External email: Use caution opening links or attachments

Quoted 32 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in one of the control
registers of ATU. This bit also needs to be programmed for proper ECRC
functionality. This is currently identified as an issue with DesignWare
IP version 4.90a.

---
V2:
* Addressed Bjorn's comments

    drivers/pci/controller/dwc/pcie-designware.c | 52
++++++++++++++++++--
    drivers/pci/controller/dwc/pcie-designware.h |  1 +
    2 files changed, 49 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c index c2dea8fc97c8..ec0d13ab6bad 100644

drivers/pci/controller/dwc/pcie-designware.c +0 -0
@@ -225,6 +225,46 @@ static void
 dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index,
 u32 reg,
         dw_pcie_writel_atu(pci, offset + reg, val);
    }
 
Quoted 3 lines

drivers/pci/controller/dwc/pcie-designware.c static void · line 230

Quoted 1 line
drivers/pci/controller/dwc/pcie-designware.c +1 -0
+static inline u32 dw_pcie_enable_ecrc(u32 val)

What is the reason to use inline here?

Quoted 7 lines

Actually, I wanted to move the programming part inside the respective APIs
but then I wanted to give some details as well in comments so to avoid
duplication, I came up with this function. But, I'm making it inline for
better code optimization by compiler.

Quoted 2 lines

I don't really care either way, but I'd be surprised if the compiler didn't inline this all by itself even without the explicit "inline".

I just checked it and you are right that compiler is indeed inlining it without explicitly mentioning 'inline'.
I hope it is ok to leave it that way.

Quoted 62 lines
Quoted 62 lines
Quoted 62 lines
Quoted 62 lines
drivers/pci/controller/dwc/pcie-designware.c +3 -0
+{
+     /*
+      * DesignWare core version 4.90A has this strange design

issue
+ * where the 'TD' bit in the Control register-1 of
the ATU outbound
+ * region acts like an override for the ECRC
setting i.e. the presence
+ * of TLP Digest(ECRC) in the outgoing TLPs is
solely determined by
+ * this bit. This is contrary to the PCIe spec
which says that the
+ * enablement of the ECRC is solely determined by
the AER registers.

drivers/pci/controller/dwc/pcie-designware.c +3 -0
+      *
+      * Because of this, even when the ECRC is enabled through AER
+      * registers, the transactions going through ATU

won't have TLP Digest
+ * as there is no way the AER sub-system could
program the TD bit which

drivers/pci/controller/dwc/pcie-designware.c +3 -0
+      * is specific to DesignWare core.
+      *
+      * The best way to handle this scenario is to program the

TD bit
+ * always. It affects only the traffic from root
port to downstream

drivers/pci/controller/dwc/pcie-designware.c +4 -0
+      * devices.
+      *
+      * At this point,
+      * When ECRC is enabled in AER registers,

everything works normally

drivers/pci/controller/dwc/pcie-designware.c +2 -0
+      * When ECRC is NOT enabled in AER registers, then,
+      * on Root Port:- TLP Digest (DWord size) gets

appended to each packet
+ * even through it is not required.
Since downstream
+ * TLPs are mostly for
configuration accesses and BAR
+ * accesses, they are not in
critical path and won't
+ * have much negative effect on the
performance.
+ * on End Point:- TLP Digest is received for
some/all the packets coming
+ * from the root port. TLP Digest
is ignored because,
+ * as per the PCIe Spec r5.0 v1.0 section
2.2.3
+ * "TLP Digest Rules", when an
endpoint receives TLP
+ * Digest when its ECRC check
functionality is disabled
+ * in AER registers, received TLP
Digest is just ignored.
+ * Since there is no issue or error reported
either side, best way to

drivers/pci/controller/dwc/pcie-designware.c +5 -0
+      * handle the scenario is to program TD bit by default.
+      */
+
+     return val | PCIE_ATU_TD;
+}
#12 lore raw
Quoted 6 lines · Rob Herring
Quoted 4 lines · Vidya Sagar

Hi Lorenzo,
Apologies to bug you, but wondering if you have any further comments on this patch that I need to take care of?

Quoted 7 lines

You can check the status of your patches in Patchwork:

https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/

If it's in 'New' state and delegated to Lorenzo or Bjorn, it's in their queue. You can shorten the queue by reviewing stuff in front of you. :)

Yes that's right. There are a couple of patches pending ahead, if this one can be rebased against my pci/dwc branch and resent I can apply it.

Thanks,
Lorenzo

Quoted 156 lines

Rob

Quoted 153 lines

Thanks,
Vidya Sagar

Quoted 148 lines · Vidya Sagar
Quoted 8 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 2 lines · Vidya Sagar

Hi Bjorn,
Please let me know if this patch needs any further modifications

I'm fine with it, but of course Lorenzo will take care of it.

Thanks Bjorn.

Hi Lorenzo,
Please let me know if you have any comments for this patch.

Thanks,
Vidya Sagar

Quoted 129 lines
Quoted 129 lines · Vidya Sagar
Quoted 127 lines

External email: Use caution opening links or attachments

Quoted 55 lines · Bjorn Helgaas

External email: Use caution opening links or attachments

Quoted 43 lines · Vidya Sagar
Quoted 36 lines · Jingoo Han

External email: Use caution opening links or attachments

Quoted 32 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in one of the control
registers of ATU. This bit also needs to be programmed for proper ECRC
functionality. This is currently identified as an issue with DesignWare
IP version 4.90a.

---
V2:
* Addressed Bjorn's comments

    drivers/pci/controller/dwc/pcie-designware.c | 52
++++++++++++++++++--
    drivers/pci/controller/dwc/pcie-designware.h |  1 +
    2 files changed, 49 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c index c2dea8fc97c8..ec0d13ab6bad 100644

drivers/pci/controller/dwc/pcie-designware.c +0 -0
@@ -225,6 +225,46 @@ static void
 dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index,
 u32 reg,
         dw_pcie_writel_atu(pci, offset + reg, val);
    }
 
Quoted 3 lines

drivers/pci/controller/dwc/pcie-designware.c static void · line 230

Quoted 1 line
drivers/pci/controller/dwc/pcie-designware.c +1 -0
+static inline u32 dw_pcie_enable_ecrc(u32 val)

What is the reason to use inline here?

Quoted 7 lines

Actually, I wanted to move the programming part inside the respective APIs
but then I wanted to give some details as well in comments so to avoid
duplication, I came up with this function. But, I'm making it inline for
better code optimization by compiler.

Quoted 2 lines

I don't really care either way, but I'd be surprised if the compiler didn't inline this all by itself even without the explicit "inline".

I just checked it and you are right that compiler is indeed inlining it without explicitly mentioning 'inline'.
I hope it is ok to leave it that way.

Quoted 62 lines
Quoted 62 lines
Quoted 62 lines
Quoted 62 lines
drivers/pci/controller/dwc/pcie-designware.c +3 -0
+{
+     /*
+      * DesignWare core version 4.90A has this strange design

issue
+ * where the 'TD' bit in the Control register-1 of
the ATU outbound
+ * region acts like an override for the ECRC
setting i.e. the presence
+ * of TLP Digest(ECRC) in the outgoing TLPs is
solely determined by
+ * this bit. This is contrary to the PCIe spec
which says that the
+ * enablement of the ECRC is solely determined by
the AER registers.

drivers/pci/controller/dwc/pcie-designware.c +3 -0
+      *
+      * Because of this, even when the ECRC is enabled through AER
+      * registers, the transactions going through ATU

won't have TLP Digest
+ * as there is no way the AER sub-system could
program the TD bit which

drivers/pci/controller/dwc/pcie-designware.c +3 -0
+      * is specific to DesignWare core.
+      *
+      * The best way to handle this scenario is to program the

TD bit
+ * always. It affects only the traffic from root
port to downstream

drivers/pci/controller/dwc/pcie-designware.c +4 -0
+      * devices.
+      *
+      * At this point,
+      * When ECRC is enabled in AER registers,

everything works normally

drivers/pci/controller/dwc/pcie-designware.c +2 -0
+      * When ECRC is NOT enabled in AER registers, then,
+      * on Root Port:- TLP Digest (DWord size) gets

appended to each packet
+ * even through it is not required.
Since downstream
+ * TLPs are mostly for
configuration accesses and BAR
+ * accesses, they are not in
critical path and won't
+ * have much negative effect on the
performance.
+ * on End Point:- TLP Digest is received for
some/all the packets coming
+ * from the root port. TLP Digest
is ignored because,
+ * as per the PCIe Spec r5.0 v1.0 section
2.2.3
+ * "TLP Digest Rules", when an
endpoint receives TLP
+ * Digest when its ECRC check
functionality is disabled
+ * in AER registers, received TLP
Digest is just ignored.
+ * Since there is no issue or error reported
either side, best way to

drivers/pci/controller/dwc/pcie-designware.c +5 -0
+      * handle the scenario is to program TD bit by default.
+      */
+
+     return val | PCIE_ATU_TD;
+}
#13 lore raw
In reply to Rob Herring
Quoted 6 lines · Rob Herring
Quoted 4 lines · Vidya Sagar

Hi Lorenzo,
Apologies to bug you, but wondering if you have any further comments on this patch that I need to take care of?

Quoted 7 lines

You can check the status of your patches in Patchwork:

https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/

If it's in 'New' state and delegated to Lorenzo or Bjorn, it's in their queue. You can shorten the queue by reviewing stuff in front of you. :)

Thanks for pointing this out, this is exactly right. I *love* it when people help review things. Obviously it saves me time, and often it raises questions I would have missed.

Even "trivial" things like spelling, grammar, whitespace, subject line formats, etc. help because they are distractions to me and I will comment on them or spend time fixing them myself. That doesn't mean "repost immediately after somebody points out a typo"; it means "collect feedback for a week or so, fix everything, *then* repost."

This is an old but still relevant list of some of my OCD tendencies that take no special expertise to review for:
https://lore.kernel.org/linux-pci/[email protected]/

reviewer
#14 lore raw
Quoted 12 lines · Bjorn Helgaas
Quoted 10 lines · Rob Herring
Quoted 4 lines · Vidya Sagar

Hi Lorenzo,
Apologies to bug you, but wondering if you have any further comments on this patch that I need to take care of?

You can check the status of your patches in Patchwork:

https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/

Quoted 7 lines
Quoted 3 lines

If it's in 'New' state and delegated to Lorenzo or Bjorn, it's in their queue. You can shorten the queue by reviewing stuff in front of you. :)

Thanks for pointing this out, this is exactly right. I *love* it when people help review things. Obviously it saves me time, and often it raises questions I would have missed.

I can't take credit, this was actually someone's strategy for getting their patches reviewed I heard about one time.

Quoted 9 lines

Even "trivial" things like spelling, grammar, whitespace, subject line formats, etc. help because they are distractions to me and I will comment on them or spend time fixing them myself. That doesn't mean "repost immediately after somebody points out a typo"; it means "collect feedback for a week or so, fix everything, *then* repost."

This is an old but still relevant list of some of my OCD tendencies that take no special expertise to review for:
https://lore.kernel.org/linux-pci/[email protected]/

It's been on my wish list for checkpatch.pl do something like: "Commit subject should begin $(git log --oneline $files | cut -d':' -f-3 | sort | uniq -c | sort -nr | head -1)". Or maybe a git commit template to fill in subject prefix and Cc list. The hard part is of course all the corner cases such as touching multiple directories/files.

Rob

reviewer Reviewed-by on v1 1/1
#15 lore raw
In reply to Vidya Sagar
Quoted 11 lines · Vidya Sagar

DesignWare core has a TLP digest (TD) override bit in one of the control registers of ATU. This bit also needs to be programmed for proper ECRC functionality. This is currently identified as an issue with DesignWare IP version 4.90a.

---
 drivers/pci/controller/dwc/pcie-designware.c | 50 ++++++++++++++++++--
 drivers/pci/controller/dwc/pcie-designware.h |  1 +
 2 files changed, 47 insertions(+), 4 deletions(-)