Skip to contentSkip to conversation

PCI: replace dublicated MRRS limit quirks

PATCH In review Artem Lapkin 7 participants 18 messages Jun 19, 2021 – Jul 12, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover PCI: replace dublicated MRRS limit quirks

Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
* drivers/pci/controller/dwc/pci-keystone.c
* drivers/pci/controller/pci-loongson.c

Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any functionality changes by one mrrs_limit_quirk

Added DesignWare PCI controller which need same quirk for
* drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3)

This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic) with HDMI scrambled picture and nvme devices at intensive writing...

come from:
* https://lore.kernel.org/linux-pci/[email protected]/

Artem Lapkin (4):
PCI: move Keystone and Loongson device IDs to pci_ids
PCI: core: quirks: add mrrs_limit_quirk
PCI: keystone move mrrs quirk to core
PCI: loongson move mrrs quirk to core

Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/4 PCI: move Keystone and Loongson device IDs to pci_ids

Moved from
* drivers/pci/controller/dwc/pci-keystone.c
* drivers/pci/controller/pci-loongson.c

---
 include/linux/pci_ids.h | 7 +++++++
 1 file changed, 7 insertions(+)
include/linux/pci_ids.h +7 -0
index 4c3fa5293d76..e19d224bbca8 100644
@@ -151,6 +151,9 @@
 /* Vendors and devices.  Sort key: vendor first, device next. */
 
 #define PCI_VENDOR_ID_LOONGSON		0x0014
+#define PCI_DEVICE_ID_LOONGSON_PCIE_PORT_0	0x7a09
+#define PCI_DEVICE_ID_LOONGSON_PCIE_PORT_1	0x7a19
+#define PCI_DEVICE_ID_LOONGSON_PCIE_PORT_2	0x7a29
 
 #define PCI_VENDOR_ID_TTTECH		0x0357
 #define PCI_DEVICE_ID_TTTECH_MC322	0x000a
@@ -885,6 +888,10 @@
 #define PCI_DEVICE_ID_TI_J721E		0xb00d
 #define PCI_DEVICE_ID_TI_DRA74x		0xb500
 #define PCI_DEVICE_ID_TI_DRA72x		0xb501
+#define PCI_DEVICE_ID_TI_RC_K2HK	0xb008
+#define PCI_DEVICE_ID_TI_RC_K2E		0xb009
+#define PCI_DEVICE_ID_TI_RC_K2L		0xb00a
+#define PCI_DEVICE_ID_TI_RC_K2G		0xb00b
 
 #define PCI_VENDOR_ID_SONY		0x104d
 
Signature · 1 line
2.25.1
#2 lore raw

PATCH 2/4 PCI: core: quirks: add mrrs_limit_quirk

In reply to Artem Lapkin

Prepare new MRRS limit quirk which can replace dublicated functionality for some controllers from Loongson, Keystone, DesignWare

---
 drivers/pci/quirks.c | 54 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)
drivers/pci/quirks.c +54 -0
index 653660e3b..73344ec71 100644
@@ -5612,3 +5612,57 @@ static void apex_pci_fixup_class(struct pci_dev *pdev)
 }
 DECLARE_PCI_FIXUP_CLASS_HEADER(0x1ac1, 0x089a,
 			       PCI_CLASS_NOT_DEFINED, 8, apex_pci_fixup_class);
+
+/*
+ * Some Loongson PCIe ports have a h/w limitation of
+ * 256 bytes maximum read request size...
+ *
+ * Keystone PCI controller has a h/w limitation of
+ * 256 bytes maximum read request size.
+ *
+ * Amlogic DesignWare PCI controller on Khadas VIM3/VIM3L have some
+ * issue with HDMI scrambled picture and nvme devices
+ * at intensive writing...
+ */
+static void mrrs_limit_quirk(struct pci_dev *dev)
+{
+	struct pci_bus *bus = dev->bus;
+	struct pci_dev *bridge;
+	int mrrs;
+	int mrrs_limit = 256;
+	static const struct pci_device_id bridge_devids[] = {
+		{ PCI_VDEVICE(LOONGSON, PCI_DEVICE_ID_LOONGSON_PCIE_PORT_0) },
+		{ PCI_VDEVICE(LOONGSON, PCI_DEVICE_ID_LOONGSON_PCIE_PORT_1) },
+		{ PCI_VDEVICE(LOONGSON, PCI_DEVICE_ID_LOONGSON_PCIE_PORT_2) },
+		{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_RC_K2HK),
+		 .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },
+		{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_RC_K2E),
+		 .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },
+		{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_RC_K2L),
+		 .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },
+		{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_RC_K2G),
+		 .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },
+		{ PCI_DEVICE(PCI_VENDOR_ID_SYNOPSYS, PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3) },
+		{ 0, },
+	};
+
+	/* look for the matching bridge */
+	while (!pci_is_root_bus(bus)) {
+		bridge = bus->self;
+		bus = bus->parent;
+		/*
+		 * 256 bytes maximum read request size. They can't handle
+		 * anything larger than this. So force this limit on
+		 * any devices attached under these ports.
+		 */
+		if (pci_match_id(bridge_devids, bridge)) {
+			mrrs = pcie_get_readrq(dev);
+			if (mrrs > mrrs_limit) {
+				pci_info(dev, "limiting MRRS %d to %d\n", mrrs, mrrs_limit);
+				pcie_set_readrq(dev, mrrs_limit);
+			}
+			break;
+		}
+	}
+}
+DECLARE_PCI_FIXUP_ENABLE(PCI_ANY_ID, PCI_ANY_ID, mrrs_limit_quirk);
Signature · 1 line
2.25.1
reviewer
#3 lore raw

Re: [PATCH 2/4] PCI: core: quirks: add mrrs_limit_quirk

Quoted 2 lines · Artem Lapkin

Prepare new MRRS limit quirk which can replace dublicated functionality

typo

Quoted 29 lines

for some controllers from Loongson, Keystone, DesignWare

---
 drivers/pci/quirks.c | 54 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)
drivers/pci/quirks.c +14 -0
index 653660e3b..73344ec71 100644
@@ -5612,3 +5612,57 @@ static void apex_pci_fixup_class(struct pci_dev *pdev)
 }
 DECLARE_PCI_FIXUP_CLASS_HEADER(0x1ac1, 0x089a,
                               PCI_CLASS_NOT_DEFINED, 8, apex_pci_fixup_class);
+
+/*
+ * Some Loongson PCIe ports have a h/w limitation of
+ * 256 bytes maximum read request size...
+ *
+ * Keystone PCI controller has a h/w limitation of
+ * 256 bytes maximum read request size.
+ *
+ * Amlogic DesignWare PCI controller on Khadas VIM3/VIM3L have some
+ * issue with HDMI scrambled picture and nvme devices
+ * at intensive writing...
+ */
+static void mrrs_limit_quirk(struct pci_dev *dev)
+{

drivers/pci/quirks.c static void apex_pci_fixup_class(struct pci_dev *pdev) · line 5638

Quoted 10 lines
drivers/pci/quirks.c +10 -0
+       struct pci_bus *bus = dev->bus;
+       struct pci_dev *bridge;
+       int mrrs;
+       int mrrs_limit = 256;
+       static const struct pci_device_id bridge_devids[] = {
+               { PCI_VDEVICE(LOONGSON, PCI_DEVICE_ID_LOONGSON_PCIE_PORT_0) },
+               { PCI_VDEVICE(LOONGSON, PCI_DEVICE_ID_LOONGSON_PCIE_PORT_1) },
+               { PCI_VDEVICE(LOONGSON, PCI_DEVICE_ID_LOONGSON_PCIE_PORT_2) },
+               { PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_RC_K2HK),
+                .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },

Seems like checking the class is redundant given the VID and Device ID seems to be pretty unique.

drivers/pci/quirks.c static void apex_pci_fixup_class(struct pci_dev *pdev) · line 5645

Quoted 7 lines
drivers/pci/quirks.c +7 -0
+               { PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_RC_K2E),
+                .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },
+               { PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_RC_K2L),
+                .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },
+               { PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_RC_K2G),
+                .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },
+               { PCI_DEVICE(PCI_VENDOR_ID_SYNOPSYS, PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3) },

This is Amlogic I guess. But they are reusing the IP default values? Any other vendor that does the same will get the same quirk applied? These are write-able,
so perhaps the driver should set them to something Amlogic specific.

drivers/pci/quirks.c static void apex_pci_fixup_class(struct pci_dev *pdev) · line 5658

Quoted 13 lines
drivers/pci/quirks.c +13 -0
+               { 0, },
+       };
+
+       /* look for the matching bridge */
+       while (!pci_is_root_bus(bus)) {
+               bridge = bus->self;
+               bus = bus->parent;
+               /*
+                * 256 bytes maximum read request size. They can't handle
+                * anything larger than this. So force this limit on
+                * any devices attached under these ports.
+                */
+               if (pci_match_id(bridge_devids, bridge)) {

I would invert this to save some indentation:

if (!pci_match_id(bridge_devids, bridge))

    continue;
Quoted 13 lines
drivers/pci/quirks.c +10 -1
+                       mrrs = pcie_get_readrq(dev);
+                       if (mrrs > mrrs_limit) {
+                               pci_info(dev, "limiting MRRS %d to %d\n", mrrs, mrrs_limit);
+                               pcie_set_readrq(dev, mrrs_limit);
+                       }
+                       break;
+               }
+       }
+}
+DECLARE_PCI_FIXUP_ENABLE(PCI_ANY_ID, PCI_ANY_ID, mrrs_limit_quirk);
--

2.25.1

#4 lore raw

PATCH 3/4 PCI: keystone move mrrs quirk to core

In reply to Artem Lapkin

Replace dublicated functionality ks_pcie_quirk to mrrs_limit_quirk from core pci quirks

---
 drivers/pci/controller/dwc/pci-keystone.c | 49 -----------------------
 1 file changed, 49 deletions(-)
drivers/pci/controller/dwc/pci-keystone.c +0 -49
index 53aa35cb3..879ae2d4f 100644
@@ -83,12 +83,6 @@
 #define ERR_IRQ_ALL			(ERR_AER | ERR_AXI | ERR_CORR | \
 					 ERR_NONFATAL | ERR_FATAL | ERR_SYS)
 
-/* PCIE controller device IDs */
-#define PCIE_RC_K2HK			0xb008
-#define PCIE_RC_K2E			0xb009
-#define PCIE_RC_K2L			0xb00a
-#define PCIE_RC_K2G			0xb00b
-
 #define KS_PCIE_DEV_TYPE_MASK		(0x3 << 1)
 #define KS_PCIE_DEV_TYPE(mode)		((mode) << 1)
 
@@ -521,49 +515,6 @@ static int ks_pcie_start_link(struct dw_pcie *pci)
 	return 0;
 }
 
-static void ks_pcie_quirk(struct pci_dev *dev)
-{
-	struct pci_bus *bus = dev->bus;
-	struct pci_dev *bridge;
-	static const struct pci_device_id rc_pci_devids[] = {
-		{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCIE_RC_K2HK),
-		 .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },
-		{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCIE_RC_K2E),
-		 .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },
-		{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCIE_RC_K2L),
-		 .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },
-		{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCIE_RC_K2G),
-		 .class = PCI_CLASS_BRIDGE_PCI << 8, .class_mask = ~0, },
-		{ 0, },
-	};
-
-	if (pci_is_root_bus(bus))
-		bridge = dev;
-
-	/* look for the host bridge */
-	while (!pci_is_root_bus(bus)) {
-		bridge = bus->self;
-		bus = bus->parent;
-	}
-
-	if (!bridge)
-		return;
-
-	/*
-	 * Keystone PCI controller has a h/w limitation of
-	 * 256 bytes maximum read request size.  It can't handle
-	 * anything higher than this.  So force this limit on
-	 * all downstream devices.
-	 */
-	if (pci_match_id(rc_pci_devids, bridge)) {
-		if (pcie_get_readrq(dev) > 256) {
-			dev_info(&dev->dev, "limiting MRRS to 256\n");
-			pcie_set_readrq(dev, 256);
-		}
-	}
-}
-DECLARE_PCI_FIXUP_ENABLE(PCI_ANY_ID, PCI_ANY_ID, ks_pcie_quirk);
-
 static void ks_pcie_msi_irq_handler(struct irq_desc *desc)
 {
 	unsigned int irq = desc->irq_data.hwirq;
Signature · 1 line
2.25.1
#5 lore raw

PATCH 4/4 PCI: loongson move mrrs quirk to core

In reply to Artem Lapkin

Replace dublicated functionality loongson_mrrs_quirk to mrrs_limit_quirk from core pci quirks

---
 drivers/pci/controller/pci-loongson.c | 42 ++-------------------------
 1 file changed, 3 insertions(+), 39 deletions(-)
drivers/pci/controller/pci-loongson.c +3 -39
index 48169b1e3..5a54faf10 100644
@@ -13,10 +13,6 @@
 #include "../pci.h"
 
 /* Device IDs */
-#define DEV_PCIE_PORT_0	0x7a09
-#define DEV_PCIE_PORT_1	0x7a19
-#define DEV_PCIE_PORT_2	0x7a29
-
 #define DEV_LS2K_APB	0x7a02
 #define DEV_LS7A_CONF	0x7a10
 #define DEV_LS7A_LPC	0x7a0c
@@ -38,11 +34,11 @@ static void bridge_class_quirk(struct pci_dev *dev)
 	dev->class = PCI_CLASS_BRIDGE_PCI << 8;
 }
 DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON,
-			DEV_PCIE_PORT_0, bridge_class_quirk);
+			PCI_DEVICE_ID_LOONGSON_PCIE_PORT_0, bridge_class_quirk);
 DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON,
-			DEV_PCIE_PORT_1, bridge_class_quirk);
+			PCI_DEVICE_ID_LOONGSON_PCIE_PORT_1, bridge_class_quirk);
 DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON,
-			DEV_PCIE_PORT_2, bridge_class_quirk);
+			PCI_DEVICE_ID_LOONGSON_PCIE_PORT_2, bridge_class_quirk);
 
 static void system_bus_quirk(struct pci_dev *pdev)
 {
@@ -60,38 +56,6 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON,
 DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON,
 			DEV_LS7A_LPC, system_bus_quirk);
 
-static void loongson_mrrs_quirk(struct pci_dev *dev)
-{
-	struct pci_bus *bus = dev->bus;
-	struct pci_dev *bridge;
-	static const struct pci_device_id bridge_devids[] = {
-		{ PCI_VDEVICE(LOONGSON, DEV_PCIE_PORT_0) },
-		{ PCI_VDEVICE(LOONGSON, DEV_PCIE_PORT_1) },
-		{ PCI_VDEVICE(LOONGSON, DEV_PCIE_PORT_2) },
-		{ 0, },
-	};
-
-	/* look for the matching bridge */
-	while (!pci_is_root_bus(bus)) {
-		bridge = bus->self;
-		bus = bus->parent;
-		/*
-		 * Some Loongson PCIe ports have a h/w limitation of
-		 * 256 bytes maximum read request size. They can't handle
-		 * anything larger than this. So force this limit on
-		 * any devices attached under these ports.
-		 */
-		if (pci_match_id(bridge_devids, bridge)) {
-			if (pcie_get_readrq(dev) > 256) {
-				pci_info(dev, "limiting MRRS to 256\n");
-				pcie_set_readrq(dev, 256);
-			}
-			break;
-		}
-	}
-}
-DECLARE_PCI_FIXUP_ENABLE(PCI_ANY_ID, PCI_ANY_ID, loongson_mrrs_quirk);
-
 static void __iomem *cfg1_map(struct loongson_pci *priv, int bus,
 				unsigned int devfn, int where)
 {
Signature · 1 line
2.25.1
#6 lore raw
In reply to Artem Lapkin

[+cc Huacai]

Quoted 3 lines · Artem Lapkin

Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
* drivers/pci/controller/dwc/pci-keystone.c
* drivers/pci/controller/pci-loongson.c

s/dublicated/duplicated/ (several occurrences)

Capitalize subject lines.

Use "git log --online" to learn conventions and follow them.

Add "()" after function names.

Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).

End sentences with periods.

A "move" patch must include both the removal and the addition and make no changes to the code itself.

Amlogic appears without explanation in 2/4. Must be separate patch to address only that specific issue. Should reference published erratum if possible. "Solves some issue" is not a compelling justification.

The tree must be consistent and functionally the same or improved after every patch.

Add to pci_ids.h only if symbol used more than one place.

See
https://lore.kernel.org/r/[email protected],
which looks similar. Combine efforts if possible and cc Huacai so you're both aware of overlapping work.

More hints in case they're useful:
https://lore.kernel.org/linux-pci/[email protected]/

Quoted 21 lines

Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any functionality changes by one mrrs_limit_quirk

Added DesignWare PCI controller which need same quirk for
* drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3)

This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic) with HDMI scrambled picture and nvme devices at intensive writing...

come from:
* https://lore.kernel.org/linux-pci/[email protected]/

Artem Lapkin (4):
PCI: move Keystone and Loongson device IDs to pci_ids
PCI: core: quirks: add mrrs_limit_quirk
PCI: keystone move mrrs quirk to core
PCI: loongson move mrrs quirk to core

Signature · 1 line
2.25.1
#7 lore raw

Hi, Bjorn,

&gt; -----原始邮件-----
&gt; 发件人: "Bjorn Helgaas" <[email protected]>
&gt; 发送时间: 2021-07-01 23:46:34 (星期四)
&gt; 收件人: "Artem Lapkin" <[email protected]>
&gt; 抄送: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "Huacai Chen" <[email protected]> &gt; 主题: Re: [PATCH 0/4] PCI: replace dublicated MRRS limit quirks
&gt;
&gt; [+cc Huacai]
&gt;
&gt; On Sat, Jun 19, 2021 at 02:39:48PM +0800, Artem Lapkin wrote:
&gt; &gt; Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
&gt; &gt; * drivers/pci/controller/dwc/pci-keystone.c
&gt; &gt; * drivers/pci/controller/pci-loongson.c
&gt;
&gt; s/dublicated/duplicated/ (several occurrences)
&gt;
&gt; Capitalize subject lines.
&gt;
&gt; Use "git log --online" to learn conventions and follow them.
&gt;
&gt; Add "()" after function names.
&gt;
&gt; Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).
&gt;
&gt; End sentences with periods.
&gt;
&gt; A "move" patch must include both the removal and the addition and make
&gt; no changes to the code itself.
&gt;
&gt; Amlogic appears without explanation in 2/4. Must be separate patch to
&gt; address only that specific issue. Should reference published erratum
&gt; if possible. "Solves some issue" is not a compelling justification.
&gt;
&gt; The tree must be consistent and functionally the same or improved
&gt; after every patch.
&gt;
&gt; Add to pci_ids.h only if symbol used more than one place.
&gt;
&gt; See
&gt; https://lore.kernel.org/r/[email protected],
&gt; which looks similar. Combine efforts if possible and cc Huacai so
&gt; you're both aware of overlapping work.
&gt;
&gt; More hints in case they're useful:
&gt; https://lore.kernel.org/linux-pci/[email protected]/
&gt;
&gt; &gt; Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
&gt; &gt; functionality changes by one mrrs_limit_quirk
Does that means keystone and Loongson has the same MRRS problem? And what should I do now?

Huacai
&gt; &gt;
&gt; &gt; Added DesignWare PCI controller which need same quirk for
&gt; &gt; * drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3) &gt; &gt;
&gt; &gt; This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic) &gt; &gt; with HDMI scrambled picture and nvme devices at intensive writing... &gt; &gt;
&gt; &gt; come from:
&gt; &gt; * https://lore.kernel.org/linux-pci/[email protected]/
&gt; &gt;
&gt; &gt; Artem Lapkin (4):

&gt; &gt;  PCI: move Keystone and Loongson device IDs to pci_ids
&gt; &gt;  PCI: core: quirks: add mrrs_limit_quirk
&gt; &gt;  PCI: keystone move mrrs quirk to core
&gt; &gt;  PCI: loongson move mrrs quirk to core

&gt; &gt;
&gt; &gt; --
&gt; &gt; 2.25.1
&gt; &gt;

</[email protected]></[email protected]></[email protected]>

author
#8 lore raw
Quoted 1 line

Does that means keystone and Loongson has the same MRRS problem? And what should I do now?

Look like yes ! and amlogic has the same problem.
I think somebody need to rewrite it all to one common quirk for this problem.

If no one has any objection, I can try to remake it again.

Quoted 76 lines · 陈华才

Hi, Bjorn,

&gt; -----原始邮件-----
&gt; 发件人: "Bjorn Helgaas" <[email protected]>
&gt; 发送时间: 2021-07-01 23:46:34 (星期四)
&gt; 收件人: "Artem Lapkin" <[email protected]>
&gt; 抄送: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "Huacai Chen" <[email protected]> &gt; 主题: Re: [PATCH 0/4] PCI: replace dublicated MRRS limit quirks
&gt;
&gt; [+cc Huacai]
&gt;
&gt; On Sat, Jun 19, 2021 at 02:39:48PM +0800, Artem Lapkin wrote:
&gt; &gt; Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
&gt; &gt; * drivers/pci/controller/dwc/pci-keystone.c
&gt; &gt; * drivers/pci/controller/pci-loongson.c
&gt;
&gt; s/dublicated/duplicated/ (several occurrences)
&gt;
&gt; Capitalize subject lines.
&gt;
&gt; Use "git log --online" to learn conventions and follow them.
&gt;
&gt; Add "()" after function names.
&gt;
&gt; Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).
&gt;
&gt; End sentences with periods.
&gt;
&gt; A "move" patch must include both the removal and the addition and make
&gt; no changes to the code itself.
&gt;
&gt; Amlogic appears without explanation in 2/4. Must be separate patch to
&gt; address only that specific issue. Should reference published erratum
&gt; if possible. "Solves some issue" is not a compelling justification.
&gt;
&gt; The tree must be consistent and functionally the same or improved
&gt; after every patch.
&gt;
&gt; Add to pci_ids.h only if symbol used more than one place.
&gt;
&gt; See
&gt; https://lore.kernel.org/r/[email protected],
&gt; which looks similar. Combine efforts if possible and cc Huacai so
&gt; you're both aware of overlapping work.
&gt;
&gt; More hints in case they're useful:
&gt; https://lore.kernel.org/linux-pci/[email protected]/
&gt;
&gt; &gt; Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
&gt; &gt; functionality changes by one mrrs_limit_quirk
Does that means keystone and Loongson has the same MRRS problem? And what should I do now?

Huacai
&gt; &gt;
&gt; &gt; Added DesignWare PCI controller which need same quirk for
&gt; &gt; * drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3) &gt; &gt;
&gt; &gt; This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic) &gt; &gt; with HDMI scrambled picture and nvme devices at intensive writing... &gt; &gt;
&gt; &gt; come from:
&gt; &gt; * https://lore.kernel.org/linux-pci/[email protected]/
&gt; &gt;
&gt; &gt; Artem Lapkin (4):

&gt; &gt;  PCI: move Keystone and Loongson device IDs to pci_ids
&gt; &gt;  PCI: core: quirks: add mrrs_limit_quirk
&gt; &gt;  PCI: keystone move mrrs quirk to core
&gt; &gt;  PCI: loongson move mrrs quirk to core

&gt; &gt;
&gt; &gt; --
&gt; &gt; 2.25.1
&gt; &gt;

</[email protected]></[email protected]></[email protected]>

#9 lore raw

Hi Art,

[...]

Quoted 1 line

If no one has any objection, I can try to remake it again.

I can't speak for everyone, of course, but I am sure that nobody would object if you decide to work on v2, so that a proper fix can be applied, and issues fixed long term.

Thank you for working on this! Much appreciated!

Krzysztof

#10 lore raw
In reply to Art Nikpal

Hi, Art,

Quoted 7 lines · Art Nikpal
Quoted 2 lines

Does that means keystone and Loongson has the same MRRS problem? And what should I do now?

Look like yes ! and amlogic has the same problem.
I think somebody need to rewrite it all to one common quirk for this problem.

If no one has any objection, I can try to remake it again.

But, Loongson platform has newer revision of hardware, and the MRRS quirk has changed, please see:
https://patchwork.kernel.org/project/linux-pci/list/?series=509497

Huacai

Quoted 78 lines · 陈华才
Quoted 76 lines

Hi, Bjorn,

&gt; -----原始邮件-----
&gt; 发件人: "Bjorn Helgaas" <[email protected]>
&gt; 发送时间: 2021-07-01 23:46:34 (星期四)
&gt; 收件人: "Artem Lapkin" <[email protected]>
&gt; 抄送: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "Huacai Chen" <[email protected]> &gt; 主题: Re: [PATCH 0/4] PCI: replace dublicated MRRS limit quirks
&gt;
&gt; [+cc Huacai]
&gt;
&gt; On Sat, Jun 19, 2021 at 02:39:48PM +0800, Artem Lapkin wrote:
&gt; &gt; Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
&gt; &gt; * drivers/pci/controller/dwc/pci-keystone.c
&gt; &gt; * drivers/pci/controller/pci-loongson.c
&gt;
&gt; s/dublicated/duplicated/ (several occurrences)
&gt;
&gt; Capitalize subject lines.
&gt;
&gt; Use "git log --online" to learn conventions and follow them.
&gt;
&gt; Add "()" after function names.
&gt;
&gt; Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).
&gt;
&gt; End sentences with periods.
&gt;
&gt; A "move" patch must include both the removal and the addition and make
&gt; no changes to the code itself.
&gt;
&gt; Amlogic appears without explanation in 2/4. Must be separate patch to
&gt; address only that specific issue. Should reference published erratum
&gt; if possible. "Solves some issue" is not a compelling justification.
&gt;
&gt; The tree must be consistent and functionally the same or improved
&gt; after every patch.
&gt;
&gt; Add to pci_ids.h only if symbol used more than one place.
&gt;
&gt; See
&gt; https://lore.kernel.org/r/[email protected],
&gt; which looks similar. Combine efforts if possible and cc Huacai so
&gt; you're both aware of overlapping work.
&gt;
&gt; More hints in case they're useful:
&gt; https://lore.kernel.org/linux-pci/[email protected]/
&gt;
&gt; &gt; Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
&gt; &gt; functionality changes by one mrrs_limit_quirk
Does that means keystone and Loongson has the same MRRS problem? And what should I do now?

Huacai
&gt; &gt;
&gt; &gt; Added DesignWare PCI controller which need same quirk for
&gt; &gt; * drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3) &gt; &gt;
&gt; &gt; This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic) &gt; &gt; with HDMI scrambled picture and nvme devices at intensive writing... &gt; &gt;
&gt; &gt; come from:
&gt; &gt; * https://lore.kernel.org/linux-pci/[email protected]/
&gt; &gt;
&gt; &gt; Artem Lapkin (4):

&gt; &gt;  PCI: move Keystone and Loongson device IDs to pci_ids
&gt; &gt;  PCI: core: quirks: add mrrs_limit_quirk
&gt; &gt;  PCI: keystone move mrrs quirk to core
&gt; &gt;  PCI: loongson move mrrs quirk to core

&gt; &gt;
&gt; &gt; --
&gt; &gt; 2.25.1
&gt; &gt;

</[email protected]></[email protected]></[email protected]>

author
#11 lore raw
Quoted 4 lines

But, Loongson platform has newer revision of hardware, and the MRRS quirk has changed, please see:
https://patchwork.kernel.org/project/linux-pci/list/?series=509497
Huacai

OK! tnx for information ! maybe we can cooperate and make one universal quirk for all

Quoted 94 lines · Huacai Chen

Hi, Art,

Quoted 7 lines · Art Nikpal
Quoted 2 lines

Does that means keystone and Loongson has the same MRRS problem? And what should I do now?

Look like yes ! and amlogic has the same problem.
I think somebody need to rewrite it all to one common quirk for this problem.

If no one has any objection, I can try to remake it again.

But, Loongson platform has newer revision of hardware, and the MRRS quirk has changed, please see:
https://patchwork.kernel.org/project/linux-pci/list/?series=509497

Huacai

Quoted 78 lines · 陈华才
Quoted 76 lines

Hi, Bjorn,

&gt; -----原始邮件-----
&gt; 发件人: "Bjorn Helgaas" <[email protected]>
&gt; 发送时间: 2021-07-01 23:46:34 (星期四)
&gt; 收件人: "Artem Lapkin" <[email protected]>
&gt; 抄送: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "Huacai Chen" <[email protected]> &gt; 主题: Re: [PATCH 0/4] PCI: replace dublicated MRRS limit quirks
&gt;
&gt; [+cc Huacai]
&gt;
&gt; On Sat, Jun 19, 2021 at 02:39:48PM +0800, Artem Lapkin wrote:
&gt; &gt; Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
&gt; &gt; * drivers/pci/controller/dwc/pci-keystone.c
&gt; &gt; * drivers/pci/controller/pci-loongson.c
&gt;
&gt; s/dublicated/duplicated/ (several occurrences)
&gt;
&gt; Capitalize subject lines.
&gt;
&gt; Use "git log --online" to learn conventions and follow them.
&gt;
&gt; Add "()" after function names.
&gt;
&gt; Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).
&gt;
&gt; End sentences with periods.
&gt;
&gt; A "move" patch must include both the removal and the addition and make
&gt; no changes to the code itself.
&gt;
&gt; Amlogic appears without explanation in 2/4. Must be separate patch to
&gt; address only that specific issue. Should reference published erratum
&gt; if possible. "Solves some issue" is not a compelling justification.
&gt;
&gt; The tree must be consistent and functionally the same or improved
&gt; after every patch.
&gt;
&gt; Add to pci_ids.h only if symbol used more than one place.
&gt;
&gt; See
&gt; https://lore.kernel.org/r/[email protected],
&gt; which looks similar. Combine efforts if possible and cc Huacai so
&gt; you're both aware of overlapping work.
&gt;
&gt; More hints in case they're useful:
&gt; https://lore.kernel.org/linux-pci/[email protected]/
&gt;
&gt; &gt; Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
&gt; &gt; functionality changes by one mrrs_limit_quirk
Does that means keystone and Loongson has the same MRRS problem? And what should I do now?

Huacai
&gt; &gt;
&gt; &gt; Added DesignWare PCI controller which need same quirk for
&gt; &gt; * drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3) &gt; &gt;
&gt; &gt; This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic) &gt; &gt; with HDMI scrambled picture and nvme devices at intensive writing... &gt; &gt;
&gt; &gt; come from:
&gt; &gt; * https://lore.kernel.org/linux-pci/[email protected]/
&gt; &gt;
&gt; &gt; Artem Lapkin (4):

&gt; &gt;  PCI: move Keystone and Loongson device IDs to pci_ids
&gt; &gt;  PCI: core: quirks: add mrrs_limit_quirk
&gt; &gt;  PCI: keystone move mrrs quirk to core
&gt; &gt;  PCI: loongson move mrrs quirk to core

&gt; &gt;
&gt; &gt; --
&gt; &gt; 2.25.1
&gt; &gt;

</[email protected]></[email protected]></[email protected]>

#12 lore raw

Hi,

Quoted 7 lines · Art Nikpal
Quoted 4 lines

But, Loongson platform has newer revision of hardware, and the MRRS quirk has changed, please see:
https://patchwork.kernel.org/project/linux-pci/list/?series=509497
Huacai

OK! tnx for information ! maybe we can cooperate and make one universal quirk for all

In their Designware PCIe controller driver, amlogic sets the Max_Payload_Size & Max_Read_Request_Size to 256:
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
in their root port PCIe Express Device Control Register.

Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size & Max_Read_Request_Size are used to decompose into AXI burst, but it seems the Max_Payload_Size & Max_Read_Request_Size are set by default to 512 but the internal Max_Payload_Size_Supported is set to 256, thus changing these values to 256 at runtime to match and optimize bandwidth.

It's said, "Reducing Outbound Decomposition" :
- "Ensure that your application master does not generate bursts of size greater than or equal to Max_Payload_Size"
- "Program your PCIe system with a larger value of Max_Payload_Size without exceeding Max_Payload_Size_Supported"
- "Program your PCIe system with a larger value of Max_Read_Request without exceeding Max_Payload_Size_Supported:

So leaving 512 in Max_Payload_Size & Max_Read_Request leads to Outbound Decomposition which decreases PCIe link and degrades the AXI bus by doubling the bursts, leading to this fix to avoid overflowing the AXI bus.

So it seems to be still needed, I assume this *should* be handled in the core somehow to propagate these settings to child endpoints to match the root port Max_Payload_Size & Max_Read_Request sizes.

Maybe by adding a core function to set these values instead of using the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers and set a state on the root port to propagate the value ?

Neil

Quoted 96 lines · Huacai Chen
Quoted 94 lines

Hi, Art,

Quoted 7 lines · Art Nikpal
Quoted 2 lines

Does that means keystone and Loongson has the same MRRS problem? And what should I do now?

Look like yes ! and amlogic has the same problem.
I think somebody need to rewrite it all to one common quirk for this problem.

If no one has any objection, I can try to remake it again.

But, Loongson platform has newer revision of hardware, and the MRRS quirk has changed, please see:
https://patchwork.kernel.org/project/linux-pci/list/?series=509497

Huacai

Quoted 78 lines · 陈华才
Quoted 76 lines

Hi, Bjorn,

&gt; -----原始邮件-----
&gt; 发件人: "Bjorn Helgaas" <[email protected]>
&gt; 发送时间: 2021-07-01 23:46:34 (星期四)
&gt; 收件人: "Artem Lapkin" <[email protected]>
&gt; 抄送: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "Huacai Chen" <[email protected]> &gt; 主题: Re: [PATCH 0/4] PCI: replace dublicated MRRS limit quirks
&gt;
&gt; [+cc Huacai]
&gt;
&gt; On Sat, Jun 19, 2021 at 02:39:48PM +0800, Artem Lapkin wrote:
&gt; &gt; Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
&gt; &gt; * drivers/pci/controller/dwc/pci-keystone.c
&gt; &gt; * drivers/pci/controller/pci-loongson.c
&gt;
&gt; s/dublicated/duplicated/ (several occurrences)
&gt;
&gt; Capitalize subject lines.
&gt;
&gt; Use "git log --online" to learn conventions and follow them.
&gt;
&gt; Add "()" after function names.
&gt;
&gt; Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).
&gt;
&gt; End sentences with periods.
&gt;
&gt; A "move" patch must include both the removal and the addition and make
&gt; no changes to the code itself.
&gt;
&gt; Amlogic appears without explanation in 2/4. Must be separate patch to
&gt; address only that specific issue. Should reference published erratum
&gt; if possible. "Solves some issue" is not a compelling justification.
&gt;
&gt; The tree must be consistent and functionally the same or improved
&gt; after every patch.
&gt;
&gt; Add to pci_ids.h only if symbol used more than one place.
&gt;
&gt; See
&gt; https://lore.kernel.org/r/[email protected],
&gt; which looks similar. Combine efforts if possible and cc Huacai so
&gt; you're both aware of overlapping work.
&gt;
&gt; More hints in case they're useful:
&gt; https://lore.kernel.org/linux-pci/[email protected]/
&gt;
&gt; &gt; Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
&gt; &gt; functionality changes by one mrrs_limit_quirk
Does that means keystone and Loongson has the same MRRS problem? And what should I do now?

Huacai
&gt; &gt;
&gt; &gt; Added DesignWare PCI controller which need same quirk for
&gt; &gt; * drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3) &gt; &gt;
&gt; &gt; This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic) &gt; &gt; with HDMI scrambled picture and nvme devices at intensive writing... &gt; &gt;
&gt; &gt; come from:
&gt; &gt; * https://lore.kernel.org/linux-pci/[email protected]/
&gt; &gt;
&gt; &gt; Artem Lapkin (4):

&gt; &gt;  PCI: move Keystone and Loongson device IDs to pci_ids
&gt; &gt;  PCI: core: quirks: add mrrs_limit_quirk
&gt; &gt;  PCI: keystone move mrrs quirk to core
&gt; &gt;  PCI: loongson move mrrs quirk to core

&gt; &gt;
&gt; &gt; --
&gt; &gt; 2.25.1
&gt; &gt;

</[email protected]></[email protected]></[email protected]>

#13 lore raw
Quoted 28 lines · Neil Armstrong

In their Designware PCIe controller driver, amlogic sets the Max_Payload_Size & Max_Read_Request_Size to 256:
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
in their root port PCIe Express Device Control Register.

Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size & Max_Read_Request_Size are used to decompose into AXI burst, but it seems the Max_Payload_Size & Max_Read_Request_Size are set by default to 512 but the internal Max_Payload_Size_Supported is set to 256, thus changing these values to 256 at runtime to match and optimize bandwidth.

It's said, "Reducing Outbound Decomposition" :
- "Ensure that your application master does not generate bursts of size greater than or equal to Max_Payload_Size"

- "Program your PCIe system with a larger value of Max_Payload_Size without exceeding Max_Payload_Size_Supported"

- "Program your PCIe system with a larger value of Max_Read_Request without exceeding Max_Payload_Size_Supported:

So leaving 512 in Max_Payload_Size & Max_Read_Request leads to Outbound Decomposition which decreases PCIe link and degrades the AXI bus by doubling the bursts, leading to this fix to avoid overflowing the AXI bus.

Quoted 7 lines

So it seems to be still needed, I assume this *should* be handled in the core somehow to propagate these settings to child endpoints to match the root port Max_Payload_Size & Max_Read_Request sizes.

Maybe by adding a core function to set these values instead of using the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers and set a state on the root port to propagate the value ?

I don't have the Synopsys DW-PCIe Databook, so I'm lacking any context. The above *seems* to say that MPS/MRRS settings affect AXI bus usage.

The MPS and MRRS registers are defined to affect traffic on *PCIe*. If a platform uses MPS and MRRS values to optimize transfers on non-PCIe links, that's a problem because the PCI core code that manages MPS and MRRS has no knowledge of those non-PCIe parts of the system.

You might be able to deal with this in Synopsys-specific code somehow, but it's going to be a bit of a hassle because I don't want it to make maintenance of the generic MPS/MRRS code harder.

Bjorn

#14 lore raw

Hi,

Quoted 33 lines · Bjorn Helgaas
Quoted 32 lines · Neil Armstrong

In their Designware PCIe controller driver, amlogic sets the Max_Payload_Size & Max_Read_Request_Size to 256:
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
in their root port PCIe Express Device Control Register.

Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size & Max_Read_Request_Size are used to decompose into AXI burst, but it seems the Max_Payload_Size & Max_Read_Request_Size are set by default to 512 but the internal Max_Payload_Size_Supported is set to 256, thus changing these values to 256 at runtime to match and optimize bandwidth.

It's said, "Reducing Outbound Decomposition" :
- "Ensure that your application master does not generate bursts of size greater than or equal to Max_Payload_Size"

- "Program your PCIe system with a larger value of Max_Payload_Size without exceeding Max_Payload_Size_Supported"

- "Program your PCIe system with a larger value of Max_Read_Request without exceeding Max_Payload_Size_Supported:

So leaving 512 in Max_Payload_Size & Max_Read_Request leads to Outbound Decomposition which decreases PCIe link and degrades the AXI bus by doubling the bursts, leading to this fix to avoid overflowing the AXI bus.

So it seems to be still needed, I assume this *should* be handled in the core somehow to propagate these settings to child endpoints to match the root port Max_Payload_Size & Max_Read_Request sizes.

Quoted 7 lines
Quoted 3 lines

Maybe by adding a core function to set these values instead of using the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers and set a state on the root port to propagate the value ?

I don't have the Synopsys DW-PCIe Databook, so I'm lacking any context. The above *seems* to say that MPS/MRRS settings affect AXI bus usage.

It does when the TLPs are directed to the RC.

Quoted 5 lines

The MPS and MRRS registers are defined to affect traffic on *PCIe*. If a platform uses MPS and MRRS values to optimize transfers on non-PCIe links, that's a problem because the PCI core code that manages MPS and MRRS has no knowledge of those non-PCIe parts of the system.

Yes and no, it only affects PCIe in P2P, in non-P2P is will certainly affect transfers on the internal SoC/Processor/Chip internal bus/fabric.

Quoted 3 lines

You might be able to deal with this in Synopsys-specific code somehow, but it's going to be a bit of a hassle because I don't want it to make maintenance of the generic MPS/MRRS code harder.

I understand, but this is why these quirks are currently implemented in the controller driver and only applies when the controller has been probed and to each endpoint detected on this particular controller.

So we may continue having separate quirks for each controller if the core isn't the right place to handle MPS/MRRS.

Neil

Quoted 2 lines

Bjorn

#15 lore raw
Quoted 38 lines · Neil Armstrong
Quoted 37 lines · Bjorn Helgaas
Quoted 35 lines · Neil Armstrong

In their Designware PCIe controller driver, amlogic sets the Max_Payload_Size & Max_Read_Request_Size to 256:
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
in their root port PCIe Express Device Control Register.

Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size & Max_Read_Request_Size are used to decompose into AXI burst, but it seems the Max_Payload_Size & Max_Read_Request_Size are set by default to 512 but the internal Max_Payload_Size_Supported is set to 256, thus changing these values to 256 at runtime to match and optimize bandwidth.

It's said, "Reducing Outbound Decomposition" :
- "Ensure that your application master does not generate bursts of size greater than or equal to Max_Payload_Size"

- "Program your PCIe system with a larger value of Max_Payload_Size without exceeding Max_Payload_Size_Supported"

- "Program your PCIe system with a larger value of Max_Read_Request without exceeding Max_Payload_Size_Supported:

So leaving 512 in Max_Payload_Size & Max_Read_Request leads to Outbound Decomposition which decreases PCIe link and degrades the AXI bus by doubling the bursts, leading to this fix to avoid overflowing the AXI bus.

So it seems to be still needed, I assume this *should* be handled in the core somehow to propagate these settings to child endpoints to match the root port Max_Payload_Size & Max_Read_Request sizes.

Maybe by adding a core function to set these values instead of using the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers and set a state on the root port to propagate the value ?

Quoted 5 lines
Quoted 3 lines

I don't have the Synopsys DW-PCIe Databook, so I'm lacking any context. The above *seems* to say that MPS/MRRS settings affect AXI bus usage.

It does when the TLPs are directed to the RC.

That's a defect in the RC.

Quoted 12 lines
Quoted 4 lines

The MPS and MRRS registers are defined to affect traffic on *PCIe*. If a platform uses MPS and MRRS values to optimize transfers on non-PCIe links, that's a problem because the PCI core code that manages MPS and MRRS has no knowledge of those non-PCIe parts of the system.

Yes and no, it only affects PCIe in P2P, in non-P2P is will certainly affect transfers on the internal SoC/Processor/Chip internal bus/fabric.

Quoted 3 lines

You might be able to deal with this in Synopsys-specific code somehow, but it's going to be a bit of a hassle because I don't want it to make maintenance of the generic MPS/MRRS code harder.

Quoted 6 lines

I understand, but this is why these quirks are currently implemented in the controller driver and only applies when the controller has been probed and to each endpoint detected on this particular controller.

So we may continue having separate quirks for each controller if the core isn't the right place to handle MPS/MRRS.

The PCI core is the correct place to handle MPS/MRRS because their behavior is defined by the PCIe spec.

Quirks are the way to work around this defect in the Synopsys PCIe IP.

Bjorn

#16 lore raw
Quoted 39 lines · Bjorn Helgaas
Quoted 38 lines · Neil Armstrong
Quoted 37 lines · Bjorn Helgaas
Quoted 35 lines · Neil Armstrong

In their Designware PCIe controller driver, amlogic sets the Max_Payload_Size & Max_Read_Request_Size to 256:
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
in their root port PCIe Express Device Control Register.

Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size & Max_Read_Request_Size are used to decompose into AXI burst, but it seems the Max_Payload_Size & Max_Read_Request_Size are set by default to 512 but the internal Max_Payload_Size_Supported is set to 256, thus changing these values to 256 at runtime to match and optimize bandwidth.

It's said, "Reducing Outbound Decomposition" :
- "Ensure that your application master does not generate bursts of size greater than or equal to Max_Payload_Size"

- "Program your PCIe system with a larger value of Max_Payload_Size without exceeding Max_Payload_Size_Supported"

- "Program your PCIe system with a larger value of Max_Read_Request without exceeding Max_Payload_Size_Supported:

So leaving 512 in Max_Payload_Size & Max_Read_Request leads to Outbound Decomposition which decreases PCIe link and degrades the AXI bus by doubling the bursts, leading to this fix to avoid overflowing the AXI bus.

So it seems to be still needed, I assume this *should* be handled in the core somehow to propagate these settings to child endpoints to match the root port Max_Payload_Size & Max_Read_Request sizes.

Maybe by adding a core function to set these values instead of using the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers and set a state on the root port to propagate the value ?

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

I don't have the Synopsys DW-PCIe Databook, so I'm lacking any context. The above *seems* to say that MPS/MRRS settings affect AXI bus usage.

It does when the TLPs are directed to the RC.

That's a defect in the RC.

I mean, it's OK if MPS affects the *performance* of traffic on AXI, but the RC must work correctly for any MPS up to the MPSS (Max Payload Size Supported) it advertises. And there's no PCIe mechanism for managing the AXI performance impact, so I think it's a mistake if Synopsys expects device-specific code in the MPS/MRRS configuration path. That code should be device-independent.

Quoted 25 lines
Quoted 12 lines
Quoted 4 lines

The MPS and MRRS registers are defined to affect traffic on *PCIe*. If a platform uses MPS and MRRS values to optimize transfers on non-PCIe links, that's a problem because the PCI core code that manages MPS and MRRS has no knowledge of those non-PCIe parts of the system.

Yes and no, it only affects PCIe in P2P, in non-P2P is will certainly affect transfers on the internal SoC/Processor/Chip internal bus/fabric.

Quoted 3 lines

You might be able to deal with this in Synopsys-specific code somehow, but it's going to be a bit of a hassle because I don't want it to make maintenance of the generic MPS/MRRS code harder.

Quoted 6 lines

I understand, but this is why these quirks are currently implemented in the controller driver and only applies when the controller has been probed and to each endpoint detected on this particular controller.

So we may continue having separate quirks for each controller if the core isn't the right place to handle MPS/MRRS.

The PCI core is the correct place to handle MPS/MRRS because their behavior is defined by the PCIe spec.

Quirks are the way to work around this defect in the Synopsys PCIe IP.

Bjorn

author
#17 lore raw
In reply to Bjorn Helgaas
Quoted 5 lines

The PCI core is the correct place to handle MPS/MRRS because their> behavior is defined by the PCIe spec. Quirks are the way to work around this defect in the Synopsys PCIe IP.

don't want it to make
maintenance of the generic MPS/MRRS code harder.

Trying summarize ( every one must use separate quirk )

which file is right place for for meson_mrrs_limit_quirk()

- pci/controller/dwc/pci-meson.c or
- pci/quirks.c

rewrited quirk just for meson:

static void meson_mrrs_limit_quirk(struct pci_dev *dev)

{
    struct pci_bus *bus = dev->bus;
    int mrrs, mrrs_limit = 256;
    static const struct pci_device_id bridge_devids[] = {
        { PCI_DEVICE(PCI_VENDOR_ID_SYNOPSYS, PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3) },
        { 0, },
    };

    /* look for the matching bridge */
    while (!pci_is_root_bus(bus)) {
        /*
         * 256 bytes maximum read request size. They can't handle
         * anything larger than this. So force this limit on
         * any devices attached under these ports.
         */
        if (!pci_match_id(bridge_devids, bus->self)){
            bus = bus->parent;
            continue;
        }
        mrrs = pcie_get_readrq(dev);
        if (mrrs > mrrs_limit) {
            pci_info(dev, "limiting MRRS %d to %d\n", mrrs, mrrs_limit);
            pcie_set_readrq(dev, mrrs_limit);
        }
        break;
    }
}

DECLARE_PCI_FIXUP_ENABLE(PCI_ANY_ID, PCI_ANY_ID, meson_mrrs_limit_quirk);

Quoted 67 lines · Bjorn Helgaas
Quoted 38 lines · Neil Armstrong
Quoted 37 lines · Bjorn Helgaas
Quoted 35 lines · Neil Armstrong

In their Designware PCIe controller driver, amlogic sets the Max_Payload_Size & Max_Read_Request_Size to 256:
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
in their root port PCIe Express Device Control Register.

Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size & Max_Read_Request_Size are used to decompose into AXI burst, but it seems the Max_Payload_Size & Max_Read_Request_Size are set by default to 512 but the internal Max_Payload_Size_Supported is set to 256, thus changing these values to 256 at runtime to match and optimize bandwidth.

It's said, "Reducing Outbound Decomposition" :
- "Ensure that your application master does not generate bursts of size greater than or equal to Max_Payload_Size"

- "Program your PCIe system with a larger value of Max_Payload_Size without exceeding Max_Payload_Size_Supported"

- "Program your PCIe system with a larger value of Max_Read_Request without exceeding Max_Payload_Size_Supported:

So leaving 512 in Max_Payload_Size & Max_Read_Request leads to Outbound Decomposition which decreases PCIe link and degrades the AXI bus by doubling the bursts, leading to this fix to avoid overflowing the AXI bus.

So it seems to be still needed, I assume this *should* be handled in the core somehow to propagate these settings to child endpoints to match the root port Max_Payload_Size & Max_Read_Request sizes.

Maybe by adding a core function to set these values instead of using the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers and set a state on the root port to propagate the value ?

Quoted 5 lines
Quoted 3 lines

I don't have the Synopsys DW-PCIe Databook, so I'm lacking any context. The above *seems* to say that MPS/MRRS settings affect AXI bus usage.

It does when the TLPs are directed to the RC.

That's a defect in the RC.

Quoted 12 lines
Quoted 4 lines

The MPS and MRRS registers are defined to affect traffic on *PCIe*. If a platform uses MPS and MRRS values to optimize transfers on non-PCIe links, that's a problem because the PCI core code that manages MPS and MRRS has no knowledge of those non-PCIe parts of the system.

Yes and no, it only affects PCIe in P2P, in non-P2P is will certainly affect transfers on the internal SoC/Processor/Chip internal bus/fabric.

Quoted 3 lines

You might be able to deal with this in Synopsys-specific code somehow, but it's going to be a bit of a hassle because I don't want it to make maintenance of the generic MPS/MRRS code harder.

Quoted 6 lines

I understand, but this is why these quirks are currently implemented in the controller driver and only applies when the controller has been probed and to each endpoint detected on this particular controller.

So we may continue having separate quirks for each controller if the core isn't the right place to handle MPS/MRRS.

Quoted 6 lines

The PCI core is the correct place to handle MPS/MRRS because their behavior is defined by the PCIe spec.

Quirks are the way to work around this defect in the Synopsys PCIe IP.

Bjorn

_______________________________________________
linux-amlogic mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-amlogic