Skip to contentSkip to conversation

PCIe: limit Max Read Request Size on i.MX to 512 bytes

PATCH Superseded Krzysztof Hałasa 3 participants 5 messages Aug 16, 2021 – Aug 17, 2021

Subsystems (4)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 PCIe: limit Max Read Request Size on i.MX to 512 bytes

DWC PCIe controller imposes limits on the Read Request Size that it can handle. For i.MX6 family it's fixed at 512 bytes by default.

If a memory read larger than the limit is requested, the CPU responds with Completer Abort (CA) (on i.MX6 Unsupported Request (UR) is returned instead due to a design error).

The i.MX6 documentation states that the limit can be changed by writing to the PCIE_PL_MRCCR0 register, however there is a fixed (and undocumented) maximum (CX_REMOTE_RD_REQ_SIZE constant). Tests indicate that values larger than 512 bytes don't work, though.

This patch makes the RTL8111 work on i.MX6.

---
This version drops CONFIG_NEED_PCIE_MAX_MRRS.

drivers/pci/controller/dwc/pci-imx6.c +1 -0
index 80fc98acf097..225380e75fff 100644
@@ -1148,6 +1148,7 @@ static int imx6_pcie_probe(struct platform_device *pdev)
 		imx6_pcie->vph = NULL;
 	}
 
+	max_pcie_mrrs = 512;
 	platform_set_drvdata(pdev, imx6_pcie);
 
 	ret = imx6_pcie_attach_pd(dev);
drivers/pci/pci.c +9 -0
index aacf575c15cf..44815af4ad85 100644
@@ -112,6 +112,10 @@ enum pcie_bus_config_types pcie_bus_config = PCIE_BUS_PEER2PEER;
 enum pcie_bus_config_types pcie_bus_config = PCIE_BUS_DEFAULT;
 #endif
 
+#ifdef CONFIG_ARM
+u16 max_pcie_mrrs = 4096; // no limit - needed mostly for DWC PCIe
+#endif
+
 /*
  * The default CLS is used if arch didn't set CLS explicitly and not
  * all pci devices agree on the same value.  Arch can override either
@@ -5816,6 +5820,11 @@ int pcie_set_readrq(struct pci_dev *dev, int rq)
 			rq = mps;
 	}
 
+#ifdef CONFIG_ARM
+	if (rq > max_pcie_mrrs)
+		rq = max_pcie_mrrs;
+#endif
+
 	v = (ffs(rq) - 8) << 12;
 
 	ret = pcie_capability_clear_and_set_word(dev, PCI_EXP_DEVCTL,
include/linux/pci.h +1 -0
index 06ff1186c1ef..1f21ec662b8e 100644
@@ -996,6 +996,7 @@ enum pcie_bus_config_types {
 };
 
 extern enum pcie_bus_config_types pcie_bus_config;
+extern u16 max_pcie_mrrs;	// currently ARM only
 
 extern struct bus_type pci_bus_type;
 
Signature · 5 lines
Krzysztof "Chris" Hałasa

Sieć Badawcza Łukasiewicz
Przemysłowy Instytut Automatyki i Pomiarów PIAP
Al. Jerozolimskie 202, 02-486 Warszawa
reviewer
#1 lore raw
Quoted 47 lines · Krzysztof Hałasa

DWC PCIe controller imposes limits on the Read Request Size that it can handle. For i.MX6 family it's fixed at 512 bytes by default.

If a memory read larger than the limit is requested, the CPU responds with Completer Abort (CA) (on i.MX6 Unsupported Request (UR) is returned instead due to a design error).

The i.MX6 documentation states that the limit can be changed by writing to the PCIE_PL_MRCCR0 register, however there is a fixed (and undocumented) maximum (CX_REMOTE_RD_REQ_SIZE constant). Tests indicate that values larger than 512 bytes don't work, though.

This patch makes the RTL8111 work on i.MX6.

---
This version drops CONFIG_NEED_PCIE_MAX_MRRS.

drivers/pci/controller/dwc/pci-imx6.c +1 -0
index 80fc98acf097..225380e75fff 100644
@@ -1148,6 +1148,7 @@ static int imx6_pcie_probe(struct platform_device *pdev)
                imx6_pcie->vph = NULL;
        }
 
+       max_pcie_mrrs = 512;
        platform_set_drvdata(pdev, imx6_pcie);
 
        ret = imx6_pcie_attach_pd(dev);
drivers/pci/pci.c +4 -0
index aacf575c15cf..44815af4ad85 100644
@@ -112,6 +112,10 @@ enum pcie_bus_config_types pcie_bus_config = PCIE_BUS_PEER2PEER;
 enum pcie_bus_config_types pcie_bus_config = PCIE_BUS_DEFAULT;
 #endif
 
+#ifdef CONFIG_ARM
+u16 max_pcie_mrrs = 4096; // no limit - needed mostly for DWC PCIe
+#endif
+
 /*
  * The default CLS is used if arch didn't set CLS explicitly and not
  * all pci devices agree on the same value.  Arch can override either

drivers/pci/pci.c int pcie_set_readrq(struct pci_dev *dev, int rq) · line 5826

Quoted 7 lines
drivers/pci/pci.c +4 -0
@@ -5816,6 +5820,11 @@ int pcie_set_readrq(struct pci_dev *dev, int rq)
                        rq = mps;
        }
 
+#ifdef CONFIG_ARM
+       if (rq > max_pcie_mrrs)
+               rq = max_pcie_mrrs;
+#endif

My objection wasn't having another kconfig option so much as I don't think we need one at all here unless Bjorn feels otherwise. It's 2 bytes of data and about 3 instructions (load, cmp, store).

If we do have a config option, using or basing on the arch is wrong. Has nothing to do with the arch. Are the other platforms needing this arm32 as well?

Also, when you do use kconfig options, use IS_ENABLED() whenever possible.

Rob

#2 lore raw
Quoted 9 lines · Rob Herring

drivers/pci/pci.c int pcie_set_readrq(struct pci_dev *dev, int rq) · line 5826

Quoted 4 lines
drivers/pci/pci.c +4 -0
+#ifdef CONFIG_ARM
+       if (rq > max_pcie_mrrs)
+               rq = max_pcie_mrrs;
+#endif

My objection wasn't having another kconfig option so much as I don't think we need one at all here unless Bjorn feels otherwise. It's 2 bytes of data and about 3 instructions (load, cmp, store).

Quoted 3 lines

If we do have a config option, using or basing on the arch is wrong. Has nothing to do with the arch. Are the other platforms needing this arm32 as well?

Yes,

I can buy the "universal ARM32 kernel" argument, but otherwise it's just nonfunctional bloat. A small one, yes.

Signature · 5 lines
Krzysztof "Chris" Hałasa

Sieć Badawcza Łukasiewicz
Przemysłowy Instytut Automatyki i Pomiarów PIAP
Al. Jerozolimskie 202, 02-486 Warszawa
#3 lore raw
In reply to Krzysztof Hałasa
Thank you for the patch! Yet something to improve:

Hi "Krzysztof,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on pci/next]
[also build test ERROR on v5.14-rc6 next-20210816]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Krzysztof-Ha-asa/PCIe-limit-Max-Read-Request-Size-on-i-MX-to-512-bytes/20210816-193000
base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: parisc-allyesconfig (attached as .config)
compiler: hppa-linux-gcc (GCC) 11.2.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/63e17da24ae7c5ee07f3a34fc32cff6455e5b0c5
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Krzysztof-Ha-asa/PCIe-limit-Max-Read-Request-Size-on-i-MX-to-512-bytes/20210816-193000 git checkout 63e17da24ae7c5ee07f3a34fc32cff6455e5b0c5
# save the attached .config to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=parisc SHELL=/bin/bash

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

hppa-linux-ld: drivers/pci/controller/dwc/pci-imx6.o: in function `imx6_pcie_probe':

>> (.text+0x698): undefined reference to `max_pcie_mrrs'
>> hppa-linux-ld: (.text+0x6a4): undefined reference to `max_pcie_mrrs'

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#4 lore raw
In reply to Krzysztof Hałasa
Thank you for the patch! Yet something to improve:

Hi "Krzysztof,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on pci/next]
[also build test ERROR on v5.14-rc6 next-20210816]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Krzysztof-Ha-asa/PCIe-limit-Max-Read-Request-Size-on-i-MX-to-512-bytes/20210816-193000
base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: arm64-randconfig-p001-20210816 (attached as .config)
compiler: aarch64-linux-gcc (GCC) 11.2.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/63e17da24ae7c5ee07f3a34fc32cff6455e5b0c5
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Krzysztof-Ha-asa/PCIe-limit-Max-Read-Request-Size-on-i-MX-to-512-bytes/20210816-193000 git checkout 63e17da24ae7c5ee07f3a34fc32cff6455e5b0c5
# save the attached .config to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=arm64 SHELL=/bin/bash

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

aarch64-linux-ld: drivers/pci/controller/dwc/pci-imx6.o: in function `imx6_pcie_probe':
pci-imx6.c:(.text+0x7c8): undefined reference to `max_pcie_mrrs'
aarch64-linux-ld: drivers/pci/controller/dwc/pci-imx6.o: relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `max_pcie_mrrs' which may bind externally can not be used when making a shared object; recompile with -fPIC pci-imx6.c:(.text+0x7c8): dangerous relocation: unsupported relocation

>> aarch64-linux-ld: pci-imx6.c:(.text+0x7d4): undefined reference to `max_pcie_mrrs'

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org