Skip to contentSkip to conversation

PCI: tegra: Use PCI_CONF1_EXT_ADDRESS() macro

PATCH Applied Pali Rohár 4 participants 10 messages Sep 28, 2022 – Oct 17, 2022

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 PCI: tegra: Use PCI_CONF1_EXT_ADDRESS() macro

Simplify pci-tegra.c driver code and use new PCI_CONF1_EXT_ADDRESS() macro for accessing PCI config space.

---
Please look also at this related patch:
https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/

---
 drivers/pci/controller/pci-tegra.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
drivers/pci/controller/pci-tegra.c +3 -8
index 5df90d183526..c9924e75e597 100644
@@ -417,13 +417,6 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
  * address (access to which generates correct config transaction) falls in
  * this 4 KiB region.
  */
-static unsigned int tegra_pcie_conf_offset(u8 bus, unsigned int devfn,
-					   unsigned int where)
-{
-	return ((where & 0xf00) << 16) | (bus << 16) | (PCI_SLOT(devfn) << 11) |
-	       (PCI_FUNC(devfn) << 8) | (where & 0xff);
-}
-
 static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
 					unsigned int devfn,
 					int where)
@@ -445,7 +438,9 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
 		unsigned int offset;
 		u32 base;
 
-		offset = tegra_pcie_conf_offset(bus->number, devfn, where);
+		offset = PCI_CONF1_EXT_ADDRESS(bus->number, PCI_SLOT(devfn),
+					       PCI_FUNC(devfn), where) &
+			 ~PCI_CONF1_ENABLE;
 
 		/* move 4 KiB window to offset within the FPCI region */
 		base = 0xfe100000 + ((offset & ~(SZ_4K - 1)) >> 8);
Signature · 1 line
2.20.1
Acked-by on v1 1/1
#1 lore raw
Quoted 10 lines · Pali Rohár

Simplify pci-tegra.c driver code and use new PCI_CONF1_EXT_ADDRESS() macro for accessing PCI config space.

---
Please look also at this related patch:
https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/

---
 drivers/pci/controller/pci-tegra.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

I had to go chase down the patch that introduces PCI_CONF1_EXT_ADDRESS. It would've been easier if this had been part of the series that introduced that, or if you had provided a link to that patch here.

Anyway, looks like this is equivalent to the existing inline function, so:

#2 lore raw
Quoted 12 lines · Thierry Reding
Quoted 10 lines · Pali Rohár

Simplify pci-tegra.c driver code and use new PCI_CONF1_EXT_ADDRESS() macro for accessing PCI config space.

---
Please look also at this related patch:
https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/

---
 drivers/pci/controller/pci-tegra.c | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
Quoted 8 lines

I had to go chase down the patch that introduces PCI_CONF1_EXT_ADDRESS. It would've been easier if this had been part of the series that introduced that, or if you had provided a link to that patch here.

Anyway, looks like this is equivalent to the existing inline function, so:

After looking at the linked patch, perhaps revise this one more time and remove the comment above the removed helper since it's now just a duplication of what the PCI_CONF1_EXT_ADDRESS comments say.

Thierry

maintainer
#4 lore raw
In reply to Pali Rohár
Quoted 30 lines · Pali Rohár

Simplify pci-tegra.c driver code and use new PCI_CONF1_EXT_ADDRESS() macro for accessing PCI config space.

---
Please look also at this related patch:
https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/

---
  drivers/pci/controller/pci-tegra.c | 11 +++--------
  1 file changed, 3 insertions(+), 8 deletions(-)
drivers/pci/controller/pci-tegra.c +0 -7
index 5df90d183526..c9924e75e597 100644
@@ -417,13 +417,6 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
   * address (access to which generates correct config transaction) falls in
   * this 4 KiB region.
   */
-static unsigned int tegra_pcie_conf_offset(u8 bus, unsigned int devfn,
-					   unsigned int where)
-{
-	return ((where & 0xf00) << 16) | (bus << 16) | (PCI_SLOT(devfn) << 11) |
-	       (PCI_FUNC(devfn) << 8) | (where & 0xff);
-}
-
  static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
  					unsigned int devfn,
  					int where)

drivers/pci/controller/pci-tegra.c static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus, · line 446

Quoted 10 lines
drivers/pci/controller/pci-tegra.c +3 -1
@@ -445,7 +438,9 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
  		unsigned int offset;
  		u32 base;
 
-		offset = tegra_pcie_conf_offset(bus->number, devfn, where);
+		offset = PCI_CONF1_EXT_ADDRESS(bus->number, PCI_SLOT(devfn),
+					       PCI_FUNC(devfn), where) &
+			 ~PCI_CONF1_ENABLE;
 
  		/* move 4 KiB window to offset within the FPCI region */
  		base = 0xfe100000 + ((offset & ~(SZ_4K - 1)) >> 8);

Our PCIe test on Tegra124 Jetson TK1 is currently failing on -next and bisect points to this commit. Looking at bit closer, the problem appears to be the PCI_CONF1_REG_MASK which has a value of 0xfc. Before this patch was applied a mask of 0xff was applied to the lower 8-bits of 'where' and now it is 0xfc. So this does not work for Tegra as it is.

Let me know if you have any thoughts?

Jon

Signature · 1 line
nvpublic
#5 lore raw
Quoted 47 lines · On Tuesday 11 October 2022 16:42:34 Jon Hunter wrote:
Quoted 30 lines · Pali Rohár

Simplify pci-tegra.c driver code and use new PCI_CONF1_EXT_ADDRESS() macro for accessing PCI config space.

---
Please look also at this related patch:
https://patchwork.kernel.org/project/linux-pci/patch/[email protected]/

---
  drivers/pci/controller/pci-tegra.c | 11 +++--------
  1 file changed, 3 insertions(+), 8 deletions(-)
drivers/pci/controller/pci-tegra.c +0 -7
index 5df90d183526..c9924e75e597 100644
@@ -417,13 +417,6 @@ static inline u32 pads_readl(struct tegra_pcie *pcie, unsigned long offset)
   * address (access to which generates correct config transaction) falls in
   * this 4 KiB region.
   */
-static unsigned int tegra_pcie_conf_offset(u8 bus, unsigned int devfn,
-					   unsigned int where)
-{
-	return ((where & 0xf00) << 16) | (bus << 16) | (PCI_SLOT(devfn) << 11) |
-	       (PCI_FUNC(devfn) << 8) | (where & 0xff);
-}
-
  static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
  					unsigned int devfn,
  					int where)

drivers/pci/controller/pci-tegra.c static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus, · line 444

Quoted 8 lines
drivers/pci/controller/pci-tegra.c +3 -1
@@ -445,7 +438,9 @@ static void __iomem *tegra_pcie_map_bus(struct pci_bus *bus,
  		unsigned int offset;
  		u32 base;
-		offset = tegra_pcie_conf_offset(bus->number, devfn, where);
+		offset = PCI_CONF1_EXT_ADDRESS(bus->number, PCI_SLOT(devfn),
+					       PCI_FUNC(devfn), where) &
+			 ~PCI_CONF1_ENABLE;
  		/* move 4 KiB window to offset within the FPCI region */
  		base = 0xfe100000 + ((offset & ~(SZ_4K - 1)) >> 8);

Our PCIe test on Tegra124 Jetson TK1 is currently failing on -next and bisect points to this commit. Looking at bit closer, the problem appears to be the PCI_CONF1_REG_MASK which has a value of 0xfc. Before this patch was applied a mask of 0xff was applied to the lower 8-bits of 'where' and now it is 0xfc. So this does not work for Tegra as it is.

Quoted 6 lines

Let me know if you have any thoughts?

Jon

Signature · 1 line
nvpublic

I see, this is stupid mistake. PCIe config read and write operations needs to be 4-byte aligned, so normally it is done by calculating 4-byte aligned base address and then using appropriate cpu load/store instruction to access just defined size/offset of 4-byte config space register.

pci-tegra.c is using common helper functions pci_generic_config_read() and pci_generic_config_write(), which expects final address with offset, and not 4-byte aligned address.

I'm not sure what should be the proper fix, but for me it looks like that pci_generic_config_read() and pci_generic_config_write() could be adjusted to handle it.

In any case, above patch is a regressions and I see there two options for now:

1) Reverting that patch

2) Adding "offset |= where & 0x3;" after the PCI_CONF1_EXT_ADDRESS() macro to set also lower 2 bits of accessed register.

Jon, Lorenzo, what do you think? Could you test if 2) is working fine?

maintainer
#6 lore raw

On 11/10/2022 17:16, Pali Rohár wrote:

...

Quoted 17 lines

I see, this is stupid mistake. PCIe config read and write operations needs to be 4-byte aligned, so normally it is done by calculating 4-byte aligned base address and then using appropriate cpu load/store instruction to access just defined size/offset of 4-byte config space register.

pci-tegra.c is using common helper functions pci_generic_config_read() and pci_generic_config_write(), which expects final address with offset, and not 4-byte aligned address.

I'm not sure what should be the proper fix, but for me it looks like that pci_generic_config_read() and pci_generic_config_write() could be adjusted to handle it.

In any case, above patch is a regressions and I see there two options for now:

Quoted 6 lines

1) Reverting that patch

2) Adding "offset |= where & 0x3;" after the PCI_CONF1_EXT_ADDRESS() macro to set also lower 2 bits of accessed register.

Jon, Lorenzo, what do you think? Could you test if 2) is working fine?

I tested 'offset |= where & 0xff' which is essentially the same as the above and that is working and so I am sure that the above works too. However, I do wonder if reverting is simpler because we already have a '& ~PCI_CONF1_ENABLE' and now adding '| where & 0x3' seems to diminish the value of this change.

Cheers
Jon

Signature · 1 line
nvpublic
#7 lore raw
Quoted 29 lines · On Tuesday 11 October 2022 17:47:50 Jon Hunter wrote:

On 11/10/2022 17:16, Pali Rohár wrote:

...

Quoted 17 lines

I see, this is stupid mistake. PCIe config read and write operations needs to be 4-byte aligned, so normally it is done by calculating 4-byte aligned base address and then using appropriate cpu load/store instruction to access just defined size/offset of 4-byte config space register.

pci-tegra.c is using common helper functions pci_generic_config_read() and pci_generic_config_write(), which expects final address with offset, and not 4-byte aligned address.

I'm not sure what should be the proper fix, but for me it looks like that pci_generic_config_read() and pci_generic_config_write() could be adjusted to handle it.

In any case, above patch is a regressions and I see there two options for now:

Quoted 6 lines

1) Reverting that patch

2) Adding "offset |= where & 0x3;" after the PCI_CONF1_EXT_ADDRESS() macro to set also lower 2 bits of accessed register.

Jon, Lorenzo, what do you think? Could you test if 2) is working fine?

Quoted 11 lines

I tested 'offset |= where & 0xff' which is essentially the same as the above and that is working and so I am sure that the above works too. However, I do wonder if reverting is simpler because we already have a '& ~PCI_CONF1_ENABLE' and now adding '| where & 0x3' seems to diminish the value of this change.

Cheers
Jon

Signature · 1 line
nvpublic

Well, if decision would be that pci_generic_config_read() could be modified in the future to handle aligning, then '|= where & 0x3' block would be moved from driver to generic function...

I'm really not sure which option to choose.

#8 lore raw
In reply to Jon Hunter
Quoted 27 lines · Jon Hunter

On 11/10/2022 17:16, Pali Rohár wrote:

...

Quoted 22 lines

I see, this is stupid mistake. PCIe config read and write operations needs to be 4-byte aligned, so normally it is done by calculating 4-byte aligned base address and then using appropriate cpu load/store instruction to access just defined size/offset of 4-byte config space register.

pci-tegra.c is using common helper functions pci_generic_config_read() and pci_generic_config_write(), which expects final address with offset, and not 4-byte aligned address.

I'm not sure what should be the proper fix, but for me it looks like that pci_generic_config_read() and pci_generic_config_write() could be adjusted to handle it.

In any case, above patch is a regressions and I see there two options for now:

1) Reverting that patch

2) Adding "offset |= where & 0x3;" after the PCI_CONF1_EXT_ADDRESS() macro to set also lower 2 bits of accessed register.

Quoted 8 lines
Quoted 1 line

Jon, Lorenzo, what do you think? Could you test if 2) is working fine?

I tested 'offset |= where & 0xff' which is essentially the same as the above and that is working and so I am sure that the above works too. However, I do wonder if reverting is simpler because we already have a '& ~PCI_CONF1_ENABLE' and now adding '| where & 0x3' seems to diminish the value of this change.

Hi Jon,

it is unfortunate but I think we should proceed with a revert, please send it and I shall ask Bjorn to send it for one of the upcoming -rcX.

Thanks,
Lorenzo

maintainer
#9 lore raw
Quoted 37 lines · Lorenzo Pieralisi
Quoted 27 lines · Jon Hunter

On 11/10/2022 17:16, Pali Rohár wrote:

...

Quoted 22 lines

I see, this is stupid mistake. PCIe config read and write operations needs to be 4-byte aligned, so normally it is done by calculating 4-byte aligned base address and then using appropriate cpu load/store instruction to access just defined size/offset of 4-byte config space register.

pci-tegra.c is using common helper functions pci_generic_config_read() and pci_generic_config_write(), which expects final address with offset, and not 4-byte aligned address.

I'm not sure what should be the proper fix, but for me it looks like that pci_generic_config_read() and pci_generic_config_write() could be adjusted to handle it.

In any case, above patch is a regressions and I see there two options for now:

1) Reverting that patch

2) Adding "offset |= where & 0x3;" after the PCI_CONF1_EXT_ADDRESS() macro to set also lower 2 bits of accessed register.

Quoted 8 lines
Quoted 1 line

Jon, Lorenzo, what do you think? Could you test if 2) is working fine?

I tested 'offset |= where & 0xff' which is essentially the same as the above and that is working and so I am sure that the above works too. However, I do wonder if reverting is simpler because we already have a '& ~PCI_CONF1_ENABLE' and now adding '| where & 0x3' seems to diminish the value of this change.

Quoted 4 lines

Hi Jon,

it is unfortunate but I think we should proceed with a revert, please send it and I shall ask Bjorn to send it for one of the upcoming -rcX.

I have sent a revert for this change.

Thanks
Jon

Signature · 1 line
nvpublic