Skip to contentSkip to conversation

PCI: rcar: Clean up remaining macros defining bits

PATCH In review [email protected] 4 participants 17 messages Apr 2, 2019 – Apr 4, 2019

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH v5 1/6 PCI: rcar: Clean up remaining macros defining bits

From: Marek Vasut <[email protected]>

Replace macros using constants with BIT()s instead, no functional change.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Add Geert's R-B
V5: Rebase on next/master 20190401

---
 drivers/pci/controller/pcie-rcar.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/pci/controller/pcie-rcar.c +3 -3
index 6a4e435bd35f..542b5bbf4bf1 100644
@@ -47,14 +47,14 @@
 /* Transfer control */
 #define PCIETCTLR		0x02000
 #define  DL_DOWN		BIT(3)
-#define  CFINIT			1
+#define  CFINIT			BIT(0)
 #define PCIETSTR		0x02004
-#define  DATA_LINK_ACTIVE	1
+#define  DATA_LINK_ACTIVE	BIT(0)
 #define PCIEERRFR		0x02020
 #define  UNSUPPORTED_REQUEST	BIT(4)
 #define PCIEMSIFR		0x02044
 #define PCIEMSIALR		0x02048
-#define  MSIFE			1
+#define  MSIFE			BIT(0)
 #define PCIEMSIAUR		0x0204c
 #define PCIEMSIIER		0x02050
 
Signature · 1 line
2.20.1
#1 lore raw

PATCH v5 2/6 PCI: rcar: Replace unsigned long with u32/unsigned int in register accessors

From: Marek Vasut <[email protected]>

Replace unsigned long with u32 and unsigned int in register accessor functions, since they access 32bit registers.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Change reg to unsigned int
V5: Rebase on next/master 20190401

---
 drivers/pci/controller/pcie-rcar.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
drivers/pci/controller/pcie-rcar.c +3 -4
index 542b5bbf4bf1..317f688b1fd2 100644
@@ -154,14 +154,13 @@ struct rcar_pcie {
 	struct			rcar_msi msi;
 };
 
-static void rcar_pci_write_reg(struct rcar_pcie *pcie, unsigned long val,
-			       unsigned long reg)
+static void rcar_pci_write_reg(struct rcar_pcie *pcie, u32 val,
+			       unsigned int reg)
 {
 	writel(val, pcie->base + reg);
 }
 
-static unsigned long rcar_pci_read_reg(struct rcar_pcie *pcie,
-				       unsigned long reg)
+static u32 rcar_pci_read_reg(struct rcar_pcie *pcie, unsigned int reg)
 {
 	return readl(pcie->base + reg);
 }
Signature · 1 line
2.20.1
#3 lore raw

PATCH v5 3/6 PCI: rcar: Replace various variable types with unsigned ones for register values

In reply to [email protected]

From: Marek Vasut <[email protected]>

Replace various variable types with u32 or unsigned int type for variables holding register values, since the registers are 32bit. Note that rcar_pcie_msi_irq() still uses various variable types because both find_first_bit() and __fls() require various variable types as an argument.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: - s@unsigned long@various variable types@ in the commit message
- Replace int with unsigned int for the $shift variable
- Replace int with unsigned int / u32 in rcar_pcie_config_access()
V3: - Change shift from u32 to unsigned int
- Change addr and data in phy_write_reg() from unsigned int to u32
V4: - Change reg to unsigned int in rcar_pcie_config_access()
V5: Rebase on next/master 20190401

---
 drivers/pci/controller/pcie-rcar.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
drivers/pci/controller/pcie-rcar.c +9 -8
index 317f688b1fd2..9d0e63945842 100644
@@ -172,7 +172,7 @@ enum {
 
 static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 {
-	int shift = 8 * (where & 3);
+	unsigned int shift = 8 * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	val &= ~(mask << shift);
@@ -182,7 +182,7 @@ static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 
 static u32 rcar_read_conf(struct rcar_pcie *pcie, int where)
 {
-	int shift = 8 * (where & 3);
+	unsigned int shift = 8 * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	return val >> shift;
@@ -193,7 +193,7 @@ static int rcar_pcie_config_access(struct rcar_pcie *pcie,
 		unsigned char access_type, struct pci_bus *bus,
 		unsigned int devfn, int where, u32 *data)
 {
-	int dev, func, reg, index;
+	unsigned int dev, func, reg, index;
 
 	dev = PCI_SLOT(devfn);
 	func = PCI_FUNC(devfn);
@@ -297,8 +297,9 @@ static int rcar_pcie_write_conf(struct pci_bus *bus, unsigned int devfn,
 				int where, int size, u32 val)
 {
 	struct rcar_pcie *pcie = bus->sysdata;
-	int shift, ret;
+	unsigned int shift;
 	u32 data;
+	int ret;
 
 	ret = rcar_pcie_config_access(pcie, RCAR_PCI_ACCESS_READ,
 				      bus, devfn, where, &data);
@@ -508,10 +509,10 @@ static int phy_wait_for_ack(struct rcar_pcie *pcie)
 }
 
 static void phy_write_reg(struct rcar_pcie *pcie,
-				 unsigned int rate, unsigned int addr,
-				 unsigned int lane, unsigned int data)
+			  unsigned int rate, u32 addr,
+			  unsigned int lane, u32 data)
 {
-	unsigned long phyaddr;
+	u32 phyaddr;
 
 	phyaddr = WRITE_CMD |
 		((rate & 1) << RATE_POS) |
@@ -1119,7 +1120,7 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
 	struct rcar_pcie *pcie;
-	unsigned int data;
+	u32 data;
 	int err;
 	int (*phy_init_fn)(struct rcar_pcie *);
 	struct pci_host_bridge *bridge;
Signature · 1 line
2.20.1
#6 lore raw

PATCH v5 4/6 PCI: rcar: Replace (8 * n) with (BITS_PER_BYTE * n)

In reply to [email protected]

From: Marek Vasut <[email protected]>

Replace (8 * n) with (BITS_PER_BYTE * n) to make bit shift operations consistent. No functional change.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Change the patch from n << 3 to BITS_PER_BYTE * n
V5: Rebase on next/master 20190401

---
 drivers/pci/controller/pcie-rcar.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/pci/controller/pcie-rcar.c +6 -6
index 9d0e63945842..d9b8c7cd9fba 100644
@@ -172,7 +172,7 @@ enum {
 
 static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 {
-	unsigned int shift = 8 * (where & 3);
+	unsigned int shift = BITS_PER_BYTE * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	val &= ~(mask << shift);
@@ -182,7 +182,7 @@ static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 
 static u32 rcar_read_conf(struct rcar_pcie *pcie, int where)
 {
-	unsigned int shift = 8 * (where & 3);
+	unsigned int shift = BITS_PER_BYTE * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	return val >> shift;
@@ -282,9 +282,9 @@ static int rcar_pcie_read_conf(struct pci_bus *bus, unsigned int devfn,
 	}
 
 	if (size == 1)
-		*val = (*val >> (8 * (where & 3))) & 0xff;
+		*val = (*val >> (BITS_PER_BYTE * (where & 3))) & 0xff;
 	else if (size == 2)
-		*val = (*val >> (8 * (where & 2))) & 0xffff;
+		*val = (*val >> (BITS_PER_BYTE * (where & 2))) & 0xffff;
 
 	dev_dbg(&bus->dev, "pcie-config-read: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08lx\n",
 		bus->number, devfn, where, size, (unsigned long)*val);
@@ -310,11 +310,11 @@ static int rcar_pcie_write_conf(struct pci_bus *bus, unsigned int devfn,
 		bus->number, devfn, where, size, (unsigned long)val);
 
 	if (size == 1) {
-		shift = 8 * (where & 3);
+		shift = BITS_PER_BYTE * (where & 3);
 		data &= ~(0xff << shift);
 		data |= ((val & 0xff) << shift);
 	} else if (size == 2) {
-		shift = 8 * (where & 2);
+		shift = BITS_PER_BYTE * (where & 2);
 		data &= ~(0xffff << shift);
 		data |= ((val & 0xffff) << shift);
 	} else
Signature · 1 line
2.20.1
#8 lore raw

PATCH v5 5/6 PCI: rcar: Clean up debug messages

In reply to [email protected]

From: Marek Vasut <[email protected]>

Drop useless casts from debug messages, they are no longer needed due to the data type cleanup.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: - Bundle this patch with other cleanups before resending
- Add R-B from Geert
V3: Add Wolfram's R-B
V4: No change
V5: Rebase on next/master 20190401

---
 drivers/pci/controller/pcie-rcar.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/pci/controller/pcie-rcar.c +4 -4
index d9b8c7cd9fba..168bc6b9bb93 100644
@@ -286,8 +286,8 @@ static int rcar_pcie_read_conf(struct pci_bus *bus, unsigned int devfn,
 	else if (size == 2)
 		*val = (*val >> (BITS_PER_BYTE * (where & 2))) & 0xffff;
 
-	dev_dbg(&bus->dev, "pcie-config-read: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08lx\n",
-		bus->number, devfn, where, size, (unsigned long)*val);
+	dev_dbg(&bus->dev, "pcie-config-read: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08x\n",
+		bus->number, devfn, where, size, *val);
 
 	return ret;
 }
@@ -306,8 +306,8 @@ static int rcar_pcie_write_conf(struct pci_bus *bus, unsigned int devfn,
 	if (ret != PCIBIOS_SUCCESSFUL)
 		return ret;
 
-	dev_dbg(&bus->dev, "pcie-config-write: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08lx\n",
-		bus->number, devfn, where, size, (unsigned long)val);
+	dev_dbg(&bus->dev, "pcie-config-write: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08x\n",
+		bus->number, devfn, where, size, val);
 
 	if (size == 1) {
 		shift = BITS_PER_BYTE * (where & 3);
Signature · 1 line
2.20.1
#10 lore raw

PATCH v5 6/6 PCI: rcar: Fix 64bit MSI message address handling

In reply to [email protected]

From: Marek Vasut <[email protected]>

The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: - s/it's/its/ in commit message
- Add R-B from Geert
V3: - Reworded commit message and thus dropped Geerts R-B
V4: - Add Geert's R-B again
V5: - Rebase on next/master 20190401
- Use {lower,upper}_32_bits() instead of >> 32

---
 drivers/pci/controller/pcie-rcar.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/pci/controller/pcie-rcar.c +3 -3
index 168bc6b9bb93..5e0102796345 100644
@@ -892,7 +892,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct rcar_msi *msi = &pcie->msi;
-	unsigned long base;
+	phys_addr_t base;
 	int err, i;
 
 	mutex_init(&msi->lock);
@@ -933,8 +933,8 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 	msi->pages = __get_free_pages(GFP_KERNEL, 0);
 	base = virt_to_phys((void *)msi->pages);
 
-	rcar_pci_write_reg(pcie, base | MSIFE, PCIEMSIALR);
-	rcar_pci_write_reg(pcie, 0, PCIEMSIAUR);
+	rcar_pci_write_reg(pcie, lower_32_bits(base) | MSIFE, PCIEMSIALR);
+	rcar_pci_write_reg(pcie, upper_32_bits(base), PCIEMSIAUR);
 
 	/* enable all MSI interrupts */
 	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
Signature · 1 line
2.20.1
Reviewed-by on v5 3/6
#4 lore raw

Re: [PATCH V5 3/6] PCI: rcar: Replace various variable types with unsigned ones for register values

In reply to [email protected]
Quoted 8 lines · On Tue, Apr 02, 2019 at 03:33:04AM +0200, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace various variable types with u32 or unsigned int type for variables holding register values, since the registers are 32bit. Note that rcar_pcie_msi_irq() still uses various variable types because both find_first_bit() and __fls() require various variable types as an argument.

Quoted 8 lines
Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Good that you found phy_write_reg meanwhile. I wanted to comment on that back then but forgot the mail in the Draft folder :(

Reviewed-by on v5 6/6
#11 lore raw

Re: [PATCH V5 6/6] PCI: rcar: Fix 64bit MSI message address handling

In reply to [email protected]
Quoted 22 lines · On Tue, Apr 02, 2019 at 03:33:07AM +0200, [email protected] wrote:

From: Marek Vasut <[email protected]>

The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

Quoted 6 lines

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Very good descripion!

Quoted 8 lines
Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

I double-checked with the datasheets previously.

Reviewed-by on v5 1/6
#16 lore raw
In reply to [email protected]
Quoted 10 lines · On Tue, Apr 02, 2019 at 03:33:02AM +0200, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace macros using constants with BIT()s instead, no functional change.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>
Quoted 3 lines

To: [email protected]

I feel a sense of déjà vu when reading this.

Quoted 34 lines

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Add Geert's R-B
V5: Rebase on next/master 20190401

---
 drivers/pci/controller/pcie-rcar.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/pci/controller/pcie-rcar.c +3 -3
index 6a4e435bd35f..542b5bbf4bf1 100644
@@ -47,14 +47,14 @@
 /* Transfer control */
 #define PCIETCTLR		0x02000
 #define  DL_DOWN		BIT(3)
-#define  CFINIT			1
+#define  CFINIT			BIT(0)
 #define PCIETSTR		0x02004
-#define  DATA_LINK_ACTIVE	1
+#define  DATA_LINK_ACTIVE	BIT(0)
 #define PCIEERRFR		0x02020
 #define  UNSUPPORTED_REQUEST	BIT(4)
 #define PCIEMSIFR		0x02044
 #define PCIEMSIALR		0x02048
-#define  MSIFE			1
+#define  MSIFE			BIT(0)
 #define PCIEMSIAUR		0x0204c
 #define PCIEMSIIER		0x02050
 
Signature · 1 line
2.20.1
Reviewed-by on v5 2/6
#2 lore raw

Re: [PATCH V5 2/6] PCI: rcar: Replace unsigned long with u32/unsigned int in register accessors

In reply to [email protected]
Quoted 11 lines · On Tue, Apr 02, 2019 at 03:33:03AM +0200, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace unsigned long with u32 and unsigned int in register accessor functions, since they access 32bit registers.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>
Quoted 3 lines

To: [email protected]

Reviewed-by on v5 4/6
#7 lore raw

Re: [PATCH V5 4/6] PCI: rcar: Replace (8 * n) with (BITS_PER_BYTE * n)

In reply to [email protected]
Quoted 11 lines · On Tue, Apr 02, 2019 at 03:33:05AM +0200, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace (8 * n) with (BITS_PER_BYTE * n) to make bit shift operations consistent. No functional change.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>
Quoted 3 lines

To: [email protected]

Reviewed-by on v5 5/6
#9 lore raw

Re: [PATCH V5 5/6] PCI: rcar: Clean up debug messages

In reply to [email protected]
Quoted 11 lines · On Tue, Apr 02, 2019 at 03:33:06AM +0200, [email protected] wrote:

From: Marek Vasut <[email protected]>

Drop useless casts from debug messages, they are no longer needed due to the data type cleanup.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>
Quoted 3 lines

To: [email protected]

Reviewed-by on v5 3/6
#5 lore raw

Re: [PATCH V5 3/6] PCI: rcar: Replace various variable types with unsigned ones for register values

In reply to [email protected]
Quoted 8 lines · On Tue, Apr 02, 2019 at 03:33:04AM +0200, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace various variable types with u32 or unsigned int type for variables holding register values, since the registers are 32bit. Note that rcar_pcie_msi_irq() still uses various variable types because both find_first_bit() and __fls() require various variable types as an argument.

Quoted 8 lines
Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Reviewed-by on v5 6/6
#12 lore raw

Re: [PATCH V5 6/6] PCI: rcar: Fix 64bit MSI message address handling

In reply to [email protected]
Quoted 29 lines · On Tue, Apr 02, 2019 at 03:33:07AM +0200, [email protected] wrote:

From: Marek Vasut <[email protected]>

The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Quoted 8 lines
Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

#13 lore raw

Re: [PATCH V5 6/6] PCI: rcar: Fix 64bit MSI message address handling

In reply to [email protected]
Quoted 36 lines · On Tue, Apr 02, 2019 at 03:33:07AM +0200, [email protected] wrote:

From: Marek Vasut <[email protected]>

The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 8 lines

---
V2: - s/it's/its/ in commit message
- Add R-B from Geert
V3: - Reworded commit message and thus dropped Geerts R-B
V4: - Add Geert's R-B again
V5: - Rebase on next/master 20190401
- Use {lower,upper}_32_bits() instead of >> 32

If that's the only reason you resent this series I will add the lower_32_bits() code myself.

Please do not rebase on top of next, apply code on top of a fixed -rc1 (we are currently using v5.1-rc1) and if there are dependencies on code already queued do let us know, we will handle conflicts in next ourselves.

Lorenzo

Quoted 31 lines
---
 drivers/pci/controller/pcie-rcar.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/pci/controller/pcie-rcar.c +3 -3
index 168bc6b9bb93..5e0102796345 100644
@@ -892,7 +892,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct rcar_msi *msi = &pcie->msi;
-	unsigned long base;
+	phys_addr_t base;
 	int err, i;
 
 	mutex_init(&msi->lock);
@@ -933,8 +933,8 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 	msi->pages = __get_free_pages(GFP_KERNEL, 0);
 	base = virt_to_phys((void *)msi->pages);
 
-	rcar_pci_write_reg(pcie, base | MSIFE, PCIEMSIALR);
-	rcar_pci_write_reg(pcie, 0, PCIEMSIAUR);
+	rcar_pci_write_reg(pcie, lower_32_bits(base) | MSIFE, PCIEMSIALR);
+	rcar_pci_write_reg(pcie, upper_32_bits(base), PCIEMSIAUR);
 
 	/* enable all MSI interrupts */
 	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
Signature · 1 line
2.20.1
#14 lore raw

Re: [PATCH V5 6/6] PCI: rcar: Fix 64bit MSI message address handling

Quoted 46 lines · Lorenzo Pieralisi
Quoted 36 lines · On Tue, Apr 02, 2019 at 03:33:07AM +0200, [email protected] wrote:

From: Marek Vasut <[email protected]>

The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 8 lines

---
V2: - s/it's/its/ in commit message
- Add R-B from Geert
V3: - Reworded commit message and thus dropped Geerts R-B
V4: - Add Geert's R-B again
V5: - Rebase on next/master 20190401
- Use {lower,upper}_32_bits() instead of >> 32

Quoted 2 lines

If that's the only reason you resent this series I will add the lower_32_bits() code myself.

Yes, you asked me to resend the whole series after the bot complained.

Quoted 4 lines

Please do not rebase on top of next, apply code on top of a fixed -rc1 (we are currently using v5.1-rc1) and if there are dependencies on code already queued do let us know, we will handle conflicts in next ourselves.

So do you want me to resend this one more time ?

Signature · 2 lines
Best regards,
Marek Vasut
#15 lore raw

Re: [PATCH V5 6/6] PCI: rcar: Fix 64bit MSI message address handling

Quoted 47 lines · Marek Vasut
Quoted 46 lines · Lorenzo Pieralisi
Quoted 44 lines · On Tue, Apr 02, 2019 at 03:33:07AM +0200, [email protected] wrote:

From: Marek Vasut <[email protected]>

The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: - s/it's/its/ in commit message
- Add R-B from Geert
V3: - Reworded commit message and thus dropped Geerts R-B
V4: - Add Geert's R-B again
V5: - Rebase on next/master 20190401
- Use {lower,upper}_32_bits() instead of >> 32

Quoted 4 lines
Quoted 2 lines

If that's the only reason you resent this series I will add the lower_32_bits() code myself.

Yes, you asked me to resend the whole series after the bot complained.

https://lists.01.org/pipermail/kbuild-all/2019-April/059428.html

Quoted 6 lines
Quoted 4 lines

Please do not rebase on top of next, apply code on top of a fixed -rc1 (we are currently using v5.1-rc1) and if there are dependencies on code already queued do let us know, we will handle conflicts in next ourselves.

So do you want me to resend this one more time ?

No, in the message above I wanted to say I would make the update myself.

Regardless, please never send patches aimed at the PCI tree on top on -next.

Thanks,
Lorenzo