Skip to contentSkip to conversation

PCI: rcar: Clean up remaining macros defining bits

PATCH Applied [email protected] 4 participants 24 messages Mar 25, 2019 – Mar 30, 2019

Subsystems (4)

v4 of 5 Applied Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/6 PCI: rcar: Clean up remaining macros defining bits

From: Marek Vasut <[email protected]>

Replace macros using constants with BIT()s instead, no functional change.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Add Geert's R-B

---
 drivers/pci/controller/pcie-rcar.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/pci/controller/pcie-rcar.c +3 -3
index c8febb009454..5b8736f0cd6b 100644
@@ -46,14 +46,14 @@
 
 /* Transfer control */
 #define PCIETCTLR		0x02000
-#define  CFINIT			1
+#define  CFINIT			BIT(0)
 #define PCIETSTR		0x02004
-#define  DATA_LINK_ACTIVE	1
+#define  DATA_LINK_ACTIVE	BIT(0)
 #define PCIEERRFR		0x02020
 #define  UNSUPPORTED_REQUEST	BIT(4)
 #define PCIEMSIFR		0x02044
 #define PCIEMSIALR		0x02048
-#define  MSIFE			1
+#define  MSIFE			BIT(0)
 #define PCIEMSIAUR		0x0204c
 #define PCIEMSIIER		0x02050
 
Signature · 1 line
2.20.1
#1 lore raw

PATCH v4 2/6 PCI: rcar: Replace unsigned long with u32/unsigned int in register accessors

From: Marek Vasut <[email protected]>

Replace unsigned long with u32 and unsigned int in register accessor functions, since they access 32bit registers.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Change reg to unsigned int

---
 drivers/pci/controller/pcie-rcar.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
drivers/pci/controller/pcie-rcar.c +3 -4
index 5b8736f0cd6b..3db693b4410e 100644
@@ -152,14 +152,13 @@ struct rcar_pcie {
 	struct			rcar_msi msi;
 };
 
-static void rcar_pci_write_reg(struct rcar_pcie *pcie, unsigned long val,
-			       unsigned long reg)
+static void rcar_pci_write_reg(struct rcar_pcie *pcie, u32 val,
+			       unsigned int reg)
 {
 	writel(val, pcie->base + reg);
 }
 
-static unsigned long rcar_pci_read_reg(struct rcar_pcie *pcie,
-				       unsigned long reg)
+static u32 rcar_pci_read_reg(struct rcar_pcie *pcie, unsigned int reg)
 {
 	return readl(pcie->base + reg);
 }
Signature · 1 line
2.20.1
Reviewed-by on v4 2/6
#2 lore raw

Re: [PATCH V4 2/6] PCI: rcar: Replace unsigned long with u32/unsigned int in register accessors

Quoted 11 lines · On Mon, Mar 25, 2019 at 12:40:57PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace unsigned long with u32 and unsigned int in register accessor functions, since they access 32bit registers.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>
Quoted 3 lines

To: [email protected]

Quoted 33 lines

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Change reg to unsigned int

---
 drivers/pci/controller/pcie-rcar.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
drivers/pci/controller/pcie-rcar.c +3 -4
index 5b8736f0cd6b..3db693b4410e 100644
@@ -152,14 +152,13 @@ struct rcar_pcie {
 	struct			rcar_msi msi;
 };
 
-static void rcar_pci_write_reg(struct rcar_pcie *pcie, unsigned long val,
-			       unsigned long reg)
+static void rcar_pci_write_reg(struct rcar_pcie *pcie, u32 val,
+			       unsigned int reg)
 {
 	writel(val, pcie->base + reg);
 }
 
-static unsigned long rcar_pci_read_reg(struct rcar_pcie *pcie,
-				       unsigned long reg)
+static u32 rcar_pci_read_reg(struct rcar_pcie *pcie, unsigned int reg)
 {
 	return readl(pcie->base + reg);
 }
Signature · 1 line
2.20.1
#3 lore raw

PATCH v4 3/6 PCI: rcar: Replace various variable types with unsigned ones for register values

In reply to [email protected]

From: Marek Vasut <[email protected]>

Replace various variable types with u32 or unsigned int type for variables holding register values, since the registers are 32bit. Note that rcar_pcie_msi_irq() still uses various variable types because both find_first_bit() and __fls() require various variable types as an argument.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: - s@unsigned long@various variable types@ in the commit message
- Replace int with unsigned int for the $shift variable
- Replace int with unsigned int / u32 in rcar_pcie_config_access()
V3: - Change shift from u32 to unsigned int
- Change addr and data in phy_write_reg() from unsigned int to u32
V4: - Change reg to unsigned int in rcar_pcie_config_access()

---
 drivers/pci/controller/pcie-rcar.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
drivers/pci/controller/pcie-rcar.c +9 -8
index 3db693b4410e..6699211fdc9a 100644
@@ -170,7 +170,7 @@ enum {
 
 static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 {
-	int shift = 8 * (where & 3);
+	unsigned int shift = 8 * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	val &= ~(mask << shift);
@@ -180,7 +180,7 @@ static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 
 static u32 rcar_read_conf(struct rcar_pcie *pcie, int where)
 {
-	int shift = 8 * (where & 3);
+	unsigned int shift = 8 * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	return val >> shift;
@@ -191,7 +191,7 @@ static int rcar_pcie_config_access(struct rcar_pcie *pcie,
 		unsigned char access_type, struct pci_bus *bus,
 		unsigned int devfn, int where, u32 *data)
 {
-	int dev, func, reg, index;
+	unsigned int dev, func, reg, index;
 
 	dev = PCI_SLOT(devfn);
 	func = PCI_FUNC(devfn);
@@ -295,8 +295,9 @@ static int rcar_pcie_write_conf(struct pci_bus *bus, unsigned int devfn,
 				int where, int size, u32 val)
 {
 	struct rcar_pcie *pcie = bus->sysdata;
-	int shift, ret;
+	unsigned int shift;
 	u32 data;
+	int ret;
 
 	ret = rcar_pcie_config_access(pcie, RCAR_PCI_ACCESS_READ,
 				      bus, devfn, where, &data);
@@ -506,10 +507,10 @@ static int phy_wait_for_ack(struct rcar_pcie *pcie)
 }
 
 static void phy_write_reg(struct rcar_pcie *pcie,
-				 unsigned int rate, unsigned int addr,
-				 unsigned int lane, unsigned int data)
+			  unsigned int rate, u32 addr,
+			  unsigned int lane, u32 data)
 {
-	unsigned long phyaddr;
+	u32 phyaddr;
 
 	phyaddr = WRITE_CMD |
 		((rate & 1) << RATE_POS) |
@@ -1117,7 +1118,7 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
 	struct rcar_pcie *pcie;
-	unsigned int data;
+	u32 data;
 	int err;
 	int (*phy_init_fn)(struct rcar_pcie *);
 	struct pci_host_bridge *bridge;
Signature · 1 line
2.20.1
Reviewed-by on v4 3/6
#4 lore raw

Re: [PATCH V4 3/6] PCI: rcar: Replace various variable types with unsigned ones for register values

Quoted 8 lines · On Mon, Mar 25, 2019 at 12:40:58PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace various variable types with u32 or unsigned int type for variables holding register values, since the registers are 32bit. Note that rcar_pcie_msi_irq() still uses various variable types because both find_first_bit() and __fls() require various variable types as an argument.

Quoted 8 lines
Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 79 lines

---
V2: - s@unsigned long@various variable types@ in the commit message
- Replace int with unsigned int for the $shift variable
- Replace int with unsigned int / u32 in rcar_pcie_config_access()
V3: - Change shift from u32 to unsigned int
- Change addr and data in phy_write_reg() from unsigned int to u32
V4: - Change reg to unsigned int in rcar_pcie_config_access()

---
 drivers/pci/controller/pcie-rcar.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
drivers/pci/controller/pcie-rcar.c +9 -8
index 3db693b4410e..6699211fdc9a 100644
@@ -170,7 +170,7 @@ enum {
 
 static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 {
-	int shift = 8 * (where & 3);
+	unsigned int shift = 8 * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	val &= ~(mask << shift);
@@ -180,7 +180,7 @@ static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 
 static u32 rcar_read_conf(struct rcar_pcie *pcie, int where)
 {
-	int shift = 8 * (where & 3);
+	unsigned int shift = 8 * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	return val >> shift;
@@ -191,7 +191,7 @@ static int rcar_pcie_config_access(struct rcar_pcie *pcie,
 		unsigned char access_type, struct pci_bus *bus,
 		unsigned int devfn, int where, u32 *data)
 {
-	int dev, func, reg, index;
+	unsigned int dev, func, reg, index;
 
 	dev = PCI_SLOT(devfn);
 	func = PCI_FUNC(devfn);
@@ -295,8 +295,9 @@ static int rcar_pcie_write_conf(struct pci_bus *bus, unsigned int devfn,
 				int where, int size, u32 val)
 {
 	struct rcar_pcie *pcie = bus->sysdata;
-	int shift, ret;
+	unsigned int shift;
 	u32 data;
+	int ret;
 
 	ret = rcar_pcie_config_access(pcie, RCAR_PCI_ACCESS_READ,
 				      bus, devfn, where, &data);
@@ -506,10 +507,10 @@ static int phy_wait_for_ack(struct rcar_pcie *pcie)
 }
 
 static void phy_write_reg(struct rcar_pcie *pcie,
-				 unsigned int rate, unsigned int addr,
-				 unsigned int lane, unsigned int data)
+			  unsigned int rate, u32 addr,
+			  unsigned int lane, u32 data)
 {
-	unsigned long phyaddr;
+	u32 phyaddr;
 
 	phyaddr = WRITE_CMD |
 		((rate & 1) << RATE_POS) |
@@ -1117,7 +1118,7 @@ static int rcar_pcie_probe(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
 	struct rcar_pcie *pcie;
-	unsigned int data;
+	u32 data;
 	int err;
 	int (*phy_init_fn)(struct rcar_pcie *);
 	struct pci_host_bridge *bridge;
Signature · 1 line
2.20.1
#5 lore raw

PATCH v4 4/6 PCI: rcar: Replace (8 * n) with (BITS_PER_BYTE * n)

In reply to [email protected]

From: Marek Vasut <[email protected]>

Replace (8 * n) with (BITS_PER_BYTE * n) to make bit shift operations consistent. No functional change.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Change the patch from n << 3 to BITS_PER_BYTE * n

---
 drivers/pci/controller/pcie-rcar.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/pci/controller/pcie-rcar.c +6 -6
index 6699211fdc9a..96210768e774 100644
@@ -170,7 +170,7 @@ enum {
 
 static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 {
-	unsigned int shift = 8 * (where & 3);
+	unsigned int shift = BITS_PER_BYTE * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	val &= ~(mask << shift);
@@ -180,7 +180,7 @@ static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 
 static u32 rcar_read_conf(struct rcar_pcie *pcie, int where)
 {
-	unsigned int shift = 8 * (where & 3);
+	unsigned int shift = BITS_PER_BYTE * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	return val >> shift;
@@ -280,9 +280,9 @@ static int rcar_pcie_read_conf(struct pci_bus *bus, unsigned int devfn,
 	}
 
 	if (size == 1)
-		*val = (*val >> (8 * (where & 3))) & 0xff;
+		*val = (*val >> (BITS_PER_BYTE * (where & 3))) & 0xff;
 	else if (size == 2)
-		*val = (*val >> (8 * (where & 2))) & 0xffff;
+		*val = (*val >> (BITS_PER_BYTE * (where & 2))) & 0xffff;
 
 	dev_dbg(&bus->dev, "pcie-config-read: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08lx\n",
 		bus->number, devfn, where, size, (unsigned long)*val);
@@ -308,11 +308,11 @@ static int rcar_pcie_write_conf(struct pci_bus *bus, unsigned int devfn,
 		bus->number, devfn, where, size, (unsigned long)val);
 
 	if (size == 1) {
-		shift = 8 * (where & 3);
+		shift = BITS_PER_BYTE * (where & 3);
 		data &= ~(0xff << shift);
 		data |= ((val & 0xff) << shift);
 	} else if (size == 2) {
-		shift = 8 * (where & 2);
+		shift = BITS_PER_BYTE * (where & 2);
 		data &= ~(0xffff << shift);
 		data |= ((val & 0xffff) << shift);
 	} else
Signature · 1 line
2.20.1
Reviewed-by on v4 4/6
#6 lore raw

Re: [PATCH V4 4/6] PCI: rcar: Replace (8 * n) with (BITS_PER_BYTE * n)

Quoted 6 lines · On Mon, Mar 25, 2019 at 12:41 PM <[email protected]> wrote:

From: Marek Vasut <[email protected]>

Replace (8 * n) with (BITS_PER_BYTE * n) to make bit shift operations consistent. No functional change.

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
Reviewed-by on v4 4/6
#7 lore raw

Re: [PATCH V4 4/6] PCI: rcar: Replace (8 * n) with (BITS_PER_BYTE * n)

In reply to [email protected]
Quoted 5 lines · On Mon, Mar 25, 2019 at 12:40:59PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace (8 * n) with (BITS_PER_BYTE * n) to make bit shift operations consistent. No functional change.

Quoted 8 lines
Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 59 lines

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Change the patch from n << 3 to BITS_PER_BYTE * n

---
 drivers/pci/controller/pcie-rcar.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/pci/controller/pcie-rcar.c +6 -6
index 6699211fdc9a..96210768e774 100644
@@ -170,7 +170,7 @@ enum {
 
 static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 {
-	unsigned int shift = 8 * (where & 3);
+	unsigned int shift = BITS_PER_BYTE * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	val &= ~(mask << shift);
@@ -180,7 +180,7 @@ static void rcar_rmw32(struct rcar_pcie *pcie, int where, u32 mask, u32 data)
 
 static u32 rcar_read_conf(struct rcar_pcie *pcie, int where)
 {
-	unsigned int shift = 8 * (where & 3);
+	unsigned int shift = BITS_PER_BYTE * (where & 3);
 	u32 val = rcar_pci_read_reg(pcie, where & ~3);
 
 	return val >> shift;
@@ -280,9 +280,9 @@ static int rcar_pcie_read_conf(struct pci_bus *bus, unsigned int devfn,
 	}
 
 	if (size == 1)
-		*val = (*val >> (8 * (where & 3))) & 0xff;
+		*val = (*val >> (BITS_PER_BYTE * (where & 3))) & 0xff;
 	else if (size == 2)
-		*val = (*val >> (8 * (where & 2))) & 0xffff;
+		*val = (*val >> (BITS_PER_BYTE * (where & 2))) & 0xffff;
 
 	dev_dbg(&bus->dev, "pcie-config-read: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08lx\n",
 		bus->number, devfn, where, size, (unsigned long)*val);
@@ -308,11 +308,11 @@ static int rcar_pcie_write_conf(struct pci_bus *bus, unsigned int devfn,
 		bus->number, devfn, where, size, (unsigned long)val);
 
 	if (size == 1) {
-		shift = 8 * (where & 3);
+		shift = BITS_PER_BYTE * (where & 3);
 		data &= ~(0xff << shift);
 		data |= ((val & 0xff) << shift);
 	} else if (size == 2) {
-		shift = 8 * (where & 2);
+		shift = BITS_PER_BYTE * (where & 2);
 		data &= ~(0xffff << shift);
 		data |= ((val & 0xffff) << shift);
 	} else
Signature · 1 line
2.20.1
#8 lore raw

PATCH v4 5/6 PCI: rcar: Clean up debug messages

In reply to [email protected]

From: Marek Vasut <[email protected]>

Drop useless casts from debug messages, they are no longer needed due to the data type cleanup.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: - Bundle this patch with other cleanups before resending
- Add R-B from Geert
V3: Add Wolfram's R-B
V4: No change

---
 drivers/pci/controller/pcie-rcar.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/pci/controller/pcie-rcar.c +4 -4
index 96210768e774..c6013f95bdb2 100644
@@ -284,8 +284,8 @@ static int rcar_pcie_read_conf(struct pci_bus *bus, unsigned int devfn,
 	else if (size == 2)
 		*val = (*val >> (BITS_PER_BYTE * (where & 2))) & 0xffff;
 
-	dev_dbg(&bus->dev, "pcie-config-read: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08lx\n",
-		bus->number, devfn, where, size, (unsigned long)*val);
+	dev_dbg(&bus->dev, "pcie-config-read: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08x\n",
+		bus->number, devfn, where, size, *val);
 
 	return ret;
 }
@@ -304,8 +304,8 @@ static int rcar_pcie_write_conf(struct pci_bus *bus, unsigned int devfn,
 	if (ret != PCIBIOS_SUCCESSFUL)
 		return ret;
 
-	dev_dbg(&bus->dev, "pcie-config-write: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08lx\n",
-		bus->number, devfn, where, size, (unsigned long)val);
+	dev_dbg(&bus->dev, "pcie-config-write: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08x\n",
+		bus->number, devfn, where, size, val);
 
 	if (size == 1) {
 		shift = BITS_PER_BYTE * (where & 3);
Signature · 1 line
2.20.1
Reviewed-by on v4 5/6
#9 lore raw

Re: [PATCH V4 5/6] PCI: rcar: Clean up debug messages

Quoted 4 lines · On Mon, Mar 25, 2019 at 12:41:00PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>

Drop useless casts from debug messages, they are no longer needed due to the data type cleanup.

Great! The fewer casts the better.

Quoted 9 lines
Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 38 lines

---
V2: - Bundle this patch with other cleanups before resending
- Add R-B from Geert
V3: Add Wolfram's R-B
V4: No change

---
 drivers/pci/controller/pcie-rcar.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/pci/controller/pcie-rcar.c +4 -4
index 96210768e774..c6013f95bdb2 100644
@@ -284,8 +284,8 @@ static int rcar_pcie_read_conf(struct pci_bus *bus, unsigned int devfn,
 	else if (size == 2)
 		*val = (*val >> (BITS_PER_BYTE * (where & 2))) & 0xffff;
 
-	dev_dbg(&bus->dev, "pcie-config-read: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08lx\n",
-		bus->number, devfn, where, size, (unsigned long)*val);
+	dev_dbg(&bus->dev, "pcie-config-read: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08x\n",
+		bus->number, devfn, where, size, *val);
 
 	return ret;
 }
@@ -304,8 +304,8 @@ static int rcar_pcie_write_conf(struct pci_bus *bus, unsigned int devfn,
 	if (ret != PCIBIOS_SUCCESSFUL)
 		return ret;
 
-	dev_dbg(&bus->dev, "pcie-config-write: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08lx\n",
-		bus->number, devfn, where, size, (unsigned long)val);
+	dev_dbg(&bus->dev, "pcie-config-write: bus=%3d devfn=0x%04x where=0x%04x size=%d val=0x%08x\n",
+		bus->number, devfn, where, size, val);
 
 	if (size == 1) {
 		shift = BITS_PER_BYTE * (where & 3);
Signature · 1 line
2.20.1
#10 lore raw

PATCH v4 6/6 PCI: rcar: Fix 64bit MSI message address handling

In reply to [email protected]

From: Marek Vasut <[email protected]>

The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: - s/it's/its/ in commit message
- Add R-B from Geert
V3: - Reworded commit message and thus dropped Geerts R-B
V4: - Add Geert's R-B again

---
 drivers/pci/controller/pcie-rcar.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/pci/controller/pcie-rcar.c +2 -2
index c6013f95bdb2..62d2de9fbf1c 100644
@@ -890,7 +890,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct rcar_msi *msi = &pcie->msi;
-	unsigned long base;
+	phys_addr_t base;
 	int err, i;
 
 	mutex_init(&msi->lock);
@@ -932,7 +932,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 	base = virt_to_phys((void *)msi->pages);
 
 	rcar_pci_write_reg(pcie, base | MSIFE, PCIEMSIALR);
-	rcar_pci_write_reg(pcie, 0, PCIEMSIAUR);
+	rcar_pci_write_reg(pcie, base >> 32, PCIEMSIAUR);
 
 	/* enable all MSI interrupts */
 	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
Signature · 1 line
2.20.1
Reviewed-by on v4 6/6
#11 lore raw

Re: [PATCH V4 6/6] PCI: rcar: Fix 64bit MSI message address handling

Quoted 29 lines · On Mon, Mar 25, 2019 at 12:41:01PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>

The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Quoted 8 lines
Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Does this warrant a Fixes tag?

That notwithstanding,

Quoted 34 lines

---
V2: - s/it's/its/ in commit message
- Add R-B from Geert
V3: - Reworded commit message and thus dropped Geerts R-B
V4: - Add Geert's R-B again

---
 drivers/pci/controller/pcie-rcar.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/pci/controller/pcie-rcar.c +2 -2
index c6013f95bdb2..62d2de9fbf1c 100644
@@ -890,7 +890,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct rcar_msi *msi = &pcie->msi;
-	unsigned long base;
+	phys_addr_t base;
 	int err, i;
 
 	mutex_init(&msi->lock);
@@ -932,7 +932,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 	base = virt_to_phys((void *)msi->pages);
 
 	rcar_pci_write_reg(pcie, base | MSIFE, PCIEMSIALR);
-	rcar_pci_write_reg(pcie, 0, PCIEMSIAUR);
+	rcar_pci_write_reg(pcie, base >> 32, PCIEMSIAUR);
 
 	/* enable all MSI interrupts */
 	rcar_pci_write_reg(pcie, 0xffffffff, PCIEMSIIER);
Signature · 1 line
2.20.1
#12 lore raw

Re: [PATCH V4 6/6] PCI: rcar: Fix 64bit MSI message address handling

Quoted 38 lines · Simon Horman
Quoted 28 lines · On Mon, Mar 25, 2019 at 12:41:01PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>
The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Quoted 8 lines
Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 1 line

Does this warrant a Fixes tag?

(digging in old sent email)

Fixes 290c1fb358605402 ("PCI: rcar: Add MSI support for PCIe")

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#13 lore raw

Re: [PATCH V4 6/6] PCI: rcar: Fix 64bit MSI message address handling

Quoted 39 lines · Geert Uytterhoeven
Quoted 38 lines · Simon Horman
Quoted 36 lines · On Mon, Mar 25, 2019 at 12:41:01PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>
The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 4 lines
Quoted 1 line

Does this warrant a Fixes tag?

(digging in old sent email)

Fixes 290c1fb358605402 ("PCI: rcar: Add MSI support for PCIe")

But does it really fix that commit, given that on Gen2 and earlier, it was not broken as those were 32bit platforms ?

Signature · 2 lines
Best regards,
Marek Vasut
#14 lore raw

Re: [PATCH V4 6/6] PCI: rcar: Fix 64bit MSI message address handling

Hi Marek,

Quoted 40 lines · Marek Vasut
Quoted 39 lines · Geert Uytterhoeven
Quoted 38 lines · Simon Horman
Quoted 36 lines · On Mon, Mar 25, 2019 at 12:41:01PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>
The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Does this warrant a Fixes tag?

(digging in old sent email)

Fixes 290c1fb358605402 ("PCI: rcar: Add MSI support for PCIe")

But does it really fix that commit, given that on Gen2 and earlier, it was not broken as those were 32bit platforms ?

It does not fix the bug on that commit, as the bug cannot happen on arm32. It does fix that commit, in that that commit used "unsigned long" for a physical address, which is wrong, even on arm32 (esp. with LPAE). If you insist on having a Fixes tag for a commit where the bug could be seen:

Fixes e015f88c368da1e6 ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")

Apart from that, drivers should use the DMA API instead of virt_to_phys(). However, now we have a better understanding of how MSI interrupts work, we don't even need to allocate that page. All we need is the physical address of a page that is guaranteed not to be backed by RAM
(i.e. not to be a valid target for a legitimate PCI bus mastering transaction).

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#15 lore raw

Re: [PATCH V4 6/6] PCI: rcar: Fix 64bit MSI message address handling

Quoted 56 lines · Geert Uytterhoeven

Hi Marek,

Quoted 40 lines · Marek Vasut
Quoted 39 lines · Geert Uytterhoeven
Quoted 38 lines · Simon Horman
Quoted 36 lines · On Mon, Mar 25, 2019 at 12:41:01PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>
The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Does this warrant a Fixes tag?

(digging in old sent email)

Fixes 290c1fb358605402 ("PCI: rcar: Add MSI support for PCIe")

But does it really fix that commit, given that on Gen2 and earlier, it was not broken as those were 32bit platforms ?

It does not fix the bug on that commit, as the bug cannot happen on arm32. It does fix that commit, in that that commit used "unsigned long" for a physical address, which is wrong, even on arm32 (esp. with LPAE). If you insist on having a Fixes tag for a commit where the bug could be seen:

Quoted 8 lines
Fixes e015f88c368da1e6 ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")

Apart from that, drivers should use the DMA API instead of virt_to_phys(). However, now we have a better understanding of how MSI interrupts work, we don't even need to allocate that page. All we need is the physical address of a page that is guaranteed not to be backed by RAM (i.e. not to be a valid target for a legitimate PCI bus mastering transaction).

Agreed but I would merge this patch first since it is a fix and update it later.

Shall I go with the Fixes: tag above ?

Thanks,
Lorenzo

Quoted 10 lines

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#16 lore raw

Re: [PATCH V4 6/6] PCI: rcar: Fix 64bit MSI message address handling

Hi Lorenzo,

Quoted 64 lines · Lorenzo Pieralisi
Quoted 54 lines · Geert Uytterhoeven
Quoted 40 lines · Marek Vasut
Quoted 39 lines · Geert Uytterhoeven
Quoted 38 lines · Simon Horman
Quoted 36 lines · On Mon, Mar 25, 2019 at 12:41:01PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>
The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Does this warrant a Fixes tag?

(digging in old sent email)

Fixes 290c1fb358605402 ("PCI: rcar: Add MSI support for PCIe")

But does it really fix that commit, given that on Gen2 and earlier, it was not broken as those were 32bit platforms ?

It does not fix the bug on that commit, as the bug cannot happen on arm32. It does fix that commit, in that that commit used "unsigned long" for a physical address, which is wrong, even on arm32 (esp. with LPAE). If you insist on having a Fixes tag for a commit where the bug could be seen:

Quoted 8 lines
Fixes e015f88c368da1e6 ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")

Apart from that, drivers should use the DMA API instead of virt_to_phys(). However, now we have a better understanding of how MSI interrupts work, we don't even need to allocate that page. All we need is the physical address of a page that is guaranteed not to be backed by RAM (i.e. not to be a valid target for a legitimate PCI bus mastering transaction).

Quoted 2 lines

Agreed but I would merge this patch first since it is a fix and update it later.

Sure, definitely.

Quoted 1 line

Shall I go with the Fixes: tag above ?

Fine for me, thanks!

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#17 lore raw

Re: [PATCH V4 6/6] PCI: rcar: Fix 64bit MSI message address handling

Quoted 68 lines · Geert Uytterhoeven

Hi Lorenzo,

Quoted 64 lines · Lorenzo Pieralisi
Quoted 62 lines · Geert Uytterhoeven
Quoted 40 lines · Marek Vasut
Quoted 39 lines · Geert Uytterhoeven
Quoted 38 lines · Simon Horman
Quoted 36 lines · On Mon, Mar 25, 2019 at 12:41:01PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>
The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Does this warrant a Fixes tag?

(digging in old sent email)

Fixes 290c1fb358605402 ("PCI: rcar: Add MSI support for PCIe")

But does it really fix that commit, given that on Gen2 and earlier, it was not broken as those were 32bit platforms ?

It does not fix the bug on that commit, as the bug cannot happen on arm32. It does fix that commit, in that that commit used "unsigned long" for a physical address, which is wrong, even on arm32 (esp. with LPAE). If you insist on having a Fixes tag for a commit where the bug could be seen:

Fixes e015f88c368da1e6 ("PCI: rcar: Add support for R-Car H3 to pcie-rcar")

Apart from that, drivers should use the DMA API instead of virt_to_phys(). However, now we have a better understanding of how MSI interrupts work, we don't even need to allocate that page. All we need is the physical address of a page that is guaranteed not to be backed by RAM (i.e. not to be a valid target for a legitimate PCI bus mastering transaction).

Quoted 8 lines
Quoted 2 lines

Agreed but I would merge this patch first since it is a fix and update it later.

Sure, definitely.

Quoted 1 line

Shall I go with the Fixes: tag above ?

Fine for me, thanks!

I don't feel strongly either way.

Signature · 2 lines
Best regards,
Marek Vasut
#18 lore raw

Re: [PATCH V4 6/6] PCI: rcar: Fix 64bit MSI message address handling

In reply to [email protected]

Hi Marek,

Quoted 60 lines · On Mon, Mar 25, 2019 at 12:41 PM <[email protected]> wrote:

From: Marek Vasut <[email protected]>

The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: - s/it's/its/ in commit message
- Add R-B from Geert
V3: - Reworded commit message and thus dropped Geerts R-B
V4: - Add Geert's R-B again

---
 drivers/pci/controller/pcie-rcar.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/pci/controller/pcie-rcar.c +1 -1
index c6013f95bdb2..62d2de9fbf1c 100644
@@ -890,7 +890,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 {
        struct device *dev = pcie->dev;
        struct rcar_msi *msi = &pcie->msi;
-       unsigned long base;
+       phys_addr_t base;
        int err, i;
 
        mutex_init(&msi->lock);

drivers/pci/controller/pcie-rcar.c static int rcar_pcie_enable_msi(struct rcar_pcie *pcie) · line 935

Quoted 5 lines
drivers/pci/controller/pcie-rcar.c +1 -1
@@ -932,7 +932,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
        base = virt_to_phys((void *)msi->pages);
 
        rcar_pci_write_reg(pcie, base | MSIFE, PCIEMSIALR);
-       rcar_pci_write_reg(pcie, 0, PCIEMSIAUR);
+       rcar_pci_write_reg(pcie, base >> 32, PCIEMSIAUR);

As reported by 0day, this causes a warning on arm32 without LPAE:

drivers/pci/controller/pcie-rcar.c:935:32: warning: right shift
count >= width of type

Using upper_32_bits() instead of an explicit shift should fix that.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#19 lore raw

Re: [PATCH V4 6/6] PCI: rcar: Fix 64bit MSI message address handling

Quoted 71 lines · Geert Uytterhoeven

Hi Marek,

Quoted 60 lines · On Mon, Mar 25, 2019 at 12:41 PM <[email protected]> wrote:

From: Marek Vasut <[email protected]>

The MSI message address in the RC address space can be 64 bit. The R-Car PCIe RC supports such a 64bit MSI message address as well. The code currently uses virt_to_phys(__get_free_pages()) to obtain a reserved page for the MSI message address, and the return value of which can be a 64 bit physical address on 64 bit system.

However, the driver only programs PCIEMSIALR register with the bottom 32 bits of the virt_to_phys(__get_free_pages()) return value and does not program the top 32 bits into PCIEMSIAUR, but rather programs the PCIEMSIAUR register with 0x0. This worked fine on older 32 bit R-Car SoCs, however may fail on new 64 bit R-Car SoCs.

Since from a PCIe controller perspective, an inbound MSI is a memory write to a special address (in case of this controller, defined by the value in PCIEMSIAUR:PCIEMSIALR), which triggers an interrupt, but never hits the DRAM _and_ because allocation of an MSI by a PCIe card driver obtains the MSI message address by reading PCIEMSIAUR:PCIEMSIALR in rcar_msi_setup_irqs(), incorrectly programmed PCIEMSIAUR cannot cause memory corruption or other issues.

There is however the possibility that if virt_to_phys(__get_free_pages()) returned address above the 32bit boundary _and_ PCIEMSIAUR was programmed to 0x0 _and_ if the system had physical RAM at the address matching the value of PCIEMSIALR, a PCIe card driver could allocate a buffer with a physical address matching the value of PCIEMSIALR and a remote write to such a buffer by a PCIe card would trigger a spurious MSI.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

---
V2: - s/it's/its/ in commit message
- Add R-B from Geert
V3: - Reworded commit message and thus dropped Geerts R-B
V4: - Add Geert's R-B again

---
 drivers/pci/controller/pcie-rcar.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/pci/controller/pcie-rcar.c +1 -1
index c6013f95bdb2..62d2de9fbf1c 100644
@@ -890,7 +890,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
 {
        struct device *dev = pcie->dev;
        struct rcar_msi *msi = &pcie->msi;
-       unsigned long base;
+       phys_addr_t base;
        int err, i;
 
        mutex_init(&msi->lock);

drivers/pci/controller/pcie-rcar.c static int rcar_pcie_enable_msi(struct rcar_pcie *pcie) · line 935

Quoted 5 lines
drivers/pci/controller/pcie-rcar.c +1 -1
@@ -932,7 +932,7 @@ static int rcar_pcie_enable_msi(struct rcar_pcie *pcie)
        base = virt_to_phys((void *)msi->pages);
 
        rcar_pci_write_reg(pcie, base | MSIFE, PCIEMSIALR);
-       rcar_pci_write_reg(pcie, 0, PCIEMSIAUR);
+       rcar_pci_write_reg(pcie, base >> 32, PCIEMSIAUR);

As reported by 0day, this causes a warning on arm32 without LPAE:

Quoted 4 lines

drivers/pci/controller/pcie-rcar.c:935:32: warning: right shift
count >= width of type

Using upper_32_bits() instead of an explicit shift should fix that.

I saw the report too.

Lorenzo, do you want a separate patch to squash with this or V5 ?

Signature · 2 lines
Best regards,
Marek Vasut
#20 lore raw
In reply to [email protected]
Quoted 13 lines · On Mon, Mar 25, 2019 at 12:40:56PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace macros using constants with BIT()s instead, no functional change.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 7 lines

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Add Geert's R-B

---
 drivers/pci/controller/pcie-rcar.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Hi Marek,

I assume this series makes:

https://patchwork.ozlabs.org/patch/1053779/
https://patchwork.ozlabs.org/patch/1057454/

obsolete, please let me know and I will drop
them from the PCI patches queue, I will get to
this series later for v5.2.

Thanks,
Lorenzo

Quoted 26 lines
drivers/pci/controller/pcie-rcar.c +3 -3
index c8febb009454..5b8736f0cd6b 100644
@@ -46,14 +46,14 @@
 
 /* Transfer control */
 #define PCIETCTLR		0x02000
-#define  CFINIT			1
+#define  CFINIT			BIT(0)
 #define PCIETSTR		0x02004
-#define  DATA_LINK_ACTIVE	1
+#define  DATA_LINK_ACTIVE	BIT(0)
 #define PCIEERRFR		0x02020
 #define  UNSUPPORTED_REQUEST	BIT(4)
 #define PCIEMSIFR		0x02044
 #define PCIEMSIALR		0x02048
-#define  MSIFE			1
+#define  MSIFE			BIT(0)
 #define PCIEMSIAUR		0x0204c
 #define PCIEMSIIER		0x02050
 
Signature · 1 line
2.20.1
#21 lore raw
Quoted 22 lines · Lorenzo Pieralisi
Quoted 13 lines · On Mon, Mar 25, 2019 at 12:40:56PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace macros using constants with BIT()s instead, no functional change.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 7 lines

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Add Geert's R-B

---
 drivers/pci/controller/pcie-rcar.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Quoted 1 line

Hi Marek,

Hi,

Quoted 8 lines

I assume this series makes:

https://patchwork.ozlabs.org/patch/1053779/
https://patchwork.ozlabs.org/patch/1057454/

obsolete, please let me know and I will drop
them from the PCI patches queue, I will get to
this series later for v5.2.

Yes, I decided to merge the two series together to make it easier to pick.

Thanks

Signature · 2 lines
Best regards,
Marek Vasut
Reviewed-by on v4 1/6
#22 lore raw
In reply to [email protected]
Quoted 10 lines · On Mon, Mar 25, 2019 at 12:40:56PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace macros using constants with BIT()s instead, no functional change.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>
Quoted 3 lines

To: [email protected]

Quoted 33 lines

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Add Geert's R-B

---
 drivers/pci/controller/pcie-rcar.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/pci/controller/pcie-rcar.c +3 -3
index c8febb009454..5b8736f0cd6b 100644
@@ -46,14 +46,14 @@
 
 /* Transfer control */
 #define PCIETCTLR		0x02000
-#define  CFINIT			1
+#define  CFINIT			BIT(0)
 #define PCIETSTR		0x02004
-#define  DATA_LINK_ACTIVE	1
+#define  DATA_LINK_ACTIVE	BIT(0)
 #define PCIEERRFR		0x02020
 #define  UNSUPPORTED_REQUEST	BIT(4)
 #define PCIEMSIFR		0x02044
 #define PCIEMSIALR		0x02048
-#define  MSIFE			1
+#define  MSIFE			BIT(0)
 #define PCIEMSIAUR		0x0204c
 #define PCIEMSIIER		0x02050
 
Signature · 1 line
2.20.1
#23 lore raw
In reply to [email protected]
Quoted 13 lines · On Mon, Mar 25, 2019 at 12:40:56PM +0100, [email protected] wrote:

From: Marek Vasut <[email protected]>

Replace macros using constants with BIT()s instead, no functional change.

Cc Geert Uytterhoeven <[email protected]>
Cc Phil Edworthy <[email protected]>
Cc Simon Horman <[email protected]>
Cc Wolfram Sang <[email protected]>

To: [email protected]

Quoted 7 lines

---
V2: Bundle this patch with other cleanups before resending
V3: Add Wolfram's R-B
V4: Add Geert's R-B

---
 drivers/pci/controller/pcie-rcar.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Applied the series to pci/rcar for v5.2, thanks.

Lorenzo

Quoted 25 lines
drivers/pci/controller/pcie-rcar.c +3 -3
index c8febb009454..5b8736f0cd6b 100644
@@ -46,14 +46,14 @@
 
 /* Transfer control */
 #define PCIETCTLR		0x02000
-#define  CFINIT			1
+#define  CFINIT			BIT(0)
 #define PCIETSTR		0x02004
-#define  DATA_LINK_ACTIVE	1
+#define  DATA_LINK_ACTIVE	BIT(0)
 #define PCIEERRFR		0x02020
 #define  UNSUPPORTED_REQUEST	BIT(4)
 #define PCIEMSIFR		0x02044
 #define PCIEMSIALR		0x02048
-#define  MSIFE			1
+#define  MSIFE			BIT(0)
 #define PCIEMSIAUR		0x0204c
 #define PCIEMSIIER		0x02050
 
Signature · 1 line
2.20.1