Skip to contentSkip to conversation

PCI: dwc: ep: Minor alignment cleanups

PATCH In review Niklas Cassel 6 participants 16 messages Oct 17, 2024 – Nov 2, 2024

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover PCI: dwc: ep: Minor alignment cleanups

Hello all,

Here comes minor cleanups related to the new address alignment APIs.

Kind regards,
Niklas

Niklas Cassel (2):
PCI: dwc: ep: Fix dw_pcie_ep_align_addr()
PCI: dwc: ep: Use align addr function for

    dw_pcie_ep_raise_{msi,msix}_irq()
 .../pci/controller/dwc/pcie-designware-ep.c   | 20 +++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
Signature · 1 line
2.47.0
#1 lore raw

PATCH 1/2 PCI: dwc: ep: Fix dw_pcie_ep_align_addr()

ep->page_size is defined by the EPC drivers.
Some drivers e.g. pci-imx6.c defines this to 4K for imx95.

dw_pcie_ep_init() will call pci_epc_mem_init() with ep->page_size. pci_epc_mem_init() will call pci_epc_multi_mem_init().

pci_epc_multi_mem_init() will initialize mem->window.page_size. If the provided page_size (ep->page_size) is smaller than PAGE_SIZE, it will initialize mem->window.page_size to PAGE_SIZE rather than ep->page_size.

Thus, mem->window.page_size can be larger than ep->page_size, e.g. for a platform built with PAGE_SIZE == 64K, while using a EPC driver that defines ep->page_size to 4k.

Therefore, modify dw_pcie_ep_align_addr() to use epc->mem->window.page_size rather than ep->page_size.

---
 drivers/pci/controller/dwc/pcie-designware-ep.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +1 -1
index 2d0e7bf17919..20f67fd85e83 100644
@@ -276,7 +276,7 @@ static u64 dw_pcie_ep_align_addr(struct pci_epc *epc, u64 pci_addr,
 	u64 mask = pci->region_align - 1;
 	size_t ofst = pci_addr & mask;
 
-	*pci_size = ALIGN(ofst + *pci_size, ep->page_size);
+	*pci_size = ALIGN(ofst + *pci_size, epc->mem->window.page_size);
 	*offset = ofst;
 
 	return pci_addr & ~mask;
Signature · 1 line
2.47.0
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] PCI: dwc: ep: Fix dw_pcie_ep_align_addr()

Quoted 11 lines · Niklas Cassel

ep->page_size is defined by the EPC drivers.
Some drivers e.g. pci-imx6.c defines this to 4K for imx95.

dw_pcie_ep_init() will call pci_epc_mem_init() with ep->page_size. pci_epc_mem_init() will call pci_epc_multi_mem_init().

pci_epc_multi_mem_init() will initialize mem->window.page_size. If the provided page_size (ep->page_size) is smaller than PAGE_SIZE, it will initialize mem->window.page_size to PAGE_SIZE rather than ep->page_size.

Quoted 8 lines

Thus, mem->window.page_size can be larger than ep->page_size, e.g. for a platform built with PAGE_SIZE == 64K, while using a EPC driver that defines ep->page_size to 4k.

Therefore, modify dw_pcie_ep_align_addr() to use epc->mem->window.page_size rather than ep->page_size.

Quoted 20 lines
---
 drivers/pci/controller/dwc/pcie-designware-ep.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +1 -2
index 2d0e7bf17919..20f67fd85e83 100644
@@ -276,7 +276,7 @@ static u64 dw_pcie_ep_align_addr(struct pci_epc *epc, u64 pci_addr,
 	u64 mask = pci->region_align - 1;
 	size_t ofst = pci_addr & mask;
 
-	*pci_size = ALIGN(ofst + *pci_size, ep->page_size);
+	*pci_size = ALIGN(ofst + *pci_size, epc->mem->window.page_size);
 	*offset = ofst;
 
 	return pci_addr & ~mask;
--

2.47.0

Reviewed-by on v1 1/2
#3 lore raw

Re: [PATCH 1/2] PCI: dwc: ep: Fix dw_pcie_ep_align_addr()

In reply to Niklas Cassel
Quoted 11 lines · Niklas Cassel

ep->page_size is defined by the EPC drivers.
Some drivers e.g. pci-imx6.c defines this to 4K for imx95.

dw_pcie_ep_init() will call pci_epc_mem_init() with ep->page_size. pci_epc_mem_init() will call pci_epc_multi_mem_init().

pci_epc_multi_mem_init() will initialize mem->window.page_size. If the provided page_size (ep->page_size) is smaller than PAGE_SIZE, it will initialize mem->window.page_size to PAGE_SIZE rather than ep->page_size.

Quoted 8 lines

Thus, mem->window.page_size can be larger than ep->page_size, e.g. for a platform built with PAGE_SIZE == 64K, while using a EPC driver that defines ep->page_size to 4k.

Therefore, modify dw_pcie_ep_align_addr() to use epc->mem->window.page_size rather than ep->page_size.

Fixes a1cd1d901a5c ("PCI: dwc: endpoint: Implement the pci_epc_ops::align_addr() operation")

- Mani

Quoted 20 lines
---
 drivers/pci/controller/dwc/pcie-designware-ep.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +1 -1
index 2d0e7bf17919..20f67fd85e83 100644
@@ -276,7 +276,7 @@ static u64 dw_pcie_ep_align_addr(struct pci_epc *epc, u64 pci_addr,
 	u64 mask = pci->region_align - 1;
 	size_t ofst = pci_addr & mask;
 
-	*pci_size = ALIGN(ofst + *pci_size, ep->page_size);
+	*pci_size = ALIGN(ofst + *pci_size, epc->mem->window.page_size);
 	*offset = ofst;
 
 	return pci_addr & ~mask;
Signature · 1 line
2.47.0
Signature · 1 line
மணிவண்ணன் சதாசிவம்
#4 lore raw

Re: [PATCH 1/2] PCI: dwc: ep: Fix dw_pcie_ep_align_addr()

Hello,

Quoted 15 lines
Quoted 15 lines

ep->page_size is defined by the EPC drivers.
Some drivers e.g. pci-imx6.c defines this to 4K for imx95.

dw_pcie_ep_init() will call pci_epc_mem_init() with ep->page_size. pci_epc_mem_init() will call pci_epc_multi_mem_init().

pci_epc_multi_mem_init() will initialize mem->window.page_size. If the provided page_size (ep->page_size) is smaller than PAGE_SIZE, it will initialize mem->window.page_size to PAGE_SIZE rather than ep->page_size.

Thus, mem->window.page_size can be larger than ep->page_size, e.g. for a platform built with PAGE_SIZE == 64K, while using a EPC driver that defines ep->page_size to 4k.

Quoted 6 lines
Quoted 4 lines

Therefore, modify dw_pcie_ep_align_addr() to use epc->mem->window.page_size rather than ep->page_size.

[...]

Fixes a1cd1d901a5c ("PCI: dwc: endpoint: Implement the pci_epc_ops::align_addr() operation")

I squashed this patch into the one this aimed to fix.

Krzysztof

#5 lore raw

PATCH 2/2 PCI: dwc: ep: Use align addr function for dw_pcie_ep_raise_{msi,msix}_irq()

In reply to Niklas Cassel

Use the dw_pcie_ep_align_addr() function to calculate the alignment in dw_pcie_ep_raise_{msi,msix}_irq() instead of open coding the same.

---
 .../pci/controller/dwc/pcie-designware-ep.c    | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +9 -9
index 20f67fd85e83..9bafa62bed1d 100644
@@ -503,7 +503,8 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	u32 msg_addr_lower, msg_addr_upper, reg;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
-	unsigned int aligned_offset;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u16 msg_ctrl, msg_data;
 	bool has_upper;
 	u64 msg_addr;
@@ -531,14 +532,13 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	}
 	msg_addr = ((u64)msg_addr_upper) << 32 | msg_addr_lower;
 
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
-				  epc->mem->window.page_size);
+				  msi_mem_size);
 	if (ret)
 		return ret;
 
-	writel(msg_data | (interrupt_num - 1), ep->msi_mem + aligned_offset);
+	writel(msg_data | (interrupt_num - 1), ep->msi_mem + offset);
 
 	dw_pcie_ep_unmap_addr(epc, func_no, 0, ep->msi_mem_phys);
 
@@ -589,8 +589,9 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
 	struct pci_epf_msix_tbl *msix_tbl;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u32 reg, msg_data, vec_ctrl;
-	unsigned int aligned_offset;
 	u32 tbl_offset;
 	u64 msg_addr;
 	int ret;
@@ -615,14 +616,13 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
 		return -EPERM;
 	}
 
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
 				  epc->mem->window.page_size);
 	if (ret)
 		return ret;
 
-	writel(msg_data, ep->msi_mem + aligned_offset);
+	writel(msg_data, ep->msi_mem + offset);
 
 	dw_pcie_ep_unmap_addr(epc, func_no, 0, ep->msi_mem_phys);
 
Signature · 1 line
2.47.0
#6 lore raw

Re: [PATCH 2/2] PCI: dwc: ep: Use align addr function for dw_pcie_ep_raise_{msi,msix}_irq()

Quoted 41 lines · Niklas Cassel

Use the dw_pcie_ep_align_addr() function to calculate the alignment in dw_pcie_ep_raise_{msi,msix}_irq() instead of open coding the same.

---
 .../pci/controller/dwc/pcie-designware-ep.c    | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +5 -5
index 20f67fd85e83..9bafa62bed1d 100644
@@ -503,7 +503,8 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	u32 msg_addr_lower, msg_addr_upper, reg;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
-	unsigned int aligned_offset;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u16 msg_ctrl, msg_data;
 	bool has_upper;
 	u64 msg_addr;
@@ -531,14 +532,13 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	}
 	msg_addr = ((u64)msg_addr_upper) << 32 | msg_addr_lower;
 
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
-				  epc->mem->window.page_size);
+				  msi_mem_size);
 	if (ret)
 		return ret;
 
-	writel(msg_data | (interrupt_num - 1), ep->msi_mem + aligned_offset);
+	writel(msg_data | (interrupt_num - 1), ep->msi_mem + offset);
 
 	dw_pcie_ep_unmap_addr(epc, func_no, 0, ep->msi_mem_phys);
 

drivers/pci/controller/dwc/pcie-designware-ep.c int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no, · line 593

Quoted 7 lines
drivers/pci/controller/dwc/pcie-designware-ep.c +2 -1
@@ -589,8 +589,9 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
 	struct pci_epf_msix_tbl *msix_tbl;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u32 reg, msg_data, vec_ctrl;
-	unsigned int aligned_offset;

why not direct use 'aligned_offset' ? just change to size_t.

Quoted 23 lines
drivers/pci/controller/dwc/pcie-designware-ep.c +2 -4
 	u32 tbl_offset;
 	u64 msg_addr;
 	int ret;
@@ -615,14 +616,13 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
 		return -EPERM;
 	}
 
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
 				  epc->mem->window.page_size);
 	if (ret)
 		return ret;
 
-	writel(msg_data, ep->msi_mem + aligned_offset);
+	writel(msg_data, ep->msi_mem + offset);
 
 	dw_pcie_ep_unmap_addr(epc, func_no, 0, ep->msi_mem_phys);
 
--

2.47.0

#7 lore raw

Re: [PATCH 2/2] PCI: dwc: ep: Use align addr function for dw_pcie_ep_raise_{msi,msix}_irq()

Hello Frank,

Quoted 41 lines · Frank Li
Quoted 40 lines · Niklas Cassel

Use the dw_pcie_ep_align_addr() function to calculate the alignment in dw_pcie_ep_raise_{msi,msix}_irq() instead of open coding the same.

---
 .../pci/controller/dwc/pcie-designware-ep.c    | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +5 -5
index 20f67fd85e83..9bafa62bed1d 100644
@@ -503,7 +503,8 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	u32 msg_addr_lower, msg_addr_upper, reg;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
-	unsigned int aligned_offset;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u16 msg_ctrl, msg_data;
 	bool has_upper;
 	u64 msg_addr;
@@ -531,14 +532,13 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	}
 	msg_addr = ((u64)msg_addr_upper) << 32 | msg_addr_lower;
 
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
-				  epc->mem->window.page_size);
+				  msi_mem_size);
 	if (ret)
 		return ret;
 
-	writel(msg_data | (interrupt_num - 1), ep->msi_mem + aligned_offset);
+	writel(msg_data | (interrupt_num - 1), ep->msi_mem + offset);
 
 	dw_pcie_ep_unmap_addr(epc, func_no, 0, ep->msi_mem_phys);
 
Quoted 10 lines

drivers/pci/controller/dwc/pcie-designware-ep.c int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no, · line 593

Quoted 8 lines
drivers/pci/controller/dwc/pcie-designware-ep.c +2 -1
@@ -589,8 +589,9 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
 	struct pci_epf_msix_tbl *msix_tbl;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u32 reg, msg_data, vec_ctrl;
-	unsigned int aligned_offset;

why not direct use 'aligned_offset' ? just change to size_t.

Because I think that that name was really bad.
aligned_offset sounds like the offset is aligned, but that is not the case.

Now when we have a dw_pcie_ep_align_addr() function, I think that simply calling the variable offset is less ambiguous. Anyone who isn't sure what the offset represents can simply read the documentation for the .align_addr endpoint controller operation.

Kind regards,
Niklas

Reviewed-by
#8 lore raw

Re: [PATCH 2/2] PCI: dwc: ep: Use align addr function for dw_pcie_ep_raise_{msi,msix}_irq()

Quoted 55 lines · Niklas Cassel

Hello Frank,

Quoted 41 lines · Frank Li
Quoted 40 lines · Niklas Cassel

Use the dw_pcie_ep_align_addr() function to calculate the alignment in dw_pcie_ep_raise_{msi,msix}_irq() instead of open coding the same.

---
 .../pci/controller/dwc/pcie-designware-ep.c    | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +5 -5
index 20f67fd85e83..9bafa62bed1d 100644
@@ -503,7 +503,8 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	u32 msg_addr_lower, msg_addr_upper, reg;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
-	unsigned int aligned_offset;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u16 msg_ctrl, msg_data;
 	bool has_upper;
 	u64 msg_addr;
@@ -531,14 +532,13 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	}
 	msg_addr = ((u64)msg_addr_upper) << 32 | msg_addr_lower;
 
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
-				  epc->mem->window.page_size);
+				  msi_mem_size);
 	if (ret)
 		return ret;
 
-	writel(msg_data | (interrupt_num - 1), ep->msi_mem + aligned_offset);
+	writel(msg_data | (interrupt_num - 1), ep->msi_mem + offset);
 
 	dw_pcie_ep_unmap_addr(epc, func_no, 0, ep->msi_mem_phys);
 
Quoted 10 lines

drivers/pci/controller/dwc/pcie-designware-ep.c int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no, · line 593

Quoted 8 lines
drivers/pci/controller/dwc/pcie-designware-ep.c +2 -1
@@ -589,8 +589,9 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
 	struct pci_epf_msix_tbl *msix_tbl;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u32 reg, msg_data, vec_ctrl;
-	unsigned int aligned_offset;

why not direct use 'aligned_offset' ? just change to size_t.

Quoted 7 lines

Because I think that that name was really bad.
aligned_offset sounds like the offset is aligned, but that is not the case.

Now when we have a dw_pcie_ep_align_addr() function, I think that simply calling the variable offset is less ambiguous. Anyone who isn't sure what the offset represents can simply read the documentation for the .align_addr endpoint controller operation.

Make sense.

Quoted 4 lines

Kind regards,
Niklas

#9 lore raw

Re: [PATCH 2/2] PCI: dwc: ep: Use align addr function for dw_pcie_ep_raise_{msi,msix}_irq()

In reply to Niklas Cassel
Quoted 23 lines · Niklas Cassel

Use the dw_pcie_ep_align_addr() function to calculate the alignment in dw_pcie_ep_raise_{msi,msix}_irq() instead of open coding the same.

---
 .../pci/controller/dwc/pcie-designware-ep.c    | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +2 -1
index 20f67fd85e83..9bafa62bed1d 100644
@@ -503,7 +503,8 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	u32 msg_addr_lower, msg_addr_upper, reg;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
-	unsigned int aligned_offset;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u16 msg_ctrl, msg_data;
 	bool has_upper;
 	u64 msg_addr;

drivers/pci/controller/dwc/pcie-designware-ep.c int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no, · line 537

Quoted 9 lines
drivers/pci/controller/dwc/pcie-designware-ep.c +2 -3
@@ -531,14 +532,13 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	}
 	msg_addr = ((u64)msg_addr_upper) << 32 | msg_addr_lower;
 
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
-				  epc->mem->window.page_size);
+				  msi_mem_size);

I haven't worked through this; just double checking that this is correct. Previously we did ALIGN_DOWN() here, but dw_pcie_ep_align_addr() uses ALIGN() (not ALIGN_DOWN()). Similar below in dw_pcie_ep_raise_msix_irq().

Quoted 39 lines
drivers/pci/controller/dwc/pcie-designware-ep.c +5 -5
 	if (ret)
 		return ret;
 
-	writel(msg_data | (interrupt_num - 1), ep->msi_mem + aligned_offset);
+	writel(msg_data | (interrupt_num - 1), ep->msi_mem + offset);
 
 	dw_pcie_ep_unmap_addr(epc, func_no, 0, ep->msi_mem_phys);
 
@@ -589,8 +589,9 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
 	struct pci_epf_msix_tbl *msix_tbl;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u32 reg, msg_data, vec_ctrl;
-	unsigned int aligned_offset;
 	u32 tbl_offset;
 	u64 msg_addr;
 	int ret;
@@ -615,14 +616,13 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
 		return -EPERM;
 	}
 
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
 				  epc->mem->window.page_size);
 	if (ret)
 		return ret;
 
-	writel(msg_data, ep->msi_mem + aligned_offset);
+	writel(msg_data, ep->msi_mem + offset);
 
 	dw_pcie_ep_unmap_addr(epc, func_no, 0, ep->msi_mem_phys);
 
Signature · 1 line
2.47.0
#10 lore raw

Re: [PATCH 2/2] PCI: dwc: ep: Use align addr function for dw_pcie_ep_raise_{msi,msix}_irq()

Quoted 27 lines · Bjorn Helgaas
Quoted 26 lines · Niklas Cassel

Use the dw_pcie_ep_align_addr() function to calculate the alignment in dw_pcie_ep_raise_{msi,msix}_irq() instead of open coding the same.

---
 .../pci/controller/dwc/pcie-designware-ep.c    | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +2 -1
index 20f67fd85e83..9bafa62bed1d 100644
@@ -503,7 +503,8 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	u32 msg_addr_lower, msg_addr_upper, reg;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
-	unsigned int aligned_offset;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u16 msg_ctrl, msg_data;
 	bool has_upper;
 	u64 msg_addr;
@@ -531,14 +532,13 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	}
 	msg_addr = ((u64)msg_addr_upper) << 32 | msg_addr_lower;
 
Quoted 11 lines

drivers/pci/controller/dwc/pcie-designware-ep.c int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no, · line 537

Quoted 6 lines
drivers/pci/controller/dwc/pcie-designware-ep.c +2 -3
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
-				  epc->mem->window.page_size);
+				  msi_mem_size);

I haven't worked through this; just double checking that this is correct. Previously we did ALIGN_DOWN() here, but dw_pcie_ep_align_addr() uses ALIGN() (not ALIGN_DOWN()). Similar below in dw_pcie_ep_raise_msix_irq().

The ALIGN() in dw_pcie_ep_align_addr() is for the mapping size. The address is aligned down manually:

	return pci_addr & ~mask;

So it is the same. We could change dw_pcie_ep_align_addr() to do:

	return ALIGN_DOWN(pci_addr, epc->mem->window.page_size);

But given that the offset calculation needs the alignment mask anyway, using the mask variable directly seems natural.

So this is functionnally identical for the PCI address being mapped, and it is even better for the mapping size since this was passing epc->mem->window.page_size before but if the PCI address range crosses over a page_size boundary, we actually need 2 x page_size as the mapping size...

Which makes me realized that there is something still wrong: the memory being mapped (ep->msi_mem) is at most one page:

	ep->msi_mem = pci_epc_mem_alloc_addr(epc, &ep->msi_mem_phys,
					     epc->mem->window.page_size);
	if (!ep->msi_mem) {
		ret = -ENOMEM;
		dev_err(dev, "Failed to reserve memory for MSI/MSI-X\n");
		goto err_exit_epc_mem;
	}

But we may need up to 2 pages depending on the PCI address we get for the MSI/MSIX. So we need to fix that as well I think.

Niklas, thoughts ?

Signature · 2 lines
Damien Le Moal
Western Digital Research
#11 lore raw

Re: [PATCH 2/2] PCI: dwc: ep: Use align addr function for dw_pcie_ep_raise_{msi,msix}_irq()

Quoted 43 lines · Damien Le Moal
Quoted 27 lines · Bjorn Helgaas
Quoted 26 lines · Niklas Cassel

Use the dw_pcie_ep_align_addr() function to calculate the alignment in dw_pcie_ep_raise_{msi,msix}_irq() instead of open coding the same.

---
 .../pci/controller/dwc/pcie-designware-ep.c    | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +2 -1
index 20f67fd85e83..9bafa62bed1d 100644
@@ -503,7 +503,8 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	u32 msg_addr_lower, msg_addr_upper, reg;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
-	unsigned int aligned_offset;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u16 msg_ctrl, msg_data;
 	bool has_upper;
 	u64 msg_addr;
@@ -531,14 +532,13 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	}
 	msg_addr = ((u64)msg_addr_upper) << 32 | msg_addr_lower;
 
Quoted 11 lines

drivers/pci/controller/dwc/pcie-designware-ep.c int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no, · line 537

Quoted 6 lines
drivers/pci/controller/dwc/pcie-designware-ep.c +2 -3
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
-				  epc->mem->window.page_size);
+				  msi_mem_size);

I haven't worked through this; just double checking that this is correct. Previously we did ALIGN_DOWN() here, but dw_pcie_ep_align_addr() uses ALIGN() (not ALIGN_DOWN()). Similar below in dw_pcie_ep_raise_msix_irq().

The ALIGN() in dw_pcie_ep_align_addr() is for the mapping size. The address is aligned down manually:

Quoted 8 lines
	return pci_addr & ~mask;

So it is the same. We could change dw_pcie_ep_align_addr() to do:

	return ALIGN_DOWN(pci_addr, epc->mem->window.page_size);

But given that the offset calculation needs the alignment mask anyway, using the mask variable directly seems natural.

I agree.

Quoted 17 lines

So this is functionnally identical for the PCI address being mapped, and it is even better for the mapping size since this was passing epc->mem->window.page_size before but if the PCI address range crosses over a page_size boundary, we actually need 2 x page_size as the mapping size...

Which makes me realized that there is something still wrong: the memory being mapped (ep->msi_mem) is at most one page:

	ep->msi_mem = pci_epc_mem_alloc_addr(epc, &ep->msi_mem_phys,
					     epc->mem->window.page_size);
	if (!ep->msi_mem) {
		ret = -ENOMEM;
		dev_err(dev, "Failed to reserve memory for MSI/MSI-X\n");
		goto err_exit_epc_mem;
	}
Quoted 4 lines

But we may need up to 2 pages depending on the PCI address we get for the MSI/MSIX. So we need to fix that as well I think.

Niklas, thoughts ?

Hmm.. the MSI data that we want to map is just two bytes.
It is just that the minimum mapping we can do is epc->mem->window.page_size.

On e.g. rk3588 the page_size is 64k.

I guess that we might be unlucky and get the MSI data at an offset that is at the end of the page.

But considering that we just need 2 bytes (for MSI, 4 bytes for MSI-X), and that the PCI word size is 4 bytes, I don't see a problem with the current code.

While the allocation of the msi_mem should still be page_size:
ep->msi_mem = pci_epc_mem_alloc_addr(epc, &ep->msi_mem_phys,

                                     epc->mem->window.page_size);

I guess that we could change dw_pcie_ep_raise_msi_irq() from:
size_t msi_mem_size = epc->mem->window.page_size; to:
size_t msi_mem_size = 2;

and change dw_pcie_ep_raise_msix_irq() from:
size_t msi_mem_size = epc->mem->window.page_size;
to:
size_t msi_mem_size = 4;

To make the code strictly more correct for a reader, but I don't think that there is a problem with the current code.

Kind regards,
Niklas

Reviewed-by on v1 2/2
#12 lore raw

Re: [PATCH 2/2] PCI: dwc: ep: Use align addr function for dw_pcie_ep_raise_{msi,msix}_irq()

In reply to Niklas Cassel
Quoted 4 lines · Niklas Cassel

Use the dw_pcie_ep_align_addr() function to calculate the alignment in dw_pcie_ep_raise_{msi,msix}_irq() instead of open coding the same.

- Mani

Quoted 67 lines
---
 .../pci/controller/dwc/pcie-designware-ep.c    | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
drivers/pci/controller/dwc/pcie-designware-ep.c +9 -9
index 20f67fd85e83..9bafa62bed1d 100644
@@ -503,7 +503,8 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	u32 msg_addr_lower, msg_addr_upper, reg;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
-	unsigned int aligned_offset;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u16 msg_ctrl, msg_data;
 	bool has_upper;
 	u64 msg_addr;
@@ -531,14 +532,13 @@ int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
 	}
 	msg_addr = ((u64)msg_addr_upper) << 32 | msg_addr_lower;
 
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
-				  epc->mem->window.page_size);
+				  msi_mem_size);
 	if (ret)
 		return ret;
 
-	writel(msg_data | (interrupt_num - 1), ep->msi_mem + aligned_offset);
+	writel(msg_data | (interrupt_num - 1), ep->msi_mem + offset);
 
 	dw_pcie_ep_unmap_addr(epc, func_no, 0, ep->msi_mem_phys);
 
@@ -589,8 +589,9 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
 	struct pci_epf_msix_tbl *msix_tbl;
 	struct dw_pcie_ep_func *ep_func;
 	struct pci_epc *epc = ep->epc;
+	size_t msi_mem_size = epc->mem->window.page_size;
+	size_t offset;
 	u32 reg, msg_data, vec_ctrl;
-	unsigned int aligned_offset;
 	u32 tbl_offset;
 	u64 msg_addr;
 	int ret;
@@ -615,14 +616,13 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no,
 		return -EPERM;
 	}
 
-	aligned_offset = msg_addr & (epc->mem->window.page_size - 1);
-	msg_addr = ALIGN_DOWN(msg_addr, epc->mem->window.page_size);
+	msg_addr = dw_pcie_ep_align_addr(epc, msg_addr, &msi_mem_size, &offset);
 	ret = dw_pcie_ep_map_addr(epc, func_no, 0, ep->msi_mem_phys, msg_addr,
 				  epc->mem->window.page_size);
 	if (ret)
 		return ret;
 
-	writel(msg_data, ep->msi_mem + aligned_offset);
+	writel(msg_data, ep->msi_mem + offset);
 
 	dw_pcie_ep_unmap_addr(epc, func_no, 0, ep->msi_mem_phys);
 
Signature · 1 line
2.47.0
Signature · 1 line
மணிவண்ணன் சதாசிவம்
Reviewed-by Tested-by on the v1 series
#14 lore raw
In reply to Niklas Cassel
Quoted 7 lines · Niklas Cassel

Hello all,

Here comes minor cleanups related to the new address alignment APIs.

Kind regards,
Niklas

Quoted 7 lines

Niklas Cassel (2):
PCI: dwc: ep: Fix dw_pcie_ep_align_addr()
PCI: dwc: ep: Use align addr function for

    dw_pcie_ep_raise_{msi,msix}_irq()
 .../pci/controller/dwc/pcie-designware-ep.c   | 20 +++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

Looks good.

Signature · 2 lines
Damien Le Moal
Western Digital Research
#15 lore raw
In reply to Niklas Cassel
Quoted 7 lines · Niklas Cassel

Hello all,

Here comes minor cleanups related to the new address alignment APIs.

Kind regards,
Niklas

Quoted 11 lines

Niklas Cassel (2):
PCI: dwc: ep: Fix dw_pcie_ep_align_addr()
PCI: dwc: ep: Use align addr function for

    dw_pcie_ep_raise_{msi,msix}_irq()
 .../pci/controller/dwc/pcie-designware-ep.c   | 20 +++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
Signature · 1 line
2.47.0

This series has been reviewed by two different people.

What is the hold up? :)

Kind regards,
Niklas