PCI: dwc: Move allocate and map page for msi out of dw_pcie_msi_init()
Subsystems (4)
PATCH 1/1 PCI: dwc: Move allocate and map page for msi out of dw_pcie_msi_init()
Currently, dw_pcie_msi_init() allocates and maps page for msi, then program the PCIE_MSI_ADDR_LO and PCIE_MSI_ADDR_HI. The Root Complex may lose power during suspend-to-RAM, so when we resume, we want to redo the latter but not the former. If designware based driver (for example, pcie-tegra194.c) calls dw_pcie_msi_init() in resume path, the previous msi page will be leaked.
Move the allocate and map msi page from dw_pcie_msi_init() to dw_pcie_host_init() to fix this problem.
--- drivers/pci/controller/dwc/pci-dra7xx.c | 18 ++++++++++++- .../pci/controller/dwc/pcie-designware-host.c | 27 +++++++++---------- 2 files changed, 30 insertions(+), 15 deletions(-)
drivers/pci/controller/dwc/pci-dra7xx.c +17 -1
@@ -490,7 +490,9 @@ static struct irq_chip dra7xx_pci_msi_bottom_irq_chip = {static int dra7xx_pcie_msi_host_init (struct pcie_port *pp) {struct dw_pcie *pci =to_dw_pcie_from_pp (pp); +struct device *dev = pci->dev;u32 ctrl, num_ctrls; +int ret; pp->msi_irq_chip = &dra7xx_pci_msi_bottom_irq_chip;
@@ -506,7 +508,21 @@ static int dra7xx_pcie_msi_host_init(struct pcie_port *pp) ~0); } -return dw_pcie_allocate_domains (pp); + ret =dw_pcie_allocate_domains (pp); +if (ret) +return ret; + + pp->msi_page =alloc_page (GFP_KERNEL ); + pp->msi_data =dma_map_page (dev, pp->msi_page,0 ,PAGE_SIZE , +DMA_FROM_DEVICE ); + ret =dma_mapping_error (dev, pp->msi_data); +if (ret) { +dev_err (dev,"Failed to map MSI data\n" ); +__free_page (pp->msi_page); + pp->msi_page =NULL ; +dw_pcie_free_msi (pp); + } +return ret; }static const struct dw_pcie_host_ops dra7xx_pcie_host_ops = {
drivers/pci/controller/dwc/pcie-designware-host.c +13 -14
@@ -294,20 +294,7 @@ void dw_pcie_free_msi(struct pcie_port *pp)void dw_pcie_msi_init (struct pcie_port *pp) { -struct dw_pcie *pci =to_dw_pcie_from_pp (pp); -struct device *dev = pci->dev; -u64 msi_target; - - pp->msi_page =alloc_page (GFP_KERNEL ); - pp->msi_data =dma_map_page (dev, pp->msi_page,0 ,PAGE_SIZE , -DMA_FROM_DEVICE ); -if (dma_mapping_error (dev, pp->msi_data)) { -dev_err (dev,"Failed to map MSI data\n" ); -__free_page (pp->msi_page); - pp->msi_page =NULL ; -return ; - } - msi_target = (u64 )pp->msi_data; +u64 msi_target = (u64 )pp->msi_data;/* Program the msi_data */ dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_LO, 4,
@@ -440,6 +427,18 @@ int dw_pcie_host_init(struct pcie_port *pp)irq_set_chained_handler_and_data (pp->msi_irq, dw_chained_msi_isr, pp); + + pp->msi_page =alloc_page (GFP_KERNEL ); + pp->msi_data =dma_map_page (pci->dev, pp->msi_page, +0 ,PAGE_SIZE , +DMA_FROM_DEVICE ); + ret =dma_mapping_error (pci->dev, pp->msi_data); +if (ret) { +dev_err (pci->dev,"Failed to map MSI data\n" ); +__free_page (pp->msi_page); + pp->msi_page =NULL ; + goto err_free_msi; + } }else { ret = pp->ops->msi_host_init (pp);if (ret <0 )
Signature · 1 line
2.28.0
Rob Herring
reviewerQuoted 7 lines · Jisheng Zhang
Currently,
dw_pcie_msi_init()allocates and maps page for msi, then program thePCIE_MSI_ADDR_LOandPCIE_MSI_ADDR_HI. The Root Complex may lose power during suspend-to-RAM, so when we resume, we want to redo the latter but not the former. If designware based driver (for example,pcie-tegra194.c) callsdw_pcie_msi_init()in resume path, the previous msi page will be leaked.
It's worse than this. I think there's also error paths too leaking the page. Also, there's never a dma_unmap_page call which should happen before freeing.
Quoted 33 lines
Move the allocate and map msi page from
dw_pcie_msi_init()todw_pcie_host_init()to fix this problem.Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")Signed-off-by Jisheng Zhang <[email protected]>--- drivers/pci/controller/dwc/pci-dra7xx.c | 18 ++++++++++++- .../pci/controller/dwc/pcie-designware-host.c | 27 +++++++++---------- 2 files changed, 30 insertions(+), 15 deletions(-)drivers/pci/controller/dwc/pci-dra7xx.c +6
-1@@ -490,7 +490,9 @@ static struct irq_chip dra7xx_pci_msi_bottom_irq_chip = {static int dra7xx_pcie_msi_host_init (struct pcie_port *pp) {struct dw_pcie *pci =to_dw_pcie_from_pp (pp); +struct device *dev = pci->dev;u32 ctrl, num_ctrls; +int ret; pp->msi_irq_chip = &dra7xx_pci_msi_bottom_irq_chip;@@ -506,7 +508,21 @@ static int dra7xx_pcie_msi_host_init(struct pcie_port *pp) ~0); } -return dw_pcie_allocate_domains (pp); + ret =dw_pcie_allocate_domains (pp); +if (ret) +return ret; +
drivers/pci/controller/dwc/pci-dra7xx.c static int dra7xx_pcie_msi_host_init(struct pcie_port *pp) · line 524
Quoted 10 lines
drivers/pci/controller/dwc/pci-dra7xx.c +10
-0+ pp->msi_page =alloc_page (GFP_KERNEL ); + pp->msi_data =dma_map_page (dev, pp->msi_page,0 ,PAGE_SIZE , +DMA_FROM_DEVICE ); + ret =dma_mapping_error (dev, pp->msi_data); +if (ret) { +dev_err (dev,"Failed to map MSI data\n" ); +__free_page (pp->msi_page); + pp->msi_page =NULL ; +dw_pcie_free_msi (pp); + }
I don't like having 2 copies of the same thing. Also, doesn't keystone need this too?
The other thing is .msi_host_init() is abused by having an empty function to disable MSI support. We should have a flag instead to enable/disable MSI support and then we can key off of that in the common code.
drivers/pci/controller/dwc/pcie-designware-host.c void dw_pcie_free_msi(struct pcie_port *pp) · line 295
Quoted 11 lines
drivers/pci/controller/dwc/pci-dra7xx.c +1
-0+return ret; }static const struct dw_pcie_host_ops dra7xx_pcie_host_ops = {drivers/pci/controller/dwc/pcie-designware-host.c +0
-0@@ -294,20 +294,7 @@ void dw_pcie_free_msi(struct pcie_port *pp)void dw_pcie_msi_init (struct pcie_port *pp)
Might be good to rename this function with exactly what it does. There's too many 'init' and 'setup' functions...
Quoted 41 lines
drivers/pci/controller/dwc/pcie-designware-host.c +13
-15{ -struct dw_pcie *pci =to_dw_pcie_from_pp (pp); -struct device *dev = pci->dev; -u64 msi_target; - - pp->msi_page =alloc_page (GFP_KERNEL ); - pp->msi_data =dma_map_page (dev, pp->msi_page,0 ,PAGE_SIZE , -DMA_FROM_DEVICE ); -if (dma_mapping_error (dev, pp->msi_data)) { -dev_err (dev,"Failed to map MSI data\n" ); -__free_page (pp->msi_page); - pp->msi_page =NULL ; -return ; - } - msi_target = (u64 )pp->msi_data; +u64 msi_target = (u64 )pp->msi_data;/* Program the msi_data */ dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_LO, 4,@@ -440,6 +427,18 @@ int dw_pcie_host_init(struct pcie_port *pp)irq_set_chained_handler_and_data (pp->msi_irq, dw_chained_msi_isr, pp); + + pp->msi_page =alloc_page (GFP_KERNEL ); + pp->msi_data =dma_map_page (pci->dev, pp->msi_page, +0 ,PAGE_SIZE , +DMA_FROM_DEVICE ); + ret =dma_mapping_error (pci->dev, pp->msi_data); +if (ret) { +dev_err (pci->dev,"Failed to map MSI data\n" ); +__free_page (pp->msi_page); + pp->msi_page =NULL ; + goto err_free_msi; + } }else { ret = pp->ops->msi_host_init (pp);if (ret <0 ) --2.28.0
Hi Rob,
Quoted 4 lines · On Wed, 23 Sep 2020 10:41:45 -0600 Rob Herring wrote:
Quoted 1 line · Jisheng Zhang
Quoted 8 lines
Quoted 6 lines
Currently,
dw_pcie_msi_init()allocates and maps page for msi, then program thePCIE_MSI_ADDR_LOandPCIE_MSI_ADDR_HI. The Root Complex may lose power during suspend-to-RAM, so when we resume, we want to redo the latter but not the former. If designware based driver (for example,pcie-tegra194.c) callsdw_pcie_msi_init()in resume path, the previous msi page will be leaked.It's worse than this. I think there's also error paths too leaking the
I think you mean the leaking in pcie-tegra194.c's error path. Synaptics SoC pcie driver(not mainlined) needs to call dw_pcie_msi_init() in resume path, pcie-tegra194.c shares the same problem, so I mentioned it in the commit msg, but the patch isn't targeting to fix all the leaking issues in pcie-tegra194.c. This patch at least fix one of the issue.
Quoted 2 lines
page. Also, there's never a
dma_unmap_pagecall which should happen before freeing.
Thanks for pointing it out. I will add it in v2
Quoted 39 lines
Quoted 39 lines
Move the allocate and map msi page from
dw_pcie_msi_init()todw_pcie_host_init()to fix this problem.Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")Signed-off-by Jisheng Zhang <[email protected]>--- drivers/pci/controller/dwc/pci-dra7xx.c | 18 ++++++++++++- .../pci/controller/dwc/pcie-designware-host.c | 27 +++++++++---------- 2 files changed, 30 insertions(+), 15 deletions(-)drivers/pci/controller/dwc/pci-dra7xx.c +11
-1@@ -490,7 +490,9 @@ static struct irq_chip dra7xx_pci_msi_bottom_irq_chip = {static int dra7xx_pcie_msi_host_init (struct pcie_port *pp) {struct dw_pcie *pci =to_dw_pcie_from_pp (pp); +struct device *dev = pci->dev;u32 ctrl, num_ctrls; +int ret; pp->msi_irq_chip = &dra7xx_pci_msi_bottom_irq_chip;@@ -506,7 +508,21 @@ static int dra7xx_pcie_msi_host_init(struct pcie_port *pp) ~0); } -return dw_pcie_allocate_domains (pp); + ret =dw_pcie_allocate_domains (pp); +if (ret) +return ret; + + pp->msi_page =alloc_page (GFP_KERNEL ); + pp->msi_data =dma_map_page (dev, pp->msi_page,0 ,PAGE_SIZE , +DMA_FROM_DEVICE ); + ret =dma_mapping_error (dev, pp->msi_data); +if (ret) {
Quoted 8 lines
drivers/pci/controller/dwc/pci-dra7xx.c static int dra7xx_pcie_msi_host_init(struct pcie_port *pp) · line 524
Quoted 5 lines
drivers/pci/controller/dwc/pci-dra7xx.c +5
-0+dev_err (dev,"Failed to map MSI data\n" ); +__free_page (pp->msi_page); + pp->msi_page =NULL ; +dw_pcie_free_msi (pp); + }I don't like having 2 copies of the same thing. Also, doesn't keystone need this too?
what about introduce dw_pcie_msi_alloc() to do this?
IIUC, keystone doesn't need this.
Quoted 5 lines
The other thing is .msi_host_init() is abused by having an empty function to disable MSI support. We should have a flag instead to enable/disable MSI support and then we can key off of that in the common code.
FWICT, the .msi_host_init() is to init soc's own msi support rather than init the DWC's integrated MSI module. So the usage is correct.
Quoted 11 lines
Quoted 11 lines
drivers/pci/controller/dwc/pci-dra7xx.c +1
-0+return ret; }static const struct dw_pcie_host_ops dra7xx_pcie_host_ops = {drivers/pci/controller/dwc/pcie-designware-host.c +0
-0@@ -294,20 +294,7 @@ void dw_pcie_free_msi(struct pcie_port *pp)
Quoted 4 lines
drivers/pci/controller/dwc/pcie-designware-host.c void dw_pcie_free_msi(struct pcie_port *pp) · line 295
Quoted 1 line
drivers/pci/controller/dwc/pcie-designware-host.c +0
-0void dw_pcie_msi_init (struct pcie_port *pp)Might be good to rename this function with exactly what it does. There's too many 'init' and 'setup' functions...
If we move the msi page allocation out of dw_pcie_msi_init(), then it only initializes the integrated MSI.
Thanks
Quoted 42 lines
Quoted 42 lines
drivers/pci/controller/dwc/pcie-designware-host.c +13
-15{ -struct dw_pcie *pci =to_dw_pcie_from_pp (pp); -struct device *dev = pci->dev; -u64 msi_target; - - pp->msi_page =alloc_page (GFP_KERNEL ); - pp->msi_data =dma_map_page (dev, pp->msi_page,0 ,PAGE_SIZE , -DMA_FROM_DEVICE ); -if (dma_mapping_error (dev, pp->msi_data)) { -dev_err (dev,"Failed to map MSI data\n" ); -__free_page (pp->msi_page); - pp->msi_page =NULL ; -return ; - } - msi_target = (u64 )pp->msi_data; +u64 msi_target = (u64 )pp->msi_data;/* Program the msi_data */ dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_LO, 4,@@ -440,6 +427,18 @@ int dw_pcie_host_init(struct pcie_port *pp)irq_set_chained_handler_and_data (pp->msi_irq, dw_chained_msi_isr, pp); + + pp->msi_page =alloc_page (GFP_KERNEL ); + pp->msi_data =dma_map_page (pci->dev, pp->msi_page, +0 ,PAGE_SIZE , +DMA_FROM_DEVICE ); + ret =dma_mapping_error (pci->dev, pp->msi_data); +if (ret) { +dev_err (pci->dev,"Failed to map MSI data\n" ); +__free_page (pp->msi_page); + pp->msi_page =NULL ; + goto err_free_msi; + } }else { ret = pp->ops->msi_host_init (pp);if (ret <0 ) --2.28.0
Quoted 8 lines · Jisheng Zhang
Currently,
dw_pcie_msi_init()allocates and maps page for msi, then program thePCIE_MSI_ADDR_LOandPCIE_MSI_ADDR_HI. The Root Complex may lose power during suspend-to-RAM, so when we resume, we want to redo the latter but not the former. If designware based driver (for example,pcie-tegra194.c) callsdw_pcie_msi_init()in resume path, the previous msi page will be leaked.
Quoted 5 lines
Move the allocate and map msi page from
dw_pcie_msi_init()todw_pcie_host_init()to fix this problem.Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")Signed-off-by Jisheng Zhang <[email protected]>
Why do you allocate a page for this in the first place? Isn't PCIE_MSI_ADDR_HI:PCIE_MSI_ADDR_LO simply a magic DMA address that never gets forwarded across to the CPU side of the host bridge, and triggers a SPI instead, which gets handled by reading PCIE_MSI_INTR0_STATUS ?
Couldn't you just map the zero page instead?
Quoted 95 lines
--- drivers/pci/controller/dwc/pci-dra7xx.c | 18 ++++++++++++- .../pci/controller/dwc/pcie-designware-host.c | 27 +++++++++---------- 2 files changed, 30 insertions(+), 15 deletions(-)drivers/pci/controller/dwc/pci-dra7xx.c +17
-1@@ -490,7 +490,9 @@ static struct irq_chip dra7xx_pci_msi_bottom_irq_chip = {static int dra7xx_pcie_msi_host_init (struct pcie_port *pp) {struct dw_pcie *pci =to_dw_pcie_from_pp (pp); +struct device *dev = pci->dev;u32 ctrl, num_ctrls; +int ret; pp->msi_irq_chip = &dra7xx_pci_msi_bottom_irq_chip;@@ -506,7 +508,21 @@ static int dra7xx_pcie_msi_host_init(struct pcie_port *pp) ~0); } -return dw_pcie_allocate_domains (pp); + ret =dw_pcie_allocate_domains (pp); +if (ret) +return ret; + + pp->msi_page =alloc_page (GFP_KERNEL ); + pp->msi_data =dma_map_page (dev, pp->msi_page,0 ,PAGE_SIZE , +DMA_FROM_DEVICE ); + ret =dma_mapping_error (dev, pp->msi_data); +if (ret) { +dev_err (dev,"Failed to map MSI data\n" ); +__free_page (pp->msi_page); + pp->msi_page =NULL ; +dw_pcie_free_msi (pp); + } +return ret; }static const struct dw_pcie_host_ops dra7xx_pcie_host_ops = {drivers/pci/controller/dwc/pcie-designware-host.c +13
-15@@ -294,20 +294,7 @@ void dw_pcie_free_msi(struct pcie_port *pp)void dw_pcie_msi_init (struct pcie_port *pp) { -struct dw_pcie *pci =to_dw_pcie_from_pp (pp); -struct device *dev = pci->dev; -u64 msi_target; - - pp->msi_page =alloc_page (GFP_KERNEL ); - pp->msi_data =dma_map_page (dev, pp->msi_page,0 ,PAGE_SIZE , -DMA_FROM_DEVICE ); -if (dma_mapping_error (dev, pp->msi_data)) { -dev_err (dev,"Failed to map MSI data\n" ); -__free_page (pp->msi_page); - pp->msi_page =NULL ; -return ; - } - msi_target = (u64 )pp->msi_data; +u64 msi_target = (u64 )pp->msi_data;/* Program the msi_data */ dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_LO, 4,@@ -440,6 +427,18 @@ int dw_pcie_host_init(struct pcie_port *pp)irq_set_chained_handler_and_data (pp->msi_irq, dw_chained_msi_isr, pp); + + pp->msi_page =alloc_page (GFP_KERNEL ); + pp->msi_data =dma_map_page (pci->dev, pp->msi_page, +0 ,PAGE_SIZE , +DMA_FROM_DEVICE ); + ret =dma_mapping_error (pci->dev, pp->msi_data); +if (ret) { +dev_err (pci->dev,"Failed to map MSI data\n" ); +__free_page (pp->msi_page); + pp->msi_page =NULL ; + goto err_free_msi; + } }else { ret = pp->ops->msi_host_init (pp);if (ret <0 ) --2.28.0
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Rob Herring
reviewerQuoted 13 lines · Ard Biesheuvel
Quoted 11 lines · Jisheng Zhang
Currently,
dw_pcie_msi_init()allocates and maps page for msi, then program thePCIE_MSI_ADDR_LOandPCIE_MSI_ADDR_HI. The Root Complex may lose power during suspend-to-RAM, so when we resume, we want to redo the latter but not the former. If designware based driver (for example,pcie-tegra194.c) callsdw_pcie_msi_init()in resume path, the previous msi page will be leaked.Move the allocate and map msi page from
dw_pcie_msi_init()todw_pcie_host_init()to fix this problem.
Quoted 8 lines
Quoted 2 lines
Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")Signed-off-by Jisheng Zhang <[email protected]>Why do you allocate a page for this in the first place? Isn't PCIE_MSI_ADDR_HI:PCIE_MSI_ADDR_LO simply a magic DMA address that never gets forwarded across to the CPU side of the host bridge, and triggers a SPI instead, which gets handled by reading
PCIE_MSI_INTR0_STATUS?
My question too after digging into this some more. I've asked the question on the thread that further complicated all this changing from virt_to_phys() to dma_map_page()[1].
Quoted 1 line
Couldn't you just map the zero page instead?
Why a page even? You could use PCIE_MSI_ADDR_LO address itself even. Or just an address in the driver data which is what some other drivers do.
Rob
[1] https://lore.kernel.org/linux-pci/20200923231846.GA1499246@bogus/
Quoted 28 lines · Rob Herring
Quoted 13 lines · Ard Biesheuvel
Quoted 11 lines · Jisheng Zhang
Currently,
dw_pcie_msi_init()allocates and maps page for msi, then program thePCIE_MSI_ADDR_LOandPCIE_MSI_ADDR_HI. The Root Complex may lose power during suspend-to-RAM, so when we resume, we want to redo the latter but not the former. If designware based driver (for example,pcie-tegra194.c) callsdw_pcie_msi_init()in resume path, the previous msi page will be leaked.Move the allocate and map msi page from
dw_pcie_msi_init()todw_pcie_host_init()to fix this problem.Quoted 8 lines
Quoted 2 lines
Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")Signed-off-by Jisheng Zhang <[email protected]>Why do you allocate a page for this in the first place? Isn't PCIE_MSI_ADDR_HI:PCIE_MSI_ADDR_LO simply a magic DMA address that never gets forwarded across to the CPU side of the host bridge, and triggers a SPI instead, which gets handled by reading
PCIE_MSI_INTR0_STATUS?My question too after digging into this some more. I've asked the question on the thread that further complicated all this changing from
virt_to_phys()to dma_map_page()[1].
Quoted 6 lines
Quoted 1 line
Couldn't you just map the zero page instead?
Why a page even? You could use
PCIE_MSI_ADDR_LOaddress itself even. Or just an address in the driver data which is what some other drivers do.
PCIE_MSI_ADDR_LO itself could collide with an actual DRAM address if any translation is applied on inbound transactions. Using an actual DRAM address avoids that.
Quoted 32 lines · Ard Biesheuvel
Quoted 30 lines · Rob Herring
Quoted 13 lines · Ard Biesheuvel
Quoted 11 lines · Jisheng Zhang
Currently,
dw_pcie_msi_init()allocates and maps page for msi, then program thePCIE_MSI_ADDR_LOandPCIE_MSI_ADDR_HI. The Root Complex may lose power during suspend-to-RAM, so when we resume, we want to redo the latter but not the former. If designware based driver (for example,pcie-tegra194.c) callsdw_pcie_msi_init()in resume path, the previous msi page will be leaked.Move the allocate and map msi page from
dw_pcie_msi_init()todw_pcie_host_init()to fix this problem.Quoted 8 lines
Quoted 2 lines
Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")Signed-off-by Jisheng Zhang <[email protected]>Why do you allocate a page for this in the first place? Isn't PCIE_MSI_ADDR_HI:PCIE_MSI_ADDR_LO simply a magic DMA address that never gets forwarded across to the CPU side of the host bridge, and triggers a SPI instead, which gets handled by reading
PCIE_MSI_INTR0_STATUS?My question too after digging into this some more. I've asked the question on the thread that further complicated all this changing from
virt_to_phys()to dma_map_page()[1].Quoted 1 line
Couldn't you just map the zero page instead?
Quoted 8 lines
Quoted 4 lines
Why a page even? You could use
PCIE_MSI_ADDR_LOaddress itself even. Or just an address in the driver data which is what some other drivers do.
PCIE_MSI_ADDR_LOitself could collide with an actual DRAM address if any translation is applied on inbound transactions. Using an actual DRAM address avoids that.
... although the MSI doorbell register on the GIC, for instance, needs to be DMA addressable as well, of course.
Rob Herring
reviewerQuoted 34 lines · Ard Biesheuvel
Quoted 32 lines · Ard Biesheuvel
Quoted 30 lines · Rob Herring
Quoted 13 lines · Ard Biesheuvel
Quoted 11 lines · Jisheng Zhang
Currently,
dw_pcie_msi_init()allocates and maps page for msi, then program thePCIE_MSI_ADDR_LOandPCIE_MSI_ADDR_HI. The Root Complex may lose power during suspend-to-RAM, so when we resume, we want to redo the latter but not the former. If designware based driver (for example,pcie-tegra194.c) callsdw_pcie_msi_init()in resume path, the previous msi page will be leaked.Move the allocate and map msi page from
dw_pcie_msi_init()todw_pcie_host_init()to fix this problem.Quoted 8 lines
Quoted 2 lines
Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")Signed-off-by Jisheng Zhang <[email protected]>Why do you allocate a page for this in the first place? Isn't PCIE_MSI_ADDR_HI:PCIE_MSI_ADDR_LO simply a magic DMA address that never gets forwarded across to the CPU side of the host bridge, and triggers a SPI instead, which gets handled by reading
PCIE_MSI_INTR0_STATUS?My question too after digging into this some more. I've asked the question on the thread that further complicated all this changing from
virt_to_phys()to dma_map_page()[1].Quoted 1 line
Couldn't you just map the zero page instead?
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Why a page even? You could use
PCIE_MSI_ADDR_LOaddress itself even. Or just an address in the driver data which is what some other drivers do.
PCIE_MSI_ADDR_LOitself could collide with an actual DRAM address if any translation is applied on inbound transactions. Using an actual DRAM address avoids that.
Good point, and the inbound windows could be less than all DRAM if there's any restrictions. However, the DWC doesn't do any inbound setup (at least for platforms with iATU), so I guess the default is all traffic is forwarded.
Quoted 2 lines
... although the MSI doorbell register on the GIC, for instance, needs to be DMA addressable as well, of course.
There's at least one DWC driver that has its own doorbell register too. I suppose there's a way for the host driver to retrieve the GIC address if configuration is needed. Doesn't seem to be needed on DWC given the above.
Rob
Quoted 30 lines · On Thu, 24 Sep 2020 07:28:25 -0600 Rob Herring <[email protected]> wrote:
CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
Quoted 13 lines · Ard Biesheuvel
Quoted 11 lines · Jisheng Zhang
Currently,
dw_pcie_msi_init()allocates and maps page for msi, then program thePCIE_MSI_ADDR_LOandPCIE_MSI_ADDR_HI. The Root Complex may lose power during suspend-to-RAM, so when we resume, we want to redo the latter but not the former. If designware based driver (for example,pcie-tegra194.c) callsdw_pcie_msi_init()in resume path, the previous msi page will be leaked.Move the allocate and map msi page from
dw_pcie_msi_init()todw_pcie_host_init()to fix this problem.Quoted 8 lines
Quoted 2 lines
Fixes 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")Signed-off-by Jisheng Zhang <[email protected]>Why do you allocate a page for this in the first place? Isn't PCIE_MSI_ADDR_HI:PCIE_MSI_ADDR_LO simply a magic DMA address that never gets forwarded across to the CPU side of the host bridge, and triggers a SPI instead, which gets handled by reading
PCIE_MSI_INTR0_STATUS?My question too after digging into this some more. I've asked the question on the thread that further complicated all this changing from
virt_to_phys()to dma_map_page()[1].
Quoted 6 lines
Quoted 1 line
Couldn't you just map the zero page instead?
Why a page even? You could use
PCIE_MSI_ADDR_LOaddress itself even. Or just an address in the driver data which is what some other drivers do.
Thank Ard and Rob. I have sent out V3 (https://lkml.org/lkml/2020/9/25/272)
which uses an address in the driver data for MSI address.
Hi Niklas,
can you please try v3 to see whether it solves your problem as well?
Thanks