Skip to contentSkip to conversation

PCI: dwc: Set 32-bit DMA mask for MSI target address allocation

PATCH Applied Vidya Sagar 3 participants 4 messages Nov 17, 2020 – Dec 10, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 PCI: dwc: Set 32-bit DMA mask for MSI target address allocation

Set DMA mask to 32-bit while allocating the MSI target address so that the address is usable for both 32-bit and 64-bit MSI capable devices. Throw a warning if it fails to set the mask to 32-bit to alert that devices that are only 32-bit MSI capable may not work properly.

---
Given the other patch that I've pushed to the MSI sub-system
http://patchwork.ozlabs.org/project/linux-pci/patch/[email protected]/
which is going to catch any mismatch between MSI capability (32-bit) of the device and system's inability to allocate the required MSI target address, I'm not sure how much sense is this patch going to be make. But, I can certainly say that if the memory allocation mechanism gives the addresses from 64-bit pool by default, this patch at least makes sure that MSI target address is allocated from 32-bit pool.

 drivers/pci/controller/dwc/pcie-designware-host.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/pci/controller/dwc/pcie-designware-host.c +8 -0
index 44c2a6572199..e6a230eddf66 100644
@@ -388,6 +388,14 @@ int dw_pcie_host_init(struct pcie_port *pp)
 							    dw_chained_msi_isr,
 							    pp);
 
+			ret = dma_set_mask(pci->dev, DMA_BIT_MASK(32));
+			if (!ret) {
+				dev_warn(pci->dev,
+					 "Failed to set DMA mask to 32-bit. "
+					 "Devices with only 32-bit MSI support"
+					 " may not work properly\n");
+			}
+
 			pp->msi_data = dma_map_single_attrs(pci->dev, &pp->msi_msg,
 						      sizeof(pp->msi_msg),
 						      DMA_FROM_DEVICE,
Signature · 1 line
2.17.1
#1 lore raw

Hi,
Could you please review this patch in the context of the following patch?
http://patchwork.ozlabs.org/project/linux-pci/patch/[email protected]/

Thanks,
Vidya Sagar

Quoted 39 lines · Vidya Sagar

Set DMA mask to 32-bit while allocating the MSI target address so that the address is usable for both 32-bit and 64-bit MSI capable devices. Throw a warning if it fails to set the mask to 32-bit to alert that devices that are only 32-bit MSI capable may not work properly.

---
Given the other patch that I've pushed to the MSI sub-system
http://patchwork.ozlabs.org/project/linux-pci/patch/[email protected]/
which is going to catch any mismatch between MSI capability (32-bit) of the device and system's inability to allocate the required MSI target address, I'm not sure how much sense is this patch going to be make. But, I can certainly say that if the memory allocation mechanism gives the addresses from 64-bit pool by default, this patch at least makes sure that MSI target address is allocated from 32-bit pool.

  drivers/pci/controller/dwc/pcie-designware-host.c | 8 ++++++++
  1 file changed, 8 insertions(+)
drivers/pci/controller/dwc/pcie-designware-host.c +8 -0
index 44c2a6572199..e6a230eddf66 100644
@@ -388,6 +388,14 @@ int dw_pcie_host_init(struct pcie_port *pp)
  							    dw_chained_msi_isr,
  							    pp);
 
+			ret = dma_set_mask(pci->dev, DMA_BIT_MASK(32));
+			if (!ret) {
+				dev_warn(pci->dev,
+					 "Failed to set DMA mask to 32-bit. "
+					 "Devices with only 32-bit MSI support"
+					 " may not work properly\n");
+			}
+
  			pp->msi_data = dma_map_single_attrs(pci->dev, &pp->msi_msg,
  						      sizeof(pp->msi_msg),
  						      DMA_FROM_DEVICE,
Reviewed-by on v1 1/1
#2 lore raw
In reply to Vidya Sagar
Quoted 4 lines · Vidya Sagar

Set DMA mask to 32-bit while allocating the MSI target address so that the address is usable for both 32-bit and 64-bit MSI capable devices. Throw a warning if it fails to set the mask to 32-bit to alert that devices that are only 32-bit MSI capable may not work properly.

This is slightly wacky, but no more so than the rest of the not-DMA shenanigans here... Ultimately it probably is the least-worst way to avoid the issue, so in terms of functionality,

Quoted 10 lines

---
Given the other patch that I've pushed to the MSI sub-system
http://patchwork.ozlabs.org/project/linux-pci/patch/[email protected]/
which is going to catch any mismatch between MSI capability (32-bit) of the device and system's inability to allocate the required MSI target address, I'm not sure how much sense is this patch going to be make. But, I can certainly say that if the memory allocation mechanism gives the addresses from 64-bit pool by default, this patch at least makes sure that MSI target address is allocated from 32-bit pool.

Note that this doesn't change where anything is allocated as such, it just means that on systems with most of their RAM above 4GB, those few bytes of private data that you map "for free" will be copied into the SWIOTLB buffer and hog 2KB of its typical 64MB capacity effectively for ever.

Quoted 8 lines
  drivers/pci/controller/dwc/pcie-designware-host.c | 8 ++++++++
  1 file changed, 8 insertions(+)
drivers/pci/controller/dwc/pcie-designware-host.c +0 -0
index 44c2a6572199..e6a230eddf66 100644

drivers/pci/controller/dwc/pcie-designware-host.c int dw_pcie_host_init(struct pcie_port *pp) · line 397

Quoted 10 lines
drivers/pci/controller/dwc/pcie-designware-host.c +7 -0
@@ -388,6 +388,14 @@ int dw_pcie_host_init(struct pcie_port *pp)
  							    dw_chained_msi_isr,
  							    pp);
 
+			ret = dma_set_mask(pci->dev, DMA_BIT_MASK(32));
+			if (!ret) {
+				dev_warn(pci->dev,
+					 "Failed to set DMA mask to 32-bit. "
+					 "Devices with only 32-bit MSI support"
+					 " may not work properly\n");
+			}

Ironically, the only real reason for that dma_set_mask() to ever fail is if the system had no 32-bit addressable memory, in which case you could likely pick any 32-bit doorbell address with impunity, just not via this mechanism (although whether it would be worthwhile is another matter).

Robin.

Quoted 5 lines
drivers/pci/controller/dwc/pcie-designware-host.c +1 -0
+
  			pp->msi_data = dma_map_single_attrs(pci->dev, &pp->msi_msg,
  						      sizeof(pp->msi_msg),
  						      DMA_FROM_DEVICE,
#3 lore raw
In reply to Vidya Sagar
Quoted 4 lines · Vidya Sagar

Set DMA mask to 32-bit while allocating the MSI target address so that the address is usable for both 32-bit and 64-bit MSI capable devices. Throw a warning if it fails to set the mask to 32-bit to alert that devices that are only 32-bit MSI capable may not work properly.

Applied to pci/dwc, thanks!

[1/1] PCI: dwc: Set 32-bit DMA mask for MSI target address allocation

      https://git.kernel.org/lpieralisi/pci/c/660c486590

Thanks,
Lorenzo