Skip to contentSkip to conversation

PCI: mvebu: Dispose INTx IRQs before to removing INTx domain

PATCH In review Marek Behún 3 participants 5 messages Jul 15, 2024 – Oct 17, 2024

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v3 cover PCI: mvebu: Dispose INTx IRQs before to removing INTx domain

Hello Bjorn, Thomas, Ilpo et al.

I've split the patch into two: the first introduces the new helper function, pci_remove_irq_domain(), and the second uses this new function instead of irq_domain_remove() in the pci-mvebu controller driver.

Marek

Marek Behún (1):
PCI: Add pci_remove_irq_domain() helper

Pali Rohár (1):
PCI: mvebu: Dispose INTx IRQs before to removing INTx domain

 drivers/pci/controller/pci-mvebu.c |  2 +-
 drivers/pci/irq.c                  | 21 +++++++++++++++++++++
 drivers/pci/pci.h                  |  7 +++++++
 3 files changed, 29 insertions(+), 1 deletion(-)
Signature · 1 line
2.44.2
#1 lore raw

PATCH v3 1/2 PCI: Add pci_remove_irq_domain() helper

Add a helper function pci_remove_irq_domain() for disposing all interrupt mappings of an IRQ domain and then removing said IRQ domain.

As explained in the attached link, the PCI INTX interrupt may be shared, and so the PCI device drivers do not dispose mapped interrupts when they are unbound from a device, since other devices may be still using those mapped interrupts. Thus the interrupts must be disposed by the PCI controller driver when the IRQ domain is being removed.

This function may be used by PCI controller drivers that wish to be removable / modular.

---
 drivers/pci/irq.c | 21 +++++++++++++++++++++
 drivers/pci/pci.h |  7 +++++++
 2 files changed, 28 insertions(+)
drivers/pci/irq.c +21 -0
index 4555630be9ec..30c8d930016a 100644
@@ -11,6 +11,7 @@
 #include <linux/errno.h>
 #include <linux/export.h>
 #include <linux/interrupt.h>
+#include <linux/irqdomain.h>
 #include <linux/pci.h>
 
 #include "pci.h"
@@ -259,6 +260,26 @@ bool pci_check_and_unmask_intx(struct pci_dev *dev)
 }
 EXPORT_SYMBOL_GPL(pci_check_and_unmask_intx);
 
+#ifdef CONFIG_IRQ_DOMAIN
+/**
+ * pci_remove_irq_domain - dispose all IRQ mappings and remove IRQ domain
+ * @domain: the IRQ domain to be removed
+ *
+ * Disposes all IRQ mappings of a given IRQ domain before removing the domain.
+ */
+void pci_remove_irq_domain(struct irq_domain *domain)
+{
+	for (irq_hw_number_t i = 0; i < domain->hwirq_max; i++) {
+		unsigned int virq = irq_find_mapping(domain, i);
+
+		if (virq)
+			irq_dispose_mapping(virq);
+	}
+
+	irq_domain_remove(domain);
+}
+#endif
+
 /**
  * pcibios_penalize_isa_irq - penalize an ISA IRQ
  * @irq: ISA IRQ to penalize
drivers/pci/pci.h +7 -0
index fd44565c4756..1ba6a6f418ac 100644
@@ -170,6 +170,13 @@ void pci_no_msi(void);
 static inline void pci_no_msi(void) { }
 #endif
 
+struct irq_domain;
+#ifdef CONFIG_IRQ_DOMAIN
+void pci_remove_irq_domain(struct irq_domain *domain);
+#else
+static inline void pci_remove_irq_domain(struct irq_domain *domain) { }
+#endif
+
 void pci_realloc_get_opt(char *);
 
 static inline int pci_no_d1d2(struct pci_dev *dev)
Signature · 1 line
2.44.2
#4 lore raw

PATCH v3 2/2 PCI: mvebu: Dispose INTx IRQs before to removing INTx domain

In reply to Marek Behún

From: Pali Rohár <[email protected]>

The documentation for the irq_domain_remove() function says that all mappings within the IRQ domain must be disposed before the domain is removed.

Use the new helper function pci_remove_irq_domain() that also disposes all IRQ mappings of a IRQ domain before removing sait domain.

Currently, the INTx IRQs are not disposed in pci-mvebu driver .remove() method, which causes the kernel to crash when unloading the driver and then reading /sys/kernel/debug/irq/irqs/<num> or /proc/interrupts.

Unmapping of the IRQs at this point of the .remove() method is safe, since the PCIe bus is already unregistered, and all its devices are unbound from their drivers and removed. If there was indeed any remaining use of PCIe resources, then it would mean that PCIe hotplug code is broken, and we have bigger problems.

Fixes ec075262648f ("PCI: mvebu: Implement support for legacy INTx interrupts")
[ Marek: refactored a little, added more explanation to commit message ]
---
 drivers/pci/controller/pci-mvebu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/pci/controller/pci-mvebu.c +1 -1
index 29fe09c99e7d..590f121bd91a 100644
@@ -1684,7 +1684,7 @@ static void mvebu_pcie_remove(struct platform_device *pdev)
 
 		/* Remove IRQ domains. */
 		if (port->intx_irq_domain)
-			irq_domain_remove(port->intx_irq_domain);
+			pci_remove_irq_domain(port->intx_irq_domain);
 
 		/* Free config space for emulated root bridge. */
 		pci_bridge_emul_cleanup(&port->bridge);
Signature · 1 line
2.44.2
Reviewed-by on v3 1/2
#2 lore raw

Re: [PATCH v3 1/2] PCI: Add pci_remove_irq_domain() helper

In reply to Marek Behún
Quoted 9 lines · Marek Behún

Add a helper function pci_remove_irq_domain() for disposing all interrupt mappings of an IRQ domain and then removing said IRQ domain.

As explained in the attached link, the PCI INTX interrupt may be shared, and so the PCI device drivers do not dispose mapped interrupts when they are unbound from a device, since other devices may be still using those mapped interrupts. Thus the interrupts must be disposed by the PCI controller driver when the IRQ domain is being removed.

Quoted 5 lines

This function may be used by PCI controller drivers that wish to be removable / modular.

Thomas shared the diff leading to this patch. Shouldn't you give some credit to him?

For the patch though,

- Mani

Quoted 65 lines
---
 drivers/pci/irq.c | 21 +++++++++++++++++++++
 drivers/pci/pci.h |  7 +++++++
 2 files changed, 28 insertions(+)
drivers/pci/irq.c +21 -0
index 4555630be9ec..30c8d930016a 100644
@@ -11,6 +11,7 @@
 #include <linux/errno.h>
 #include <linux/export.h>
 #include <linux/interrupt.h>
+#include <linux/irqdomain.h>
 #include <linux/pci.h>
 
 #include "pci.h"
@@ -259,6 +260,26 @@ bool pci_check_and_unmask_intx(struct pci_dev *dev)
 }
 EXPORT_SYMBOL_GPL(pci_check_and_unmask_intx);
 
+#ifdef CONFIG_IRQ_DOMAIN
+/**
+ * pci_remove_irq_domain - dispose all IRQ mappings and remove IRQ domain
+ * @domain: the IRQ domain to be removed
+ *
+ * Disposes all IRQ mappings of a given IRQ domain before removing the domain.
+ */
+void pci_remove_irq_domain(struct irq_domain *domain)
+{
+	for (irq_hw_number_t i = 0; i < domain->hwirq_max; i++) {
+		unsigned int virq = irq_find_mapping(domain, i);
+
+		if (virq)
+			irq_dispose_mapping(virq);
+	}
+
+	irq_domain_remove(domain);
+}
+#endif
+
 /**
  * pcibios_penalize_isa_irq - penalize an ISA IRQ
  * @irq: ISA IRQ to penalize
drivers/pci/pci.h +7 -0
index fd44565c4756..1ba6a6f418ac 100644
@@ -170,6 +170,13 @@ void pci_no_msi(void);
 static inline void pci_no_msi(void) { }
 #endif
 
+struct irq_domain;
+#ifdef CONFIG_IRQ_DOMAIN
+void pci_remove_irq_domain(struct irq_domain *domain);
+#else
+static inline void pci_remove_irq_domain(struct irq_domain *domain) { }
+#endif
+
 void pci_realloc_get_opt(char *);
 
 static inline int pci_no_d1d2(struct pci_dev *dev)
Signature · 1 line
2.44.2
Signature · 1 line
மணிவண்ணன் சதாசிவம்
#3 lore raw

Re: [PATCH v3 1/2] PCI: Add pci_remove_irq_domain() helper

In reply to Marek Behún
Quoted 7 lines · Marek Behún
drivers/pci/irq.c +6 -0
+#ifdef CONFIG_IRQ_DOMAIN
+/**
+ * pci_remove_irq_domain - dispose all IRQ mappings and remove IRQ domain
+ * @domain: the IRQ domain to be removed
+ *
+ * Disposes all IRQ mappings of a given IRQ domain before removing

the domain.

Sure, but this lacks information what this is about and where this should be used.

Thanks,

tglx