Skip to contentSkip to conversation

PCI: MSI parent domain conversion

PATCH Applied Nam Cao 9 participants 58 messages Jun 26, 2025 – Jul 16, 2025

Subsystems (19)
lore
Share & tools
lore

Nam Cao

author
#0 lore raw

cover PCI: MSI parent domain conversion

The initial implementation of PCI/MSI interrupt domains in the hierarchical interrupt domain model used a shortcut by providing a global PCI/MSI domain.

This works because the PCI/MSI[X] hardware is standardized and uniform, but it violates the basic design principle of hierarchical interrupt domains:
Each hardware block involved in the interrupt delivery chain should have a separate interrupt domain.

For PCI/MSI[X], the interrupt controller is per PCI device and not a global made-up entity.

Unsurprisingly, the shortcut turned out to have downsides as it does not allow dynamic allocation of interrupt vectors after initialization and it prevents supporting IMS on PCI. For further details, see:

https://lore.kernel.org/lkml/[email protected]/

The solution is implementing per device MSI domains, this means the entities which provide global PCI/MSI domain so far have to implement MSI parent domain functionality instead.

This series converts the PCI controller drivers to implement MSI parent domain.

 drivers/pci/Kconfig                           |   1 +
 drivers/pci/controller/Kconfig                |  11 +
 drivers/pci/controller/dwc/Kconfig            |   1 +
 .../pci/controller/dwc/pcie-designware-host.c |  68 ++----
 drivers/pci/controller/dwc/pcie-designware.h  |   1 -
 drivers/pci/controller/mobiveil/Kconfig       |   1 +
 .../controller/mobiveil/pcie-mobiveil-host.c  |  42 ++--
 .../pci/controller/mobiveil/pcie-mobiveil.h   |   1 -
 drivers/pci/controller/pci-aardvark.c         |  59 ++---
 drivers/pci/controller/pci-hyperv.c           |  98 ++++++--
 drivers/pci/controller/pcie-altera-msi.c      |  44 ++--
 drivers/pci/controller/pcie-brcmstb.c         |  44 ++--
 drivers/pci/controller/pcie-iproc-msi.c       |  45 ++--
 drivers/pci/controller/pcie-mediatek-gen3.c   |  67 ++---
 drivers/pci/controller/pcie-mediatek.c        |  46 ++--
 drivers/pci/controller/pcie-rcar-host.c       |  69 ++----
 drivers/pci/controller/pcie-xilinx-dma-pl.c   |  48 ++--
 drivers/pci/controller/pcie-xilinx-nwl.c      |  45 ++--
 drivers/pci/controller/pcie-xilinx.c          |  55 +++--
 drivers/pci/controller/plda/Kconfig           |   1 +
 drivers/pci/controller/plda/pcie-plda-host.c  |  44 ++--
 drivers/pci/controller/plda/pcie-plda.h       |   1 -
 drivers/pci/controller/vmd.c                  | 229 +++++++++---------
 23 files changed, 504 insertions(+), 517 deletions(-)
Signature · 1 line
2.39.5
author
#1 lore raw

PATCH 1/16 PCI: dwc: Switch to msi_create_parent_irq_domain()

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Jingoo Han <[email protected]>

---
 drivers/pci/controller/dwc/Kconfig            |  1 +
 .../pci/controller/dwc/pcie-designware-host.c | 68 +++++++------------
 drivers/pci/controller/dwc/pcie-designware.h  |  1 -
 3 files changed, 25 insertions(+), 45 deletions(-)
drivers/pci/controller/dwc/Kconfig +1 -0
index bb95877b2c6c4..8a63e9bc0c039 100644
@@ -19,6 +19,7 @@ config PCIE_DW_DEBUGFS
 config PCIE_DW_HOST
 	bool
 	select PCIE_DW
+	select IRQ_MSI_LIB
 
 config PCIE_DW_EP
 	bool
drivers/pci/controller/dwc/pcie-designware-host.c +24 -44

97 lines not on this page · raw message

drivers/pci/controller/dwc/pcie-designware.h +0 -1
index ce9e18554e426..d9daee4ce220d 100644
@@ -417,7 +417,6 @@ struct dw_pcie_rp {
 	const struct dw_pcie_host_ops *ops;
 	int			msi_irq[MAX_MSI_CTRLS];
 	struct irq_domain	*irq_domain;
-	struct irq_domain	*msi_domain;
 	dma_addr_t		msi_data;
 	struct irq_chip		*msi_irq_chip;
 	u32			num_vectors;
Signature · 1 line
2.39.5
author
#3 lore raw

PATCH 2/16 PCI: mobiveil: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Karthikeyan Mitran <[email protected]>
Cc: Hou Zhiqiang <[email protected]>

---
 drivers/pci/controller/mobiveil/Kconfig       |  1 +
 .../controller/mobiveil/pcie-mobiveil-host.c  | 42 ++++++++++---------
 .../pci/controller/mobiveil/pcie-mobiveil.h   |  1 -
 3 files changed, 23 insertions(+), 21 deletions(-)
drivers/pci/controller/mobiveil/Kconfig +1 -0
index 58ce034f701ab..c50c4625937f8 100644
@@ -9,6 +9,7 @@ config PCIE_MOBIVEIL
 config PCIE_MOBIVEIL_HOST
 	bool
 	depends on PCI_MSI
+	select IRQ_MSI_LIB
 	select PCIE_MOBIVEIL
 
 config PCIE_LAYERSCAPE_GEN4
drivers/pci/controller/mobiveil/pcie-mobiveil-host.c +22 -20

63 lines not on this page · raw message

drivers/pci/controller/mobiveil/pcie-mobiveil.h +0 -1
index 662f17f9bf65c..7246de6a71768 100644
@@ -135,7 +135,6 @@
 
 struct mobiveil_msi {			/* MSI information */
 	struct mutex lock;		/* protect bitmap variable */
-	struct irq_domain *msi_domain;
 	struct irq_domain *dev_domain;
 	phys_addr_t msi_pages_phys;
 	int num_of_vectors;
Signature · 1 line
2.39.5
author
#5 lore raw

PATCH 3/16 PCI: aardvark: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Thomas Petazzoni <[email protected]>
Cc: "Pali Rohár" <[email protected]>
Cc: [email protected]

---
 drivers/pci/controller/Kconfig        |  1 +
 drivers/pci/controller/pci-aardvark.c | 59 +++++++++++----------------
 2 files changed, 24 insertions(+), 36 deletions(-)
drivers/pci/controller/Kconfig +1 -0
index 886f6f43a895f..91a2d4ffc3ac4 100644
@@ -13,6 +13,7 @@ config PCI_AARDVARK
 	depends on OF
 	depends on PCI_MSI
 	select PCI_BRIDGE_EMUL
+	select IRQ_MSI_LIB
 	help
 	 Add support for Aardvark 64bit PCIe Host Controller. This
 	 controller is part of the South Bridge of the Marvel Armada
drivers/pci/controller/pci-aardvark.c +23 -36

97 lines not on this page · raw message

Signature · 1 line
2.39.5
author
#7 lore raw

PATCH 4/16 PCI: altera-msi: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Joyce Ooi <[email protected]>

---
 drivers/pci/controller/Kconfig           |  1 +
 drivers/pci/controller/pcie-altera-msi.c | 44 +++++++++++-------------
 2 files changed, 22 insertions(+), 23 deletions(-)
drivers/pci/controller/Kconfig +1 -0
index 91a2d4ffc3ac4..012c18c67d9c6 100644
@@ -30,6 +30,7 @@ config PCIE_ALTERA_MSI
 	tristate "Altera PCIe MSI feature"
 	depends on PCIE_ALTERA
 	depends on PCI_MSI
+	select IRQ_MSI_LIB
 	help
 	  Say Y here if you want PCIe MSI support for the Altera FPGA.
 	  This MSI driver supports Altera MSI to GIC controller IP.
drivers/pci/controller/pcie-altera-msi.c +21 -23

78 lines not on this page · raw message

Signature · 1 line
2.39.5
author
#9 lore raw

PATCH 5/16 PCI: brcmstb: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Florian Fainelli <[email protected]>
Cc: Broadcom internal kernel review list <[email protected]>
Cc: Jim Quinlan <[email protected]>
Cc: Nicolas Saenz Julienne <[email protected]>
Cc: [email protected]
Cc: [email protected]

---
 drivers/pci/controller/Kconfig        |  1 +
 drivers/pci/controller/pcie-brcmstb.c | 44 +++++++++++++--------------
 2 files changed, 23 insertions(+), 22 deletions(-)
drivers/pci/controller/Kconfig +1 -0
index 012c18c67d9c6..0f6cec244d4fa 100644
@@ -64,6 +64,7 @@ config PCIE_BRCMSTB
 		   BMIPS_GENERIC || COMPILE_TEST
 	depends on OF
 	depends on PCI_MSI
+	select IRQ_MSI_LIB
 	default ARCH_BRCMSTB || BMIPS_GENERIC
 	help
 	  Say Y here to enable PCIe host controller support for
drivers/pci/controller/pcie-brcmstb.c +22 -22

77 lines not on this page · raw message

Signature · 1 line
2.39.5
author
#12 lore raw

PATCH 6/16 PCI: iproc: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Ray Jui <[email protected]>
Cc: Scott Branden <[email protected]>
Cc: Broadcom internal kernel review list <[email protected]>
Cc: [email protected]

---
 drivers/pci/controller/Kconfig          |  1 +
 drivers/pci/controller/pcie-iproc-msi.c | 45 ++++++++++++-------------
 2 files changed, 22 insertions(+), 24 deletions(-)
drivers/pci/controller/Kconfig +1 -0
index 0f6cec244d4fa..375a019f35bd9 100644
@@ -101,6 +101,7 @@ config PCIE_IPROC_MSI
 	bool "Broadcom iProc PCIe MSI support"
 	depends on PCIE_IPROC_PLATFORM || PCIE_IPROC_BCMA
 	depends on PCI_MSI
+	select IRQ_MSI_LIB
 	default ARCH_BCM_IPROC
 	help
 	  Say Y here if you want to enable MSI support for Broadcom's iProc
drivers/pci/controller/pcie-iproc-msi.c +21 -24

84 lines not on this page · raw message

Signature · 1 line
2.39.5
author
#15 lore raw

PATCH 7/16 PCI: mediatek-gen3: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Ryder Lee <[email protected]>
Cc: Jianjun Wang <[email protected]>
Cc: Matthias Brugger <[email protected]>
Cc: AngeloGioacchino Del Regno <[email protected]>
Cc: [email protected]
Cc: [email protected]

---
 drivers/pci/controller/Kconfig              |  1 +
 drivers/pci/controller/pcie-mediatek-gen3.c | 67 ++++++++-------------
 2 files changed, 26 insertions(+), 42 deletions(-)
drivers/pci/controller/Kconfig +1 -0
index 375a019f35bd9..ec32c343a751d 100644
@@ -203,6 +203,7 @@ config PCIE_MEDIATEK_GEN3
 	tristate "MediaTek Gen3 PCIe controller"
 	depends on ARCH_AIROHA || ARCH_MEDIATEK || COMPILE_TEST
 	depends on PCI_MSI
+	select IRQ_MSI_LIB
 	help
 	  Adds support for PCIe Gen3 MAC controller for MediaTek SoCs.
 	  This PCIe controller is compatible with Gen3, Gen2 and Gen1 speed,
drivers/pci/controller/pcie-mediatek-gen3.c +25 -42

112 lines not on this page · raw message

Signature · 1 line
2.39.5
author
#17 lore raw

PATCH 8/16 PCI: mediatek: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Ryder Lee <[email protected]>
Cc: Jianjun Wang <[email protected]>
Cc: Matthias Brugger <[email protected]>
Cc: AngeloGioacchino Del Regno <[email protected]>
Cc: [email protected]
Cc: [email protected]

---
 drivers/pci/controller/Kconfig         |  1 +
 drivers/pci/controller/pcie-mediatek.c | 46 ++++++++++++--------------
 2 files changed, 23 insertions(+), 24 deletions(-)
drivers/pci/controller/Kconfig +1 -0
index ec32c343a751d..65289a171333c 100644
@@ -195,6 +195,7 @@ config PCIE_MEDIATEK
 	depends on ARCH_AIROHA || ARCH_MEDIATEK || COMPILE_TEST
 	depends on OF
 	depends on PCI_MSI
+	select IRQ_MSI_LIB
 	help
 	  Say Y here if you want to enable PCIe controller support on
 	  MediaTek SoCs.
drivers/pci/controller/pcie-mediatek.c +22 -24

88 lines not on this page · raw message

Signature · 1 line
2.39.5
author
#19 lore raw

PATCH 9/16 PCI: rcar-host: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Marek Vasut <[email protected]>
Cc: Yoshihiro Shimoda <[email protected]>
Cc: Geert Uytterhoeven <[email protected]>
Cc: Magnus Damm <[email protected]>
Cc: [email protected]

---
 drivers/pci/controller/Kconfig          |  1 +
 drivers/pci/controller/pcie-rcar-host.c | 69 +++++++++----------------
 2 files changed, 26 insertions(+), 44 deletions(-)
drivers/pci/controller/Kconfig +1 -0
index 65289a171333c..8b9492e9ae693 100644
@@ -243,6 +243,7 @@ config PCIE_RCAR_HOST
 	bool "Renesas R-Car PCIe controller (host mode)"
 	depends on ARCH_RENESAS || COMPILE_TEST
 	depends on PCI_MSI
+	select IRQ_MSI_LIB
 	help
 	  Say Y here if you want PCIe controller support on R-Car SoCs in host
 	  mode.
drivers/pci/controller/pcie-rcar-host.c +25 -44

101 lines not on this page · raw message

Signature · 1 line
2.39.5
author
#21 lore raw

PATCH 10/16 PCI: xilinx-xdma: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Michal Simek <[email protected]>
Cc: [email protected]

---
 drivers/pci/controller/Kconfig              |  1 +
 drivers/pci/controller/pcie-xilinx-dma-pl.c | 48 +++++++++------------
 2 files changed, 22 insertions(+), 27 deletions(-)
drivers/pci/controller/Kconfig +1 -0
index 8b9492e9ae693..c9b6180239732 100644
@@ -331,6 +331,7 @@ config PCIE_XILINX_DMA_PL
 	depends on ARCH_ZYNQMP || COMPILE_TEST
 	depends on PCI_MSI
 	select PCI_HOST_COMMON
+	select IRQ_MSI_LIB
 	help
 	  Say 'Y' here if you want kernel support for the Xilinx PL DMA
 	  PCIe host bridge. The controller is a Soft IP which can act as
drivers/pci/controller/pcie-xilinx-dma-pl.c +21 -27

83 lines not on this page · raw message

Signature · 1 line
2.39.5
author
#23 lore raw

PATCH 11/16 PCI: xilinx-nwl: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Michal Simek <[email protected]>
Cc: [email protected]

---
 drivers/pci/controller/Kconfig           |  1 +
 drivers/pci/controller/pcie-xilinx-nwl.c | 45 ++++++++++++------------
 2 files changed, 24 insertions(+), 22 deletions(-)
drivers/pci/controller/Kconfig +1 -0
index c9b6180239732..118ff622fa69e 100644
@@ -342,6 +342,7 @@ config PCIE_XILINX_NWL
 	bool "Xilinx NWL PCIe controller"
 	depends on ARCH_ZYNQMP || COMPILE_TEST
 	depends on PCI_MSI
+	select IRQ_MSI_LIB
 	help
 	 Say 'Y' here if you want kernel support for Xilinx
 	 NWL PCIe controller. The controller can act as Root Port
drivers/pci/controller/pcie-xilinx-nwl.c +23 -22

75 lines not on this page · raw message

Signature · 1 line
2.39.5
author
#25 lore raw

PATCH 12/16 PCI: xilinx: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Michal Simek <[email protected]>
Cc: [email protected]

---
 drivers/pci/controller/Kconfig       |  1 +
 drivers/pci/controller/pcie-xilinx.c | 55 ++++++++++++++++------------
 2 files changed, 32 insertions(+), 24 deletions(-)
drivers/pci/controller/Kconfig +1 -0
index 118ff622fa69e..8f56ffd029ba2 100644
@@ -322,6 +322,7 @@ config PCIE_XILINX
 	bool "Xilinx AXI PCIe controller"
 	depends on OF
 	depends on PCI_MSI
+	select IRQ_MSI_LIB
 	help
 	  Say 'Y' here if you want kernel to support the Xilinx AXI PCIe
 	  Host Bridge driver.
drivers/pci/controller/pcie-xilinx.c +31 -24

87 lines not on this page · raw message

Signature · 1 line
2.39.5
author
#27 lore raw

PATCH 13/16 PCI: plda: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Daire McNamara <[email protected]>

---
 drivers/pci/controller/plda/Kconfig          |  1 +
 drivers/pci/controller/plda/pcie-plda-host.c | 44 ++++++++++----------
 drivers/pci/controller/plda/pcie-plda.h      |  1 -
 3 files changed, 23 insertions(+), 23 deletions(-)
drivers/pci/controller/plda/Kconfig +1 -0
index c0e14146d7e45..62120101139cb 100644
@@ -5,6 +5,7 @@ menu "PLDA-based PCIe controllers"
 
 config PCIE_PLDA_HOST
 	bool
+	select IRQ_MSI_LIB
 
 config PCIE_MICROCHIP_HOST
 	tristate "Microchip AXI PCIe controller"
drivers/pci/controller/plda/pcie-plda-host.c +22 -22

73 lines not on this page · raw message

drivers/pci/controller/plda/pcie-plda.h +0 -1
index 61ece26065ea0..6b8665df7bf0f 100644
@@ -164,7 +164,6 @@ struct plda_pcie_host_ops {
 
 struct plda_msi {
 	struct mutex lock;		/* Protect used bitmap */
-	struct irq_domain *msi_domain;
 	struct irq_domain *dev_domain;
 	u32 num_vectors;
 	u64 vector_phy;
Signature · 1 line
2.39.5
author
#29 lore raw

PATCH 14/16 PCI: hv: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: K. Y. Srinivasan <[email protected]>
Cc: Haiyang Zhang <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: Dexuan Cui <[email protected]>
Cc: [email protected]

---
 drivers/pci/Kconfig                 |  1 +
 drivers/pci/controller/pci-hyperv.c | 98 +++++++++++++++++++++++------
 2 files changed, 80 insertions(+), 19 deletions(-)
drivers/pci/Kconfig +1 -0
index 9c0e4aaf4e8cb..9a249c65aedcd 100644
@@ -223,6 +223,7 @@ config PCI_HYPERV
 	tristate "Hyper-V PCI Frontend"
 	depends on ((X86 && X86_64) || ARM64) && HYPERV && PCI_MSI && SYSFS
 	select PCI_HYPERV_INTERFACE
+	select IRQ_MSI_LIB
 	help
 	  The PCI device frontend driver allows the kernel to import arbitrary
 	  PCI devices from a PCI backend to support PCI driver domains.
drivers/pci/controller/pci-hyperv.c +79 -19

152 lines not on this page · raw message

Signature · 1 line
2.39.5
#31 lore raw

RE: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

From: Nam Cao <[email protected]> Sent: Thursday, June 26, 2025 7:48 AM

Quoted 3 lines

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

From a build standpoint, this patch does not apply cleanly to linux-next20250630. See also an issue below where a needed irq function isn't exported.

At runtime, I've done basic smoke testing on an x86 VM in the Azure cloud that has a Mellanox NIC VF and two NVMe devices as PCI devices. So far everything looks good. But I'm still doing additional testing, and I want to also test on an ARM64 VM. Please give me another day or two to be completely satisfied.

Michael Kelley

Quoted 148 lines

---
Cc: K. Y. Srinivasan <[email protected]>
Cc: Haiyang Zhang <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: Dexuan Cui <[email protected]>
Cc: [email protected]

---
 drivers/pci/Kconfig                 |  1 +
 drivers/pci/controller/pci-hyperv.c | 98 +++++++++++++++++++++++------
 2 files changed, 80 insertions(+), 19 deletions(-)
drivers/pci/Kconfig +1 -0
index 9c0e4aaf4e8cb..9a249c65aedcd 100644
@@ -223,6 +223,7 @@ config PCI_HYPERV
 	tristate "Hyper-V PCI Frontend"
 	depends on ((X86 && X86_64) || ARM64) && HYPERV && PCI_MSI && SYSFS
 	select PCI_HYPERV_INTERFACE
+	select IRQ_MSI_LIB
 	help
 	  The PCI device frontend driver allows the kernel to import arbitrary
 	  PCI devices from a PCI backend to support PCI driver domains.
drivers/pci/controller/pci-hyperv.c +40 -4
index ef5d655a0052c..3a24fadddb83b 100644
@@ -44,6 +44,7 @@
 #include <linux/delay.h>
 #include <linux/semaphore.h>
 #include <linux/irq.h>
+#include <linux/irqchip/irq-msi-lib.h>
 #include <linux/msi.h>
 #include <linux/hyperv.h>
 #include <linux/refcount.h>
@@ -508,7 +509,6 @@ struct hv_pcibus_device {
 	struct list_head children;
 	struct list_head dr_list;
 
-	struct msi_domain_info msi_info;
 	struct irq_domain *irq_domain;
 
 	struct workqueue_struct *wq;
@@ -1687,7 +1687,7 @@ static void hv_msi_free(struct irq_domain *domain, struct msi_domain_info *info,
 	struct msi_desc *msi = irq_data_get_msi_desc(irq_data);
 
 	pdev = msi_desc_to_pci_dev(msi);
-	hbus = info->data;
+	hbus = domain->host_data;
 	int_desc = irq_data_get_irq_chip_data(irq_data);
 	if (!int_desc)
 		return;
@@ -1705,7 +1705,6 @@ static void hv_msi_free(struct irq_domain *domain, struct msi_domain_info *info,
 
 static void hv_irq_mask(struct irq_data *data)
 {
-	pci_msi_mask_irq(data);
 	if (data->parent_data->chip->irq_mask)
 		irq_chip_mask_parent(data);
 }
@@ -1716,7 +1715,6 @@ static void hv_irq_unmask(struct irq_data *data)
 
 	if (data->parent_data->chip->irq_unmask)
 		irq_chip_unmask_parent(data);
-	pci_msi_unmask_irq(data);
 }
 
 struct compose_comp_ctxt {
@@ -2101,6 +2099,44 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
 	msg->data = 0;
 }
 
+static bool hv_pcie_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+				      struct irq_domain *real_parent, struct msi_domain_info *info)
+{
+	struct irq_chip *chip = info->chip;
+
+	if (!msi_lib_init_dev_msi_info(dev, domain, real_parent, info))
+		return false;
+
+	info->ops->msi_prepare = hv_msi_prepare;
+
+	chip->irq_set_affinity = irq_chip_set_affinity_parent;
+
+	if (IS_ENABLED(CONFIG_X86))
+		chip->flags |= IRQCHIP_MOVE_DEFERRED;
+
+	return true;
+}
+
+#define HV_PCIE_MSI_FLAGS_REQUIRED (MSI_FLAG_USE_DEF_DOM_OPS
 	| \
+				    MSI_FLAG_USE_DEF_CHIP_OPS		| \
+				    MSI_FLAG_PCI_MSI_MASK_PARENT)
+#define HV_PCIE_MSI_FLAGS_SUPPORTED (MSI_FLAG_MULTI_PCI_MSI
 	| \
+				     MSI_FLAG_PCI_MSIX			| \
+				     MSI_GENERIC_FLAGS_MASK)
+
+static const struct msi_parent_ops hv_pcie_msi_parent_ops = {
+	.required_flags		= HV_PCIE_MSI_FLAGS_REQUIRED,
+	.supported_flags	= HV_PCIE_MSI_FLAGS_SUPPORTED,
+	.bus_select_token	= DOMAIN_BUS_PCI_MSI,
+#ifdef CONFIG_X86
+	.chip_flags		= MSI_CHIP_FLAG_SET_ACK,
+#elif defined(CONFIG_ARM64)
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI,
+#endif
+	.prefix			= "HV-",
+	.init_dev_msi_info	= hv_pcie_init_dev_msi_info,
+};
+
 /* HW Interrupt Chip Descriptor */

static struct irq_chip hv_msi_irq_chip = {

drivers/pci/controller/pci-hyperv.c +16 -4
 	.name			= "Hyper-V PCIe MSI",
@@ -2108,7 +2144,6 @@ static struct irq_chip hv_msi_irq_chip = {
 	.irq_set_affinity	= irq_chip_set_affinity_parent,
 #ifdef CONFIG_X86
 	.irq_ack		= irq_chip_ack_parent,
-	.flags			= IRQCHIP_MOVE_DEFERRED,
 #elif defined(CONFIG_ARM64)
 	.irq_eoi		= irq_chip_eoi_parent,
 #endif
@@ -2116,9 +2151,37 @@ static struct irq_chip hv_msi_irq_chip = {
 	.irq_unmask		= hv_irq_unmask,
 };
 
-static struct msi_domain_ops hv_msi_ops = {
-	.msi_prepare	= hv_msi_prepare,
-	.msi_free	= hv_msi_free,
+static int hv_pcie_domain_alloc(struct irq_domain *d, unsigned int virq, unsigned int nr_irqs,
+			       void *arg)
+{
+	/* TODO: move the content of hv_compose_msi_msg() in here */
+	int ret;
+
+	ret = irq_domain_alloc_irqs_parent(d, virq, nr_irqs, arg);
+	if (ret < 0)
+		return ret;
+
+	for (int i = 0; i < nr_irqs; i++) {
+		irq_domain_set_info(d, virq + i, 0, &hv_msi_irq_chip, NULL, FLOW_HANDLER, NULL,
+				    FLOW_NAME);
+	}
+
+	return 0;

drivers/pci/controller/pci-hyperv.c static struct irq_chip hv_msi_irq_chip = { · line 2179

Quoted 10 lines
drivers/pci/controller/pci-hyperv.c +10 -0
+}
+
+static void hv_pcie_domain_free(struct irq_domain *d, unsigned int virq, unsigned int nr_irqs)
+{
+	struct msi_domain_info *info = d->host_data;
+
+	for (int i = 0; i < nr_irqs; i++)
+		hv_msi_free(d, info, virq + i);
+
+	irq_domain_free_irqs_top(d, virq, nr_irqs);

This code can be built as a module, so irq_domain_free_irqs_top() needs to be exported, which it currently is not.

Quoted 37 lines
drivers/pci/controller/pci-hyperv.c +13 -12
+}
+
+static const struct irq_domain_ops hv_pcie_domain_ops = {
+	.alloc	= hv_pcie_domain_alloc,
+	.free	= hv_pcie_domain_free,
 };
 
 /**
@@ -2136,17 +2199,14 @@ static struct msi_domain_ops hv_msi_ops = {
  */
 static int hv_pcie_init_irq_domain(struct hv_pcibus_device *hbus)
 {
-	hbus->msi_info.chip = &hv_msi_irq_chip;
-	hbus->msi_info.ops = &hv_msi_ops;
-	hbus->msi_info.flags = (MSI_FLAG_USE_DEF_DOM_OPS |
-		MSI_FLAG_USE_DEF_CHIP_OPS | MSI_FLAG_MULTI_PCI_MSI |
-		MSI_FLAG_PCI_MSIX);
-	hbus->msi_info.handler = FLOW_HANDLER;
-	hbus->msi_info.handler_name = FLOW_NAME;
-	hbus->msi_info.data = hbus;
-	hbus->irq_domain = pci_msi_create_irq_domain(hbus->fwnode,
-						     &hbus->msi_info,
-						     hv_pci_get_root_domain());
+	struct irq_domain_info info = {
+		.fwnode		= hbus->fwnode,
+		.ops		= &hv_pcie_domain_ops,
+		.host_data	= hbus,
+		.parent		= hv_pci_get_root_domain(),
+	};
+
+	hbus->irq_domain = msi_create_parent_irq_domain(&info, &hv_pcie_msi_parent_ops);
 	if (!hbus->irq_domain) {
 		dev_err(&hbus->hdev->device,
 			"Failed to build an MSI IRQ domain\n");
--

2.39.5

#32 lore raw

RE: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

Quoted 8 lines · Michael Kelley

From: Nam Cao <[email protected]> Sent: Thursday, June 26, 2025 7:48 AM

Quoted 3 lines

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

From a build standpoint, this patch does not apply cleanly to linux-next20250630. See also an issue below where a needed irq function isn't exported.

Does it conflict against the PCI tree?

Quoted 5 lines

At runtime, I've done basic smoke testing on an x86 VM in the Azure cloud that has a Mellanox NIC VF and two NVMe devices as PCI devices. So far everything looks good. But I'm still doing additional testing, and I want to also test on an ARM64 VM. Please give me another day or two to be completely satisfied.

Sure.

Quoted 11 lines

drivers/pci/controller/pci-hyperv.c static struct irq_chip hv_msi_irq_chip = { · line 2179

Quoted 8 lines
drivers/pci/controller/pci-hyperv.c +8 -0
+static void hv_pcie_domain_free(struct irq_domain *d, unsigned int virq, unsigned int nr_irqs)
+{
+	struct msi_domain_info *info = d->host_data;
+
+	for (int i = 0; i < nr_irqs; i++)
+		hv_msi_free(d, info, virq + i);
+
+	irq_domain_free_irqs_top(d, virq, nr_irqs);

This code can be built as a module, so irq_domain_free_irqs_top() needs to be exported, which it currently is not.

Nam, can you please create a seperate patch, which exports this and take care of the conflict?

Thanks,

tglx

#33 lore raw

RE: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

From: Thomas Gleixner <[email protected]> Sent: Thursday, July 3, 2025 1:00 PM

Quoted 12 lines · Michael Kelley
Quoted 8 lines

From: Nam Cao <[email protected]> Sent: Thursday, June 26, 2025 7:48 AM

Quoted 3 lines

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

From a build standpoint, this patch does not apply cleanly to linux-next20250630. See also an issue below where a needed irq function isn't exported.

Does it conflict against the PCI tree?

There's no conflict in the "next" or "for-linus" tags in
https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git/.

The conflict is with Patch 2 of this series:

https://lore.kernel.org/linux-hyperv/1749650984-9193-1-git-send-email-shradhagupta@linux.microsoft.com/

which is in netdev/net-next.

Michael

Quoted 26 lines
Quoted 6 lines

At runtime, I've done basic smoke testing on an x86 VM in the Azure cloud that has a Mellanox NIC VF and two NVMe devices as PCI devices. So far everything looks good. But I'm still doing additional testing, and I want to also test on an ARM64 VM. Please give me another day or two to be completely satisfied.

Sure.

Quoted 11 lines

drivers/pci/controller/pci-hyperv.c static struct irq_chip hv_msi_irq_chip = { · line 2179

Quoted 8 lines
drivers/pci/controller/pci-hyperv.c +8 -0
+static void hv_pcie_domain_free(struct irq_domain *d, unsigned int virq, unsigned int nr_irqs)
+{
+	struct msi_domain_info *info = d->host_data;
+
+	for (int i = 0; i < nr_irqs; i++)
+		hv_msi_free(d, info, virq + i);
+
+	irq_domain_free_irqs_top(d, virq, nr_irqs);

This code can be built as a module, so irq_domain_free_irqs_top() needs to be exported, which it currently is not.

Nam, can you please create a seperate patch, which exports this and take care of the conflict?

Thanks,

tglx

author
#34 lore raw

Re: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

Quoted 14 lines · Michael Kelley

From: Thomas Gleixner <[email protected]> Sent: Thursday, July 3, 2025 1:00 PM

Quoted 12 lines · Michael Kelley
Quoted 8 lines

From: Nam Cao <[email protected]> Sent: Thursday, June 26, 2025 7:48 AM

Quoted 3 lines

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

From a build standpoint, this patch does not apply cleanly to linux-next20250630. See also an issue below where a needed irq function isn't exported.

Does it conflict against the PCI tree?

Quoted 8 lines

There's no conflict in the "next" or "for-linus" tags in
https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git/.

The conflict is with Patch 2 of this series:

https://lore.kernel.org/linux-hyperv/1749650984-9193-1-git-send-email-shradhagupta@linux.microsoft.com/

which is in netdev/net-next.

I need some guidance here. If I make it apply cleanly to linux-next, it won't apply to pci tree.

I saw this type of conflict being resolved during merging to Linus's tree. Shouldn't we do the same for this case?

Quoted 8 lines

Michael

Quoted 6 lines
Quoted 6 lines

At runtime, I've done basic smoke testing on an x86 VM in the Azure cloud that has a Mellanox NIC VF and two NVMe devices as PCI devices. So far everything looks good. But I'm still doing additional testing, and I want to also test on an ARM64 VM. Please give me another day or two to be completely satisfied.

Good to hear, thanks!

Quoted 13 lines
Quoted 13 lines

Sure.

Quoted 11 lines

drivers/pci/controller/pci-hyperv.c static struct irq_chip hv_msi_irq_chip = { · line 2179

Quoted 8 lines
drivers/pci/controller/pci-hyperv.c +8 -0
+static void hv_pcie_domain_free(struct irq_domain *d, unsigned int virq, unsigned int nr_irqs)
+{
+	struct msi_domain_info *info = d->host_data;
+
+	for (int i = 0; i < nr_irqs; i++)
+		hv_msi_free(d, info, virq + i);
+
+	irq_domain_free_irqs_top(d, virq, nr_irqs);

This code can be built as a module, so irq_domain_free_irqs_top() needs to be exported, which it currently is not.

Quoted 2 lines
Quoted 2 lines

Nam, can you please create a seperate patch, which exports this and take care of the conflict?

Will do.

Best regards,
Nam

#35 lore raw

Re: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

Quoted 8 lines · Nam Cao
Quoted 1 line · Michael Kelley

which is in netdev/net-next.

I need some guidance here. If I make it apply cleanly to linux-next, it won't apply to pci tree.

I saw this type of conflict being resolved during merging to Linus's tree. Shouldn't we do the same for this case?

There are many ways to skin a cat. See my other reply.

#36 lore raw

RE: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

In reply to Michael Kelley
Quoted 11 lines · Michael Kelley

From: Thomas Gleixner <[email protected]> Sent: Thursday, July 3, 2025 1:00 PM

Quoted 1 line

Does it conflict against the PCI tree?

There's no conflict in the "next" or "for-linus" tags in
https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git/.

The conflict is with Patch 2 of this series:

https://lore.kernel.org/linux-hyperv/1749650984-9193-1-git-send-email-shradhagupta@linux.microsoft.com/

which is in netdev/net-next.

That's a trivial one. There are two ways to handle it:

1) Take it through the PCI tree and provide a conflict resolution for linux-next and later for Linus as reference.

2) Route it through the net-next tree with an updated patch.

As there are no further dependencies (aside of the missing export which is needed anyway) it's obvious to pick #2 as it creates the least headaches. Assumed that the PCI folks have no objections.

Michael, as you have resolved the conflict already, can you please either take care of it yourself or provide the resolution here as reference for Nam?

Thanks,

tglx

#37 lore raw

RE: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

From: Thomas Gleixner <[email protected]> Sent: Thursday, July 3, 2025 2:22 PM

Quoted 21 lines · Michael Kelley
Quoted 11 lines

From: Thomas Gleixner <[email protected]> Sent: Thursday, July 3, 2025 1:00 PM

Quoted 1 line

Does it conflict against the PCI tree?

There's no conflict in the "next" or "for-linus" tags in
https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git/

The conflict is with Patch 2 of this series:

https://lore.kernel.org/linux-hyperv/1749650984-9193-1-git-send-email-shradhagupta@linux.microsoft.com/

which is in netdev/net-next.

That's a trivial one. There are two ways to handle it:

1) Take it through the PCI tree and provide a conflict resolution for linux-next and later for Linus as reference.

2) Route it through the net-next tree with an updated patch.

Quoted 7 lines

As there are no further dependencies (aside of the missing export which is needed anyway) it's obvious to pick #2 as it creates the least headaches. Assumed that the PCI folks have no objections.

Michael, as you have resolved the conflict already, can you please either take care of it yourself or provide the resolution here as reference for Nam?

I haven't resolved the conflict. As a shortcut for testing I just removed the conflicting patch since it is for a Microsoft custom NIC ("MANA") that's not in the configuration I'm testing with. I'll have to look more closely to figure out the resolution.

Separately, this patch (the switch to misc_create_parent_irq_domain) isn't working for Linux VMs on Hyper-V on ARM64. The initial symptom is that interrupts from the NVMe controller aren't getting handled and everything hangs. Here's the dmesg output:

[   84.463419] hv_vmbus: registering driver hv_pci
[   84.463875] hv_pci abee639e-0b9d-49b7-9a07-c54ba8cd5734: PCI VMBus probing: Using version 0x10004
[   84.464518] hv_pci abee639e-0b9d-49b7-9a07-c54ba8cd5734: PCI host bridge to bus 0b9d:00
[   84.464529] pci_bus 0b9d:00: root bus resource [mem 0xfc0000000-0xfc00fffff window]
[   84.464531] pci_bus 0b9d:00: No busn resource found for root bus, will use [bus 00-ff]
[   84.465211] pci 0b9d:00:00.0: [1414:b111] type 00 class 0x010802 PCIe Endpoint
[   84.466657] pci 0b9d:00:00.0: BAR 0 [mem 0xfc0000000-0xfc00fffff 64bit]
[   84.481923] pci_bus 0b9d:00: busn_res: [bus 00-ff] end is updated to 00
[   84.481936] pci 0b9d:00:00.0: BAR 0 [mem 0xfc0000000-0xfc00fffff 64bit]: assigned
[   84.482413] nvme nvme0: pci function 0b9d:00:00.0
[   84.482513] nvme 0b9d:00:00.0: enabling device (0000 -> 0002)
[   84.556871] irq 17, desc: 00000000e8529819, depth: 0, count: 0, unhandled: 0
[   84.556883] ->handle_irq():  0000000062fa78bc, handle_bad_irq+0x0/0x270
[   84.556892] ->irq_data.chip(): 00000000ba07832f, 0xffff00011469dc30
[   84.556895] ->action(): 0000000069f160b3
[   84.556896] ->action->handler(): 00000000e15d8191, nvme_irq+0x0/0x3e8
[  172.307920] watchdog: BUG: soft lockup - CPU#6 stuck for 26s! [kworker/6:1H:195]

Everything looks normal up to the "irq 17" line. Meanwhile, the device probe path is waiting with this stack trace, which I suspect is the first Interaction with the NVMe controller:

[<0>] blk_execute_rq+0x1ec/0x348
[<0>] nvme_execute_rq+0x20/0x68
[<0>] __nvme_submit_sync_cmd+0xc8/0x170
[<0>] nvme_identify_ctrl.isra.0+0x90/0xf0
[<0>] nvme_init_identify+0x44/0xee0
[<0>] nvme_init_ctrl_finish+0x84/0x370
[<0>] nvme_probe+0x668/0x7d8
[<0>] local_pci_probe+0x48/0xd0
[<0>] pci_device_probe+0xd0/0x248
[<0>] really_probe+0xd4/0x388
[<0>] __driver_probe_device+0x90/0x1a8
[<0>] driver_probe_device+0x48/0x150
[<0>] __device_attach_driver+0xe0/0x1b8
[<0>] bus_for_each_drv+0x8c/0xf8
[<0>] __device_attach+0x104/0x1e8
[<0>] device_attach+0x1c/0x30
[<0>] pci_bus_add_device+0xe0/0x188
[<0>] pci_bus_add_devices+0x40/0x98
[<0>] hv_pci_probe+0x4b0/0x690 [pci_hyperv]
[<0>] vmbus_probe+0x4c/0xb0 [hv_vmbus]
[<0>] really_probe+0xd4/0x388
[<0>] __driver_probe_device+0x90/0x1a8
[<0>] driver_probe_device+0x48/0x150
[<0>] __driver_attach+0xe8/0x1c8
[<0>] bus_for_each_dev+0x80/0xf0
[<0>] driver_attach+0x2c/0x40
[<0>] bus_add_driver+0x118/0x260
[<0>] driver_register+0x68/0x138
[<0>] __vmbus_driver_register+0x70/0x98 [hv_vmbus]
[<0>] init_hv_pci_drv+0x1b8/0xfff8 [pci_hyperv]
[<0>] do_one_initcall+0x4c/0x2c8
[<0>] do_init_module+0x60/0x280
[<0>] load_module+0x2318/0x2448
[<0>] init_module_from_file+0x94/0xe0
[<0>] __arm64_sys_finit_module+0x228/0x3e8
[<0>] invoke_syscall+0x6c/0xf8
[<0>] el0_svc_common.constprop.0+0xc8/0xf0
[<0>] do_el0_svc+0x24/0x38
[<0>] el0_svc+0x40/0x1a0
[<0>] el0t_64_sync_handler+0xd0/0xe8
[<0>] el0t_64_sync+0x1b0/0x1b8

I'll try to figure out what's going wrong.

Michael

author
#38 lore raw

Re: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

Quoted 19 lines · Michael Kelley

I haven't resolved the conflict. As a shortcut for testing I just removed the conflicting patch since it is for a Microsoft custom NIC ("MANA") that's not in the configuration I'm testing with. I'll have to look more closely to figure out the resolution.

Separately, this patch (the switch to misc_create_parent_irq_domain) isn't working for Linux VMs on Hyper-V on ARM64. The initial symptom is that interrupts from the NVMe controller aren't getting handled and everything hangs. Here's the dmesg output:

[   84.463419] hv_vmbus: registering driver hv_pci
[   84.463875] hv_pci abee639e-0b9d-49b7-9a07-c54ba8cd5734: PCI VMBus probing: Using version 0x10004
[   84.464518] hv_pci abee639e-0b9d-49b7-9a07-c54ba8cd5734: PCI host bridge to bus 0b9d:00
[   84.464529] pci_bus 0b9d:00: root bus resource [mem 0xfc0000000-0xfc00fffff window]
[   84.464531] pci_bus 0b9d:00: No busn resource found for root bus, will use [bus 00-ff]
[   84.465211] pci 0b9d:00:00.0: [1414:b111] type 00 class 0x010802 PCIe Endpoint
[   84.466657] pci 0b9d:00:00.0: BAR 0 [mem 0xfc0000000-0xfc00fffff 64bit]
[   84.481923] pci_bus 0b9d:00: busn_res: [bus 00-ff] end is updated to 00
[   84.481936] pci 0b9d:00:00.0: BAR 0 [mem 0xfc0000000-0xfc00fffff 64bit]: assigned
Quoted 8 lines
[   84.482413] nvme nvme0: pci function 0b9d:00:00.0
[   84.482513] nvme 0b9d:00:00.0: enabling device (0000 -> 0002)
[   84.556871] irq 17, desc: 00000000e8529819, depth: 0, count: 0, unhandled: 0
[   84.556883] ->handle_irq():  0000000062fa78bc, handle_bad_irq+0x0/0x270
[   84.556892] ->irq_data.chip(): 00000000ba07832f, 0xffff00011469dc30
[   84.556895] ->action(): 0000000069f160b3
[   84.556896] ->action->handler(): 00000000e15d8191, nvme_irq+0x0/0x3e8
[  172.307920] watchdog: BUG: soft lockup - CPU#6 stuck for 26s! [kworker/6:1H:195]

Thanks for the report.

On arm64, this driver relies on the parent irq domain to set handler. So the driver must not overwrite it to NULL.

This should cures it:

drivers/pci/controller/pci-hyperv.c +3 -6
index 3a24fadddb83..f4a435b0456c 100644
@@ -577,8 +577,6 @@ static void hv_pci_onchannelcallback(void *context);
 
 #ifdef CONFIG_X86
 #define DELIVERY_MODE	APIC_DELIVERY_MODE_FIXED
-#define FLOW_HANDLER	handle_edge_irq
-#define FLOW_NAME	"edge"
 
 static int hv_pci_irqchip_init(void)
 {
@@ -723,8 +721,6 @@ static void hv_arch_irq_unmask(struct irq_data *data)
 #define HV_PCI_MSI_SPI_START	64
 #define HV_PCI_MSI_SPI_NR	(1020 - HV_PCI_MSI_SPI_START)
 #define DELIVERY_MODE		0
-#define FLOW_HANDLER		NULL
-#define FLOW_NAME		NULL
 #define hv_msi_prepare		NULL
 
 struct hv_pci_chip_data {
@@ -2162,8 +2158,9 @@ static int hv_pcie_domain_alloc(struct irq_domain *d, unsigned int virq, unsigne
 		return ret;
 
 	for (int i = 0; i < nr_irqs; i++) {
-		irq_domain_set_info(d, virq + i, 0, &hv_msi_irq_chip, NULL, FLOW_HANDLER, NULL,
-				    FLOW_NAME);
+		irq_domain_set_hwirq_and_chip(d, virq + i, 0, &hv_msi_irq_chip, NULL);
+		if (IS_ENABLED(CONFIG_X86))
+			__irq_set_handler(virq + i, handle_edge_irq, 0, "edge");
 	}
 
 	return 0;
#39 lore raw

RE: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

From: Nam Cao <[email protected]> Sent: Thursday, July 3, 2025 9:33 PM

Quoted 62 lines · Michael Kelley
Quoted 19 lines

I haven't resolved the conflict. As a shortcut for testing I just removed the conflicting patch since it is for a Microsoft custom NIC ("MANA") that's not in the configuration I'm testing with. I'll have to look more closely to figure out the resolution.

Separately, this patch (the switch to misc_create_parent_irq_domain) isn't working for Linux VMs on Hyper-V on ARM64. The initial symptom is that interrupts from the NVMe controller aren't getting handled and everything hangs. Here's the dmesg output:

[   84.463419] hv_vmbus: registering driver hv_pci
[   84.463875] hv_pci abee639e-0b9d-49b7-9a07-c54ba8cd5734: PCI VMBus probing: Using version 0x10004
[   84.464518] hv_pci abee639e-0b9d-49b7-9a07-c54ba8cd5734: PCI host bridge to bus 0b9d:00
[   84.464529] pci_bus 0b9d:00: root bus resource [mem 0xfc0000000-0xfc00fffff window]
[   84.464531] pci_bus 0b9d:00: No busn resource found for root bus, will use [bus 00-ff]
[   84.465211] pci 0b9d:00:00.0: [1414:b111] type 00 class 0x010802 PCIe Endpoint
[   84.466657] pci 0b9d:00:00.0: BAR 0 [mem 0xfc0000000-0xfc00fffff 64bit]
[   84.481923] pci_bus 0b9d:00: busn_res: [bus 00-ff] end is updated to 00
[   84.481936] pci 0b9d:00:00.0: BAR 0 [mem 0xfc0000000-0xfc00fffff 64bit]: assigned
Quoted 8 lines
[   84.482413] nvme nvme0: pci function 0b9d:00:00.0
[   84.482513] nvme 0b9d:00:00.0: enabling device (0000 -> 0002)
[   84.556871] irq 17, desc: 00000000e8529819, depth: 0, count: 0, unhandled: 0
[   84.556883] ->handle_irq():  0000000062fa78bc, handle_bad_irq+0x0/0x270
[   84.556892] ->irq_data.chip(): 00000000ba07832f, 0xffff00011469dc30
[   84.556895] ->action(): 0000000069f160b3
[   84.556896] ->action->handler(): 00000000e15d8191, nvme_irq+0x0/0x3e8
[  172.307920] watchdog: BUG: soft lockup - CPU#6 stuck for 26s! [kworker/6:1H:195]

Thanks for the report.

On arm64, this driver relies on the parent irq domain to set handler. So the driver must not overwrite it to NULL.

This should cures it:

drivers/pci/controller/pci-hyperv.c +0 -4
index 3a24fadddb83..f4a435b0456c 100644
@@ -577,8 +577,6 @@ static void hv_pci_onchannelcallback(void *context);
 
 #ifdef CONFIG_X86
 #define DELIVERY_MODE	APIC_DELIVERY_MODE_FIXED
-#define FLOW_HANDLER	handle_edge_irq
-#define FLOW_NAME	"edge"
 
 static int hv_pci_irqchip_init(void)
 {
@@ -723,8 +721,6 @@ static void hv_arch_irq_unmask(struct irq_data *data)
 #define HV_PCI_MSI_SPI_START	64
 #define HV_PCI_MSI_SPI_NR	(1020 - HV_PCI_MSI_SPI_START)
 #define DELIVERY_MODE		0
-#define FLOW_HANDLER		NULL
-#define FLOW_NAME		NULL
 #define hv_msi_prepare		NULL
 
 struct hv_pci_chip_data {
@@ -2162,8 +2158,9 @@ static int hv_pcie_domain_alloc(struct irq_domain *d,

unsigned int virq, unsigne

drivers/pci/controller/pci-hyperv.c +0 -0
 		return ret;

drivers/pci/controller/pci-hyperv.c static int hv_pcie_domain_alloc(struct irq_domain *d, unsigned int virq, unsigne · line 2166

Quoted 10 lines
drivers/pci/controller/pci-hyperv.c +3 -2
 
 	for (int i = 0; i < nr_irqs; i++) {
-		irq_domain_set_info(d, virq + i, 0, &hv_msi_irq_chip, NULL, FLOW_HANDLER, NULL,
-				    FLOW_NAME);
+		irq_domain_set_hwirq_and_chip(d, virq + i, 0, &hv_msi_irq_chip, NULL);
+		if (IS_ENABLED(CONFIG_X86))
+			__irq_set_handler(virq + i, handle_edge_irq, 0, "edge");
 	}
 
 	return 0;

Yes, that fixes the problem. Linux now boots with the PCI NIC VF and two NVMe controllers being visible and operational. Thanks for the fix! It would have taken me a while to figure it out.

I want to do some additional testing tomorrow, and look more closely at the code, but now I have something that works well enough to make further progress.

Michael

#40 lore raw

Re: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao
kernel test robot noticed the following build errors:

Hi Nam,

kernel test robot noticed the following build errors:

[auto build test ERROR on pci/next]
[cannot apply to pci/for-linus linus/master v6.16-rc4 next-20250704] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Nam-Cao/PCI-dwc-Switch-to-msi_create_parent_irq_domain/20250626-230029
base:   https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git next
patch link:    https://lore.kernel.org/r/024f0122314198fe0a42fef01af53e8953a687ec.1750858083.git.namcao%40linutronix.de
patch subject: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()
config: x86_64-rhel-9.4 (https://download.01.org/0day-ci/archive/20250705/[email protected]/config)
compiler: gcc-12 (Debian 12.2.0-14+deb12u1) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250705/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>, old ones prefixed by <<):

>> ERROR: modpost: "irq_domain_free_irqs_top" [drivers/pci/controller/pci-hyperv.ko] undefined!
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#41 lore raw

RE: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

From: Nam Cao <[email protected]> Sent: Thursday, June 26, 2025 7:48 AM

Quoted 3 lines

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

With the additional tweak to this patch that you supplied separately, everything in my testing on both x86 and arm64 seems to work OK. So that's all good.

On arm64, I did notice the following IRQ domain information from /sys/kernel/debug/irq/domains:

# cat HV-PCI-MSIX-1e03\:00\:00.0-12
name:   HV-PCI-MSIX-1e03:00:00.0-12
 size:   0
 mapped: 7
 flags:  0x00000213
            IRQ_DOMAIN_FLAG_HIERARCHY
            IRQ_DOMAIN_NAME_ALLOCATED
            IRQ_DOMAIN_FLAG_MSI
            IRQ_DOMAIN_FLAG_MSI_DEVICE
 parent: 5D202AA8-1E03-4F0F-A786-390A0D2749E9-3
    name:   5D202AA8-1E03-4F0F-A786-390A0D2749E9-3
     size:   0
     mapped: 7
     flags:  0x00000103
                IRQ_DOMAIN_FLAG_HIERARCHY
                IRQ_DOMAIN_NAME_ALLOCATED
                IRQ_DOMAIN_FLAG_MSI_PARENT
     parent: hv_vpci_arm64
        name:   hv_vpci_arm64
         size:   956
         mapped: 31
         flags:  0x00000003
                    IRQ_DOMAIN_FLAG_HIERARCHY
                    IRQ_DOMAIN_NAME_ALLOCATED
         parent: irqchip@0x00000000ffff0000-1
            name:   irqchip@0x00000000ffff0000-1
             size:   0
             mapped: 47
             flags:  0x00000003
                        IRQ_DOMAIN_FLAG_HIERARCHY
                        IRQ_DOMAIN_NAME_ALLOCATED

The 5D202AA8-1E03-4F0F-A786-390A0D2749E9-3 domain has IRQ_DOMAIN_FLAG_MSI_PARENT set. But the hv_vpci_arm64 and irqchip@... domains do not. Is that a problem? On x86, the output is this, with IRQ_DOMAIN_FLAG_MSI_PARENT set in the next level up VECTOR domain:

# cat HV-PCI-MSIX-6b71\:00\:02.0-12
name:   HV-PCI-MSIX-6b71:00:02.0-12
 size:   0
 mapped: 17
 flags:  0x00000213
            IRQ_DOMAIN_FLAG_HIERARCHY
            IRQ_DOMAIN_NAME_ALLOCATED
            IRQ_DOMAIN_FLAG_MSI
            IRQ_DOMAIN_FLAG_MSI_DEVICE
 parent: 8564CB14-6B71-477C-B189-F175118E6FF0-3
    name:   8564CB14-6B71-477C-B189-F175118E6FF0-3
     size:   0
     mapped: 17
     flags:  0x00000103
                IRQ_DOMAIN_FLAG_HIERARCHY
                IRQ_DOMAIN_NAME_ALLOCATED
                IRQ_DOMAIN_FLAG_MSI_PARENT
     parent: VECTOR
        name:   VECTOR
         size:   0
         mapped: 67
         flags:  0x00000103
                    IRQ_DOMAIN_FLAG_HIERARCHY
                    IRQ_DOMAIN_NAME_ALLOCATED
                    IRQ_DOMAIN_FLAG_MSI_PARENT

Finally, I've noted a couple of code review comments below. These comments may reflect my lack of fully understanding the MSI IRQ handling, in which case, please set me straight. Thanks,

Michael

Quoted 116 lines

---
Cc: K. Y. Srinivasan <[email protected]>
Cc: Haiyang Zhang <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: Dexuan Cui <[email protected]>
Cc: [email protected]

---
 drivers/pci/Kconfig                 |  1 +
 drivers/pci/controller/pci-hyperv.c | 98 +++++++++++++++++++++++------
 2 files changed, 80 insertions(+), 19 deletions(-)
drivers/pci/Kconfig +1 -0
index 9c0e4aaf4e8cb..9a249c65aedcd 100644
@@ -223,6 +223,7 @@ config PCI_HYPERV
 	tristate "Hyper-V PCI Frontend"
 	depends on ((X86 && X86_64) || ARM64) && HYPERV && PCI_MSI && SYSFS
 	select PCI_HYPERV_INTERFACE
+	select IRQ_MSI_LIB
 	help
 	  The PCI device frontend driver allows the kernel to import arbitrary
 	  PCI devices from a PCI backend to support PCI driver domains.
drivers/pci/controller/pci-hyperv.c +40 -4
index ef5d655a0052c..3a24fadddb83b 100644
@@ -44,6 +44,7 @@
 #include <linux/delay.h>
 #include <linux/semaphore.h>
 #include <linux/irq.h>
+#include <linux/irqchip/irq-msi-lib.h>
 #include <linux/msi.h>
 #include <linux/hyperv.h>
 #include <linux/refcount.h>
@@ -508,7 +509,6 @@ struct hv_pcibus_device {
 	struct list_head children;
 	struct list_head dr_list;
 
-	struct msi_domain_info msi_info;
 	struct irq_domain *irq_domain;
 
 	struct workqueue_struct *wq;
@@ -1687,7 +1687,7 @@ static void hv_msi_free(struct irq_domain *domain, struct msi_domain_info *info,
 	struct msi_desc *msi = irq_data_get_msi_desc(irq_data);
 
 	pdev = msi_desc_to_pci_dev(msi);
-	hbus = info->data;
+	hbus = domain->host_data;
 	int_desc = irq_data_get_irq_chip_data(irq_data);
 	if (!int_desc)
 		return;
@@ -1705,7 +1705,6 @@ static void hv_msi_free(struct irq_domain *domain, struct msi_domain_info *info,
 
 static void hv_irq_mask(struct irq_data *data)
 {
-	pci_msi_mask_irq(data);
 	if (data->parent_data->chip->irq_mask)
 		irq_chip_mask_parent(data);
 }
@@ -1716,7 +1715,6 @@ static void hv_irq_unmask(struct irq_data *data)
 
 	if (data->parent_data->chip->irq_unmask)
 		irq_chip_unmask_parent(data);
-	pci_msi_unmask_irq(data);
 }
 
 struct compose_comp_ctxt {
@@ -2101,6 +2099,44 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
 	msg->data = 0;
 }
 
+static bool hv_pcie_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+				      struct irq_domain *real_parent, struct msi_domain_info *info)
+{
+	struct irq_chip *chip = info->chip;
+
+	if (!msi_lib_init_dev_msi_info(dev, domain, real_parent, info))
+		return false;
+
+	info->ops->msi_prepare = hv_msi_prepare;
+
+	chip->irq_set_affinity = irq_chip_set_affinity_parent;
+
+	if (IS_ENABLED(CONFIG_X86))
+		chip->flags |= IRQCHIP_MOVE_DEFERRED;
+
+	return true;
+}
+
+#define HV_PCIE_MSI_FLAGS_REQUIRED (MSI_FLAG_USE_DEF_DOM_OPS	| \
+				    MSI_FLAG_USE_DEF_CHIP_OPS		| \
+				    MSI_FLAG_PCI_MSI_MASK_PARENT)
+#define HV_PCIE_MSI_FLAGS_SUPPORTED (MSI_FLAG_MULTI_PCI_MSI	| \
+				     MSI_FLAG_PCI_MSIX			| \
+				     MSI_GENERIC_FLAGS_MASK)
+
+static const struct msi_parent_ops hv_pcie_msi_parent_ops = {
+	.required_flags		= HV_PCIE_MSI_FLAGS_REQUIRED,
+	.supported_flags	= HV_PCIE_MSI_FLAGS_SUPPORTED,
+	.bus_select_token	= DOMAIN_BUS_PCI_MSI,
+#ifdef CONFIG_X86
+	.chip_flags		= MSI_CHIP_FLAG_SET_ACK,
+#elif defined(CONFIG_ARM64)
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI,
+#endif
+	.prefix			= "HV-",
+	.init_dev_msi_info	= hv_pcie_init_dev_msi_info,
+};
+
 /* HW Interrupt Chip Descriptor */
 static struct irq_chip hv_msi_irq_chip = {
 	.name			= "Hyper-V PCIe MSI",

drivers/pci/controller/pci-hyperv.c static struct irq_chip hv_msi_irq_chip = { · line 2149

Quoted 7 lines
drivers/pci/controller/pci-hyperv.c +0 -1
@@ -2108,7 +2144,6 @@ static struct irq_chip hv_msi_irq_chip = {
 	.irq_set_affinity	= irq_chip_set_affinity_parent,
 #ifdef CONFIG_X86
 	.irq_ack		= irq_chip_ack_parent,
-	.flags			= IRQCHIP_MOVE_DEFERRED,
 #elif defined(CONFIG_ARM64)
 	.irq_eoi		= irq_chip_eoi_parent,
 #endif

Would it work to drop the #ifdef's and always set both .irq_ack and .irq_eoi on x86 and on ARM64? Is which one gets called controlled by the child HV-PCI-MSIX- ... domain, based on the .chip_flags? I'm trying to reduce the #ifdef clutter. I tested without the #ifdefs on both x86 and arm64, and everything works, but I know that doesn't prove that it's OK.

If the #ifdefs can go away, then I'd like to see a tweak to the way .chip_flags is set. Rather than do an #ifdef inline for struct msi_parent_ops hv_pcie_msi_parent_ops, add a #define HV_MSI_CHIP_FLAGS in the existing #ifdef X86 and #ifdef ARM64 sections respectively near the top of this source file, and then use HV_MSI_CHIP_FLAGS in struct msi_parent_ops hv_pcie_msi_parent_ops. As much as is reasonable, I'd like to not clutter the code with #ifdef X86 #elseif ARM64, but instead group all the differences under the existing #ifdefs near the top. There are some places where this isn't practical, but this seems like a place that is practical.

drivers/pci/controller/pci-hyperv.c static struct irq_chip hv_msi_irq_chip = { · line 2157

Quoted 11 lines
drivers/pci/controller/pci-hyperv.c +4 -3
@@ -2116,9 +2151,37 @@ static struct irq_chip hv_msi_irq_chip = {
 	.irq_unmask		= hv_irq_unmask,
 };
 
-static struct msi_domain_ops hv_msi_ops = {
-	.msi_prepare	= hv_msi_prepare,
-	.msi_free	= hv_msi_free,
+static int hv_pcie_domain_alloc(struct irq_domain *d, unsigned int virq, unsigned int nr_irqs,
+			       void *arg)
+{
+	/* TODO: move the content of hv_compose_msi_msg() in here */

Could you elaborate on this TODO? Is the idea to loop through all the IRQs and generate the MSI message for each one? What is the advantage to doing it here? I noticed in Patch 3 of the series, the Aardvark controller has advk_msi_irq_compose_msi_msg(), but you had not moved it into the domain allocation path.

Also, is there some point in the time in the future where the "TODO" is likely to become a "MUST DO"?

Quoted 59 lines
drivers/pci/controller/pci-hyperv.c +35 -12
+	int ret;
+
+	ret = irq_domain_alloc_irqs_parent(d, virq, nr_irqs, arg);
+	if (ret < 0)
+		return ret;
+
+	for (int i = 0; i < nr_irqs; i++) {
+		irq_domain_set_info(d, virq + i, 0, &hv_msi_irq_chip, NULL, FLOW_HANDLER, NULL,
+				    FLOW_NAME);
+	}
+
+	return 0;
+}
+
+static void hv_pcie_domain_free(struct irq_domain *d, unsigned int virq, unsigned int nr_irqs)
+{
+	struct msi_domain_info *info = d->host_data;
+
+	for (int i = 0; i < nr_irqs; i++)
+		hv_msi_free(d, info, virq + i);
+
+	irq_domain_free_irqs_top(d, virq, nr_irqs);
+}
+
+static const struct irq_domain_ops hv_pcie_domain_ops = {
+	.alloc	= hv_pcie_domain_alloc,
+	.free	= hv_pcie_domain_free,
 };
 
 /**
@@ -2136,17 +2199,14 @@ static struct msi_domain_ops hv_msi_ops = {
  */
 static int hv_pcie_init_irq_domain(struct hv_pcibus_device *hbus)
 {
-	hbus->msi_info.chip = &hv_msi_irq_chip;
-	hbus->msi_info.ops = &hv_msi_ops;
-	hbus->msi_info.flags = (MSI_FLAG_USE_DEF_DOM_OPS |
-		MSI_FLAG_USE_DEF_CHIP_OPS | MSI_FLAG_MULTI_PCI_MSI |
-		MSI_FLAG_PCI_MSIX);
-	hbus->msi_info.handler = FLOW_HANDLER;
-	hbus->msi_info.handler_name = FLOW_NAME;
-	hbus->msi_info.data = hbus;
-	hbus->irq_domain = pci_msi_create_irq_domain(hbus->fwnode,
-						     &hbus->msi_info,
-						     hv_pci_get_root_domain());
+	struct irq_domain_info info = {
+		.fwnode		= hbus->fwnode,
+		.ops		= &hv_pcie_domain_ops,
+		.host_data	= hbus,
+		.parent		= hv_pci_get_root_domain(),
+	};
+
+	hbus->irq_domain = msi_create_parent_irq_domain(&info, &hv_pcie_msi_parent_ops);
 	if (!hbus->irq_domain) {
 		dev_err(&hbus->hdev->device,
 			"Failed to build an MSI IRQ domain\n");
--

2.39.5

author
#42 lore raw

Re: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

Quoted 44 lines · Michael Kelley

From: Nam Cao <[email protected]> Sent: Thursday, June 26, 2025 7:48 AM

Quoted 3 lines

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

With the additional tweak to this patch that you supplied separately, everything in my testing on both x86 and arm64 seems to work OK. So that's all good.

On arm64, I did notice the following IRQ domain information from /sys/kernel/debug/irq/domains:

# cat HV-PCI-MSIX-1e03\:00\:00.0-12
name:   HV-PCI-MSIX-1e03:00:00.0-12
 size:   0
 mapped: 7
 flags:  0x00000213
            IRQ_DOMAIN_FLAG_HIERARCHY
            IRQ_DOMAIN_NAME_ALLOCATED
            IRQ_DOMAIN_FLAG_MSI
            IRQ_DOMAIN_FLAG_MSI_DEVICE
 parent: 5D202AA8-1E03-4F0F-A786-390A0D2749E9-3
    name:   5D202AA8-1E03-4F0F-A786-390A0D2749E9-3
     size:   0
     mapped: 7
     flags:  0x00000103
                IRQ_DOMAIN_FLAG_HIERARCHY
                IRQ_DOMAIN_NAME_ALLOCATED
                IRQ_DOMAIN_FLAG_MSI_PARENT
     parent: hv_vpci_arm64
        name:   hv_vpci_arm64
         size:   956
         mapped: 31
         flags:  0x00000003
                    IRQ_DOMAIN_FLAG_HIERARCHY
                    IRQ_DOMAIN_NAME_ALLOCATED
         parent: irqchip@0x00000000ffff0000-1
            name:   irqchip@0x00000000ffff0000-1
             size:   0
             mapped: 47
             flags:  0x00000003
                        IRQ_DOMAIN_FLAG_HIERARCHY
                        IRQ_DOMAIN_NAME_ALLOCATED
Quoted 5 lines

The 5D202AA8-1E03-4F0F-A786-390A0D2749E9-3 domain has IRQ_DOMAIN_FLAG_MSI_PARENT set. But the hv_vpci_arm64 and irqchip@... domains do not. Is that a problem? On x86, the output is this, with IRQ_DOMAIN_FLAG_MSI_PARENT set in the next level up VECTOR domain:

That looks normal. IRQ_DOMAIN_FLAG_MSI_PARENT is set for domains which provide MSI parent domain capability, which happens to be the case for x86 vector.

Quoted 160 lines
# cat HV-PCI-MSIX-6b71\:00\:02.0-12
name:   HV-PCI-MSIX-6b71:00:02.0-12
 size:   0
 mapped: 17
 flags:  0x00000213
            IRQ_DOMAIN_FLAG_HIERARCHY
            IRQ_DOMAIN_NAME_ALLOCATED
            IRQ_DOMAIN_FLAG_MSI
            IRQ_DOMAIN_FLAG_MSI_DEVICE
 parent: 8564CB14-6B71-477C-B189-F175118E6FF0-3
    name:   8564CB14-6B71-477C-B189-F175118E6FF0-3
     size:   0
     mapped: 17
     flags:  0x00000103
                IRQ_DOMAIN_FLAG_HIERARCHY
                IRQ_DOMAIN_NAME_ALLOCATED
                IRQ_DOMAIN_FLAG_MSI_PARENT
     parent: VECTOR
        name:   VECTOR
         size:   0
         mapped: 67
         flags:  0x00000103
                    IRQ_DOMAIN_FLAG_HIERARCHY
                    IRQ_DOMAIN_NAME_ALLOCATED
                    IRQ_DOMAIN_FLAG_MSI_PARENT

Finally, I've noted a couple of code review comments below. These comments may reflect my lack of fully understanding the MSI IRQ handling, in which case, please set me straight. Thanks,

Michael

Quoted 116 lines

---
Cc: K. Y. Srinivasan <[email protected]>
Cc: Haiyang Zhang <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: Dexuan Cui <[email protected]>
Cc: [email protected]

---
 drivers/pci/Kconfig                 |  1 +
 drivers/pci/controller/pci-hyperv.c | 98 +++++++++++++++++++++++------
 2 files changed, 80 insertions(+), 19 deletions(-)
drivers/pci/Kconfig +1 -0
index 9c0e4aaf4e8cb..9a249c65aedcd 100644
@@ -223,6 +223,7 @@ config PCI_HYPERV
 	tristate "Hyper-V PCI Frontend"
 	depends on ((X86 && X86_64) || ARM64) && HYPERV && PCI_MSI && SYSFS
 	select PCI_HYPERV_INTERFACE
+	select IRQ_MSI_LIB
 	help
 	  The PCI device frontend driver allows the kernel to import arbitrary
 	  PCI devices from a PCI backend to support PCI driver domains.
drivers/pci/controller/pci-hyperv.c +40 -4
index ef5d655a0052c..3a24fadddb83b 100644
@@ -44,6 +44,7 @@
 #include <linux/delay.h>
 #include <linux/semaphore.h>
 #include <linux/irq.h>
+#include <linux/irqchip/irq-msi-lib.h>
 #include <linux/msi.h>
 #include <linux/hyperv.h>
 #include <linux/refcount.h>
@@ -508,7 +509,6 @@ struct hv_pcibus_device {
 	struct list_head children;
 	struct list_head dr_list;
 
-	struct msi_domain_info msi_info;
 	struct irq_domain *irq_domain;
 
 	struct workqueue_struct *wq;
@@ -1687,7 +1687,7 @@ static void hv_msi_free(struct irq_domain *domain, struct msi_domain_info *info,
 	struct msi_desc *msi = irq_data_get_msi_desc(irq_data);
 
 	pdev = msi_desc_to_pci_dev(msi);
-	hbus = info->data;
+	hbus = domain->host_data;
 	int_desc = irq_data_get_irq_chip_data(irq_data);
 	if (!int_desc)
 		return;
@@ -1705,7 +1705,6 @@ static void hv_msi_free(struct irq_domain *domain, struct msi_domain_info *info,
 
 static void hv_irq_mask(struct irq_data *data)
 {
-	pci_msi_mask_irq(data);
 	if (data->parent_data->chip->irq_mask)
 		irq_chip_mask_parent(data);
 }
@@ -1716,7 +1715,6 @@ static void hv_irq_unmask(struct irq_data *data)
 
 	if (data->parent_data->chip->irq_unmask)
 		irq_chip_unmask_parent(data);
-	pci_msi_unmask_irq(data);
 }
 
 struct compose_comp_ctxt {
@@ -2101,6 +2099,44 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
 	msg->data = 0;
 }
 
+static bool hv_pcie_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+				      struct irq_domain *real_parent, struct msi_domain_info *info)
+{
+	struct irq_chip *chip = info->chip;
+
+	if (!msi_lib_init_dev_msi_info(dev, domain, real_parent, info))
+		return false;
+
+	info->ops->msi_prepare = hv_msi_prepare;
+
+	chip->irq_set_affinity = irq_chip_set_affinity_parent;
+
+	if (IS_ENABLED(CONFIG_X86))
+		chip->flags |= IRQCHIP_MOVE_DEFERRED;
+
+	return true;
+}
+
+#define HV_PCIE_MSI_FLAGS_REQUIRED (MSI_FLAG_USE_DEF_DOM_OPS	| \
+				    MSI_FLAG_USE_DEF_CHIP_OPS		| \
+				    MSI_FLAG_PCI_MSI_MASK_PARENT)
+#define HV_PCIE_MSI_FLAGS_SUPPORTED (MSI_FLAG_MULTI_PCI_MSI	| \
+				     MSI_FLAG_PCI_MSIX			| \
+				     MSI_GENERIC_FLAGS_MASK)
+
+static const struct msi_parent_ops hv_pcie_msi_parent_ops = {
+	.required_flags		= HV_PCIE_MSI_FLAGS_REQUIRED,
+	.supported_flags	= HV_PCIE_MSI_FLAGS_SUPPORTED,
+	.bus_select_token	= DOMAIN_BUS_PCI_MSI,
+#ifdef CONFIG_X86
+	.chip_flags		= MSI_CHIP_FLAG_SET_ACK,
+#elif defined(CONFIG_ARM64)
+	.chip_flags		= MSI_CHIP_FLAG_SET_EOI,
+#endif
+	.prefix			= "HV-",
+	.init_dev_msi_info	= hv_pcie_init_dev_msi_info,
+};
+
 /* HW Interrupt Chip Descriptor */
 static struct irq_chip hv_msi_irq_chip = {
 	.name			= "Hyper-V PCIe MSI",

drivers/pci/controller/pci-hyperv.c static struct irq_chip hv_msi_irq_chip = { · line 2149

Quoted 7 lines
drivers/pci/controller/pci-hyperv.c +0 -1
@@ -2108,7 +2144,6 @@ static struct irq_chip hv_msi_irq_chip = {
 	.irq_set_affinity	= irq_chip_set_affinity_parent,
 #ifdef CONFIG_X86
 	.irq_ack		= irq_chip_ack_parent,
-	.flags			= IRQCHIP_MOVE_DEFERRED,
 #elif defined(CONFIG_ARM64)
 	.irq_eoi		= irq_chip_eoi_parent,
 #endif

Would it work to drop the #ifdef's and always set both .irq_ack and .irq_eoi on x86 and on ARM64? Is which one gets called controlled by the child HV-PCI-MSIX- ... domain, based on the .chip_flags?

Quoted 4 lines

I'm trying to reduce the #ifdef clutter. I tested without the #ifdefs on both x86 and arm64, and everything works, but I know that doesn't prove that it's OK.

Nothing is wrong with that, as far as I can tell.

Quoted 11 lines

If the #ifdefs can go away, then I'd like to see a tweak to the way .chip_flags is set. Rather than do an #ifdef inline for struct msi_parent_ops hv_pcie_msi_parent_ops, add a #define HV_MSI_CHIP_FLAGS in the existing #ifdef X86 and #ifdef ARM64 sections respectively near the top of this source file, and then use HV_MSI_CHIP_FLAGS in struct msi_parent_ops hv_pcie_msi_parent_ops. As much as is reasonable, I'd like to not clutter the code with #ifdef X86 #elseif ARM64, but instead group all the differences under the existing #ifdefs near the top. There are some places where this isn't practical, but this seems like a place that is practical.

Yes, that would be better. I will do it in v2.

Quoted 4 lines
Quoted 4 lines
drivers/pci/controller/pci-hyperv.c +0 -0
@@ -2116,9 +2151,37 @@ static struct irq_chip hv_msi_irq_chip = {
 	.irq_unmask		= hv_irq_unmask,
 };
 
Quoted 13 lines

drivers/pci/controller/pci-hyperv.c static struct irq_chip hv_msi_irq_chip = { · line 2157

Quoted 7 lines
drivers/pci/controller/pci-hyperv.c +4 -3
-static struct msi_domain_ops hv_msi_ops = {
-	.msi_prepare	= hv_msi_prepare,
-	.msi_free	= hv_msi_free,
+static int hv_pcie_domain_alloc(struct irq_domain *d, unsigned int virq, unsigned int nr_irqs,
+			       void *arg)
+{
+	/* TODO: move the content of hv_compose_msi_msg() in here */

Could you elaborate on this TODO? Is the idea to loop through all the IRQs and generate the MSI message for each one? What is the advantage to doing it here? I noticed in Patch 3 of the series, the Aardvark controller has advk_msi_irq_compose_msi_msg(), but you had not moved it into the domain allocation path.

Sorry for being unclear. hv_compose_msi_msg() should not be moved here entirely. Let me elaborate this in v2.

What I meant is that, hv_compose_msi_msg() is doing more than what this callback is supposed to do (composing message). It works, but it is not correct. Interrupt allocation is the responsibility of irq_domain_ops::alloc(). Allocating and populating int_desc should be in hv_pcie_domain_alloc() instead.

irq_domain_ops's .alloc() and .free() should be asymmetric.

Quoted 3 lines

Also, is there some point in the time in the future where the "TODO" is likely to become a "MUST DO"?

There's nothing planned that would make this non-functional, as far as I know.

Thanks so much for examining the patch,
Nam

author
#43 lore raw

Re: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

Quoted 11 lines · Nam Cao
Quoted 8 lines · Michael Kelley

From: Nam Cao <[email protected]> Sent: Thursday, June 26, 2025 7:48 AM

Quoted 3 lines

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

With the additional tweak to this patch that you supplied separately, everything in my testing on both x86 and arm64 seems to work OK. So that's all good.

Thanks so much for examining the patch,

Btw, you probably would also be interested in
https://lore.kernel.org/lkml/[email protected]/

which does a similar conversion for the other hyperv driver.

Nam

#44 lore raw

RE: [PATCH 14/16] PCI: hv: Switch to msi_create_parent_irq_domain()

From: Nam Cao <[email protected]> Sent: Saturday, July 5, 2025 3:02 AM

Quoted 12 lines · Nam Cao
Quoted 10 lines · Michael Kelley
Quoted 8 lines

From: Nam Cao <[email protected]> Sent: Thursday, June 26, 2025 7:48 AM

Quoted 3 lines

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

With the additional tweak to this patch that you supplied separately, everything in my testing on both x86 and arm64 seems to work OK. So that's all good.

Quoted 7 lines
Quoted 1 line

Thanks so much for examining the patch,

Btw, you probably would also be interested in
https://lore.kernel.org/lkml/[email protected]/

which does a similar conversion for the other hyperv driver.

Thanks. I had seen this other small series for the driver used by Linux when running as the root partition (i.e., "dom0") on the Hyper-V hypervisor. Unfortunately, I have less familiarity with that configuration, and I lack access to hardware needed to run and test it. Someone at Microsoft would need to look at the patches and give them a basic smoke test with Linux-as-root.

FWIW, I was a Microsoft employee for many years, working on Linux guests on Hyper-V. But I retired about 2 years ago. Though I've kept my hands in Linux kernel work, my access to hardware configs is more limited than when I was an employee -- now it's guest VMs on my personal laptop and Azure VMs.

Michael

author
#45 lore raw

PATCH 15/16 PCI: vmd: Convert to lock guards

In reply to Nam Cao

Convert lock/unlock pairs to lock guard and tidy up the code.

---
Cc: Nirmal Patel <[email protected]>
Cc: Jonathan Derrick <[email protected]>

---
 drivers/pci/controller/vmd.c | 73 ++++++++++++++----------------------
 1 file changed, 29 insertions(+), 44 deletions(-)
drivers/pci/controller/vmd.c +29 -44
index 375ce9d6d9f6a..d9b893bf4e456 100644
@@ -180,13 +180,12 @@ static void vmd_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
 static void vmd_irq_enable(struct irq_data *data)
 {
 	struct vmd_irq *vmdirq = data->chip_data;
-	unsigned long flags;
 
-	raw_spin_lock_irqsave(&list_lock, flags);
-	WARN_ON(vmdirq->enabled);
-	list_add_tail_rcu(&vmdirq->node, &vmdirq->irq->irq_list);
-	vmdirq->enabled = true;
-	raw_spin_unlock_irqrestore(&list_lock, flags);
+	scoped_guard(raw_spinlock_irqsave, &list_lock) {
+		WARN_ON(vmdirq->enabled);
+		list_add_tail_rcu(&vmdirq->node, &vmdirq->irq->irq_list);
+		vmdirq->enabled = true;
+	}
 
 	data->chip->irq_unmask(data);
 }
@@ -194,16 +193,15 @@ static void vmd_irq_enable(struct irq_data *data)
 static void vmd_irq_disable(struct irq_data *data)
 {
 	struct vmd_irq *vmdirq = data->chip_data;
-	unsigned long flags;
 
 	data->chip->irq_mask(data);
 
-	raw_spin_lock_irqsave(&list_lock, flags);
-	if (vmdirq->enabled) {
-		list_del_rcu(&vmdirq->node);
-		vmdirq->enabled = false;
+	scoped_guard(raw_spinlock_irqsave, &list_lock) {
+		if (vmdirq->enabled) {
+			list_del_rcu(&vmdirq->node);
+			vmdirq->enabled = false;
+		}
 	}
-	raw_spin_unlock_irqrestore(&list_lock, flags);
 }
 
 static struct irq_chip vmd_msi_controller = {
@@ -225,7 +223,6 @@ static irq_hw_number_t vmd_get_hwirq(struct msi_domain_info *info,
  */
 static struct vmd_irq_list *vmd_next_irq(struct vmd_dev *vmd, struct msi_desc *desc)
 {
-	unsigned long flags;
 	int i, best;
 
 	if (vmd->msix_count == 1 + vmd->first_vec)
@@ -242,13 +239,13 @@ static struct vmd_irq_list *vmd_next_irq(struct vmd_dev *vmd, struct msi_desc *d
 		return &vmd->irqs[vmd->first_vec];
 	}
 
-	raw_spin_lock_irqsave(&list_lock, flags);
-	best = vmd->first_vec + 1;
-	for (i = best; i < vmd->msix_count; i++)
-		if (vmd->irqs[i].count < vmd->irqs[best].count)
-			best = i;
-	vmd->irqs[best].count++;
-	raw_spin_unlock_irqrestore(&list_lock, flags);
+	scoped_guard(raw_spinlock_irq, &list_lock) {
+		best = vmd->first_vec + 1;
+		for (i = best; i < vmd->msix_count; i++)
+			if (vmd->irqs[i].count < vmd->irqs[best].count)
+				best = i;
+		vmd->irqs[best].count++;
+	}
 
 	return &vmd->irqs[best];
 }
@@ -277,14 +274,12 @@ static void vmd_msi_free(struct irq_domain *domain,
 			struct msi_domain_info *info, unsigned int virq)
 {
 	struct vmd_irq *vmdirq = irq_get_chip_data(virq);
-	unsigned long flags;
 
 	synchronize_srcu(&vmdirq->irq->srcu);
 
 	/* XXX: Potential optimization to rebalance */
-	raw_spin_lock_irqsave(&list_lock, flags);
-	vmdirq->irq->count--;
-	raw_spin_unlock_irqrestore(&list_lock, flags);
+	scoped_guard(raw_spinlock_irq, &list_lock)
+		vmdirq->irq->count--;
 
 	kfree(vmdirq);
 }
@@ -387,29 +382,24 @@ static int vmd_pci_read(struct pci_bus *bus, unsigned int devfn, int reg,
 {
 	struct vmd_dev *vmd = vmd_from_bus(bus);
 	void __iomem *addr = vmd_cfg_addr(vmd, bus, devfn, reg, len);
-	unsigned long flags;
-	int ret = 0;
 
 	if (!addr)
 		return -EFAULT;
 
-	raw_spin_lock_irqsave(&vmd->cfg_lock, flags);
+	guard(raw_spinlock_irqsave)(&vmd->cfg_lock);
 	switch (len) {
 	case 1:
 		*value = readb(addr);
-		break;
+		return 0;
 	case 2:
 		*value = readw(addr);
-		break;
+		return 0;
 	case 4:
 		*value = readl(addr);
-		break;
+		return 0;
 	default:
-		ret = -EINVAL;
-		break;
+		return -EINVAL;
 	}
-	raw_spin_unlock_irqrestore(&vmd->cfg_lock, flags);
-	return ret;
 }
 
 /*
@@ -422,32 +412,27 @@ static int vmd_pci_write(struct pci_bus *bus, unsigned int devfn, int reg,
 {
 	struct vmd_dev *vmd = vmd_from_bus(bus);
 	void __iomem *addr = vmd_cfg_addr(vmd, bus, devfn, reg, len);
-	unsigned long flags;
-	int ret = 0;
 
 	if (!addr)
 		return -EFAULT;
 
-	raw_spin_lock_irqsave(&vmd->cfg_lock, flags);
+	guard(raw_spinlock_irqsave)(&vmd->cfg_lock);
 	switch (len) {
 	case 1:
 		writeb(value, addr);
 		readb(addr);
-		break;
+		return 0;
 	case 2:
 		writew(value, addr);
 		readw(addr);
-		break;
+		return 0;
 	case 4:
 		writel(value, addr);
 		readl(addr);
-		break;
+		return 0;
 	default:
-		ret = -EINVAL;
-		break;
+		return -EINVAL;
 	}
-	raw_spin_unlock_irqrestore(&vmd->cfg_lock, flags);
-	return ret;
 }
 
 static struct pci_ops vmd_ops = {
Signature · 1 line
2.39.5
author
#47 lore raw

PATCH 16/16 PCI: vmd: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Nirmal Patel <[email protected]>
Cc: Jonathan Derrick <[email protected]>

---
 drivers/pci/controller/Kconfig |   1 +
 drivers/pci/controller/vmd.c   | 160 +++++++++++++++++----------------
 2 files changed, 82 insertions(+), 79 deletions(-)
drivers/pci/controller/Kconfig +1 -0
index 8f56ffd029ba2..41748d083b933 100644
@@ -156,6 +156,7 @@ config PCI_IXP4XX
 config VMD
 	depends on PCI_MSI && X86_64 && !UML
 	tristate "Intel Volume Management Device Driver"
+	select IRQ_MSI_LIB
 	help
 	  Adds support for the Intel Volume Management Device (VMD). VMD is a
 	  secondary PCI host bridge that allows PCI Express root ports,
drivers/pci/controller/vmd.c +81 -79

242 lines not on this page · raw message

Signature · 1 line
2.39.5
maintainer
#49 lore raw

Re: [PATCH 16/16] PCI: vmd: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao
Quoted 108 lines · On Thu, 26 Jun 2025 16:48:06 +0200 Nam Cao <[email protected]> wrote:

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

---
Cc: Nirmal Patel <[email protected]>
Cc: Jonathan Derrick <[email protected]>

---
 drivers/pci/controller/Kconfig |   1 +
 drivers/pci/controller/vmd.c   | 160

+++++++++++++++++---------------- 2 files changed, 82 insertions(+), 79 deletions(-)

(quoted diff) +0 -0

b/drivers/pci/controller/Kconfig index 8f56ffd029ba2..41748d083b933 100644 --- a/drivers/pci/controller/Kconfig
+++ b/drivers/pci/controller/Kconfig

drivers/pci/controller/Kconfig +1 -0
@@ -156,6 +156,7 @@ config PCI_IXP4XX
 config VMD
 	depends on PCI_MSI && X86_64 && !UML
 	tristate "Intel Volume Management Device Driver"
+	select IRQ_MSI_LIB
 	help
 	  Adds support for the Intel Volume Management Device (VMD).
 VMD is a secondary PCI host bridge that allows PCI Express root ports,
(quoted diff) +0 -0

b/drivers/pci/controller/vmd.c index d9b893bf4e456..38693a9487d9b 100644 --- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c

drivers/pci/controller/vmd.c +1 -2
@@ -7,6 +7,7 @@
 #include <linux/device.h>
 #include <linux/interrupt.h>
 #include <linux/irq.h>
+#include <linux/irqchip/irq-msi-lib.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/msi.h>
@@ -174,9 +175,6 @@ static void vmd_compose_msi_msg(struct irq_data
 *data, struct msi_msg *msg) msg->arch_addr_lo.destid_0_7 =
 index_from_irqs(vmd, irq); }
 
-/*
- * We rely on MSI_FLAG_USE_DEF_CHIP_OPS to set the IRQ mask/unmask

ops.
- */
static void vmd_irq_enable(struct irq_data *data) {

drivers/pci/controller/vmd.c +10 -10
 	struct vmd_irq *vmdirq = data->chip_data;
@@ -186,7 +184,11 @@ static void vmd_irq_enable(struct irq_data *data)
 		list_add_tail_rcu(&vmdirq->node,
 &vmdirq->irq->irq_list); vmdirq->enabled = true;
 	}
+}
 
+static void vmd_pci_msi_enable(struct irq_data *data)
+{
+	vmd_irq_enable(data->parent_data);
 	data->chip->irq_unmask(data);
 }
 
@@ -194,8 +196,6 @@ static void vmd_irq_disable(struct irq_data *data)
 {
 	struct vmd_irq *vmdirq = data->chip_data;
 
-	data->chip->irq_mask(data);
-
 	scoped_guard(raw_spinlock_irqsave, &list_lock) {
 		if (vmdirq->enabled) {
 			list_del_rcu(&vmdirq->node);
@@ -204,19 +204,17 @@ static void vmd_irq_disable(struct irq_data
 *data) }
 }
 
+static void vmd_pci_msi_disable(struct irq_data *data)
+{
+	data->chip->irq_mask(data);
+	vmd_irq_disable(data->parent_data);
+}
+
 static struct irq_chip vmd_msi_controller = {
 	.name			= "VMD-MSI",
-	.irq_enable		= vmd_irq_enable,
-	.irq_disable		= vmd_irq_disable,
 	.irq_compose_msi_msg	= vmd_compose_msi_msg,
 };
 
-static irq_hw_number_t vmd_get_hwirq(struct msi_domain_info *info,
-				     msi_alloc_info_t *arg)
-{
-	return 0;
-}
-
 /*
  * XXX: We can be even smarter selecting the best IRQ once we solve

the
* affinity problem.

(quoted diff) +1 -3
@@ -250,100 +248,110 @@ static struct vmd_irq_list
 *vmd_next_irq(struct vmd_dev *vmd, struct msi_desc *d return
 &vmd->irqs[best]; }
 
-static int vmd_msi_init(struct irq_domain *domain, struct
 msi_domain_info *info,
-			unsigned int virq, irq_hw_number_t hwirq,
-			msi_alloc_info_t *arg)
+static void vmd_msi_free(struct irq_domain *domain, unsigned int

virq, unsigned int nr_irqs); +
+static int vmd_msi_alloc(struct irq_domain *domain, unsigned int virq, unsigned int nr_irqs,

drivers/pci/controller/vmd.c static struct vmd_irq_list *vmd_next_irq(struct vmd_dev *vmd, struct msi_desc *d · line 254

Quoted 1 line
drivers/pci/controller/vmd.c +1 -0
+			 void *arg)

Is this wrapped in 80 columns? I can see few lines are more than 80. Disregard this if it is wrapped and it can be my claws mail client issue.

Quoted 188 lines

{

(quoted diff) +0 -2
-	struct msi_desc *desc = arg->desc;
-	struct vmd_dev *vmd =

vmd_from_bus(msi_desc_to_pci_dev(desc)->bus);

(quoted diff) +0 -1
-	struct vmd_irq *vmdirq = kzalloc(sizeof(*vmdirq),

GFP_KERNEL);

drivers/pci/controller/vmd.c +14 -5
+	struct msi_desc *desc = ((msi_alloc_info_t *)arg)->desc;
+	struct vmd_dev *vmd = domain->host_data;
+	struct vmd_irq *vmdirq;
 
-	if (!vmdirq)
-		return -ENOMEM;
+	for (int i = 0; i < nr_irqs; ++i) {
+		vmdirq = kzalloc(sizeof(*vmdirq), GFP_KERNEL);
+		if (!vmdirq) {
+			vmd_msi_free(domain, virq, i);
+			return -ENOMEM;
+		}
 
-	INIT_LIST_HEAD(&vmdirq->node);
-	vmdirq->irq = vmd_next_irq(vmd, desc);
-	vmdirq->virq = virq;
+		INIT_LIST_HEAD(&vmdirq->node);
+		vmdirq->irq = vmd_next_irq(vmd, desc);
+		vmdirq->virq = virq + i;
+
+		irq_domain_set_info(domain, virq + i,

vmdirq->irq->virq, &vmd_msi_controller,

(quoted diff) +1 -0
+				    vmdirq, handle_untracked_irq,

vmd, NULL);

drivers/pci/controller/vmd.c +1 -1
+	}
 
-	irq_domain_set_info(domain, virq, vmdirq->irq->virq,

info->chip, vmdirq,

drivers/pci/controller/vmd.c +0 -3
-			    handle_untracked_irq, vmd, NULL);
 	return 0;
 }
 
-static void vmd_msi_free(struct irq_domain *domain,
-			struct msi_domain_info *info, unsigned int

virq) +static void vmd_msi_free(struct irq_domain *domain, unsigned int virq, unsigned int nr_irqs) {

drivers/pci/controller/vmd.c +7 -6
 	struct vmd_irq *vmdirq = irq_get_chip_data(virq);
 
-	synchronize_srcu(&vmdirq->irq->srcu);
+	for (int i = 0; i < nr_irqs; ++i) {
+		synchronize_srcu(&vmdirq->irq->srcu);
 
-	/* XXX: Potential optimization to rebalance */
-	scoped_guard(raw_spinlock_irq, &list_lock)
-		vmdirq->irq->count--;
+		/* XXX: Potential optimization to rebalance */
+		scoped_guard(raw_spinlock_irq, &list_lock)
+			vmdirq->irq->count--;
 
-	kfree(vmdirq);
+		kfree(vmdirq);
+	}
 }
 
-static int vmd_msi_prepare(struct irq_domain *domain, struct device

*dev,

drivers/pci/controller/vmd.c +6 -1
-			   int nvec, msi_alloc_info_t *arg)
+static const struct irq_domain_ops vmd_msi_domain_ops = {
+	.alloc		= vmd_msi_alloc,
+	.free		= vmd_msi_free,
+};
+
+static bool vmd_init_dev_msi_info(struct device *dev, struct

irq_domain *domain,

(quoted diff) +1 -0
+				  struct irq_domain *real_parent,

struct msi_domain_info *info) {

drivers/pci/controller/vmd.c +1 -2
-	struct pci_dev *pdev = to_pci_dev(dev);
-	struct vmd_dev *vmd = vmd_from_bus(pdev->bus);
+	if (WARN_ON_ONCE(info->bus_token !=

DOMAIN_BUS_PCI_DEVICE_MSIX))

drivers/pci/controller/vmd.c +2 -2
+		return false;
 
-	if (nvec > vmd->msix_count)
-		return vmd->msix_count;
+	if (!msi_lib_init_dev_msi_info(dev, domain, real_parent,

info))

drivers/pci/controller/vmd.c +4 -3
+		return false;
 
-	memset(arg, 0, sizeof(*arg));
-	return 0;
+	info->chip->irq_enable		= vmd_pci_msi_enable;
+	info->chip->irq_disable		= vmd_pci_msi_disable;
+	return true;
 }
 
-static void vmd_set_desc(msi_alloc_info_t *arg, struct msi_desc

*desc) -{

drivers/pci/controller/vmd.c +1 -2
-	arg->desc = desc;
-}
+#define VMD_MSI_FLAGS_SUPPORTED

(MSI_GENERIC_FLAGS_MASK | MSI_FLAG_PCI_MSIX) +#define
VMD_MSI_FLAGS_REQUIRED (MSI_FLAG_USE_DEF_DOM_OPS |
MSI_FLAG_NO_AFFINITY) -static struct msi_domain_ops vmd_msi_domain_ops = {

drivers/pci/controller/vmd.c +7 -7
-	.get_hwirq	= vmd_get_hwirq,
-	.msi_init	= vmd_msi_init,
-	.msi_free	= vmd_msi_free,
-	.msi_prepare	= vmd_msi_prepare,
-	.set_desc	= vmd_set_desc,
+static const struct msi_parent_ops vmd_msi_parent_ops = {
+	.supported_flags	= VMD_MSI_FLAGS_SUPPORTED,
+	.required_flags		= VMD_MSI_FLAGS_REQUIRED,
+	.bus_select_token	= DOMAIN_BUS_VMD_MSI,
+	.bus_select_mask	= MATCH_PCI_MSI,
+	.prefix			= "VMD-",
+	.init_dev_msi_info	= vmd_init_dev_msi_info,
 };
 
-static struct msi_domain_info vmd_msi_domain_info = {
-	.flags		= MSI_FLAG_USE_DEF_DOM_OPS |

MSI_FLAG_USE_DEF_CHIP_OPS |

drivers/pci/controller/vmd.c +5 -17
-			  MSI_FLAG_NO_AFFINITY | MSI_FLAG_PCI_MSIX,
-	.ops		= &vmd_msi_domain_ops,
-	.chip		= &vmd_msi_controller,
-};
-
-static void vmd_set_msi_remapping(struct vmd_dev *vmd, bool enable)
-{
-	u16 reg;
-
-	pci_read_config_word(vmd->dev, PCI_REG_VMCONFIG, &reg);
-	reg = enable ? (reg & ~VMCONFIG_MSI_REMAP) :
-		       (reg | VMCONFIG_MSI_REMAP);
-	pci_write_config_word(vmd->dev, PCI_REG_VMCONFIG, reg);
-}
-
 static int vmd_create_irq_domain(struct vmd_dev *vmd)
 {
-	struct fwnode_handle *fn;
+	struct irq_domain_info info = {
+		.size		= vmd->msix_count,
+		.ops		= &vmd_msi_domain_ops,
+		.host_data	= vmd,
+	};
 
-	fn = irq_domain_alloc_named_id_fwnode("VMD-MSI",

vmd->sysdata.domain);

(quoted diff) +1 -1
-	if (!fn)
+	info.fwnode = irq_domain_alloc_named_id_fwnode("VMD-MSI",

vmd->sysdata.domain);

drivers/pci/controller/vmd.c +1 -1
+	if (!info.fwnode)
 		return -ENODEV;
 
-	vmd->irq_domain = pci_msi_create_irq_domain(fn,

&vmd_msi_domain_info, NULL);

(quoted diff) +1 -0
+	vmd->irq_domain = msi_create_parent_irq_domain(&info,

&vmd_msi_parent_ops); if (!vmd->irq_domain) {

drivers/pci/controller/vmd.c +11 -4
-		irq_domain_free_fwnode(fn);
+		irq_domain_free_fwnode(info.fwnode);
 		return -ENODEV;
 	}
 
 	return 0;
 }
 
+static void vmd_set_msi_remapping(struct vmd_dev *vmd, bool enable)
+{
+	u16 reg;
+
+	pci_read_config_word(vmd->dev, PCI_REG_VMCONFIG, &reg);
+	reg = enable ? (reg & ~VMCONFIG_MSI_REMAP) :
+		       (reg | VMCONFIG_MSI_REMAP);
+	pci_write_config_word(vmd->dev, PCI_REG_VMCONFIG, reg);
+}
+
 static void vmd_remove_irq_domain(struct vmd_dev *vmd)
 {
 	/*
@@ -874,12 +882,6 @@ static int vmd_enable_domain(struct vmd_dev
 *vmd, unsigned long features) ret = vmd_create_irq_domain(vmd);
 		if (ret)
 			return ret;
-
-		/*
-		 * Override the IRQ domain bus token so the domain

can be

drivers/pci/controller/vmd.c +0 -3
-		 * distinguished from a regular PCI/MSI domain.
-		 */
-		irq_domain_update_bus_token(vmd->irq_domain,

DOMAIN_BUS_VMD_MSI); } else {

drivers/pci/controller/vmd.c +0 -0
 		vmd_set_msi_remapping(vmd, false);
 	}
#50 lore raw

Re: [PATCH 16/16] PCI: vmd: Switch to msi_create_parent_irq_domain()

Quoted 6 lines · Nirmal Patel
Quoted 2 lines · On Thu, 26 Jun 2025 16:48:06 +0200 Nam Cao <[email protected]> wrote:

Move away from the legacy MSI domain setup, switch to use msi_create_parent_irq_domain().

Quoted 11 lines

drivers/pci/controller/vmd.c static struct vmd_irq_list *vmd_next_irq(struct vmd_dev *vmd, struct msi_desc *d · line 254

Quoted 7 lines
drivers/pci/controller/vmd.c +1 -2
-			unsigned int virq, irq_hw_number_t hwirq,
-			msi_alloc_info_t *arg)
+static void vmd_msi_free(struct irq_domain *domain, unsigned int

virq, unsigned int nr_irqs); +
+static int vmd_msi_alloc(struct irq_domain *domain, unsigned int virq, unsigned int nr_irqs,

drivers/pci/controller/vmd.c +1 -0
+			 void *arg)

Is this wrapped in 80 columns? I can see few lines are more than 80. Disregard this if it is wrapped and it can be my claws mail client issue.

Wrapped locally, thanks Nirmal.

Bjorn

#51 lore raw

Re: [PATCH 16/16] PCI: vmd: Switch to msi_create_parent_irq_domain()

In reply to Nam Cao

Hi Nam,

On 26/06/2025 16:48, Nam Cao wrote:
[...]

Quoted 10 lines
drivers/pci/controller/vmd.c +3 -3
-static void vmd_msi_free(struct irq_domain *domain,
-			struct msi_domain_info *info, unsigned int virq)
+static void vmd_msi_free(struct irq_domain *domain, unsigned int virq, unsigned int nr_irqs)
  {
  	struct vmd_irq *vmdirq = irq_get_chip_data(virq);
 
-	synchronize_srcu(&vmdirq->irq->srcu);
+	for (int i = 0; i < nr_irqs; ++i) {
+		synchronize_srcu(&vmdirq->irq->srcu);
 

drivers/pci/controller/vmd.c static struct vmd_irq_list *vmd_next_irq(struct vmd_dev *vmd, struct msi_desc *d · line 290

Quoted 10 lines
drivers/pci/controller/vmd.c +5 -4
-	/* XXX: Potential optimization to rebalance */
-	scoped_guard(raw_spinlock_irq, &list_lock)
-		vmdirq->irq->count--;
+		/* XXX: Potential optimization to rebalance */
+		scoped_guard(raw_spinlock_irq, &list_lock)
+			vmdirq->irq->count--;
 
-	kfree(vmdirq);
+		kfree(vmdirq);
+	}

By introducing a for loop in this function, you are re-using vmdirq after free'ing it.

I can't send a patch because I am not faimliar with this API and I don't know how to fix it.

However, the issue was reported today by Coverity.

Any idea? :-)

Regards,

Signature · 5 lines
Antonio Quartulli

CEO and Co-Founder
Mandelbit Srl
https://www.mandelbit.com
author
#52 lore raw

Re: [PATCH 16/16] PCI: vmd: Switch to msi_create_parent_irq_domain()

Quoted 1 line · Antonio Quartulli

Hi Nam,

Hi Antonio,

Quoted 23 lines · Nam Cao

[...]

Quoted 7 lines
(quoted diff) +2 -3
-static void vmd_msi_free(struct irq_domain *domain,
-			struct msi_domain_info *info, unsigned int virq)
+static void vmd_msi_free(struct irq_domain *domain, unsigned int virq, unsigned int nr_irqs)
  {
  	struct vmd_irq *vmdirq = irq_get_chip_data(virq);
-	synchronize_srcu(&vmdirq->irq->srcu);
+	for (int i = 0; i < nr_irqs; ++i) {

Review

Quoted 10 lines
(quoted diff) +6 -4
+		synchronize_srcu(&vmdirq->irq->srcu);
-	/* XXX: Potential optimization to rebalance */
-	scoped_guard(raw_spinlock_irq, &list_lock)
-		vmdirq->irq->count--;
+		/* XXX: Potential optimization to rebalance */
+		scoped_guard(raw_spinlock_irq, &list_lock)
+			vmdirq->irq->count--;
-	kfree(vmdirq);
+		kfree(vmdirq);
+	}

By introducing a for loop in this function, you are re-using vmdirq after free'ing it.

Quoted 6 lines

I can't send a patch because I am not faimliar with this API and I don't know how to fix it.

However, the issue was reported today by Coverity.

Any idea? :-)

Thanks for the report. That was indeed a mistake from my side.

I hope PCI maintainers don't mind squashing the below diff.

Sorry for the troubles so far,
Nam

drivers/pci/controller/vmd.c +3 -1
index 48a6096cbbc0..50f0c91d561c 100644
@@ -280,9 +280,11 @@ static int vmd_msi_alloc(struct irq_domain *domain, unsigned int virq,
 static void vmd_msi_free(struct irq_domain *domain, unsigned int virq,
 			 unsigned int nr_irqs)
 {
-	struct vmd_irq *vmdirq = irq_get_chip_data(virq);
+	struct vmd_irq *vmdirq;
 
 	for (int i = 0; i < nr_irqs; ++i) {
+		vmdirq = irq_get_chip_data(virq + i);
+
 		synchronize_srcu(&vmdirq->irq->srcu);
 
 		/* XXX: Potential optimization to rebalance */
#53 lore raw

Re: [PATCH 16/16] PCI: vmd: Switch to msi_create_parent_irq_domain()

Quoted 30 lines · Nam Cao
Quoted 1 line · Antonio Quartulli

Hi Nam,

Hi Antonio,

Quoted 26 lines · Nam Cao

[...]

Quoted 7 lines
(quoted diff) +2 -3
-static void vmd_msi_free(struct irq_domain *domain,
-			struct msi_domain_info *info, unsigned int virq)
+static void vmd_msi_free(struct irq_domain *domain, unsigned int virq, unsigned int nr_irqs)
  {
  	struct vmd_irq *vmdirq = irq_get_chip_data(virq);
-	synchronize_srcu(&vmdirq->irq->srcu);
+	for (int i = 0; i < nr_irqs; ++i) {

Review

Quoted 10 lines
(quoted diff) +6 -4
+		synchronize_srcu(&vmdirq->irq->srcu);
-	/* XXX: Potential optimization to rebalance */
-	scoped_guard(raw_spinlock_irq, &list_lock)
-		vmdirq->irq->count--;
+		/* XXX: Potential optimization to rebalance */
+		scoped_guard(raw_spinlock_irq, &list_lock)
+			vmdirq->irq->count--;
-	kfree(vmdirq);
+		kfree(vmdirq);
+	}

By introducing a for loop in this function, you are re-using vmdirq after free'ing it.

I can't send a patch because I am not faimliar with this API and I don't know how to fix it.

Quoted 7 lines
Quoted 3 lines

However, the issue was reported today by Coverity.

Any idea? :-)

Thanks for the report. That was indeed a mistake from my side.

I hope PCI maintainers don't mind squashing the below diff.

Squashed, thanks! Updated commit:

https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git/commit/?id=4246b7fccf26

Quoted 18 lines
drivers/pci/controller/vmd.c +3 -1
index 48a6096cbbc0..50f0c91d561c 100644
@@ -280,9 +280,11 @@ static int vmd_msi_alloc(struct irq_domain *domain, unsigned int virq,
 static void vmd_msi_free(struct irq_domain *domain, unsigned int virq,
 			 unsigned int nr_irqs)
 {
-	struct vmd_irq *vmdirq = irq_get_chip_data(virq);
+	struct vmd_irq *vmdirq;
 
 	for (int i = 0; i < nr_irqs; ++i) {
+		vmdirq = irq_get_chip_data(virq + i);
+
 		synchronize_srcu(&vmdirq->irq->srcu);
 
 		/* XXX: Potential optimization to rebalance */
Acked-by on the v1 series
#54 lore raw
In reply to Nam Cao
Quoted 41 lines · Nam Cao

The initial implementation of PCI/MSI interrupt domains in the hierarchical interrupt domain model used a shortcut by providing a global PCI/MSI domain.

This works because the PCI/MSI[X] hardware is standardized and uniform, but it violates the basic design principle of hierarchical interrupt domains:
Each hardware block involved in the interrupt delivery chain should have a separate interrupt domain.

For PCI/MSI[X], the interrupt controller is per PCI device and not a global made-up entity.

Unsurprisingly, the shortcut turned out to have downsides as it does not allow dynamic allocation of interrupt vectors after initialization and it prevents supporting IMS on PCI. For further details, see:

https://lore.kernel.org/lkml/[email protected]/

The solution is implementing per device MSI domains, this means the entities which provide global PCI/MSI domain so far have to implement MSI parent domain functionality instead.

This series converts the PCI controller drivers to implement MSI parent domain.

 drivers/pci/Kconfig                           |   1 +
 drivers/pci/controller/Kconfig                |  11 +
 drivers/pci/controller/dwc/Kconfig            |   1 +
 .../pci/controller/dwc/pcie-designware-host.c |  68 ++----
 drivers/pci/controller/dwc/pcie-designware.h  |   1 -
 drivers/pci/controller/mobiveil/Kconfig       |   1 +
 .../controller/mobiveil/pcie-mobiveil-host.c  |  42 ++--
 .../pci/controller/mobiveil/pcie-mobiveil.h   |   1 -
 drivers/pci/controller/pci-aardvark.c         |  59 ++---
 drivers/pci/controller/pci-hyperv.c           |  98 ++++++--
 drivers/pci/controller/pcie-altera-msi.c      |  44 ++--
 drivers/pci/controller/pcie-brcmstb.c         |  44 ++--
 drivers/pci/controller/pcie-iproc-msi.c       |  45 ++--
 drivers/pci/controller/pcie-mediatek-gen3.c   |  67 ++---
 drivers/pci/controller/pcie-mediatek.c        |  46 ++--
 drivers/pci/controller/pcie-rcar-host.c       |  69 ++----
Quoted 8 lines
 drivers/pci/controller/pcie-xilinx-dma-pl.c   |  48 ++--
 drivers/pci/controller/pcie-xilinx-nwl.c      |  45 ++--
 drivers/pci/controller/pcie-xilinx.c          |  55 +++--
 drivers/pci/controller/plda/Kconfig           |   1 +
 drivers/pci/controller/plda/pcie-plda-host.c  |  44 ++--
 drivers/pci/controller/plda/pcie-plda.h       |   1 -
 drivers/pci/controller/vmd.c                  | 229 +++++++++---------
 23 files changed, 504 insertions(+), 517 deletions(-)

Looks good to me, thanks! I think Mani will probably pick this up.

I might have included the specific "legacy MSI domain" thing you're replacing. It looks like you're replacing pci_msi_create_irq_domain() with msi_create_parent_irq_domain()?

Minor merge conflict in pcie-mediatek-gen3.c with dcbea1c7e94e ("PCI:
mediatek-gen3: Use dev_fwnode() for irq_domain_create_linear()"). No problem, we can easily fix that up.

The "++i" in vmd.c stuck out to me since "i++" is so much more common.

author
#55 lore raw
Quoted 51 lines · Bjorn Helgaas
Quoted 41 lines · Nam Cao

The initial implementation of PCI/MSI interrupt domains in the hierarchical interrupt domain model used a shortcut by providing a global PCI/MSI domain.

This works because the PCI/MSI[X] hardware is standardized and uniform, but it violates the basic design principle of hierarchical interrupt domains:
Each hardware block involved in the interrupt delivery chain should have a separate interrupt domain.

For PCI/MSI[X], the interrupt controller is per PCI device and not a global made-up entity.

Unsurprisingly, the shortcut turned out to have downsides as it does not allow dynamic allocation of interrupt vectors after initialization and it prevents supporting IMS on PCI. For further details, see:

https://lore.kernel.org/lkml/[email protected]/

The solution is implementing per device MSI domains, this means the entities which provide global PCI/MSI domain so far have to implement MSI parent domain functionality instead.

This series converts the PCI controller drivers to implement MSI parent domain.

 drivers/pci/Kconfig                           |   1 +
 drivers/pci/controller/Kconfig                |  11 +
 drivers/pci/controller/dwc/Kconfig            |   1 +
 .../pci/controller/dwc/pcie-designware-host.c |  68 ++----
 drivers/pci/controller/dwc/pcie-designware.h  |   1 -
 drivers/pci/controller/mobiveil/Kconfig       |   1 +
 .../controller/mobiveil/pcie-mobiveil-host.c  |  42 ++--
 .../pci/controller/mobiveil/pcie-mobiveil.h   |   1 -
 drivers/pci/controller/pci-aardvark.c         |  59 ++---
 drivers/pci/controller/pci-hyperv.c           |  98 ++++++--
 drivers/pci/controller/pcie-altera-msi.c      |  44 ++--
 drivers/pci/controller/pcie-brcmstb.c         |  44 ++--
 drivers/pci/controller/pcie-iproc-msi.c       |  45 ++--
 drivers/pci/controller/pcie-mediatek-gen3.c   |  67 ++---
 drivers/pci/controller/pcie-mediatek.c        |  46 ++--
 drivers/pci/controller/pcie-rcar-host.c       |  69 ++----
Quoted 8 lines
 drivers/pci/controller/pcie-xilinx-dma-pl.c   |  48 ++--
 drivers/pci/controller/pcie-xilinx-nwl.c      |  45 ++--
 drivers/pci/controller/pcie-xilinx.c          |  55 +++--
 drivers/pci/controller/plda/Kconfig           |   1 +
 drivers/pci/controller/plda/pcie-plda-host.c  |  44 ++--
 drivers/pci/controller/plda/pcie-plda.h       |   1 -
 drivers/pci/controller/vmd.c                  | 229 +++++++++---------
 23 files changed, 504 insertions(+), 517 deletions(-)
Quoted 5 lines

Looks good to me, thanks! I think Mani will probably pick this up.

I might have included the specific "legacy MSI domain" thing you're replacing. It looks like you're replacing pci_msi_create_irq_domain() with msi_create_parent_irq_domain()?

Yes, pci_msi_create_irq_domain() is legacy. We will delete it once everything is converted.

Quoted 3 lines

Minor merge conflict in pcie-mediatek-gen3.c with dcbea1c7e94e ("PCI:
mediatek-gen3: Use dev_fwnode() for irq_domain_create_linear()"). No problem, we can easily fix that up.

Thanks!

Quoted 1 line

The "++i" in vmd.c stuck out to me since "i++" is so much more common.

I always do "++i", maybe I'm the weird one..

Best regards,
Nam

#56 lore raw
Quoted 52 lines · Nam Cao
Quoted 51 lines · Bjorn Helgaas
Quoted 49 lines · Nam Cao

The initial implementation of PCI/MSI interrupt domains in the hierarchical interrupt domain model used a shortcut by providing a global PCI/MSI domain.

This works because the PCI/MSI[X] hardware is standardized and uniform, but it violates the basic design principle of hierarchical interrupt domains:
Each hardware block involved in the interrupt delivery chain should have a separate interrupt domain.

For PCI/MSI[X], the interrupt controller is per PCI device and not a global made-up entity.

Unsurprisingly, the shortcut turned out to have downsides as it does not allow dynamic allocation of interrupt vectors after initialization and it prevents supporting IMS on PCI. For further details, see:

https://lore.kernel.org/lkml/[email protected]/

The solution is implementing per device MSI domains, this means the entities which provide global PCI/MSI domain so far have to implement MSI parent domain functionality instead.

This series converts the PCI controller drivers to implement MSI parent domain.

 drivers/pci/Kconfig                           |   1 +
 drivers/pci/controller/Kconfig                |  11 +
 drivers/pci/controller/dwc/Kconfig            |   1 +
 .../pci/controller/dwc/pcie-designware-host.c |  68 ++----
 drivers/pci/controller/dwc/pcie-designware.h  |   1 -
 drivers/pci/controller/mobiveil/Kconfig       |   1 +
 .../controller/mobiveil/pcie-mobiveil-host.c  |  42 ++--
 .../pci/controller/mobiveil/pcie-mobiveil.h   |   1 -
 drivers/pci/controller/pci-aardvark.c         |  59 ++---
 drivers/pci/controller/pci-hyperv.c           |  98 ++++++--
 drivers/pci/controller/pcie-altera-msi.c      |  44 ++--
 drivers/pci/controller/pcie-brcmstb.c         |  44 ++--
 drivers/pci/controller/pcie-iproc-msi.c       |  45 ++--
 drivers/pci/controller/pcie-mediatek-gen3.c   |  67 ++---
 drivers/pci/controller/pcie-mediatek.c        |  46 ++--
 drivers/pci/controller/pcie-rcar-host.c       |  69 ++----
 drivers/pci/controller/pcie-xilinx-dma-pl.c   |  48 ++--
 drivers/pci/controller/pcie-xilinx-nwl.c      |  45 ++--
 drivers/pci/controller/pcie-xilinx.c          |  55 +++--
 drivers/pci/controller/plda/Kconfig           |   1 +
 drivers/pci/controller/plda/pcie-plda-host.c  |  44 ++--
 drivers/pci/controller/plda/pcie-plda.h       |   1 -
 drivers/pci/controller/vmd.c                  | 229 +++++++++---------
 23 files changed, 504 insertions(+), 517 deletions(-)
Quoted 9 lines
Quoted 5 lines

Looks good to me, thanks! I think Mani will probably pick this up.

I might have included the specific "legacy MSI domain" thing you're replacing. It looks like you're replacing pci_msi_create_irq_domain() with msi_create_parent_irq_domain()?

Yes, pci_msi_create_irq_domain() is legacy. We will delete it once everything is converted.

I'll amend the commit message to include pci_msi_create_irq_domain().

Quoted 10 lines
Quoted 3 lines

Minor merge conflict in pcie-mediatek-gen3.c with dcbea1c7e94e ("PCI:
mediatek-gen3: Use dev_fwnode() for irq_domain_create_linear()"). No problem, we can easily fix that up.

Thanks!

Quoted 1 line

The "++i" in vmd.c stuck out to me since "i++" is so much more common.

I always do "++i", maybe I'm the weird one..

Just for the sake of uniformity, I'll do 's/++i/i++' while applying.

And based on the discussion on patch 14/16, I'll drop it to be applied through netdev and apply the rest of the series through PCI tree.

- Mani

Signature · 1 line
மணிவண்ணன் சதாசிவம்
maintainer Applied
#57 lore raw
In reply to Nam Cao
Quoted 16 lines · Nam Cao

The initial implementation of PCI/MSI interrupt domains in the hierarchical interrupt domain model used a shortcut by providing a global PCI/MSI domain.

This works because the PCI/MSI[X] hardware is standardized and uniform, but it violates the basic design principle of hierarchical interrupt domains:
Each hardware block involved in the interrupt delivery chain should have a separate interrupt domain.

For PCI/MSI[X], the interrupt controller is per PCI device and not a global made-up entity.

Unsurprisingly, the shortcut turned out to have downsides as it does not allow dynamic allocation of interrupt vectors after initialization and it prevents supporting IMS on PCI. For further details, see:

Quoted 9 lines

https://lore.kernel.org/lkml/[email protected]/

The solution is implementing per device MSI domains, this means the entities which provide global PCI/MSI domain so far have to implement MSI parent domain functionality instead.

This series converts the PCI controller drivers to implement MSI parent domain.

Applied the series except patch 14/16 to pci/controller/msi-parent, thanks!

- Mani

Quoted 28 lines
 drivers/pci/Kconfig                           |   1 +
 drivers/pci/controller/Kconfig                |  11 +
 drivers/pci/controller/dwc/Kconfig            |   1 +
 .../pci/controller/dwc/pcie-designware-host.c |  68 ++----
 drivers/pci/controller/dwc/pcie-designware.h  |   1 -
 drivers/pci/controller/mobiveil/Kconfig       |   1 +
 .../controller/mobiveil/pcie-mobiveil-host.c  |  42 ++--
 .../pci/controller/mobiveil/pcie-mobiveil.h   |   1 -
 drivers/pci/controller/pci-aardvark.c         |  59 ++---
 drivers/pci/controller/pci-hyperv.c           |  98 ++++++--
 drivers/pci/controller/pcie-altera-msi.c      |  44 ++--
 drivers/pci/controller/pcie-brcmstb.c         |  44 ++--
 drivers/pci/controller/pcie-iproc-msi.c       |  45 ++--
 drivers/pci/controller/pcie-mediatek-gen3.c   |  67 ++---
 drivers/pci/controller/pcie-mediatek.c        |  46 ++--
 drivers/pci/controller/pcie-rcar-host.c       |  69 ++----
 drivers/pci/controller/pcie-xilinx-dma-pl.c   |  48 ++--
 drivers/pci/controller/pcie-xilinx-nwl.c      |  45 ++--
 drivers/pci/controller/pcie-xilinx.c          |  55 +++--
 drivers/pci/controller/plda/Kconfig           |   1 +
 drivers/pci/controller/plda/pcie-plda-host.c  |  44 ++--
 drivers/pci/controller/plda/pcie-plda.h       |   1 -
 drivers/pci/controller/vmd.c                  | 229 +++++++++---------
 23 files changed, 504 insertions(+), 517 deletions(-)
Signature · 1 line
2.39.5
Signature · 1 line
மணிவண்ணன் சதாசிவம்