Skip to contentSkip to conversation

genirq, irqchip: Convert ARM MSI handling to per device MSI domains

PATCH Applied Thomas Gleixner 9 participants 94 messages Jun 23, 2024 – Jul 18, 2024

Subsystems (11)
lore
Share & tools
lore
#0 lore raw

v4 cover genirq, irqchip: Convert ARM MSI handling to per device MSI domains

This is version 4 of the series to convert ARM MSI handling over to per device MSI domains. Version 3 can be found here:

https://lore.kernel.org/lkml/[email protected]

The conversion aims to replace the existing platform MSI mechanism and enables ARM to support the future PCI/IMS mechanism.

The infrastructure to replace the platform MSI mechanism is already upstream and in use by RISC-V and has been tested on various ARM platforms during the V2 development.

Changes vs. V3:

- Fix the conversion of the GIC V3 MBI driver - Marc

- Dropped a few stray MSI_FLAG_PCI_MSI_MASK_PARENT flags

- Dropped the trivial cleanup patches as they have been merged

- Picked up tags

The series is only lightly tested due to lack of hardware, so we rely on the people who have access to affected machines to help with testing.

If there are no major objections raised or testing fallout reported, I'm aiming this series for the next merge window.

The series is also available from git:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git devmsi-arm-v4

Thanks,

tglx

---
 b/drivers/base/platform-msi.c                 |  350 --------------------------
 b/drivers/irqchip/Kconfig                     |    8 
 b/drivers/irqchip/Makefile                    |    4 
 b/drivers/irqchip/irq-gic-common.h            |    3 
 b/drivers/irqchip/irq-gic-v2m.c               |   80 +----
 b/drivers/irqchip/irq-gic-v3-its-msi-parent.c |  210 +++++++++++++++
 b/drivers/irqchip/irq-gic-v3-its.c            |    5 
 b/drivers/irqchip/irq-gic-v3-mbi.c            |  130 +++------
 b/drivers/irqchip/irq-imx-mu-msi.c            |   48 +--
 b/drivers/irqchip/irq-mbigen.c                |   96 ++-----
 b/drivers/irqchip/irq-msi-lib.c               |  135 ++++++++++
 b/drivers/irqchip/irq-msi-lib.h               |   27 ++
 b/drivers/irqchip/irq-mvebu-gicp.c            |   44 +--
 b/drivers/irqchip/irq-mvebu-icu.c             |  275 ++++++++------------
 b/drivers/irqchip/irq-mvebu-odmi.c            |   37 +-
 b/drivers/irqchip/irq-mvebu-sei.c             |   52 +--
 b/drivers/pci/msi/irqdomain.c                 |   21 +
 b/include/linux/msi.h                         |   52 ---
 b/kernel/irq/msi.c                            |   95 +------
 drivers/irqchip/irq-gic-v3-its-pci-msi.c      |  202 ---------------
 drivers/irqchip/irq-gic-v3-its-platform-msi.c |  163 ------------
 21 files changed, 738 insertions(+), 1299 deletions(-)
#1 lore raw

PATCH v4 1/21 PCI/MSI: Provide MSI_FLAG_PCI_MSI_MASK_PARENT

From: Shivamurthy Shastri <[email protected]>

Most ARM(64) PCI/MSI domains mask and unmask in the parent domain after or before the PCI mask/unmask operation takes place. So there are more than a dozen of the same wrapper implementation all over the place.

Don't make the same mistake with the new per device PCI/MSI domains and provide a new MSI feature flag, which lets the domain implementation enable this sequence in the PCI/MSI code.

---
v3: new patch to replace the global static key - Marc Zyngier

---
 drivers/pci/msi/irqdomain.c |   21 +++++++++++++++++++++
 include/linux/msi.h         |    2 ++
 2 files changed, 23 insertions(+)
drivers/pci/msi/irqdomain.c +21 -0
@@ -148,17 +148,35 @@ static void pci_device_domain_set_desc(m
 	arg->hwirq = desc->msi_index;
 }
 
+static __always_inline void cond_mask_parent(struct irq_data *data)
+{
+	struct msi_domain_info *info = data->domain->host_data;
+
+	if (unlikely(info->flags & MSI_FLAG_PCI_MSI_MASK_PARENT))
+		irq_chip_mask_parent(data);
+}
+
+static __always_inline void cond_unmask_parent(struct irq_data *data)
+{
+	struct msi_domain_info *info = data->domain->host_data;
+
+	if (unlikely(info->flags & MSI_FLAG_PCI_MSI_MASK_PARENT))
+		irq_chip_unmask_parent(data);
+}
+
 static void pci_irq_mask_msi(struct irq_data *data)
 {
 	struct msi_desc *desc = irq_data_get_msi_desc(data);
 
 	pci_msi_mask(desc, BIT(data->irq - desc->irq));
+	cond_mask_parent(data);
 }
 
 static void pci_irq_unmask_msi(struct irq_data *data)
 {
 	struct msi_desc *desc = irq_data_get_msi_desc(data);
 
+	cond_unmask_parent(data);
 	pci_msi_unmask(desc, BIT(data->irq - desc->irq));
 }
 
@@ -170,6 +188,7 @@ static void pci_irq_unmask_msi(struct ir
 
 #define MSI_COMMON_FLAGS	(MSI_FLAG_FREE_MSI_DESCS |	\
 				 MSI_FLAG_ACTIVATE_EARLY |	\
+				 MSI_FLAG_PCI_MSI_MASK_PARENT |	\
 				 MSI_FLAG_DEV_SYSFS |		\
 				 MSI_REACTIVATE)
 
@@ -195,10 +214,12 @@ static const struct msi_domain_template
 static void pci_irq_mask_msix(struct irq_data *data)
 {
 	pci_msix_mask(irq_data_get_msi_desc(data));
+	cond_mask_parent(data);
 }
 
 static void pci_irq_unmask_msix(struct irq_data *data)
 {
+	cond_unmask_parent(data);
 	pci_msix_unmask(irq_data_get_msi_desc(data));
 }
 
include/linux/msi.h +2 -0
@@ -556,6 +556,8 @@ enum {
 	MSI_FLAG_USE_DEV_FWNODE		= (1 << 7),
 	/* Set parent->dev into domain->pm_dev on device domain creation */
 	MSI_FLAG_PARENT_PM_DEV		= (1 << 8),
+	/* Support for parent mask/unmask */
+	MSI_FLAG_PCI_MSI_MASK_PARENT	= (1 << 9),
 
 	/* Mask for the generic functionality */
 	MSI_GENERIC_FLAGS_MASK		= GENMASK(15, 0),
#2 lore raw

PATCH v4 1/21 PCI/MSI: Provide MSI_FLAG_PCI_MSI_MASK_PARENT

From: Shivamurthy Shastri <[email protected]>

Most ARM(64) PCI/MSI domains mask and unmask in the parent domain after or before the PCI mask/unmask operation takes place. So there are more than a dozen of the same wrapper implementation all over the place.

Don't make the same mistake with the new per device PCI/MSI domains and provide a new MSI feature flag, which lets the domain implementation enable this sequence in the PCI/MSI code.

---
v4-1: Remove MSI_FLAG_PCI_MSI_MASK_PARENT from the defaults - Rob
v3: new patch to replace the global static key - Marc Zyngier

---
 drivers/pci/msi/irqdomain.c |   20 ++++++++++++++++++++
 include/linux/msi.h         |    2 ++
 2 files changed, 22 insertions(+)
drivers/pci/msi/irqdomain.c +20 -0
@@ -148,17 +148,35 @@ static void pci_device_domain_set_desc(m
 	arg->hwirq = desc->msi_index;
 }
 
+static __always_inline void cond_mask_parent(struct irq_data *data)
+{
+	struct msi_domain_info *info = data->domain->host_data;
+
+	if (unlikely(info->flags & MSI_FLAG_PCI_MSI_MASK_PARENT))
+		irq_chip_mask_parent(data);
+}
+
+static __always_inline void cond_unmask_parent(struct irq_data *data)
+{
+	struct msi_domain_info *info = data->domain->host_data;
+
+	if (unlikely(info->flags & MSI_FLAG_PCI_MSI_MASK_PARENT))
+		irq_chip_unmask_parent(data);
+}
+
 static void pci_irq_mask_msi(struct irq_data *data)
 {
 	struct msi_desc *desc = irq_data_get_msi_desc(data);
 
 	pci_msi_mask(desc, BIT(data->irq - desc->irq));
+	cond_mask_parent(data);
 }
 
 static void pci_irq_unmask_msi(struct irq_data *data)
 {
 	struct msi_desc *desc = irq_data_get_msi_desc(data);
 
+	cond_unmask_parent(data);
 	pci_msi_unmask(desc, BIT(data->irq - desc->irq));
 }
 
@@ -195,10 +213,12 @@ static const struct msi_domain_template
 static void pci_irq_mask_msix(struct irq_data *data)
 {
 	pci_msix_mask(irq_data_get_msi_desc(data));
+	cond_mask_parent(data);
 }
 
 static void pci_irq_unmask_msix(struct irq_data *data)
 {
+	cond_unmask_parent(data);
 	pci_msix_unmask(irq_data_get_msi_desc(data));
 }
 
include/linux/msi.h +2 -0
@@ -556,6 +556,8 @@ enum {
 	MSI_FLAG_USE_DEV_FWNODE		= (1 << 7),
 	/* Set parent->dev into domain->pm_dev on device domain creation */
 	MSI_FLAG_PARENT_PM_DEV		= (1 << 8),
+	/* Support for parent mask/unmask */
+	MSI_FLAG_PCI_MSI_MASK_PARENT	= (1 << 9),
 
 	/* Mask for the generic functionality */
 	MSI_GENERIC_FLAGS_MASK		= GENMASK(15, 0),
#3 lore raw

[tip: irq/core] PCI/MSI: Provide MSI_FLAG_PCI_MSI_MASK_PARENT

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     698c86faa99ae60a19aacbb74ec1df243bc5eba6
Gitweb:        https://git.kernel.org/tip/698c86faa99ae60a19aacbb74ec1df243bc5eba6
Author:        Shivamurthy Shastri <[email protected]>
AuthorDate:    Wed, 26 Jun 2024 21:05:12 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:23 +02:00

PCI/MSI: Provide MSI_FLAG_PCI_MSI_MASK_PARENT

Most ARM(64) PCI/MSI domains mask and unmask in the parent domain after or before the PCI mask/unmask operation takes place. So there are more than a dozen of the same wrapper implementation all over the place.

Don't make the same mistake with the new per device PCI/MSI domains and provide a new MSI feature flag, which lets the domain implementation enable this sequence in the PCI/MSI code.

---
 drivers/pci/msi/irqdomain.c | 20 ++++++++++++++++++++
 include/linux/msi.h         |  2 ++
 2 files changed, 22 insertions(+)
drivers/pci/msi/irqdomain.c +20 -0
index 03d2dd2..5691257 100644
@@ -148,17 +148,35 @@ static void pci_device_domain_set_desc(msi_alloc_info_t *arg, struct msi_desc *d
 	arg->hwirq = desc->msi_index;
 }
 
+static __always_inline void cond_mask_parent(struct irq_data *data)
+{
+	struct msi_domain_info *info = data->domain->host_data;
+
+	if (unlikely(info->flags & MSI_FLAG_PCI_MSI_MASK_PARENT))
+		irq_chip_mask_parent(data);
+}
+
+static __always_inline void cond_unmask_parent(struct irq_data *data)
+{
+	struct msi_domain_info *info = data->domain->host_data;
+
+	if (unlikely(info->flags & MSI_FLAG_PCI_MSI_MASK_PARENT))
+		irq_chip_unmask_parent(data);
+}
+
 static void pci_irq_mask_msi(struct irq_data *data)
 {
 	struct msi_desc *desc = irq_data_get_msi_desc(data);
 
 	pci_msi_mask(desc, BIT(data->irq - desc->irq));
+	cond_mask_parent(data);
 }
 
 static void pci_irq_unmask_msi(struct irq_data *data)
 {
 	struct msi_desc *desc = irq_data_get_msi_desc(data);
 
+	cond_unmask_parent(data);
 	pci_msi_unmask(desc, BIT(data->irq - desc->irq));
 }
 
@@ -195,10 +213,12 @@ static const struct msi_domain_template pci_msi_template = {
 static void pci_irq_mask_msix(struct irq_data *data)
 {
 	pci_msix_mask(irq_data_get_msi_desc(data));
+	cond_mask_parent(data);
 }
 
 static void pci_irq_unmask_msix(struct irq_data *data)
 {
+	cond_unmask_parent(data);
 	pci_msix_unmask(irq_data_get_msi_desc(data));
 }
 
include/linux/msi.h +2 -0
index dc27cf3..04f33e7 100644
@@ -556,6 +556,8 @@ enum {
 	MSI_FLAG_USE_DEV_FWNODE		= (1 << 7),
 	/* Set parent->dev into domain->pm_dev on device domain creation */
 	MSI_FLAG_PARENT_PM_DEV		= (1 << 8),
+	/* Support for parent mask/unmask */
+	MSI_FLAG_PCI_MSI_MASK_PARENT	= (1 << 9),
 
 	/* Mask for the generic functionality */
 	MSI_GENERIC_FLAGS_MASK		= GENMASK(15, 0),
#4 lore raw

[tip: irq/msi] PCI/MSI: Provide MSI_FLAG_PCI_MSI_MASK_PARENT

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     7d189c77106ed6df09829f7a419e35ada67b2bd0
Gitweb:        https://git.kernel.org/tip/7d189c77106ed6df09829f7a419e35ada67b2bd0
Author:        Shivamurthy Shastri <[email protected]>
AuthorDate:    Wed, 26 Jun 2024 21:05:12 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:19 +02:00

PCI/MSI: Provide MSI_FLAG_PCI_MSI_MASK_PARENT

Most ARM(64) PCI/MSI domains mask and unmask in the parent domain after or before the PCI mask/unmask operation takes place. So there are more than a dozen of the same wrapper implementation all over the place.

Don't make the same mistake with the new per device PCI/MSI domains and provide a new MSI feature flag, which lets the domain implementation enable this sequence in the PCI/MSI code.

---
 drivers/pci/msi/irqdomain.c | 20 ++++++++++++++++++++
 include/linux/msi.h         |  2 ++
 2 files changed, 22 insertions(+)
drivers/pci/msi/irqdomain.c +20 -0
index 03d2dd2..5691257 100644
@@ -148,17 +148,35 @@ static void pci_device_domain_set_desc(msi_alloc_info_t *arg, struct msi_desc *d
 	arg->hwirq = desc->msi_index;
 }
 
+static __always_inline void cond_mask_parent(struct irq_data *data)
+{
+	struct msi_domain_info *info = data->domain->host_data;
+
+	if (unlikely(info->flags & MSI_FLAG_PCI_MSI_MASK_PARENT))
+		irq_chip_mask_parent(data);
+}
+
+static __always_inline void cond_unmask_parent(struct irq_data *data)
+{
+	struct msi_domain_info *info = data->domain->host_data;
+
+	if (unlikely(info->flags & MSI_FLAG_PCI_MSI_MASK_PARENT))
+		irq_chip_unmask_parent(data);
+}
+
 static void pci_irq_mask_msi(struct irq_data *data)
 {
 	struct msi_desc *desc = irq_data_get_msi_desc(data);
 
 	pci_msi_mask(desc, BIT(data->irq - desc->irq));
+	cond_mask_parent(data);
 }
 
 static void pci_irq_unmask_msi(struct irq_data *data)
 {
 	struct msi_desc *desc = irq_data_get_msi_desc(data);
 
+	cond_unmask_parent(data);
 	pci_msi_unmask(desc, BIT(data->irq - desc->irq));
 }
 
@@ -195,10 +213,12 @@ static const struct msi_domain_template pci_msi_template = {
 static void pci_irq_mask_msix(struct irq_data *data)
 {
 	pci_msix_mask(irq_data_get_msi_desc(data));
+	cond_mask_parent(data);
 }
 
 static void pci_irq_unmask_msix(struct irq_data *data)
 {
+	cond_unmask_parent(data);
 	pci_msix_unmask(irq_data_get_msi_desc(data));
 }
 
include/linux/msi.h +2 -0
index dc27cf3..04f33e7 100644
@@ -556,6 +556,8 @@ enum {
 	MSI_FLAG_USE_DEV_FWNODE		= (1 << 7),
 	/* Set parent->dev into domain->pm_dev on device domain creation */
 	MSI_FLAG_PARENT_PM_DEV		= (1 << 8),
+	/* Support for parent mask/unmask */
+	MSI_FLAG_PCI_MSI_MASK_PARENT	= (1 << 9),
 
 	/* Mask for the generic functionality */
 	MSI_GENERIC_FLAGS_MASK		= GENMASK(15, 0),
#5 lore raw

PATCH v4 2/21 irqchip: Provide irq-msi-lib

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

All irqdomains which provide MSI parent domain functionality for per device MSI domains need to provide a select() callback for the irqdomain and a function to initialize the child domain.

Most of these functions would just be copy&paste with minimal modifications, so provide a library function which implements the required functionality and is customizable via parent_domain::msi_parent_ops. The check for the supported bus tokens in msi_lib_init_dev_msi_info() is expanded step by step within the next patches.

---
v3: renamed file and functions as the library is independent of gic - Frank Li

---
 drivers/irqchip/Kconfig       |   3 +
 drivers/irqchip/Makefile      |   1 +
 drivers/irqchip/irq-msi-lib.c | 112 ++++++++++++++++++++++++++++++++++
 drivers/irqchip/irq-msi-lib.h |  19 ++++++
 4 files changed, 135 insertions(+)
 create mode 100644 drivers/irqchip/irq-msi-lib.c
 create mode 100644 drivers/irqchip/irq-msi-lib.h
drivers/irqchip/Kconfig +3 -0
index 14464716bacb..2bf8d940504c 100644
@@ -74,6 +74,9 @@ config ARM_VIC_NR
 	  The maximum number of VICs available in the system, for
 	  power management.
 
+config IRQ_MSI_LIB
+	bool
+
 config ARMADA_370_XP_IRQ
 	bool
 	select GENERIC_IRQ_CHIP
drivers/irqchip/Makefile +1 -0
index d9dc3d99aaa8..72c7f6289411 100644
@@ -29,6 +29,7 @@ obj-$(CONFIG_ARCH_SPEAR3XX)		+= spear-shirq.o
 obj-$(CONFIG_ARM_GIC)			+= irq-gic.o irq-gic-common.o
 obj-$(CONFIG_ARM_GIC_PM)		+= irq-gic-pm.o
 obj-$(CONFIG_ARCH_REALVIEW)		+= irq-gic-realview.o
+obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
 obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o
drivers/irqchip/irq-msi-lib.c new file +112 -0

112 lines not on this page · raw message

drivers/irqchip/irq-msi-lib.h new file +19 -0
new file mode 100644
index 000000000000..f0706cc28264
@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: GPL-2.0-only
+// Copyright (C) 2022 Linutronix GmbH
+// Copyright (C) 2022 Intel
+
+#ifndef _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H
+#define _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H
+
+#include <linux/bits.h>
+#include <linux/irqdomain.h>
+#include <linux/msi.h>
+
+int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
+			      enum irq_domain_bus_token bus_token);
+
+bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+			       struct irq_domain *real_parent,
+			       struct msi_domain_info *info);
+
+#endif /* _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H */
Signature · 1 line
2.34.1
#6 lore raw

Re: [patch V4 02/21] irqchip: Provide irq-msi-lib

On Sun, Jun 23, 2024 at 05:18:34PM +0200, Thomas Gleixner wrote:

[...]

Quoted 42 lines
drivers/irqchip/irq-msi-lib.c new file +36 -0
new file mode 100644
index 000000000000..acbccf8f7f5b
@@ -0,0 +1,112 @@
+// SPDX-License-Identifier: GPL-2.0-only
+// Copyright (C) 2022 Linutronix GmbH
+// Copyright (C) 2022 Intel
+
+#include <linux/export.h>
+
+#include "irq-msi-lib.h"
+
+/**
+ * msi_lib_init_dev_msi_info - Domain info setup for MSI domains
+ * @dev:		The device for which the domain is created for
+ * @domain:		The domain providing this callback
+ * @real_parent:	The real parent domain of the domain to be initialized
+ *			which might be a domain built on top of @domain or
+ *			@domain itself
+ * @info:		The domain info for the domain to be initialize
+ *
+ * This function is to be used for all types of MSI domains above the root
+ * parent domain and any intermediates. The topmost parent domain specific
+ * functionality is determined via @real_parent.
+ *
+ * All intermediate domains between the root and the device domain must
+ * have either msi_parent_ops.init_dev_msi_info = msi_parent_init_dev_msi_info
+ * or invoke it down the line.
+ */
+bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+			       struct irq_domain *real_parent,
+			       struct msi_domain_info *info)
+{
+	const struct msi_parent_ops *pops = real_parent->msi_parent_ops;
+
+	/*
+	 * MSI parent domain specific settings. For now there is only the
+	 * root parent domain, e.g. NEXUS, acting as a MSI parent, but it is
+	 * possible to stack MSI parents. See x86 vector -> irq remapping
+	 */

drivers/irqchip/irq-msi-lib.c line 46

Quoted 10 lines
drivers/irqchip/irq-msi-lib.c new file +10 -0
+	if (domain->bus_token == pops->bus_select_token) {
+		if (WARN_ON_ONCE(domain != real_parent))
+			return false;
+	} else {
+		WARN_ON_ONCE(1);
+		return false;
+	}
+
+	/* Parent ops available? */
+	if (WARN_ON_ONCE(!pops))

We have already dereferenced pops above, we should move this warning before we dereference it (ie checked devmsi-arm-v4-2 too - branch same comment applies there too).

Thanks
Lorenzo

Quoted 95 lines
drivers/irqchip/irq-msi-lib.c new file +66 -0
+		return false;
+
+	/* Is the target domain bus token supported? */
+	switch(info->bus_token) {
+	default:
+		/*
+		 * This should never be reached. See
+		 * msi_lib_irq_domain_select()
+		 */
+		WARN_ON_ONCE(1);
+		return false;
+	}
+
+	/*
+	 * Mask out the domain specific MSI feature flags which are not
+	 * supported by the real parent.
+	 */
+	info->flags			&= pops->supported_flags;
+	/* Enforce the required flags */
+	info->flags			|= pops->required_flags;
+
+	/* Chip updates for all child bus types */
+	if (!info->chip->irq_eoi)
+		info->chip->irq_eoi	= irq_chip_eoi_parent;
+
+	/*
+	 * The device MSI domain can never have a set affinity callback. It
+	 * always has to rely on the parent domain to handle affinity
+	 * settings. The device MSI domain just has to write the resulting
+	 * MSI message into the hardware which is the whole purpose of the
+	 * device MSI domain aside of mask/unmask which is provided e.g. by
+	 * PCI/MSI device domains.
+	 */
+	info->chip->irq_set_affinity	= msi_domain_set_affinity;
+	return true;
+}
+EXPORT_SYMBOL_GPL(msi_lib_init_dev_msi_info);
+
+/**
+ * msi_lib_irq_domain_select - Shared select function for NEXUS domains
+ * @d:		Pointer to the irq domain on which select is invoked
+ * @fwspec:	Firmware spec describing what is searched
+ * @bus_token:	The bus token for which a matching irq domain is looked up
+ *
+ * Returns:	%0 if @d is not what is being looked for
+ *
+ *		%1 if @d is either the domain which is directly searched for or
+ *		   if @d is providing the parent MSI domain for the functionality
+ *			 requested with @bus_token.
+ */
+int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
+			      enum irq_domain_bus_token bus_token)
+{
+	const struct msi_parent_ops *ops = d->msi_parent_ops;
+	u32 busmask = BIT(bus_token);
+
+	if (fwspec->fwnode != d->fwnode || fwspec->param_count != 0)
+		return 0;
+
+	/* Handle pure domain searches */
+	if (bus_token == ops->bus_select_token)
+		return 1;
+
+	return ops && !!(ops->bus_select_mask & busmask);
+}
+EXPORT_SYMBOL_GPL(msi_lib_irq_domain_select);
drivers/irqchip/irq-msi-lib.h new file +19 -0
new file mode 100644
index 000000000000..f0706cc28264
@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: GPL-2.0-only
+// Copyright (C) 2022 Linutronix GmbH
+// Copyright (C) 2022 Intel
+
+#ifndef _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H
+#define _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H
+
+#include <linux/bits.h>
+#include <linux/irqdomain.h>
+#include <linux/msi.h>
+
+int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
+			      enum irq_domain_bus_token bus_token);
+
+bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+			       struct irq_domain *real_parent,
+			       struct msi_domain_info *info);
+
+#endif /* _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H */
Signature · 1 line
2.34.1
#7 lore raw

Re: [patch V4 02/21] irqchip: Provide irq-msi-lib

Quoted 18 lines · Lorenzo Pieralisi
Quoted 17 lines · Thomas Gleixner
drivers/irqchip/irq-msi-lib.c new file +17 -0
+bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+			       struct irq_domain *real_parent,
+			       struct msi_domain_info *info)
+{
+	const struct msi_parent_ops *pops = real_parent->msi_parent_ops;
+
+	/*
+	 * MSI parent domain specific settings. For now there is only the
+	 * root parent domain, e.g. NEXUS, acting as a MSI parent, but it is
+	 * possible to stack MSI parents. See x86 vector -> irq remapping
+	 */
+	if (domain->bus_token == pops->bus_select_token) {
+		if (WARN_ON_ONCE(domain != real_parent))
+			return false;
+	} else {
+		WARN_ON_ONCE(1);
+		return false;
Quoted 8 lines

drivers/irqchip/irq-msi-lib.c line 46

Quoted 4 lines
drivers/irqchip/irq-msi-lib.c new file +4 -0
+	}
+
+	/* Parent ops available? */
+	if (WARN_ON_ONCE(!pops))

We have already dereferenced pops above, we should move this warning before we dereference it (ie checked devmsi-arm-v4-2 too - branch same comment applies there too).

Oops. Yes.

#8 lore raw

[tip: irq/core] irqchip: Provide irq-msi-lib

In reply to Thomas Gleixner
The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     64e4f92275a52e1c1b89a556a45807fdbfd4b28c
Gitweb:        https://git.kernel.org/tip/64e4f92275a52e1c1b89a556a45807fdbfd4b28c
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:34 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:23 +02:00

irqchip: Provide irq-msi-lib

All irqdomains which provide MSI parent domain functionality for per device MSI domains need to provide a select() callback for the irqdomain and a function to initialize the child domain.

Most of these functions would just be copy&paste with minimal modifications, so provide a library function which implements the required functionality and is customizable via parent_domain::msi_parent_ops. The check for the supported bus tokens in msi_lib_init_dev_msi_info() is expanded step by step within the next patches.

---
 drivers/irqchip/Kconfig       |   3 +-
 drivers/irqchip/Makefile      |   1 +-
 drivers/irqchip/irq-msi-lib.c | 112 +++++++++++++++++++++++++++++++++-
 drivers/irqchip/irq-msi-lib.h |  19 ++++++-
 4 files changed, 135 insertions(+)
 create mode 100644 drivers/irqchip/irq-msi-lib.c
 create mode 100644 drivers/irqchip/irq-msi-lib.h
drivers/irqchip/Kconfig +3 -0
index 344c484..3e3cbe9 100644
@@ -74,6 +74,9 @@ config ARM_VIC_NR
 	  The maximum number of VICs available in the system, for
 	  power management.
 
+config IRQ_MSI_LIB
+	bool
+
 config ARMADA_370_XP_IRQ
 	bool
 	select GENERIC_IRQ_CHIP
drivers/irqchip/Makefile +1 -0
index 2df72b7..f5e0fa9 100644
@@ -29,6 +29,7 @@ obj-$(CONFIG_ARCH_SPEAR3XX)		+= spear-shirq.o
 obj-$(CONFIG_ARM_GIC)			+= irq-gic.o irq-gic-common.o
 obj-$(CONFIG_ARM_GIC_PM)		+= irq-gic-pm.o
 obj-$(CONFIG_ARCH_REALVIEW)		+= irq-gic-realview.o
+obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
 obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o
drivers/irqchip/irq-msi-lib.c new file +112 -0

112 lines not on this page · raw message

drivers/irqchip/irq-msi-lib.h new file +19 -0
new file mode 100644
index 0000000..f0706cc
@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: GPL-2.0-only
+// Copyright (C) 2022 Linutronix GmbH
+// Copyright (C) 2022 Intel
+
+#ifndef _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H
+#define _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H
+
+#include <linux/bits.h>
+#include <linux/irqdomain.h>
+#include <linux/msi.h>
+
+int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
+			      enum irq_domain_bus_token bus_token);
+
+bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+			       struct irq_domain *real_parent,
+			       struct msi_domain_info *info);
+
+#endif /* _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H */
#9 lore raw

[tip: irq/msi] irqchip: Provide irq-msi-lib

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     72e257c6f058032daba1c4fe0c81003d545d0f81
Gitweb:        https://git.kernel.org/tip/72e257c6f058032daba1c4fe0c81003d545d0f81
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:34 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:19 +02:00

irqchip: Provide irq-msi-lib

All irqdomains which provide MSI parent domain functionality for per device MSI domains need to provide a select() callback for the irqdomain and a function to initialize the child domain.

Most of these functions would just be copy&paste with minimal modifications, so provide a library function which implements the required functionality and is customizable via parent_domain::msi_parent_ops. The check for the supported bus tokens in msi_lib_init_dev_msi_info() is expanded step by step within the next patches.

---
 drivers/irqchip/Kconfig       |   3 +-
 drivers/irqchip/Makefile      |   1 +-
 drivers/irqchip/irq-msi-lib.c | 112 +++++++++++++++++++++++++++++++++-
 drivers/irqchip/irq-msi-lib.h |  19 ++++++-
 4 files changed, 135 insertions(+)
 create mode 100644 drivers/irqchip/irq-msi-lib.c
 create mode 100644 drivers/irqchip/irq-msi-lib.h
drivers/irqchip/Kconfig +3 -0
index 1446471..2bf8d94 100644
@@ -74,6 +74,9 @@ config ARM_VIC_NR
 	  The maximum number of VICs available in the system, for
 	  power management.
 
+config IRQ_MSI_LIB
+	bool
+
 config ARMADA_370_XP_IRQ
 	bool
 	select GENERIC_IRQ_CHIP
drivers/irqchip/Makefile +1 -0
index d9dc3d9..72c7f62 100644
@@ -29,6 +29,7 @@ obj-$(CONFIG_ARCH_SPEAR3XX)		+= spear-shirq.o
 obj-$(CONFIG_ARM_GIC)			+= irq-gic.o irq-gic-common.o
 obj-$(CONFIG_ARM_GIC_PM)		+= irq-gic-pm.o
 obj-$(CONFIG_ARCH_REALVIEW)		+= irq-gic-realview.o
+obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
 obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o
drivers/irqchip/irq-msi-lib.c new file +112 -0

112 lines not on this page · raw message

drivers/irqchip/irq-msi-lib.h new file +19 -0
new file mode 100644
index 0000000..f0706cc
@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: GPL-2.0-only
+// Copyright (C) 2022 Linutronix GmbH
+// Copyright (C) 2022 Intel
+
+#ifndef _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H
+#define _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H
+
+#include <linux/bits.h>
+#include <linux/irqdomain.h>
+#include <linux/msi.h>
+
+int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
+			      enum irq_domain_bus_token bus_token);
+
+bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+			       struct irq_domain *real_parent,
+			       struct msi_domain_info *info);
+
+#endif /* _DRIVERS_IRQCHIP_IRQ_MSI_LIB_H */
#10 lore raw

PATCH v4 3/21 irqchip/gic-v3-its: Provide MSI parent infrastructure

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

To support per device MSI domains the ITS must provide MSI parent domain functionality.

Provide the basic skeleton for this:

- msi_parent_ops
- child domain init callback
- the MSI parent flag set in irqdomain::flags

This does not make ITS a functional parent domain as there is no bit set in the bus_select_mask yet, but it provides the base to implement PCI and platform MSI support gradually on top.

---
v3: enabled MSI_FLAG_PCI_MSI_MASK_PARENT in msi_parent_ops::supported_flags

---
 drivers/irqchip/Kconfig                     |  1 +
 drivers/irqchip/Makefile                    |  2 +-
 drivers/irqchip/irq-gic-common.h            |  3 ++
 drivers/irqchip/irq-gic-v3-its-msi-parent.c | 31 +++++++++++++++++++++
 drivers/irqchip/irq-gic-v3-its.c            |  5 ++++
 5 files changed, 41 insertions(+), 1 deletion(-)
 create mode 100644 drivers/irqchip/irq-gic-v3-its-msi-parent.c
drivers/irqchip/Kconfig +1 -0
index 2bf8d940504c..b51863fa9b38 100644
@@ -41,6 +41,7 @@ config ARM_GIC_V3
 config ARM_GIC_V3_ITS
 	bool
 	select GENERIC_MSI_IRQ
+	select IRQ_MSI_LIB
 	default ARM_GIC_V3
 
 config ARM_GIC_V3_ITS_PCI
drivers/irqchip/Makefile +1 -1
index 72c7f6289411..6e4f7715206d 100644
@@ -32,7 +32,7 @@ obj-$(CONFIG_ARCH_REALVIEW)		+= irq-gic-realview.o
 obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
-obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o
+obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_PCI)	+= irq-gic-v3-its-pci-msi.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_FSL_MC)	+= irq-gic-v3-its-fsl-mc-msi.o
 obj-$(CONFIG_PARTITION_PERCPU)		+= irq-partition-percpu.o
drivers/irqchip/irq-gic-common.h +3 -0
index f407cce9ecaa..eb4a220dd6ad 100644
@@ -8,6 +8,7 @@
 
 #include <linux/of.h>
 #include <linux/irqdomain.h>
+#include <linux/msi.h>
 #include <linux/irqchip/arm-gic-common.h>
 
 struct gic_quirk {
@@ -29,6 +30,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
 void gic_enable_of_quirks(const struct device_node *np,
 			  const struct gic_quirk *quirks, void *data);
 
+extern const struct msi_parent_ops gic_v3_its_msi_parent_ops;
+
 #define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING    (1 << 0)
 #define RDIST_FLAGS_RD_TABLES_PREALLOCATED     (1 << 1)
 #define RDIST_FLAGS_FORCE_NON_SHAREABLE        (1 << 2)
drivers/irqchip/irq-gic-v3-its-msi-parent.c new file +31 -0
new file mode 100644
index 000000000000..cdc0844229b5
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0-only
+// Copyright (C) 2022 Linutronix GmbH
+// Copyright (C) 2022 Intel
+
+#include "irq-gic-common.h"
+#include "irq-msi-lib.h"
+
+#define ITS_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
+				 MSI_FLAG_USE_DEF_CHIP_OPS)
+
+#define ITS_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK |	\
+				 MSI_FLAG_PCI_MSIX      |	\
+				 MSI_FLAG_MULTI_PCI_MSI |	\
+				 MSI_FLAG_PCI_MSI_MASK_PARENT)
+
+static bool its_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+				  struct irq_domain *real_parent, struct msi_domain_info *info)
+{
+	if (!msi_lib_init_dev_msi_info(dev, domain, real_parent, info))
+		return false;
+
+	return true;
+}
+
+const struct msi_parent_ops gic_v3_its_msi_parent_ops = {
+	.supported_flags	= ITS_MSI_FLAGS_SUPPORTED,
+	.required_flags		= ITS_MSI_FLAGS_REQUIRED,
+	.bus_select_token	= DOMAIN_BUS_NEXUS,
+	.prefix			= "ITS-",
+	.init_dev_msi_info	= its_init_dev_msi_info,
+};
drivers/irqchip/irq-gic-v3-its.c +5 -0
index 3c755d5dad6e..d770d6aedb29 100644
@@ -38,6 +38,7 @@
 #include <asm/exception.h>
 
 #include "irq-gic-common.h"
+#include "irq-msi-lib.h"
 
 #define ITS_FLAGS_CMDQ_NEEDS_FLUSHING		(1ULL << 0)
 #define ITS_FLAGS_WORKAROUND_CAVIUM_22375	(1ULL << 1)
@@ -3688,6 +3689,7 @@ static void its_irq_domain_free(struct irq_domain *domain, unsigned int virq,
 }
 
 static const struct irq_domain_ops its_domain_ops = {
+	.select			= msi_lib_irq_domain_select,
 	.alloc			= its_irq_domain_alloc,
 	.free			= its_irq_domain_free,
 	.activate		= its_irq_domain_activate,
@@ -4993,6 +4995,9 @@ static int its_init_domain(struct its_node *its)
 
 	irq_domain_update_bus_token(inner_domain, DOMAIN_BUS_NEXUS);
 
+	inner_domain->msi_parent_ops = &gic_v3_its_msi_parent_ops;
+	inner_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+
 	return 0;
 }
 
Signature · 1 line
2.34.1
#11 lore raw

[tip: irq/core] irqchip/gic-v3-its: Provide MSI parent infrastructure

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     e3baf5e45373c788bc7e0d899d1563f2acdb0fa8
Gitweb:        https://git.kernel.org/tip/e3baf5e45373c788bc7e0d899d1563f2acdb0fa8
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:36 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:23 +02:00

irqchip/gic-v3-its: Provide MSI parent infrastructure

To support per device MSI domains the ITS must provide MSI parent domain functionality.

Provide the basic skeleton for this:

- msi_parent_ops
- child domain init callback
- the MSI parent flag set in irqdomain::flags

This does not make ITS a functional parent domain as there is no bit set in the bus_select_mask yet, but it provides the base to implement PCI and platform MSI support gradually on top.

---
 drivers/irqchip/Kconfig                     |  1 +-
 drivers/irqchip/Makefile                    |  2 +-
 drivers/irqchip/irq-gic-common.h            |  3 ++-
 drivers/irqchip/irq-gic-v3-its-msi-parent.c | 31 ++++++++++++++++++++-
 drivers/irqchip/irq-gic-v3-its.c            |  5 +++-
 5 files changed, 41 insertions(+), 1 deletion(-)
 create mode 100644 drivers/irqchip/irq-gic-v3-its-msi-parent.c
drivers/irqchip/Kconfig +1 -0
index 3e3cbe9..ee78716 100644
@@ -41,6 +41,7 @@ config ARM_GIC_V3
 config ARM_GIC_V3_ITS
 	bool
 	select GENERIC_MSI_IRQ
+	select IRQ_MSI_LIB
 	default ARM_GIC_V3
 
 config ARM_GIC_V3_ITS_PCI
drivers/irqchip/Makefile +1 -1
index f5e0fa9..def6fea 100644
@@ -32,7 +32,7 @@ obj-$(CONFIG_ARCH_REALVIEW)		+= irq-gic-realview.o
 obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
-obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o
+obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_PCI)	+= irq-gic-v3-its-pci-msi.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_FSL_MC)	+= irq-gic-v3-its-fsl-mc-msi.o
 obj-$(CONFIG_PARTITION_PERCPU)		+= irq-partition-percpu.o
drivers/irqchip/irq-gic-common.h +3 -0
index f407cce..eb4a220 100644
@@ -8,6 +8,7 @@
 
 #include <linux/of.h>
 #include <linux/irqdomain.h>
+#include <linux/msi.h>
 #include <linux/irqchip/arm-gic-common.h>
 
 struct gic_quirk {
@@ -29,6 +30,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
 void gic_enable_of_quirks(const struct device_node *np,
 			  const struct gic_quirk *quirks, void *data);
 
+extern const struct msi_parent_ops gic_v3_its_msi_parent_ops;
+
 #define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING    (1 << 0)
 #define RDIST_FLAGS_RD_TABLES_PREALLOCATED     (1 << 1)
 #define RDIST_FLAGS_FORCE_NON_SHAREABLE        (1 << 2)
drivers/irqchip/irq-gic-v3-its-msi-parent.c new file +31 -0
new file mode 100644
index 0000000..cdc0844
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0-only
+// Copyright (C) 2022 Linutronix GmbH
+// Copyright (C) 2022 Intel
+
+#include "irq-gic-common.h"
+#include "irq-msi-lib.h"
+
+#define ITS_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
+				 MSI_FLAG_USE_DEF_CHIP_OPS)
+
+#define ITS_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK |	\
+				 MSI_FLAG_PCI_MSIX      |	\
+				 MSI_FLAG_MULTI_PCI_MSI |	\
+				 MSI_FLAG_PCI_MSI_MASK_PARENT)
+
+static bool its_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+				  struct irq_domain *real_parent, struct msi_domain_info *info)
+{
+	if (!msi_lib_init_dev_msi_info(dev, domain, real_parent, info))
+		return false;
+
+	return true;
+}
+
+const struct msi_parent_ops gic_v3_its_msi_parent_ops = {
+	.supported_flags	= ITS_MSI_FLAGS_SUPPORTED,
+	.required_flags		= ITS_MSI_FLAGS_REQUIRED,
+	.bus_select_token	= DOMAIN_BUS_NEXUS,
+	.prefix			= "ITS-",
+	.init_dev_msi_info	= its_init_dev_msi_info,
+};
drivers/irqchip/irq-gic-v3-its.c +5 -0
index af5297e..047e566 100644
@@ -38,6 +38,7 @@
 #include <asm/exception.h>
 
 #include "irq-gic-common.h"
+#include "irq-msi-lib.h"
 
 #define ITS_FLAGS_CMDQ_NEEDS_FLUSHING		(1ULL << 0)
 #define ITS_FLAGS_WORKAROUND_CAVIUM_22375	(1ULL << 1)
@@ -3708,6 +3709,7 @@ static void its_irq_domain_free(struct irq_domain *domain, unsigned int virq,
 }
 
 static const struct irq_domain_ops its_domain_ops = {
+	.select			= msi_lib_irq_domain_select,
 	.alloc			= its_irq_domain_alloc,
 	.free			= its_irq_domain_free,
 	.activate		= its_irq_domain_activate,
@@ -5013,6 +5015,9 @@ static int its_init_domain(struct its_node *its)
 
 	irq_domain_update_bus_token(inner_domain, DOMAIN_BUS_NEXUS);
 
+	inner_domain->msi_parent_ops = &gic_v3_its_msi_parent_ops;
+	inner_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+
 	return 0;
 }
 
#12 lore raw

[tip: irq/msi] irqchip/gic-v3-its: Provide MSI parent infrastructure

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     48f71d56e2b87839052d2a2ec32fc97a79c3e264
Gitweb:        https://git.kernel.org/tip/48f71d56e2b87839052d2a2ec32fc97a79c3e264
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:36 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:19 +02:00

irqchip/gic-v3-its: Provide MSI parent infrastructure

To support per device MSI domains the ITS must provide MSI parent domain functionality.

Provide the basic skeleton for this:

- msi_parent_ops
- child domain init callback
- the MSI parent flag set in irqdomain::flags

This does not make ITS a functional parent domain as there is no bit set in the bus_select_mask yet, but it provides the base to implement PCI and platform MSI support gradually on top.

---
 drivers/irqchip/Kconfig                     |  1 +-
 drivers/irqchip/Makefile                    |  2 +-
 drivers/irqchip/irq-gic-common.h            |  3 ++-
 drivers/irqchip/irq-gic-v3-its-msi-parent.c | 31 ++++++++++++++++++++-
 drivers/irqchip/irq-gic-v3-its.c            |  5 +++-
 5 files changed, 41 insertions(+), 1 deletion(-)
 create mode 100644 drivers/irqchip/irq-gic-v3-its-msi-parent.c
drivers/irqchip/Kconfig +1 -0
index 2bf8d94..b51863f 100644
@@ -41,6 +41,7 @@ config ARM_GIC_V3
 config ARM_GIC_V3_ITS
 	bool
 	select GENERIC_MSI_IRQ
+	select IRQ_MSI_LIB
 	default ARM_GIC_V3
 
 config ARM_GIC_V3_ITS_PCI
drivers/irqchip/Makefile +1 -1
index 72c7f62..6e4f771 100644
@@ -32,7 +32,7 @@ obj-$(CONFIG_ARCH_REALVIEW)		+= irq-gic-realview.o
 obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
-obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o
+obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_PCI)	+= irq-gic-v3-its-pci-msi.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_FSL_MC)	+= irq-gic-v3-its-fsl-mc-msi.o
 obj-$(CONFIG_PARTITION_PERCPU)		+= irq-partition-percpu.o
drivers/irqchip/irq-gic-common.h +3 -0
index f407cce..eb4a220 100644
@@ -8,6 +8,7 @@
 
 #include <linux/of.h>
 #include <linux/irqdomain.h>
+#include <linux/msi.h>
 #include <linux/irqchip/arm-gic-common.h>
 
 struct gic_quirk {
@@ -29,6 +30,8 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
 void gic_enable_of_quirks(const struct device_node *np,
 			  const struct gic_quirk *quirks, void *data);
 
+extern const struct msi_parent_ops gic_v3_its_msi_parent_ops;
+
 #define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING    (1 << 0)
 #define RDIST_FLAGS_RD_TABLES_PREALLOCATED     (1 << 1)
 #define RDIST_FLAGS_FORCE_NON_SHAREABLE        (1 << 2)
drivers/irqchip/irq-gic-v3-its-msi-parent.c new file +31 -0
new file mode 100644
index 0000000..cdc0844
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0-only
+// Copyright (C) 2022 Linutronix GmbH
+// Copyright (C) 2022 Intel
+
+#include "irq-gic-common.h"
+#include "irq-msi-lib.h"
+
+#define ITS_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
+				 MSI_FLAG_USE_DEF_CHIP_OPS)
+
+#define ITS_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK |	\
+				 MSI_FLAG_PCI_MSIX      |	\
+				 MSI_FLAG_MULTI_PCI_MSI |	\
+				 MSI_FLAG_PCI_MSI_MASK_PARENT)
+
+static bool its_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+				  struct irq_domain *real_parent, struct msi_domain_info *info)
+{
+	if (!msi_lib_init_dev_msi_info(dev, domain, real_parent, info))
+		return false;
+
+	return true;
+}
+
+const struct msi_parent_ops gic_v3_its_msi_parent_ops = {
+	.supported_flags	= ITS_MSI_FLAGS_SUPPORTED,
+	.required_flags		= ITS_MSI_FLAGS_REQUIRED,
+	.bus_select_token	= DOMAIN_BUS_NEXUS,
+	.prefix			= "ITS-",
+	.init_dev_msi_info	= its_init_dev_msi_info,
+};
drivers/irqchip/irq-gic-v3-its.c +5 -0
index 3c755d5..d770d6a 100644
@@ -38,6 +38,7 @@
 #include <asm/exception.h>
 
 #include "irq-gic-common.h"
+#include "irq-msi-lib.h"
 
 #define ITS_FLAGS_CMDQ_NEEDS_FLUSHING		(1ULL << 0)
 #define ITS_FLAGS_WORKAROUND_CAVIUM_22375	(1ULL << 1)
@@ -3688,6 +3689,7 @@ static void its_irq_domain_free(struct irq_domain *domain, unsigned int virq,
 }
 
 static const struct irq_domain_ops its_domain_ops = {
+	.select			= msi_lib_irq_domain_select,
 	.alloc			= its_irq_domain_alloc,
 	.free			= its_irq_domain_free,
 	.activate		= its_irq_domain_activate,
@@ -4993,6 +4995,9 @@ static int its_init_domain(struct its_node *its)
 
 	irq_domain_update_bus_token(inner_domain, DOMAIN_BUS_NEXUS);
 
+	inner_domain->msi_parent_ops = &gic_v3_its_msi_parent_ops;
+	inner_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+
 	return 0;
 }
 
#13 lore raw

PATCH v4 4/21 irqchip/irq-msi-lib: Prepare for PCI MSI/MSIX

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

Add the bus tokens for DOMAIN_BUS_PCI_DEVICE_MSI and DOMAIN_BUS_PCI_DEVICE_MSIX to the common child init function.

Provide the match mask which can be used by parent domain implementation so the bitmask based child bus token match works.

---
v3: removed pci_device_msi_mask_unmask_parent_enable call as there is no more global static key.

---
 drivers/irqchip/irq-msi-lib.c | 6 ++++++
 drivers/irqchip/irq-msi-lib.h | 6 ++++++
 2 files changed, 12 insertions(+)
drivers/irqchip/irq-msi-lib.c +6 -0
index acbccf8f7f5b..0b359c5d8c6c 100644
@@ -48,6 +48,12 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 
 	/* Is the target domain bus token supported? */
 	switch(info->bus_token) {
+	case DOMAIN_BUS_PCI_DEVICE_MSI:
+	case DOMAIN_BUS_PCI_DEVICE_MSIX:
+		if (WARN_ON_ONCE(!IS_ENABLED(CONFIG_PCI_MSI)))
+			return false;
+
+		break;
 	default:
 		/*
 		 * This should never be reached. See
drivers/irqchip/irq-msi-lib.h +6 -0
index f0706cc28264..525aa5284a99 100644
@@ -9,6 +9,12 @@
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
 
+#ifdef CONFIG_PCI_MSI
+#define MATCH_PCI_MSI		BIT(DOMAIN_BUS_PCI_MSI)
+#else
+#define MATCH_PCI_MSI		(0)
+#endif
+
 int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
 			      enum irq_domain_bus_token bus_token);
 
Signature · 1 line
2.34.1
#14 lore raw

[tip: irq/core] irqchip/irq-msi-lib: Prepare for PCI MSI/MSIX

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     3ed8ef2eb156f93c32e935f0f668902923f9a02c
Gitweb:        https://git.kernel.org/tip/3ed8ef2eb156f93c32e935f0f668902923f9a02c
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:38 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:23 +02:00

irqchip/irq-msi-lib: Prepare for PCI MSI/MSIX

Add the bus tokens for DOMAIN_BUS_PCI_DEVICE_MSI and DOMAIN_BUS_PCI_DEVICE_MSIX to the common child init function.

Provide the match mask which can be used by parent domain implementation so the bitmask based child bus token match works.

---
 drivers/irqchip/irq-msi-lib.c | 11 ++++++++++-
 drivers/irqchip/irq-msi-lib.h |  6 ++++++
 2 files changed, 16 insertions(+), 1 deletion(-)
drivers/irqchip/irq-msi-lib.c +10 -1
index ec1a10f..ef26962 100644
@@ -28,6 +28,7 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 			       struct msi_domain_info *info)
 {
 	const struct msi_parent_ops *pops = real_parent->msi_parent_ops;
+	u32 required_flags;
 
 	/* Parent ops available? */
 	if (WARN_ON_ONCE(!pops))
@@ -46,8 +47,16 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 		return false;
 	}
 
+	required_flags = pops->required_flags;
+
 	/* Is the target domain bus token supported? */
 	switch(info->bus_token) {
+	case DOMAIN_BUS_PCI_DEVICE_MSI:
+	case DOMAIN_BUS_PCI_DEVICE_MSIX:
+		if (WARN_ON_ONCE(!IS_ENABLED(CONFIG_PCI_MSI)))
+			return false;
+
+		break;
 	default:
 		/*
 		 * This should never be reached. See
@@ -63,7 +72,7 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 	 */
 	info->flags			&= pops->supported_flags;
 	/* Enforce the required flags */
-	info->flags			|= pops->required_flags;
+	info->flags			|= required_flags;
 
 	/* Chip updates for all child bus types */
 	if (!info->chip->irq_eoi)
drivers/irqchip/irq-msi-lib.h +6 -0
index f0706cc..525aa52 100644
@@ -9,6 +9,12 @@
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
 
+#ifdef CONFIG_PCI_MSI
+#define MATCH_PCI_MSI		BIT(DOMAIN_BUS_PCI_MSI)
+#else
+#define MATCH_PCI_MSI		(0)
+#endif
+
 int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
 			      enum irq_domain_bus_token bus_token);
 
#15 lore raw

[tip: irq/msi] irqchip/irq-msi-lib: Prepare for PCI MSI/MSIX

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     8c41ccec839c622b2d1be769a95405e4e9a4cb20
Gitweb:        https://git.kernel.org/tip/8c41ccec839c622b2d1be769a95405e4e9a4cb20
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:38 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:19 +02:00

irqchip/irq-msi-lib: Prepare for PCI MSI/MSIX

Add the bus tokens for DOMAIN_BUS_PCI_DEVICE_MSI and DOMAIN_BUS_PCI_DEVICE_MSIX to the common child init function.

Provide the match mask which can be used by parent domain implementation so the bitmask based child bus token match works.

---
 drivers/irqchip/irq-msi-lib.c | 11 ++++++++++-
 drivers/irqchip/irq-msi-lib.h |  6 ++++++
 2 files changed, 16 insertions(+), 1 deletion(-)
drivers/irqchip/irq-msi-lib.c +10 -1
index ec1a10f..ef26962 100644
@@ -28,6 +28,7 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 			       struct msi_domain_info *info)
 {
 	const struct msi_parent_ops *pops = real_parent->msi_parent_ops;
+	u32 required_flags;
 
 	/* Parent ops available? */
 	if (WARN_ON_ONCE(!pops))
@@ -46,8 +47,16 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 		return false;
 	}
 
+	required_flags = pops->required_flags;
+
 	/* Is the target domain bus token supported? */
 	switch(info->bus_token) {
+	case DOMAIN_BUS_PCI_DEVICE_MSI:
+	case DOMAIN_BUS_PCI_DEVICE_MSIX:
+		if (WARN_ON_ONCE(!IS_ENABLED(CONFIG_PCI_MSI)))
+			return false;
+
+		break;
 	default:
 		/*
 		 * This should never be reached. See
@@ -63,7 +72,7 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 	 */
 	info->flags			&= pops->supported_flags;
 	/* Enforce the required flags */
-	info->flags			|= pops->required_flags;
+	info->flags			|= required_flags;
 
 	/* Chip updates for all child bus types */
 	if (!info->chip->irq_eoi)
drivers/irqchip/irq-msi-lib.h +6 -0
index f0706cc..525aa52 100644
@@ -9,6 +9,12 @@
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
 
+#ifdef CONFIG_PCI_MSI
+#define MATCH_PCI_MSI		BIT(DOMAIN_BUS_PCI_MSI)
+#else
+#define MATCH_PCI_MSI		(0)
+#endif
+
 int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
 			      enum irq_domain_bus_token bus_token);
 
#16 lore raw

PATCH v4 5/21 irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

The its_pci_msi_prepare() function from the ITS-PCI/MSI code provides the 'global' PCI/MSI domains. Move this function to the ITS-MSI parent code and amend the function to use the domain hardware size, which is the MSI[X] vector count, for allocating the ITS slots for the PCI device.

Enable PCI matching in msi_parent_ops and provide the necessary update to the ITS specific child domain initialization function so that the prepare callback gets invoked on allocations.

The latter might be optimized to do the allocation right at the point where the child domain is initialized, but keep it simple for now.

---
v3: adopted to IMS->MSI rename

---
 drivers/irqchip/Makefile                    |   1 -
 drivers/irqchip/irq-gic-v3-its-msi-parent.c | 108 +++++++++++
 drivers/irqchip/irq-gic-v3-its-pci-msi.c    | 202 --------------------
 3 files changed, 108 insertions(+), 203 deletions(-)
 delete mode 100644 drivers/irqchip/irq-gic-v3-its-pci-msi.c
drivers/irqchip/Makefile +0 -1
index 6e4f7715206d..c5316634637f 100644
@@ -33,7 +33,6 @@ obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
 obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
-obj-$(CONFIG_ARM_GIC_V3_ITS_PCI)	+= irq-gic-v3-its-pci-msi.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_FSL_MC)	+= irq-gic-v3-its-fsl-mc-msi.o
 obj-$(CONFIG_PARTITION_PERCPU)		+= irq-partition-percpu.o
 obj-$(CONFIG_HISILICON_IRQ_MBIGEN)	+= irq-mbigen.o
drivers/irqchip/irq-gic-v3-its-msi-parent.c +108 -0

133 lines not on this page · raw message

drivers/irqchip/irq-gic-v3-its-pci-msi.c deleted +0 -202

202 lines not on this page · raw message

Signature · 1 line
2.34.1
#17 lore raw

Re: [patch V4 05/21] irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

Hi Thomas,

Quoted 11 lines · Thomas Gleixner

From: Thomas Gleixner <[email protected]>

The its_pci_msi_prepare() function from the ITS-PCI/MSI code provides the 'global' PCI/MSI domains. Move this function to the ITS-MSI parent code and amend the function to use the domain hardware size, which is the MSI[X] vector count, for allocating the ITS slots for the PCI device.

Enable PCI matching in msi_parent_ops and provide the necessary update to the ITS specific child domain initialization function so that the prepare callback gets invoked on allocations.

Quoted 7 lines

The latter might be optimized to do the allocation right at the point where the child domain is initialized, but keep it simple for now.

I just noticed guests (under KVM) failing to boot on my TX2 with your latest branch. I bisected to this patch as the first bad commit.

I'm away this weekend, so won't have time to dive deeper. It looks like the CPU is stuck in do_idle() (no timer interrupts?). Also sysrq did not seem able to get the stack trace on the other CPUs. It fails both with a single or multiple CPUs in the same way place (shortly before mounting the rootfs and starting user space).

Not sure whether it's related by Red Hat's CI is also reporting boot failures:

https://lore.kernel.org/r/[email protected]

I'll drop your branch from the arm64 for-kernelci for now and have a look again on Monday.

Signature · 1 line
Catalin
#18 lore raw

Re: [patch V4 05/21] irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

Quoted 8 lines · Catalin Marinas

I just noticed guests (under KVM) failing to boot on my TX2 with your latest branch. I bisected to this patch as the first bad commit.

I'm away this weekend, so won't have time to dive deeper. It looks like the CPU is stuck in do_idle() (no timer interrupts?). Also sysrq did not seem able to get the stack trace on the other CPUs. It fails both with a single or multiple CPUs in the same way place (shortly before mounting the rootfs and starting user space).

From the RH log it's clear that PCI interrupts are not delivered.

Quoted 2 lines

I'll drop your branch from the arm64 for-kernelci for now and have a look again on Monday.

I stare too. Unfortunately I don't have access to such hardware :(

Thanks,

Thomas

maintainer
#19 lore raw

Re: [patch V4 05/21] irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

Quoted 11 lines · Thomas Gleixner
Quoted 8 lines · Catalin Marinas

I just noticed guests (under KVM) failing to boot on my TX2 with your latest branch. I bisected to this patch as the first bad commit.

I'm away this weekend, so won't have time to dive deeper. It looks like the CPU is stuck in do_idle() (no timer interrupts?). Also sysrq did not seem able to get the stack trace on the other CPUs. It fails both with a single or multiple CPUs in the same way place (shortly before mounting the rootfs and starting user space).

Quoted 6 lines

From the RH log it's clear that PCI interrupts are not delivered.

Quoted 2 lines

I'll drop your branch from the arm64 for-kernelci for now and have a look again on Monday.

I stare too. Unfortunately I don't have access to such hardware :(

On the face of it, the LPIs are never unmasked (grepping in /sys/kernel/debug/kvm/*/vgic-state):

Distributor
===========
vgic_model: GICv3
nr_spis: 32
nr_lpis: 7
enabled: 1

P=pending_latch, L=line_level, A=active
E=enabled, H=hw, C=config (level=1, edge=0)
G=group

VCPU 0 TYP ID TGT_ID PLAEHCG HWID TARGET SRC PRI VCPU_ID
----------------------------------------------------------------
[...]

       LPI 8192      0 1000001        0        0   0 160      -1 
       LPI 8193      1 0000001        0        0   0 160      -1 
       LPI 8194      2 0000001        0        0   0 160      -1 
       LPI 8256      3 0000001        0        0   0 160      -1 
       LPI 8257      4 0000001        0        0   0 160      -1 
       LPI 8320      5 0000001        0        0   0 160      -1 
       LPI 8321      6 1000001        0        0   0 160      -1

8192 and 8321 are pending, but never enabled.

This is further confirmed by placing traces in the guest. Now trying to find my way through the new maze of callbacks, because something is clearly missing there.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#20 lore raw

Re: [patch V4 05/21] irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

Quoted 46 lines · Marc Zyngier
Quoted 11 lines · Thomas Gleixner
Quoted 8 lines · Catalin Marinas

I just noticed guests (under KVM) failing to boot on my TX2 with your latest branch. I bisected to this patch as the first bad commit.

I'm away this weekend, so won't have time to dive deeper. It looks like the CPU is stuck in do_idle() (no timer interrupts?). Also sysrq did not seem able to get the stack trace on the other CPUs. It fails both with a single or multiple CPUs in the same way place (shortly before mounting the rootfs and starting user space).

Quoted 6 lines

From the RH log it's clear that PCI interrupts are not delivered.

Quoted 2 lines

I'll drop your branch from the arm64 for-kernelci for now and have a look again on Monday.

I stare too. Unfortunately I don't have access to such hardware :(

On the face of it, the LPIs are never unmasked (grepping in /sys/kernel/debug/kvm/*/vgic-state):

Distributor
===========
vgic_model: GICv3
nr_spis: 32
nr_lpis: 7
enabled: 1

P=pending_latch, L=line_level, A=active
E=enabled, H=hw, C=config (level=1, edge=0)
G=group

VCPU 0 TYP ID TGT_ID PLAEHCG HWID TARGET SRC PRI VCPU_ID
----------------------------------------------------------------
[...]

       LPI 8192      0 1000001        0        0   0 160      -1 
       LPI 8193      1 0000001        0        0   0 160      -1 
       LPI 8194      2 0000001        0        0   0 160      -1 
       LPI 8256      3 0000001        0        0   0 160      -1 
       LPI 8257      4 0000001        0        0   0 160      -1 
       LPI 8320      5 0000001        0        0   0 160      -1 
       LPI 8321      6 1000001        0        0   0 160      -1
Quoted 5 lines

8192 and 8321 are pending, but never enabled.

This is further confirmed by placing traces in the guest. Now trying to find my way through the new maze of callbacks, because something is clearly missing there.

This is clearly related to MSI_FLAG_PCI_MSI_MASK_PARENT which is not seen as being set from cond_unmask_parent(), and ignoring this condition results in a booting VM.

I have the ugly feeling that the flag is applied at the wrong level, or not propagated.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#21 lore raw

Re: [patch V4 05/21] irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

Quoted 55 lines · Marc Zyngier
Quoted 46 lines · Marc Zyngier
Quoted 11 lines · Thomas Gleixner
Quoted 8 lines · Catalin Marinas

I just noticed guests (under KVM) failing to boot on my TX2 with your latest branch. I bisected to this patch as the first bad commit.

I'm away this weekend, so won't have time to dive deeper. It looks like the CPU is stuck in do_idle() (no timer interrupts?). Also sysrq did not seem able to get the stack trace on the other CPUs. It fails both with a single or multiple CPUs in the same way place (shortly before mounting the rootfs and starting user space).

Quoted 6 lines

From the RH log it's clear that PCI interrupts are not delivered.

Quoted 2 lines

I'll drop your branch from the arm64 for-kernelci for now and have a look again on Monday.

I stare too. Unfortunately I don't have access to such hardware :(

On the face of it, the LPIs are never unmasked (grepping in /sys/kernel/debug/kvm/*/vgic-state):

Distributor
===========
vgic_model: GICv3
nr_spis: 32
nr_lpis: 7
enabled: 1

P=pending_latch, L=line_level, A=active
E=enabled, H=hw, C=config (level=1, edge=0)
G=group

VCPU 0 TYP ID TGT_ID PLAEHCG HWID TARGET SRC PRI VCPU_ID
----------------------------------------------------------------
[...]

       LPI 8192      0 1000001        0        0   0 160      -1 
       LPI 8193      1 0000001        0        0   0 160      -1 
       LPI 8194      2 0000001        0        0   0 160      -1 
       LPI 8256      3 0000001        0        0   0 160      -1 
       LPI 8257      4 0000001        0        0   0 160      -1 
       LPI 8320      5 0000001        0        0   0 160      -1 
       LPI 8321      6 1000001        0        0   0 160      -1
Quoted 5 lines

8192 and 8321 are pending, but never enabled.

This is further confirmed by placing traces in the guest. Now trying to find my way through the new maze of callbacks, because something is clearly missing there.

Quoted 6 lines

This is clearly related to MSI_FLAG_PCI_MSI_MASK_PARENT which is not seen as being set from cond_unmask_parent(), and ignoring this condition results in a booting VM.

I have the ugly feeling that the flag is applied at the wrong level, or not propagated.

Here's a possible fix. Making the masking at the ITS level optional is not an option (haha). It is the PCI masking that is totally superfluous and that could completely be elided.

With this hack, I can boot a GICv3+ITS guest as usual.

	M.
drivers/irqchip/irq-gic-v3-its-msi-parent.c +4 -4
index 21daa452ffa6d..b66e64eaae440 100644
@@ -10,13 +10,13 @@
 #include "irq-gic-common.h"
 #include "irq-msi-lib.h"
 
-#define ITS_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
-				 MSI_FLAG_USE_DEF_CHIP_OPS)
+#define ITS_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS  |	\
+				 MSI_FLAG_USE_DEF_CHIP_OPS |	\
+				 MSI_FLAG_PCI_MSI_MASK_PARENT)
 
 #define ITS_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK |	\
 				 MSI_FLAG_PCI_MSIX      |	\
-				 MSI_FLAG_MULTI_PCI_MSI |	\
-				 MSI_FLAG_PCI_MSI_MASK_PARENT)
+				 MSI_FLAG_MULTI_PCI_MSI)
 
 #ifdef CONFIG_PCI_MSI
 static int its_pci_msi_vec_count(struct pci_dev *pdev, void *data)
Signature · 1 line
Without deviation from the norm, progress is not possible.
#22 lore raw

Re: [patch V4 05/21] irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

Marc!

Quoted 2 lines · Marc Zyngier
Quoted 2 lines

I have the ugly feeling that the flag is applied at the wrong level, or not propagated.

Indeed.

Quoted 3 lines

Here's a possible fix. Making the masking at the ITS level optional is not an option (haha). It is the PCI masking that is totally superfluous and that could completely be elided.

It's the right fix because ITS requires this bit to be set.

Vs. PCI masking, you are right from a pure ITS point of view, but not from the PCI side. PCI can't be unmsaked until there is a valid message and we need to mask it on shutdown.

It's not a run time issue at all because PCI/MSI is edge triggered so the mask/unmask dance only matters during startup, shutdown and message update.

Quoted 1 line

With this hack, I can boot a GICv3+ITS guest as usual.

It's not a hack. It's the proper solution. Let me fold that back and look at the other PCI conversions which probably have the same issue.

Thanks for digging into this. This help is truly welcome right now.

Thanks,

tglx

#23 lore raw

Re: [patch V4 05/21] irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

Quoted 6 lines · Thomas Gleixner
Quoted 1 line

With this hack, I can boot a GICv3+ITS guest as usual.

It's not a hack. It's the proper solution. Let me fold that back and look at the other PCI conversions which probably have the same issue.

Thanks for digging into this. This help is truly welcome right now.

So while I pondered to do it slightly differently for a moment I went back to this approach and fixed up the other affected ones too. Full delta vs. devmsi-arm-v4-1 below.

Updated branch:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git devmsi-arm-v4-2

Thanks,

tglx
---

drivers/irqchip/irq-gic-v2m.c +5 -4
index 428132aa26cc..51af63c046ed 100644
@@ -243,13 +243,14 @@ static void __init gicv2m_teardown(void)
 	}
 }
 
-#define GICV2M_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
-				    MSI_FLAG_USE_DEF_CHIP_OPS)
+
+#define GICV2M_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |		\
+				    MSI_FLAG_USE_DEF_CHIP_OPS |		\
+				    MSI_FLAG_PCI_MSI_MASK_PARENT)
 
 #define GICV2M_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK |	\
 				    MSI_FLAG_PCI_MSIX      |	\
-				    MSI_FLAG_MULTI_PCI_MSI |	\
-				    MSI_FLAG_PCI_MSI_MASK_PARENT)
+				    MSI_FLAG_MULTI_PCI_MSI)
 
 static struct msi_parent_ops gicv2m_msi_parent_ops = {
 	.supported_flags	= GICV2M_MSI_FLAGS_SUPPORTED,
drivers/irqchip/irq-gic-v3-its-msi-parent.c +3 -3
index 21daa452ffa6..780e1bc9df54 100644
@@ -11,12 +11,12 @@
 #include "irq-msi-lib.h"
 
 #define ITS_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
-				 MSI_FLAG_USE_DEF_CHIP_OPS)
+				 MSI_FLAG_USE_DEF_CHIP_OPS |	\
+				 MSI_FLAG_PCI_MSI_MASK_PARENT)
 
 #define ITS_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK |	\
 				 MSI_FLAG_PCI_MSIX      |	\
-				 MSI_FLAG_MULTI_PCI_MSI |	\
-				 MSI_FLAG_PCI_MSI_MASK_PARENT)
+				 MSI_FLAG_MULTI_PCI_MSI)
 
 #ifdef CONFIG_PCI_MSI
 static int its_pci_msi_vec_count(struct pci_dev *pdev, void *data)
drivers/irqchip/irq-gic-v3-mbi.c +3 -3
index 0aad9699be33..3fe870f8ee17 100644
@@ -191,12 +191,12 @@ static bool mbi_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 }
 
 #define MBI_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
-				 MSI_FLAG_USE_DEF_CHIP_OPS)
+				 MSI_FLAG_USE_DEF_CHIP_OPS |	\
+				 MSI_FLAG_PCI_MSI_MASK_PARENT)
 
 #define MBI_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK |	\
 				 MSI_FLAG_PCI_MSIX      |	\
-				 MSI_FLAG_MULTI_PCI_MSI |	\
-				 MSI_FLAG_PCI_MSI_MASK_PARENT)
+				 MSI_FLAG_MULTI_PCI_MSI)
 
 static const struct msi_parent_ops gic_v3_mbi_msi_parent_ops = {
 	.supported_flags	= MBI_MSI_FLAGS_SUPPORTED,
drivers/irqchip/irq-msi-lib.c +5 -2
index 15f92ce4d1e8..e93917b98d7d 100644
@@ -28,6 +28,7 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 			       struct msi_domain_info *info)
 {
 	const struct msi_parent_ops *pops = real_parent->msi_parent_ops;
+	u32 required_flags = pops->required_flags;
 
 	/*
 	 * MSI parent domain specific settings. For now there is only the
@@ -66,8 +67,10 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 
 		/* Core managed MSI descriptors */
 		info->flags = MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS | MSI_FLAG_FREE_MSI_DESCS;
-		break;
+		fallthrough;
 	case DOMAIN_BUS_WIRED_TO_MSI:
+		/* Remove PCI specific flags */
+		required_flags &= ~MSI_FLAG_PCI_MSI_MASK_PARENT;
 		break;
 	default:
 		/*
@@ -84,7 +87,7 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 	 */
 	info->flags			&= pops->supported_flags;
 	/* Enforce the required flags */
-	info->flags			|= pops->required_flags;
+	info->flags			|= required_flags;
 
 	/* Chip updates for all child bus types */
 	if (!info->chip->irq_eoi)
#24 lore raw

Re: [patch V4 05/21] irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

Quoted 8 lines · Thomas Gleixner
Quoted 6 lines · Thomas Gleixner
Quoted 1 line

With this hack, I can boot a GICv3+ITS guest as usual.

It's not a hack. It's the proper solution. Let me fold that back and look at the other PCI conversions which probably have the same issue.

Thanks for digging into this. This help is truly welcome right now.

Quoted 7 lines

So while I pondered to do it slightly differently for a moment I went back to this approach and fixed up the other affected ones too. Full delta vs. devmsi-arm-v4-1 below.

Updated branch:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git devmsi-arm-v4-2

This seems to work for me as well. I re-added this branch to arm64 for-kernelci for wider exposure.

Thanks.

Signature · 1 line
Catalin
maintainer
#25 lore raw

Re: [patch V4 05/21] irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

In reply to Catalin Marinas
Quoted 18 lines · Catalin Marinas

Hi Thomas,

Quoted 14 lines · Thomas Gleixner

From: Thomas Gleixner <[email protected]>

The its_pci_msi_prepare() function from the ITS-PCI/MSI code provides the 'global' PCI/MSI domains. Move this function to the ITS-MSI parent code and amend the function to use the domain hardware size, which is the MSI[X] vector count, for allocating the ITS slots for the PCI device.

Enable PCI matching in msi_parent_ops and provide the necessary update to the ITS specific child domain initialization function so that the prepare callback gets invoked on allocations.

The latter might be optimized to do the allocation right at the point where the child domain is initialized, but keep it simple for now.

Quoted 7 lines
Quoted 4 lines

I just noticed guests (under KVM) failing to boot on my TX2 with your latest branch. I bisected to this patch as the first bad commit.

Reproduced here on a different host (M1), so this is not specific to TX2 (which would have been odd since KVM emulates the ITS entirely).

I'll start digging.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
#26 lore raw

[tip: irq/core] irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

In reply to Thomas Gleixner
The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     233db05bc37f82fb4564bc16ff9d545f00f5770e
Gitweb:        https://git.kernel.org/tip/233db05bc37f82fb4564bc16ff9d545f00f5770e
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:39 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:23 +02:00

irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

The its_pci_msi_prepare() function from the ITS-PCI/MSI code provides the 'global' PCI/MSI domains. Move this function to the ITS-MSI parent code and amend the function to use the domain hardware size, which is the MSI[X] vector count, for allocating the ITS slots for the PCI device.

Enable PCI matching in msi_parent_ops and provide the necessary update to the ITS specific child domain initialization function so that the prepare callback gets invoked on allocations.

The latter might be optimized to do the allocation right at the point where the child domain is initialized, but keep it simple for now.

---
 drivers/irqchip/Makefile                    |   1 +-
 drivers/irqchip/irq-gic-v3-its-msi-parent.c | 114 ++++++++++-
 drivers/irqchip/irq-gic-v3-its-pci-msi.c    | 202 +-------------------
 3 files changed, 111 insertions(+), 206 deletions(-)
 delete mode 100644 drivers/irqchip/irq-gic-v3-its-pci-msi.c
drivers/irqchip/Makefile +0 -1
index def6fea..ac0627a 100644
@@ -33,7 +33,6 @@ obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
 obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
-obj-$(CONFIG_ARM_GIC_V3_ITS_PCI)	+= irq-gic-v3-its-pci-msi.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_FSL_MC)	+= irq-gic-v3-its-fsl-mc-msi.o
 obj-$(CONFIG_PARTITION_PERCPU)		+= irq-partition-percpu.o
 obj-$(CONFIG_HISILICON_IRQ_MBIGEN)	+= irq-mbigen.o
drivers/irqchip/irq-gic-v3-its-msi-parent.c +111 -3

140 lines not on this page · raw message

drivers/irqchip/irq-gic-v3-its-pci-msi.c deleted +0 -202

202 lines not on this page · raw message

#27 lore raw

[tip: irq/msi] irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     b5712bf89b4bbc5bcc9ebde8753ad222f1f68296
Gitweb:        https://git.kernel.org/tip/b5712bf89b4bbc5bcc9ebde8753ad222f1f68296
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:39 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]

The its_pci_msi_prepare() function from the ITS-PCI/MSI code provides the 'global' PCI/MSI domains. Move this function to the ITS-MSI parent code and amend the function to use the domain hardware size, which is the MSI[X] vector count, for allocating the ITS slots for the PCI device.

Enable PCI matching in msi_parent_ops and provide the necessary update to the ITS specific child domain initialization function so that the prepare callback gets invoked on allocations.

The latter might be optimized to do the allocation right at the point where the child domain is initialized, but keep it simple for now.

---
 drivers/irqchip/Makefile                    |   1 +-
 drivers/irqchip/irq-gic-v3-its-msi-parent.c | 114 ++++++++++-
 drivers/irqchip/irq-gic-v3-its-pci-msi.c    | 202 +-------------------
 3 files changed, 111 insertions(+), 206 deletions(-)
 delete mode 100644 drivers/irqchip/irq-gic-v3-its-pci-msi.c
drivers/irqchip/Makefile +0 -1
index 6e4f771..c531663 100644
@@ -33,7 +33,6 @@ obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
 obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
-obj-$(CONFIG_ARM_GIC_V3_ITS_PCI)	+= irq-gic-v3-its-pci-msi.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_FSL_MC)	+= irq-gic-v3-its-fsl-mc-msi.o
 obj-$(CONFIG_PARTITION_PERCPU)		+= irq-partition-percpu.o
 obj-$(CONFIG_HISILICON_IRQ_MBIGEN)	+= irq-mbigen.o
drivers/irqchip/irq-gic-v3-its-msi-parent.c +111 -3

140 lines not on this page · raw message

drivers/irqchip/irq-gic-v3-its-pci-msi.c deleted +0 -202

202 lines not on this page · raw message

#28 lore raw

PATCH v4 6/21 irqchip/irq-msi-lib: Prepare for DEVICE MSI to replace platform MSI

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

Add the prerequisites for DEVICE MSI into the shared select() and child domain init function. These domains are really trivial and just provide a custom irq chip callback to write the MSI message.

---
v3: adopted to IMS->MSI rename

---
 drivers/irqchip/irq-msi-lib.c | 13 +++++++++++++
 drivers/irqchip/irq-msi-lib.h |  2 ++
 2 files changed, 15 insertions(+)
drivers/irqchip/irq-msi-lib.c +13 -0
index 0b359c5d8c6c..6aa4974d2d12 100644
@@ -53,6 +53,19 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 		if (WARN_ON_ONCE(!IS_ENABLED(CONFIG_PCI_MSI)))
 			return false;
 
+		break;
+	case DOMAIN_BUS_DEVICE_MSI:
+		/*
+		 * Per device MSI should never have any MSI feature bits
+		 * set. It's sole purpose is to create a dumb interrupt
+		 * chip which has a device specific irq_write_msi_msg()
+		 * callback.
+		 */
+		if (WARN_ON_ONCE(info->flags))
+			return false;
+
+		/* Core managed MSI descriptors */
+		info->flags = MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS | MSI_FLAG_FREE_MSI_DESCS;
 		break;
 	default:
 		/*
drivers/irqchip/irq-msi-lib.h +2 -0
index 525aa5284a99..681ceabb7bc7 100644
@@ -15,6 +15,8 @@
 #define MATCH_PCI_MSI		(0)
 #endif
 
+#define MATCH_PLATFORM_MSI	BIT(DOMAIN_BUS_PLATFORM_MSI)
+
 int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
 			      enum irq_domain_bus_token bus_token);
 
Signature · 1 line
2.34.1
#29 lore raw

[tip: irq/core] irqchip/irq-msi-lib: Prepare for DEVICE MSI to replace platform MSI

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     e12eba0f080304db4aae02f51e4b6ff873ea13c5
Gitweb:        https://git.kernel.org/tip/e12eba0f080304db4aae02f51e4b6ff873ea13c5
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:41 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:23 +02:00

irqchip/irq-msi-lib: Prepare for DEVICE MSI to replace platform MSI

Add the prerequisites for DEVICE MSI into the shared select() and child domain init function. These domains are really trivial and just provide a custom irq chip callback to write the MSI message.

---
 drivers/irqchip/irq-msi-lib.c | 15 +++++++++++++++
 drivers/irqchip/irq-msi-lib.h |  2 ++
 2 files changed, 17 insertions(+)
drivers/irqchip/irq-msi-lib.c +15 -0
index ef26962..b98a219 100644
@@ -57,6 +57,21 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 			return false;
 
 		break;
+	case DOMAIN_BUS_DEVICE_MSI:
+		/*
+		 * Per device MSI should never have any MSI feature bits
+		 * set. It's sole purpose is to create a dumb interrupt
+		 * chip which has a device specific irq_write_msi_msg()
+		 * callback.
+		 */
+		if (WARN_ON_ONCE(info->flags))
+			return false;
+
+		/* Core managed MSI descriptors */
+		info->flags = MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS | MSI_FLAG_FREE_MSI_DESCS;
+		/* Remove PCI specific flags */
+		required_flags &= ~MSI_FLAG_PCI_MSI_MASK_PARENT;
+		break;
 	default:
 		/*
 		 * This should never be reached. See
drivers/irqchip/irq-msi-lib.h +2 -0
index 525aa52..681ceab 100644
@@ -15,6 +15,8 @@
 #define MATCH_PCI_MSI		(0)
 #endif
 
+#define MATCH_PLATFORM_MSI	BIT(DOMAIN_BUS_PLATFORM_MSI)
+
 int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
 			      enum irq_domain_bus_token bus_token);
 
#30 lore raw

[tip: irq/msi] irqchip/irq-msi-lib: Prepare for DEVICE MSI to replace platform MSI

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     496436f4a514a3fb4bc7aecd41f0dd4b38e39b1f
Gitweb:        https://git.kernel.org/tip/496436f4a514a3fb4bc7aecd41f0dd4b38e39b1f
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:41 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

irqchip/irq-msi-lib: Prepare for DEVICE MSI to replace platform MSI

Add the prerequisites for DEVICE MSI into the shared select() and child domain init function. These domains are really trivial and just provide a custom irq chip callback to write the MSI message.

---
 drivers/irqchip/irq-msi-lib.c | 15 +++++++++++++++
 drivers/irqchip/irq-msi-lib.h |  2 ++
 2 files changed, 17 insertions(+)
drivers/irqchip/irq-msi-lib.c +15 -0
index ef26962..b98a219 100644
@@ -57,6 +57,21 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 			return false;
 
 		break;
+	case DOMAIN_BUS_DEVICE_MSI:
+		/*
+		 * Per device MSI should never have any MSI feature bits
+		 * set. It's sole purpose is to create a dumb interrupt
+		 * chip which has a device specific irq_write_msi_msg()
+		 * callback.
+		 */
+		if (WARN_ON_ONCE(info->flags))
+			return false;
+
+		/* Core managed MSI descriptors */
+		info->flags = MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS | MSI_FLAG_FREE_MSI_DESCS;
+		/* Remove PCI specific flags */
+		required_flags &= ~MSI_FLAG_PCI_MSI_MASK_PARENT;
+		break;
 	default:
 		/*
 		 * This should never be reached. See
drivers/irqchip/irq-msi-lib.h +2 -0
index 525aa52..681ceab 100644
@@ -15,6 +15,8 @@
 #define MATCH_PCI_MSI		(0)
 #endif
 
+#define MATCH_PLATFORM_MSI	BIT(DOMAIN_BUS_PLATFORM_MSI)
+
 int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
 			      enum irq_domain_bus_token bus_token);
 
#31 lore raw

PATCH v4 7/21 irqchip/mbigen: Prepare for real per device MSI

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

The core infrastructure has everything in place to switch MBIGEN to per device MSI domains and avoid the convoluted construct of the existing platform-MSI layering violation.

The new infrastructure provides a wired interrupt specific interface in the MSI core which converts the 'hardware interrupt number + trigger type' allocation which is required for wired interrupts in the regular irqdomain code to a normal MSI allocation.

The hardware interrupt number and the trigger type are stored in the MSI descriptor device cookie by the core code so the MBIGEN specific code can retrieve them.

The new per device domain is only instantiated when the irqdomain which is associated to the MBIGEN device provides MSI parent functionality. Up to that point it invokes the existing code. Once the parent is converted the code for the current platform-MSI mechanism is removed.

The new domain shares the interrupt chip callbacks and the translation function. The only new functionality aside of filling out the msi_domain_template is a domain specific set_desc() callback, which will go away once all platform-MSI code has been converted.

---
v3: removed unused variable 'parent' in the function 'mbigen_of_create_domain'

---
 drivers/irqchip/irq-mbigen.c | 98 +++++++++++++++++++++++++-----------
 1 file changed, 70 insertions(+), 28 deletions(-)
drivers/irqchip/irq-mbigen.c +70 -28
index 58881d313979..db0fa80330d9 100644
@@ -135,24 +135,14 @@ static int mbigen_set_type(struct irq_data *data, unsigned int type)
 	return 0;
 }
 
-static struct irq_chip mbigen_irq_chip = {
-	.name =			"mbigen-v2",
-	.irq_mask =		irq_chip_mask_parent,
-	.irq_unmask =		irq_chip_unmask_parent,
-	.irq_eoi =		mbigen_eoi_irq,
-	.irq_set_type =		mbigen_set_type,
-	.irq_set_affinity =	irq_chip_set_affinity_parent,
-};
-
-static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
+static void mbigen_write_msi_msg(struct irq_data *d, struct msi_msg *msg)
 {
-	struct irq_data *d = irq_get_irq_data(desc->irq);
 	void __iomem *base = d->chip_data;
 	u32 val;
 
 	if (!msg->address_lo && !msg->address_hi)
 		return;
- 
+
 	base += get_mbigen_vec_reg(d->hwirq);
 	val = readl_relaxed(base);
 
@@ -165,10 +155,8 @@ static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
 	writel_relaxed(val, base);
 }
 
-static int mbigen_domain_translate(struct irq_domain *d,
-				    struct irq_fwspec *fwspec,
-				    unsigned long *hwirq,
-				    unsigned int *type)
+static int mbigen_domain_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
+				   unsigned long *hwirq, unsigned int *type)
 {
 	if (is_of_node(fwspec->fwnode) || is_acpi_device_node(fwspec->fwnode)) {
 		if (fwspec->param_count != 2)
@@ -192,6 +180,17 @@ static int mbigen_domain_translate(struct irq_domain *d,
 	return -EINVAL;
 }
 
+/* The following section will go away once ITS provides a MSI parent */
+
+static struct irq_chip mbigen_irq_chip = {
+	.name =			"mbigen-v2",
+	.irq_mask =		irq_chip_mask_parent,
+	.irq_unmask =		irq_chip_unmask_parent,
+	.irq_eoi =		mbigen_eoi_irq,
+	.irq_set_type =		mbigen_set_type,
+	.irq_set_affinity =	irq_chip_set_affinity_parent,
+};
+
 static int mbigen_irq_domain_alloc(struct irq_domain *domain,
 					unsigned int virq,
 					unsigned int nr_irqs,
@@ -232,11 +231,63 @@ static const struct irq_domain_ops mbigen_domain_ops = {
 	.free		= mbigen_irq_domain_free,
 };
 
+static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
+{
+	mbigen_write_msi_msg(irq_get_irq_data(desc->irq), msg);
+}
+
+/* End of to be removed section */
+
+static void mbigen_domain_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
+{
+	arg->desc = desc;
+	arg->hwirq = (u32)desc->data.icookie.value;
+}
+
+static const struct msi_domain_template mbigen_msi_template = {
+	.chip = {
+		.name			= "mbigen-v2",
+		.irq_mask		= irq_chip_mask_parent,
+		.irq_unmask		= irq_chip_unmask_parent,
+		.irq_eoi		= mbigen_eoi_irq,
+		.irq_set_type		= mbigen_set_type,
+		.irq_write_msi_msg	= mbigen_write_msi_msg,
+	},
+
+	.ops = {
+		.set_desc		= mbigen_domain_set_desc,
+		.msi_translate		= mbigen_domain_translate,
+	},
+
+	.info = {
+		.bus_token		= DOMAIN_BUS_WIRED_TO_MSI,
+		.flags			= MSI_FLAG_USE_DEV_FWNODE,
+	},
+};
+
+static bool mbigen_create_device_domain(struct device *dev, unsigned int size,
+					struct mbigen_device *mgn_chip)
+{
+	struct irq_domain *domain = dev->msi.domain;
+
+	if (WARN_ON_ONCE(!domain))
+		return false;
+
+	if (irq_domain_is_msi_parent(domain)) {
+		return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
+						    &mbigen_msi_template, size,
+						    NULL, mgn_chip->base);
+	}
+
+	/* Remove once ITS provides MSI parent */
+	return !!platform_msi_create_device_domain(dev, size, mbigen_write_msg,
+						   &mbigen_domain_ops, mgn_chip);
+}
+
 static int mbigen_of_create_domain(struct platform_device *pdev,
 				   struct mbigen_device *mgn_chip)
 {
 	struct platform_device *child;
-	struct irq_domain *domain;
 	struct device_node *np;
 	u32 num_pins;
 	int ret = 0;
@@ -258,11 +309,7 @@ static int mbigen_of_create_domain(struct platform_device *pdev,
 			break;
 		}
 
-		domain = platform_msi_create_device_domain(&child->dev, num_pins,
-							   mbigen_write_msg,
-							   &mbigen_domain_ops,
-							   mgn_chip);
-		if (!domain) {
+		if (!mbigen_create_device_domain(&child->dev, num_pins, mgn_chip)) {
 			ret = -ENOMEM;
 			break;
 		}
@@ -284,7 +331,6 @@ MODULE_DEVICE_TABLE(acpi, mbigen_acpi_match);
 static int mbigen_acpi_create_domain(struct platform_device *pdev,
 				     struct mbigen_device *mgn_chip)
 {
-	struct irq_domain *domain;
 	u32 num_pins = 0;
 	int ret;
 
@@ -315,11 +361,7 @@ static int mbigen_acpi_create_domain(struct platform_device *pdev,
 	if (ret || num_pins == 0)
 		return -EINVAL;
 
-	domain = platform_msi_create_device_domain(&pdev->dev, num_pins,
-						   mbigen_write_msg,
-						   &mbigen_domain_ops,
-						   mgn_chip);
-	if (!domain)
+	if (!mbigen_create_device_domain(&pdev->dev, num_pins, mgn_chip))
 		return -ENOMEM;
 
 	return 0;
Signature · 1 line
2.34.1
#32 lore raw

[tip: irq/core] irqchip/mbigen: Prepare for real per device MSI

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     dc34b1f74e0c97356d3ba00f1b33ea1e8ff774f1
Gitweb:        https://git.kernel.org/tip/dc34b1f74e0c97356d3ba00f1b33ea1e8ff774f1
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:43 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:23 +02:00

irqchip/mbigen: Prepare for real per device MSI

The core infrastructure has everything in place to switch MBIGEN to per device MSI domains and avoid the convoluted construct of the existing platform-MSI layering violation.

The new infrastructure provides a wired interrupt specific interface in the MSI core which converts the 'hardware interrupt number + trigger type' allocation which is required for wired interrupts in the regular irqdomain code to a normal MSI allocation.

The hardware interrupt number and the trigger type are stored in the MSI descriptor device cookie by the core code so the MBIGEN specific code can retrieve them.

The new per device domain is only instantiated when the irqdomain which is associated to the MBIGEN device provides MSI parent functionality. Up to that point it invokes the existing code. Once the parent is converted the code for the current platform-MSI mechanism is removed.

The new domain shares the interrupt chip callbacks and the translation function. The only new functionality aside of filling out the msi_domain_template is a domain specific set_desc() callback, which will go away once all platform-MSI code has been converted.

---
 drivers/irqchip/irq-mbigen.c | 98 +++++++++++++++++++++++++----------
 1 file changed, 70 insertions(+), 28 deletions(-)
drivers/irqchip/irq-mbigen.c +70 -28
index 58881d3..db0fa80 100644
@@ -135,24 +135,14 @@ static int mbigen_set_type(struct irq_data *data, unsigned int type)
 	return 0;
 }
 
-static struct irq_chip mbigen_irq_chip = {
-	.name =			"mbigen-v2",
-	.irq_mask =		irq_chip_mask_parent,
-	.irq_unmask =		irq_chip_unmask_parent,
-	.irq_eoi =		mbigen_eoi_irq,
-	.irq_set_type =		mbigen_set_type,
-	.irq_set_affinity =	irq_chip_set_affinity_parent,
-};
-
-static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
+static void mbigen_write_msi_msg(struct irq_data *d, struct msi_msg *msg)
 {
-	struct irq_data *d = irq_get_irq_data(desc->irq);
 	void __iomem *base = d->chip_data;
 	u32 val;
 
 	if (!msg->address_lo && !msg->address_hi)
 		return;
- 
+
 	base += get_mbigen_vec_reg(d->hwirq);
 	val = readl_relaxed(base);
 
@@ -165,10 +155,8 @@ static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
 	writel_relaxed(val, base);
 }
 
-static int mbigen_domain_translate(struct irq_domain *d,
-				    struct irq_fwspec *fwspec,
-				    unsigned long *hwirq,
-				    unsigned int *type)
+static int mbigen_domain_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
+				   unsigned long *hwirq, unsigned int *type)
 {
 	if (is_of_node(fwspec->fwnode) || is_acpi_device_node(fwspec->fwnode)) {
 		if (fwspec->param_count != 2)
@@ -192,6 +180,17 @@ static int mbigen_domain_translate(struct irq_domain *d,
 	return -EINVAL;
 }
 
+/* The following section will go away once ITS provides a MSI parent */
+
+static struct irq_chip mbigen_irq_chip = {
+	.name =			"mbigen-v2",
+	.irq_mask =		irq_chip_mask_parent,
+	.irq_unmask =		irq_chip_unmask_parent,
+	.irq_eoi =		mbigen_eoi_irq,
+	.irq_set_type =		mbigen_set_type,
+	.irq_set_affinity =	irq_chip_set_affinity_parent,
+};
+
 static int mbigen_irq_domain_alloc(struct irq_domain *domain,
 					unsigned int virq,
 					unsigned int nr_irqs,
@@ -232,11 +231,63 @@ static const struct irq_domain_ops mbigen_domain_ops = {
 	.free		= mbigen_irq_domain_free,
 };
 
+static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
+{
+	mbigen_write_msi_msg(irq_get_irq_data(desc->irq), msg);
+}
+
+/* End of to be removed section */
+
+static void mbigen_domain_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
+{
+	arg->desc = desc;
+	arg->hwirq = (u32)desc->data.icookie.value;
+}
+
+static const struct msi_domain_template mbigen_msi_template = {
+	.chip = {
+		.name			= "mbigen-v2",
+		.irq_mask		= irq_chip_mask_parent,
+		.irq_unmask		= irq_chip_unmask_parent,
+		.irq_eoi		= mbigen_eoi_irq,
+		.irq_set_type		= mbigen_set_type,
+		.irq_write_msi_msg	= mbigen_write_msi_msg,
+	},
+
+	.ops = {
+		.set_desc		= mbigen_domain_set_desc,
+		.msi_translate		= mbigen_domain_translate,
+	},
+
+	.info = {
+		.bus_token		= DOMAIN_BUS_WIRED_TO_MSI,
+		.flags			= MSI_FLAG_USE_DEV_FWNODE,
+	},
+};
+
+static bool mbigen_create_device_domain(struct device *dev, unsigned int size,
+					struct mbigen_device *mgn_chip)
+{
+	struct irq_domain *domain = dev->msi.domain;
+
+	if (WARN_ON_ONCE(!domain))
+		return false;
+
+	if (irq_domain_is_msi_parent(domain)) {
+		return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
+						    &mbigen_msi_template, size,
+						    NULL, mgn_chip->base);
+	}
+
+	/* Remove once ITS provides MSI parent */
+	return !!platform_msi_create_device_domain(dev, size, mbigen_write_msg,
+						   &mbigen_domain_ops, mgn_chip);
+}
+
 static int mbigen_of_create_domain(struct platform_device *pdev,
 				   struct mbigen_device *mgn_chip)
 {
 	struct platform_device *child;
-	struct irq_domain *domain;
 	struct device_node *np;
 	u32 num_pins;
 	int ret = 0;
@@ -258,11 +309,7 @@ static int mbigen_of_create_domain(struct platform_device *pdev,
 			break;
 		}
 
-		domain = platform_msi_create_device_domain(&child->dev, num_pins,
-							   mbigen_write_msg,
-							   &mbigen_domain_ops,
-							   mgn_chip);
-		if (!domain) {
+		if (!mbigen_create_device_domain(&child->dev, num_pins, mgn_chip)) {
 			ret = -ENOMEM;
 			break;
 		}
@@ -284,7 +331,6 @@ MODULE_DEVICE_TABLE(acpi, mbigen_acpi_match);
 static int mbigen_acpi_create_domain(struct platform_device *pdev,
 				     struct mbigen_device *mgn_chip)
 {
-	struct irq_domain *domain;
 	u32 num_pins = 0;
 	int ret;
 
@@ -315,11 +361,7 @@ static int mbigen_acpi_create_domain(struct platform_device *pdev,
 	if (ret || num_pins == 0)
 		return -EINVAL;
 
-	domain = platform_msi_create_device_domain(&pdev->dev, num_pins,
-						   mbigen_write_msg,
-						   &mbigen_domain_ops,
-						   mgn_chip);
-	if (!domain)
+	if (!mbigen_create_device_domain(&pdev->dev, num_pins, mgn_chip))
 		return -ENOMEM;
 
 	return 0;
#33 lore raw

[tip: irq/msi] irqchip/mbigen: Prepare for real per device MSI

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     fbfe7e13641efe811a273e1fd237929245376fe0
Gitweb:        https://git.kernel.org/tip/fbfe7e13641efe811a273e1fd237929245376fe0
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:43 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

irqchip/mbigen: Prepare for real per device MSI

The core infrastructure has everything in place to switch MBIGEN to per device MSI domains and avoid the convoluted construct of the existing platform-MSI layering violation.

The new infrastructure provides a wired interrupt specific interface in the MSI core which converts the 'hardware interrupt number + trigger type' allocation which is required for wired interrupts in the regular irqdomain code to a normal MSI allocation.

The hardware interrupt number and the trigger type are stored in the MSI descriptor device cookie by the core code so the MBIGEN specific code can retrieve them.

The new per device domain is only instantiated when the irqdomain which is associated to the MBIGEN device provides MSI parent functionality. Up to that point it invokes the existing code. Once the parent is converted the code for the current platform-MSI mechanism is removed.

The new domain shares the interrupt chip callbacks and the translation function. The only new functionality aside of filling out the msi_domain_template is a domain specific set_desc() callback, which will go away once all platform-MSI code has been converted.

---
 drivers/irqchip/irq-mbigen.c | 98 +++++++++++++++++++++++++----------
 1 file changed, 70 insertions(+), 28 deletions(-)
drivers/irqchip/irq-mbigen.c +70 -28
index 58881d3..db0fa80 100644
@@ -135,24 +135,14 @@ static int mbigen_set_type(struct irq_data *data, unsigned int type)
 	return 0;
 }
 
-static struct irq_chip mbigen_irq_chip = {
-	.name =			"mbigen-v2",
-	.irq_mask =		irq_chip_mask_parent,
-	.irq_unmask =		irq_chip_unmask_parent,
-	.irq_eoi =		mbigen_eoi_irq,
-	.irq_set_type =		mbigen_set_type,
-	.irq_set_affinity =	irq_chip_set_affinity_parent,
-};
-
-static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
+static void mbigen_write_msi_msg(struct irq_data *d, struct msi_msg *msg)
 {
-	struct irq_data *d = irq_get_irq_data(desc->irq);
 	void __iomem *base = d->chip_data;
 	u32 val;
 
 	if (!msg->address_lo && !msg->address_hi)
 		return;
- 
+
 	base += get_mbigen_vec_reg(d->hwirq);
 	val = readl_relaxed(base);
 
@@ -165,10 +155,8 @@ static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
 	writel_relaxed(val, base);
 }
 
-static int mbigen_domain_translate(struct irq_domain *d,
-				    struct irq_fwspec *fwspec,
-				    unsigned long *hwirq,
-				    unsigned int *type)
+static int mbigen_domain_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
+				   unsigned long *hwirq, unsigned int *type)
 {
 	if (is_of_node(fwspec->fwnode) || is_acpi_device_node(fwspec->fwnode)) {
 		if (fwspec->param_count != 2)
@@ -192,6 +180,17 @@ static int mbigen_domain_translate(struct irq_domain *d,
 	return -EINVAL;
 }
 
+/* The following section will go away once ITS provides a MSI parent */
+
+static struct irq_chip mbigen_irq_chip = {
+	.name =			"mbigen-v2",
+	.irq_mask =		irq_chip_mask_parent,
+	.irq_unmask =		irq_chip_unmask_parent,
+	.irq_eoi =		mbigen_eoi_irq,
+	.irq_set_type =		mbigen_set_type,
+	.irq_set_affinity =	irq_chip_set_affinity_parent,
+};
+
 static int mbigen_irq_domain_alloc(struct irq_domain *domain,
 					unsigned int virq,
 					unsigned int nr_irqs,
@@ -232,11 +231,63 @@ static const struct irq_domain_ops mbigen_domain_ops = {
 	.free		= mbigen_irq_domain_free,
 };
 
+static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
+{
+	mbigen_write_msi_msg(irq_get_irq_data(desc->irq), msg);
+}
+
+/* End of to be removed section */
+
+static void mbigen_domain_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
+{
+	arg->desc = desc;
+	arg->hwirq = (u32)desc->data.icookie.value;
+}
+
+static const struct msi_domain_template mbigen_msi_template = {
+	.chip = {
+		.name			= "mbigen-v2",
+		.irq_mask		= irq_chip_mask_parent,
+		.irq_unmask		= irq_chip_unmask_parent,
+		.irq_eoi		= mbigen_eoi_irq,
+		.irq_set_type		= mbigen_set_type,
+		.irq_write_msi_msg	= mbigen_write_msi_msg,
+	},
+
+	.ops = {
+		.set_desc		= mbigen_domain_set_desc,
+		.msi_translate		= mbigen_domain_translate,
+	},
+
+	.info = {
+		.bus_token		= DOMAIN_BUS_WIRED_TO_MSI,
+		.flags			= MSI_FLAG_USE_DEV_FWNODE,
+	},
+};
+
+static bool mbigen_create_device_domain(struct device *dev, unsigned int size,
+					struct mbigen_device *mgn_chip)
+{
+	struct irq_domain *domain = dev->msi.domain;
+
+	if (WARN_ON_ONCE(!domain))
+		return false;
+
+	if (irq_domain_is_msi_parent(domain)) {
+		return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
+						    &mbigen_msi_template, size,
+						    NULL, mgn_chip->base);
+	}
+
+	/* Remove once ITS provides MSI parent */
+	return !!platform_msi_create_device_domain(dev, size, mbigen_write_msg,
+						   &mbigen_domain_ops, mgn_chip);
+}
+
 static int mbigen_of_create_domain(struct platform_device *pdev,
 				   struct mbigen_device *mgn_chip)
 {
 	struct platform_device *child;
-	struct irq_domain *domain;
 	struct device_node *np;
 	u32 num_pins;
 	int ret = 0;
@@ -258,11 +309,7 @@ static int mbigen_of_create_domain(struct platform_device *pdev,
 			break;
 		}
 
-		domain = platform_msi_create_device_domain(&child->dev, num_pins,
-							   mbigen_write_msg,
-							   &mbigen_domain_ops,
-							   mgn_chip);
-		if (!domain) {
+		if (!mbigen_create_device_domain(&child->dev, num_pins, mgn_chip)) {
 			ret = -ENOMEM;
 			break;
 		}
@@ -284,7 +331,6 @@ MODULE_DEVICE_TABLE(acpi, mbigen_acpi_match);
 static int mbigen_acpi_create_domain(struct platform_device *pdev,
 				     struct mbigen_device *mgn_chip)
 {
-	struct irq_domain *domain;
 	u32 num_pins = 0;
 	int ret;
 
@@ -315,11 +361,7 @@ static int mbigen_acpi_create_domain(struct platform_device *pdev,
 	if (ret || num_pins == 0)
 		return -EINVAL;
 
-	domain = platform_msi_create_device_domain(&pdev->dev, num_pins,
-						   mbigen_write_msg,
-						   &mbigen_domain_ops,
-						   mgn_chip);
-	if (!domain)
+	if (!mbigen_create_device_domain(&pdev->dev, num_pins, mgn_chip))
 		return -ENOMEM;
 
 	return 0;
#34 lore raw

PATCH v4 8/21 irqchip/irq-msi-lib: Prepare for DOMAIN_BUS_WIRED_TO_MSI

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

Add the new bus token to the accepted list of child domain tokens.

---
 drivers/irqchip/irq-msi-lib.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/irqchip/irq-msi-lib.c +2 -0
index 6aa4974d2d12..40c19087d719 100644
@@ -67,6 +67,8 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 		/* Core managed MSI descriptors */
 		info->flags = MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS | MSI_FLAG_FREE_MSI_DESCS;
 		break;
+	case DOMAIN_BUS_WIRED_TO_MSI:
+		break;
 	default:
 		/*
 		 * This should never be reached. See
Signature · 1 line
2.34.1
#35 lore raw

[tip: irq/core] irqchip/irq-msi-lib: Prepare for DOMAIN_BUS_WIRED_TO_MSI

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     d318b8855a34e1ade18b69fde60b0c394aa0e462
Gitweb:        https://git.kernel.org/tip/d318b8855a34e1ade18b69fde60b0c394aa0e462
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:44 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:24 +02:00

irqchip/irq-msi-lib: Prepare for DOMAIN_BUS_WIRED_TO_MSI

Add the new bus token to the accepted list of child domain tokens.

---
 drivers/irqchip/irq-msi-lib.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/irqchip/irq-msi-lib.c +2 -0
index b98a219..d20a0e6 100644
@@ -69,6 +69,8 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 
 		/* Core managed MSI descriptors */
 		info->flags = MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS | MSI_FLAG_FREE_MSI_DESCS;
+		fallthrough;
+	case DOMAIN_BUS_WIRED_TO_MSI:
 		/* Remove PCI specific flags */
 		required_flags &= ~MSI_FLAG_PCI_MSI_MASK_PARENT;
 		break;
#36 lore raw

[tip: irq/msi] irqchip/irq-msi-lib: Prepare for DOMAIN_BUS_WIRED_TO_MSI

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     64a855324311ddad7a85fc0b25ce1f3914ed3425
Gitweb:        https://git.kernel.org/tip/64a855324311ddad7a85fc0b25ce1f3914ed3425
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:44 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

irqchip/irq-msi-lib: Prepare for DOMAIN_BUS_WIRED_TO_MSI

Add the new bus token to the accepted list of child domain tokens.

---
 drivers/irqchip/irq-msi-lib.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/irqchip/irq-msi-lib.c +2 -0
index b98a219..d20a0e6 100644
@@ -69,6 +69,8 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 
 		/* Core managed MSI descriptors */
 		info->flags = MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS | MSI_FLAG_FREE_MSI_DESCS;
+		fallthrough;
+	case DOMAIN_BUS_WIRED_TO_MSI:
 		/* Remove PCI specific flags */
 		required_flags &= ~MSI_FLAG_PCI_MSI_MASK_PARENT;
 		break;
#37 lore raw

PATCH v4 9/21 irqchip/gic-v3-its: Switch platform MSI to MSI parent

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

Similar to the previous conversion of the PCI/MSI support lift the prepare() callback from the existing platform MSI code and enable platform MSI and the related device domain bus tokens in select and the child domain initialization code.

All platform MSI users are automatically using the new per device MSI model now.

---
v3: adopted to IMS->MSI rename

---
 drivers/irqchip/Makefile                      |   2 +-
 drivers/irqchip/irq-gic-v3-its-msi-parent.c   |  73 +++++++-
 drivers/irqchip/irq-gic-v3-its-platform-msi.c | 163 ------------------
 3 files changed, 73 insertions(+), 165 deletions(-)
 delete mode 100644 drivers/irqchip/irq-gic-v3-its-platform-msi.c
drivers/irqchip/Makefile +1 -1
index c5316634637f..afc44f4709a6 100644
@@ -32,7 +32,7 @@ obj-$(CONFIG_ARCH_REALVIEW)		+= irq-gic-realview.o
 obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
-obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
+obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_FSL_MC)	+= irq-gic-v3-its-fsl-mc-msi.o
 obj-$(CONFIG_PARTITION_PERCPU)		+= irq-partition-percpu.o
 obj-$(CONFIG_HISILICON_IRQ_MBIGEN)	+= irq-mbigen.o
drivers/irqchip/irq-gic-v3-its-msi-parent.c +72 -1

97 lines not on this page · raw message

drivers/irqchip/irq-gic-v3-its-platform-msi.c deleted +0 -163

163 lines not on this page · raw message

Signature · 1 line
2.34.1
#38 lore raw

[tip: irq/core] irqchip/gic-v3-its: Switch platform MSI to MSI parent

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     3d1c927c08fcf579c0b73dd8b2d92329b2ca9712
Gitweb:        https://git.kernel.org/tip/3d1c927c08fcf579c0b73dd8b2d92329b2ca9712
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:46 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:24 +02:00

irqchip/gic-v3-its: Switch platform MSI to MSI parent

Similar to the previous conversion of the PCI/MSI support lift the prepare() callback from the existing platform MSI code and enable platform MSI and the related device domain bus tokens in select and the child domain initialization code.

All platform MSI users are automatically using the new per device MSI model now.

---
 drivers/irqchip/Makefile                      |   2 +-
 drivers/irqchip/irq-gic-v3-its-msi-parent.c   |  73 +++++++-
 drivers/irqchip/irq-gic-v3-its-platform-msi.c | 163 +-----------------
 3 files changed, 73 insertions(+), 165 deletions(-)
 delete mode 100644 drivers/irqchip/irq-gic-v3-its-platform-msi.c
drivers/irqchip/Makefile +1 -1
index ac0627a..1563581 100644
@@ -32,7 +32,7 @@ obj-$(CONFIG_ARCH_REALVIEW)		+= irq-gic-realview.o
 obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
-obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
+obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_FSL_MC)	+= irq-gic-v3-its-fsl-mc-msi.o
 obj-$(CONFIG_PARTITION_PERCPU)		+= irq-partition-percpu.o
 obj-$(CONFIG_HISILICON_IRQ_MBIGEN)	+= irq-mbigen.o
drivers/irqchip/irq-gic-v3-its-msi-parent.c +72 -1

97 lines not on this page · raw message

drivers/irqchip/irq-gic-v3-its-platform-msi.c deleted +0 -163

163 lines not on this page · raw message

#39 lore raw

[tip: irq/msi] irqchip/gic-v3-its: Switch platform MSI to MSI parent

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     7f2baef05d6aaae7bd6d0c863eeec50ae5b2997f
Gitweb:        https://git.kernel.org/tip/7f2baef05d6aaae7bd6d0c863eeec50ae5b2997f
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:46 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

irqchip/gic-v3-its: Switch platform MSI to MSI parent

Similar to the previous conversion of the PCI/MSI support lift the prepare() callback from the existing platform MSI code and enable platform MSI and the related device domain bus tokens in select and the child domain initialization code.

All platform MSI users are automatically using the new per device MSI model now.

---
 drivers/irqchip/Makefile                      |   2 +-
 drivers/irqchip/irq-gic-v3-its-msi-parent.c   |  73 +++++++-
 drivers/irqchip/irq-gic-v3-its-platform-msi.c | 163 +-----------------
 3 files changed, 73 insertions(+), 165 deletions(-)
 delete mode 100644 drivers/irqchip/irq-gic-v3-its-platform-msi.c
drivers/irqchip/Makefile +1 -1
index c531663..afc44f4 100644
@@ -32,7 +32,7 @@ obj-$(CONFIG_ARCH_REALVIEW)		+= irq-gic-realview.o
 obj-$(CONFIG_IRQ_MSI_LIB)		+= irq-msi-lib.o
 obj-$(CONFIG_ARM_GIC_V2M)		+= irq-gic-v2m.o
 obj-$(CONFIG_ARM_GIC_V3)		+= irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
-obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v3-its-platform-msi.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
+obj-$(CONFIG_ARM_GIC_V3_ITS)		+= irq-gic-v3-its.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
 obj-$(CONFIG_ARM_GIC_V3_ITS_FSL_MC)	+= irq-gic-v3-its-fsl-mc-msi.o
 obj-$(CONFIG_PARTITION_PERCPU)		+= irq-partition-percpu.o
 obj-$(CONFIG_HISILICON_IRQ_MBIGEN)	+= irq-mbigen.o
drivers/irqchip/irq-gic-v3-its-msi-parent.c +72 -1

97 lines not on this page · raw message

drivers/irqchip/irq-gic-v3-its-platform-msi.c deleted +0 -163

163 lines not on this page · raw message

#40 lore raw

PATCH v4 10/21 irqchip/mbigen: Remove platform_msi_create_device_domain() fallback

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

Now that ITS provides the MSI parent domain, remove the unused fallback code.

---
 drivers/irqchip/irq-mbigen.c | 74 ++----------------------------------
 1 file changed, 4 insertions(+), 70 deletions(-)
drivers/irqchip/irq-mbigen.c +4 -70
index db0fa80330d9..093fd42893a7 100644
@@ -180,64 +180,6 @@ static int mbigen_domain_translate(struct irq_domain *d, struct irq_fwspec *fwsp
 	return -EINVAL;
 }
 
-/* The following section will go away once ITS provides a MSI parent */
-
-static struct irq_chip mbigen_irq_chip = {
-	.name =			"mbigen-v2",
-	.irq_mask =		irq_chip_mask_parent,
-	.irq_unmask =		irq_chip_unmask_parent,
-	.irq_eoi =		mbigen_eoi_irq,
-	.irq_set_type =		mbigen_set_type,
-	.irq_set_affinity =	irq_chip_set_affinity_parent,
-};
-
-static int mbigen_irq_domain_alloc(struct irq_domain *domain,
-					unsigned int virq,
-					unsigned int nr_irqs,
-					void *args)
-{
-	struct irq_fwspec *fwspec = args;
-	irq_hw_number_t hwirq;
-	unsigned int type;
-	struct mbigen_device *mgn_chip;
-	int i, err;
-
-	err = mbigen_domain_translate(domain, fwspec, &hwirq, &type);
-	if (err)
-		return err;
-
-	err = platform_msi_device_domain_alloc(domain, virq, nr_irqs);
-	if (err)
-		return err;
-
-	mgn_chip = platform_msi_get_host_data(domain);
-
-	for (i = 0; i < nr_irqs; i++)
-		irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
-				      &mbigen_irq_chip, mgn_chip->base);
-
-	return 0;
-}
-
-static void mbigen_irq_domain_free(struct irq_domain *domain, unsigned int virq,
-				   unsigned int nr_irqs)
-{
-	platform_msi_device_domain_free(domain, virq, nr_irqs);
-}
-
-static const struct irq_domain_ops mbigen_domain_ops = {
-	.translate	= mbigen_domain_translate,
-	.alloc		= mbigen_irq_domain_alloc,
-	.free		= mbigen_irq_domain_free,
-};
-
-static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
-{
-	mbigen_write_msi_msg(irq_get_irq_data(desc->irq), msg);
-}
-
-/* End of to be removed section */
-
 static void mbigen_domain_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
 {
 	arg->desc = desc;
@@ -268,20 +210,12 @@ static const struct msi_domain_template mbigen_msi_template = {
 static bool mbigen_create_device_domain(struct device *dev, unsigned int size,
 					struct mbigen_device *mgn_chip)
 {
-	struct irq_domain *domain = dev->msi.domain;
-
-	if (WARN_ON_ONCE(!domain))
+	if (WARN_ON_ONCE(!dev->msi.domain))
 		return false;
 
-	if (irq_domain_is_msi_parent(domain)) {
-		return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
-						    &mbigen_msi_template, size,
-						    NULL, mgn_chip->base);
-	}
-
-	/* Remove once ITS provides MSI parent */
-	return !!platform_msi_create_device_domain(dev, size, mbigen_write_msg,
-						   &mbigen_domain_ops, mgn_chip);
+	return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
+					    &mbigen_msi_template, size,
+					    NULL, mgn_chip->base);
 }
 
 static int mbigen_of_create_domain(struct platform_device *pdev,
Signature · 1 line
2.34.1
#41 lore raw

Re: [patch V4 10/21] irqchip/mbigen: Remove platform_msi_create_device_domain() fallback

On Sun, Jun 23, 2024 at 05:18:48PM +0200, Thomas Gleixner wrote:

[+Hanjun]

Hanjun, are you able to test this series (or find someone who can) and in particular mbigen changes on affected HW and report back here please ?

Thanks,
Lorenzo

Quoted 113 lines

From: Thomas Gleixner <[email protected]>

Now that ITS provides the MSI parent domain, remove the unused fallback code.

---
 drivers/irqchip/irq-mbigen.c | 74 ++----------------------------------
 1 file changed, 4 insertions(+), 70 deletions(-)
drivers/irqchip/irq-mbigen.c +4 -70
index db0fa80330d9..093fd42893a7 100644
@@ -180,64 +180,6 @@ static int mbigen_domain_translate(struct irq_domain *d, struct irq_fwspec *fwsp
 	return -EINVAL;
 }
 
-/* The following section will go away once ITS provides a MSI parent */
-
-static struct irq_chip mbigen_irq_chip = {
-	.name =			"mbigen-v2",
-	.irq_mask =		irq_chip_mask_parent,
-	.irq_unmask =		irq_chip_unmask_parent,
-	.irq_eoi =		mbigen_eoi_irq,
-	.irq_set_type =		mbigen_set_type,
-	.irq_set_affinity =	irq_chip_set_affinity_parent,
-};
-
-static int mbigen_irq_domain_alloc(struct irq_domain *domain,
-					unsigned int virq,
-					unsigned int nr_irqs,
-					void *args)
-{
-	struct irq_fwspec *fwspec = args;
-	irq_hw_number_t hwirq;
-	unsigned int type;
-	struct mbigen_device *mgn_chip;
-	int i, err;
-
-	err = mbigen_domain_translate(domain, fwspec, &hwirq, &type);
-	if (err)
-		return err;
-
-	err = platform_msi_device_domain_alloc(domain, virq, nr_irqs);
-	if (err)
-		return err;
-
-	mgn_chip = platform_msi_get_host_data(domain);
-
-	for (i = 0; i < nr_irqs; i++)
-		irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
-				      &mbigen_irq_chip, mgn_chip->base);
-
-	return 0;
-}
-
-static void mbigen_irq_domain_free(struct irq_domain *domain, unsigned int virq,
-				   unsigned int nr_irqs)
-{
-	platform_msi_device_domain_free(domain, virq, nr_irqs);
-}
-
-static const struct irq_domain_ops mbigen_domain_ops = {
-	.translate	= mbigen_domain_translate,
-	.alloc		= mbigen_irq_domain_alloc,
-	.free		= mbigen_irq_domain_free,
-};
-
-static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
-{
-	mbigen_write_msi_msg(irq_get_irq_data(desc->irq), msg);
-}
-
-/* End of to be removed section */
-
 static void mbigen_domain_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
 {
 	arg->desc = desc;
@@ -268,20 +210,12 @@ static const struct msi_domain_template mbigen_msi_template = {
 static bool mbigen_create_device_domain(struct device *dev, unsigned int size,
 					struct mbigen_device *mgn_chip)
 {
-	struct irq_domain *domain = dev->msi.domain;
-
-	if (WARN_ON_ONCE(!domain))
+	if (WARN_ON_ONCE(!dev->msi.domain))
 		return false;
 
-	if (irq_domain_is_msi_parent(domain)) {
-		return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
-						    &mbigen_msi_template, size,
-						    NULL, mgn_chip->base);
-	}
-
-	/* Remove once ITS provides MSI parent */
-	return !!platform_msi_create_device_domain(dev, size, mbigen_write_msg,
-						   &mbigen_domain_ops, mgn_chip);
+	return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
+					    &mbigen_msi_template, size,
+					    NULL, mgn_chip->base);
 }
 
 static int mbigen_of_create_domain(struct platform_device *pdev,
Signature · 1 line
2.34.1
#42 lore raw

Re: [patch V4 10/21] irqchip/mbigen: Remove platform_msi_create_device_domain() fallback

Quoted 3 lines · Lorenzo Pieralisi

On Sun, Jun 23, 2024 at 05:18:48PM +0200, Thomas Gleixner wrote:

[+Hanjun]

Thanks for let me know.

Quoted 3 lines

Hanjun, are you able to test this series (or find someone who can) and in particular mbigen changes on affected HW and report back here please ?

I tested on a Kunpeng920 ARM server (with this patch set on top of 6.10-rc3), which the mbigen interrupts are used by SoC PMU counters, I can see that the PMU counters can generate overflow interrupts as before, it works for me.

Thanks
Hanjun

#43 lore raw

[tip: irq/core] irqchip/mbigen: Remove platform_msi_create_device_domain() fallback

In reply to Thomas Gleixner
The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     b040ac38c07737eaf9a9237db07da5702ce4fe5c
Gitweb:        https://git.kernel.org/tip/b040ac38c07737eaf9a9237db07da5702ce4fe5c
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:48 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:24 +02:00

irqchip/mbigen: Remove platform_msi_create_device_domain() fallback

Now that ITS provides the MSI parent domain, remove the unused fallback code.

---
 drivers/irqchip/irq-mbigen.c | 74 +----------------------------------
 1 file changed, 4 insertions(+), 70 deletions(-)
drivers/irqchip/irq-mbigen.c +4 -70
index db0fa80..093fd42 100644
@@ -180,64 +180,6 @@ static int mbigen_domain_translate(struct irq_domain *d, struct irq_fwspec *fwsp
 	return -EINVAL;
 }
 
-/* The following section will go away once ITS provides a MSI parent */
-
-static struct irq_chip mbigen_irq_chip = {
-	.name =			"mbigen-v2",
-	.irq_mask =		irq_chip_mask_parent,
-	.irq_unmask =		irq_chip_unmask_parent,
-	.irq_eoi =		mbigen_eoi_irq,
-	.irq_set_type =		mbigen_set_type,
-	.irq_set_affinity =	irq_chip_set_affinity_parent,
-};
-
-static int mbigen_irq_domain_alloc(struct irq_domain *domain,
-					unsigned int virq,
-					unsigned int nr_irqs,
-					void *args)
-{
-	struct irq_fwspec *fwspec = args;
-	irq_hw_number_t hwirq;
-	unsigned int type;
-	struct mbigen_device *mgn_chip;
-	int i, err;
-
-	err = mbigen_domain_translate(domain, fwspec, &hwirq, &type);
-	if (err)
-		return err;
-
-	err = platform_msi_device_domain_alloc(domain, virq, nr_irqs);
-	if (err)
-		return err;
-
-	mgn_chip = platform_msi_get_host_data(domain);
-
-	for (i = 0; i < nr_irqs; i++)
-		irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
-				      &mbigen_irq_chip, mgn_chip->base);
-
-	return 0;
-}
-
-static void mbigen_irq_domain_free(struct irq_domain *domain, unsigned int virq,
-				   unsigned int nr_irqs)
-{
-	platform_msi_device_domain_free(domain, virq, nr_irqs);
-}
-
-static const struct irq_domain_ops mbigen_domain_ops = {
-	.translate	= mbigen_domain_translate,
-	.alloc		= mbigen_irq_domain_alloc,
-	.free		= mbigen_irq_domain_free,
-};
-
-static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
-{
-	mbigen_write_msi_msg(irq_get_irq_data(desc->irq), msg);
-}
-
-/* End of to be removed section */
-
 static void mbigen_domain_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
 {
 	arg->desc = desc;
@@ -268,20 +210,12 @@ static const struct msi_domain_template mbigen_msi_template = {
 static bool mbigen_create_device_domain(struct device *dev, unsigned int size,
 					struct mbigen_device *mgn_chip)
 {
-	struct irq_domain *domain = dev->msi.domain;
-
-	if (WARN_ON_ONCE(!domain))
+	if (WARN_ON_ONCE(!dev->msi.domain))
 		return false;
 
-	if (irq_domain_is_msi_parent(domain)) {
-		return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
-						    &mbigen_msi_template, size,
-						    NULL, mgn_chip->base);
-	}
-
-	/* Remove once ITS provides MSI parent */
-	return !!platform_msi_create_device_domain(dev, size, mbigen_write_msg,
-						   &mbigen_domain_ops, mgn_chip);
+	return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
+					    &mbigen_msi_template, size,
+					    NULL, mgn_chip->base);
 }
 
 static int mbigen_of_create_domain(struct platform_device *pdev,
#44 lore raw

[tip: irq/msi] irqchip/mbigen: Remove platform_msi_create_device_domain() fallback

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     752e021f5b9be0ad42752deaa241ae631f293f9f
Gitweb:        https://git.kernel.org/tip/752e021f5b9be0ad42752deaa241ae631f293f9f
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:48 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

irqchip/mbigen: Remove platform_msi_create_device_domain() fallback

Now that ITS provides the MSI parent domain, remove the unused fallback code.

---
 drivers/irqchip/irq-mbigen.c | 74 +----------------------------------
 1 file changed, 4 insertions(+), 70 deletions(-)
drivers/irqchip/irq-mbigen.c +4 -70
index db0fa80..093fd42 100644
@@ -180,64 +180,6 @@ static int mbigen_domain_translate(struct irq_domain *d, struct irq_fwspec *fwsp
 	return -EINVAL;
 }
 
-/* The following section will go away once ITS provides a MSI parent */
-
-static struct irq_chip mbigen_irq_chip = {
-	.name =			"mbigen-v2",
-	.irq_mask =		irq_chip_mask_parent,
-	.irq_unmask =		irq_chip_unmask_parent,
-	.irq_eoi =		mbigen_eoi_irq,
-	.irq_set_type =		mbigen_set_type,
-	.irq_set_affinity =	irq_chip_set_affinity_parent,
-};
-
-static int mbigen_irq_domain_alloc(struct irq_domain *domain,
-					unsigned int virq,
-					unsigned int nr_irqs,
-					void *args)
-{
-	struct irq_fwspec *fwspec = args;
-	irq_hw_number_t hwirq;
-	unsigned int type;
-	struct mbigen_device *mgn_chip;
-	int i, err;
-
-	err = mbigen_domain_translate(domain, fwspec, &hwirq, &type);
-	if (err)
-		return err;
-
-	err = platform_msi_device_domain_alloc(domain, virq, nr_irqs);
-	if (err)
-		return err;
-
-	mgn_chip = platform_msi_get_host_data(domain);
-
-	for (i = 0; i < nr_irqs; i++)
-		irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
-				      &mbigen_irq_chip, mgn_chip->base);
-
-	return 0;
-}
-
-static void mbigen_irq_domain_free(struct irq_domain *domain, unsigned int virq,
-				   unsigned int nr_irqs)
-{
-	platform_msi_device_domain_free(domain, virq, nr_irqs);
-}
-
-static const struct irq_domain_ops mbigen_domain_ops = {
-	.translate	= mbigen_domain_translate,
-	.alloc		= mbigen_irq_domain_alloc,
-	.free		= mbigen_irq_domain_free,
-};
-
-static void mbigen_write_msg(struct msi_desc *desc, struct msi_msg *msg)
-{
-	mbigen_write_msi_msg(irq_get_irq_data(desc->irq), msg);
-}
-
-/* End of to be removed section */
-
 static void mbigen_domain_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
 {
 	arg->desc = desc;
@@ -268,20 +210,12 @@ static const struct msi_domain_template mbigen_msi_template = {
 static bool mbigen_create_device_domain(struct device *dev, unsigned int size,
 					struct mbigen_device *mgn_chip)
 {
-	struct irq_domain *domain = dev->msi.domain;
-
-	if (WARN_ON_ONCE(!domain))
+	if (WARN_ON_ONCE(!dev->msi.domain))
 		return false;
 
-	if (irq_domain_is_msi_parent(domain)) {
-		return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
-						    &mbigen_msi_template, size,
-						    NULL, mgn_chip->base);
-	}
-
-	/* Remove once ITS provides MSI parent */
-	return !!platform_msi_create_device_domain(dev, size, mbigen_write_msg,
-						   &mbigen_domain_ops, mgn_chip);
+	return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
+					    &mbigen_msi_template, size,
+					    NULL, mgn_chip->base);
 }
 
 static int mbigen_of_create_domain(struct platform_device *pdev,
#45 lore raw

PATCH v4 11/21 genirq/msi: Remove platform_msi_create_device_domain()

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

No more users.

---
 include/linux/msi.h | 2 --
 1 file changed, 2 deletions(-)
include/linux/msi.h +0 -2
index 04f33e7f6f8b..4ae036d0c7db 100644
@@ -660,8 +660,6 @@ __platform_msi_create_device_domain(struct device *dev,
 				    const struct irq_domain_ops *ops,
 				    void *host_data);
 
-#define platform_msi_create_device_domain(dev, nvec, write, ops, data)	\
-	__platform_msi_create_device_domain(dev, nvec, false, write, ops, data)
 #define platform_msi_create_device_tree_domain(dev, nvec, write, ops, data) \
 	__platform_msi_create_device_domain(dev, nvec, true, write, ops, data)
 
Signature · 1 line
2.34.1
#46 lore raw

[tip: irq/core] genirq/msi: Remove platform_msi_create_device_domain()

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     808d004f0784dd5706bf725cadd7193cc1e4a442
Gitweb:        https://git.kernel.org/tip/808d004f0784dd5706bf725cadd7193cc1e4a442
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:50 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:24 +02:00

genirq/msi: Remove platform_msi_create_device_domain()

No more users.

---
 include/linux/msi.h | 2 --
 1 file changed, 2 deletions(-)
include/linux/msi.h +0 -2
index 04f33e7..4ae036d 100644
@@ -660,8 +660,6 @@ __platform_msi_create_device_domain(struct device *dev,
 				    const struct irq_domain_ops *ops,
 				    void *host_data);
 
-#define platform_msi_create_device_domain(dev, nvec, write, ops, data)	\
-	__platform_msi_create_device_domain(dev, nvec, false, write, ops, data)
 #define platform_msi_create_device_tree_domain(dev, nvec, write, ops, data) \
 	__platform_msi_create_device_domain(dev, nvec, true, write, ops, data)
 
#47 lore raw

[tip: irq/msi] genirq/msi: Remove platform_msi_create_device_domain()

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     f6a9886a9e55a1e6bd78c7e505205d05ef50a71e
Gitweb:        https://git.kernel.org/tip/f6a9886a9e55a1e6bd78c7e505205d05ef50a71e
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:50 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

genirq/msi: Remove platform_msi_create_device_domain()

No more users.

---
 include/linux/msi.h | 2 --
 1 file changed, 2 deletions(-)
include/linux/msi.h +0 -2
index 04f33e7..4ae036d 100644
@@ -660,8 +660,6 @@ __platform_msi_create_device_domain(struct device *dev,
 				    const struct irq_domain_ops *ops,
 				    void *host_data);
 
-#define platform_msi_create_device_domain(dev, nvec, write, ops, data)	\
-	__platform_msi_create_device_domain(dev, nvec, false, write, ops, data)
 #define platform_msi_create_device_tree_domain(dev, nvec, write, ops, data) \
 	__platform_msi_create_device_domain(dev, nvec, true, write, ops, data)
 
#48 lore raw

PATCH v4 12/21 irqchip/gic_v3_mbi: Switch over to parent domain

In reply to Thomas Gleixner

The MBI chip creates two MSI domains:
- PCI/MSI
- Platform device domain

Both have the MBI domain as parent and differ slightly in the interrupt chip callbacks and the platform device domain supports level type signaling.

Convert it over to the MSI parent domain mechanism by:

- Providing the required templates

- Implementing a custom init_dev_msi_info() callback which sets the chip callbacks and the level support flags depending on the domain bus token type of the per device domain.

---
V3-1: New patch to replace the broken conversion - Marc

---
 drivers/irqchip/irq-gic-v3-mbi.c |  128 ++++++++++++++-------------------------
 1 file changed, 46 insertions(+), 82 deletions(-)
drivers/irqchip/irq-gic-v3-mbi.c +46 -82
@@ -18,6 +18,8 @@
 
 #include <linux/irqchip/arm-gic-v3.h>
 
+#include "irq-msi-lib.h"
+
 struct mbi_range {
 	u32			spi_start;
 	u32			nr_spis;
@@ -138,6 +140,7 @@ static void mbi_irq_domain_free(struct i
 }
 
 static const struct irq_domain_ops mbi_domain_ops = {
+	.select			= msi_lib_irq_domain_select,
 	.alloc			= mbi_irq_domain_alloc,
 	.free			= mbi_irq_domain_free,
 };
@@ -151,54 +154,6 @@ static void mbi_compose_msi_msg(struct i
 	iommu_dma_compose_msi_msg(irq_data_get_msi_desc(data), msg);
 }
 
-#ifdef CONFIG_PCI_MSI
-/* PCI-specific irqchip */
-static void mbi_mask_msi_irq(struct irq_data *d)
-{
-	pci_msi_mask_irq(d);
-	irq_chip_mask_parent(d);
-}
-
-static void mbi_unmask_msi_irq(struct irq_data *d)
-{
-	pci_msi_unmask_irq(d);
-	irq_chip_unmask_parent(d);
-}
-
-static struct irq_chip mbi_msi_irq_chip = {
-	.name			= "MSI",
-	.irq_mask		= mbi_mask_msi_irq,
-	.irq_unmask		= mbi_unmask_msi_irq,
-	.irq_eoi		= irq_chip_eoi_parent,
-	.irq_compose_msi_msg	= mbi_compose_msi_msg,
-};
-
-static struct msi_domain_info mbi_msi_domain_info = {
-	.flags	= (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
-		   MSI_FLAG_PCI_MSIX | MSI_FLAG_MULTI_PCI_MSI),
-	.chip	= &mbi_msi_irq_chip,
-};
-
-static int mbi_allocate_pci_domain(struct irq_domain *nexus_domain,
-				   struct irq_domain **pci_domain)
-{
-	*pci_domain = pci_msi_create_irq_domain(nexus_domain->parent->fwnode,
-						&mbi_msi_domain_info,
-						nexus_domain);
-	if (!*pci_domain)
-		return -ENOMEM;
-
-	return 0;
-}
-#else
-static int mbi_allocate_pci_domain(struct irq_domain *nexus_domain,
-				   struct irq_domain **pci_domain)
-{
-	*pci_domain = NULL;
-	return 0;
-}
-#endif
-
 static void mbi_compose_mbi_msg(struct irq_data *data, struct msi_msg *msg)
 {
 	mbi_compose_msi_msg(data, msg);
@@ -210,28 +165,51 @@ static void mbi_compose_mbi_msg(struct i
 	iommu_dma_compose_msi_msg(irq_data_get_msi_desc(data), &msg[1]);
 }
 
-/* Platform-MSI specific irqchip */
-static struct irq_chip mbi_pmsi_irq_chip = {
-	.name			= "pMSI",
-	.irq_set_type		= irq_chip_set_type_parent,
-	.irq_compose_msi_msg	= mbi_compose_mbi_msg,
-	.flags			= IRQCHIP_SUPPORTS_LEVEL_MSI,
-};
+static bool mbi_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+				  struct irq_domain *real_parent, struct msi_domain_info *info)
+{
+	if (!msi_lib_init_dev_msi_info(dev, domain, real_parent, info))
+		return false;
 
-static struct msi_domain_ops mbi_pmsi_ops = {
-};
+	switch (info->bus_token) {
+	case DOMAIN_BUS_PCI_DEVICE_MSI:
+	case DOMAIN_BUS_PCI_DEVICE_MSIX:
+		info->chip->irq_compose_msi_msg = mbi_compose_msi_msg;
+		return true;
+
+	case DOMAIN_BUS_DEVICE_MSI:
+		info->chip->irq_compose_msi_msg = mbi_compose_mbi_msg;
+		info->chip->irq_set_type = irq_chip_set_type_parent;
+		info->chip->flags |= IRQCHIP_SUPPORTS_LEVEL_MSI;
+		info->flags |= MSI_FLAG_LEVEL_CAPABLE;
+		return true;
+
+	default:
+		WARN_ON_ONCE(1);
+		return false;
+	}
+}
+
+#define MBI_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
+				 MSI_FLAG_USE_DEF_CHIP_OPS)
 
-static struct msi_domain_info mbi_pmsi_domain_info = {
-	.flags	= (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
-		   MSI_FLAG_LEVEL_CAPABLE),
-	.ops	= &mbi_pmsi_ops,
-	.chip	= &mbi_pmsi_irq_chip,
+#define MBI_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK |	\
+				 MSI_FLAG_PCI_MSIX      |	\
+				 MSI_FLAG_MULTI_PCI_MSI |	\
+				 MSI_FLAG_PCI_MSI_MASK_PARENT)
+
+static const struct msi_parent_ops gic_v3_mbi_msi_parent_ops = {
+	.supported_flags	= MBI_MSI_FLAGS_SUPPORTED,
+	.required_flags		= MBI_MSI_FLAGS_REQUIRED,
+	.bus_select_token	= DOMAIN_BUS_NEXUS,
+	.bus_select_mask	= MATCH_PCI_MSI | MATCH_PLATFORM_MSI,
+	.prefix			= "MBI-",
+	.init_dev_msi_info	= mbi_init_dev_msi_info,
 };
 
-static int mbi_allocate_domains(struct irq_domain *parent)
+static int mbi_allocate_domain(struct irq_domain *parent)
 {
-	struct irq_domain *nexus_domain, *pci_domain, *plat_domain;
-	int err;
+	struct irq_domain *nexus_domain;
 
 	nexus_domain = irq_domain_create_hierarchy(parent, 0, 0, parent->fwnode,
 						   &mbi_domain_ops, NULL);
@@ -239,22 +217,8 @@ static int mbi_allocate_domains(struct i
 		return -ENOMEM;
 
 	irq_domain_update_bus_token(nexus_domain, DOMAIN_BUS_NEXUS);
-
-	err = mbi_allocate_pci_domain(nexus_domain, &pci_domain);
-
-	plat_domain = platform_msi_create_irq_domain(parent->fwnode,
-						     &mbi_pmsi_domain_info,
-						     nexus_domain);
-
-	if (err || !plat_domain) {
-		if (plat_domain)
-			irq_domain_remove(plat_domain);
-		if (pci_domain)
-			irq_domain_remove(pci_domain);
-		irq_domain_remove(nexus_domain);
-		return -ENOMEM;
-	}
-
+	nexus_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+	nexus_domain->msi_parent_ops = &gic_v3_mbi_msi_parent_ops;
 	return 0;
 }
 
@@ -317,7 +281,7 @@ int __init mbi_init(struct fwnode_handle
 
 	pr_info("Using MBI frame %pa\n", &mbi_phys_base);
 
-	ret = mbi_allocate_domains(parent);
+	ret = mbi_allocate_domain(parent);
 	if (ret)
 		goto err_free_mbi;
 
#49 lore raw

[tip: irq/core] irqchip/gic_v3_mbi: Switch over to parent domain

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     2e63b1cc9cf75c6c213ae0b9b9cec97111ea64a9
Gitweb:        https://git.kernel.org/tip/2e63b1cc9cf75c6c213ae0b9b9cec97111ea64a9
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:51 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:24 +02:00

irqchip/gic_v3_mbi: Switch over to parent domain

The MBI chip creates two MSI domains:
- PCI/MSI
- Platform device domain

Both have the MBI domain as parent and differ slightly in the interrupt chip callbacks and the platform device domain supports level type signaling.

Convert it over to the MSI parent domain mechanism by:

- Providing the required templates

- Implementing a custom init_dev_msi_info() callback which sets the chip callbacks and the level support flags depending on the domain bus token type of the per device domain.

---
 drivers/irqchip/irq-gic-v3-mbi.c | 130 ++++++++++--------------------
 1 file changed, 47 insertions(+), 83 deletions(-)
drivers/irqchip/irq-gic-v3-mbi.c +47 -83
index dbb8b1e..3fe870f 100644
@@ -18,6 +18,8 @@
 
 #include <linux/irqchip/arm-gic-v3.h>
 
+#include "irq-msi-lib.h"
+
 struct mbi_range {
 	u32			spi_start;
 	u32			nr_spis;
@@ -138,6 +140,7 @@ static void mbi_irq_domain_free(struct irq_domain *domain,
 }
 
 static const struct irq_domain_ops mbi_domain_ops = {
+	.select			= msi_lib_irq_domain_select,
 	.alloc			= mbi_irq_domain_alloc,
 	.free			= mbi_irq_domain_free,
 };
@@ -151,54 +154,6 @@ static void mbi_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
 	iommu_dma_compose_msi_msg(irq_data_get_msi_desc(data), msg);
 }
 
-#ifdef CONFIG_PCI_MSI
-/* PCI-specific irqchip */
-static void mbi_mask_msi_irq(struct irq_data *d)
-{
-	pci_msi_mask_irq(d);
-	irq_chip_mask_parent(d);
-}
-
-static void mbi_unmask_msi_irq(struct irq_data *d)
-{
-	pci_msi_unmask_irq(d);
-	irq_chip_unmask_parent(d);
-}
-
-static struct irq_chip mbi_msi_irq_chip = {
-	.name			= "MSI",
-	.irq_mask		= mbi_mask_msi_irq,
-	.irq_unmask		= mbi_unmask_msi_irq,
-	.irq_eoi		= irq_chip_eoi_parent,
-	.irq_compose_msi_msg	= mbi_compose_msi_msg,
-};
-
-static struct msi_domain_info mbi_msi_domain_info = {
-	.flags	= (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
-		   MSI_FLAG_PCI_MSIX | MSI_FLAG_MULTI_PCI_MSI),
-	.chip	= &mbi_msi_irq_chip,
-};
-
-static int mbi_allocate_pci_domain(struct irq_domain *nexus_domain,
-				   struct irq_domain **pci_domain)
-{
-	*pci_domain = pci_msi_create_irq_domain(nexus_domain->parent->fwnode,
-						&mbi_msi_domain_info,
-						nexus_domain);
-	if (!*pci_domain)
-		return -ENOMEM;
-
-	return 0;
-}
-#else
-static int mbi_allocate_pci_domain(struct irq_domain *nexus_domain,
-				   struct irq_domain **pci_domain)
-{
-	*pci_domain = NULL;
-	return 0;
-}
-#endif
-
 static void mbi_compose_mbi_msg(struct irq_data *data, struct msi_msg *msg)
 {
 	mbi_compose_msi_msg(data, msg);
@@ -210,28 +165,51 @@ static void mbi_compose_mbi_msg(struct irq_data *data, struct msi_msg *msg)
 	iommu_dma_compose_msi_msg(irq_data_get_msi_desc(data), &msg[1]);
 }
 
-/* Platform-MSI specific irqchip */
-static struct irq_chip mbi_pmsi_irq_chip = {
-	.name			= "pMSI",
-	.irq_set_type		= irq_chip_set_type_parent,
-	.irq_compose_msi_msg	= mbi_compose_mbi_msg,
-	.flags			= IRQCHIP_SUPPORTS_LEVEL_MSI,
-};
-
-static struct msi_domain_ops mbi_pmsi_ops = {
-};
+static bool mbi_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+				  struct irq_domain *real_parent, struct msi_domain_info *info)
+{
+	if (!msi_lib_init_dev_msi_info(dev, domain, real_parent, info))
+		return false;
+
+	switch (info->bus_token) {
+	case DOMAIN_BUS_PCI_DEVICE_MSI:
+	case DOMAIN_BUS_PCI_DEVICE_MSIX:
+		info->chip->irq_compose_msi_msg = mbi_compose_msi_msg;
+		return true;
+
+	case DOMAIN_BUS_DEVICE_MSI:
+		info->chip->irq_compose_msi_msg = mbi_compose_mbi_msg;
+		info->chip->irq_set_type = irq_chip_set_type_parent;
+		info->chip->flags |= IRQCHIP_SUPPORTS_LEVEL_MSI;
+		info->flags |= MSI_FLAG_LEVEL_CAPABLE;
+		return true;
+
+	default:
+		WARN_ON_ONCE(1);
+		return false;
+	}
+}
 
-static struct msi_domain_info mbi_pmsi_domain_info = {
-	.flags	= (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
-		   MSI_FLAG_LEVEL_CAPABLE),
-	.ops	= &mbi_pmsi_ops,
-	.chip	= &mbi_pmsi_irq_chip,
+#define MBI_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
+				 MSI_FLAG_USE_DEF_CHIP_OPS |	\
+				 MSI_FLAG_PCI_MSI_MASK_PARENT)
+
+#define MBI_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK |	\
+				 MSI_FLAG_PCI_MSIX      |	\
+				 MSI_FLAG_MULTI_PCI_MSI)
+
+static const struct msi_parent_ops gic_v3_mbi_msi_parent_ops = {
+	.supported_flags	= MBI_MSI_FLAGS_SUPPORTED,
+	.required_flags		= MBI_MSI_FLAGS_REQUIRED,
+	.bus_select_token	= DOMAIN_BUS_NEXUS,
+	.bus_select_mask	= MATCH_PCI_MSI | MATCH_PLATFORM_MSI,
+	.prefix			= "MBI-",
+	.init_dev_msi_info	= mbi_init_dev_msi_info,
 };
 
-static int mbi_allocate_domains(struct irq_domain *parent)
+static int mbi_allocate_domain(struct irq_domain *parent)
 {
-	struct irq_domain *nexus_domain, *pci_domain, *plat_domain;
-	int err;
+	struct irq_domain *nexus_domain;
 
 	nexus_domain = irq_domain_create_hierarchy(parent, 0, 0, parent->fwnode,
 						   &mbi_domain_ops, NULL);
@@ -239,22 +217,8 @@ static int mbi_allocate_domains(struct irq_domain *parent)
 		return -ENOMEM;
 
 	irq_domain_update_bus_token(nexus_domain, DOMAIN_BUS_NEXUS);
-
-	err = mbi_allocate_pci_domain(nexus_domain, &pci_domain);
-
-	plat_domain = platform_msi_create_irq_domain(parent->fwnode,
-						     &mbi_pmsi_domain_info,
-						     nexus_domain);
-
-	if (err || !plat_domain) {
-		if (plat_domain)
-			irq_domain_remove(plat_domain);
-		if (pci_domain)
-			irq_domain_remove(pci_domain);
-		irq_domain_remove(nexus_domain);
-		return -ENOMEM;
-	}
-
+	nexus_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+	nexus_domain->msi_parent_ops = &gic_v3_mbi_msi_parent_ops;
 	return 0;
 }
 
@@ -317,7 +281,7 @@ int __init mbi_init(struct fwnode_handle *fwnode, struct irq_domain *parent)
 
 	pr_info("Using MBI frame %pa\n", &mbi_phys_base);
 
-	ret = mbi_allocate_domains(parent);
+	ret = mbi_allocate_domain(parent);
 	if (ret)
 		goto err_free_mbi;
 
#50 lore raw

[tip: irq/msi] irqchip/gic_v3_mbi: Switch over to parent domain

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     da8ec7956efb5dc2be87f5f3beb3c9ed74316baf
Gitweb:        https://git.kernel.org/tip/da8ec7956efb5dc2be87f5f3beb3c9ed74316baf
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:51 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

irqchip/gic_v3_mbi: Switch over to parent domain

The MBI chip creates two MSI domains:
- PCI/MSI
- Platform device domain

Both have the MBI domain as parent and differ slightly in the interrupt chip callbacks and the platform device domain supports level type signaling.

Convert it over to the MSI parent domain mechanism by:

- Providing the required templates

- Implementing a custom init_dev_msi_info() callback which sets the chip callbacks and the level support flags depending on the domain bus token type of the per device domain.

---
 drivers/irqchip/irq-gic-v3-mbi.c | 130 ++++++++++--------------------
 1 file changed, 47 insertions(+), 83 deletions(-)
drivers/irqchip/irq-gic-v3-mbi.c +47 -83
index dbb8b1e..3fe870f 100644
@@ -18,6 +18,8 @@
 
 #include <linux/irqchip/arm-gic-v3.h>
 
+#include "irq-msi-lib.h"
+
 struct mbi_range {
 	u32			spi_start;
 	u32			nr_spis;
@@ -138,6 +140,7 @@ static void mbi_irq_domain_free(struct irq_domain *domain,
 }
 
 static const struct irq_domain_ops mbi_domain_ops = {
+	.select			= msi_lib_irq_domain_select,
 	.alloc			= mbi_irq_domain_alloc,
 	.free			= mbi_irq_domain_free,
 };
@@ -151,54 +154,6 @@ static void mbi_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
 	iommu_dma_compose_msi_msg(irq_data_get_msi_desc(data), msg);
 }
 
-#ifdef CONFIG_PCI_MSI
-/* PCI-specific irqchip */
-static void mbi_mask_msi_irq(struct irq_data *d)
-{
-	pci_msi_mask_irq(d);
-	irq_chip_mask_parent(d);
-}
-
-static void mbi_unmask_msi_irq(struct irq_data *d)
-{
-	pci_msi_unmask_irq(d);
-	irq_chip_unmask_parent(d);
-}
-
-static struct irq_chip mbi_msi_irq_chip = {
-	.name			= "MSI",
-	.irq_mask		= mbi_mask_msi_irq,
-	.irq_unmask		= mbi_unmask_msi_irq,
-	.irq_eoi		= irq_chip_eoi_parent,
-	.irq_compose_msi_msg	= mbi_compose_msi_msg,
-};
-
-static struct msi_domain_info mbi_msi_domain_info = {
-	.flags	= (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
-		   MSI_FLAG_PCI_MSIX | MSI_FLAG_MULTI_PCI_MSI),
-	.chip	= &mbi_msi_irq_chip,
-};
-
-static int mbi_allocate_pci_domain(struct irq_domain *nexus_domain,
-				   struct irq_domain **pci_domain)
-{
-	*pci_domain = pci_msi_create_irq_domain(nexus_domain->parent->fwnode,
-						&mbi_msi_domain_info,
-						nexus_domain);
-	if (!*pci_domain)
-		return -ENOMEM;
-
-	return 0;
-}
-#else
-static int mbi_allocate_pci_domain(struct irq_domain *nexus_domain,
-				   struct irq_domain **pci_domain)
-{
-	*pci_domain = NULL;
-	return 0;
-}
-#endif
-
 static void mbi_compose_mbi_msg(struct irq_data *data, struct msi_msg *msg)
 {
 	mbi_compose_msi_msg(data, msg);
@@ -210,28 +165,51 @@ static void mbi_compose_mbi_msg(struct irq_data *data, struct msi_msg *msg)
 	iommu_dma_compose_msi_msg(irq_data_get_msi_desc(data), &msg[1]);
 }
 
-/* Platform-MSI specific irqchip */
-static struct irq_chip mbi_pmsi_irq_chip = {
-	.name			= "pMSI",
-	.irq_set_type		= irq_chip_set_type_parent,
-	.irq_compose_msi_msg	= mbi_compose_mbi_msg,
-	.flags			= IRQCHIP_SUPPORTS_LEVEL_MSI,
-};
-
-static struct msi_domain_ops mbi_pmsi_ops = {
-};
+static bool mbi_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
+				  struct irq_domain *real_parent, struct msi_domain_info *info)
+{
+	if (!msi_lib_init_dev_msi_info(dev, domain, real_parent, info))
+		return false;
+
+	switch (info->bus_token) {
+	case DOMAIN_BUS_PCI_DEVICE_MSI:
+	case DOMAIN_BUS_PCI_DEVICE_MSIX:
+		info->chip->irq_compose_msi_msg = mbi_compose_msi_msg;
+		return true;
+
+	case DOMAIN_BUS_DEVICE_MSI:
+		info->chip->irq_compose_msi_msg = mbi_compose_mbi_msg;
+		info->chip->irq_set_type = irq_chip_set_type_parent;
+		info->chip->flags |= IRQCHIP_SUPPORTS_LEVEL_MSI;
+		info->flags |= MSI_FLAG_LEVEL_CAPABLE;
+		return true;
+
+	default:
+		WARN_ON_ONCE(1);
+		return false;
+	}
+}
 
-static struct msi_domain_info mbi_pmsi_domain_info = {
-	.flags	= (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
-		   MSI_FLAG_LEVEL_CAPABLE),
-	.ops	= &mbi_pmsi_ops,
-	.chip	= &mbi_pmsi_irq_chip,
+#define MBI_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
+				 MSI_FLAG_USE_DEF_CHIP_OPS |	\
+				 MSI_FLAG_PCI_MSI_MASK_PARENT)
+
+#define MBI_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK |	\
+				 MSI_FLAG_PCI_MSIX      |	\
+				 MSI_FLAG_MULTI_PCI_MSI)
+
+static const struct msi_parent_ops gic_v3_mbi_msi_parent_ops = {
+	.supported_flags	= MBI_MSI_FLAGS_SUPPORTED,
+	.required_flags		= MBI_MSI_FLAGS_REQUIRED,
+	.bus_select_token	= DOMAIN_BUS_NEXUS,
+	.bus_select_mask	= MATCH_PCI_MSI | MATCH_PLATFORM_MSI,
+	.prefix			= "MBI-",
+	.init_dev_msi_info	= mbi_init_dev_msi_info,
 };
 
-static int mbi_allocate_domains(struct irq_domain *parent)
+static int mbi_allocate_domain(struct irq_domain *parent)
 {
-	struct irq_domain *nexus_domain, *pci_domain, *plat_domain;
-	int err;
+	struct irq_domain *nexus_domain;
 
 	nexus_domain = irq_domain_create_hierarchy(parent, 0, 0, parent->fwnode,
 						   &mbi_domain_ops, NULL);
@@ -239,22 +217,8 @@ static int mbi_allocate_domains(struct irq_domain *parent)
 		return -ENOMEM;
 
 	irq_domain_update_bus_token(nexus_domain, DOMAIN_BUS_NEXUS);
-
-	err = mbi_allocate_pci_domain(nexus_domain, &pci_domain);
-
-	plat_domain = platform_msi_create_irq_domain(parent->fwnode,
-						     &mbi_pmsi_domain_info,
-						     nexus_domain);
-
-	if (err || !plat_domain) {
-		if (plat_domain)
-			irq_domain_remove(plat_domain);
-		if (pci_domain)
-			irq_domain_remove(pci_domain);
-		irq_domain_remove(nexus_domain);
-		return -ENOMEM;
-	}
-
+	nexus_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+	nexus_domain->msi_parent_ops = &gic_v3_mbi_msi_parent_ops;
 	return 0;
 }
 
@@ -317,7 +281,7 @@ int __init mbi_init(struct fwnode_handle *fwnode, struct irq_domain *parent)
 
 	pr_info("Using MBI frame %pa\n", &mbi_phys_base);
 
-	ret = mbi_allocate_domains(parent);
+	ret = mbi_allocate_domain(parent);
 	if (ret)
 		goto err_free_mbi;
 
#51 lore raw

PATCH v4 13/21 irqchip/gic-v2m: Switch to device MSI

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" PCI/MSI and platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
v3: enabled MSI_FLAG_PCI_MSI_MASK_PARENT in msi_parent_ops::supported_flags

---
 drivers/irqchip/Kconfig       |  1 +
 drivers/irqchip/irq-gic-v2m.c | 80 +++++++++++------------------------
 2 files changed, 25 insertions(+), 56 deletions(-)
drivers/irqchip/Kconfig +1 -0
index b51863fa9b38..2104b8727b1a 100644
@@ -26,6 +26,7 @@ config ARM_GIC_V2M
 	bool
 	depends on PCI
 	select ARM_GIC
+	select IRQ_MSI_LIB
 	select PCI_MSI
 
 config GIC_NON_BANKED
drivers/irqchip/irq-gic-v2m.c +24 -56

124 lines not on this page · raw message

Signature · 1 line
2.34.1
#52 lore raw

[tip: irq/core] irqchip/gic-v2m: Switch to device MSI

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     7b89a879cce4f81fb08e7aeb7b2639fe755803bb
Gitweb:        https://git.kernel.org/tip/7b89a879cce4f81fb08e7aeb7b2639fe755803bb
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:53 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:24 +02:00

irqchip/gic-v2m: Switch to device MSI

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" PCI/MSI and platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
 drivers/irqchip/Kconfig       |  1 +-
 drivers/irqchip/irq-gic-v2m.c | 81 ++++++++++------------------------
 2 files changed, 26 insertions(+), 56 deletions(-)
drivers/irqchip/Kconfig +1 -0
index ee78716..d3fbe56 100644
@@ -26,6 +26,7 @@ config ARM_GIC_V2M
 	bool
 	depends on PCI
 	select ARM_GIC
+	select IRQ_MSI_LIB
 	select PCI_MSI
 
 config GIC_NON_BANKED
drivers/irqchip/irq-gic-v2m.c +25 -56

125 lines not on this page · raw message

#53 lore raw

[tip: irq/msi] irqchip/gic-v2m: Switch to device MSI

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     74e44454aafefd682706248eb3846e25a1a05c6d
Gitweb:        https://git.kernel.org/tip/74e44454aafefd682706248eb3846e25a1a05c6d
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:53 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

irqchip/gic-v2m: Switch to device MSI

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" PCI/MSI and platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
 drivers/irqchip/Kconfig       |  1 +-
 drivers/irqchip/irq-gic-v2m.c | 81 ++++++++++------------------------
 2 files changed, 26 insertions(+), 56 deletions(-)
drivers/irqchip/Kconfig +1 -0
index b51863f..2104b87 100644
@@ -26,6 +26,7 @@ config ARM_GIC_V2M
 	bool
 	depends on PCI
 	select ARM_GIC
+	select IRQ_MSI_LIB
 	select PCI_MSI
 
 config GIC_NON_BANKED
drivers/irqchip/irq-gic-v2m.c +25 -56

125 lines not on this page · raw message

#54 lore raw

PATCH v4 14/21 irqchip/imx-mu-msi: Switch to MSI parent

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---

---
 drivers/irqchip/Kconfig          |    1 
 drivers/irqchip/irq-imx-mu-msi.c |   48 +++++++++++++++++----------------------
 drivers/irqchip/irq-msi-lib.c    |    2 +
 3 files changed, 24 insertions(+), 27 deletions(-)
drivers/irqchip/Kconfig +1 -0
@@ -492,6 +492,7 @@ config IMX_MU_MSI
 	select IRQ_DOMAIN
 	select IRQ_DOMAIN_HIERARCHY
 	select GENERIC_MSI_IRQ
+	select IRQ_MSI_LIB
 	help
 	  Provide a driver for the i.MX Messaging Unit block used as a
 	  CPU-to-CPU MSI controller. This requires a specially crafted DT
drivers/irqchip/irq-imx-mu-msi.c +21 -27

78 lines not on this page · raw message

drivers/irqchip/irq-msi-lib.c +2 -0
@@ -89,6 +89,8 @@ bool msi_lib_init_dev_msi_info(struct de
 	/* Chip updates for all child bus types */
 	if (!info->chip->irq_eoi)
 		info->chip->irq_eoi	= irq_chip_eoi_parent;
+	if (!info->chip->irq_ack)
+		info->chip->irq_ack	= irq_chip_ack_parent;
 
 	/*
 	 * The device MSI domain can never have a set affinity callback. It
#55 lore raw

[tip: irq/core] irqchip/imx-mu-msi: Switch to MSI parent

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     0ce1d34b8040577c1ddd23eea63977c7be832419
Gitweb:        https://git.kernel.org/tip/0ce1d34b8040577c1ddd23eea63977c7be832419
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:55 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:24 +02:00

irqchip/imx-mu-msi: Switch to MSI parent

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
 drivers/irqchip/Kconfig          |  1 +-
 drivers/irqchip/irq-imx-mu-msi.c | 48 +++++++++++++------------------
 drivers/irqchip/irq-msi-lib.c    |  2 +-
 3 files changed, 24 insertions(+), 27 deletions(-)
drivers/irqchip/Kconfig +1 -0
index d3fbe56..5b93a56 100644
@@ -513,6 +513,7 @@ config IMX_MU_MSI
 	select IRQ_DOMAIN
 	select IRQ_DOMAIN_HIERARCHY
 	select GENERIC_MSI_IRQ
+	select IRQ_MSI_LIB
 	help
 	  Provide a driver for the i.MX Messaging Unit block used as a
 	  CPU-to-CPU MSI controller. This requires a specially crafted DT
drivers/irqchip/irq-imx-mu-msi.c +21 -27

78 lines not on this page · raw message

drivers/irqchip/irq-msi-lib.c +2 -0
index d20a0e6..b5b9000 100644
@@ -94,6 +94,8 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 	/* Chip updates for all child bus types */
 	if (!info->chip->irq_eoi)
 		info->chip->irq_eoi	= irq_chip_eoi_parent;
+	if (!info->chip->irq_ack)
+		info->chip->irq_ack	= irq_chip_ack_parent;
 
 	/*
 	 * The device MSI domain can never have a set affinity callback. It
#56 lore raw

[tip: irq/msi] irqchip/imx-mu-msi: Switch to MSI parent

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     7b2f8aa005bc141ac9144f2fcf2cfb63abd7a0b1
Gitweb:        https://git.kernel.org/tip/7b2f8aa005bc141ac9144f2fcf2cfb63abd7a0b1
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:55 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

irqchip/imx-mu-msi: Switch to MSI parent

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
 drivers/irqchip/Kconfig          |  1 +-
 drivers/irqchip/irq-imx-mu-msi.c | 48 +++++++++++++------------------
 drivers/irqchip/irq-msi-lib.c    |  2 +-
 3 files changed, 24 insertions(+), 27 deletions(-)
drivers/irqchip/Kconfig +1 -0
index 2104b87..e7a57b3 100644
@@ -492,6 +492,7 @@ config IMX_MU_MSI
 	select IRQ_DOMAIN
 	select IRQ_DOMAIN_HIERARCHY
 	select GENERIC_MSI_IRQ
+	select IRQ_MSI_LIB
 	help
 	  Provide a driver for the i.MX Messaging Unit block used as a
 	  CPU-to-CPU MSI controller. This requires a specially crafted DT
drivers/irqchip/irq-imx-mu-msi.c +21 -27

78 lines not on this page · raw message

drivers/irqchip/irq-msi-lib.c +2 -0
index d20a0e6..b5b9000 100644
@@ -94,6 +94,8 @@ bool msi_lib_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
 	/* Chip updates for all child bus types */
 	if (!info->chip->irq_eoi)
 		info->chip->irq_eoi	= irq_chip_eoi_parent;
+	if (!info->chip->irq_ack)
+		info->chip->irq_ack	= irq_chip_ack_parent;
 
 	/*
 	 * The device MSI domain can never have a set affinity callback. It
#57 lore raw

PATCH v4 15/21 irqchip/irq-mvebu-icu: Prepare for real per device MSI

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

The core infrastructure has everything in place to switch ICU to per device MSI domains and avoid the convoluted construct of the existing platform-MSI layering violation.

The new infrastructure provides a wired interrupt specific interface in the MSI core which converts the 'hardware interrupt number + trigger type' allocation which is required for wired interrupts in the regular irqdomain code to a normal MSI allocation.

The hardware interrupt number and the trigger type are stored in the MSI descriptor device cookie by the core code so the ICU specific code can retrieve them.

The new per device domain is only instantiated when the irqdomain which is associated to the ICU device provides MSI parent functionality. Up to that point it invokes the existing code. Once the parent is converted the code for the current platform-MSI mechanism is removed.

The new domain shares the interrupt chip callbacks and the translation function. The only new functionality aside of filling out the msi_domain_templates is a domain specific set_desc() callback, which will go away once all platform-MSI code has been converted.

---
 drivers/irqchip/irq-mvebu-icu.c | 181 ++++++++++++++++++++++++++++++--
 1 file changed, 170 insertions(+), 11 deletions(-)
drivers/irqchip/irq-mvebu-icu.c +170 -11
index 3c77acc7ec6a..2a210cd9482e 100644
@@ -20,6 +20,8 @@
 #include <linux/of_platform.h>
 #include <linux/platform_device.h>
 
+#include "irq-msi-lib.h"
+
 #include <dt-bindings/interrupt-controller/mvebu-icu.h>
 
 /* ICU registers */
@@ -60,14 +62,52 @@ struct mvebu_icu_msi_data {
 	const struct mvebu_icu_subset_data *subset_data;
 };
 
-struct mvebu_icu_irq_data {
-	struct mvebu_icu *icu;
-	unsigned int icu_group;
-	unsigned int type;
-};
-
 static DEFINE_STATIC_KEY_FALSE(legacy_bindings);
 
+static int mvebu_icu_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
+			       unsigned long *hwirq, unsigned int *type)
+{
+	unsigned int param_count = static_branch_unlikely(&legacy_bindings) ? 3 : 2;
+	struct mvebu_icu_msi_data *msi_data = d->host_data;
+	struct mvebu_icu *icu = msi_data->icu;
+
+	/* Check the count of the parameters in dt */
+	if (WARN_ON(fwspec->param_count != param_count)) {
+		dev_err(icu->dev, "wrong ICU parameter count %d\n",
+			fwspec->param_count);
+		return -EINVAL;
+	}
+
+	if (static_branch_unlikely(&legacy_bindings)) {
+		*hwirq = fwspec->param[1];
+		*type = fwspec->param[2] & IRQ_TYPE_SENSE_MASK;
+		if (fwspec->param[0] != ICU_GRP_NSR) {
+			dev_err(icu->dev, "wrong ICU group type %x\n",
+				fwspec->param[0]);
+			return -EINVAL;
+		}
+	} else {
+		*hwirq = fwspec->param[0];
+		*type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
+
+		/*
+		 * The ICU receives level interrupts. While the NSR are also
+		 * level interrupts, SEI are edge interrupts. Force the type
+		 * here in this case. Please note that this makes the interrupt
+		 * handling unreliable.
+		 */
+		if (msi_data->subset_data->icu_group == ICU_GRP_SEI)
+			*type = IRQ_TYPE_EDGE_RISING;
+	}
+
+	if (*hwirq >= ICU_MAX_IRQS) {
+		dev_err(icu->dev, "invalid interrupt number %ld\n", *hwirq);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static void mvebu_icu_init(struct mvebu_icu *icu,
 			   struct mvebu_icu_msi_data *msi_data,
 			   struct msi_msg *msg)
@@ -89,6 +129,14 @@ static void mvebu_icu_init(struct mvebu_icu *icu,
 	writel_relaxed(msg[1].address_lo, icu->base + subset->offset_clr_al);
 }
 
+/* Start of area to be removed once all parent chips provide MSI parent */
+
+struct mvebu_icu_irq_data {
+	struct mvebu_icu *icu;
+	unsigned int icu_group;
+	unsigned int type;
+};
+
 static void mvebu_icu_write_msg(struct msi_desc *desc, struct msi_msg *msg)
 {
 	struct irq_data *d = irq_get_irq_data(desc->irq);
@@ -269,6 +317,109 @@ static const struct irq_domain_ops mvebu_icu_domain_ops = {
 	.free      = mvebu_icu_irq_domain_free,
 };
 
+/* End of removal area */
+
+static int mvebu_icu_msi_init(struct irq_domain *domain, struct msi_domain_info *info,
+			      unsigned int virq, irq_hw_number_t hwirq, msi_alloc_info_t *arg)
+{
+	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, info->chip, info->chip_data);
+	return irq_set_irqchip_state(virq, IRQCHIP_STATE_PENDING, false);
+}
+
+static void mvebu_icu_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
+{
+	arg->desc = desc;
+	arg->hwirq = (u32)desc->data.icookie.value;
+}
+
+static void mvebu_icu_write_msi_msg(struct irq_data *d, struct msi_msg *msg)
+{
+	struct mvebu_icu_msi_data *msi_data = d->chip_data;
+	unsigned int icu_group = msi_data->subset_data->icu_group;
+	struct msi_desc *desc = irq_data_get_msi_desc(d);
+	struct mvebu_icu *icu = msi_data->icu;
+	unsigned int type;
+	u32 icu_int;
+
+	if (msg->address_lo || msg->address_hi) {
+		/* One off initialization per domain */
+		mvebu_icu_init(icu, msi_data, msg);
+		/* Configure the ICU with irq number & type */
+		icu_int = msg->data | ICU_INT_ENABLE;
+		type = (unsigned int)(desc->data.icookie.value >> 32);
+		if (type & IRQ_TYPE_EDGE_RISING)
+			icu_int |= ICU_IS_EDGE;
+		icu_int |= icu_group << ICU_GROUP_SHIFT;
+	} else {
+		/* De-configure the ICU */
+		icu_int = 0;
+	}
+
+	writel_relaxed(icu_int, icu->base + ICU_INT_CFG(d->hwirq));
+
+	/*
+	 * The SATA unit has 2 ports, and a dedicated ICU entry per
+	 * port. The ahci sata driver supports only one irq interrupt
+	 * per SATA unit. To solve this conflict, we configure the 2
+	 * SATA wired interrupts in the south bridge into 1 GIC
+	 * interrupt in the north bridge. Even if only a single port
+	 * is enabled, if sata node is enabled, both interrupts are
+	 * configured (regardless of which port is actually in use).
+	 */
+	if (d->hwirq == ICU_SATA0_ICU_ID || d->hwirq == ICU_SATA1_ICU_ID) {
+		writel_relaxed(icu_int, icu->base + ICU_INT_CFG(ICU_SATA0_ICU_ID));
+		writel_relaxed(icu_int, icu->base + ICU_INT_CFG(ICU_SATA1_ICU_ID));
+	}
+}
+
+static const struct msi_domain_template mvebu_icu_nsr_msi_template = {
+	.chip = {
+		.name			= "ICU-NSR",
+		.irq_mask		= irq_chip_mask_parent,
+		.irq_unmask		= irq_chip_unmask_parent,
+		.irq_eoi		= irq_chip_eoi_parent,
+		.irq_set_type		= irq_chip_set_type_parent,
+		.irq_write_msi_msg	= mvebu_icu_write_msi_msg,
+		.flags			= IRQCHIP_SUPPORTS_LEVEL_MSI,
+	},
+
+	.ops = {
+		.msi_translate		= mvebu_icu_translate,
+		.msi_init		= mvebu_icu_msi_init,
+		.set_desc		= mvebu_icu_set_desc,
+	},
+
+	.info = {
+		.bus_token		= DOMAIN_BUS_WIRED_TO_MSI,
+		.flags			= MSI_FLAG_LEVEL_CAPABLE |
+					  MSI_FLAG_USE_DEV_FWNODE,
+	},
+};
+
+static const struct msi_domain_template mvebu_icu_sei_msi_template = {
+	.chip = {
+		.name			= "ICU-SEI",
+		.irq_mask		= irq_chip_mask_parent,
+		.irq_unmask		= irq_chip_unmask_parent,
+		.irq_ack		= irq_chip_ack_parent,
+		.irq_set_type		= irq_chip_set_type_parent,
+		.irq_write_msi_msg	= mvebu_icu_write_msi_msg,
+		.flags			= IRQCHIP_SUPPORTS_LEVEL_MSI,
+	},
+
+	.ops = {
+		.msi_translate		= mvebu_icu_translate,
+		.msi_init		= mvebu_icu_msi_init,
+		.set_desc		= mvebu_icu_set_desc,
+	},
+
+	.info = {
+		.bus_token		= DOMAIN_BUS_WIRED_TO_MSI,
+		.flags			= MSI_FLAG_LEVEL_CAPABLE |
+					  MSI_FLAG_USE_DEV_FWNODE,
+	},
+};
+
 static const struct mvebu_icu_subset_data mvebu_icu_nsr_subset_data = {
 	.icu_group = ICU_GRP_NSR,
 	.offset_set_ah = ICU_SETSPI_NSR_AH,
@@ -298,7 +449,6 @@ static const struct of_device_id mvebu_icu_subset_of_match[] = {
 static int mvebu_icu_subset_probe(struct platform_device *pdev)
 {
 	struct mvebu_icu_msi_data *msi_data;
-	struct device_node *msi_parent_dn;
 	struct device *dev = &pdev->dev;
 	struct irq_domain *irq_domain;
 
@@ -314,15 +464,24 @@ static int mvebu_icu_subset_probe(struct platform_device *pdev)
 		msi_data->subset_data = of_device_get_match_data(dev);
 	}
 
-	dev->msi.domain = of_msi_get_domain(dev, dev->of_node,
-					    DOMAIN_BUS_PLATFORM_MSI);
+	dev->msi.domain = of_msi_get_domain(dev, dev->of_node, DOMAIN_BUS_PLATFORM_MSI);
 	if (!dev->msi.domain)
 		return -EPROBE_DEFER;
 
-	msi_parent_dn = irq_domain_get_of_node(dev->msi.domain);
-	if (!msi_parent_dn)
+	if (!irq_domain_get_of_node(dev->msi.domain))
 		return -ENODEV;
 
+	if (irq_domain_is_msi_parent(dev->msi.domain)) {
+		bool sei = msi_data->subset_data->icu_group == ICU_GRP_SEI;
+		const struct msi_domain_template *tmpl;
+
+		tmpl = sei ? &mvebu_icu_sei_msi_template : &mvebu_icu_nsr_msi_template;
+
+		if (!msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN, tmpl,
+						  ICU_MAX_IRQS, NULL, msi_data))
+			return -ENOMEM;
+	}
+
 	irq_domain = platform_msi_create_device_tree_domain(dev, ICU_MAX_IRQS,
 							    mvebu_icu_write_msg,
 							    &mvebu_icu_domain_ops,
Signature · 1 line
2.34.1
#58 lore raw

[tip: irq/core] irqchip/irq-mvebu-icu: Prepare for real per device MSI

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     315145d9ede827295d5a6911b36dbf3af7a25dff
Gitweb:        https://git.kernel.org/tip/315145d9ede827295d5a6911b36dbf3af7a25dff
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:56 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:24 +02:00

irqchip/irq-mvebu-icu: Prepare for real per device MSI

The core infrastructure has everything in place to switch ICU to per device MSI domains and avoid the convoluted construct of the existing platform-MSI layering violation.

The new infrastructure provides a wired interrupt specific interface in the MSI core which converts the 'hardware interrupt number + trigger type' allocation which is required for wired interrupts in the regular irqdomain code to a normal MSI allocation.

The hardware interrupt number and the trigger type are stored in the MSI descriptor device cookie by the core code so the ICU specific code can retrieve them.

The new per device domain is only instantiated when the irqdomain which is associated to the ICU device provides MSI parent functionality. Up to that point it invokes the existing code. Once the parent is converted the code for the current platform-MSI mechanism is removed.

The new domain shares the interrupt chip callbacks and the translation function. The only new functionality aside of filling out the msi_domain_templates is a domain specific set_desc() callback, which will go away once all platform-MSI code has been converted.

---
 drivers/irqchip/irq-mvebu-icu.c | 181 +++++++++++++++++++++++++++++--
 1 file changed, 170 insertions(+), 11 deletions(-)
drivers/irqchip/irq-mvebu-icu.c +170 -11
index 3c77acc..2a210cd 100644
@@ -20,6 +20,8 @@
 #include <linux/of_platform.h>
 #include <linux/platform_device.h>
 
+#include "irq-msi-lib.h"
+
 #include <dt-bindings/interrupt-controller/mvebu-icu.h>
 
 /* ICU registers */
@@ -60,14 +62,52 @@ struct mvebu_icu_msi_data {
 	const struct mvebu_icu_subset_data *subset_data;
 };
 
-struct mvebu_icu_irq_data {
-	struct mvebu_icu *icu;
-	unsigned int icu_group;
-	unsigned int type;
-};
-
 static DEFINE_STATIC_KEY_FALSE(legacy_bindings);
 
+static int mvebu_icu_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
+			       unsigned long *hwirq, unsigned int *type)
+{
+	unsigned int param_count = static_branch_unlikely(&legacy_bindings) ? 3 : 2;
+	struct mvebu_icu_msi_data *msi_data = d->host_data;
+	struct mvebu_icu *icu = msi_data->icu;
+
+	/* Check the count of the parameters in dt */
+	if (WARN_ON(fwspec->param_count != param_count)) {
+		dev_err(icu->dev, "wrong ICU parameter count %d\n",
+			fwspec->param_count);
+		return -EINVAL;
+	}
+
+	if (static_branch_unlikely(&legacy_bindings)) {
+		*hwirq = fwspec->param[1];
+		*type = fwspec->param[2] & IRQ_TYPE_SENSE_MASK;
+		if (fwspec->param[0] != ICU_GRP_NSR) {
+			dev_err(icu->dev, "wrong ICU group type %x\n",
+				fwspec->param[0]);
+			return -EINVAL;
+		}
+	} else {
+		*hwirq = fwspec->param[0];
+		*type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
+
+		/*
+		 * The ICU receives level interrupts. While the NSR are also
+		 * level interrupts, SEI are edge interrupts. Force the type
+		 * here in this case. Please note that this makes the interrupt
+		 * handling unreliable.
+		 */
+		if (msi_data->subset_data->icu_group == ICU_GRP_SEI)
+			*type = IRQ_TYPE_EDGE_RISING;
+	}
+
+	if (*hwirq >= ICU_MAX_IRQS) {
+		dev_err(icu->dev, "invalid interrupt number %ld\n", *hwirq);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static void mvebu_icu_init(struct mvebu_icu *icu,
 			   struct mvebu_icu_msi_data *msi_data,
 			   struct msi_msg *msg)
@@ -89,6 +129,14 @@ static void mvebu_icu_init(struct mvebu_icu *icu,
 	writel_relaxed(msg[1].address_lo, icu->base + subset->offset_clr_al);
 }
 
+/* Start of area to be removed once all parent chips provide MSI parent */
+
+struct mvebu_icu_irq_data {
+	struct mvebu_icu *icu;
+	unsigned int icu_group;
+	unsigned int type;
+};
+
 static void mvebu_icu_write_msg(struct msi_desc *desc, struct msi_msg *msg)
 {
 	struct irq_data *d = irq_get_irq_data(desc->irq);
@@ -269,6 +317,109 @@ static const struct irq_domain_ops mvebu_icu_domain_ops = {
 	.free      = mvebu_icu_irq_domain_free,
 };
 
+/* End of removal area */
+
+static int mvebu_icu_msi_init(struct irq_domain *domain, struct msi_domain_info *info,
+			      unsigned int virq, irq_hw_number_t hwirq, msi_alloc_info_t *arg)
+{
+	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, info->chip, info->chip_data);
+	return irq_set_irqchip_state(virq, IRQCHIP_STATE_PENDING, false);
+}
+
+static void mvebu_icu_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
+{
+	arg->desc = desc;
+	arg->hwirq = (u32)desc->data.icookie.value;
+}
+
+static void mvebu_icu_write_msi_msg(struct irq_data *d, struct msi_msg *msg)
+{
+	struct mvebu_icu_msi_data *msi_data = d->chip_data;
+	unsigned int icu_group = msi_data->subset_data->icu_group;
+	struct msi_desc *desc = irq_data_get_msi_desc(d);
+	struct mvebu_icu *icu = msi_data->icu;
+	unsigned int type;
+	u32 icu_int;
+
+	if (msg->address_lo || msg->address_hi) {
+		/* One off initialization per domain */
+		mvebu_icu_init(icu, msi_data, msg);
+		/* Configure the ICU with irq number & type */
+		icu_int = msg->data | ICU_INT_ENABLE;
+		type = (unsigned int)(desc->data.icookie.value >> 32);
+		if (type & IRQ_TYPE_EDGE_RISING)
+			icu_int |= ICU_IS_EDGE;
+		icu_int |= icu_group << ICU_GROUP_SHIFT;
+	} else {
+		/* De-configure the ICU */
+		icu_int = 0;
+	}
+
+	writel_relaxed(icu_int, icu->base + ICU_INT_CFG(d->hwirq));
+
+	/*
+	 * The SATA unit has 2 ports, and a dedicated ICU entry per
+	 * port. The ahci sata driver supports only one irq interrupt
+	 * per SATA unit. To solve this conflict, we configure the 2
+	 * SATA wired interrupts in the south bridge into 1 GIC
+	 * interrupt in the north bridge. Even if only a single port
+	 * is enabled, if sata node is enabled, both interrupts are
+	 * configured (regardless of which port is actually in use).
+	 */
+	if (d->hwirq == ICU_SATA0_ICU_ID || d->hwirq == ICU_SATA1_ICU_ID) {
+		writel_relaxed(icu_int, icu->base + ICU_INT_CFG(ICU_SATA0_ICU_ID));
+		writel_relaxed(icu_int, icu->base + ICU_INT_CFG(ICU_SATA1_ICU_ID));
+	}
+}
+
+static const struct msi_domain_template mvebu_icu_nsr_msi_template = {
+	.chip = {
+		.name			= "ICU-NSR",
+		.irq_mask		= irq_chip_mask_parent,
+		.irq_unmask		= irq_chip_unmask_parent,
+		.irq_eoi		= irq_chip_eoi_parent,
+		.irq_set_type		= irq_chip_set_type_parent,
+		.irq_write_msi_msg	= mvebu_icu_write_msi_msg,
+		.flags			= IRQCHIP_SUPPORTS_LEVEL_MSI,
+	},
+
+	.ops = {
+		.msi_translate		= mvebu_icu_translate,
+		.msi_init		= mvebu_icu_msi_init,
+		.set_desc		= mvebu_icu_set_desc,
+	},
+
+	.info = {
+		.bus_token		= DOMAIN_BUS_WIRED_TO_MSI,
+		.flags			= MSI_FLAG_LEVEL_CAPABLE |
+					  MSI_FLAG_USE_DEV_FWNODE,
+	},
+};
+
+static const struct msi_domain_template mvebu_icu_sei_msi_template = {
+	.chip = {
+		.name			= "ICU-SEI",
+		.irq_mask		= irq_chip_mask_parent,
+		.irq_unmask		= irq_chip_unmask_parent,
+		.irq_ack		= irq_chip_ack_parent,
+		.irq_set_type		= irq_chip_set_type_parent,
+		.irq_write_msi_msg	= mvebu_icu_write_msi_msg,
+		.flags			= IRQCHIP_SUPPORTS_LEVEL_MSI,
+	},
+
+	.ops = {
+		.msi_translate		= mvebu_icu_translate,
+		.msi_init		= mvebu_icu_msi_init,
+		.set_desc		= mvebu_icu_set_desc,
+	},
+
+	.info = {
+		.bus_token		= DOMAIN_BUS_WIRED_TO_MSI,
+		.flags			= MSI_FLAG_LEVEL_CAPABLE |
+					  MSI_FLAG_USE_DEV_FWNODE,
+	},
+};
+
 static const struct mvebu_icu_subset_data mvebu_icu_nsr_subset_data = {
 	.icu_group = ICU_GRP_NSR,
 	.offset_set_ah = ICU_SETSPI_NSR_AH,
@@ -298,7 +449,6 @@ static const struct of_device_id mvebu_icu_subset_of_match[] = {
 static int mvebu_icu_subset_probe(struct platform_device *pdev)
 {
 	struct mvebu_icu_msi_data *msi_data;
-	struct device_node *msi_parent_dn;
 	struct device *dev = &pdev->dev;
 	struct irq_domain *irq_domain;
 
@@ -314,15 +464,24 @@ static int mvebu_icu_subset_probe(struct platform_device *pdev)
 		msi_data->subset_data = of_device_get_match_data(dev);
 	}
 
-	dev->msi.domain = of_msi_get_domain(dev, dev->of_node,
-					    DOMAIN_BUS_PLATFORM_MSI);
+	dev->msi.domain = of_msi_get_domain(dev, dev->of_node, DOMAIN_BUS_PLATFORM_MSI);
 	if (!dev->msi.domain)
 		return -EPROBE_DEFER;
 
-	msi_parent_dn = irq_domain_get_of_node(dev->msi.domain);
-	if (!msi_parent_dn)
+	if (!irq_domain_get_of_node(dev->msi.domain))
 		return -ENODEV;
 
+	if (irq_domain_is_msi_parent(dev->msi.domain)) {
+		bool sei = msi_data->subset_data->icu_group == ICU_GRP_SEI;
+		const struct msi_domain_template *tmpl;
+
+		tmpl = sei ? &mvebu_icu_sei_msi_template : &mvebu_icu_nsr_msi_template;
+
+		if (!msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN, tmpl,
+						  ICU_MAX_IRQS, NULL, msi_data))
+			return -ENOMEM;
+	}
+
 	irq_domain = platform_msi_create_device_tree_domain(dev, ICU_MAX_IRQS,
 							    mvebu_icu_write_msg,
 							    &mvebu_icu_domain_ops,
#59 lore raw

[tip: irq/msi] irqchip/irq-mvebu-icu: Prepare for real per device MSI

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     d929e4db22b61555a6b091450c26d1f7281ca576
Gitweb:        https://git.kernel.org/tip/d929e4db22b61555a6b091450c26d1f7281ca576
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:56 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

irqchip/irq-mvebu-icu: Prepare for real per device MSI

The core infrastructure has everything in place to switch ICU to per device MSI domains and avoid the convoluted construct of the existing platform-MSI layering violation.

The new infrastructure provides a wired interrupt specific interface in the MSI core which converts the 'hardware interrupt number + trigger type' allocation which is required for wired interrupts in the regular irqdomain code to a normal MSI allocation.

The hardware interrupt number and the trigger type are stored in the MSI descriptor device cookie by the core code so the ICU specific code can retrieve them.

The new per device domain is only instantiated when the irqdomain which is associated to the ICU device provides MSI parent functionality. Up to that point it invokes the existing code. Once the parent is converted the code for the current platform-MSI mechanism is removed.

The new domain shares the interrupt chip callbacks and the translation function. The only new functionality aside of filling out the msi_domain_templates is a domain specific set_desc() callback, which will go away once all platform-MSI code has been converted.

---
 drivers/irqchip/irq-mvebu-icu.c | 181 +++++++++++++++++++++++++++++--
 1 file changed, 170 insertions(+), 11 deletions(-)
drivers/irqchip/irq-mvebu-icu.c +170 -11
index 3c77acc..2a210cd 100644
@@ -20,6 +20,8 @@
 #include <linux/of_platform.h>
 #include <linux/platform_device.h>
 
+#include "irq-msi-lib.h"
+
 #include <dt-bindings/interrupt-controller/mvebu-icu.h>
 
 /* ICU registers */
@@ -60,14 +62,52 @@ struct mvebu_icu_msi_data {
 	const struct mvebu_icu_subset_data *subset_data;
 };
 
-struct mvebu_icu_irq_data {
-	struct mvebu_icu *icu;
-	unsigned int icu_group;
-	unsigned int type;
-};
-
 static DEFINE_STATIC_KEY_FALSE(legacy_bindings);
 
+static int mvebu_icu_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
+			       unsigned long *hwirq, unsigned int *type)
+{
+	unsigned int param_count = static_branch_unlikely(&legacy_bindings) ? 3 : 2;
+	struct mvebu_icu_msi_data *msi_data = d->host_data;
+	struct mvebu_icu *icu = msi_data->icu;
+
+	/* Check the count of the parameters in dt */
+	if (WARN_ON(fwspec->param_count != param_count)) {
+		dev_err(icu->dev, "wrong ICU parameter count %d\n",
+			fwspec->param_count);
+		return -EINVAL;
+	}
+
+	if (static_branch_unlikely(&legacy_bindings)) {
+		*hwirq = fwspec->param[1];
+		*type = fwspec->param[2] & IRQ_TYPE_SENSE_MASK;
+		if (fwspec->param[0] != ICU_GRP_NSR) {
+			dev_err(icu->dev, "wrong ICU group type %x\n",
+				fwspec->param[0]);
+			return -EINVAL;
+		}
+	} else {
+		*hwirq = fwspec->param[0];
+		*type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
+
+		/*
+		 * The ICU receives level interrupts. While the NSR are also
+		 * level interrupts, SEI are edge interrupts. Force the type
+		 * here in this case. Please note that this makes the interrupt
+		 * handling unreliable.
+		 */
+		if (msi_data->subset_data->icu_group == ICU_GRP_SEI)
+			*type = IRQ_TYPE_EDGE_RISING;
+	}
+
+	if (*hwirq >= ICU_MAX_IRQS) {
+		dev_err(icu->dev, "invalid interrupt number %ld\n", *hwirq);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static void mvebu_icu_init(struct mvebu_icu *icu,
 			   struct mvebu_icu_msi_data *msi_data,
 			   struct msi_msg *msg)
@@ -89,6 +129,14 @@ static void mvebu_icu_init(struct mvebu_icu *icu,
 	writel_relaxed(msg[1].address_lo, icu->base + subset->offset_clr_al);
 }
 
+/* Start of area to be removed once all parent chips provide MSI parent */
+
+struct mvebu_icu_irq_data {
+	struct mvebu_icu *icu;
+	unsigned int icu_group;
+	unsigned int type;
+};
+
 static void mvebu_icu_write_msg(struct msi_desc *desc, struct msi_msg *msg)
 {
 	struct irq_data *d = irq_get_irq_data(desc->irq);
@@ -269,6 +317,109 @@ static const struct irq_domain_ops mvebu_icu_domain_ops = {
 	.free      = mvebu_icu_irq_domain_free,
 };
 
+/* End of removal area */
+
+static int mvebu_icu_msi_init(struct irq_domain *domain, struct msi_domain_info *info,
+			      unsigned int virq, irq_hw_number_t hwirq, msi_alloc_info_t *arg)
+{
+	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, info->chip, info->chip_data);
+	return irq_set_irqchip_state(virq, IRQCHIP_STATE_PENDING, false);
+}
+
+static void mvebu_icu_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
+{
+	arg->desc = desc;
+	arg->hwirq = (u32)desc->data.icookie.value;
+}
+
+static void mvebu_icu_write_msi_msg(struct irq_data *d, struct msi_msg *msg)
+{
+	struct mvebu_icu_msi_data *msi_data = d->chip_data;
+	unsigned int icu_group = msi_data->subset_data->icu_group;
+	struct msi_desc *desc = irq_data_get_msi_desc(d);
+	struct mvebu_icu *icu = msi_data->icu;
+	unsigned int type;
+	u32 icu_int;
+
+	if (msg->address_lo || msg->address_hi) {
+		/* One off initialization per domain */
+		mvebu_icu_init(icu, msi_data, msg);
+		/* Configure the ICU with irq number & type */
+		icu_int = msg->data | ICU_INT_ENABLE;
+		type = (unsigned int)(desc->data.icookie.value >> 32);
+		if (type & IRQ_TYPE_EDGE_RISING)
+			icu_int |= ICU_IS_EDGE;
+		icu_int |= icu_group << ICU_GROUP_SHIFT;
+	} else {
+		/* De-configure the ICU */
+		icu_int = 0;
+	}
+
+	writel_relaxed(icu_int, icu->base + ICU_INT_CFG(d->hwirq));
+
+	/*
+	 * The SATA unit has 2 ports, and a dedicated ICU entry per
+	 * port. The ahci sata driver supports only one irq interrupt
+	 * per SATA unit. To solve this conflict, we configure the 2
+	 * SATA wired interrupts in the south bridge into 1 GIC
+	 * interrupt in the north bridge. Even if only a single port
+	 * is enabled, if sata node is enabled, both interrupts are
+	 * configured (regardless of which port is actually in use).
+	 */
+	if (d->hwirq == ICU_SATA0_ICU_ID || d->hwirq == ICU_SATA1_ICU_ID) {
+		writel_relaxed(icu_int, icu->base + ICU_INT_CFG(ICU_SATA0_ICU_ID));
+		writel_relaxed(icu_int, icu->base + ICU_INT_CFG(ICU_SATA1_ICU_ID));
+	}
+}
+
+static const struct msi_domain_template mvebu_icu_nsr_msi_template = {
+	.chip = {
+		.name			= "ICU-NSR",
+		.irq_mask		= irq_chip_mask_parent,
+		.irq_unmask		= irq_chip_unmask_parent,
+		.irq_eoi		= irq_chip_eoi_parent,
+		.irq_set_type		= irq_chip_set_type_parent,
+		.irq_write_msi_msg	= mvebu_icu_write_msi_msg,
+		.flags			= IRQCHIP_SUPPORTS_LEVEL_MSI,
+	},
+
+	.ops = {
+		.msi_translate		= mvebu_icu_translate,
+		.msi_init		= mvebu_icu_msi_init,
+		.set_desc		= mvebu_icu_set_desc,
+	},
+
+	.info = {
+		.bus_token		= DOMAIN_BUS_WIRED_TO_MSI,
+		.flags			= MSI_FLAG_LEVEL_CAPABLE |
+					  MSI_FLAG_USE_DEV_FWNODE,
+	},
+};
+
+static const struct msi_domain_template mvebu_icu_sei_msi_template = {
+	.chip = {
+		.name			= "ICU-SEI",
+		.irq_mask		= irq_chip_mask_parent,
+		.irq_unmask		= irq_chip_unmask_parent,
+		.irq_ack		= irq_chip_ack_parent,
+		.irq_set_type		= irq_chip_set_type_parent,
+		.irq_write_msi_msg	= mvebu_icu_write_msi_msg,
+		.flags			= IRQCHIP_SUPPORTS_LEVEL_MSI,
+	},
+
+	.ops = {
+		.msi_translate		= mvebu_icu_translate,
+		.msi_init		= mvebu_icu_msi_init,
+		.set_desc		= mvebu_icu_set_desc,
+	},
+
+	.info = {
+		.bus_token		= DOMAIN_BUS_WIRED_TO_MSI,
+		.flags			= MSI_FLAG_LEVEL_CAPABLE |
+					  MSI_FLAG_USE_DEV_FWNODE,
+	},
+};
+
 static const struct mvebu_icu_subset_data mvebu_icu_nsr_subset_data = {
 	.icu_group = ICU_GRP_NSR,
 	.offset_set_ah = ICU_SETSPI_NSR_AH,
@@ -298,7 +449,6 @@ static const struct of_device_id mvebu_icu_subset_of_match[] = {
 static int mvebu_icu_subset_probe(struct platform_device *pdev)
 {
 	struct mvebu_icu_msi_data *msi_data;
-	struct device_node *msi_parent_dn;
 	struct device *dev = &pdev->dev;
 	struct irq_domain *irq_domain;
 
@@ -314,15 +464,24 @@ static int mvebu_icu_subset_probe(struct platform_device *pdev)
 		msi_data->subset_data = of_device_get_match_data(dev);
 	}
 
-	dev->msi.domain = of_msi_get_domain(dev, dev->of_node,
-					    DOMAIN_BUS_PLATFORM_MSI);
+	dev->msi.domain = of_msi_get_domain(dev, dev->of_node, DOMAIN_BUS_PLATFORM_MSI);
 	if (!dev->msi.domain)
 		return -EPROBE_DEFER;
 
-	msi_parent_dn = irq_domain_get_of_node(dev->msi.domain);
-	if (!msi_parent_dn)
+	if (!irq_domain_get_of_node(dev->msi.domain))
 		return -ENODEV;
 
+	if (irq_domain_is_msi_parent(dev->msi.domain)) {
+		bool sei = msi_data->subset_data->icu_group == ICU_GRP_SEI;
+		const struct msi_domain_template *tmpl;
+
+		tmpl = sei ? &mvebu_icu_sei_msi_template : &mvebu_icu_nsr_msi_template;
+
+		if (!msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN, tmpl,
+						  ICU_MAX_IRQS, NULL, msi_data))
+			return -ENOMEM;
+	}
+
 	irq_domain = platform_msi_create_device_tree_domain(dev, ICU_MAX_IRQS,
 							    mvebu_icu_write_msg,
 							    &mvebu_icu_domain_ops,
#60 lore raw

PATCH v4 16/21 irqchip/mvebu-gicp: Switch to MSI parent

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
v3: enabled MSI_FLAG_PCI_MSI_MASK_PARENT in msi_parent_ops::supported_flags

---
 drivers/irqchip/Kconfig          |    1 
 drivers/irqchip/irq-mvebu-gicp.c |   44 ++++++++++++++++-----------------------
 2 files changed, 20 insertions(+), 25 deletions(-)
drivers/irqchip/Kconfig +1 -0
@@ -371,6 +371,7 @@ config MSCC_OCELOT_IRQ
 	select GENERIC_IRQ_CHIP
 
 config MVEBU_GICP
+	select IRQ_MSI_LIB
 	bool
 
 config MVEBU_ICU
drivers/irqchip/irq-mvebu-gicp.c +19 -25

74 lines not on this page · raw message

#61 lore raw

[tip: irq/core] irqchip/mvebu-gicp: Switch to MSI parent

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     cf8793968627f2699be2fd3de2d217e7de0fabae
Gitweb:        https://git.kernel.org/tip/cf8793968627f2699be2fd3de2d217e7de0fabae
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:58 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:25 +02:00

irqchip/mvebu-gicp: Switch to MSI parent

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
 drivers/irqchip/Kconfig          |  1 +-
 drivers/irqchip/irq-mvebu-gicp.c | 44 +++++++++++++------------------
 2 files changed, 20 insertions(+), 25 deletions(-)
drivers/irqchip/Kconfig +1 -0
index 5b93a56..85df667 100644
@@ -383,6 +383,7 @@ config MSCC_OCELOT_IRQ
 	select GENERIC_IRQ_CHIP
 
 config MVEBU_GICP
+	select IRQ_MSI_LIB
 	bool
 
 config MVEBU_ICU
drivers/irqchip/irq-mvebu-gicp.c +19 -25

74 lines not on this page · raw message

#62 lore raw

[tip: irq/msi] irqchip/mvebu-gicp: Switch to MSI parent

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     cdb238723018eb766040d7be5d879b4c81ad3d50
Gitweb:        https://git.kernel.org/tip/cdb238723018eb766040d7be5d879b4c81ad3d50
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:18:58 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:20 +02:00

irqchip/mvebu-gicp: Switch to MSI parent

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
 drivers/irqchip/Kconfig          |  1 +-
 drivers/irqchip/irq-mvebu-gicp.c | 44 +++++++++++++------------------
 2 files changed, 20 insertions(+), 25 deletions(-)
drivers/irqchip/Kconfig +1 -0
index e7a57b3..a0fa599 100644
@@ -371,6 +371,7 @@ config MSCC_OCELOT_IRQ
 	select GENERIC_IRQ_CHIP
 
 config MVEBU_GICP
+	select IRQ_MSI_LIB
 	bool
 
 config MVEBU_ICU
drivers/irqchip/irq-mvebu-gicp.c +19 -25

74 lines not on this page · raw message

#63 lore raw

PATCH v4 17/21 irqchip/mvebu-odmi: Switch to parent MSI

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---

---
 drivers/irqchip/Kconfig          |    1 +
 drivers/irqchip/irq-mvebu-odmi.c |   37 ++++++++++++++++++-------------------
 2 files changed, 19 insertions(+), 19 deletions(-)
drivers/irqchip/Kconfig +1 -0
@@ -379,6 +379,7 @@ config MVEBU_ICU
 
 config MVEBU_ODMI
 	bool
+	select IRQ_MSI_LIB
 	select GENERIC_MSI_IRQ
 
 config MVEBU_PIC
drivers/irqchip/irq-mvebu-odmi.c +18 -19
@@ -17,6 +17,9 @@
 #include <linux/msi.h>
 #include <linux/of_address.h>
 #include <linux/slab.h>
+
+#include "irq-msi-lib.h"
+
 #include <dt-bindings/interrupt-controller/arm-gic.h>
 
 #define GICP_ODMIN_SET			0x40
@@ -141,27 +144,29 @@ static void odmi_irq_domain_free(struct
 }
 
 static const struct irq_domain_ops odmi_domain_ops = {
+	.select	= msi_lib_irq_domain_select,
 	.alloc	= odmi_irq_domain_alloc,
 	.free	= odmi_irq_domain_free,
 };
 
-static struct irq_chip odmi_msi_irq_chip = {
-	.name	= "ODMI",
-};
+#define ODMI_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
+				  MSI_FLAG_USE_DEF_CHIP_OPS)
 
-static struct msi_domain_ops odmi_msi_ops = {
-};
+#define ODMI_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK)
 
-static struct msi_domain_info odmi_msi_domain_info = {
-	.flags	= (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS),
-	.ops	= &odmi_msi_ops,
-	.chip	= &odmi_msi_irq_chip,
+static const struct msi_parent_ops odmi_msi_parent_ops = {
+	.supported_flags	= ODMI_MSI_FLAGS_SUPPORTED,
+	.required_flags		= ODMI_MSI_FLAGS_REQUIRED,
+	.bus_select_token	= DOMAIN_BUS_GENERIC_MSI,
+	.bus_select_mask	= MATCH_PLATFORM_MSI,
+	.prefix			= "ODMI-",
+	.init_dev_msi_info	= msi_lib_init_dev_msi_info,
 };
 
 static int __init mvebu_odmi_init(struct device_node *node,
 				  struct device_node *parent)
 {
-	struct irq_domain *parent_domain, *inner_domain, *plat_domain;
+	struct irq_domain *parent_domain, *inner_domain;
 	int ret, i;
 
 	if (of_property_read_u32(node, "marvell,odmi-frames", &odmis_count))
@@ -208,18 +213,12 @@ static int __init mvebu_odmi_init(struct
 		goto err_unmap;
 	}
 
-	plat_domain = platform_msi_create_irq_domain(of_node_to_fwnode(node),
-						     &odmi_msi_domain_info,
-						     inner_domain);
-	if (!plat_domain) {
-		ret = -ENOMEM;
-		goto err_remove_inner;
-	}
+	irq_domain_update_bus_token(inner_domain, DOMAIN_BUS_GENERIC_MSI);
+	inner_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+	inner_domain->msi_parent_ops = &odmi_msi_parent_ops;
 
 	return 0;
 
-err_remove_inner:
-	irq_domain_remove(inner_domain);
 err_unmap:
 	for (i = 0; i < odmis_count; i++) {
 		struct odmi_data *odmi = &odmis[i];
#64 lore raw

[tip: irq/core] irqchip/mvebu-odmi: Switch to parent MSI

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     e7fa0449a49f4589ee5cb2183492865c18aef6c0
Gitweb:        https://git.kernel.org/tip/e7fa0449a49f4589ee5cb2183492865c18aef6c0
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:19:00 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:25 +02:00

irqchip/mvebu-odmi: Switch to parent MSI

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
 drivers/irqchip/Kconfig          |  1 +-
 drivers/irqchip/irq-mvebu-odmi.c | 37 +++++++++++++++----------------
 2 files changed, 19 insertions(+), 19 deletions(-)
drivers/irqchip/Kconfig +1 -0
index 85df667..d078bdc 100644
@@ -391,6 +391,7 @@ config MVEBU_ICU
 
 config MVEBU_ODMI
 	bool
+	select IRQ_MSI_LIB
 	select GENERIC_MSI_IRQ
 
 config MVEBU_PIC
drivers/irqchip/irq-mvebu-odmi.c +18 -19
index 1080915..ff19bfd 100644
@@ -17,6 +17,9 @@
 #include <linux/msi.h>
 #include <linux/of_address.h>
 #include <linux/slab.h>
+
+#include "irq-msi-lib.h"
+
 #include <dt-bindings/interrupt-controller/arm-gic.h>
 
 #define GICP_ODMIN_SET			0x40
@@ -141,27 +144,29 @@ static void odmi_irq_domain_free(struct irq_domain *domain,
 }
 
 static const struct irq_domain_ops odmi_domain_ops = {
+	.select	= msi_lib_irq_domain_select,
 	.alloc	= odmi_irq_domain_alloc,
 	.free	= odmi_irq_domain_free,
 };
 
-static struct irq_chip odmi_msi_irq_chip = {
-	.name	= "ODMI",
-};
+#define ODMI_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
+				  MSI_FLAG_USE_DEF_CHIP_OPS)
 
-static struct msi_domain_ops odmi_msi_ops = {
-};
+#define ODMI_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK)
 
-static struct msi_domain_info odmi_msi_domain_info = {
-	.flags	= (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS),
-	.ops	= &odmi_msi_ops,
-	.chip	= &odmi_msi_irq_chip,
+static const struct msi_parent_ops odmi_msi_parent_ops = {
+	.supported_flags	= ODMI_MSI_FLAGS_SUPPORTED,
+	.required_flags		= ODMI_MSI_FLAGS_REQUIRED,
+	.bus_select_token	= DOMAIN_BUS_GENERIC_MSI,
+	.bus_select_mask	= MATCH_PLATFORM_MSI,
+	.prefix			= "ODMI-",
+	.init_dev_msi_info	= msi_lib_init_dev_msi_info,
 };
 
 static int __init mvebu_odmi_init(struct device_node *node,
 				  struct device_node *parent)
 {
-	struct irq_domain *parent_domain, *inner_domain, *plat_domain;
+	struct irq_domain *parent_domain, *inner_domain;
 	int ret, i;
 
 	if (of_property_read_u32(node, "marvell,odmi-frames", &odmis_count))
@@ -208,18 +213,12 @@ static int __init mvebu_odmi_init(struct device_node *node,
 		goto err_unmap;
 	}
 
-	plat_domain = platform_msi_create_irq_domain(of_node_to_fwnode(node),
-						     &odmi_msi_domain_info,
-						     inner_domain);
-	if (!plat_domain) {
-		ret = -ENOMEM;
-		goto err_remove_inner;
-	}
+	irq_domain_update_bus_token(inner_domain, DOMAIN_BUS_GENERIC_MSI);
+	inner_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+	inner_domain->msi_parent_ops = &odmi_msi_parent_ops;
 
 	return 0;
 
-err_remove_inner:
-	irq_domain_remove(inner_domain);
 err_unmap:
 	for (i = 0; i < odmis_count; i++) {
 		struct odmi_data *odmi = &odmis[i];
#65 lore raw

[tip: irq/msi] irqchip/mvebu-odmi: Switch to parent MSI

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     e0b99c4c5917759c257a5c41d2c3e4d7c014578b
Gitweb:        https://git.kernel.org/tip/e0b99c4c5917759c257a5c41d2c3e4d7c014578b
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:19:00 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:21 +02:00

irqchip/mvebu-odmi: Switch to parent MSI

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
 drivers/irqchip/Kconfig          |  1 +-
 drivers/irqchip/irq-mvebu-odmi.c | 37 +++++++++++++++----------------
 2 files changed, 19 insertions(+), 19 deletions(-)
drivers/irqchip/Kconfig +1 -0
index a0fa599..4def6dc 100644
@@ -379,6 +379,7 @@ config MVEBU_ICU
 
 config MVEBU_ODMI
 	bool
+	select IRQ_MSI_LIB
 	select GENERIC_MSI_IRQ
 
 config MVEBU_PIC
drivers/irqchip/irq-mvebu-odmi.c +18 -19
index 1080915..ff19bfd 100644
@@ -17,6 +17,9 @@
 #include <linux/msi.h>
 #include <linux/of_address.h>
 #include <linux/slab.h>
+
+#include "irq-msi-lib.h"
+
 #include <dt-bindings/interrupt-controller/arm-gic.h>
 
 #define GICP_ODMIN_SET			0x40
@@ -141,27 +144,29 @@ static void odmi_irq_domain_free(struct irq_domain *domain,
 }
 
 static const struct irq_domain_ops odmi_domain_ops = {
+	.select	= msi_lib_irq_domain_select,
 	.alloc	= odmi_irq_domain_alloc,
 	.free	= odmi_irq_domain_free,
 };
 
-static struct irq_chip odmi_msi_irq_chip = {
-	.name	= "ODMI",
-};
+#define ODMI_MSI_FLAGS_REQUIRED  (MSI_FLAG_USE_DEF_DOM_OPS |	\
+				  MSI_FLAG_USE_DEF_CHIP_OPS)
 
-static struct msi_domain_ops odmi_msi_ops = {
-};
+#define ODMI_MSI_FLAGS_SUPPORTED (MSI_GENERIC_FLAGS_MASK)
 
-static struct msi_domain_info odmi_msi_domain_info = {
-	.flags	= (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS),
-	.ops	= &odmi_msi_ops,
-	.chip	= &odmi_msi_irq_chip,
+static const struct msi_parent_ops odmi_msi_parent_ops = {
+	.supported_flags	= ODMI_MSI_FLAGS_SUPPORTED,
+	.required_flags		= ODMI_MSI_FLAGS_REQUIRED,
+	.bus_select_token	= DOMAIN_BUS_GENERIC_MSI,
+	.bus_select_mask	= MATCH_PLATFORM_MSI,
+	.prefix			= "ODMI-",
+	.init_dev_msi_info	= msi_lib_init_dev_msi_info,
 };
 
 static int __init mvebu_odmi_init(struct device_node *node,
 				  struct device_node *parent)
 {
-	struct irq_domain *parent_domain, *inner_domain, *plat_domain;
+	struct irq_domain *parent_domain, *inner_domain;
 	int ret, i;
 
 	if (of_property_read_u32(node, "marvell,odmi-frames", &odmis_count))
@@ -208,18 +213,12 @@ static int __init mvebu_odmi_init(struct device_node *node,
 		goto err_unmap;
 	}
 
-	plat_domain = platform_msi_create_irq_domain(of_node_to_fwnode(node),
-						     &odmi_msi_domain_info,
-						     inner_domain);
-	if (!plat_domain) {
-		ret = -ENOMEM;
-		goto err_remove_inner;
-	}
+	irq_domain_update_bus_token(inner_domain, DOMAIN_BUS_GENERIC_MSI);
+	inner_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+	inner_domain->msi_parent_ops = &odmi_msi_parent_ops;
 
 	return 0;
 
-err_remove_inner:
-	irq_domain_remove(inner_domain);
 err_unmap:
 	for (i = 0; i < odmis_count; i++) {
 		struct odmi_data *odmi = &odmis[i];
#66 lore raw

PATCH v4 18/21 irqchip/irq-mvebu-sei: Switch to MSI parent

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
v3: enabled MSI_FLAG_PCI_MSI_MASK_PARENT in msi_parent_ops::supported_flags

---
 drivers/irqchip/irq-mvebu-sei.c |   52 +++++++++++++++-------------------------
 1 file changed, 20 insertions(+), 32 deletions(-)
drivers/irqchip/irq-mvebu-sei.c +20 -32
@@ -14,6 +14,8 @@
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
 
+#include "irq-msi-lib.h"
+
 /* Cause register */
 #define GICP_SECR(idx)		(0x0  + ((idx) * 0x4))
 /* Mask register */
@@ -190,6 +192,7 @@ static void mvebu_sei_domain_free(struct
 }
 
 static const struct irq_domain_ops mvebu_sei_domain_ops = {
+	.select	= msi_lib_irq_domain_select,
 	.alloc	= mvebu_sei_domain_alloc,
 	.free	= mvebu_sei_domain_free,
 };
@@ -307,21 +310,6 @@ static const struct irq_domain_ops mvebu
 	.free	= mvebu_sei_cp_domain_free,
 };
 
-static struct irq_chip mvebu_sei_msi_irq_chip = {
-	.name		= "SEI pMSI",
-	.irq_ack	= irq_chip_ack_parent,
-	.irq_set_type	= irq_chip_set_type_parent,
-};
-
-static struct msi_domain_ops mvebu_sei_msi_ops = {
-};
-
-static struct msi_domain_info mvebu_sei_msi_domain_info = {
-	.flags	= MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS,
-	.ops	= &mvebu_sei_msi_ops,
-	.chip	= &mvebu_sei_msi_irq_chip,
-};
-
 static void mvebu_sei_handle_cascade_irq(struct irq_desc *desc)
 {
 	struct mvebu_sei *sei = irq_desc_get_handler_data(desc);
@@ -360,10 +348,23 @@ static void mvebu_sei_reset(struct mvebu
 	}
 }
 
+#define SEI_MSI_FLAGS_REQUIRED	(MSI_FLAG_USE_DEF_DOM_OPS |	\
+				 MSI_FLAG_USE_DEF_CHIP_OPS)
+
+#define SEI_MSI_FLAGS_SUPPORTED	(MSI_GENERIC_FLAGS_MASK)
+
+static const struct msi_parent_ops sei_msi_parent_ops = {
+	.supported_flags	= SEI_MSI_FLAGS_SUPPORTED,
+	.required_flags		= SEI_MSI_FLAGS_REQUIRED,
+	.bus_select_mask	= MATCH_PLATFORM_MSI,
+	.bus_select_token	= DOMAIN_BUS_GENERIC_MSI,
+	.prefix			= "SEI-",
+	.init_dev_msi_info	= msi_lib_init_dev_msi_info,
+};
+
 static int mvebu_sei_probe(struct platform_device *pdev)
 {
 	struct device_node *node = pdev->dev.of_node;
-	struct irq_domain *plat_domain;
 	struct mvebu_sei *sei;
 	u32 parent_irq;
 	int ret;
@@ -440,33 +441,20 @@ static int mvebu_sei_probe(struct platfo
 	}
 
 	irq_domain_update_bus_token(sei->cp_domain, DOMAIN_BUS_GENERIC_MSI);
-
-	plat_domain = platform_msi_create_irq_domain(of_node_to_fwnode(node),
-						     &mvebu_sei_msi_domain_info,
-						     sei->cp_domain);
-	if (!plat_domain) {
-		pr_err("Failed to create CPs MSI domain\n");
-		ret = -ENOMEM;
-		goto remove_cp_domain;
-	}
+	sei->cp_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+	sei->cp_domain->msi_parent_ops = &sei_msi_parent_ops;
 
 	mvebu_sei_reset(sei);
 
-	irq_set_chained_handler_and_data(parent_irq,
-					 mvebu_sei_handle_cascade_irq,
-					 sei);
-
+	irq_set_chained_handler_and_data(parent_irq, mvebu_sei_handle_cascade_irq, sei);
 	return 0;
 
-remove_cp_domain:
-	irq_domain_remove(sei->cp_domain);
 remove_ap_domain:
 	irq_domain_remove(sei->ap_domain);
 remove_sei_domain:
 	irq_domain_remove(sei->sei_domain);
 dispose_irq:
 	irq_dispose_mapping(parent_irq);
-
 	return ret;
 }
 
#67 lore raw

[tip: irq/core] irqchip/irq-mvebu-sei: Switch to MSI parent

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     8c4eeedc3795172347abb5e1ad635415fba14f8a
Gitweb:        https://git.kernel.org/tip/8c4eeedc3795172347abb5e1ad635415fba14f8a
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:19:02 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:25 +02:00

irqchip/irq-mvebu-sei: Switch to MSI parent

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
 drivers/irqchip/irq-mvebu-sei.c | 52 ++++++++++++--------------------
 1 file changed, 20 insertions(+), 32 deletions(-)
drivers/irqchip/irq-mvebu-sei.c +20 -32
index a48dbe9..f8c70f2 100644
@@ -14,6 +14,8 @@
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
 
+#include "irq-msi-lib.h"
+
 /* Cause register */
 #define GICP_SECR(idx)		(0x0  + ((idx) * 0x4))
 /* Mask register */
@@ -190,6 +192,7 @@ static void mvebu_sei_domain_free(struct irq_domain *domain, unsigned int virq,
 }
 
 static const struct irq_domain_ops mvebu_sei_domain_ops = {
+	.select	= msi_lib_irq_domain_select,
 	.alloc	= mvebu_sei_domain_alloc,
 	.free	= mvebu_sei_domain_free,
 };
@@ -307,21 +310,6 @@ static const struct irq_domain_ops mvebu_sei_cp_domain_ops = {
 	.free	= mvebu_sei_cp_domain_free,
 };
 
-static struct irq_chip mvebu_sei_msi_irq_chip = {
-	.name		= "SEI pMSI",
-	.irq_ack	= irq_chip_ack_parent,
-	.irq_set_type	= irq_chip_set_type_parent,
-};
-
-static struct msi_domain_ops mvebu_sei_msi_ops = {
-};
-
-static struct msi_domain_info mvebu_sei_msi_domain_info = {
-	.flags	= MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS,
-	.ops	= &mvebu_sei_msi_ops,
-	.chip	= &mvebu_sei_msi_irq_chip,
-};
-
 static void mvebu_sei_handle_cascade_irq(struct irq_desc *desc)
 {
 	struct mvebu_sei *sei = irq_desc_get_handler_data(desc);
@@ -360,10 +348,23 @@ static void mvebu_sei_reset(struct mvebu_sei *sei)
 	}
 }
 
+#define SEI_MSI_FLAGS_REQUIRED	(MSI_FLAG_USE_DEF_DOM_OPS |	\
+				 MSI_FLAG_USE_DEF_CHIP_OPS)
+
+#define SEI_MSI_FLAGS_SUPPORTED	(MSI_GENERIC_FLAGS_MASK)
+
+static const struct msi_parent_ops sei_msi_parent_ops = {
+	.supported_flags	= SEI_MSI_FLAGS_SUPPORTED,
+	.required_flags		= SEI_MSI_FLAGS_REQUIRED,
+	.bus_select_mask	= MATCH_PLATFORM_MSI,
+	.bus_select_token	= DOMAIN_BUS_GENERIC_MSI,
+	.prefix			= "SEI-",
+	.init_dev_msi_info	= msi_lib_init_dev_msi_info,
+};
+
 static int mvebu_sei_probe(struct platform_device *pdev)
 {
 	struct device_node *node = pdev->dev.of_node;
-	struct irq_domain *plat_domain;
 	struct mvebu_sei *sei;
 	u32 parent_irq;
 	int ret;
@@ -440,33 +441,20 @@ static int mvebu_sei_probe(struct platform_device *pdev)
 	}
 
 	irq_domain_update_bus_token(sei->cp_domain, DOMAIN_BUS_GENERIC_MSI);
-
-	plat_domain = platform_msi_create_irq_domain(of_node_to_fwnode(node),
-						     &mvebu_sei_msi_domain_info,
-						     sei->cp_domain);
-	if (!plat_domain) {
-		pr_err("Failed to create CPs MSI domain\n");
-		ret = -ENOMEM;
-		goto remove_cp_domain;
-	}
+	sei->cp_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+	sei->cp_domain->msi_parent_ops = &sei_msi_parent_ops;
 
 	mvebu_sei_reset(sei);
 
-	irq_set_chained_handler_and_data(parent_irq,
-					 mvebu_sei_handle_cascade_irq,
-					 sei);
-
+	irq_set_chained_handler_and_data(parent_irq, mvebu_sei_handle_cascade_irq, sei);
 	return 0;
 
-remove_cp_domain:
-	irq_domain_remove(sei->cp_domain);
 remove_ap_domain:
 	irq_domain_remove(sei->ap_domain);
 remove_sei_domain:
 	irq_domain_remove(sei->sei_domain);
 dispose_irq:
 	irq_dispose_mapping(parent_irq);
-
 	return ret;
 }
 
#68 lore raw

[tip: irq/msi] irqchip/irq-mvebu-sei: Switch to MSI parent

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     fbdf14e90ce445fedfb387413c3d8dc9d90db2a7
Gitweb:        https://git.kernel.org/tip/fbdf14e90ce445fedfb387413c3d8dc9d90db2a7
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:19:02 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:21 +02:00

irqchip/irq-mvebu-sei: Switch to MSI parent

All platform MSI users and the PCI/MSI code handle per device MSI domains when the irqdomain associated to the device provides MSI parent functionality.

Remove the "global" platform domain related code and provide the MSI parent functionality by filling in msi_parent_ops.

---
 drivers/irqchip/irq-mvebu-sei.c | 52 ++++++++++++--------------------
 1 file changed, 20 insertions(+), 32 deletions(-)
drivers/irqchip/irq-mvebu-sei.c +20 -32
index a48dbe9..f8c70f2 100644
@@ -14,6 +14,8 @@
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
 
+#include "irq-msi-lib.h"
+
 /* Cause register */
 #define GICP_SECR(idx)		(0x0  + ((idx) * 0x4))
 /* Mask register */
@@ -190,6 +192,7 @@ static void mvebu_sei_domain_free(struct irq_domain *domain, unsigned int virq,
 }
 
 static const struct irq_domain_ops mvebu_sei_domain_ops = {
+	.select	= msi_lib_irq_domain_select,
 	.alloc	= mvebu_sei_domain_alloc,
 	.free	= mvebu_sei_domain_free,
 };
@@ -307,21 +310,6 @@ static const struct irq_domain_ops mvebu_sei_cp_domain_ops = {
 	.free	= mvebu_sei_cp_domain_free,
 };
 
-static struct irq_chip mvebu_sei_msi_irq_chip = {
-	.name		= "SEI pMSI",
-	.irq_ack	= irq_chip_ack_parent,
-	.irq_set_type	= irq_chip_set_type_parent,
-};
-
-static struct msi_domain_ops mvebu_sei_msi_ops = {
-};
-
-static struct msi_domain_info mvebu_sei_msi_domain_info = {
-	.flags	= MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS,
-	.ops	= &mvebu_sei_msi_ops,
-	.chip	= &mvebu_sei_msi_irq_chip,
-};
-
 static void mvebu_sei_handle_cascade_irq(struct irq_desc *desc)
 {
 	struct mvebu_sei *sei = irq_desc_get_handler_data(desc);
@@ -360,10 +348,23 @@ static void mvebu_sei_reset(struct mvebu_sei *sei)
 	}
 }
 
+#define SEI_MSI_FLAGS_REQUIRED	(MSI_FLAG_USE_DEF_DOM_OPS |	\
+				 MSI_FLAG_USE_DEF_CHIP_OPS)
+
+#define SEI_MSI_FLAGS_SUPPORTED	(MSI_GENERIC_FLAGS_MASK)
+
+static const struct msi_parent_ops sei_msi_parent_ops = {
+	.supported_flags	= SEI_MSI_FLAGS_SUPPORTED,
+	.required_flags		= SEI_MSI_FLAGS_REQUIRED,
+	.bus_select_mask	= MATCH_PLATFORM_MSI,
+	.bus_select_token	= DOMAIN_BUS_GENERIC_MSI,
+	.prefix			= "SEI-",
+	.init_dev_msi_info	= msi_lib_init_dev_msi_info,
+};
+
 static int mvebu_sei_probe(struct platform_device *pdev)
 {
 	struct device_node *node = pdev->dev.of_node;
-	struct irq_domain *plat_domain;
 	struct mvebu_sei *sei;
 	u32 parent_irq;
 	int ret;
@@ -440,33 +441,20 @@ static int mvebu_sei_probe(struct platform_device *pdev)
 	}
 
 	irq_domain_update_bus_token(sei->cp_domain, DOMAIN_BUS_GENERIC_MSI);
-
-	plat_domain = platform_msi_create_irq_domain(of_node_to_fwnode(node),
-						     &mvebu_sei_msi_domain_info,
-						     sei->cp_domain);
-	if (!plat_domain) {
-		pr_err("Failed to create CPs MSI domain\n");
-		ret = -ENOMEM;
-		goto remove_cp_domain;
-	}
+	sei->cp_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
+	sei->cp_domain->msi_parent_ops = &sei_msi_parent_ops;
 
 	mvebu_sei_reset(sei);
 
-	irq_set_chained_handler_and_data(parent_irq,
-					 mvebu_sei_handle_cascade_irq,
-					 sei);
-
+	irq_set_chained_handler_and_data(parent_irq, mvebu_sei_handle_cascade_irq, sei);
 	return 0;
 
-remove_cp_domain:
-	irq_domain_remove(sei->cp_domain);
 remove_ap_domain:
 	irq_domain_remove(sei->ap_domain);
 remove_sei_domain:
 	irq_domain_remove(sei->sei_domain);
 dispose_irq:
 	irq_dispose_mapping(parent_irq);
-
 	return ret;
 }
 
#69 lore raw

PATCH v4 19/21 irqchip/irq-mvebu-icu: Remove platform MSI leftovers

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

All related domains provide MSI parent functionality, so the fallback code to the original platform MSI implementation is not longer required.

---
 drivers/irqchip/irq-mvebu-icu.c | 212 +-------------------------------
 1 file changed, 6 insertions(+), 206 deletions(-)
drivers/irqchip/irq-mvebu-icu.c +6 -206
index 2a210cd9482e..b337f6c05f18 100644
@@ -129,196 +129,6 @@ static void mvebu_icu_init(struct mvebu_icu *icu,
 	writel_relaxed(msg[1].address_lo, icu->base + subset->offset_clr_al);
 }
 
-/* Start of area to be removed once all parent chips provide MSI parent */
-
-struct mvebu_icu_irq_data {
-	struct mvebu_icu *icu;
-	unsigned int icu_group;
-	unsigned int type;
-};
-
-static void mvebu_icu_write_msg(struct msi_desc *desc, struct msi_msg *msg)
-{
-	struct irq_data *d = irq_get_irq_data(desc->irq);
-	struct mvebu_icu_msi_data *msi_data = platform_msi_get_host_data(d->domain);
-	struct mvebu_icu_irq_data *icu_irqd = d->chip_data;
-	struct mvebu_icu *icu = icu_irqd->icu;
-	unsigned int icu_int;
-
-	if (msg->address_lo || msg->address_hi) {
-		/* One off initialization per domain */
-		mvebu_icu_init(icu, msi_data, msg);
-		/* Configure the ICU with irq number & type */
-		icu_int = msg->data | ICU_INT_ENABLE;
-		if (icu_irqd->type & IRQ_TYPE_EDGE_RISING)
-			icu_int |= ICU_IS_EDGE;
-		icu_int |= icu_irqd->icu_group << ICU_GROUP_SHIFT;
-	} else {
-		/* De-configure the ICU */
-		icu_int = 0;
-	}
-
-	writel_relaxed(icu_int, icu->base + ICU_INT_CFG(d->hwirq));
-
-	/*
-	 * The SATA unit has 2 ports, and a dedicated ICU entry per
-	 * port. The ahci sata driver supports only one irq interrupt
-	 * per SATA unit. To solve this conflict, we configure the 2
-	 * SATA wired interrupts in the south bridge into 1 GIC
-	 * interrupt in the north bridge. Even if only a single port
-	 * is enabled, if sata node is enabled, both interrupts are
-	 * configured (regardless of which port is actually in use).
-	 */
-	if (d->hwirq == ICU_SATA0_ICU_ID || d->hwirq == ICU_SATA1_ICU_ID) {
-		writel_relaxed(icu_int,
-			       icu->base + ICU_INT_CFG(ICU_SATA0_ICU_ID));
-		writel_relaxed(icu_int,
-			       icu->base + ICU_INT_CFG(ICU_SATA1_ICU_ID));
-	}
-}
-
-static struct irq_chip mvebu_icu_nsr_chip = {
-	.name			= "ICU-NSR",
-	.irq_mask		= irq_chip_mask_parent,
-	.irq_unmask		= irq_chip_unmask_parent,
-	.irq_eoi		= irq_chip_eoi_parent,
-	.irq_set_type		= irq_chip_set_type_parent,
-	.irq_set_affinity	= irq_chip_set_affinity_parent,
-};
-
-static struct irq_chip mvebu_icu_sei_chip = {
-	.name			= "ICU-SEI",
-	.irq_ack		= irq_chip_ack_parent,
-	.irq_mask		= irq_chip_mask_parent,
-	.irq_unmask		= irq_chip_unmask_parent,
-	.irq_set_type		= irq_chip_set_type_parent,
-	.irq_set_affinity	= irq_chip_set_affinity_parent,
-};
-
-static int
-mvebu_icu_irq_domain_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
-			       unsigned long *hwirq, unsigned int *type)
-{
-	unsigned int param_count = static_branch_unlikely(&legacy_bindings) ? 3 : 2;
-	struct mvebu_icu_msi_data *msi_data = platform_msi_get_host_data(d);
-	struct mvebu_icu *icu = msi_data->icu;
-
-	/* Check the count of the parameters in dt */
-	if (WARN_ON(fwspec->param_count != param_count)) {
-		dev_err(icu->dev, "wrong ICU parameter count %d\n",
-			fwspec->param_count);
-		return -EINVAL;
-	}
-
-	if (static_branch_unlikely(&legacy_bindings)) {
-		*hwirq = fwspec->param[1];
-		*type = fwspec->param[2] & IRQ_TYPE_SENSE_MASK;
-		if (fwspec->param[0] != ICU_GRP_NSR) {
-			dev_err(icu->dev, "wrong ICU group type %x\n",
-				fwspec->param[0]);
-			return -EINVAL;
-		}
-	} else {
-		*hwirq = fwspec->param[0];
-		*type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
-
-		/*
-		 * The ICU receives level interrupts. While the NSR are also
-		 * level interrupts, SEI are edge interrupts. Force the type
-		 * here in this case. Please note that this makes the interrupt
-		 * handling unreliable.
-		 */
-		if (msi_data->subset_data->icu_group == ICU_GRP_SEI)
-			*type = IRQ_TYPE_EDGE_RISING;
-	}
-
-	if (*hwirq >= ICU_MAX_IRQS) {
-		dev_err(icu->dev, "invalid interrupt number %ld\n", *hwirq);
-		return -EINVAL;
-	}
-
-	return 0;
-}
-
-static int
-mvebu_icu_irq_domain_alloc(struct irq_domain *domain, unsigned int virq,
-			   unsigned int nr_irqs, void *args)
-{
-	int err;
-	unsigned long hwirq;
-	struct irq_fwspec *fwspec = args;
-	struct mvebu_icu_msi_data *msi_data = platform_msi_get_host_data(domain);
-	struct mvebu_icu *icu = msi_data->icu;
-	struct mvebu_icu_irq_data *icu_irqd;
-	struct irq_chip *chip = &mvebu_icu_nsr_chip;
-
-	icu_irqd = kmalloc(sizeof(*icu_irqd), GFP_KERNEL);
-	if (!icu_irqd)
-		return -ENOMEM;
-
-	err = mvebu_icu_irq_domain_translate(domain, fwspec, &hwirq,
-					     &icu_irqd->type);
-	if (err) {
-		dev_err(icu->dev, "failed to translate ICU parameters\n");
-		goto free_irqd;
-	}
-
-	if (static_branch_unlikely(&legacy_bindings))
-		icu_irqd->icu_group = fwspec->param[0];
-	else
-		icu_irqd->icu_group = msi_data->subset_data->icu_group;
-	icu_irqd->icu = icu;
-
-	err = platform_msi_device_domain_alloc(domain, virq, nr_irqs);
-	if (err) {
-		dev_err(icu->dev, "failed to allocate ICU interrupt in parent domain\n");
-		goto free_irqd;
-	}
-
-	/* Make sure there is no interrupt left pending by the firmware */
-	err = irq_set_irqchip_state(virq, IRQCHIP_STATE_PENDING, false);
-	if (err)
-		goto free_msi;
-
-	if (icu_irqd->icu_group == ICU_GRP_SEI)
-		chip = &mvebu_icu_sei_chip;
-
-	err = irq_domain_set_hwirq_and_chip(domain, virq, hwirq,
-					    chip, icu_irqd);
-	if (err) {
-		dev_err(icu->dev, "failed to set the data to IRQ domain\n");
-		goto free_msi;
-	}
-
-	return 0;
-
-free_msi:
-	platform_msi_device_domain_free(domain, virq, nr_irqs);
-free_irqd:
-	kfree(icu_irqd);
-	return err;
-}
-
-static void
-mvebu_icu_irq_domain_free(struct irq_domain *domain, unsigned int virq,
-			  unsigned int nr_irqs)
-{
-	struct irq_data *d = irq_get_irq_data(virq);
-	struct mvebu_icu_irq_data *icu_irqd = d->chip_data;
-
-	kfree(icu_irqd);
-
-	platform_msi_device_domain_free(domain, virq, nr_irqs);
-}
-
-static const struct irq_domain_ops mvebu_icu_domain_ops = {
-	.translate = mvebu_icu_irq_domain_translate,
-	.alloc     = mvebu_icu_irq_domain_alloc,
-	.free      = mvebu_icu_irq_domain_free,
-};
-
-/* End of removal area */
-
 static int mvebu_icu_msi_init(struct irq_domain *domain, struct msi_domain_info *info,
 			      unsigned int virq, irq_hw_number_t hwirq, msi_alloc_info_t *arg)
 {
@@ -448,9 +258,10 @@ static const struct of_device_id mvebu_icu_subset_of_match[] = {
 
 static int mvebu_icu_subset_probe(struct platform_device *pdev)
 {
+	const struct msi_domain_template *tmpl;
 	struct mvebu_icu_msi_data *msi_data;
 	struct device *dev = &pdev->dev;
-	struct irq_domain *irq_domain;
+	bool sei;
 
 	msi_data = devm_kzalloc(dev, sizeof(*msi_data), GFP_KERNEL);
 	if (!msi_data)
@@ -471,22 +282,11 @@ static int mvebu_icu_subset_probe(struct platform_device *pdev)
 	if (!irq_domain_get_of_node(dev->msi.domain))
 		return -ENODEV;
 
-	if (irq_domain_is_msi_parent(dev->msi.domain)) {
-		bool sei = msi_data->subset_data->icu_group == ICU_GRP_SEI;
-		const struct msi_domain_template *tmpl;
-
-		tmpl = sei ? &mvebu_icu_sei_msi_template : &mvebu_icu_nsr_msi_template;
-
-		if (!msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN, tmpl,
-						  ICU_MAX_IRQS, NULL, msi_data))
-			return -ENOMEM;
-	}
+	sei = msi_data->subset_data->icu_group == ICU_GRP_SEI;
+	tmpl = sei ? &mvebu_icu_sei_msi_template : &mvebu_icu_nsr_msi_template;
 
-	irq_domain = platform_msi_create_device_tree_domain(dev, ICU_MAX_IRQS,
-							    mvebu_icu_write_msg,
-							    &mvebu_icu_domain_ops,
-							    msi_data);
-	if (!irq_domain) {
+	if (!msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN, tmpl,
+					  ICU_MAX_IRQS, NULL, msi_data)) {
 		dev_err(dev, "Failed to create ICU MSI domain\n");
 		return -ENOMEM;
 	}
Signature · 1 line
2.34.1
#70 lore raw

[tip: irq/core] irqchip/irq-mvebu-icu: Remove platform MSI leftovers

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     5aef068dd2adc5192eda5a471a502301bf876b62
Gitweb:        https://git.kernel.org/tip/5aef068dd2adc5192eda5a471a502301bf876b62
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:19:04 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:25 +02:00

irqchip/irq-mvebu-icu: Remove platform MSI leftovers

All related domains provide MSI parent functionality, so the fallback code to the original platform MSI implementation is not longer required.

---
 drivers/irqchip/irq-mvebu-icu.c | 212 +-------------------------------
 1 file changed, 6 insertions(+), 206 deletions(-)
drivers/irqchip/irq-mvebu-icu.c +6 -206
index 2a210cd..b337f6c 100644
@@ -129,196 +129,6 @@ static void mvebu_icu_init(struct mvebu_icu *icu,
 	writel_relaxed(msg[1].address_lo, icu->base + subset->offset_clr_al);
 }
 
-/* Start of area to be removed once all parent chips provide MSI parent */
-
-struct mvebu_icu_irq_data {
-	struct mvebu_icu *icu;
-	unsigned int icu_group;
-	unsigned int type;
-};
-
-static void mvebu_icu_write_msg(struct msi_desc *desc, struct msi_msg *msg)
-{
-	struct irq_data *d = irq_get_irq_data(desc->irq);
-	struct mvebu_icu_msi_data *msi_data = platform_msi_get_host_data(d->domain);
-	struct mvebu_icu_irq_data *icu_irqd = d->chip_data;
-	struct mvebu_icu *icu = icu_irqd->icu;
-	unsigned int icu_int;
-
-	if (msg->address_lo || msg->address_hi) {
-		/* One off initialization per domain */
-		mvebu_icu_init(icu, msi_data, msg);
-		/* Configure the ICU with irq number & type */
-		icu_int = msg->data | ICU_INT_ENABLE;
-		if (icu_irqd->type & IRQ_TYPE_EDGE_RISING)
-			icu_int |= ICU_IS_EDGE;
-		icu_int |= icu_irqd->icu_group << ICU_GROUP_SHIFT;
-	} else {
-		/* De-configure the ICU */
-		icu_int = 0;
-	}
-
-	writel_relaxed(icu_int, icu->base + ICU_INT_CFG(d->hwirq));
-
-	/*
-	 * The SATA unit has 2 ports, and a dedicated ICU entry per
-	 * port. The ahci sata driver supports only one irq interrupt
-	 * per SATA unit. To solve this conflict, we configure the 2
-	 * SATA wired interrupts in the south bridge into 1 GIC
-	 * interrupt in the north bridge. Even if only a single port
-	 * is enabled, if sata node is enabled, both interrupts are
-	 * configured (regardless of which port is actually in use).
-	 */
-	if (d->hwirq == ICU_SATA0_ICU_ID || d->hwirq == ICU_SATA1_ICU_ID) {
-		writel_relaxed(icu_int,
-			       icu->base + ICU_INT_CFG(ICU_SATA0_ICU_ID));
-		writel_relaxed(icu_int,
-			       icu->base + ICU_INT_CFG(ICU_SATA1_ICU_ID));
-	}
-}
-
-static struct irq_chip mvebu_icu_nsr_chip = {
-	.name			= "ICU-NSR",
-	.irq_mask		= irq_chip_mask_parent,
-	.irq_unmask		= irq_chip_unmask_parent,
-	.irq_eoi		= irq_chip_eoi_parent,
-	.irq_set_type		= irq_chip_set_type_parent,
-	.irq_set_affinity	= irq_chip_set_affinity_parent,
-};
-
-static struct irq_chip mvebu_icu_sei_chip = {
-	.name			= "ICU-SEI",
-	.irq_ack		= irq_chip_ack_parent,
-	.irq_mask		= irq_chip_mask_parent,
-	.irq_unmask		= irq_chip_unmask_parent,
-	.irq_set_type		= irq_chip_set_type_parent,
-	.irq_set_affinity	= irq_chip_set_affinity_parent,
-};
-
-static int
-mvebu_icu_irq_domain_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
-			       unsigned long *hwirq, unsigned int *type)
-{
-	unsigned int param_count = static_branch_unlikely(&legacy_bindings) ? 3 : 2;
-	struct mvebu_icu_msi_data *msi_data = platform_msi_get_host_data(d);
-	struct mvebu_icu *icu = msi_data->icu;
-
-	/* Check the count of the parameters in dt */
-	if (WARN_ON(fwspec->param_count != param_count)) {
-		dev_err(icu->dev, "wrong ICU parameter count %d\n",
-			fwspec->param_count);
-		return -EINVAL;
-	}
-
-	if (static_branch_unlikely(&legacy_bindings)) {
-		*hwirq = fwspec->param[1];
-		*type = fwspec->param[2] & IRQ_TYPE_SENSE_MASK;
-		if (fwspec->param[0] != ICU_GRP_NSR) {
-			dev_err(icu->dev, "wrong ICU group type %x\n",
-				fwspec->param[0]);
-			return -EINVAL;
-		}
-	} else {
-		*hwirq = fwspec->param[0];
-		*type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
-
-		/*
-		 * The ICU receives level interrupts. While the NSR are also
-		 * level interrupts, SEI are edge interrupts. Force the type
-		 * here in this case. Please note that this makes the interrupt
-		 * handling unreliable.
-		 */
-		if (msi_data->subset_data->icu_group == ICU_GRP_SEI)
-			*type = IRQ_TYPE_EDGE_RISING;
-	}
-
-	if (*hwirq >= ICU_MAX_IRQS) {
-		dev_err(icu->dev, "invalid interrupt number %ld\n", *hwirq);
-		return -EINVAL;
-	}
-
-	return 0;
-}
-
-static int
-mvebu_icu_irq_domain_alloc(struct irq_domain *domain, unsigned int virq,
-			   unsigned int nr_irqs, void *args)
-{
-	int err;
-	unsigned long hwirq;
-	struct irq_fwspec *fwspec = args;
-	struct mvebu_icu_msi_data *msi_data = platform_msi_get_host_data(domain);
-	struct mvebu_icu *icu = msi_data->icu;
-	struct mvebu_icu_irq_data *icu_irqd;
-	struct irq_chip *chip = &mvebu_icu_nsr_chip;
-
-	icu_irqd = kmalloc(sizeof(*icu_irqd), GFP_KERNEL);
-	if (!icu_irqd)
-		return -ENOMEM;
-
-	err = mvebu_icu_irq_domain_translate(domain, fwspec, &hwirq,
-					     &icu_irqd->type);
-	if (err) {
-		dev_err(icu->dev, "failed to translate ICU parameters\n");
-		goto free_irqd;
-	}
-
-	if (static_branch_unlikely(&legacy_bindings))
-		icu_irqd->icu_group = fwspec->param[0];
-	else
-		icu_irqd->icu_group = msi_data->subset_data->icu_group;
-	icu_irqd->icu = icu;
-
-	err = platform_msi_device_domain_alloc(domain, virq, nr_irqs);
-	if (err) {
-		dev_err(icu->dev, "failed to allocate ICU interrupt in parent domain\n");
-		goto free_irqd;
-	}
-
-	/* Make sure there is no interrupt left pending by the firmware */
-	err = irq_set_irqchip_state(virq, IRQCHIP_STATE_PENDING, false);
-	if (err)
-		goto free_msi;
-
-	if (icu_irqd->icu_group == ICU_GRP_SEI)
-		chip = &mvebu_icu_sei_chip;
-
-	err = irq_domain_set_hwirq_and_chip(domain, virq, hwirq,
-					    chip, icu_irqd);
-	if (err) {
-		dev_err(icu->dev, "failed to set the data to IRQ domain\n");
-		goto free_msi;
-	}
-
-	return 0;
-
-free_msi:
-	platform_msi_device_domain_free(domain, virq, nr_irqs);
-free_irqd:
-	kfree(icu_irqd);
-	return err;
-}
-
-static void
-mvebu_icu_irq_domain_free(struct irq_domain *domain, unsigned int virq,
-			  unsigned int nr_irqs)
-{
-	struct irq_data *d = irq_get_irq_data(virq);
-	struct mvebu_icu_irq_data *icu_irqd = d->chip_data;
-
-	kfree(icu_irqd);
-
-	platform_msi_device_domain_free(domain, virq, nr_irqs);
-}
-
-static const struct irq_domain_ops mvebu_icu_domain_ops = {
-	.translate = mvebu_icu_irq_domain_translate,
-	.alloc     = mvebu_icu_irq_domain_alloc,
-	.free      = mvebu_icu_irq_domain_free,
-};
-
-/* End of removal area */
-
 static int mvebu_icu_msi_init(struct irq_domain *domain, struct msi_domain_info *info,
 			      unsigned int virq, irq_hw_number_t hwirq, msi_alloc_info_t *arg)
 {
@@ -448,9 +258,10 @@ static const struct of_device_id mvebu_icu_subset_of_match[] = {
 
 static int mvebu_icu_subset_probe(struct platform_device *pdev)
 {
+	const struct msi_domain_template *tmpl;
 	struct mvebu_icu_msi_data *msi_data;
 	struct device *dev = &pdev->dev;
-	struct irq_domain *irq_domain;
+	bool sei;
 
 	msi_data = devm_kzalloc(dev, sizeof(*msi_data), GFP_KERNEL);
 	if (!msi_data)
@@ -471,22 +282,11 @@ static int mvebu_icu_subset_probe(struct platform_device *pdev)
 	if (!irq_domain_get_of_node(dev->msi.domain))
 		return -ENODEV;
 
-	if (irq_domain_is_msi_parent(dev->msi.domain)) {
-		bool sei = msi_data->subset_data->icu_group == ICU_GRP_SEI;
-		const struct msi_domain_template *tmpl;
-
-		tmpl = sei ? &mvebu_icu_sei_msi_template : &mvebu_icu_nsr_msi_template;
-
-		if (!msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN, tmpl,
-						  ICU_MAX_IRQS, NULL, msi_data))
-			return -ENOMEM;
-	}
+	sei = msi_data->subset_data->icu_group == ICU_GRP_SEI;
+	tmpl = sei ? &mvebu_icu_sei_msi_template : &mvebu_icu_nsr_msi_template;
 
-	irq_domain = platform_msi_create_device_tree_domain(dev, ICU_MAX_IRQS,
-							    mvebu_icu_write_msg,
-							    &mvebu_icu_domain_ops,
-							    msi_data);
-	if (!irq_domain) {
+	if (!msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN, tmpl,
+					  ICU_MAX_IRQS, NULL, msi_data)) {
 		dev_err(dev, "Failed to create ICU MSI domain\n");
 		return -ENOMEM;
 	}
#71 lore raw

[tip: irq/msi] irqchip/irq-mvebu-icu: Remove platform MSI leftovers

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     ce44d1ff463dcf812b43c71a0e16859e3451bc1e
Gitweb:        https://git.kernel.org/tip/ce44d1ff463dcf812b43c71a0e16859e3451bc1e
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:19:04 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:21 +02:00

irqchip/irq-mvebu-icu: Remove platform MSI leftovers

All related domains provide MSI parent functionality, so the fallback code to the original platform MSI implementation is not longer required.

---
 drivers/irqchip/irq-mvebu-icu.c | 212 +-------------------------------
 1 file changed, 6 insertions(+), 206 deletions(-)
drivers/irqchip/irq-mvebu-icu.c +6 -206
index 2a210cd..b337f6c 100644
@@ -129,196 +129,6 @@ static void mvebu_icu_init(struct mvebu_icu *icu,
 	writel_relaxed(msg[1].address_lo, icu->base + subset->offset_clr_al);
 }
 
-/* Start of area to be removed once all parent chips provide MSI parent */
-
-struct mvebu_icu_irq_data {
-	struct mvebu_icu *icu;
-	unsigned int icu_group;
-	unsigned int type;
-};
-
-static void mvebu_icu_write_msg(struct msi_desc *desc, struct msi_msg *msg)
-{
-	struct irq_data *d = irq_get_irq_data(desc->irq);
-	struct mvebu_icu_msi_data *msi_data = platform_msi_get_host_data(d->domain);
-	struct mvebu_icu_irq_data *icu_irqd = d->chip_data;
-	struct mvebu_icu *icu = icu_irqd->icu;
-	unsigned int icu_int;
-
-	if (msg->address_lo || msg->address_hi) {
-		/* One off initialization per domain */
-		mvebu_icu_init(icu, msi_data, msg);
-		/* Configure the ICU with irq number & type */
-		icu_int = msg->data | ICU_INT_ENABLE;
-		if (icu_irqd->type & IRQ_TYPE_EDGE_RISING)
-			icu_int |= ICU_IS_EDGE;
-		icu_int |= icu_irqd->icu_group << ICU_GROUP_SHIFT;
-	} else {
-		/* De-configure the ICU */
-		icu_int = 0;
-	}
-
-	writel_relaxed(icu_int, icu->base + ICU_INT_CFG(d->hwirq));
-
-	/*
-	 * The SATA unit has 2 ports, and a dedicated ICU entry per
-	 * port. The ahci sata driver supports only one irq interrupt
-	 * per SATA unit. To solve this conflict, we configure the 2
-	 * SATA wired interrupts in the south bridge into 1 GIC
-	 * interrupt in the north bridge. Even if only a single port
-	 * is enabled, if sata node is enabled, both interrupts are
-	 * configured (regardless of which port is actually in use).
-	 */
-	if (d->hwirq == ICU_SATA0_ICU_ID || d->hwirq == ICU_SATA1_ICU_ID) {
-		writel_relaxed(icu_int,
-			       icu->base + ICU_INT_CFG(ICU_SATA0_ICU_ID));
-		writel_relaxed(icu_int,
-			       icu->base + ICU_INT_CFG(ICU_SATA1_ICU_ID));
-	}
-}
-
-static struct irq_chip mvebu_icu_nsr_chip = {
-	.name			= "ICU-NSR",
-	.irq_mask		= irq_chip_mask_parent,
-	.irq_unmask		= irq_chip_unmask_parent,
-	.irq_eoi		= irq_chip_eoi_parent,
-	.irq_set_type		= irq_chip_set_type_parent,
-	.irq_set_affinity	= irq_chip_set_affinity_parent,
-};
-
-static struct irq_chip mvebu_icu_sei_chip = {
-	.name			= "ICU-SEI",
-	.irq_ack		= irq_chip_ack_parent,
-	.irq_mask		= irq_chip_mask_parent,
-	.irq_unmask		= irq_chip_unmask_parent,
-	.irq_set_type		= irq_chip_set_type_parent,
-	.irq_set_affinity	= irq_chip_set_affinity_parent,
-};
-
-static int
-mvebu_icu_irq_domain_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
-			       unsigned long *hwirq, unsigned int *type)
-{
-	unsigned int param_count = static_branch_unlikely(&legacy_bindings) ? 3 : 2;
-	struct mvebu_icu_msi_data *msi_data = platform_msi_get_host_data(d);
-	struct mvebu_icu *icu = msi_data->icu;
-
-	/* Check the count of the parameters in dt */
-	if (WARN_ON(fwspec->param_count != param_count)) {
-		dev_err(icu->dev, "wrong ICU parameter count %d\n",
-			fwspec->param_count);
-		return -EINVAL;
-	}
-
-	if (static_branch_unlikely(&legacy_bindings)) {
-		*hwirq = fwspec->param[1];
-		*type = fwspec->param[2] & IRQ_TYPE_SENSE_MASK;
-		if (fwspec->param[0] != ICU_GRP_NSR) {
-			dev_err(icu->dev, "wrong ICU group type %x\n",
-				fwspec->param[0]);
-			return -EINVAL;
-		}
-	} else {
-		*hwirq = fwspec->param[0];
-		*type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
-
-		/*
-		 * The ICU receives level interrupts. While the NSR are also
-		 * level interrupts, SEI are edge interrupts. Force the type
-		 * here in this case. Please note that this makes the interrupt
-		 * handling unreliable.
-		 */
-		if (msi_data->subset_data->icu_group == ICU_GRP_SEI)
-			*type = IRQ_TYPE_EDGE_RISING;
-	}
-
-	if (*hwirq >= ICU_MAX_IRQS) {
-		dev_err(icu->dev, "invalid interrupt number %ld\n", *hwirq);
-		return -EINVAL;
-	}
-
-	return 0;
-}
-
-static int
-mvebu_icu_irq_domain_alloc(struct irq_domain *domain, unsigned int virq,
-			   unsigned int nr_irqs, void *args)
-{
-	int err;
-	unsigned long hwirq;
-	struct irq_fwspec *fwspec = args;
-	struct mvebu_icu_msi_data *msi_data = platform_msi_get_host_data(domain);
-	struct mvebu_icu *icu = msi_data->icu;
-	struct mvebu_icu_irq_data *icu_irqd;
-	struct irq_chip *chip = &mvebu_icu_nsr_chip;
-
-	icu_irqd = kmalloc(sizeof(*icu_irqd), GFP_KERNEL);
-	if (!icu_irqd)
-		return -ENOMEM;
-
-	err = mvebu_icu_irq_domain_translate(domain, fwspec, &hwirq,
-					     &icu_irqd->type);
-	if (err) {
-		dev_err(icu->dev, "failed to translate ICU parameters\n");
-		goto free_irqd;
-	}
-
-	if (static_branch_unlikely(&legacy_bindings))
-		icu_irqd->icu_group = fwspec->param[0];
-	else
-		icu_irqd->icu_group = msi_data->subset_data->icu_group;
-	icu_irqd->icu = icu;
-
-	err = platform_msi_device_domain_alloc(domain, virq, nr_irqs);
-	if (err) {
-		dev_err(icu->dev, "failed to allocate ICU interrupt in parent domain\n");
-		goto free_irqd;
-	}
-
-	/* Make sure there is no interrupt left pending by the firmware */
-	err = irq_set_irqchip_state(virq, IRQCHIP_STATE_PENDING, false);
-	if (err)
-		goto free_msi;
-
-	if (icu_irqd->icu_group == ICU_GRP_SEI)
-		chip = &mvebu_icu_sei_chip;
-
-	err = irq_domain_set_hwirq_and_chip(domain, virq, hwirq,
-					    chip, icu_irqd);
-	if (err) {
-		dev_err(icu->dev, "failed to set the data to IRQ domain\n");
-		goto free_msi;
-	}
-
-	return 0;
-
-free_msi:
-	platform_msi_device_domain_free(domain, virq, nr_irqs);
-free_irqd:
-	kfree(icu_irqd);
-	return err;
-}
-
-static void
-mvebu_icu_irq_domain_free(struct irq_domain *domain, unsigned int virq,
-			  unsigned int nr_irqs)
-{
-	struct irq_data *d = irq_get_irq_data(virq);
-	struct mvebu_icu_irq_data *icu_irqd = d->chip_data;
-
-	kfree(icu_irqd);
-
-	platform_msi_device_domain_free(domain, virq, nr_irqs);
-}
-
-static const struct irq_domain_ops mvebu_icu_domain_ops = {
-	.translate = mvebu_icu_irq_domain_translate,
-	.alloc     = mvebu_icu_irq_domain_alloc,
-	.free      = mvebu_icu_irq_domain_free,
-};
-
-/* End of removal area */
-
 static int mvebu_icu_msi_init(struct irq_domain *domain, struct msi_domain_info *info,
 			      unsigned int virq, irq_hw_number_t hwirq, msi_alloc_info_t *arg)
 {
@@ -448,9 +258,10 @@ static const struct of_device_id mvebu_icu_subset_of_match[] = {
 
 static int mvebu_icu_subset_probe(struct platform_device *pdev)
 {
+	const struct msi_domain_template *tmpl;
 	struct mvebu_icu_msi_data *msi_data;
 	struct device *dev = &pdev->dev;
-	struct irq_domain *irq_domain;
+	bool sei;
 
 	msi_data = devm_kzalloc(dev, sizeof(*msi_data), GFP_KERNEL);
 	if (!msi_data)
@@ -471,22 +282,11 @@ static int mvebu_icu_subset_probe(struct platform_device *pdev)
 	if (!irq_domain_get_of_node(dev->msi.domain))
 		return -ENODEV;
 
-	if (irq_domain_is_msi_parent(dev->msi.domain)) {
-		bool sei = msi_data->subset_data->icu_group == ICU_GRP_SEI;
-		const struct msi_domain_template *tmpl;
-
-		tmpl = sei ? &mvebu_icu_sei_msi_template : &mvebu_icu_nsr_msi_template;
-
-		if (!msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN, tmpl,
-						  ICU_MAX_IRQS, NULL, msi_data))
-			return -ENOMEM;
-	}
+	sei = msi_data->subset_data->icu_group == ICU_GRP_SEI;
+	tmpl = sei ? &mvebu_icu_sei_msi_template : &mvebu_icu_nsr_msi_template;
 
-	irq_domain = platform_msi_create_device_tree_domain(dev, ICU_MAX_IRQS,
-							    mvebu_icu_write_msg,
-							    &mvebu_icu_domain_ops,
-							    msi_data);
-	if (!irq_domain) {
+	if (!msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN, tmpl,
+					  ICU_MAX_IRQS, NULL, msi_data)) {
 		dev_err(dev, "Failed to create ICU MSI domain\n");
 		return -ENOMEM;
 	}
#72 lore raw

PATCH v4 20/21 genirq/msi: Remove platform MSI leftovers

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

No more users!

---
 drivers/base/platform-msi.c | 350 +-----------------------------------
 include/linux/msi.h         |  30 ----
 kernel/irq/msi.c            |  75 +-------
 3 files changed, 5 insertions(+), 450 deletions(-)
drivers/base/platform-msi.c +3 -347
index 11f5fdf65b9e..0e60dd650b5e 100644
@@ -4,346 +4,12 @@
  *
  * Copyright (C) 2015 ARM Limited, All Rights Reserved.
  * Author: Marc Zyngier <[email protected]>
+ * Copyright (C) 2022 Linutronix GmbH
  */
 
 #include <linux/device.h>
-#include <linux/idr.h>
-#include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
-#include <linux/slab.h>
-
-/* Begin of removal area. Once everything is converted over. Cleanup the includes too! */
-
-#define DEV_ID_SHIFT	21
-#define MAX_DEV_MSIS	(1 << (32 - DEV_ID_SHIFT))
-
-/*
- * Internal data structure containing a (made up, but unique) devid
- * and the callback to write the MSI message.
- */
-struct platform_msi_priv_data {
-	struct device			*dev;
-	void				*host_data;
-	msi_alloc_info_t		arg;
-	irq_write_msi_msg_t		write_msg;
-	int				devid;
-};
-
-/* The devid allocator */
-static DEFINE_IDA(platform_msi_devid_ida);
-
-#ifdef GENERIC_MSI_DOMAIN_OPS
-/*
- * Convert an msi_desc to a globaly unique identifier (per-device
- * devid + msi_desc position in the msi_list).
- */
-static irq_hw_number_t platform_msi_calc_hwirq(struct msi_desc *desc)
-{
-	u32 devid = desc->dev->msi.data->platform_data->devid;
-
-	return (devid << (32 - DEV_ID_SHIFT)) | desc->msi_index;
-}
-
-static void platform_msi_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
-{
-	arg->desc = desc;
-	arg->hwirq = platform_msi_calc_hwirq(desc);
-}
-
-static int platform_msi_init(struct irq_domain *domain,
-			     struct msi_domain_info *info,
-			     unsigned int virq, irq_hw_number_t hwirq,
-			     msi_alloc_info_t *arg)
-{
-	return irq_domain_set_hwirq_and_chip(domain, virq, hwirq,
-					     info->chip, info->chip_data);
-}
-
-static void platform_msi_set_proxy_dev(msi_alloc_info_t *arg)
-{
-	arg->flags |= MSI_ALLOC_FLAGS_PROXY_DEVICE;
-}
-#else
-#define platform_msi_set_desc		NULL
-#define platform_msi_init		NULL
-#define platform_msi_set_proxy_dev(x)	do {} while(0)
-#endif
-
-static void platform_msi_update_dom_ops(struct msi_domain_info *info)
-{
-	struct msi_domain_ops *ops = info->ops;
-
-	BUG_ON(!ops);
-
-	if (ops->msi_init == NULL)
-		ops->msi_init = platform_msi_init;
-	if (ops->set_desc == NULL)
-		ops->set_desc = platform_msi_set_desc;
-}
-
-static void platform_msi_write_msg(struct irq_data *data, struct msi_msg *msg)
-{
-	struct msi_desc *desc = irq_data_get_msi_desc(data);
-
-	desc->dev->msi.data->platform_data->write_msg(desc, msg);
-}
-
-static void platform_msi_update_chip_ops(struct msi_domain_info *info)
-{
-	struct irq_chip *chip = info->chip;
-
-	BUG_ON(!chip);
-	if (!chip->irq_mask)
-		chip->irq_mask = irq_chip_mask_parent;
-	if (!chip->irq_unmask)
-		chip->irq_unmask = irq_chip_unmask_parent;
-	if (!chip->irq_eoi)
-		chip->irq_eoi = irq_chip_eoi_parent;
-	if (!chip->irq_set_affinity)
-		chip->irq_set_affinity = msi_domain_set_affinity;
-	if (!chip->irq_write_msi_msg)
-		chip->irq_write_msi_msg = platform_msi_write_msg;
-	if (WARN_ON((info->flags & MSI_FLAG_LEVEL_CAPABLE) &&
-		    !(chip->flags & IRQCHIP_SUPPORTS_LEVEL_MSI)))
-		info->flags &= ~MSI_FLAG_LEVEL_CAPABLE;
-}
-
-/**
- * platform_msi_create_irq_domain - Create a platform MSI interrupt domain
- * @fwnode:		Optional fwnode of the interrupt controller
- * @info:	MSI domain info
- * @parent:	Parent irq domain
- *
- * Updates the domain and chip ops and creates a platform MSI
- * interrupt domain.
- *
- * Returns:
- * A domain pointer or NULL in case of failure.
- */
-struct irq_domain *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
-						  struct msi_domain_info *info,
-						  struct irq_domain *parent)
-{
-	struct irq_domain *domain;
-
-	if (info->flags & MSI_FLAG_USE_DEF_DOM_OPS)
-		platform_msi_update_dom_ops(info);
-	if (info->flags & MSI_FLAG_USE_DEF_CHIP_OPS)
-		platform_msi_update_chip_ops(info);
-	info->flags |= MSI_FLAG_DEV_SYSFS | MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS |
-		       MSI_FLAG_FREE_MSI_DESCS;
-
-	domain = msi_create_irq_domain(fwnode, info, parent);
-	if (domain)
-		irq_domain_update_bus_token(domain, DOMAIN_BUS_PLATFORM_MSI);
-
-	return domain;
-}
-EXPORT_SYMBOL_GPL(platform_msi_create_irq_domain);
-
-static int platform_msi_alloc_priv_data(struct device *dev, unsigned int nvec,
-					irq_write_msi_msg_t write_msi_msg)
-{
-	struct platform_msi_priv_data *datap;
-	int err;
-
-	/*
-	 * Limit the number of interrupts to 2048 per device. Should we
-	 * need to bump this up, DEV_ID_SHIFT should be adjusted
-	 * accordingly (which would impact the max number of MSI
-	 * capable devices).
-	 */
-	if (!dev->msi.domain || !write_msi_msg || !nvec || nvec > MAX_DEV_MSIS)
-		return -EINVAL;
-
-	if (dev->msi.domain->bus_token != DOMAIN_BUS_PLATFORM_MSI) {
-		dev_err(dev, "Incompatible msi_domain, giving up\n");
-		return -EINVAL;
-	}
-
-	err = msi_setup_device_data(dev);
-	if (err)
-		return err;
-
-	/* Already initialized? */
-	if (dev->msi.data->platform_data)
-		return -EBUSY;
-
-	datap = kzalloc(sizeof(*datap), GFP_KERNEL);
-	if (!datap)
-		return -ENOMEM;
-
-	datap->devid = ida_alloc_max(&platform_msi_devid_ida,
-				     (1 << DEV_ID_SHIFT) - 1, GFP_KERNEL);
-	if (datap->devid < 0) {
-		err = datap->devid;
-		kfree(datap);
-		return err;
-	}
-
-	datap->write_msg = write_msi_msg;
-	datap->dev = dev;
-	dev->msi.data->platform_data = datap;
-	return 0;
-}
-
-static void platform_msi_free_priv_data(struct device *dev)
-{
-	struct platform_msi_priv_data *data = dev->msi.data->platform_data;
-
-	dev->msi.data->platform_data = NULL;
-	ida_free(&platform_msi_devid_ida, data->devid);
-	kfree(data);
-}
-
-/**
- * platform_msi_domain_alloc_irqs - Allocate MSI interrupts for @dev
- * @dev:		The device for which to allocate interrupts
- * @nvec:		The number of interrupts to allocate
- * @write_msi_msg:	Callback to write an interrupt message for @dev
- *
- * Returns:
- * Zero for success, or an error code in case of failure
- */
-static int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
-					  irq_write_msi_msg_t write_msi_msg)
-{
-	int err;
-
-	err = platform_msi_alloc_priv_data(dev, nvec, write_msi_msg);
-	if (err)
-		return err;
-
-	err = msi_domain_alloc_irqs_range(dev, MSI_DEFAULT_DOMAIN, 0, nvec - 1);
-	if (err)
-		platform_msi_free_priv_data(dev);
-
-	return err;
-}
-
-/**
- * platform_msi_get_host_data - Query the private data associated with
- *                              a platform-msi domain
- * @domain:	The platform-msi domain
- *
- * Return: The private data provided when calling
- * platform_msi_create_device_domain().
- */
-void *platform_msi_get_host_data(struct irq_domain *domain)
-{
-	struct platform_msi_priv_data *data = domain->host_data;
-
-	return data->host_data;
-}
-
-static struct lock_class_key platform_device_msi_lock_class;
-
-/**
- * __platform_msi_create_device_domain - Create a platform-msi device domain
- *
- * @dev:		The device generating the MSIs
- * @nvec:		The number of MSIs that need to be allocated
- * @is_tree:		flag to indicate tree hierarchy
- * @write_msi_msg:	Callback to write an interrupt message for @dev
- * @ops:		The hierarchy domain operations to use
- * @host_data:		Private data associated to this domain
- *
- * Return: An irqdomain for @nvec interrupts on success, NULL in case of error.
- *
- * This is for interrupt domains which stack on a platform-msi domain
- * created by platform_msi_create_irq_domain(). @dev->msi.domain points to
- * that platform-msi domain which is the parent for the new domain.
- */
-struct irq_domain *
-__platform_msi_create_device_domain(struct device *dev,
-				    unsigned int nvec,
-				    bool is_tree,
-				    irq_write_msi_msg_t write_msi_msg,
-				    const struct irq_domain_ops *ops,
-				    void *host_data)
-{
-	struct platform_msi_priv_data *data;
-	struct irq_domain *domain;
-	int err;
-
-	err = platform_msi_alloc_priv_data(dev, nvec, write_msi_msg);
-	if (err)
-		return NULL;
-
-	/*
-	 * Use a separate lock class for the MSI descriptor mutex on
-	 * platform MSI device domains because the descriptor mutex nests
-	 * into the domain mutex. See alloc/free below.
-	 */
-	lockdep_set_class(&dev->msi.data->mutex, &platform_device_msi_lock_class);
-
-	data = dev->msi.data->platform_data;
-	data->host_data = host_data;
-	domain = irq_domain_create_hierarchy(dev->msi.domain, 0,
-					     is_tree ? 0 : nvec,
-					     dev->fwnode, ops, data);
-	if (!domain)
-		goto free_priv;
-
-	platform_msi_set_proxy_dev(&data->arg);
-	err = msi_domain_prepare_irqs(domain->parent, dev, nvec, &data->arg);
-	if (err)
-		goto free_domain;
-
-	return domain;
-
-free_domain:
-	irq_domain_remove(domain);
-free_priv:
-	platform_msi_free_priv_data(dev);
-	return NULL;
-}
-
-/**
- * platform_msi_device_domain_free - Free interrupts associated with a platform-msi
- *				     device domain
- *
- * @domain:	The platform-msi device domain
- * @virq:	The base irq from which to perform the free operation
- * @nr_irqs:	How many interrupts to free from @virq
- */
-void platform_msi_device_domain_free(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nr_irqs)
-{
-	struct platform_msi_priv_data *data = domain->host_data;
-
-	msi_lock_descs(data->dev);
-	msi_domain_depopulate_descs(data->dev, virq, nr_irqs);
-	irq_domain_free_irqs_common(domain, virq, nr_irqs);
-	msi_free_msi_descs_range(data->dev, virq, virq + nr_irqs - 1);
-	msi_unlock_descs(data->dev);
-}
-
-/**
- * platform_msi_device_domain_alloc - Allocate interrupts associated with
- *				      a platform-msi device domain
- *
- * @domain:	The platform-msi device domain
- * @virq:	The base irq from which to perform the allocate operation
- * @nr_irqs:	How many interrupts to allocate from @virq
- *
- * Return 0 on success, or an error code on failure. Must be called
- * with irq_domain_mutex held (which can only be done as part of a
- * top-level interrupt allocation).
- */
-int platform_msi_device_domain_alloc(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nr_irqs)
-{
-	struct platform_msi_priv_data *data = domain->host_data;
-	struct device *dev = data->dev;
-
-	return msi_domain_populate_irqs(domain->parent, dev, virq, nr_irqs, &data->arg);
-}
-
-/* End of removal area */
-
-/* Real per device domain interfaces */
 
 /*
  * This indirection can go when platform_device_msi_init_and_alloc_irqs()
@@ -357,7 +23,7 @@ static void platform_msi_write_msi_msg(struct irq_data *d, struct msi_msg *msg)
 	cb(irq_data_get_msi_desc(d), msg);
 }
 
-static void platform_msi_set_desc_byindex(msi_alloc_info_t *arg, struct msi_desc *desc)
+static void platform_msi_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
 {
 	arg->desc = desc;
 	arg->hwirq = desc->msi_index;
@@ -373,7 +39,7 @@ static const struct msi_domain_template platform_msi_template = {
 	},
 
 	.ops = {
-		.set_desc		= platform_msi_set_desc_byindex,
+		.set_desc		= platform_msi_set_desc,
 	},
 
 	.info = {
@@ -408,10 +74,6 @@ int platform_device_msi_init_and_alloc_irqs(struct device *dev, unsigned int nve
 	if (!domain || !write_msi_msg)
 		return -EINVAL;
 
-	/* Migration support. Will go away once everything is converted */
-	if (!irq_domain_is_msi_parent(domain))
-		return platform_msi_domain_alloc_irqs(dev, nvec, write_msi_msg);
-
 	/*
 	 * @write_msi_msg is stored in the resulting msi_domain_info::data.
 	 * The underlying domain creation mechanism will assign that
@@ -432,12 +94,6 @@ EXPORT_SYMBOL_GPL(platform_device_msi_init_and_alloc_irqs);
  */
 void platform_device_msi_free_irqs_all(struct device *dev)
 {
-	struct irq_domain *domain = dev->msi.domain;
-
 	msi_domain_free_irqs_all(dev, MSI_DEFAULT_DOMAIN);
-
-	/* Migration support. Will go away once everything is converted */
-	if (!irq_domain_is_msi_parent(domain))
-		platform_msi_free_priv_data(dev);
 }
 EXPORT_SYMBOL_GPL(platform_device_msi_free_irqs_all);
include/linux/msi.h +0 -30
index 4ae036d0c7db..4c3462a6a97b 100644
@@ -81,7 +81,6 @@ extern int pci_msi_ignore_mask;
 /* Helper functions */
 struct msi_desc;
 struct pci_dev;
-struct platform_msi_priv_data;
 struct device_attribute;
 struct irq_domain;
 struct irq_affinity_desc;
@@ -231,14 +230,12 @@ struct msi_dev_domain {
 /**
  * msi_device_data - MSI per device data
  * @properties:		MSI properties which are interesting to drivers
- * @platform_data:	Platform-MSI specific data
  * @mutex:		Mutex protecting the MSI descriptor store
  * @__domains:		Internal data for per device MSI domains
  * @__iter_idx:		Index to search the next entry for iterators
  */
 struct msi_device_data {
 	unsigned long			properties;
-	struct platform_msi_priv_data	*platform_data;
 	struct mutex			mutex;
 	struct msi_dev_domain		__domains[MSI_MAX_DEVICE_IRQDOMAINS];
 	unsigned long			__iter_idx;
@@ -641,33 +638,6 @@ void msi_domain_free_irqs_all(struct device *dev, unsigned int domid);
 
 struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain);
 
-struct irq_domain *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
-						  struct msi_domain_info *info,
-						  struct irq_domain *parent);
-
-/* When an MSI domain is used as an intermediate domain */
-int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev,
-			    int nvec, msi_alloc_info_t *args);
-int msi_domain_populate_irqs(struct irq_domain *domain, struct device *dev,
-			     int virq, int nvec, msi_alloc_info_t *args);
-void msi_domain_depopulate_descs(struct device *dev, int virq, int nvec);
-
-struct irq_domain *
-__platform_msi_create_device_domain(struct device *dev,
-				    unsigned int nvec,
-				    bool is_tree,
-				    irq_write_msi_msg_t write_msi_msg,
-				    const struct irq_domain_ops *ops,
-				    void *host_data);
-
-#define platform_msi_create_device_tree_domain(dev, nvec, write, ops, data) \
-	__platform_msi_create_device_domain(dev, nvec, true, write, ops, data)
-
-int platform_msi_device_domain_alloc(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nr_irqs);
-void platform_msi_device_domain_free(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nvec);
-void *platform_msi_get_host_data(struct irq_domain *domain);
 /* Per device platform MSI */
 int platform_device_msi_init_and_alloc_irqs(struct device *dev, unsigned int nvec,
 					    irq_write_msi_msg_t write_msi_msg);
kernel/irq/msi.c +2 -73

87 lines not on this page · raw message

Signature · 1 line
2.34.1
#74 lore raw

[tip: irq/core] genirq/msi: Remove platform MSI leftovers

In reply to Thomas Gleixner
The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     57ca43b37c414086fb0f331eda0cdf830c3159ff
Gitweb:        https://git.kernel.org/tip/57ca43b37c414086fb0f331eda0cdf830c3159ff
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:19:05 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:25 +02:00

genirq/msi: Remove platform MSI leftovers

No more users!

---
 drivers/base/platform-msi.c | 350 +-----------------------------------
 include/linux/msi.h         |  30 +---
 kernel/irq/msi.c            |  75 +--------
 3 files changed, 5 insertions(+), 450 deletions(-)
drivers/base/platform-msi.c +3 -347
index 11f5fdf..0e60dd6 100644
@@ -4,346 +4,12 @@
  *
  * Copyright (C) 2015 ARM Limited, All Rights Reserved.
  * Author: Marc Zyngier <[email protected]>
+ * Copyright (C) 2022 Linutronix GmbH
  */
 
 #include <linux/device.h>
-#include <linux/idr.h>
-#include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
-#include <linux/slab.h>
-
-/* Begin of removal area. Once everything is converted over. Cleanup the includes too! */
-
-#define DEV_ID_SHIFT	21
-#define MAX_DEV_MSIS	(1 << (32 - DEV_ID_SHIFT))
-
-/*
- * Internal data structure containing a (made up, but unique) devid
- * and the callback to write the MSI message.
- */
-struct platform_msi_priv_data {
-	struct device			*dev;
-	void				*host_data;
-	msi_alloc_info_t		arg;
-	irq_write_msi_msg_t		write_msg;
-	int				devid;
-};
-
-/* The devid allocator */
-static DEFINE_IDA(platform_msi_devid_ida);
-
-#ifdef GENERIC_MSI_DOMAIN_OPS
-/*
- * Convert an msi_desc to a globaly unique identifier (per-device
- * devid + msi_desc position in the msi_list).
- */
-static irq_hw_number_t platform_msi_calc_hwirq(struct msi_desc *desc)
-{
-	u32 devid = desc->dev->msi.data->platform_data->devid;
-
-	return (devid << (32 - DEV_ID_SHIFT)) | desc->msi_index;
-}
-
-static void platform_msi_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
-{
-	arg->desc = desc;
-	arg->hwirq = platform_msi_calc_hwirq(desc);
-}
-
-static int platform_msi_init(struct irq_domain *domain,
-			     struct msi_domain_info *info,
-			     unsigned int virq, irq_hw_number_t hwirq,
-			     msi_alloc_info_t *arg)
-{
-	return irq_domain_set_hwirq_and_chip(domain, virq, hwirq,
-					     info->chip, info->chip_data);
-}
-
-static void platform_msi_set_proxy_dev(msi_alloc_info_t *arg)
-{
-	arg->flags |= MSI_ALLOC_FLAGS_PROXY_DEVICE;
-}
-#else
-#define platform_msi_set_desc		NULL
-#define platform_msi_init		NULL
-#define platform_msi_set_proxy_dev(x)	do {} while(0)
-#endif
-
-static void platform_msi_update_dom_ops(struct msi_domain_info *info)
-{
-	struct msi_domain_ops *ops = info->ops;
-
-	BUG_ON(!ops);
-
-	if (ops->msi_init == NULL)
-		ops->msi_init = platform_msi_init;
-	if (ops->set_desc == NULL)
-		ops->set_desc = platform_msi_set_desc;
-}
-
-static void platform_msi_write_msg(struct irq_data *data, struct msi_msg *msg)
-{
-	struct msi_desc *desc = irq_data_get_msi_desc(data);
-
-	desc->dev->msi.data->platform_data->write_msg(desc, msg);
-}
-
-static void platform_msi_update_chip_ops(struct msi_domain_info *info)
-{
-	struct irq_chip *chip = info->chip;
-
-	BUG_ON(!chip);
-	if (!chip->irq_mask)
-		chip->irq_mask = irq_chip_mask_parent;
-	if (!chip->irq_unmask)
-		chip->irq_unmask = irq_chip_unmask_parent;
-	if (!chip->irq_eoi)
-		chip->irq_eoi = irq_chip_eoi_parent;
-	if (!chip->irq_set_affinity)
-		chip->irq_set_affinity = msi_domain_set_affinity;
-	if (!chip->irq_write_msi_msg)
-		chip->irq_write_msi_msg = platform_msi_write_msg;
-	if (WARN_ON((info->flags & MSI_FLAG_LEVEL_CAPABLE) &&
-		    !(chip->flags & IRQCHIP_SUPPORTS_LEVEL_MSI)))
-		info->flags &= ~MSI_FLAG_LEVEL_CAPABLE;
-}
-
-/**
- * platform_msi_create_irq_domain - Create a platform MSI interrupt domain
- * @fwnode:		Optional fwnode of the interrupt controller
- * @info:	MSI domain info
- * @parent:	Parent irq domain
- *
- * Updates the domain and chip ops and creates a platform MSI
- * interrupt domain.
- *
- * Returns:
- * A domain pointer or NULL in case of failure.
- */
-struct irq_domain *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
-						  struct msi_domain_info *info,
-						  struct irq_domain *parent)
-{
-	struct irq_domain *domain;
-
-	if (info->flags & MSI_FLAG_USE_DEF_DOM_OPS)
-		platform_msi_update_dom_ops(info);
-	if (info->flags & MSI_FLAG_USE_DEF_CHIP_OPS)
-		platform_msi_update_chip_ops(info);
-	info->flags |= MSI_FLAG_DEV_SYSFS | MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS |
-		       MSI_FLAG_FREE_MSI_DESCS;
-
-	domain = msi_create_irq_domain(fwnode, info, parent);
-	if (domain)
-		irq_domain_update_bus_token(domain, DOMAIN_BUS_PLATFORM_MSI);
-
-	return domain;
-}
-EXPORT_SYMBOL_GPL(platform_msi_create_irq_domain);
-
-static int platform_msi_alloc_priv_data(struct device *dev, unsigned int nvec,
-					irq_write_msi_msg_t write_msi_msg)
-{
-	struct platform_msi_priv_data *datap;
-	int err;
-
-	/*
-	 * Limit the number of interrupts to 2048 per device. Should we
-	 * need to bump this up, DEV_ID_SHIFT should be adjusted
-	 * accordingly (which would impact the max number of MSI
-	 * capable devices).
-	 */
-	if (!dev->msi.domain || !write_msi_msg || !nvec || nvec > MAX_DEV_MSIS)
-		return -EINVAL;
-
-	if (dev->msi.domain->bus_token != DOMAIN_BUS_PLATFORM_MSI) {
-		dev_err(dev, "Incompatible msi_domain, giving up\n");
-		return -EINVAL;
-	}
-
-	err = msi_setup_device_data(dev);
-	if (err)
-		return err;
-
-	/* Already initialized? */
-	if (dev->msi.data->platform_data)
-		return -EBUSY;
-
-	datap = kzalloc(sizeof(*datap), GFP_KERNEL);
-	if (!datap)
-		return -ENOMEM;
-
-	datap->devid = ida_alloc_max(&platform_msi_devid_ida,
-				     (1 << DEV_ID_SHIFT) - 1, GFP_KERNEL);
-	if (datap->devid < 0) {
-		err = datap->devid;
-		kfree(datap);
-		return err;
-	}
-
-	datap->write_msg = write_msi_msg;
-	datap->dev = dev;
-	dev->msi.data->platform_data = datap;
-	return 0;
-}
-
-static void platform_msi_free_priv_data(struct device *dev)
-{
-	struct platform_msi_priv_data *data = dev->msi.data->platform_data;
-
-	dev->msi.data->platform_data = NULL;
-	ida_free(&platform_msi_devid_ida, data->devid);
-	kfree(data);
-}
-
-/**
- * platform_msi_domain_alloc_irqs - Allocate MSI interrupts for @dev
- * @dev:		The device for which to allocate interrupts
- * @nvec:		The number of interrupts to allocate
- * @write_msi_msg:	Callback to write an interrupt message for @dev
- *
- * Returns:
- * Zero for success, or an error code in case of failure
- */
-static int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
-					  irq_write_msi_msg_t write_msi_msg)
-{
-	int err;
-
-	err = platform_msi_alloc_priv_data(dev, nvec, write_msi_msg);
-	if (err)
-		return err;
-
-	err = msi_domain_alloc_irqs_range(dev, MSI_DEFAULT_DOMAIN, 0, nvec - 1);
-	if (err)
-		platform_msi_free_priv_data(dev);
-
-	return err;
-}
-
-/**
- * platform_msi_get_host_data - Query the private data associated with
- *                              a platform-msi domain
- * @domain:	The platform-msi domain
- *
- * Return: The private data provided when calling
- * platform_msi_create_device_domain().
- */
-void *platform_msi_get_host_data(struct irq_domain *domain)
-{
-	struct platform_msi_priv_data *data = domain->host_data;
-
-	return data->host_data;
-}
-
-static struct lock_class_key platform_device_msi_lock_class;
-
-/**
- * __platform_msi_create_device_domain - Create a platform-msi device domain
- *
- * @dev:		The device generating the MSIs
- * @nvec:		The number of MSIs that need to be allocated
- * @is_tree:		flag to indicate tree hierarchy
- * @write_msi_msg:	Callback to write an interrupt message for @dev
- * @ops:		The hierarchy domain operations to use
- * @host_data:		Private data associated to this domain
- *
- * Return: An irqdomain for @nvec interrupts on success, NULL in case of error.
- *
- * This is for interrupt domains which stack on a platform-msi domain
- * created by platform_msi_create_irq_domain(). @dev->msi.domain points to
- * that platform-msi domain which is the parent for the new domain.
- */
-struct irq_domain *
-__platform_msi_create_device_domain(struct device *dev,
-				    unsigned int nvec,
-				    bool is_tree,
-				    irq_write_msi_msg_t write_msi_msg,
-				    const struct irq_domain_ops *ops,
-				    void *host_data)
-{
-	struct platform_msi_priv_data *data;
-	struct irq_domain *domain;
-	int err;
-
-	err = platform_msi_alloc_priv_data(dev, nvec, write_msi_msg);
-	if (err)
-		return NULL;
-
-	/*
-	 * Use a separate lock class for the MSI descriptor mutex on
-	 * platform MSI device domains because the descriptor mutex nests
-	 * into the domain mutex. See alloc/free below.
-	 */
-	lockdep_set_class(&dev->msi.data->mutex, &platform_device_msi_lock_class);
-
-	data = dev->msi.data->platform_data;
-	data->host_data = host_data;
-	domain = irq_domain_create_hierarchy(dev->msi.domain, 0,
-					     is_tree ? 0 : nvec,
-					     dev->fwnode, ops, data);
-	if (!domain)
-		goto free_priv;
-
-	platform_msi_set_proxy_dev(&data->arg);
-	err = msi_domain_prepare_irqs(domain->parent, dev, nvec, &data->arg);
-	if (err)
-		goto free_domain;
-
-	return domain;
-
-free_domain:
-	irq_domain_remove(domain);
-free_priv:
-	platform_msi_free_priv_data(dev);
-	return NULL;
-}
-
-/**
- * platform_msi_device_domain_free - Free interrupts associated with a platform-msi
- *				     device domain
- *
- * @domain:	The platform-msi device domain
- * @virq:	The base irq from which to perform the free operation
- * @nr_irqs:	How many interrupts to free from @virq
- */
-void platform_msi_device_domain_free(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nr_irqs)
-{
-	struct platform_msi_priv_data *data = domain->host_data;
-
-	msi_lock_descs(data->dev);
-	msi_domain_depopulate_descs(data->dev, virq, nr_irqs);
-	irq_domain_free_irqs_common(domain, virq, nr_irqs);
-	msi_free_msi_descs_range(data->dev, virq, virq + nr_irqs - 1);
-	msi_unlock_descs(data->dev);
-}
-
-/**
- * platform_msi_device_domain_alloc - Allocate interrupts associated with
- *				      a platform-msi device domain
- *
- * @domain:	The platform-msi device domain
- * @virq:	The base irq from which to perform the allocate operation
- * @nr_irqs:	How many interrupts to allocate from @virq
- *
- * Return 0 on success, or an error code on failure. Must be called
- * with irq_domain_mutex held (which can only be done as part of a
- * top-level interrupt allocation).
- */
-int platform_msi_device_domain_alloc(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nr_irqs)
-{
-	struct platform_msi_priv_data *data = domain->host_data;
-	struct device *dev = data->dev;
-
-	return msi_domain_populate_irqs(domain->parent, dev, virq, nr_irqs, &data->arg);
-}
-
-/* End of removal area */
-
-/* Real per device domain interfaces */
 
 /*
  * This indirection can go when platform_device_msi_init_and_alloc_irqs()
@@ -357,7 +23,7 @@ static void platform_msi_write_msi_msg(struct irq_data *d, struct msi_msg *msg)
 	cb(irq_data_get_msi_desc(d), msg);
 }
 
-static void platform_msi_set_desc_byindex(msi_alloc_info_t *arg, struct msi_desc *desc)
+static void platform_msi_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
 {
 	arg->desc = desc;
 	arg->hwirq = desc->msi_index;
@@ -373,7 +39,7 @@ static const struct msi_domain_template platform_msi_template = {
 	},
 
 	.ops = {
-		.set_desc		= platform_msi_set_desc_byindex,
+		.set_desc		= platform_msi_set_desc,
 	},
 
 	.info = {
@@ -408,10 +74,6 @@ int platform_device_msi_init_and_alloc_irqs(struct device *dev, unsigned int nve
 	if (!domain || !write_msi_msg)
 		return -EINVAL;
 
-	/* Migration support. Will go away once everything is converted */
-	if (!irq_domain_is_msi_parent(domain))
-		return platform_msi_domain_alloc_irqs(dev, nvec, write_msi_msg);
-
 	/*
 	 * @write_msi_msg is stored in the resulting msi_domain_info::data.
 	 * The underlying domain creation mechanism will assign that
@@ -432,12 +94,6 @@ EXPORT_SYMBOL_GPL(platform_device_msi_init_and_alloc_irqs);
  */
 void platform_device_msi_free_irqs_all(struct device *dev)
 {
-	struct irq_domain *domain = dev->msi.domain;
-
 	msi_domain_free_irqs_all(dev, MSI_DEFAULT_DOMAIN);
-
-	/* Migration support. Will go away once everything is converted */
-	if (!irq_domain_is_msi_parent(domain))
-		platform_msi_free_priv_data(dev);
 }
 EXPORT_SYMBOL_GPL(platform_device_msi_free_irqs_all);
include/linux/msi.h +0 -30
index 4ae036d..4c3462a 100644
@@ -81,7 +81,6 @@ extern int pci_msi_ignore_mask;
 /* Helper functions */
 struct msi_desc;
 struct pci_dev;
-struct platform_msi_priv_data;
 struct device_attribute;
 struct irq_domain;
 struct irq_affinity_desc;
@@ -231,14 +230,12 @@ struct msi_dev_domain {
 /**
  * msi_device_data - MSI per device data
  * @properties:		MSI properties which are interesting to drivers
- * @platform_data:	Platform-MSI specific data
  * @mutex:		Mutex protecting the MSI descriptor store
  * @__domains:		Internal data for per device MSI domains
  * @__iter_idx:		Index to search the next entry for iterators
  */
 struct msi_device_data {
 	unsigned long			properties;
-	struct platform_msi_priv_data	*platform_data;
 	struct mutex			mutex;
 	struct msi_dev_domain		__domains[MSI_MAX_DEVICE_IRQDOMAINS];
 	unsigned long			__iter_idx;
@@ -641,33 +638,6 @@ void msi_domain_free_irqs_all(struct device *dev, unsigned int domid);
 
 struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain);
 
-struct irq_domain *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
-						  struct msi_domain_info *info,
-						  struct irq_domain *parent);
-
-/* When an MSI domain is used as an intermediate domain */
-int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev,
-			    int nvec, msi_alloc_info_t *args);
-int msi_domain_populate_irqs(struct irq_domain *domain, struct device *dev,
-			     int virq, int nvec, msi_alloc_info_t *args);
-void msi_domain_depopulate_descs(struct device *dev, int virq, int nvec);
-
-struct irq_domain *
-__platform_msi_create_device_domain(struct device *dev,
-				    unsigned int nvec,
-				    bool is_tree,
-				    irq_write_msi_msg_t write_msi_msg,
-				    const struct irq_domain_ops *ops,
-				    void *host_data);
-
-#define platform_msi_create_device_tree_domain(dev, nvec, write, ops, data) \
-	__platform_msi_create_device_domain(dev, nvec, true, write, ops, data)
-
-int platform_msi_device_domain_alloc(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nr_irqs);
-void platform_msi_device_domain_free(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nvec);
-void *platform_msi_get_host_data(struct irq_domain *domain);
 /* Per device platform MSI */
 int platform_device_msi_init_and_alloc_irqs(struct device *dev, unsigned int nvec,
 					    irq_write_msi_msg_t write_msi_msg);
kernel/irq/msi.c +2 -73

87 lines not on this page · raw message

#75 lore raw

[tip: irq/msi] genirq/msi: Remove platform MSI leftovers

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     e9894248994ca8291838baf063f045eab28e5a0e
Gitweb:        https://git.kernel.org/tip/e9894248994ca8291838baf063f045eab28e5a0e
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:19:05 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:21 +02:00

genirq/msi: Remove platform MSI leftovers

No more users!

---
 drivers/base/platform-msi.c | 350 +-----------------------------------
 include/linux/msi.h         |  30 +---
 kernel/irq/msi.c            |  75 +--------
 3 files changed, 5 insertions(+), 450 deletions(-)
drivers/base/platform-msi.c +3 -347
index 11f5fdf..0e60dd6 100644
@@ -4,346 +4,12 @@
  *
  * Copyright (C) 2015 ARM Limited, All Rights Reserved.
  * Author: Marc Zyngier <[email protected]>
+ * Copyright (C) 2022 Linutronix GmbH
  */
 
 #include <linux/device.h>
-#include <linux/idr.h>
-#include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
-#include <linux/slab.h>
-
-/* Begin of removal area. Once everything is converted over. Cleanup the includes too! */
-
-#define DEV_ID_SHIFT	21
-#define MAX_DEV_MSIS	(1 << (32 - DEV_ID_SHIFT))
-
-/*
- * Internal data structure containing a (made up, but unique) devid
- * and the callback to write the MSI message.
- */
-struct platform_msi_priv_data {
-	struct device			*dev;
-	void				*host_data;
-	msi_alloc_info_t		arg;
-	irq_write_msi_msg_t		write_msg;
-	int				devid;
-};
-
-/* The devid allocator */
-static DEFINE_IDA(platform_msi_devid_ida);
-
-#ifdef GENERIC_MSI_DOMAIN_OPS
-/*
- * Convert an msi_desc to a globaly unique identifier (per-device
- * devid + msi_desc position in the msi_list).
- */
-static irq_hw_number_t platform_msi_calc_hwirq(struct msi_desc *desc)
-{
-	u32 devid = desc->dev->msi.data->platform_data->devid;
-
-	return (devid << (32 - DEV_ID_SHIFT)) | desc->msi_index;
-}
-
-static void platform_msi_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
-{
-	arg->desc = desc;
-	arg->hwirq = platform_msi_calc_hwirq(desc);
-}
-
-static int platform_msi_init(struct irq_domain *domain,
-			     struct msi_domain_info *info,
-			     unsigned int virq, irq_hw_number_t hwirq,
-			     msi_alloc_info_t *arg)
-{
-	return irq_domain_set_hwirq_and_chip(domain, virq, hwirq,
-					     info->chip, info->chip_data);
-}
-
-static void platform_msi_set_proxy_dev(msi_alloc_info_t *arg)
-{
-	arg->flags |= MSI_ALLOC_FLAGS_PROXY_DEVICE;
-}
-#else
-#define platform_msi_set_desc		NULL
-#define platform_msi_init		NULL
-#define platform_msi_set_proxy_dev(x)	do {} while(0)
-#endif
-
-static void platform_msi_update_dom_ops(struct msi_domain_info *info)
-{
-	struct msi_domain_ops *ops = info->ops;
-
-	BUG_ON(!ops);
-
-	if (ops->msi_init == NULL)
-		ops->msi_init = platform_msi_init;
-	if (ops->set_desc == NULL)
-		ops->set_desc = platform_msi_set_desc;
-}
-
-static void platform_msi_write_msg(struct irq_data *data, struct msi_msg *msg)
-{
-	struct msi_desc *desc = irq_data_get_msi_desc(data);
-
-	desc->dev->msi.data->platform_data->write_msg(desc, msg);
-}
-
-static void platform_msi_update_chip_ops(struct msi_domain_info *info)
-{
-	struct irq_chip *chip = info->chip;
-
-	BUG_ON(!chip);
-	if (!chip->irq_mask)
-		chip->irq_mask = irq_chip_mask_parent;
-	if (!chip->irq_unmask)
-		chip->irq_unmask = irq_chip_unmask_parent;
-	if (!chip->irq_eoi)
-		chip->irq_eoi = irq_chip_eoi_parent;
-	if (!chip->irq_set_affinity)
-		chip->irq_set_affinity = msi_domain_set_affinity;
-	if (!chip->irq_write_msi_msg)
-		chip->irq_write_msi_msg = platform_msi_write_msg;
-	if (WARN_ON((info->flags & MSI_FLAG_LEVEL_CAPABLE) &&
-		    !(chip->flags & IRQCHIP_SUPPORTS_LEVEL_MSI)))
-		info->flags &= ~MSI_FLAG_LEVEL_CAPABLE;
-}
-
-/**
- * platform_msi_create_irq_domain - Create a platform MSI interrupt domain
- * @fwnode:		Optional fwnode of the interrupt controller
- * @info:	MSI domain info
- * @parent:	Parent irq domain
- *
- * Updates the domain and chip ops and creates a platform MSI
- * interrupt domain.
- *
- * Returns:
- * A domain pointer or NULL in case of failure.
- */
-struct irq_domain *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
-						  struct msi_domain_info *info,
-						  struct irq_domain *parent)
-{
-	struct irq_domain *domain;
-
-	if (info->flags & MSI_FLAG_USE_DEF_DOM_OPS)
-		platform_msi_update_dom_ops(info);
-	if (info->flags & MSI_FLAG_USE_DEF_CHIP_OPS)
-		platform_msi_update_chip_ops(info);
-	info->flags |= MSI_FLAG_DEV_SYSFS | MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS |
-		       MSI_FLAG_FREE_MSI_DESCS;
-
-	domain = msi_create_irq_domain(fwnode, info, parent);
-	if (domain)
-		irq_domain_update_bus_token(domain, DOMAIN_BUS_PLATFORM_MSI);
-
-	return domain;
-}
-EXPORT_SYMBOL_GPL(platform_msi_create_irq_domain);
-
-static int platform_msi_alloc_priv_data(struct device *dev, unsigned int nvec,
-					irq_write_msi_msg_t write_msi_msg)
-{
-	struct platform_msi_priv_data *datap;
-	int err;
-
-	/*
-	 * Limit the number of interrupts to 2048 per device. Should we
-	 * need to bump this up, DEV_ID_SHIFT should be adjusted
-	 * accordingly (which would impact the max number of MSI
-	 * capable devices).
-	 */
-	if (!dev->msi.domain || !write_msi_msg || !nvec || nvec > MAX_DEV_MSIS)
-		return -EINVAL;
-
-	if (dev->msi.domain->bus_token != DOMAIN_BUS_PLATFORM_MSI) {
-		dev_err(dev, "Incompatible msi_domain, giving up\n");
-		return -EINVAL;
-	}
-
-	err = msi_setup_device_data(dev);
-	if (err)
-		return err;
-
-	/* Already initialized? */
-	if (dev->msi.data->platform_data)
-		return -EBUSY;
-
-	datap = kzalloc(sizeof(*datap), GFP_KERNEL);
-	if (!datap)
-		return -ENOMEM;
-
-	datap->devid = ida_alloc_max(&platform_msi_devid_ida,
-				     (1 << DEV_ID_SHIFT) - 1, GFP_KERNEL);
-	if (datap->devid < 0) {
-		err = datap->devid;
-		kfree(datap);
-		return err;
-	}
-
-	datap->write_msg = write_msi_msg;
-	datap->dev = dev;
-	dev->msi.data->platform_data = datap;
-	return 0;
-}
-
-static void platform_msi_free_priv_data(struct device *dev)
-{
-	struct platform_msi_priv_data *data = dev->msi.data->platform_data;
-
-	dev->msi.data->platform_data = NULL;
-	ida_free(&platform_msi_devid_ida, data->devid);
-	kfree(data);
-}
-
-/**
- * platform_msi_domain_alloc_irqs - Allocate MSI interrupts for @dev
- * @dev:		The device for which to allocate interrupts
- * @nvec:		The number of interrupts to allocate
- * @write_msi_msg:	Callback to write an interrupt message for @dev
- *
- * Returns:
- * Zero for success, or an error code in case of failure
- */
-static int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
-					  irq_write_msi_msg_t write_msi_msg)
-{
-	int err;
-
-	err = platform_msi_alloc_priv_data(dev, nvec, write_msi_msg);
-	if (err)
-		return err;
-
-	err = msi_domain_alloc_irqs_range(dev, MSI_DEFAULT_DOMAIN, 0, nvec - 1);
-	if (err)
-		platform_msi_free_priv_data(dev);
-
-	return err;
-}
-
-/**
- * platform_msi_get_host_data - Query the private data associated with
- *                              a platform-msi domain
- * @domain:	The platform-msi domain
- *
- * Return: The private data provided when calling
- * platform_msi_create_device_domain().
- */
-void *platform_msi_get_host_data(struct irq_domain *domain)
-{
-	struct platform_msi_priv_data *data = domain->host_data;
-
-	return data->host_data;
-}
-
-static struct lock_class_key platform_device_msi_lock_class;
-
-/**
- * __platform_msi_create_device_domain - Create a platform-msi device domain
- *
- * @dev:		The device generating the MSIs
- * @nvec:		The number of MSIs that need to be allocated
- * @is_tree:		flag to indicate tree hierarchy
- * @write_msi_msg:	Callback to write an interrupt message for @dev
- * @ops:		The hierarchy domain operations to use
- * @host_data:		Private data associated to this domain
- *
- * Return: An irqdomain for @nvec interrupts on success, NULL in case of error.
- *
- * This is for interrupt domains which stack on a platform-msi domain
- * created by platform_msi_create_irq_domain(). @dev->msi.domain points to
- * that platform-msi domain which is the parent for the new domain.
- */
-struct irq_domain *
-__platform_msi_create_device_domain(struct device *dev,
-				    unsigned int nvec,
-				    bool is_tree,
-				    irq_write_msi_msg_t write_msi_msg,
-				    const struct irq_domain_ops *ops,
-				    void *host_data)
-{
-	struct platform_msi_priv_data *data;
-	struct irq_domain *domain;
-	int err;
-
-	err = platform_msi_alloc_priv_data(dev, nvec, write_msi_msg);
-	if (err)
-		return NULL;
-
-	/*
-	 * Use a separate lock class for the MSI descriptor mutex on
-	 * platform MSI device domains because the descriptor mutex nests
-	 * into the domain mutex. See alloc/free below.
-	 */
-	lockdep_set_class(&dev->msi.data->mutex, &platform_device_msi_lock_class);
-
-	data = dev->msi.data->platform_data;
-	data->host_data = host_data;
-	domain = irq_domain_create_hierarchy(dev->msi.domain, 0,
-					     is_tree ? 0 : nvec,
-					     dev->fwnode, ops, data);
-	if (!domain)
-		goto free_priv;
-
-	platform_msi_set_proxy_dev(&data->arg);
-	err = msi_domain_prepare_irqs(domain->parent, dev, nvec, &data->arg);
-	if (err)
-		goto free_domain;
-
-	return domain;
-
-free_domain:
-	irq_domain_remove(domain);
-free_priv:
-	platform_msi_free_priv_data(dev);
-	return NULL;
-}
-
-/**
- * platform_msi_device_domain_free - Free interrupts associated with a platform-msi
- *				     device domain
- *
- * @domain:	The platform-msi device domain
- * @virq:	The base irq from which to perform the free operation
- * @nr_irqs:	How many interrupts to free from @virq
- */
-void platform_msi_device_domain_free(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nr_irqs)
-{
-	struct platform_msi_priv_data *data = domain->host_data;
-
-	msi_lock_descs(data->dev);
-	msi_domain_depopulate_descs(data->dev, virq, nr_irqs);
-	irq_domain_free_irqs_common(domain, virq, nr_irqs);
-	msi_free_msi_descs_range(data->dev, virq, virq + nr_irqs - 1);
-	msi_unlock_descs(data->dev);
-}
-
-/**
- * platform_msi_device_domain_alloc - Allocate interrupts associated with
- *				      a platform-msi device domain
- *
- * @domain:	The platform-msi device domain
- * @virq:	The base irq from which to perform the allocate operation
- * @nr_irqs:	How many interrupts to allocate from @virq
- *
- * Return 0 on success, or an error code on failure. Must be called
- * with irq_domain_mutex held (which can only be done as part of a
- * top-level interrupt allocation).
- */
-int platform_msi_device_domain_alloc(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nr_irqs)
-{
-	struct platform_msi_priv_data *data = domain->host_data;
-	struct device *dev = data->dev;
-
-	return msi_domain_populate_irqs(domain->parent, dev, virq, nr_irqs, &data->arg);
-}
-
-/* End of removal area */
-
-/* Real per device domain interfaces */
 
 /*
  * This indirection can go when platform_device_msi_init_and_alloc_irqs()
@@ -357,7 +23,7 @@ static void platform_msi_write_msi_msg(struct irq_data *d, struct msi_msg *msg)
 	cb(irq_data_get_msi_desc(d), msg);
 }
 
-static void platform_msi_set_desc_byindex(msi_alloc_info_t *arg, struct msi_desc *desc)
+static void platform_msi_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
 {
 	arg->desc = desc;
 	arg->hwirq = desc->msi_index;
@@ -373,7 +39,7 @@ static const struct msi_domain_template platform_msi_template = {
 	},
 
 	.ops = {
-		.set_desc		= platform_msi_set_desc_byindex,
+		.set_desc		= platform_msi_set_desc,
 	},
 
 	.info = {
@@ -408,10 +74,6 @@ int platform_device_msi_init_and_alloc_irqs(struct device *dev, unsigned int nve
 	if (!domain || !write_msi_msg)
 		return -EINVAL;
 
-	/* Migration support. Will go away once everything is converted */
-	if (!irq_domain_is_msi_parent(domain))
-		return platform_msi_domain_alloc_irqs(dev, nvec, write_msi_msg);
-
 	/*
 	 * @write_msi_msg is stored in the resulting msi_domain_info::data.
 	 * The underlying domain creation mechanism will assign that
@@ -432,12 +94,6 @@ EXPORT_SYMBOL_GPL(platform_device_msi_init_and_alloc_irqs);
  */
 void platform_device_msi_free_irqs_all(struct device *dev)
 {
-	struct irq_domain *domain = dev->msi.domain;
-
 	msi_domain_free_irqs_all(dev, MSI_DEFAULT_DOMAIN);
-
-	/* Migration support. Will go away once everything is converted */
-	if (!irq_domain_is_msi_parent(domain))
-		platform_msi_free_priv_data(dev);
 }
 EXPORT_SYMBOL_GPL(platform_device_msi_free_irqs_all);
include/linux/msi.h +0 -30
index 4ae036d..4c3462a 100644
@@ -81,7 +81,6 @@ extern int pci_msi_ignore_mask;
 /* Helper functions */
 struct msi_desc;
 struct pci_dev;
-struct platform_msi_priv_data;
 struct device_attribute;
 struct irq_domain;
 struct irq_affinity_desc;
@@ -231,14 +230,12 @@ struct msi_dev_domain {
 /**
  * msi_device_data - MSI per device data
  * @properties:		MSI properties which are interesting to drivers
- * @platform_data:	Platform-MSI specific data
  * @mutex:		Mutex protecting the MSI descriptor store
  * @__domains:		Internal data for per device MSI domains
  * @__iter_idx:		Index to search the next entry for iterators
  */
 struct msi_device_data {
 	unsigned long			properties;
-	struct platform_msi_priv_data	*platform_data;
 	struct mutex			mutex;
 	struct msi_dev_domain		__domains[MSI_MAX_DEVICE_IRQDOMAINS];
 	unsigned long			__iter_idx;
@@ -641,33 +638,6 @@ void msi_domain_free_irqs_all(struct device *dev, unsigned int domid);
 
 struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain);
 
-struct irq_domain *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
-						  struct msi_domain_info *info,
-						  struct irq_domain *parent);
-
-/* When an MSI domain is used as an intermediate domain */
-int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev,
-			    int nvec, msi_alloc_info_t *args);
-int msi_domain_populate_irqs(struct irq_domain *domain, struct device *dev,
-			     int virq, int nvec, msi_alloc_info_t *args);
-void msi_domain_depopulate_descs(struct device *dev, int virq, int nvec);
-
-struct irq_domain *
-__platform_msi_create_device_domain(struct device *dev,
-				    unsigned int nvec,
-				    bool is_tree,
-				    irq_write_msi_msg_t write_msi_msg,
-				    const struct irq_domain_ops *ops,
-				    void *host_data);
-
-#define platform_msi_create_device_tree_domain(dev, nvec, write, ops, data) \
-	__platform_msi_create_device_domain(dev, nvec, true, write, ops, data)
-
-int platform_msi_device_domain_alloc(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nr_irqs);
-void platform_msi_device_domain_free(struct irq_domain *domain, unsigned int virq,
-				     unsigned int nvec);
-void *platform_msi_get_host_data(struct irq_domain *domain);
 /* Per device platform MSI */
 int platform_device_msi_init_and_alloc_irqs(struct device *dev, unsigned int nvec,
 					    irq_write_msi_msg_t write_msi_msg);
kernel/irq/msi.c +2 -73

87 lines not on this page · raw message

#76 lore raw

PATCH v4 21/21 genirq/msi: Move msi_device_data to core

In reply to Thomas Gleixner

From: Thomas Gleixner <[email protected]>

Now that the platform MSI hack is gone, nothing needs to know about struct msi_device_data outside of the core code.

---
v3: fixed warning by kernel test robot
warning: cannot understand function prototype: 'struct msi_device_data '

---
 include/linux/msi.h | 18 ------------------
 kernel/irq/msi.c    | 20 ++++++++++++++++++--
 2 files changed, 18 insertions(+), 20 deletions(-)
include/linux/msi.h +0 -18
index 4c3462a6a97b..369367ecae5e 100644
@@ -21,11 +21,7 @@
 #include <linux/irqdomain_defs.h>
 #include <linux/cpumask.h>
 #include <linux/msi_api.h>
-#include <linux/xarray.h>
-#include <linux/mutex.h>
-#include <linux/list.h>
 #include <linux/irq.h>
-#include <linux/bits.h>
 
 #include <asm/msi.h>
 
@@ -227,20 +223,6 @@ struct msi_dev_domain {
 	struct irq_domain	*domain;
 };
 
-/**
- * msi_device_data - MSI per device data
- * @properties:		MSI properties which are interesting to drivers
- * @mutex:		Mutex protecting the MSI descriptor store
- * @__domains:		Internal data for per device MSI domains
- * @__iter_idx:		Index to search the next entry for iterators
- */
-struct msi_device_data {
-	unsigned long			properties;
-	struct mutex			mutex;
-	struct msi_dev_domain		__domains[MSI_MAX_DEVICE_IRQDOMAINS];
-	unsigned long			__iter_idx;
-};
-
 int msi_setup_device_data(struct device *dev);
 
 void msi_lock_descs(struct device *dev);
kernel/irq/msi.c +18 -2
index 8314b1d4a903..5fa0547ece0c 100644
@@ -8,17 +8,33 @@
  * This file contains common code to support Message Signaled Interrupts for
  * PCI compatible and non PCI compatible devices.
  */
-#include <linux/types.h>
 #include <linux/device.h>
 #include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
+#include <linux/mutex.h>
+#include <linux/pci.h>
 #include <linux/slab.h>
 #include <linux/sysfs.h>
-#include <linux/pci.h>
+#include <linux/types.h>
+#include <linux/xarray.h>
 
 #include "internals.h"
 
+/**
+ * struct msi_device_data - MSI per device data
+ * @properties:		MSI properties which are interesting to drivers
+ * @mutex:		Mutex protecting the MSI descriptor store
+ * @__domains:		Internal data for per device MSI domains
+ * @__iter_idx:		Index to search the next entry for iterators
+ */
+struct msi_device_data {
+	unsigned long			properties;
+	struct mutex			mutex;
+	struct msi_dev_domain		__domains[MSI_MAX_DEVICE_IRQDOMAINS];
+	unsigned long			__iter_idx;
+};
+
 /**
  * struct msi_ctrl - MSI internal management control structure
  * @domid:	ID of the domain on which management operations should be done
Signature · 1 line
2.34.1
#77 lore raw

[tip: irq/core] genirq/msi: Move msi_device_data to core

The following commit has been merged into the irq/core branch of tip:

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     ccde7dc94768b4777402247e1cabb58c2b00f495
Gitweb:        https://git.kernel.org/tip/ccde7dc94768b4777402247e1cabb58c2b00f495
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:19:07 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 10 Jul 2024 18:19:25 +02:00

genirq/msi: Move msi_device_data to core

Now that the platform MSI hack is gone, nothing needs to know about struct msi_device_data outside of the core code.

---
 include/linux/msi.h | 18 ------------------
 kernel/irq/msi.c    | 20 ++++++++++++++++++--
 2 files changed, 18 insertions(+), 20 deletions(-)
include/linux/msi.h +0 -18
index 4c3462a..369367e 100644
@@ -21,11 +21,7 @@
 #include <linux/irqdomain_defs.h>
 #include <linux/cpumask.h>
 #include <linux/msi_api.h>
-#include <linux/xarray.h>
-#include <linux/mutex.h>
-#include <linux/list.h>
 #include <linux/irq.h>
-#include <linux/bits.h>
 
 #include <asm/msi.h>
 
@@ -227,20 +223,6 @@ struct msi_dev_domain {
 	struct irq_domain	*domain;
 };
 
-/**
- * msi_device_data - MSI per device data
- * @properties:		MSI properties which are interesting to drivers
- * @mutex:		Mutex protecting the MSI descriptor store
- * @__domains:		Internal data for per device MSI domains
- * @__iter_idx:		Index to search the next entry for iterators
- */
-struct msi_device_data {
-	unsigned long			properties;
-	struct mutex			mutex;
-	struct msi_dev_domain		__domains[MSI_MAX_DEVICE_IRQDOMAINS];
-	unsigned long			__iter_idx;
-};
-
 int msi_setup_device_data(struct device *dev);
 
 void msi_lock_descs(struct device *dev);
kernel/irq/msi.c +18 -2
index 8314b1d..5fa0547 100644
@@ -8,18 +8,34 @@
  * This file contains common code to support Message Signaled Interrupts for
  * PCI compatible and non PCI compatible devices.
  */
-#include <linux/types.h>
 #include <linux/device.h>
 #include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
+#include <linux/mutex.h>
+#include <linux/pci.h>
 #include <linux/slab.h>
 #include <linux/sysfs.h>
-#include <linux/pci.h>
+#include <linux/types.h>
+#include <linux/xarray.h>
 
 #include "internals.h"
 
 /**
+ * struct msi_device_data - MSI per device data
+ * @properties:		MSI properties which are interesting to drivers
+ * @mutex:		Mutex protecting the MSI descriptor store
+ * @__domains:		Internal data for per device MSI domains
+ * @__iter_idx:		Index to search the next entry for iterators
+ */
+struct msi_device_data {
+	unsigned long			properties;
+	struct mutex			mutex;
+	struct msi_dev_domain		__domains[MSI_MAX_DEVICE_IRQDOMAINS];
+	unsigned long			__iter_idx;
+};
+
+/**
  * struct msi_ctrl - MSI internal management control structure
  * @domid:	ID of the domain on which management operations should be done
  * @first:	First (hardware) slot index to operate on
#78 lore raw

[tip: irq/msi] genirq/msi: Move msi_device_data to core

In reply to Thomas Gleixner
The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     2fdda02a8749fdaff5621c96aaf24a61d2f8c5a2
Gitweb:        https://git.kernel.org/tip/2fdda02a8749fdaff5621c96aaf24a61d2f8c5a2
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Sun, 23 Jun 2024 17:19:07 +02:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Thu, 18 Jul 2024 20:31:21 +02:00

genirq/msi: Move msi_device_data to core

Now that the platform MSI hack is gone, nothing needs to know about struct msi_device_data outside of the core code.

---
 include/linux/msi.h | 18 ------------------
 kernel/irq/msi.c    | 20 ++++++++++++++++++--
 2 files changed, 18 insertions(+), 20 deletions(-)
include/linux/msi.h +0 -18
index 4c3462a..369367e 100644
@@ -21,11 +21,7 @@
 #include <linux/irqdomain_defs.h>
 #include <linux/cpumask.h>
 #include <linux/msi_api.h>
-#include <linux/xarray.h>
-#include <linux/mutex.h>
-#include <linux/list.h>
 #include <linux/irq.h>
-#include <linux/bits.h>
 
 #include <asm/msi.h>
 
@@ -227,20 +223,6 @@ struct msi_dev_domain {
 	struct irq_domain	*domain;
 };
 
-/**
- * msi_device_data - MSI per device data
- * @properties:		MSI properties which are interesting to drivers
- * @mutex:		Mutex protecting the MSI descriptor store
- * @__domains:		Internal data for per device MSI domains
- * @__iter_idx:		Index to search the next entry for iterators
- */
-struct msi_device_data {
-	unsigned long			properties;
-	struct mutex			mutex;
-	struct msi_dev_domain		__domains[MSI_MAX_DEVICE_IRQDOMAINS];
-	unsigned long			__iter_idx;
-};
-
 int msi_setup_device_data(struct device *dev);
 
 void msi_lock_descs(struct device *dev);
kernel/irq/msi.c +18 -2
index 8314b1d..5fa0547 100644
@@ -8,18 +8,34 @@
  * This file contains common code to support Message Signaled Interrupts for
  * PCI compatible and non PCI compatible devices.
  */
-#include <linux/types.h>
 #include <linux/device.h>
 #include <linux/irq.h>
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
+#include <linux/mutex.h>
+#include <linux/pci.h>
 #include <linux/slab.h>
 #include <linux/sysfs.h>
-#include <linux/pci.h>
+#include <linux/types.h>
+#include <linux/xarray.h>
 
 #include "internals.h"
 
 /**
+ * struct msi_device_data - MSI per device data
+ * @properties:		MSI properties which are interesting to drivers
+ * @mutex:		Mutex protecting the MSI descriptor store
+ * @__domains:		Internal data for per device MSI domains
+ * @__iter_idx:		Index to search the next entry for iterators
+ */
+struct msi_device_data {
+	unsigned long			properties;
+	struct mutex			mutex;
+	struct msi_dev_domain		__domains[MSI_MAX_DEVICE_IRQDOMAINS];
+	unsigned long			__iter_idx;
+};
+
+/**
  * struct msi_ctrl - MSI internal management control structure
  * @domid:	ID of the domain on which management operations should be done
  * @first:	First (hardware) slot index to operate on
reviewer
#79 lore raw
In reply to Thomas Gleixner
Quoted 25 lines · Thomas Gleixner

This is version 4 of the series to convert ARM MSI handling over to per device MSI domains. Version 3 can be found here:

https://lore.kernel.org/lkml/[email protected]

The conversion aims to replace the existing platform MSI mechanism and enables ARM to support the future PCI/IMS mechanism.

The infrastructure to replace the platform MSI mechanism is already upstream and in use by RISC-V and has been tested on various ARM platforms during the V2 development.

Changes vs. V3:

- Fix the conversion of the GIC V3 MBI driver - Marc

- Dropped a few stray MSI_FLAG_PCI_MSI_MASK_PARENT flags

- Dropped the trivial cleanup patches as they have been merged

- Picked up tags

The series is only lightly tested due to lack of hardware, so we rely on the people who have access to affected machines to help with testing.

Quoted 6 lines

If there are no major objections raised or testing fallout reported, I'm aiming this series for the next merge window.

The series is also available from git:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git devmsi-arm-v4

Running this thru kernelCI has some failures on x86 QEMU boots[1]. Here's the backtrace:

<1>[    2.199948] BUG: kernel NULL pointer dereference, address: 0000000000000000
<1>[    2.199948] #PF: supervisor instruction fetch in kernel mode
<1>[    2.199948] #PF: error_code(0x0010) - not-present page
<6>[    2.199948] PGD 0 P4D 0 
<4>[    2.199948] Oops: Oops: 0010 [#1] PREEMPT SMP NOPTI
<4>[    2.199948] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.10.0-rc3 #1
<4>[    2.199948] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014
<4>[    2.199948] RIP: 0010:0x0
<4>[    2.199948] Code: Unable to access opcode bytes at 0xffffffffffffffd6.
<4>[    2.199948] RSP: 0018:ffffa7ac80013a90 EFLAGS: 00000002
<4>[    2.199948] RAX: 0000000000000000 RBX: ffffa4050333d600 RCX: 0000000000000000
<4>[    2.199948] RDX: ffffa4050333d430 RSI: 0000000000000001 RDI: ffffa40502ff3100
<4>[    2.199948] RBP: ffffa4050333d600 R08: ffffa405032f1c00 R09: 0000000000000000
<4>[    2.199948] R10: 0000000000000246 R11: ffffa405032f1d80 R12: ffffa405032f1d80
<4>[    2.199948] R13: 0000000000000001 R14: 0000000000000000 R15: ffffa4050333d760
<4>[    2.199948] FS:  0000000000000000(0000) GS:ffffa4053e400000(0000) knlGS:0000000000000000
<4>[    2.199948] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[    2.199948] CR2: ffffffffffffffd6 CR3: 000000002a22e000 CR4: 00000000000006f0
<4>[    2.199948] Call Trace:
<4>[    2.199948]  <TASK>
<4>[    2.199948]  ? __die+0x1f/0x70
<4>[    2.199948]  ? page_fault_oops+0x155/0x440
<4>[    2.199948]  ? ondemand_readahead+0x2c0/0x370
<4>[    2.199948]  ? bitmap_find_next_zero_area_off+0x7b/0x90
<4>[    2.199948]  ? exc_page_fault+0x69/0x150
<4>[    2.199948]  ? asm_exc_page_fault+0x26/0x30
<4>[    2.199948]  pci_irq_unmask_msix+0x53/0x60
<4>[    2.199948]  irq_enable+0x32/0x80
<4>[    2.199948]  __irq_startup+0x51/0x70
<4>[    2.199948]  irq_startup+0x62/0x120
<4>[    2.199948]  __setup_irq+0x326/0x730
<4>[    2.199948]  ? __pfx_vp_config_changed+0x10/0x10
<4>[    2.199948]  request_threaded_irq+0x10b/0x180
<4>[    2.199948]  vp_find_vqs_msix+0x16b/0x470
<4>[    2.199948]  vp_find_vqs+0x34/0x1a0
<4>[    2.199948]  vp_modern_find_vqs+0x16/0x60
<4>[    2.199948]  init_vqs+0x3ee/0x690
<4>[    2.199948]  virtnet_probe+0x50c/0xd10
<4>[    2.199948]  virtio_dev_probe+0x1dd/0x2b0
<4>[    2.199948]  really_probe+0xbc/0x2b0
<4>[    2.199948]  __driver_probe_device+0x6e/0x120
<4>[    2.199948]  driver_probe_device+0x19/0xe0
<4>[    2.199948]  __driver_attach+0x85/0x180
<4>[    2.199948]  ? __pfx___driver_attach+0x10/0x10
<4>[    2.199948]  bus_for_each_dev+0x76/0xd0
<4>[    2.199948]  bus_add_driver+0xe3/0x210
<4>[    2.199948]  driver_register+0x5b/0x110
<4>[    2.199948]  ? __pfx_virtio_net_driver_init+0x10/0x10
<4>[    2.199948]  virtio_net_driver_init+0x8b/0xb0
<4>[    2.199948]  ? __pfx_virtio_net_driver_init+0x10/0x10
<4>[    2.199948]  do_one_initcall+0x43/0x210
<4>[    2.199948]  kernel_init_freeable+0x19b/0x2d0
<4>[    2.199948]  ? __pfx_kernel_init+0x10/0x10
<4>[    2.199948]  kernel_init+0x15/0x1c0
<4>[    2.199948]  ret_from_fork+0x2f/0x50
<4>[    2.199948]  ? __pfx_kernel_init+0x10/0x10
<4>[    2.199948]  ret_from_fork_asm+0x1a/0x30
<4>[    2.199948]  </TASK>
<4>[    2.199948] Modules linked in:
<4>[    2.199948] CR2: 0000000000000000
<4>[    2.199948] ---[ end trace 0000000000000000 ]---

Rob

[1] https://linux.kernelci.org/test/job/robh/branch/for-kernelci/kernel/v6.10-rc3-21-gd27f9f4a2dd80/plan/baseline/

#80 lore raw
Quoted 1 line · Rob Herring

Running this thru kernelCI has some failures on x86 QEMU boots[1].

oops

Quoted 2 lines
<4>[    2.199948]  pci_irq_unmask_msix+0x53/0x60
<4>[    2.199948]  irq_enable+0x32/0x80

I'm sure that I fixed that before.

Updated branch:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git devmsi-arm-v4-1

Thanks,

tglx

maintainer
#81 lore raw
In reply to Thomas Gleixner
Quoted 11 lines · Thomas Gleixner

This is version 4 of the series to convert ARM MSI handling over to per device MSI domains.

The conversion aims to replace the existing platform MSI mechanism and enables ARM to support the future PCI/IMS mechanism.

The series is only lightly tested due to lack of hardware, so we rely on the people who have access to affected machines to help with testing.

If there are no major objections raised or testing fallout reported, I'm aiming this series for the next merge window.

This series only showed up in linux-next last Friday and broke interrupt handling on Qualcomm platforms like sc8280xp (e.g. Lenovo ThinkPad X13s) and x1e80100 that use the GIC ITS for PCIe MSIs.

I've applied the series (21 commits from linux-next) on top of 6.10 and can confirm that the breakage is caused by commits:

	3d1c927c08fc ("irqchip/gic-v3-its: Switch platform MSI to MSI parent")
	233db05bc37f ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")

Applying the series up until the change before 3d1c927c08fc unbreaks the wifi on one machine:

	ath11k_pci 0006:01:00.0: failed to enable msi: -22
	ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -22

and backing up until the commit before 233db05bc37f makes the NVMe come up again during boot on another.

I have not tried to debug this further.

Johan

maintainer
#82 lore raw
Quoted 27 lines · Johan Hovold
Quoted 11 lines · Thomas Gleixner

This is version 4 of the series to convert ARM MSI handling over to per device MSI domains.

The conversion aims to replace the existing platform MSI mechanism and enables ARM to support the future PCI/IMS mechanism.

The series is only lightly tested due to lack of hardware, so we rely on the people who have access to affected machines to help with testing.

If there are no major objections raised or testing fallout reported, I'm aiming this series for the next merge window.

This series only showed up in linux-next last Friday and broke interrupt handling on Qualcomm platforms like sc8280xp (e.g. Lenovo ThinkPad X13s) and x1e80100 that use the GIC ITS for PCIe MSIs.

I've applied the series (21 commits from linux-next) on top of 6.10 and can confirm that the breakage is caused by commits:

	3d1c927c08fc ("irqchip/gic-v3-its: Switch platform MSI to MSI parent")
	233db05bc37f ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")

Applying the series up until the change before 3d1c927c08fc unbreaks the wifi on one machine:

Quoted 7 lines
	ath11k_pci 0006:01:00.0: failed to enable msi: -22
	ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -22

and backing up until the commit before 233db05bc37f makes the NVMe come up again during boot on another.

I have not tried to debug this further.

I need a few things from you though, because you're not giving much to help you (and I'm travelling, which doesn't help).

Can you at least investigate what in ath11k_pci_alloc_msi() causes the wifi driver to be upset? Does it normally use a single MSI vector or MSI-X? How about your nVME device?

It would also help if you could define the DEBUG symbol at the very top of irq-gic-v3-its.c and report the debug information that the ITS driver dumps.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#83 lore raw
Quoted 22 lines · Marc Zyngier
Quoted 20 lines · Johan Hovold
Quoted 2 lines · Thomas Gleixner

This is version 4 of the series to convert ARM MSI handling over to per device MSI domains.

This series only showed up in linux-next last Friday and broke interrupt handling on Qualcomm platforms like sc8280xp (e.g. Lenovo ThinkPad X13s) and x1e80100 that use the GIC ITS for PCIe MSIs.

I've applied the series (21 commits from linux-next) on top of 6.10 and can confirm that the breakage is caused by commits:

	3d1c927c08fc ("irqchip/gic-v3-its: Switch platform MSI to MSI parent")
	233db05bc37f ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")

Applying the series up until the change before 3d1c927c08fc unbreaks the wifi on one machine:

	ath11k_pci 0006:01:00.0: failed to enable msi: -22
	ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -22
Quoted 7 lines
Quoted 4 lines

and backing up until the commit before 233db05bc37f makes the NVMe come up again during boot on another.

I have not tried to debug this further.

I need a few things from you though, because you're not giving much to help you (and I'm travelling, which doesn't help).

Yeah, this was just an early heads up.

Quoted 3 lines

Can you at least investigate what in ath11k_pci_alloc_msi() causes the wifi driver to be upset? Does it normally use a single MSI vector or MSI-X? How about your nVME device?

It uses multiple vectors, but now it falls back to trying to allocate a single one and even that fails with -ENOSPC:

	ath11k_pci 0006:01:00.0: ath11k_pci_alloc_msi - requesting one vector failed: -28

Similar for the NVMe, it uses multiple vectors normally, but now only the AER interrupts appears to be allocated for each controller and there is a GICv3 interrupt for the NVMe:

208:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0006:00:00.0   0 Edge      PCIe PME, aerdrv
212:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0004:00:00.0   0 Edge      PCIe PME, aerdrv
214:        161          0          0          0          0          0          0          0     GICv3 562 Level     nvme0q0, nvme0q1
215:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0002:00:00.0   0 Edge      PCIe PME, aerdrv

Next boot, after disabling PCIe controller async probing, it's an MSI-X?!:

201:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0006:00:00.0   0 Edge      PCIe PME, aerdrv
203:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0004:00:00.0   0 Edge      PCIe PME, aerdrv
205:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0002:00:00.0   0 Edge      PCIe PME, aerdrv
206:          0          0          0          0          0          0          0          0  ITS-PCI-MSIX-0002:01:00.0   0 Edge      nvme0q0

This time ath11k vector allocation succeeded, but the driver times out eventually:

[    8.984619] ath11k_pci 0006:01:00.0: MSI vectors: 32
[   29.690841] ath11k_pci 0006:01:00.0: failed to power up mhi: -110
[   29.697136] ath11k_pci 0006:01:00.0: failed to start mhi: -110
[   29.703153] ath11k_pci 0006:01:00.0: failed to power up :-110
[   29.732144] ath11k_pci 0006:01:00.0: failed to create soc core: -110
[   29.738694] ath11k_pci 0006:01:00.0: failed to init core: -110
[   32.841758] ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -110
Quoted 3 lines

It would also help if you could define the DEBUG symbol at the very top of irq-gic-v3-its.c and report the debug information that the ITS driver dumps.

See below (with synchronous probing of the pcie controllers).

Johan

[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: 960 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GICv3: GICv3 features: 16 PPIs
[    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x0000000017a60000
[    0.000000] ITS [mem 0x17a40000-0x17a5ffff]
[    0.000000] ITS@0x0000000017a40000: allocated 8192 Devices @100100000 (indirect, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x0000000017a40000: allocated 32768 Interrupt Collections @100110000 (flat, esz 2, psz 64K, shr 1)
[    0.000000] GICv3: using LPI property table @0x0000000100120000
[    0.000000] ITS: Allocator initialized for 57344 LPIs
[    0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100130000
[    0.010428] GICv3: CPU1: found redistributor 100 region 0:0x0000000017a80000
[    0.010438] GICv3: CPU1: using allocated LPI pending table @0x0000000100140000
[    0.010477] CPU1: Booted secondary processor 0x0000000100 [0x410fd4b0]
[    0.011496] Detected PIPT I-cache on CPU2
[    0.011535] GICv3: CPU2: found redistributor 200 region 0:0x0000000017aa0000
[    0.011545] GICv3: CPU2: using allocated LPI pending table @0x0000000100150000
[    0.011576] CPU2: Booted secondary processor 0x0000000200 [0x410fd4b0]
[    0.012593] Detected PIPT I-cache on CPU3
[    0.012631] GICv3: CPU3: found redistributor 300 region 0:0x0000000017ac0000
[    0.012641] GICv3: CPU3: using allocated LPI pending table @0x0000000100160000
[    0.012671] CPU3: Booted secondary processor 0x0000000300 [0x410fd4b0]
[    0.015590] Detected PIPT I-cache on CPU4
[    0.015637] GICv3: CPU4: found redistributor 400 region 0:0x0000000017ae0000
[    0.015647] GICv3: CPU4: using allocated LPI pending table @0x0000000100170000
[    0.015675] CPU4: Booted secondary processor 0x0000000400 [0x410fd4c0]
[    0.016698] Detected PIPT I-cache on CPU5
[    0.016733] GICv3: CPU5: found redistributor 500 region 0:0x0000000017b00000
[    0.016742] GICv3: CPU5: using allocated LPI pending table @0x0000000100180000
[    0.016772] CPU5: Booted secondary processor 0x0000000500 [0x410fd4c0]
[    0.020807] Detected PIPT I-cache on CPU6
[    0.020841] GICv3: CPU6: found redistributor 600 region 0:0x0000000017b20000
[    0.020851] GICv3: CPU6: using allocated LPI pending table @0x0000000100190000
[    0.020879] CPU6: Booted secondary processor 0x0000000600 [0x410fd4c0]
[    0.021878] Detected PIPT I-cache on CPU7
[    0.021914] GICv3: CPU7: found redistributor 700 region 0:0x0000000017b40000
[    0.021922] GICv3: CPU7: using allocated LPI pending table @0x00000001001a0000
[    0.021952] CPU7: Booted secondary processor 0x0000000700 [0x410fd4c0]
[    8.358586] qcom-pcie 1c00000.pcie: host bridge /soc@0/pcie@1c00000 ranges:
[    8.365787] qcom-pcie 1c00000.pcie:       IO 0x0030200000..0x00302fffff -> 0x0000000000
[    8.381670] qcom-pcie 1c00000.pcie:      MEM 0x0030300000..0x0031ffffff -> 0x0030300000
[    8.507519] qcom-pcie 1c00000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 1024G
[    8.603797] qcom-pcie 1c00000.pcie: PCIe Gen.2 x1 link up
[    8.610023] qcom-pcie 1c00000.pcie: PCI host bridge to bus 0006:00
[    8.616805] pci_bus 0006:00: root bus resource [bus 00-ff]
[    8.622872] pci_bus 0006:00: root bus resource [io  0x0000-0xfffff]
[    8.629844] pci_bus 0006:00: root bus resource [mem 0x30300000-0x31ffffff]
[    8.636981] pci 0006:00:00.0: [17cb:010e] type 01 class 0x060400 PCIe Root Port
[    8.655493] pci 0006:00:00.0: BAR 0 [mem 0x00000000-0x00000fff]
[    8.672909] pci 0006:00:00.0: PCI bridge to [bus 01-ff]
[    8.688721] pci 0006:00:00.0:   bridge window [io  0x0000-0x0fff]
[    8.703805] pci 0006:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    8.719789] pci 0006:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    8.736680] pci 0006:00:00.0: PME# supported from D0 D3hot D3cold
[    8.745548] pci 0006:01:00.0: [17cb:1103] type 00 class 0x028000 PCIe Endpoint
[    8.745646] pci 0006:01:00.0: BAR 0 [mem 0x00000000-0x001fffff 64bit]
[    8.746274] pci 0006:01:00.0: PME# supported from D0 D3hot D3cold
[    8.746442] pci 0006:01:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x1 link at 0006:00:00.0 (capable of 7.876 Gb/s with 8.0 GT/s PCIe x1 link)
[    8.836195] pci 0006:00:00.0: bridge window [mem 0x30400000-0x305fffff]: assigned
[    8.853287] pci 0006:00:00.0: BAR 0 [mem 0x30300000-0x30300fff]: assigned
[    8.870163] pci 0006:01:00.0: BAR 0 [mem 0x30400000-0x305fffff 64bit]: assigned
[    8.887617] pci 0006:00:00.0: PCI bridge to [bus 01-ff]
[    8.902850] pci 0006:00:00.0:   bridge window [mem 0x30400000-0x305fffff]
[    8.933586] ITS: alloc 8192:32
[    8.933599] ITT 32 entries, 5 bits
[    8.951573] ID:0 pID:8192 vID:201
[    8.951585] ID:1 pID:8193 vID:202
[    8.951591] ID:2 pID:8194 vID:203
[    8.951597] ID:3 pID:8195 vID:204
[    8.951603] ID:4 pID:8196 vID:205
[    8.951609] ID:5 pID:8197 vID:206
[    8.951615] ID:6 pID:8198 vID:207
[    8.951621] ID:7 pID:8199 vID:208
[    8.951627] ID:8 pID:8200 vID:209
[    8.951633] ID:9 pID:8201 vID:210
[    8.951639] ID:10 pID:8202 vID:211
[    8.951645] ID:11 pID:8203 vID:212
[    8.951650] ID:12 pID:8204 vID:213
[    8.951656] ID:13 pID:8205 vID:214
[    8.951662] ID:14 pID:8206 vID:215
[    8.951667] ID:15 pID:8207 vID:216
[    8.951673] ID:16 pID:8208 vID:217
[    8.951679] ID:17 pID:8209 vID:218
[    8.951685] ID:18 pID:8210 vID:219
[    8.951691] ID:19 pID:8211 vID:220
[    8.951696] ID:20 pID:8212 vID:221
[    8.951702] ID:21 pID:8213 vID:222
[    8.951708] ID:22 pID:8214 vID:223
[    8.951714] ID:23 pID:8215 vID:224
[    8.951720] ID:24 pID:8216 vID:225
[    8.951725] ID:25 pID:8217 vID:226
[    8.951772] ID:26 pID:8218 vID:227
[    8.951778] ID:27 pID:8219 vID:228
[    8.951784] ID:28 pID:8220 vID:229
[    8.951790] ID:29 pID:8221 vID:230
[    8.951796] ID:30 pID:8222 vID:231
[    8.951802] ID:31 pID:8223 vID:232
[    8.951919] IRQ201 -> 0-7 CPU0
[    8.951940] IRQ202 -> 0-7 CPU1
[    8.951952] IRQ203 -> 0-7 CPU2
[    8.951963] IRQ204 -> 0-7 CPU3
[    8.951975] IRQ205 -> 0-7 CPU4
[    8.951987] IRQ206 -> 0-7 CPU5
[    8.951998] IRQ207 -> 0-7 CPU6
[    8.952010] IRQ208 -> 0-7 CPU7
[    8.952022] IRQ209 -> 0-7 CPU0
[    8.952033] IRQ210 -> 0-7 CPU1
[    8.952045] IRQ211 -> 0-7 CPU2
[    8.952056] IRQ212 -> 0-7 CPU3
[    8.952068] IRQ213 -> 0-7 CPU4
[    8.952079] IRQ214 -> 0-7 CPU5
[    8.952091] IRQ215 -> 0-7 CPU6
[    8.952103] IRQ216 -> 0-7 CPU7
[    8.952115] IRQ217 -> 0-7 CPU0
[    8.952126] IRQ218 -> 0-7 CPU1
[    8.952138] IRQ219 -> 0-7 CPU2
[    8.952150] IRQ220 -> 0-7 CPU3
[    8.952162] IRQ221 -> 0-7 CPU4
[    8.952174] IRQ222 -> 0-7 CPU5
[    8.952185] IRQ223 -> 0-7 CPU6
[    8.952197] IRQ224 -> 0-7 CPU7
[    8.952209] IRQ225 -> 0-7 CPU0
[    8.952220] IRQ226 -> 0-7 CPU1
[    8.952232] IRQ227 -> 0-7 CPU2
[    8.952244] IRQ228 -> 0-7 CPU3
[    8.952255] IRQ229 -> 0-7 CPU4
[    8.952267] IRQ230 -> 0-7 CPU5
[    8.952278] IRQ231 -> 0-7 CPU6
[    8.952290] IRQ232 -> 0-7 CPU7
[    8.954072] ITS: alloc 8192:32
[    8.954081] ITT 32 entries, 5 bits
[    8.954128] ID:0 pID:8192 vID:201
[    8.954137] IRQ201 -> 0-7 CPU0
[    8.954328] IRQ201 -> 0-7 CPU0
[    8.954357] pcieport 0006:00:00.0: PME: Signaling with IRQ 201
[    8.960980] pcieport 0006:00:00.0: AER: enabled with IRQ 201
[    8.967607] ath11k_pci 0006:01:00.0: BAR 0 [mem 0x30400000-0x305fffff 64bit]: assigned
[    8.976146] ath11k_pci 0006:01:00.0: enabling device (0000 -> 0002)
[    8.983071] ITS: alloc 8224:32
[    8.983080] ITT 32 entries, 5 bits
[    8.983842] ID:0 pID:8224 vID:202
[    8.983849] ID:1 pID:8225 vID:203
[    8.983855] ID:2 pID:8226 vID:204
[    8.983861] ID:3 pID:8227 vID:205
[    8.983867] ID:4 pID:8228 vID:206
[    8.983873] ID:5 pID:8229 vID:207
[    8.983878] ID:6 pID:8230 vID:208
[    8.983884] ID:7 pID:8231 vID:209
[    8.983890] ID:8 pID:8232 vID:210
[    8.983895] ID:9 pID:8233 vID:211
[    8.983901] ID:10 pID:8234 vID:212
[    8.983907] ID:11 pID:8235 vID:213
[    8.983913] ID:12 pID:8236 vID:214
[    8.983919] ID:13 pID:8237 vID:215
[    8.983925] ID:14 pID:8238 vID:216
[    8.983931] ID:15 pID:8239 vID:217
[    8.983937] ID:16 pID:8240 vID:218
[    8.983942] ID:17 pID:8241 vID:219
[    8.983948] ID:18 pID:8242 vID:220
[    8.983954] ID:19 pID:8243 vID:221
[    8.983960] ID:20 pID:8244 vID:222
[    8.983965] ID:21 pID:8245 vID:223
[    8.983971] ID:22 pID:8246 vID:224
[    8.983977] ID:23 pID:8247 vID:225
[    8.983983] ID:24 pID:8248 vID:226
[    8.983989] ID:25 pID:8249 vID:227
[    8.983995] ID:26 pID:8250 vID:228
[    8.984000] ID:27 pID:8251 vID:229
[    8.984006] ID:28 pID:8252 vID:230
[    8.984012] ID:29 pID:8253 vID:231
[    8.984018] ID:30 pID:8254 vID:232
[    8.984024] ID:31 pID:8255 vID:233
[    8.984102] IRQ202 -> 0-7 CPU1
[    8.984148] IRQ203 -> 0-7 CPU2
[    8.984160] IRQ204 -> 0-7 CPU3
[    8.984172] IRQ205 -> 0-7 CPU4
[    8.984184] IRQ206 -> 0-7 CPU5
[    8.984196] IRQ207 -> 0-7 CPU6
[    8.984208] IRQ208 -> 0-7 CPU7
[    8.984220] IRQ209 -> 0-7 CPU0
[    8.984231] IRQ210 -> 0-7 CPU1
[    8.984243] IRQ211 -> 0-7 CPU2
[    8.984255] IRQ212 -> 0-7 CPU3
[    8.984267] IRQ213 -> 0-7 CPU4
[    8.984279] IRQ214 -> 0-7 CPU5
[    8.984291] IRQ215 -> 0-7 CPU6
[    8.984303] IRQ216 -> 0-7 CPU7
[    8.984315] IRQ217 -> 0-7 CPU0
[    8.984326] IRQ218 -> 0-7 CPU1
[    8.984338] IRQ219 -> 0-7 CPU2
[    8.984350] IRQ220 -> 0-7 CPU3
[    8.984362] IRQ221 -> 0-7 CPU4
[    8.984373] IRQ222 -> 0-7 CPU5
[    8.984385] IRQ223 -> 0-7 CPU6
[    8.984398] IRQ224 -> 0-7 CPU7
[    8.984409] IRQ225 -> 0-7 CPU0
[    8.984422] IRQ226 -> 0-7 CPU1
[    8.984434] IRQ227 -> 0-7 CPU2
[    8.984445] IRQ228 -> 0-7 CPU3
[    8.984457] IRQ229 -> 0-7 CPU4
[    8.984469] IRQ230 -> 0-7 CPU5
[    8.984481] IRQ231 -> 0-7 CPU6
[    8.984492] IRQ232 -> 0-7 CPU7
[    8.984504] IRQ233 -> 0-7 CPU0
[    8.984619] ath11k_pci 0006:01:00.0: MSI vectors: 32
[    8.990070] ath11k_pci 0006:01:00.0: wcn6855 hw2.0
[    8.998289] IRQ202 -> 0-7 CPU1
[    8.998348] IRQ203 -> 0-7 CPU2
[    8.998376] IRQ204 -> 0-7 CPU3
[    9.001890] IRQ205 -> 0-7 CPU4
[    9.001923] IRQ206 -> 0-7 CPU5
[    9.001953] IRQ207 -> 0-7 CPU6
[    9.001977] IRQ208 -> 0-7 CPU7
[    9.002003] IRQ209 -> 0-7 CPU0
[    9.002031] IRQ210 -> 0-7 CPU1
[    9.002055] IRQ211 -> 0-7 CPU2
[    9.002117] IRQ216 -> 0-7 CPU7
[    9.002168] IRQ217 -> 0-7 CPU0
[    9.002210] IRQ218 -> 0-7 CPU1
[    9.002257] IRQ220 -> 0-7 CPU3
[    9.002296] IRQ221 -> 0-7 CPU4
[    9.002337] IRQ222 -> 0-7 CPU5
[    9.002381] IRQ223 -> 0-7 CPU6
[    9.002421] IRQ224 -> 0-7 CPU7
[    9.002460] IRQ225 -> 0-7 CPU0
[    9.002499] IRQ226 -> 0-7 CPU1
[    9.162382] mhi mhi0: Requested to power ON
[    9.167114] mhi mhi0: Power on setup success
[   29.680356] mhi mhi0: Device link is not accessible
[   29.685437] mhi mhi0: MHI did not enter READY state
[   29.690841] ath11k_pci 0006:01:00.0: failed to power up mhi: -110
[   29.697136] ath11k_pci 0006:01:00.0: failed to start mhi: -110
[   29.703153] ath11k_pci 0006:01:00.0: failed to power up :-110
[   29.732144] ath11k_pci 0006:01:00.0: failed to create soc core: -110
[   29.738694] ath11k_pci 0006:01:00.0: failed to init core: -110
[   32.841758] ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -110
[   32.852799] qcom-pcie 1c10000.pcie: supply vdda not found, using dummy regulator
[   32.860924] qcom-pcie 1c10000.pcie: host bridge /soc@0/pcie@1c10000 ranges:
[   32.868157] qcom-pcie 1c10000.pcie:       IO 0x0034200000..0x00342fffff -> 0x0000000000
[   32.876428] qcom-pcie 1c10000.pcie:      MEM 0x0034300000..0x0035ffffff -> 0x0034300000
[   33.001705] qcom-pcie 1c10000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 1024G
[   33.111456] qcom-pcie 1c10000.pcie: PCIe Gen.3 x2 link up
[   33.117554] qcom-pcie 1c10000.pcie: PCI host bridge to bus 0004:00
[   33.124000] pci_bus 0004:00: root bus resource [bus 00-ff]
[   33.129745] pci_bus 0004:00: root bus resource [io  0x100000-0x1fffff] (bus address [0x0000-0xfffff])
[   33.139324] pci_bus 0004:00: root bus resource [mem 0x34300000-0x35ffffff]
[   33.146525] pci 0004:00:00.0: [17cb:010e] type 01 class 0x060400 PCIe Root Port
[   33.154167] pci 0004:00:00.0: BAR 0 [mem 0x00000000-0x00000fff]
[   33.160373] pci 0004:00:00.0: PCI bridge to [bus 01-ff]
[   33.165804] pci 0004:00:00.0:   bridge window [io  0x100000-0x100fff]
[   33.172482] pci 0004:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[   33.179515] pci 0004:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[   33.187622] pci 0004:00:00.0: PME# supported from D0 D3hot D3cold
[   33.195555] pci 0004:01:00.0: [17cb:0306] type 00 class 0xff0000 PCIe Endpoint
[   33.203462] pci 0004:01:00.0: BAR 0 [mem 0x00000000-0x00000fff 64bit]
[   33.210163] pci 0004:01:00.0: BAR 2 [mem 0x00000000-0x00000fff 64bit]
[   33.217379] pci 0004:01:00.0: PME# supported from D0 D3hot D3cold
[   33.223825] pci 0004:01:00.0: 15.752 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x2 link at 0004:00:00.0 (capable of 31.506 Gb/s with 16.0 GT/s PCIe x2 link)
[   33.251876] pci 0004:00:00.0: bridge window [mem 0x34300000-0x343fffff]: assigned
[   33.259599] pci 0004:00:00.0: BAR 0 [mem 0x34400000-0x34400fff]: assigned
[   33.266621] pci 0004:01:00.0: BAR 0 [mem 0x34300000-0x34300fff 64bit]: assigned
[   33.274186] pci 0004:01:00.0: BAR 2 [mem 0x34301000-0x34301fff 64bit]: assigned
[   33.281748] pci 0004:00:00.0: PCI bridge to [bus 01-ff]
[   33.287133] pci 0004:00:00.0:   bridge window [mem 0x34300000-0x343fffff]
[   33.294322] Reusing ITT for devID 0
[   33.296005] Reusing ITT for devID 0
[   33.296053] ID:1 pID:8193 vID:203
[   33.296066] IRQ203 -> 0-7 CPU1
[   33.296176] IRQ203 -> 0-7 CPU1
[   33.296240] pcieport 0004:00:00.0: PME: Signaling with IRQ 203
[   33.302538] pcieport 0004:00:00.0: AER: enabled with IRQ 203
[   33.308587] mhi-pci-generic 0004:01:00.0: MHI PCI device found: foxconn-sdx55
[   33.315945] mhi-pci-generic 0004:01:00.0: BAR 0 [mem 0x34300000-0x34300fff 64bit]: assigned
[   33.324583] mhi-pci-generic 0004:01:00.0: enabling device (0000 -> 0002)
[   33.331610] ITS: alloc 8224:8
[   33.331619] ITT 8 entries, 3 bits
[   33.331750] ID:0 pID:8224 vID:204
[   33.331756] ID:1 pID:8225 vID:205
[   33.331762] ID:2 pID:8226 vID:206
[   33.331769] ID:3 pID:8227 vID:207
[   33.331774] ID:4 pID:8228 vID:208
[   33.331791] IRQ204 -> 0-7 CPU2
[   33.331837] IRQ205 -> 0-7 CPU3
[   33.331848] IRQ206 -> 0-7 CPU4
[   33.331860] IRQ207 -> 0-7 CPU5
[   33.331872] IRQ208 -> 0-7 CPU6
[   33.332711] IRQ204 -> 0-7 CPU2
[   33.333016] IRQ205 -> 0-7 CPU3
[   33.333042] IRQ206 -> 0-7 CPU4
[   33.333066] IRQ207 -> 0-7 CPU5
[   33.333090] IRQ208 -> 0-7 CPU6
[   33.335976] mhi mhi0: Requested to power ON
[   33.340327] mhi mhi0: Power on setup success
[   54.242353] mhi-pci-generic 0004:01:00.0: failed to power up MHI controller
[   54.251547] mhi-pci-generic 0004:01:00.0: probe with driver mhi-pci-generic failed with error -110
[   54.262662] qcom-pcie 1c20000.pcie: supply vdda not found, using dummy regulator
[   54.270794] qcom-pcie 1c20000.pcie: host bridge /soc@0/pcie@1c20000 ranges:
[   54.278042] qcom-pcie 1c20000.pcie:       IO 0x003c200000..0x003c2fffff -> 0x0000000000
[   54.286340] qcom-pcie 1c20000.pcie:      MEM 0x003c300000..0x003dffffff -> 0x003c300000
[   54.409356] qcom-pcie 1c20000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 1024G
[   54.519604] qcom-pcie 1c20000.pcie: PCIe Gen.3 x4 link up
[   54.525609] qcom-pcie 1c20000.pcie: PCI host bridge to bus 0002:00
[   54.532017] pci_bus 0002:00: root bus resource [bus 00-ff]
[   54.537732] pci_bus 0002:00: root bus resource [io  0x200000-0x2fffff] (bus address [0x0000-0xfffff])
[   54.547830] pci_bus 0002:00: root bus resource [mem 0x3c300000-0x3dffffff]
[   54.555523] pci 0002:00:00.0: [17cb:010e] type 01 class 0x060400 PCIe Root Port
[   54.563629] pci 0002:00:00.0: BAR 0 [mem 0x00000000-0x00000fff]
[   54.570244] pci 0002:00:00.0: PCI bridge to [bus 01-ff]
[   54.576099] pci 0002:00:00.0:   bridge window [io  0x200000-0x200fff]
[   54.583121] pci 0002:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[   54.590473] pci 0002:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[   54.598841] pci 0002:00:00.0: PME# supported from D0 D3hot D3cold
[   54.606657] pci 0002:01:00.0: [1e0f:0001] type 00 class 0x010802 PCIe Endpoint
[   54.614458] pci 0002:01:00.0: BAR 0 [mem 0x00000000-0x00003fff 64bit]
[   54.621900] pci 0002:01:00.0: PME# supported from D0 D3hot
[   54.635232] sd 0:0:0:0: [sda] Starting disk
[   54.641117] pci 0002:00:00.0: bridge window [mem 0x3c300000-0x3c3fffff]: assigned
[   54.649086] pci 0002:00:00.0: BAR 0 [mem 0x3c400000-0x3c400fff]: assigned
[   54.656299] pci 0002:01:00.0: BAR 0 [mem 0x3c300000-0x3c303fff 64bit]: assigned
[   54.664083] pci 0002:00:00.0: PCI bridge to [bus 01-ff]
[   54.669688] pci 0002:00:00.0:   bridge window [mem 0x3c300000-0x3c3fffff]
[   54.677113] Reusing ITT for devID 0
[   54.678960] Reusing ITT for devID 0
[   54.678994] ID:2 pID:8194 vID:205
[   54.679005] IRQ205 -> 0-7 CPU2
[   54.679103] IRQ205 -> 0-7 CPU2
[   54.679123] pcieport 0002:00:00.0: PME: Signaling with IRQ 205
[   54.685994] pcieport 0002:00:00.0: AER: enabled with IRQ 205
[   54.693042] nvme nvme0: pci function 0002:01:00.0
[   54.698150] nvme 0002:01:00.0: enabling device (0000 -> 0002)
[   54.704457] Reusing ITT for devID 100
[   54.704500] ID:0 pID:8224 vID:206
[   54.704509] IRQ206 -> 0-7 CPU3
[   54.706919] IRQ206 -> 0-7 CPU3
[  115.695904] nvme nvme0: I/O tag 0 (1000) QID 0 timeout, completion polled
[  177.135829] nvme nvme0: I/O tag 1 (1001) QID 0 timeout, completion polled
[  238.575830] nvme nvme0: I/O tag 2 (1002) QID 0 timeout, completion polled
[  300.023834] nvme nvme0: I/O tag 3 (1003) QID 0 timeout, completion polled
[  300.055992] nvme nvme0: allocated 61 MiB host memory buffer.
maintainer
#84 lore raw
Quoted 43 lines · Johan Hovold
Quoted 22 lines · Marc Zyngier
Quoted 20 lines · Johan Hovold
Quoted 2 lines · Thomas Gleixner

This is version 4 of the series to convert ARM MSI handling over to per device MSI domains.

This series only showed up in linux-next last Friday and broke interrupt handling on Qualcomm platforms like sc8280xp (e.g. Lenovo ThinkPad X13s) and x1e80100 that use the GIC ITS for PCIe MSIs.

I've applied the series (21 commits from linux-next) on top of 6.10 and can confirm that the breakage is caused by commits:

	3d1c927c08fc ("irqchip/gic-v3-its: Switch platform MSI to MSI parent")
	233db05bc37f ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")

Applying the series up until the change before 3d1c927c08fc unbreaks the wifi on one machine:

	ath11k_pci 0006:01:00.0: failed to enable msi: -22
	ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -22
Quoted 7 lines
Quoted 4 lines

and backing up until the commit before 233db05bc37f makes the NVMe come up again during boot on another.

I have not tried to debug this further.

I need a few things from you though, because you're not giving much to help you (and I'm travelling, which doesn't help).

Yeah, this was just an early heads up.

Quoted 3 lines

Can you at least investigate what in ath11k_pci_alloc_msi() causes the wifi driver to be upset? Does it normally use a single MSI vector or MSI-X? How about your nVME device?

It uses multiple vectors, but now it falls back to trying to allocate a single one and even that fails with -ENOSPC:

	ath11k_pci 0006:01:00.0: ath11k_pci_alloc_msi - requesting one vector failed: -28
Quoted 9 lines

Similar for the NVMe, it uses multiple vectors normally, but now only the AER interrupts appears to be allocated for each controller and there is a GICv3 interrupt for the NVMe:

208:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0006:00:00.0   0 Edge      PCIe PME, aerdrv
212:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0004:00:00.0   0 Edge      PCIe PME, aerdrv
214:        161          0          0          0          0          0          0          0     GICv3 562 Level     nvme0q0, nvme0q1
215:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0002:00:00.0   0 Edge      PCIe PME, aerdrv

That's an indication of the driver having failed its MSI allocation and gone back to INTx signalling.

Quoted 7 lines

Next boot, after disabling PCIe controller async probing, it's an MSI-X?!:

201:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0006:00:00.0   0 Edge      PCIe PME, aerdrv
203:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0004:00:00.0   0 Edge      PCIe PME, aerdrv
205:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0002:00:00.0   0 Edge      PCIe PME, aerdrv
206:          0          0          0          0          0          0          0          0  ITS-PCI-MSIX-0002:01:00.0   0 Edge      nvme0q0

So is this issue actually tied to the async probing? Does it always work if you disable it?

Quoted 11 lines

This time ath11k vector allocation succeeded, but the driver times out eventually:

[    8.984619] ath11k_pci 0006:01:00.0: MSI vectors: 32
[   29.690841] ath11k_pci 0006:01:00.0: failed to power up mhi: -110
[   29.697136] ath11k_pci 0006:01:00.0: failed to start mhi: -110
[   29.703153] ath11k_pci 0006:01:00.0: failed to power up :-110
[   29.732144] ath11k_pci 0006:01:00.0: failed to create soc core: -110
[   29.738694] ath11k_pci 0006:01:00.0: failed to init core: -110
[   32.841758] ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -110
Quoted 5 lines
Quoted 3 lines

It would also help if you could define the DEBUG symbol at the very top of irq-gic-v3-its.c and report the debug information that the ITS driver dumps.

See below (with synchronous probing of the pcie controllers).

I don't see much going wrong there, and the ITS driver correctly dishes out interrupts. I'll take the current -next for a ride on my own HW and see what happens.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#85 lore raw
Quoted 16 lines · Marc Zyngier
Quoted 14 lines · Johan Hovold
Quoted 13 lines · Marc Zyngier
Quoted 11 lines · Johan Hovold
Quoted 2 lines · Thomas Gleixner

This is version 4 of the series to convert ARM MSI handling over to per device MSI domains.

This series only showed up in linux-next last Friday and broke interrupt handling on Qualcomm platforms like sc8280xp (e.g. Lenovo ThinkPad X13s) and x1e80100 that use the GIC ITS for PCIe MSIs.

I've applied the series (21 commits from linux-next) on top of 6.10 and can confirm that the breakage is caused by commits:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
	3d1c927c08fc ("irqchip/gic-v3-its: Switch platform MSI to MSI parent")
	233db05bc37f ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")

Applying the series up until the change before 3d1c927c08fc unbreaks the wifi on one machine:

	ath11k_pci 0006:01:00.0: failed to enable msi: -22
	ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -22

Correction, this doesn't fix the wifi, but I'm not seeing these errors with the commit before cc23d1dfc959 as the ath11k driver doesn't get this far (or doesn't probe at all).

Quoted 34 lines
Quoted 19 lines
Quoted 7 lines
Quoted 4 lines

and backing up until the commit before 233db05bc37f makes the NVMe come up again during boot on another.

I have not tried to debug this further.

I need a few things from you though, because you're not giving much to help you (and I'm travelling, which doesn't help).

Yeah, this was just an early heads up.

Quoted 3 lines

Can you at least investigate what in ath11k_pci_alloc_msi() causes the wifi driver to be upset? Does it normally use a single MSI vector or MSI-X? How about your nVME device?

It uses multiple vectors, but now it falls back to trying to allocate a single one and even that fails with -ENOSPC:

	ath11k_pci 0006:01:00.0: ath11k_pci_alloc_msi - requesting one vector failed: -28
Quoted 9 lines

Similar for the NVMe, it uses multiple vectors normally, but now only the AER interrupts appears to be allocated for each controller and there is a GICv3 interrupt for the NVMe:

208:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0006:00:00.0   0 Edge      PCIe PME, aerdrv
212:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0004:00:00.0   0 Edge      PCIe PME, aerdrv
214:        161          0          0          0          0          0          0          0     GICv3 562 Level     nvme0q0, nvme0q1
215:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0002:00:00.0   0 Edge      PCIe PME, aerdrv

That's an indication of the driver having failed its MSI allocation and gone back to INTx signalling.

Quoted 2 lines

Next boot, after disabling PCIe controller async probing, it's an MSI-X?!:

Quoted 8 lines
Quoted 5 lines
201:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0006:00:00.0   0 Edge      PCIe PME, aerdrv
203:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0004:00:00.0   0 Edge      PCIe PME, aerdrv
205:          0          0          0          0          0          0          0          0  ITS-PCI-MSI-0002:00:00.0   0 Edge      PCIe PME, aerdrv
206:          0          0          0          0          0          0          0          0  ITS-PCI-MSIX-0002:01:00.0   0 Edge      nvme0q0

So is this issue actually tied to the async probing? Does it always work if you disable it?

There seem to multiple issues here.

With the full series applied and normal async (i.e. parallel) probing of the PCIe controllers I sometimes see allocation failing with -ENOSPC (e.g. the above ath11k errors). This seems to indicate broken locking somewhere.

With synchronous probing, allocation always seems to succeed but the ath11k (and modem) drivers time out as no interrupts are received.

The NVMe driver sometimes falls back to INTx signalling and can access the drive, but often end up with an MSIX (?!) allocation and then fails to probe:

	[  132.084740] nvme nvme0: I/O tag 17 (1011) QID 0 timeout, completion polled

Johan

maintainer
#86 lore raw

[Dropping [email protected] who is now bouncing...]

Quoted 24 lines · Johan Hovold
Quoted 22 lines · Marc Zyngier
Quoted 20 lines · Johan Hovold
Quoted 19 lines · Marc Zyngier
Quoted 17 lines · Johan Hovold
Quoted 2 lines · Thomas Gleixner

This is version 4 of the series to convert ARM MSI handling over to per device MSI domains.

This series only showed up in linux-next last Friday and broke interrupt handling on Qualcomm platforms like sc8280xp (e.g. Lenovo ThinkPad X13s) and x1e80100 that use the GIC ITS for PCIe MSIs.

I've applied the series (21 commits from linux-next) on top of 6.10 and can confirm that the breakage is caused by commits:

	3d1c927c08fc ("irqchip/gic-v3-its: Switch platform MSI to MSI parent")
	233db05bc37f ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")

Applying the series up until the change before 3d1c927c08fc unbreaks the wifi on one machine:

Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
	ath11k_pci 0006:01:00.0: failed to enable msi: -22
	ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -22

Correction, this doesn't fix the wifi, but I'm not seeing these errors with the commit before cc23d1dfc959 as the ath11k driver doesn't get this far (or doesn't probe at all).

I think we need to track one thing at a time. The wifi and nvme problems seem subtly different... Which is the exact commit that breaks nvme on your machine?

[...]

Quoted 9 lines
Quoted 2 lines

So is this issue actually tied to the async probing? Does it always work if you disable it?

There seem to multiple issues here.

With the full series applied and normal async (i.e. parallel) probing of the PCIe controllers I sometimes see allocation failing with -ENOSPC (e.g. the above ath11k errors). This seems to indicate broken locking somewhere.

Your log doesn't support this theory. At least not from an ITS perspective, as it keeps dishing out INTIDs (and it is very hard to run out of IRQs with the ITS).

Quoted 9 lines

With synchronous probing, allocation always seems to succeed but the ath11k (and modem) drivers time out as no interrupts are received.

The NVMe driver sometimes falls back to INTx signalling and can access the drive, but often end up with an MSIX (?!) allocation and then fails to probe:

	[  132.084740] nvme nvme0: I/O tag 17 (1011) QID 0 timeout, completion polled

So one of my test boxes (ThunderX) fails this exact way, while another (Synquacer) is pretty happy. Still trying to understand the difference in behaviour.

How do you enforce synchronous probing?

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#87 lore raw
Quoted 19 lines · Marc Zyngier
Quoted 17 lines · Johan Hovold
Quoted 16 lines · Marc Zyngier
Quoted 14 lines · Johan Hovold
Quoted 13 lines · Marc Zyngier
Quoted 10 lines · Johan Hovold

This series only showed up in linux-next last Friday and broke interrupt handling on Qualcomm platforms like sc8280xp (e.g. Lenovo ThinkPad X13s) and x1e80100 that use the GIC ITS for PCIe MSIs.

I've applied the series (21 commits from linux-next) on top of 6.10 and can confirm that the breakage is caused by commits:

	3d1c927c08fc ("irqchip/gic-v3-its: Switch platform MSI to MSI parent")
	233db05bc37f ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Applying the series up until the change before 3d1c927c08fc unbreaks the wifi on one machine:

	ath11k_pci 0006:01:00.0: failed to enable msi: -22
	ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -22

Correction, this doesn't fix the wifi, but I'm not seeing these errors with the commit before cc23d1dfc959 as the ath11k driver doesn't get

[ This was supposed to say 3d1c927c08fc, which is the mainline hash, sorry. ]

Quoted 5 lines
Quoted 1 line

this far (or doesn't probe at all).

I think we need to track one thing at a time. The wifi and nvme problems seem subtly different... Which is the exact commit that breaks nvme on your machine?

Yeah, forget about 3d1c927c08fc for now, which may have been a red herring since we're also appear to be dealing with some sort of race and (some) symptoms keep changing from boot to boot. The only thing that for certain is that the series breaks MSI and that the NVMe breaks with commit 233db05bc37f ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]").

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

So is this issue actually tied to the async probing? Does it always work if you disable it?

There seem to multiple issues here.

Quoted 8 lines
Quoted 4 lines

With the full series applied and normal async (i.e. parallel) probing of the PCIe controllers I sometimes see allocation failing with -ENOSPC (e.g. the above ath11k errors). This seems to indicate broken locking somewhere.

Your log doesn't support this theory. At least not from an ITS perspective, as it keeps dishing out INTIDs (and it is very hard to run out of IRQs with the ITS).

The log I shared was with synchronous probing which takes parallel allocation out of the equation (and gives more readable logs) so that is expected. See below for a log with normal async probing that may give some more insight into the race as well (i.e. when ath11k allocation fails with -ENOSPC.)

Quoted 7 lines
Quoted 7 lines

With synchronous probing, allocation always seems to succeed but the ath11k (and modem) drivers time out as no interrupts are received.

The NVMe driver sometimes falls back to INTx signalling and can access the drive, but often end up with an MSIX (?!) allocation and then fails to probe:

Quoted 7 lines
Quoted 1 line
	[  132.084740] nvme nvme0: I/O tag 17 (1011) QID 0 timeout, completion polled

So one of my test boxes (ThunderX) fails this exact way, while another (Synquacer) is pretty happy. Still trying to understand the difference in behaviour.

How do you enforce synchronous probing?

I believe there is a kernel parameter for this (e.g. module.async_probe), but I just disable async probing for the Qualcomm PCIe driver I'm using:

drivers/pci/controller/dwc/pcie-qcom.c +1 -1
@@ -1684,7 +1684,7 @@ static struct platform_driver qcom_pcie_driver = {
                .name = "qcom-pcie",
                .of_match_table = qcom_pcie_match,
                .pm = &qcom_pcie_pm_ops,
-               .probe_type = PROBE_PREFER_ASYNCHRONOUS,
+               //.probe_type = PROBE_PREFER_ASYNCHRONOUS,
        },
 };
 

Johan

[    8.323957] qcom-pcie 1c00000.pcie: host bridge /soc@0/pcie@1c00000 ranges:
[    8.334800] qcom-pcie 1c00000.pcie:       IO 0x0030200000..0x00302fffff -> 0x0000000000
[    8.348124] qcom-pcie 1c00000.pcie:      MEM 0x0030300000..0x0031ffffff -> 0x0030300000
[    8.378334] qcom-pcie 1c10000.pcie: host bridge /soc@0/pcie@1c10000 ranges:
[    8.378632] qcom-pcie 1c20000.pcie: host bridge /soc@0/pcie@1c20000 ranges:
[    8.378654] qcom-pcie 1c20000.pcie:       IO 0x003c200000..0x003c2fffff -> 0x0000000000
[    8.378666] qcom-pcie 1c20000.pcie:      MEM 0x003c300000..0x003dffffff -> 0x003c300000
[    8.391084] qcom-pcie 1c10000.pcie:       IO 0x0034200000..0x00342fffff -> 0x0000000000
[    8.419252] qcom-pcie 1c10000.pcie:      MEM 0x0034300000..0x0035ffffff -> 0x0034300000
[    8.477255] qcom-pcie 1c00000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 1024G
[    8.497259] qcom-pcie 1c20000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 1024G
[    8.537258] qcom-pcie 1c10000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 1024G
[    8.583746] qcom-pcie 1c00000.pcie: PCIe Gen.2 x1 link up
[    8.590079] qcom-pcie 1c00000.pcie: PCI host bridge to bus 0006:00
[    8.596838] pci_bus 0006:00: root bus resource [bus 00-ff]
[    8.602874] pci_bus 0006:00: root bus resource [io  0x0000-0xfffff]
[    8.603809] qcom-pcie 1c20000.pcie: PCIe Gen.3 x4 link up
[    8.609322] pci_bus 0006:00: root bus resource [mem 0x30300000-0x31ffffff]
[    8.609393] pci 0006:00:00.0: [17cb:010e] type 01 class 0x060400 PCIe Root Port
[    8.615040] qcom-pcie 1c20000.pcie: PCI host bridge to bus 0002:00
[    8.621951] pci 0006:00:00.0: BAR 0 [mem 0x00000000-0x00000fff]
[    8.629452] pci_bus 0002:00: root bus resource [bus 00-ff]
[    8.629706] pci_bus 0002:00: root bus resource [io  0x100000-0x1fffff] (bus address [0x0000-0xfffff])
[    8.635822] pci 0006:00:00.0: PCI bridge to [bus 01-ff]
[    8.641903] pci_bus 0002:00: root bus resource [mem 0x3c300000-0x3dffffff]
[    8.643728] qcom-pcie 1c10000.pcie: PCIe Gen.3 x2 link up
[    8.643851] qcom-pcie 1c10000.pcie: PCI host bridge to bus 0004:00
[    8.643854] pci_bus 0004:00: root bus resource [bus 00-ff]
[    8.643857] pci_bus 0004:00: root bus resource [io  0x200000-0x2fffff] (bus address [0x0000-0xfffff])
[    8.643859] pci_bus 0004:00: root bus resource [mem 0x34300000-0x35ffffff]
[    8.643873] pci 0004:00:00.0: [17cb:010e] type 01 class 0x060400 PCIe Root Port
[    8.643881] pci 0004:00:00.0: BAR 0 [mem 0x00000000-0x00000fff]
[    8.643890] pci 0004:00:00.0: PCI bridge to [bus 01-ff]
[    8.643894] pci 0004:00:00.0:   bridge window [io  0x200000-0x200fff]
[    8.643897] pci 0004:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    8.643903] pci 0004:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    8.643982] pci 0004:00:00.0: PME# supported from D0 D3hot D3cold
[    8.644933] pci 0004:01:00.0: [17cb:0306] type 00 class 0xff0000 PCIe Endpoint
[    8.645012] pci 0004:01:00.0: BAR 0 [mem 0x00000000-0x00000fff 64bit]
[    8.645063] pci 0004:01:00.0: BAR 2 [mem 0x00000000-0x00000fff 64bit]
[    8.645614] pci 0004:01:00.0: PME# supported from D0 D3hot D3cold
[    8.645768] pci 0004:01:00.0: 15.752 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x2 link at 0004:00:00.0 (capable of 31.506 Gb/s with 16.0 GT/s PCIe x2 link)
[    8.647523] pci 0006:00:00.0:   bridge window [io  0x0000-0x0fff]
[    8.659851] pci 0004:00:00.0: bridge window [mem 0x34300000-0x343fffff]: assigned
[    8.659862] pci 0002:00:00.0: [17cb:010e] type 01 class 0x060400 PCIe Root Port
[    8.659873] pci 0002:00:00.0: BAR 0 [mem 0x00000000-0x00000fff]
[    8.659883] pci 0002:00:00.0: PCI bridge to [bus 01-ff]
[    8.659889] pci 0002:00:00.0:   bridge window [io  0x100000-0x100fff]
[    8.659893] pci 0002:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    8.659900] pci 0002:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    8.659962] pci 0002:00:00.0: PME# supported from D0 D3hot D3cold
[    8.661170] pci 0002:01:00.0: [1e0f:0001] type 00 class 0x010802 PCIe Endpoint
[    8.661259] pci 0002:01:00.0: BAR 0 [mem 0x00000000-0x00003fff 64bit]
[    8.661825] pci 0002:01:00.0: PME# supported from D0 D3hot
[    8.662365] pci 0006:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    8.669410] pci 0004:00:00.0: BAR 0 [mem 0x34400000-0x34400fff]: assigned
[    8.671873] pci 0002:00:00.0: bridge window [mem 0x3c300000-0x3c3fffff]: assigned
[    8.671879] pci 0002:00:00.0: BAR 0 [mem 0x3c400000-0x3c400fff]: assigned
[    8.671887] pci 0002:01:00.0: BAR 0 [mem 0x3c300000-0x3c303fff 64bit]: assigned
[    8.671931] pci 0002:00:00.0: PCI bridge to [bus 01-ff]
[    8.671936] pci 0002:00:00.0:   bridge window [mem 0x3c300000-0x3c3fffff]
[    8.672719] ITS: alloc 8192:32
[    8.672737] ITT 32 entries, 5 bits
[    8.674420] ID:0 pID:8192 vID:196
[    8.674436] ID:1 pID:8193 vID:197
[    8.674444] ID:2 pID:8194 vID:198
[    8.674452] ID:3 pID:8195 vID:199
[    8.674461] ID:4 pID:8196 vID:200
[    8.674469] ID:5 pID:8197 vID:201
[    8.674476] ID:6 pID:8198 vID:202
[    8.674485] ID:7 pID:8199 vID:203
[    8.674493] ID:8 pID:8200 vID:204
[    8.674501] ID:9 pID:8201 vID:205
[    8.674508] ID:10 pID:8202 vID:206
[    8.674517] ID:11 pID:8203 vID:207
[    8.674525] ID:12 pID:8204 vID:208
[    8.674532] ID:13 pID:8205 vID:209
[    8.674540] ID:14 pID:8206 vID:210
[    8.674548] ID:15 pID:8207 vID:211
[    8.674556] ID:16 pID:8208 vID:212
[    8.674564] ID:17 pID:8209 vID:213
[    8.674572] ID:18 pID:8210 vID:214
[    8.674580] ID:19 pID:8211 vID:215
[    8.674588] ID:20 pID:8212 vID:216
[    8.674596] ID:21 pID:8213 vID:217
[    8.674604] ID:22 pID:8214 vID:218
[    8.674612] ID:23 pID:8215 vID:219
[    8.674620] ID:24 pID:8216 vID:220
[    8.674628] ID:25 pID:8217 vID:221
[    8.674636] ID:26 pID:8218 vID:222
[    8.674643] ID:27 pID:8219 vID:223
[    8.674651] ID:28 pID:8220 vID:224
[    8.674659] ID:29 pID:8221 vID:225
[    8.674667] ID:30 pID:8222 vID:226
[    8.674675] ID:31 pID:8223 vID:227
[    8.674824] IRQ196 -> 0-7 CPU0
[    8.674850] IRQ197 -> 0-7 CPU1
[    8.674864] IRQ198 -> 0-7 CPU2
[    8.674878] IRQ199 -> 0-7 CPU3
[    8.674891] IRQ200 -> 0-7 CPU4
[    8.674905] IRQ201 -> 0-7 CPU5
[    8.674918] IRQ202 -> 0-7 CPU6
[    8.674932] IRQ203 -> 0-7 CPU7
[    8.674945] IRQ204 -> 0-7 CPU0
[    8.674951] pci 0006:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    8.675005] pci 0006:00:00.0: PME# supported from D0 D3hot D3cold
[    8.675887] pci 0006:01:00.0: [17cb:1103] type 00 class 0x028000 PCIe Endpoint
[    8.675983] pci 0006:01:00.0: BAR 0 [mem 0x00000000-0x001fffff 64bit]
[    8.676613] pci 0006:01:00.0: PME# supported from D0 D3hot D3cold
[    8.676779] pci 0006:01:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x1 link at 0006:00:00.0 (capable of 7.876 Gb/s with 8.0 GT/s PCIe x1 link)
[    8.681292] pci 0004:01:00.0: BAR 0 [mem 0x34300000-0x34300fff 64bit]: assigned
[    8.681332] pci 0004:01:00.0: BAR 2 [mem 0x34301000-0x34301fff 64bit]: assigned
[    8.686968] IRQ205 -> 0-7 CPU1
[    8.691823] pci 0006:00:00.0: bridge window [mem 0x30400000-0x305fffff]: assigned
[    8.691825] pci 0006:00:00.0: BAR 0 [mem 0x30300000-0x30300fff]: assigned
[    8.691829] pci 0006:01:00.0: BAR 0 [mem 0x30400000-0x305fffff 64bit]: assigned
[    8.691877] pci 0006:00:00.0: PCI bridge to [bus 01-ff]
[    8.691880] pci 0006:00:00.0:   bridge window [mem 0x30400000-0x305fffff]
[    8.692011] Reusing ITT for devID 0
[    8.693668] Reusing ITT for devID 0
[    8.693871] pcieport 0006:00:00.0: PME: Signaling with IRQ 228
[    8.694116] pcieport 0006:00:00.0: AER: enabled with IRQ 228
[    8.696453] pci 0004:00:00.0: PCI bridge to [bus 01-ff]
[    8.703760] IRQ206 -> 0-7 CPU2
[    8.710986] pci 0004:00:00.0:   bridge window [mem 0x34300000-0x343fffff]
[    8.711136] Reusing ITT for devID 0
[    8.717093] IRQ207 -> 0-7 CPU3
[    8.723889] Reusing ITT for devID 0
[    8.729600] IRQ208 -> 0-7 CPU4
[    8.736507] pcieport 0004:00:00.0: PME: Signaling with IRQ 229
[    8.744261] IRQ209 -> 0-7 CPU5
[    8.750757] pcieport 0004:00:00.0: AER: enabled with IRQ 229
[    8.758038] IRQ210 -> 0-7 CPU6
[    9.071793] IRQ211 -> 0-7 CPU7
[    9.071807] IRQ212 -> 0-7 CPU0
[    9.071819] IRQ213 -> 0-7 CPU1
[    9.071831] IRQ214 -> 0-7 CPU2
[    9.071842] IRQ215 -> 0-7 CPU3
[    9.071852] IRQ216 -> 0-7 CPU4
[    9.071863] IRQ217 -> 0-7 CPU5
[    9.071875] IRQ218 -> 0-7 CPU6
[    9.071886] IRQ219 -> 0-7 CPU7
[    9.071897] IRQ220 -> 0-7 CPU0
[    9.071907] IRQ221 -> 0-7 CPU1
[    9.071920] IRQ222 -> 0-7 CPU2
[    9.071930] IRQ223 -> 0-7 CPU3
[    9.071941] IRQ224 -> 0-7 CPU4
[    9.071952] IRQ225 -> 0-7 CPU5
[    9.071962] IRQ226 -> 0-7 CPU6
[    9.071973] IRQ227 -> 0-7 CPU7
[    9.073568] Reusing ITT for devID 0
[    9.073607] ID:0 pID:8192 vID:196
[    9.073618] IRQ196 -> 0-7 CPU0
[    9.073717] IRQ196 -> 0-7 CPU0
[    9.073737] pcieport 0002:00:00.0: PME: Signaling with IRQ 196
[    9.086532] pcieport 0002:00:00.0: AER: enabled with IRQ 196
[    9.102057] mhi-pci-generic 0004:01:00.0: MHI PCI device found: foxconn-sdx55
[    9.109830] mhi-pci-generic 0004:01:00.0: BAR 0 [mem 0x34300000-0x34300fff 64bit]: assigned
[    9.119027] mhi-pci-generic 0004:01:00.0: enabling device (0000 -> 0002)
[    9.127271] ITS: alloc 8224:8
[    9.141500] ITT 8 entries, 3 bits
[    9.144502] ID:0 pID:8224 vID:198
[    9.144597] ID:1 pID:8225 vID:199
[    9.144605] ID:2 pID:8226 vID:200
[    9.144612] ID:3 pID:8227 vID:201
[    9.144619] ID:4 pID:8228 vID:202
[    9.144689] IRQ198 -> 0-7 CPU1
[    9.144888] IRQ199 -> 0-7 CPU2
[    9.144901] IRQ200 -> 0-7 CPU3
[    9.144914] IRQ201 -> 0-7 CPU4
[    9.144927] IRQ202 -> 0-7 CPU5
[    9.151264] IRQ198 -> 0-7 CPU1
[    9.151479] IRQ199 -> 0-7 CPU2
[    9.151673] IRQ200 -> 0-7 CPU3
[    9.151849] IRQ201 -> 0-7 CPU4
[    9.152056] IRQ202 -> 0-7 CPU5
[    9.159972] mhi mhi0: Requested to power ON
[    9.165275] mhi mhi0: Power on setup success
[    9.279951] ath11k_pci 0006:01:00.0: BAR 0 [mem 0x30400000-0x305fffff 64bit]: assigned
[    9.288208] ath11k_pci 0006:01:00.0: enabling device (0000 -> 0002)
[    9.301708] nvme nvme0: pci function 0002:01:00.0
[    9.307052] Reusing ITT for devID 100
[    9.315457] nvme 0002:01:00.0: enabling device (0000 -> 0002)
[    9.326554] Reusing ITT for devID 100
[    9.336332] ath11k_pci 0006:01:00.0: ath11k_pci_alloc_msi - requesting one vector failed: -28
[    9.344362] Reusing ITT for devID 100
[    9.351639] ath11k_pci 0006:01:00.0: failed to enable msi: -22
[    9.351866] ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -22
[    9.360327] Reusing ITT for devID 100
[    9.654429] nvme nvme0: allocated 61 MiB host memory buffer.
[    9.814664] Reusing ITT for devID 100
[    9.815000] Reusing ITT for devID 100
[    9.815553] Reusing ITT for devID 100
[    9.843417] nvme nvme0: 1/0/0 default/read/poll queues
[    9.875782]  nvme0n1: p1
[   29.666877] mhi-pci-generic 0004:01:00.0: failed to power up MHI controller
[   29.681492] mhi-pci-generic 0004:01:00.0: probe with driver mhi-pci-generic failed with error -110
maintainer
#88 lore raw
Quoted 60 lines · Johan Hovold
Quoted 19 lines · Marc Zyngier
Quoted 17 lines · Johan Hovold
Quoted 16 lines · Marc Zyngier
Quoted 14 lines · Johan Hovold
Quoted 13 lines · Marc Zyngier
Quoted 10 lines · Johan Hovold

This series only showed up in linux-next last Friday and broke interrupt handling on Qualcomm platforms like sc8280xp (e.g. Lenovo ThinkPad X13s) and x1e80100 that use the GIC ITS for PCIe MSIs.

I've applied the series (21 commits from linux-next) on top of 6.10 and can confirm that the breakage is caused by commits:

	3d1c927c08fc ("irqchip/gic-v3-its: Switch platform MSI to MSI parent")
	233db05bc37f ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Applying the series up until the change before 3d1c927c08fc unbreaks the wifi on one machine:

	ath11k_pci 0006:01:00.0: failed to enable msi: -22
	ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -22

Correction, this doesn't fix the wifi, but I'm not seeing these errors with the commit before cc23d1dfc959 as the ath11k driver doesn't get

[ This was supposed to say 3d1c927c08fc, which is the mainline hash, sorry. ]

Quoted 5 lines
Quoted 1 line

this far (or doesn't probe at all).

I think we need to track one thing at a time. The wifi and nvme problems seem subtly different... Which is the exact commit that breaks nvme on your machine?

Yeah, forget about 3d1c927c08fc for now, which may have been a red herring since we're also appear to be dealing with some sort of race and (some) symptoms keep changing from boot to boot. The only thing that for certain is that the series breaks MSI and that the NVMe breaks with commit 233db05bc37f ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]").

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

So is this issue actually tied to the async probing? Does it always work if you disable it?

There seem to multiple issues here.

Quoted 8 lines
Quoted 4 lines

With the full series applied and normal async (i.e. parallel) probing of the PCIe controllers I sometimes see allocation failing with -ENOSPC (e.g. the above ath11k errors). This seems to indicate broken locking somewhere.

Your log doesn't support this theory. At least not from an ITS perspective, as it keeps dishing out INTIDs (and it is very hard to run out of IRQs with the ITS).

Quoted 5 lines

The log I shared was with synchronous probing which takes parallel allocation out of the equation (and gives more readable logs) so that is expected. See below for a log with normal async probing that may give some more insight into the race as well (i.e. when ath11k allocation fails with -ENOSPC.)

Huh, this log is actually pointing at something very ugly. Not a race, but some horrible ID confusion. See below.

Quoted 14 lines
Quoted 14 lines
Quoted 9 lines

With synchronous probing, allocation always seems to succeed but the ath11k (and modem) drivers time out as no interrupts are received.

The NVMe driver sometimes falls back to INTx signalling and can access the drive, but often end up with an MSIX (?!) allocation and then fails to probe:

	[  132.084740] nvme nvme0: I/O tag 17 (1011) QID 0 timeout, completion polled

So one of my test boxes (ThunderX) fails this exact way, while another (Synquacer) is pretty happy. Still trying to understand the difference in behaviour.

Quoted 5 lines
Quoted 1 line

How do you enforce synchronous probing?

I believe there is a kernel parameter for this (e.g. module.async_probe), but I just disable async probing for the Qualcomm PCIe driver I'm using:

I had tried this module parameter, but it didn't change anything on my end.

drivers/pci/controller/dwc/pcie-qcom.c static struct platform_driver qcom_pcie_driver = { · line 1689

Quoted 11 lines
drivers/pci/controller/dwc/pcie-qcom.c +1 -1
@@ -1684,7 +1684,7 @@ static struct platform_driver qcom_pcie_driver = {
                .name = "qcom-pcie",
                .of_match_table = qcom_pcie_match,
                .pm = &qcom_pcie_pm_ops,
-               .probe_type = PROBE_PREFER_ASYNCHRONOUS,
+               //.probe_type = PROBE_PREFER_ASYNCHRONOUS,
        },
 };

I'll have a look whether the TX1 PCIe driver uses this. It's positively ancient, so I wouldn't bet that it has been touched significantly in the past 5 years.

[...]

Quoted 2 lines
[    8.692011] Reusing ITT for devID 0
[    8.693668] Reusing ITT for devID 0

This is really odd. It indicates that you have several devices sharing the same DeviceID, which I seriously doubt it is the case in a laptop. Do you have any non-transparent bridge here? lspci would help.

Quoted 6 lines
[    8.693871] pcieport 0006:00:00.0: PME: Signaling with IRQ 228
[    8.694116] pcieport 0006:00:00.0: AER: enabled with IRQ 228
[    8.696453] pci 0004:00:00.0: PCI bridge to [bus 01-ff]
[    8.703760] IRQ206 -> 0-7 CPU2
[    8.710986] pci 0004:00:00.0:   bridge window [mem 0x34300000-0x343fffff]
[    8.711136] Reusing ITT for devID 0

Where is the bus number gone?

Quoted 49 lines
[    8.717093] IRQ207 -> 0-7 CPU3
[    8.723889] Reusing ITT for devID 0
[    8.729600] IRQ208 -> 0-7 CPU4
[    8.736507] pcieport 0004:00:00.0: PME: Signaling with IRQ 229
[    8.744261] IRQ209 -> 0-7 CPU5
[    8.750757] pcieport 0004:00:00.0: AER: enabled with IRQ 229
[    8.758038] IRQ210 -> 0-7 CPU6
[    9.071793] IRQ211 -> 0-7 CPU7
[    9.071807] IRQ212 -> 0-7 CPU0
[    9.071819] IRQ213 -> 0-7 CPU1
[    9.071831] IRQ214 -> 0-7 CPU2
[    9.071842] IRQ215 -> 0-7 CPU3
[    9.071852] IRQ216 -> 0-7 CPU4
[    9.071863] IRQ217 -> 0-7 CPU5
[    9.071875] IRQ218 -> 0-7 CPU6
[    9.071886] IRQ219 -> 0-7 CPU7
[    9.071897] IRQ220 -> 0-7 CPU0
[    9.071907] IRQ221 -> 0-7 CPU1
[    9.071920] IRQ222 -> 0-7 CPU2
[    9.071930] IRQ223 -> 0-7 CPU3
[    9.071941] IRQ224 -> 0-7 CPU4
[    9.071952] IRQ225 -> 0-7 CPU5
[    9.071962] IRQ226 -> 0-7 CPU6
[    9.071973] IRQ227 -> 0-7 CPU7
[    9.073568] Reusing ITT for devID 0
[    9.073607] ID:0 pID:8192 vID:196
[    9.073618] IRQ196 -> 0-7 CPU0
[    9.073717] IRQ196 -> 0-7 CPU0
[    9.073737] pcieport 0002:00:00.0: PME: Signaling with IRQ 196
[    9.086532] pcieport 0002:00:00.0: AER: enabled with IRQ 196
[    9.102057] mhi-pci-generic 0004:01:00.0: MHI PCI device found: foxconn-sdx55
[    9.109830] mhi-pci-generic 0004:01:00.0: BAR 0 [mem 0x34300000-0x34300fff 64bit]: assigned
[    9.119027] mhi-pci-generic 0004:01:00.0: enabling device (0000 -> 0002)
[    9.127271] ITS: alloc 8224:8
[    9.141500] ITT 8 entries, 3 bits
[    9.144502] ID:0 pID:8224 vID:198
[    9.144597] ID:1 pID:8225 vID:199
[    9.144605] ID:2 pID:8226 vID:200
[    9.144612] ID:3 pID:8227 vID:201
[    9.144619] ID:4 pID:8228 vID:202
[    9.144689] IRQ198 -> 0-7 CPU1
[    9.144888] IRQ199 -> 0-7 CPU2
[    9.144901] IRQ200 -> 0-7 CPU3
[    9.144914] IRQ201 -> 0-7 CPU4
[    9.144927] IRQ202 -> 0-7 CPU5
[    9.151264] IRQ198 -> 0-7 CPU1
[    9.151479] IRQ199 -> 0-7 CPU2
[    9.151673] IRQ200 -> 0-7 CPU3
[    9.151849] IRQ201 -> 0-7 CPU4
Quoted 8 lines
[    9.152056] IRQ202 -> 0-7 CPU5
[    9.159972] mhi mhi0: Requested to power ON
[    9.165275] mhi mhi0: Power on setup success
[    9.279951] ath11k_pci 0006:01:00.0: BAR 0 [mem 0x30400000-0x305fffff 64bit]: assigned
[    9.288208] ath11k_pci 0006:01:00.0: enabling device (0000 -> 0002)
[    9.301708] nvme nvme0: pci function 0002:01:00.0
[    9.307052] Reusing ITT for devID 100
[    9.315457] nvme 0002:01:00.0: enabling device (0000 -> 0002)

This is device 0002:01:00.0...

Quoted 1 line
[    9.326554] Reusing ITT for devID 100

... seen as device 0000:01:00.0. WTF???

Quoted 1 line
[    9.336332] ath11k_pci 0006:01:00.0: ath11k_pci_alloc_msi - requesting one vector failed: -28

I'm starting to suspect that the new code doesn't carry all the required bits for the DevID, and that we end-up trying to allocated interrupts from the pool allocated to another device, which can never be a good thing, and would explain why everything dies a painful death.

Can you run the same trace with the whole thing reverted? I think we're on something here.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#89 lore raw
Quoted 7 lines · Marc Zyngier
Quoted 3 lines · Johan Hovold

I believe there is a kernel parameter for this (e.g. module.async_probe), but I just disable async probing for the Qualcomm PCIe driver I'm using:

Quoted 6 lines

I had tried this module parameter, but it didn't change anything on my end.

I'll have a look whether the TX1 PCIe driver uses this. It's positively ancient, so I wouldn't bet that it has been touched significantly in the past 5 years.

Perhaps async probing just changes the symptoms, the NVMe and wifi doesn't work in either case.

Quoted 6 lines
Quoted 2 lines
[    8.692011] Reusing ITT for devID 0
[    8.693668] Reusing ITT for devID 0

This is really odd. It indicates that you have several devices sharing the same DeviceID, which I seriously doubt it is the case in a laptop. Do you have any non-transparent bridge here? lspci would help.

Yeah, and these messages do not show up without the series (see log below). They are there in the previous synchronous log however.

0002:00:00.0 PCI bridge: Qualcomm Technologies, Inc SC8280XP PCI Express Root Port 0002:01:00.0 Non-Volatile memory controller: KIOXIA Corporation NVMe SSD Controller BG4 (DRAM-less) 0004:00:00.0 PCI bridge: Qualcomm Technologies, Inc SC8280XP PCI Express Root Port 0004:01:00.0 Unassigned class [ff00]: Qualcomm Technologies, Inc SDX55 [Snapdragon X55 5G] 0006:00:00.0 PCI bridge: Qualcomm Technologies, Inc SC8280XP PCI Express Root Port 0006:01:00.0 Network controller: Qualcomm Technologies, Inc QCNFA765 Wireless Network Adapter (rev 01)

Quoted 8 lines

I'm starting to suspect that the new code doesn't carry all the required bits for the DevID, and that we end-up trying to allocated interrupts from the pool allocated to another device, which can never be a good thing, and would explain why everything dies a painful death.

Can you run the same trace with the whole thing reverted? I think we're on something here.

See below, using normal asynchronous probing like the previous log.

Johan

[    8.129424] qcom-pcie 1c10000.pcie: host bridge /soc@0/pcie@1c10000 ranges:
[    8.136886] qcom-pcie 1c10000.pcie:       IO 0x0034200000..0x00342fffff -> 0x0000000000
[    8.145351] qcom-pcie 1c00000.pcie: host bridge /soc@0/pcie@1c00000 ranges:
[    8.145372] qcom-pcie 1c10000.pcie:      MEM 0x0034300000..0x0035ffffff -> 0x0034300000
[    8.146042] qcom-pcie 1c20000.pcie: host bridge /soc@0/pcie@1c20000 ranges:
[    8.146063] qcom-pcie 1c20000.pcie:       IO 0x003c200000..0x003c2fffff -> 0x0000000000
[    8.146073] qcom-pcie 1c20000.pcie:      MEM 0x003c300000..0x003dffffff -> 0x003c300000
[    8.152546] qcom-pcie 1c00000.pcie:       IO 0x0030200000..0x00302fffff -> 0x0000000000
[    8.176372] qcom-pcie 1c00000.pcie:      MEM 0x0030300000..0x0031ffffff -> 0x0030300000
[    8.266560] qcom-pcie 1c20000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 1024G
[    8.298587] qcom-pcie 1c10000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 1024G
[    8.318753] qcom-pcie 1c00000.pcie: iATU: unroll T, 8 ob, 8 ib, align 4K, limit 1024G
[    8.377720] qcom-pcie 1c20000.pcie: PCIe Gen.3 x4 link up
[    8.384650] qcom-pcie 1c20000.pcie: PCI host bridge to bus 0002:00
[    8.392099] pci_bus 0002:00: root bus resource [bus 00-ff]
[    8.398766] pci_bus 0002:00: root bus resource [io  0x100000-0x1fffff] (bus address [0x0000-0xfffff])
[    8.405033] qcom-pcie 1c10000.pcie: PCIe Gen.3 x2 link up
[    8.408250] pci_bus 0002:00: root bus resource [mem 0x3c300000-0x3dffffff]
[    8.413899] qcom-pcie 1c10000.pcie: PCI host bridge to bus 0004:00
[    8.420959] pci 0002:00:00.0: [17cb:010e] type 01 class 0x060400 PCIe Root Port
[    8.427201] pci_bus 0004:00: root bus resource [bus 00-ff]
[    8.427204] pci_bus 0004:00: root bus resource [io  0x0000-0xfffff]
[    8.427206] pci_bus 0004:00: root bus resource [mem 0x34300000-0x35ffffff]
[    8.427219] pci 0004:00:00.0: [17cb:010e] type 01 class 0x060400 PCIe Root Port
[    8.430158] qcom-pcie 1c00000.pcie: PCIe Gen.2 x1 link up
[    8.430263] qcom-pcie 1c00000.pcie: PCI host bridge to bus 0006:00
[    8.430266] pci_bus 0006:00: root bus resource [bus 00-ff]
[    8.430269] pci_bus 0006:00: root bus resource [io  0x200000-0x2fffff] (bus address [0x0000-0xfffff])
[    8.430271] pci_bus 0006:00: root bus resource [mem 0x30300000-0x31ffffff]
[    8.430285] pci 0006:00:00.0: [17cb:010e] type 01 class 0x060400 PCIe Root Port
[    8.430297] pci 0006:00:00.0: BAR 0 [mem 0x00000000-0x00000fff]
[    8.430307] pci 0006:00:00.0: PCI bridge to [bus 01-ff]
[    8.430313] pci 0006:00:00.0:   bridge window [io  0x200000-0x200fff]
[    8.430317] pci 0006:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    8.430324] pci 0006:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    8.430414] pci 0006:00:00.0: PME# supported from D0 D3hot D3cold
[    8.431430] pci 0006:01:00.0: [17cb:1103] type 00 class 0x028000 PCIe Endpoint
[    8.431526] pci 0006:01:00.0: BAR 0 [mem 0x00000000-0x001fffff 64bit]
[    8.432154] pci 0006:01:00.0: PME# supported from D0 D3hot D3cold
[    8.432320] pci 0006:01:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5.0 GT/s PCIe x1 link at 0006:00:00.0 (capable of 7.876 Gb/s with 8.0 GT/s PCIe x1 link)
[    8.434723] pci 0002:00:00.0: BAR 0 [mem 0x00000000-0x00000fff]
[    8.440358] pci 0004:00:00.0: BAR 0 [mem 0x00000000-0x00000fff]
[    8.445157] pci 0006:00:00.0: bridge window [mem 0x30400000-0x305fffff]: assigned
[    8.445160] pci 0006:00:00.0: BAR 0 [mem 0x30300000-0x30300fff]: assigned
[    8.445163] pci 0006:01:00.0: BAR 0 [mem 0x30400000-0x305fffff 64bit]: assigned
[    8.445211] pci 0006:00:00.0: PCI bridge to [bus 01-ff]
[    8.445214] pci 0006:00:00.0:   bridge window [mem 0x30400000-0x305fffff]
[    8.445526] ITS: alloc 8192:32
[    8.445537] ITT 32 entries, 5 bits
[    8.446675] ID:0 pID:8192 vID:196
[    8.446697] ID:1 pID:8193 vID:197
[    8.446702] ID:2 pID:8194 vID:198
[    8.446707] ID:3 pID:8195 vID:199
[    8.446712] ID:4 pID:8196 vID:200
[    8.446718] ID:5 pID:8197 vID:201
[    8.446722] ID:6 pID:8198 vID:202
[    8.446727] ID:7 pID:8199 vID:203
[    8.446732] ID:8 pID:8200 vID:204
[    8.446738] ID:9 pID:8201 vID:205
[    8.446743] ID:10 pID:8202 vID:206
[    8.446748] ID:11 pID:8203 vID:207
[    8.446753] ID:12 pID:8204 vID:208
[    8.446758] ID:13 pID:8205 vID:209
[    8.446763] ID:14 pID:8206 vID:210
[    8.446768] ID:15 pID:8207 vID:211
[    8.446773] ID:16 pID:8208 vID:212
[    8.446777] ID:17 pID:8209 vID:213
[    8.446783] ID:18 pID:8210 vID:214
[    8.446788] ID:19 pID:8211 vID:215
[    8.446805] pci 0002:00:00.0: PCI bridge to [bus 01-ff]
[    8.446812] pci 0002:00:00.0:   bridge window [io  0x100000-0x100fff]
[    8.446817] pci 0002:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    8.446827] pci 0002:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    8.446899] pci 0002:00:00.0: PME# supported from D0 D3hot D3cold
[    8.448399] pci 0002:01:00.0: [1e0f:0001] type 00 class 0x010802 PCIe Endpoint
[    8.448489] pci 0002:01:00.0: BAR 0 [mem 0x00000000-0x00003fff 64bit]
[    8.449076] pci 0002:01:00.0: PME# supported from D0 D3hot
[    8.453855] pci 0004:00:00.0: PCI bridge to [bus 01-ff]
[    8.453860] pci 0004:00:00.0:   bridge window [io  0x0000-0x0fff]
[    8.461133] pci 0002:00:00.0: bridge window [mem 0x3c300000-0x3c3fffff]: assigned
[    8.461137] pci 0002:00:00.0: BAR 0 [mem 0x3c400000-0x3c400fff]: assigned
[    8.461141] pci 0002:01:00.0: BAR 0 [mem 0x3c300000-0x3c303fff 64bit]: assigned
[    8.461182] pci 0002:00:00.0: PCI bridge to [bus 01-ff]
[    8.461185] pci 0002:00:00.0:   bridge window [mem 0x3c300000-0x3c3fffff]
[    8.461378] ID:20 pID:8212 vID:216
[    8.466916] pci 0004:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    8.473265] ID:21 pID:8213 vID:217
[    8.478893] pci 0004:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    8.488351] ID:22 pID:8214 vID:218
[    8.495446] pci 0004:00:00.0: PME# supported from D0 D3hot D3cold
[    8.502905] ID:23 pID:8215 vID:219
[    8.509868] pci 0004:01:00.0: [17cb:0306] type 00 class 0xff0000 PCIe Endpoint
[    8.514345] ID:24 pID:8216 vID:220
[    8.521029] pci 0004:01:00.0: BAR 0 [mem 0x00000000-0x00000fff 64bit]
[    8.527916] ID:25 pID:8217 vID:221
[    8.535900] pci 0004:01:00.0: BAR 2 [mem 0x00000000-0x00000fff 64bit]
[    8.542116] ID:26 pID:8218 vID:222
[    8.550074] pci 0004:01:00.0: PME# supported from D0 D3hot D3cold
[    8.556138] ID:27 pID:8219 vID:223
[    8.562538] pci 0004:01:00.0: 15.752 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x2 link at 0004:00:00.0 (capable of 31.506 Gb/s with 16.0 GT/s PCIe x2 link)
[    8.577637] ID:28 pID:8220 vID:224
[    8.597112] pci 0004:00:00.0: bridge window [mem 0x34300000-0x343fffff]: assigned
[    8.597753] ID:29 pID:8221 vID:225
[    8.604711] pci 0004:00:00.0: BAR 0 [mem 0x34400000-0x34400fff]: assigned
[    8.612214] ID:30 pID:8222 vID:226
[    8.617572] pci 0004:01:00.0: BAR 0 [mem 0x34300000-0x34300fff 64bit]: assigned
[    8.624536] ID:31 pID:8223 vID:227
[    8.624836] pci 0004:01:00.0: BAR 2 [mem 0x34301000-0x34301fff 64bit]: assigned
[    8.625174] IRQ196 -> 0-7 CPU0
[    8.625221] ITS: alloc 8224:32
[    8.625230] ITT 32 entries, 5 bits
[    8.625370] pci 0004:00:00.0: PCI bridge to [bus 01-ff]
[    8.625633] IRQ197 -> 0-7 CPU1
[    8.625888] pci 0004:00:00.0:   bridge window [mem 0x34300000-0x343fffff]
[    8.626014] ID:0 pID:8224 vID:229
[    8.626020] ID:1 pID:8225 vID:230
[    8.626025] ID:2 pID:8226 vID:231
[    8.626031] ID:3 pID:8227 vID:232
[    8.626036] ID:4 pID:8228 vID:233
[    8.626041] ID:5 pID:8229 vID:234
[    8.626046] ID:6 pID:8230 vID:235
[    8.626051] ID:7 pID:8231 vID:236
[    8.626056] ID:8 pID:8232 vID:237
[    8.626061] ID:9 pID:8233 vID:238
[    8.626066] ID:10 pID:8234 vID:239
[    8.626071] ID:11 pID:8235 vID:240
[    8.626076] ID:12 pID:8236 vID:241
[    8.626081] ID:13 pID:8237 vID:242
[    8.626086] ID:14 pID:8238 vID:243
[    8.626092] ID:15 pID:8239 vID:244
[    8.626097] ID:16 pID:8240 vID:245
[    8.626102] ID:17 pID:8241 vID:246
[    8.626107] ID:18 pID:8242 vID:247
[    8.626112] ID:19 pID:8243 vID:248
[    8.626117] ID:20 pID:8244 vID:249
[    8.626122] ID:21 pID:8245 vID:250
[    8.626127] ID:22 pID:8246 vID:251
[    8.626132] ID:23 pID:8247 vID:252
[    8.626137] ID:24 pID:8248 vID:253
[    8.626143] ID:25 pID:8249 vID:254
[    8.626148] ID:26 pID:8250 vID:255
[    8.626153] ID:27 pID:8251 vID:256
[    8.626158] ID:28 pID:8252 vID:257
[    8.626166] IRQ198 -> 0-7 CPU2
[    8.626177] IRQ199 -> 0-7 CPU3
[    8.626188] IRQ200 -> 0-7 CPU4
[    8.626199] IRQ201 -> 0-7 CPU5
[    8.626210] IRQ202 -> 0-7 CPU6
[    8.626221] IRQ203 -> 0-7 CPU7
[    8.626232] IRQ204 -> 0-7 CPU0
[    8.626243] IRQ205 -> 0-7 CPU1
[    8.626254] IRQ206 -> 0-7 CPU2
[    8.626264] IRQ207 -> 0-7 CPU3
[    8.626275] IRQ208 -> 0-7 CPU4
[    8.626286] IRQ209 -> 0-7 CPU5
[    8.626297] IRQ210 -> 0-7 CPU6
[    8.626308] IRQ211 -> 0-7 CPU7
[    8.626319] IRQ212 -> 0-7 CPU0
[    8.626330] IRQ213 -> 0-7 CPU1
[    8.626341] IRQ214 -> 0-7 CPU2
[    8.626352] IRQ215 -> 0-7 CPU3
[    8.626363] IRQ216 -> 0-7 CPU4
[    8.626374] IRQ217 -> 0-7 CPU5
[    8.626385] IRQ218 -> 0-7 CPU6
[    8.626396] IRQ219 -> 0-7 CPU7
[    8.626407] IRQ220 -> 0-7 CPU0
[    8.626418] IRQ221 -> 0-7 CPU1
[    8.626429] IRQ222 -> 0-7 CPU2
[    8.626704] ID:29 pID:8253 vID:258
[    8.626965] IRQ223 -> 0-7 CPU3
[    8.627214] ID:30 pID:8254 vID:259
[    8.627467] IRQ224 -> 0-7 CPU4
[    8.627722] ID:31 pID:8255 vID:260
[    8.627977] IRQ225 -> 0-7 CPU5
[    8.628312] IRQ229 -> 0-7 CPU5
[    8.628372] ITS: alloc 8256:32
[    8.628380] ITT 32 entries, 5 bits
[    8.628479] IRQ226 -> 0-7 CPU6
[    8.628723] IRQ230 -> 0-7 CPU6
[    8.628957] IRQ227 -> 0-7 CPU7
[    8.629094] ID:0 pID:8256 vID:262
[    8.629099] ID:1 pID:8257 vID:263
[    8.629104] ID:2 pID:8258 vID:264
[    8.629109] ID:3 pID:8259 vID:265
[    8.629114] ID:4 pID:8260 vID:266
[    8.629119] ID:5 pID:8261 vID:267
[    8.629124] ID:6 pID:8262 vID:268
[    8.629129] ID:7 pID:8263 vID:269
[    8.629134] ID:8 pID:8264 vID:270
[    8.629139] ID:9 pID:8265 vID:271
[    8.629144] ID:10 pID:8266 vID:272
[    8.629149] ID:11 pID:8267 vID:273
[    8.629153] ID:12 pID:8268 vID:274
[    8.629158] ID:13 pID:8269 vID:275
[    8.629163] ID:14 pID:8270 vID:276
[    8.629168] ID:15 pID:8271 vID:277
[    8.629173] ID:16 pID:8272 vID:278
[    8.629178] ID:17 pID:8273 vID:279
[    8.629183] ID:18 pID:8274 vID:280
[    8.629188] ID:19 pID:8275 vID:281
[    8.629200] IRQ231 -> 0-7 CPU7
[    8.629211] IRQ232 -> 0-7 CPU0
[    8.629222] IRQ233 -> 0-7 CPU1
[    8.629233] IRQ234 -> 0-7 CPU2
[    8.629244] IRQ235 -> 0-7 CPU3
[    8.629255] IRQ236 -> 0-7 CPU4
[    8.629266] IRQ237 -> 0-7 CPU7
[    8.629277] IRQ238 -> 0-7 CPU0
[    8.629287] IRQ239 -> 0-7 CPU1
[    8.629298] IRQ240 -> 0-7 CPU2
[    8.629309] IRQ241 -> 0-7 CPU3
[    8.629319] IRQ242 -> 0-7 CPU4
[    8.629336] IRQ243 -> 0-7 CPU5
[    8.629346] IRQ244 -> 0-7 CPU6
[    8.629357] IRQ245 -> 0-7 CPU7
[    8.629368] IRQ246 -> 0-7 CPU0
[    8.629379] IRQ247 -> 0-7 CPU1
[    8.629390] IRQ248 -> 0-7 CPU2
[    8.629401] IRQ249 -> 0-7 CPU3
[    8.629411] IRQ250 -> 0-7 CPU4
[    8.629422] IRQ251 -> 0-7 CPU5
[    8.629433] IRQ252 -> 0-7 CPU6
[    8.629670] ID:20 pID:8276 vID:282
[    8.629908] IRQ253 -> 0-7 CPU0
[    8.630134] ID:21 pID:8277 vID:283
[    8.635511] IRQ254 -> 0-7 CPU1
[    8.642115] ID:22 pID:8278 vID:284
[    8.649085] IRQ255 -> 0-7 CPU2
[    8.657029] ID:23 pID:8279 vID:285
[    8.663285] IRQ256 -> 0-7 CPU3
[    8.670689] ID:24 pID:8280 vID:286
[    8.677302] IRQ257 -> 0-7 CPU4
[    8.682925] ID:25 pID:8281 vID:287
[    8.688293] IRQ258 -> 0-7 CPU5
[    8.694547] ID:26 pID:8282 vID:288
[    8.702234] IRQ259 -> 0-7 CPU6
[    8.709197] ID:27 pID:8283 vID:289
[    8.709204] ID:28 pID:8284 vID:290
[    8.716722] IRQ260 -> 0-7 CPU7
[    8.722081] ID:29 pID:8285 vID:291
[    8.842813] ID:30 pID:8286 vID:292
[    8.842818] ID:31 pID:8287 vID:293
[    8.842966] IRQ262 -> 0-7 CPU0
[    8.842982] IRQ263 -> 0-7 CPU1
[    8.842993] IRQ264 -> 0-7 CPU2
[    8.843004] IRQ265 -> 0-7 CPU3
[    8.843016] IRQ266 -> 0-7 CPU4
[    8.843028] IRQ267 -> 0-7 CPU5
[    8.843040] IRQ268 -> 0-7 CPU6
[    8.843051] IRQ269 -> 0-7 CPU7
[    8.843063] IRQ270 -> 0-7 CPU0
[    8.843075] IRQ271 -> 0-7 CPU1
[    8.843087] IRQ272 -> 0-7 CPU2
[    8.843098] IRQ273 -> 0-7 CPU3
[    8.843110] IRQ274 -> 0-7 CPU4
[    8.843122] IRQ275 -> 0-7 CPU5
[    8.843133] IRQ276 -> 0-7 CPU6
[    8.843145] IRQ277 -> 0-7 CPU7
[    8.843157] IRQ278 -> 0-7 CPU0
[    8.843168] IRQ279 -> 0-7 CPU1
[    8.843180] IRQ280 -> 0-7 CPU2
[    8.843192] IRQ281 -> 0-7 CPU3
[    8.843203] IRQ282 -> 0-7 CPU4
[    8.843215] IRQ283 -> 0-7 CPU5
[    8.843227] IRQ284 -> 0-7 CPU6
[    8.843238] IRQ285 -> 0-7 CPU7
[    8.843250] IRQ286 -> 0-7 CPU0
[    8.843262] IRQ287 -> 0-7 CPU1
[    8.843273] IRQ288 -> 0-7 CPU2
[    8.843284] IRQ289 -> 0-7 CPU3
[    8.843296] IRQ290 -> 0-7 CPU4
[    8.843308] IRQ291 -> 0-7 CPU5
[    8.843319] IRQ292 -> 0-7 CPU6
[    8.843331] IRQ293 -> 0-7 CPU7
[    8.844444] ITS: alloc 8192:1
[    8.844455] ITT 1 entries, 0 bits
[    8.845389] ID:0 pID:8192 vID:196
[    8.845395] ITS: alloc 8193:1
[    8.845403] IRQ196 -> 0-7 CPU0
[    8.845405] ITT 1 entries, 0 bits
[    8.845604] IRQ196 -> 0-7 CPU0
[    8.845631] pcieport 0006:00:00.0: PME: Signaling with IRQ 196
[    8.846380] ID:0 pID:8193 vID:197
[    8.846414] ITS: alloc 8194:1
[    8.846423] ITT 1 entries, 0 bits
[    8.857408] IRQ197 -> 0-7 CPU1
[    8.857440] ID:0 pID:8194 vID:198
[    8.857450] IRQ198 -> 0-7 CPU2
[    8.857499] IRQ197 -> 0-7 CPU1
[    8.857515] pcieport 0002:00:00.0: PME: Signaling with IRQ 197
[    8.857529] IRQ198 -> 0-7 CPU2
[    8.858291] pcieport 0006:00:00.0: AER: enabled with IRQ 196
[    8.866563] pcieport 0002:00:00.0: AER: enabled with IRQ 197
[    8.872342] pcieport 0004:00:00.0: PME: Signaling with IRQ 198
[    8.885618] pcieport 0004:00:00.0: AER: enabled with IRQ 198
[    8.909946] mhi-pci-generic 0004:01:00.0: MHI PCI device found: foxconn-sdx55
[    8.914659] nvme nvme0: pci function 0002:01:00.0
[    8.917541] mhi-pci-generic 0004:01:00.0: BAR 0 [mem 0x34300000-0x34300fff 64bit]: assigned
[    8.922185] nvme 0002:01:00.0: enabling device (0000 -> 0002)
[    8.930939] mhi-pci-generic 0004:01:00.0: enabling device (0000 -> 0002)
[    8.937318] ITS: alloc 8195:1
[    8.944985] ITT 1 entries, 0 bits
[    8.945289] ITS: alloc 8196:8
[    8.945303] ITT 8 entries, 3 bits
[    8.947818] ID:0 pID:8195 vID:201
[    8.947910] IRQ201 -> 0-7 CPU3
[    8.948702] ID:0 pID:8196 vID:202
[    8.948720] ID:1 pID:8197 vID:203
[    8.950480] IRQ201 -> 0-7 CPU3
[    8.965330] ID:2 pID:8198 vID:204
[    8.974909] ID:3 pID:8199 vID:205
[    8.987215] ID:4 pID:8200 vID:206
[    9.001562] IRQ202 -> 0-7 CPU4
[    9.001759] IRQ203 -> 0-7 CPU5
[    9.001771] IRQ204 -> 0-7 CPU6
[    9.001849] IRQ205 -> 0-7 CPU7
[    9.001862] IRQ206 -> 0-7 CPU0
[    9.003223] IRQ202 -> 0-7 CPU4
[    9.003449] IRQ203 -> 0-7 CPU5
[    9.003638] IRQ204 -> 0-7 CPU6
[    9.003836] IRQ205 -> 0-7 CPU7
[    9.004007] IRQ206 -> 0-7 CPU0
[    9.005127] mhi mhi0: Requested to power ON
[    9.009901] mhi mhi0: Power on setup success
[    9.015403] nvme nvme0: allocated 61 MiB host memory buffer.
[    9.169296] ITS: alloc 8204:16
[    9.169319] ITT 16 entries, 4 bits
[    9.169492] ID:0 pID:8204 vID:201
[    9.169516] IRQ201 -> 0-7 CPU3
[    9.169620] ID:1 pID:8205 vID:211
[    9.169633] IRQ211 -> 0-7 CPU0
[    9.169702] ID:2 pID:8206 vID:212
[    9.169713] IRQ212 -> 0-7 CPU1
[    9.169904] ID:3 pID:8207 vID:213
[    9.169917] IRQ213 -> 0-7 CPU2
[    9.169982] ID:4 pID:8208 vID:214
[    9.169993] IRQ214 -> 0-7 CPU3
[    9.170070] ID:5 pID:8209 vID:215
[    9.170082] IRQ215 -> 0-7 CPU4
[    9.170143] ID:6 pID:8210 vID:216
[    9.170155] IRQ216 -> 0-7 CPU5
[    9.170221] ID:7 pID:8211 vID:217
[    9.170232] IRQ217 -> 0-7 CPU6
[    9.170294] ID:8 pID:8212 vID:218
[    9.170319] IRQ218 -> 0-7 CPU7
[    9.170460] IRQ201 -> 0-7 CPU3
[    9.179969] IRQ211 -> 0 CPU0
[    9.180329] IRQ212 -> 1 CPU1
[    9.180663] IRQ213 -> 2 CPU2
[    9.181001] IRQ214 -> 3 CPU3
[    9.181355] IRQ215 -> 4 CPU4
[    9.181702] IRQ216 -> 5 CPU5
[    9.188542] IRQ217 -> 6 CPU6
[    9.196576] IRQ218 -> 7 CPU7
[    9.196623] nvme nvme0: 8/0/0 default/read/poll queues
[    9.206751]  nvme0n1: p1
[    9.278797] ath11k_pci 0006:01:00.0: BAR 0 [mem 0x30400000-0x305fffff 64bit]: assigned
[    9.294555] ath11k_pci 0006:01:00.0: enabling device (0000 -> 0002)
[    9.295634] wwan wwan0: port wwan0qcdm0 attached
[    9.296105] wwan wwan0: port wwan0mbim0 attached
[    9.296789] wwan wwan0: port wwan0at0 attached
[    9.304915] ITS: alloc 8220:32
[    9.314316] ITT 32 entries, 5 bits
[    9.324270] ID:0 pID:8220 vID:262
[    9.338759] ID:1 pID:8221 vID:263
[    9.338765] ID:2 pID:8222 vID:264
[    9.338770] ID:3 pID:8223 vID:265
[    9.338775] ID:4 pID:8224 vID:266
[    9.338779] ID:5 pID:8225 vID:267
[    9.338784] ID:6 pID:8226 vID:268
[    9.338789] ID:7 pID:8227 vID:269
[    9.338794] ID:8 pID:8228 vID:270
[    9.338798] ID:9 pID:8229 vID:271
[    9.338803] ID:10 pID:8230 vID:272
[    9.338808] ID:11 pID:8231 vID:273
[    9.338812] ID:12 pID:8232 vID:274
[    9.338817] ID:13 pID:8233 vID:275
[    9.338821] ID:14 pID:8234 vID:276
[    9.338826] ID:15 pID:8235 vID:277
[    9.338831] ID:16 pID:8236 vID:278
[    9.338836] ID:17 pID:8237 vID:279
[    9.338841] ID:18 pID:8238 vID:280
[    9.338845] ID:19 pID:8239 vID:281
[    9.338850] ID:20 pID:8240 vID:282
[    9.338855] ID:21 pID:8241 vID:283
[    9.338859] ID:22 pID:8242 vID:284
[    9.338864] ID:23 pID:8243 vID:285
[    9.338868] ID:24 pID:8244 vID:286
[    9.338873] ID:25 pID:8245 vID:287
[    9.338877] ID:26 pID:8246 vID:288
[    9.338882] ID:27 pID:8247 vID:289
[    9.338887] ID:28 pID:8248 vID:290
[    9.338891] ID:29 pID:8249 vID:291
[    9.338896] ID:30 pID:8250 vID:292
[    9.338900] ID:31 pID:8251 vID:293
[    9.338980] IRQ262 -> 0-7 CPU1
[    9.362613] IRQ263 -> 0-7 CPU2
[    9.370142] IRQ264 -> 0-7 CPU3
[    9.377656] IRQ265 -> 0-7 CPU4
[    9.400274] IRQ266 -> 0-7 CPU5
[    9.409009] IRQ267 -> 0-7 CPU6
[    9.409021] IRQ268 -> 0-7 CPU7
[    9.409033] IRQ269 -> 0-7 CPU0
[    9.409044] IRQ270 -> 0-7 CPU1
[    9.409056] IRQ271 -> 0-7 CPU2
[    9.409067] IRQ272 -> 0-7 CPU3
[    9.409078] IRQ273 -> 0-7 CPU4
[    9.409089] IRQ274 -> 0-7 CPU5
[    9.409100] IRQ275 -> 0-7 CPU6
[    9.409111] IRQ276 -> 0-7 CPU7
[    9.409123] IRQ277 -> 0-7 CPU0
[    9.409134] IRQ278 -> 0-7 CPU1
[    9.409145] IRQ279 -> 0-7 CPU2
[    9.409157] IRQ280 -> 0-7 CPU3
[    9.409168] IRQ281 -> 0-7 CPU4
[    9.409179] IRQ282 -> 0-7 CPU5
[    9.409190] IRQ283 -> 0-7 CPU6
[    9.409201] IRQ284 -> 0-7 CPU7
[    9.409213] IRQ285 -> 0-7 CPU0
[    9.409224] IRQ286 -> 0-7 CPU1
[    9.409235] IRQ287 -> 0-7 CPU2
[    9.409247] IRQ288 -> 0-7 CPU3
[    9.409258] IRQ289 -> 0-7 CPU4
[    9.409270] IRQ290 -> 0-7 CPU5
[    9.409281] IRQ291 -> 0-7 CPU6
[    9.409292] IRQ292 -> 0-7 CPU7
[    9.409303] IRQ293 -> 0-7 CPU0
[    9.409438] ath11k_pci 0006:01:00.0: MSI vectors: 32
[    9.426507] ath11k_pci 0006:01:00.0: wcn6855 hw2.0
[    9.456885] IRQ262 -> 0-7 CPU1
[    9.467067] IRQ263 -> 0-7 CPU2
[    9.481466] IRQ264 -> 0-7 CPU3
[    9.630594] IRQ265 -> 0-7 CPU4
[    9.630629] IRQ266 -> 0-7 CPU5
[    9.630655] IRQ267 -> 0-7 CPU6
[    9.630682] IRQ268 -> 0-7 CPU7
[    9.630709] IRQ269 -> 0-7 CPU0
[    9.630735] IRQ270 -> 0-7 CPU1
[    9.630764] IRQ271 -> 0-7 CPU2
[    9.640971] IRQ276 -> 0-7 CPU7
[    9.641039] IRQ277 -> 0-7 CPU0
[    9.641088] IRQ278 -> 0-7 CPU1
[    9.641138] IRQ280 -> 0-7 CPU3
[    9.641182] IRQ281 -> 0-7 CPU4
[    9.641227] IRQ282 -> 0-7 CPU5
[    9.651400] IRQ283 -> 0-7 CPU6
[    9.651442] IRQ284 -> 0-7 CPU7
[    9.651490] IRQ285 -> 0-7 CPU0
[    9.651534] IRQ286 -> 0-7 CPU1
[    9.813900] mhi mhi1: Requested to power ON
[    9.818607] mhi mhi1: Power on setup success
[   10.017482] mhi mhi1: Wait for device to enter SBL or Mission mode
[   10.862765] ath11k_pci 0006:01:00.0: chip_id 0x2 chip_family 0xb board_id 0x8c soc_id 0x400c0200
[   10.872101] ath11k_pci 0006:01:00.0: fw_version 0x1106196e fw_build_timestamp 2024-01-12 11:30 fw_build_id WLAN.HSP.1.1-03125-QCAHSPSWPL_V1_V2_SILICONZ_LITE-3.6510.37
maintainer
#90 lore raw
Quoted 12 lines · Johan Hovold
Quoted 9 lines · Marc Zyngier
Quoted 3 lines · Johan Hovold

I believe there is a kernel parameter for this (e.g. module.async_probe), but I just disable async probing for the Qualcomm PCIe driver I'm using:

I had tried this module parameter, but it didn't change anything on my end.

Quoted 6 lines
Quoted 3 lines

I'll have a look whether the TX1 PCIe driver uses this. It's positively ancient, so I wouldn't bet that it has been touched significantly in the past 5 years.

Perhaps async probing just changes the symptoms, the NVMe and wifi doesn't work in either case.

Yeah, my impression is that this changes the order in which LPIs get allocated, but the core symptom is the same.

Quoted 11 lines
Quoted 7 lines
Quoted 3 lines
[    8.692011] Reusing ITT for devID 0
[    8.693668] Reusing ITT for devID 0

This is really odd. It indicates that you have several devices sharing the same DeviceID, which I seriously doubt it is the case in a laptop. Do you have any non-transparent bridge here? lspci would help.

Yeah, and these messages do not show up without the series (see log below). They are there in the previous synchronous log however.

Quoted 6 lines

0002:00:00.0 PCI bridge: Qualcomm Technologies, Inc SC8280XP PCI Express Root Port 0002:01:00.0 Non-Volatile memory controller: KIOXIA Corporation NVMe SSD Controller BG4 (DRAM-less) 0004:00:00.0 PCI bridge: Qualcomm Technologies, Inc SC8280XP PCI Express Root Port 0004:01:00.0 Unassigned class [ff00]: Qualcomm Technologies, Inc SDX55 [Snapdragon X55 5G] 0006:00:00.0 PCI bridge: Qualcomm Technologies, Inc SC8280XP PCI Express Root Port 0006:01:00.0 Network controller: Qualcomm Technologies, Inc QCNFA765 Wireless Network Adapter (rev 01)

Right, this is a very straightforward setup, Design-crap-ware-style. Nothing that would alias any device.

Quoted 11 lines
Quoted 9 lines

I'm starting to suspect that the new code doesn't carry all the required bits for the DevID, and that we end-up trying to allocated interrupts from the pool allocated to another device, which can never be a good thing, and would explain why everything dies a painful death.

Can you run the same trace with the whole thing reverted? I think we're on something here.

See below, using normal asynchronous probing like the previous log.

And as expected, no aliasing showing up in this log. Somehow, we're not able to distinguish between the different PCI domains anymore, leading to all sorts of funnies.

For the record, I've added some extra debug in the its driver and ran the result on TX1, old and new kernels.

Before this series:

[   10.139806] nvme nvme0: pci function 0006:58:00.0
[   10.158599] nvme 0006:58:00.0: devid = 35800

With this series:

[   10.143729] nvme nvme0: pci function 0006:58:00.0
[   10.181775] nvme 0006:58:00.0: devid = 5800

Clearly, we've lost something in the battle. I'll keep digging.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#91 lore raw
In reply to Johan Hovold
Quoted 8 lines · Johan Hovold
Quoted 6 lines · Marc Zyngier
Quoted 2 lines · Johan Hovold
[    8.692011] Reusing ITT for devID 0
[    8.693668] Reusing ITT for devID 0
Quoted 6 lines
Quoted 3 lines

This is really odd. It indicates that you have several devices sharing the same DeviceID, which I seriously doubt it is the case in a laptop. Do you have any non-transparent bridge here? lspci would help.

Yeah, and these messages do not show up without the series (see log below). They are there in the previous synchronous log however.

I think I've finally nailed the sucker, and posted a potential fix[1].

It definitely restore my TX1 to a state that is no worse than normal, so something must be less wrong there. I'm pretty sure that the platform-msi equivalent is equally broken, but I don't have the energy to verify/debug that tonight.

Thomas, feel free to squash this into your series or keep it as is, as you prefer.

	M.

[1] https://lore.kernel.org/r/[email protected]

Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#92 lore raw
Quoted 8 lines · Marc Zyngier

I think I've finally nailed the sucker, and posted a potential fix[1].

It definitely restore my TX1 to a state that is no worse than normal, so something must be less wrong there. I'm pretty sure that the platform-msi equivalent is equally broken, but I don't have the energy to verify/debug that tonight.

[1] https://lore.kernel.org/r/[email protected]

This seems to fix the regression here too, thanks!

201:          0       ...         0  ITS-PCI-MSI-0006:00:00.0   0 Edge      PCIe PME, aerdrv
202:          0                   0  ITS-PCI-MSI-0006:01:00.0   0 Edge      bhi
203:          0                   0  ITS-PCI-MSI-0006:01:00.0   1 Edge      mhi
204:          0                   0  ITS-PCI-MSI-0006:01:00.0   2 Edge      mhi
205:          0                   0  ITS-PCI-MSI-0006:01:00.0   3 Edge      ce0
206:          0                   0  ITS-PCI-MSI-0006:01:00.0   4 Edge      ce1
207:          0                   0  ITS-PCI-MSI-0006:01:00.0   5 Edge      ce2
208:          0                   2  ITS-PCI-MSI-0006:01:00.0   6 Edge      ce3
209:          2                   0  ITS-PCI-MSI-0006:01:00.0   7 Edge      ce5
210:          0                   0  ITS-PCI-MSI-0006:01:00.0   8 Edge      ce7
211:          0                   0  ITS-PCI-MSI-0006:01:00.0   9 Edge      ce8
216:          0                   0  ITS-PCI-MSI-0006:01:00.0  14 Edge      DP_EXT_IRQ
217:          0                   0  ITS-PCI-MSI-0006:01:00.0  15 Edge      DP_EXT_IRQ
218:          0                   0  ITS-PCI-MSI-0006:01:00.0  16 Edge      DP_EXT_IRQ
220:          0                   0  ITS-PCI-MSI-0006:01:00.0  18 Edge      DP_EXT_IRQ
221:          0                   0  ITS-PCI-MSI-0006:01:00.0  19 Edge      DP_EXT_IRQ
222:          0                   0  ITS-PCI-MSI-0006:01:00.0  20 Edge      DP_EXT_IRQ
223:          0                   0  ITS-PCI-MSI-0006:01:00.0  21 Edge      DP_EXT_IRQ
224:          0                   0  ITS-PCI-MSI-0006:01:00.0  22 Edge      DP_EXT_IRQ
225:          0                   0  ITS-PCI-MSI-0006:01:00.0  23 Edge      DP_EXT_IRQ
226:          0                   0  ITS-PCI-MSI-0006:01:00.0  24 Edge      DP_EXT_IRQ
235:          0                   0  ITS-PCI-MSI-0004:00:00.0   0 Edge      PCIe PME, aerdrv
236:          0                   0  ITS-PCI-MSI-0004:01:00.0   0 Edge      bhi
237:          0                   0  ITS-PCI-MSI-0004:01:00.0   1 Edge      mhi
238:          0                   0  ITS-PCI-MSI-0004:01:00.0   2 Edge      mhi
239:          0                   0  ITS-PCI-MSI-0004:01:00.0   3 Edge      mhi
240:          0                   0  ITS-PCI-MSI-0004:01:00.0   4 Edge      mhi
242:          0                   0  ITS-PCI-MSI-0002:00:00.0   0 Edge      PCIe PME, aerdrv
243:         22                   0  ITS-PCI-MSIX-0002:01:00.0   0 Edge      nvme0q0
244:          0                   0  ITS-PCI-MSIX-0002:01:00.0   1 Edge      nvme0q1
245:          0                   0  ITS-PCI-MSIX-0002:01:00.0   2 Edge      nvme0q2
246:          0                   0  ITS-PCI-MSIX-0002:01:00.0   3 Edge      nvme0q3
247:          0                   0  ITS-PCI-MSIX-0002:01:00.0   4 Edge      nvme0q4
248:          0                   0  ITS-PCI-MSIX-0002:01:00.0   5 Edge      nvme0q5
249:          0                   0  ITS-PCI-MSIX-0002:01:00.0   6 Edge      nvme0q6
250:          0                   0  ITS-PCI-MSIX-0002:01:00.0   7 Edge      nvme0q7
251:          0                   0  ITS-PCI-MSIX-0002:01:00.0   8 Edge      nvme0q8

Johan

#93 lore raw
In reply to Johan Hovold
Quoted 11 lines · Johan Hovold

I've applied the series (21 commits from linux-next) on top of 6.10 and can confirm that the breakage is caused by commits:

	3d1c927c08fc ("irqchip/gic-v3-its: Switch platform MSI to MSI parent")
	233db05bc37f ("irqchip/gic-v3-its: Provide MSI parent for PCI/MSI[-X]")

Applying the series up until the change before 3d1c927c08fc unbreaks the wifi on one machine:

	ath11k_pci 0006:01:00.0: failed to enable msi: -22
	ath11k_pci 0006:01:00.0: probe with driver ath11k_pci failed with error -22

3d1c927c08fc converts the platform MSI stuff over which is unrelated to PCI/MSI. I'm confused how this affects PCI/MSI of the WIFI card.

Quoted 2 lines

and backing up until the commit before 233db05bc37f makes the NVMe come up again during boot on another.

So that undoes the PCI/MSI change. Hrm.

Quoted 1 line

I have not tried to debug this further.

Any hint would be appreciated.

Thanks,

tglx