Skip to contentSkip to conversation

Linux RISC-V AIA Support

PATCH Applied Anup Patel 11 participants 56 messages Feb 22, 2024 – Jul 9, 2024

Subsystems (7)

v14 of 16 Applied Latest is v16 →

lore
Share & tools
lore

Anup Patel

author
#0 lore raw

v14 cover Linux RISC-V AIA Support

The RISC-V AIA specification is ratified as-per the RISC-V international process. The latest ratified AIA specifcation can be found at:
https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interrupts-1.0.pdf

At a high-level, the AIA specification adds three things:
1) AIA CSRs
- Improved local interrupt support
2) Incoming Message Signaled Interrupt Controller (IMSIC)
- Per-HART MSI controller
- Support MSI virtualization
- Support IPI along with virtualization
3) Advanced Platform-Level Interrupt Controller (APLIC)
- Wired interrupt controller
- In MSI-mode, converts wired interrupt into MSIs (i.e. MSI generator)
- In Direct-mode, injects external interrupts directly into HARTs

For an overview of the AIA specification, refer the AIA virtualization talk at KVM Forum 2022:
https://static.sched.com/hosted_files/kvmforum2022/a1/AIA_Virtualization_in_KVM_RISCV_final.pdf
https://www.youtube.com/watch?v=r071dL8Z0yo

To test this series, use QEMU v7.2 (or higher) and OpenSBI v1.2 (or higher).

This series depends upon per-device MSI domain patches merged by Thomas (tglx) which are available in irq/msi branch at:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git

These patches can also be found in the riscv_aia_v14 branch at:
https://github.com/avpatel/linux.git

Changes since v13:
- Split PATCH1 into six granular patches
- Addressed nit comments from Thomas and Bjorn

Changes since v12:
- Rebased on Linux-6.8-rc5
- Dropped per-device MSI domain patches which are already merged by Thomas (tglx)
- Addressed nit comments from Thomas and Clement
- Added a new patch2 to fix lock dependency warning
- Replaced local sync IPI in the IMSIC driver with per-CPU timer
- Simplified locking in the IMSIC driver to avoid lock dependency issues
- Added a dirty bitmap in the IMSIC driver to optimize per-CPU local sync loop

Changes since v11:
- Rebased on Linux-6.8-rc1
- Included kernel/irq related patches from "genirq, irqchip: Convert ARM MSI handling to per device MSI domains" series by Thomas. (PATCH7, PATCH8, PATCH9, PATCH14, PATCH16, PATCH17, PATCH18, PATCH19,

    PATCH20, PATCH21, PATCH22, PATCH23, and PATCH32 of
    https://lore.kernel.org/linux-arm-kernel/[email protected]/)

- Updated APLIC MSI-mode driver to use the new WIRED_TO_MSI mechanism.
- Updated IMSIC driver to support per-device MSI domains for PCI and platform devices.

Changes since v10:
- Rebased on Linux-6.6-rc7
- Dropped PATCH3 of v10 series since this has been merged by MarcZ for Linux-6.6-rc7
- Changed the IMSIC ID management strategy from 1-n approach to x86-style 1-1 approach

Changes since v9:
- Rebased on Linux-6.6-rc4
- Use builtin_platform_driver() in PATCH5, PATCH9, and PATCH12

Changes since v8:
- Rebased on Linux-6.6-rc3
- Dropped PATCH2 of v8 series since we won't be requiring riscv_get_intc_hartid() based on Marc Z's comments on ACPI AIA support.
- Addressed Saravana's comments in PATCH3 of v8 series
- Update PATCH9 and PATCH13 of v8 series based on comments from Sunil

Changes since v7:
- Rebased on Linux-6.6-rc1
- Addressed comments on PATCH1 of v7 series and split it into two PATCHes
- Use DEFINE_SIMPLE_PROP() in PATCH2 of v7 series

Changes since v6:
- Rebased on Linux-6.5-rc4
- Updated PATCH2 to use IS_ENABLED(CONFIG_SPARC) instead of !IS_ENABLED(CONFIG_OF_IRQ)
- Added new PATCH4 to fix syscore registration in PLIC driver
- Update PATCH5 to convert PLIC driver into full-blown platform driver with a re-written probe function.

Changes since v5:
- Rebased on Linux-6.5-rc2
- Updated the overall series to ensure that only IPI, timer, and INTC drivers are probed very early whereas rest of the interrupt controllers (such as PLIC, APLIC, and IMISC) are probed as regular platform drivers.
- Renamed riscv_fw_parent_hartid() to riscv_get_intc_hartid()
- New PATCH1 to add fw_devlink support for msi-parent DT property
- New PATCH2 to ensure all INTC suppliers are initialized which in-turn fixes the probing issue for PLIC, APLIC and IMSIC as platform driver
- New PATCH3 to use platform driver probing for PLIC
- Re-structured the IMSIC driver into two separate drivers: early and platform. The IMSIC early driver (PATCH7) only initialized IMSIC state and provides IPIs whereas the IMSIC platform driver (PATCH8) is probed provides MSI domain for platform devices.
- Re-structure the APLIC platform driver into three separe sources: main, direct mode, and MSI mode.

Changes since v4:
- Rebased on Linux-6.5-rc1
- Added "Dependencies" in the APLIC bindings (PATCH6 in v4)
- Dropped the PATCH6 which was changing the IOMMU DMA domain APIs
- Dropped use of IOMMU DMA APIs in the IMSIC driver (PATCH4)

Changes since v3:
- Rebased on Linux-6.4-rc6
- Dropped PATCH2 of v3 series instead we now set FWNODE_FLAG_BEST_EFFORT via IRQCHIP_DECLARE()
- Extend riscv_fw_parent_hartid() to support both DT and ACPI in PATCH1
- Extend iommu_dma_compose_msi_msg() instead of adding iommu_dma_select_msi() in PATCH6
- Addressed Conor's comments in PATCH3
- Addressed Conor's and Rob's comments in PATCH7

Changes since v2:
- Rebased on Linux-6.4-rc1
- Addressed Rob's comments on DT bindings patches 4 and 8.
- Addessed Marc's comments on IMSIC driver PATCH5
- Replaced use of OF apis in APLIC and IMSIC drivers with FWNODE apis this makes both drivers easily portable for ACPI support. This also removes unnecessary indirection from the APLIC and IMSIC drivers.
- PATCH1 is a new patch for portability with ACPI support
- PATCH2 is a new patch to fix probing in APLIC drivers for APLIC-only systems.
- PATCH7 is a new patch which addresses the IOMMU DMA domain issues pointed out by SiFive

Changes since v1:
- Rebased on Linux-6.2-rc2
- Addressed comments on IMSIC DT bindings for PATCH4
- Use raw_spin_lock_irqsave() on ids_lock for PATCH5
- Improved MMIO alignment checks in PATCH5 to allow MMIO regions with holes.
- Addressed comments on APLIC DT bindings for PATCH6
- Fixed warning splat in aplic_msi_write_msg() caused by zeroed MSI message in PATCH7
- Dropped DT property riscv,slow-ipi instead will have module parameter in future.

Anup Patel (17):
irqchip/sifive-plic: Convert PLIC driver into a platform driver irqchip/sifive-plic: Use dev_xyz() in-place of pr_xyz() irqchip/sifive-plic: Use devm_xyz() for managed allocation irqchip/sifive-plic: Use riscv_get_intc_hwnode() to get parent fwnode irqchip/sifive-plic: Cleanup PLIC contexts upon irqdomain creation
failure
irqchip/sifive-plic: Parse number of irqs and contexts early in
plic_probe
irqchip/sifive-plic: Improve locking safety by using
irqsave/irqrestore
irqchip/riscv-intc: Add support for RISC-V AIA
dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller
irqchip: Add RISC-V incoming MSI controller early driver irqchip/riscv-imsic: Add device MSI domain support for platform
devices
irqchip/riscv-imsic: Add device MSI domain support for PCI devices
dt-bindings: interrupt-controller: Add RISC-V advanced PLIC
irqchip: Add RISC-V advanced PLIC driver for direct-mode irqchip/riscv-aplic: Add support for MSI-mode
RISC-V: Select APLIC and IMSIC drivers
MAINTAINERS: Add entry for RISC-V AIA drivers

Björn Töpel (1):
genirq/matrix: Dynamic bitmap allocation

 .../interrupt-controller/riscv,aplic.yaml     | 172 ++++
 .../interrupt-controller/riscv,imsics.yaml    | 172 ++++
 MAINTAINERS                                   |  14 +
 arch/riscv/Kconfig                            |   2 +
 arch/x86/include/asm/hw_irq.h                 |   2 -
 drivers/irqchip/Kconfig                       |  25 +
 drivers/irqchip/Makefile                      |   3 +
 drivers/irqchip/irq-riscv-aplic-direct.c      | 326 +++++++
 drivers/irqchip/irq-riscv-aplic-main.c        | 211 +++++
 drivers/irqchip/irq-riscv-aplic-main.h        |  52 ++
 drivers/irqchip/irq-riscv-aplic-msi.c         | 263 ++++++
 drivers/irqchip/irq-riscv-imsic-early.c       | 201 ++++
 drivers/irqchip/irq-riscv-imsic-platform.c    | 375 ++++++++
 drivers/irqchip/irq-riscv-imsic-state.c       | 870 ++++++++++++++++++
 drivers/irqchip/irq-riscv-imsic-state.h       | 108 +++
 drivers/irqchip/irq-riscv-intc.c              |  32 +-
 drivers/irqchip/irq-sifive-plic.c             | 269 ++++--
 include/linux/irqchip/riscv-aplic.h           | 145 +++
 include/linux/irqchip/riscv-imsic.h           |  87 ++
 kernel/irq/matrix.c                           |  28 +-
 20 files changed, 3235 insertions(+), 122 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml
 create mode 100644 drivers/irqchip/irq-riscv-aplic-direct.c
 create mode 100644 drivers/irqchip/irq-riscv-aplic-main.c
 create mode 100644 drivers/irqchip/irq-riscv-aplic-main.h
 create mode 100644 drivers/irqchip/irq-riscv-aplic-msi.c
 create mode 100644 drivers/irqchip/irq-riscv-imsic-early.c
 create mode 100644 drivers/irqchip/irq-riscv-imsic-platform.c
 create mode 100644 drivers/irqchip/irq-riscv-imsic-state.c
 create mode 100644 drivers/irqchip/irq-riscv-imsic-state.h
 create mode 100644 include/linux/irqchip/riscv-aplic.h
 create mode 100644 include/linux/irqchip/riscv-imsic.h
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH v14 1/18 irqchip/sifive-plic: Convert PLIC driver into a platform driver

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

---
 drivers/irqchip/irq-sifive-plic.c | 101 ++++++++++++++++++------------
 1 file changed, 61 insertions(+), 40 deletions(-)
drivers/irqchip/irq-sifive-plic.c +61 -40
index 5b7bc4fd9517..7400a07fc479 100644
@@ -64,6 +64,7 @@
 #define PLIC_QUIRK_EDGE_INTERRUPT	0
 
 struct plic_priv {
+	struct device *dev;
 	struct cpumask lmask;
 	struct irq_domain *irqdomain;
 	void __iomem *regs;
@@ -406,30 +407,50 @@ static int plic_starting_cpu(unsigned int cpu)
 	return 0;
 }
 
-static int __init __plic_init(struct device_node *node,
-			      struct device_node *parent,
-			      unsigned long plic_quirks)
+static const struct of_device_id plic_match[] = {
+	{ .compatible = "sifive,plic-1.0.0" },
+	{ .compatible = "riscv,plic0" },
+	{ .compatible = "andestech,nceplic100",
+	  .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+	{ .compatible = "thead,c900-plic",
+	  .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+	{}
+};
+
+static int plic_probe(struct platform_device *pdev)
 {
 	int error = 0, nr_contexts, nr_handlers = 0, i;
-	u32 nr_irqs;
-	struct plic_priv *priv;
+	struct device *dev = &pdev->dev;
+	unsigned long plic_quirks = 0;
 	struct plic_handler *handler;
+	struct plic_priv *priv;
+	bool cpuhp_setup;
 	unsigned int cpu;
+	u32 nr_irqs;
+
+	if (is_of_node(dev->fwnode)) {
+		const struct of_device_id *id;
+
+		id = of_match_node(plic_match, to_of_node(dev->fwnode));
+		if (id)
+			plic_quirks = (unsigned long)id->data;
+	}
 
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
 	if (!priv)
 		return -ENOMEM;
 
+	priv->dev = dev;
 	priv->plic_quirks = plic_quirks;
 
-	priv->regs = of_iomap(node, 0);
+	priv->regs = of_iomap(to_of_node(dev->fwnode), 0);
 	if (WARN_ON(!priv->regs)) {
 		error = -EIO;
 		goto out_free_priv;
 	}
 
 	error = -EINVAL;
-	of_property_read_u32(node, "riscv,ndev", &nr_irqs);
+	of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", &nr_irqs);
 	if (WARN_ON(!nr_irqs))
 		goto out_iounmap;
 
@@ -439,13 +460,13 @@ static int __init __plic_init(struct device_node *node,
 	if (!priv->prio_save)
 		goto out_free_priority_reg;
 
-	nr_contexts = of_irq_count(node);
+	nr_contexts = of_irq_count(to_of_node(dev->fwnode));
 	if (WARN_ON(!nr_contexts))
 		goto out_free_priority_reg;
 
 	error = -ENOMEM;
-	priv->irqdomain = irq_domain_add_linear(node, nr_irqs + 1,
-			&plic_irqdomain_ops, priv);
+	priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
+						&plic_irqdomain_ops, priv);
 	if (WARN_ON(!priv->irqdomain))
 		goto out_free_priority_reg;
 
@@ -455,7 +476,7 @@ static int __init __plic_init(struct device_node *node,
 		int cpu;
 		unsigned long hartid;
 
-		if (of_irq_parse_one(node, i, &parent)) {
+		if (of_irq_parse_one(to_of_node(dev->fwnode), i, &parent)) {
 			pr_err("failed to parse parent for context %d.\n", i);
 			continue;
 		}
@@ -491,7 +512,7 @@ static int __init __plic_init(struct device_node *node,
 
 		/* Find parent domain and register chained handler */
 		if (!plic_parent_irq && irq_find_host(parent.np)) {
-			plic_parent_irq = irq_of_parse_and_map(node, i);
+			plic_parent_irq = irq_of_parse_and_map(to_of_node(dev->fwnode), i);
 			if (plic_parent_irq)
 				irq_set_chained_handler(plic_parent_irq,
 							plic_handle_irq);
@@ -533,20 +554,29 @@ static int __init __plic_init(struct device_node *node,
 
 	/*
 	 * We can have multiple PLIC instances so setup cpuhp state
-	 * and register syscore operations only when context handler
-	 * for current/boot CPU is present.
+	 * and register syscore operations only once after context
+	 * handlers of all online CPUs are initialized.
 	 */
-	handler = this_cpu_ptr(&plic_handlers);
-	if (handler->present && !plic_cpuhp_setup_done) {
-		cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
-				  "irqchip/sifive/plic:starting",
-				  plic_starting_cpu, plic_dying_cpu);
-		register_syscore_ops(&plic_irq_syscore_ops);
-		plic_cpuhp_setup_done = true;
+	if (!plic_cpuhp_setup_done) {
+		cpuhp_setup = true;
+		for_each_online_cpu(cpu) {
+			handler = per_cpu_ptr(&plic_handlers, cpu);
+			if (!handler->present) {
+				cpuhp_setup = false;
+				break;
+			}
+		}
+		if (cpuhp_setup) {
+			cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
+					  "irqchip/sifive/plic:starting",
+					  plic_starting_cpu, plic_dying_cpu);
+			register_syscore_ops(&plic_irq_syscore_ops);
+			plic_cpuhp_setup_done = true;
+		}
 	}
 
-	pr_info("%pOFP: mapped %d interrupts with %d handlers for"
-		" %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
+	pr_info("%pOFP: mapped %d interrupts with %d handlers for %d contexts.\n",
+		to_of_node(dev->fwnode), nr_irqs, nr_handlers, nr_contexts);
 	return 0;
 
 out_free_enable_reg:
@@ -563,20 +593,11 @@ static int __init __plic_init(struct device_node *node,
 	return error;
 }
 
-static int __init plic_init(struct device_node *node,
-			    struct device_node *parent)
-{
-	return __plic_init(node, parent, 0);
-}
-
-IRQCHIP_DECLARE(sifive_plic, "sifive,plic-1.0.0", plic_init);
-IRQCHIP_DECLARE(riscv_plic0, "riscv,plic0", plic_init); /* for legacy systems */
-
-static int __init plic_edge_init(struct device_node *node,
-				 struct device_node *parent)
-{
-	return __plic_init(node, parent, BIT(PLIC_QUIRK_EDGE_INTERRUPT));
-}
-
-IRQCHIP_DECLARE(andestech_nceplic100, "andestech,nceplic100", plic_edge_init);
-IRQCHIP_DECLARE(thead_c900_plic, "thead,c900-plic", plic_edge_init);
+static struct platform_driver plic_driver = {
+	.driver = {
+		.name		= "riscv-plic",
+		.of_match_table	= plic_match,
+	},
+	.probe = plic_probe,
+};
+builtin_platform_driver(plic_driver);
Signature · 1 line
2.34.1
merge bot Applied
#2 lore raw

[tip: irq/msi] irqchip/sifive-plic: Convert PLIC driver into a platform driver

The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     8ec99b033147ef3bb8f0a560c24eb1baec3bc0be
Gitweb:        https://git.kernel.org/tip/8ec99b033147ef3bb8f0a560c24eb1baec3bc0be
Author:        Anup Patel <[email protected]>
AuthorDate:    Thu, 22 Feb 2024 15:09:49 +05:30
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Fri, 23 Feb 2024 10:18:43 +01:00

irqchip/sifive-plic: Convert PLIC driver into a platform driver

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

---
 drivers/irqchip/irq-sifive-plic.c | 101 +++++++++++++++++------------
 1 file changed, 61 insertions(+), 40 deletions(-)
drivers/irqchip/irq-sifive-plic.c +61 -40
index 5b7bc4f..7400a07 100644
@@ -64,6 +64,7 @@
 #define PLIC_QUIRK_EDGE_INTERRUPT	0
 
 struct plic_priv {
+	struct device *dev;
 	struct cpumask lmask;
 	struct irq_domain *irqdomain;
 	void __iomem *regs;
@@ -406,30 +407,50 @@ static int plic_starting_cpu(unsigned int cpu)
 	return 0;
 }
 
-static int __init __plic_init(struct device_node *node,
-			      struct device_node *parent,
-			      unsigned long plic_quirks)
+static const struct of_device_id plic_match[] = {
+	{ .compatible = "sifive,plic-1.0.0" },
+	{ .compatible = "riscv,plic0" },
+	{ .compatible = "andestech,nceplic100",
+	  .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+	{ .compatible = "thead,c900-plic",
+	  .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+	{}
+};
+
+static int plic_probe(struct platform_device *pdev)
 {
 	int error = 0, nr_contexts, nr_handlers = 0, i;
-	u32 nr_irqs;
-	struct plic_priv *priv;
+	struct device *dev = &pdev->dev;
+	unsigned long plic_quirks = 0;
 	struct plic_handler *handler;
+	struct plic_priv *priv;
+	bool cpuhp_setup;
 	unsigned int cpu;
+	u32 nr_irqs;
+
+	if (is_of_node(dev->fwnode)) {
+		const struct of_device_id *id;
+
+		id = of_match_node(plic_match, to_of_node(dev->fwnode));
+		if (id)
+			plic_quirks = (unsigned long)id->data;
+	}
 
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
 	if (!priv)
 		return -ENOMEM;
 
+	priv->dev = dev;
 	priv->plic_quirks = plic_quirks;
 
-	priv->regs = of_iomap(node, 0);
+	priv->regs = of_iomap(to_of_node(dev->fwnode), 0);
 	if (WARN_ON(!priv->regs)) {
 		error = -EIO;
 		goto out_free_priv;
 	}
 
 	error = -EINVAL;
-	of_property_read_u32(node, "riscv,ndev", &nr_irqs);
+	of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", &nr_irqs);
 	if (WARN_ON(!nr_irqs))
 		goto out_iounmap;
 
@@ -439,13 +460,13 @@ static int __init __plic_init(struct device_node *node,
 	if (!priv->prio_save)
 		goto out_free_priority_reg;
 
-	nr_contexts = of_irq_count(node);
+	nr_contexts = of_irq_count(to_of_node(dev->fwnode));
 	if (WARN_ON(!nr_contexts))
 		goto out_free_priority_reg;
 
 	error = -ENOMEM;
-	priv->irqdomain = irq_domain_add_linear(node, nr_irqs + 1,
-			&plic_irqdomain_ops, priv);
+	priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
+						&plic_irqdomain_ops, priv);
 	if (WARN_ON(!priv->irqdomain))
 		goto out_free_priority_reg;
 
@@ -455,7 +476,7 @@ static int __init __plic_init(struct device_node *node,
 		int cpu;
 		unsigned long hartid;
 
-		if (of_irq_parse_one(node, i, &parent)) {
+		if (of_irq_parse_one(to_of_node(dev->fwnode), i, &parent)) {
 			pr_err("failed to parse parent for context %d.\n", i);
 			continue;
 		}
@@ -491,7 +512,7 @@ static int __init __plic_init(struct device_node *node,
 
 		/* Find parent domain and register chained handler */
 		if (!plic_parent_irq && irq_find_host(parent.np)) {
-			plic_parent_irq = irq_of_parse_and_map(node, i);
+			plic_parent_irq = irq_of_parse_and_map(to_of_node(dev->fwnode), i);
 			if (plic_parent_irq)
 				irq_set_chained_handler(plic_parent_irq,
 							plic_handle_irq);
@@ -533,20 +554,29 @@ done:
 
 	/*
 	 * We can have multiple PLIC instances so setup cpuhp state
-	 * and register syscore operations only when context handler
-	 * for current/boot CPU is present.
+	 * and register syscore operations only once after context
+	 * handlers of all online CPUs are initialized.
 	 */
-	handler = this_cpu_ptr(&plic_handlers);
-	if (handler->present && !plic_cpuhp_setup_done) {
-		cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
-				  "irqchip/sifive/plic:starting",
-				  plic_starting_cpu, plic_dying_cpu);
-		register_syscore_ops(&plic_irq_syscore_ops);
-		plic_cpuhp_setup_done = true;
+	if (!plic_cpuhp_setup_done) {
+		cpuhp_setup = true;
+		for_each_online_cpu(cpu) {
+			handler = per_cpu_ptr(&plic_handlers, cpu);
+			if (!handler->present) {
+				cpuhp_setup = false;
+				break;
+			}
+		}
+		if (cpuhp_setup) {
+			cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
+					  "irqchip/sifive/plic:starting",
+					  plic_starting_cpu, plic_dying_cpu);
+			register_syscore_ops(&plic_irq_syscore_ops);
+			plic_cpuhp_setup_done = true;
+		}
 	}
 
-	pr_info("%pOFP: mapped %d interrupts with %d handlers for"
-		" %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
+	pr_info("%pOFP: mapped %d interrupts with %d handlers for %d contexts.\n",
+		to_of_node(dev->fwnode), nr_irqs, nr_handlers, nr_contexts);
 	return 0;
 
 out_free_enable_reg:
@@ -563,20 +593,11 @@ out_free_priv:
 	return error;
 }
 
-static int __init plic_init(struct device_node *node,
-			    struct device_node *parent)
-{
-	return __plic_init(node, parent, 0);
-}
-
-IRQCHIP_DECLARE(sifive_plic, "sifive,plic-1.0.0", plic_init);
-IRQCHIP_DECLARE(riscv_plic0, "riscv,plic0", plic_init); /* for legacy systems */
-
-static int __init plic_edge_init(struct device_node *node,
-				 struct device_node *parent)
-{
-	return __plic_init(node, parent, BIT(PLIC_QUIRK_EDGE_INTERRUPT));
-}
-
-IRQCHIP_DECLARE(andestech_nceplic100, "andestech,nceplic100", plic_edge_init);
-IRQCHIP_DECLARE(thead_c900_plic, "thead,c900-plic", plic_edge_init);
+static struct platform_driver plic_driver = {
+	.driver = {
+		.name		= "riscv-plic",
+		.of_match_table	= plic_match,
+	},
+	.probe = plic_probe,
+};
+builtin_platform_driver(plic_driver);
#3 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

In reply to Anup Patel

Hi Anup,

Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Quoted 5 lines
---
 drivers/irqchip/irq-sifive-plic.c | 101 ++++++++++++++++++------------
 1 file changed, 61 insertions(+), 40 deletions(-)

This patch seems to have broken things on RZ/Five SoC, after reverting this patch I get to boot it back again on v6.9-rc2. Looks like there is some probe order issue after switching to platform driver?

Cheers,
Prabhakar

Quoted 183 lines
drivers/irqchip/irq-sifive-plic.c +61 -41
index 5b7bc4fd9517..7400a07fc479 100644
@@ -64,6 +64,7 @@
 #define PLIC_QUIRK_EDGE_INTERRUPT      0
 
 struct plic_priv {
+       struct device *dev;
        struct cpumask lmask;
        struct irq_domain *irqdomain;
        void __iomem *regs;
@@ -406,30 +407,50 @@ static int plic_starting_cpu(unsigned int cpu)
        return 0;
 }
 
-static int __init __plic_init(struct device_node *node,
-                             struct device_node *parent,
-                             unsigned long plic_quirks)
+static const struct of_device_id plic_match[] = {
+       { .compatible = "sifive,plic-1.0.0" },
+       { .compatible = "riscv,plic0" },
+       { .compatible = "andestech,nceplic100",
+         .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+       { .compatible = "thead,c900-plic",
+         .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+       {}
+};
+
+static int plic_probe(struct platform_device *pdev)
 {
        int error = 0, nr_contexts, nr_handlers = 0, i;
-       u32 nr_irqs;
-       struct plic_priv *priv;
+       struct device *dev = &pdev->dev;
+       unsigned long plic_quirks = 0;
        struct plic_handler *handler;
+       struct plic_priv *priv;
+       bool cpuhp_setup;
        unsigned int cpu;
+       u32 nr_irqs;
+
+       if (is_of_node(dev->fwnode)) {
+               const struct of_device_id *id;
+
+               id = of_match_node(plic_match, to_of_node(dev->fwnode));
+               if (id)
+                       plic_quirks = (unsigned long)id->data;
+       }
 
        priv = kzalloc(sizeof(*priv), GFP_KERNEL);
        if (!priv)
                return -ENOMEM;
 
+       priv->dev = dev;
        priv->plic_quirks = plic_quirks;
 
-       priv->regs = of_iomap(node, 0);
+       priv->regs = of_iomap(to_of_node(dev->fwnode), 0);
        if (WARN_ON(!priv->regs)) {
                error = -EIO;
                goto out_free_priv;
        }
 
        error = -EINVAL;
-       of_property_read_u32(node, "riscv,ndev", &nr_irqs);
+       of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", &nr_irqs);
        if (WARN_ON(!nr_irqs))
                goto out_iounmap;
 
@@ -439,13 +460,13 @@ static int __init __plic_init(struct device_node *node,
        if (!priv->prio_save)
                goto out_free_priority_reg;
 
-       nr_contexts = of_irq_count(node);
+       nr_contexts = of_irq_count(to_of_node(dev->fwnode));
        if (WARN_ON(!nr_contexts))
                goto out_free_priority_reg;
 
        error = -ENOMEM;
-       priv->irqdomain = irq_domain_add_linear(node, nr_irqs + 1,
-                       &plic_irqdomain_ops, priv);
+       priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
+                                               &plic_irqdomain_ops, priv);
        if (WARN_ON(!priv->irqdomain))
                goto out_free_priority_reg;
 
@@ -455,7 +476,7 @@ static int __init __plic_init(struct device_node *node,
                int cpu;
                unsigned long hartid;
 
-               if (of_irq_parse_one(node, i, &parent)) {
+               if (of_irq_parse_one(to_of_node(dev->fwnode), i, &parent)) {
                        pr_err("failed to parse parent for context %d.\n", i);
                        continue;
                }
@@ -491,7 +512,7 @@ static int __init __plic_init(struct device_node *node,
 
                /* Find parent domain and register chained handler */
                if (!plic_parent_irq && irq_find_host(parent.np)) {
-                       plic_parent_irq = irq_of_parse_and_map(node, i);
+                       plic_parent_irq = irq_of_parse_and_map(to_of_node(dev->fwnode), i);
                        if (plic_parent_irq)
                                irq_set_chained_handler(plic_parent_irq,
                                                        plic_handle_irq);
@@ -533,20 +554,29 @@ static int __init __plic_init(struct device_node *node,
 
        /*
         * We can have multiple PLIC instances so setup cpuhp state
-        * and register syscore operations only when context handler
-        * for current/boot CPU is present.
+        * and register syscore operations only once after context
+        * handlers of all online CPUs are initialized.
         */
-       handler = this_cpu_ptr(&plic_handlers);
-       if (handler->present && !plic_cpuhp_setup_done) {
-               cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
-                                 "irqchip/sifive/plic:starting",
-                                 plic_starting_cpu, plic_dying_cpu);
-               register_syscore_ops(&plic_irq_syscore_ops);
-               plic_cpuhp_setup_done = true;
+       if (!plic_cpuhp_setup_done) {
+               cpuhp_setup = true;
+               for_each_online_cpu(cpu) {
+                       handler = per_cpu_ptr(&plic_handlers, cpu);
+                       if (!handler->present) {
+                               cpuhp_setup = false;
+                               break;
+                       }
+               }
+               if (cpuhp_setup) {
+                       cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
+                                         "irqchip/sifive/plic:starting",
+                                         plic_starting_cpu, plic_dying_cpu);
+                       register_syscore_ops(&plic_irq_syscore_ops);
+                       plic_cpuhp_setup_done = true;
+               }
        }
 
-       pr_info("%pOFP: mapped %d interrupts with %d handlers for"
-               " %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
+       pr_info("%pOFP: mapped %d interrupts with %d handlers for %d contexts.\n",
+               to_of_node(dev->fwnode), nr_irqs, nr_handlers, nr_contexts);
        return 0;
 
 out_free_enable_reg:
@@ -563,20 +593,11 @@ static int __init __plic_init(struct device_node *node,
        return error;
 }
 
-static int __init plic_init(struct device_node *node,
-                           struct device_node *parent)
-{
-       return __plic_init(node, parent, 0);
-}
-
-IRQCHIP_DECLARE(sifive_plic, "sifive,plic-1.0.0", plic_init);
-IRQCHIP_DECLARE(riscv_plic0, "riscv,plic0", plic_init); /* for legacy systems */
-
-static int __init plic_edge_init(struct device_node *node,
-                                struct device_node *parent)
-{
-       return __plic_init(node, parent, BIT(PLIC_QUIRK_EDGE_INTERRUPT));
-}
-
-IRQCHIP_DECLARE(andestech_nceplic100, "andestech,nceplic100", plic_edge_init);
-IRQCHIP_DECLARE(thead_c900_plic, "thead,c900-plic", plic_edge_init);
+static struct platform_driver plic_driver = {
+       .driver = {
+               .name           = "riscv-plic",
+               .of_match_table = plic_match,
+       },
+       .probe = plic_probe,
+};
+builtin_platform_driver(plic_driver);
--

2.34.1

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author
#4 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Quoted 13 lines · Prabhakar

Hi Anup,

Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Quoted 8 lines
Quoted 5 lines
---
 drivers/irqchip/irq-sifive-plic.c | 101 ++++++++++++++++++------------
 1 file changed, 61 insertions(+), 40 deletions(-)

This patch seems to have broken things on RZ/Five SoC, after reverting this patch I get to boot it back again on v6.9-rc2. Looks like there is some probe order issue after switching to platform driver?

Yes, this is most likely related to probe ordering based on your DT.

Can you share the failing boot log and DT ?

Regards,
Anup

Quoted 187 lines

Cheers,
Prabhakar

Quoted 183 lines
drivers/irqchip/irq-sifive-plic.c +61 -41
index 5b7bc4fd9517..7400a07fc479 100644
@@ -64,6 +64,7 @@
 #define PLIC_QUIRK_EDGE_INTERRUPT      0
 
 struct plic_priv {
+       struct device *dev;
        struct cpumask lmask;
        struct irq_domain *irqdomain;
        void __iomem *regs;
@@ -406,30 +407,50 @@ static int plic_starting_cpu(unsigned int cpu)
        return 0;
 }
 
-static int __init __plic_init(struct device_node *node,
-                             struct device_node *parent,
-                             unsigned long plic_quirks)
+static const struct of_device_id plic_match[] = {
+       { .compatible = "sifive,plic-1.0.0" },
+       { .compatible = "riscv,plic0" },
+       { .compatible = "andestech,nceplic100",
+         .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+       { .compatible = "thead,c900-plic",
+         .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+       {}
+};
+
+static int plic_probe(struct platform_device *pdev)
 {
        int error = 0, nr_contexts, nr_handlers = 0, i;
-       u32 nr_irqs;
-       struct plic_priv *priv;
+       struct device *dev = &pdev->dev;
+       unsigned long plic_quirks = 0;
        struct plic_handler *handler;
+       struct plic_priv *priv;
+       bool cpuhp_setup;
        unsigned int cpu;
+       u32 nr_irqs;
+
+       if (is_of_node(dev->fwnode)) {
+               const struct of_device_id *id;
+
+               id = of_match_node(plic_match, to_of_node(dev->fwnode));
+               if (id)
+                       plic_quirks = (unsigned long)id->data;
+       }
 
        priv = kzalloc(sizeof(*priv), GFP_KERNEL);
        if (!priv)
                return -ENOMEM;
 
+       priv->dev = dev;
        priv->plic_quirks = plic_quirks;
 
-       priv->regs = of_iomap(node, 0);
+       priv->regs = of_iomap(to_of_node(dev->fwnode), 0);
        if (WARN_ON(!priv->regs)) {
                error = -EIO;
                goto out_free_priv;
        }
 
        error = -EINVAL;
-       of_property_read_u32(node, "riscv,ndev", &nr_irqs);
+       of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", &nr_irqs);
        if (WARN_ON(!nr_irqs))
                goto out_iounmap;
 
@@ -439,13 +460,13 @@ static int __init __plic_init(struct device_node *node,
        if (!priv->prio_save)
                goto out_free_priority_reg;
 
-       nr_contexts = of_irq_count(node);
+       nr_contexts = of_irq_count(to_of_node(dev->fwnode));
        if (WARN_ON(!nr_contexts))
                goto out_free_priority_reg;
 
        error = -ENOMEM;
-       priv->irqdomain = irq_domain_add_linear(node, nr_irqs + 1,
-                       &plic_irqdomain_ops, priv);
+       priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
+                                               &plic_irqdomain_ops, priv);
        if (WARN_ON(!priv->irqdomain))
                goto out_free_priority_reg;
 
@@ -455,7 +476,7 @@ static int __init __plic_init(struct device_node *node,
                int cpu;
                unsigned long hartid;
 
-               if (of_irq_parse_one(node, i, &parent)) {
+               if (of_irq_parse_one(to_of_node(dev->fwnode), i, &parent)) {
                        pr_err("failed to parse parent for context %d.\n", i);
                        continue;
                }
@@ -491,7 +512,7 @@ static int __init __plic_init(struct device_node *node,
 
                /* Find parent domain and register chained handler */
                if (!plic_parent_irq && irq_find_host(parent.np)) {
-                       plic_parent_irq = irq_of_parse_and_map(node, i);
+                       plic_parent_irq = irq_of_parse_and_map(to_of_node(dev->fwnode), i);
                        if (plic_parent_irq)
                                irq_set_chained_handler(plic_parent_irq,
                                                        plic_handle_irq);
@@ -533,20 +554,29 @@ static int __init __plic_init(struct device_node *node,
 
        /*
         * We can have multiple PLIC instances so setup cpuhp state
-        * and register syscore operations only when context handler
-        * for current/boot CPU is present.
+        * and register syscore operations only once after context
+        * handlers of all online CPUs are initialized.
         */
-       handler = this_cpu_ptr(&plic_handlers);
-       if (handler->present && !plic_cpuhp_setup_done) {
-               cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
-                                 "irqchip/sifive/plic:starting",
-                                 plic_starting_cpu, plic_dying_cpu);
-               register_syscore_ops(&plic_irq_syscore_ops);
-               plic_cpuhp_setup_done = true;
+       if (!plic_cpuhp_setup_done) {
+               cpuhp_setup = true;
+               for_each_online_cpu(cpu) {
+                       handler = per_cpu_ptr(&plic_handlers, cpu);
+                       if (!handler->present) {
+                               cpuhp_setup = false;
+                               break;
+                       }
+               }
+               if (cpuhp_setup) {
+                       cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
+                                         "irqchip/sifive/plic:starting",
+                                         plic_starting_cpu, plic_dying_cpu);
+                       register_syscore_ops(&plic_irq_syscore_ops);
+                       plic_cpuhp_setup_done = true;
+               }
        }
 
-       pr_info("%pOFP: mapped %d interrupts with %d handlers for"
-               " %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
+       pr_info("%pOFP: mapped %d interrupts with %d handlers for %d contexts.\n",
+               to_of_node(dev->fwnode), nr_irqs, nr_handlers, nr_contexts);
        return 0;
 
 out_free_enable_reg:
@@ -563,20 +593,11 @@ static int __init __plic_init(struct device_node *node,
        return error;
 }
 
-static int __init plic_init(struct device_node *node,
-                           struct device_node *parent)
-{
-       return __plic_init(node, parent, 0);
-}
-
-IRQCHIP_DECLARE(sifive_plic, "sifive,plic-1.0.0", plic_init);
-IRQCHIP_DECLARE(riscv_plic0, "riscv,plic0", plic_init); /* for legacy systems */
-
-static int __init plic_edge_init(struct device_node *node,
-                                struct device_node *parent)
-{
-       return __plic_init(node, parent, BIT(PLIC_QUIRK_EDGE_INTERRUPT));
-}
-
-IRQCHIP_DECLARE(andestech_nceplic100, "andestech,nceplic100", plic_edge_init);
-IRQCHIP_DECLARE(thead_c900_plic, "thead,c900-plic", plic_edge_init);
+static struct platform_driver plic_driver = {
+       .driver = {
+               .name           = "riscv-plic",
+               .of_match_table = plic_match,
+       },
+       .probe = plic_probe,
+};
+builtin_platform_driver(plic_driver);
--

2.34.1

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#5 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Quoted 21 lines · Anup Patel
Quoted 18 lines · Prabhakar

Hi Anup,

Quoted 14 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

---
 drivers/irqchip/irq-sifive-plic.c | 101 ++++++++++++++++++------------
 1 file changed, 61 insertions(+), 40 deletions(-)
Quoted 7 lines
Quoted 3 lines

This patch seems to have broken things on RZ/Five SoC, after reverting this patch I get to boot it back again on v6.9-rc2. Looks like there is some probe order issue after switching to platform driver?

Yes, this is most likely related to probe ordering based on your DT.

Can you share the failing boot log and DT ?

non working case, https://paste.debian.net/1312947/
after reverting, https://paste.debian.net/1312948/
(attached is the DTB)

Cheers,
Prabhakar

maintainer
#6 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Hi Prabhakar,

Quoted 26 lines · Prabhakar
Quoted 25 lines · Anup Patel
Quoted 21 lines · Prabhakar

Hi Anup,

Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Quoted 5 lines
---
 drivers/irqchip/irq-sifive-plic.c | 101 ++++++++++++++++++------------
 1 file changed, 61 insertions(+), 40 deletions(-)

This patch seems to have broken things on RZ/Five SoC, after reverting this patch I get to boot it back again on v6.9-rc2. Looks like there is some probe order issue after switching to platform driver?

Quoted 5 lines
Quoted 3 lines

Yes, this is most likely related to probe ordering based on your DT.

Can you share the failing boot log and DT ?

non working case, https://paste.debian.net/1312947/

Looks like you need to add "keep_bootcon" to your kernel command line to get a full log here.

Quoted 2 lines

after reverting, https://paste.debian.net/1312948/
(attached is the DTB)

I don't see anything suspicious between the "riscv-intc" lines and the "Fixed dependency cycle(s)" lines that looks like it would depend on the PLIC IRQ domain. Maybe there is some driver that does not handle -EPROBE_DEFER? It's hard to tell without the full log from the failure case.

Regards,
Samuel

#7 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Hi Samuel and Anup,

Quoted 30 lines · Samuel Holland

Hi Prabhakar,

Quoted 26 lines · Prabhakar
Quoted 25 lines · Anup Patel
Quoted 21 lines · Prabhakar

Hi Anup,

Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Quoted 5 lines
---
 drivers/irqchip/irq-sifive-plic.c | 101 ++++++++++++++++++------------
 1 file changed, 61 insertions(+), 40 deletions(-)

This patch seems to have broken things on RZ/Five SoC, after reverting this patch I get to boot it back again on v6.9-rc2. Looks like there is some probe order issue after switching to platform driver?

Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

Yes, this is most likely related to probe ordering based on your DT.

Can you share the failing boot log and DT ?

non working case, https://paste.debian.net/1312947/

Looks like you need to add "keep_bootcon" to your kernel command line to get a full log here.

Thanks for the pointer, that helped me to get to the root cause.

Quoted 8 lines
Quoted 2 lines

after reverting, https://paste.debian.net/1312948/
(attached is the DTB)

I don't see anything suspicious between the "riscv-intc" lines and the "Fixed dependency cycle(s)" lines that looks like it would depend on the PLIC IRQ domain. Maybe there is some driver that does not handle -EPROBE_DEFER? It's hard to tell without the full log from the failure case.

The clock required for the PLIC wasnt available during the probe of this driver. This bug got hidden when the PLIC driver was probed earlier in boot where it used an incorrect clock source. Ive created a patch which adds a missing clock for the PLIC.

Sorry for the noise!

Cheers,
Prabhakar

maintainer
#8 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

In reply to Lad, Prabhakar
Quoted 27 lines · Prabhakar
Quoted 25 lines · Anup Patel
Quoted 21 lines · Prabhakar

Hi Anup,

Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Quoted 5 lines
---
 drivers/irqchip/irq-sifive-plic.c | 101 ++++++++++++++++++------------
 1 file changed, 61 insertions(+), 40 deletions(-)

This patch seems to have broken things on RZ/Five SoC, after reverting this patch I get to boot it back again on v6.9-rc2. Looks like there is some probe order issue after switching to platform driver?

Quoted 8 lines
Quoted 3 lines

Yes, this is most likely related to probe ordering based on your DT.

Can you share the failing boot log and DT ?

non working case, https://paste.debian.net/1312947/

after reverting, https://paste.debian.net/1312948/
(attached is the DTB)

Can you add "console=ttySC0,115200" to kernel parameters and share updated boot logs ?

Regards,
Anup

maintainer
#9 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

In reply to Lad, Prabhakar
Quoted 27 lines · Prabhakar
Quoted 25 lines · Anup Patel
Quoted 21 lines · Prabhakar

Hi Anup,

Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Quoted 5 lines
---
 drivers/irqchip/irq-sifive-plic.c | 101 ++++++++++++++++++------------
 1 file changed, 61 insertions(+), 40 deletions(-)

This patch seems to have broken things on RZ/Five SoC, after reverting this patch I get to boot it back again on v6.9-rc2. Looks like there is some probe order issue after switching to platform driver?

Quoted 7 lines
Quoted 3 lines

Yes, this is most likely related to probe ordering based on your DT.

Can you share the failing boot log and DT ?

non working case, https://paste.debian.net/1312947/
after reverting, https://paste.debian.net/1312948/
(attached is the DTB)

One potential problem is that drivers/clocksource/renesas-ostm.c is probed early using TIMER_OF_DECLARE() but the timer interrupt is connected to PLIC which is probed late hence the timer probe will fail.

We have two possible options:
1) Disable OSTM nodes
2) Improve the OSTM driver to probe like a regular platform device on RISC-V

Regards,
Anup

#10 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

In reply to Anup Patel

Hi Anup,

Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Thanks for your patch, which is now commit 8ec99b033147ef3b ("irqchip/sifive-plic: Convert PLIC driver into a platform driver") in v6.9.

It looks like this conversion is causing issues on BeagleV Starlight Beta. After updating esmil/visionfive to v6.10-rc1, the kernel usually fails to boot. Adding "earlycon keep_bootcon" reveals these differences:

-riscv-plic c000000.interrupt-controller: mapped 133 interrupts with 2 handlers for 4 contexts.
+------------[ cut here ]------------
+WARNING: CPU: 0 PID: 1 at drivers/irqchip/irq-sifive-plic.c:373 plic_handle_irq+0xf2/0xf6
+Modules linked in:
+CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.10.0-rc1-starlight-02342-g0ba4c76ca0e8-dirty #323
+Hardware name: BeagleV Starlight Beta (DT)
+epc : plic_handle_irq+0xf2/0xf6
+ ra : generic_handle_domain_irq+0x1c/0x2a
+epc : ffffffff8033f994 ra : ffffffff8006319a sp : ffffffc800003f50
+ gp : ffffffff812d63f0 tp : ffffffd8800b8000 t0 : 0000000000000040
+ t1 : 0000000000000000 t2 : 0000000000001000 s0 : ffffffc800003fa0
+ s1 : 0000000000000009 a0 : ffffffd880183600 a1 : 0000000000000009
+ a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000
+ a5 : 0000000000000000 a6 : ffffffd880400248 a7 : ffffffd8804002b8
+ s2 : ffffffd9f8fac458 s3 : 0000000000000004 s4 : 0000000000000000
+ s5 : ffffffff81293f58 s6 : ffffffd88014ac00 s7 : 0000000000000004
+ s8 : ffffffc800013b2c s9 : ffffffc800013b34 s10: 0000000000000006
+ s11: ffffffd9f8fc1458 t3 : 0000000000000002 t4 : 0000000000000402
+ t5 : ffffffd8800610c0 t6 : ffffffd8800610e0
+status: 0000000200000100 badaddr: ffffffd9f8fac458 cause: 0000000000000003 +[<ffffffff8033f994>] plic_handle_irq+0xf2/0xf6 +[<ffffffff8006319a>] generic_handle_domain_irq+0x1c/0x2a +[<ffffffff8033d7aa>] riscv_intc_irq+0x26/0x60 +[<ffffffff806c92ee>] handle_riscv_irq+0x4a/0x74 +[<ffffffff806d2346>] call_on_irq_stack+0x32/0x40
+---[ end trace 0000000000000000 ]---
+Unable to handle kernel NULL pointer dereference at virtual address 0000000000000004
+Oops [#1]
+Modules linked in:
+CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W
6.10.0-rc1-starlight-02342-g0ba4c76ca0e8-dirty #323
+Hardware name: BeagleV Starlight Beta (DT)
+epc : plic_handle_irq+0x66/0xf6
+ ra : generic_handle_domain_irq+0x1c/0x2a
+epc : ffffffff8033f908 ra : ffffffff8006319a sp : ffffffc800003f50
+ gp : ffffffff812d63f0 tp : ffffffd8800b8000 t0 : 0000000000000040
+ t1 : 0000000000000000 t2 : 0000000000001000 s0 : ffffffc800003fa0
+ s1 : 0000000000000009 a0 : ffffffd880183600 a1 : 0000000000000009
+ a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000
+ a5 : ffffffff8033d72a a6 : ffffffd880400248 a7 : ffffffd8804002b8
+ s2 : ffffffd9f8fac458 s3 : 0000000000000004 s4 : ffffffd880183630
+ s5 : ffffffff81293f58 s6 : ffffffff812948a0 s7 : ffffffff80c4e660
+ s8 : ffffffff80d9eea0 s9 : ffffffc800013b34 s10: 0000000000000006
+ s11: ffffffd9f8fc1458 t3 : 0000000000000002 t4 : 0000000000000402
+ t5 : ffffffd8800610c0 t6 : ffffffd8800610e0
+status: 0000000200000100 badaddr: 0000000000000004 cause: 000000000000000d +[<ffffffff8033f908>] plic_handle_irq+0x66/0xf6 +[<ffffffff8006319a>] generic_handle_domain_irq+0x1c/0x2a +[<ffffffff8033d7aa>] riscv_intc_irq+0x26/0x60 +[<ffffffff806c92ee>] handle_riscv_irq+0x4a/0x74 +[<ffffffff806d2346>] call_on_irq_stack+0x32/0x40
+Code: 8b93 d70b 5b17 00f5 0b13 fa8b fc17 00a5 0c13 5a0c (a783) 0009 +---[ end trace 0000000000000000 ]---
+Kernel panic - not syncing: Fatal exception in interrupt
+SMP: stopping secondary CPUs
+---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

As "mapped 133 interrupts" is no longer printed, it looks like an unexpected early interrupt comes in while still in plic_probe().

Esmil suggested reverting all of:
a7fb69ffd7ce438a irqchip/sifive-plic: Avoid explicit cpumask allocation on stack abb7205794900503 irqchip/sifive-plic: Improve locking safety by using irqsave/irqrestore
95652106478030f5 irqchip/sifive-plic: Parse number of interrupts and contexts early in plic_probe()
a15587277a246c38 irqchip/sifive-plic: Cleanup PLIC contexts upon irqdomain creation failure
6c725f33d67b53f2 irqchip/sifive-plic: Use riscv_get_intc_hwnode() to get parent fwnode
b68d0ff529a939a1 irqchip/sifive-plic: Use devm_xyz() for managed allocation 25d862e183d4efeb irqchip/sifive-plic: Use dev_xyz() in-place of pr_xyz() 8ec99b033147ef3b irqchip/sifive-plic: Convert PLIC driver into a platform driver

After this, the PLIC is initialized earlier again, and this indeed seems to fix the issue for me.
Before, the kernel booted fine in only ca. 1 out of 5 tries. After the reverts, it booted 5/5.

Do you know what's going on? Is there a simpler fix?

Thanks!

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
maintainer
#11 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Hi Geert,

Quoted 99 lines · Geert Uytterhoeven

Hi Anup,

Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Thanks for your patch, which is now commit 8ec99b033147ef3b ("irqchip/sifive-plic: Convert PLIC driver into a platform driver") in v6.9.

It looks like this conversion is causing issues on BeagleV Starlight Beta. After updating esmil/visionfive to v6.10-rc1, the kernel usually fails to boot. Adding "earlycon keep_bootcon" reveals these differences:

-riscv-plic c000000.interrupt-controller: mapped 133 interrupts with 2 handlers for 4 contexts.
+------------[ cut here ]------------
+WARNING: CPU: 0 PID: 1 at drivers/irqchip/irq-sifive-plic.c:373 plic_handle_irq+0xf2/0xf6

(quoted diff) +2 -0
+Modules linked in:
+CPU: 0 PID: 1 Comm: swapper/0 Not tainted

6.10.0-rc1-starlight-02342-g0ba4c76ca0e8-dirty #323

(quoted diff) +22 -0
+Hardware name: BeagleV Starlight Beta (DT)
+epc : plic_handle_irq+0xf2/0xf6
+ ra : generic_handle_domain_irq+0x1c/0x2a
+epc : ffffffff8033f994 ra : ffffffff8006319a sp : ffffffc800003f50
+ gp : ffffffff812d63f0 tp : ffffffd8800b8000 t0 : 0000000000000040
+ t1 : 0000000000000000 t2 : 0000000000001000 s0 : ffffffc800003fa0
+ s1 : 0000000000000009 a0 : ffffffd880183600 a1 : 0000000000000009
+ a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000
+ a5 : 0000000000000000 a6 : ffffffd880400248 a7 : ffffffd8804002b8
+ s2 : ffffffd9f8fac458 s3 : 0000000000000004 s4 : 0000000000000000
+ s5 : ffffffff81293f58 s6 : ffffffd88014ac00 s7 : 0000000000000004
+ s8 : ffffffc800013b2c s9 : ffffffc800013b34 s10: 0000000000000006
+ s11: ffffffd9f8fc1458 t3 : 0000000000000002 t4 : 0000000000000402
+ t5 : ffffffd8800610c0 t6 : ffffffd8800610e0
+status: 0000000200000100 badaddr: ffffffd9f8fac458 cause: 0000000000000003
+[<ffffffff8033f994>] plic_handle_irq+0xf2/0xf6
+[<ffffffff8006319a>] generic_handle_domain_irq+0x1c/0x2a
+[<ffffffff8033d7aa>] riscv_intc_irq+0x26/0x60
+[<ffffffff806c92ee>] handle_riscv_irq+0x4a/0x74
+[<ffffffff806d2346>] call_on_irq_stack+0x32/0x40
+---[ end trace 0000000000000000 ]---
+Unable to handle kernel NULL pointer dereference at virtual address

0000000000000004

(quoted diff) +3 -0
+Oops [#1]
+Modules linked in:
+CPU: 0 PID: 1 Comm: swapper/0 Tainted: G        W

6.10.0-rc1-starlight-02342-g0ba4c76ca0e8-dirty #323

(quoted diff) +25 -0
+Hardware name: BeagleV Starlight Beta (DT)
+epc : plic_handle_irq+0x66/0xf6
+ ra : generic_handle_domain_irq+0x1c/0x2a
+epc : ffffffff8033f908 ra : ffffffff8006319a sp : ffffffc800003f50
+ gp : ffffffff812d63f0 tp : ffffffd8800b8000 t0 : 0000000000000040
+ t1 : 0000000000000000 t2 : 0000000000001000 s0 : ffffffc800003fa0
+ s1 : 0000000000000009 a0 : ffffffd880183600 a1 : 0000000000000009
+ a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000
+ a5 : ffffffff8033d72a a6 : ffffffd880400248 a7 : ffffffd8804002b8
+ s2 : ffffffd9f8fac458 s3 : 0000000000000004 s4 : ffffffd880183630
+ s5 : ffffffff81293f58 s6 : ffffffff812948a0 s7 : ffffffff80c4e660
+ s8 : ffffffff80d9eea0 s9 : ffffffc800013b34 s10: 0000000000000006
+ s11: ffffffd9f8fc1458 t3 : 0000000000000002 t4 : 0000000000000402
+ t5 : ffffffd8800610c0 t6 : ffffffd8800610e0
+status: 0000000200000100 badaddr: 0000000000000004 cause: 000000000000000d
+[<ffffffff8033f908>] plic_handle_irq+0x66/0xf6
+[<ffffffff8006319a>] generic_handle_domain_irq+0x1c/0x2a
+[<ffffffff8033d7aa>] riscv_intc_irq+0x26/0x60
+[<ffffffff806c92ee>] handle_riscv_irq+0x4a/0x74
+[<ffffffff806d2346>] call_on_irq_stack+0x32/0x40
+Code: 8b93 d70b 5b17 00f5 0b13 fa8b fc17 00a5 0c13 5a0c (a783) 0009
+---[ end trace 0000000000000000 ]---
+Kernel panic - not syncing: Fatal exception in interrupt
+SMP: stopping secondary CPUs
+---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

As "mapped 133 interrupts" is no longer printed, it looks like an unexpected early interrupt comes in while still in plic_probe().

Esmil suggested reverting all of:
a7fb69ffd7ce438a irqchip/sifive-plic: Avoid explicit cpumask allocation on stack abb7205794900503 irqchip/sifive-plic: Improve locking safety by using irqsave/irqrestore
95652106478030f5 irqchip/sifive-plic: Parse number of interrupts and contexts early in plic_probe()
a15587277a246c38 irqchip/sifive-plic: Cleanup PLIC contexts upon irqdomain creation failure
6c725f33d67b53f2 irqchip/sifive-plic: Use riscv_get_intc_hwnode() to get parent fwnode
b68d0ff529a939a1 irqchip/sifive-plic: Use devm_xyz() for managed allocation 25d862e183d4efeb irqchip/sifive-plic: Use dev_xyz() in-place of pr_xyz() 8ec99b033147ef3b irqchip/sifive-plic: Convert PLIC driver into a platform driver

Quoted 6 lines

After this, the PLIC is initialized earlier again, and this indeed seems to fix the issue for me.
Before, the kernel booted fine in only ca. 1 out of 5 tries. After the reverts, it booted 5/5.

Do you know what's going on? Is there a simpler fix?

The fact that you hit the warning indicates that plic_handle_irq() was called before handler->present was set. Previously the PLIC driver was probed very early, so it is unlikely that some peripheral already had a pending interrupt. Now, while platform device drivers would not yet be able to request interrupts (because the irqdomain is not registered yet), they could have programmed the hardware in a way that generates an interrupt. If that interrupt was enabled at the PLIC (e.g. by the bootloader), then we could expect plic_handle_irq() to be called as soon as irq_set_chained_handler() is called.

So the fix is to not call irq_set_chained_handler() until after the handlers are completely set up.

I've sent a patch doing this:
https://lore.kernel.org/linux-riscv/[email protected]/

Regards,
Samuel

#12 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Hi Samuel,

Quoted 36 lines · Samuel Holland
Quoted 22 lines · Geert Uytterhoeven
Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Thanks for your patch, which is now commit 8ec99b033147ef3b ("irqchip/sifive-plic: Convert PLIC driver into a platform driver") in v6.9.

It looks like this conversion is causing issues on BeagleV Starlight Beta. After updating esmil/visionfive to v6.10-rc1, the kernel usually fails to boot. Adding "earlycon keep_bootcon" reveals these differences:

-riscv-plic c000000.interrupt-controller: mapped 133 interrupts with 2 handlers for 4 contexts.
+------------[ cut here ]------------

Review

Quoted 3 lines
(quoted diff) +2 -0
+WARNING: CPU: 0 PID: 1 at drivers/irqchip/irq-sifive-plic.c:373
 
+Unable to handle kernel NULL pointer dereference at virtual address

The fact that you hit the warning indicates that plic_handle_irq() was called before handler->present was set. Previously the PLIC driver was probed very early, so it is unlikely that some peripheral already had a pending interrupt. Now, while platform device drivers would not yet be able to request interrupts (because the irqdomain is not registered yet), they could have programmed the hardware in a way that generates an interrupt. If that interrupt was enabled at the PLIC (e.g. by the bootloader), then we could expect plic_handle_irq() to be called as soon as irq_set_chained_handler() is called.

Quoted 5 lines

So the fix is to not call irq_set_chained_handler() until after the handlers are completely set up.

I've sent a patch doing this:
https://lore.kernel.org/linux-riscv/[email protected]/

Thanks, that fixed the issue!

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#13 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

In reply to Anup Patel
Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Hi Anup,

Sorry for the late reply to the mailing list, but ever since 6.9 where this was applied my Allwinner D1 based boards no longer boot. This is the log of my LicheeRV Dock booting plain 6.10-rc4, locking up and then rebooting due to the the watchdog timing out:

https://pastebin.com/raw/nsbzgEKW

On 6.10-rc4 I can bring the same board to boot by reverting this patch and all patches building on it. Eg.:

  git revert e306a894bd51 a7fb69ffd7ce abb720579490 \
             956521064780 a15587277a24 6c725f33d67b \
	     b68d0ff529a9 25d862e183d4 8ec99b033147

After that it boots but run into this separate issue:
https://lore.kernel.org/all/DM6PR01MB58047C810DDD5D0AE397CADFF7C22@DM6PR01MB5804.prod.exchangelabs.com/

Samuel: After the reverts above applying this also prevents my board from booting:
https://lore.kernel.org/all/[email protected]/

/Emil

Quoted 187 lines
---
 drivers/irqchip/irq-sifive-plic.c | 101 ++++++++++++++++++------------
 1 file changed, 61 insertions(+), 40 deletions(-)
drivers/irqchip/irq-sifive-plic.c +61 -41
index 5b7bc4fd9517..7400a07fc479 100644
@@ -64,6 +64,7 @@
 #define PLIC_QUIRK_EDGE_INTERRUPT	0
 
 struct plic_priv {
+	struct device *dev;
 	struct cpumask lmask;
 	struct irq_domain *irqdomain;
 	void __iomem *regs;
@@ -406,30 +407,50 @@ static int plic_starting_cpu(unsigned int cpu)
 	return 0;
 }
 
-static int __init __plic_init(struct device_node *node,
-			      struct device_node *parent,
-			      unsigned long plic_quirks)
+static const struct of_device_id plic_match[] = {
+	{ .compatible = "sifive,plic-1.0.0" },
+	{ .compatible = "riscv,plic0" },
+	{ .compatible = "andestech,nceplic100",
+	  .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+	{ .compatible = "thead,c900-plic",
+	  .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+	{}
+};
+
+static int plic_probe(struct platform_device *pdev)
 {
 	int error = 0, nr_contexts, nr_handlers = 0, i;
-	u32 nr_irqs;
-	struct plic_priv *priv;
+	struct device *dev = &pdev->dev;
+	unsigned long plic_quirks = 0;
 	struct plic_handler *handler;
+	struct plic_priv *priv;
+	bool cpuhp_setup;
 	unsigned int cpu;
+	u32 nr_irqs;
+
+	if (is_of_node(dev->fwnode)) {
+		const struct of_device_id *id;
+
+		id = of_match_node(plic_match, to_of_node(dev->fwnode));
+		if (id)
+			plic_quirks = (unsigned long)id->data;
+	}
 
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
 	if (!priv)
 		return -ENOMEM;
 
+	priv->dev = dev;
 	priv->plic_quirks = plic_quirks;
 
-	priv->regs = of_iomap(node, 0);
+	priv->regs = of_iomap(to_of_node(dev->fwnode), 0);
 	if (WARN_ON(!priv->regs)) {
 		error = -EIO;
 		goto out_free_priv;
 	}
 
 	error = -EINVAL;
-	of_property_read_u32(node, "riscv,ndev", &nr_irqs);
+	of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", &nr_irqs);
 	if (WARN_ON(!nr_irqs))
 		goto out_iounmap;
 
@@ -439,13 +460,13 @@ static int __init __plic_init(struct device_node *node,
 	if (!priv->prio_save)
 		goto out_free_priority_reg;
 
-	nr_contexts = of_irq_count(node);
+	nr_contexts = of_irq_count(to_of_node(dev->fwnode));
 	if (WARN_ON(!nr_contexts))
 		goto out_free_priority_reg;
 
 	error = -ENOMEM;
-	priv->irqdomain = irq_domain_add_linear(node, nr_irqs + 1,
-			&plic_irqdomain_ops, priv);
+	priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
+						&plic_irqdomain_ops, priv);
 	if (WARN_ON(!priv->irqdomain))
 		goto out_free_priority_reg;
 
@@ -455,7 +476,7 @@ static int __init __plic_init(struct device_node *node,
 		int cpu;
 		unsigned long hartid;
 
-		if (of_irq_parse_one(node, i, &parent)) {
+		if (of_irq_parse_one(to_of_node(dev->fwnode), i, &parent)) {
 			pr_err("failed to parse parent for context %d.\n", i);
 			continue;
 		}
@@ -491,7 +512,7 @@ static int __init __plic_init(struct device_node *node,
 
 		/* Find parent domain and register chained handler */
 		if (!plic_parent_irq && irq_find_host(parent.np)) {
-			plic_parent_irq = irq_of_parse_and_map(node, i);
+			plic_parent_irq = irq_of_parse_and_map(to_of_node(dev->fwnode), i);
 			if (plic_parent_irq)
 				irq_set_chained_handler(plic_parent_irq,
 							plic_handle_irq);
@@ -533,20 +554,29 @@ static int __init __plic_init(struct device_node *node,
 
 	/*
 	 * We can have multiple PLIC instances so setup cpuhp state
-	 * and register syscore operations only when context handler
-	 * for current/boot CPU is present.
+	 * and register syscore operations only once after context
+	 * handlers of all online CPUs are initialized.
 	 */
-	handler = this_cpu_ptr(&plic_handlers);
-	if (handler->present && !plic_cpuhp_setup_done) {
-		cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
-				  "irqchip/sifive/plic:starting",
-				  plic_starting_cpu, plic_dying_cpu);
-		register_syscore_ops(&plic_irq_syscore_ops);
-		plic_cpuhp_setup_done = true;
+	if (!plic_cpuhp_setup_done) {
+		cpuhp_setup = true;
+		for_each_online_cpu(cpu) {
+			handler = per_cpu_ptr(&plic_handlers, cpu);
+			if (!handler->present) {
+				cpuhp_setup = false;
+				break;
+			}
+		}
+		if (cpuhp_setup) {
+			cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
+					  "irqchip/sifive/plic:starting",
+					  plic_starting_cpu, plic_dying_cpu);
+			register_syscore_ops(&plic_irq_syscore_ops);
+			plic_cpuhp_setup_done = true;
+		}
 	}
 
-	pr_info("%pOFP: mapped %d interrupts with %d handlers for"
-		" %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
+	pr_info("%pOFP: mapped %d interrupts with %d handlers for %d contexts.\n",
+		to_of_node(dev->fwnode), nr_irqs, nr_handlers, nr_contexts);
 	return 0;
 
 out_free_enable_reg:
@@ -563,20 +593,11 @@ static int __init __plic_init(struct device_node *node,
 	return error;
 }
 
-static int __init plic_init(struct device_node *node,
-			    struct device_node *parent)
-{
-	return __plic_init(node, parent, 0);
-}
-
-IRQCHIP_DECLARE(sifive_plic, "sifive,plic-1.0.0", plic_init);
-IRQCHIP_DECLARE(riscv_plic0, "riscv,plic0", plic_init); /* for legacy systems */
-
-static int __init plic_edge_init(struct device_node *node,
-				 struct device_node *parent)
-{
-	return __plic_init(node, parent, BIT(PLIC_QUIRK_EDGE_INTERRUPT));
-}
-
-IRQCHIP_DECLARE(andestech_nceplic100, "andestech,nceplic100", plic_edge_init);
-IRQCHIP_DECLARE(thead_c900_plic, "thead,c900-plic", plic_edge_init);
+static struct platform_driver plic_driver = {
+	.driver = {
+		.name		= "riscv-plic",
+		.of_match_table	= plic_match,
+	},
+	.probe = plic_probe,
+};
+builtin_platform_driver(plic_driver);
--

2.34.1

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

author
#14 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Quoted 19 lines · Emil Renner Berthing
Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Hi Anup,

Sorry for the late reply to the mailing list, but ever since 6.9 where this was applied my Allwinner D1 based boards no longer boot. This is the log of my LicheeRV Dock booting plain 6.10-rc4, locking up and then rebooting due to the the watchdog timing out:

Quoted 8 lines

https://pastebin.com/raw/nsbzgEKW

On 6.10-rc4 I can bring the same board to boot by reverting this patch and all patches building on it. Eg.:

  git revert e306a894bd51 a7fb69ffd7ce abb720579490 \
             956521064780 a15587277a24 6c725f33d67b \
             b68d0ff529a9 25d862e183d4 8ec99b033147

Does your board boot with only SBI timer driver enabled ?

If yes then probing of Allwinner timer driver needs to be fixed since it depends on PLIC.

Regards,
Anup

Quoted 197 lines

After that it boots but run into this separate issue:
https://lore.kernel.org/all/DM6PR01MB58047C810DDD5D0AE397CADFF7C22@DM6PR01MB5804.prod.exchangelabs.com/

Samuel: After the reverts above applying this also prevents my board from booting:
https://lore.kernel.org/all/[email protected]/

/Emil

Quoted 187 lines
---
 drivers/irqchip/irq-sifive-plic.c | 101 ++++++++++++++++++------------
 1 file changed, 61 insertions(+), 40 deletions(-)
drivers/irqchip/irq-sifive-plic.c +61 -41
index 5b7bc4fd9517..7400a07fc479 100644
@@ -64,6 +64,7 @@
 #define PLIC_QUIRK_EDGE_INTERRUPT    0
 
 struct plic_priv {
+     struct device *dev;
      struct cpumask lmask;
      struct irq_domain *irqdomain;
      void __iomem *regs;
@@ -406,30 +407,50 @@ static int plic_starting_cpu(unsigned int cpu)
      return 0;
 }
 
-static int __init __plic_init(struct device_node *node,
-                           struct device_node *parent,
-                           unsigned long plic_quirks)
+static const struct of_device_id plic_match[] = {
+     { .compatible = "sifive,plic-1.0.0" },
+     { .compatible = "riscv,plic0" },
+     { .compatible = "andestech,nceplic100",
+       .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+     { .compatible = "thead,c900-plic",
+       .data = (const void *)BIT(PLIC_QUIRK_EDGE_INTERRUPT) },
+     {}
+};
+
+static int plic_probe(struct platform_device *pdev)
 {
      int error = 0, nr_contexts, nr_handlers = 0, i;
-     u32 nr_irqs;
-     struct plic_priv *priv;
+     struct device *dev = &pdev->dev;
+     unsigned long plic_quirks = 0;
      struct plic_handler *handler;
+     struct plic_priv *priv;
+     bool cpuhp_setup;
      unsigned int cpu;
+     u32 nr_irqs;
+
+     if (is_of_node(dev->fwnode)) {
+             const struct of_device_id *id;
+
+             id = of_match_node(plic_match, to_of_node(dev->fwnode));
+             if (id)
+                     plic_quirks = (unsigned long)id->data;
+     }
 
      priv = kzalloc(sizeof(*priv), GFP_KERNEL);
      if (!priv)
              return -ENOMEM;
 
+     priv->dev = dev;
      priv->plic_quirks = plic_quirks;
 
-     priv->regs = of_iomap(node, 0);
+     priv->regs = of_iomap(to_of_node(dev->fwnode), 0);
      if (WARN_ON(!priv->regs)) {
              error = -EIO;
              goto out_free_priv;
      }
 
      error = -EINVAL;
-     of_property_read_u32(node, "riscv,ndev", &nr_irqs);
+     of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", &nr_irqs);
      if (WARN_ON(!nr_irqs))
              goto out_iounmap;
 
@@ -439,13 +460,13 @@ static int __init __plic_init(struct device_node *node,
      if (!priv->prio_save)
              goto out_free_priority_reg;
 
-     nr_contexts = of_irq_count(node);
+     nr_contexts = of_irq_count(to_of_node(dev->fwnode));
      if (WARN_ON(!nr_contexts))
              goto out_free_priority_reg;
 
      error = -ENOMEM;
-     priv->irqdomain = irq_domain_add_linear(node, nr_irqs + 1,
-                     &plic_irqdomain_ops, priv);
+     priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
+                                             &plic_irqdomain_ops, priv);
      if (WARN_ON(!priv->irqdomain))
              goto out_free_priority_reg;
 
@@ -455,7 +476,7 @@ static int __init __plic_init(struct device_node *node,
              int cpu;
              unsigned long hartid;
 
-             if (of_irq_parse_one(node, i, &parent)) {
+             if (of_irq_parse_one(to_of_node(dev->fwnode), i, &parent)) {
                      pr_err("failed to parse parent for context %d.\n", i);
                      continue;
              }
@@ -491,7 +512,7 @@ static int __init __plic_init(struct device_node *node,
 
              /* Find parent domain and register chained handler */
              if (!plic_parent_irq && irq_find_host(parent.np)) {
-                     plic_parent_irq = irq_of_parse_and_map(node, i);
+                     plic_parent_irq = irq_of_parse_and_map(to_of_node(dev->fwnode), i);
                      if (plic_parent_irq)
                              irq_set_chained_handler(plic_parent_irq,
                                                      plic_handle_irq);
@@ -533,20 +554,29 @@ static int __init __plic_init(struct device_node *node,
 
      /*
       * We can have multiple PLIC instances so setup cpuhp state
-      * and register syscore operations only when context handler
-      * for current/boot CPU is present.
+      * and register syscore operations only once after context
+      * handlers of all online CPUs are initialized.
       */
-     handler = this_cpu_ptr(&plic_handlers);
-     if (handler->present && !plic_cpuhp_setup_done) {
-             cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
-                               "irqchip/sifive/plic:starting",
-                               plic_starting_cpu, plic_dying_cpu);
-             register_syscore_ops(&plic_irq_syscore_ops);
-             plic_cpuhp_setup_done = true;
+     if (!plic_cpuhp_setup_done) {
+             cpuhp_setup = true;
+             for_each_online_cpu(cpu) {
+                     handler = per_cpu_ptr(&plic_handlers, cpu);
+                     if (!handler->present) {
+                             cpuhp_setup = false;
+                             break;
+                     }
+             }
+             if (cpuhp_setup) {
+                     cpuhp_setup_state(CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
+                                       "irqchip/sifive/plic:starting",
+                                       plic_starting_cpu, plic_dying_cpu);
+                     register_syscore_ops(&plic_irq_syscore_ops);
+                     plic_cpuhp_setup_done = true;
+             }
      }
 
-     pr_info("%pOFP: mapped %d interrupts with %d handlers for"
-             " %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
+     pr_info("%pOFP: mapped %d interrupts with %d handlers for %d contexts.\n",
+             to_of_node(dev->fwnode), nr_irqs, nr_handlers, nr_contexts);
      return 0;
 
 out_free_enable_reg:
@@ -563,20 +593,11 @@ static int __init __plic_init(struct device_node *node,
      return error;
 }
 
-static int __init plic_init(struct device_node *node,
-                         struct device_node *parent)
-{
-     return __plic_init(node, parent, 0);
-}
-
-IRQCHIP_DECLARE(sifive_plic, "sifive,plic-1.0.0", plic_init);
-IRQCHIP_DECLARE(riscv_plic0, "riscv,plic0", plic_init); /* for legacy systems */
-
-static int __init plic_edge_init(struct device_node *node,
-                              struct device_node *parent)
-{
-     return __plic_init(node, parent, BIT(PLIC_QUIRK_EDGE_INTERRUPT));
-}
-
-IRQCHIP_DECLARE(andestech_nceplic100, "andestech,nceplic100", plic_edge_init);
-IRQCHIP_DECLARE(thead_c900_plic, "thead,c900-plic", plic_edge_init);
+static struct platform_driver plic_driver = {
+     .driver = {
+             .name           = "riscv-plic",
+             .of_match_table = plic_match,
+     },
+     .probe = plic_probe,
+};
+builtin_platform_driver(plic_driver);
--

2.34.1

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#15 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Quoted 23 lines · Anup Patel
Quoted 21 lines · Emil Renner Berthing
Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Hi Anup,

Sorry for the late reply to the mailing list, but ever since 6.9 where this was applied my Allwinner D1 based boards no longer boot. This is the log of my LicheeRV Dock booting plain 6.10-rc4, locking up and then rebooting due to the the watchdog timing out:

https://pastebin.com/raw/nsbzgEKW

Quoted 8 lines
Quoted 6 lines

On 6.10-rc4 I can bring the same board to boot by reverting this patch and all patches building on it. Eg.:

  git revert e306a894bd51 a7fb69ffd7ce abb720579490 \
             956521064780 a15587277a24 6c725f33d67b \
             b68d0ff529a9 25d862e183d4 8ec99b033147

Does your board boot with only SBI timer driver enabled ?

I'm not 100% sure this is what you mean, but with this change I can disable CONFIG_SUN4I_TIMER:

arch/riscv/Kconfig.socs +0 -1
index f51bb24bc84c..0143545348eb 100644
@@ -39,7 +39,6 @@ config ARCH_SUNXI
        bool "Allwinner sun20i SoCs"
        depends on MMU && !XIP_KERNEL
        select ERRATA_THEAD
-       select SUN4I_TIMER
        help
          This enables support for Allwinner sun20i platform hardware,
          including boards based on the D1 and D1s SoCs.

But unfortunately the board still doesn't boot:
https://pastebin.com/raw/AwRxcfeu

/Emil

maintainer
#16 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Quoted 51 lines · Emil Renner Berthing
Quoted 23 lines · Anup Patel
Quoted 21 lines · Emil Renner Berthing
Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Hi Anup,

Sorry for the late reply to the mailing list, but ever since 6.9 where this was applied my Allwinner D1 based boards no longer boot. This is the log of my LicheeRV Dock booting plain 6.10-rc4, locking up and then rebooting due to the the watchdog timing out:

https://pastebin.com/raw/nsbzgEKW

Quoted 8 lines
Quoted 6 lines

On 6.10-rc4 I can bring the same board to boot by reverting this patch and all patches building on it. Eg.:

  git revert e306a894bd51 a7fb69ffd7ce abb720579490 \
             956521064780 a15587277a24 6c725f33d67b \
             b68d0ff529a9 25d862e183d4 8ec99b033147

Does your board boot with only SBI timer driver enabled ?

I'm not 100% sure this is what you mean, but with this change I can disable CONFIG_SUN4I_TIMER:

arch/riscv/Kconfig.socs +0 -1
index f51bb24bc84c..0143545348eb 100644
@@ -39,7 +39,6 @@ config ARCH_SUNXI
        bool "Allwinner sun20i SoCs"
        depends on MMU && !XIP_KERNEL
        select ERRATA_THEAD
-       select SUN4I_TIMER
        help
          This enables support for Allwinner sun20i platform hardware,
          including boards based on the D1 and D1s SoCs.
Quoted 2 lines

But unfortunately the board still doesn't boot:
https://pastebin.com/raw/AwRxcfeu

I think we should enable debug prints in DD core and see which device is not getting probed due to lack of a provider.

Just add "#define DEBUG" at the top in drivers/base/core.c and boot again with "loglevel=8" kernel parameter (along with the above change).

Regards,
Anup

#17 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Quoted 59 lines · Anup Patel
Quoted 51 lines · Emil Renner Berthing
Quoted 23 lines · Anup Patel
Quoted 21 lines · Emil Renner Berthing
Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Hi Anup,

Sorry for the late reply to the mailing list, but ever since 6.9 where this was applied my Allwinner D1 based boards no longer boot. This is the log of my LicheeRV Dock booting plain 6.10-rc4, locking up and then rebooting due to the the watchdog timing out:

https://pastebin.com/raw/nsbzgEKW

Quoted 8 lines
Quoted 6 lines

On 6.10-rc4 I can bring the same board to boot by reverting this patch and all patches building on it. Eg.:

  git revert e306a894bd51 a7fb69ffd7ce abb720579490 \
             956521064780 a15587277a24 6c725f33d67b \
             b68d0ff529a9 25d862e183d4 8ec99b033147

Does your board boot with only SBI timer driver enabled ?

I'm not 100% sure this is what you mean, but with this change I can disable CONFIG_SUN4I_TIMER:

arch/riscv/Kconfig.socs +0 -1
index f51bb24bc84c..0143545348eb 100644
@@ -39,7 +39,6 @@ config ARCH_SUNXI
        bool "Allwinner sun20i SoCs"
        depends on MMU && !XIP_KERNEL
        select ERRATA_THEAD
-       select SUN4I_TIMER
        help
          This enables support for Allwinner sun20i platform hardware,
          including boards based on the D1 and D1s SoCs.
Quoted 2 lines

But unfortunately the board still doesn't boot:
https://pastebin.com/raw/AwRxcfeu

I think we should enable debug prints in DD core and see which device is not getting probed due to lack of a provider.

Quoted 3 lines

Just add "#define DEBUG" at the top in drivers/base/core.c and boot again with "loglevel=8" kernel parameter (along with the above change).

With the above changes this is what I get:
https://pastebin.com/raw/JfRrEahT

/Emil

author
#18 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Quoted 61 lines · Emil Renner Berthing
Quoted 59 lines · Anup Patel
Quoted 51 lines · Emil Renner Berthing
Quoted 23 lines · Anup Patel
Quoted 21 lines · Emil Renner Berthing
Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Hi Anup,

Sorry for the late reply to the mailing list, but ever since 6.9 where this was applied my Allwinner D1 based boards no longer boot. This is the log of my LicheeRV Dock booting plain 6.10-rc4, locking up and then rebooting due to the the watchdog timing out:

https://pastebin.com/raw/nsbzgEKW

Quoted 8 lines
Quoted 6 lines

On 6.10-rc4 I can bring the same board to boot by reverting this patch and all patches building on it. Eg.:

  git revert e306a894bd51 a7fb69ffd7ce abb720579490 \
             956521064780 a15587277a24 6c725f33d67b \
             b68d0ff529a9 25d862e183d4 8ec99b033147

Does your board boot with only SBI timer driver enabled ?

I'm not 100% sure this is what you mean, but with this change I can disable CONFIG_SUN4I_TIMER:

arch/riscv/Kconfig.socs +0 -1
index f51bb24bc84c..0143545348eb 100644
@@ -39,7 +39,6 @@ config ARCH_SUNXI
        bool "Allwinner sun20i SoCs"
        depends on MMU && !XIP_KERNEL
        select ERRATA_THEAD
-       select SUN4I_TIMER
        help
          This enables support for Allwinner sun20i platform hardware,
          including boards based on the D1 and D1s SoCs.
Quoted 2 lines

But unfortunately the board still doesn't boot:
https://pastebin.com/raw/AwRxcfeu

I think we should enable debug prints in DD core and see which device is not getting probed due to lack of a provider.

Quoted 6 lines
Quoted 3 lines

Just add "#define DEBUG" at the top in drivers/base/core.c and boot again with "loglevel=8" kernel parameter (along with the above change).

With the above changes this is what I get:
https://pastebin.com/raw/JfRrEahT

You should see prints like below which show producer consumer relation:

[    0.214589] /soc/rtc@101000 Linked as a fwnode consumer to /soc/plic@c000000
[    0.214966] /soc/serial@10000000 Linked as a fwnode consumer to
/soc/plic@c000000
[    0.215443] /soc/virtio_mmio@10008000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.216041] /soc/virtio_mmio@10007000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.216482] /soc/virtio_mmio@10006000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.216868] /soc/virtio_mmio@10005000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.217477] /soc/virtio_mmio@10004000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.217949] /soc/virtio_mmio@10003000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.218595] /soc/virtio_mmio@10002000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.219280] /soc/virtio_mmio@10001000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.219908] /soc/plic@c000000 Linked as a fwnode consumer to
/cpus/cpu@0/interrupt-controller
[    0.220800] /soc/plic@c000000 Linked as a fwnode consumer to
/cpus/cpu@1/interrupt-controller
[    0.221323] /soc/plic@c000000 Linked as a fwnode consumer to
/cpus/cpu@2/interrupt-controller
[    0.221838] /soc/plic@c000000 Linked as a fwnode consumer to
/cpus/cpu@3/interrupt-controller
[    0.222347] /soc/clint@2000000 Linked as a fwnode consumer to
/cpus/cpu@0/interrupt-controller
[    0.222769] /soc/clint@2000000 Linked as a fwnode consumer to
/cpus/cpu@1/interrupt-controller
[    0.223864] /soc/clint@2000000 Linked as a fwnode consumer to
/cpus/cpu@2/interrupt-controller
[    0.224370] /soc/clint@2000000 Linked as a fwnode consumer to
/cpus/cpu@3/interrupt-controller
[    0.225217] /soc/pci@30000000 Linked as a fwnode consumer to

/soc/plic@c000000

To get further prints, I suggest enabling SBI_HVC console and use "console=hvc0" as kernel parameter.

Regards,
Anup

#19 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Quoted 109 lines · Anup Patel
Quoted 61 lines · Emil Renner Berthing
Quoted 59 lines · Anup Patel
Quoted 51 lines · Emil Renner Berthing
Quoted 23 lines · Anup Patel
Quoted 21 lines · Emil Renner Berthing
Quoted 9 lines · Anup Patel

The PLIC driver does not require very early initialization so convert it into a platform driver.

After conversion, the PLIC driver is probed after CPUs are brought-up so setup cpuhp state after context handler of all online CPUs are initialized otherwise PLIC driver crashes for platforms with multiple PLIC instances.

Hi Anup,

Sorry for the late reply to the mailing list, but ever since 6.9 where this was applied my Allwinner D1 based boards no longer boot. This is the log of my LicheeRV Dock booting plain 6.10-rc4, locking up and then rebooting due to the the watchdog timing out:

https://pastebin.com/raw/nsbzgEKW

Quoted 8 lines
Quoted 6 lines

On 6.10-rc4 I can bring the same board to boot by reverting this patch and all patches building on it. Eg.:

  git revert e306a894bd51 a7fb69ffd7ce abb720579490 \
             956521064780 a15587277a24 6c725f33d67b \
             b68d0ff529a9 25d862e183d4 8ec99b033147

Does your board boot with only SBI timer driver enabled ?

I'm not 100% sure this is what you mean, but with this change I can disable CONFIG_SUN4I_TIMER:

arch/riscv/Kconfig.socs +0 -1
index f51bb24bc84c..0143545348eb 100644
@@ -39,7 +39,6 @@ config ARCH_SUNXI
        bool "Allwinner sun20i SoCs"
        depends on MMU && !XIP_KERNEL
        select ERRATA_THEAD
-       select SUN4I_TIMER
        help
          This enables support for Allwinner sun20i platform hardware,
          including boards based on the D1 and D1s SoCs.
Quoted 2 lines

But unfortunately the board still doesn't boot:
https://pastebin.com/raw/AwRxcfeu

I think we should enable debug prints in DD core and see which device is not getting probed due to lack of a provider.

Quoted 6 lines
Quoted 3 lines

Just add "#define DEBUG" at the top in drivers/base/core.c and boot again with "loglevel=8" kernel parameter (along with the above change).

With the above changes this is what I get:
https://pastebin.com/raw/JfRrEahT

You should see prints like below which show producer consumer relation:

[    0.214589] /soc/rtc@101000 Linked as a fwnode consumer to /soc/plic@c000000
[    0.214966] /soc/serial@10000000 Linked as a fwnode consumer to
/soc/plic@c000000
[    0.215443] /soc/virtio_mmio@10008000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.216041] /soc/virtio_mmio@10007000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.216482] /soc/virtio_mmio@10006000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.216868] /soc/virtio_mmio@10005000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.217477] /soc/virtio_mmio@10004000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.217949] /soc/virtio_mmio@10003000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.218595] /soc/virtio_mmio@10002000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.219280] /soc/virtio_mmio@10001000 Linked as a fwnode consumer
to /soc/plic@c000000
[    0.219908] /soc/plic@c000000 Linked as a fwnode consumer to
/cpus/cpu@0/interrupt-controller
[    0.220800] /soc/plic@c000000 Linked as a fwnode consumer to
/cpus/cpu@1/interrupt-controller
[    0.221323] /soc/plic@c000000 Linked as a fwnode consumer to
/cpus/cpu@2/interrupt-controller
[    0.221838] /soc/plic@c000000 Linked as a fwnode consumer to
/cpus/cpu@3/interrupt-controller
[    0.222347] /soc/clint@2000000 Linked as a fwnode consumer to
/cpus/cpu@0/interrupt-controller
[    0.222769] /soc/clint@2000000 Linked as a fwnode consumer to
/cpus/cpu@1/interrupt-controller
[    0.223864] /soc/clint@2000000 Linked as a fwnode consumer to
/cpus/cpu@2/interrupt-controller
[    0.224370] /soc/clint@2000000 Linked as a fwnode consumer to
/cpus/cpu@3/interrupt-controller
[    0.225217] /soc/pci@30000000 Linked as a fwnode consumer to
Quoted 7 lines

/soc/plic@c000000

To get further prints, I suggest enabling SBI_HVC console and use "console=hvc0" as kernel parameter.

Regards,
Anup

I did some follow-up research into this. The hanging after "cpuidle:
using governor menu" is due to being stuck inside of check_unaligned_access(). Specifically, there is a check that appears to be waiting for jiffies to start ticking, but they never do:

while ((now = jiffies) == start_jiffies)

	cpu_relax();

jiffies is fixed at 0xfffedb08, effectively making this a while(true) loop. This happens with and without SUN4I_TIMER.

This hang unfortunately happens before the "Linked as a fwnode consumer" print statements start.

After bypassing this with the configs

CONFIG_NONPORTABLE=y
CONFIG_RISCV_EFFICIENT_UNALIGNED_ACCESS=y

A new warning is tripped:

[    1.015134] No max_rate, ignoring min_rate of clock 9 - pll-video0
[    1.021322] WARNING: CPU: 0 PID: 1 at drivers/clk/sunxi-ng/ccu_common.c:155 sunxi_ccu_probe+0x144/0x1a2
[    1.021351] Modules linked in:
[    1.021360] CPU: 0 PID: 1 Comm: swapper Tainted: G        W          6.10.0-rc6 #1
[    1.021372] Hardware name: Allwinner D1 Nezha (changed) (DT)
[    1.021377] epc : sunxi_ccu_probe+0x144/0x1a2
[    1.021386]  ra : sunxi_ccu_probe+0x144/0x1a2
[    1.021397] epc : ffffffff80405a50 ra : ffffffff80405a50 sp : ffffffc80000bb80
[    1.021406]  gp : ffffffff815f69c8 tp : ffffffd801df8000 t0 : 6100000000000000
[    1.021414]  t1 : 000000000000004e t2 : 61725f78616d206f s0 : ffffffc80000bbe0
[    1.021422]  s1 : ffffffff81537498 a0 : 0000000000000036 a1 : 000000000000054b
[    1.021430]  a2 : 00000000ffffefff a3 : 0000000000000000 a4 : ffffffff8141f628
[    1.021438]  a5 : 0000000000000000 a6 : 0000000000000000 a7 : 000000004442434e
[    1.021446]  s2 : 0000000000000009 s3 : 0000000000000000 s4 : ffffffd801dc9010
[    1.021453]  s5 : ffffffd802428a00 s6 : ffffffd83ffdcf20 s7 : ffffffc800015000
[    1.021462]  s8 : ffffffff80e55360 s9 : ffffffff81034598 s10: 0000000000000000
[    1.021470]  s11: 0000000000000000 t3 : ffffffff8160a257 t4 : ffffffff8160a257
[    1.021478]  t5 : ffffffff8160a258 t6 : ffffffc80000b990
[    1.021485] status: 0000000200000120 badaddr: 0000000000000000 cause: 0000000000000003
[    1.021493] [<ffffffff80405a50>] sunxi_ccu_probe+0x144/0x1a2
[    1.021510] [<ffffffff80405af6>] devm_sunxi_ccu_probe+0x48/0x82
[    1.021524] [<ffffffff80409020>] sun20i_d1_ccu_probe+0xba/0xfa
[    1.021546] [<ffffffff804a8b40>] platform_probe+0x4e/0xa6
[    1.021562] [<ffffffff808d81ee>] really_probe+0x10a/0x2dc
[    1.021581] [<ffffffff808d8472>] __driver_probe_device.part.0+0xb2/0xe8
[    1.021597] [<ffffffff804a67aa>] driver_probe_device+0x7a/0xca
[    1.021621] [<ffffffff804a6912>] __driver_attach+0x52/0x164
[    1.021638] [<ffffffff804a4c7a>] bus_for_each_dev+0x56/0x8c
[    1.021656] [<ffffffff804a6382>] driver_attach+0x1a/0x22
[    1.021673] [<ffffffff804a5c18>] bus_add_driver+0xea/0x1d8
[    1.021690] [<ffffffff804a7852>] driver_register+0x3e/0xd8
[    1.021709] [<ffffffff804a8826>] __platform_driver_register+0x1c/0x24
Emil[    1.021725] [<ffffffff80a17488>] sun20i_d1_ccu_driver_init+0x1a/0x22
[    1.021746] [<ffffffff800026ae>] do_one_initcall+0x46/0x1be
[    1.021762] [<ffffffff80a00ef2>] kernel_init_freeable+0x1c6/0x220
[    1.021791] [<ffffffff808e0b46>] kernel_init+0x1e/0x112

Linked as a fwnode consumer[ 1.021807] [<ffffffff808e7632>] ret_from_fork+0xe/0x1c

The warning is not fatal, so execution continues until hanging at

[    2.110919] printk: legacy console [ttyS0] disabled
[    2.136911] 2500000.serial: ttyS0 at MMIO 0x2500000 (irq = 205, base_baud = 1500000) is a 16550A�[    2.145674] printk: legacy console [ttyS0] enabled
[    2.145674] printk: legacy console [ttyS0] enabled
[    2.155095] printk: legacy bootconsole [sbi0] disabled
[    2.155095] printk: legacy bootconsole [sbi0] disabled

I have not been able to discover why it hangs here.

The clock is somehow relying on the previous behavior of this PLIC driver.

- Charlie

Quoted 5 lines

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#20 lore raw

Re: [PATCH v14 01/18] irqchip/sifive-plic: Convert PLIC driver into a platform driver

Quoted 45 lines · Charlie Jenkins

CONFIG_NONPORTABLE=y
CONFIG_RISCV_EFFICIENT_UNALIGNED_ACCESS=y

A new warning is tripped:

[    1.015134] No max_rate, ignoring min_rate of clock 9 - pll-video0
[    1.021322] WARNING: CPU: 0 PID: 1 at drivers/clk/sunxi-ng/ccu_common.c:155 sunxi_ccu_probe+0x144/0x1a2
[    1.021351] Modules linked in:
[    1.021360] CPU: 0 PID: 1 Comm: swapper Tainted: G        W          6.10.0-rc6 #1
[    1.021372] Hardware name: Allwinner D1 Nezha (changed) (DT)
[    1.021377] epc : sunxi_ccu_probe+0x144/0x1a2
[    1.021386]  ra : sunxi_ccu_probe+0x144/0x1a2
[    1.021397] epc : ffffffff80405a50 ra : ffffffff80405a50 sp : ffffffc80000bb80
[    1.021406]  gp : ffffffff815f69c8 tp : ffffffd801df8000 t0 : 6100000000000000
[    1.021414]  t1 : 000000000000004e t2 : 61725f78616d206f s0 : ffffffc80000bbe0
[    1.021422]  s1 : ffffffff81537498 a0 : 0000000000000036 a1 : 000000000000054b
[    1.021430]  a2 : 00000000ffffefff a3 : 0000000000000000 a4 : ffffffff8141f628
[    1.021438]  a5 : 0000000000000000 a6 : 0000000000000000 a7 : 000000004442434e
[    1.021446]  s2 : 0000000000000009 s3 : 0000000000000000 s4 : ffffffd801dc9010
[    1.021453]  s5 : ffffffd802428a00 s6 : ffffffd83ffdcf20 s7 : ffffffc800015000
[    1.021462]  s8 : ffffffff80e55360 s9 : ffffffff81034598 s10: 0000000000000000
[    1.021470]  s11: 0000000000000000 t3 : ffffffff8160a257 t4 : ffffffff8160a257
[    1.021478]  t5 : ffffffff8160a258 t6 : ffffffc80000b990
[    1.021485] status: 0000000200000120 badaddr: 0000000000000000 cause: 0000000000000003
[    1.021493] [<ffffffff80405a50>] sunxi_ccu_probe+0x144/0x1a2
[    1.021510] [<ffffffff80405af6>] devm_sunxi_ccu_probe+0x48/0x82
[    1.021524] [<ffffffff80409020>] sun20i_d1_ccu_probe+0xba/0xfa
[    1.021546] [<ffffffff804a8b40>] platform_probe+0x4e/0xa6
[    1.021562] [<ffffffff808d81ee>] really_probe+0x10a/0x2dc
[    1.021581] [<ffffffff808d8472>] __driver_probe_device.part.0+0xb2/0xe8
[    1.021597] [<ffffffff804a67aa>] driver_probe_device+0x7a/0xca
[    1.021621] [<ffffffff804a6912>] __driver_attach+0x52/0x164
[    1.021638] [<ffffffff804a4c7a>] bus_for_each_dev+0x56/0x8c
[    1.021656] [<ffffffff804a6382>] driver_attach+0x1a/0x22
[    1.021673] [<ffffffff804a5c18>] bus_add_driver+0xea/0x1d8
[    1.021690] [<ffffffff804a7852>] driver_register+0x3e/0xd8
[    1.021709] [<ffffffff804a8826>] __platform_driver_register+0x1c/0x24
Emil[    1.021725] [<ffffffff80a17488>] sun20i_d1_ccu_driver_init+0x1a/0x22
[    1.021746] [<ffffffff800026ae>] do_one_initcall+0x46/0x1be
[    1.021762] [<ffffffff80a00ef2>] kernel_init_freeable+0x1c6/0x220
[    1.021791] [<ffffffff808e0b46>] kernel_init+0x1e/0x112

Linked as a fwnode consumer[ 1.021807] [<ffffffff808e7632>] ret_from_fork+0xe/0x1c

The warning is not fatal, so execution continues until hanging at

Quoted 7 lines
[    2.110919] printk: legacy console [ttyS0] disabled
[    2.136911] 2500000.serial: ttyS0 at MMIO 0x2500000 (irq = 205, base_baud = 1500000) is a 16550A�[    2.145674] printk: legacy console [ttyS0] enabled
[    2.145674] printk: legacy console [ttyS0] enabled
[    2.155095] printk: legacy bootconsole [sbi0] disabled
[    2.155095] printk: legacy bootconsole [sbi0] disabled

I have not been able to discover why it hangs here.

FWIW, that's probably because the CCU is the clock driver providing the clock for the uart, so when the sbi console goes away you lose output cos the uart driver cannot get the right rate for its input. You'd probably get further if you set keep_bootcon in your cmdline - but realistically the clock driver failing to probe is gonna have a load of knock on effects that it's probably enough to just have the failure you link here.

author
#21 lore raw

PATCH v14 2/18 irqchip/sifive-plic: Use dev_xyz() in-place of pr_xyz()

In reply to Anup Patel

Use dev_info(), dev_warn(), and dev_err() in-place of pr_info(), pr_warn(), and pr_err().

---
 drivers/irqchip/irq-sifive-plic.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)
drivers/irqchip/irq-sifive-plic.c +9 -10
index 7400a07fc479..3d7a61c35572 100644
@@ -3,7 +3,6 @@
  * Copyright (C) 2017 SiFive
  * Copyright (C) 2018 Christoph Hellwig
  */
-#define pr_fmt(fmt) "plic: " fmt
 #include <linux/cpu.h>
 #include <linux/interrupt.h>
 #include <linux/io.h>
@@ -372,8 +371,8 @@ static void plic_handle_irq(struct irq_desc *desc)
 		int err = generic_handle_domain_irq(handler->priv->irqdomain,
 						    hwirq);
 		if (unlikely(err))
-			pr_warn_ratelimited("can't find mapping for hwirq %lu\n",
-					hwirq);
+			dev_warn_ratelimited(handler->priv->dev,
+					     "can't find mapping for hwirq %lu\n", hwirq);
 	}
 
 	chained_irq_exit(chip, desc);
@@ -401,7 +400,7 @@ static int plic_starting_cpu(unsigned int cpu)
 		enable_percpu_irq(plic_parent_irq,
 				  irq_get_trigger_type(plic_parent_irq));
 	else
-		pr_warn("cpu%d: parent irq not available\n", cpu);
+		dev_warn(handler->priv->dev, "cpu%d: parent irq not available\n", cpu);
 	plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
 
 	return 0;
@@ -477,7 +476,7 @@ static int plic_probe(struct platform_device *pdev)
 		unsigned long hartid;
 
 		if (of_irq_parse_one(to_of_node(dev->fwnode), i, &parent)) {
-			pr_err("failed to parse parent for context %d.\n", i);
+			dev_err(dev, "failed to parse parent for context %d.\n", i);
 			continue;
 		}
 
@@ -500,13 +499,13 @@ static int plic_probe(struct platform_device *pdev)
 
 		error = riscv_of_parent_hartid(parent.np, &hartid);
 		if (error < 0) {
-			pr_warn("failed to parse hart ID for context %d.\n", i);
+			dev_warn(dev, "failed to parse hart ID for context %d.\n", i);
 			continue;
 		}
 
 		cpu = riscv_hartid_to_cpuid(hartid);
 		if (cpu < 0) {
-			pr_warn("Invalid cpuid for context %d\n", i);
+			dev_warn(dev, "Invalid cpuid for context %d\n", i);
 			continue;
 		}
 
@@ -525,7 +524,7 @@ static int plic_probe(struct platform_device *pdev)
 		 */
 		handler = per_cpu_ptr(&plic_handlers, cpu);
 		if (handler->present) {
-			pr_warn("handler already present for context %d.\n", i);
+			dev_warn(dev, "handler already present for context %d.\n", i);
 			plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
 			goto done;
 		}
@@ -575,8 +574,8 @@ static int plic_probe(struct platform_device *pdev)
 		}
 	}
 
-	pr_info("%pOFP: mapped %d interrupts with %d handlers for %d contexts.\n",
-		to_of_node(dev->fwnode), nr_irqs, nr_handlers, nr_contexts);
+	dev_info(dev, "mapped %d interrupts with %d handlers for %d contexts.\n",
+		 nr_irqs, nr_handlers, nr_contexts);
 	return 0;
 
 out_free_enable_reg:
Signature · 1 line
2.34.1
merge bot Applied
#22 lore raw

[tip: irq/msi] irqchip/sifive-plic: Use dev_xyz() in-place of pr_xyz()

The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     25d862e183d4efeb5e8b9843d783c90aaae4b14a
Gitweb:        https://git.kernel.org/tip/25d862e183d4efeb5e8b9843d783c90aaae4b14a
Author:        Anup Patel <[email protected]>
AuthorDate:    Thu, 22 Feb 2024 15:09:50 +05:30
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Fri, 23 Feb 2024 10:18:43 +01:00

irqchip/sifive-plic: Use dev_xyz() in-place of pr_xyz()

Use dev_info(), dev_warn(), and dev_err() in-place of pr_info(), pr_warn(), and pr_err().

---
 drivers/irqchip/irq-sifive-plic.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)
drivers/irqchip/irq-sifive-plic.c +11 -11
index 7400a07..892666f 100644
@@ -3,7 +3,6 @@
  * Copyright (C) 2017 SiFive
  * Copyright (C) 2018 Christoph Hellwig
  */
-#define pr_fmt(fmt) "plic: " fmt
 #include <linux/cpu.h>
 #include <linux/interrupt.h>
 #include <linux/io.h>
@@ -371,9 +370,10 @@ static void plic_handle_irq(struct irq_desc *desc)
 	while ((hwirq = readl(claim))) {
 		int err = generic_handle_domain_irq(handler->priv->irqdomain,
 						    hwirq);
-		if (unlikely(err))
-			pr_warn_ratelimited("can't find mapping for hwirq %lu\n",
-					hwirq);
+		if (unlikely(err)) {
+			dev_warn_ratelimited(handler->priv->dev,
+					     "can't find mapping for hwirq %lu\n", hwirq);
+		}
 	}
 
 	chained_irq_exit(chip, desc);
@@ -401,7 +401,7 @@ static int plic_starting_cpu(unsigned int cpu)
 		enable_percpu_irq(plic_parent_irq,
 				  irq_get_trigger_type(plic_parent_irq));
 	else
-		pr_warn("cpu%d: parent irq not available\n", cpu);
+		dev_warn(handler->priv->dev, "cpu%d: parent irq not available\n", cpu);
 	plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
 
 	return 0;
@@ -477,7 +477,7 @@ static int plic_probe(struct platform_device *pdev)
 		unsigned long hartid;
 
 		if (of_irq_parse_one(to_of_node(dev->fwnode), i, &parent)) {
-			pr_err("failed to parse parent for context %d.\n", i);
+			dev_err(dev, "failed to parse parent for context %d.\n", i);
 			continue;
 		}
 
@@ -500,13 +500,13 @@ static int plic_probe(struct platform_device *pdev)
 
 		error = riscv_of_parent_hartid(parent.np, &hartid);
 		if (error < 0) {
-			pr_warn("failed to parse hart ID for context %d.\n", i);
+			dev_warn(dev, "failed to parse hart ID for context %d.\n", i);
 			continue;
 		}
 
 		cpu = riscv_hartid_to_cpuid(hartid);
 		if (cpu < 0) {
-			pr_warn("Invalid cpuid for context %d\n", i);
+			dev_warn(dev, "Invalid cpuid for context %d\n", i);
 			continue;
 		}
 
@@ -525,7 +525,7 @@ static int plic_probe(struct platform_device *pdev)
 		 */
 		handler = per_cpu_ptr(&plic_handlers, cpu);
 		if (handler->present) {
-			pr_warn("handler already present for context %d.\n", i);
+			dev_warn(dev, "handler already present for context %d.\n", i);
 			plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
 			goto done;
 		}
@@ -575,8 +575,8 @@ done:
 		}
 	}
 
-	pr_info("%pOFP: mapped %d interrupts with %d handlers for %d contexts.\n",
-		to_of_node(dev->fwnode), nr_irqs, nr_handlers, nr_contexts);
+	dev_info(dev, "mapped %d interrupts with %d handlers for %d contexts.\n",
+		 nr_irqs, nr_handlers, nr_contexts);
 	return 0;
 
 out_free_enable_reg:
author
#23 lore raw

PATCH v14 3/18 irqchip/sifive-plic: Use devm_xyz() for managed allocation

In reply to Anup Patel

Use devm_xyz() for allocations and mappings managed by the Linux device driver framework.

---
 drivers/irqchip/irq-sifive-plic.c | 49 ++++++++++---------------------
 1 file changed, 16 insertions(+), 33 deletions(-)
drivers/irqchip/irq-sifive-plic.c +16 -33
index 3d7a61c35572..558ae63795d9 100644
@@ -435,39 +435,30 @@ static int plic_probe(struct platform_device *pdev)
 			plic_quirks = (unsigned long)id->data;
 	}
 
-	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
 	if (!priv)
 		return -ENOMEM;
 
 	priv->dev = dev;
 	priv->plic_quirks = plic_quirks;
 
-	priv->regs = of_iomap(to_of_node(dev->fwnode), 0);
-	if (WARN_ON(!priv->regs)) {
-		error = -EIO;
-		goto out_free_priv;
-	}
+	priv->regs = devm_platform_ioremap_resource(pdev, 0);
+	if (WARN_ON(!priv->regs))
+		return -EIO;
 
-	error = -EINVAL;
 	of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", &nr_irqs);
 	if (WARN_ON(!nr_irqs))
-		goto out_iounmap;
+		return -EINVAL;
 
 	priv->nr_irqs = nr_irqs;
 
-	priv->prio_save = bitmap_alloc(nr_irqs, GFP_KERNEL);
+	priv->prio_save = devm_bitmap_zalloc(dev, nr_irqs, GFP_KERNEL);
 	if (!priv->prio_save)
-		goto out_free_priority_reg;
+		return -ENOMEM;
 
 	nr_contexts = of_irq_count(to_of_node(dev->fwnode));
 	if (WARN_ON(!nr_contexts))
-		goto out_free_priority_reg;
-
-	error = -ENOMEM;
-	priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
-						&plic_irqdomain_ops, priv);
-	if (WARN_ON(!priv->irqdomain))
-		goto out_free_priority_reg;
+		return -EINVAL;
 
 	for (i = 0; i < nr_contexts; i++) {
 		struct of_phandle_args parent;
@@ -538,10 +529,10 @@ static int plic_probe(struct platform_device *pdev)
 			i * CONTEXT_ENABLE_SIZE;
 		handler->priv = priv;
 
-		handler->enable_save =  kcalloc(DIV_ROUND_UP(nr_irqs, 32),
-						sizeof(*handler->enable_save), GFP_KERNEL);
+		handler->enable_save = devm_kcalloc(dev, DIV_ROUND_UP(nr_irqs, 32),
+						    sizeof(*handler->enable_save), GFP_KERNEL);
 		if (!handler->enable_save)
-			goto out_free_enable_reg;
+			return -ENOMEM;
 done:
 		for (hwirq = 1; hwirq <= nr_irqs; hwirq++) {
 			plic_toggle(handler, hwirq, 0);
@@ -551,6 +542,11 @@ static int plic_probe(struct platform_device *pdev)
 		nr_handlers++;
 	}
 
+	priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
+						&plic_irqdomain_ops, priv);
+	if (WARN_ON(!priv->irqdomain))
+		return -ENOMEM;
+
 	/*
 	 * We can have multiple PLIC instances so setup cpuhp state
 	 * and register syscore operations only once after context
@@ -577,19 +573,6 @@ static int plic_probe(struct platform_device *pdev)
 	dev_info(dev, "mapped %d interrupts with %d handlers for %d contexts.\n",
 		 nr_irqs, nr_handlers, nr_contexts);
 	return 0;
-
-out_free_enable_reg:
-	for_each_cpu(cpu, cpu_present_mask) {
-		handler = per_cpu_ptr(&plic_handlers, cpu);
-		kfree(handler->enable_save);
-	}
-out_free_priority_reg:
-	kfree(priv->prio_save);
-out_iounmap:
-	iounmap(priv->regs);
-out_free_priv:
-	kfree(priv);
-	return error;
 }
 
 static struct platform_driver plic_driver = {
Signature · 1 line
2.34.1
merge bot Applied
#24 lore raw

[tip: irq/msi] irqchip/sifive-plic: Use devm_xyz() for managed allocation

The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     b68d0ff529a939a118ec52f271be8cad5d99e79a
Gitweb:        https://git.kernel.org/tip/b68d0ff529a939a118ec52f271be8cad5d99e79a
Author:        Anup Patel <[email protected]>
AuthorDate:    Thu, 22 Feb 2024 15:09:51 +05:30
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Fri, 23 Feb 2024 10:18:43 +01:00

irqchip/sifive-plic: Use devm_xyz() for managed allocation

Use devm_xyz() for allocations and mappings managed by the Linux device driver framework.

---
 drivers/irqchip/irq-sifive-plic.c | 49 +++++++++---------------------
 1 file changed, 16 insertions(+), 33 deletions(-)
drivers/irqchip/irq-sifive-plic.c +16 -33
index 892666f..299feef 100644
@@ -436,39 +436,30 @@ static int plic_probe(struct platform_device *pdev)
 			plic_quirks = (unsigned long)id->data;
 	}
 
-	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
 	if (!priv)
 		return -ENOMEM;
 
 	priv->dev = dev;
 	priv->plic_quirks = plic_quirks;
 
-	priv->regs = of_iomap(to_of_node(dev->fwnode), 0);
-	if (WARN_ON(!priv->regs)) {
-		error = -EIO;
-		goto out_free_priv;
-	}
+	priv->regs = devm_platform_ioremap_resource(pdev, 0);
+	if (WARN_ON(!priv->regs))
+		return -EIO;
 
-	error = -EINVAL;
 	of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", &nr_irqs);
 	if (WARN_ON(!nr_irqs))
-		goto out_iounmap;
+		return -EINVAL;
 
 	priv->nr_irqs = nr_irqs;
 
-	priv->prio_save = bitmap_alloc(nr_irqs, GFP_KERNEL);
+	priv->prio_save = devm_bitmap_zalloc(dev, nr_irqs, GFP_KERNEL);
 	if (!priv->prio_save)
-		goto out_free_priority_reg;
+		return -ENOMEM;
 
 	nr_contexts = of_irq_count(to_of_node(dev->fwnode));
 	if (WARN_ON(!nr_contexts))
-		goto out_free_priority_reg;
-
-	error = -ENOMEM;
-	priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
-						&plic_irqdomain_ops, priv);
-	if (WARN_ON(!priv->irqdomain))
-		goto out_free_priority_reg;
+		return -EINVAL;
 
 	for (i = 0; i < nr_contexts; i++) {
 		struct of_phandle_args parent;
@@ -539,10 +530,10 @@ static int plic_probe(struct platform_device *pdev)
 			i * CONTEXT_ENABLE_SIZE;
 		handler->priv = priv;
 
-		handler->enable_save =  kcalloc(DIV_ROUND_UP(nr_irqs, 32),
-						sizeof(*handler->enable_save), GFP_KERNEL);
+		handler->enable_save = devm_kcalloc(dev, DIV_ROUND_UP(nr_irqs, 32),
+						    sizeof(*handler->enable_save), GFP_KERNEL);
 		if (!handler->enable_save)
-			goto out_free_enable_reg;
+			return -ENOMEM;
 done:
 		for (hwirq = 1; hwirq <= nr_irqs; hwirq++) {
 			plic_toggle(handler, hwirq, 0);
@@ -552,6 +543,11 @@ done:
 		nr_handlers++;
 	}
 
+	priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
+						&plic_irqdomain_ops, priv);
+	if (WARN_ON(!priv->irqdomain))
+		return -ENOMEM;
+
 	/*
 	 * We can have multiple PLIC instances so setup cpuhp state
 	 * and register syscore operations only once after context
@@ -578,19 +574,6 @@ done:
 	dev_info(dev, "mapped %d interrupts with %d handlers for %d contexts.\n",
 		 nr_irqs, nr_handlers, nr_contexts);
 	return 0;
-
-out_free_enable_reg:
-	for_each_cpu(cpu, cpu_present_mask) {
-		handler = per_cpu_ptr(&plic_handlers, cpu);
-		kfree(handler->enable_save);
-	}
-out_free_priority_reg:
-	kfree(priv->prio_save);
-out_iounmap:
-	iounmap(priv->regs);
-out_free_priv:
-	kfree(priv);
-	return error;
 }
 
 static struct platform_driver plic_driver = {
author
#25 lore raw

PATCH v14 4/18 irqchip/sifive-plic: Use riscv_get_intc_hwnode() to get parent fwnode

In reply to Anup Patel

The RISC-V INTC irqdomain is always the parent irqdomain of SiFive PLIC so use riscv_get_intc_hwnode() to get the parent fwnode similar to other RISC-V drivers which use local interrupts.

---
 drivers/irqchip/irq-sifive-plic.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/irqchip/irq-sifive-plic.c +4 -2
index 558ae63795d9..615498071b6e 100644
@@ -422,6 +422,7 @@ static int plic_probe(struct platform_device *pdev)
 	struct device *dev = &pdev->dev;
 	unsigned long plic_quirks = 0;
 	struct plic_handler *handler;
+	struct irq_domain *domain;
 	struct plic_priv *priv;
 	bool cpuhp_setup;
 	unsigned int cpu;
@@ -501,8 +502,9 @@ static int plic_probe(struct platform_device *pdev)
 		}
 
 		/* Find parent domain and register chained handler */
-		if (!plic_parent_irq && irq_find_host(parent.np)) {
-			plic_parent_irq = irq_of_parse_and_map(to_of_node(dev->fwnode), i);
+		domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(), DOMAIN_BUS_ANY);
+		if (!plic_parent_irq && domain) {
+			plic_parent_irq = irq_create_mapping(domain, RV_IRQ_EXT);
 			if (plic_parent_irq)
 				irq_set_chained_handler(plic_parent_irq,
 							plic_handle_irq);
Signature · 1 line
2.34.1
merge bot Applied
#26 lore raw

[tip: irq/msi] irqchip/sifive-plic: Use riscv_get_intc_hwnode() to get parent fwnode

The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     6c725f33d67b53f2d302c2c4509deae953fc6ade
Gitweb:        https://git.kernel.org/tip/6c725f33d67b53f2d302c2c4509deae953fc6ade
Author:        Anup Patel <[email protected]>
AuthorDate:    Thu, 22 Feb 2024 15:09:52 +05:30
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Fri, 23 Feb 2024 10:18:44 +01:00

irqchip/sifive-plic: Use riscv_get_intc_hwnode() to get parent fwnode

The RISC-V INTC irqdomain is always the parent irqdomain of SiFive PLIC so use riscv_get_intc_hwnode() to get the parent fwnode similar to other RISC-V drivers which use local interrupts.

---
 drivers/irqchip/irq-sifive-plic.c |  9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/irqchip/irq-sifive-plic.c +5 -4
index 299feef..208fad7 100644
@@ -423,6 +423,7 @@ static int plic_probe(struct platform_device *pdev)
 	struct device *dev = &pdev->dev;
 	unsigned long plic_quirks = 0;
 	struct plic_handler *handler;
+	struct irq_domain *domain;
 	struct plic_priv *priv;
 	bool cpuhp_setup;
 	unsigned int cpu;
@@ -502,11 +503,11 @@ static int plic_probe(struct platform_device *pdev)
 		}
 
 		/* Find parent domain and register chained handler */
-		if (!plic_parent_irq && irq_find_host(parent.np)) {
-			plic_parent_irq = irq_of_parse_and_map(to_of_node(dev->fwnode), i);
+		domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(), DOMAIN_BUS_ANY);
+		if (!plic_parent_irq && domain) {
+			plic_parent_irq = irq_create_mapping(domain, RV_IRQ_EXT);
 			if (plic_parent_irq)
-				irq_set_chained_handler(plic_parent_irq,
-							plic_handle_irq);
+				irq_set_chained_handler(plic_parent_irq, plic_handle_irq);
 		}
 
 		/*
author
#27 lore raw

PATCH v14 5/18 irqchip/sifive-plic: Cleanup PLIC contexts upon irqdomain creation failure

In reply to Anup Patel

The SiFive PLIC contexts should not be left dangling if irqdomain creation fails because plic_starting_cpu() can crash accessing unmapped registers.

---
 drivers/irqchip/irq-sifive-plic.c | 73 ++++++++++++++++++++++---------
 1 file changed, 53 insertions(+), 20 deletions(-)
drivers/irqchip/irq-sifive-plic.c +53 -20
index 615498071b6e..f0df5d0cb76e 100644
@@ -416,17 +416,45 @@ static const struct of_device_id plic_match[] = {
 	{}
 };
 
+static int plic_parse_context_parent(struct platform_device *pdev, u32 context,
+				     u32 *parent_hwirq, int *parent_cpu)
+{
+	struct device *dev = &pdev->dev;
+	struct of_phandle_args parent;
+	unsigned long hartid;
+	int rc;
+
+	/*
+	 * Currently, only OF fwnode is supported so extend this
+	 * function for ACPI support.
+	 */
+	if (!is_of_node(dev->fwnode))
+		return -EINVAL;
+
+	rc = of_irq_parse_one(to_of_node(dev->fwnode), context, &parent);
+	if (rc)
+		return rc;
+
+	rc = riscv_of_parent_hartid(parent.np, &hartid);
+	if (rc)
+		return rc;
+
+	*parent_hwirq = parent.args[0];
+	*parent_cpu = riscv_hartid_to_cpuid(hartid);
+	return 0;
+}
+
 static int plic_probe(struct platform_device *pdev)
 {
-	int error = 0, nr_contexts, nr_handlers = 0, i;
+	int error = 0, nr_contexts, nr_handlers = 0, cpu, i;
 	struct device *dev = &pdev->dev;
 	unsigned long plic_quirks = 0;
 	struct plic_handler *handler;
+	u32 nr_irqs, parent_hwirq;
 	struct irq_domain *domain;
 	struct plic_priv *priv;
+	irq_hw_number_t hwirq;
 	bool cpuhp_setup;
-	unsigned int cpu;
-	u32 nr_irqs;
 
 	if (is_of_node(dev->fwnode)) {
 		const struct of_device_id *id;
@@ -462,13 +490,9 @@ static int plic_probe(struct platform_device *pdev)
 		return -EINVAL;
 
 	for (i = 0; i < nr_contexts; i++) {
-		struct of_phandle_args parent;
-		irq_hw_number_t hwirq;
-		int cpu;
-		unsigned long hartid;
-
-		if (of_irq_parse_one(to_of_node(dev->fwnode), i, &parent)) {
-			dev_err(dev, "failed to parse parent for context %d.\n", i);
+		error = plic_parse_context_parent(pdev, i, &parent_hwirq, &cpu);
+		if (error) {
+			dev_warn(dev, "hwirq for context%d not found\n", i);
 			continue;
 		}
 
@@ -476,7 +500,7 @@ static int plic_probe(struct platform_device *pdev)
 		 * Skip contexts other than external interrupts for our
 		 * privilege level.
 		 */
-		if (parent.args[0] != RV_IRQ_EXT) {
+		if (parent_hwirq != RV_IRQ_EXT) {
 			/* Disable S-mode enable bits if running in M-mode. */
 			if (IS_ENABLED(CONFIG_RISCV_M_MODE)) {
 				void __iomem *enable_base = priv->regs +
@@ -489,13 +513,6 @@ static int plic_probe(struct platform_device *pdev)
 			continue;
 		}
 
-		error = riscv_of_parent_hartid(parent.np, &hartid);
-		if (error < 0) {
-			dev_warn(dev, "failed to parse hart ID for context %d.\n", i);
-			continue;
-		}
-
-		cpu = riscv_hartid_to_cpuid(hartid);
 		if (cpu < 0) {
 			dev_warn(dev, "Invalid cpuid for context %d\n", i);
 			continue;
@@ -534,7 +551,7 @@ static int plic_probe(struct platform_device *pdev)
 		handler->enable_save = devm_kcalloc(dev, DIV_ROUND_UP(nr_irqs, 32),
 						    sizeof(*handler->enable_save), GFP_KERNEL);
 		if (!handler->enable_save)
-			return -ENOMEM;
+			goto fail_cleanup_contexts;
 done:
 		for (hwirq = 1; hwirq <= nr_irqs; hwirq++) {
 			plic_toggle(handler, hwirq, 0);
@@ -547,7 +564,7 @@ static int plic_probe(struct platform_device *pdev)
 	priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
 						&plic_irqdomain_ops, priv);
 	if (WARN_ON(!priv->irqdomain))
-		return -ENOMEM;
+		goto fail_cleanup_contexts;
 
 	/*
 	 * We can have multiple PLIC instances so setup cpuhp state
@@ -575,6 +592,22 @@ static int plic_probe(struct platform_device *pdev)
 	dev_info(dev, "mapped %d interrupts with %d handlers for %d contexts.\n",
 		 nr_irqs, nr_handlers, nr_contexts);
 	return 0;
+
+fail_cleanup_contexts:
+	for (i = 0; i < nr_contexts; i++) {
+		if (plic_parse_context_parent(pdev, i, &parent_hwirq, &cpu))
+			continue;
+		if (parent_hwirq != RV_IRQ_EXT || cpu < 0)
+			continue;
+
+		handler = per_cpu_ptr(&plic_handlers, cpu);
+		handler->present = false;
+		handler->hart_base = NULL;
+		handler->enable_base = NULL;
+		handler->enable_save = NULL;
+		handler->priv = NULL;
+	}
+	return -ENOMEM;
 }
 
 static struct platform_driver plic_driver = {
Signature · 1 line
2.34.1
merge bot Applied
#28 lore raw

[tip: irq/msi] irqchip/sifive-plic: Cleanup PLIC contexts upon irqdomain creation failure

The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     a15587277a246c388c83b1cd9cf7c1a868cd752f
Gitweb:        https://git.kernel.org/tip/a15587277a246c388c83b1cd9cf7c1a868cd752f
Author:        Anup Patel <[email protected]>
AuthorDate:    Thu, 22 Feb 2024 15:09:53 +05:30
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Fri, 23 Feb 2024 10:18:44 +01:00

irqchip/sifive-plic: Cleanup PLIC contexts upon irqdomain creation failure

The SiFive PLIC contexts should not be left dangling if irqdomain creation fails because plic_starting_cpu() can crash accessing unmapped registers.

---
 drivers/irqchip/irq-sifive-plic.c | 73 +++++++++++++++++++++---------
 1 file changed, 53 insertions(+), 20 deletions(-)
drivers/irqchip/irq-sifive-plic.c +53 -20
index 208fad7..a399cb3 100644
@@ -417,17 +417,45 @@ static const struct of_device_id plic_match[] = {
 	{}
 };
 
+static int plic_parse_context_parent(struct platform_device *pdev, u32 context,
+				     u32 *parent_hwirq, int *parent_cpu)
+{
+	struct device *dev = &pdev->dev;
+	struct of_phandle_args parent;
+	unsigned long hartid;
+	int rc;
+
+	/*
+	 * Currently, only OF fwnode is supported so extend this
+	 * function for ACPI support.
+	 */
+	if (!is_of_node(dev->fwnode))
+		return -EINVAL;
+
+	rc = of_irq_parse_one(to_of_node(dev->fwnode), context, &parent);
+	if (rc)
+		return rc;
+
+	rc = riscv_of_parent_hartid(parent.np, &hartid);
+	if (rc)
+		return rc;
+
+	*parent_hwirq = parent.args[0];
+	*parent_cpu = riscv_hartid_to_cpuid(hartid);
+	return 0;
+}
+
 static int plic_probe(struct platform_device *pdev)
 {
-	int error = 0, nr_contexts, nr_handlers = 0, i;
+	int error = 0, nr_contexts, nr_handlers = 0, cpu, i;
 	struct device *dev = &pdev->dev;
 	unsigned long plic_quirks = 0;
 	struct plic_handler *handler;
+	u32 nr_irqs, parent_hwirq;
 	struct irq_domain *domain;
 	struct plic_priv *priv;
+	irq_hw_number_t hwirq;
 	bool cpuhp_setup;
-	unsigned int cpu;
-	u32 nr_irqs;
 
 	if (is_of_node(dev->fwnode)) {
 		const struct of_device_id *id;
@@ -463,13 +491,9 @@ static int plic_probe(struct platform_device *pdev)
 		return -EINVAL;
 
 	for (i = 0; i < nr_contexts; i++) {
-		struct of_phandle_args parent;
-		irq_hw_number_t hwirq;
-		int cpu;
-		unsigned long hartid;
-
-		if (of_irq_parse_one(to_of_node(dev->fwnode), i, &parent)) {
-			dev_err(dev, "failed to parse parent for context %d.\n", i);
+		error = plic_parse_context_parent(pdev, i, &parent_hwirq, &cpu);
+		if (error) {
+			dev_warn(dev, "hwirq for context%d not found\n", i);
 			continue;
 		}
 
@@ -477,7 +501,7 @@ static int plic_probe(struct platform_device *pdev)
 		 * Skip contexts other than external interrupts for our
 		 * privilege level.
 		 */
-		if (parent.args[0] != RV_IRQ_EXT) {
+		if (parent_hwirq != RV_IRQ_EXT) {
 			/* Disable S-mode enable bits if running in M-mode. */
 			if (IS_ENABLED(CONFIG_RISCV_M_MODE)) {
 				void __iomem *enable_base = priv->regs +
@@ -490,13 +514,6 @@ static int plic_probe(struct platform_device *pdev)
 			continue;
 		}
 
-		error = riscv_of_parent_hartid(parent.np, &hartid);
-		if (error < 0) {
-			dev_warn(dev, "failed to parse hart ID for context %d.\n", i);
-			continue;
-		}
-
-		cpu = riscv_hartid_to_cpuid(hartid);
 		if (cpu < 0) {
 			dev_warn(dev, "Invalid cpuid for context %d\n", i);
 			continue;
@@ -534,7 +551,7 @@ static int plic_probe(struct platform_device *pdev)
 		handler->enable_save = devm_kcalloc(dev, DIV_ROUND_UP(nr_irqs, 32),
 						    sizeof(*handler->enable_save), GFP_KERNEL);
 		if (!handler->enable_save)
-			return -ENOMEM;
+			goto fail_cleanup_contexts;
 done:
 		for (hwirq = 1; hwirq <= nr_irqs; hwirq++) {
 			plic_toggle(handler, hwirq, 0);
@@ -547,7 +564,7 @@ done:
 	priv->irqdomain = irq_domain_add_linear(to_of_node(dev->fwnode), nr_irqs + 1,
 						&plic_irqdomain_ops, priv);
 	if (WARN_ON(!priv->irqdomain))
-		return -ENOMEM;
+		goto fail_cleanup_contexts;
 
 	/*
 	 * We can have multiple PLIC instances so setup cpuhp state
@@ -575,6 +592,22 @@ done:
 	dev_info(dev, "mapped %d interrupts with %d handlers for %d contexts.\n",
 		 nr_irqs, nr_handlers, nr_contexts);
 	return 0;
+
+fail_cleanup_contexts:
+	for (i = 0; i < nr_contexts; i++) {
+		if (plic_parse_context_parent(pdev, i, &parent_hwirq, &cpu))
+			continue;
+		if (parent_hwirq != RV_IRQ_EXT || cpu < 0)
+			continue;
+
+		handler = per_cpu_ptr(&plic_handlers, cpu);
+		handler->present = false;
+		handler->hart_base = NULL;
+		handler->enable_base = NULL;
+		handler->enable_save = NULL;
+		handler->priv = NULL;
+	}
+	return -ENOMEM;
 }
 
 static struct platform_driver plic_driver = {
author
#29 lore raw

PATCH v14 6/18 irqchip/sifive-plic: Parse number of irqs and contexts early in plic_probe

In reply to Anup Patel

The SiFive PLIC driver needs to know the number of irqs and contexts to complete initialization. Parse these details early in plic_probe() to avoid unnecessary memory allocations and register mappings if these details are not available.

---
 drivers/irqchip/irq-sifive-plic.c | 43 ++++++++++++++++++++++++-------
 1 file changed, 33 insertions(+), 10 deletions(-)
drivers/irqchip/irq-sifive-plic.c +33 -10
index f0df5d0cb76e..e6d23fde51ed 100644
@@ -416,6 +416,34 @@ static const struct of_device_id plic_match[] = {
 	{}
 };
 
+static int plic_parse_nr_irqs_and_contexts(struct platform_device *pdev,
+					   u32 *nr_irqs, u32 *nr_contexts)
+{
+	struct device *dev = &pdev->dev;
+	int rc;
+
+	/*
+	 * Currently, only OF fwnode is supported so extend this
+	 * function for ACPI support.
+	 */
+	if (!is_of_node(dev->fwnode))
+		return -EINVAL;
+
+	rc = of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", nr_irqs);
+	if (rc) {
+		dev_err(dev, "riscv,ndev property not available\n");
+		return rc;
+	}
+
+	*nr_contexts = of_irq_count(to_of_node(dev->fwnode));
+	if (WARN_ON(!(*nr_contexts))) {
+		dev_err(dev, "no PLIC context available\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static int plic_parse_context_parent(struct platform_device *pdev, u32 context,
 				     u32 *parent_hwirq, int *parent_cpu)
 {
@@ -464,31 +492,26 @@ static int plic_probe(struct platform_device *pdev)
 			plic_quirks = (unsigned long)id->data;
 	}
 
+	error = plic_parse_nr_irqs_and_contexts(pdev, &nr_irqs, &nr_contexts);
+	if (error)
+		return error;
+
 	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
 	if (!priv)
 		return -ENOMEM;
 
 	priv->dev = dev;
 	priv->plic_quirks = plic_quirks;
+	priv->nr_irqs = nr_irqs;
 
 	priv->regs = devm_platform_ioremap_resource(pdev, 0);
 	if (WARN_ON(!priv->regs))
 		return -EIO;
 
-	of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", &nr_irqs);
-	if (WARN_ON(!nr_irqs))
-		return -EINVAL;
-
-	priv->nr_irqs = nr_irqs;
-
 	priv->prio_save = devm_bitmap_zalloc(dev, nr_irqs, GFP_KERNEL);
 	if (!priv->prio_save)
 		return -ENOMEM;
 
-	nr_contexts = of_irq_count(to_of_node(dev->fwnode));
-	if (WARN_ON(!nr_contexts))
-		return -EINVAL;
-
 	for (i = 0; i < nr_contexts; i++) {
 		error = plic_parse_context_parent(pdev, i, &parent_hwirq, &cpu);
 		if (error) {
Signature · 1 line
2.34.1
merge bot Applied
#30 lore raw

[tip: irq/msi] irqchip/sifive-plic: Parse number of interrupts and contexts early in plic_probe()

The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     95652106478030f54620b1f0d28f78ab110b3212
Gitweb:        https://git.kernel.org/tip/95652106478030f54620b1f0d28f78ab110b3212
Author:        Anup Patel <[email protected]>
AuthorDate:    Thu, 22 Feb 2024 15:09:54 +05:30
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Fri, 23 Feb 2024 10:18:44 +01:00

irqchip/sifive-plic: Parse number of interrupts and contexts early in plic_probe()

The SiFive PLIC driver needs to know the number of interrupts and contexts to complete initialization. Parse these details early in plic_probe() to avoid unnecessary memory allocations and register mappings if these details are not available.

---
 drivers/irqchip/irq-sifive-plic.c | 43 +++++++++++++++++++++++-------
 1 file changed, 33 insertions(+), 10 deletions(-)
drivers/irqchip/irq-sifive-plic.c +33 -10
index a399cb3..474ddc3 100644
@@ -417,6 +417,34 @@ static const struct of_device_id plic_match[] = {
 	{}
 };
 
+static int plic_parse_nr_irqs_and_contexts(struct platform_device *pdev,
+					   u32 *nr_irqs, u32 *nr_contexts)
+{
+	struct device *dev = &pdev->dev;
+	int rc;
+
+	/*
+	 * Currently, only OF fwnode is supported so extend this
+	 * function for ACPI support.
+	 */
+	if (!is_of_node(dev->fwnode))
+		return -EINVAL;
+
+	rc = of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", nr_irqs);
+	if (rc) {
+		dev_err(dev, "riscv,ndev property not available\n");
+		return rc;
+	}
+
+	*nr_contexts = of_irq_count(to_of_node(dev->fwnode));
+	if (WARN_ON(!(*nr_contexts))) {
+		dev_err(dev, "no PLIC context available\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static int plic_parse_context_parent(struct platform_device *pdev, u32 context,
 				     u32 *parent_hwirq, int *parent_cpu)
 {
@@ -465,31 +493,26 @@ static int plic_probe(struct platform_device *pdev)
 			plic_quirks = (unsigned long)id->data;
 	}
 
+	error = plic_parse_nr_irqs_and_contexts(pdev, &nr_irqs, &nr_contexts);
+	if (error)
+		return error;
+
 	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
 	if (!priv)
 		return -ENOMEM;
 
 	priv->dev = dev;
 	priv->plic_quirks = plic_quirks;
+	priv->nr_irqs = nr_irqs;
 
 	priv->regs = devm_platform_ioremap_resource(pdev, 0);
 	if (WARN_ON(!priv->regs))
 		return -EIO;
 
-	of_property_read_u32(to_of_node(dev->fwnode), "riscv,ndev", &nr_irqs);
-	if (WARN_ON(!nr_irqs))
-		return -EINVAL;
-
-	priv->nr_irqs = nr_irqs;
-
 	priv->prio_save = devm_bitmap_zalloc(dev, nr_irqs, GFP_KERNEL);
 	if (!priv->prio_save)
 		return -ENOMEM;
 
-	nr_contexts = of_irq_count(to_of_node(dev->fwnode));
-	if (WARN_ON(!nr_contexts))
-		return -EINVAL;
-
 	for (i = 0; i < nr_contexts; i++) {
 		error = plic_parse_context_parent(pdev, i, &parent_hwirq, &cpu);
 		if (error) {
author
#31 lore raw

PATCH v14 7/18 irqchip/sifive-plic: Improve locking safety by using irqsave/irqrestore

In reply to Anup Patel

Now that PLIC driver is probed as a regular platform driver, the lock dependency validator complains about the safety of handler->enable_lock usage:

[    0.956775]  Possible interrupt unsafe locking scenario:
[    0.956998]        CPU0                    CPU1
[    0.957247]        ----                    ----
[    0.957439]   lock(&handler->enable_lock);
[    0.957607]                                local_irq_disable();
[    0.957793]                                lock(&irq_desc_lock_class);
[    0.958021]                                lock(&handler->enable_lock);
[    0.958246]   <Interrupt>
[    0.958342]     lock(&irq_desc_lock_class);
[    0.958501]
                *** DEADLOCK ***

To address above, use raw_spin_lock_irqsave/unlock_irqrestore() instead of raw_spin_lock/unlock().

---
 drivers/irqchip/irq-sifive-plic.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)
drivers/irqchip/irq-sifive-plic.c +10 -6
index e6d23fde51ed..175d02f7fd64 100644
@@ -103,9 +103,11 @@ static void __plic_toggle(void __iomem *enable_base, int hwirq, int enable)
 
 static void plic_toggle(struct plic_handler *handler, int hwirq, int enable)
 {
-	raw_spin_lock(&handler->enable_lock);
+	unsigned long flags;
+
+	raw_spin_lock_irqsave(&handler->enable_lock, flags);
 	__plic_toggle(handler->enable_base, hwirq, enable);
-	raw_spin_unlock(&handler->enable_lock);
+	raw_spin_unlock_irqrestore(&handler->enable_lock, flags);
 }
 
 static inline void plic_irq_toggle(const struct cpumask *mask,
@@ -236,6 +238,7 @@ static int plic_irq_set_type(struct irq_data *d, unsigned int type)
 static int plic_irq_suspend(void)
 {
 	unsigned int i, cpu;
+	unsigned long flags;
 	u32 __iomem *reg;
 	struct plic_priv *priv;
 
@@ -253,12 +256,12 @@ static int plic_irq_suspend(void)
 		if (!handler->present)
 			continue;
 
-		raw_spin_lock(&handler->enable_lock);
+		raw_spin_lock_irqsave(&handler->enable_lock, flags);
 		for (i = 0; i < DIV_ROUND_UP(priv->nr_irqs, 32); i++) {
 			reg = handler->enable_base + i * sizeof(u32);
 			handler->enable_save[i] = readl(reg);
 		}
-		raw_spin_unlock(&handler->enable_lock);
+		raw_spin_unlock_irqrestore(&handler->enable_lock, flags);
 	}
 
 	return 0;
@@ -267,6 +270,7 @@ static int plic_irq_suspend(void)
 static void plic_irq_resume(void)
 {
 	unsigned int i, index, cpu;
+	unsigned long flags;
 	u32 __iomem *reg;
 	struct plic_priv *priv;
 
@@ -284,12 +288,12 @@ static void plic_irq_resume(void)
 		if (!handler->present)
 			continue;
 
-		raw_spin_lock(&handler->enable_lock);
+		raw_spin_lock_irqsave(&handler->enable_lock, flags);
 		for (i = 0; i < DIV_ROUND_UP(priv->nr_irqs, 32); i++) {
 			reg = handler->enable_base + i * sizeof(u32);
 			writel(handler->enable_save[i], reg);
 		}
-		raw_spin_unlock(&handler->enable_lock);
+		raw_spin_unlock_irqrestore(&handler->enable_lock, flags);
 	}
 }
 
Signature · 1 line
2.34.1
merge bot Applied
#32 lore raw

[tip: irq/msi] irqchip/sifive-plic: Improve locking safety by using irqsave/irqrestore

The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     abb7205794900503d6358ef1fb645373753a794d
Gitweb:        https://git.kernel.org/tip/abb7205794900503d6358ef1fb645373753a794d
Author:        Anup Patel <[email protected]>
AuthorDate:    Thu, 22 Feb 2024 15:09:55 +05:30
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Fri, 23 Feb 2024 10:18:44 +01:00

irqchip/sifive-plic: Improve locking safety by using irqsave/irqrestore

Now that PLIC driver is probed as a regular platform driver, the lock dependency validator complains about the safety of handler->enable_lock usage:

[    0.956775]  Possible interrupt unsafe locking scenario:
[    0.956998]        CPU0                    CPU1
[    0.957247]        ----                    ----
[    0.957439]   lock(&handler->enable_lock);
[    0.957607]                                local_irq_disable();
[    0.957793]                                lock(&irq_desc_lock_class);
[    0.958021]                                lock(&handler->enable_lock);
[    0.958246]   <Interrupt>
[    0.958342]     lock(&irq_desc_lock_class);
[    0.958501]
                *** DEADLOCK ***

To address above, use raw_spin_lock_irqsave/unlock_irqrestore() instead of raw_spin_lock/unlock().

---
 drivers/irqchip/irq-sifive-plic.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)
drivers/irqchip/irq-sifive-plic.c +10 -6
index 474ddc3..601000d 100644
@@ -103,9 +103,11 @@ static void __plic_toggle(void __iomem *enable_base, int hwirq, int enable)
 
 static void plic_toggle(struct plic_handler *handler, int hwirq, int enable)
 {
-	raw_spin_lock(&handler->enable_lock);
+	unsigned long flags;
+
+	raw_spin_lock_irqsave(&handler->enable_lock, flags);
 	__plic_toggle(handler->enable_base, hwirq, enable);
-	raw_spin_unlock(&handler->enable_lock);
+	raw_spin_unlock_irqrestore(&handler->enable_lock, flags);
 }
 
 static inline void plic_irq_toggle(const struct cpumask *mask,
@@ -236,6 +238,7 @@ static int plic_irq_set_type(struct irq_data *d, unsigned int type)
 static int plic_irq_suspend(void)
 {
 	unsigned int i, cpu;
+	unsigned long flags;
 	u32 __iomem *reg;
 	struct plic_priv *priv;
 
@@ -253,12 +256,12 @@ static int plic_irq_suspend(void)
 		if (!handler->present)
 			continue;
 
-		raw_spin_lock(&handler->enable_lock);
+		raw_spin_lock_irqsave(&handler->enable_lock, flags);
 		for (i = 0; i < DIV_ROUND_UP(priv->nr_irqs, 32); i++) {
 			reg = handler->enable_base + i * sizeof(u32);
 			handler->enable_save[i] = readl(reg);
 		}
-		raw_spin_unlock(&handler->enable_lock);
+		raw_spin_unlock_irqrestore(&handler->enable_lock, flags);
 	}
 
 	return 0;
@@ -267,6 +270,7 @@ static int plic_irq_suspend(void)
 static void plic_irq_resume(void)
 {
 	unsigned int i, index, cpu;
+	unsigned long flags;
 	u32 __iomem *reg;
 	struct plic_priv *priv;
 
@@ -284,12 +288,12 @@ static void plic_irq_resume(void)
 		if (!handler->present)
 			continue;
 
-		raw_spin_lock(&handler->enable_lock);
+		raw_spin_lock_irqsave(&handler->enable_lock, flags);
 		for (i = 0; i < DIV_ROUND_UP(priv->nr_irqs, 32); i++) {
 			reg = handler->enable_base + i * sizeof(u32);
 			writel(handler->enable_save[i], reg);
 		}
-		raw_spin_unlock(&handler->enable_lock);
+		raw_spin_unlock_irqrestore(&handler->enable_lock, flags);
 	}
 }
 
author
#33 lore raw

PATCH v14 8/18 irqchip/riscv-intc: Add support for RISC-V AIA

In reply to Anup Patel

The RISC-V advanced interrupt architecture (AIA) extends the per-HART local interrupts in following ways:
1. Minimum 64 local interrupts for both RV32 and RV64
2. Ability to process multiple pending local interrupts in same interrupt handler
3. Priority configuration for each local interrupts
4. Special CSRs to configure/access the per-HART MSI controller

Add support for #1 and #2 described above in the RISC-V intc driver.

---
 drivers/irqchip/irq-riscv-intc.c | 32 +++++++++++++++++++++++++-------
 1 file changed, 25 insertions(+), 7 deletions(-)
drivers/irqchip/irq-riscv-intc.c +25 -7
index e8d01b14ccdd..8997f6986f89 100644
@@ -17,6 +17,7 @@
 #include <linux/module.h>
 #include <linux/of.h>
 #include <linux/smp.h>
+#include <asm/hwcap.h>
 
 static struct irq_domain *intc_domain;
 
@@ -30,6 +31,14 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 	generic_handle_domain_irq(intc_domain, cause);
 }
 
+static asmlinkage void riscv_intc_aia_irq(struct pt_regs *regs)
+{
+	unsigned long topi;
+
+	while ((topi = csr_read(CSR_TOPI)))
+		generic_handle_domain_irq(intc_domain, topi >> TOPI_IID_SHIFT);
+}
+
 /*
  * On RISC-V systems local interrupts are masked or unmasked by writing
  * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be written
@@ -39,12 +48,18 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 
 static void riscv_intc_irq_mask(struct irq_data *d)
 {
-	csr_clear(CSR_IE, BIT(d->hwirq));
+	if (IS_ENABLED(CONFIG_32BIT) && d->hwirq >= BITS_PER_LONG)
+		csr_clear(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));
+	else
+		csr_clear(CSR_IE, BIT(d->hwirq));
 }
 
 static void riscv_intc_irq_unmask(struct irq_data *d)
 {
-	csr_set(CSR_IE, BIT(d->hwirq));
+	if (IS_ENABLED(CONFIG_32BIT) && d->hwirq >= BITS_PER_LONG)
+		csr_set(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));
+	else
+		csr_set(CSR_IE, BIT(d->hwirq));
 }
 
 static void riscv_intc_irq_eoi(struct irq_data *d)
@@ -115,16 +130,18 @@ static struct fwnode_handle *riscv_intc_hwnode(void)
 
 static int __init riscv_intc_init_common(struct fwnode_handle *fn)
 {
-	int rc;
+	int rc, nr_irqs = riscv_isa_extension_available(NULL, SxAIA) ? 64 : BITS_PER_LONG;
 
-	intc_domain = irq_domain_create_linear(fn, BITS_PER_LONG,
-					       &riscv_intc_domain_ops, NULL);
+	intc_domain = irq_domain_create_linear(fn, nr_irqs, &riscv_intc_domain_ops, NULL);
 	if (!intc_domain) {
 		pr_err("unable to add IRQ domain\n");
 		return -ENXIO;
 	}
 
-	rc = set_handle_irq(&riscv_intc_irq);
+	if (riscv_isa_extension_available(NULL, SxAIA))
+		rc = set_handle_irq(&riscv_intc_aia_irq);
+	else
+		rc = set_handle_irq(&riscv_intc_irq);
 	if (rc) {
 		pr_err("failed to set irq handler\n");
 		return rc;
@@ -132,7 +149,8 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
 
 	riscv_set_intc_hwnode_fn(riscv_intc_hwnode);
 
-	pr_info("%d local interrupts mapped\n", BITS_PER_LONG);
+	pr_info("%d local interrupts mapped%s\n", nr_irqs,
+		riscv_isa_extension_available(NULL, SxAIA) ? " using AIA" : "");
 
 	return 0;
 }
Signature · 1 line
2.34.1
merge bot Applied
#34 lore raw

[tip: irq/msi] irqchip/riscv-intc: Add support for RISC-V AIA

The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     3c46fc5b5507be1f4aa144a1fbd83b0ccba04cc6
Gitweb:        https://git.kernel.org/tip/3c46fc5b5507be1f4aa144a1fbd83b0ccba04cc6
Author:        Anup Patel <[email protected]>
AuthorDate:    Thu, 22 Feb 2024 15:09:56 +05:30
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Fri, 23 Feb 2024 10:18:44 +01:00

irqchip/riscv-intc: Add support for RISC-V AIA

The RISC-V advanced interrupt architecture (AIA) extends the per-HART local interrupts in following ways:
1. Minimum 64 local interrupts for both RV32 and RV64
2. Ability to process multiple pending local interrupts in same interrupt handler
3. Priority configuration for each local interrupts
4. Special CSRs to configure/access the per-HART MSI controller

Add support for #1 and #2 described above in the RISC-V intc driver.

---
 drivers/irqchip/irq-riscv-intc.c | 32 ++++++++++++++++++++++---------
 1 file changed, 23 insertions(+), 9 deletions(-)
drivers/irqchip/irq-riscv-intc.c +23 -9
index 0cd6b48..cccb653 100644
@@ -19,6 +19,8 @@
 #include <linux/smp.h>
 #include <linux/soc/andes/irq.h>
 
+#include <asm/hwcap.h>
+
 static struct irq_domain *intc_domain;
 static unsigned int riscv_intc_nr_irqs __ro_after_init = BITS_PER_LONG;
 static unsigned int riscv_intc_custom_base __ro_after_init = BITS_PER_LONG;
@@ -32,6 +34,14 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 		pr_warn_ratelimited("Failed to handle interrupt (cause: %ld)\n", cause);
 }
 
+static asmlinkage void riscv_intc_aia_irq(struct pt_regs *regs)
+{
+	unsigned long topi;
+
+	while ((topi = csr_read(CSR_TOPI)))
+		generic_handle_domain_irq(intc_domain, topi >> TOPI_IID_SHIFT);
+}
+
 /*
  * On RISC-V systems local interrupts are masked or unmasked by writing
  * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be written
@@ -41,12 +51,18 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 
 static void riscv_intc_irq_mask(struct irq_data *d)
 {
-	csr_clear(CSR_IE, BIT(d->hwirq));
+	if (IS_ENABLED(CONFIG_32BIT) && d->hwirq >= BITS_PER_LONG)
+		csr_clear(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));
+	else
+		csr_clear(CSR_IE, BIT(d->hwirq));
 }
 
 static void riscv_intc_irq_unmask(struct irq_data *d)
 {
-	csr_set(CSR_IE, BIT(d->hwirq));
+	if (IS_ENABLED(CONFIG_32BIT) && d->hwirq >= BITS_PER_LONG)
+		csr_set(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));
+	else
+		csr_set(CSR_IE, BIT(d->hwirq));
 }
 
 static void andes_intc_irq_mask(struct irq_data *d)
@@ -157,8 +173,7 @@ static struct fwnode_handle *riscv_intc_hwnode(void)
 	return intc_domain->fwnode;
 }
 
-static int __init riscv_intc_init_common(struct fwnode_handle *fn,
-					 struct irq_chip *chip)
+static int __init riscv_intc_init_common(struct fwnode_handle *fn, struct irq_chip *chip)
 {
 	int rc;
 
@@ -176,11 +191,10 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn,
 
 	riscv_set_intc_hwnode_fn(riscv_intc_hwnode);
 
-	pr_info("%d local interrupts mapped\n", riscv_intc_nr_irqs);
-	if (riscv_intc_custom_nr_irqs) {
-		pr_info("%d custom local interrupts mapped\n",
-			riscv_intc_custom_nr_irqs);
-	}
+	pr_info("%d local interrupts mapped\n",
+		riscv_isa_extension_available(NULL, SxAIA) ? 64 : riscv_intc_nr_irqs);
+	if (riscv_intc_custom_nr_irqs)
+		pr_info("%d custom local interrupts mapped\n", riscv_intc_custom_nr_irqs);
 
 	return 0;
 }
author
#35 lore raw

PATCH v14 9/18 dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller

In reply to Anup Patel

Add DT bindings document for the RISC-V incoming MSI controller (IMSIC) defined by the RISC-V advanced interrupt architecture (AIA) specification.

---
 .../interrupt-controller/riscv,imsics.yaml    | 172 ++++++++++++++++++
 1 file changed, 172 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml
Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml new file +172 -0
new file mode 100644
index 000000000000..84976f17a4a1
@@ -0,0 +1,172 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/interrupt-controller/riscv,imsics.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: RISC-V Incoming MSI Controller (IMSIC)
+
+maintainers:
+  - Anup Patel <[email protected]>
+
+description: |
+  The RISC-V advanced interrupt architecture (AIA) defines a per-CPU incoming
+  MSI controller (IMSIC) for handling MSIs in a RISC-V platform. The RISC-V
+  AIA specification can be found at https://github.com/riscv/riscv-aia.
+
+  The IMSIC is a per-CPU (or per-HART) device with separate interrupt file
+  for each privilege level (machine or supervisor). The configuration of
+  a IMSIC interrupt file is done using AIA CSRs and it also has a 4KB MMIO
+  space to receive MSIs from devices. Each IMSIC interrupt file supports a
+  fixed number of interrupt identities (to distinguish MSIs from devices)
+  which is same for given privilege level across CPUs (or HARTs).
+
+  The device tree of a RISC-V platform will have one IMSIC device tree node
+  for each privilege level (machine or supervisor) which collectively describe
+  IMSIC interrupt files at that privilege level across CPUs (or HARTs).
+
+  The arrangement of IMSIC interrupt files in MMIO space of a RISC-V platform
+  follows a particular scheme defined by the RISC-V AIA specification. A IMSIC
+  group is a set of IMSIC interrupt files co-located in MMIO space and we can
+  have multiple IMSIC groups (i.e. clusters, sockets, chiplets, etc) in a
+  RISC-V platform. The MSI target address of a IMSIC interrupt file at given
+  privilege level (machine or supervisor) encodes group index, HART index,
+  and guest index (shown below).
+
+  XLEN-1            > (HART Index MSB)                  12    0
+  |                  |                                  |     |
+  -------------------------------------------------------------
+  |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index|  0  |
+  -------------------------------------------------------------
+
+allOf:
+  - $ref: /schemas/interrupt-controller.yaml#
+  - $ref: /schemas/interrupt-controller/msi-controller.yaml#
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - qemu,imsics
+      - const: riscv,imsics
+
+  reg:
+    minItems: 1
+    maxItems: 16384
+    description:
+      Base address of each IMSIC group.
+
+  interrupt-controller: true
+
+  "#interrupt-cells":
+    const: 0
+
+  msi-controller: true
+
+  "#msi-cells":
+    const: 0
+
+  interrupts-extended:
+    minItems: 1
+    maxItems: 16384
+    description:
+      This property represents the set of CPUs (or HARTs) for which given
+      device tree node describes the IMSIC interrupt files. Each node pointed
+      to should be a riscv,cpu-intc node, which has a CPU node (i.e. RISC-V
+      HART) as parent.
+
+  riscv,num-ids:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    minimum: 63
+    maximum: 2047
+    description:
+      Number of interrupt identities supported by IMSIC interrupt file.
+
+  riscv,num-guest-ids:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    minimum: 63
+    maximum: 2047
+    description:
+      Number of interrupt identities are supported by IMSIC guest interrupt
+      file. When not specified it is assumed to be same as specified by the
+      riscv,num-ids property.
+
+  riscv,guest-index-bits:
+    minimum: 0
+    maximum: 7
+    default: 0
+    description:
+      Number of guest index bits in the MSI target address.
+
+  riscv,hart-index-bits:
+    minimum: 0
+    maximum: 15
+    description:
+      Number of HART index bits in the MSI target address. When not
+      specified it is calculated based on the interrupts-extended property.
+
+  riscv,group-index-bits:
+    minimum: 0
+    maximum: 7
+    default: 0
+    description:
+      Number of group index bits in the MSI target address.
+
+  riscv,group-index-shift:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    minimum: 0
+    maximum: 55
+    default: 24
+    description:
+      The least significant bit position of the group index bits in the
+      MSI target address.
+
+required:
+  - compatible
+  - reg
+  - interrupt-controller
+  - msi-controller
+  - "#msi-cells"
+  - interrupts-extended
+  - riscv,num-ids
+
+unevaluatedProperties: false
+
+examples:
+  - |
+    // Example 1 (Machine-level IMSIC files with just one group):
+
+    interrupt-controller@24000000 {
+      compatible = "qemu,imsics", "riscv,imsics";
+      interrupts-extended = <&cpu1_intc 11>,
+                            <&cpu2_intc 11>,
+                            <&cpu3_intc 11>,
+                            <&cpu4_intc 11>;
+      reg = <0x28000000 0x4000>;
+      interrupt-controller;
+      #interrupt-cells = <0>;
+      msi-controller;
+      #msi-cells = <0>;
+      riscv,num-ids = <127>;
+    };
+
+  - |
+    // Example 2 (Supervisor-level IMSIC files with two groups):
+
+    interrupt-controller@28000000 {
+      compatible = "qemu,imsics", "riscv,imsics";
+      interrupts-extended = <&cpu1_intc 9>,
+                            <&cpu2_intc 9>,
+                            <&cpu3_intc 9>,
+                            <&cpu4_intc 9>;
+      reg = <0x28000000 0x2000>, /* Group0 IMSICs */
+            <0x29000000 0x2000>; /* Group1 IMSICs */
+      interrupt-controller;
+      #interrupt-cells = <0>;
+      msi-controller;
+      #msi-cells = <0>;
+      riscv,num-ids = <127>;
+      riscv,group-index-bits = <1>;
+      riscv,group-index-shift = <24>;
+    };
+...
Signature · 1 line
2.34.1
author
#36 lore raw

PATCH v14 10/18 genirq/matrix: Dynamic bitmap allocation

In reply to Anup Patel

From: Björn Töpel <[email protected]>

Some (future) users of the irq matrix allocator, do not know the size of the matrix bitmaps at compile time.

To avoid wasting memory on unnecessary large bitmaps, size the bitmap at matrix allocation time.

---
 arch/x86/include/asm/hw_irq.h |  2 --
 kernel/irq/matrix.c           | 28 +++++++++++++++++-----------
 2 files changed, 17 insertions(+), 13 deletions(-)
arch/x86/include/asm/hw_irq.h +0 -2
index b02c3cd3c0f6..edebf1020e04 100644
@@ -16,8 +16,6 @@
 
 #include <asm/irq_vectors.h>
 
-#define IRQ_MATRIX_BITS		NR_VECTORS
-
 #ifndef __ASSEMBLY__
 
 #include <linux/percpu.h>
kernel/irq/matrix.c +17 -11
index 75d0ae490e29..8f222d1cccec 100644
@@ -8,8 +8,6 @@
 #include <linux/cpu.h>
 #include <linux/irq.h>
 
-#define IRQ_MATRIX_SIZE	(BITS_TO_LONGS(IRQ_MATRIX_BITS))
-
 struct cpumap {
 	unsigned int		available;
 	unsigned int		allocated;
@@ -17,8 +15,8 @@ struct cpumap {
 	unsigned int		managed_allocated;
 	bool			initialized;
 	bool			online;
-	unsigned long		alloc_map[IRQ_MATRIX_SIZE];
-	unsigned long		managed_map[IRQ_MATRIX_SIZE];
+	unsigned long		*managed_map;
+	unsigned long		alloc_map[];
 };
 
 struct irq_matrix {
@@ -32,8 +30,8 @@ struct irq_matrix {
 	unsigned int		total_allocated;
 	unsigned int		online_maps;
 	struct cpumap __percpu	*maps;
-	unsigned long		scratch_map[IRQ_MATRIX_SIZE];
-	unsigned long		system_map[IRQ_MATRIX_SIZE];
+	unsigned long		*system_map;
+	unsigned long		scratch_map[];
 };
 
 #define CREATE_TRACE_POINTS
@@ -50,24 +48,32 @@ __init struct irq_matrix *irq_alloc_matrix(unsigned int matrix_bits,
 					   unsigned int alloc_start,
 					   unsigned int alloc_end)
 {
+	unsigned int cpu, matrix_size = BITS_TO_LONGS(matrix_bits);
 	struct irq_matrix *m;
 
-	if (matrix_bits > IRQ_MATRIX_BITS)
-		return NULL;
-
-	m = kzalloc(sizeof(*m), GFP_KERNEL);
+	m = kzalloc(struct_size(m, scratch_map, matrix_size * 2), GFP_KERNEL);
 	if (!m)
 		return NULL;
 
+	m->system_map = &m->scratch_map[matrix_size];
+
 	m->matrix_bits = matrix_bits;
 	m->alloc_start = alloc_start;
 	m->alloc_end = alloc_end;
 	m->alloc_size = alloc_end - alloc_start;
-	m->maps = alloc_percpu(*m->maps);
+	m->maps = __alloc_percpu(struct_size(m->maps, alloc_map, matrix_size * 2),
+				 __alignof__(*m->maps));
 	if (!m->maps) {
 		kfree(m);
 		return NULL;
 	}
+
+	for_each_possible_cpu(cpu) {
+		struct cpumap *cm = per_cpu_ptr(m->maps, cpu);
+
+		cm->managed_map = &cm->alloc_map[matrix_size];
+	}
+
 	return m;
 }
 
Signature · 1 line
2.34.1
merge bot Applied
#37 lore raw

[tip: irq/msi] genirq/matrix: Dynamic bitmap allocation

The following commit has been merged into the irq/msi branch of tip:

The following commit has been merged into the irq/msi branch of tip:

Commit-ID:     5b98d210ac1e4eb35abfbd940df50dec10ae81e1
Gitweb:        https://git.kernel.org/tip/5b98d210ac1e4eb35abfbd940df50dec10ae81e1
Author:        Björn Töpel <[email protected]>
AuthorDate:    Thu, 22 Feb 2024 15:09:58 +05:30
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Fri, 23 Feb 2024 10:18:44 +01:00

genirq/matrix: Dynamic bitmap allocation

A future user of the matrix allocator, does not know the size of the matrix bitmaps at compile time.

To avoid wasting memory on unnecessary large bitmaps, size the bitmap at matrix allocation time.

---
 arch/x86/include/asm/hw_irq.h |  2 --
 kernel/irq/matrix.c           | 28 +++++++++++++++++-----------
 2 files changed, 17 insertions(+), 13 deletions(-)
arch/x86/include/asm/hw_irq.h +0 -2
index b02c3cd..edebf10 100644
@@ -16,8 +16,6 @@
 
 #include <asm/irq_vectors.h>
 
-#define IRQ_MATRIX_BITS		NR_VECTORS
-
 #ifndef __ASSEMBLY__
 
 #include <linux/percpu.h>
kernel/irq/matrix.c +17 -11
index 75d0ae4..8f222d1 100644
@@ -8,8 +8,6 @@
 #include <linux/cpu.h>
 #include <linux/irq.h>
 
-#define IRQ_MATRIX_SIZE	(BITS_TO_LONGS(IRQ_MATRIX_BITS))
-
 struct cpumap {
 	unsigned int		available;
 	unsigned int		allocated;
@@ -17,8 +15,8 @@ struct cpumap {
 	unsigned int		managed_allocated;
 	bool			initialized;
 	bool			online;
-	unsigned long		alloc_map[IRQ_MATRIX_SIZE];
-	unsigned long		managed_map[IRQ_MATRIX_SIZE];
+	unsigned long		*managed_map;
+	unsigned long		alloc_map[];
 };
 
 struct irq_matrix {
@@ -32,8 +30,8 @@ struct irq_matrix {
 	unsigned int		total_allocated;
 	unsigned int		online_maps;
 	struct cpumap __percpu	*maps;
-	unsigned long		scratch_map[IRQ_MATRIX_SIZE];
-	unsigned long		system_map[IRQ_MATRIX_SIZE];
+	unsigned long		*system_map;
+	unsigned long		scratch_map[];
 };
 
 #define CREATE_TRACE_POINTS
@@ -50,24 +48,32 @@ __init struct irq_matrix *irq_alloc_matrix(unsigned int matrix_bits,
 					   unsigned int alloc_start,
 					   unsigned int alloc_end)
 {
+	unsigned int cpu, matrix_size = BITS_TO_LONGS(matrix_bits);
 	struct irq_matrix *m;
 
-	if (matrix_bits > IRQ_MATRIX_BITS)
-		return NULL;
-
-	m = kzalloc(sizeof(*m), GFP_KERNEL);
+	m = kzalloc(struct_size(m, scratch_map, matrix_size * 2), GFP_KERNEL);
 	if (!m)
 		return NULL;
 
+	m->system_map = &m->scratch_map[matrix_size];
+
 	m->matrix_bits = matrix_bits;
 	m->alloc_start = alloc_start;
 	m->alloc_end = alloc_end;
 	m->alloc_size = alloc_end - alloc_start;
-	m->maps = alloc_percpu(*m->maps);
+	m->maps = __alloc_percpu(struct_size(m->maps, alloc_map, matrix_size * 2),
+				 __alignof__(*m->maps));
 	if (!m->maps) {
 		kfree(m);
 		return NULL;
 	}
+
+	for_each_possible_cpu(cpu) {
+		struct cpumap *cm = per_cpu_ptr(m->maps, cpu);
+
+		cm->managed_map = &cm->alloc_map[matrix_size];
+	}
+
 	return m;
 }
 
author
#38 lore raw

PATCH v14 11/18 irqchip: Add RISC-V incoming MSI controller early driver

In reply to Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new MSI controller called incoming message signalled interrupt controller (IMSIC) which manages MSI on per-HART (or per-CPU) basis. It also supports IPIs as software injected MSIs. (For more details refer https://github.com/riscv/riscv-aia)

Add an early irqchip driver for RISC-V IMSIC which sets up the IMSIC state and provide IPIs.

---
 drivers/irqchip/Kconfig                 |   7 +
 drivers/irqchip/Makefile                |   1 +
 drivers/irqchip/irq-riscv-imsic-early.c | 201 ++++++
 drivers/irqchip/irq-riscv-imsic-state.c | 870 ++++++++++++++++++++++++
 drivers/irqchip/irq-riscv-imsic-state.h | 107 +++
 include/linux/irqchip/riscv-imsic.h     |  87 +++
 6 files changed, 1273 insertions(+)
 create mode 100644 drivers/irqchip/irq-riscv-imsic-early.c
 create mode 100644 drivers/irqchip/irq-riscv-imsic-state.c
 create mode 100644 drivers/irqchip/irq-riscv-imsic-state.h
 create mode 100644 include/linux/irqchip/riscv-imsic.h
drivers/irqchip/Kconfig +7 -0
index f7149d0f3d45..85f86e31c996 100644
@@ -546,6 +546,13 @@ config SIFIVE_PLIC
 	select IRQ_DOMAIN_HIERARCHY
 	select GENERIC_IRQ_EFFECTIVE_AFF_MASK if SMP
 
+config RISCV_IMSIC
+	bool
+	depends on RISCV
+	select IRQ_DOMAIN_HIERARCHY
+	select GENERIC_IRQ_MATRIX_ALLOCATOR
+	select GENERIC_MSI_IRQ
+
 config EXYNOS_IRQ_COMBINER
 	bool "Samsung Exynos IRQ combiner support" if COMPILE_TEST
 	depends on (ARCH_EXYNOS && ARM) || COMPILE_TEST
drivers/irqchip/Makefile +1 -0
index ffd945fe71aa..d714724387ce 100644
@@ -95,6 +95,7 @@ obj-$(CONFIG_QCOM_MPM)			+= irq-qcom-mpm.o
 obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
 obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
 obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
+obj-$(CONFIG_RISCV_IMSIC)		+= irq-riscv-imsic-state.o irq-riscv-imsic-early.o
 obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
 obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
 obj-$(CONFIG_IMX_INTMUX)		+= irq-imx-intmux.o
drivers/irqchip/irq-riscv-imsic-early.c new file +201 -0

201 lines not on this page · raw message

drivers/irqchip/irq-riscv-imsic-state.c new file +870 -0

870 lines not on this page · raw message

drivers/irqchip/irq-riscv-imsic-state.h new file +107 -0

107 lines not on this page · raw message

include/linux/irqchip/riscv-imsic.h new file +87 -0

87 lines not on this page · raw message

Signature · 1 line
2.34.1
#39 lore raw

Re: [PATCH v14 11/18] irqchip: Add RISC-V incoming MSI controller early driver

Quoted 6 lines · Anup Patel
drivers/irqchip/irq-riscv-imsic-state.c new file +0 -0
new file mode 100644
index 000000000000..0c19ffb9ca3e
@@ -0,0 +1,870 @@

[...]

drivers/irqchip/irq-riscv-imsic-state.c line 135

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+static void __imsic_local_sync(struct imsic_local_priv *lpriv)
+{
+	struct imsic_local_config *mlocal;
+	struct imsic_vector *vec, *mvec;
+	int i;
+
+	lockdep_assert_held(&lpriv->lock);
+
+	/* This pairs with the barrier in __imsic_remote_sync(). */
+	smp_mb();

I'm trying to figure out why this barrier is needed? All the updates are done behind the spinlocks. If there're some ordering constraints that I'm missing, please document them.

Quoted 9 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +9 -0
+
+	for_each_set_bit(i, lpriv->dirty_bitmap, imsic->global.nr_ids + 1) {
+		if (!i || i == IMSIC_IPI_ID)
+			goto skip;
+		vec = &lpriv->vectors[i];
+
+		if (READ_ONCE(vec->enable))
+			__imsic_id_set_enable(i);
+		else

drivers/irqchip/irq-riscv-imsic-state.c line 154

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+			__imsic_id_clear_enable(i);
+
+		/*
+		 * If the ID was being moved to a new ID on some other CPU
+		 * then we can get a MSI during the movement so check the
+		 * ID pending bit and re-trigger the new ID on other CPU
+		 * using MMIO write.
+		 */
+		mvec = READ_ONCE(vec->move);
+		WRITE_ONCE(vec->move, NULL);

mvec = xchg(&vec->move, NULL) ?

Quoted 45 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +45 -0
+		if (mvec && mvec != vec) {
+			if (__imsic_id_read_clear_pending(i)) {
+				mlocal = per_cpu_ptr(imsic->global.local, mvec->cpu);
+				writel_relaxed(mvec->local_id, mlocal->msi_va);
+			}
+
+			imsic_vector_free(&lpriv->vectors[i]);
+		}
+
+skip:
+		bitmap_clear(lpriv->dirty_bitmap, i, 1);
+	}
+}
+
+void imsic_local_sync_all(void)
+{
+	struct imsic_local_priv *lpriv = this_cpu_ptr(imsic->lpriv);
+	unsigned long flags;
+
+	raw_spin_lock_irqsave(&lpriv->lock, flags);
+	bitmap_fill(lpriv->dirty_bitmap, imsic->global.nr_ids + 1);
+	__imsic_local_sync(lpriv);
+	raw_spin_unlock_irqrestore(&lpriv->lock, flags);
+}
+
+void imsic_local_delivery(bool enable)
+{
+	if (enable) {
+		imsic_csr_write(IMSIC_EITHRESHOLD, IMSIC_ENABLE_EITHRESHOLD);
+		imsic_csr_write(IMSIC_EIDELIVERY, IMSIC_ENABLE_EIDELIVERY);
+		return;
+	}
+
+	imsic_csr_write(IMSIC_EIDELIVERY, IMSIC_DISABLE_EIDELIVERY);
+	imsic_csr_write(IMSIC_EITHRESHOLD, IMSIC_DISABLE_EITHRESHOLD);
+}
+
+#ifdef CONFIG_SMP
+static void imsic_local_timer_callback(struct timer_list *timer)
+{
+	struct imsic_local_priv *lpriv = this_cpu_ptr(imsic->lpriv);
+	unsigned long flags;
+
+	raw_spin_lock_irqsave(&lpriv->lock, flags);
+	__imsic_local_sync(lpriv);

drivers/irqchip/irq-riscv-imsic-state.c line 209

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+	raw_spin_unlock_irqrestore(&lpriv->lock, flags);
+}
+
+static void __imsic_remote_sync(struct imsic_local_priv *lpriv, unsigned int cpu)
+{
+	lockdep_assert_held(&lpriv->lock);
+
+	/*
+	 * Ensure that changes to vector enable, vector move and
+	 * dirty bitmap are visible to the target CPU.

...which case the spinlock(s) are enough, no?

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+	 *
+	 * This pairs with the barrier in __imsic_local_sync().
+	 */
+	smp_mb();
+
+	/*

drivers/irqchip/irq-riscv-imsic-state.c line 225

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+	 * We schedule a timer on the target CPU if the target CPU is not
+	 * same as the current CPU. An offline CPU will unconditionally
+	 * synchronize IDs through imsic_starting_cpu() when the
+	 * CPU is brought up.
+	 */
+	if (cpu_online(cpu)) {
+		if (cpu == smp_processor_id()) {
+			__imsic_local_sync(lpriv);
+			return;
+		}

Maybe move this if-clause out from the cpu_online(), and only have something like
if (cpu_online(cpu) && !timer_pending(&lpriv->timer)) { ... }
inside the CONFIG_SMP guard...

Quoted 4 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +4 -0
+
+		if (!timer_pending(&lpriv->timer)) {
+			lpriv->timer.expires = jiffies + 1;
+			add_timer_on(&lpriv->timer, cpu);

drivers/irqchip/irq-riscv-imsic-state.c line 239

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+		}
+	}
+}
+#else
+static void __imsic_remote_sync(struct imsic_local_priv *lpriv, unsigned int cpu)
+{
+	lockdep_assert_held(&lpriv->lock);
+	__imsic_local_sync(lpriv);
+}
+#endif

...where we can get rid of this special !SMP all together for this function.

Quoted 7 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +7 -0
+
+void imsic_vector_mask(struct imsic_vector *vec)
+{
+	struct imsic_local_priv *lpriv;
+
+	lpriv = per_cpu_ptr(imsic->lpriv, vec->cpu);
+	if (WARN_ON_ONCE(&lpriv->vectors[vec->local_id] != vec))

drivers/irqchip/irq-riscv-imsic-state.c line 256

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+		return;
+
+	/*
+	 * This function is called through Linux irq subsystem with
+	 * irqs disabled so no need to save/restore irq flags.
+	 */
+
+	raw_spin_lock(&lpriv->lock);
+
+	vec->enable = false;

Should have WRITE_ONCE to make the checkers happy.

Quoted 12 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +12 -0
+	bitmap_set(lpriv->dirty_bitmap, vec->local_id, 1);
+	__imsic_remote_sync(lpriv, vec->cpu);
+
+	raw_spin_unlock(&lpriv->lock);
+}
+
+void imsic_vector_unmask(struct imsic_vector *vec)
+{
+	struct imsic_local_priv *lpriv;
+
+	lpriv = per_cpu_ptr(imsic->lpriv, vec->cpu);
+	if (WARN_ON_ONCE(&lpriv->vectors[vec->local_id] != vec))

drivers/irqchip/irq-riscv-imsic-state.c line 278

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+		return;
+
+	/*
+	 * This function is called through Linux irq subsystem with
+	 * irqs disabled so no need to save/restore irq flags.
+	 */
+
+	raw_spin_lock(&lpriv->lock);
+
+	vec->enable = true;

Dito.

Quoted 7 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +7 -0
+	bitmap_set(lpriv->dirty_bitmap, vec->local_id, 1);
+	__imsic_remote_sync(lpriv, vec->cpu);
+
+	raw_spin_unlock(&lpriv->lock);
+}
+
+static bool imsic_vector_move_update(struct imsic_local_priv *lpriv, struct imsic_vector *vec,

drivers/irqchip/irq-riscv-imsic-state.c line 295

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+				     bool new_enable, struct imsic_vector *new_move)
+{
+	unsigned long flags;
+	bool enabled;
+
+	raw_spin_lock_irqsave(&lpriv->lock, flags);
+
+	/* Update enable and move details */
+	enabled = READ_ONCE(vec->enable);
+	WRITE_ONCE(vec->enable, new_enable);

Again, xchg() would be easier to read.

Björn

maintainer
#40 lore raw

Re: [PATCH v14 11/18] irqchip: Add RISC-V incoming MSI controller early driver

Quoted 40 lines · Björn Töpel
Quoted 6 lines · Anup Patel
drivers/irqchip/irq-riscv-imsic-state.c new file +0 -0
new file mode 100644
index 000000000000..0c19ffb9ca3e
@@ -0,0 +1,870 @@

[...]

drivers/irqchip/irq-riscv-imsic-state.c line 135

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+static void __imsic_local_sync(struct imsic_local_priv *lpriv)
+{
+     struct imsic_local_config *mlocal;
+     struct imsic_vector *vec, *mvec;
+     int i;
+
+     lockdep_assert_held(&lpriv->lock);
+
+     /* This pairs with the barrier in __imsic_remote_sync(). */
+     smp_mb();

I'm trying to figure out why this barrier is needed? All the updates are done behind the spinlocks. If there're some ordering constraints that I'm missing, please document them.

Quoted 13 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +13 -0
+
+     for_each_set_bit(i, lpriv->dirty_bitmap, imsic->global.nr_ids + 1) {
+             if (!i || i == IMSIC_IPI_ID)
+                     goto skip;
+             vec = &lpriv->vectors[i];
+
+             if (READ_ONCE(vec->enable))
+                     __imsic_id_set_enable(i);
+             else
+                     __imsic_id_clear_enable(i);
+
+             /*
+              * If the ID was being moved to a new ID on some other CPU
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 154

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+              * then we can get a MSI during the movement so check the
+              * ID pending bit and re-trigger the new ID on other CPU
+              * using MMIO write.
+              */
+             mvec = READ_ONCE(vec->move);
+             WRITE_ONCE(vec->move, NULL);

mvec = xchg(&vec->move, NULL) ?

The __imsic_local_sync() is called with spinlock held.

Quoted 50 lines
Quoted 50 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +49 -0
+             if (mvec && mvec != vec) {
+                     if (__imsic_id_read_clear_pending(i)) {
+                             mlocal = per_cpu_ptr(imsic->global.local, mvec->cpu);
+                             writel_relaxed(mvec->local_id, mlocal->msi_va);
+                     }
+
+                     imsic_vector_free(&lpriv->vectors[i]);
+             }
+
+skip:
+             bitmap_clear(lpriv->dirty_bitmap, i, 1);
+     }
+}
+
+void imsic_local_sync_all(void)
+{
+     struct imsic_local_priv *lpriv = this_cpu_ptr(imsic->lpriv);
+     unsigned long flags;
+
+     raw_spin_lock_irqsave(&lpriv->lock, flags);
+     bitmap_fill(lpriv->dirty_bitmap, imsic->global.nr_ids + 1);
+     __imsic_local_sync(lpriv);
+     raw_spin_unlock_irqrestore(&lpriv->lock, flags);
+}
+
+void imsic_local_delivery(bool enable)
+{
+     if (enable) {
+             imsic_csr_write(IMSIC_EITHRESHOLD, IMSIC_ENABLE_EITHRESHOLD);
+             imsic_csr_write(IMSIC_EIDELIVERY, IMSIC_ENABLE_EIDELIVERY);
+             return;
+     }
+
+     imsic_csr_write(IMSIC_EIDELIVERY, IMSIC_DISABLE_EIDELIVERY);
+     imsic_csr_write(IMSIC_EITHRESHOLD, IMSIC_DISABLE_EITHRESHOLD);
+}
+
+#ifdef CONFIG_SMP
+static void imsic_local_timer_callback(struct timer_list *timer)
+{
+     struct imsic_local_priv *lpriv = this_cpu_ptr(imsic->lpriv);
+     unsigned long flags;
+
+     raw_spin_lock_irqsave(&lpriv->lock, flags);
+     __imsic_local_sync(lpriv);
+     raw_spin_unlock_irqrestore(&lpriv->lock, flags);
+}
+
+static void __imsic_remote_sync(struct imsic_local_priv *lpriv, unsigned int cpu)
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 209

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+{
+     lockdep_assert_held(&lpriv->lock);
+
+     /*
+      * Ensure that changes to vector enable, vector move and
+      * dirty bitmap are visible to the target CPU.

...which case the spinlock(s) are enough, no?

spinlocks are not fences.

If the timer wheel on the target cpu is already running and we don't have a fence here then the target cpu might not see the changes done by this cpu.

Quoted 32 lines
Quoted 7 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+      *
+      * This pairs with the barrier in __imsic_local_sync().
+      */
+     smp_mb();
+
+     /*

drivers/irqchip/irq-riscv-imsic-state.c line 225

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+      * We schedule a timer on the target CPU if the target CPU is not
+      * same as the current CPU. An offline CPU will unconditionally
+      * synchronize IDs through imsic_starting_cpu() when the
+      * CPU is brought up.
+      */
+     if (cpu_online(cpu)) {
+             if (cpu == smp_processor_id()) {
+                     __imsic_local_sync(lpriv);
+                     return;
+             }

Maybe move this if-clause out from the cpu_online(), and only have something like
if (cpu_online(cpu) && !timer_pending(&lpriv->timer)) { ... }
inside the CONFIG_SMP guard...

Quoted 9 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +9 -0
+
+             if (!timer_pending(&lpriv->timer)) {
+                     lpriv->timer.expires = jiffies + 1;
+                     add_timer_on(&lpriv->timer, cpu);
+             }
+     }
+}
+#else
+static void __imsic_remote_sync(struct imsic_local_priv *lpriv, unsigned int cpu)
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 239

Quoted 5 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +5 -0
+{
+     lockdep_assert_held(&lpriv->lock);
+     __imsic_local_sync(lpriv);
+}
+#endif

...where we can get rid of this special !SMP all together for this function.

I failed to understand what is wrong the current code.

Quoted 12 lines
Quoted 12 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +11 -0
+
+void imsic_vector_mask(struct imsic_vector *vec)
+{
+     struct imsic_local_priv *lpriv;
+
+     lpriv = per_cpu_ptr(imsic->lpriv, vec->cpu);
+     if (WARN_ON_ONCE(&lpriv->vectors[vec->local_id] != vec))
+             return;
+
+     /*
+      * This function is called through Linux irq subsystem with
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 256

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+      * irqs disabled so no need to save/restore irq flags.
+      */
+
+     raw_spin_lock(&lpriv->lock);
+
+     vec->enable = false;

Should have WRITE_ONCE to make the checkers happy.

Okay, I will update.

Quoted 17 lines
Quoted 17 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +16 -0
+     bitmap_set(lpriv->dirty_bitmap, vec->local_id, 1);
+     __imsic_remote_sync(lpriv, vec->cpu);
+
+     raw_spin_unlock(&lpriv->lock);
+}
+
+void imsic_vector_unmask(struct imsic_vector *vec)
+{
+     struct imsic_local_priv *lpriv;
+
+     lpriv = per_cpu_ptr(imsic->lpriv, vec->cpu);
+     if (WARN_ON_ONCE(&lpriv->vectors[vec->local_id] != vec))
+             return;
+
+     /*
+      * This function is called through Linux irq subsystem with
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 278

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+      * irqs disabled so no need to save/restore irq flags.
+      */
+
+     raw_spin_lock(&lpriv->lock);
+
+     vec->enable = true;

Dito.

Okay, I will update.

Quoted 12 lines
Quoted 12 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +11 -0
+     bitmap_set(lpriv->dirty_bitmap, vec->local_id, 1);
+     __imsic_remote_sync(lpriv, vec->cpu);
+
+     raw_spin_unlock(&lpriv->lock);
+}
+
+static bool imsic_vector_move_update(struct imsic_local_priv *lpriv, struct imsic_vector *vec,
+                                  bool new_enable, struct imsic_vector *new_move)
+{
+     unsigned long flags;
+     bool enabled;
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 295

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+
+     raw_spin_lock_irqsave(&lpriv->lock, flags);
+
+     /* Update enable and move details */
+     enabled = READ_ONCE(vec->enable);
+     WRITE_ONCE(vec->enable, new_enable);

Again, xchg() would be easier to read.

why ? spinlock is not enough?

Regards,
Anup

#41 lore raw

Re: [PATCH v14 11/18] irqchip: Add RISC-V incoming MSI controller early driver

Quoted 50 lines · Anup Patel
Quoted 40 lines · Björn Töpel
Quoted 6 lines · Anup Patel
drivers/irqchip/irq-riscv-imsic-state.c new file +0 -0
new file mode 100644
index 000000000000..0c19ffb9ca3e
@@ -0,0 +1,870 @@

[...]

drivers/irqchip/irq-riscv-imsic-state.c line 135

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+static void __imsic_local_sync(struct imsic_local_priv *lpriv)
+{
+     struct imsic_local_config *mlocal;
+     struct imsic_vector *vec, *mvec;
+     int i;
+
+     lockdep_assert_held(&lpriv->lock);
+
+     /* This pairs with the barrier in __imsic_remote_sync(). */
+     smp_mb();

I'm trying to figure out why this barrier is needed? All the updates are done behind the spinlocks. If there're some ordering constraints that I'm missing, please document them.

Quoted 13 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +13 -0
+
+     for_each_set_bit(i, lpriv->dirty_bitmap, imsic->global.nr_ids + 1) {
+             if (!i || i == IMSIC_IPI_ID)
+                     goto skip;
+             vec = &lpriv->vectors[i];
+
+             if (READ_ONCE(vec->enable))
+                     __imsic_id_set_enable(i);
+             else
+                     __imsic_id_clear_enable(i);
+
+             /*
+              * If the ID was being moved to a new ID on some other CPU
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 154

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+              * then we can get a MSI during the movement so check the
+              * ID pending bit and re-trigger the new ID on other CPU
+              * using MMIO write.
+              */
+             mvec = READ_ONCE(vec->move);
+             WRITE_ONCE(vec->move, NULL);

mvec = xchg(&vec->move, NULL) ?

Quoted 1 line

The __imsic_local_sync() is called with spinlock held.

Yeah, this was a readability comment.

Quoted 58 lines
Quoted 49 lines
Quoted 49 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +49 -0
+             if (mvec && mvec != vec) {
+                     if (__imsic_id_read_clear_pending(i)) {
+                             mlocal = per_cpu_ptr(imsic->global.local, mvec->cpu);
+                             writel_relaxed(mvec->local_id, mlocal->msi_va);
+                     }
+
+                     imsic_vector_free(&lpriv->vectors[i]);
+             }
+
+skip:
+             bitmap_clear(lpriv->dirty_bitmap, i, 1);
+     }
+}
+
+void imsic_local_sync_all(void)
+{
+     struct imsic_local_priv *lpriv = this_cpu_ptr(imsic->lpriv);
+     unsigned long flags;
+
+     raw_spin_lock_irqsave(&lpriv->lock, flags);
+     bitmap_fill(lpriv->dirty_bitmap, imsic->global.nr_ids + 1);
+     __imsic_local_sync(lpriv);
+     raw_spin_unlock_irqrestore(&lpriv->lock, flags);
+}
+
+void imsic_local_delivery(bool enable)
+{
+     if (enable) {
+             imsic_csr_write(IMSIC_EITHRESHOLD, IMSIC_ENABLE_EITHRESHOLD);
+             imsic_csr_write(IMSIC_EIDELIVERY, IMSIC_ENABLE_EIDELIVERY);
+             return;
+     }
+
+     imsic_csr_write(IMSIC_EIDELIVERY, IMSIC_DISABLE_EIDELIVERY);
+     imsic_csr_write(IMSIC_EITHRESHOLD, IMSIC_DISABLE_EITHRESHOLD);
+}
+
+#ifdef CONFIG_SMP
+static void imsic_local_timer_callback(struct timer_list *timer)
+{
+     struct imsic_local_priv *lpriv = this_cpu_ptr(imsic->lpriv);
+     unsigned long flags;
+
+     raw_spin_lock_irqsave(&lpriv->lock, flags);
+     __imsic_local_sync(lpriv);
+     raw_spin_unlock_irqrestore(&lpriv->lock, flags);
+}
+
+static void __imsic_remote_sync(struct imsic_local_priv *lpriv, unsigned int cpu)
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 209

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+{
+     lockdep_assert_held(&lpriv->lock);
+
+     /*
+      * Ensure that changes to vector enable, vector move and
+      * dirty bitmap are visible to the target CPU.

...which case the spinlock(s) are enough, no?

Quoted 1 line

spinlocks are not fences.

They are indeed, and it would be hard to use them as locks otherwise (acq/rel -- good ol' Documentation/memory-barriers.txt).

Quoted 3 lines

If the timer wheel on the target cpu is already running and we don't have a fence here then the target cpu might not see the changes done by this cpu.

Remove the smp_mb() pair, the spinlocks are enough for this scenario!

Quoted 40 lines
Quoted 31 lines
Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+      *
+      * This pairs with the barrier in __imsic_local_sync().
+      */
+     smp_mb();
+
+     /*

drivers/irqchip/irq-riscv-imsic-state.c line 225

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+      * We schedule a timer on the target CPU if the target CPU is not
+      * same as the current CPU. An offline CPU will unconditionally
+      * synchronize IDs through imsic_starting_cpu() when the
+      * CPU is brought up.
+      */
+     if (cpu_online(cpu)) {
+             if (cpu == smp_processor_id()) {
+                     __imsic_local_sync(lpriv);
+                     return;
+             }

Maybe move this if-clause out from the cpu_online(), and only have something like
if (cpu_online(cpu) && !timer_pending(&lpriv->timer)) { ... }
inside the CONFIG_SMP guard...

Quoted 9 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +9 -0
+
+             if (!timer_pending(&lpriv->timer)) {
+                     lpriv->timer.expires = jiffies + 1;
+                     add_timer_on(&lpriv->timer, cpu);
+             }
+     }
+}
+#else
+static void __imsic_remote_sync(struct imsic_local_priv *lpriv, unsigned int cpu)
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 239

Quoted 5 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +5 -0
+{
+     lockdep_assert_held(&lpriv->lock);
+     __imsic_local_sync(lpriv);
+}
+#endif

...where we can get rid of this special !SMP all together for this function.

Quoted 1 line

I failed to understand what is wrong the current code.

Oh, nothing is wrong. Just trying to get rid of some ifdeffery.

Quoted 70 lines
Quoted 10 lines
Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +10 -0
+void imsic_vector_mask(struct imsic_vector *vec)
+{
+     struct imsic_local_priv *lpriv;
+
+     lpriv = per_cpu_ptr(imsic->lpriv, vec->cpu);
+     if (WARN_ON_ONCE(&lpriv->vectors[vec->local_id] != vec))
+             return;
+
+     /*
+      * This function is called through Linux irq subsystem with
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 256

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+      * irqs disabled so no need to save/restore irq flags.
+      */
+
+     raw_spin_lock(&lpriv->lock);
+
+     vec->enable = false;

Should have WRITE_ONCE to make the checkers happy.

Okay, I will update.

Quoted 17 lines
Quoted 17 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +16 -0
+     bitmap_set(lpriv->dirty_bitmap, vec->local_id, 1);
+     __imsic_remote_sync(lpriv, vec->cpu);
+
+     raw_spin_unlock(&lpriv->lock);
+}
+
+void imsic_vector_unmask(struct imsic_vector *vec)
+{
+     struct imsic_local_priv *lpriv;
+
+     lpriv = per_cpu_ptr(imsic->lpriv, vec->cpu);
+     if (WARN_ON_ONCE(&lpriv->vectors[vec->local_id] != vec))
+             return;
+
+     /*
+      * This function is called through Linux irq subsystem with
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 278

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+      * irqs disabled so no need to save/restore irq flags.
+      */
+
+     raw_spin_lock(&lpriv->lock);
+
+     vec->enable = true;

Dito.

Okay, I will update.

Quoted 12 lines
Quoted 12 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +11 -0
+     bitmap_set(lpriv->dirty_bitmap, vec->local_id, 1);
+     __imsic_remote_sync(lpriv, vec->cpu);
+
+     raw_spin_unlock(&lpriv->lock);
+}
+
+static bool imsic_vector_move_update(struct imsic_local_priv *lpriv, struct imsic_vector *vec,
+                                  bool new_enable, struct imsic_vector *new_move)
+{
+     unsigned long flags;
+     bool enabled;
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-state.c line 295

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +6 -0
+
+     raw_spin_lock_irqsave(&lpriv->lock, flags);
+
+     /* Update enable and move details */
+     enabled = READ_ONCE(vec->enable);
+     WRITE_ONCE(vec->enable, new_enable);

Again, xchg() would be easier to read.

Quoted 1 line

why ? spinlock is not enough?

They're enough! Just readbaility/personal taste.

I'm running tests for this series now! Would be awesome to have the series land for 6.9!

Cheers,
Björn

#42 lore raw

Re: [PATCH v14 11/18] irqchip: Add RISC-V incoming MSI controller early driver

In reply to Anup Patel

drivers/irqchip/irq-riscv-imsic-early.c line 174

Quoted 9 lines · Anup Patel
drivers/irqchip/irq-riscv-imsic-early.c new file +9 -0
+	/*
+	 * Setup cpuhp state (must be done after setting imsic_parent_irq)
+	 *
+	 * Don't disable per-CPU IMSIC file when CPU goes offline
+	 * because this affects IPI and the masking/unmasking of
+	 * virtual IPIs is done via generic IPI-Mux
+	 */
+	cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "irqchip/riscv/imsic:starting",
+			  imsic_starting_cpu, imsic_dying_cpu);

This is not really correct. IPIs should be working right away when a CPU comes online and on the unplug side until it really goes offline.

So this wants to be in the starting range, i.e. between CPUHP_AP_OFFLINE and CPUHP_AP_ONLINE. No?

Thanks,

tglx

maintainer
#43 lore raw

Re: [PATCH v14 11/18] irqchip: Add RISC-V incoming MSI controller early driver

Quoted 12 lines · Thomas Gleixner

drivers/irqchip/irq-riscv-imsic-early.c line 174

Quoted 9 lines · Anup Patel
drivers/irqchip/irq-riscv-imsic-early.c new file +9 -0
+     /*
+      * Setup cpuhp state (must be done after setting imsic_parent_irq)
+      *
+      * Don't disable per-CPU IMSIC file when CPU goes offline
+      * because this affects IPI and the masking/unmasking of
+      * virtual IPIs is done via generic IPI-Mux
+      */
+     cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "irqchip/riscv/imsic:starting",
+                       imsic_starting_cpu, imsic_dying_cpu);
Quoted 5 lines

This is not really correct. IPIs should be working right away when a CPU comes online and on the unplug side until it really goes offline.

So this wants to be in the starting range, i.e. between CPUHP_AP_OFFLINE and CPUHP_AP_ONLINE. No?

Yes, it has to be as early as possible but I was not sure about introducing yet another driver specific CPUHP_AP_xyz state considering CPUHP_AP_ONLINE_DYN worked fine.

Since you suggested, let me introduce CPUHP_AP_IRQ_RISCV_IMSIC_STARTING in the next revision.

Regards,
Anup

author
#44 lore raw

PATCH v14 12/18 irqchip/riscv-imsic: Add device MSI domain support for platform devices

In reply to Anup Patel

The Linux platform MSI support allows per-device MSI domains so add a platform irqchip driver for RISC-V IMSIC which provides a base IRQ domain with MSI parent support for platform device domains.

The IMSIC platform driver assumes that the IMSIC state is already initialized by the IMSIC early driver.

---
 drivers/irqchip/Makefile                   |   2 +-
 drivers/irqchip/irq-riscv-imsic-platform.c | 344 +++++++++++++++++++++
 drivers/irqchip/irq-riscv-imsic-state.h    |   1 +
 3 files changed, 346 insertions(+), 1 deletion(-)
 create mode 100644 drivers/irqchip/irq-riscv-imsic-platform.c
drivers/irqchip/Makefile +1 -1
index d714724387ce..abca445a3229 100644
@@ -95,7 +95,7 @@ obj-$(CONFIG_QCOM_MPM)			+= irq-qcom-mpm.o
 obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
 obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
 obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
-obj-$(CONFIG_RISCV_IMSIC)		+= irq-riscv-imsic-state.o irq-riscv-imsic-early.o
+obj-$(CONFIG_RISCV_IMSIC)		+= irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o
 obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
 obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
 obj-$(CONFIG_IMX_INTMUX)		+= irq-imx-intmux.o
drivers/irqchip/irq-riscv-imsic-platform.c new file +344 -0

344 lines not on this page · raw message

drivers/irqchip/irq-riscv-imsic-state.h +1 -0
index 8ec9649d0d01..5ae2f69b035b 100644
@@ -103,5 +103,6 @@ void imsic_vector_debug_show_summary(struct seq_file *m, int ind);
 void imsic_state_online(void);
 void imsic_state_offline(void);
 int imsic_setup_state(struct fwnode_handle *fwnode);
+int imsic_irqdomain_init(void);
 
 #endif
Signature · 1 line
2.34.1
#45 lore raw

Re: [PATCH v14 12/18] irqchip/riscv-imsic: Add device MSI domain support for platform devices

Quoted 61 lines · Anup Patel

The Linux platform MSI support allows per-device MSI domains so add a platform irqchip driver for RISC-V IMSIC which provides a base IRQ domain with MSI parent support for platform device domains.

The IMSIC platform driver assumes that the IMSIC state is already initialized by the IMSIC early driver.

---
 drivers/irqchip/Makefile                   |   2 +-
 drivers/irqchip/irq-riscv-imsic-platform.c | 344 +++++++++++++++++++++
 drivers/irqchip/irq-riscv-imsic-state.h    |   1 +
 3 files changed, 346 insertions(+), 1 deletion(-)
 create mode 100644 drivers/irqchip/irq-riscv-imsic-platform.c
drivers/irqchip/Makefile +1 -1
index d714724387ce..abca445a3229 100644
@@ -95,7 +95,7 @@ obj-$(CONFIG_QCOM_MPM)			+= irq-qcom-mpm.o
 obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
 obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
 obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
-obj-$(CONFIG_RISCV_IMSIC)		+= irq-riscv-imsic-state.o irq-riscv-imsic-early.o
+obj-$(CONFIG_RISCV_IMSIC)		+= irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o
 obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
 obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
 obj-$(CONFIG_IMX_INTMUX)		+= irq-imx-intmux.o
drivers/irqchip/irq-riscv-imsic-platform.c new file +27 -0
new file mode 100644
index 000000000000..e2344fc08dca
@@ -0,0 +1,344 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 Western Digital Corporation or its affiliates.
+ * Copyright (C) 2022 Ventana Micro Systems Inc.
+ */
+
+#define pr_fmt(fmt) "riscv-imsic: " fmt
+#include <linux/bitmap.h>
+#include <linux/cpu.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/irq.h>
+#include <linux/irqchip.h>
+#include <linux/irqdomain.h>
+#include <linux/module.h>
+#include <linux/msi.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/smp.h>
+
+#include "irq-riscv-imsic-state.h"
+
+static bool imsic_cpu_page_phys(unsigned int cpu, unsigned int guest_index,
+				phys_addr_t *out_msi_pa)
+{
+	struct imsic_global_config *global;
+	struct imsic_local_config *local;

drivers/irqchip/irq-riscv-imsic-platform.c line 37

Quoted 10 lines
drivers/irqchip/irq-riscv-imsic-platform.c new file +10 -0
+
+	global = &imsic->global;
+	local = per_cpu_ptr(global->local, cpu);
+
+	if (BIT(global->guest_index_bits) <= guest_index)
+		return false;
+
+	if (out_msi_pa)
+		*out_msi_pa = local->msi_pa +
+			      (guest_index * IMSIC_MMIO_PAGE_SZ);

Nit: And one more redundant parenthesis and 100 char! ;-)

Björn

maintainer
#46 lore raw

Re: [PATCH v14 12/18] irqchip/riscv-imsic: Add device MSI domain support for platform devices

Quoted 68 lines · Björn Töpel
Quoted 65 lines · Anup Patel

The Linux platform MSI support allows per-device MSI domains so add a platform irqchip driver for RISC-V IMSIC which provides a base IRQ domain with MSI parent support for platform device domains.

The IMSIC platform driver assumes that the IMSIC state is already initialized by the IMSIC early driver.

---
 drivers/irqchip/Makefile                   |   2 +-
 drivers/irqchip/irq-riscv-imsic-platform.c | 344 +++++++++++++++++++++
 drivers/irqchip/irq-riscv-imsic-state.h    |   1 +
 3 files changed, 346 insertions(+), 1 deletion(-)
 create mode 100644 drivers/irqchip/irq-riscv-imsic-platform.c
drivers/irqchip/Makefile +1 -1
index d714724387ce..abca445a3229 100644
@@ -95,7 +95,7 @@ obj-$(CONFIG_QCOM_MPM)                      += irq-qcom-mpm.o
 obj-$(CONFIG_CSKY_MPINTC)            += irq-csky-mpintc.o
 obj-$(CONFIG_CSKY_APB_INTC)          += irq-csky-apb-intc.o
 obj-$(CONFIG_RISCV_INTC)             += irq-riscv-intc.o
-obj-$(CONFIG_RISCV_IMSIC)            += irq-riscv-imsic-state.o irq-riscv-imsic-early.o
+obj-$(CONFIG_RISCV_IMSIC)            += irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o
 obj-$(CONFIG_SIFIVE_PLIC)            += irq-sifive-plic.o
 obj-$(CONFIG_IMX_IRQSTEER)           += irq-imx-irqsteer.o
 obj-$(CONFIG_IMX_INTMUX)             += irq-imx-intmux.o
drivers/irqchip/irq-riscv-imsic-platform.c new file +31 -0
new file mode 100644
index 000000000000..e2344fc08dca
@@ -0,0 +1,344 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 Western Digital Corporation or its affiliates.
+ * Copyright (C) 2022 Ventana Micro Systems Inc.
+ */
+
+#define pr_fmt(fmt) "riscv-imsic: " fmt
+#include <linux/bitmap.h>
+#include <linux/cpu.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/irq.h>
+#include <linux/irqchip.h>
+#include <linux/irqdomain.h>
+#include <linux/module.h>
+#include <linux/msi.h>
+#include <linux/platform_device.h>
+#include <linux/spinlock.h>
+#include <linux/smp.h>
+
+#include "irq-riscv-imsic-state.h"
+
+static bool imsic_cpu_page_phys(unsigned int cpu, unsigned int guest_index,
+                             phys_addr_t *out_msi_pa)
+{
+     struct imsic_global_config *global;
+     struct imsic_local_config *local;
+
+     global = &imsic->global;
+     local = per_cpu_ptr(global->local, cpu);
+
Quoted 8 lines

drivers/irqchip/irq-riscv-imsic-platform.c line 37

Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-platform.c new file +6 -0
+     if (BIT(global->guest_index_bits) <= guest_index)
+             return false;
+
+     if (out_msi_pa)
+             *out_msi_pa = local->msi_pa +
+                           (guest_index * IMSIC_MMIO_PAGE_SZ);

Nit: And one more redundant parenthesis and 100 char! ;-)

Ahh, I missed updating this line. I will update in the next revision.

Regards,
Anup

author
#47 lore raw

PATCH v14 13/18 irqchip/riscv-imsic: Add device MSI domain support for PCI devices

In reply to Anup Patel

The Linux PCI framework supports per-device MSI domains for PCI devices so extend the IMSIC driver to allow PCI per-device MSI domains.

---
 drivers/irqchip/Kconfig                    |  7 +++++
 drivers/irqchip/irq-riscv-imsic-platform.c | 35 ++++++++++++++++++++--
 2 files changed, 40 insertions(+), 2 deletions(-)
drivers/irqchip/Kconfig +7 -0
index 85f86e31c996..2fc0cb32341a 100644
@@ -553,6 +553,13 @@ config RISCV_IMSIC
 	select GENERIC_IRQ_MATRIX_ALLOCATOR
 	select GENERIC_MSI_IRQ
 
+config RISCV_IMSIC_PCI
+	bool
+	depends on RISCV_IMSIC
+	depends on PCI
+	depends on PCI_MSI
+	default RISCV_IMSIC
+
 config EXYNOS_IRQ_COMBINER
 	bool "Samsung Exynos IRQ combiner support" if COMPILE_TEST
 	depends on (ARCH_EXYNOS && ARM) || COMPILE_TEST
drivers/irqchip/irq-riscv-imsic-platform.c +33 -2
index e2344fc08dca..90ddcdd0bba5 100644
@@ -14,6 +14,7 @@
 #include <linux/irqdomain.h>
 #include <linux/module.h>
 #include <linux/msi.h>
+#include <linux/pci.h>
 #include <linux/platform_device.h>
 #include <linux/spinlock.h>
 #include <linux/smp.h>
@@ -208,6 +209,28 @@ static const struct irq_domain_ops imsic_base_domain_ops = {
 #endif
 };
 
+#ifdef CONFIG_RISCV_IMSIC_PCI
+
+static void imsic_pci_mask_irq(struct irq_data *d)
+{
+	pci_msi_mask_irq(d);
+	irq_chip_mask_parent(d);
+}
+
+static void imsic_pci_unmask_irq(struct irq_data *d)
+{
+	irq_chip_unmask_parent(d);
+	pci_msi_unmask_irq(d);
+}
+
+#define MATCH_PCI_MSI		BIT(DOMAIN_BUS_PCI_MSI)
+
+#else
+
+#define MATCH_PCI_MSI		0
+
+#endif
+
 static bool imsic_init_dev_msi_info(struct device *dev,
 				    struct irq_domain *domain,
 				    struct irq_domain *real_parent,
@@ -231,6 +254,13 @@ static bool imsic_init_dev_msi_info(struct device *dev,
 
 	/* Is the target supported? */
 	switch (info->bus_token) {
+#ifdef CONFIG_RISCV_IMSIC_PCI
+	case DOMAIN_BUS_PCI_DEVICE_MSI:
+	case DOMAIN_BUS_PCI_DEVICE_MSIX:
+		info->chip->irq_mask = imsic_pci_mask_irq;
+		info->chip->irq_unmask = imsic_pci_unmask_irq;
+		break;
+#endif
 	case DOMAIN_BUS_DEVICE_MSI:
 		/*
 		 * Per-device MSI should never have any MSI feature bits
@@ -270,11 +300,12 @@ static bool imsic_init_dev_msi_info(struct device *dev,
 #define MATCH_PLATFORM_MSI		BIT(DOMAIN_BUS_PLATFORM_MSI)
 
 static const struct msi_parent_ops imsic_msi_parent_ops = {
-	.supported_flags	= MSI_GENERIC_FLAGS_MASK,
+	.supported_flags	= MSI_GENERIC_FLAGS_MASK |
+				  MSI_FLAG_PCI_MSIX,
 	.required_flags		= MSI_FLAG_USE_DEF_DOM_OPS |
 				  MSI_FLAG_USE_DEF_CHIP_OPS,
 	.bus_select_token	= DOMAIN_BUS_NEXUS,
-	.bus_select_mask	= MATCH_PLATFORM_MSI,
+	.bus_select_mask	= MATCH_PCI_MSI | MATCH_PLATFORM_MSI,
 	.init_dev_msi_info	= imsic_init_dev_msi_info,
 };
 
Signature · 1 line
2.34.1
#48 lore raw

Re: [PATCH v14 13/18] irqchip/riscv-imsic: Add device MSI domain support for PCI devices

Quoted 69 lines · Anup Patel

The Linux PCI framework supports per-device MSI domains for PCI devices so extend the IMSIC driver to allow PCI per-device MSI domains.

---
 drivers/irqchip/Kconfig                    |  7 +++++
 drivers/irqchip/irq-riscv-imsic-platform.c | 35 ++++++++++++++++++++--
 2 files changed, 40 insertions(+), 2 deletions(-)
drivers/irqchip/Kconfig +7 -0
index 85f86e31c996..2fc0cb32341a 100644
@@ -553,6 +553,13 @@ config RISCV_IMSIC
 	select GENERIC_IRQ_MATRIX_ALLOCATOR
 	select GENERIC_MSI_IRQ
 
+config RISCV_IMSIC_PCI
+	bool
+	depends on RISCV_IMSIC
+	depends on PCI
+	depends on PCI_MSI
+	default RISCV_IMSIC
+
 config EXYNOS_IRQ_COMBINER
 	bool "Samsung Exynos IRQ combiner support" if COMPILE_TEST
 	depends on (ARCH_EXYNOS && ARM) || COMPILE_TEST
drivers/irqchip/irq-riscv-imsic-platform.c +23 -0
index e2344fc08dca..90ddcdd0bba5 100644
@@ -14,6 +14,7 @@
 #include <linux/irqdomain.h>
 #include <linux/module.h>
 #include <linux/msi.h>
+#include <linux/pci.h>
 #include <linux/platform_device.h>
 #include <linux/spinlock.h>
 #include <linux/smp.h>
@@ -208,6 +209,28 @@ static const struct irq_domain_ops imsic_base_domain_ops = {
 #endif
 };
 
+#ifdef CONFIG_RISCV_IMSIC_PCI
+
+static void imsic_pci_mask_irq(struct irq_data *d)
+{
+	pci_msi_mask_irq(d);
+	irq_chip_mask_parent(d);
+}
+
+static void imsic_pci_unmask_irq(struct irq_data *d)
+{
+	irq_chip_unmask_parent(d);
+	pci_msi_unmask_irq(d);
+}
+
+#define MATCH_PCI_MSI		BIT(DOMAIN_BUS_PCI_MSI)
+
+#else
+
+#define MATCH_PCI_MSI		0
+
+#endif
+
 static bool imsic_init_dev_msi_info(struct device *dev,
 				    struct irq_domain *domain,
 				    struct irq_domain *real_parent,

drivers/irqchip/irq-riscv-imsic-platform.c static bool imsic_init_dev_msi_info(struct device *dev, · line 261

Quoted 8 lines
drivers/irqchip/irq-riscv-imsic-platform.c +5 -0
@@ -231,6 +254,13 @@ static bool imsic_init_dev_msi_info(struct device *dev,
 
 	/* Is the target supported? */
 	switch (info->bus_token) {
+#ifdef CONFIG_RISCV_IMSIC_PCI
+	case DOMAIN_BUS_PCI_DEVICE_MSI:
+	case DOMAIN_BUS_PCI_DEVICE_MSIX:
+		info->chip->irq_mask = imsic_pci_mask_irq;
+		info->chip->irq_unmask = imsic_pci_unmask_irq;

irq_set_affinity()?

Björn

maintainer
#49 lore raw

Re: [PATCH v14 13/18] irqchip/riscv-imsic: Add device MSI domain support for PCI devices

Quoted 73 lines · Björn Töpel
Quoted 70 lines · Anup Patel

The Linux PCI framework supports per-device MSI domains for PCI devices so extend the IMSIC driver to allow PCI per-device MSI domains.

---
 drivers/irqchip/Kconfig                    |  7 +++++
 drivers/irqchip/irq-riscv-imsic-platform.c | 35 ++++++++++++++++++++--
 2 files changed, 40 insertions(+), 2 deletions(-)
drivers/irqchip/Kconfig +7 -0
index 85f86e31c996..2fc0cb32341a 100644
@@ -553,6 +553,13 @@ config RISCV_IMSIC
      select GENERIC_IRQ_MATRIX_ALLOCATOR
      select GENERIC_MSI_IRQ
 
+config RISCV_IMSIC_PCI
+     bool
+     depends on RISCV_IMSIC
+     depends on PCI
+     depends on PCI_MSI
+     default RISCV_IMSIC
+
 config EXYNOS_IRQ_COMBINER
      bool "Samsung Exynos IRQ combiner support" if COMPILE_TEST
      depends on (ARCH_EXYNOS && ARM) || COMPILE_TEST
drivers/irqchip/irq-riscv-imsic-platform.c +23 -0
index e2344fc08dca..90ddcdd0bba5 100644
@@ -14,6 +14,7 @@
 #include <linux/irqdomain.h>
 #include <linux/module.h>
 #include <linux/msi.h>
+#include <linux/pci.h>
 #include <linux/platform_device.h>
 #include <linux/spinlock.h>
 #include <linux/smp.h>
@@ -208,6 +209,28 @@ static const struct irq_domain_ops imsic_base_domain_ops = {
 #endif
 };
 
+#ifdef CONFIG_RISCV_IMSIC_PCI
+
+static void imsic_pci_mask_irq(struct irq_data *d)
+{
+     pci_msi_mask_irq(d);
+     irq_chip_mask_parent(d);
+}
+
+static void imsic_pci_unmask_irq(struct irq_data *d)
+{
+     irq_chip_unmask_parent(d);
+     pci_msi_unmask_irq(d);
+}
+
+#define MATCH_PCI_MSI                BIT(DOMAIN_BUS_PCI_MSI)
+
+#else
+
+#define MATCH_PCI_MSI                0
+
+#endif
+
 static bool imsic_init_dev_msi_info(struct device *dev,
                                  struct irq_domain *domain,
                                  struct irq_domain *real_parent,
@@ -231,6 +254,13 @@ static bool imsic_init_dev_msi_info(struct device *dev,
 
Quoted 9 lines

drivers/irqchip/irq-riscv-imsic-platform.c static bool imsic_init_dev_msi_info(struct device *dev, · line 261

Quoted 7 lines
drivers/irqchip/irq-riscv-imsic-platform.c +5 -0
      /* Is the target supported? */
      switch (info->bus_token) {
+#ifdef CONFIG_RISCV_IMSIC_PCI
+     case DOMAIN_BUS_PCI_DEVICE_MSI:
+     case DOMAIN_BUS_PCI_DEVICE_MSIX:
+             info->chip->irq_mask = imsic_pci_mask_irq;
+             info->chip->irq_unmask = imsic_pci_unmask_irq;

irq_set_affinity()?

It's already set by the switch-case above.

Regards,
Anup

#50 lore raw

Re: [PATCH v14 13/18] irqchip/riscv-imsic: Add device MSI domain support for PCI devices

Quoted 84 lines · Anup Patel
Quoted 73 lines · Björn Töpel
Quoted 70 lines · Anup Patel

The Linux PCI framework supports per-device MSI domains for PCI devices so extend the IMSIC driver to allow PCI per-device MSI domains.

---
 drivers/irqchip/Kconfig                    |  7 +++++
 drivers/irqchip/irq-riscv-imsic-platform.c | 35 ++++++++++++++++++++--
 2 files changed, 40 insertions(+), 2 deletions(-)
drivers/irqchip/Kconfig +7 -0
index 85f86e31c996..2fc0cb32341a 100644
@@ -553,6 +553,13 @@ config RISCV_IMSIC
      select GENERIC_IRQ_MATRIX_ALLOCATOR
      select GENERIC_MSI_IRQ
 
+config RISCV_IMSIC_PCI
+     bool
+     depends on RISCV_IMSIC
+     depends on PCI
+     depends on PCI_MSI
+     default RISCV_IMSIC
+
 config EXYNOS_IRQ_COMBINER
      bool "Samsung Exynos IRQ combiner support" if COMPILE_TEST
      depends on (ARCH_EXYNOS && ARM) || COMPILE_TEST
drivers/irqchip/irq-riscv-imsic-platform.c +23 -0
index e2344fc08dca..90ddcdd0bba5 100644
@@ -14,6 +14,7 @@
 #include <linux/irqdomain.h>
 #include <linux/module.h>
 #include <linux/msi.h>
+#include <linux/pci.h>
 #include <linux/platform_device.h>
 #include <linux/spinlock.h>
 #include <linux/smp.h>
@@ -208,6 +209,28 @@ static const struct irq_domain_ops imsic_base_domain_ops = {
 #endif
 };
 
+#ifdef CONFIG_RISCV_IMSIC_PCI
+
+static void imsic_pci_mask_irq(struct irq_data *d)
+{
+     pci_msi_mask_irq(d);
+     irq_chip_mask_parent(d);
+}
+
+static void imsic_pci_unmask_irq(struct irq_data *d)
+{
+     irq_chip_unmask_parent(d);
+     pci_msi_unmask_irq(d);
+}
+
+#define MATCH_PCI_MSI                BIT(DOMAIN_BUS_PCI_MSI)
+
+#else
+
+#define MATCH_PCI_MSI                0
+
+#endif
+
 static bool imsic_init_dev_msi_info(struct device *dev,
                                  struct irq_domain *domain,
                                  struct irq_domain *real_parent,
@@ -231,6 +254,13 @@ static bool imsic_init_dev_msi_info(struct device *dev,
 
Quoted 9 lines

drivers/irqchip/irq-riscv-imsic-platform.c static bool imsic_init_dev_msi_info(struct device *dev, · line 261

Quoted 7 lines
drivers/irqchip/irq-riscv-imsic-platform.c +5 -0
      /* Is the target supported? */
      switch (info->bus_token) {
+#ifdef CONFIG_RISCV_IMSIC_PCI
+     case DOMAIN_BUS_PCI_DEVICE_MSI:
+     case DOMAIN_BUS_PCI_DEVICE_MSIX:
+             info->chip->irq_mask = imsic_pci_mask_irq;
+             info->chip->irq_unmask = imsic_pci_unmask_irq;

irq_set_affinity()?

Quoted 1 line

It's already set by the switch-case above.

Ah, right -- the new .bus_select_token! Thank you!

author
#51 lore raw

PATCH v14 14/18 dt-bindings: interrupt-controller: Add RISC-V advanced PLIC

In reply to Anup Patel

Add DT bindings document for RISC-V advanced platform level interrupt controller (APLIC) defined by the RISC-V advanced interrupt architecture (AIA) specification.

---
 .../interrupt-controller/riscv,aplic.yaml     | 172 ++++++++++++++++++
 1 file changed, 172 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml
Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml new file +172 -0
new file mode 100644
index 000000000000..190a6499c932
@@ -0,0 +1,172 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/interrupt-controller/riscv,aplic.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: RISC-V Advanced Platform Level Interrupt Controller (APLIC)
+
+maintainers:
+  - Anup Patel <[email protected]>
+
+description:
+  The RISC-V advanced interrupt architecture (AIA) defines an advanced
+  platform level interrupt controller (APLIC) for handling wired interrupts
+  in a RISC-V platform. The RISC-V AIA specification can be found at
+  https://github.com/riscv/riscv-aia.
+
+  The RISC-V APLIC is implemented as hierarchical APLIC domains where all
+  interrupt sources connect to the root APLIC domain and a parent APLIC
+  domain can delegate interrupt sources to it's child APLIC domains. There
+  is one device tree node for each APLIC domain.
+
+allOf:
+  - $ref: /schemas/interrupt-controller.yaml#
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - qemu,aplic
+      - const: riscv,aplic
+
+  reg:
+    maxItems: 1
+
+  interrupt-controller: true
+
+  "#interrupt-cells":
+    const: 2
+
+  interrupts-extended:
+    minItems: 1
+    maxItems: 16384
+    description:
+      Given APLIC domain directly injects external interrupts to a set of
+      RISC-V HARTS (or CPUs). Each node pointed to should be a riscv,cpu-intc
+      node, which has a CPU node (i.e. RISC-V HART) as parent.
+
+  msi-parent:
+    description:
+      Given APLIC domain forwards wired interrupts as MSIs to a AIA incoming
+      message signaled interrupt controller (IMSIC). If both "msi-parent" and
+      "interrupts-extended" properties are present then it means the APLIC
+      domain supports both MSI mode and Direct mode in HW. In this case, the
+      APLIC driver has to choose between MSI mode or Direct mode.
+
+  riscv,num-sources:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    minimum: 1
+    maximum: 1023
+    description:
+      Specifies the number of wired interrupt sources supported by this
+      APLIC domain.
+
+  riscv,children:
+    $ref: /schemas/types.yaml#/definitions/phandle-array
+    minItems: 1
+    maxItems: 1024
+    items:
+      maxItems: 1
+    description:
+      A list of child APLIC domains for the given APLIC domain. Each child
+      APLIC domain is assigned a child index in increasing order, with the
+      first child APLIC domain assigned child index 0. The APLIC domain child
+      index is used by firmware to delegate interrupts from the given APLIC
+      domain to a particular child APLIC domain.
+
+  riscv,delegation:
+    $ref: /schemas/types.yaml#/definitions/phandle-array
+    minItems: 1
+    maxItems: 1024
+    items:
+      items:
+        - description: child APLIC domain phandle
+        - description: first interrupt number of the parent APLIC domain (inclusive)
+        - description: last interrupt number of the parent APLIC domain (inclusive)
+    description:
+      A interrupt delegation list where each entry is a triple consisting
+      of child APLIC domain phandle, first interrupt number of the parent
+      APLIC domain, and last interrupt number of the parent APLIC domain.
+      Firmware must configure interrupt delegation registers based on
+      interrupt delegation list.
+
+dependencies:
+  riscv,delegation: [ "riscv,children" ]
+
+required:
+  - compatible
+  - reg
+  - interrupt-controller
+  - "#interrupt-cells"
+  - riscv,num-sources
+
+anyOf:
+  - required:
+      - interrupts-extended
+  - required:
+      - msi-parent
+
+unevaluatedProperties: false
+
+examples:
+  - |
+    // Example 1 (APLIC domains directly injecting interrupt to HARTs):
+
+    interrupt-controller@c000000 {
+      compatible = "qemu,aplic", "riscv,aplic";
+      interrupts-extended = <&cpu1_intc 11>,
+                            <&cpu2_intc 11>,
+                            <&cpu3_intc 11>,
+                            <&cpu4_intc 11>;
+      reg = <0xc000000 0x4080>;
+      interrupt-controller;
+      #interrupt-cells = <2>;
+      riscv,num-sources = <63>;
+      riscv,children = <&aplic1>, <&aplic2>;
+      riscv,delegation = <&aplic1 1 63>;
+    };
+
+    aplic1: interrupt-controller@d000000 {
+      compatible = "qemu,aplic", "riscv,aplic";
+      interrupts-extended = <&cpu1_intc 9>,
+                            <&cpu2_intc 9>;
+      reg = <0xd000000 0x4080>;
+      interrupt-controller;
+      #interrupt-cells = <2>;
+      riscv,num-sources = <63>;
+    };
+
+    aplic2: interrupt-controller@e000000 {
+      compatible = "qemu,aplic", "riscv,aplic";
+      interrupts-extended = <&cpu3_intc 9>,
+                            <&cpu4_intc 9>;
+      reg = <0xe000000 0x4080>;
+      interrupt-controller;
+      #interrupt-cells = <2>;
+      riscv,num-sources = <63>;
+    };
+
+  - |
+    // Example 2 (APLIC domains forwarding interrupts as MSIs):
+
+    interrupt-controller@c000000 {
+      compatible = "qemu,aplic", "riscv,aplic";
+      msi-parent = <&imsic_mlevel>;
+      reg = <0xc000000 0x4000>;
+      interrupt-controller;
+      #interrupt-cells = <2>;
+      riscv,num-sources = <63>;
+      riscv,children = <&aplic3>;
+      riscv,delegation = <&aplic3 1 63>;
+    };
+
+    aplic3: interrupt-controller@d000000 {
+      compatible = "qemu,aplic", "riscv,aplic";
+      msi-parent = <&imsic_slevel>;
+      reg = <0xd000000 0x4000>;
+      interrupt-controller;
+      #interrupt-cells = <2>;
+      riscv,num-sources = <63>;
+    };
+...
Signature · 1 line
2.34.1
author
#52 lore raw

PATCH v14 15/18 irqchip: Add RISC-V advanced PLIC driver for direct-mode

In reply to Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines advanced platform-level interrupt controller (APLIC) which has two modes of operation: 1) Direct mode and 2) MSI mode.
(For more details, refer https://github.com/riscv/riscv-aia)

In APLIC direct-mode, wired interrupts are forwared to CPUs (or HARTs) as a local external interrupt.

Add a platform irqchip driver for the RISC-V APLIC direct-mode to support RISC-V platforms having only wired interrupts.

---
 drivers/irqchip/Kconfig                  |   5 +
 drivers/irqchip/Makefile                 |   1 +
 drivers/irqchip/irq-riscv-aplic-direct.c | 326 +++++++++++++++++++++++
 drivers/irqchip/irq-riscv-aplic-main.c   | 211 +++++++++++++++
 drivers/irqchip/irq-riscv-aplic-main.h   |  44 +++
 include/linux/irqchip/riscv-aplic.h      | 145 ++++++++++
 6 files changed, 732 insertions(+)
 create mode 100644 drivers/irqchip/irq-riscv-aplic-direct.c
 create mode 100644 drivers/irqchip/irq-riscv-aplic-main.c
 create mode 100644 drivers/irqchip/irq-riscv-aplic-main.h
 create mode 100644 include/linux/irqchip/riscv-aplic.h
drivers/irqchip/Kconfig +5 -0
index 2fc0cb32341a..dbc8811d3764 100644
@@ -546,6 +546,11 @@ config SIFIVE_PLIC
 	select IRQ_DOMAIN_HIERARCHY
 	select GENERIC_IRQ_EFFECTIVE_AFF_MASK if SMP
 
+config RISCV_APLIC
+	bool
+	depends on RISCV
+	select IRQ_DOMAIN_HIERARCHY
+
 config RISCV_IMSIC
 	bool
 	depends on RISCV
drivers/irqchip/Makefile +1 -0
index abca445a3229..7f8289790ed8 100644
@@ -95,6 +95,7 @@ obj-$(CONFIG_QCOM_MPM)			+= irq-qcom-mpm.o
 obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
 obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
 obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
+obj-$(CONFIG_RISCV_APLIC)		+= irq-riscv-aplic-main.o irq-riscv-aplic-direct.o
 obj-$(CONFIG_RISCV_IMSIC)		+= irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o
 obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
 obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
drivers/irqchip/irq-riscv-aplic-direct.c new file +326 -0

326 lines not on this page · raw message

drivers/irqchip/irq-riscv-aplic-main.c new file +211 -0

211 lines not on this page · raw message

drivers/irqchip/irq-riscv-aplic-main.h new file +44 -0

44 lines not on this page · raw message

include/linux/irqchip/riscv-aplic.h new file +145 -0

145 lines not on this page · raw message

Signature · 1 line
2.34.1
author
#53 lore raw

PATCH v14 16/18 irqchip/riscv-aplic: Add support for MSI-mode

In reply to Anup Patel

The RISC-V advanced platform-level interrupt controller (APLIC) has two modes of operation: 1) Direct mode and 2) MSI mode. (For more details, refer https://github.com/riscv/riscv-aia)

In APLIC MSI-mode, wired interrupts are forwared as message signaled interrupts (MSIs) to CPUs via IMSIC.

Extend the existing APLIC irqchip driver to support MSI-mode for RISC-V platforms having both wired interrupts and MSIs.

---
 drivers/irqchip/Kconfig                |   6 +
 drivers/irqchip/Makefile               |   1 +
 drivers/irqchip/irq-riscv-aplic-main.c |   2 +-
 drivers/irqchip/irq-riscv-aplic-main.h |   8 +
 drivers/irqchip/irq-riscv-aplic-msi.c  | 263 +++++++++++++++++++++++++
 5 files changed, 279 insertions(+), 1 deletion(-)
 create mode 100644 drivers/irqchip/irq-riscv-aplic-msi.c
drivers/irqchip/Kconfig +6 -0
index dbc8811d3764..806b5fccb3e8 100644
@@ -551,6 +551,12 @@ config RISCV_APLIC
 	depends on RISCV
 	select IRQ_DOMAIN_HIERARCHY
 
+config RISCV_APLIC_MSI
+	bool
+	depends on RISCV_APLIC
+	select GENERIC_MSI_IRQ
+	default RISCV_APLIC
+
 config RISCV_IMSIC
 	bool
 	depends on RISCV
drivers/irqchip/Makefile +1 -0
index 7f8289790ed8..47995fdb2c60 100644
@@ -96,6 +96,7 @@ obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
 obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
 obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
 obj-$(CONFIG_RISCV_APLIC)		+= irq-riscv-aplic-main.o irq-riscv-aplic-direct.o
+obj-$(CONFIG_RISCV_APLIC_MSI)		+= irq-riscv-aplic-msi.o
 obj-$(CONFIG_RISCV_IMSIC)		+= irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o
 obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
 obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
drivers/irqchip/irq-riscv-aplic-main.c +1 -1
index 160ff99d6979..774a0c97fdab 100644
@@ -187,7 +187,7 @@ static int aplic_probe(struct platform_device *pdev)
 	if (is_of_node(dev->fwnode))
 		msi_mode = of_property_present(to_of_node(dev->fwnode), "msi-parent");
 	if (msi_mode)
-		rc = -ENODEV;
+		rc = aplic_msi_setup(dev, regs);
 	else
 		rc = aplic_direct_setup(dev, regs);
 	if (rc)
drivers/irqchip/irq-riscv-aplic-main.h +8 -0
index 4cfbadf37ddc..4393927d8c80 100644
@@ -40,5 +40,13 @@ int aplic_irqdomain_translate(struct irq_fwspec *fwspec, u32 gsi_base,
 void aplic_init_hw_global(struct aplic_priv *priv, bool msi_mode);
 int aplic_setup_priv(struct aplic_priv *priv, struct device *dev, void __iomem *regs);
 int aplic_direct_setup(struct device *dev, void __iomem *regs);
+#ifdef CONFIG_RISCV_APLIC_MSI
+int aplic_msi_setup(struct device *dev, void __iomem *regs);
+#else
+static inline int aplic_msi_setup(struct device *dev, void __iomem *regs)
+{
+	return -ENODEV;
+}
+#endif
 
 #endif
drivers/irqchip/irq-riscv-aplic-msi.c new file +263 -0

263 lines not on this page · raw message

Signature · 1 line
2.34.1
author
#54 lore raw

PATCH v14 17/18 RISC-V: Select APLIC and IMSIC drivers

In reply to Anup Patel

The QEMU virt machine supports AIA emulation and quite a few RISC-V platforms with AIA support are under development so select APLIC and IMSIC drivers for all RISC-V platforms.

---
 arch/riscv/Kconfig | 2 ++
 1 file changed, 2 insertions(+)
arch/riscv/Kconfig +2 -0
index bffbd869a068..569f2b6fd60a 100644
@@ -162,6 +162,8 @@ config RISCV
 	select PCI_DOMAINS_GENERIC if PCI
 	select PCI_MSI if PCI
 	select RISCV_ALTERNATIVE if !XIP_KERNEL
+	select RISCV_APLIC
+	select RISCV_IMSIC
 	select RISCV_INTC
 	select RISCV_TIMER if RISCV_SBI
 	select SIFIVE_PLIC
Signature · 1 line
2.34.1
author
#55 lore raw

PATCH v14 18/18 MAINTAINERS: Add entry for RISC-V AIA drivers

In reply to Anup Patel

Add myself as maintainer for RISC-V AIA drivers including the RISC-V INTC driver which supports both AIA and non-AIA platforms.

---
 MAINTAINERS | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
MAINTAINERS +14 -0
index 9ed4d3868539..d948f9210f1b 100644
@@ -18801,6 +18801,20 @@ S:	Maintained
 F:	drivers/mtd/nand/raw/r852.c
 F:	drivers/mtd/nand/raw/r852.h
 
+RISC-V AIA DRIVERS
+M:	Anup Patel <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml
+F:	Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml
+F:	drivers/irqchip/irq-riscv-aplic-*.c
+F:	drivers/irqchip/irq-riscv-aplic-*.h
+F:	drivers/irqchip/irq-riscv-imsic-*.c
+F:	drivers/irqchip/irq-riscv-imsic-*.h
+F:	drivers/irqchip/irq-riscv-intc.c
+F:	include/linux/irqchip/riscv-aplic.h
+F:	include/linux/irqchip/riscv-imsic.h
+
 RISC-V ARCHITECTURE
 M:	Paul Walmsley <[email protected]>
 M:	Palmer Dabbelt <[email protected]>
Signature · 1 line
2.34.1