Skip to contentSkip to conversation

Linux RISC-V AIA Support

PATCH Superseded Anup Patel 5 participants 23 messages Jul 10, 2023 – Jul 17, 2023

Subsystems (4)

v5 of 16 Superseded Latest is v16 →

lore
Share & tools
lore

Anup Patel

author
#0 lore raw

v5 cover Linux RISC-V AIA Support

The RISC-V AIA specification is now frozen as-per the RISC-V international process. The latest frozen specifcation can be found at:
https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interrupts-1.0.pdf

At a high-level, the AIA specification adds three things:
1) AIA CSRs
- Improved local interrupt support
2) Incoming Message Signaled Interrupt Controller (IMSIC)
- Per-HART MSI controller
- Support MSI virtualization
- Support IPI along with virtualization
3) Advanced Platform-Level Interrupt Controller (APLIC)
- Wired interrupt controller
- In MSI-mode, converts wired interrupt into MSIs (i.e. MSI generator)
- In Direct-mode, injects external interrupts directly into HARTs

For an overview of the AIA specification, refer the AIA virtualization talk at KVM Forum 2022:
https://static.sched.com/hosted_files/kvmforum2022/a1/AIA_Virtualization_in_KVM_RISCV_final.pdf
https://www.youtube.com/watch?v=r071dL8Z0yo

To test this series, use QEMU v7.2 (or higher) and OpenSBI v1.2 (or higher).

These patches can also be found in the riscv_aia_v5 branch at:
https://github.com/avpatel/linux.git

Changes since v4:
- Rebased on Linux-6.5-rc1
- Added "Dependencies" in the APLIC bindings (PATCH6 in v4)
- Dropped the PATCH6 which was changing the IOMMU DMA domain APIs
- Dropped use of IOMMU DMA APIs in the IMSIC driver (PATCH4)

Changes since v3:
- Rebased on Linux-6.4-rc6
- Droped PATCH2 of v3 series instead we now set FWNODE_FLAG_BEST_EFFORT via IRQCHIP_DECLARE()
- Extend riscv_fw_parent_hartid() to support both DT and ACPI in PATCH1
- Extend iommu_dma_compose_msi_msg() instead of adding iommu_dma_select_msi() in PATCH6
- Addressed Conor's comments in PATCH3
- Addressed Conor's and Rob's comments in PATCH7

Changes since v2:
- Rebased on Linux-6.4-rc1
- Addressed Rob's comments on DT bindings patches 4 and 8.
- Addessed Marc's comments on IMSIC driver PATCH5
- Replaced use of OF apis in APLIC and IMSIC drivers with FWNODE apis this makes both drivers easily portable for ACPI support. This also removes unnecessary indirection from the APLIC and IMSIC drivers.
- PATCH1 is a new patch for portability with ACPI support
- PATCH2 is a new patch to fix probing in APLIC drivers for APLIC-only systems.
- PATCH7 is a new patch which addresses the IOMMU DMA domain issues pointed out by SiFive

Changes since v1:
- Rebased on Linux-6.2-rc2
- Addressed comments on IMSIC DT bindings for PATCH4
- Use raw_spin_lock_irqsave() on ids_lock for PATCH5
- Improved MMIO alignment checks in PATCH5 to allow MMIO regions with holes.
- Addressed comments on APLIC DT bindings for PATCH6
- Fixed warning splat in aplic_msi_write_msg() caused by zeroed MSI message in PATCH7
- Dropped DT property riscv,slow-ipi instead will have module parameter in future.

Anup Patel (9):
RISC-V: Add riscv_fw_parent_hartid() function irqchip/riscv-intc: Add support for RISC-V AIA
dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller
irqchip: Add RISC-V incoming MSI controller driver irqchip/riscv-imsic: Add support for PCI MSI irqdomain
dt-bindings: interrupt-controller: Add RISC-V advanced PLIC
irqchip: Add RISC-V advanced PLIC driver
RISC-V: Select APLIC and IMSIC drivers
MAINTAINERS: Add entry for RISC-V AIA drivers

 .../interrupt-controller/riscv,aplic.yaml     |  172 +++
 .../interrupt-controller/riscv,imsics.yaml    |  172 +++
 MAINTAINERS                                   |   12 +
 arch/riscv/Kconfig                            |    2 +
 arch/riscv/include/asm/processor.h            |    3 +
 arch/riscv/kernel/cpu.c                       |   16 +
 drivers/irqchip/Kconfig                       |   20 +-
 drivers/irqchip/Makefile                      |    2 +
 drivers/irqchip/irq-riscv-aplic.c             |  774 ++++++++++++
 drivers/irqchip/irq-riscv-imsic.c             | 1060 +++++++++++++++++
 drivers/irqchip/irq-riscv-intc.c              |   36 +-
 include/linux/irqchip/riscv-aplic.h           |  119 ++
 include/linux/irqchip/riscv-imsic.h           |   86 ++
 13 files changed, 2467 insertions(+), 7 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml
 create mode 100644 drivers/irqchip/irq-riscv-aplic.c
 create mode 100644 drivers/irqchip/irq-riscv-imsic.c
 create mode 100644 include/linux/irqchip/riscv-aplic.h
 create mode 100644 include/linux/irqchip/riscv-imsic.h
Signature · 7 lines
2.34.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#1 lore raw

PATCH v5 1/9 RISC-V: Add riscv_fw_parent_hartid() function

We add common riscv_fw_parent_hartid() which help device drivers to get parent hartid of the INTC (i.e. local interrupt controller) fwnode. This should work for both DT and ACPI.

---
 arch/riscv/include/asm/processor.h |  3 +++
 arch/riscv/kernel/cpu.c            | 16 ++++++++++++++++
 2 files changed, 19 insertions(+)
arch/riscv/include/asm/processor.h +3 -0
index c950a8d9edef..39dc23a18f88 100644
@@ -81,6 +81,9 @@ int riscv_of_processor_hartid(struct device_node *node, unsigned long *hartid);
 int riscv_early_of_processor_hartid(struct device_node *node, unsigned long *hartid);
 int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid);
 
+struct fwnode_handle;
+int riscv_fw_parent_hartid(struct fwnode_handle *node, unsigned long *hartid);
+
 extern void riscv_fill_hwcap(void);
 extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src);
 
arch/riscv/kernel/cpu.c +16 -0
index a2fc952318e9..9be9b3b1f333 100644
@@ -96,6 +96,22 @@ int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid)
 	return -1;
 }
 
+/* Find hart ID of the CPU fwnode under which given fwnode falls. */
+int riscv_fw_parent_hartid(struct fwnode_handle *node, unsigned long *hartid)
+{
+	int rc;
+	u64 temp;
+
+	if (!is_of_node(node)) {
+		rc = fwnode_property_read_u64_array(node, "hartid", &temp, 1);
+		if (!rc)
+			*hartid = temp;
+	} else
+		rc = riscv_of_parent_hartid(to_of_node(node), hartid);
+
+	return rc;
+}
+
 DEFINE_PER_CPU(struct riscv_cpuinfo, riscv_cpuinfo);
 
 unsigned long riscv_cached_mvendorid(unsigned int cpu_id)
Signature · 7 lines
2.34.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#4 lore raw

PATCH v5 2/9 irqchip/riscv-intc: Add support for RISC-V AIA

In reply to Anup Patel

The RISC-V advanced interrupt architecture (AIA) extends the per-HART local interrupts in following ways:
1. Minimum 64 local interrupts for both RV32 and RV64
2. Ability to process multiple pending local interrupts in same interrupt handler
3. Priority configuration for each local interrupts
4. Special CSRs to configure/access the per-HART MSI controller

This patch adds support for RISC-V AIA in the RISC-V intc driver.

---
 drivers/irqchip/irq-riscv-intc.c | 36 ++++++++++++++++++++++++++------
 1 file changed, 30 insertions(+), 6 deletions(-)
drivers/irqchip/irq-riscv-intc.c +30 -6
index 4adeee1bc391..e235bf1708a4 100644
@@ -17,6 +17,7 @@
 #include <linux/module.h>
 #include <linux/of.h>
 #include <linux/smp.h>
+#include <asm/hwcap.h>
 
 static struct irq_domain *intc_domain;
 
@@ -30,6 +31,15 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 	generic_handle_domain_irq(intc_domain, cause);
 }
 
+static asmlinkage void riscv_intc_aia_irq(struct pt_regs *regs)
+{
+	unsigned long topi;
+
+	while ((topi = csr_read(CSR_TOPI)))
+		generic_handle_domain_irq(intc_domain,
+					  topi >> TOPI_IID_SHIFT);
+}
+
 /*
  * On RISC-V systems local interrupts are masked or unmasked by writing
  * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be written
@@ -39,12 +49,18 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 
 static void riscv_intc_irq_mask(struct irq_data *d)
 {
-	csr_clear(CSR_IE, BIT(d->hwirq));
+	if (d->hwirq < BITS_PER_LONG)
+		csr_clear(CSR_IE, BIT(d->hwirq));
+	else
+		csr_clear(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));
 }
 
 static void riscv_intc_irq_unmask(struct irq_data *d)
 {
-	csr_set(CSR_IE, BIT(d->hwirq));
+	if (d->hwirq < BITS_PER_LONG)
+		csr_set(CSR_IE, BIT(d->hwirq));
+	else
+		csr_set(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));
 }
 
 static void riscv_intc_irq_eoi(struct irq_data *d)
@@ -115,16 +131,22 @@ static struct fwnode_handle *riscv_intc_hwnode(void)
 
 static int __init riscv_intc_init_common(struct fwnode_handle *fn)
 {
-	int rc;
+	int rc, nr_irqs = BITS_PER_LONG;
+
+	if (riscv_isa_extension_available(NULL, SxAIA) && BITS_PER_LONG == 32)
+		nr_irqs = nr_irqs * 2;
 
-	intc_domain = irq_domain_create_linear(fn, BITS_PER_LONG,
+	intc_domain = irq_domain_create_linear(fn, nr_irqs,
 					       &riscv_intc_domain_ops, NULL);
 	if (!intc_domain) {
 		pr_err("unable to add IRQ domain\n");
 		return -ENXIO;
 	}
 
-	rc = set_handle_irq(&riscv_intc_irq);
+	if (riscv_isa_extension_available(NULL, SxAIA))
+		rc = set_handle_irq(&riscv_intc_aia_irq);
+	else
+		rc = set_handle_irq(&riscv_intc_irq);
 	if (rc) {
 		pr_err("failed to set irq handler\n");
 		return rc;
@@ -132,7 +154,9 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
 
 	riscv_set_intc_hwnode_fn(riscv_intc_hwnode);
 
-	pr_info("%d local interrupts mapped\n", BITS_PER_LONG);
+	pr_info("%d local interrupts mapped%s\n",
+		nr_irqs, (riscv_isa_extension_available(NULL, SxAIA)) ?
+			 " using AIA" : "");
 
 	return 0;
 }
Signature · 7 lines
2.34.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#7 lore raw

PATCH v5 3/9 dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller

In reply to Anup Patel

We add DT bindings document for the RISC-V incoming MSI controller (IMSIC) defined by the RISC-V advanced interrupt architecture (AIA) specification.

---
 .../interrupt-controller/riscv,imsics.yaml    | 172 ++++++++++++++++++
 1 file changed, 172 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml
Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml new file +172 -0
new file mode 100644
index 000000000000..84976f17a4a1
@@ -0,0 +1,172 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/interrupt-controller/riscv,imsics.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: RISC-V Incoming MSI Controller (IMSIC)
+
+maintainers:
+  - Anup Patel <[email protected]>
+
+description: |
+  The RISC-V advanced interrupt architecture (AIA) defines a per-CPU incoming
+  MSI controller (IMSIC) for handling MSIs in a RISC-V platform. The RISC-V
+  AIA specification can be found at https://github.com/riscv/riscv-aia.
+
+  The IMSIC is a per-CPU (or per-HART) device with separate interrupt file
+  for each privilege level (machine or supervisor). The configuration of
+  a IMSIC interrupt file is done using AIA CSRs and it also has a 4KB MMIO
+  space to receive MSIs from devices. Each IMSIC interrupt file supports a
+  fixed number of interrupt identities (to distinguish MSIs from devices)
+  which is same for given privilege level across CPUs (or HARTs).
+
+  The device tree of a RISC-V platform will have one IMSIC device tree node
+  for each privilege level (machine or supervisor) which collectively describe
+  IMSIC interrupt files at that privilege level across CPUs (or HARTs).
+
+  The arrangement of IMSIC interrupt files in MMIO space of a RISC-V platform
+  follows a particular scheme defined by the RISC-V AIA specification. A IMSIC
+  group is a set of IMSIC interrupt files co-located in MMIO space and we can
+  have multiple IMSIC groups (i.e. clusters, sockets, chiplets, etc) in a
+  RISC-V platform. The MSI target address of a IMSIC interrupt file at given
+  privilege level (machine or supervisor) encodes group index, HART index,
+  and guest index (shown below).
+
+  XLEN-1            > (HART Index MSB)                  12    0
+  |                  |                                  |     |
+  -------------------------------------------------------------
+  |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index|  0  |
+  -------------------------------------------------------------
+
+allOf:
+  - $ref: /schemas/interrupt-controller.yaml#
+  - $ref: /schemas/interrupt-controller/msi-controller.yaml#
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - qemu,imsics
+      - const: riscv,imsics
+
+  reg:
+    minItems: 1
+    maxItems: 16384
+    description:
+      Base address of each IMSIC group.
+
+  interrupt-controller: true
+
+  "#interrupt-cells":
+    const: 0
+
+  msi-controller: true
+
+  "#msi-cells":
+    const: 0
+
+  interrupts-extended:
+    minItems: 1
+    maxItems: 16384
+    description:
+      This property represents the set of CPUs (or HARTs) for which given
+      device tree node describes the IMSIC interrupt files. Each node pointed
+      to should be a riscv,cpu-intc node, which has a CPU node (i.e. RISC-V
+      HART) as parent.
+
+  riscv,num-ids:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    minimum: 63
+    maximum: 2047
+    description:
+      Number of interrupt identities supported by IMSIC interrupt file.
+
+  riscv,num-guest-ids:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    minimum: 63
+    maximum: 2047
+    description:
+      Number of interrupt identities are supported by IMSIC guest interrupt
+      file. When not specified it is assumed to be same as specified by the
+      riscv,num-ids property.
+
+  riscv,guest-index-bits:
+    minimum: 0
+    maximum: 7
+    default: 0
+    description:
+      Number of guest index bits in the MSI target address.
+
+  riscv,hart-index-bits:
+    minimum: 0
+    maximum: 15
+    description:
+      Number of HART index bits in the MSI target address. When not
+      specified it is calculated based on the interrupts-extended property.
+
+  riscv,group-index-bits:
+    minimum: 0
+    maximum: 7
+    default: 0
+    description:
+      Number of group index bits in the MSI target address.
+
+  riscv,group-index-shift:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    minimum: 0
+    maximum: 55
+    default: 24
+    description:
+      The least significant bit position of the group index bits in the
+      MSI target address.
+
+required:
+  - compatible
+  - reg
+  - interrupt-controller
+  - msi-controller
+  - "#msi-cells"
+  - interrupts-extended
+  - riscv,num-ids
+
+unevaluatedProperties: false
+
+examples:
+  - |
+    // Example 1 (Machine-level IMSIC files with just one group):
+
+    interrupt-controller@24000000 {
+      compatible = "qemu,imsics", "riscv,imsics";
+      interrupts-extended = <&cpu1_intc 11>,
+                            <&cpu2_intc 11>,
+                            <&cpu3_intc 11>,
+                            <&cpu4_intc 11>;
+      reg = <0x28000000 0x4000>;
+      interrupt-controller;
+      #interrupt-cells = <0>;
+      msi-controller;
+      #msi-cells = <0>;
+      riscv,num-ids = <127>;
+    };
+
+  - |
+    // Example 2 (Supervisor-level IMSIC files with two groups):
+
+    interrupt-controller@28000000 {
+      compatible = "qemu,imsics", "riscv,imsics";
+      interrupts-extended = <&cpu1_intc 9>,
+                            <&cpu2_intc 9>,
+                            <&cpu3_intc 9>,
+                            <&cpu4_intc 9>;
+      reg = <0x28000000 0x2000>, /* Group0 IMSICs */
+            <0x29000000 0x2000>; /* Group1 IMSICs */
+      interrupt-controller;
+      #interrupt-cells = <0>;
+      msi-controller;
+      #msi-cells = <0>;
+      riscv,num-ids = <127>;
+      riscv,group-index-bits = <1>;
+      riscv,group-index-shift = <24>;
+    };
+...
Signature · 7 lines
2.34.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#8 lore raw

PATCH v5 4/9 irqchip: Add RISC-V incoming MSI controller driver

In reply to Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new MSI controller for managing MSIs and IPIs on a RISC-V platform.

This new MSI controller is referred to as incoming message signalled interrupt controller (IMSIC) which manages MSI on per-HART (or per-CPU) basis. (For more details refer https://github.com/riscv/riscv-aia)

This patch adds an irqchip driver for RISC-V IMSIC which provides IPIs and platform MSIs to the Linux RISC-V kernel.

---
 drivers/irqchip/Kconfig             |    7 +-
 drivers/irqchip/Makefile            |    1 +
 drivers/irqchip/irq-riscv-imsic.c   | 1011 +++++++++++++++++++++++++++
 include/linux/irqchip/riscv-imsic.h |   86 +++
 4 files changed, 1104 insertions(+), 1 deletion(-)
 create mode 100644 drivers/irqchip/irq-riscv-imsic.c
 create mode 100644 include/linux/irqchip/riscv-imsic.h
drivers/irqchip/Kconfig +6 -1
index 09e422da482f..8ef18be5f37b 100644
@@ -30,7 +30,6 @@ config ARM_GIC_V2M
 
 config GIC_NON_BANKED
 	bool
-
 config ARM_GIC_V3
 	bool
 	select IRQ_DOMAIN_HIERARCHY
@@ -545,6 +544,12 @@ config SIFIVE_PLIC
 	select IRQ_DOMAIN_HIERARCHY
 	select GENERIC_IRQ_EFFECTIVE_AFF_MASK if SMP
 
+config RISCV_IMSIC
+	bool
+	depends on RISCV
+	select IRQ_DOMAIN_HIERARCHY
+	select GENERIC_MSI_IRQ
+
 config EXYNOS_IRQ_COMBINER
 	bool "Samsung Exynos IRQ combiner support" if COMPILE_TEST
 	depends on (ARCH_EXYNOS && ARM) || COMPILE_TEST
drivers/irqchip/Makefile +1 -0
index ffd945fe71aa..577bde3e986b 100644
@@ -95,6 +95,7 @@ obj-$(CONFIG_QCOM_MPM)			+= irq-qcom-mpm.o
 obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
 obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
 obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
+obj-$(CONFIG_RISCV_IMSIC)		+= irq-riscv-imsic.o
 obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
 obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
 obj-$(CONFIG_IMX_INTMUX)		+= irq-imx-intmux.o
drivers/irqchip/irq-riscv-imsic.c new file +1011 -0

1,011 lines not on this page · raw message

include/linux/irqchip/riscv-imsic.h new file +86 -0

86 lines not on this page · raw message

Signature · 7 lines
2.34.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#9 lore raw

PATCH v5 5/9 irqchip/riscv-imsic: Add support for PCI MSI irqdomain

In reply to Anup Patel

The Linux PCI framework requires it's own dedicated MSI irqdomain so let us create PCI MSI irqdomain as child of the IMSIC base irqdomain.

---
 drivers/irqchip/Kconfig           |  7 +++++
 drivers/irqchip/irq-riscv-imsic.c | 49 +++++++++++++++++++++++++++++++
 2 files changed, 56 insertions(+)
drivers/irqchip/Kconfig +7 -0
index 8ef18be5f37b..d700980372ef 100644
@@ -550,6 +550,13 @@ config RISCV_IMSIC
 	select IRQ_DOMAIN_HIERARCHY
 	select GENERIC_MSI_IRQ
 
+config RISCV_IMSIC_PCI
+	bool
+	depends on RISCV_IMSIC
+	depends on PCI
+	depends on PCI_MSI
+	default RISCV_IMSIC
+
 config EXYNOS_IRQ_COMBINER
 	bool "Samsung Exynos IRQ combiner support" if COMPILE_TEST
 	depends on (ARCH_EXYNOS && ARM) || COMPILE_TEST
drivers/irqchip/irq-riscv-imsic.c +49 -0

79 lines not on this page · raw message

Signature · 7 lines
2.34.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#10 lore raw

PATCH v5 6/9 dt-bindings: interrupt-controller: Add RISC-V advanced PLIC

In reply to Anup Patel

We add DT bindings document for RISC-V advanced platform level interrupt controller (APLIC) defined by the RISC-V advanced interrupt architecture (AIA) specification.

---
 .../interrupt-controller/riscv,aplic.yaml     | 172 ++++++++++++++++++
 1 file changed, 172 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml
Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml new file +172 -0
new file mode 100644
index 000000000000..190a6499c932
@@ -0,0 +1,172 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/interrupt-controller/riscv,aplic.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: RISC-V Advanced Platform Level Interrupt Controller (APLIC)
+
+maintainers:
+  - Anup Patel <[email protected]>
+
+description:
+  The RISC-V advanced interrupt architecture (AIA) defines an advanced
+  platform level interrupt controller (APLIC) for handling wired interrupts
+  in a RISC-V platform. The RISC-V AIA specification can be found at
+  https://github.com/riscv/riscv-aia.
+
+  The RISC-V APLIC is implemented as hierarchical APLIC domains where all
+  interrupt sources connect to the root APLIC domain and a parent APLIC
+  domain can delegate interrupt sources to it's child APLIC domains. There
+  is one device tree node for each APLIC domain.
+
+allOf:
+  - $ref: /schemas/interrupt-controller.yaml#
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - qemu,aplic
+      - const: riscv,aplic
+
+  reg:
+    maxItems: 1
+
+  interrupt-controller: true
+
+  "#interrupt-cells":
+    const: 2
+
+  interrupts-extended:
+    minItems: 1
+    maxItems: 16384
+    description:
+      Given APLIC domain directly injects external interrupts to a set of
+      RISC-V HARTS (or CPUs). Each node pointed to should be a riscv,cpu-intc
+      node, which has a CPU node (i.e. RISC-V HART) as parent.
+
+  msi-parent:
+    description:
+      Given APLIC domain forwards wired interrupts as MSIs to a AIA incoming
+      message signaled interrupt controller (IMSIC). If both "msi-parent" and
+      "interrupts-extended" properties are present then it means the APLIC
+      domain supports both MSI mode and Direct mode in HW. In this case, the
+      APLIC driver has to choose between MSI mode or Direct mode.
+
+  riscv,num-sources:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    minimum: 1
+    maximum: 1023
+    description:
+      Specifies the number of wired interrupt sources supported by this
+      APLIC domain.
+
+  riscv,children:
+    $ref: /schemas/types.yaml#/definitions/phandle-array
+    minItems: 1
+    maxItems: 1024
+    items:
+      maxItems: 1
+    description:
+      A list of child APLIC domains for the given APLIC domain. Each child
+      APLIC domain is assigned a child index in increasing order, with the
+      first child APLIC domain assigned child index 0. The APLIC domain child
+      index is used by firmware to delegate interrupts from the given APLIC
+      domain to a particular child APLIC domain.
+
+  riscv,delegation:
+    $ref: /schemas/types.yaml#/definitions/phandle-array
+    minItems: 1
+    maxItems: 1024
+    items:
+      items:
+        - description: child APLIC domain phandle
+        - description: first interrupt number of the parent APLIC domain (inclusive)
+        - description: last interrupt number of the parent APLIC domain (inclusive)
+    description:
+      A interrupt delegation list where each entry is a triple consisting
+      of child APLIC domain phandle, first interrupt number of the parent
+      APLIC domain, and last interrupt number of the parent APLIC domain.
+      Firmware must configure interrupt delegation registers based on
+      interrupt delegation list.
+
+dependencies:
+  riscv,delegation: [ "riscv,children" ]
+
+required:
+  - compatible
+  - reg
+  - interrupt-controller
+  - "#interrupt-cells"
+  - riscv,num-sources
+
+anyOf:
+  - required:
+      - interrupts-extended
+  - required:
+      - msi-parent
+
+unevaluatedProperties: false
+
+examples:
+  - |
+    // Example 1 (APLIC domains directly injecting interrupt to HARTs):
+
+    interrupt-controller@c000000 {
+      compatible = "qemu,aplic", "riscv,aplic";
+      interrupts-extended = <&cpu1_intc 11>,
+                            <&cpu2_intc 11>,
+                            <&cpu3_intc 11>,
+                            <&cpu4_intc 11>;
+      reg = <0xc000000 0x4080>;
+      interrupt-controller;
+      #interrupt-cells = <2>;
+      riscv,num-sources = <63>;
+      riscv,children = <&aplic1>, <&aplic2>;
+      riscv,delegation = <&aplic1 1 63>;
+    };
+
+    aplic1: interrupt-controller@d000000 {
+      compatible = "qemu,aplic", "riscv,aplic";
+      interrupts-extended = <&cpu1_intc 9>,
+                            <&cpu2_intc 9>;
+      reg = <0xd000000 0x4080>;
+      interrupt-controller;
+      #interrupt-cells = <2>;
+      riscv,num-sources = <63>;
+    };
+
+    aplic2: interrupt-controller@e000000 {
+      compatible = "qemu,aplic", "riscv,aplic";
+      interrupts-extended = <&cpu3_intc 9>,
+                            <&cpu4_intc 9>;
+      reg = <0xe000000 0x4080>;
+      interrupt-controller;
+      #interrupt-cells = <2>;
+      riscv,num-sources = <63>;
+    };
+
+  - |
+    // Example 2 (APLIC domains forwarding interrupts as MSIs):
+
+    interrupt-controller@c000000 {
+      compatible = "qemu,aplic", "riscv,aplic";
+      msi-parent = <&imsic_mlevel>;
+      reg = <0xc000000 0x4000>;
+      interrupt-controller;
+      #interrupt-cells = <2>;
+      riscv,num-sources = <63>;
+      riscv,children = <&aplic3>;
+      riscv,delegation = <&aplic3 1 63>;
+    };
+
+    aplic3: interrupt-controller@d000000 {
+      compatible = "qemu,aplic", "riscv,aplic";
+      msi-parent = <&imsic_slevel>;
+      reg = <0xd000000 0x4000>;
+      interrupt-controller;
+      #interrupt-cells = <2>;
+      riscv,num-sources = <63>;
+    };
+...
Signature · 7 lines
2.34.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#11 lore raw

PATCH v5 7/9 irqchip: Add RISC-V advanced PLIC driver

In reply to Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new interrupt controller for managing wired interrupts on a RISC-V platform. This new interrupt controller is referred to as advanced platform-level interrupt controller (APLIC) which can forward wired interrupts to CPUs (or HARTs) as local interrupts OR as message signaled interrupts.
(For more details refer https://github.com/riscv/riscv-aia)

This patch adds an irqchip driver for RISC-V APLIC found on RISC-V platforms.

---
 drivers/irqchip/Kconfig             |   6 +
 drivers/irqchip/Makefile            |   1 +
 drivers/irqchip/irq-riscv-aplic.c   | 774 ++++++++++++++++++++++++++++
 include/linux/irqchip/riscv-aplic.h | 119 +++++
 4 files changed, 900 insertions(+)
 create mode 100644 drivers/irqchip/irq-riscv-aplic.c
 create mode 100644 include/linux/irqchip/riscv-aplic.h
drivers/irqchip/Kconfig +6 -0
index d700980372ef..834c0329f583 100644
@@ -544,6 +544,12 @@ config SIFIVE_PLIC
 	select IRQ_DOMAIN_HIERARCHY
 	select GENERIC_IRQ_EFFECTIVE_AFF_MASK if SMP
 
+config RISCV_APLIC
+	bool
+	depends on RISCV
+	select IRQ_DOMAIN_HIERARCHY
+	select GENERIC_MSI_IRQ
+
 config RISCV_IMSIC
 	bool
 	depends on RISCV
drivers/irqchip/Makefile +1 -0
index 577bde3e986b..438b8e1a152c 100644
@@ -95,6 +95,7 @@ obj-$(CONFIG_QCOM_MPM)			+= irq-qcom-mpm.o
 obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
 obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
 obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
+obj-$(CONFIG_RISCV_APLIC)		+= irq-riscv-aplic.o
 obj-$(CONFIG_RISCV_IMSIC)		+= irq-riscv-imsic.o
 obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
 obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
drivers/irqchip/irq-riscv-aplic.c new file +774 -0

774 lines not on this page · raw message

include/linux/irqchip/riscv-aplic.h new file +119 -0

119 lines not on this page · raw message

Signature · 7 lines
2.34.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#21 lore raw

PATCH v5 8/9 RISC-V: Select APLIC and IMSIC drivers

In reply to Anup Patel

The QEMU virt machine supports AIA emulation and we also have quite a few RISC-V platforms with AIA support under development so let us select APLIC and IMSIC drivers for all RISC-V platforms.

---
 arch/riscv/Kconfig | 2 ++
 1 file changed, 2 insertions(+)
arch/riscv/Kconfig +2 -0
index 4c07b9189c86..318f62a0a187 100644
@@ -148,6 +148,8 @@ config RISCV
 	select PCI_DOMAINS_GENERIC if PCI
 	select PCI_MSI if PCI
 	select RISCV_ALTERNATIVE if !XIP_KERNEL
+	select RISCV_APLIC
+	select RISCV_IMSIC
 	select RISCV_INTC
 	select RISCV_TIMER if RISCV_SBI
 	select SIFIVE_PLIC
Signature · 7 lines
2.34.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#22 lore raw

PATCH v5 9/9 MAINTAINERS: Add entry for RISC-V AIA drivers

In reply to Anup Patel

Add myself as maintainer for RISC-V AIA drivers including the RISC-V INTC driver which supports both AIA and non-AIA platforms.

---
 MAINTAINERS | 12 ++++++++++++
 1 file changed, 12 insertions(+)
MAINTAINERS +12 -0
index 3be1bdfe8ecc..a85fc0102179 100644
@@ -18245,6 +18245,18 @@ S:	Maintained
 F:	drivers/mtd/nand/raw/r852.c
 F:	drivers/mtd/nand/raw/r852.h
 
+RISC-V AIA DRIVERS
+M:	Anup Patel <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml
+F:	Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml
+F:	drivers/irqchip/irq-riscv-aplic.c
+F:	drivers/irqchip/irq-riscv-imsic.c
+F:	drivers/irqchip/irq-riscv-intc.c
+F:	include/linux/irqchip/riscv-aplic.h
+F:	include/linux/irqchip/riscv-imsic.h
+
 RISC-V ARCHITECTURE
 M:	Paul Walmsley <[email protected]>
 M:	Palmer Dabbelt <[email protected]>
Signature · 7 lines
2.34.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#2 lore raw

Re: [PATCH v5 1/9] RISC-V: Add riscv_fw_parent_hartid() function

In reply to Anup Patel
Quoted 29 lines · Anup Patel

We add common riscv_fw_parent_hartid() which help device drivers to get parent hartid of the INTC (i.e. local interrupt controller) fwnode. This should work for both DT and ACPI.

---
 arch/riscv/include/asm/processor.h |  3 +++
 arch/riscv/kernel/cpu.c            | 16 ++++++++++++++++
 2 files changed, 19 insertions(+)
arch/riscv/include/asm/processor.h +3 -0
index c950a8d9edef..39dc23a18f88 100644
@@ -81,6 +81,9 @@ int riscv_of_processor_hartid(struct device_node *node, unsigned long *hartid);
 int riscv_early_of_processor_hartid(struct device_node *node, unsigned long *hartid);
 int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid);
 
+struct fwnode_handle;
+int riscv_fw_parent_hartid(struct fwnode_handle *node, unsigned long *hartid);
+
 extern void riscv_fill_hwcap(void);
 extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src);
 
arch/riscv/kernel/cpu.c +0 -0
index a2fc952318e9..9be9b3b1f333 100644

arch/riscv/kernel/cpu.c int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid) · line 99

Quoted 4 lines
arch/riscv/kernel/cpu.c +1 -0
@@ -96,6 +96,22 @@ int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid)
 	return -1;
 }
 
+/* Find hart ID of the CPU fwnode under which given fwnode falls. */

This comment matches the comment for riscv_of_parent_hartid(), but I don't think it will be correct for the !is_of_node(node) case since fwnode_property_read_u64_array() isn't obliged to walk up its tree. Looking ahead it appears riscv_fw_parent_hartid() is only called with the parent node, so we could just drop this function and use fwnode_property_read_u64_array() directly at the two call sites.

Thanks,
drew

Quoted 21 lines
arch/riscv/kernel/cpu.c +15 -0
+int riscv_fw_parent_hartid(struct fwnode_handle *node, unsigned long *hartid)
+{
+	int rc;
+	u64 temp;
+
+	if (!is_of_node(node)) {
+		rc = fwnode_property_read_u64_array(node, "hartid", &temp, 1);
+		if (!rc)
+			*hartid = temp;
+	} else
+		rc = riscv_of_parent_hartid(to_of_node(node), hartid);
+
+	return rc;
+}
+
 DEFINE_PER_CPU(struct riscv_cpuinfo, riscv_cpuinfo);
 
 unsigned long riscv_cached_mvendorid(unsigned int cpu_id)
Signature · 1 line
2.34.1

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#5 lore raw

Re: [PATCH v5 2/9] irqchip/riscv-intc: Add support for RISC-V AIA

In reply to Anup Patel
Quoted 7 lines · Anup Patel

The RISC-V advanced interrupt architecture (AIA) extends the per-HART local interrupts in following ways:
1. Minimum 64 local interrupts for both RV32 and RV64
2. Ability to process multiple pending local interrupts in same interrupt handler
3. Priority configuration for each local interrupts
4. Special CSRs to configure/access the per-HART MSI controller

afaict, we're only doing (1) and (2) from this list in this patch.

Quoted 37 lines

This patch adds support for RISC-V AIA in the RISC-V intc driver.

---
 drivers/irqchip/irq-riscv-intc.c | 36 ++++++++++++++++++++++++++------
 1 file changed, 30 insertions(+), 6 deletions(-)
drivers/irqchip/irq-riscv-intc.c +10 -0
index 4adeee1bc391..e235bf1708a4 100644
@@ -17,6 +17,7 @@
 #include <linux/module.h>
 #include <linux/of.h>
 #include <linux/smp.h>
+#include <asm/hwcap.h>
 
 static struct irq_domain *intc_domain;
 
@@ -30,6 +31,15 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 	generic_handle_domain_irq(intc_domain, cause);
 }
 
+static asmlinkage void riscv_intc_aia_irq(struct pt_regs *regs)
+{
+	unsigned long topi;
+
+	while ((topi = csr_read(CSR_TOPI)))
+		generic_handle_domain_irq(intc_domain,
+					  topi >> TOPI_IID_SHIFT);
+}
+
 /*
  * On RISC-V systems local interrupts are masked or unmasked by writing
  * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be written

drivers/irqchip/irq-riscv-intc.c static asmlinkage void riscv_intc_irq(struct pt_regs *regs) · line 55

Quoted 8 lines
drivers/irqchip/irq-riscv-intc.c +4 -1
@@ -39,12 +49,18 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 
 static void riscv_intc_irq_mask(struct irq_data *d)
 {
-	csr_clear(CSR_IE, BIT(d->hwirq));
+	if (d->hwirq < BITS_PER_LONG)
+		csr_clear(CSR_IE, BIT(d->hwirq));
+	else
+		csr_clear(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));

We can optimize rv64 by allowing the compiler to remove the branch

if (IS_ENABLED(CONFIG_32BIT) && d->hwirq >= 32)

    csr_clear(CSR_IEH, BIT(d->hwirq - 32));
 else
    csr_clear(CSR_IE, BIT(d->hwirq));

drivers/irqchip/irq-riscv-intc.c static asmlinkage void riscv_intc_irq(struct pt_regs *regs) · line 63

Quoted 9 lines
drivers/irqchip/irq-riscv-intc.c +4 -1
 }
 
 static void riscv_intc_irq_unmask(struct irq_data *d)
 {
-	csr_set(CSR_IE, BIT(d->hwirq));
+	if (d->hwirq < BITS_PER_LONG)
+		csr_set(CSR_IE, BIT(d->hwirq));
+	else
+		csr_set(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));

Same comment as above.

drivers/irqchip/irq-riscv-intc.c static struct fwnode_handle *riscv_intc_hwnode(void) · line 137

Quoted 12 lines
drivers/irqchip/irq-riscv-intc.c +4 -1
 }
 
 static void riscv_intc_irq_eoi(struct irq_data *d)
@@ -115,16 +131,22 @@ static struct fwnode_handle *riscv_intc_hwnode(void)
 
 static int __init riscv_intc_init_common(struct fwnode_handle *fn)
 {
-	int rc;
+	int rc, nr_irqs = BITS_PER_LONG;
+
+	if (riscv_isa_extension_available(NULL, SxAIA) && BITS_PER_LONG == 32)
+		nr_irqs = nr_irqs * 2;

The AIA spec states sie and sip are explicitly 64, so how about writing this as

int rc, nr_irqs = BITS_PER_LONG;

if (riscv_isa_extension_available(NULL, SxAIA))

     nr_irqs = 64;
Quoted 4 lines
drivers/irqchip/irq-riscv-intc.c +1 -1
-	intc_domain = irq_domain_create_linear(fn, BITS_PER_LONG,
+	intc_domain = irq_domain_create_linear(fn, nr_irqs,
 					       &riscv_intc_domain_ops, NULL);

drivers/irqchip/irq-riscv-intc.c static struct fwnode_handle *riscv_intc_hwnode(void) · line 149

Quoted 10 lines
drivers/irqchip/irq-riscv-intc.c +4 -1
 	if (!intc_domain) {
 		pr_err("unable to add IRQ domain\n");
 		return -ENXIO;
 	}
 
-	rc = set_handle_irq(&riscv_intc_irq);
+	if (riscv_isa_extension_available(NULL, SxAIA))
+		rc = set_handle_irq(&riscv_intc_aia_irq);
+	else
+		rc = set_handle_irq(&riscv_intc_irq);

nit: blank line here

drivers/irqchip/irq-riscv-intc.c static int __init riscv_intc_init_common(struct fwnode_handle *fn) · line 158

Quoted 10 lines
drivers/irqchip/irq-riscv-intc.c +2 -1
 	if (rc) {
 		pr_err("failed to set irq handler\n");
 		return rc;
@@ -132,7 +154,9 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
 
 	riscv_set_intc_hwnode_fn(riscv_intc_hwnode);
 
-	pr_info("%d local interrupts mapped\n", BITS_PER_LONG);
+	pr_info("%d local interrupts mapped%s\n",
+		nr_irqs, (riscv_isa_extension_available(NULL, SxAIA)) ?

nit: unnecessary ()

Quoted 7 lines
drivers/irqchip/irq-riscv-intc.c +1 -0
+			 " using AIA" : "");
 
 	return 0;
 }
Signature · 1 line
2.34.1

Thanks,
drew

#12 lore raw

Re: [PATCH v5 7/9] irqchip: Add RISC-V advanced PLIC driver

In reply to Anup Patel
Quoted 817 lines · Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new interrupt controller for managing wired interrupts on a RISC-V platform. This new interrupt controller is referred to as advanced platform-level interrupt controller (APLIC) which can forward wired interrupts to CPUs (or HARTs) as local interrupts OR as message signaled interrupts.
(For more details refer https://github.com/riscv/riscv-aia)

This patch adds an irqchip driver for RISC-V APLIC found on RISC-V platforms.

---
 drivers/irqchip/Kconfig             |   6 +
 drivers/irqchip/Makefile            |   1 +
 drivers/irqchip/irq-riscv-aplic.c   | 774 ++++++++++++++++++++++++++++
 include/linux/irqchip/riscv-aplic.h | 119 +++++
 4 files changed, 900 insertions(+)
 create mode 100644 drivers/irqchip/irq-riscv-aplic.c
 create mode 100644 include/linux/irqchip/riscv-aplic.h
drivers/irqchip/Kconfig +6 -0
index d700980372ef..834c0329f583 100644
@@ -544,6 +544,12 @@ config SIFIVE_PLIC
        select IRQ_DOMAIN_HIERARCHY
        select GENERIC_IRQ_EFFECTIVE_AFF_MASK if SMP
 
+config RISCV_APLIC
+       bool
+       depends on RISCV
+       select IRQ_DOMAIN_HIERARCHY
+       select GENERIC_MSI_IRQ
+
 config RISCV_IMSIC
        bool
        depends on RISCV
drivers/irqchip/Makefile +1 -0
index 577bde3e986b..438b8e1a152c 100644
@@ -95,6 +95,7 @@ obj-$(CONFIG_QCOM_MPM)                        += irq-qcom-mpm.o
 obj-$(CONFIG_CSKY_MPINTC)              += irq-csky-mpintc.o
 obj-$(CONFIG_CSKY_APB_INTC)            += irq-csky-apb-intc.o
 obj-$(CONFIG_RISCV_INTC)               += irq-riscv-intc.o
+obj-$(CONFIG_RISCV_APLIC)              += irq-riscv-aplic.o
 obj-$(CONFIG_RISCV_IMSIC)              += irq-riscv-imsic.o
 obj-$(CONFIG_SIFIVE_PLIC)              += irq-sifive-plic.o
 obj-$(CONFIG_IMX_IRQSTEER)             += irq-imx-irqsteer.o
drivers/irqchip/irq-riscv-aplic.c new file +760 -0
new file mode 100644
index 000000000000..b4db29d4179b
@@ -0,0 +1,774 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 Western Digital Corporation or its affiliates.
+ * Copyright (C) 2022 Ventana Micro Systems Inc.
+ */
+
+#define pr_fmt(fmt) "riscv-aplic: " fmt
+#include <linux/bitops.h>
+#include <linux/cpu.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/irq.h>
+#include <linux/irqchip.h>
+#include <linux/irqchip/chained_irq.h>
+#include <linux/irqchip/riscv-aplic.h>
+#include <linux/irqchip/riscv-imsic.h>
+#include <linux/irqdomain.h>
+#include <linux/module.h>
+#include <linux/msi.h>
+#include <linux/platform_device.h>
+#include <linux/smp.h>
+
+#define APLIC_DEFAULT_PRIORITY         1
+#define APLIC_DISABLE_IDELIVERY                0
+#define APLIC_ENABLE_IDELIVERY         1
+#define APLIC_DISABLE_ITHRESHOLD       1
+#define APLIC_ENABLE_ITHRESHOLD                0
+
+struct aplic_msicfg {
+       phys_addr_t             base_ppn;
+       u32                     hhxs;
+       u32                     hhxw;
+       u32                     lhxs;
+       u32                     lhxw;
+};
+
+struct aplic_idc {
+       unsigned int            hart_index;
+       void __iomem            *regs;
+       struct aplic_priv       *priv;
+};
+
+struct aplic_priv {
+       struct fwnode_handle    *fwnode;
+       u32                     gsi_base;
+       u32                     nr_irqs;
+       u32                     nr_idcs;
+       void __iomem            *regs;
+       struct irq_domain       *irqdomain;
+       struct aplic_msicfg     msicfg;
+       struct cpumask          lmask;
+};
+
+static unsigned int aplic_idc_parent_irq;
+static DEFINE_PER_CPU(struct aplic_idc, aplic_idcs);
+
+static void aplic_irq_unmask(struct irq_data *d)
+{
+       struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
+
+       writel(d->hwirq, priv->regs + APLIC_SETIENUM);
+
+       if (!priv->nr_idcs)
+               irq_chip_unmask_parent(d);
+}
+
+static void aplic_irq_mask(struct irq_data *d)
+{
+       struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
+
+       writel(d->hwirq, priv->regs + APLIC_CLRIENUM);
+
+       if (!priv->nr_idcs)
+               irq_chip_mask_parent(d);
+}
+
+static int aplic_set_type(struct irq_data *d, unsigned int type)
+{
+       u32 val = 0;
+       void __iomem *sourcecfg;
+       struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
+
+       switch (type) {
+       case IRQ_TYPE_NONE:
+               val = APLIC_SOURCECFG_SM_INACTIVE;
+               break;
+       case IRQ_TYPE_LEVEL_LOW:
+               val = APLIC_SOURCECFG_SM_LEVEL_LOW;
+               break;
+       case IRQ_TYPE_LEVEL_HIGH:
+               val = APLIC_SOURCECFG_SM_LEVEL_HIGH;
+               break;
+       case IRQ_TYPE_EDGE_FALLING:
+               val = APLIC_SOURCECFG_SM_EDGE_FALL;
+               break;
+       case IRQ_TYPE_EDGE_RISING:
+               val = APLIC_SOURCECFG_SM_EDGE_RISE;
+               break;
+       default:
+               return -EINVAL;
+       }
+
+       sourcecfg = priv->regs + APLIC_SOURCECFG_BASE;
+       sourcecfg += (d->hwirq - 1) * sizeof(u32);
+       writel(val, sourcecfg);
+
+       return 0;
+}
+
+static void aplic_irq_eoi(struct irq_data *d)
+{
+       struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
+       u32 reg_off, reg_mask;
+
+       /*
+        * EOI handling only required only for level-triggered
+        * interrupts in APLIC MSI mode.
+        */
+
+       if (priv->nr_idcs)
+               return;
+
+       reg_off = APLIC_CLRIP_BASE + ((d->hwirq / APLIC_IRQBITS_PER_REG) * 4);
+       reg_mask = BIT(d->hwirq % APLIC_IRQBITS_PER_REG);
+       switch (irqd_get_trigger_type(d)) {
+       case IRQ_TYPE_LEVEL_LOW:
+               if (!(readl(priv->regs + reg_off) & reg_mask))
+                       writel(d->hwirq, priv->regs + APLIC_SETIPNUM_LE);
+               break;
+       case IRQ_TYPE_LEVEL_HIGH:
+               if (readl(priv->regs + reg_off) & reg_mask)
+                       writel(d->hwirq, priv->regs + APLIC_SETIPNUM_LE);
+               break;
+       }
+}
+
+#ifdef CONFIG_SMP
+static int aplic_set_affinity(struct irq_data *d,
+                             const struct cpumask *mask_val, bool force)
+{
+       struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
+       struct aplic_idc *idc;
+       unsigned int cpu, val;
+       struct cpumask amask;
+       void __iomem *target;
+
+       if (!priv->nr_idcs)
+               return irq_chip_set_affinity_parent(d, mask_val, force);
+
+       cpumask_and(&amask, &priv->lmask, mask_val);
+
+       if (force)
+               cpu = cpumask_first(&amask);
+       else
+               cpu = cpumask_any_and(&amask, cpu_online_mask);
+
+       if (cpu >= nr_cpu_ids)
+               return -EINVAL;
+
+       idc = per_cpu_ptr(&aplic_idcs, cpu);
+       target = priv->regs + APLIC_TARGET_BASE;
+       target += (d->hwirq - 1) * sizeof(u32);
+       val = idc->hart_index & APLIC_TARGET_HART_IDX_MASK;
+       val <<= APLIC_TARGET_HART_IDX_SHIFT;
+       val |= APLIC_DEFAULT_PRIORITY;
+       writel(val, target);
+
+       irq_data_update_effective_affinity(d, cpumask_of(cpu));
+
+       return IRQ_SET_MASK_OK_DONE;
+}
+#endif
+
+static struct irq_chip aplic_chip = {
+       .name           = "RISC-V APLIC",
+       .irq_mask       = aplic_irq_mask,
+       .irq_unmask     = aplic_irq_unmask,
+       .irq_set_type   = aplic_set_type,
+       .irq_eoi        = aplic_irq_eoi,
+#ifdef CONFIG_SMP
+       .irq_set_affinity = aplic_set_affinity,
+#endif
+       .flags          = IRQCHIP_SET_TYPE_MASKED |
+                         IRQCHIP_SKIP_SET_WAKE |
+                         IRQCHIP_MASK_ON_SUSPEND,
+};
+
+static int aplic_irqdomain_translate(struct irq_fwspec *fwspec,
+                                    u32 gsi_base,
+                                    unsigned long *hwirq,
+                                    unsigned int *type)
+{
+       if (WARN_ON(fwspec->param_count < 2))
+               return -EINVAL;
+       if (WARN_ON(!fwspec->param[0]))
+               return -EINVAL;
+
+       /* For DT, gsi_base is always zero. */
+       *hwirq = fwspec->param[0] - gsi_base;
+       *type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
+
+       WARN_ON(*type == IRQ_TYPE_NONE);
+
+       return 0;
+}
+
+static int aplic_irqdomain_msi_translate(struct irq_domain *d,
+                                        struct irq_fwspec *fwspec,
+                                        unsigned long *hwirq,
+                                        unsigned int *type)
+{
+       struct aplic_priv *priv = platform_msi_get_host_data(d);
+
+       return aplic_irqdomain_translate(fwspec, priv->gsi_base, hwirq, type);
+}
+
+static int aplic_irqdomain_msi_alloc(struct irq_domain *domain,
+                                    unsigned int virq, unsigned int nr_irqs,
+                                    void *arg)
+{
+       int i, ret;
+       unsigned int type;
+       irq_hw_number_t hwirq;
+       struct irq_fwspec *fwspec = arg;
+       struct aplic_priv *priv = platform_msi_get_host_data(domain);
+
+       ret = aplic_irqdomain_translate(fwspec, priv->gsi_base, &hwirq, &type);
+       if (ret)
+               return ret;
+
+       ret = platform_msi_device_domain_alloc(domain, virq, nr_irqs);
+       if (ret)
+               return ret;
+
+       for (i = 0; i < nr_irqs; i++) {
+               irq_domain_set_info(domain, virq + i, hwirq + i,
+                                   &aplic_chip, priv, handle_fasteoi_irq,
+                                   NULL, NULL);
+               /*
+                * APLIC does not implement irq_disable() so Linux interrupt
+                * subsystem will take a lazy approach for disabling an APLIC
+                * interrupt. This means APLIC interrupts are left unmasked
+                * upon system suspend and interrupts are not processed
+                * immediately upon system wake up. To tackle this, we disable
+                * the lazy approach for all APLIC interrupts.
+                */
+               irq_set_status_flags(virq + i, IRQ_DISABLE_UNLAZY);
+       }
+
+       return 0;
+}
+
+static const struct irq_domain_ops aplic_irqdomain_msi_ops = {
+       .translate      = aplic_irqdomain_msi_translate,
+       .alloc          = aplic_irqdomain_msi_alloc,
+       .free           = platform_msi_device_domain_free,
+};
+
+static int aplic_irqdomain_idc_translate(struct irq_domain *d,
+                                        struct irq_fwspec *fwspec,
+                                        unsigned long *hwirq,
+                                        unsigned int *type)
+{
+       struct aplic_priv *priv = d->host_data;
+
+       return aplic_irqdomain_translate(fwspec, priv->gsi_base, hwirq, type);
+}
+
+static int aplic_irqdomain_idc_alloc(struct irq_domain *domain,
+                                    unsigned int virq, unsigned int nr_irqs,
+                                    void *arg)
+{
+       int i, ret;
+       unsigned int type;
+       irq_hw_number_t hwirq;
+       struct irq_fwspec *fwspec = arg;
+       struct aplic_priv *priv = domain->host_data;
+
+       ret = aplic_irqdomain_translate(fwspec, priv->gsi_base, &hwirq, &type);
+       if (ret)
+               return ret;
+
+       for (i = 0; i < nr_irqs; i++) {
+               irq_domain_set_info(domain, virq + i, hwirq + i,
+                                   &aplic_chip, priv, handle_fasteoi_irq,
+                                   NULL, NULL);
+               irq_set_affinity(virq + i, &priv->lmask);
+               /* See the reason described in aplic_irqdomain_msi_alloc() */
+               irq_set_status_flags(virq + i, IRQ_DISABLE_UNLAZY);
+       }
+
+       return 0;
+}
+
+static const struct irq_domain_ops aplic_irqdomain_idc_ops = {
+       .translate      = aplic_irqdomain_idc_translate,
+       .alloc          = aplic_irqdomain_idc_alloc,
+       .free           = irq_domain_free_irqs_top,
+};
+
+static void aplic_init_hw_irqs(struct aplic_priv *priv)
+{
+       int i;
+
+       /* Disable all interrupts */
+       for (i = 0; i <= priv->nr_irqs; i += 32)
+               writel(-1U, priv->regs + APLIC_CLRIE_BASE +
+                           (i / 32) * sizeof(u32));
+
+       /* Set interrupt type and default priority for all interrupts */
+       for (i = 1; i <= priv->nr_irqs; i++) {
+               writel(0, priv->regs + APLIC_SOURCECFG_BASE +
+                         (i - 1) * sizeof(u32));
+               writel(APLIC_DEFAULT_PRIORITY,
+                      priv->regs + APLIC_TARGET_BASE +
+                      (i - 1) * sizeof(u32));
+       }
+
+       /* Clear APLIC domaincfg */
+       writel(0, priv->regs + APLIC_DOMAINCFG);
+}
+
+static void aplic_init_hw_global(struct aplic_priv *priv)
+{
+       u32 val;
+#ifdef CONFIG_RISCV_M_MODE
+       u32 valH;
+
+       if (!priv->nr_idcs) {
+               val = priv->msicfg.base_ppn;
+               valH = (priv->msicfg.base_ppn >> 32) &
+                       APLIC_xMSICFGADDRH_BAPPN_MASK;
+               valH |= (priv->msicfg.lhxw & APLIC_xMSICFGADDRH_LHXW_MASK)
+                       << APLIC_xMSICFGADDRH_LHXW_SHIFT;
+               valH |= (priv->msicfg.hhxw & APLIC_xMSICFGADDRH_HHXW_MASK)
+                       << APLIC_xMSICFGADDRH_HHXW_SHIFT;
+               valH |= (priv->msicfg.lhxs & APLIC_xMSICFGADDRH_LHXS_MASK)
+                       << APLIC_xMSICFGADDRH_LHXS_SHIFT;
+               valH |= (priv->msicfg.hhxs & APLIC_xMSICFGADDRH_HHXS_MASK)
+                       << APLIC_xMSICFGADDRH_HHXS_SHIFT;
+               writel(val, priv->regs + APLIC_xMSICFGADDR);
+               writel(valH, priv->regs + APLIC_xMSICFGADDRH);
+       }
+#endif
+
+       /* Setup APLIC domaincfg register */
+       val = readl(priv->regs + APLIC_DOMAINCFG);
+       val |= APLIC_DOMAINCFG_IE;
+       if (!priv->nr_idcs)
+               val |= APLIC_DOMAINCFG_DM;
+       writel(val, priv->regs + APLIC_DOMAINCFG);
+       if (readl(priv->regs + APLIC_DOMAINCFG) != val)
+               pr_warn("%pfwP: unable to write 0x%x in domaincfg\n",
+                       priv->fwnode, val);
+}
+
+static void aplic_msi_write_msg(struct msi_desc *desc, struct msi_msg *msg)
+{
+       unsigned int group_index, hart_index, guest_index, val;
+       struct irq_data *d = irq_get_irq_data(desc->irq);
+       struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
+       struct aplic_msicfg *mc = &priv->msicfg;
+       phys_addr_t tppn, tbppn, msg_addr;
+       void __iomem *target;
+
+       /* For zeroed MSI, simply write zero into the target register */
+       if (!msg->address_hi && !msg->address_lo && !msg->data) {
+               target = priv->regs + APLIC_TARGET_BASE;
+               target += (d->hwirq - 1) * sizeof(u32);
+               writel(0, target);
+               return;
+       }
+
+       /* Sanity check on message data */
+       WARN_ON(msg->data > APLIC_TARGET_EIID_MASK);
+
+       /* Compute target MSI address */
+       msg_addr = (((u64)msg->address_hi) << 32) | msg->address_lo;
+       tppn = msg_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
+
+       /* Compute target HART Base PPN */
+       tbppn = tppn;
+       tbppn &= ~APLIC_xMSICFGADDR_PPN_HART(mc->lhxs);
+       tbppn &= ~APLIC_xMSICFGADDR_PPN_LHX(mc->lhxw, mc->lhxs);
+       tbppn &= ~APLIC_xMSICFGADDR_PPN_HHX(mc->hhxw, mc->hhxs);
+       WARN_ON(tbppn != mc->base_ppn);
+
+       /* Compute target group and hart indexes */
+       group_index = (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(mc->hhxs)) &
+                    APLIC_xMSICFGADDR_PPN_HHX_MASK(mc->hhxw);
+       hart_index = (tppn >> APLIC_xMSICFGADDR_PPN_LHX_SHIFT(mc->lhxs)) &
+                    APLIC_xMSICFGADDR_PPN_LHX_MASK(mc->lhxw);
+       hart_index |= (group_index << mc->lhxw);
+       WARN_ON(hart_index > APLIC_TARGET_HART_IDX_MASK);
+
+       /* Compute target guest index */
+       guest_index = tppn & APLIC_xMSICFGADDR_PPN_HART(mc->lhxs);
+       WARN_ON(guest_index > APLIC_TARGET_GUEST_IDX_MASK);
+
+       /* Update IRQ TARGET register */
+       target = priv->regs + APLIC_TARGET_BASE;
+       target += (d->hwirq - 1) * sizeof(u32);
+       val = (hart_index & APLIC_TARGET_HART_IDX_MASK)
+                               << APLIC_TARGET_HART_IDX_SHIFT;
+       val |= (guest_index & APLIC_TARGET_GUEST_IDX_MASK)
+                               << APLIC_TARGET_GUEST_IDX_SHIFT;
+       val |= (msg->data & APLIC_TARGET_EIID_MASK);
+       writel(val, target);
+}
+
+static int aplic_setup_msi(struct aplic_priv *priv)
+{
+       struct aplic_msicfg *mc = &priv->msicfg;
+       const struct imsic_global_config *imsic_global;
+
+       /*
+        * The APLIC outgoing MSI config registers assume target MSI
+        * controller to be RISC-V AIA IMSIC controller.
+        */
+       imsic_global = imsic_get_global_config();
+       if (!imsic_global) {
+               pr_err("%pfwP: IMSIC global config not found\n",
+                       priv->fwnode);
+               return -ENODEV;
+       }
+
+       /* Find number of guest index bits (LHXS) */
+       mc->lhxs = imsic_global->guest_index_bits;
+       if (APLIC_xMSICFGADDRH_LHXS_MASK < mc->lhxs) {
+               pr_err("%pfwP: IMSIC guest index bits big for APLIC LHXS\n",
+                       priv->fwnode);
+               return -EINVAL;
+       }
+
+       /* Find number of HART index bits (LHXW) */
+       mc->lhxw = imsic_global->hart_index_bits;
+       if (APLIC_xMSICFGADDRH_LHXW_MASK < mc->lhxw) {
+               pr_err("%pfwP: IMSIC hart index bits big for APLIC LHXW\n",
+                       priv->fwnode);
+               return -EINVAL;
+       }
+
+       /* Find number of group index bits (HHXW) */
+       mc->hhxw = imsic_global->group_index_bits;
+       if (APLIC_xMSICFGADDRH_HHXW_MASK < mc->hhxw) {
+               pr_err("%pfwP: IMSIC group index bits big for APLIC HHXW\n",
+                       priv->fwnode);
+               return -EINVAL;
+       }
+
+       /* Find first bit position of group index (HHXS) */
+       mc->hhxs = imsic_global->group_index_shift;
+       if (mc->hhxs < (2 * APLIC_xMSICFGADDR_PPN_SHIFT)) {
+               pr_err("%pfwP: IMSIC group index shift should be >= %d\n",
+                       priv->fwnode, (2 * APLIC_xMSICFGADDR_PPN_SHIFT));
+               return -EINVAL;
+       }
+       mc->hhxs -= (2 * APLIC_xMSICFGADDR_PPN_SHIFT);
+       if (APLIC_xMSICFGADDRH_HHXS_MASK < mc->hhxs) {
+               pr_err("%pfwP: IMSIC group index shift big for APLIC HHXS\n",
+                       priv->fwnode);
+               return -EINVAL;
+       }
+
+       /* Compute PPN base */
+       mc->base_ppn = imsic_global->base_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
+       mc->base_ppn &= ~APLIC_xMSICFGADDR_PPN_HART(mc->lhxs);
+       mc->base_ppn &= ~APLIC_xMSICFGADDR_PPN_LHX(mc->lhxw, mc->lhxs);
+       mc->base_ppn &= ~APLIC_xMSICFGADDR_PPN_HHX(mc->hhxw, mc->hhxs);
+
+       /* Use all possible CPUs as lmask */
+       cpumask_copy(&priv->lmask, cpu_possible_mask);
+
+       return 0;
+}
+
+/*
+ * To handle an APLIC IDC interrupts, we just read the CLAIMI register
+ * which will return highest priority pending interrupt and clear the
+ * pending bit of the interrupt. This process is repeated until CLAIMI
+ * register return zero value.
+ */
+static void aplic_idc_handle_irq(struct irq_desc *desc)
+{
+       struct aplic_idc *idc = this_cpu_ptr(&aplic_idcs);
+       struct irq_chip *chip = irq_desc_get_chip(desc);
+       irq_hw_number_t hw_irq;
+       int irq;
+
+       chained_irq_enter(chip, desc);
+
+       while ((hw_irq = readl(idc->regs + APLIC_IDC_CLAIMI))) {
+               hw_irq = hw_irq >> APLIC_IDC_TOPI_ID_SHIFT;
+               irq = irq_find_mapping(idc->priv->irqdomain, hw_irq);
+
+               if (unlikely(irq <= 0))
+                       pr_warn_ratelimited("hw_irq %lu mapping not found\n",
+                                           hw_irq);
+               else
+                       generic_handle_irq(irq);
+       }
+
+       chained_irq_exit(chip, desc);
+}
+
+static void aplic_idc_set_delivery(struct aplic_idc *idc, bool en)
+{
+       u32 de = (en) ? APLIC_ENABLE_IDELIVERY : APLIC_DISABLE_IDELIVERY;
+       u32 th = (en) ? APLIC_ENABLE_ITHRESHOLD : APLIC_DISABLE_ITHRESHOLD;
+
+       /* Priority must be less than threshold for interrupt triggering */
+       writel(th, idc->regs + APLIC_IDC_ITHRESHOLD);
+
+       /* Delivery must be set to 1 for interrupt triggering */
+       writel(de, idc->regs + APLIC_IDC_IDELIVERY);
+}
+
+static int aplic_idc_dying_cpu(unsigned int cpu)
+{
+       if (aplic_idc_parent_irq)
+               disable_percpu_irq(aplic_idc_parent_irq);
+
+       return 0;
+}
+
+static int aplic_idc_starting_cpu(unsigned int cpu)
+{
+       if (aplic_idc_parent_irq)
+               enable_percpu_irq(aplic_idc_parent_irq,
+                                 irq_get_trigger_type(aplic_idc_parent_irq));
+
+       return 0;
+}
+
+static int aplic_setup_idc(struct aplic_priv *priv)
+{
+       int i, j, rc, cpu, setup_count = 0;
+       struct fwnode_reference_args parent;
+       struct irq_domain *domain;
+       unsigned long hartid;
+       struct aplic_idc *idc;
+       u32 val;
+
+       /* Setup per-CPU IDC and target CPU mask */
+       for (i = 0; i < priv->nr_idcs; i++) {
+               rc = fwnode_property_get_reference_args(priv->fwnode,
+                               "interrupts-extended", "#interrupt-cells",
+                               0, i, &parent);
+               if (rc) {
+                       pr_warn("%pfwP: parent irq for IDC%d not found\n",
+                               priv->fwnode, i);
+                       continue;
+               }
+
+               /*
+                * Skip interrupts other than external interrupts for
+                * current privilege level.
+                */
+               if (parent.args[0] != RV_IRQ_EXT)
+                       continue;
+
+               rc = riscv_fw_parent_hartid(parent.fwnode, &hartid);
+               if (rc) {
+                       pr_warn("%pfwP: invalid hartid for IDC%d\n",
+                               priv->fwnode, i);
+                       continue;
+               }
+
+               cpu = riscv_hartid_to_cpuid(hartid);
+               if (cpu < 0) {
+                       pr_warn("%pfwP: invalid cpuid for IDC%d\n",
+                               priv->fwnode, i);
+                       continue;
+               }
+
+               cpumask_set_cpu(cpu, &priv->lmask);
+
+               idc = per_cpu_ptr(&aplic_idcs, cpu);
+               idc->hart_index = i;
+               idc->regs = priv->regs + APLIC_IDC_BASE + i * APLIC_IDC_SIZE;
+               idc->priv = priv;
+
+               aplic_idc_set_delivery(idc, true);
+
+               /*
+                * Boot cpu might not have APLIC hart_index = 0 so check
+                * and update target registers of all interrupts.
+                */
+               if (cpu == smp_processor_id() && idc->hart_index) {
+                       val = idc->hart_index & APLIC_TARGET_HART_IDX_MASK;
+                       val <<= APLIC_TARGET_HART_IDX_SHIFT;
+                       val |= APLIC_DEFAULT_PRIORITY;
+                       for (j = 1; j <= priv->nr_irqs; j++)
+                               writel(val, priv->regs + APLIC_TARGET_BASE +
+                                           (j - 1) * sizeof(u32));
+               }
+
+               setup_count++;
+       }
+
+       /* Find parent domain and register chained handler */
+       domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(),
+                                         DOMAIN_BUS_ANY);
+       if (!aplic_idc_parent_irq && domain) {
+               aplic_idc_parent_irq = irq_create_mapping(domain, RV_IRQ_EXT);
+               if (aplic_idc_parent_irq) {
+                       irq_set_chained_handler(aplic_idc_parent_irq,
+                                               aplic_idc_handle_irq);
+
+                       /*
+                        * Setup CPUHP notifier to enable IDC parent
+                        * interrupt on all CPUs
+                        */
+                       cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+                                         "irqchip/riscv/aplic:starting",
+                                         aplic_idc_starting_cpu,
+                                         aplic_idc_dying_cpu);
+               }
+       }
+
+       /* Fail if we were not able to setup IDC for any CPU */
+       return (setup_count) ? 0 : -ENODEV;
+}
+
+static int aplic_probe(struct platform_device *pdev)
+{
+       struct fwnode_handle *fwnode = pdev->dev.fwnode;
+       struct fwnode_reference_args parent;
+       struct aplic_priv *priv;
+       struct resource *res;
+       phys_addr_t pa;
+       int rc;
+
+       priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+       if (!priv)
+               return -ENOMEM;
+       priv->fwnode = fwnode;
+
+       /* Map the MMIO registers */
+       res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+       if (!res) {
+               pr_err("%pfwP: failed to get MMIO resource\n", fwnode);
+               return -EINVAL;
+       }
+       priv->regs = devm_ioremap(&pdev->dev, res->start, resource_size(res));
+       if (!priv->regs) {
+               pr_err("%pfwP: failed map MMIO registers\n", fwnode);
+               return -ENOMEM;
+       }
+
+       /*
+        * Find out GSI base number
+        *
+        * Note: DT does not define "riscv,gsi-base" property so GSI
+        * base is always zero for DT.
+        */
+       rc = fwnode_property_read_u32_array(fwnode, "riscv,gsi-base",
+                                           &priv->gsi_base, 1);
+       if (rc)
+               priv->gsi_base = 0;
+
+       /* Find out number of interrupt sources */
+       rc = fwnode_property_read_u32_array(fwnode, "riscv,num-sources",
+                                           &priv->nr_irqs, 1);
+       if (rc) {
+               pr_err("%pfwP: failed to get number of interrupt sources\n",
+                       fwnode);
+               return rc;
+       }
+
+       /* Setup initial state APLIC interrupts */
+       aplic_init_hw_irqs(priv);
+
+       /*
+        * Find out number of IDCs based on parent interrupts
+        *
+        * If "msi-parent" property is present then we ignore the
+        * APLIC IDCs which forces the APLIC driver to use MSI mode.
+        */
+       if (!fwnode_property_present(fwnode, "msi-parent")) {
+               while (!fwnode_property_get_reference_args(fwnode,
+                               "interrupts-extended", "#interrupt-cells",
+                               0, priv->nr_idcs, &parent))
+                       priv->nr_idcs++;
+       }
+
+       /* Setup IDCs or MSIs based on number of IDCs */
+       if (priv->nr_idcs)
+               rc = aplic_setup_idc(priv);
+       else
+               rc = aplic_setup_msi(priv);
+       if (rc) {
+               pr_err("%pfwP: failed setup %s\n",
+                       fwnode, priv->nr_idcs ? "IDCs" : "MSIs");
+               return rc;
+       }
+
+       /* Setup global config and interrupt delivery */
+       aplic_init_hw_global(priv);
+
+       /* Create irq domain instance for the APLIC */
+       if (priv->nr_idcs)
+               priv->irqdomain = irq_domain_create_linear(
+                                               priv->fwnode,
+                                               priv->nr_irqs + 1,
+                                               &aplic_irqdomain_idc_ops,
+                                               priv);
+       else
+               priv->irqdomain = platform_msi_create_device_domain(
+                                               &pdev->dev,
+                                               priv->nr_irqs + 1,
+                                               aplic_msi_write_msg,
+                                               &aplic_irqdomain_msi_ops,
+                                               priv);
+       if (!priv->irqdomain) {
+               pr_err("%pfwP: failed to add irq domain\n", priv->fwnode);
+               return -ENOMEM;
+       }
+
+       /* Advertise the interrupt controller */
+       if (priv->nr_idcs) {
+               pr_info("%pfwP: %d interrupts directly connected to %d CPUs\n",
+                       priv->fwnode, priv->nr_irqs, priv->nr_idcs);
+       } else {
+               pa = priv->msicfg.base_ppn << APLIC_xMSICFGADDR_PPN_SHIFT;
+               pr_info("%pfwP: %d interrupts forwared to MSI base %pa\n",
+                       priv->fwnode, priv->nr_irqs, &pa);
+       }
+
+       return 0;
+}
+
+static const struct of_device_id aplic_match[] = {
+       { .compatible = "riscv,aplic" },
+       {}
+};
+
+static struct platform_driver aplic_driver = {
+       .driver = {
+               .name           = "riscv-aplic",
+               .of_match_table = aplic_match,
+       },
+       .probe = aplic_probe,
+};
+
+static int __init aplic_init(void)
+{
+       /*
+        * Register APLIC driver as early as possible so that APLIC
+        * platform device is probed as soon as it is created.
+        */
+       return platform_driver_register(&aplic_driver);
+}
+core_initcall(aplic_init);
+
+static int __init aplic_dt_init(struct device_node *node,
+                               struct device_node *parent)
+{
+       /*
+        * The APLIC platform driver needs to be probed early

drivers/irqchip/irq-riscv-aplic.c line 770

Quoted 10 lines
drivers/irqchip/irq-riscv-aplic.c new file +10 -0
+        * so for device tree:
+        *
+        * 1) Set the FWNODE_FLAG_BEST_EFFORT flag in fwnode which
+        *    provides a hint to the device driver core to probe the
+        *    platform driver early.
+        * 2) Clear the OF_POPULATED flag in device_node because
+        *    of_irq_init() sets it which prevents creation of
+        *    platform device.
+        */
+       node->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;

Please stop spamming us with broken patches. Already told you this is not an option.

Nack.

-Saravana

Quoted 132 lines
drivers/irqchip/irq-riscv-aplic.c new file +4 -0
+       of_node_clear_flag(node, OF_POPULATED);
+       return 0;
+}
+IRQCHIP_DECLARE(riscv_aplic, "riscv,aplic", aplic_dt_init);
include/linux/irqchip/riscv-aplic.h new file +119 -1
new file mode 100644
index 000000000000..97e198ea0109
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2021 Western Digital Corporation or its affiliates.
+ * Copyright (C) 2022 Ventana Micro Systems Inc.
+ */
+#ifndef __LINUX_IRQCHIP_RISCV_APLIC_H
+#define __LINUX_IRQCHIP_RISCV_APLIC_H
+
+#include <linux/bitops.h>
+
+#define APLIC_MAX_IDC                  BIT(14)
+#define APLIC_MAX_SOURCE               1024
+
+#define APLIC_DOMAINCFG                        0x0000
+#define APLIC_DOMAINCFG_RDONLY         0x80000000
+#define APLIC_DOMAINCFG_IE             BIT(8)
+#define APLIC_DOMAINCFG_DM             BIT(2)
+#define APLIC_DOMAINCFG_BE             BIT(0)
+
+#define APLIC_SOURCECFG_BASE           0x0004
+#define APLIC_SOURCECFG_D              BIT(10)
+#define APLIC_SOURCECFG_CHILDIDX_MASK  0x000003ff
+#define APLIC_SOURCECFG_SM_MASK        0x00000007
+#define APLIC_SOURCECFG_SM_INACTIVE    0x0
+#define APLIC_SOURCECFG_SM_DETACH      0x1
+#define APLIC_SOURCECFG_SM_EDGE_RISE   0x4
+#define APLIC_SOURCECFG_SM_EDGE_FALL   0x5
+#define APLIC_SOURCECFG_SM_LEVEL_HIGH  0x6
+#define APLIC_SOURCECFG_SM_LEVEL_LOW   0x7
+
+#define APLIC_MMSICFGADDR              0x1bc0
+#define APLIC_MMSICFGADDRH             0x1bc4
+#define APLIC_SMSICFGADDR              0x1bc8
+#define APLIC_SMSICFGADDRH             0x1bcc
+
+#ifdef CONFIG_RISCV_M_MODE
+#define APLIC_xMSICFGADDR              APLIC_MMSICFGADDR
+#define APLIC_xMSICFGADDRH             APLIC_MMSICFGADDRH
+#else
+#define APLIC_xMSICFGADDR              APLIC_SMSICFGADDR
+#define APLIC_xMSICFGADDRH             APLIC_SMSICFGADDRH
+#endif
+
+#define APLIC_xMSICFGADDRH_L           BIT(31)
+#define APLIC_xMSICFGADDRH_HHXS_MASK   0x1f
+#define APLIC_xMSICFGADDRH_HHXS_SHIFT  24
+#define APLIC_xMSICFGADDRH_LHXS_MASK   0x7
+#define APLIC_xMSICFGADDRH_LHXS_SHIFT  20
+#define APLIC_xMSICFGADDRH_HHXW_MASK   0x7
+#define APLIC_xMSICFGADDRH_HHXW_SHIFT  16
+#define APLIC_xMSICFGADDRH_LHXW_MASK   0xf
+#define APLIC_xMSICFGADDRH_LHXW_SHIFT  12
+#define APLIC_xMSICFGADDRH_BAPPN_MASK  0xfff
+
+#define APLIC_xMSICFGADDR_PPN_SHIFT    12
+
+#define APLIC_xMSICFGADDR_PPN_HART(__lhxs) \
+       (BIT(__lhxs) - 1)
+
+#define APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) \
+       (BIT(__lhxw) - 1)
+#define APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs) \
+       ((__lhxs))
+#define APLIC_xMSICFGADDR_PPN_LHX(__lhxw, __lhxs) \
+       (APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) << \
+        APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs))
+
+#define APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) \
+       (BIT(__hhxw) - 1)
+#define APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs) \
+       ((__hhxs) + APLIC_xMSICFGADDR_PPN_SHIFT)
+#define APLIC_xMSICFGADDR_PPN_HHX(__hhxw, __hhxs) \
+       (APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) << \
+        APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs))
+
+#define APLIC_IRQBITS_PER_REG          32
+
+#define APLIC_SETIP_BASE               0x1c00
+#define APLIC_SETIPNUM                 0x1cdc
+
+#define APLIC_CLRIP_BASE               0x1d00
+#define APLIC_CLRIPNUM                 0x1ddc
+
+#define APLIC_SETIE_BASE               0x1e00
+#define APLIC_SETIENUM                 0x1edc
+
+#define APLIC_CLRIE_BASE               0x1f00
+#define APLIC_CLRIENUM                 0x1fdc
+
+#define APLIC_SETIPNUM_LE              0x2000
+#define APLIC_SETIPNUM_BE              0x2004
+
+#define APLIC_GENMSI                   0x3000
+
+#define APLIC_TARGET_BASE              0x3004
+#define APLIC_TARGET_HART_IDX_SHIFT    18
+#define APLIC_TARGET_HART_IDX_MASK     0x3fff
+#define APLIC_TARGET_GUEST_IDX_SHIFT   12
+#define APLIC_TARGET_GUEST_IDX_MASK    0x3f
+#define APLIC_TARGET_IPRIO_MASK        0xff
+#define APLIC_TARGET_EIID_MASK 0x7ff
+
+#define APLIC_IDC_BASE                 0x4000
+#define APLIC_IDC_SIZE                 32
+
+#define APLIC_IDC_IDELIVERY            0x00
+
+#define APLIC_IDC_IFORCE               0x04
+
+#define APLIC_IDC_ITHRESHOLD           0x08
+
+#define APLIC_IDC_TOPI                 0x18
+#define APLIC_IDC_TOPI_ID_SHIFT        16
+#define APLIC_IDC_TOPI_ID_MASK 0x3ff
+#define APLIC_IDC_TOPI_PRIO_MASK       0xff
+
+#define APLIC_IDC_CLAIMI               0x1c
+
+#endif
--

2.34.1

#13 lore raw

Re: [PATCH v5 7/9] irqchip: Add RISC-V advanced PLIC driver

Anup,

Quoted 15 lines · Saravana Kannan
Quoted 13 lines · Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new interrupt controller for managing wired interrupts on a RISC-V platform. This new interrupt controller is referred to as advanced platform-level interrupt controller (APLIC) which can forward wired interrupts to CPUs (or HARTs) as local interrupts OR as message signaled interrupts.
(For more details refer https://github.com/riscv/riscv-aia)

This patch adds an irqchip driver for RISC-V APLIC found on RISC-V platforms.

[...]

Quoted 19 lines
Quoted 5 lines
drivers/irqchip/irq-riscv-aplic.c new file +5 -0
+static int __init aplic_dt_init(struct device_node *node,
+                               struct device_node *parent)
+{
+       /*
+        * The APLIC platform driver needs to be probed early

drivers/irqchip/irq-riscv-aplic.c line 770

Quoted 10 lines
drivers/irqchip/irq-riscv-aplic.c new file +10 -0
+        * so for device tree:
+        *
+        * 1) Set the FWNODE_FLAG_BEST_EFFORT flag in fwnode which
+        *    provides a hint to the device driver core to probe the
+        *    platform driver early.
+        * 2) Clear the OF_POPULATED flag in device_node because
+        *    of_irq_init() sets it which prevents creation of
+        *    platform device.
+        */
+       node->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;

Please stop spamming us with broken patches. Already told you this is not an option.

Quoted 1 line

Nack.

What puzzles me here is that *no other arch* requires this sort of hack. What is so special about the APLIC that it requires it? I see nothing in this patch that even hints at it, despite the "discussion" in the last round.

The rules are simple:

- either the APLIC is so fundamental to the system that it has to be initialised super early, much like the GIC on arm64, at which point it cannot be a platform device, and the story is pretty simple.

- or it isn't that fundamental, and it can be probed as a platform device using the dependency infrastructure that is already used by multiple other interrupt controller drivers, without any need to mess with internal flags. Again, this should be simple enough.

If these rules don't apply to your stuff, please explain what is so different. And I mean actually explain the issue. Which isn't telling us "it doesn't work without it". Because as things stand, there is no way I will even consider taking this ugly mix of probing methods.

In any case, reposting the same stuff ad nauseam is only going to result in this series being ignored, which I don't think is what you want.

Thanks,

	M.
Signature · 6 lines
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
author
#14 lore raw

Re: [PATCH v5 7/9] irqchip: Add RISC-V advanced PLIC driver

Quoted 51 lines · Marc Zyngier

Anup,

Quoted 15 lines · Saravana Kannan
Quoted 13 lines · Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new interrupt controller for managing wired interrupts on a RISC-V platform. This new interrupt controller is referred to as advanced platform-level interrupt controller (APLIC) which can forward wired interrupts to CPUs (or HARTs) as local interrupts OR as message signaled interrupts.
(For more details refer https://github.com/riscv/riscv-aia)

This patch adds an irqchip driver for RISC-V APLIC found on RISC-V platforms.

[...]

Quoted 19 lines
Quoted 5 lines
drivers/irqchip/irq-riscv-aplic.c new file +5 -0
+static int __init aplic_dt_init(struct device_node *node,
+                               struct device_node *parent)
+{
+       /*
+        * The APLIC platform driver needs to be probed early

drivers/irqchip/irq-riscv-aplic.c line 770

Quoted 10 lines
drivers/irqchip/irq-riscv-aplic.c new file +10 -0
+        * so for device tree:
+        *
+        * 1) Set the FWNODE_FLAG_BEST_EFFORT flag in fwnode which
+        *    provides a hint to the device driver core to probe the
+        *    platform driver early.
+        * 2) Clear the OF_POPULATED flag in device_node because
+        *    of_irq_init() sets it which prevents creation of
+        *    platform device.
+        */
+       node->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;

Please stop spamming us with broken patches. Already told you this is not an option.

Quoted 1 line

Nack.

What puzzles me here is that *no other arch* requires this sort of hack. What is so special about the APLIC that it requires it? I see nothing in this patch that even hints at it, despite the "discussion" in the last round.

The rules are simple:

Quoted 8 lines

- either the APLIC is so fundamental to the system that it has to be initialised super early, much like the GIC on arm64, at which point it cannot be a platform device, and the story is pretty simple.

- or it isn't that fundamental, and it can be probed as a platform device using the dependency infrastructure that is already used by multiple other interrupt controller drivers, without any need to mess with internal flags. Again, this should be simple enough.

APLIC manages all wired interrupts whereas IMSIC manages all MSIs. Both APLIC and IMSIC are fundamental devices which need to be probed super early.

Now APLIC has two modes of operations:
1) Direct mode where there is no IMSIC in the system and APLIC directly injects interrupt to CPUs
2) MSI mode where IMSIC is present in the system and APLIC converts wired interrupts into MSIs

The APLIC driver added by this patch is a common driver for both above modes.

For #2, APLIC needs to be a platform device to create a device MSI domain using platform_msi_create_device_domain() which is why the APLIC driver is a platform driver.

Quoted 5 lines

If these rules don't apply to your stuff, please explain what is so different. And I mean actually explain the issue. Which isn't telling us "it doesn't work without it". Because as things stand, there is no way I will even consider taking this ugly mix of probing methods.

Yes, I don't want this ugly FWNODE_FLAG_BEST_EFFORT hack in this driver.

I tried several things but setting the FWNODE_FLAG_BEST_EFFORT flag is the only thing which works right now.

Quoted 11 lines

In any case, reposting the same stuff ad nauseam is only going to result in this series being ignored, which I don't think is what you want.

Thanks,

        M.

--
Without deviation from the norm, progress is not possible.

Regards,
Anup

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#15 lore raw

Re: [PATCH v5 7/9] irqchip: Add RISC-V advanced PLIC driver

Quoted 66 lines · Anup Patel
Quoted 51 lines · Marc Zyngier

Anup,

Quoted 15 lines · Saravana Kannan
Quoted 13 lines · Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new interrupt controller for managing wired interrupts on a RISC-V platform. This new interrupt controller is referred to as advanced platform-level interrupt controller (APLIC) which can forward wired interrupts to CPUs (or HARTs) as local interrupts OR as message signaled interrupts.
(For more details refer https://github.com/riscv/riscv-aia)

This patch adds an irqchip driver for RISC-V APLIC found on RISC-V platforms.

[...]

Quoted 19 lines
Quoted 5 lines
drivers/irqchip/irq-riscv-aplic.c new file +5 -0
+static int __init aplic_dt_init(struct device_node *node,
+                               struct device_node *parent)
+{
+       /*
+        * The APLIC platform driver needs to be probed early

drivers/irqchip/irq-riscv-aplic.c line 770

Quoted 10 lines
drivers/irqchip/irq-riscv-aplic.c new file +10 -0
+        * so for device tree:
+        *
+        * 1) Set the FWNODE_FLAG_BEST_EFFORT flag in fwnode which
+        *    provides a hint to the device driver core to probe the
+        *    platform driver early.
+        * 2) Clear the OF_POPULATED flag in device_node because
+        *    of_irq_init() sets it which prevents creation of
+        *    platform device.
+        */
+       node->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;

Please stop spamming us with broken patches. Already told you this is not an option.

Quoted 1 line

Nack.

What puzzles me here is that *no other arch* requires this sort of hack. What is so special about the APLIC that it requires it? I see nothing in this patch that even hints at it, despite the "discussion" in the last round.

The rules are simple:

Quoted 8 lines

- either the APLIC is so fundamental to the system that it has to be initialised super early, much like the GIC on arm64, at which point it cannot be a platform device, and the story is pretty simple.

- or it isn't that fundamental, and it can be probed as a platform device using the dependency infrastructure that is already used by multiple other interrupt controller drivers, without any need to mess with internal flags. Again, this should be simple enough.

APLIC manages all wired interrupts whereas IMSIC manages all MSIs. Both APLIC and IMSIC are fundamental devices which need to be probed super early.

Quoted 8 lines

Now APLIC has two modes of operations:
1) Direct mode where there is no IMSIC in the system and APLIC directly injects interrupt to CPUs
2) MSI mode where IMSIC is present in the system and APLIC converts wired interrupts into MSIs

The APLIC driver added by this patch is a common driver for both above modes.

Which it doesn't need to be. You are pointlessly making life difficult for yourself, and everyone else. The MSI bridge behaviour has *zero* reason to be the same driver as the main "I need it super early" driver. They may be called the same, but they *are* different things in the system.

They can share code, but they are fundamentally a different thing in the system. And I guess this silly approach has other ramifications:
the IMSIC is also some early driver when it really doesn't need to be. Who needs MSIs that early in the life of the system? I don't buy this for even a second.

Frankly, this whole thing needs to be taken apart and rebuilt from the ground up.

Quoted 3 lines

For #2, APLIC needs to be a platform device to create a device MSI domain using platform_msi_create_device_domain() which is why the APLIC driver is a platform driver.

You can't have your cake and eat it. If needed super early, and it cannot be a platform driver. End of the story.

And to my earlier point: IMSIC and APLIC-as-MSI-bridge have no purpose being early drivers. They must be platform drivers, and only that.

Quoted 7 lines
Quoted 4 lines

If these rules don't apply to your stuff, please explain what is so different. And I mean actually explain the issue. Which isn't telling us "it doesn't work without it". Because as things stand, there is no way I will even consider taking this ugly mix of probing methods.

Yes, I don't want this ugly FWNODE_FLAG_BEST_EFFORT hack in this driver.

And yet you are hammering it even when told this is wrong.

Quoted 2 lines

I tried several things but setting the FWNODE_FLAG_BEST_EFFORT flag is the only thing which works right now.

How about you take a step back and realise that the way you've architected your drivers makes little sense? I don't think you have tried *that*.

Thanks,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
maintainer
#16 lore raw

Re: [PATCH v5 7/9] irqchip: Add RISC-V advanced PLIC driver

Quoted 84 lines · Marc Zyngier
Quoted 66 lines · Anup Patel
Quoted 51 lines · Marc Zyngier

Anup,

Quoted 15 lines · Saravana Kannan
Quoted 13 lines · Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new interrupt controller for managing wired interrupts on a RISC-V platform. This new interrupt controller is referred to as advanced platform-level interrupt controller (APLIC) which can forward wired interrupts to CPUs (or HARTs) as local interrupts OR as message signaled interrupts.
(For more details refer https://github.com/riscv/riscv-aia)

This patch adds an irqchip driver for RISC-V APLIC found on RISC-V platforms.

[...]

Quoted 19 lines
Quoted 5 lines
drivers/irqchip/irq-riscv-aplic.c new file +5 -0
+static int __init aplic_dt_init(struct device_node *node,
+                               struct device_node *parent)
+{
+       /*
+        * The APLIC platform driver needs to be probed early

drivers/irqchip/irq-riscv-aplic.c line 770

Quoted 10 lines
drivers/irqchip/irq-riscv-aplic.c new file +10 -0
+        * so for device tree:
+        *
+        * 1) Set the FWNODE_FLAG_BEST_EFFORT flag in fwnode which
+        *    provides a hint to the device driver core to probe the
+        *    platform driver early.
+        * 2) Clear the OF_POPULATED flag in device_node because
+        *    of_irq_init() sets it which prevents creation of
+        *    platform device.
+        */
+       node->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;

Please stop spamming us with broken patches. Already told you this is not an option.

Quoted 1 line

Nack.

What puzzles me here is that *no other arch* requires this sort of hack. What is so special about the APLIC that it requires it? I see nothing in this patch that even hints at it, despite the "discussion" in the last round.

The rules are simple:

Quoted 8 lines

- either the APLIC is so fundamental to the system that it has to be initialised super early, much like the GIC on arm64, at which point it cannot be a platform device, and the story is pretty simple.

- or it isn't that fundamental, and it can be probed as a platform device using the dependency infrastructure that is already used by multiple other interrupt controller drivers, without any need to mess with internal flags. Again, this should be simple enough.

APLIC manages all wired interrupts whereas IMSIC manages all MSIs. Both APLIC and IMSIC are fundamental devices which need to be probed super early.

Quoted 8 lines

Now APLIC has two modes of operations:
1) Direct mode where there is no IMSIC in the system and APLIC directly injects interrupt to CPUs
2) MSI mode where IMSIC is present in the system and APLIC converts wired interrupts into MSIs

The APLIC driver added by this patch is a common driver for both above modes.

Which it doesn't need to be. You are pointlessly making life difficult for yourself, and everyone else. The MSI bridge behaviour has *zero* reason to be the same driver as the main "I need it super early" driver. They may be called the same, but they *are* different things in the system.

Quoted 5 lines

They can share code, but they are fundamentally a different thing in the system. And I guess this silly approach has other ramifications:
the IMSIC is also some early driver when it really doesn't need to be. Who needs MSIs that early in the life of the system? I don't buy this for even a second.

IMSIC also provides IPIs which are required super early so I think we can't make IMSIC as a platform driver.

Quoted 8 lines

Frankly, this whole thing needs to be taken apart and rebuilt from the ground up.

Quoted 3 lines

For #2, APLIC needs to be a platform device to create a device MSI domain using platform_msi_create_device_domain() which is why the APLIC driver is a platform driver.

Quoted 5 lines

You can't have your cake and eat it. If needed super early, and it cannot be a platform driver. End of the story.

And to my earlier point: IMSIC and APLIC-as-MSI-bridge have no purpose being early drivers. They must be platform drivers, and only that.

We can have IMSIC and APLIC-Direct-Mode as non-platform driver whereas APLIC-as-MSI-bridge will be a platform driver.

Both APLIC-Direct-Mode and APLIC-as-MSI-bridge can share a large part of the current driver.

Quoted 9 lines
Quoted 8 lines
Quoted 5 lines

If these rules don't apply to your stuff, please explain what is so different. And I mean actually explain the issue. Which isn't telling us "it doesn't work without it". Because as things stand, there is no way I will even consider taking this ugly mix of probing methods.

Yes, I don't want this ugly FWNODE_FLAG_BEST_EFFORT hack in this driver.

Quoted 8 lines

And yet you are hammering it even when told this is wrong.

Quoted 2 lines

I tried several things but setting the FWNODE_FLAG_BEST_EFFORT flag is the only thing which works right now.

How about you take a step back and realise that the way you've architected your drivers makes little sense? I don't think you have tried *that*.

Both APLIC and IMSIC are separate devices as defined by the AIA spec.

There are three possible systems:
1) Systems with only APLIC (i.e. only wired interrupts)
2) Systems with only IMSIC (i.e. only MSIs)
3) Systems with both APLIC and IMSIC (i.e. both wired interrupts and MSIs)

To address the above, APLIC and IMSIC are separate drivers. I am okay with splitting the APLIC driver into two separate drivers .

Regards,
Anup

author
#3 lore raw

Re: [PATCH v5 1/9] RISC-V: Add riscv_fw_parent_hartid() function

In reply to Andrew Jones
Quoted 34 lines · Andrew Jones
Quoted 32 lines · Anup Patel

We add common riscv_fw_parent_hartid() which help device drivers to get parent hartid of the INTC (i.e. local interrupt controller) fwnode. This should work for both DT and ACPI.

---
 arch/riscv/include/asm/processor.h |  3 +++
 arch/riscv/kernel/cpu.c            | 16 ++++++++++++++++
 2 files changed, 19 insertions(+)
arch/riscv/include/asm/processor.h +3 -0
index c950a8d9edef..39dc23a18f88 100644
@@ -81,6 +81,9 @@ int riscv_of_processor_hartid(struct device_node *node, unsigned long *hartid);
 int riscv_early_of_processor_hartid(struct device_node *node, unsigned long *hartid);
 int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid);
 
+struct fwnode_handle;
+int riscv_fw_parent_hartid(struct fwnode_handle *node, unsigned long *hartid);
+
 extern void riscv_fill_hwcap(void);
 extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src);
 
arch/riscv/kernel/cpu.c +0 -0
index a2fc952318e9..9be9b3b1f333 100644
@@ -96,6 +96,22 @@ int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid)
      return -1;
 }
 
Quoted 8 lines

arch/riscv/kernel/cpu.c int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid) · line 99

Quoted 1 line
arch/riscv/kernel/cpu.c +1 -0
+/* Find hart ID of the CPU fwnode under which given fwnode falls. */

This comment matches the comment for riscv_of_parent_hartid(), but I don't think it will be correct for the !is_of_node(node) case since fwnode_property_read_u64_array() isn't obliged to walk up its tree. Looking ahead it appears riscv_fw_parent_hartid() is only called with the parent node, so we could just drop this function and use fwnode_property_read_u64_array() directly at the two call sites.

I think the function name riscv_fw_parent_hartid() is misleading. It should be riscv_get_intc_hartid() because it is retrieving hartid based on INTC fwnode.

Currently, this function is used in APLIC and IMSIC drivers but soon it will be also used in PLIC driver with the upcoming PLIC ACPI support.

In fact, this patch should also replace riscv_of_parent_hartid() with riscv_get_intc_hartid() in INTC and PLIC.

Regards,
Anup

Quoted 25 lines

Thanks,
drew

Quoted 21 lines
arch/riscv/kernel/cpu.c +15 -1
+int riscv_fw_parent_hartid(struct fwnode_handle *node, unsigned long *hartid)
+{
+     int rc;
+     u64 temp;
+
+     if (!is_of_node(node)) {
+             rc = fwnode_property_read_u64_array(node, "hartid", &temp, 1);
+             if (!rc)
+                     *hartid = temp;
+     } else
+             rc = riscv_of_parent_hartid(to_of_node(node), hartid);
+
+     return rc;
+}
+
 DEFINE_PER_CPU(struct riscv_cpuinfo, riscv_cpuinfo);
 
 unsigned long riscv_cached_mvendorid(unsigned int cpu_id)
--

2.34.1

author
#6 lore raw

Re: [PATCH v5 2/9] irqchip/riscv-intc: Add support for RISC-V AIA

In reply to Andrew Jones
Quoted 11 lines · Andrew Jones
Quoted 7 lines · Anup Patel

The RISC-V advanced interrupt architecture (AIA) extends the per-HART local interrupts in following ways:
1. Minimum 64 local interrupts for both RV32 and RV64
2. Ability to process multiple pending local interrupts in same interrupt handler
3. Priority configuration for each local interrupts
4. Special CSRs to configure/access the per-HART MSI controller

afaict, we're only doing (1) and (2) from this list in this patch.

Okay, I will update the commit description.

Quoted 49 lines
Quoted 38 lines

This patch adds support for RISC-V AIA in the RISC-V intc driver.

---
 drivers/irqchip/irq-riscv-intc.c | 36 ++++++++++++++++++++++++++------
 1 file changed, 30 insertions(+), 6 deletions(-)
drivers/irqchip/irq-riscv-intc.c +10 -0
index 4adeee1bc391..e235bf1708a4 100644
@@ -17,6 +17,7 @@
 #include <linux/module.h>
 #include <linux/of.h>
 #include <linux/smp.h>
+#include <asm/hwcap.h>
 
 static struct irq_domain *intc_domain;
 
@@ -30,6 +31,15 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
      generic_handle_domain_irq(intc_domain, cause);
 }
 
+static asmlinkage void riscv_intc_aia_irq(struct pt_regs *regs)
+{
+     unsigned long topi;
+
+     while ((topi = csr_read(CSR_TOPI)))
+             generic_handle_domain_irq(intc_domain,
+                                       topi >> TOPI_IID_SHIFT);
+}
+
 /*
  * On RISC-V systems local interrupts are masked or unmasked by writing
  * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be written

drivers/irqchip/irq-riscv-intc.c static asmlinkage void riscv_intc_irq(struct pt_regs *regs) · line 55

Quoted 8 lines
drivers/irqchip/irq-riscv-intc.c +4 -1
@@ -39,12 +49,18 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 
 static void riscv_intc_irq_mask(struct irq_data *d)
 {
-     csr_clear(CSR_IE, BIT(d->hwirq));
+     if (d->hwirq < BITS_PER_LONG)
+             csr_clear(CSR_IE, BIT(d->hwirq));
+     else
+             csr_clear(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));

We can optimize rv64 by allowing the compiler to remove the branch

Quoted 5 lines

if (IS_ENABLED(CONFIG_32BIT) && d->hwirq >= 32)

    csr_clear(CSR_IEH, BIT(d->hwirq - 32));
 else
    csr_clear(CSR_IE, BIT(d->hwirq));

Makes sense, I will update.

Quoted 11 lines

drivers/irqchip/irq-riscv-intc.c static asmlinkage void riscv_intc_irq(struct pt_regs *regs) · line 63

Quoted 10 lines
drivers/irqchip/irq-riscv-intc.c +4 -1
 }
 
 static void riscv_intc_irq_unmask(struct irq_data *d)
 {
-     csr_set(CSR_IE, BIT(d->hwirq));
+     if (d->hwirq < BITS_PER_LONG)
+             csr_set(CSR_IE, BIT(d->hwirq));
+     else
+             csr_set(CSR_IEH, BIT(d->hwirq - BITS_PER_LONG));
Quoted 1 line

Same comment as above.

Okay, I will update.

Quoted 17 lines

drivers/irqchip/irq-riscv-intc.c static struct fwnode_handle *riscv_intc_hwnode(void) · line 137

Quoted 13 lines
drivers/irqchip/irq-riscv-intc.c +4 -1
 }
 
 static void riscv_intc_irq_eoi(struct irq_data *d)
@@ -115,16 +131,22 @@ static struct fwnode_handle *riscv_intc_hwnode(void)
 
 static int __init riscv_intc_init_common(struct fwnode_handle *fn)
 {
-     int rc;
+     int rc, nr_irqs = BITS_PER_LONG;
+
+     if (riscv_isa_extension_available(NULL, SxAIA) && BITS_PER_LONG == 32)
+             nr_irqs = nr_irqs * 2;

The AIA spec states sie and sip are explicitly 64, so how about writing this as

Quoted 4 lines

int rc, nr_irqs = BITS_PER_LONG;

if (riscv_isa_extension_available(NULL, SxAIA))

     nr_irqs = 64;

Okay, I will update.

Quoted 10 lines
Quoted 10 lines
drivers/irqchip/irq-riscv-intc.c +1 -1
-     intc_domain = irq_domain_create_linear(fn, BITS_PER_LONG,
+     intc_domain = irq_domain_create_linear(fn, nr_irqs,
                                             &riscv_intc_domain_ops, NULL);
      if (!intc_domain) {
              pr_err("unable to add IRQ domain\n");
              return -ENXIO;
      }
 
Quoted 7 lines

drivers/irqchip/irq-riscv-intc.c static struct fwnode_handle *riscv_intc_hwnode(void) · line 149

Quoted 5 lines
drivers/irqchip/irq-riscv-intc.c +4 -1
-     rc = set_handle_irq(&riscv_intc_irq);
+     if (riscv_isa_extension_available(NULL, SxAIA))
+             rc = set_handle_irq(&riscv_intc_aia_irq);
+     else
+             rc = set_handle_irq(&riscv_intc_irq);

nit: blank line here

I prefer no blank line here because the "if (rc)" below checks for errors for the above calls.

Quoted 6 lines
Quoted 6 lines
drivers/irqchip/irq-riscv-intc.c +0 -0
      if (rc) {
              pr_err("failed to set irq handler\n");
              return rc;
@@ -132,7 +154,9 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
 
Quoted 7 lines

drivers/irqchip/irq-riscv-intc.c static int __init riscv_intc_init_common(struct fwnode_handle *fn) · line 158

Quoted 5 lines
drivers/irqchip/irq-riscv-intc.c +2 -1
      riscv_set_intc_hwnode_fn(riscv_intc_hwnode);
 
-     pr_info("%d local interrupts mapped\n", BITS_PER_LONG);
+     pr_info("%d local interrupts mapped%s\n",
+             nr_irqs, (riscv_isa_extension_available(NULL, SxAIA)) ?

nit: unnecessary ()

Okay, I will update.

Quoted 11 lines
Quoted 8 lines
drivers/irqchip/irq-riscv-intc.c +1 -1
+                      " using AIA" : "");
 
      return 0;
 }
--

2.34.1

Thanks,
drew

Regards,
Anup

#17 lore raw

Re: [PATCH v5 7/9] irqchip: Add RISC-V advanced PLIC driver

In reply to Anup Patel
Quoted 86 lines · Anup Patel
Quoted 84 lines · Marc Zyngier
Quoted 66 lines · Anup Patel
Quoted 51 lines · Marc Zyngier

Anup,

Quoted 15 lines · Saravana Kannan
Quoted 13 lines · Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new interrupt controller for managing wired interrupts on a RISC-V platform. This new interrupt controller is referred to as advanced platform-level interrupt controller (APLIC) which can forward wired interrupts to CPUs (or HARTs) as local interrupts OR as message signaled interrupts.
(For more details refer https://github.com/riscv/riscv-aia)

This patch adds an irqchip driver for RISC-V APLIC found on RISC-V platforms.

[...]

Quoted 19 lines
Quoted 5 lines
drivers/irqchip/irq-riscv-aplic.c new file +5 -0
+static int __init aplic_dt_init(struct device_node *node,
+                               struct device_node *parent)
+{
+       /*
+        * The APLIC platform driver needs to be probed early

drivers/irqchip/irq-riscv-aplic.c line 770

Quoted 10 lines
drivers/irqchip/irq-riscv-aplic.c new file +10 -0
+        * so for device tree:
+        *
+        * 1) Set the FWNODE_FLAG_BEST_EFFORT flag in fwnode which
+        *    provides a hint to the device driver core to probe the
+        *    platform driver early.
+        * 2) Clear the OF_POPULATED flag in device_node because
+        *    of_irq_init() sets it which prevents creation of
+        *    platform device.
+        */
+       node->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;

Please stop spamming us with broken patches. Already told you this is not an option.

Quoted 1 line

Nack.

What puzzles me here is that *no other arch* requires this sort of hack. What is so special about the APLIC that it requires it? I see nothing in this patch that even hints at it, despite the "discussion" in the last round.

The rules are simple:

Quoted 8 lines

- either the APLIC is so fundamental to the system that it has to be initialised super early, much like the GIC on arm64, at which point it cannot be a platform device, and the story is pretty simple.

- or it isn't that fundamental, and it can be probed as a platform device using the dependency infrastructure that is already used by multiple other interrupt controller drivers, without any need to mess with internal flags. Again, this should be simple enough.

APLIC manages all wired interrupts whereas IMSIC manages all MSIs. Both APLIC and IMSIC are fundamental devices which need to be probed super early.

Quoted 8 lines

Now APLIC has two modes of operations:
1) Direct mode where there is no IMSIC in the system and APLIC directly injects interrupt to CPUs
2) MSI mode where IMSIC is present in the system and APLIC converts wired interrupts into MSIs

The APLIC driver added by this patch is a common driver for both above modes.

Which it doesn't need to be. You are pointlessly making life difficult for yourself, and everyone else. The MSI bridge behaviour has *zero* reason to be the same driver as the main "I need it super early" driver. They may be called the same, but they *are* different things in the system.

Quoted 8 lines
Quoted 5 lines

They can share code, but they are fundamentally a different thing in the system. And I guess this silly approach has other ramifications:
the IMSIC is also some early driver when it really doesn't need to be. Who needs MSIs that early in the life of the system? I don't buy this for even a second.

IMSIC also provides IPIs which are required super early so I think we can't make IMSIC as a platform driver.

Then split this part further. Just because the architecture lumps two completely unrelated concepts together doesn't mean we need to follow the same organisation.

Quoted 39 lines
Quoted 9 lines

Frankly, this whole thing needs to be taken apart and rebuilt from the ground up.

Quoted 3 lines

For #2, APLIC needs to be a platform device to create a device MSI domain using platform_msi_create_device_domain() which is why the APLIC driver is a platform driver.

Quoted 5 lines

You can't have your cake and eat it. If needed super early, and it cannot be a platform driver. End of the story.

And to my earlier point: IMSIC and APLIC-as-MSI-bridge have no purpose being early drivers. They must be platform drivers, and only that.

We can have IMSIC and APLIC-Direct-Mode as non-platform driver whereas APLIC-as-MSI-bridge will be a platform driver.

Both APLIC-Direct-Mode and APLIC-as-MSI-bridge can share a large part of the current driver.

Quoted 9 lines
Quoted 8 lines
Quoted 5 lines

If these rules don't apply to your stuff, please explain what is so different. And I mean actually explain the issue. Which isn't telling us "it doesn't work without it". Because as things stand, there is no way I will even consider taking this ugly mix of probing methods.

Yes, I don't want this ugly FWNODE_FLAG_BEST_EFFORT hack in this driver.

Quoted 8 lines

And yet you are hammering it even when told this is wrong.

Quoted 2 lines

I tried several things but setting the FWNODE_FLAG_BEST_EFFORT flag is the only thing which works right now.

How about you take a step back and realise that the way you've architected your drivers makes little sense? I don't think you have tried *that*.

Quoted 5 lines

Both APLIC and IMSIC are separate devices as defined by the AIA spec.

There are three possible systems:
1) Systems with only APLIC (i.e. only wired interrupts)
2) Systems with only IMSIC (i.e. only MSIs)

How is that possible? Are you saying that even things like timers are firing as MSIs?

Quoted 4 lines

3) Systems with both APLIC and IMSIC (i.e. both wired interrupts and MSIs)

To address the above, APLIC and IMSIC are separate drivers. I am okay with splitting the APLIC driver into two separate drivers .

Again, we don't have to follow the split established by the architecture. Instead, we should follow what is *functionally correct* for the kernel. If we were to write Risc-V-OS, that'd be an acceptable solution. But this is Linux, and the constraints are different.

My take on this discussion is that we should have:

- Direct-mode APLIC + IPI support as a an early irqchip driver

- MSI-bridge APLIC + MSI support as platform driver

Yes, these will likely share most of their code. But at least the split will be manageable, and will avoid ugly hacks.

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
author
#18 lore raw

Re: [PATCH v5 7/9] irqchip: Add RISC-V advanced PLIC driver

Quoted 95 lines · Marc Zyngier
Quoted 92 lines · Anup Patel
Quoted 89 lines · Marc Zyngier
Quoted 66 lines · Anup Patel
Quoted 51 lines · Marc Zyngier

Anup,

Quoted 15 lines · Saravana Kannan
Quoted 13 lines · Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new interrupt controller for managing wired interrupts on a RISC-V platform. This new interrupt controller is referred to as advanced platform-level interrupt controller (APLIC) which can forward wired interrupts to CPUs (or HARTs) as local interrupts OR as message signaled interrupts.
(For more details refer https://github.com/riscv/riscv-aia)

This patch adds an irqchip driver for RISC-V APLIC found on RISC-V platforms.

[...]

Quoted 19 lines
Quoted 5 lines
(quoted diff) +5 -0
+static int __init aplic_dt_init(struct device_node *node,
+                               struct device_node *parent)
+{
+       /*
+        * The APLIC platform driver needs to be probed early

Review

Quoted 10 lines
(quoted diff) +10 -0
+        * so for device tree:
+        *
+        * 1) Set the FWNODE_FLAG_BEST_EFFORT flag in fwnode which
+        *    provides a hint to the device driver core to probe the
+        *    platform driver early.
+        * 2) Clear the OF_POPULATED flag in device_node because
+        *    of_irq_init() sets it which prevents creation of
+        *    platform device.
+        */
+       node->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;

Please stop spamming us with broken patches. Already told you this is not an option.

Quoted 1 line

Nack.

What puzzles me here is that *no other arch* requires this sort of hack. What is so special about the APLIC that it requires it? I see nothing in this patch that even hints at it, despite the "discussion" in the last round.

The rules are simple:

Quoted 8 lines

- either the APLIC is so fundamental to the system that it has to be initialised super early, much like the GIC on arm64, at which point it cannot be a platform device, and the story is pretty simple.

- or it isn't that fundamental, and it can be probed as a platform device using the dependency infrastructure that is already used by multiple other interrupt controller drivers, without any need to mess with internal flags. Again, this should be simple enough.

APLIC manages all wired interrupts whereas IMSIC manages all MSIs. Both APLIC and IMSIC are fundamental devices which need to be probed super early.

Quoted 8 lines

Now APLIC has two modes of operations:
1) Direct mode where there is no IMSIC in the system and APLIC directly injects interrupt to CPUs
2) MSI mode where IMSIC is present in the system and APLIC converts wired interrupts into MSIs

The APLIC driver added by this patch is a common driver for both above modes.

Which it doesn't need to be. You are pointlessly making life difficult for yourself, and everyone else. The MSI bridge behaviour has *zero* reason to be the same driver as the main "I need it super early" driver. They may be called the same, but they *are* different things in the system.

They can share code, but they are fundamentally a different thing in the system. And I guess this silly approach has other ramifications:
the IMSIC is also some early driver when it really doesn't need to be. Who needs MSIs that early in the life of the system? I don't buy this for even a second.

Quoted 6 lines
Quoted 2 lines

IMSIC also provides IPIs which are required super early so I think we can't make IMSIC as a platform driver.

Then split this part further. Just because the architecture lumps two completely unrelated concepts together doesn't mean we need to follow the same organisation.

IPIs are supported as software injected MSIs. Basically, each HART has its own MSI controller and one HART can inject IPI to other HART by writing its MSI MMIO register.

Quoted 40 lines
Quoted 40 lines
Quoted 10 lines

Frankly, this whole thing needs to be taken apart and rebuilt from the ground up.

Quoted 3 lines

For #2, APLIC needs to be a platform device to create a device MSI domain using platform_msi_create_device_domain() which is why the APLIC driver is a platform driver.

Quoted 5 lines

You can't have your cake and eat it. If needed super early, and it cannot be a platform driver. End of the story.

And to my earlier point: IMSIC and APLIC-as-MSI-bridge have no purpose being early drivers. They must be platform drivers, and only that.

We can have IMSIC and APLIC-Direct-Mode as non-platform driver whereas APLIC-as-MSI-bridge will be a platform driver.

Both APLIC-Direct-Mode and APLIC-as-MSI-bridge can share a large part of the current driver.

Quoted 17 lines
Quoted 8 lines
Quoted 5 lines

If these rules don't apply to your stuff, please explain what is so different. And I mean actually explain the issue. Which isn't telling us "it doesn't work without it". Because as things stand, there is no way I will even consider taking this ugly mix of probing methods.

Yes, I don't want this ugly FWNODE_FLAG_BEST_EFFORT hack in this driver.

And yet you are hammering it even when told this is wrong.

Quoted 2 lines

I tried several things but setting the FWNODE_FLAG_BEST_EFFORT flag is the only thing which works right now.

How about you take a step back and realise that the way you've architected your drivers makes little sense? I don't think you have tried *that*.

Quoted 8 lines
Quoted 5 lines

Both APLIC and IMSIC are separate devices as defined by the AIA spec.

There are three possible systems:
1) Systems with only APLIC (i.e. only wired interrupts)
2) Systems with only IMSIC (i.e. only MSIs)

How is that possible? Are you saying that even things like timers are firing as MSIs?

No, timer interrupts are triggered through INTC.

In a full-blown AIA based system, we have three irqchips:
1) RISC-V INTC (per-HART local interrupts): This irqchip handles per-HART local interrupts such as timer, external interrupt, etc.
2) IMSIC (per-HART MSI controller): This is the per-HART MSI controller. Whenever an IMSIC of a HART receives MSI, it sets the local external interrupt of that HART.
3) APLIC (system-level wired interrupt): This is a system level interrupt controller for managing wired interrupt. In direct mode, APLIC will set a local external interrupt of that HART whereas in MSI mode, APLIC will do a MSI write to one of the per-HART IMSIC.

Quoted 11 lines
Quoted 5 lines

3) Systems with both APLIC and IMSIC (i.e. both wired interrupts and MSIs)

To address the above, APLIC and IMSIC are separate drivers. I am okay with splitting the APLIC driver into two separate drivers .

Again, we don't have to follow the split established by the architecture. Instead, we should follow what is *functionally correct* for the kernel. If we were to write Risc-V-OS, that'd be an acceptable solution. But this is Linux, and the constraints are different.

Quoted 8 lines

My take on this discussion is that we should have:

- Direct-mode APLIC + IPI support as a an early irqchip driver

- MSI-bridge APLIC + MSI support as platform driver

Yes, these will likely share most of their code. But at least the split will be manageable, and will avoid ugly hacks.

APLIC only manages wired interrupts and does not support IPIs.

For systems with only wired interrupts, APLIC is used in direct mode and IPI are triggered using SBI calls (firmware calls)

To summarize IPIs and device interrupts, here are the three combinations:
1) RISC-V systems with only wired interrupts: These systems will use APLIC in direct mode for wired interrupts and IPIs are injected using SBI calls.
2) RISC-V systems with only MSIs: These systems will use IMSICs for handling MSIs and IPIs are injected as software injected MSIs
3) RISC-V systems with both wired interrupts and MSIs: These systems use APLIC in MSI-mode for wired interrupts and IMSIC for MSIs. The IPIs are injected as software injected MSIs.

Regards,
Anup

author
#19 lore raw

Re: [PATCH v5 7/9] irqchip: Add RISC-V advanced PLIC driver

Quoted 196 lines · Anup Patel
Quoted 95 lines · Marc Zyngier
Quoted 92 lines · Anup Patel
Quoted 89 lines · Marc Zyngier
Quoted 66 lines · Anup Patel
Quoted 51 lines · Marc Zyngier

Anup,

Quoted 15 lines · Saravana Kannan
Quoted 13 lines · Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new interrupt controller for managing wired interrupts on a RISC-V platform. This new interrupt controller is referred to as advanced platform-level interrupt controller (APLIC) which can forward wired interrupts to CPUs (or HARTs) as local interrupts OR as message signaled interrupts.
(For more details refer https://github.com/riscv/riscv-aia)

This patch adds an irqchip driver for RISC-V APLIC found on RISC-V platforms.

[...]

Quoted 19 lines
Quoted 5 lines
(quoted diff) +5 -0
+static int __init aplic_dt_init(struct device_node *node,
+                               struct device_node *parent)
+{
+       /*
+        * The APLIC platform driver needs to be probed early

Review

Quoted 10 lines
(quoted diff) +10 -0
+        * so for device tree:
+        *
+        * 1) Set the FWNODE_FLAG_BEST_EFFORT flag in fwnode which
+        *    provides a hint to the device driver core to probe the
+        *    platform driver early.
+        * 2) Clear the OF_POPULATED flag in device_node because
+        *    of_irq_init() sets it which prevents creation of
+        *    platform device.
+        */
+       node->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;

Please stop spamming us with broken patches. Already told you this is not an option.

Quoted 1 line

Nack.

What puzzles me here is that *no other arch* requires this sort of hack. What is so special about the APLIC that it requires it? I see nothing in this patch that even hints at it, despite the "discussion" in the last round.

The rules are simple:

Quoted 8 lines

- either the APLIC is so fundamental to the system that it has to be initialised super early, much like the GIC on arm64, at which point it cannot be a platform device, and the story is pretty simple.

- or it isn't that fundamental, and it can be probed as a platform device using the dependency infrastructure that is already used by multiple other interrupt controller drivers, without any need to mess with internal flags. Again, this should be simple enough.

APLIC manages all wired interrupts whereas IMSIC manages all MSIs. Both APLIC and IMSIC are fundamental devices which need to be probed super early.

Quoted 8 lines

Now APLIC has two modes of operations:
1) Direct mode where there is no IMSIC in the system and APLIC directly injects interrupt to CPUs
2) MSI mode where IMSIC is present in the system and APLIC converts wired interrupts into MSIs

The APLIC driver added by this patch is a common driver for both above modes.

Which it doesn't need to be. You are pointlessly making life difficult for yourself, and everyone else. The MSI bridge behaviour has *zero* reason to be the same driver as the main "I need it super early" driver. They may be called the same, but they *are* different things in the system.

They can share code, but they are fundamentally a different thing in the system. And I guess this silly approach has other ramifications:
the IMSIC is also some early driver when it really doesn't need to be. Who needs MSIs that early in the life of the system? I don't buy this for even a second.

Quoted 6 lines
Quoted 2 lines

IMSIC also provides IPIs which are required super early so I think we can't make IMSIC as a platform driver.

Then split this part further. Just because the architecture lumps two completely unrelated concepts together doesn't mean we need to follow the same organisation.

IPIs are supported as software injected MSIs. Basically, each HART has its own MSI controller and one HART can inject IPI to other HART by writing its MSI MMIO register.

Quoted 40 lines
Quoted 40 lines
Quoted 10 lines

Frankly, this whole thing needs to be taken apart and rebuilt from the ground up.

Quoted 3 lines

For #2, APLIC needs to be a platform device to create a device MSI domain using platform_msi_create_device_domain() which is why the APLIC driver is a platform driver.

Quoted 5 lines

You can't have your cake and eat it. If needed super early, and it cannot be a platform driver. End of the story.

And to my earlier point: IMSIC and APLIC-as-MSI-bridge have no purpose being early drivers. They must be platform drivers, and only that.

We can have IMSIC and APLIC-Direct-Mode as non-platform driver whereas APLIC-as-MSI-bridge will be a platform driver.

Both APLIC-Direct-Mode and APLIC-as-MSI-bridge can share a large part of the current driver.

Quoted 17 lines
Quoted 8 lines
Quoted 5 lines

If these rules don't apply to your stuff, please explain what is so different. And I mean actually explain the issue. Which isn't telling us "it doesn't work without it". Because as things stand, there is no way I will even consider taking this ugly mix of probing methods.

Yes, I don't want this ugly FWNODE_FLAG_BEST_EFFORT hack in this driver.

And yet you are hammering it even when told this is wrong.

Quoted 2 lines

I tried several things but setting the FWNODE_FLAG_BEST_EFFORT flag is the only thing which works right now.

How about you take a step back and realise that the way you've architected your drivers makes little sense? I don't think you have tried *that*.

Quoted 8 lines
Quoted 5 lines

Both APLIC and IMSIC are separate devices as defined by the AIA spec.

There are three possible systems:
1) Systems with only APLIC (i.e. only wired interrupts)
2) Systems with only IMSIC (i.e. only MSIs)

How is that possible? Are you saying that even things like timers are firing as MSIs?

No, timer interrupts are triggered through INTC.

In a full-blown AIA based system, we have three irqchips:
1) RISC-V INTC (per-HART local interrupts): This irqchip handles per-HART local interrupts such as timer, external interrupt, etc.
2) IMSIC (per-HART MSI controller): This is the per-HART MSI controller. Whenever an IMSIC of a HART receives MSI, it sets the local external interrupt of that HART.
3) APLIC (system-level wired interrupt): This is a system level interrupt controller for managing wired interrupt. In direct mode, APLIC will set a local external interrupt of that HART whereas in MSI mode, APLIC will do a MSI write to one of the per-HART IMSIC.

Quoted 11 lines
Quoted 5 lines

3) Systems with both APLIC and IMSIC (i.e. both wired interrupts and MSIs)

To address the above, APLIC and IMSIC are separate drivers. I am okay with splitting the APLIC driver into two separate drivers .

Again, we don't have to follow the split established by the architecture. Instead, we should follow what is *functionally correct* for the kernel. If we were to write Risc-V-OS, that'd be an acceptable solution. But this is Linux, and the constraints are different.

Quoted 8 lines

My take on this discussion is that we should have:

- Direct-mode APLIC + IPI support as a an early irqchip driver

- MSI-bridge APLIC + MSI support as platform driver

Yes, these will likely share most of their code. But at least the split will be manageable, and will avoid ugly hacks.

APLIC only manages wired interrupts and does not support IPIs.

For systems with only wired interrupts, APLIC is used in direct mode and IPI are triggered using SBI calls (firmware calls)

Quoted 11 lines

To summarize IPIs and device interrupts, here are the three combinations:
1) RISC-V systems with only wired interrupts: These systems will use APLIC in direct mode for wired interrupts and IPIs are injected using SBI calls.
2) RISC-V systems with only MSIs: These systems will use IMSICs for handling MSIs and IPIs are injected as software injected MSIs
3) RISC-V systems with both wired interrupts and MSIs: These systems use APLIC in MSI-mode for wired interrupts and IMSIC for MSIs. The IPIs are injected as software injected MSIs.

I have created two ascii diagrams which might help understand the RISC-V AIA.

Systems with only wired interrupts
----------------------------------

----------

|  HART0 |
|        |
|  timer |
|    |   |
|    v   |
| ------ |
| |INTC|<-----------------------\
| ------ |                      |
----------                      |
     .                      ---------
     .                      |       |<------
     .                      | APLIC |<------ Device Wired Interrupts
     .                      |       |<------
     .                      ---------
----------                      |
|  HARTN |                      |
|        |                      |
|  timer |                      |
|    |   |                      |
|    v   |                      |
| ------ |                      |
| |INTC|<-----------------------/
| ------ |

----------

Systems with wired interrupts and MSIs
--------------------------------------

----------      /------------------- Device MSIs
|  HART0 |      |
|        |      |
|  timer |      |
|    |   |      |
|    v   |      v
| ------ |  ---------
| |INTC|<---| IMSIC |<----------\
| ------ |  ---------           |
----------                      |
     .                      ---------
     .                      |       |<------
     .                      | APLIC |<------ Device Wired Interrupts
     .                      |       |<------
     .                      ---------
----------                      |
|  HARTN |                      |
|        |                      |
|  timer |                      |
|    |   |                      |
|    v   |                      |
| ------ |  ---------           |
| |INTC|<---| IMSIC |<----------/
| ------ |  ---------
----------      ^
                |
                \------------------- Device MSIs

As shown in the figures, we are dealing with three different irqchip.

The INTC is inside the HART whereas we have one IMSIC next to each HART. The APLIC is a system-level irqchip.

Furthermore, AIA is a modular specification so platforms might implement only APLIC or only IMSICs or both APLIC+IMSIC.

Regards,
Anup

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#20 lore raw

Re: [PATCH v5 7/9] irqchip: Add RISC-V advanced PLIC driver

In reply to Anup Patel
Quoted 150 lines · Anup Patel
Quoted 95 lines · Marc Zyngier
Quoted 92 lines · Anup Patel
Quoted 89 lines · Marc Zyngier
Quoted 66 lines · Anup Patel
Quoted 51 lines · Marc Zyngier

Anup,

Quoted 15 lines · Saravana Kannan
Quoted 13 lines · Anup Patel

The RISC-V advanced interrupt architecture (AIA) specification defines a new interrupt controller for managing wired interrupts on a RISC-V platform. This new interrupt controller is referred to as advanced platform-level interrupt controller (APLIC) which can forward wired interrupts to CPUs (or HARTs) as local interrupts OR as message signaled interrupts.
(For more details refer https://github.com/riscv/riscv-aia)

This patch adds an irqchip driver for RISC-V APLIC found on RISC-V platforms.

[...]

Quoted 19 lines
Quoted 5 lines
(quoted diff) +5 -0
+static int __init aplic_dt_init(struct device_node *node,
+                               struct device_node *parent)
+{
+       /*
+        * The APLIC platform driver needs to be probed early

Review

Quoted 10 lines
(quoted diff) +10 -0
+        * so for device tree:
+        *
+        * 1) Set the FWNODE_FLAG_BEST_EFFORT flag in fwnode which
+        *    provides a hint to the device driver core to probe the
+        *    platform driver early.
+        * 2) Clear the OF_POPULATED flag in device_node because
+        *    of_irq_init() sets it which prevents creation of
+        *    platform device.
+        */
+       node->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;

Please stop spamming us with broken patches. Already told you this is not an option.

Quoted 1 line

Nack.

What puzzles me here is that *no other arch* requires this sort of hack. What is so special about the APLIC that it requires it? I see nothing in this patch that even hints at it, despite the "discussion" in the last round.

The rules are simple:

Quoted 8 lines

- either the APLIC is so fundamental to the system that it has to be initialised super early, much like the GIC on arm64, at which point it cannot be a platform device, and the story is pretty simple.

- or it isn't that fundamental, and it can be probed as a platform device using the dependency infrastructure that is already used by multiple other interrupt controller drivers, without any need to mess with internal flags. Again, this should be simple enough.

APLIC manages all wired interrupts whereas IMSIC manages all MSIs. Both APLIC and IMSIC are fundamental devices which need to be probed super early.

Quoted 8 lines

Now APLIC has two modes of operations:
1) Direct mode where there is no IMSIC in the system and APLIC directly injects interrupt to CPUs
2) MSI mode where IMSIC is present in the system and APLIC converts wired interrupts into MSIs

The APLIC driver added by this patch is a common driver for both above modes.

Which it doesn't need to be. You are pointlessly making life difficult for yourself, and everyone else. The MSI bridge behaviour has *zero* reason to be the same driver as the main "I need it super early" driver. They may be called the same, but they *are* different things in the system.

They can share code, but they are fundamentally a different thing in the system. And I guess this silly approach has other ramifications:
the IMSIC is also some early driver when it really doesn't need to be. Who needs MSIs that early in the life of the system? I don't buy this for even a second.

Quoted 6 lines
Quoted 2 lines

IMSIC also provides IPIs which are required super early so I think we can't make IMSIC as a platform driver.

Then split this part further. Just because the architecture lumps two completely unrelated concepts together doesn't mean we need to follow the same organisation.

IPIs are supported as software injected MSIs. Basically, each HART has its own MSI controller and one HART can inject IPI to other HART by writing its MSI MMIO register.

Quoted 42 lines
Quoted 42 lines
Quoted 10 lines

Frankly, this whole thing needs to be taken apart and rebuilt from the ground up.

Quoted 3 lines

For #2, APLIC needs to be a platform device to create a device MSI domain using platform_msi_create_device_domain() which is why the APLIC driver is a platform driver.

Quoted 5 lines

You can't have your cake and eat it. If needed super early, and it cannot be a platform driver. End of the story.

And to my earlier point: IMSIC and APLIC-as-MSI-bridge have no purpose being early drivers. They must be platform drivers, and only that.

We can have IMSIC and APLIC-Direct-Mode as non-platform driver whereas APLIC-as-MSI-bridge will be a platform driver.

Both APLIC-Direct-Mode and APLIC-as-MSI-bridge can share a large part of the current driver.

Quoted 9 lines
Quoted 8 lines
Quoted 5 lines

If these rules don't apply to your stuff, please explain what is so different. And I mean actually explain the issue. Which isn't telling us "it doesn't work without it". Because as things stand, there is no way I will even consider taking this ugly mix of probing methods.

Yes, I don't want this ugly FWNODE_FLAG_BEST_EFFORT hack in this driver.

Quoted 8 lines

And yet you are hammering it even when told this is wrong.

Quoted 2 lines

I tried several things but setting the FWNODE_FLAG_BEST_EFFORT flag is the only thing which works right now.

How about you take a step back and realise that the way you've architected your drivers makes little sense? I don't think you have tried *that*.

Both APLIC and IMSIC are separate devices as defined by the AIA spec.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

There are three possible systems:
1) Systems with only APLIC (i.e. only wired interrupts)
2) Systems with only IMSIC (i.e. only MSIs)

How is that possible? Are you saying that even things like timers are firing as MSIs?

No, timer interrupts are triggered through INTC.

So all the above is BS. All you need is timers and IPIs to be supported early. Everything else can be postponed until you probe the rest of the interrupt hierarchy.

	M.
Signature · 6 lines
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv