Linux RISC-V AIA Support
Subsystems (5)
v16 cover Linux RISC-V AIA Support
The RISC-V AIA specification is ratified as-per the RISC-V international process. The latest ratified AIA specifcation can be found at:
https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interrupts-1.0.pdf
At a high-level, the AIA specification adds three things:
1) AIA CSRs
- Improved local interrupt support
2) Incoming Message Signaled Interrupt Controller (IMSIC)
- Per-HART MSI controller
- Support MSI virtualization
- Support IPI along with virtualization
3) Advanced Platform-Level Interrupt Controller (APLIC)
- Wired interrupt controller
- In MSI-mode, converts wired interrupt into MSIs (i.e. MSI generator)
- In Direct-mode, injects external interrupts directly into HARTs
For an overview of the AIA specification, refer the AIA virtualization talk at KVM Forum 2022:
https://static.sched.com/hosted_files/kvmforum2022/a1/AIA_Virtualization_in_KVM_RISCV_final.pdf
https://www.youtube.com/watch?v=r071dL8Z0yo
To test this series, use QEMU v7.2 (or higher) and OpenSBI v1.2 (or higher).
This series depends upon per-device MSI domain (and few other) patches merged by Thomas (tglx) which are available in irq/msi branch at:git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
These patches can also be found in the riscv_aia_v16 branch at:
https://github.com/avpatel/linux.git
Changes since v15:
- Dropped PATCH1 since it is already merged by Thomas (tglx) and available in his irq/msi branch at git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
- Simplified aplic_msi_irq_eoi() in PATCH7 based on the section "4.9.2 Special consideration for level-sensitive interrupt sources" of the RISC-V AIA specification and also provided handler name.
Changes since v14:
- Dropped 9 patches which are already merged by Thomas (tglx) and available in his irq/msi branch at git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
- Added new PATCH1 which adds changes missed out in merging AIA support for RISC-V INTC.
- Added a separate cpuhp state for IMSIC driver in PATCH3 which ensures that cpuhp notifiers of IMSIC are called as early as possible.
- Removed redundant barriers in PATCH3.
- Addressed few other nit comments.
Changes since v13:
- Split PATCH1 into six granular patches
- Addressed nit comments from Thomas and Bjorn
Changes since v12:
- Rebased on Linux-6.8-rc5
- Dropped per-device MSI domain patches which are already merged by Thomas (tglx)
- Addressed nit comments from Thomas and Clement
- Added a new patch2 to fix lock dependency warning
- Replaced local sync IPI in the IMSIC driver with per-CPU timer
- Simplified locking in the IMSIC driver to avoid lock dependency issues
- Added a dirty bitmap in the IMSIC driver to optimize per-CPU local sync loop
Changes since v11:
- Rebased on Linux-6.8-rc1
- Included kernel/irq related patches from "genirq, irqchip: Convert ARM MSI handling to per device MSI domains" series by Thomas. (PATCH7, PATCH8, PATCH9, PATCH14, PATCH16, PATCH17, PATCH18, PATCH19,
PATCH20, PATCH21, PATCH22, PATCH23, and PATCH32 of
https://lore.kernel.org/linux-arm-kernel/[email protected]/) - Updated APLIC MSI-mode driver to use the new WIRED_TO_MSI mechanism.
- Updated IMSIC driver to support per-device MSI domains for PCI and platform devices.
Changes since v10:
- Rebased on Linux-6.6-rc7
- Dropped PATCH3 of v10 series since this has been merged by MarcZ for Linux-6.6-rc7
- Changed the IMSIC ID management strategy from 1-n approach to x86-style 1-1 approach
Changes since v9:
- Rebased on Linux-6.6-rc4
- Use builtin_platform_driver() in PATCH5, PATCH9, and PATCH12
Changes since v8:
- Rebased on Linux-6.6-rc3
- Dropped PATCH2 of v8 series since we won't be requiring riscv_get_intc_hartid() based on Marc Z's comments on ACPI AIA support.
- Addressed Saravana's comments in PATCH3 of v8 series
- Update PATCH9 and PATCH13 of v8 series based on comments from Sunil
Changes since v7:
- Rebased on Linux-6.6-rc1
- Addressed comments on PATCH1 of v7 series and split it into two PATCHes
- Use DEFINE_SIMPLE_PROP() in PATCH2 of v7 series
Changes since v6:
- Rebased on Linux-6.5-rc4
- Updated PATCH2 to use IS_ENABLED(CONFIG_SPARC) instead of !IS_ENABLED(CONFIG_OF_IRQ)
- Added new PATCH4 to fix syscore registration in PLIC driver
- Update PATCH5 to convert PLIC driver into full-blown platform driver with a re-written probe function.
Changes since v5:
- Rebased on Linux-6.5-rc2
- Updated the overall series to ensure that only IPI, timer, and INTC drivers are probed very early whereas rest of the interrupt controllers (such as PLIC, APLIC, and IMISC) are probed as regular platform drivers.
- Renamed riscv_fw_parent_hartid() to riscv_get_intc_hartid()
- New PATCH1 to add fw_devlink support for msi-parent DT property
- New PATCH2 to ensure all INTC suppliers are initialized which in-turn fixes the probing issue for PLIC, APLIC and IMSIC as platform driver
- New PATCH3 to use platform driver probing for PLIC
- Re-structured the IMSIC driver into two separate drivers: early and platform. The IMSIC early driver (PATCH7) only initialized IMSIC state and provides IPIs whereas the IMSIC platform driver (PATCH8) is probed provides MSI domain for platform devices.
- Re-structure the APLIC platform driver into three separe sources: main, direct mode, and MSI mode.
Changes since v4:
- Rebased on Linux-6.5-rc1
- Added "Dependencies" in the APLIC bindings (PATCH6 in v4)
- Dropped the PATCH6 which was changing the IOMMU DMA domain APIs
- Dropped use of IOMMU DMA APIs in the IMSIC driver (PATCH4)
Changes since v3:
- Rebased on Linux-6.4-rc6
- Dropped PATCH2 of v3 series instead we now set FWNODE_FLAG_BEST_EFFORT via IRQCHIP_DECLARE()
- Extend riscv_fw_parent_hartid() to support both DT and ACPI in PATCH1
- Extend iommu_dma_compose_msi_msg() instead of adding iommu_dma_select_msi() in PATCH6
- Addressed Conor's comments in PATCH3
- Addressed Conor's and Rob's comments in PATCH7
Changes since v2:
- Rebased on Linux-6.4-rc1
- Addressed Rob's comments on DT bindings patches 4 and 8.
- Addessed Marc's comments on IMSIC driver PATCH5
- Replaced use of OF apis in APLIC and IMSIC drivers with FWNODE apis this makes both drivers easily portable for ACPI support. This also removes unnecessary indirection from the APLIC and IMSIC drivers.
- PATCH1 is a new patch for portability with ACPI support
- PATCH2 is a new patch to fix probing in APLIC drivers for APLIC-only systems.
- PATCH7 is a new patch which addresses the IOMMU DMA domain issues pointed out by SiFive
Changes since v1:
- Rebased on Linux-6.2-rc2
- Addressed comments on IMSIC DT bindings for PATCH4
- Use raw_spin_lock_irqsave() on ids_lock for PATCH5
- Improved MMIO alignment checks in PATCH5 to allow MMIO regions with holes.
- Addressed comments on APLIC DT bindings for PATCH6
- Fixed warning splat in aplic_msi_write_msg() caused by zeroed MSI message in PATCH7
- Dropped DT property riscv,slow-ipi instead will have module parameter in future.
Anup Patel (9):
dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller
irqchip: Add RISC-V incoming MSI controller early driver irqchip/riscv-imsic: Add device MSI domain support for platform
devices
irqchip/riscv-imsic: Add device MSI domain support for PCI devices
dt-bindings: interrupt-controller: Add RISC-V advanced PLIC
irqchip: Add RISC-V advanced PLIC driver for direct-mode irqchip/riscv-aplic: Add support for MSI-mode
RISC-V: Select APLIC and IMSIC drivers
MAINTAINERS: Add entry for RISC-V AIA drivers
.../interrupt-controller/riscv,aplic.yaml | 172 ++++ .../interrupt-controller/riscv,imsics.yaml | 172 ++++ MAINTAINERS | 14 + arch/riscv/Kconfig | 2 + drivers/irqchip/Kconfig | 25 + drivers/irqchip/Makefile | 3 + drivers/irqchip/irq-riscv-aplic-direct.c | 326 +++++++ drivers/irqchip/irq-riscv-aplic-main.c | 211 +++++ drivers/irqchip/irq-riscv-aplic-main.h | 52 ++ drivers/irqchip/irq-riscv-aplic-msi.c | 257 ++++++ drivers/irqchip/irq-riscv-imsic-early.c | 201 ++++ drivers/irqchip/irq-riscv-imsic-platform.c | 374 ++++++++ drivers/irqchip/irq-riscv-imsic-state.c | 865 ++++++++++++++++++ drivers/irqchip/irq-riscv-imsic-state.h | 108 +++ include/linux/cpuhotplug.h | 1 + include/linux/irqchip/riscv-aplic.h | 145 +++ include/linux/irqchip/riscv-imsic.h | 87 ++ 17 files changed, 3015 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml create mode 100644 drivers/irqchip/irq-riscv-aplic-direct.c create mode 100644 drivers/irqchip/irq-riscv-aplic-main.c create mode 100644 drivers/irqchip/irq-riscv-aplic-main.h create mode 100644 drivers/irqchip/irq-riscv-aplic-msi.c create mode 100644 drivers/irqchip/irq-riscv-imsic-early.c create mode 100644 drivers/irqchip/irq-riscv-imsic-platform.c create mode 100644 drivers/irqchip/irq-riscv-imsic-state.c create mode 100644 drivers/irqchip/irq-riscv-imsic-state.h create mode 100644 include/linux/irqchip/riscv-aplic.h create mode 100644 include/linux/irqchip/riscv-imsic.h
Signature · 1 line
2.34.1
PATCH v16 1/9 dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller
Add DT bindings document for the RISC-V incoming MSI controller (IMSIC) defined by the RISC-V advanced interrupt architecture (AIA) specification.
--- .../interrupt-controller/riscv,imsics.yaml | 172 ++++++++++++++++++ 1 file changed, 172 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml
Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml new file +172 -0
@@ -0,0 +1,172 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/interrupt-controller/riscv,imsics.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: RISC-V Incoming MSI Controller (IMSIC) + +maintainers: + - Anup Patel <[email protected]> + +description: | + The RISC-V advanced interrupt architecture (AIA) defines a per-CPU incoming + MSI controller (IMSIC) for handling MSIs in a RISC-V platform. The RISC-V + AIA specification can be found at https://github.com/riscv/riscv-aia. + + The IMSIC is a per-CPU (or per-HART) device with separate interrupt file + for each privilege level (machine or supervisor). The configuration of + a IMSIC interrupt file is done using AIA CSRs and it also has a 4KB MMIO + space to receive MSIs from devices. Each IMSIC interrupt file supports a + fixed number of interrupt identities (to distinguish MSIs from devices) + which is same for given privilege level across CPUs (or HARTs). + + The device tree of a RISC-V platform will have one IMSIC device tree node + for each privilege level (machine or supervisor) which collectively describe + IMSIC interrupt files at that privilege level across CPUs (or HARTs). + + The arrangement of IMSIC interrupt files in MMIO space of a RISC-V platform + follows a particular scheme defined by the RISC-V AIA specification. A IMSIC + group is a set of IMSIC interrupt files co-located in MMIO space and we can + have multiple IMSIC groups (i.e. clusters, sockets, chiplets, etc) in a + RISC-V platform. The MSI target address of a IMSIC interrupt file at given + privilege level (machine or supervisor) encodes group index, HART index, + and guest index (shown below). + + XLEN-1 > (HART Index MSB) 12 0 + | | | | + ------------------------------------------------------------- + |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index| 0 | + ------------------------------------------------------------- + +allOf: + - $ref: /schemas/interrupt-controller.yaml# + - $ref: /schemas/interrupt-controller/msi-controller.yaml# + +properties: + compatible: + items: + - enum: + - qemu,imsics + - const: riscv,imsics + + reg: + minItems: 1 + maxItems: 16384 + description: + Base address of each IMSIC group. + + interrupt-controller: true + + "#interrupt-cells": + const: 0 + + msi-controller: true + + "#msi-cells": + const: 0 + + interrupts-extended: + minItems: 1 + maxItems: 16384 + description: + This property represents the set of CPUs (or HARTs) for which given + device tree node describes the IMSIC interrupt files. Each node pointed + to should be a riscv,cpu-intc node, which has a CPU node (i.e. RISC-V + HART) as parent. + + riscv,num-ids: + $ref: /schemas/types.yaml#/definitions/uint32 + minimum: 63 + maximum: 2047 + description: + Number of interrupt identities supported by IMSIC interrupt file. + + riscv,num-guest-ids: + $ref: /schemas/types.yaml#/definitions/uint32 + minimum: 63 + maximum: 2047 + description: + Number of interrupt identities are supported by IMSIC guest interrupt + file. When not specified it is assumed to be same as specified by the + riscv,num-ids property. + + riscv,guest-index-bits: + minimum: 0 + maximum: 7 + default: 0 + description: + Number of guest index bits in the MSI target address. + + riscv,hart-index-bits: + minimum: 0 + maximum: 15 + description: + Number of HART index bits in the MSI target address. When not + specified it is calculated based on the interrupts-extended property. + + riscv,group-index-bits: + minimum: 0 + maximum: 7 + default: 0 + description: + Number of group index bits in the MSI target address. + + riscv,group-index-shift: + $ref: /schemas/types.yaml#/definitions/uint32 + minimum: 0 + maximum: 55 + default: 24 + description: + The least significant bit position of the group index bits in the + MSI target address. + +required: + - compatible + - reg + - interrupt-controller + - msi-controller + - "#msi-cells" + - interrupts-extended + - riscv,num-ids + +unevaluatedProperties: false + +examples: + - | + // Example 1 (Machine-level IMSIC files with just one group): + + interrupt-controller@24000000 { + compatible = "qemu,imsics", "riscv,imsics"; + interrupts-extended = <&cpu1_intc 11>, + <&cpu2_intc 11>, + <&cpu3_intc 11>, + <&cpu4_intc 11>; + reg = <0x28000000 0x4000>; + interrupt-controller; + #interrupt-cells = <0>; + msi-controller; + #msi-cells = <0>; + riscv,num-ids = <127>; + }; + + - | + // Example 2 (Supervisor-level IMSIC files with two groups): + + interrupt-controller@28000000 { + compatible = "qemu,imsics", "riscv,imsics"; + interrupts-extended = <&cpu1_intc 9>, + <&cpu2_intc 9>, + <&cpu3_intc 9>, + <&cpu4_intc 9>; + reg = <0x28000000 0x2000>, /* Group0 IMSICs */ + <0x29000000 0x2000>; /* Group1 IMSICs */ + interrupt-controller; + #interrupt-cells = <0>; + msi-controller; + #msi-cells = <0>; + riscv,num-ids = <127>; + riscv,group-index-bits = <1>; + riscv,group-index-shift = <24>; + }; +...
Signature · 1 line
2.34.1
[tip: irq/core] dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 0151a8db49b0a88f967dca0ea5ae2bee2d67b22a Gitweb: https://git.kernel.org/tip/0151a8db49b0a88f967dca0ea5ae2bee2d67b22a Author: Anup Patel <[email protected]> AuthorDate: Thu, 07 Mar 2024 19:32:59 +05:30 Committer: Thomas Gleixner <[email protected]> CommitterDate: Mon, 25 Mar 2024 17:38:28 +01:00
dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller
Add DT bindings document for the RISC-V incoming MSI controller (IMSIC) defined by the RISC-V advanced interrupt architecture (AIA) specification.
--- Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 172 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml
Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml new file +172 -0
@@ -0,0 +1,172 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/interrupt-controller/riscv,imsics.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: RISC-V Incoming MSI Controller (IMSIC) + +maintainers: + - Anup Patel <[email protected]> + +description: | + The RISC-V advanced interrupt architecture (AIA) defines a per-CPU incoming + MSI controller (IMSIC) for handling MSIs in a RISC-V platform. The RISC-V + AIA specification can be found at https://github.com/riscv/riscv-aia. + + The IMSIC is a per-CPU (or per-HART) device with separate interrupt file + for each privilege level (machine or supervisor). The configuration of + a IMSIC interrupt file is done using AIA CSRs and it also has a 4KB MMIO + space to receive MSIs from devices. Each IMSIC interrupt file supports a + fixed number of interrupt identities (to distinguish MSIs from devices) + which is same for given privilege level across CPUs (or HARTs). + + The device tree of a RISC-V platform will have one IMSIC device tree node + for each privilege level (machine or supervisor) which collectively describe + IMSIC interrupt files at that privilege level across CPUs (or HARTs). + + The arrangement of IMSIC interrupt files in MMIO space of a RISC-V platform + follows a particular scheme defined by the RISC-V AIA specification. A IMSIC + group is a set of IMSIC interrupt files co-located in MMIO space and we can + have multiple IMSIC groups (i.e. clusters, sockets, chiplets, etc) in a + RISC-V platform. The MSI target address of a IMSIC interrupt file at given + privilege level (machine or supervisor) encodes group index, HART index, + and guest index (shown below). + + XLEN-1 > (HART Index MSB) 12 0 + | | | | + ------------------------------------------------------------- + |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index| 0 | + ------------------------------------------------------------- + +allOf: + - $ref: /schemas/interrupt-controller.yaml# + - $ref: /schemas/interrupt-controller/msi-controller.yaml# + +properties: + compatible: + items: + - enum: + - qemu,imsics + - const: riscv,imsics + + reg: + minItems: 1 + maxItems: 16384 + description: + Base address of each IMSIC group. + + interrupt-controller: true + + "#interrupt-cells": + const: 0 + + msi-controller: true + + "#msi-cells": + const: 0 + + interrupts-extended: + minItems: 1 + maxItems: 16384 + description: + This property represents the set of CPUs (or HARTs) for which given + device tree node describes the IMSIC interrupt files. Each node pointed + to should be a riscv,cpu-intc node, which has a CPU node (i.e. RISC-V + HART) as parent. + + riscv,num-ids: + $ref: /schemas/types.yaml#/definitions/uint32 + minimum: 63 + maximum: 2047 + description: + Number of interrupt identities supported by IMSIC interrupt file. + + riscv,num-guest-ids: + $ref: /schemas/types.yaml#/definitions/uint32 + minimum: 63 + maximum: 2047 + description: + Number of interrupt identities are supported by IMSIC guest interrupt + file. When not specified it is assumed to be same as specified by the + riscv,num-ids property. + + riscv,guest-index-bits: + minimum: 0 + maximum: 7 + default: 0 + description: + Number of guest index bits in the MSI target address. + + riscv,hart-index-bits: + minimum: 0 + maximum: 15 + description: + Number of HART index bits in the MSI target address. When not + specified it is calculated based on the interrupts-extended property. + + riscv,group-index-bits: + minimum: 0 + maximum: 7 + default: 0 + description: + Number of group index bits in the MSI target address. + + riscv,group-index-shift: + $ref: /schemas/types.yaml#/definitions/uint32 + minimum: 0 + maximum: 55 + default: 24 + description: + The least significant bit position of the group index bits in the + MSI target address. + +required: + - compatible + - reg + - interrupt-controller + - msi-controller + - "#msi-cells" + - interrupts-extended + - riscv,num-ids + +unevaluatedProperties: false + +examples: + - | + // Example 1 (Machine-level IMSIC files with just one group): + + interrupt-controller@24000000 { + compatible = "qemu,imsics", "riscv,imsics"; + interrupts-extended = <&cpu1_intc 11>, + <&cpu2_intc 11>, + <&cpu3_intc 11>, + <&cpu4_intc 11>; + reg = <0x28000000 0x4000>; + interrupt-controller; + #interrupt-cells = <0>; + msi-controller; + #msi-cells = <0>; + riscv,num-ids = <127>; + }; + + - | + // Example 2 (Supervisor-level IMSIC files with two groups): + + interrupt-controller@28000000 { + compatible = "qemu,imsics", "riscv,imsics"; + interrupts-extended = <&cpu1_intc 9>, + <&cpu2_intc 9>, + <&cpu3_intc 9>, + <&cpu4_intc 9>; + reg = <0x28000000 0x2000>, /* Group0 IMSICs */ + <0x29000000 0x2000>; /* Group1 IMSICs */ + interrupt-controller; + #interrupt-cells = <0>; + msi-controller; + #msi-cells = <0>; + riscv,num-ids = <127>; + riscv,group-index-bits = <1>; + riscv,group-index-shift = <24>; + }; +...
PATCH v16 2/9 irqchip: Add RISC-V incoming MSI controller early driver
In reply to Anup PatelThe RISC-V advanced interrupt architecture (AIA) specification defines a new MSI controller called incoming message signalled interrupt controller (IMSIC) which manages MSI on per-HART (or per-CPU) basis. It also supports IPIs as software injected MSIs. (For more details refer https://github.com/riscv/riscv-aia)
Add an early irqchip driver for RISC-V IMSIC which sets up the IMSIC state and provide IPIs.
--- drivers/irqchip/Kconfig | 7 + drivers/irqchip/Makefile | 1 + drivers/irqchip/irq-riscv-imsic-early.c | 201 ++++++ drivers/irqchip/irq-riscv-imsic-state.c | 865 ++++++++++++++++++++++++ drivers/irqchip/irq-riscv-imsic-state.h | 107 +++ include/linux/cpuhotplug.h | 1 + include/linux/irqchip/riscv-imsic.h | 87 +++ 7 files changed, 1269 insertions(+) create mode 100644 drivers/irqchip/irq-riscv-imsic-early.c create mode 100644 drivers/irqchip/irq-riscv-imsic-state.c create mode 100644 drivers/irqchip/irq-riscv-imsic-state.h create mode 100644 include/linux/irqchip/riscv-imsic.h
drivers/irqchip/Kconfig +7 -0
@@ -546,6 +546,13 @@ config SIFIVE_PLIC select IRQ_DOMAIN_HIERARCHY select GENERIC_IRQ_EFFECTIVE_AFF_MASK if SMP +config RISCV_IMSIC + bool + depends on RISCV + select IRQ_DOMAIN_HIERARCHY + select GENERIC_IRQ_MATRIX_ALLOCATOR + select GENERIC_MSI_IRQ + config EXYNOS_IRQ_COMBINER bool "Samsung Exynos IRQ combiner support" if COMPILE_TEST depends on (ARCH_EXYNOS && ARM) || COMPILE_TEST
drivers/irqchip/Makefile +1 -0
@@ -95,6 +95,7 @@ obj-$(CONFIG_QCOM_MPM) += irq-qcom-mpm.o obj-$(CONFIG_CSKY_MPINTC) += irq-csky-mpintc.o obj-$(CONFIG_CSKY_APB_INTC) += irq-csky-apb-intc.o obj-$(CONFIG_RISCV_INTC) += irq-riscv-intc.o +obj-$(CONFIG_RISCV_IMSIC) += irq-riscv-imsic-state.o irq-riscv-imsic-early.o obj-$(CONFIG_SIFIVE_PLIC) += irq-sifive-plic.o obj-$(CONFIG_IMX_IRQSTEER) += irq-imx-irqsteer.o obj-$(CONFIG_IMX_INTMUX) += irq-imx-intmux.o
drivers/irqchip/irq-riscv-imsic-early.c new file +201 -0
201 lines not on this page · raw message
drivers/irqchip/irq-riscv-imsic-state.c new file +865 -0
865 lines not on this page · raw message
drivers/irqchip/irq-riscv-imsic-state.h new file +107 -0
107 lines not on this page · raw message
include/linux/cpuhotplug.h +1 -0
@@ -146,6 +146,7 @@ enum cpuhp_state {CPUHP_AP_IRQ_MIPS_GIC_STARTING ,CPUHP_AP_IRQ_LOONGARCH_STARTING ,CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING , +CPUHP_AP_IRQ_RISCV_IMSIC_STARTING ,CPUHP_AP_ARM_MVEBU_COHERENCY ,CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING , CPUHP_AP_PERF_X86_STARTING,
include/linux/irqchip/riscv-imsic.h new file +87 -0
87 lines not on this page · raw message
Signature · 1 line
2.34.1
[tip: irq/core] irqchip: Add RISC-V incoming MSI controller early driver
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 21a8f8a0eb35ceb21e2c9ddd87468bc3b5ac87c0 Gitweb: https://git.kernel.org/tip/21a8f8a0eb35ceb21e2c9ddd87468bc3b5ac87c0 Author: Anup Patel <[email protected]> AuthorDate: Thu, 07 Mar 2024 19:33:00 +05:30 Committer: Thomas Gleixner <[email protected]> CommitterDate: Mon, 25 Mar 2024 17:38:28 +01:00
irqchip: Add RISC-V incoming MSI controller early driver
The RISC-V advanced interrupt architecture (AIA) specification defines a new MSI controller called incoming message signalled interrupt controller (IMSIC) which manages MSI on per-HART (or per-CPU) basis. It also supports IPIs as software injected MSIs. (For more details refer https://github.com/riscv/riscv-aia)
Add an early irqchip driver for RISC-V IMSIC which sets up the IMSIC state and provide IPIs.
--- drivers/irqchip/Kconfig | 7 +- drivers/irqchip/Makefile | 1 +- drivers/irqchip/irq-riscv-imsic-early.c | 201 +++++- drivers/irqchip/irq-riscv-imsic-state.c | 865 +++++++++++++++++++++++- drivers/irqchip/irq-riscv-imsic-state.h | 107 +++- include/linux/cpuhotplug.h | 1 +- include/linux/irqchip/riscv-imsic.h | 87 ++- 7 files changed, 1269 insertions(+) create mode 100644 drivers/irqchip/irq-riscv-imsic-early.c create mode 100644 drivers/irqchip/irq-riscv-imsic-state.c create mode 100644 drivers/irqchip/irq-riscv-imsic-state.h create mode 100644 include/linux/irqchip/riscv-imsic.h
drivers/irqchip/Kconfig +7 -0
@@ -540,6 +540,13 @@ config RISCV_INTC depends on RISCV select IRQ_DOMAIN_HIERARCHY +config RISCV_IMSIC + bool + depends on RISCV + select IRQ_DOMAIN_HIERARCHY + select GENERIC_IRQ_MATRIX_ALLOCATOR + select GENERIC_MSI_IRQ + config SIFIVE_PLIC bool depends on RISCV
drivers/irqchip/Makefile +1 -0
@@ -95,6 +95,7 @@ obj-$(CONFIG_QCOM_MPM) += irq-qcom-mpm.o obj-$(CONFIG_CSKY_MPINTC) += irq-csky-mpintc.o obj-$(CONFIG_CSKY_APB_INTC) += irq-csky-apb-intc.o obj-$(CONFIG_RISCV_INTC) += irq-riscv-intc.o +obj-$(CONFIG_RISCV_IMSIC) += irq-riscv-imsic-state.o irq-riscv-imsic-early.o obj-$(CONFIG_SIFIVE_PLIC) += irq-sifive-plic.o obj-$(CONFIG_STARFIVE_JH8100_INTC) += irq-starfive-jh8100-intc.o obj-$(CONFIG_IMX_IRQSTEER) += irq-imx-irqsteer.o
drivers/irqchip/irq-riscv-imsic-early.c new file +201 -0
201 lines not on this page · raw message
drivers/irqchip/irq-riscv-imsic-state.c new file +865 -0
865 lines not on this page · raw message
drivers/irqchip/irq-riscv-imsic-state.h new file +107 -0
107 lines not on this page · raw message
include/linux/cpuhotplug.h +1 -0
@@ -146,6 +146,7 @@ enum cpuhp_state {CPUHP_AP_IRQ_MIPS_GIC_STARTING ,CPUHP_AP_IRQ_LOONGARCH_STARTING ,CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING , +CPUHP_AP_IRQ_RISCV_IMSIC_STARTING ,CPUHP_AP_ARM_MVEBU_COHERENCY ,CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING , CPUHP_AP_PERF_X86_STARTING,
include/linux/irqchip/riscv-imsic.h new file +87 -0
87 lines not on this page · raw message
Re: [PATCH v16 2/9] irqchip: Add RISC-V incoming MSI controller early driver
In reply to Anup PatelQuoted 5 lines · Anup Patel
drivers/irqchip/irq-riscv-imsic-state.c new file +5
-0+static int __init imsic_parse_fwnode (struct fwnode_handle *fwnode, +struct imsic_global_config *global, +u32 *nr_parent_irqs, +u32 *nr_mmios) +{
...
drivers/irqchip/irq-riscv-imsic-state.c line 614
Quoted 12 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +12
-0+/* +* Find first bit position of group index. +* If not specified assumed the default APLIC-IMSIC configuration. +*/ + rc =of_property_read_u32 (to_of_node (fwnode),"riscv,group-index-shift" , + &global->group_index_shift); +if (rc) + global->group_index_shift =IMSIC_MMIO_PAGE_SHIFT *2 ; + +/* Find number of interrupt identities */ + rc =of_property_read_u32 (to_of_node (fwnode),"riscv,num-ids" , + &global->nr_ids);
Will here check if the pass-in interrupt identity number is (multi-64 -1) && between [63, 2047]? Per spec AIA v1.0, Sec. 3.1 Interrupt files and interrupt identities:
"The number of interrupt identities supported by an interrupt file (and hence the number of active bits in each array) is one less than a multiple of 64, and may be a minimum of 63 and a maximum of 2047."
Quoted 5 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +5
-0+if (rc) { +pr_err ("%pfwP: number of interrupt identities not found\n" , fwnode); +return rc; + } +
...
Re: [PATCH v16 2/9] irqchip: Add RISC-V incoming MSI controller early driver
Quoted 24 lines · Eric Cheng
Quoted 5 lines · Anup Patel
drivers/irqchip/irq-riscv-imsic-state.c new file +5
-0+static int __init imsic_parse_fwnode (struct fwnode_handle *fwnode, +struct imsic_global_config *global, +u32 *nr_parent_irqs, +u32 *nr_mmios) +{...
drivers/irqchip/irq-riscv-imsic-state.c line 614
Quoted 12 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +12
-0+/* +* Find first bit position of group index. +* If not specified assumed the default APLIC-IMSIC configuration. +*/ + rc =of_property_read_u32 (to_of_node (fwnode),"riscv,group-index-shift" , + &global->group_index_shift); +if (rc) + global->group_index_shift =IMSIC_MMIO_PAGE_SHIFT *2 ; + +/* Find number of interrupt identities */ + rc =of_property_read_u32 (to_of_node (fwnode),"riscv,num-ids" , + &global->nr_ids);
Quoted 8 lines
Will here check if the pass-in interrupt identity number is (multi-64 -1) && between [63, 2047]?
Per spec AIA v1.0, Sec. 3.1 Interrupt files and interrupt identities:
"The number of interrupt identities supported by an interrupt file (and hence the number of active
bits in each array) is one less than a multiple of 64, and may be a minimum of 63 and a maximum
of 2047."
There is already a check on global->nr_ids to ensure the above is satisfied. Please see further below in imsic_parse_fwnode() function.
Quoted 9 lines
Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +5
-0+if (rc) { +pr_err ("%pfwP: number of interrupt identities not found\n" , fwnode); +return rc; + } +...
Regards,
Anup
Re: [PATCH v16 2/9] irqchip: Add RISC-V incoming MSI controller early driver
Quoted 34 lines · Anup Patel
Quoted 24 lines · Eric Cheng
Quoted 5 lines · Anup Patel
drivers/irqchip/irq-riscv-imsic-state.c new file +5
-0+static int __init imsic_parse_fwnode (struct fwnode_handle *fwnode, +struct imsic_global_config *global, +u32 *nr_parent_irqs, +u32 *nr_mmios) +{...
drivers/irqchip/irq-riscv-imsic-state.c line 614
Quoted 12 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +12
-0+/* +* Find first bit position of group index. +* If not specified assumed the default APLIC-IMSIC configuration. +*/ + rc =of_property_read_u32 (to_of_node (fwnode),"riscv,group-index-shift" , + &global->group_index_shift); +if (rc) + global->group_index_shift =IMSIC_MMIO_PAGE_SHIFT *2 ; + +/* Find number of interrupt identities */ + rc =of_property_read_u32 (to_of_node (fwnode),"riscv,num-ids" , + &global->nr_ids);Quoted 8 lines
Will here check if the pass-in interrupt identity number is (multi-64 -1) && between [63, 2047]?
Per spec AIA v1.0, Sec. 3.1 Interrupt files and interrupt identities:
"The number of interrupt identities supported by an interrupt file (and hence the number of active
bits in each array) is one less than a multiple of 64, and may be a minimum of 63 and a maximum
of 2047."
Quoted 3 lines
There is already a check on
global->nr_idsto ensure the above is satisfied. Please see further below inimsic_parse_fwnode()function.
Right, thanks.
Quoted 12 lines
Quoted 9 lines
Quoted 6 lines
drivers/irqchip/irq-riscv-imsic-state.c new file +5
-0+if (rc) { +pr_err ("%pfwP: number of interrupt identities not found\n" , fwnode); +return rc; + } +...
Regards,
Anup
PATCH v16 3/9 irqchip/riscv-imsic: Add device MSI domain support for platform devices
In reply to Anup PatelThe Linux platform MSI support allows per-device MSI domains so add a platform irqchip driver for RISC-V IMSIC which provides a base IRQ domain with MSI parent support for platform device domains.
The IMSIC platform driver assumes that the IMSIC state is already initialized by the IMSIC early driver.
--- drivers/irqchip/Makefile | 2 +- drivers/irqchip/irq-riscv-imsic-platform.c | 343 +++++++++++++++++++++ drivers/irqchip/irq-riscv-imsic-state.h | 1 + 3 files changed, 345 insertions(+), 1 deletion(-) create mode 100644 drivers/irqchip/irq-riscv-imsic-platform.c
drivers/irqchip/Makefile +1 -1
@@ -95,7 +95,7 @@ obj-$(CONFIG_QCOM_MPM) += irq-qcom-mpm.o obj-$(CONFIG_CSKY_MPINTC) += irq-csky-mpintc.o obj-$(CONFIG_CSKY_APB_INTC) += irq-csky-apb-intc.o obj-$(CONFIG_RISCV_INTC) += irq-riscv-intc.o -obj-$(CONFIG_RISCV_IMSIC) += irq-riscv-imsic-state.o irq-riscv-imsic-early.o +obj-$(CONFIG_RISCV_IMSIC) += irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o obj-$(CONFIG_SIFIVE_PLIC) += irq-sifive-plic.o obj-$(CONFIG_IMX_IRQSTEER) += irq-imx-irqsteer.o obj-$(CONFIG_IMX_INTMUX) += irq-imx-intmux.o
drivers/irqchip/irq-riscv-imsic-platform.c new file +343 -0
343 lines not on this page · raw message
drivers/irqchip/irq-riscv-imsic-state.h +1 -0
@@ -103,5 +103,6 @@ void imsic_vector_debug_show_summary(struct seq_file *m, int ind);void imsic_state_online (void );void imsic_state_offline (void );int imsic_setup_state (struct fwnode_handle *fwnode); +int imsic_irqdomain_init (void );#endif
Signature · 1 line
2.34.1
[tip: irq/core] irqchip/riscv-imsic: Add device MSI domain support for platform devices
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 027e125acdbad79e9a7274940e8bf92299b208af Gitweb: https://git.kernel.org/tip/027e125acdbad79e9a7274940e8bf92299b208af Author: Anup Patel <[email protected]> AuthorDate: Thu, 07 Mar 2024 19:33:01 +05:30 Committer: Thomas Gleixner <[email protected]> CommitterDate: Mon, 25 Mar 2024 17:38:28 +01:00
irqchip/riscv-imsic: Add device MSI domain support for platform devices
The Linux platform MSI support allows per-device MSI domains so add a platform irqchip driver for RISC-V IMSIC which provides a base IRQ domain with MSI parent support for platform device domains.
The IMSIC platform driver assumes that the IMSIC state is already initialized by the IMSIC early driver.
--- drivers/irqchip/Makefile | 2 +- drivers/irqchip/irq-riscv-imsic-platform.c | 343 ++++++++++++++++++++- drivers/irqchip/irq-riscv-imsic-state.h | 1 +- 3 files changed, 345 insertions(+), 1 deletion(-) create mode 100644 drivers/irqchip/irq-riscv-imsic-platform.c
drivers/irqchip/Makefile +1 -1
@@ -95,7 +95,7 @@ obj-$(CONFIG_QCOM_MPM) += irq-qcom-mpm.o obj-$(CONFIG_CSKY_MPINTC) += irq-csky-mpintc.o obj-$(CONFIG_CSKY_APB_INTC) += irq-csky-apb-intc.o obj-$(CONFIG_RISCV_INTC) += irq-riscv-intc.o -obj-$(CONFIG_RISCV_IMSIC) += irq-riscv-imsic-state.o irq-riscv-imsic-early.o +obj-$(CONFIG_RISCV_IMSIC) += irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o obj-$(CONFIG_SIFIVE_PLIC) += irq-sifive-plic.o obj-$(CONFIG_STARFIVE_JH8100_INTC) += irq-starfive-jh8100-intc.o obj-$(CONFIG_IMX_IRQSTEER) += irq-imx-irqsteer.o
drivers/irqchip/irq-riscv-imsic-platform.c new file +343 -0
343 lines not on this page · raw message
drivers/irqchip/irq-riscv-imsic-state.h +1 -0
@@ -103,5 +103,6 @@ void imsic_vector_debug_show_summary(struct seq_file *m, int ind);void imsic_state_online (void );void imsic_state_offline (void );int imsic_setup_state (struct fwnode_handle *fwnode); +int imsic_irqdomain_init (void );#endif
PATCH v16 4/9 irqchip/riscv-imsic: Add device MSI domain support for PCI devices
In reply to Anup PatelThe Linux PCI framework supports per-device MSI domains for PCI devices so extend the IMSIC driver to allow PCI per-device MSI domains.
--- drivers/irqchip/Kconfig | 7 +++++ drivers/irqchip/irq-riscv-imsic-platform.c | 35 ++++++++++++++++++++-- 2 files changed, 40 insertions(+), 2 deletions(-)
drivers/irqchip/Kconfig +7 -0
@@ -553,6 +553,13 @@ config RISCV_IMSIC select GENERIC_IRQ_MATRIX_ALLOCATOR select GENERIC_MSI_IRQ +config RISCV_IMSIC_PCI + bool + depends on RISCV_IMSIC + depends on PCI + depends on PCI_MSI + default RISCV_IMSIC + config EXYNOS_IRQ_COMBINER bool "Samsung Exynos IRQ combiner support" if COMPILE_TEST depends on (ARCH_EXYNOS && ARM) || COMPILE_TEST
drivers/irqchip/irq-riscv-imsic-platform.c +33 -2
@@ -14,6 +14,7 @@#include <linux/irqdomain.h> #include <linux/module.h> #include <linux/msi.h> +#include <linux/pci.h> #include <linux/platform_device.h> #include <linux/spinlock.h> #include <linux/smp.h>
@@ -207,6 +208,28 @@ static const struct irq_domain_ops imsic_base_domain_ops = {#endif }; +#ifdef CONFIG_RISCV_IMSIC_PCI + +static void imsic_pci_mask_irq (struct irq_data *d) +{ +pci_msi_mask_irq (d); +irq_chip_mask_parent (d); +} + +static void imsic_pci_unmask_irq (struct irq_data *d) +{ +irq_chip_unmask_parent (d); +pci_msi_unmask_irq (d); +} + +#define MATCH_PCI_MSI BIT(DOMAIN_BUS_PCI_MSI) + +#else + +#define MATCH_PCI_MSI 0 + +#endif + static bool imsic_init_dev_msi_info(struct device *dev, struct irq_domain *domain, struct irq_domain *real_parent,
@@ -230,6 +253,13 @@ static bool imsic_init_dev_msi_info(struct device *dev,/* Is the target supported? */ switch (info->bus_token) { +#ifdef CONFIG_RISCV_IMSIC_PCI +case DOMAIN_BUS_PCI_DEVICE_MSI : +case DOMAIN_BUS_PCI_DEVICE_MSIX : + info->chip->irq_mask = imsic_pci_mask_irq; + info->chip->irq_unmask = imsic_pci_unmask_irq; +break ; +#endif case DOMAIN_BUS_DEVICE_MSI :/* * Per-device MSI should never have any MSI feature bits
@@ -269,11 +299,12 @@ static bool imsic_init_dev_msi_info(struct device *dev,#define MATCH_PLATFORM_MSI BIT(DOMAIN_BUS_PLATFORM_MSI)static const struct msi_parent_ops imsic_msi_parent_ops = { - .supported_flags =MSI_GENERIC_FLAGS_MASK , + .supported_flags =MSI_GENERIC_FLAGS_MASK | +MSI_FLAG_PCI_MSIX , .required_flags =MSI_FLAG_USE_DEF_DOM_OPS |MSI_FLAG_USE_DEF_CHIP_OPS , .bus_select_token =DOMAIN_BUS_NEXUS , - .bus_select_mask =MATCH_PLATFORM_MSI , + .bus_select_mask =MATCH_PCI_MSI |MATCH_PLATFORM_MSI , .init_dev_msi_info = imsic_init_dev_msi_info, };
Signature · 1 line
2.34.1
[tip: irq/core] irqchip/riscv-imsic: Add device MSI domain support for PCI devices
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 5c5a71d0434093cd42d09afd4e2032c0b16a7da8 Gitweb: https://git.kernel.org/tip/5c5a71d0434093cd42d09afd4e2032c0b16a7da8 Author: Anup Patel <[email protected]> AuthorDate: Thu, 07 Mar 2024 19:33:02 +05:30 Committer: Thomas Gleixner <[email protected]> CommitterDate: Mon, 25 Mar 2024 17:38:28 +01:00
irqchip/riscv-imsic: Add device MSI domain support for PCI devices
The Linux PCI framework supports per-device MSI domains for PCI devices so extend the IMSIC driver to allow PCI per-device MSI domains.
--- drivers/irqchip/Kconfig | 7 ++++- drivers/irqchip/irq-riscv-imsic-platform.c | 35 +++++++++++++++++++-- 2 files changed, 40 insertions(+), 2 deletions(-)
drivers/irqchip/Kconfig +7 -0
@@ -547,6 +547,13 @@ config RISCV_IMSIC select GENERIC_IRQ_MATRIX_ALLOCATOR select GENERIC_MSI_IRQ +config RISCV_IMSIC_PCI + bool + depends on RISCV_IMSIC + depends on PCI + depends on PCI_MSI + default RISCV_IMSIC + config SIFIVE_PLIC bool depends on RISCV
drivers/irqchip/irq-riscv-imsic-platform.c +33 -2
@@ -14,6 +14,7 @@#include <linux/irqdomain.h> #include <linux/module.h> #include <linux/msi.h> +#include <linux/pci.h> #include <linux/platform_device.h> #include <linux/spinlock.h> #include <linux/smp.h>
@@ -207,6 +208,28 @@ static const struct irq_domain_ops imsic_base_domain_ops = {#endif }; +#ifdef CONFIG_RISCV_IMSIC_PCI + +static void imsic_pci_mask_irq (struct irq_data *d) +{ +pci_msi_mask_irq (d); +irq_chip_mask_parent (d); +} + +static void imsic_pci_unmask_irq (struct irq_data *d) +{ +irq_chip_unmask_parent (d); +pci_msi_unmask_irq (d); +} + +#define MATCH_PCI_MSI BIT(DOMAIN_BUS_PCI_MSI) + +#else + +#define MATCH_PCI_MSI 0 + +#endif + static bool imsic_init_dev_msi_info(struct device *dev, struct irq_domain *domain, struct irq_domain *real_parent,
@@ -230,6 +253,13 @@ static bool imsic_init_dev_msi_info(struct device *dev,/* Is the target supported? */ switch (info->bus_token) { +#ifdef CONFIG_RISCV_IMSIC_PCI +case DOMAIN_BUS_PCI_DEVICE_MSI : +case DOMAIN_BUS_PCI_DEVICE_MSIX : + info->chip->irq_mask = imsic_pci_mask_irq; + info->chip->irq_unmask = imsic_pci_unmask_irq; +break ; +#endif case DOMAIN_BUS_DEVICE_MSI :/* * Per-device MSI should never have any MSI feature bits
@@ -269,11 +299,12 @@ static bool imsic_init_dev_msi_info(struct device *dev,#define MATCH_PLATFORM_MSI BIT(DOMAIN_BUS_PLATFORM_MSI)static const struct msi_parent_ops imsic_msi_parent_ops = { - .supported_flags =MSI_GENERIC_FLAGS_MASK , + .supported_flags =MSI_GENERIC_FLAGS_MASK | +MSI_FLAG_PCI_MSIX , .required_flags =MSI_FLAG_USE_DEF_DOM_OPS |MSI_FLAG_USE_DEF_CHIP_OPS , .bus_select_token =DOMAIN_BUS_NEXUS , - .bus_select_mask =MATCH_PLATFORM_MSI , + .bus_select_mask =MATCH_PCI_MSI |MATCH_PLATFORM_MSI , .init_dev_msi_info = imsic_init_dev_msi_info, };
PATCH v16 5/9 dt-bindings: interrupt-controller: Add RISC-V advanced PLIC
In reply to Anup PatelAdd DT bindings document for RISC-V advanced platform level interrupt controller (APLIC) defined by the RISC-V advanced interrupt architecture (AIA) specification.
--- .../interrupt-controller/riscv,aplic.yaml | 172 ++++++++++++++++++ 1 file changed, 172 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml
Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml new file +172 -0
@@ -0,0 +1,172 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/interrupt-controller/riscv,aplic.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: RISC-V Advanced Platform Level Interrupt Controller (APLIC) + +maintainers: + - Anup Patel <[email protected]> + +description: + The RISC-V advanced interrupt architecture (AIA) defines an advanced + platform level interrupt controller (APLIC) for handling wired interrupts + in a RISC-V platform. The RISC-V AIA specification can be found at + https://github.com/riscv/riscv-aia. + + The RISC-V APLIC is implemented as hierarchical APLIC domains where all + interrupt sources connect to the root APLIC domain and a parent APLIC + domain can delegate interrupt sources to it's child APLIC domains. There + is one device tree node for each APLIC domain. + +allOf: + - $ref: /schemas/interrupt-controller.yaml# + +properties: + compatible: + items: + - enum: + - qemu,aplic + - const: riscv,aplic + + reg: + maxItems: 1 + + interrupt-controller: true + + "#interrupt-cells": + const: 2 + + interrupts-extended: + minItems: 1 + maxItems: 16384 + description: + Given APLIC domain directly injects external interrupts to a set of + RISC-V HARTS (or CPUs). Each node pointed to should be a riscv,cpu-intc + node, which has a CPU node (i.e. RISC-V HART) as parent. + + msi-parent: + description: + Given APLIC domain forwards wired interrupts as MSIs to a AIA incoming + message signaled interrupt controller (IMSIC). If both "msi-parent" and + "interrupts-extended" properties are present then it means the APLIC + domain supports both MSI mode and Direct mode in HW. In this case, the + APLIC driver has to choose between MSI mode or Direct mode. + + riscv,num-sources: + $ref: /schemas/types.yaml#/definitions/uint32 + minimum: 1 + maximum: 1023 + description: + Specifies the number of wired interrupt sources supported by this + APLIC domain. + + riscv,children: + $ref: /schemas/types.yaml#/definitions/phandle-array + minItems: 1 + maxItems: 1024 + items: + maxItems: 1 + description: + A list of child APLIC domains for the given APLIC domain. Each child + APLIC domain is assigned a child index in increasing order, with the + first child APLIC domain assigned child index 0. The APLIC domain child + index is used by firmware to delegate interrupts from the given APLIC + domain to a particular child APLIC domain. + + riscv,delegation: + $ref: /schemas/types.yaml#/definitions/phandle-array + minItems: 1 + maxItems: 1024 + items: + items: + - description: child APLIC domain phandle + - description: first interrupt number of the parent APLIC domain (inclusive) + - description: last interrupt number of the parent APLIC domain (inclusive) + description: + A interrupt delegation list where each entry is a triple consisting + of child APLIC domain phandle, first interrupt number of the parent + APLIC domain, and last interrupt number of the parent APLIC domain. + Firmware must configure interrupt delegation registers based on + interrupt delegation list. + +dependencies: + riscv,delegation: [ "riscv,children" ] + +required: + - compatible + - reg + - interrupt-controller + - "#interrupt-cells" + - riscv,num-sources + +anyOf: + - required: + - interrupts-extended + - required: + - msi-parent + +unevaluatedProperties: false + +examples: + - | + // Example 1 (APLIC domains directly injecting interrupt to HARTs): + + interrupt-controller@c000000 { + compatible = "qemu,aplic", "riscv,aplic"; + interrupts-extended = <&cpu1_intc 11>, + <&cpu2_intc 11>, + <&cpu3_intc 11>, + <&cpu4_intc 11>; + reg = <0xc000000 0x4080>; + interrupt-controller; + #interrupt-cells = <2>; + riscv,num-sources = <63>; + riscv,children = <&aplic1>, <&aplic2>; + riscv,delegation = <&aplic1 1 63>; + }; + + aplic1: interrupt-controller@d000000 { + compatible = "qemu,aplic", "riscv,aplic"; + interrupts-extended = <&cpu1_intc 9>, + <&cpu2_intc 9>; + reg = <0xd000000 0x4080>; + interrupt-controller; + #interrupt-cells = <2>; + riscv,num-sources = <63>; + }; + + aplic2: interrupt-controller@e000000 { + compatible = "qemu,aplic", "riscv,aplic"; + interrupts-extended = <&cpu3_intc 9>, + <&cpu4_intc 9>; + reg = <0xe000000 0x4080>; + interrupt-controller; + #interrupt-cells = <2>; + riscv,num-sources = <63>; + }; + + - | + // Example 2 (APLIC domains forwarding interrupts as MSIs): + + interrupt-controller@c000000 { + compatible = "qemu,aplic", "riscv,aplic"; + msi-parent = <&imsic_mlevel>; + reg = <0xc000000 0x4000>; + interrupt-controller; + #interrupt-cells = <2>; + riscv,num-sources = <63>; + riscv,children = <&aplic3>; + riscv,delegation = <&aplic3 1 63>; + }; + + aplic3: interrupt-controller@d000000 { + compatible = "qemu,aplic", "riscv,aplic"; + msi-parent = <&imsic_slevel>; + reg = <0xd000000 0x4000>; + interrupt-controller; + #interrupt-cells = <2>; + riscv,num-sources = <63>; + }; +...
Signature · 1 line
2.34.1
[tip: irq/core] dt-bindings: interrupt-controller: Add RISC-V advanced PLIC
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 3b806a5a1a39d593d6fe0def2ef474402f551f91 Gitweb: https://git.kernel.org/tip/3b806a5a1a39d593d6fe0def2ef474402f551f91 Author: Anup Patel <[email protected]> AuthorDate: Thu, 07 Mar 2024 19:33:03 +05:30 Committer: Thomas Gleixner <[email protected]> CommitterDate: Mon, 25 Mar 2024 17:38:28 +01:00
dt-bindings: interrupt-controller: Add RISC-V advanced PLIC
Add DT bindings document for RISC-V advanced platform level interrupt controller (APLIC) defined by the RISC-V advanced interrupt architecture (AIA) specification.
--- Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml | 172 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 172 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml
Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml new file +172 -0
@@ -0,0 +1,172 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/interrupt-controller/riscv,aplic.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: RISC-V Advanced Platform Level Interrupt Controller (APLIC) + +maintainers: + - Anup Patel <[email protected]> + +description: + The RISC-V advanced interrupt architecture (AIA) defines an advanced + platform level interrupt controller (APLIC) for handling wired interrupts + in a RISC-V platform. The RISC-V AIA specification can be found at + https://github.com/riscv/riscv-aia. + + The RISC-V APLIC is implemented as hierarchical APLIC domains where all + interrupt sources connect to the root APLIC domain and a parent APLIC + domain can delegate interrupt sources to it's child APLIC domains. There + is one device tree node for each APLIC domain. + +allOf: + - $ref: /schemas/interrupt-controller.yaml# + +properties: + compatible: + items: + - enum: + - qemu,aplic + - const: riscv,aplic + + reg: + maxItems: 1 + + interrupt-controller: true + + "#interrupt-cells": + const: 2 + + interrupts-extended: + minItems: 1 + maxItems: 16384 + description: + Given APLIC domain directly injects external interrupts to a set of + RISC-V HARTS (or CPUs). Each node pointed to should be a riscv,cpu-intc + node, which has a CPU node (i.e. RISC-V HART) as parent. + + msi-parent: + description: + Given APLIC domain forwards wired interrupts as MSIs to a AIA incoming + message signaled interrupt controller (IMSIC). If both "msi-parent" and + "interrupts-extended" properties are present then it means the APLIC + domain supports both MSI mode and Direct mode in HW. In this case, the + APLIC driver has to choose between MSI mode or Direct mode. + + riscv,num-sources: + $ref: /schemas/types.yaml#/definitions/uint32 + minimum: 1 + maximum: 1023 + description: + Specifies the number of wired interrupt sources supported by this + APLIC domain. + + riscv,children: + $ref: /schemas/types.yaml#/definitions/phandle-array + minItems: 1 + maxItems: 1024 + items: + maxItems: 1 + description: + A list of child APLIC domains for the given APLIC domain. Each child + APLIC domain is assigned a child index in increasing order, with the + first child APLIC domain assigned child index 0. The APLIC domain child + index is used by firmware to delegate interrupts from the given APLIC + domain to a particular child APLIC domain. + + riscv,delegation: + $ref: /schemas/types.yaml#/definitions/phandle-array + minItems: 1 + maxItems: 1024 + items: + items: + - description: child APLIC domain phandle + - description: first interrupt number of the parent APLIC domain (inclusive) + - description: last interrupt number of the parent APLIC domain (inclusive) + description: + A interrupt delegation list where each entry is a triple consisting + of child APLIC domain phandle, first interrupt number of the parent + APLIC domain, and last interrupt number of the parent APLIC domain. + Firmware must configure interrupt delegation registers based on + interrupt delegation list. + +dependencies: + riscv,delegation: [ "riscv,children" ] + +required: + - compatible + - reg + - interrupt-controller + - "#interrupt-cells" + - riscv,num-sources + +anyOf: + - required: + - interrupts-extended + - required: + - msi-parent + +unevaluatedProperties: false + +examples: + - | + // Example 1 (APLIC domains directly injecting interrupt to HARTs): + + interrupt-controller@c000000 { + compatible = "qemu,aplic", "riscv,aplic"; + interrupts-extended = <&cpu1_intc 11>, + <&cpu2_intc 11>, + <&cpu3_intc 11>, + <&cpu4_intc 11>; + reg = <0xc000000 0x4080>; + interrupt-controller; + #interrupt-cells = <2>; + riscv,num-sources = <63>; + riscv,children = <&aplic1>, <&aplic2>; + riscv,delegation = <&aplic1 1 63>; + }; + + aplic1: interrupt-controller@d000000 { + compatible = "qemu,aplic", "riscv,aplic"; + interrupts-extended = <&cpu1_intc 9>, + <&cpu2_intc 9>; + reg = <0xd000000 0x4080>; + interrupt-controller; + #interrupt-cells = <2>; + riscv,num-sources = <63>; + }; + + aplic2: interrupt-controller@e000000 { + compatible = "qemu,aplic", "riscv,aplic"; + interrupts-extended = <&cpu3_intc 9>, + <&cpu4_intc 9>; + reg = <0xe000000 0x4080>; + interrupt-controller; + #interrupt-cells = <2>; + riscv,num-sources = <63>; + }; + + - | + // Example 2 (APLIC domains forwarding interrupts as MSIs): + + interrupt-controller@c000000 { + compatible = "qemu,aplic", "riscv,aplic"; + msi-parent = <&imsic_mlevel>; + reg = <0xc000000 0x4000>; + interrupt-controller; + #interrupt-cells = <2>; + riscv,num-sources = <63>; + riscv,children = <&aplic3>; + riscv,delegation = <&aplic3 1 63>; + }; + + aplic3: interrupt-controller@d000000 { + compatible = "qemu,aplic", "riscv,aplic"; + msi-parent = <&imsic_slevel>; + reg = <0xd000000 0x4000>; + interrupt-controller; + #interrupt-cells = <2>; + riscv,num-sources = <63>; + }; +...
PATCH v16 6/9 irqchip: Add RISC-V advanced PLIC driver for direct-mode
In reply to Anup PatelThe RISC-V advanced interrupt architecture (AIA) specification defines advanced platform-level interrupt controller (APLIC) which has two modes of operation: 1) Direct mode and 2) MSI mode.
(For more details, refer https://github.com/riscv/riscv-aia)
In APLIC direct-mode, wired interrupts are forwared to CPUs (or HARTs) as a local external interrupt.
Add a platform irqchip driver for the RISC-V APLIC direct-mode to support RISC-V platforms having only wired interrupts.
--- drivers/irqchip/Kconfig | 5 + drivers/irqchip/Makefile | 1 + drivers/irqchip/irq-riscv-aplic-direct.c | 326 +++++++++++++++++++++++ drivers/irqchip/irq-riscv-aplic-main.c | 211 +++++++++++++++ drivers/irqchip/irq-riscv-aplic-main.h | 44 +++ include/linux/irqchip/riscv-aplic.h | 145 ++++++++++ 6 files changed, 732 insertions(+) create mode 100644 drivers/irqchip/irq-riscv-aplic-direct.c create mode 100644 drivers/irqchip/irq-riscv-aplic-main.c create mode 100644 drivers/irqchip/irq-riscv-aplic-main.h create mode 100644 include/linux/irqchip/riscv-aplic.h
drivers/irqchip/Kconfig +5 -0
@@ -546,6 +546,11 @@ config SIFIVE_PLIC select IRQ_DOMAIN_HIERARCHY select GENERIC_IRQ_EFFECTIVE_AFF_MASK if SMP +config RISCV_APLIC + bool + depends on RISCV + select IRQ_DOMAIN_HIERARCHY + config RISCV_IMSIC bool depends on RISCV
drivers/irqchip/Makefile +1 -0
@@ -95,6 +95,7 @@ obj-$(CONFIG_QCOM_MPM) += irq-qcom-mpm.o obj-$(CONFIG_CSKY_MPINTC) += irq-csky-mpintc.o obj-$(CONFIG_CSKY_APB_INTC) += irq-csky-apb-intc.o obj-$(CONFIG_RISCV_INTC) += irq-riscv-intc.o +obj-$(CONFIG_RISCV_APLIC) += irq-riscv-aplic-main.o irq-riscv-aplic-direct.o obj-$(CONFIG_RISCV_IMSIC) += irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o obj-$(CONFIG_SIFIVE_PLIC) += irq-sifive-plic.o obj-$(CONFIG_IMX_IRQSTEER) += irq-imx-irqsteer.o
drivers/irqchip/irq-riscv-aplic-direct.c new file +326 -0
326 lines not on this page · raw message
drivers/irqchip/irq-riscv-aplic-main.c new file +211 -0
211 lines not on this page · raw message
drivers/irqchip/irq-riscv-aplic-main.h new file +44 -0
44 lines not on this page · raw message
include/linux/irqchip/riscv-aplic.h new file +145 -0
145 lines not on this page · raw message
Signature · 1 line
2.34.1
[tip: irq/core] irqchip: Add RISC-V advanced PLIC driver for direct-mode
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 2333df5ae51ead2188d07c99e841e159a664741e Gitweb: https://git.kernel.org/tip/2333df5ae51ead2188d07c99e841e159a664741e Author: Anup Patel <[email protected]> AuthorDate: Thu, 07 Mar 2024 19:33:04 +05:30 Committer: Thomas Gleixner <[email protected]> CommitterDate: Mon, 25 Mar 2024 17:38:29 +01:00
irqchip: Add RISC-V advanced PLIC driver for direct-mode
The RISC-V advanced interrupt architecture (AIA) specification defines advanced platform-level interrupt controller (APLIC) which has two modes of operation: 1) Direct mode and 2) MSI mode.
(For more details, refer https://github.com/riscv/riscv-aia)
In APLIC direct-mode, wired interrupts are forwared to CPUs (or HARTs) as a local external interrupt.
Add a platform irqchip driver for the RISC-V APLIC direct-mode to support RISC-V platforms having only wired interrupts.
--- drivers/irqchip/Kconfig | 5 +- drivers/irqchip/Makefile | 1 +- drivers/irqchip/irq-riscv-aplic-direct.c | 326 ++++++++++++++++++++++- drivers/irqchip/irq-riscv-aplic-main.c | 211 ++++++++++++++- drivers/irqchip/irq-riscv-aplic-main.h | 44 +++- include/linux/irqchip/riscv-aplic.h | 145 ++++++++++- 6 files changed, 732 insertions(+) create mode 100644 drivers/irqchip/irq-riscv-aplic-direct.c create mode 100644 drivers/irqchip/irq-riscv-aplic-main.c create mode 100644 drivers/irqchip/irq-riscv-aplic-main.h create mode 100644 include/linux/irqchip/riscv-aplic.h
drivers/irqchip/Kconfig +5 -0
@@ -540,6 +540,11 @@ config RISCV_INTC depends on RISCV select IRQ_DOMAIN_HIERARCHY +config RISCV_APLIC + bool + depends on RISCV + select IRQ_DOMAIN_HIERARCHY + config RISCV_IMSIC bool depends on RISCV
drivers/irqchip/Makefile +1 -0
@@ -95,6 +95,7 @@ obj-$(CONFIG_QCOM_MPM) += irq-qcom-mpm.o obj-$(CONFIG_CSKY_MPINTC) += irq-csky-mpintc.o obj-$(CONFIG_CSKY_APB_INTC) += irq-csky-apb-intc.o obj-$(CONFIG_RISCV_INTC) += irq-riscv-intc.o +obj-$(CONFIG_RISCV_APLIC) += irq-riscv-aplic-main.o irq-riscv-aplic-direct.o obj-$(CONFIG_RISCV_IMSIC) += irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o obj-$(CONFIG_SIFIVE_PLIC) += irq-sifive-plic.o obj-$(CONFIG_STARFIVE_JH8100_INTC) += irq-starfive-jh8100-intc.o
drivers/irqchip/irq-riscv-aplic-direct.c new file +326 -0
326 lines not on this page · raw message
drivers/irqchip/irq-riscv-aplic-main.c new file +211 -0
211 lines not on this page · raw message
drivers/irqchip/irq-riscv-aplic-main.h new file +44 -0
44 lines not on this page · raw message
include/linux/irqchip/riscv-aplic.h new file +145 -0
145 lines not on this page · raw message
Re: [PATCH v16 6/9] irqchip: Add RISC-V advanced PLIC driver for direct-mode
In reply to Anup PatelQuoted 4 lines
drivers/irqchip/irq-riscv-aplic-main.c +3
-0+static const struct of_device_id aplic_match[] = { + { .compatible ="riscv,aplic" }, + {}
drivers/irqchip/irq-riscv-aplic-main.c line 211
Quoted 10 lines
drivers/irqchip/irq-riscv-aplic-main.c +10
-0+}; + +static struct platform_driver aplic_driver = { + .driver = { + .name ="riscv-aplic" , + .of_match_table = aplic_match, + }, + .probe = aplic_probe, +}; +builtin_platform_driver (aplic_driver);
Dear Anup Patel and all concerned,
I am writing to inquire about the historical rationale behind defining the APLIC driver's initialization priority using builtin_platform_driver in the current implementation.
In our environment, we are encountering an issue where this priority level causes ACPI-based PCIe enumeration to be executed in the system_unbound_wq work queue. This parallel execution model results in PCIe devices being enumerated in an arbitrary order rather than strictly following the sequence defined in the ACPI DSDT table.
The random enumeration order is adversely affecting customer experience, particularly in scenarios where device ordering is critical for proper system operation or application compatibility.
We are considering modifying the APLIC driver's initialization priority to ensure PCIe enumeration occurs sequentially according to the DSDT specification. However, before proceeding with such changes, we wanted to consult with you regarding:
1. Were there specific technical considerations that led to the current priority selection?
2. Are there any potential side effects or broader impacts that we might have overlooked?
3. Would you support such a priority adjustment, or do you have alternative suggestions to address the enumeration order issue?
We greatly appreciate your insights and expertise on this matter, as it will help us make an informed decision while maintaining system stability and compatibility.
Thank you for your time and consideration.
Best regards,
Liqiang
Re: [PATCH v16 6/9] irqchip: Add RISC-V advanced PLIC driver for direct-mode
Hi Liqiang,
Quoted 37 lines · niliqiang
Quoted 4 lines
drivers/irqchip/irq-riscv-aplic-main.c +3
-0+static const struct of_device_id aplic_match[] = { + { .compatible ="riscv,aplic" }, + {}drivers/irqchip/irq-riscv-aplic-main.c line 211
Quoted 10 lines
drivers/irqchip/irq-riscv-aplic-main.c +10
-0+}; + +static struct platform_driver aplic_driver = { + .driver = { + .name ="riscv-aplic" , + .of_match_table = aplic_match, + }, + .probe = aplic_probe, +}; +builtin_platform_driver (aplic_driver);Dear Anup Patel and all concerned,
I am writing to inquire about the historical rationale behind defining the APLIC driver's initialization priority using
builtin_platform_driverin the current implementation.In our environment, we are encountering an issue where this priority level causes ACPI-based PCIe enumeration to be executed in the
system_unbound_wqwork queue. This parallel execution model results in PCIe devices being enumerated in an arbitrary order rather than strictly following the sequence defined in the ACPI DSDT table.The random enumeration order is adversely affecting customer experience, particularly in scenarios where device ordering is critical for proper system operation or application compatibility.
We are considering modifying the APLIC driver's initialization priority to ensure PCIe enumeration occurs sequentially according to the DSDT specification. However, before proceeding with such changes, we wanted to consult with you regarding:
1. Were there specific technical considerations that led to the current priority selection?
2. Are there any potential side effects or broader impacts that we might have overlooked?
3. Would you support such a priority adjustment, or do you have alternative suggestions to address the enumeration order issue?
Quoted 5 lines
We greatly appreciate your insights and expertise on this matter, as it will help us make an informed decision while maintaining system stability and compatibility.
Thank you for your time and consideration.
IRQ subsystem maintainers rejected the idea of relying on initcalls to enforce probe order because initcalls do not guarantee ordering. The Linux driver model instead ensures probe order through device dependencies. Since PCI INTx depends on the APLIC being probed first, the PCI host bridge probe cannot occur until after the APLIC probe completes. This requirement and behavior are the same for both DT and ACPI. In DT, the driver model uses fw_devlink to establish probe ordering, while in ACPI this is handled through either an explicit _DEP or, on RISC-V, the GSI mapping.
Typically, this dependency appears in the DSDT only for the PCI host bridge. Individual PCIe devices are enumerated through the standard PCI scan once the host bridge has been probed. Therefore, I’m not sure what you meant by a probe sequence defined in the DSDT for PCIe devices.
Regards,
Sunil
Re: [PATCH v16 6/9] irqchip: Add RISC-V advanced PLIC driver for direct-mode
Dear Sunil,
Quoted 43 lines
Quoted 37 lines
Quoted 4 lines
drivers/irqchip/irq-riscv-aplic-main.c +3
-0+static const struct of_device_id aplic_match[] = { + { .compatible ="riscv,aplic" }, + {}drivers/irqchip/irq-riscv-aplic-main.c line 211
Quoted 10 lines
drivers/irqchip/irq-riscv-aplic-main.c +10
-0+}; + +static struct platform_driver aplic_driver = { + .driver = { + .name ="riscv-aplic" , + .of_match_table = aplic_match, + }, + .probe = aplic_probe, +}; +builtin_platform_driver (aplic_driver);Dear Anup Patel and all concerned,
I am writing to inquire about the historical rationale behind defining the APLIC driver's initialization priority using
builtin_platform_driverin the current implementation.In our environment, we are encountering an issue where this priority level causes ACPI-based PCIe enumeration to be executed in the
system_unbound_wqwork queue. This parallel execution model results in PCIe devices being enumerated in an arbitrary order rather than strictly following the sequence defined in the ACPI DSDT table.The random enumeration order is adversely affecting customer experience, particularly in scenarios where device ordering is critical for proper system operation or application compatibility.
We are considering modifying the APLIC driver's initialization priority to ensure PCIe enumeration occurs sequentially according to the DSDT specification. However, before proceeding with such changes, we wanted to consult with you regarding:
1. Were there specific technical considerations that led to the current priority selection?
2. Are there any potential side effects or broader impacts that we might have overlooked?
3. Would you support such a priority adjustment, or do you have alternative suggestions to address the enumeration order issue?Quoted 5 lines
We greatly appreciate your insights and expertise on this matter, as it will help us make an informed decision while maintaining system stability and compatibility.
Thank you for your time and consideration.
Quoted 15 lines
IRQ subsystem maintainers rejected the idea of relying on initcalls to enforce probe order because initcalls do not guarantee ordering. The Linux driver model instead ensures probe order through device dependencies. Since PCI INTx depends on the APLIC being probed first, the PCI host bridge probe cannot occur until after the APLIC probe completes. This requirement and behavior are the same for both DT and ACPI. In DT, the driver model uses
fw_devlinkto establish probe ordering, while in ACPI this is handled through either an explicit_DEPor, on RISC-V, the GSI mapping.
Typically, this dependency appears in the DSDT only for the PCI host bridge. Individual PCIe devices are enumerated through the standard PCI scan once the host bridge has been probed. Therefore, I’m not sure what you meant by a probe sequence defined in the DSDT for PCIe devices. Regards,
Sunil
I understand the scenario you described with a single PCI host bridge, where devices are enumerated through standard PCIe scanning after the host bridge completes probing. However, in ARM and RISC-V architectures, systems often have multiple PCI host bridges. We're currently facing an issue in a 6-host-bridge system where all bridges depend on the APLIC driver. They must wait until the APLIC driver completes and callsacpi_dev_clear_dependenciesto resolve dependencies, after which they're sequentially added to the system_unbound_wq work queue. However, during execution in the work queue, these 6 host bridges undergo parallel enumeration, preventing them from following the order defined in the firmware's ACPI DSDT table. Specifically:
1. The ACPI DSDT table declares the 6 host bridges in a fixed sequence
Device(PC06) {
Name(_HID, "PNP0A08")
Name(_CID, "PNP0A03")
Name(_UID, 0x6)
......
}
Device(PC07) {
Name(_HID, "PNP0A08")
Name(_CID, "PNP0A03")
Name(_UID, 0x7)
......
}
Device(PC08) {
Name(_HID, "PNP0A08")
Name(_CID, "PNP0A03")
Name(_UID, 0x8)
.....
}
...
Device(PC11) {
Name(_HID, "PNP0A08")
Name(_CID, "PNP0A03")
Name(_UID, 0xB)......
}
2. But the OS enumerates them in random order upon each boot (first boot sequence ≠ second boot sequence) first boot sequence ~ # dmesg |grep -i "PCI Root"
[ 8794.588531] ACPI: PCI Root Bridge [PC08] (domain 0006 [bus 80-ff]) [ 8794.624478] ACPI: PCI Root Bridge [PC06] (domain 0005 [bus 00-ff]) [ 8794.672741] ACPI: PCI Root Bridge [PC10] (domain 0008 [bus 00-ff]) [ 8794.696680] ACPI: PCI Root Bridge [PC07] (domain 0006 [bus 00-7f]) [ 8794.728234] ACPI: PCI Root Bridge [PC11] (domain 0009 [bus 00-ff]) [ 8794.755098] ACPI: PCI Root Bridge [PC09] (domain 0007 [bus 00-ff]) second boot sequence ~ # dmesg |grep -i "PCI Root" [ 8794.588531] ACPI: PCI Root Bridge [PC09] (domain 0007 [bus 00-ff]) [ 8794.624478] ACPI: PCI Root Bridge [PC06] (domain 0005 [bus 00-ff]) [ 8794.672741] ACPI: PCI Root Bridge [PC08] (domain 0006 [bus 80-ff]) [ 8794.696680] ACPI: PCI Root Bridge [PC11] (domain 0009 [bus 00-ff]) [ 8794.728234] ACPI: PCI Root Bridge [PC07] (domain 0006 [bus 00-7f]) [ 8794.755098] ACPI: PCI Root Bridge [PC10] (domain 0008 [bus 00-ff])
This creates a critical issue: when NVMe devices are connected to these host bridges, the unpredictable kernel scanning sequence causes device identifiers (e.g., /dev/nvme0n1, /dev/nvme1n1) to change across reboots. In server environments, such device naming instability is unacceptable as it breaks storage configuration reliability and consistency.
So far, we've only observed this disorderly enumeration in RISC-V multi-host-bridge scenarios, where APLIC dependency leads to enumeration via system_unbound_wq. We'd like to consult kernel experts:
1. Has the impact of enumeration disorder in multi-host-bridge scenarios been considered?
2. Are there viable solutions to address the random enumeration caused by system_unbound_wq?
Best regards,
Liqiang
Re: [PATCH v16 6/9] irqchip: Add RISC-V advanced PLIC driver for direct-mode
Quoted 1 line · niliqiang
[...]
Quoted 16 lines
2. But the OS enumerates them in random order upon each boot (first boot sequence ≠ second boot sequence) first boot sequence ~ # dmesg |grep -i "PCI Root"
[ 8794.588531] ACPI: PCI Root Bridge [PC08] (domain 0006 [bus 80-ff]) [ 8794.624478] ACPI: PCI Root Bridge [PC06] (domain 0005 [bus 00-ff]) [ 8794.672741] ACPI: PCI Root Bridge [PC10] (domain 0008 [bus 00-ff]) [ 8794.696680] ACPI: PCI Root Bridge [PC07] (domain 0006 [bus 00-7f]) [ 8794.728234] ACPI: PCI Root Bridge [PC11] (domain 0009 [bus 00-ff]) [ 8794.755098] ACPI: PCI Root Bridge [PC09] (domain 0007 [bus 00-ff]) second boot sequence ~ # dmesg |grep -i "PCI Root" [ 8794.588531] ACPI: PCI Root Bridge [PC09] (domain 0007 [bus 00-ff]) [ 8794.624478] ACPI: PCI Root Bridge [PC06] (domain 0005 [bus 00-ff]) [ 8794.672741] ACPI: PCI Root Bridge [PC08] (domain 0006 [bus 80-ff]) [ 8794.696680] ACPI: PCI Root Bridge [PC11] (domain 0009 [bus 00-ff]) [ 8794.728234] ACPI: PCI Root Bridge [PC07] (domain 0006 [bus 00-7f]) [ 8794.755098] ACPI: PCI Root Bridge [PC10] (domain 0008 [bus 00-ff])
Quoted 6 lines
This creates a critical issue: when NVMe devices are connected to these host bridges, the unpredictable kernel scanning sequence causes device identifiers (e.g.,
/dev/nvme0n1,/dev/nvme1n1) to change across reboots. In server environments, such device naming instability is unacceptable as it breaks storage configuration reliability and consistency.
You're barking up the wrong tree. It is *userspace*'s job to ensure this consistency. That's why udev exists. That's why device and partition UUIDs exist. In short, this isn't the kernel's job. The kernel gives you all the tools you need already.
Thanks,
M.
Signature · 1 line
Jazz isn't dead. It just smells funny.
Re: [PATCH v16 6/9] irqchip: Add RISC-V advanced PLIC driver for direct-mode
Dear Marc and all concerned,
Quoted 115 lines
Quoted 43 lines
Quoted 37 lines
Quoted 4 lines
drivers/irqchip/irq-riscv-aplic-main.c +3
-0+static const struct of_device_id aplic_match[] = { + { .compatible ="riscv,aplic" }, + {}drivers/irqchip/irq-riscv-aplic-main.c line 211
Quoted 10 lines
drivers/irqchip/irq-riscv-aplic-main.c +10
-0+}; + +static struct platform_driver aplic_driver = { + .driver = { + .name ="riscv-aplic" , + .of_match_table = aplic_match, + }, + .probe = aplic_probe, +}; +builtin_platform_driver (aplic_driver);Dear Anup Patel and all concerned,
I am writing to inquire about the historical rationale behind defining the APLIC driver's initialization priority using
builtin_platform_driverin the current implementation.In our environment, we are encountering an issue where this priority level causes ACPI-based PCIe enumeration to be executed in the
system_unbound_wqwork queue. This parallel execution model results in PCIe devices being enumerated in an arbitrary order rather than strictly following the sequence defined in the ACPI DSDT table.The random enumeration order is adversely affecting customer experience, particularly in scenarios where device ordering is critical for proper system operation or application compatibility.
We are considering modifying the APLIC driver's initialization priority to ensure PCIe enumeration occurs sequentially according to the DSDT specification. However, before proceeding with such changes, we wanted to consult with you regarding:
1. Were there specific technical considerations that led to the current priority selection?
2. Are there any potential side effects or broader impacts that we might have overlooked?
3. Would you support such a priority adjustment, or do you have alternative suggestions to address the enumeration order issue?Quoted 5 lines
We greatly appreciate your insights and expertise on this matter, as it will help us make an informed decision while maintaining system stability and compatibility.
Thank you for your time and consideration.
Quoted 15 lines
IRQ subsystem maintainers rejected the idea of relying on initcalls to enforce probe order because initcalls do not guarantee ordering. The Linux driver model instead ensures probe order through device dependencies. Since PCI INTx depends on the APLIC being probed first, the PCI host bridge probe cannot occur until after the APLIC probe completes. This requirement and behavior are the same for both DT and ACPI. In DT, the driver model uses
fw_devlinkto establish probe ordering, while in ACPI this is handled through either an explicit_DEPor, on RISC-V, the GSI mapping.
Typically, this dependency appears in the DSDT only for the PCI host bridge. Individual PCIe devices are enumerated through the standard PCI scan once the host bridge has been probed. Therefore, I’m not sure what you meant by a probe sequence defined in the DSDT for PCIe devices. Regards,
SunilI understand the scenario you described with a single PCI host bridge, where devices are enumerated through standard PCIe scanning after the host bridge completes probing. However, in ARM and RISC-V architectures, systems often have multiple PCI host bridges. We're currently facing an issue in a 6-host-bridge system where all bridges depend on the APLIC driver. They must wait until the APLIC driver completes and
callsacpi_dev_clear_dependenciestoresolve dependencies, after which they're sequentially added to thesystem_unbound_wqwork queue. However, during execution in the work queue, these 6 host bridges undergo parallel enumeration, preventing them from following the order defined in the firmware's ACPI DSDT table. Specifically:
1. The ACPI DSDT table declares the 6 host bridges in a fixed sequenceDevice(PC06) { Name(_HID, "PNP0A08") Name(_CID, "PNP0A03") Name(_UID, 0x6) ...... } Device(PC07) { Name(_HID, "PNP0A08") Name(_CID, "PNP0A03") Name(_UID, 0x7) ...... } Device(PC08) { Name(_HID, "PNP0A08") Name(_CID, "PNP0A03") Name(_UID, 0x8) ..... } ... Device(PC11) { Name(_HID, "PNP0A08") Name(_CID, "PNP0A03") Name(_UID, 0xB)......
}2. But the OS enumerates them in random order upon each boot (first boot sequence ≠ second boot sequence) first boot sequence ~ # dmesg |grep -i "PCI Root"
[ 8794.588531] ACPI: PCI Root Bridge [PC08] (domain 0006 [bus 80-ff]) [ 8794.624478] ACPI: PCI Root Bridge [PC06] (domain 0005 [bus 00-ff]) [ 8794.672741] ACPI: PCI Root Bridge [PC10] (domain 0008 [bus 00-ff]) [ 8794.696680] ACPI: PCI Root Bridge [PC07] (domain 0006 [bus 00-7f]) [ 8794.728234] ACPI: PCI Root Bridge [PC11] (domain 0009 [bus 00-ff]) [ 8794.755098] ACPI: PCI Root Bridge [PC09] (domain 0007 [bus 00-ff]) second boot sequence ~ # dmesg |grep -i "PCI Root" [ 8794.588531] ACPI: PCI Root Bridge [PC09] (domain 0007 [bus 00-ff]) [ 8794.624478] ACPI: PCI Root Bridge [PC06] (domain 0005 [bus 00-ff]) [ 8794.672741] ACPI: PCI Root Bridge [PC08] (domain 0006 [bus 80-ff]) [ 8794.696680] ACPI: PCI Root Bridge [PC11] (domain 0009 [bus 00-ff]) [ 8794.728234] ACPI: PCI Root Bridge [PC07] (domain 0006 [bus 00-7f]) [ 8794.755098] ACPI: PCI Root Bridge [PC10] (domain 0008 [bus 00-ff])This creates a critical issue: when NVMe devices are connected to these host bridges, the unpredictable kernel scanning sequence causes device identifiers (e.g.,
/dev/nvme0n1,/dev/nvme1n1) to change across reboots. In server environments, such device naming instability is unacceptable as it breaks storage configuration reliability and consistency.
Quoted 4 lines
So far, we've only observed this disorderly enumeration in RISC-V multi-host-bridge scenarios, where APLIC dependency leads to enumeration via
system_unbound_wq. We'd like to consult kernel experts:
1. Has the impact of enumeration disorder in multi-host-bridge scenarios been considered?
2. Are there viable solutions to address the random enumeration caused bysystem_unbound_wq?
Thank you for your previous suggestion regarding the use of udev. We have carefully implemented this approach, but found that while udev can create symbolic links, it doesn't resolve the underlying issue of device ordering in lsblk output, which remains important for our use case.
In our large-scale server deployment comprising hundreds of machines, the udev approach presents several operational challenges:
1. Management Complexity: Maintaining per-machine udev rules across our entire infrastructure is not scalable.
2. Customer Workflow Dependencies: Our customers rely on consistent lsblk output and direct device names for their operational workflows and automated scripts.
3. Scalability Limitations: We use standardized hardware configurations where consistent behavior should be the default, rather than requiring per-machine customization.
We've noticed that PCIe enumeration order tends to vary across system reboots (for example: first boot showed PC08->PC06->PC10->PC07->PC11->PC09, while second boot showed PC09->PC06->PC08->PC11->PC07->PC10), even though the ACPI firmware consistently reports the root bridge sequence as PC06->PC07->PC08->PC09->PC10->PC11.
In our testing, we found that adjusting the registration priority of the aplic driver seems to help ensure the interrupt controller initializes before PCI enumeration, leading to more consistent device ordering.
We'd be grateful for your expert advice on a few questions:
1. Are there any dependencies or implications we might have missed with this priority adjustment?
2. Would you happen to know of any other kernel-level approaches that could help with enumeration ordering?
3. We're curious if there might be any ongoing discussions about making enumeration more deterministic for standardized server environments?
We fully understand there are good reasons behind the current design, but the variable ordering has been somewhat challenging for our users. We'd appreciate any guidance you might offer on the best way forward.
Thank you for your time and valuable insights.
Best regards,
Liqiang
Re: [PATCH v16 6/9] irqchip: Add RISC-V advanced PLIC driver for direct-mode
Quoted 8 lines · niliqiang
We've noticed that PCIe enumeration order tends to vary across system reboots (for example: first boot showed
PC08->PC06->PC10->PC07->PC11->PC09, while second boot showedPC09->PC06->PC08->PC11->PC07->PC10), even though the ACPI firmware consistently reports the root bridge sequence asPC06->PC07->PC08->PC09->PC10->PC11.In our testing, we found that adjusting the registration priority of the aplic driver seems to help ensure the interrupt controller initializes before PCI enumeration, leading to more consistent device ordering.
You're repeating yourself over and over, but you're missing the point.
The ACPI table does not provide a sequence, it provides a collection and it's nowhere written in stone that this collection has to be processed in order.
If you want a sequence then put dependencies into the table and be done with it, no?
Thanks,
tglx
Re: [PATCH v16 6/9] irqchip: Add RISC-V advanced PLIC driver for direct-mode
Hi, tglx
Quoted 9 lines
Quoted 8 lines
We've noticed that PCIe enumeration order tends to vary across system reboots (for example: first boot showed
PC08->PC06->PC10->PC07->PC11->PC09, while second boot showedPC09->PC06->PC08->PC11->PC07->PC10), even though the ACPI firmware consistently reports the root bridge sequence asPC06->PC07->PC08->PC09->PC10->PC11.In our testing, we found that adjusting the registration priority of the aplic driver seems to help ensure the interrupt controller initializes before PCI enumeration, leading to more consistent device ordering.
Quoted 5 lines
The ACPI table does not provide a sequence, it provides a collection and it's nowhere written in stone that this collection has to be processed in order.
If you want a sequence then put dependencies into the table and be done with it, no?
Thank you for your suggestions.
Through our validation, We have observed that the current kernel appears to lack an effective dependency resolution mechanism between ACPI PCIe host bridge devices.
The specific manifestation is as follows: when establishing dependencies between host bridges via the _DEP method (for example, PC07 depends on PC06), only PC06 is enumerated successfully, while PC07 is skipped. Our investigation reveals that the root cause lies in the current limitation where ACPI PCIe devices have only a single opportunity to resolve dependencies—specifically within the acpi_dev_clear_dependencies function inside the APLIC driver. PC06, having its dependency resolved earlier by the APLIC driver, is asynchronously processed in the system_unbound_wq queue. At this point, PC07 still depends on PC06, which has not yet completed enumeration. As a result, PC07 cannot proceed and subsequently misses its enumeration opportunity.
To address this issue, we have introduced the following dependency resolution mechanism at the end of the acpi_pci_root_add function in the kernel's PCIe enumeration path to achieve the desired fixed-order enumeration:
#ifdef CONFIG_ACPI
if (!acpi_disabled)acpi_dev_clear_dependencies(device);
#endif
We have observed that other drivers in the kernel also utilize acpi_dev_clear_dependencies to handle device dependencies. We seek confirmation on whether this approach is reasonable and welcome feedback regarding alternative effective solutions.
Best regards,
Liqiang
PATCH v16 7/9 irqchip/riscv-aplic: Add support for MSI-mode
In reply to Anup PatelThe RISC-V advanced platform-level interrupt controller (APLIC) has two modes of operation: 1) Direct mode and 2) MSI mode. (For more details, refer https://github.com/riscv/riscv-aia)
In APLIC MSI-mode, wired interrupts are forwared as message signaled interrupts (MSIs) to CPUs via IMSIC.
Extend the existing APLIC irqchip driver to support MSI-mode for RISC-V platforms having both wired interrupts and MSIs.
--- drivers/irqchip/Kconfig | 6 + drivers/irqchip/Makefile | 1 + drivers/irqchip/irq-riscv-aplic-main.c | 2 +- drivers/irqchip/irq-riscv-aplic-main.h | 8 + drivers/irqchip/irq-riscv-aplic-msi.c | 257 +++++++++++++++++++++++++ 5 files changed, 273 insertions(+), 1 deletion(-) create mode 100644 drivers/irqchip/irq-riscv-aplic-msi.c
drivers/irqchip/Kconfig +6 -0
@@ -551,6 +551,12 @@ config RISCV_APLIC depends on RISCV select IRQ_DOMAIN_HIERARCHY +config RISCV_APLIC_MSI + bool + depends on RISCV_APLIC + select GENERIC_MSI_IRQ + default RISCV_APLIC + config RISCV_IMSIC bool depends on RISCV
drivers/irqchip/Makefile +1 -0
@@ -96,6 +96,7 @@ obj-$(CONFIG_CSKY_MPINTC) += irq-csky-mpintc.o obj-$(CONFIG_CSKY_APB_INTC) += irq-csky-apb-intc.o obj-$(CONFIG_RISCV_INTC) += irq-riscv-intc.o obj-$(CONFIG_RISCV_APLIC) += irq-riscv-aplic-main.o irq-riscv-aplic-direct.o +obj-$(CONFIG_RISCV_APLIC_MSI) += irq-riscv-aplic-msi.o obj-$(CONFIG_RISCV_IMSIC) += irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o obj-$(CONFIG_SIFIVE_PLIC) += irq-sifive-plic.o obj-$(CONFIG_IMX_IRQSTEER) += irq-imx-irqsteer.o
drivers/irqchip/irq-riscv-aplic-main.c +1 -1
@@ -187,7 +187,7 @@ static int aplic_probe(struct platform_device *pdev)if (is_of_node (dev->fwnode)) msi_mode =of_property_present (to_of_node (dev->fwnode),"msi-parent" );if (msi_mode) - rc = -ENODEV ; + rc =aplic_msi_setup (dev, regs);else rc =aplic_direct_setup (dev, regs);if (rc)
drivers/irqchip/irq-riscv-aplic-main.h +8 -0
@@ -40,5 +40,13 @@ int aplic_irqdomain_translate(struct irq_fwspec *fwspec, u32 gsi_base,void aplic_init_hw_global (struct aplic_priv *priv,bool msi_mode);int aplic_setup_priv (struct aplic_priv *priv,struct device *dev,void __iomem *regs);int aplic_direct_setup (struct device *dev,void __iomem *regs); +#ifdef CONFIG_RISCV_APLIC_MSI +int aplic_msi_setup (struct device *dev,void __iomem *regs); +#else +static inline int aplic_msi_setup (struct device *dev,void __iomem *regs) +{ +return -ENODEV ; +} +#endif #endif
drivers/irqchip/irq-riscv-aplic-msi.c new file +257 -0
257 lines not on this page · raw message
Signature · 1 line
2.34.1
[tip: irq/core] irqchip/riscv-aplic: Add support for MSI-mode
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: ca8df97fe6798afbe395fc4a8e23bac0c7fbd248 Gitweb: https://git.kernel.org/tip/ca8df97fe6798afbe395fc4a8e23bac0c7fbd248 Author: Anup Patel <[email protected]> AuthorDate: Thu, 07 Mar 2024 19:33:05 +05:30 Committer: Thomas Gleixner <[email protected]> CommitterDate: Mon, 25 Mar 2024 17:38:29 +01:00
irqchip/riscv-aplic: Add support for MSI-mode
The RISC-V advanced platform-level interrupt controller (APLIC) has two modes of operation: 1) Direct mode and 2) MSI mode. (For more details, refer https://github.com/riscv/riscv-aia)
In APLIC MSI-mode, wired interrupts are forwared as message signaled interrupts (MSIs) to CPUs via IMSIC.
Extend the existing APLIC irqchip driver to support MSI-mode for RISC-V platforms having both wired interrupts and MSIs.
--- drivers/irqchip/Kconfig | 6 +- drivers/irqchip/Makefile | 1 +- drivers/irqchip/irq-riscv-aplic-main.c | 2 +- drivers/irqchip/irq-riscv-aplic-main.h | 8 +- drivers/irqchip/irq-riscv-aplic-msi.c | 257 ++++++++++++++++++++++++- 5 files changed, 273 insertions(+), 1 deletion(-) create mode 100644 drivers/irqchip/irq-riscv-aplic-msi.c
drivers/irqchip/Kconfig +6 -0
@@ -545,6 +545,12 @@ config RISCV_APLIC depends on RISCV select IRQ_DOMAIN_HIERARCHY +config RISCV_APLIC_MSI + bool + depends on RISCV_APLIC + select GENERIC_MSI_IRQ + default RISCV_APLIC + config RISCV_IMSIC bool depends on RISCV
drivers/irqchip/Makefile +1 -0
@@ -96,6 +96,7 @@ obj-$(CONFIG_CSKY_MPINTC) += irq-csky-mpintc.o obj-$(CONFIG_CSKY_APB_INTC) += irq-csky-apb-intc.o obj-$(CONFIG_RISCV_INTC) += irq-riscv-intc.o obj-$(CONFIG_RISCV_APLIC) += irq-riscv-aplic-main.o irq-riscv-aplic-direct.o +obj-$(CONFIG_RISCV_APLIC_MSI) += irq-riscv-aplic-msi.o obj-$(CONFIG_RISCV_IMSIC) += irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o obj-$(CONFIG_SIFIVE_PLIC) += irq-sifive-plic.o obj-$(CONFIG_STARFIVE_JH8100_INTC) += irq-starfive-jh8100-intc.o
drivers/irqchip/irq-riscv-aplic-main.c +1 -1
@@ -187,7 +187,7 @@ static int aplic_probe(struct platform_device *pdev)if (is_of_node (dev->fwnode)) msi_mode =of_property_present (to_of_node (dev->fwnode),"msi-parent" );if (msi_mode) - rc = -ENODEV ; + rc =aplic_msi_setup (dev, regs);else rc =aplic_direct_setup (dev, regs);if (rc)
drivers/irqchip/irq-riscv-aplic-main.h +8 -0
@@ -40,5 +40,13 @@ int aplic_irqdomain_translate(struct irq_fwspec *fwspec, u32 gsi_base,void aplic_init_hw_global (struct aplic_priv *priv,bool msi_mode);int aplic_setup_priv (struct aplic_priv *priv,struct device *dev,void __iomem *regs);int aplic_direct_setup (struct device *dev,void __iomem *regs); +#ifdef CONFIG_RISCV_APLIC_MSI +int aplic_msi_setup (struct device *dev,void __iomem *regs); +#else +static inline int aplic_msi_setup (struct device *dev,void __iomem *regs) +{ +return -ENODEV ; +} +#endif #endif
drivers/irqchip/irq-riscv-aplic-msi.c new file +257 -0
257 lines not on this page · raw message
PATCH v16 8/9 RISC-V: Select APLIC and IMSIC drivers
In reply to Anup PatelThe QEMU virt machine supports AIA emulation and quite a few RISC-V platforms with AIA support are under development so select APLIC and IMSIC drivers for all RISC-V platforms.
--- arch/riscv/Kconfig | 2 ++ 1 file changed, 2 insertions(+)
arch/riscv/Kconfig +2 -0
@@ -162,6 +162,8 @@ config RISCV select PCI_DOMAINS_GENERIC if PCI select PCI_MSI if PCI select RISCV_ALTERNATIVE if !XIP_KERNEL + select RISCV_APLIC + select RISCV_IMSIC select RISCV_INTC select RISCV_TIMER if RISCV_SBI select SIFIVE_PLIC
Signature · 1 line
2.34.1
[tip: irq/core] RISC-V: Select APLIC and IMSIC drivers
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 0eebc69db358fd2f6fe34cc4db6428df6a540dd7 Gitweb: https://git.kernel.org/tip/0eebc69db358fd2f6fe34cc4db6428df6a540dd7 Author: Anup Patel <[email protected]> AuthorDate: Thu, 07 Mar 2024 19:33:06 +05:30 Committer: Thomas Gleixner <[email protected]> CommitterDate: Mon, 25 Mar 2024 17:38:29 +01:00
RISC-V: Select APLIC and IMSIC drivers
The QEMU virt machine supports AIA emulation and quite a few RISC-V platforms with AIA support are under development so select APLIC and IMSIC drivers for all RISC-V platforms.
--- arch/riscv/Kconfig | 2 ++ 1 file changed, 2 insertions(+)
arch/riscv/Kconfig +2 -0
@@ -173,6 +173,8 @@ config RISCV select PCI_DOMAINS_GENERIC if PCI select PCI_MSI if PCI select RISCV_ALTERNATIVE if !XIP_KERNEL + select RISCV_APLIC + select RISCV_IMSIC select RISCV_INTC select RISCV_TIMER if RISCV_SBI select SIFIVE_PLIC
PATCH v16 9/9 MAINTAINERS: Add entry for RISC-V AIA drivers
In reply to Anup PatelAdd myself as maintainer for RISC-V AIA drivers including the RISC-V INTC driver which supports both AIA and non-AIA platforms.
--- MAINTAINERS | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
MAINTAINERS +14 -0
@@ -18803,6 +18803,20 @@ S: Maintained F: drivers/mtd/nand/raw/r852.c F: drivers/mtd/nand/raw/r852.h +RISC-V AIA DRIVERS +M: Anup Patel <[email protected]> +L: [email protected] +S: Maintained +F: Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml +F: Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml +F: drivers/irqchip/irq-riscv-aplic-*.c +F: drivers/irqchip/irq-riscv-aplic-*.h +F: drivers/irqchip/irq-riscv-imsic-*.c +F: drivers/irqchip/irq-riscv-imsic-*.h +F: drivers/irqchip/irq-riscv-intc.c +F: include/linux/irqchip/riscv-aplic.h +F: include/linux/irqchip/riscv-imsic.h + RISC-V ARCHITECTURE M: Paul Walmsley <[email protected]> M: Palmer Dabbelt <[email protected]>
Signature · 1 line
2.34.1
[tip: irq/core] MAINTAINERS: Add entry for RISC-V AIA drivers
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: f4e116b2c5eec0dc1ab65e7105c180d4f8627b39 Gitweb: https://git.kernel.org/tip/f4e116b2c5eec0dc1ab65e7105c180d4f8627b39 Author: Anup Patel <[email protected]> AuthorDate: Thu, 07 Mar 2024 19:33:07 +05:30 Committer: Thomas Gleixner <[email protected]> CommitterDate: Mon, 25 Mar 2024 17:38:29 +01:00
MAINTAINERS: Add entry for RISC-V AIA drivers
Add myself as maintainer for RISC-V AIA drivers including the RISC-V INTC driver which supports both AIA and non-AIA platforms.
--- MAINTAINERS | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
MAINTAINERS +14 -0
@@ -18922,6 +18922,20 @@ S: Maintained F: drivers/mtd/nand/raw/r852.c F: drivers/mtd/nand/raw/r852.h +RISC-V AIA DRIVERS +M: Anup Patel <[email protected]> +L: [email protected] +S: Maintained +F: Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml +F: Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml +F: drivers/irqchip/irq-riscv-aplic-*.c +F: drivers/irqchip/irq-riscv-aplic-*.h +F: drivers/irqchip/irq-riscv-imsic-*.c +F: drivers/irqchip/irq-riscv-imsic-*.h +F: drivers/irqchip/irq-riscv-intc.c +F: include/linux/irqchip/riscv-aplic.h +F: include/linux/irqchip/riscv-imsic.h + RISC-V ARCHITECTURE M: Paul Walmsley <[email protected]> M: Palmer Dabbelt <[email protected]>
Hi Thomas,
Quoted 161 lines · Anup Patel
The RISC-V AIA specification is ratified as-per the RISC-V international process. The latest ratified AIA specifcation can be found at:
https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interrupts-1.0.pdfAt a high-level, the AIA specification adds three things:
1) AIA CSRs
- Improved local interrupt support
2) Incoming Message Signaled Interrupt Controller (IMSIC)
- Per-HART MSI controller
- Support MSI virtualization
- Support IPI along with virtualization
3) Advanced Platform-Level Interrupt Controller (APLIC)
- Wired interrupt controller
- In MSI-mode, converts wired interrupt into MSIs (i.e. MSI generator)
- In Direct-mode, injects external interrupts directly into HARTsFor an overview of the AIA specification, refer the AIA virtualization talk at KVM Forum 2022:
https://static.sched.com/hosted_files/kvmforum2022/a1/AIA_Virtualization_in_KVM_RISCV_final.pdf
https://www.youtube.com/watch?v=r071dL8Z0yoTo test this series, use QEMU v7.2 (or higher) and OpenSBI v1.2 (or higher).
This series depends upon per-device MSI domain (and few other) patches merged by Thomas (tglx) which are available in irq/msi branch at:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitThese patches can also be found in the
riscv_aia_v16branch at:
https://github.com/avpatel/linux.gitChanges since v15:
- Dropped PATCH1 since it is already merged by Thomas (tglx) and available in his irq/msi branch atgit://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
- Simplifiedaplic_msi_irq_eoi()in PATCH7 based on the section "4.9.2 Special consideration for level-sensitive interrupt sources" of the RISC-V AIA specification and also provided handler name.Changes since v14:
- Dropped 9 patches which are already merged by Thomas (tglx) and available in his irq/msi branch atgit://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
- Added new PATCH1 which adds changes missed out in merging AIA support for RISC-V INTC.
- Added a separate cpuhp state for IMSIC driver in PATCH3 which ensures that cpuhp notifiers of IMSIC are called as early as possible.
- Removed redundant barriers in PATCH3.
- Addressed few other nit comments.Changes since v13:
- Split PATCH1 into six granular patches
- Addressed nit comments from Thomas and BjornChanges since v12:
- Rebased on Linux-6.8-rc5
- Dropped per-device MSI domain patches which are already merged by Thomas (tglx)
- Addressed nit comments from Thomas and Clement
- Added a new patch2 to fix lock dependency warning
- Replaced local sync IPI in the IMSIC driver with per-CPU timer
- Simplified locking in the IMSIC driver to avoid lock dependency issues
- Added a dirty bitmap in the IMSIC driver to optimize per-CPU local sync loopChanges since v11:
- Rebased on Linux-6.8-rc1
- Included kernel/irq related patches from "genirq, irqchip: Convert ARM MSI handling to per device MSI domains" series by Thomas. (PATCH7, PATCH8, PATCH9, PATCH14, PATCH16, PATCH17, PATCH18, PATCH19,PATCH20, PATCH21, PATCH22, PATCH23, and PATCH32 of https://lore.kernel.org/linux-arm-kernel/[email protected]/)- Updated APLIC MSI-mode driver to use the new
WIRED_TO_MSImechanism.
- Updated IMSIC driver to support per-device MSI domains for PCI and platform devices.Changes since v10:
- Rebased on Linux-6.6-rc7
- Dropped PATCH3 of v10 series since this has been merged by MarcZ for Linux-6.6-rc7
- Changed the IMSIC ID management strategy from 1-n approach to x86-style 1-1 approachChanges since v9:
- Rebased on Linux-6.6-rc4
- Usebuiltin_platform_driver()in PATCH5, PATCH9, and PATCH12Changes since v8:
- Rebased on Linux-6.6-rc3
- Dropped PATCH2 of v8 series since we won't be requiringriscv_get_intc_hartid()based on Marc Z's comments on ACPI AIA support.
- Addressed Saravana's comments in PATCH3 of v8 series
- Update PATCH9 and PATCH13 of v8 series based on comments from SunilChanges since v7:
- Rebased on Linux-6.6-rc1
- Addressed comments on PATCH1 of v7 series and split it into two PATCHes
- UseDEFINE_SIMPLE_PROP()in PATCH2 of v7 seriesChanges since v6:
- Rebased on Linux-6.5-rc4
- Updated PATCH2 to useIS_ENABLED(CONFIG_SPARC)instead of !IS_ENABLED(CONFIG_OF_IRQ)
- Added new PATCH4 to fix syscore registration in PLIC driver
- Update PATCH5 to convert PLIC driver into full-blown platform driver with a re-written probe function.Changes since v5:
- Rebased on Linux-6.5-rc2
- Updated the overall series to ensure that only IPI, timer, and INTC drivers are probed very early whereas rest of the interrupt controllers (such as PLIC, APLIC, and IMISC) are probed as regular platform drivers.
- Renamedriscv_fw_parent_hartid()toriscv_get_intc_hartid()
- New PATCH1 to addfw_devlinksupport for msi-parent DT property
- New PATCH2 to ensure all INTC suppliers are initialized which in-turn fixes the probing issue for PLIC, APLIC and IMSIC as platform driver
- New PATCH3 to use platform driver probing for PLIC
- Re-structured the IMSIC driver into two separate drivers: early and platform. The IMSIC early driver (PATCH7) only initialized IMSIC state and provides IPIs whereas the IMSIC platform driver (PATCH8) is probed provides MSI domain for platform devices.
- Re-structure the APLIC platform driver into three separe sources: main, direct mode, and MSI mode.Changes since v4:
- Rebased on Linux-6.5-rc1
- Added "Dependencies" in the APLIC bindings (PATCH6 in v4)
- Dropped the PATCH6 which was changing the IOMMU DMA domain APIs
- Dropped use of IOMMU DMA APIs in the IMSIC driver (PATCH4)Changes since v3:
- Rebased on Linux-6.4-rc6
- Dropped PATCH2 of v3 series instead we now setFWNODE_FLAG_BEST_EFFORTviaIRQCHIP_DECLARE()
- Extendriscv_fw_parent_hartid()to support both DT and ACPI in PATCH1
- Extendiommu_dma_compose_msi_msg()instead of addingiommu_dma_select_msi()in PATCH6
- Addressed Conor's comments in PATCH3
- Addressed Conor's and Rob's comments in PATCH7Changes since v2:
- Rebased on Linux-6.4-rc1
- Addressed Rob's comments on DT bindings patches 4 and 8.
- Addessed Marc's comments on IMSIC driver PATCH5
- Replaced use of OF apis in APLIC and IMSIC drivers with FWNODE apis this makes both drivers easily portable for ACPI support. This also removes unnecessary indirection from the APLIC and IMSIC drivers.
- PATCH1 is a new patch for portability with ACPI support
- PATCH2 is a new patch to fix probing in APLIC drivers for APLIC-only systems.
- PATCH7 is a new patch which addresses the IOMMU DMA domain issues pointed out by SiFiveChanges since v1:
- Rebased on Linux-6.2-rc2
- Addressed comments on IMSIC DT bindings for PATCH4
- Useraw_spin_lock_irqsave()onids_lockfor PATCH5
- Improved MMIO alignment checks in PATCH5 to allow MMIO regions with holes.
- Addressed comments on APLIC DT bindings for PATCH6
- Fixed warning splat inaplic_msi_write_msg()caused by zeroed MSI message in PATCH7
- Dropped DT property riscv,slow-ipi instead will have module parameter in future.
Quoted 11 lines
Anup Patel (9):
dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller
irqchip: Add RISC-V incoming MSI controller early driver irqchip/riscv-imsic: Add device MSI domain support for platform
devices
irqchip/riscv-imsic: Add device MSI domain support for PCI devices
dt-bindings: interrupt-controller: Add RISC-V advanced PLIC
irqchip: Add RISC-V advanced PLIC driver for direct-mode irqchip/riscv-aplic: Add support for MSI-mode
RISC-V: Select APLIC and IMSIC drivers
MAINTAINERS: Add entry for RISC-V AIA drivers
Can this series be considered for Linux-6.9 ?
Regards,
Anup
Quoted 35 lines
.../interrupt-controller/riscv,aplic.yaml | 172 ++++ .../interrupt-controller/riscv,imsics.yaml | 172 ++++ MAINTAINERS | 14 + arch/riscv/Kconfig | 2 + drivers/irqchip/Kconfig | 25 + drivers/irqchip/Makefile | 3 + drivers/irqchip/irq-riscv-aplic-direct.c | 326 +++++++ drivers/irqchip/irq-riscv-aplic-main.c | 211 +++++ drivers/irqchip/irq-riscv-aplic-main.h | 52 ++ drivers/irqchip/irq-riscv-aplic-msi.c | 257 ++++++ drivers/irqchip/irq-riscv-imsic-early.c | 201 ++++ drivers/irqchip/irq-riscv-imsic-platform.c | 374 ++++++++ drivers/irqchip/irq-riscv-imsic-state.c | 865 ++++++++++++++++++ drivers/irqchip/irq-riscv-imsic-state.h | 108 +++ include/linux/cpuhotplug.h | 1 + include/linux/irqchip/riscv-aplic.h | 145 +++ include/linux/irqchip/riscv-imsic.h | 87 ++ 17 files changed, 3015 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml create mode 100644 drivers/irqchip/irq-riscv-aplic-direct.c create mode 100644 drivers/irqchip/irq-riscv-aplic-main.c create mode 100644 drivers/irqchip/irq-riscv-aplic-main.h create mode 100644 drivers/irqchip/irq-riscv-aplic-msi.c create mode 100644 drivers/irqchip/irq-riscv-imsic-early.c create mode 100644 drivers/irqchip/irq-riscv-imsic-platform.c create mode 100644 drivers/irqchip/irq-riscv-imsic-state.c create mode 100644 drivers/irqchip/irq-riscv-imsic-state.h create mode 100644 include/linux/irqchip/riscv-aplic.h create mode 100644 include/linux/irqchip/riscv-imsic.h--
2.34.1
Quoted 2 lines · Anup Patel
Can this series be considered for Linux-6.9 ?
Sorry no. I stop taking patches which are not fixes or trivial cleanups at rc7 latest. I marked it for inclusion into my post-merge window tree.
Thanks,
tglx
Quoted 3 lines · Anup Patel
The RISC-V AIA specification is ratified as-per the RISC-V international process. The latest ratified AIA specifcation can be found at:
https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interrupts-1.0.pdf
It's been a been a long ride, and I got in late (v10)... Thanks for the hard work, Anup!
Note that Alex' text-patching/IPI series [1] needs to go into the RV tree at roughly the same time as AIA.
I've not caught any regressions for kselftests on qemu for this version (with Alex' series applied). I have not run it on real hardware.
For the series:
Quoted 8 lines · Björn Töpel
Quoted 3 lines · Anup Patel
The RISC-V AIA specification is ratified as-per the RISC-V international process. The latest ratified AIA specifcation can be found at:
https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interrupts-1.0.pdfIt's been a been a long ride, and I got in late (v10)... Thanks for the hard work, Anup!
Too excited! ;-)
s/a been//
Quoted 2 lines
Note that Alex' text-patching/IPI series [1] needs to go into the RV tree at roughly the same time as AIA.
...and missing reference:
[1] https://lore.kernel.org/linux-riscv/[email protected]/
Quoted 7 lines
I've not caught any regressions for kselftests on qemu for this version (with Alex' series applied). I have not run it on real hardware.
For the series:
Tested-by Björn Töpel <[email protected]>Reviewed-by Björn Töpel <[email protected]>
Have a nice w/e,
Björn