irqchip/riscv-intc: Mark INTC nodes for secondary CPUs as initialized.
Subsystems (3)
PATCH 1/1 irqchip/riscv-intc: Mark INTC nodes for secondary CPUs as initialized.
The current Linux driver irq-riscv-intc initialize IRQ domain only once, when init function called on primary hart. In other cases no IRQ domain is created and no operation on interrupt-controller node is performed. This is cause of that no common Linux driver can use per-cpu interrupts mapped to several CPUs because fwnode of secondary cores INTC is not marked as initialized. This device is always will be marked as deferred. For example the system with devicetree
cpu0: cpu@0 {
cpu0_intc: interrupt-controller {
interrupt-controller;
compatible = riscv,cpu-intc;
};
};
cpu1: cpu@1 {
cpu1_intc: interrupt-controller {
interrupt-controller;
compatible = riscv,cpu-intc;
};
};
buserr {
compatible = riscv,buserr;
interrupts-extended = <&cpu0_intc 16 &cpu1_intc 16>;
};will always report 'buserr' node as deferred without calling any bus probe function.
This patch will mark all secondary nodes passed to irq-riscv-intc driver init function as initialized to be able to act as correct IRQ phandle node.
--- drivers/irqchip/irq-riscv-intc.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/irqchip/irq-riscv-intc.c +6 -2
@@ -155,8 +155,10 @@ static int __init riscv_intc_init(struct device_node *node,* for each INTC DT node. We only need to do INTC initialization * for the INTC DT node belonging to boot CPU (or boot HART). */ -if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) +if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) { +fwnode_dev_initialized (of_node_to_fwnode (node), true);return 0 ; + }return riscv_intc_init_common (of_node_to_fwnode (node)); }
@@ -179,8 +181,10 @@ static int __init riscv_intc_acpi_init(union acpi_subtable_headers *header,* for each INTC. We only do INTC initialization * for the INTC belonging to the boot CPU (or boot HART). */ -if (riscv_hartid_to_cpuid (rintc->hart_id) !=smp_processor_id ()) +if (riscv_hartid_to_cpuid (rintc->hart_id) !=smp_processor_id ()) { +fwnode_dev_initialized (of_node_to_fwnode (node), true);return 0 ; + } fn =irq_domain_alloc_named_fwnode ("RISCV-INTC" );if (!fn) {
Signature · 7 lines
2.34.1 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 45 lines · Dmitry Dunaev
The current Linux driver irq-riscv-intc initialize IRQ domain only once, when init function called on primary hart. In other cases no IRQ domain is created and no operation on interrupt-controller node is performed. This is cause of that no common Linux driver can use per-cpu interrupts mapped to several CPUs because fwnode of secondary cores INTC is not marked as initialized. This device is always will be marked as deferred. For example the system with devicetree
cpu0: cpu@0 { cpu0_intc: interrupt-controller { interrupt-controller; compatible = riscv,cpu-intc; }; }; cpu1: cpu@1 { cpu1_intc: interrupt-controller { interrupt-controller; compatible = riscv,cpu-intc; }; }; buserr { compatible = riscv,buserr; interrupts-extended = <&cpu0_intc 16 &cpu1_intc 16>; };will always report 'buserr' node as deferred without calling any bus probe function.
This patch will mark all secondary nodes passed to irq-riscv-intc driver init function as initialized to be able to act as correct IRQ phandle node.
Signed-off-by Dmitry Dunaev <[email protected]>--- drivers/irqchip/irq-riscv-intc.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)drivers/irqchip/irq-riscv-intc.c +0
-0
drivers/irqchip/irq-riscv-intc.c static int __init riscv_intc_init(struct device_node *node, · line 159
Quoted 6 lines
drivers/irqchip/irq-riscv-intc.c +2
-1@@ -155,8 +155,10 @@ static int __init riscv_intc_init(struct device_node *node,* for each INTC DT node. We only need to do INTC initialization * for the INTC DT node belonging to boot CPU (or boot HART). */ -if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) +if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) { +fwnode_dev_initialized (of_node_to_fwnode (node), true);
There is already a patch on LKML to address this.
https://www.spinics.net/lists/kernel/msg4929886.html
Quoted 6 lines
drivers/irqchip/irq-riscv-intc.c +1
-0return 0 ; + }return riscv_intc_init_common (of_node_to_fwnode (node)); }
drivers/irqchip/irq-riscv-intc.c static int __init riscv_intc_acpi_init(union acpi_subtable_headers *header, · line 187
Quoted 8 lines
drivers/irqchip/irq-riscv-intc.c +3
-1@@ -179,8 +181,10 @@ static int __init riscv_intc_acpi_init(union acpi_subtable_headers *header,* for each INTC. We only do INTC initialization * for the INTC belonging to the boot CPU (or boot HART). */ -if (riscv_hartid_to_cpuid (rintc->hart_id) !=smp_processor_id ()) +if (riscv_hartid_to_cpuid (rintc->hart_id) !=smp_processor_id ()) { +fwnode_dev_initialized (of_node_to_fwnode (node), true);return 0 ; + }
We don't need this change for ACPI because we don't have devlink dependency between devices and INTC.
Regards,
Anup
Quoted 11 lines
drivers/irqchip/irq-riscv-intc.c +0
-1fn =irq_domain_alloc_named_fwnode ("RISCV-INTC" );if (!fn) { --2.34.1
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 42 lines · Anup Patel
Quoted 40 lines · Dmitry Dunaev
The current Linux driver irq-riscv-intc initialize IRQ domain only once, when init function called on primary hart. In other cases no IRQ domain is created and no operation on interrupt-controller node is performed. This is cause of that no common Linux driver can use per-cpu interrupts mapped to several CPUs because fwnode of secondary cores INTC is not marked as initialized. This device is always will be marked as deferred. For example the system with devicetree
cpu0: cpu@0 { cpu0_intc: interrupt-controller { interrupt-controller; compatible = riscv,cpu-intc; }; }; cpu1: cpu@1 { cpu1_intc: interrupt-controller { interrupt-controller; compatible = riscv,cpu-intc; }; }; buserr { compatible = riscv,buserr; interrupts-extended = <&cpu0_intc 16 &cpu1_intc 16>; };will always report 'buserr' node as deferred without calling any bus probe function.
This patch will mark all secondary nodes passed to irq-riscv-intc driver init function as initialized to be able to act as correct IRQ phandle node.
Signed-off-by Dmitry Dunaev <[email protected]>--- drivers/irqchip/irq-riscv-intc.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
Quoted 14 lines
drivers/irqchip/irq-riscv-intc.c static int __init riscv_intc_init(struct device_node *node, · line 159
Quoted 11 lines
drivers/irqchip/irq-riscv-intc.c +2
-1@@ -155,8 +155,10 @@ static int __init riscv_intc_init(struct device_node *node,* for each INTC DT node. We only need to do INTC initialization * for the INTC DT node belonging to boot CPU (or boot HART). */ -if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) +if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) { +fwnode_dev_initialized (of_node_to_fwnode (node), true);There is already a patch on LKML to address this.
https://www.spinics.net/lists/kernel/msg4929886.html
If this is a fix, why is it buried in a huge series and not brought forward as an independent fix that needs to be picked early?
M.
Signature · 6 lines
Without deviation from the norm, progress is not possible. _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 60 lines · Marc Zyngier
Quoted 42 lines · Anup Patel
Quoted 40 lines · Dmitry Dunaev
The current Linux driver irq-riscv-intc initialize IRQ domain only once, when init function called on primary hart. In other cases no IRQ domain is created and no operation on interrupt-controller node is performed. This is cause of that no common Linux driver can use per-cpu interrupts mapped to several CPUs because fwnode of secondary cores INTC is not marked as initialized. This device is always will be marked as deferred. For example the system with devicetree
cpu0: cpu@0 { cpu0_intc: interrupt-controller { interrupt-controller; compatible = riscv,cpu-intc; }; }; cpu1: cpu@1 { cpu1_intc: interrupt-controller { interrupt-controller; compatible = riscv,cpu-intc; }; }; buserr { compatible = riscv,buserr; interrupts-extended = <&cpu0_intc 16 &cpu1_intc 16>; };will always report 'buserr' node as deferred without calling any bus probe function.
This patch will mark all secondary nodes passed to irq-riscv-intc driver init function as initialized to be able to act as correct IRQ phandle node.
Signed-off-by Dmitry Dunaev <[email protected]>--- drivers/irqchip/irq-riscv-intc.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)Quoted 14 lines
drivers/irqchip/irq-riscv-intc.c static int __init riscv_intc_init(struct device_node *node, · line 159
Quoted 11 lines
drivers/irqchip/irq-riscv-intc.c +2
-1@@ -155,8 +155,10 @@ static int __init riscv_intc_init(struct device_node *node,* for each INTC DT node. We only need to do INTC initialization * for the INTC DT node belonging to boot CPU (or boot HART). */ -if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) +if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) { +fwnode_dev_initialized (of_node_to_fwnode (node), true);There is already a patch on LKML to address this.
https://www.spinics.net/lists/kernel/msg4929886.html
Quoted 2 lines
If this is a fix, why is it buried in a huge series and not brought forward as an independent fix that needs to be picked early?
Dmitry saw this issue in a totally different context which is not reproducible with existing DTS files in kernel sources.
This issue only manifests when some platform driver DT node points to the per-HART INTC nodes. For example, RISC-V irqchip device DT nodes point to per-HART INTC nodes.
Currently, all RISC-V irqchip drivers (INTC and PLIC) are probed early (not as platform drivers) so we don't see this issue with existing irqchip drivers.
The Linux AIA series implements RISC-V irqchip drivers (PLIC, APLIC, and IMSIC) as regular platform drivers (like you suggested) so this issue is easily seen with Linux AIA series hence the corresponding fix is part of the Linux AIA series.
(https://www.spinics.net/lists/devicetree/msg638068.html)
Regards,
Anup
Quoted 5 lines
M.--
Without deviation from the norm, progress is not possible.
Quoted 62 lines · Anup Patel
Quoted 60 lines · Marc Zyngier
Quoted 56 lines · Anup Patel
Quoted 45 lines · Dmitry Dunaev
The current Linux driver irq-riscv-intc initialize IRQ domain only once, when init function called on primary hart. In other cases no IRQ domain is created and no operation on interrupt-controller node is performed. This is cause of that no common Linux driver can use per-cpu interrupts mapped to several CPUs because fwnode of secondary cores INTC is not marked as initialized. This device is always will be marked as deferred. For example the system with devicetree
cpu0: cpu@0 { cpu0_intc: interrupt-controller { interrupt-controller; compatible = riscv,cpu-intc; }; }; cpu1: cpu@1 { cpu1_intc: interrupt-controller { interrupt-controller; compatible = riscv,cpu-intc; }; }; buserr { compatible = riscv,buserr; interrupts-extended = <&cpu0_intc 16 &cpu1_intc 16>; };will always report 'buserr' node as deferred without calling any bus probe function.
This patch will mark all secondary nodes passed to irq-riscv-intc driver init function as initialized to be able to act as correct IRQ phandle node.
Signed-off-by Dmitry Dunaev <[email protected]>--- drivers/irqchip/irq-riscv-intc.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)drivers/irqchip/irq-riscv-intc.c +0
-0drivers/irqchip/irq-riscv-intc.c static int __init riscv_intc_init(struct device_node *node, · line 159
Quoted 6 lines
drivers/irqchip/irq-riscv-intc.c +2
-1@@ -155,8 +155,10 @@ static int __init riscv_intc_init(struct device_node *node,* for each INTC DT node. We only need to do INTC initialization * for the INTC DT node belonging to boot CPU (or boot HART). */ -if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) +if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) { +fwnode_dev_initialized (of_node_to_fwnode (node), true);There is already a patch on LKML to address this.
https://www.spinics.net/lists/kernel/msg4929886.html
Quoted 5 lines
Quoted 2 lines
If this is a fix, why is it buried in a huge series and not brought forward as an independent fix that needs to be picked early?
Dmitry saw this issue in a totally different context which is not reproducible with existing DTS files in kernel sources.
I hope you're not suggesting that only the DTs that are present in the kernel tree are valid. Because as far as I'm concern, the DTs in the kernel tree are only some *examples*, and not a reference.
I fully expect the vast majority of DTs to live *outside* of the kernel tree, provided by the firmware, and never upstreamed. Would you expect every PC vendor to upstream their ACPI tables?
Quoted 3 lines
This issue only manifests when some platform driver DT node points to the per-HART INTC nodes. For example, RISC-V irqchip device DT nodes point to per-HART INTC nodes.
Is this configuration legal or not as per the DT binding? I don't see anything that suggests it isn't legal, and having per-CPU interrupts isn't exactly a new thing.
Quoted 3 lines
Currently, all RISC-V irqchip drivers (INTC and PLIC) are probed early (not as platform drivers) so we don't see this issue with existing irqchip drivers.
You don't, but Dimitry does. Who wins?
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 68 lines · Marc Zyngier
Quoted 65 lines · Anup Patel
Quoted 62 lines · Marc Zyngier
Quoted 42 lines · Anup Patel
Quoted 40 lines · Dmitry Dunaev
The current Linux driver irq-riscv-intc initialize IRQ domain only once, when init function called on primary hart. In other cases no IRQ domain is created and no operation on interrupt-controller node is performed. This is cause of that no common Linux driver can use per-cpu interrupts mapped to several CPUs because fwnode of secondary cores INTC is not marked as initialized. This device is always will be marked as deferred. For example the system with devicetree
cpu0: cpu@0 { cpu0_intc: interrupt-controller { interrupt-controller; compatible = riscv,cpu-intc; }; }; cpu1: cpu@1 { cpu1_intc: interrupt-controller { interrupt-controller; compatible = riscv,cpu-intc; }; }; buserr { compatible = riscv,buserr; interrupts-extended = <&cpu0_intc 16 &cpu1_intc 16>; };will always report 'buserr' node as deferred without calling any bus probe function.
This patch will mark all secondary nodes passed to irq-riscv-intc driver init function as initialized to be able to act as correct IRQ phandle node.
Signed-off-by Dmitry Dunaev <[email protected]>--- drivers/irqchip/irq-riscv-intc.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)Quoted 14 lines
drivers/irqchip/irq-riscv-intc.c static int __init riscv_intc_init(struct device_node *node, · line 159
Quoted 11 lines
drivers/irqchip/irq-riscv-intc.c +2
-1@@ -155,8 +155,10 @@ static int __init riscv_intc_init(struct device_node *node,* for each INTC DT node. We only need to do INTC initialization * for the INTC DT node belonging to boot CPU (or boot HART). */ -if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) +if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) { +fwnode_dev_initialized (of_node_to_fwnode (node), true);There is already a patch on LKML to address this.
https://www.spinics.net/lists/kernel/msg4929886.htmlIf this is a fix, why is it buried in a huge series and not brought forward as an independent fix that needs to be picked early?
Quoted 6 lines
Quoted 2 lines
Dmitry saw this issue in a totally different context which is not reproducible with existing DTS files in kernel sources.
I hope you're not suggesting that only the DTs that are present in the kernel tree are valid. Because as far as I'm concern, the DTs in the kernel tree are only some *examples*, and not a reference.
I am only saying why this issue was not observed before.
Quoted 4 lines
I fully expect the vast majority of DTs to live *outside* of the kernel tree, provided by the firmware, and never upstreamed. Would you expect every PC vendor to upstream their ACPI tables?
I agree. We can't expect all vendors to submit DT to kernel sources.
Quoted 8 lines
Quoted 4 lines
This issue only manifests when some platform driver DT node points to the per-HART INTC nodes. For example, RISC-V irqchip device DT nodes point to per-HART INTC nodes.
Is this configuration legal or not as per the DT binding? I don't see anything that suggests it isn't legal, and having per-CPU interrupts isn't exactly a new thing.
This is a perfect legal configuration in the RISC-V world. We have similar DT binding for AIA drivers as well.
Quoted 6 lines
Quoted 4 lines
Currently, all RISC-V irqchip drivers (INTC and PLIC) are probed early (not as platform drivers) so we don't see this issue with existing irqchip drivers.
You don't, but Dimitry does. Who wins?
I am totally fine taking PATCH3 of the Linux AIA v10 series as a fix PATCH for 6.6-rcX. The PATCH3 is pretty self contained and does not depend on any other PATCH of Linux AIA v10 series.
Do you want me to re-send it as an individual PATCH ?
Regards,
Anup
Quoted 5 lines
M.--
Without deviation from the norm, progress is not possible.
kernel test robot
build botkernel test robot noticed the following build errors:
Hi Dmitry,
kernel test robot noticed the following build errors:
[auto build test ERROR on tip/irq/core]
[also build test ERROR on linus/master v6.6-rc3 next-20230926] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Dmitry-Dunaev/irqchip-riscv-intc-Mark-INTC-nodes-for-secondary-CPUs-as-initialized/20230926-183500 base: tip/irq/core patch link: https://lore.kernel.org/r/20230926102801.1591126-1-dunaev%40tecon.ru patch subject: [PATCH] irqchip/riscv-intc: Mark INTC nodes for secondary CPUs as initialized. config: riscv-randconfig-001-20230926 (https://download.01.org/0day-ci/archive/20230927/[email protected]/config) compiler: riscv64-linux-gcc (GCC) 13.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20230927/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
drivers/irqchip/irq-riscv-intc.c: In function 'riscv_intc_acpi_init':
>> drivers/irqchip/irq-riscv-intc.c:185:58: error: 'node' undeclared (first use in this function)
185 | fwnode_dev_initialized(of_node_to_fwnode(node), true);
| ^~~~ drivers/irqchip/irq-riscv-intc.c:185:58: note: each undeclared identifier is reported only once for each function it appears in
vim +/node +185 drivers/irqchip/irq-riscv-intc.c
169
170 static int __init riscv_intc_acpi_init(union acpi_subtable_headers *header,
171 const unsigned long end)
172 {
173 struct fwnode_handle *fn;
174 struct acpi_madt_rintc *rintc;
175
176 rintc = (struct acpi_madt_rintc *)header;
177
178 /*
179 * The ACPI MADT will have one INTC for each CPU (or HART)
180 * so riscv_intc_acpi_init() function will be called once
181 * for each INTC. We only do INTC initialization
182 * for the INTC belonging to the boot CPU (or boot HART).
183 */
184 if (riscv_hartid_to_cpuid(rintc->hart_id) != smp_processor_id()) {
> 185 fwnode_dev_initialized(of_node_to_fwnode(node), true);
186 return 0;
187 }
188
189 fn = irq_domain_alloc_named_fwnode("RISCV-INTC");
190 if (!fn) {
191 pr_err("unable to allocate INTC FW node\n");
192 return -ENOMEM;
193 }
194
195 return riscv_intc_init_common(fn);
196 }
197
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
[irqchip: irq/irqchip-fixes] irqchip/riscv-intc: Mark all INTC nodes as initialized
In reply to Dmitry DunaevThe following commit has been merged into the irq/irqchip-fixes branch of irqchip:
The following commit has been merged into the irq/irqchip-fixes branch of irqchip:
Commit-ID: e13cd66bd821be417c498a34928652db4ac6b436 Gitweb: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms/e13cd66bd821be417c498a34928652db4ac6b436 Author: Anup Patel <[email protected]> AuthorDate: Tue, 03 Oct 2023 10:13:51 +05:30 Committer: Marc Zyngier <[email protected]> CommitterDate: Sat, 07 Oct 2023 12:47:12 +01:00
irqchip/riscv-intc: Mark all INTC nodes as initialized
The RISC-V INTC local interrupts are per-HART (or per-CPU) so we create INTC IRQ domain only for the INTC node belonging to the boot HART. This means only the boot HART INTC node will be marked as initialized and other INTC nodes won't be marked which results downstream interrupt controllers (such as PLIC, IMSIC and APLIC direct-mode) not being probed due to missing device suppliers.
To address this issue, we mark all INTC node for which we don't create IRQ domain as initialized.
--- drivers/irqchip/irq-riscv-intc.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
drivers/irqchip/irq-riscv-intc.c +9 -1
@@ -155,8 +155,16 @@ static int __init riscv_intc_init(struct device_node *node,* for each INTC DT node. We only need to do INTC initialization * for the INTC DT node belonging to boot CPU (or boot HART). */ -if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) +if (riscv_hartid_to_cpuid (hartid) !=smp_processor_id ()) { +/* +* The INTC nodes of each CPU are suppliers for downstream +* interrupt controllers (such as PLIC, IMSIC and APLIC +* direct-mode) so we should mark an INTC node as initialized +* if we are not creating IRQ domain for it. +*/ +fwnode_dev_initialized (of_fwnode_handle (node), true);return 0 ; + }return riscv_intc_init_common (of_node_to_fwnode (node)); }