arm64: ACPI GTDT watchdog fixes
Subsystems (3)
cover arm64: ACPI GTDT watchdog fixes
Dann recently reported that his ThunderX machine failed to boot since 64b499d8df40 ("irqchip/gic-v3: Configure SGIs as standard interrupts"), with a not so pretty crash while trying to send an IPI.
It turned out to be caused by a mix of broken firmware and a buggy GTDT watchdog driver. Both have forever been buggy, but the above commit revealed that the error handling path of the driver was probably the worse part of it all.
Anyway, this short series has two goals:
- handle broken firmware in a less broken way
- make sure that the route cause of the problem can be identified quickly
Thanks,
M.
Marc Zyngier (2):
ACPI: GTDT: Don't corrupt interrupt mappings on watchdow probe failure
ACPI: irq: Prevent unregistering of GIC SGIs
drivers/acpi/arm64/gtdt.c | 10 ++++++---- drivers/acpi/irq.c | 6 +++++- 2 files changed, 11 insertions(+), 5 deletions(-)
Signature · 1 line
2.29.2
PATCH 1/2 ACPI: GTDT: Don't corrupt interrupt mappings on watchdow probe failure
When failing the driver probe because of invalid firmware properties, the GTDT driver unmaps the interrupt that it mapped earlier.
However, it never checks whether the mapping of the interrupt actially succeeded. Even more, should the firmware report an illegal interrupt number that overlaps with the GIC SGI range, this can result in an IPI being unmapped, and subsequent fireworks (as reported by Dann Frazier).
Rework the driver to have a slightly saner behaviour and actually check whether the interrupt has been mapped before unmapping things.
--- drivers/acpi/arm64/gtdt.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/acpi/arm64/gtdt.c +6 -4
@@ -329,7 +329,7 @@ static int __init gtdt_import_sbsa_gwdt(struct acpi_gtdt_watchdog *wd, int index) {struct platform_device *pdev; -int irq =map_gt_gsi (wd->timer_interrupt, wd->timer_flags); +int irq;/* * According to SBSA specification the size of refresh and control
@@ -338,7 +338,7 @@ static int __init gtdt_import_sbsa_gwdt(struct acpi_gtdt_watchdog *wd,struct resource res[] = {DEFINE_RES_MEM (wd->control_frame_address,SZ_4K ),DEFINE_RES_MEM (wd->refresh_frame_address,SZ_4K ), -DEFINE_RES_IRQ (irq), + {}, };int nr_res =ARRAY_SIZE (res);
@@ -348,10 +348,11 @@ static int __init gtdt_import_sbsa_gwdt(struct acpi_gtdt_watchdog *wd,if (!(wd->refresh_frame_address && wd->control_frame_address)) {pr_err (FW_BUG "failed to get the Watchdog base address.\n" ); -acpi_unregister_gsi (wd->timer_interrupt);return -EINVAL ; } + irq =map_gt_gsi (wd->timer_interrupt, wd->timer_flags); + res[2 ] = (struct resource )DEFINE_RES_IRQ (irq);if (irq <=0 ) {pr_warn ("failed to map the Watchdog interrupt.\n" ); nr_res--;
@@ -364,7 +365,8 @@ static int __init gtdt_import_sbsa_gwdt(struct acpi_gtdt_watchdog *wd,*/ pdev =platform_device_register_simple ("sbsa-gwdt" , index, res, nr_res);if (IS_ERR (pdev)) { -acpi_unregister_gsi (wd->timer_interrupt); +if (irq >0 ) +acpi_unregister_gsi (wd->timer_interrupt);return PTR_ERR (pdev); }
Signature · 1 line
2.29.2
Re: [PATCH 1/2] ACPI: GTDT: Don't corrupt interrupt mappings on watchdow probe failure
Quoted 12 lines · Marc Zyngier
When failing the driver probe because of invalid firmware properties, the GTDT driver unmaps the interrupt that it mapped earlier.
However, it never checks whether the mapping of the interrupt actially succeeded. Even more, should the firmware report an illegal interrupt number that overlaps with the GIC SGI range, this can result in an IPI being unmapped, and subsequent fireworks (as reported by Dann Frazier).
Rework the driver to have a slightly saner behaviour and actually check whether the interrupt has been mapped before unmapping things.
Signature · 2 lines
Regards, Sudeep
Re: [PATCH 1/2] ACPI: GTDT: Don't corrupt interrupt mappings on watchdow probe failure
In reply to Marc Zyngier(with stable email fixed, but may need separate posting anyways)
Quoted 9 lines · Marc Zyngier
When failing the driver probe because of invalid firmware properties, the GTDT driver unmaps the interrupt that it mapped earlier.
However, it never checks whether the mapping of the interrupt actially succeeded. Even more, should the firmware report an illegal interrupt number that overlaps with the GIC SGI range, this can result in an IPI being unmapped, and subsequent fireworks (as reported by Dann Frazier).
Quoted 8 lines
Rework the driver to have a slightly saner behaviour and actually check whether the interrupt has been mapped before unmapping things.
Reported-by dann frazier <[email protected]>Fixes ca9ae5ec4ef0 ("acpi/arm64: Add SBSA Generic Watchdog support in GTDT driver")Signed-off-by Marc Zyngier <[email protected]>
^^^
I thought I messed up while replying with my fat fingers, but here is the culprit. I got mail delivery failure to my reply, thought I will let you know as it may be difficult to notice this when using git send-email.
Signature · 2 lines
Regards, Sudeep
Re: [PATCH 1/2] ACPI: GTDT: Don't corrupt interrupt mappings on watchdow probe failure
Quoted 21 lines · Sudeep Holla
(with stable email fixed, but may need separate posting anyways)
Quoted 9 lines · Marc Zyngier
When failing the driver probe because of invalid firmware properties, the GTDT driver unmaps the interrupt that it mapped earlier.
However, it never checks whether the mapping of the interrupt actially succeeded. Even more, should the firmware report an illegal interrupt number that overlaps with the GIC SGI range, this can result in an IPI being unmapped, and subsequent fireworks (as reported by Dann Frazier).
Quoted 8 lines
Rework the driver to have a slightly saner behaviour and actually check whether the interrupt has been mapped before unmapping things.
Reported-by dann frazier <[email protected]>Fixes ca9ae5ec4ef0 ("acpi/arm64: Add SBSA Generic Watchdog support in GTDT driver")Signed-off-by Marc Zyngier <[email protected]>
Quoted 5 lines
^^^I thought I messed up while replying with my fat fingers, but here is the culprit. I got mail delivery failure to my reply, thought I will let you know as it may be difficult to notice this when using git send-email.
I'm definitely the one with very fat fingers. I'll fix the patch in my tree.
Thanks for the heads up,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
PATCH 2/2 ACPI: irq: Prevent unregistering of GIC SGIs
In reply to Marc ZyngierWhen using ACPI on arm64, which implies the GIC IRQ model, no table should ever provide a GSI number in the range [0:15], as these are reserved for IPIs.
However, drivers tend to call acpi_unregister_gsi() with any random GSI number provided by half baked tables, which results in an exploding kernel when its IPIs have been unconfigured.
In order to catch this, check for the silly case early, warn that something is going wrong and avoid the above disaster.
--- drivers/acpi/irq.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/acpi/irq.c +5 -1
@@ -75,8 +75,12 @@ void acpi_unregister_gsi(u32 gsi) {struct irq_domain *d =irq_find_matching_fwnode (acpi_gsi_domain_id,DOMAIN_BUS_ANY ); -int irq =irq_find_mapping (d, gsi); +int irq; +if (WARN_ON (acpi_irq_model ==ACPI_IRQ_MODEL_GIC && gsi <16 )) +return ; + + irq =irq_find_mapping (d, gsi);irq_dispose_mapping (irq); }EXPORT_SYMBOL_GPL (acpi_unregister_gsi);
Signature · 1 line
2.29.2
Re: [PATCH 2/2] ACPI: irq: Prevent unregistering of GIC SGIs
Quoted 11 lines · Marc Zyngier
When using ACPI on arm64, which implies the GIC IRQ model, no table should ever provide a GSI number in the range [0:15], as these are reserved for IPIs.
However, drivers tend to call
acpi_unregister_gsi()with any random GSI number provided by half baked tables, which results in an exploding kernel when its IPIs have been unconfigured.In order to catch this, check for the silly case early, warn that something is going wrong and avoid the above disaster.
Just curious if this is just precaution or do we have a platform doing something stupid like this ?
--
Regards,
Sudeep
Re: [PATCH 2/2] ACPI: irq: Prevent unregistering of GIC SGIs
Quoted 10 lines · Sudeep Holla
Quoted 8 lines · Marc Zyngier
When using ACPI on arm64, which implies the GIC IRQ model, no table should ever provide a GSI number in the range [0:15], as these are reserved for IPIs.
However, drivers tend to call
acpi_unregister_gsi()with any random GSI number provided by half baked tables, which results in an exploding kernel when its IPIs have been unconfigured.
Quoted 8 lines
Quoted 3 lines
In order to catch this, check for the silly case early, warn that something is going wrong and avoid the above disaster.
Reviewed-by Sudeep Holla <[email protected]>Just curious if this is just precaution or do we have a platform doing something stupid like this ?
Without this, it could be really hard to pinpoint which driver messes with IPIs. Having this in place would have caught the GTDT bug much earlier (several years ago actually).
The only reason I managed to track it down in a short amount of time is that the driver actually printed an error message before the kernel exploded while probing a completely unrelated driver. Without this message, I'd still be scratching my head.
The WARN_ON() would definitely point at the guilty party, and keep the kernel running.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Quoted 18 lines · Marc Zyngier
Dann recently reported that his ThunderX machine failed to boot since
64b499d8df40("irqchip/gic-v3: Configure SGIs as standard interrupts"), with a not so pretty crash while trying to send an IPI.It turned out to be caused by a mix of broken firmware and a buggy GTDT watchdog driver. Both have forever been buggy, but the above commit revealed that the error handling path of the driver was probably the worse part of it all.
Anyway, this short series has two goals:
- handle broken firmware in a less broken way
- make sure that the route cause of the problem can be identified quicklyThanks,
M.
Quoted 7 lines
Marc Zyngier (2):
ACPI: GTDT: Don't corrupt interrupt mappings on watchdow probe failure
ACPI: irq: Prevent unregistering of GIC SGIsdrivers/acpi/arm64/gtdt.c | 10 ++++++---- drivers/acpi/irq.c | 6 +++++- 2 files changed, 11 insertions(+), 5 deletions(-)
For the series:
Quoted 9 lines · Marc Zyngier
Dann recently reported that his ThunderX machine failed to boot since
64b499d8df40("irqchip/gic-v3: Configure SGIs as standard interrupts"), with a not so pretty crash while trying to send an IPI.It turned out to be caused by a mix of broken firmware and a buggy GTDT watchdog driver. Both have forever been buggy, but the above commit revealed that the error handling path of the driver was probably the worse part of it all.
Quoted 4 lines
Anyway, this short series has two goals:
- handle broken firmware in a less broken way
- make sure that the route cause of the problem can be identified quickly
Tested on Kunpeng920 ARM64 server, didn't any issue after applying this patch set,
Thanks
Hanjun
Quoted 18 lines · Marc Zyngier
Dann recently reported that his ThunderX machine failed to boot since
64b499d8df40("irqchip/gic-v3: Configure SGIs as standard interrupts"), with a not so pretty crash while trying to send an IPI.It turned out to be caused by a mix of broken firmware and a buggy GTDT watchdog driver. Both have forever been buggy, but the above commit revealed that the error handling path of the driver was probably the worse part of it all.
Anyway, this short series has two goals:
- handle broken firmware in a less broken way
- make sure that the route cause of the problem can be identified quicklyThanks,
M.
Quoted 7 lines
Marc Zyngier (2):
ACPI: GTDT: Don't corrupt interrupt mappings on watchdow probe failure
ACPI: irq: Prevent unregistering of GIC SGIsdrivers/acpi/arm64/gtdt.c | 10 ++++++---- drivers/acpi/irq.c | 6 +++++- 2 files changed, 11 insertions(+), 5 deletions(-)
Patch(2) needs an ACK from Rafael - usually these patches go via the ARM64 tree but I don't think it is compulsory for this series.
Thank you !
Catalin Marinas
maintainerQuoted 27 lines · Lorenzo Pieralisi
Quoted 18 lines · Marc Zyngier
Dann recently reported that his ThunderX machine failed to boot since
64b499d8df40("irqchip/gic-v3: Configure SGIs as standard interrupts"), with a not so pretty crash while trying to send an IPI.It turned out to be caused by a mix of broken firmware and a buggy GTDT watchdog driver. Both have forever been buggy, but the above commit revealed that the error handling path of the driver was probably the worse part of it all.
Anyway, this short series has two goals:
- handle broken firmware in a less broken way
- make sure that the route cause of the problem can be identified quicklyThanks,
M.Quoted 7 lines
Marc Zyngier (2):
ACPI: GTDT: Don't corrupt interrupt mappings on watchdow probe failure
ACPI: irq: Prevent unregistering of GIC SGIsdrivers/acpi/arm64/gtdt.c | 10 ++++++---- drivers/acpi/irq.c | 6 +++++- 2 files changed, 11 insertions(+), 5 deletions(-)
Quoted 6 lines
Patch(2) needs an ACK from Rafael - usually these patches go via the ARM64 tree but I don't think it is compulsory for this series.
Thank you !
Reviewed-by Lorenzo Pieralisi <[email protected]>
Thanks Lorenzo.
Rafael, if there are no objections, I'll take these two patches in the arm64 tree.
Signature · 1 line
Catalin
Quoted 10 lines · Marc Zyngier
Dann recently reported that his ThunderX machine failed to boot since
64b499d8df40("irqchip/gic-v3: Configure SGIs as standard interrupts"), with a not so pretty crash while trying to send an IPI.It turned out to be caused by a mix of broken firmware and a buggy GTDT watchdog driver. Both have forever been buggy, but the above commit revealed that the error handling path of the driver was probably the worse part of it all.
[...]
Applied to arm64 (for-next/core), thanks!
[1/2] ACPI: GTDT: Don't corrupt interrupt mappings on watchdow probe failure
https://git.kernel.org/arm64/c/1ecd5b129252 [2/2] ACPI: irq: Prevent unregistering of GIC SGIs https://git.kernel.org/arm64/c/2a20b08f06e7
Signature · 1 line
Catalin