x86/ioapic: Don't return 0 from arch_dynirq_lower_bound()
PATCH v2 1/1 x86/ioapic: Don't return 0 from arch_dynirq_lower_bound()
arch_dynirq_lower_bound() is invoked by the core interrupt code to retrieve the lowest possible Linux interrupt number for dynamically allocated interrupts like MSI.
The x86 implementation uses this to exclude the IO/APIC GSI space. This works correctly as long as there is an IO/APIC registered, but returns 0 if not. This has been observed in VMs where the BIOS does not advertise an IO/APIC.
0 is an invalid interrupt number except for the legacy timer interrupt on x86. The return value is unchecked in the core code, so it ends up to allocate interrupt number 0 which is subsequently considered to be invalid by the caller, e.g. the MSI allocation code.
The function has already a check for 0 in the case that an IO/APIC is registered, but ioapic_dynirq_base is 0 in case of device tree setups.
Consolidate this and zero check for both ioapic_dynirq_base and gsi_top, which is used in the case that no IO/APIC is registered.
---
[V2]
- Edit commit message
- Consolidated the 0 check for ioapic_dynirq_base as well
arch/x86/kernel/apic/io_apic.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
arch/x86/kernel/apic/io_apic.c +9 -5
@@ -2477,17 +2477,21 @@ static int io_apic_get_redir_entries(int ioapic)unsigned int arch_dynirq_lower_bound (unsigned int from) { +unsigned int ret; +/* * dmar_alloc_hwirq() may be called before setup_IO_APIC(), so use * gsi_top if ioapic_dynirq_base hasn't been initialized yet. */ -if (!ioapic_initialized) -return gsi_top; + ret = ioapic_dynirq_base ? : gsi_top; +/* -* For DT enabled machines ioapic_dynirq_base is irrelevant and not -* updated. So simply return @from if ioapic_dynirq_base == 0. +* For DT enabled machines ioapic_dynirq_base is irrelevant and +* always 0. gsi_top can be 0 if there is no IO/APIC registered. +* 0 is an invalid interrupt number for dynamic allocations. Return +* @from instead. */ -return ioapic_dynirq_base ? : from; +return ret ? : from; }#ifdef CONFIG_X86_32
Signature · 1 line
2.34.1
H. Peter Anvin
reviewerQuoted 55 lines · Saurabh Sengar
arch_dynirq_lower_bound()is invoked by the core interrupt code to retrieve the lowest possible Linux interrupt number for dynamically allocated interrupts like MSI.The x86 implementation uses this to exclude the IO/APIC GSI space. This works correctly as long as there is an IO/APIC registered, but returns 0 if not. This has been observed in VMs where the BIOS does not advertise an IO/APIC.
0 is an invalid interrupt number except for the legacy timer interrupt on x86. The return value is unchecked in the core code, so it ends up to allocate interrupt number 0 which is subsequently considered to be invalid by the caller, e.g. the MSI allocation code.
The function has already a check for 0 in the case that an IO/APIC is registered, but
ioapic_dynirq_baseis 0 in case of device tree setups.Consolidate this and zero check for both
ioapic_dynirq_baseandgsi_top, which is used in the case that no IO/APIC is registered.Fixes 3e5bedc2c258 ("x86/apic: Fix arch_dynirq_lower_bound() bug for DT enabled machines")Co-developed-by Thomas Gleixner <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Signed-off-by Saurabh Sengar <[email protected]>Cc Andy Shevchenko <[email protected]>Cc Thomas Gleixner <[email protected]>---
[V2]
- Edit commit message
- Consolidated the 0 check forioapic_dynirq_baseas well
arch/x86/kernel/apic/io_apic.c| 14 +++++++++-----1 file changed, 9 insertions(+), 5 deletions(-)arch/x86/kernel/apic/io_apic.c +4
-4@@ -2477,17 +2477,21 @@ static int io_apic_get_redir_entries(int ioapic)unsigned int arch_dynirq_lower_bound (unsigned int from) { +unsigned int ret; +/* * dmar_alloc_hwirq() may be called before setup_IO_APIC(), so use * gsi_top if ioapic_dynirq_base hasn't been initialized yet. */ -if (!ioapic_initialized) -return gsi_top; + ret = ioapic_dynirq_base ? : gsi_top; +/* -* For DT enabled machines ioapic_dynirq_base is irrelevant and not -* updated. So simply return @from if ioapic_dynirq_base == 0.
arch/x86/kernel/apic/io_apic.c static int io_apic_get_redir_entries(int ioapic) · line 2497
Quoted 10 lines
arch/x86/kernel/apic/io_apic.c +5
-1+* For DT enabled machines ioapic_dynirq_base is irrelevant and +* always 0. gsi_top can be 0 if there is no IO/APIC registered. +* 0 is an invalid interrupt number for dynamic allocations. Return +* @from instead. */ -return ioapic_dynirq_base ? : from; +return ret ? : from; }#ifdef CONFIG_X86_32
Is there any reason why this variable can't be initialized to a fixed nonzero number, like 16?
Quoted 65 lines · H. Peter Anvin
Quoted 64 lines · Saurabh Sengar
arch_dynirq_lower_bound()is invoked by the core interrupt code to retrieve the lowest possible Linux interrupt number for dynamically allocated interrupts like MSI.The x86 implementation uses this to exclude the IO/APIC GSI space. This works correctly as long as there is an IO/APIC registered, but returns 0 if not. This has been observed in VMs where the BIOS does not advertise an IO/APIC.
0 is an invalid interrupt number except for the legacy timer interrupt on x86. The return value is unchecked in the core code, so it ends up to allocate interrupt number 0 which is subsequently considered to be invalid by the caller, e.g. the MSI allocation code.
The function has already a check for 0 in the case that an IO/APIC is registered, but
ioapic_dynirq_baseis 0 in case of device tree setups.Consolidate this and zero check for both
ioapic_dynirq_baseandgsi_top, which is used in the case that no IO/APIC is registered.Fixes 3e5bedc2c258 ("x86/apic: Fix arch_dynirq_lower_bound() bug for DT enabled machines")Co-developed-by Thomas Gleixner <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Signed-off-by Saurabh Sengar <[email protected]>Cc Andy Shevchenko <[email protected]>Cc Thomas Gleixner <[email protected]>---
[V2]
- Edit commit message
- Consolidated the 0 check forioapic_dynirq_baseas well
arch/x86/kernel/apic/io_apic.c| 14 +++++++++-----1 file changed, 9 insertions(+), 5 deletions(-)arch/x86/kernel/apic/io_apic.c +9
-5@@ -2477,17 +2477,21 @@ static int io_apic_get_redir_entries(int ioapic)unsigned int arch_dynirq_lower_bound (unsigned int from) { +unsigned int ret; +/* * dmar_alloc_hwirq() may be called before setup_IO_APIC(), so use * gsi_top if ioapic_dynirq_base hasn't been initialized yet. */ -if (!ioapic_initialized) -return gsi_top; + ret = ioapic_dynirq_base ? : gsi_top; +/* -* For DT enabled machines ioapic_dynirq_base is irrelevant and not -* updated. So simply return @from if ioapic_dynirq_base == 0. +* For DT enabled machines ioapic_dynirq_base is irrelevant and +* always 0. gsi_top can be 0 if there is no IO/APIC registered. +* 0 is an invalid interrupt number for dynamic allocations. Return +* @from instead. */ -return ioapic_dynirq_base ? : from; +return ret ? : from; }
Quoted 3 lines
arch/x86/kernel/apic/io_apic.c static int io_apic_get_redir_entries(int ioapic) · line 2497
Quoted 1 line
arch/x86/kernel/apic/io_apic.c +0
-0#ifdef CONFIG_X86_32 Is there any reason why this variable can't be initialized to a fixed nonzero number, like 16?
Yes, initializing gst_top to any non-zero value should fix this issue. At first I thought to intialize gst_top to 1. But then I looked at how the ioapic_dynirq_base case is handled and followed a similar approach.
Regards,
Saurabh
Quoted 6 lines · Saurabh Singh Sengar
Quoted 2 lines · H. Peter Anvin
Is there any reason why this variable can't be initialized to a fixed nonzero number, like 16?
Yes, initializing
gst_topto any non-zero value should fix this issue. At first I thought to intializegst_topto 1.
That works only in your case. Some boot time registrations of IO_APICs use gsi_top as the base. So initializing gsi_top to N would move IOAPIC[0] interrupts out to irq N... and make the legacy interrupts fail.
That whole IOAPIC registration could do with some major cleanup, but that's a different story.
Thanks,
tglx
Quoted 55 lines · Saurabh Sengar
arch_dynirq_lower_bound()is invoked by the core interrupt code to retrieve the lowest possible Linux interrupt number for dynamically allocated interrupts like MSI.The x86 implementation uses this to exclude the IO/APIC GSI space. This works correctly as long as there is an IO/APIC registered, but returns 0 if not. This has been observed in VMs where the BIOS does not advertise an IO/APIC.
0 is an invalid interrupt number except for the legacy timer interrupt on x86. The return value is unchecked in the core code, so it ends up to allocate interrupt number 0 which is subsequently considered to be invalid by the caller, e.g. the MSI allocation code.
The function has already a check for 0 in the case that an IO/APIC is registered, but
ioapic_dynirq_baseis 0 in case of device tree setups.Consolidate this and zero check for both
ioapic_dynirq_baseandgsi_top, which is used in the case that no IO/APIC is registered.Fixes 3e5bedc2c258 ("x86/apic: Fix arch_dynirq_lower_bound() bug for DT enabled machines")Co-developed-by Thomas Gleixner <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Signed-off-by Saurabh Sengar <[email protected]>Cc Andy Shevchenko <[email protected]>Cc Thomas Gleixner <[email protected]>---
[V2]
- Edit commit message
- Consolidated the 0 check forioapic_dynirq_baseas wellarch/x86/kernel/apic/io_apic.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)arch/x86/kernel/apic/io_apic.c +4
-4@@ -2477,17 +2477,21 @@ static int io_apic_get_redir_entries(int ioapic)unsigned int arch_dynirq_lower_bound (unsigned int from) { +unsigned int ret; +/* * dmar_alloc_hwirq() may be called before setup_IO_APIC(), so use * gsi_top if ioapic_dynirq_base hasn't been initialized yet. */ -if (!ioapic_initialized) -return gsi_top; + ret = ioapic_dynirq_base ? : gsi_top; +/* -* For DT enabled machines ioapic_dynirq_base is irrelevant and not -* updated. So simply return @from if ioapic_dynirq_base == 0.
arch/x86/kernel/apic/io_apic.c static int io_apic_get_redir_entries(int ioapic) · line 2497
Quoted 10 lines
arch/x86/kernel/apic/io_apic.c +5
-1+* For DT enabled machines ioapic_dynirq_base is irrelevant and +* always 0. gsi_top can be 0 if there is no IO/APIC registered. +* 0 is an invalid interrupt number for dynamic allocations. Return +* @from instead. */ -return ioapic_dynirq_base ? : from; +return ret ? : from; }#ifdef CONFIG_X86_32
Is this good to get accepted ? Please let me know if anything pending from my end on this.
- Saurabh
Quoted 2 lines
Signature · 1 line
2.34.1
The following commit has been merged into the x86/apic branch of tip:
The following commit has been merged into the x86/apic branch of tip:
Commit-ID: 5af507bef93c09a94fb8f058213b489178f4cbe5 Gitweb: https://git.kernel.org/tip/5af507bef93c09a94fb8f058213b489178f4cbe5 Author: Saurabh Sengar <[email protected]> AuthorDate: Tue, 28 Mar 2023 00:30:04 -07:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Wed, 12 Apr 2023 17:45:50 +02:00
x86/ioapic: Don't return 0 from arch_dynirq_lower_bound()
arch_dynirq_lower_bound() is invoked by the core interrupt code to retrieve the lowest possible Linux interrupt number for dynamically allocated interrupts like MSI.
The x86 implementation uses this to exclude the IO/APIC GSI space. This works correctly as long as there is an IO/APIC registered, but returns 0 if not. This has been observed in VMs where the BIOS does not advertise an IO/APIC.
0 is an invalid interrupt number except for the legacy timer interrupt on x86. The return value is unchecked in the core code, so it ends up to allocate interrupt number 0 which is subsequently considered to be invalid by the caller, e.g. the MSI allocation code.
The function has already a check for 0 in the case that an IO/APIC is registered, as ioapic_dynirq_base is 0 in case of device tree setups.
Consolidate this and zero check for both ioapic_dynirq_base and gsi_top, which is used in the case that no IO/APIC is registered.
--- arch/x86/kernel/apic/io_apic.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
arch/x86/kernel/apic/io_apic.c +9 -5
@@ -2477,17 +2477,21 @@ static int io_apic_get_redir_entries(int ioapic)unsigned int arch_dynirq_lower_bound (unsigned int from) { +unsigned int ret; +/* * dmar_alloc_hwirq() may be called before setup_IO_APIC(), so use * gsi_top if ioapic_dynirq_base hasn't been initialized yet. */ -if (!ioapic_initialized) -return gsi_top; + ret = ioapic_dynirq_base ? : gsi_top; +/* -* For DT enabled machines ioapic_dynirq_base is irrelevant and not -* updated. So simply return @from if ioapic_dynirq_base == 0. +* For DT enabled machines ioapic_dynirq_base is irrelevant and +* always 0. gsi_top can be 0 if there is no IO/APIC registered. +* 0 is an invalid interrupt number for dynamic allocations. Return +* @from instead. */ -return ioapic_dynirq_base ? : from; +return ret ? : from; }#ifdef CONFIG_X86_32