Skip to contentSkip to conversation

x86/ioapic: Don't return 0 from arch_dynirq_lower_bound()

PATCH Applied Saurabh Sengar 4 participants 6 messages Mar 28, 2023 – Apr 12, 2023 x86

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 x86/ioapic: Don't return 0 from arch_dynirq_lower_bound()

arch_dynirq_lower_bound() is invoked by the core interrupt code to retrieve the lowest possible Linux interrupt number for dynamically allocated interrupts like MSI.

The x86 implementation uses this to exclude the IO/APIC GSI space. This works correctly as long as there is an IO/APIC registered, but returns 0 if not. This has been observed in VMs where the BIOS does not advertise an IO/APIC.

0 is an invalid interrupt number except for the legacy timer interrupt on x86. The return value is unchecked in the core code, so it ends up to allocate interrupt number 0 which is subsequently considered to be invalid by the caller, e.g. the MSI allocation code.

The function has already a check for 0 in the case that an IO/APIC is registered, but ioapic_dynirq_base is 0 in case of device tree setups.

Consolidate this and zero check for both ioapic_dynirq_base and gsi_top, which is used in the case that no IO/APIC is registered.

Fixes 3e5bedc2c258 ("x86/apic: Fix arch_dynirq_lower_bound() bug for DT enabled machines")
Cc Andy Shevchenko <[email protected]>
Cc Thomas Gleixner <[email protected]>

---
[V2]
- Edit commit message
- Consolidated the 0 check for ioapic_dynirq_base as well

 arch/x86/kernel/apic/io_apic.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
arch/x86/kernel/apic/io_apic.c +9 -5
index 1f83b052bb74..f980b38b0227 100644
@@ -2477,17 +2477,21 @@ static int io_apic_get_redir_entries(int ioapic)
 
 unsigned int arch_dynirq_lower_bound(unsigned int from)
 {
+	unsigned int ret;
+
 	/*
 	 * dmar_alloc_hwirq() may be called before setup_IO_APIC(), so use
 	 * gsi_top if ioapic_dynirq_base hasn't been initialized yet.
 	 */
-	if (!ioapic_initialized)
-		return gsi_top;
+	ret = ioapic_dynirq_base ? : gsi_top;
+
 	/*
-	 * For DT enabled machines ioapic_dynirq_base is irrelevant and not
-	 * updated. So simply return @from if ioapic_dynirq_base == 0.
+	 * For DT enabled machines ioapic_dynirq_base is irrelevant and
+	 * always 0. gsi_top can be 0 if there is no IO/APIC registered.
+	 * 0 is an invalid interrupt number for dynamic allocations. Return
+	 * @from instead.
 	 */
-	return ioapic_dynirq_base ? : from;
+	return ret ? : from;
 }
 
 #ifdef CONFIG_X86_32
Signature · 1 line
2.34.1
#1 lore raw
Quoted 55 lines · Saurabh Sengar

arch_dynirq_lower_bound() is invoked by the core interrupt code to retrieve the lowest possible Linux interrupt number for dynamically allocated interrupts like MSI.

The x86 implementation uses this to exclude the IO/APIC GSI space. This works correctly as long as there is an IO/APIC registered, but returns 0 if not. This has been observed in VMs where the BIOS does not advertise an IO/APIC.

0 is an invalid interrupt number except for the legacy timer interrupt on x86. The return value is unchecked in the core code, so it ends up to allocate interrupt number 0 which is subsequently considered to be invalid by the caller, e.g. the MSI allocation code.

The function has already a check for 0 in the case that an IO/APIC is registered, but ioapic_dynirq_base is 0 in case of device tree setups.

Consolidate this and zero check for both ioapic_dynirq_base and gsi_top, which is used in the case that no IO/APIC is registered.

Fixes 3e5bedc2c258 ("x86/apic: Fix arch_dynirq_lower_bound() bug for DT enabled machines")
Cc Andy Shevchenko <[email protected]>
Cc Thomas Gleixner <[email protected]>

---
[V2]
- Edit commit message
- Consolidated the 0 check for ioapic_dynirq_base as well

arch/x86/kernel/apic/io_apic.c | 14 +++++++++-----

1 file changed, 9 insertions(+), 5 deletions(-)
arch/x86/kernel/apic/io_apic.c +4 -4
index 1f83b052bb74..f980b38b0227 100644
@@ -2477,17 +2477,21 @@ static int io_apic_get_redir_entries(int ioapic)
 
 unsigned int arch_dynirq_lower_bound(unsigned int from)
 {
+	unsigned int ret;
+
 	/*
 	 * dmar_alloc_hwirq() may be called before setup_IO_APIC(), so use
 	 * gsi_top if ioapic_dynirq_base hasn't been initialized yet.
 	 */
-	if (!ioapic_initialized)
-		return gsi_top;
+	ret = ioapic_dynirq_base ? : gsi_top;
+
 	/*
-	 * For DT enabled machines ioapic_dynirq_base is irrelevant and not
-	 * updated. So simply return @from if ioapic_dynirq_base == 0.

arch/x86/kernel/apic/io_apic.c static int io_apic_get_redir_entries(int ioapic) · line 2497

Quoted 10 lines
arch/x86/kernel/apic/io_apic.c +5 -1
+	 * For DT enabled machines ioapic_dynirq_base is irrelevant and
+	 * always 0. gsi_top can be 0 if there is no IO/APIC registered.
+	 * 0 is an invalid interrupt number for dynamic allocations. Return
+	 * @from instead.
 	 */
-	return ioapic_dynirq_base ? : from;
+	return ret ? : from;
 }
 
 #ifdef CONFIG_X86_32

Is there any reason why this variable can't be initialized to a fixed nonzero number, like 16?

#2 lore raw
Quoted 65 lines · H. Peter Anvin
Quoted 64 lines · Saurabh Sengar

arch_dynirq_lower_bound() is invoked by the core interrupt code to retrieve the lowest possible Linux interrupt number for dynamically allocated interrupts like MSI.

The x86 implementation uses this to exclude the IO/APIC GSI space. This works correctly as long as there is an IO/APIC registered, but returns 0 if not. This has been observed in VMs where the BIOS does not advertise an IO/APIC.

0 is an invalid interrupt number except for the legacy timer interrupt on x86. The return value is unchecked in the core code, so it ends up to allocate interrupt number 0 which is subsequently considered to be invalid by the caller, e.g. the MSI allocation code.

The function has already a check for 0 in the case that an IO/APIC is registered, but ioapic_dynirq_base is 0 in case of device tree setups.

Consolidate this and zero check for both ioapic_dynirq_base and gsi_top, which is used in the case that no IO/APIC is registered.

Fixes 3e5bedc2c258 ("x86/apic: Fix arch_dynirq_lower_bound() bug for DT enabled machines")
Cc Andy Shevchenko <[email protected]>
Cc Thomas Gleixner <[email protected]>

---
[V2]
- Edit commit message
- Consolidated the 0 check for ioapic_dynirq_base as well

arch/x86/kernel/apic/io_apic.c | 14 +++++++++-----

1 file changed, 9 insertions(+), 5 deletions(-)
arch/x86/kernel/apic/io_apic.c +9 -5
index 1f83b052bb74..f980b38b0227 100644
@@ -2477,17 +2477,21 @@ static int io_apic_get_redir_entries(int ioapic)
 
 unsigned int arch_dynirq_lower_bound(unsigned int from)
 {
+	unsigned int ret;
+
 	/*
 	 * dmar_alloc_hwirq() may be called before setup_IO_APIC(), so use
 	 * gsi_top if ioapic_dynirq_base hasn't been initialized yet.
 	 */
-	if (!ioapic_initialized)
-		return gsi_top;
+	ret = ioapic_dynirq_base ? : gsi_top;
+
 	/*
-	 * For DT enabled machines ioapic_dynirq_base is irrelevant and not
-	 * updated. So simply return @from if ioapic_dynirq_base == 0.
+	 * For DT enabled machines ioapic_dynirq_base is irrelevant and
+	 * always 0. gsi_top can be 0 if there is no IO/APIC registered.
+	 * 0 is an invalid interrupt number for dynamic allocations. Return
+	 * @from instead.
 	 */
-	return ioapic_dynirq_base ? : from;
+	return ret ? : from;
 }
 
Quoted 3 lines

arch/x86/kernel/apic/io_apic.c static int io_apic_get_redir_entries(int ioapic) · line 2497

Quoted 1 line
arch/x86/kernel/apic/io_apic.c +0 -0
 #ifdef CONFIG_X86_32

Is there any reason why this variable can't be initialized to a fixed nonzero number, like 16?

Yes, initializing gst_top to any non-zero value should fix this issue. At first I thought to intialize gst_top to 1. But then I looked at how the ioapic_dynirq_base case is handled and followed a similar approach.

Regards,
Saurabh

#3 lore raw
Quoted 6 lines · Saurabh Singh Sengar
Quoted 2 lines · H. Peter Anvin

Is there any reason why this variable can't be initialized to a fixed nonzero number, like 16?

Yes, initializing gst_top to any non-zero value should fix this issue. At first I thought to intialize gst_top to 1.

That works only in your case. Some boot time registrations of IO_APICs use gsi_top as the base. So initializing gsi_top to N would move IOAPIC[0] interrupts out to irq N... and make the legacy interrupts fail.

That whole IOAPIC registration could do with some major cleanup, but that's a different story.

Thanks,

tglx

#4 lore raw
In reply to Saurabh Sengar
Quoted 55 lines · Saurabh Sengar

arch_dynirq_lower_bound() is invoked by the core interrupt code to retrieve the lowest possible Linux interrupt number for dynamically allocated interrupts like MSI.

The x86 implementation uses this to exclude the IO/APIC GSI space. This works correctly as long as there is an IO/APIC registered, but returns 0 if not. This has been observed in VMs where the BIOS does not advertise an IO/APIC.

0 is an invalid interrupt number except for the legacy timer interrupt on x86. The return value is unchecked in the core code, so it ends up to allocate interrupt number 0 which is subsequently considered to be invalid by the caller, e.g. the MSI allocation code.

The function has already a check for 0 in the case that an IO/APIC is registered, but ioapic_dynirq_base is 0 in case of device tree setups.

Consolidate this and zero check for both ioapic_dynirq_base and gsi_top, which is used in the case that no IO/APIC is registered.

Fixes 3e5bedc2c258 ("x86/apic: Fix arch_dynirq_lower_bound() bug for DT enabled machines")
Cc Andy Shevchenko <[email protected]>
Cc Thomas Gleixner <[email protected]>

---
[V2]
- Edit commit message
- Consolidated the 0 check for ioapic_dynirq_base as well

 arch/x86/kernel/apic/io_apic.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
arch/x86/kernel/apic/io_apic.c +4 -4
index 1f83b052bb74..f980b38b0227 100644
@@ -2477,17 +2477,21 @@ static int io_apic_get_redir_entries(int ioapic)
 
 unsigned int arch_dynirq_lower_bound(unsigned int from)
 {
+	unsigned int ret;
+
 	/*
 	 * dmar_alloc_hwirq() may be called before setup_IO_APIC(), so use
 	 * gsi_top if ioapic_dynirq_base hasn't been initialized yet.
 	 */
-	if (!ioapic_initialized)
-		return gsi_top;
+	ret = ioapic_dynirq_base ? : gsi_top;
+
 	/*
-	 * For DT enabled machines ioapic_dynirq_base is irrelevant and not
-	 * updated. So simply return @from if ioapic_dynirq_base == 0.

arch/x86/kernel/apic/io_apic.c static int io_apic_get_redir_entries(int ioapic) · line 2497

Quoted 10 lines
arch/x86/kernel/apic/io_apic.c +5 -1
+	 * For DT enabled machines ioapic_dynirq_base is irrelevant and
+	 * always 0. gsi_top can be 0 if there is no IO/APIC registered.
+	 * 0 is an invalid interrupt number for dynamic allocations. Return
+	 * @from instead.
 	 */
-	return ioapic_dynirq_base ? : from;
+	return ret ? : from;
 }
 
 #ifdef CONFIG_X86_32

Is this good to get accepted ? Please let me know if anything pending from my end on this.

- Saurabh

Quoted 2 lines
Signature · 1 line
2.34.1
merge bot Applied
#5 lore raw
In reply to Saurabh Sengar
The following commit has been merged into the x86/apic branch of tip:

The following commit has been merged into the x86/apic branch of tip:

Commit-ID:     5af507bef93c09a94fb8f058213b489178f4cbe5
Gitweb:        https://git.kernel.org/tip/5af507bef93c09a94fb8f058213b489178f4cbe5
Author:        Saurabh Sengar <[email protected]>
AuthorDate:    Tue, 28 Mar 2023 00:30:04 -07:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 12 Apr 2023 17:45:50 +02:00

x86/ioapic: Don't return 0 from arch_dynirq_lower_bound()

arch_dynirq_lower_bound() is invoked by the core interrupt code to retrieve the lowest possible Linux interrupt number for dynamically allocated interrupts like MSI.

The x86 implementation uses this to exclude the IO/APIC GSI space. This works correctly as long as there is an IO/APIC registered, but returns 0 if not. This has been observed in VMs where the BIOS does not advertise an IO/APIC.

0 is an invalid interrupt number except for the legacy timer interrupt on x86. The return value is unchecked in the core code, so it ends up to allocate interrupt number 0 which is subsequently considered to be invalid by the caller, e.g. the MSI allocation code.

The function has already a check for 0 in the case that an IO/APIC is registered, as ioapic_dynirq_base is 0 in case of device tree setups.

Consolidate this and zero check for both ioapic_dynirq_base and gsi_top, which is used in the case that no IO/APIC is registered.

Fixes 3e5bedc2c258 ("x86/apic: Fix arch_dynirq_lower_bound() bug for DT enabled machines")
---
 arch/x86/kernel/apic/io_apic.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
arch/x86/kernel/apic/io_apic.c +9 -5
index 1f83b05..f980b38 100644
@@ -2477,17 +2477,21 @@ static int io_apic_get_redir_entries(int ioapic)
 
 unsigned int arch_dynirq_lower_bound(unsigned int from)
 {
+	unsigned int ret;
+
 	/*
 	 * dmar_alloc_hwirq() may be called before setup_IO_APIC(), so use
 	 * gsi_top if ioapic_dynirq_base hasn't been initialized yet.
 	 */
-	if (!ioapic_initialized)
-		return gsi_top;
+	ret = ioapic_dynirq_base ? : gsi_top;
+
 	/*
-	 * For DT enabled machines ioapic_dynirq_base is irrelevant and not
-	 * updated. So simply return @from if ioapic_dynirq_base == 0.
+	 * For DT enabled machines ioapic_dynirq_base is irrelevant and
+	 * always 0. gsi_top can be 0 if there is no IO/APIC registered.
+	 * 0 is an invalid interrupt number for dynamic allocations. Return
+	 * @from instead.
 	 */
-	return ioapic_dynirq_base ? : from;
+	return ret ? : from;
 }
 
 #ifdef CONFIG_X86_32