Skip to contentSkip to conversation

Increase the number of IRQ descriptors for SPARSEIRQ

PATCH Superseded Shanker Donthineni 1 participant 4 messages May 10, 2023 IRQ

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover Increase the number of IRQ descriptors for SPARSEIRQ

The ARM64 architecture uses SPARSEIRQ with a default value of NR_IRQS, which is set to 64. This means that only 64+8192 IRQ descriptors are allowed, which may not be sufficient for modern ARM64 servers that have a large number of IO devices and GIC hardware that supports direct vSGI and vLPI injection features.

This limitation has caused issues when attempting to launch multiple virtual machines with GICv4.1 features, resulting in the error message 'kvm_err("VPE IRQ allocation failure\n")'. The root cause of this issue is the ~8K IRQ descriptor limit.

To address this issue, an initial proposal was made to define NR_IRQS to 2^19 for ARM64. However, Marc Zyngier suggested implementing a generic solution instead of hard-coded values. Thomas Gleixner advised to use the maple tree data structure and provided most of the necessary functions.

For more information, refer to the discussion thread at
https://lore.kernel.org/linux-arm-kernel/[email protected]/.

This patch series converts the static memory allocation to dynamic using the maple tree, and increases the maximum number of IRQ descriptors to INT_MAX from NR_IRQS+8192. This change has been tested on an ARM64 server with CONFIG_SPARSE_IRQ=y, where 256 virtual machines were launched, creating a total of 128K+ IRQ descriptors, and IRQ injection was verified.

Changes in v4:
- Fix the iterator function using mt_find() instead of nt_next()
- Tested CPU hot-unplug and hot-plug on ARM64 system. This is completely broken before v4.

Changes in v3:
- Edited commit text
- Added a helper function irq_resend_init()
- Rebased to v6.3-rc6

Changes in v2:
- The patches have been updated to v6.3-rc5.
- The patches 2/5 and 4/5 have been removed as they are unnecessary.
- The review comments from Thomas have been addressed.

Shanker Donthineni (3):
genirq: Use hlist for managing resend handlers
genirq: Encapsulate sparse bitmap handling
genirq: Use the maple tree for IRQ descriptors management

 include/linux/irqdesc.h |  3 ++
 kernel/irq/chip.c       |  1 +
 kernel/irq/internals.h  |  6 ++--
 kernel/irq/irqdesc.c    | 77 +++++++++++++++++++++++++----------------
 kernel/irq/resend.c     | 47 ++++++++++++++++---------
 5 files changed, 86 insertions(+), 48 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

PATCH v4 1/3 genirq: Use hlist for managing resend handlers

The current implementation utilizes a bitmap for managing IRQ resend handlers, which is allocated based on the SPARSE_IRQ/NR_IRQS macros. However, this method may not efficiently utilize memory during runtime, particularly when IRQ_BITMAP_BITS is large.

Address this issue by using the hlist to manage IRQ resend handlers instead of relying on a static bitmap memory allocation. Additionally, a new function, clear_irq_resend(), is introduced and called from irq_shutdown to ensure a graceful teardown of the interrupt.

---
 include/linux/irqdesc.h |  3 +++
 kernel/irq/chip.c       |  1 +
 kernel/irq/internals.h  |  2 ++
 kernel/irq/irqdesc.c    |  2 ++
 kernel/irq/resend.c     | 47 ++++++++++++++++++++++++++---------------
 5 files changed, 38 insertions(+), 17 deletions(-)
include/linux/irqdesc.h +3 -0
index 844a8e30e6de..d9451d456a73 100644
@@ -102,6 +102,9 @@ struct irq_desc {
 	int			parent_irq;
 	struct module		*owner;
 	const char		*name;
+#ifdef CONFIG_HARDIRQS_SW_RESEND
+	struct hlist_node	resend_node;
+#endif
 } ____cacheline_internodealigned_in_smp;
 
 #ifdef CONFIG_SPARSE_IRQ
kernel/irq/chip.c +1 -0
index 49e7bc871fec..2eac5532c3c8 100644
@@ -306,6 +306,7 @@ static void __irq_disable(struct irq_desc *desc, bool mask);
 void irq_shutdown(struct irq_desc *desc)
 {
 	if (irqd_is_started(&desc->irq_data)) {
+		clear_irq_resend(desc);
 		desc->depth = 1;
 		if (desc->irq_data.chip->irq_shutdown) {
 			desc->irq_data.chip->irq_shutdown(&desc->irq_data);
kernel/irq/internals.h +2 -0
index 5fdc0b557579..51fc8c497c22 100644
@@ -113,6 +113,8 @@ irqreturn_t handle_irq_event(struct irq_desc *desc);
 
 /* Resending of interrupts :*/
 int check_irq_resend(struct irq_desc *desc, bool inject);
+void clear_irq_resend(struct irq_desc *desc);
+void irq_resend_init(struct irq_desc *desc);
 bool irq_wait_for_poll(struct irq_desc *desc);
 void __irq_wake_thread(struct irq_desc *desc, struct irqaction *action);
 
kernel/irq/irqdesc.c +2 -0
index fd0996274401..f4cb34c87ae7 100644
@@ -415,6 +415,7 @@ static struct irq_desc *alloc_desc(int irq, int node, unsigned int flags,
 	desc_set_defaults(irq, desc, node, affinity, owner);
 	irqd_set(&desc->irq_data, flags);
 	kobject_init(&desc->kobj, &irq_kobj_type);
+	irq_resend_init(desc);
 
 	return desc;
 
@@ -581,6 +582,7 @@ int __init early_irq_init(void)
 		mutex_init(&desc[i].request_mutex);
 		init_waitqueue_head(&desc[i].wait_for_threads);
 		desc_set_defaults(i, &desc[i], node, NULL, NULL);
+		irq_resend_init(desc);
 	}
 	return arch_early_irq_init();
 }
kernel/irq/resend.c +30 -17

80 lines not on this page · raw message

Signature · 1 line
2.25.1
#2 lore raw

PATCH v4 2/3 genirq: Encapsulate sparse bitmap handling

In reply to Shanker Donthineni

Move the open coded sparse bitmap handling into helper functions as a preparatory step for converting the sparse interrupt management to a maple tree.

No functional change.

---
 kernel/irq/internals.h |  4 ++--
 kernel/irq/irqdesc.c   | 28 +++++++++++++++++++---------
 2 files changed, 21 insertions(+), 11 deletions(-)
kernel/irq/internals.h +2 -2
index 51fc8c497c22..f3f2090dd2de 100644
@@ -12,9 +12,9 @@
 #include <linux/sched/clock.h>
 
 #ifdef CONFIG_SPARSE_IRQ
-# define IRQ_BITMAP_BITS	(NR_IRQS + 8196)
+# define MAX_SPARSE_IRQS	(NR_IRQS + 8196)
 #else
-# define IRQ_BITMAP_BITS	NR_IRQS
+# define MAX_SPARSE_IRQS	NR_IRQS
 #endif
 
 #define istate core_internal_state__do_not_mess_with_it
kernel/irq/irqdesc.c +19 -9
index f4cb34c87ae7..a741a37d3641 100644
@@ -131,7 +131,18 @@ int nr_irqs = NR_IRQS;
 EXPORT_SYMBOL_GPL(nr_irqs);
 
 static DEFINE_MUTEX(sparse_irq_lock);
-static DECLARE_BITMAP(allocated_irqs, IRQ_BITMAP_BITS);
+static DECLARE_BITMAP(allocated_irqs, MAX_SPARSE_IRQS);
+
+static int irq_find_free_area(unsigned int from, unsigned int cnt)
+{
+	return bitmap_find_next_zero_area(allocated_irqs, MAX_SPARSE_IRQS,
+					  from, cnt, 0);
+}
+
+static unsigned int irq_find_next_irq(unsigned int offset)
+{
+	return find_next_bit(allocated_irqs, nr_irqs, offset);
+}
 
 #ifdef CONFIG_SPARSE_IRQ
 
@@ -517,7 +528,7 @@ static int alloc_descs(unsigned int start, unsigned int cnt, int node,
 
 static int irq_expand_nr_irqs(unsigned int nr)
 {
-	if (nr > IRQ_BITMAP_BITS)
+	if (nr > MAX_SPARSE_IRQS)
 		return -ENOMEM;
 	nr_irqs = nr;
 	return 0;
@@ -535,11 +546,11 @@ int __init early_irq_init(void)
 	printk(KERN_INFO "NR_IRQS: %d, nr_irqs: %d, preallocated irqs: %d\n",
 	       NR_IRQS, nr_irqs, initcnt);
 
-	if (WARN_ON(nr_irqs > IRQ_BITMAP_BITS))
-		nr_irqs = IRQ_BITMAP_BITS;
+	if (WARN_ON(nr_irqs > MAX_SPARSE_IRQS))
+		nr_irqs = MAX_SPARSE_IRQS;
 
-	if (WARN_ON(initcnt > IRQ_BITMAP_BITS))
-		initcnt = IRQ_BITMAP_BITS;
+	if (WARN_ON(initcnt > MAX_SPARSE_IRQS))
+		initcnt = MAX_SPARSE_IRQS;
 
 	if (initcnt > nr_irqs)
 		nr_irqs = initcnt;
@@ -812,8 +823,7 @@ __irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
 
 	mutex_lock(&sparse_irq_lock);
 
-	start = bitmap_find_next_zero_area(allocated_irqs, IRQ_BITMAP_BITS,
-					   from, cnt, 0);
+	start = irq_find_free_area(from, cnt);
 	ret = -EEXIST;
 	if (irq >=0 && start != irq)
 		goto unlock;
@@ -838,7 +848,7 @@ EXPORT_SYMBOL_GPL(__irq_alloc_descs);
  */
 unsigned int irq_get_next_irq(unsigned int offset)
 {
-	return find_next_bit(allocated_irqs, nr_irqs, offset);
+	return irq_find_next_irq(offset);
 }
 
 struct irq_desc *
Signature · 1 line
2.25.1
#3 lore raw

PATCH v4 3/3 genirq: Use the maple tree for IRQ descriptors management

In reply to Shanker Donthineni

The current implementation uses a static bitmap and a radix tree to manage IRQ allocation and irq_desc pointer store respectively. However, the size of the bitmap is constrained by the build time macro MAX_SPARSE_IRQS, which may not be sufficient to support the high-end servers, particularly those with GICv4.1 hardware, which require a large interrupt space to cover LPIs and vSGIs.

The maple tree is a highly efficient data structure for storing non-overlapping ranges and can handle a large number of entries, up to ULONG_MAX. It can be utilized for both storing interrupt descriptors and identifying available free spaces.

The interrupt descriptors management can be simplified by switching to a maple tree data structure, which offers greater flexibility and scalability. To support modern servers, the maximum number of IRQs has been increased to INT_MAX, which provides a more adequate value than the previous limit of NR_IRQS+8192.

---
 kernel/irq/internals.h |  2 +-
 kernel/irq/irqdesc.c   | 55 ++++++++++++++++++++++++------------------
 2 files changed, 32 insertions(+), 25 deletions(-)
kernel/irq/internals.h +1 -1
index f3f2090dd2de..7bdb7507efb0 100644
@@ -12,7 +12,7 @@
 #include <linux/sched/clock.h>
 
 #ifdef CONFIG_SPARSE_IRQ
-# define MAX_SPARSE_IRQS	(NR_IRQS + 8196)
+# define MAX_SPARSE_IRQS	INT_MAX
 #else
 # define MAX_SPARSE_IRQS	NR_IRQS
 #endif
kernel/irq/irqdesc.c +31 -24

124 lines not on this page · raw message

Signature · 1 line
2.25.1