Skip to contentSkip to conversation

sparseirq: Increase nr_irqs if needed

PATCH New Yinghai Lu 2 participants 4 messages Nov 23, 2010 – Dec 28, 2010 7 messages not in the archive

Subsystems (2)
lore
Share & tools
lore

Yinghai Lu

author
#0 lore raw

PATCH 1/1 sparseirq: Increase nr_irqs if needed

only do that when we need more.

For x86_64 system:
when we have 128 cpus with 5 ioapics, will have nr_irqs = 3064

	120 + 8 * 128 + 120 * 16

systems could take 20 pcie, when intel 10g are used with sriov and ixgbevf, every vf will need 3 irqs, and one device have 64 vf. so will need 20 * 3 * 64 = 3840.
some 6 ports Intel 10gb may need more.

with this patch, nr_irqs will increase 25% every time if nr_irqs is not big enough.

Also later We could change nr_irqs initial value to

	 min(nr_irqs_gsi, 224*nr_cpu_ids).

for x86.

---
 kernel/irq/irqdesc.c |   29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)
kernel/irq/irqdesc.c +26 -3
Index: linux-2.6/kernel/irq/irqdesc.c
===================================================================
@@ -301,6 +301,27 @@ void irq_free_descs(unsigned int from, u
 	mutex_unlock(&sparse_irq_lock);
 }
 
+static bool increase_nr_irqs(void)
+{
+#ifdef CONFIG_SPARSE_IRQ
+	int new;
+
+	if (nr_irqs == NR_IRQS)
+		return false;
+
+	new = nr_irqs;
+	new += nr_irqs >> 2;
+	if (new > NR_IRQS)
+		new = NR_IRQS;
+	if (new > nr_irqs) {
+		nr_irqs = new;
+		printk(KERN_INFO "nr_irqs increase to %d\n", nr_irqs);
+		return true;
+	}
+#endif
+	return false;
+}
+
 /**
  * irq_alloc_descs - allocate and initialize a range of irq descriptors
  * @irq:	Allocate for specific irq number if irq >= 0
@@ -320,14 +341,16 @@ irq_alloc_descs(int irq, unsigned int fr
 
 	mutex_lock(&sparse_irq_lock);
 
-	start = bitmap_find_next_zero_area(allocated_irqs, nr_irqs, from, cnt, 0);
+	start = bitmap_find_next_zero_area(allocated_irqs, NR_IRQS, from, cnt, 0);
 	ret = -EEXIST;
 	if (irq >=0 && start != irq)
 		goto err;
 
 	ret = -ENOMEM;
-	if (start >= nr_irqs)
-		goto err;
+	while ((start + cnt) >= nr_irqs) {
+		if (!increase_nr_irqs())
+			goto err;
+	}
 
 	bitmap_set(allocated_irqs, start, cnt);
 	mutex_unlock(&sparse_irq_lock);

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#8 lore raw

PATCH 1/1 x86, sparseirq: let nr_irqs equal to NR_IRQS

For x86_64 system:
when we have 128 cpus with 5 ioapics, will have nr_irqs = 3064

	120 + 8 * 128 + 120 * 16

systems could take 20 pcie, when intel 10g are used with sriov and ixgbevf, every vf will need 3 irqs, and one device have 64 vf. so will need 20 * 3 * 64 = 3840.
some 6 ports Intel 10gb may need more.

Just remove that function for x86, and let nr_irqs to NR_IRQS because We already have radix-tree and bit_map for searching desc for irq.

Notes: long before same vresion cause one of Ingo's setup udev hang...

---
 arch/x86/kernel/apic/io_apic.c |   22 ----------------------
 1 file changed, 22 deletions(-)
arch/x86/kernel/apic/io_apic.c +0 -22
Index: linux-2.6/arch/x86/kernel/apic/io_apic.c
===================================================================
@@ -3637,28 +3637,6 @@ int get_nr_irqs_gsi(void)
 	return nr_irqs_gsi;
 }
 
-#ifdef CONFIG_SPARSE_IRQ
-int __init arch_probe_nr_irqs(void)
-{
-	int nr;
-
-	if (nr_irqs > (NR_VECTORS * nr_cpu_ids))
-		nr_irqs = NR_VECTORS * nr_cpu_ids;
-
-	nr = nr_irqs_gsi + 8 * nr_cpu_ids;
-#if defined(CONFIG_PCI_MSI) || defined(CONFIG_HT_IRQ)
-	/*
-	 * for MSI and HT dyn irq
-	 */
-	nr += nr_irqs_gsi * 16;
-#endif
-	if (nr < nr_irqs)
-		nr_irqs = nr;
-
-	return NR_IRQS_LEGACY;
-}
-#endif
-
 static int __io_apic_set_pci_routing(struct device *dev, int irq,
 				struct io_apic_irq_attr *irq_attr)
 {
#9 lore raw

Re: [PATCH] x86, sparseirq: let nr_irqs equal to NR_IRQS

Quoted 4 lines · Yinghai Lu

For x86_64 system:
when we have 128 cpus with 5 ioapics, will have nr_irqs = 3064

Quoted 8 lines
	120 + 8 * 128 + 120 * 16

systems could take 20 pcie, when intel 10g are used with sriov and ixgbevf, every vf will need 3 irqs, and one device have 64 vf. so will need 20 * 3 * 64 = 3840.
some 6 ports Intel 10gb may need more.

Just remove that function for x86, and let nr_irqs to NR_IRQS because We already have radix-tree and bit_map for searching desc for irq.

Won't that make the arch_probe_nr_irqs be NR_IRQS_LEGACY?

I am looking at Linus's today tree, or is this patch based on some other tree?

Quoted 47 lines

Notes: long before same vresion cause one of Ingo's setup udev hang...

---
 arch/x86/kernel/apic/io_apic.c |   22 ----------------------
 1 file changed, 22 deletions(-)

Index: linux-2.6/arch/x86/kernel/apic/io_apic.c
===================================================================

arch/x86/kernel/apic/io_apic.c +0 -23
@@ -3637,28 +3637,6 @@ int get_nr_irqs_gsi(void)
 	return nr_irqs_gsi;
 }
 
-#ifdef CONFIG_SPARSE_IRQ
-int __init arch_probe_nr_irqs(void)
-{
-	int nr;
-
-	if (nr_irqs > (NR_VECTORS * nr_cpu_ids))
-		nr_irqs = NR_VECTORS * nr_cpu_ids;
-
-	nr = nr_irqs_gsi + 8 * nr_cpu_ids;
-#if defined(CONFIG_PCI_MSI) || defined(CONFIG_HT_IRQ)
-	/*
-	 * for MSI and HT dyn irq
-	 */
-	nr += nr_irqs_gsi * 16;
-#endif
-	if (nr < nr_irqs)
-		nr_irqs = nr;
-
-	return NR_IRQS_LEGACY;
-}
-#endif
-
 static int __io_apic_set_pci_routing(struct device *dev, int irq,
 				struct io_apic_irq_attr *irq_attr)
 {
--

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

author
#10 lore raw

Re: [PATCH] x86, sparseirq: let nr_irqs equal to NR_IRQS

Quoted 11 lines · Konrad Rzeszutek Wilk
Quoted 10 lines · Yinghai Lu

For x86_64 system:
when we have 128 cpus with 5 ioapics, will have nr_irqs = 3064 120 + 8 * 128 + 120 * 16
systems could take 20 pcie,  when intel 10g are used with sriov and ixgbevf, every vf will need 3 irqs, and one device have 64 vf. so will need 20 * 3 * 64 = 3840.
some 6 ports Intel 10gb may need more.

Quoted 4 lines
Quoted 2 lines

Just remove that function for x86, and let nr_irqs to NR_IRQS because We already have radix-tree and bit_map for searching desc for irq.

Won't that make the arch_probe_nr_irqs be NR_IRQS_LEGACY?

int nr_irqs = NR_IRQS;

and

int __init __weak arch_probe_nr_irqs(void)

{
        return NR_IRQS_LEGACY;
}

int __init early_irq_init(void)

{
        int i, initcnt, node = first_online_node;
        struct irq_desc *desc;

        init_irq_default_affinity();

        /* Let arch update nr_irqs and return the nr of preallocated irqs */
        initcnt = arch_probe_nr_irqs();
        printk(KERN_INFO "NR_IRQS:%d nr_irqs:%d %d\n", NR_IRQS,

nr_irqs, initcnt);

that arch_probe_nr_irqs() is return for initcnt...

should change nr_irqs in the functions...

some confusing...

Quoted 56 lines

I am looking at Linus's today tree, or is this patch based on some other tree?

Quoted 43 lines

Notes: long before same vresion cause one of Ingo's setup udev hang...

---
 arch/x86/kernel/apic/io_apic.c |   22 ----------------------

 1 file changed, 22 deletions(-)

Index: linux-2.6/arch/x86/kernel/apic/io_apic.c
===================================================================

arch/x86/kernel/apic/io_apic.c +0 -23
@@ -3637,28 +3637,6 @@ int get_nr_irqs_gsi(void)
       return nr_irqs_gsi;
  }
 
-#ifdef CONFIG_SPARSE_IRQ
-int __init arch_probe_nr_irqs(void)
-{
-     int nr;
-
-     if (nr_irqs > (NR_VECTORS * nr_cpu_ids))
-             nr_irqs = NR_VECTORS * nr_cpu_ids;
-
-     nr = nr_irqs_gsi + 8 * nr_cpu_ids;
-#if defined(CONFIG_PCI_MSI) || defined(CONFIG_HT_IRQ)
-     /*
-      * for MSI and HT dyn irq
-      */
-     nr += nr_irqs_gsi * 16;
-#endif
-     if (nr < nr_irqs)
-             nr_irqs = nr;
-
-     return NR_IRQS_LEGACY;
-}
-#endif
-
  static int __io_apic_set_pci_routing(struct device *dev, int irq,
                               struct io_apic_irq_attr *irq_attr)
  {
--
Quoted 4 lines

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/