Skip to contentSkip to conversation

Use NUMA node cpu mask in irq affinity

PATCH In review Prarit Bhargava 3 participants 4 messages Feb 8, 2012 – Feb 9, 2012 IRQ

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Use NUMA node cpu mask in irq affinity

The irq affinity files (/proc/irq/.../smp_affinity) contain a mask that is used to "pin" an irq to a set of cpus. On boot this set is currently all cpus. This can be incorrect as ACPI SRAT may tell us that a specific device or bus is attached to a particular node and it's cpus.

When setting up the irq affinity we should take into account the NUMA node cpu mask by and'ing it into the irq's affinity mask.

---
 kernel/irq/manage.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
kernel/irq/manage.c +2 -0
index a9a9dbe..2fb3469 100644
@@ -301,6 +301,8 @@ setup_affinity(unsigned int irq, struct irq_desc *desc, struct cpumask *mask)
 	}
 
 	cpumask_and(mask, cpu_online_mask, set);
+	if (desc->irq_data.node != -1)
+		cpumask_and(mask, mask, cpumask_of_node(desc->irq_data.node));
 	ret = chip->irq_set_affinity(&desc->irq_data, mask, false);
 	switch (ret) {
 	case IRQ_SET_MASK_OK:
Signature · 1 line
1.7.1
#1 lore raw
Quoted 26 lines · Prarit Bhargava

The irq affinity files (/proc/irq/.../smp_affinity) contain a mask that is used to "pin" an irq to a set of cpus.  On boot this set is currently all cpus. This can be incorrect as ACPI SRAT may tell us that a specific device or bus is attached to a particular node and it's cpus.

When setting up the irq affinity we should take into account the NUMA node cpu mask by and'ing it into the irq's affinity mask.

---
 kernel/irq/manage.c |    2 ++

 1 files changed, 2 insertions(+), 0 deletions(-)
kernel/irq/manage.c +2 -0
index a9a9dbe..2fb3469 100644
@@ -301,6 +301,8 @@ setup_affinity(unsigned int irq, struct irq_desc *desc, struct cpumask *mask)
        }
 
        cpumask_and(mask, cpu_online_mask, set);
+       if (desc->irq_data.node != -1)
+               cpumask_and(mask, mask, cpumask_of_node(desc->irq_data.node));
        ret = chip->irq_set_affinity(&desc->irq_data, mask, false);
        switch (ret) {
Quoted 2 lines

       case IRQ_SET_MASK_OK:
--

How about all cpus on that node get offlined?

Yinghai

#2 lore raw
Quoted 22 lines · Yinghai Lu
Quoted 21 lines · Prarit Bhargava

The irq affinity files (/proc/irq/.../smp_affinity) contain a mask that is used to "pin" an irq to a set of cpus. On boot this set is currently all cpus. This can be incorrect as ACPI SRAT may tell us that a specific device or bus is attached to a particular node and it's cpus.

When setting up the irq affinity we should take into account the NUMA node cpu mask by and'ing it into the irq's affinity mask.

---
 kernel/irq/manage.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
kernel/irq/manage.c +0 -0
index a9a9dbe..2fb3469 100644
@@ -301,6 +301,8 @@ setup_affinity(unsigned int irq, struct irq_desc *desc, struct cpumask *mask)
       }
 
Quoted 9 lines

kernel/irq/manage.c setup_affinity(unsigned int irq, struct irq_desc *desc, struct cpumask *mask)

Quoted 7 lines
kernel/irq/manage.c +2 -1
       cpumask_and(mask, cpu_online_mask, set);
+       if (desc->irq_data.node != -1)
+               cpumask_and(mask, mask, cpumask_of_node(desc->irq_data.node));
       ret = chip->irq_set_affinity(&desc->irq_data, mask, false);
       switch (ret) {
       case IRQ_SET_MASK_OK:
--

How about all cpus on that node get offlined?

Good point. I guess that also goes to tglx's comments wondering what happens if the mask ends up being zero.

I'll think about that and send out a new patch shortly.

P.

Quoted 2 lines

Yinghai

#3 lore raw
In reply to Prarit Bhargava

On Wed, 8 Feb 2012, Prarit Bhargava wrote:

Could you please CC the relevant maintainers ?

Quoted 19 lines

The irq affinity files (/proc/irq/.../smp_affinity) contain a mask that is used to "pin" an irq to a set of cpus. On boot this set is currently all cpus. This can be incorrect as ACPI SRAT may tell us that a specific device or bus is attached to a particular node and it's cpus.

When setting up the irq affinity we should take into account the NUMA node cpu mask by and'ing it into the irq's affinity mask.

---
 kernel/irq/manage.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
kernel/irq/manage.c +0 -0
index a9a9dbe..2fb3469 100644

kernel/irq/manage.c setup_affinity(unsigned int irq, struct irq_desc *desc, struct cpumask *mask) · line 305

Quoted 5 lines
kernel/irq/manage.c +2 -0
@@ -301,6 +301,8 @@ setup_affinity(unsigned int irq, struct irq_desc *desc, struct cpumask *mask)
 	}
 
 	cpumask_and(mask, cpu_online_mask, set);
+	if (desc->irq_data.node != -1)
+		cpumask_and(mask, mask, cpumask_of_node(desc->irq_data.node));

What prevents mask from becoming empty if mask does not intersect with the node mask?

Thanks,

tglx