Skip to contentSkip to conversation

smp_affinity, MSI-X and 2.6.21.1

DISCUSSION Rick Jones 4 participants 7 messages May 7, 2007 – May 15, 2007

Subsystems (3)
Networking Network drivers IRQ
lore
Share & tools
lore

Rick Jones

author
#0 lore raw

Folks -

Is it a bug, or a feature that after changing a device's smp_affinity via echo "N" >> /proc/irq/M/smp_affinity that the new mask isn't visible via cat /proc/irq/M/smp_affinity until after actual interrupts are taken?

rick jones

#1 lore raw
Quoted 5 lines · Rick Jones

Folks -

Is it a bug, or a feature that after changing a device's smp_affinity via echo "N" >> /proc/irq/M/smp_affinity that the new mask isn't visible via cat /proc/irq/M/smp_affinity until after actual interrupts are taken?\

that's known and expected behavior

author
#2 lore raw
Quoted 8 lines · Arjan van de Ven
Quoted 3 lines · Rick Jones

Folks -

Is it a bug, or a feature that after changing a device's smp_affinity via echo "N" >> /proc/irq/M/smp_affinity that the new mask isn't visible via cat /proc/irq/M/smp_affinity until after actual interrupts are taken?\

that's known and expected behavior

well, expected by those in the know anyway :)

thanks.

rick jones

#3 lore raw
In reply to Rick Jones
Quoted 6 lines · Rick Jones

Folks -

Is it a bug, or a feature that after changing a device's smp_affinity via echo "N" >> /proc/irq/M/smp_affinity that the new mask isn't visible via cat /proc/irq/M/smp_affinity until after actual interrupts are taken?

Intel chipsets can only safely update affinity during interrupt processing. You see a side effect of the code implementing this restriction.

-Andi

maintainer
#4 lore raw

From: Andi Kleen <[email protected]>
Date: 09 May 2007 12:35:29 +0200

Quoted 11 lines
Quoted 6 lines · Rick Jones

Folks -

Is it a bug, or a feature that after changing a device's smp_affinity via echo "N" >> /proc/irq/M/smp_affinity that the new mask isn't visible via cat /proc/irq/M/smp_affinity until after actual interrupts are taken?

Intel chipsets can only safely update affinity during interrupt processing. You see a side effect of the code implementing this restriction.

That's true, but we are talking about software state so in some sense it might be better that the affinity-to-be is reported to the user in this case.

Delayed register updates are an implementation detail the user does not need to know about here.

#5 lore raw
Quoted 6 lines

That's true, but we are talking about software state so in some sense it might be better that the affinity-to-be is reported to the user in this case.

Delayed register updates are an implementation detail the user does not need to know about here.

This patch should fix it.
-Andi

Report the pending irq if available in smp_affinity

Otherwise smp_affinity would only update after the next interrupt on x86 systems.

Cc [email protected]
Cc [email protected]
Signed-off-by Andi Kleen <[email protected]>
kernel/irq/proc.c +8 -1
Index: linux/kernel/irq/proc.c
===================================================================
@@ -19,7 +19,14 @@ static struct proc_dir_entry *root_irq_d
 static int irq_affinity_read_proc(char *page, char **start, off_t off,
 				  int count, int *eof, void *data)
 {
-	int len = cpumask_scnprintf(page, count, irq_desc[(long)data].affinity);
+	struct irq_desc *desc = irq_desc + (long)data;
+	cpumask_t *mask = &desc->affinity;
+	int len;
+#ifdef CONFIG_GENERIC_PENDING_IRQ
+	if (desc->status & IRQ_MOVE_PENDING)
+		mask = &desc->pending_mask;
+#endif
+	len = cpumask_scnprintf(page, count, *mask);
 
 	if (count - len < 2)
 		return -EINVAL;
author
#6 lore raw
Quoted 9 lines · Andi Kleen
Quoted 6 lines

That's true, but we are talking about software state so in some sense it might be better that the affinity-to-be is reported to the user in this case.

Delayed register updates are an implementation detail the user does not need to know about here.

This patch should fix it.

And it seems to when I apply it against the 2.6.21.1 kernel I'm messing about with:

hpcpc106:~/s2io-2.0.19-8893# cat /proc/irq/69/smp_affinity
ffffffff,ffffffff
hpcpc106:~/s2io-2.0.19-8893# echo "4" >> /proc/irq/69/smp_affinity
hpcpc106:~/s2io-2.0.19-8893# cat /proc/irq/69/smp_affinity
00000000,00000004
hpcpc106:~/s2io-2.0.19-8893# cat /proc/interrupts | grep 69
 69:          0          0          0          0         PCI-MSI  eth2:MSI-X-6-RX

It would be nice if this could find its way into the kernel at some point - 2.6.23 or 2.6.24 perhaps?

rick jones

Quoted 32 lines

-Andi

Report the pending irq if available in smp_affinity

Otherwise smp_affinity would only update after the next interrupt on x86 systems.

Cc [email protected]
Cc [email protected]
Signed-off-by Andi Kleen <[email protected]>

Index: linux/kernel/irq/proc.c
===================================================================

kernel/irq/proc.c +8 -1
@@ -19,7 +19,14 @@ static struct proc_dir_entry *root_irq_d
 static int irq_affinity_read_proc(char *page, char **start, off_t off,
 				  int count, int *eof, void *data)
 {
-	int len = cpumask_scnprintf(page, count, irq_desc[(long)data].affinity);
+	struct irq_desc *desc = irq_desc + (long)data;
+	cpumask_t *mask = &desc->affinity;
+	int len;
+#ifdef CONFIG_GENERIC_PENDING_IRQ
+	if (desc->status & IRQ_MOVE_PENDING)
+		mask = &desc->pending_mask;
+#endif
+	len = cpumask_scnprintf(page, count, *mask);
 
 	if (count - len < 2)
 		return -EINVAL;