genirq: avoid long loops in handle_edge_irq
PATCH v2 1/1 genirq: avoid long loops in handle_edge_irq
When there are a large number of interrupts occurring on the tx queue(irq smp_affinity=1) of the network card, changing the CPU affinity of the tx queue (echo 2 > /proc/irq/xx/smp_affinity) will cause handle_edge_irq to loop for a long time in the do {} while() loop.
After setting the IRQ CPU affinity, the next interrupt will only be activated when it arrives. Therefore, the next interrupt will still be on CPU 0. When a new CPU affinity is activated on CPU 0, subsequent interrupts will be processed on CPU 1.
cpu 0 cpu 1
- handle_edge_irq
- apic_ack_irq
- irq_do_set_affinity
- handle_edge_irq
- do {
- handle_irq_event
- istate &= ~IRQS_PENDIN
- IRQD_IRQ_INPROGRESS
- spin_unlock()
- spin_lock()
- istate |= IRQS_PENDIN
- handle_irq_event_percpu - mask_ack_irq()
- spin_unlock()
- spin_unlock
} while(IRQS_PENDIN &&
!irq_disable)
Therefore, when determining whether to continue looping, we add a check to see if the current CPU belongs to the affinity table of the interrupt.
--- kernel/irq/chip.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/irq/chip.c +2 -1
@@ -831,7 +831,8 @@ void handle_edge_irq(struct irq_desc *desc)handle_irq_event (desc); }while ((desc->istate &IRQS_PENDING ) && - !irqd_irq_disabled (&desc->irq_data)); + !irqd_irq_disabled (&desc->irq_data) && +cpumask_test_cpu (smp_processor_id (),irq_data_get_affinity_mask (&desc->irq_data))); out_unlock:raw_spin_unlock (&desc->lock);
Signature · 1 line
2.32.1 (Apple Git-133)
kernel/irq/chip.c void handle_edge_irq(struct irq_desc *desc) · line 835
Quoted 11 lines · Wei Gong
kernel/irq/chip.c +2
-1@@ -831,7 +831,8 @@ void handle_edge_irq(struct irq_desc *desc)handle_irq_event (desc); }while ((desc->istate &IRQS_PENDING ) && - !irqd_irq_disabled (&desc->irq_data)); + !irqd_irq_disabled (&desc->irq_data) && +cpumask_test_cpu (smp_processor_id (),irq_data_get_affinity_mask (&desc->irq_data)));
Assume affinty mask has CPU0 and CPU1 set and the loop is on CPU0, but the effective affinity is on CPU1 then how is this going to move the interrupt?
Thanks,
tglx
Quoted 8 lines · Thomas Gleixner
Quoted 7 lines · Wei Gong
kernel/irq/chip.c +0
-0@@ -831,7 +831,8 @@ void handle_edge_irq(struct irq_desc *desc)handle_irq_event (desc);
Quoted 8 lines
kernel/irq/chip.c void handle_edge_irq(struct irq_desc *desc) · line 835
Quoted 4 lines
kernel/irq/chip.c +2
-1}while ((desc->istate &IRQS_PENDING ) && - !irqd_irq_disabled (&desc->irq_data)); + !irqd_irq_disabled (&desc->irq_data) && +cpumask_test_cpu (smp_processor_id (),irq_data_get_affinity_mask (&desc->irq_data)));Assume affinty mask has CPU0 and CPU1 set and the loop is on CPU0, but the effective affinity is on CPU1 then how is this going to move the interrupt?
Loop is on the CPU0 means that the previous effective affinity was on CPU0. When the previous effective affinity is a subset of the new affinity mask, the effective affinity will not be updated.
Therefore, I understand that the scenario you mentioned will not occur?
Quoted 4 lines
Thanks,
tglx
Thanks,
Wei Gong
Quoted 18 lines · Wei Gong
Quoted 8 lines · Thomas Gleixner
Quoted 7 lines · Wei Gong
kernel/irq/chip.c +0
-0@@ -831,7 +831,8 @@ void handle_edge_irq(struct irq_desc *desc)handle_irq_event (desc);Quoted 8 lines
kernel/irq/chip.c void handle_edge_irq(struct irq_desc *desc) · line 835
Quoted 4 lines
kernel/irq/chip.c +2
-1}while ((desc->istate &IRQS_PENDING ) && - !irqd_irq_disabled (&desc->irq_data)); + !irqd_irq_disabled (&desc->irq_data) && +cpumask_test_cpu (smp_processor_id (),irq_data_get_affinity_mask (&desc->irq_data)));Assume affinty mask has CPU0 and CPU1 set and the loop is on CPU0, but the effective affinity is on CPU1 then how is this going to move the interrupt?
Quoted 3 lines
Loop is on the CPU0 means that the previous effective affinity was on CPU0. When the previous effective affinity is a subset of the new affinity mask, the effective affinity will not be updated.
That's an implementation detail of a particular interrupt chip driver, but not a general guaranteed behaviour.
Quoted 10 lines · Thomas Gleixner
Quoted 9 lines · Wei Gong
Quoted 8 lines · Thomas Gleixner
Quoted 7 lines · Wei Gong
kernel/irq/chip.c +0
-0@@ -831,7 +831,8 @@ void handle_edge_irq(struct irq_desc *desc)handle_irq_event (desc);
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
kernel/irq/chip.c void handle_edge_irq(struct irq_desc *desc) · line 835
Quoted 4 lines
kernel/irq/chip.c +2
-1}while ((desc->istate &IRQS_PENDING ) && - !irqd_irq_disabled (&desc->irq_data)); + !irqd_irq_disabled (&desc->irq_data) && +cpumask_test_cpu (smp_processor_id (),irq_data_get_affinity_mask (&desc->irq_data)));Assume affinty mask has CPU0 and CPU1 set and the loop is on CPU0, but the effective affinity is on CPU1 then how is this going to move the interrupt?
Can replacing irq_data_get_affinity_mask with irq_data_get_effective_affinity_mask solve this issue?
Quoted 8 lines
Quoted 4 lines
Loop is on the CPU0 means that the previous effective affinity was on CPU0. When the previous effective affinity is a subset of the new affinity mask, the effective affinity will not be updated.
That's an implementation detail of a particular interrupt chip driver, but not a general guaranteed behaviour.
Quoted 24 lines · Wei Gong
Quoted 23 lines · Thomas Gleixner
Quoted 21 lines · Wei Gong
Quoted 8 lines · Thomas Gleixner
Quoted 7 lines · Wei Gong
kernel/irq/chip.c +0
-0@@ -831,7 +831,8 @@ void handle_edge_irq(struct irq_desc *desc)handle_irq_event (desc);Quoted 8 lines
kernel/irq/chip.c void handle_edge_irq(struct irq_desc *desc) · line 835
Quoted 4 lines
kernel/irq/chip.c +2
-1}while ((desc->istate &IRQS_PENDING ) && - !irqd_irq_disabled (&desc->irq_data)); + !irqd_irq_disabled (&desc->irq_data) && +cpumask_test_cpu (smp_processor_id (),irq_data_get_affinity_mask (&desc->irq_data)));Assume affinty mask has CPU0 and CPU1 set and the loop is on CPU0, but the effective affinity is on CPU1 then how is this going to move the interrupt?
Loop is on the CPU0 means that the previous effective affinity was on CPU0. When the previous effective affinity is a subset of the new affinity mask, the effective affinity will not be updated.
Quoted 6 lines
Quoted 3 lines
That's an implementation detail of a particular interrupt chip driver, but not a general guaranteed behaviour.
Can replacing
irq_data_get_affinity_maskwithirq_data_get_effective_affinity_masksolve this issue?
Yes.