limiting IRQ load, irq-rewrite-2.4.11-B5
Quoted 11 lines · jamal
Quoted 9 lines · Ingo Molnar
Quoted 4 lines
but the objectives, judging from the description you gave, are i think largely orthogonal, with some overlapping in the polling part.
yes. Weve done a lot of thoroughly thought work in that area and i think it will be a sin to throw it out.
Quoted 6 lines
I hit the send button to fast..
The dynamic irq limiting (it must not be set by a system admin to conserve the principle of work) could be used as a last resort. The point is, if you are not generating a lot of interupts to begin with (as is the case with NAPI), i dont see the irq rate limiting kicking in at all.
A few notes as seen for low-end nics:
Forcing an irq limit without asking the driver is bad - it must be the opposite way around.
e.g. the winbond nic contains a bug that forces it to 1 interrupt/packet tx, but I can switch to rx polling/mitigation.
I'm sure the ne2k-pci users would also complain if a fixed irq limit is added - I bet the majority of the drivers perform worse with a fixed limit, only some perform better, and most perform best if they are given a notice that they should reduce their irq rate. (e.g. disable rx_packet, tx_packet. Leave the error interrupts on, and do the rx_packet, tx_packet work in the poll handler)
But a hint for the driver ("now switch mitigation on/off") seems to be a good idea. And that hint should not be the return value of netif_rx - what if the driver is only sending packets?
What if it's not even a network driver?
NAPI seems to be very promising to fix the total system overload case (so many packets arrive that despite irq mitigation the system is still overloaded).
But the implementation of irq mitigation is driver specific, and a 10 millisecond stop is far too long.
--
Manfred
Quoted 18 lines · Manfred Spraul
Quoted 11 lines · jamal
Quoted 9 lines · Ingo Molnar
Quoted 4 lines
but the objectives, judging from the description you gave, are i think largely orthogonal, with some overlapping in the polling part.
yes. Weve done a lot of thoroughly thought work in that area and i think it will be a sin to throw it out.
Quoted 6 lines
I hit the send button to fast..
The dynamic irq limiting (it must not be set by a system admin to conserve the principle of work) could be used as a last resort. The point is, if you are not generating a lot of interupts to begin with (as is the case with NAPI), i dont see the irq rate limiting kicking in at all.
Quoted 6 lines
A few notes as seen for low-end nics:
Forcing an irq limit without asking the driver is bad - it must be the opposite way around.
e.g. the winbond nic contains a bug that forces it to 1 interrupt/packet tx, but I can switch to rx polling/mitigation.
Indeed this is a weird case that we have not encountered but it does make the point that the driver knows best what to do.
Quoted 7 lines
I'm sure the ne2k-pci users would also complain if a fixed irq limit is added - I bet the majority of the drivers perform worse with a fixed limit, only some perform better, and most perform best if they are given a notice that they should reduce their irq rate. (e.g. disable
rx_packet,tx_packet. Leave the error interrupts on, and do therx_packet,tx_packetwork in the poll handler)
agreed. The reaction should be left to the driver's policy.
Quoted 4 lines
But a hint for the driver ("now switch mitigation on/off") seems to be a good idea. And that hint should not be the return value of
netif_rx- what if the driver is only sending packets?
What if it's not even a network driver?
For 2.4, unfortunately there was no other way to pass that feedback without the driver sending a packet up the stack. Our system feedback probe is based on sampling the backlog queue.
Quoted 7 lines
NAPI seems to be very promising to fix the total system overload case (so many packets arrive that despite irq mitigation the system is still overloaded).
But the implementation of irq mitigation is driver specific, and a 10 millisecond stop is far too long.
violent agreement.
cheers,
jamal
Quoted 9 lines · jamal
Quoted 7 lines
NAPI seems to be very promising to fix the total system overload case (so many packets arrive that despite irq mitigation the system is still overloaded).
But the implementation of irq mitigation is driver specific, and a 10 millisecond stop is far too long.
violent agreement.
The Ingo's solution move the mitigation control into the kernel with the immediate advantage that it'll work right now with existing drivers. I think that the idea of kirqpoll is good but the long term solution should be the move of the mitigation knowledge inside the drivers that will register their own kirqpoll callbacks when they're going to mask irqs.
In this case the "intelligence" about irq rates is left in the place where there's more knowledge about the I/O traffic nature.
- Davide