ksoftirqd doesn't really become active
Subsystems (2)
While testing with arpflood, I stumbled across what looks like a general flaw in the softirq processing: softirqs were keeping one CPU busy; only 1% of this CPU capacity was left to other applications. But still ksoftirqd didn't get scheduled much, most of the time softirqs were processed in the context of irq_exit. Hence the scheduler didn't really influence the balancing between softirq processing and other tasks, as probably intended. This looks like a general flaw which should affect every platform in one way or the other.
Here's what seems to happen: an arp storm leads to the NAPI activating softirqs all the time. After 10 softirqs in a row the softirq processing is suspended and ksoftirqd is woken up. Now the scheduler influences the remaining time until the next interrupt, after which 10 softirqs are processed in the context of irq_exit again.
Let
s = average time spent on processing 10 softirqs; t = average time distance between hardirqs
If hardirqs come in regularly, then this CPU spends
ca. s/(t+t*int(s/t)) of its time on softirq processing in irq_exit context:
s
_________________/\________________
/ \
|=============|=============|=======>-----|
\_____________/
t
Once s approaches or exceeds t, the CPU is at least 50% busy processing softirqs, regardless of the scheduler's decisions.
What are useful ways to increase the scheduler's influence? One way might be to suspend softirq processing in irq_exit context as long as ksoftirqd feels responsible. I made a test with a quick hack, and this looks promising. Are there other suggestions? Or should the whole issue be dealt with in a completely different way?