Skip to content

RFC Multi-thread per-cpu ksoftirqd

Dmitry Safonov lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest6 patches 3 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/6softirq: Add softirq_groups boot parameterDocumentation/admin-guide/kernel-parameters.txt include/linux/interrupt.h kernel/softirq.c+104 -00 replies
2/6softirq: Introduce mask for __do_softirq()include/linux/interrupt.h kernel/softirq.c+18 -170 replies
3/6softirq: Add reverse group-to-softirq mapkernel/softirq.c+18 -00 replies
4/6softirq: Run per-group per-cpu ksoftirqd threadinclude/linux/interrupt.h kernel/sched/cputime.c kernel/softirq.c +1 more+177 -553 replies
5/6softirq: Add time accounting per-softirq typeinclude/linux/hardirq.h include/linux/vtime.h init/Kconfig +3 more+65 -150 replies
6/6softirq/sched: Account si cpu time to ksoftirqd(s)include/linux/interrupt.h kernel/sched/fair.c kernel/sched/sched.h +1 more+95 -80 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC 0/6] Multi-thread per-cpu ksoftirqd

Another attempt to solve softirq deferring problems. There are at least two problems, AFAIK:
o deferring one softirq to ksoftirqd results in latencies for other (different type) softirqs by the reason of ksoftirqd_running() decision for deferring/servicing.
o The logic in __do_softirq() that checks if (pending) after 2ms of processing doesn't work on some machines during i.e. UDP storm.

So, what's done here in attempt to improve this is:
- added boot param to separate softirqs in deffer-groups
- per each softirq-group there is a ksoftirqd (per-cpu also)

The last two patches might be just a brain fart as I tried to improve the metric on which the decision to defer is based.
I measure the time spent to serve each softirq and account that time to ksoftirqd thread of that softirq-group. After that the decision to serve/defer a softirq is based on the comparison:
(current->vruntime < ksoftirqd->vruntime)
Ugh, time measures and updating ksoftirqd cpu time each tick might be costly.. And it looks like it doesn't work as expected: a new task is being started with normalized vruntime (min_vruntime), which is lower than ksoftirqd's. And time spent on servicing softirqs are still bigger than any running task.
Anyway, sending this as RFC, may be some one will like the approach (or suggests some other ideas).

Cc Andrew Morton <[email protected]>
Cc David Miller <[email protected]>
Cc Eric Dumazet <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Hannes Frederic Sowa <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc "Levin, Alexander (Sasha Levin)" <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Mauro Carvalho Chehab <[email protected]>
Cc Mike Galbraith <[email protected]>
Cc Paolo Abeni <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Radu Rendec <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Stanislaw Gruszka <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Wanpeng Li <[email protected]>

Dmitry Safonov (6):
softirq: Add softirq_groups boot parameter
softirq: Introduce mask for __do_softirq()
softirq: Add reverse group-to-softirq map
softirq: Run per-group per-cpu ksoftirqd thread
softirq: Add time accounting per-softirq type softirq/sched: Account si cpu time to ksoftirqd(s)

 Documentation/admin-guide/kernel-parameters.txt |  16 ++
 include/linux/hardirq.h                         |   2 +-
 include/linux/interrupt.h                       |  26 +-
 include/linux/vtime.h                           |  10 +-
 init/Kconfig                                    |  10 +
 kernel/sched/cputime.c                          |  60 +++-
 kernel/sched/fair.c                             |  38 +++
 kernel/sched/sched.h                            |  20 ++
 kernel/softirq.c                                | 362 ++++++++++++++++++++----
 net/ipv4/tcp_output.c                           |   2 +-
 10 files changed, 464 insertions(+), 82 deletions(-)
Signature · 1 line
2.13.6