Revisit NUMA imbalance tolerance and fork balancing
Status from this message: The following commit has been merged into the sched/core branch of tip:
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v3 Latest4 patches 6 repliesApplied
- v24 patches 0 repliesSuperseded
- v1 RFC3 patches 11 repliesSuperseded
Changes in v3 from the cover letter
o Build fix for !NUMA_BALANCING configurations
Patches in v3 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/4 | sched/numa: Rename nr_running and break out the magic numberkernel/ | +6 | 1 reply | |
| 2/4 | sched: Avoid unnecessary calculation of load imbalance at clone timekernel/ | +5 | 1 reply | |
| 3/4 | sched/numa: Allow a floating imbalance between NUMA nodeskernel/ | +11 | 1 reply | |
| 4/4 | sched: Limit the amount of NUMA imbalance that can exist at fork timekernel/ | +15 | 1 reply |
Reviews trailers per patch
All 4 patches have a Reviewed-by or Acked-by.
| Patch | Vincent Guittot |
|---|---|
| 1/4 | R-b |
| 2/4 | R-b |
| 3/4 | R-b |
| 4/4 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v3 0/4] Revisit NUMA imbalance tolerance and fork balancing
Changelog since v2
o Build fix for !NUMA_BALANCING configurations
Changelog since v1
o Split out patch that moves imbalance calculation
o Strongly connect fork imbalance considerations with adjust_numa_imbalance
When NUMA and CPU balancing were reconciled, there was an attempt to allow a degree of imbalance but it caused more problems than it solved. Instead, imbalance was only allowed with an almost idle NUMA domain. A lot of the problems have since been addressed so it's time for a revisit. There is also an issue with how fork is balanced across threads. It's mentioned in this context as patch 3 and 4 should share similar behaviour in terms of a nodes utilisation.
Patch 1 is just a cosmetic rename
Patch 2 moves an imbalance calculation. It is both a micro-optimisation
and avoids confusing what imbalance means for different group types.
Patch 3 allows a "floating" imbalance to exist so communicating tasks can
remain on the same domain until utilisation is higher. It aims to balance compute availability with memory bandwidth.
Patch 4 is the interesting one. Currently fork can allow a NUMA node
to be completely utilised as long as there are idle CPUs until the load balancer gets involved. This caused serious problems with a real workload that unfortunately I cannot share many details about but there is a proxy reproducer.
Signature · 13 lines
2.26.2 Mel Gorman (4): sched/numa: Rename nr_running and break out the magic number sched: Avoid unnecessary calculation of load imbalance at clone time sched/numa: Allow a floating imbalance between NUMA nodes sched: Limit the amount of NUMA imbalance that can exist at fork time kernel/sched/fair.c | 44 +++++++++++++++++++++++++++++++------------- 1 file changed, 31 insertions(+), 13 deletions(-) -- 2.26.2