Skip to contentSkip to conversation

sched: Split out kernel/sched/fair_balance.c, numa_balancing.c and syscalls.c, plus other updates

PATCH Applied Ingo Molnar 4 participants 15 messages Apr 7, 2024 – May 27, 2024 Scheduler

lore
Share & tools
lore
#0 lore raw

cover sched: Split out kernel/sched/fair_balance.c, numa_balancing.c and syscalls.c, plus other updates

kernel/sched/core.c and fair.c have become a hodgepodge of scheduler functionality that obscures core scheduling and fair-balancing class scheduling functionality quite a bit.

Improve it all a bit by splitting syscall.c out of core.c, and fair_balance.c and numa_balancing.c out of fair.c:

                                # of Lines of code
                                BEFORE => AFTER
                                ---------------
 kernel/sched/sched.h             3493 =>  3891
 kernel/sched/core.c             12067 => 10308
 kernel/sched/syscalls.c             - =>  1691
 kernel/sched/fair.c             13295 =>  5707
 kernel/sched/fair_balance.c         - =>  5103
 kernel/sched/numa_balancing.c       - =>  2277

total 49650 => 49765

This is only a first-approximation split.

Thanks,

Ingo

=================>

Ingo Molnar (5):
sched: Split out kernel/sched/syscalls.c from kernel/sched/core.c
sched: Split out kernel/sched/fair_balance.c from kernel/sched/fair.c
sched: Split out kernel/sched/numa_balancing.c from kernel/sched/fair.c sched/fair: Remove NEXT_BUDDY
sched/fair: Rename set_next_buddy() to set_next_pick()

 kernel/sched/Makefile         |     3 +
 kernel/sched/core.c           |  1951 +--------
 kernel/sched/fair.c           | 13184 +++++++++++++-----------------------------------------------
 kernel/sched/fair_balance.c   |  5103 +++++++++++++++++++++++
 kernel/sched/features.h       |     7 -
 kernel/sched/numa_balancing.c |  2277 +++++++++++
 kernel/sched/sched.h          |   400 +-
 kernel/sched/syscalls.c       |  1691 ++++++++
 8 files changed, 12367 insertions(+), 12249 deletions(-)
 create mode 100644 kernel/sched/fair_balance.c
 create mode 100644 kernel/sched/numa_balancing.c
 create mode 100644 kernel/sched/syscalls.c
Signature · 1 line
2.40.1
#1 lore raw

PATCH 1/5 sched: Split out kernel/sched/syscalls.c from kernel/sched/core.c

core.c has become rather large, move most scheduler syscall related functionality into a separate file, syscalls.c.

Move the alloc_user_cpus_ptr(), __rt_effective_prio(), rt_effective_prio(), uclamp_none(), uclamp_se_set() and uclamp_bucket_id() inlines to kernel/sched/sched.h.

Internally export the __sched_setscheduler(), __sched_setaffinity(), __setscheduler_prio(), set_load_weight(), enqueue_task(), dequeue_task(), check_class_changed(), splice_balance_callbacks() and balance_callbacks() methods to better facilitate this.

---
 kernel/sched/Makefile   |    1 +
 kernel/sched/core.c     | 1950 ++++---------------------------------------------------------------
 kernel/sched/sched.h    |  106 +++-
 kernel/sched/syscalls.c | 1691 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 1892 insertions(+), 1856 deletions(-)
kernel/sched/Makefile +1 -0
index 976092b7bd45..c7afe445480a 100644
@@ -29,6 +29,7 @@ endif
 # build parallelizes well and finishes roughly at once:
 #
 obj-y += core.o
+obj-y += syscalls.o
 obj-y += fair.o
 obj-y += build_policy.o
 obj-y += build_utility.o
kernel/sched/core.c +95 -1855

2,115 lines not on this page · raw message

kernel/sched/sched.h +105 -1

136 lines not on this page · raw message

kernel/sched/syscalls.c new file +1691 -0

742 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.40.1

Part of the diff is not shown here. View the raw message.

#2 lore raw

Re: [PATCH 1/5] sched: Split out kernel/sched/syscalls.c from kernel/sched/core.c

kernel test robot noticed the following build warnings:

Hi Ingo,

kernel test robot noticed the following build warnings:

[auto build test WARNING on tip/sched/core]
[also build test WARNING on linux/master linus/master peterz-queue/sched/core v6.9-rc2 next-20240405] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Ingo-Molnar/sched-Split-out-kernel-sched-syscalls-c-from-kernel-sched-core-c/20240407-164646
base:   tip/sched/core
patch link:    https://lore.kernel.org/r/20240407084319.1462211-2-mingo%40kernel.org
patch subject: [PATCH 1/5] sched: Split out kernel/sched/syscalls.c from kernel/sched/core.c
config: i386-randconfig-061-20240408 (https://download.01.org/0day-ci/archive/20240408/[email protected]/config)
compiler: gcc-13 (Ubuntu 13.2.0-4ubuntu3) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240408/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)

>> kernel/sched/syscalls.c:197:22: sparse: sparse: incompatible types in comparison expression (different address spaces):

kernel/sched/syscalls.c:197:22: sparse: struct task_struct [noderef] __rcu *
kernel/sched/syscalls.c:197:22: sparse: struct task_struct *
kernel/sched/syscalls.c: note: in included file:
kernel/sched/sched.h:2172:25: sparse: sparse: incompatible types in comparison expression (different address spaces):
kernel/sched/sched.h:2172:25: sparse: struct task_struct [noderef] __rcu *
kernel/sched/sched.h:2172:25: sparse: struct task_struct *
kernel/sched/sched.h:2344:9: sparse: sparse: incompatible types in comparison expression (different address spaces):
kernel/sched/sched.h:2344:9: sparse: struct task_struct [noderef] __rcu *
kernel/sched/sched.h:2344:9: sparse: struct task_struct *
kernel/sched/sched.h:2172:25: sparse: sparse: incompatible types in comparison expression (different address spaces):
kernel/sched/sched.h:2172:25: sparse: struct task_struct [noderef] __rcu *
kernel/sched/sched.h:2172:25: sparse: struct task_struct *
kernel/sched/sched.h:2344:9: sparse: sparse: incompatible types in comparison expression (different address spaces):
kernel/sched/sched.h:2344:9: sparse: struct task_struct [noderef] __rcu *
kernel/sched/sched.h:2344:9: sparse: struct task_struct *
kernel/sched/syscalls.c:1228:5: sparse: sparse: context imbalance in 'dl_task_check_affinity' - wrong count at exit kernel/sched/syscalls.c:1331:9: sparse: sparse: context imbalance in 'sched_setaffinity' - different lock contexts for basic block kernel/sched/syscalls.c:1393:6: sparse: sparse: context imbalance in 'sched_getaffinity' - wrong count at exit

vim +197 kernel/sched/syscalls.c

186
187 /**
188 * idle_cpu - is a given CPU idle currently?
189 * @cpu: the processor in question.
190 *
191 * Return: 1 if the CPU is currently idle. 0 otherwise.
192 */
193 int idle_cpu(int cpu)
194 {
195 struct rq *rq = cpu_rq(cpu);
196

 > 197		if (rq->curr != rq->idle)

198 return 0;
199
200 if (rq->nr_running)
201 return 0;
202

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
merge bot Applied
#3 lore raw

[tip: sched/core] sched/syscalls: Split out kernel/sched/syscalls.c from kernel/sched/core.c

In reply to Ingo Molnar
The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     04746ed80bcf3130951ed4d5c1bc5b0bcabdde22
Gitweb:        https://git.kernel.org/tip/04746ed80bcf3130951ed4d5c1bc5b0bcabdde22
Author:        Ingo Molnar <[email protected]>
AuthorDate:    Sun, 07 Apr 2024 10:43:15 +02:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Mon, 27 May 2024 13:56:10 +02:00

sched/syscalls: Split out kernel/sched/syscalls.c from kernel/sched/core.c

core.c has become rather large, move most scheduler syscall related functionality into a separate file, syscalls.c.

This is about ~15% of core.c's raw linecount.

Move the alloc_user_cpus_ptr(), __rt_effective_prio(), rt_effective_prio(), uclamp_none(), uclamp_se_set() and uclamp_bucket_id() inlines to kernel/sched/sched.h.

Internally export the __sched_setscheduler(), __sched_setaffinity(), __setscheduler_prio(), set_load_weight(), enqueue_task(), dequeue_task(), check_class_changed(), splice_balance_callbacks() and balance_callbacks() methods to better facilitate this.

Move the new file's build to sched_policy.c, because it fits there semantically, but also because it's the smallest of the 4 build units under an allmodconfig build:

-rw-rw-r-- 1 mingo mingo 7.3M May 27 12:35 kernel/sched/core.i -rw-rw-r-- 1 mingo mingo 6.4M May 27 12:36 kernel/sched/build_utility.i -rw-rw-r-- 1 mingo mingo 6.3M May 27 12:36 kernel/sched/fair.i -rw-rw-r-- 1 mingo mingo 5.8M May 27 12:36 kernel/sched/build_policy.i

This better balances build time for scheduler subsystem rebuilds.

I build-tested this new file as a standalone syscalls.o file for a bit, to make sure all the encapsulations & abstractions are robust.

Also update/add my copyright notices to these files.

Build time measurements:

# -Before/+After:

kepler:~/tip> perf stat -e 'cycles,instructions,duration_time' --sync --repeat 5 --pre 'rm -f kernel/sched/*.o' m kernel/sched/built-in.a >/dev/null

Performance counter stats for 'm kernel/sched/built-in.a' (5 runs):

 -    71,938,508,607      cycles                                                                  ( +-  0.17% )
 +    71,992,916,493      cycles                                                                  ( +-  0.22% )
 -   106,214,780,964      instructions                     #    1.48  insn per cycle              ( +-  0.01% )
 +   105,450,231,154      instructions                     #    1.46  insn per cycle              ( +-  0.01% )
 -     5,878,232,620 ns   duration_time                                                           ( +-  0.38% )
 +     5,290,085,069 ns   duration_time                                                           ( +-  0.21% )
 -            5.8782 +- 0.0221 seconds time elapsed  ( +-  0.38% )
 +            5.2901 +- 0.0111 seconds time elapsed  ( +-  0.21% )

Build time improvement of -11.1% (duration_time) is expected: the parallel build time of the scheduler subsystem is determined by the largest, slowest to build object file, which is kernel/sched/core.o. By moving ~15% of its complexity into another build unit, we reduced build time by -11%.

Measured cycles spent on building is within its ~0.2% stddev noise envelope.

The -0.7% reduction in instructions spent on building the scheduler is statistically reliable and somewhat surprising - I can only speculate:
maybe compilers aren't that efficient at building & optimizing 10+ KLOC files (core.c), and it's an overall win to balance the linecount a bit.

Anyway, this might be a data point that suggests that reducing the linecount of our largest files will improve not just code readability and maintainability, but might also improve build times a bit.

Code generation got a bit worse, by 0.5kb text on an x86 defconfig build:

# -Before/+After:

kepler:~/tip> size vmlinux
text data bss dec hex filename
-26475475 10439178 1740804 38655457 24dd5e1 vmlinux
+26476003 10439178 1740804 38655985 24dd7f1 vmlinux

kepler:~/tip> size kernel/sched/built-in.a

     text	   data	    bss	    dec	    hex	filename
  - 76056	  30025	    489	 106570	  1a04a	kernel/sched/core.o (ex kernel/sched/built-in.a)
  + 63452	  29453	    489	  93394	  16cd2	kernel/sched/core.o (ex kernel/sched/built-in.a)
    44299	   2181	    104	  46584	   b5f8	kernel/sched/fair.o (ex kernel/sched/built-in.a)
  - 42764	   3424	    120	  46308	   b4e4	kernel/sched/build_policy.o (ex kernel/sched/built-in.a)
  + 55651	   4044	    120	  59815	   e9a7	kernel/sched/build_policy.o (ex kernel/sched/built-in.a)
    44866	  12655	   2192	  59713	   e941	kernel/sched/build_utility.o (ex kernel/sched/built-in.a)
    44866	  12655	   2192	  59713	   e941	kernel/sched/build_utility.o (ex kernel/sched/built-in.a)

This is primarily due to the extra functions exported, and the size gets exaggerated somewhat by __pfx CFI function padding:

	ffffffff810cc710 <__pfx_enqueue_task>:
	ffffffff810cc710:	90                   	nop
	ffffffff810cc711:	90                   	nop
	ffffffff810cc712:	90                   	nop
	ffffffff810cc713:	90                   	nop
	ffffffff810cc714:	90                   	nop
	ffffffff810cc715:	90                   	nop
	ffffffff810cc716:	90                   	nop
	ffffffff810cc717:	90                   	nop
	ffffffff810cc718:	90                   	nop
	ffffffff810cc719:	90                   	nop
	ffffffff810cc71a:	90                   	nop
	ffffffff810cc71b:	90                   	nop
	ffffffff810cc71c:	90                   	nop
	ffffffff810cc71d:	90                   	nop
	ffffffff810cc71e:	90                   	nop
	ffffffff810cc71f:	90                   	nop

AFAICS the cost is primarily not to core.o and fair.o though (which contain most performance sensitive scheduler functions), only to syscalls.o that get called with much lower frequency - so I think this is an acceptable trade-off for better code separation.

---
 kernel/sched/build_policy.c |    1 +-
 kernel/sched/core.c         | 1953 +---------------------------------
 kernel/sched/sched.h        |  106 +-
 kernel/sched/syscalls.c     | 1699 ++++++++++++++++++++++++++++++-
 4 files changed, 1902 insertions(+), 1857 deletions(-)
 create mode 100644 kernel/sched/syscalls.c
kernel/sched/build_policy.c +1 -0
index d9dc9ab..39c3151 100644
@@ -52,3 +52,4 @@
 #include "cputime.c"
 #include "deadline.c"
 
+#include "syscalls.c"
kernel/sched/core.c +97 -1856

2,126 lines not on this page · raw message

kernel/sched/sched.h +105 -1

136 lines not on this page · raw message

kernel/sched/syscalls.c new file +1699 -0

734 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Part of the diff is not shown here. View the raw message.

#4 lore raw

PATCH 2/5 sched: Split out kernel/sched/fair_balance.c from kernel/sched/fair.c

In reply to Ingo Molnar

Move the SMP load-balancing code into the new fair_balance.c file, because it's mostly self-contained code that comprised about 50% of the lines of code in fair.c.

Expose the sched_balance_softirq(), sched_balance_find_dst_group(), cpu_load_without(), cpu_runnable_without(), cpu_util_without(), sched_balance_newidle(), task_h_load(), throttled_lb_pair(), task_util(), task_util_est() and a number of other methods internally to better facilitate this code separation.

---
 kernel/sched/Makefile       |     1 +
 kernel/sched/core.c         |     1 +
 kernel/sched/fair.c         | 10366 +++++++++++++++-----------------------------------------------
 kernel/sched/fair_balance.c |  5103 +++++++++++++++++++++++++++++++
 kernel/sched/sched.h        |   256 ++
 5 files changed, 7909 insertions(+), 7818 deletions(-)
kernel/sched/Makefile +1 -0
index c7afe445480a..898f6062a2a7 100644
@@ -31,5 +31,6 @@ endif
 obj-y += core.o
 obj-y += syscalls.o
 obj-y += fair.o
+obj-y += fair_balance.o
 obj-y += build_policy.o
 obj-y += build_utility.o
kernel/sched/core.c +1 -0
index 7fbb53d27229..013ce552941a 100644
@@ -8337,6 +8337,7 @@ void __init sched_init(void)
 	balance_push_set(smp_processor_id(), false);
 #endif
 	init_sched_fair_class();
+	init_sched_fair_class_balance();
 
 	psi_init();
 
kernel/sched/fair.c +2548 -7818

2,987 lines not on this page · raw message

kernel/sched/fair_balance.c new file +5103 -0

5,103 lines not on this page · raw message

kernel/sched/sched.h +256 -0

256 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.40.1

Part of the diff is not shown here. View the raw message.

#5 lore raw

Re: [PATCH 2/5] sched: Split out kernel/sched/fair_balance.c from kernel/sched/fair.c

kernel test robot noticed the following build errors:

Hi Ingo,

kernel test robot noticed the following build errors:

[auto build test ERROR on tip/sched/core]
[cannot apply to linux/master linus/master peterz-queue/sched/core v6.9-rc2] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Ingo-Molnar/sched-Split-out-kernel-sched-syscalls-c-from-kernel-sched-core-c/20240407-164646
base:   tip/sched/core
patch link:    https://lore.kernel.org/r/20240407084319.1462211-3-mingo%40kernel.org
patch subject: [PATCH 2/5] sched: Split out kernel/sched/fair_balance.c from kernel/sched/fair.c
config: arc-defconfig (https://download.01.org/0day-ci/archive/20240407/[email protected]/config)
compiler: arc-elf-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240407/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All error/warnings (new ones prefixed by >>):

In file included from kernel/sched/core.c:87:

>> kernel/sched/sched.h:3611:22: warning: 'task_h_load' defined but not used [-Wunused-function]

3611 | static unsigned long task_h_load(struct task_struct *p)
| ^~~~~~~~~~~
--
arc-elf-ld: kernel/sched/fair.o: in function `switched_from_fair':

>> fair.c:(.text+0x1f0): undefined reference to `update_tg_load_avg'
>> arc-elf-ld: fair.c:(.text+0x1f0): undefined reference to `update_tg_load_avg'
   arc-elf-ld: kernel/sched/fair.o: in function `rq_offline_fair':
>> fair.c:(.text+0x766): undefined reference to `clear_tg_offline_cfs_rqs'
>> arc-elf-ld: fair.c:(.text+0x766): undefined reference to `clear_tg_offline_cfs_rqs'
   arc-elf-ld: fair.c:(.text+0x774): undefined reference to `clear_tg_offline_cfs_rqs'
   arc-elf-ld: fair.c:(.text+0x774): undefined reference to `clear_tg_offline_cfs_rqs'
   arc-elf-ld: kernel/sched/fair.o: in function `switched_to_fair':
   fair.c:(.text+0x158a): undefined reference to `update_tg_load_avg'
   arc-elf-ld: fair.c:(.text+0x158a): undefined reference to `update_tg_load_avg'
   arc-elf-ld: kernel/sched/fair_balance.o: in function `update_load_avg':
>> fair_balance.c:(.text+0x12c8): undefined reference to `update_tg_load_avg'
>> arc-elf-ld: fair_balance.c:(.text+0x12c8): undefined reference to `update_tg_load_avg'
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#6 lore raw

Re: [PATCH 2/5] sched: Split out kernel/sched/fair_balance.c from kernel/sched/fair.c

In reply to Ingo Molnar
kernel test robot noticed the following build errors:

Hi Ingo,

kernel test robot noticed the following build errors:

[auto build test ERROR on tip/sched/core]
[also build test ERROR on next-20240405]
[cannot apply to linux/master linus/master peterz-queue/sched/core v6.9-rc2] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Ingo-Molnar/sched-Split-out-kernel-sched-syscalls-c-from-kernel-sched-core-c/20240407-164646
base:   tip/sched/core
patch link:    https://lore.kernel.org/r/20240407084319.1462211-3-mingo%40kernel.org
patch subject: [PATCH 2/5] sched: Split out kernel/sched/fair_balance.c from kernel/sched/fair.c
config: arm64-allnoconfig (https://download.01.org/0day-ci/archive/20240407/[email protected]/config)
compiler: aarch64-linux-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240407/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

aarch64-linux-ld: Unexpected GOT/PLT entries detected!
aarch64-linux-ld: Unexpected run-time procedure linkages detected!
aarch64-linux-ld: kernel/sched/fair.o: in function `switched_from_fair':
fair.c:(.text+0x16c): undefined reference to `update_tg_load_avg'
aarch64-linux-ld: kernel/sched/fair.o: in function `rq_offline_fair':
fair.c:(.text+0x230): undefined reference to `clear_tg_offline_cfs_rqs'

>> aarch64-linux-ld: fair.c:(.text+0x24c): undefined reference to `clear_tg_offline_cfs_rqs'
   aarch64-linux-ld: kernel/sched/fair.o: in function `switched_to_fair':
   fair.c:(.text+0x87c): undefined reference to `update_tg_load_avg'
   aarch64-linux-ld: kernel/sched/fair_balance.o: in function `update_load_avg':
   fair_balance.c:(.text+0x1520): undefined reference to `update_tg_load_avg'
>> aarch64-linux-ld: fair_balance.c:(.text+0x15ec): undefined reference to `update_tg_load_avg'

aarch64-linux-ld: fair_balance.c:(.text+0x1608): undefined reference to `update_tg_load_avg'

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#7 lore raw

Re: [PATCH 2/5] sched: Split out kernel/sched/fair_balance.c from kernel/sched/fair.c

In reply to Ingo Molnar
kernel test robot noticed the following build warnings:

Hi Ingo,

kernel test robot noticed the following build warnings:

[auto build test WARNING on tip/sched/core]
[also build test WARNING on next-20240405]
[cannot apply to linux/master linus/master peterz-queue/sched/core v6.9-rc2] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Ingo-Molnar/sched-Split-out-kernel-sched-syscalls-c-from-kernel-sched-core-c/20240407-164646
base:   tip/sched/core
patch link:    https://lore.kernel.org/r/20240407084319.1462211-3-mingo%40kernel.org
patch subject: [PATCH 2/5] sched: Split out kernel/sched/fair_balance.c from kernel/sched/fair.c
config: i386-randconfig-061-20240408 (https://download.01.org/0day-ci/archive/20240408/[email protected]/config)
compiler: gcc-13 (Ubuntu 13.2.0-4ubuntu3) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240408/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)

>> kernel/sched/fair_balance.c:2171:40: sparse: sparse: incorrect type in initializer (different address spaces) @@     expected struct sched_domain *child @@     got struct sched_domain [noderef] __rcu *child @@
   kernel/sched/fair_balance.c:2171:40: sparse:     expected struct sched_domain *child
   kernel/sched/fair_balance.c:2171:40: sparse:     got struct sched_domain [noderef] __rcu *child
>> kernel/sched/fair_balance.c:2793:22: sparse: sparse: incompatible types in comparison expression (different address spaces):
   kernel/sched/fair_balance.c:2793:22: sparse:    struct task_struct [noderef] __rcu *
   kernel/sched/fair_balance.c:2793:22: sparse:    struct task_struct *
>> kernel/sched/fair_balance.c:4218:9: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *[assigned] sd @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/fair_balance.c:4218:9: sparse:     expected struct sched_domain *[assigned] sd
   kernel/sched/fair_balance.c:4218:9: sparse:     got struct sched_domain [noderef] __rcu *parent
>> kernel/sched/fair_balance.c:3866:44: sparse: sparse: incorrect type in initializer (different address spaces) @@     expected struct sched_domain *sd_parent @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/fair_balance.c:3866:44: sparse:     expected struct sched_domain *sd_parent
   kernel/sched/fair_balance.c:3866:44: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/fair_balance.c:4327:9: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *[assigned] sd @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/fair_balance.c:4327:9: sparse:     expected struct sched_domain *[assigned] sd
   kernel/sched/fair_balance.c:4327:9: sparse:     got struct sched_domain [noderef] __rcu *parent
   kernel/sched/fair_balance.c:4972:9: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct sched_domain *[assigned] sd @@     got struct sched_domain [noderef] __rcu *parent @@
   kernel/sched/fair_balance.c:4972:9: sparse:     expected struct sched_domain *[assigned] sd
   kernel/sched/fair_balance.c:4972:9: sparse:     got struct sched_domain [noderef] __rcu *parent
>> kernel/sched/fair_balance.c:1921:24: sparse: sparse: dereference of noderef expression

kernel/sched/fair_balance.c:4050:64: sparse: sparse: dereference of noderef expression kernel/sched/fair_balance.c:4050:64: sparse: sparse: dereference of noderef expression

vim +2171 kernel/sched/fair_balance.c

2168
2169 void update_group_capacity(struct sched_domain *sd, int cpu)
2170 {

> 2171		struct sched_domain *child = sd->child;

2172 struct sched_group *group, *sdg = sd->groups;
2173 unsigned long capacity, min_capacity, max_capacity;
2174 unsigned long interval;
2175
2176 interval = msecs_to_jiffies(sd->balance_interval);
2177 interval = clamp(interval, 1UL, max_load_balance_interval);
2178 sdg->sgc->next_update = jiffies + interval;
2179
2180 if (!child) {
2181 update_cpu_capacity(sd, cpu);
2182 return;
2183 }
2184
2185 capacity = 0;
2186 min_capacity = ULONG_MAX;
2187 max_capacity = 0;
2188
2189 if (child->flags & SD_OVERLAP) {
2190 /*
2191 * SD_OVERLAP domains cannot assume that child groups
2192 * span the current group.
2193 */
2194
2195 for_each_cpu(cpu, sched_group_span(sdg)) {
2196 unsigned long cpu_cap = capacity_of(cpu);
2197
2198 capacity += cpu_cap;
2199 min_capacity = min(cpu_cap, min_capacity);
2200 max_capacity = max(cpu_cap, max_capacity);
2201 }
2202 } else {
2203 /*
2204 * !SD_OVERLAP domains can assume that child groups
2205 * span the current group.
2206 */
2207
2208 group = child->groups;
2209 do {
2210 struct sched_group_capacity *sgc = group->sgc;
2211
2212 capacity += sgc->capacity;
2213 min_capacity = min(sgc->min_capacity, min_capacity);
2214 max_capacity = max(sgc->max_capacity, max_capacity);
2215 group = group->next;
2216 } while (group != child->groups);
2217 }
2218
2219 sdg->sgc->capacity = capacity;
2220 sdg->sgc->min_capacity = min_capacity;
2221 sdg->sgc->max_capacity = max_capacity;
2222 }
2223

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#8 lore raw

PATCH 3/5 sched: Split out kernel/sched/numa_balancing.c from kernel/sched/fair.c

In reply to Ingo Molnar

Much of the NUMA balancing code already lives in a single #ifdef block - move it over into its own file: kernel/sched/numa_balancing.c.

Expose a handful of methods internally to facilitate this.

This further shrinks the rather large kernel/sched/fair.c file.

---
 kernel/sched/Makefile         |    1 +
 kernel/sched/fair.c           | 2307 +------------------------------------------------------------
 kernel/sched/numa_balancing.c | 2277 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 kernel/sched/sched.h          |   40 +-
 4 files changed, 2318 insertions(+), 2307 deletions(-)
kernel/sched/Makefile +1 -0
index 898f6062a2a7..45ab29e60fc7 100644
@@ -32,5 +32,6 @@ obj-y += core.o
 obj-y += syscalls.o
 obj-y += fair.o
 obj-y += fair_balance.o
+obj-y += numa_balancing.o
 obj-y += build_policy.o
 obj-y += build_utility.o
kernel/sched/fair.c +1 -2306

2,337 lines not on this page · raw message

kernel/sched/numa_balancing.c new file +2277 -0

657 lines not on this page · raw message

kernel/sched/sched.h +39 -1
index 7f1d856fdc3b..d687b9a272fc 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.40.1

Part of the diff is not shown here. View the raw message.

#9 lore raw

Re: [PATCH 3/5] sched: Split out kernel/sched/numa_balancing.c from kernel/sched/fair.c

kernel test robot noticed the following build warnings:

Hi Ingo,

kernel test robot noticed the following build warnings:

[auto build test WARNING on tip/sched/core]
[also build test WARNING on next-20240405]
[cannot apply to linux/master linus/master peterz-queue/sched/core v6.9-rc2] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Ingo-Molnar/sched-Split-out-kernel-sched-syscalls-c-from-kernel-sched-core-c/20240407-164646
base:   tip/sched/core
patch link:    https://lore.kernel.org/r/20240407084319.1462211-4-mingo%40kernel.org
patch subject: [PATCH 3/5] sched: Split out kernel/sched/numa_balancing.c from kernel/sched/fair.c
config: x86_64-randconfig-121-20240408 (https://download.01.org/0day-ci/archive/20240408/[email protected]/config)
compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240408/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)

>> kernel/sched/numa_balancing.c:1654:13: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct task_struct *tsk @@     got struct task_struct [noderef] __rcu * @@

kernel/sched/numa_balancing.c:1654:13: sparse: expected struct task_struct *tsk
kernel/sched/numa_balancing.c:1654:13: sparse: got struct task_struct [noderef] __rcu *
kernel/sched/numa_balancing.c: note: in included file (through include/linux/mmzone.h, include/linux/memory-tiers.h):
include/linux/page-flags.h:242:46: sparse: sparse: self-comparison always evaluates to false include/linux/page-flags.h:242:46: sparse: sparse: self-comparison always evaluates to false kernel/sched/numa_balancing.c:1600:9: sparse: sparse: context imbalance in 'task_numa_placement' - different lock contexts for basic block

vim +1654 kernel/sched/numa_balancing.c

1619
1620 static void task_numa_group(struct task_struct *p, int cpupid, int flags,
1621 int *priv)
1622 {
1623 struct numa_group *grp, *my_grp;
1624 struct task_struct *tsk;
1625 bool join = false;
1626 int cpu = cpupid_to_cpu(cpupid);
1627 int i;
1628
1629 if (unlikely(!deref_curr_numa_group(p))) {
1630 unsigned int size = sizeof(struct numa_group) +
1631 NR_NUMA_HINT_FAULT_STATS *
1632 nr_node_ids * sizeof(unsigned long);
1633
1634 grp = kzalloc(size, GFP_KERNEL | __GFP_NOWARN);
1635 if (!grp)
1636 return;
1637
1638 refcount_set(&grp->refcount, 1);
1639 grp->active_nodes = 1;
1640 grp->max_faults_cpu = 0;
1641 spin_lock_init(&grp->lock);
1642 grp->gid = p->pid;
1643
1644 for (i = 0; i < NR_NUMA_HINT_FAULT_STATS * nr_node_ids; i++)
1645 grp->faults[i] = p->numa_faults[i];
1646
1647 grp->total_faults = p->total_numa_faults;
1648
1649 grp->nr_tasks++;
1650 rcu_assign_pointer(p->numa_group, grp);
1651 }
1652
1653 rcu_read_lock();

> 1654		tsk = READ_ONCE(cpu_rq(cpu)->curr);

1655
1656 if (!cpupid_match_pid(tsk, cpupid))
1657 goto no_join;
1658
1659 grp = rcu_dereference(tsk->numa_group);
1660 if (!grp)
1661 goto no_join;
1662
1663 my_grp = deref_curr_numa_group(p);
1664 if (grp == my_grp)
1665 goto no_join;
1666
1667 /*
1668 * Only join the other group if its bigger; if we're the bigger group,
1669 * the other task will join us.
1670 */
1671 if (my_grp->nr_tasks > grp->nr_tasks)
1672 goto no_join;
1673
1674 /*
1675 * Tie-break on the grp address.
1676 */
1677 if (my_grp->nr_tasks == grp->nr_tasks && my_grp > grp)
1678 goto no_join;
1679
1680 /* Always join threads in the same process. */
1681 if (tsk->mm == current->mm)
1682 join = true;
1683
1684 /* Simple filter to avoid false positives due to PID collisions */
1685 if (flags & TNF_SHARED)
1686 join = true;
1687
1688 /* Update priv based on whether false sharing was detected */
1689 *priv = !join;
1690
1691 if (join && !get_numa_group(grp))
1692 goto no_join;
1693
1694 rcu_read_unlock();
1695
1696 if (!join)
1697 return;
1698
1699 WARN_ON_ONCE(irqs_disabled());
1700 double_lock_irq(&my_grp->lock, &grp->lock);
1701
1702 for (i = 0; i < NR_NUMA_HINT_FAULT_STATS * nr_node_ids; i++) {
1703 my_grp->faults[i] -= p->numa_faults[i];
1704 grp->faults[i] += p->numa_faults[i];
1705 }
1706 my_grp->total_faults -= p->total_numa_faults;
1707 grp->total_faults += p->total_numa_faults;
1708
1709 my_grp->nr_tasks--;
1710 grp->nr_tasks++;
1711
1712 spin_unlock(&my_grp->lock);
1713 spin_unlock_irq(&grp->lock);
1714
1715 rcu_assign_pointer(p->numa_group, grp);
1716
1717 put_numa_group(my_grp);
1718 return;
1719
1720 no_join:
1721 rcu_read_unlock();
1722 return;
1723 }
1724

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#10 lore raw

PATCH 4/5 sched/fair: Remove NEXT_BUDDY

In reply to Ingo Molnar

It has been turned off in 2009 and hasn't been enabled since, 15 years ought to be enough to remove it.

---
 kernel/sched/fair.c     | 11 -----------
 kernel/sched/features.h |  7 -------
 2 files changed, 18 deletions(-)
kernel/sched/fair.c +0 -11
index 0197ba78b89c..93ea653065f5 100644
@@ -1901,13 +1901,6 @@ set_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *se)
 static struct sched_entity *
 pick_next_entity(struct cfs_rq *cfs_rq)
 {
-	/*
-	 * Enabling NEXT_BUDDY will affect latency but not fairness.
-	 */
-	if (sched_feat(NEXT_BUDDY) &&
-	    cfs_rq->next && entity_eligible(cfs_rq, cfs_rq->next))
-		return cfs_rq->next;
-
 	return pick_eevdf(cfs_rq);
 }
 
@@ -4671,10 +4664,6 @@ static void check_preempt_wakeup_fair(struct rq *rq, struct task_struct *p, int
 	if (unlikely(throttled_hierarchy(cfs_rq_of(pse))))
 		return;
 
-	if (sched_feat(NEXT_BUDDY) && !(wake_flags & WF_FORK)) {
-		set_next_buddy(pse);
-	}
-
 	/*
 	 * We can come here with TIF_NEED_RESCHED already set from new task
 	 * wake up path.
kernel/sched/features.h +0 -7
index 143f55df890b..f0df03fe24d8 100644
@@ -8,13 +8,6 @@ SCHED_FEAT(PLACE_LAG, true)
 SCHED_FEAT(PLACE_DEADLINE_INITIAL, true)
 SCHED_FEAT(RUN_TO_PARITY, true)
 
-/*
- * Prefer to schedule the task we woke last (assuming it failed
- * wakeup-preemption), since its likely going to consume data we
- * touched, increases cache locality.
- */
-SCHED_FEAT(NEXT_BUDDY, false)
-
 /*
  * Consider buddies to be cache hot, decreases the likeliness of a
  * cache buddy being migrated away, increases cache locality.
Signature · 1 line
2.40.1
#11 lore raw

PATCH 5/5 sched/fair: Rename set_next_buddy() to set_next_pick()

In reply to Ingo Molnar

This is a mechanism to set the next task_pick target, 'buddy' is too ambiguous and refers to a historic feature we don't have anymore.

---
 kernel/sched/fair.c | 28 +++++++++++++---------------
 1 file changed, 13 insertions(+), 15 deletions(-)
kernel/sched/fair.c +13 -15
index 93ea653065f5..fe730f232ffd 100644
@@ -3200,7 +3200,16 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 	hrtick_update(rq);
 }
 
-static void set_next_buddy(struct sched_entity *se);
+static void set_next_pick(struct sched_entity *se)
+{
+	for_each_sched_entity(se) {
+		if (SCHED_WARN_ON(!se->on_rq))
+			return;
+		if (se_is_idle(se))
+			return;
+		cfs_rq_of(se)->next = se;
+	}
+}
 
 /*
  * The dequeue_task method is called before nr_running is
@@ -3240,7 +3249,7 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 			 * p is sleeping when it is within its sched_slice.
 			 */
 			if (task_sleep && se && !throttled_hierarchy(cfs_rq))
-				set_next_buddy(se);
+				set_next_pick(se);
 			break;
 		}
 		flags |= DEQUEUE_SLEEP;
@@ -4631,17 +4640,6 @@ balance_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
 static inline void set_task_max_allowed_capacity(struct task_struct *p) {}
 #endif /* CONFIG_SMP */
 
-static void set_next_buddy(struct sched_entity *se)
-{
-	for_each_sched_entity(se) {
-		if (SCHED_WARN_ON(!se->on_rq))
-			return;
-		if (se_is_idle(se))
-			return;
-		cfs_rq_of(se)->next = se;
-	}
-}
-
 /*
  * Preempt the current task with a newly woken task if needed:
  */
@@ -4769,7 +4767,7 @@ pick_next_task_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf
 		goto simple;
 
 	/*
-	 * Because of the set_next_buddy() in dequeue_task_fair() it is rather
+	 * Because of the set_next_pick() in dequeue_task_fair() it is rather
 	 * likely that a next task is from the same cgroup as the current.
 	 *
 	 * Therefore attempt to avoid putting and setting the entire cgroup
@@ -4957,7 +4955,7 @@ static bool yield_to_task_fair(struct rq *rq, struct task_struct *p)
 		return false;
 
 	/* Tell the scheduler that we'd really like se to run next. */
-	set_next_buddy(se);
+	set_next_pick(se);
 
 	yield_task_fair(rq);
 
Signature · 1 line
2.40.1
maintainer
#12 lore raw

Re: [PATCH 5/5] sched/fair: Rename set_next_buddy() to set_next_pick()

Quoted 45 lines · Ingo Molnar

This is a mechanism to set the next task_pick target, 'buddy' is too ambiguous and refers to a historic feature we don't have anymore.

---
 kernel/sched/fair.c | 28 +++++++++++++---------------
 1 file changed, 13 insertions(+), 15 deletions(-)
kernel/sched/fair.c +11 -3
index 93ea653065f5..fe730f232ffd 100644
@@ -3200,7 +3200,16 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 	hrtick_update(rq);
 }
 
-static void set_next_buddy(struct sched_entity *se);
+static void set_next_pick(struct sched_entity *se)
+{
+	for_each_sched_entity(se) {
+		if (SCHED_WARN_ON(!se->on_rq))
+			return;
+		if (se_is_idle(se))
+			return;
+		cfs_rq_of(se)->next = se;
+	}
+}
 
 /*
  * The dequeue_task method is called before nr_running is
@@ -3240,7 +3249,7 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 			 * p is sleeping when it is within its sched_slice.
 			 */
 			if (task_sleep && se && !throttled_hierarchy(cfs_rq))
-				set_next_buddy(se);
+				set_next_pick(se);
 			break;
 		}
 		flags |= DEQUEUE_SLEEP;
@@ -4631,17 +4640,6 @@ balance_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
 static inline void set_task_max_allowed_capacity(struct task_struct *p) {}
 #endif /* CONFIG_SMP */
 
-static void set_next_buddy(struct sched_entity *se)

kernel/sched/fair.c balance_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) · line 4644

Quoted 10 lines
kernel/sched/fair.c +0 -10
-{
-	for_each_sched_entity(se) {
-		if (SCHED_WARN_ON(!se->on_rq))
-			return;
-		if (se_is_idle(se))
-			return;
-		cfs_rq_of(se)->next = se;
-	}
-}
-

Hurmmm.. afaict the only actual user of cfs_rq->next left is task_hot(), no? Is that thing worth it?

That is, should we not totally nuke the thing?

kernel/sched/fair.c pick_next_task_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf · line 4772

Quoted 11 lines
kernel/sched/fair.c +1 -1
 /*
  * Preempt the current task with a newly woken task if needed:
  */
@@ -4769,7 +4767,7 @@ pick_next_task_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf
 		goto simple;
 
 	/*
-	 * Because of the set_next_buddy() in dequeue_task_fair() it is rather
+	 * Because of the set_next_pick() in dequeue_task_fair() it is rather
 	 * likely that a next task is from the same cgroup as the current.
 	 *

So, given you killed the ->next consideration in pick, isn't this comment 'misleading' at best?

#13 lore raw

Re: [PATCH 5/5] sched/fair: Rename set_next_buddy() to set_next_pick()

Quoted 45 lines · Peter Zijlstra
Quoted 44 lines · Ingo Molnar

This is a mechanism to set the next task_pick target, 'buddy' is too ambiguous and refers to a historic feature we don't have anymore.

---
 kernel/sched/fair.c | 28 +++++++++++++---------------
 1 file changed, 13 insertions(+), 15 deletions(-)
kernel/sched/fair.c +11 -2
index 93ea653065f5..fe730f232ffd 100644
@@ -3200,7 +3200,16 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 	hrtick_update(rq);
 }
 
-static void set_next_buddy(struct sched_entity *se);
+static void set_next_pick(struct sched_entity *se)
+{
+	for_each_sched_entity(se) {
+		if (SCHED_WARN_ON(!se->on_rq))
+			return;
+		if (se_is_idle(se))
+			return;
+		cfs_rq_of(se)->next = se;
+	}
+}
 
 /*
  * The dequeue_task method is called before nr_running is
@@ -3240,7 +3249,7 @@ static void dequeue_task_fair(struct rq *rq, struct task_struct *p, int flags)
 			 * p is sleeping when it is within its sched_slice.
 			 */
 			if (task_sleep && se && !throttled_hierarchy(cfs_rq))
-				set_next_buddy(se);
+				set_next_pick(se);
 			break;
 		}
 		flags |= DEQUEUE_SLEEP;
@@ -4631,17 +4640,6 @@ balance_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf)
 static inline void set_task_max_allowed_capacity(struct task_struct *p) {}
 #endif /* CONFIG_SMP */
 
Quoted 14 lines

kernel/sched/fair.c balance_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) · line 4644

Quoted 11 lines
kernel/sched/fair.c +0 -11
-static void set_next_buddy(struct sched_entity *se)
-{
-	for_each_sched_entity(se) {
-		if (SCHED_WARN_ON(!se->on_rq))
-			return;
-		if (se_is_idle(se))
-			return;
-		cfs_rq_of(se)->next = se;
-	}
-}
-

Hurmmm.. afaict the only actual user of cfs_rq->next left is task_hot(), no? Is that thing worth it?

Yeah, so:

1)

While you are correct in the context of my patch, I think that might be a bug - the yield_to() methods are intending to use ->next:

        /* Tell the scheduler that we'd really like se to run next. */
        set_next_buddy(se);

        yield_task_fair(rq);

... and yield_to() would rather fundamentally rely on ->next overriding the next-task-pick selection, but it won't due to NEXT_BUDDY being false:

static struct sched_entity *
pick_next_entity(struct cfs_rq *cfs_rq)

{               
        /*      
         * Enabling NEXT_BUDDY will affect latency but not fairness.
         */
        if (sched_feat(NEXT_BUDDY) &&
            cfs_rq->next && entity_eligible(cfs_rq, cfs_rq->next))
                return cfs_rq->next;
Quoted 1 line

That is, should we not totally nuke the thing?

I don't think we want to nuke it - there's 3 users:

- yield()
- CFS bandwidth
- wakeup

I think the yield() and CFS bandwidth ones are genuine, but non-working due to NEXT_BUDDY at 0. Wakeup was the original intended NEXT_BUDDY logic, but it got turned off due to some performance or latency considerations that might or might not be valid & relevant today.

2)

Even the task_hot() use of ->next isn't spurious: if a task has been marked as run-next, then presumably the current task is descheduling and we should probably not tear its ->next away in load-balancing.

3)

Side note: a set rq->next should probably reduce a candidate runqueue's weight both in periodic load-balancing and in idle-balancing, by rq->curr's weight or so?

So what I think we should do is to keep ->next and fix all its intended uses, and make it all unconditional by removing both NEXT_BUDDY and CACHE_HOT_BUDDY. I can cook up a series if you agree in principle.

Thanks,

Ingo

maintainer
#14 lore raw

Re: [PATCH 5/5] sched/fair: Rename set_next_buddy() to set_next_pick()

Quoted 19 lines · Ingo Molnar

I don't think we want to nuke it - there's 3 users:

- yield()
- CFS bandwidth
- wakeup

I think the yield() and CFS bandwidth ones are genuine, but non-working due to NEXT_BUDDY at 0. Wakeup was the original intended NEXT_BUDDY logic, but it got turned off due to some performance or latency considerations that might or might not be valid & relevant today.

2)

Even the task_hot() use of ->next isn't spurious: if a task has been marked as run-next, then presumably the current task is descheduling and we should probably not tear its ->next away in load-balancing.

3)

Quoted 7 lines

Side note: a set rq->next should probably reduce a candidate runqueue's weight both in periodic load-balancing and in idle-balancing, by rq->curr's weight or so?

So what I think we should do is to keep ->next and fix all its intended uses, and make it all unconditional by removing both NEXT_BUDDY and CACHE_HOT_BUDDY. I can cook up a series if you agree in principle.

So yes on fixing those yield_to() and cfs_bandwidth thingies, but put then under a new knob -- if the fix regresses we can simply flip it.

Ack on removing the current knobs, for them not having been changed in forever.