Skip to contentSkip to conversation

sched/headers: Optimize scheduler build time

PATCH In review Ingo Molnar 3 participants 18 messages Mar 15, 2022 – Aug 24, 2022

Subsystems (12)
x86 MM core arm64 Scheduler +8 more subsystems
lore
Share & tools
lore
#0 lore raw

cover sched/headers: Optimize scheduler build time

Optimize the kernel scheduler build time by organizing the code into larger build units. This cuts CPU usage by 60%.

These patches originate from the fast-headers tree.

The numbers:

Cumulative scheduler (kernel/sched/) build time speedup on a Linux distribution's config, which enables all scheduler features, compared to the vanilla kernel:

      _____________________________________________________________________________
     |
     |  Vanilla kernel (v5.13-rc7):
     |_____________________________________________________________________________
     |
     |  Performance counter stats for 'make -j96 kernel/sched/' (3 runs):
     |
     |   126,975,564,374      instructions              #    1.45  insn per cycle           ( +-  0.00% )
     |    87,637,847,671      cycles                    #    3.959 GHz                      ( +-  0.30% )
     |         22,136.96 msec cpu-clock                 #    7.499 CPUs utilized            ( +-  0.29% )
     |
     |            2.9520 +- 0.0169 seconds time elapsed  ( +-  0.57% )
     |_____________________________________________________________________________
     |
     |  Patched kernel:
     |_____________________________________________________________________________
     |
     | Performance counter stats for 'make -j96 kernel/sched/' (3 runs):
     |
     |    50,420,496,914      instructions              #    1.47  insn per cycle           ( +-  0.00% )
     |    34,234,322,038      cycles                    #    3.946 GHz                      ( +-  0.31% )
     |          8,675.81 msec cpu-clock                 #    3.053 CPUs utilized            ( +-  0.45% )
     |
     |            2.8420 +- 0.0181 seconds time elapsed  ( +-  0.64% )
     |_____________________________________________________________________________

    Summary:

      - CPU time used to build the scheduler dropped by -60.9%, a reduction
        from 22.1 clock-seconds to 8.7 clock-seconds.

      - Wall-clock time to build the scheduler dropped by -3.9%, a reduction
        from 2.95 seconds to 2.84 seconds.

Ingo Molnar (15):
sched/headers: Fix header to build standalone: <linux/sched_clock.h> sched/headers: Add header guard to kernel/sched/sched.h
sched/headers: Add header guard to kernel/sched/stats.h and kernel/sched/autogroup.h sched/headers: sched/clock: Mark all functions 'notrace', remove CC_FLAGS_FTRACE build asymmetry sched/headers: Add initial new headers as identity mappings
sched/headers: Fix comment typo in kernel/sched/cpudeadline.c
sched/headers: Make the <linux/sched/deadline.h> header build standalone sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there sched/headers: Introduce kernel/sched/build_policy.c and build multiple .c files there sched/headers: Standardize kernel/sched/sched.h header dependencies sched/headers: Reorganize, clean up and optimize kernel/sched/core.c dependencies sched/headers: Reorganize, clean up and optimize kernel/sched/fair.c dependencies sched/headers: Reorganize, clean up and optimize kernel/sched/build_policy.c dependencies sched/headers: Reorganize, clean up and optimize kernel/sched/build_utility.c dependencies sched/headers: Reorganize, clean up and optimize kernel/sched/sched.h dependencies

 arch/arm64/include/asm/paravirt_api_clock.h |   1 +
 arch/x86/include/asm/paravirt_api_clock.h   |   1 +
 include/linux/cgroup_api.h                  |   1 +
 include/linux/cpumask_api.h                 |   1 +
 include/linux/fs_api.h                      |   1 +
 include/linux/gfp_api.h                     |   1 +
 include/linux/hashtable_api.h               |   1 +
 include/linux/hrtimer_api.h                 |   1 +
 include/linux/kobject_api.h                 |   1 +
 include/linux/kref_api.h                    |   1 +
 include/linux/ktime_api.h                   |   1 +
 include/linux/llist_api.h                   |   1 +
 include/linux/lockdep_api.h                 |   1 +
 include/linux/mm_api.h                      |   1 +
 include/linux/mutex_api.h                   |   1 +
 include/linux/perf_event_api.h              |   1 +
 include/linux/pgtable_api.h                 |   1 +
 include/linux/ptrace_api.h                  |   1 +
 include/linux/rcuwait_api.h                 |   1 +
 include/linux/refcount_api.h                |   1 +
 include/linux/sched/affinity.h              |   1 +
 include/linux/sched/cond_resched.h          |   1 +
 include/linux/sched/deadline.h              |   2 +
 include/linux/sched/posix-timers.h          |   1 +
 include/linux/sched/rseq_api.h              |   1 +
 include/linux/sched/task_flags.h            |   1 +
 include/linux/sched/thread_info_api.h       |   1 +
 include/linux/sched_clock.h                 |   2 +
 include/linux/seqlock_api.h                 |   1 +
 include/linux/softirq.h                     |   1 +
 include/linux/spinlock_api.h                |   1 +
 include/linux/swait_api.h                   |   1 +
 include/linux/syscalls_api.h                |   1 +
 include/linux/u64_stats_sync_api.h          |   1 +
 include/linux/wait_api.h                    |   1 +
 include/linux/workqueue_api.h               |   1 +
 kernel/sched/Makefile                       |  28 +++++-------
 kernel/sched/autogroup.c                    |   3 +-
 kernel/sched/autogroup.h                    |   5 +++
 kernel/sched/build_policy.c                 |  52 ++++++++++++++++++++++
 kernel/sched/build_utility.c                | 109 +++++++++++++++++++++++++++++++++++++++++++++
 kernel/sched/clock.c                        |  44 +++++++++----------
 kernel/sched/completion.c                   |   2 +-
 kernel/sched/core.c                         |  81 +++++++++++++++++++++++++++++-----
 kernel/sched/core_sched.c                   |   3 --
 kernel/sched/cpuacct.c                      |   3 +-
 kernel/sched/cpudeadline.c                  |   3 +-
 kernel/sched/cpufreq.c                      |   3 --
 kernel/sched/cpufreq_schedutil.c            |   7 ---
 kernel/sched/cpupri.c                       |   1 -
 kernel/sched/cputime.c                      |   1 -
 kernel/sched/deadline.c                     |   2 -
 kernel/sched/debug.c                        |   1 -
 kernel/sched/fair.c                         |  31 +++++++++++++
 kernel/sched/idle.c                         |   3 --
 kernel/sched/isolation.c                    |   1 -
 kernel/sched/loadavg.c                      |   1 -
 kernel/sched/membarrier.c                   |   1 -
 kernel/sched/pelt.c                         |   4 --
 kernel/sched/psi.c                          |  15 -------
 kernel/sched/rt.c                           |   3 --
 kernel/sched/sched.h                        | 114 ++++++++++++++++++++++++++----------------------
 kernel/sched/stats.c                        |   1 -
 kernel/sched/stats.h                        |   4 ++
 kernel/sched/stop_task.c                    |   1 -
 kernel/sched/swait.c                        |   1 -
 kernel/sched/topology.c                     |   1 -
 kernel/sched/wait.c                         |   1 -
 kernel/sched/wait_bit.c                     |   2 +-
 69 files changed, 407 insertions(+), 163 deletions(-)
 create mode 100644 arch/arm64/include/asm/paravirt_api_clock.h
 create mode 100644 arch/x86/include/asm/paravirt_api_clock.h
 create mode 100644 include/linux/cgroup_api.h
 create mode 100644 include/linux/cpumask_api.h
 create mode 100644 include/linux/fs_api.h
 create mode 100644 include/linux/gfp_api.h
 create mode 100644 include/linux/hashtable_api.h
 create mode 100644 include/linux/hrtimer_api.h
 create mode 100644 include/linux/kobject_api.h
 create mode 100644 include/linux/kref_api.h
 create mode 100644 include/linux/ktime_api.h
 create mode 100644 include/linux/llist_api.h
 create mode 100644 include/linux/lockdep_api.h
 create mode 100644 include/linux/mm_api.h
 create mode 100644 include/linux/mutex_api.h
 create mode 100644 include/linux/perf_event_api.h
 create mode 100644 include/linux/pgtable_api.h
 create mode 100644 include/linux/ptrace_api.h
 create mode 100644 include/linux/rcuwait_api.h
 create mode 100644 include/linux/refcount_api.h
 create mode 100644 include/linux/sched/affinity.h
 create mode 100644 include/linux/sched/cond_resched.h
 create mode 100644 include/linux/sched/posix-timers.h
 create mode 100644 include/linux/sched/rseq_api.h
 create mode 100644 include/linux/sched/task_flags.h
 create mode 100644 include/linux/sched/thread_info_api.h
 create mode 100644 include/linux/seqlock_api.h
 create mode 100644 include/linux/softirq.h
 create mode 100644 include/linux/spinlock_api.h
 create mode 100644 include/linux/swait_api.h
 create mode 100644 include/linux/syscalls_api.h
 create mode 100644 include/linux/u64_stats_sync_api.h
 create mode 100644 include/linux/wait_api.h
 create mode 100644 include/linux/workqueue_api.h
 create mode 100644 kernel/sched/build_policy.c
 create mode 100644 kernel/sched/build_utility.c
Signature · 1 line
2.32.0
#1 lore raw

PATCH 1/15 sched/headers: Fix header to build standalone: <linux/sched_clock.h>

Uses various kernel types that don't build standalone.

---
 include/linux/sched_clock.h | 2 ++
 1 file changed, 2 insertions(+)
include/linux/sched_clock.h +2 -0
index 835ee87ed792..cb41c5edb4d4 100644
@@ -5,6 +5,8 @@
 #ifndef LINUX_SCHED_CLOCK
 #define LINUX_SCHED_CLOCK
 
+#include <linux/types.h>
+
 #ifdef CONFIG_GENERIC_SCHED_CLOCK
 /**
  * struct clock_read_data - data required to read from sched_clock()
Signature · 1 line
2.32.0
#2 lore raw

PATCH 2/15 sched/headers: Add header guard to kernel/sched/sched.h

In reply to Ingo Molnar

Use the canonical header guard naming of the full path to the header.

---
 kernel/sched/sched.h | 4 ++++
 1 file changed, 4 insertions(+)
kernel/sched/sched.h +4 -0
index 3da5718cd641..eab4a18f71c2 100644
@@ -2,6 +2,9 @@
 /*
  * Scheduler internal types and methods:
  */
+#ifndef _KERNEL_SCHED_SCHED_H
+#define _KERNEL_SCHED_SCHED_H
+
 #include <linux/sched.h>
 
 #include <linux/sched/autogroup.h>
@@ -3137,3 +3140,4 @@ extern int sched_dynamic_mode(const char *str);
 extern void sched_dynamic_update(int mode);
 #endif
 
+#endif /* _KERNEL_SCHED_SCHED_H */
Signature · 1 line
2.32.0
#3 lore raw

PATCH 3/15 sched/headers: Add header guard to kernel/sched/stats.h and kernel/sched/autogroup.h

In reply to Ingo Molnar

Protect against multiple inclusion.

Also include "sched.h" in "stat.h", as it relies on it.

---
 kernel/sched/autogroup.h | 5 +++++
 kernel/sched/stats.h     | 6 ++++++
 2 files changed, 11 insertions(+)
kernel/sched/autogroup.h +5 -0
index 90fcbfdd70c3..90d69f2c5eaf 100644
@@ -1,4 +1,7 @@
 /* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _KERNEL_SCHED_AUTOGROUP_H
+#define _KERNEL_SCHED_AUTOGROUP_H
+
 #ifdef CONFIG_SCHED_AUTOGROUP
 
 struct autogroup {
@@ -59,3 +62,5 @@ static inline int autogroup_path(struct task_group *tg, char *buf, int buflen)
 }
 
 #endif /* CONFIG_SCHED_AUTOGROUP */
+
+#endif /* _KERNEL_SCHED_AUTOGROUP_H */
kernel/sched/stats.h +6 -0
index 3a3c826dd83a..edc0d13fc61b 100644
@@ -1,7 +1,11 @@
 /* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _KERNEL_STATS_H
+#define _KERNEL_STATS_H
 
 #ifdef CONFIG_SCHEDSTATS
 
+#include "sched.h"
+
 extern struct static_key_false sched_schedstats;
 
 /*
@@ -298,3 +302,5 @@ sched_info_switch(struct rq *rq, struct task_struct *prev, struct task_struct *n
 # define sched_info_dequeue(rq, t)	do { } while (0)
 # define sched_info_switch(rq, t, next)	do { } while (0)
 #endif /* CONFIG_SCHED_INFO */
+
+#endif /* _KERNEL_STATS_H */
Signature · 1 line
2.32.0
#5 lore raw

PATCH 5/15 sched/headers: Add initial new headers as identity mappings

In reply to Ingo Molnar

This allows code sharing between fast-headers tree and the vanilla scheduler tree.

---
 arch/arm64/include/asm/paravirt_api_clock.h | 1 +
 arch/x86/include/asm/paravirt_api_clock.h   | 1 +
 include/linux/cgroup_api.h                  | 1 +
 include/linux/cpumask_api.h                 | 1 +
 include/linux/fs_api.h                      | 1 +
 include/linux/gfp_api.h                     | 1 +
 include/linux/hashtable_api.h               | 1 +
 include/linux/hrtimer_api.h                 | 1 +
 include/linux/kobject_api.h                 | 1 +
 include/linux/kref_api.h                    | 1 +
 include/linux/ktime_api.h                   | 1 +
 include/linux/llist_api.h                   | 1 +
 include/linux/lockdep_api.h                 | 1 +
 include/linux/mm_api.h                      | 1 +
 include/linux/mutex_api.h                   | 1 +
 include/linux/perf_event_api.h              | 1 +
 include/linux/pgtable_api.h                 | 1 +
 include/linux/ptrace_api.h                  | 1 +
 include/linux/rcuwait_api.h                 | 1 +
 include/linux/refcount_api.h                | 1 +
 include/linux/sched/affinity.h              | 1 +
 include/linux/sched/cond_resched.h          | 1 +
 include/linux/sched/posix-timers.h          | 1 +
 include/linux/sched/rseq_api.h              | 1 +
 include/linux/sched/task_flags.h            | 1 +
 include/linux/sched/thread_info_api.h       | 1 +
 include/linux/seqlock_api.h                 | 1 +
 include/linux/softirq.h                     | 1 +
 include/linux/spinlock_api.h                | 1 +
 include/linux/swait_api.h                   | 1 +
 include/linux/syscalls_api.h                | 1 +
 include/linux/u64_stats_sync_api.h          | 1 +
 include/linux/wait_api.h                    | 1 +
 include/linux/workqueue_api.h               | 1 +
 34 files changed, 34 insertions(+)
arch/arm64/include/asm/paravirt_api_clock.h new file +1 -0
new file mode 100644
index 000000000000..65ac7cee0dad
@@ -0,0 +1 @@
+#include <asm/paravirt.h>
arch/x86/include/asm/paravirt_api_clock.h new file +1 -0
new file mode 100644
index 000000000000..65ac7cee0dad
@@ -0,0 +1 @@
+#include <asm/paravirt.h>
include/linux/cgroup_api.h new file +1 -0
new file mode 100644
index 000000000000..d0cfe8025111
@@ -0,0 +1 @@
+#include <linux/cgroup.h>
include/linux/cpumask_api.h new file +1 -0
new file mode 100644
index 000000000000..83bd3ebe82b0
@@ -0,0 +1 @@
+#include <linux/cpumask.h>
include/linux/fs_api.h new file +1 -0
new file mode 100644
index 000000000000..83be38d6d413
@@ -0,0 +1 @@
+#include <linux/fs.h>
include/linux/gfp_api.h new file +1 -0
new file mode 100644
index 000000000000..5a05a2764a86
@@ -0,0 +1 @@
+#include <linux/gfp.h>
include/linux/hashtable_api.h new file +1 -0
new file mode 100644
index 000000000000..c268ac2c5c0e
@@ -0,0 +1 @@
+#include <linux/hashtable.h>
include/linux/hrtimer_api.h new file +1 -0
new file mode 100644
index 000000000000..8d9700894468
@@ -0,0 +1 @@
+#include <linux/hrtimer.h>
include/linux/kobject_api.h new file +1 -0
new file mode 100644
index 000000000000..6e36a054c2d6
@@ -0,0 +1 @@
+#include <linux/kobject.h>
include/linux/kref_api.h new file +1 -0
new file mode 100644
index 000000000000..d67e554721d2
@@ -0,0 +1 @@
+#include <linux/kref.h>
include/linux/ktime_api.h new file +1 -0
new file mode 100644
index 000000000000..f697d493960f
@@ -0,0 +1 @@
+#include <linux/ktime.h>
include/linux/llist_api.h new file +1 -0
new file mode 100644
index 000000000000..625bec0393a1
@@ -0,0 +1 @@
+#include <linux/llist.h>
include/linux/lockdep_api.h new file +1 -0
new file mode 100644
index 000000000000..907e66979ab2
@@ -0,0 +1 @@
+#include <linux/lockdep.h>
include/linux/mm_api.h new file +1 -0
new file mode 100644
index 000000000000..a5ace2b198b8
@@ -0,0 +1 @@
+#include <linux/mm.h>
include/linux/mutex_api.h new file +1 -0
new file mode 100644
index 000000000000..85ab9491e13e
@@ -0,0 +1 @@
+#include <linux/mutex.h>
include/linux/perf_event_api.h new file +1 -0
new file mode 100644
index 000000000000..c2fd6048b790
@@ -0,0 +1 @@
+#include <linux/perf_event.h>
include/linux/pgtable_api.h new file +1 -0
new file mode 100644
index 000000000000..ff367a4ba8c4
@@ -0,0 +1 @@
+#include <linux/pgtable.h>
include/linux/ptrace_api.h new file +1 -0
new file mode 100644
index 000000000000..26e7d275ad8d
@@ -0,0 +1 @@
+#include <linux/ptrace.h>
include/linux/rcuwait_api.h new file +1 -0
new file mode 100644
index 000000000000..f962e28544dd
@@ -0,0 +1 @@
+#include <linux/rcuwait.h>
include/linux/refcount_api.h new file +1 -0
new file mode 100644
index 000000000000..5f032589f568
@@ -0,0 +1 @@
+#include <linux/refcount.h>
include/linux/sched/affinity.h new file +1 -0
new file mode 100644
index 000000000000..227f5be81bcd
@@ -0,0 +1 @@
+#include <linux/sched.h>
include/linux/sched/cond_resched.h new file +1 -0
new file mode 100644
index 000000000000..227f5be81bcd
@@ -0,0 +1 @@
+#include <linux/sched.h>
include/linux/sched/posix-timers.h new file +1 -0
new file mode 100644
index 000000000000..523a381d6c88
@@ -0,0 +1 @@
+#include <linux/posix-timers.h>
include/linux/sched/rseq_api.h new file +1 -0
new file mode 100644
index 000000000000..cf2af72693e1
@@ -0,0 +1 @@
+#include <linux/rseq.h>
include/linux/sched/task_flags.h new file +1 -0
new file mode 100644
index 000000000000..227f5be81bcd
@@ -0,0 +1 @@
+#include <linux/sched.h>
include/linux/sched/thread_info_api.h new file +1 -0
new file mode 100644
index 000000000000..2c60fbc16c08
@@ -0,0 +1 @@
+#include <linux/thread_info.h>
include/linux/seqlock_api.h new file +1 -0
new file mode 100644
index 000000000000..be91e7d3b826
@@ -0,0 +1 @@
+#include <linux/seqlock.h>
include/linux/softirq.h new file +1 -0
new file mode 100644
index 000000000000..c73d7dcb4cb5
@@ -0,0 +1 @@
+#include <linux/interrupt.h>
include/linux/spinlock_api.h new file +1 -0
new file mode 100644
index 000000000000..6338b27f98df
@@ -0,0 +1 @@
+#include <linux/spinlock.h>
include/linux/swait_api.h new file +1 -0
new file mode 100644
index 000000000000..1eeaaaaa5ea7
@@ -0,0 +1 @@
+#include <linux/swait.h>
include/linux/syscalls_api.h new file +1 -0
new file mode 100644
index 000000000000..23e012b04db4
@@ -0,0 +1 @@
+#include <linux/syscalls.h>
include/linux/u64_stats_sync_api.h new file +1 -0
new file mode 100644
index 000000000000..c72ca63da44b
@@ -0,0 +1 @@
+#include <linux/u64_stats_sync.h>
include/linux/wait_api.h new file +1 -0
new file mode 100644
index 000000000000..4e930548935a
@@ -0,0 +1 @@
+#include <linux/wait.h>
include/linux/workqueue_api.h new file +1 -0
new file mode 100644
index 000000000000..77debb5d2760
@@ -0,0 +1 @@
+#include <linux/workqueue.h>
Signature · 1 line
2.32.0
#6 lore raw

PATCH 6/15 sched/headers: Fix comment typo in kernel/sched/cpudeadline.c

In reply to Ingo Molnar

File name changed.

---
 kernel/sched/cpudeadline.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/cpudeadline.c +1 -1
index ceb03d76c0cc..0e196f0de492 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0-only
 /*
- *  kernel/sched/cpudl.c
+ *  kernel/sched/cpudeadline.c
  *
  *  Global CPU deadline management
  *
Signature · 1 line
2.32.0
#4 lore raw

PATCH 4/15 sched/headers: sched/clock: Mark all functions 'notrace', remove CC_FLAGS_FTRACE build asymmetry

In reply to Ingo Molnar

Mark all non-init functions in kernel/sched.c as 'notrace', instead of turning them all off via CC_FLAGS_FTRACE.

This is going to allow the treatment of this file as any other scheduler file, and it can be #include-ed in compound compilation units as well.

---
 kernel/sched/Makefile |  3 ---
 kernel/sched/clock.c  | 42 +++++++++++++++++++++---------------------
 2 files changed, 21 insertions(+), 24 deletions(-)
kernel/sched/Makefile +0 -3
index c83b37af155b..c0c52026ad0d 100644
@@ -1,7 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
-ifdef CONFIG_FUNCTION_TRACER
-CFLAGS_REMOVE_clock.o = $(CC_FLAGS_FTRACE)
-endif
 
 # The compilers are complaining about unused variables inside an if(0) scope
 # block. This is daft, shut them up.
kernel/sched/clock.c +21 -21

159 lines not on this page · raw message

Signature · 1 line
2.32.0
#7 lore raw

PATCH 7/15 sched/headers: Make the <linux/sched/deadline.h> header build standalone

In reply to Ingo Molnar

This header depends on various scheduler definitions.

---
 include/linux/sched/deadline.h | 2 ++
 1 file changed, 2 insertions(+)
include/linux/sched/deadline.h +2 -0
index 1aff00b65f3c..7c83d4d5a971 100644
@@ -6,6 +6,8 @@
  * NORMAL/BATCH tasks.
  */
 
+#include <linux/sched.h>
+
 #define MAX_DL_PRIO		0
 
 static inline int dl_prio(int prio)
Signature · 1 line
2.32.0
#12 lore raw

PATCH 10/15 sched/headers: Standardize kernel/sched/sched.h header dependencies

In reply to Ingo Molnar

kernel/sched/sched.h is a weird mix of ad-hoc headers included in the middle of the header.

Two of them rely on being included in the middle of kernel/sched/sched.h, due to definitions they require:

- "stat.h" needs the rq definitions.
- "autogroup.h" needs the task_group definition.

Move the inclusion of these two files out of kernel/sched/sched.h, and include them in all files that require them.

Move of the rest of the header dependencies to the top of the kernel/sched/sched.h file.

---
 kernel/sched/build_policy.c  |  3 +++
 kernel/sched/build_utility.c |  2 ++
 kernel/sched/core.c          |  2 ++
 kernel/sched/fair.c          |  2 ++
 kernel/sched/sched.h         | 20 +++++++++++---------
 kernel/sched/stats.h         |  4 ++--
 6 files changed, 22 insertions(+), 11 deletions(-)
kernel/sched/build_policy.c +3 -0
index 6bb384ddcfab..9a169b2f97e6 100644
@@ -13,6 +13,9 @@
  */
 
 #include "sched.h"
+
+#include "autogroup.h"
+#include "stats.h"
 #include "pelt.h"
 
 #include "idle.c"
kernel/sched/build_utility.c +2 -0
index 31216ce0b4b3..8b2b199983bf 100644
@@ -12,6 +12,8 @@
 
 #include "sched.h"
 #include "sched-pelt.h"
+#include "stats.h"
+#include "autogroup.h"
 
 #include <linux/sched_clock.h>
 
kernel/sched/core.c +2 -0
index ef946123e9af..22de53710ee9 100644
@@ -11,6 +11,8 @@
 #undef CREATE_TRACE_POINTS
 
 #include "sched.h"
+#include "stats.h"
+#include "autogroup.h"
 
 #include <linux/nospec.h>
 #include <linux/blkdev.h>
kernel/sched/fair.c +2 -0
index 16874e112fe6..7391c9287503 100644
@@ -21,6 +21,8 @@
  *  Copyright (C) 2007 Red Hat, Inc., Peter Zijlstra
  */
 #include "sched.h"
+#include "stats.h"
+#include "autogroup.h"
 
 /*
  * Targeted preemption latency for CPU-bound tasks:
kernel/sched/sched.h +11 -9
index f7d10b98e911..f255ec2afeca 100644
@@ -96,11 +96,20 @@
 # include <asm/paravirt.h>
 #endif
 
+#include <trace/events/sched.h>
+
+#ifdef CONFIG_CGROUP_SCHED
+#include <linux/cgroup.h>
+#include <linux/psi.h>
+#endif
+
+#ifdef CONFIG_SCHED_DEBUG
+# include <linux/static_key.h>
+#endif
+
 #include "cpupri.h"
 #include "cpudeadline.h"
 
-#include <trace/events/sched.h>
-
 #ifdef CONFIG_SCHED_DEBUG
 # define SCHED_WARN_ON(x)	WARN_ONCE(x, #x)
 #else
@@ -417,9 +426,6 @@ extern bool dl_cpu_busy(unsigned int cpu);
 
 #ifdef CONFIG_CGROUP_SCHED
 
-#include <linux/cgroup.h>
-#include <linux/psi.h>
-
 struct cfs_rq;
 struct rt_rq;
 
@@ -1919,9 +1925,6 @@ extern void flush_smp_call_function_from_idle(void);
 static inline void flush_smp_call_function_from_idle(void) { }
 #endif
 
-#include "stats.h"
-#include "autogroup.h"
-
 #if defined(CONFIG_SCHED_CORE) && defined(CONFIG_SCHEDSTATS)
 
 extern void __sched_core_account_forceidle(struct rq *rq);
@@ -2016,7 +2019,6 @@ static inline void __set_task_cpu(struct task_struct *p, unsigned int cpu)
  * Tunables that become constants when CONFIG_SCHED_DEBUG is off:
  */
 #ifdef CONFIG_SCHED_DEBUG
-# include <linux/static_key.h>
 # define const_debug __read_mostly
 #else
 # define const_debug const
kernel/sched/stats.h +2 -2
index edc0d13fc61b..a0df79e6038c 100644
@@ -2,10 +2,10 @@
 #ifndef _KERNEL_STATS_H
 #define _KERNEL_STATS_H
 
-#ifdef CONFIG_SCHEDSTATS
-
 #include "sched.h"
 
+#ifdef CONFIG_SCHEDSTATS
+
 extern struct static_key_false sched_schedstats;
 
 /*
Signature · 1 line
2.32.0
#9 lore raw

PATCH 9/15 sched/headers: Introduce kernel/sched/build_policy.c and build multiple .c files there

In reply to Ingo Molnar

Similarly to kernel/sched/build_utility.c, collect all 'scheduling policy' related source code files into kernel/sched/build_policy.c:

kernel/sched/idle.c

kernel/sched/rt.c

kernel/sched/cpudeadline.c
kernel/sched/pelt.c

kernel/sched/cputime.c
kernel/sched/deadline.c

With the exception of fair.c, which we continue to build as a separate file for build efficiency and parallelism reasons.

---
 kernel/sched/Makefile       |  2 +-
 kernel/sched/build_policy.c | 29 +++++++++++++++++++++++++++++
 kernel/sched/cpudeadline.c  |  1 -
 kernel/sched/cputime.c      |  1 -
 kernel/sched/deadline.c     |  2 --
 kernel/sched/idle.c         |  3 ---
 kernel/sched/pelt.c         |  4 ----
 kernel/sched/rt.c           |  3 ---
 kernel/sched/sched.h        |  8 ++++++++
 9 files changed, 38 insertions(+), 15 deletions(-)
kernel/sched/Makefile +1 -1
index 4a4785cb3cd2..976092b7bd45 100644
@@ -30,5 +30,5 @@ endif
 #
 obj-y += core.o
 obj-y += fair.o
+obj-y += build_policy.o
 obj-y += build_utility.o
-obj-y += idle.o rt.o deadline.o cputime.o cpudeadline.o pelt.o
kernel/sched/build_policy.c new file +29 -0
new file mode 100644
index 000000000000..6bb384ddcfab
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * These are the scheduling policy related scheduler files, built
+ * in a single compilation unit for build efficiency reasons.
+ *
+ * ( Incidentally, the size of the compilation unit is roughly
+ *   comparable to core.c and fair.c, the other two big
+ *   compilation units. This helps balance build time, while
+ *   coalescing source files to amortize header inclusion
+ *   cost. )
+ *
+ * core.c and fair.c are built separately.
+ */
+
+#include "sched.h"
+#include "pelt.h"
+
+#include "idle.c"
+
+#include "rt.c"
+
+#ifdef CONFIG_SMP
+# include "cpudeadline.c"
+# include "pelt.c"
+#endif
+
+#include "cputime.c"
+#include "deadline.c"
+
kernel/sched/cpudeadline.c +0 -1
index 0e196f0de492..02d970a879ed 100644
@@ -6,7 +6,6 @@
  *
  *  Author: Juri Lelli <[email protected]>
  */
-#include "sched.h"
 
 static inline int parent(int i)
 {
kernel/sched/cputime.c +0 -1
index b7ec42732b28..78a233d43757 100644
@@ -2,7 +2,6 @@
 /*
  * Simple CPU accounting cgroup controller
  */
-#include "sched.h"
 
 #ifdef CONFIG_IRQ_TIME_ACCOUNTING
 
kernel/sched/deadline.c +0 -2
index d2c072b0ef01..fca2d7de4d3d 100644
@@ -15,8 +15,6 @@
  *                    Michael Trimarchi <[email protected]>,
  *                    Fabio Checconi <[email protected]>
  */
-#include "sched.h"
-#include "pelt.h"
 
 struct dl_bandwidth def_dl_bandwidth;
 
kernel/sched/idle.c +0 -3
index d17b0a5ce6ac..8f8b5020e76a 100644
@@ -6,9 +6,6 @@
  * (NOTE: these are not related to SCHED_IDLE batch scheduled
  *        tasks which are handled in sched/fair.c )
  */
-#include "sched.h"
-
-#include <trace/events/power.h>
 
 /* Linker adds these: start and end of __cpuidle functions */
 extern char __cpuidle_text_start[], __cpuidle_text_end[];
kernel/sched/pelt.c +0 -4
index a554e3bbab2b..0f310768260c 100644
@@ -24,10 +24,6 @@
  *  Author: Vincent Guittot <[email protected]>
  */
 
-#include <linux/sched.h>
-#include "sched.h"
-#include "pelt.h"
-
 /*
  * Approximate:
  *   val * y^n,    where y^32 ~= 0.5 (~1 scheduling period)
kernel/sched/rt.c +0 -3
index 7b4f4fbbb404..ff4c044aed12 100644
@@ -3,9 +3,6 @@
  * Real-Time Scheduling Class (mapped to the SCHED_FIFO and SCHED_RR
  * policies)
  */
-#include "sched.h"
-
-#include "pelt.h"
 
 int sched_rr_timeslice = RR_TIMESLICE;
 int sysctl_sched_rr_timeslice = (MSEC_PER_SEC / HZ) * RR_TIMESLICE;
kernel/sched/sched.h +8 -0
index 79c7a8a2be65..f7d10b98e911 100644
@@ -13,6 +13,7 @@
 #include <linux/prctl.h>
 #include <linux/ptrace_api.h>
 #include <linux/gfp_api.h>
+#include <linux/posix-timers.h>
 #include <linux/sched/clock.h>
 #include <linux/workqueue_api.h>
 #include <linux/tick.h>
@@ -113,8 +114,11 @@
 #include <linux/cpumask_api.h>
 #include <linux/ctype.h>
 #include <linux/file.h>
+#include <linux/hrtimer_api.h>
+#include <linux/interrupt.h>
 #include <linux/jiffies.h>
 #include <linux/kref_api.h>
+#include <linux/ktime_api.h>
 #include <linux/lockdep_api.h>
 #include <linux/module.h>
 #include <linux/mutex_api.h>
@@ -126,12 +130,16 @@
 #include <linux/sched/loadavg.h>
 #include <linux/sched/mm.h>
 #include <linux/sched/rseq_api.h>
+#include <linux/sched/signal.h>
 #include <linux/seq_file.h>
 #include <linux/seqlock.h>
+#include <linux/softirq.h>
+#include <linux/spinlock_api.h>
 #include <linux/syscalls_api.h>
 #include <linux/syscalls.h>
 #include <linux/topology.h>
 #include <linux/types.h>
+#include <linux/u64_stats_sync_api.h>
 #include <linux/uaccess.h>
 #include <linux/wait_api.h>
 #include <linux/workqueue_api.h>
Signature · 1 line
2.32.0
#8 lore raw

PATCH 8/15 sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there

In reply to Ingo Molnar

Collect all utility functionality source code files into a single kernel/sched/build_utility.c file, via #include-ing the .c files:

kernel/sched/clock.c
kernel/sched/completion.c
kernel/sched/loadavg.c
kernel/sched/swait.c
kernel/sched/wait_bit.c
kernel/sched/wait.c

CONFIG_CPU_FREQ:
kernel/sched/cpufreq.c

CONFIG_CPU_FREQ_GOV_SCHEDUTIL:
kernel/sched/cpufreq_schedutil.c

CONFIG_CGROUP_CPUACCT:
kernel/sched/cpuacct.c

CONFIG_SCHED_DEBUG:
kernel/sched/debug.c

CONFIG_SCHEDSTATS:
kernel/sched/stats.c

CONFIG_SMP:
kernel/sched/cpupri.c
kernel/sched/stop_task.c
kernel/sched/topology.c

CONFIG_SCHED_CORE:
kernel/sched/core_sched.c

CONFIG_PSI:
kernel/sched/psi.c

CONFIG_MEMBARRIER:
kernel/sched/membarrier.c

CONFIG_CPU_ISOLATION:
kernel/sched/isolation.c

CONFIG_SCHED_AUTOGROUP:
kernel/sched/autogroup.c

The goal is to amortize the 60+ KLOC header bloat from over a dozen build units into a single build unit.

The build time of build_utility.c also roughly matches the build time of core.c and fair.c - allowing better load-balancing of scheduler-only rebuilds.

---
 kernel/sched/Makefile            | 25 +++++++++-------------
 kernel/sched/autogroup.c         |  3 +--
 kernel/sched/build_utility.c     | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 kernel/sched/clock.c             |  2 --
 kernel/sched/completion.c        |  2 +-
 kernel/sched/core_sched.c        |  3 ---
 kernel/sched/cpuacct.c           |  3 +--
 kernel/sched/cpufreq.c           |  3 ---
 kernel/sched/cpufreq_schedutil.c |  7 ------
 kernel/sched/cpupri.c            |  1 -
 kernel/sched/debug.c             |  1 -
 kernel/sched/isolation.c         |  1 -
 kernel/sched/loadavg.c           |  1 -
 kernel/sched/membarrier.c        |  1 -
 kernel/sched/psi.c               | 15 -------------
 kernel/sched/sched.h             | 57 +++++++++++++++++++++++++++++++++++++++++++++++--
 kernel/sched/stats.c             |  1 -
 kernel/sched/stop_task.c         |  1 -
 kernel/sched/swait.c             |  1 -
 kernel/sched/topology.c          |  1 -
 kernel/sched/wait.c              |  1 -
 kernel/sched/wait_bit.c          |  2 +-
 22 files changed, 139 insertions(+), 63 deletions(-)
kernel/sched/Makefile +10 -15
index c0c52026ad0d..4a4785cb3cd2 100644
@@ -22,18 +22,13 @@ ifneq ($(CONFIG_SCHED_OMIT_FRAME_POINTER),y)
 CFLAGS_core.o := $(PROFILING) -fno-omit-frame-pointer
 endif
 
-obj-y += core.o loadavg.o clock.o cputime.o
-obj-y += idle.o fair.o rt.o deadline.o
-obj-y += wait.o wait_bit.o swait.o completion.o
-
-obj-$(CONFIG_SMP) += cpupri.o cpudeadline.o topology.o stop_task.o pelt.o
-obj-$(CONFIG_SCHED_AUTOGROUP) += autogroup.o
-obj-$(CONFIG_SCHEDSTATS) += stats.o
-obj-$(CONFIG_SCHED_DEBUG) += debug.o
-obj-$(CONFIG_CGROUP_CPUACCT) += cpuacct.o
-obj-$(CONFIG_CPU_FREQ) += cpufreq.o
-obj-$(CONFIG_CPU_FREQ_GOV_SCHEDUTIL) += cpufreq_schedutil.o
-obj-$(CONFIG_MEMBARRIER) += membarrier.o
-obj-$(CONFIG_CPU_ISOLATION) += isolation.o
-obj-$(CONFIG_PSI) += psi.o
-obj-$(CONFIG_SCHED_CORE) += core_sched.o
+#
+# Build efficiency:
+#
+# These compilation units have roughly the same size and complexity - so their
+# build parallelizes well and finishes roughly at once:
+#
+obj-y += core.o
+obj-y += fair.o
+obj-y += build_utility.o
+obj-y += idle.o rt.o deadline.o cputime.o cpudeadline.o pelt.o
kernel/sched/autogroup.c +1 -2
index 31dd2593145e..16092b49ff6a 100644
@@ -1,9 +1,8 @@
 // SPDX-License-Identifier: GPL-2.0
+
 /*
  * Auto-group scheduling implementation:
  */
-#include <linux/nospec.h>
-#include "sched.h"
 
 unsigned int __read_mostly sysctl_sched_autogroup_enabled = 1;
 static struct autogroup autogroup_default;
kernel/sched/build_utility.c new file +70 -0

70 lines not on this page · raw message

kernel/sched/clock.c +0 -2
index 540d0e50e31c..d9272d9061a3 100644
@@ -53,8 +53,6 @@
  * that is otherwise invisible (TSC gets stopped).
  *
  */
-#include "sched.h"
-#include <linux/sched_clock.h>
 
 /*
  * Scheduler clock - returns current time in nanosec units.
kernel/sched/completion.c +1 -1
index a778554f9dad..35f15c26ed54 100644
@@ -1,4 +1,5 @@
 // SPDX-License-Identifier: GPL-2.0
+
 /*
  * Generic wait-for-completion handler;
  *
@@ -11,7 +12,6 @@
  * typically be used for exclusion which gives rise to priority inversion.
  * Waiting for completion is a typically sync point, but not an exclusion point.
  */
-#include "sched.h"
 
 /**
  * complete: - signals a single thread waiting on this completion
kernel/sched/core_sched.c +0 -3
index c8746a9a7ada..38a2cec21014 100644
@@ -1,8 +1,5 @@
 // SPDX-License-Identifier: GPL-2.0-only
 
-#include <linux/prctl.h>
-#include "sched.h"
-
 /*
  * A simple wrapper around refcount. An allocated sched_core_cookie's
  * address is used to compute the cookie of the task.
kernel/sched/cpuacct.c +1 -2
index 3d06c5e4220d..2c505cf800aa 100644
@@ -1,12 +1,11 @@
 // SPDX-License-Identifier: GPL-2.0
+
 /*
  * CPU accounting code for task groups.
  *
  * Based on the work by Paul Menage ([email protected]) and Balbir Singh
  * ([email protected]).
  */
-#include <asm/irq_regs.h>
-#include "sched.h"
 
 /* Time spent by the tasks of the CPU accounting group executing in ... */
 enum cpuacct_stat_index {
kernel/sched/cpufreq.c +0 -3
index 7c2fe50fd76d..5252fb191fae 100644
@@ -5,9 +5,6 @@
  * Copyright (C) 2016, Intel Corporation
  * Author: Rafael J. Wysocki <[email protected]>
  */
-#include <linux/cpufreq.h>
-
-#include "sched.h"
 
 DEFINE_PER_CPU(struct update_util_data __rcu *, cpufreq_update_util_data);
 
kernel/sched/cpufreq_schedutil.c +0 -7
index 6d65ab6e484e..f68885d049f2 100644
@@ -6,13 +6,6 @@
  * Author: Rafael J. Wysocki <[email protected]>
  */
 
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include "sched.h"
-
-#include <linux/sched/cpufreq.h>
-#include <trace/events/power.h>
-
 #define IOWAIT_BOOST_MIN	(SCHED_CAPACITY_SCALE / 8)
 
 struct sugov_tunables {
kernel/sched/cpupri.c +0 -1
index d583f2aa744e..fa9ce9d83683 100644
@@ -22,7 +22,6 @@
  *  worst case complexity of O(min(101, nr_domcpus)), though the scenario that
  *  yields the worst case search is fairly contrived.
  */
-#include "sched.h"
 
 /*
  * p->rt_priority   p->prio   newpri   cpupri
kernel/sched/debug.c +0 -1
index 102d6f70e84d..bb3d63bdf4ae 100644
@@ -6,7 +6,6 @@
  *
  * Copyright(C) 2007, Red Hat, Inc., Ingo Molnar
  */
-#include "sched.h"
 
 /*
  * This allows printing both to /proc/sched_debug and
kernel/sched/isolation.c +0 -1
index b4d10815c45a..373d42c707bc 100644
@@ -7,7 +7,6 @@
  * Copyright (C) 2017-2018 SUSE, Frederic Weisbecker
  *
  */
-#include "sched.h"
 
 enum hk_flags {
 	HK_FLAG_TIMER		= BIT(HK_TYPE_TIMER),
kernel/sched/loadavg.c +0 -1
index 954b229868d9..52c8f8226b0d 100644
@@ -6,7 +6,6 @@
  * figure. Its a silly number but people think its important. We go through
  * great pains to make it work on big machines and tickless kernels.
  */
-#include "sched.h"
 
 /*
  * Global load-average calculations
kernel/sched/membarrier.c +0 -1
index 3d2825408e3a..0c5be7ebb1dc 100644
@@ -4,7 +4,6 @@
  *
  * membarrier system call
  */
-#include "sched.h"
 
 /*
  * For documentation purposes, here are some membarrier ordering
kernel/sched/psi.c +0 -15
index 8fb08a12f602..a4fa3aadfcba 100644
@@ -137,21 +137,6 @@
  * sampling of the aggregate task states would be.
  */
 
-#include "../workqueue_internal.h"
-#include <linux/sched/loadavg.h>
-#include <linux/seq_file.h>
-#include <linux/proc_fs.h>
-#include <linux/seqlock.h>
-#include <linux/uaccess.h>
-#include <linux/cgroup.h>
-#include <linux/module.h>
-#include <linux/sched.h>
-#include <linux/ctype.h>
-#include <linux/file.h>
-#include <linux/poll.h>
-#include <linux/psi.h>
-#include "sched.h"
-
 static int psi_bug __read_mostly;
 
 DEFINE_STATIC_KEY_FALSE(psi_disabled);
kernel/sched/sched.h +55 -2

88 lines not on this page · raw message

kernel/sched/stats.c +0 -1
index 07dde2928c79..857f837f52cb 100644
@@ -2,7 +2,6 @@
 /*
  * /proc/schedstat implementation
  */
-#include "sched.h"
 
 void __update_stats_wait_start(struct rq *rq, struct task_struct *p,
 			       struct sched_statistics *stats)
kernel/sched/stop_task.c +0 -1
index 0b165a25f22f..d04073a93eb4 100644
@@ -7,7 +7,6 @@
  *
  * See kernel/stop_machine.c
  */
-#include "sched.h"
 
 #ifdef CONFIG_SMP
 static int
kernel/sched/swait.c +0 -1
index e1c655f928c7..76b9b796e695 100644
@@ -2,7 +2,6 @@
 /*
  * <linux/swait.h> (simple wait queues ) implementation:
  */
-#include "sched.h"
 
 void __init_swait_queue_head(struct swait_queue_head *q, const char *name,
 			     struct lock_class_key *key)
kernel/sched/topology.c +0 -1
index 32841c6741d1..e8af72fd70bd 100644
@@ -2,7 +2,6 @@
 /*
  * Scheduler topology setup/handling methods
  */
-#include "sched.h"
 
 DEFINE_MUTEX(sched_domains_mutex);
 
kernel/sched/wait.c +0 -1
index eca38107b32f..9860bb9a847c 100644
@@ -4,7 +4,6 @@
  *
  * (C) 2004 Nadia Yvette Chambers, Oracle
  */
-#include "sched.h"
 
 void __init_waitqueue_head(struct wait_queue_head *wq_head, const char *name, struct lock_class_key *key)
 {
kernel/sched/wait_bit.c +1 -1
index 02ce292b9bc0..d4788f810b55 100644
@@ -1,8 +1,8 @@
 // SPDX-License-Identifier: GPL-2.0-only
+
 /*
  * The implementation of the wait_bit*() and related waiting APIs:
  */
-#include "sched.h"
 
 #define WAIT_TABLE_BITS 8
 #define WAIT_TABLE_SIZE (1 << WAIT_TABLE_BITS)
Signature · 1 line
2.32.0
#13 lore raw

PATCH 11/15 sched/headers: Reorganize, clean up and optimize kernel/sched/core.c dependencies

In reply to Ingo Molnar

Use all generic headers from kernel/sched/sched.h that are required for it to build.

Sort the sections alphabetically.

---
 kernel/sched/core.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 70 insertions(+), 11 deletions(-)
kernel/sched/core.c +70 -11
index 22de53710ee9..5eaa4211d93c 100644
@@ -6,7 +6,73 @@
  *
  *  Copyright (C) 1991-2002  Linus Torvalds
  */
+#include <linux/highmem.h>
+#include <linux/hrtimer_api.h>
+#include <linux/ktime_api.h>
+#include <linux/sched/signal.h>
+#include <linux/syscalls_api.h>
+#include <linux/debug_locks.h>
+#include <linux/prefetch.h>
+#include <linux/capability.h>
+#include <linux/pgtable_api.h>
+#include <linux/wait_bit.h>
+#include <linux/jiffies.h>
+#include <linux/spinlock_api.h>
+#include <linux/cpumask_api.h>
+#include <linux/lockdep_api.h>
+#include <linux/hardirq.h>
+#include <linux/softirq.h>
+#include <linux/refcount_api.h>
+#include <linux/topology.h>
+#include <linux/sched/clock.h>
+#include <linux/sched/cond_resched.h>
+#include <linux/sched/debug.h>
+#include <linux/sched/isolation.h>
+#include <linux/sched/loadavg.h>
+#include <linux/sched/mm.h>
+#include <linux/sched/nohz.h>
+#include <linux/sched/rseq_api.h>
+#include <linux/sched/rt.h>
+
+#include <linux/blkdev.h>
+#include <linux/context_tracking.h>
+#include <linux/cpuset.h>
+#include <linux/delayacct.h>
+#include <linux/init_task.h>
+#include <linux/interrupt.h>
+#include <linux/ioprio.h>
+#include <linux/kallsyms.h>
+#include <linux/kcov.h>
+#include <linux/kprobes.h>
+#include <linux/llist_api.h>
+#include <linux/mmu_context.h>
+#include <linux/mmzone.h>
+#include <linux/mutex_api.h>
+#include <linux/nmi.h>
+#include <linux/nospec.h>
+#include <linux/perf_event_api.h>
+#include <linux/profile.h>
+#include <linux/psi.h>
+#include <linux/rcuwait_api.h>
+#include <linux/sched/wake_q.h>
+#include <linux/scs.h>
+#include <linux/slab.h>
+#include <linux/syscalls.h>
+#include <linux/vtime.h>
+#include <linux/wait_api.h>
+#include <linux/workqueue_api.h>
+
+#ifdef CONFIG_PREEMPT_DYNAMIC
+# include <linux/entry-common.h>
+#endif
+
+#include <uapi/linux/sched/types.h>
+
+#include <asm/switch_to.h>
+#include <asm/tlb.h>
+
 #define CREATE_TRACE_POINTS
+#include <linux/sched/rseq_api.h>
 #include <trace/events/sched.h>
 #undef CREATE_TRACE_POINTS
 
@@ -14,22 +80,15 @@
 #include "stats.h"
 #include "autogroup.h"
 
-#include <linux/nospec.h>
-#include <linux/blkdev.h>
-#include <linux/jump_label.h>
-#include <linux/kcov.h>
-#include <linux/scs.h>
-
-#include <asm/switch_to.h>
-#include <asm/tlb.h>
+#include "autogroup.h"
+#include "pelt.h"
+#include "smp.h"
+#include "stats.h"
 
 #include "../workqueue_internal.h"
 #include "../../fs/io-wq.h"
 #include "../smpboot.h"
 
-#include "pelt.h"
-#include "smp.h"
-
 /*
  * Export tracepoints that act as a bare tracehook (ie: have no trace event
  * associated with them) to allow external modules to probe them.
Signature · 1 line
2.32.0
#15 lore raw

PATCH 13/15 sched/headers: Reorganize, clean up and optimize kernel/sched/build_policy.c dependencies

In reply to Ingo Molnar

Use all generic headers from kernel/sched/sched.h that are required for it to build.

Sort the sections alphabetically.

---
 kernel/sched/build_policy.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
kernel/sched/build_policy.c +24 -0
index 9a169b2f97e6..a2e4023771a0 100644
@@ -12,12 +12,36 @@
  * core.c and fair.c are built separately.
  */
 
+/* Headers: */
+#include <linux/sched/clock.h>
+#include <linux/sched/cputime.h>
+#include <linux/sched/posix-timers.h>
+#include <linux/sched/rt.h>
+
+#include <linux/cpuidle.h>
+#include <linux/jiffies.h>
+#include <linux/livepatch.h>
+#include <linux/psi.h>
+#include <linux/seqlock_api.h>
+#include <linux/slab.h>
+#include <linux/suspend.h>
+#include <linux/tsacct_kern.h>
+#include <linux/vtime.h>
+
+#ifdef CONFIG_PARAVIRT
+# include <asm/paravirt.h>
+#endif
+
+#include <uapi/linux/sched/types.h>
+
 #include "sched.h"
 
 #include "autogroup.h"
 #include "stats.h"
 #include "pelt.h"
 
+/* Source code modules: */
+
 #include "idle.c"
 
 #include "rt.c"
Signature · 1 line
2.32.0
#14 lore raw

PATCH 12/15 sched/headers: Reorganize, clean up and optimize kernel/sched/fair.c dependencies

In reply to Ingo Molnar

Use all generic headers from kernel/sched/sched.h that are required for it to build.

Sort the sections alphabetically.

---
 kernel/sched/fair.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
kernel/sched/fair.c +29 -0
index 7391c9287503..89d21fda106c 100644
@@ -20,6 +20,35 @@
  *  Adaptive scheduling granularity, math enhancements by Peter Zijlstra
  *  Copyright (C) 2007 Red Hat, Inc., Peter Zijlstra
  */
+#include <linux/energy_model.h>
+#include <linux/mmap_lock.h>
+#include <linux/hugetlb_inline.h>
+#include <linux/jiffies.h>
+#include <linux/mm_api.h>
+#include <linux/highmem.h>
+#include <linux/spinlock_api.h>
+#include <linux/cpumask_api.h>
+#include <linux/lockdep_api.h>
+#include <linux/softirq.h>
+#include <linux/refcount_api.h>
+#include <linux/topology.h>
+#include <linux/sched/clock.h>
+#include <linux/sched/cond_resched.h>
+#include <linux/sched/cputime.h>
+#include <linux/sched/isolation.h>
+
+#include <linux/cpuidle.h>
+#include <linux/interrupt.h>
+#include <linux/mempolicy.h>
+#include <linux/mutex_api.h>
+#include <linux/profile.h>
+#include <linux/psi.h>
+#include <linux/ratelimit.h>
+
+#include <asm/switch_to.h>
+
+#include <linux/sched/cond_resched.h>
+
 #include "sched.h"
 #include "stats.h"
 #include "autogroup.h"
Signature · 1 line
2.32.0
#16 lore raw

PATCH 14/15 sched/headers: Reorganize, clean up and optimize kernel/sched/build_utility.c dependencies

In reply to Ingo Molnar

Use all generic headers from kernel/sched/sched.h that are required for it to build.

Sort the sections alphabetically.

---
 kernel/sched/build_utility.c | 43 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)
kernel/sched/build_utility.c +41 -2
index 8b2b199983bf..bc2f4d648209 100644
@@ -9,14 +9,53 @@
  *   coalescing source files to amortize header inclusion
  *   cost. )
  */
+#include <linux/sched/clock.h>
+#include <linux/sched/cputime.h>
+#include <linux/sched/debug.h>
+#include <linux/sched/isolation.h>
+#include <linux/sched/loadavg.h>
+#include <linux/sched/mm.h>
+#include <linux/sched/rseq_api.h>
+#include <linux/sched/task_stack.h>
+
+#include <linux/cpufreq.h>
+#include <linux/cpumask_api.h>
+#include <linux/cpuset.h>
+#include <linux/ctype.h>
+#include <linux/debugfs.h>
+#include <linux/energy_model.h>
+#include <linux/hashtable_api.h>
+#include <linux/irq.h>
+#include <linux/kobject_api.h>
+#include <linux/membarrier.h>
+#include <linux/mempolicy.h>
+#include <linux/nmi.h>
+#include <linux/nospec.h>
+#include <linux/proc_fs.h>
+#include <linux/psi.h>
+#include <linux/psi.h>
+#include <linux/ptrace_api.h>
+#include <linux/sched_clock.h>
+#include <linux/security.h>
+#include <linux/spinlock_api.h>
+#include <linux/swait_api.h>
+#include <linux/timex.h>
+#include <linux/utsname.h>
+#include <linux/wait_api.h>
+#include <linux/workqueue_api.h>
+
+#ifdef CONFIG_PARAVIRT
+# include <asm/paravirt.h>
+#endif
+
+#include <uapi/linux/prctl.h>
+#include <uapi/linux/sched/types.h>
 
 #include "sched.h"
 #include "sched-pelt.h"
 #include "stats.h"
 #include "autogroup.h"
 
-#include <linux/sched_clock.h>
-
 #include "clock.c"
 
 #ifdef CONFIG_CGROUP_CPUACCT
Signature · 1 line
2.32.0
#17 lore raw

PATCH 15/15 sched/headers: Reorganize, clean up and optimize kernel/sched/sched.h dependencies

In reply to Ingo Molnar

Remove all headers, except the ones required to make this header build standalone.

Also include stats.h in sched.h explicitly - dependencies already require this.

Summary of the build speedup gained through the last ~15 scheduler build & header dependency patches:

Cumulative scheduler (kernel/sched/) build time speedup on a Linux distribution's config, which enables all scheduler features, compared to the vanilla kernel:

_____________________________________________________________________________

 |
 |  Vanilla kernel (v5.13-rc7):
 |_____________________________________________________________________________
 |
 |  Performance counter stats for 'make -j96 kernel/sched/' (3 runs):
 |
 |   126,975,564,374      instructions              #    1.45  insn per cycle           ( +-  0.00% )
 |    87,637,847,671      cycles                    #    3.959 GHz                      ( +-  0.30% )
 |         22,136.96 msec cpu-clock                 #    7.499 CPUs utilized            ( +-  0.29% )
 |
 |            2.9520 +- 0.0169 seconds time elapsed  ( +-  0.57% )
 |_____________________________________________________________________________
 |
 |  Patched kernel:
 |_____________________________________________________________________________
 |
 | Performance counter stats for 'make -j96 kernel/sched/' (3 runs):
 |
 |    50,420,496,914      instructions              #    1.47  insn per cycle           ( +-  0.00% )
 |    34,234,322,038      cycles                    #    3.946 GHz                      ( +-  0.31% )
 |          8,675.81 msec cpu-clock                 #    3.053 CPUs utilized            ( +-  0.45% )
 |
 |            2.8420 +- 0.0181 seconds time elapsed  ( +-  0.64% )
 |_____________________________________________________________________________

Summary:

- CPU time used to build the scheduler dropped by -60.9%, a reduction from 22.1 clock-seconds to 8.7 clock-seconds.

- Wall-clock time to build the scheduler dropped by -3.9%, a reduction

    from 2.95 seconds to 2.84 seconds.
---
 kernel/sched/build_policy.c  |   4 --
 kernel/sched/build_utility.c |   6 +--
 kernel/sched/sched.h         | 147 +++++++++++++++++++--------------------------------------------
 kernel/sched/stats.h         |   2 -
 4 files changed, 46 insertions(+), 113 deletions(-)
kernel/sched/build_policy.c +0 -4
index a2e4023771a0..e0104b45029a 100644
@@ -28,10 +28,6 @@
 #include <linux/tsacct_kern.h>
 #include <linux/vtime.h>
 
-#ifdef CONFIG_PARAVIRT
-# include <asm/paravirt.h>
-#endif
-
 #include <uapi/linux/sched/types.h>
 
 #include "sched.h"
kernel/sched/build_utility.c +2 -4
index bc2f4d648209..eec0849b2aae 100644
@@ -44,13 +44,11 @@
 #include <linux/wait_api.h>
 #include <linux/workqueue_api.h>
 
-#ifdef CONFIG_PARAVIRT
-# include <asm/paravirt.h>
-#endif
-
 #include <uapi/linux/prctl.h>
 #include <uapi/linux/sched/types.h>
 
+#include <asm/switch_to.h>
+
 #include "sched.h"
 #include "sched-pelt.h"
 #include "stats.h"
kernel/sched/sched.h +44 -103

208 lines not on this page · raw message

kernel/sched/stats.h +0 -2
index a0df79e6038c..baa839c1ba96 100644
@@ -2,8 +2,6 @@
 #ifndef _KERNEL_STATS_H
 #define _KERNEL_STATS_H
 
-#include "sched.h"
-
 #ifdef CONFIG_SCHEDSTATS
 
 extern struct static_key_false sched_schedstats;
Signature · 1 line
2.32.0
#10 lore raw

Re: [PATCH 09/15] sched/headers: Introduce kernel/sched/build_policy.c and build multiple .c files there

In reply to Ingo Molnar

Hi Ingo, Since commit f96eca432015ddc1b621632488ebc345bca06791 merged, the scripts/clang-tools/gen_compile_commands.py can't work very well.
In this patch the build_policy.c and build_utility.c include kernel/sched/xxx.c(
such as rt.c idle.c...), so it compile the build_utility.o build_policy.o directly,
and it won't generate kernel/sched/xxx.c(such as rt.o idle.o), so the gen_compile_commands.py can't work.
It will report "[8/23/2022, 3:24:06 PM] "rt.c" not found in "${workspaceFolder}/compile_commands.json".
'includePath' from c_cpp_properties.json in folder 'linux' will be used for this file instead.".
And i test the kdump & crash tools, it can find the functions by line in kernel/sched/xxx.c(
such as rt.c idle.c), it works well.
I can't solve this report error, so i want to get help.
Maybe we change the gen_compile_commands.py code can solve this problem or
can we revert this change?(3.9% build wall time decrease really matter? it will
destory all tool's parse result which depends on the compiled file result).

#11 lore raw

Re: [PATCH 09/15] sched/headers: Introduce kernel/sched/build_policy.c and build multiple .c files there

Quoted 7 lines · suhui kernel

Hi Ingo, Since commit f96eca432015ddc1b621632488ebc345bca06791 merged, the scripts/clang-tools/gen_compile_commands.py can't work very well.

In this patch the build_policy.c and build_utility.c include kernel/sched/xxx.c(
such as rt.c idle.c...), so it compile the build_utility.o build_policy.o directly, and it won't generate kernel/sched/xxx.c(such as rt.o idle.o), so the gen_compile_commands.py can't work.

gen_compile_commands.py works perfectly fine and generates the entries for build_policy.o and built_utility.o as expected.

gen_compile_commands.py scans the build tree for .cmd files and extracts the command line from those files into the the json file. As there is no .cmd file for the .c files which are included into build_*.c there wont be entries in the json file either.

Quoted 3 lines

It will report "[8/23/2022, 3:24:06 PM] "rt.c" not found in "${workspaceFolder}/compile_commands.json".
'includePath' from c_cpp_properties.json in folder 'linux' will be
used for this file instead.".

No idea where this message comes from, but certainly not from gen_compile_commands.py because that generates the file.

The clang-tidy kernel build target generates and uses that file and that works perfectly fine.

So what exactly broke?

Thanks,

tglx