Skip to contentSkip to conversation

tools/sched_ext: Add example C schedulers

PATCH In review Emil Tsalapatis 2 participants 6 messages Jan 23 – Jan 28 Scheduler - sched_ext

lore
Share & tools
lore
#0 lore raw

v3 cover tools/sched_ext: Add example C schedulers

Add to the tree several C sched-ext schedulers developed and maintained out-of-tree by the sched-ext project over the years. These schedulers demonstrate sched-ext's feature set and provide a starting point for scheduler development. Place the schedulers together with the existing examples in tools/sched_ext.

v2->v3 (https://lore.kernel.org/sched-ext/[email protected]/)

- Fix inverted condition in scx_sdt for zeroing out freed memory (AI)

v1->v2 (https://lore.kernel.org/sched-ext/[email protected]/

- Remove scx_nest scheduler from patchset (Changwoo, Andrea)

Emil Tsalapatis (3):
tools/sched_ext: add scx_userland scheduler
tools/sched_ext: add scx_pair scheduler
tools/sched_ext: add arena based scheduler

 tools/sched_ext/Makefile           |   2 +-
 tools/sched_ext/scx_pair.bpf.c     | 610 +++++++++++++++++++++++++
 tools/sched_ext/scx_pair.c         | 180 ++++++++
 tools/sched_ext/scx_pair.h         |   9 +
 tools/sched_ext/scx_sdt.bpf.c      | 710 +++++++++++++++++++++++++++++
 tools/sched_ext/scx_sdt.c          | 101 ++++
 tools/sched_ext/scx_sdt.h          | 113 +++++
 tools/sched_ext/scx_userland.bpf.c | 344 ++++++++++++++
 tools/sched_ext/scx_userland.c     | 437 ++++++++++++++++++
 tools/sched_ext/scx_userland.h     |  17 +
 10 files changed, 2522 insertions(+), 1 deletion(-)
 create mode 100644 tools/sched_ext/scx_pair.bpf.c
 create mode 100644 tools/sched_ext/scx_pair.c
 create mode 100644 tools/sched_ext/scx_pair.h
 create mode 100644 tools/sched_ext/scx_sdt.bpf.c
 create mode 100644 tools/sched_ext/scx_sdt.c
 create mode 100644 tools/sched_ext/scx_sdt.h
 create mode 100644 tools/sched_ext/scx_userland.bpf.c
 create mode 100644 tools/sched_ext/scx_userland.c
 create mode 100644 tools/sched_ext/scx_userland.h
Signature · 1 line
2.49.0
#1 lore raw

PATCH v3 1/3 tools/sched_ext: add scx_userland scheduler

Add in the scx_userland scheduler that does vruntime-based scheduling in userspace code and communicates scheduling decisions to BPF by accessing and modifying globals through the skeleton.

---
 tools/sched_ext/Makefile           |   2 +-
 tools/sched_ext/scx_userland.bpf.c | 344 +++++++++++++++++++++++
 tools/sched_ext/scx_userland.c     | 437 +++++++++++++++++++++++++++++
 tools/sched_ext/scx_userland.h     |  17 ++
 4 files changed, 799 insertions(+), 1 deletion(-)
 create mode 100644 tools/sched_ext/scx_userland.bpf.c
 create mode 100644 tools/sched_ext/scx_userland.c
 create mode 100644 tools/sched_ext/scx_userland.h
tools/sched_ext/Makefile +1 -1
index e4bda2474060..12043a82a1a9 100644
@@ -189,7 +189,7 @@ $(INCLUDE_DIR)/%.bpf.skel.h: $(SCXOBJ_DIR)/%.bpf.o $(INCLUDE_DIR)/vmlinux.h $(BP
 
 SCX_COMMON_DEPS := include/scx/common.h include/scx/user_exit_info.h | $(BINDIR)
 
-c-sched-targets = scx_simple scx_cpu0 scx_qmap scx_central scx_flatcg
+c-sched-targets = scx_simple scx_cpu0 scx_qmap scx_central scx_flatcg scx_userland
 
 $(addprefix $(BINDIR)/,$(c-sched-targets)): \
 	$(BINDIR)/%: \
tools/sched_ext/scx_userland.bpf.c new file +344 -0

344 lines not on this page · raw message

tools/sched_ext/scx_userland.c new file +437 -0

437 lines not on this page · raw message

tools/sched_ext/scx_userland.h new file +17 -0
new file mode 100644
index 000000000000..684fb2dd5de9
@@ -0,0 +1,17 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2022 Meta, Inc */
+
+#ifndef __SCX_USERLAND_COMMON_H
+#define __SCX_USERLAND_COMMON_H
+
+/*
+ * An instance of a task that has been enqueued by the kernel for consumption
+ * by a user space global scheduler thread.
+ */
+struct scx_userland_enqueued_task {
+	__s32 pid;
+	u64 sum_exec_runtime;
+	u64 weight;
+};
+
+#endif  // __SCX_USERLAND_COMMON_H
Signature · 1 line
2.49.0
#2 lore raw

PATCH v3 2/3 tools/sched_ext: add scx_pair scheduler

In reply to Emil Tsalapatis

Add the scx_pair cgroup-based core scheduler.

---
 tools/sched_ext/Makefile       |   2 +-
 tools/sched_ext/scx_pair.bpf.c | 610 +++++++++++++++++++++++++++++++++
 tools/sched_ext/scx_pair.c     | 180 ++++++++++
 tools/sched_ext/scx_pair.h     |   9 +
 4 files changed, 800 insertions(+), 1 deletion(-)
 create mode 100644 tools/sched_ext/scx_pair.bpf.c
 create mode 100644 tools/sched_ext/scx_pair.c
 create mode 100644 tools/sched_ext/scx_pair.h
tools/sched_ext/Makefile +1 -1
index 12043a82a1a9..208e8f8fe4d8 100644
@@ -189,7 +189,7 @@ $(INCLUDE_DIR)/%.bpf.skel.h: $(SCXOBJ_DIR)/%.bpf.o $(INCLUDE_DIR)/vmlinux.h $(BP
 
 SCX_COMMON_DEPS := include/scx/common.h include/scx/user_exit_info.h | $(BINDIR)
 
-c-sched-targets = scx_simple scx_cpu0 scx_qmap scx_central scx_flatcg scx_userland
+c-sched-targets = scx_simple scx_cpu0 scx_qmap scx_central scx_flatcg scx_userland scx_pair
 
 $(addprefix $(BINDIR)/,$(c-sched-targets)): \
 	$(BINDIR)/%: \
tools/sched_ext/scx_pair.bpf.c new file +610 -0

610 lines not on this page · raw message

tools/sched_ext/scx_pair.c new file +180 -0

180 lines not on this page · raw message

tools/sched_ext/scx_pair.h new file +9 -0
new file mode 100644
index 000000000000..d9666a447d3f
@@ -0,0 +1,9 @@
+#ifndef __SCX_EXAMPLE_PAIR_H
+#define __SCX_EXAMPLE_PAIR_H
+
+enum {
+	MAX_QUEUED		= 4096,
+	MAX_CGRPS		= 4096,
+};
+
+#endif /* __SCX_EXAMPLE_PAIR_H */
Signature · 1 line
2.49.0
#3 lore raw

PATCH v3 3/3 tools/sched_ext: add arena based scheduler

In reply to Emil Tsalapatis

Add a scheduler that uses BPF arenas to manage task context data.

---
 tools/sched_ext/Makefile      |   2 +-
 tools/sched_ext/scx_sdt.bpf.c | 710 ++++++++++++++++++++++++++++++++++
 tools/sched_ext/scx_sdt.c     | 101 +++++
 tools/sched_ext/scx_sdt.h     | 113 ++++++
 4 files changed, 925 insertions(+), 1 deletion(-)
 create mode 100644 tools/sched_ext/scx_sdt.bpf.c
 create mode 100644 tools/sched_ext/scx_sdt.c
 create mode 100644 tools/sched_ext/scx_sdt.h
tools/sched_ext/Makefile +1 -1
index 208e8f8fe4d8..47ad7444677e 100644
@@ -189,7 +189,7 @@ $(INCLUDE_DIR)/%.bpf.skel.h: $(SCXOBJ_DIR)/%.bpf.o $(INCLUDE_DIR)/vmlinux.h $(BP
 
 SCX_COMMON_DEPS := include/scx/common.h include/scx/user_exit_info.h | $(BINDIR)
 
-c-sched-targets = scx_simple scx_cpu0 scx_qmap scx_central scx_flatcg scx_userland scx_pair
+c-sched-targets = scx_simple scx_cpu0 scx_qmap scx_central scx_flatcg scx_userland scx_pair scx_sdt
 
 $(addprefix $(BINDIR)/,$(c-sched-targets)): \
 	$(BINDIR)/%: \
tools/sched_ext/scx_sdt.bpf.c new file +710 -0

710 lines not on this page · raw message

tools/sched_ext/scx_sdt.c new file +101 -0

101 lines not on this page · raw message

tools/sched_ext/scx_sdt.h new file +113 -0

113 lines not on this page · raw message

Signature · 1 line
2.49.0
reviewer
#4 lore raw

Re: [PATCH v3 3/3] tools/sched_ext: add arena based scheduler

[This is a AI generated review by Claude Opus 4.5]

tools/sched_ext/scx_sdt.bpf.c line 317

Quoted 6 lines
tools/sched_ext/scx_sdt.bpf.c new file +6 -0
+	pos = idx & mask;
+	data = chunk->data[pos];
+	if (likely(data)) {
+		data[pos] = (struct sdt_data) {
+			.tid.genn = data->tid.genn + 1,
+		};

data already points to chunk->data[pos], so data[pos] double-indexes and writes to the wrong memory location. This should be *data. The payload zeroing loop below correctly uses data->payload[i], so this is just the struct assignment that's off.

Please send a fix.

reviewer
#5 lore raw
In reply to Emil Tsalapatis
Quoted 4 lines

Emil Tsalapatis (3):
tools/sched_ext: add scx_userland scheduler
tools/sched_ext: add scx_pair scheduler
tools/sched_ext: add arena based scheduler

Applied 1-3 to sched_ext/for-6.20.

Note that there's a bug in 3/3 which I noted in a separate reply. Please send a fix.

Thanks.
--
tejun