Skip to contentSkip to conversation

sched_ext: Document the ops compat strategy in compat.h/compat.bpf.h

PATCH Applied Tejun Heo 3 participants 4 messages Apr 19 Scheduler - sched_ext

lore
Share & tools
lore

Tejun Heo

author reviewer
#0 lore raw

PATCH 1/1 sched_ext: Document the ops compat strategy in compat.h/compat.bpf.h

The comments around SCX_OPS_DEFINE() and SCX_OPS_OPEN() were vague about how backward compatibility actually works. Expand them to describe the two
mechanisms: load-time BTF fix-up for additive changes, and multi-variant struct_ops for incompatible ones.

---
 tools/sched_ext/include/scx/compat.bpf.h |    6 ++++--
 tools/sched_ext/include/scx/compat.h     |   23 +++++++++++++++++++----
 2 files changed, 23 insertions(+), 6 deletions(-)
tools/sched_ext/include/scx/compat.bpf.h +4 -2
@@ -423,8 +423,10 @@ static inline void scx_bpf_dsq_reenq(u64
 }
 
 /*
- * Define sched_ext_ops. This may be expanded to define multiple variants for
- * backward compatibility. See compat.h::SCX_OPS_LOAD/ATTACH().
+ * Define sched_ext_ops. See compat.h::SCX_OPS_OPEN() for how backward
+ * compatibility is handled (this macro can be expanded to emit multiple
+ * variants for incompatible op changes; SCX_OPS_OPEN() handles purely
+ * additive changes at load time).
  */
 #define SCX_OPS_DEFINE(__name, ...)						\
 	SEC(".struct_ops.link")							\
tools/sched_ext/include/scx/compat.h +19 -4
@@ -149,10 +149,24 @@ static inline long scx_hotplug_seq(void)
 }
 
 /*
- * struct sched_ext_ops can change over time. If compat.bpf.h::SCX_OPS_DEFINE()
- * is used to define ops and compat.h::SCX_OPS_LOAD/ATTACH() are used to load
- * and attach it, backward compatibility is automatically maintained where
- * reasonable.
+ * Open the sched_ext_ops skeleton.
+ *
+ * struct sched_ext_ops can change over time. Two complementary mechanisms
+ * keep BPF schedulers built against newer headers running on older kernels:
+ *
+ * 1. Load-time fix-up (this macro). For each optional ops callback or field
+ *    added to struct sched_ext_ops, an explicit stanza below probes the
+ *    running kernel's BTF via __COMPAT_struct_has_field() and, if the field
+ *    is missing, clears it in the in-memory struct_ops (with a warning to
+ *    stderr) before load. Handles additive changes - a new stanza must be
+ *    added here for each new optional field.
+ *
+ * 2. Multi-variant struct_ops via compat.bpf.h::SCX_OPS_DEFINE(). That
+ *    macro can be expanded to emit several variants of struct sched_ext_ops,
+ *    and SCX_OPS_LOAD()/ATTACH() can pick the right one based on what the
+ *    kernel supports. Needed when an existing operation has to change
+ *    incompatibly (e.g. a callback signature changes); the load-time
+ *    fix-up above only handles purely additive changes.
  *
  * ec7e3b0463e1 ("implement-ops") in https://github.com/sched-ext/sched_ext is
  * the current minimum required kernel version.
@@ -225,6 +239,7 @@ static inline void __scx_ops_assoc_prog(
 }
 #endif
 
+/* See SCX_OPS_OPEN() above for backward-compatibility handling. */
 #define SCX_OPS_LOAD(__skel, __ops_name, __scx_name, __uei_name) ({		\
 	struct bpf_program *__prog;						\
 	UEI_SET_SIZE(__skel, __ops_name, __uei_name);				\
reviewer Acked-by on v1 1/1
#1 lore raw

Hi Tejun,

Quoted 4 lines · Tejun Heo

The comments around SCX_OPS_DEFINE() and SCX_OPS_OPEN() were vague about how backward compatibility actually works. Expand them to describe the two
mechanisms: load-time BTF fix-up for additive changes, and multi-variant struct_ops for incompatible ones.

Thanks for clarifying these comments.

-Andrea

Quoted 61 lines
---
 tools/sched_ext/include/scx/compat.bpf.h |    6 ++++--
 tools/sched_ext/include/scx/compat.h     |   23 +++++++++++++++++++----
 2 files changed, 23 insertions(+), 6 deletions(-)
tools/sched_ext/include/scx/compat.bpf.h +4 -2
@@ -423,8 +423,10 @@ static inline void scx_bpf_dsq_reenq(u64
 }
 
 /*
- * Define sched_ext_ops. This may be expanded to define multiple variants for
- * backward compatibility. See compat.h::SCX_OPS_LOAD/ATTACH().
+ * Define sched_ext_ops. See compat.h::SCX_OPS_OPEN() for how backward
+ * compatibility is handled (this macro can be expanded to emit multiple
+ * variants for incompatible op changes; SCX_OPS_OPEN() handles purely
+ * additive changes at load time).
  */
 #define SCX_OPS_DEFINE(__name, ...)						\
 	SEC(".struct_ops.link")							\
tools/sched_ext/include/scx/compat.h +19 -4
@@ -149,10 +149,24 @@ static inline long scx_hotplug_seq(void)
 }
 
 /*
- * struct sched_ext_ops can change over time. If compat.bpf.h::SCX_OPS_DEFINE()
- * is used to define ops and compat.h::SCX_OPS_LOAD/ATTACH() are used to load
- * and attach it, backward compatibility is automatically maintained where
- * reasonable.
+ * Open the sched_ext_ops skeleton.
+ *
+ * struct sched_ext_ops can change over time. Two complementary mechanisms
+ * keep BPF schedulers built against newer headers running on older kernels:
+ *
+ * 1. Load-time fix-up (this macro). For each optional ops callback or field
+ *    added to struct sched_ext_ops, an explicit stanza below probes the
+ *    running kernel's BTF via __COMPAT_struct_has_field() and, if the field
+ *    is missing, clears it in the in-memory struct_ops (with a warning to
+ *    stderr) before load. Handles additive changes - a new stanza must be
+ *    added here for each new optional field.
+ *
+ * 2. Multi-variant struct_ops via compat.bpf.h::SCX_OPS_DEFINE(). That
+ *    macro can be expanded to emit several variants of struct sched_ext_ops,
+ *    and SCX_OPS_LOAD()/ATTACH() can pick the right one based on what the
+ *    kernel supports. Needed when an existing operation has to change
+ *    incompatibly (e.g. a callback signature changes); the load-time
+ *    fix-up above only handles purely additive changes.
  *
  * ec7e3b0463e1 ("implement-ops") in https://github.com/sched-ext/sched_ext is
  * the current minimum required kernel version.
@@ -225,6 +239,7 @@ static inline void __scx_ops_assoc_prog(
 }
 #endif
 
+/* See SCX_OPS_OPEN() above for backward-compatibility handling. */
 #define SCX_OPS_LOAD(__skel, __ops_name, __scx_name, __uei_name) ({		\
 	struct bpf_program *__prog;						\
 	UEI_SET_SIZE(__skel, __ops_name, __uei_name);				\
Acked-by on v1 1/1
#2 lore raw
In reply to Tejun Heo

Hi Tejun,

Quoted 4 lines · Tejun Heo

The comments around SCX_OPS_DEFINE() and SCX_OPS_OPEN() were vague about how backward compatibility actually works. Expand them to describe the two
mechanisms: load-time BTF fix-up for additive changes, and multi-variant struct_ops for incompatible ones.

Thanks for improving this. It is much clearer now.

Quoted 40 lines
---
 tools/sched_ext/include/scx/compat.bpf.h |    6 ++++--
 tools/sched_ext/include/scx/compat.h     |   23 +++++++++++++++++++----
 2 files changed, 23 insertions(+), 6 deletions(-)
tools/sched_ext/include/scx/compat.bpf.h +4 -2
@@ -423,8 +423,10 @@ static inline void scx_bpf_dsq_reenq(u64
 }
 
 /*
- * Define sched_ext_ops. This may be expanded to define multiple variants for
- * backward compatibility. See compat.h::SCX_OPS_LOAD/ATTACH().
+ * Define sched_ext_ops. See compat.h::SCX_OPS_OPEN() for how backward
+ * compatibility is handled (this macro can be expanded to emit multiple
+ * variants for incompatible op changes; SCX_OPS_OPEN() handles purely
+ * additive changes at load time).
  */
 #define SCX_OPS_DEFINE(__name, ...)						\
 	SEC(".struct_ops.link")							\
tools/sched_ext/include/scx/compat.h +8 -4
@@ -149,10 +149,24 @@ static inline long scx_hotplug_seq(void)
 }
 
 /*
- * struct sched_ext_ops can change over time. If compat.bpf.h::SCX_OPS_DEFINE()
- * is used to define ops and compat.h::SCX_OPS_LOAD/ATTACH() are used to load
- * and attach it, backward compatibility is automatically maintained where
- * reasonable.
+ * Open the sched_ext_ops skeleton.
+ *
+ * struct sched_ext_ops can change over time. Two complementary mechanisms
+ * keep BPF schedulers built against newer headers running on older kernels:
+ *
+ * 1. Load-time fix-up (this macro). For each optional ops callback or field
+ *    added to struct sched_ext_ops, an explicit stanza below probes the
+ *    running kernel's BTF via __COMPAT_struct_has_field() and, if the field

tools/sched_ext/include/scx/compat.h static inline long scx_hotplug_seq(void) · line 169

Quoted 10 lines
tools/sched_ext/include/scx/compat.h +10 -0
+ *    is missing, clears it in the in-memory struct_ops (with a warning to
+ *    stderr) before load. Handles additive changes - a new stanza must be
+ *    added here for each new optional field.
+ *
+ * 2. Multi-variant struct_ops via compat.bpf.h::SCX_OPS_DEFINE(). That
+ *    macro can be expanded to emit several variants of struct sched_ext_ops,
+ *    and SCX_OPS_LOAD()/ATTACH() can pick the right one based on what the
+ *    kernel supports. Needed when an existing operation has to change
+ *    incompatibly (e.g. a callback signature changes); the load-time
+ *    fix-up above only handles purely additive changes.

This is very clear now, thank u! ^0^

Quoted 11 lines
tools/sched_ext/include/scx/compat.h +1 -0
  *
  * ec7e3b0463e1 ("implement-ops") in https://github.com/sched-ext/sched_ext is
  * the current minimum required kernel version.
@@ -225,6 +239,7 @@ static inline void __scx_ops_assoc_prog(
 }
 #endif
 
+/* See SCX_OPS_OPEN() above for backward-compatibility handling. */
 #define SCX_OPS_LOAD(__skel, __ops_name, __scx_name, __uei_name) ({		\
 	struct bpf_program *__prog;						\
 	UEI_SET_SIZE(__skel, __ops_name, __uei_name);				\
Signature · 2 lines
Cheers,
Cheng-Yang