sched_ext: Document the ops compat strategy in compat.h/compat.bpf.h
Tejun Heo
reviewerPATCH 1/1 sched_ext: Document the ops compat strategy in compat.h/compat.bpf.h
The comments around SCX_OPS_DEFINE() and SCX_OPS_OPEN() were vague about how backward compatibility actually works. Expand them to describe the two
mechanisms: load-time BTF fix-up for additive changes, and multi-variant struct_ops for incompatible ones.
--- tools/sched_ext/include/scx/compat.bpf.h | 6 ++++-- tools/sched_ext/include/scx/compat.h | 23 +++++++++++++++++++---- 2 files changed, 23 insertions(+), 6 deletions(-)
tools/sched_ext/include/scx/compat.bpf.h +4 -2
@@ -423,8 +423,10 @@ static inline void scx_bpf_dsq_reenq(u64 }/* -* Define sched_ext_ops. This may be expanded to define multiple variants for -* backward compatibility. See compat.h::SCX_OPS_LOAD/ATTACH(). +* Define sched_ext_ops. See compat.h::SCX_OPS_OPEN() for how backward +* compatibility is handled (this macro can be expanded to emit multiple +* variants for incompatible op changes; SCX_OPS_OPEN() handles purely +* additive changes at load time). */ #define SCX_OPS_DEFINE (__name, ...) \ SEC(".struct_ops.link") \
tools/sched_ext/include/scx/compat.h +19 -4
@@ -149,10 +149,24 @@ static inline long scx_hotplug_seq(void) }/* -* struct sched_ext_ops can change over time. If compat.bpf.h::SCX_OPS_DEFINE() -* is used to define ops and compat.h::SCX_OPS_LOAD/ATTACH() are used to load -* and attach it, backward compatibility is automatically maintained where -* reasonable. +* Open the sched_ext_ops skeleton. +* +* struct sched_ext_ops can change over time. Two complementary mechanisms +* keep BPF schedulers built against newer headers running on older kernels: +* +* 1. Load-time fix-up (this macro). For each optional ops callback or field +* added to struct sched_ext_ops, an explicit stanza below probes the +* running kernel's BTF via __COMPAT_struct_has_field() and, if the field +* is missing, clears it in the in-memory struct_ops (with a warning to +* stderr) before load. Handles additive changes - a new stanza must be +* added here for each new optional field. +* +* 2. Multi-variant struct_ops via compat.bpf.h::SCX_OPS_DEFINE(). That +* macro can be expanded to emit several variants of struct sched_ext_ops, +* and SCX_OPS_LOAD()/ATTACH() can pick the right one based on what the +* kernel supports. Needed when an existing operation has to change +* incompatibly (e.g. a callback signature changes); the load-time +* fix-up above only handles purely additive changes. * * ec7e3b0463e1 ("implement-ops") in https://github.com/sched-ext/sched_ext is * the current minimum required kernel version.
@@ -225,6 +239,7 @@ static inline void __scx_ops_assoc_prog( }#endif +/* See SCX_OPS_OPEN() above for backward-compatibility handling. */ #define SCX_OPS_LOAD (__skel, __ops_name, __scx_name, __uei_name) ({ \ struct bpf_program *__prog; \ UEI_SET_SIZE(__skel, __ops_name, __uei_name); \
Hi Tejun,
Quoted 4 lines · Tejun Heo
The comments around
SCX_OPS_DEFINE()andSCX_OPS_OPEN()were vague about how backward compatibility actually works. Expand them to describe the two
mechanisms: load-time BTF fix-up for additive changes, and multi-variantstruct_opsfor incompatible ones.
Thanks for clarifying these comments.
-Andrea
Quoted 61 lines
Signed-off-by Tejun Heo <[email protected]>--- tools/sched_ext/include/scx/compat.bpf.h | 6 ++++-- tools/sched_ext/include/scx/compat.h | 23 +++++++++++++++++++---- 2 files changed, 23 insertions(+), 6 deletions(-)tools/sched_ext/include/scx/compat.bpf.h +4
-2@@ -423,8 +423,10 @@ static inline void scx_bpf_dsq_reenq(u64 }/* -* Define sched_ext_ops. This may be expanded to define multiple variants for -* backward compatibility. See compat.h::SCX_OPS_LOAD/ATTACH(). +* Define sched_ext_ops. See compat.h::SCX_OPS_OPEN() for how backward +* compatibility is handled (this macro can be expanded to emit multiple +* variants for incompatible op changes; SCX_OPS_OPEN() handles purely +* additive changes at load time). */ #define SCX_OPS_DEFINE (__name, ...) \ SEC(".struct_ops.link") \tools/sched_ext/include/scx/compat.h +19
-4@@ -149,10 +149,24 @@ static inline long scx_hotplug_seq(void) }/* -* struct sched_ext_ops can change over time. If compat.bpf.h::SCX_OPS_DEFINE() -* is used to define ops and compat.h::SCX_OPS_LOAD/ATTACH() are used to load -* and attach it, backward compatibility is automatically maintained where -* reasonable. +* Open the sched_ext_ops skeleton. +* +* struct sched_ext_ops can change over time. Two complementary mechanisms +* keep BPF schedulers built against newer headers running on older kernels: +* +* 1. Load-time fix-up (this macro). For each optional ops callback or field +* added to struct sched_ext_ops, an explicit stanza below probes the +* running kernel's BTF via __COMPAT_struct_has_field() and, if the field +* is missing, clears it in the in-memory struct_ops (with a warning to +* stderr) before load. Handles additive changes - a new stanza must be +* added here for each new optional field. +* +* 2. Multi-variant struct_ops via compat.bpf.h::SCX_OPS_DEFINE(). That +* macro can be expanded to emit several variants of struct sched_ext_ops, +* and SCX_OPS_LOAD()/ATTACH() can pick the right one based on what the +* kernel supports. Needed when an existing operation has to change +* incompatibly (e.g. a callback signature changes); the load-time +* fix-up above only handles purely additive changes. * * ec7e3b0463e1 ("implement-ops") in https://github.com/sched-ext/sched_ext is * the current minimum required kernel version. @@ -225,6 +239,7 @@ static inline void __scx_ops_assoc_prog( }#endif +/* See SCX_OPS_OPEN() above for backward-compatibility handling. */ #define SCX_OPS_LOAD (__skel, __ops_name, __scx_name, __uei_name) ({ \ struct bpf_program *__prog; \ UEI_SET_SIZE(__skel, __ops_name, __uei_name); \
Hi Tejun,
Quoted 4 lines · Tejun Heo
The comments around
SCX_OPS_DEFINE()andSCX_OPS_OPEN()were vague about how backward compatibility actually works. Expand them to describe the two
mechanisms: load-time BTF fix-up for additive changes, and multi-variantstruct_opsfor incompatible ones.
Thanks for improving this. It is much clearer now.
Quoted 40 lines
Signed-off-by Tejun Heo <[email protected]>--- tools/sched_ext/include/scx/compat.bpf.h | 6 ++++-- tools/sched_ext/include/scx/compat.h | 23 +++++++++++++++++++---- 2 files changed, 23 insertions(+), 6 deletions(-)tools/sched_ext/include/scx/compat.bpf.h +4
-2@@ -423,8 +423,10 @@ static inline void scx_bpf_dsq_reenq(u64 }/* -* Define sched_ext_ops. This may be expanded to define multiple variants for -* backward compatibility. See compat.h::SCX_OPS_LOAD/ATTACH(). +* Define sched_ext_ops. See compat.h::SCX_OPS_OPEN() for how backward +* compatibility is handled (this macro can be expanded to emit multiple +* variants for incompatible op changes; SCX_OPS_OPEN() handles purely +* additive changes at load time). */ #define SCX_OPS_DEFINE (__name, ...) \ SEC(".struct_ops.link") \tools/sched_ext/include/scx/compat.h +8
-4@@ -149,10 +149,24 @@ static inline long scx_hotplug_seq(void) }/* -* struct sched_ext_ops can change over time. If compat.bpf.h::SCX_OPS_DEFINE() -* is used to define ops and compat.h::SCX_OPS_LOAD/ATTACH() are used to load -* and attach it, backward compatibility is automatically maintained where -* reasonable. +* Open the sched_ext_ops skeleton. +* +* struct sched_ext_ops can change over time. Two complementary mechanisms +* keep BPF schedulers built against newer headers running on older kernels: +* +* 1. Load-time fix-up (this macro). For each optional ops callback or field +* added to struct sched_ext_ops, an explicit stanza below probes the +* running kernel's BTF via __COMPAT_struct_has_field() and, if the field
tools/sched_ext/include/scx/compat.h static inline long scx_hotplug_seq(void) · line 169
Quoted 10 lines
tools/sched_ext/include/scx/compat.h +10
-0+* is missing, clears it in the in-memory struct_ops (with a warning to +* stderr) before load. Handles additive changes - a new stanza must be +* added here for each new optional field. +* +* 2. Multi-variant struct_ops via compat.bpf.h::SCX_OPS_DEFINE(). That +* macro can be expanded to emit several variants of struct sched_ext_ops, +* and SCX_OPS_LOAD()/ATTACH() can pick the right one based on what the +* kernel supports. Needed when an existing operation has to change +* incompatibly (e.g. a callback signature changes); the load-time +* fix-up above only handles purely additive changes.
This is very clear now, thank u! ^0^
Quoted 11 lines
tools/sched_ext/include/scx/compat.h +1
-0* * ec7e3b0463e1 ("implement-ops") in https://github.com/sched-ext/sched_ext is * the current minimum required kernel version. @@ -225,6 +239,7 @@ static inline void __scx_ops_assoc_prog( }#endif +/* See SCX_OPS_OPEN() above for backward-compatibility handling. */ #define SCX_OPS_LOAD (__skel, __ops_name, __scx_name, __uei_name) ({ \ struct bpf_program *__prog; \ UEI_SET_SIZE(__skel, __ops_name, __uei_name); \
Signature · 2 lines
Cheers, Cheng-Yang
Hello,
Applied to sched_ext/for-7.2.
Thanks.
Signature · 1 line
tejun