net_sched: Introduce eBPF based Qdisc
Subsystems (7)
v7 cover net_sched: Introduce eBPF based Qdisc
Hi,
I am continuing the work of ebpf-based Qdisc based on Cong’s previous RFC. The followings are some use cases of eBPF Qdisc:
1. Allow customizing Qdiscs in an easier way. So that people don't have to write a complete Qdisc kernel module just to experiment some new queuing theory.
2. Solve EDT's problem. EDT calcuates the "tokens" in clsact which is before enqueue, it is impossible to adjust those "tokens" after packets get dropped in enqueue. With eBPF Qdisc, it is easy to be solved with a shared map between clsact and sch_bpf.
3. Replace qevents, as now the user gains much more control over the skb and queues.
4. Provide a new way to reuse TC filters. Currently TC relies on filter chain and block to reuse the TC filters, but they are too complicated to understand. With eBPF helper bpf_skb_tc_classify(), we can invoke TC filters on _any_ Qdisc (even on a different netdev) to do the classification.
5. Potentially pave a way for ingress to queue packets, although current implementation is still only for egress.
I’ve combed through previous comments and appreciated the feedbacks. Some major changes in this RFC is the use of kptr to skb to maintain the validility of skb during its lifetime in the Qdisc, dropping rbtree maps, and the inclusion of two examples.
Some questions for discussion:
1. We now pass a trusted kptr of sk_buff to the program instead of __sk_buff. This makes most helpers using __sk_buff incompatible with eBPF qdisc. An alternative is to still use __sk_buff in the context and use bpf_cast_to_kern_ctx() to acquire the kptr. However, this can only be applied to enqueue program, since in dequeue program skbs do not come from ctx but kptrs exchanged out of maps (i.e., there is no __sk_buff). Any suggestion for making skb kptr and helper functions compatible?
2. The current patchset uses netlink. Do we also want to use bpf_link for attachment?
3. People have suggested struct_ops. We chose not to use struct_ops since users might want to create multiple bpf qdiscs with different implementations. Current struct_ops attachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong.
Todo:
- Add selftest
- Make bpf list/rbtree use list/rbnode in skb so that developers don't need to allocate bpf objects for storing skb kptrs.
Note:
- This patchset requires bpf support of exchanging kptr into allocated objects (local kptr), which Dave Marchevsky is working on.
- The user space programs in the sample are adapted from the example Peihao Yang written in RFC v5 thread.
---
v7: Reference skb using kptr to sk_buff instead of __sk_buff
Use the new bpf rbtree/link to for skb queues Add reset and init programs
Add a bpf fq qdisc sample
Add a bpf netem qdisc sample
v6: switch to kptr based approach
v5: mv kernel/bpf/skb_map.c net/core/skb_map.c
implement flow map as map-in-map
rename bpf_skb_tc_classify() and move it to net/sched/cls_api.c clean up eBPF qdisc program context
v4: get rid of PIFO, use rbtree directly
v3: move priority queue from sch_bpf to skb map
introduce skb map and its helpers
introduce bpf_skb_classify()
use netdevice notifier to reset skb's
Rebase on latest bpf-next
v2: Rebase on latest net-next
Make the code more complete (but still incomplete)
Amery Hung (5):
net_sched: Add reset program
net_sched: Add init program
tools/libbpf: Add support for BPF_PROG_TYPE_QDISC samples/bpf: Add an example of bpf fq qdisc samples/bpf: Add an example of bpf netem qdisc
Cong Wang (3):
net_sched: Introduce eBPF based Qdisc
net_sched: Add kfuncs for working with skb
net_sched: Introduce kfunc bpf_skb_tc_classify()
include/linux/bpf_types.h | 4 + include/uapi/linux/bpf.h | 23 + include/uapi/linux/pkt_sched.h | 24 ++ kernel/bpf/btf.c | 5 + kernel/bpf/helpers.c | 1 + kernel/bpf/syscall.c | 10 + net/core/filter.c | 100 +++++ net/sched/Kconfig | 15 + net/sched/Makefile | 1 + net/sched/sch_bpf.c | 729 ++++++++++++++++++++++++++++++++ samples/bpf/Makefile | 14 +- samples/bpf/bpf_experimental.h | 134 ++++++ samples/bpf/tc_clsact_edt.bpf.c | 103 +++++ samples/bpf/tc_sch_fq.bpf.c | 666 +++++++++++++++++++++++++++++ samples/bpf/tc_sch_fq.c | 321 ++++++++++++++ samples/bpf/tc_sch_netem.bpf.c | 256 +++++++++++ samples/bpf/tc_sch_netem.c | 347 +++++++++++++++ tools/include/uapi/linux/bpf.h | 23 + tools/lib/bpf/libbpf.c | 4 + 19 files changed, 2779 insertions(+), 1 deletion(-) create mode 100644 net/sched/sch_bpf.c create mode 100644 samples/bpf/bpf_experimental.h create mode 100644 samples/bpf/tc_clsact_edt.bpf.c create mode 100644 samples/bpf/tc_sch_fq.bpf.c create mode 100644 samples/bpf/tc_sch_fq.c create mode 100644 samples/bpf/tc_sch_netem.bpf.c create mode 100644 samples/bpf/tc_sch_netem.c
Signature · 1 line
2.20.1
RFC PATCH v7 1/8 net_sched: Introduce eBPF based Qdisc
From: Cong Wang <[email protected]>
Introduce a new Qdisc which is completely managed by eBPF program of type BPF_PROG_TYPE_QDISC. It accepts two eBPF programs of the same type, but one for enqueue and the other for dequeue.
It interacts with Qdisc layer in two ways:
1) It relies on Qdisc watchdog to handle throttling;
2) It could pass the skb enqueue/dequeue down to child classes
The context is used differently for enqueue and dequeue, as shown below:
┌──────────┬───────────────┬──────────────────────────────────┐ │ prog │ input │ output │ ├──────────┼───────────────┼──────────────────────────────────┤ │ │ ctx->skb │ SCH_BPF_THROTTLE: ctx->expire │ │ │ │ ctx->delta_ns │ │ │ ctx->classid │ │ │ │ │ SCH_BPF_QUEUED: None │ │ │ │ │ │ │ │ SCH_BPF_BYPASS: None │ │ enqueue │ │ │ │ │ │ SCH_BPF_STOLEN: None │ │ │ │ │ │ │ │ SCH_BPF_DROP: None │ │ │ │ │ │ │ │ SCH_BPF_CN: None │ │ │ │ │ │ │ │ SCH_BPF_PASS: ctx->classid │ ├──────────┼───────────────┼──────────────────────────────────┤ │ │ ctx->classid │ SCH_BPF_THROTTLE: ctx->expire │ │ │ │ ctx->delta_ns │ │ │ │ │ │ dequeue │ │ SCH_BPF_DEQUEUED: None │ │ │ │ │ │ │ │ SCH_BPF_DROP: None │ │ │ │ │ │ │ │ SCH_BPF_PASS: ctx->classid │ └──────────┴───────────────┴──────────────────────────────────┘
--- include/linux/bpf_types.h | 4 + include/uapi/linux/bpf.h | 21 ++ include/uapi/linux/pkt_sched.h | 16 + kernel/bpf/btf.c | 5 + kernel/bpf/helpers.c | 1 + kernel/bpf/syscall.c | 8 + net/core/filter.c | 96 ++++++ net/sched/Kconfig | 15 + net/sched/Makefile | 1 + net/sched/sch_bpf.c | 537 +++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 21 ++ 11 files changed, 725 insertions(+) create mode 100644 net/sched/sch_bpf.c
include/linux/bpf_types.h +4 -0
@@ -83,6 +83,10 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_SYSCALL, bpf_syscall,BPF_PROG_TYPE (BPF_PROG_TYPE_NETFILTER , netfilter, struct bpf_nf_ctx, struct bpf_nf_ctx)#endif +#ifdef CONFIG_NET +BPF_PROG_TYPE (BPF_PROG_TYPE_QDISC , tc_qdisc, + struct bpf_qdisc_ctx, struct bpf_qdisc_ctx) +#endif BPF_MAP_TYPE(BPF_MAP_TYPE_ARRAY, array_map_ops)BPF_MAP_TYPE (BPF_MAP_TYPE_PERCPU_ARRAY , percpu_array_map_ops)
include/uapi/linux/bpf.h +21 -0
@@ -997,6 +997,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_SK_LOOKUP ,BPF_PROG_TYPE_SYSCALL ,/* a program that can execute syscalls */ BPF_PROG_TYPE_NETFILTER , + BPF_PROG_TYPE_QDISC, }; enum bpf_attach_type {
@@ -1056,6 +1057,8 @@ enum bpf_attach_type {BPF_CGROUP_UNIX_GETSOCKNAME ,BPF_NETKIT_PRIMARY ,BPF_NETKIT_PEER , +BPF_QDISC_ENQUEUE , +BPF_QDISC_DEQUEUE , __MAX_BPF_ATTACH_TYPE };
@@ -7357,4 +7360,22 @@ struct bpf_iter_num {__u64 __opaque[1 ]; } __attribute__((aligned(8))); +struct bpf_qdisc_ctx { + __bpf_md_ptr(struct sk_buff *, skb); +__u32 classid; +__u64 expire; +__u64 delta_ns; +}; + +enum { +SCH_BPF_QUEUED , +SCH_BPF_DEQUEUED =SCH_BPF_QUEUED , +SCH_BPF_DROP , +SCH_BPF_CN , +SCH_BPF_THROTTLE , +SCH_BPF_PASS , +SCH_BPF_BYPASS , +SCH_BPF_STOLEN , +}; +#endif /* _UAPI__LINUX_BPF_H__ */
include/uapi/linux/pkt_sched.h +16 -0
@@ -1317,4 +1317,20 @@ enum {#define TCA_ETS_MAX (__TCA_ETS_MAX - 1) +#define TCA_SCH_BPF_FLAG_DIRECT _BITUL(0) +enum { +TCA_SCH_BPF_UNSPEC , +TCA_SCH_BPF_ENQUEUE_PROG_NAME ,/* string */ +TCA_SCH_BPF_ENQUEUE_PROG_FD ,/* u32 */ +TCA_SCH_BPF_ENQUEUE_PROG_ID ,/* u32 */ +TCA_SCH_BPF_ENQUEUE_PROG_TAG ,/* data */ +TCA_SCH_BPF_DEQUEUE_PROG_NAME ,/* string */ +TCA_SCH_BPF_DEQUEUE_PROG_FD ,/* u32 */ +TCA_SCH_BPF_DEQUEUE_PROG_ID ,/* u32 */ +TCA_SCH_BPF_DEQUEUE_PROG_TAG ,/* data */ + __TCA_SCH_BPF_MAX, +}; + +#define TCA_SCH_BPF_MAX (__TCA_SCH_BPF_MAX - 1) +#endif
kernel/bpf/btf.c +5 -0
@@ -217,6 +217,7 @@ enum btf_kfunc_hook {BTF_KFUNC_HOOK_SOCKET_FILTER ,BTF_KFUNC_HOOK_LWT ,BTF_KFUNC_HOOK_NETFILTER , +BTF_KFUNC_HOOK_QDISC , BTF_KFUNC_HOOK_MAX, };
@@ -5928,6 +5929,8 @@ static bool prog_args_trusted(const struct bpf_prog *prog)return bpf_lsm_is_trusted (prog);case BPF_PROG_TYPE_STRUCT_OPS :return true; +case BPF_PROG_TYPE_QDISC : +return true;default :return false; }
@@ -7865,6 +7868,8 @@ static int bpf_prog_type_to_kfunc_hook(enum bpf_prog_type prog_type)return BTF_KFUNC_HOOK_LWT ;case BPF_PROG_TYPE_NETFILTER :return BTF_KFUNC_HOOK_NETFILTER ; +case BPF_PROG_TYPE_QDISC : +return BTF_KFUNC_HOOK_QDISC ;default :return BTF_KFUNC_HOOK_MAX ; }
kernel/bpf/helpers.c +1 -0
@@ -2610,6 +2610,7 @@ static int __init kfunc_init(void) ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &generic_kfunc_set); ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_SCHED_CLS , &generic_kfunc_set); + ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_QDISC , &generic_kfunc_set); ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &generic_kfunc_set); ret = ret ?: register_btf_id_dtor_kfuncs(generic_dtors, ARRAY_SIZE(generic_dtors),
kernel/bpf/syscall.c +8 -0
@@ -2502,6 +2502,14 @@ bpf_prog_load_check_attach(enum bpf_prog_type prog_type,if (expected_attach_type ==BPF_NETFILTER )return 0 ;return -EINVAL ; +case BPF_PROG_TYPE_QDISC : +switch (expected_attach_type) { +case BPF_QDISC_ENQUEUE : +case BPF_QDISC_DEQUEUE : +return 0 ; +default : +return -EINVAL ; + }case BPF_PROG_TYPE_SYSCALL :case BPF_PROG_TYPE_EXT :if (expected_attach_type)
net/core/filter.c +96 -0
108 lines not on this page · raw message
net/sched/Kconfig +15 -0
@@ -403,6 +403,21 @@ config NET_SCH_ETS If unsure, say N. +config NET_SCH_BPF + tristate "eBPF based programmable queue discipline" + help + This eBPF based queue discipline offers a way to program your + own packet scheduling algorithm. This is a classful qdisc which + also allows you to decide the hierarchy. + + Say Y here if you want to use the eBPF based programmable queue + discipline. + + To compile this driver as a module, choose M here: the module + will be called sch_bpf. + + If unsure, say N. + menuconfig NET_SCH_DEFAULT bool "Allow override default queue discipline" help
net/sched/Makefile +1 -0
@@ -63,6 +63,7 @@ obj-$(CONFIG_NET_SCH_FQ_PIE) += sch_fq_pie.o obj-$(CONFIG_NET_SCH_CBS) += sch_cbs.o obj-$(CONFIG_NET_SCH_ETF) += sch_etf.o obj-$(CONFIG_NET_SCH_TAPRIO) += sch_taprio.o +obj-$(CONFIG_NET_SCH_BPF) += sch_bpf.o obj-$(CONFIG_NET_CLS_U32) += cls_u32.o obj-$(CONFIG_NET_CLS_ROUTE4) += cls_route.o
net/sched/sch_bpf.c new file +537 -0
537 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +21 -0
@@ -997,6 +997,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_SK_LOOKUP ,BPF_PROG_TYPE_SYSCALL ,/* a program that can execute syscalls */ BPF_PROG_TYPE_NETFILTER , + BPF_PROG_TYPE_QDISC, }; enum bpf_attach_type {
@@ -1056,6 +1057,8 @@ enum bpf_attach_type {BPF_CGROUP_UNIX_GETSOCKNAME ,BPF_NETKIT_PRIMARY ,BPF_NETKIT_PEER , +BPF_QDISC_ENQUEUE , +BPF_QDISC_DEQUEUE , __MAX_BPF_ATTACH_TYPE };
@@ -7357,4 +7360,22 @@ struct bpf_iter_num {__u64 __opaque[1 ]; } __attribute__((aligned(8))); +struct bpf_qdisc_ctx { + __bpf_md_ptr(struct sk_buff *, skb); +__u32 classid; +__u64 expire; +__u64 delta_ns; +}; + +enum { +SCH_BPF_QUEUED , +SCH_BPF_DEQUEUED =SCH_BPF_QUEUED , +SCH_BPF_DROP , +SCH_BPF_CN , +SCH_BPF_THROTTLE , +SCH_BPF_PASS , +SCH_BPF_BYPASS , +SCH_BPF_STOLEN , +}; +#endif /* _UAPI__LINUX_BPF_H__ */
Signature · 1 line
2.20.1
kernel test robot
build botkernel test robot noticed the following build warnings:
Hi Amery,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:
[auto build test WARNING on mst-vhost/linux-next]
[also build test WARNING on v6.7]
[cannot apply to bpf-next/master bpf/master net/main net-next/main linus/master horms-ipvs/master next-20240119] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Amery-Hung/net_sched-Introduce-eBPF-based-Qdisc/20240118-055917 base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next patch link: https://lore.kernel.org/r/232881645a5c4c05a35df4ff1f08a19ef9a02662.1705432850.git.amery.hung%40bytedance.com patch subject: [RFC PATCH v7 1/8] net_sched: Introduce eBPF based Qdisc config: i386-randconfig-013-20240120 (https://download.01.org/0day-ci/archive/20240120/[email protected]/config) compiler: gcc-9 (Debian 9.3.0-22) 9.3.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240120/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
net/core/filter.c: In function 'tc_qdisc_is_valid_access':
net/core/filter.c:8929:9: error: implicit declaration of function 'bpf_get_btf_vmlinux' [-Werror=implicit-function-declaration]
8929 | btf = bpf_get_btf_vmlinux();
| ^~~~~~~~~~~~~~~~~~~
>> net/core/filter.c:8929:7: warning: assignment to 'struct btf *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
8929 | btf = bpf_get_btf_vmlinux();
| ^cc1: some warnings being treated as errors
vim +8929 net/core/filter.c
8909
8910 static bool tc_qdisc_is_valid_access(int off, int size,
8911 enum bpf_access_type type,
8912 const struct bpf_prog *prog,
8913 struct bpf_insn_access_aux *info)
8914 {
8915 struct btf *btf;
8916
8917 if (off < 0 || off >= sizeof(struct bpf_qdisc_ctx))
8918 return false;
8919
8920 switch (off) {
8921 case offsetof(struct bpf_qdisc_ctx, skb):
8922 if (type == BPF_WRITE ||
8923 size != sizeof_field(struct bpf_qdisc_ctx, skb))
8924 return false;
8925
8926 if (prog->expected_attach_type != BPF_QDISC_ENQUEUE)
8927 return false;
8928
> 8929 btf = bpf_get_btf_vmlinux(); 8930 if (IS_ERR_OR_NULL(btf))
8931 return false;
8932
8933 info->btf = btf;
8934 info->btf_id = tc_qdisc_ctx_access_btf_ids[0];
8935 info->reg_type = PTR_TO_BTF_ID | PTR_TRUSTED;
8936 return true;
8937 case bpf_ctx_range(struct bpf_qdisc_ctx, classid):
8938 return size == sizeof_field(struct bpf_qdisc_ctx, classid);
8939 case bpf_ctx_range(struct bpf_qdisc_ctx, expire):
8940 return size == sizeof_field(struct bpf_qdisc_ctx, expire);
8941 case bpf_ctx_range(struct bpf_qdisc_ctx, delta_ns):
8942 return size == sizeof_field(struct bpf_qdisc_ctx, delta_ns);
8943 default:
8944 return false;
8945 }
8946
8947 return false;
8948 }
8949
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botkernel test robot noticed the following build errors:
Hi Amery,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:
[auto build test ERROR on mst-vhost/linux-next]
[also build test ERROR on v6.7]
[cannot apply to bpf-next/master bpf/master net/main net-next/main linus/master horms-ipvs/master next-20240119] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Amery-Hung/net_sched-Introduce-eBPF-based-Qdisc/20240118-055917 base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next patch link: https://lore.kernel.org/r/232881645a5c4c05a35df4ff1f08a19ef9a02662.1705432850.git.amery.hung%40bytedance.com patch subject: [RFC PATCH v7 1/8] net_sched: Introduce eBPF based Qdisc config: i386-buildonly-randconfig-002-20240120 (https://download.01.org/0day-ci/archive/20240120/[email protected]/config) compiler: ClangBuiltLinux clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240120/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> net/core/filter.c:8929:9: error: call to undeclared function 'bpf_get_btf_vmlinux'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
8929 | btf = bpf_get_btf_vmlinux();
| ^
>> net/core/filter.c:8929:7: error: incompatible integer to pointer conversion assigning to 'struct btf *' from 'int' [-Wint-conversion]
8929 | btf = bpf_get_btf_vmlinux();
| ^ ~~~~~~~~~~~~~~~~~~~~~2 errors generated.
vim +/bpf_get_btf_vmlinux +8929 net/core/filter.c
8909
8910 static bool tc_qdisc_is_valid_access(int off, int size,
8911 enum bpf_access_type type,
8912 const struct bpf_prog *prog,
8913 struct bpf_insn_access_aux *info)
8914 {
8915 struct btf *btf;
8916
8917 if (off < 0 || off >= sizeof(struct bpf_qdisc_ctx))
8918 return false;
8919
8920 switch (off) {
8921 case offsetof(struct bpf_qdisc_ctx, skb):
8922 if (type == BPF_WRITE ||
8923 size != sizeof_field(struct bpf_qdisc_ctx, skb))
8924 return false;
8925
8926 if (prog->expected_attach_type != BPF_QDISC_ENQUEUE)
8927 return false;
8928
> 8929 btf = bpf_get_btf_vmlinux(); 8930 if (IS_ERR_OR_NULL(btf))
8931 return false;
8932
8933 info->btf = btf;
8934 info->btf_id = tc_qdisc_ctx_access_btf_ids[0];
8935 info->reg_type = PTR_TO_BTF_ID | PTR_TRUSTED;
8936 return true;
8937 case bpf_ctx_range(struct bpf_qdisc_ctx, classid):
8938 return size == sizeof_field(struct bpf_qdisc_ctx, classid);
8939 case bpf_ctx_range(struct bpf_qdisc_ctx, expire):
8940 return size == sizeof_field(struct bpf_qdisc_ctx, expire);
8941 case bpf_ctx_range(struct bpf_qdisc_ctx, delta_ns):
8942 return size == sizeof_field(struct bpf_qdisc_ctx, delta_ns);
8943 default:
8944 return false;
8945 }
8946
8947 return false;
8948 }
8949
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botkernel test robot noticed the following build errors:
Hi Amery,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:
[auto build test ERROR on mst-vhost/linux-next]
[also build test ERROR on v6.7]
[cannot apply to bpf-next/master bpf/master net/main net-next/main linus/master horms-ipvs/master next-20240119] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Amery-Hung/net_sched-Introduce-eBPF-based-Qdisc/20240118-055917 base: https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git linux-next patch link: https://lore.kernel.org/r/232881645a5c4c05a35df4ff1f08a19ef9a02662.1705432850.git.amery.hung%40bytedance.com patch subject: [RFC PATCH v7 1/8] net_sched: Introduce eBPF based Qdisc config: i386-randconfig-013-20240120 (https://download.01.org/0day-ci/archive/20240120/[email protected]/config) compiler: gcc-9 (Debian 9.3.0-22) 9.3.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240120/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
net/core/filter.c: In function 'tc_qdisc_is_valid_access':
>> net/core/filter.c:8929:9: error: implicit declaration of function 'bpf_get_btf_vmlinux' [-Werror=implicit-function-declaration]
8929 | btf = bpf_get_btf_vmlinux();
| ^~~~~~~~~~~~~~~~~~~
net/core/filter.c:8929:7: warning: assignment to 'struct btf *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
8929 | btf = bpf_get_btf_vmlinux();
| ^cc1: some warnings being treated as errors
vim +/bpf_get_btf_vmlinux +8929 net/core/filter.c
8909
8910 static bool tc_qdisc_is_valid_access(int off, int size,
8911 enum bpf_access_type type,
8912 const struct bpf_prog *prog,
8913 struct bpf_insn_access_aux *info)
8914 {
8915 struct btf *btf;
8916
8917 if (off < 0 || off >= sizeof(struct bpf_qdisc_ctx))
8918 return false;
8919
8920 switch (off) {
8921 case offsetof(struct bpf_qdisc_ctx, skb):
8922 if (type == BPF_WRITE ||
8923 size != sizeof_field(struct bpf_qdisc_ctx, skb))
8924 return false;
8925
8926 if (prog->expected_attach_type != BPF_QDISC_ENQUEUE)
8927 return false;
8928
> 8929 btf = bpf_get_btf_vmlinux(); 8930 if (IS_ERR_OR_NULL(btf))
8931 return false;
8932
8933 info->btf = btf;
8934 info->btf_id = tc_qdisc_ctx_access_btf_ids[0];
8935 info->reg_type = PTR_TO_BTF_ID | PTR_TRUSTED;
8936 return true;
8937 case bpf_ctx_range(struct bpf_qdisc_ctx, classid):
8938 return size == sizeof_field(struct bpf_qdisc_ctx, classid);
8939 case bpf_ctx_range(struct bpf_qdisc_ctx, expire):
8940 return size == sizeof_field(struct bpf_qdisc_ctx, expire);
8941 case bpf_ctx_range(struct bpf_qdisc_ctx, delta_ns):
8942 return size == sizeof_field(struct bpf_qdisc_ctx, delta_ns);
8943 default:
8944 return false;
8945 }
8946
8947 return false;
8948 }
8949
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Martin KaFai Lau
reviewerQuoted 44 lines · Amery Hung
include/uapi/linux/bpf.h +21
-0@@ -997,6 +997,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_SK_LOOKUP ,BPF_PROG_TYPE_SYSCALL ,/* a program that can execute syscalls */ BPF_PROG_TYPE_NETFILTER , + BPF_PROG_TYPE_QDISC, }; enum bpf_attach_type {@@ -1056,6 +1057,8 @@ enum bpf_attach_type {BPF_CGROUP_UNIX_GETSOCKNAME ,BPF_NETKIT_PRIMARY ,BPF_NETKIT_PEER , +BPF_QDISC_ENQUEUE , +BPF_QDISC_DEQUEUE , __MAX_BPF_ATTACH_TYPE };@@ -7357,4 +7360,22 @@ struct bpf_iter_num {__u64 __opaque[1 ]; } __attribute__((aligned(8))); +struct bpf_qdisc_ctx { + __bpf_md_ptr(struct sk_buff *, skb); +__u32 classid; +__u64 expire; +__u64 delta_ns; +}; + +enum { +SCH_BPF_QUEUED , +SCH_BPF_DEQUEUED =SCH_BPF_QUEUED , +SCH_BPF_DROP , +SCH_BPF_CN , +SCH_BPF_THROTTLE , +SCH_BPF_PASS , +SCH_BPF_BYPASS , +SCH_BPF_STOLEN , +}; +#endif /* _UAPI__LINUX_BPF_H__ */
[ ... ]
Quoted 40 lines
include/uapi/linux/bpf.h +40
-0+static bool tc_qdisc_is_valid_access (int off,int size, +enum bpf_access_type type, +const struct bpf_prog *prog, +struct bpf_insn_access_aux *info) +{ +struct btf *btf; + +if (off <0 || off >=sizeof (struct bpf_qdisc_ctx )) +return false; + +switch (off) { +case offsetof(struct bpf_qdisc_ctx , skb): +if (type ==BPF_WRITE || + size != sizeof_field(struct bpf_qdisc_ctx, skb)) +return false; + +if (prog->expected_attach_type !=BPF_QDISC_ENQUEUE ) +return false; + + btf =bpf_get_btf_vmlinux (); +if (IS_ERR_OR_NULL (btf)) +return false; + + info->btf = btf; + info->btf_id = tc_qdisc_ctx_access_btf_ids[0 ]; + info->reg_type =PTR_TO_BTF_ID |PTR_TRUSTED ; +return true; + case bpf_ctx_range(struct bpf_qdisc_ctx, classid): + return size == sizeof_field(struct bpf_qdisc_ctx, classid); + case bpf_ctx_range(struct bpf_qdisc_ctx, expire): + return size == sizeof_field(struct bpf_qdisc_ctx, expire); + case bpf_ctx_range(struct bpf_qdisc_ctx, delta_ns): + return size == sizeof_field(struct bpf_qdisc_ctx, delta_ns); +default : +return false; + } + +return false; +} +
[ ... ]
Quoted 105 lines
include/uapi/linux/bpf.h +105
-0+static int sch_bpf_enqueue (struct sk_buff *skb,struct Qdisc *sch, +struct sk_buff **to_free) +{ +struct bpf_sched_data *q =qdisc_priv (sch); +unsigned int len =qdisc_pkt_len (skb); +struct bpf_qdisc_ctx ctx = {}; +int res =NET_XMIT_SUCCESS ; +struct sch_bpf_class *cl; +struct bpf_prog *enqueue; + + enqueue =rcu_dereference (q->enqueue_prog.prog); +if (!enqueue) +return NET_XMIT_DROP ; + + ctx.skb = skb; + ctx.classid = sch->handle; + res =bpf_prog_run (enqueue, &ctx); +switch (res) { +case SCH_BPF_THROTTLE : +qdisc_watchdog_schedule_range_ns (&q->watchdog, ctx.expire, ctx.delta_ns); +qdisc_qstats_overlimit (sch); + fallthrough; +case SCH_BPF_QUEUED : +qdisc_qstats_backlog_inc (sch, skb); +return NET_XMIT_SUCCESS ; +case SCH_BPF_BYPASS : +qdisc_qstats_drop (sch); +__qdisc_drop (skb, to_free); +return NET_XMIT_SUCCESS | __NET_XMIT_BYPASS; +case SCH_BPF_STOLEN : +__qdisc_drop (skb, to_free); +return NET_XMIT_SUCCESS | __NET_XMIT_STOLEN; +case SCH_BPF_CN : +return NET_XMIT_CN ; +case SCH_BPF_PASS : +break ; +default : +return qdisc_drop (skb, sch, to_free); + } + + cl =sch_bpf_find (sch, ctx.classid); +if (!cl || !cl->qdisc) +return qdisc_drop (skb, sch, to_free); + + res =qdisc_enqueue (skb, cl->qdisc, to_free); +if (res !=NET_XMIT_SUCCESS ) { +if (net_xmit_drop_count (res)) { +qdisc_qstats_drop (sch); + cl->drops++; + } +return res; + } + + sch->qstats.backlog += len; + sch->q.qlen++; +return res; +} + +DEFINE_PER_CPU(struct sk_buff*, bpf_skb_dequeue); + +static struct sk_buff *sch_bpf_dequeue (struct Qdisc *sch) +{ +struct bpf_sched_data *q =qdisc_priv (sch); +struct bpf_qdisc_ctx ctx = {}; +struct sk_buff *skb =NULL ; +struct bpf_prog *dequeue; +struct sch_bpf_class *cl; +int res; + + dequeue =rcu_dereference (q->dequeue_prog.prog); +if (!dequeue) +return NULL ; + +__this_cpu_write (bpf_skb_dequeue,NULL ); + ctx.classid = sch->handle; + res =bpf_prog_run (dequeue, &ctx); +switch (res) { +case SCH_BPF_DEQUEUED : + skb =__this_cpu_read (bpf_skb_dequeue); +qdisc_bstats_update (sch, skb); +qdisc_qstats_backlog_dec (sch, skb); +break ; +case SCH_BPF_THROTTLE : +qdisc_watchdog_schedule_range_ns (&q->watchdog, ctx.expire, ctx.delta_ns); +qdisc_qstats_overlimit (sch); + cl =sch_bpf_find (sch, ctx.classid); +if (cl) + cl->overlimits++; +return NULL ; +case SCH_BPF_PASS : + cl =sch_bpf_find (sch, ctx.classid); +if (!cl || !cl->qdisc) +return NULL ; + skb =qdisc_dequeue_peeked (cl->qdisc); +if (skb) { +bstats_update (&cl->bstats, skb); +qdisc_bstats_update (sch, skb); +qdisc_qstats_backlog_dec (sch, skb); + sch->q.qlen--; + } +break ; + } + +return skb; +}
[ ... ]
Quoted 36 lines
include/uapi/linux/bpf.h +36
-0+static int sch_bpf_init (struct Qdisc *sch,struct nlattr *opt, +struct netlink_ext_ack *extack) +{ +struct bpf_sched_data *q =qdisc_priv (sch); +int err; + +qdisc_watchdog_init (&q->watchdog, sch); +if (opt) { + err =sch_bpf_change (sch, opt, extack); +if (err) +return err; + } + + err =tcf_block_get (&q->block, &q->filter_list, sch, extack); +if (err) +return err; + +return qdisc_class_hash_init (&q->clhash); +} + +static void sch_bpf_reset (struct Qdisc *sch) +{ +struct bpf_sched_data *q =qdisc_priv (sch); +struct sch_bpf_class *cl; +unsigned int i; + +for (i =0 ; i < q->clhash.hashsize; i++) { +hlist_for_each_entry (cl, &q->clhash.hash[i], common.hnode) { +if (cl->qdisc) +qdisc_reset (cl->qdisc); + } + } + +qdisc_watchdog_cancel (&q->watchdog); +} +
[ ... ]
Quoted 10 lines
include/uapi/linux/bpf.h +10
-0+static const struct Qdisc_class_ops sch_bpf_class_ops = { + .graft = sch_bpf_graft, + .leaf = sch_bpf_leaf, + .find = sch_bpf_search, + .change = sch_bpf_change_class, + .delete = sch_bpf_delete, + .tcf_block = sch_bpf_tcf_block, + .bind_tcf = sch_bpf_bind, + .unbind_tcf = sch_bpf_unbind, + .dump = sch_bpf_dump_class,
include/uapi/linux/bpf.h
Quoted 10 lines
include/uapi/linux/bpf.h +10
-0+ .dump_stats = sch_bpf_dump_class_stats, + .walk = sch_bpf_walk, +}; + +static struct Qdisc_ops sch_bpf_qdisc_ops__read_mostly = { + .cl_ops = &sch_bpf_class_ops, + .id ="bpf" , + .priv_size =sizeof (struct bpf_sched_data ), + .enqueue = sch_bpf_enqueue, + .dequeue = sch_bpf_dequeue,
I looked at the high level of the patchset. The major ops that it wants to be programmable in bpf is the ".enqueue" and ".dequeue" (+ ".init" and ".reset" in patch 4 and patch 5).
This patch adds a new prog type BPF_PROG_TYPE_QDISC, four attach types (each for ".enqueue", ".dequeue", ".init", and ".reset"), and a new "bpf_qdisc_ctx" in the uapi. It is no long an acceptable way to add new bpf extension.
Can the ".enqueue", ".dequeue", ".init", and ".reset" be completely implemented in bpf (with the help of new kfuncs if needed)? Then a struct_ops for Qdisc_ops can be created. The bpf Qdisc_ops can be loaded through the existing struct_ops api.
If other ops (like ".dump", ".dump_stats"...) do not have good use case to be programmable in bpf, it can stay with the kernel implementation for now and only allows the userspace to load the a bpf Qdisc_ops with .equeue/dequeue/init/reset implemented.
You mentioned in the cover letter that:
"Current struct_ops attachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong."
I assumed you meant allow bpf to replace only "some" ops of the Qdisc_ops? Yes, it can be done through the struct_ops's ".init_member". Take a look at bpf_tcp_ca_init_member. The kernel can assign the kernel implementation for ".dump" (for example) when loading the bpf Qdisc_ops.
Quoted 18 lines
include/uapi/linux/bpf.h +18
-0+ .peek = qdisc_peek_dequeued, + .init = sch_bpf_init, + .reset = sch_bpf_reset, > + .destroy = sch_bpf_destroy, + .change = sch_bpf_change, + .dump = sch_bpf_dump, + .dump_stats = sch_bpf_dump_stats, + .owner = THIS_MODULE, +}; + +static int __init sch_bpf_mod_init (void ) +{ +return register_qdisc (&sch_bpf_qdisc_ops); +} + +static void __exit sch_bpf_mod_exit(void) +{ +unregister_qdisc (&sch_bpf_qdisc_ops); +}
Quoted 264 lines · Martin KaFai Lau
Quoted 44 lines · Amery Hung
include/uapi/linux/bpf.h +21
-0@@ -997,6 +997,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_SK_LOOKUP ,BPF_PROG_TYPE_SYSCALL ,/* a program that can execute syscalls */ BPF_PROG_TYPE_NETFILTER , + BPF_PROG_TYPE_QDISC, }; enum bpf_attach_type {@@ -1056,6 +1057,8 @@ enum bpf_attach_type {BPF_CGROUP_UNIX_GETSOCKNAME ,BPF_NETKIT_PRIMARY ,BPF_NETKIT_PEER , +BPF_QDISC_ENQUEUE , +BPF_QDISC_DEQUEUE , __MAX_BPF_ATTACH_TYPE };@@ -7357,4 +7360,22 @@ struct bpf_iter_num {__u64 __opaque[1 ]; } __attribute__((aligned(8))); +struct bpf_qdisc_ctx { + __bpf_md_ptr(struct sk_buff *, skb); +__u32 classid; +__u64 expire; +__u64 delta_ns; +}; + +enum { +SCH_BPF_QUEUED , +SCH_BPF_DEQUEUED =SCH_BPF_QUEUED , +SCH_BPF_DROP , +SCH_BPF_CN , +SCH_BPF_THROTTLE , +SCH_BPF_PASS , +SCH_BPF_BYPASS , +SCH_BPF_STOLEN , +}; +#endif /* _UAPI__LINUX_BPF_H__ */ [ ... ]
Quoted 40 lines
include/uapi/linux/bpf.h +40
-0+static bool tc_qdisc_is_valid_access (int off,int size, +enum bpf_access_type type, +const struct bpf_prog *prog, +struct bpf_insn_access_aux *info) +{ +struct btf *btf; + +if (off <0 || off >=sizeof (struct bpf_qdisc_ctx )) +return false; + +switch (off) { +case offsetof(struct bpf_qdisc_ctx , skb): +if (type ==BPF_WRITE || + size != sizeof_field(struct bpf_qdisc_ctx, skb)) +return false; + +if (prog->expected_attach_type !=BPF_QDISC_ENQUEUE ) +return false; + + btf =bpf_get_btf_vmlinux (); +if (IS_ERR_OR_NULL (btf)) +return false; + + info->btf = btf; + info->btf_id = tc_qdisc_ctx_access_btf_ids[0 ]; + info->reg_type =PTR_TO_BTF_ID |PTR_TRUSTED ; +return true; + case bpf_ctx_range(struct bpf_qdisc_ctx, classid): + return size == sizeof_field(struct bpf_qdisc_ctx, classid); + case bpf_ctx_range(struct bpf_qdisc_ctx, expire): + return size == sizeof_field(struct bpf_qdisc_ctx, expire); + case bpf_ctx_range(struct bpf_qdisc_ctx, delta_ns): + return size == sizeof_field(struct bpf_qdisc_ctx, delta_ns); +default : +return false; + } + +return false; +} +[ ... ]
Quoted 105 lines
include/uapi/linux/bpf.h +105
-0+static int sch_bpf_enqueue (struct sk_buff *skb,struct Qdisc *sch, +struct sk_buff **to_free) +{ +struct bpf_sched_data *q =qdisc_priv (sch); +unsigned int len =qdisc_pkt_len (skb); +struct bpf_qdisc_ctx ctx = {}; +int res =NET_XMIT_SUCCESS ; +struct sch_bpf_class *cl; +struct bpf_prog *enqueue; + + enqueue =rcu_dereference (q->enqueue_prog.prog); +if (!enqueue) +return NET_XMIT_DROP ; + + ctx.skb = skb; + ctx.classid = sch->handle; + res =bpf_prog_run (enqueue, &ctx); +switch (res) { +case SCH_BPF_THROTTLE : +qdisc_watchdog_schedule_range_ns (&q->watchdog, ctx.expire, ctx.delta_ns); +qdisc_qstats_overlimit (sch); + fallthrough; +case SCH_BPF_QUEUED : +qdisc_qstats_backlog_inc (sch, skb); +return NET_XMIT_SUCCESS ; +case SCH_BPF_BYPASS : +qdisc_qstats_drop (sch); +__qdisc_drop (skb, to_free); +return NET_XMIT_SUCCESS | __NET_XMIT_BYPASS; +case SCH_BPF_STOLEN : +__qdisc_drop (skb, to_free); +return NET_XMIT_SUCCESS | __NET_XMIT_STOLEN; +case SCH_BPF_CN : +return NET_XMIT_CN ; +case SCH_BPF_PASS : +break ; +default : +return qdisc_drop (skb, sch, to_free); + } + + cl =sch_bpf_find (sch, ctx.classid); +if (!cl || !cl->qdisc) +return qdisc_drop (skb, sch, to_free); + + res =qdisc_enqueue (skb, cl->qdisc, to_free); +if (res !=NET_XMIT_SUCCESS ) { +if (net_xmit_drop_count (res)) { +qdisc_qstats_drop (sch); + cl->drops++; + } +return res; + } + + sch->qstats.backlog += len; + sch->q.qlen++; +return res; +} + +DEFINE_PER_CPU(struct sk_buff*, bpf_skb_dequeue); + +static struct sk_buff *sch_bpf_dequeue (struct Qdisc *sch) +{ +struct bpf_sched_data *q =qdisc_priv (sch); +struct bpf_qdisc_ctx ctx = {}; +struct sk_buff *skb =NULL ; +struct bpf_prog *dequeue; +struct sch_bpf_class *cl; +int res; + + dequeue =rcu_dereference (q->dequeue_prog.prog); +if (!dequeue) +return NULL ; + +__this_cpu_write (bpf_skb_dequeue,NULL ); + ctx.classid = sch->handle; + res =bpf_prog_run (dequeue, &ctx); +switch (res) { +case SCH_BPF_DEQUEUED : + skb =__this_cpu_read (bpf_skb_dequeue); +qdisc_bstats_update (sch, skb); +qdisc_qstats_backlog_dec (sch, skb); +break ; +case SCH_BPF_THROTTLE : +qdisc_watchdog_schedule_range_ns (&q->watchdog, ctx.expire, ctx.delta_ns); +qdisc_qstats_overlimit (sch); + cl =sch_bpf_find (sch, ctx.classid); +if (cl) + cl->overlimits++; +return NULL ; +case SCH_BPF_PASS : + cl =sch_bpf_find (sch, ctx.classid); +if (!cl || !cl->qdisc) +return NULL ; + skb =qdisc_dequeue_peeked (cl->qdisc); +if (skb) { +bstats_update (&cl->bstats, skb); +qdisc_bstats_update (sch, skb); +qdisc_qstats_backlog_dec (sch, skb); + sch->q.qlen--; + } +break ; + } + +return skb; +}[ ... ]
Quoted 36 lines
include/uapi/linux/bpf.h +36
-0+static int sch_bpf_init (struct Qdisc *sch,struct nlattr *opt, +struct netlink_ext_ack *extack) +{ +struct bpf_sched_data *q =qdisc_priv (sch); +int err; + +qdisc_watchdog_init (&q->watchdog, sch); +if (opt) { + err =sch_bpf_change (sch, opt, extack); +if (err) +return err; + } + + err =tcf_block_get (&q->block, &q->filter_list, sch, extack); +if (err) +return err; + +return qdisc_class_hash_init (&q->clhash); +} + +static void sch_bpf_reset (struct Qdisc *sch) +{ +struct bpf_sched_data *q =qdisc_priv (sch); +struct sch_bpf_class *cl; +unsigned int i; + +for (i =0 ; i < q->clhash.hashsize; i++) { +hlist_for_each_entry (cl, &q->clhash.hash[i], common.hnode) { +if (cl->qdisc) +qdisc_reset (cl->qdisc); + } + } + +qdisc_watchdog_cancel (&q->watchdog); +} +[ ... ]
Quoted 10 lines
include/uapi/linux/bpf.h +10
-0+static const struct Qdisc_class_ops sch_bpf_class_ops = { + .graft = sch_bpf_graft, + .leaf = sch_bpf_leaf, + .find = sch_bpf_search, + .change = sch_bpf_change_class, + .delete = sch_bpf_delete, + .tcf_block = sch_bpf_tcf_block, + .bind_tcf = sch_bpf_bind, + .unbind_tcf = sch_bpf_unbind, + .dump = sch_bpf_dump_class,include/uapi/linux/bpf.h
Quoted 10 lines
include/uapi/linux/bpf.h +10
-0+ .dump_stats = sch_bpf_dump_class_stats, + .walk = sch_bpf_walk, +}; + +static struct Qdisc_ops sch_bpf_qdisc_ops__read_mostly = { + .cl_ops = &sch_bpf_class_ops, + .id ="bpf" , + .priv_size =sizeof (struct bpf_sched_data ), + .enqueue = sch_bpf_enqueue, + .dequeue = sch_bpf_dequeue,I looked at the high level of the patchset. The major ops that it wants to be programmable in bpf is the ".enqueue" and ".dequeue" (+ ".init" and ".reset" in patch 4 and patch 5).
Quoted 8 lines
This patch adds a new prog type
BPF_PROG_TYPE_QDISC, four attach types (each for ".enqueue", ".dequeue", ".init", and ".reset"), and a new "bpf_qdisc_ctx" in the uapi. It is no long an acceptable way to add new bpf extension.Can the ".enqueue", ".dequeue", ".init", and ".reset" be completely implemented in bpf (with the help of new kfuncs if needed)? Then a
struct_opsforQdisc_opscan be created. The bpfQdisc_opscan be loaded through the existingstruct_opsapi.
Partially. If using struct_ops, I think we'll need another structure like the following in bpf qdisc to be implemented with struct_ops bpf:
struct bpf_qdisc_ops {
int (*enqueue) (struct sk_buff *skb)
void (*dequeue) (void)
void (*init) (void)
void (*reset) (void)
};
Then, Qdisc_ops will wrap around them to handle things that cannot be implemented with bpf (e.g., sch_tree_lock, returning a skb ptr).
Quoted 5 lines
If other ops (like ".dump", ".dump_stats"...) do not have good use case to be programmable in bpf, it can stay with the kernel implementation for now and only allows the userspace to load the a bpf
Qdisc_opswith .equeue/dequeue/init/reset implemented.
Quoted 9 lines
You mentioned in the cover letter that:
"Currentstruct_opsattachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong."I assumed you meant allow bpf to replace only "some" ops of the
Qdisc_ops? Yes, it can be done through the struct_ops's ".init_member". Take a look atbpf_tcp_ca_init_member. The kernel can assign the kernel implementation for ".dump" (for example) when loading the bpfQdisc_ops.
I have no problem with partially replacing a struct, which like you mentioned has been demonstrated by congestion control or sched_ext. What I am not sure about is the ability to create multiple bpf qdiscs, where each has different ".enqueue", ".dequeue", and so on. I like the struct_ops approach and would love to try it if struct_ops support this.
Thanks,
Amery
Quoted 19 lines
Quoted 18 lines
include/uapi/linux/bpf.h +18
-0+ .peek = qdisc_peek_dequeued, + .init = sch_bpf_init, + .reset = sch_bpf_reset, > + .destroy = sch_bpf_destroy, + .change = sch_bpf_change, + .dump = sch_bpf_dump, + .dump_stats = sch_bpf_dump_stats, + .owner = THIS_MODULE, +}; + +static int __init sch_bpf_mod_init (void ) +{ +return register_qdisc (&sch_bpf_qdisc_ops); +} + +static void __exit sch_bpf_mod_exit(void) +{ +unregister_qdisc (&sch_bpf_qdisc_ops); +}
Martin KaFai Lau
reviewerQuoted 22 lines · Amery Hung
Quoted 4 lines
I looked at the high level of the patchset. The major ops that it wants to be programmable in bpf is the ".enqueue" and ".dequeue" (+ ".init" and ".reset" in patch 4 and patch 5).
Quoted 8 lines
This patch adds a new prog type
BPF_PROG_TYPE_QDISC, four attach types (each for ".enqueue", ".dequeue", ".init", and ".reset"), and a new "bpf_qdisc_ctx" in the uapi. It is no long an acceptable way to add new bpf extension.Can the ".enqueue", ".dequeue", ".init", and ".reset" be completely implemented in bpf (with the help of new kfuncs if needed)? Then a
struct_opsforQdisc_opscan be created. The bpfQdisc_opscan be loaded through the existingstruct_opsapi.Partially. If using
struct_ops, I think we'll need another structure like the following in bpf qdisc to be implemented withstruct_opsbpf:struct
bpf_qdisc_ops{
int (*enqueue) (structsk_buff*skb)
void (*dequeue) (void)
void (*init) (void)
void (*reset) (void)
};
Quoted 2 lines
Then,
Qdisc_opswill wrap around them to handle things that cannot be implemented with bpf (e.g.,sch_tree_lock, returning a skb ptr).
We can see how those limitations (calling sch_tree_lock() and returning a ptr) can be addressed in bpf. This will also help other similar use cases.
Other than sch_tree_lock and returning a ptr from a bpf prog. What else do you see that blocks directly implementing the enqueue/dequeue/init/reset in the struct Qdisc_ops?
Have you thought above ".priv_size"? It is now fixed to sizeof(struct bpf_sched_data). It should be useful to allow the bpf prog to store its own data there?
Quoted 15 lines
Quoted 6 lines
If other ops (like ".dump", ".dump_stats"...) do not have good use case to be programmable in bpf, it can stay with the kernel implementation for now and only allows the userspace to load the a bpf
Qdisc_opswith .equeue/dequeue/init/reset implemented.Quoted 9 lines
You mentioned in the cover letter that:
"Currentstruct_opsattachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong."I assumed you meant allow bpf to replace only "some" ops of the
Qdisc_ops? Yes, it can be done through the struct_ops's ".init_member". Take a look atbpf_tcp_ca_init_member. The kernel can assign the kernel implementation for ".dump" (for example) when loading the bpfQdisc_ops.
Quoted 6 lines
I have no problem with partially replacing a struct, which like you mentioned has been demonstrated by congestion control or
sched_ext. What I am not sure about is the ability to create multiple bpf qdiscs, where each has different ".enqueue", ".dequeue", and so on. I like thestruct_opsapproach and would love to try it ifstruct_opssupport this.
The need for allowing different ".enqueue/.dequeue/..." bpf (BPF_PROG_TYPE_QDISC) programs loaded into different qdisc instances is because there is only one ".id == bpf" Qdisc_ops native kernel implementation which is then because of the limitation you mentioned above?
Am I understanding your reason correctly on why it requires to load different bpf prog for different qdisc instances?
If the ".enqueue/.dequeue/..." in the "struct Qdisc_ops" can be directly implemented in bpf prog itself, it can just load another bpf struct_ops which has a different ".enqueue/.dequeue/..." implementation:
#> bpftool struct_ops register bpf_simple_fq_v1.bpf.o #> bpftool struct_ops register bpf_simple_fq_v2.bpf.o #> bpftool struct_ops register bpf_simple_fq_xyz.bpf.o
From reading the test bpf prog, I think the set is on a good footing. Instead of working around the limitation by wrapping the bpf prog in a predefined "struct Qdisc_ops sch_bpf_qdisc_ops", lets first understand what is missing in bpf and see how we could address them.
Quoted 24 lines · Martin KaFai Lau
Quoted 22 lines · Amery Hung
Quoted 4 lines
I looked at the high level of the patchset. The major ops that it wants to be programmable in bpf is the ".enqueue" and ".dequeue" (+ ".init" and ".reset" in patch 4 and patch 5).
Quoted 8 lines
This patch adds a new prog type
BPF_PROG_TYPE_QDISC, four attach types (each for ".enqueue", ".dequeue", ".init", and ".reset"), and a new "bpf_qdisc_ctx" in the uapi. It is no long an acceptable way to add new bpf extension.Can the ".enqueue", ".dequeue", ".init", and ".reset" be completely implemented in bpf (with the help of new kfuncs if needed)? Then a
struct_opsforQdisc_opscan be created. The bpfQdisc_opscan be loaded through the existingstruct_opsapi.Partially. If using
struct_ops, I think we'll need another structure like the following in bpf qdisc to be implemented withstruct_opsbpf:struct
bpf_qdisc_ops{
int (*enqueue) (structsk_buff*skb)
void (*dequeue) (void)
void (*init) (void)
void (*reset) (void)
};
Quoted 6 lines
Quoted 2 lines
Then,
Qdisc_opswill wrap around them to handle things that cannot be implemented with bpf (e.g.,sch_tree_lock, returning a skb ptr).We can see how those limitations (calling
sch_tree_lock()and returning a ptr) can be addressed in bpf. This will also help other similar use cases.
For kptr, I wonder if we can support the following semantics in bpf if they make sense:
1. Passing a referenced kptr into a bpf program, which will also need to be released, or exchanged into maps or allocated objects.
2. Returning a kptr from a program and treating it as releasing the reference.
Quoted 4 lines
Other than
sch_tree_lockand returning a ptr from a bpf prog. What else do you see that blocks directly implementing the enqueue/dequeue/init/reset in the structQdisc_ops?
Not much! We can deal with sch_tree_lock later since enqueue/dequeue/init/reset are unlikely to use it.
Quoted 4 lines
Have you thought above ".priv_size"? It is now fixed to
sizeof(struct bpf_sched_data). It should be useful to allow the bpf prog to store its own data there?
Maybe we can let bpf qdiscs store statistics here and make it work with netlink. I haven't explored much in how bpf qdiscs record and share statistics with user space.
Quoted 34 lines
Quoted 15 lines
Quoted 6 lines
If other ops (like ".dump", ".dump_stats"...) do not have good use case to be programmable in bpf, it can stay with the kernel implementation for now and only allows the userspace to load the a bpf
Qdisc_opswith .equeue/dequeue/init/reset implemented.Quoted 9 lines
You mentioned in the cover letter that:
"Currentstruct_opsattachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong."I assumed you meant allow bpf to replace only "some" ops of the
Qdisc_ops? Yes, it can be done through the struct_ops's ".init_member". Take a look atbpf_tcp_ca_init_member. The kernel can assign the kernel implementation for ".dump" (for example) when loading the bpfQdisc_ops.Quoted 6 lines
I have no problem with partially replacing a struct, which like you mentioned has been demonstrated by congestion control or
sched_ext. What I am not sure about is the ability to create multiple bpf qdiscs, where each has different ".enqueue", ".dequeue", and so on. I like thestruct_opsapproach and would love to try it ifstruct_opssupport this.The need for allowing different ".enqueue/.dequeue/..." bpf (
BPF_PROG_TYPE_QDISC) programs loaded into different qdisc instances is because there is only one ".id == bpf"Qdisc_opsnative kernel implementation which is then because of the limitation you mentioned above?Am I understanding your reason correctly on why it requires to load different bpf prog for different qdisc instances?
If the ".enqueue/.dequeue/..." in the "struct
Qdisc_ops" can be directly implemented in bpf prog itself, it can just load another bpfstruct_opswhich has a different ".enqueue/.dequeue/..." implementation:
Quoted 9 lines
#> bpftool
struct_opsregister bpf_simple_fq_v1.bpf.o #> bpftoolstruct_opsregister bpf_simple_fq_v2.bpf.o #> bpftoolstruct_opsregister bpf_simple_fq_xyz.bpf.oFrom reading the test bpf prog, I think the set is on a good footing. Instead
of working around the limitation by wrapping the bpf prog in a predefined "structQdisc_opssch_bpf_qdisc_ops", lets first understand what is missing in bpf and see how we could address them.
Thank you so much for the clarification. I had the wrong impression since I was thinking about using a structure in the bpf qdisc for struct_ops. It makes sense to try making "struct Qdisc_ops" work with struct_ops. I will send the next patch set with struct_ops.
Thanks,
Amery
Quoted 1 line
Martin KaFai Lau
reviewerQuoted 28 lines · Amery Hung
Quoted 27 lines · Martin KaFai Lau
Quoted 24 lines · Amery Hung
Quoted 4 lines
I looked at the high level of the patchset. The major ops that it wants to be programmable in bpf is the ".enqueue" and ".dequeue" (+ ".init" and ".reset" in patch 4 and patch 5).
Quoted 8 lines
This patch adds a new prog type
BPF_PROG_TYPE_QDISC, four attach types (each for ".enqueue", ".dequeue", ".init", and ".reset"), and a new "bpf_qdisc_ctx" in the uapi. It is no long an acceptable way to add new bpf extension.Can the ".enqueue", ".dequeue", ".init", and ".reset" be completely implemented in bpf (with the help of new kfuncs if needed)? Then a
struct_opsforQdisc_opscan be created. The bpfQdisc_opscan be loaded through the existingstruct_opsapi.Partially. If using
struct_ops, I think we'll need another structure like the following in bpf qdisc to be implemented withstruct_opsbpf:struct
bpf_qdisc_ops{
int (*enqueue) (structsk_buff*skb)
void (*dequeue) (void)
void (*init) (void)
void (*reset) (void)
};Then,
Qdisc_opswill wrap around them to handle things that cannot be implemented with bpf (e.g.,sch_tree_lock, returning a skb ptr).
Quoted 6 lines
Quoted 3 lines
We can see how those limitations (calling
sch_tree_lock()and returning a ptr) can be addressed in bpf. This will also help other similar use cases.For kptr, I wonder if we can support the following semantics in bpf if they make sense:
I think they are useful but they are not fully supported now.
Some thoughts below.
Quoted 2 lines
1. Passing a referenced kptr into a bpf program, which will also need to be released, or exchanged into maps or allocated objects.
"enqueue" should be the one considering here:
struct Qdisc_ops {
/* ... */
int (*enqueue)(struct sk_buff *skb,
struct Qdisc *sch,
struct sk_buff **to_free);};
The verifier only marks the skb as a trusted kptr but does not mark its reg->ref_obj_id. Take a look at btf_ctx_access(). In particular:
if (prog_args_trusted(prog)) info->reg_type |= PTR_TRUSTED;
The verifier does not know the skb ownership is passed into the ".enqueue" ops and does not know the bpf prog needs to release it or store it in a map.
The verifier tracks the reference state when a KF_ACQUIRE kfunc is called (just an example, not saying we need to use KF_ACQUIRE kfunc). Take a look at acquire_reference_state() which is the useful one here.
Whenever the verifier is loading the ".enqueue" bpf_prog, the verifier can always acquire_reference_state() for the "struct sk_buff *skb" argument.
Take a look at a recent RFC: https://lore.kernel.org/bpf/[email protected]/
which is tagging the argument of an ops (e.g. ".enqueue" here). That RFC patch is tagging the argument could be NULL by appending "__nullable" to the argument name. The verifier will enforce that the bpf prog must check for NULL first.
The similar idea can be used here but with a different tagging (for example, "__must_release", admittedly not a good name). While the RFC patch is in-progress, for now, may be hardcode for the ".enqueue" ops in check_struct_ops_btf_id() and always acquire_reference_state() for the skb. This part can be adjusted later once the RFC patch will be in shape.
Then one more thing is to track when the struct_ops bpf prog is actually reading the value of the skb pointer. One thing is worth to mention here, e.g. a struct_ops prog for enqueue:
SEC("struct_ops")
int BPF_PROG(bpf_dropall_enqueue, struct sk_buff *skb, struct Qdisc *sch,
struct sk_buff **to_free)
{
return bpf_qdisc_drop(skb, sch, to_free);
}Take a look at the BPF_PROG macro, the bpf prog is getting a pointer to an array of __u64 as the only argument. The skb is actually in ctx[0], sch is in ctx[1]...etc. When ctx[0] is read to get the skb pointer (e.g. r1 = ctx[0]), btf_ctx_access() marks the reg_type to PTR_TRUSTED. It needs to also initialize the reg->ref_obj_id by the id obtained earlier from acquire_reference_state() during check_struct_ops_btf_id() somehow.
Quoted 1 line
2. Returning a kptr from a program and treating it as releasing the reference.
e.g. for dequeue:
struct Qdisc_ops {
/* ... */
struct sk_buff * (*dequeue)(struct Qdisc *);
};Right now the verifier should complain on check_reference_leak() if the struct_ops bpf prog is returning a referenced kptr.
Unlike an argument, the return type of a function does not have a name to tag. It is the first case that a struct_ops bpf_prog returning a pointer. One idea is to assume it must be a trusted pointer (PTR_TRUSTED) and the verifier should check it is indeed with PTR_TRUSTED flag.
May be release_reference_state() can be called to assume the kernel will release it as long as the return pointer type is PTR_TRUSTED and the type matches the return type of the ops. Take a look at check_return_code().
Quoted 92 lines · Martin KaFai Lau
Quoted 28 lines · Amery Hung
Quoted 27 lines · Martin KaFai Lau
Quoted 24 lines · Amery Hung
Quoted 4 lines
I looked at the high level of the patchset. The major ops that it wants to be programmable in bpf is the ".enqueue" and ".dequeue" (+ ".init" and ".reset" in patch 4 and patch 5).
Quoted 8 lines
This patch adds a new prog type
BPF_PROG_TYPE_QDISC, four attach types (each for ".enqueue", ".dequeue", ".init", and ".reset"), and a new "bpf_qdisc_ctx" in the uapi. It is no long an acceptable way to add new bpf extension.Can the ".enqueue", ".dequeue", ".init", and ".reset" be completely implemented in bpf (with the help of new kfuncs if needed)? Then a
struct_opsforQdisc_opscan be created. The bpfQdisc_opscan be loaded through the existingstruct_opsapi.Partially. If using
struct_ops, I think we'll need another structure like the following in bpf qdisc to be implemented withstruct_opsbpf:struct
bpf_qdisc_ops{
int (*enqueue) (structsk_buff*skb)
void (*dequeue) (void)
void (*init) (void)
void (*reset) (void)
};Then,
Qdisc_opswill wrap around them to handle things that cannot be implemented with bpf (e.g.,sch_tree_lock, returning a skb ptr).Quoted 6 lines
Quoted 3 lines
We can see how those limitations (calling
sch_tree_lock()and returning a ptr) can be addressed in bpf. This will also help other similar use cases.For kptr, I wonder if we can support the following semantics in bpf if they make sense:
I think they are useful but they are not fully supported now.
Some thoughts below.
Quoted 2 lines
1. Passing a referenced kptr into a bpf program, which will also need to be released, or exchanged into maps or allocated objects.
"enqueue" should be the one considering here:
struct
Qdisc_ops{
/* ... */
int (*enqueue)(structsk_buff*skb, struct Qdisc *sch,
structsk_buff**to_free);};
The verifier only marks the skb as a trusted kptr but does not mark its
reg->ref_obj_id. Take a look atbtf_ctx_access(). In particular:if (
prog_args_trusted(prog))
info->reg_type|=PTR_TRUSTED;The verifier does not know the skb ownership is passed into the ".enqueue" ops and does not know the bpf prog needs to release it or store it in a map.
The verifier tracks the reference state when a
KF_ACQUIREkfunc is called (just an example, not saying we need to useKF_ACQUIREkfunc). Take a look atacquire_reference_state()which is the useful one here.Whenever the verifier is loading the ".enqueue"
bpf_prog, the verifier can alwaysacquire_reference_state()for the "structsk_buff*skb" argument.Take a look at a recent RFC: https://lore.kernel.org/bpf/[email protected]/
which is tagging the argument of an ops (e.g. ".enqueue" here). That RFC patch is tagging the argument could be NULL by appending "__nullable" to the argument name. The verifier will enforce that the bpf prog must check for NULL first.The similar idea can be used here but with a different tagging (for example, "
__must_release", admittedly not a good name). While the RFC patch is in-progress, for now, may be hardcode for the ".enqueue" ops incheck_struct_ops_btf_id()and alwaysacquire_reference_state()for the skb. This part can be adjusted later once the RFC patch will be in shape.Then one more thing is to track when the
struct_opsbpf prog is actually reading the value of the skb pointer. One thing is worth to mention here, e.g. astruct_opsprog for enqueue:
SEC("struct_ops")
int BPF_PROG(bpf_dropall_enqueue, structsk_buff*skb, struct Qdisc *sch,
structsk_buff**to_free)
{
returnbpf_qdisc_drop(skb, sch, to_free);
}Take a look at the
BPF_PROGmacro, the bpf prog is getting a pointer to an array of__u64as the only argument. The skb is actually in ctx[0], sch is in ctx[1]...etc. When ctx[0] is read to get the skb pointer (e.g. r1 = ctx[0]),btf_ctx_access()marks thereg_typetoPTR_TRUSTED. It needs to also initialize thereg->ref_obj_idby the id obtained earlier fromacquire_reference_state()duringcheck_struct_ops_btf_id()somehow.Quoted 1 line
2. Returning a kptr from a program and treating it as releasing the reference.
e.g. for dequeue:
struct
Qdisc_ops{
/* ... */
structsk_buff* (*dequeue)(struct Qdisc *);
};
Quoted 3 lines
Right now the verifier should complain on
check_reference_leak()if thestruct_opsbpf prog is returning a referenced kptr.Unlike an argument, the return type of a function does not have a name to tag. It is the first case that a
struct_opsbpf_progreturning a
We may tag the stub functions instead, right?
Is the purpose here to return a referenced pointer from a struct_ops operator without verifier complaining?
Quoted 4 lines
pointer. One idea is to assume it must be a trusted pointer (
PTR_TRUSTED) and the verifier should check it is indeed withPTR_TRUSTEDflag.May be
release_reference_state()can be called to assume the kernel will release it as long as the return pointer type isPTR_TRUSTEDand the type matches the return type of the ops. Take a look atcheck_return_code().
Martin KaFai Lau
reviewerQuoted 10 lines · Kui-Feng Lee
Quoted 10 lines
Quoted 1 line
2. Returning a kptr from a program and treating it as releasing the reference.
e.g. for dequeue:
struct
Qdisc_ops{
/* ... */
structsk_buff* (*dequeue)(struct Qdisc *); };
Quoted 5 lines
Quoted 3 lines
Right now the verifier should complain on
check_reference_leak()if thestruct_opsbpf prog is returning a referenced kptr.Unlike an argument, the return type of a function does not have a name to tag. It is the first case that a
struct_opsbpf_progreturning aWe may tag the stub functions instead, right?
What is the suggestion on how to tag the return type?
I was suggesting it doesn't need to tag and it should by default require a trusted ptr for the pointer returned by struct_ops. The pointer argument and the return pointer of a struct_ops should be a trusted ptr.
Quoted 2 lines
Is the purpose here to return a referenced pointer from a
struct_opsoperator without verifier complaining?
Yes, basically need to teach the verifier the kernel will do the reference release.
Quoted 4 lines
Quoted 4 lines
pointer. One idea is to assume it must be a trusted pointer (
PTR_TRUSTED) and the verifier should check it is indeed withPTR_TRUSTEDflag.May be
release_reference_state()can be called to assume the kernel will release it as long as the return pointer type isPTR_TRUSTEDand the type matches the return type of the ops. Take a look atcheck_return_code().
Quoted 13 lines · Martin KaFai Lau
Quoted 12 lines · Kui-Feng Lee
Quoted 12 lines
Quoted 1 line
2. Returning a kptr from a program and treating it as releasing the reference.
e.g. for dequeue:
struct
Qdisc_ops{
/* ... */
structsk_buff* (*dequeue)(struct Qdisc *); };Right now the verifier should complain on
check_reference_leak()if thestruct_opsbpf prog is returning a referenced kptr.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Unlike an argument, the return type of a function does not have a name to tag. It is the first case that a
struct_opsbpf_progreturning aWe may tag the stub functions instead, right?
What is the suggestion on how to tag the return type?
I was suggesting it doesn't need to tag and it should by default require a trusted ptr for the pointer returned by
struct_ops. The pointer argument and the return pointer of astruct_opsshould be a trusted ptr.
That make sense to me. Should we also allow operators to return a null pointer?
Quoted 11 lines
Quoted 3 lines
Is the purpose here to return a referenced pointer from a
struct_opsoperator without verifier complaining?Yes, basically need to teach the verifier the kernel will do the reference release.
Quoted 4 lines
Quoted 4 lines
pointer. One idea is to assume it must be a trusted pointer (
PTR_TRUSTED) and the verifier should check it is indeed withPTR_TRUSTEDflag.May be
release_reference_state()can be called to assume the kernel will release it as long as the return pointer type isPTR_TRUSTEDand the type matches the return type of the ops. Take a look atcheck_return_code().
Quoted 26 lines · Kui-Feng Lee
Quoted 22 lines · Martin KaFai Lau
Quoted 11 lines · Kui-Feng Lee
Quoted 11 lines
Quoted 2 lines
2. Returning a kptr from a program and treating it as releasing the reference.
e.g. for dequeue:
struct Qdisc_ops { /* ... */ struct sk_buff * (*dequeue)(struct Qdisc *); };Quoted 7 lines
Quoted 5 lines
Right now the verifier should complain on
check_reference_leak()if thestruct_opsbpf prog is returning a referenced kptr.Unlike an argument, the return type of a function does not have a name to tag. It is the first case that a
struct_opsbpf_progreturning aWe may tag the stub functions instead, right?
What is the suggestion on how to tag the return type?
Quoted 8 lines
Quoted 3 lines
I was suggesting it doesn't need to tag and it should by default require a trusted ptr for the pointer returned by
struct_ops. The pointer argument and the return pointer of astruct_opsshould be a trusted ptr.That make sense to me. Should we also allow operators to return a null pointer?
.dequeue in Qdisc_ops can return a null pointer when there is no skb to be dequeued so I think that should be allowed.
Quoted 17 lines
Quoted 17 lines
Quoted 3 lines
Is the purpose here to return a referenced pointer from a
struct_opsoperator without verifier complaining?Yes, basically need to teach the verifier the kernel will do the reference release.
Quoted 9 lines
Quoted 9 lines
pointer. One idea is to assume it must be a trusted pointer (
PTR_TRUSTED) and the verifier should check it is indeed withPTR_TRUSTEDflag.May be
release_reference_state()can be called to assume the kernel will release it as long as the return pointer type isPTR_TRUSTEDand the type matches the return type of the ops. Take a look atcheck_return_code().
Quoted 46 lines · Martin KaFai Lau
Quoted 6 lines
Quoted 3 lines
We can see how those limitations (calling
sch_tree_lock()and returning a ptr) can be addressed in bpf. This will also help other similar use cases.For kptr, I wonder if we can support the following semantics in bpf if they make sense:
I think they are useful but they are not fully supported now.
Some thoughts below.
Quoted 2 lines
1. Passing a referenced kptr into a bpf program, which will also need to be released, or exchanged into maps or allocated objects.
"enqueue" should be the one considering here:
struct
Qdisc_ops{
/* ... */
int (*enqueue)(structsk_buff*skb,
struct Qdisc *sch, structsk_buff**to_free);};
The verifier only marks the skb as a trusted kptr but does not mark its
reg->ref_obj_id. Take a look atbtf_ctx_access(). In particular:if (prog_args_trusted(prog)) info->reg_type |= PTR_TRUSTED;The verifier does not know the skb ownership is passed into the ".enqueue" ops and does not know the bpf prog needs to release it or store it in a map.
The verifier tracks the reference state when a
KF_ACQUIREkfunc is called (just an example, not saying we need to useKF_ACQUIREkfunc). Take a look atacquire_reference_state()which is the useful one here.Whenever the verifier is loading the ".enqueue"
bpf_prog, the verifier can alwaysacquire_reference_state()for the "structsk_buff*skb" argument.Take a look at a recent RFC:
https://lore.kernel.org/bpf/[email protected]/
which is tagging the argument of an ops (e.g. ".enqueue" here). That RFC patch is tagging the argument could be NULL by appending "__nullable" to the argument name. The verifier will enforce that the bpf prog must check for NULL first.
Quoted 6 lines
The similar idea can be used here but with a different tagging (for example, "
__must_release", admittedly not a good name). While the RFC patch is in-progress, for now, may be hardcode for the ".enqueue" ops incheck_struct_ops_btf_id()and alwaysacquire_reference_state()for the skb. This part can be adjusted later once the RFC patch will be in shape.
Make sense. One more thing to consider here is that .enqueue is actually a reference acquiring and releasing function at the same time. Assuming ctx written to by a struct_ops program can be seen by the kernel, another new tag for the "to_free" argument will still be needed so that the verifier can recognize when writing skb to "to_free".
Quoted 41 lines
Then one more thing is to track when the
struct_opsbpf prog is actually reading the value of the skb pointer. One thing is worth to mention here, e.g. astruct_opsprog for enqueue:
SEC("struct_ops")
int BPF_PROG(bpf_dropall_enqueue, structsk_buff*skb, struct Qdisc *sch,
structsk_buff**to_free){ return bpf_qdisc_drop(skb, sch, to_free); }Take a look at the
BPF_PROGmacro, the bpf prog is getting a pointer to an array of__u64as the only argument. The skb is actually in ctx[0], sch is in ctx[1]...etc. When ctx[0] is read to get the skb pointer (e.g. r1 = ctx[0]),btf_ctx_access()marks thereg_typetoPTR_TRUSTED. It needs to also initialize thereg->ref_obj_idby the id obtained earlier fromacquire_reference_state()duringcheck_struct_ops_btf_id()somehow.Quoted 1 line
2. Returning a kptr from a program and treating it as releasing the reference.
e.g. for dequeue:
struct Qdisc_ops { /* ... */ struct sk_buff * (*dequeue)(struct Qdisc *); };Right now the verifier should complain on
check_reference_leak()if thestruct_opsbpf prog is returning a referenced kptr.Unlike an argument, the return type of a function does not have a name to tag. It is the first case that a
struct_opsbpf_progreturning a pointer. One idea is to assume it must be a trusted pointer (PTR_TRUSTED) and the verifier should check it is indeed withPTR_TRUSTEDflag.May be
release_reference_state()can be called to assume the kernel will release it as long as the return pointer type isPTR_TRUSTEDand the type matches the return type of the ops. Take a look atcheck_return_code().
Martin KaFai Lau
reviewerQuoted 40 lines · Amery Hung
Quoted 34 lines
Quoted 2 lines
1. Passing a referenced kptr into a bpf program, which will also need to be released, or exchanged into maps or allocated objects.
"enqueue" should be the one considering here:
struct
Qdisc_ops{
/* ... */
int (*enqueue)(structsk_buff*skb,
struct Qdisc *sch, structsk_buff**to_free);};
The verifier only marks the skb as a trusted kptr but does not mark its
reg->ref_obj_id. Take a look atbtf_ctx_access(). In particular:if (prog_args_trusted(prog)) info->reg_type |= PTR_TRUSTED;The verifier does not know the skb ownership is passed into the ".enqueue" ops and does not know the bpf prog needs to release it or store it in a map.
The verifier tracks the reference state when a
KF_ACQUIREkfunc is called (just an example, not saying we need to useKF_ACQUIREkfunc). Take a look atacquire_reference_state()which is the useful one here.Whenever the verifier is loading the ".enqueue"
bpf_prog, the verifier can alwaysacquire_reference_state()for the "structsk_buff*skb" argument.Take a look at a recent RFC:
https://lore.kernel.org/bpf/[email protected]/
which is tagging the argument of an ops (e.g. ".enqueue" here). That RFC patch is tagging the argument could be NULL by appending "__nullable" to the argument name. The verifier will enforce that the bpf prog must check for NULL first.Quoted 6 lines
The similar idea can be used here but with a different tagging (for example, "
__must_release", admittedly not a good name). While the RFC patch is in-progress, for now, may be hardcode for the ".enqueue" ops incheck_struct_ops_btf_id()and alwaysacquire_reference_state()for the skb. This part can be adjusted later once the RFC patch will be in shape.
Quoted 6 lines
Make sense. One more thing to consider here is that .enqueue is actually a reference acquiring and releasing function at the same time. Assuming ctx written to by a
struct_opsprogram can be seen by the kernel, another new tag for the "to_free" argument will still be needed so that the verifier can recognize when writing skb to "to_free".
I don't think "to_free" needs special tagging. I was thinking the "bpf_qdisc_drop" kfunc could be a KF_RELEASE. Ideally, it should be like
__bpf_kfunc int bpf_qdisc_drop(struct sk_buff *skb, struct Qdisc *sch,
struct sk_buff **to_free)
{
return qdisc_drop(skb, sch, to_free);
}However, I don't think the verifier supports pointer to pointer now. Meaning "struct sk_buff **to_free" does not work.
If the ptr indirection spinning in my head is sound, one possible solution to unblock the qdisc work is to introduce:
struct bpf_sk_buff_ptr {
struct sk_buff *skb;
};and the bpf_qdisc_drop kfunc:
__bpf_kfunc int bpf_qdisc_drop(struct sk_buff *skb, struct Qdisc *sch,
struct bpf_sk_buff_ptr *to_free_list)
and the enqueue prog:
SEC("struct_ops/enqueue")
int BPF_PROG(test_enqueue, struct sk_buff *skb,
struct Qdisc *sch,
struct bpf_sk_buff_ptr *to_free_list)
{
return bpf_qdisc_drop(skb, sch, to_free_list);
}and the ".is_valid_access" needs to change the btf_type from "struct sk_buff **" to "struct bpf_sk_buff_ptr *" which is sort of similar to the bpf_tcp_ca.c that is changing the "struct sock *" type to the "struct tcp_sock *" type.
I have the compiler-tested idea here: https://git.kernel.org/pub/scm/linux/kernel/git/martin.lau/bpf-next.git/log/?h=qdisc-ideas
Quoted 18 lines
Quoted 18 lines
Then one more thing is to track when the
struct_opsbpf prog is actually reading the value of the skb pointer. One thing is worth to mention here, e.g. astruct_opsprog for enqueue:
SEC("struct_ops")
int BPF_PROG(bpf_dropall_enqueue, structsk_buff*skb, struct Qdisc *sch,
structsk_buff**to_free){ return bpf_qdisc_drop(skb, sch, to_free); }Take a look at the
BPF_PROGmacro, the bpf prog is getting a pointer to an array of__u64as the only argument. The skb is actually in ctx[0], sch is in ctx[1]...etc. When ctx[0] is read to get the skb pointer (e.g. r1 = ctx[0]),btf_ctx_access()marks thereg_typetoPTR_TRUSTED. It needs to also initialize thereg->ref_obj_idby the id obtained earlier fromacquire_reference_state()duringcheck_struct_ops_btf_id()somehow.
Quoted 83 lines · Martin KaFai Lau
Quoted 40 lines · Amery Hung
Quoted 34 lines
Quoted 2 lines
1. Passing a referenced kptr into a bpf program, which will also need to be released, or exchanged into maps or allocated objects.
"enqueue" should be the one considering here:
struct
Qdisc_ops{
/* ... */
int (*enqueue)(structsk_buff*skb,
struct Qdisc *sch, structsk_buff**to_free);};
The verifier only marks the skb as a trusted kptr but does not mark its
reg->ref_obj_id. Take a look atbtf_ctx_access(). In particular:if (prog_args_trusted(prog)) info->reg_type |= PTR_TRUSTED;The verifier does not know the skb ownership is passed into the ".enqueue" ops and does not know the bpf prog needs to release it or store it in a map.
The verifier tracks the reference state when a
KF_ACQUIREkfunc is called (just an example, not saying we need to useKF_ACQUIREkfunc). Take a look atacquire_reference_state()which is the useful one here.Whenever the verifier is loading the ".enqueue"
bpf_prog, the verifier can alwaysacquire_reference_state()for the "structsk_buff*skb" argument.Take a look at a recent RFC:
https://lore.kernel.org/bpf/[email protected]/
which is tagging the argument of an ops (e.g. ".enqueue" here). That RFC patch is tagging the argument could be NULL by appending "__nullable" to the argument name. The verifier will enforce that the bpf prog must check for NULL first.Quoted 6 lines
The similar idea can be used here but with a different tagging (for example, "
__must_release", admittedly not a good name). While the RFC patch is in-progress, for now, may be hardcode for the ".enqueue" ops incheck_struct_ops_btf_id()and alwaysacquire_reference_state()for the skb. This part can be adjusted later once the RFC patch will be in shape.Quoted 6 lines
Make sense. One more thing to consider here is that .enqueue is actually a reference acquiring and releasing function at the same time. Assuming ctx written to by a
struct_opsprogram can be seen by the kernel, another new tag for the "to_free" argument will still be needed so that the verifier can recognize when writing skb to "to_free".I don't think "
to_free" needs special tagging. I was thinking the "bpf_qdisc_drop" kfunc could be aKF_RELEASE. Ideally, it should be like
__bpf_kfuncint bpf_qdisc_drop(structsk_buff*skb, struct Qdisc *sch,
structsk_buff**to_free){ return qdisc_drop(skb, sch, to_free); }However, I don't think the verifier supports pointer to pointer now. Meaning "struct
sk_buff**to_free" does not work.If the ptr indirection spinning in my head is sound, one possible solution to unblock the qdisc work is to introduce:
struct bpf_sk_buff_ptr { struct sk_buff *skb; };and the
bpf_qdisc_dropkfunc:
__bpf_kfuncint bpf_qdisc_drop(structsk_buff*skb, struct Qdisc *sch,
structbpf_sk_buff_ptr*to_free_list)and the enqueue prog:
SEC("struct_ops/enqueue")
int BPF_PROG(test_enqueue, structsk_buff*skb,
struct Qdisc *sch,
structbpf_sk_buff_ptr*to_free_list){ return bpf_qdisc_drop(skb, sch, to_free_list); }
Quoted 27 lines
and the ".is_valid_access" needs to change the
btf_typefrom "structsk_buff**" to "structbpf_sk_buff_ptr*" which is sort of similar to thebpf_tcp_ca.cthat is changing the "struct sock *" type to the "structtcp_sock*" type.I have the compiler-tested idea here:
https://git.kernel.org/pub/scm/linux/kernel/git/martin.lau/bpf-next.git/log/?h=qdisc-ideasQuoted 18 lines
Quoted 18 lines
Then one more thing is to track when the
struct_opsbpf prog is actually reading the value of the skb pointer. One thing is worth to mention here, e.g. astruct_opsprog for enqueue:
SEC("struct_ops")
int BPF_PROG(bpf_dropall_enqueue, structsk_buff*skb, struct Qdisc *sch,
structsk_buff**to_free){ return bpf_qdisc_drop(skb, sch, to_free); }Take a look at the
BPF_PROGmacro, the bpf prog is getting a pointer to an array of__u64as the only argument. The skb is actually in ctx[0], sch is in ctx[1]...etc. When ctx[0] is read to get the skb pointer (e.g. r1 = ctx[0]),btf_ctx_access()marks thereg_typetoPTR_TRUSTED. It needs to also initialize thereg->ref_obj_idby the id obtained earlier fromacquire_reference_state()duringcheck_struct_ops_btf_id()somehow.
I appreciate the idea. The pointer redirection works without problems. I now have a working fifo bpf qdisc using struct_ops. I will explore how other parts of qdisc work with struct_ops.
Thanks,
Amery
RFC PATCH v7 2/8 net_sched: Add kfuncs for working with skb
In reply to Amery HungFrom: Cong Wang <[email protected]>
This patch introduces four kfuncs available to developers:
struct sk_buff *bpf_skb_acquire(struct sk_buff *skb); void bpf_skb_release(struct sk_buff *skb); bool bpf_qdisc_set_skb_dequeue(struct sk_buff *skb); u32 bpf_skb_get_hash(struct sk_buff *skb)
kptr is used to ensure the vailidility of skbs throughout their lifetime in eBPF qdiscs. First, in the enqueue program, bpf_skb_acquire() can be used to acquire a referenced kptr to an skb from ctx->skb. Then, it can be stored in bpf maps or allocated objects serving as queues. Otherwise, the program should call bpf_skb_release() to release the reference. Finally, in the dequeue program, a skb kptr can be exchanged out of queues and passed to bpf_qdisc_set_skb_dequeue() to set the skb to be dequeued. The kfunc will also release the reference.
Since skb kptr is incompatible with helpers taking __sk_buff, bpf_skb_get_hash() is added for now for the ease of implementing flow-based queueing algorithms.
--- net/sched/sch_bpf.c | 83 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 82 insertions(+), 1 deletion(-)
net/sched/sch_bpf.c +82 -1
@@ -15,6 +15,7 @@#include <linux/slab.h> #include <linux/filter.h> #include <linux/bpf.h> +#include <linux/btf_ids.h> #include <net/netlink.h> #include <net/pkt_sched.h> #include <net/pkt_cls.h>
@@ -520,9 +521,89 @@ static struct Qdisc_ops sch_bpf_qdisc_ops __read_mostly = { .owner = THIS_MODULE, }; +__diag_push (); +__diag_ignore_all ("-Wmissing-prototypes" , +"Global functions as their definitions will be in vmlinux BTF" ); + +/* bpf_skb_acquire - Acquire a reference to an skb. An skb acquired by this +* kfunc which is not stored in a map as a kptr, must be released by calling +* bpf_skb_release(). +* @skb: The skb on which a reference is being acquired. +*/ +__bpf_kfunc struct sk_buff *bpf_skb_acquire(struct sk_buff *skb) +{ +return skb_get (skb); +} + +/* bpf_skb_release - Release the reference acquired on an skb. +* @skb: The skb on which a reference is being released. +*/ +__bpf_kfunc void bpf_skb_release(struct sk_buff *skb) +{ +skb_unref (skb); +} + +/* bpf_skb_destroy - Release an skb reference acquired and exchanged into +* an allocated object or a map. +* @skb: The skb on which a reference is being released. +*/ +__bpf_kfunc void bpf_skb_destroy(struct sk_buff *skb) +{ +skb_unref (skb); +consume_skb (skb); +} + +/* bpf_skb_get_hash - Get the flow hash of an skb. +* @skb: The skb to get the flow hash from. +*/ +__bpf_kfunc u32 bpf_skb_get_hash(struct sk_buff *skb) +{ +return skb_get_hash (skb); +} + +/* bpf_qdisc_set_skb_dequeue - Set the skb to be dequeued. This will also +* release the reference to the skb. +* @skb: The skb to be dequeued by the qdisc. +*/ +__bpf_kfunc void bpf_qdisc_set_skb_dequeue(struct sk_buff *skb) +{ +consume_skb (skb); +__this_cpu_write (bpf_skb_dequeue, skb); +} + +__diag_pop (); + +BTF_SET8_START (skb_kfunc_btf_ids) +BTF_ID_FLAGS (func ,bpf_skb_acquire ,KF_ACQUIRE ) +BTF_ID_FLAGS (func, bpf_skb_release,KF_RELEASE ) +BTF_ID_FLAGS (func, bpf_skb_get_hash) +BTF_ID_FLAGS (func, bpf_qdisc_set_skb_dequeue,KF_RELEASE ) +BTF_SET8_END (skb_kfunc_btf_ids) + +static const struct btf_kfunc_id_set skb_kfunc_set = { + .owner =THIS_MODULE , + .set = &skb_kfunc_btf_ids, +}; + +BTF_ID_LIST (skb_kfunc_dtor_ids) +BTF_ID(struct, sk_buff) +BTF_ID_FLAGS (func, bpf_skb_destroy,KF_RELEASE ) + static int__init sch_bpf_mod_init (void) { -return register_qdisc (&sch_bpf_qdisc_ops); +int ret; +const struct btf_id_dtor_kfunc skb_kfunc_dtors[] = { + { + .btf_id = skb_kfunc_dtor_ids[0 ], + .kfunc_btf_id = skb_kfunc_dtor_ids[1 ] + }, + }; + + ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_QDISC , &skb_kfunc_set); + ret = ret ?:register_btf_id_dtor_kfuncs (skb_kfunc_dtors, +ARRAY_SIZE (skb_kfunc_dtors), +THIS_MODULE ); +return ret ?:register_qdisc (&sch_bpf_qdisc_ops); } static void __exit sch_bpf_mod_exit(void)
Signature · 1 line
2.20.1
RFC PATCH v7 3/8 net_sched: Introduce kfunc bpf_skb_tc_classify()
In reply to Amery HungFrom: Cong Wang <[email protected]>
Introduce a kfunc, bpf_skb_tc_classify(), to reuse exising TC filters on *any* Qdisc to classify the skb.
--- net/sched/sch_bpf.c | 68 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+)
net/sched/sch_bpf.c +68 -0
@@ -571,6 +571,73 @@ __bpf_kfunc void bpf_qdisc_set_skb_dequeue(struct sk_buff *skb)__this_cpu_write (bpf_skb_dequeue, skb); } +/* bpf_skb_tc_classify - Classify an skb using an existing filter referred +* to by the specified handle on the net device of index ifindex. +* @skb: The skb to be classified. +* @handle: The handle of the filter to be referenced. +* @ifindex: The ifindex of the net device where the filter is attached. +* +* Returns a 64-bit integer containing the tc action verdict and the classid, +* created as classid << 32 | action. +*/ +__bpf_kfunc u64 bpf_skb_tc_classify(struct sk_buff *skb, int ifindex, +u32 handle) +{ +struct net *net =dev_net (skb->dev); +const struct Qdisc_class_ops *cops; +struct tcf_result res = {}; +struct tcf_block *block; +struct tcf_chain *chain; +struct net_device *dev; +int result =TC_ACT_OK ; +unsigned long cl =0 ; +struct Qdisc *q; + +rcu_read_lock (); + dev =dev_get_by_index_rcu (net, ifindex); +if (!dev) + goto out; + q =qdisc_lookup_rcu (dev, handle); +if (!q) + goto out; + + cops = q->ops->cl_ops; +if (!cops) + goto out; +if (!cops->tcf_block) + goto out; +if (TC_H_MIN (handle)) { + cl = cops->find (q, handle); +if (cl ==0 ) + goto out; + } + block = cops->tcf_block (q, cl,NULL ); +if (!block) + goto out; + +for (chain =tcf_get_next_chain (block,NULL ); + chain; + chain =tcf_get_next_chain (block, chain)) { +struct tcf_proto *tp; + + result =tcf_classify (skb,NULL , tp, &res, false); +if (result >=0 ) { +switch (result) { +case TC_ACT_QUEUED : +case TC_ACT_STOLEN : +case TC_ACT_TRAP : + fallthrough; +case TC_ACT_SHOT : +rcu_read_unlock (); +return result; + } + } + } +out: +rcu_read_unlock (); +return (res.class <<32 | result); +} +__diag_pop ();BTF_SET8_START (skb_kfunc_btf_ids)
@@ -578,6 +645,7 @@ BTF_ID_FLAGS(func, bpf_skb_acquire, KF_ACQUIRE) BTF_ID_FLAGS(func, bpf_skb_release, KF_RELEASE) BTF_ID_FLAGS(func, bpf_skb_get_hash) BTF_ID_FLAGS(func, bpf_qdisc_set_skb_dequeue, KF_RELEASE) +BTF_ID_FLAGS(func, bpf_skb_tc_classify)BTF_SET8_END (skb_kfunc_btf_ids) static const struct btf_kfunc_id_set skb_kfunc_set = {
Signature · 1 line
2.20.1
RFC PATCH v7 4/8 net_sched: Add reset program
In reply to Amery HungAllow developers to implement customized reset logic through an optional reset program. The program also takes bpf_qdisc_ctx as context, but currently cannot access any field.
To release skbs, the program can release all references to bpf list or rbtree serving as skb queues. The destructor kfunc bpf_skb_destroy() will be called by bpf_map_free_deferred(). This prevents the qdisc from holding the sch_tree_lock for too long when there are many packets in the qdisc.
--- include/uapi/linux/bpf.h | 1 + include/uapi/linux/pkt_sched.h | 4 ++++ kernel/bpf/syscall.c | 1 + net/core/filter.c | 3 +++ net/sched/sch_bpf.c | 30 ++++++++++++++++++++++++++---- tools/include/uapi/linux/bpf.h | 1 + 6 files changed, 36 insertions(+), 4 deletions(-)
include/uapi/linux/bpf.h +1 -0
@@ -1059,6 +1059,7 @@ enum bpf_attach_type {BPF_NETKIT_PEER ,BPF_QDISC_ENQUEUE ,BPF_QDISC_DEQUEUE , +BPF_QDISC_RESET , __MAX_BPF_ATTACH_TYPE };
include/uapi/linux/pkt_sched.h +4 -0
@@ -1328,6 +1328,10 @@ enum {TCA_SCH_BPF_DEQUEUE_PROG_FD ,/* u32 */ TCA_SCH_BPF_DEQUEUE_PROG_ID ,/* u32 */ TCA_SCH_BPF_DEQUEUE_PROG_TAG ,/* data */ +TCA_SCH_BPF_RESET_PROG_NAME ,/* string */ +TCA_SCH_BPF_RESET_PROG_FD ,/* u32 */ +TCA_SCH_BPF_RESET_PROG_ID ,/* u32 */ +TCA_SCH_BPF_RESET_PROG_TAG ,/* data */ __TCA_SCH_BPF_MAX, };
kernel/bpf/syscall.c +1 -0
@@ -2506,6 +2506,7 @@ bpf_prog_load_check_attach(enum bpf_prog_type prog_type,switch (expected_attach_type) {case BPF_QDISC_ENQUEUE :case BPF_QDISC_DEQUEUE : +case BPF_QDISC_RESET :return 0 ;default :return -EINVAL ;
net/core/filter.c +3 -0
@@ -8905,6 +8905,9 @@ static bool tc_qdisc_is_valid_access(int off, int size, {struct btf *btf; +if (prog->expected_attach_type ==BPF_QDISC_RESET ) +return false; +if (off <0 || off >=sizeof (struct bpf_qdisc_ctx ))return false;
net/sched/sch_bpf.c +26 -4
@@ -42,6 +42,7 @@ struct bpf_sched_data {struct Qdisc_class_hash clhash;struct sch_bpf_prog __rcu enqueue_prog;struct sch_bpf_prog __rcu dequeue_prog; +struct sch_bpf_prog __rcu reset_prog;struct qdisc_watchdog watchdog; };
@@ -51,6 +52,9 @@ static int sch_bpf_dump_prog(const struct sch_bpf_prog *prog, struct sk_buff *sk {struct nlattr *nla; +if (!prog->prog) +return 0 ; +if (prog->name &&nla_put_string (skb, name, prog->name))return -EMSGSIZE ;
@@ -81,6 +85,9 @@ static int sch_bpf_dump(struct Qdisc *sch, struct sk_buff *skb)if (sch_bpf_dump_prog (&q->dequeue_prog, skb,TCA_SCH_BPF_DEQUEUE_PROG_NAME ,TCA_SCH_BPF_DEQUEUE_PROG_ID ,TCA_SCH_BPF_DEQUEUE_PROG_TAG )) goto nla_put_failure; +if (sch_bpf_dump_prog (&q->reset_prog, skb,TCA_SCH_BPF_RESET_PROG_NAME , +TCA_SCH_BPF_RESET_PROG_ID ,TCA_SCH_BPF_RESET_PROG_TAG )) + goto nla_put_failure;return nla_nest_end (skb, opts);
@@ -259,16 +266,21 @@ static const struct nla_policy sch_bpf_policy[TCA_SCH_BPF_MAX + 1] = { [TCA_SCH_BPF_DEQUEUE_PROG_FD] = { .type = NLA_U32 }, [TCA_SCH_BPF_DEQUEUE_PROG_NAME ] = { .type =NLA_NUL_STRING , .len = ACT_BPF_NAME_LEN }, + [TCA_SCH_BPF_RESET_PROG_FD] = { .type = NLA_U32 }, + [TCA_SCH_BPF_RESET_PROG_NAME ] = { .type =NLA_NUL_STRING , + .len = ACT_BPF_NAME_LEN }, }; -static int bpf_init_prog (struct nlattr *fd,struct nlattr *name,struct sch_bpf_prog *prog) +static int bpf_init_prog (struct nlattr *fd,struct nlattr *name, +struct sch_bpf_prog *prog,bool optional) {struct bpf_prog *fp, *old_fp;char *prog_name =NULL ;u32 bpf_fd;if (!fd) -return -EINVAL ; +return optional ?0 : -EINVAL ; + bpf_fd =nla_get_u32 (fd); fp =bpf_prog_get_type (bpf_fd,BPF_PROG_TYPE_QDISC );
@@ -327,11 +339,15 @@ static int sch_bpf_change(struct Qdisc *sch, struct nlattr *opt,sch_tree_lock (sch); err =bpf_init_prog (tb[TCA_SCH_BPF_ENQUEUE_PROG_FD ], - tb[TCA_SCH_BPF_ENQUEUE_PROG_NAME ], &q->enqueue_prog); + tb[TCA_SCH_BPF_ENQUEUE_PROG_NAME ], &q->enqueue_prog, false);if (err) goto failure; err =bpf_init_prog (tb[TCA_SCH_BPF_DEQUEUE_PROG_FD ], - tb[TCA_SCH_BPF_DEQUEUE_PROG_NAME ], &q->dequeue_prog); + tb[TCA_SCH_BPF_DEQUEUE_PROG_NAME ], &q->dequeue_prog, false); +if (err) + goto failure; + err =bpf_init_prog (tb[TCA_SCH_BPF_RESET_PROG_FD ], + tb[TCA_SCH_BPF_RESET_PROG_NAME ], &q->reset_prog, true); failure:sch_tree_unlock (sch);return err;
@@ -360,7 +376,9 @@ static int sch_bpf_init(struct Qdisc *sch, struct nlattr *opt,static void sch_bpf_reset (struct Qdisc *sch) {struct bpf_sched_data *q =qdisc_priv (sch); +struct bpf_qdisc_ctx ctx = {};struct sch_bpf_class *cl; +struct bpf_prog *reset;unsigned int i;for (i =0 ; i < q->clhash.hashsize; i++) {
@@ -371,6 +389,9 @@ static void sch_bpf_reset(struct Qdisc *sch) }qdisc_watchdog_cancel (&q->watchdog); + reset =rcu_dereference (q->reset_prog.prog); +if (reset) +bpf_prog_run (reset, &ctx); }static void sch_bpf_destroy_class (struct Qdisc *sch,struct sch_bpf_class *cl)
@@ -398,6 +419,7 @@ static void sch_bpf_destroy(struct Qdisc *sch)sch_tree_lock (sch);bpf_cleanup_prog (&q->enqueue_prog);bpf_cleanup_prog (&q->dequeue_prog); +bpf_cleanup_prog (&q->reset_prog);sch_tree_unlock (sch); }
tools/include/uapi/linux/bpf.h +1 -0
@@ -1059,6 +1059,7 @@ enum bpf_attach_type {BPF_NETKIT_PEER ,BPF_QDISC_ENQUEUE ,BPF_QDISC_DEQUEUE , +BPF_QDISC_RESET , __MAX_BPF_ATTACH_TYPE };
Signature · 1 line
2.20.1
RFC PATCH v7 5/8 net_sched: Add init program
In reply to Amery HungThis patch adds another optional program to be called during the creation of a qdisc for initializating data in the bpf world. The program takes bpf_qdisc_ctx as context, but cannot not access any field.
--- include/uapi/linux/bpf.h | 1 + include/uapi/linux/pkt_sched.h | 4 ++++ kernel/bpf/syscall.c | 1 + net/core/filter.c | 3 ++- net/sched/sch_bpf.c | 23 ++++++++++++++++++++++- tools/include/uapi/linux/bpf.h | 1 + 6 files changed, 31 insertions(+), 2 deletions(-)
include/uapi/linux/bpf.h +1 -0
@@ -1060,6 +1060,7 @@ enum bpf_attach_type {BPF_QDISC_ENQUEUE ,BPF_QDISC_DEQUEUE ,BPF_QDISC_RESET , +BPF_QDISC_INIT , __MAX_BPF_ATTACH_TYPE };
include/uapi/linux/pkt_sched.h +4 -0
@@ -1332,6 +1332,10 @@ enum {TCA_SCH_BPF_RESET_PROG_FD ,/* u32 */ TCA_SCH_BPF_RESET_PROG_ID ,/* u32 */ TCA_SCH_BPF_RESET_PROG_TAG ,/* data */ +TCA_SCH_BPF_INIT_PROG_NAME ,/* string */ +TCA_SCH_BPF_INIT_PROG_FD ,/* u32 */ +TCA_SCH_BPF_INIT_PROG_ID ,/* u32 */ +TCA_SCH_BPF_INIT_PROG_TAG ,/* data */ __TCA_SCH_BPF_MAX, };
kernel/bpf/syscall.c +1 -0
@@ -2507,6 +2507,7 @@ bpf_prog_load_check_attach(enum bpf_prog_type prog_type,case BPF_QDISC_ENQUEUE :case BPF_QDISC_DEQUEUE :case BPF_QDISC_RESET : +case BPF_QDISC_INIT :return 0 ;default :return -EINVAL ;
net/core/filter.c +2 -1
@@ -8905,7 +8905,8 @@ static bool tc_qdisc_is_valid_access(int off, int size, {struct btf *btf; -if (prog->expected_attach_type ==BPF_QDISC_RESET ) +if (prog->expected_attach_type ==BPF_QDISC_RESET || + prog->expected_attach_type ==BPF_QDISC_INIT )return false;if (off <0 || off >=sizeof (struct bpf_qdisc_ctx ))
net/sched/sch_bpf.c +22 -1
@@ -43,6 +43,7 @@ struct bpf_sched_data {struct sch_bpf_prog __rcu enqueue_prog;struct sch_bpf_prog __rcu dequeue_prog;struct sch_bpf_prog __rcu reset_prog; +struct sch_bpf_prog __rcu init_prog;struct qdisc_watchdog watchdog; };
@@ -88,6 +89,9 @@ static int sch_bpf_dump(struct Qdisc *sch, struct sk_buff *skb)if (sch_bpf_dump_prog (&q->reset_prog, skb,TCA_SCH_BPF_RESET_PROG_NAME ,TCA_SCH_BPF_RESET_PROG_ID ,TCA_SCH_BPF_RESET_PROG_TAG )) goto nla_put_failure; +if (sch_bpf_dump_prog (&q->init_prog, skb,TCA_SCH_BPF_INIT_PROG_NAME , +TCA_SCH_BPF_INIT_PROG_ID ,TCA_SCH_BPF_INIT_PROG_TAG )) + goto nla_put_failure;return nla_nest_end (skb, opts);
@@ -269,6 +273,9 @@ static const struct nla_policy sch_bpf_policy[TCA_SCH_BPF_MAX + 1] = { [TCA_SCH_BPF_RESET_PROG_FD] = { .type = NLA_U32 }, [TCA_SCH_BPF_RESET_PROG_NAME ] = { .type =NLA_NUL_STRING , .len = ACT_BPF_NAME_LEN }, + [TCA_SCH_BPF_INIT_PROG_FD] = { .type = NLA_U32 }, + [TCA_SCH_BPF_INIT_PROG_NAME ] = { .type =NLA_NUL_STRING , + .len = ACT_BPF_NAME_LEN }, }; static int bpf_init_prog(struct nlattr *fd, struct nlattr *name,
@@ -348,6 +355,10 @@ static int sch_bpf_change(struct Qdisc *sch, struct nlattr *opt, goto failure; err =bpf_init_prog (tb[TCA_SCH_BPF_RESET_PROG_FD ], tb[TCA_SCH_BPF_RESET_PROG_NAME ], &q->reset_prog, true); +if (err) + goto failure; + err =bpf_init_prog (tb[TCA_SCH_BPF_INIT_PROG_FD ], + tb[TCA_SCH_BPF_INIT_PROG_NAME ], &q->init_prog, true); failure:sch_tree_unlock (sch);return err;
@@ -357,6 +368,8 @@ static int sch_bpf_init(struct Qdisc *sch, struct nlattr *opt, struct netlink_ext_ack *extack) {struct bpf_sched_data *q =qdisc_priv (sch); +struct bpf_qdisc_ctx ctx = {}; +struct bpf_prog *init;int err;qdisc_watchdog_init (&q->watchdog, sch);
@@ -370,7 +383,14 @@ static int sch_bpf_init(struct Qdisc *sch, struct nlattr *opt,if (err)return err; -return qdisc_class_hash_init (&q->clhash); + err =qdisc_class_hash_init (&q->clhash); +if (err <0 ) +return err; + + init =rcu_dereference (q->init_prog.prog); +if (init) +bpf_prog_run (init, &ctx); +return 0 ; }static void sch_bpf_reset (struct Qdisc *sch)
@@ -420,6 +440,7 @@ static void sch_bpf_destroy(struct Qdisc *sch)bpf_cleanup_prog (&q->enqueue_prog);bpf_cleanup_prog (&q->dequeue_prog);bpf_cleanup_prog (&q->reset_prog); +bpf_cleanup_prog (&q->init_prog);sch_tree_unlock (sch); }
tools/include/uapi/linux/bpf.h +1 -0
@@ -1060,6 +1060,7 @@ enum bpf_attach_type {BPF_QDISC_ENQUEUE ,BPF_QDISC_DEQUEUE ,BPF_QDISC_RESET , +BPF_QDISC_INIT , __MAX_BPF_ATTACH_TYPE };
Signature · 1 line
2.20.1
RFC PATCH v7 6/8 tools/libbpf: Add support for BPF_PROG_TYPE_QDISC
In reply to Amery HungWhile eBPF qdisc uses NETLINK for attachment, expected_attach_type is required at load time to verify context access from different programs. This patch adds the section definition for this.
--- tools/lib/bpf/libbpf.c | 4 ++++ 1 file changed, 4 insertions(+)
tools/lib/bpf/libbpf.c +4 -0
@@ -8991,6 +8991,10 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("struct_ops.s+" ,STRUCT_OPS ,0 ,SEC_SLEEPABLE ),SEC_DEF ("sk_lookup" ,SK_LOOKUP ,BPF_SK_LOOKUP ,SEC_ATTACHABLE ),SEC_DEF ("netfilter" ,NETFILTER ,BPF_NETFILTER ,SEC_NONE ), +SEC_DEF ("qdisc/enqueue" ,QDISC ,BPF_QDISC_ENQUEUE ,SEC_ATTACHABLE_OPT ), +SEC_DEF ("qdisc/dequeue" ,QDISC ,BPF_QDISC_DEQUEUE ,SEC_ATTACHABLE_OPT ), +SEC_DEF ("qdisc/reset" ,QDISC ,BPF_QDISC_RESET ,SEC_ATTACHABLE_OPT ), + SEC_DEF("qdisc/init", QDISC, BPF_QDISC_INIT, SEC_ATTACHABLE_OPT), }; int libbpf_register_prog_handler(const char *sec,
Signature · 1 line
2.20.1
Re: [RFC PATCH v7 6/8] tools/libbpf: Add support for BPF_PROG_TYPE_QDISC
Quoted 15 lines · Amery Hung
While eBPF qdisc uses NETLINK for attachment,
expected_attach_typeis required at load time to verify context access from different programs. This patch adds the section definition for this.Signed-off-by Amery Hung <[email protected]>--- tools/lib/bpf/libbpf.c | 4 ++++ 1 file changed, 4 insertions(+)tools/lib/bpf/libbpf.c +0
-0
tools/lib/bpf/libbpf.c static const struct bpf_sec_def section_defs[] = { · line 8997
Quoted 7 lines
tools/lib/bpf/libbpf.c +4
-0@@ -8991,6 +8991,10 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("struct_ops.s+" ,STRUCT_OPS ,0 ,SEC_SLEEPABLE ),SEC_DEF ("sk_lookup" ,SK_LOOKUP ,BPF_SK_LOOKUP ,SEC_ATTACHABLE ),SEC_DEF ("netfilter" ,NETFILTER ,BPF_NETFILTER ,SEC_NONE ), +SEC_DEF ("qdisc/enqueue" ,QDISC ,BPF_QDISC_ENQUEUE ,SEC_ATTACHABLE_OPT ), +SEC_DEF ("qdisc/dequeue" ,QDISC ,BPF_QDISC_DEQUEUE ,SEC_ATTACHABLE_OPT ), +SEC_DEF ("qdisc/reset" ,QDISC ,BPF_QDISC_RESET ,SEC_ATTACHABLE_OPT ), + SEC_DEF("qdisc/init", QDISC, BPF_QDISC_INIT, SEC_ATTACHABLE_OPT),
seems like SEC_ATTACHABLE (or just 0) is what you want. expected_attach_type shouldn't be optional for any new program type
Quoted 7 lines
tools/lib/bpf/libbpf.c +0
-1}; int libbpf_register_prog_handler(const char *sec, --2.20.1
Re: [RFC PATCH v7 6/8] tools/libbpf: Add support for BPF_PROG_TYPE_QDISC
Quoted 12 lines · Andrii Nakryiko
Quoted 10 lines · Amery Hung
While eBPF qdisc uses NETLINK for attachment,
expected_attach_typeis required at load time to verify context access from different programs. This patch adds the section definition for this.Signed-off-by Amery Hung <[email protected]>--- tools/lib/bpf/libbpf.c | 4 ++++ 1 file changed, 4 insertions(+)
Quoted 16 lines
tools/lib/bpf/libbpf.c static const struct bpf_sec_def section_defs[] = { · line 8997
Quoted 12 lines
tools/lib/bpf/libbpf.c +4
-0@@ -8991,6 +8991,10 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("struct_ops.s+" ,STRUCT_OPS ,0 ,SEC_SLEEPABLE ),SEC_DEF ("sk_lookup" ,SK_LOOKUP ,BPF_SK_LOOKUP ,SEC_ATTACHABLE ),SEC_DEF ("netfilter" ,NETFILTER ,BPF_NETFILTER ,SEC_NONE ), +SEC_DEF ("qdisc/enqueue" ,QDISC ,BPF_QDISC_ENQUEUE ,SEC_ATTACHABLE_OPT ), +SEC_DEF ("qdisc/dequeue" ,QDISC ,BPF_QDISC_DEQUEUE ,SEC_ATTACHABLE_OPT ), +SEC_DEF ("qdisc/reset" ,QDISC ,BPF_QDISC_RESET ,SEC_ATTACHABLE_OPT ), + SEC_DEF("qdisc/init", QDISC, BPF_QDISC_INIT, SEC_ATTACHABLE_OPT),seems like
SEC_ATTACHABLE(or just 0) is what you want.expected_attach_typeshouldn't be optional for any new program type
Got it. Will change the flags to SEC_NONE.
Quoted 7 lines
Quoted 7 lines
tools/lib/bpf/libbpf.c +0
-1}; int libbpf_register_prog_handler(const char *sec, --2.20.1
RFC PATCH v7 7/8 samples/bpf: Add an example of bpf fq qdisc
In reply to Amery Hungtc_sch_fq.bpf.c
A simple bpf fair queueing (fq) qdisc that gives each flow a euqal chance to transmit data. The qdisc respects the timestamp in a skb set by an clsact rate limiter. It can also inform the rate limiter about packet drop when enabled to adjust timestamps. The implementation does not prevent hash collision of flows nor does it recycle flows.
tc_sch_fq.c
A user space program to load and attach the eBPF-based fq qdisc, which by default add the bpf fq to the loopback device, but can also add to other dev and class with '-d' and '-p' options.
To test the bpf fq qdisc with the EDT rate limiter:
$ tc qdisc add dev lo clsact $ tc filter add dev lo egress bpf obj tc_clsact_edt.bpf.o sec classifier $ ./tc_sch_fq -s
--- samples/bpf/Makefile | 8 +- samples/bpf/bpf_experimental.h | 134 +++++++ samples/bpf/tc_clsact_edt.bpf.c | 103 +++++ samples/bpf/tc_sch_fq.bpf.c | 666 ++++++++++++++++++++++++++++++++ samples/bpf/tc_sch_fq.c | 321 +++++++++++++++ 5 files changed, 1231 insertions(+), 1 deletion(-) create mode 100644 samples/bpf/bpf_experimental.h create mode 100644 samples/bpf/tc_clsact_edt.bpf.c create mode 100644 samples/bpf/tc_sch_fq.bpf.c create mode 100644 samples/bpf/tc_sch_fq.c
samples/bpf/Makefile +7 -1
@@ -46,6 +46,7 @@ tprogs-y += xdp_fwd tprogs-y += task_fd_query tprogs-y += ibumad tprogs-y += hbm +tprogs-y += tc_sch_fq # Libbpf dependencies LIBBPF_SRC = $(TOOLS_PATH)/lib/bpf
@@ -98,6 +99,7 @@ ibumad-objs := ibumad_user.o hbm-objs := hbm.o $(CGROUP_HELPERS) xdp_router_ipv4-objs := xdp_router_ipv4_user.o $(XDP_SAMPLE) +tc_sch_fq-objs := tc_sch_fq.o # Tell kbuild to always build the programs always-y := $(tprogs-y)
@@ -149,6 +151,7 @@ always-y += task_fd_query_kern.o always-y += ibumad_kern.o always-y += hbm_out_kern.o always-y += hbm_edt_kern.o +always-y += tc_sch_fq.bpf.o TPROGS_CFLAGS = $(TPROGS_USER_CFLAGS) TPROGS_LDFLAGS = $(TPROGS_USER_LDFLAGS)
@@ -195,6 +198,7 @@ TPROGLDLIBS_tracex4 += -lrt TPROGLDLIBS_trace_output += -lrt TPROGLDLIBS_map_perf_test += -lrt TPROGLDLIBS_test_overhead += -lrt +TPROGLDLIBS_tc_sch_fq += -lmnl # Allows pointing LLC/CLANG to a LLVM backend with bpf support, redefine on cmdline: # make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang
@@ -306,6 +310,7 @@ $(obj)/$(TRACE_HELPERS) $(obj)/$(CGROUP_HELPERS) $(obj)/$(XDP_SAMPLE): | libbpf_ .PHONY: libbpf_hdrs $(obj)/xdp_router_ipv4_user.o: $(obj)/xdp_router_ipv4.skel.h +$(obj)/tc_sch_fq.o: $(obj)/tc_sch_fq.skel.h $(obj)/tracex5.bpf.o: $(obj)/syscall_nrs.h $(obj)/hbm_out_kern.o: $(src)/hbm.h $(src)/hbm_kern.h
@@ -370,10 +375,11 @@ $(obj)/%.bpf.o: $(src)/%.bpf.c $(obj)/vmlinux.h $(src)/xdp_sample.bpf.h $(src)/x -I$(LIBBPF_INCLUDE) $(CLANG_SYS_INCLUDES) \ -c $(filter %.bpf.c,$^) -o $@ -LINKED_SKELS := xdp_router_ipv4.skel.h +LINKED_SKELS := xdp_router_ipv4.skel.h tc_sch_fq.skel.h clean-files += $(LINKED_SKELS) xdp_router_ipv4.skel.h-deps := xdp_router_ipv4.bpf.o xdp_sample.bpf.o +tc_sch_fq.skel.h-deps := tc_sch_fq.bpf.o LINKED_BPF_SRCS := $(patsubst %.bpf.o,%.bpf.c,$(foreach skel,$(LINKED_SKELS),$($(skel)-deps)))
samples/bpf/bpf_experimental.h new file +134 -0
134 lines not on this page · raw message
samples/bpf/tc_clsact_edt.bpf.c new file +103 -0
103 lines not on this page · raw message
samples/bpf/tc_sch_fq.bpf.c new file +666 -0
666 lines not on this page · raw message
samples/bpf/tc_sch_fq.c new file +321 -0
321 lines not on this page · raw message
Signature · 1 line
2.20.1
Daniel Borkmann
maintainerRe: [RFC PATCH v7 7/8] samples/bpf: Add an example of bpf fq qdisc
Quoted 12 lines · Amery Hung
tc_sch_fq.bpf.c
A simple bpf fair queueing (fq) qdisc that gives each flow a euqal chance to transmit data. The qdisc respects the timestamp in a skb set by an clsact rate limiter. It can also inform the rate limiter about packet drop when enabled to adjust timestamps. The implementation does not prevent hash collision of flows nor does it recycle flows.
tc_sch_fq.c
A user space program to load and attach the eBPF-based fq qdisc, which by default add the bpf fq to the loopback device, but can also add to other dev and class with '-d' and '-p' options.
Quoted 4 lines
To test the bpf fq qdisc with the EDT rate limiter:
$ tc qdisc add dev lo clsact $ tc filter add dev lo egress bpf obj tc_clsact_edt.bpf.o sec classifier $ ./tc_sch_fq -s
Would be nice if you also include a performance comparison (did you do production tests with it?) with side-by-side to native fq and if you see a delta elaborate on what would be needed to address it.
Quoted 12 lines
Signed-off-by Amery Hung <[email protected]>--- samples/bpf/Makefile | 8 +- samples/bpf/bpf_experimental.h | 134 +++++++ samples/bpf/tc_clsact_edt.bpf.c | 103 +++++ samples/bpf/tc_sch_fq.bpf.c | 666 ++++++++++++++++++++++++++++++++ samples/bpf/tc_sch_fq.c | 321 +++++++++++++++ 5 files changed, 1231 insertions(+), 1 deletion(-)create mode 100644
samples/bpf/bpf_experimental.hcreate mode 100644samples/bpf/tc_clsact_edt.bpf.ccreate mode 100644samples/bpf/tc_sch_fq.bpf.ccreate mode 100644samples/bpf/tc_sch_fq.c
Re: [RFC PATCH v7 7/8] samples/bpf: Add an example of bpf fq qdisc
Quoted 14 lines · Daniel Borkmann
Quoted 12 lines · Amery Hung
tc_sch_fq.bpf.c
A simple bpf fair queueing (fq) qdisc that gives each flow a euqal chance to transmit data. The qdisc respects the timestamp in a skb set by an clsact rate limiter. It can also inform the rate limiter about packet drop when enabled to adjust timestamps. The implementation does not prevent hash collision of flows nor does it recycle flows.
tc_sch_fq.c
A user space program to load and attach the eBPF-based fq qdisc, which by default add the bpf fq to the loopback device, but can also add to other dev and class with '-d' and '-p' options.
Quoted 8 lines
Quoted 4 lines
To test the bpf fq qdisc with the EDT rate limiter:
$ tc qdisc add dev lo clsact $ tc filter add dev lo egress bpf obj tc_clsact_edt.bpf.o sec classifier $ ./tc_sch_fq -sWould be nice if you also include a performance comparison (did you do production tests with it?) with side-by-side to native fq and if you see a delta elaborate on what would be needed to address it.
I did a simple test by adding a fq to the loopback device and then sending a single stream traffic via iperf. The bpf implementation of fq achieves 90% throughput compared with the native one.
I think the overhead mainly comes from allocating bpf objects (struct skb_node) to store skb kptrs. This part can be removed if bpf list/rbtree recognizes skb->list/rbnode. On the kfunc implementation side, I think we can do it by saving struct bpf_rb_node_kern into skb->rb_node and skb->cb. I haven't looked into the verifier to see what needs to be done.
I will move the test cases from samples to selftests and include more testing in the next patchset.
Quoted 13 lines
Quoted 13 lines
Signed-off-by Amery Hung <[email protected]>--- samples/bpf/Makefile | 8 +- samples/bpf/bpf_experimental.h | 134 +++++++ samples/bpf/tc_clsact_edt.bpf.c | 103 +++++ samples/bpf/tc_sch_fq.bpf.c | 666 ++++++++++++++++++++++++++++++++ samples/bpf/tc_sch_fq.c | 321 +++++++++++++++ 5 files changed, 1231 insertions(+), 1 deletion(-)create mode 100644
samples/bpf/bpf_experimental.hcreate mode 100644samples/bpf/tc_clsact_edt.bpf.ccreate mode 100644samples/bpf/tc_sch_fq.bpf.ccreate mode 100644samples/bpf/tc_sch_fq.c
RFC PATCH v7 8/8 samples/bpf: Add an example of bpf netem qdisc
In reply to Amery Hungtc_sch_netem.bpf.c
A simple bpf network emulator (netem) qdisc that simulates packet drops, loss, and delay. The qdisc shares the state machine of Gilbert-Elliott model via a eBPF map when it is added to multiple tx queues.
tc_sch_netem.c
A user space program to load and attach the eBPF-based netem qdisc, which by default add the bpf fq to the loopback device, but can also add to other dev and class with '-d' and '-p' options.
To test mq + netem with shared state machine:
$ tc qdisc add dev ens5 root handle 1: mq $ ./tc_sch_netem -d ens5 -p 1:1 -h 801 -s $ ./tc_sch_netem -d ens5 -p 1:2 -h 802 -s $ ./tc_sch_netem -d ens5 -p 1:3 -h 803 -s $ ./tc_sch_netem -d ens5 -p 1:4 -h 804 -s
--- samples/bpf/Makefile | 8 +- samples/bpf/tc_sch_netem.bpf.c | 256 ++++++++++++++++++++++++ samples/bpf/tc_sch_netem.c | 347 +++++++++++++++++++++++++++++++++ 3 files changed, 610 insertions(+), 1 deletion(-) create mode 100644 samples/bpf/tc_sch_netem.bpf.c create mode 100644 samples/bpf/tc_sch_netem.c
samples/bpf/Makefile +7 -1
@@ -47,6 +47,7 @@ tprogs-y += task_fd_query tprogs-y += ibumad tprogs-y += hbm tprogs-y += tc_sch_fq +tprogs-y += tc_sch_netem # Libbpf dependencies LIBBPF_SRC = $(TOOLS_PATH)/lib/bpf
@@ -100,6 +101,7 @@ hbm-objs := hbm.o $(CGROUP_HELPERS) xdp_router_ipv4-objs := xdp_router_ipv4_user.o $(XDP_SAMPLE) tc_sch_fq-objs := tc_sch_fq.o +tc_sch_netem-objs := tc_sch_netem.o # Tell kbuild to always build the programs always-y := $(tprogs-y)
@@ -152,6 +154,7 @@ always-y += ibumad_kern.o always-y += hbm_out_kern.o always-y += hbm_edt_kern.o always-y += tc_sch_fq.bpf.o +always-y += tc_sch_netem.bpf.o TPROGS_CFLAGS = $(TPROGS_USER_CFLAGS) TPROGS_LDFLAGS = $(TPROGS_USER_LDFLAGS)
@@ -199,6 +202,7 @@ TPROGLDLIBS_trace_output += -lrt TPROGLDLIBS_map_perf_test += -lrt TPROGLDLIBS_test_overhead += -lrt TPROGLDLIBS_tc_sch_fq += -lmnl +TPROGLDLIBS_tc_sch_netem += -lmnl # Allows pointing LLC/CLANG to a LLVM backend with bpf support, redefine on cmdline: # make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang
@@ -311,6 +315,7 @@ $(obj)/$(TRACE_HELPERS) $(obj)/$(CGROUP_HELPERS) $(obj)/$(XDP_SAMPLE): | libbpf_ $(obj)/xdp_router_ipv4_user.o: $(obj)/xdp_router_ipv4.skel.h $(obj)/tc_sch_fq.o: $(obj)/tc_sch_fq.skel.h +$(obj)/tc_sch_netem.o: $(obj)/tc_sch_netem.skel.h $(obj)/tracex5.bpf.o: $(obj)/syscall_nrs.h $(obj)/hbm_out_kern.o: $(src)/hbm.h $(src)/hbm_kern.h
@@ -375,11 +380,12 @@ $(obj)/%.bpf.o: $(src)/%.bpf.c $(obj)/vmlinux.h $(src)/xdp_sample.bpf.h $(src)/x -I$(LIBBPF_INCLUDE) $(CLANG_SYS_INCLUDES) \ -c $(filter %.bpf.c,$^) -o $@ -LINKED_SKELS := xdp_router_ipv4.skel.h tc_sch_fq.skel.h +LINKED_SKELS := xdp_router_ipv4.skel.h tc_sch_fq.skel.h tc_sch_netem.skel.h clean-files += $(LINKED_SKELS) xdp_router_ipv4.skel.h-deps := xdp_router_ipv4.bpf.o xdp_sample.bpf.o tc_sch_fq.skel.h-deps := tc_sch_fq.bpf.o +tc_sch_netem.skel.h-deps := tc_sch_netem.bpf.o LINKED_BPF_SRCS := $(patsubst %.bpf.o,%.bpf.c,$(foreach skel,$(LINKED_SKELS),$($(skel)-deps)))
samples/bpf/tc_sch_netem.bpf.c new file +256 -0
256 lines not on this page · raw message
samples/bpf/tc_sch_netem.c new file +347 -0
347 lines not on this page · raw message
Signature · 1 line
2.20.1
Quoted 44 lines · Amery Hung
Hi,
I am continuing the work of ebpf-based Qdisc based on Cong’s previous RFC. The followings are some use cases of eBPF Qdisc:
1. Allow customizing Qdiscs in an easier way. So that people don't have to write a complete Qdisc kernel module just to experiment some new queuing theory.
2. Solve EDT's problem. EDT calcuates the "tokens" in clsact which is before enqueue, it is impossible to adjust those "tokens" after packets get dropped in enqueue. With eBPF Qdisc, it is easy to be solved with a shared map between clsact and
sch_bpf.3. Replace qevents, as now the user gains much more control over the skb and queues.
4. Provide a new way to reuse TC filters. Currently TC relies on filter chain and block to reuse the TC filters, but they are too complicated to understand. With eBPF helper
bpf_skb_tc_classify(), we can invoke TC filters on _any_ Qdisc (even on a different netdev) to do the classification.5. Potentially pave a way for ingress to queue packets, although current implementation is still only for egress.
I’ve combed through previous comments and appreciated the feedbacks. Some major changes in this RFC is the use of kptr to skb to maintain the validility of skb during its lifetime in the Qdisc, dropping rbtree maps, and the inclusion of two examples.
Some questions for discussion:
1. We now pass a trusted kptr of
sk_buffto the program instead of__sk_buff. This makes most helpers using__sk_buffincompatible with eBPF qdisc. An alternative is to still use__sk_buffin the context and usebpf_cast_to_kern_ctx()to acquire the kptr. However, this can only be applied to enqueue program, since in dequeue program skbs do not come from ctx but kptrs exchanged out of maps (i.e., there is no__sk_buff). Any suggestion for making skb kptr and helper functions compatible?2. The current patchset uses netlink. Do we also want to use
bpf_linkfor attachment?
[..]
Quoted 5 lines
3. People have suggested
struct_ops. We chose not to usestruct_opssince users might want to create multiple bpf qdiscs with different implementations. Currentstruct_opsattachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong.
I still feel like it deserves at leasta try. Maybe we can find some potential path where struct_ops can allow different implementations (Martin probably has some ideas about that). I looked at the bpf qdisc itself and it doesn't really have anything complicated (besides trying to play nicely with other tc classes/actions, but I'm not sure how relevant that is).
With struct_ops you can also get your (2) addressed.
Daniel Borkmann
maintainerQuoted 54 lines · Stanislav Fomichev
Quoted 44 lines · Amery Hung
Hi,
I am continuing the work of ebpf-based Qdisc based on Cong’s previous RFC. The followings are some use cases of eBPF Qdisc:
1. Allow customizing Qdiscs in an easier way. So that people don't have to write a complete Qdisc kernel module just to experiment some new queuing theory.
2. Solve EDT's problem. EDT calcuates the "tokens" in clsact which is before enqueue, it is impossible to adjust those "tokens" after packets get dropped in enqueue. With eBPF Qdisc, it is easy to be solved with a shared map between clsact and
sch_bpf.3. Replace qevents, as now the user gains much more control over the skb and queues.
4. Provide a new way to reuse TC filters. Currently TC relies on filter chain and block to reuse the TC filters, but they are too complicated to understand. With eBPF helper
bpf_skb_tc_classify(), we can invoke TC filters on _any_ Qdisc (even on a different netdev) to do the classification.5. Potentially pave a way for ingress to queue packets, although current implementation is still only for egress.
I’ve combed through previous comments and appreciated the feedbacks. Some major changes in this RFC is the use of kptr to skb to maintain the validility of skb during its lifetime in the Qdisc, dropping rbtree maps, and the inclusion of two examples.
Some questions for discussion:
1. We now pass a trusted kptr of
sk_buffto the program instead of__sk_buff. This makes most helpers using__sk_buffincompatible with eBPF qdisc. An alternative is to still use__sk_buffin the context and usebpf_cast_to_kern_ctx()to acquire the kptr. However, this can only be applied to enqueue program, since in dequeue program skbs do not come from ctx but kptrs exchanged out of maps (i.e., there is no__sk_buff). Any suggestion for making skb kptr and helper functions compatible?2. The current patchset uses netlink. Do we also want to use
bpf_linkfor attachment?[..]
Quoted 5 lines
3. People have suggested
struct_ops. We chose not to usestruct_opssince users might want to create multiple bpf qdiscs with different implementations. Currentstruct_opsattachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong.
Quoted 5 lines
I still feel like it deserves at leasta try. Maybe we can find some potential path where
struct_opscan allow different implementations (Martin probably has some ideas about that). I looked at the bpf qdisc itself and it doesn't really have anything complicated (besides trying to play nicely with other tc classes/actions, but I'm not sure how relevant that is).
Plus it's also not used in the two sample implementations, given you can implement this as part of the enqueue operation in bpf. It would make sense to drop the kfunc from the set. One other note.. the BPF samples have been bitrotting for quite a while, please convert this into a proper BPF selftest so that BPF CI can run this.
Quoted 1 line
With
struct_opsyou can also get your (2) addressed.
+1
Thanks,
Daniel
Jamal Hadi Salim
maintainerQuoted 55 lines · Stanislav Fomichev
Quoted 44 lines · Amery Hung
Hi,
I am continuing the work of ebpf-based Qdisc based on Cong’s previous RFC. The followings are some use cases of eBPF Qdisc:
1. Allow customizing Qdiscs in an easier way. So that people don't have to write a complete Qdisc kernel module just to experiment some new queuing theory.
2. Solve EDT's problem. EDT calcuates the "tokens" in clsact which is before enqueue, it is impossible to adjust those "tokens" after packets get dropped in enqueue. With eBPF Qdisc, it is easy to be solved with a shared map between clsact and
sch_bpf.3. Replace qevents, as now the user gains much more control over the skb and queues.
4. Provide a new way to reuse TC filters. Currently TC relies on filter chain and block to reuse the TC filters, but they are too complicated to understand. With eBPF helper
bpf_skb_tc_classify(), we can invoke TC filters on _any_ Qdisc (even on a different netdev) to do the classification.5. Potentially pave a way for ingress to queue packets, although current implementation is still only for egress.
I’ve combed through previous comments and appreciated the feedbacks. Some major changes in this RFC is the use of kptr to skb to maintain the validility of skb during its lifetime in the Qdisc, dropping rbtree maps, and the inclusion of two examples.
Some questions for discussion:
1. We now pass a trusted kptr of
sk_buffto the program instead of__sk_buff. This makes most helpers using__sk_buffincompatible with eBPF qdisc. An alternative is to still use__sk_buffin the context and usebpf_cast_to_kern_ctx()to acquire the kptr. However, this can only be applied to enqueue program, since in dequeue program skbs do not come from ctx but kptrs exchanged out of maps (i.e., there is no__sk_buff). Any suggestion for making skb kptr and helper functions compatible?2. The current patchset uses netlink. Do we also want to use
bpf_linkfor attachment?[..]
Quoted 5 lines
3. People have suggested
struct_ops. We chose not to usestruct_opssince users might want to create multiple bpf qdiscs with different implementations. Currentstruct_opsattachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong.
Quoted 5 lines
I still feel like it deserves at leasta try. Maybe we can find some potential path where
struct_opscan allow different implementations (Martin probably has some ideas about that). I looked at the bpf qdisc itself and it doesn't really have anything complicated (besides trying to play nicely with other tc classes/actions, but I'm not sure how relevant that is).
Are you suggesting that it is a nuisance to integrate with the existing infra? I would consider it being a lot more than "trying to play nicely". Besides, it's a kfunc and people will not be forced to use it.
cheers,
jamal
Quoted 1 line
With
struct_opsyou can also get your (2) addressed.
Daniel Borkmann
maintainerQuoted 61 lines · Jamal Hadi Salim
Quoted 54 lines · Stanislav Fomichev
Quoted 44 lines · Amery Hung
Hi,
I am continuing the work of ebpf-based Qdisc based on Cong’s previous RFC. The followings are some use cases of eBPF Qdisc:
1. Allow customizing Qdiscs in an easier way. So that people don't have to write a complete Qdisc kernel module just to experiment some new queuing theory.
2. Solve EDT's problem. EDT calcuates the "tokens" in clsact which is before enqueue, it is impossible to adjust those "tokens" after packets get dropped in enqueue. With eBPF Qdisc, it is easy to be solved with a shared map between clsact and
sch_bpf.3. Replace qevents, as now the user gains much more control over the skb and queues.
4. Provide a new way to reuse TC filters. Currently TC relies on filter chain and block to reuse the TC filters, but they are too complicated to understand. With eBPF helper
bpf_skb_tc_classify(), we can invoke TC filters on _any_ Qdisc (even on a different netdev) to do the classification.5. Potentially pave a way for ingress to queue packets, although current implementation is still only for egress.
I’ve combed through previous comments and appreciated the feedbacks. Some major changes in this RFC is the use of kptr to skb to maintain the validility of skb during its lifetime in the Qdisc, dropping rbtree maps, and the inclusion of two examples.
Some questions for discussion:
1. We now pass a trusted kptr of
sk_buffto the program instead of__sk_buff. This makes most helpers using__sk_buffincompatible with eBPF qdisc. An alternative is to still use__sk_buffin the context and usebpf_cast_to_kern_ctx()to acquire the kptr. However, this can only be applied to enqueue program, since in dequeue program skbs do not come from ctx but kptrs exchanged out of maps (i.e., there is no__sk_buff). Any suggestion for making skb kptr and helper functions compatible?2. The current patchset uses netlink. Do we also want to use
bpf_linkfor attachment?[..]
Quoted 5 lines
3. People have suggested
struct_ops. We chose not to usestruct_opssince users might want to create multiple bpf qdiscs with different implementations. Currentstruct_opsattachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong.Quoted 5 lines
I still feel like it deserves at leasta try. Maybe we can find some potential path where
struct_opscan allow different implementations (Martin probably has some ideas about that). I looked at the bpf qdisc itself and it doesn't really have anything complicated (besides trying to play nicely with other tc classes/actions, but I'm not sure how relevant that is).
Quoted 4 lines
Are you suggesting that it is a nuisance to integrate with the existing infra? I would consider it being a lot more than "trying to play nicely". Besides, it's a kfunc and people will not be forced to use it.
What's the use case? If you already go that route to implement your own qdisc, why is there a need to take the performane hit and go all the way into old style cls/act infra when it can be done in a more straight forward way natively? For the vast majority of cases this will be some very lightweight classification anyway (if not outsourced to tc egress given the lock). If there is a concrete production need, it could be added, otherwise if there is no immediate use case which cannot be solved otherwise I would not add unnecessary kfuncs.
Cheers,
Daniel
Jamal Hadi Salim
maintainerQuoted 63 lines · Daniel Borkmann
Quoted 61 lines · Jamal Hadi Salim
Quoted 59 lines · Stanislav Fomichev
Quoted 44 lines · Amery Hung
Hi,
I am continuing the work of ebpf-based Qdisc based on Cong’s previous RFC. The followings are some use cases of eBPF Qdisc:
1. Allow customizing Qdiscs in an easier way. So that people don't have to write a complete Qdisc kernel module just to experiment some new queuing theory.
2. Solve EDT's problem. EDT calcuates the "tokens" in clsact which is before enqueue, it is impossible to adjust those "tokens" after packets get dropped in enqueue. With eBPF Qdisc, it is easy to be solved with a shared map between clsact and
sch_bpf.3. Replace qevents, as now the user gains much more control over the skb and queues.
4. Provide a new way to reuse TC filters. Currently TC relies on filter chain and block to reuse the TC filters, but they are too complicated to understand. With eBPF helper
bpf_skb_tc_classify(), we can invoke TC filters on _any_ Qdisc (even on a different netdev) to do the classification.5. Potentially pave a way for ingress to queue packets, although current implementation is still only for egress.
I’ve combed through previous comments and appreciated the feedbacks. Some major changes in this RFC is the use of kptr to skb to maintain the validility of skb during its lifetime in the Qdisc, dropping rbtree maps, and the inclusion of two examples.
Some questions for discussion:
1. We now pass a trusted kptr of
sk_buffto the program instead of__sk_buff. This makes most helpers using__sk_buffincompatible with eBPF qdisc. An alternative is to still use__sk_buffin the context and usebpf_cast_to_kern_ctx()to acquire the kptr. However, this can only be applied to enqueue program, since in dequeue program skbs do not come from ctx but kptrs exchanged out of maps (i.e., there is no__sk_buff). Any suggestion for making skb kptr and helper functions compatible?2. The current patchset uses netlink. Do we also want to use
bpf_linkfor attachment?[..]
Quoted 5 lines
3. People have suggested
struct_ops. We chose not to usestruct_opssince users might want to create multiple bpf qdiscs with different implementations. Currentstruct_opsattachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong.I still feel like it deserves at leasta try. Maybe we can find some potential path where
struct_opscan allow different implementations (Martin probably has some ideas about that). I looked at the bpf qdisc itself and it doesn't really have anything complicated (besides trying to play nicely with other tc classes/actions, but I'm not sure how relevant that is).
Quoted 6 lines
Quoted 4 lines
Are you suggesting that it is a nuisance to integrate with the existing infra? I would consider it being a lot more than "trying to play nicely". Besides, it's a kfunc and people will not be forced to use it.
What's the use case?
What's the use case for enabling existing infra to work? Sure, let's rewrite everything from scratch in ebpf. And then introduce new tooling which well funded companies are capable of owning the right resources to build and manage. Open source is about choices and sharing and this is about choices and sharing.
Quoted 4 lines
If you already go that route to implement your own
qdisc, why is there a need to take the performane hit and go all the way into old style cls/act infra when it can be done in a more straight forward way natively?
Who is forcing you to use the kfunc? This is about choice. What is ebpf these days anyways? Is it a) a programming environment or
b) is it the only way to do things? I see it as available infra i.e #a not as the answer looking for a question. IOW, as something we can use to build the infra we need and use kfuncs when it makes sense. Not everybody has infinite resources to keep hacking things into ebpf.
Quoted 5 lines
For the vast majority of cases this will be some
very lightweight classification anyway (if not outsourced to tc egress given the lock). If there is a concrete production need, it could be added, otherwise if there is no immediate use case which cannot be solved otherwise I would not add unnecessary kfuncs.
"Unnecessary" is really your view.
cheers,
jamal
Quoted 2 lines
Cheers,
Daniel
Daniel Borkmann
maintainerQuoted 88 lines · Jamal Hadi Salim
Quoted 62 lines · Daniel Borkmann
Quoted 61 lines · Jamal Hadi Salim
Quoted 59 lines · Stanislav Fomichev
Quoted 44 lines · Amery Hung
Hi,
I am continuing the work of ebpf-based Qdisc based on Cong’s previous RFC. The followings are some use cases of eBPF Qdisc:
1. Allow customizing Qdiscs in an easier way. So that people don't have to write a complete Qdisc kernel module just to experiment some new queuing theory.
2. Solve EDT's problem. EDT calcuates the "tokens" in clsact which is before enqueue, it is impossible to adjust those "tokens" after packets get dropped in enqueue. With eBPF Qdisc, it is easy to be solved with a shared map between clsact and
sch_bpf.3. Replace qevents, as now the user gains much more control over the skb and queues.
4. Provide a new way to reuse TC filters. Currently TC relies on filter chain and block to reuse the TC filters, but they are too complicated to understand. With eBPF helper
bpf_skb_tc_classify(), we can invoke TC filters on _any_ Qdisc (even on a different netdev) to do the classification.5. Potentially pave a way for ingress to queue packets, although current implementation is still only for egress.
I’ve combed through previous comments and appreciated the feedbacks. Some major changes in this RFC is the use of kptr to skb to maintain the validility of skb during its lifetime in the Qdisc, dropping rbtree maps, and the inclusion of two examples.
Some questions for discussion:
1. We now pass a trusted kptr of
sk_buffto the program instead of__sk_buff. This makes most helpers using__sk_buffincompatible with eBPF qdisc. An alternative is to still use__sk_buffin the context and usebpf_cast_to_kern_ctx()to acquire the kptr. However, this can only be applied to enqueue program, since in dequeue program skbs do not come from ctx but kptrs exchanged out of maps (i.e., there is no__sk_buff). Any suggestion for making skb kptr and helper functions compatible?2. The current patchset uses netlink. Do we also want to use
bpf_linkfor attachment?[..]
Quoted 5 lines
3. People have suggested
struct_ops. We chose not to usestruct_opssince users might want to create multiple bpf qdiscs with different implementations. Currentstruct_opsattachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong.I still feel like it deserves at leasta try. Maybe we can find some potential path where
struct_opscan allow different implementations (Martin probably has some ideas about that). I looked at the bpf qdisc itself and it doesn't really have anything complicated (besides trying to play nicely with other tc classes/actions, but I'm not sure how relevant that is).Quoted 6 lines
Quoted 4 lines
Are you suggesting that it is a nuisance to integrate with the existing infra? I would consider it being a lot more than "trying to play nicely". Besides, it's a kfunc and people will not be forced to use it.
What's the use case?
What's the use case for enabling existing infra to work? Sure, let's rewrite everything from scratch in ebpf. And then introduce new tooling which well funded companies are capable of owning the right resources to build and manage. Open source is about choices and sharing and this is about choices and sharing.
Quoted 4 lines
If you already go that route to implement your own
qdisc, why is there a need to take the performane hit and go all the way into old style cls/act infra when it can be done in a more straight forward way natively?Who is forcing you to use the kfunc? This is about choice. What is ebpf these days anyways? Is it a) a programming environment or
b) is it the only way to do things? I see it as available infra i.e #a not as the answer looking for a question. IOW, as something we can use to build the infra we need and use kfuncs when it makes sense. Not everybody has infinite resources to keep hacking things into ebpf.
Quoted 7 lines
Quoted 5 lines
For the vast majority of cases this will be some
very lightweight classification anyway (if not outsourced to tc egress given the lock). If there is a concrete production need, it could be added, otherwise if there is no immediate use case which cannot be solved otherwise I would not add unnecessary kfuncs."Unnecessary" is really your view.
Looks like we're talking past each other? If there is no plan to use it in production (I assume Amery would be able to answer?), why add it right now to the initial series, only to figure out later on (worst case in few years) when the time comes that the kfunc does not fit the actual need? You've probably seen the life cycle doc (Documentation/bpf/kfuncs.rst) and while changes can be made, they should still be mindful about potential breakages the longer it's out in the wild, hence my question if it's planning to be used given it wasn't in the samples.
Thanks,
Daniel
Quoted 98 lines · Daniel Borkmann
Quoted 88 lines · Jamal Hadi Salim
Quoted 62 lines · Daniel Borkmann
Quoted 61 lines · Jamal Hadi Salim
Quoted 59 lines · Stanislav Fomichev
Quoted 44 lines · Amery Hung
Hi,
I am continuing the work of ebpf-based Qdisc based on Cong’s previous RFC. The followings are some use cases of eBPF Qdisc:
1. Allow customizing Qdiscs in an easier way. So that people don't have to write a complete Qdisc kernel module just to experiment some new queuing theory.
2. Solve EDT's problem. EDT calcuates the "tokens" in clsact which is before enqueue, it is impossible to adjust those "tokens" after packets get dropped in enqueue. With eBPF Qdisc, it is easy to be solved with a shared map between clsact and
sch_bpf.3. Replace qevents, as now the user gains much more control over the skb and queues.
4. Provide a new way to reuse TC filters. Currently TC relies on filter chain and block to reuse the TC filters, but they are too complicated to understand. With eBPF helper
bpf_skb_tc_classify(), we can invoke TC filters on _any_ Qdisc (even on a different netdev) to do the classification.5. Potentially pave a way for ingress to queue packets, although current implementation is still only for egress.
I’ve combed through previous comments and appreciated the feedbacks. Some major changes in this RFC is the use of kptr to skb to maintain the validility of skb during its lifetime in the Qdisc, dropping rbtree maps, and the inclusion of two examples.
Some questions for discussion:
1. We now pass a trusted kptr of
sk_buffto the program instead of__sk_buff. This makes most helpers using__sk_buffincompatible with eBPF qdisc. An alternative is to still use__sk_buffin the context and usebpf_cast_to_kern_ctx()to acquire the kptr. However, this can only be applied to enqueue program, since in dequeue program skbs do not come from ctx but kptrs exchanged out of maps (i.e., there is no__sk_buff). Any suggestion for making skb kptr and helper functions compatible?2. The current patchset uses netlink. Do we also want to use
bpf_linkfor attachment?[..]
Quoted 5 lines
3. People have suggested
struct_ops. We chose not to usestruct_opssince users might want to create multiple bpf qdiscs with different implementations. Currentstruct_opsattachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong.I still feel like it deserves at leasta try. Maybe we can find some potential path where
struct_opscan allow different implementations (Martin probably has some ideas about that). I looked at the bpf qdisc itself and it doesn't really have anything complicated (besides trying to play nicely with other tc classes/actions, but I'm not sure how relevant that is).Quoted 6 lines
Quoted 4 lines
Are you suggesting that it is a nuisance to integrate with the existing infra? I would consider it being a lot more than "trying to play nicely". Besides, it's a kfunc and people will not be forced to use it.
What's the use case?
What's the use case for enabling existing infra to work? Sure, let's rewrite everything from scratch in ebpf. And then introduce new tooling which well funded companies are capable of owning the right resources to build and manage. Open source is about choices and sharing and this is about choices and sharing.
Quoted 4 lines
If you already go that route to implement your own
qdisc, why is there a need to take the performane hit and go all the way into old style cls/act infra when it can be done in a more straight forward way natively?Who is forcing you to use the kfunc? This is about choice. What is ebpf these days anyways? Is it a) a programming environment or
b) is it the only way to do things? I see it as available infra i.e #a not as the answer looking for a question. IOW, as something we can use to build the infra we need and use kfuncs when it makes sense. Not everybody has infinite resources to keep hacking things into ebpf.Quoted 7 lines
Quoted 5 lines
For the vast majority of cases this will be some
very lightweight classification anyway (if not outsourced to tc egress given the lock). If there is a concrete production need, it could be added, otherwise if there is no immediate use case which cannot be solved otherwise I would not add unnecessary kfuncs."Unnecessary" is really your view.
Quoted 9 lines
Looks like we're talking past each other? If there is no plan to use it in production (I assume Amery would be able to answer?), why add it right now to the initial series, only to figure out later on (worst case in few years) when the time comes that the kfunc does not fit the actual need? You've probably seen the life cycle doc (
Documentation/bpf/kfuncs.rst) and while changes can be made, they should still be mindful about potential breakages the longer it's out in the wild, hence my question if it's planning to be used given it wasn't in the samples.
We would like to reuse existing TC filters. Like Jamal says, changing filter rules in production can be done easily with existing tooling. Besides, when the user is only interested in exploring scheduling algorithms but not classifying traffics, they don't need to replicate the filter again in bpf. I can add bpf_skb_tc_classify() test cases in the next series if that helps.
Thanks,
Amery
Quoted 2 lines
Thanks,
Daniel
Daniel Borkmann
maintainerQuoted 107 lines · Amery Hung
Quoted 97 lines · Daniel Borkmann
Quoted 88 lines · Jamal Hadi Salim
Quoted 62 lines · Daniel Borkmann
Quoted 61 lines · Jamal Hadi Salim
Quoted 59 lines · Stanislav Fomichev
Quoted 44 lines · Amery Hung
Hi,
I am continuing the work of ebpf-based Qdisc based on Cong’s previous RFC. The followings are some use cases of eBPF Qdisc:
1. Allow customizing Qdiscs in an easier way. So that people don't have to write a complete Qdisc kernel module just to experiment some new queuing theory.
2. Solve EDT's problem. EDT calcuates the "tokens" in clsact which is before enqueue, it is impossible to adjust those "tokens" after packets get dropped in enqueue. With eBPF Qdisc, it is easy to be solved with a shared map between clsact and
sch_bpf.3. Replace qevents, as now the user gains much more control over the skb and queues.
4. Provide a new way to reuse TC filters. Currently TC relies on filter chain and block to reuse the TC filters, but they are too complicated to understand. With eBPF helper
bpf_skb_tc_classify(), we can invoke TC filters on _any_ Qdisc (even on a different netdev) to do the classification.5. Potentially pave a way for ingress to queue packets, although current implementation is still only for egress.
I’ve combed through previous comments and appreciated the feedbacks. Some major changes in this RFC is the use of kptr to skb to maintain the validility of skb during its lifetime in the Qdisc, dropping rbtree maps, and the inclusion of two examples.
Some questions for discussion:
1. We now pass a trusted kptr of
sk_buffto the program instead of__sk_buff. This makes most helpers using__sk_buffincompatible with eBPF qdisc. An alternative is to still use__sk_buffin the context and usebpf_cast_to_kern_ctx()to acquire the kptr. However, this can only be applied to enqueue program, since in dequeue program skbs do not come from ctx but kptrs exchanged out of maps (i.e., there is no__sk_buff). Any suggestion for making skb kptr and helper functions compatible?2. The current patchset uses netlink. Do we also want to use
bpf_linkfor attachment?[..]
Quoted 5 lines
3. People have suggested
struct_ops. We chose not to usestruct_opssince users might want to create multiple bpf qdiscs with different implementations. Currentstruct_opsattachment model does not seem to support replacing only functions of a specific instance of a module, but I might be wrong.I still feel like it deserves at leasta try. Maybe we can find some potential path where
struct_opscan allow different implementations (Martin probably has some ideas about that). I looked at the bpf qdisc itself and it doesn't really have anything complicated (besides trying to play nicely with other tc classes/actions, but I'm not sure how relevant that is).Quoted 6 lines
Quoted 4 lines
Are you suggesting that it is a nuisance to integrate with the existing infra? I would consider it being a lot more than "trying to play nicely". Besides, it's a kfunc and people will not be forced to use it.
What's the use case?
What's the use case for enabling existing infra to work? Sure, let's rewrite everything from scratch in ebpf. And then introduce new tooling which well funded companies are capable of owning the right resources to build and manage. Open source is about choices and sharing and this is about choices and sharing.
Quoted 4 lines
If you already go that route to implement your own
qdisc, why is there a need to take the performane hit and go all the way into old style cls/act infra when it can be done in a more straight forward way natively?Who is forcing you to use the kfunc? This is about choice. What is ebpf these days anyways? Is it a) a programming environment or
b) is it the only way to do things? I see it as available infra i.e #a not as the answer looking for a question. IOW, as something we can use to build the infra we need and use kfuncs when it makes sense. Not everybody has infinite resources to keep hacking things into ebpf.Quoted 7 lines
Quoted 5 lines
For the vast majority of cases this will be some
very lightweight classification anyway (if not outsourced to tc egress given the lock). If there is a concrete production need, it could be added, otherwise if there is no immediate use case which cannot be solved otherwise I would not add unnecessary kfuncs."Unnecessary" is really your view.
Quoted 8 lines
Looks like we're talking past each other? If there is no plan to use it in production (I assume Amery would be able to answer?), why add it right now to the initial series, only to figure out later on (worst case in few years) when the time comes that the kfunc does not fit the actual need? You've probably seen the life cycle doc (
Documentation/bpf/kfuncs.rst) and while changes can be made, they should still be mindful about potential breakages the longer it's out in the wild, hence my question if it's planning to be used given it wasn't in the samples.
Quoted 6 lines
We would like to reuse existing TC filters. Like Jamal says, changing filter rules in production can be done easily with existing tooling. Besides, when the user is only interested in exploring scheduling algorithms but not classifying traffics, they don't need to replicate the filter again in bpf. I can add
bpf_skb_tc_classify()test cases in the next series if that helps.
In that case, please add a BPF selftest for exercising the kfunc, and also expand the commit description with the above rationale.
Thanks,
Daniel