bpf: getsockopt and setsockopt hooks
Subsystems (11)
v4 cover bpf: getsockopt and setsockopt hooks
This series implements two new per-cgroup hooks: getsockopt and setsockopt along with a new sockopt program type. The idea is pretty similar to recently introduced cgroup sysctl hooks, but implementation is simpler (no need to convert to/from strings).
What this can be applied to:
* move business logic of what tos/priority/etc can be set by containers (either pass or reject)
* handle existing options (or introduce new ones) differently by propagating some information in cgroup/socket local storage
Compared to a simple syscall/{g,s}etsockopt tracepoint, those hooks are context aware. Meaning, they can access underlying socket and use cgroup and socket local storage.
Stanislav Fomichev (8):
bpf: implement getsockopt and setsockopt hooks
bpf: sync bpf.h to tools/
libbpf: support sockopt hooks
selftests/bpf: test sockopt section name selftests/bpf: add sockopt test
selftests/bpf: add sockopt test that exercises sk helpers
bpf: add sockopt documentation
bpftool: support cgroup sockopt
Documentation/bpf/index.rst | 1 + Documentation/bpf/prog_cgroup_sockopt.rst | 39 + include/linux/bpf-cgroup.h | 29 + include/linux/bpf.h | 45 + include/linux/bpf_types.h | 1 + include/linux/filter.h | 13 + include/uapi/linux/bpf.h | 13 + kernel/bpf/cgroup.c | 262 ++++++ kernel/bpf/core.c | 9 + kernel/bpf/syscall.c | 19 + kernel/bpf/verifier.c | 15 + net/core/filter.c | 2 +- net/socket.c | 18 + .../bpftool/Documentation/bpftool-cgroup.rst | 7 +- .../bpftool/Documentation/bpftool-prog.rst | 2 +- tools/bpf/bpftool/bash-completion/bpftool | 8 +- tools/bpf/bpftool/cgroup.c | 5 +- tools/bpf/bpftool/main.h | 1 + tools/bpf/bpftool/prog.c | 3 +- tools/include/uapi/linux/bpf.h | 14 + tools/lib/bpf/libbpf.c | 5 + tools/lib/bpf/libbpf_probes.c | 1 + tools/testing/selftests/bpf/.gitignore | 2 + tools/testing/selftests/bpf/Makefile | 4 +- .../testing/selftests/bpf/progs/sockopt_sk.c | 67 ++ .../selftests/bpf/test_section_names.c | 10 + tools/testing/selftests/bpf/test_sockopt.c | 773 ++++++++++++++++++ tools/testing/selftests/bpf/test_sockopt_sk.c | 156 ++++ 28 files changed, 1514 insertions(+), 10 deletions(-) create mode 100644 Documentation/bpf/prog_cgroup_sockopt.rst create mode 100644 tools/testing/selftests/bpf/progs/sockopt_sk.c create mode 100644 tools/testing/selftests/bpf/test_sockopt.c create mode 100644 tools/testing/selftests/bpf/test_sockopt_sk.c
Signature · 1 line
2.22.0.rc2.383.gf4fbbf30c2-goog
PATCH v4 1/8 bpf: implement getsockopt and setsockopt hooks
Implement new BPF_PROG_TYPE_CGROUP_SOCKOPT program type and BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
BPF_CGROUP_SETSOCKOPT get a read-only view of the setsockopt arguments. BPF_CGROUP_GETSOCKOPT can modify the supplied buffer.
Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.
The buffer memory is pre-allocated (because I don't think there is a precedent for working with __user memory from bpf). This might be slow to do for each {s,g}etsockopt call, that's why I've added __cgroup_bpf_prog_array_is_empty that exits early if there is nothing attached to a cgroup. Note, however, that there is a race between __cgroup_bpf_prog_array_is_empty and BPF_PROG_RUN_ARRAY where cgroup program layout might have changed; this should not be a problem because in general there is a race between multiple calls to {s,g}etsocktop and user adding/removing bpf progs from a cgroup.
The return code of the BPF program is handled as follows:
* 0: EPERM
* 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
* 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF prog exits
v4:
* don't export bpf_sk_fullsock helper (Martin Lau)
* size != sizeof(__u64) for uapi pointers (Martin Lau)
* offsetof instead of bpf_ctx_range when checking ctx access (Martin Lau)
v3:
* typos in BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY comments (Andrii Nakryiko)
* reverse christmas tree in BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY (Andrii Nakryiko)
* use __bpf_md_ptr instead of __u32 for optval{,_end} (Martin Lau)
* use BPF_FIELD_SIZEOF() for consistency (Martin Lau)
* new CG_SOCKOPT_ACCESS macro to wrap repeated parts
v2:
* moved bpf_sockopt_kern fields around to remove a hole (Martin Lau)
* aligned bpf_sockopt_kern->buf to 8 bytes (Martin Lau)
* bpf_prog_array_is_empty instead of bpf_prog_array_length (Martin Lau)
* added [0,2] return code check to verifier (Martin Lau)
* dropped unused buf[64] from the stack (Martin Lau)
* use PTR_TO_SOCKET for bpf_sockopt->sk (Martin Lau)
* dropped bpf_target_off from ctx rewrites (Martin Lau)
* use return code for kernel bypass (Martin Lau & Andrii Nakryiko)
--- include/linux/bpf-cgroup.h | 29 ++++ include/linux/bpf.h | 45 +++++++ include/linux/bpf_types.h | 1 + include/linux/filter.h | 13 ++ include/uapi/linux/bpf.h | 13 ++ kernel/bpf/cgroup.c | 262 +++++++++++++++++++++++++++++++++++++ kernel/bpf/core.c | 9 ++ kernel/bpf/syscall.c | 19 +++ kernel/bpf/verifier.c | 15 +++ net/core/filter.c | 2 +- net/socket.c | 18 +++ 11 files changed, 425 insertions(+), 1 deletion(-)
include/linux/bpf-cgroup.h +29 -0
@@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, loff_t *ppos, void **new_buf, enum bpf_attach_type type); +int __cgroup_bpf_run_filter_setsockopt (struct sock *sock,int level, +int optname,char __user *optval, +unsigned int optlen); +int __cgroup_bpf_run_filter_getsockopt (struct sock *sock,int level, +int optname,char __user *optval, +int __user *optlen); +static inline enum bpf_cgroup_storage_type cgroup_storage_type (struct bpf_map *map) {
@@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key, __ret; \ }) +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock, level, optname, optval, optlen) \ +({ \ + int __ret = 0; \ + if (cgroup_bpf_enabled) \ + __ret = __cgroup_bpf_run_filter_setsockopt(sock, level, \ + optname, optval, \ + optlen); \ + __ret; \ +}) + +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock, level, optname, optval, optlen) \ +({ \ + int __ret = 0; \ + if (cgroup_bpf_enabled) \ + __ret = __cgroup_bpf_run_filter_getsockopt(sock, level, \ + optname, optval, \ + optlen); \ + __ret; \ +}) +int cgroup_bpf_prog_attach (const union bpf_attr *attr,enum bpf_prog_type ptype,struct bpf_prog *prog); int cgroup_bpf_prog_detach(const union bpf_attr *attr,
@@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,#define BPF_CGROUP_RUN_PROG_SOCK_OPS (sock_ops) ({ 0; })#define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP (type,major,minor,access) ({ 0; })#define BPF_CGROUP_RUN_PROG_SYSCTL (head,table,write,buf,count,pos,nbuf) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock, level, optname, optval, optlen) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock, level, optname, optval, optlen) ({ 0; })#define for_each_cgroup_storage_type (stype) for (; false; )
include/linux/bpf.h +45 -0
63 lines not on this page · raw message
include/linux/bpf_types.h +1 -0
@@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)#ifdef CONFIG_CGROUP_BPF BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev) BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl) +BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)#endif #ifdef CONFIG_BPF_LIRC_MODE2 BPF_PROG_TYPE (BPF_PROG_TYPE_LIRC_MODE2 , lirc_mode2)
include/linux/filter.h +13 -0
@@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern { u64 tmp_reg; }; +struct bpf_sockopt_kern { +struct sock *sk; +u8 *optval; +u8 *optval_end; +s32 level; +s32 optname; +u32 optlen; + +/* Small on-stack optval buffer to avoid small allocations. +*/ +u8 buf[64 ]__aligned(8) ; +}; +#endif /* __LINUX_FILTER_H__ */
include/uapi/linux/bpf.h +13 -0
@@ -170,6 +170,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_FLOW_DISSECTOR ,BPF_PROG_TYPE_CGROUP_SYSCTL ,BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE , + BPF_PROG_TYPE_CGROUP_SOCKOPT, }; enum bpf_attach_type {
@@ -192,6 +193,8 @@ enum bpf_attach_type {BPF_LIRC_MODE2 ,BPF_FLOW_DISSECTOR ,BPF_CGROUP_SYSCTL , +BPF_CGROUP_GETSOCKOPT , +BPF_CGROUP_SETSOCKOPT , __MAX_BPF_ATTACH_TYPE };
@@ -3533,4 +3536,14 @@ struct bpf_sysctl {*/ }; +struct bpf_sockopt { + __bpf_md_ptr(struct bpf_sock *, sk); + __bpf_md_ptr(void *, optval); + __bpf_md_ptr(void *, optval_end); + +__s32 level; +__s32 optname; +__u32 optlen; +}; +#endif /* _UAPI__LINUX_BPF_H__ */
kernel/bpf/cgroup.c +262 -0
277 lines not on this page · raw message
kernel/bpf/core.c +9 -0
@@ -1813,6 +1813,15 @@ int bpf_prog_array_length(struct bpf_prog_array *array)return cnt; } +bool bpf_prog_array_is_empty (struct bpf_prog_array *array) +{ +struct bpf_prog_array_item *item; + +for (item = array->items; item->prog; item++) +if (item->prog != &dummy_bpf_prog.prog) +return false; +return true; +} static bool bpf_prog_array_copy_core(struct bpf_prog_array *array, u32 *prog_ids,
kernel/bpf/syscall.c +19 -0
@@ -1596,6 +1596,14 @@ bpf_prog_load_check_attach_type(enum bpf_prog_type prog_type,default :return -EINVAL ; } +case BPF_PROG_TYPE_CGROUP_SOCKOPT : +switch (expected_attach_type) { +case BPF_CGROUP_SETSOCKOPT : +case BPF_CGROUP_GETSOCKOPT : +return 0 ; +default : +return -EINVAL ; + }default :return 0 ; }
@@ -1846,6 +1854,7 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,switch (prog->type) {case BPF_PROG_TYPE_CGROUP_SOCK :case BPF_PROG_TYPE_CGROUP_SOCK_ADDR : +case BPF_PROG_TYPE_CGROUP_SOCKOPT :return attach_type == prog->expected_attach_type ?0 : -EINVAL ;case BPF_PROG_TYPE_CGROUP_SKB : return prog->enforce_expected_attach_type &&
@@ -1916,6 +1925,10 @@ static int bpf_prog_attach(const union bpf_attr *attr)case BPF_CGROUP_SYSCTL : ptype =BPF_PROG_TYPE_CGROUP_SYSCTL ;break ; +case BPF_CGROUP_GETSOCKOPT : +case BPF_CGROUP_SETSOCKOPT : + ptype =BPF_PROG_TYPE_CGROUP_SOCKOPT ; +break ;default :return -EINVAL ; }
@@ -1997,6 +2010,10 @@ static int bpf_prog_detach(const union bpf_attr *attr)case BPF_CGROUP_SYSCTL : ptype =BPF_PROG_TYPE_CGROUP_SYSCTL ;break ; +case BPF_CGROUP_GETSOCKOPT : +case BPF_CGROUP_SETSOCKOPT : + ptype =BPF_PROG_TYPE_CGROUP_SOCKOPT ; +break ;default :return -EINVAL ; }
@@ -2031,6 +2048,8 @@ static int bpf_prog_query(const union bpf_attr *attr,case BPF_CGROUP_SOCK_OPS :case BPF_CGROUP_DEVICE :case BPF_CGROUP_SYSCTL : +case BPF_CGROUP_GETSOCKOPT : +case BPF_CGROUP_SETSOCKOPT :break ;case BPF_LIRC_MODE2 :return lirc_prog_query (attr, uattr);
kernel/bpf/verifier.c +15 -0
@@ -1717,6 +1717,18 @@ static bool may_access_direct_pkt_data(struct bpf_verifier_env *env, env->seen_direct_write = true;return true; + +case BPF_PROG_TYPE_CGROUP_SOCKOPT : +if (t ==BPF_WRITE ) { +if (env->prog->expected_attach_type == +BPF_CGROUP_GETSOCKOPT ) { + env->seen_direct_write = true; +return true; + } +return false; + } +return true; +default :return false; }
@@ -5524,6 +5536,9 @@ static int check_return_code(struct bpf_verifier_env *env)case BPF_PROG_TYPE_CGROUP_DEVICE :case BPF_PROG_TYPE_CGROUP_SYSCTL :break ; +case BPF_PROG_TYPE_CGROUP_SOCKOPT : + range =tnum_range (0 ,2 ); +break ;default :return 0 ; }
net/core/filter.c +1 -1
@@ -5636,7 +5636,7 @@ BPF_CALL_1(bpf_tcp_sock, struct sock *, sk)return (unsigned long )NULL ; } -static const struct bpf_func_proto bpf_tcp_sock_proto = { +const struct bpf_func_proto bpf_tcp_sock_proto = { .func = bpf_tcp_sock, .gpl_only = false, .ret_type =RET_PTR_TO_TCP_SOCK_OR_NULL ,
net/socket.c +18 -0
@@ -2069,6 +2069,15 @@ static int __sys_setsockopt(int fd, int level, int optname,if (err) goto out_put; + err =BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock->sk, level, optname, + optval, optlen); +if (err <0 ) { + goto out_put; + }else if (err >0 ) { + err =0 ; + goto out_put; + } +if (level ==SOL_SOCKET ) err = sock_setsockopt(sock, level, optname, optval,
@@ -2106,6 +2115,15 @@ static int __sys_getsockopt(int fd, int level, int optname,if (err) goto out_put; + err =BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock->sk, level, optname, + optval, optlen); +if (err <0 ) { + goto out_put; + }else if (err >0 ) { + err =0 ; + goto out_put; + } +if (level ==SOL_SOCKET ) err = sock_getsockopt(sock, level, optname, optval,
Signature · 1 line
2.22.0.rc2.383.gf4fbbf30c2-goog
Re: [PATCH bpf-next v4 1/8] bpf: implement getsockopt and setsockopt hooks
Quoted 381 lines · Stanislav Fomichev
Implement new
BPF_PROG_TYPE_CGROUP_SOCKOPTprogram type and BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
BPF_CGROUP_SETSOCKOPTget a read-only view of the setsockopt arguments.BPF_CGROUP_GETSOCKOPTcan modify the supplied buffer.
Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.The buffer memory is pre-allocated (because I don't think there is a precedent for working with
__usermemory from bpf). This might be slow to do for each {s,g}etsockopt call, that's why I've added__cgroup_bpf_prog_array_is_emptythat exits early if there is nothing attached to a cgroup. Note, however, that there is a race between__cgroup_bpf_prog_array_is_emptyandBPF_PROG_RUN_ARRAYwhere cgroup program layout might have changed; this should not be a problem because in general there is a race between multiple calls to {s,g}etsocktop and user adding/removing bpf progs from a cgroup.The return code of the BPF program is handled as follows:
* 0: EPERM
* 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
* 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF prog exitsv4:
* don't exportbpf_sk_fullsockhelper (Martin Lau)
* size !=sizeof(__u64)for uapi pointers (Martin Lau)
* offsetof instead ofbpf_ctx_rangewhen checking ctx access (Martin Lau)v3:
* typos inBPF_PROG_CGROUP_SOCKOPT_RUN_ARRAYcomments (Andrii Nakryiko)
* reverse christmas tree inBPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(Andrii Nakryiko)
* use__bpf_md_ptrinstead of__u32for optval{,_end} (Martin Lau)
* useBPF_FIELD_SIZEOF()for consistency (Martin Lau)
* newCG_SOCKOPT_ACCESSmacro to wrap repeated partsv2:
* movedbpf_sockopt_kernfields around to remove a hole (Martin Lau)
* alignedbpf_sockopt_kern->bufto 8 bytes (Martin Lau)
*bpf_prog_array_is_emptyinstead ofbpf_prog_array_length(Martin Lau)
* added [0,2] return code check to verifier (Martin Lau)
* dropped unused buf[64] from the stack (Martin Lau)
* usePTR_TO_SOCKETforbpf_sockopt->sk(Martin Lau)
* droppedbpf_target_offfrom ctx rewrites (Martin Lau)
* use return code for kernel bypass (Martin Lau & Andrii Nakryiko)Cc Martin Lau <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Stanislav Fomichev <[email protected]>--- include/linux/bpf-cgroup.h | 29 ++++ include/linux/bpf.h | 45 +++++++ include/linux/bpf_types.h | 1 + include/linux/filter.h | 13 ++ include/uapi/linux/bpf.h | 13 ++ kernel/bpf/cgroup.c | 262 +++++++++++++++++++++++++++++++++++++ kernel/bpf/core.c | 9 ++ kernel/bpf/syscall.c | 19 +++ kernel/bpf/verifier.c | 15 +++ net/core/filter.c | 2 +- net/socket.c | 18 +++ 11 files changed, 425 insertions(+), 1 deletion(-)include/linux/bpf-cgroup.h +29
-0@@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, loff_t *ppos, void **new_buf, enum bpf_attach_type type); +int __cgroup_bpf_run_filter_setsockopt (struct sock *sock,int level, +int optname,char __user *optval, +unsigned int optlen); +int __cgroup_bpf_run_filter_getsockopt (struct sock *sock,int level, +int optname,char __user *optval, +int __user *optlen); +static inline enum bpf_cgroup_storage_type cgroup_storage_type (struct bpf_map *map) {@@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key, __ret; \ }) +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock, level, optname, optval, optlen) \ +({ \ + int __ret = 0; \ + if (cgroup_bpf_enabled) \ + __ret = __cgroup_bpf_run_filter_setsockopt(sock, level, \ + optname, optval, \ + optlen); \ + __ret; \ +}) + +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock, level, optname, optval, optlen) \ +({ \ + int __ret = 0; \ + if (cgroup_bpf_enabled) \ + __ret = __cgroup_bpf_run_filter_getsockopt(sock, level, \ + optname, optval, \ + optlen); \ + __ret; \ +}) +int cgroup_bpf_prog_attach (const union bpf_attr *attr,enum bpf_prog_type ptype,struct bpf_prog *prog); int cgroup_bpf_prog_detach(const union bpf_attr *attr,@@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,#define BPF_CGROUP_RUN_PROG_SOCK_OPS (sock_ops) ({ 0; })#define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP (type,major,minor,access) ({ 0; })#define BPF_CGROUP_RUN_PROG_SYSCTL (head,table,write,buf,count,pos,nbuf) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock, level, optname, optval, optlen) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock, level, optname, optval, optlen) ({ 0; })#define for_each_cgroup_storage_type (stype) for (; false; )include/linux/bpf.h +45
-0@@ -520,6 +520,7 @@ struct bpf_prog_array {struct bpf_prog_array *bpf_prog_array_alloc (u32 prog_cnt,gfp_t flags);void bpf_prog_array_free (struct bpf_prog_array *progs);int bpf_prog_array_length (struct bpf_prog_array *progs); +bool bpf_prog_array_is_empty (struct bpf_prog_array *array);int bpf_prog_array_copy_to_user (struct bpf_prog_array *progs,__u32 __user *prog_ids,u32 cnt);@@ -606,6 +607,49 @@ _out: \ _ret; \ }) +/* To be used by BPF_PROG_TYPE_CGROUP_SOCKOPT program type. +* +* Expected BPF program return values are: +* 0: return -EPERM to the userspace +* 1: sockopt was not handled by BPF, kernel should do it +* 2: sockopt was handled by BPF, kernel should _not_ do it and return +* to the userspace instead +* +* Note, that return '0' takes precedence over everything else. In other +* words, if any single program in the prog array has returned 0, +* the userspace will get -EPERM (regardless of what other programs +* return). +* +* The macro itself returns: +* 0: sockopt was not handled by BPF, kernel should do it +* 1: sockopt was handled by BPF, kernel should _not_ do it +* -EPERM: return error back to userspace +*/ +#define BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY (array, ctx, func) \ + ({ \ + struct bpf_prog_array_item *_item; \ + struct bpf_prog_array *_array; \ + struct bpf_prog *_prog; \ + u32 _success = 1; \ + u32 _bypass = 0; \ + u32 ret; \ + preempt_disable(); \ + rcu_read_lock(); \ + _array = rcu_dereference(array); \ + _item = &_array->items[0]; \ + while ((_prog = READ_ONCE(_item->prog))) { \ + bpf_cgroup_storage_set(_item->cgroup_storage); \ + ret = func(_prog, ctx); \ + _success &= (ret > 0); \ + _bypass |= (ret == 2); \ + _item++; \ + } \ + rcu_read_unlock(); \ + preempt_enable(); \ + ret = _success ? _bypass : -EPERM; \ + ret; \ + }) +#define BPF_PROG_RUN_ARRAY (array, ctx, func) \ __BPF_PROG_RUN_ARRAY(array, ctx, func, false)@@ -1054,6 +1098,7 @@ extern const struct bpf_func_proto bpf_spin_unlock_proto;extern const struct bpf_func_proto bpf_get_local_storage_proto;extern const struct bpf_func_proto bpf_strtol_proto;extern const struct bpf_func_proto bpf_strtoul_proto; +extern const struct bpf_func_proto bpf_tcp_sock_proto;/* Shared helpers among cBPF and eBPF. */ void bpf_user_rnd_init_once (void );include/linux/bpf_types.h +1
-0@@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)#ifdef CONFIG_CGROUP_BPF BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev) BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl) +BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)#endif #ifdef CONFIG_BPF_LIRC_MODE2 BPF_PROG_TYPE (BPF_PROG_TYPE_LIRC_MODE2 , lirc_mode2)include/linux/filter.h +13
-0@@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern { u64 tmp_reg; }; +struct bpf_sockopt_kern { +struct sock *sk; +u8 *optval; +u8 *optval_end; +s32 level; +s32 optname; +u32 optlen; + +/* Small on-stack optval buffer to avoid small allocations. +*/ +u8 buf[64 ]__aligned(8) ; +}; +#endif /* __LINUX_FILTER_H__ */ include/uapi/linux/bpf.h +13
-0@@ -170,6 +170,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_FLOW_DISSECTOR ,BPF_PROG_TYPE_CGROUP_SYSCTL ,BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE , + BPF_PROG_TYPE_CGROUP_SOCKOPT, }; enum bpf_attach_type {@@ -192,6 +193,8 @@ enum bpf_attach_type {BPF_LIRC_MODE2 ,BPF_FLOW_DISSECTOR ,BPF_CGROUP_SYSCTL , +BPF_CGROUP_GETSOCKOPT , +BPF_CGROUP_SETSOCKOPT , __MAX_BPF_ATTACH_TYPE };@@ -3533,4 +3536,14 @@ struct bpf_sysctl {*/ }; +struct bpf_sockopt { + __bpf_md_ptr(struct bpf_sock *, sk); + __bpf_md_ptr(void *, optval); + __bpf_md_ptr(void *, optval_end); + +__s32 level; +__s32 optname; +__u32 optlen; +}; +#endif /* _UAPI__LINUX_BPF_H__ */ kernel/bpf/cgroup.c +109
-0@@ -18,6 +18,7 @@#include <linux/bpf.h> #include <linux/bpf-cgroup.h> #include <net/sock.h> +#include <net/bpf_sk_storage.h> DEFINE_STATIC_KEY_FALSE (cgroup_bpf_enabled_key);EXPORT_SYMBOL (cgroup_bpf_enabled_key);@@ -924,6 +925,142 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, }EXPORT_SYMBOL (__cgroup_bpf_run_filter_sysctl); +static bool __cgroup_bpf_prog_array_is_empty (struct cgroup *cgrp, +enum bpf_attach_type attach_type) +{ +struct bpf_prog_array *prog_array; +bool empty; + +rcu_read_lock (); + prog_array =rcu_dereference (cgrp->bpf.effective[attach_type]); + empty =bpf_prog_array_is_empty (prog_array); +rcu_read_unlock (); + +return empty; +} + +static int sockopt_alloc_buf (struct bpf_sockopt_kern *ctx,int max_optlen) +{ +if (unlikely (max_optlen >PAGE_SIZE )) +return -EINVAL ; + +if (likely (max_optlen <=sizeof (ctx->buf))) { + ctx->optval = ctx->buf; + }else { + ctx->optval =kzalloc (max_optlen,GFP_USER ); +if (!ctx->optval) +return -ENOMEM ; + } + + ctx->optval_end = ctx->optval + max_optlen; + ctx->optlen = max_optlen; + +return 0 ; +} + +static void sockopt_free_buf (struct bpf_sockopt_kern *ctx) +{ +if (unlikely (ctx->optval != ctx->buf)) +kfree (ctx->optval); +} + +int __cgroup_bpf_run_filter_setsockopt (struct sock *sk,int level, +int optname,char __user *optval, +unsigned int optlen) +{ +struct cgroup *cgrp =sock_cgroup_ptr (&sk->sk_cgrp_data); +struct bpf_sockopt_kern ctx = { + .sk = sk, + .level = level, + .optname = optname, + }; +int ret; + +/* Opportunistic check to see whether we have any BPF program +* attached to the hook so we don't waste time allocating +* memory and locking the socket. +*/ +if (__cgroup_bpf_prog_array_is_empty (cgrp,BPF_CGROUP_SETSOCKOPT )) +return 0 ; + + ret =sockopt_alloc_buf (&ctx, optlen); +if (ret) +return ret; + +if (copy_from_user (ctx.optval, optval, optlen) !=0 ) { +sockopt_free_buf (&ctx); +return -EFAULT ; + } + +lock_sock (sk); + ret =BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY ( + cgrp->bpf.effective[BPF_CGROUP_SETSOCKOPT ], + &ctx,BPF_PROG_RUN ); +release_sock (sk); + +sockopt_free_buf (&ctx); + +return ret; +} +EXPORT_SYMBOL (__cgroup_bpf_run_filter_setsockopt); + +int __cgroup_bpf_run_filter_getsockopt (struct sock *sk,int level, +int optname,char __user *optval, +int __user *optlen) +{ +struct cgroup *cgrp =sock_cgroup_ptr (&sk->sk_cgrp_data); +struct bpf_sockopt_kern ctx = { + .sk = sk, + .level = level, + .optname = optname, + }; +int max_optlen; +int ret; + +/* Opportunistic check to see whether we have any BPF program +* attached to the hook so we don't waste time allocating +* memory and locking the socket. +*/ +if (__cgroup_bpf_prog_array_is_empty (cgrp,BPF_CGROUP_GETSOCKOPT )) +return 0 ; + +if (get_user (max_optlen, optlen)) +return -EFAULT ; + + ret =sockopt_alloc_buf (&ctx, max_optlen); +if (ret) +return ret; + +lock_sock (sk); + ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(
kernel/bpf/cgroup.c int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, · line 1045
Quoted 10 lines
kernel/bpf/cgroup.c +10
-0+ cgrp->bpf.effective[BPF_CGROUP_GETSOCKOPT ], + &ctx, BPF_PROG_RUN); +release_sock (sk); + +if (ret <0 ) { +sockopt_free_buf (&ctx); +return ret; + } + +if (ctx.optlen > max_optlen) {
v4 looks very good.
One minor question here, what may be the use case if the bpf_prog makes ctx.optlen < max_optlen(==user's input) but sets ret == 0 (i.e. kernel should continue to handle it)?
Quoted 18 lines
kernel/bpf/cgroup.c +18
-0+sockopt_free_buf (&ctx); +return -EFAULT ; + } + +if (copy_to_user (optval, ctx.optval, ctx.optlen) !=0 ) { +sockopt_free_buf (&ctx); +return -EFAULT ; + } + +sockopt_free_buf (&ctx); + +if (put_user (ctx.optlen, optlen)) +return -EFAULT ; + +return ret; +} +EXPORT_SYMBOL (__cgroup_bpf_run_filter_getsockopt); +
Stanislav Fomichev
reviewerRe: [PATCH bpf-next v4 1/8] bpf: implement getsockopt and setsockopt hooks
Quoted 394 lines · Martin Lau
Quoted 381 lines · Stanislav Fomichev
Implement new
BPF_PROG_TYPE_CGROUP_SOCKOPTprogram type and BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
BPF_CGROUP_SETSOCKOPTget a read-only view of the setsockopt arguments.BPF_CGROUP_GETSOCKOPTcan modify the supplied buffer.
Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.The buffer memory is pre-allocated (because I don't think there is a precedent for working with
__usermemory from bpf). This might be slow to do for each {s,g}etsockopt call, that's why I've added__cgroup_bpf_prog_array_is_emptythat exits early if there is nothing attached to a cgroup. Note, however, that there is a race between__cgroup_bpf_prog_array_is_emptyandBPF_PROG_RUN_ARRAYwhere cgroup program layout might have changed; this should not be a problem because in general there is a race between multiple calls to {s,g}etsocktop and user adding/removing bpf progs from a cgroup.The return code of the BPF program is handled as follows:
* 0: EPERM
* 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
* 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF prog exitsv4:
* don't exportbpf_sk_fullsockhelper (Martin Lau)
* size !=sizeof(__u64)for uapi pointers (Martin Lau)
* offsetof instead ofbpf_ctx_rangewhen checking ctx access (Martin Lau)v3:
* typos inBPF_PROG_CGROUP_SOCKOPT_RUN_ARRAYcomments (Andrii Nakryiko)
* reverse christmas tree inBPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(Andrii Nakryiko)
* use__bpf_md_ptrinstead of__u32for optval{,_end} (Martin Lau)
* useBPF_FIELD_SIZEOF()for consistency (Martin Lau)
* newCG_SOCKOPT_ACCESSmacro to wrap repeated partsv2:
* movedbpf_sockopt_kernfields around to remove a hole (Martin Lau)
* alignedbpf_sockopt_kern->bufto 8 bytes (Martin Lau)
*bpf_prog_array_is_emptyinstead ofbpf_prog_array_length(Martin Lau)
* added [0,2] return code check to verifier (Martin Lau)
* dropped unused buf[64] from the stack (Martin Lau)
* usePTR_TO_SOCKETforbpf_sockopt->sk(Martin Lau)
* droppedbpf_target_offfrom ctx rewrites (Martin Lau)
* use return code for kernel bypass (Martin Lau & Andrii Nakryiko)Cc Martin Lau <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Stanislav Fomichev <[email protected]>--- include/linux/bpf-cgroup.h | 29 ++++ include/linux/bpf.h | 45 +++++++ include/linux/bpf_types.h | 1 + include/linux/filter.h | 13 ++ include/uapi/linux/bpf.h | 13 ++ kernel/bpf/cgroup.c | 262 +++++++++++++++++++++++++++++++++++++ kernel/bpf/core.c | 9 ++ kernel/bpf/syscall.c | 19 +++ kernel/bpf/verifier.c | 15 +++ net/core/filter.c | 2 +- net/socket.c | 18 +++ 11 files changed, 425 insertions(+), 1 deletion(-)include/linux/bpf-cgroup.h +29
-0@@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, loff_t *ppos, void **new_buf, enum bpf_attach_type type); +int __cgroup_bpf_run_filter_setsockopt (struct sock *sock,int level, +int optname,char __user *optval, +unsigned int optlen); +int __cgroup_bpf_run_filter_getsockopt (struct sock *sock,int level, +int optname,char __user *optval, +int __user *optlen); +static inline enum bpf_cgroup_storage_type cgroup_storage_type (struct bpf_map *map) {@@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key, __ret; \ }) +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock, level, optname, optval, optlen) \ +({ \ + int __ret = 0; \ + if (cgroup_bpf_enabled) \ + __ret = __cgroup_bpf_run_filter_setsockopt(sock, level, \ + optname, optval, \ + optlen); \ + __ret; \ +}) + +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock, level, optname, optval, optlen) \ +({ \ + int __ret = 0; \ + if (cgroup_bpf_enabled) \ + __ret = __cgroup_bpf_run_filter_getsockopt(sock, level, \ + optname, optval, \ + optlen); \ + __ret; \ +}) +int cgroup_bpf_prog_attach (const union bpf_attr *attr,enum bpf_prog_type ptype,struct bpf_prog *prog); int cgroup_bpf_prog_detach(const union bpf_attr *attr,@@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,#define BPF_CGROUP_RUN_PROG_SOCK_OPS (sock_ops) ({ 0; })#define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP (type,major,minor,access) ({ 0; })#define BPF_CGROUP_RUN_PROG_SYSCTL (head,table,write,buf,count,pos,nbuf) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock, level, optname, optval, optlen) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock, level, optname, optval, optlen) ({ 0; })#define for_each_cgroup_storage_type (stype) for (; false; )include/linux/bpf.h +45
-0@@ -520,6 +520,7 @@ struct bpf_prog_array {struct bpf_prog_array *bpf_prog_array_alloc (u32 prog_cnt,gfp_t flags);void bpf_prog_array_free (struct bpf_prog_array *progs);int bpf_prog_array_length (struct bpf_prog_array *progs); +bool bpf_prog_array_is_empty (struct bpf_prog_array *array);int bpf_prog_array_copy_to_user (struct bpf_prog_array *progs,__u32 __user *prog_ids,u32 cnt);@@ -606,6 +607,49 @@ _out: \ _ret; \ }) +/* To be used by BPF_PROG_TYPE_CGROUP_SOCKOPT program type. +* +* Expected BPF program return values are: +* 0: return -EPERM to the userspace +* 1: sockopt was not handled by BPF, kernel should do it +* 2: sockopt was handled by BPF, kernel should _not_ do it and return +* to the userspace instead +* +* Note, that return '0' takes precedence over everything else. In other +* words, if any single program in the prog array has returned 0, +* the userspace will get -EPERM (regardless of what other programs +* return). +* +* The macro itself returns: +* 0: sockopt was not handled by BPF, kernel should do it +* 1: sockopt was handled by BPF, kernel should _not_ do it +* -EPERM: return error back to userspace +*/ +#define BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY (array, ctx, func) \ + ({ \ + struct bpf_prog_array_item *_item; \ + struct bpf_prog_array *_array; \ + struct bpf_prog *_prog; \ + u32 _success = 1; \ + u32 _bypass = 0; \ + u32 ret; \ + preempt_disable(); \ + rcu_read_lock(); \ + _array = rcu_dereference(array); \ + _item = &_array->items[0]; \ + while ((_prog = READ_ONCE(_item->prog))) { \ + bpf_cgroup_storage_set(_item->cgroup_storage); \ + ret = func(_prog, ctx); \ + _success &= (ret > 0); \ + _bypass |= (ret == 2); \ + _item++; \ + } \ + rcu_read_unlock(); \ + preempt_enable(); \ + ret = _success ? _bypass : -EPERM; \ + ret; \ + }) +#define BPF_PROG_RUN_ARRAY (array, ctx, func) \ __BPF_PROG_RUN_ARRAY(array, ctx, func, false)@@ -1054,6 +1098,7 @@ extern const struct bpf_func_proto bpf_spin_unlock_proto;extern const struct bpf_func_proto bpf_get_local_storage_proto;extern const struct bpf_func_proto bpf_strtol_proto;extern const struct bpf_func_proto bpf_strtoul_proto; +extern const struct bpf_func_proto bpf_tcp_sock_proto;/* Shared helpers among cBPF and eBPF. */ void bpf_user_rnd_init_once (void );include/linux/bpf_types.h +1
-0@@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)#ifdef CONFIG_CGROUP_BPF BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev) BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl) +BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)#endif #ifdef CONFIG_BPF_LIRC_MODE2 BPF_PROG_TYPE (BPF_PROG_TYPE_LIRC_MODE2 , lirc_mode2)include/linux/filter.h +13
-0@@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern { u64 tmp_reg; }; +struct bpf_sockopt_kern { +struct sock *sk; +u8 *optval; +u8 *optval_end; +s32 level; +s32 optname; +u32 optlen; + +/* Small on-stack optval buffer to avoid small allocations. +*/ +u8 buf[64 ]__aligned(8) ; +}; +#endif /* __LINUX_FILTER_H__ */ include/uapi/linux/bpf.h +13
-0@@ -170,6 +170,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_FLOW_DISSECTOR ,BPF_PROG_TYPE_CGROUP_SYSCTL ,BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE , + BPF_PROG_TYPE_CGROUP_SOCKOPT, }; enum bpf_attach_type {@@ -192,6 +193,8 @@ enum bpf_attach_type {BPF_LIRC_MODE2 ,BPF_FLOW_DISSECTOR ,BPF_CGROUP_SYSCTL , +BPF_CGROUP_GETSOCKOPT , +BPF_CGROUP_SETSOCKOPT , __MAX_BPF_ATTACH_TYPE };@@ -3533,4 +3536,14 @@ struct bpf_sysctl {*/ }; +struct bpf_sockopt { + __bpf_md_ptr(struct bpf_sock *, sk); + __bpf_md_ptr(void *, optval); + __bpf_md_ptr(void *, optval_end); + +__s32 level; +__s32 optname; +__u32 optlen; +}; +#endif /* _UAPI__LINUX_BPF_H__ */ kernel/bpf/cgroup.c +109
-0@@ -18,6 +18,7 @@#include <linux/bpf.h> #include <linux/bpf-cgroup.h> #include <net/sock.h> +#include <net/bpf_sk_storage.h> DEFINE_STATIC_KEY_FALSE (cgroup_bpf_enabled_key);EXPORT_SYMBOL (cgroup_bpf_enabled_key);@@ -924,6 +925,142 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, }EXPORT_SYMBOL (__cgroup_bpf_run_filter_sysctl); +static bool __cgroup_bpf_prog_array_is_empty (struct cgroup *cgrp, +enum bpf_attach_type attach_type) +{ +struct bpf_prog_array *prog_array; +bool empty; + +rcu_read_lock (); + prog_array =rcu_dereference (cgrp->bpf.effective[attach_type]); + empty =bpf_prog_array_is_empty (prog_array); +rcu_read_unlock (); + +return empty; +} + +static int sockopt_alloc_buf (struct bpf_sockopt_kern *ctx,int max_optlen) +{ +if (unlikely (max_optlen >PAGE_SIZE )) +return -EINVAL ; + +if (likely (max_optlen <=sizeof (ctx->buf))) { + ctx->optval = ctx->buf; + }else { + ctx->optval =kzalloc (max_optlen,GFP_USER ); +if (!ctx->optval) +return -ENOMEM ; + } + + ctx->optval_end = ctx->optval + max_optlen; + ctx->optlen = max_optlen; + +return 0 ; +} + +static void sockopt_free_buf (struct bpf_sockopt_kern *ctx) +{ +if (unlikely (ctx->optval != ctx->buf)) +kfree (ctx->optval); +} + +int __cgroup_bpf_run_filter_setsockopt (struct sock *sk,int level, +int optname,char __user *optval, +unsigned int optlen) +{ +struct cgroup *cgrp =sock_cgroup_ptr (&sk->sk_cgrp_data); +struct bpf_sockopt_kern ctx = { + .sk = sk, + .level = level, + .optname = optname, + }; +int ret; + +/* Opportunistic check to see whether we have any BPF program +* attached to the hook so we don't waste time allocating +* memory and locking the socket. +*/ +if (__cgroup_bpf_prog_array_is_empty (cgrp,BPF_CGROUP_SETSOCKOPT )) +return 0 ; + + ret =sockopt_alloc_buf (&ctx, optlen); +if (ret) +return ret; + +if (copy_from_user (ctx.optval, optval, optlen) !=0 ) { +sockopt_free_buf (&ctx); +return -EFAULT ; + } + +lock_sock (sk); + ret =BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY ( + cgrp->bpf.effective[BPF_CGROUP_SETSOCKOPT ], + &ctx,BPF_PROG_RUN ); +release_sock (sk); + +sockopt_free_buf (&ctx); + +return ret; +} +EXPORT_SYMBOL (__cgroup_bpf_run_filter_setsockopt); + +int __cgroup_bpf_run_filter_getsockopt (struct sock *sk,int level, +int optname,char __user *optval, +int __user *optlen) +{ +struct cgroup *cgrp =sock_cgroup_ptr (&sk->sk_cgrp_data); +struct bpf_sockopt_kern ctx = { + .sk = sk, + .level = level, + .optname = optname, + }; +int max_optlen; +int ret; + +/* Opportunistic check to see whether we have any BPF program +* attached to the hook so we don't waste time allocating +* memory and locking the socket. +*/ +if (__cgroup_bpf_prog_array_is_empty (cgrp,BPF_CGROUP_GETSOCKOPT )) +return 0 ; + +if (get_user (max_optlen, optlen)) +return -EFAULT ; + + ret =sockopt_alloc_buf (&ctx, max_optlen); +if (ret) +return ret; + +lock_sock (sk); + ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(kernel/bpf/cgroup.c int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, · line 1045
Quoted 10 lines
kernel/bpf/cgroup.c +10
-0+ cgrp->bpf.effective[BPF_CGROUP_GETSOCKOPT ], + &ctx, BPF_PROG_RUN); +release_sock (sk); + +if (ret <0 ) { +sockopt_free_buf (&ctx); +return ret; + } + +if (ctx.optlen > max_optlen) {v4 looks very good.
Quoted 3 lines
One minor question here, what may be the use case if the
bpf_progmakes ctx.optlen <max_optlen(==user's input)but sets ret == 0 (i.e. kernel should continue to handle it)?
Do you mean bpf fills in optval + adjusts optlen, but returns 0? I don't see any usecase for that, bpf's output would be ignored in that case.
Do you think we should have an additional check to make sure ctx.optval data is zero and ctx.optlen is not changed in that case?
Quoted 18 lines
Quoted 18 lines
kernel/bpf/cgroup.c +18
-0+sockopt_free_buf (&ctx); +return -EFAULT ; + } + +if (copy_to_user (optval, ctx.optval, ctx.optlen) !=0 ) { +sockopt_free_buf (&ctx); +return -EFAULT ; + } + +sockopt_free_buf (&ctx); + +if (put_user (ctx.optlen, optlen)) +return -EFAULT ; + +return ret; +} +EXPORT_SYMBOL (__cgroup_bpf_run_filter_getsockopt); +
Re: [PATCH bpf-next v4 1/8] bpf: implement getsockopt and setsockopt hooks
Quoted 395 lines · Stanislav Fomichev
Quoted 394 lines · Martin Lau
Quoted 381 lines · Stanislav Fomichev
Implement new
BPF_PROG_TYPE_CGROUP_SOCKOPTprogram type and BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
BPF_CGROUP_SETSOCKOPTget a read-only view of the setsockopt arguments.BPF_CGROUP_GETSOCKOPTcan modify the supplied buffer.
Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.The buffer memory is pre-allocated (because I don't think there is a precedent for working with
__usermemory from bpf). This might be slow to do for each {s,g}etsockopt call, that's why I've added__cgroup_bpf_prog_array_is_emptythat exits early if there is nothing attached to a cgroup. Note, however, that there is a race between__cgroup_bpf_prog_array_is_emptyandBPF_PROG_RUN_ARRAYwhere cgroup program layout might have changed; this should not be a problem because in general there is a race between multiple calls to {s,g}etsocktop and user adding/removing bpf progs from a cgroup.The return code of the BPF program is handled as follows:
* 0: EPERM
* 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
* 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF prog exitsv4:
* don't exportbpf_sk_fullsockhelper (Martin Lau)
* size !=sizeof(__u64)for uapi pointers (Martin Lau)
* offsetof instead ofbpf_ctx_rangewhen checking ctx access (Martin Lau)v3:
* typos inBPF_PROG_CGROUP_SOCKOPT_RUN_ARRAYcomments (Andrii Nakryiko)
* reverse christmas tree inBPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(Andrii Nakryiko)
* use__bpf_md_ptrinstead of__u32for optval{,_end} (Martin Lau)
* useBPF_FIELD_SIZEOF()for consistency (Martin Lau)
* newCG_SOCKOPT_ACCESSmacro to wrap repeated partsv2:
* movedbpf_sockopt_kernfields around to remove a hole (Martin Lau)
* alignedbpf_sockopt_kern->bufto 8 bytes (Martin Lau)
*bpf_prog_array_is_emptyinstead ofbpf_prog_array_length(Martin Lau)
* added [0,2] return code check to verifier (Martin Lau)
* dropped unused buf[64] from the stack (Martin Lau)
* usePTR_TO_SOCKETforbpf_sockopt->sk(Martin Lau)
* droppedbpf_target_offfrom ctx rewrites (Martin Lau)
* use return code for kernel bypass (Martin Lau & Andrii Nakryiko)Cc Martin Lau <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Stanislav Fomichev <[email protected]>--- include/linux/bpf-cgroup.h | 29 ++++ include/linux/bpf.h | 45 +++++++ include/linux/bpf_types.h | 1 + include/linux/filter.h | 13 ++ include/uapi/linux/bpf.h | 13 ++ kernel/bpf/cgroup.c | 262 +++++++++++++++++++++++++++++++++++++ kernel/bpf/core.c | 9 ++ kernel/bpf/syscall.c | 19 +++ kernel/bpf/verifier.c | 15 +++ net/core/filter.c | 2 +- net/socket.c | 18 +++ 11 files changed, 425 insertions(+), 1 deletion(-)include/linux/bpf-cgroup.h +29
-0@@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, loff_t *ppos, void **new_buf, enum bpf_attach_type type); +int __cgroup_bpf_run_filter_setsockopt (struct sock *sock,int level, +int optname,char __user *optval, +unsigned int optlen); +int __cgroup_bpf_run_filter_getsockopt (struct sock *sock,int level, +int optname,char __user *optval, +int __user *optlen); +static inline enum bpf_cgroup_storage_type cgroup_storage_type (struct bpf_map *map) {@@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key, __ret; \ }) +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock, level, optname, optval, optlen) \ +({ \ + int __ret = 0; \ + if (cgroup_bpf_enabled) \ + __ret = __cgroup_bpf_run_filter_setsockopt(sock, level, \ + optname, optval, \ + optlen); \ + __ret; \ +}) + +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock, level, optname, optval, optlen) \ +({ \ + int __ret = 0; \ + if (cgroup_bpf_enabled) \ + __ret = __cgroup_bpf_run_filter_getsockopt(sock, level, \ + optname, optval, \ + optlen); \ + __ret; \ +}) +int cgroup_bpf_prog_attach (const union bpf_attr *attr,enum bpf_prog_type ptype,struct bpf_prog *prog); int cgroup_bpf_prog_detach(const union bpf_attr *attr,@@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,#define BPF_CGROUP_RUN_PROG_SOCK_OPS (sock_ops) ({ 0; })#define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP (type,major,minor,access) ({ 0; })#define BPF_CGROUP_RUN_PROG_SYSCTL (head,table,write,buf,count,pos,nbuf) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock, level, optname, optval, optlen) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock, level, optname, optval, optlen) ({ 0; })#define for_each_cgroup_storage_type (stype) for (; false; )include/linux/bpf.h +45
-0@@ -520,6 +520,7 @@ struct bpf_prog_array {struct bpf_prog_array *bpf_prog_array_alloc (u32 prog_cnt,gfp_t flags);void bpf_prog_array_free (struct bpf_prog_array *progs);int bpf_prog_array_length (struct bpf_prog_array *progs); +bool bpf_prog_array_is_empty (struct bpf_prog_array *array);int bpf_prog_array_copy_to_user (struct bpf_prog_array *progs,__u32 __user *prog_ids,u32 cnt);@@ -606,6 +607,49 @@ _out: \ _ret; \ }) +/* To be used by BPF_PROG_TYPE_CGROUP_SOCKOPT program type. +* +* Expected BPF program return values are: +* 0: return -EPERM to the userspace +* 1: sockopt was not handled by BPF, kernel should do it +* 2: sockopt was handled by BPF, kernel should _not_ do it and return +* to the userspace instead +* +* Note, that return '0' takes precedence over everything else. In other +* words, if any single program in the prog array has returned 0, +* the userspace will get -EPERM (regardless of what other programs +* return). +* +* The macro itself returns: +* 0: sockopt was not handled by BPF, kernel should do it +* 1: sockopt was handled by BPF, kernel should _not_ do it +* -EPERM: return error back to userspace +*/ +#define BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY (array, ctx, func) \ + ({ \ + struct bpf_prog_array_item *_item; \ + struct bpf_prog_array *_array; \ + struct bpf_prog *_prog; \ + u32 _success = 1; \ + u32 _bypass = 0; \ + u32 ret; \ + preempt_disable(); \ + rcu_read_lock(); \ + _array = rcu_dereference(array); \ + _item = &_array->items[0]; \ + while ((_prog = READ_ONCE(_item->prog))) { \ + bpf_cgroup_storage_set(_item->cgroup_storage); \ + ret = func(_prog, ctx); \ + _success &= (ret > 0); \ + _bypass |= (ret == 2); \ + _item++; \ + } \ + rcu_read_unlock(); \ + preempt_enable(); \ + ret = _success ? _bypass : -EPERM; \ + ret; \ + }) +#define BPF_PROG_RUN_ARRAY (array, ctx, func) \ __BPF_PROG_RUN_ARRAY(array, ctx, func, false)@@ -1054,6 +1098,7 @@ extern const struct bpf_func_proto bpf_spin_unlock_proto;extern const struct bpf_func_proto bpf_get_local_storage_proto;extern const struct bpf_func_proto bpf_strtol_proto;extern const struct bpf_func_proto bpf_strtoul_proto; +extern const struct bpf_func_proto bpf_tcp_sock_proto;/* Shared helpers among cBPF and eBPF. */ void bpf_user_rnd_init_once (void );include/linux/bpf_types.h +1
-0@@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)#ifdef CONFIG_CGROUP_BPF BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev) BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl) +BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)#endif #ifdef CONFIG_BPF_LIRC_MODE2 BPF_PROG_TYPE (BPF_PROG_TYPE_LIRC_MODE2 , lirc_mode2)include/linux/filter.h +13
-0@@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern { u64 tmp_reg; }; +struct bpf_sockopt_kern { +struct sock *sk; +u8 *optval; +u8 *optval_end; +s32 level; +s32 optname; +u32 optlen; + +/* Small on-stack optval buffer to avoid small allocations. +*/ +u8 buf[64 ]__aligned(8) ; +}; +#endif /* __LINUX_FILTER_H__ */ include/uapi/linux/bpf.h +13
-0@@ -170,6 +170,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_FLOW_DISSECTOR ,BPF_PROG_TYPE_CGROUP_SYSCTL ,BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE , + BPF_PROG_TYPE_CGROUP_SOCKOPT, }; enum bpf_attach_type {@@ -192,6 +193,8 @@ enum bpf_attach_type {BPF_LIRC_MODE2 ,BPF_FLOW_DISSECTOR ,BPF_CGROUP_SYSCTL , +BPF_CGROUP_GETSOCKOPT , +BPF_CGROUP_SETSOCKOPT , __MAX_BPF_ATTACH_TYPE };@@ -3533,4 +3536,14 @@ struct bpf_sysctl {*/ }; +struct bpf_sockopt { + __bpf_md_ptr(struct bpf_sock *, sk); + __bpf_md_ptr(void *, optval); + __bpf_md_ptr(void *, optval_end); + +__s32 level; +__s32 optname; +__u32 optlen; +}; +#endif /* _UAPI__LINUX_BPF_H__ */ kernel/bpf/cgroup.c +109
-0@@ -18,6 +18,7 @@#include <linux/bpf.h> #include <linux/bpf-cgroup.h> #include <net/sock.h> +#include <net/bpf_sk_storage.h> DEFINE_STATIC_KEY_FALSE (cgroup_bpf_enabled_key);EXPORT_SYMBOL (cgroup_bpf_enabled_key);@@ -924,6 +925,142 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, }EXPORT_SYMBOL (__cgroup_bpf_run_filter_sysctl); +static bool __cgroup_bpf_prog_array_is_empty (struct cgroup *cgrp, +enum bpf_attach_type attach_type) +{ +struct bpf_prog_array *prog_array; +bool empty; + +rcu_read_lock (); + prog_array =rcu_dereference (cgrp->bpf.effective[attach_type]); + empty =bpf_prog_array_is_empty (prog_array); +rcu_read_unlock (); + +return empty; +} + +static int sockopt_alloc_buf (struct bpf_sockopt_kern *ctx,int max_optlen) +{ +if (unlikely (max_optlen >PAGE_SIZE )) +return -EINVAL ; + +if (likely (max_optlen <=sizeof (ctx->buf))) { + ctx->optval = ctx->buf; + }else { + ctx->optval =kzalloc (max_optlen,GFP_USER ); +if (!ctx->optval) +return -ENOMEM ; + } + + ctx->optval_end = ctx->optval + max_optlen; + ctx->optlen = max_optlen; + +return 0 ; +} + +static void sockopt_free_buf (struct bpf_sockopt_kern *ctx) +{ +if (unlikely (ctx->optval != ctx->buf)) +kfree (ctx->optval); +} + +int __cgroup_bpf_run_filter_setsockopt (struct sock *sk,int level, +int optname,char __user *optval, +unsigned int optlen) +{ +struct cgroup *cgrp =sock_cgroup_ptr (&sk->sk_cgrp_data); +struct bpf_sockopt_kern ctx = { + .sk = sk, + .level = level, + .optname = optname, + }; +int ret; + +/* Opportunistic check to see whether we have any BPF program +* attached to the hook so we don't waste time allocating +* memory and locking the socket. +*/ +if (__cgroup_bpf_prog_array_is_empty (cgrp,BPF_CGROUP_SETSOCKOPT )) +return 0 ; + + ret =sockopt_alloc_buf (&ctx, optlen); +if (ret) +return ret; + +if (copy_from_user (ctx.optval, optval, optlen) !=0 ) { +sockopt_free_buf (&ctx); +return -EFAULT ; + } + +lock_sock (sk); + ret =BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY ( + cgrp->bpf.effective[BPF_CGROUP_SETSOCKOPT ], + &ctx,BPF_PROG_RUN ); +release_sock (sk); + +sockopt_free_buf (&ctx); + +return ret; +} +EXPORT_SYMBOL (__cgroup_bpf_run_filter_setsockopt); + +int __cgroup_bpf_run_filter_getsockopt (struct sock *sk,int level, +int optname,char __user *optval, +int __user *optlen) +{ +struct cgroup *cgrp =sock_cgroup_ptr (&sk->sk_cgrp_data); +struct bpf_sockopt_kern ctx = { + .sk = sk, + .level = level, + .optname = optname, + }; +int max_optlen; +int ret; + +/* Opportunistic check to see whether we have any BPF program +* attached to the hook so we don't waste time allocating +* memory and locking the socket. +*/ +if (__cgroup_bpf_prog_array_is_empty (cgrp,BPF_CGROUP_GETSOCKOPT )) +return 0 ; + +if (get_user (max_optlen, optlen)) +return -EFAULT ; + + ret =sockopt_alloc_buf (&ctx, max_optlen); +if (ret) +return ret; + +lock_sock (sk); + ret = BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(kernel/bpf/cgroup.c int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, · line 1045
Quoted 10 lines
kernel/bpf/cgroup.c +10
-0+ cgrp->bpf.effective[BPF_CGROUP_GETSOCKOPT ], + &ctx, BPF_PROG_RUN); +release_sock (sk); + +if (ret <0 ) { +sockopt_free_buf (&ctx); +return ret; + } + +if (ctx.optlen > max_optlen) {v4 looks very good.
Quoted 5 lines
Quoted 3 lines
One minor question here, what may be the use case if the
bpf_progmakes ctx.optlen <max_optlen(==user's input)but sets ret == 0 (i.e. kernel should continue to handle it)?Do you mean bpf fills in optval + adjusts optlen, but returns 0? I don't see any usecase for that, bpf's output would be ignored in that case.
Yes,
Quoted 3 lines
Do you think we should have an additional check to make sure ctx.optval data is zero and ctx.optlen is not changed in that case?
or skip the copy_to_user() and put_user() when ret == 0?
Quoted 19 lines
Quoted 19 lines
Quoted 19 lines
kernel/bpf/cgroup.c +18
-0+sockopt_free_buf (&ctx); +return -EFAULT ; + } + +if (copy_to_user (optval, ctx.optval, ctx.optlen) !=0 ) { +sockopt_free_buf (&ctx); +return -EFAULT ; + } + +sockopt_free_buf (&ctx); + +if (put_user (ctx.optlen, optlen)) +return -EFAULT ; + +return ret; +} +EXPORT_SYMBOL (__cgroup_bpf_run_filter_getsockopt); +
PATCH v4 2/8 bpf: sync bpf.h to tools/
In reply to Stanislav FomichevExport new prog type and hook points to the libbpf.
--- tools/include/uapi/linux/bpf.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
tools/include/uapi/linux/bpf.h +14 -0
@@ -170,6 +170,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_FLOW_DISSECTOR ,BPF_PROG_TYPE_CGROUP_SYSCTL ,BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE , + BPF_PROG_TYPE_CGROUP_SOCKOPT, }; enum bpf_attach_type {
@@ -192,6 +193,8 @@ enum bpf_attach_type {BPF_LIRC_MODE2 ,BPF_FLOW_DISSECTOR ,BPF_CGROUP_SYSCTL , +BPF_CGROUP_GETSOCKOPT , +BPF_CGROUP_SETSOCKOPT , __MAX_BPF_ATTACH_TYPE };
@@ -3533,4 +3536,15 @@ struct bpf_sysctl {*/ }; +struct bpf_sockopt { + __bpf_md_ptr(struct bpf_sock *, sk); + __bpf_md_ptr(void *, optval); + __bpf_md_ptr(void *, optval_end); + +__s32 level; +__s32 optname; + +__u32 optlen; +}; +#endif /* _UAPI__LINUX_BPF_H__ */
Signature · 1 line
2.22.0.rc2.383.gf4fbbf30c2-goog
PATCH v4 3/8 libbpf: support sockopt hooks
In reply to Stanislav FomichevMake libbpf aware of new sockopt hooks so it can derive prog type and hook point from the section names.
--- tools/lib/bpf/libbpf.c | 5 +++++ tools/lib/bpf/libbpf_probes.c | 1 + 2 files changed, 6 insertions(+)
tools/lib/bpf/libbpf.c +5 -0
@@ -2243,6 +2243,7 @@ static bool bpf_prog_type__needs_kver(enum bpf_prog_type type)case BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE :case BPF_PROG_TYPE_PERF_EVENT :case BPF_PROG_TYPE_CGROUP_SYSCTL : +case BPF_PROG_TYPE_CGROUP_SOCKOPT :return false;case BPF_PROG_TYPE_KPROBE :default :
@@ -3196,6 +3197,10 @@ static const struct { BPF_CGROUP_UDP6_SENDMSG),BPF_EAPROG_SEC ("cgroup/sysctl" ,BPF_PROG_TYPE_CGROUP_SYSCTL ,BPF_CGROUP_SYSCTL ), +BPF_EAPROG_SEC ("cgroup/getsockopt" ,BPF_PROG_TYPE_CGROUP_SOCKOPT , +BPF_CGROUP_GETSOCKOPT ), +BPF_EAPROG_SEC ("cgroup/setsockopt" ,BPF_PROG_TYPE_CGROUP_SOCKOPT , + BPF_CGROUP_SETSOCKOPT), };#undef BPF_PROG_SEC_IMPL
tools/lib/bpf/libbpf_probes.c +1 -0
@@ -101,6 +101,7 @@ probe_load(enum bpf_prog_type prog_type, const struct bpf_insn *insns,case BPF_PROG_TYPE_SK_REUSEPORT :case BPF_PROG_TYPE_FLOW_DISSECTOR :case BPF_PROG_TYPE_CGROUP_SYSCTL : +case BPF_PROG_TYPE_CGROUP_SOCKOPT :default :break ; }
Signature · 1 line
2.22.0.rc2.383.gf4fbbf30c2-goog
PATCH v4 4/8 selftests/bpf: test sockopt section name
In reply to Stanislav FomichevAdd tests that make sure libbpf section detection works.
--- tools/testing/selftests/bpf/test_section_names.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
tools/testing/selftests/bpf/test_section_names.c +10 -0
@@ -124,6 +124,16 @@ static struct sec_name_test tests[] = { {0, BPF_PROG_TYPE_CGROUP_SYSCTL, BPF_CGROUP_SYSCTL}, {0, BPF_CGROUP_SYSCTL}, }, + { +"cgroup/getsockopt" , + {0, BPF_PROG_TYPE_CGROUP_SOCKOPT, BPF_CGROUP_GETSOCKOPT}, + {0, BPF_CGROUP_GETSOCKOPT}, + }, + { +"cgroup/setsockopt" , + {0, BPF_PROG_TYPE_CGROUP_SOCKOPT, BPF_CGROUP_SETSOCKOPT}, + {0, BPF_CGROUP_SETSOCKOPT}, + }, };static int test_prog_type_by_name (const struct sec_name_test *test)
Signature · 1 line
2.22.0.rc2.383.gf4fbbf30c2-goog
PATCH v4 5/8 selftests/bpf: add sockopt test
In reply to Stanislav FomichevAdd sockopt selftests:
* require proper expected_attach_type
* enforce context field read/write access
* test bpf_sockopt_handled handler
* test EPERM
* test limiting optlen from getsockopt
* test out-of-bounds access
v3:
* use DW for optval{,_end} loads
v2:
* use return code 2 for kernel bypass
--- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 3 +- tools/testing/selftests/bpf/test_sockopt.c | 773 +++++++++++++++++++++ 3 files changed, 776 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/test_sockopt.c
tools/testing/selftests/bpf/.gitignore +1 -0
@@ -39,3 +39,4 @@ libbpf.so.* test_hashmap test_btf_dump xdping +test_sockopt
tools/testing/selftests/bpf/Makefile +2 -1
@@ -26,7 +26,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test test_sock test_btf test_sockmap test_lirc_mode2_user get_cgroup_id_user \ test_socket_cookie test_cgroup_storage test_select_reuseport test_section_names \ test_netcnt test_tcpnotify_user test_sock_fields test_sysctl test_hashmap \ - test_btf_dump test_cgroup_attach xdping + test_btf_dump test_cgroup_attach xdping test_sockopt BPF_OBJ_FILES = $(patsubst %.c,%.o, $(notdir $(wildcard progs/*.c))) TEST_GEN_FILES = $(BPF_OBJ_FILES)
@@ -101,6 +101,7 @@ $(OUTPUT)/test_netcnt: cgroup_helpers.c $(OUTPUT)/test_sock_fields: cgroup_helpers.c $(OUTPUT)/test_sysctl: cgroup_helpers.c $(OUTPUT)/test_cgroup_attach: cgroup_helpers.c +$(OUTPUT)/test_sockopt: cgroup_helpers.c .PHONY: force
tools/testing/selftests/bpf/test_sockopt.c new file +773 -0
773 lines not on this page · raw message
Signature · 1 line
2.22.0.rc2.383.gf4fbbf30c2-goog
PATCH v4 6/8 selftests/bpf: add sockopt test that exercises sk helpers
In reply to Stanislav Fomichevsocktop test that introduces new SOL_CUSTOM sockopt level and stores whatever users sets in sk storage. Whenever getsockopt is called, the original value is retrieved.
v4:
* don't call bpf_sk_fullsock helper
v3:
* drop (__u8 *)(long) casts for optval{,_end}
v2:
* new test
--- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 3 +- .../testing/selftests/bpf/progs/sockopt_sk.c | 67 ++++++++ tools/testing/selftests/bpf/test_sockopt_sk.c | 156 ++++++++++++++++++ 4 files changed, 226 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/progs/sockopt_sk.c create mode 100644 tools/testing/selftests/bpf/test_sockopt_sk.c
tools/testing/selftests/bpf/.gitignore +1 -0
@@ -40,3 +40,4 @@ test_hashmap test_btf_dump xdping test_sockopt +test_sockopt_sk
tools/testing/selftests/bpf/Makefile +2 -1
@@ -26,7 +26,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test test_sock test_btf test_sockmap test_lirc_mode2_user get_cgroup_id_user \ test_socket_cookie test_cgroup_storage test_select_reuseport test_section_names \ test_netcnt test_tcpnotify_user test_sock_fields test_sysctl test_hashmap \ - test_btf_dump test_cgroup_attach xdping test_sockopt + test_btf_dump test_cgroup_attach xdping test_sockopt test_sockopt_sk BPF_OBJ_FILES = $(patsubst %.c,%.o, $(notdir $(wildcard progs/*.c))) TEST_GEN_FILES = $(BPF_OBJ_FILES)
@@ -102,6 +102,7 @@ $(OUTPUT)/test_sock_fields: cgroup_helpers.c $(OUTPUT)/test_sysctl: cgroup_helpers.c $(OUTPUT)/test_cgroup_attach: cgroup_helpers.c $(OUTPUT)/test_sockopt: cgroup_helpers.c +$(OUTPUT)/test_sockopt_sk: cgroup_helpers.c .PHONY: force
tools/testing/selftests/bpf/progs/sockopt_sk.c new file +67 -0
67 lines not on this page · raw message
tools/testing/selftests/bpf/test_sockopt_sk.c new file +156 -0
156 lines not on this page · raw message
Signature · 1 line
2.22.0.rc2.383.gf4fbbf30c2-goog
PATCH v4 7/8 bpf: add sockopt documentation
In reply to Stanislav FomichevProvide user documentation about sockopt prog type and cgroup hooks.
v2:
* use return code 2 for kernel bypass
--- Documentation/bpf/index.rst | 1 + Documentation/bpf/prog_cgroup_sockopt.rst | 39 +++++++++++++++++++++++ 2 files changed, 40 insertions(+) create mode 100644 Documentation/bpf/prog_cgroup_sockopt.rst
Documentation/bpf/index.rst +1 -0
@@ -42,6 +42,7 @@ Program types .. toctree:: :maxdepth: 1 + prog_cgroup_sockopt prog_cgroup_sysctl prog_flow_dissector
Documentation/bpf/prog_cgroup_sockopt.rst new file +39 -0
@@ -0,0 +1,39 @@ +.. SPDX-License-Identifier: GPL-2.0 + +===================== +BPF_PROG_TYPE_SOCKOPT +===================== + +``BPF_PROG_TYPE_SOCKOPT`` program type can be attached to two cgroup hooks: + +* ``BPF_CGROUP_GETSOCKOPT`` - called every time process executes ``getsockopt`` + system call. +* ``BPF_CGROUP_SETSOCKOPT`` - called every time process executes ``setsockopt`` + system call. + +The context (``struct bpf_sockopt``) has associated socket (``sk``) and +all input arguments: ``level``, ``optname``, ``optval`` and ``optlen``. + +BPF_CGROUP_SETSOCKOPT +===================== + +``BPF_CGROUP_SETSOCKOPT`` has a read-only context and this hook has +access to cgroup and socket local storage. + +BPF_CGROUP_GETSOCKOPT +===================== + +``BPF_CGROUP_GETSOCKOPT`` has to fill in ``optval`` and adjust +``optlen`` accordingly. Input ``optlen`` contains the maximum length +of data that can be returned to the userspace. In other words, BPF +program can't increase ``optlen``, it can only decrease it. + +Return Type +=========== + +* ``0`` - reject the syscall, ``EPERM`` will be returned to the userspace. +* ``1`` - success: after returning from the BPF hook, kernel will also + handle this socket option. +* ``2`` - success: after returning from the BPF hook, kernel will _not_ + handle this socket option; control will be returned to the userspace + instead.
Signature · 1 line
2.22.0.rc2.383.gf4fbbf30c2-goog
PATCH v4 8/8 bpftool: support cgroup sockopt
In reply to Stanislav FomichevSupport sockopt prog type and cgroup hooks in the bpftool.
--- tools/bpf/bpftool/Documentation/bpftool-cgroup.rst | 7 +++++-- tools/bpf/bpftool/Documentation/bpftool-prog.rst | 2 +- tools/bpf/bpftool/bash-completion/bpftool | 8 +++++--- tools/bpf/bpftool/cgroup.c | 5 ++++- tools/bpf/bpftool/main.h | 1 + tools/bpf/bpftool/prog.c | 3 ++- 6 files changed, 18 insertions(+), 8 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-cgroup.rst +5 -2
@@ -29,7 +29,8 @@ CGROUP COMMANDS | *PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* } | *ATTACH_TYPE* := { **ingress** | **egress** | **sock_create** | **sock_ops** | **device** | | **bind4** | **bind6** | **post_bind4** | **post_bind6** | **connect4** | **connect6** | -| **sendmsg4** | **sendmsg6** | **sysctl** } +| **sendmsg4** | **sendmsg6** | **sysctl** | **getsockopt** | +| **setsockopt** } | *ATTACH_FLAGS* := { **multi** | **override** } DESCRIPTION
@@ -86,7 +87,9 @@ DESCRIPTION unconnected udp4 socket (since 4.18); **sendmsg6** call to sendto(2), sendmsg(2), sendmmsg(2) for an unconnected udp6 socket (since 4.18); - **sysctl** sysctl access (since 5.2). + **sysctl** sysctl access (since 5.2); + **getsockopt** call to getsockopt (since 5.3); + **setsockopt** call to setsockopt (since 5.3). **bpftool cgroup detach** *CGROUP* *ATTACH_TYPE* *PROG* Detach *PROG* from the cgroup *CGROUP* and attach type
tools/bpf/bpftool/Documentation/bpftool-prog.rst +1 -1
@@ -40,7 +40,7 @@ PROG COMMANDS | **lwt_seg6local** | **sockops** | **sk_skb** | **sk_msg** | **lirc_mode2** | | **cgroup/bind4** | **cgroup/bind6** | **cgroup/post_bind4** | **cgroup/post_bind6** | | **cgroup/connect4** | **cgroup/connect6** | **cgroup/sendmsg4** | **cgroup/sendmsg6** | -| **cgroup/sysctl** +| **cgroup/sysctl** | **cgroup/getsockopt** | **cgroup/setsockopt** | } | *ATTACH_TYPE* := { | **msg_verdict** | **stream_verdict** | **stream_parser** | **flow_dissector**
tools/bpf/bpftool/bash-completion/bpftool +5 -3
@@ -378,7 +378,8 @@ _bpftool() cgroup/connect4 cgroup/connect6 \ cgroup/sendmsg4 cgroup/sendmsg6 \ cgroup/post_bind4 cgroup/post_bind6 \ - cgroup/sysctl" -- \ + cgroup/sysctl cgroup/getsockopt \ + cgroup/setsockopt" -- \ "$cur" ) ) return 0 ;;
@@ -688,7 +689,8 @@ _bpftool() attach|detach) local ATTACH_TYPES='ingress egress sock_create sock_ops \ device bind4 bind6 post_bind4 post_bind6 connect4 \ - connect6 sendmsg4 sendmsg6 sysctl' + connect6 sendmsg4 sendmsg6 sysctl getsockopt \ + setsockopt' local ATTACH_FLAGS='multi override' local PROG_TYPE='id pinned tag' case $prev in
@@ -698,7 +700,7 @@ _bpftool() ;; ingress|egress|sock_create|sock_ops|device|bind4|bind6|\ post_bind4|post_bind6|connect4|connect6|sendmsg4|\ - sendmsg6|sysctl) + sendmsg6|sysctl|getsockopt|setsockopt) COMPREPLY=( $( compgen -W "$PROG_TYPE" -- \ "$cur" ) ) return 0
tools/bpf/bpftool/cgroup.c +4 -1
@@ -25,7 +25,8 @@" ATTACH_TYPE := { ingress | egress | sock_create |\n" \" sock_ops | device | bind4 | bind6 |\n" \" post_bind4 | post_bind6 | connect4 |\n" \ -" connect6 | sendmsg4 | sendmsg6 | sysctl }" +" connect6 | sendmsg4 | sendmsg6 | sysctl |\n" \ +" getsockopt | setsockopt }" static const char * const attach_type_strings[] = { [BPF_CGROUP_INET_INGRESS] = "ingress",
@@ -42,6 +43,8 @@ static const char * const attach_type_strings[] = { [BPF_CGROUP_UDP4_SENDMSG] = "sendmsg4", [BPF_CGROUP_UDP6_SENDMSG] = "sendmsg6", [BPF_CGROUP_SYSCTL] = "sysctl", + [BPF_CGROUP_GETSOCKOPT] = "getsockopt", + [BPF_CGROUP_SETSOCKOPT] = "setsockopt", [__MAX_BPF_ATTACH_TYPE] = NULL, };
tools/bpf/bpftool/main.h +1 -0
@@ -74,6 +74,7 @@ static const char * const prog_type_name[] = { [BPF_PROG_TYPE_SK_REUSEPORT] = "sk_reuseport", [BPF_PROG_TYPE_FLOW_DISSECTOR] = "flow_dissector", [BPF_PROG_TYPE_CGROUP_SYSCTL] = "cgroup_sysctl", + [BPF_PROG_TYPE_CGROUP_SOCKOPT] = "cgroup_sockopt", };extern const char *const map_type_name[];
tools/bpf/bpftool/prog.c +2 -1
@@ -1070,7 +1070,8 @@ static int do_help(int argc, char **argv)" sk_reuseport | flow_dissector | cgroup/sysctl |\n" " cgroup/bind4 | cgroup/bind6 | cgroup/post_bind4 |\n" " cgroup/post_bind6 | cgroup/connect4 | cgroup/connect6 |\n" -" cgroup/sendmsg4 | cgroup/sendmsg6 }\n" +" cgroup/sendmsg4 | cgroup/sendmsg6 | cgroup/getsockopt |\n" +" cgroup/setsockopt }\n" " ATTACH_TYPE := { msg_verdict | stream_verdict | stream_parser |\n" " flow_dissector }\n" " " HELP_SPEC_OPTIONS "\n"
Signature · 1 line
2.22.0.rc2.383.gf4fbbf30c2-goog
Quoted 12 lines · Stanislav Fomichev
This series implements two new per-cgroup hooks: getsockopt and setsockopt along with a new sockopt program type. The idea is pretty similar to recently introduced cgroup sysctl hooks, but implementation is simpler (no need to convert to/from strings).
What this can be applied to:
* move business logic of what tos/priority/etc can be set by containers (either pass or reject)
* handle existing options (or introduce new ones) differently by propagating some information in cgroup/socket local storage
Quoted 3 lines
Compared to a simple syscall/{g,s}etsockopt tracepoint, those hooks are context aware. Meaning, they can access underlying socket and use cgroup and socket local storage.
It's customary to include version change log for the whole patch set in a cover letter vs first patch. Please include it in the future. Thanks!
Quoted 47 lines
Stanislav Fomichev (8):
bpf: implement getsockopt and setsockopt hooks
bpf: syncbpf.htotools/
libbpf: support sockopt hooks
selftests/bpf: test sockopt section name selftests/bpf: add sockopt test
selftests/bpf: add sockopt test that exercises sk helpers
bpf: add sockopt documentation
bpftool: support cgroup sockoptDocumentation/bpf/index.rst | 1 + Documentation/bpf/prog_cgroup_sockopt.rst | 39 + include/linux/bpf-cgroup.h | 29 + include/linux/bpf.h | 45 + include/linux/bpf_types.h | 1 + include/linux/filter.h | 13 + include/uapi/linux/bpf.h | 13 + kernel/bpf/cgroup.c | 262 ++++++ kernel/bpf/core.c | 9 + kernel/bpf/syscall.c | 19 + kernel/bpf/verifier.c | 15 + net/core/filter.c | 2 +- net/socket.c | 18 + .../bpftool/Documentation/bpftool-cgroup.rst | 7 +- .../bpftool/Documentation/bpftool-prog.rst | 2 +- tools/bpf/bpftool/bash-completion/bpftool | 8 +- tools/bpf/bpftool/cgroup.c | 5 +- tools/bpf/bpftool/main.h | 1 + tools/bpf/bpftool/prog.c | 3 +- tools/include/uapi/linux/bpf.h | 14 + tools/lib/bpf/libbpf.c | 5 + tools/lib/bpf/libbpf_probes.c | 1 + tools/testing/selftests/bpf/.gitignore | 2 + tools/testing/selftests/bpf/Makefile | 4 +- .../testing/selftests/bpf/progs/sockopt_sk.c | 67 ++ .../selftests/bpf/test_section_names.c | 10 + tools/testing/selftests/bpf/test_sockopt.c | 773 ++++++++++++++++++ tools/testing/selftests/bpf/test_sockopt_sk.c | 156 ++++ 28 files changed, 1514 insertions(+), 10 deletions(-) create mode 100644 Documentation/bpf/prog_cgroup_sockopt.rst create mode 100644 tools/testing/selftests/bpf/progs/sockopt_sk.c create mode 100644 tools/testing/selftests/bpf/test_sockopt.c create mode 100644 tools/testing/selftests/bpf/test_sockopt_sk.c--
2.22.0.rc2.383.gf4fbbf30c2-goog
Stanislav Fomichev
reviewerQuoted 13 lines · Andrii Nakryiko
Quoted 12 lines · Stanislav Fomichev
This series implements two new per-cgroup hooks: getsockopt and setsockopt along with a new sockopt program type. The idea is pretty similar to recently introduced cgroup sysctl hooks, but implementation is simpler (no need to convert to/from strings).
What this can be applied to:
* move business logic of what tos/priority/etc can be set by containers (either pass or reject)
* handle existing options (or introduce new ones) differently by propagating some information in cgroup/socket local storage
Quoted 7 lines
Quoted 3 lines
Compared to a simple syscall/{g,s}etsockopt tracepoint, those hooks are context aware. Meaning, they can access underlying socket and use cgroup and socket local storage.
It's customary to include version change log for the whole patch set in a cover letter vs first patch. Please include it in the future. Thanks!
I don't think there is a precedent that strongly favors one way or the other. If you search the mailing list, you can find both versions:
cover letter has short version log vs each patch has detailed version log.
My reasoning for putting version log in the particular patches is to make life of people who review the changes easier. For example, if a particular patch doesn't have a version change log, it means that the patch is in the same state as in the previous version and doesn't need another round of scrutiny.
Quoted 46 lines
Quoted 46 lines
Stanislav Fomichev (8):
bpf: implement getsockopt and setsockopt hooks
bpf: syncbpf.htotools/
libbpf: support sockopt hooks
selftests/bpf: test sockopt section name selftests/bpf: add sockopt test
selftests/bpf: add sockopt test that exercises sk helpers
bpf: add sockopt documentation
bpftool: support cgroup sockoptDocumentation/bpf/index.rst | 1 + Documentation/bpf/prog_cgroup_sockopt.rst | 39 + include/linux/bpf-cgroup.h | 29 + include/linux/bpf.h | 45 + include/linux/bpf_types.h | 1 + include/linux/filter.h | 13 + include/uapi/linux/bpf.h | 13 + kernel/bpf/cgroup.c | 262 ++++++ kernel/bpf/core.c | 9 + kernel/bpf/syscall.c | 19 + kernel/bpf/verifier.c | 15 + net/core/filter.c | 2 +- net/socket.c | 18 + .../bpftool/Documentation/bpftool-cgroup.rst | 7 +- .../bpftool/Documentation/bpftool-prog.rst | 2 +- tools/bpf/bpftool/bash-completion/bpftool | 8 +- tools/bpf/bpftool/cgroup.c | 5 +- tools/bpf/bpftool/main.h | 1 + tools/bpf/bpftool/prog.c | 3 +- tools/include/uapi/linux/bpf.h | 14 + tools/lib/bpf/libbpf.c | 5 + tools/lib/bpf/libbpf_probes.c | 1 + tools/testing/selftests/bpf/.gitignore | 2 + tools/testing/selftests/bpf/Makefile | 4 +- .../testing/selftests/bpf/progs/sockopt_sk.c | 67 ++ .../selftests/bpf/test_section_names.c | 10 + tools/testing/selftests/bpf/test_sockopt.c | 773 ++++++++++++++++++ tools/testing/selftests/bpf/test_sockopt_sk.c | 156 ++++ 28 files changed, 1514 insertions(+), 10 deletions(-) create mode 100644 Documentation/bpf/prog_cgroup_sockopt.rst create mode 100644 tools/testing/selftests/bpf/progs/sockopt_sk.c create mode 100644 tools/testing/selftests/bpf/test_sockopt.c create mode 100644 tools/testing/selftests/bpf/test_sockopt_sk.c--
2.22.0.rc2.383.gf4fbbf30c2-goog