bpf: getsockopt and setsockopt hooks
Subsystems (11)
v3 cover bpf: getsockopt and setsockopt hooks
This series implements two new per-cgroup hooks: getsockopt and setsockopt along with a new sockopt program type. The idea is pretty similar to recently introduced cgroup sysctl hooks, but implementation is simpler (no need to convert to/from strings).
What this can be applied to:
* move business logic of what tos/priority/etc can be set by containers (either pass or reject)
* handle existing options (or introduce new ones) differently by propagating some information in cgroup/socket local storage
Compared to a simple syscall/{g,s}etsockopt tracepoint, those hooks are context aware. Meaning, they can access underlying socket and use cgroup and socket local storage.
Stanislav Fomichev (8):
bpf: implement getsockopt and setsockopt hooks
bpf: sync bpf.h to tools/
libbpf: support sockopt hooks
selftests/bpf: test sockopt section name selftests/bpf: add sockopt test
selftests/bpf: add sockopt test that exercises sk helpers
bpf: add sockopt documentation
bpftool: support cgroup sockopt
Documentation/bpf/index.rst | 1 + Documentation/bpf/prog_cgroup_sockopt.rst | 39 + include/linux/bpf-cgroup.h | 29 + include/linux/bpf.h | 46 ++ include/linux/bpf_types.h | 1 + include/linux/filter.h | 13 + include/uapi/linux/bpf.h | 13 + kernel/bpf/cgroup.c | 264 ++++++ kernel/bpf/core.c | 9 + kernel/bpf/syscall.c | 19 + kernel/bpf/verifier.c | 15 + net/core/filter.c | 4 +- net/socket.c | 18 + .../bpftool/Documentation/bpftool-cgroup.rst | 7 +- .../bpftool/Documentation/bpftool-prog.rst | 2 +- tools/bpf/bpftool/bash-completion/bpftool | 8 +- tools/bpf/bpftool/cgroup.c | 5 +- tools/bpf/bpftool/main.h | 1 + tools/bpf/bpftool/prog.c | 3 +- tools/include/uapi/linux/bpf.h | 14 + tools/lib/bpf/libbpf.c | 5 + tools/lib/bpf/libbpf_probes.c | 1 + tools/testing/selftests/bpf/.gitignore | 2 + tools/testing/selftests/bpf/Makefile | 4 +- .../testing/selftests/bpf/progs/sockopt_sk.c | 77 ++ .../selftests/bpf/test_section_names.c | 10 + tools/testing/selftests/bpf/test_sockopt.c | 773 ++++++++++++++++++ tools/testing/selftests/bpf/test_sockopt_sk.c | 156 ++++ 28 files changed, 1528 insertions(+), 11 deletions(-) create mode 100644 Documentation/bpf/prog_cgroup_sockopt.rst create mode 100644 tools/testing/selftests/bpf/progs/sockopt_sk.c create mode 100644 tools/testing/selftests/bpf/test_sockopt.c create mode 100644 tools/testing/selftests/bpf/test_sockopt_sk.c
Signature · 1 line
2.22.0.rc1.311.g5d7573a151-goog
PATCH v3 1/8 bpf: implement getsockopt and setsockopt hooks
Implement new BPF_PROG_TYPE_CGROUP_SOCKOPT program type and BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
BPF_CGROUP_SETSOCKOPT get a read-only view of the setsockopt arguments. BPF_CGROUP_GETSOCKOPT can modify the supplied buffer.
Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.
The buffer memory is pre-allocated (because I don't think there is a precedent for working with __user memory from bpf). This might be slow to do for each {s,g}etsockopt call, that's why I've added __cgroup_bpf_prog_array_is_empty that exits early if there is nothing attached to a cgroup. Note, however, that there is a race between __cgroup_bpf_prog_array_is_empty and BPF_PROG_RUN_ARRAY where cgroup program layout might have changed; this should not be a problem because in general there is a race between multiple calls to {s,g}etsocktop and user adding/removing bpf progs from a cgroup.
The return code of the BPF program is handled as follows:
* 0: EPERM
* 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
* 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF prog exits
v3:
* typos in BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY comments (Andrii Nakryiko)
* reverse christmas tree in BPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY (Andrii Nakryiko)
* use __bpf_md_ptr instead of __u32 for optval{,_end} (Martin Lau)
* use BPF_FIELD_SIZEOF() for consistency (Martin Lau)
* new CG_SOCKOPT_ACCESS macro to wrap repeated parts
v2:
* moved bpf_sockopt_kern fields around to remove a hole (Martin Lau)
* aligned bpf_sockopt_kern->buf to 8 bytes (Martin Lau)
* bpf_prog_array_is_empty instead of bpf_prog_array_length (Martin Lau)
* added [0,2] return code check to verifier (Martin Lau)
* dropped unused buf[64] from the stack (Martin Lau)
* use PTR_TO_SOCKET for bpf_sockopt->sk (Martin Lau)
* dropped bpf_target_off from ctx rewrites (Martin Lau)
* use return code for kernel bypass (Martin Lau & Andrii Nakryiko)
--- include/linux/bpf-cgroup.h | 29 ++++ include/linux/bpf.h | 46 +++++++ include/linux/bpf_types.h | 1 + include/linux/filter.h | 13 ++ include/uapi/linux/bpf.h | 13 ++ kernel/bpf/cgroup.c | 264 +++++++++++++++++++++++++++++++++++++ kernel/bpf/core.c | 9 ++ kernel/bpf/syscall.c | 19 +++ kernel/bpf/verifier.c | 15 +++ net/core/filter.c | 4 +- net/socket.c | 18 +++ 11 files changed, 429 insertions(+), 2 deletions(-)
include/linux/bpf-cgroup.h +29 -0
@@ -124,6 +124,13 @@ int __cgroup_bpf_run_filter_sysctl(struct ctl_table_header *head, loff_t *ppos, void **new_buf, enum bpf_attach_type type); +int __cgroup_bpf_run_filter_setsockopt (struct sock *sock,int level, +int optname,char __user *optval, +unsigned int optlen); +int __cgroup_bpf_run_filter_getsockopt (struct sock *sock,int level, +int optname,char __user *optval, +int __user *optlen); +static inline enum bpf_cgroup_storage_type cgroup_storage_type (struct bpf_map *map) {
@@ -280,6 +287,26 @@ int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key, __ret; \ }) +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock, level, optname, optval, optlen) \ +({ \ + int __ret = 0; \ + if (cgroup_bpf_enabled) \ + __ret = __cgroup_bpf_run_filter_setsockopt(sock, level, \ + optname, optval, \ + optlen); \ + __ret; \ +}) + +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock, level, optname, optval, optlen) \ +({ \ + int __ret = 0; \ + if (cgroup_bpf_enabled) \ + __ret = __cgroup_bpf_run_filter_getsockopt(sock, level, \ + optname, optval, \ + optlen); \ + __ret; \ +}) +int cgroup_bpf_prog_attach (const union bpf_attr *attr,enum bpf_prog_type ptype,struct bpf_prog *prog); int cgroup_bpf_prog_detach(const union bpf_attr *attr,
@@ -349,6 +376,8 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map,#define BPF_CGROUP_RUN_PROG_SOCK_OPS (sock_ops) ({ 0; })#define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP (type,major,minor,access) ({ 0; })#define BPF_CGROUP_RUN_PROG_SYSCTL (head,table,write,buf,count,pos,nbuf) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock, level, optname, optval, optlen) ({ 0; }) +#define BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock, level, optname, optval, optlen) ({ 0; })#define for_each_cgroup_storage_type (stype) for (; false; )
include/linux/bpf.h +46 -0
64 lines not on this page · raw message
include/linux/bpf_types.h +1 -0
@@ -30,6 +30,7 @@ BPF_PROG_TYPE(BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE, raw_tracepoint_writable)#ifdef CONFIG_CGROUP_BPF BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_DEVICE, cg_dev) BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SYSCTL, cg_sysctl) +BPF_PROG_TYPE(BPF_PROG_TYPE_CGROUP_SOCKOPT, cg_sockopt)#endif #ifdef CONFIG_BPF_LIRC_MODE2 BPF_PROG_TYPE (BPF_PROG_TYPE_LIRC_MODE2 , lirc_mode2)
include/linux/filter.h +13 -0
@@ -1199,4 +1199,17 @@ struct bpf_sysctl_kern { u64 tmp_reg; }; +struct bpf_sockopt_kern { +struct sock *sk; +u8 *optval; +u8 *optval_end; +s32 level; +s32 optname; +u32 optlen; + +/* Small on-stack optval buffer to avoid small allocations. +*/ +u8 buf[64 ]__aligned(8) ; +}; +#endif /* __LINUX_FILTER_H__ */
include/uapi/linux/bpf.h +13 -0
@@ -170,6 +170,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_FLOW_DISSECTOR ,BPF_PROG_TYPE_CGROUP_SYSCTL ,BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE , + BPF_PROG_TYPE_CGROUP_SOCKOPT, }; enum bpf_attach_type {
@@ -192,6 +193,8 @@ enum bpf_attach_type {BPF_LIRC_MODE2 ,BPF_FLOW_DISSECTOR ,BPF_CGROUP_SYSCTL , +BPF_CGROUP_GETSOCKOPT , +BPF_CGROUP_SETSOCKOPT , __MAX_BPF_ATTACH_TYPE };
@@ -3533,4 +3536,14 @@ struct bpf_sysctl {*/ }; +struct bpf_sockopt { + __bpf_md_ptr(struct bpf_sock *, sk); + __bpf_md_ptr(void *, optval); + __bpf_md_ptr(void *, optval_end); + +__s32 level; +__s32 optname; +__u32 optlen; +}; +#endif /* _UAPI__LINUX_BPF_H__ */
kernel/bpf/cgroup.c +264 -0
279 lines not on this page · raw message
kernel/bpf/core.c +9 -0
@@ -1813,6 +1813,15 @@ int bpf_prog_array_length(struct bpf_prog_array *array)return cnt; } +bool bpf_prog_array_is_empty (struct bpf_prog_array *array) +{ +struct bpf_prog_array_item *item; + +for (item = array->items; item->prog; item++) +if (item->prog != &dummy_bpf_prog.prog) +return false; +return true; +} static bool bpf_prog_array_copy_core(struct bpf_prog_array *array, u32 *prog_ids,
kernel/bpf/syscall.c +19 -0
@@ -1596,6 +1596,14 @@ bpf_prog_load_check_attach_type(enum bpf_prog_type prog_type,default :return -EINVAL ; } +case BPF_PROG_TYPE_CGROUP_SOCKOPT : +switch (expected_attach_type) { +case BPF_CGROUP_SETSOCKOPT : +case BPF_CGROUP_GETSOCKOPT : +return 0 ; +default : +return -EINVAL ; + }default :return 0 ; }
@@ -1846,6 +1854,7 @@ static int bpf_prog_attach_check_attach_type(const struct bpf_prog *prog,switch (prog->type) {case BPF_PROG_TYPE_CGROUP_SOCK :case BPF_PROG_TYPE_CGROUP_SOCK_ADDR : +case BPF_PROG_TYPE_CGROUP_SOCKOPT :return attach_type == prog->expected_attach_type ?0 : -EINVAL ;case BPF_PROG_TYPE_CGROUP_SKB : return prog->enforce_expected_attach_type &&
@@ -1916,6 +1925,10 @@ static int bpf_prog_attach(const union bpf_attr *attr)case BPF_CGROUP_SYSCTL : ptype =BPF_PROG_TYPE_CGROUP_SYSCTL ;break ; +case BPF_CGROUP_GETSOCKOPT : +case BPF_CGROUP_SETSOCKOPT : + ptype =BPF_PROG_TYPE_CGROUP_SOCKOPT ; +break ;default :return -EINVAL ; }
@@ -1997,6 +2010,10 @@ static int bpf_prog_detach(const union bpf_attr *attr)case BPF_CGROUP_SYSCTL : ptype =BPF_PROG_TYPE_CGROUP_SYSCTL ;break ; +case BPF_CGROUP_GETSOCKOPT : +case BPF_CGROUP_SETSOCKOPT : + ptype =BPF_PROG_TYPE_CGROUP_SOCKOPT ; +break ;default :return -EINVAL ; }
@@ -2031,6 +2048,8 @@ static int bpf_prog_query(const union bpf_attr *attr,case BPF_CGROUP_SOCK_OPS :case BPF_CGROUP_DEVICE :case BPF_CGROUP_SYSCTL : +case BPF_CGROUP_GETSOCKOPT : +case BPF_CGROUP_SETSOCKOPT :break ;case BPF_LIRC_MODE2 :return lirc_prog_query (attr, uattr);
kernel/bpf/verifier.c +15 -0
@@ -1717,6 +1717,18 @@ static bool may_access_direct_pkt_data(struct bpf_verifier_env *env, env->seen_direct_write = true;return true; + +case BPF_PROG_TYPE_CGROUP_SOCKOPT : +if (t ==BPF_WRITE ) { +if (env->prog->expected_attach_type == +BPF_CGROUP_GETSOCKOPT ) { + env->seen_direct_write = true; +return true; + } +return false; + } +return true; +default :return false; }
@@ -5524,6 +5536,9 @@ static int check_return_code(struct bpf_verifier_env *env)case BPF_PROG_TYPE_CGROUP_DEVICE :case BPF_PROG_TYPE_CGROUP_SYSCTL :break ; +case BPF_PROG_TYPE_CGROUP_SOCKOPT : + range =tnum_range (0 ,2 ); +break ;default :return 0 ; }
net/core/filter.c +2 -2
@@ -1835,7 +1835,7 @@ BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk)return sk_fullsock (sk) ? (unsigned long )sk : (unsigned long )NULL ; } -static const struct bpf_func_proto bpf_sk_fullsock_proto = { +const struct bpf_func_proto bpf_sk_fullsock_proto = { .func = bpf_sk_fullsock, .gpl_only = false, .ret_type =RET_PTR_TO_SOCKET_OR_NULL ,
@@ -5636,7 +5636,7 @@ BPF_CALL_1(bpf_tcp_sock, struct sock *, sk)return (unsigned long )NULL ; } -static const struct bpf_func_proto bpf_tcp_sock_proto = { +const struct bpf_func_proto bpf_tcp_sock_proto = { .func = bpf_tcp_sock, .gpl_only = false, .ret_type =RET_PTR_TO_TCP_SOCK_OR_NULL ,
net/socket.c +18 -0
@@ -2069,6 +2069,15 @@ static int __sys_setsockopt(int fd, int level, int optname,if (err) goto out_put; + err =BPF_CGROUP_RUN_PROG_SETSOCKOPT (sock->sk, level, optname, + optval, optlen); +if (err <0 ) { + goto out_put; + }else if (err >0 ) { + err =0 ; + goto out_put; + } +if (level ==SOL_SOCKET ) err = sock_setsockopt(sock, level, optname, optval,
@@ -2106,6 +2115,15 @@ static int __sys_getsockopt(int fd, int level, int optname,if (err) goto out_put; + err =BPF_CGROUP_RUN_PROG_GETSOCKOPT (sock->sk, level, optname, + optval, optlen); +if (err <0 ) { + goto out_put; + }else if (err >0 ) { + err =0 ; + goto out_put; + } +if (level ==SOL_SOCKET ) err = sock_getsockopt(sock, level, optname, optval,
Signature · 1 line
2.22.0.rc1.311.g5d7573a151-goog
PATCH v3 2/8 bpf: sync bpf.h to tools/
In reply to Stanislav FomichevExport new prog type and hook points to the libbpf.
--- tools/include/uapi/linux/bpf.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
tools/include/uapi/linux/bpf.h +14 -0
@@ -170,6 +170,7 @@ enum bpf_prog_type {BPF_PROG_TYPE_FLOW_DISSECTOR ,BPF_PROG_TYPE_CGROUP_SYSCTL ,BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE , + BPF_PROG_TYPE_CGROUP_SOCKOPT, }; enum bpf_attach_type {
@@ -192,6 +193,8 @@ enum bpf_attach_type {BPF_LIRC_MODE2 ,BPF_FLOW_DISSECTOR ,BPF_CGROUP_SYSCTL , +BPF_CGROUP_GETSOCKOPT , +BPF_CGROUP_SETSOCKOPT , __MAX_BPF_ATTACH_TYPE };
@@ -3533,4 +3536,15 @@ struct bpf_sysctl {*/ }; +struct bpf_sockopt { + __bpf_md_ptr(struct bpf_sock *, sk); + __bpf_md_ptr(void *, optval); + __bpf_md_ptr(void *, optval_end); + +__s32 level; +__s32 optname; + +__u32 optlen; +}; +#endif /* _UAPI__LINUX_BPF_H__ */
Signature · 1 line
2.22.0.rc1.311.g5d7573a151-goog
PATCH v3 3/8 libbpf: support sockopt hooks
In reply to Stanislav FomichevMake libbpf aware of new sockopt hooks so it can derive prog type and hook point from the section names.
--- tools/lib/bpf/libbpf.c | 5 +++++ tools/lib/bpf/libbpf_probes.c | 1 + 2 files changed, 6 insertions(+)
tools/lib/bpf/libbpf.c +5 -0
@@ -2243,6 +2243,7 @@ static bool bpf_prog_type__needs_kver(enum bpf_prog_type type)case BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE :case BPF_PROG_TYPE_PERF_EVENT :case BPF_PROG_TYPE_CGROUP_SYSCTL : +case BPF_PROG_TYPE_CGROUP_SOCKOPT :return false;case BPF_PROG_TYPE_KPROBE :default :
@@ -3196,6 +3197,10 @@ static const struct { BPF_CGROUP_UDP6_SENDMSG),BPF_EAPROG_SEC ("cgroup/sysctl" ,BPF_PROG_TYPE_CGROUP_SYSCTL ,BPF_CGROUP_SYSCTL ), +BPF_EAPROG_SEC ("cgroup/getsockopt" ,BPF_PROG_TYPE_CGROUP_SOCKOPT , +BPF_CGROUP_GETSOCKOPT ), +BPF_EAPROG_SEC ("cgroup/setsockopt" ,BPF_PROG_TYPE_CGROUP_SOCKOPT , + BPF_CGROUP_SETSOCKOPT), };#undef BPF_PROG_SEC_IMPL
tools/lib/bpf/libbpf_probes.c +1 -0
@@ -101,6 +101,7 @@ probe_load(enum bpf_prog_type prog_type, const struct bpf_insn *insns,case BPF_PROG_TYPE_SK_REUSEPORT :case BPF_PROG_TYPE_FLOW_DISSECTOR :case BPF_PROG_TYPE_CGROUP_SYSCTL : +case BPF_PROG_TYPE_CGROUP_SOCKOPT :default :break ; }
Signature · 1 line
2.22.0.rc1.311.g5d7573a151-goog
PATCH v3 4/8 selftests/bpf: test sockopt section name
In reply to Stanislav FomichevAdd tests that make sure libbpf section detection works.
--- tools/testing/selftests/bpf/test_section_names.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
tools/testing/selftests/bpf/test_section_names.c +10 -0
@@ -124,6 +124,16 @@ static struct sec_name_test tests[] = { {0, BPF_PROG_TYPE_CGROUP_SYSCTL, BPF_CGROUP_SYSCTL}, {0, BPF_CGROUP_SYSCTL}, }, + { +"cgroup/getsockopt" , + {0, BPF_PROG_TYPE_CGROUP_SOCKOPT, BPF_CGROUP_GETSOCKOPT}, + {0, BPF_CGROUP_GETSOCKOPT}, + }, + { +"cgroup/setsockopt" , + {0, BPF_PROG_TYPE_CGROUP_SOCKOPT, BPF_CGROUP_SETSOCKOPT}, + {0, BPF_CGROUP_SETSOCKOPT}, + }, };static int test_prog_type_by_name (const struct sec_name_test *test)
Signature · 1 line
2.22.0.rc1.311.g5d7573a151-goog
PATCH v3 5/8 selftests/bpf: add sockopt test
In reply to Stanislav FomichevAdd sockopt selftests:
* require proper expected_attach_type
* enforce context field read/write access
* test bpf_sockopt_handled handler
* test EPERM
* test limiting optlen from getsockopt
* test out-of-bounds access
v3:
* use DW for optval{,_end} loads
v2:
* use return code 2 for kernel bypass
--- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 3 +- tools/testing/selftests/bpf/test_sockopt.c | 773 +++++++++++++++++++++ 3 files changed, 776 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/test_sockopt.c
tools/testing/selftests/bpf/.gitignore +1 -0
@@ -39,3 +39,4 @@ libbpf.so.* test_hashmap test_btf_dump xdping +test_sockopt
tools/testing/selftests/bpf/Makefile +2 -1
@@ -26,7 +26,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test test_sock test_btf test_sockmap test_lirc_mode2_user get_cgroup_id_user \ test_socket_cookie test_cgroup_storage test_select_reuseport test_section_names \ test_netcnt test_tcpnotify_user test_sock_fields test_sysctl test_hashmap \ - test_btf_dump test_cgroup_attach xdping + test_btf_dump test_cgroup_attach xdping test_sockopt BPF_OBJ_FILES = $(patsubst %.c,%.o, $(notdir $(wildcard progs/*.c))) TEST_GEN_FILES = $(BPF_OBJ_FILES)
@@ -101,6 +101,7 @@ $(OUTPUT)/test_netcnt: cgroup_helpers.c $(OUTPUT)/test_sock_fields: cgroup_helpers.c $(OUTPUT)/test_sysctl: cgroup_helpers.c $(OUTPUT)/test_cgroup_attach: cgroup_helpers.c +$(OUTPUT)/test_sockopt: cgroup_helpers.c .PHONY: force
tools/testing/selftests/bpf/test_sockopt.c new file +773 -0
773 lines not on this page · raw message
Signature · 1 line
2.22.0.rc1.311.g5d7573a151-goog
PATCH v3 7/8 bpf: add sockopt documentation
In reply to Stanislav FomichevProvide user documentation about sockopt prog type and cgroup hooks.
v2:
* use return code 2 for kernel bypass
--- Documentation/bpf/index.rst | 1 + Documentation/bpf/prog_cgroup_sockopt.rst | 39 +++++++++++++++++++++++ 2 files changed, 40 insertions(+) create mode 100644 Documentation/bpf/prog_cgroup_sockopt.rst
Documentation/bpf/index.rst +1 -0
@@ -42,6 +42,7 @@ Program types .. toctree:: :maxdepth: 1 + prog_cgroup_sockopt prog_cgroup_sysctl prog_flow_dissector
Documentation/bpf/prog_cgroup_sockopt.rst new file +39 -0
@@ -0,0 +1,39 @@ +.. SPDX-License-Identifier: GPL-2.0 + +===================== +BPF_PROG_TYPE_SOCKOPT +===================== + +``BPF_PROG_TYPE_SOCKOPT`` program type can be attached to two cgroup hooks: + +* ``BPF_CGROUP_GETSOCKOPT`` - called every time process executes ``getsockopt`` + system call. +* ``BPF_CGROUP_SETSOCKOPT`` - called every time process executes ``setsockopt`` + system call. + +The context (``struct bpf_sockopt``) has associated socket (``sk``) and +all input arguments: ``level``, ``optname``, ``optval`` and ``optlen``. + +BPF_CGROUP_SETSOCKOPT +===================== + +``BPF_CGROUP_SETSOCKOPT`` has a read-only context and this hook has +access to cgroup and socket local storage. + +BPF_CGROUP_GETSOCKOPT +===================== + +``BPF_CGROUP_GETSOCKOPT`` has to fill in ``optval`` and adjust +``optlen`` accordingly. Input ``optlen`` contains the maximum length +of data that can be returned to the userspace. In other words, BPF +program can't increase ``optlen``, it can only decrease it. + +Return Type +=========== + +* ``0`` - reject the syscall, ``EPERM`` will be returned to the userspace. +* ``1`` - success: after returning from the BPF hook, kernel will also + handle this socket option. +* ``2`` - success: after returning from the BPF hook, kernel will _not_ + handle this socket option; control will be returned to the userspace + instead.
Signature · 1 line
2.22.0.rc1.311.g5d7573a151-goog
PATCH v3 6/8 selftests/bpf: add sockopt test that exercises sk helpers
In reply to Stanislav Fomichevsocktop test that introduces new SOL_CUSTOM sockopt level and stores whatever users sets in sk storage. Whenever getsockopt is called, the original value is retrieved.
v3:
* drop (__u8 *)(long) casts for optval{,_end}
v2:
* new test
--- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 3 +- .../testing/selftests/bpf/progs/sockopt_sk.c | 77 +++++++++ tools/testing/selftests/bpf/test_sockopt_sk.c | 156 ++++++++++++++++++ 4 files changed, 236 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/progs/sockopt_sk.c create mode 100644 tools/testing/selftests/bpf/test_sockopt_sk.c
tools/testing/selftests/bpf/.gitignore +1 -0
@@ -40,3 +40,4 @@ test_hashmap test_btf_dump xdping test_sockopt +test_sockopt_sk
tools/testing/selftests/bpf/Makefile +2 -1
@@ -26,7 +26,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test test_sock test_btf test_sockmap test_lirc_mode2_user get_cgroup_id_user \ test_socket_cookie test_cgroup_storage test_select_reuseport test_section_names \ test_netcnt test_tcpnotify_user test_sock_fields test_sysctl test_hashmap \ - test_btf_dump test_cgroup_attach xdping test_sockopt + test_btf_dump test_cgroup_attach xdping test_sockopt test_sockopt_sk BPF_OBJ_FILES = $(patsubst %.c,%.o, $(notdir $(wildcard progs/*.c))) TEST_GEN_FILES = $(BPF_OBJ_FILES)
@@ -102,6 +102,7 @@ $(OUTPUT)/test_sock_fields: cgroup_helpers.c $(OUTPUT)/test_sysctl: cgroup_helpers.c $(OUTPUT)/test_cgroup_attach: cgroup_helpers.c $(OUTPUT)/test_sockopt: cgroup_helpers.c +$(OUTPUT)/test_sockopt_sk: cgroup_helpers.c .PHONY: force
tools/testing/selftests/bpf/progs/sockopt_sk.c new file +77 -0
77 lines not on this page · raw message
tools/testing/selftests/bpf/test_sockopt_sk.c new file +156 -0
156 lines not on this page · raw message
Signature · 1 line
2.22.0.rc1.311.g5d7573a151-goog
PATCH v3 8/8 bpftool: support cgroup sockopt
In reply to Stanislav FomichevSupport sockopt prog type and cgroup hooks in the bpftool.
--- tools/bpf/bpftool/Documentation/bpftool-cgroup.rst | 7 +++++-- tools/bpf/bpftool/Documentation/bpftool-prog.rst | 2 +- tools/bpf/bpftool/bash-completion/bpftool | 8 +++++--- tools/bpf/bpftool/cgroup.c | 5 ++++- tools/bpf/bpftool/main.h | 1 + tools/bpf/bpftool/prog.c | 3 ++- 6 files changed, 18 insertions(+), 8 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-cgroup.rst +5 -2
@@ -29,7 +29,8 @@ CGROUP COMMANDS | *PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* } | *ATTACH_TYPE* := { **ingress** | **egress** | **sock_create** | **sock_ops** | **device** | | **bind4** | **bind6** | **post_bind4** | **post_bind6** | **connect4** | **connect6** | -| **sendmsg4** | **sendmsg6** | **sysctl** } +| **sendmsg4** | **sendmsg6** | **sysctl** | **getsockopt** | +| **setsockopt** } | *ATTACH_FLAGS* := { **multi** | **override** } DESCRIPTION
@@ -86,7 +87,9 @@ DESCRIPTION unconnected udp4 socket (since 4.18); **sendmsg6** call to sendto(2), sendmsg(2), sendmmsg(2) for an unconnected udp6 socket (since 4.18); - **sysctl** sysctl access (since 5.2). + **sysctl** sysctl access (since 5.2); + **getsockopt** call to getsockopt (since 5.3); + **setsockopt** call to setsockopt (since 5.3). **bpftool cgroup detach** *CGROUP* *ATTACH_TYPE* *PROG* Detach *PROG* from the cgroup *CGROUP* and attach type
tools/bpf/bpftool/Documentation/bpftool-prog.rst +1 -1
@@ -40,7 +40,7 @@ PROG COMMANDS | **lwt_seg6local** | **sockops** | **sk_skb** | **sk_msg** | **lirc_mode2** | | **cgroup/bind4** | **cgroup/bind6** | **cgroup/post_bind4** | **cgroup/post_bind6** | | **cgroup/connect4** | **cgroup/connect6** | **cgroup/sendmsg4** | **cgroup/sendmsg6** | -| **cgroup/sysctl** +| **cgroup/sysctl** | **cgroup/getsockopt** | **cgroup/setsockopt** | } | *ATTACH_TYPE* := { | **msg_verdict** | **stream_verdict** | **stream_parser** | **flow_dissector**
tools/bpf/bpftool/bash-completion/bpftool +5 -3
@@ -378,7 +378,8 @@ _bpftool() cgroup/connect4 cgroup/connect6 \ cgroup/sendmsg4 cgroup/sendmsg6 \ cgroup/post_bind4 cgroup/post_bind6 \ - cgroup/sysctl" -- \ + cgroup/sysctl cgroup/getsockopt \ + cgroup/setsockopt" -- \ "$cur" ) ) return 0 ;;
@@ -688,7 +689,8 @@ _bpftool() attach|detach) local ATTACH_TYPES='ingress egress sock_create sock_ops \ device bind4 bind6 post_bind4 post_bind6 connect4 \ - connect6 sendmsg4 sendmsg6 sysctl' + connect6 sendmsg4 sendmsg6 sysctl getsockopt \ + setsockopt' local ATTACH_FLAGS='multi override' local PROG_TYPE='id pinned tag' case $prev in
@@ -698,7 +700,7 @@ _bpftool() ;; ingress|egress|sock_create|sock_ops|device|bind4|bind6|\ post_bind4|post_bind6|connect4|connect6|sendmsg4|\ - sendmsg6|sysctl) + sendmsg6|sysctl|getsockopt|setsockopt) COMPREPLY=( $( compgen -W "$PROG_TYPE" -- \ "$cur" ) ) return 0
tools/bpf/bpftool/cgroup.c +4 -1
@@ -25,7 +25,8 @@" ATTACH_TYPE := { ingress | egress | sock_create |\n" \" sock_ops | device | bind4 | bind6 |\n" \" post_bind4 | post_bind6 | connect4 |\n" \ -" connect6 | sendmsg4 | sendmsg6 | sysctl }" +" connect6 | sendmsg4 | sendmsg6 | sysctl |\n" \ +" getsockopt | setsockopt }" static const char * const attach_type_strings[] = { [BPF_CGROUP_INET_INGRESS] = "ingress",
@@ -42,6 +43,8 @@ static const char * const attach_type_strings[] = { [BPF_CGROUP_UDP4_SENDMSG] = "sendmsg4", [BPF_CGROUP_UDP6_SENDMSG] = "sendmsg6", [BPF_CGROUP_SYSCTL] = "sysctl", + [BPF_CGROUP_GETSOCKOPT] = "getsockopt", + [BPF_CGROUP_SETSOCKOPT] = "setsockopt", [__MAX_BPF_ATTACH_TYPE] = NULL, };
tools/bpf/bpftool/main.h +1 -0
@@ -74,6 +74,7 @@ static const char * const prog_type_name[] = { [BPF_PROG_TYPE_SK_REUSEPORT] = "sk_reuseport", [BPF_PROG_TYPE_FLOW_DISSECTOR] = "flow_dissector", [BPF_PROG_TYPE_CGROUP_SYSCTL] = "cgroup_sysctl", + [BPF_PROG_TYPE_CGROUP_SOCKOPT] = "cgroup_sockopt", };extern const char *const map_type_name[];
tools/bpf/bpftool/prog.c +2 -1
@@ -1070,7 +1070,8 @@ static int do_help(int argc, char **argv)" sk_reuseport | flow_dissector | cgroup/sysctl |\n" " cgroup/bind4 | cgroup/bind6 | cgroup/post_bind4 |\n" " cgroup/post_bind6 | cgroup/connect4 | cgroup/connect6 |\n" -" cgroup/sendmsg4 | cgroup/sendmsg6 }\n" +" cgroup/sendmsg4 | cgroup/sendmsg6 | cgroup/getsockopt |\n" +" cgroup/setsockopt }\n" " ATTACH_TYPE := { msg_verdict | stream_verdict | stream_parser |\n" " flow_dissector }\n" " " HELP_SPEC_OPTIONS "\n"
Signature · 1 line
2.22.0.rc1.311.g5d7573a151-goog
Re: [PATCH bpf-next v3 1/8] bpf: implement getsockopt and setsockopt hooks
In reply to Stanislav FomichevQuoted 46 lines · Stanislav Fomichev
Implement new
BPF_PROG_TYPE_CGROUP_SOCKOPTprogram type and BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
BPF_CGROUP_SETSOCKOPTget a read-only view of the setsockopt arguments.BPF_CGROUP_GETSOCKOPTcan modify the supplied buffer.
Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.The buffer memory is pre-allocated (because I don't think there is a precedent for working with
__usermemory from bpf). This might be slow to do for each {s,g}etsockopt call, that's why I've added__cgroup_bpf_prog_array_is_emptythat exits early if there is nothing attached to a cgroup. Note, however, that there is a race between__cgroup_bpf_prog_array_is_emptyandBPF_PROG_RUN_ARRAYwhere cgroup program layout might have changed; this should not be a problem because in general there is a race between multiple calls to {s,g}etsocktop and user adding/removing bpf progs from a cgroup.The return code of the BPF program is handled as follows:
* 0: EPERM
* 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
* 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF prog exitsv3:
* typos inBPF_PROG_CGROUP_SOCKOPT_RUN_ARRAYcomments (Andrii Nakryiko)
* reverse christmas tree inBPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(Andrii Nakryiko)
* use__bpf_md_ptrinstead of__u32for optval{,_end} (Martin Lau)
* useBPF_FIELD_SIZEOF()for consistency (Martin Lau)
* newCG_SOCKOPT_ACCESSmacro to wrap repeated partsv2:
* movedbpf_sockopt_kernfields around to remove a hole (Martin Lau)
* alignedbpf_sockopt_kern->bufto 8 bytes (Martin Lau)
*bpf_prog_array_is_emptyinstead ofbpf_prog_array_length(Martin Lau)
* added [0,2] return code check to verifier (Martin Lau)
* dropped unused buf[64] from the stack (Martin Lau)
* usePTR_TO_SOCKETforbpf_sockopt->sk(Martin Lau)
* droppedbpf_target_offfrom ctx rewrites (Martin Lau)
* use return code for kernel bypass (Martin Lau & Andrii Nakryiko)kernel/bpf/cgroup.c +0
-0
[ ... ]
kernel/bpf/cgroup.c const struct bpf_verifier_ops cg_sysctl_verifier_ops = { · line 1330
Quoted 6 lines
kernel/bpf/cgroup.c +6
-0+static const struct bpf_func_proto * +cg_sockopt_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog) +{ +switch (func_id) { +case BPF_FUNC_sk_fullsock: +return &bpf_sk_fullsock_proto;
May be my v2 comment has been missed.
sk here (i.e. PTR_TO_SOCKET) must be a fullsock. bpf_sk_fullsock() will be a no-op. Hence, there is no need to expose bpf_sk_fullsock_proto.
Quoted 31 lines
kernel/bpf/cgroup.c +31
-0+case BPF_FUNC_sk_storage_get: +return &bpf_sk_storage_get_proto; +case BPF_FUNC_sk_storage_delete: +return &bpf_sk_storage_delete_proto; +#ifdef CONFIG_INET +case BPF_FUNC_tcp_sock: +return &bpf_tcp_sock_proto; +#endif +default : +return cgroup_base_func_proto (func_id, prog); + } +} + +static bool cg_sockopt_is_valid_access (int off,int size, +enum bpf_access_type type, +const struct bpf_prog *prog, +struct bpf_insn_access_aux *info) +{ +const int size_default =sizeof (__u32); + +if (off <0 || off >=sizeof (struct bpf_sockopt )) +return false; + +if (off % size !=0 ) +return false; + +if (type ==BPF_WRITE ) { +switch (off) { +case offsetof(struct bpf_sockopt , optlen): +if (size != size_default) +return false;
kernel/bpf/cgroup.c const struct bpf_verifier_ops cg_sysctl_verifier_ops = { · line 1371
Quoted 10 lines
kernel/bpf/cgroup.c +10
-0+return prog->expected_attach_type == +BPF_CGROUP_GETSOCKOPT ; +default : +return false; + } + } + +switch (off) { +case offsetof(struct bpf_sockopt , sk): +if (size !=sizeof (struct bpf_sock *))
Based on my understanding in commit b7df9ada9a77 ("bpf: fix pointer offsets in context for 32 bit"), I think it should be 'size != sizeof(__u64)'
Same for the optval and optval_end below.
kernel/bpf/cgroup.c const struct bpf_verifier_ops cg_sysctl_verifier_ops = { · line 1375
Quoted 4 lines
kernel/bpf/cgroup.c +4
-0+return false; + info->reg_type =PTR_TO_SOCKET ; +break ; + case bpf_ctx_range(struct bpf_sockopt, optval):
offsetof(struct bpf_sockopt, optval)
kernel/bpf/cgroup.c const struct bpf_verifier_ops cg_sysctl_verifier_ops = { · line 1380
Quoted 5 lines
kernel/bpf/cgroup.c +5
-0+if (size !=sizeof (void *)) +return false; + info->reg_type =PTR_TO_PACKET ; +break ; + case bpf_ctx_range(struct bpf_sockopt, optval_end):
offsetof(struct bpf_sockopt, optval_end)
Quoted 12 lines
kernel/bpf/cgroup.c +12
-0+if (size !=sizeof (void *)) +return false; + info->reg_type =PTR_TO_PACKET_END ; +break ; +default : +if (size != size_default) +return false; +break ; + } +return true; +} +
[ ... ]
net/core/filter.c BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk) · line 1838
Quoted 10 lines
net/core/filter.c +1
-1@@ -1835,7 +1835,7 @@ BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk)return sk_fullsock (sk) ? (unsigned long )sk : (unsigned long )NULL ; } -static const struct bpf_func_proto bpf_sk_fullsock_proto = { +const struct bpf_func_proto bpf_sk_fullsock_proto = {
As mentioned above, this change is also not needed.
Others LGTM.
Quoted 7 lines
net/core/filter.c +0
-0.func = bpf_sk_fullsock, .gpl_only = false, .ret_type = RET_PTR_TO_SOCKET_OR_NULL,@@ -5636,7 +5636,7 @@ BPF_CALL_1(bpf_tcp_sock, struct sock *, sk)return (unsigned long )NULL ; }
Stanislav Fomichev
reviewerRe: [PATCH bpf-next v3 1/8] bpf: implement getsockopt and setsockopt hooks
Quoted 58 lines · Martin Lau
Quoted 46 lines · Stanislav Fomichev
Implement new
BPF_PROG_TYPE_CGROUP_SOCKOPTprogram type and BPF_CGROUP_{G,S}ETSOCKOPT cgroup hooks.
BPF_CGROUP_SETSOCKOPTget a read-only view of the setsockopt arguments.BPF_CGROUP_GETSOCKOPTcan modify the supplied buffer.
Both of them reuse existing PTR_TO_PACKET{,_END} infrastructure.The buffer memory is pre-allocated (because I don't think there is a precedent for working with
__usermemory from bpf). This might be slow to do for each {s,g}etsockopt call, that's why I've added__cgroup_bpf_prog_array_is_emptythat exits early if there is nothing attached to a cgroup. Note, however, that there is a race between__cgroup_bpf_prog_array_is_emptyandBPF_PROG_RUN_ARRAYwhere cgroup program layout might have changed; this should not be a problem because in general there is a race between multiple calls to {s,g}etsocktop and user adding/removing bpf progs from a cgroup.The return code of the BPF program is handled as follows:
* 0: EPERM
* 1: success, execute kernel {s,g}etsockopt path after BPF prog exits
* 2: success, do _not_ execute kernel {s,g}etsockopt path after BPF prog exitsv3:
* typos inBPF_PROG_CGROUP_SOCKOPT_RUN_ARRAYcomments (Andrii Nakryiko)
* reverse christmas tree inBPF_PROG_CGROUP_SOCKOPT_RUN_ARRAY(Andrii Nakryiko)
* use__bpf_md_ptrinstead of__u32for optval{,_end} (Martin Lau)
* useBPF_FIELD_SIZEOF()for consistency (Martin Lau)
* newCG_SOCKOPT_ACCESSmacro to wrap repeated partsv2:
* movedbpf_sockopt_kernfields around to remove a hole (Martin Lau)
* alignedbpf_sockopt_kern->bufto 8 bytes (Martin Lau)
*bpf_prog_array_is_emptyinstead ofbpf_prog_array_length(Martin Lau)
* added [0,2] return code check to verifier (Martin Lau)
* dropped unused buf[64] from the stack (Martin Lau)
* usePTR_TO_SOCKETforbpf_sockopt->sk(Martin Lau)
* droppedbpf_target_offfrom ctx rewrites (Martin Lau)
* use return code for kernel bypass (Martin Lau & Andrii Nakryiko)kernel/bpf/cgroup.c +0
-0[ ... ]
kernel/bpf/cgroup.c const struct bpf_verifier_ops cg_sysctl_verifier_ops = { · line 1330
Quoted 6 lines
kernel/bpf/cgroup.c +6
-0+static const struct bpf_func_proto * +cg_sockopt_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog) +{ +switch (func_id) { +case BPF_FUNC_sk_fullsock: +return &bpf_sk_fullsock_proto;May be my v2 comment has been missed.
Quoted 3 lines
sk here (i.e.
PTR_TO_SOCKET) must be a fullsock.bpf_sk_fullsock()will be a no-op. Hence, there is no need to exposebpf_sk_fullsock_proto.
I think I missed that fact that PTR_TO_SOCKET implies fullsock. Will remove, thanks!
Quoted 44 lines
Quoted 31 lines
kernel/bpf/cgroup.c +31
-0+case BPF_FUNC_sk_storage_get: +return &bpf_sk_storage_get_proto; +case BPF_FUNC_sk_storage_delete: +return &bpf_sk_storage_delete_proto; +#ifdef CONFIG_INET +case BPF_FUNC_tcp_sock: +return &bpf_tcp_sock_proto; +#endif +default : +return cgroup_base_func_proto (func_id, prog); + } +} + +static bool cg_sockopt_is_valid_access (int off,int size, +enum bpf_access_type type, +const struct bpf_prog *prog, +struct bpf_insn_access_aux *info) +{ +const int size_default =sizeof (__u32); + +if (off <0 || off >=sizeof (struct bpf_sockopt )) +return false; + +if (off % size !=0 ) +return false; + +if (type ==BPF_WRITE ) { +switch (off) { +case offsetof(struct bpf_sockopt , optlen): +if (size != size_default) +return false;kernel/bpf/cgroup.c const struct bpf_verifier_ops cg_sysctl_verifier_ops = { · line 1371
Quoted 10 lines
kernel/bpf/cgroup.c +10
-0+return prog->expected_attach_type == +BPF_CGROUP_GETSOCKOPT ; +default : +return false; + } + } + +switch (off) { +case offsetof(struct bpf_sockopt , sk): +if (size !=sizeof (struct bpf_sock *))Based on my understanding in commit
b7df9ada9a77("bpf: fix pointer offsets in context for 32 bit"), I think it should be 'size !=sizeof(__u64)'
Quoted 1 line
Same for the optval and
optval_endbelow.
Good point. I was actually wondering when converting BPF_W to BPF_DW in the tests whether that would work correctly on 32 bits. Thanks for commit pointer, that should, indeed, always be all sizeof(__u64).
Quoted 5 lines
kernel/bpf/cgroup.c const struct bpf_verifier_ops cg_sysctl_verifier_ops = { · line 1375
Quoted 4 lines
kernel/bpf/cgroup.c +4
-0+return false; + info->reg_type =PTR_TO_SOCKET ; +break ; + case bpf_ctx_range(struct bpf_sockopt, optval):
offsetof(struct bpf_sockopt, optval)
Ack. No narrow loads for the pointers.
Quoted 30 lines
kernel/bpf/cgroup.c const struct bpf_verifier_ops cg_sysctl_verifier_ops = { · line 1380
Quoted 5 lines
kernel/bpf/cgroup.c +5
-0+if (size !=sizeof (void *)) +return false; + info->reg_type =PTR_TO_PACKET ; +break ; + case bpf_ctx_range(struct bpf_sockopt, optval_end):
offsetof(struct bpf_sockopt, optval_end)Quoted 12 lines
kernel/bpf/cgroup.c +12
-0+if (size !=sizeof (void *)) +return false; + info->reg_type =PTR_TO_PACKET_END ; +break ; +default : +if (size != size_default) +return false; +break ; + } +return true; +} +[ ... ]
Quoted 8 lines
net/core/filter.c +0
-0@@ -1835,7 +1835,7 @@ BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk)return sk_fullsock (sk) ? (unsigned long )sk : (unsigned long )NULL ; }
Quoted 5 lines
net/core/filter.c BPF_CALL_1(bpf_sk_fullsock, struct sock *, sk) · line 1838
Quoted 2 lines
net/core/filter.c +1
-1-static const struct bpf_func_proto bpf_sk_fullsock_proto = { +const struct bpf_func_proto bpf_sk_fullsock_proto = {As mentioned above, this change is also not needed.
Others LGTM.
Agreed, will not export.
Quoted 7 lines
Quoted 7 lines
net/core/filter.c +0
-0.func = bpf_sk_fullsock, .gpl_only = false, .ret_type = RET_PTR_TO_SOCKET_OR_NULL,@@ -5636,7 +5636,7 @@ BPF_CALL_1(bpf_tcp_sock, struct sock *, sk)return (unsigned long )NULL ; }