Introduce unstable CT lookup helpers
Subsystems (10)
Kumar Kartikeya Dwivedi
maintainerv8 cover Introduce unstable CT lookup helpers
This series adds unstable conntrack lookup helpers using BPF kfunc support. The patch adding the lookup helper is based off of Maxim's recent patch to aid in rebasing their series on top of this, all adjusted to work with module kfuncs [0].
[0]: https://lore.kernel.org/bpf/[email protected]
To enable returning a reference to struct nf_conn, the verifier is extended to support reference tracking for PTR_TO_BTF_ID, and kfunc is extended with support for working as acquire/release functions, similar to existing BPF helpers. kfunc returning pointer (limited to PTR_TO_BTF_ID in the kernel) can also return a PTR_TO_BTF_ID_OR_NULL now, typically needed when acquiring a resource can fail. kfunc can also receive PTR_TO_CTX and PTR_TO_MEM (with some limitations) as arguments now. There is also support for passing a mem, len pair as argument to kfunc now. In such cases, passing pointer to unsized type (void) is also permitted.
Please see individual commits for details.
Changelog:
----------
v7 -> v8:
v7: https://lore.kernel.org/bpf/[email protected]
* Move enum btf_kfunc_hook to btf.c (Alexei)
* Drop verbose log for unlikely failure case in __find_kfunc_desc_btf (Alexei)
* Remove unnecessary barrier in register_btf_kfunc_id_set (Alexei)
* Switch macro in bpf_nf test to __always_inline function (Alexei)
v6 -> v7:
v6: https://lore.kernel.org/bpf/[email protected]
* Drop try_module_get_live patch, use flag in btf_module struct (Alexei)
* Add comments and expand commit message detailing why we have to concatenate and sort vmlinux kfunc BTF ID sets (Alexei)
* Use bpf_testmod for testing btf_try_get_module race (Alexei)
* Use bpf_prog_type for both btf_kfunc_id_set_contains and register_btf_kfunc_id_set calls (Alexei)
* In case of module set registration, directly assign set (Alexei)
* Add CONFIG_USERFAULTFD=y to selftest config
* Fix other nits
v5 -> v6:
v5: https://lore.kernel.org/bpf/[email protected]
* Fix for a bug in btf_try_get_module leading to use-after-free
* Drop *kallsyms_on_each_symbol loop, reinstate register_btf_kfunc_id_set (Alexei)
* btf_free_kfunc_set_tab now takes struct btf, and handles resetting tab to NULL
* Check return value btf_name_by_offset for param_name
* Instead of using tmp_set, use btf->kfunc_set_tab directly, and simplify cleanup
v4 -> v5:
v4: https://lore.kernel.org/bpf/[email protected]
* Move nf_conntrack helpers code to its own separate file (Toke, Pablo)
* Remove verifier callbacks, put btf_id_sets in struct btf (Alexei)
* Convert the in-kernel users away from the old API
* Change len__ prefix convention to __sz suffix (Alexei)
* Drop parent_ref_obj_id patch (Alexei)
v3 -> v4:
v3: https://lore.kernel.org/bpf/[email protected]
* Guard unstable CT helpers with CONFIG_DEBUG_INFO_BTF_MODULES
* Move addition of prog_test test kfuncs to selftest commit
* Move negative kfunc tests to test_verifier suite
* Limit struct nesting depth to 4, which should be enough for now
v2 -> v3:
v2: https://lore.kernel.org/bpf/[email protected]
* Fix build error for !CONFIG_BPF_SYSCALL (Patchwork)
RFC v1 -> v2:
v1: https://lore.kernel.org/bpf/[email protected]
* Limit PTR_TO_MEM support to pointer to scalar, or struct with scalars (Alexei)
* Use btf_id_set for checking acquire, release, ret type null (Alexei)
* Introduce opts struct for CT helpers, move int err parameter to it
* Add l4proto as parameter to CT helper's opts, remove separate tcp/udp helpers
* Add support for mem, len argument pair to kfunc
* Allow void * as pointer type for mem, len argument pair
* Extend selftests to cover new additions to kfuncs
* Copy ref_obj_id to PTR_TO_BTF_ID dst_reg on btf_struct_access, test it
* Fix other misc nits, bugs, and expand commit messages
Kumar Kartikeya Dwivedi (10):
bpf: Fix UAF due to race between btf_try_get_module and load_module
bpf: Populate kfunc BTF ID sets in struct btf
bpf: Remove check_kfunc_call callback and old kfunc BTF ID API
bpf: Introduce mem, size argument pair support for kfunc
bpf: Add reference tracking support to kfunc net/netfilter: Add unstable CT lookup helpers for XDP and TC-BPF selftests/bpf: Add test for unstable CT lookup API selftests/bpf: Add test_verifier support to fixup kfunc call insns selftests/bpf: Extend kfunc selftests
selftests/bpf: Add test for race in btf_try_get_module
include/linux/bpf.h | 8 - include/linux/bpf_verifier.h | 7 + include/linux/btf.h | 75 ++-- include/linux/btf_ids.h | 13 +- include/net/netfilter/nf_conntrack_bpf.h | 23 ++ kernel/bpf/btf.c | 368 ++++++++++++++++-- kernel/bpf/verifier.c | 196 ++++++---- net/bpf/test_run.c | 150 ++++++- net/core/filter.c | 1 - net/core/net_namespace.c | 1 + net/ipv4/bpf_tcp_ca.c | 22 +- net/ipv4/tcp_bbr.c | 18 +- net/ipv4/tcp_cubic.c | 17 +- net/ipv4/tcp_dctcp.c | 18 +- net/netfilter/Makefile | 5 + net/netfilter/nf_conntrack_bpf.c | 257 ++++++++++++ net/netfilter/nf_conntrack_core.c | 8 + .../selftests/bpf/bpf_testmod/bpf_testmod.c | 21 +- tools/testing/selftests/bpf/config | 5 + .../selftests/bpf/prog_tests/bpf_mod_race.c | 230 +++++++++++ .../testing/selftests/bpf/prog_tests/bpf_nf.c | 48 +++ .../selftests/bpf/prog_tests/kfunc_call.c | 6 + .../selftests/bpf/progs/bpf_mod_race.c | 100 +++++ .../selftests/bpf/progs/kfunc_call_race.c | 14 + .../selftests/bpf/progs/kfunc_call_test.c | 52 ++- tools/testing/selftests/bpf/progs/ksym_race.c | 13 + .../testing/selftests/bpf/progs/test_bpf_nf.c | 109 ++++++ tools/testing/selftests/bpf/test_verifier.c | 28 ++ tools/testing/selftests/bpf/verifier/calls.c | 75 ++++ 29 files changed, 1674 insertions(+), 214 deletions(-) create mode 100644 include/net/netfilter/nf_conntrack_bpf.h create mode 100644 net/netfilter/nf_conntrack_bpf.c create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_mod_race.c create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_nf.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_mod_race.c create mode 100644 tools/testing/selftests/bpf/progs/kfunc_call_race.c create mode 100644 tools/testing/selftests/bpf/progs/ksym_race.c create mode 100644 tools/testing/selftests/bpf/progs/test_bpf_nf.c
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerPATCH v8 1/10 bpf: Fix UAF due to race between btf_try_get_module and load_module
While working on code to populate kfunc BTF ID sets for module BTF from its initcall, I noticed that by the time the initcall is invoked, the module BTF can already be seen by userspace (and the BPF verifier). The existing btf_try_get_module calls try_module_get which only fails if mod->state == MODULE_STATE_GOING, i.e. it can increment module reference when module initcall is happening in parallel.
Currently, BTF parsing happens from MODULE_STATE_COMING notifier callback. At this point, the module initcalls have not been invoked. The notifier callback parses and prepares the module BTF, allocates an ID, which publishes it to userspace, and then adds it to the btf_modules list allowing the kernel to invoke btf_try_get_module for the BTF.
However, at this point, the module has not been fully initialized (i.e. its initcalls have not finished). The code in module.c can still fail and free the module, without caring for other users. However, nothing stops btf_try_get_module from succeeding between the state transition from MODULE_STATE_COMING to MODULE_STATE_LIVE.
This leads to a use-after-free issue when BPF program loads successfully in the state transition, load_module's do_init_module call fails and frees the module, and BPF program fd on close calls module_put for the freed module. Future patch has test case to verify we don't regress in this area in future.
There are multiple points after prepare_coming_module (in load_module) where failure can occur and module loading can return error. We illustrate and test for the race using the last point where it can practically occur (in module __init function).
An illustration of the race:
CPU 0 CPU 1
load_module
notifier_call(MODULE_STATE_COMING)
btf_parse_module
btf_alloc_id // Published to userspace
list_add(&btf_mod->list, btf_modules)
mod->init(...)
... ^
bpf_check |
check_pseudo_btf_id |
btf_try_get_module |
returns true | ...
... | module __init in progress
return prog_fd | ...
... V
if (ret < 0)
free_module(mod)
...close(prog_fd)
...
bpf_prog_free_deferred
module_put(used_btf.mod) // use-after-free
We fix this issue by setting a flag BTF_MODULE_F_LIVE, from the notifier callback when MODULE_STATE_LIVE state is reached for the module, so that we return NULL from btf_try_get_module for modules that are not fully formed. Since try_module_get already checks that module is not in MODULE_STATE_GOING state, and that is the only transition a live module can make before being removed from btf_modules list, this is enough to close the race and prevent the bug.
A later selftest patch crafts the race condition artifically to verify that it has been fixed, and that verifier fails to load program (with ENXIO).
Lastly, a couple of comments:
1. Even if this race didn't exist, it seems more appropriate to only access resources (ksyms and kfuncs) of a fully formed module which has been initialized completely.
2. This patch was born out of need for synchronization against module initcall for the next patch, so it is needed for correctness even without the aforementioned race condition. The BTF resources initialized by module initcall are set up once and then only looked up, so just waiting until the initcall has finished ensures correct behavior.
--- kernel/bpf/btf.c | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-)
kernel/bpf/btf.c +24 -2
@@ -6200,12 +6200,17 @@ bool btf_id_set_contains(const struct btf_id_set *set, u32 id)return bsearch (&id, set->ids, set->cnt,sizeof (u32), btf_id_cmp_func) !=NULL ; } +enum { +BTF_MODULE_F_LIVE = (1 <<0 ), +}; +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES struct btf_module {struct list_head list;struct module *module;struct btf *btf;struct bin_attribute *sysfs_attr; +int flags; };static LIST_HEAD (btf_modules);
@@ -6233,7 +6238,8 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,int err =0 ;if (mod->btf_data_size ==0 || - (op !=MODULE_STATE_COMING && op !=MODULE_STATE_GOING )) + (op !=MODULE_STATE_COMING && op !=MODULE_STATE_LIVE && + op !=MODULE_STATE_GOING )) goto out;switch (op) {
@@ -6291,6 +6297,17 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op, btf_mod->sysfs_attr = attr; } +break ; +case MODULE_STATE_LIVE : +mutex_lock (&btf_module_mutex); +list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) { +if (btf_mod->module != module) +continue ; + + btf_mod->flags |=BTF_MODULE_F_LIVE ; +break ; + } +mutex_unlock (&btf_module_mutex);break ;case MODULE_STATE_GOING :mutex_lock (&btf_module_mutex);
@@ -6338,7 +6355,12 @@ struct module *btf_try_get_module(const struct btf *btf)if (btf_mod->btf != btf)continue ; -if (try_module_get (btf_mod->module)) +/* We must only consider module whose __init routine has +* finished, hence we must check for BTF_MODULE_F_LIVE flag, +* which is set from the notifier callback for +* MODULE_STATE_LIVE. +*/ +if ((btf_mod->flags &BTF_MODULE_F_LIVE ) &&try_module_get (btf_mod->module)) res = btf_mod->module;break ;
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerPATCH v8 2/10 bpf: Populate kfunc BTF ID sets in struct btf
In reply to Kumar Kartikeya DwivediThis patch prepares the kernel to support putting all kinds of kfunc BTF ID sets in the struct btf itself. The various kernel subsystems will make register_btf_kfunc_id_set call in the initcalls (for built-in code and modules).
The 'hook' is one of the many program types, e.g. XDP and TC/SCHED_CLS, STRUCT_OPS, and 'types' are check (allowed or not), acquire, release, and ret_null (with PTR_TO_BTF_ID_OR_NULL return type).
A maximum of BTF_KFUNC_SET_MAX_CNT (32) kfunc BTF IDs are permitted in a set of certain hook and type for vmlinux sets, since they are allocated on demand, and otherwise set as NULL. Module sets can only be registered once per hook and type, hence they are directly assigned.
A new btf_kfunc_id_set_contains function is exposed for use in verifier, this new method is faster than the existing list searching method, and is also automatic. It also lets other code not care whether the set is unallocated or not.
Note that module code can only do single register_btf_kfunc_id_set call per hook. This is why sorting is only done for in-kernel vmlinux sets, because there might be multiple sets for the same hook and type that must be concatenated, hence sorting them is required to ensure bsearch in btf_id_set_contains continues to work correctly.
Next commit will update the kernel users to make use of this infrastructure.
Finally, add __maybe_unused annotation for BTF ID macros for the !CONFIG_DEBUG_INFO_BTF case, so that they don't produce warnings during build time.
The previous patch is also needed to provide synchronization against initialization for module BTF's kfunc_set_tab introduced here, as described below:
The kfunc_set_tab pointer in struct btf is write-once (if we consider the registration phase (comprised of multiple register_btf_kfunc_id_set calls) as a single operation). In this sense, once it has been fully prepared, it isn't modified, only used for lookup (from the verifier context).
For btf_vmlinux, it is initialized fully during the do_initcalls phase, which happens fairly early in the boot process, before any processes are present. This also eliminates the possibility of bpf_check being called at that point, thus relieving us of ensuring any synchronization between the registration and lookup function (btf_kfunc_id_set_contains).
However, the case for module BTF is a bit tricky. The BTF is parsed, prepared, and published from the MODULE_STATE_COMING notifier callback. After this, the module initcalls are invoked, where our registration function will be called to populate the kfunc_set_tab for module BTF.
At this point, BTF may be available to userspace while its corresponding module is still intializing. A BTF fd can then be passed to verifier using bpf syscall (e.g. for kfunc call insn).
Hence, there is a race window where verifier may concurrently try to lookup the kfunc_set_tab. To prevent this race, we must ensure the operations are serialized, or waiting for the __init functions to complete.
In the earlier registration API, this race was alleviated as verifier bpf_check_mod_kfunc_call didn't find the kfunc BTF ID until it was added by the registration function (called usually at the end of module __init function after all module resources have been initialized). If the verifier made the check_kfunc_call before kfunc BTF ID was added to the list, it would fail verification (saying call isn't allowed). The access to list was protected using a mutex.
Now, it would still fail verification, but for a different reason (returning ENXIO due to the failed btf_try_get_module call in add_kfunc_call), because if the __init call is in progress the module will be in the middle of MODULE_STATE_COMING -> MODULE_STATE_LIVE transition, and the BTF_MODULE_LIVE flag for btf_module instance will not be set, so the btf_try_get_module call will fail.
--- include/linux/btf.h | 39 +++++++ include/linux/btf_ids.h | 13 ++- kernel/bpf/btf.c | 244 +++++++++++++++++++++++++++++++++++++++- 3 files changed, 289 insertions(+), 7 deletions(-)
include/linux/btf.h +39 -0
@@ -12,11 +12,33 @@#define BTF_TYPE_EMIT (type) ((void)(type *)0)#define BTF_TYPE_EMIT_ENUM (enum_val) ((void)enum_val) +enum btf_kfunc_type { +BTF_KFUNC_TYPE_CHECK , +BTF_KFUNC_TYPE_ACQUIRE , +BTF_KFUNC_TYPE_RELEASE , +BTF_KFUNC_TYPE_RET_NULL , +BTF_KFUNC_TYPE_MAX , +}; +struct btf ;struct btf_member ;struct btf_type ;union bpf_attr ;struct btf_show ; +struct btf_id_set ; + +struct btf_kfunc_id_set { +struct module *owner; +union { +struct { +struct btf_id_set *check_set; +struct btf_id_set *acquire_set; +struct btf_id_set *release_set; +struct btf_id_set *ret_null_set; + }; +struct btf_id_set *sets[BTF_KFUNC_TYPE_MAX ]; + }; +};extern const struct file_operations btf_fops;
@@ -307,6 +329,11 @@ const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id);const char *btf_name_by_offset (const struct btf *btf,u32 offset);struct btf *btf_parse_vmlinux (void );struct btf *bpf_prog_get_target_btf (const struct bpf_prog *prog); +bool btf_kfunc_id_set_contains (const struct btf *btf, +enum bpf_prog_type prog_type, +enum btf_kfunc_type type,u32 kfunc_btf_id); +int register_btf_kfunc_id_set (enum bpf_prog_type prog_type, +const struct btf_kfunc_id_set *s);#else static inline const struct btf_type *btf_type_by_id (const struct btf *btf,u32 type_id)
@@ -318,6 +345,18 @@ static inline const char *btf_name_by_offset(const struct btf *btf, {return NULL ; } +static inline bool btf_kfunc_id_set_contains (const struct btf *btf, +enum bpf_prog_type prog_type, +enum btf_kfunc_type type, +u32 kfunc_btf_id) +{ +return false; +} +static inline int register_btf_kfunc_id_set (enum bpf_prog_type prog_type, +const struct btf_kfunc_id_set *s) +{ +return 0 ; +}#endif struct kfunc_btf_id_set {
include/linux/btf_ids.h +7 -6
@@ -11,6 +11,7 @@ struct btf_id_set {#ifdef CONFIG_DEBUG_INFO_BTF #include <linux/compiler.h> /* for __PASTE */ +#include <linux/compiler_attributes.h> /* for __maybe_unused */ /* * Following macros help to define lists of BTF IDs placed
@@ -146,14 +147,14 @@ extern struct btf_id_set name;#else -#define BTF_ID_LIST (name) static u32 name[5]; +#define BTF_ID_LIST (name) static u32 __maybe_unused name[5];#define BTF_ID (prefix, name)#define BTF_ID_UNUSED -#define BTF_ID_LIST_GLOBAL (name, n) u32 name[n]; -#define BTF_ID_LIST_SINGLE (name, prefix, typename) static u32 name[1]; -#define BTF_ID_LIST_GLOBAL_SINGLE (name, prefix, typename) u32 name[1]; -#define BTF_SET_START (name) static struct btf_id_set name = { 0 }; -#define BTF_SET_START_GLOBAL (name) static struct btf_id_set name = { 0 }; +#define BTF_ID_LIST_GLOBAL (name, n) u32 __maybe_unused name[n]; +#define BTF_ID_LIST_SINGLE (name, prefix, typename) static u32 __maybe_unused name[1]; +#define BTF_ID_LIST_GLOBAL_SINGLE (name, prefix, typename) u32 __maybe_unused name[1]; +#define BTF_SET_START (name) static struct btf_id_set __maybe_unused name = { 0 }; +#define BTF_SET_START_GLOBAL (name) static struct btf_id_set __maybe_unused name = { 0 };#define BTF_SET_END (name)#endif /* CONFIG_DEBUG_INFO_BTF */
kernel/bpf/btf.c +243 -1
276 lines not on this page · raw message
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerPATCH v8 3/10 bpf: Remove check_kfunc_call callback and old kfunc BTF ID API
In reply to Kumar Kartikeya DwivediCompletely remove the old code for check_kfunc_call to help it work with modules, and also the callback itself.
The previous commit adds infrastructure to register all sets and put them in vmlinux or module BTF, and concatenates all related sets organized by the hook and the type. Once populated, these sets remain immutable for the lifetime of the struct btf.
Also, since we don't need the 'owner' module anywhere when doing check_kfunc_call, drop the 'btf_modp' module parameter from find_kfunc_desc_btf.
--- include/linux/bpf.h | 8 ---- include/linux/btf.h | 44 ------------------ kernel/bpf/btf.c | 46 ------------------- kernel/bpf/verifier.c | 20 ++++---- net/bpf/test_run.c | 23 ++++++---- net/core/filter.c | 1 - net/ipv4/bpf_tcp_ca.c | 22 +++++---- net/ipv4/tcp_bbr.c | 18 ++++---- net/ipv4/tcp_cubic.c | 17 +++---- net/ipv4/tcp_dctcp.c | 18 ++++---- .../selftests/bpf/bpf_testmod/bpf_testmod.c | 17 +++---- 11 files changed, 73 insertions(+), 161 deletions(-)
include/linux/bpf.h +0 -8
@@ -573,7 +573,6 @@ struct bpf_verifier_ops { const struct btf_type *t, int off, int size, enum bpf_access_type atype, u32 *next_btf_id); -bool (*check_kfunc_call)(u32 kfunc_btf_id,struct module *owner); };struct bpf_prog_offload_ops {
@@ -1719,7 +1718,6 @@ int bpf_prog_test_run_raw_tp(struct bpf_prog *prog,int bpf_prog_test_run_sk_lookup (struct bpf_prog *prog,const union bpf_attr *kattr,union bpf_attr __user *uattr); -bool bpf_prog_test_check_kfunc_call (u32 kfunc_id,struct module *owner);bool btf_ctx_access (int off,int size,enum bpf_access_type type,const struct bpf_prog *prog,struct bpf_insn_access_aux *info);
@@ -1971,12 +1969,6 @@ static inline int bpf_prog_test_run_sk_lookup(struct bpf_prog *prog,return -ENOTSUPP ; } -static inline bool bpf_prog_test_check_kfunc_call (u32 kfunc_id, -struct module *owner) -{ -return false; -} -static inline void bpf_map_put (struct bpf_map *map) { }
include/linux/btf.h +0 -44
48 lines not on this page · raw message
kernel/bpf/btf.c +0 -46
52 lines not on this page · raw message
kernel/bpf/verifier.c +8 -12
@@ -1741,7 +1741,7 @@ find_kfunc_desc(const struct bpf_prog *prog, u32 func_id, u16 offset) }static struct btf *__find_kfunc_desc_btf (struct bpf_verifier_env *env, -s16 offset,struct module **btf_modp) +s16 offset) {struct bpf_kfunc_btf kf_btf = { .offset = offset };struct bpf_kfunc_btf_tab *tab;
@@ -1795,8 +1795,6 @@ static struct btf *__find_kfunc_desc_btf(struct bpf_verifier_env *env,sort (tab->descs, tab->nr_descs,sizeof (tab->descs[0 ]), kfunc_btf_cmp_by_off,NULL ); } -if (btf_modp) - *btf_modp = b->module;return b->btf; }
@@ -1813,8 +1811,7 @@ void bpf_free_kfunc_btf_tab(struct bpf_kfunc_btf_tab *tab) }static struct btf *find_kfunc_desc_btf (struct bpf_verifier_env *env, -u32 func_id,s16 offset, -struct module **btf_modp) +u32 func_id,s16 offset) {if (offset) {if (offset <0 ) {
@@ -1825,7 +1822,7 @@ static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env,return ERR_PTR (-EINVAL ); } -return __find_kfunc_desc_btf (env, offset, btf_modp); +return __find_kfunc_desc_btf (env, offset); }return btf_vmlinux ?:ERR_PTR (-ENOENT ); }
@@ -1888,7 +1885,7 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) prog_aux->kfunc_btf_tab = btf_tab; } - desc_btf =find_kfunc_desc_btf (env, func_id, offset,NULL ); + desc_btf =find_kfunc_desc_btf (env, func_id, offset);if (IS_ERR (desc_btf)) {verbose (env,"failed to find BTF for kernel function\n" );return PTR_ERR (desc_btf);
@@ -2349,7 +2346,7 @@ static const char *disasm_kfunc_name(void *data, const struct bpf_insn *insn)if (insn->src_reg !=BPF_PSEUDO_KFUNC_CALL )return NULL ; - desc_btf =find_kfunc_desc_btf (data, insn->imm, insn->off,NULL ); + desc_btf =find_kfunc_desc_btf (data, insn->imm, insn->off);if (IS_ERR (desc_btf))return "<error>" ;
@@ -6820,7 +6817,6 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn)struct bpf_reg_state *regs =cur_regs (env);const char *func_name, *ptr_type_name;u32 i, nargs, func_id, ptr_type_id; -struct module *btf_mod =NULL ;const struct btf_param *args;struct btf *desc_btf;int err;
@@ -6829,7 +6825,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn)if (!insn->imm)return 0 ; - desc_btf =find_kfunc_desc_btf (env, insn->imm, insn->off, &btf_mod); + desc_btf =find_kfunc_desc_btf (env, insn->imm, insn->off);if (IS_ERR (desc_btf))return PTR_ERR (desc_btf);
@@ -6838,8 +6834,8 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) func_name =btf_name_by_offset (desc_btf, func->name_off); func_proto =btf_type_by_id (desc_btf, func->type); -if (!env->ops->check_kfunc_call || - !env->ops->check_kfunc_call (func_id, btf_mod)) { +if (!btf_kfunc_id_set_contains (desc_btf,resolve_prog_type (env->prog), +BTF_KFUNC_TYPE_CHECK , func_id)) {verbose (env,"calling kernel function %s is not allowed\n" , func_name);return -EACCES ;
net/bpf/test_run.c +14 -9
@@ -5,6 +5,7 @@#include <linux/btf.h> #include <linux/btf_ids.h> #include <linux/slab.h> +#include <linux/init.h> #include <linux/vmalloc.h> #include <linux/etherdevice.h> #include <linux/filter.h>
@@ -236,18 +237,11 @@ __diag_pop();ALLOW_ERROR_INJECTION (bpf_modify_return_test,ERRNO ); -BTF_SET_START (test_sk_kfunc_ids) +BTF_SET_START (test_sk_check_kfunc_ids) BTF_ID(func, bpf_kfunc_call_test1) BTF_ID(func, bpf_kfunc_call_test2) BTF_ID(func, bpf_kfunc_call_test3) -BTF_SET_END (test_sk_kfunc_ids) - -bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, struct module *owner) -{ -if (btf_id_set_contains (&test_sk_kfunc_ids, kfunc_id)) -return true; -return bpf_check_mod_kfunc_call (&prog_test_kfunc_list, kfunc_id, owner); -} +BTF_SET_END(test_sk_check_kfunc_ids) static void *bpf_test_init(const union bpf_attr *kattr, u32 size,u32 headroom,u32 tailroom)
@@ -1067,3 +1061,14 @@ int bpf_prog_test_run_syscall(struct bpf_prog *prog,kfree (ctx);return err; } + +static const struct btf_kfunc_id_set bpf_prog_test_kfunc_set = { + .owner =THIS_MODULE , + .check_set = &test_sk_check_kfunc_ids, +}; + +static int __init bpf_prog_test_run_init (void ) +{ +return register_btf_kfunc_id_set (BPF_PROG_TYPE_SCHED_CLS , &bpf_prog_test_kfunc_set); +} +late_initcall (bpf_prog_test_run_init);
net/core/filter.c +0 -1
@@ -10062,7 +10062,6 @@ const struct bpf_verifier_ops tc_cls_act_verifier_ops = { .convert_ctx_access = tc_cls_act_convert_ctx_access, .gen_prologue = tc_cls_act_prologue, .gen_ld_abs = bpf_gen_ld_abs, - .check_kfunc_call = bpf_prog_test_check_kfunc_call, };const struct bpf_prog_ops tc_cls_act_prog_ops = {
net/ipv4/bpf_tcp_ca.c +13 -9
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2019 Facebook */ +#include <linux/init.h> #include <linux/types.h> #include <linux/bpf_verifier.h> #include <linux/bpf.h>
@@ -212,26 +213,23 @@ bpf_tcp_ca_get_func_proto(enum bpf_func_id func_id, } } -BTF_SET_START (bpf_tcp_ca_kfunc_ids) +BTF_SET_START (bpf_tcp_ca_check_kfunc_ids)BTF_ID (func ,tcp_reno_ssthresh )BTF_ID (func, tcp_reno_cong_avoid)BTF_ID (func, tcp_reno_undo_cwnd)BTF_ID (func, tcp_slow_start)BTF_ID (func, tcp_cong_avoid_ai) -BTF_SET_END (bpf_tcp_ca_kfunc_ids) +BTF_SET_END (bpf_tcp_ca_check_kfunc_ids) -static bool bpf_tcp_ca_check_kfunc_call(u32 kfunc_btf_id, struct module *owner) -{ -if (btf_id_set_contains (&bpf_tcp_ca_kfunc_ids, kfunc_btf_id)) -return true; -return bpf_check_mod_kfunc_call (&bpf_tcp_ca_kfunc_list, kfunc_btf_id, owner); -} +static const struct btf_kfunc_id_set bpf_tcp_ca_kfunc_set = { + .owner =THIS_MODULE , + .check_set = &bpf_tcp_ca_check_kfunc_ids, +};static const struct bpf_verifier_ops bpf_tcp_ca_verifier_ops = { .get_func_proto = bpf_tcp_ca_get_func_proto, .is_valid_access = bpf_tcp_ca_is_valid_access, .btf_struct_access = bpf_tcp_ca_btf_struct_access, - .check_kfunc_call = bpf_tcp_ca_check_kfunc_call, }; static int bpf_tcp_ca_init_member(const struct btf_type *t,
@@ -300,3 +298,9 @@ struct bpf_struct_ops bpf_tcp_congestion_ops = { .init = bpf_tcp_ca_init, .name = "tcp_congestion_ops", }; + +static int __init bpf_tcp_ca_kfunc_init (void ) +{ +return register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &bpf_tcp_ca_kfunc_set); +} +late_initcall (bpf_tcp_ca_kfunc_init);
net/ipv4/tcp_bbr.c +10 -8
@@ -1154,7 +1154,7 @@ static struct tcp_congestion_ops tcp_bbr_cong_ops __read_mostly = { .set_state = bbr_set_state, }; -BTF_SET_START(tcp_bbr_kfunc_ids) +BTF_SET_START(tcp_bbr_check_kfunc_ids)#ifdef CONFIG_X86 #ifdef CONFIG_DYNAMIC_FTRACE BTF_ID (func, bbr_init)
@@ -1167,25 +1167,27 @@ BTF_ID(func, bbr_min_tso_segs) BTF_ID(func, bbr_set_state)#endif #endif -BTF_SET_END (tcp_bbr_kfunc_ids) +BTF_SET_END (tcp_bbr_check_kfunc_ids) -static DEFINE_KFUNC_BTF_ID_SET(&tcp_bbr_kfunc_ids, tcp_bbr_kfunc_btf_set); +static const struct btf_kfunc_id_set tcp_bbr_kfunc_set = { + .owner =THIS_MODULE , + .check_set = &tcp_bbr_check_kfunc_ids, +};static int __init bbr_register (void ) {int ret;BUILD_BUG_ON (sizeof (struct bbr ) >ICSK_CA_PRIV_SIZE ); - ret =tcp_register_congestion_control (&tcp_bbr_cong_ops); -if (ret) + + ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &tcp_bbr_kfunc_set); +if (ret <0 )return ret; -register_kfunc_btf_id_set (&bpf_tcp_ca_kfunc_list, &tcp_bbr_kfunc_btf_set); -return 0 ; +return tcp_register_congestion_control (&tcp_bbr_cong_ops); } static void __exit bbr_unregister(void) { -unregister_kfunc_btf_id_set (&bpf_tcp_ca_kfunc_list, &tcp_bbr_kfunc_btf_set);tcp_unregister_congestion_control (&tcp_bbr_cong_ops); }
net/ipv4/tcp_cubic.c +9 -8
@@ -485,7 +485,7 @@ static struct tcp_congestion_ops cubictcp __read_mostly = { .name = "cubic", }; -BTF_SET_START(tcp_cubic_kfunc_ids) +BTF_SET_START(tcp_cubic_check_kfunc_ids)#ifdef CONFIG_X86 #ifdef CONFIG_DYNAMIC_FTRACE BTF_ID (func, cubictcp_init)
@@ -496,9 +496,12 @@ BTF_ID(func, cubictcp_cwnd_event) BTF_ID(func, cubictcp_acked)#endif #endif -BTF_SET_END (tcp_cubic_kfunc_ids) +BTF_SET_END (tcp_cubic_check_kfunc_ids) -static DEFINE_KFUNC_BTF_ID_SET(&tcp_cubic_kfunc_ids, tcp_cubic_kfunc_btf_set); +static const struct btf_kfunc_id_set tcp_cubic_kfunc_set = { + .owner =THIS_MODULE , + .check_set = &tcp_cubic_check_kfunc_ids, +};static int __init cubictcp_register (void ) {
@@ -534,16 +537,14 @@ static int __init cubictcp_register(void)/* divide by bic_scale and by constant Srtt (100ms) */ do_div (cube_factor, bic_scale *10 ); - ret =tcp_register_congestion_control (&cubictcp); -if (ret) + ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &tcp_cubic_kfunc_set); +if (ret <0 )return ret; -register_kfunc_btf_id_set (&bpf_tcp_ca_kfunc_list, &tcp_cubic_kfunc_btf_set); -return 0 ; +return tcp_register_congestion_control (&cubictcp); } static void __exit cubictcp_unregister(void) { -unregister_kfunc_btf_id_set (&bpf_tcp_ca_kfunc_list, &tcp_cubic_kfunc_btf_set);tcp_unregister_congestion_control (&cubictcp); }
net/ipv4/tcp_dctcp.c +10 -8
@@ -238,7 +238,7 @@ static struct tcp_congestion_ops dctcp_reno __read_mostly = { .name = "dctcp-reno", }; -BTF_SET_START(tcp_dctcp_kfunc_ids) +BTF_SET_START(tcp_dctcp_check_kfunc_ids)#ifdef CONFIG_X86 #ifdef CONFIG_DYNAMIC_FTRACE BTF_ID (func, dctcp_init)
@@ -249,25 +249,27 @@ BTF_ID(func, dctcp_cwnd_undo) BTF_ID(func, dctcp_state)#endif #endif -BTF_SET_END (tcp_dctcp_kfunc_ids) +BTF_SET_END (tcp_dctcp_check_kfunc_ids) -static DEFINE_KFUNC_BTF_ID_SET(&tcp_dctcp_kfunc_ids, tcp_dctcp_kfunc_btf_set); +static const struct btf_kfunc_id_set tcp_dctcp_kfunc_set = { + .owner =THIS_MODULE , + .check_set = &tcp_dctcp_check_kfunc_ids, +};static int __init dctcp_register (void ) {int ret;BUILD_BUG_ON (sizeof (struct dctcp ) >ICSK_CA_PRIV_SIZE ); - ret =tcp_register_congestion_control (&dctcp); -if (ret) + + ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &tcp_dctcp_kfunc_set); +if (ret <0 )return ret; -register_kfunc_btf_id_set (&bpf_tcp_ca_kfunc_list, &tcp_dctcp_kfunc_btf_set); -return 0 ; +return tcp_register_congestion_control (&dctcp); } static void __exit dctcp_unregister(void) { -unregister_kfunc_btf_id_set (&bpf_tcp_ca_kfunc_list, &tcp_dctcp_kfunc_btf_set);tcp_unregister_congestion_control (&dctcp); }
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +9 -8
@@ -109,26 +109,27 @@ static struct bin_attribute bin_attr_bpf_testmod_file __ro_after_init = { .write = bpf_testmod_test_write, }; -BTF_SET_START(bpf_testmod_kfunc_ids) +BTF_SET_START (bpf_testmod_check_kfunc_ids)BTF_ID (func ,bpf_testmod_test_mod_kfunc ) -BTF_SET_END(bpf_testmod_kfunc_ids) +BTF_SET_END (bpf_testmod_check_kfunc_ids) -static DEFINE_KFUNC_BTF_ID_SET(&bpf_testmod_kfunc_ids, bpf_testmod_kfunc_btf_set); +static const struct btf_kfunc_id_set bpf_testmod_kfunc_set = { + .owner =THIS_MODULE , + .check_set = &bpf_testmod_check_kfunc_ids, +};static int bpf_testmod_init (void ) {int ret; - ret =sysfs_create_bin_file (kernel_kobj, &bin_attr_bpf_testmod_file); -if (ret) + ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_SCHED_CLS , &bpf_testmod_kfunc_set); +if (ret <0 )return ret; -register_kfunc_btf_id_set (&prog_test_kfunc_list, &bpf_testmod_kfunc_btf_set); -return 0 ; +return sysfs_create_bin_file (kernel_kobj, &bin_attr_bpf_testmod_file); }static void bpf_testmod_exit (void ) { -unregister_kfunc_btf_id_set (&prog_test_kfunc_list, &bpf_testmod_kfunc_btf_set);return sysfs_remove_bin_file (kernel_kobj, &bin_attr_bpf_testmod_file); }
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerPATCH v8 4/10 bpf: Introduce mem, size argument pair support for kfunc
In reply to Kumar Kartikeya DwivediBPF helpers can associate two adjacent arguments together to pass memory of certain size, using ARG_PTR_TO_MEM and ARG_CONST_SIZE arguments. Since we don't use bpf_func_proto for kfunc, we need to leverage BTF to implement similar support.
The ARG_CONST_SIZE processing for helpers is refactored into a common check_mem_size_reg helper that is shared with kfunc as well. kfunc ptr_to_mem support follows logic similar to global functions, where verification is done as if pointer is not null, even when it may be null.
This leads to a simple to follow rule for writing kfunc: always check the argument pointer for NULL, except when it is PTR_TO_CTX. Also, the PTR_TO_CTX case is also only safe when the helper expecting pointer to program ctx is not exposed to other programs where same struct is not ctx type. In that case, the type check will fall through to other cases and would permit passing other types of pointers, possibly NULL at runtime.
Currently, we require the size argument to be suffixed with "__sz" in the parameter name. This information is then recorded in kernel BTF and verified during function argument checking. In the future we can use BTF tagging instead, and modify the kernel function definitions. This will be a purely kernel-side change.
This allows us to have some form of backwards compatibility for structures that are passed in to the kernel function with their size, and allow variable length structures to be passed in if they are accompanied by a size parameter.
--- include/linux/bpf_verifier.h | 2 + kernel/bpf/btf.c | 48 +++++++++++++- kernel/bpf/verifier.c | 124 ++++++++++++++++++++++------------- 3 files changed, 126 insertions(+), 48 deletions(-)
include/linux/bpf_verifier.h +2 -0
@@ -521,6 +521,8 @@ bpf_prog_offload_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt);int check_ctx_reg (struct bpf_verifier_env *env,const struct bpf_reg_state *reg,int regno); +int check_kfunc_mem_size_reg (struct bpf_verifier_env *env,struct bpf_reg_state *reg, +u32 regno);int check_mem_reg (struct bpf_verifier_env *env,struct bpf_reg_state *reg,u32 regno,u32 mem_size);
kernel/bpf/btf.c +45 -3
68 lines not on this page · raw message
kernel/bpf/verifier.c +79 -45
142 lines not on this page · raw message
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerPATCH v8 5/10 bpf: Add reference tracking support to kfunc
In reply to Kumar Kartikeya DwivediThis patch adds verifier support for PTR_TO_BTF_ID return type of kfunc to be a reference, by reusing acquire_reference_state/release_reference support for existing in-kernel bpf helpers.
We make use of the three kfunc types:
- BTF_KFUNC_TYPE_ACQUIRE
Return true if kfunc_btf_id is an acquire kfunc. This will acquire_reference_state for the returned PTR_TO_BTF_ID (this is the only allow return value). Note that acquire kfunc must always return a PTR_TO_BTF_ID{_OR_NULL}, otherwise the program is rejected.
- BTF_KFUNC_TYPE_RELEASE
Return true if kfunc_btf_id is a release kfunc. This will release the reference to the passed in PTR_TO_BTF_ID which has a reference state (from earlier acquire kfunc).
The btf_check_func_arg_match returns the regno (of argument register, hence > 0) if the kfunc is a release kfunc, and a proper referenced PTR_TO_BTF_ID is being passed to it.
This is similar to how helper call check uses bpf_call_arg_meta to store the ref_obj_id that is later used to release the reference. Similar to in-kernel helper, we only allow passing one referenced PTR_TO_BTF_ID as an argument. It can also be passed in to normal kfunc, but in case of release kfunc there must always be one PTR_TO_BTF_ID argument that is referenced.
- BTF_KFUNC_TYPE_RET_NULL
For kfunc returning PTR_TO_BTF_ID, tells if it can be NULL, hence force caller to mark the pointer not null (using check) before accessing it. Note that taking into account the case fixed by commit 93c230e3f5bd ("bpf: Enforce id generation for all may-be-null register type") we assign a non-zero id for mark_ptr_or_null_reg logic. Later, if more return types are supported by kfunc, which have a _OR_NULL variant, it might be better to move this id generation under a common reg_type_may_be_null check, similar to the case in the commit.
Referenced PTR_TO_BTF_ID is currently only limited to kfunc, but can be extended in the future to other BPF helpers as well. For now, we can rely on the btf_struct_ids_match check to ensure we get the pointer to the expected struct type. In the future, care needs to be taken to avoid ambiguity for reference PTR_TO_BTF_ID passed to release function, in case multiple candidates can release same BTF ID.
e.g. there might be two release kfuncs (or kfunc and helper):
foo(struct abc *p);bar(struct abc *p);
... such that both release a PTR_TO_BTF_ID with btf_id of struct abc. In this case we would need to track the acquire function corresponding to the release function to avoid type confusion, and store this information in the register state so that an incorrect program can be rejected. This is not a problem right now, hence it is left as an exercise for the future patch introducing such a case in the kernel.
--- include/linux/bpf_verifier.h | 5 ++++ kernel/bpf/btf.c | 32 ++++++++++++++++++++-- kernel/bpf/verifier.c | 52 +++++++++++++++++++++++++++++------- 3 files changed, 77 insertions(+), 12 deletions(-)
include/linux/bpf_verifier.h +5 -0
@@ -566,4 +566,9 @@ static inline u32 type_flag(u32 type) return type & ~BPF_BASE_TYPE_MASK; } +static inline enum bpf_prog_type resolve_prog_type (struct bpf_prog *prog) +{ +return prog->aux->dst_prog ? prog->aux->dst_prog->type : prog->type; +} +#endif /* _LINUX_BPF_VERIFIER_H */
kernel/bpf/btf.c +30 -2
@@ -5686,11 +5686,13 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, bool ptr_to_mem_ok) {struct bpf_verifier_log *log = &env->log; +u32 i, nargs, ref_id, ref_obj_id =0 ;bool is_kfunc =btf_is_kernel (btf);const char *func_name, *ref_tname;const struct btf_type *t, *ref_t;const struct btf_param *args; -u32 i, nargs, ref_id; +int ref_regno =0 ; +bool rel = false; t =btf_type_by_id (btf, func_id);if (!t || !btf_type_is_func (t)) {
@@ -5768,6 +5770,16 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,if (reg->type ==PTR_TO_BTF_ID ) { reg_btf = reg->btf; reg_ref_id = reg->btf_id; +/* Ensure only one argument is referenced PTR_TO_BTF_ID */ +if (reg->ref_obj_id) { +if (ref_obj_id) { +bpf_log (log,"verifier internal error: more than one arg with ref_obj_id R%d %u %u\n" , + regno, reg->ref_obj_id, ref_obj_id); +return -EFAULT ; + } + ref_regno = regno; + ref_obj_id = reg->ref_obj_id; + } }else { reg_btf = btf_vmlinux; reg_ref_id = *reg2btf_ids[reg->type];
@@ -5838,7 +5850,23 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, } } -return 0 ; +/* Either both are set, or neither */ +WARN_ON_ONCE ((ref_obj_id && !ref_regno) || (!ref_obj_id && ref_regno)); +if (is_kfunc) { + rel =btf_kfunc_id_set_contains (btf,resolve_prog_type (env->prog), +BTF_KFUNC_TYPE_RELEASE , func_id); +/* We already made sure ref_obj_id is set only for one argument */ +if (rel && !ref_obj_id) { +bpf_log (log,"release kernel function %s expects refcounted PTR_TO_BTF_ID\n" , + func_name); +return -EINVAL ; + } +/* Allow (!rel && ref_obj_id), so that passing such referenced PTR_TO_BTF_ID to +* other kfuncs works +*/ + } +/* returns argument register number > 0 in case of reference release kfunc */ +return rel ? ref_regno :0 ; }/* Compare BTF of a function with given bpf_reg_state.
kernel/bpf/verifier.c +42 -10
105 lines not on this page · raw message
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerPATCH v8 6/10 net/netfilter: Add unstable CT lookup helpers for XDP and TC-BPF
In reply to Kumar Kartikeya DwivediThis change adds conntrack lookup helpers using the unstable kfunc call interface for the XDP and TC-BPF hooks. The primary usecase is implementing a synproxy in XDP, see Maxim's patchset [0].
Export get_net_ns_by_id as nf_conntrack_bpf.c needs to call it.
This object is only built when CONFIG_DEBUG_INFO_BTF_MODULES is enabled.
[0]: https://lore.kernel.org/bpf/[email protected]
--- include/net/netfilter/nf_conntrack_bpf.h | 23 ++ net/core/net_namespace.c | 1 + net/netfilter/Makefile | 5 + net/netfilter/nf_conntrack_bpf.c | 257 +++++++++++++++++++++++ net/netfilter/nf_conntrack_core.c | 8 + 5 files changed, 294 insertions(+) create mode 100644 include/net/netfilter/nf_conntrack_bpf.h create mode 100644 net/netfilter/nf_conntrack_bpf.c
include/net/netfilter/nf_conntrack_bpf.h new file +23 -0
@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _NF_CONNTRACK_BPF_H +#define _NF_CONNTRACK_BPF_H + +#include <linux/btf.h> +#include <linux/kconfig.h> + +#if (IS_BUILTIN (CONFIG_NF_CONNTRACK ) &&IS_ENABLED (CONFIG_DEBUG_INFO_BTF )) || \ + (IS_MODULE (CONFIG_NF_CONNTRACK ) &&IS_ENABLED (CONFIG_DEBUG_INFO_BTF_MODULES )) + +extern int register_nf_conntrack_bpf (void ); + +#else + +static inline int register_nf_conntrack_bpf (void ) +{ +return 0 ; +} + +#endif + +#endif /* _NF_CONNTRACK_BPF_H */
net/core/net_namespace.c +1 -0
@@ -299,6 +299,7 @@ struct net *get_net_ns_by_id(const struct net *net, int id)return peer; } +EXPORT_SYMBOL_GPL (get_net_ns_by_id);/* * setup_net runs the initializers for the network namespace object.
net/netfilter/Makefile +5 -0
@@ -14,6 +14,11 @@ nf_conntrack-$(CONFIG_NF_CONNTRACK_LABELS) += nf_conntrack_labels.o nf_conntrack-$(CONFIG_NF_CT_PROTO_DCCP) += nf_conntrack_proto_dccp.o nf_conntrack-$(CONFIG_NF_CT_PROTO_SCTP) += nf_conntrack_proto_sctp.o nf_conntrack-$(CONFIG_NF_CT_PROTO_GRE) += nf_conntrack_proto_gre.o +ifeq ($(CONFIG_NF_CONNTRACK),m) +nf_conntrack-$(CONFIG_DEBUG_INFO_BTF_MODULES) += nf_conntrack_bpf.o +else ifeq ($(CONFIG_NF_CONNTRACK),y) +nf_conntrack-$(CONFIG_DEBUG_INFO_BTF) += nf_conntrack_bpf.o +endif obj-$(CONFIG_NETFILTER) = netfilter.o
net/netfilter/nf_conntrack_bpf.c new file +257 -0
257 lines not on this page · raw message
net/netfilter/nf_conntrack_core.c +8 -0
@@ -34,6 +34,7 @@#include <linux/rculist_nulls.h> #include <net/netfilter/nf_conntrack.h> +#include <net/netfilter/nf_conntrack_bpf.h> #include <net/netfilter/nf_conntrack_l4proto.h> #include <net/netfilter/nf_conntrack_expect.h> #include <net/netfilter/nf_conntrack_helper.h>
@@ -2748,8 +2749,15 @@ int nf_conntrack_init_start(void)conntrack_gc_work_init (&conntrack_gc_work);queue_delayed_work (system_power_efficient_wq, &conntrack_gc_work.dwork,HZ ); + ret =register_nf_conntrack_bpf (); +if (ret <0 ) + goto err_kfunc; +return 0 ; +err_kfunc: +cancel_delayed_work_sync (&conntrack_gc_work.dwork); +nf_conntrack_proto_fini (); err_proto:nf_conntrack_seqadj_fini (); err_seqadj:
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerPATCH v8 7/10 selftests/bpf: Add test for unstable CT lookup API
In reply to Kumar Kartikeya DwivediThis tests that we return errors as documented, and also that the kfunc calls work from both XDP and TC hooks.
--- tools/testing/selftests/bpf/config | 4 + .../testing/selftests/bpf/prog_tests/bpf_nf.c | 48 ++++++++ .../testing/selftests/bpf/progs/test_bpf_nf.c | 109 ++++++++++++++++++ 3 files changed, 161 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_nf.c create mode 100644 tools/testing/selftests/bpf/progs/test_bpf_nf.c
tools/testing/selftests/bpf/config +4 -0
@@ -48,3 +48,7 @@ CONFIG_IMA_READ_POLICY=y CONFIG_BLK_DEV_LOOP=y CONFIG_FUNCTION_TRACER=y CONFIG_DYNAMIC_FTRACE=y +CONFIG_NETFILTER=y +CONFIG_NF_DEFRAG_IPV4=y +CONFIG_NF_DEFRAG_IPV6=y +CONFIG_NF_CONNTRACK=y
tools/testing/selftests/bpf/prog_tests/bpf_nf.c new file +48 -0
48 lines not on this page · raw message
tools/testing/selftests/bpf/progs/test_bpf_nf.c new file +109 -0
109 lines not on this page · raw message
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerPATCH v8 8/10 selftests/bpf: Add test_verifier support to fixup kfunc call insns
In reply to Kumar Kartikeya DwivediThis allows us to add tests (esp. negative tests) where we only want to ensure the program doesn't pass through the verifier, and also verify the error. The next commit will add the tests making use of this.
--- tools/testing/selftests/bpf/test_verifier.c | 28 +++++++++++++++++++++ 1 file changed, 28 insertions(+)
tools/testing/selftests/bpf/test_verifier.c +28 -0
@@ -31,6 +31,7 @@#include <linux/if_ether.h> #include <linux/btf.h> +#include <bpf/btf.h> #include <bpf/bpf.h> #include <bpf/libbpf.h>
@@ -66,6 +67,11 @@ static bool unpriv_disabled = false;static int skips;static bool verbose = false; +struct kfunc_btf_id_pair { +const char *kfunc; +int insn_idx; +}; +struct bpf_test {const char *descr;struct bpf_insn insns[MAX_INSNS ];
@@ -92,6 +98,7 @@ struct bpf_test {int fixup_map_reuseport_array[MAX_FIXUPS ];int fixup_map_ringbuf[MAX_FIXUPS ];int fixup_map_timer[MAX_FIXUPS ]; +struct kfunc_btf_id_pair fixup_kfunc_btf_id[MAX_FIXUPS ];/* Expected verifier log output for result REJECT or VERBOSE_ACCEPT. * Can be a tab-separated sequence of expected strings. An empty string * means no log verification.
@@ -744,6 +751,7 @@ static void do_test_fixup(struct bpf_test *test, enum bpf_prog_type prog_type,int *fixup_map_reuseport_array = test->fixup_map_reuseport_array;int *fixup_map_ringbuf = test->fixup_map_ringbuf;int *fixup_map_timer = test->fixup_map_timer; +struct kfunc_btf_id_pair *fixup_kfunc_btf_id = test->fixup_kfunc_btf_id;if (test->fill_helper) { test->fill_insns =calloc (MAX_TEST_INSNS ,sizeof (struct bpf_insn ));
@@ -936,6 +944,26 @@ static void do_test_fixup(struct bpf_test *test, enum bpf_prog_type prog_type, fixup_map_timer++; }while (*fixup_map_timer); } + +/* Patch in kfunc BTF IDs */ +if (fixup_kfunc_btf_id->kfunc) { +struct btf *btf; +int btf_id; + +do { + btf_id =0 ; + btf =btf__load_vmlinux_btf (); +if (btf) { + btf_id =btf__find_by_name_kind (btf, + fixup_kfunc_btf_id->kfunc, +BTF_KIND_FUNC ); + btf_id = btf_id <0 ?0 : btf_id; + } +btf__free (btf); + prog[fixup_kfunc_btf_id->insn_idx].imm = btf_id; + fixup_kfunc_btf_id++; + }while (fixup_kfunc_btf_id->kfunc); + } }struct libcap {
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerPATCH v8 9/10 selftests/bpf: Extend kfunc selftests
In reply to Kumar Kartikeya DwivediUse the prog_test kfuncs to test the referenced PTR_TO_BTF_ID kfunc support, and PTR_TO_CTX, PTR_TO_MEM argument passing support. Also testing the various failure cases for invalid kfunc prototypes.
--- net/bpf/test_run.c | 129 +++++++++++++++++- .../selftests/bpf/prog_tests/kfunc_call.c | 6 + .../selftests/bpf/progs/kfunc_call_test.c | 52 ++++++- tools/testing/selftests/bpf/verifier/calls.c | 75 ++++++++++ 4 files changed, 258 insertions(+), 4 deletions(-)
net/bpf/test_run.c +127 -2
@@ -233,6 +233,105 @@ struct sock * noinline bpf_kfunc_call_test3(struct sock *sk)return sk; } +struct prog_test_ref_kfunc { +int a; +int b; +struct prog_test_ref_kfunc *next; +}; + +static struct prog_test_ref_kfunc prog_test_struct = { + .a =42 , + .b =108 , + .next = &prog_test_struct, +}; + +noinline struct prog_test_ref_kfunc * +bpf_kfunc_call_test_acquire (unsigned long *scalar_ptr) +{ +/* randomly return NULL */ +if (get_jiffies_64 () %2 ) +return NULL ; +return &prog_test_struct; +} + +noinline void bpf_kfunc_call_test_release(struct prog_test_ref_kfunc *p) +{ +} + +struct prog_test_pass1 { +int x0; +struct { +int x1; +struct { +int x2; +struct { +int x3; + }; + }; + }; +}; + +struct prog_test_pass2 { +int len; +short arr1[4 ]; +struct { +char arr2[4 ]; +unsigned long arr3[8 ]; + } x; +}; + +struct prog_test_fail1 { +void *p; +int x; +}; + +struct prog_test_fail2 { +int x8; +struct prog_test_pass1 x; +}; + +struct prog_test_fail3 { +int len; +char arr1[2 ]; +char arr2[0 ]; +}; + +noinline void bpf_kfunc_call_test_pass_ctx(struct __sk_buff *skb) +{ +} + +noinline void bpf_kfunc_call_test_pass1(struct prog_test_pass1 *p) +{ +} + +noinline void bpf_kfunc_call_test_pass2(struct prog_test_pass2 *p) +{ +} + +noinline void bpf_kfunc_call_test_fail1(struct prog_test_fail1 *p) +{ +} + +noinline void bpf_kfunc_call_test_fail2(struct prog_test_fail2 *p) +{ +} + +noinline void bpf_kfunc_call_test_fail3(struct prog_test_fail3 *p) +{ +} + +noinline void bpf_kfunc_call_test_mem_len_pass1(void *mem, int mem__sz) +{ +} + +noinline void bpf_kfunc_call_test_mem_len_fail1(void *mem, int len) +{ +} + +noinline void bpf_kfunc_call_test_mem_len_fail2(u64 *mem, int len) +{ +} +__diag_pop ();ALLOW_ERROR_INJECTION (bpf_modify_return_test,ERRNO );
@@ -241,8 +340,31 @@ BTF_SET_START(test_sk_check_kfunc_ids) BTF_ID(func, bpf_kfunc_call_test1) BTF_ID(func, bpf_kfunc_call_test2) BTF_ID(func, bpf_kfunc_call_test3) +BTF_ID(func, bpf_kfunc_call_test_acquire) +BTF_ID(func, bpf_kfunc_call_test_release) +BTF_ID(func, bpf_kfunc_call_test_pass_ctx) +BTF_ID(func, bpf_kfunc_call_test_pass1) +BTF_ID(func, bpf_kfunc_call_test_pass2) +BTF_ID(func, bpf_kfunc_call_test_fail1) +BTF_ID(func, bpf_kfunc_call_test_fail2) +BTF_ID(func, bpf_kfunc_call_test_fail3) +BTF_ID(func, bpf_kfunc_call_test_mem_len_pass1) +BTF_ID(func, bpf_kfunc_call_test_mem_len_fail1) +BTF_ID(func, bpf_kfunc_call_test_mem_len_fail2)BTF_SET_END (test_sk_check_kfunc_ids) +BTF_SET_START (test_sk_acquire_kfunc_ids ) +BTF_ID (func, bpf_kfunc_call_test_acquire) +BTF_SET_END (test_sk_acquire_kfunc_ids) + +BTF_SET_START (test_sk_release_kfunc_ids) +BTF_ID (func, bpf_kfunc_call_test_release) +BTF_SET_END (test_sk_release_kfunc_ids) + +BTF_SET_START (test_sk_ret_null_kfunc_ids) +BTF_ID (func, bpf_kfunc_call_test_acquire) +BTF_SET_END (test_sk_ret_null_kfunc_ids) + static void *bpf_test_init(const union bpf_attr *kattr, u32 size, u32 headroom, u32 tailroom) {
@@ -1063,8 +1185,11 @@ int bpf_prog_test_run_syscall(struct bpf_prog *prog, }static const struct btf_kfunc_id_set bpf_prog_test_kfunc_set = { - .owner =THIS_MODULE , - .check_set = &test_sk_check_kfunc_ids, + .owner =THIS_MODULE , + .check_set = &test_sk_check_kfunc_ids, + .acquire_set = &test_sk_acquire_kfunc_ids, + .release_set = &test_sk_release_kfunc_ids, + .ret_null_set = &test_sk_ret_null_kfunc_ids, };static int __init bpf_prog_test_run_init (void )
tools/testing/selftests/bpf/prog_tests/kfunc_call.c +6 -0
@@ -27,6 +27,12 @@ static void test_main(void)ASSERT_OK (err,"bpf_prog_test_run(test2)" );ASSERT_EQ (retval,3 ,"test2-retval" ); + prog_fd = skel->progs.kfunc_call_test_ref_btf_id.prog_fd; + err =bpf_prog_test_run (prog_fd,1 , &pkt_v4,sizeof (pkt_v4), +NULL ,NULL , (__u32 *)&retval,NULL ); +ASSERT_OK (err,"bpf_prog_test_run(test_ref_btf_id)" ); +ASSERT_EQ (retval,0 ,"test_ref_btf_id-retval" ); +kfunc_call_test_lskel__destroy (skel); }
tools/testing/selftests/bpf/progs/kfunc_call_test.c +50 -2
67 lines not on this page · raw message
tools/testing/selftests/bpf/verifier/calls.c +75 -0
81 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH bpf-next v8 09/10] selftests/bpf: Extend kfunc selftests
Quoted 4 lines · Kumar Kartikeya Dwivedi
Use the
prog_testkfuncs to test the referencedPTR_TO_BTF_IDkfunc support, andPTR_TO_CTX,PTR_TO_MEMargument passing support. Also testing the various failure cases for invalid kfunc prototypes.
Quoted 8 lines
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- net/bpf/test_run.c | 129 +++++++++++++++++- .../selftests/bpf/prog_tests/kfunc_call.c | 6 + .../selftests/bpf/progs/kfunc_call_test.c | 52 ++++++- tools/testing/selftests/bpf/verifier/calls.c | 75 ++++++++++ 4 files changed, 258 insertions(+), 4 deletions(-)
It looks like this patch broke building the bpf tests:
tools/testing/selftests/bpf$ make
CLNG-BPF [test_maps] kfunc_call_test.oprogs/kfunc_call_test.c:13:46: error: declaration of 'struct prog_test_pass1' will not be visible outside of this function [-Werror,-Wvisibility]
extern void bpf_kfunc_call_test_pass1(struct prog_test_pass1 *p) __ksym;
^The only definition of struct prog_test_pass1 that I see is in net/bpf/test_run.c. How is this supposed to work?
commit 87091063df5d ("selftests/bpf: Add test for unstable CT lookup API") from the same series added a similar problem in progs/test_bpf_nf.c:
progs/test_bpf_nf.c:31:21: error: variable has incomplete type 'struct bpf_ct_opts'
struct bpf_ct_opts opts_def = { .l4proto = IPPROTO_TCP, .netns_id = -1 };
^Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v8 09/10] selftests/bpf: Extend kfunc selftests
Quoted 26 lines · Benjamin Poirier
Quoted 4 lines · Kumar Kartikeya Dwivedi
Use the
prog_testkfuncs to test the referencedPTR_TO_BTF_IDkfunc support, andPTR_TO_CTX,PTR_TO_MEMargument passing support. Also testing the various failure cases for invalid kfunc prototypes.Quoted 8 lines
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- net/bpf/test_run.c | 129 +++++++++++++++++- .../selftests/bpf/prog_tests/kfunc_call.c | 6 + .../selftests/bpf/progs/kfunc_call_test.c | 52 ++++++- tools/testing/selftests/bpf/verifier/calls.c | 75 ++++++++++ 4 files changed, 258 insertions(+), 4 deletions(-)It looks like this patch broke building the bpf tests:
tools/testing/selftests/bpf$make
CLNG-BPF [test_maps] kfunc_call_test.oprogs/kfunc_call_test.c:13:46: error: declaration of 'structprog_test_pass1' will not be visible outside of this function [-Werror,-Wvisibility]extern void bpf_kfunc_call_test_pass1(struct prog_test_pass1 *p) __ksym; ^The only definition of struct
prog_test_pass1that I see is innet/bpf/test_run.c. How is this supposed to work?
Quoted 7 lines
commit
87091063df5d("selftests/bpf: Add test for unstable CT lookup API") from the same series added a similar problem inprogs/test_bpf_nf.c:
progs/test_bpf_nf.c:31:21: error: variable has incomplete type 'structbpf_ct_opts'struct bpf_ct_opts opts_def = { .l4proto = IPPROTO_TCP, .netns_id = -1 };
Both of them should have their definition in vmlinux.h. Can you check? Also for BPF selftests we require conntrack to be built into the kernel, instead of as a module.
--
Kartikeya
Re: [PATCH bpf-next v8 09/10] selftests/bpf: Extend kfunc selftests
Quoted 31 lines · Kumar Kartikeya Dwivedi
Quoted 30 lines · Benjamin Poirier
Quoted 4 lines · Kumar Kartikeya Dwivedi
Use the
prog_testkfuncs to test the referencedPTR_TO_BTF_IDkfunc support, andPTR_TO_CTX,PTR_TO_MEMargument passing support. Also testing the various failure cases for invalid kfunc prototypes.Quoted 8 lines
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- net/bpf/test_run.c | 129 +++++++++++++++++- .../selftests/bpf/prog_tests/kfunc_call.c | 6 + .../selftests/bpf/progs/kfunc_call_test.c | 52 ++++++- tools/testing/selftests/bpf/verifier/calls.c | 75 ++++++++++ 4 files changed, 258 insertions(+), 4 deletions(-)It looks like this patch broke building the bpf tests:
tools/testing/selftests/bpf$make
CLNG-BPF [test_maps] kfunc_call_test.oprogs/kfunc_call_test.c:13:46: error: declaration of 'structprog_test_pass1' will not be visible outside of this function [-Werror,-Wvisibility]extern void bpf_kfunc_call_test_pass1(struct prog_test_pass1 *p) __ksym; ^The only definition of struct
prog_test_pass1that I see is innet/bpf/test_run.c. How is this supposed to work?commit
87091063df5d("selftests/bpf: Add test for unstable CT lookup API") from the same series added a similar problem inprogs/test_bpf_nf.c:
Quoted 5 lines
Quoted 3 lines
progs/test_bpf_nf.c:31:21: error: variable has incomplete type 'structbpf_ct_opts'struct bpf_ct_opts opts_def = { .l4proto = IPPROTO_TCP, .netns_id = -1 };Both of them should have their definition in
vmlinux.h. Can you check?
My bad, I didn't realize it was generated from the running kernel. The build works after trying again while running v5.18.
Kumar Kartikeya Dwivedi
maintainerPATCH v8 10/10 selftests/bpf: Add test for race in btf_try_get_module
In reply to Kumar Kartikeya DwivediThis adds a complete test case to ensure we never take references to modules not in MODULE_STATE_LIVE, which can lead to UAF, and it also ensures we never access btf->kfunc_set_tab in an inconsistent state.
The test uses userfaultfd to artificially widen the race.
When run on an unpatched kernel, it leads to the following splat:
[root@(none) bpf]# ./test_progs -t bpf_mod_race/ksym
[ 55.498171] BUG: unable to handle page fault for address: fffffbfff802548b [ 55.499206] #PF: supervisor read access in kernel mode [ 55.499855] #PF: error_code(0x0000) - not-present page [ 55.500555] PGD a4fa9067 P4D a4fa9067 PUD a4fa5067 PMD 1b44067 PTE 0 [ 55.501499] Oops: 0000 [#1] PREEMPT SMP KASAN NOPTI [ 55.502195] CPU: 0 PID: 83 Comm: kworker/0:2 Tainted: G OE 5.16.0-rc4+ #151 [ 55.503388] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS ArchLinux 1.15.0-1 04/01/2014 [ 55.504777] Workqueue: events bpf_prog_free_deferred [ 55.505563] RIP: 0010:kasan_check_range+0x184/0x1d0 [ 55.506363] Code: 12 83 e0 07 48 39 d0 7d 8a 41 bb 01 00 00 00 5b 5d 44 89 d8 41 5c c3 48 85 d2 74 ed 48 01 ea eb 09 48 83 c0 01 48 39 d0 74 df <80> 38 00 74 f2 e9 39 ff ff ff b8 01 00 00 00 c3 48 29 c3 48 89 da [ 55.509140] RSP: 0018:ffff88800560fcf0 EFLAGS: 00010282 [ 55.509977] RAX: fffffbfff802548b RBX: fffffbfff802548c RCX: ffffffff9337b6ba [ 55.511096] RDX: fffffbfff802548c RSI: 0000000000000004 RDI: ffffffffc012a458 [ 55.512143] RBP: fffffbfff802548b R08: 0000000000000001 R09: ffffffffc012a45b [ 55.513228] R10: fffffbfff802548b R11: 0000000000000001 R12: ffff888001b5f598 [ 55.514332] R13: ffff888004f49ac8 R14: 0000000000000000 R15: ffff888092449400 [ 55.515418] FS: 0000000000000000(0000) GS:ffff888092400000(0000) knlGS:0000000000000000 [ 55.516705] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 55.517560] CR2: fffffbfff802548b CR3: 0000000007c10006 CR4: 0000000000770ef0
[ 55.518672] PKRU: 55555554 [ 55.519022] Call Trace: [ 55.519483] <TASK> [ 55.519884] module_put.part.0+0x2a/0x180 [ 55.520642] bpf_prog_free_deferred+0x129/0x2e0 [ 55.521478] process_one_work+0x4fa/0x9e0 [ 55.522122] ? pwq_dec_nr_in_flight+0x100/0x100 [ 55.522878] ? rwlock_bug.part.0+0x60/0x60 [ 55.523551] worker_thread+0x2eb/0x700 [ 55.524176] ? __kthread_parkme+0xd8/0xf0 [ 55.524853] ? process_one_work+0x9e0/0x9e0 [ 55.525544] kthread+0x23a/0x270 [ 55.526088] ? set_kthread_struct+0x80/0x80 [ 55.526798] ret_from_fork+0x1f/0x30 [ 55.527413] </TASK> [ 55.527813] Modules linked in: bpf_testmod(OE) crc32_pclmul(E) intel_rapl_msr(E) intel_rapl_common(E) rapl(E) ghash_clmulni_intel(E) crct10dif_pclmul(E) crc32c_intel(E) serio_raw(E) [last unloaded: bpf_testmod] [ 55.530846] CR2: fffffbfff802548b [ 55.531341] ---[ end trace 1af41803c054ad6d ]--- [ 55.532136] RIP: 0010:kasan_check_range+0x184/0x1d0 [ 55.532918] Code: 12 83 e0 07 48 39 d0 7d 8a 41 bb 01 00 00 00 5b 5d 44 89 d8 41 5c c3 48 85 d2 74 ed 48 01 ea eb 09 48 83 c0 01 48 39 d0 74 df <80> 38 00 74 f2 e9 39 ff ff ff b8 01 00 00 00 c3 48 29 c3 48 89 da [ 55.535887] RSP: 0018:ffff88800560fcf0 EFLAGS: 00010282 [ 55.536711] RAX: fffffbfff802548b RBX: fffffbfff802548c RCX: ffffffff9337b6ba [ 55.537821] RDX: fffffbfff802548c RSI: 0000000000000004 RDI: ffffffffc012a458 [ 55.538899] RBP: fffffbfff802548b R08: 0000000000000001 R09: ffffffffc012a45b [ 55.539928] R10: fffffbfff802548b R11: 0000000000000001 R12: ffff888001b5f598 [ 55.541021] R13: ffff888004f49ac8 R14: 0000000000000000 R15: ffff888092449400 [ 55.542108] FS: 0000000000000000(0000) GS:ffff888092400000(0000) knlGS:0000000000000000 [ 55.543260]CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 55.544136] CR2: fffffbfff802548b CR3: 0000000007c10006 CR4: 0000000000770ef0 [ 55.545317] PKRU: 55555554 [ 55.545671] note: kworker/0:2[83] exited with preempt_count 1
--- net/bpf/test_run.c | 2 + .../selftests/bpf/bpf_testmod/bpf_testmod.c | 4 + tools/testing/selftests/bpf/config | 1 + .../selftests/bpf/prog_tests/bpf_mod_race.c | 230 ++++++++++++++++++ .../selftests/bpf/progs/bpf_mod_race.c | 100 ++++++++ .../selftests/bpf/progs/kfunc_call_race.c | 14 ++ tools/testing/selftests/bpf/progs/ksym_race.c | 13 + 7 files changed, 364 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_mod_race.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_mod_race.c create mode 100644 tools/testing/selftests/bpf/progs/kfunc_call_race.c create mode 100644 tools/testing/selftests/bpf/progs/ksym_race.c
net/bpf/test_run.c +2 -0
@@ -172,6 +172,8 @@ int noinline bpf_fentry_test1(int a) {return a +1 ; } +EXPORT_SYMBOL_GPL (bpf_fentry_test1); +ALLOW_ERROR_INJECTION (bpf_fentry_test1,ERRNO ); int noinline bpf_fentry_test2(int a, u64 b) {
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +4 -0
@@ -118,6 +118,8 @@ static const struct btf_kfunc_id_set bpf_testmod_kfunc_set = { .check_set = &bpf_testmod_check_kfunc_ids, }; +extern int bpf_fentry_test1 (int a); +static int bpf_testmod_init (void ) {int ret;
@@ -125,6 +127,8 @@ static int bpf_testmod_init(void) ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_SCHED_CLS , &bpf_testmod_kfunc_set);if (ret <0 )return ret; +if (bpf_fentry_test1 (0 ) <0 ) +return -EINVAL ;return sysfs_create_bin_file (kernel_kobj, &bin_attr_bpf_testmod_file); }
tools/testing/selftests/bpf/config +1 -0
@@ -52,3 +52,4 @@ CONFIG_NETFILTER=y CONFIG_NF_DEFRAG_IPV4=y CONFIG_NF_DEFRAG_IPV6=y CONFIG_NF_CONNTRACK=y +CONFIG_USERFAULTFD=y
tools/testing/selftests/bpf/prog_tests/bpf_mod_race.c new file +230 -0
230 lines not on this page · raw message
tools/testing/selftests/bpf/progs/bpf_mod_race.c new file +100 -0
100 lines not on this page · raw message
tools/testing/selftests/bpf/progs/kfunc_call_race.c new file +14 -0
@@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <vmlinux.h> +#include <bpf/bpf_helpers.h> + +extern void bpf_testmod_test_mod_kfunc(int i) __ksym; + +SEC("tc") +int kfunc_call_fail (struct __sk_buff *ctx) +{ +bpf_testmod_test_mod_kfunc (0 ); +return 0 ; +} + +char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/ksym_race.c new file +13 -0
@@ -0,0 +1,13 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <vmlinux.h> +#include <bpf/bpf_helpers.h> + +extern int bpf_testmod_ksym_percpu __ksym; + +SEC("tc") +int ksym_fail (struct __sk_buff *ctx) +{ +return *(int *)bpf_this_cpu_ptr (&bpf_testmod_ksym_percpu); +} + +char _license[] SEC("license") = "GPL";
Signature · 1 line
2.34.1
Re: [PATCH bpf-next v8 10/10] selftests/bpf: Add test for race in btf_try_get_module
Quoted 7 lines · Kumar Kartikeya Dwivedi
This adds a complete test case to ensure we never take references to modules not in
MODULE_STATE_LIVE, which can lead to UAF, and it also ensures we never accessbtf->kfunc_set_tabin an inconsistent state.The test uses userfaultfd to artificially widen the race.
Quoted 8 lines
When run on an unpatched kernel, it leads to the following splat:
[root@(none) bpf]# ./test_progs -t bpf_mod_race/ksym
[ 55.498171] BUG: unable to handle page fault for address: fffffbfff802548b [ 55.499206] #PF: supervisor read access in kernel mode [ 55.499855] #PF: error_code(0x0000) - not-present page [ 55.500555] PGD a4fa9067 P4D a4fa9067 PUD a4fa5067 PMD 1b44067 PTE 0 [ 55.501499] Oops: 0000 [#1] PREEMPT SMP KASAN NOPTI [ 55.502195] CPU: 0 PID: 83 Comm: kworker/0:2 Tainted: G OE 5.16.0-rc4+ #151 [ 55.503388] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS ArchLinux 1.15.0-1 04/01/2014
The commit log was messed up in a few places.
I've cleaned it up while applying.
Thanks!
Hi,
we are having a problem loading nf_conntrack on linux-next:
# modprobe nf_conntrack modprobe: ERROR: could not insert 'nf_conntrack': Unknown symbol in module, or unknown parameter (see dmesg) modprobe: ERROR: Error running install command '/sbin/modprobe --ignore-install nf_conntrack && /sbin/sysctl --quiet --pattern 'net[.]netfilter[.]nf_conntrack.*' --system' for module nf_conntrack: retcode 1 modprobe: ERROR: could not insert 'nf_conntrack': Invalid argument
# dmesg [ 3728.188969] missing module BTF, cannot register kfuncs [ 3748.208674] missing module BTF, cannot register kfuncs [ 3748.567123] missing module BTF, cannot register kfuncs [ 3873.597276] missing module BTF, cannot register kfuncs [ 3874.017125] missing module BTF, cannot register kfuncs [ 3882.637097] missing module BTF, cannot register kfuncs [ 3883.507213] missing module BTF, cannot register kfuncs [ 3883.876878] missing module BTF, cannot register kfuncs
# zgrep BTF /proc/config.gz CONFIG_DEBUG_INFO_BTF=y CONFIG_PAHOLE_HAS_SPLIT_BTF=y CONFIG_DEBUG_INFO_BTF_MODULES=y
It seems that nf_conntrack.ko is missing a .BTF section
which is present in debuginfo within /usr/lib/debug/lib/modules/*/kernel/net/netfilter/nf_conntrack.ko.debug instead.
Am i correct in assuming that this is not supported (yet) ?
We use pahole 1.22 and build linux-next on Fedora 35 as a set of custom packages. Architecture is s390x.
Thanks
Regards
Alex
Kumar Kartikeya Dwivedi
maintainerQuoted 25 lines · Alexander Egorenkov
Hi,
we are having a problem loading
nf_conntrackon linux-next:# modprobe nf_conntrack modprobe: ERROR: could not insert 'nf_conntrack': Unknown symbol in module, or unknown parameter (see dmesg) modprobe: ERROR: Error running install command '/sbin/modprobe --ignore-install nf_conntrack && /sbin/sysctl --quiet --pattern 'net[.]netfilter[.]nf_conntrack.*' --system' for module nf_conntrack: retcode 1 modprobe: ERROR: could not insert 'nf_conntrack': Invalid argument# dmesg [ 3728.188969] missing module BTF, cannot register kfuncs [ 3748.208674] missing module BTF, cannot register kfuncs [ 3748.567123] missing module BTF, cannot register kfuncs [ 3873.597276] missing module BTF, cannot register kfuncs [ 3874.017125] missing module BTF, cannot register kfuncs [ 3882.637097] missing module BTF, cannot register kfuncs [ 3883.507213] missing module BTF, cannot register kfuncs [ 3883.876878] missing module BTF, cannot register kfuncs# zgrep BTF /proc/config.gz CONFIG_DEBUG_INFO_BTF=y CONFIG_PAHOLE_HAS_SPLIT_BTF=y CONFIG_DEBUG_INFO_BTF_MODULES=y
Quoted 9 lines
It seems that nf_conntrack.ko is missing a .BTF section
which is present in debuginfo within/usr/lib/debug/lib/modules/*/kernel/net/netfilter/nf_conntrack.ko.debuginstead.Am i correct in assuming that this is not supported (yet) ?
We use pahole 1.22 and build linux-next on Fedora 35 as a set of custom packages. Architecture is s390x.
+Cc Ilya
Thanks for the report, Alex.
My assumption was that if DEBUG_INFO_BTF options was enabled, and BTF was not present, it is a problem, but it seems it can happen even when the options are enabled.
I guess if .BTF section isn't supported/emitted on s390x, we have to relax the error and print a warning and ignore it, but I am not sure. Ilya would probably know the current status.
We have already relaxed it once in (bpf-next):
c446fdacb10d ("bpf: fix register_btf_kfunc_id_set for !CONFIG_DEBUG_INFO_BTF")
If this doesn't work on s390x, we should probably just print a warning when CONFIG_DEBUG_INFO_BTF is enabled and btf == NULL, and return 0.
Quoted 4 lines
Thanks
Regards
Alex
--
Kartikeya
Hi Kartikeya,
sorry if i wasn't clear in my first message but just to be sure we have no misunderstandings :)
.BTF sections work on s390x with linux-next and nf_conntrack as well if the corresponding .BTF section is a part of the kernel module itself. I tested it myself by building a linux-next kernel and testing it with a KVM guest, i'm able to load nf_conntrack and it works.
In contrast, in the case where the corresponding .BTF section is separate from the kernel module nf_conntrack, it fails with the message i provided in the first email.
Therefore, my question is, does BTF for kernel modules work if the corresponding BTF section is NOT a part of the kernel module but instead is stored within the corresponding debuginfo file /usr/lib/debug/lib/modules/*/kernel/net/netfilter/nf_conntrack.ko.debug ?
Thanks
Regards
Alex
Kumar Kartikeya Dwivedi
maintainerQuoted 11 lines · Alexander Egorenkov
Hi Kartikeya,
sorry if i wasn't clear in my first message but just to be sure we have no misunderstandings :)
.BTF sections work on s390x with linux-next and
nf_conntrackas well if the corresponding .BTF section is a part of the kernel module itself. I tested it myself by building a linux-next kernel and testing it with a KVM guest, i'm able to loadnf_conntrackand it works.
Quoted 9 lines
In contrast, in the case where the corresponding .BTF section is separate from the kernel module
nf_conntrack, it fails with the message i provided in the first email.Therefore, my question is, does BTF for kernel modules work if the corresponding BTF section is NOT a part of the kernel module but instead is stored within the corresponding debuginfo file
/usr/lib/debug/lib/modules/*/kernel/net/netfilter/nf_conntrack.ko.debug?
Ah, I see, thanks for the clarification.
Currently module BTF is parsed during module load, so unless we can extend the kernel to pass BTF from a separate debuginfo during modprobe, and associate it with the loaded module, it won't work. You also won't be able to make the kfunc call from BPF program, as it would require module BTF fd during prog load.
Still, we can probably remove the error and return 0 when btf == NULL, even if Kconfig option CONFIG_DEBUG_INFO_BTF is set. I was assuming it would never be the case that the option is set and btf is not available after successful parsing, but clearly that is wrong.
If the BPF maintainers agree this is ok, feel free to send a fix to remove the error.
Quoted 6 lines
Thanks
Regards
Alex
--
Kartikeya
Quoted 12 lines · Alexander Egorenkov
Hi Kartikeya,
sorry if i wasn't clear in my first message but just to be sure we have no misunderstandings :)
.BTF sections work on s390x with linux-next and
nf_conntrackas well if the corresponding .BTF section is a part of the kernel module itself. I tested it myself by building a linux-next kernel and testing it with a KVM guest, i'm able to loadnf_conntrackand it works.
Quoted 8 lines
In contrast, in the case where the corresponding .BTF section is separate from the kernel module
nf_conntrack, it fails with the message i provided in the first email.Therefore, my question is, does BTF for kernel modules work if the corresponding BTF section is NOT a part of the kernel module but instead is stored within the corresponding debuginfo file
/usr/lib/debug/lib/modules/*/kernel/net/netfilter/nf_conntrack.ko.debug?
No, it doesn't. .BTF has to be part of the module's .ko ELF file. DWARF can be stripped out, it's only needed for generating BTF during the build process.
It actually would be great to have an option to generate BTF without leaving any DWARF, but that's a separate topic.
Quoted 7 lines
Thanks
Regards
Alex