Various improvements to cgroup helpers
Subsystems (7)
Daniel Borkmann
maintainercover Various improvements to cgroup helpers
This adds various straight-forward helper improvements and additions to BPF cgroup based connect(), sendmsg(), recvmsg() and bind-related hooks which would allow to implement more fine-grained policies and improve current load balancer limitations we're seeing. For details please see individual patches. I've tested them on Kubernetes & Cilium and also added selftests for the small verifier extension. Thanks!
Daniel Borkmann (7):
bpf: enable retrieval of socket cookie for bind/post-bind hook
bpf: enable perf event rb output for bpf cgroup progs
bpf: add netns cookie and enable it for bpf cgroup hooks
bpf: allow to retrieve cgroup v1 classid from v2 hooks
bpf: enable bpf cgroup hooks to retrieve cgroup v2 and ancestor id
bpf: enable retrival of pid/tgid/comm from bpf cgroup hooks
bpf: add selftest cases for ctx_or_null argument type
include/linux/bpf.h | 2 + include/net/cls_cgroup.h | 7 +- include/net/net_namespace.h | 10 ++ include/uapi/linux/bpf.h | 35 ++++++- kernel/bpf/core.c | 1 + kernel/bpf/helpers.c | 18 ++++ kernel/bpf/verifier.c | 16 ++-- net/core/filter.c | 106 ++++++++++++++++++++- net/core/net_namespace.c | 15 +++ tools/include/uapi/linux/bpf.h | 35 ++++++- tools/testing/selftests/bpf/verifier/ctx.c | 105 ++++++++++++++++++++ 11 files changed, 336 insertions(+), 14 deletions(-)
Signature · 1 line
2.21.0
Daniel Borkmann
maintainerPATCH 2/7 bpf: enable perf event rb output for bpf cgroup progs
Currently, connect(), sendmsg(), recvmsg() and bind-related hooks are all lacking perf event rb output in order to push notifications or monitoring events up to user space. Back in commit a5a3a828cd00 ("bpf: add perf event notificaton support for sock_ops"), I've worked with Sowmini to enable them for sock_ops where the context part is not used (as opposed to skbs for example where the packet data can be appended). Make the bpf_sockopt_event_output() helper generic and enable it for mentioned hooks.
--- net/core/filter.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
net/core/filter.c +9 -5
@@ -4159,8 +4159,8 @@ static const struct bpf_func_proto bpf_get_socket_uid_proto = { .arg1_type = ARG_PTR_TO_CTX, }; -BPF_CALL_5(bpf_sockopt_event_output, struct bpf_sock_ops_kern *, bpf_sock, - struct bpf_map *, map, u64, flags, void *, data, u64, size) +BPF_CALL_5(bpf_event_output_data, void *, ctx, struct bpf_map *, map, u64, flags, + void *, data, u64, size) {if (unlikely (flags & ~(BPF_F_INDEX_MASK )))return -EINVAL ;
@@ -4168,8 +4168,8 @@ BPF_CALL_5(bpf_sockopt_event_output, struct bpf_sock_ops_kern *, bpf_sock,return bpf_event_output (map, flags, data, size,NULL ,0 ,NULL ); } -static const struct bpf_func_proto bpf_sockopt_event_output_proto = { - .func = bpf_sockopt_event_output, +static const struct bpf_func_proto bpf_event_output_data_proto = { + .func = bpf_event_output_data, .gpl_only = true, .ret_type =RET_INTEGER , .arg1_type =ARG_PTR_TO_CTX ,
@@ -5968,6 +5968,8 @@ sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_local_storage_proto;case BPF_FUNC_get_socket_cookie:return &bpf_get_socket_cookie_sock_proto; +case BPF_FUNC_perf_event_output: +return &bpf_event_output_data_proto;default :return bpf_base_func_proto (func_id); }
@@ -5994,6 +5996,8 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_socket_cookie_sock_addr_proto;case BPF_FUNC_get_local_storage:return &bpf_get_local_storage_proto; +case BPF_FUNC_perf_event_output: +return &bpf_event_output_data_proto;#ifdef CONFIG_INET case BPF_FUNC_sk_lookup_tcp:return &bpf_sock_addr_sk_lookup_tcp_proto;
@@ -6236,7 +6240,7 @@ sock_ops_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)case BPF_FUNC_get_local_storage:return &bpf_get_local_storage_proto;case BPF_FUNC_perf_event_output: -return &bpf_sockopt_event_output_proto; +return &bpf_event_output_data_proto;case BPF_FUNC_sk_storage_get:return &bpf_sk_storage_get_proto;case BPF_FUNC_sk_storage_delete:
Signature · 1 line
2.21.0
Daniel Borkmann
maintainerPATCH 3/7 bpf: add netns cookie and enable it for bpf cgroup hooks
In reply to Daniel BorkmannIn Cilium we're mainly using BPF cgroup hooks today in order to implement kube-proxy free Kubernetes service translation for ClusterIP, NodePort (*), ExternalIP, and LoadBalancer as well as HostPort mapping [0] for all traffic between Cilium managed nodes. While this works in its current shape and avoids packet-level NAT for inter Cilium managed node traffic, there is one major limitation we're facing today, that is, lack of netns awareness.
In Kubernetes, the concept of Pods (which hold one or multiple containers) has been built around network namespaces, so while we can use the global scope of attaching to root BPF cgroup hooks also to our advantage (e.g. for exposing NodePort ports on loopback addresses), we also have the need to differentiate between initial network namespaces and non-initial one. For example, ExternalIP services mandate that non-local service IPs are not to be translated from the host (initial) network namespace as one example. Right now, we have an ugly work-around in place where non-local service IPs for ExternalIP services are not xlated from connect() and friends BPF hooks but instead via less efficient packet-level NAT on the veth tc ingress hook for Pod traffic.
On top of determining whether we're in initial or non-initial network namespace we also have a need for a socket-cookie like mechanism for network namespaces scope. Socket cookies have the nice property that they can be combined as part of the key structure e.g. for BPF LRU maps without having to worry that the cookie could be recycled. We are planning to use this for our sessionAffinity implementation for services. Therefore, add a new bpf_get_netns_cookie() helper which would resolve both use cases at once: bpf_get_netns_cookie(NULL) would provide the cookie for the initial network namespace while passing the context instead of NULL would provide the cookie from the application's network namespace. We're using a hole, so no size increase; the assignment happens only once. Therefore this allows for a comparison on initial namespace as well as regular cookie usage as we have today with socket cookies. We could later on enable this helper for other program types as well as we would see need.
(*) Both externalTrafficPolicy={Local|Cluster} types
[0] https://github.com/cilium/cilium/blob/master/bpf/bpf_sock.c
--- include/linux/bpf.h | 1 + include/net/net_namespace.h | 10 +++++++++ include/uapi/linux/bpf.h | 16 ++++++++++++++- kernel/bpf/verifier.c | 16 +++++++++------ net/core/filter.c | 37 ++++++++++++++++++++++++++++++++++ net/core/net_namespace.c | 15 ++++++++++++++ tools/include/uapi/linux/bpf.h | 16 ++++++++++++++- 7 files changed, 103 insertions(+), 8 deletions(-)
include/linux/bpf.h +1 -0
@@ -233,6 +233,7 @@ enum bpf_arg_type {ARG_CONST_SIZE_OR_ZERO ,/* number of bytes accessed from memory or 0 */ ARG_PTR_TO_CTX ,/* pointer to context */ +ARG_PTR_TO_CTX_OR_NULL ,/* pointer to context or NULL */ ARG_ANYTHING ,/* any (initialized) argument is ok */ ARG_PTR_TO_SPIN_LOCK ,/* pointer to bpf_spin_lock */ ARG_PTR_TO_SOCK_COMMON, /* pointer to sock_common */
include/net/net_namespace.h +10 -0
@@ -168,6 +168,9 @@ struct net {#ifdef CONFIG_XFRM struct netns_xfrm xfrm;#endif + +atomic64_t net_cookie;/* written once */ +#if IS_ENABLED (CONFIG_IP_VS )struct netns_ipvs *ipvs;#endif
@@ -273,6 +276,8 @@ static inline int check_net(const struct net *net)void net_drop_ns (void *); +u64 net_gen_cookie (struct net *net); +#else static inline struct net *get_net (struct net *net)
@@ -300,6 +305,11 @@ static inline int check_net(const struct net *net) return 1; } +static inline u64 net_gen_cookie (struct net *net) +{ +return 0 ; +} +#define net_drop_ns NULL#endif
include/uapi/linux/bpf.h +15 -1
@@ -2950,6 +2950,19 @@ union bpf_attr {* restricted to raw_tracepoint bpf programs. * Return * 0 on success, or a negative error in case of failure. +* +* u64 bpf_get_netns_cookie(void *ctx) +* Description +* Retrieve the cookie (generated by the kernel) of the network +* namespace the input *ctx* is associated with. The network +* namespace cookie remains stable for its lifetime and provides +* a global identifier that can be assumed unique. If *ctx* is +* NULL, then the helper returns the cookie for the initial +* network namespace. The cookie itself is very similar to that +* of bpf_get_socket_cookie() helper, but for network namespaces +* instead of sockets. +* Return +* A 8-byte long opaque number. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \
@@ -3073,7 +3086,8 @@ union bpf_attr {FN (jiffies64), \FN (read_branch_records), \FN (get_ns_current_pid_tgid), \ -FN (xdp_output), +FN (xdp_output), \ +FN (get_netns_cookie),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
kernel/bpf/verifier.c +10 -6
@@ -3461,13 +3461,17 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 regno, expected_type =CONST_PTR_TO_MAP ;if (type != expected_type) goto err_type; - }else if (arg_type ==ARG_PTR_TO_CTX ) { + }else if (arg_type ==ARG_PTR_TO_CTX || + arg_type ==ARG_PTR_TO_CTX_OR_NULL ) { expected_type =PTR_TO_CTX ; -if (type != expected_type) - goto err_type; - err =check_ctx_reg (env, reg, regno); -if (err <0 ) -return err; +if (!(register_is_null (reg) && + arg_type ==ARG_PTR_TO_CTX_OR_NULL )) { +if (type != expected_type) + goto err_type; + err =check_ctx_reg (env, reg, regno); +if (err <0 ) +return err; + } }else if (arg_type ==ARG_PTR_TO_SOCK_COMMON ) { expected_type =PTR_TO_SOCK_COMMON ;/* Any sk pointer can be ARG_PTR_TO_SOCK_COMMON */
net/core/filter.c +37 -0
@@ -4141,6 +4141,39 @@ static const struct bpf_func_proto bpf_get_socket_cookie_sock_ops_proto = { .arg1_type = ARG_PTR_TO_CTX, }; +static u64 __bpf_get_netns_cookie (struct sock *sk) +{ +#ifdef CONFIG_NET_NS +return net_gen_cookie (sk ? sk->sk_net.net : &init_net); +#else +return 0 ; +#endif +} + +BPF_CALL_1(bpf_get_netns_cookie_sock, struct sock *, ctx) +{ +return __bpf_get_netns_cookie (ctx); +} + +static const struct bpf_func_proto bpf_get_netns_cookie_sock_proto = { + .func = bpf_get_netns_cookie_sock, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX_OR_NULL , +}; + +BPF_CALL_1(bpf_get_netns_cookie_sock_addr, struct bpf_sock_addr_kern *, ctx) +{ +return __bpf_get_netns_cookie (ctx ? ctx->sk :NULL ); +} + +static const struct bpf_func_proto bpf_get_netns_cookie_sock_addr_proto = { + .func = bpf_get_netns_cookie_sock_addr, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX_OR_NULL , +}; + BPF_CALL_1(bpf_get_socket_uid, struct sk_buff *, skb) {struct sock *sk =sk_to_full_sk (skb->sk);
@@ -5968,6 +6001,8 @@ sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_local_storage_proto;case BPF_FUNC_get_socket_cookie:return &bpf_get_socket_cookie_sock_proto; +case BPF_FUNC_get_netns_cookie: +return &bpf_get_netns_cookie_sock_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto;default :
@@ -5994,6 +6029,8 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) }case BPF_FUNC_get_socket_cookie:return &bpf_get_socket_cookie_sock_addr_proto; +case BPF_FUNC_get_netns_cookie: +return &bpf_get_netns_cookie_sock_addr_proto;case BPF_FUNC_get_local_storage:return &bpf_get_local_storage_proto;case BPF_FUNC_perf_event_output:
net/core/net_namespace.c +15 -0
@@ -69,6 +69,20 @@ EXPORT_SYMBOL_GPL(pernet_ops_rwsem);static unsigned int max_gen_ptrs =INITIAL_NET_GEN_PTRS ; +static atomic64_t cookie_gen; + +u64 net_gen_cookie (struct net *net) +{ +while (1 ) { +u64 res =atomic64_read (&net->net_cookie); + +if (res) +return res; + res =atomic64_inc_return (&cookie_gen); +atomic64_cmpxchg (&net->net_cookie,0 , res); + } +} +static struct net_generic *net_alloc_generic (void ) {struct net_generic *ng;
@@ -1087,6 +1101,7 @@ static int __init net_ns_init(void)panic ("Could not allocate generic netns" );rcu_assign_pointer (init_net.gen, ng); +net_gen_cookie (&init_net);down_write (&pernet_ops_rwsem);if (setup_net (&init_net, &init_user_ns))
tools/include/uapi/linux/bpf.h +15 -1
@@ -2950,6 +2950,19 @@ union bpf_attr {* restricted to raw_tracepoint bpf programs. * Return * 0 on success, or a negative error in case of failure. +* +* u64 bpf_get_netns_cookie(void *ctx) +* Description +* Retrieve the cookie (generated by the kernel) of the network +* namespace the input *ctx* is associated with. The network +* namespace cookie remains stable for its lifetime and provides +* a global identifier that can be assumed unique. If *ctx* is +* NULL, then the helper returns the cookie for the initial +* network namespace. The cookie itself is very similar to that +* of bpf_get_socket_cookie() helper, but for network namespaces +* instead of sockets. +* Return +* A 8-byte long opaque number. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \
@@ -3073,7 +3086,8 @@ union bpf_attr {FN (jiffies64), \FN (read_branch_records), \FN (get_ns_current_pid_tgid), \ -FN (xdp_output), +FN (xdp_output), \ +FN (get_netns_cookie),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
Signature · 1 line
2.21.0
Daniel Borkmann
maintainerPATCH 5/7 bpf: enable bpf cgroup hooks to retrieve cgroup v2 and ancestor id
In reply to Daniel BorkmannEnable the bpf_get_current_cgroup_id() helper for connect(), sendmsg(), recvmsg() and bind-related hooks in order to retrieve the cgroup v2 context which can then be used as part of the key for BPF map lookups, for example. Given these hooks operate in process context 'current' is always valid and pointing to the app that is performing mentioned syscalls if it's subject to a v2 cgroup. Also with same motivation of commit 7723628101aa ("bpf: Introduce bpf_skb_ancestor_cgroup_id helper") enable retrieval of ancestor from current so the cgroup id can be used for policy lookups which can then forbid connect() / bind(), for example.
--- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 21 ++++++++++++++++++++- kernel/bpf/core.c | 1 + kernel/bpf/helpers.c | 18 ++++++++++++++++++ net/core/filter.c | 12 ++++++++++++ tools/include/uapi/linux/bpf.h | 21 ++++++++++++++++++++- 6 files changed, 72 insertions(+), 2 deletions(-)
include/linux/bpf.h +1 -0
@@ -1501,6 +1501,7 @@ extern const struct bpf_func_proto bpf_get_stack_proto;extern const struct bpf_func_proto bpf_sock_map_update_proto;extern const struct bpf_func_proto bpf_sock_hash_update_proto;extern const struct bpf_func_proto bpf_get_current_cgroup_id_proto; +extern const struct bpf_func_proto bpf_get_current_ancestor_cgroup_id_proto;extern const struct bpf_func_proto bpf_msg_redirect_hash_proto;extern const struct bpf_func_proto bpf_msg_redirect_map_proto;extern const struct bpf_func_proto bpf_sk_redirect_hash_proto;
include/uapi/linux/bpf.h +20 -1
@@ -2963,6 +2963,24 @@ union bpf_attr {* instead of sockets. * Return * A 8-byte long opaque number. +* +* u64 bpf_get_current_ancestor_cgroup_id(int ancestor_level) +* Description +* Return id of cgroup v2 that is ancestor of the cgroup associated +* with the current task at the *ancestor_level*. The root cgroup +* is at *ancestor_level* zero and each step down the hierarchy +* increments the level. If *ancestor_level* == level of cgroup +* associated with the current task, then return value will be the +* same as that of **bpf_get_current_cgroup_id**\ (). +* +* The helper is useful to implement policies based on cgroups +* that are upper in hierarchy than immediate cgroup associated +* with the current task. +* +* The format of returned id and helper limitations are same as in +* **bpf_get_current_cgroup_id**\ (). +* Return +* The id is returned or 0 in case the id could not be retrieved. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \
@@ -3087,7 +3105,8 @@ union bpf_attr {FN (read_branch_records), \FN (get_ns_current_pid_tgid), \FN (xdp_output), \ -FN (get_netns_cookie), +FN (get_netns_cookie), \ +FN (get_current_ancestor_cgroup_id),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
kernel/bpf/core.c +1 -0
@@ -2156,6 +2156,7 @@ const struct bpf_func_proto bpf_get_current_pid_tgid_proto __weak;const struct bpf_func_proto bpf_get_current_uid_gid_proto__weak ;const struct bpf_func_proto bpf_get_current_comm_proto__weak ;const struct bpf_func_proto bpf_get_current_cgroup_id_proto__weak ; +const struct bpf_func_proto bpf_get_current_ancestor_cgroup_id_proto__weak ;const struct bpf_func_proto bpf_get_local_storage_proto__weak ;const struct bpf_func_proto bpf_get_ns_current_pid_tgid_proto__weak ;
kernel/bpf/helpers.c +18 -0
@@ -340,6 +340,24 @@ const struct bpf_func_proto bpf_get_current_cgroup_id_proto = { .ret_type = RET_INTEGER, }; +BPF_CALL_1(bpf_get_current_ancestor_cgroup_id, int, ancestor_level) +{ +struct cgroup *cgrp =task_dfl_cgroup (current); +struct cgroup *ancestor; + + ancestor =cgroup_ancestor (cgrp, ancestor_level); +if (!ancestor) +return 0 ; +return cgroup_id (ancestor); +} + +const struct bpf_func_proto bpf_get_current_ancestor_cgroup_id_proto = { + .func = bpf_get_current_ancestor_cgroup_id, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_ANYTHING , +}; +#ifdef CONFIG_CGROUP_BPF DECLARE_PER_CPU(struct bpf_cgroup_storage*, bpf_cgroup_storage[MAX_BPF_CGROUP_STORAGE_TYPE ]);
net/core/filter.c +12 -0
@@ -6018,6 +6018,12 @@ sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_netns_cookie_sock_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +#ifdef CONFIG_CGROUPS +case BPF_FUNC_get_current_cgroup_id: +return &bpf_get_current_cgroup_id_proto; +case BPF_FUNC_get_current_ancestor_cgroup_id: +return &bpf_get_current_ancestor_cgroup_id_proto; +#endif #ifdef CONFIG_CGROUP_NET_CLASSID case BPF_FUNC_get_cgroup_classid:return &bpf_get_cgroup_classid_curr_proto;
@@ -6052,6 +6058,12 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_local_storage_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +#ifdef CONFIG_CGROUPS +case BPF_FUNC_get_current_cgroup_id: +return &bpf_get_current_cgroup_id_proto; +case BPF_FUNC_get_current_ancestor_cgroup_id: +return &bpf_get_current_ancestor_cgroup_id_proto; +#endif #ifdef CONFIG_CGROUP_NET_CLASSID case BPF_FUNC_get_cgroup_classid:return &bpf_get_cgroup_classid_curr_proto;
tools/include/uapi/linux/bpf.h +20 -1
@@ -2963,6 +2963,24 @@ union bpf_attr {* instead of sockets. * Return * A 8-byte long opaque number. +* +* u64 bpf_get_current_ancestor_cgroup_id(int ancestor_level) +* Description +* Return id of cgroup v2 that is ancestor of the cgroup associated +* with the current task at the *ancestor_level*. The root cgroup +* is at *ancestor_level* zero and each step down the hierarchy +* increments the level. If *ancestor_level* == level of cgroup +* associated with the current task, then return value will be the +* same as that of **bpf_get_current_cgroup_id**\ (). +* +* The helper is useful to implement policies based on cgroups +* that are upper in hierarchy than immediate cgroup associated +* with the current task. +* +* The format of returned id and helper limitations are same as in +* **bpf_get_current_cgroup_id**\ (). +* Return +* The id is returned or 0 in case the id could not be retrieved. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \
@@ -3087,7 +3105,8 @@ union bpf_attr {FN (read_branch_records), \FN (get_ns_current_pid_tgid), \FN (xdp_output), \ -FN (get_netns_cookie), +FN (get_netns_cookie), \ +FN (get_current_ancestor_cgroup_id),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
Signature · 1 line
2.21.0
Daniel Borkmann
maintainerPATCH 6/7 bpf: enable retrival of pid/tgid/comm from bpf cgroup hooks
In reply to Daniel BorkmannWe already have the bpf_get_current_uid_gid() helper enabled, and given we now have perf event RB output available for connect(), sendmsg(), recvmsg() and bind-related hooks, add a trivial change to enable bpf_get_current_pid_tgid() and bpf_get_current_comm() as well.
--- net/core/filter.c | 8 ++++++++ 1 file changed, 8 insertions(+)
net/core/filter.c +8 -0
@@ -6018,6 +6018,10 @@ sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_netns_cookie_sock_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +case BPF_FUNC_get_current_pid_tgid: +return &bpf_get_current_pid_tgid_proto; +case BPF_FUNC_get_current_comm: +return &bpf_get_current_comm_proto;#ifdef CONFIG_CGROUPS case BPF_FUNC_get_current_cgroup_id:return &bpf_get_current_cgroup_id_proto;
@@ -6058,6 +6062,10 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_local_storage_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +case BPF_FUNC_get_current_pid_tgid: +return &bpf_get_current_pid_tgid_proto; +case BPF_FUNC_get_current_comm: +return &bpf_get_current_comm_proto;#ifdef CONFIG_CGROUPS case BPF_FUNC_get_current_cgroup_id:return &bpf_get_current_cgroup_id_proto;
Signature · 1 line
2.21.0
Daniel Borkmann
maintainerPATCH 7/7 bpf: add selftest cases for ctx_or_null argument type
In reply to Daniel BorkmannAdd various tests to make sure the verifier keeps catching them:
# ./test_verifier [...] #230/p pass ctx or null check, 1: ctx OK #231/p pass ctx or null check, 2: null OK #232/p pass ctx or null check, 3: 1 OK #233/p pass ctx or null check, 4: ctx - const OK #234/p pass ctx or null check, 5: null (connect) OK #235/p pass ctx or null check, 6: null (bind) OK #236/p pass ctx or null check, 7: ctx (bind) OK #237/p pass ctx or null check, 8: null (bind) OK [...] Summary: 1595 PASSED, 0 SKIPPED, 0 FAILED
--- tools/testing/selftests/bpf/verifier/ctx.c | 105 +++++++++++++++++++++ 1 file changed, 105 insertions(+)
tools/testing/selftests/bpf/verifier/ctx.c +105 -0
@@ -91,3 +91,108 @@ .result = REJECT, .errstr = "variable ctx access var_off=(0x0; 0x4)", }, +{ + "pass ctx or null check, 1: ctx", + .insns = { +BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,0 ,0 , + BPF_FUNC_get_netns_cookie), +BPF_MOV64_IMM (BPF_REG_0 ,0 ), + BPF_EXIT_INSN(), + }, + .prog_type =BPF_PROG_TYPE_CGROUP_SOCK_ADDR , + .expected_attach_type = BPF_CGROUP_UDP6_SENDMSG, + .result = ACCEPT, +}, +{ +"pass ctx or null check, 2: null" , + .insns = { +BPF_MOV64_IMM (BPF_REG_1 ,0 ), +BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,0 ,0 , + BPF_FUNC_get_netns_cookie), +BPF_MOV64_IMM (BPF_REG_0 ,0 ), + BPF_EXIT_INSN(), + }, + .prog_type =BPF_PROG_TYPE_CGROUP_SOCK_ADDR , + .expected_attach_type = BPF_CGROUP_UDP6_SENDMSG, + .result = ACCEPT, +}, +{ +"pass ctx or null check, 3: 1" , + .insns = { +BPF_MOV64_IMM (BPF_REG_1 ,1 ), +BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,0 ,0 , + BPF_FUNC_get_netns_cookie), +BPF_MOV64_IMM (BPF_REG_0 ,0 ), + BPF_EXIT_INSN(), + }, + .prog_type =BPF_PROG_TYPE_CGROUP_SOCK_ADDR , + .expected_attach_type = BPF_CGROUP_UDP6_SENDMSG, + .result = REJECT, + .errstr = "R1 type=inv expected=ctx", +}, +{ +"pass ctx or null check, 4: ctx - const" , + .insns = { +BPF_ALU64_IMM (BPF_ADD ,BPF_REG_1 ,-612 ), +BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,0 ,0 , + BPF_FUNC_get_netns_cookie), +BPF_MOV64_IMM (BPF_REG_0 ,0 ), + BPF_EXIT_INSN(), + }, + .prog_type =BPF_PROG_TYPE_CGROUP_SOCK_ADDR , + .expected_attach_type = BPF_CGROUP_UDP6_SENDMSG, + .result = REJECT, + .errstr = "dereference of modified ctx ptr", +}, +{ +"pass ctx or null check, 5: null (connect)" , + .insns = { +BPF_MOV64_IMM (BPF_REG_1 ,0 ), +BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,0 ,0 , + BPF_FUNC_get_netns_cookie), +BPF_MOV64_IMM (BPF_REG_0 ,0 ), + BPF_EXIT_INSN(), + }, + .prog_type =BPF_PROG_TYPE_CGROUP_SOCK_ADDR , + .expected_attach_type = BPF_CGROUP_INET4_CONNECT, + .result = ACCEPT, +}, +{ +"pass ctx or null check, 6: null (bind)" , + .insns = { +BPF_MOV64_IMM (BPF_REG_1 ,0 ), +BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,0 ,0 , + BPF_FUNC_get_netns_cookie), +BPF_MOV64_IMM (BPF_REG_0 ,0 ), + BPF_EXIT_INSN(), + }, + .prog_type =BPF_PROG_TYPE_CGROUP_SOCK , + .expected_attach_type = BPF_CGROUP_INET4_POST_BIND, + .result = ACCEPT, +}, +{ +"pass ctx or null check, 7: ctx (bind)" , + .insns = { +BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,0 ,0 , + BPF_FUNC_get_socket_cookie), +BPF_MOV64_IMM (BPF_REG_0 ,0 ), + BPF_EXIT_INSN(), + }, + .prog_type =BPF_PROG_TYPE_CGROUP_SOCK , + .expected_attach_type = BPF_CGROUP_INET4_POST_BIND, + .result = ACCEPT, +}, +{ +"pass ctx or null check, 8: null (bind)" , + .insns = { +BPF_MOV64_IMM (BPF_REG_1 ,0 ), +BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,0 ,0 , + BPF_FUNC_get_socket_cookie), +BPF_MOV64_IMM (BPF_REG_0 ,0 ), + BPF_EXIT_INSN(), + }, + .prog_type =BPF_PROG_TYPE_CGROUP_SOCK , + .expected_attach_type = BPF_CGROUP_INET4_POST_BIND, + .result = REJECT, + .errstr = "R1 type=inv expected=ctx", +},
Signature · 1 line
2.21.0
Daniel Borkmann
maintainerPATCH 1/7 bpf: enable retrieval of socket cookie for bind/post-bind hook
In reply to Daniel BorkmannWe currently make heavy use of the socket cookie in BPF's connect(), sendmsg() and recvmsg() hooks for load-balancing decisions. However, it is currently not enabled/implemented in BPF {post-}bind hooks where it can later be used in combination for correlation in the tc egress path, for example.
--- net/core/filter.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
net/core/filter.c +14 -0
@@ -4117,6 +4117,18 @@ static const struct bpf_func_proto bpf_get_socket_cookie_sock_addr_proto = { .arg1_type = ARG_PTR_TO_CTX, }; +BPF_CALL_1(bpf_get_socket_cookie_sock, struct sock *, ctx) +{ +return sock_gen_cookie (ctx); +} + +static const struct bpf_func_proto bpf_get_socket_cookie_sock_proto = { + .func = bpf_get_socket_cookie_sock, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX , +}; + BPF_CALL_1(bpf_get_socket_cookie_sock_ops, struct bpf_sock_ops_kern *, ctx) {return sock_gen_cookie (ctx->sk);
@@ -5954,6 +5966,8 @@ sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_current_uid_gid_proto;case BPF_FUNC_get_local_storage:return &bpf_get_local_storage_proto; +case BPF_FUNC_get_socket_cookie: +return &bpf_get_socket_cookie_sock_proto;default :return bpf_base_func_proto (func_id); }
Signature · 1 line
2.21.0
Daniel Borkmann
maintainerPATCH 4/7 bpf: allow to retrieve cgroup v1 classid from v2 hooks
In reply to Daniel BorkmannToday, Kubernetes is still operating on cgroups v1, however, it is possible to retrieve the task's classid based on 'current' out of connect(), sendmsg(), recvmsg() and bind-related hooks for orchestrators which attach to the root cgroup v2 hook in a mixed env like in case of Cilium, for example, in order to then correlate certain pod traffic and use it as part of the key for BPF map lookups.
--- include/net/cls_cgroup.h | 7 ++++++- net/core/filter.c | 21 +++++++++++++++++++++ 2 files changed, 27 insertions(+), 1 deletion(-)
include/net/cls_cgroup.h +6 -1
@@ -45,9 +45,14 @@ static inline void sock_update_classid(struct sock_cgroup_data *skcd)sock_cgroup_set_classid (skcd, classid); } +static inline u32 __task_get_classid (struct task_struct *task) +{ +return task_cls_state (task)->classid; +} +static inline u32 task_get_classid (const struct sk_buff *skb) { -u32 classid =task_cls_state (current)->classid; +u32 classid =__task_get_classid (current);/* Due to the nature of the classifier it is required to ignore all * packets originating from softirq context as accessing `current'
net/core/filter.c +21 -0
@@ -2642,6 +2642,19 @@ static const struct bpf_func_proto bpf_msg_pop_data_proto = { .arg4_type = ARG_ANYTHING, }; +#ifdef CONFIG_CGROUP_NET_CLASSID +BPF_CALL_0 (bpf_get_cgroup_classid_curr) +{ +return __task_get_classid (current); +} + +static const struct bpf_func_proto bpf_get_cgroup_classid_curr_proto = { + .func = bpf_get_cgroup_classid_curr, + .gpl_only = false, + .ret_type =RET_INTEGER , +}; +#endif + BPF_CALL_1(bpf_get_cgroup_classid, const struct sk_buff *, skb) {return task_get_classid (skb);
@@ -6005,6 +6018,10 @@ sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_netns_cookie_sock_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +#ifdef CONFIG_CGROUP_NET_CLASSID +case BPF_FUNC_get_cgroup_classid: +return &bpf_get_cgroup_classid_curr_proto; +#endif default :return bpf_base_func_proto (func_id); }
@@ -6035,6 +6052,10 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_local_storage_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +#ifdef CONFIG_CGROUP_NET_CLASSID +case BPF_FUNC_get_cgroup_classid: +return &bpf_get_cgroup_classid_curr_proto; +#endif #ifdef CONFIG_INET case BPF_FUNC_sk_lookup_tcp:return &bpf_sock_addr_sk_lookup_tcp_proto;
Signature · 1 line
2.21.0
Re: [PATCH bpf-next 3/7] bpf: add netns cookie and enable it for bpf cgroup hooks
In reply to Daniel BorkmannQuoted 47 lines · Daniel Borkmann
In Cilium we're mainly using BPF cgroup hooks today in order to implement kube-proxy free Kubernetes service translation for ClusterIP, NodePort (*), ExternalIP, and LoadBalancer as well as HostPort mapping [0] for all traffic between Cilium managed nodes. While this works in its current shape and avoids packet-level NAT for inter Cilium managed node traffic, there is one major limitation we're facing today, that is, lack of netns awareness.
In Kubernetes, the concept of Pods (which hold one or multiple containers) has been built around network namespaces, so while we can use the global scope of attaching to root BPF cgroup hooks also to our advantage (e.g. for exposing NodePort ports on loopback addresses), we also have the need to differentiate between initial network namespaces and non-initial one. For example, ExternalIP services mandate that non-local service IPs are not to be translated from the host (initial) network namespace as one example. Right now, we have an ugly work-around in place where non-local service IPs for ExternalIP services are not xlated from
connect()and friends BPF hooks but instead via less efficient packet-level NAT on the veth tc ingress hook for Pod traffic.On top of determining whether we're in initial or non-initial network namespace we also have a need for a socket-cookie like mechanism for network namespaces scope. Socket cookies have the nice property that they can be combined as part of the key structure e.g. for BPF LRU maps without having to worry that the cookie could be recycled. We are planning to use this for our sessionAffinity implementation for services. Therefore, add a new
bpf_get_netns_cookie()helper which would resolve both use cases at once:bpf_get_netns_cookie(NULL)would provide the cookie for the initial network namespace while passing the context instead of NULL would provide the cookie from the application's network namespace. We're using a hole, so no size increase; the assignment happens only once. Therefore this allows for a comparison on initial namespace as well as regular cookie usage as we have today with socket cookies. We could later on enable this helper for other program types as well as we would see need.(*) Both externalTrafficPolicy={Local|Cluster} types
[0] https://github.com/cilium/cilium/blob/master/bpf/bpf_sock.cSigned-off-by Daniel Borkmann <[email protected]>--- include/linux/bpf.h | 1 + include/net/net_namespace.h | 10 +++++++++ include/uapi/linux/bpf.h | 16 ++++++++++++++- kernel/bpf/verifier.c | 16 +++++++++------ net/core/filter.c | 37 ++++++++++++++++++++++++++++++++++ net/core/net_namespace.c | 15 ++++++++++++++ tools/include/uapi/linux/bpf.h | 16 ++++++++++++++- 7 files changed, 103 insertions(+), 8 deletions(-)
[...]
Quoted 9 lines
kernel/bpf/verifier.c +0
-1@@ -3461,13 +3461,17 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 regno, expected_type =CONST_PTR_TO_MAP ;if (type != expected_type) goto err_type; - }else if (arg_type ==ARG_PTR_TO_CTX ) {
kernel/bpf/verifier.c static int check_func_arg(struct bpf_verifier_env *env, u32 regno, · line 3468
Quoted 10 lines
kernel/bpf/verifier.c +4
-5+ }else if (arg_type ==ARG_PTR_TO_CTX || + arg_type ==ARG_PTR_TO_CTX_OR_NULL ) { expected_type =PTR_TO_CTX ; -if (type != expected_type) - goto err_type; - err =check_ctx_reg (env, reg, regno); -if (err <0 ) -return err; +if (!(register_is_null (reg) && + arg_type ==ARG_PTR_TO_CTX_OR_NULL )) {
Other parts of check_func_arg() have different pattern that doesn't negate this complicated condition:
if (register_is_null(reg) && arg_type == ARG_PTR_TO_CTX_OR_NULL)
;
else {
...
}It's marginally easier to follow, though still increases nestedness :(
Quoted 9 lines
kernel/bpf/verifier.c +6
-0+if (type != expected_type) + goto err_type; + err =check_ctx_reg (env, reg, regno); +if (err <0 ) +return err; + } }else if (arg_type ==ARG_PTR_TO_SOCK_COMMON ) { expected_type =PTR_TO_SOCK_COMMON ;/* Any sk pointer can be ARG_PTR_TO_SOCK_COMMON */
[...]
kernel/bpf/verifier.c
Quoted 5 lines
kernel/bpf/verifier.c +5
-0+static const struct bpf_func_proto bpf_get_netns_cookie_sock_addr_proto = { + .func = bpf_get_netns_cookie_sock_addr, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX_OR_NULL ,
Just for completeness sake, have you considered using two helpers - one accepting context and other accepting nothing instead of adding ARG_PTR_TO_CTX_OR_NULL? Would it be too bad?
Quoted 2 lines
kernel/bpf/verifier.c +2
-0+}; +
[...]
kernel/bpf/verifier.c
Quoted 11 lines
kernel/bpf/verifier.c +11
-0+static atomic64_t cookie_gen; + +u64 net_gen_cookie (struct net *net) +{ +while (1 ) { +u64 res =atomic64_read (&net->net_cookie); + +if (res) +return res; + res =atomic64_inc_return (&cookie_gen); +atomic64_cmpxchg (&net->net_cookie,0 , res);
you'll always do extra atomic64_read, even if you succeed on the first try. Why not:
while (1) {
u64 res = atomic64_read(&net->net_cookie);
if (res)
return res;
res = atomic64_inc_return(&cookie_gen);
if (atomic64_cmpxchg(&net->net_cookie, 0, res) == 0)
return res;
}Quoted 3 lines
kernel/bpf/verifier.c +3
-0+ } +} +
[...]
Re: [PATCH bpf-next 4/7] bpf: allow to retrieve cgroup v1 classid from v2 hooks
In reply to Daniel BorkmannQuoted 8 lines · Daniel Borkmann
Today, Kubernetes is still operating on cgroups v1, however, it is possible to retrieve the task's classid based on 'current' out of
connect(),sendmsg(),recvmsg()and bind-related hooks for orchestrators which attach to the root cgroup v2 hook in a mixed env like in case of Cilium, for example, in order to then correlate certain pod traffic and use it as part of the key for BPF map lookups.
Have you tried getting this classid directly from task_struct in your BPF program with vmlinux.h and CO-RE? Seems like it should be pretty straightforward and not requiring a special BPF handler just for that?
Quoted 6 lines
Signed-off-by Daniel Borkmann <[email protected]>--- include/net/cls_cgroup.h | 7 ++++++- net/core/filter.c | 21 +++++++++++++++++++++ 2 files changed, 27 insertions(+), 1 deletion(-)
[...]
Re: [PATCH bpf-next 5/7] bpf: enable bpf cgroup hooks to retrieve cgroup v2 and ancestor id
In reply to Daniel BorkmannQuoted 11 lines · Daniel Borkmann
Enable the
bpf_get_current_cgroup_id()helper forconnect(),sendmsg(),recvmsg()and bind-related hooks in order to retrieve the cgroup v2 context which can then be used as part of the key for BPF map lookups, for example. Given these hooks operate in process context 'current' is always valid and pointing to the app that is performing mentioned syscalls if it's subject to a v2 cgroup. Also with same motivation of commit7723628101aa("bpf: Introducebpf_skb_ancestor_cgroup_idhelper") enable retrieval of ancestor from current so the cgroup id can be used for policy lookups which can then forbidconnect()/bind(), for example.
Quoted 2 lines
Signed-off-by Daniel Borkmann <[email protected]>---
Same question about just directly getting this from current through CO-RE.
Quoted 172 lines
include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 21 ++++++++++++++++++++- kernel/bpf/core.c | 1 + kernel/bpf/helpers.c | 18 ++++++++++++++++++ net/core/filter.c | 12 ++++++++++++ tools/include/uapi/linux/bpf.h | 21 ++++++++++++++++++++- 6 files changed, 72 insertions(+), 2 deletions(-)include/linux/bpf.h +1
-0@@ -1501,6 +1501,7 @@ extern const struct bpf_func_proto bpf_get_stack_proto;extern const struct bpf_func_proto bpf_sock_map_update_proto;extern const struct bpf_func_proto bpf_sock_hash_update_proto;extern const struct bpf_func_proto bpf_get_current_cgroup_id_proto; +extern const struct bpf_func_proto bpf_get_current_ancestor_cgroup_id_proto;extern const struct bpf_func_proto bpf_msg_redirect_hash_proto;extern const struct bpf_func_proto bpf_msg_redirect_map_proto;extern const struct bpf_func_proto bpf_sk_redirect_hash_proto;include/uapi/linux/bpf.h +20
-1@@ -2963,6 +2963,24 @@ union bpf_attr {* instead of sockets. * Return * A 8-byte long opaque number. +* +* u64 bpf_get_current_ancestor_cgroup_id(int ancestor_level) +* Description +* Return id of cgroup v2 that is ancestor of the cgroup associated +* with the current task at the *ancestor_level*. The root cgroup +* is at *ancestor_level* zero and each step down the hierarchy +* increments the level. If *ancestor_level* == level of cgroup +* associated with the current task, then return value will be the +* same as that of **bpf_get_current_cgroup_id**\ (). +* +* The helper is useful to implement policies based on cgroups +* that are upper in hierarchy than immediate cgroup associated +* with the current task. +* +* The format of returned id and helper limitations are same as in +* **bpf_get_current_cgroup_id**\ (). +* Return +* The id is returned or 0 in case the id could not be retrieved. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \@@ -3087,7 +3105,8 @@ union bpf_attr {FN (read_branch_records), \FN (get_ns_current_pid_tgid), \FN (xdp_output), \ -FN (get_netns_cookie), +FN (get_netns_cookie), \ +FN (get_current_ancestor_cgroup_id),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call kernel/bpf/core.c +1
-0@@ -2156,6 +2156,7 @@ const struct bpf_func_proto bpf_get_current_pid_tgid_proto __weak;const struct bpf_func_proto bpf_get_current_uid_gid_proto__weak ;const struct bpf_func_proto bpf_get_current_comm_proto__weak ;const struct bpf_func_proto bpf_get_current_cgroup_id_proto__weak ; +const struct bpf_func_proto bpf_get_current_ancestor_cgroup_id_proto__weak ;const struct bpf_func_proto bpf_get_local_storage_proto__weak ;const struct bpf_func_proto bpf_get_ns_current_pid_tgid_proto__weak ;kernel/bpf/helpers.c +18
-0@@ -340,6 +340,24 @@ const struct bpf_func_proto bpf_get_current_cgroup_id_proto = { .ret_type = RET_INTEGER, }; +BPF_CALL_1(bpf_get_current_ancestor_cgroup_id, int, ancestor_level) +{ +struct cgroup *cgrp =task_dfl_cgroup (current); +struct cgroup *ancestor; + + ancestor =cgroup_ancestor (cgrp, ancestor_level); +if (!ancestor) +return 0 ; +return cgroup_id (ancestor); +} + +const struct bpf_func_proto bpf_get_current_ancestor_cgroup_id_proto = { + .func = bpf_get_current_ancestor_cgroup_id, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_ANYTHING , +}; +#ifdef CONFIG_CGROUP_BPF DECLARE_PER_CPU(struct bpf_cgroup_storage*, bpf_cgroup_storage[MAX_BPF_CGROUP_STORAGE_TYPE ]);net/core/filter.c +12
-0@@ -6018,6 +6018,12 @@ sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_netns_cookie_sock_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +#ifdef CONFIG_CGROUPS +case BPF_FUNC_get_current_cgroup_id: +return &bpf_get_current_cgroup_id_proto; +case BPF_FUNC_get_current_ancestor_cgroup_id: +return &bpf_get_current_ancestor_cgroup_id_proto; +#endif #ifdef CONFIG_CGROUP_NET_CLASSID case BPF_FUNC_get_cgroup_classid:return &bpf_get_cgroup_classid_curr_proto;@@ -6052,6 +6058,12 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_local_storage_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +#ifdef CONFIG_CGROUPS +case BPF_FUNC_get_current_cgroup_id: +return &bpf_get_current_cgroup_id_proto; +case BPF_FUNC_get_current_ancestor_cgroup_id: +return &bpf_get_current_ancestor_cgroup_id_proto; +#endif #ifdef CONFIG_CGROUP_NET_CLASSID case BPF_FUNC_get_cgroup_classid:return &bpf_get_cgroup_classid_curr_proto;tools/include/uapi/linux/bpf.h +20
-2@@ -2963,6 +2963,24 @@ union bpf_attr {* instead of sockets. * Return * A 8-byte long opaque number. +* +* u64 bpf_get_current_ancestor_cgroup_id(int ancestor_level) +* Description +* Return id of cgroup v2 that is ancestor of the cgroup associated +* with the current task at the *ancestor_level*. The root cgroup +* is at *ancestor_level* zero and each step down the hierarchy +* increments the level. If *ancestor_level* == level of cgroup +* associated with the current task, then return value will be the +* same as that of **bpf_get_current_cgroup_id**\ (). +* +* The helper is useful to implement policies based on cgroups +* that are upper in hierarchy than immediate cgroup associated +* with the current task. +* +* The format of returned id and helper limitations are same as in +* **bpf_get_current_cgroup_id**\ (). +* Return +* The id is returned or 0 in case the id could not be retrieved. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \@@ -3087,7 +3105,8 @@ union bpf_attr {FN (read_branch_records), \FN (get_ns_current_pid_tgid), \FN (xdp_output), \ -FN (get_netns_cookie), +FN (get_netns_cookie), \ +FN (get_current_ancestor_cgroup_id),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call -- 2.21.0
Re: [PATCH bpf-next 6/7] bpf: enable retrival of pid/tgid/comm from bpf cgroup hooks
In reply to Daniel BorkmannQuoted 9 lines · Daniel Borkmann
We already have the
bpf_get_current_uid_gid()helper enabled, and given we now have perf event RB output available forconnect(),sendmsg(),recvmsg()and bind-related hooks, add a trivial change to enablebpf_get_current_pid_tgid()andbpf_get_current_comm()as well.Signed-off-by Daniel Borkmann <[email protected]>---
LGTM, there was probably never a good reason this wasn't available from the very beginning :)
Might as well add bpf_get_current_uid_gid() if it's not there yet.
Quoted 8 lines
net/core/filter.c | 8 ++++++++ 1 file changed, 8 insertions(+)net/core/filter.c +0
-0
net/core/filter.c sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 6024
Quoted 7 lines
net/core/filter.c +4
-0@@ -6018,6 +6018,10 @@ sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_netns_cookie_sock_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +case BPF_FUNC_get_current_pid_tgid: +return &bpf_get_current_pid_tgid_proto; +case BPF_FUNC_get_current_comm: +return &bpf_get_current_comm_proto;
So you are not adding it to bpf_base_func_proto() instead, because that one can be used in BPF programs that don't have a valid current, is that right? If yes, would it make sense to have a common bpf_base_process_ctx_func_proto() function for cases where there is a valid current and add all the functions there (including uid_gid and whatever else makes sense?)
Quoted 17 lines
net/core/filter.c +4
-1#ifdef CONFIG_CGROUPS case BPF_FUNC_get_current_cgroup_id:return &bpf_get_current_cgroup_id_proto;@@ -6058,6 +6062,10 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_local_storage_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +case BPF_FUNC_get_current_pid_tgid: +return &bpf_get_current_pid_tgid_proto; +case BPF_FUNC_get_current_comm: +return &bpf_get_current_comm_proto;#ifdef CONFIG_CGROUPS case BPF_FUNC_get_current_cgroup_id:return &bpf_get_current_cgroup_id_proto; --2.21.0
Re: [PATCH bpf-next 7/7] bpf: add selftest cases for ctx_or_null argument type
In reply to Daniel BorkmannQuoted 26 lines · Daniel Borkmann
Add various tests to make sure the verifier keeps catching them:
# ./test_verifier [...] #230/p pass ctx or null check, 1: ctx OK #231/p pass ctx or null check, 2: null OK #232/p pass ctx or null check, 3: 1 OK #233/p pass ctx or null check, 4: ctx - const OK #234/p pass ctx or null check, 5: null (connect) OK #235/p pass ctx or null check, 6: null (bind) OK #236/p pass ctx or null check, 7: ctx (bind) OK #237/p pass ctx or null check, 8: null (bind) OK [...] Summary: 1595 PASSED, 0 SKIPPED, 0 FAILEDSigned-off-by Daniel Borkmann <[email protected]>--- tools/testing/selftests/bpf/verifier/ctx.c | 105 +++++++++++++++++++++ 1 file changed, 105 insertions(+)tools/testing/selftests/bpf/verifier/ctx.c +0
-0
tools/testing/selftests/bpf/verifier/ctx.c line 98
Quoted 8 lines
tools/testing/selftests/bpf/verifier/ctx.c +5
-0@@ -91,3 +91,108 @@ .result = REJECT, .errstr = "variable ctx access var_off=(0x0; 0x4)", }, +{ + "pass ctx or null check, 1: ctx", + .insns = { +BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,0 ,0 , + BPF_FUNC_get_netns_cookie),
nit: seems like it deserves its own helper, e.g.,BPF_CALL_HELPER(BPF_FUNC_get_netns_cookie)?
Quoted 7 lines
tools/testing/selftests/bpf/verifier/ctx.c +7
-0+ BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_EXIT_INSN(), + }, + .prog_type = BPF_PROG_TYPE_CGROUP_SOCK_ADDR, + .expected_attach_type = BPF_CGROUP_UDP6_SENDMSG, + .result = ACCEPT, +},
[...]
Daniel Borkmann
maintainerRe: [PATCH bpf-next 3/7] bpf: add netns cookie and enable it for bpf cgroup hooks
In reply to Andrii NakryikoQuoted 74 lines · Andrii Nakryiko
Quoted 47 lines · Daniel Borkmann
In Cilium we're mainly using BPF cgroup hooks today in order to implement kube-proxy free Kubernetes service translation for ClusterIP, NodePort (*), ExternalIP, and LoadBalancer as well as HostPort mapping [0] for all traffic between Cilium managed nodes. While this works in its current shape and avoids packet-level NAT for inter Cilium managed node traffic, there is one major limitation we're facing today, that is, lack of netns awareness.
In Kubernetes, the concept of Pods (which hold one or multiple containers) has been built around network namespaces, so while we can use the global scope of attaching to root BPF cgroup hooks also to our advantage (e.g. for exposing NodePort ports on loopback addresses), we also have the need to differentiate between initial network namespaces and non-initial one. For example, ExternalIP services mandate that non-local service IPs are not to be translated from the host (initial) network namespace as one example. Right now, we have an ugly work-around in place where non-local service IPs for ExternalIP services are not xlated from
connect()and friends BPF hooks but instead via less efficient packet-level NAT on the veth tc ingress hook for Pod traffic.On top of determining whether we're in initial or non-initial network namespace we also have a need for a socket-cookie like mechanism for network namespaces scope. Socket cookies have the nice property that they can be combined as part of the key structure e.g. for BPF LRU maps without having to worry that the cookie could be recycled. We are planning to use this for our sessionAffinity implementation for services. Therefore, add a new
bpf_get_netns_cookie()helper which would resolve both use cases at once:bpf_get_netns_cookie(NULL)would provide the cookie for the initial network namespace while passing the context instead of NULL would provide the cookie from the application's network namespace. We're using a hole, so no size increase; the assignment happens only once. Therefore this allows for a comparison on initial namespace as well as regular cookie usage as we have today with socket cookies. We could later on enable this helper for other program types as well as we would see need.(*) Both externalTrafficPolicy={Local|Cluster} types
[0] https://github.com/cilium/cilium/blob/master/bpf/bpf_sock.cSigned-off-by Daniel Borkmann <[email protected]>--- include/linux/bpf.h | 1 + include/net/net_namespace.h | 10 +++++++++ include/uapi/linux/bpf.h | 16 ++++++++++++++- kernel/bpf/verifier.c | 16 +++++++++------ net/core/filter.c | 37 ++++++++++++++++++++++++++++++++++ net/core/net_namespace.c | 15 ++++++++++++++ tools/include/uapi/linux/bpf.h | 16 ++++++++++++++- 7 files changed, 103 insertions(+), 8 deletions(-)[...]
Quoted 9 lines
kernel/bpf/verifier.c +0
-1@@ -3461,13 +3461,17 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 regno, expected_type =CONST_PTR_TO_MAP ;if (type != expected_type) goto err_type; - }else if (arg_type ==ARG_PTR_TO_CTX ) {kernel/bpf/verifier.c static int check_func_arg(struct bpf_verifier_env *env, u32 regno, · line 3468
Quoted 10 lines
kernel/bpf/verifier.c +4
-5+ }else if (arg_type ==ARG_PTR_TO_CTX || + arg_type ==ARG_PTR_TO_CTX_OR_NULL ) { expected_type =PTR_TO_CTX ; -if (type != expected_type) - goto err_type; - err =check_ctx_reg (env, reg, regno); -if (err <0 ) -return err; +if (!(register_is_null (reg) && + arg_type ==ARG_PTR_TO_CTX_OR_NULL )) {Other parts of
check_func_arg()have different pattern that doesn't negate this complicated condition:
Quoted 7 lines
if (
register_is_null(reg)&&arg_type==ARG_PTR_TO_CTX_OR_NULL); else { ... }It's marginally easier to follow, though still increases nestedness :(
Yeah, that looks a bit ugly tbh, but perhaps personal style/preference. I tend to avoid such empty bodies.
Quoted 18 lines
Quoted 9 lines
kernel/bpf/verifier.c +6
-0+if (type != expected_type) + goto err_type; + err =check_ctx_reg (env, reg, regno); +if (err <0 ) +return err; + } }else if (arg_type ==ARG_PTR_TO_SOCK_COMMON ) { expected_type =PTR_TO_SOCK_COMMON ;/* Any sk pointer can be ARG_PTR_TO_SOCK_COMMON */ [...]
kernel/bpf/verifier.c
Quoted 5 lines
kernel/bpf/verifier.c +5
-0+static const struct bpf_func_proto bpf_get_netns_cookie_sock_addr_proto = { + .func = bpf_get_netns_cookie_sock_addr, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX_OR_NULL ,
Quoted 3 lines
Just for completeness sake, have you considered using two helpers - one accepting context and other accepting nothing instead of adding
ARG_PTR_TO_CTX_OR_NULL? Would it be too bad?
I haven't considered it since it feels a bit too much churn to the helper side for just this simple addition. Both helpers would be doing the same and I would need to have it duplicated for sock_addr and sock once again given we need it in both connect et al and bind contexts.
Quoted 20 lines
Quoted 2 lines
kernel/bpf/verifier.c +2
-0+}; +[...]
kernel/bpf/verifier.c
Quoted 11 lines
kernel/bpf/verifier.c +11
-0+static atomic64_t cookie_gen; + +u64 net_gen_cookie (struct net *net) +{ +while (1 ) { +u64 res =atomic64_read (&net->net_cookie); + +if (res) +return res; + res =atomic64_inc_return (&cookie_gen); +atomic64_cmpxchg (&net->net_cookie,0 , res);you'll always do extra
atomic64_read, even if you succeed on the first try. Why not:
Quoted 8 lines
while (1) {
u64 res = atomic64_read(&net->net_cookie);
if (res)
return res;
res = atomic64_inc_return(&cookie_gen);
if (atomic64_cmpxchg(&net->net_cookie, 0, res) == 0)
return res;
}Right, though it's on same CPU, so might not make too much of a noticeable difference. I've used same scheme here as with socket cookies actually. I thought about consolidating the generator into a separate function for both as next step and reuse some of the ideas from get_next_ino() for batching to optimize the atomic op cost, which should make a difference, but will do as a separate patch.
Quoted 6 lines
Quoted 3 lines
kernel/bpf/verifier.c +3
-0+ } +} +[...]
Daniel Borkmann
maintainerRe: [PATCH bpf-next 6/7] bpf: enable retrival of pid/tgid/comm from bpf cgroup hooks
In reply to Andrii NakryikoQuoted 8 lines · Andrii Nakryiko
Quoted 7 lines · Daniel Borkmann
We already have the
bpf_get_current_uid_gid()helper enabled, and given we now have perf event RB output available forconnect(),sendmsg(),recvmsg()and bind-related hooks, add a trivial change to enablebpf_get_current_pid_tgid()andbpf_get_current_comm()as well.
Quoted 5 lines
Quoted 2 lines
Signed-off-by Daniel Borkmann <[email protected]>---
LGTM, there was probably never a good reason this wasn't available from the very beginning :)
Right. :-)
Quoted 1 line
Might as well add
bpf_get_current_uid_gid()if it's not there yet.
It's already there. ;-)
Quoted 16 lines
Acked-by Andrii Nakryiko <[email protected]>Quoted 14 lines
net/core/filter.c | 8 ++++++++ 1 file changed, 8 insertions(+)net/core/filter.c +3
-0@@ -6018,6 +6018,10 @@ sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_netns_cookie_sock_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +case BPF_FUNC_get_current_pid_tgid: +return &bpf_get_current_pid_tgid_proto; +case BPF_FUNC_get_current_comm:
Quoted 8 lines
net/core/filter.c sock_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) · line 6024
Quoted 1 line
net/core/filter.c +1
-0+return &bpf_get_current_comm_proto;So you are not adding it to
bpf_base_func_proto()instead, because that one can be used in BPF programs that don't have a valid current, is that right? If yes, would it make sense to have a commonbpf_base_process_ctx_func_proto()function for cases where there is a valid current and add all the functions there (includinguid_gidand whatever else makes sense?)
I didn't add it to bpf_base_func_proto() since we might not always have a useful 'current' available. My focus in this series was on sock_addr and sock helpers for our LB where 'current' is always the application doing the syscall. A common base_func_proto() for both could be useful though this only works with helpers that do not rely on the input context since both are different. Tbh, the whole net/core/filter.c feels quite convoluted these days where it's getting hard to follow which func_proto and access checker belongs to which program type. I can check if we could get some more order in there in general through some larger refactoring to make it easier for people to extend, though not sure if more bpf_base_func_proto()-like helpers would add much, I think it needs a larger revamp.
Quoted 17 lines
Quoted 17 lines
net/core/filter.c +4
-1#ifdef CONFIG_CGROUPS case BPF_FUNC_get_current_cgroup_id:return &bpf_get_current_cgroup_id_proto;@@ -6058,6 +6062,10 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_local_storage_proto;case BPF_FUNC_perf_event_output:return &bpf_event_output_data_proto; +case BPF_FUNC_get_current_pid_tgid: +return &bpf_get_current_pid_tgid_proto; +case BPF_FUNC_get_current_comm: +return &bpf_get_current_comm_proto;#ifdef CONFIG_CGROUPS case BPF_FUNC_get_current_cgroup_id:return &bpf_get_current_cgroup_id_proto; --2.21.0
Re: [PATCH bpf-next 3/7] bpf: add netns cookie and enable it for bpf cgroup hooks
In reply to Daniel BorkmannQuoted 11 lines · Daniel Borkmann
+ *
+ * u64bpf_get_netns_cookie(void *ctx)
+ * Description
+ * Retrieve the cookie (generated by the kernel) of the network
+ * namespace the input *ctx* is associated with. The network
+ * namespace cookie remains stable for its lifetime and provides
+ * a global identifier that can be assumed unique. If *ctx* is
+ * NULL, then the helper returns the cookie for the initial
+ * network namespace. The cookie itself is very similar to that
+ * ofbpf_get_socket_cookie()helper, but for network namespaces
+ * instead of sockets.
All new helpers in this patch and few others are missing 'flags' argument. Yes. It's kinda hard right now to come up with a use case for the flags, since all helpers look kinda trivial, simple, and single purpose. But the same thing happened with bpf_send_signal(). It felt that there is no way to extend it. Yet later we had to add bpf_send_signal_thread() which could have been handled with flags if they were there. So please add flags to all new helpers though it might seem redundant.
Daniel Borkmann
maintainerRe: [PATCH bpf-next 4/7] bpf: allow to retrieve cgroup v1 classid from v2 hooks
In reply to Andrii NakryikoQuoted 9 lines · Andrii Nakryiko
Quoted 7 lines · Daniel Borkmann
Today, Kubernetes is still operating on cgroups v1, however, it is possible to retrieve the task's classid based on 'current' out of
connect(),sendmsg(),recvmsg()and bind-related hooks for orchestrators which attach to the root cgroup v2 hook in a mixed env like in case of Cilium, for example, in order to then correlate certain pod traffic and use it as part of the key for BPF map lookups.
Quoted 3 lines
Have you tried getting this classid directly from
task_structin your BPF program withvmlinux.hand CO-RE? Seems like it should be pretty straightforward and not requiring a special BPF handler just for that?
To answer both questions (5/7 and this one) in the same mail here: my understanding is that this would require to install additional tracing programs on these hooks instead of being able to integrate them into [0] for usage out of sock_addr and sock progs (similar as they are available as well from tc from skb)?
Thanks,
Daniel
[0] https://github.com/cilium/cilium/blob/master/bpf/bpf_sock.c
Daniel Borkmann
maintainerRe: [PATCH bpf-next 3/7] bpf: add netns cookie and enable it for bpf cgroup hooks
In reply to Alexei StarovoitovQuoted 13 lines · Alexei Starovoitov
Quoted 11 lines · Daniel Borkmann
+ *
+ * u64bpf_get_netns_cookie(void *ctx)
+ * Description
+ * Retrieve the cookie (generated by the kernel) of the network
+ * namespace the input *ctx* is associated with. The network
+ * namespace cookie remains stable for its lifetime and provides
+ * a global identifier that can be assumed unique. If *ctx* is
+ * NULL, then the helper returns the cookie for the initial
+ * network namespace. The cookie itself is very similar to that
+ * ofbpf_get_socket_cookie()helper, but for network namespaces
+ * instead of sockets.
Quoted 7 lines
All new helpers in this patch and few others are missing 'flags' argument. Yes. It's kinda hard right now to come up with a use case for the flags, since all helpers look kinda trivial, simple, and single purpose. But the same thing happened with
bpf_send_signal(). It felt that there is no way to extend it. Yet later we had to addbpf_send_signal_thread()which could have been handled with flags if they were there. So please add flags to all new helpers though it might seem redundant.
We have very similar helpers for almost 2yrs now, that is, bpf_get_socket_cookie() and bpf_skb_ancestor_cgroup_id(). Both no extra 'unused flags' arg and they are simple enough to do exactly what we expect them to do, I also haven't seen any reason to extend them further so far (otherwise we have have new ones by now). The two added here are very much analogue to this, so breaking this consistency is super ugly just to add empty flags now. :/ Given the timeframe we have these by now and given they do one simple thing, what is the harm to add a new helper in future iff really needed rather than uglifying with flags now (I would understand it for complex helpers though where we use this practice)? Just recently we added bpf_jiffies64() (5576b991e9c1 ("bpf: Add BPF_FUNC_jiffies64")); no flag either and it has similar simplicity.
Thanks,
Daniel
Re: [PATCH bpf-next 3/7] bpf: add netns cookie and enable it for bpf cgroup hooks
Quoted 22 lines · Daniel Borkmann
Quoted 13 lines · Alexei Starovoitov
Quoted 11 lines · Daniel Borkmann
+ *
+ * u64bpf_get_netns_cookie(void *ctx)
+ * Description
+ * Retrieve the cookie (generated by the kernel) of the network
+ * namespace the input *ctx* is associated with. The network
+ * namespace cookie remains stable for its lifetime and provides
+ * a global identifier that can be assumed unique. If *ctx* is
+ * NULL, then the helper returns the cookie for the initial
+ * network namespace. The cookie itself is very similar to that
+ * ofbpf_get_socket_cookie()helper, but for network namespaces
+ * instead of sockets.Quoted 7 lines
All new helpers in this patch and few others are missing 'flags' argument. Yes. It's kinda hard right now to come up with a use case for the flags, since all helpers look kinda trivial, simple, and single purpose. But the same thing happened with
bpf_send_signal(). It felt that there is no way to extend it. Yet later we had to addbpf_send_signal_thread()which could have been handled with flags if they were there. So please add flags to all new helpers though it might seem redundant.
Quoted 11 lines
We have very similar helpers for almost 2yrs now, that is,
bpf_get_socket_cookie()andbpf_skb_ancestor_cgroup_id(). Both no extra 'unused flags' arg and they are simple enough to do exactly what we expect them to do, I also haven't seen any reason to extend them further so far (otherwise we have have new ones by now). The two added here are very much analogue to this, so breaking this consistency is super ugly just to add empty flags now. :/ Given the timeframe we have these by now and given they do one simple thing, what is the harm to add a new helper in future iff really needed rather than uglifying with flags now (I would understand it for complex helpers though where we use this practice)? Just recently we addedbpf_jiffies64()(5576b991e9c1("bpf: AddBPF_FUNC_jiffies64")); no flag either and it has similar simplicity.
Fair enough, but I reserve the right to say "I told you so" later ;) The series applied. Thanks.
Re: [PATCH bpf-next 4/7] bpf: allow to retrieve cgroup v1 classid from v2 hooks
In reply to Daniel BorkmannQuoted 15 lines · Daniel Borkmann
Quoted 9 lines · Andrii Nakryiko
Quoted 7 lines · Daniel Borkmann
Today, Kubernetes is still operating on cgroups v1, however, it is possible to retrieve the task's classid based on 'current' out of
connect(),sendmsg(),recvmsg()and bind-related hooks for orchestrators which attach to the root cgroup v2 hook in a mixed env like in case of Cilium, for example, in order to then correlate certain pod traffic and use it as part of the key for BPF map lookups.Quoted 3 lines
Have you tried getting this classid directly from
task_structin your BPF program withvmlinux.hand CO-RE? Seems like it should be pretty straightforward and not requiring a special BPF handler just for that?
Quoted 5 lines
To answer both questions (5/7 and this one) in the same mail here: my understanding is that this would require to install additional tracing programs on these hooks instead of being able to integrate them into [0] for usage out of
sock_addrand sock progs (similar as they are available as well from tc from skb)?
No, not really, assuming bpf_get_current_task() helper is available for those programs. Something like this should work, can't really check because I don't know what classid value is supposed to be, but all the relocations succeed, so at least typing wise it should be good:
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_core_read.h>
static __always_inline u32 get_cgroup_classid(void)
{
struct task_struct *task = (void *)bpf_get_current_task(); struct cgroup_cls_state *state = (void *)BPF_CORE_READ(task,
cgroups, subsys[net_cls_cgrp_id]);
return BPF_CORE_READ(state, classid); }
I've cheated with conversion from struct cgroup_subsys_state * to struct cgroup_cls_state *, given that they match right now. But it is possible to have relocatable equivalent of container_of() macro for CO-RE, I'd be happy to play with that and provide it as part of bpf_core_read.h, if necessary.
Hope this clarifies what I meant by implementing with CO-RE.
Quoted 5 lines
Thanks,
Daniel[0] https://github.com/cilium/cilium/blob/master/bpf/bpf_sock.c
kbuild test robot
build botRe: [PATCH bpf-next 3/7] bpf: add netns cookie and enable it for bpf cgroup hooks
In reply to Daniel BorkmannI love your patch! Yet something to improve:
Hi Daniel,
I love your patch! Yet something to improve:
[auto build test ERROR on bpf-next/master]
[also build test ERROR on net-next/master next-20200327]
[cannot apply to bpf/master net/master linus/master v5.6]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Daniel-Borkmann/Various-improvements-to-cgroup-helpers/20200328-042919 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master config: x86_64-randconfig-s0-20200330 (attached as .config) compiler: gcc-7 (Debian 7.5.0-5) 7.5.0 reproduce: # save the attached .config to linux build tree make ARCH=x86_64
If you fix the issue, kindly add following tag
All errors (new ones prefixed by >>):
>> net/core/net_namespace.c:74:5: error: redefinition of 'net_gen_cookie'
u64 net_gen_cookie(struct net *net)
^~~~~~~~~~~~~~
In file included from include/linux/netdevice.h:38:0,
from include/linux/rtnetlink.h:7,
from net/core/net_namespace.c:5:
include/net/net_namespace.h:308:19: note: previous definition of 'net_gen_cookie' was here
static inline u64 net_gen_cookie(struct net *net)
^~~~~~~~~~~~~~
vim +/net_gen_cookie +74 net/core/net_namespace.c
73
> 74 u64 net_gen_cookie(struct net *net)
75 {
76 while (1) {
77 u64 res = atomic64_read(&net->net_cookie);
78
79 if (res)
80 return res;
81 res = atomic64_inc_return(&cookie_gen);
82 atomic64_cmpxchg(&net->net_cookie, 0, res);
83 }
84 }
85 ---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org