implement bpf iterator for tcp and udp sockets
Subsystems (12)
v2 cover implement bpf iterator for tcp and udp sockets
bpf iterator implments traversal of kernel data structures and these data structures are passed to a bpf program for processing. This gives great flexibility for users to examine kernel data structure without using e.g. /proc/net which has limited and fixed format.
Commit 138d0be35b14 ("net: bpf: Add netlink and ipv6_route bpf_iter targets") implemented bpf iterators for netlink and ipv6_route.
This patch set intends to implement bpf iterators for tcp and udp.
Currently, /proc/net/tcp is used to print tcp4 stats and /proc/net/tcp6 is used to print tcp6 stats. /proc/net/udp[6] have similar usage model. In contrast, only one tcp iterator is implemented and it is bpf program resposibility to filter based on socket family. The same is for udp. This will avoid another unnecessary traversal pass if users want to check both tcp4 and tcp6.
Several helpers are also implemented in this patch
bpf_skc_to_{tcp, tcp6, tcp_timewait, tcp_request, udp6}_sock
The argument for these helpers is not a fixed btf_id. For example,
bpf_skc_to_tcp(struct sock_common *), or bpf_skc_to_tcp(struct sock *), or
bpf_skc_to_tcp(struct inet_sock *), ...
are all valid. At runtime, the helper will check whether pointer cast is legal or not. Please see Patch #5 for details.
Since btf_id's for both arguments and return value are known at build time, the btf_id's are pre-computed once vmlinux btf becomes valid. Jiri's "adding d_path helper" patch set
https://lore.kernel.org/bpf/[email protected]/T/
provides a way to pre-compute btf id during vmlinux build time. This can be applied here as well. A followup patch can convert to build time btf id computation after Jiri's patch landed.
Changelogs:
v1 -> v2:
- guard init_sock_cast_types() defination properly with CONFIG_NET (Martin)
- reuse the btf_ids, computed for new helper argument, for return values (Martin)
- using BTF_TYPE_EMIT to express intent of btf type generation (Andrii)
- abstract out common net macros into bpf_tracing_net.h (Andrii)
Yonghong Song (15):
bpf: add bpf_seq_afinfo in tcp_iter_state
net: bpf: implement bpf iterator for tcp
bpf: support 'X' in bpf_seq_printf() helper
bpf: allow tracing programs to use bpf_jiffies64() helper
bpf: add bpf_skc_to_tcp6_sock() helper
bpf: add bpf_skc_to_{tcp,tcp_timewait,tcp_request}_sock() helpers
bpf: add bpf_seq_afinfo in udp_iter_state
net: bpf: implement bpf iterator for udp
bpf: add bpf_skc_to_udp6_sock() helper
bpf/selftests: move newer bpf_iter_* type redefining to a new header
file
tools/bpf: refactor some net macros to libbpf bpf_tracing_net.h tools/libbpf: add more common macros to bpf_tracing_net.h tools/bpf: selftests: implement sample tcp/tcp6 bpf_iter programs tools/bpf: add udp4/udp6 bpf iterator
bpf/selftests: add tcp/udp iterator programs to selftests
include/linux/bpf.h | 16 ++ include/net/tcp.h | 1 + include/net/udp.h | 1 + include/uapi/linux/bpf.h | 37 ++- kernel/bpf/btf.c | 1 + kernel/bpf/verifier.c | 43 ++- kernel/trace/bpf_trace.c | 15 +- net/core/filter.c | 153 +++++++++++ net/ipv4/tcp_ipv4.c | 153 ++++++++++- net/ipv4/udp.c | 144 +++++++++- scripts/bpf_helpers_doc.py | 10 + tools/include/uapi/linux/bpf.h | 37 ++- tools/lib/bpf/Makefile | 1 + tools/lib/bpf/bpf_tracing_net.h | 51 ++++ .../selftests/bpf/prog_tests/bpf_iter.c | 68 +++++ tools/testing/selftests/bpf/progs/bpf_iter.h | 80 ++++++ .../selftests/bpf/progs/bpf_iter_bpf_map.c | 18 +- .../selftests/bpf/progs/bpf_iter_ipv6_route.c | 25 +- .../selftests/bpf/progs/bpf_iter_netlink.c | 22 +- .../selftests/bpf/progs/bpf_iter_task.c | 18 +- .../selftests/bpf/progs/bpf_iter_task_file.c | 20 +- .../selftests/bpf/progs/bpf_iter_tcp4.c | 235 ++++++++++++++++ .../selftests/bpf/progs/bpf_iter_tcp6.c | 250 ++++++++++++++++++ .../selftests/bpf/progs/bpf_iter_test_kern3.c | 17 +- .../selftests/bpf/progs/bpf_iter_test_kern4.c | 17 +- .../bpf/progs/bpf_iter_test_kern_common.h | 18 +- .../selftests/bpf/progs/bpf_iter_udp4.c | 71 +++++ .../selftests/bpf/progs/bpf_iter_udp6.c | 79 ++++++ 28 files changed, 1431 insertions(+), 170 deletions(-) create mode 100644 tools/lib/bpf/bpf_tracing_net.h create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter.h create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_tcp4.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_tcp6.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_udp4.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_udp6.c
Signature · 1 line
2.24.1
PATCH v2 1/15 bpf: add bpf_seq_afinfo in tcp_iter_state
A new field bpf_seq_afinfo is added to tcp_iter_state to provide bpf tcp iterator afinfo. There are two reasons on why we did this.
First, the current way to get afinfo from PDE_DATA does not work for bpf iterator as its seq_file inode does not conform to /proc/net/{tcp,tcp6} inode structures. More specifically, anonymous bpf iterator will use an anonymous inode which is shared in the system and we cannot change inode private data structure at all.
Second, bpf iterator for tcp/tcp6 wants to traverse all tcp and tcp6 sockets in one pass and bpf program can control whether they want to skip one sk_family or not. Having a different afinfo with family AF_UNSPEC make it easier to understand in the code.
This patch does not change /proc/net/{tcp,tcp6} behavior as the bpf_seq_afinfo will be NULL for these two proc files.
--- include/net/tcp.h | 1 + net/ipv4/tcp_ipv4.c | 30 ++++++++++++++++++++++++------ 2 files changed, 25 insertions(+), 6 deletions(-)
include/net/tcp.h +1 -0
@@ -1935,6 +1935,7 @@ struct tcp_iter_state {struct seq_net_private p;enum tcp_seq_states state;struct sock *syn_wait_sk; +struct tcp_seq_afinfo *bpf_seq_afinfo;int bucket, offset, sbucket, num;loff_t last_pos; };
net/ipv4/tcp_ipv4.c +24 -6
@@ -2211,13 +2211,18 @@ EXPORT_SYMBOL(tcp_v4_destroy_sock);*/ static void *listening_get_next (struct seq_file *seq,void *cur) { -struct tcp_seq_afinfo *afinfo =PDE_DATA (file_inode (seq->file)); +struct tcp_seq_afinfo *afinfo;struct tcp_iter_state *st = seq->private;struct net *net =seq_file_net (seq);struct inet_listen_hashbucket *ilb;struct hlist_nulls_node *node;struct sock *sk = cur; +if (st->bpf_seq_afinfo) + afinfo = st->bpf_seq_afinfo; +else + afinfo =PDE_DATA (file_inode (seq->file)); +if (!sk) { get_head: ilb = &tcp_hashinfo.listening_hash[st->bucket];
@@ -2235,7 +2240,8 @@ static void *listening_get_next(struct seq_file *seq, void *cur)sk_nulls_for_each_from (sk, node) {if (!net_eq (sock_net (sk), net))continue ; -if (sk->sk_family == afinfo->family) +if (afinfo->family ==AF_UNSPEC || + sk->sk_family == afinfo->family)return sk; }spin_unlock (&ilb->lock);
@@ -2272,11 +2278,16 @@ static inline bool empty_bucket(const struct tcp_iter_state *st)*/ static void *established_get_first (struct seq_file *seq) { -struct tcp_seq_afinfo *afinfo =PDE_DATA (file_inode (seq->file)); +struct tcp_seq_afinfo *afinfo;struct tcp_iter_state *st = seq->private;struct net *net =seq_file_net (seq);void *rc =NULL ; +if (st->bpf_seq_afinfo) + afinfo = st->bpf_seq_afinfo; +else + afinfo =PDE_DATA (file_inode (seq->file)); + st->offset =0 ;for (; st->bucket <= tcp_hashinfo.ehash_mask; ++st->bucket) {struct sock *sk;
@@ -2289,7 +2300,8 @@ static void *established_get_first(struct seq_file *seq)spin_lock_bh (lock);sk_nulls_for_each (sk, node, &tcp_hashinfo.ehash[st->bucket].chain) { -if (sk->sk_family != afinfo->family || +if ((afinfo->family !=AF_UNSPEC && + sk->sk_family != afinfo->family) || !net_eq (sock_net (sk), net)) {continue ; }
@@ -2304,19 +2316,25 @@ static void *established_get_first(struct seq_file *seq)static void *established_get_next (struct seq_file *seq,void *cur) { -struct tcp_seq_afinfo *afinfo =PDE_DATA (file_inode (seq->file)); +struct tcp_seq_afinfo *afinfo;struct sock *sk = cur;struct hlist_nulls_node *node;struct tcp_iter_state *st = seq->private;struct net *net =seq_file_net (seq); +if (st->bpf_seq_afinfo) + afinfo = st->bpf_seq_afinfo; +else + afinfo =PDE_DATA (file_inode (seq->file)); + ++st->num; ++st->offset; sk =sk_nulls_next (sk);sk_nulls_for_each_from (sk, node) { -if (sk->sk_family == afinfo->family && +if ((afinfo->family ==AF_UNSPEC || + sk->sk_family == afinfo->family) &&net_eq (sock_net (sk), net))return sk; }
Signature · 1 line
2.24.1
PATCH v2 2/15 net: bpf: implement bpf iterator for tcp
In reply to Yonghong SongThe bpf iterator for tcp is implemented. Both tcp4 and tcp6 sockets will be traversed. It is up to bpf program to filter for tcp4 or tcp6 only, or both families of sockets.
--- net/ipv4/tcp_ipv4.c | 123 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 123 insertions(+)
net/ipv4/tcp_ipv4.c +123 -0
@@ -2613,6 +2613,74 @@ static int tcp4_seq_show(struct seq_file *seq, void *v)return 0 ; } +#ifdef CONFIG_BPF_SYSCALL +struct bpf_iter__tcp { + __bpf_md_ptr(struct bpf_iter_meta *, meta); + __bpf_md_ptr(struct sock_common *, sk_common); +uid_t uid__aligned(8) ; +}; + +static int tcp_prog_seq_show (struct bpf_prog *prog,struct bpf_iter_meta *meta, +struct sock_common *sk_common,uid_t uid) +{ +struct bpf_iter__tcp ctx; + + meta->seq_num--;/* skip SEQ_START_TOKEN */ + ctx.meta = meta; + ctx.sk_common = sk_common; + ctx.uid = uid; +return bpf_iter_run_prog (prog, &ctx); +} + +static int bpf_iter_tcp_seq_show (struct seq_file *seq,void *v) +{ +struct bpf_iter_meta meta; +struct bpf_prog *prog; +struct sock *sk = v; +uid_t uid; + +if (v ==SEQ_START_TOKEN ) +return 0 ; + +if (sk->sk_state ==TCP_TIME_WAIT ) { + uid =0 ; + }else if (sk->sk_state ==TCP_NEW_SYN_RECV ) { +const struct request_sock *req = v; + + uid =from_kuid_munged (seq_user_ns (seq), +sock_i_uid (req->rsk_listener)); + }else { + uid =from_kuid_munged (seq_user_ns (seq),sock_i_uid (sk)); + } + + meta.seq = seq; + prog =bpf_iter_get_info (&meta, false); +return tcp_prog_seq_show (prog, &meta, v, uid); +} + +static void bpf_iter_tcp_seq_stop (struct seq_file *seq,void *v) +{ +struct bpf_iter_meta meta; +struct bpf_prog *prog; + +if (!v) { + meta.seq = seq; + prog =bpf_iter_get_info (&meta, true); +if (prog) + (void )tcp_prog_seq_show (prog, &meta, v,0 ); + } + +tcp_seq_stop (seq, v); +} + +static const struct seq_operations bpf_iter_tcp_seq_ops = { + .show = bpf_iter_tcp_seq_show, + .start = tcp_seq_start, + .next = tcp_seq_next, + .stop = bpf_iter_tcp_seq_stop, +}; +#endif +static const struct seq_operations tcp4_seq_ops = { .show = tcp4_seq_show, .start = tcp_seq_start,
@@ -2844,8 +2912,63 @@ static struct pernet_operations __net_initdata tcp_sk_ops = { .exit_batch = tcp_sk_exit_batch, }; +#if defined(CONFIG_BPF_SYSCALL ) && defined(CONFIG_PROC_FS ) +DEFINE_BPF_ITER_FUNC(tcp, struct bpf_iter_meta *meta, + struct sock_common *sk_common, uid_t uid) + +static int bpf_iter_init_tcp (void *priv_data) +{ +struct tcp_iter_state *st = priv_data; +struct tcp_seq_afinfo *afinfo; +int ret; + + afinfo =kmalloc (sizeof (*afinfo),GFP_USER | __GFP_NOWARN); +if (!afinfo) +return -ENOMEM ; + + afinfo->family =AF_UNSPEC ; + st->bpf_seq_afinfo = afinfo; + ret =bpf_iter_init_seq_net (priv_data); +if (ret) +kfree (afinfo); +return ret; +} + +static void bpf_iter_fini_tcp (void *priv_data) +{ +struct tcp_iter_state *st = priv_data; + +kfree (st->bpf_seq_afinfo); +bpf_iter_fini_seq_net (priv_data); +} + +static const struct bpf_iter_reg tcp_reg_info = { + .target ="tcp" , + .seq_ops = &bpf_iter_tcp_seq_ops, + .init_seq_private = bpf_iter_init_tcp, + .fini_seq_private = bpf_iter_fini_tcp, + .seq_priv_size =sizeof (struct tcp_iter_state ), + .ctx_arg_info_size =1 , + .ctx_arg_info = { + { offsetof(struct bpf_iter__tcp , sk_common), +PTR_TO_BTF_ID_OR_NULL }, + }, +}; + +static void __init bpf_iter_register (void ) +{ +if (bpf_iter_reg_target (&tcp_reg_info)) +pr_warn ("Warning: could not register bpf iterator tcp\n" ); +} + +#endif +void __init tcp_v4_init (void ) {if (register_pernet_subsys (&tcp_sk_ops))panic ("Failed to create the TCP control socket.\n" ); + +#if defined(CONFIG_BPF_SYSCALL ) && defined(CONFIG_PROC_FS ) +bpf_iter_register (); +#endif }
Signature · 1 line
2.24.1
Re: [PATCH bpf-next v2 02/15] net: bpf: implement bpf iterator for tcp
Quoted 3 lines · Yonghong Song
The bpf iterator for tcp is implemented. Both tcp4 and tcp6 sockets will be traversed. It is up to bpf program to filter for tcp4 or tcp6 only, or both families of sockets.
PATCH v2 3/15 bpf: support 'X' in bpf_seq_printf() helper
In reply to Yonghong Song'X' tells kernel to print hex with upper case letters. /proc/net/tcp{4,6} seq_file show() used this, and supports it in bpf_seq_printf() helper too.
--- kernel/trace/bpf_trace.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/trace/bpf_trace.c +2 -1
@@ -681,7 +681,8 @@ BPF_CALL_5(bpf_seq_printf, struct seq_file *, m, char *, fmt, u32, fmt_size, }if (fmt[i] !='i' && fmt[i] !='d' && - fmt[i] !='u' && fmt[i] !='x' ) { + fmt[i] !='u' && fmt[i] !='x' && + fmt[i] !='X' ) { err = -EINVAL ; goto out; }
Signature · 1 line
2.24.1
PATCH v2 4/15 bpf: allow tracing programs to use bpf_jiffies64() helper
In reply to Yonghong Song/proc/net/tcp{4,6} uses jiffies for various computations. Let us add bpf_jiffies64() helper to tracing program so bpf_iter and other programs can use it.
--- kernel/trace/bpf_trace.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/trace/bpf_trace.c +2 -0
@@ -1135,6 +1135,8 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_ringbuf_discard_proto;case BPF_FUNC_ringbuf_query:return &bpf_ringbuf_query_proto; +case BPF_FUNC_jiffies64: +return &bpf_jiffies64_proto;default :return NULL ; }
Signature · 1 line
2.24.1
PATCH v2 5/15 bpf: add bpf_skc_to_tcp6_sock() helper
In reply to Yonghong SongThe helper is used in tracing programs to cast a socket pointer to a tcp6_sock pointer.
The return value could be NULL if the casting is illegal.
A new helper return type RET_PTR_TO_BTF_ID_OR_NULL is added so the verifier is able to deduce proper return types for the helper.
Different from the previous BTF_ID based helpers,
the bpf_skc_to_tcp6_sock() argument can be several possible btf_ids. More specifically, all possible socket data structures with sock_common appearing in the first in the memory layout. This patch only added socket types related to tcp and udp.
All possible argument btf_id and return value btf_id for helper bpf_skc_to_tcp6_sock() are pre-calculcated and cached. In the future, it is even possible to precompute these btf_id's at kernel build time.
--- include/linux/bpf.h | 12 +++++ include/uapi/linux/bpf.h | 9 +++- kernel/bpf/btf.c | 1 + kernel/bpf/verifier.c | 43 +++++++++++++----- kernel/trace/bpf_trace.c | 2 + net/core/filter.c | 80 ++++++++++++++++++++++++++++++++++ scripts/bpf_helpers_doc.py | 2 + tools/include/uapi/linux/bpf.h | 9 +++- 8 files changed, 145 insertions(+), 13 deletions(-)
include/linux/bpf.h +12 -0
@@ -261,6 +261,7 @@ enum bpf_return_type {RET_PTR_TO_TCP_SOCK_OR_NULL ,/* returns a pointer to a tcp_sock or NULL */ RET_PTR_TO_SOCK_COMMON_OR_NULL ,/* returns a pointer to a sock_common or NULL */ RET_PTR_TO_ALLOC_MEM_OR_NULL ,/* returns a pointer to dynamically allocated memory or NULL */ + RET_PTR_TO_BTF_ID_OR_NULL, /* returns a pointer to a btf_id or NULL */ };/* eBPF function prototype used by verifier to allow BPF_CALLs from eBPF programs
@@ -283,6 +284,12 @@ struct bpf_func_proto {enum bpf_arg_type arg_type[5 ]; };int *btf_id;/* BTF ids of arguments */ +bool (*check_btf_id)(u32 btf_id,u32 arg);/* if the argument btf_id is +* valid. Often used if more +* than one btf id is permitted +* for this argument. +*/ +int *ret_btf_id;/* return value btf_id */ };/* bpf_context is intentionally undefined structure. Pointer to bpf_context is
@@ -1629,6 +1636,7 @@ extern const struct bpf_func_proto bpf_ringbuf_reserve_proto;extern const struct bpf_func_proto bpf_ringbuf_submit_proto;extern const struct bpf_func_proto bpf_ringbuf_discard_proto;extern const struct bpf_func_proto bpf_ringbuf_query_proto; +extern const struct bpf_func_proto bpf_skc_to_tcp6_sock_proto;const struct bpf_func_proto *bpf_tracing_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog);
@@ -1652,6 +1660,7 @@ u32 bpf_sock_convert_ctx_access(enum bpf_access_type type,struct bpf_insn *insn_buf, struct bpf_prog *prog, u32 *target_size); +void init_sock_cast_types (struct btf *btf);#else static inline bool bpf_sock_common_is_valid_access(int off, int size, enum bpf_access_type type,
@@ -1673,6 +1682,9 @@ static inline u32 bpf_sock_convert_ctx_access(enum bpf_access_type type, {return 0 ; } +static inline void init_sock_cast_types (struct btf *btf) +{ +}#endif #ifdef CONFIG_INET
include/uapi/linux/bpf.h +8 -1
@@ -3252,6 +3252,12 @@ union bpf_attr {* case of **BPF_CSUM_LEVEL_QUERY**, the current skb->csum_level * is returned or the error code -EACCES in case the skb is not * subject to CHECKSUM_UNNECESSARY. +* +* struct tcp6_sock *bpf_skc_to_tcp6_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *tcp6_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \
@@ -3389,7 +3395,8 @@ union bpf_attr {FN (ringbuf_submit), \FN (ringbuf_discard), \FN (ringbuf_query), \ -FN (csum_level), +FN (csum_level), \ +FN (skc_to_tcp6_sock),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
kernel/bpf/btf.c +1 -0
@@ -3645,6 +3645,7 @@ struct btf *btf_parse_vmlinux(void) }bpf_struct_ops_init (btf, log); +init_sock_cast_types (btf);btf_verifier_env_free (env);refcount_set (&btf->refcnt,1 );
kernel/bpf/verifier.c +32 -11
76 lines not on this page · raw message
kernel/trace/bpf_trace.c +2 -0
@@ -1515,6 +1515,8 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_skb_output_proto;case BPF_FUNC_xdp_output:return &bpf_xdp_output_proto; +case BPF_FUNC_skc_to_tcp6_sock: +return &bpf_skc_to_tcp6_sock_proto;#endif case BPF_FUNC_seq_printf: return prog->expected_attach_type == BPF_TRACE_ITER ?
net/core/filter.c +80 -0
89 lines not on this page · raw message
scripts/bpf_helpers_doc.py +2 -0
@@ -421,6 +421,7 @@ class PrinterHelpers(Printer): 'struct sockaddr', 'struct tcphdr', 'struct seq_file', + 'struct tcp6_sock', 'struct __sk_buff', 'struct sk_msg_md',
@@ -458,6 +459,7 @@ class PrinterHelpers(Printer): 'struct sockaddr', 'struct tcphdr', 'struct seq_file', + 'struct tcp6_sock', } mapped_types = { 'u8': '__u8',
tools/include/uapi/linux/bpf.h +8 -1
@@ -3252,6 +3252,12 @@ union bpf_attr {* case of **BPF_CSUM_LEVEL_QUERY**, the current skb->csum_level * is returned or the error code -EACCES in case the skb is not * subject to CHECKSUM_UNNECESSARY. +* +* struct tcp6_sock *bpf_skc_to_tcp6_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *tcp6_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \
@@ -3389,7 +3395,8 @@ union bpf_attr {FN (ringbuf_submit), \FN (ringbuf_discard), \FN (ringbuf_query), \ -FN (csum_level), +FN (csum_level), \ +FN (skc_to_tcp6_sock),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
Signature · 1 line
2.24.1
Re: [PATCH bpf-next v2 05/15] bpf: add bpf_skc_to_tcp6_sock() helper
Quoted 18 lines · Yonghong Song
The helper is used in tracing programs to cast a socket pointer to a
tcp6_sockpointer.
The return value could be NULL if the casting is illegal.A new helper return type
RET_PTR_TO_BTF_ID_OR_NULLis added so the verifier is able to deduce proper return types for the helper.Different from the previous
BTF_IDbased helpers,
thebpf_skc_to_tcp6_sock()argument can be several possiblebtf_ids. More specifically, all possible socket data structures withsock_commonappearing in the first in the memory layout. This patch only added socket types related to tcp and udp.All possible argument
btf_idand return valuebtf_idfor helperbpf_skc_to_tcp6_sock()are pre-calculcated and cached. In the future, it is even possible to precompute these btf_id's at kernel build time.
[ ... ]
kernel/bpf/verifier.c static int check_helper_call(struct bpf_verifier_env *env, int func_id, int insn · line 4612
Quoted 6 lines
kernel/bpf/verifier.c +1
-1@@ -4600,7 +4609,7 @@ static int check_helper_call(struct bpf_verifier_env *env, int func_id, int insnstruct bpf_reg_state *regs;struct bpf_call_arg_meta meta;bool changes_data; -int i, err; +int i, err, ret_btf_id;
Nit.
Try to keep the rev xmas tree.
or just move "int ret_btf_id;" to the latter "else if (fn->ret_type...)"
Quoted 90 lines
kernel/bpf/verifier.c +16
-4/* find function prototype */ if (func_id <0 || func_id >= __BPF_FUNC_MAX_ID) {@@ -4644,10 +4653,12 @@ static int check_helper_call(struct bpf_verifier_env *env, int func_id, int insn meta.func_id = func_id;/* check args */ for (i =0 ; i <5 ; i++) { - err =btf_resolve_helper_id (&env->log, fn, i); -if (err >0 ) - meta.btf_id = err; - err =check_func_arg (env,BPF_REG_1 + i, fn->arg_type[i], &meta); +if (!fn->check_btf_id) { + err =btf_resolve_helper_id (&env->log, fn, i); +if (err >0 ) + meta.btf_id = err; + } + err =check_func_arg (env, i, &meta, fn);if (err)return err; }@@ -4750,6 +4761,16 @@ static int check_helper_call(struct bpf_verifier_env *env, int func_id, int insn regs[BPF_REG_0 ].type =PTR_TO_MEM_OR_NULL ; regs[BPF_REG_0 ].id = ++env->id_gen; regs[BPF_REG_0 ].mem_size = meta.mem_size; + }else if (fn->ret_type ==RET_PTR_TO_BTF_ID_OR_NULL ) { +mark_reg_known_zero (env, regs,BPF_REG_0 ); + regs[BPF_REG_0 ].type =PTR_TO_BTF_ID_OR_NULL ; + ret_btf_id = *fn->ret_btf_id; +if (ret_btf_id ==0 ) { +verbose (env,"invalid return type %d of func %s#%d\n" , + fn->ret_type,func_id_name (func_id), func_id); +return -EINVAL ; + } + regs[BPF_REG_0 ].btf_id = ret_btf_id; }else {verbose (env,"unknown return type %d of func %s#%d\n" , fn->ret_type,func_id_name (func_id), func_id);kernel/trace/bpf_trace.c +2
-0@@ -1515,6 +1515,8 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_skb_output_proto;case BPF_FUNC_xdp_output:return &bpf_xdp_output_proto; +case BPF_FUNC_skc_to_tcp6_sock: +return &bpf_skc_to_tcp6_sock_proto;#endif case BPF_FUNC_seq_printf: return prog->expected_attach_type == BPF_TRACE_ITER ?net/core/filter.c +25
-0@@ -47,6 +47,7 @@#include <linux/seccomp.h> #include <linux/if_vlan.h> #include <linux/bpf.h> +#include <linux/btf.h> #include <net/sch_generic.h> #include <net/cls_cgroup.h> #include <net/dst_metadata.h> @@ -9191,3 +9192,82 @@ void bpf_prog_change_xdp(struct bpf_prog *prev_prog, struct bpf_prog *prog) {bpf_dispatcher_change_prog (BPF_DISPATCHER_PTR (xdp), prev_prog, prog); } + +/* Define a list of socket types which can be the argument for +* skc_to_*_sock() helpers. All these sockets should have +* sock_common as the first argument in its memory layout. +*/ +#define BPF_SOCK_CAST_TYPES \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_INET_CONN_SOCK, "inet_connection_sock") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_INET_REQ_SOCK, "inet_request_sock") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_INET_SOCK, "inet_sock") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_INET_TW_SOCK, "inet_timewait_sock") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_REQ_SOCK, "request_sock") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_SOCK, "sock") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_SOCK_COMMON, "sock_common") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_TCP_SOCK, "tcp_sock") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_TCP_REQ_SOCK, "tcp_request_sock") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_TCP_TW_SOCK, "tcp_timewait_sock") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_TCP6_SOCK, "tcp6_sock") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_UDP_SOCK, "udp_sock") \ + BPF_SOCK_CAST_TYPE(SOCK_CAST_UDP6_SOCK, "udp6_sock") + +enum { +#define BPF_SOCK_CAST_TYPE(name, str) name, +BPF_SOCK_CAST_TYPES +MAX_SOCK_CAST_TYPE,
net/core/filter.c void bpf_prog_change_xdp(struct bpf_prog *prev_prog, struct bpf_prog *prog) · line 9228
Quoted 10 lines
net/core/filter.c +10
-0+#undef BPF_SOCK_CAST_TYPE +}; + +static const char *sock_cast_types[] = { +#define BPF_SOCK_CAST_TYPE(name, str) str, +BPF_SOCK_CAST_TYPES +#undef BPF_SOCK_CAST_TYPE +}; + +static int sock_cast_btf_ids[MAX_SOCK_CAST_TYPE ];
Thanks for doing this.
I think they can be reused outside of casting in the future, e.g. bpf_tcp_ca.c. If you respin, do you mind to remove the "_cast_" and "_CAST_". May be naming it to BTF_SOCK_TYPE_xxx, btf_sock_ids?
[ ... ]
net/core/filter.c void bpf_prog_change_xdp(struct bpf_prog *prev_prog, struct bpf_prog *prog) · line 9260
Quoted 8 lines
net/core/filter.c +8
-0+BPF_CALL_1(bpf_skc_to_tcp6_sock, struct sock *, sk) +{ +/* tcp6_sock type is not generated in dwarf and hence btf, +* trigger an explicit type generation here. +*/ +BTF_TYPE_EMIT (struct tcp6_sock ); +if (sk_fullsock (sk) && sk->sk_protocol ==IPPROTO_TCP && + sk->sk_family ==AF_INET6 )
IS_ENABLED(CONFIG_IPV6) just came to my mind. I think it should be fine without checking it to keep the #if macro to minimal. No sk should be in AF_INET6 in that case.
net/core/filter.c void bpf_prog_change_xdp(struct bpf_prog *prev_prog, struct bpf_prog *prog) · line 9268
Quoted 8 lines
net/core/filter.c +8
-0+return (unsigned long )sk; + +return (unsigned long )NULL ; +} + +> +const struct bpf_func_proto bpf_skc_to_tcp6_sock_proto = { + .func = bpf_skc_to_tcp6_sock, + .gpl_only = true,
s/true/false/ With that,
Quoted 5 lines
net/core/filter.c +5
-0+ .ret_type = RET_PTR_TO_BTF_ID_OR_NULL, + .arg1_type = ARG_PTR_TO_BTF_ID, + .check_btf_id = check_arg_btf_id, + .ret_btf_id = &sock_cast_btf_ids[SOCK_CAST_TCP6_SOCK], +};
PATCH v2 6/15 bpf: add bpf_skc_to_{tcp,tcp_timewait,tcp_request}_sock() helpers
In reply to Yonghong SongThree more helpers are added to cast a sock_common pointer to an tcp_sock, tcp_timewait_sock or a tcp_request_sock for tracing programs.
--- include/linux/bpf.h | 3 ++ include/uapi/linux/bpf.h | 23 ++++++++++++++- kernel/trace/bpf_trace.c | 6 ++++ net/core/filter.c | 51 ++++++++++++++++++++++++++++++++++ scripts/bpf_helpers_doc.py | 6 ++++ tools/include/uapi/linux/bpf.h | 23 ++++++++++++++- 6 files changed, 110 insertions(+), 2 deletions(-)
include/linux/bpf.h +3 -0
@@ -1637,6 +1637,9 @@ extern const struct bpf_func_proto bpf_ringbuf_submit_proto;extern const struct bpf_func_proto bpf_ringbuf_discard_proto;extern const struct bpf_func_proto bpf_ringbuf_query_proto;extern const struct bpf_func_proto bpf_skc_to_tcp6_sock_proto; +extern const struct bpf_func_proto bpf_skc_to_tcp_sock_proto; +extern const struct bpf_func_proto bpf_skc_to_tcp_timewait_sock_proto; +extern const struct bpf_func_proto bpf_skc_to_tcp_request_sock_proto;const struct bpf_func_proto *bpf_tracing_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog);
include/uapi/linux/bpf.h +22 -1
@@ -3258,6 +3258,24 @@ union bpf_attr {* Dynamically cast a *sk* pointer to a *tcp6_sock* pointer. * Return * *sk* if casting is valid, or NULL otherwise. +* +* struct tcp_sock *bpf_skc_to_tcp_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *tcp_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. +* +* struct tcp_timewait_sock *bpf_skc_to_tcp_timewait_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *tcp_timewait_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. +* +* struct tcp_request_sock *bpf_skc_to_tcp_request_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *tcp_request_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \
@@ -3396,7 +3414,10 @@ union bpf_attr {FN (ringbuf_discard), \FN (ringbuf_query), \FN (csum_level), \ -FN (skc_to_tcp6_sock), +FN (skc_to_tcp6_sock), \ +FN (skc_to_tcp_sock), \ +FN (skc_to_tcp_timewait_sock), \ +FN (skc_to_tcp_request_sock),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
kernel/trace/bpf_trace.c +6 -0
@@ -1517,6 +1517,12 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_xdp_output_proto;case BPF_FUNC_skc_to_tcp6_sock:return &bpf_skc_to_tcp6_sock_proto; +case BPF_FUNC_skc_to_tcp_sock: +return &bpf_skc_to_tcp_sock_proto; +case BPF_FUNC_skc_to_tcp_timewait_sock: +return &bpf_skc_to_tcp_timewait_sock_proto; +case BPF_FUNC_skc_to_tcp_request_sock: +return &bpf_skc_to_tcp_request_sock_proto;#endif case BPF_FUNC_seq_printf: return prog->expected_attach_type == BPF_TRACE_ITER ?
net/core/filter.c +51 -0
54 lines not on this page · raw message
scripts/bpf_helpers_doc.py +6 -0
@@ -422,6 +422,9 @@ class PrinterHelpers(Printer): 'struct tcphdr', 'struct seq_file', 'struct tcp6_sock', + 'struct tcp_sock', + 'struct tcp_timewait_sock', + 'struct tcp_request_sock', 'struct __sk_buff', 'struct sk_msg_md',
@@ -460,6 +463,9 @@ class PrinterHelpers(Printer): 'struct tcphdr', 'struct seq_file', 'struct tcp6_sock', + 'struct tcp_sock', + 'struct tcp_timewait_sock', + 'struct tcp_request_sock', } mapped_types = { 'u8': '__u8',
tools/include/uapi/linux/bpf.h +22 -1
@@ -3258,6 +3258,24 @@ union bpf_attr {* Dynamically cast a *sk* pointer to a *tcp6_sock* pointer. * Return * *sk* if casting is valid, or NULL otherwise. +* +* struct tcp_sock *bpf_skc_to_tcp_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *tcp_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. +* +* struct tcp_timewait_sock *bpf_skc_to_tcp_timewait_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *tcp_timewait_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. +* +* struct tcp_request_sock *bpf_skc_to_tcp_request_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *tcp_request_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \
@@ -3396,7 +3414,10 @@ union bpf_attr {FN (ringbuf_discard), \FN (ringbuf_query), \FN (csum_level), \ -FN (skc_to_tcp6_sock), +FN (skc_to_tcp6_sock), \ +FN (skc_to_tcp_sock), \ +FN (skc_to_tcp_timewait_sock), \ +FN (skc_to_tcp_request_sock),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
Signature · 1 line
2.24.1
Re: [PATCH bpf-next v2 06/15] bpf: add bpf_skc_to_{tcp,tcp_timewait,tcp_request}_sock() helpers
Quoted 96 lines · Yonghong Song
Three more helpers are added to cast a
sock_commonpointer to antcp_sock,tcp_timewait_sockor atcp_request_sockfor tracing programs.Signed-off-by Yonghong Song <[email protected]>--- include/linux/bpf.h | 3 ++ include/uapi/linux/bpf.h | 23 ++++++++++++++- kernel/trace/bpf_trace.c | 6 ++++ net/core/filter.c | 51 ++++++++++++++++++++++++++++++++++ scripts/bpf_helpers_doc.py | 6 ++++ tools/include/uapi/linux/bpf.h | 23 ++++++++++++++- 6 files changed, 110 insertions(+), 2 deletions(-)include/linux/bpf.h +3
-0@@ -1637,6 +1637,9 @@ extern const struct bpf_func_proto bpf_ringbuf_submit_proto;extern const struct bpf_func_proto bpf_ringbuf_discard_proto;extern const struct bpf_func_proto bpf_ringbuf_query_proto;extern const struct bpf_func_proto bpf_skc_to_tcp6_sock_proto; +extern const struct bpf_func_proto bpf_skc_to_tcp_sock_proto; +extern const struct bpf_func_proto bpf_skc_to_tcp_timewait_sock_proto; +extern const struct bpf_func_proto bpf_skc_to_tcp_request_sock_proto;const struct bpf_func_proto *bpf_tracing_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog);include/uapi/linux/bpf.h +22
-1@@ -3258,6 +3258,24 @@ union bpf_attr {* Dynamically cast a *sk* pointer to a *tcp6_sock* pointer. * Return * *sk* if casting is valid, or NULL otherwise. +* +* struct tcp_sock *bpf_skc_to_tcp_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *tcp_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. +* +* struct tcp_timewait_sock *bpf_skc_to_tcp_timewait_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *tcp_timewait_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. +* +* struct tcp_request_sock *bpf_skc_to_tcp_request_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *tcp_request_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \@@ -3396,7 +3414,10 @@ union bpf_attr {FN (ringbuf_discard), \FN (ringbuf_query), \FN (csum_level), \ -FN (skc_to_tcp6_sock), +FN (skc_to_tcp6_sock), \ +FN (skc_to_tcp_sock), \ +FN (skc_to_tcp_timewait_sock), \ +FN (skc_to_tcp_request_sock),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call kernel/trace/bpf_trace.c +6
-0@@ -1517,6 +1517,12 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_xdp_output_proto;case BPF_FUNC_skc_to_tcp6_sock:return &bpf_skc_to_tcp6_sock_proto; +case BPF_FUNC_skc_to_tcp_sock: +return &bpf_skc_to_tcp_sock_proto; +case BPF_FUNC_skc_to_tcp_timewait_sock: +return &bpf_skc_to_tcp_timewait_sock_proto; +case BPF_FUNC_skc_to_tcp_request_sock: +return &bpf_skc_to_tcp_request_sock_proto;#endif case BPF_FUNC_seq_printf: return prog->expected_attach_type == BPF_TRACE_ITER ?net/core/filter.c +2
-0@@ -9271,3 +9271,54 @@ const struct bpf_func_proto bpf_skc_to_tcp6_sock_proto = { .check_btf_id = check_arg_btf_id, .ret_btf_id = &sock_cast_btf_ids[SOCK_CAST_TCP6_SOCK], }; + +BPF_CALL_1(bpf_skc_to_tcp_sock, struct sock *, sk)
net/core/filter.c const struct bpf_func_proto bpf_skc_to_tcp6_sock_proto = { · line 9285
Quoted 10 lines
net/core/filter.c +10
-0+{ +if (sk_fullsock (sk) && sk->sk_protocol ==IPPROTO_TCP ) +return (unsigned long )sk; + +return (unsigned long )NULL ; +} + +const struct bpf_func_proto bpf_skc_to_tcp_sock_proto = { + .func = bpf_skc_to_tcp_sock, + .gpl_only = true,
s/true/false/
and for other bpf_func_proto below.
net/core/filter.c const struct bpf_func_proto bpf_skc_to_tcp6_sock_proto = { · line 9294
Quoted 9 lines
net/core/filter.c +9
-0+ .ret_type = RET_PTR_TO_BTF_ID_OR_NULL, + .arg1_type = ARG_PTR_TO_BTF_ID, + .check_btf_id = check_arg_btf_id, + .ret_btf_id = &sock_cast_btf_ids[SOCK_CAST_TCP_SOCK], +}; + +BPF_CALL_1(bpf_skc_to_tcp_timewait_sock, struct sock *, sk) +{ +if (sk->sk_state ==TCP_TIME_WAIT )
Not sure if checking TCP_TIME_WAIT can guarantee a tcp_timewait_sock. dccp seems to have the same state aliased to DCCP_TIME_WAIT. The same goes for the TCP_NEW_SYN_RECV check.
may be checking sk->sk_prot instead?
Quoted 30 lines
net/core/filter.c +30
-0+return (unsigned long )sk; + +return (unsigned long )NULL ; +} + +const struct bpf_func_proto bpf_skc_to_tcp_timewait_sock_proto = { + .func = bpf_skc_to_tcp_timewait_sock, + .gpl_only = true, + .ret_type =RET_PTR_TO_BTF_ID_OR_NULL , + .arg1_type =ARG_PTR_TO_BTF_ID , + .check_btf_id = check_arg_btf_id, + .ret_btf_id = &sock_cast_btf_ids[SOCK_CAST_TCP_TW_SOCK ], +}; + +BPF_CALL_1(bpf_skc_to_tcp_request_sock, struct sock *, sk) +{ +if (sk->sk_state ==TCP_NEW_SYN_RECV ) +return (unsigned long )sk; + +return (unsigned long )NULL ; +} + +const struct bpf_func_proto bpf_skc_to_tcp_request_sock_proto = { + .func = bpf_skc_to_tcp_request_sock, + .gpl_only = true, + .ret_type =RET_PTR_TO_BTF_ID_OR_NULL , + .arg1_type =ARG_PTR_TO_BTF_ID , + .check_btf_id = check_arg_btf_id, + .ret_btf_id = &sock_cast_btf_ids[SOCK_CAST_TCP_REQ_SOCK ], +};
PATCH v2 7/15 bpf: add bpf_seq_afinfo in udp_iter_state
In reply to Yonghong SongSimilar to tcp_iter_state, a new field bpf_seq_afinfo is added to udp_iter_state to provide bpf udp iterator afinfo.
This does not change /proc/net/{udp, udp6} behavior. But it enables bpf iterator to avoid get afinfo from PDE_DATA and iterate through all udp and udp6 sockets in one pass.
--- include/net/udp.h | 1 + net/ipv4/udp.c | 28 +++++++++++++++++++++++----- 2 files changed, 24 insertions(+), 5 deletions(-)
include/net/udp.h +1 -0
@@ -440,6 +440,7 @@ struct udp_seq_afinfo {struct udp_iter_state {struct seq_net_private p;int bucket; +struct udp_seq_afinfo *bpf_seq_afinfo; };void *udp_seq_start (struct seq_file *seq,loff_t *pos);
net/ipv4/udp.c +23 -5
@@ -2826,10 +2826,15 @@ EXPORT_SYMBOL(udp_prot);static struct sock *udp_get_first (struct seq_file *seq,int start) {struct sock *sk; -struct udp_seq_afinfo *afinfo =PDE_DATA (file_inode (seq->file)); +struct udp_seq_afinfo *afinfo;struct udp_iter_state *state = seq->private;struct net *net =seq_file_net (seq); +if (state->bpf_seq_afinfo) + afinfo = state->bpf_seq_afinfo; +else + afinfo =PDE_DATA (file_inode (seq->file)); +for (state->bucket = start; state->bucket <= afinfo->udp_table->mask; ++state->bucket) {struct udp_hslot *hslot = &afinfo->udp_table->hash[state->bucket];
@@ -2841,7 +2846,8 @@ static struct sock *udp_get_first(struct seq_file *seq, int start)sk_for_each (sk, &hslot->head) {if (!net_eq (sock_net (sk), net))continue ; -if (sk->sk_family == afinfo->family) +if (afinfo->family ==AF_UNSPEC || + sk->sk_family == afinfo->family) goto found; }spin_unlock_bh (&hslot->lock);
@@ -2853,13 +2859,20 @@ static struct sock *udp_get_first(struct seq_file *seq, int start)static struct sock *udp_get_next (struct seq_file *seq,struct sock *sk) { -struct udp_seq_afinfo *afinfo =PDE_DATA (file_inode (seq->file)); +struct udp_seq_afinfo *afinfo;struct udp_iter_state *state = seq->private;struct net *net =seq_file_net (seq); +if (state->bpf_seq_afinfo) + afinfo = state->bpf_seq_afinfo; +else + afinfo =PDE_DATA (file_inode (seq->file)); +do { sk =sk_next (sk); - }while (sk && (!net_eq (sock_net (sk), net) || sk->sk_family != afinfo->family)); + }while (sk && (!net_eq (sock_net (sk), net) || + (afinfo->family !=AF_UNSPEC && + sk->sk_family != afinfo->family)));if (!sk) {if (state->bucket <= afinfo->udp_table->mask)
@@ -2904,9 +2917,14 @@ EXPORT_SYMBOL(udp_seq_next);void udp_seq_stop (struct seq_file *seq,void *v) { -struct udp_seq_afinfo *afinfo =PDE_DATA (file_inode (seq->file)); +struct udp_seq_afinfo *afinfo;struct udp_iter_state *state = seq->private; +if (state->bpf_seq_afinfo) + afinfo = state->bpf_seq_afinfo; +else + afinfo =PDE_DATA (file_inode (seq->file)); +if (state->bucket <= afinfo->udp_table->mask)spin_unlock_bh (&afinfo->udp_table->hash[state->bucket].lock); }
Signature · 1 line
2.24.1
Re: [PATCH bpf-next v2 07/15] bpf: add bpf_seq_afinfo in udp_iter_state
Quoted 7 lines · Yonghong Song
Similar to
tcp_iter_state, a new fieldbpf_seq_afinfois added toudp_iter_stateto provide bpf udp iterator afinfo.This does not change
/proc/net/{udp, udp6} behavior. But it enables bpf iterator to avoid get afinfo fromPDE_DATAand iterate through all udp and udp6 sockets in one pass.
PATCH v2 8/15 net: bpf: implement bpf iterator for udp
In reply to Yonghong SongThe bpf iterator for udp is implemented. Both udp4 and udp6 sockets will be traversed. It is up to bpf program to filter for udp4 or udp6 only, or both families of sockets.
--- net/ipv4/udp.c | 116 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+)
net/ipv4/udp.c +116 -0
@@ -2968,6 +2968,67 @@ int udp4_seq_show(struct seq_file *seq, void *v)return 0 ; } +#ifdef CONFIG_BPF_SYSCALL +struct bpf_iter__udp { + __bpf_md_ptr(struct bpf_iter_meta *, meta); + __bpf_md_ptr(struct udp_sock *, udp_sk); +uid_t uid__aligned(8) ; +int bucket__aligned(8) ; +}; + +static int udp_prog_seq_show (struct bpf_prog *prog,struct bpf_iter_meta *meta, +struct udp_sock *udp_sk,uid_t uid,int bucket) +{ +struct bpf_iter__udp ctx; + + meta->seq_num--;/* skip SEQ_START_TOKEN */ + ctx.meta = meta; + ctx.udp_sk = udp_sk; + ctx.uid = uid; + ctx.bucket = bucket; +return bpf_iter_run_prog (prog, &ctx); +} + +static int bpf_iter_udp_seq_show (struct seq_file *seq,void *v) +{ +struct udp_iter_state *state = seq->private; +struct bpf_iter_meta meta; +struct bpf_prog *prog; +struct sock *sk = v; +uid_t uid; + +if (v ==SEQ_START_TOKEN ) +return 0 ; + + uid =from_kuid_munged (seq_user_ns (seq),sock_i_uid (sk)); + meta.seq = seq; + prog =bpf_iter_get_info (&meta, false); +return udp_prog_seq_show (prog, &meta, v, uid, state->bucket); +} + +static void bpf_iter_udp_seq_stop (struct seq_file *seq,void *v) +{ +struct bpf_iter_meta meta; +struct bpf_prog *prog; + +if (!v) { + meta.seq = seq; + prog =bpf_iter_get_info (&meta, true); +if (prog) + (void )udp_prog_seq_show (prog, &meta, v,0 ,0 ); + } + +udp_seq_stop (seq, v); +} + +static const struct seq_operations bpf_iter_udp_seq_ops = { + .start = udp_seq_start, + .next = udp_seq_next, + .stop = bpf_iter_udp_seq_stop, + .show = bpf_iter_udp_seq_show, +}; +#endif +const struct seq_operations udp_seq_ops = { .start = udp_seq_start, .next = udp_seq_next,
@@ -3085,6 +3146,57 @@ static struct pernet_operations __net_initdata udp_sysctl_ops = { .init = udp_sysctl_init, }; +#if defined(CONFIG_BPF_SYSCALL ) && defined(CONFIG_PROC_FS ) +DEFINE_BPF_ITER_FUNC(udp, struct bpf_iter_meta *meta, + struct udp_sock *udp_sk, uid_t uid, int bucket) + +static int bpf_iter_init_udp (void *priv_data) +{ +struct udp_iter_state *st = priv_data; +struct udp_seq_afinfo *afinfo; +int ret; + + afinfo =kmalloc (sizeof (*afinfo),GFP_USER | __GFP_NOWARN); +if (!afinfo) +return -ENOMEM ; + + afinfo->family =AF_UNSPEC ; + afinfo->udp_table = &udp_table; + st->bpf_seq_afinfo = afinfo; + ret =bpf_iter_init_seq_net (priv_data); +if (ret) +kfree (afinfo); +return ret; +} + +static void bpf_iter_fini_udp (void *priv_data) +{ +struct udp_iter_state *st = priv_data; + +kfree (st->bpf_seq_afinfo); +bpf_iter_fini_seq_net (priv_data); +} + +static const struct bpf_iter_reg udp_reg_info = { + .target ="udp" , + .seq_ops = &bpf_iter_udp_seq_ops, + .init_seq_private = bpf_iter_init_udp, + .fini_seq_private = bpf_iter_fini_udp, + .seq_priv_size =sizeof (struct udp_iter_state ), + .ctx_arg_info_size =1 , + .ctx_arg_info = { + { offsetof(struct bpf_iter__udp , udp_sk), +PTR_TO_BTF_ID_OR_NULL }, + }, +}; + +static void __init bpf_iter_register (void ) +{ +if (bpf_iter_reg_target (&udp_reg_info)) +pr_warn ("Warning: could not register bpf iterator udp\n" ); +} +#endif +void __init udp_init (void ) {unsigned long limit;
@@ -3110,4 +3222,8 @@ void __init udp_init(void)if (register_pernet_subsys (&udp_sysctl_ops))panic ("UDP: failed to init sysctl parameters.\n" ); + +#if defined(CONFIG_BPF_SYSCALL ) && defined(CONFIG_PROC_FS ) +bpf_iter_register (); +#endif }
Signature · 1 line
2.24.1
Re: [Potential Spoof] [PATCH bpf-next v2 08/15] net: bpf: implement bpf iterator for udp
Quoted 3 lines · Yonghong Song
The bpf iterator for udp is implemented. Both udp4 and udp6 sockets will be traversed. It is up to bpf program to filter for udp4 or udp6 only, or both families of sockets.
PATCH v2 9/15 bpf: add bpf_skc_to_udp6_sock() helper
In reply to Yonghong SongThe helper is used in tracing programs to cast a socket pointer to a udp6_sock pointer.
The return value could be NULL if the casting is illegal.
--- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 9 ++++++++- kernel/trace/bpf_trace.c | 2 ++ net/core/filter.c | 22 ++++++++++++++++++++++ scripts/bpf_helpers_doc.py | 2 ++ tools/include/uapi/linux/bpf.h | 9 ++++++++- 6 files changed, 43 insertions(+), 2 deletions(-)
include/linux/bpf.h +1 -0
@@ -1640,6 +1640,7 @@ extern const struct bpf_func_proto bpf_skc_to_tcp6_sock_proto;extern const struct bpf_func_proto bpf_skc_to_tcp_sock_proto;extern const struct bpf_func_proto bpf_skc_to_tcp_timewait_sock_proto;extern const struct bpf_func_proto bpf_skc_to_tcp_request_sock_proto; +extern const struct bpf_func_proto bpf_skc_to_udp6_sock_proto;const struct bpf_func_proto *bpf_tracing_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog);
include/uapi/linux/bpf.h +8 -1
@@ -3276,6 +3276,12 @@ union bpf_attr {* Dynamically cast a *sk* pointer to a *tcp_request_sock* pointer. * Return * *sk* if casting is valid, or NULL otherwise. +* +* struct udp6_sock *bpf_skc_to_udp6_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *udp6_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \
@@ -3417,7 +3423,8 @@ union bpf_attr {FN (skc_to_tcp6_sock), \FN (skc_to_tcp_sock), \FN (skc_to_tcp_timewait_sock), \ -FN (skc_to_tcp_request_sock), +FN (skc_to_tcp_request_sock), \ +FN (skc_to_udp6_sock),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
kernel/trace/bpf_trace.c +2 -0
@@ -1523,6 +1523,8 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_skc_to_tcp_timewait_sock_proto;case BPF_FUNC_skc_to_tcp_request_sock:return &bpf_skc_to_tcp_request_sock_proto; +case BPF_FUNC_skc_to_udp6_sock: +return &bpf_skc_to_udp6_sock_proto;#endif case BPF_FUNC_seq_printf: return prog->expected_attach_type == BPF_TRACE_ITER ?
net/core/filter.c +22 -0
@@ -9322,3 +9322,25 @@ const struct bpf_func_proto bpf_skc_to_tcp_request_sock_proto = { .check_btf_id = check_arg_btf_id, .ret_btf_id = &sock_cast_btf_ids[SOCK_CAST_TCP_REQ_SOCK], }; + +BPF_CALL_1(bpf_skc_to_udp6_sock, struct sock *, sk) +{ +/* udp6_sock type is not generated in dwarf and hence btf, +* trigger an explicit type generation here. +*/ +BTF_TYPE_EMIT (struct udp6_sock ); +if (sk_fullsock (sk) && sk->sk_protocol ==IPPROTO_UDP && + sk->sk_family ==AF_INET6 ) +return (unsigned long )sk; + +return (unsigned long )NULL ; +} + +const struct bpf_func_proto bpf_skc_to_udp6_sock_proto = { + .func = bpf_skc_to_udp6_sock, + .gpl_only = true, + .ret_type =RET_PTR_TO_BTF_ID_OR_NULL , + .arg1_type =ARG_PTR_TO_BTF_ID , + .check_btf_id = check_arg_btf_id, + .ret_btf_id = &sock_cast_btf_ids[SOCK_CAST_UDP6_SOCK ], +};
scripts/bpf_helpers_doc.py +2 -0
@@ -425,6 +425,7 @@ class PrinterHelpers(Printer): 'struct tcp_sock', 'struct tcp_timewait_sock', 'struct tcp_request_sock', + 'struct udp6_sock', 'struct __sk_buff', 'struct sk_msg_md',
@@ -466,6 +467,7 @@ class PrinterHelpers(Printer): 'struct tcp_sock', 'struct tcp_timewait_sock', 'struct tcp_request_sock', + 'struct udp6_sock', } mapped_types = { 'u8': '__u8',
tools/include/uapi/linux/bpf.h +8 -1
@@ -3276,6 +3276,12 @@ union bpf_attr {* Dynamically cast a *sk* pointer to a *tcp_request_sock* pointer. * Return * *sk* if casting is valid, or NULL otherwise. +* +* struct udp6_sock *bpf_skc_to_udp6_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *udp6_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \
@@ -3417,7 +3423,8 @@ union bpf_attr {FN (skc_to_tcp6_sock), \FN (skc_to_tcp_sock), \FN (skc_to_tcp_timewait_sock), \ -FN (skc_to_tcp_request_sock), +FN (skc_to_tcp_request_sock), \ +FN (skc_to_udp6_sock),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
Signature · 1 line
2.24.1
Re: [PATCH bpf-next v2 09/15] bpf: add bpf_skc_to_udp6_sock() helper
Quoted 81 lines · Yonghong Song
The helper is used in tracing programs to cast a socket pointer to a
udp6_sockpointer.
The return value could be NULL if the casting is illegal.Signed-off-by Yonghong Song <[email protected]>--- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 9 ++++++++- kernel/trace/bpf_trace.c | 2 ++ net/core/filter.c | 22 ++++++++++++++++++++++ scripts/bpf_helpers_doc.py | 2 ++ tools/include/uapi/linux/bpf.h | 9 ++++++++- 6 files changed, 43 insertions(+), 2 deletions(-)include/linux/bpf.h +1
-0@@ -1640,6 +1640,7 @@ extern const struct bpf_func_proto bpf_skc_to_tcp6_sock_proto;extern const struct bpf_func_proto bpf_skc_to_tcp_sock_proto;extern const struct bpf_func_proto bpf_skc_to_tcp_timewait_sock_proto;extern const struct bpf_func_proto bpf_skc_to_tcp_request_sock_proto; +extern const struct bpf_func_proto bpf_skc_to_udp6_sock_proto;const struct bpf_func_proto *bpf_tracing_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog);include/uapi/linux/bpf.h +8
-1@@ -3276,6 +3276,12 @@ union bpf_attr {* Dynamically cast a *sk* pointer to a *tcp_request_sock* pointer. * Return * *sk* if casting is valid, or NULL otherwise. +* +* struct udp6_sock *bpf_skc_to_udp6_sock(void *sk) +* Description +* Dynamically cast a *sk* pointer to a *udp6_sock* pointer. +* Return +* *sk* if casting is valid, or NULL otherwise. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \@@ -3417,7 +3423,8 @@ union bpf_attr {FN (skc_to_tcp6_sock), \FN (skc_to_tcp_sock), \FN (skc_to_tcp_timewait_sock), \ -FN (skc_to_tcp_request_sock), +FN (skc_to_tcp_request_sock), \ +FN (skc_to_udp6_sock),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call kernel/trace/bpf_trace.c +2
-0@@ -1523,6 +1523,8 @@ tracing_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_skc_to_tcp_timewait_sock_proto;case BPF_FUNC_skc_to_tcp_request_sock:return &bpf_skc_to_tcp_request_sock_proto; +case BPF_FUNC_skc_to_udp6_sock: +return &bpf_skc_to_udp6_sock_proto;#endif case BPF_FUNC_seq_printf: return prog->expected_attach_type == BPF_TRACE_ITER ?net/core/filter.c +7
-0@@ -9322,3 +9322,25 @@ const struct bpf_func_proto bpf_skc_to_tcp_request_sock_proto = { .check_btf_id = check_arg_btf_id, .ret_btf_id = &sock_cast_btf_ids[SOCK_CAST_TCP_REQ_SOCK], }; + +BPF_CALL_1(bpf_skc_to_udp6_sock, struct sock *, sk) +{ +/* udp6_sock type is not generated in dwarf and hence btf, +* trigger an explicit type generation here. +*/ +BTF_TYPE_EMIT (struct udp6_sock );
net/core/filter.c const struct bpf_func_proto bpf_skc_to_tcp_request_sock_proto = { · line 9341
Quoted 10 lines
net/core/filter.c +10
-0+if (sk_fullsock (sk) && sk->sk_protocol ==IPPROTO_UDP && + sk->sk_family ==AF_INET6 ) +return (unsigned long )sk; + +return (unsigned long )NULL ; +} + +const struct bpf_func_proto bpf_skc_to_udp6_sock_proto = { + .func = bpf_skc_to_udp6_sock, + .gpl_only = true,
Same gpl_only change is needed.
PATCH v2 10/15 bpf/selftests: move newer bpf_iter_* type redefining to a new header file
In reply to Yonghong SongCommit b9f4c01f3e0b ("selftest/bpf: Make bpf_iter selftest compilable against old vmlinux.h") and Commit dda18a5c0b75 ("selftests/bpf: Convert bpf_iter_test_kern{3, 4}.c to define own bpf_iter_meta") redefined newly introduced types in bpf programs so the bpf program can still compile properly with old kernels although loading may fail.
Since this patch set introduced new types and the same workaround is needed, so let us move the workaround to a separate header file so they do not clutter bpf programs.
--- tools/testing/selftests/bpf/progs/bpf_iter.h | 49 +++++++++++++++++++ .../selftests/bpf/progs/bpf_iter_bpf_map.c | 18 +------ .../selftests/bpf/progs/bpf_iter_ipv6_route.c | 18 +------ .../selftests/bpf/progs/bpf_iter_netlink.c | 18 +------ .../selftests/bpf/progs/bpf_iter_task.c | 18 +------ .../selftests/bpf/progs/bpf_iter_task_file.c | 20 +------- .../selftests/bpf/progs/bpf_iter_test_kern3.c | 17 +------ .../selftests/bpf/progs/bpf_iter_test_kern4.c | 17 +------ .../bpf/progs/bpf_iter_test_kern_common.h | 18 +------ 9 files changed, 57 insertions(+), 136 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter.h
tools/testing/selftests/bpf/progs/bpf_iter.h new file +49 -0
@@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2020 Facebook */ +/* "undefine" structs in vmlinux.h, because we "override" them below */ +#define bpf_iter_meta bpf_iter_meta___not_used +#define bpf_iter__bpf_map bpf_iter__bpf_map___not_used +#define bpf_iter__ipv6_route bpf_iter__ipv6_route___not_used +#define bpf_iter__netlink bpf_iter__netlink___not_used +#define bpf_iter__task bpf_iter__task___not_used +#define bpf_iter__task_file bpf_iter__task_file___not_used +#include "vmlinux.h" +#undef bpf_iter_meta +#undef bpf_iter__bpf_map +#undef bpf_iter__ipv6_route +#undef bpf_iter__netlink +#undef bpf_iter__task +#undef bpf_iter__task_file + +struct bpf_iter_meta { +struct seq_file *seq; +__u64 session_id; +__u64 seq_num; +} __attribute__((preserve_access_index)); + +struct bpf_iter__ipv6_route { +struct bpf_iter_meta *meta; +struct fib6_info *rt; +} __attribute__((preserve_access_index)); + +struct bpf_iter__netlink { +struct bpf_iter_meta *meta; +struct netlink_sock *sk; +} __attribute__((preserve_access_index)); + +struct bpf_iter__task { +struct bpf_iter_meta *meta; +struct task_struct *task; +} __attribute__((preserve_access_index)); + +struct bpf_iter__task_file { +struct bpf_iter_meta *meta; +struct task_struct *task; +__u32 fd; +struct file *file; +} __attribute__((preserve_access_index)); + +struct bpf_iter__bpf_map { +struct bpf_iter_meta *meta; +struct bpf_map *map; +} __attribute__((preserve_access_index));
tools/testing/selftests/bpf/progs/bpf_iter_bpf_map.c +1 -17
@@ -1,27 +1,11 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ -/* "undefine" structs in vmlinux.h, because we "override" them below */ -#define bpf_iter_meta bpf_iter_meta___not_used -#define bpf_iter__bpf_map bpf_iter__bpf_map___not_used -#include "vmlinux.h" -#undef bpf_iter_meta -#undef bpf_iter__bpf_map +#include "bpf_iter.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_tracing.h> char _license[] SEC("license") = "GPL"; -struct bpf_iter_meta { -struct seq_file *seq; -__u64 session_id; -__u64 seq_num; -} __attribute__((preserve_access_index)); - -struct bpf_iter__bpf_map { -struct bpf_iter_meta *meta; -struct bpf_map *map; -} __attribute__((preserve_access_index)); - SEC("iter/bpf_map")int dump_bpf_map (struct bpf_iter__bpf_map *ctx) {
tools/testing/selftests/bpf/progs/bpf_iter_ipv6_route.c +1 -17
@@ -1,25 +1,9 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ -/* "undefine" structs in vmlinux.h, because we "override" them below */ -#define bpf_iter_meta bpf_iter_meta___not_used -#define bpf_iter__ipv6_route bpf_iter__ipv6_route___not_used -#include "vmlinux.h" -#undef bpf_iter_meta -#undef bpf_iter__ipv6_route +#include "bpf_iter.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_tracing.h> -struct bpf_iter_meta { -struct seq_file *seq; -__u64 session_id; -__u64 seq_num; -} __attribute__((preserve_access_index)); - -struct bpf_iter__ipv6_route { -struct bpf_iter_meta *meta; -struct fib6_info *rt; -} __attribute__((preserve_access_index)); - char _license[] SEC("license") = "GPL"; extern bool CONFIG_IPV6_SUBTREES __kconfig __weak;
tools/testing/selftests/bpf/progs/bpf_iter_netlink.c +1 -17
@@ -1,11 +1,6 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ -/* "undefine" structs in vmlinux.h, because we "override" them below */ -#define bpf_iter_meta bpf_iter_meta___not_used -#define bpf_iter__netlink bpf_iter__netlink___not_used -#include "vmlinux.h" -#undef bpf_iter_meta -#undef bpf_iter__netlink +#include "bpf_iter.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_tracing.h>
@@ -14,17 +9,6 @@ char _license[] SEC("license") = "GPL";#define sk_rmem_alloc sk_backlog.rmem_alloc#define sk_refcnt __sk_common.skc_refcnt -struct bpf_iter_meta { -struct seq_file *seq; -__u64 session_id; -__u64 seq_num; -} __attribute__((preserve_access_index)); - -struct bpf_iter__netlink { -struct bpf_iter_meta *meta; -struct netlink_sock *sk; -} __attribute__((preserve_access_index)); -static inline struct inode *SOCK_INODE (struct socket *socket) { return &container_of(socket, struct socket_alloc, socket)->vfs_inode;
tools/testing/selftests/bpf/progs/bpf_iter_task.c +1 -17
@@ -1,27 +1,11 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ -/* "undefine" structs in vmlinux.h, because we "override" them below */ -#define bpf_iter_meta bpf_iter_meta___not_used -#define bpf_iter__task bpf_iter__task___not_used -#include "vmlinux.h" -#undef bpf_iter_meta -#undef bpf_iter__task +#include "bpf_iter.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_tracing.h> char _license[] SEC("license") = "GPL"; -struct bpf_iter_meta { -struct seq_file *seq; -__u64 session_id; -__u64 seq_num; -} __attribute__((preserve_access_index)); - -struct bpf_iter__task { -struct bpf_iter_meta *meta; -struct task_struct *task; -} __attribute__((preserve_access_index)); - SEC("iter/task")int dump_task (struct bpf_iter__task *ctx) {
tools/testing/selftests/bpf/progs/bpf_iter_task_file.c +1 -19
@@ -1,29 +1,11 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ -/* "undefine" structs in vmlinux.h, because we "override" them below */ -#define bpf_iter_meta bpf_iter_meta___not_used -#define bpf_iter__task_file bpf_iter__task_file___not_used -#include "vmlinux.h" -#undef bpf_iter_meta -#undef bpf_iter__task_file +#include "bpf_iter.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_tracing.h> char _license[] SEC("license") = "GPL"; -struct bpf_iter_meta { -struct seq_file *seq; -__u64 session_id; -__u64 seq_num; -} __attribute__((preserve_access_index)); - -struct bpf_iter__task_file { -struct bpf_iter_meta *meta; -struct task_struct *task; -__u32 fd; -struct file *file; -} __attribute__((preserve_access_index)); - SEC("iter/task_file")int dump_task_file (struct bpf_iter__task_file *ctx) {
tools/testing/selftests/bpf/progs/bpf_iter_test_kern3.c +1 -16
@@ -1,25 +1,10 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ -#define bpf_iter_meta bpf_iter_meta___not_used -#define bpf_iter__task bpf_iter__task___not_used -#include "vmlinux.h" -#undef bpf_iter_meta -#undef bpf_iter__task +#include "bpf_iter.h" #include <bpf/bpf_helpers.h> char _license[] SEC("license") = "GPL"; -struct bpf_iter_meta { -struct seq_file *seq; -__u64 session_id; -__u64 seq_num; -} __attribute__((preserve_access_index)); - -struct bpf_iter__task { -struct bpf_iter_meta *meta; -struct task_struct *task; -} __attribute__((preserve_access_index)); - SEC("iter/task")int dump_task (struct bpf_iter__task *ctx) {
tools/testing/selftests/bpf/progs/bpf_iter_test_kern4.c +1 -16
@@ -1,25 +1,10 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ -#define bpf_iter_meta bpf_iter_meta___not_used -#define bpf_iter__bpf_map bpf_iter__bpf_map___not_used -#include "vmlinux.h" -#undef bpf_iter_meta -#undef bpf_iter__bpf_map +#include "bpf_iter.h" #include <bpf/bpf_helpers.h> char _license[] SEC("license") = "GPL"; -struct bpf_iter_meta { -struct seq_file *seq; -__u64 session_id; -__u64 seq_num; -} __attribute__((preserve_access_index)); - -struct bpf_iter__bpf_map { -struct bpf_iter_meta *meta; -struct bpf_map *map; -} __attribute__((preserve_access_index)); -__u32 map1_id =0 , map2_id =0 ;__u32 map1_accessed =0 , map2_accessed =0 ;__u64 map1_seqnum =0 , map2_seqnum1 =0 , map2_seqnum2 =0 ;
tools/testing/selftests/bpf/progs/bpf_iter_test_kern_common.h +1 -17
@@ -1,27 +1,11 @@/* SPDX-License-Identifier: GPL-2.0 */ /* Copyright (c) 2020 Facebook */ -/* "undefine" structs in vmlinux.h, because we "override" them below */ -#define bpf_iter_meta bpf_iter_meta___not_used -#define bpf_iter__task bpf_iter__task___not_used -#include "vmlinux.h" -#undef bpf_iter_meta -#undef bpf_iter__task +#include "bpf_iter.h" #include <bpf/bpf_helpers.h> char _license[] SEC("license") = "GPL";int count =0 ; -struct bpf_iter_meta { -struct seq_file *seq; -__u64 session_id; -__u64 seq_num; -} __attribute__((preserve_access_index)); - -struct bpf_iter__task { -struct bpf_iter_meta *meta; -struct task_struct *task; -} __attribute__((preserve_access_index)); - SEC("iter/task")int dump_task (struct bpf_iter__task *ctx) {
Signature · 1 line
2.24.1
PATCH v2 11/15 tools/bpf: refactor some net macros to libbpf bpf_tracing_net.h
In reply to Yonghong SongRefactor bpf_iter_ipv6_route.c and bpf_iter_netlink.c so net macros, originally from various include/linux header files, are moved to a new libbpf installable header file bpf_tracing_net.h. The goal is to improve reuse so networking tracing programs do not need to
copy these macros every time they use them.
--- tools/lib/bpf/Makefile | 1 + tools/lib/bpf/bpf_tracing_net.h | 16 ++++++++++++++++ .../selftests/bpf/progs/bpf_iter_ipv6_route.c | 7 +------ .../selftests/bpf/progs/bpf_iter_netlink.c | 4 +--- 4 files changed, 19 insertions(+), 9 deletions(-) create mode 100644 tools/lib/bpf/bpf_tracing_net.h
tools/lib/bpf/Makefile +1 -0
@@ -257,6 +257,7 @@ install_headers: $(BPF_HELPER_DEFS) $(call do_install,bpf_helpers.h,$(prefix)/include/bpf,644); \ $(call do_install,$(BPF_HELPER_DEFS),$(prefix)/include/bpf,644); \ $(call do_install,bpf_tracing.h,$(prefix)/include/bpf,644); \ + $(call do_install,bpf_tracing_net.h,$(prefix)/include/bpf,644); \ $(call do_install,bpf_endian.h,$(prefix)/include/bpf,644); \ $(call do_install,bpf_core_read.h,$(prefix)/include/bpf,644);
tools/lib/bpf/bpf_tracing_net.h new file +16 -0
@@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __BPF_TRACING_NET_H__ +#define __BPF_TRACING_NET_H__ + +#define IFNAMSIZ 16 + +#define RTF_GATEWAY 0x0002 + +#define fib_nh_dev nh_common.nhc_dev +#define fib_nh_gw_family nh_common.nhc_gw_family +#define fib_nh_gw6 nh_common.nhc_gw.ipv6 + +#define sk_rmem_alloc sk_backlog.rmem_alloc +#define sk_refcnt __sk_common.skc_refcnt + +#endif
tools/testing/selftests/bpf/progs/bpf_iter_ipv6_route.c +1 -6
@@ -3,17 +3,12 @@#include "bpf_iter.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_tracing.h> +#include <bpf/bpf_tracing_net.h> char _license[] SEC("license") = "GPL"; extern bool CONFIG_IPV6_SUBTREES __kconfig __weak; -#define RTF_GATEWAY 0x0002 -#define IFNAMSIZ 16 -#define fib_nh_gw_family nh_common.nhc_gw_family -#define fib_nh_gw6 nh_common.nhc_gw.ipv6 -#define fib_nh_dev nh_common.nhc_dev - SEC("iter/ipv6_route")int dump_ipv6_route (struct bpf_iter__ipv6_route *ctx) {
tools/testing/selftests/bpf/progs/bpf_iter_netlink.c +1 -3
@@ -3,12 +3,10 @@#include "bpf_iter.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_tracing.h> +#include <bpf/bpf_tracing_net.h> char _license[] SEC("license") = "GPL"; -#define sk_rmem_alloc sk_backlog.rmem_alloc -#define sk_refcnt __sk_common.skc_refcnt -static inline struct inode *SOCK_INODE (struct socket *socket) { return &container_of(socket, struct socket_alloc, socket)->vfs_inode;
Signature · 1 line
2.24.1
PATCH v2 12/15 tools/libbpf: add more common macros to bpf_tracing_net.h
In reply to Yonghong SongThese newly added macros will be used in subsequent bpf iterator tcp{4,6} and udp{4,6} programs.
--- tools/lib/bpf/bpf_tracing_net.h | 35 +++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
tools/lib/bpf/bpf_tracing_net.h +35 -0
@@ -2,15 +2,50 @@#ifndef __BPF_TRACING_NET_H__#define __BPF_TRACING_NET_H__ +#define AF_INET 2 +#define AF_INET6 10 + +#define ICSK_TIME_RETRANS 1 +#define ICSK_TIME_PROBE0 3 +#define ICSK_TIME_LOSS_PROBE 5 +#define ICSK_TIME_REO_TIMEOUT 6 +#define IFNAMSIZ 16#define RTF_GATEWAY 0x0002 +#define TCP_INFINITE_SSTHRESH 0x7fffffff +#define TCP_PINGPONG_THRESH 3 +#define fib_nh_dev nh_common.nhc_dev#define fib_nh_gw_family nh_common.nhc_gw_family#define fib_nh_gw6 nh_common.nhc_gw.ipv6 +#define inet_daddr sk.__sk_common.skc_daddr +#define inet_rcv_saddr sk.__sk_common.skc_rcv_saddr +#define inet_dport sk.__sk_common.skc_dport + +#define ir_loc_addr req.__req_common.skc_rcv_saddr +#define ir_num req.__req_common.skc_num +#define ir_rmt_addr req.__req_common.skc_daddr +#define ir_rmt_port req.__req_common.skc_dport +#define ir_v6_rmt_addr req.__req_common.skc_v6_daddr +#define ir_v6_loc_addr req.__req_common.skc_v6_rcv_saddr + +#define sk_family __sk_common.skc_family#define sk_rmem_alloc sk_backlog.rmem_alloc#define sk_refcnt __sk_common.skc_refcnt +#define sk_state __sk_common.skc_state +#define sk_v6_daddr __sk_common.skc_v6_daddr +#define sk_v6_rcv_saddr __sk_common.skc_v6_rcv_saddr + +#define s6_addr32 in6_u.u6_addr32 + +#define tw_daddr __tw_common.skc_daddr +#define tw_rcv_saddr __tw_common.skc_rcv_saddr +#define tw_dport __tw_common.skc_dport +#define tw_refcnt __tw_common.skc_refcnt +#define tw_v6_daddr __tw_common.skc_v6_daddr +#define tw_v6_rcv_saddr __tw_common.skc_v6_rcv_saddr#endif
Signature · 1 line
2.24.1
PATCH v2 13/15 tools/bpf: selftests: implement sample tcp/tcp6 bpf_iter programs
In reply to Yonghong SongIn my VM, I got identical result compared to /proc/net/{tcp,tcp6}. For tcp6:
$ cat /proc/net/tcp6
sl local_address remote_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
0: 00000000000000000000000000000000:0016 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000001 00000000 0 0 17955 1 000000003eb3102e 100 0 0 10 0 $ cat /sys/fs/bpf/p1
sl local_address remote_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
0: 00000000000000000000000000000000:0016 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 17955 1 000000003eb3102e 100 0 0 10 0For tcp:
$ cat /proc/net/tcp sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode 0: 00000000:0016 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 2666 1 000000007152e43f 100 0 0 10 0 $ cat /sys/fs/bpf/p2 sl local_address remote_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode 1: 00000000:0016 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 2666 1 000000007152e43f 100 0 0 10 0
--- tools/testing/selftests/bpf/progs/bpf_iter.h | 15 ++ .../selftests/bpf/progs/bpf_iter_tcp4.c | 235 ++++++++++++++++ .../selftests/bpf/progs/bpf_iter_tcp6.c | 250 ++++++++++++++++++ 3 files changed, 500 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_tcp4.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_tcp6.c
tools/testing/selftests/bpf/progs/bpf_iter.h +15 -0
@@ -7,6 +7,8 @@#define bpf_iter__netlink bpf_iter__netlink___not_used#define bpf_iter__task bpf_iter__task___not_used#define bpf_iter__task_file bpf_iter__task_file___not_used +#define bpf_iter__tcp bpf_iter__tcp___not_used +#define tcp6_sock tcp6_sock___not_used#include "vmlinux.h" #undef bpf_iter_meta#undef bpf_iter__bpf_map
@@ -14,6 +16,8 @@#undef bpf_iter__netlink#undef bpf_iter__task#undef bpf_iter__task_file +#undef bpf_iter__tcp +#undef tcp6_sockstruct bpf_iter_meta {struct seq_file *seq;
@@ -47,3 +51,14 @@ struct bpf_iter__bpf_map {struct bpf_iter_meta *meta;struct bpf_map *map; } __attribute__((preserve_access_index)); + +struct bpf_iter__tcp { +struct bpf_iter_meta *meta; +struct sock_common *sk_common; +uid_t uid; +} __attribute__((preserve_access_index)); + +struct tcp6_sock { +struct tcp_sock tcp; +struct ipv6_pinfo inet6; +} __attribute__((preserve_access_index));
tools/testing/selftests/bpf/progs/bpf_iter_tcp4.c new file +235 -0
235 lines not on this page · raw message
tools/testing/selftests/bpf/progs/bpf_iter_tcp6.c new file +250 -0
250 lines not on this page · raw message
Signature · 1 line
2.24.1
PATCH v2 14/15 tools/bpf: add udp4/udp6 bpf iterator
In reply to Yonghong SongOn my VM, I got identical results between /proc/net/udp[6] and the udp{4,6} bpf iterator.
For udp6:
$ cat /sys/fs/bpf/p1
sl local_address remote_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
1405: 000080FE00000000FF7CC4D0D9EFE4FE:0222 00000000000000000000000000000000:0000 07 00000000:00000000 00:00000000 00000000 193 0 19183 2 0000000029eab111 0
$ cat /proc/net/udp6
sl local_address remote_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
1405: 000080FE00000000FF7CC4D0D9EFE4FE:0222 00000000000000000000000000000000:0000 07 00000000:00000000 00:00000000 00000000 193 0 19183 2 0000000029eab111 0For udp4:
$ cat /sys/fs/bpf/p4
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
2007: 00000000:1F90 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 72540 2 000000004ede477a 0
$ cat /proc/net/udp
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode ref pointer drops
2007: 00000000:1F90 00000000:0000 07 00000000:00000000 00:00000000 00000000 0 0 72540 2 000000004ede477a 0
---
tools/testing/selftests/bpf/progs/bpf_iter.h | 16 ++++
.../selftests/bpf/progs/bpf_iter_udp4.c | 71 +++++++++++++++++
.../selftests/bpf/progs/bpf_iter_udp6.c | 79 +++++++++++++++++++
3 files changed, 166 insertions(+)
create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_udp4.c
create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_udp6.ctools/testing/selftests/bpf/progs/bpf_iter.h +16 -0
@@ -9,6 +9,8 @@#define bpf_iter__task_file bpf_iter__task_file___not_used#define bpf_iter__tcp bpf_iter__tcp___not_used#define tcp6_sock tcp6_sock___not_used +#define bpf_iter__udp bpf_iter__udp___not_used +#define udp6_sock udp6_sock___not_used#include "vmlinux.h" #undef bpf_iter_meta#undef bpf_iter__bpf_map
@@ -18,6 +20,8 @@#undef bpf_iter__task_file#undef bpf_iter__tcp#undef tcp6_sock +#undef bpf_iter__udp +#undef udp6_sockstruct bpf_iter_meta {struct seq_file *seq;
@@ -62,3 +66,15 @@ struct tcp6_sock {struct tcp_sock tcp;struct ipv6_pinfo inet6; } __attribute__((preserve_access_index)); + +struct bpf_iter__udp { +struct bpf_iter_meta *meta; +struct udp_sock *udp_sk; +uid_t uid __attribute__((aligned (8 ))); +int bucket __attribute__((aligned (8 ))); +} __attribute__((preserve_access_index)); + +struct udp6_sock { +struct udp_sock udp; +struct ipv6_pinfo inet6; +} __attribute__((preserve_access_index));
tools/testing/selftests/bpf/progs/bpf_iter_udp4.c new file +71 -0
71 lines not on this page · raw message
tools/testing/selftests/bpf/progs/bpf_iter_udp6.c new file +79 -0
79 lines not on this page · raw message
Signature · 1 line
2.24.1
PATCH v2 15/15 bpf/selftests: add tcp/udp iterator programs to selftests
In reply to Yonghong SongAdded tcp{4,6} and udp{4,6} bpf programs into test_progs selftest so that they at least can load successfully.
$ ./test_progs -n 3 ... #3/7 tcp4:OK #3/8 tcp6:OK #3/9 udp4:OK #3/10 udp6:OK ... #3 bpf_iter:OK Summary: 1/16 PASSED, 0 SKIPPED, 0 FAILED
--- .../selftests/bpf/prog_tests/bpf_iter.c | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+)
tools/testing/selftests/bpf/prog_tests/bpf_iter.c +68 -0
@@ -6,6 +6,10 @@#include "bpf_iter_bpf_map.skel.h" #include "bpf_iter_task.skel.h" #include "bpf_iter_task_file.skel.h" +#include "bpf_iter_tcp4.skel.h" +#include "bpf_iter_tcp6.skel.h" +#include "bpf_iter_udp4.skel.h" +#include "bpf_iter_udp6.skel.h" #include "bpf_iter_test_kern1.skel.h" #include "bpf_iter_test_kern2.skel.h" #include "bpf_iter_test_kern3.skel.h"
@@ -120,6 +124,62 @@ static void test_task_file(void)bpf_iter_task_file__destroy (skel); } +static void test_tcp4 (void ) +{ +struct bpf_iter_tcp4 *skel; + + skel =bpf_iter_tcp4__open_and_load (); +if (CHECK (!skel,"bpf_iter_tcp4__open_and_load" , +"skeleton open_and_load failed\n" )) +return ; + +do_dummy_read (skel->progs.dump_tcp4); + +bpf_iter_tcp4__destroy (skel); +} + +static void test_tcp6 (void ) +{ +struct bpf_iter_tcp6 *skel; + + skel =bpf_iter_tcp6__open_and_load (); +if (CHECK (!skel,"bpf_iter_tcp6__open_and_load" , +"skeleton open_and_load failed\n" )) +return ; + +do_dummy_read (skel->progs.dump_tcp6); + +bpf_iter_tcp6__destroy (skel); +} + +static void test_udp4 (void ) +{ +struct bpf_iter_udp4 *skel; + + skel =bpf_iter_udp4__open_and_load (); +if (CHECK (!skel,"bpf_iter_udp4__open_and_load" , +"skeleton open_and_load failed\n" )) +return ; + +do_dummy_read (skel->progs.dump_udp4); + +bpf_iter_udp4__destroy (skel); +} + +static void test_udp6 (void ) +{ +struct bpf_iter_udp6 *skel; + + skel =bpf_iter_udp6__open_and_load (); +if (CHECK (!skel,"bpf_iter_udp6__open_and_load" , +"skeleton open_and_load failed\n" )) +return ; + +do_dummy_read (skel->progs.dump_udp6); + +bpf_iter_udp6__destroy (skel); +} +/* The expected string is less than 16 bytes */ static int do_read_with_fd (int iter_fd,const char *expected,bool read_one_char)
@@ -394,6 +454,14 @@ void test_bpf_iter(void)test_task ();if (test__start_subtest ("task_file" ))test_task_file (); +if (test__start_subtest ("tcp4" )) +test_tcp4 (); +if (test__start_subtest ("tcp6" )) +test_tcp6 (); +if (test__start_subtest ("udp4" )) +test_udp4 (); +if (test__start_subtest ("udp6" )) +test_udp6 ();if (test__start_subtest ("anon" ))test_anon_iter (false);if (test__start_subtest ("anon-read-one-char" ))
Signature · 1 line
2.24.1
Jakub Kicinski
maintainerQuoted 7 lines · On Sat, 20 Jun 2020 22:54:59 -0700 Yonghong Song wrote:
Changelogs:
v1 -> v2:
- guardinit_sock_cast_types()defination properly withCONFIG_NET(Martin)
- reuse thebtf_ids, computed for new helper argument, for return values (Martin)
- usingBTF_TYPE_EMITto express intent of btf type generation (Andrii)
- abstract out common net macros intobpf_tracing_net.h(Andrii)
netdev@ has not been CCed either on v1 or v2.
Seems more than appropriate to do so.
Quoted 11 lines · Jakub Kicinski
Quoted 7 lines · On Sat, 20 Jun 2020 22:54:59 -0700 Yonghong Song wrote:
Changelogs:
v1 -> v2:
- guardinit_sock_cast_types()defination properly withCONFIG_NET(Martin)
- reuse thebtf_ids, computed for new helper argument, for return values (Martin)
- usingBTF_TYPE_EMITto express intent of btf type generation (Andrii)
- abstract out common net macros intobpf_tracing_net.h(Andrii)netdev@ has not been CCed either on v1 or v2.
Seems more than appropriate to do so.
Sorry about this. I forgot to include netdev. I will send out v3 shortly also cc'ing netdev.