bpf: More sock_ops callbacks
Subsystems (8)
v8 cover bpf: More sock_ops callbacks
This patchset adds support for:
- direct R or R/W access to many tcp_sock fields
- passing up to 4 arguments to sock_ops BPF functions
- tcp_sock field bpf_sock_ops_cb_flags for controlling callbacks
- optionally calling sock_ops BPF program when RTO fires
- optionally calling sock_ops BPF program when packet is retransmitted
- optionally calling sock_ops BPF program when TCP state changes
- access to tclass and sk_txhash
- new selftest
v2: Fixed commit message 0/11. The commit is to "bpf-next" but the patch
below used "bpf" and Patchwork didn't work correctly.
v3: Cleaned RTO callback as per Yuchung's comment
Added BPF enum for TCP states as per Alexei's comment
v4: Fixed compile warnings related to detecting changes between TCP
internal states and the BPF defined states.
v5: Fixed comment issues in some selftest files
Fixed accesss issue with u64 fields in bpf_sock_ops struct
v6: Made fixes based on comments form Eric Dumazet:
The field bpf_sock_ops_cb_flags was addded in a hole on 64bit kernels Field bpf_sock_ops_cb_flags is now set through a helper function which returns an error when a BPF program tries to set bits for callbacks that are not supported in the current kernel. Added a comment indicating that when adding fields to bpf_sock_ops_kern they should be added before the field named "temp" if they need to be cleared before calling the BPF function.
v7: Enfornced fields "op" and "replylong[1] .. replylong[3]" not be writable
based on comments form Eric Dumazet and Alexei Starovoitov. Filled 32 bit hole in bpf_sock_ops struct with sk_txhash based on comments from Daniel Borkmann.
Removed unused functions (tcp_call_bpf_1arg, tcp_call_bpf_4arg) based on comments from Daniel Borkmann.
v8: Add commit message 00/12
Add Acked-by as appropriate
Consists of the following patches:
[PATCH bpf-next v8 01/12] bpf: Make SOCK_OPS_GET_TCP size independent [PATCH bpf-next v8 02/12] bpf: Make SOCK_OPS_GET_TCP struct [PATCH bpf-next v8 03/12] bpf: Add write access to tcp_sock and sock [PATCH bpf-next v8 04/12] bpf: Only reply field should be writeable [PATCH bpf-next v8 05/12] bpf: Support passing args to sock_ops bpf [PATCH bpf-next v8 06/12] bpf: Adds field bpf_sock_ops_cb_flags to [PATCH bpf-next v8 07/12] bpf: Add sock_ops RTO callback [PATCH bpf-next v8 08/12] bpf: Add support for reading sk_state and [PATCH bpf-next v8 09/12] bpf: Add sock_ops R/W access to tclass [PATCH bpf-next v8 10/12] bpf: Add BPF_SOCK_OPS_RETRANS_CB [PATCH bpf-next v8 11/12] bpf: Add BPF_SOCK_OPS_STATE_CB [PATCH bpf-next v8 12/12] bpf: add selftest for tcpbpf
include/linux/filter.h | 10 ++ include/linux/tcp.h | 11 ++ include/net/tcp.h | 42 ++++- include/uapi/linux/bpf.h | 72 +++++++- include/uapi/linux/tcp.h | 8 + net/core/filter.c | 282 ++++++++++++++++++++++++++++--- net/ipv4/tcp.c | 26 ++- net/ipv4/tcp_nv.c | 2 +- net/ipv4/tcp_output.c | 5 +- net/ipv4/tcp_timer.c | 7 + tools/include/uapi/linux/bpf.h | 74 +++++++- tools/testing/selftests/bpf/Makefile | 4 +- tools/testing/selftests/bpf/bpf_helpers.h | 2 + tools/testing/selftests/bpf/tcp_client.py | 52 ++++++ tools/testing/selftests/bpf/tcp_server.py | 79 +++++++++ tools/testing/selftests/bpf/test_tcpbpf.h | 16 ++ tools/testing/selftests/bpf/test_tcpbpf_kern.c | 131 ++++++++++++++ tools/testing/selftests/bpf/test_tcpbpf_user.c | 126 ++++++++++++++ 18 files changed, 910 insertions(+), 39 deletions(-)
PATCH v8 1/12 bpf: Make SOCK_OPS_GET_TCP size independent
Make SOCK_OPS_GET_TCP helper macro size independent (before only worked with 4-byte fields.
--- net/core/filter.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
net/core/filter.c +8 -5
@@ -4471,9 +4471,10 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type,break ;/* Helper macro for adding read access to tcp_sock fields. */ -#define SOCK_OPS_GET_TCP32 (FIELD_NAME ) \ +#define SOCK_OPS_GET_TCP (FIELD_NAME ) \ do { \ - BUILD_BUG_ON(FIELD_SIZEOF(struct tcp_sock, FIELD_NAME) != 4); \ + BUILD_BUG_ON(FIELD_SIZEOF(struct tcp_sock, FIELD_NAME) > \ + FIELD_SIZEOF(struct bpf_sock_ops, FIELD_NAME)); \ *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( \ struct bpf_sock_ops_kern, \ is_fullsock), \
@@ -4485,16 +4486,18 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, struct bpf_sock_ops_kern, sk),\ si->dst_reg, si->src_reg, \ offsetof(struct bpf_sock_ops_kern, sk));\ - *insn++ =BPF_LDX_MEM (BPF_W , si->dst_reg, si->dst_reg, \ + *insn++ = BPF_LDX_MEM(FIELD_SIZEOF(struct tcp_sock, \ +FIELD_NAME ), si->dst_reg, \ + si->dst_reg, \ offsetof(struct tcp_sock , FIELD_NAME)); \ }while (0 )case offsetof(struct bpf_sock_ops , snd_cwnd): -SOCK_OPS_GET_TCP32 (snd_cwnd); +SOCK_OPS_GET_TCP (snd_cwnd);break ;case offsetof(struct bpf_sock_ops , srtt_us): -SOCK_OPS_GET_TCP32 (srtt_us); +SOCK_OPS_GET_TCP (srtt_us);break ; }return insn - insn_buf;
Signature · 1 line
2.9.5
PATCH v8 2/12 bpf: Make SOCK_OPS_GET_TCP struct independent
In reply to Lawrence BrakmoChanged SOCK_OPS_GET_TCP to SOCK_OPS_GET_FIELD and added 2 arguments so now it can also work with struct sock fields. The first argument is the name of the field in the bpf_sock_ops struct, the 2nd argument is the name of the field in the OBJ struct.
Previous: SOCK_OPS_GET_TCP(FIELD_NAME) New: SOCK_OPS_GET_FIELD(BPF_FIELD, OBJ_FIELD, OBJ)
Where OBJ is either "struct tcp_sock" or "struct sock" (without quotation). BPF_FIELD is the name of the field in the bpf_sock_ops struct and OBJ_FIELD is the name of the field in the OBJ struct.
Although the field names are currently the same, the kernel struct names could change in the future and this change makes it easier to support that.
Note that adding access to tcp_sock fields in sock_ops programs does not preclude the tcp_sock fields from being removed as long as we are willing to do one of the following:
1) Return a fixed value (e.x. 0 or 0xffffffff), or
2) Make the verifier fail if that field is accessed (i.e. program fails to load) so the user will know that field is no longer supported.
--- net/core/filter.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
net/core/filter.c +10 -10
@@ -4470,11 +4470,11 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, is_fullsock));break ; -/* Helper macro for adding read access to tcp_sock fields. */ -#define SOCK_OPS_GET_TCP (FIELD_NAME ) \ +/* Helper macro for adding read access to tcp_sock or sock fields. */ +#define SOCK_OPS_GET_FIELD (BPF_FIELD ,OBJ_FIELD ,OBJ ) \ do { \ - BUILD_BUG_ON(FIELD_SIZEOF(struct tcp_sock, FIELD_NAME) > \ - FIELD_SIZEOF(struct bpf_sock_ops, FIELD_NAME)); \ + BUILD_BUG_ON(FIELD_SIZEOF(OBJ, OBJ_FIELD) > \ + FIELD_SIZEOF(struct bpf_sock_ops, BPF_FIELD)); \ *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( \ struct bpf_sock_ops_kern, \ is_fullsock), \
@@ -4486,18 +4486,18 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, struct bpf_sock_ops_kern, sk),\ si->dst_reg, si->src_reg, \ offsetof(struct bpf_sock_ops_kern, sk));\ - *insn++ = BPF_LDX_MEM(FIELD_SIZEOF(struct tcp_sock, \ -FIELD_NAME ), si->dst_reg, \ - si->dst_reg, \ - offsetof(struct tcp_sock , FIELD_NAME)); \ + *insn++ =BPF_LDX_MEM (BPF_FIELD_SIZEOF (OBJ , \ +OBJ_FIELD ), \ + si->dst_reg, si->dst_reg, \ + offsetof(OBJ , OBJ_FIELD)); \ }while (0 )case offsetof(struct bpf_sock_ops , snd_cwnd): -SOCK_OPS_GET_TCP (snd_cwnd); + SOCK_OPS_GET_FIELD(snd_cwnd, snd_cwnd, struct tcp_sock);break ;case offsetof(struct bpf_sock_ops , srtt_us): -SOCK_OPS_GET_TCP (srtt_us); + SOCK_OPS_GET_FIELD(srtt_us, srtt_us, struct tcp_sock);break ; }return insn - insn_buf;
Signature · 1 line
2.9.5
PATCH v8 3/12 bpf: Add write access to tcp_sock and sock fields
In reply to Lawrence BrakmoThis patch adds a macro, SOCK_OPS_SET_FIELD, for writing to struct tcp_sock or struct sock fields. This required adding a new field "temp" to struct bpf_sock_ops_kern for temporary storage that is used by sock_ops_convert_ctx_access. It is used to store and recover the contents of a register, so the register can be used to store the address of the sk. Since we cannot overwrite the dst_reg because it contains the pointer to ctx, nor the src_reg since it contains the value we want to store, we need an extra register to contain the address of the sk.
Also adds the macro SOCK_OPS_GET_OR_SET_FIELD that calls one of the GET or SET macros depending on the value of the TYPE field.
--- include/linux/filter.h | 9 +++++++++ include/net/tcp.h | 2 +- net/core/filter.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+), 1 deletion(-)
include/linux/filter.h +9 -0
@@ -1007,6 +1007,15 @@ struct bpf_sock_ops_kern { u32 replylong[4]; };u32 is_fullsock; +u64 temp;/* temp and everything after is not +* initialized to 0 before calling +* the BPF program. New fields that +* should be initialized to 0 should +* be inserted before temp. +* temp is scratch storage used by +* sock_ops_convert_ctx_access +* as temporary storage of a register. +*/ };#endif /* __LINUX_FILTER_H__ */
include/net/tcp.h +1 -1
@@ -2011,7 +2011,7 @@ static inline int tcp_call_bpf(struct sock *sk, int op)struct bpf_sock_ops_kern sock_ops;int ret; -memset (&sock_ops,0 ,sizeof (sock_ops)); +memset (&sock_ops,0 , offsetof(struct bpf_sock_ops_kern , temp));if (sk_fullsock (sk)) { sock_ops.is_fullsock =1 ;sock_owned_by_me (sk);
net/core/filter.c +48 -0
54 lines not on this page · raw message
Signature · 1 line
2.9.5
PATCH v8 4/12 bpf: Only reply field should be writeable
In reply to Lawrence BrakmoCurrently, a sock_ops BPF program can write the op field and all the reply fields (reply and replylong). This is a bug. The op field should not have been writeable and there is currently no way to use replylong field for indices >= 1. This patch enforces that only the reply field (which equals replylong[0]) is writeable.
--- net/core/filter.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
net/core/filter.c +1 -2
@@ -3845,8 +3845,7 @@ static bool sock_ops_is_valid_access(int off, int size, {if (type ==BPF_WRITE ) {switch (off) { - case offsetof(struct bpf_sock_ops, op) ... - offsetof(struct bpf_sock_ops, replylong[3]): +case offsetof(struct bpf_sock_ops , reply):break ;default :return false;
Signature · 1 line
2.9.5
Re: [PATCH bpf-next v8 04/12] bpf: Only reply field should be writeable
Quoted 5 lines · Lawrence Brakmo
Currently, a
sock_opsBPF program can write the op field and all the reply fields (reply and replylong). This is a bug. The op field should not have been writeable and there is currently no way to use replylong field for indices >= 1. This patch enforces that only the reply field (which equals replylong[0]) is writeable.
Would this patch be more suitable for -net ?
Quoted 3 lines
Fixes 40304b2a1567 ("bpf: BPF support for sock_ops")Signed-off-by Lawrence Brakmo <[email protected]>
Quoted 21 lines
--- net/core/filter.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)net/core/filter.c +1
-3@@ -3845,8 +3845,7 @@ static bool sock_ops_is_valid_access(int off, int size, {if (type ==BPF_WRITE ) {switch (off) { - case offsetof(struct bpf_sock_ops, op) ... - offsetof(struct bpf_sock_ops, replylong[3]): +case offsetof(struct bpf_sock_ops , reply):break ;default :return false; --2.9.5
Re: [PATCH bpf-next v8 04/12] bpf: Only reply field should be writeable
Quoted 7 lines · Yuchung Cheng
Quoted 5 lines · Lawrence Brakmo
Currently, a
sock_opsBPF program can write the op field and all the reply fields (reply and replylong). This is a bug. The op field should not have been writeable and there is currently no way to use replylong field for indices >= 1. This patch enforces that only the reply field (which equals replylong[0]) is writeable.Would this patch be more suitable for -net ?
yes. we will send it to 4.15 and 4.14 as soon as it's released. See discussion with Eric.
Quoted 26 lines
Quoted 3 lines
Fixes 40304b2a1567 ("bpf: BPF support for sock_ops")Signed-off-by Lawrence Brakmo <[email protected]>Acked-by Yuchung Cheng <[email protected]>Quoted 21 lines
--- net/core/filter.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)net/core/filter.c +1
-3@@ -3845,8 +3845,7 @@ static bool sock_ops_is_valid_access(int off, int size, {if (type ==BPF_WRITE ) {switch (off) { - case offsetof(struct bpf_sock_ops, op) ... - offsetof(struct bpf_sock_ops, replylong[3]): +case offsetof(struct bpf_sock_ops , reply):break ;default :return false; --2.9.5
PATCH v8 5/12 bpf: Support passing args to sock_ops bpf function
In reply to Lawrence BrakmoAdds support for passing up to 4 arguments to sock_ops bpf functions. It reusues the reply union, so the bpf_sock_ops structures are not increased in size.
--- include/linux/filter.h | 1 + include/net/tcp.h | 40 +++++++++++++++++++++++++++++++++++----- include/uapi/linux/bpf.h | 5 +++-- net/ipv4/tcp.c | 2 +- net/ipv4/tcp_nv.c | 2 +- net/ipv4/tcp_output.c | 2 +- 6 files changed, 42 insertions(+), 10 deletions(-)
include/linux/filter.h +1 -0
@@ -1003,6 +1003,7 @@ struct bpf_sock_ops_kern {struct sock *sk;u32 op;union { +u32 args[4 ];u32 reply;u32 replylong[4 ]; };
include/net/tcp.h +35 -5
@@ -2006,7 +2006,7 @@ void tcp_cleanup_ulp(struct sock *sk);* program loaded). */ #ifdef CONFIG_BPF -static inline int tcp_call_bpf (struct sock *sk,int op) +static inline int tcp_call_bpf (struct sock *sk,int op,u32 nargs,u32 *args) {struct bpf_sock_ops_kern sock_ops;int ret;
@@ -2019,6 +2019,8 @@ static inline int tcp_call_bpf(struct sock *sk, int op) sock_ops.sk = sk; sock_ops.op = op; +if (nargs >0 ) +memcpy (sock_ops.args, args, nargs *sizeof (*args)); ret =BPF_CGROUP_RUN_PROG_SOCK_OPS (&sock_ops);if (ret ==0 )
@@ -2027,18 +2029,46 @@ static inline int tcp_call_bpf(struct sock *sk, int op) ret =-1 ;return ret; } + +static inline int tcp_call_bpf_2arg (struct sock *sk,int op,u32 arg1,u32 arg2) +{ +u32 args[2 ] = {arg1, arg2}; + +return tcp_call_bpf (sk, op,2 , args); +} + +static inline int tcp_call_bpf_3arg (struct sock *sk,int op,u32 arg1,u32 arg2, +u32 arg3) +{ +u32 args[3 ] = {arg1, arg2, arg3}; + +return tcp_call_bpf (sk, op,3 , args); +} +#else -static inline int tcp_call_bpf (struct sock *sk,int op) +static inline int tcp_call_bpf (struct sock *sk,int op,u32 nargs,u32 *args) {return -EPERM ; } + +static inline int tcp_call_bpf_2arg (struct sock *sk,int op,u32 arg1,u32 arg2) +{ +return -EPERM ; +} + +static inline int tcp_call_bpf_3arg (struct sock *sk,int op,u32 arg1,u32 arg2, +u32 arg3) +{ +return -EPERM ; +} +#endif static inline u32 tcp_timeout_init (struct sock *sk) {int timeout; - timeout =tcp_call_bpf (sk,BPF_SOCK_OPS_TIMEOUT_INIT ); + timeout =tcp_call_bpf (sk,BPF_SOCK_OPS_TIMEOUT_INIT ,0 ,NULL );if (timeout <=0 ) timeout =TCP_TIMEOUT_INIT ;
@@ -2049,7 +2079,7 @@ static inline u32 tcp_rwnd_init_bpf(struct sock *sk) {int rwnd; - rwnd =tcp_call_bpf (sk,BPF_SOCK_OPS_RWND_INIT ); + rwnd =tcp_call_bpf (sk,BPF_SOCK_OPS_RWND_INIT ,0 ,NULL );if (rwnd <0 ) rwnd =0 ;
@@ -2058,7 +2088,7 @@ static inline u32 tcp_rwnd_init_bpf(struct sock *sk)static inline bool tcp_bpf_ca_needs_ecn (struct sock *sk) { -return (tcp_call_bpf (sk,BPF_SOCK_OPS_NEEDS_ECN ) ==1 ); +return (tcp_call_bpf (sk,BPF_SOCK_OPS_NEEDS_ECN ,0 ,NULL ) ==1 ); }#if IS_ENABLED (CONFIG_SMC )
include/uapi/linux/bpf.h +3 -2
@@ -952,8 +952,9 @@ struct bpf_map_info {struct bpf_sock_ops {__u32 op;union { -__u32 reply; -__u32 replylong[4 ]; +__u32 args[4 ];/* Optionally passed to bpf program */ +__u32 reply;/* Returned by bpf program */ +__u32 replylong[4 ];/* Optionally returned by bpf prog */ };__u32 family;__u32 remote_ip4;/* Stored in network byte order */
net/ipv4/tcp.c +1 -1
@@ -463,7 +463,7 @@ void tcp_init_transfer(struct sock *sk, int bpf_op)tcp_mtup_init (sk); icsk->icsk_af_ops->rebuild_header (sk);tcp_init_metrics (sk); -tcp_call_bpf (sk, bpf_op); +tcp_call_bpf (sk, bpf_op,0 ,NULL );tcp_init_congestion_control (sk);tcp_init_buffer_space (sk); }
net/ipv4/tcp_nv.c +1 -1
@@ -146,7 +146,7 @@ static void tcpnv_init(struct sock *sk)* within a datacenter, where we have reasonable estimates of * RTTs */ - base_rtt =tcp_call_bpf (sk,BPF_SOCK_OPS_BASE_RTT ); + base_rtt =tcp_call_bpf (sk,BPF_SOCK_OPS_BASE_RTT ,0 ,NULL );if (base_rtt >0 ) { ca->nv_base_rtt = base_rtt; ca->nv_lower_bound_rtt = (base_rtt *205 ) >>8 ;/* 80% */
net/ipv4/tcp_output.c +1 -1
@@ -3469,7 +3469,7 @@ int tcp_connect(struct sock *sk)struct sk_buff *buff;int err; -tcp_call_bpf (sk,BPF_SOCK_OPS_TCP_CONNECT_CB ); +tcp_call_bpf (sk,BPF_SOCK_OPS_TCP_CONNECT_CB ,0 ,NULL );if (inet_csk (sk)->icsk_af_ops->rebuild_header (sk))return -EHOSTUNREACH ;/* Routing failure or similar. */
Signature · 1 line
2.9.5
PATCH v8 6/12 bpf: Adds field bpf_sock_ops_cb_flags to tcp_sock
In reply to Lawrence BrakmoAdds field bpf_sock_ops_cb_flags to tcp_sock and bpf_sock_ops. Its primary use is to determine if there should be calls to sock_ops bpf program at various points in the TCP code. The field is initialized to zero, disabling the calls. A sock_ops BPF program can set it, per connection and as necessary, when the connection is established.
It also adds support for reading and writting the field within a sock_ops BPF program. Reading is done by accessing the field directly. However, writing is done through the helper function bpf_sock_ops_cb_flags_set, in order to return an error if a BPF program is trying to set a callback that is not supported in the current kernel (i.e. running an older kernel). The helper function returns 0 if it was able to set all of the bits set in the argument, a positive number containing the bits that could not be set, or -EINVAL if the socket is not a full TCP socket.
Examples of where one could call the bpf program:
1) When RTO fires
2) When a packet is retransmitted
3) When the connection terminates
4) When a packet is sent
5) When a packet is received
--- include/linux/tcp.h | 11 +++++++++++ include/uapi/linux/bpf.h | 12 +++++++++++- include/uapi/linux/tcp.h | 5 +++++ net/core/filter.c | 34 ++++++++++++++++++++++++++++++++++ 4 files changed, 61 insertions(+), 1 deletion(-)
include/linux/tcp.h +11 -0
@@ -335,6 +335,17 @@ struct tcp_sock {int linger2; + +/* Sock_ops bpf program related variables */ +#ifdef CONFIG_BPF +u8 bpf_sock_ops_cb_flags;/* Control calling BPF programs +* values defined in uapi/linux/tcp.h +*/ +#define BPF_SOCK_OPS_TEST_FLAG (TP ,ARG ) (TP->bpf_sock_ops_cb_flags & ARG) +#else +#define BPF_SOCK_OPS_TEST_FLAG (TP ,ARG ) 0 +#endif +/* Receiver side RTT estimation */ struct {u32 rtt_us;
include/uapi/linux/bpf.h +11 -1
@@ -642,6 +642,14 @@ union bpf_attr {* @optlen: length of optval in bytes * Return: 0 or negative error * +* int bpf_sock_ops_cb_flags_set(bpf_sock_ops, flags) +* Set callback flags for sock_ops +* @bpf_sock_ops: pointer to bpf_sock_ops_kern struct +* @flags: flags value +* Return: 0 for no error +* -EINVAL if there is no full tcp socket +* bits in flags that are not supported by current kernel +* * int bpf_skb_adjust_room(skb, len_diff, mode, flags) * Grow or shrink room in sk_buff. * @skb: pointer to skb
@@ -748,7 +756,8 @@ union bpf_attr {FN (perf_event_read_value), \FN (perf_prog_read_value), \FN (getsockopt), \ -FN (override_return), +FN (override_return), \ +FN (sock_ops_cb_flags_set),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
@@ -969,6 +978,7 @@ struct bpf_sock_ops {*/ __u32 snd_cwnd;__u32 srtt_us;/* Averaged RTT << 3 in usecs */ +__u32 bpf_sock_ops_cb_flags;/* flags defined in uapi/linux/tcp.h */ };/* List of known BPF sock_ops operators.
include/uapi/linux/tcp.h +5 -0
@@ -268,4 +268,9 @@ struct tcp_diag_md5sig { __u8 tcpm_key[TCP_MD5SIG_MAXKEYLEN]; }; +/* Definitions for bpf_sock_ops_cb_flags */ +#define BPF_SOCK_OPS_ALL_CB_FLAGS 0/* Mask of all currently +* supported cb flags +*/ +#endif /* _UAPI_LINUX_TCP_H */
net/core/filter.c +34 -0
@@ -3328,6 +3328,33 @@ static const struct bpf_func_proto bpf_getsockopt_proto = { .arg5_type = ARG_CONST_SIZE, }; +BPF_CALL_2(bpf_sock_ops_cb_flags_set, struct bpf_sock_ops_kern *, bpf_sock, + int, argval) +{ +struct sock *sk = bpf_sock->sk; +int val = argval &BPF_SOCK_OPS_ALL_CB_FLAGS ; + +if (!sk_fullsock (sk)) +return -EINVAL ; + +#ifdef CONFIG_INET +if (val) +tcp_sk (sk)->bpf_sock_ops_cb_flags = val; + +return argval & (~BPF_SOCK_OPS_ALL_CB_FLAGS ); +#else +return -EINVAL ; +#endif +} + +static const struct bpf_func_proto bpf_sock_ops_cb_flags_set_proto = { + .func = bpf_sock_ops_cb_flags_set, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX , + .arg2_type =ARG_ANYTHING , +}; +static const struct bpf_func_proto *bpf_base_func_proto (enum bpf_func_id func_id) {
@@ -3510,6 +3537,8 @@ static const struct bpf_func_proto *return &bpf_setsockopt_proto;case BPF_FUNC_getsockopt:return &bpf_getsockopt_proto; +case BPF_FUNC_sock_ops_cb_flags_set: +return &bpf_sock_ops_cb_flags_set_proto;case BPF_FUNC_sock_map_update:return &bpf_sock_map_update_proto;default :
@@ -4546,6 +4575,11 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type,case offsetof(struct bpf_sock_ops , srtt_us): SOCK_OPS_GET_FIELD(srtt_us, srtt_us, struct tcp_sock);break ; + +case offsetof(struct bpf_sock_ops , bpf_sock_ops_cb_flags): +SOCK_OPS_GET_FIELD (bpf_sock_ops_cb_flags, bpf_sock_ops_cb_flags, + struct tcp_sock); +break ; }return insn - insn_buf; }
Signature · 1 line
2.9.5
PATCH v8 7/12 bpf: Add sock_ops RTO callback
In reply to Lawrence BrakmoAdds an optional call to sock_ops BPF program based on whether the BPF_SOCK_OPS_RTO_CB_FLAG is set in bpf_sock_ops_flags.
The BPF program is passed 2 arguments: icsk_retransmits and whether the RTO has expired.
--- include/uapi/linux/bpf.h | 5 +++++ include/uapi/linux/tcp.h | 3 ++- net/ipv4/tcp_timer.c | 7 +++++++ 3 files changed, 14 insertions(+), 1 deletion(-)
include/uapi/linux/bpf.h +5 -0
@@ -1014,6 +1014,11 @@ enum {* a congestion threshold. RTTs above * this indicate congestion */ + BPF_SOCK_OPS_RTO_CB, /* Called when an RTO has triggered. +* Arg1: value of icsk_retransmits +* Arg2: value of icsk_rto +* Arg3: whether RTO has expired +*/ };#define TCP_BPF_IW 1001/* Set TCP initial congestion window */
include/uapi/linux/tcp.h +2 -1
@@ -269,7 +269,8 @@ struct tcp_diag_md5sig { };/* Definitions for bpf_sock_ops_cb_flags */ -#define BPF_SOCK_OPS_ALL_CB_FLAGS 0/* Mask of all currently +#define BPF_SOCK_OPS_RTO_CB_FLAG (1<<0) +#define BPF_SOCK_OPS_ALL_CB_FLAGS 0x1/* Mask of all currently * supported cb flags */
net/ipv4/tcp_timer.c +7 -0
@@ -213,11 +213,18 @@ static int tcp_write_timeout(struct sock *sk) icsk->icsk_user_timeout); }tcp_fastopen_active_detect_blackhole (sk, expired); + +if (BPF_SOCK_OPS_TEST_FLAG (tp,BPF_SOCK_OPS_RTO_CB_FLAG )) +tcp_call_bpf_3arg (sk,BPF_SOCK_OPS_RTO_CB , + icsk->icsk_retransmits, + icsk->icsk_rto, (int )expired); +if (expired) {/* Has it gone just too far? */ tcp_write_err (sk);return 1 ; } +return 0 ; }
Signature · 1 line
2.9.5
PATCH v8 8/12 bpf: Add support for reading sk_state and more
In reply to Lawrence BrakmoAdd support for reading many more tcp_sock fields
state, same as sk->sk_state
rtt_min same as sk->rtt_min.s[0].v (current rtt_min)
snd_ssthresh
rcv_nxt
snd_nxt
snd_una
mss_cache
ecn_flags
rate_delivered
rate_interval_us
packets_out
retrans_out
total_retrans
segs_in
data_segs_in
segs_out
data_segs_out
sk_txhash
bytes_received (__u64)
bytes_acked (__u64)
--- include/uapi/linux/bpf.h | 20 +++++++ net/core/filter.c | 135 +++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 144 insertions(+), 11 deletions(-)
include/uapi/linux/bpf.h +20 -0
@@ -979,6 +979,26 @@ struct bpf_sock_ops {__u32 snd_cwnd;__u32 srtt_us;/* Averaged RTT << 3 in usecs */ __u32 bpf_sock_ops_cb_flags;/* flags defined in uapi/linux/tcp.h */ +__u32 state; +__u32 rtt_min; +__u32 snd_ssthresh; +__u32 rcv_nxt; +__u32 snd_nxt; +__u32 snd_una; +__u32 mss_cache; +__u32 ecn_flags; +__u32 rate_delivered; +__u32 rate_interval_us; +__u32 packets_out; +__u32 retrans_out; +__u32 total_retrans; +__u32 segs_in; +__u32 data_segs_in; +__u32 segs_out; +__u32 data_segs_out; +__u32 sk_txhash; +__u64 bytes_received; +__u64 bytes_acked; };/* List of known BPF sock_ops operators.
net/core/filter.c +124 -11
169 lines not on this page · raw message
Signature · 1 line
2.9.5
Re: [PATCH bpf-next v8 08/12] bpf: Add support for reading sk_state and more
Quoted 14 lines · Lawrence Brakmo
Add support for reading many more
tcp_sockfieldsstate, same as
sk->sk_state
rtt_minsame as sk->rtt_min.s[0].v (currentrtt_min)
snd_ssthresh
rcv_nxt
snd_nxt
snd_una
mss_cache
ecn_flags
rate_delivered
rate_interval_us
packets_out
retrans_out
Might as well get ca_state, sacked_out and lost_out to estimate CA states and the packets in flight?
Quoted 225 lines
total_retrans
segs_in
data_segs_in
segs_out
data_segs_out
sk_txhash
bytes_received(__u64)
bytes_acked(__u64)Signed-off-by Lawrence Brakmo <[email protected]>--- include/uapi/linux/bpf.h | 20 +++++++ net/core/filter.c | 135 +++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 144 insertions(+), 11 deletions(-)include/uapi/linux/bpf.h +20
-0@@ -979,6 +979,26 @@ struct bpf_sock_ops {__u32 snd_cwnd;__u32 srtt_us;/* Averaged RTT << 3 in usecs */ __u32 bpf_sock_ops_cb_flags;/* flags defined in uapi/linux/tcp.h */ +__u32 state; +__u32 rtt_min; +__u32 snd_ssthresh; +__u32 rcv_nxt; +__u32 snd_nxt; +__u32 snd_una; +__u32 mss_cache; +__u32 ecn_flags; +__u32 rate_delivered; +__u32 rate_interval_us; +__u32 packets_out; +__u32 retrans_out; +__u32 total_retrans; +__u32 segs_in; +__u32 data_segs_in; +__u32 segs_out; +__u32 data_segs_out; +__u32 sk_txhash; +__u64 bytes_received; +__u64 bytes_acked; };/* List of known BPF sock_ops operators. net/core/filter.c +124
-12@@ -3855,33 +3855,43 @@ void bpf_warn_invalid_xdp_action(u32 act) }EXPORT_SYMBOL_GPL (bpf_warn_invalid_xdp_action); -static bool __is_valid_sock_ops_access (int off,int size) +static bool sock_ops_is_valid_access (int off,int size, +enum bpf_access_type type, +struct bpf_insn_access_aux *info) { +const int size_default =sizeof (__u32); +if (off <0 || off >=sizeof (struct bpf_sock_ops ))return false; +/* The verifier guarantees that size > 0. */ if (off % size !=0 )return false; -if (size !=sizeof (__u32)) -return false; - -return true; -} -static bool sock_ops_is_valid_access (int off,int size, -enum bpf_access_type type, -struct bpf_insn_access_aux *info) -{if (type ==BPF_WRITE ) {switch (off) {case offsetof(struct bpf_sock_ops , reply): +if (size != size_default) +return false;break ;default :return false; } + }else { +switch (off) { + case bpf_ctx_range_till(struct bpf_sock_ops, bytes_received, + bytes_acked): +if (size !=sizeof (__u64)) +return false; +break ; +default : +if (size != size_default) +return false; +break ; + } } -return __is_valid_sock_ops_access (off, size); +return true; } static int sk_skb_prologue(struct bpf_insn *insn_buf, bool direct_write,@@ -4498,6 +4508,32 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, is_fullsock));break ; +case offsetof(struct bpf_sock_ops , state): + BUILD_BUG_ON(FIELD_SIZEOF(struct sock_common, skc_state) != 1); + + *insn++ =BPF_LDX_MEM (BPF_FIELD_SIZEOF ( + struct bpf_sock_ops_kern, sk), + si->dst_reg, si->src_reg, + offsetof(struct bpf_sock_ops_kern , sk)); + *insn++ =BPF_LDX_MEM (BPF_B , si->dst_reg, si->dst_reg, + offsetof(struct sock_common , skc_state)); +break ; + +case offsetof(struct bpf_sock_ops , rtt_min): + BUILD_BUG_ON(FIELD_SIZEOF(struct tcp_sock, rtt_min) != + sizeof(struct minmax)); +BUILD_BUG_ON (sizeof (struct minmax ) < +sizeof (struct minmax_sample )); + + *insn++ =BPF_LDX_MEM (BPF_FIELD_SIZEOF ( + struct bpf_sock_ops_kern, sk), + si->dst_reg, si->src_reg, + offsetof(struct bpf_sock_ops_kern , sk)); + *insn++ =BPF_LDX_MEM (BPF_W , si->dst_reg, si->dst_reg, + offsetof(struct tcp_sock , rtt_min) + + FIELD_SIZEOF(struct minmax_sample, t)); +break ; +/* Helper macro for adding read access to tcp_sock or sock fields. */ #define SOCK_OPS_GET_FIELD (BPF_FIELD ,OBJ_FIELD ,OBJ ) \ do { \@@ -4580,6 +4616,83 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type,SOCK_OPS_GET_FIELD (bpf_sock_ops_cb_flags, bpf_sock_ops_cb_flags, struct tcp_sock);break ; + +case offsetof(struct bpf_sock_ops , snd_ssthresh): + SOCK_OPS_GET_FIELD(snd_ssthresh, snd_ssthresh, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , rcv_nxt): + SOCK_OPS_GET_FIELD(rcv_nxt, rcv_nxt, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , snd_nxt): + SOCK_OPS_GET_FIELD(snd_nxt, snd_nxt, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , snd_una): + SOCK_OPS_GET_FIELD(snd_una, snd_una, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , mss_cache): + SOCK_OPS_GET_FIELD(mss_cache, mss_cache, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , ecn_flags): + SOCK_OPS_GET_FIELD(ecn_flags, ecn_flags, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , rate_delivered): +SOCK_OPS_GET_FIELD (rate_delivered, rate_delivered, + struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , rate_interval_us): +SOCK_OPS_GET_FIELD (rate_interval_us, rate_interval_us, + struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , packets_out): + SOCK_OPS_GET_FIELD(packets_out, packets_out, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , retrans_out): + SOCK_OPS_GET_FIELD(retrans_out, retrans_out, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , total_retrans): +SOCK_OPS_GET_FIELD (total_retrans, total_retrans, + struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , segs_in): + SOCK_OPS_GET_FIELD(segs_in, segs_in, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , data_segs_in): + SOCK_OPS_GET_FIELD(data_segs_in, data_segs_in, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , segs_out): + SOCK_OPS_GET_FIELD(segs_out, segs_out, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , data_segs_out): +SOCK_OPS_GET_FIELD (data_segs_out, data_segs_out, + struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , sk_txhash): + SOCK_OPS_GET_FIELD(sk_txhash, sk_txhash, struct sock); +break ; + +case offsetof(struct bpf_sock_ops , bytes_received): +SOCK_OPS_GET_FIELD (bytes_received, bytes_received, + struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , bytes_acked): + SOCK_OPS_GET_FIELD(bytes_acked, bytes_acked, struct tcp_sock); +break ; }return insn - insn_buf; } --2.9.5
Re: [PATCH bpf-next v8 08/12] bpf: Add support for reading sk_state and more
On 1/24/18, 12:07 PM, "[email protected] on behalf of Yuchung Cheng" <[email protected] on behalf of [email protected]> wrote:
On Tue, Jan 23, 2018 at 11:57 PM, Lawrence Brakmo <[email protected]> wrote: > Add support for reading many more tcp_sock fields > > state, same as sk->sk_state > rtt_min same as sk->rtt_min.s[0].v (current rtt_min) > snd_ssthresh > rcv_nxt > snd_nxt > snd_una > mss_cache > ecn_flags > rate_delivered > rate_interval_us > packets_out > retrans_out Might as well get ca_state, sacked_out and lost_out to estimate CA states and the packets in flight?
Will try to add in updated patchset. If not, I will add as a new patch.
Quoted 225 lines
total_retrans
segs_in
data_segs_in
segs_out
data_segs_out
sk_txhash
bytes_received(__u64)
bytes_acked(__u64)Signed-off-by Lawrence Brakmo <[email protected]>--- include/uapi/linux/bpf.h | 20 +++++++ net/core/filter.c | 135 +++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 144 insertions(+), 11 deletions(-)include/uapi/linux/bpf.h +20
-0@@ -979,6 +979,26 @@ struct bpf_sock_ops {__u32 snd_cwnd;__u32 srtt_us;/* Averaged RTT << 3 in usecs */ __u32 bpf_sock_ops_cb_flags;/* flags defined in uapi/linux/tcp.h */ +__u32 state; +__u32 rtt_min; +__u32 snd_ssthresh; +__u32 rcv_nxt; +__u32 snd_nxt; +__u32 snd_una; +__u32 mss_cache; +__u32 ecn_flags; +__u32 rate_delivered; +__u32 rate_interval_us; +__u32 packets_out; +__u32 retrans_out; +__u32 total_retrans; +__u32 segs_in; +__u32 data_segs_in; +__u32 segs_out; +__u32 data_segs_out; +__u32 sk_txhash; +__u64 bytes_received; +__u64 bytes_acked; };/* List of known BPF sock_ops operators. net/core/filter.c +124
-12@@ -3855,33 +3855,43 @@ void bpf_warn_invalid_xdp_action(u32 act) }EXPORT_SYMBOL_GPL (bpf_warn_invalid_xdp_action); -static bool __is_valid_sock_ops_access (int off,int size) +static bool sock_ops_is_valid_access (int off,int size, +enum bpf_access_type type, +struct bpf_insn_access_aux *info) { +const int size_default =sizeof (__u32); +if (off <0 || off >=sizeof (struct bpf_sock_ops ))return false; +/* The verifier guarantees that size > 0. */ if (off % size !=0 )return false; -if (size !=sizeof (__u32)) -return false; - -return true; -} -static bool sock_ops_is_valid_access (int off,int size, -enum bpf_access_type type, -struct bpf_insn_access_aux *info) -{if (type ==BPF_WRITE ) {switch (off) {case offsetof(struct bpf_sock_ops , reply): +if (size != size_default) +return false;break ;default :return false; } + }else { +switch (off) { + case bpf_ctx_range_till(struct bpf_sock_ops, bytes_received, + bytes_acked): +if (size !=sizeof (__u64)) +return false; +break ; +default : +if (size != size_default) +return false; +break ; + } } -return __is_valid_sock_ops_access (off, size); +return true; } static int sk_skb_prologue(struct bpf_insn *insn_buf, bool direct_write,@@ -4498,6 +4508,32 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, is_fullsock));break ; +case offsetof(struct bpf_sock_ops , state): + BUILD_BUG_ON(FIELD_SIZEOF(struct sock_common, skc_state) != 1); + + *insn++ =BPF_LDX_MEM (BPF_FIELD_SIZEOF ( + struct bpf_sock_ops_kern, sk), + si->dst_reg, si->src_reg, + offsetof(struct bpf_sock_ops_kern , sk)); + *insn++ =BPF_LDX_MEM (BPF_B , si->dst_reg, si->dst_reg, + offsetof(struct sock_common , skc_state)); +break ; + +case offsetof(struct bpf_sock_ops , rtt_min): + BUILD_BUG_ON(FIELD_SIZEOF(struct tcp_sock, rtt_min) != + sizeof(struct minmax)); +BUILD_BUG_ON (sizeof (struct minmax ) < +sizeof (struct minmax_sample )); + + *insn++ =BPF_LDX_MEM (BPF_FIELD_SIZEOF ( + struct bpf_sock_ops_kern, sk), + si->dst_reg, si->src_reg, + offsetof(struct bpf_sock_ops_kern , sk)); + *insn++ =BPF_LDX_MEM (BPF_W , si->dst_reg, si->dst_reg, + offsetof(struct tcp_sock , rtt_min) + + FIELD_SIZEOF(struct minmax_sample, t)); +break ; +/* Helper macro for adding read access to tcp_sock or sock fields. */ #define SOCK_OPS_GET_FIELD (BPF_FIELD ,OBJ_FIELD ,OBJ ) \ do { \@@ -4580,6 +4616,83 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type,SOCK_OPS_GET_FIELD (bpf_sock_ops_cb_flags, bpf_sock_ops_cb_flags, struct tcp_sock);break ; + +case offsetof(struct bpf_sock_ops , snd_ssthresh): + SOCK_OPS_GET_FIELD(snd_ssthresh, snd_ssthresh, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , rcv_nxt): + SOCK_OPS_GET_FIELD(rcv_nxt, rcv_nxt, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , snd_nxt): + SOCK_OPS_GET_FIELD(snd_nxt, snd_nxt, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , snd_una): + SOCK_OPS_GET_FIELD(snd_una, snd_una, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , mss_cache): + SOCK_OPS_GET_FIELD(mss_cache, mss_cache, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , ecn_flags): + SOCK_OPS_GET_FIELD(ecn_flags, ecn_flags, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , rate_delivered): +SOCK_OPS_GET_FIELD (rate_delivered, rate_delivered, + struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , rate_interval_us): +SOCK_OPS_GET_FIELD (rate_interval_us, rate_interval_us, + struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , packets_out): + SOCK_OPS_GET_FIELD(packets_out, packets_out, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , retrans_out): + SOCK_OPS_GET_FIELD(retrans_out, retrans_out, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , total_retrans): +SOCK_OPS_GET_FIELD (total_retrans, total_retrans, + struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , segs_in): + SOCK_OPS_GET_FIELD(segs_in, segs_in, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , data_segs_in): + SOCK_OPS_GET_FIELD(data_segs_in, data_segs_in, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , segs_out): + SOCK_OPS_GET_FIELD(segs_out, segs_out, struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , data_segs_out): +SOCK_OPS_GET_FIELD (data_segs_out, data_segs_out, + struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , sk_txhash): + SOCK_OPS_GET_FIELD(sk_txhash, sk_txhash, struct sock); +break ; + +case offsetof(struct bpf_sock_ops , bytes_received): +SOCK_OPS_GET_FIELD (bytes_received, bytes_received, + struct tcp_sock); +break ; + +case offsetof(struct bpf_sock_ops , bytes_acked): + SOCK_OPS_GET_FIELD(bytes_acked, bytes_acked, struct tcp_sock); +break ; }return insn - insn_buf; } --2.9.5
PATCH v8 9/12 bpf: Add sock_ops R/W access to tclass
In reply to Lawrence BrakmoAdds direct write access to sk_txhash and access to tclass for ipv6 flows through getsockopt and setsockopt. Sample usage for tclass:
bpf_getsockopt(skops, SOL_IPV6, IPV6_TCLASS, &v, sizeof(v))
where skops is a pointer to the ctx (struct bpf_sock_ops).
--- net/core/filter.c | 47 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 45 insertions(+), 2 deletions(-)
net/core/filter.c +45 -2
@@ -3232,6 +3232,29 @@ BPF_CALL_5(bpf_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, ret = -EINVAL ; } #ifdef CONFIG_INET +#if IS_ENABLED(CONFIG_IPV6) + } else if (level == SOL_IPV6) { + if (optlen != sizeof(int) || sk->sk_family != AF_INET6) + return -EINVAL; + + val = *((int *)optval); + /* Only some options are supported */ + switch (optname) { + case IPV6_TCLASS: + if (val < -1 || val > 0xff) { + ret = -EINVAL; + } else { + struct ipv6_pinfo *np = inet6_sk(sk); + + if (val == -1) + val = 0; + np->tclass = val; + } + break; + default: + ret = -EINVAL; + } +#endif } else if (level == SOL_TCP && sk->sk_prot->setsockopt == tcp_setsockopt) { if (optname == TCP_CONGESTION) {
@@ -3241,7 +3264,8 @@ BPF_CALL_5(bpf_setsockopt, struct bpf_sock_ops_kern *, bpf_sock,strncpy (name, optval,min_t (long, optlen,TCP_CA_NAME_MAX -1 )); name[TCP_CA_NAME_MAX -1 ] =0 ; - ret =tcp_set_congestion_control (sk, name, false, reinit); + ret =tcp_set_congestion_control (sk, name, false, + reinit); }else {struct tcp_sock *tp =tcp_sk (sk);
@@ -3307,6 +3331,22 @@ BPF_CALL_5(bpf_getsockopt, struct bpf_sock_ops_kern *, bpf_sock, }else { goto err_clear; } +#if IS_ENABLED (CONFIG_IPV6 ) + } else if (level == SOL_IPV6) { +struct ipv6_pinfo *np =inet6_sk (sk); + +if (optlen !=sizeof (int ) || sk->sk_family !=AF_INET6 ) + goto err_clear; + +/* Only some options are supported */ +switch (optname) { +case IPV6_TCLASS : + *((int *)optval) = (int )np->tclass; +break ; +default : + goto err_clear; + } +#endif }else { goto err_clear; }
@@ -3871,6 +3911,7 @@ static bool sock_ops_is_valid_access(int off, int size,if (type ==BPF_WRITE ) {switch (off) {case offsetof(struct bpf_sock_ops , reply): +case offsetof(struct bpf_sock_ops , sk_txhash):if (size != size_default)return false;break ;
@@ -4682,7 +4723,8 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type,break ;case offsetof(struct bpf_sock_ops , sk_txhash): - SOCK_OPS_GET_FIELD(sk_txhash, sk_txhash, struct sock); +SOCK_OPS_GET_OR_SET_FIELD (sk_txhash, sk_txhash, + struct sock, type);break ;case offsetof(struct bpf_sock_ops , bytes_received):
@@ -4693,6 +4735,7 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type,case offsetof(struct bpf_sock_ops , bytes_acked): SOCK_OPS_GET_FIELD(bytes_acked, bytes_acked, struct tcp_sock);break ; + }return insn - insn_buf; }
Signature · 1 line
2.9.5
PATCH v8 10/12 bpf: Add BPF_SOCK_OPS_RETRANS_CB
In reply to Lawrence BrakmoAdds support for calling sock_ops BPF program when there is a retransmission. Two arguments are used; one for the sequence number and other for the number of segments retransmitted. Does not include syn-ack retransmissions.
New op: BPF_SOCK_OPS_RETRANS_CB.
--- include/uapi/linux/bpf.h | 4 ++++ include/uapi/linux/tcp.h | 3 ++- net/ipv4/tcp_output.c | 3 +++ 3 files changed, 9 insertions(+), 1 deletion(-)
include/uapi/linux/bpf.h +4 -0
@@ -1039,6 +1039,10 @@ enum {* Arg2: value of icsk_rto * Arg3: whether RTO has expired */ + BPF_SOCK_OPS_RETRANS_CB, /* Called when skb is retransmitted. +* Arg1: sequence number of 1st byte +* Arg2: # segments +*/ };#define TCP_BPF_IW 1001/* Set TCP initial congestion window */
include/uapi/linux/tcp.h +2 -1
@@ -270,7 +270,8 @@ struct tcp_diag_md5sig {/* Definitions for bpf_sock_ops_cb_flags */ #define BPF_SOCK_OPS_RTO_CB_FLAG (1<<0) -#define BPF_SOCK_OPS_ALL_CB_FLAGS 0x1/* Mask of all currently +#define BPF_SOCK_OPS_RETRANS_CB_FLAG (1<<1) +#define BPF_SOCK_OPS_ALL_CB_FLAGS 0x3/* Mask of all currently * supported cb flags */
net/ipv4/tcp_output.c +3 -0
@@ -2908,6 +2908,9 @@ int __tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb, int segs)if (likely (!err)) {TCP_SKB_CB (skb)->sacked |=TCPCB_EVER_RETRANS ;trace_tcp_retransmit_skb (sk, skb); +if (BPF_SOCK_OPS_TEST_FLAG (tp,BPF_SOCK_OPS_RETRANS_CB_FLAG )) +tcp_call_bpf_2arg (sk,BPF_SOCK_OPS_RETRANS_CB , +TCP_SKB_CB (skb)->seq, segs); }else if (err != -EBUSY ) {NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPRETRANSFAIL ); }
Signature · 1 line
2.9.5
Re: [PATCH bpf-next v8 10/12] bpf: Add BPF_SOCK_OPS_RETRANS_CB
Quoted 48 lines · Lawrence Brakmo
Adds support for calling
sock_opsBPF program when there is a retransmission. Two arguments are used; one for the sequence number and other for the number of segments retransmitted. Does not include syn-ack retransmissions.New op:
BPF_SOCK_OPS_RETRANS_CB.Signed-off-by Lawrence Brakmo <[email protected]>--- include/uapi/linux/bpf.h | 4 ++++ include/uapi/linux/tcp.h | 3 ++- net/ipv4/tcp_output.c | 3 +++ 3 files changed, 9 insertions(+), 1 deletion(-)include/uapi/linux/bpf.h +4
-0@@ -1039,6 +1039,10 @@ enum {* Arg2: value of icsk_rto * Arg3: whether RTO has expired */ + BPF_SOCK_OPS_RETRANS_CB, /* Called when skb is retransmitted. +* Arg1: sequence number of 1st byte +* Arg2: # segments +*/ };#define TCP_BPF_IW 1001/* Set TCP initial congestion window */ include/uapi/linux/tcp.h +2
-1@@ -270,7 +270,8 @@ struct tcp_diag_md5sig {/* Definitions for bpf_sock_ops_cb_flags */ #define BPF_SOCK_OPS_RTO_CB_FLAG (1<<0) -#define BPF_SOCK_OPS_ALL_CB_FLAGS 0x1/* Mask of all currently +#define BPF_SOCK_OPS_RETRANS_CB_FLAG (1<<1) +#define BPF_SOCK_OPS_ALL_CB_FLAGS 0x3/* Mask of all currently * supported cb flags */ net/ipv4/tcp_output.c +0
-0
net/ipv4/tcp_output.c int __tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb, int segs) · line 2913
Quoted 6 lines
net/ipv4/tcp_output.c +3
-0@@ -2908,6 +2908,9 @@ int __tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb, int segs)if (likely (!err)) {TCP_SKB_CB (skb)->sacked |=TCPCB_EVER_RETRANS ;trace_tcp_retransmit_skb (sk, skb); +if (BPF_SOCK_OPS_TEST_FLAG (tp,BPF_SOCK_OPS_RETRANS_CB_FLAG )) +tcp_call_bpf_2arg (sk,BPF_SOCK_OPS_RETRANS_CB , +TCP_SKB_CB (skb)->seq, segs);
Any reason to skip failed retransmission? I would think that's helpful as well.
Quoted 6 lines
net/ipv4/tcp_output.c +0
-1}else if (err != -EBUSY ) {NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPRETRANSFAIL ); } --2.9.5
Re: [PATCH bpf-next v8 10/12] bpf: Add BPF_SOCK_OPS_RETRANS_CB
On 1/24/18, 12:02 PM, "[email protected] on behalf of Yuchung Cheng" <[email protected] on behalf of [email protected]> wrote:
On Tue, Jan 23, 2018 at 11:58 PM, Lawrence Brakmo <[email protected]> wrote: > Adds support for calling sock_ops BPF program when there is a > retransmission. Two arguments are used; one for the sequence number and > other for the number of segments retransmitted. Does not include syn-ack > retransmissions. > > New op: BPF_SOCK_OPS_RETRANS_CB. > > Signed-off-by: Lawrence Brakmo <[email protected]> > --- > include/uapi/linux/bpf.h | 4 ++++ > include/uapi/linux/tcp.h | 3 ++- > net/ipv4/tcp_output.c | 3 +++ > 3 files changed, 9 insertions(+), 1 deletion(-) > > diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h > index 6998032..eb26cdb 100644 > --- a/include/uapi/linux/bpf.h > +++ b/include/uapi/linux/bpf.h > @@ -1039,6 +1039,10 @@ enum { > * Arg2: value of icsk_rto > * Arg3: whether RTO has expired > */ > + BPF_SOCK_OPS_RETRANS_CB, /* Called when skb is retransmitted. > + * Arg1: sequence number of 1st byte > + * Arg2: # segments > + */ > }; > > #define TCP_BPF_IW 1001 /* Set TCP initial congestion window */ > diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h > index 129032ca..ec03a2b 100644 > --- a/include/uapi/linux/tcp.h > +++ b/include/uapi/linux/tcp.h > @@ -270,7 +270,8 @@ struct tcp_diag_md5sig { > > /* Definitions for bpf_sock_ops_cb_flags */ > #define BPF_SOCK_OPS_RTO_CB_FLAG (1<<0) > -#define BPF_SOCK_OPS_ALL_CB_FLAGS 0x1 /* Mask of all currently > +#define BPF_SOCK_OPS_RETRANS_CB_FLAG (1<<1) > +#define BPF_SOCK_OPS_ALL_CB_FLAGS 0x3 /* Mask of all currently > * supported cb flags > */ > > diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c > index d12f7f7..f7d34f01 100644 > --- a/net/ipv4/tcp_output.c > +++ b/net/ipv4/tcp_output.c > @@ -2908,6 +2908,9 @@ int __tcp_retransmit_skb(struct sock *sk, struct sk_buff *skb, int segs) > if (likely(!err)) { > TCP_SKB_CB(skb)->sacked |= TCPCB_EVER_RETRANS; > trace_tcp_retransmit_skb(sk, skb); > + if (BPF_SOCK_OPS_TEST_FLAG(tp, BPF_SOCK_OPS_RETRANS_CB_FLAG)) > + tcp_call_bpf_2arg(sk, BPF_SOCK_OPS_RETRANS_CB, > + TCP_SKB_CB(skb)->seq, segs); Any reason to skip failed retransmission? I would think that's helpful as well.
Good point, thanks Yuchung. I will do a new patch shortly that will also pass the err value to the BPF program.
Quoted 6 lines
} else if (err != -EBUSY) { NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPRETRANSFAIL); }--
2.9.5
PATCH v8 11/12 bpf: Add BPF_SOCK_OPS_STATE_CB
In reply to Lawrence BrakmoAdds support for calling sock_ops BPF program when there is a TCP state change. Two arguments are used; one for the old state and another for the new state.
There is a new enum in include/uapi/linux/bpf.h that exports the TCP states that prepends BPF_ to the current TCP state names. If it is ever necessary to change the internal TCP state values (other than adding more to the end), then it will become necessary to convert from the internal TCP state value to the BPF value before calling the BPF sock_ops function. There are a set of compile checks added in tcp.c to detect if the internal and BPF values differ so we can make the necessary fixes.
New op: BPF_SOCK_OPS_STATE_CB.
--- include/uapi/linux/bpf.h | 26 ++++++++++++++++++++++++++ include/uapi/linux/tcp.h | 3 ++- net/ipv4/tcp.c | 24 ++++++++++++++++++++++++ 3 files changed, 52 insertions(+), 1 deletion(-)
include/uapi/linux/bpf.h +26 -0
@@ -1043,6 +1043,32 @@ enum {* Arg1: sequence number of 1st byte * Arg2: # segments */ + BPF_SOCK_OPS_STATE_CB, /* Called when TCP changes state. +* Arg1: old_state +* Arg2: new_state +*/ +}; + +/* List of TCP states. There is a build check in net/ipv4/tcp.c to detect +* changes between the TCP and BPF versions. Ideally this should never happen. +* If it does, we need to add code to convert them before calling +* the BPF sock_ops function. +*/ +enum { +BPF_TCP_ESTABLISHED =1 , +BPF_TCP_SYN_SENT , +BPF_TCP_SYN_RECV , +BPF_TCP_FIN_WAIT1 , +BPF_TCP_FIN_WAIT2 , +BPF_TCP_TIME_WAIT , +BPF_TCP_CLOSE , +BPF_TCP_CLOSE_WAIT , +BPF_TCP_LAST_ACK , +BPF_TCP_LISTEN , +BPF_TCP_CLOSING ,/* Now a valid state */ +BPF_TCP_NEW_SYN_RECV , + +BPF_TCP_MAX_STATES /* Leave at the end! */ };#define TCP_BPF_IW 1001/* Set TCP initial congestion window */
include/uapi/linux/tcp.h +2 -1
@@ -271,7 +271,8 @@ struct tcp_diag_md5sig {/* Definitions for bpf_sock_ops_cb_flags */ #define BPF_SOCK_OPS_RTO_CB_FLAG (1<<0)#define BPF_SOCK_OPS_RETRANS_CB_FLAG (1<<1) -#define BPF_SOCK_OPS_ALL_CB_FLAGS 0x3/* Mask of all currently +#define BPF_SOCK_OPS_STATE_CB_FLAG (1<<2) +#define BPF_SOCK_OPS_ALL_CB_FLAGS 0x7/* Mask of all currently * supported cb flags */
net/ipv4/tcp.c +24 -0
@@ -2042,6 +2042,30 @@ void tcp_set_state(struct sock *sk, int state) {int oldstate = sk->sk_state; +/* We defined a new enum for TCP states that are exported in BPF +* so as not force the internal TCP states to be frozen. The +* following checks will detect if an internal state value ever +* differs from the BPF value. If this ever happens, then we will +* need to remap the internal value to the BPF value before calling +* tcp_call_bpf_2arg. +*/ +BUILD_BUG_ON ((int )BPF_TCP_ESTABLISHED != (int )TCP_ESTABLISHED ); +BUILD_BUG_ON ((int )BPF_TCP_SYN_SENT != (int )TCP_SYN_SENT ); +BUILD_BUG_ON ((int )BPF_TCP_SYN_RECV != (int )TCP_SYN_RECV ); +BUILD_BUG_ON ((int )BPF_TCP_FIN_WAIT1 != (int )TCP_FIN_WAIT1 ); +BUILD_BUG_ON ((int )BPF_TCP_FIN_WAIT2 != (int )TCP_FIN_WAIT2 ); +BUILD_BUG_ON ((int )BPF_TCP_TIME_WAIT != (int )TCP_TIME_WAIT ); +BUILD_BUG_ON ((int )BPF_TCP_CLOSE != (int )TCP_CLOSE ); +BUILD_BUG_ON ((int )BPF_TCP_CLOSE_WAIT != (int )TCP_CLOSE_WAIT ); +BUILD_BUG_ON ((int )BPF_TCP_LAST_ACK != (int )TCP_LAST_ACK ); +BUILD_BUG_ON ((int )BPF_TCP_LISTEN != (int )TCP_LISTEN ); +BUILD_BUG_ON ((int )BPF_TCP_CLOSING != (int )TCP_CLOSING ); +BUILD_BUG_ON ((int )BPF_TCP_NEW_SYN_RECV != (int )TCP_NEW_SYN_RECV ); +BUILD_BUG_ON ((int )BPF_TCP_MAX_STATES != (int )TCP_MAX_STATES ); + +if (BPF_SOCK_OPS_TEST_FLAG (tcp_sk (sk),BPF_SOCK_OPS_STATE_CB_FLAG )) +tcp_call_bpf_2arg (sk,BPF_SOCK_OPS_STATE_CB , oldstate, state); +switch (state) {case TCP_ESTABLISHED :if (oldstate !=TCP_ESTABLISHED )
Signature · 1 line
2.9.5
PATCH v8 12/12 bpf: add selftest for tcpbpf
In reply to Lawrence BrakmoAdded a selftest for tcpbpf (sock_ops) that checks that the appropriate callbacks occured and that it can access tcp_sock fields and that their values are correct.
Run with command: ./test_tcpbpf_user
--- tools/include/uapi/linux/bpf.h | 74 +++++++++++++- tools/testing/selftests/bpf/Makefile | 4 +- tools/testing/selftests/bpf/bpf_helpers.h | 2 + tools/testing/selftests/bpf/tcp_client.py | 52 ++++++++++ tools/testing/selftests/bpf/tcp_server.py | 79 +++++++++++++++ tools/testing/selftests/bpf/test_tcpbpf.h | 16 +++ tools/testing/selftests/bpf/test_tcpbpf_kern.c | 131 +++++++++++++++++++++++++ tools/testing/selftests/bpf/test_tcpbpf_user.c | 126 ++++++++++++++++++++++++ 8 files changed, 478 insertions(+), 6 deletions(-) create mode 100755 tools/testing/selftests/bpf/tcp_client.py create mode 100755 tools/testing/selftests/bpf/tcp_server.py create mode 100644 tools/testing/selftests/bpf/test_tcpbpf.h create mode 100644 tools/testing/selftests/bpf/test_tcpbpf_kern.c create mode 100644 tools/testing/selftests/bpf/test_tcpbpf_user.c
tools/include/uapi/linux/bpf.h +70 -4
@@ -17,7 +17,7 @@#define BPF_ALU64 0x07/* alu mode in double word width */ /* ld/ldx fields */ -#define BPF_DW 0x18/* double word */ +#define BPF_DW 0x18/* double word (64-bit) */ #define BPF_XADD 0xc0/* exclusive add */ /* alu/jmp fields */
@@ -642,6 +642,14 @@ union bpf_attr {* @optlen: length of optval in bytes * Return: 0 or negative error * +* int bpf_sock_ops_cb_flags_set(bpf_sock_ops, flags) +* Set callback flags for sock_ops +* @bpf_sock_ops: pointer to bpf_sock_ops_kern struct +* @flags: flags value +* Return: 0 for no error +* -EINVAL if there is no full tcp socket +* bits in flags that are not supported by current kernel +* * int bpf_skb_adjust_room(skb, len_diff, mode, flags) * Grow or shrink room in sk_buff. * @skb: pointer to skb
@@ -748,7 +756,8 @@ union bpf_attr {FN (perf_event_read_value), \FN (perf_prog_read_value), \FN (getsockopt), \ -FN (override_return), +FN (override_return), \ +FN (sock_ops_cb_flags_set),/* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call
@@ -952,8 +961,9 @@ struct bpf_map_info {struct bpf_sock_ops {__u32 op;union { -__u32 reply; -__u32 replylong[4 ]; +__u32 args[4 ];/* Optionally passed to bpf program */ +__u32 reply;/* Returned by bpf program */ +__u32 replylong[4 ];/* Optionally returned by bpf prog */ };__u32 family;__u32 remote_ip4;/* Stored in network byte order */
@@ -968,6 +978,27 @@ struct bpf_sock_ops {*/ __u32 snd_cwnd;__u32 srtt_us;/* Averaged RTT << 3 in usecs */ +__u32 bpf_sock_ops_cb_flags;/* flags defined in uapi/linux/tcp.h */ +__u32 state; +__u32 rtt_min; +__u32 snd_ssthresh; +__u32 rcv_nxt; +__u32 snd_nxt; +__u32 snd_una; +__u32 mss_cache; +__u32 ecn_flags; +__u32 rate_delivered; +__u32 rate_interval_us; +__u32 packets_out; +__u32 retrans_out; +__u32 total_retrans; +__u32 segs_in; +__u32 data_segs_in; +__u32 segs_out; +__u32 data_segs_out; +__u32 sk_txhash; +__u64 bytes_received; +__u64 bytes_acked; };/* List of known BPF sock_ops operators.
@@ -1003,6 +1034,41 @@ enum {* a congestion threshold. RTTs above * this indicate congestion */ +BPF_SOCK_OPS_RTO_CB ,/* Called when an RTO has triggered. +* Arg1: value of icsk_retransmits +* Arg2: value of icsk_rto +* Arg3: whether RTO has expired +*/ +BPF_SOCK_OPS_RETRANS_CB ,/* Called when skb is retransmitted. +* Arg1: sequence number of 1st byte +* Arg2: # segments +*/ + BPF_SOCK_OPS_STATE_CB, /* Called when TCP changes state. + * Arg1: old_state + * Arg2: new_state + */ +}; + +/* List of TCP states. There is a build check in net/ipv4/tcp.c to detect +* changes between the TCP and BPF versions. Ideally this should never happen. +* If it does, we need to add code to convert them before calling +* the BPF sock_ops function. +*/ +enum { +BPF_TCP_ESTABLISHED =1 , +BPF_TCP_SYN_SENT , +BPF_TCP_SYN_RECV , +BPF_TCP_FIN_WAIT1 , +BPF_TCP_FIN_WAIT2 , +BPF_TCP_TIME_WAIT , +BPF_TCP_CLOSE , +BPF_TCP_CLOSE_WAIT , +BPF_TCP_LAST_ACK , +BPF_TCP_LISTEN , +BPF_TCP_CLOSING ,/* Now a valid state */ +BPF_TCP_NEW_SYN_RECV , + +BPF_TCP_MAX_STATES /* Leave at the end! */ };#define TCP_BPF_IW 1001/* Set TCP initial congestion window */
tools/testing/selftests/bpf/Makefile +2 -2
@@ -14,13 +14,13 @@ CFLAGS += -Wall -O2 -I$(APIDIR) -I$(LIBDIR) -I$(GENDIR) $(GENFLAGS) -I../../../i LDLIBS += -lcap -lelf -lrt TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test_progs \ - test_align test_verifier_log test_dev_cgroup + test_align test_verifier_log test_dev_cgroup test_tcpbpf_user TEST_GEN_FILES = test_pkt_access.o test_xdp.o test_l4lb.o test_tcp_estats.o test_obj_id.o \ test_pkt_md_access.o test_xdp_redirect.o test_xdp_meta.o sockmap_parse_prog.o \ sockmap_verdict_prog.o dev_cgroup.o sample_ret0.o test_tracepoint.o \ test_l4lb_noinline.o test_xdp_noinline.o test_stacktrace_map.o \ - sample_map_ret0.o + sample_map_ret0.o test_tcpbpf_kern.o TEST_PROGS := test_kmod.sh test_xdp_redirect.sh test_xdp_meta.sh \ test_offload.py
tools/testing/selftests/bpf/bpf_helpers.h +2 -0
@@ -71,6 +71,8 @@ static int (*bpf_setsockopt)(void *ctx, int level, int optname, void *optval,static int (*bpf_getsockopt)(void *ctx,int level,int optname,void *optval,int optlen) = (void *) BPF_FUNC_getsockopt; +static int (*bpf_sock_ops_cb_flags_set)(void *ctx,int flags) = + (void *) BPF_FUNC_sock_ops_cb_flags_set;static int (*bpf_sk_redirect_map)(void *ctx,void *map,int key,int flags) = (void *) BPF_FUNC_sk_redirect_map; static int (*bpf_sock_map_update)(void *map, void *key, void *value,
tools/testing/selftests/bpf/tcp_client.py new file +52 -0
52 lines not on this page · raw message
tools/testing/selftests/bpf/tcp_server.py new file +79 -0
79 lines not on this page · raw message
tools/testing/selftests/bpf/test_tcpbpf.h new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 + +#ifndef _TEST_TCPBPF_H +#define _TEST_TCPBPF_H + +struct tcpbpf_globals { +__u32 event_map; +__u32 total_retrans; +__u32 data_segs_in; +__u32 data_segs_out; +__u32 bad_cb_test_rv; +__u32 good_cb_test_rv; +__u64 bytes_received; +__u64 bytes_acked; +}; +#endif
tools/testing/selftests/bpf/test_tcpbpf_kern.c new file +131 -0
131 lines not on this page · raw message
tools/testing/selftests/bpf/test_tcpbpf_user.c new file +126 -0
126 lines not on this page · raw message
Signature · 1 line
2.9.5
Quoted 25 lines · Lawrence Brakmo
This patchset adds support for:
- direct R or R/W access to many
tcp_sockfields
- passing up to 4 arguments tosock_opsBPF functions
-tcp_sockfieldbpf_sock_ops_cb_flagsfor controlling callbacks
- optionally callingsock_opsBPF program when RTO fires
- optionally callingsock_opsBPF program when packet is retransmitted
- optionally callingsock_opsBPF program when TCP state changes
- access to tclass andsk_txhash
- new selftestv2: Fixed commit message 0/11. The commit is to "bpf-next" but the patch
below used "bpf" and Patchwork didn't work correctly.
v3: Cleaned RTO callback as per Yuchung's comment
Added BPF enum for TCP states as per Alexei's comment
v4: Fixed compile warnings related to detecting changes between TCP
internal states and the BPF defined states.
v5: Fixed comment issues in some selftest files
Fixed accesss issue with u64 fields inbpf_sock_opsstruct
v6: Made fixes based on comments form Eric Dumazet:
The fieldbpf_sock_ops_cb_flagswas addded in a hole on 64bit kernels Fieldbpf_sock_ops_cb_flagsis now set through a helper function which returns an error when a BPF program tries to set bits for callbacks that are not supported in the current kernel. Added a comment indicating that when adding fields tobpf_sock_ops_kern
Quoted 8 lines
they should be added before the field named "temp" if they need to be cleared before calling the BPF function.
v7: Enfornced fields "op" and "replylong[1] .. replylong[3]" not be writable
based on comments form Eric Dumazet and Alexei Starovoitov. Filled 32 bit hole inbpf_sock_opsstruct withsk_txhashbased on comments from Daniel Borkmann.
Removed unused functions (tcp_call_bpf_1arg,tcp_call_bpf_4arg) based on comments from Daniel Borkmann.
Please independently lets fix the congestion control bug before pushing more stuff.
Fix needs to be backported to 4.13, so would target net tree.
Then your new functionalities can target net-next
By mixing all the stuff, you make stable trees maintainer life more difficult than needed.
Thanks.
Quoted 38 lines · Eric Dumazet
Quoted 25 lines · Lawrence Brakmo
This patchset adds support for:
- direct R or R/W access to many
tcp_sockfields
- passing up to 4 arguments tosock_opsBPF functions
-tcp_sockfieldbpf_sock_ops_cb_flagsfor controlling callbacks
- optionally callingsock_opsBPF program when RTO fires
- optionally callingsock_opsBPF program when packet is retransmitted
- optionally callingsock_opsBPF program when TCP state changes
- access to tclass andsk_txhash
- new selftestv2: Fixed commit message 0/11. The commit is to "bpf-next" but the patch
below used "bpf" and Patchwork didn't work correctly.
v3: Cleaned RTO callback as per Yuchung's comment
Added BPF enum for TCP states as per Alexei's comment
v4: Fixed compile warnings related to detecting changes between TCP
internal states and the BPF defined states.
v5: Fixed comment issues in some selftest files
Fixed accesss issue with u64 fields inbpf_sock_opsstruct
v6: Made fixes based on comments form Eric Dumazet:
The fieldbpf_sock_ops_cb_flagswas addded in a hole on 64bit kernels Fieldbpf_sock_ops_cb_flagsis now set through a helper function which returns an error when a BPF program tries to set bits for callbacks that are not supported in the current kernel. Added a comment indicating that when adding fields tobpf_sock_ops_kernQuoted 8 lines
they should be added before the field named "temp" if they need to be cleared before calling the BPF function.
v7: Enfornced fields "op" and "replylong[1] .. replylong[3]" not be writable
based on comments form Eric Dumazet and Alexei Starovoitov. Filled 32 bit hole inbpf_sock_opsstruct withsk_txhashbased on comments from Daniel Borkmann.
Removed unused functions (tcp_call_bpf_1arg,tcp_call_bpf_4arg) based on comments from Daniel Borkmann.Please independently lets fix the congestion control bug before pushing more stuff.
Quoted 8 lines
Fix needs to be backported to 4.13, so would target net tree.
Fixes 91b5b21c7c16 ("bpf: Add support for changing congestion control")Then your new functionalities can target net-next
By mixing all the stuff, you make stable trees maintainer life more difficult than needed.
Most of the time, Yes, but it's the other way around this time. I specifically asked Larry to do it this way, since net tree is practically closed (only critical fixes allowed). When 4.15 is released on Sunday we'll send this patch independently to 4.15 and 4.14
Quoted 7 lines · Alexei Starovoitov
Most of the time, Yes, but it's the other way around this time. I specifically asked Larry to do it this way, since net tree is practically closed (only critical fixes allowed). When 4.15 is released on Sunday we'll send this patch independently to 4.15 and 4.14
How hard would it be to put the fix first in the series then ? If this proves complex, then maybe we have a bigger problem.
I did not know merging bpf into bpf-next would be hard :)
Thanks.
Quoted 11 lines · Eric Dumazet
Quoted 7 lines · Alexei Starovoitov
Most of the time, Yes, but it's the other way around this time. I specifically asked Larry to do it this way, since net tree is practically closed (only critical fixes allowed). When 4.15 is released on Sunday we'll send this patch independently to 4.15 and 4.14
How hard would it be to put the fix first in the series then ? If this proves complex, then maybe we have a bigger problem.
you mean to put patch 4 to be first in the series ? I don't think that matters. It applies fine as-is on net tree and builds without issues. I only need to double check that nothing else needed before adding it to stable queue.
Quoted 1 line
I did not know merging bpf into bpf-next would be hard :)
yeah :) It's double pain for us too due to backports into our kernels.
I think the speed of bpf development and attention from the bigger kernel community increased this cycle a lot. Hopefully next release things will calm down.
Quoted 10 lines · Alexei Starovoitov
Quoted 9 lines · Eric Dumazet
Quoted 7 lines · Alexei Starovoitov
Most of the time, Yes, but it's the other way around this time. I specifically asked Larry to do it this way, since net tree is practically closed (only critical fixes allowed). When 4.15 is released on Sunday we'll send this patch independently to 4.15 and 4.14
Quoted 7 lines
Quoted 2 lines
How hard would it be to put the fix first in the series then ? If this proves complex, then maybe we have a bigger problem.
you mean to put patch 4 to be first in the series ? I don't think that matters. It applies fine as-is on net tree and builds without issues. I only need to double check that nothing else needed before adding it to stable queue.
Yes, this is exactly what we need.
We had a major issue with this bug, and we are lucky that we disabled unpriv eBPF for other reasons.