net/smc: Introduce smc_ops
Subsystems (11)
D. Wythe
maintainerv5 cover net/smc: Introduce smc_ops
This patch aims to introduce BPF injection capabilities for SMC and includes a self-test to ensure code stability.
Since the SMC protocol isn't ideal for every situation, especially short-lived ones, most applications can't guarantee the absence of such scenarios. Consequently, applications may need specific strategies to decide whether to use SMC. For example, an application might limit SMC usage to certain IP addresses or ports.
To maintain the principle of transparent replacement, we want applications to remain unaffected even if they need specific SMC strategies. In other words, they should not require recompilation of their code.
Additionally, we need to ensure the scalability of strategy implementation. While using socket options or sysctl might be straightforward, it could complicate future expansions.
Fortunately, BPF addresses these concerns effectively. Users can write their own strategies in eBPF to determine whether to use SMC, and they can easily modify those strategies in the future.
v2:
1. Rename smc_bpf_ops to smc_ops.
2. Change the scope of smc_ops from global to per netns.
3. Directly pass parameters to ops instead of smc_ops_ctx.
4. Remove struct smc_ops_ctx.
5. Remove exports that are no longer needed.
v3:
1. Remove find_ksym_btf_id_by_prefix_kind.
2. Enhance selftest, introduce a complete ops for filtering smc connections based on ip pairs and a realistic topology test to verify it.
v4:
1. Remove unless func: smc_bpf_ops_check_member()
2. Remove unless inline func: smc_ops_find_by_name()
3. Change CONFIG_SMC=y to complete CI testing
4. Change smc_sock to smc_sock___local in test to avoid compiling failed with CONFIG_SMC=y
5. Improve test cases, remove unnecessary timeouts and multi-thread test, using network_helpers to start testing between server and client.
6. Fix issues when the return value of the ops function is neither 0
nor 1.
v5:
1. Fix incorrect CI config from CONFIG_SMC=Y to CONFIG_SMC=y.
D. Wythe (5):
bpf: export necessary sympols for modules with struct_ops net/smc: Introduce generic hook smc_ops
net/smc: bpf: register smc_ops info struct_ops
libbpf: fix error when st-prefix_ops and ops from differ btf bpf/selftests: add selftest for bpf_smc_ops
include/net/netns/smc.h | 3 + include/net/smc.h | 51 +++ kernel/bpf/bpf_struct_ops.c | 2 + kernel/bpf/syscall.c | 1 + net/ipv4/tcp_output.c | 15 +- net/smc/Kconfig | 12 + net/smc/Makefile | 1 + net/smc/af_smc.c | 10 + net/smc/smc_ops.c | 130 ++++++ net/smc/smc_ops.h | 30 ++ net/smc/smc_sysctl.c | 95 +++++ tools/lib/bpf/libbpf.c | 25 +- tools/testing/selftests/bpf/config | 4 + .../selftests/bpf/prog_tests/test_bpf_smc.c | 390 ++++++++++++++++++ tools/testing/selftests/bpf/progs/bpf_smc.c | 116 ++++++ 15 files changed, 873 insertions(+), 12 deletions(-) create mode 100644 net/smc/smc_ops.c create mode 100644 net/smc/smc_ops.h create mode 100644 tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_smc.c
Signature · 1 line
2.45.0
D. Wythe
maintainerPATCH v5 1/5 bpf: export necessary sympols for modules with struct_ops
Exports three necessary symbols for implementing struct_ops with tristate subsystem.
To hold or release refcnt of struct_ops refcnt by inline funcs bpf_try_module_get and bpf_module_put which use bpf_struct_ops_get(put) conditionally.
And to copy obj name from one to the other with effective checks by bpf_obj_name_cpy.
--- kernel/bpf/bpf_struct_ops.c | 2 ++ kernel/bpf/syscall.c | 1 + 2 files changed, 3 insertions(+)
kernel/bpf/bpf_struct_ops.c +2 -0
@@ -1119,6 +1119,7 @@ bool bpf_struct_ops_get(const void *kdata) map =__bpf_map_inc_not_zero (&st_map->map, false);return !IS_ERR (map); } +EXPORT_SYMBOL_GPL (bpf_struct_ops_get);void bpf_struct_ops_put (const void *kdata) {
@@ -1130,6 +1131,7 @@ void bpf_struct_ops_put(const void *kdata)bpf_map_put (&st_map->map); } +EXPORT_SYMBOL_GPL (bpf_struct_ops_put);int bpf_struct_ops_supported (const struct bpf_struct_ops *st_ops,u32 moff) {
kernel/bpf/syscall.c +1 -0
@@ -1167,6 +1167,7 @@ int bpf_obj_name_cpy(char *dst, const char *src, unsigned int size)return src - orig_src; } +EXPORT_SYMBOL_GPL (bpf_obj_name_cpy); int map_check_no_btf(const struct bpf_map *map, const struct btf *btf,
Signature · 1 line
2.45.0
D. Wythe
maintainerPATCH v5 2/5 net/smc: Introduce generic hook smc_ops
In reply to D. WytheThe introduction of IPPROTO_SMC enables eBPF programs to determine whether to use SMC based on the context of socket creation, such as network namespaces, PID and comm name, etc.
As a subsequent enhancement, to introduce a new generic hook that allows decisions on whether to use SMC or not at runtime, including but not limited to local/remote IP address or ports.
Moreover, in the future, we can achieve more complex extensions to the protocol stack by extending this ops.
--- include/net/netns/smc.h | 3 ++ include/net/smc.h | 51 ++++++++++++++++++++++ net/ipv4/tcp_output.c | 15 +++++-- net/smc/Kconfig | 12 ++++++ net/smc/Makefile | 1 + net/smc/smc_ops.c | 51 ++++++++++++++++++++++ net/smc/smc_ops.h | 25 +++++++++++ net/smc/smc_sysctl.c | 95 +++++++++++++++++++++++++++++++++++++++++ 8 files changed, 249 insertions(+), 4 deletions(-) create mode 100644 net/smc/smc_ops.c create mode 100644 net/smc/smc_ops.h
include/net/netns/smc.h +3 -0
@@ -17,6 +17,9 @@ struct netns_smc {#ifdef CONFIG_SYSCTL struct ctl_table_header *smc_hdr;#endif +#if IS_ENABLED (CONFIG_SMC_OPS ) +struct smc_ops __rcu *ops; +#endif /* CONFIG_SMC_OPS */ unsigned int sysctl_autocorking_size;unsigned int sysctl_smcr_buf_type;int sysctl_smcr_testlink_time;
include/net/smc.h +51 -0
61 lines not on this page · raw message
net/ipv4/tcp_output.c +11 -4
@@ -48,6 +48,7 @@#include <linux/skbuff_ref.h> #include <trace/events/tcp.h> +#include <net/smc.h> /* Refresh clocks of a TCP socket, * ensuring monotically increasing values.
@@ -759,14 +760,17 @@ static void tcp_options_write(struct tcphdr *th, struct tcp_sock *tp,mptcp_options_write (th, ptr, tp, opts); } -static void smc_set_option (const struct tcp_sock *tp, +static void smc_set_option (struct tcp_sock *tp,struct tcp_out_options *opts,unsigned int *remaining) {#if IS_ENABLED (CONFIG_SMC )if (static_branch_unlikely (&tcp_have_smc)) {if (tp->syn_smc) { -if (*remaining >=TCPOLEN_EXP_SMC_BASE_ALIGNED ) { + tp->syn_smc =smc_call_retops (1 , &tp->inet_conn.icsk_inet.sk, + set_option, tp); +/* re-check syn_smc */ +if (tp->syn_smc && *remaining >=TCPOLEN_EXP_SMC_BASE_ALIGNED ) { opts->options |=OPTION_SMC ; *remaining -=TCPOLEN_EXP_SMC_BASE_ALIGNED ; }
@@ -776,14 +780,17 @@ static void smc_set_option(const struct tcp_sock *tp, }static void smc_set_option_cond (const struct tcp_sock *tp, -const struct inet_request_sock *ireq, +struct inet_request_sock *ireq,struct tcp_out_options *opts,unsigned int *remaining) {#if IS_ENABLED (CONFIG_SMC )if (static_branch_unlikely (&tcp_have_smc)) {if (tp->syn_smc && ireq->smc_ok) { -if (*remaining >=TCPOLEN_EXP_SMC_BASE_ALIGNED ) { + ireq->smc_ok =smc_call_retops (1 , &tp->inet_conn.icsk_inet.sk, + set_option_cond, tp, ireq); +/* re-check smc_ok */ +if (ireq->smc_ok && *remaining >=TCPOLEN_EXP_SMC_BASE_ALIGNED ) { opts->options |=OPTION_SMC ; *remaining -=TCPOLEN_EXP_SMC_BASE_ALIGNED ; }
net/smc/Kconfig +12 -0
@@ -33,3 +33,15 @@ config SMC_LO of architecture or hardware. if unsure, say N. + +config SMC_OPS + bool "Generic hook for SMC subsystem" + depends on SMC && BPF_SYSCALL + default n + help + SMC_OPS enables support to register genericfor hook via eBPF programs + for SMC subsystem. eBPF programs offer much greater flexibility + in modifying the behavior of the SMC protocol stack compared + to a complete kernel-based approach. + + if unsure, say N.
net/smc/Makefile +1 -0
@@ -7,3 +7,4 @@ smc-y += smc_cdc.o smc_tx.o smc_rx.o smc_close.o smc_ism.o smc_netlink.o smc_sta smc-y += smc_tracepoint.o smc_inet.o smc-$(CONFIG_SYSCTL) += smc_sysctl.o smc-$(CONFIG_SMC_LO) += smc_loopback.o +smc-$(CONFIG_SMC_OPS) += smc_ops.o
net/smc/smc_ops.c new file +51 -0
51 lines not on this page · raw message
net/smc/smc_ops.h new file +25 -0
@@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Shared Memory Communications over RDMA (SMC-R) and RoCE +* +* Generic hook for SMC subsystem. +* +* Copyright IBM Corp. 2016 +* Copyright (c) 2024, Alibaba Inc. +* +* Author: D. Wythe <[email protected]> +*/ + +#ifndef __SMC_OPS +#define __SMC_OPS + +#include <net/smc.h> + +/* Find ops by the target name, which required to be a c-string. +* Return NULL if no such ops was found,otherwise, return a valid ops. +* +* Note: Caller MUST ensure it's was invoked under rcu_read_lock. +*/ +struct smc_ops *smc_ops_find_by_name (const char *name); + +#endif /* __SMC_OPS */
net/smc/smc_sysctl.c +95 -0
131 lines not on this page · raw message
Signature · 1 line
2.45.0
Dust Li
maintainerRe: [PATCH bpf-next v5 2/5] net/smc: Introduce generic hook smc_ops
Quoted 30 lines · D. Wythe
The introduction of
IPPROTO_SMCenables eBPF programs to determine whether to use SMC based on the context of socket creation, such as network namespaces, PID and comm name, etc.As a subsequent enhancement, to introduce a new generic hook that allows decisions on whether to use SMC or not at runtime, including but not limited to local/remote IP address or ports.
Moreover, in the future, we can achieve more complex extensions to the protocol stack by extending this ops.
Signed-off-by D. Wythe <[email protected]>---
include/net/netns/smc.h | 3 ++ include/net/smc.h | 51 ++++++++++++++++++++++ net/ipv4/tcp_output.c | 15 +++++-- net/smc/Kconfig | 12 ++++++ net/smc/Makefile | 1 + net/smc/smc_ops.c | 51 ++++++++++++++++++++++ net/smc/smc_ops.h | 25 +++++++++++ net/smc/smc_sysctl.c | 95 +++++++++++++++++++++++++++++++++++++++++ 8 files changed, 249 insertions(+), 4 deletions(-)create mode 100644
net/smc/smc_ops.c
create mode 100644net/smc/smc_ops.hinclude/net/netns/smc.h +0
-0
include/net/netns/smc.h struct netns_smc { · line 21
Quoted 5 lines
include/net/netns/smc.h +2
-0@@ -17,6 +17,9 @@ struct netns_smc {#ifdef CONFIG_SYSCTL struct ctl_table_header *smc_hdr;#endif +#if IS_ENABLED (CONFIG_SMC_OPS ) +struct smc_ops __rcu *ops;
^^^
Align with other fields
Quoted 76 lines
include/net/netns/smc.h +1
-0+#endif /* CONFIG_SMC_OPS */ unsigned int sysctl_autocorking_size;unsigned int sysctl_smcr_buf_type;int sysctl_smcr_testlink_time;include/net/smc.h +51
-0@@ -18,6 +18,8 @@#include "linux/ism.h" struct sock ; +struct tcp_sock ; +struct inet_request_sock ;#define SMC_MAX_PNETID_LEN 16/* Max. length of PNET id */ @@ -97,4 +99,53 @@ struct smcd_dev { u8 going_away : 1; }; +#define SMC_OPS_NAME_MAX 16 + +enum { +/* ops can be inherit from init_net */ +SMC_OPS_FLAG_INHERITABLE =0x1 , + +SMC_OPS_ALL_FLAGS =SMC_OPS_FLAG_INHERITABLE , +}; + +struct smc_ops { +/* priavte */ + +struct list_head list; +struct module *owner; + +/* public */ + +/* unique name */ +char name[SMC_OPS_NAME_MAX ]; +int flags; + +/* Invoked before computing SMC option for SYN packets. +* We can control whether to set SMC options by returning varios value. +* Return 0 to disable SMC, or return any other value to enable it. +*/ +int (*set_option)(struct tcp_sock *tp); + +/* Invoked before Set up SMC options for SYN-ACK packets +* We can control whether to respond SMC options by returning varios value. +* Return 0 to disable SMC, or return any other value to enable it. +*/ +int (*set_option_cond)(const struct tcp_sock *tp,struct inet_request_sock *ireq); +}; + +#if IS_ENABLED (CONFIG_SMC_OPS ) +#define smc_call_retops (init_val, sk, func, ...) ({ \ + typeof(init_val) __ret = (init_val); \ + struct smc_ops *ops; \ + rcu_read_lock(); \ + ops = READ_ONCE(sock_net(sk)->smc.ops); \ + if (ops && ops->func) \ + __ret = ops->func(__VA_ARGS__); \ + rcu_read_unlock(); \ + !!__ret; \ +}) +#else +#define smc_call_retops (init_val, ...) (init_val) +#endif /* CONFIG_SMC_OPS */ +#endif /* _SMC_H */ net/ipv4/tcp_output.c +0
-0
net/ipv4/tcp_output.c line 51
Quoted 4 lines
net/ipv4/tcp_output.c +1
-0@@ -48,6 +48,7 @@#include <linux/skbuff_ref.h> #include <trace/events/tcp.h> +#include <net/smc.h>
Put this include right after #include <net/proto_memory.h> ? I notice all the includes here are well classified.
Quoted 48 lines
net/ipv4/tcp_output.c +10
-4/* Refresh clocks of a TCP socket, * ensuring monotically increasing values. @@ -759,14 +760,17 @@ static void tcp_options_write(struct tcphdr *th, struct tcp_sock *tp,mptcp_options_write (th, ptr, tp, opts); } -static void smc_set_option (const struct tcp_sock *tp, +static void smc_set_option (struct tcp_sock *tp,struct tcp_out_options *opts,unsigned int *remaining) {#if IS_ENABLED (CONFIG_SMC )if (static_branch_unlikely (&tcp_have_smc)) {if (tp->syn_smc) { -if (*remaining >=TCPOLEN_EXP_SMC_BASE_ALIGNED ) { + tp->syn_smc =smc_call_retops (1 , &tp->inet_conn.icsk_inet.sk, + set_option, tp); +/* re-check syn_smc */ +if (tp->syn_smc && *remaining >=TCPOLEN_EXP_SMC_BASE_ALIGNED ) { opts->options |=OPTION_SMC ; *remaining -=TCPOLEN_EXP_SMC_BASE_ALIGNED ; }@@ -776,14 +780,17 @@ static void smc_set_option(const struct tcp_sock *tp, }static void smc_set_option_cond (const struct tcp_sock *tp, -const struct inet_request_sock *ireq, +struct inet_request_sock *ireq,struct tcp_out_options *opts,unsigned int *remaining) {#if IS_ENABLED (CONFIG_SMC )if (static_branch_unlikely (&tcp_have_smc)) {if (tp->syn_smc && ireq->smc_ok) { -if (*remaining >=TCPOLEN_EXP_SMC_BASE_ALIGNED ) { + ireq->smc_ok =smc_call_retops (1 , &tp->inet_conn.icsk_inet.sk, + set_option_cond, tp, ireq); +/* re-check smc_ok */ +if (ireq->smc_ok && *remaining >=TCPOLEN_EXP_SMC_BASE_ALIGNED ) { opts->options |=OPTION_SMC ; *remaining -=TCPOLEN_EXP_SMC_BASE_ALIGNED ; }net/smc/Kconfig +0
-0
net/smc/Kconfig config SMC_LO · line 42
Quoted 10 lines
net/smc/Kconfig +7
-0@@ -33,3 +33,15 @@ config SMC_LO of architecture or hardware. if unsure, say N. + +config SMC_OPS + bool "Generic hook for SMC subsystem" + depends on SMC && BPF_SYSCALL + default n + help + SMC_OPS enables support to register genericfor hook via eBPF programs
genericfor ?
Quoted 147 lines
net/smc/Kconfig +5
-0+ for SMC subsystem. eBPF programs offer much greater flexibility + in modifying the behavior of the SMC protocol stack compared + to a complete kernel-based approach. + + if unsure, say N.net/smc/Makefile +0
-0@@ -7,3 +7,4 @@ smc-y += smc_cdc.o smc_tx.o smc_rx.o smc_close.o smc_ism.o smc_netlink.o smc_stasmc-y += smc_tracepoint.o smc_inet.o
smc-$(CONFIG_SYSCTL) += smc_sysctl.o
smc-$(CONFIG_SMC_LO) += smc_loopback.onet/smc/Makefile +1
-0+smc-$(CONFIG_SMC_OPS) += smc_ops.onet/smc/smc_ops.c new file +51
-0@@ -0,0 +1,51 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Shared Memory Communications over RDMA (SMC-R) and RoCE +* +* Generic hook for SMC subsystem. +* +* Copyright IBM Corp. 2016 +* Copyright (c) 2024, Alibaba Inc. +* +* Author: D. Wythe <[email protected]> +*/ + +#include "smc_ops.h" + +static DEFINE_SPINLOCK (smc_ops_list_lock); +static LIST_HEAD (smc_ops_list); + +static int smc_ops_reg (struct smc_ops *ops) +{ +int ret =0 ; + +spin_lock (&smc_ops_list_lock); +/* already exist or duplicate name */ +if (smc_ops_find_by_name (ops->name)) + ret = -EEXIST ; +else +list_add_tail_rcu (&ops->list, &smc_ops_list); +spin_unlock (&smc_ops_list_lock); +return ret; +} + +static void smc_ops_unreg (struct smc_ops *ops) +{ +spin_lock (&smc_ops_list_lock); +list_del_rcu (&ops->list); +spin_unlock (&smc_ops_list_lock); + +/* Ensure that all readers to complete */ +synchronize_rcu (); +} + +struct smc_ops *smc_ops_find_by_name (const char *name) +{ +struct smc_ops *ops; + +list_for_each_entry_rcu (ops, &smc_ops_list, list) { +if (strcmp (ops->name, name) ==0 ) +return ops; + } +return NULL ; +}net/smc/smc_ops.h new file +25
-0@@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Shared Memory Communications over RDMA (SMC-R) and RoCE +* +* Generic hook for SMC subsystem. +* +* Copyright IBM Corp. 2016 +* Copyright (c) 2024, Alibaba Inc. +* +* Author: D. Wythe <[email protected]> +*/ + +#ifndef __SMC_OPS +#define __SMC_OPS + +#include <net/smc.h> + +/* Find ops by the target name, which required to be a c-string. +* Return NULL if no such ops was found,otherwise, return a valid ops. +* +* Note: Caller MUST ensure it's was invoked under rcu_read_lock. +*/ +struct smc_ops *smc_ops_find_by_name (const char *name); + +#endif /* __SMC_OPS */ net/smc/smc_sysctl.c +29
-0@@ -18,6 +18,7 @@#include "smc_core.h" #include "smc_llc.h" #include "smc_sysctl.h" +#include "smc_ops.h" static int min_sndbuf =SMC_BUF_MIN_SIZE ;static int min_rcvbuf =SMC_BUF_MIN_SIZE ;@@ -30,6 +31,70 @@ static int links_per_lgr_max = SMC_LINKS_ADD_LNK_MAX;static int conns_per_lgr_min =SMC_CONN_PER_LGR_MIN ;static int conns_per_lgr_max =SMC_CONN_PER_LGR_MAX ; +#if IS_ENABLED (CONFIG_SMC_OPS ) +static int smc_net_replace_smc_ops (struct net *net,const char *name) +{ +struct smc_ops *ops =NULL ; + +rcu_read_lock (); +/* null or empty name ask to clear current ops */ +if (name && name[0 ]) { + ops =smc_ops_find_by_name (name); +if (!ops) { +rcu_read_unlock (); +return -EINVAL ; + } +/* no change, just return */ +if (ops ==rcu_dereference (net->smc.ops)) { +rcu_read_unlock (); +return 0 ; + } + } +if (!ops ||bpf_try_module_get (ops, ops->owner)) { +/* xhcg */ + ops =rcu_replace_pointer (net->smc.ops, ops, true); +/* release old ops */ +if (ops) +bpf_module_put (ops, ops->owner); + }else if (ops) { +rcu_read_unlock (); +return -EBUSY ;
net/smc/smc_sysctl.c static int links_per_lgr_max = SMC_LINKS_ADD_LNK_MAX; · line 71
Quoted 10 lines
net/smc/smc_sysctl.c +10
-0+ } +rcu_read_unlock (); +return 0 ; +} + +static int proc_smc_ops (const struct ctl_table *ctl,int write, +void *buffer,size_t *lenp,loff_t *ppos) +{ + struct net *net = container_of(ctl->data, struct net, + smc.ops);
This line won't exceed 80 characters, no need to split into 2 lines.
Best regards,
Dust
D. Wythe
maintainerRe: [PATCH bpf-next v5 2/5] net/smc: Introduce generic hook smc_ops
Quoted 37 lines · Dust Li
Quoted 30 lines · D. Wythe
The introduction of
IPPROTO_SMCenables eBPF programs to determine whether to use SMC based on the context of socket creation, such as network namespaces, PID and comm name, etc.As a subsequent enhancement, to introduce a new generic hook that allows decisions on whether to use SMC or not at runtime, including but not limited to local/remote IP address or ports.
Moreover, in the future, we can achieve more complex extensions to the protocol stack by extending this ops.
Signed-off-by D. Wythe <[email protected]>---
include/net/netns/smc.h | 3 ++ include/net/smc.h | 51 ++++++++++++++++++++++ net/ipv4/tcp_output.c | 15 +++++-- net/smc/Kconfig | 12 ++++++ net/smc/Makefile | 1 + net/smc/smc_ops.c | 51 ++++++++++++++++++++++ net/smc/smc_ops.h | 25 +++++++++++ net/smc/smc_sysctl.c | 95 +++++++++++++++++++++++++++++++++++++++++ 8 files changed, 249 insertions(+), 4 deletions(-)create mode 100644
net/smc/smc_ops.c
create mode 100644net/smc/smc_ops.hinclude/net/netns/smc.h +0
-0include/net/netns/smc.h struct netns_smc { · line 21
Quoted 5 lines
include/net/netns/smc.h +2
-0@@ -17,6 +17,9 @@ struct netns_smc {#ifdef CONFIG_SYSCTL struct ctl_table_header *smc_hdr;#endif +#if IS_ENABLED (CONFIG_SMC_OPS ) +struct smc_ops __rcu *ops;^^^
Quoted 2 lines
Align with other fields
Oops, nice catch.
Quoted 4 lines
Quoted 4 lines
include/net/netns/smc.h +1
-0+#endif /* CONFIG_SMC_OPS */ unsigned int sysctl_autocorking_size;unsigned int sysctl_smcr_buf_type;int sysctl_smcr_testlink_time;
Quoted 10 lines
include/net/smc.h
Quoted 5 lines
include/net/smc.h +1
-0+#include <net/smc.h> Put this include right after #include <
net/proto_memory.h> ? I notice all the includes here are well classified.
Done, I didn't notice this detail before.
Quoted 7 lines
Quoted 7 lines
/* Refresh clocks of a TCP socket,
* ensuring monotically increasing values.include/net/smc.h +0
-0@@ -759,14 +760,17 @@ static void tcp_options_write(struct tcphdr *th, struct tcp_sock *tp,mptcp_options_write (th, ptr, tp, opts); }
Quoted 9 lines
include/net/smc.h static void tcp_options_write(struct tcphdr *th, struct tcp_sock *tp, · line 767
Quoted 6 lines
include/net/smc.h +2
-1-static void smc_set_option (const struct tcp_sock *tp, +static void smc_set_option (struct tcp_sock *tp,struct tcp_out_options *opts,unsigned int *remaining) { + SMC_OPS enables support to register genericfor hook via eBPF programsgenericfor ?
generic.. I will fix it in next version.
Quoted 5 lines
Quoted 5 lines
include/net/smc.h +5
-0+ for SMC subsystem. eBPF programs offer much greater flexibility + in modifying the behavior of the SMC protocol stack compared + to a complete kernel-based approach. + + if unsure, say N.
Quoted 9 lines
net/smc/Makefile
Quoted 6 lines
net/smc/Makefile +2
-0+ struct net *net = container_of(ctl->data, struct net, + smc.ops);This line won't exceed 80 characters, no need to split into 2 lines.
Take it. Thanks for all the advices.
Best wishes,
D. Wythe
Quoted 3 lines
Best regards,
Dust
Dust Li
maintainerRe: [PATCH bpf-next v5 2/5] net/smc: Introduce generic hook smc_ops
In reply to D. WytheQuoted 56 lines · D. Wythe
The introduction of
IPPROTO_SMCenables eBPF programs to determine whether to use SMC based on the context of socket creation, such as network namespaces, PID and comm name, etc.As a subsequent enhancement, to introduce a new generic hook that allows decisions on whether to use SMC or not at runtime, including but not limited to local/remote IP address or ports.
Moreover, in the future, we can achieve more complex extensions to the protocol stack by extending this ops.
Signed-off-by D. Wythe <[email protected]>---
include/net/netns/smc.h | 3 ++ include/net/smc.h | 51 ++++++++++++++++++++++ net/ipv4/tcp_output.c | 15 +++++-- net/smc/Kconfig | 12 ++++++ net/smc/Makefile | 1 + net/smc/smc_ops.c | 51 ++++++++++++++++++++++ net/smc/smc_ops.h | 25 +++++++++++ net/smc/smc_sysctl.c | 95 +++++++++++++++++++++++++++++++++++++++++ 8 files changed, 249 insertions(+), 4 deletions(-)create mode 100644
net/smc/smc_ops.c
create mode 100644net/smc/smc_ops.hinclude/net/netns/smc.h +3
-0@@ -17,6 +17,9 @@ struct netns_smc {#ifdef CONFIG_SYSCTL struct ctl_table_header *smc_hdr;#endif +#if IS_ENABLED (CONFIG_SMC_OPS ) +struct smc_ops __rcu *ops; +#endif /* CONFIG_SMC_OPS */ unsigned int sysctl_autocorking_size;unsigned int sysctl_smcr_buf_type;int sysctl_smcr_testlink_time;include/net/smc.h +2
-0@@ -18,6 +18,8 @@#include "linux/ism.h" struct sock ; +struct tcp_sock ; +struct inet_request_sock ;#define SMC_MAX_PNETID_LEN 16/* Max. length of PNET id */ @@ -97,4 +99,53 @@ struct smcd_dev { u8 going_away : 1; };
include/net/smc.h struct smcd_dev { · line 111
Quoted 10 lines
include/net/smc.h +10
-0+#define SMC_OPS_NAME_MAX 16 + +enum { +/* ops can be inherit from init_net */ +SMC_OPS_FLAG_INHERITABLE =0x1 , + +SMC_OPS_ALL_FLAGS =SMC_OPS_FLAG_INHERITABLE , +}; + +struct smc_ops {
One more thing.
Can we call it smc_bpf_ops ? I think smc_ops is a bit ambiguous. Same for smc_ops.h/c source file.
Best regards,
Dust
D. Wythe
maintainerRe: [PATCH bpf-next v5 2/5] net/smc: Introduce generic hook smc_ops
Quoted 26 lines · Dust Li
Quoted 25 lines · D. Wythe
The introduction of
IPPROTO_SMCenables eBPF programs to determine whether to use SMC based on the context of socket creation, such as network namespaces, PID and comm name, etc.As a subsequent enhancement, to introduce a new generic hook that allows decisions on whether to use SMC or not at runtime, including but not limited to local/remote IP address or ports.
Moreover, in the future, we can achieve more complex extensions to the protocol stack by extending this ops.
Signed-off-by D. Wythe <[email protected]>---
include/net/netns/smc.h | 3 ++ include/net/smc.h | 51 ++++++++++++++++++++++ net/ipv4/tcp_output.c | 15 +++++-- net/smc/Kconfig | 12 ++++++ net/smc/Makefile | 1 + net/smc/smc_ops.c | 51 ++++++++++++++++++++++ net/smc/smc_ops.h | 25 +++++++++++ net/smc/smc_sysctl.c | 95 +++++++++++++++++++++++++++++++++++++++++ 8 files changed, 249 insertions(+), 4 deletions(-)create mode 100644
net/smc/smc_ops.c
create mode 100644net/smc/smc_ops.h
Quoted 6 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ +struct smc_ops {One more thing.
Can we call itsmc_bpf_ops? I thinksmc_opsis a bit ambiguous. Same for smc_ops.h/c source file.
I don't think smc_bpf_ops is a good idea. BPF is just a way to implement smc_ops. Similarly, we can also implement this ops within the kernel module, just like tcp_congestion_ops dose. If you think this is ambiguous, perhaps we can call it as smc_handshake_ops ? This should eliminate the ambiguity.
Best wishes,
D. Wythe
Quoted 3 lines
Best regards,
Dust
D. Wythe
maintainerPATCH v5 3/5 net/smc: bpf: register smc_ops info struct_ops
In reply to D. WytheTo implement injection capability for smc via struct_ops, so that user can make their own smc_ops to modify the behavior of smc stack.
Currently, user can write their own implememtion to choose whether to use SMC or not before TCP 3rd handshake to be comleted. In the future, users can implement more complex functions on smc by expanding it.
--- net/smc/af_smc.c | 10 ++++++ net/smc/smc_ops.c | 79 +++++++++++++++++++++++++++++++++++++++++++++++ net/smc/smc_ops.h | 5 +++ 3 files changed, 94 insertions(+)
net/smc/af_smc.c +10 -0
@@ -55,6 +55,7 @@#include "smc_sysctl.h" #include "smc_loopback.h" #include "smc_inet.h" +#include "smc_ops.h" static DEFINE_MUTEX (smc_server_lgr_pending);/* serialize link group * creation on server
@@ -3576,8 +3577,17 @@ static int __init smc_init(void)pr_err ("%s: smc_inet_init fails with %d\n" , __func__, rc); goto out_ulp; } + + rc =smc_bpf_struct_ops_init (); +if (rc) { +pr_err ("%s: smc_bpf_struct_ops_init fails with %d\n" , __func__, rc); + goto out_inet; + } +static_branch_enable (&tcp_have_smc);return 0 ; +out_inet: +smc_inet_exit (); out_ulp:tcp_unregister_ulp (&smc_ulp_ops); out_lo:
net/smc/smc_ops.c +79 -0
88 lines not on this page · raw message
net/smc/smc_ops.h +5 -0
@@ -21,5 +21,10 @@* Note: Caller MUST ensure it's was invoked under rcu_read_lock. */ struct smc_ops *smc_ops_find_by_name (const char *name); +#if IS_ENABLED (CONFIG_SMC_OPS ) +int smc_bpf_struct_ops_init (void ); +#else +static inline int smc_bpf_struct_ops_init (void ) {return 0 ; } +#endif #endif /* __SMC_OPS */
Signature · 1 line
2.45.0
D. Wythe
maintainerPATCH v5 4/5 libbpf: fix error when st-prefix_ops and ops from differ btf
In reply to D. WytheWhen a struct_ops named xxx_ops was registered by a module, and it will be used in both built-in modules and the module itself, so that the btf_type of xxx_ops will be present in btf_vmlinux instead of in btf_mod, which means that the btf_type of bpf_struct_ops_xxx_ops and xxx_ops will not be in the same btf.
Here are four possible case:
+--------+-------------+-------------+---------------------------------+ | | st_opx_xxx | xxx | | +--------+-------------+-------------+---------------------------------+ | case 0 | btf_vmlinux | bft_vmlinux | be used and reg only in vmlinux | +--------+-------------+-------------+---------------------------------+ | case 1 | btf_vmlinux | bpf_mod | INVALID | +--------+-------------+-------------+---------------------------------+ | case 2 | btf_mod | btf_vmlinux | reg in mod but be used both in | | | | | vmlinux and mod. | +--------+-------------+-------------+---------------------------------+ | case 3 | btf_mod | btf_mod | be used and reg only in mod | +--------+-------------+-------------+---------------------------------+
At present, cases 0, 1, and 3 can be correctly identified, because st_ops_xxx is searched from the same btf with xxx. In order to handle case 2 correctly without affecting other cases, we cannot simply change the search method for st_ops_xxx from find_btf_by_prefix_kind() to find_ksym_btf_id(), because in this way, case 1 will not be recognized anymore.
To address this issue, if st_ops_xxx cannot be found in the btf with xxx and mod_btf does not exist, do find_ksym_btf_id() again to avoid such issue.
--- tools/lib/bpf/libbpf.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-)
tools/lib/bpf/libbpf.c +17 -8
@@ -1005,7 +1005,8 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,const struct btf_member *kern_data_member;struct btf *btf =NULL ;__s32 kern_vtype_id, kern_type_id; -char tname[256 ]; +char tname[256 ], stname[256 ]; +int ret;__u32 i; snprintf(tname, sizeof(tname), "%.*s",
@@ -1020,17 +1021,25 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw, } kern_type =btf__type_by_id (btf, kern_type_id); + ret =snprintf (stname,sizeof (stname),"%s%s" ,STRUCT_OPS_VALUE_PREFIX , tname); +if (ret <0 || ret >=sizeof (stname)) +return -ENAMETOOLONG ; +/* Find the corresponding "map_value" type that will be used * in map_update(BPF_MAP_TYPE_STRUCT_OPS). For example, * find "struct bpf_struct_ops_tcp_congestion_ops" from the * btf_vmlinux. */ - kern_vtype_id =find_btf_by_prefix_kind (btf,STRUCT_OPS_VALUE_PREFIX , - tname,BTF_KIND_STRUCT ); + kern_vtype_id =btf__find_by_name_kind (btf, stname,BTF_KIND_STRUCT );if (kern_vtype_id <0 ) { -pr_warn ("struct_ops init_kern: struct %s%s is not found in kernel BTF\n" , -STRUCT_OPS_VALUE_PREFIX , tname); -return kern_vtype_id; +if (kern_vtype_id == -ENOENT && !*mod_btf) + kern_vtype_id =find_ksym_btf_id (obj, stname,BTF_KIND_STRUCT , + &btf, mod_btf); +if (kern_vtype_id <0 ) { +pr_warn ("struct_ops init_kern: struct %s is not found in kernel BTF\n" , + stname); +return kern_vtype_id; + } } kern_vtype =btf__type_by_id (btf, kern_vtype_id);
@@ -1046,8 +1055,8 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,break ; }if (i ==btf_vlen (kern_vtype)) { -pr_warn ("struct_ops init_kern: struct %s data is not found in struct %s%s\n" , - tname,STRUCT_OPS_VALUE_PREFIX , tname); +pr_warn ("struct_ops init_kern: struct %s data is not found in struct %s\n" , + tname, stname);return -EINVAL ; }
Signature · 1 line
2.45.0
D. Wythe
maintainerPATCH v5 5/5 bpf/selftests: add selftest for bpf_smc_ops
In reply to D. WytheThis tests introduces a tiny smc_ops for filtering SMC connections based on IP pairs, and also adds a realistic topology model to verify this ops.
Also, we can only use SMC loopback under CI test, so an additional configuration needs to be enabled.
Follow the steps below to run this test.
make -C tools/testing/selftests/bpf
cd tools/testing/selftests/bpf
sudo ./test_progs -t smc
Results shows:
Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
--- tools/testing/selftests/bpf/config | 4 + .../selftests/bpf/prog_tests/test_bpf_smc.c | 390 ++++++++++++++++++ tools/testing/selftests/bpf/progs/bpf_smc.c | 116 ++++++ 3 files changed, 510 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_smc.c
tools/testing/selftests/bpf/config +4 -0
@@ -113,3 +113,7 @@ CONFIG_XDP_SOCKETS=y CONFIG_XFRM_INTERFACE=y CONFIG_TCP_CONG_DCTCP=y CONFIG_TCP_CONG_BBR=y +CONFIG_INFINIBAND=y +CONFIG_SMC=y +CONFIG_SMC_OPS=y +CONFIG_SMC_LO=y
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +390 -0
390 lines not on this page · raw message
tools/testing/selftests/bpf/progs/bpf_smc.c new file +116 -0
116 lines not on this page · raw message
Signature · 1 line
2.45.0
Martin KaFai Lau
reviewerRe: [PATCH bpf-next v5 5/5] bpf/selftests: add selftest for bpf_smc_ops
Quoted 27 lines · D. Wythe
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +27
-0+static int send_cmd (int fd,__u16 nlmsg_type,__u32 nlmsg_pid,__u16 nlmsg_flags, +__u8 genl_cmd,__u16 nla_type, +void *nla_data,int nla_len) +{ +struct nlattr *na; +struct sockaddr_nl nladdr; +int r, buflen; +char *buf; + +struct msgtemplate msg = {0 }; + + msg.n.nlmsg_len =NLMSG_LENGTH (GENL_HDRLEN ); + msg.n.nlmsg_type = nlmsg_type; + msg.n.nlmsg_flags = nlmsg_flags; + msg.n.nlmsg_seq =0 ; + msg.n.nlmsg_pid = nlmsg_pid; + msg.g.cmd = genl_cmd; + msg.g.version =1 ; + na = (struct nlattr *)GENLMSG_DATA (&msg); + na->nla_type = nla_type; + na->nla_len = nla_len +1 +NLA_HDRLEN ; +memcpy (NLA_DATA (na), nla_data, nla_len); + msg.n.nlmsg_len +=NLMSG_ALIGN (na->nla_len); + + buf = (char *) &msg; + buflen = msg.n.nlmsg_len; +memset (&nladdr,0 ,sizeof (nladdr));
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c line 95
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +10
-0+ nladdr.nl_family =AF_NETLINK ; + +while ((r =sendto (fd, buf, buflen,0 , (struct sockaddr *) &nladdr, +sizeof (nladdr))) < buflen) { +if (r >0 ) { + buf += r; + buflen -= r; + }else if (errno !=EAGAIN ) +return -1 ; + }
The "}" indentation is off.
I was wondering if it missed a "}" for the while loop. Turns out the "else if" does not have braces while the "if" has. I would add braces to the "else if" also to avoid confusion like this.
Quoted 68 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +68
-0+return 0 ; +} + +static bool get_smc_nl_family_id (void ) +{ +struct sockaddr_nl nl_src; +struct msgtemplate msg; +struct nlattr *nl; +int fd, ret; +pid_t pid; + + fd =socket (AF_NETLINK ,SOCK_RAW ,NETLINK_GENERIC ); +if (!ASSERT_GT (fd,0 ,"nl_family socket" )) +return false; + + pid =getpid (); + +memset (&nl_src,0 ,sizeof (nl_src)); + nl_src.nl_family =AF_NETLINK ; + nl_src.nl_pid = pid; + + ret =bind (fd, (struct sockaddr *) &nl_src,sizeof (nl_src)); +if (!ASSERT_GE (ret,0 ,"nl_family bind" )) + goto fail; + + ret =send_cmd (fd,GENL_ID_CTRL , pid, +NLM_F_REQUEST ,CTRL_CMD_GETFAMILY , +CTRL_ATTR_FAMILY_NAME , (void *)SMC_GENL_FAMILY_NAME , +strlen (SMC_GENL_FAMILY_NAME )); +if (!ASSERT_EQ (ret,0 ,"nl_family query" )) + goto fail; + + ret =recv (fd, &msg,sizeof (msg),0 ); +if (!ASSERT_FALSE (msg.n.nlmsg_type ==NLMSG_ERROR || (ret <0 ) || + !NLMSG_OK ((&msg.n), ret),"nl_family response" )) + goto fail; + + nl = (struct nlattr *)GENLMSG_DATA (&msg); + nl = (struct nlattr *) ((char *) nl +NLA_ALIGN (nl->nla_len)); +if (!ASSERT_EQ (nl->nla_type,CTRL_ATTR_FAMILY_ID ,"nl_family nla type" )) + goto fail; + + smc_nl_family_id = *(uint16_t *)NLA_DATA (nl); +close (fd); +return true; +fail: +close (fd); +return false; +} + +static bool smc_ueid (int op) +{ +struct sockaddr_nl nl_src; +struct msgtemplate msg; +struct nlmsgerr *err; +char test_ueid[32 ]; +int fd, ret; +pid_t pid; + +/* UEID required */ +memset (test_ueid,'\x20' ,sizeof (test_ueid)); +memcpy (test_ueid,SMC_BPFTEST_UEID ,strlen (SMC_BPFTEST_UEID )); + fd =socket (AF_NETLINK ,SOCK_RAW ,NETLINK_GENERIC ); +if (!ASSERT_GT (fd,0 ,"ueid socket" )) +return false; + + pid =getpid (); +memset (&nl_src,0 ,sizeof (nl_src));
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c line 173
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +10
-0+ nl_src.nl_family =AF_NETLINK ; + nl_src.nl_pid = pid; + + ret =bind (fd, (struct sockaddr *) &nl_src,sizeof (nl_src)); +if (!ASSERT_GE (ret,0 ,"ueid bind" )) + goto fail; + + ret =send_cmd (fd, smc_nl_family_id, pid, +NLM_F_REQUEST |NLM_F_ACK , op,SMC_NLA_EID_TABLE_ENTRY , + (void *)test_ueid,sizeof (test_ueid));
Same. Indentation is off.
Quoted 25 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +25
-0+if (!ASSERT_EQ (ret,0 ,"ueid cmd" )) + goto fail; + + ret =recv (fd, &msg,sizeof (msg),0 ); +if (!ASSERT_FALSE ((ret <0 ) || !NLMSG_OK ((&msg.n), ret),"ueid response" )) + goto fail; + +if (msg.n.nlmsg_type ==NLMSG_ERROR ) { + err =NLMSG_DATA (&msg); +switch (op) { +case SMC_NETLINK_REMOVE_UEID : +if (!ASSERT_FALSE ((err->error && err->error != -ENOENT ),"ueid remove" )) + goto fail; +break ; +case SMC_NETLINK_ADD_UEID : +if (!ASSERT_EQ (err->error,0 ,"ueid add" )) + goto fail; +break ; +default : +break ; + } + } +close (fd); +return true; +fail:
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c line 208
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +10
-0+close (fd); +return false; +} + +static bool setup_netns (void ) +{ +if (!ASSERT_OK (make_netns (TEST_NS ),"create net namespace" )) +return false; + + nstoken =open_netns (TEST_NS );
Instead of make_netns and then immediately open_netns, try netns_new(TEST_NS, true) from the test_progs.c.
Quoted 133 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +133
-0+if (!ASSERT_OK_PTR (nstoken,"open net namespace" )) + goto fail_open_netns; + +if (!ASSERT_OK (system ("ip addr add 127.0.1.0/8 dev lo" ),"add server node" )) + goto fail_ip; + +if (!ASSERT_OK (system ("ip addr add 127.0.2.0/8 dev lo" ),"server via risk path" )) + goto fail_ip; + +return true; +fail_open_netns: +remove_netns (TEST_NS ); +fail_ip: +close_netns (nstoken); +return false; +} + +static void cleanup_netns (void ) +{ +close_netns (nstoken); +remove_netns (TEST_NS ); +} + +static bool setup_ueid (void ) +{ +/* get smc nl id */ +if (!get_smc_nl_family_id ()) +return false; +/* clear old ueid for bpftest */ +smc_ueid (SMC_NETLINK_REMOVE_UEID ); +/* smc-loopback required ueid */ +return smc_ueid (SMC_NETLINK_ADD_UEID ); +} + +static void cleanup_ueid (void ) +{ +smc_ueid (SMC_NETLINK_REMOVE_UEID ); +} + +static bool setup_smc (void ) +{ +if (!setup_ueid ()) +return false; + +if (!setup_netns ()) + goto fail_netns; + +return true; +fail_netns: +cleanup_ueid (); +return false; +} + +static int set_client_addr_cb (int fd,void *opts) +{ +const char *src = (const char *)opts; +struct sockaddr_in localaddr; + + localaddr.sin_family =AF_INET ; + localaddr.sin_port =htons (0 ); + localaddr.sin_addr.s_addr =inet_addr (src); +return !ASSERT_EQ (bind (fd, &localaddr,sizeof (localaddr)),0 ,"client bind" ); +} + +static void run_link (const char *src,const char *dst,int port) +{ +struct network_helper_opts opts = {0 }; +int server, client; + + server =start_server_str (AF_INET ,SOCK_STREAM , dst, port,NULL ); +if (!ASSERT_OK_FD (server,"start service_1" )) +return ; + + opts.proto =IPPROTO_TCP ; + opts.post_socket_cb = set_client_addr_cb; + opts.cb_opts = (void *)src; + + client =connect_to_fd_opts (server, &opts); +if (!ASSERT_OK_FD (client,"start connect" )) + goto fail_client; + +close (client); +fail_client: +close (server); +} + +static void block_link (int map_fd,const char *src,const char *dst) +{ +struct smc_strat_ip_value val = { .mode =/* block */ 0 }; +struct smc_strat_ip_key key = { + .sip =inet_addr (src), + .dip =inet_addr (dst), + }; + +bpf_map_update_elem (map_fd, &key, &val,BPF_ANY ); +} + +/* +* This test describes a real-life service topology as follows: +* +* +-------------> service_1 +* link1 | | +* +--------------------> server | link 2 +* | | V +* | +-------------> service_2 +* | link 3 +* client -------------------> server_via_unsafe_path -> service_3 +* +* Among them, +* 1. link-1 is very suitable for using SMC. +* 2. link-2 is not suitable for using SMC, because the mode of this link is kind of +* short-link services. +* 3. link-3 is also not suitable for using SMC, because the RDMA link is unavailable and +* needs to go through a long timeout before it can fallback to TCP. +* +* To achieve this goal, we use a customized SMC ip strategy via smc_ops. +*/ +static void test_topo (void ) +{ +struct bpf_smc *skel; +int rc, map_fd; + + skel =bpf_smc__open_and_load (); +if (!ASSERT_OK_PTR (skel,"bpf_smc__open_and_load" )) +return ; + + rc =bpf_smc__attach (skel); +if (!ASSERT_EQ (rc,0 ,"bpf_smc__attach" )) + goto fail; + + map_fd =bpf_map__fd (skel->maps.smc_strats_ip); +if (!ASSERT_GT (map_fd,0 ,"bpf_map__fd" )) + goto fail;
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c line 351
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +10
-0+ +/* Mock the process of transparent replacement, since we will modify protocol +* to ipproto_smc accropding to it via fmod_ret/update_socket_protocol. +*/ +system ("sysctl -w net.smc.ops=linkcheck" ); + +/* Configure ip strat */ +block_link (map_fd,CLIENT_IP ,SERVER_IP_VIA_RISK_PATH ); +block_link (map_fd,SERVER_IP ,SERVER_IP ); +close (map_fd);
No need to close(map-fd) here. bpf_smc__destroy(skel) will do it.
It seems the new selftest fails also. not always though which is concerning.
pw-bot: cr
Quoted 24 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +24
-0+ +/* should go with smc */ +run_link (CLIENT_IP ,SERVER_IP ,SERVICE_1 ); +/* should go with smc fallback */ +run_link (SERVER_IP ,SERVER_IP ,SERVICE_2 ); + +ASSERT_EQ (skel->bss->smc_cnt,2 ,"smc count" ); +ASSERT_EQ (skel->bss->fallback_cnt,1 ,"fallback count" ); + +/* should go with smc */ +run_link (CLIENT_IP ,SERVER_IP ,SERVICE_2 ); + +ASSERT_EQ (skel->bss->smc_cnt,3 ,"smc count" ); +ASSERT_EQ (skel->bss->fallback_cnt,1 ,"fallback count" ); + +/* should go with smc fallback */ +run_link (CLIENT_IP ,SERVER_IP_VIA_RISK_PATH ,SERVICE_3 ); + +ASSERT_EQ (skel->bss->smc_cnt,4 ,"smc count" ); +ASSERT_EQ (skel->bss->fallback_cnt,2 ,"fallback count" ); + +fail: +bpf_smc__destroy (skel); +}
D. Wythe
maintainerRe: [PATCH bpf-next v5 5/5] bpf/selftests: add selftest for bpf_smc_ops
Quoted 14 lines · Martin KaFai Lau
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c line 95
Quoted 10 lines · D. Wythe
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +10
-0+static int send_cmd (int fd,__u16 nlmsg_type,__u32 nlmsg_pid,__u16 nlmsg_flags, +__u8 genl_cmd,__u16 nla_type, + while ((r = sendto(fd, buf, buflen, 0, (struct sockaddr *) &nladdr, + sizeof(nladdr))) < buflen) { +if (r >0 ) { + buf += r; + buflen -= r; + }else if (errno !=EAGAIN ) +return -1 ; + }The "}" indentation is off.
Quoted 4 lines
I was wondering if it missed a "}" for the while loop. Turns out the "else if" does not have braces while the "if" has. I would add braces to the "else if" also to avoid confusion like this.
Take it. I fix change it in next version.
Quoted 4 lines
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +4
-0+return 0 ; +} + +static bool get_smc_nl_family_id (void )
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c line 173
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +6
-0+{ +struct sockaddr_nl nl_src; +struct msgtemplate msg; + ret =send_cmd (fd, smc_nl_family_id, pid, +NLM_F_REQUEST |NLM_F_ACK , op,SMC_NLA_EID_TABLE_ENTRY , + (void *)test_ueid,sizeof (test_ueid));Same. Indentation is off.
Take it. Thanks for pointing it out.
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c line 208
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +4
-0+if (!ASSERT_EQ (ret,0 ,"ueid cmd" )) + goto fail; + + nstoken =open_netns (TEST_NS );Instead of
make_netnsand then immediatelyopen_netns, trynetns_new(TEST_NS, true)from thetest_progs.c.
Got it, I'll try it in next version.
Quoted 10 lines
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +9
-0+if (!ASSERT_OK_PTR (nstoken,"open net namespace" )) + goto fail_open_netns; + +if (!ASSERT_OK (system ("ip addr add 127.0.1.0/8 dev lo" ),"add server node" )) + goto fail_ip; + +if (!ASSERT_OK (system ("ip addr add 127.0.2.0/8 dev lo" ),"server via risk path" )) +close_netns (nstoken); +return false;
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c line 351
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +6
-0+} + +/* Configure ip strat */ +block_link (map_fd,CLIENT_IP ,SERVER_IP_VIA_RISK_PATH ); +block_link (map_fd,SERVER_IP ,SERVER_IP ); +close (map_fd);No need to close(map-fd) here.
bpf_smc__destroy(skel)will do it.
Got it. Many thanks.
Quoted 3 lines
It seems the new selftest fails also. not always though which is concerning.
This might not be a random failure, but rather related to s390x, which carries a seid by default, which may affect my action of deleting ueid. I am requesting IBM folks to help me analyze this issue since i have no s390x machine.
Anyway, I will solve it in the next version.
Best wishes,
D. Wythe
Quoted 27 lines
pw-bot: cr
Quoted 24 lines
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +24
-0+ +/* should go with smc */ +run_link (CLIENT_IP ,SERVER_IP ,SERVICE_1 ); +/* should go with smc fallback */ +run_link (SERVER_IP ,SERVER_IP ,SERVICE_2 ); + +ASSERT_EQ (skel->bss->smc_cnt,2 ,"smc count" ); +ASSERT_EQ (skel->bss->fallback_cnt,1 ,"fallback count" ); + +/* should go with smc */ +run_link (CLIENT_IP ,SERVER_IP ,SERVICE_2 ); + +ASSERT_EQ (skel->bss->smc_cnt,3 ,"smc count" ); +ASSERT_EQ (skel->bss->fallback_cnt,1 ,"fallback count" ); + +/* should go with smc fallback */ +run_link (CLIENT_IP ,SERVER_IP_VIA_RISK_PATH ,SERVICE_3 ); + +ASSERT_EQ (skel->bss->smc_cnt,4 ,"smc count" ); +ASSERT_EQ (skel->bss->fallback_cnt,2 ,"fallback count" ); + +fail: +bpf_smc__destroy (skel); +}
D. Wythe
maintainerOn Tue, Jan 07, 2025 at 12:17:10PM +0800, D. Wythe wrote:
Hi smc folks,
Given that there are almost no more serious issues blocking the progress of this series in the BPF community regarding the BPF part, your input on the SMC part of this series are very valuable.
Additionally, I encountered some errors during CI testing on the S390x architecture. These errors seem to be related to adding ueid via netlink, and I suspect they might be connected to seid. However, since I have no s390x machine, I am unable to further pinpoint the exact cause of the problem. If IBM could help confirm this, it would be greatly appreciated. If there is no time to address this issue, I plan to temporarily skip the BPF CI testing of my case in S390x.
Best wishes,
D. Wythe
Quoted 80 lines
This patch aims to introduce BPF injection capabilities for SMC and includes a self-test to ensure code stability.
Since the SMC protocol isn't ideal for every situation, especially short-lived ones, most applications can't guarantee the absence of such scenarios. Consequently, applications may need specific strategies to decide whether to use SMC. For example, an application might limit SMC usage to certain IP addresses or ports.
To maintain the principle of transparent replacement, we want applications to remain unaffected even if they need specific SMC strategies. In other words, they should not require recompilation of their code.
Additionally, we need to ensure the scalability of strategy implementation. While using socket options or sysctl might be straightforward, it could complicate future expansions.
Fortunately, BPF addresses these concerns effectively. Users can write their own strategies in eBPF to determine whether to use SMC, and they can easily modify those strategies in the future.
v2:
1. Renamesmc_bpf_opstosmc_ops.
2. Change the scope ofsmc_opsfrom global to per netns.
3. Directly pass parameters to ops instead ofsmc_ops_ctx.
4. Remove structsmc_ops_ctx.
5. Remove exports that are no longer needed.v3:
1. Removefind_ksym_btf_id_by_prefix_kind.
2. Enhance selftest, introduce a complete ops for filtering smc connections based on ip pairs and a realistic topology test to verify it.v4:
1. Remove unless func:smc_bpf_ops_check_member()
2. Remove unless inline func:smc_ops_find_by_name()
3. ChangeCONFIG_SMC=yto complete CI testing
4. Changesmc_socktosmc_sock___localin test to avoid compiling failed withCONFIG_SMC=y
5. Improve test cases, remove unnecessary timeouts and multi-thread test, usingnetwork_helpersto start testing between server and client.
6. Fix issues when the return value of the ops function is neither 0nor 1.v5:
1. Fix incorrect CI config fromCONFIG_SMC=YtoCONFIG_SMC=y.D. Wythe (5):
bpf: export necessary sympols for modules withstruct_opsnet/smc: Introduce generic hooksmc_ops
net/smc: bpf: registersmc_opsinfostruct_ops
libbpf: fix error when st-prefix_ops and ops from differ btf bpf/selftests: add selftest forbpf_smc_opsinclude/net/netns/smc.h | 3 + include/net/smc.h | 51 +++ kernel/bpf/bpf_struct_ops.c | 2 + kernel/bpf/syscall.c | 1 + net/ipv4/tcp_output.c | 15 +- net/smc/Kconfig | 12 + net/smc/Makefile | 1 + net/smc/af_smc.c | 10 + net/smc/smc_ops.c | 130 ++++++ net/smc/smc_ops.h | 30 ++ net/smc/smc_sysctl.c | 95 +++++ tools/lib/bpf/libbpf.c | 25 +- tools/testing/selftests/bpf/config | 4 + .../selftests/bpf/prog_tests/test_bpf_smc.c | 390 ++++++++++++++++++ tools/testing/selftests/bpf/progs/bpf_smc.c | 116 ++++++ 15 files changed, 873 insertions(+), 12 deletions(-) create mode 100644 net/smc/smc_ops.c create mode 100644 net/smc/smc_ops.h create mode 100644 tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_smc.cSignature · 1 line
2.45.0