Skip to contentSkip to conversation

net/smc: Introduce smc_ops

PATCH Superseded D. Wythe 5 participants 19 messages Dec 18, 2024 – Jan 14, 2025 4 messages not in the archive

Subsystems (11)

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

D. Wythe

author maintainer
#0 lore raw

v3 cover net/smc: Introduce smc_ops

This patches attempt to introduce BPF injection capability for SMC, and add selftest to ensure code stability.

Since the SMC protocol is not suitable for all scenarios, especially for short-lived. For most applications, they cannot guarantee that there are no such scenarios at all. Therefore, apps may need some specific strategies to decide shall we need to use SMC or not, for example, apps can limit the scope of the SMC to a specific IP address or port.

Based on the consideration of transparent replacement, we hope that apps can remain transparent even if they need to formulate some specific strategies for SMC using. That is, do not need to recompile their code.

On the other hand, we need to ensure the scalability of strategies implementation. Although it is simple to use socket options or sysctl, it will bring more complexity to subsequent expansion.

Fortunately, BPF can solve these concerns very well, users can write thire own strategies in eBPF to choose whether to use SMC or not. And it's quite easy for them to modify their strategies in the future.

v2:
1. Rename smc_bpf_ops to smc_ops.
2. Change the scope of smc_ops from global to per netns.
3. Directly pass parameters to ops instead of smc_ops_ctx.
4. Remove struct smc_ops_ctx.
5. Remove exports that are no longer needed.

v3:
1. Remove find_ksym_btf_id_by_prefix_kind.
2. Enhance selftest, introduce a complete ops for filtering smc connections based on ip pairs and a realistic topology test to verify it.

D. Wythe (5):
bpf: export necessary sympols for modules with struct_ops net/smc: Introduce generic hook smc_ops
net/smc: bpf: register smc_ops info struct_ops
libbpf: fix error when st-prefix_ops and ops from differ btf bpf/selftests: add selftest for bpf_smc_ops

 include/net/netns/smc.h                       |   3 +
 include/net/smc.h                             |  51 ++
 kernel/bpf/bpf_struct_ops.c                   |   2 +
 kernel/bpf/syscall.c                          |   1 +
 net/ipv4/tcp_output.c                         |  15 +-
 net/smc/Kconfig                               |  12 +
 net/smc/Makefile                              |   1 +
 net/smc/af_smc.c                              |  10 +
 net/smc/smc_ops.c                             | 150 +++++
 net/smc/smc_ops.h                             |  31 +
 net/smc/smc_sysctl.c                          |  95 ++++
 tools/lib/bpf/libbpf.c                        |  25 +-
 tools/testing/selftests/bpf/config            |   4 +
 .../selftests/bpf/prog_tests/test_bpf_smc.c   | 535 ++++++++++++++++++
 tools/testing/selftests/bpf/progs/bpf_smc.c   | 109 ++++
 15 files changed, 1032 insertions(+), 12 deletions(-)
 create mode 100644 net/smc/smc_ops.c
 create mode 100644 net/smc/smc_ops.h
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_smc.c
Signature · 1 line
2.45.0
author maintainer
#1 lore raw

PATCH v3 1/5 bpf: export necessary sympols for modules with struct_ops

Exports three necessary symbols for implementing struct_ops with tristate subsystem.

To hold or release refcnt of struct_ops refcnt by inline funcs bpf_try_module_get and bpf_module_put which use bpf_struct_ops_get(put) conditionally.

And to copy obj name from one to the other with effective checks by bpf_obj_name_cpy.

---
 kernel/bpf/bpf_struct_ops.c | 2 ++
 kernel/bpf/syscall.c        | 1 +
 2 files changed, 3 insertions(+)
kernel/bpf/bpf_struct_ops.c +2 -0
index 606efe32485a..00c212e0ad39 100644
@@ -1119,6 +1119,7 @@ bool bpf_struct_ops_get(const void *kdata)
 	map = __bpf_map_inc_not_zero(&st_map->map, false);
 	return !IS_ERR(map);
 }
+EXPORT_SYMBOL_GPL(bpf_struct_ops_get);
 
 void bpf_struct_ops_put(const void *kdata)
 {
@@ -1130,6 +1131,7 @@ void bpf_struct_ops_put(const void *kdata)
 
 	bpf_map_put(&st_map->map);
 }
+EXPORT_SYMBOL_GPL(bpf_struct_ops_put);
 
 int bpf_struct_ops_supported(const struct bpf_struct_ops *st_ops, u32 moff)
 {
kernel/bpf/syscall.c +1 -0
index 5684e8ce132d..62238ec989dc 100644
@@ -1167,6 +1167,7 @@ int bpf_obj_name_cpy(char *dst, const char *src, unsigned int size)
 
 	return src - orig_src;
 }
+EXPORT_SYMBOL_GPL(bpf_obj_name_cpy);
 
 int map_check_no_btf(const struct bpf_map *map,
 		     const struct btf *btf,
Signature · 1 line
2.45.0
author maintainer
#2 lore raw

PATCH v3 2/5 net/smc: Introduce generic hook smc_ops

In reply to D. Wythe

The introduction of IPPROTO_SMC enables eBPF programs to determine whether to use SMC based on the context of socket creation, such as network namespaces, PID and comm name, etc.

As a subsequent enhancement, to introduce a new generic hook that allows decisions on whether to use SMC or not at runtime, including but not limited to local/remote IP address or ports.

Moreover, in the future, we can achieve more complex extensions to the protocol stack by extending this ops.

---
 include/net/netns/smc.h |  3 ++
 include/net/smc.h       | 51 ++++++++++++++++++++++
 net/ipv4/tcp_output.c   | 15 +++++--
 net/smc/Kconfig         | 12 ++++++
 net/smc/Makefile        |  1 +
 net/smc/smc_ops.c       | 51 ++++++++++++++++++++++
 net/smc/smc_ops.h       | 29 +++++++++++++
 net/smc/smc_sysctl.c    | 95 +++++++++++++++++++++++++++++++++++++++++
 8 files changed, 253 insertions(+), 4 deletions(-)
 create mode 100644 net/smc/smc_ops.c
 create mode 100644 net/smc/smc_ops.h
include/net/netns/smc.h +3 -0
index fc752a50f91b..59d069f56b2d 100644
@@ -17,6 +17,9 @@ struct netns_smc {
 #ifdef CONFIG_SYSCTL
 	struct ctl_table_header		*smc_hdr;
 #endif
+#if IS_ENABLED(CONFIG_SMC_OPS)
+	struct smc_ops __rcu *ops;
+#endif /* CONFIG_SMC_OPS */
 	unsigned int			sysctl_autocorking_size;
 	unsigned int			sysctl_smcr_buf_type;
 	int				sysctl_smcr_testlink_time;
include/net/smc.h +51 -0

61 lines not on this page · raw message

net/ipv4/tcp_output.c +11 -4
index 5485a70b5fe5..7b402167fb4d 100644
@@ -48,6 +48,7 @@
 #include <linux/skbuff_ref.h>
 
 #include <trace/events/tcp.h>
+#include <net/smc.h>
 
 /* Refresh clocks of a TCP socket,
  * ensuring monotically increasing values.
@@ -759,14 +760,17 @@ static void tcp_options_write(struct tcphdr *th, struct tcp_sock *tp,
 	mptcp_options_write(th, ptr, tp, opts);
 }
 
-static void smc_set_option(const struct tcp_sock *tp,
+static void smc_set_option(struct tcp_sock *tp,
 			   struct tcp_out_options *opts,
 			   unsigned int *remaining)
 {
 #if IS_ENABLED(CONFIG_SMC)
 	if (static_branch_unlikely(&tcp_have_smc)) {
 		if (tp->syn_smc) {
-			if (*remaining >= TCPOLEN_EXP_SMC_BASE_ALIGNED) {
+			tp->syn_smc = smc_call_retops(1, &tp->inet_conn.icsk_inet.sk,
+						      set_option, tp);
+			/* re-check syn_smc */
+			if (tp->syn_smc && *remaining >= TCPOLEN_EXP_SMC_BASE_ALIGNED) {
 				opts->options |= OPTION_SMC;
 				*remaining -= TCPOLEN_EXP_SMC_BASE_ALIGNED;
 			}
@@ -776,14 +780,17 @@ static void smc_set_option(const struct tcp_sock *tp,
 }
 
 static void smc_set_option_cond(const struct tcp_sock *tp,
-				const struct inet_request_sock *ireq,
+				struct inet_request_sock *ireq,
 				struct tcp_out_options *opts,
 				unsigned int *remaining)
 {
 #if IS_ENABLED(CONFIG_SMC)
 	if (static_branch_unlikely(&tcp_have_smc)) {
 		if (tp->syn_smc && ireq->smc_ok) {
-			if (*remaining >= TCPOLEN_EXP_SMC_BASE_ALIGNED) {
+			ireq->smc_ok = smc_call_retops(1, &tp->inet_conn.icsk_inet.sk,
+						       set_option_cond, tp, ireq);
+			/* re-check smc_ok */
+			if (ireq->smc_ok && *remaining >= TCPOLEN_EXP_SMC_BASE_ALIGNED) {
 				opts->options |= OPTION_SMC;
 				*remaining -= TCPOLEN_EXP_SMC_BASE_ALIGNED;
 			}
net/smc/Kconfig +12 -0
index ba5e6a2dd2fd..0ee16ec8dceb 100644
@@ -33,3 +33,15 @@ config SMC_LO
 	  of architecture or hardware.
 
 	  if unsure, say N.
+
+config SMC_OPS
+	bool "Generic hook for SMC subsystem"
+	depends on SMC && BPF_SYSCALL
+	default n
+	help
+	  SMC_OPS enables support to register genericfor hook via eBPF programs
+	  for SMC subsystem. eBPF programs offer much greater flexibility
+	  in modifying the behavior of the SMC protocol stack compared
+	  to a complete kernel-based approach.
+
+	  if unsure, say N.
net/smc/Makefile +1 -0
index 60f1c87d5212..5dd706b2927a 100644
@@ -7,3 +7,4 @@ smc-y += smc_cdc.o smc_tx.o smc_rx.o smc_close.o smc_ism.o smc_netlink.o smc_sta
 smc-y += smc_tracepoint.o smc_inet.o
 smc-$(CONFIG_SYSCTL) += smc_sysctl.o
 smc-$(CONFIG_SMC_LO) += smc_loopback.o
+smc-$(CONFIG_SMC_OPS) += smc_ops.o
\ No newline at end of file
net/smc/smc_ops.c new file +51 -0

51 lines not on this page · raw message

net/smc/smc_ops.h new file +29 -0
new file mode 100644
index 000000000000..214f4c99efd4
@@ -0,0 +1,29 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ *  Shared Memory Communications over RDMA (SMC-R) and RoCE
+ *
+ *  Generic hook for SMC subsystem.
+ *
+ *  Copyright IBM Corp. 2016
+ *  Copyright (c) 2024, Alibaba Inc.
+ *
+ *  Author: D. Wythe <[email protected]>
+ */
+
+#ifndef __SMC_OPS
+#define __SMC_OPS
+
+#include <net/smc.h>
+
+#if IS_ENABLED(CONFIG_SMC_OPS)
+/* Find ops by the target name, which required to be a c-string.
+ * Return NULL if no such ops was found,otherwise, return a valid ops.
+ *
+ * Note: Caller MUST ensure it's was invoked under rcu_read_lock.
+ */
+struct smc_ops *smc_ops_find_by_name(const char *name);
+#else
+static inline struct smc_ops *smc_ops_find_by_name(const char *name) { return NULL; }
+#endif /* CONFIG_SMC_OPS*/
+
+#endif /* __SMC_OPS */
net/smc/smc_sysctl.c +95 -0

131 lines not on this page · raw message

Signature · 1 line
2.45.0
author maintainer
#3 lore raw

PATCH v3 3/5 net/smc: bpf: register smc_ops info struct_ops

In reply to D. Wythe

To implement injection capability for smc via struct_ops, so that user can make their own smc_ops to modify the behavior of smc stack.

Currently, user can write their own implememtion to choose whether to use SMC or not before TCP 3rd handshake to be comleted. In the future, users can implement more complex functions on smc by expanding it.

---
 net/smc/af_smc.c  | 10 +++++
 net/smc/smc_ops.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++
 net/smc/smc_ops.h |  2 +
 3 files changed, 111 insertions(+)
net/smc/af_smc.c +10 -0
index 9d76e902fd77..6adedae2986d 100644
@@ -55,6 +55,7 @@
 #include "smc_sysctl.h"
 #include "smc_loopback.h"
 #include "smc_inet.h"
+#include "smc_ops.h"
 
 static DEFINE_MUTEX(smc_server_lgr_pending);	/* serialize link group
 						 * creation on server
@@ -3576,8 +3577,17 @@ static int __init smc_init(void)
 		pr_err("%s: smc_inet_init fails with %d\n", __func__, rc);
 		goto out_ulp;
 	}
+
+	rc = smc_bpf_struct_ops_init();
+	if (rc) {
+		pr_err("%s: smc_bpf_struct_ops_init fails with %d\n", __func__, rc);
+		goto out_inet;
+	}
+
 	static_branch_enable(&tcp_have_smc);
 	return 0;
+out_inet:
+	smc_inet_exit();
 out_ulp:
 	tcp_unregister_ulp(&smc_ulp_ops);
 out_lo:
net/smc/smc_ops.c +99 -0

108 lines not on this page · raw message

net/smc/smc_ops.h +2 -0
index 214f4c99efd4..f4e50eae13f6 100644
@@ -22,8 +22,10 @@
  * Note: Caller MUST ensure it's was invoked under rcu_read_lock.
  */
 struct smc_ops *smc_ops_find_by_name(const char *name);
+int smc_bpf_struct_ops_init(void);
 #else
 static inline struct smc_ops *smc_ops_find_by_name(const char *name) { return NULL; }
+static inline int smc_bpf_struct_ops_init(void) { return 0; }
 #endif /* CONFIG_SMC_OPS*/
 
 #endif /* __SMC_OPS */
Signature · 1 line
2.45.0
#4 lore raw

Re: [PATCH bpf-next v3 3/5] net/smc: bpf: register smc_ops info struct_ops

Quoted 8 lines · D. Wythe
net/smc/smc_ops.c +8 -0
+static int smc_bpf_ops_check_member(const struct btf_type *t,
+				    const struct btf_member *member,
+				    const struct bpf_prog *prog)
+{
+	u32 moff = __btf_member_bit_offset(t, member) / 8;
+
+	switch (moff) {
+	case offsetof(struct smc_ops, name):

net/smc/smc_ops.c struct smc_ops *smc_ops_find_by_name(const char *name) · line 128

Quoted 10 lines
net/smc/smc_ops.c +10 -0
+	case offsetof(struct smc_ops, flags):
+	case offsetof(struct smc_ops, set_option):
+	case offsetof(struct smc_ops, set_option_cond):
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}

The whole smc_bpf_ops_check_member() should not be needed.

author maintainer
#5 lore raw

Re: [PATCH bpf-next v3 3/5] net/smc: bpf: register smc_ops info struct_ops

Quoted 13 lines · Martin KaFai Lau
Quoted 12 lines · D. Wythe
net/smc/smc_ops.c +12 -0
+static int smc_bpf_ops_check_member(const struct btf_type *t,
+				    const struct btf_member *member,
+				    const struct bpf_prog *prog)
+{
+	u32 moff = __btf_member_bit_offset(t, member) / 8;
+
+	switch (moff) {
+	case offsetof(struct smc_ops, name):
+	case offsetof(struct smc_ops, flags):
+	case offsetof(struct smc_ops, set_option):
+	case offsetof(struct smc_ops, set_option_cond):
+		break;
Quoted 8 lines

net/smc/smc_ops.c struct smc_ops *smc_ops_find_by_name(const char *name) · line 128

Quoted 6 lines
net/smc/smc_ops.c +6 -0
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}

The whole smc_bpf_ops_check_member() should not be needed.

Got it, thanks for that.

D. Wythe

#6 lore raw

Re: [PATCH bpf-next v3 3/5] net/smc: bpf: register smc_ops info struct_ops

In reply to D. Wythe
Quoted 163 lines · D. Wythe

To implement injection capability for smc via struct_ops, so that user can make their own smc_ops to modify the behavior of smc stack.

Currently, user can write their own implememtion to choose whether to use SMC or not before TCP 3rd handshake to be comleted. In the future, users can implement more complex functions on smc by expanding it.

---
  net/smc/af_smc.c  | 10 +++++
  net/smc/smc_ops.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++
  net/smc/smc_ops.h |  2 +
  3 files changed, 111 insertions(+)
net/smc/af_smc.c +10 -0
index 9d76e902fd77..6adedae2986d 100644
@@ -55,6 +55,7 @@
  #include "smc_sysctl.h"
  #include "smc_loopback.h"
  #include "smc_inet.h"
+#include "smc_ops.h"
 
  static DEFINE_MUTEX(smc_server_lgr_pending);	/* serialize link group
  						 * creation on server
@@ -3576,8 +3577,17 @@ static int __init smc_init(void)
  		pr_err("%s: smc_inet_init fails with %d\n", __func__, rc);
  		goto out_ulp;
  	}
+
+	rc = smc_bpf_struct_ops_init();
+	if (rc) {
+		pr_err("%s: smc_bpf_struct_ops_init fails with %d\n", __func__, rc);
+		goto out_inet;
+	}
+
  	static_branch_enable(&tcp_have_smc);
  	return 0;
+out_inet:
+	smc_inet_exit();
  out_ulp:
  	tcp_unregister_ulp(&smc_ulp_ops);
  out_lo:
net/smc/smc_ops.c +99 -0
index 0fc19cadd760..0f07652f4837 100644
@@ -10,6 +10,10 @@
   *  Author: D. Wythe <[email protected]>
   */
 
+#include <linux/bpf_verifier.h>
+#include <linux/bpf.h>
+#include <linux/btf.h>
+
  #include "smc_ops.h"
 
  static DEFINE_SPINLOCK(smc_ops_list_lock);
@@ -49,3 +53,98 @@ struct smc_ops *smc_ops_find_by_name(const char *name)
  	}
  	return NULL;
  }
+
+static int __bpf_smc_stub_set_tcp_option(struct tcp_sock *tp) { return 1; }
+static int __bpf_smc_stub_set_tcp_option_cond(const struct tcp_sock *tp,
+					      struct inet_request_sock *ireq)
+{
+	return 1;
+}
+
+static struct smc_ops __bpf_smc_bpf_ops = {
+	.set_option		= __bpf_smc_stub_set_tcp_option,
+	.set_option_cond	= __bpf_smc_stub_set_tcp_option_cond,
+};
+
+static int smc_bpf_ops_init(struct btf *btf) { return 0; }
+
+static int smc_bpf_ops_reg(void *kdata, struct bpf_link *link)
+{
+	return smc_ops_reg(kdata);
+}
+
+static void smc_bpf_ops_unreg(void *kdata, struct bpf_link *link)
+{
+	smc_ops_unreg(kdata);
+}
+
+static int smc_bpf_ops_init_member(const struct btf_type *t,
+				   const struct btf_member *member,
+				   void *kdata, const void *udata)
+{
+	const struct smc_ops *u_ops;
+	struct smc_ops *k_ops;
+	u32 moff;
+
+	u_ops = (const struct smc_ops *)udata;
+	k_ops = (struct smc_ops *)kdata;
+
+	moff = __btf_member_bit_offset(t, member) / 8;
+	switch (moff) {
+	case offsetof(struct smc_ops, name):
+		if (bpf_obj_name_cpy(k_ops->name, u_ops->name,
+				     sizeof(u_ops->name)) <= 0)
+			return -EINVAL;
+		return 1;
+	case offsetof(struct smc_ops, flags):
+		if (u_ops->flags & ~SMC_OPS_ALL_FLAGS)
+			return -EINVAL;
+		k_ops->flags = u_ops->flags;
+		return 1;
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+static int smc_bpf_ops_check_member(const struct btf_type *t,
+				    const struct btf_member *member,
+				    const struct bpf_prog *prog)
+{
+	u32 moff = __btf_member_bit_offset(t, member) / 8;
+
+	switch (moff) {
+	case offsetof(struct smc_ops, name):
+	case offsetof(struct smc_ops, flags):
+	case offsetof(struct smc_ops, set_option):
+	case offsetof(struct smc_ops, set_option_cond):
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static const struct bpf_verifier_ops smc_bpf_verifier_ops = {
+	.get_func_proto		= bpf_base_func_proto,
+	.is_valid_access	= bpf_tracing_btf_ctx_access,
+};
+
+static struct bpf_struct_ops bpf_smc_bpf_ops = {
+	.name		= "smc_ops",
+	.init		= smc_bpf_ops_init,
+	.reg		= smc_bpf_ops_reg,
+	.unreg		= smc_bpf_ops_unreg,
+	.cfi_stubs	= &__bpf_smc_bpf_ops,
+	.verifier_ops	= &smc_bpf_verifier_ops,
+	.init_member	= smc_bpf_ops_init_member,
+	.check_member	= smc_bpf_ops_check_member,
+	.owner		= THIS_MODULE,
+};
+
+int smc_bpf_struct_ops_init(void)
+{
+	return register_bpf_struct_ops(&bpf_smc_bpf_ops, smc_ops);
+}
net/smc/smc_ops.h +0 -0
index 214f4c99efd4..f4e50eae13f6 100644

net/smc/smc_ops.h line 28

Quoted 7 lines
net/smc/smc_ops.h +2 -0
@@ -22,8 +22,10 @@
   * Note: Caller MUST ensure it's was invoked under rcu_read_lock.
   */
  struct smc_ops *smc_ops_find_by_name(const char *name);
+int smc_bpf_struct_ops_init(void);
  #else
  static inline struct smc_ops *smc_ops_find_by_name(const char *name) { return NULL; }
+static inline int smc_bpf_struct_ops_init(void) { return 0; }

Both smc_ops_find_by_name and smc_bpf_struct_ops_init seem to be dead codes. Enabling/Disabling CONFIG_SMC_OPS, the above 2 inline functions will not be called. The 2 functions should be removed.

Zhu Yanjun

Quoted 3 lines
net/smc/smc_ops.h +0 -0
  #endif /* CONFIG_SMC_OPS*/
 
  #endif /* __SMC_OPS */
author maintainer
#7 lore raw

Re: [PATCH bpf-next v3 3/5] net/smc: bpf: register smc_ops info struct_ops

Quoted 32 lines · Zhu Yanjun
Quoted 19 lines · D. Wythe

To implement injection capability for smc via struct_ops, so that user can make their own smc_ops to modify the behavior of smc stack.

Currently, user can write their own implememtion to choose whether to use SMC or not before TCP 3rd handshake to be comleted. In the future, users can implement more complex functions on smc by expanding it.

---
 net/smc/af_smc.c  | 10 +++++
 net/smc/smc_ops.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++
 net/smc/smc_ops.h |  2 +
 3 files changed, 111 insertions(+)
net/smc/af_smc.c +0 -0
index 9d76e902fd77..6adedae2986d 100644

net/smc/af_smc.c line 61

Quoted 7 lines
net/smc/af_smc.c +2 -0
@@ -55,6 +55,7 @@
 #include "smc_sysctl.h"
 #include "smc_loopback.h"
 #include "smc_inet.h"
+static struct bpf_struct_ops bpf_smc_bpf_ops = {
 #else
 static inline struct smc_ops *smc_ops_find_by_name(const char *name) { return NULL; }
+static inline int smc_bpf_struct_ops_init(void) { return 0; }

Both smc_ops_find_by_name and smc_bpf_struct_ops_init seem to be dead codes. Enabling/Disabling CONFIG_SMC_OPS, the above 2 inline functions will not be called. The 2 functions should be removed.

Quoted 2 lines

Zhu Yanjun

Good catch. I will fix this in the next version.

Thanks,
D. Wythe

Quoted 2 lines
Quoted 2 lines

#endif /* CONFIG_SMC_OPS*/
#endif /* __SMC_OPS */

author maintainer
#8 lore raw

PATCH v3 4/5 libbpf: fix error when st-prefix_ops and ops from differ btf

In reply to D. Wythe

When a struct_ops named xxx_ops was registered by a module, and it will be used in both built-in modules and the module itself, so that the btf_type of xxx_ops will be present in btf_vmlinux instead of in btf_mod, which means that the btf_type of bpf_struct_ops_xxx_ops and xxx_ops will not be in the same btf.

Here are four possible case:

+--------+-------------+-------------+---------------------------------+
|        | st_opx_xxx  | xxx         |                                 |
+--------+-------------+-------------+---------------------------------+
| case 0 | btf_vmlinux | bft_vmlinux | be used and reg only in vmlinux |
+--------+-------------+-------------+---------------------------------+
| case 1 | btf_vmlinux | bpf_mod     | INVALID                         |
+--------+-------------+-------------+---------------------------------+
| case 2 | btf_mod     | btf_vmlinux | reg in mod but be used both in  |
|        |             |             | vmlinux and mod.                |
+--------+-------------+-------------+---------------------------------+
| case 3 | btf_mod     | btf_mod     | be used and reg only in mod     |
+--------+-------------+-------------+---------------------------------+

At present, cases 0, 1, and 3 can be correctly identified, because st_ops_xxx is searched from the same btf with xxx. In order to handle case 2 correctly without affecting other cases, we cannot simply change the search method for st_ops_xxx from find_btf_by_prefix_kind() to find_ksym_btf_id(), because in this way, case 1 will not be recognized anymore.

To address this issue, if st_ops_xxx cannot be found in the btf with xxx and mod_btf does not exist, do find_ksym_btf_id() again to avoid such issue.

Fixes 590a00888250 ("bpf: libbpf: Add STRUCT_OPS support")
---
 tools/lib/bpf/libbpf.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)
tools/lib/bpf/libbpf.c +17 -8
index 66173ddb5a2d..56bf74894110 100644
@@ -1005,7 +1005,8 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
 	const struct btf_member *kern_data_member;
 	struct btf *btf = NULL;
 	__s32 kern_vtype_id, kern_type_id;
-	char tname[256];
+	char tname[256], stname[256];
+	int ret;
 	__u32 i;
 
 	snprintf(tname, sizeof(tname), "%.*s",
@@ -1020,17 +1021,25 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
 	}
 	kern_type = btf__type_by_id(btf, kern_type_id);
 
+	ret = snprintf(stname, sizeof(stname), "%s%s", STRUCT_OPS_VALUE_PREFIX, tname);
+	if (ret < 0 || ret >= sizeof(stname))
+		return -ENAMETOOLONG;
+
 	/* Find the corresponding "map_value" type that will be used
 	 * in map_update(BPF_MAP_TYPE_STRUCT_OPS).  For example,
 	 * find "struct bpf_struct_ops_tcp_congestion_ops" from the
 	 * btf_vmlinux.
 	 */
-	kern_vtype_id = find_btf_by_prefix_kind(btf, STRUCT_OPS_VALUE_PREFIX,
-						tname, BTF_KIND_STRUCT);
+	kern_vtype_id = btf__find_by_name_kind(btf, stname, BTF_KIND_STRUCT);
 	if (kern_vtype_id < 0) {
-		pr_warn("struct_ops init_kern: struct %s%s is not found in kernel BTF\n",
-			STRUCT_OPS_VALUE_PREFIX, tname);
-		return kern_vtype_id;
+		if (kern_vtype_id == -ENOENT && !*mod_btf)
+			kern_vtype_id = find_ksym_btf_id(obj, stname, BTF_KIND_STRUCT,
+							 &btf, mod_btf);
+		if (kern_vtype_id < 0) {
+			pr_warn("struct_ops init_kern: struct %s is not found in kernel BTF\n",
+				stname);
+			return kern_vtype_id;
+		}
 	}
 	kern_vtype = btf__type_by_id(btf, kern_vtype_id);
 
@@ -1046,8 +1055,8 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
 			break;
 	}
 	if (i == btf_vlen(kern_vtype)) {
-		pr_warn("struct_ops init_kern: struct %s data is not found in struct %s%s\n",
-			tname, STRUCT_OPS_VALUE_PREFIX, tname);
+		pr_warn("struct_ops init_kern: struct %s data is not found in struct %s\n",
+			tname, stname);
 		return -EINVAL;
 	}
 
Signature · 1 line
2.45.0
#9 lore raw

Re: [PATCH bpf-next v3 4/5] libbpf: fix error when st-prefix_ops and ops from differ btf

Quoted 53 lines · D. Wythe

When a struct_ops named xxx_ops was registered by a module, and it will be used in both built-in modules and the module itself, so that the btf_type of xxx_ops will be present in btf_vmlinux instead of in btf_mod, which means that the btf_type of bpf_struct_ops_xxx_ops and xxx_ops will not be in the same btf.

Here are four possible case:

+--------+-------------+-------------+---------------------------------+
|        | st_opx_xxx  | xxx         |                                 |
+--------+-------------+-------------+---------------------------------+
| case 0 | btf_vmlinux | bft_vmlinux | be used and reg only in vmlinux |
+--------+-------------+-------------+---------------------------------+
| case 1 | btf_vmlinux | bpf_mod     | INVALID                         |
+--------+-------------+-------------+---------------------------------+
| case 2 | btf_mod     | btf_vmlinux | reg in mod but be used both in  |
|        |             |             | vmlinux and mod.                |
+--------+-------------+-------------+---------------------------------+
| case 3 | btf_mod     | btf_mod     | be used and reg only in mod     |
+--------+-------------+-------------+---------------------------------+

At present, cases 0, 1, and 3 can be correctly identified, because st_ops_xxx is searched from the same btf with xxx. In order to handle case 2 correctly without affecting other cases, we cannot simply change the search method for st_ops_xxx from find_btf_by_prefix_kind() to find_ksym_btf_id(), because in this way, case 1 will not be recognized anymore.

To address this issue, if st_ops_xxx cannot be found in the btf with xxx and mod_btf does not exist, do find_ksym_btf_id() again to avoid such issue.

Fixes 590a00888250 ("bpf: libbpf: Add STRUCT_OPS support")
---
  tools/lib/bpf/libbpf.c | 25 +++++++++++++++++--------
  1 file changed, 17 insertions(+), 8 deletions(-)
tools/lib/bpf/libbpf.c +2 -1
index 66173ddb5a2d..56bf74894110 100644
@@ -1005,7 +1005,8 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
  	const struct btf_member *kern_data_member;
  	struct btf *btf = NULL;
  	__s32 kern_vtype_id, kern_type_id;
-	char tname[256];
+	char tname[256], stname[256];
+	int ret;
  	__u32 i;
 
  	snprintf(tname, sizeof(tname), "%.*s",

tools/lib/bpf/libbpf.c find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw, · line 1024

Quoted 4 lines
tools/lib/bpf/libbpf.c +1 -0
@@ -1020,17 +1021,25 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
  	}
  	kern_type = btf__type_by_id(btf, kern_type_id);
 
+	ret = snprintf(stname, sizeof(stname), "%s%s", STRUCT_OPS_VALUE_PREFIX, tname);

How about always look for "struct bpf_struct_ops_smc_ops" first, figure out the btf, and then look for "struct smc_ops", would it work?

If CONFIG_SMC=y instead of =m, this change cannot be tested?

Quoted 37 lines
tools/lib/bpf/libbpf.c +14 -7
+	if (ret < 0 || ret >= sizeof(stname))
+		return -ENAMETOOLONG;
+
  	/* Find the corresponding "map_value" type that will be used
  	 * in map_update(BPF_MAP_TYPE_STRUCT_OPS).  For example,
  	 * find "struct bpf_struct_ops_tcp_congestion_ops" from the
  	 * btf_vmlinux.
  	 */
-	kern_vtype_id = find_btf_by_prefix_kind(btf, STRUCT_OPS_VALUE_PREFIX,
-						tname, BTF_KIND_STRUCT);
+	kern_vtype_id = btf__find_by_name_kind(btf, stname, BTF_KIND_STRUCT);
  	if (kern_vtype_id < 0) {
-		pr_warn("struct_ops init_kern: struct %s%s is not found in kernel BTF\n",
-			STRUCT_OPS_VALUE_PREFIX, tname);
-		return kern_vtype_id;
+		if (kern_vtype_id == -ENOENT && !*mod_btf)
+			kern_vtype_id = find_ksym_btf_id(obj, stname, BTF_KIND_STRUCT,
+							 &btf, mod_btf);
+		if (kern_vtype_id < 0) {
+			pr_warn("struct_ops init_kern: struct %s is not found in kernel BTF\n",
+				stname);
+			return kern_vtype_id;
+		}
  	}
  	kern_vtype = btf__type_by_id(btf, kern_vtype_id);
 
@@ -1046,8 +1055,8 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
  			break;
  	}
  	if (i == btf_vlen(kern_vtype)) {
-		pr_warn("struct_ops init_kern: struct %s data is not found in struct %s%s\n",
-			tname, STRUCT_OPS_VALUE_PREFIX, tname);
+		pr_warn("struct_ops init_kern: struct %s data is not found in struct %s\n",
+			tname, stname);
  		return -EINVAL;
  	}
 
author maintainer
#10 lore raw

Re: [PATCH bpf-next v3 4/5] libbpf: fix error when st-prefix_ops and ops from differ btf

Quoted 16 lines · Martin KaFai Lau
Quoted 15 lines · D. Wythe

Here are four possible case:

+--------+-------------+-------------+---------------------------------+
|        | st_opx_xxx  | xxx         |                                 |
+--------+-------------+-------------+---------------------------------+
| case 0 | btf_vmlinux | bft_vmlinux | be used and reg only in vmlinux |
+--------+-------------+-------------+---------------------------------+
| case 1 | btf_vmlinux | bpf_mod     | INVALID                         |
+--------+-------------+-------------+---------------------------------+
| case 2 | btf_mod     | btf_vmlinux | reg in mod but be used both in  |
|        |             |             | vmlinux and mod.                |
+--------+-------------+-------------+---------------------------------+
| case 3 | btf_mod     | btf_mod     | be used and reg only in mod     |
+--------+-------------+-------------+---------------------------------+
Quoted 15 lines

tools/lib/bpf/libbpf.c find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw, · line 1023

Quoted 11 lines

At present, cases 0, 1, and 3 can be correctly identified, because st_ops_xxx is searched from the same btf with xxx. In order to handle case 2 correctly without affecting other cases, we cannot simply change the search method for st_ops_xxx from find_btf_by_prefix_kind() to find_ksym_btf_id(), because in this way, case 1 will not be recognized anymore.

tools/lib/bpf/libbpf.c +1 -0
 	snprintf(tname, sizeof(tname), "%.*s",
@@ -1020,17 +1021,25 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
 	}
 	kern_type = btf__type_by_id(btf, kern_type_id);
+	ret = snprintf(stname, sizeof(stname), "%s%s", STRUCT_OPS_VALUE_PREFIX, tname);

How about always look for "struct bpf_struct_ops_smc_ops" first, figure out the btf, and then look for "struct smc_ops", would it work?

I think this might not work, as the core issue lies in the fact that bpf_struct_ops_smc_ops and smc_ops are located on different btf. Searching for one fisrt cannot lead to the inference of the other.

Quoted 3 lines

If CONFIG_SMC=y instead of =m, this change cannot be tested?

That is indeed a problem, but currently there is no better solution unless the CI can add a step to run 'make modules_install'.

Best wishes,
D. Wythe

Quoted 35 lines
Quoted 35 lines
tools/lib/bpf/libbpf.c +14 -7
+	if (ret < 0 || ret >= sizeof(stname))
+		return -ENAMETOOLONG;
+
 	/* Find the corresponding "map_value" type that will be used
 	 * in map_update(BPF_MAP_TYPE_STRUCT_OPS).  For example,
 	 * find "struct bpf_struct_ops_tcp_congestion_ops" from the
 	 * btf_vmlinux.
 	 */
-	kern_vtype_id = find_btf_by_prefix_kind(btf, STRUCT_OPS_VALUE_PREFIX,
-						tname, BTF_KIND_STRUCT);
+	kern_vtype_id = btf__find_by_name_kind(btf, stname, BTF_KIND_STRUCT);
 	if (kern_vtype_id < 0) {
-		pr_warn("struct_ops init_kern: struct %s%s is not found in kernel BTF\n",
-			STRUCT_OPS_VALUE_PREFIX, tname);
-		return kern_vtype_id;
+		if (kern_vtype_id == -ENOENT && !*mod_btf)
+			kern_vtype_id = find_ksym_btf_id(obj, stname, BTF_KIND_STRUCT,
+							 &btf, mod_btf);
+		if (kern_vtype_id < 0) {
+			pr_warn("struct_ops init_kern: struct %s is not found in kernel BTF\n",
+				stname);
+			return kern_vtype_id;
+		}
 	}
 	kern_vtype = btf__type_by_id(btf, kern_vtype_id);
@@ -1046,8 +1055,8 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
 			break;
 	}
 	if (i == btf_vlen(kern_vtype)) {
-		pr_warn("struct_ops init_kern: struct %s data is not found in struct %s%s\n",
-			tname, STRUCT_OPS_VALUE_PREFIX, tname);
+		pr_warn("struct_ops init_kern: struct %s data is not found in struct %s\n",
+			tname, stname);
 		return -EINVAL;
 	}
#11 lore raw

Re: [PATCH bpf-next v3 4/5] libbpf: fix error when st-prefix_ops and ops from differ btf

Quoted 33 lines · D. Wythe
Quoted 16 lines · Martin KaFai Lau
Quoted 15 lines · D. Wythe

Here are four possible case:

+--------+-------------+-------------+---------------------------------+
|        | st_opx_xxx  | xxx         |                                 |
+--------+-------------+-------------+---------------------------------+
| case 0 | btf_vmlinux | bft_vmlinux | be used and reg only in vmlinux |
+--------+-------------+-------------+---------------------------------+
| case 1 | btf_vmlinux | bpf_mod     | INVALID                         |
+--------+-------------+-------------+---------------------------------+
| case 2 | btf_mod     | btf_vmlinux | reg in mod but be used both in  |
|        |             |             | vmlinux and mod.                |
+--------+-------------+-------------+---------------------------------+
| case 3 | btf_mod     | btf_mod     | be used and reg only in mod     |
+--------+-------------+-------------+---------------------------------+
Quoted 15 lines

tools/lib/bpf/libbpf.c find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw, · line 1023

Quoted 11 lines

At present, cases 0, 1, and 3 can be correctly identified, because st_ops_xxx is searched from the same btf with xxx. In order to handle case 2 correctly without affecting other cases, we cannot simply change the search method for st_ops_xxx from find_btf_by_prefix_kind() to find_ksym_btf_id(), because in this way, case 1 will not be recognized anymore.
snprintf(tname, sizeof(tname), "%.*s",

tools/lib/bpf/libbpf.c +1 -0
@@ -1020,17 +1021,25 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
  	}
  	kern_type = btf__type_by_id(btf, kern_type_id);
+	ret = snprintf(stname, sizeof(stname), "%s%s", STRUCT_OPS_VALUE_PREFIX, tname);

How about always look for "struct bpf_struct_ops_smc_ops" first, figure out the btf, and then look for "struct smc_ops", would it work?

Quoted 3 lines

I think this might not work, as the core issue lies in the fact that bpf_struct_ops_smc_ops and smc_ops are located on different btf. Searching for one fisrt cannot lead to the inference of the other.

Take a look at btf_find_by_name_kind(btf, 1 /* from base_btf */, ...) and also btf_type_by_id(). It starts searching from the btf->base_btf which should be the btf_vmlinux here and should have the "struct smc_ops". Please try.

author maintainer
#12 lore raw

Re: [PATCH bpf-next v3 4/5] libbpf: fix error when st-prefix_ops and ops from differ btf

Quoted 34 lines · Martin KaFai Lau
Quoted 33 lines · D. Wythe
Quoted 31 lines · Martin KaFai Lau
Quoted 23 lines · D. Wythe

Here are four possible case:

+--------+-------------+-------------+---------------------------------+
|        | st_opx_xxx  | xxx         |                                 |
+--------+-------------+-------------+---------------------------------+
| case 0 | btf_vmlinux | bft_vmlinux | be used and reg only in vmlinux |
+--------+-------------+-------------+---------------------------------+
| case 1 | btf_vmlinux | bpf_mod     | INVALID                         |
+--------+-------------+-------------+---------------------------------+
| case 2 | btf_mod     | btf_vmlinux | reg in mod but be used both in  |
|        |             |             | vmlinux and mod.                |
+--------+-------------+-------------+---------------------------------+
| case 3 | btf_mod     | btf_mod     | be used and reg only in mod     |
+--------+-------------+-------------+---------------------------------+

At present, cases 0, 1, and 3 can be correctly identified, because st_ops_xxx is searched from the same btf with xxx. In order to handle case 2 correctly without affecting other cases, we cannot simply change the search method for st_ops_xxx from find_btf_by_prefix_kind() to find_ksym_btf_id(), because in this way, case 1 will not be recognized anymore.

tools/lib/bpf/libbpf.c +0 -0
 	snprintf(tname, sizeof(tname), "%.*s",

tools/lib/bpf/libbpf.c find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw, · line 1023

Quoted 3 lines
tools/lib/bpf/libbpf.c +1 -0
@@ -1020,17 +1021,25 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
 	}
 	kern_type = btf__type_by_id(btf, kern_type_id);
+	ret = snprintf(stname, sizeof(stname), "%s%s", STRUCT_OPS_VALUE_PREFIX, tname);

How about always look for "struct bpf_struct_ops_smc_ops" first, figure out the btf, and then look for "struct smc_ops", would it work?

Quoted 8 lines
Quoted 3 lines

I think this might not work, as the core issue lies in the fact that bpf_struct_ops_smc_ops and smc_ops are located on different btf. Searching for one fisrt cannot lead to the inference of the other.

Take a look at btf_find_by_name_kind(btf, 1 /* from base_btf */, ...) and also btf_type_by_id(). It starts searching from the btf->base_btf which should be the btf_vmlinux here and should have the "struct smc_ops". Please try.

Got it, I will try it, thanks for your suggestion.

D. Wythe

#13 lore raw

Re: [PATCH bpf-next v3 4/5] libbpf: fix error when st-prefix_ops and ops from differ btf

In reply to D. Wythe
Quoted 71 lines · D. Wythe

When a struct_ops named xxx_ops was registered by a module, and it will be used in both built-in modules and the module itself, so that the btf_type of xxx_ops will be present in btf_vmlinux instead of in btf_mod, which means that the btf_type of bpf_struct_ops_xxx_ops and xxx_ops will not be in the same btf.

Here are four possible case:

+--------+-------------+-------------+---------------------------------+
|        | st_opx_xxx  | xxx         |                                 |
+--------+-------------+-------------+---------------------------------+
| case 0 | btf_vmlinux | bft_vmlinux | be used and reg only in vmlinux |
+--------+-------------+-------------+---------------------------------+
| case 1 | btf_vmlinux | bpf_mod     | INVALID                         |
+--------+-------------+-------------+---------------------------------+
| case 2 | btf_mod     | btf_vmlinux | reg in mod but be used both in  |
|        |             |             | vmlinux and mod.                |
+--------+-------------+-------------+---------------------------------+
| case 3 | btf_mod     | btf_mod     | be used and reg only in mod     |
+--------+-------------+-------------+---------------------------------+

At present, cases 0, 1, and 3 can be correctly identified, because st_ops_xxx is searched from the same btf with xxx. In order to handle case 2 correctly without affecting other cases, we cannot simply change the search method for st_ops_xxx from find_btf_by_prefix_kind() to find_ksym_btf_id(), because in this way, case 1 will not be recognized anymore.

To address this issue, if st_ops_xxx cannot be found in the btf with xxx and mod_btf does not exist, do find_ksym_btf_id() again to avoid such issue.

Fixes 590a00888250 ("bpf: libbpf: Add STRUCT_OPS support")
---
 tools/lib/bpf/libbpf.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)
tools/lib/bpf/libbpf.c +7 -4
index 66173ddb5a2d..56bf74894110 100644
@@ -1005,7 +1005,8 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
        const struct btf_member *kern_data_member;
        struct btf *btf = NULL;
        __s32 kern_vtype_id, kern_type_id;
-       char tname[256];
+       char tname[256], stname[256];
+       int ret;
        __u32 i;
 
        snprintf(tname, sizeof(tname), "%.*s",
@@ -1020,17 +1021,25 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
        }
        kern_type = btf__type_by_id(btf, kern_type_id);
 
+       ret = snprintf(stname, sizeof(stname), "%s%s", STRUCT_OPS_VALUE_PREFIX, tname);
+       if (ret < 0 || ret >= sizeof(stname))
+               return -ENAMETOOLONG;
+
        /* Find the corresponding "map_value" type that will be used
         * in map_update(BPF_MAP_TYPE_STRUCT_OPS).  For example,
         * find "struct bpf_struct_ops_tcp_congestion_ops" from the
         * btf_vmlinux.
         */
-       kern_vtype_id = find_btf_by_prefix_kind(btf, STRUCT_OPS_VALUE_PREFIX,
-                                               tname, BTF_KIND_STRUCT);
+       kern_vtype_id = btf__find_by_name_kind(btf, stname, BTF_KIND_STRUCT);
        if (kern_vtype_id < 0) {
-               pr_warn("struct_ops init_kern: struct %s%s is not found in kernel BTF\n",

tools/lib/bpf/libbpf.c find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw, · line 1042

Quoted 10 lines
tools/lib/bpf/libbpf.c +8 -2
-                       STRUCT_OPS_VALUE_PREFIX, tname);
-               return kern_vtype_id;
+               if (kern_vtype_id == -ENOENT && !*mod_btf)
+                       kern_vtype_id = find_ksym_btf_id(obj, stname, BTF_KIND_STRUCT,
+                                                        &btf, mod_btf);
+               if (kern_vtype_id < 0) {
+                       pr_warn("struct_ops init_kern: struct %s is not found in kernel BTF\n",
+                               stname);
+                       return kern_vtype_id;
+               }

purely from the coding perspective, this is unnecessarily nested and convoluted. Wouldn't this work the same but be less nested:

kern_vtype_id = btf__find_by_name_kind(btf, stname, BTF_KIND_STRUCT); if (kern_vtype_id == -ENOENT && !*mod_btf)

    kern_vtype_id = find_ksym_btf_id(...);
if (kern_vtype_id < 0) {
    pr_warn(...);
    return kern_vtype_id;
}
Quoted 17 lines
tools/lib/bpf/libbpf.c +2 -3
        }
        kern_vtype = btf__type_by_id(btf, kern_vtype_id);
 
@@ -1046,8 +1055,8 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
                        break;
        }
        if (i == btf_vlen(kern_vtype)) {
-               pr_warn("struct_ops init_kern: struct %s data is not found in struct %s%s\n",
-                       tname, STRUCT_OPS_VALUE_PREFIX, tname);
+               pr_warn("struct_ops init_kern: struct %s data is not found in struct %s\n",
+                       tname, stname);
                return -EINVAL;
        }
 
--

2.45.0

author maintainer
#14 lore raw

Re: [PATCH bpf-next v3 4/5] libbpf: fix error when st-prefix_ops and ops from differ btf

Quoted 38 lines · Andrii Nakryiko
Quoted 29 lines · D. Wythe

When a struct_ops named xxx_ops was registered by a module, and it will be used in both built-in modules and the module itself, so that the btf_type of xxx_ops will be present in btf_vmlinux instead of in btf_mod, which means that the btf_type of bpf_struct_ops_xxx_ops and xxx_ops will not be in the same btf.

Here are four possible case:

+--------+-------------+-------------+---------------------------------+
|        | st_opx_xxx  | xxx         |                                 |
+--------+-------------+-------------+---------------------------------+
| case 0 | btf_vmlinux | bft_vmlinux | be used and reg only in vmlinux |
+--------+-------------+-------------+---------------------------------+
| case 1 | btf_vmlinux | bpf_mod     | INVALID                         |
+--------+-------------+-------------+---------------------------------+
| case 2 | btf_mod     | btf_vmlinux | reg in mod but be used both in  |
|        |             |             | vmlinux and mod.                |
+--------+-------------+-------------+---------------------------------+
| case 3 | btf_mod     | btf_mod     | be used and reg only in mod     |
+--------+-------------+-------------+---------------------------------+

At present, cases 0, 1, and 3 can be correctly identified, because st_ops_xxx is searched from the same btf with xxx. In order to handle case 2 correctly without affecting other cases, we cannot simply change the search method for st_ops_xxx from find_btf_by_prefix_kind() to find_ksym_btf_id(), because in this way, case 1 will not be recognized anymore.

Quoted 4 lines

To address this issue, if st_ops_xxx cannot be found in the btf with xxx and mod_btf does not exist, do find_ksym_btf_id() again to avoid such issue.
+ }

purely from the coding perspective, this is unnecessarily nested and convoluted. Wouldn't this work the same but be less nested:

Quoted 7 lines

kern_vtype_id = btf__find_by_name_kind(btf, stname, BTF_KIND_STRUCT); if (kern_vtype_id == -ENOENT && !*mod_btf)

    kern_vtype_id = find_ksym_btf_id(...);
if (kern_vtype_id < 0) {
    pr_warn(...);
    return kern_vtype_id;
}

Hi Andrii,

It's indeed more concise with your code. Thanks very much for your suggestion. And Martin has provided us with another suggestion to address this issue, and according to his plan, there shall be no such complex conditional checks too.

Anyway, I will keep my code concise in the next version. Thank you for the reminder.

Best wishes,
D. Wythe

Quoted 19 lines
Quoted 19 lines

        }
        kern_vtype = btf__type_by_id(btf, kern_vtype_id);
tools/lib/bpf/libbpf.c +2 -3
@@ -1046,8 +1055,8 @@ find_struct_ops_kern_types(struct bpf_object *obj, const char *tname_raw,
                        break;
        }
        if (i == btf_vlen(kern_vtype)) {
-               pr_warn("struct_ops init_kern: struct %s data is not found in struct %s%s\n",
-                       tname, STRUCT_OPS_VALUE_PREFIX, tname);
+               pr_warn("struct_ops init_kern: struct %s data is not found in struct %s\n",
+                       tname, stname);
                return -EINVAL;
        }
 
--

2.45.0

author maintainer
#15 lore raw

PATCH v3 5/5 bpf/selftests: add selftest for bpf_smc_ops

In reply to D. Wythe

This tests introduces a tiny smc_ops for filtering SMC connections based on IP pairs, and also adds a realistic topology model to verify this ops.

Also, we can only use SMC loopback under CI test, so an additional configuration needs to be enabled.

Follow the steps below to run this test.

make -C tools/testing/selftests/bpf
cd tools/testing/selftests/bpf
sudo ./test_progs -t smc

Results shows:
Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

---
 tools/testing/selftests/bpf/config            |   4 +
 .../selftests/bpf/prog_tests/test_bpf_smc.c   | 535 ++++++++++++++++++
 tools/testing/selftests/bpf/progs/bpf_smc.c   | 109 ++++
 3 files changed, 648 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_smc.c
tools/testing/selftests/bpf/config +4 -0
index c378d5d07e02..79248eb04ec4 100644
@@ -113,3 +113,7 @@ CONFIG_XDP_SOCKETS=y
 CONFIG_XFRM_INTERFACE=y
 CONFIG_TCP_CONG_DCTCP=y
 CONFIG_TCP_CONG_BBR=y
+CONFIG_INFINIBAND=m
+CONFIG_SMC=m
+CONFIG_SMC_OPS=y
+CONFIG_SMC_LO=y
\ No newline at end of file
tools/testing/selftests/bpf/prog_tests/test_bpf_smc.c new file +535 -0

535 lines not on this page · raw message

tools/testing/selftests/bpf/progs/bpf_smc.c new file +109 -0

109 lines not on this page · raw message

Signature · 1 line
2.45.0
#16 lore raw

Re: [PATCH bpf-next v3 5/5] bpf/selftests: add selftest for bpf_smc_ops

tools/testing/selftests/bpf/progs/bpf_smc.c line 11

Quoted 11 lines · D. Wythe
tools/testing/selftests/bpf/progs/bpf_smc.c new file +11 -0
+// SPDX-License-Identifier: GPL-2.0
+
+#include "vmlinux.h"
+
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_tracing_net.h"
+
+char _license[] SEC("license") = "GPL";
+
+struct smc_sock {

I suspect this should be "smc_sock___local". Otherwise, it can't compile if the same type is found in vmlinux.h.

I only looked at the high level of prog_tests/test_bpf_smc.c. A few comments,

Try to reuse the helpers in network_helpers.c and test_progs.c, e.g. netns creation helpers, start_server, ...etc. There are many examples in selftests/bpf/prog_tests using them.

I see 1s timeout everywhere. BPF CI could be slow some time. Please consider how reliable the multi-thread test is. If the test is too flaky, it will be put in the selftests/bpf/DENYLIST.

Quoted 5 lines
tools/testing/selftests/bpf/progs/bpf_smc.c new file +5 -0
+	struct sock sk;
+	struct smc_sock *listen_smc;
+	bool use_fallback;
+} __attribute__((preserve_access_index));
+
author maintainer
#17 lore raw

Re: [PATCH bpf-next v3 5/5] bpf/selftests: add selftest for bpf_smc_ops

Quoted 16 lines · Martin KaFai Lau

tools/testing/selftests/bpf/progs/bpf_smc.c line 11

Quoted 11 lines · D. Wythe
tools/testing/selftests/bpf/progs/bpf_smc.c new file +11 -0
+// SPDX-License-Identifier: GPL-2.0
+
+#include "vmlinux.h"
+
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_tracing_net.h"
+
+char _license[] SEC("license") = "GPL";
+
+struct smc_sock {

I suspect this should be "smc_sock___local". Otherwise, it can't compile if the same type is found in vmlinux.h.

Yes, it has been changed to ___local.

Quoted 10 lines

I only looked at the high level of prog_tests/test_bpf_smc.c. A few comments,

Try to reuse the helpers in network_helpers.c and test_progs.c, e.g. netns creation helpers, start_server, ...etc. There are many examples in selftests/bpf/prog_tests using them.

I see 1s timeout everywhere. BPF CI could be slow some time. Please consider how reliable the multi-thread test is. If the test is too flaky, it will be put in the selftests/bpf/DENYLIST.

Got it, I will remove those timeouts in the next version. I have tried reusing these helpers, it is very convenient and makes code more concise!

D. Wythe

Quoted 5 lines
Quoted 5 lines
tools/testing/selftests/bpf/progs/bpf_smc.c new file +5 -0
+	struct sock sk;
+	struct smc_sock *listen_smc;
+	bool use_fallback;
+} __attribute__((preserve_access_index));
+

A message that is not in the archive (sent off-list or not yet received): bf1eef2ada1330c92b6326e90e23482e53a759fd40cd6c25832ae7f72207930a@mail.kernel.org

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#22 lore raw
Quoted 69 lines · D. Wythe
Quoted 27 lines · Daniel Borkmann
Quoted 26 lines · D. Wythe
Quoted 25 lines · On Wed, Dec 18, 2024 at 03:23:01AM +0000, [email protected] wrote:

Dear patch submitter,

CI has tested the following submission:

Status:     FAILURE
Name:       [bpf-next,v3,0/5] net/smc: Introduce smc_ops
Patchwork:  https://patchwork.kernel.org/project/netdevbpf/list/?series=918946&state=*
Matrix:     https://github.com/kernel-patches/bpf/actions/runs/12385547644

Failed jobs:
test_progs-aarch64-gcc: https://github.com/kernel-patches/bpf/actions/runs/12385547644/job/34572137789
test_progs_no_alu32-aarch64-gcc: https://github.com/kernel-patches/bpf/actions/runs/12385547644/job/34572137985
test_progs-s390x-gcc: https://github.com/kernel-patches/bpf/actions/runs/12385547644/job/34572116351
test_progs_no_alu32-s390x-gcc: https://github.com/kernel-patches/bpf/actions/runs/12385547644/job/34572116670
test_progs-x86_64-gcc: https://github.com/kernel-patches/bpf/actions/runs/12385547644/job/34572147114
test_progs_no_alu32-x86_64-gcc: https://github.com/kernel-patches/bpf/actions/runs/12385547644/job/34572147579
test_progs-x86_64-llvm-17: https://github.com/kernel-patches/bpf/actions/runs/12385547644/job/34572152627
test_progs_no_alu32-x86_64-llvm-17: https://github.com/kernel-patches/bpf/actions/runs/12385547644/job/34572152400
test_progs-x86_64-llvm-18: https://github.com/kernel-patches/bpf/actions/runs/12385547644/job/34572134514
test_progs_cpuv4-x86_64-llvm-18: https://github.com/kernel-patches/bpf/actions/runs/12385547644/job/34572134779
test_progs_no_alu32-x86_64-llvm-18: https://github.com/kernel-patches/bpf/actions/runs/12385547644/job/34572135179

First test_progs failure (test_progs-aarch64-gcc):
#27 bpf_smc
load_smc_module:FAIL:create ipproto_smc unexpected create ipproto_smc: actual -1 < expected 0 setup for smc test failed, test SKIP:

Quoted 8 lines
Quoted 3 lines

A bit weird. How can I reproduce this test on my own? Also, it seems that I shouldn't use the ASSERT_XX macros in the setup phase, so that it can properly skip.

In the BPF selftest dir, you could try to repro with vmtest.sh script :

   ./vmtest.sh -- ./test_progs -t smc

Potentially tools/testing/selftests/bpf/config* needs to be extended as first step.

Hi Daniel,

Thanks very much for your prompt. I think I have identified the issue.

I reproduced this issue by vmtest, and I suspect it may have been caused by not execute "make modules_install" before executing vmtest since CONFIG_SMC=m in tools/testing/selftests/bpf/config*;

[root@localhost linux]$ vmtest -k arch/x86_64/boot/bzImage "modprobe smc;" => bzImage
===> Booting
===> Setting up VM
===> Running command
modprobe: FATAL: Module smc not found in directory /lib/modules/6.13.0-rc3-00116-gafe40385f116
Command failed with exit code: 1

When I first run make modules_install and then vmtest, the issue is resolved!

[root@localhost linux]$ vmtest -k arch/x86_64/boot/bzImage "cd ./tools/testing/selftests/bpf/; ./test_progs -t smc" => bzImage
===> Booting
===> Setting up VM
===> Running command

[    1.829268] bpf_testmod: loading out-of-tree module taints kernel.
[    1.906274] NET: Registered PF_SMC protocol family
[    1.906603] smc: adding smcd device loopback-ism

net.smc.ops = linkcheck
#27/1 bpf_smc/topo:OK
#27 bpf_smc:OK
Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

Quoted 5 lines

Of course, I don't think it's necessarily appropriate to have the CI modify this, so I decided to change CONFIG_SMC=m to CONFIG_SMC=y in the config, which also solves the problem.

What do you think?

[cc bpf list]

CONFIG_SMC=y makes sense to me.

Thanks,
(Other) Daniel