Skip to contentSkip to conversation

Libbpf API and memfd_create() fixes

PATCH Applied Andrii Nakryiko 3 participants 8 messages Feb 1, 2024

Subsystems (4)
lore
Share & tools
lore

Andrii Nakryiko

author maintainer
#0 lore raw

v2 cover Libbpf API and memfd_create() fixes

Few small fixes identified over last few days. Main fix is for memfd_create() which causes problems on Android platforms. See individual patches for details.

v1->v2:
- added extra Fixes tag in patch #2 (Yonghong);
- improved commit message in patch #5 (Yonghong).

Andrii Nakryiko (5):
libbpf: call memfd_create() syscall directly
libbpf: add missing LIBBPF_API annotation to libbpf_set_memlock_rlim
API
libbpf: add btf__new_split() API that was declared but not implemented
libbpf: add missed btf_ext__raw_data() API
selftests/bpf: fix bench runner SIGSEGV

 tools/lib/bpf/bpf.h                 |  2 +-
 tools/lib/bpf/btf.c                 | 11 ++++++++++-
 tools/lib/bpf/libbpf.c              | 11 ++++++++++-
 tools/lib/bpf/libbpf.map            |  5 +++--
 tools/lib/bpf/linker.c              |  2 +-
 tools/testing/selftests/bpf/bench.c | 12 ++++++++++--
 6 files changed, 35 insertions(+), 8 deletions(-)
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH v2 1/5 libbpf: call memfd_create() syscall directly

Some versions of Android do not implement memfd_create() wrapper in their libc implementation, leading to build failures ([0]). On the other hand, memfd_create() is available as a syscall on quite old kernels (3.17+, while bpf() syscall itself is available since 3.18+), so it is ok to assume that syscall availability and call into it with syscall() helper to avoid Android-specific workarounds.

Validated in libbpf-bootstrap's CI ([1]).

[0] https://github.com/libbpf/libbpf-bootstrap/actions/runs/7701003207/job/20986080319#step:5:83
[1] https://github.com/libbpf/libbpf-bootstrap/actions/runs/7715988887/job/21031767212?pr=253

---
 tools/lib/bpf/libbpf.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
tools/lib/bpf/libbpf.c +10 -1
index f6953d7faff1..6932f2c4ddfd 100644
@@ -1525,11 +1525,20 @@ static Elf64_Sym *find_elf_var_sym(const struct bpf_object *obj, const char *nam
 	return ERR_PTR(-ENOENT);
 }
 
+/* Some versions of Android don't provide memfd_create() in their libc
+ * implementation, so avoid complications and just go straight to Linux
+ * syscall.
+ */
+static int sys_memfd_create(const char *name, unsigned flags)
+{
+	return syscall(__NR_memfd_create, name, flags);
+}
+
 static int create_placeholder_fd(void)
 {
 	int fd;
 
-	fd = ensure_good_fd(memfd_create("libbpf-placeholder-fd", MFD_CLOEXEC));
+	fd = ensure_good_fd(sys_memfd_create("libbpf-placeholder-fd", MFD_CLOEXEC));
 	if (fd < 0)
 		return -errno;
 	return fd;
Signature · 1 line
2.34.1
author maintainer
#2 lore raw

PATCH v2 2/5 libbpf: add missing LIBBPF_API annotation to libbpf_set_memlock_rlim API

In reply to Andrii Nakryiko

LIBBPF_API annotation seems missing on libbpf_set_memlock_rlim API, so add it to make this API callable from libbpf's shared library version.

Fixes e542f2c4cd16 ("libbpf: Auto-bump RLIMIT_MEMLOCK if kernel needs it for BPF")
Fixes ab9a5a05dc48 ("libbpf: fix up few libbpf.map problems")
---
 tools/lib/bpf/bpf.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/lib/bpf/bpf.h +1 -1
index 1441f642c563..f866e98b2436 100644
@@ -35,7 +35,7 @@
 extern "C" {
 #endif
 
-int libbpf_set_memlock_rlim(size_t memlock_bytes);
+LIBBPF_API int libbpf_set_memlock_rlim(size_t memlock_bytes);
 
 struct bpf_map_create_opts {
 	size_t sz; /* size of this struct for forward/backward compatibility */
Signature · 1 line
2.34.1
author maintainer
#3 lore raw

PATCH v2 3/5 libbpf: add btf__new_split() API that was declared but not implemented

In reply to Andrii Nakryiko

Seems like original commit adding split BTF support intended to add btf__new_split() API, and even declared it in libbpf.map, but never added (trivial) implementation. Fix this.

Fixes ba451366bf44 ("libbpf: Implement basic split BTF support")
---
 tools/lib/bpf/btf.c      | 5 +++++
 tools/lib/bpf/libbpf.map | 3 ++-
 2 files changed, 7 insertions(+), 1 deletion(-)
tools/lib/bpf/btf.c +5 -0
index 95db88b36cf3..845034d15420 100644
@@ -1079,6 +1079,11 @@ struct btf *btf__new(const void *data, __u32 size)
 	return libbpf_ptr(btf_new(data, size, NULL));
 }
 
+struct btf *btf__new_split(const void *data, __u32 size, struct btf *base_btf)
+{
+	return libbpf_ptr(btf_new(data, size, base_btf));
+}
+
 static struct btf *btf_parse_elf(const char *path, struct btf *base_btf,
 				 struct btf_ext **btf_ext)
 {
tools/lib/bpf/libbpf.map +2 -1
index d9e1f57534fa..386964f572a8 100644
@@ -245,7 +245,6 @@ LIBBPF_0.3.0 {
 		btf__parse_raw_split;
 		btf__parse_split;
 		btf__new_empty_split;
-		btf__new_split;
 		ring_buffer__epoll_fd;
 } LIBBPF_0.2.0;
 
@@ -411,5 +410,7 @@ LIBBPF_1.3.0 {
 } LIBBPF_1.2.0;
 
 LIBBPF_1.4.0 {
+	global:
 		bpf_token_create;
+		btf__new_split;
 } LIBBPF_1.3.0;
Signature · 1 line
2.34.1
author maintainer
#4 lore raw

PATCH v2 4/5 libbpf: add missed btf_ext__raw_data() API

In reply to Andrii Nakryiko

Another API that was declared in libbpf.map but actual implementation was missing. btf_ext__get_raw_data() was intended as a discouraged alias to consistently-named btf_ext__raw_data(), so make this an actuality.

Fixes 20eccf29e297 ("libbpf: hide and discourage inconsistently named getters")
---
 tools/lib/bpf/btf.c      | 6 +++++-
 tools/lib/bpf/libbpf.map | 2 +-
 tools/lib/bpf/linker.c   | 2 +-
 3 files changed, 7 insertions(+), 3 deletions(-)
tools/lib/bpf/btf.c +5 -1
index 845034d15420..a17b4c9c4213 100644
@@ -3050,12 +3050,16 @@ struct btf_ext *btf_ext__new(const __u8 *data, __u32 size)
 	return btf_ext;
 }
 
-const void *btf_ext__get_raw_data(const struct btf_ext *btf_ext, __u32 *size)
+const void *btf_ext__raw_data(const struct btf_ext *btf_ext, __u32 *size)
 {
 	*size = btf_ext->data_size;
 	return btf_ext->data;
 }
 
+__attribute__((alias("btf_ext__raw_data")))
+const void *btf_ext__get_raw_data(const struct btf_ext *btf_ext, __u32 *size);
+
+
 struct btf_dedup;
 
 static struct btf_dedup *btf_dedup_new(struct btf *btf, const struct btf_dedup_opts *opts);
tools/lib/bpf/libbpf.map +1 -1
index 386964f572a8..86804fd90dd1 100644
@@ -325,7 +325,6 @@ LIBBPF_0.7.0 {
 		bpf_xdp_detach;
 		bpf_xdp_query;
 		bpf_xdp_query_id;
-		btf_ext__raw_data;
 		libbpf_probe_bpf_helper;
 		libbpf_probe_bpf_map_type;
 		libbpf_probe_bpf_prog_type;
@@ -413,4 +412,5 @@ LIBBPF_1.4.0 {
 	global:
 		bpf_token_create;
 		btf__new_split;
+		btf_ext__raw_data;
 } LIBBPF_1.3.0;
tools/lib/bpf/linker.c +1 -1
index 16bca56002ab..0d4be829551b 100644
@@ -2732,7 +2732,7 @@ static int finalize_btf(struct bpf_linker *linker)
 
 	/* Emit .BTF.ext section */
 	if (linker->btf_ext) {
-		raw_data = btf_ext__get_raw_data(linker->btf_ext, &raw_sz);
+		raw_data = btf_ext__raw_data(linker->btf_ext, &raw_sz);
 		if (!raw_data)
 			return -ENOMEM;
 
Signature · 1 line
2.34.1
author maintainer
#5 lore raw

PATCH v2 5/5 selftests/bpf: fix bench runner SIGSEGV

In reply to Andrii Nakryiko

Some benchmarks don't have either "consumer" or "producer" sides. For example, trig-tp and other BPF triggering benchmarks don't have consumers, as they only do "producing" by calling into syscall or predefined uproes. As such it's valid for some benchmarks to have zero consumers or producers. So allows to specify -c0 explicitly.

This triggers another problem. If benchmark doesn't support either consumer or producer side, consumer_thread/producer_thread callback will be NULL, but benchmark runner will attempt to use those NULL callback to create threads anyways. So instead of crashing with SIGSEGV in case of misconfigured benchmark, detect the condition and report error.

---
 tools/testing/selftests/bpf/bench.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/bench.c +10 -2
index 73ce11b0547d..1724d50ba942 100644
@@ -323,14 +323,14 @@ static error_t parse_arg(int key, char *arg, struct argp_state *state)
 		break;
 	case 'p':
 		env.producer_cnt = strtol(arg, NULL, 10);
-		if (env.producer_cnt <= 0) {
+		if (env.producer_cnt < 0) {
 			fprintf(stderr, "Invalid producer count: %s\n", arg);
 			argp_usage(state);
 		}
 		break;
 	case 'c':
 		env.consumer_cnt = strtol(arg, NULL, 10);
-		if (env.consumer_cnt <= 0) {
+		if (env.consumer_cnt < 0) {
 			fprintf(stderr, "Invalid consumer count: %s\n", arg);
 			argp_usage(state);
 		}
@@ -607,6 +607,10 @@ static void setup_benchmark(void)
 		bench->setup();
 
 	for (i = 0; i < env.consumer_cnt; i++) {
+		if (!bench->consumer_thread) {
+			fprintf(stderr, "benchmark doesn't support consumers!\n");
+			exit(1);
+		}
 		err = pthread_create(&state.consumers[i], NULL,
 				     bench->consumer_thread, (void *)(long)i);
 		if (err) {
@@ -626,6 +630,10 @@ static void setup_benchmark(void)
 		env.prod_cpus.next_cpu = env.cons_cpus.next_cpu;
 
 	for (i = 0; i < env.producer_cnt; i++) {
+		if (!bench->producer_thread) {
+			fprintf(stderr, "benchmark doesn't support producers!\n");
+			exit(1);
+		}
 		err = pthread_create(&state.producers[i], NULL,
 				     bench->producer_thread, (void *)(long)i);
 		if (err) {
Signature · 1 line
2.34.1
maintainer Acked-by on the v2 series
#6 lore raw
In reply to Andrii Nakryiko
Quoted 7 lines · Andrii Nakryiko

Few small fixes identified over last few days. Main fix is for memfd_create() which causes problems on Android platforms. See individual patches for details.

v1->v2:
- added extra Fixes tag in patch #2 (Yonghong);
- improved commit message in patch #5 (Yonghong).

Full patch-set looks good to me.

#7 lore raw
In reply to Andrii Nakryiko
This series was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:

Quoted 9 lines · On Thu, 1 Feb 2024 09:20:22 -0800 you wrote:

Few small fixes identified over last few days. Main fix is for memfd_create() which causes problems on Android platforms. See individual patches for details.

v1->v2:
- added extra Fixes tag in patch #2 (Yonghong);
- improved commit message in patch #5 (Yonghong).

[...]

Here is the summary with links:
- [v2,bpf-next,1/5] libbpf: call memfd_create() syscall directly

    https://git.kernel.org/bpf/bpf-next/c/9fa5e1a180aa
  - [v2,bpf-next,2/5] libbpf: add missing LIBBPF_API annotation to libbpf_set_memlock_rlim API
    https://git.kernel.org/bpf/bpf-next/c/93ee1eb85e28
  - [v2,bpf-next,3/5] libbpf: add btf__new_split() API that was declared but not implemented
    https://git.kernel.org/bpf/bpf-next/c/c81a8ab196b5
  - [v2,bpf-next,4/5] libbpf: add missed btf_ext__raw_data() API
    https://git.kernel.org/bpf/bpf-next/c/b9551da8cf3a
  - [v2,bpf-next,5/5] selftests/bpf: fix bench runner SIGSEGV
    https://git.kernel.org/bpf/bpf-next/c/943b043aeecc

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html