Skip to contentSkip to conversation

bpftool: Skip non-autoload programs when generating light skeletons

PATCH Superseded Thiébaud Weksteen 4 participants 9 messages Sep 9 – Sep 17

Subsystems (4)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/3 bpftool: Skip non-autoload programs when generating light skeletons

When generating a light skeleton (bpftool gen skeleton -L), bpf_object__load() skips loading programs marked as non-autoload (e.g. SEC("?...")), so the generated loader program only records and populates file descriptors for autoloaded programs.

Previously, bpftool emitted struct bpf_prog_desc fields, link fields, and attach/detach/destroy functions for all programs in the BPF object, causing the loader program to store subsequent program FDs into incorrect skeleton struct fields when non-autoload programs were present.

Furthermore, bpf_object__load() can update a program's autoload status during preparation (e.g. for struct_ops programs when resolving kernel BTF members or adjusting autoload based on map autocreate settings). Move bpf_object__gen_loader() and bpf_object__load() out of gen_trace() into do_skeleton() before counting programs and emitting struct fields so that struct field declarations and attach/detach/destroy functions all observe the final post-load autoload state.

Skip programs with !bpf_program__autoload(prog) when counting programs and generating progs/links struct fields as well as attach, detach, and destroy functions for light skeletons.

Fixes d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.")

---
Changes since v1:
- Move bpf_object__gen_loader() and bpf_object__load() out of gen_trace()

 .../bpf/bpftool/Documentation/bpftool-gen.rst |  4 +-
 tools/bpf/bpftool/gen.c                       | 64 ++++++++++++-------
 2 files changed, 44 insertions(+), 24 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-gen.rst +3 -1
index d0a36f442db7..1cdecf3e4fa5 100644
@@ -184,7 +184,9 @@ OPTIONS
 -L, --use-loader
     For skeletons, generate a "light" skeleton (also known as "loader"
     skeleton). A light skeleton contains a loader eBPF program. It does not use
-    the majority of the libbpf infrastructure, and does not need libelf.
+    the majority of the libbpf infrastructure, and does not need libelf. BPF
+    programs marked as non-autoload (e.g., via **SEC("?...")**) are skipped and
+    not included in the generated skeleton.
 
 -S, --sign
     For skeletons, generate a signed skeleton. This option must be used with
tools/bpf/bpftool/gen.c +41 -23

159 lines not on this page · raw message

Signature · 1 line
2.55.0.979.g7e5102b832-goog
#1 lore raw

PATCH v2 2/3 bpftool: Compute light skeleton ctx.sz without relying on links

When generating a light skeleton, gen_trace() unconditionally sets skel->ctx.sz using (char *)&skel->links - (char *)skel. However, if a BPF object has no programs and no struct_ops maps (prog_cnt + attach_map_cnt == 0), do_skeleton() omits the links struct, causing the generated skeleton header to fail compilation.

Compute skel->ctx.sz from the end of progs (if prog_cnt > 0), maps (if map_cnt > 0), or ctx instead of relying on skel->links. Also remove the unused opts.data_sz argument passed to codegen().

Fixes d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.")
---
 tools/bpf/bpftool/gen.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
tools/bpf/bpftool/gen.c +11 -4
index e9a1a018f270..4a93c392857a 100644
@@ -711,7 +711,7 @@ static void codegen_destroy(struct bpf_object *obj, const char *obj_name)
 }
 
 static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *header_guard,
-		     const struct gen_loader_opts *opts)
+		     const struct gen_loader_opts *opts, size_t prog_cnt, size_t map_cnt)
 {
 	struct bpf_load_and_run_opts sopts = {};
 	char sig_buf[MAX_SIG_SIZE];
@@ -746,9 +746,16 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 			skel = (struct %1$s *)skel_alloc(sizeof(*skel));    \n\
 			if (!skel)					    \n\
 				goto cleanup;				    \n\
-			skel->ctx.sz = (char *)&skel->links - (char *)skel; \n\
 		",
-		obj_name, opts->data_sz);
+		obj_name);
+	if (prog_cnt)
+		printf("\tskel->ctx.sz = (char *)&skel->progs - (char *)skel\n"
+		       "\t\t       + sizeof(skel->progs);\n");
+	else if (map_cnt)
+		printf("\tskel->ctx.sz = (char *)&skel->maps - (char *)skel\n"
+		       "\t\t       + sizeof(skel->maps);\n");
+	else
+		printf("\tskel->ctx.sz = sizeof(skel->ctx);\n");
 	bpf_object__for_each_map(map, obj) {
 		const void *mmap_data = NULL;
 		size_t mmap_size = 0;
@@ -1469,7 +1476,7 @@ static int do_skeleton(int argc, char **argv)
 			goto out;
 	}
 	if (use_loader) {
-		err = gen_trace(obj, obj_name, header_guard, &gen_opts);
+		err = gen_trace(obj, obj_name, header_guard, &gen_opts, prog_cnt, map_cnt);
 		goto out;
 	}
 
Signature · 1 line
2.55.0.979.g7e5102b832-goog
maintainer Acked-by on v2 2/3
#2 lore raw

Re: [PATCH bpf-next v2 2/3] bpftool: Compute light skeleton ctx.sz without relying on links

Quoted 6 lines · Thiébaud Weksteen

When generating a light skeleton, gen_trace() unconditionally sets skel->ctx.sz using (char *)&skel->links - (char *)skel. However, if a BPF object has no programs and no struct_ops maps (prog_cnt + attach_map_cnt == 0), do_skeleton() omits the links struct, causing the generated skeleton header to fail compilation.

Quoted 6 lines

Compute skel->ctx.sz from the end of progs (if prog_cnt > 0), maps (if map_cnt > 0), or ctx instead of relying on skel->links. Also remove the unused opts.data_sz argument passed to codegen().

Fixes d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.")

Thank you

#3 lore raw

PATCH v2 3/3 selftests/bpf: Verify program FD after non-autoload programs in lskel

In reply to Thiébaud Weksteen

In test_global_percpu_data.c, dump_percpu_data is defined after two non-autoload SEC("?kprobe") programs (verifier_strncmp and verifier_snprintf). Verify in test_global_percpu_data_lskel() that lskel->progs.dump_percpu_data.prog_fd is properly populated (> 0) when loaded via the light skeleton.

---
No changes since v1.

 tools/testing/selftests/bpf/prog_tests/global_data_init.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/global_data_init.c +2 -0
index 7c539cbcf3a1..ebda60200b71 100644
@@ -214,6 +214,8 @@ static void test_global_percpu_data_lskel(void)
 	lskel = test_global_percpu_data_lskel__open_and_load();
 	if (!ASSERT_OK_PTR(lskel, "test_global_percpu_data_lskel__open_and_load"))
 		goto out;
+	if (!ASSERT_GT(lskel->progs.dump_percpu_data.prog_fd, 0, "dump_percpu_data.prog_fd"))
+		goto out;
 
 	map_fd = lskel->maps.percpu.map_fd;
 	prog_fd = lskel->progs.update_percpu_data.prog_fd;
Signature · 1 line
2.55.0.979.g7e5102b832-goog
#4 lore raw

Re: [PATCH bpf-next v2 3/3] selftests/bpf: Verify program FD after non-autoload programs in lskel

Quoted 9 lines · Thiébaud Weksteen

In test_global_percpu_data.c, dump_percpu_data is defined after two non-autoload SEC("?kprobe") programs (verifier_strncmp and verifier_snprintf). Verify in test_global_percpu_data_lskel() that lskel->progs.dump_percpu_data.prog_fd is properly populated (> 0) when loaded via the light skeleton.

---
No changes since v1.

The Fixes tags of v1 are dropped.

Quoted 9 lines
 tools/testing/selftests/bpf/prog_tests/global_data_init.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/global_data_init.c +0 -0
index 7c539cbcf3a1..ebda60200b71 100644

tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_lskel(void) · line 218

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +2 -0
@@ -214,6 +214,8 @@ static void test_global_percpu_data_lskel(void)
 	lskel = test_global_percpu_data_lskel__open_and_load();
 	if (!ASSERT_OK_PTR(lskel, "test_global_percpu_data_lskel__open_and_load"))
 		goto out;
+	if (!ASSERT_GT(lskel->progs.dump_percpu_data.prog_fd, 0, "dump_percpu_data.prog_fd"))
+		goto out;

When the light skeleton does not skip the non-autoload progs, lskel->progs.dump_percpu_data.prog_fd is 0, actually.

But, theoretically, this ASSERT_GT() is not good enough to prove the FD is the correct FD of dump_percpu_data prog.

How about running the iter subtest against the lskel? Factor out test_global_percpu_data_iter_fd() helper first. Then, add an iter subtest using lskel.
When factoring out test_global_percpu_data_iter_fd() helper, pls change 'do { } while (0);' to '{ /* no-op */ }' btw.

Thanks,
Leon

#5 lore raw

Re: [PATCH bpf-next v2 3/3] selftests/bpf: Verify program FD after non-autoload programs in lskel

Quoted 32 lines · Leon Hwang
Quoted 9 lines · Thiébaud Weksteen

In test_global_percpu_data.c, dump_percpu_data is defined after two non-autoload SEC("?kprobe") programs (verifier_strncmp and verifier_snprintf). Verify in test_global_percpu_data_lskel() that lskel->progs.dump_percpu_data.prog_fd is properly populated (> 0) when loaded via the light skeleton.

---
No changes since v1.

The Fixes tags of v1 are dropped.

Quoted 9 lines
 tools/testing/selftests/bpf/prog_tests/global_data_init.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/global_data_init.c +0 -0
index 7c539cbcf3a1..ebda60200b71 100644

tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_lskel(void) · line 218

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +2 -0
@@ -214,6 +214,8 @@ static void test_global_percpu_data_lskel(void)
      lskel = test_global_percpu_data_lskel__open_and_load();
      if (!ASSERT_OK_PTR(lskel, "test_global_percpu_data_lskel__open_and_load"))
              goto out;
+     if (!ASSERT_GT(lskel->progs.dump_percpu_data.prog_fd, 0, "dump_percpu_data.prog_fd"))
+             goto out;

When the light skeleton does not skip the non-autoload progs, lskel->progs.dump_percpu_data.prog_fd is 0, actually.

Quoted 9 lines

But, theoretically, this ASSERT_GT() is not good enough to prove the FD is the correct FD of dump_percpu_data prog.

How about running the iter subtest against the lskel? Factor out test_global_percpu_data_iter_fd() helper first. Then, add an iter subtest using lskel.
When factoring out test_global_percpu_data_iter_fd() helper, pls change 'do { } while (0);' to '{ /* no-op */ }' btw.

Thanks for the review Leon. That sounds good, I can implement these changes. I am just waiting for Quentin's feedback on the first patch in this series before spinning up a new patch set.

maintainer
#6 lore raw

Re: [PATCH bpf-next v2 3/3] selftests/bpf: Verify program FD after non-autoload programs in lskel

2026-09-17 11:06 UTC+1000 ~ Thiébaud Weksteen <[email protected]>

Quoted 43 lines · Leon Hwang
Quoted 32 lines · Thiébaud Weksteen
Quoted 9 lines

In test_global_percpu_data.c, dump_percpu_data is defined after two non-autoload SEC("?kprobe") programs (verifier_strncmp and verifier_snprintf). Verify in test_global_percpu_data_lskel() that lskel->progs.dump_percpu_data.prog_fd is properly populated (> 0) when loaded via the light skeleton.

---
No changes since v1.

The Fixes tags of v1 are dropped.

Quoted 9 lines
 tools/testing/selftests/bpf/prog_tests/global_data_init.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/global_data_init.c +0 -0
index 7c539cbcf3a1..ebda60200b71 100644

tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_lskel(void) · line 218

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +2 -0
@@ -214,6 +214,8 @@ static void test_global_percpu_data_lskel(void)
      lskel = test_global_percpu_data_lskel__open_and_load();
      if (!ASSERT_OK_PTR(lskel, "test_global_percpu_data_lskel__open_and_load"))
              goto out;
+     if (!ASSERT_GT(lskel->progs.dump_percpu_data.prog_fd, 0, "dump_percpu_data.prog_fd"))
+             goto out;

When the light skeleton does not skip the non-autoload progs, lskel->progs.dump_percpu_data.prog_fd is 0, actually.

Quoted 9 lines

But, theoretically, this ASSERT_GT() is not good enough to prove the FD is the correct FD of dump_percpu_data prog.

How about running the iter subtest against the lskel? Factor out test_global_percpu_data_iter_fd() helper first. Then, add an iter subtest using lskel.
When factoring out test_global_percpu_data_iter_fd() helper, pls change 'do { } while (0);' to '{ /* no-op */ }' btw.

Quoted 4 lines

Thanks for the review Leon. That sounds good, I can implement these changes. I am just waiting for Quentin's feedback on the first patch in this series before spinning up a new patch set.

Apologies for the delay. Patch 1 looks good to me, I just posted some minor suggestions regarding comments. Thank you!

Quentin

#7 lore raw
In reply to Thiébaud Weksteen
This isn't a bug, but now that the bpf_object__load() call moved to do_skeleton(), would this comment be clearer next to the new call site, or reworded as a precondition on the opts parameter? The…
Quoted 4 lines
tools/bpf/bpftool/gen.c +0 -0
index a50540ef6521..e9a1a018f270 100644

[ ... ]

Quoted 22 lines
tools/bpf/bpftool/gen.c +2 -8
@@ -701,9 +710,9 @@ static void codegen_destroy(struct bpf_object *obj, const char *obj_name)
 		obj_name);
 }
 
-static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *header_guard)
+static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *header_guard,
+		     const struct gen_loader_opts *opts)
 {
-	DECLARE_LIBBPF_OPTS(gen_loader_opts, opts);
 	struct bpf_load_and_run_opts sopts = {};
 	char sig_buf[MAX_SIG_SIZE];
 	__u8 prog_sha[SHA256_DIGEST_LENGTH];
@@ -712,19 +721,6 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 	char ident[256];
 	int err = 0;
 
-	if (sign_progs)
-		opts.gen_hash = true;
-
-	err = bpf_object__gen_loader(obj, &opts);
-	if (err)
-		return err;

tools/bpf/bpftool/gen.c static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h · line 726

Quoted 10 lines
tools/bpf/bpftool/gen.c +0 -7
-
-	err = bpf_object__load(obj);
-	if (err) {
-		p_err("failed to load object file");
-		goto out;
-	}
-
 	/* If there was no error during load then gen_loader_opts
 	 * are populated with the loader program.
 	 */

This isn't a bug, but now that the bpf_object__load() call moved to do_skeleton(), would this comment be clearer next to the new call site, or reworded as a precondition on the opts parameter? The comment currently sits at the top of gen_trace() followed immediately by another unrelated comment, and a reader has to look at the caller to find the load it refers to.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34311359322

maintainer
#8 lore raw
In reply to Thiébaud Weksteen
Quoted 52 lines · Thiébaud Weksteen

When generating a light skeleton (bpftool gen skeleton -L), bpf_object__load() skips loading programs marked as non-autoload (e.g. SEC("?...")), so the generated loader program only records and populates file descriptors for autoloaded programs.

Previously, bpftool emitted struct bpf_prog_desc fields, link fields, and attach/detach/destroy functions for all programs in the BPF object, causing the loader program to store subsequent program FDs into incorrect skeleton struct fields when non-autoload programs were present.

Furthermore, bpf_object__load() can update a program's autoload status during preparation (e.g. for struct_ops programs when resolving kernel BTF members or adjusting autoload based on map autocreate settings). Move bpf_object__gen_loader() and bpf_object__load() out of gen_trace() into do_skeleton() before counting programs and emitting struct fields so that struct field declarations and attach/detach/destroy functions all observe the final post-load autoload state.

Skip programs with !bpf_program__autoload(prog) when counting programs and generating progs/links struct fields as well as attach, detach, and destroy functions for light skeletons.

Fixes d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.")

---
Changes since v1:
- Move bpf_object__gen_loader() and bpf_object__load() out of gen_trace()

 .../bpf/bpftool/Documentation/bpftool-gen.rst |  4 +-
 tools/bpf/bpftool/gen.c                       | 64 ++++++++++++-------
 2 files changed, 44 insertions(+), 24 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-gen.rst +3 -1
index d0a36f442db7..1cdecf3e4fa5 100644
@@ -184,7 +184,9 @@ OPTIONS
 -L, --use-loader
     For skeletons, generate a "light" skeleton (also known as "loader"
     skeleton). A light skeleton contains a loader eBPF program. It does not use
-    the majority of the libbpf infrastructure, and does not need libelf.
+    the majority of the libbpf infrastructure, and does not need libelf. BPF
+    programs marked as non-autoload (e.g., via **SEC("?...")**) are skipped and
+    not included in the generated skeleton.
 
 -S, --sign
     For skeletons, generate a signed skeleton. This option must be used with
tools/bpf/bpftool/gen.c +0 -0
index a50540ef6521..e9a1a018f270 100644

[...]

Quoted 10 lines
tools/bpf/bpftool/gen.c +0 -6
@@ -712,19 +721,6 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 	char ident[256];
 	int err = 0;
 
-	if (sign_progs)
-		opts.gen_hash = true;
-
-	err = bpf_object__gen_loader(obj, &opts);
-	if (err)
-		return err;

tools/bpf/bpftool/gen.c static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h · line 726

Quoted 10 lines
tools/bpf/bpftool/gen.c +0 -7
-
-	err = bpf_object__load(obj);
-	if (err) {
-		p_err("failed to load object file");
-		goto out;
-	}
-
 	/* If there was no error during load then gen_loader_opts
 	 * are populated with the loader program.
 	 */

This comment could be updated (or moved to do_skeleton()).

Quoted 54 lines
tools/bpf/bpftool/gen.c +12 -7
@@ -752,7 +748,7 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 				goto cleanup;				    \n\
 			skel->ctx.sz = (char *)&skel->links - (char *)skel; \n\
 		",
-		obj_name, opts.data_sz);
+		obj_name, opts->data_sz);
 	bpf_object__for_each_map(map, obj) {
 		const void *mmap_data = NULL;
 		size_t mmap_size = 0;
@@ -795,22 +791,22 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 			static const char opts_data[] __attribute__((__aligned__(8))) = \"\\\n\
 		",
 		obj_name);
-	print_hex(opts.data, opts.data_sz);
+	print_hex(opts->data, opts->data_sz);
 	codegen("\
 		\n\
 		\";							    \n\
 			static const char opts_insn[] __attribute__((__aligned__(8))) = \"\\\n\
 		");
-	print_hex(opts.insns, opts.insns_sz);
+	print_hex(opts->insns, opts->insns_sz);
 	codegen("\
 		\n\
 		\";\n");
 
 	if (sign_progs) {
-		sopts.insns = opts.insns;
-		sopts.insns_sz = opts.insns_sz;
-		sopts.data = opts.data;
-		sopts.data_sz = opts.data_sz;
+		sopts.insns = opts->insns;
+		sopts.insns_sz = opts->insns_sz;
+		sopts.data = opts->data;
+		sopts.data_sz = opts->data_sz;
 		sopts.excl_prog_hash = prog_sha;
 		sopts.excl_prog_hash_sz = sizeof(prog_sha);
 		sopts.signature = sig_buf;
@@ -1250,6 +1246,7 @@ static int do_skeleton(int argc, char **argv)
 	char header_guard[MAX_OBJ_NAME_LEN + sizeof("__SKEL_H__")];
 	size_t map_cnt = 0, prog_cnt = 0, attach_map_cnt = 0, file_sz, mmap_sz;
 	DECLARE_LIBBPF_OPTS(bpf_object_open_opts, opts);
+	DECLARE_LIBBPF_OPTS(gen_loader_opts, gen_opts);
 	char obj_name[MAX_OBJ_NAME_LEN] = "", *obj_data;
 	struct bpf_object *obj = NULL;
 	const char *file;
@@ -1326,6 +1323,21 @@ static int do_skeleton(int argc, char **argv)
 		goto out_obj;
 	}
 
+	if (use_loader) {
+		if (sign_progs)
+			gen_opts.gen_hash = true;
+

tools/bpf/bpftool/gen.c static int do_skeleton(int argc, char **argv) · line 1339

Quoted 10 lines
tools/bpf/bpftool/gen.c +10 -0
+		err = bpf_object__gen_loader(obj, &gen_opts);
+		if (err)
+			goto out;
+
+		err = bpf_object__load(obj);
+		if (err) {
+			p_err("failed to load object file");
+			goto out;
+		}
+	}

Could you please add a comment above this block to explain why it's at the current location (to avoid autoload flags update), to avoid people moving it by accident, please?

Looks good otherwise, thank you!