Skip to contentSkip to conversation

bpftool: Skip non-autoload programs when generating light skeletons

PATCH Superseded Thiébaud Weksteen 4 participants 9 messages Sep 9 – Sep 17

Subsystems (4)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/3 bpftool: Skip non-autoload programs when generating light skeletons

When generating a light skeleton (bpftool gen skeleton -L), bpf_object__load() skips loading programs marked as non-autoload (e.g. SEC("?...")), so the generated loader program only records and populates file descriptors for autoloaded programs.

Previously, bpftool emitted struct bpf_prog_desc fields, link fields, and attach/detach/destroy functions for all programs in the BPF object, causing the loader program to store subsequent program FDs into incorrect skeleton struct fields when non-autoload programs were present.

Furthermore, bpf_object__load() can update a program's autoload status during preparation (e.g. for struct_ops programs when resolving kernel BTF members or adjusting autoload based on map autocreate settings). Move bpf_object__gen_loader() and bpf_object__load() out of gen_trace() into do_skeleton() before counting programs and emitting struct fields so that struct field declarations and attach/detach/destroy functions all observe the final post-load autoload state.

Skip programs with !bpf_program__autoload(prog) when counting programs and generating progs/links struct fields as well as attach, detach, and destroy functions for light skeletons.

Fixes d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.")

---
Changes since v1:
- Move bpf_object__gen_loader() and bpf_object__load() out of gen_trace()

 .../bpf/bpftool/Documentation/bpftool-gen.rst |  4 +-
 tools/bpf/bpftool/gen.c                       | 64 ++++++++++++-------
 2 files changed, 44 insertions(+), 24 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-gen.rst +3 -1
index d0a36f442db7..1cdecf3e4fa5 100644
@@ -184,7 +184,9 @@ OPTIONS
 -L, --use-loader
     For skeletons, generate a "light" skeleton (also known as "loader"
     skeleton). A light skeleton contains a loader eBPF program. It does not use
-    the majority of the libbpf infrastructure, and does not need libelf.
+    the majority of the libbpf infrastructure, and does not need libelf. BPF
+    programs marked as non-autoload (e.g., via **SEC("?...")**) are skipped and
+    not included in the generated skeleton.
 
 -S, --sign
     For skeletons, generate a signed skeleton. This option must be used with
tools/bpf/bpftool/gen.c +41 -23

159 lines not on this page · raw message

Signature · 1 line
2.55.0.979.g7e5102b832-goog
#1 lore raw

PATCH v2 2/3 bpftool: Compute light skeleton ctx.sz without relying on links

When generating a light skeleton, gen_trace() unconditionally sets skel->ctx.sz using (char *)&skel->links - (char *)skel. However, if a BPF object has no programs and no struct_ops maps (prog_cnt + attach_map_cnt == 0), do_skeleton() omits the links struct, causing the generated skeleton header to fail compilation.

Compute skel->ctx.sz from the end of progs (if prog_cnt > 0), maps (if map_cnt > 0), or ctx instead of relying on skel->links. Also remove the unused opts.data_sz argument passed to codegen().

Fixes d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.")
---
 tools/bpf/bpftool/gen.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
tools/bpf/bpftool/gen.c +11 -4
index e9a1a018f270..4a93c392857a 100644
@@ -711,7 +711,7 @@ static void codegen_destroy(struct bpf_object *obj, const char *obj_name)
 }
 
 static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *header_guard,
-		     const struct gen_loader_opts *opts)
+		     const struct gen_loader_opts *opts, size_t prog_cnt, size_t map_cnt)
 {
 	struct bpf_load_and_run_opts sopts = {};
 	char sig_buf[MAX_SIG_SIZE];
@@ -746,9 +746,16 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 			skel = (struct %1$s *)skel_alloc(sizeof(*skel));    \n\
 			if (!skel)					    \n\
 				goto cleanup;				    \n\
-			skel->ctx.sz = (char *)&skel->links - (char *)skel; \n\
 		",
-		obj_name, opts->data_sz);
+		obj_name);
+	if (prog_cnt)
+		printf("\tskel->ctx.sz = (char *)&skel->progs - (char *)skel\n"
+		       "\t\t       + sizeof(skel->progs);\n");
+	else if (map_cnt)
+		printf("\tskel->ctx.sz = (char *)&skel->maps - (char *)skel\n"
+		       "\t\t       + sizeof(skel->maps);\n");
+	else
+		printf("\tskel->ctx.sz = sizeof(skel->ctx);\n");
 	bpf_object__for_each_map(map, obj) {
 		const void *mmap_data = NULL;
 		size_t mmap_size = 0;
@@ -1469,7 +1476,7 @@ static int do_skeleton(int argc, char **argv)
 			goto out;
 	}
 	if (use_loader) {
-		err = gen_trace(obj, obj_name, header_guard, &gen_opts);
+		err = gen_trace(obj, obj_name, header_guard, &gen_opts, prog_cnt, map_cnt);
 		goto out;
 	}
 
Signature · 1 line
2.55.0.979.g7e5102b832-goog
#3 lore raw

PATCH v2 3/3 selftests/bpf: Verify program FD after non-autoload programs in lskel

In reply to Thiébaud Weksteen

In test_global_percpu_data.c, dump_percpu_data is defined after two non-autoload SEC("?kprobe") programs (verifier_strncmp and verifier_snprintf). Verify in test_global_percpu_data_lskel() that lskel->progs.dump_percpu_data.prog_fd is properly populated (> 0) when loaded via the light skeleton.

---
No changes since v1.

 tools/testing/selftests/bpf/prog_tests/global_data_init.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/global_data_init.c +2 -0
index 7c539cbcf3a1..ebda60200b71 100644
@@ -214,6 +214,8 @@ static void test_global_percpu_data_lskel(void)
 	lskel = test_global_percpu_data_lskel__open_and_load();
 	if (!ASSERT_OK_PTR(lskel, "test_global_percpu_data_lskel__open_and_load"))
 		goto out;
+	if (!ASSERT_GT(lskel->progs.dump_percpu_data.prog_fd, 0, "dump_percpu_data.prog_fd"))
+		goto out;
 
 	map_fd = lskel->maps.percpu.map_fd;
 	prog_fd = lskel->progs.update_percpu_data.prog_fd;
Signature · 1 line
2.55.0.979.g7e5102b832-goog
#7 lore raw
In reply to Thiébaud Weksteen
This isn't a bug, but now that the bpf_object__load() call moved to do_skeleton(), would this comment be clearer next to the new call site, or reworded as a precondition on the opts parameter? The…
Quoted 4 lines
tools/bpf/bpftool/gen.c +0 -0
index a50540ef6521..e9a1a018f270 100644

[ ... ]

Quoted 22 lines
tools/bpf/bpftool/gen.c +2 -8
@@ -701,9 +710,9 @@ static void codegen_destroy(struct bpf_object *obj, const char *obj_name)
 		obj_name);
 }
 
-static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *header_guard)
+static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *header_guard,
+		     const struct gen_loader_opts *opts)
 {
-	DECLARE_LIBBPF_OPTS(gen_loader_opts, opts);
 	struct bpf_load_and_run_opts sopts = {};
 	char sig_buf[MAX_SIG_SIZE];
 	__u8 prog_sha[SHA256_DIGEST_LENGTH];
@@ -712,19 +721,6 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 	char ident[256];
 	int err = 0;
 
-	if (sign_progs)
-		opts.gen_hash = true;
-
-	err = bpf_object__gen_loader(obj, &opts);
-	if (err)
-		return err;

tools/bpf/bpftool/gen.c static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h · line 726

Quoted 10 lines
tools/bpf/bpftool/gen.c +0 -7
-
-	err = bpf_object__load(obj);
-	if (err) {
-		p_err("failed to load object file");
-		goto out;
-	}
-
 	/* If there was no error during load then gen_loader_opts
 	 * are populated with the loader program.
 	 */

This isn't a bug, but now that the bpf_object__load() call moved to do_skeleton(), would this comment be clearer next to the new call site, or reworded as a precondition on the opts parameter? The comment currently sits at the top of gen_trace() followed immediately by another unrelated comment, and a reader has to look at the caller to find the load it refers to.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34311359322

#4 lore raw

Re: [PATCH bpf-next v2 3/3] selftests/bpf: Verify program FD after non-autoload programs in lskel

In reply to Thiébaud Weksteen
Quoted 9 lines · Thiébaud Weksteen

In test_global_percpu_data.c, dump_percpu_data is defined after two non-autoload SEC("?kprobe") programs (verifier_strncmp and verifier_snprintf). Verify in test_global_percpu_data_lskel() that lskel->progs.dump_percpu_data.prog_fd is properly populated (> 0) when loaded via the light skeleton.

---
No changes since v1.

The Fixes tags of v1 are dropped.

Quoted 9 lines
 tools/testing/selftests/bpf/prog_tests/global_data_init.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/global_data_init.c +0 -0
index 7c539cbcf3a1..ebda60200b71 100644

tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_lskel(void) · line 218

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +2 -0
@@ -214,6 +214,8 @@ static void test_global_percpu_data_lskel(void)
 	lskel = test_global_percpu_data_lskel__open_and_load();
 	if (!ASSERT_OK_PTR(lskel, "test_global_percpu_data_lskel__open_and_load"))
 		goto out;
+	if (!ASSERT_GT(lskel->progs.dump_percpu_data.prog_fd, 0, "dump_percpu_data.prog_fd"))
+		goto out;

When the light skeleton does not skip the non-autoload progs, lskel->progs.dump_percpu_data.prog_fd is 0, actually.

But, theoretically, this ASSERT_GT() is not good enough to prove the FD is the correct FD of dump_percpu_data prog.

How about running the iter subtest against the lskel? Factor out test_global_percpu_data_iter_fd() helper first. Then, add an iter subtest using lskel.
When factoring out test_global_percpu_data_iter_fd() helper, pls change 'do { } while (0);' to '{ /* no-op */ }' btw.

Thanks,
Leon

#5 lore raw

Re: [PATCH bpf-next v2 3/3] selftests/bpf: Verify program FD after non-autoload programs in lskel

Quoted 32 lines · Leon Hwang
Quoted 9 lines · Thiébaud Weksteen

In test_global_percpu_data.c, dump_percpu_data is defined after two non-autoload SEC("?kprobe") programs (verifier_strncmp and verifier_snprintf). Verify in test_global_percpu_data_lskel() that lskel->progs.dump_percpu_data.prog_fd is properly populated (> 0) when loaded via the light skeleton.

---
No changes since v1.

The Fixes tags of v1 are dropped.

Quoted 9 lines
 tools/testing/selftests/bpf/prog_tests/global_data_init.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/global_data_init.c +0 -0
index 7c539cbcf3a1..ebda60200b71 100644

tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_lskel(void) · line 218

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +2 -0
@@ -214,6 +214,8 @@ static void test_global_percpu_data_lskel(void)
      lskel = test_global_percpu_data_lskel__open_and_load();
      if (!ASSERT_OK_PTR(lskel, "test_global_percpu_data_lskel__open_and_load"))
              goto out;
+     if (!ASSERT_GT(lskel->progs.dump_percpu_data.prog_fd, 0, "dump_percpu_data.prog_fd"))
+             goto out;

When the light skeleton does not skip the non-autoload progs, lskel->progs.dump_percpu_data.prog_fd is 0, actually.

Quoted 9 lines

But, theoretically, this ASSERT_GT() is not good enough to prove the FD is the correct FD of dump_percpu_data prog.

How about running the iter subtest against the lskel? Factor out test_global_percpu_data_iter_fd() helper first. Then, add an iter subtest using lskel.
When factoring out test_global_percpu_data_iter_fd() helper, pls change 'do { } while (0);' to '{ /* no-op */ }' btw.

Thanks for the review Leon. That sounds good, I can implement these changes. I am just waiting for Quentin's feedback on the first patch in this series before spinning up a new patch set.

maintainer
#8 lore raw
In reply to Thiébaud Weksteen
Quoted 52 lines · Thiébaud Weksteen

When generating a light skeleton (bpftool gen skeleton -L), bpf_object__load() skips loading programs marked as non-autoload (e.g. SEC("?...")), so the generated loader program only records and populates file descriptors for autoloaded programs.

Previously, bpftool emitted struct bpf_prog_desc fields, link fields, and attach/detach/destroy functions for all programs in the BPF object, causing the loader program to store subsequent program FDs into incorrect skeleton struct fields when non-autoload programs were present.

Furthermore, bpf_object__load() can update a program's autoload status during preparation (e.g. for struct_ops programs when resolving kernel BTF members or adjusting autoload based on map autocreate settings). Move bpf_object__gen_loader() and bpf_object__load() out of gen_trace() into do_skeleton() before counting programs and emitting struct fields so that struct field declarations and attach/detach/destroy functions all observe the final post-load autoload state.

Skip programs with !bpf_program__autoload(prog) when counting programs and generating progs/links struct fields as well as attach, detach, and destroy functions for light skeletons.

Fixes d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.")

---
Changes since v1:
- Move bpf_object__gen_loader() and bpf_object__load() out of gen_trace()

 .../bpf/bpftool/Documentation/bpftool-gen.rst |  4 +-
 tools/bpf/bpftool/gen.c                       | 64 ++++++++++++-------
 2 files changed, 44 insertions(+), 24 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-gen.rst +3 -1
index d0a36f442db7..1cdecf3e4fa5 100644
@@ -184,7 +184,9 @@ OPTIONS
 -L, --use-loader
     For skeletons, generate a "light" skeleton (also known as "loader"
     skeleton). A light skeleton contains a loader eBPF program. It does not use
-    the majority of the libbpf infrastructure, and does not need libelf.
+    the majority of the libbpf infrastructure, and does not need libelf. BPF
+    programs marked as non-autoload (e.g., via **SEC("?...")**) are skipped and
+    not included in the generated skeleton.
 
 -S, --sign
     For skeletons, generate a signed skeleton. This option must be used with
tools/bpf/bpftool/gen.c +0 -0
index a50540ef6521..e9a1a018f270 100644

[...]

Quoted 10 lines
tools/bpf/bpftool/gen.c +0 -6
@@ -712,19 +721,6 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 	char ident[256];
 	int err = 0;
 
-	if (sign_progs)
-		opts.gen_hash = true;
-
-	err = bpf_object__gen_loader(obj, &opts);
-	if (err)
-		return err;

tools/bpf/bpftool/gen.c static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h · line 726

Quoted 10 lines
tools/bpf/bpftool/gen.c +0 -7
-
-	err = bpf_object__load(obj);
-	if (err) {
-		p_err("failed to load object file");
-		goto out;
-	}
-
 	/* If there was no error during load then gen_loader_opts
 	 * are populated with the loader program.
 	 */

This comment could be updated (or moved to do_skeleton()).

Quoted 54 lines
tools/bpf/bpftool/gen.c +12 -7
@@ -752,7 +748,7 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 				goto cleanup;				    \n\
 			skel->ctx.sz = (char *)&skel->links - (char *)skel; \n\
 		",
-		obj_name, opts.data_sz);
+		obj_name, opts->data_sz);
 	bpf_object__for_each_map(map, obj) {
 		const void *mmap_data = NULL;
 		size_t mmap_size = 0;
@@ -795,22 +791,22 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 			static const char opts_data[] __attribute__((__aligned__(8))) = \"\\\n\
 		",
 		obj_name);
-	print_hex(opts.data, opts.data_sz);
+	print_hex(opts->data, opts->data_sz);
 	codegen("\
 		\n\
 		\";							    \n\
 			static const char opts_insn[] __attribute__((__aligned__(8))) = \"\\\n\
 		");
-	print_hex(opts.insns, opts.insns_sz);
+	print_hex(opts->insns, opts->insns_sz);
 	codegen("\
 		\n\
 		\";\n");
 
 	if (sign_progs) {
-		sopts.insns = opts.insns;
-		sopts.insns_sz = opts.insns_sz;
-		sopts.data = opts.data;
-		sopts.data_sz = opts.data_sz;
+		sopts.insns = opts->insns;
+		sopts.insns_sz = opts->insns_sz;
+		sopts.data = opts->data;
+		sopts.data_sz = opts->data_sz;
 		sopts.excl_prog_hash = prog_sha;
 		sopts.excl_prog_hash_sz = sizeof(prog_sha);
 		sopts.signature = sig_buf;
@@ -1250,6 +1246,7 @@ static int do_skeleton(int argc, char **argv)
 	char header_guard[MAX_OBJ_NAME_LEN + sizeof("__SKEL_H__")];
 	size_t map_cnt = 0, prog_cnt = 0, attach_map_cnt = 0, file_sz, mmap_sz;
 	DECLARE_LIBBPF_OPTS(bpf_object_open_opts, opts);
+	DECLARE_LIBBPF_OPTS(gen_loader_opts, gen_opts);
 	char obj_name[MAX_OBJ_NAME_LEN] = "", *obj_data;
 	struct bpf_object *obj = NULL;
 	const char *file;
@@ -1326,6 +1323,21 @@ static int do_skeleton(int argc, char **argv)
 		goto out_obj;
 	}
 
+	if (use_loader) {
+		if (sign_progs)
+			gen_opts.gen_hash = true;
+

tools/bpf/bpftool/gen.c static int do_skeleton(int argc, char **argv) · line 1339

Quoted 10 lines
tools/bpf/bpftool/gen.c +10 -0
+		err = bpf_object__gen_loader(obj, &gen_opts);
+		if (err)
+			goto out;
+
+		err = bpf_object__load(obj);
+		if (err) {
+			p_err("failed to load object file");
+			goto out;
+		}
+	}

Could you please add a comment above this block to explain why it's at the current location (to avoid autoload flags update), to avoid people moving it by accident, please?

Looks good otherwise, thank you!

maintainer Acked-by on v2 2/3
#2 lore raw

Re: [PATCH bpf-next v2 2/3] bpftool: Compute light skeleton ctx.sz without relying on links

In reply to Thiébaud Weksteen
Quoted 6 lines · Thiébaud Weksteen

When generating a light skeleton, gen_trace() unconditionally sets skel->ctx.sz using (char *)&skel->links - (char *)skel. However, if a BPF object has no programs and no struct_ops maps (prog_cnt + attach_map_cnt == 0), do_skeleton() omits the links struct, causing the generated skeleton header to fail compilation.

Quoted 6 lines

Compute skel->ctx.sz from the end of progs (if prog_cnt > 0), maps (if map_cnt > 0), or ctx instead of relying on skel->links. Also remove the unused opts.data_sz argument passed to codegen().

Fixes d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.")

Thank you

maintainer
#6 lore raw

Re: [PATCH bpf-next v2 3/3] selftests/bpf: Verify program FD after non-autoload programs in lskel

In reply to Thiébaud Weksteen

2026-09-17 11:06 UTC+1000 ~ Thiébaud Weksteen <[email protected]>

Quoted 43 lines · Leon Hwang
Quoted 32 lines · Thiébaud Weksteen
Quoted 9 lines

In test_global_percpu_data.c, dump_percpu_data is defined after two non-autoload SEC("?kprobe") programs (verifier_strncmp and verifier_snprintf). Verify in test_global_percpu_data_lskel() that lskel->progs.dump_percpu_data.prog_fd is properly populated (> 0) when loaded via the light skeleton.

---
No changes since v1.

The Fixes tags of v1 are dropped.

Quoted 9 lines
 tools/testing/selftests/bpf/prog_tests/global_data_init.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/global_data_init.c +0 -0
index 7c539cbcf3a1..ebda60200b71 100644

tools/testing/selftests/bpf/prog_tests/global_data_init.c static void test_global_percpu_data_lskel(void) · line 218

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/global_data_init.c +2 -0
@@ -214,6 +214,8 @@ static void test_global_percpu_data_lskel(void)
      lskel = test_global_percpu_data_lskel__open_and_load();
      if (!ASSERT_OK_PTR(lskel, "test_global_percpu_data_lskel__open_and_load"))
              goto out;
+     if (!ASSERT_GT(lskel->progs.dump_percpu_data.prog_fd, 0, "dump_percpu_data.prog_fd"))
+             goto out;

When the light skeleton does not skip the non-autoload progs, lskel->progs.dump_percpu_data.prog_fd is 0, actually.

Quoted 9 lines

But, theoretically, this ASSERT_GT() is not good enough to prove the FD is the correct FD of dump_percpu_data prog.

How about running the iter subtest against the lskel? Factor out test_global_percpu_data_iter_fd() helper first. Then, add an iter subtest using lskel.
When factoring out test_global_percpu_data_iter_fd() helper, pls change 'do { } while (0);' to '{ /* no-op */ }' btw.

Quoted 4 lines

Thanks for the review Leon. That sounds good, I can implement these changes. I am just waiting for Quentin's feedback on the first patch in this series before spinning up a new patch set.

Apologies for the delay. Patch 1 looks good to me, I just posted some minor suggestions regarding comments. Thank you!

Quentin