Skip to contentSkip to conversation

libbpf/bpftool: support merging split BTFs

PATCH Superseded Josef Bacik 5 participants 12 messages Feb 20 – Feb 23

Subsystems (5)
Kselftest BPF BPF selftests libbpf +1 more subsystems

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v4 cover libbpf/bpftool: support merging split BTFs

v1: https://lore.kernel.org/bpf/[email protected]/
v2: https://lore.kernel.org/bpf/[email protected]/
v3: https://lore.kernel.org/bpf/[email protected]/

v1->v2:
- Added a btf__dedup() call to btf__add_btf() to ensure that we don't have duplicate types in the merged BTF.
v2->v3:
- AI review got confused about the UAF comment, so the comment was expanded to clarify the UAF potential.
- Fixed potential clobbering of errno in the error path.
v3->v4:
- Fixed a potential silent corruption pointed out by the AI review bot.

--- Original email ---

Hello,

I'm extending systing to do introspection on vfio devices, which requires having the structs I need from the kernel available in userspace. Normally these are loadable modules, but in the case of vfio there's multiple structs across multiple modules. Normally you'd do the following to generate your vmlinux.h with a module

bpftool btf dump file /sys/kernel/btf/<module> format c \
--base /sys/kernel/btf/vmlinux > vmlinux.h

but if you need multiple modules you have to hack together multiple dumps and merge them together. This patch series adds support for merging multiple BTF sources together, so you can do

bpftool btf dump file /sys/kernel/btf/<module1> \
	file /sys/kernel/btf/<module2> format c \
	--base /sys/kernel/btf/vmlinux > vmlinux.h

I tested this with my usecase and it works. Thanks,

Josef

Josef Bacik (3):
libbpf: support appending split BTF in btf__add_btf()
bpftool: support merging multiple module BTFs in btf dump selftests/bpf: add test for btf__add_btf() with split BTF sources

 tools/bpf/bpftool/btf.c                       | 135 ++++++++++++++++--
 tools/lib/bpf/btf.c                           |  44 ++++--
 .../selftests/bpf/prog_tests/btf_write.c      |  89 ++++++++++++
 3 files changed, 247 insertions(+), 21 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH v4 1/3 libbpf: support appending split BTF in btf__add_btf()

btf__add_btf() currently rejects split BTF sources with -ENOTSUP. This prevents merging types from multiple kernel module BTFs that are all split against the same vmlinux base.

Extend btf__add_btf() to handle split BTF sources by:

- Replacing the blanket -ENOTSUP with a validation that src and dst share the same base BTF pointer when both are split.

- Computing src_start_id from the source's base to distinguish base type ID references (which must remain unchanged) from split type IDs (which must be remapped to new positions in the destination).

- Using src_btf->nr_types instead of btf__type_cnt()-1 for the type count, which is correct for both split and non-split sources.

- Pre-emptively calling btf_ensure_modifiable() on the destination's base BTF to prevent a use-after-free: btf_rewrite_str() resolves strings via btf__str_by_offset(src) which may return pointers into the shared base's string data; btf__add_str(dst) then calls btf__find_str(base) which can trigger btf_ensure_modifiable(base), reallocating that string data and invalidating the pointer.

For non-split sources the behavior is identical: src_start_id is 1, the type_id < 1 guard is never true (VOID is already skipped), and the remapping formula reduces to the original.

---
 tools/lib/bpf/btf.c | 44 ++++++++++++++++++++++++++++++++++++--------
 1 file changed, 36 insertions(+), 8 deletions(-)
tools/lib/bpf/btf.c +36 -8
index 83fe79ffcb8f..654fb9ffa01f 100644
@@ -2004,24 +2004,47 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 {
 	struct btf_pipe p = { .src = src_btf, .dst = btf };
 	int data_sz, sz, cnt, i, err, old_strs_len;
+	__u32 src_start_id;
 	__u32 *off;
 	void *t;
 
-	/* appending split BTF isn't supported yet */
-	if (src_btf->base_btf)
-		return libbpf_err(-ENOTSUP);
+	/* When appending split BTF, the destination must share the same base
+	 * BTF so that base type ID references remain valid.
+	 */
+	if (src_btf->base_btf &&
+	    src_btf->base_btf != btf->base_btf)
+		return libbpf_err(-EINVAL);
+
+	src_start_id = src_btf->base_btf ? btf__type_cnt(src_btf->base_btf) : 1;
 
 	/* deconstruct BTF, if necessary, and invalidate raw_data */
 	if (btf_ensure_modifiable(btf))
 		return libbpf_err(-ENOMEM);
 
+	/* If dst has a base BTF, ensure it is modifiable now so that
+	 * btf__str_by_offset() on the base returns pointers from the
+	 * strset copy rather than from raw_data.
+	 *
+	 * Without this, the first btf_rewrite_str() call that encounters
+	 * a base string offset resolves it via btf__str_by_offset(src),
+	 * which for split src returns a pointer into base->raw_data.
+	 * That pointer is then passed to btf__add_str(dst), which calls
+	 * btf__find_str(base) -> btf_ensure_modifiable(base).  The
+	 * first btf_ensure_modifiable() call copies strings into a
+	 * strset but then frees raw_data via btf_invalidate_raw_data(),
+	 * leaving the pointer dangling before strset__find_str() uses
+	 * it.
+	 */
+	if (btf->base_btf && btf_ensure_modifiable(btf->base_btf))
+		return libbpf_err(-ENOMEM);
+
 	/* remember original strings section size if we have to roll back
 	 * partial strings section changes
 	 */
 	old_strs_len = btf->hdr->str_len;
 
 	data_sz = src_btf->hdr->type_len;
-	cnt = btf__type_cnt(src_btf) - 1;
+	cnt = src_btf->nr_types;
 
 	/* pre-allocate enough memory for new types */
 	t = btf_add_type_mem(btf, data_sz);
@@ -2074,11 +2097,16 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 			if (!*type_id) /* nothing to do for VOID references */
 				continue;
 
-			/* we haven't updated btf's type count yet, so
-			 * btf->start_id + btf->nr_types - 1 is the type ID offset we should
-			 * add to all newly added BTF types
+			/* For split BTF sources, type IDs referencing the
+			 * base BTF (< src_start_id) remain unchanged since
+			 * dst shares the same base.  Only remap IDs in the
+			 * split portion.  For non-split sources src_start_id
+			 * is 1 so all IDs are remapped as before.
 			 */
-			*type_id += btf->start_id + btf->nr_types - 1;
+			if (*type_id < src_start_id)
+				continue;
+
+			*type_id += btf->start_id + btf->nr_types - src_start_id;
 		}
 
 		/* go to next type data and type offset index entry */
Signature · 1 line
2.53.0
#5 lore raw

PATCH v4 2/3 bpftool: support merging multiple module BTFs in btf dump

In reply to Josef Bacik

Add support for specifying multiple file sources in 'bpftool btf dump' to generate a single C header containing types from vmlinux plus multiple kernel modules:

bpftool btf dump file /sys/kernel/btf/mod1 file /sys/kernel/btf/mod2 format c

This is useful for BPF programs that need to access types defined in kernel modules. Previously this required a separate bpftool invocation for each module, producing separate headers that could not be combined due to overlapping vmlinux type definitions.

The implementation collects all file paths, then for the multi-file case creates an empty split BTF on the vmlinux base and iteratively merges each module's types into it via btf__add_btf(). The single-file code path is preserved exactly to avoid any regression risk.

Auto-detection of vmlinux as the base BTF from sysfs paths works as before. If vmlinux itself appears in the file list it is skipped with a warning since its types are already provided by the base.

---
 tools/bpf/bpftool/btf.c | 135 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 122 insertions(+), 13 deletions(-)
tools/bpf/bpftool/btf.c +122 -13
index 946612029dee..16bc1332639f 100644
@@ -28,6 +28,7 @@
 #define FASTCALL_DECL_TAG	"bpf_fastcall"
 
 #define MAX_ROOT_IDS		16
+#define MAX_BTF_FILES		64
 
 static const char * const btf_kind_str[NR_BTF_KINDS] = {
 	[BTF_KIND_UNKN]		= "UNKNOWN",
@@ -958,20 +959,127 @@ static int do_dump(int argc, char **argv)
 		NEXT_ARG();
 	} else if (is_prefix(src, "file")) {
 		const char sysfs_prefix[] = "/sys/kernel/btf/";
+		const char *files[MAX_BTF_FILES];
+		int nr_files = 0;
 
-		if (!base_btf &&
-		    strncmp(*argv, sysfs_prefix, sizeof(sysfs_prefix) - 1) == 0 &&
-		    strcmp(*argv, sysfs_vmlinux) != 0)
-			base = get_vmlinux_btf_from_sysfs();
-
-		btf = btf__parse_split(*argv, base ?: base_btf);
-		if (!btf) {
-			err = -errno;
-			p_err("failed to load BTF from %s: %s",
-			      *argv, strerror(errno));
-			goto done;
-		}
+		files[nr_files++] = *argv;
 		NEXT_ARG();
+
+		/* Collect additional file arguments. Use strcmp (exact match)
+		 * to avoid ambiguity with file paths that are prefixes of
+		 * "file" (e.g., "./f").
+		 */
+		while (argc && strcmp(*argv, "file") == 0) {
+			NEXT_ARG();
+			if (!REQ_ARGS(1)) {
+				err = -EINVAL;
+				goto done;
+			}
+			if (nr_files >= MAX_BTF_FILES) {
+				p_err("too many BTF files (max %d)",
+				      MAX_BTF_FILES);
+				err = -E2BIG;
+				goto done;
+			}
+			files[nr_files++] = *argv;
+			NEXT_ARG();
+		}
+
+		if (nr_files == 1) {
+			/* Single file — preserve existing behavior exactly */
+			if (!base_btf &&
+			    strncmp(files[0], sysfs_prefix,
+				    sizeof(sysfs_prefix) - 1) == 0 &&
+			    strcmp(files[0], sysfs_vmlinux) != 0)
+				base = get_vmlinux_btf_from_sysfs();
+
+			btf = btf__parse_split(files[0], base ?: base_btf);
+			if (!btf) {
+				err = -errno;
+				p_err("failed to load BTF from %s: %s",
+				      files[0], strerror(errno));
+				goto done;
+			}
+		} else {
+			struct btf *vmlinux_base = base_btf;
+			struct btf *combined, *mod;
+			int j, ret;
+
+			/* Auto-detect vmlinux base from sysfs if needed */
+			if (!vmlinux_base) {
+				for (j = 0; j < nr_files; j++) {
+					if (strncmp(files[j], sysfs_prefix,
+						    sizeof(sysfs_prefix) - 1) == 0 &&
+					    strcmp(files[j], sysfs_vmlinux) != 0) {
+						base = get_vmlinux_btf_from_sysfs();
+						vmlinux_base = base;
+						break;
+					}
+				}
+			}
+			if (!vmlinux_base) {
+				p_err("base BTF is required when merging multiple BTF files; use -B/--base-btf or use sysfs paths");
+				err = -EINVAL;
+				goto done;
+			}
+
+			/* Filter out vmlinux from the file list */
+			for (j = 0; j < nr_files; j++) {
+				if (strcmp(files[j], sysfs_vmlinux) == 0) {
+					p_info("skipping %s (already loaded as base)",
+					       sysfs_vmlinux);
+					memmove(&files[j], &files[j + 1],
+						(nr_files - j - 1) * sizeof(files[0]));
+					nr_files--;
+					j--;
+				}
+			}
+			if (nr_files == 0) {
+				p_err("no module BTF files to merge (all paths were vmlinux)");
+				err = -EINVAL;
+				goto done;
+			}
+
+			combined = btf__new_empty_split(vmlinux_base);
+			if (!combined) {
+				err = -errno;
+				p_err("failed to create combined BTF: %s",
+				      strerror(-err));
+				goto done;
+			}
+
+			for (j = 0; j < nr_files; j++) {
+				mod = btf__parse_split(files[j], vmlinux_base);
+				if (!mod) {
+					err = -errno;
+					p_err("failed to load BTF from %s: %s",
+					      files[j], strerror(-err));
+					btf__free(combined);
+					goto done;
+				}
+
+				ret = btf__add_btf(combined, mod);
+				btf__free(mod);
+				if (ret < 0) {
+					p_err("failed to merge BTF from %s: %s",
+					      files[j], strerror(-ret));
+					btf__free(combined);
+					err = ret;
+					goto done;
+				}
+			}
+
+			ret = btf__dedup(combined, NULL);
+			if (ret) {
+				p_err("failed to dedup combined BTF: %s",
+				      strerror(-ret));
+				btf__free(combined);
+				err = ret;
+				goto done;
+			}
+
+			btf = combined;
+		}
 	} else {
 		err = -1;
 		p_err("unrecognized BTF source specifier: '%s'", src);
@@ -1445,7 +1553,8 @@ static int do_help(int argc, char **argv)
 		"       %1$s %2$s dump BTF_SRC [format FORMAT] [root_id ROOT_ID]\n"
 		"       %1$s %2$s help\n"
 		"\n"
-		"       BTF_SRC := { id BTF_ID | prog PROG | map MAP [{key | value | kv | all}] | file FILE }\n"
+		"       BTF_SRC := { id BTF_ID | prog PROG | map MAP [{key | value | kv | all}] |\n"
+		"                    file FILE [file FILE]... }\n"
 		"       FORMAT  := { raw | c [unsorted] }\n"
 		"       " HELP_SPEC_MAP "\n"
 		"       " HELP_SPEC_PROGRAM "\n"
Signature · 1 line
2.53.0
#8 lore raw

PATCH v4 3/3 selftests/bpf: add test for btf__add_btf() with split BTF sources

In reply to Josef Bacik

Add a test that verifies btf__add_btf() correctly handles merging multiple split BTF objects that share the same base BTF. The test creates two sibling split BTFs on a common base, merges them into a combined split BTF, and validates that base type references are preserved while split type references are properly remapped.

---
 .../selftests/bpf/prog_tests/btf_write.c      | 89 +++++++++++++++++++
 1 file changed, 89 insertions(+)
tools/testing/selftests/bpf/prog_tests/btf_write.c +89 -0
index 6e36de1302fc..80353a545cdd 100644
@@ -497,10 +497,99 @@ static void test_btf_add_btf()
 	btf__free(btf2);
 }
 
+static void test_btf_add_btf_split()
+{
+	struct btf *base = NULL, *split1 = NULL, *split2 = NULL;
+	struct btf *combined = NULL;
+	int id;
+
+	/* Create a base BTF with an INT and a PTR to it */
+	base = btf__new_empty();
+	if (!ASSERT_OK_PTR(base, "base"))
+		return;
+
+	id = btf__add_int(base, "int", 4, BTF_INT_SIGNED);
+	ASSERT_EQ(id, 1, "base_int_id");
+	id = btf__add_ptr(base, 1);
+	ASSERT_EQ(id, 2, "base_ptr_id");
+
+	/* base has 2 types, type IDs 1..2 */
+	ASSERT_EQ(btf__type_cnt(base), 3, "base_type_cnt");
+
+	/* Create split1 on base: a STRUCT referencing base's int (ID 1) */
+	split1 = btf__new_empty_split(base);
+	if (!ASSERT_OK_PTR(split1, "split1"))
+		goto cleanup;
+
+	id = btf__add_struct(split1, "s1", 4);
+	/* split types start at base_type_cnt = 3 */
+	ASSERT_EQ(id, 3, "split1_struct_id");
+	btf__add_field(split1, "x", 1, 0, 0); /* refers to base int */
+
+	id = btf__add_ptr(split1, 3);
+	ASSERT_EQ(id, 4, "split1_ptr_id"); /* ptr to the struct (split self-ref) */
+
+	/* Create split2 on base: a TYPEDEF referencing base's ptr (ID 2) */
+	split2 = btf__new_empty_split(base);
+	if (!ASSERT_OK_PTR(split2, "split2"))
+		goto cleanup;
+
+	id = btf__add_typedef(split2, "int_ptr", 2); /* refers to base ptr */
+	ASSERT_EQ(id, 3, "split2_typedef_id");
+
+	id = btf__add_struct(split2, "s2", 8);
+	ASSERT_EQ(id, 4, "split2_struct_id");
+	btf__add_field(split2, "p", 3, 0, 0); /* refers to split2's own typedef */
+
+	/* Create combined split BTF on same base and merge both */
+	combined = btf__new_empty_split(base);
+	if (!ASSERT_OK_PTR(combined, "combined"))
+		goto cleanup;
+
+	/* Merge split1: its types (3,4) should land at IDs 3,4 in combined */
+	id = btf__add_btf(combined, split1);
+	if (!ASSERT_GE(id, 0, "add_split1"))
+		goto cleanup;
+	ASSERT_EQ(id, 3, "split1_first_id");
+
+	/* Merge split2: its types (3,4) should be remapped to IDs 5,6 */
+	id = btf__add_btf(combined, split2);
+	if (!ASSERT_GE(id, 0, "add_split2"))
+		goto cleanup;
+	ASSERT_EQ(id, 5, "split2_first_id");
+
+	/* combined should have: base (2 types) + split1 (2) + split2 (2) = 6 types + void */
+	ASSERT_EQ(btf__type_cnt(combined), 7, "combined_type_cnt");
+
+	VALIDATE_RAW_BTF(
+		combined,
+		/* base types (IDs 1-2) */
+		"[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+		"[2] PTR '(anon)' type_id=1",
+
+		/* split1 types (IDs 3-4): base refs unchanged */
+		"[3] STRUCT 's1' size=4 vlen=1\n"
+		"\t'x' type_id=1 bits_offset=0",      /* refers to base int=1 */
+		"[4] PTR '(anon)' type_id=3",          /* refers to split1's struct=3 */
+
+		/* split2 types (IDs 5-6): remapped from 3,4 to 5,6 */
+		"[5] TYPEDEF 'int_ptr' type_id=2",     /* base ptr=2, unchanged */
+		"[6] STRUCT 's2' size=8 vlen=1\n"
+		"\t'p' type_id=5 bits_offset=0");      /* split2 typedef: 3->5 */
+
+cleanup:
+	btf__free(combined);
+	btf__free(split2);
+	btf__free(split1);
+	btf__free(base);
+}
+
 void test_btf_write()
 {
 	if (test__start_subtest("btf_add"))
 		test_btf_add();
 	if (test__start_subtest("btf_add_btf"))
 		test_btf_add_btf();
+	if (test__start_subtest("btf_add_btf_split"))
+		test_btf_add_btf_split();
 }
Signature · 1 line
2.53.0
#9 lore raw

Re: [PATCH v4 3/3] selftests/bpf: add test for btf__add_btf() with split BTF sources

Quoted 8 lines · Josef Bacik

Add a test that verifies btf__add_btf() correctly handles merging multiple split BTF objects that share the same base BTF. The test creates two sibling split BTFs on a common base, merges them into a combined split BTF, and validates that base type references are preserved while split type references are properly remapped.

v4 and it is still missing your SOB.

Chris,
would be great if AI catch this.

#10 lore raw

Re: [PATCH v4 3/3] selftests/bpf: add test for btf__add_btf() with split BTF sources

Quoted 9 lines · Alexei Starovoitov
Quoted 7 lines · Josef Bacik

Add a test that verifies btf__add_btf() correctly handles merging multiple split BTF objects that share the same base BTF. The test creates two sibling split BTFs on a common base, merges them into a combined split BTF, and validates that base type references are preserved while split type references are properly remapped.

Quoted 4 lines
Quoted 1 line

v4 and it is still missing your SOB.

I didn't want you to feel left out, had to leave you something to yell at me over. Thanks,

Josef

#6 lore raw

Re: [PATCH v4 2/3] bpftool: support merging multiple module BTFs in btf dump

In reply to Josef Bacik
Quoted 16 lines · Josef Bacik

Add support for specifying multiple file sources in 'bpftool btf dump' to generate a single C header containing types from vmlinux plus multiple kernel modules:

bpftool btf dump file /sys/kernel/btf/mod1 file /sys/kernel/btf/mod2 format c

This is useful for BPF programs that need to access types defined in kernel modules. Previously this required a separate bpftool invocation for each module, producing separate headers that could not be combined due to overlapping vmlinux type definitions.

The implementation collects all file paths, then for the multi-file case creates an empty split BTF on the vmlinux base and iteratively merges each module's types into it via btf__add_btf(). The single-file code path is preserved exactly to avoid any regression risk.

Quoted 6 lines

Auto-detection of vmlinux as the base BTF from sysfs paths works as before. If vmlinux itself appears in the file list it is skipped with a warning since its types are already provided by the base.

neat idea! some suggestions below, and I think needs a Documentation/bpftool-btf.rst doc update too.

Quoted 56 lines
---
 tools/bpf/bpftool/btf.c | 135 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 122 insertions(+), 13 deletions(-)
tools/bpf/bpftool/btf.c +23 -12
index 946612029dee..16bc1332639f 100644
@@ -28,6 +28,7 @@
 #define FASTCALL_DECL_TAG	"bpf_fastcall"
 
 #define MAX_ROOT_IDS		16
+#define MAX_BTF_FILES		64
 
 static const char * const btf_kind_str[NR_BTF_KINDS] = {
 	[BTF_KIND_UNKN]		= "UNKNOWN",
@@ -958,20 +959,127 @@ static int do_dump(int argc, char **argv)
 		NEXT_ARG();
 	} else if (is_prefix(src, "file")) {
 		const char sysfs_prefix[] = "/sys/kernel/btf/";
+		const char *files[MAX_BTF_FILES];
+		int nr_files = 0;
 
-		if (!base_btf &&
-		    strncmp(*argv, sysfs_prefix, sizeof(sysfs_prefix) - 1) == 0 &&
-		    strcmp(*argv, sysfs_vmlinux) != 0)
-			base = get_vmlinux_btf_from_sysfs();
-
-		btf = btf__parse_split(*argv, base ?: base_btf);
-		if (!btf) {
-			err = -errno;
-			p_err("failed to load BTF from %s: %s",
-			      *argv, strerror(errno));
-			goto done;
-		}
+		files[nr_files++] = *argv;
 		NEXT_ARG();
+
+		/* Collect additional file arguments. Use strcmp (exact match)
+		 * to avoid ambiguity with file paths that are prefixes of
+		 * "file" (e.g., "./f").
+		 */
+		while (argc && strcmp(*argv, "file") == 0) {
+			NEXT_ARG();
+			if (!REQ_ARGS(1)) {
+				err = -EINVAL;
+				goto done;
+			}
+			if (nr_files >= MAX_BTF_FILES) {
+				p_err("too many BTF files (max %d)",
+				      MAX_BTF_FILES);
+				err = -E2BIG;
+				goto done;
+			}
+			files[nr_files++] = *argv;
+			NEXT_ARG();

tools/bpf/bpftool/btf.c static int do_dump(int argc, char **argv) · line 995

Quoted 10 lines
tools/bpf/bpftool/btf.c +10 -0
+		}
+
+		if (nr_files == 1) {
+			/* Single file — preserve existing behavior exactly */
+			if (!base_btf &&
+			    strncmp(files[0], sysfs_prefix,
+				    sizeof(sysfs_prefix) - 1) == 0 &&
+			    strcmp(files[0], sysfs_vmlinux) != 0)
+				base = get_vmlinux_btf_from_sysfs();
+

looks to me like you could avoid the repetition here for the base BTF retrieval to above the "if (nr_files == 1)" check, since it'll only be done once if nr_files == 1

		vmlinux_base = base_btf;

		/* Auto-detect vmlinux base from sysfs if needed */
		if (!vmlinux_base) {
			for (j = 0; j < nr_files; j++) {
				if (strncmp(files[j], sysfs_prefix,
					    sizeof(sysfs_prefix) - 1) == 0 &&
				    strcmp(files[j], sysfs_vmlinux) != 0) {
					base = get_vmlinux_btf_from_sysfs();
					vmlinux_base = base;
					break;
				}
			}
		}
Quoted 31 lines
tools/bpf/bpftool/btf.c +31 -0
+			btf = btf__parse_split(files[0], base ?: base_btf);
+			if (!btf) {
+				err = -errno;
+				p_err("failed to load BTF from %s: %s",
+				      files[0], strerror(errno));
+				goto done;
+			}
+		} else {
+			struct btf *vmlinux_base = base_btf;
+			struct btf *combined, *mod;
+			int j, ret;
+
+			/* Auto-detect vmlinux base from sysfs if needed */
+			if (!vmlinux_base) {
+				for (j = 0; j < nr_files; j++) {
+					if (strncmp(files[j], sysfs_prefix,
+						    sizeof(sysfs_prefix) - 1) == 0 &&
+					    strcmp(files[j], sysfs_vmlinux) != 0) {
+						base = get_vmlinux_btf_from_sysfs();
+						vmlinux_base = base;
+						break;
+					}
+				}
+			}
+			if (!vmlinux_base) {
+				p_err("base BTF is required when merging multiple BTF files; use -B/--base-btf or use sysfs paths");
+				err = -EINVAL;
+				goto done;
+			}
+
+			/* Filter out vmlinux from the file list */

tools/bpf/bpftool/btf.c static int do_dump(int argc, char **argv) · line 1036

Quoted 10 lines
tools/bpf/bpftool/btf.c +10 -0
+			for (j = 0; j < nr_files; j++) {
+				if (strcmp(files[j], sysfs_vmlinux) == 0) {
+					p_info("skipping %s (already loaded as base)",
+					       sysfs_vmlinux);
+					memmove(&files[j], &files[j + 1],
+						(nr_files - j - 1) * sizeof(files[0]));
+					nr_files--;
+					j--;
+				}
+			}

would it be simpler to filter this above at collection time?

Quoted 58 lines
tools/bpf/bpftool/btf.c +47 -1
+			if (nr_files == 0) {
+				p_err("no module BTF files to merge (all paths were vmlinux)");
+				err = -EINVAL;
+				goto done;
+			}
+
+			combined = btf__new_empty_split(vmlinux_base);
+			if (!combined) {
+				err = -errno;
+				p_err("failed to create combined BTF: %s",
+				      strerror(-err));
+				goto done;
+			}
+
+			for (j = 0; j < nr_files; j++) {
+				mod = btf__parse_split(files[j], vmlinux_base);
+				if (!mod) {
+					err = -errno;
+					p_err("failed to load BTF from %s: %s",
+					      files[j], strerror(-err));
+					btf__free(combined);
+					goto done;
+				}
+
+				ret = btf__add_btf(combined, mod);
+				btf__free(mod);
+				if (ret < 0) {
+					p_err("failed to merge BTF from %s: %s",
+					      files[j], strerror(-ret));
+					btf__free(combined);
+					err = ret;
+					goto done;
+				}
+			}
+
+			ret = btf__dedup(combined, NULL);
+			if (ret) {
+				p_err("failed to dedup combined BTF: %s",
+				      strerror(-ret));
+				btf__free(combined);
+				err = ret;
+				goto done;
+			}
+
+			btf = combined;> +		}
 	} else {
 		err = -1;
 		p_err("unrecognized BTF source specifier: '%s'", src);
@@ -1445,7 +1553,8 @@ static int do_help(int argc, char **argv)
 		"       %1$s %2$s dump BTF_SRC [format FORMAT] [root_id ROOT_ID]\n"
 		"       %1$s %2$s help\n"
 		"\n"
-		"       BTF_SRC := { id BTF_ID | prog PROG | map MAP [{key | value | kv | all}] | file FILE }\n"
+		"       BTF_SRC := { id BTF_ID | prog PROG | map MAP [{key | value | kv | all}] |\n"
+		"                    file FILE [file FILE]... }\n"
 		"       FORMAT  := { raw | c [unsorted] }\n"
 		"       " HELP_SPEC_MAP "\n"
 		"       " HELP_SPEC_PROGRAM "\n"
#11 lore raw

Re: [PATCH v4 3/3] selftests/bpf: add test for btf__add_btf() with split BTF sources

In reply to Josef Bacik
Quoted 7 lines · Josef Bacik

Add a test that verifies btf__add_btf() correctly handles merging multiple split BTF objects that share the same base BTF. The test creates two sibling split BTFs on a common base, merges them into a combined split BTF, and validates that base type references are preserved while split type references are properly remapped.

looks great; one suggetion below..

Quoted 63 lines
---
 .../selftests/bpf/prog_tests/btf_write.c      | 89 +++++++++++++++++++
 1 file changed, 89 insertions(+)
tools/testing/selftests/bpf/prog_tests/btf_write.c +51 -0
index 6e36de1302fc..80353a545cdd 100644
@@ -497,10 +497,99 @@ static void test_btf_add_btf()
 	btf__free(btf2);
 }
 
+static void test_btf_add_btf_split()
+{
+	struct btf *base = NULL, *split1 = NULL, *split2 = NULL;
+	struct btf *combined = NULL;
+	int id;
+
+	/* Create a base BTF with an INT and a PTR to it */
+	base = btf__new_empty();
+	if (!ASSERT_OK_PTR(base, "base"))
+		return;
+
+	id = btf__add_int(base, "int", 4, BTF_INT_SIGNED);
+	ASSERT_EQ(id, 1, "base_int_id");
+	id = btf__add_ptr(base, 1);
+	ASSERT_EQ(id, 2, "base_ptr_id");
+
+	/* base has 2 types, type IDs 1..2 */
+	ASSERT_EQ(btf__type_cnt(base), 3, "base_type_cnt");
+
+	/* Create split1 on base: a STRUCT referencing base's int (ID 1) */
+	split1 = btf__new_empty_split(base);
+	if (!ASSERT_OK_PTR(split1, "split1"))
+		goto cleanup;
+
+	id = btf__add_struct(split1, "s1", 4);
+	/* split types start at base_type_cnt = 3 */
+	ASSERT_EQ(id, 3, "split1_struct_id");
+	btf__add_field(split1, "x", 1, 0, 0); /* refers to base int */
+
+	id = btf__add_ptr(split1, 3);
+	ASSERT_EQ(id, 4, "split1_ptr_id"); /* ptr to the struct (split self-ref) */
+
+	/* Create split2 on base: a TYPEDEF referencing base's ptr (ID 2) */
+	split2 = btf__new_empty_split(base);
+	if (!ASSERT_OK_PTR(split2, "split2"))
+		goto cleanup;
+
+	id = btf__add_typedef(split2, "int_ptr", 2); /* refers to base ptr */
+	ASSERT_EQ(id, 3, "split2_typedef_id");
+
+	id = btf__add_struct(split2, "s2", 8);
+	ASSERT_EQ(id, 4, "split2_struct_id");
+	btf__add_field(split2, "p", 3, 0, 0); /* refers to split2's own typedef */
+
+	/* Create combined split BTF on same base and merge both */
+	combined = btf__new_empty_split(base);
+	if (!ASSERT_OK_PTR(combined, "combined"))
+		goto cleanup;
+
+	/* Merge split1: its types (3,4) should land at IDs 3,4 in combined */
+	id = btf__add_btf(combined, split1);

tools/testing/selftests/bpf/prog_tests/btf_write.c static void test_btf_add_btf() · line 560

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/btf_write.c +10 -0
+	if (!ASSERT_GE(id, 0, "add_split1"))
+		goto cleanup;
+	ASSERT_EQ(id, 3, "split1_first_id");
+
+	/* Merge split2: its types (3,4) should be remapped to IDs 5,6 */
+	id = btf__add_btf(combined, split2);
+	if (!ASSERT_GE(id, 0, "add_split2"))
+		goto cleanup;
+	ASSERT_EQ(id, 5, "split2_first_id");
+

could we add a duplicated type to both split1 and split2 and call btf__dedup() on the combination to more closely mirror the bpftool code flow?

Quoted 35 lines
tools/testing/selftests/bpf/prog_tests/btf_write.c +28 -0
+	/* combined should have: base (2 types) + split1 (2) + split2 (2) = 6 types + void */
+	ASSERT_EQ(btf__type_cnt(combined), 7, "combined_type_cnt");
+
+	VALIDATE_RAW_BTF(
+		combined,
+		/* base types (IDs 1-2) */
+		"[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+		"[2] PTR '(anon)' type_id=1",
+
+		/* split1 types (IDs 3-4): base refs unchanged */
+		"[3] STRUCT 's1' size=4 vlen=1\n"
+		"\t'x' type_id=1 bits_offset=0",      /* refers to base int=1 */
+		"[4] PTR '(anon)' type_id=3",          /* refers to split1's struct=3 */
+
+		/* split2 types (IDs 5-6): remapped from 3,4 to 5,6 */
+		"[5] TYPEDEF 'int_ptr' type_id=2",     /* base ptr=2, unchanged */
+		"[6] STRUCT 's2' size=8 vlen=1\n"
+		"\t'p' type_id=5 bits_offset=0");      /* split2 typedef: 3->5 */
+
+cleanup:
+	btf__free(combined);
+	btf__free(split2);
+	btf__free(split1);
+	btf__free(base);
+}
+
 void test_btf_write()
 {
 	if (test__start_subtest("btf_add"))
 		test_btf_add();
 	if (test__start_subtest("btf_add_btf"))
 		test_btf_add_btf();
+	if (test__start_subtest("btf_add_btf_split"))
+		test_btf_add_btf_split();
 }
#7 lore raw

Re: [PATCH v4 2/3] bpftool: support merging multiple module BTFs in btf dump

In reply to Josef Bacik

2026-02-20 17:43 UTC-0500 ~ Josef Bacik <[email protected]>

Quoted 16 lines

Add support for specifying multiple file sources in 'bpftool btf dump' to generate a single C header containing types from vmlinux plus multiple kernel modules:

bpftool btf dump file /sys/kernel/btf/mod1 file /sys/kernel/btf/mod2 format c

This is useful for BPF programs that need to access types defined in kernel modules. Previously this required a separate bpftool invocation for each module, producing separate headers that could not be combined due to overlapping vmlinux type definitions.

The implementation collects all file paths, then for the multi-file case creates an empty split BTF on the vmlinux base and iteratively merges each module's types into it via btf__add_btf(). The single-file code path is preserved exactly to avoid any regression risk.

Quoted 5 lines

Auto-detection of vmlinux as the base BTF from sysfs paths works as before. If vmlinux itself appears in the file list it is skipped with a warning since its types are already provided by the base.

Thanks for this patch!

Quoted 1 line

Looking at Documentation/process/coding-assistants.rst and git logs, I believe this tag should be like "Assisted-by: Claude:claude-opus-4-6". (Same for other patches in the series.)

Quoted 32 lines
---
 tools/bpf/bpftool/btf.c | 135 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 122 insertions(+), 13 deletions(-)
tools/bpf/bpftool/btf.c +3 -9
index 946612029dee..16bc1332639f 100644
@@ -28,6 +28,7 @@
 #define FASTCALL_DECL_TAG	"bpf_fastcall"
 
 #define MAX_ROOT_IDS		16
+#define MAX_BTF_FILES		64
 
 static const char * const btf_kind_str[NR_BTF_KINDS] = {
 	[BTF_KIND_UNKN]		= "UNKNOWN",
@@ -958,20 +959,127 @@ static int do_dump(int argc, char **argv)
 		NEXT_ARG();
 	} else if (is_prefix(src, "file")) {
 		const char sysfs_prefix[] = "/sys/kernel/btf/";
+		const char *files[MAX_BTF_FILES];
+		int nr_files = 0;
 
-		if (!base_btf &&
-		    strncmp(*argv, sysfs_prefix, sizeof(sysfs_prefix) - 1) == 0 &&
-		    strcmp(*argv, sysfs_vmlinux) != 0)
-			base = get_vmlinux_btf_from_sysfs();
-
-		btf = btf__parse_split(*argv, base ?: base_btf);
-		if (!btf) {
-			err = -errno;
-			p_err("failed to load BTF from %s: %s",

tools/bpf/bpftool/btf.c static int do_dump(int argc, char **argv) · line 971

Quoted 10 lines
tools/bpf/bpftool/btf.c +6 -3
-			      *argv, strerror(errno));
-			goto done;
-		}
+		files[nr_files++] = *argv;
 		NEXT_ARG();
+
+		/* Collect additional file arguments. Use strcmp (exact match)
+		 * to avoid ambiguity with file paths that are prefixes of
+		 * "file" (e.g., "./f").
+		 */

I don't understand why we'd remove subcommand prefixing here, each file name should be preceded with the "file" keyword so from command parsing, there's no ambiguity?

Quoted 8 lines
tools/bpf/bpftool/btf.c +8 -0
+		while (argc && strcmp(*argv, "file") == 0) {
+			NEXT_ARG();
+			if (!REQ_ARGS(1)) {
+				err = -EINVAL;
+				goto done;
+			}
+			if (nr_files >= MAX_BTF_FILES) {
+				p_err("too many BTF files (max %d)",

tools/bpf/bpftool/btf.c static int do_dump(int argc, char **argv) · line 989

Quoted 10 lines
tools/bpf/bpftool/btf.c +10 -0
+				      MAX_BTF_FILES);
+				err = -E2BIG;
+				goto done;
+			}
+			files[nr_files++] = *argv;
+			NEXT_ARG();
+		}
+
+		if (nr_files == 1) {
+			/* Single file — preserve existing behavior exactly */

I guess Claude inserted this comment. Please use a regular dash, there's no need to turn the file into UTF-8 for this (or remove the comment).

Quoted 4 lines
tools/bpf/bpftool/btf.c +4 -0
+			if (!base_btf &&
+			    strncmp(files[0], sysfs_prefix,
+				    sizeof(sysfs_prefix) - 1) == 0 &&
+			    strcmp(files[0], sysfs_vmlinux) != 0)

tools/bpf/bpftool/btf.c static int do_dump(int argc, char **argv) · line 1003

Quoted 10 lines
tools/bpf/bpftool/btf.c +10 -0
+				base = get_vmlinux_btf_from_sysfs();
+
+			btf = btf__parse_split(files[0], base ?: base_btf);
+			if (!btf) {
+				err = -errno;
+				p_err("failed to load BTF from %s: %s",
+				      files[0], strerror(errno));
+				goto done;
+			}
+		} else {

That's quite a lot of processing in the middle of the argument parsing code, I wouldn't mind having this as a separate function.

Quoted 83 lines
tools/bpf/bpftool/btf.c +79 -0
+			struct btf *vmlinux_base = base_btf;
+			struct btf *combined, *mod;
+			int j, ret;
+
+			/* Auto-detect vmlinux base from sysfs if needed */
+			if (!vmlinux_base) {
+				for (j = 0; j < nr_files; j++) {
+					if (strncmp(files[j], sysfs_prefix,
+						    sizeof(sysfs_prefix) - 1) == 0 &&
+					    strcmp(files[j], sysfs_vmlinux) != 0) {
+						base = get_vmlinux_btf_from_sysfs();
+						vmlinux_base = base;
+						break;
+					}
+				}
+			}
+			if (!vmlinux_base) {
+				p_err("base BTF is required when merging multiple BTF files; use -B/--base-btf or use sysfs paths");
+				err = -EINVAL;
+				goto done;
+			}
+
+			/* Filter out vmlinux from the file list */
+			for (j = 0; j < nr_files; j++) {
+				if (strcmp(files[j], sysfs_vmlinux) == 0) {
+					p_info("skipping %s (already loaded as base)",
+					       sysfs_vmlinux);
+					memmove(&files[j], &files[j + 1],
+						(nr_files - j - 1) * sizeof(files[0]));
+					nr_files--;
+					j--;
+				}
+			}
+			if (nr_files == 0) {
+				p_err("no module BTF files to merge (all paths were vmlinux)");
+				err = -EINVAL;
+				goto done;
+			}
+
+			combined = btf__new_empty_split(vmlinux_base);
+			if (!combined) {
+				err = -errno;
+				p_err("failed to create combined BTF: %s",
+				      strerror(-err));
+				goto done;
+			}
+
+			for (j = 0; j < nr_files; j++) {
+				mod = btf__parse_split(files[j], vmlinux_base);
+				if (!mod) {
+					err = -errno;
+					p_err("failed to load BTF from %s: %s",
+					      files[j], strerror(-err));
+					btf__free(combined);
+					goto done;
+				}
+
+				ret = btf__add_btf(combined, mod);
+				btf__free(mod);
+				if (ret < 0) {
+					p_err("failed to merge BTF from %s: %s",
+					      files[j], strerror(-ret));
+					btf__free(combined);
+					err = ret;
+					goto done;
+				}
+			}
+
+			ret = btf__dedup(combined, NULL);
+			if (ret) {
+				p_err("failed to dedup combined BTF: %s",
+				      strerror(-ret));
+				btf__free(combined);
+				err = ret;
+				goto done;
+			}
+
+			btf = combined;
+		}
 	} else {
 		err = -1;
 		p_err("unrecognized BTF source specifier: '%s'", src);

tools/bpf/bpftool/btf.c static int do_help(int argc, char **argv) · line 1557

Quoted 6 lines
tools/bpf/bpftool/btf.c +2 -1
@@ -1445,7 +1553,8 @@ static int do_help(int argc, char **argv)
 		"       %1$s %2$s dump BTF_SRC [format FORMAT] [root_id ROOT_ID]\n"
 		"       %1$s %2$s help\n"
 		"\n"
-		"       BTF_SRC := { id BTF_ID | prog PROG | map MAP [{key | value | kv | all}] | file FILE }\n"
+		"       BTF_SRC := { id BTF_ID | prog PROG | map MAP [{key | value | kv | all}] |\n"
+		"                    file FILE [file FILE]... }\n"

As Alan said, can you please update the related man page? There's the command summary at the top, and then the command description to update.

Can you please also update bash-completion/bpftool? You could use:

    diff --git i/tools/bpf/bpftool/bash-completion/bpftool w/tools/bpf/bpftool/bash-completion/bpftool
    index a28f0cc522e4..babb0d4e9753 100644
    --- i/tools/bpf/bpftool/bash-completion/bpftool
    +++ w/tools/bpf/bpftool/bash-completion/bpftool
    @@ -961,10 +961,14 @@ _bpftool()
                             *)
                                 # emit extra options
                                 case ${words[3]} in
    -                                id|file)
    +                                id)
                                         COMPREPLY=( $( compgen -W "root_id" -- "$cur" ) )
                                         _bpftool_once_attr 'format'
                                         ;;
    +                                file)
    +                                    COMPREPLY=( $( compgen -W "root_id file" -- "$cur" ) )
    +                                    _bpftool_once_attr 'format'
    +                                    ;;
                                     map|prog)
                                         if [[ ${words[3]} == "map" ]] && [[ $cword == 6 ]]; then
                                             COMPREPLY+=( $( compgen -W "key value kv all" -- "$cur" ) )

Thanks,
Quentin

#2 lore raw

Re: [PATCH v4 1/3] libbpf: support appending split BTF in btf__add_btf()

In reply to Josef Bacik
Quoted 45 lines · Josef Bacik

btf__add_btf() currently rejects split BTF sources with -ENOTSUP. This prevents merging types from multiple kernel module BTFs that are all split against the same vmlinux base.

Extend btf__add_btf() to handle split BTF sources by:

- Replacing the blanket -ENOTSUP with a validation that src and dst share the same base BTF pointer when both are split.

- Computing src_start_id from the source's base to distinguish base type ID references (which must remain unchanged) from split type IDs (which must be remapped to new positions in the destination).

- Using src_btf->nr_types instead of btf__type_cnt()-1 for the type count, which is correct for both split and non-split sources.

- Pre-emptively calling btf_ensure_modifiable() on the destination's base BTF to prevent a use-after-free: btf_rewrite_str() resolves strings via btf__str_by_offset(src) which may return pointers into the shared base's string data; btf__add_str(dst) then calls btf__find_str(base) which can trigger btf_ensure_modifiable(base), reallocating that string data and invalidating the pointer.

For non-split sources the behavior is identical: src_start_id is 1, the type_id < 1 guard is never true (VOID is already skipped), and the remapping formula reduces to the original.

---
 tools/lib/bpf/btf.c | 44 ++++++++++++++++++++++++++++++++++++--------
 1 file changed, 36 insertions(+), 8 deletions(-)
tools/lib/bpf/btf.c +1 -0
index 83fe79ffcb8f..654fb9ffa01f 100644
@@ -2004,24 +2004,47 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 {
        struct btf_pipe p = { .src = src_btf, .dst = btf };
        int data_sz, sz, cnt, i, err, old_strs_len;
+       __u32 src_start_id;
        __u32 *off;
        void *t;

tools/lib/bpf/btf.c int btf__add_btf(struct btf *btf, const struct btf *src_btf) · line 2016

Quoted 10 lines
tools/lib/bpf/btf.c +6 -3
 
-       /* appending split BTF isn't supported yet */
-       if (src_btf->base_btf)
-               return libbpf_err(-ENOTSUP);
+       /* When appending split BTF, the destination must share the same base
+        * BTF so that base type ID references remain valid.
+        */
+       if (src_btf->base_btf &&
+           src_btf->base_btf != btf->base_btf)
+               return libbpf_err(-EINVAL);

let's make it -EOPNOTSUPP (ENOTSUP is not a public error, so let's use the proper one), it can still be meaningfully supported if bases are different, we'd just need to add types from base and split parts, and shift all type IDs (but no use case right now)

Quoted 11 lines
tools/lib/bpf/btf.c +6 -0
+
+       src_start_id = src_btf->base_btf ? btf__type_cnt(src_btf->base_btf) : 1;
 
        /* deconstruct BTF, if necessary, and invalidate raw_data */
        if (btf_ensure_modifiable(btf))
                return libbpf_err(-ENOMEM);
 
+       /* If dst has a base BTF, ensure it is modifiable now so that
+        * btf__str_by_offset() on the base returns pointers from the
+        * strset copy rather than from raw_data.
+        *

tools/lib/bpf/btf.c int btf__add_btf(struct btf *btf, const struct btf *src_btf) · line 2037

Quoted 10 lines
tools/lib/bpf/btf.c +10 -0
+        * Without this, the first btf_rewrite_str() call that encounters
+        * a base string offset resolves it via btf__str_by_offset(src),
+        * which for split src returns a pointer into base->raw_data.
+        * That pointer is then passed to btf__add_str(dst), which calls
+        * btf__find_str(base) -> btf_ensure_modifiable(base).  The
+        * first btf_ensure_modifiable() call copies strings into a
+        * strset but then frees raw_data via btf_invalidate_raw_data(),
+        * leaving the pointer dangling before strset__find_str() uses
+        * it.
+        */

This is very surprising behavior, tbh... Base BTF should stay immutable, we never modify it. It seems like we switch to modifiable mode for base (implicitly) just to be able to find string by contents efficiently, which is just a side-effect.

Anyways, instead of this, can't we just detect that str_off we are trying to remap is coming from an immutable base BTF, and pass it through? That would be also more efficient, IMO.

Quoted 20 lines
tools/lib/bpf/btf.c +4 -3
+       if (btf->base_btf && btf_ensure_modifiable(btf->base_btf))
+               return libbpf_err(-ENOMEM);
+
        /* remember original strings section size if we have to roll back
         * partial strings section changes
         */
        old_strs_len = btf->hdr->str_len;
 
        data_sz = src_btf->hdr->type_len;
-       cnt = btf__type_cnt(src_btf) - 1;
+       cnt = src_btf->nr_types;
 
        /* pre-allocate enough memory for new types */
        t = btf_add_type_mem(btf, data_sz);
@@ -2074,11 +2097,16 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
                        if (!*type_id) /* nothing to do for VOID references */
                                continue;
 
-                       /* we haven't updated btf's type count yet, so
-                        * btf->start_id + btf->nr_types - 1 is the type ID offset we should

tools/lib/bpf/btf.c int btf__add_btf(struct btf *btf, const struct btf *src_btf) · line 2107

Quoted 10 lines
tools/lib/bpf/btf.c +7 -2
-                        * add to all newly added BTF types
+                       /* For split BTF sources, type IDs referencing the
+                        * base BTF (< src_start_id) remain unchanged since
+                        * dst shares the same base.  Only remap IDs in the
+                        * split portion.  For non-split sources src_start_id
+                        * is 1 so all IDs are remapped as before.
                         */
-                       *type_id += btf->start_id + btf->nr_types - 1;
+                       if (*type_id < src_start_id)
+                               continue;

see, just like here: you know type is from immutable base, and doesn't need remapping, we can do the same with string offsets

pw-bot: cr

Quoted 9 lines
tools/lib/bpf/btf.c +2 -1
+
+                       *type_id += btf->start_id + btf->nr_types - src_start_id;
                }
 
                /* go to next type data and type offset index entry */
--

2.53.0

#3 lore raw

Re: [PATCH v4 1/3] libbpf: support appending split BTF in btf__add_btf()

Quoted 128 lines · Andrii Nakryiko
Quoted 45 lines · Josef Bacik

btf__add_btf() currently rejects split BTF sources with -ENOTSUP. This prevents merging types from multiple kernel module BTFs that are all split against the same vmlinux base.

Extend btf__add_btf() to handle split BTF sources by:

- Replacing the blanket -ENOTSUP with a validation that src and dst share the same base BTF pointer when both are split.

- Computing src_start_id from the source's base to distinguish base type ID references (which must remain unchanged) from split type IDs (which must be remapped to new positions in the destination).

- Using src_btf->nr_types instead of btf__type_cnt()-1 for the type count, which is correct for both split and non-split sources.

- Pre-emptively calling btf_ensure_modifiable() on the destination's base BTF to prevent a use-after-free: btf_rewrite_str() resolves strings via btf__str_by_offset(src) which may return pointers into the shared base's string data; btf__add_str(dst) then calls btf__find_str(base) which can trigger btf_ensure_modifiable(base), reallocating that string data and invalidating the pointer.

For non-split sources the behavior is identical: src_start_id is 1, the type_id < 1 guard is never true (VOID is already skipped), and the remapping formula reduces to the original.

---
 tools/lib/bpf/btf.c | 44 ++++++++++++++++++++++++++++++++++++--------
 1 file changed, 36 insertions(+), 8 deletions(-)
tools/lib/bpf/btf.c +1 -0
index 83fe79ffcb8f..654fb9ffa01f 100644
@@ -2004,24 +2004,47 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 {
        struct btf_pipe p = { .src = src_btf, .dst = btf };
        int data_sz, sz, cnt, i, err, old_strs_len;
+       __u32 src_start_id;
        __u32 *off;
        void *t;

tools/lib/bpf/btf.c int btf__add_btf(struct btf *btf, const struct btf *src_btf) · line 2016

Quoted 10 lines
tools/lib/bpf/btf.c +6 -3
 
-       /* appending split BTF isn't supported yet */
-       if (src_btf->base_btf)
-               return libbpf_err(-ENOTSUP);
+       /* When appending split BTF, the destination must share the same base
+        * BTF so that base type ID references remain valid.
+        */
+       if (src_btf->base_btf &&
+           src_btf->base_btf != btf->base_btf)
+               return libbpf_err(-EINVAL);

let's make it -EOPNOTSUPP (ENOTSUP is not a public error, so let's use the proper one), it can still be meaningfully supported if bases are different, we'd just need to add types from base and split parts, and shift all type IDs (but no use case right now)

Quoted 11 lines
tools/lib/bpf/btf.c +6 -0
+
+       src_start_id = src_btf->base_btf ? btf__type_cnt(src_btf->base_btf) : 1;
 
        /* deconstruct BTF, if necessary, and invalidate raw_data */
        if (btf_ensure_modifiable(btf))
                return libbpf_err(-ENOMEM);
 
+       /* If dst has a base BTF, ensure it is modifiable now so that
+        * btf__str_by_offset() on the base returns pointers from the
+        * strset copy rather than from raw_data.
+        *

tools/lib/bpf/btf.c int btf__add_btf(struct btf *btf, const struct btf *src_btf) · line 2037

Quoted 10 lines
tools/lib/bpf/btf.c +10 -0
+        * Without this, the first btf_rewrite_str() call that encounters
+        * a base string offset resolves it via btf__str_by_offset(src),
+        * which for split src returns a pointer into base->raw_data.
+        * That pointer is then passed to btf__add_str(dst), which calls
+        * btf__find_str(base) -> btf_ensure_modifiable(base).  The
+        * first btf_ensure_modifiable() call copies strings into a
+        * strset but then frees raw_data via btf_invalidate_raw_data(),
+        * leaving the pointer dangling before strset__find_str() uses
+        * it.
+        */

This is very surprising behavior, tbh... Base BTF should stay immutable, we never modify it. It seems like we switch to modifiable mode for base (implicitly) just to be able to find string by contents efficiently, which is just a side-effect.

Anyways, instead of this, can't we just detect that str_off we are trying to remap is coming from an immutable base BTF, and pass it through? That would be also more efficient, IMO.

Quoted 20 lines
tools/lib/bpf/btf.c +4 -3
+       if (btf->base_btf && btf_ensure_modifiable(btf->base_btf))
+               return libbpf_err(-ENOMEM);
+
        /* remember original strings section size if we have to roll back
         * partial strings section changes
         */
        old_strs_len = btf->hdr->str_len;
 
        data_sz = src_btf->hdr->type_len;
-       cnt = btf__type_cnt(src_btf) - 1;
+       cnt = src_btf->nr_types;
 
        /* pre-allocate enough memory for new types */
        t = btf_add_type_mem(btf, data_sz);
@@ -2074,11 +2097,16 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
                        if (!*type_id) /* nothing to do for VOID references */
                                continue;
 
-                       /* we haven't updated btf's type count yet, so
-                        * btf->start_id + btf->nr_types - 1 is the type ID offset we should

tools/lib/bpf/btf.c int btf__add_btf(struct btf *btf, const struct btf *src_btf) · line 2107

Quoted 10 lines
tools/lib/bpf/btf.c +7 -2
-                        * add to all newly added BTF types
+                       /* For split BTF sources, type IDs referencing the
+                        * base BTF (< src_start_id) remain unchanged since
+                        * dst shares the same base.  Only remap IDs in the
+                        * split portion.  For non-split sources src_start_id
+                        * is 1 so all IDs are remapped as before.
                         */
-                       *type_id += btf->start_id + btf->nr_types - 1;
+                       if (*type_id < src_start_id)
+                               continue;

see, just like here: you know type is from immutable base, and doesn't need remapping, we can do the same with string offsets

Quoted 1 line

pw-bot: cr

oh, and let's please have a test for this case of adding split BTF with the same base?

Quoted 10 lines
Quoted 10 lines
tools/lib/bpf/btf.c +2 -1
+
+                       *type_id += btf->start_id + btf->nr_types - src_start_id;
                }
 
                /* go to next type data and type offset index entry */
--

2.53.0

#4 lore raw

Re: [PATCH v4 1/3] libbpf: support appending split BTF in btf__add_btf()

Quoted 131 lines · Andrii Nakryiko
Quoted 128 lines · Andrii Nakryiko
Quoted 45 lines · Josef Bacik

btf__add_btf() currently rejects split BTF sources with -ENOTSUP. This prevents merging types from multiple kernel module BTFs that are all split against the same vmlinux base.

Extend btf__add_btf() to handle split BTF sources by:

- Replacing the blanket -ENOTSUP with a validation that src and dst share the same base BTF pointer when both are split.

- Computing src_start_id from the source's base to distinguish base type ID references (which must remain unchanged) from split type IDs (which must be remapped to new positions in the destination).

- Using src_btf->nr_types instead of btf__type_cnt()-1 for the type count, which is correct for both split and non-split sources.

- Pre-emptively calling btf_ensure_modifiable() on the destination's base BTF to prevent a use-after-free: btf_rewrite_str() resolves strings via btf__str_by_offset(src) which may return pointers into the shared base's string data; btf__add_str(dst) then calls btf__find_str(base) which can trigger btf_ensure_modifiable(base), reallocating that string data and invalidating the pointer.

For non-split sources the behavior is identical: src_start_id is 1, the type_id < 1 guard is never true (VOID is already skipped), and the remapping formula reduces to the original.

---
 tools/lib/bpf/btf.c | 44 ++++++++++++++++++++++++++++++++++++--------
 1 file changed, 36 insertions(+), 8 deletions(-)
tools/lib/bpf/btf.c +1 -0
index 83fe79ffcb8f..654fb9ffa01f 100644
@@ -2004,24 +2004,47 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 {
        struct btf_pipe p = { .src = src_btf, .dst = btf };
        int data_sz, sz, cnt, i, err, old_strs_len;
+       __u32 src_start_id;
        __u32 *off;
        void *t;

tools/lib/bpf/btf.c int btf__add_btf(struct btf *btf, const struct btf *src_btf) · line 2016

Quoted 10 lines
tools/lib/bpf/btf.c +6 -3
 
-       /* appending split BTF isn't supported yet */
-       if (src_btf->base_btf)
-               return libbpf_err(-ENOTSUP);
+       /* When appending split BTF, the destination must share the same base
+        * BTF so that base type ID references remain valid.
+        */
+       if (src_btf->base_btf &&
+           src_btf->base_btf != btf->base_btf)
+               return libbpf_err(-EINVAL);

let's make it -EOPNOTSUPP (ENOTSUP is not a public error, so let's use the proper one), it can still be meaningfully supported if bases are different, we'd just need to add types from base and split parts, and shift all type IDs (but no use case right now)

Quoted 11 lines
tools/lib/bpf/btf.c +6 -0
+
+       src_start_id = src_btf->base_btf ? btf__type_cnt(src_btf->base_btf) : 1;
 
        /* deconstruct BTF, if necessary, and invalidate raw_data */
        if (btf_ensure_modifiable(btf))
                return libbpf_err(-ENOMEM);
 
+       /* If dst has a base BTF, ensure it is modifiable now so that
+        * btf__str_by_offset() on the base returns pointers from the
+        * strset copy rather than from raw_data.
+        *

tools/lib/bpf/btf.c int btf__add_btf(struct btf *btf, const struct btf *src_btf) · line 2037

Quoted 10 lines
tools/lib/bpf/btf.c +10 -0
+        * Without this, the first btf_rewrite_str() call that encounters
+        * a base string offset resolves it via btf__str_by_offset(src),
+        * which for split src returns a pointer into base->raw_data.
+        * That pointer is then passed to btf__add_str(dst), which calls
+        * btf__find_str(base) -> btf_ensure_modifiable(base).  The
+        * first btf_ensure_modifiable() call copies strings into a
+        * strset but then frees raw_data via btf_invalidate_raw_data(),
+        * leaving the pointer dangling before strset__find_str() uses
+        * it.
+        */

This is very surprising behavior, tbh... Base BTF should stay immutable, we never modify it. It seems like we switch to modifiable mode for base (implicitly) just to be able to find string by contents efficiently, which is just a side-effect.

Anyways, instead of this, can't we just detect that str_off we are trying to remap is coming from an immutable base BTF, and pass it through? That would be also more efficient, IMO.

Quoted 20 lines
tools/lib/bpf/btf.c +4 -3
+       if (btf->base_btf && btf_ensure_modifiable(btf->base_btf))
+               return libbpf_err(-ENOMEM);
+
        /* remember original strings section size if we have to roll back
         * partial strings section changes
         */
        old_strs_len = btf->hdr->str_len;
 
        data_sz = src_btf->hdr->type_len;
-       cnt = btf__type_cnt(src_btf) - 1;
+       cnt = src_btf->nr_types;
 
        /* pre-allocate enough memory for new types */
        t = btf_add_type_mem(btf, data_sz);
@@ -2074,11 +2097,16 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
                        if (!*type_id) /* nothing to do for VOID references */
                                continue;
 
-                       /* we haven't updated btf's type count yet, so
-                        * btf->start_id + btf->nr_types - 1 is the type ID offset we should

tools/lib/bpf/btf.c int btf__add_btf(struct btf *btf, const struct btf *src_btf) · line 2107

Quoted 10 lines
tools/lib/bpf/btf.c +7 -2
-                        * add to all newly added BTF types
+                       /* For split BTF sources, type IDs referencing the
+                        * base BTF (< src_start_id) remain unchanged since
+                        * dst shares the same base.  Only remap IDs in the
+                        * split portion.  For non-split sources src_start_id
+                        * is 1 so all IDs are remapped as before.
                         */
-                       *type_id += btf->start_id + btf->nr_types - 1;
+                       if (*type_id < src_start_id)
+                               continue;

see, just like here: you know type is from immutable base, and doesn't need remapping, we can do the same with string offsets

Quoted 5 lines
Quoted 1 line

pw-bot: cr

oh, and let's please have a test for this case of adding split BTF with the same base?

doh, brain farted, there is a self test in patch #3, ignore me

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
tools/lib/bpf/btf.c +2 -1
+
+                       *type_id += btf->start_id + btf->nr_types - src_start_id;
                }
 
                /* go to next type data and type offset index entry */
--

2.53.0