Skip to contentSkip to conversation

libbpf: Fix CO-RE relocations of unsigned 32-bit enum values

PATCH Superseded Yonghong Song 2 participants 4 messages Nov 19 – Nov 20

Subsystems (4)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Yonghong Song

author reviewer
#0 lore raw

v3 cover libbpf: Fix CO-RE relocations of unsigned 32-bit enum values

For a BPF_CORE_ENUMVAL_VALUE relocation, libbpf sign-extends the value of a 32-bit enumerator even if the enum is unsigned. A value in [0x80000000, 0xffffffff], e.g., DCACHE_PRIVATE of enum dentry_flags, then doesn't match the zero-extended value clang emits, and the whole object is refused.

Patch 1 extends the value as given by the kflag of the enum, which is the form clang emits. The kernel builds the same relo_core.c, so it gets the fix too.

GCC emits every 32-bit enum value sign-extended, since its __builtin_preserve_enum_value() returns int. So with the change above alone, GCC objects that load today would be refused. Patch 1 therefore also accepts the sign-extended form of an unsigned value, and patches the target value in the same form.

Patch 2 tests it, with clang and GCC.

Changelog:
v2 -> v3:
- v2: https://lore.kernel.org/bpf/[email protected]/
- Fold patch 3 into patch 1 and patch 4 into patch 2 to avoid test failures when bisecting.
v1 -> v2:
- v1: https://lore.kernel.org/bpf/[email protected]/
- Accept the sign-extended enum values emitted by GCC (patch 3), and make the new test work with GCC (patch 4).

Yonghong Song (2):
libbpf: Fix CO-RE relocations of unsigned 32-bit enum values selftests/bpf: Test CO-RE relocations of unsigned 32-bit enum values

 include/linux/btf.h                           |  5 +++
 tools/lib/bpf/relo_core.c                     | 31 ++++++++++++++++---
 .../selftests/bpf/prog_tests/core_reloc.c     |  2 ++
 .../selftests/bpf/progs/core_reloc_types.h    |  8 +++++
 .../bpf/progs/test_core_reloc_enumval.c       | 21 +++++++++++++
 5 files changed, 63 insertions(+), 4 deletions(-)
Signature · 1 line
2.53.1-Meta
author reviewer
#1 lore raw

PATCH v3 1/2 libbpf: Fix CO-RE relocations of unsigned 32-bit enum values

For a BPF_CORE_ENUMVAL_VALUE relocation, bpf_core_calc_enumval_relo() reads the value of a 32-bit enumerator from btf_enum->val, which is __s32, so the value is sign-extended to 64 bits even if the enum is unsigned. A value in [0x80000000, 0xffffffff] of an unsigned enum becomes 0xffffffff80000000 or above.

Clang emits such a value zero-extended in the ld_imm64 of the relocation, so the check of the original value fails, and the whole object is refused even if the target value is the same:

  enum dentry_flags { DCACHE_PRIVATE = 0x80000000 };
  out = bpf_core_enum_value(enum dentry_flags, DCACHE_PRIVATE);

libbpf: prog 'p': relo #0: unexpected insn #0 (LDIMM64) value: got

          2147483648, exp 18446744071562067968 -> 18446744071562067968

libbpf: prog 'p': relo #0: failed to patch insn #0: -EINVAL

And a target value in that range is patched sign-extended, which is wrong wherever the program uses it as a 64-bit value. The x86 vmlinux has 29 enums with such values, e.g., dentry_flags, init_ns_ino and e820_type.

Commit 89e7a42e3030 ("libbpf: Normalize enum values according to signedness") added btf_enum_value(), which extends the value as given by the kflag of the enum, for btf_dump and map fields, but not for CO-RE relocations, whose relo_core.c is also built into the kernel. Use it in bpf_core_calc_enumval_relo() as well, and add it to the kernel's include/linux/btf.h, so that the kernel gets the same fix.

GCC, however, sign-extends every 32-bit enum value, since its __builtin_preserve_enum_value() returns int. For the example above:

clang: r1 = 0x80000000 ll
gcc: lddw %r1,-2147483648

An object built by GCC is accepted today, because the original value is sign-extended too, and the change above alone would refuse it:

libbpf: prog 'p': relo #0: unexpected insn #0 (LDIMM64) value: got

          18446744071562067968, exp 2147483648 -> 2147483648

The two forms differ only for an unsigned value in [0x80000000, 0xffffffff]. For such a value, accept the sign-extended form as well. In that case, also sign-extend the target value, since GCC code uses the ld_imm64 as the int value widened to 64 bits.

An original value below 0x80000000 is the same in both forms, so the instruction doesn't show which compiler emitted it. The target value then gets the extension given by the kflag of the target enum, which is the form clang emits, as its builtin returns long. A target value that doesn't fit in 32 bits, e.g., from an enum64 in the target kernel, is patched unchanged.

Fixes 776281652ddc ("libbpf: Permit 64bit relocation value")
---
 include/linux/btf.h       |  5 +++++
 tools/lib/bpf/relo_core.c | 31 +++++++++++++++++++++++++++----
 2 files changed, 32 insertions(+), 4 deletions(-)
include/linux/btf.h +5 -0
index 87fad4076f0e..ceb27984df14 100644
@@ -525,6 +525,11 @@ static inline struct btf_enum *btf_enum(const struct btf_type *t)
 	return (struct btf_enum *)(t + 1);
 }
 
+static inline u64 btf_enum_value(const struct btf_type *t, const struct btf_enum *e)
+{
+	return btf_type_kflag(t) ? (s64)e->val : (u32)e->val;
+}
+
 static inline struct btf_enum64 *btf_enum64(const struct btf_type *t)
 {
 	return (struct btf_enum64 *)(t + 1);
tools/lib/bpf/relo_core.c +27 -4
index 2672623a4198..4891c8efa916 100644
@@ -861,6 +861,13 @@ static int bpf_core_calc_type_relo(const struct bpf_core_relo *relo,
 	return 0;
 }
 
+static __u64 bpf_core_enumval(const struct btf_type *t, int idx)
+{
+	if (btf_is_enum(t))
+		return btf_enum_value(t, btf_enum(t) + idx);
+	return btf_enum64_value(btf_enum64(t) + idx);
+}
+
 static int bpf_core_calc_enumval_relo(const struct bpf_core_relo *relo,
 				      const struct bpf_core_spec *spec,
 				      __u64 *val)
@@ -875,10 +882,7 @@ static int bpf_core_calc_enumval_relo(const struct bpf_core_relo *relo,
 		if (!spec)
 			return -EUCLEAN; /* request instruction poisoning */
 		t = btf_type_by_id(spec->btf, spec->spec[0].type_id);
-		if (btf_is_enum(t))
-			*val = btf_enum(t)[spec->spec[0].idx].val;
-		else
-			*val = btf_enum64_value(btf_enum64(t) + spec->spec[0].idx);
+		*val = bpf_core_enumval(t, spec->spec[0].idx);
 		break;
 	default:
 		return -EOPNOTSUPP;
@@ -1028,6 +1032,20 @@ static int insn_bytes_to_bpf_size(__u32 sz)
 	}
 }
 
+/*
+ * A 32-bit enum value is in the low 32 bits of the ld_imm64, extended as
+ * the compiler chooses: clang zero-extends an unsigned value, GCC
+ * sign-extends every value, as its __builtin_preserve_enum_value() returns
+ * int. Check whether the instruction has the sign-extended form of an
+ * unsigned value, so that the target value gets that form too.
+ */
+static bool bpf_core_enumval_is_sext(const struct bpf_core_relo *relo,
+				     __u64 imm, __u64 orig_val)
+{
+	return relo->kind == BPF_CORE_ENUMVAL_VALUE && orig_val <= UINT_MAX &&
+	       imm != orig_val && imm == (__u64)(__s32)orig_val;
+}
+
 /*
  * Patch relocatable BPF instruction.
  *
@@ -1144,6 +1162,11 @@ int bpf_core_patch_insn(const char *prog_name, struct bpf_insn *insn,
 			return bpf_core_poison_insn(prog_name, relo_idx, insn, insn_idx);
 
 		imm = (__u32)insn[0].imm | ((__u64)insn[1].imm << 32);
+		if (res->validate && bpf_core_enumval_is_sext(relo, imm, orig_val)) {
+			orig_val = imm;
+			if (new_val <= UINT_MAX)
+				new_val = (__s32)new_val;
+		}
 		if (res->validate && imm != orig_val) {
 			pr_warn("prog '%s': relo #%d: unexpected insn #%d (LDIMM64) value: got %llu, exp %llu -> %llu\n",
 				prog_name, relo_idx,
Signature · 1 line
2.53.1-Meta
#2 lore raw
TL;DR: Possible regression in libbpf CO-RE: on targets whose enum BTF has no kflag (pre-6.0 kernels), negative enumerators like -1 may be zero-extended to 4294967295 instead of sign-extended.…
Quoted 1 line

libbpf: Fix CO-RE relocations of unsigned 32-bit enum values

TL;DR: Possible regression in libbpf CO-RE: on targets whose enum BTF has no kflag (pre-6.0 kernels), negative enumerators like -1 may be zero-extended to 4294967295 instead of sign-extended. Selftest doesn't cover it.

Quoted 12 lines
include/linux/btf.h +5 -0
@@ -525,6 +525,11 @@ static inline struct btf_enum *btf_enum(const struct btf_type *t)
 	return (struct btf_enum *)(t + 1);
 }
 
+static inline u64 btf_enum_value(const struct btf_type *t, const struct btf_enum *e)
+{
+	return btf_type_kflag(t) ? (s64)e->val : (u32)e->val;
+}
+

[ ... ]

Quoted 3 lines
tools/lib/bpf/relo_core.c +0 -0

[ ... ]

Quoted 11 lines
tools/lib/bpf/relo_core.c +7 -0
+static __u64 bpf_core_enumval(const struct btf_type *t, int idx)
+{
+	if (btf_is_enum(t))
+		return btf_enum_value(t, btf_enum(t) + idx);
+	return btf_enum64_value(btf_enum64(t) + idx);
+}
+
 static int bpf_core_calc_enumval_relo(const struct bpf_core_relo *relo,
 				      const struct bpf_core_spec *spec,
 				      __u64 *val)

tools/lib/bpf/relo_core.c static int bpf_core_calc_enumval_relo(const struct bpf_core_relo *relo, · line 886

Quoted 9 lines
tools/lib/bpf/relo_core.c +1 -4
@@ -875,10 +882,7 @@ static int bpf_core_calc_enumval_relo(const struct bpf_core_relo *relo,
 		if (!spec)
 			return -EUCLEAN; /* request instruction poisoning */
 		t = btf_type_by_id(spec->btf, spec->spec[0].type_id);
-		if (btf_is_enum(t))
-			*val = btf_enum(t)[spec->spec[0].idx].val;
-		else
-			*val = btf_enum64_value(btf_enum64(t) + spec->spec[0].idx);
+		*val = bpf_core_enumval(t, spec->spec[0].idx);
 		break;

On the target side, bpf_core_enumval() treats kflag == 0 as unsigned:

    btf_enum_value(): return btf_type_kflag(t) ? (s64)e->val : (u32)e->val;

Is kflag == 0 really a safe indication of an unsigned enum here?

Before commit 6089fb325cf7 ("bpf: Add btf enum64 support"), btf_enum_check_meta() refused any enum with kflag set, so the vmlinux and module BTF of every pre-6.0 kernel has kflag == 0 on every enum, signed ones included. libbpf itself relies on this, since bpf_object__sanitize_btf() clears the enum kflag when !FEAT_BTF_ENUM64. Kernels built with pahole 1.22/1.23, which lib/Kconfig.debug still allows, have no signedness bit either.

Take a program built with clang >= 15 against a recent vmlinux.h, running on such a kernel:

    enum perf_event_state { ..., PERF_EVENT_STATE_OFF = -1, ... };
    long v = bpf_core_enum_value(enum perf_event_state,
                                 PERF_EVENT_STATE_OFF);

- Local BTF has kflag = 1, so orig_val = 0xffffffffffffffff and clang emits the same value in the ld_imm64.
- bpf_core_enumval_is_sext() returns false, because orig_val > UINT_MAX.
- Target BTF has kflag = 0, so new_val = (__u32)-1 = 0x00000000ffffffff.
- bpf_core_patch_insn() writes insn[0].imm = 0xffffffff and insn[1].imm = 0.

The program now gets 4294967295 instead of -1. Before this patch, new_val was btf_enum(t)[idx].val (an __s32) widened to __u64, i.e. sign-extended, which was the right value here. Comparisons such as
state == bpf_core_enum_value(...) (state widened to long) would stop matching, and other negative enumerators such as CPUHP_INVALID = -1 hit the same path. Kernel-side CO-RE (bpf_core_apply(), light skeletons) runs the same code and is affected when vmlinux BTF has no enum kflag.

The commit message says the target value "gets the extension given by the kflag of the target enum". That only works when the target BTF records signedness. Could the extension of a 32-bit target value come from the signedness of the local enum (or of orig_val) when the target enum has kflag == 0, as the GCC sign-extension case already does from the instruction?

The selftest added later in the series covers only unsigned values on a kflag-aware target, so it does not exercise this case.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/44640008097

author reviewer
#3 lore raw

PATCH v3 2/2 selftests/bpf: Test CO-RE relocations of unsigned 32-bit enum values

In reply to Yonghong Song

Test the extension of unsigned 32-bit enum values in CO-RE relocations from the previous patch.

test_core_reloc_enumval gets an optional local enum with values out of the range of int, i.e., an unsigned 4-byte enum, whose values clang emits zero-extended in the ld_imm64 of the relocations. The enumval target has the same enum with different values, 0x80000001 and 0xffffffff, which are expected zero-extended in the 64-bit outputs.

GCC's __builtin_preserve_enum_value() returns int, so storing the value of enum named_uint_enum into a long field sign-extends it, and uint_val1 would read back as 0xffffffff80000001. Convert the result to unsigned int for GCC to get the enumerator value. Clang still stores the 64-bit relocated value, so its upper 32 bits keep being checked.

Without the fix, the check of the original value fails, and the whole object is refused:

libbpf: prog 'test_core_enumval': relo #11: unexpected insn #53 (LDIMM64)

          value: got 2147483648, exp 18446744071562067968 -> 18446744071562067969

libbpf: prog 'test_core_enumval': relo #11: failed to patch insn #53: -EINVAL #122/71 core_reloc/enumval:FAIL

---
 .../selftests/bpf/prog_tests/core_reloc.c     |  2 ++
 .../selftests/bpf/progs/core_reloc_types.h    |  8 +++++++
 .../bpf/progs/test_core_reloc_enumval.c       | 21 +++++++++++++++++++
 3 files changed, 31 insertions(+)
tools/testing/selftests/bpf/prog_tests/core_reloc.c +2 -0
index 80432c4656b5..98b3c5fb1028 100644
@@ -904,6 +904,8 @@ static const struct core_reloc_test_case test_cases[] = {
 		.named_val2 = 2,
 		.anon_val1 = 0x10,
 		.anon_val2 = 0x20,
+		.uint_val1 = 0x80000001,
+		.uint_val2 = 0xffffffff,
 	}),
 	ENUMVAL_CASE(enumval___diff, {
 		.named_val1_exists = true,
tools/testing/selftests/bpf/progs/core_reloc_types.h +8 -0
index 5760ae015e09..6d758e74bd58 100644
@@ -1215,6 +1215,8 @@ struct core_reloc_enumval_output {
 	int named_val2;
 	int anon_val1;
 	int anon_val2;
+	long uint_val1;
+	long uint_val2;
 };
 
 struct core_reloc_enum64val_output {
@@ -1243,9 +1245,15 @@ typedef enum {
 	ANON_ENUM_VAL3 = 0x30,
 } anon_enum;
 
+enum named_uint_enum {
+	UINT_ENUM_VAL1 = 0x80000001,
+	UINT_ENUM_VAL2 = 0xffffffff,
+};
+
 struct core_reloc_enumval {
 	enum named_enum f1;
 	anon_enum f2;
+	enum named_uint_enum f3;
 };
 
 enum named_unsigned_enum64 {
tools/testing/selftests/bpf/progs/test_core_reloc_enumval.c +21 -0
index e7ef3dada2bf..8b8abbaaf533 100644
@@ -27,6 +27,12 @@ typedef enum {
 	ANON_ENUM_VAL3 = 0x30,
 } anon_enum;
 
+/* 4-byte enum with values out of the range of int, so it is unsigned */
+enum named_uint_enum {
+	UINT_ENUM_VAL1 = 0x80000000,
+	UINT_ENUM_VAL2 = 0xfffffff0,
+};
+
 struct core_reloc_enumval_output {
 	bool named_val1_exists;
 	bool named_val2_exists;
@@ -39,8 +45,17 @@ struct core_reloc_enumval_output {
 	int named_val2;
 	int anon_val1;
 	int anon_val2;
+	long uint_val1;
+	long uint_val2;
 };
 
+#ifdef __clang__
+#define UINT_ENUM_VALUE(v) bpf_core_enum_value(enum named_uint_enum, v)
+#else
+/* GCC's __builtin_preserve_enum_value() returns int */
+#define UINT_ENUM_VALUE(v) ((unsigned int)bpf_core_enum_value(enum named_uint_enum, v))
+#endif
+
 SEC("raw_tracepoint/sys_enter")
 int test_core_enumval(void *ctx)
 {
@@ -64,6 +79,12 @@ int test_core_enumval(void *ctx)
 	out->anon_val1 = bpf_core_enum_value(anon, ANON_ENUM_VAL1);
 	out->anon_val2 = bpf_core_enum_value(anon, ANON_ENUM_VAL2);
 	/* ANON_ENUM_VAL3 value is optional */
+
+	/* enum named_uint_enum is optional */
+	if (bpf_core_enum_value_exists(enum named_uint_enum, UINT_ENUM_VAL1))
+		out->uint_val1 = UINT_ENUM_VALUE(UINT_ENUM_VAL1);
+	if (bpf_core_enum_value_exists(enum named_uint_enum, UINT_ENUM_VAL2))
+		out->uint_val2 = UINT_ENUM_VALUE(UINT_ENUM_VAL2);
 #else
 	data.skip = true;
 #endif
Signature · 1 line
2.53.1-Meta