Skip to contentSkip to conversation

libbpf: fix LDX/STX/ST CO-RE relocation size adjustment logic

PATCH Applied Andrii Nakryiko 5 participants 8 messages Feb 7, 2025 – Feb 15, 2025

Subsystems (4)
lore
Share & tools
lore

Andrii Nakryiko

author maintainer
#0 lore raw

PATCH 1/2 libbpf: fix LDX/STX/ST CO-RE relocation size adjustment logic

Libbpf has a somewhat obscure feature of automatically adjusting the "size" of LDX/STX/ST instruction (memory store and load instructions), based on originally recorded access size (u8, u16, u32, or u64) and the actual size of the field on target kernel. This is meant to facilitate using BPF CO-RE on 32-bit architectures (pointers are always 64-bit in BPF, but host kernel's BTF will have it as 32-bit type), as well as generally supporting safe type changes (unsigned integer type changes can be transparently "relocated").

One issue that surfaced only now, 5 years after this logic was implemented, is how this all works when dealing with fields that are arrays. This isn't all that easy and straightforward to hit (see selftests that reproduce this condition), but one of sched_ext BPF programs did hit it with innocent looking loop.

Long story short, libbpf used to calculate entire array size, instead of making sure to only calculate array's element size. But it's the element that is loaded by LDX/STX/ST instructions (1, 2, 4, or 8 bytes), so that's what libbpf should check. This patch adjusts the logic for arrays and fixed the issue.

---
 tools/lib/bpf/relo_core.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)
tools/lib/bpf/relo_core.c +20 -4
index 7632e9d41827..2b83c98a1137 100644
@@ -683,7 +683,7 @@ static int bpf_core_calc_field_relo(const char *prog_name,
 {
 	const struct bpf_core_accessor *acc;
 	const struct btf_type *t;
-	__u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id;
+	__u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id, elem_id;
 	const struct btf_member *m;
 	const struct btf_type *mt;
 	bool bitfield;
@@ -706,8 +706,14 @@ static int bpf_core_calc_field_relo(const char *prog_name,
 	if (!acc->name) {
 		if (relo->kind == BPF_CORE_FIELD_BYTE_OFFSET) {
 			*val = spec->bit_offset / 8;
-			/* remember field size for load/store mem size */
-			sz = btf__resolve_size(spec->btf, acc->type_id);
+			/* remember field size for load/store mem size;
+			 * note, for arrays we care about individual element
+			 * sizes, not the overall array size
+			 */
+			t = skip_mods_and_typedefs(spec->btf, acc->type_id, &elem_id);
+			while (btf_is_array(t))
+				t = skip_mods_and_typedefs(spec->btf, btf_array(t)->type, &elem_id);
+			sz = btf__resolve_size(spec->btf, elem_id);
 			if (sz < 0)
 				return -EINVAL;
 			*field_sz = sz;
@@ -767,7 +773,17 @@ static int bpf_core_calc_field_relo(const char *prog_name,
 	case BPF_CORE_FIELD_BYTE_OFFSET:
 		*val = byte_off;
 		if (!bitfield) {
-			*field_sz = byte_sz;
+			/* remember field size for load/store mem size;
+			 * note, for arrays we care about individual element
+			 * sizes, not the overall array size
+			 */
+			t = skip_mods_and_typedefs(spec->btf, field_type_id, &elem_id);
+			while (btf_is_array(t))
+				t = skip_mods_and_typedefs(spec->btf, btf_array(t)->type, &elem_id);
+			sz = btf__resolve_size(spec->btf, elem_id);
+			if (sz < 0)
+				return -EINVAL;
+			*field_sz = sz;
 			*type_id = field_type_id;
 		}
 		break;
Signature · 1 line
2.43.5
author maintainer
#1 lore raw

PATCH 2/2 selftests/bpf: add test for LDX/STX/ST relocations over array field

Add a simple repro for the issue of miscalculating LDX/STX/ST CO-RE relocation size adjustment when the CO-RE relocation target type is an ARRAY.

We need to make sure that compiler generates LDX/STX/ST instruction with CO-RE relocation against entire ARRAY type, not ARRAY's element. With the code pattern in selftest, we get this:

      59:       61 71 00 00 00 00 00 00 w1 = *(u32 *)(r7 + 0x0)
                00000000000001d8:  CO-RE <byte_off> [5] struct core_reloc_arrays::a (0:0)

Where offset of int a[5] is embedded (through CO-RE relocation) into memory load instruction itself.

---
 tools/testing/selftests/bpf/prog_tests/core_reloc.c    |  6 ++++--
 ...f__core_reloc_arrays___err_bad_signed_arr_elem_sz.c |  3 +++
 tools/testing/selftests/bpf/progs/core_reloc_types.h   | 10 ++++++++++
 .../selftests/bpf/progs/test_core_reloc_arrays.c       |  5 +++++
 4 files changed, 22 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_arrays___err_bad_signed_arr_elem_sz.c
tools/testing/selftests/bpf/prog_tests/core_reloc.c +4 -2
index e10ea92c3fe2..08963c82f30b 100644
@@ -85,11 +85,11 @@ static int duration = 0;
 #define NESTING_ERR_CASE(name) {					\
 	NESTING_CASE_COMMON(name),					\
 	.fails = true,							\
-	.run_btfgen_fails = true,							\
+	.run_btfgen_fails = true,					\
 }
 
 #define ARRAYS_DATA(struct_name) STRUCT_TO_CHAR_PTR(struct_name) {	\
-	.a = { [2] = 1 },						\
+	.a = { [2] = 1, [3] = 11 },					\
 	.b = { [1] = { [2] = { [3] = 2 } } },				\
 	.c = { [1] = { .c =  3 } },					\
 	.d = { [0] = { [0] = { .d = 4 } } },				\
@@ -108,6 +108,7 @@ static int duration = 0;
 	.input_len = sizeof(struct core_reloc_##name),			\
 	.output = STRUCT_TO_CHAR_PTR(core_reloc_arrays_output) {	\
 		.a2   = 1,						\
+		.a3   = 12,						\
 		.b123 = 2,						\
 		.c1c  = 3,						\
 		.d00d = 4,						\
@@ -602,6 +603,7 @@ static const struct core_reloc_test_case test_cases[] = {
 	ARRAYS_ERR_CASE(arrays___err_non_array),
 	ARRAYS_ERR_CASE(arrays___err_wrong_val_type),
 	ARRAYS_ERR_CASE(arrays___err_bad_zero_sz_arr),
+	ARRAYS_ERR_CASE(arrays___err_bad_signed_arr_elem_sz),
 
 	/* enum/ptr/int handling scenarios */
 	PRIMITIVES_CASE(primitives),
tools/testing/selftests/bpf/progs/btf__core_reloc_arrays___err_bad_signed_arr_elem_sz.c new file +3 -0
new file mode 100644
index 000000000000..21a560427b10
@@ -0,0 +1,3 @@
+#include "core_reloc_types.h"
+
+void f(struct core_reloc_arrays___err_bad_signed_arr_elem_sz x) {}
tools/testing/selftests/bpf/progs/core_reloc_types.h +10 -0
index fd8e1b4c6762..5760ae015e09 100644
@@ -347,6 +347,7 @@ struct core_reloc_nesting___err_too_deep {
  */
 struct core_reloc_arrays_output {
 	int a2;
+	int a3;
 	char b123;
 	int c1c;
 	int d00d;
@@ -455,6 +456,15 @@ struct core_reloc_arrays___err_bad_zero_sz_arr {
 	struct core_reloc_arrays_substruct d[1][2];
 };
 
+struct core_reloc_arrays___err_bad_signed_arr_elem_sz {
+	/* int -> short (signed!): not supported case */
+	short a[5];
+	char b[2][3][4];
+	struct core_reloc_arrays_substruct c[3];
+	struct core_reloc_arrays_substruct d[1][2];
+	struct core_reloc_arrays_substruct f[][2];
+};
+
 /*
  * PRIMITIVES
  */
tools/testing/selftests/bpf/progs/test_core_reloc_arrays.c +5 -0
index 51b3f79df523..448403634eea 100644
@@ -15,6 +15,7 @@ struct {
 
 struct core_reloc_arrays_output {
 	int a2;
+	int a3;
 	char b123;
 	int c1c;
 	int d00d;
@@ -41,6 +42,7 @@ int test_core_arrays(void *ctx)
 {
 	struct core_reloc_arrays *in = (void *)&data.in;
 	struct core_reloc_arrays_output *out = (void *)&data.out;
+	int *a;
 
 	if (CORE_READ(&out->a2, &in->a[2]))
 		return 1;
@@ -53,6 +55,9 @@ int test_core_arrays(void *ctx)
 	if (CORE_READ(&out->f01c, &in->f[0][1].c))
 		return 1;
 
+	a = __builtin_preserve_access_index(({ in->a; }));
+	out->a3 = a[0] + a[1] + a[2] + a[3];
+
 	return 0;
 }
 
Signature · 1 line
2.43.5
#2 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: add test for LDX/STX/ST relocations over array field

Hi Andrii,

Quoted 115 lines · Andrii Nakryiko

Add a simple repro for the issue of miscalculating LDX/STX/ST CO-RE relocation size adjustment when the CO-RE relocation target type is an ARRAY.

We need to make sure that compiler generates LDX/STX/ST instruction with CO-RE relocation against entire ARRAY type, not ARRAY's element. With the code pattern in selftest, we get this:

       59:       61 71 00 00 00 00 00 00 w1 = *(u32 *)(r7 + 0x0)
                 00000000000001d8:  CO-RE <byte_off> [5] struct core_reloc_arrays::a (0:0)

Where offset of int a[5] is embedded (through CO-RE relocation) into memory load instruction itself.

---
  tools/testing/selftests/bpf/prog_tests/core_reloc.c    |  6 ++++--
  ...f__core_reloc_arrays___err_bad_signed_arr_elem_sz.c |  3 +++
  tools/testing/selftests/bpf/progs/core_reloc_types.h   | 10 ++++++++++
  .../selftests/bpf/progs/test_core_reloc_arrays.c       |  5 +++++
  4 files changed, 22 insertions(+), 2 deletions(-)

create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_arrays___err_bad_signed_arr_elem_sz.c

tools/testing/selftests/bpf/prog_tests/core_reloc.c +4 -2
index e10ea92c3fe2..08963c82f30b 100644
@@ -85,11 +85,11 @@ static int duration = 0;
  #define NESTING_ERR_CASE(name) {					\
  	NESTING_CASE_COMMON(name),					\
  	.fails = true,							\
-	.run_btfgen_fails = true,							\
+	.run_btfgen_fails = true,					\
  }
 
  #define ARRAYS_DATA(struct_name) STRUCT_TO_CHAR_PTR(struct_name) {	\
-	.a = { [2] = 1 },						\
+	.a = { [2] = 1, [3] = 11 },					\
  	.b = { [1] = { [2] = { [3] = 2 } } },				\
  	.c = { [1] = { .c =  3 } },					\
  	.d = { [0] = { [0] = { .d = 4 } } },				\
@@ -108,6 +108,7 @@ static int duration = 0;
  	.input_len = sizeof(struct core_reloc_##name),			\
  	.output = STRUCT_TO_CHAR_PTR(core_reloc_arrays_output) {	\
  		.a2   = 1,						\
+		.a3   = 12,						\
  		.b123 = 2,						\
  		.c1c  = 3,						\
  		.d00d = 4,						\
@@ -602,6 +603,7 @@ static const struct core_reloc_test_case test_cases[] = {
  	ARRAYS_ERR_CASE(arrays___err_non_array),
  	ARRAYS_ERR_CASE(arrays___err_wrong_val_type),
  	ARRAYS_ERR_CASE(arrays___err_bad_zero_sz_arr),
+	ARRAYS_ERR_CASE(arrays___err_bad_signed_arr_elem_sz),
 
  	/* enum/ptr/int handling scenarios */
  	PRIMITIVES_CASE(primitives),
tools/testing/selftests/bpf/progs/btf__core_reloc_arrays___err_bad_signed_arr_elem_sz.c new file +3 -0
new file mode 100644
index 000000000000..21a560427b10
@@ -0,0 +1,3 @@
+#include "core_reloc_types.h"
+
+void f(struct core_reloc_arrays___err_bad_signed_arr_elem_sz x) {}
tools/testing/selftests/bpf/progs/core_reloc_types.h +10 -0
index fd8e1b4c6762..5760ae015e09 100644
@@ -347,6 +347,7 @@ struct core_reloc_nesting___err_too_deep {
   */
  struct core_reloc_arrays_output {
  	int a2;
+	int a3;
  	char b123;
  	int c1c;
  	int d00d;
@@ -455,6 +456,15 @@ struct core_reloc_arrays___err_bad_zero_sz_arr {
  	struct core_reloc_arrays_substruct d[1][2];
  };
 
+struct core_reloc_arrays___err_bad_signed_arr_elem_sz {
+	/* int -> short (signed!): not supported case */
+	short a[5];
+	char b[2][3][4];
+	struct core_reloc_arrays_substruct c[3];
+	struct core_reloc_arrays_substruct d[1][2];
+	struct core_reloc_arrays_substruct f[][2];
+};
+
  /*
   * PRIMITIVES
   */
tools/testing/selftests/bpf/progs/test_core_reloc_arrays.c +2 -0
index 51b3f79df523..448403634eea 100644
@@ -15,6 +15,7 @@ struct {
 
  struct core_reloc_arrays_output {
  	int a2;
+	int a3;
  	char b123;
  	int c1c;
  	int d00d;
@@ -41,6 +42,7 @@ int test_core_arrays(void *ctx)
  {
  	struct core_reloc_arrays *in = (void *)&data.in;
  	struct core_reloc_arrays_output *out = (void *)&data.out;
+	int *a;
 
  	if (CORE_READ(&out->a2, &in->a[2]))
  		return 1;

tools/testing/selftests/bpf/progs/test_core_reloc_arrays.c int test_core_arrays(void *ctx) · line 59

Quoted 5 lines
tools/testing/selftests/bpf/progs/test_core_reloc_arrays.c +2 -0
@@ -53,6 +55,9 @@ int test_core_arrays(void *ctx)
  	if (CORE_READ(&out->f01c, &in->f[0][1].c))
  		return 1;
 
+	a = __builtin_preserve_access_index(({ in->a; }));
+	out->a3 = a[0] + a[1] + a[2] + a[3];

Just to try to understand what seems to be the expectation from the compiler and CO-RE in this case.
Do you expect that all those a[n] accesses would be generating CO-RE relocations assuming the size for the elements in in->a ?

Quoted 4 lines
tools/testing/selftests/bpf/progs/test_core_reloc_arrays.c +1 -0
+
  	return 0;
  }
 
#3 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: add test for LDX/STX/ST relocations over array field

Quoted 122 lines · Cupertino Miranda

Hi Andrii,

Quoted 118 lines · Andrii Nakryiko

Add a simple repro for the issue of miscalculating LDX/STX/ST CO-RE relocation size adjustment when the CO-RE relocation target type is an ARRAY.

We need to make sure that compiler generates LDX/STX/ST instruction with CO-RE relocation against entire ARRAY type, not ARRAY's element. With the code pattern in selftest, we get this:

       59:       61 71 00 00 00 00 00 00 w1 = *(u32 *)(r7 + 0x0)
                 00000000000001d8:  CO-RE <byte_off> [5] struct core_reloc_arrays::a (0:0)

Where offset of int a[5] is embedded (through CO-RE relocation) into memory load instruction itself.

---
  tools/testing/selftests/bpf/prog_tests/core_reloc.c    |  6 ++++--
  ...f__core_reloc_arrays___err_bad_signed_arr_elem_sz.c |  3 +++
  tools/testing/selftests/bpf/progs/core_reloc_types.h   | 10 ++++++++++
  .../selftests/bpf/progs/test_core_reloc_arrays.c       |  5 +++++
  4 files changed, 22 insertions(+), 2 deletions(-)

create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_arrays___err_bad_signed_arr_elem_sz.c

tools/testing/selftests/bpf/prog_tests/core_reloc.c +4 -2
index e10ea92c3fe2..08963c82f30b 100644
@@ -85,11 +85,11 @@ static int duration = 0;
  #define NESTING_ERR_CASE(name) {                                    \
      NESTING_CASE_COMMON(name),                                      \
      .fails = true,                                                  \
-     .run_btfgen_fails = true,                                                       \
+     .run_btfgen_fails = true,                                       \
  }
 
  #define ARRAYS_DATA(struct_name) STRUCT_TO_CHAR_PTR(struct_name) {  \
-     .a = { [2] = 1 },                                               \
+     .a = { [2] = 1, [3] = 11 },                                     \
      .b = { [1] = { [2] = { [3] = 2 } } },                           \
      .c = { [1] = { .c =  3 } },                                     \
      .d = { [0] = { [0] = { .d = 4 } } },                            \
@@ -108,6 +108,7 @@ static int duration = 0;
      .input_len = sizeof(struct core_reloc_##name),                  \
      .output = STRUCT_TO_CHAR_PTR(core_reloc_arrays_output) {        \
              .a2   = 1,                                              \
+             .a3   = 12,                                             \
              .b123 = 2,                                              \
              .c1c  = 3,                                              \
              .d00d = 4,                                              \
@@ -602,6 +603,7 @@ static const struct core_reloc_test_case test_cases[] = {
      ARRAYS_ERR_CASE(arrays___err_non_array),
      ARRAYS_ERR_CASE(arrays___err_wrong_val_type),
      ARRAYS_ERR_CASE(arrays___err_bad_zero_sz_arr),
+     ARRAYS_ERR_CASE(arrays___err_bad_signed_arr_elem_sz),
 
      /* enum/ptr/int handling scenarios */
      PRIMITIVES_CASE(primitives),
tools/testing/selftests/bpf/progs/btf__core_reloc_arrays___err_bad_signed_arr_elem_sz.c new file +3 -0
new file mode 100644
index 000000000000..21a560427b10
@@ -0,0 +1,3 @@
+#include "core_reloc_types.h"
+
+void f(struct core_reloc_arrays___err_bad_signed_arr_elem_sz x) {}
tools/testing/selftests/bpf/progs/core_reloc_types.h +10 -0
index fd8e1b4c6762..5760ae015e09 100644
@@ -347,6 +347,7 @@ struct core_reloc_nesting___err_too_deep {
   */
  struct core_reloc_arrays_output {
      int a2;
+     int a3;
      char b123;
      int c1c;
      int d00d;
@@ -455,6 +456,15 @@ struct core_reloc_arrays___err_bad_zero_sz_arr {
      struct core_reloc_arrays_substruct d[1][2];
  };
 
+struct core_reloc_arrays___err_bad_signed_arr_elem_sz {
+     /* int -> short (signed!): not supported case */
+     short a[5];
+     char b[2][3][4];
+     struct core_reloc_arrays_substruct c[3];
+     struct core_reloc_arrays_substruct d[1][2];
+     struct core_reloc_arrays_substruct f[][2];
+};
+
  /*
   * PRIMITIVES
   */
tools/testing/selftests/bpf/progs/test_core_reloc_arrays.c +2 -0
index 51b3f79df523..448403634eea 100644
@@ -15,6 +15,7 @@ struct {
 
  struct core_reloc_arrays_output {
      int a2;
+     int a3;
      char b123;
      int c1c;
      int d00d;
@@ -41,6 +42,7 @@ int test_core_arrays(void *ctx)
  {
      struct core_reloc_arrays *in = (void *)&data.in;
      struct core_reloc_arrays_output *out = (void *)&data.out;
+     int *a;
 
      if (CORE_READ(&out->a2, &in->a[2]))
              return 1;
@@ -53,6 +55,9 @@ int test_core_arrays(void *ctx)
      if (CORE_READ(&out->f01c, &in->f[0][1].c))
              return 1;
 
Quoted 7 lines

tools/testing/selftests/bpf/progs/test_core_reloc_arrays.c int test_core_arrays(void *ctx) · line 59

Quoted 2 lines
tools/testing/selftests/bpf/progs/test_core_reloc_arrays.c +2 -0
+     a = __builtin_preserve_access_index(({ in->a; }));
+     out->a3 = a[0] + a[1] + a[2] + a[3];

Just to try to understand what seems to be the expectation from the compiler and CO-RE in this case.
Do you expect that all those a[n] accesses would be generating CO-RE relocations assuming the size for the elements in in->a ?

Well, I only care to get LDX instruction with associated in->a CO-RE relocation. This is what Clang currently generates for this piece of code. You can see that it combines both LDX+CO-RE relo for a[0], and then non-CO-RE relocated LDX for a[1], a[2], a[3], where the base was relocated with CO-RE a bit earlier.

      44:       18 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r7 = 0x0 ll
                0000000000000160:  R_BPF_64_64  data

...

      55:       b7 01 00 00 00 00 00 00 r1 = 0x0
                00000000000001b8:  CO-RE <byte_off> [5] struct
core_reloc_arrays::a (0:0)
      56:       18 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r2 = 0x0 ll
                00000000000001c0:  R_BPF_64_64  data
      58:       0f 12 00 00 00 00 00 00 r2 += r1
      59:       61 71 00 00 00 00 00 00 w1 = *(u32 *)(r7 + 0x0)
                00000000000001d8:  CO-RE <byte_off> [5] struct
core_reloc_arrays::a (0:0)
      60:       61 23 04 00 00 00 00 00 w3 = *(u32 *)(r2 + 0x4)
      61:       0c 13 00 00 00 00 00 00 w3 += w1
      62:       61 21 08 00 00 00 00 00 w1 = *(u32 *)(r2 + 0x8)
      63:       0c 13 00 00 00 00 00 00 w3 += w1
      64:       61 21 0c 00 00 00 00 00 w1 = *(u32 *)(r2 + 0xc)
      65:       0c 13 00 00 00 00 00 00 w3 += w1
      66:       63 37 04 01 00 00 00 00 *(u32 *)(r7 + 0x104) = w3

Clang might change code generation pattern in the future, of course, but at least as of right now I know I did test this logic :) Ideally I'd be able to generate embedded asm with CO-RE relocation, but I'm not sure that's supported today.

Quoted 5 lines
Quoted 4 lines

+

      return 0;

}

#4 lore raw

Re: [PATCH bpf-next 2/2] selftests/bpf: add test for LDX/STX/ST relocations over array field

Quoted 160 lines · Andrii Nakryiko
Quoted 122 lines · Cupertino Miranda

Hi Andrii,

Quoted 118 lines · Andrii Nakryiko

Add a simple repro for the issue of miscalculating LDX/STX/ST CO-RE relocation size adjustment when the CO-RE relocation target type is an ARRAY.

We need to make sure that compiler generates LDX/STX/ST instruction with CO-RE relocation against entire ARRAY type, not ARRAY's element. With the code pattern in selftest, we get this:

        59:       61 71 00 00 00 00 00 00 w1 = *(u32 *)(r7 + 0x0)
                  00000000000001d8:  CO-RE <byte_off> [5] struct core_reloc_arrays::a (0:0)

Where offset of int a[5] is embedded (through CO-RE relocation) into memory load instruction itself.

---
   tools/testing/selftests/bpf/prog_tests/core_reloc.c    |  6 ++++--
   ...f__core_reloc_arrays___err_bad_signed_arr_elem_sz.c |  3 +++
   tools/testing/selftests/bpf/progs/core_reloc_types.h   | 10 ++++++++++
   .../selftests/bpf/progs/test_core_reloc_arrays.c       |  5 +++++
   4 files changed, 22 insertions(+), 2 deletions(-)

create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_arrays___err_bad_signed_arr_elem_sz.c

tools/testing/selftests/bpf/prog_tests/core_reloc.c +4 -2
index e10ea92c3fe2..08963c82f30b 100644
@@ -85,11 +85,11 @@ static int duration = 0;
   #define NESTING_ERR_CASE(name) {                                    \
       NESTING_CASE_COMMON(name),                                      \
       .fails = true,                                                  \
-     .run_btfgen_fails = true,                                                       \
+     .run_btfgen_fails = true,                                       \
   }
 
   #define ARRAYS_DATA(struct_name) STRUCT_TO_CHAR_PTR(struct_name) {  \
-     .a = { [2] = 1 },                                               \
+     .a = { [2] = 1, [3] = 11 },                                     \
       .b = { [1] = { [2] = { [3] = 2 } } },                           \
       .c = { [1] = { .c =  3 } },                                     \
       .d = { [0] = { [0] = { .d = 4 } } },                            \
@@ -108,6 +108,7 @@ static int duration = 0;
       .input_len = sizeof(struct core_reloc_##name),                  \
       .output = STRUCT_TO_CHAR_PTR(core_reloc_arrays_output) {        \
               .a2   = 1,                                              \
+             .a3   = 12,                                             \
               .b123 = 2,                                              \
               .c1c  = 3,                                              \
               .d00d = 4,                                              \
@@ -602,6 +603,7 @@ static const struct core_reloc_test_case test_cases[] = {
       ARRAYS_ERR_CASE(arrays___err_non_array),
       ARRAYS_ERR_CASE(arrays___err_wrong_val_type),
       ARRAYS_ERR_CASE(arrays___err_bad_zero_sz_arr),
+     ARRAYS_ERR_CASE(arrays___err_bad_signed_arr_elem_sz),
 
       /* enum/ptr/int handling scenarios */
       PRIMITIVES_CASE(primitives),
tools/testing/selftests/bpf/progs/btf__core_reloc_arrays___err_bad_signed_arr_elem_sz.c new file +3 -0
new file mode 100644
index 000000000000..21a560427b10
@@ -0,0 +1,3 @@
+#include "core_reloc_types.h"
+
+void f(struct core_reloc_arrays___err_bad_signed_arr_elem_sz x) {}
tools/testing/selftests/bpf/progs/core_reloc_types.h +10 -0
index fd8e1b4c6762..5760ae015e09 100644
@@ -347,6 +347,7 @@ struct core_reloc_nesting___err_too_deep {
    */
   struct core_reloc_arrays_output {
       int a2;
+     int a3;
       char b123;
       int c1c;
       int d00d;
@@ -455,6 +456,15 @@ struct core_reloc_arrays___err_bad_zero_sz_arr {
       struct core_reloc_arrays_substruct d[1][2];
   };
 
+struct core_reloc_arrays___err_bad_signed_arr_elem_sz {
+     /* int -> short (signed!): not supported case */
+     short a[5];
+     char b[2][3][4];
+     struct core_reloc_arrays_substruct c[3];
+     struct core_reloc_arrays_substruct d[1][2];
+     struct core_reloc_arrays_substruct f[][2];
+};
+
   /*
    * PRIMITIVES
    */
tools/testing/selftests/bpf/progs/test_core_reloc_arrays.c +2 -0
index 51b3f79df523..448403634eea 100644
@@ -15,6 +15,7 @@ struct {
 
   struct core_reloc_arrays_output {
       int a2;
+     int a3;
       char b123;
       int c1c;
       int d00d;
@@ -41,6 +42,7 @@ int test_core_arrays(void *ctx)
   {
       struct core_reloc_arrays *in = (void *)&data.in;
       struct core_reloc_arrays_output *out = (void *)&data.out;
+     int *a;
 
       if (CORE_READ(&out->a2, &in->a[2]))
               return 1;
@@ -53,6 +55,9 @@ int test_core_arrays(void *ctx)
       if (CORE_READ(&out->f01c, &in->f[0][1].c))
               return 1;
 
Quoted 7 lines

tools/testing/selftests/bpf/progs/test_core_reloc_arrays.c int test_core_arrays(void *ctx) · line 59

Quoted 2 lines
tools/testing/selftests/bpf/progs/test_core_reloc_arrays.c +2 -0
+     a = __builtin_preserve_access_index(({ in->a; }));
+     out->a3 = a[0] + a[1] + a[2] + a[3];

Just to try to understand what seems to be the expectation from the compiler and CO-RE in this case.
Do you expect that all those a[n] accesses would be generating CO-RE relocations assuming the size for the elements in in->a ?

Well, I only care to get LDX instruction with associated in->a CO-RE relocation. This is what Clang currently generates for this piece of code. You can see that it combines both LDX+CO-RE relo for a[0], and then non-CO-RE relocated LDX for a[1], a[2], a[3], where the base was relocated with CO-RE a bit earlier.

       44:       18 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r7 = 0x0 ll
                 0000000000000160:  R_BPF_64_64  data

...

       55:       b7 01 00 00 00 00 00 00 r1 = 0x0
                 00000000000001b8:  CO-RE <byte_off> [5] struct
core_reloc_arrays::a (0:0)
       56:       18 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r2 = 0x0 ll
                 00000000000001c0:  R_BPF_64_64  data
       58:       0f 12 00 00 00 00 00 00 r2 += r1
       59:       61 71 00 00 00 00 00 00 w1 = *(u32 *)(r7 + 0x0)
                 00000000000001d8:  CO-RE <byte_off> [5] struct
core_reloc_arrays::a (0:0)
       60:       61 23 04 00 00 00 00 00 w3 = *(u32 *)(r2 + 0x4)
       61:       0c 13 00 00 00 00 00 00 w3 += w1
       62:       61 21 08 00 00 00 00 00 w1 = *(u32 *)(r2 + 0x8)
       63:       0c 13 00 00 00 00 00 00 w3 += w1
       64:       61 21 0c 00 00 00 00 00 w1 = *(u32 *)(r2 + 0xc)
       65:       0c 13 00 00 00 00 00 00 w3 += w1
       66:       63 37 04 01 00 00 00 00 *(u32 *)(r7 + 0x104) = w3
Quoted 4 lines

Clang might change code generation pattern in the future, of course, but at least as of right now I know I did test this logic :) Ideally I'd be able to generate embedded asm with CO-RE relocation, but I'm not sure that's supported today.

Ok, good! I just miss read it. :)
Thanks!

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

+

       return 0;

}

maintainer Acked-by on v1 1/2
#5 lore raw
In reply to Andrii Nakryiko
Quoted 21 lines · Andrii Nakryiko

Libbpf has a somewhat obscure feature of automatically adjusting the "size" of LDX/STX/ST instruction (memory store and load instructions), based on originally recorded access size (u8, u16, u32, or u64) and the actual size of the field on target kernel. This is meant to facilitate using BPF CO-RE on 32-bit architectures (pointers are always 64-bit in BPF, but host kernel's BTF will have it as 32-bit type), as well as generally supporting safe type changes (unsigned integer type changes can be transparently "relocated").

One issue that surfaced only now, 5 years after this logic was implemented, is how this all works when dealing with fields that are arrays. This isn't all that easy and straightforward to hit (see selftests that reproduce this condition), but one of sched_ext BPF programs did hit it with innocent looking loop.

Long story short, libbpf used to calculate entire array size, instead of making sure to only calculate array's element size. But it's the element that is loaded by LDX/STX/ST instructions (1, 2, 4, or 8 bytes), so that's what libbpf should check. This patch adjusts the logic for arrays and fixed the issue.

Quoted 3 lines

---

Do I understand correctly, that for nested arrays relocation size would be resolved to the innermost element size?
To allow e.g.:

    struct { int a[2][3]; }
    ...
    int *a = __builtin_preserve_access_index(({ in->a; }));
    a[0] = 42;

With a justification that nothing useful could be done with 'int **a' type when dimensions are not known?
I guess this makes sense.

Quoted 20 lines
 tools/lib/bpf/relo_core.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)
tools/lib/bpf/relo_core.c +1 -1
index 7632e9d41827..2b83c98a1137 100644
@@ -683,7 +683,7 @@ static int bpf_core_calc_field_relo(const char *prog_name,
 {
 	const struct bpf_core_accessor *acc;
 	const struct btf_type *t;
-	__u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id;
+	__u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id, elem_id;
 	const struct btf_member *m;
 	const struct btf_type *mt;
 	bool bitfield;
@@ -706,8 +706,14 @@ static int bpf_core_calc_field_relo(const char *prog_name,
 	if (!acc->name) {
 		if (relo->kind == BPF_CORE_FIELD_BYTE_OFFSET) {
 			*val = spec->bit_offset / 8;

tools/lib/bpf/relo_core.c static int bpf_core_calc_field_relo(const char *prog_name, · line 716

Quoted 10 lines
tools/lib/bpf/relo_core.c +8 -2
-			/* remember field size for load/store mem size */
-			sz = btf__resolve_size(spec->btf, acc->type_id);
+			/* remember field size for load/store mem size;
+			 * note, for arrays we care about individual element
+			 * sizes, not the overall array size
+			 */
+			t = skip_mods_and_typedefs(spec->btf, acc->type_id, &elem_id);
+			while (btf_is_array(t))
+				t = skip_mods_and_typedefs(spec->btf, btf_array(t)->type, &elem_id);
+			sz = btf__resolve_size(spec->btf, elem_id);

Nit: while trying to figure out what this change is about
I commented out the above hunk and this did not trigger any test failures.

[...]

#6 lore raw
Quoted 31 lines · Eduard Zingerman
Quoted 21 lines · Andrii Nakryiko

Libbpf has a somewhat obscure feature of automatically adjusting the "size" of LDX/STX/ST instruction (memory store and load instructions), based on originally recorded access size (u8, u16, u32, or u64) and the actual size of the field on target kernel. This is meant to facilitate using BPF CO-RE on 32-bit architectures (pointers are always 64-bit in BPF, but host kernel's BTF will have it as 32-bit type), as well as generally supporting safe type changes (unsigned integer type changes can be transparently "relocated").

One issue that surfaced only now, 5 years after this logic was implemented, is how this all works when dealing with fields that are arrays. This isn't all that easy and straightforward to hit (see selftests that reproduce this condition), but one of sched_ext BPF programs did hit it with innocent looking loop.

Long story short, libbpf used to calculate entire array size, instead of making sure to only calculate array's element size. But it's the element that is loaded by LDX/STX/ST instructions (1, 2, 4, or 8 bytes), so that's what libbpf should check. This patch adjusts the logic for arrays and fixed the issue.

Quoted 3 lines

---

Do I understand correctly, that for nested arrays relocation size would be resolved to the innermost element size?
To allow e.g.:

Quoted 8 lines
    struct { int a[2][3]; }
    ...
    int *a = __builtin_preserve_access_index(({ in->a; }));
    a[0] = 42;

With a justification that nothing useful could be done with 'int **a' type when dimensions are not known?
I guess this makes sense.

Known or not, a multi-dimensional array at the lowest level is still an array of elements, and it is the elements that will be read (up to u64), so that's why I'm flattening the array and getting to the actual item.

Quoted 28 lines
Quoted 25 lines
 tools/lib/bpf/relo_core.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)
tools/lib/bpf/relo_core.c +4 -3
index 7632e9d41827..2b83c98a1137 100644
@@ -683,7 +683,7 @@ static int bpf_core_calc_field_relo(const char *prog_name,
 {
      const struct bpf_core_accessor *acc;
      const struct btf_type *t;
-     __u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id;
+     __u32 byte_off, byte_sz, bit_off, bit_sz, field_type_id, elem_id;
      const struct btf_member *m;
      const struct btf_type *mt;
      bool bitfield;
@@ -706,8 +706,14 @@ static int bpf_core_calc_field_relo(const char *prog_name,
      if (!acc->name) {
              if (relo->kind == BPF_CORE_FIELD_BYTE_OFFSET) {
                      *val = spec->bit_offset / 8;
-                     /* remember field size for load/store mem size */
-                     sz = btf__resolve_size(spec->btf, acc->type_id);
+                     /* remember field size for load/store mem size;
+                      * note, for arrays we care about individual element
+                      * sizes, not the overall array size
Quoted 8 lines

tools/lib/bpf/relo_core.c static int bpf_core_calc_field_relo(const char *prog_name, · line 716

Quoted 5 lines
tools/lib/bpf/relo_core.c +5 -0
+                      */
+                     t = skip_mods_and_typedefs(spec->btf, acc->type_id, &elem_id);
+                     while (btf_is_array(t))
+                             t = skip_mods_and_typedefs(spec->btf, btf_array(t)->type, &elem_id);
+                     sz = btf__resolve_size(spec->btf, elem_id);

Nit: while trying to figure out what this change is about
I commented out the above hunk and this did not trigger any test failures.

I don't remember exactly under which conditions we'll have this branch, something about array element access. But this whole logic has to stay in sync with non-array-element CO-RE relocation.

Quoted 3 lines

[...]

#7 lore raw
In reply to Andrii Nakryiko
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 10 lines · On Thu, 6 Feb 2025 17:48:08 -0800 you wrote:

Libbpf has a somewhat obscure feature of automatically adjusting the "size" of LDX/STX/ST instruction (memory store and load instructions), based on originally recorded access size (u8, u16, u32, or u64) and the actual size of the field on target kernel. This is meant to facilitate using BPF CO-RE on 32-bit architectures (pointers are always 64-bit in BPF, but host kernel's BTF will have it as 32-bit type), as well as generally supporting safe type changes (unsigned integer type changes can be transparently "relocated").

[...]

Here is the summary with links:
- [bpf-next,1/2] libbpf: fix LDX/STX/ST CO-RE relocation size adjustment logic

    https://git.kernel.org/bpf/bpf-next/c/06096d19ee38
  - [bpf-next,2/2] selftests/bpf: add test for LDX/STX/ST relocations over array field
    https://git.kernel.org/bpf/bpf-next/c/4eb93fea5919

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html