Skip to contentSkip to conversation

btf_dump fixes for s390

PATCH Superseded Ilya Leoshkevich 2 participants 11 messages Oct 12, 2021 – Oct 20, 2021

Subsystems (4)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover btf_dump fixes for s390

Hi,

This series along with [1] and [2] fixes all the failures in the btf_dump testsuite currently present on s390, in particular:

* [1] fixes intermittent build bug causing "failed to encode tag ..."
* error messages.
* [2] fixes missing VAR entries on s390.
* Patch 1 disables Intel-specific code in a testcase.
* Patch 2 fixes an endianness-related bug.
* Patch 3 fixes an alignment-related bug.

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/

Best regards,
Ilya

Ilya Leoshkevich (3):
selftests/bpf: Use cpu_number only on arches that have it
libbpf: Fix dumping big-endian bitfields
libbpf: Fix dumping __int128

 tools/lib/bpf/btf_dump.c                          | 12 ++++++++----
 tools/testing/selftests/bpf/prog_tests/btf_dump.c |  2 ++
 2 files changed, 10 insertions(+), 4 deletions(-)
Signature · 1 line
2.31.1
#5 lore raw

PATCH 2/3 libbpf: Fix dumping big-endian bitfields

On big-endian arches not only bytes, but also bits are numbered in reverse order (see e.g. S/390 ELF ABI Supplement, but this is also true for other big-endian arches as well).

---
 tools/lib/bpf/btf_dump.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tools/lib/bpf/btf_dump.c +2 -1
index ad6df97295ae..ab45771d0cb4 100644
@@ -1577,14 +1577,15 @@ static int btf_dump_get_bitfield_value(struct btf_dump *d,
 	/* Bitfield value retrieval is done in two steps; first relevant bytes are
 	 * stored in num, then we left/right shift num to eliminate irrelevant bits.
 	 */
-	nr_copy_bits = bit_sz + bits_offset;
 	nr_copy_bytes = t->size;
 #if __BYTE_ORDER == __LITTLE_ENDIAN
 	for (i = nr_copy_bytes - 1; i >= 0; i--)
 		num = num * 256 + bytes[i];
+	nr_copy_bits = bit_sz + bits_offset;
 #elif __BYTE_ORDER == __BIG_ENDIAN
 	for (i = 0; i < nr_copy_bytes; i++)
 		num = num * 256 + bytes[i];
+	nr_copy_bits = nr_copy_bytes * 8 - bits_offset;
 #else
 # error "Unrecognized __BYTE_ORDER__"
 #endif
Signature · 1 line
2.31.1
#8 lore raw

PATCH 3/3 libbpf: Fix dumping __int128

In reply to Ilya Leoshkevich

On s390 __int128 can be 8-byte aligned, therefore in libbpf will occasionally consider variables of this type non-aligned and try to dump them as a bitfield, which is supported for at most 64-bit integers.

Fix by using the same trick as btf_dump_float_data(): copy non-aligned values to the local buffer.

---
 tools/lib/bpf/btf_dump.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tools/lib/bpf/btf_dump.c +6 -3
index ab45771d0cb4..d8264c1762e8 100644
@@ -1672,9 +1672,10 @@ static int btf_dump_int_data(struct btf_dump *d,
 {
 	__u8 encoding = btf_int_encoding(t);
 	bool sign = encoding & BTF_INT_SIGNED;
+	char buf[16] __aligned(16);
 	int sz = t->size;
 
-	if (sz == 0) {
+	if (sz == 0 || sz > sizeof(buf)) {
 		pr_warn("unexpected size %d for id [%u]\n", sz, type_id);
 		return -EINVAL;
 	}
@@ -1682,8 +1683,10 @@ static int btf_dump_int_data(struct btf_dump *d,
 	/* handle packed int data - accesses of integers not aligned on
 	 * int boundaries can cause problems on some platforms.
 	 */
-	if (!ptr_is_aligned(data, sz))
-		return btf_dump_bitfield_data(d, t, data, 0, 0);
+	if (!ptr_is_aligned(data, sz)) {
+		memcpy(buf, data, sz);
+		data = buf;
+	}
 
 	switch (sz) {
 	case 16: {
Signature · 1 line
2.31.1
#1 lore raw

PATCH 1/3 selftests/bpf: Use cpu_number only on arches that have it

In reply to Ilya Leoshkevich

cpu_number exists only on Intel and aarch64, so skip the test involing it on other arches. An alternative would be to replace it with an exported non-ifdefed primitive-typed percpu variable from the common code, but there appears to be none.

---
 tools/testing/selftests/bpf/prog_tests/btf_dump.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/btf_dump.c +2 -0
index 87f9df653e4e..12f457b6786d 100644
@@ -778,8 +778,10 @@ static void test_btf_dump_struct_data(struct btf *btf, struct btf_dump *d,
 static void test_btf_dump_var_data(struct btf *btf, struct btf_dump *d,
 				   char *str)
 {
+#if defined(__i386__) || defined(__x86_64__) || defined(__aarch64__)
 	TEST_BTF_DUMP_VAR(btf, d, NULL, str, "cpu_number", int, BTF_F_COMPACT,
 			  "int cpu_number = (int)100", 100);
+#endif
 	TEST_BTF_DUMP_VAR(btf, d, NULL, str, "cpu_profile_flip", int, BTF_F_COMPACT,
 			  "static int cpu_profile_flip = (int)2", 2);
 }
Signature · 1 line
2.31.1
#9 lore raw

Re: [PATCH bpf-next 3/3] libbpf: Fix dumping __int128

In reply to Ilya Leoshkevich
Quoted 31 lines · Ilya Leoshkevich

On s390 __int128 can be 8-byte aligned, therefore in libbpf will occasionally consider variables of this type non-aligned and try to dump them as a bitfield, which is supported for at most 64-bit integers.

Fix by using the same trick as btf_dump_float_data(): copy non-aligned values to the local buffer.

---
 tools/lib/bpf/btf_dump.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tools/lib/bpf/btf_dump.c +2 -1
index ab45771d0cb4..d8264c1762e8 100644
@@ -1672,9 +1672,10 @@ static int btf_dump_int_data(struct btf_dump *d,
 {
        __u8 encoding = btf_int_encoding(t);
        bool sign = encoding & BTF_INT_SIGNED;
+       char buf[16] __aligned(16);
        int sz = t->size;
 
-       if (sz == 0) {
+       if (sz == 0 || sz > sizeof(buf)) {
                pr_warn("unexpected size %d for id [%u]\n", sz, type_id);
                return -EINVAL;
        }

tools/lib/bpf/btf_dump.c static int btf_dump_int_data(struct btf_dump *d, · line 1686

Quoted 6 lines
tools/lib/bpf/btf_dump.c +1 -2
@@ -1682,8 +1683,10 @@ static int btf_dump_int_data(struct btf_dump *d,
        /* handle packed int data - accesses of integers not aligned on
         * int boundaries can cause problems on some platforms.
         */
-       if (!ptr_is_aligned(data, sz))
-               return btf_dump_bitfield_data(d, t, data, 0, 0);
+       if (!ptr_is_aligned(data, sz)) {

I think ptr_is_aligned() logic is wrong. We should probably not assume that __int128 has 16-byte alignment. Can you try fixing it by using btf__align_of() API to get the natural alignment?

Quoted 9 lines
tools/lib/bpf/btf_dump.c +3 -1
+               memcpy(buf, data, sz);
+               data = buf;
+       }
 
        switch (sz) {
        case 16: {
--

2.31.1

#2 lore raw

Re: [PATCH bpf-next 1/3] selftests/bpf: Use cpu_number only on arches that have it

In reply to Ilya Leoshkevich
Quoted 16 lines · Ilya Leoshkevich

cpu_number exists only on Intel and aarch64, so skip the test involing it on other arches. An alternative would be to replace it with an exported non-ifdefed primitive-typed percpu variable from the common code, but there appears to be none.

---
 tools/testing/selftests/bpf/prog_tests/btf_dump.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/btf_dump.c +0 -0
index 87f9df653e4e..12f457b6786d 100644

tools/testing/selftests/bpf/prog_tests/btf_dump.c static void test_btf_dump_struct_data(struct btf *btf, struct btf_dump *d, · line 784

Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/btf_dump.c +2 -0
@@ -778,8 +778,10 @@ static void test_btf_dump_struct_data(struct btf *btf, struct btf_dump *d,
 static void test_btf_dump_var_data(struct btf *btf, struct btf_dump *d,
                                   char *str)
 {
+#if defined(__i386__) || defined(__x86_64__) || defined(__aarch64__)
        TEST_BTF_DUMP_VAR(btf, d, NULL, str, "cpu_number", int, BTF_F_COMPACT,
                          "int cpu_number = (int)100", 100);
+#endif

We are in the talks about supporting cross-compilation of selftests, and this will be just another breakage that we'll have to undo.

Can we find some other variable that will be available on all architectures? Maybe "runqueues"?

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/btf_dump.c +0 -1
        TEST_BTF_DUMP_VAR(btf, d, NULL, str, "cpu_profile_flip", int, BTF_F_COMPACT,
                          "static int cpu_profile_flip = (int)2", 2);
 }
--

2.31.1

#6 lore raw

Re: [PATCH bpf-next 2/3] libbpf: Fix dumping big-endian bitfields

In reply to Ilya Leoshkevich
Quoted 18 lines · Ilya Leoshkevich

On big-endian arches not only bytes, but also bits are numbered in reverse order (see e.g. S/390 ELF ABI Supplement, but this is also true for other big-endian arches as well).

---
 tools/lib/bpf/btf_dump.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tools/lib/bpf/btf_dump.c +0 -0
index ad6df97295ae..ab45771d0cb4 100644
@@ -1577,14 +1577,15 @@ static int btf_dump_get_bitfield_value(struct btf_dump *d,
        /* Bitfield value retrieval is done in two steps; first relevant bytes are
         * stored in num, then we left/right shift num to eliminate irrelevant bits.
         */

tools/lib/bpf/btf_dump.c static int btf_dump_get_bitfield_value(struct btf_dump *d, · line 1588

Quoted 10 lines
tools/lib/bpf/btf_dump.c +2 -1
-       nr_copy_bits = bit_sz + bits_offset;
        nr_copy_bytes = t->size;
 #if __BYTE_ORDER == __LITTLE_ENDIAN
        for (i = nr_copy_bytes - 1; i >= 0; i--)
                num = num * 256 + bytes[i];
+       nr_copy_bits = bit_sz + bits_offset;
 #elif __BYTE_ORDER == __BIG_ENDIAN
        for (i = 0; i < nr_copy_bytes; i++)
                num = num * 256 + bytes[i];
+       nr_copy_bits = nr_copy_bytes * 8 - bits_offset;

oh, I remember dealing with this in the context of pahole. Just one nit, please use t->size instead of nr_copy_bytes, I think it will make it a bit more explicit (nr_copy_bytes is logically mutable, though only in little-endian case, but still).

Quoted 6 lines
tools/lib/bpf/btf_dump.c +0 -1
 #else
 # error "Unrecognized __BYTE_ORDER__"
 #endif
--

2.31.1

#3 lore raw

Re: [PATCH bpf-next 1/3] selftests/bpf: Use cpu_number only on arches that have it

In reply to Andrii Nakryiko
Quoted 27 lines · Andrii Nakryiko
Quoted 25 lines · Ilya Leoshkevich

cpu_number exists only on Intel and aarch64, so skip the test involing
it on other arches. An alternative would be to replace it with an exported non-ifdefed primitive-typed percpu variable from the common
code, but there appears to be none.

---
 tools/testing/selftests/bpf/prog_tests/btf_dump.c | 2 ++

 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/btf_dump.c +0 -0
index 87f9df653e4e..12f457b6786d 100644
@@ -778,8 +778,10 @@ static void test_btf_dump_struct_data(struct
 btf *btf, struct btf_dump *d,

 static void test_btf_dump_var_data(struct btf *btf, struct btf_dump *d,
                                   char *str)
 {
+#if defined(__i386__) || defined(__x86_64__) ||

Quoted 8 lines
Quoted 5 lines

defined(__aarch64__)
        TEST_BTF_DUMP_VAR(btf, d, NULL, str, "cpu_number", int, BTF_F_COMPACT,
                          "int cpu_number = (int)100", 100); +#endif

We are in the talks about supporting cross-compilation of selftests, and this will be just another breakage that we'll have to undo.

Why would this break? Cross-compilation should define these macros based on target, not build system.

Quoted 2 lines

Can we find some other variable that will be available on all architectures? Maybe "runqueues"?

Wouldn't runqueues be pointless? We already have cpu_profile_flip. I thought the idea here was to have something marked with EXPORT_PER_CPU_SYMBOL.

Quoted 7 lines
Quoted 7 lines

        TEST_BTF_DUMP_VAR(btf, d, NULL, str, "cpu_profile_flip", int, BTF_F_COMPACT,
                          "static int cpu_profile_flip = (int)2", 2);
 }
--
2.31.1

#7 lore raw

Re: [PATCH bpf-next 2/3] libbpf: Fix dumping big-endian bitfields

In reply to Andrii Nakryiko
Quoted 32 lines · Andrii Nakryiko
Quoted 30 lines · Ilya Leoshkevich

On big-endian arches not only bytes, but also bits are numbered in reverse order (see e.g. S/390 ELF ABI Supplement, but this is also true
for other big-endian arches as well).

---
 tools/lib/bpf/btf_dump.c | 3 ++-

 1 file changed, 2 insertions(+), 1 deletion(-)
tools/lib/bpf/btf_dump.c +0 -0
index ad6df97295ae..ab45771d0cb4 100644
@@ -1577,14 +1577,15 @@ static int
 btf_dump_get_bitfield_value(struct btf_dump *d,
         /* Bitfield value retrieval is done in two steps; first

relevant bytes are
         * stored in num, then we left/right shift num to eliminate irrelevant bits.
         */
-       nr_copy_bits = bit_sz + bits_offset; nr_copy_bytes = t->size;
 #if __BYTE_ORDER == __LITTLE_ENDIAN
        for (i = nr_copy_bytes - 1; i >= 0; i--)
                num = num * 256 + bytes[i];
+       nr_copy_bits = bit_sz + bits_offset;
 #elif __BYTE_ORDER == __BIG_ENDIAN
        for (i = 0; i < nr_copy_bytes; i++)

Quoted 8 lines
Quoted 2 lines

                num = num * 256 + bytes[i];
+       nr_copy_bits = nr_copy_bytes * 8 - bits_offset;

oh, I remember dealing with this in the context of pahole. Just one nit, please use t->size instead of nr_copy_bytes, I think it will make
it a bit more explicit (nr_copy_bytes is logically mutable, though only in little-endian case, but still).

Both sz and num_copy_bytes look redundant to be honest. What do you think about dropping both completely and just using t->size everywhere instead?

Quoted 7 lines
Quoted 7 lines

 #else

 # error "Unrecognized __BYTE_ORDER__"
 #endif
--
2.31.1
#10 lore raw

Re: [PATCH bpf-next 3/3] libbpf: Fix dumping __int128

In reply to Andrii Nakryiko
Quoted 41 lines · Andrii Nakryiko
Quoted 39 lines · Ilya Leoshkevich

On s390 __int128 can be 8-byte aligned, therefore in libbpf will occasionally consider variables of this type non-aligned and try to dump them as a bitfield, which is supported for at most 64-bit integers.

Fix by using the same trick as btf_dump_float_data(): copy non- aligned
values to the local buffer.

---
 tools/lib/bpf/btf_dump.c | 9 ++++++---

 1 file changed, 6 insertions(+), 3 deletions(-)
tools/lib/bpf/btf_dump.c +2 -1
index ab45771d0cb4..d8264c1762e8 100644
@@ -1672,9 +1672,10 @@ static int btf_dump_int_data(struct btf_dump
 *d,
  {
         __u8 encoding = btf_int_encoding(t);
         bool sign = encoding & BTF_INT_SIGNED;
+       char buf[16] __aligned(16);
         int sz = t->size;
 
-       if (sz == 0) {
+       if (sz == 0 || sz > sizeof(buf)) {
                 pr_warn("unexpected size %d for id [%u]\n", sz,
 type_id);

                return -EINVAL;
        }

tools/lib/bpf/btf_dump.c +0 -0
@@ -1682,8 +1683,10 @@ static int btf_dump_int_data(struct btf_dump
 *d,
         /* handle packed int data - accesses of integers not

aligned on
         * int boundaries can cause problems on some platforms. */

Quoted 8 lines
Quoted 3 lines

-       if (!ptr_is_aligned(data, sz))
-               return btf_dump_bitfield_data(d, t, data, 0, 0); +       if (!ptr_is_aligned(data, sz)) {

I think ptr_is_aligned() logic is wrong. We should probably not assume
that __int128 has 16-byte alignment. Can you try fixing it by using btf__align_of() API to get the natural alignment?

Ok, but this patch makes sense anyway, doesn't it? I can fix ptr_is_aligned() separately.

Quoted 10 lines
Quoted 10 lines

+               memcpy(buf, data, sz);
+               data = buf;
+       }

        switch (sz) {
        case 16: {
--
2.31.1

#4 lore raw

Re: [PATCH bpf-next 1/3] selftests/bpf: Use cpu_number only on arches that have it

In reply to Ilya Leoshkevich
Quoted 38 lines · Ilya Leoshkevich
Quoted 27 lines · Andrii Nakryiko
Quoted 25 lines · Ilya Leoshkevich

cpu_number exists only on Intel and aarch64, so skip the test involing
it on other arches. An alternative would be to replace it with an exported non-ifdefed primitive-typed percpu variable from the common
code, but there appears to be none.

---
 tools/testing/selftests/bpf/prog_tests/btf_dump.c | 2 ++
 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/prog_tests/btf_dump.c +1 -0
index 87f9df653e4e..12f457b6786d 100644
@@ -778,8 +778,10 @@ static void test_btf_dump_struct_data(struct
 btf *btf, struct btf_dump *d,
 static void test_btf_dump_var_data(struct btf *btf, struct
 btf_dump *d,
                                   char *str)
 {
+#if defined(__i386__) || defined(__x86_64__) ||
Quoted 8 lines

tools/testing/selftests/bpf/prog_tests/btf_dump.c static void test_btf_dump_struct_data(struct

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/btf_dump.c +1 -0
 defined(__aarch64__)
        TEST_BTF_DUMP_VAR(btf, d, NULL, str, "cpu_number", int,
 BTF_F_COMPACT,
                          "int cpu_number = (int)100", 100);
+#endif

We are in the talks about supporting cross-compilation of selftests, and this will be just another breakage that we'll have to undo.

Quoted 2 lines

Why would this break? Cross-compilation should define these macros based on target, not build system.

Ok, then it should be good.

Quoted 7 lines
Quoted 3 lines

Can we find some other variable that will be available on all architectures? Maybe "runqueues"?

Wouldn't runqueues be pointless? We already have cpu_profile_flip. I thought the idea here was to have something marked with EXPORT_PER_CPU_SYMBOL.

No idea what the idea was, tbh.

Quoted 9 lines
Quoted 8 lines
Quoted 8 lines

TEST_BTF_DUMP_VAR(btf, d, NULL, str, "cpu_profile_flip",
int, BTF_F_COMPACT,
"static int cpu_profile_flip = (int)2",
2);
}
--
2.31.1