Skip to contentSkip to conversation

bpf: Fix a verifier bug due to incorrect branch offset comparison with cpu=v4

PATCH Superseded Yonghong Song 3 participants 5 messages Nov 29, 2023 – Nov 30, 2023

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Yonghong Song

author reviewer
#0 lore raw

PATCH 1/1 bpf: Fix a verifier bug due to incorrect branch offset comparison with cpu=v4

Bpf cpu=v4 support is introduced in [1] and Commit 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction") added support for new 32bit offset jmp instruction. Unfortunately, in function bpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset (plus/minor a small delta) compares to 16-bit offset bound [S16_MIN, S16_MAX], which caused the following verification failure:

  $ ./test_progs-cpuv4 -t verif_scale_pyperf180
  ...
  insn 10 cannot be patched due to 16-bit range
  ...
  libbpf: failed to load object 'pyperf180.bpf.o'
  scale_test:FAIL:expect_success unexpected error: -12 (errno 12)
  #405     verif_scale_pyperf180:FAIL

Note that due to recent llvm18 development, the patch [2] (already applied in bpf-next) needs to be applied to bpf tree for testing purpose.

The fix is rather simple. For 32bit offset branch insn, the adjusted offset compares to [S32_MIN, S32_MAX] and then verification succeeded.

[1] https://lore.kernel.org/all/[email protected]
[2] https://lore.kernel.org/bpf/[email protected]

Fixes 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction")
---
 kernel/bpf/core.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
kernel/bpf/core.c +6 -3
index cd3afe57ece3..74f2fd48148c 100644
@@ -371,14 +371,17 @@ static int bpf_adj_delta_to_imm(struct bpf_insn *insn, u32 pos, s32 end_old,
 static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old,
 				s32 end_new, s32 curr, const bool probe_pass)
 {
-	const s32 off_min = S16_MIN, off_max = S16_MAX;
+	s32 off_min = S16_MIN, off_max = S16_MAX;
 	s32 delta = end_new - end_old;
 	s32 off;
 
-	if (insn->code == (BPF_JMP32 | BPF_JA))
+	if (insn->code == (BPF_JMP32 | BPF_JA)) {
 		off = insn->imm;
-	else
+		off_min = S32_MIN;
+		off_max = S32_MAX;
+	} else {
 		off = insn->off;
+	}
 
 	if (curr < pos && curr + off + 1 >= end_old)
 		off += delta;
Signature · 1 line
2.34.1
Acked-by on v1 1/1
#1 lore raw
Quoted 17 lines · Yonghong Song

Bpf cpu=v4 support is introduced in [1] and Commit 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction") added support for new 32bit offset jmp instruction. Unfortunately, in function bpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset (plus/minor a small delta) compares to 16-bit offset bound [S16_MIN, S16_MAX], which caused the following verification failure:

  $ ./test_progs-cpuv4 -t verif_scale_pyperf180
  ...
  insn 10 cannot be patched due to 16-bit range
  ...
  libbpf: failed to load object 'pyperf180.bpf.o'
  scale_test:FAIL:expect_success unexpected error: -12 (errno 12)
  #405     verif_scale_pyperf180:FAIL

Note that due to recent llvm18 development, the patch [2] (already applied in bpf-next) needs to be applied to bpf tree for testing purpose.

Quoted 8 lines

The fix is rather simple. For 32bit offset branch insn, the adjusted offset compares to [S32_MIN, S32_MAX] and then verification succeeded.

[1] https://lore.kernel.org/all/[email protected]
[2] https://lore.kernel.org/bpf/[email protected]

Fixes 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction")
maintainer
#2 lore raw
In reply to Yonghong Song
Quoted 34 lines · Yonghong Song

Bpf cpu=v4 support is introduced in [1] and Commit 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction") added support for new 32bit offset jmp instruction. Unfortunately, in function bpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset (plus/minor a small delta) compares to 16-bit offset bound [S16_MIN, S16_MAX], which caused the following verification failure:

   $ ./test_progs-cpuv4 -t verif_scale_pyperf180
   ...
   insn 10 cannot be patched due to 16-bit range
   ...
   libbpf: failed to load object 'pyperf180.bpf.o'
   scale_test:FAIL:expect_success unexpected error: -12 (errno 12)
   #405     verif_scale_pyperf180:FAIL

Note that due to recent llvm18 development, the patch [2] (already applied in bpf-next) needs to be applied to bpf tree for testing purpose.

The fix is rather simple. For 32bit offset branch insn, the adjusted offset compares to [S32_MIN, S32_MAX] and then verification succeeded.

   [1] https://lore.kernel.org/all/[email protected]
   [2] https://lore.kernel.org/bpf/[email protected]
Fixes 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction")
---
  kernel/bpf/core.c | 9 ++++++---
  1 file changed, 6 insertions(+), 3 deletions(-)
kernel/bpf/core.c +0 -0
index cd3afe57ece3..74f2fd48148c 100644

kernel/bpf/core.c static int bpf_adj_delta_to_imm(struct bpf_insn *insn, u32 pos, s32 end_old, · line 376

Quoted 7 lines
kernel/bpf/core.c +1 -1
@@ -371,14 +371,17 @@ static int bpf_adj_delta_to_imm(struct bpf_insn *insn, u32 pos, s32 end_old,
  static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old,
  				s32 end_new, s32 curr, const bool probe_pass)
  {
-	const s32 off_min = S16_MIN, off_max = S16_MAX;
+	s32 off_min = S16_MIN, off_max = S16_MAX;
  	s32 delta = end_new - end_old;
  	s32 off;

These should all be converted to s64, no? E.g. further below the test will never trigger then for jmp32:

       if (off < off_min || off > off_max)
                return -ERANGE;
Quoted 13 lines
kernel/bpf/core.c +5 -2
-	if (insn->code == (BPF_JMP32 | BPF_JA))
+	if (insn->code == (BPF_JMP32 | BPF_JA)) {
  		off = insn->imm;
-	else
+		off_min = S32_MIN;
+		off_max = S32_MAX;
+	} else {
  		off = insn->off;
+	}
 
  	if (curr < pos && curr + off + 1 >= end_old)
  		off += delta;
 
author reviewer
#3 lore raw
Quoted 47 lines · Daniel Borkmann
Quoted 34 lines · Yonghong Song

Bpf cpu=v4 support is introduced in [1] and Commit 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction") added support for new 32bit offset jmp instruction. Unfortunately, in function bpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset (plus/minor a small delta) compares to 16-bit offset bound [S16_MIN, S16_MAX], which caused the following verification failure:

   $ ./test_progs-cpuv4 -t verif_scale_pyperf180
   ...
   insn 10 cannot be patched due to 16-bit range
   ...
   libbpf: failed to load object 'pyperf180.bpf.o'
   scale_test:FAIL:expect_success unexpected error: -12 (errno 12)
   #405     verif_scale_pyperf180:FAIL

Note that due to recent llvm18 development, the patch [2] (already applied in bpf-next) needs to be applied to bpf tree for testing purpose.

The fix is rather simple. For 32bit offset branch insn, the adjusted offset compares to [S32_MIN, S32_MAX] and then verification succeeded.

   [1] https://lore.kernel.org/all/[email protected]
   [2] https://lore.kernel.org/bpf/[email protected]

Fixes 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction")

---
  kernel/bpf/core.c | 9 ++++++---

  1 file changed, 6 insertions(+), 3 deletions(-)
kernel/bpf/core.c +0 -0
index cd3afe57ece3..74f2fd48148c 100644

kernel/bpf/core.c static int bpf_adj_delta_to_imm(struct bpf_insn · line 377

Quoted 8 lines
kernel/bpf/core.c +1 -1
@@ -371,14 +371,17 @@ static int bpf_adj_delta_to_imm(struct bpf_insn 
 *insn, u32 pos, s32 end_old,
   static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old,
                   s32 end_new, s32 curr, const bool probe_pass)
   {
-    const s32 off_min = S16_MIN, off_max = S16_MAX;
+    s32 off_min = S16_MIN, off_max = S16_MAX;
       s32 delta = end_new - end_old;
       s32 off;

These should all be converted to s64, no? E.g. further below the test will never trigger then for jmp32:

Quoted 2 lines

       if (off < off_min || off > off_max)
                return -ERANGE;

good point! Let us use s64 for potential overflows. Will send v2 soon.

Quoted 14 lines
Quoted 13 lines
kernel/bpf/core.c +5 -2
-    if (insn->code == (BPF_JMP32 | BPF_JA))
+    if (insn->code == (BPF_JMP32 | BPF_JA)) {
           off = insn->imm;
-    else
+        off_min = S32_MIN;
+        off_max = S32_MAX;
+    } else {
           off = insn->off;
+    }
         if (curr < pos && curr + off + 1 >= end_old)
           off += delta;
 
author reviewer
#4 lore raw
Quoted 49 lines · Yonghong Song
Quoted 47 lines · Daniel Borkmann
Quoted 34 lines · Yonghong Song

Bpf cpu=v4 support is introduced in [1] and Commit 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction") added support for new 32bit offset jmp instruction. Unfortunately, in function bpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset (plus/minor a small delta) compares to 16-bit offset bound [S16_MIN, S16_MAX], which caused the following verification failure:

   $ ./test_progs-cpuv4 -t verif_scale_pyperf180
   ...
   insn 10 cannot be patched due to 16-bit range
   ...
   libbpf: failed to load object 'pyperf180.bpf.o'
   scale_test:FAIL:expect_success unexpected error: -12 (errno 12)
   #405     verif_scale_pyperf180:FAIL

Note that due to recent llvm18 development, the patch [2] (already applied in bpf-next) needs to be applied to bpf tree for testing purpose.

The fix is rather simple. For 32bit offset branch insn, the adjusted offset compares to [S32_MIN, S32_MAX] and then verification succeeded.

   [1] https://lore.kernel.org/all/[email protected]
   [2] https://lore.kernel.org/bpf/[email protected]

Fixes 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction")

---
  kernel/bpf/core.c | 9 ++++++---

  1 file changed, 6 insertions(+), 3 deletions(-)
kernel/bpf/core.c +0 -0
index cd3afe57ece3..74f2fd48148c 100644

kernel/bpf/core.c static int bpf_adj_delta_to_imm(struct · line 377

Quoted 8 lines
kernel/bpf/core.c +1 -1
@@ -371,14 +371,17 @@ static int bpf_adj_delta_to_imm(struct 
 bpf_insn *insn, u32 pos, s32 end_old,
   static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old,
                   s32 end_new, s32 curr, const bool probe_pass)
   {
-    const s32 off_min = S16_MIN, off_max = S16_MAX;
+    s32 off_min = S16_MIN, off_max = S16_MAX;
       s32 delta = end_new - end_old;
       s32 off;

These should all be converted to s64, no? E.g. further below the test will never trigger then for jmp32:

Quoted 6 lines
Quoted 2 lines

       if (off < off_min || off > off_max)
                return -ERANGE;

good point! Let us use s64 for potential overflows. Will send v2 soon.

I didn't change 's32 delta' type to be consistent with bpf_adj_delta_to_imm() such that the delta should be within s32 range. Technically off_min/off_max can remain as 's32' but I changed them to 's64' to be consistent with bpf_adj_delta_to_imm().

Quoted 16 lines
Quoted 15 lines
Quoted 14 lines
kernel/bpf/core.c +5 -2
-    if (insn->code == (BPF_JMP32 | BPF_JA))
+    if (insn->code == (BPF_JMP32 | BPF_JA)) {
           off = insn->imm;
-    else
+        off_min = S32_MIN;
+        off_max = S32_MAX;
+    } else {
           off = insn->off;
+    }
         if (curr < pos && curr + off + 1 >= end_old)
           off += delta;