bpf: Fix a verifier bug due to incorrect branch offset comparison with cpu=v4
Yonghong Song
reviewerPATCH 1/1 bpf: Fix a verifier bug due to incorrect branch offset comparison with cpu=v4
Bpf cpu=v4 support is introduced in [1] and Commit 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction") added support for new 32bit offset jmp instruction. Unfortunately, in function bpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset (plus/minor a small delta) compares to 16-bit offset bound [S16_MIN, S16_MAX], which caused the following verification failure:
$ ./test_progs-cpuv4 -t verif_scale_pyperf180 ... insn 10 cannot be patched due to 16-bit range ... libbpf: failed to load object 'pyperf180.bpf.o' scale_test:FAIL:expect_success unexpected error: -12 (errno 12) #405 verif_scale_pyperf180:FAIL
Note that due to recent llvm18 development, the patch [2] (already applied in bpf-next) needs to be applied to bpf tree for testing purpose.
The fix is rather simple. For 32bit offset branch insn, the adjusted offset compares to [S32_MIN, S32_MAX] and then verification succeeded.
[1] https://lore.kernel.org/all/[email protected]
[2] https://lore.kernel.org/bpf/[email protected]
--- kernel/bpf/core.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
kernel/bpf/core.c +6 -3
@@ -371,14 +371,17 @@ static int bpf_adj_delta_to_imm(struct bpf_insn *insn, u32 pos, s32 end_old,static int bpf_adj_delta_to_off (struct bpf_insn *insn,u32 pos,s32 end_old,s32 end_new,s32 curr,const bool probe_pass) { -const s32 off_min =S16_MIN , off_max =S16_MAX ; +s32 off_min =S16_MIN , off_max =S16_MAX ;s32 delta = end_new - end_old;s32 off; -if (insn->code == (BPF_JMP32 |BPF_JA )) +if (insn->code == (BPF_JMP32 |BPF_JA )) { off = insn->imm; -else + off_min =S32_MIN ; + off_max =S32_MAX ; + }else { off = insn->off; + }if (curr < pos && curr + off +1 >= end_old) off += delta;
Signature · 1 line
2.34.1
Quoted 17 lines · Yonghong Song
Bpf cpu=v4 support is introduced in [1] and Commit
4cd58e9af8b9("bpf: Support new 32bit offset jmp instruction") added support for new 32bit offset jmp instruction. Unfortunately, in functionbpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset (plus/minor a small delta) compares to 16-bit offset bound [S16_MIN,S16_MAX], which caused the following verification failure:$ ./test_progs-cpuv4 -t verif_scale_pyperf180 ... insn 10 cannot be patched due to 16-bit range ... libbpf: failed to load object 'pyperf180.bpf.o' scale_test:FAIL:expect_success unexpected error: -12 (errno 12) #405 verif_scale_pyperf180:FAILNote that due to recent llvm18 development, the patch [2] (already applied in bpf-next) needs to be applied to bpf tree for testing purpose.
Quoted 8 lines
The fix is rather simple. For 32bit offset branch insn, the adjusted offset compares to [
S32_MIN,S32_MAX] and then verification succeeded.[1] https://lore.kernel.org/all/[email protected]
[2] https://lore.kernel.org/bpf/[email protected]Fixes 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction")Signed-off-by Yonghong Song <[email protected]>
Daniel Borkmann
maintainerQuoted 34 lines · Yonghong Song
Bpf cpu=v4 support is introduced in [1] and Commit
4cd58e9af8b9("bpf: Support new 32bit offset jmp instruction") added support for new 32bit offset jmp instruction. Unfortunately, in functionbpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset (plus/minor a small delta) compares to 16-bit offset bound [S16_MIN,S16_MAX], which caused the following verification failure:$ ./test_progs-cpuv4 -t verif_scale_pyperf180 ... insn 10 cannot be patched due to 16-bit range ... libbpf: failed to load object 'pyperf180.bpf.o' scale_test:FAIL:expect_success unexpected error: -12 (errno 12) #405 verif_scale_pyperf180:FAILNote that due to recent llvm18 development, the patch [2] (already applied in bpf-next) needs to be applied to bpf tree for testing purpose.
The fix is rather simple. For 32bit offset branch insn, the adjusted offset compares to [
S32_MIN,S32_MAX] and then verification succeeded.[1] https://lore.kernel.org/all/[email protected] [2] https://lore.kernel.org/bpf/[email protected]Fixes 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction")Signed-off-by Yonghong Song <[email protected]>--- kernel/bpf/core.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)kernel/bpf/core.c +0
-0
kernel/bpf/core.c static int bpf_adj_delta_to_imm(struct bpf_insn *insn, u32 pos, s32 end_old, · line 376
Quoted 7 lines
kernel/bpf/core.c +1
-1@@ -371,14 +371,17 @@ static int bpf_adj_delta_to_imm(struct bpf_insn *insn, u32 pos, s32 end_old,static int bpf_adj_delta_to_off (struct bpf_insn *insn,u32 pos,s32 end_old,s32 end_new,s32 curr,const bool probe_pass) { -const s32 off_min =S16_MIN , off_max =S16_MAX ; +s32 off_min =S16_MIN , off_max =S16_MAX ;s32 delta = end_new - end_old;s32 off;
These should all be converted to s64, no? E.g. further below the test will never trigger then for jmp32:
if (off < off_min || off > off_max)
return -ERANGE;Quoted 13 lines
kernel/bpf/core.c +5
-2-if (insn->code == (BPF_JMP32 |BPF_JA )) +if (insn->code == (BPF_JMP32 |BPF_JA )) { off = insn->imm; -else + off_min =S32_MIN ; + off_max =S32_MAX ; + }else { off = insn->off; + }if (curr < pos && curr + off +1 >= end_old) off += delta;
Yonghong Song
reviewerQuoted 47 lines · Daniel Borkmann
Quoted 34 lines · Yonghong Song
Bpf cpu=v4 support is introduced in [1] and Commit
4cd58e9af8b9("bpf: Support new 32bit offset jmp instruction") added support for new 32bit offset jmp instruction. Unfortunately, in functionbpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset (plus/minor a small delta) compares to 16-bit offset bound [S16_MIN,S16_MAX], which caused the following verification failure:$ ./test_progs-cpuv4 -t verif_scale_pyperf180 ... insn 10 cannot be patched due to 16-bit range ... libbpf: failed to load object 'pyperf180.bpf.o' scale_test:FAIL:expect_success unexpected error: -12 (errno 12) #405 verif_scale_pyperf180:FAILNote that due to recent llvm18 development, the patch [2] (already applied in bpf-next) needs to be applied to bpf tree for testing purpose.
The fix is rather simple. For 32bit offset branch insn, the adjusted offset compares to [
S32_MIN,S32_MAX] and then verification succeeded.[1] https://lore.kernel.org/all/[email protected]
[2] https://lore.kernel.org/bpf/[email protected]Fixes 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction")Signed-off-by Yonghong Song <[email protected]>---
kernel/bpf/core.c| 9 ++++++---1 file changed, 6 insertions(+), 3 deletions(-)kernel/bpf/core.c +0
-0kernel/bpf/core.c static int bpf_adj_delta_to_imm(struct bpf_insn · line 377
Quoted 8 lines
kernel/bpf/core.c +1
-1@@ -371,14 +371,17 @@ static int bpf_adj_delta_to_imm(struct bpf_insn *insn, u32 pos, s32 end_old, static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old, s32 end_new, s32 curr, const bool probe_pass) { - const s32 off_min = S16_MIN, off_max = S16_MAX; + s32 off_min = S16_MIN, off_max = S16_MAX; s32 delta = end_new - end_old; s32 off;These should all be converted to s64, no? E.g. further below the test will never trigger then for jmp32:
Quoted 2 lines
if (off <
off_min|| off >off_max)
return-ERANGE;
good point! Let us use s64 for potential overflows. Will send v2 soon.
Quoted 14 lines
Quoted 13 lines
kernel/bpf/core.c +5
-2- if (insn->code == (BPF_JMP32 | BPF_JA)) + if (insn->code == (BPF_JMP32 | BPF_JA)) { off = insn->imm; - else + off_min = S32_MIN; + off_max = S32_MAX; + } else { off = insn->off; + } if (curr < pos && curr + off + 1 >= end_old) off += delta;
Yonghong Song
reviewerQuoted 49 lines · Yonghong Song
Quoted 47 lines · Daniel Borkmann
Quoted 34 lines · Yonghong Song
Bpf cpu=v4 support is introduced in [1] and Commit
4cd58e9af8b9("bpf: Support new 32bit offset jmp instruction") added support for new 32bit offset jmp instruction. Unfortunately, in functionbpf_adj_delta_to_off(), for new branch insn with 32bit offset, the offset (plus/minor a small delta) compares to 16-bit offset bound [S16_MIN,S16_MAX], which caused the following verification failure:$ ./test_progs-cpuv4 -t verif_scale_pyperf180 ... insn 10 cannot be patched due to 16-bit range ... libbpf: failed to load object 'pyperf180.bpf.o' scale_test:FAIL:expect_success unexpected error: -12 (errno 12) #405 verif_scale_pyperf180:FAILNote that due to recent llvm18 development, the patch [2] (already applied in bpf-next) needs to be applied to bpf tree for testing purpose.
The fix is rather simple. For 32bit offset branch insn, the adjusted offset compares to [
S32_MIN,S32_MAX] and then verification succeeded.[1] https://lore.kernel.org/all/[email protected]
[2] https://lore.kernel.org/bpf/[email protected]Fixes 4cd58e9af8b9 ("bpf: Support new 32bit offset jmp instruction")Signed-off-by Yonghong Song <[email protected]>---
kernel/bpf/core.c| 9 ++++++---1 file changed, 6 insertions(+), 3 deletions(-)kernel/bpf/core.c +0
-0kernel/bpf/core.c static int bpf_adj_delta_to_imm(struct · line 377
Quoted 8 lines
kernel/bpf/core.c +1
-1@@ -371,14 +371,17 @@ static int bpf_adj_delta_to_imm(struct bpf_insn *insn, u32 pos, s32 end_old, static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old, s32 end_new, s32 curr, const bool probe_pass) { - const s32 off_min = S16_MIN, off_max = S16_MAX; + s32 off_min = S16_MIN, off_max = S16_MAX; s32 delta = end_new - end_old; s32 off;These should all be converted to s64, no? E.g. further below the test will never trigger then for jmp32:
Quoted 6 lines
Quoted 2 lines
if (off <
off_min|| off >off_max)
return-ERANGE;good point! Let us use s64 for potential overflows. Will send v2 soon.
I didn't change 's32 delta' type to be consistent with bpf_adj_delta_to_imm() such that the delta should be within s32 range. Technically off_min/off_max can remain as 's32' but I changed them to 's64' to be consistent with bpf_adj_delta_to_imm().
Quoted 16 lines
Quoted 15 lines
Quoted 14 lines
kernel/bpf/core.c +5
-2- if (insn->code == (BPF_JMP32 | BPF_JA)) + if (insn->code == (BPF_JMP32 | BPF_JA)) { off = insn->imm; - else + off_min = S32_MIN; + off_max = S32_MAX; + } else { off = insn->off; + } if (curr < pos && curr + off + 1 >= end_old) off += delta;