selftests/bpf: limit unroll_count for pyperf600 test
Subsystems (3)
PATCH 1/1 selftests/bpf: limit unroll_count for pyperf600 test
LLVM commit [1] changed loop pragma behavior such that full loop unroll is always honored with user pragma. Previously, unroll count also depends on the unrolled code size. For pyperf600, without [1], the loop unroll count is 150. With [1], the loop unroll count is 600.
The unroll count of 600 caused the program size close to 298k and this caused the following code is generated:
0: 7b 1a 00 ff 00 00 00 00 *(u64 *)(r10 - 256) = r1
; uint64_t pid_tgid = bpf_get_current_pid_tgid();
1: 85 00 00 00 0e 00 00 00 call 14
2: bf 06 00 00 00 00 00 00 r6 = r0
; pid_t pid = (pid_t)(pid_tgid >> 32);
3: bf 61 00 00 00 00 00 00 r1 = r6
4: 77 01 00 00 20 00 00 00 r1 >>= 32
5: 63 1a fc ff 00 00 00 00 *(u32 *)(r10 - 4) = r1
6: bf a2 00 00 00 00 00 00 r2 = r10
7: 07 02 00 00 fc ff ff ff r2 += -4
; PidData* pidData = bpf_map_lookup_elem(&pidmap, &pid);
8: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll
10: 85 00 00 00 01 00 00 00 call 1
11: bf 08 00 00 00 00 00 00 r8 = r0
; if (!pidData)
12: 15 08 15 e8 00 00 00 00 if r8 == 0 goto -6123 <LBB0_27588+0xffffffffffdae100>Note that insn 12 has a branch offset -6123 which is clearly illegal and will be rejected by the verifier. The negative offset is due to the branch range is greater than INT16_MAX.
This patch changed the unroll count to be 150 to avoid above branch target insn out-of-range issue. Also the llvm is enhanced ([2]) to assert if the branch target insn is out of INT16 range.
[1] https://reviews.llvm.org/D119148
[2] https://reviews.llvm.org/D123877
--- tools/testing/selftests/bpf/progs/pyperf.h | 4 ++++ tools/testing/selftests/bpf/progs/pyperf600.c | 11 +++++++---- 2 files changed, 11 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/progs/pyperf.h +4 -0
@@ -299,7 +299,11 @@ int __on_event(struct bpf_raw_tracepoint_args *ctx)#ifdef NO_UNROLL #pragma clang loop unroll(disable)#else +#ifdef UNROLL_COUNT +#pragma clang loop unroll_count(UNROLL_COUNT) +#else #pragma clang loop unroll(full) +#endif #endif /* NO_UNROLL */ /* Unwind python stack */ for (int i =0 ; i <STACK_MAX_LEN ; ++i) {
tools/testing/selftests/bpf/progs/pyperf600.c +7 -4
@@ -1,9 +1,12 @@// SPDX-License-Identifier: GPL-2.0 // Copyright (c) 2019 Facebook #define STACK_MAX_LEN 600 -/* clang will not unroll the loop 600 times. -* Instead it will unroll it to the amount it deemed -* appropriate, but the loop will still execute 600 times. -* Total program size is around 90k insns +/* Full unroll of 600 iterations will have total +* program size close to 298k insns and this may +* cause BPF_JMP insn out of 16-bit integer range. +* So limit the unroll size to 150 so the +* total program size is around 80k insns but +* the loop will still execute 600 times. */ +#define UNROLL_COUNT 150#include "pyperf.h"
Signature · 1 line
2.30.2
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 25 lines · On Mon, 18 Apr 2022 21:32:30 -0700 you wrote:
LLVM commit [1] changed loop pragma behavior such that full loop unroll is always honored with user pragma. Previously, unroll count also depends on the unrolled code size. For pyperf600, without [1], the loop unroll count is 150. With [1], the loop unroll count is 600.
The unroll count of 600 caused the program size close to 298k and this caused the following code is generated:
0: 7b 1a 00 ff 00 00 00 00 *(u64 *)(r10 - 256) = r1 ; uint64_t pid_tgid = bpf_get_current_pid_tgid(); 1: 85 00 00 00 0e 00 00 00 call 14 2: bf 06 00 00 00 00 00 00 r6 = r0 ; pid_t pid = (pid_t)(pid_tgid >> 32); 3: bf 61 00 00 00 00 00 00 r1 = r6 4: 77 01 00 00 20 00 00 00 r1 >>= 32 5: 63 1a fc ff 00 00 00 00 *(u32 *)(r10 - 4) = r1 6: bf a2 00 00 00 00 00 00 r2 = r10 7: 07 02 00 00 fc ff ff ff r2 += -4 ; PidData* pidData = bpf_map_lookup_elem(&pidmap, &pid); 8: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll 10: 85 00 00 00 01 00 00 00 call 1 11: bf 08 00 00 00 00 00 00 r8 = r0 ; if (!pidData) 12: 15 08 15 e8 00 00 00 00 if r8 == 0 goto -6123 <LBB0_27588+0xffffffffffdae100>
Quoted 1 line
[...]
Here is the summary with links:
- [bpf-next] selftests/bpf: limit unroll_count for pyperf600 test
https://git.kernel.org/bpf/bpf-next/c/8c89b5db7a28
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html