bpf: Support stack arguments for bpf functions
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 63 lines · Yonghong Song
Currently BPF functions (subprogs) are limited to 5 register arguments. With [1], the compiler can emit code that passes additional arguments via a dedicated stack area through bpf register
BPF_REG_PARAMS(r11), introduced in the previous patch.The compiler uses positive r11 offsets for incoming (callee-side) args and negative r11 offsets for outgoing (caller-side) args, following the x86_64/arm64 calling convention direction. There is an 8-byte gap at offset 0 separating the two regions:
Incoming (callee reads): r11+8 (arg6), r11+16 (arg7), ...
Outgoing (caller writes): r11-8 (arg6), r11-16 (arg7), ...The following is an example to show how stack arguments are saved and transferred between caller and callee:
int foo(int a1, int a2, int a3, int a4, int a5, int a6, int a7) { ... bar(a1, a2, a3, a4, a5, a6, a7, a8); ... }Caller (foo) Callee (bar) ============ ============ Incoming (positive offsets): Incoming (positive offsets):r11+8: [incoming arg 6] r11+8: [incoming arg 6] <-+
r11+16: [incoming arg 7] r11+16: [incoming arg 7] <-|+ r11+24: [incoming arg 8] <-||+ Outgoing (negative offsets): |||r11-8: [outgoing arg 6 to bar] -------->-------------------------+||
r11-16: [outgoing arg 7 to bar] -------->--------------------------+|
r11-24: [outgoing arg 8 to bar] -------->---------------------------+If the bpf function has more than one call:
int foo(int a1, int a2, int a3, int a4, int a5, int a6, int a7) { ... bar1(a1, a2, a3, a4, a5, a6, a7, a8); ... bar2(a1, a2, a3, a4, a5, a6, a7, a8, a9); ... }Caller (foo) Callee (bar2) ============ ============== Incoming (positive offsets): Incoming (positive offsets):r11+8: [incoming arg 6] r11+8: [incoming arg 6] <+
r11+16: [incoming arg 7] r11+16: [incoming arg 7] <|+ r11+24: [incoming arg 8] <||+ Outgoing for bar2 (negative offsets): r11+32: [incoming arg 9] <|||+ r11-8: [outgoing arg 6] ---->----------->-------------------------+||| r11-16: [outgoing arg 7] ---->----------->--------------------------+|| r11-24: [outgoing arg 8] ---->----------->---------------------------+|r11-32: [outgoing arg 9] ---->----------->----------------------------+
The verifier tracks stack arguments separately from the regular r10 stack. The
stack_arg_regsare stored inbpf_func_state. This separation keeps the stack arg area from interfering with the normal stack and frame pointer (r10) bookkeeping. Similar tostacksafe(), introducestack_arg_safe()to do pruning check.
Quoted 10 lines
A per-state bitmask
out_stack_arg_masktracks which outgoing stack arg slots have been written on the current path. Each bit corresponds to an outgoing slot index (bit 0 = r11-8 = arg6, bit 1 = r11-16 = arg7, etc.). At a call site, the verifier checks that all slots required by the callee have their corresponding mask bits set. This enables precise per-path tracking: if one branch of a conditional writes arg6 but another does not, the mask correctly reflects the difference and the verifier rejects the uninitialized path. The mask is included instack_arg_safe()so that states with different sets of initialized slots are not incorrectly pruned together.
you didn't address my comments.
pw-bot: cr
Yonghong Song
reviewerQuoted 73 lines · Alexei Starovoitov
Quoted 62 lines · Yonghong Song
Currently BPF functions (subprogs) are limited to 5 register arguments. With [1], the compiler can emit code that passes additional arguments via a dedicated stack area through bpf register
BPF_REG_PARAMS(r11), introduced in the previous patch.The compiler uses positive r11 offsets for incoming (callee-side) args and negative r11 offsets for outgoing (caller-side) args, following the x86_64/arm64 calling convention direction. There is an 8-byte gap at offset 0 separating the two regions:
Incoming (callee reads): r11+8 (arg6), r11+16 (arg7), ... Outgoing (caller writes): r11-8 (arg6), r11-16 (arg7), ...The following is an example to show how stack arguments are saved and transferred between caller and callee:
int foo(int a1, int a2, int a3, int a4, int a5, int a6, int a7) { ... bar(a1, a2, a3, a4, a5, a6, a7, a8); ... }Caller (foo) Callee (bar) ============ ============ Incoming (positive offsets): Incoming (positive offsets):r11+8: [incoming arg 6] r11+8: [incoming arg 6] <-+ r11+16: [incoming arg 7] r11+16: [incoming arg 7] <-|+ r11+24: [incoming arg 8] <-||+ Outgoing (negative offsets): ||| r11-8: [outgoing arg 6 to bar] -------->-------------------------+|| r11-16: [outgoing arg 7 to bar] -------->--------------------------+| r11-24: [outgoing arg 8 to bar] -------->---------------------------+If the bpf function has more than one call:
int foo(int a1, int a2, int a3, int a4, int a5, int a6, int a7) { ... bar1(a1, a2, a3, a4, a5, a6, a7, a8); ... bar2(a1, a2, a3, a4, a5, a6, a7, a8, a9); ... }Caller (foo) Callee (bar2) ============ ============== Incoming (positive offsets): Incoming (positive offsets):r11+8: [incoming arg 6] r11+8: [incoming arg 6] <+ r11+16: [incoming arg 7] r11+16: [incoming arg 7] <|+ r11+24: [incoming arg 8] <||+ Outgoing for bar2 (negative offsets): r11+32: [incoming arg 9] <|||+ r11-8: [outgoing arg 6] ---->----------->-------------------------+||| r11-16: [outgoing arg 7] ---->----------->--------------------------+|| r11-24: [outgoing arg 8] ---->----------->---------------------------+| r11-32: [outgoing arg 9] ---->----------->----------------------------+The verifier tracks stack arguments separately from the regular r10 stack. The
stack_arg_regsare stored inbpf_func_state. This separation keeps the stack arg area from interfering with the normal stack and frame pointer (r10) bookkeeping. Similar tostacksafe(), introducestack_arg_safe()to do pruning check.Quoted 10 lines
A per-state bitmask
out_stack_arg_masktracks which outgoing stack arg slots have been written on the current path. Each bit corresponds to an outgoing slot index (bit 0 = r11-8 = arg6, bit 1 = r11-16 = arg7, etc.). At a call site, the verifier checks that all slots required by the callee have their corresponding mask bits set. This enables precise per-path tracking: if one branch of a conditional writes arg6 but another does not, the mask correctly reflects the difference and the verifier rejects the uninitialized path. The mask is included instack_arg_safe()so that states with different sets of initialized slots are not incorrectly pruned together.
Quoted 1 line
you didn't address my comments.
After sending out v7, I then found you have some comments for v6. I will address all issues in the next revision.
Quoted 3 lines
pw-bot: cr