sparc: bpf_jit: fix loads from negative offsets
Subsystems (5)
PATCH 1/1 sparc: bpf_jit: fix loads from negative offsets
- fix BPF_LD|ABS|IND from negative offsets:
make sure to sign extend lower 32 bits in 64-bit register before calling C helpers from JITed code, otherwise 'int k' argument of bpf_internal_load_pointer_neg_helper() function will be added as large unsigned integer, causing packet size check to trigger and abort the program.
It's worth noting that JITed code for 'A = A op K' will affect upper 32 bits differently depending whether K is simm13 or not. Since small constants are sign extended, whereas large constants are stored in temp register and zero extended.
That is ok and we don't have to pay a penalty of sign extension for every sethi, since all classic BPF instructions have 32-bit semantics and we only need to set correct upper bits when transitioning from JITed code into C.
- though instructions 'A &= 0' and 'A *= 0' are odd, JIT compiler should not optimize them out
---
More detailed explanation of 1st bug:
BPF_STMT(BPF_LD | BPF_B | BPF_ABS, SKF_LL_OFF)
will be JITed as:
sethi %hi(0xffe00000), %g3
or %g3, 0, %g3
call bpf_jit_load_byte_negative_offset
which will do:
mov %g3, %o1
call bpf_internal_load_pointer_neg_helper
mov 1, %o2
void *bpf_internal_load_pointer_neg_helper(const struct sk_buff *skb, int k, unsigned int size) {
...
ptr = skb_mac_header(skb) + k - SKF_LL_OFF;
here 'k' will be added as 0xffe00000 instead of SKF_LL_OFF == -0x200000
After these fixes, test_bpf is now clean on sparc64. Compile tested only on sparc32
arch/sparc/net/bpf_jit_asm.S | 3 +++ arch/sparc/net/bpf_jit_comp.c | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-)
arch/sparc/net/bpf_jit_asm.S +3 -0
@@ -6,10 +6,12 @@ #define SAVE_SZ 176 #define SCRATCH_OFF STACK_BIAS + 128 #define BE_PTR(label) be,pn %xcc, label +#define SIGN_EXTEND(reg) sra reg, 0, reg #else #define SAVE_SZ 96 #define SCRATCH_OFF 72 #define BE_PTR(label) be label +#define SIGN_EXTEND(reg) #endif #define SKF_MAX_NEG_OFF (-0x200000) /* SKF_LL_OFF from filter.h */
@@ -135,6 +137,7 @@ bpf_slow_path_byte_msh: save %sp, -SAVE_SZ, %sp; \ mov %i0, %o0; \ mov r_OFF, %o1; \ + SIGN_EXTEND(%o1); \ call bpf_internal_load_pointer_neg_helper; \ mov (LEN), %o2; \ mov %o0, r_TMP; \
arch/sparc/net/bpf_jit_comp.c +1 -1
@@ -184,7 +184,7 @@ do { \*/ #define emit_alu_K (OPCODE ,K ) \ do { \ - if (K) { \ + if (K || OPCODE == AND || OPCODE == MUL) { \ unsigned int _insn = OPCODE; \ _insn |= RS1(r_A) | RD(r_A); \ if (is_simm13(K)) { \
Signature · 1 line
1.7.9.5
From: Alexei Starovoitov <[email protected]>
Date: Tue, 23 Sep 2014 13:50:10 -0700
Quoted 16 lines
- fix BPF_LD|ABS|IND from negative offsets:
make sure to sign extend lower 32 bits in 64-bit register before calling C helpers from JITed code, otherwise 'int k' argument ofbpf_internal_load_pointer_neg_helper()function will be added as large unsigned integer, causing packet size check to trigger and abort the program.It's worth noting that JITed code for 'A = A op K' will affect upper 32 bits differently depending whether K is simm13 or not. Since small constants are sign extended, whereas large constants are stored in temp register and zero extended.
That is ok and we don't have to pay a penalty of sign extension for every sethi, since all classic BPF instructions have 32-bit semantics and we only need to set correct upper bits when transitioning from JITed code into C.
Quoted 4 lines
- though instructions 'A &= 0' and 'A *= 0' are odd, JIT compiler should not optimize them out
Signed-off-by Alexei Starovoitov <[email protected]>
Applied, thanks for fixing this.
What we could do longer-term is change is_simm13() into is_simm12() and therefore know that all constants loaded are 32-bit zero extended regardless of size.
BTW, there is a trick to load arbitrary negative 32-bit constants sign extended to 64-bit, in just two instructions:
sethi %(~value), reg xor reg, ~(0x400 | (value & 0x3ff)), reg
Or something like that, it's been a while. GCC knows how to emit that sequence too.
Anyways, we could adjust the JIT to do that too.
Quoted 9 lines · David Miller
BTW, there is a trick to load arbitrary negative 32-bit constants sign extended to 64-bit, in just two instructions:
sethi %(~value), reg
xor reg, ~(0x400 | (value &0x3ff)), regOr something like that, it's been a while. GCC knows how to emit that sequence too.
nice, that's a neat trick.
gcc computes it as:
sethi %(~value), reg
xor reg, -0x400 | (value & 0x3ff), reg
and llvm as:
sethi %(~value), reg
xor reg, ~(~value & 0x3ff), reg
both are equivalent. I'll try to teach JIT to emit this.
Thanks!