bpf, x86_32: Fix incorrect encoding in BPF_LDX zero-extension
Subsystems (5)
PATCH v2 1/2 bpf, x86_32: Fix incorrect encoding in BPF_LDX zero-extension
The current JIT uses the following sequence to zero-extend into the upper 32 bits of the destination register for BPF_LDX BPF_{B,H,W}, when the destination register is not on the stack:
EMIT3(0xC7, add_1reg(0xC0, dst_hi), 0);
The problem is that C7 /0 encodes a MOV instruction that requires a 4-byte immediate; the current code emits only 1 byte of the immediate. This means that the first 3 bytes of the next instruction will be treated as the rest of the immediate, breaking the stream of instructions.
This patch fixes the problem by instead emitting "xor dst_hi,dst_hi" to clear the upper 32 bits. This fixes the problem and is more efficient than using MOV to load a zero immediate.
This bug may not be currently triggerable as BPF_REG_AX is the only register not stored on the stack and the verifier uses it in a limited way, and the verifier implements a zero-extension optimization. But the JIT should avoid emitting incorrect encodings regardless.
---
v1 -> v2: Updated commit message to better reflect the bug.
(H. Peter Anvin and Brian Gerst)
--- arch/x86/net/bpf_jit_comp32.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/net/bpf_jit_comp32.c +3 -1
@@ -1854,7 +1854,9 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, STACK_VAR(dst_hi));EMIT (0x0 ,4 ); }else { -EMIT3 (0xC7 ,add_1reg (0xC0 , dst_hi),0 ); +/* xor dst_hi,dst_hi */ +EMIT2 (0x33 , +add_2reg (0xC0 , dst_hi, dst_hi)); }break ;case BPF_DW :
Signature · 1 line
2.17.1
PATCH v2 2/2 bpf, x86_32: Fix clobbering of dst for BPF_JSET
The current JIT clobbers the destination register for BPF_JSET BPF_X and BPF_K by using "and" and "or" instructions. This is fine when the destination register is a temporary loaded from a register stored on the stack but not otherwise.
This patch fixes the problem (for both BPF_K and BPF_X) by always loading the destination register into temporaries since BPF_JSET should not modify the destination register.
This bug may not be currently triggerable as BPF_REG_AX is the only register not stored on the stack and the verifier uses it in a limited way.
---
v1 -> v2: No changes.
--- arch/x86/net/bpf_jit_comp32.c | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-)
arch/x86/net/bpf_jit_comp32.c +18 -4
@@ -2015,8 +2015,8 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image,case BPF_JMP |BPF_JSET |BPF_X :case BPF_JMP32 |BPF_JSET |BPF_X : {bool is_jmp64 =BPF_CLASS (insn->code) ==BPF_JMP ; -u8 dreg_lo = dstk ?IA32_EAX : dst_lo; -u8 dreg_hi = dstk ?IA32_EDX : dst_hi; +u8 dreg_lo =IA32_EAX ; +u8 dreg_hi =IA32_EDX ;u8 sreg_lo = sstk ?IA32_ECX : src_lo;u8 sreg_hi = sstk ?IA32_EBX : src_hi;
@@ -2028,6 +2028,13 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image,add_2reg (0x40 ,IA32_EBP ,IA32_EDX ), STACK_VAR(dst_hi)); + }else { +/* mov dreg_lo,dst_lo */ +EMIT2 (0x89 ,add_2reg (0xC0 , dreg_lo, dst_lo)); +if (is_jmp64) +/* mov dreg_hi,dst_hi */ +EMIT2 (0x89 , +add_2reg (0xC0 , dreg_hi, dst_hi)); }if (sstk) {
@@ -2052,8 +2059,8 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image,case BPF_JMP |BPF_JSET |BPF_K :case BPF_JMP32 |BPF_JSET |BPF_K : {bool is_jmp64 =BPF_CLASS (insn->code) ==BPF_JMP ; -u8 dreg_lo = dstk ?IA32_EAX : dst_lo; -u8 dreg_hi = dstk ?IA32_EDX : dst_hi; +u8 dreg_lo =IA32_EAX ; +u8 dreg_hi =IA32_EDX ;u8 sreg_lo =IA32_ECX ;u8 sreg_hi =IA32_EBX ;u32 hi;
@@ -2066,6 +2073,13 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image,add_2reg (0x40 ,IA32_EBP ,IA32_EDX ), STACK_VAR(dst_hi)); + }else { +/* mov dreg_lo,dst_lo */ +EMIT2 (0x89 ,add_2reg (0xC0 , dreg_lo, dst_lo)); +if (is_jmp64) +/* mov dreg_hi,dst_hi */ +EMIT2 (0x89 , +add_2reg (0xC0 , dreg_hi, dst_hi)); }/* mov ecx,imm32 */
Signature · 1 line
2.17.1
Re: [PATCH bpf v2 2/2] bpf, x86_32: Fix clobbering of dst for BPF_JSET
Quoted 9 lines · Luke Nelson
The current JIT clobbers the destination register for
BPF_JSETBPF_XandBPF_Kby using "and" and "or" instructions. This is fine when the destination register is a temporary loaded from a register stored on the stack but not otherwise.This patch fixes the problem (for both
BPF_KandBPF_X) by always loading the destination register into temporaries sinceBPF_JSETshould not modify the destination register.
Quoted 7 lines
This bug may not be currently triggerable as
BPF_REG_AXis the only register not stored on the stack and the verifier uses it in a limited way.Fixes 03f5781be2c7b ("bpf, x86_32: add eBPF JIT compiler for ia32")Signed-off-by Xi Wang <[email protected]>Signed-off-by Luke Nelson <[email protected]>
Quoted 15 lines · Luke Nelson
The current JIT uses the following sequence to zero-extend into the upper 32 bits of the destination register for
BPF_LDXBPF_{B,H,W}, when the destination register is not on the stack:EMIT3(0xC7, add_1reg(0xC0, dst_hi), 0);The problem is that C7 /0 encodes a MOV instruction that requires a 4-byte immediate; the current code emits only 1 byte of the immediate. This means that the first 3 bytes of the next instruction will be treated as the rest of the immediate, breaking the stream of instructions.
This patch fixes the problem by instead emitting "xor dst_hi,dst_hi" to clear the upper 32 bits. This fixes the problem and is more efficient than using MOV to load a zero immediate.
Quoted 8 lines
This bug may not be currently triggerable as
BPF_REG_AXis the only register not stored on the stack and the verifier uses it in a limited way, and the verifier implements a zero-extension optimization. But the JIT should avoid emitting incorrect encodings regardless.Fixes 03f5781be2c7b ("bpf, x86_32: add eBPF JIT compiler for ia32")Signed-off-by Xi Wang <[email protected]>Signed-off-by Luke Nelson <[email protected]>
Quoted 48 lines · Luke Nelson
The current JIT uses the following sequence to zero-extend into the upper 32 bits of the destination register for
BPF_LDXBPF_{B,H,W}, when the destination register is not on the stack:
EMIT3(0xC7, add_1reg(0xC0, dst_hi), 0);The problem is that C7 /0 encodes a MOV instruction that requires a 4-byte
immediate; the current code emits only 1 byte of the immediate. This means that the first 3 bytes of the next instruction will be treated as the rest of the immediate, breaking the stream of instructions.This patch fixes the problem by instead emitting "xor dst_hi,dst_hi" to clear the upper 32 bits. This fixes the problem and is more efficient
than using MOV to load a zero immediate.This bug may not be currently triggerable as
BPF_REG_AXis the only register not stored on the stack and the verifier uses it in a limited way, and the verifier implements a zero-extension optimization. But the JIT should avoid emitting incorrect encodings regardless.Fixes 03f5781be2c7b ("bpf, x86_32: add eBPF JIT compiler for ia32")Signed-off-by Xi Wang <[email protected]>Signed-off-by Luke Nelson <[email protected]>---
v1 -> v2: Updated commit message to better reflect the bug.
(H. Peter Anvin and Brian Gerst)
---arch/x86/net/bpf_jit_comp32.c| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)arch/x86/net/bpf_jit_comp32.c +3
-1@@ -1854,7 +1854,9 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, STACK_VAR(dst_hi));EMIT (0x0 ,4 ); }else { -EMIT3 (0xC7 ,add_1reg (0xC0 , dst_hi),0 ); +/* xor dst_hi,dst_hi */ +EMIT2 (0x33 , +add_2reg (0xC0 , dst_hi, dst_hi)); }break ;
Quoted 1 line
case
BPF_DW:
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Alexei Starovoitov
AppliedQuoted 16 lines · Luke Nelson
The current JIT uses the following sequence to zero-extend into the upper 32 bits of the destination register for
BPF_LDXBPF_{B,H,W}, when the destination register is not on the stack:EMIT3(0xC7, add_1reg(0xC0, dst_hi), 0);The problem is that C7 /0 encodes a MOV instruction that requires a 4-byte immediate; the current code emits only 1 byte of the immediate. This means that the first 3 bytes of the next instruction will be treated as the rest of the immediate, breaking the stream of instructions.
This patch fixes the problem by instead emitting "xor dst_hi,dst_hi" to clear the upper 32 bits. This fixes the problem and is more efficient than using MOV to load a zero immediate.
Quoted 8 lines
This bug may not be currently triggerable as
BPF_REG_AXis the only register not stored on the stack and the verifier uses it in a limited way, and the verifier implements a zero-extension optimization. But the JIT should avoid emitting incorrect encodings regardless.Fixes 03f5781be2c7b ("bpf, x86_32: add eBPF JIT compiler for ia32")Signed-off-by Xi Wang <[email protected]>Signed-off-by Luke Nelson <[email protected]>
Applied. Thanks