bpf, x86: Fix encoding for lower 8-bit registers in BPF_STX BPF_B
Subsystems (7)
PATCH 1/2 bpf, x86: Fix encoding for lower 8-bit registers in BPF_STX BPF_B
This patch fixes an encoding bug in emit_stx for BPF_B when the source register is BPF_REG_FP.
The current implementation for BPF_STX BPF_B in emit_stx saves one REX byte when the operands can be encoded using Mod-R/M alone. The lower 8 bits of registers %rax, %rbx, %rcx, and %rdx can be accessed without using a REX prefix via %al, %bl, %cl, and %dl, respectively. Other registers, (e.g., %rsi, %rdi, %rbp, %rsp) require a REX prefix to use their 8-bit equivalents (%sil, %dil, %bpl, %spl).
The current code checks if the source for BPF_STX BPF_B is BPF_REG_1 or BPF_REG_2 (which map to %rdi and %rsi), in which case it emits the required REX prefix. However, it misses the case when the source is BPF_REG_FP (mapped to %rbp).
The result is that BPF_STX BPF_B with BPF_REG_FP as the source operand will read from register %ch instead of the correct %bpl. This patch fixes the problem by fixing and refactoring the check on which registers need the extra REX byte. Since no BPF registers map to %rsp, there is no need to handle %spl.
--- arch/x86/net/bpf_jit_comp.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
arch/x86/net/bpf_jit_comp.c +15 -3
@@ -158,6 +158,19 @@ static bool is_ereg(u32 reg) BIT(BPF_REG_AX)); } +/* +* is_ereg_8l() == true if BPF register 'reg' is mapped to access x86-64 +* lower 8-bit registers dil,sil,bpl,spl,r8b..r15b, which need extra byte +* of encoding. al,cl,dl,bl have simpler encoding. +*/ +static bool is_ereg_8l (u32 reg) +{ +return is_ereg (reg) || + (1 << reg) & (BIT (BPF_REG_1 ) | +BIT (BPF_REG_2 ) | +BIT (BPF_REG_FP )); +} +static bool is_axreg (u32 reg) {return reg ==BPF_REG_0 ;
@@ -598,9 +611,8 @@ static void emit_stx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)switch (size) {case BPF_B :/* Emit 'mov byte ptr [rax + off], al' */ -if (is_ereg (dst_reg) ||is_ereg (src_reg) || -/* We have to add extra byte for x86 SIL, DIL regs */ - src_reg ==BPF_REG_1 || src_reg ==BPF_REG_2 ) +if (is_ereg (dst_reg) ||is_ereg_8l (src_reg)) +/* Add extra byte for eregs or SIL,DIL,BPL in src_reg */ EMIT2 (add_2mod (0x40 , dst_reg, src_reg),0x88 );else EMIT1 (0x88 );
Signature · 1 line
2.17.1
PATCH 2/2 bpf, selftests: Add test for BPF_STX BPF_B storing R10
This patch adds a test to test_verifier that writes the lower 8 bits of R10 (aka FP) using BPF_B to an array map and reads the result back. The expected behavior is that the result should be the same as first copying R10 to R9, and then storing / loading the lower 8 bits of R9.
This test catches a bug that was present in the x86-64 JIT that caused an incorrect encoding for BPF_STX BPF_B when the source operand is R10.
--- .../selftests/bpf/verifier/stack_ptr.c | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+)
tools/testing/selftests/bpf/verifier/stack_ptr.c +40 -0
@@ -315,3 +315,43 @@ }, .result = ACCEPT, }, +{ + "store PTR_TO_STACK in R10 to array map using BPF_B", + .insns = { +/* Load pointer to map. */ +BPF_MOV64_REG (BPF_REG_2 ,BPF_REG_10 ), +BPF_ALU64_IMM (BPF_ADD ,BPF_REG_2 ,-8 ), +BPF_ST_MEM (BPF_DW ,BPF_REG_2 ,0 ,0 ), +BPF_LD_MAP_FD (BPF_REG_1 ,0 ), +BPF_EMIT_CALL (BPF_FUNC_map_lookup_elem), +BPF_JMP_IMM (BPF_JNE ,BPF_REG_0 ,0 ,2 ), +BPF_MOV64_IMM (BPF_REG_0 ,2 ), +BPF_EXIT_INSN (), +BPF_MOV64_REG (BPF_REG_1 ,BPF_REG_0 ), +/* Copy R10 to R9. */ +BPF_MOV64_REG (BPF_REG_9 ,BPF_REG_10 ), +/* Pollute other registers with unaligned values. */ +BPF_MOV64_IMM (BPF_REG_2 ,-1 ), +BPF_MOV64_IMM (BPF_REG_3 ,-1 ), +BPF_MOV64_IMM (BPF_REG_4 ,-1 ), +BPF_MOV64_IMM (BPF_REG_5 ,-1 ), +BPF_MOV64_IMM (BPF_REG_6 ,-1 ), +BPF_MOV64_IMM (BPF_REG_7 ,-1 ), +BPF_MOV64_IMM (BPF_REG_8 ,-1 ), +/* Store both R9 and R10 with BPF_B and read back. */ +BPF_STX_MEM (BPF_B ,BPF_REG_1 ,BPF_REG_10 ,0 ), +BPF_LDX_MEM (BPF_B ,BPF_REG_2 ,BPF_REG_1 ,0 ), +BPF_STX_MEM (BPF_B ,BPF_REG_1 ,BPF_REG_9 ,0 ), +BPF_LDX_MEM (BPF_B ,BPF_REG_3 ,BPF_REG_1 ,0 ), +/* Should read back as same value. */ +BPF_JMP_REG (BPF_JEQ ,BPF_REG_2 ,BPF_REG_3 ,2 ), +BPF_MOV64_IMM (BPF_REG_0 ,1 ), +BPF_EXIT_INSN (), +BPF_MOV64_IMM (BPF_REG_0 ,42 ), + BPF_EXIT_INSN(), + }, + .fixup_map_array_48b = { 3 }, + .result = ACCEPT, + .retval = 42, + .prog_type = BPF_PROG_TYPE_SCHED_CLS, +},
Signature · 1 line
2.17.1
Alexei Starovoitov
AppliedQuoted 22 lines · Luke Nelson
This patch fixes an encoding bug in
emit_stxforBPF_Bwhen the source register isBPF_REG_FP.The current implementation for
BPF_STXBPF_Binemit_stxsaves one REX byte when the operands can be encoded using Mod-R/M alone. The lower 8 bits of registers %rax, %rbx, %rcx, and %rdx can be accessed without using a REX prefix via %al, %bl, %cl, and %dl, respectively. Other registers, (e.g., %rsi, %rdi, %rbp, %rsp) require a REX prefix to use their 8-bit equivalents (%sil, %dil, %bpl, %spl).The current code checks if the source for
BPF_STXBPF_BisBPF_REG_1orBPF_REG_2(which map to %rdi and %rsi), in which case it emits the required REX prefix. However, it misses the case when the source isBPF_REG_FP(mapped to %rbp).The result is that
BPF_STXBPF_BwithBPF_REG_FPas the source operand will read from register %ch instead of the correct %bpl. This patch fixes the problem by fixing and refactoring the check on which registers need the extra REX byte. Since no BPF registers map to %rsp, there is no need to handle %spl.
Quoted 3 lines
Fixes 622582786c9e0 ("net: filter: x86: internal BPF JIT")Signed-off-by Xi Wang <[email protected]>Signed-off-by Luke Nelson <[email protected]>
Applied. Thanks for the fix.
It's questionable whether the verifier should have allowed such insn in the first place, but JIT fix is good regardless.