Skip to contentSkip to conversation

Limit tail calls to 33 in all JIT compilers

PATCH Applied Paul Chaignon 6 participants 13 messages Dec 9, 2019 – Dec 18, 2019

Subsystems (8)
Networking Network drivers MIPS BPF +4 more subsystems
lore
Share & tools
lore
#0 lore raw

cover Limit tail calls to 33 in all JIT compilers

The BPF interpreter and all JIT compilers, except RISC-V's and MIPS', enforce a 33-tail calls limit at runtime. Because of this discrepancy, a BPF program can have a different behavior and output depending on whether it is interpreted or JIT compiled, or depending on the underlying architecture.

This patchset changes the RISC-V and MIPS JIT compilers to limit tail calls to 33 instead of 32. I have checked other BPF JIT compilers for the same discrepancy.

Paul Chaignon (2):
bpf, riscv: limit to 33 tail calls
bpf, mips: limit to 33 tail calls

 arch/mips/net/ebpf_jit.c      | 9 +++++----
 arch/riscv/net/bpf_jit_comp.c | 4 ++--
 2 files changed, 7 insertions(+), 6 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/2 bpf, riscv: limit to 33 tail calls

All BPF JIT compilers except RISC-V's and MIPS' enforce a 33-tail calls limit at runtime. In addition, a test was recently added, in tailcalls2, to check this limit.

This patch updates the tail call limit in RISC-V's JIT compiler to allow 33 tail calls. I tested it using the above selftest on an emulated RISCV64.

Fixes 2353ecc6f91f ("bpf, riscv: add BPF JIT for RV64G")
---
 arch/riscv/net/bpf_jit_comp.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/riscv/net/bpf_jit_comp.c +2 -2
index 5451ef3845f2..7fbf56aab661 100644
@@ -631,14 +631,14 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx)
 		return -1;
 	emit(rv_bgeu(RV_REG_A2, RV_REG_T1, off >> 1), ctx);
 
-	/* if (--TCC < 0)
+	/* if (TCC-- < 0)
 	 *     goto out;
 	 */
 	emit(rv_addi(RV_REG_T1, tcc, -1), ctx);
 	off = (tc_ninsn - (ctx->ninsns - start_insn)) << 2;
 	if (is_13b_check(off, insn))
 		return -1;
-	emit(rv_blt(RV_REG_T1, RV_REG_ZERO, off >> 1), ctx);
+	emit(rv_blt(tcc, RV_REG_ZERO, off >> 1), ctx);
 
 	/* prog = array->ptrs[index];
 	 * if (!prog)
Signature · 1 line
2.17.1
#5 lore raw

PATCH 2/2 bpf, mips: limit to 33 tail calls

In reply to Paul Chaignon

All BPF JIT compilers except RISC-V's and MIPS' enforce a 33-tail calls limit at runtime. In addition, a test was recently added, in tailcalls2, to check this limit.

This patch updates the tail call limit in MIPS' JIT compiler to allow 33 tail calls.

Fixes b6bd53f9c4e8 ("MIPS: Add missing file for eBPF JIT.")
---
 arch/mips/net/ebpf_jit.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
arch/mips/net/ebpf_jit.c +5 -4
index 46b76751f3a5..3ec69d9cbe88 100644
@@ -604,6 +604,7 @@ static void emit_const_to_reg(struct jit_ctx *ctx, int dst, u64 value)
 static int emit_bpf_tail_call(struct jit_ctx *ctx, int this_idx)
 {
 	int off, b_off;
+	int tcc_reg;
 
 	ctx->flags |= EBPF_SEEN_TC;
 	/*
@@ -616,14 +617,14 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx, int this_idx)
 	b_off = b_imm(this_idx + 1, ctx);
 	emit_instr(ctx, bne, MIPS_R_AT, MIPS_R_ZERO, b_off);
 	/*
-	 * if (--TCC < 0)
+	 * if (TCC-- < 0)
 	 *     goto out;
 	 */
 	/* Delay slot */
-	emit_instr(ctx, daddiu, MIPS_R_T5,
-		   (ctx->flags & EBPF_TCC_IN_V1) ? MIPS_R_V1 : MIPS_R_S4, -1);
+	tcc_reg = (ctx->flags & EBPF_TCC_IN_V1) ? MIPS_R_V1 : MIPS_R_S4;
+	emit_instr(ctx, daddiu, MIPS_R_T5, tcc_reg, -1);
 	b_off = b_imm(this_idx + 1, ctx);
-	emit_instr(ctx, bltz, MIPS_R_T5, b_off);
+	emit_instr(ctx, bltz, tcc_reg, b_off);
 	/*
 	 * prog = array->ptrs[index];
 	 * if (prog == NULL)
Signature · 1 line
2.17.1
Acked-by on v1 1/2
#2 lore raw

Re: [PATCH bpf 1/2] bpf, riscv: limit to 33 tail calls

In reply to Paul Chaignon
Quoted 9 lines · Paul Chaignon

All BPF JIT compilers except RISC-V's and MIPS' enforce a 33-tail calls limit at runtime. In addition, a test was recently added, in tailcalls2, to check this limit.

This patch updates the tail call limit in RISC-V's JIT compiler to allow 33 tail calls. I tested it using the above selftest on an emulated RISCV64.

33! ICK! ;-) Thanks for finding this!

Quoted 31 lines
Fixes 2353ecc6f91f ("bpf, riscv: add BPF JIT for RV64G")
---
 arch/riscv/net/bpf_jit_comp.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/riscv/net/bpf_jit_comp.c +2 -3
index 5451ef3845f2..7fbf56aab661 100644
@@ -631,14 +631,14 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx)
                return -1;
        emit(rv_bgeu(RV_REG_A2, RV_REG_T1, off >> 1), ctx);
 
-       /* if (--TCC < 0)
+       /* if (TCC-- < 0)
         *     goto out;
         */
        emit(rv_addi(RV_REG_T1, tcc, -1), ctx);
        off = (tc_ninsn - (ctx->ninsns - start_insn)) << 2;
        if (is_13b_check(off, insn))
                return -1;
-       emit(rv_blt(RV_REG_T1, RV_REG_ZERO, off >> 1), ctx);
+       emit(rv_blt(tcc, RV_REG_ZERO, off >> 1), ctx);
 
        /* prog = array->ptrs[index];
         * if (!prog)
--

2.17.1

#3 lore raw

Re: [PATCH bpf 1/2] bpf, riscv: limit to 33 tail calls

Quoted 7 lines · Björn Töpel
Quoted 5 lines · Paul Chaignon

All BPF JIT compilers except RISC-V's and MIPS' enforce a 33-tail calls limit at runtime. In addition, a test was recently added, in tailcalls2, to check this limit.

Quoted 9 lines
Quoted 4 lines

This patch updates the tail call limit in RISC-V's JIT compiler to allow 33 tail calls. I tested it using the above selftest on an emulated RISCV64.

33! ICK! ;-) Thanks for finding this!

...and somewhat related; One of the tailcall tests fail due to missing far-branch support in the emitter. I'll address this in the v2 of the "far branch" series.

#4 lore raw

Re: [PATCH bpf 1/2] bpf, riscv: limit to 33 tail calls

In reply to Björn Töpel
Quoted 6 lines · Björn Töpel
Quoted 5 lines · Paul Chaignon

All BPF JIT compilers except RISC-V's and MIPS' enforce a 33-tail calls limit at runtime. In addition, a test was recently added, in tailcalls2, to check this limit.

Quoted 6 lines
Quoted 4 lines

This patch updates the tail call limit in RISC-V's JIT compiler to allow 33 tail calls. I tested it using the above selftest on an emulated RISCV64.

33! ICK! ;-) Thanks for finding this!

Actually, Mahshid found it during her internship because she wanted to check that the number of tail calls was limited. And now I feel so naive for trusting the doc...

Quoted 34 lines
Quoted 31 lines
Fixes 2353ecc6f91f ("bpf, riscv: add BPF JIT for RV64G")
---
 arch/riscv/net/bpf_jit_comp.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/riscv/net/bpf_jit_comp.c +2 -3
index 5451ef3845f2..7fbf56aab661 100644
@@ -631,14 +631,14 @@ static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx)
                return -1;
        emit(rv_bgeu(RV_REG_A2, RV_REG_T1, off >> 1), ctx);
 
-       /* if (--TCC < 0)
+       /* if (TCC-- < 0)
         *     goto out;
         */
        emit(rv_addi(RV_REG_T1, tcc, -1), ctx);
        off = (tc_ninsn - (ctx->ninsns - start_insn)) << 2;
        if (is_13b_check(off, insn))
                return -1;
-       emit(rv_blt(RV_REG_T1, RV_REG_ZERO, off >> 1), ctx);
+       emit(rv_blt(tcc, RV_REG_ZERO, off >> 1), ctx);
 
        /* prog = array->ptrs[index];
         * if (!prog)
--

2.17.1

Acked-by on the v1 series
#11 lore raw
In reply to Paul Chaignon
Quoted 9 lines · Paul Chaignon

The BPF interpreter and all JIT compilers, except RISC-V's and MIPS', enforce a 33-tail calls limit at runtime. Because of this discrepancy, a BPF program can have a different behavior and output depending on whether it is interpreted or JIT compiled, or depending on the underlying architecture.

This patchset changes the RISC-V and MIPS JIT compilers to limit tail calls to 33 instead of 32. I have checked other BPF JIT compilers for the same discrepancy.

maintainer Acked-by on v1 2/2
#6 lore raw

Re: [PATCH bpf 2/2] bpf, mips: limit to 33 tail calls

In reply to Paul Chaignon

Hi Paul,

Quoted 10 lines · Paul Chaignon

All BPF JIT compilers except RISC-V's and MIPS' enforce a 33-tail calls limit at runtime. In addition, a test was recently added, in tailcalls2, to check this limit.

This patch updates the tail call limit in MIPS' JIT compiler to allow 33 tail calls.

Fixes b6bd53f9c4e8 ("MIPS: Add missing file for eBPF JIT.")

I'd be happy to take this through mips-fixes, but equally happy for it to go through the BPF/net trees in which case:

Thanks,
Paul

Quoted 38 lines
---
 arch/mips/net/ebpf_jit.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
arch/mips/net/ebpf_jit.c +5 -4
index 46b76751f3a5..3ec69d9cbe88 100644
@@ -604,6 +604,7 @@ static void emit_const_to_reg(struct jit_ctx *ctx, int dst, u64 value)
 static int emit_bpf_tail_call(struct jit_ctx *ctx, int this_idx)
 {
 	int off, b_off;
+	int tcc_reg;
 
 	ctx->flags |= EBPF_SEEN_TC;
 	/*
@@ -616,14 +617,14 @@ static int emit_bpf_tail_call(struct jit_ctx *ctx, int this_idx)
 	b_off = b_imm(this_idx + 1, ctx);
 	emit_instr(ctx, bne, MIPS_R_AT, MIPS_R_ZERO, b_off);
 	/*
-	 * if (--TCC < 0)
+	 * if (TCC-- < 0)
 	 *     goto out;
 	 */
 	/* Delay slot */
-	emit_instr(ctx, daddiu, MIPS_R_T5,
-		   (ctx->flags & EBPF_TCC_IN_V1) ? MIPS_R_V1 : MIPS_R_S4, -1);
+	tcc_reg = (ctx->flags & EBPF_TCC_IN_V1) ? MIPS_R_V1 : MIPS_R_S4;
+	emit_instr(ctx, daddiu, MIPS_R_T5, tcc_reg, -1);
 	b_off = b_imm(this_idx + 1, ctx);
-	emit_instr(ctx, bltz, MIPS_R_T5, b_off);
+	emit_instr(ctx, bltz, tcc_reg, b_off);
 	/*
 	 * prog = array->ptrs[index];
 	 * if (prog == NULL)
Signature · 1 line
2.17.1
maintainer Applied
#12 lore raw
In reply to Paul Chaignon
Quoted 10 lines · Paul Chaignon

The BPF interpreter and all JIT compilers, except RISC-V's and MIPS', enforce a 33-tail calls limit at runtime. Because of this discrepancy, a BPF program can have a different behavior and output depending on whether it is interpreted or JIT compiled, or depending on the underlying architecture.

This patchset changes the RISC-V and MIPS JIT compilers to limit tail calls to 33 instead of 32. I have checked other BPF JIT compilers for the same discrepancy.

Quoted 7 lines

Paul Chaignon (2):
bpf, riscv: limit to 33 tail calls
bpf, mips: limit to 33 tail calls

 arch/mips/net/ebpf_jit.c      | 9 +++++----
 arch/riscv/net/bpf_jit_comp.c | 4 ++--
 2 files changed, 7 insertions(+), 6 deletions(-)

Applied, thanks!

maintainer
#7 lore raw

Re: [PATCH bpf 2/2] bpf, mips: limit to 33 tail calls

In reply to Paul Burton
Quoted 8 lines · Paul Burton
Quoted 7 lines · Paul Chaignon

All BPF JIT compilers except RISC-V's and MIPS' enforce a 33-tail calls limit at runtime. In addition, a test was recently added, in tailcalls2, to check this limit.

This patch updates the tail call limit in MIPS' JIT compiler to allow 33 tail calls.

Quoted 6 lines
Quoted 3 lines
Fixes b6bd53f9c4e8 ("MIPS: Add missing file for eBPF JIT.")

I'd be happy to take this through mips-fixes, but equally happy for it to go through the BPF/net trees in which case:

We took the series via bpf, thanks!

#8 lore raw

Re: [PATCH bpf 2/2] bpf, mips: limit to 33 tail calls

In reply to Paul Burton

Paul Burton wrote 11.12.2019 02:23:

Quoted 9 lines

Hi Paul,

Quoted 6 lines · Paul Chaignon

All BPF JIT compilers except RISC-V's and MIPS' enforce a 33-tail calls limit at runtime. In addition, a test was recently added, in tailcalls2, to check this limit.

This patch updates the tail call limit in MIPS' JIT compiler to allow 33 tail calls.

Hi Paul,

You've restored MIPS cBPF in mips-fixes tree, doesn't it require any changes to limit tail calls to 33? This series includes only eBPF as there was no MIPS cBPF at the moment of writing.

Quoted 44 lines
Quoted 3 lines
Fixes b6bd53f9c4e8 ("MIPS: Add missing file for eBPF JIT.")

I'd be happy to take this through mips-fixes, but equally happy for it to go through the BPF/net trees in which case:

Thanks,
Paul

Quoted 32 lines
---
 arch/mips/net/ebpf_jit.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
arch/mips/net/ebpf_jit.c +1 -0
index 46b76751f3a5..3ec69d9cbe88 100644
@@ -604,6 +604,7 @@ static void emit_const_to_reg(struct jit_ctx *ctx, 
 int dst, u64 value)
 static int emit_bpf_tail_call(struct jit_ctx *ctx, int this_idx)
 {
 	int off, b_off;
+	int tcc_reg;
 
 	ctx->flags |= EBPF_SEEN_TC;
arch/mips/net/ebpf_jit.c +3 -3
 	/*
@@ -616,14 +617,14 @@ static int emit_bpf_tail_call(struct jit_ctx 
 *ctx, int this_idx)
 	b_off = b_imm(this_idx + 1, ctx);
 	emit_instr(ctx, bne, MIPS_R_AT, MIPS_R_ZERO, b_off);
 	/*
-	 * if (--TCC < 0)
+	 * if (TCC-- < 0)
 	 *     goto out;
 	 */
 	/* Delay slot */
-	emit_instr(ctx, daddiu, MIPS_R_T5,
-		   (ctx->flags & EBPF_TCC_IN_V1) ? MIPS_R_V1 : MIPS_R_S4, -1);
+	tcc_reg = (ctx->flags & EBPF_TCC_IN_V1) ? MIPS_R_V1 : MIPS_R_S4;
+	emit_instr(ctx, daddiu, MIPS_R_T5, tcc_reg, -1);
 	b_off = b_imm(this_idx + 1, ctx);
Quoted 8 lines
Quoted 8 lines
arch/mips/net/ebpf_jit.c +1 -1
-	emit_instr(ctx, bltz, MIPS_R_T5, b_off);
+	emit_instr(ctx, bltz, tcc_reg, b_off);
 	/*
 	 * prog = array->ptrs[index];

* if (prog == NULL)
--
2.17.1

Regards,
ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ

#9 lore raw

Re: [PATCH bpf 2/2] bpf, mips: limit to 33 tail calls

Quoted 10 lines · Alexander Lobakin

Paul Burton wrote 11.12.2019 02:23:

Quoted 9 lines

Hi Paul,

Quoted 6 lines · Paul Chaignon

All BPF JIT compilers except RISC-V's and MIPS' enforce a 33-tail calls
limit at runtime. In addition, a test was recently added, in tailcalls2,
to check this limit.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

This patch updates the tail call limit in MIPS' JIT compiler to allow 33 tail calls.

Hi Paul,

You've restored MIPS cBPF in mips-fixes tree, doesn't it require any changes to limit tail calls to 33? This series includes only eBPF as there was no MIPS cBPF at the moment of writing.

cBPF doesn't support tail calls or even the call instruction for helpers.

Paul C.

Quoted 56 lines
Quoted 45 lines
Quoted 4 lines
Fixes b6bd53f9c4e8 ("MIPS: Add missing file for eBPF JIT.")

I'd be happy to take this through mips-fixes, but equally happy for it to go through the BPF/net trees in which case:

Thanks,
Paul

Quoted 32 lines
---
 arch/mips/net/ebpf_jit.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
arch/mips/net/ebpf_jit.c +1 -0
index 46b76751f3a5..3ec69d9cbe88 100644
@@ -604,6 +604,7 @@ static void emit_const_to_reg(struct jit_ctx
 *ctx, int dst, u64 value)
 static int emit_bpf_tail_call(struct jit_ctx *ctx, int this_idx)
 {
 	int off, b_off;
+	int tcc_reg;
 
 	ctx->flags |= EBPF_SEEN_TC;
arch/mips/net/ebpf_jit.c +3 -3
 	/*
@@ -616,14 +617,14 @@ static int emit_bpf_tail_call(struct jit_ctx
 *ctx, int this_idx)
 	b_off = b_imm(this_idx + 1, ctx);
 	emit_instr(ctx, bne, MIPS_R_AT, MIPS_R_ZERO, b_off);
 	/*
-	 * if (--TCC < 0)
+	 * if (TCC-- < 0)
 	 *     goto out;
 	 */
 	/* Delay slot */
-	emit_instr(ctx, daddiu, MIPS_R_T5,
-		   (ctx->flags & EBPF_TCC_IN_V1) ? MIPS_R_V1 : MIPS_R_S4, -1);
+	tcc_reg = (ctx->flags & EBPF_TCC_IN_V1) ? MIPS_R_V1 : MIPS_R_S4;
+	emit_instr(ctx, daddiu, MIPS_R_T5, tcc_reg, -1);
 	b_off = b_imm(this_idx + 1, ctx);
Quoted 8 lines
Quoted 8 lines
arch/mips/net/ebpf_jit.c +1 -1
-	emit_instr(ctx, bltz, MIPS_R_T5, b_off);
+	emit_instr(ctx, bltz, tcc_reg, b_off);
 	/*
 	 * prog = array->ptrs[index];

* if (prog == NULL)
--
2.17.1

Regards,
ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ

#10 lore raw

Re: [PATCH bpf 2/2] bpf, mips: limit to 33 tail calls

Paul Chaignon wrote 18.12.2019 12:58:

Quoted 14 lines · Alexander Lobakin
Quoted 13 lines

Paul Burton wrote 11.12.2019 02:23:

Quoted 11 lines

Hi Paul,

Quoted 8 lines · Paul Chaignon

All BPF JIT compilers except RISC-V's and MIPS' enforce a 33-tail calls
limit at runtime. In addition, a test was recently added, in tailcalls2,
to check this limit.

This patch updates the tail call limit in MIPS' JIT compiler to allow 33 tail calls.

Quoted 8 lines
Quoted 5 lines

Hi Paul,

You've restored MIPS cBPF in mips-fixes tree, doesn't it require any changes to limit tail calls to 33? This series includes only eBPF as there was no MIPS cBPF at the moment of writing.

cBPF doesn't support tail calls or even the call instruction for helpers.

Aww, didn't know that. Thanks!

Quoted 58 lines

Paul C.

Quoted 56 lines
Quoted 45 lines
Quoted 4 lines
Fixes b6bd53f9c4e8 ("MIPS: Add missing file for eBPF JIT.")

I'd be happy to take this through mips-fixes, but equally happy for it to go through the BPF/net trees in which case:

Thanks,
Paul

Quoted 32 lines
---
 arch/mips/net/ebpf_jit.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
arch/mips/net/ebpf_jit.c +1 -0
index 46b76751f3a5..3ec69d9cbe88 100644
@@ -604,6 +604,7 @@ static void emit_const_to_reg(struct jit_ctx
 *ctx, int dst, u64 value)
 static int emit_bpf_tail_call(struct jit_ctx *ctx, int this_idx)
 {
 	int off, b_off;
+	int tcc_reg;
 
 	ctx->flags |= EBPF_SEEN_TC;
arch/mips/net/ebpf_jit.c +3 -3
 	/*
@@ -616,14 +617,14 @@ static int emit_bpf_tail_call(struct jit_ctx
 *ctx, int this_idx)
 	b_off = b_imm(this_idx + 1, ctx);
 	emit_instr(ctx, bne, MIPS_R_AT, MIPS_R_ZERO, b_off);
 	/*
-	 * if (--TCC < 0)
+	 * if (TCC-- < 0)
 	 *     goto out;
 	 */
 	/* Delay slot */
-	emit_instr(ctx, daddiu, MIPS_R_T5,
-		   (ctx->flags & EBPF_TCC_IN_V1) ? MIPS_R_V1 : MIPS_R_S4, -1);
+	tcc_reg = (ctx->flags & EBPF_TCC_IN_V1) ? MIPS_R_V1 : MIPS_R_S4;
+	emit_instr(ctx, daddiu, MIPS_R_T5, tcc_reg, -1);
 	b_off = b_imm(this_idx + 1, ctx);
Quoted 8 lines
Quoted 8 lines
arch/mips/net/ebpf_jit.c +1 -1
-	emit_instr(ctx, bltz, MIPS_R_T5, b_off);
+	emit_instr(ctx, bltz, tcc_reg, b_off);
 	/*
 	 * prog = array->ptrs[index];

* if (prog == NULL)
--
2.17.1

Regards,
ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ

Regards,
ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ