Skip to contentSkip to conversation

s390/bpf: Write back tail call counter

PATCH Applied Ilya Leoshkevich 2 participants 6 messages Aug 13, 2025 – Aug 18, 2025

Subsystems (5)
Kselftest BPF s390 BPF selftests +1 more subsystems
lore
Share & tools
lore

Ilya Leoshkevich

author maintainer
#0 lore raw

v2 cover s390/bpf: Write back tail call counter

v1: https://lore.kernel.org/bpf/[email protected]/
v1 -> v2: Write back tail call counter only for BPF_PSEUDO_CALL

Hi,

This series fixes the tailcall_bpf2bpf_hierarchy test failures on s390.

It takes a simpler approach than x86_64 and aarch64: instead of introducing a pointer to the tail call counter, it copies the updated value back from the callee's frame to the caller's frame. This needs to be done in two locations: after BPF_PSEUDO_CALL and after BPF_TRAMP_F_CALL_ORIG.

Patch 1 is a cleanup, patches 2 and 3 are the actual fixes, patch 4 improves the hierarchy tests in order to catch issues with accidentally clobbering the counter.

Best regards,
Ilya

Ilya Leoshkevich (4):
s390/bpf: Do not write tail call counter into helper and kfunc frames s390/bpf: Write back tail call counter for BPF_PSEUDO_CALL s390/bpf: Write back tail call counter for BPF_TRAMP_F_CALL_ORIG selftests/bpf: Clobber a lot of registers in
tailcall_bpf2bpf_hierarchy tests

 arch/s390/net/bpf_jit_comp.c                  | 42 ++++++++++++++-----
 .../selftests/bpf/progs/bpf_test_utils.h      | 18 ++++++++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy1.c   |  3 ++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy2.c   |  3 ++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy3.c   |  3 ++
 .../progs/tailcall_bpf2bpf_hierarchy_fentry.c |  3 ++
 6 files changed, 61 insertions(+), 11 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_test_utils.h
Signature · 1 line
2.50.1
author maintainer
#1 lore raw

PATCH v2 1/4 s390/bpf: Do not write tail call counter into helper and kfunc frames

Only BPF functions make use of the tail call counter; helpers and kfuncs ignore and most likely also clobber it. Writing it into these functions' frames is pointless and misleading, so do not do it.

Fixes dd691e847d28 ("s390/bpf: Implement bpf_jit_supports_subprog_tailcalls()")
---
 arch/s390/net/bpf_jit_comp.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
arch/s390/net/bpf_jit_comp.c +12 -4
index bb17efe29d65..bfac1ddf3447 100644
@@ -1790,6 +1790,7 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 
 		REG_SET_SEEN(BPF_REG_5);
 		jit->seen |= SEEN_FUNC;
+
 		/*
 		 * Copy the tail call counter to where the callee expects it.
 		 *
@@ -1800,10 +1801,17 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 		 * Note 2: We assume that the verifier does not let us call the
 		 * main program, which clears the tail call counter on entry.
 		 */
-		/* mvc tail_call_cnt(4,%r15),frame_off+tail_call_cnt(%r15) */
-		_EMIT6(0xd203f000 | offsetof(struct prog_frame, tail_call_cnt),
-		       0xf000 | (jit->frame_off +
-				 offsetof(struct prog_frame, tail_call_cnt)));
+
+		if (insn->src_reg == BPF_PSEUDO_CALL)
+			/*
+			 * mvc tail_call_cnt(4,%r15),
+			 *     frame_off+tail_call_cnt(%r15)
+			 */
+			_EMIT6(0xd203f000 | offsetof(struct prog_frame,
+						     tail_call_cnt),
+			       0xf000 | (jit->frame_off +
+					 offsetof(struct prog_frame,
+						  tail_call_cnt)));
 
 		/* Sign-extend the kfunc arguments. */
 		if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL) {
Signature · 1 line
2.50.1
author maintainer
#2 lore raw

PATCH v2 2/4 s390/bpf: Write back tail call counter for BPF_PSEUDO_CALL

In reply to Ilya Leoshkevich

The tailcall_bpf2bpf_hierarchy_1 test hangs on s390. Its call graph is as follows:

entry()

    subprog_tail()
      bpf_tail_call_static(0) -> entry + tail_call_start
    subprog_tail()
      bpf_tail_call_static(0) -> entry + tail_call_start

entry() copies its tail call counter to the subprog_tail()'s frame, which then increments it. However, the incremented result is discarded, leading to an astronomically large number of tail calls.

Fix by writing the incremented counter back to the entry()'s frame.

Fixes dd691e847d28 ("s390/bpf: Implement bpf_jit_supports_subprog_tailcalls()")
---
 arch/s390/net/bpf_jit_comp.c | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)
arch/s390/net/bpf_jit_comp.c +16 -7
index bfac1ddf3447..ccb83ac3e6f3 100644
@@ -1793,13 +1793,6 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 
 		/*
 		 * Copy the tail call counter to where the callee expects it.
-		 *
-		 * Note 1: The callee can increment the tail call counter, but
-		 * we do not load it back, since the x86 JIT does not do this
-		 * either.
-		 *
-		 * Note 2: We assume that the verifier does not let us call the
-		 * main program, which clears the tail call counter on entry.
 		 */
 
 		if (insn->src_reg == BPF_PSEUDO_CALL)
@@ -1833,6 +1826,22 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 		call_r1(jit);
 		/* lgr %b0,%r2: load return value into %b0 */
 		EMIT4(0xb9040000, BPF_REG_0, REG_2);
+
+		/*
+		 * Copy the potentially updated tail call counter back.
+		 */
+
+		if (insn->src_reg == BPF_PSEUDO_CALL)
+			/*
+			 * mvc frame_off+tail_call_cnt(%r15),
+			 *     tail_call_cnt(4,%r15)
+			 */
+			_EMIT6(0xd203f000 | (jit->frame_off +
+					     offsetof(struct prog_frame,
+						      tail_call_cnt)),
+			       0xf000 | offsetof(struct prog_frame,
+						 tail_call_cnt));
+
 		break;
 	}
 	case BPF_JMP | BPF_TAIL_CALL: {
Signature · 1 line
2.50.1
author maintainer
#3 lore raw

PATCH v2 3/4 s390/bpf: Write back tail call counter for BPF_TRAMP_F_CALL_ORIG

In reply to Ilya Leoshkevich

The tailcall_bpf2bpf_hierarchy_fentry test hangs on s390. Its call graph is as follows:

entry()
subprog_tail()
trampoline()
fentry()
the rest of subprog_tail() # via BPF_TRAMP_F_CALL_ORIG return to entry()

The problem is that the rest of subprog_tail() increments the tail call counter, but the trampoline discards the incremented value. This results in an astronomically large number of tail calls.

Fix by making the trampoline write the incremented tail call counter back.

Fixes 528eb2cb87bc ("s390/bpf: Implement arch_prepare_bpf_trampoline()")
---
 arch/s390/net/bpf_jit_comp.c | 3 +++
 1 file changed, 3 insertions(+)
arch/s390/net/bpf_jit_comp.c +3 -0
index ccb83ac3e6f3..b2b8eb62b82e 100644
@@ -2839,6 +2839,9 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
 		/* stg %r2,retval_off(%r15) */
 		EMIT6_DISP_LH(0xe3000000, 0x0024, REG_2, REG_0, REG_15,
 			      tjit->retval_off);
+		/* mvc tccnt_off(%r15),tail_call_cnt(4,%r15) */
+		_EMIT6(0xd203f000 | tjit->tccnt_off,
+		       0xf000 | offsetof(struct prog_frame, tail_call_cnt));
 
 		im->ip_after_call = jit->prg_buf + jit->prg;
 
Signature · 1 line
2.50.1
author maintainer
#4 lore raw

PATCH v2 4/4 selftests/bpf: Clobber a lot of registers in tailcall_bpf2bpf_hierarchy tests

In reply to Ilya Leoshkevich

Clobbering a lot of registers and stack slots helps exposing tail call counter overwrite bugs in JITs.

---
 .../selftests/bpf/progs/bpf_test_utils.h       | 18 ++++++++++++++++++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy1.c    |  3 +++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy2.c    |  3 +++
 .../bpf/progs/tailcall_bpf2bpf_hierarchy3.c    |  3 +++
 .../progs/tailcall_bpf2bpf_hierarchy_fentry.c  |  3 +++
 5 files changed, 30 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_test_utils.h
tools/testing/selftests/bpf/progs/bpf_test_utils.h new file +18 -0
new file mode 100644
index 000000000000..f4e67b492dd2
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __BPF_TEST_UTILS_H__
+#define __BPF_TEST_UTILS_H__
+
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+/* Clobber as many native registers and stack slots as possible. */
+static __always_inline void clobber_regs_stack(void)
+{
+	char tmp_str[] = "123456789";
+	unsigned long tmp;
+
+	bpf_strtoul(tmp_str, sizeof(tmp_str), 0, &tmp);
+	__sink(tmp);
+}
+
+#endif
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy1.c +3 -0
index 327ca395e860..d556b19413d7 100644
@@ -2,6 +2,7 @@
 #include <linux/bpf.h>
 #include <bpf/bpf_helpers.h>
 #include "bpf_legacy.h"
+#include "bpf_test_utils.h"
 
 struct {
 	__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
@@ -24,6 +25,8 @@ int entry(struct __sk_buff *skb)
 {
 	int ret = 1;
 
+	clobber_regs_stack();
+
 	count++;
 	subprog_tail(skb);
 	subprog_tail(skb);
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy2.c +3 -0
index 72fd0d577506..ae94c9c70ab7 100644
@@ -2,6 +2,7 @@
 #include <linux/bpf.h>
 #include <bpf/bpf_helpers.h>
 #include "bpf_misc.h"
+#include "bpf_test_utils.h"
 
 int classifier_0(struct __sk_buff *skb);
 int classifier_1(struct __sk_buff *skb);
@@ -60,6 +61,8 @@ int tailcall_bpf2bpf_hierarchy_2(struct __sk_buff *skb)
 {
 	int ret = 0;
 
+	clobber_regs_stack();
+
 	subprog_tail0(skb);
 	subprog_tail1(skb);
 
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy3.c +3 -0
index a7fb91cb05b7..56b6b0099840 100644
@@ -2,6 +2,7 @@
 #include <linux/bpf.h>
 #include <bpf/bpf_helpers.h>
 #include "bpf_misc.h"
+#include "bpf_test_utils.h"
 
 int classifier_0(struct __sk_buff *skb);
 
@@ -53,6 +54,8 @@ int tailcall_bpf2bpf_hierarchy_3(struct __sk_buff *skb)
 {
 	int ret = 0;
 
+	clobber_regs_stack();
+
 	bpf_tail_call_static(skb, &jmp_table0, 0);
 
 	__sink(ret);
tools/testing/selftests/bpf/progs/tailcall_bpf2bpf_hierarchy_fentry.c +3 -0
index c87f9ca982d3..5261395713cd 100644
@@ -4,6 +4,7 @@
 #include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
+#include "bpf_test_utils.h"
 
 struct {
 	__uint(type, BPF_MAP_TYPE_PROG_ARRAY);
@@ -24,6 +25,8 @@ int subprog_tail(void *ctx)
 SEC("fentry/dummy")
 int BPF_PROG(fentry, struct sk_buff *skb)
 {
+	clobber_regs_stack();
+
 	count++;
 	subprog_tail(ctx);
 	subprog_tail(ctx);
Signature · 1 line
2.50.1
#5 lore raw
In reply to Ilya Leoshkevich
This series was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:

Quoted 8 lines · On Wed, 13 Aug 2025 14:06:27 +0200 you wrote:

v1: https://lore.kernel.org/bpf/[email protected]/
v1 -> v2: Write back tail call counter only for BPF_PSEUDO_CALL

Hi,

This series fixes the tailcall_bpf2bpf_hierarchy test failures on s390.

[...]

Here is the summary with links:
- [bpf-next,v2,1/4] s390/bpf: Do not write tail call counter into helper and kfunc frames

    https://git.kernel.org/bpf/bpf-next/c/eada40e057fc
  - [bpf-next,v2,2/4] s390/bpf: Write back tail call counter for BPF_PSEUDO_CALL
    https://git.kernel.org/bpf/bpf-next/c/c861a6b14713
  - [bpf-next,v2,3/4] s390/bpf: Write back tail call counter for BPF_TRAMP_F_CALL_ORIG
    https://git.kernel.org/bpf/bpf-next/c/bc3905a71f02
  - [bpf-next,v2,4/4] selftests/bpf: Clobber a lot of registers in tailcall_bpf2bpf_hierarchy tests
    https://git.kernel.org/bpf/bpf-next/c/12741630350c

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html