Skip to contentSkip to conversation

powerpc64/bpf: Support tailcalls with subprogs & BPF exceptions

PATCH Superseded [email protected] 3 participants 14 messages Jan 22 – Jan 23 1 message not in the archive

Subsystems (4)

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover powerpc64/bpf: Support tailcalls with subprogs & BPF exceptions

From: Abhishek Dubey <[email protected]>

This patch series enables support for two BPF JIT features on powerpc64. The first three patches target support for tail calls with subprogram combinations. The first patch supports realignment of tail_call_cnt offset in stack frame. Implementation details are provided in the commit messages.

The last three patches add support for BPF exceptions. An architecture-specific stack walker is implemented to assist with stack walk during exceptions.

All selftests related to tailcalls and exceptions are passing:

# ./test_progs -t tailcalls
#442/1   tailcalls/tailcall_1:OK
#442/2   tailcalls/tailcall_2:OK
#442/3   tailcalls/tailcall_3:OK
#442/4   tailcalls/tailcall_4:OK
#442/5   tailcalls/tailcall_5:OK
#442/6   tailcalls/tailcall_6:OK
#442/7   tailcalls/tailcall_bpf2bpf_1:OK
#442/8   tailcalls/tailcall_bpf2bpf_2:OK
#442/9   tailcalls/tailcall_bpf2bpf_3:OK
#442/10  tailcalls/tailcall_bpf2bpf_4:OK
#442/11  tailcalls/tailcall_bpf2bpf_5:OK
#442/12  tailcalls/tailcall_bpf2bpf_6:OK
#442/13  tailcalls/tailcall_bpf2bpf_fentry:OK
#442/14  tailcalls/tailcall_bpf2bpf_fexit:OK
#442/15  tailcalls/tailcall_bpf2bpf_fentry_fexit:OK
#442/16  tailcalls/tailcall_bpf2bpf_fentry_entry:OK
#442/17  tailcalls/tailcall_poke:OK
#442/18  tailcalls/tailcall_bpf2bpf_hierarchy_1:OK
#442/19  tailcalls/tailcall_bpf2bpf_hierarchy_fentry:OK
#442/20  tailcalls/tailcall_bpf2bpf_hierarchy_fexit:OK
#442/21  tailcalls/tailcall_bpf2bpf_hierarchy_fentry_fexit:OK
#442/22  tailcalls/tailcall_bpf2bpf_hierarchy_fentry_entry:OK
#442/23  tailcalls/tailcall_bpf2bpf_hierarchy_2:OK
#442/24  tailcalls/tailcall_bpf2bpf_hierarchy_3:OK
#442/25  tailcalls/tailcall_freplace:OK
#442/26  tailcalls/tailcall_bpf2bpf_freplace:OK
#442/27  tailcalls/tailcall_failure:OK
#442/28  tailcalls/reject_tail_call_spin_lock:OK
#442/29  tailcalls/reject_tail_call_rcu_lock:OK
#442/30  tailcalls/reject_tail_call_preempt_lock:OK
#442/31  tailcalls/reject_tail_call_ref:OK
#442     tailcalls:OK
Summary: 1/31 PASSED, 0 SKIPPED, 0 FAILED
# ./test_progs -t exceptions
#105/1   exceptions/exception_throw_always_1:OK
#105/2   exceptions/exception_throw_always_2:OK
#105/3   exceptions/exception_throw_unwind_1:OK
#105/4   exceptions/exception_throw_unwind_2:OK
#105/5   exceptions/exception_throw_default:OK
#105/6   exceptions/exception_throw_default_value:OK
#105/7   exceptions/exception_tail_call:OK
#105/8   exceptions/exception_ext:OK
#105/9   exceptions/exception_ext_mod_cb_runtime:OK
#105/10  exceptions/exception_throw_subprog:OK
#105/11  exceptions/exception_assert_nz_gfunc:OK
#105/12  exceptions/exception_assert_zero_gfunc:OK
#105/13  exceptions/exception_assert_neg_gfunc:OK
#105/14  exceptions/exception_assert_pos_gfunc:OK
#105/15  exceptions/exception_assert_negeq_gfunc:OK
#105/16  exceptions/exception_assert_poseq_gfunc:OK
#105/17  exceptions/exception_assert_nz_gfunc_with:OK
#105/18  exceptions/exception_assert_zero_gfunc_with:OK
#105/19  exceptions/exception_assert_neg_gfunc_with:OK
#105/20  exceptions/exception_assert_pos_gfunc_with:OK
#105/21  exceptions/exception_assert_negeq_gfunc_with:OK
#105/22  exceptions/exception_assert_poseq_gfunc_with:OK
#105/23  exceptions/exception_bad_assert_nz_gfunc:OK
#105/24  exceptions/exception_bad_assert_zero_gfunc:OK
#105/25  exceptions/exception_bad_assert_neg_gfunc:OK
#105/26  exceptions/exception_bad_assert_pos_gfunc:OK
#105/27  exceptions/exception_bad_assert_negeq_gfunc:OK
#105/28  exceptions/exception_bad_assert_poseq_gfunc:OK
#105/29  exceptions/exception_bad_assert_nz_gfunc_with:OK
#105/30  exceptions/exception_bad_assert_zero_gfunc_with:OK
#105/31  exceptions/exception_bad_assert_neg_gfunc_with:OK
#105/32  exceptions/exception_bad_assert_pos_gfunc_with:OK
#105/33  exceptions/exception_bad_assert_negeq_gfunc_with:OK
#105/34  exceptions/exception_bad_assert_poseq_gfunc_with:OK
#105/35  exceptions/exception_assert_range:OK
#105/36  exceptions/exception_assert_range_with:OK
#105/37  exceptions/exception_bad_assert_range:OK
#105/38  exceptions/exception_bad_assert_range_with:OK
#105/39  exceptions/non-throwing fentry -> exception_cb:OK
#105/40  exceptions/throwing fentry -> exception_cb:OK
#105/41  exceptions/non-throwing fexit -> exception_cb:OK
#105/42  exceptions/throwing fexit -> exception_cb:OK
#105/43  exceptions/throwing extension (with custom cb) -> exception_cb:OK
#105/44  exceptions/throwing extension -> global func in exception_cb:OK
#105/45  exceptions/exception_ext_mod_cb_runtime:OK
#105/46  exceptions/throwing extension (with custom cb) -> global func in exception_cb:OK
#105/47  exceptions/exception_ext:OK
#105/48  exceptions/non-throwing fentry -> non-throwing subprog:OK
#105/49  exceptions/throwing fentry -> non-throwing subprog:OK
#105/50  exceptions/non-throwing fentry -> throwing subprog:OK
#105/51  exceptions/throwing fentry -> throwing subprog:OK
#105/52  exceptions/non-throwing fexit -> non-throwing subprog:OK
#105/53  exceptions/throwing fexit -> non-throwing subprog:OK
#105/54  exceptions/non-throwing fexit -> throwing subprog:OK
#105/55  exceptions/throwing fexit -> throwing subprog:OK
#105/56  exceptions/non-throwing fmod_ret -> non-throwing subprog:OK
#105/57  exceptions/non-throwing fmod_ret -> non-throwing global subprog:OK
#105/58  exceptions/non-throwing extension -> non-throwing subprog:OK
#105/59  exceptions/non-throwing extension -> throwing subprog:OK
#105/60  exceptions/non-throwing extension -> non-throwing subprog:OK
#105/61  exceptions/non-throwing extension -> throwing global subprog:OK
#105/62  exceptions/throwing extension -> throwing global subprog:OK
#105/63  exceptions/throwing extension -> non-throwing global subprog:OK
#105/64  exceptions/non-throwing extension -> main subprog:OK
#105/65  exceptions/throwing extension -> main subprog:OK
#105/66  exceptions/reject_exception_cb_type_1:OK
#105/67  exceptions/reject_exception_cb_type_2:OK
#105/68  exceptions/reject_exception_cb_type_3:OK
#105/69  exceptions/reject_exception_cb_type_4:OK
#105/70  exceptions/reject_async_callback_throw:OK
#105/71  exceptions/reject_with_lock:OK
#105/72  exceptions/reject_subprog_with_lock:OK
#105/73  exceptions/reject_with_rcu_read_lock:OK
#105/74  exceptions/reject_subprog_with_rcu_read_lock:OK
#105/75  exceptions/reject_with_rbtree_add_throw:OK
#105/76  exceptions/reject_with_reference:OK
#105/77  exceptions/reject_with_cb_reference:OK
#105/78  exceptions/reject_with_cb:OK
#105/79  exceptions/reject_with_subprog_reference:OK
#105/80  exceptions/reject_throwing_exception_cb:OK
#105/81  exceptions/reject_exception_cb_call_global_func:OK
#105/82  exceptions/reject_exception_cb_call_static_func:OK
#105/83  exceptions/reject_multiple_exception_cb:OK
#105/84  exceptions/reject_exception_throw_cb:OK
#105/85  exceptions/reject_exception_throw_cb_diff:OK
#105/86  exceptions/reject_set_exception_cb_bad_ret1:OK
#105/87  exceptions/reject_set_exception_cb_bad_ret2:OK
#105/88  exceptions/check_assert_eq_int_min:OK
#105/89  exceptions/check_assert_eq_int_max:OK
#105/90  exceptions/check_assert_eq_zero:OK
#105/91  exceptions/check_assert_eq_llong_min:OK
#105/92  exceptions/check_assert_eq_llong_max:OK
#105/93  exceptions/check_assert_lt_pos:OK
#105/94  exceptions/check_assert_lt_zero:OK
#105/95  exceptions/check_assert_lt_neg:OK
#105/96  exceptions/check_assert_le_pos:OK
#105/97  exceptions/check_assert_le_zero:OK
#105/98  exceptions/check_assert_le_neg:OK
#105/99  exceptions/check_assert_gt_pos:OK
#105/100 exceptions/check_assert_gt_zero:OK
#105/101 exceptions/check_assert_gt_neg:OK
#105/102 exceptions/check_assert_ge_pos:OK
#105/103 exceptions/check_assert_ge_zero:OK
#105/104 exceptions/check_assert_ge_neg:OK
#105/105 exceptions/check_assert_range_s64:OK
#105/106 exceptions/check_assert_range_u64:OK
#105/107 exceptions/check_assert_single_range_s64:OK
#105/108 exceptions/check_assert_single_range_u64:OK
#105/109 exceptions/check_assert_generic:OK
#105/110 exceptions/check_assert_with_return:OK
#105     exceptions:OK
Summary: 1/110 PASSED, 0 SKIPPED, 0 FAILED

[v1]: https://lore.kernel.org/all/[email protected]/
[v2]: https://lore.kernel.org/all/[email protected]/
[v3]: https://lore.kernel.org/all/[email protected]/

Changes v3->v4:
Handle bpf-ci warnings
Changes v2->v3:
Added PPC_BCC_CONST_SHORT for short jumps of constant offset Optimize tailcall allocation for BPF_TRAMP_F_CALL_ORIG flag New helper for stack size calculation during exceptions Prologue JIT optimizations during non exception prog case
Changes v1->v2:
Move tail_call_cnt to offset 0 in stack frame Remove trampoline NVR remapping-patch3/6[v1]

Abhishek Dubey (6):
powerpc64/bpf: Moving tail_call_cnt to bottom of frame powerpc64/bpf: Support tailcalls with subprogs powerpc64/bpf: Avoid tailcall restore from trampoline powerpc64/bpf: Add arch_bpf_stack_walk() for BPF JIT powerpc64/bpf: Support exceptions powerpc64/bpf: Additional NVR handling for bpf_throw

 arch/powerpc/net/bpf_jit.h        |  16 ++
 arch/powerpc/net/bpf_jit_comp.c   |  88 ++++++++--
 arch/powerpc/net/bpf_jit_comp64.c | 256 ++++++++++++++++++++++++------
 3 files changed, 297 insertions(+), 63 deletions(-)
Signature · 1 line
2.48.1
#1 lore raw

PATCH v4 1/6 powerpc64/bpf: Moving tail_call_cnt to bottom of frame

From: Abhishek Dubey <[email protected]>

In the conventional stack frame, the position of tail_call_cnt is after the NVR save area (BPF_PPC_STACK_SAVE). Whereas, the offset of tail_call_cnt in the trampoline frame is after the stack alignment padding. BPF JIT logic could become complex when dealing with frame-sensitive offset calculation of tail_call_cnt. Having the same offset in both frames is the desired objective.

The trampoline frame does not have a BPF_PPC_STACK_SAVE area. Introducing it leads to under-utilization of extra memory meant only for the offset alignment of tail_call_cnt. Another challenge is the variable alignment padding sitting at the bottom of the trampoline frame, which requires additional handling to compute tail_call_cnt offset.

This patch addresses the above issues by moving tail_call_cnt to the bottom of the stack frame at offset 0 for both types of frames. This saves additional bytes required by BPF_PPC_STACK_SAVE in trampoline frame, and a common offset computation for tail_call_cnt serves both frames.

The changes in this patch are required by the second patch in the series, where the 'reference to tail_call_info' of the main frame is copied into the trampoline frame from the previous frame.

---
 arch/powerpc/net/bpf_jit.h        |  1 +
 arch/powerpc/net/bpf_jit_comp.c   | 15 ++++++++++++---
 arch/powerpc/net/bpf_jit_comp64.c | 31 ++++++++++++++++++++-----------
 3 files changed, 33 insertions(+), 14 deletions(-)
arch/powerpc/net/bpf_jit.h +1 -0
index 8334cd667bba..9f6ec00bd02e 100644
@@ -24,6 +24,7 @@
 
 #define SZL			sizeof(unsigned long)
 #define BPF_INSN_SAFETY		64
+#define BPF_PPC_TAILCALL	8
 
 #define PLANT_INSTR(d, idx, instr)					      \
 	do { if (d) { (d)[idx] = instr; } idx++; } while (0)
arch/powerpc/net/bpf_jit_comp.c +12 -3
index 5e976730b2f5..d51c696221d7 100644
@@ -604,8 +604,8 @@ static void bpf_trampoline_setup_tail_call_cnt(u32 *image, struct codegen_contex
 					       int func_frame_offset, int r4_off)
 {
 	if (IS_ENABLED(CONFIG_PPC64)) {
-		/* See bpf_jit_stack_tailcallcnt() */
-		int tailcallcnt_offset = 7 * 8;
+		/* See Generated stack layout */
+		int tailcallcnt_offset = BPF_PPC_TAILCALL;
 
 		EMIT(PPC_RAW_LL(_R3, _R1, func_frame_offset - tailcallcnt_offset));
 		EMIT(PPC_RAW_STL(_R3, _R1, -tailcallcnt_offset));
@@ -620,7 +620,7 @@ static void bpf_trampoline_restore_tail_call_cnt(u32 *image, struct codegen_cont
 {
 	if (IS_ENABLED(CONFIG_PPC64)) {
 		/* See bpf_jit_stack_tailcallcnt() */
-		int tailcallcnt_offset = 7 * 8;
+		int tailcallcnt_offset = BPF_PPC_TAILCALL;
 
 		EMIT(PPC_RAW_LL(_R3, _R1, -tailcallcnt_offset));
 		EMIT(PPC_RAW_STL(_R3, _R1, func_frame_offset - tailcallcnt_offset));
@@ -714,6 +714,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
 	 * LR save area                 [ r0 save (64-bit)  ]   | header
 	 *                              [ r0 save (32-bit)  ]   |
 	 * dummy frame for unwind       [ back chain 1      ] --
+	 *                              [ tail_call_cnt     ] optional - 64-bit powerpc
 	 *                              [ padding           ] align stack frame
 	 *       r4_off                 [ r4 (tailcallcnt)  ] optional - 32-bit powerpc
 	 *       alt_lr_off             [ real lr (ool stub)] optional - actual lr
@@ -795,6 +796,14 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
 		}
 	}
 
+	/*
+	 * Save tailcall count pointer at the same offset on the
+	 * stack where subprogs expect it
+	 */
+	if ((flags & BPF_TRAMP_F_CALL_ORIG) &&
+		(flags & BPF_TRAMP_F_TAIL_CALL_CTX))
+		bpf_frame_size += BPF_PPC_TAILCALL;
+
 	/* Padding to align stack frame, if any */
 	bpf_frame_size = round_up(bpf_frame_size, SZL * 2);
 
arch/powerpc/net/bpf_jit_comp64.c +20 -11
index 1fe37128c876..296e9ea14f2e 100644
@@ -20,13 +20,15 @@
 #include "bpf_jit.h"
 
 /*
- * Stack layout:
+ * Stack layout with frame:
+ * Layout when setting up our own stack frame.
+ * Note: r1 at bottom, component offsets positive wrt r1.
  * Ensure the top half (upto local_tmp_var) stays consistent
  * with our redzone usage.
  *
  *		[	prev sp		] <-------------
- *		[   nv gpr save area	] 6*8		|
  *		[    tail_call_cnt	] 8		|
+ *		[   nv gpr save area	] 6*8		|
  *		[    local_tmp_var	] 24		|
  * fp (r31) -->	[   ebpf stack space	] upto 512	|
  *		[     frame header	] 32/112	|
@@ -36,10 +38,12 @@
 /* for gpr non volatile registers BPG_REG_6 to 10 */
 #define BPF_PPC_STACK_SAVE	(6*8)
 /* for bpf JIT code internal usage */
-#define BPF_PPC_STACK_LOCALS	32
+#define BPF_PPC_STACK_LOCALS	24
 /* stack frame excluding BPF stack, ensure this is quadword aligned */
 #define BPF_PPC_STACKFRAME	(STACK_FRAME_MIN_SIZE + \
-				 BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SAVE)
+				 BPF_PPC_STACK_LOCALS + \
+				 BPF_PPC_STACK_SAVE   + \
+				 BPF_PPC_TAILCALL)
 
 /* BPF register usage */
 #define TMP_REG_1	(MAX_BPF_JIT_REG + 0)
@@ -87,27 +91,32 @@ static inline bool bpf_has_stack_frame(struct codegen_context *ctx)
 }
 
 /*
+ * Stack layout with redzone:
  * When not setting up our own stackframe, the redzone (288 bytes) usage is:
+ * Note: r1 from prev frame. Component offset negative wrt r1.
  *
  *		[	prev sp		] <-------------
  *		[	  ...       	] 		|
  * sp (r1) --->	[    stack pointer	] --------------
- *		[   nv gpr save area	] 6*8
  *		[    tail_call_cnt	] 8
+ *		[   nv gpr save area	] 6*8
  *		[    local_tmp_var	] 24
  *		[   unused red zone	] 224
  */
 static int bpf_jit_stack_local(struct codegen_context *ctx)
 {
-	if (bpf_has_stack_frame(ctx))
+	if (bpf_has_stack_frame(ctx)) {
+		/* Stack layout with frame */
 		return STACK_FRAME_MIN_SIZE + ctx->stack_size;
-	else
-		return -(BPF_PPC_STACK_SAVE + 32);
+	} else {
+		/* Stack layout with redzone */
+		return -(BPF_PPC_TAILCALL + BPF_PPC_STACK_SAVE + BPF_PPC_STACK_LOCALS);
+	}
 }
 
 static int bpf_jit_stack_tailcallcnt(struct codegen_context *ctx)
 {
-	return bpf_jit_stack_local(ctx) + 24;
+	return bpf_jit_stack_local(ctx) + BPF_PPC_STACK_LOCALS + BPF_PPC_STACK_SAVE;
 }
 
 static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg)
@@ -115,7 +124,7 @@ static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg)
 	if (reg >= BPF_PPC_NVR_MIN && reg < 32)
 		return (bpf_has_stack_frame(ctx) ?
 			(BPF_PPC_STACKFRAME + ctx->stack_size) : 0)
-				- (8 * (32 - reg));
+				- (8 * (32 - reg)) - BPF_PPC_TAILCALL;
 
 	pr_err("BPF JIT is asking about unknown registers");
 	BUG();
@@ -145,7 +154,7 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx)
 	if (ctx->seen & SEEN_TAILCALL) {
 		EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0));
 		/* this goes in the redzone */
-		EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_STACK_SAVE + 8)));
+		EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL)));
 	} else {
 		EMIT(PPC_RAW_NOP());
 		EMIT(PPC_RAW_NOP());
Signature · 1 line
2.48.1
maintainer
#2 lore raw

Re: [PATCH v4 1/6] powerpc64/bpf: Moving tail_call_cnt to bottom of frame

Quoted 23 lines · On 23/01/26 2:48 am, [email protected] wrote:

From: Abhishek Dubey <[email protected]>

In the conventional stack frame, the position of tail_call_cnt is after the NVR save area (BPF_PPC_STACK_SAVE). Whereas, the offset of tail_call_cnt in the trampoline frame is after the stack alignment padding. BPF JIT logic could become complex when dealing with frame-sensitive offset calculation of tail_call_cnt. Having the same offset in both frames is the desired objective.

The trampoline frame does not have a BPF_PPC_STACK_SAVE area. Introducing it leads to under-utilization of extra memory meant only for the offset alignment of tail_call_cnt. Another challenge is the variable alignment padding sitting at the bottom of the trampoline frame, which requires additional handling to compute tail_call_cnt offset.

This patch addresses the above issues by moving tail_call_cnt to the bottom of the stack frame at offset 0 for both types of frames. This saves additional bytes required by BPF_PPC_STACK_SAVE in trampoline frame, and a common offset computation for tail_call_cnt serves both frames.

Quoted 3 lines

The changes in this patch are required by the second patch in the series, where the 'reference to tail_call_info' of the main frame is copied into the trampoline frame from the previous frame.

The changelog needs to be simplified. Something like below:

To support tailcalls in subprogs, tail_call_cnt needs to be on the BPF trampoline stack frame. In a regular BPF program or subprog stack frame, the position of tail_call_cnt is after the NVR save area (BPF_PPC_STACK_SAVE). To avoid complex logic in deducing offset for tail_call_cnt, it has to be kept at the same offset on the trampoline frame as well. But doing that wastes nearly all of BPF_PPC_STACK_SAVE bytes on the BPF trampoline stack frame as the NVR save area is not the same for BPF trampoline and regular BPF programs. Address this by moving tail_call_cnt to the bottom of the frame.

This change avoids the need to account for BPF_PPC_STACK_SAVE bytes in the BPF trampoline stack frame when support for tailcalls in BPF subprogs is added later. Also, this change makes offset calculation of tail_call_cnt field simpler all across.

Quoted 56 lines
---
  arch/powerpc/net/bpf_jit.h        |  1 +
  arch/powerpc/net/bpf_jit_comp.c   | 15 ++++++++++++---
  arch/powerpc/net/bpf_jit_comp64.c | 31 ++++++++++++++++++++-----------
  3 files changed, 33 insertions(+), 14 deletions(-)
arch/powerpc/net/bpf_jit.h +1 -0
index 8334cd667bba..9f6ec00bd02e 100644
@@ -24,6 +24,7 @@
 
  #define SZL			sizeof(unsigned long)
  #define BPF_INSN_SAFETY		64
+#define BPF_PPC_TAILCALL	8
 
  #define PLANT_INSTR(d, idx, instr)					      \
  	do { if (d) { (d)[idx] = instr; } idx++; } while (0)
arch/powerpc/net/bpf_jit_comp.c +4 -3
index 5e976730b2f5..d51c696221d7 100644
@@ -604,8 +604,8 @@ static void bpf_trampoline_setup_tail_call_cnt(u32 *image, struct codegen_contex
  					       int func_frame_offset, int r4_off)
  {
  	if (IS_ENABLED(CONFIG_PPC64)) {
-		/* See bpf_jit_stack_tailcallcnt() */
-		int tailcallcnt_offset = 7 * 8;
+		/* See Generated stack layout */
+		int tailcallcnt_offset = BPF_PPC_TAILCALL;
 
  		EMIT(PPC_RAW_LL(_R3, _R1, func_frame_offset - tailcallcnt_offset));
  		EMIT(PPC_RAW_STL(_R3, _R1, -tailcallcnt_offset));
@@ -620,7 +620,7 @@ static void bpf_trampoline_restore_tail_call_cnt(u32 *image, struct codegen_cont
  {
  	if (IS_ENABLED(CONFIG_PPC64)) {
  		/* See bpf_jit_stack_tailcallcnt() */
-		int tailcallcnt_offset = 7 * 8;
+		int tailcallcnt_offset = BPF_PPC_TAILCALL;
 
  		EMIT(PPC_RAW_LL(_R3, _R1, -tailcallcnt_offset));
  		EMIT(PPC_RAW_STL(_R3, _R1, func_frame_offset - tailcallcnt_offset));
@@ -714,6 +714,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
  	 * LR save area                 [ r0 save (64-bit)  ]   | header
  	 *                              [ r0 save (32-bit)  ]   |
  	 * dummy frame for unwind       [ back chain 1      ] --
+	 *                              [ tail_call_cnt     ] optional - 64-bit powerpc
  	 *                              [ padding           ] align stack frame
  	 *       r4_off                 [ r4 (tailcallcnt)  ] optional - 32-bit powerpc
  	 *       alt_lr_off             [ real lr (ool stub)] optional - actual lr
@@ -795,6 +796,14 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
  		}

arch/powerpc/net/bpf_jit_comp.c static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im · line 806

Quoted 10 lines
arch/powerpc/net/bpf_jit_comp.c +8 -0
  	}
 
+	/*
+	 * Save tailcall count pointer at the same offset on the
+	 * stack where subprogs expect it
+	 */
+	if ((flags & BPF_TRAMP_F_CALL_ORIG) &&
+		(flags & BPF_TRAMP_F_TAIL_CALL_CTX))
+		bpf_frame_size += BPF_PPC_TAILCALL;
+

The above hunk is relevant in the next patch where tailcalls support in subprogs is added. Drop it here and move it to patch#2.

- Hari

#3 lore raw

PATCH v4 2/6 powerpc64/bpf: Support tailcalls with subprogs

In reply to [email protected]

From: Abhishek Dubey <[email protected]>

Enabling tailcalls with subprog combinations by referencing method. The actual tailcall count is always maintained in the tail_call_info variable present in the frame of main function (also called entry function). The tail_call_info field in the stack frame of subprogs contains reference to the tail_call_info field in the stack frame of main BPF program.

Dynamic resolution interprets the tail_call_info either as value or reference depending on the context of active frame while tailcall is invoked.

---
 arch/powerpc/net/bpf_jit.h        | 13 ++++++
 arch/powerpc/net/bpf_jit_comp.c   | 59 +++++++++++++++++++++++----
 arch/powerpc/net/bpf_jit_comp64.c | 68 +++++++++++++++++++++++--------
 3 files changed, 117 insertions(+), 23 deletions(-)
arch/powerpc/net/bpf_jit.h +13 -0
index 9f6ec00bd02e..56f56fdd4969 100644
@@ -52,6 +52,13 @@
 		EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (offset & 0xfffc));					\
 	} while (0)
 
+/* When constant jump offset is known prior */
+#define PPC_BCC_CONST_SHORT(cond, offset)							\
+	do {											\
+		BUILD_BUG_ON(offset < -0x8000 || offset > 0x7fff || (offset & 0x3));		\
+		EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (offset & 0xfffc));	\
+	} while (0)
+
 /*
  * Sign-extended 32-bit immediate load
  *
@@ -73,6 +80,10 @@
 	} } while (0)
 
 #ifdef CONFIG_PPC64
+
+/* for gpr non volatile registers BPG_REG_6 to 10 */
+#define BPF_PPC_STACK_SAVE	(6 * 8)
+
 /* If dummy pass (!image), account for maximum possible instructions */
 #define PPC_LI64(d, i)		do {					      \
 	if (!image)							      \
@@ -167,6 +178,7 @@ struct codegen_context {
 	unsigned int alt_exit_addr;
 	u64 arena_vm_start;
 	u64 user_vm_start;
+	bool is_subprog;
 };
 
 #define bpf_to_ppc(r)	(ctx->b2p[r])
@@ -206,6 +218,7 @@ int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass
 			  struct codegen_context *ctx, int insn_idx,
 			  int jmp_off, int dst_reg, u32 code);
 
+int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx);
 #endif
 
 #endif
arch/powerpc/net/bpf_jit_comp.c +52 -7

93 lines not on this page · raw message

arch/powerpc/net/bpf_jit_comp64.c +52 -16

118 lines not on this page · raw message

Signature · 1 line
2.48.1
maintainer
#4 lore raw

Re: [PATCH v4 2/6] powerpc64/bpf: Support tailcalls with subprogs

Quoted 8 lines · On 23/01/26 2:48 am, [email protected] wrote:

From: Abhishek Dubey <[email protected]>

Enabling tailcalls with subprog combinations by referencing method. The actual tailcall count is always maintained in the tail_call_info variable present in the frame of main function (also called entry function). The tail_call_info field in the stack frame of subprogs contains reference to the tail_call_info field in the stack frame of main BPF program.

Describe the changes in imperative mood..

Instead of:

"Enabling tailcalls with subprog combinations by referencing method. The actual tailcall count is always maintained in the tail_call_info variable present in the frame of main function (also called entry function). The tail_call_info field in the stack frame of subprogs contains reference to the tail_call_info field in the stack frame of main BPF program."

use:

"Enable tailcalls support in subprogs by passing tail call count as reference instead of value. The actual tailcall count is always maintained in the tailcall field present in the frame of main function (also called entry function). The tailcall field in the stack frame of subprogs contains reference to the tailcall field in the stack frame of main BPF program. Accordingly, rename tail_call_cnt field in the stack layout to tail_call_info."

Quoted 82 lines

Dynamic resolution interprets the tail_call_info either as value or reference depending on the context of active frame while tailcall is invoked.

---
  arch/powerpc/net/bpf_jit.h        | 13 ++++++
  arch/powerpc/net/bpf_jit_comp.c   | 59 +++++++++++++++++++++++----
  arch/powerpc/net/bpf_jit_comp64.c | 68 +++++++++++++++++++++++--------
  3 files changed, 117 insertions(+), 23 deletions(-)
arch/powerpc/net/bpf_jit.h +13 -0
index 9f6ec00bd02e..56f56fdd4969 100644
@@ -52,6 +52,13 @@
  		EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (offset & 0xfffc));					\
  	} while (0)
 
+/* When constant jump offset is known prior */
+#define PPC_BCC_CONST_SHORT(cond, offset)							\
+	do {											\
+		BUILD_BUG_ON(offset < -0x8000 || offset > 0x7fff || (offset & 0x3));		\
+		EMIT(PPC_INST_BRANCH_COND | (((cond) & 0x3ff) << 16) | (offset & 0xfffc));	\
+	} while (0)
+
  /*
   * Sign-extended 32-bit immediate load
   *
@@ -73,6 +80,10 @@
  	} } while (0)
 
  #ifdef CONFIG_PPC64
+
+/* for gpr non volatile registers BPG_REG_6 to 10 */
+#define BPF_PPC_STACK_SAVE	(6 * 8)
+
  /* If dummy pass (!image), account for maximum possible instructions */
  #define PPC_LI64(d, i)		do {					      \
  	if (!image)							      \
@@ -167,6 +178,7 @@ struct codegen_context {
  	unsigned int alt_exit_addr;
  	u64 arena_vm_start;
  	u64 user_vm_start;
+	bool is_subprog;
  };
 
  #define bpf_to_ppc(r)	(ctx->b2p[r])
@@ -206,6 +218,7 @@ int bpf_add_extable_entry(struct bpf_prog *fp, u32 *image, u32 *fimage, int pass
  			  struct codegen_context *ctx, int insn_idx,
  			  int jmp_off, int dst_reg, u32 code);
 
+int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx);
  #endif
 
  #endif
arch/powerpc/net/bpf_jit_comp.c +6 -0
index d51c696221d7..93355ba5382a 100644
@@ -206,6 +206,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
  	cgctx.stack_size = round_up(fp->aux->stack_depth, 16);
  	cgctx.arena_vm_start = bpf_arena_get_kern_vm_start(fp->aux->arena);
  	cgctx.user_vm_start = bpf_arena_get_user_vm_start(fp->aux->arena);
+	cgctx.is_subprog = bpf_is_subprog(fp);
 
  	/* Scouting faux-generate pass 0 */
  	if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
@@ -435,6 +436,11 @@ void bpf_jit_free(struct bpf_prog *fp)
  	bpf_prog_unlock_free(fp);
  }
 
+bool bpf_jit_supports_subprog_tailcalls(void)
+{
+	return IS_ENABLED(CONFIG_PPC64);
+}
+
  bool bpf_jit_supports_kfunc_call(void)
  {
  	return true;

arch/powerpc/net/bpf_jit_comp.c static int invoke_bpf_mod_ret(u32 *image, u32 *ro_image, struct codegen_context · line 612

Quoted 9 lines
arch/powerpc/net/bpf_jit_comp.c +3 -2
@@ -600,15 +606,53 @@ static int invoke_bpf_mod_ret(u32 *image, u32 *ro_image, struct codegen_context
  	return 0;
  }
 
-static void bpf_trampoline_setup_tail_call_cnt(u32 *image, struct codegen_context *ctx,
-					       int func_frame_offset, int r4_off)
+/*
 
+ * Refer the label 'Generated stack layout' in this file for actual stack
+ * layout during trampoline invocation.

The above and the below comment seem to mean the same thing. One is enough?

arch/powerpc/net/bpf_jit_comp.c static int invoke_bpf_mod_ret(u32 *image, u32 *ro_image, struct codegen_context · line 618

Quoted 10 lines
arch/powerpc/net/bpf_jit_comp.c +7 -0
+ *
+ * Refer __arch_prepare_bpf_trampoline() for stack component details.
 
 
 
+ *
+ * The tailcall count/reference is present in caller's stack frame. Its required
+ * to copy the content of tail_call_info before calling the actual function
+ * to which the trampoline is attached.
+ */

Instead of:

"Its required to copy the content of tail_call_info before calling the actual function to which the trampoline is attached."

use:

"The tail_call_info is saved at the same offset on the trampoline frame for the traced function (BPF subprog/callee) to fetch it."

- Hari

#5 lore raw

PATCH v4 3/6 powerpc64/bpf: Avoid tailcall restore from trampoline

In reply to [email protected]

From: Abhishek Dubey <[email protected]>

Back propagation of tailcall count is no longer needed for powerpc64 due to use of reference, which updates the tailcall count in the tail_call_info field in the frame of the main program only.

Back propagation is still required for 32-bit powerpc.

---
 arch/powerpc/net/bpf_jit_comp.c | 13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)
arch/powerpc/net/bpf_jit_comp.c +5 -8
index 93355ba5382a..1a305f0fed27 100644
@@ -662,14 +662,11 @@ static void bpf_trampoline_setup_tail_call_info(u32 *image, struct codegen_conte
 static void bpf_trampoline_restore_tail_call_cnt(u32 *image, struct codegen_context *ctx,
 						 int func_frame_offset, int r4_off)
 {
-	if (IS_ENABLED(CONFIG_PPC64)) {
-		/* See bpf_jit_stack_tailcallcnt() */
-		int tailcallcnt_offset = BPF_PPC_TAILCALL;
-
-		EMIT(PPC_RAW_LL(_R3, _R1, -tailcallcnt_offset));
-		EMIT(PPC_RAW_STL(_R3, _R1, func_frame_offset - tailcallcnt_offset));
-	} else {
-		/* See bpf_jit_stack_offsetof() and BPF_PPC_TC */
+	if (IS_ENABLED(CONFIG_PPC32)) {
+		/*
+		 * Restore tailcall for 32-bit powerpc
+		 * See bpf_jit_stack_offsetof() and BPF_PPC_TC
+		 */
 		EMIT(PPC_RAW_STL(_R4, _R1, r4_off));
 	}
 }
Signature · 1 line
2.48.1
#6 lore raw

PATCH v4 4/6 powerpc64/bpf: Add arch_bpf_stack_walk() for BPF JIT

In reply to [email protected]

From: Abhishek Dubey <[email protected]>

This function is used by bpf_throw() to unwind the stack until frame of exception-boundary during BPF exception handling.

This function is necessary to support BPF exceptions on PowerPC.

---
 arch/powerpc/net/bpf_jit_comp64.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
arch/powerpc/net/bpf_jit_comp64.c +28 -0
index 18da5a866447..c25ba1ad587a 100644
@@ -247,6 +247,34 @@ void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx)
 	bpf_jit_build_fentry_stubs(image, ctx);
 }
 
+void arch_bpf_stack_walk(bool (*consume_fn)(void *, u64, u64, u64), void *cookie)
+{
+	// callback processing always in current context
+	unsigned long fp = current_stack_frame();
+
+	for (;;) {
+		unsigned long *frame = (unsigned long *) fp;
+		unsigned long ip;
+
+		if (!validate_sp(fp, current))
+			return;
+
+		ip = frame[STACK_FRAME_LR_SAVE];
+		if (!ip)
+			break;
+
+		/*
+		 * consume_fn common code expects stack pointer(sp) in third
+		 * argument. There is no sp in ppc64, rather pass frame
+		 * pointer.
+		 */
+		if (ip && !consume_fn(cookie, ip, fp, fp))
+			break;
+
+		fp = frame[0];
+	}
+}
+
 int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_context *ctx, u64 func)
 {
 	unsigned long func_addr = func ? ppc_function_entry((void *)func) : 0;
Signature · 1 line
2.48.1
maintainer
#7 lore raw

Re: [PATCH v4 4/6] powerpc64/bpf: Add arch_bpf_stack_walk() for BPF JIT

Quoted 43 lines · On 23/01/26 2:48 am, [email protected] wrote:

From: Abhishek Dubey <[email protected]>

This function is used by bpf_throw() to unwind the stack until frame of exception-boundary during BPF exception handling.

This function is necessary to support BPF exceptions on PowerPC.

---
  arch/powerpc/net/bpf_jit_comp64.c | 28 ++++++++++++++++++++++++++++
  1 file changed, 28 insertions(+)
arch/powerpc/net/bpf_jit_comp64.c +21 -0
index 18da5a866447..c25ba1ad587a 100644
@@ -247,6 +247,34 @@ void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx)
  	bpf_jit_build_fentry_stubs(image, ctx);
  }
 
+void arch_bpf_stack_walk(bool (*consume_fn)(void *, u64, u64, u64), void *cookie)
+{
+	// callback processing always in current context
+	unsigned long fp = current_stack_frame();
+
+	for (;;) {
+		unsigned long *frame = (unsigned long *) fp;
+		unsigned long ip;
+
+		if (!validate_sp(fp, current))
+			return;
+
+		ip = frame[STACK_FRAME_LR_SAVE];
+		if (!ip)
+			break;
+
+		/*
+		 * consume_fn common code expects stack pointer(sp) in third
+		 * argument. There is no sp in ppc64, rather pass frame
+		 * pointer.
+		 */

arch/powerpc/net/bpf_jit_comp64.c void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx) · line 280

Quoted 10 lines
arch/powerpc/net/bpf_jit_comp64.c +7 -0
+		if (ip && !consume_fn(cookie, ip, fp, fp))
+			break;
+
+		fp = frame[0];
+	}
+}
+
  int bpf_jit_emit_func_call_rel(u32 *image, u32 *fimage, struct codegen_context *ctx, u64 func)
  {
  	unsigned long func_addr = func ? ppc_function_entry((void *)func) : 0;

Ok. Refactoring of arch_stack_walk() & arch_bpf_stack_walk() can be done as a follow-up, but keep the code as identical as possible. Use 'sp' instead of 'fp' and 'stack' instead of 'frame' to make it easy to spot the similarities. Also, a "TODO:" comment on refactoring would be good..

- Hari

#8 lore raw

PATCH v4 5/6 powerpc64/bpf: Support exceptions

In reply to [email protected]

From: Abhishek Dubey <[email protected]>

The modified prologue/epilogue generation code now enables exception-callback to use the stack frame of the program marked as exception boundary, where callee saved registers are stored.

As per ppc64 ABIv2 documentation[1], r14-r31 are callee saved registers. BPF programs on ppc64 already saves r26-r31 registers. Saving the remaining set of callee saved registers(r14-r25) is handled in the next patch.

[1] https://ftp.rtems.org/pub/rtems/people/sebh/ABI64BitOpenPOWERv1.1_16July2015_pub.pdf

---
 arch/powerpc/net/bpf_jit.h        |  2 ++
 arch/powerpc/net/bpf_jit_comp.c   |  7 ++++
 arch/powerpc/net/bpf_jit_comp64.c | 58 +++++++++++++++++++++----------
 3 files changed, 48 insertions(+), 19 deletions(-)
arch/powerpc/net/bpf_jit.h +2 -0
index 56f56fdd4969..82bbf63f0e57 100644
@@ -179,6 +179,8 @@ struct codegen_context {
 	u64 arena_vm_start;
 	u64 user_vm_start;
 	bool is_subprog;
+	bool exception_boundary;
+	bool exception_cb;
 };
 
 #define bpf_to_ppc(r)	(ctx->b2p[r])
arch/powerpc/net/bpf_jit_comp.c +7 -0
index 1a305f0fed27..2607ea0bedef 100644
@@ -207,6 +207,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
 	cgctx.arena_vm_start = bpf_arena_get_kern_vm_start(fp->aux->arena);
 	cgctx.user_vm_start = bpf_arena_get_user_vm_start(fp->aux->arena);
 	cgctx.is_subprog = bpf_is_subprog(fp);
+	cgctx.exception_boundary = fp->aux->exception_boundary;
+	cgctx.exception_cb = fp->aux->exception_cb;
 
 	/* Scouting faux-generate pass 0 */
 	if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
@@ -436,6 +438,11 @@ void bpf_jit_free(struct bpf_prog *fp)
 	bpf_prog_unlock_free(fp);
 }
 
+bool bpf_jit_supports_exceptions(void)
+{
+	return IS_ENABLED(CONFIG_PPC64);
+}
+
 bool bpf_jit_supports_subprog_tailcalls(void)
 {
 	return IS_ENABLED(CONFIG_PPC64);
arch/powerpc/net/bpf_jit_comp64.c +39 -19

102 lines not on this page · raw message

Signature · 1 line
2.48.1
maintainer
#9 lore raw

Re: [PATCH v4 5/6] powerpc64/bpf: Support exceptions

Quoted 144 lines · On 23/01/26 2:48 am, [email protected] wrote:

From: Abhishek Dubey <[email protected]>

The modified prologue/epilogue generation code now enables exception-callback to use the stack frame of the program marked as exception boundary, where callee saved registers are stored.

As per ppc64 ABIv2 documentation[1], r14-r31 are callee saved registers. BPF programs on ppc64 already saves r26-r31 registers. Saving the remaining set of callee saved registers(r14-r25) is handled in the next patch.

[1] https://ftp.rtems.org/pub/rtems/people/sebh/ABI64BitOpenPOWERv1.1_16July2015_pub.pdf

---
  arch/powerpc/net/bpf_jit.h        |  2 ++
  arch/powerpc/net/bpf_jit_comp.c   |  7 ++++
  arch/powerpc/net/bpf_jit_comp64.c | 58 +++++++++++++++++++++----------
  3 files changed, 48 insertions(+), 19 deletions(-)
arch/powerpc/net/bpf_jit.h +2 -0
index 56f56fdd4969..82bbf63f0e57 100644
@@ -179,6 +179,8 @@ struct codegen_context {
  	u64 arena_vm_start;
  	u64 user_vm_start;
  	bool is_subprog;
+	bool exception_boundary;
+	bool exception_cb;
  };
 
  #define bpf_to_ppc(r)	(ctx->b2p[r])
arch/powerpc/net/bpf_jit_comp.c +7 -0
index 1a305f0fed27..2607ea0bedef 100644
@@ -207,6 +207,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
  	cgctx.arena_vm_start = bpf_arena_get_kern_vm_start(fp->aux->arena);
  	cgctx.user_vm_start = bpf_arena_get_user_vm_start(fp->aux->arena);
  	cgctx.is_subprog = bpf_is_subprog(fp);
+	cgctx.exception_boundary = fp->aux->exception_boundary;
+	cgctx.exception_cb = fp->aux->exception_cb;
 
  	/* Scouting faux-generate pass 0 */
  	if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {
@@ -436,6 +438,11 @@ void bpf_jit_free(struct bpf_prog *fp)
  	bpf_prog_unlock_free(fp);
  }
 
+bool bpf_jit_supports_exceptions(void)
+{
+	return IS_ENABLED(CONFIG_PPC64);
+}
+
  bool bpf_jit_supports_subprog_tailcalls(void)
  {
  	return IS_ENABLED(CONFIG_PPC64);
arch/powerpc/net/bpf_jit_comp64.c +9 -2
index c25ba1ad587a..d7cd8ab6559c 100644
@@ -89,7 +89,9 @@ static inline bool bpf_has_stack_frame(struct codegen_context *ctx)
  	 * - the bpf program uses its stack area
  	 * The latter condition is deduced from the usage of BPF_REG_FP
  	 */
-	return ctx->seen & SEEN_FUNC || bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP));
+	return ctx->seen & SEEN_FUNC ||
+	       bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)) ||
+	       ctx->exception_cb;
  }
 
  /*
@@ -161,8 +163,13 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx)
 
 
 
  		EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0));
  		/* this goes in the redzone */
  		EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL)));
-	} else {
+	} else if (!ctx->exception_cb) {
  		/*
+		 * Tailcall jitting for non exception_cb progs only.
+		 * exception_cb won't require tail_call_info to be setup.
+		 *
+		 * tail_call_info interpretation logic:
+		 *

* if tail_call_info < MAX_TAIL_CALL_CNT
* main prog calling first subprog -> copy reference
* else

arch/powerpc/net/bpf_jit_comp64.c +20 -14
@@ -177,8 +184,12 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx)
  		EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, -(BPF_PPC_TAILCALL)));
  	}
 
-	if (bpf_has_stack_frame(ctx)) {
+	if (bpf_has_stack_frame(ctx) && !ctx->exception_cb) {
  		/*
+		 * exception_cb uses boundary frame after stack walk.
+		 * It can simply use redzone, this optimization reduces
+		 * stack walk loop by one level.
+		 *
  		 * We need a stack frame, but we don't necessarily need to
  		 * save/restore LR unless we call other functions
  		 */
@@ -190,23 +201,32 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx)
  		EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx->stack_size)));
  	}
 
-	/*
-	 * Back up non-volatile regs -- BPF registers 6-10
-	 * If we haven't created our own stack frame, we save these
-	 * in the protected zone below the previous stack frame
-	 */
-	for (i = BPF_REG_6; i <= BPF_REG_10; i++)
-		if (bpf_is_seen_register(ctx, bpf_to_ppc(i)))
-			EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(i))));
+	if (!ctx->exception_cb) {
+		/*
+		 * Back up non-volatile regs -- BPF registers 6-10
+		 * If we haven't created our own stack frame, we save these
+		 * in the protected zone below the previous stack frame
+		 */
+		for (i = BPF_REG_6; i <= BPF_REG_10; i++)
+			if (ctx->exception_boundary || bpf_is_seen_register(ctx, bpf_to_ppc(i)))
+				EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1,
+					bpf_jit_stack_offsetof(ctx, bpf_to_ppc(i))));
 
-	if (ctx->arena_vm_start)
-		EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1,
+		if (ctx->exception_boundary || ctx->arena_vm_start)
+			EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1,
  				 bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START))));
 
-	/* Setup frame pointer to point to the bpf stack area */
-	if (bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)))
-		EMIT(PPC_RAW_ADDI(bpf_to_ppc(BPF_REG_FP), _R1,
+		/* Setup frame pointer to point to the bpf stack area */
+		if (bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)))
+			EMIT(PPC_RAW_ADDI(bpf_to_ppc(BPF_REG_FP), _R1,
  				STACK_FRAME_MIN_SIZE + ctx->stack_size));

arch/powerpc/net/bpf_jit_comp64.c void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx) · line 232

Quoted 10 lines
arch/powerpc/net/bpf_jit_comp64.c +7 -0
+	} else {
+		/*
+		 * Exception callback receives Frame Pointer of main
+		 * program as third arg
+		 */
+		EMIT(PPC_RAW_MR(_R1, _R5));
+	}
 
  	if (ctx->arena_vm_start)
  		PPC_LI64(bpf_to_ppc(ARENA_VM_START), ctx->arena_vm_start);

For all practical purposes, the only thing that matters for the prologue of execption_cb subprog seems to be, to set r1 and move on. The below snippet before tailcall setup in the prologue should live the rest of the prologue code unchanged?

if (ctx->exception_cb) {
	/*
	 * Exception callback receives Frame Pointer of main 

	 * program as third arg 

	 */
	EMIT(PPC_RAW_MR(_R1, _R5));
	return;
}

- Hari

A message that is not in the archive (sent off-list or not yet received): [email protected]

maintainer
#11 lore raw

Re: [PATCH v4 5/6] powerpc64/bpf: Support exceptions

Quoted 169 lines · adubey
Quoted 168 lines · Hari Bathini
Quoted 147 lines · On 23/01/26 2:48 am, [email protected] wrote:

From: Abhishek Dubey <[email protected]>

The modified prologue/epilogue generation code now enables exception-callback to use the stack frame of the program marked as exception boundary, where callee saved registers are stored.

As per ppc64 ABIv2 documentation[1], r14-r31 are callee saved registers. BPF programs on ppc64 already saves r26-r31 registers. Saving the remaining set of callee saved registers(r14-r25) is handled in the next patch.

[1] https://ftp.rtems.org/pub/rtems/people/sebh/ ABI64BitOpenPOWERv1.1_16July2015_pub.pdf

---
  arch/powerpc/net/bpf_jit.h        |  2 ++ arch/powerpc/net/bpf_jit_comp.c   |  7 ++++ arch/powerpc/net/bpf_jit_comp64.c | 58 +++++++++++++++++++++----------

  3 files changed, 48 insertions(+), 19 deletions(-)
arch/powerpc/net/bpf_jit.h +2 -0
index 56f56fdd4969..82bbf63f0e57 100644
@@ -179,6 +179,8 @@ struct codegen_context {
       u64 arena_vm_start;
       u64 user_vm_start;
       bool is_subprog;
+    bool exception_boundary;
+    bool exception_cb;
   };
     #define bpf_to_ppc(r)    (ctx->b2p[r])
arch/powerpc/net/bpf_jit_comp.c +0 -0
index 1a305f0fed27..2607ea0bedef 100644
@@ -207,6 +207,8 @@ struct bpf_prog *bpf_int_jit_compile(struct 
 bpf_prog *fp)

      cgctx.arena_vm_start = bpf_arena_get_kern_vm_start(fp->aux-

Quoted 1 line

arena);

      cgctx.user_vm_start = bpf_arena_get_user_vm_start(fp->aux->arena); cgctx.is_subprog = bpf_is_subprog(fp);
+    cgctx.exception_boundary = fp->aux->exception_boundary; +    cgctx.exception_cb = fp->aux->exception_cb;
        /* Scouting faux-generate pass 0 */
      if (bpf_jit_build_body(fp, NULL, NULL, &cgctx, addrs, 0, false)) {

arch/powerpc/net/bpf_jit_comp.c +4 -0
@@ -436,6 +438,11 @@ void bpf_jit_free(struct bpf_prog *fp)
       bpf_prog_unlock_free(fp);
   }
   +bool bpf_jit_supports_exceptions(void)
+{
+    return IS_ENABLED(CONFIG_PPC64);
+}
+
   bool bpf_jit_supports_subprog_tailcalls(void)
   {
       return IS_ENABLED(CONFIG_PPC64);
arch/powerpc/net/bpf_jit_comp64.c +3 -1
index c25ba1ad587a..d7cd8ab6559c 100644
@@ -89,7 +89,9 @@ static inline bool bpf_has_stack_frame(struct 
 codegen_context *ctx)
        * - the bpf program uses its stack area
        * The latter condition is deduced from the usage of BPF_REG_FP
        */
-    return ctx->seen & SEEN_FUNC || bpf_is_seen_register(ctx, 
 bpf_to_ppc(BPF_REG_FP));
+    return ctx->seen & SEEN_FUNC ||
+           bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)) ||
+           ctx->exception_cb;
   }

    /*

arch/powerpc/net/bpf_jit_comp64.c +1 -1
@@ -161,8 +163,13 @@ void bpf_jit_build_prologue(u32 *image, struct 
 codegen_context *ctx)
 
 
 
           EMIT(PPC_RAW_LI(bpf_to_ppc(TMP_REG_1), 0));
           /* this goes in the redzone */
           EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, - (BPF_PPC_TAILCALL)));
-    } else {
+    } else if (!ctx->exception_cb) {

          /*
+         * Tailcall jitting for non exception_cb progs only. +         * exception_cb won't require tail_call_info to be setup. +         *
+         * tail_call_info interpretation logic:
+         *
           * if tail_call_info < MAX_TAIL_CALL_CNT
           *      main prog calling first subprog -> copy reference
           * else

arch/powerpc/net/bpf_jit_comp64.c +5 -0
@@ -177,8 +184,12 @@ void bpf_jit_build_prologue(u32 *image, struct 
 codegen_context *ctx)
           EMIT(PPC_RAW_STD(bpf_to_ppc(TMP_REG_1), _R1, - (BPF_PPC_TAILCALL)));
       }
   -    if (bpf_has_stack_frame(ctx)) {
+    if (bpf_has_stack_frame(ctx) && !ctx->exception_cb) {
           /*
+         * exception_cb uses boundary frame after stack walk.
+         * It can simply use redzone, this optimization reduces
+         * stack walk loop by one level.
+         *
            * We need a stack frame, but we don't necessarily need to
            * save/restore LR unless we call other functions

           */

arch/powerpc/net/bpf_jit_comp64.c +0 -0
@@ -190,23 +201,32 @@ void bpf_jit_build_prologue(u32 *image, struct 
 codegen_context *ctx)

          EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx-

Quoted 1 line

stack_size)));

      }
  -    /*
-     * Back up non-volatile regs -- BPF registers 6-10
-     * If we haven't created our own stack frame, we save these -     * in the protected zone below the previous stack frame
-     */
-    for (i = BPF_REG_6; i <= BPF_REG_10; i++)
-        if (bpf_is_seen_register(ctx, bpf_to_ppc(i)))
-            EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(i))));
+    if (!ctx->exception_cb) {
+        /*
+         * Back up non-volatile regs -- BPF registers 6-10
+         * If we haven't created our own stack frame, we save these +         * in the protected zone below the previous stack frame +         */
+        for (i = BPF_REG_6; i <= BPF_REG_10; i++)
+            if (ctx->exception_boundary || bpf_is_seen_register(ctx, bpf_to_ppc(i)))
+                EMIT(PPC_RAW_STD(bpf_to_ppc(i), _R1,
+                    bpf_jit_stack_offsetof(ctx, bpf_to_ppc(i)))); -    if (ctx->arena_vm_start)
-        EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1,
+        if (ctx->exception_boundary || ctx->arena_vm_start)
+            EMIT(PPC_RAW_STD(bpf_to_ppc(ARENA_VM_START), _R1, bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START)))); -    /* Setup frame pointer to point to the bpf stack area */
-    if (bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)))
-        EMIT(PPC_RAW_ADDI(bpf_to_ppc(BPF_REG_FP), _R1,
+        /* Setup frame pointer to point to the bpf stack area */ +        if (bpf_is_seen_register(ctx, bpf_to_ppc(BPF_REG_FP)))
+            EMIT(PPC_RAW_ADDI(bpf_to_ppc(BPF_REG_FP), _R1, STACK_FRAME_MIN_SIZE + ctx->stack_size));
+    } else {

Quoted 8 lines

+        /*
+         * Exception callback receives Frame Pointer of main +         * program as third arg
+         */
+        EMIT(PPC_RAW_MR(_R1, _R5));
+    }
        if (ctx->arena_vm_start) PPC_LI64(bpf_to_ppc(ARENA_VM_START), ctx->arena_vm_start);

For all practical purposes, the only thing that matters for the prologue of execption_cb subprog seems to be, to set r1 and move on. The below snippet before tailcall setup in the prologue should live the rest of the prologue code unchanged?

if (ctx->exception_cb) {
    /*
     * Exception callback receives Frame Pointer of main

     * program as third arg

Quoted 8 lines
Quoted 4 lines

     */
    EMIT(PPC_RAW_MR(_R1, _R5));
    return;
}

This approach could work, but it may be fragile. If future changes to the callback logic start accessing fields from the active boundary frame, this could lead to incorrect memory access or corruption. Other archs have kept it. Shall we keep it?

True. In fact, exception_cb isn't seem to be restricted from using the BPF stack or arena...

So, the "Setup frame pointer to point to the bpf stack area" part may also have to come out of that !exception_cb condition..

- Hari

#12 lore raw

PATCH v4 6/6 powerpc64/bpf: Additional NVR handling for bpf_throw

In reply to [email protected]

From: Abhishek Dubey <[email protected]>

The bpf_throw() function never returns, if it has clobbered any callee-saved register, those will remain clobbered. The prologue must take care of saving all callee-saved registers in the frame of exception boundary program. Later these additional non volatile registers R14-R25 along with other NVRs are restored back in the epilogue of exception callback.

To achieve above objective, the frame size is determined dynamically to accommodate additional non volatile registers in exception boundary's frame. For non-exception boundary program, the frame size remains optimal. The additional instructions to save & restore r14-r25 registers are emitted only during exception boundary and exception callback program respectively.

---
 arch/powerpc/net/bpf_jit_comp64.c | 81 ++++++++++++++++++++++++++++---
 1 file changed, 74 insertions(+), 7 deletions(-)
arch/powerpc/net/bpf_jit_comp64.c +74 -7
index d7cd8ab6559c..df07375d1595 100644
@@ -32,21 +32,37 @@
  *
  *		[	prev sp		] <-------------
  *		[    tail_call_info	] 8		|
- *		[   nv gpr save area	] 6*8		|
+ *		[   nv gpr save area	] 6*8 + (12*8)	|
  *		[    local_tmp_var	] 24		|
  * fp (r31) -->	[   ebpf stack space	] upto 512	|
  *		[     frame header	] 32/112	|
  * sp (r1) --->	[    stack pointer	] --------------
+ *
+ * Additional (12*8) in 'nv gpr save area' only in case of
+ * exception boundary.
  */
 
 /* for bpf JIT code internal usage */
 #define BPF_PPC_STACK_LOCALS	24
+/*
+ * for additional non volatile registers(r14-r25) to be saved
+ * at exception boundary
+ */
+#define BPF_PPC_EXC_STACK_SAVE (12*8)
+
 /* stack frame excluding BPF stack, ensure this is quadword aligned */
 #define BPF_PPC_STACKFRAME	(STACK_FRAME_MIN_SIZE + \
 				 BPF_PPC_STACK_LOCALS + \
 				 BPF_PPC_STACK_SAVE   + \
 				 BPF_PPC_TAILCALL)
 
+/*
+ * same as BPF_PPC_STACKFRAME with save area for additional
+ * non volatile registers saved at exception boundary.
+ * This is quad-word aligned.
+ */
+#define BPF_PPC_EXC_STACKFRAME (BPF_PPC_STACKFRAME + BPF_PPC_EXC_STACK_SAVE)
+
 /* BPF register usage */
 #define TMP_REG_1	(MAX_BPF_JIT_REG + 0)
 #define TMP_REG_2	(MAX_BPF_JIT_REG + 1)
@@ -103,9 +119,12 @@ static inline bool bpf_has_stack_frame(struct codegen_context *ctx)
  *		[	  ...       	] 		|
  * sp (r1) --->	[    stack pointer	] --------------
  *		[    tail_call_info	] 8
- *		[   nv gpr save area	] 6*8
+ *		[   nv gpr save area	] 6*8 + (12*8)
  *		[    local_tmp_var	] 24
  *		[   unused red zone	] 224
+ *
+ * Additional (12*8) in 'nv gpr save area' only in case of
+ * exception boundary.
  */
 static int bpf_jit_stack_local(struct codegen_context *ctx)
 {
@@ -114,7 +133,12 @@ static int bpf_jit_stack_local(struct codegen_context *ctx)
 		return STACK_FRAME_MIN_SIZE + ctx->stack_size;
 	} else {
 		/* Stack layout with redzone */
-		return -(BPF_PPC_TAILCALL + BPF_PPC_STACK_SAVE + BPF_PPC_STACK_LOCALS);
+		return -(BPF_PPC_TAILCALL
+			+BPF_PPC_STACK_SAVE
+			+(ctx->exception_boundary || ctx->exception_cb ?
+						BPF_PPC_EXC_STACK_SAVE : 0)
+			+BPF_PPC_STACK_LOCALS
+			);
 	}
 }
 
@@ -125,9 +149,19 @@ int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx)
 
 static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg)
 {
-	if (reg >= BPF_PPC_NVR_MIN && reg < 32)
+	int min_valid_nvreg = BPF_PPC_NVR_MIN;
+	/* Default frame size for all cases except exception boundary */
+	int frame_nvr_size = BPF_PPC_STACKFRAME;
+
+	/* Consider all nv regs for handling exceptions */
+	if (ctx->exception_boundary || ctx->exception_cb) {
+		min_valid_nvreg = _R14;
+		frame_nvr_size = BPF_PPC_EXC_STACKFRAME;
+	}
+
+	if (reg >= min_valid_nvreg && reg < 32)
 		return (bpf_has_stack_frame(ctx) ?
-			(BPF_PPC_STACKFRAME + ctx->stack_size) : 0)
+			(frame_nvr_size + ctx->stack_size) : 0)
 				- (8 * (32 - reg)) - BPF_PPC_TAILCALL;
 
 	pr_err("BPF JIT is asking about unknown registers");
@@ -138,6 +172,17 @@ void bpf_jit_realloc_regs(struct codegen_context *ctx)
 {
 }
 
+/*
+ * For exception boundary & exception_cb progs:
+ *     return increased size to accommodate additional NVRs.
+ */
+static int bpf_jit_stack_size(struct codegen_context *ctx)
+{
+	return ctx->exception_boundary || ctx->exception_cb ?
+					BPF_PPC_EXC_STACKFRAME :
+					BPF_PPC_STACKFRAME;
+}
+
 void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx)
 {
 	int i;
@@ -198,7 +243,19 @@ void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx)
 			EMIT(PPC_RAW_STD(_R0, _R1, PPC_LR_STKOFF));
 		}
 
-		EMIT(PPC_RAW_STDU(_R1, _R1, -(BPF_PPC_STACKFRAME + ctx->stack_size)));
+		EMIT(PPC_RAW_STDU(_R1, _R1,
+				-(bpf_jit_stack_size(ctx) + ctx->stack_size)));
+	}
+
+	/*
+	 * Program acting as exception boundary pushes R14..R25 in addition to
+	 * BPF callee-saved non volatile registers. Exception callback uses
+	 * the boundary program's stack frame, recover additionally saved
+	 * registers in epilogue of exception callback.
+	 */
+	if (ctx->exception_boundary) {
+		for (i = _R14; i <= _R25; i++)
+			EMIT(PPC_RAW_STD(i, _R1, bpf_jit_stack_offsetof(ctx, i)));
 	}
 
 	if (!ctx->exception_cb) {
@@ -245,9 +302,19 @@ static void bpf_jit_emit_common_epilogue(u32 *image, struct codegen_context *ctx
 		EMIT(PPC_RAW_LD(bpf_to_ppc(ARENA_VM_START), _R1,
 				bpf_jit_stack_offsetof(ctx, bpf_to_ppc(ARENA_VM_START))));
 
+	if (ctx->exception_cb) {
+		/*
+		 * Recover additionally saved non volatile registers from stack
+		 * frame of exception boundary program.
+		 */
+		for (i = _R14; i <= _R25; i++)
+			EMIT(PPC_RAW_LD(i, _R1, bpf_jit_stack_offsetof(ctx, i)));
+	}
+
 	/* Tear down our stack frame */
 	if (bpf_has_stack_frame(ctx)) {
-		EMIT(PPC_RAW_ADDI(_R1, _R1, BPF_PPC_STACKFRAME + ctx->stack_size));
+		EMIT(PPC_RAW_ADDI(_R1, _R1, bpf_jit_stack_size(ctx) + ctx->stack_size));
+
 		if (ctx->seen & SEEN_FUNC || ctx->exception_cb) {
 			EMIT(PPC_RAW_LD(_R0, _R1, PPC_LR_STKOFF));
 			EMIT(PPC_RAW_MTLR(_R0));
Signature · 1 line
2.48.1
#13 lore raw

Re: [PATCH v4 6/6] powerpc64/bpf: Additional NVR handling for bpf_throw

For the redzone case (no stack frame) with exception_boundary, this formula computes R23 at offset -80 and R24 at offset -72 from R1.
Quoted 4 lines
arch/powerpc/net/bpf_jit_comp64.c +0 -0
index c284a12fce6e..98780a15fc69 100644

[ ... ]

Quoted 10 lines
arch/powerpc/net/bpf_jit_comp64.c +5 -1
@@ -125,9 +149,19 @@ int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx)
 
 static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg)
 {
-	if (reg >= BPF_PPC_NVR_MIN && reg < 32)
+	int min_valid_nvreg = BPF_PPC_NVR_MIN;
+	/* Default frame size for all cases except exception boundary */
+	int frame_nvr_size = BPF_PPC_STACKFRAME;
+
+	/* Consider all nv regs for handling exceptions */

arch/powerpc/net/bpf_jit_comp64.c int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx) · line 165

Quoted 10 lines
arch/powerpc/net/bpf_jit_comp64.c +7 -1
+	if (ctx->exception_boundary || ctx->exception_cb) {
+		min_valid_nvreg = _R14;
+		frame_nvr_size = BPF_PPC_EXC_STACKFRAME;
+	}
+
+	if (reg >= min_valid_nvreg && reg < 32)
 		return (bpf_has_stack_frame(ctx) ?
-			(BPF_PPC_STACKFRAME + ctx->stack_size) : 0)
+			(frame_nvr_size + ctx->stack_size) : 0)
 				- (8 * (32 - reg)) - BPF_PPC_TAILCALL;

For the redzone case (no stack frame) with exception_boundary, this formula computes R23 at offset -80 and R24 at offset -72 from R1.

However, bpf_stf_barrier() uses hardcoded offsets -80(1) and -72(1) to temporarily save R21 and R22 during STF barrier execution:

    asm (
    "	bpf_stf_barrier:		;"
    "		std	21,-80(1)	;"
    "		std	22,-72(1)	;"
    "		sync			;"
    "		ld	21,-80(1)	;"
    "		ld	22,-72(1)	;"
    ...
    );

When an exception_boundary program without its own stack frame executes a memory load that triggers STF_BARRIER_FALLBACK, can bpf_stf_barrier() corrupt the saved R23 and R24 values? The hardcoded offsets would overlap with the extended NVR save area.

If exception_cb later restores these registers, it would get R21/R22 values instead of the original R23/R24 values.

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/21265643939

maintainer
#14 lore raw

Re: [PATCH v4 6/6] powerpc64/bpf: Additional NVR handling for bpf_throw

Quoted 44 lines · On 23/01/26 3:10 am, [email protected] wrote:
Quoted 4 lines
arch/powerpc/net/bpf_jit_comp64.c +0 -0
index c284a12fce6e..98780a15fc69 100644

[ ... ]

Quoted 10 lines
arch/powerpc/net/bpf_jit_comp64.c +5 -1
@@ -125,9 +149,19 @@ int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx)
 
  static int bpf_jit_stack_offsetof(struct codegen_context *ctx, int reg)
  {
-	if (reg >= BPF_PPC_NVR_MIN && reg < 32)
+	int min_valid_nvreg = BPF_PPC_NVR_MIN;
+	/* Default frame size for all cases except exception boundary */
+	int frame_nvr_size = BPF_PPC_STACKFRAME;
+
+	/* Consider all nv regs for handling exceptions */

arch/powerpc/net/bpf_jit_comp64.c int bpf_jit_stack_tailcallinfo_offset(struct codegen_context *ctx) · line 165

Quoted 10 lines
arch/powerpc/net/bpf_jit_comp64.c +7 -1
+	if (ctx->exception_boundary || ctx->exception_cb) {
+		min_valid_nvreg = _R14;
+		frame_nvr_size = BPF_PPC_EXC_STACKFRAME;
+	}
+
+	if (reg >= min_valid_nvreg && reg < 32)
  		return (bpf_has_stack_frame(ctx) ?
-			(BPF_PPC_STACKFRAME + ctx->stack_size) : 0)
+			(frame_nvr_size + ctx->stack_size) : 0)
  				- (8 * (32 - reg)) - BPF_PPC_TAILCALL;

For the redzone case (no stack frame) with exception_boundary, this formula computes R23 at offset -80 and R24 at offset -72 from R1.

However, bpf_stf_barrier() uses hardcoded offsets -80(1) and -72(1) to temporarily save R21 and R22 during STF barrier execution:

     asm (
     "	bpf_stf_barrier:		;"
     "		std	21,-80(1)	;"
     "		std	22,-72(1)	;"
     "		sync			;"
     "		ld	21,-80(1)	;"
     "		ld	22,-72(1)	;"
     ...
     );
Quoted 8 lines

When an exception_boundary program without its own stack frame executes a memory load that triggers STF_BARRIER_FALLBACK, can bpf_stf_barrier() corrupt the saved R23 and R24 values? The hardcoded offsets would overlap with the extended NVR save area.

If exception_cb later restores these registers, it would get R21/R22 values instead of the original R23/R24 values.

That is practically not possible as exception_boundary would mean
at least bpf_throw() is in the program (SEEN_FUNC) and SEEN_FUNC
would setup a frame for the exception_boundary program. Also, bpf_stf_barrier() always uses the redzone. So, it never stomps on the stack of exception_boundary program...

@abhishek, better add the above comment and probably also make bpf_has_stack_frame() return true explicitly for exception_boundary as well (though SEEN_FUNC can't be false in case of exception_boundary)..

- Hari