uprobes/x86: Fix red zone issue for optimized uprobes
Subsystems (11)
Jiri Olsa
reviewerv5 cover uprobes/x86: Fix red zone issue for optimized uprobes
hi,
Andrii reported an issue with optimized uprobes [1] that can clobber redzone area with call instruction storing return address on stack where user code may keep temporary data without adjusting rsp.
Fixing this by moving the optimized uprobes on top of 10-bytes nop instruction, so we can squeeze another instruction to escape the redzone area before doing the call.
Note we need upstream update first for patch 3 (github.com/libbpf/usdt), if we decide to take this change.
thanks,
jirka
v1: https://lore.kernel.org/bpf/[email protected]/
v2: https://lore.kernel.org/bpf/[email protected]/
v3: https://lore.kernel.org/bpf/[email protected]/
v4: https://lore.kernel.org/bpf/[email protected]/
v5 changes:
- several selftests changes and reviewed-by tags [Jakub]
- add more comments in int3_update_unoptimize [Andrii]
- several other minor changes and acks [Oleg]
- move insn_decode out of uprobe_init_insn to simplify the code
- align uprobe_red_zone_test to 64 to make sure nop10 is not on page boundary
v4 changes:
- do not use 2nd int3 (ont +5 offset) because the call instruction is allways the same for the given nop10 address [Andrii/Peter]
- unmap unused trampoline vma after unsuccesfull optimization [sashiko]
- small change to patch#2 moved user_64bit_mode earlier in the path and pass/use mm_struct pointer directly from arch_uprobe_optimize instead of gettting current->mm
Andrii, keeping your ack, please shout otherwise
v3 changes:
- use nop10 update suggested by Peter in [2]
- remove struct uprobe_trampoline object, use vma objects directly instead
- selftests fixes [sashiko]
- ack from Andrii
v2 changes:
- several selftest fixes [sashiko]
- consolidate is_lea_insn and is_call_insn insto single check [Jakub Sitnicki]
- use proper mm_struct object in __in_uprobe_trampoline check [sashiko]
- allow to copy uprobe trampolines vma objects on fork [sashiko]
- change uprobe syscall detection error from -ENXIO to -EPROTO [Andrii]
- added fork/clone tests
- I kept the selftest changes and nop5->nop10 changes in separate commits for easier review, we can squash them later if we want to keep bisect working properly
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/#t
---
Andrii Nakryiko (1):
selftests/bpf: Add tests for uprobe nop10 red zone clobbering
Jiri Olsa (12):
uprobes/x86: Use proper mm_struct in __in_uprobe_trampoline uprobes/x86: Remove struct uprobe_trampoline object uprobes/x86: Do not leak trampoline vma mapping on optimization failure uprobes/x86: Allow to copy uprobe trampolines on fork uprobes/x86: Move optimized uprobe from nop5 to nop10
libbpf: Change has_nop_combo to work on top of nop10
libbpf: Detect uprobe syscall with new error
selftests/bpf: Emit nop,nop10 instructions combo for x86_64 arch selftests/bpf: Change uprobe syscall tests to use nop10 selftests/bpf: Change uprobe/usdt trigger bench code to use nop10 selftests/bpf: Add reattach tests for uprobe syscall selftests/bpf: Add tests for forked/cloned optimized uprobes
arch/x86/kernel/uprobes.c | 416 +++++++++++++++++++++++++++++++++++++++++++----------------------------- include/linux/uprobes.h | 5 - kernel/events/uprobes.c | 10 -- kernel/fork.c | 1 - tools/lib/bpf/features.c | 4 +- tools/lib/bpf/usdt.c | 16 +-- tools/testing/selftests/bpf/bench.c | 20 ++-- tools/testing/selftests/bpf/benchs/bench_trigger.c | 38 +++---- tools/testing/selftests/bpf/benchs/run_bench_uprobes.sh | 2 +- tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c | 326 ++++++++++++++++++++++++++++++++++++++++++++++++++++---- tools/testing/selftests/bpf/prog_tests/usdt.c | 74 +++++++++++-- tools/testing/selftests/bpf/progs/test_usdt.c | 25 +++++ tools/testing/selftests/bpf/usdt.h | 2 +- tools/testing/selftests/bpf/usdt_2.c | 15 ++- 14 files changed, 698 insertions(+), 256 deletions(-)
Jiri Olsa
reviewerPATCH v5 1/13 uprobes/x86: Use proper mm_struct in __in_uprobe_trampoline
In the unregister path we use __in_uprobe_trampoline check with current->mm for the VMA lookup, which is wrong, because we are in the tracer context, not the traced process.
Add mm_struct pointer argument to __in_uprobe_trampoline and changing related callers to pass proper mm_struct pointer.
--- arch/x86/kernel/uprobes.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
arch/x86/kernel/uprobes.c +7 -7
@@ -761,9 +761,9 @@ void arch_uprobe_clear_state(struct mm_struct *mm)destroy_uprobe_trampoline (tramp); } -static bool __in_uprobe_trampoline (unsigned long ip) +static bool __in_uprobe_trampoline (struct mm_struct *mm,unsigned long ip) { -struct vm_area_struct *vma =vma_lookup (current->mm, ip); +struct vm_area_struct *vma =vma_lookup (mm, ip);return vma &&vma_is_special_mapping (vma, &tramp_mapping); }
@@ -776,14 +776,14 @@ static bool in_uprobe_trampoline(unsigned long ip)rcu_read_lock ();if (mmap_lock_speculate_try_begin (mm, &seq)) { - found =__in_uprobe_trampoline (ip); + found =__in_uprobe_trampoline (mm, ip); retry =mmap_lock_speculate_retry (mm, seq); }rcu_read_unlock ();if (retry) {mmap_read_lock (mm); - found =__in_uprobe_trampoline (ip); + found =__in_uprobe_trampoline (mm, ip);mmap_read_unlock (mm); }return found;
@@ -1044,7 +1044,7 @@ static int copy_from_vaddr(struct mm_struct *mm, unsigned long vaddr, void *dst,return 0 ; } -static bool __is_optimized (uprobe_opcode_t *insn,unsigned long vaddr) +static bool __is_optimized (struct mm_struct *mm,uprobe_opcode_t *insn,unsigned long vaddr) {struct __packed __arch_relative_insn {u8 op;
@@ -1053,7 +1053,7 @@ static bool __is_optimized(uprobe_opcode_t *insn, unsigned long vaddr)if (!is_call_insn (insn))return false; -return __in_uprobe_trampoline (vaddr +5 + call->raddr); +return __in_uprobe_trampoline (mm, vaddr +5 + call->raddr); }static int is_optimized (struct mm_struct *mm,unsigned long vaddr)
@@ -1064,7 +1064,7 @@ static int is_optimized(struct mm_struct *mm, unsigned long vaddr) err =copy_from_vaddr (mm, vaddr, &insn,5 );if (err)return err; -return __is_optimized ((uprobe_opcode_t *)&insn, vaddr); +return __is_optimized (mm, (uprobe_opcode_t *)&insn, vaddr); }static bool should_optimize (struct arch_uprobe *auprobe)
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCHv5 01/13] uprobes/x86: Use proper mm_struct in __in_uprobe_trampoline
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Missing vma_start_write() before modifying page tables under mmap_write_lock() in uprobe modification paths. --
commit e98c98f0853de23f35b1d0253c941b8b823e5967
Author: Jiri Olsa <[email protected]>
uprobes/x86: Use proper mm_struct in __in_uprobe_trampoline
This commit updates the unregister path to pass the correct mm_struct to __in_uprobe_trampoline instead of relying on current->mm. It adds the mm_struct argument and updates related callers to provide the proper pointer in the tracer context.
arch/x86/kernel/uprobes.c
Quoted 4 lines
arch/x86/kernel/uprobes.c +0
-0
[ ... ]
[Severity: High]
This isn't a bug introduced by this patch, but there is a pre-existing issue in the uprobes subsystem regarding the per-VMA locks.
Does the uprobe registration and optimization path race with concurrent per-VMA lock readers?
In arch_uprobe_optimize(), the code takes the mmap_write_lock but doesn't call vma_start_write() on the target VMA:
arch/x86/kernel/uprobes.c:arch_uprobe_optimize() {
...
mmap_write_lock(mm);
...
if (__arch_uprobe_optimize(auprobe, mm, vaddr))
...
}A similar sequence happens in register_for_each_vma():
kernel/events/uprobes.c:register_for_each_vma() {
...
mmap_write_lock(mm);
if (check_stable_address_space(mm))
goto unlock;
vma = find_vma(mm, info->vaddr);
...
err = install_breakpoint(uprobe, vma, info->vaddr);
...
}Both of these paths eventually call down to __uprobe_write(), which modifies the page tables directly:
kernel/events/uprobes.c:__uprobe_write() {
...
fw->pte = ptep_clear_flush(vma, vaddr, fw->ptep);
...
}Since mmap_write_lock() alone does not drain existing per-VMA read lock holders (its sequence counter only prevents new acquisitions), could this cause a race where concurrent paths executing under the per-VMA lock (like page faults or MADV_DONTNEED) read stale page table state while __uprobe_write() clears and flushes the PTEs?
Should these paths call vma_start_write() to safely drain the per-VMA readers before making page table modifications?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
[tip: perf/urgent] uprobes/x86: Use proper mm_struct in __in_uprobe_trampoline
In reply to Jiri OlsaThe following commit has been merged into the perf/urgent branch of tip:
The following commit has been merged into the perf/urgent branch of tip:
Commit-ID: 169328645663bae30e9abad4012d52441e085a71 Gitweb: https://git.kernel.org/tip/169328645663bae30e9abad4012d52441e085a71 Author: Jiri Olsa <[email protected]> AuthorDate: Wed, 01 Jul 2026 13:13:25 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 02 Jul 2026 13:21:49 +02:00
uprobes/x86: Use proper mm_struct in __in_uprobe_trampoline
In the unregister path we use __in_uprobe_trampoline check with current->mm for the VMA lookup, which is wrong, because we are in the tracer context, not the traced process.
Add mm_struct pointer argument to __in_uprobe_trampoline and changing related callers to pass proper mm_struct pointer.
--- arch/x86/kernel/uprobes.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
arch/x86/kernel/uprobes.c +7 -7
@@ -761,9 +761,9 @@ void arch_uprobe_clear_state(struct mm_struct *mm)destroy_uprobe_trampoline (tramp); } -static bool __in_uprobe_trampoline (unsigned long ip) +static bool __in_uprobe_trampoline (struct mm_struct *mm,unsigned long ip) { -struct vm_area_struct *vma =vma_lookup (current->mm, ip); +struct vm_area_struct *vma =vma_lookup (mm, ip);return vma &&vma_is_special_mapping (vma, &tramp_mapping); }
@@ -776,14 +776,14 @@ static bool in_uprobe_trampoline(unsigned long ip)rcu_read_lock ();if (mmap_lock_speculate_try_begin (mm, &seq)) { - found =__in_uprobe_trampoline (ip); + found =__in_uprobe_trampoline (mm, ip); retry =mmap_lock_speculate_retry (mm, seq); }rcu_read_unlock ();if (retry) {mmap_read_lock (mm); - found =__in_uprobe_trampoline (ip); + found =__in_uprobe_trampoline (mm, ip);mmap_read_unlock (mm); }return found;
@@ -1044,7 +1044,7 @@ static int copy_from_vaddr(struct mm_struct *mm, unsigned long vaddr, void *dst,return 0 ; } -static bool __is_optimized (uprobe_opcode_t *insn,unsigned long vaddr) +static bool __is_optimized (struct mm_struct *mm,uprobe_opcode_t *insn,unsigned long vaddr) {struct __packed __arch_relative_insn {u8 op;
@@ -1053,7 +1053,7 @@ static bool __is_optimized(uprobe_opcode_t *insn, unsigned long vaddr)if (!is_call_insn (insn))return false; -return __in_uprobe_trampoline (vaddr +5 + call->raddr); +return __in_uprobe_trampoline (mm, vaddr +5 + call->raddr); }static int is_optimized (struct mm_struct *mm,unsigned long vaddr)
@@ -1064,7 +1064,7 @@ static int is_optimized(struct mm_struct *mm, unsigned long vaddr) err =copy_from_vaddr (mm, vaddr, &insn,5 );if (err)return err; -return __is_optimized ((uprobe_opcode_t *)&insn, vaddr); +return __is_optimized (mm, (uprobe_opcode_t *)&insn, vaddr); }static bool should_optimize (struct arch_uprobe *auprobe)
Jiri Olsa
reviewerPATCH v5 2/13 uprobes/x86: Remove struct uprobe_trampoline object
In reply to Jiri OlsaRemoving struct uprobe_trampoline object and it's tracking code, because it's not needed. We can do same thing directly on top of struct vm_area_struct objects.
This makes the code simpler and allows easy propagation of the trampoline vma object into child process in following change.
Note the original code called destroy_uprobe_trampoline if the optimiation failed, but it only freed the struct uprobe_trampoline object, not the vma. The new vma leak is fixed in following change.
--- arch/x86/kernel/uprobes.c | 106 ++++++++------------------------------ include/linux/uprobes.h | 5 -- kernel/events/uprobes.c | 10 ---- kernel/fork.c | 1 - 4 files changed, 22 insertions(+), 100 deletions(-)
arch/x86/kernel/uprobes.c +22 -84
@@ -631,11 +631,6 @@ static struct vm_special_mapping tramp_mapping = { .pages = tramp_mapping_pages, }; -struct uprobe_trampoline { -struct hlist_node node; -unsigned long vaddr; -}; -static bool is_reachable_by_call (unsigned long vtramp,unsigned long vaddr) {long delta = (long )(vaddr +5 - vtramp);
@@ -682,83 +677,28 @@ static unsigned long find_nearest_trampoline(unsigned long vaddr)return high_tramp; } -static struct uprobe_trampoline *create_uprobe_trampoline (unsigned long vaddr) +static struct vm_area_struct *get_uprobe_trampoline (struct mm_struct *mm,unsigned long vaddr) { -struct pt_regs *regs =task_pt_regs (current); -struct mm_struct *mm = current->mm; -struct uprobe_trampoline *tramp; +VMA_ITERATOR (vmi, mm,0 );struct vm_area_struct *vma; -if (!user_64bit_mode (regs)) -return NULL ; +if (vaddr >TASK_SIZE || vaddr <PAGE_SIZE ) +return ERR_PTR (-EINVAL ); + +for_each_vma (vmi, vma) { +if (!vma_is_special_mapping (vma, &tramp_mapping)) +continue ; +if (is_reachable_by_call (vma->vm_start, vaddr)) +return vma; + } vaddr =find_nearest_trampoline (vaddr);if (IS_ERR_VALUE (vaddr)) -return NULL ; +return ERR_PTR (vaddr); - tramp =kzalloc_obj (*tramp); -if (unlikely (!tramp)) -return NULL ; - - tramp->vaddr = vaddr; - vma =_install_special_mapping (mm, tramp->vaddr,PAGE_SIZE , +return _install_special_mapping (mm, vaddr,PAGE_SIZE ,VM_READ |VM_EXEC |VM_MAYEXEC |VM_MAYREAD |VM_DONTCOPY |VM_IO , &tramp_mapping); -if (IS_ERR (vma)) { -kfree (tramp); -return NULL ; - } -return tramp; -} - -static struct uprobe_trampoline *get_uprobe_trampoline (unsigned long vaddr,bool *new) -{ -struct uprobes_state *state = ¤t->mm->uprobes_state; -struct uprobe_trampoline *tramp =NULL ; - -if (vaddr >TASK_SIZE || vaddr <PAGE_SIZE ) -return NULL ; - -hlist_for_each_entry (tramp, &state->head_tramps, node) { -if (is_reachable_by_call (tramp->vaddr, vaddr)) { - *new = false; -return tramp; - } - } - - tramp =create_uprobe_trampoline (vaddr); -if (!tramp) -return NULL ; - - *new = true; -hlist_add_head (&tramp->node, &state->head_tramps); -return tramp; -} - -static void destroy_uprobe_trampoline (struct uprobe_trampoline *tramp) -{ -/* -* We do not unmap and release uprobe trampoline page itself, -* because there's no easy way to make sure none of the threads -* is still inside the trampoline. -*/ -hlist_del (&tramp->node); -kfree (tramp); -} - -void arch_uprobe_init_state (struct mm_struct *mm) -{ -INIT_HLIST_HEAD (&mm->uprobes_state.head_tramps); -} - -void arch_uprobe_clear_state (struct mm_struct *mm) -{ -struct uprobes_state *state = &mm->uprobes_state; -struct uprobe_trampoline *tramp; -struct hlist_node *n; - -hlist_for_each_entry_safe (tramp, n, &state->head_tramps, node) -destroy_uprobe_trampoline (tramp); }static bool __in_uprobe_trampoline (struct mm_struct *mm,unsigned long ip)
@@ -1111,21 +1051,19 @@ int set_orig_insn(struct arch_uprobe *auprobe, struct vm_area_struct *vma,static int __arch_uprobe_optimize (struct arch_uprobe *auprobe,struct mm_struct *mm,unsigned long vaddr) { -struct uprobe_trampoline *tramp; -struct vm_area_struct *vma; -bool new = false; -int err =0 ; +struct pt_regs *regs =task_pt_regs (current); +struct vm_area_struct *vma, *tramp; +int ret; +if (!user_64bit_mode (regs)) +return -EINVAL ; vma =find_vma (mm, vaddr);if (!vma)return -EINVAL ; - tramp =get_uprobe_trampoline (vaddr, &new); -if (!tramp) -return -EINVAL ; - err =swbp_optimize (auprobe, vma, vaddr, tramp->vaddr); -if (WARN_ON_ONCE (err) && new) -destroy_uprobe_trampoline (tramp); -return err; + tramp =get_uprobe_trampoline (mm, vaddr); +if (IS_ERR (tramp)) +return PTR_ERR (tramp); +return WARN_ON_ONCE (swbp_optimize (auprobe, vma, vaddr, tramp->vm_start)); }void arch_uprobe_optimize (struct arch_uprobe *auprobe,unsigned long vaddr)
include/linux/uprobes.h +0 -5
@@ -186,9 +186,6 @@ struct xol_area;struct uprobes_state {struct xol_area *xol_area; -#ifdef CONFIG_X86_64 -struct hlist_head head_tramps; -#endif }; typedef int (*uprobe_write_verify_t)(struct page *page, unsigned long vaddr,
@@ -238,8 +235,6 @@ extern void uprobe_handle_trampoline(struct pt_regs *regs);extern void *arch_uretprobe_trampoline (unsigned long *psize);extern unsigned long uprobe_get_trampoline_vaddr (void );extern void uprobe_copy_from_page (struct page *page,unsigned long vaddr,void *dst,int len); -extern void arch_uprobe_clear_state (struct mm_struct *mm); -extern void arch_uprobe_init_state (struct mm_struct *mm);extern void handle_syscall_uprobe (struct pt_regs *regs,unsigned long bp_vaddr);extern void arch_uprobe_optimize (struct arch_uprobe *auprobe,unsigned long vaddr);extern unsigned long arch_uprobe_get_xol_area (void );
kernel/events/uprobes.c +0 -10
@@ -1806,14 +1806,6 @@ static struct xol_area *get_xol_area(void)return area; } -void __weak arch_uprobe_clear_state (struct mm_struct *mm) -{ -} - -void __weak arch_uprobe_init_state (struct mm_struct *mm) -{ -} -/* * uprobe_clear_state - Free the area allocated for slots. */
@@ -1825,8 +1817,6 @@ void uprobe_clear_state(struct mm_struct *mm)delayed_uprobe_remove (NULL , mm);mutex_unlock (&delayed_uprobe_lock); -arch_uprobe_clear_state (mm); -if (!area)return ;
kernel/fork.c +0 -1
@@ -1064,7 +1064,6 @@ static void mm_init_uprobes_state(struct mm_struct *mm) {#ifdef CONFIG_UPROBES mm->uprobes_state.xol_area =NULL ; -arch_uprobe_init_state (mm);#endif }
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCHv5 02/13] uprobes/x86: Remove struct uprobe_trampoline object
The variable ret is declared but never used in the refactored __arch_uprobe_optimize(). The old code used err which was assigned and returned, but the new code returns directly. Should ret be…
Quoted 4 lines
arch/x86/kernel/uprobes.c +0
-0
[ ... ]
Quoted 19 lines
arch/x86/kernel/uprobes.c +5
-6@@ -1111,21 +1051,19 @@ int set_orig_insn(struct arch_uprobe *auprobe, struct vm_area_struct *vma,static int __arch_uprobe_optimize (struct arch_uprobe *auprobe,struct mm_struct *mm,unsigned long vaddr) { -struct uprobe_trampoline *tramp; -struct vm_area_struct *vma; -bool new = false; -int err =0 ; +struct pt_regs *regs =task_pt_regs (current); +struct vm_area_struct *vma, *tramp; +int ret; +if (!user_64bit_mode (regs)) +return -EINVAL ; vma =find_vma (mm, vaddr);if (!vma)return -EINVAL ; - tramp =get_uprobe_trampoline (vaddr, &new); -if (!tramp)
arch/x86/kernel/uprobes.c int set_orig_insn(struct arch_uprobe *auprobe, struct vm_area_struct *vma, · line 1067
Quoted 10 lines
arch/x86/kernel/uprobes.c +4
-5-return -EINVAL ; - err =swbp_optimize (auprobe, vma, vaddr, tramp->vaddr); -if (WARN_ON_ONCE (err) && new) -destroy_uprobe_trampoline (tramp); -return err; + tramp =get_uprobe_trampoline (mm, vaddr); +if (IS_ERR (tramp)) +return PTR_ERR (tramp); +return WARN_ON_ONCE (swbp_optimize (auprobe, vma, vaddr, tramp->vm_start)); }
The variable ret is declared but never used in the refactored __arch_uprobe_optimize(). The old code used err which was assigned and returned, but the new code returns directly. Should ret be removed to avoid a compiler warning?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28514315910
Jiri Olsa
reviewerPATCH v5 3/13 uprobes/x86: Do not leak trampoline vma mapping on optimization failure
In reply to Jiri OlsaIn case the optimization fails, we leak new-ly created trampoline vma mapping (in case we just created it), let's unmap it.
--- arch/x86/kernel/uprobes.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
arch/x86/kernel/uprobes.c +11 -3
@@ -677,11 +677,14 @@ static unsigned long find_nearest_trampoline(unsigned long vaddr)return high_tramp; } -static struct vm_area_struct *get_uprobe_trampoline (struct mm_struct *mm,unsigned long vaddr) +static struct vm_area_struct *get_uprobe_trampoline (struct mm_struct *mm,unsigned long vaddr, +bool *new_mapping) {VMA_ITERATOR (vmi, mm,0 );struct vm_area_struct *vma; + *new_mapping = false; +if (vaddr >TASK_SIZE || vaddr <PAGE_SIZE )return ERR_PTR (-EINVAL );
@@ -696,6 +699,7 @@ static struct vm_area_struct *get_uprobe_trampoline(struct mm_struct *mm, unsignif (IS_ERR_VALUE (vaddr))return ERR_PTR (vaddr); + *new_mapping = true;return _install_special_mapping (mm, vaddr,PAGE_SIZE ,VM_READ |VM_EXEC |VM_MAYEXEC |VM_MAYREAD |VM_DONTCOPY |VM_IO , &tramp_mapping);
@@ -1053,6 +1057,7 @@ static int __arch_uprobe_optimize(struct arch_uprobe *auprobe, struct mm_struct {struct pt_regs *regs =task_pt_regs (current);struct vm_area_struct *vma, *tramp; +bool new_mapping;int ret;if (!user_64bit_mode (regs))
@@ -1060,10 +1065,13 @@ static int __arch_uprobe_optimize(struct arch_uprobe *auprobe, struct mm_struct vma =find_vma (mm, vaddr);if (!vma)return -EINVAL ; - tramp =get_uprobe_trampoline (mm, vaddr); + tramp =get_uprobe_trampoline (mm, vaddr, &new_mapping);if (IS_ERR (tramp))return PTR_ERR (tramp); -return WARN_ON_ONCE (swbp_optimize (auprobe, vma, vaddr, tramp->vm_start)); + ret =swbp_optimize (auprobe, vma, vaddr, tramp->vm_start); +if (WARN_ON_ONCE (ret) && new_mapping) +WARN_ON_ONCE (do_munmap (mm, tramp->vm_start,PAGE_SIZE ,NULL )); +return ret; }void arch_uprobe_optimize (struct arch_uprobe *auprobe,unsigned long vaddr)
Signature · 1 line
2.54.0
Jiri Olsa
reviewerPATCH v5 4/13 uprobes/x86: Allow to copy uprobe trampolines on fork
In reply to Jiri OlsaWhen we do fork or clone without CLONE_VM the new process won't have uprobe trampoline vma objects and at the same time it will have optimized code calling that trampoline and crash.
Fixing this by allowing vma uprobe trampoline objects to be copied on fork to the new process.
--- arch/x86/kernel/uprobes.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/uprobes.c +1 -1
@@ -701,7 +701,7 @@ static struct vm_area_struct *get_uprobe_trampoline(struct mm_struct *mm, unsign *new_mapping = true;return _install_special_mapping (mm, vaddr,PAGE_SIZE , -VM_READ |VM_EXEC |VM_MAYEXEC |VM_MAYREAD |VM_DONTCOPY |VM_IO , +VM_READ |VM_EXEC |VM_MAYEXEC |VM_MAYREAD |VM_IO , &tramp_mapping); }
Signature · 1 line
2.54.0
Jiri Olsa
reviewerPATCH v5 5/13 uprobes/x86: Move optimized uprobe from nop5 to nop10
In reply to Jiri OlsaAndrii reported an issue with optimized uprobes [1] that can clobber redzone area with call instruction storing return address on stack where user code may keep temporary data without adjusting rsp.
Fixing this by moving the optimized uprobes on top of 10-bytes nop instruction, so we can squeeze another instruction to escape the redzone area before doing the call, like:
lea -0x80(%rsp), %rsp
call tramp
Note the lea instruction is used to adjust the rsp register without changing the flags.
We use nop10 and following transformation to optimized instructions above and back as suggested by Peterz [2].
Optimize path (int3_update_optimize):
1) Initial state after set_swbp() installed the uprobe:
cc 2e 0f 1f 84 00 00 00 00 00
From offset 0 this is INT3 followed by the tail of the original 10-byte NOP.
After a previous unoptimization bytes 5..9 may still contain the old call instruction, which remains valid for threads already there.
2) Rewrite the LEA tail and call displacement:
cc [8d 64 24 80 e8 d0 d1 d2 d3]
From offset 0 this traps on the uprobe INT3. Bytes 1..9 are not executable entry points while byte 0 is trapped.
3) Publish the first LEA byte:
[48] 8d 64 24 80 e8 d0 d1 d2 d3
From offset 0 this is:
lea -0x80(%rsp), %rsp
call <uprobe-trampoline>Unoptimize path (int3_update_unoptimize):
1) Initial optimized state:
48 8d 64 24 80 e8 d0 d1 d2 d3
Same as 3) above.
2) Trap new entries before restoring the NOP bytes:
[cc] 8d 64 24 80 e8 d0 d1 d2 d3
From offset 0 this traps. A thread that had already executed the LEA can still reach the intact CALL at offset 5.
3) Restore bytes 1..4 of the original NOP while keeping byte 0 trapped and byte 5 as CALL.
cc [2e 0f 1f 84] e8 d0 d1 d2 d3
From offset 0 this still traps. Offset 5 is still the CALL for any thread that was already past the first LEA byte.
4) Publish the first byte of the original NOP:
[66] 2e 0f 1f 84 e8 d0 d1 d2 d3
From offset 0 this is the restored 10-byte NOP; the CALL opcode and displacement are now only NOP operands. Offset 5 still decodes as CALL for a thread that was already there.
Tthere is only a single target uprobe-trampoline for the given nop10 instruction address, so the CALL instruction will not be changed across unoptimization/optimization cycles.
Therefore, any task that is preempted at the CALL instruction is guaranteed to observe that CALL and not anything else.
Note as explained in [2] we need to use following nop10:
PF1 PF2 ESC NOPL MOD SIB DISP32
NOP10: 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00 -- cs nopw 0x00000000(%rax,%rax,1)
which means we need to allow 0x2e prefix which maps to INAT_PFX_CS attribute in is_prefix_bad function.
Also changing the uprobe syscall error when called out of uprobe trampoline to -EPROTO, so we are able to detect the fixed kernel.
The optimized uprobe performance stays the same:
uprobe-nop : 3.129 ± 0.013M/s
uprobe-push : 3.045 ± 0.006M/s
uprobe-ret : 1.095 ± 0.004M/s
--> uprobe-nop10 : 7.170 ± 0.020M/s
uretprobe-nop : 2.143 ± 0.021M/s
uretprobe-push : 2.090 ± 0.000M/s
uretprobe-ret : 0.942 ± 0.000M/s
--> uretprobe-nop10: 3.381 ± 0.003M/s
usdt-nop : 3.245 ± 0.004M/s
--> usdt-nop10 : 7.256 ± 0.023M/s[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/#t
--- arch/x86/kernel/uprobes.c | 292 ++++++++++++++++++++++++++++---------- 1 file changed, 216 insertions(+), 76 deletions(-)
arch/x86/kernel/uprobes.c +216 -76
@@ -276,15 +276,9 @@ static bool is_prefix_bad(struct insn *insn)return false; } -static int uprobe_init_insn (struct arch_uprobe *auprobe,struct insn *insn,bool x86_64) +static int uprobe_init_insn (struct arch_uprobe *auprobe,struct insn *insn) { -enum insn_mode m = x86_64 ?INSN_MODE_64 :INSN_MODE_32 ;u32 volatile *good_insns; -int ret; - - ret =insn_decode (insn, auprobe->insn,sizeof (auprobe->insn), m); -if (ret <0 ) -return -ENOEXEC ;if (is_prefix_bad (insn))return -ENOTSUPP ;
@@ -293,7 +287,7 @@ static int uprobe_init_insn(struct arch_uprobe *auprobe, struct insn *insn, boolif (insn_masking_exception (insn))return -ENOTSUPP ; -if (x86_64) +if (insn->x86_64) good_insns = good_insns_64;else good_insns = good_insns_32;
@@ -631,9 +625,29 @@ static struct vm_special_mapping tramp_mapping = { .pages = tramp_mapping_pages, }; + +#define LEA_INSN_SIZE 5 +#define OPT_INSN_SIZE (LEA_INSN_SIZE + CALL_INSN_SIZE) +#define REDZONE_SIZE 0x80 + +static const u8 lea_rsp[] = {0x48 ,0x8d ,0x64 ,0x24 ,0x80 }; + +static bool is_opt_insns (const uprobe_opcode_t *insn) +{ +return !memcmp (insn, lea_rsp,LEA_INSN_SIZE ) && + insn[LEA_INSN_SIZE ] ==CALL_INSN_OPCODE ; +} + +static bool is_swbp_opt_insns (uprobe_opcode_t *insn) +{ +return is_swbp_insn (&insn[0 ]) && + !memcmp (&insn[1 ], &lea_rsp[1 ],LEA_INSN_SIZE -1 ) && + insn[LEA_INSN_SIZE ] ==CALL_INSN_OPCODE ; +} +static bool is_reachable_by_call (unsigned long vtramp,unsigned long vaddr) { -long delta = (long )(vaddr +5 - vtramp); +long delta = (long )(vaddr +OPT_INSN_SIZE - vtramp);return delta >=INT_MIN && delta <=INT_MAX ; }
@@ -646,7 +660,7 @@ static unsigned long find_nearest_trampoline(unsigned long vaddr) };unsigned long low_limit, high_limit;unsigned long low_tramp, high_tramp; -unsigned long call_end = vaddr +5 ; +unsigned long call_end = vaddr +OPT_INSN_SIZE ;if (check_add_overflow (call_end,INT_MIN , &low_limit)) low_limit =PAGE_SIZE ;
@@ -754,7 +768,7 @@ SYSCALL_DEFINE0(uprobe)/* Allow execution only from uprobe trampolines. */ if (!in_uprobe_trampoline (regs->ip)) -return -ENXIO ; +return -EPROTO ; err =copy_from_user (&args, (void __user *)regs->sp,sizeof (args));if (err)
@@ -770,8 +784,8 @@ SYSCALL_DEFINE0(uprobe) regs->ax = args.ax; regs->r11 = args.r11; regs->cx = args.cx; - regs->ip = args.retaddr -5 ; - regs->sp +=sizeof (args); + regs->ip = args.retaddr -OPT_INSN_SIZE ; + regs->sp +=sizeof (args) +REDZONE_SIZE ; regs->orig_ax =-1 ; sp = regs->sp;
@@ -788,12 +802,12 @@ SYSCALL_DEFINE0(uprobe)*/ if (regs->sp != sp) {/* skip the trampoline call */ -if (args.retaddr -5 == regs->ip) - regs->ip +=5 ; +if (args.retaddr -OPT_INSN_SIZE == regs->ip) + regs->ip +=OPT_INSN_SIZE ;return regs->ax; } - regs->sp -=sizeof (args); + regs->sp -=sizeof (args) +REDZONE_SIZE ;/* for the case uprobe_consumer has changed ax/r11/cx */ args.ax = regs->ax;
@@ -801,7 +815,7 @@ SYSCALL_DEFINE0(uprobe) args.cx = regs->cx;/* keep return address unless we are instructed otherwise */ -if (args.retaddr -5 != regs->ip) +if (args.retaddr -OPT_INSN_SIZE != regs->ip) args.retaddr = regs->ip;if (shstk_push (args.retaddr) == -EFAULT )
@@ -835,7 +849,7 @@ asm ("pop %rax\n" "pop %r11\n" "pop %rcx\n" -"ret\n" + "ret $" __stringify(REDZONE_SIZE) "\n""int3\n" ".balign " __stringify(PAGE_SIZE) "\n"".popsection\n"
@@ -853,7 +867,8 @@ late_initcall(arch_uprobes_init);enum {EXPECT_SWBP , -EXPECT_CALL , +EXPECT_OPTIMIZED , +EXPECT_SWBP_OPTIMIZED , };struct write_opcode_ctx {
@@ -861,30 +876,29 @@ struct write_opcode_ctx {int expect; }; -static int is_call_insn (uprobe_opcode_t *insn) -{ -return *insn ==CALL_INSN_OPCODE ; -} -/* -* Verification callback used by int3_update uprobe_write calls to make sure -* the underlying instruction is as expected - either int3 or call. +* Verification callback used by uprobe_write calls to make sure the underlying +* instruction is in the expected stage of the INT3 update sequence. */ static int verify_insn (struct page *page,unsigned long vaddr,uprobe_opcode_t *new_opcode,int nbytes,void *data) {struct write_opcode_ctx *ctx = data; -uprobe_opcode_t old_opcode[5 ]; +uprobe_opcode_t old_opcode[OPT_INSN_SIZE ]; -uprobe_copy_from_page (page, ctx->base, (uprobe_opcode_t *) &old_opcode,5 ); +uprobe_copy_from_page (page, ctx->base, old_opcode,OPT_INSN_SIZE );switch (ctx->expect) {case EXPECT_SWBP :if (is_swbp_insn (&old_opcode[0 ]))return 1 ;break ; -case EXPECT_CALL : -if (is_call_insn (&old_opcode[0 ])) +case EXPECT_OPTIMIZED : +if (is_opt_insns (&old_opcode[0 ])) +return 1 ; +break ; +case EXPECT_SWBP_OPTIMIZED : +if (is_swbp_opt_insns (&old_opcode[0 ]))return 1 ;break ; }
@@ -893,48 +907,122 @@ static int verify_insn(struct page *page, unsigned long vaddr, uprobe_opcode_t * }/* -* Modify multi-byte instructions by using INT3 breakpoints on SMP. +* Modify the optimized instruction by using INT3 breakpoints on SMP. * We completely avoid using stop_machine() here, and achieve the * synchronization using INT3 breakpoints and SMP cross-calls. * (borrowed comment from smp_text_poke_batch_finish) * -* The way it is done: -* - Add an INT3 trap to the address that will be patched -* - SMP sync all CPUs -* - Update all but the first byte of the patched range -* - SMP sync all CPUs -* - Replace the first byte (INT3) by the first byte of the replacing opcode -* - SMP sync all CPUs +* For optimization (int3_update_optimize): +* 1) Start with the uprobe INT3 trap already installed +* 2) Update everything but the first byte +* 3) Replace the first INT3 by the first byte of the LEA instruction +* +* For unoptimization (int3_update_unoptimize): +* 1) Start with the optimized uprobe lea/call instructions +* 2) Add an INT3 trap to the address that will be patched +* 3) Restore the NOP bytes before the call opcode +* 4) Replace the first INT3 by the first byte of the NOP instruction +* +* Note that unoptimization deliberately keeps the call opcode and displacement +* in bytes 5..9. Those bytes become operands of the restored 10-byte NOP. +* +* Since there is only a single target uprobe-trampoline for the given nop10 +* instruction address, the CALL instruction will not be changed across +* unoptimization/optimization cycles. +* Therefore, any task that is preempted at the CALL instruction is guaranteed +* to observe that CALL and not anything else. */ -static int int3_update (struct arch_uprobe *auprobe,struct vm_area_struct *vma, -unsigned long vaddr,char *insn,bool optimize) +static int int3_update_optimize (struct arch_uprobe *auprobe,struct vm_area_struct *vma, +unsigned long vaddr,uprobe_opcode_t *insn) { -uprobe_opcode_t int3 =UPROBE_SWBP_INSN ;struct write_opcode_ctx ctx = { .base = vaddr, };int err;/* -* Write int3 trap. +* 1) Initial state after set_swbp() installed the uprobe: +* cc 2e 0f 1f 84 00 00 00 00 00 * -* The swbp_optimize path comes with breakpoint already installed, -* so we can skip this step for optimize == true. +* After a previous unoptimization bytes 5..9 may still contain the +* old call instruction, which remains valid for threads already there. */ -if (!optimize) { - ctx.expect =EXPECT_CALL ; - err =uprobe_write (auprobe, vma, vaddr, &int3,1 , verify_insn, - true/* is_register */ , false/* do_update_ref_ctr */ , - &ctx); -if (err) -return err; - } +smp_text_poke_sync_each_cpu (); + +/* +* 2) Rewrite the LEA tail and call displacement: +* cc [8d 64 24 80 e8 d0 d1 d2 d3] +*/ + ctx.expect =EXPECT_SWBP ; + err =uprobe_write (auprobe, vma, vaddr +1 , insn +1 , +OPT_INSN_SIZE -1 , verify_insn, + true/* is_register */ , false/* do_update_ref_ctr */ , + &ctx); +if (err) +return err; + +smp_text_poke_sync_each_cpu (); + +/* +* 3) Publish the first LEA byte: +* [48] 8d 64 24 80 e8 d0 d1 d2 d3 +* +* From offset 0 this is: +* lea -0x80(%rsp), %rsp +* call <uprobe-trampoline> +*/ + ctx.expect =EXPECT_SWBP_OPTIMIZED ; + err =uprobe_write (auprobe, vma, vaddr, insn,1 , verify_insn, + true/* is_register */ , false/* do_update_ref_ctr */ , + &ctx); +if (err) + goto error;smp_text_poke_sync_each_cpu (); +return 0 ; -/* Write all but the first byte of the patched range. */ +error: +/* +* In all intermediate states byte 0 is INT3, so EXPECT_SWBP covers every +* case. Restore NOP bytes 1..4, but keep the valid CALL at bytes 5..9 +* for a thread that had already executed the LEA before a previous +* unoptimization. +*/ ctx.expect =EXPECT_SWBP ; - err =uprobe_write (auprobe, vma, vaddr +1 , insn +1 ,4 , verify_insn, +uprobe_write (auprobe, vma, vaddr +1 , auprobe->insn +1 , +LEA_INSN_SIZE -1 , verify_insn, true, false, &ctx); +smp_text_poke_sync_each_cpu (); +return err; +} + +static int int3_update_unoptimize (struct arch_uprobe *auprobe,struct vm_area_struct *vma, +unsigned long vaddr,uprobe_opcode_t *insn) +{ +uprobe_opcode_t int3 =UPROBE_SWBP_INSN ; +struct write_opcode_ctx ctx = { + .base = vaddr, + .expect =EXPECT_OPTIMIZED , + }; +int err; + +/* +* Note the first two uprobe_write calls use is_register=true, because they +* are intermediate patching states while the probe is still active, so +* we force the exclusive anonymous page for the update. +* Also we use do_update_ref_ctr=false because refctr was already updated by +* the initial int3 install. +* +* The last uprobe_write to nop10 instruction is called with is_register=false +* and do_update_ref_ctr=true to trigger the refctr update and to instruct +* uprobe_write to zap the anonymous page if it now matches the file page. +* +* 1) Initial optimized state: +* 48 8d 64 24 80 e8 d0 d1 d2 d3 +* +* 2) Trap new entries before restoring the NOP bytes: +* [cc] 8d 64 24 80 e8 d0 d1 d2 d3 +*/ + err =uprobe_write (auprobe, vma, vaddr, &int3,1 , verify_insn, true/* is_register */ , false/* do_update_ref_ctr */ , &ctx);if (err)
@@ -943,13 +1031,31 @@ static int int3_update(struct arch_uprobe *auprobe, struct vm_area_struct *vma,smp_text_poke_sync_each_cpu ();/* -* Write first byte. +* 3) Restore bytes 1..4 of the original NOP while keeping byte 0 trapped +* and byte 5 as CALL: +* cc [2e 0f 1f 84] e8 d0 d1 d2 d3 +*/ + ctx.expect =EXPECT_SWBP_OPTIMIZED ; + err =uprobe_write (auprobe, vma, vaddr +1 , insn +1 , +LEA_INSN_SIZE -1 , verify_insn, + true/* is_register */ , false/* do_update_ref_ctr */ , + &ctx); +if (err) +return err; + +smp_text_poke_sync_each_cpu (); + +/* +* 4) Publish the first byte of the original NOP: +* [66] 2e 0f 1f 84 e8 d0 d1 d2 d3 * -* The swbp_unoptimize needs to finish uprobe removal together -* with ref_ctr update, using uprobe_write with proper flags. +* From offset 0 this is the restored 10-byte NOP; the CALL opcode and +* displacement are now only NOP operands. Offset 5 still decodes as +* CALL for a thread that was already there. */ + ctx.expect =EXPECT_SWBP ; err =uprobe_write (auprobe, vma, vaddr, insn,1 , verify_insn, - optimize/* is_register */ , !optimize/* do_update_ref_ctr */ , + false/* is_register */ , true/* do_update_ref_ctr */ , &ctx);if (err)return err;
@@ -961,17 +1067,25 @@ static int int3_update(struct arch_uprobe *auprobe, struct vm_area_struct *vma,static int swbp_optimize (struct arch_uprobe *auprobe,struct vm_area_struct *vma,unsigned long vaddr,unsigned long tramp) { -u8 call[5 ]; +u8 insn[OPT_INSN_SIZE ], *call = &insn[LEA_INSN_SIZE ]; -__text_gen_insn (call,CALL_INSN_OPCODE , (const void *) vaddr, +/* +* We have nop10 instruction (with first byte overwritten to int3), +* changing it to: +* lea -0x80(%rsp), %rsp +* call tramp +*/ +memcpy (insn, lea_rsp,LEA_INSN_SIZE ); +__text_gen_insn (call,CALL_INSN_OPCODE , + (const void *) (vaddr +LEA_INSN_SIZE ), (const void *) tramp,CALL_INSN_SIZE ); -return int3_update (auprobe, vma, vaddr, call, true/* optimize */ ); +return int3_update_optimize (auprobe, vma, vaddr, insn); }static int swbp_unoptimize (struct arch_uprobe *auprobe,struct vm_area_struct *vma,unsigned long vaddr) { -return int3_update (auprobe, vma, vaddr, auprobe->insn, false/* optimize */ ); +return int3_update_unoptimize (auprobe, vma, vaddr, auprobe->insn); }static int copy_from_vaddr (struct mm_struct *mm,unsigned long vaddr,void *dst,int len)
@@ -993,19 +1107,19 @@ static bool __is_optimized(struct mm_struct *mm, uprobe_opcode_t *insn, unsignedstruct __packed __arch_relative_insn {u8 op;s32 raddr; - } *call = (struct __arch_relative_insn *) insn; + } *call = (struct __arch_relative_insn *)(insn +LEA_INSN_SIZE ); -if (!is_call_insn (insn)) +if (!is_opt_insns (insn))return false; -return __in_uprobe_trampoline (mm, vaddr +5 + call->raddr); +return __in_uprobe_trampoline (mm, vaddr +OPT_INSN_SIZE + call->raddr); }static int is_optimized (struct mm_struct *mm,unsigned long vaddr) { -uprobe_opcode_t insn[5 ]; +uprobe_opcode_t insn[OPT_INSN_SIZE ];int err; - err =copy_from_vaddr (mm, vaddr, &insn,5 ); + err =copy_from_vaddr (mm, vaddr, &insn,OPT_INSN_SIZE );if (err)return err;return __is_optimized (mm, (uprobe_opcode_t *)&insn, vaddr);
@@ -1077,7 +1191,7 @@ static int __arch_uprobe_optimize(struct arch_uprobe *auprobe, struct mm_structvoid arch_uprobe_optimize (struct arch_uprobe *auprobe,unsigned long vaddr) {struct mm_struct *mm = current->mm; -uprobe_opcode_t insn[5 ]; +uprobe_opcode_t insn[OPT_INSN_SIZE ];if (!should_optimize (auprobe))return ;
@@ -1088,7 +1202,7 @@ void arch_uprobe_optimize(struct arch_uprobe *auprobe, unsigned long vaddr)* Check if some other thread already optimized the uprobe for us, * if it's the case just go away silently. */ -if (copy_from_vaddr (mm, vaddr, &insn,5 )) +if (copy_from_vaddr (mm, vaddr, &insn,OPT_INSN_SIZE )) goto unlock;if (!is_swbp_insn ((uprobe_opcode_t *) &insn)) goto unlock;
@@ -1104,16 +1218,32 @@ void arch_uprobe_optimize(struct arch_uprobe *auprobe, unsigned long vaddr) mmap_write_unlock(mm); } +static bool is_optimizable_nop10 (struct insn *insn) +{ +static const u8 nop10_prefix[] = { +0x66 ,0x2e ,0x0f ,0x1f ,0x84 + }; + +/* +* Restrict this to the 10-byte NOP form whose last 5 bytes are +* SIB/displacement operands. Unoptimization keeps the call opcode and +* displacement in those bytes, so other NOP encodings are not safe. +*/ +return insn->length ==OPT_INSN_SIZE && +insn_is_nop (insn) && + !memcmp (insn->kaddr, nop10_prefix,ARRAY_SIZE (nop10_prefix)); +} +static bool can_optimize (struct insn *insn,unsigned long vaddr) { -if (!insn->x86_64 || insn->length !=5 ) +if (!insn->x86_64)return false; -if (!insn_is_nop (insn)) +if (!is_optimizable_nop10 (insn))return false;/* We can't do cross page atomic writes yet. */ -return PAGE_SIZE - (vaddr & ~PAGE_MASK ) >=5 ; +return PAGE_SIZE - (vaddr & ~PAGE_MASK ) >=OPT_INSN_SIZE ; }#else /* 32-bit: */ /*
@@ -1485,16 +1615,26 @@ static int push_setup_xol_ops(struct arch_uprobe *auprobe, struct insn *insn)*/ int arch_uprobe_analyze_insn (struct arch_uprobe *auprobe,struct mm_struct *mm,unsigned long addr) { +enum insn_mode m =is_64bit_mm (mm) ?INSN_MODE_64 :INSN_MODE_32 ;u8 fix_ip_or_call =UPROBE_FIX_IP ;struct insn insn;int ret; - ret =uprobe_init_insn (auprobe, &insn,is_64bit_mm (mm)); -if (ret) -return ret; + ret =insn_decode (&insn, auprobe->insn,sizeof (auprobe->insn), m); +if (ret <0 ) +return -ENOEXEC ; -if (can_optimize (&insn, addr)) +/* +* No need to check instruction in uprobe_init_insn in case we +* are on top of optimizable nop10. +*/ +if (can_optimize (&insn, addr)) {set_bit (ARCH_UPROBE_FLAG_CAN_OPTIMIZE , &auprobe->flags); + }else { + ret =uprobe_init_insn (auprobe, &insn); +if (ret) +return ret; + } ret =branch_setup_xol_ops (auprobe, &insn);if (ret != -ENOSYS )
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCHv5 05/13] uprobes/x86: Move optimized uprobe from nop5 to nop10
This isn't a bug, but could the changelog be updated to describe the approach actually taken? The patch bypasses uprobe_init_insn() (and therefore is_prefix_bad()) entirely when can_optimize() is…
Quoted 6 lines
Note as explained in [2] we need to use following nop10:
PF1 PF2 ESC NOPL MOD SIB DISP32
NOP10:0x66,0x2e,0x0f,0x1f,0x84,0x00,0x00,0x00,0x00,0x00-- cs nopw 0x00000000(%rax,%rax,1)which means we need to allow
0x2eprefix which maps toINAT_PFX_CSattribute inis_prefix_badfunction.
This isn't a bug, but could the changelog be updated to describe the approach actually taken? The patch bypasses uprobe_init_insn() (and therefore is_prefix_bad()) entirely when can_optimize() is true for the nop10. The is_prefix_bad() function itself remains unchanged and still returns true for the CS prefix.
Quoted 22 lines
Unoptimize path (
int3_update_unoptimize):1) Initial optimized state:
48 8d 64 24 80 e8 d0 d1 d2 d3
Same as 3) above.2) Trap new entries before restoring the NOP bytes:
[cc] 8d 64 24 80 e8 d0 d1 d2 d3From offset 0 this traps. A thread that had already executed the LEA can still reach the intact CALL at offset 5.
3) Restore bytes 1..4 of the original NOP while keeping byte 0 trapped and byte 5 as CALL.
cc [2e 0f 1f 84] e8 d0 d1 d2 d3From offset 0 this still traps. Offset 5 is still the CALL for any thread that was already past the first LEA byte.
4) Publish the first byte of the original NOP:
[66] 2e 0f 1f 84 e8 d0 d1 d2 d3
Quoted 7 lines
From offset 0 this is the restored 10-byte NOP; the CALL opcode and displacement are now only NOP operands. Offset 5 still decodes as CALL for a thread that was already there.
Tthere is only a single target uprobe-trampoline for the given nop10 instruction address, so the CALL instruction will not be changed across unoptimization/optimization cycles.
This isn't a bug, but there's a typo: "Tthere is only a single target" should be "There is only a single target".
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28514315910
Jiri Olsa
reviewerPATCH v5 6/13 libbpf: Change has_nop_combo to work on top of nop10
In reply to Jiri OlsaWe now expect nop combo with 10 bytes nop instead of 5 bytes nop, fixing has_nop_combo to reflect that.
--- tools/lib/bpf/usdt.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
tools/lib/bpf/usdt.c +8 -8
@@ -305,7 +305,7 @@ struct usdt_manager *usdt_manager_new(struct bpf_object *obj)/* * Detect kernel support for uprobe() syscall, it's presence means we can -* take advantage of faster nop5 uprobe handling. +* take advantage of faster nop10 uprobe handling. * Added in: 56101b69c919 ("uprobes/x86: Add uprobe syscall to speed up uprobe") */ man->has_uprobe_syscall =kernel_supports (obj,FEAT_UPROBE_SYSCALL );
@@ -605,14 +605,14 @@ static int parse_usdt_spec(struct usdt_spec *spec, const struct usdt_note *note,#if defined(__x86_64__)static bool has_nop_combo (int fd,long off) { -unsigned char nop_combo[6 ] = { -0x90 ,0x0f ,0x1f ,0x44 ,0x00 ,0x00 /* nop,nop5 */ +unsigned char nop_combo[11 ] = { +0x90 ,0x66 ,0x2e ,0x0f ,0x1f ,0x84 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 , }; -unsigned char buf[6 ]; +unsigned char buf[11 ]; -if (pread (fd, buf,6 , off) !=6 ) +if (pread (fd, buf,11 , off) !=11 )return false; -return memcmp (buf, nop_combo,6 ) ==0 ; +return memcmp (buf, nop_combo,11 ) ==0 ; }#else static bool has_nop_combo (int fd,long off)
@@ -825,8 +825,8 @@ static int collect_usdt_targets(struct usdt_manager *man, struct elf_fd *elf_fd,memset (target,0 ,sizeof (*target));/* -* We have uprobe syscall and usdt with nop,nop5 instructions combo, -* so we can place the uprobe directly on nop5 (+1) and get this probe +* We have uprobe syscall and usdt with nop,nop10 instructions combo, +* so we can place the uprobe directly on nop10 (+1) and get this probe * optimized. */ if (man->has_uprobe_syscall &&has_nop_combo (elf_fd->fd, usdt_rel_ip)) {
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCHv5 06/13] libbpf: Change has_nop_combo to work on top of nop10
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The patch breaks USDT optimization selftests and functionally fails to enable the optimization…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The patch breaks USDT optimization selftests and functionally fails to enable the optimization because the kernel does not support optimizing 10-byte NOPs. --
commit 7b26e4395329f0d2e4ebce57947c8615c8552ea2
Author: Jiri Olsa <[email protected]>
libbpf: Change has_nop_combo to work on top of nop10
This commit modifies libbpf's USDT parsing to check for an 11-byte NOP combination (nop + nop10) instead of the previous 6-byte combination (nop + nop5). The intent is to leverage faster uprobe handling.
Quoted 4 lines
tools/lib/bpf/usdt.c +0
-0
[ ... ]
Quoted 8 lines
tools/lib/bpf/usdt.c +2
-2@@ -604,14 +604,14 @@ static int parse_usdt_spec(struct usdt_spec *spec, const struct usdt_note *note, #if defined(__x86_64__) static bool has_nop_combo(int fd, long off) { - unsigned char nop_combo[6] = { - 0x90, 0x0f, 0x1f, 0x44, 0x00, 0x00 /* nop,nop5 */ + unsigned char nop_combo[11] = { + 0x90, 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00,
tools/lib/bpf/usdt.c static int parse_usdt_spec(struct usdt_spec *spec, const struct usdt_note *note, · line 615
Quoted 10 lines
tools/lib/bpf/usdt.c +3
-3}; -unsigned char buf[6 ]; +unsigned char buf[11 ]; -if (pread (fd, buf,6 , off) !=6 ) +if (pread (fd, buf,11 , off) !=11 )return false; -return memcmp (buf, nop_combo,6 ) ==0 ; +return memcmp (buf, nop_combo,11 ) ==0 ; }
[Severity: High]
Will this modification break the USDT optimization selftests and prevent uprobes from being optimized?
The BPF selftests still emit a 6-byte NOP combo:
tools/testing/selftests/bpf/usdt.h:
#define USDT_NOP .byte 0x90, 0x0f, 0x1f, 0x44, 0x00, 0x0 /* nop, nop5 */
Because libbpf now strictly requires an 11-byte sequence, it skips the optimization for existing tests. This causes subtest_optimized_attach() in tools/testing/selftests/bpf/prog_tests/usdt.c to fail its assertion:
ASSERT_EQ(*(addr_2 + 1), 0xe8, "call");
The assertion fails because the uprobe falls back to an unoptimized 1-byte INT3 (0xcc) instead of the optimized 5-byte call (0xe8) at addr_2 + 1.
Furthermore, if user-space were to emit the 10-byte NOP, is the kernel currently capable of optimizing it? Looking at arch/x86/kernel/uprobes.c:
arch/x86/kernel/uprobes.c:can_optimize() {
...
if (!insn->x86_64 || insn->length != 5)
return false;
...
}Because the kernel strictly requires the instruction length to be exactly 5 bytes, a 10-byte NOP will be rejected for optimization. The kernel will fall back to an INT3 uprobe, meaning the intended uprobe syscall optimization is not achieved.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Jiri Olsa
reviewerPATCH v5 7/13 libbpf: Detect uprobe syscall with new error
In reply to Jiri OlsaIn the previous optimized uprobe fix we changed the syscall error used for its detection from ENXIO to EPROTO.
Changing related probe_uprobe_syscall detection check.
--- tools/lib/bpf/features.c | 4 ++-- tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)
tools/lib/bpf/features.c +2 -2
@@ -577,10 +577,10 @@ static int probe_ldimm64_full_range_off(int token_fd)static int probe_uprobe_syscall (int token_fd) {/* -* If kernel supports uprobe() syscall, it will return -ENXIO when called +* If kernel supports uprobe() syscall, it will return -EPROTO when called * from the outside of a kernel-generated uprobe trampoline. */ -return syscall (__NR_uprobe) <0 && errno ==ENXIO ; +return syscall (__NR_uprobe) <0 && errno ==EPROTO ; }#else static int probe_uprobe_syscall (int token_fd)
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +1 -1
@@ -762,7 +762,7 @@ static void test_uprobe_error(void)long err =syscall (__NR_uprobe);ASSERT_EQ (err,-1 ,"error" ); -ASSERT_EQ (errno,ENXIO ,"errno" ); +ASSERT_EQ (errno,EPROTO ,"errno" ); }static void __test_uprobe_syscall (void )
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCHv5 07/13] libbpf: Detect uprobe syscall with new error
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Unmerged cross-tree dependency breaks `test_uprobe_error()` selftest and drops backward…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Unmerged cross-tree dependency breaks test_uprobe_error() selftest and drops backward compatibility. --
commit 7f7a698fa2699de0d1d3b920e07eb13831d5223e
Author: Jiri Olsa <[email protected]>
libbpf: Detect uprobe syscall with new error
This commit updates the uprobe syscall feature detection in libbpf and the corresponding selftest to expect EPROTO instead of ENXIO.
Quoted 4 lines
tools/lib/bpf/features.c +0
-0
[ ... ]
tools/lib/bpf/features.c static int probe_ldimm64_full_range_off(int token_fd) · line 583
Quoted 10 lines
tools/lib/bpf/features.c +2
-2@@ -577,10 +577,10 @@ static int probe_ldimm64_full_range_off(int token_fd)static int probe_uprobe_syscall (int token_fd) {/* -* If kernel supports uprobe() syscall, it will return -ENXIO when called +* If kernel supports uprobe() syscall, it will return -EPROTO when called * from the outside of a kernel-generated uprobe trampoline. */ -return syscall (__NR_uprobe) <0 && errno ==ENXIO ; +return syscall (__NR_uprobe) <0 && errno ==EPROTO ;
[Severity: Medium]
Does modifying this check to strictly require EPROTO break backward compatibility for libbpf? Older kernels that support the uprobe syscall but return ENXIO will no longer be detected, which might needlessly prevent the use of optimized uprobes on those versions.
Could this check accept both ENXIO and EPROTO?
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +0
-0
[ ... ]
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c static void test_uprobe_error(void) · line 765
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +1
-1@@ -762,7 +762,7 @@ static void test_uprobe_error(void)long err =syscall (__NR_uprobe);ASSERT_EQ (err,-1 ,"error" ); -ASSERT_EQ (errno,ENXIO ,"errno" ); +ASSERT_EQ (errno,EPROTO ,"errno" );
[Severity: Medium]
Will this test fail against the current kernel tree? If the kernel patch changing the return value from ENXIO to EPROTO hasn't been merged yet, the syscall might still return -ENXIO, causing this assertion to fail when running the BPF selftests.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Jiri Olsa
reviewerPATCH v5 8/13 selftests/bpf: Emit nop,nop10 instructions combo for x86_64 arch
In reply to Jiri OlsaSyncing latest usdt.h change [1].
Now that we have nop10 optimization support in kernel, let's emit nop,nop10 for usdt probe. We leave it up to the library to use desirable nop instruction.
[1] TBD
--- tools/testing/selftests/bpf/usdt.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/usdt.h +1 -1
@@ -313,7 +313,7 @@ struct usdt_sema { volatile unsigned short active; };#if defined(__ia64__) || defined(__s390__) || defined(__s390x__)#define USDT_NOP nop 0#elif defined(__x86_64__) -#define USDT_NOP .byte 0x90, 0x0f, 0x1f, 0x44, 0x00, 0x0/* nop, nop5 */ +#define USDT_NOP .byte 0x90, 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00/* nop, nop10 */ #else #define USDT_NOP nop#endif
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCHv5 08/13] selftests/bpf: Emit nop,nop10 instructions combo for x86_64 arch
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The change to default `USDT_NOP` breaks `prog_tests/usdt.c` and fails to achieve the intended…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The change to default USDT_NOP breaks prog_tests/usdt.c and fails to achieve the intended optimization because libbpf is not updated. --
commit 94041e33868287f19cab80050a0ae7ab5dca238b
Author: Jiri Olsa <[email protected]>
selftests/bpf: Emit nop,nop10 instructions combo for x86_64 arch
This commit synchronizes the latest usdt.h changes. It updates the x86_64 USDT_NOP macro to emit a nop,nop10 instruction combination to utilize the new nop10 optimization support in the kernel.
tools/testing/selftests/bpf/usdt.h struct usdt_sema { volatile unsigned short active; }; · line 316
Quoted 10 lines
tools/testing/selftests/bpf/usdt.h +1
-1@@ -313,7 +313,7 @@ struct usdt_sema { volatile unsigned short active; };#if defined(__ia64__) || defined(__s390__) || defined(__s390x__)#define USDT_NOP nop 0#elif defined(__x86_64__) -#define USDT_NOP .byte 0x90, 0x0f, 0x1f, 0x44, 0x00, 0x0/* nop, nop5 */ +#define USDT_NOP .byte 0x90, 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00/* nop, nop10 */
[Severity: High]
Will this change cause the usdt selftest to fail?
Looking at tools/testing/selftests/bpf/prog_tests/usdt.c, subtest_optimized_attach() hardcodes a check for the 6-byte combo:
tools/testing/selftests/bpf/prog_tests/usdt.c:subtest_optimized_attach() {
...
/* usdt_2 USDT probe has nop,nop5 instructions combo */
addr_2 = find_instr(usdt_2, nop1_nop5_combo, 6);
if (!ASSERT_OK_PTR(addr_2, "usdt_2_find_nop1_nop5_combo"))
return;
...
}Since USDT_NOP now emits an 11-byte combo, find_instr() will return NULL and fail the assertion.
Additionally, will this break the intended uprobe optimization in libbpf?
In tools/lib/bpf/usdt.c, has_nop_combo() only checks for the 6-byte combination:
tools/lib/bpf/usdt.c:has_nop_combo() {
unsigned char nop_combo[6] = {
0x90, 0x0f, 0x1f, 0x44, 0x00, 0x00 /* nop,nop5 */
};
...
if (pread(fd, buf, 6, off) != 6)
return false;
return memcmp(buf, nop_combo, 6) == 0;
}When libbpf encounters the new nop10 bytes, it will return false, skip the optimization, and attach the uprobe at the first nop byte. Should libbpf and the selftest be updated to recognize the new 11-byte nop10 sequence?
Quoted 3 lines
tools/testing/selftests/bpf/usdt.h +0
-0#else #define USDT_NOP nop#endif
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Jiri Olsa
reviewerPATCH v5 9/13 selftests/bpf: Change uprobe syscall tests to use nop10
In reply to Jiri OlsaOptimized uprobes are now on top of 10-bytes nop instructions, reflect that in existing tests.
--- .../selftests/bpf/benchs/bench_trigger.c | 2 +- .../selftests/bpf/prog_tests/uprobe_syscall.c | 30 +++++++++++-------- tools/testing/selftests/bpf/prog_tests/usdt.c | 25 +++++++++------- tools/testing/selftests/bpf/usdt_2.c | 2 +- 4 files changed, 34 insertions(+), 25 deletions(-)
tools/testing/selftests/bpf/benchs/bench_trigger.c +1 -1
@@ -398,7 +398,7 @@ static void *uprobe_producer_ret(void *input)#ifdef __x86_64__ __nocf_check __weak void uprobe_target_nop5(void) { - asmvolatile (".byte 0x0f, 0x1f, 0x44, 0x00, 0x00" ); + asmvolatile (".byte 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00" ); }static void *uprobe_producer_nop5 (void *input)
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +17 -13
@@ -17,7 +17,7 @@#include "uprobe_syscall_executed.skel.h" #include "bpf/libbpf_internal.h" -#define USDT_NOP .byte 0x0f, 0x1f, 0x44, 0x00, 0x00 +#define USDT_NOP .byte 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00#include "usdt.h" #pragma GCC diagnostic ignored "-Wattributes"
@@ -26,7 +26,7 @@ __attribute__((aligned(16)))__nocf_check __weak __naked unsigned long uprobe_regs_trigger (void ) { asmvolatile ( -".byte 0x0f, 0x1f, 0x44, 0x00, 0x00\n" /* nop5 */ +".byte 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00\n" /* nop10 */ "movq $0xdeadbeef, %rax\n" "ret\n" );
@@ -345,9 +345,9 @@ static void test_uretprobe_syscall_call(void) __attribute__((aligned (16 ))) __nocf_check __weak __naked void uprobe_test(void) { - asmvolatile (" \n" -".byte 0x0f, 0x1f, 0x44, 0x00, 0x00 \n" -"ret \n" + asmvolatile ( +".byte 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00\n" /* nop10 */ +"ret\n" ); }
@@ -388,14 +388,15 @@ static int find_uprobes_trampoline(void *tramp_addr)return ret; } -static unsigned char nop5[5 ] = {0x0f ,0x1f ,0x44 ,0x00 ,0x00 }; +static unsigned char nop10[10 ] = {0x66 ,0x2e ,0x0f ,0x1f ,0x84 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 }; +static unsigned char lea_rsp[5 ] = {0x48 ,0x8d ,0x64 ,0x24 ,0x80 }; -static void *find_nop5 (void *fn) +static void *find_nop10 (void *fn) {int i; -for (i =0 ; i <10 ; i++) { -if (!memcmp (nop5, fn + i,5 )) +for (i =0 ; i <128 ; i++) { +if (!memcmp (nop10, fn + i,10 ))return fn + i; }return NULL ;
@@ -420,7 +421,8 @@ static void *check_attach(struct uprobe_syscall_executed *skel, trigger_t triggeASSERT_EQ (skel->bss->executed, executed,"executed" );/* .. and check the trampoline is as expected. */ - call = (struct __arch_relative_insn *) addr; +ASSERT_OK (memcmp (addr, lea_rsp,5 ),"lea_rsp" ); + call = (struct __arch_relative_insn *)(addr +5 ); tramp = (void *) (call +1 ) + call->raddr;ASSERT_EQ (call->op,0xe8 ,"call" );ASSERT_OK (find_uprobes_trampoline (tramp),"uprobes_trampoline" );
@@ -430,9 +432,11 @@ static void *check_attach(struct uprobe_syscall_executed *skel, trigger_t triggestatic void check_detach (void *addr,void *tramp) { +static const unsigned char nop10_prefix[] = {0x66 ,0x2e ,0x0f ,0x1f ,0x84 }; +/* [uprobes_trampoline] stays after detach */ ASSERT_OK (find_uprobes_trampoline (tramp),"uprobes_trampoline" ); -ASSERT_OK (memcmp (addr, nop5,5 ),"nop5" ); +ASSERT_OK (memcmp (addr, nop10_prefix,5 ),"nop10_prefix" ); } static void check(struct uprobe_syscall_executed *skel, struct bpf_link *link,
@@ -568,8 +572,8 @@ static void test_uprobe_usdt(void)void *addr; errno =0 ; - addr =find_nop5 (usdt_test); -if (!ASSERT_OK_PTR (addr,"find_nop5" )) + addr =find_nop10 (usdt_test); +if (!ASSERT_OK_PTR (addr,"find_nop10" ))return ; skel =uprobe_syscall_executed__open_and_load ();
tools/testing/selftests/bpf/prog_tests/usdt.c +15 -10
@@ -252,7 +252,7 @@ extern void usdt_1(void);extern void usdt_2 (void );static unsigned char nop1[1 ] = {0x90 }; -static unsigned char nop1_nop5_combo[6 ] = {0x90 ,0x0f ,0x1f ,0x44 ,0x00 ,0x00 }; +static unsigned char nop1_nop10_combo[11 ] = {0x90 ,0x66 ,0x2e ,0x0f ,0x1f ,0x84 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 };static void *find_instr (void *fn,unsigned char *instr,size_t cnt) {
@@ -271,17 +271,17 @@ static void subtest_optimized_attach(void)__u8 *addr_1, *addr_2;/* usdt_1 USDT probe has single nop instruction */ - addr_1 =find_instr (usdt_1, nop1_nop5_combo,6 ); -if (!ASSERT_NULL (addr_1,"usdt_1_find_nop1_nop5_combo" )) + addr_1 =find_instr (usdt_1, nop1_nop10_combo,11 ); +if (!ASSERT_NULL (addr_1,"usdt_1_find_nop1_nop10_combo" ))return ; addr_1 =find_instr (usdt_1, nop1,1 );if (!ASSERT_OK_PTR (addr_1,"usdt_1_find_nop1" ))return ; -/* usdt_2 USDT probe has nop,nop5 instructions combo */ - addr_2 =find_instr (usdt_2, nop1_nop5_combo,6 ); -if (!ASSERT_OK_PTR (addr_2,"usdt_2_find_nop1_nop5_combo" )) +/* usdt_2 USDT probe has nop,nop10 instructions combo */ + addr_2 =find_instr (usdt_2, nop1_nop10_combo,11 ); +if (!ASSERT_OK_PTR (addr_2,"usdt_2_find_nop1_nop10_combo" ))return ; skel =test_usdt__open_and_load ();
@@ -309,12 +309,12 @@ static void subtest_optimized_attach(void)bpf_link__destroy (skel->links.usdt_executed); -/* we expect the nop5 ip */ +/* we expect the nop10 ip */ skel->bss->expected_ip = (unsigned long ) addr_2 +1 ;/* * Attach program on top of usdt_2 which is probe defined on top -* of nop1,nop5 combo, so the probe gets optimized on top of nop5. +* of nop1,nop10 combo, so the probe gets optimized on top of nop10. */ skel->links.usdt_executed = bpf_program__attach_usdt(skel->progs.usdt_executed, 0 /*self*/, "/proc/self/exe",
@@ -328,8 +328,13 @@ static void subtest_optimized_attach(void)/* nop stays on addr_2 address */ ASSERT_EQ (*addr_2,0x90 ,"nop" ); -/* call is on addr_2 + 1 address */ -ASSERT_EQ (*(addr_2 +1 ),0xe8 ,"call" ); +/* +* lea -0x80(%rsp), %rsp +* call ... +*/ +static unsigned char expected[] = {0x48 ,0x8d ,0x64 ,0x24 ,0x80 ,0xe8 }; + +ASSERT_MEMEQ (addr_2 +1 , expected,sizeof (expected),"lea_and_call" );ASSERT_EQ (skel->bss->executed,4 ,"executed" ); cleanup:
tools/testing/selftests/bpf/usdt_2.c +1 -1
@@ -3,7 +3,7 @@#if defined(__x86_64__)/* -* Include usdt.h with default nop,nop5 instructions combo. +* Include usdt.h with default nop,nop10 instructions combo. */ #include "usdt.h"
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCHv5 09/13] selftests/bpf: Change uprobe syscall tests to use nop10
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] The patch updates test expectations to look for an 11-byte NOP sequence but fails to update the…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] The patch updates test expectations to look for an 11-byte NOP sequence but fails to update the default USDT_NOP in usdt.h, causing a guaranteed test failure.
- [Medium] check_detach() only verifies the first 5 bytes of the restored 10-byte NOP instruction, potentially masking an instruction stream corruption bug in the kernel. --
commit 6e149e271696edf6debf9408dedc04512556082d
Author: Jiri Olsa <[email protected]>
Subject: selftests/bpf: Change uprobe syscall tests to use nop10
This patch updates the uprobe syscall tests to reflect that optimized uprobes now operate on top of 10-byte nop instructions.
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +0
-0
[ ... ]
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c static void *check_attach(struct uprobe_syscall_executed *skel, trigger_t trigge · line 439
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +3
-1@@ -430,9 +432,11 @@ static void *check_attach(struct uprobe_syscall_executed *skel, trigger_t triggestatic void check_detach (void *addr,void *tramp) { +static const unsigned char nop10_prefix[] = {0x66 ,0x2e ,0x0f ,0x1f ,0x84 }; +/* [uprobes_trampoline] stays after detach */ ASSERT_OK (find_uprobes_trampoline (tramp),"uprobes_trampoline" ); -ASSERT_OK (memcmp (addr, nop5,5 ),"nop5" ); +ASSERT_OK (memcmp (addr, nop10_prefix,5 ),"nop10_prefix" );
[Severity: Medium]
Does checking only the first 5 bytes mask potential kernel-side unpatching bugs?
The optimized uprobe logic patches a 10-byte sequence, but check_detach() only verifies the first 5 bytes of the restored instruction. If the kernel's unpatching logic fails to restore the remaining 5 bytes, they will still contain part of the call instruction, leaving the execution stream corrupted.
Could we verify the full 10 bytes to ensure the instruction is correctly and completely restored?
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/usdt.c +0
-0
[ ... ]
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/usdt.c +2
-2@@ -271,17 +271,17 @@ static void subtest_optimized_attach(void)__u8 *addr_1, *addr_2;/* usdt_1 USDT probe has single nop instruction */ - addr_1 =find_instr (usdt_1, nop1_nop5_combo,6 ); -if (!ASSERT_NULL (addr_1,"usdt_1_find_nop1_nop5_combo" )) + addr_1 =find_instr (usdt_1, nop1_nop10_combo,11 ); +if (!ASSERT_NULL (addr_1,"usdt_1_find_nop1_nop10_combo" ))return ;
tools/testing/selftests/bpf/prog_tests/usdt.c static void subtest_optimized_attach(void) · line 284
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/usdt.c +3
-3addr_1 =find_instr (usdt_1, nop1,1 );if (!ASSERT_OK_PTR (addr_1,"usdt_1_find_nop1" ))return ; -/* usdt_2 USDT probe has nop,nop5 instructions combo */ - addr_2 =find_instr (usdt_2, nop1_nop5_combo,6 ); -if (!ASSERT_OK_PTR (addr_2,"usdt_2_find_nop1_nop5_combo" )) +/* usdt_2 USDT probe has nop,nop10 instructions combo */ + addr_2 =find_instr (usdt_2, nop1_nop10_combo,11 ); +if (!ASSERT_OK_PTR (addr_2,"usdt_2_find_nop1_nop10_combo" ))
[Severity: Medium]
Will this test fail because usdt.h wasn't updated?
It appears the default USDT_NOP in tools/testing/selftests/bpf/usdt.h for x86_64 is still defined as a 6-byte instruction:
#elif defined(__x86_64__)
#define USDT_NOP .byte 0x90, 0x0f, 0x1f, 0x44, 0x00, 0x0 /* nop, nop5 */
Since usdt_2.c relies on this default macro, won't it still compile with the old 6-byte sequence, causing this find_instr() check for 11 bytes to return NULL?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Jiri Olsa
reviewerPATCH v5 10/13 selftests/bpf: Change uprobe/usdt trigger bench code to use nop10
In reply to Jiri OlsaChanging uprobe/usdt trigger bench code to use nop10 instead of nop5. Also changing run_bench_uprobes.sh to use nop10 triggers.
--- tools/testing/selftests/bpf/bench.c | 20 +++++------ .../selftests/bpf/benchs/bench_trigger.c | 36 +++++++++---------- .../selftests/bpf/benchs/run_bench_uprobes.sh | 2 +- 3 files changed, 29 insertions(+), 29 deletions(-)
tools/testing/selftests/bpf/bench.c +10 -10
@@ -539,12 +539,12 @@ extern const struct bench bench_trig_uretprobe_multi_push;extern const struct bench bench_trig_uprobe_multi_ret;extern const struct bench bench_trig_uretprobe_multi_ret;#ifdef __x86_64__ -extern const struct bench bench_trig_uprobe_nop5; -extern const struct bench bench_trig_uretprobe_nop5; -extern const struct bench bench_trig_uprobe_multi_nop5; -extern const struct bench bench_trig_uretprobe_multi_nop5; +extern const struct bench bench_trig_uprobe_nop10; +extern const struct bench bench_trig_uretprobe_nop10; +extern const struct bench bench_trig_uprobe_multi_nop10; +extern const struct bench bench_trig_uretprobe_multi_nop10;extern const struct bench bench_trig_usdt_nop; -extern const struct bench bench_trig_usdt_nop5; +extern const struct bench bench_trig_usdt_nop10;#endif extern const struct bench bench_rb_libbpf;
@@ -622,12 +622,12 @@ static const struct bench *benchs[] = { &bench_trig_uprobe_multi_ret, &bench_trig_uretprobe_multi_ret, #ifdef __x86_64__ - &bench_trig_uprobe_nop5, - &bench_trig_uretprobe_nop5, - &bench_trig_uprobe_multi_nop5, - &bench_trig_uretprobe_multi_nop5, + &bench_trig_uprobe_nop10, + &bench_trig_uretprobe_nop10, + &bench_trig_uprobe_multi_nop10, + &bench_trig_uretprobe_multi_nop10, &bench_trig_usdt_nop, - &bench_trig_usdt_nop5, + &bench_trig_usdt_nop10, #endif/* ringbuf/perfbuf benchmarks */ &bench_rb_libbpf,
tools/testing/selftests/bpf/benchs/bench_trigger.c +18 -18
@@ -396,15 +396,15 @@ static void *uprobe_producer_ret(void *input) }#ifdef __x86_64__ -__nocf_check __weak void uprobe_target_nop5(void) +__nocf_check __weak void uprobe_target_nop10(void) { asmvolatile (".byte 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00" ); } -static void *uprobe_producer_nop5 (void *input) +static void *uprobe_producer_nop10 (void *input) {while (true) -uprobe_target_nop5 (); +uprobe_target_nop10 ();return NULL ; }
@@ -418,7 +418,7 @@ static void *uprobe_producer_usdt_nop(void *input)return NULL ; } -static void *uprobe_producer_usdt_nop5 (void *input) +static void *uprobe_producer_usdt_nop10 (void *input) {while (true)usdt_2 ();
@@ -542,24 +542,24 @@ static void uretprobe_multi_ret_setup(void) }#ifdef __x86_64__ -static void uprobe_nop5_setup (void ) +static void uprobe_nop10_setup (void ) { -usetup (false, false/* !use_multi */ , &uprobe_target_nop5); +usetup (false, false/* !use_multi */ , &uprobe_target_nop10); } -static void uretprobe_nop5_setup (void ) +static void uretprobe_nop10_setup (void ) { -usetup (true, false/* !use_multi */ , &uprobe_target_nop5); +usetup (true, false/* !use_multi */ , &uprobe_target_nop10); } -static void uprobe_multi_nop5_setup (void ) +static void uprobe_multi_nop10_setup (void ) { -usetup (false, true/* use_multi */ , &uprobe_target_nop5); +usetup (false, true/* use_multi */ , &uprobe_target_nop10); } -static void uretprobe_multi_nop5_setup (void ) +static void uretprobe_multi_nop10_setup (void ) { -usetup (true, true/* use_multi */ , &uprobe_target_nop5); +usetup (true, true/* use_multi */ , &uprobe_target_nop10); }static void usdt_setup (const char *name)
@@ -598,7 +598,7 @@ static void usdt_nop_setup(void)usdt_setup ("usdt_1" ); } -static void usdt_nop5_setup (void ) +static void usdt_nop10_setup (void ) {usdt_setup ("usdt_2" ); }
@@ -665,10 +665,10 @@ BENCH_TRIG_USERMODE(uretprobe_multi_nop, nop, "uretprobe-multi-nop");BENCH_TRIG_USERMODE (uretprobe_multi_push, push,"uretprobe-multi-push" );BENCH_TRIG_USERMODE (uretprobe_multi_ret, ret,"uretprobe-multi-ret" );#ifdef __x86_64__ -BENCH_TRIG_USERMODE (uprobe_nop5, nop5,"uprobe-nop5" ); -BENCH_TRIG_USERMODE (uretprobe_nop5, nop5,"uretprobe-nop5" ); -BENCH_TRIG_USERMODE (uprobe_multi_nop5, nop5,"uprobe-multi-nop5" ); -BENCH_TRIG_USERMODE (uretprobe_multi_nop5, nop5,"uretprobe-multi-nop5" ); +BENCH_TRIG_USERMODE (uprobe_nop10, nop10,"uprobe-nop10" ); +BENCH_TRIG_USERMODE (uretprobe_nop10, nop10,"uretprobe-nop10" ); +BENCH_TRIG_USERMODE (uprobe_multi_nop10, nop10,"uprobe-multi-nop10" ); +BENCH_TRIG_USERMODE (uretprobe_multi_nop10, nop10,"uretprobe-multi-nop10" );BENCH_TRIG_USERMODE (usdt_nop, usdt_nop,"usdt-nop" ); -BENCH_TRIG_USERMODE (usdt_nop5, usdt_nop5,"usdt-nop5" ); +BENCH_TRIG_USERMODE (usdt_nop10, usdt_nop10,"usdt-nop10" );#endif
tools/testing/selftests/bpf/benchs/run_bench_uprobes.sh +1 -1
@@ -2,7 +2,7 @@ set -eufo pipefail -for i in usermode-count syscall-count {uprobe,uretprobe}-{nop,push,ret,nop5} usdt-nop usdt-nop5 +for i in usermode-count syscall-count {uprobe,uretprobe}-{nop,push,ret,nop10} usdt-nop usdt-nop10 do summary=$(sudo ./bench -w2 -d5 -a trig-$i | tail -n1 | cut -d'(' -f1 | cut -d' ' -f3-) printf "%-15s: %s\n" $i "$summary"
Signature · 1 line
2.54.0
Jiri Olsa
reviewerPATCH v5 11/13 selftests/bpf: Add reattach tests for uprobe syscall
In reply to Jiri OlsaAdding reattach tests for uprobe syscall tests to make sure we can re-attach and optimize same uprobe multiple times.
--- .../selftests/bpf/prog_tests/uprobe_syscall.c | 130 ++++++++++++++++-- 1 file changed, 120 insertions(+), 10 deletions(-)
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +120 -10
@@ -404,6 +404,16 @@ static void *find_nop10(void *fn) typedef void (__attribute__((nocf_check)) *trigger_t)(void); +static void check_attach_notrigger (struct uprobe_syscall_executed *skel, +void *addr,int executed) +{ +unsigned char *op = addr; + +/* Make sure bpf program was not executed. */ +ASSERT_EQ (skel->bss->executed, executed,"executed" ); +ASSERT_EQ (*op,0xcc ,"int3" ); +} +static void *check_attach (struct uprobe_syscall_executed *skel,trigger_t trigger,void *addr,int executed) {
@@ -430,23 +440,26 @@ static void *check_attach(struct uprobe_syscall_executed *skel, trigger_t triggereturn tramp; } -static void check_detach (void *addr,void *tramp) +static bool check_detach (void *addr,void *tramp) {static const unsigned char nop10_prefix[] = {0x66 ,0x2e ,0x0f ,0x1f ,0x84 }; +bool ok = true;/* [uprobes_trampoline] stays after detach */ -ASSERT_OK (find_uprobes_trampoline (tramp),"uprobes_trampoline" ); -ASSERT_OK (memcmp (addr, nop10_prefix,5 ),"nop10_prefix" ); + ok &=ASSERT_OK (find_uprobes_trampoline (tramp),"uprobes_trampoline" ); + ok &=ASSERT_OK (memcmp (addr, nop10_prefix,5 ),"nop10_prefix" ); +return ok; } -static void check (struct uprobe_syscall_executed *skel,struct bpf_link *link, -trigger_t trigger,void *addr,int executed) +static void *check (struct uprobe_syscall_executed *skel,struct bpf_link *link, +trigger_t trigger,void *addr,int executed) {void *tramp; tramp =check_attach (skel, trigger, addr, executed);bpf_link__destroy (link);check_detach (addr, tramp); +return tramp; }static void test_uprobe_legacy (void )
@@ -457,6 +470,7 @@ static void test_uprobe_legacy(void) );struct bpf_link *link;unsigned long offset; +void *tramp; offset =get_uprobe_offset (&uprobe_test);if (!ASSERT_GE (offset,0 ,"get_uprobe_offset" ))
@@ -474,7 +488,30 @@ static void test_uprobe_legacy(void)if (!ASSERT_OK_PTR (link,"bpf_program__attach_uprobe_opts" )) goto cleanup; -check (skel, link, uprobe_test, uprobe_test,2 ); + tramp =check (skel, link, uprobe_test, uprobe_test,2 ); + +/* reattach and detach without triggering optimization */ + link =bpf_program__attach_uprobe_opts (skel->progs.test_uprobe, +0 ,"/proc/self/exe" , offset,NULL ); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_uprobe_opts" )) + goto cleanup; + +check_attach_notrigger (skel, uprobe_test,2 ); + +bpf_link__destroy (link); +if (!check_detach (uprobe_test, tramp)) + goto cleanup; + +uprobe_test (); +ASSERT_EQ (skel->bss->executed,2 ,"executed_no_probe" ); + +/* reattach with triggering optimization */ + link =bpf_program__attach_uprobe_opts (skel->progs.test_uprobe, +0 ,"/proc/self/exe" , offset,NULL ); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_uprobe_opts" )) + goto cleanup; + +check (skel, link, uprobe_test, uprobe_test,4 );/* uretprobe */ skel->bss->executed =0 ;
@@ -496,6 +533,7 @@ static void test_uprobe_multi(void)LIBBPF_OPTS (bpf_uprobe_multi_opts, opts);struct bpf_link *link;unsigned long offset; +void *tramp; offset =get_uprobe_offset (&uprobe_test);if (!ASSERT_GE (offset,0 ,"get_uprobe_offset" ))
@@ -516,7 +554,30 @@ static void test_uprobe_multi(void)if (!ASSERT_OK_PTR (link,"bpf_program__attach_uprobe_multi" )) goto cleanup; -check (skel, link, uprobe_test, uprobe_test,2 ); + tramp =check (skel, link, uprobe_test, uprobe_test,2 ); + +/* reattach and detach without triggering optimization */ + link =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe_multi, +0 ,"/proc/self/exe" ,NULL , &opts); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + +check_attach_notrigger (skel, uprobe_test,2 ); + +bpf_link__destroy (link); +if (!check_detach (uprobe_test, tramp)) + goto cleanup; + +uprobe_test (); +ASSERT_EQ (skel->bss->executed,2 ,"executed_no_probe" ); + +/* reattach with triggering optimization */ + link =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe_multi, +0 ,"/proc/self/exe" ,NULL , &opts); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + +check (skel, link, uprobe_test, uprobe_test,4 );/* uretprobe.multi */ skel->bss->executed =0 ;
@@ -540,6 +601,7 @@ static void test_uprobe_session(void) );struct bpf_link *link;unsigned long offset; +void *tramp; offset =get_uprobe_offset (&uprobe_test);if (!ASSERT_GE (offset,0 ,"get_uprobe_offset" ))
@@ -559,7 +621,30 @@ static void test_uprobe_session(void)if (!ASSERT_OK_PTR (link,"bpf_program__attach_uprobe_multi" )) goto cleanup; -check (skel, link, uprobe_test, uprobe_test,4 ); + tramp =check (skel, link, uprobe_test, uprobe_test,4 ); + +/* reattach and detach without triggering optimization */ + link =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe_session, +0 ,"/proc/self/exe" ,NULL , &opts); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + +check_attach_notrigger (skel, uprobe_test,4 ); + +bpf_link__destroy (link); +if (!check_detach (uprobe_test, tramp)) + goto cleanup; + +uprobe_test (); +ASSERT_EQ (skel->bss->executed,4 ,"executed_no_probe" ); + +/* reattach with triggering optimization */ + link =bpf_program__attach_uprobe_multi (skel->progs.test_uprobe_session, +0 ,"/proc/self/exe" ,NULL , &opts); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_uprobe_multi" )) + goto cleanup; + +check (skel, link, uprobe_test, uprobe_test,8 ); cleanup:uprobe_syscall_executed__destroy (skel);
@@ -569,7 +654,7 @@ static void test_uprobe_usdt(void) {struct uprobe_syscall_executed *skel;struct bpf_link *link; -void *addr; +void *addr, *tramp; errno =0 ; addr =find_nop10 (usdt_test);
@@ -588,7 +673,32 @@ static void test_uprobe_usdt(void)if (!ASSERT_OK_PTR (link,"bpf_program__attach_usdt" )) goto cleanup; -check (skel, link, usdt_test, addr,2 ); + tramp =check (skel, link, usdt_test, addr,2 ); + +/* reattach and detach without triggering optimization */ + link =bpf_program__attach_usdt (skel->progs.test_usdt, +-1 /* all PIDs */ ,"/proc/self/exe" , +"optimized_uprobe" ,"usdt" ,NULL ); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_usdt" )) + goto cleanup; + +check_attach_notrigger (skel, addr,2 ); + +bpf_link__destroy (link); +if (!check_detach (addr, tramp)) + goto cleanup; + +usdt_test (); +ASSERT_EQ (skel->bss->executed,2 ,"executed_no_probe" ); + +/* reattach with triggering optimization */ + link =bpf_program__attach_usdt (skel->progs.test_usdt, +-1 /* all PIDs */ ,"/proc/self/exe" , +"optimized_uprobe" ,"usdt" ,NULL ); +if (!ASSERT_OK_PTR (link,"bpf_program__attach_usdt" )) + goto cleanup; + +check (skel, link, usdt_test, addr,4 ); cleanup:uprobe_syscall_executed__destroy (skel);
Signature · 1 line
2.54.0
Jiri Olsa
reviewerPATCH v5 12/13 selftests/bpf: Add tests for uprobe nop10 red zone clobbering
In reply to Jiri OlsaFrom: Andrii Nakryiko <[email protected]>
The uprobe nop5 optimization used to replace a 5-byte NOP with a 5-byte CALL to a trampoline. The CALL pushes a return address onto the stack at [rsp-8], clobbering whatever was stored there.
On x86-64, the red zone is the 128 bytes below rsp that user code may use for temporary storage without adjusting rsp. Compilers can place USDT argument operands there, generating specs like "8@-8(%rbp)" when rbp == rsp. With the CALL-based optimization, the return address overwrites that argument before the BPF-side USDT argument fetch runs.
Add two tests for this case. The uprobe_syscall subtest stores known values at -8(%rsp), -16(%rsp), and -24(%rsp), executes an optimized nop10 uprobe, and verifies the red-zone data is still intact. The USDT subtest triggers a probe in a function where the compiler places three USDT operands in the red zone and verifies that all 10 optimized invocations deliver the expected argument values to BPF.
On an unfixed kernel, the first hit goes through the INT3 path and later hits use the optimized CALL path, so the red-zone checks fail after optimization.
--- .../selftests/bpf/prog_tests/uprobe_syscall.c | 77 +++++++++++++++++++ tools/testing/selftests/bpf/prog_tests/usdt.c | 49 ++++++++++++ tools/testing/selftests/bpf/progs/test_usdt.c | 25 ++++++ tools/testing/selftests/bpf/usdt_2.c | 13 ++++ 4 files changed, 164 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +77 -0
@@ -357,6 +357,50 @@ __nocf_check __weak void usdt_test(void)USDT (optimized_uprobe, usdt); } +/* +* Assembly-level red zone clobbering test. Stores known values in the +* red zone (below RSP), executes a nop10 (uprobe site), and checks that +* the values survived. Returns 0 if intact, 1 if clobbered. +* +* The nop5 optimization used CALL (which pushes a return address to +* [rsp-8]), the value at -8(%rsp) was overwritten. The nop10 optimization +* should escape that by moving stackpointer below the redzone before +* doing the CALL. +* +* Align the code at 64 bytes, to make sure nop10 is not on page boundary. +*/ +__attribute__((aligned (64 ))) +__nocf_check __weak __naked unsigned long uprobe_red_zone_test (void ) +{ + asmvolatile ( +"movabs $0x1111111111111111, %%rax\n" +"movq %%rax, -8(%%rsp)\n" +"movabs $0x2222222222222222, %%rax\n" +"movq %%rax, -16(%%rsp)\n" +"movabs $0x3333333333333333, %%rax\n" +"movq %%rax, -24(%%rsp)\n" + +".byte 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00\n" /* nop10: uprobe site */ + +"movabs $0x1111111111111111, %%rax\n" +"cmpq %%rax, -8(%%rsp)\n" +"jne 1f\n" +"movabs $0x2222222222222222, %%rax\n" +"cmpq %%rax, -16(%%rsp)\n" +"jne 1f\n" +"movabs $0x3333333333333333, %%rax\n" +"cmpq %%rax, -24(%%rsp)\n" +"jne 1f\n" + +"xorl %%eax, %%eax\n" +"retq\n" +"1:\n" +"movl $1, %%eax\n" +"retq\n" + :::"rax" ,"memory" + ); +} +static int find_uprobes_trampoline (void *tramp_addr) {void *start, *end;
@@ -871,6 +915,37 @@ static void test_uprobe_race(void)#define __NR_uprobe 336#endif +static void test_uprobe_red_zone (void ) +{ +struct uprobe_syscall_executed *skel; +struct bpf_link *link; +void *nop10_addr; +size_t offset; +int i; + + nop10_addr =find_nop10 (uprobe_red_zone_test); +if (!ASSERT_NEQ (nop10_addr,NULL ,"find_nop10" )) +return ; + + skel =uprobe_syscall_executed__open_and_load (); +if (!ASSERT_OK_PTR (skel,"open_and_load" )) +return ; + + offset =get_uprobe_offset (nop10_addr); + link =bpf_program__attach_uprobe_opts (skel->progs.test_uprobe, +0 ,"/proc/self/exe" , offset,NULL ); +if (!ASSERT_OK_PTR (link,"attach_uprobe" )) + goto cleanup; + +for (i =0 ; i <10 ; i++) +ASSERT_EQ (uprobe_red_zone_test (),0 ,"red_zone_intact" ); + +bpf_link__destroy (link); + +cleanup: +uprobe_syscall_executed__destroy (skel); +} +static void test_uprobe_error (void ) {long err =syscall (__NR_uprobe);
@@ -897,6 +972,8 @@ static void __test_uprobe_syscall(void)test_uprobe_usdt ();if (test__start_subtest ("uprobe_race" ))test_uprobe_race (); +if (test__start_subtest ("uprobe_red_zone" )) +test_uprobe_red_zone ();if (test__start_subtest ("uprobe_error" ))test_uprobe_error ();if (test__start_subtest ("uprobe_regs_equal" ))
tools/testing/selftests/bpf/prog_tests/usdt.c +49 -0
67 lines not on this page · raw message
tools/testing/selftests/bpf/progs/test_usdt.c +25 -0
@@ -149,5 +149,30 @@ int usdt_executed(struct pt_regs *ctx) executed++;return 0 ; } + +int arg_total; +int arg_bad; +long arg_last[3 ]; +long expected_arg[3 ]; +int expected_pid; + +SEC("usdt") +int BPF_USDT (usdt_check_arg ,long arg1,long arg2,long arg3) +{ +if (expected_pid != (bpf_get_current_pid_tgid () >>32 )) +return 0 ; + +__sync_fetch_and_add (&arg_total,1 ); + arg_last[0 ] = arg1; + arg_last[1 ] = arg2; + arg_last[2 ] = arg3; + +if (arg1 != expected_arg[0 ] || + arg2 != expected_arg[1 ] || + arg3 != expected_arg[2 ]) +__sync_fetch_and_add (&arg_bad,1 ); + +return 0 ; +}#endif char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/usdt_2.c +13 -0
@@ -13,4 +13,17 @@ void usdt_2(void) USDT(optimized_attach, usdt_2); } +static volatile unsigned long usdt_red_zone_arg1 =0xDEADBEEF ; +static volatile unsigned long usdt_red_zone_arg2 =0xCAFEBABE ; +static volatile unsigned long usdt_red_zone_arg3 =0xFEEDFACE ; + +void __attribute__((noinline))usdt_red_zone_trigger (void ) +{ +unsigned long a1 = usdt_red_zone_arg1; +unsigned long a2 = usdt_red_zone_arg2; +unsigned long a3 = usdt_red_zone_arg3; + +USDT (optimized_attach, usdt_red_zone, a1, a2, a3); +} +#endif
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCHv5 12/13] selftests/bpf: Add tests for uprobe nop10 red zone clobbering
This isn't a bug, but the four-line "nop5 optimization used CALL ..." explanation here reads nearly verbatim to the comment added above uprobe_red_zone_test() in prog_tests/uprobe_syscall.c.
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/usdt.c +0
-0
[ ... ]
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/usdt.c +4
-0@@ -340,6 +341,52 @@ static void subtest_optimized_attach(void) cleanup:test_usdt__destroy (skel); } + +/* +* Test that USDT arguments survive nop10 optimization in a function where +* the compiler places operands in the red zone.
tools/testing/selftests/bpf/prog_tests/usdt.c static void subtest_optimized_attach(void) · line 357
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/usdt.c +10
-0+* +* Signal handlers are prone to having the compiler place USDT argument +* operands in the red zone (below rsp). +* +* The nop5 optimization used CALL (which pushes a return address to +* [rsp-8]), the value at -8(%rsp) was overwritten. The nop10 optimization +* should escape that by moving stackpointer below the redzone before +* doing the CALL. +*/ +static void subtest_optimized_red_zone (void )
This isn't a bug, but the four-line "nop5 optimization used CALL ..." explanation here reads nearly verbatim to the comment added above uprobe_red_zone_test() in prog_tests/uprobe_syscall.c.
Since both tests probe the same mechanism, would a single copy (or a shorter note pointing at the other) be preferable, so the two don't drift apart?
One small wording note: "moving stackpointer below the redzone" uses "stackpointer" and "redzone", while the rest of the change and the commit message use "stack pointer" and "red zone".
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28514315910
Jiri Olsa
reviewerPATCH v5 13/13 selftests/bpf: Add tests for forked/cloned optimized uprobes
In reply to Jiri OlsaAdding tests for forked/cloned optimized uprobes and make sure the child can properly execute optimized probe for both fork (dups mm) and clone with CLONE_VM.
--- .../selftests/bpf/prog_tests/uprobe_syscall.c | 89 +++++++++++++++++++ 1 file changed, 89 insertions(+)
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +89 -0
@@ -4,6 +4,8 @@#ifdef __x86_64__ +#define _GNU_SOURCE +#include <sched.h> #include <unistd.h> #include <asm/ptrace.h> #include <linux/compiler.h>
@@ -13,6 +15,7 @@#include <sys/syscall.h> #include <sys/prctl.h> #include <asm/prctl.h> +#include <stdnoreturn.h> #include "uprobe_syscall.skel.h" #include "uprobe_syscall_executed.skel.h" #include "bpf/libbpf_internal.h"
@@ -954,6 +957,88 @@ static void test_uprobe_error(void)ASSERT_EQ (errno,EPROTO ,"errno" ); } +__attribute__((aligned (16 ))) +__nocf_check __weak __naked void uprobe_fork_test(void) +{ + asmvolatile ( +".byte 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00\n" /* nop10 */ +"ret\n" + ); +} + +static noreturnint child_func (void *arg) +{ +struct uprobe_syscall_executed *skel = arg; + +/* Make sure the child's probe is still there and optimized.. */ +if (memcmp (uprobe_fork_test, lea_rsp,sizeof (lea_rsp))) +_exit (1 ); + + skel->bss->pid =getpid (); + +/* .. and it executes properly. */ +uprobe_fork_test (); + +if (skel->bss->executed !=3 ) +_exit (2 ); + +_exit (0 ); +} + +static void test_uprobe_fork_optimized (bool clone_vm) +{ +struct uprobe_syscall_executed *skel =NULL ; +struct bpf_link *link =NULL ; +unsigned long offset; +int pid, status, err; +char stack[65535 ]; + + offset =get_uprobe_offset (&uprobe_fork_test); +if (!ASSERT_GE (offset,0 ,"get_uprobe_offset" )) +return ; + + skel =uprobe_syscall_executed__open_and_load (); +if (!ASSERT_OK_PTR (skel,"open_and_load" )) + goto cleanup; + + link =bpf_program__attach_uprobe_opts (skel->progs.test_uprobe, +-1 ,"/proc/self/exe" , offset,NULL ); +if (!ASSERT_OK_PTR (link,"attach_uprobe" )) + goto cleanup; + + skel->bss->pid =getpid (); + +/* Trigger optimization of uprobe in uprobe_fork_test. */ +uprobe_fork_test (); +uprobe_fork_test (); + +/* Make sure it got optimied. */ +if (!ASSERT_OK (memcmp (uprobe_fork_test, lea_rsp,sizeof (lea_rsp)),"optimized" )) + goto cleanup; + +if (clone_vm) { + pid =clone (child_func, stack +sizeof (stack),CLONE_VM |SIGCHLD , skel); +if (!ASSERT_GT (pid,0 ,"clone" )) + goto cleanup; + }else { + pid =fork (); +if (!ASSERT_GE (pid,0 ,"fork" )) + goto cleanup; +if (pid ==0 ) +child_func (skel); + } + +/* Wait for the child and verify it exited properly with 0. */ + err =waitpid (pid, &status,0 ); +if (ASSERT_EQ (err, pid,"waitpid" )) { +ASSERT_EQ (WIFEXITED (status),1 ,"child_exited" ); +ASSERT_EQ (WEXITSTATUS (status),0 ,"child_exit_code" ); + } + +cleanup: +uprobe_syscall_executed__destroy (skel); +} +static void __test_uprobe_syscall (void ) {if (test__start_subtest ("uretprobe_regs_equal" ))
@@ -974,6 +1059,10 @@ static void __test_uprobe_syscall(void)test_uprobe_race ();if (test__start_subtest ("uprobe_red_zone" ))test_uprobe_red_zone (); +if (test__start_subtest ("uprobe_optimized_fork" )) +test_uprobe_fork_optimized (false); +if (test__start_subtest ("uprobe_optimized_clone_vm" )) +test_uprobe_fork_optimized (true);if (test__start_subtest ("uprobe_error" ))test_uprobe_error ();if (test__start_subtest ("uprobe_regs_equal" ))
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCHv5 13/13] selftests/bpf: Add tests for forked/cloned optimized uprobes
This isn't a bug, but should "optimied" in this comment read "optimized", to match the spelling used elsewhere in the file?
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +0
-0
[ ... ]
Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +2
-0@@ -954,6 +957,88 @@ static void test_uprobe_error(void) +static void test_uprobe_fork_optimized (bool clone_vm) +{
[ ... ]
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c static void test_uprobe_error(void) · line 1016
Quoted 2 lines
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +2
-0+/* Make sure it got optimied. */ +if (!ASSERT_OK (memcmp (uprobe_fork_test, lea_rsp,sizeof (lea_rsp)),"optimized" ))
This isn't a bug, but should "optimied" in this comment read "optimized", to match the spelling used elsewhere in the file?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28514315910
Quoted 56 lines · Jiri Olsa
hi,
Andrii reported an issue with optimized uprobes [1] that can clobber redzone area with call instruction storing return address on stack where user code may keep temporary data without adjusting rsp.Fixing this by moving the optimized uprobes on top of 10-bytes nop instruction, so we can squeeze another instruction to escape the redzone area before doing the call.
Note we need upstream update first for patch 3 (github.com/libbpf/usdt), if we decide to take this change.
thanks,
jirkav1: https://lore.kernel.org/bpf/[email protected]/
v2: https://lore.kernel.org/bpf/[email protected]/
v3: https://lore.kernel.org/bpf/[email protected]/
v4: https://lore.kernel.org/bpf/[email protected]/v5 changes:
- several selftests changes and reviewed-by tags [Jakub]
- add more comments inint3_update_unoptimize[Andrii]
- several other minor changes and acks [Oleg]
- moveinsn_decodeout ofuprobe_init_insnto simplify the code
- alignuprobe_red_zone_testto 64 to make sure nop10 is not on page boundaryv4 changes:
- do not use 2nd int3 (ont +5 offset) because the call instruction is allways the same for the given nop10 address [Andrii/Peter]
- unmap unused trampoline vma after unsuccesfull optimization [sashiko]
- small change to patch#2 moveduser_64bit_modeearlier in the path and pass/usemm_structpointer directly fromarch_uprobe_optimizeinstead of getttingcurrent->mm
Andrii, keeping your ack, please shout otherwisev3 changes:
- use nop10 update suggested by Peter in [2]
- remove structuprobe_trampolineobject, use vma objects directly instead
- selftests fixes [sashiko]
- ack from Andriiv2 changes:
- several selftest fixes [sashiko]
- consolidateis_lea_insnandis_call_insninsto single check [Jakub Sitnicki]
- use propermm_structobject in__in_uprobe_trampolinecheck [sashiko]
- allow to copy uprobe trampolines vma objects on fork [sashiko]
- change uprobe syscall detection error from-ENXIOto-EPROTO[Andrii]
- added fork/clone tests
- I kept the selftest changes andnop5->nop10changes in separate commits for easier review, we can squash them later if we want to keep bisect working properly
Quoted 3 lines
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/#t
---
ASAN-enabled test_progs runs are not happy in CI, can you please check?
Quoted 32 lines
Andrii Nakryiko (1):
selftests/bpf: Add tests for uprobe nop10 red zone clobberingJiri Olsa (12):
uprobes/x86: Use propermm_structin__in_uprobe_trampolineuprobes/x86: Remove structuprobe_trampolineobject uprobes/x86: Do not leak trampoline vma mapping on optimization failure uprobes/x86: Allow to copy uprobe trampolines on fork uprobes/x86: Move optimized uprobe from nop5 to nop10
libbpf: Changehas_nop_comboto work on top of nop10
libbpf: Detect uprobe syscall with new error
selftests/bpf: Emit nop,nop10 instructions combo forx86_64arch selftests/bpf: Change uprobe syscall tests to use nop10 selftests/bpf: Change uprobe/usdt trigger bench code to use nop10 selftests/bpf: Add reattach tests for uprobe syscall selftests/bpf: Add tests for forked/cloned optimized uprobesarch/x86/kernel/uprobes.c | 416 +++++++++++++++++++++++++++++++++++++++++++----------------------------- include/linux/uprobes.h | 5 - kernel/events/uprobes.c | 10 -- kernel/fork.c | 1 - tools/lib/bpf/features.c | 4 +- tools/lib/bpf/usdt.c | 16 +-- tools/testing/selftests/bpf/bench.c | 20 ++-- tools/testing/selftests/bpf/benchs/bench_trigger.c | 38 +++---- tools/testing/selftests/bpf/benchs/run_bench_uprobes.sh | 2 +- tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c | 326 ++++++++++++++++++++++++++++++++++++++++++++++++++++---- tools/testing/selftests/bpf/prog_tests/usdt.c | 74 +++++++++++-- tools/testing/selftests/bpf/progs/test_usdt.c | 25 +++++ tools/testing/selftests/bpf/usdt.h | 2 +- tools/testing/selftests/bpf/usdt_2.c | 15 ++- 14 files changed, 698 insertions(+), 256 deletions(-)
Quoted 57 lines · Andrii Nakryiko
Quoted 56 lines · Jiri Olsa
hi,
Andrii reported an issue with optimized uprobes [1] that can clobber redzone area with call instruction storing return address on stack where user code may keep temporary data without adjusting rsp.Fixing this by moving the optimized uprobes on top of 10-bytes nop instruction, so we can squeeze another instruction to escape the redzone area before doing the call.
Note we need upstream update first for patch 3 (github.com/libbpf/usdt), if we decide to take this change.
thanks,
jirkav1: https://lore.kernel.org/bpf/[email protected]/
v2: https://lore.kernel.org/bpf/[email protected]/
v3: https://lore.kernel.org/bpf/[email protected]/
v4: https://lore.kernel.org/bpf/[email protected]/v5 changes:
- several selftests changes and reviewed-by tags [Jakub]
- add more comments inint3_update_unoptimize[Andrii]
- several other minor changes and acks [Oleg]
- moveinsn_decodeout ofuprobe_init_insnto simplify the code
- alignuprobe_red_zone_testto 64 to make sure nop10 is not on page boundaryv4 changes:
- do not use 2nd int3 (ont +5 offset) because the call instruction is allways the same for the given nop10 address [Andrii/Peter]
- unmap unused trampoline vma after unsuccesfull optimization [sashiko]
- small change to patch#2 moveduser_64bit_modeearlier in the path and pass/usemm_structpointer directly fromarch_uprobe_optimizeinstead of getttingcurrent->mm
Andrii, keeping your ack, please shout otherwisev3 changes:
- use nop10 update suggested by Peter in [2]
- remove structuprobe_trampolineobject, use vma objects directly instead
- selftests fixes [sashiko]
- ack from Andriiv2 changes:
- several selftest fixes [sashiko]
- consolidateis_lea_insnandis_call_insninsto single check [Jakub Sitnicki]
- use propermm_structobject in__in_uprobe_trampolinecheck [sashiko]
- allow to copy uprobe trampolines vma objects on fork [sashiko]
- change uprobe syscall detection error from-ENXIOto-EPROTO[Andrii]
- added fork/clone tests
- I kept the selftest changes andnop5->nop10changes in separate commits for easier review, we can squash them later if we want to keep bisect working properly
Quoted 5 lines
Quoted 3 lines
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/#t
---ASAN-enabled
test_progsruns are not happy in CI, can you please check?
I failed to release link in test_uprobe_fork_optimized, fix is below I can send new version or separate fix
also there's 2 things to solve/discuss once kernel changes are acked:
- selftest changes depend on:
selftests/bpf: Emit nop,nop10 instructions combo for x86_64 arch that is taken from libbpf/usdt, I pushed the PR in here [1]
- as bots complained the patchset breaks bisection, because kernel changes break selftests.. not sure what's prefered solution, as for me I'd keep it that way rather than mixing kernel/user space changes
thanks,
jirka
[1] https://github.com/libbpf/usdt/pull/16
---
tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +3 -4
@@ -988,7 +988,6 @@ static noreturn int child_func(void *arg)static void test_uprobe_fork_optimized (bool clone_vm) {struct uprobe_syscall_executed *skel =NULL ; -struct bpf_link *link =NULL ;unsigned long offset;int pid, status, err;char stack[65535 ];
@@ -1001,9 +1000,9 @@ static void test_uprobe_fork_optimized(bool clone_vm)if (!ASSERT_OK_PTR (skel,"open_and_load" )) goto cleanup; - link =bpf_program__attach_uprobe_opts (skel->progs.test_uprobe, --1 ,"/proc/self/exe" , offset,NULL ); -if (!ASSERT_OK_PTR (link,"attach_uprobe" )) + skel->links.test_uprobe =bpf_program__attach_uprobe_opts (skel->progs.test_uprobe, +-1 ,"/proc/self/exe" , offset,NULL ); +if (!ASSERT_OK_PTR (skel->links.test_uprobe,"attach_uprobe" )) goto cleanup; skel->bss->pid =getpid ();
Quoted 59 lines · Jiri Olsa
Quoted 57 lines · Andrii Nakryiko
Quoted 56 lines · Jiri Olsa
hi,
Andrii reported an issue with optimized uprobes [1] that can clobber redzone area with call instruction storing return address on stack where user code may keep temporary data without adjusting rsp.Fixing this by moving the optimized uprobes on top of 10-bytes nop instruction, so we can squeeze another instruction to escape the redzone area before doing the call.
Note we need upstream update first for patch 3 (github.com/libbpf/usdt), if we decide to take this change.
thanks,
jirkav1: https://lore.kernel.org/bpf/[email protected]/
v2: https://lore.kernel.org/bpf/[email protected]/
v3: https://lore.kernel.org/bpf/[email protected]/
v4: https://lore.kernel.org/bpf/[email protected]/v5 changes:
- several selftests changes and reviewed-by tags [Jakub]
- add more comments inint3_update_unoptimize[Andrii]
- several other minor changes and acks [Oleg]
- moveinsn_decodeout ofuprobe_init_insnto simplify the code
- alignuprobe_red_zone_testto 64 to make sure nop10 is not on page boundaryv4 changes:
- do not use 2nd int3 (ont +5 offset) because the call instruction is allways the same for the given nop10 address [Andrii/Peter]
- unmap unused trampoline vma after unsuccesfull optimization [sashiko]
- small change to patch#2 moveduser_64bit_modeearlier in the path and pass/usemm_structpointer directly fromarch_uprobe_optimizeinstead of getttingcurrent->mm
Andrii, keeping your ack, please shout otherwisev3 changes:
- use nop10 update suggested by Peter in [2]
- remove structuprobe_trampolineobject, use vma objects directly instead
- selftests fixes [sashiko]
- ack from Andriiv2 changes:
- several selftest fixes [sashiko]
- consolidateis_lea_insnandis_call_insninsto single check [Jakub Sitnicki]
- use propermm_structobject in__in_uprobe_trampolinecheck [sashiko]
- allow to copy uprobe trampolines vma objects on fork [sashiko]
- change uprobe syscall detection error from-ENXIOto-EPROTO[Andrii]
- added fork/clone tests
- I kept the selftest changes andnop5->nop10changes in separate commits for easier review, we can squash them later if we want to keep bisect working properly
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/#t
---ASAN-enabled
test_progsruns are not happy in CI, can you please check?I failed to release link in
test_uprobe_fork_optimized, fix is below I can send new version or separate fix
yeah, please fix the test, adjust comments as pointed out by AI and send v6. Seems like Peter wants to pick it up through tip, I don't mind.
Quoted 7 lines
also there's 2 things to solve/discuss once kernel changes are acked:
- selftest changes depend on:
selftests/bpf: Emit nop,nop10 instructions combo forx86_64arch that is taken from libbpf/usdt, I pushed the PR in here [1]
merged that one, we are good
Quoted 3 lines
- as bots complained the patchset breaks bisection, because kernel changes break selftests.. not sure what's prefered solution, as for me I'd keep it that way rather than mixing kernel/user space changes
I think it's fine to keep them separate
Quoted 32 lines
thanks,
jirka[1] https://github.com/libbpf/usdt/pull/16
---tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +3
-4@@ -988,7 +988,6 @@ static noreturn int child_func(void *arg)static void test_uprobe_fork_optimized (bool clone_vm) {struct uprobe_syscall_executed *skel =NULL ; -struct bpf_link *link =NULL ;unsigned long offset;int pid, status, err;char stack[65535 ];@@ -1001,9 +1000,9 @@ static void test_uprobe_fork_optimized(bool clone_vm)if (!ASSERT_OK_PTR (skel,"open_and_load" )) goto cleanup; - link =bpf_program__attach_uprobe_opts (skel->progs.test_uprobe, --1 ,"/proc/self/exe" , offset,NULL ); -if (!ASSERT_OK_PTR (link,"attach_uprobe" )) + skel->links.test_uprobe =bpf_program__attach_uprobe_opts (skel->progs.test_uprobe, +-1 ,"/proc/self/exe" , offset,NULL ); +if (!ASSERT_OK_PTR (skel->links.test_uprobe,"attach_uprobe" )) goto cleanup; skel->bss->pid =getpid ();
Peter Zijlstra
maintainerQuoted 6 lines · Andrii Nakryiko
Quoted 2 lines
I failed to release link in
test_uprobe_fork_optimized, fix is below I can send new version or separate fixyeah, please fix the test, adjust comments as pointed out by AI and send v6. Seems like Peter wants to pick it up through tip, I don't mind.
So I should wait for v6, and not munge the below delta in?
Quoted 10 lines · Peter Zijlstra
Quoted 6 lines · Andrii Nakryiko
Quoted 2 lines
I failed to release link in
test_uprobe_fork_optimized, fix is below I can send new version or separate fixyeah, please fix the test, adjust comments as pointed out by AI and send v6. Seems like Peter wants to pick it up through tip, I don't mind.
So I should wait for v6, and not munge the below delta in?
yep, I'll send it out later today
thanks,
jirka