tracing: wprobe: x86: Add wprobe for watchpoint

Hi,

Here is the 19th version of the series for adding new wprobe (watch probe) which provides memory access tracing event. Moreover, this can be used via event trigger. Thus it can trace memory access on dynamically allocated objects too.
The previous version (v17) is here (It seems v18 failed to send the patch.):

https://lore.kernel.org/all/179005108298.388919.4535333252892590932.stgit@devnote2/

v19 is just rebased on the latest probes/for-next branch.
In the version 18:
- Dropped MCE related fix because it is merged in -tip tree.
- Dropped the patches related to making DR7 updates NMI-safe (and the associated KVM guest debug register isolation patch) because wprobe itself does not support updating DR7 in NMI context (wprobe explicitly rejects execution in NMI context via in_nmi()). Those patches are still valid for KGDB, but not related to wprobe.
- Addressed review comments from Sashiko bot on wprobe triggers, selftests, and BTF struct offset resolution:
- Add register_wide_hw_breakpoint_cpuslocked() API to hw_breakpoint and use it under cpus_read_lock() in __register_trace_wprobe() to avoid both the CPU hotplug race and lockdep recursive locking warnings.
- Added early check for in_nmi() in wprobe_trigger() before acquiring

    tw->lock to avoid deadlock if interrupted by an NMI.

- Always evaluate __wprobe_count_func() under tw->lock in wprobe_trigger() to eliminate the TOCTOU race for counted triggers.
- Added "set_wprobe:<wprobe-event>":README requirement to wprobe trigger selftests so they are gracefully skipped if CONFIG_WPROBE_TRIGGERS=n.
- Added clear_trace in trigger-wprobe-hotplug.tc after taking CPU offline to avoid falsely matching events logged prior to offlining.
- In get_offset_of_field(), clarified that the containing structure/union type is checked for kflag (renamed mtype to container with comments) and verified that the resolved bit offset is byte-aligned.
- Updated all "Assisted-by:" tags across all commits to "Assisted-by: LLM".

Detailed changes are here.

Changes in v18:
- Drop patch "x86/mce: Fix hardware debug register corruption on task migration"
- Drop patch "perf/x86, KVM: Prevent host debug register leak into guest OS on NMI".
- Drop patch "x86/hw_breakpoints: Make DR7 updates NMI safe".
- Update "Assisted-by:" tags across all commits to "Assisted-by: LLM".
[01/11]
- Drop perf_guest_in_guest() check and cpu_dr7_seq sequence loop
since wprobe does not support updating DR7 in NMI context.
[03/11]
- Add register_wide_hw_breakpoint_cpuslocked() API.
[08/11]
- Use register_wide_hw_breakpoint_cpuslocked() under cpus_read_lock() in __register_trace_wprobe() to avoid CPU hotplug race and lockdep recursive locking warnings.
- Return early on in_nmi() in wprobe_trigger() before taking tw->lock to prevent NMI deadlock.
- Always evaluate __wprobe_count_func() under tw->lock in wprobe_trigger() to fix TOCTOU race for counted triggers.
- Drop redundant in_nmi() check from __wprobe_count_func().
- Defer data->count decrement to wprobe_trigger() under tw->lock after successful hardware update to avoid consuming count on missed triggers and prevent counter underflow.
[09/11]
- Add "set_wprobe:<wprobe-event>":README to requires line in wprobe trigger tests so they are skipped if CONFIG_WPROBE_TRIGGERS is disabled.
- Clear trace buffer after taking CPU offline in trigger-wprobe-hotplug.tc to avoid matching watchpoint events logged prior to offlining.
[10/11]
- Add "set_wprobe:<wprobe-event>":README to requires line in trigger-wprobe-btf-typecast.tc so the test is skipped if CONFIG_WPROBE_TRIGGERS is disabled.
[11/11]
- In get_offset_of_field(), rename mtype to container and document that it represents the containing structure/union type (handling anonymous structures/unions) rather than member's own type.
- In get_offset_of_field(), verify bit offset is byte-aligned before returning byte offset.
- Add "set_wprobe:<wprobe-event>":README to requires line in trigger-wprobe-btf-offset.tc so the test is skipped if CONFIG_WPROBE_TRIGGERS is disabled.

Public branch
-------------
I will push this branch as topic/wprobe-v2 to my tree so that it can be easily tested.

https://git.kernel.org/pub/scm/linux/kernel/git/mhiramat/linux.git/log/?h=topic/wprobe-v2

This is based on linux-trace tree's linux-trace/probes/fixes.

Usage
-----

The basic usage of this wprobe is similar to other probes;

w:[GRP/][EVENT] [r|w|rw]@<ADDRESS|SYMBOL[+|-OFFS]>[:LEN] [FETCHARGS]

This defines a new wprobe event. For example, to trace jiffies update, you can do;

echo 'w:my_jiffies w@jiffies:8 value=+0($addr)' >> dynamic_events echo 1 > events/wprobes/my_jiffies/enable

Moreover, this can be combined with event trigger to trace the memory access on slab objects. The trigger syntax is;

set_wprobe:WPROBE_EVENT:FIELD[+|-OFFSET][:COUNT] [if FILTER] clear_wprobe:WPROBE_EVENT[:FIELD[+|-OFFSET][:COUNT]] [if FILTER]

set_wprobe sets WPROBE_EVENT's watch address on FIELD[+|-OFFSET]. clear_wprobe clears WPROBE_EVENT's watch address if it is set to FIELD[+|-OFFSET]. If FIELD is omitted, forcibly clear the watch address when trigger event is hit.

For example, trace the first 8 byte of the dentry data structure passed to do_truncate() until it is deleted by dentry_kill().
(Note: all tracefs setup uses '>>' so that it does not kick do_truncate())

  # echo 'w:watch rw@-1:8 address=$addr value=+0($addr)' > dynamic_events

  # echo 'f:truncate do_truncate dentry=$arg2' >> dynamic_events
  # echo 'set_wprobe:watch:dentry' >> events/fprobes/truncate/trigger

  # echo 'f:dentry_kill dentry_kill dentry=$arg1' >> dynamic_events
  # echo 'clear_wprobe:watch:dentry' >> events/fprobes/dentry_kill/trigger

  # echo 1 >> events/fprobes/truncate/enable
  # echo 1 >> events/fprobes/dentry_kill/enable

  # echo aaa > /tmp/hoge
  # echo bbb > /tmp/hoge
  # echo ccc > /tmp/hoge
  # rm /tmp/hoge

Then, the trace data will show;

# tracer: nop
#
# entries-in-buffer/entries-written: 32/32 #P:8
#
# _-----=> irqs-off/BH-disabled
# / _----=> need-resched

 #                              | / _---=> hardirq/softirq
 #                              || / _--=> preempt-depth
 #                              ||| / _-=> migrate-disable
 #                              |||| /     delay
 #           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
 #              | |         |   |||||     |         |
               sh-107     [004] ...1.     9.990418: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
               sh-107     [004] ...1.     9.990914: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b3de78
               sh-107     [004] ...1.     9.993175: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049ddd40
               sh-107     [004] .....     9.995198: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
               sh-107     [004] ...1.     9.995389: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db998
               sh-107     [004] ..Zff     9.997503: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
               sh-107     [004] ..Zff     9.997509: watch: (path_openat+0x211/0xda0) address=0xffff8880048083a8 value=0x8200080
               sh-107     [004] ..Zff     9.997514: watch: (path_openat+0xa56/0xda0) address=0xffff8880048083a8 value=0x8200080
               sh-107     [004] ..Zff     9.997518: watch: (path_openat+0xae2/0xda0) address=0xffff8880048083a8 value=0x8200080
               sh-107     [004] .....     9.997521: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
               sh-107     [004] ...1.     9.997582: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004808270
               sh-107     [004] ...1.     9.999365: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db728
               sh-107     [004] ...1.     9.999388: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b1c000
               rm-113     [005] ..Zff    10.000965: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.000971: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.000984: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.000988: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.001010: watch: (lookup_one_qstr_excl+0x28/0x140) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.001014: watch: (lookup_one_qstr_excl+0xd1/0x140) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.001018: watch: (may_delete_dentry+0x1c/0x200) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.001021: watch: (may_delete_dentry+0x195/0x200) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] ..Zff    10.001031: watch: (vfs_unlink+0x5e/0x260) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] d.Z..    10.001067: watch: (d_make_discardable+0x1b/0x40) address=0xffff8880048083a8 value=0x8200080
               rm-113     [005] d.Z..    10.001071: watch: (d_make_discardable+0x29/0x40) address=0xffff8880048083a8 value=0x200080
               rm-113     [005] ...1.    10.001072: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
               rm-113     [005] ...1.    10.001218: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
               sh-107     [004] ...1.    10.001416: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db110
               sh-107     [004] ...1.    10.001444: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db248
               sh-107     [004] ...1.    10.001500: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
               sh-107     [004] ...1.    10.002067: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
               sh-107     [004] ...1.    10.904920: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
               sh-107     [004] ...1.    10.905129: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618

Thank you,

---
base-commit: 06e0d3b89da12f24ed5ed434acc8e2cb504f5e4f

Jinchao Wang (1):
x86/hw_breakpoints: Add arch_modify_local_hw_breakpoint_addr() API

Masami Hiramatsu (Google) (10):
HWBP: Add modify_local_hw_breakpoint_addr() API perf/hw_breakpoint: Add register_wide_hw_breakpoint_cpuslocked() API tracing/wprobe: Add wprobe (watchpoint probe) trace event support
x86: hw_breakpoint: Add a kconfig to clarify when a breakpoint fires
selftests: tracing: Add a basic testcase for wprobe
selftests: tracing: Add syntax testcase for wprobe tracing/wprobe: Add set_wprobe and clear_wprobe event triggers
selftests: tracing: Add wprobe trigger testcases tracing/wprobe: Support BTF typecast in fetchargs tracing/wprobe: Support BTF struct offset resolution in set_wprobe trigger

 Documentation/trace/index.rst                      |    1 
 Documentation/trace/wprobetrace.rst                |  200 ++
 arch/Kconfig                                       |   18 
 arch/x86/Kconfig                                   |    2 
 arch/x86/include/asm/hw_breakpoint.h               |    2 
 arch/x86/kernel/hw_breakpoint.c                    |   51 +
 include/linux/hw_breakpoint.h                      |   15 
 include/linux/trace_events.h                       |    3 
 kernel/events/hw_breakpoint.c                      |   86 +
 kernel/trace/Kconfig                               |   24 
 kernel/trace/Makefile                              |    1 
 kernel/trace/trace.c                               |    9 
 kernel/trace/trace.h                               |    6 
 kernel/trace/trace_events_trigger.c                |    2 
 kernel/trace/trace_probe.c                         |   36 
 kernel/trace/trace_probe.h                         |   22 
 kernel/trace/trace_wprobe.c                        | 1707 ++++++++++++++++++++
 tools/testing/selftests/ftrace/config              |    3 
 .../ftrace/test.d/dynevent/add_remove_wprobe.tc    |   66 +
 .../test.d/dynevent/wprobes_syntax_errors.tc       |   23 
 .../test.d/trigger/trigger-wprobe-btf-offset.tc    |   85 +
 .../test.d/trigger/trigger-wprobe-btf-typecast.tc  |   85 +
 .../test.d/trigger/trigger-wprobe-hotplug.tc       |  153 ++
 .../test.d/trigger/trigger-wprobe-syntax-errors.tc |   37 
 .../ftrace/test.d/trigger/trigger-wprobe.tc        |  107 +
 25 files changed, 2731 insertions(+), 13 deletions(-)
 create mode 100644 Documentation/trace/wprobetrace.rst
 create mode 100644 kernel/trace/trace_wprobe.c
 create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc

--
Masami Hiramatsu (Google) <[email protected]>

From: Jinchao Wang <[email protected]>

Wprobe needs to move an active per-CPU watchpoint without releasing and reserving its hardware slot.

Add arch_modify_local_hw_breakpoint_addr() as the architecture backend for modify_local_hw_breakpoint_addr(). It validates the new address, finds the installed local slot, and updates the hardware debug register and per-CPU shadow register. The core layer (modify_local_hw_breakpoint_addr()) handles updating bp->attr.bp_addr and counter_arch_bp(bp)->address.

The caller must provide an installed local event and a valid address. Slot ownership, breakpoint type, length, mask and DR7 remain unchanged.

---
Changes in v18:
- Drop perf_guest_in_guest() check and cpu_dr7_seq sequence loop since wprobe does not support updating DR7 in NMI context.
Changes in v17:
- In arch_modify_local_hw_breakpoint_addr(), check perf_guest_in_guest() instead of cpu_dr_in_guest.
Changes in v16:
- Clear breakpoint in shadow cpu_dr7 before setting hardware DR7 so an intervening NMI cannot re-enable the breakpoint via local_db_restore() while the address register is being updated, preventing spurious #DB exceptions.
- Check cpu_dr_in_guest and return -EBUSY to reject modifying local breakpoints while executing a KVM guest.
Changes in v15:
- Temporarily clear the breakpoint enable bit in DR7 before updating the debug address register, and re-enable it afterward, to comply with the x86 hardware specification and avoid spurious #DB exceptions.
Changes in v13:
- Add compiler barrier (barrier()) before checking cpu_dr7_seq to prevent hoisting the sequence check before set_debugreg().
- Clarify in commit log and code comment that the core layer manages updating bp->attr.bp_addr and counter_arch_bp(bp)->address.
Changes in v11:
- Return int error code instead of void.
- Validate the new address using hw_breakpoint_arch_parse() before updating registers.
- Increment cpu_dr7_seq in a sequence loop for NMI protection.
- Return -ENOENT if the breakpoint slot is not found on the local CPU.

---
 arch/x86/include/asm/hw_breakpoint.h |    2 +
 arch/x86/kernel/hw_breakpoint.c      |   51 ++++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)

index 0bc931cd0698..f35ec491f6dd 100644
@@ -59,6 +59,8 @@ extern int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
 
 
 int arch_install_hw_breakpoint(struct perf_event *bp);
+int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp,
+					 unsigned long addr);
 void arch_uninstall_hw_breakpoint(struct perf_event *bp);
 void hw_breakpoint_pmu_read(struct perf_event *bp);
 void hw_breakpoint_pmu_unthrottle(struct perf_event *bp);

From: Masami Hiramatsu (Google) <[email protected]>

Add modify_local_hw_breakpoint_addr() to update only the watched address of an installed hardware breakpoint on the local CPU without releasing and reserving its hardware slot. This is available when the architecture selects CONFIG_HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR.

The caller must provide an installed local event and a valid address, and update other CPUs separately.

---
Changes in v15:
- Use WRITE_ONCE() for counter_arch_bp(bp)->address and bp->attr.bp_addr in both update and rollback paths to prevent data races with concurrent readers (e.g. NMI handlers).
Changes in v13:
- Add comment to modify_local_hw_breakpoint_addr() clarifying the early update behavior of bp->attr.bp_addr during the NMI race window.
Changes in v12:
- Update bp->attr.bp_addr and counter_arch_bp(bp)->address before calling arch_modify_local_hw_breakpoint_addr() to prevent race conditions during interrupt handling.
- Restore counter_arch_bp(bp)->address and bp->attr.bp_addr if arch_modify_local_hw_breakpoint_addr() fails.
Changes in v11:
- Check return value of arch_modify_local_hw_breakpoint_addr() and return error on failure.
- Update bp->attr.bp_addr and counter_arch_bp(bp)->address only after arch modification succeeds.

---
 arch/Kconfig                  |    8 +++++++
 arch/x86/Kconfig              |    1 +
 include/linux/hw_breakpoint.h |    6 +++++
 kernel/events/hw_breakpoint.c |   48 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 63 insertions(+)

index 45c657772362..d021bc31ef14 100644
@@ -474,6 +474,14 @@ config HAVE_MIXED_BREAKPOINTS_REGS
 	  Select this option if your arch implements breakpoints under the
 	  latter fashion.
 
+config HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
+	bool
+	depends on HAVE_HW_BREAKPOINT
+	help
+	  Select this if the architecture can modify the address of an
+	  installed hardware breakpoint on the local CPU without releasing
+	  and reserving its hardware slot.
+
 config HAVE_USER_RETURN_NOTIFIER
 	bool
 

index 15fd9ec5ecac..38e29e465022 100644
@@ -249,6 +249,7 @@ config X86
 	select HAVE_FUNCTION_TRACER
 	select HAVE_GCC_PLUGINS
 	select HAVE_HW_BREAKPOINT
+	select HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
 	select HAVE_IOREMAP_PROT
 	select HAVE_IRQ_EXIT_ON_IRQ_STACK	if X86_64
 	select HAVE_IRQ_TIME_ACCOUNTING

index db199d653dd1..bf65c7fffd99 100644
@@ -81,6 +81,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
 			    perf_overflow_handler_t triggered,
 			    void *context);
 
+int modify_local_hw_breakpoint_addr(struct perf_event *bp,
+				    unsigned long addr);
+
 extern int register_perf_hw_breakpoint(struct perf_event *bp);
 extern void unregister_hw_breakpoint(struct perf_event *bp);
 extern void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cpu_events);

@@ -124,6 +127,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
 			    perf_overflow_handler_t triggered,
 			    void *context)		{ return NULL; }
 static inline int
+modify_local_hw_breakpoint_addr(struct perf_event *bp,
+				unsigned long addr) { return -EOPNOTSUPP; }
+static inline int
 register_perf_hw_breakpoint(struct perf_event *bp)	{ return -ENOSYS; }
 static inline void unregister_hw_breakpoint(struct perf_event *bp)	{ }
 static inline void

From: Masami Hiramatsu (Google) <[email protected]>

Add register_wide_hw_breakpoint_cpuslocked() which registers a wide hardware breakpoint across all online CPUs while caller holds cpus_read_lock(). This allows callers who need to perform additional setup under cpus_read_lock() without releasing it to avoid races with CPU hotplug and without triggering recursive lockdep warnings.

Refactor register_wide_hw_breakpoint() as a wrapper around register_wide_hw_breakpoint_cpuslocked().

---
 include/linux/hw_breakpoint.h |    9 +++++++++
 kernel/events/hw_breakpoint.c |   38 ++++++++++++++++++++++++++++++++------
 2 files changed, 41 insertions(+), 6 deletions(-)

index bf65c7fffd99..65efe55fef14 100644
@@ -76,6 +76,11 @@ register_wide_hw_breakpoint_cpu(struct perf_event_attr *attr,
 				void *context,
 				int cpu);
 
+extern struct perf_event * __percpu *
+register_wide_hw_breakpoint_cpuslocked(struct perf_event_attr *attr,
+				       perf_overflow_handler_t triggered,
+				       void *context);
+
 extern struct perf_event * __percpu *
 register_wide_hw_breakpoint(struct perf_event_attr *attr,
 			    perf_overflow_handler_t triggered,

@@ -123,6 +128,10 @@ register_wide_hw_breakpoint_cpu(struct perf_event_attr *attr,
 				void *context,
 				int cpu)		{ return NULL; }
 static inline struct perf_event * __percpu *
+register_wide_hw_breakpoint_cpuslocked(struct perf_event_attr *attr,
+				       perf_overflow_handler_t triggered,
+				       void *context)		{ return NULL; }
+static inline struct perf_event * __percpu *
 register_wide_hw_breakpoint(struct perf_event_attr *attr,
 			    perf_overflow_handler_t triggered,
 			    void *context)		{ return NULL; }

index 69bfb17b87c4..5627ad97600a 100644
@@ -832,27 +832,30 @@ void unregister_hw_breakpoint(struct perf_event *bp)
 EXPORT_SYMBOL_GPL(unregister_hw_breakpoint);
 
 /**
- * register_wide_hw_breakpoint - register a wide breakpoint in the kernel
+ * register_wide_hw_breakpoint_cpuslocked - register a wide breakpoint with cpus locked
  * @attr: breakpoint attributes
  * @triggered: callback to trigger when we hit the breakpoint
  * @context: context data could be used in the triggered callback
  *
+ * Same as register_wide_hw_breakpoint(), but caller must hold cpus_read_lock().
+ *
  * @return a set of per_cpu pointers to perf events
  */
 struct perf_event * __percpu *
-register_wide_hw_breakpoint(struct perf_event_attr *attr,
-			    perf_overflow_handler_t triggered,
-			    void *context)
+register_wide_hw_breakpoint_cpuslocked(struct perf_event_attr *attr,
+				       perf_overflow_handler_t triggered,
+				       void *context)
 {
 	struct perf_event * __percpu *cpu_events, *bp;
 	long err = 0;
 	int cpu;
 
+	lockdep_assert_cpus_held();
+
 	cpu_events = alloc_percpu(typeof(*cpu_events));
 	if (!cpu_events)
 		return ERR_PTR_PCPU(-ENOMEM);
 
-	cpus_read_lock();
 	for_each_online_cpu(cpu) {
 		bp = perf_event_create_kernel_counter(attr, cpu, NULL,
 						      triggered, context);

@@ -863,7 +866,6 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
 
 		per_cpu(*cpu_events, cpu) = bp;
 	}
-	cpus_read_unlock();
 
 	if (likely(!err))
 		return cpu_events;

@@ -871,6 +873,30 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
 	unregister_wide_hw_breakpoint(cpu_events);
 	return ERR_PTR_PCPU(err);
 }
+EXPORT_SYMBOL_GPL(register_wide_hw_breakpoint_cpuslocked);
+
+/**
+ * register_wide_hw_breakpoint - register a wide breakpoint in the kernel
+ * @attr: breakpoint attributes
+ * @triggered: callback to trigger when we hit the breakpoint
+ * @context: context data could be used in the triggered callback
+ *
+ * @return a set of per_cpu pointers to perf events
+ */
+struct perf_event * __percpu *
+register_wide_hw_breakpoint(struct perf_event_attr *attr,
+			    perf_overflow_handler_t triggered,
+			    void *context)
+{
+	struct perf_event * __percpu *cpu_events;
+
+	cpus_read_lock();
+	cpu_events = register_wide_hw_breakpoint_cpuslocked(attr, triggered,
+							    context);
+	cpus_read_unlock();
+
+	return cpu_events;
+}
 EXPORT_SYMBOL_GPL(register_wide_hw_breakpoint);
 
 /**

From: Masami Hiramatsu (Google) <[email protected]>

Add hardware-breakpoint-based dynamic trace event support (wprobe). Wprobe creates a dynamic event on data read/write accesses using hardware breakpoints and logs the access context and fetchargs.

---
Changes in v16:
- Remove non-existent wprobe_events file description from readme_msg in kernel/trace/trace.c as wprobe events are created via dynamic_events.
Changes in v15:
- Introduce trace_wprobe_is_valid_addr() to validate target address and natural alignment for wprobe length during event creation.
- Move local variable declarations to function entry in trace_wprobe_show() to comply with kernel C coding standards.
- Fix indentation in kernel/trace/trace.c to use tabs instead of spaces.
Changes in v14:
- In parse_address_spec(), reject address 0 and user-space addresses by changing the check to '_addr < TASK_SIZE'.
- In trace_wprobe_show(), format symbol and offset explicitly instead of using '%pS' to ensure dynamic event restoration works properly.
- Add offset field to struct trace_wprobe and pass it to alloc_trace_wprobe().
Changes in v13:
- Fix documentation notation to SYMBOL[+|-OFFS] as same as others.
- Fix documentation indents and add default value.
- Fix README file to add [<args>].
- Add a note about module unloading.
Changes in v12:
- Fix syntax comment typo ('b' to 'w') and remove dead 'ret = 0' initialization in __trace_wprobe_create().
- Fix documentation notation to SYMBOL[[+|-]OFFS].
Changes in v11:
- Add trace_wprobe_is_busy() to prevent releasing busy events.
Changes in v9:
- Use kzalloc_flex for alloc_trace_wprobe.

---
 Documentation/trace/index.rst       |    1 
 Documentation/trace/wprobetrace.rst |   78 ++++
 include/linux/trace_events.h        |    2 
 kernel/trace/Kconfig                |   13 +
 kernel/trace/Makefile               |    1 
 kernel/trace/trace.c                |    5 
 kernel/trace/trace.h                |    5 
 kernel/trace/trace_probe.c          |   21 +
 kernel/trace/trace_probe.h          |   10 
 kernel/trace/trace_wprobe.c         |  775 +++++++++++++++++++++++++++++++++++
 10 files changed, 907 insertions(+), 4 deletions(-)
 create mode 100644 Documentation/trace/wprobetrace.rst
 create mode 100644 kernel/trace/trace_wprobe.c

index f4058e8e92e3..4f3fa60e4ecd 100644
@@ -36,6 +36,7 @@ the Linux kernel.
    kprobes
    kprobetrace
    fprobetrace
+   wprobetrace
    eprobetrace
    fprobe
    ring-buffer-design

index 5cbd09c8be8d..43ffd9a76d88 100644
@@ -337,6 +337,7 @@ enum {
 	TRACE_EVENT_FL_UPROBE_BIT,
 	TRACE_EVENT_FL_EPROBE_BIT,
 	TRACE_EVENT_FL_FPROBE_BIT,
+	TRACE_EVENT_FL_WPROBE_BIT,
 	TRACE_EVENT_FL_CUSTOM_BIT,
 	TRACE_EVENT_FL_TEST_STR_BIT,
 };

@@ -367,6 +368,7 @@ enum {
 	TRACE_EVENT_FL_UPROBE		= (1 << TRACE_EVENT_FL_UPROBE_BIT),
 	TRACE_EVENT_FL_EPROBE		= (1 << TRACE_EVENT_FL_EPROBE_BIT),
 	TRACE_EVENT_FL_FPROBE		= (1 << TRACE_EVENT_FL_FPROBE_BIT),
+	TRACE_EVENT_FL_WPROBE		= (1 << TRACE_EVENT_FL_WPROBE_BIT),
 	TRACE_EVENT_FL_CUSTOM		= (1 << TRACE_EVENT_FL_CUSTOM_BIT),
 	TRACE_EVENT_FL_TEST_STR		= (1 << TRACE_EVENT_FL_TEST_STR_BIT),
 };

index 0ab5916575a9..b58c2565024f 100644
@@ -862,6 +862,19 @@ config EPROBE_EVENTS
 	  convert the type of an event field. For example, turn an
 	  address into a string.
 
+config WPROBE_EVENTS
+	bool "Enable wprobe-based dynamic events"
+	depends on TRACING
+	depends on HAVE_HW_BREAKPOINT
+	select PROBE_EVENTS
+	select DYNAMIC_EVENTS
+	help
+	  This allows the user to add watchpoint tracing events based on
+	  hardware breakpoints on the fly via the ftrace interface.
+
+	  Those events can be inserted wherever hardware breakpoints can be
+	  set, and record accessed memory address and values.
+
 config BPF_EVENTS
 	depends on BPF_SYSCALL
 	depends on (KPROBE_EVENTS || UPROBE_EVENTS) && PERF_EVENTS

index f934ff586bd4..141c8323de20 100644
@@ -126,6 +126,7 @@ obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o
 obj-$(CONFIG_FPROBE) += fprobe.o
 obj-$(CONFIG_RETHOOK) += rethook.o
 obj-$(CONFIG_FPROBE_EVENTS) += trace_fprobe.o
+obj-$(CONFIG_WPROBE_EVENTS) += trace_wprobe.o
 
 obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
 obj-$(CONFIG_RV) += rv/

index e4a490d3d08c..438787e4eec3 100644
@@ -4295,7 +4295,7 @@ static const char readme_msg[] =
 	"\t\t\t  Write into this file to define/undefine new trace events.\n"
 #endif
 #if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS) || \
-    defined(CONFIG_FPROBE_EVENTS)
+	defined(CONFIG_FPROBE_EVENTS) || defined(CONFIG_WPROBE_EVENTS)
 	"\t  accepts: event-definitions (one definition per line)\n"
 #if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS)
 	"\t   Format: p[:[<group>/][<event>]] <place> [<args>]\n"

@@ -4305,6 +4305,9 @@ static const char readme_msg[] =
 	"\t           f[:[<group>/][<event>]] <func-name>[%return] [<args>]\n"
 	"\t           t[:[<group>/][<event>]] <tracepoint> [<args>]\n"
 #endif
+#ifdef CONFIG_WPROBE_EVENTS
+	"\t           w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>] [<args>]\n"
+#endif
 #ifdef CONFIG_HIST_TRIGGERS
 	"\t           s:[synthetic/]<event> <field> [<field>]\n"
 #endif

index 3749485a7d85..e44b5dd325bb 100644
@@ -179,6 +179,11 @@ struct fexit_trace_entry_head {
 	unsigned long		ret_ip;
 };
 
+struct wprobe_trace_entry_head {
+	struct trace_entry	ent;
+	unsigned long		ip;
+};
+
 #define TRACE_BUF_SIZE		1024
 
 struct trace_array;

index cfb710095f65..b4e7668df589 100644
@@ -1403,6 +1403,23 @@ static int parse_probe_vars(char *orig_arg, const struct fetch_type *t,
 			return 0;
 	}
 
+	/* wprobe only support "$addr" and "$value" variable */
+	if (ctx->flags & TPARG_FL_WPROBE) {
+		if (!strcmp(arg, "addr")) {
+			code->op = FETCH_OP_BADDR;
+			return 0;
+		}
+		if (!strcmp(arg, "value")) {
+			code->op = FETCH_OP_BADDR;
+			code++;
+			code->op = FETCH_OP_DEREF;
+			code->offset = 0;
+			*pcode = code;
+			return 0;
+		}
+		goto inval;
+	}
+
 	if (strcmp(arg, "comm") == 0 || strcmp(arg, "COMM") == 0) {
 		code->op = FETCH_OP_COMM;
 		return 0;

@@ -1462,8 +1479,8 @@ static int parse_probe_arg_register(char *arg, struct fetch_insn *code,
 {
 	int ret;
 
-	if (ctx->flags & (TPARG_FL_TEVENT | TPARG_FL_FPROBE)) {
-		/* eprobe and fprobe do not handle registers */
+	if (ctx->flags & (TPARG_FL_TEVENT | TPARG_FL_FPROBE | TPARG_FL_WPROBE)) {
+		/* eprobe, fprobe and wprobe do not handle registers */
 		trace_probe_log_err(ctx->offset, BAD_VAR);
 		return -EINVAL;
 	}

index d1fb3520700f..f20cafb87cc6 100644
@@ -90,6 +90,7 @@ typedef int (*print_type_func_t)(struct trace_seq *, void *, void *);
 	FETCH_OP(STACK, param),		/* Stack: .param = index */	\
 	FETCH_OP(STACKP, none),		/* Stack pointer */		\
 	FETCH_OP(RETVAL, none),		/* Return value */		\
+	FETCH_OP(BADDR, none),		/* Break address */		\
 	FETCH_OP(IMM, imm),		/* Immediate: .immediate */	\
 	FETCH_OP(COMM, none),		/* Current comm */		\
 	FETCH_OP(CURRENT, none),	/* Current task_struct address */\

@@ -420,6 +421,7 @@ static inline int traceprobe_get_entry_data_size(struct trace_probe *tp)
 #define TPARG_FL_USER   BIT(4)
 #define TPARG_FL_FPROBE BIT(5)
 #define TPARG_FL_TPOINT BIT(6)
+#define TPARG_FL_WPROBE BIT(7)
 #define TPARG_FL_LOC_MASK	GENMASK(4, 0)
 
 static inline bool tparg_is_function_entry(unsigned int flags)

@@ -546,6 +548,10 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
 	C(ARG_TOO_LONG,		"Argument expression is too long"),		\
 	C(ARRAY_NO_CLOSE,	"Array is not closed"),		\
 	C(ARRAY_TOO_BIG,	"Array number is too big"),		\
+	C(BAD_ACCESS_ADDR,	"Invalid access memory address"),		\
+	C(BAD_ACCESS_FMT,	"Access memory address requires @"),		\
+	C(BAD_ACCESS_LEN,	"This memory access length is not supported"),	\
+	C(BAD_ACCESS_TYPE,	"Bad memory access type"),			\
 	C(BAD_ADDR_SUFFIX,	"Invalid probed address suffix"),		\
 	C(BAD_ARG_NAME,		"Argument name must follow the same rules as C identifiers"),	\
 	C(BAD_ARG_NUM,		"Invalid argument number"),		\

@@ -628,7 +634,9 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
 	C(TYPECAST_NOT_EVENT,	"Typecasts are only for eprobe fields"),	\
 	C(TYPECAST_REQ_FIELD,	"Typecast requires a field access"),	\
 	C(TYPECAST_SYM_OFFSET,	"@SYM+/-OFFSET with typecast needs parentheses"),	\
-	C(USED_ARG_NAME,	"This argument name is already used"),
+	C(USED_ARG_NAME,	"This argument name is already used"),		\
+	C(WPROBE_NO_MAXACT,	"Watchpoint probe does not support maxactive"),	\
+	C(WPROBE_NO_SIBLING,	"Watchpoint probe does not support sibling probes"),
 
 #undef C
 #define C(a, b)		TP_ERR_##a

From: Masami Hiramatsu (Google) <[email protected]>

Add CONFIG_HAVE_POST_BREAKPOINT_HOOK which indicates the hw_breakpoint on that architecture fires after the target memory has been modified. This is currently x86 only behavior.

---
Changes in v11:
- Move select to alphabetically sorted place.

---
 arch/Kconfig         |   10 ++++++++++
 arch/x86/Kconfig     |    1 +
 kernel/trace/Kconfig |    1 +
 3 files changed, 12 insertions(+)

index d021bc31ef14..5292bd05d5d4 100644
@@ -474,6 +474,16 @@ config HAVE_MIXED_BREAKPOINTS_REGS
 	  Select this option if your arch implements breakpoints under the
 	  latter fashion.
 
+config HAVE_POST_BREAKPOINT_HOOK
+	bool
+	depends on HAVE_HW_BREAKPOINT
+	help
+	  Depending on the arch implementation of hardware breakpoints,
+	  some of them provide breakpoint hook after the target memory
+	  is modified.
+	  Select this option if your arch implements breakpoints overflow
+	  handler hooks after the target memory is modified.
+
 config HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
 	bool
 	depends on HAVE_HW_BREAKPOINT

index 38e29e465022..e4eaf0b607d3 100644
@@ -289,6 +289,7 @@ config X86
 	select MMU_GATHER_RCU_TABLE_FREE
 	select MMU_GATHER_MERGE_VMAS
 	select HAVE_POSIX_CPU_TIMERS_TASK_WORK
+	select HAVE_POST_BREAKPOINT_HOOK
 	select HAVE_REGS_AND_STACK_ACCESS_API
 	select HAVE_RELIABLE_STACKTRACE		if UNWINDER_ORC || STACK_VALIDATION
 	select HAVE_FUNCTION_ARG_ACCESS_API

index b58c2565024f..d9b6fa5c35d9 100644
@@ -866,6 +866,7 @@ config WPROBE_EVENTS
 	bool "Enable wprobe-based dynamic events"
 	depends on TRACING
 	depends on HAVE_HW_BREAKPOINT
+	depends on HAVE_POST_BREAKPOINT_HOOK
 	select PROBE_EVENTS
 	select DYNAMIC_EVENTS
 	help

From: Masami Hiramatsu (Google) <[email protected]>

Add 'add_remove_wprobe.tc' testcase for testing wprobe event that tests adding and removing operations of the wprobe event.

---
Changes in v17:
- Catch failures when enabling wprobe and call exit_unresolved in case hardware watchpoints are exhausted.
Changes in v9:
- Fix command check logic to prevent early exit under 'set -e' (errexit) when grep or test fails.
- Simplify enable/disable status checks by removing cat pipes.
Changes in v8:
- Fixed silently test failure path.

---
 tools/testing/selftests/ftrace/config              |    1 
 .../ftrace/test.d/dynevent/add_remove_wprobe.tc    |   66 ++++++++++++++++++++
 2 files changed, 67 insertions(+)
 create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc

index 29f05f7c6e2c..c2caea1ba1f5 100644
@@ -33,3 +33,4 @@ CONFIG_SYNTH_EVENTS=y
 CONFIG_TRACER_SNAPSHOT=y
 CONFIG_UPROBES=y
 CONFIG_UPROBE_EVENTS=y
+CONFIG_WPROBE_EVENTS=y

From: Masami Hiramatsu (Google) <[email protected]>

Add "wprobe_syntax_errors.tc" testcase for testing syntax errors of the watch probe events.

---
Changes in v14:
- Update comment for invalid offset syntax check to BAD_PROBE_ADDR.
- Add testcase for invalid address 0 error check (BAD_ACCESS_ADDR).
Changes in v12:
- Add NO_ARG_BODY error case.
Changes in v11:
- Add WPROBE_NO_MAXACT syntax error case.

---
 .../test.d/dynevent/wprobes_syntax_errors.tc       |   23 ++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc

new file mode 100644
index 000000000000..e0df58fe7aa3
@@ -0,0 +1,23 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Watch probe event parser error log check
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README
+
+check_error() { # command-with-error-pos-by-^
+    ftrace_errlog_check 'wprobe' "$1" 'dynamic_events'
+}
+
+check_error '^w'			# NO_ARG_BODY
+check_error 'w^10 w@jiffies'		# WPROBE_NO_MAXACT
+check_error 'w ^symbol'			# BAD_ACCESS_FMT
+check_error 'w ^a@symbol'		# BAD_ACCESS_TYPE
+check_error 'w w@^symbol'		# BAD_ACCESS_ADDR
+check_error 'w w@^0:4'			# BAD_ACCESS_ADDR
+check_error 'w w@jiffies^+offset'	# BAD_PROBE_ADDR
+check_error 'w w@jiffies:^100'		# BAD_ACCESS_LEN
+check_error 'w w@jiffies ^$arg1'	# BAD_VAR
+check_error 'w w@jiffies ^$retval'	# BAD_VAR
+check_error 'w w@jiffies ^$stack'	# BAD_VAR
+check_error 'w w@jiffies ^%ax'		# BAD_VAR
+
+exit 0

From: Masami Hiramatsu (Google) <[email protected]>

Add set_wprobe and clear_wprobe event triggers to dynamically attach and detach hardware breakpoint address monitoring based on event field contents.

---
Changes in v18:
- Use register_wide_hw_breakpoint_cpuslocked() under cpus_read_lock() in __register_trace_wprobe() to avoid CPU hotplug race and lockdep recursive locking warnings.
- Return early on in_nmi() in wprobe_trigger() before taking tw->lock to prevent NMI deadlock.
- Always evaluate __wprobe_count_func() under tw->lock in wprobe_trigger() to fix TOCTOU race for counted triggers.
- Drop redundant in_nmi() check from __wprobe_count_func().
- Defer data->count decrement to wprobe_trigger() under tw->lock after successful hardware update to avoid consuming count on missed triggers and prevent counter underflow.
Changes in v17:
- Add dynamic CPU hotplug callbacks to handle CPU offline/online.
- Restore EVENT_FILE_FL_SOFT_DISABLED_BIT in wprobe_trigger_free().
- Check whether trigger event has "unlimited" field before skipping "unlimited" in clear_wprobe trigger parsing.
- Call tracepoint_synchronize_unregister() before freeing trigger data via data->cmd_ops->free() on registration failure.
- Implement __wprobe_count_func()/wprobe_count_func() helpers to consolidate precondition checks and use it for count_func() and inside wprobe_trigger() function.
Changes in v16:
- Immediately update the hardware breakpoint on the local CPU via trace_wprobe_update_local() in wprobe_trigger() to avoid workqueue latency, and only kick irq_work for other CPUs.
- Skip redundant updates in trace_wprobe_update_local() if bp->attr.bp_addr already matches the target address.
- Use strcmp() instead of strncmp() for "unlimited" to avoid misinterpreting field names starting with "unlimited" as count.
- Normalize "unlimited" and "count=" parameters before creating trigger data so trigger parsing handles count keywords correctly.
Changes in v15:
- Use trace_wprobe_is_valid_addr() in wprobe_trigger() to validate address and natural alignment for wprobe length.
- Record SMP local update failures in tw->missed counter.
- Expose missed event count as a trailing comment in trigger file.
- Add set_wprobe and clear_wprobe triggers to readme_msg in trace.c.
- Fix line length exceeding 100 columns in wprobe_trigger_log_err macro.
- Initialize irq_work and work structs immediately after allocation in alloc_trace_wprobe() and check tw->work.func in free_trace_wprobe() to prevent WARN_ON(!work->func) on early allocation failure.
Changes in v14:
- Allow '-1' as a dummy address in parse_address_spec() to set initial address to WPROBE_DEFAULT_CLEAR_ADDRESS for trigger-only wprobes.
- In trace_wprobe_show(), format WPROBE_DEFAULT_CLEAR_ADDRESS as '-1'.
- Update documentation and examples to use '-1' instead of '0'.
- Skip field parsing in clear_wprobe if parameter is a count keyword.
- Set tw->addr to WPROBE_DEFAULT_CLEAR_ADDRESS before soft-enabling event and roll back on failure.
- Fix memory and refcount leak of trigger_data/wprobe_data on error path.
Changes in v13:
- Align WPROBE_DEFAULT_CLEAR_ADDRESS to 8 bytes using dedicated u64.
- Remove manual count decrement to rely on core event_trigger_count().
- Parse optional FIELD argument in clear_wprobe command.
- Fix trigger reference count leak and error cleanup in command parsing.
- Update document to add [:COUNT] to trigger syntax and fix adjust forms.
- Add notes about the exclusive trigger setting behavior.
- Remove unused dyn_event_ops_mutex.
Changes in v12:
- Decrement trigger data->count only when watchpoint state is actually changed.
- Remove dyn_event_ops_mutex to fix lockdep circular dependency deadlock.
- Add event_trigger_init() call to prevent premature freeing of trigger_data.

---
 Documentation/trace/wprobetrace.rst |  107 +++++
 include/linux/trace_events.h        |    1 
 kernel/trace/Kconfig                |   10 
 kernel/trace/trace.c                |    4 
 kernel/trace/trace.h                |    1 
 kernel/trace/trace_events_trigger.c |    2 
 kernel/trace/trace_probe.c          |    2 
 kernel/trace/trace_probe.h          |    9 
 kernel/trace/trace_wprobe.c         |  788 +++++++++++++++++++++++++++++++++++
 9 files changed, 919 insertions(+), 5 deletions(-)

index 31cc68b464e9..df8985283312 100644
@@ -76,3 +76,110 @@ Wprobe event does not disable itself even if the module is unloaded.
 For example, if you add a wprobe event on a module variable, and then
 unload the module, the wprobe event will still be enabled. This is for
 watching the address is used unexpectedly after the module is unloaded.
+
+Combination with trigger action
+-------------------------------
+The event trigger action can extend the utilization of this wprobe.
+
+- set_wprobe:WPEVENT:FIELD[+|-ADJUST][:COUNT]
+- clear_wprobe:WPEVENT[:FIELD[+|-ADJUST][:COUNT]]
+
+Set these triggers to the target event, then the WPROBE event will be
+setup to trace the memory access at FIELD[+|-ADJUST] address.
+When clear_wprobe is hit, if FIELD is NOT specified, the WPEVENT is
+forcibly cleared. If FIELD[+|-ADJUST] is set, it clears WPEVENT only
+if its watching address is the same as the FIELD[+|-ADJUST] value.
+If COUNT is specified, it will set/clear WPEVENT only if it hits COUNT
+times.
+
+Notes:
+- set_wprobe only works on the wprobe which is NOT set a valid address yet,
+  and it must be enabled after the trigger is set.
+- clear_wprobe only works on the wprobe which is set a valid address, and it
+  will be soft-disabled after the trigger is cleared.
+- Therefore, if a trigger sets/clears a wprobe, other/same trigger events
+  will not work (on the same event) while the wprobe is set.
+
+The set_wprobe trigger does not change the type and length, these
+must be set when creating a new wprobe.
+
+The WPROBE event must be disabled when setting the new trigger
+and it will be busy afterwards. Recommended usage is to add a new
+wprobe at invalid dummy address (-1) and keep disabled.
+
+Wprobe triggers only support target addresses in kernel memory. If a
+set_wprobe trigger evaluates to a user-space memory address or NULL
+pointer, the trigger action ignores the update and skips setting the
+watchpoint.
+
+Wprobe triggers are not supported on kprobe_events, because kprobes
+themselves can use software breakpoints which conflicts with wprobe
+operation.
+
+
+For example, trace the first 8 bytes of the dentry data structure passed
+to do_truncate() until it is deleted by dentry_kill().
+(Note: all tracefs setup uses '>>' so that it does not kick do_truncate())
+::
+
+  # echo 'w:watch rw@-1:8 address=$addr value=+0($addr)' >> dynamic_events
+  # echo 'f:truncate do_truncate dentry=$arg2' >> dynamic_events
+  # echo 'set_wprobe:watch:dentry' >> events/fprobes/truncate/trigger
+  # echo 'f:dentry_kill dentry_kill dentry=$arg1' >> dynamic_events
+  # echo 'clear_wprobe:watch:dentry' >> events/fprobes/dentry_kill/trigger
+  # echo 1 >> events/fprobes/truncate/enable
+  # echo 1 >> events/fprobes/dentry_kill/enable
+
+  # echo aaa > /tmp/hoge
+  # echo bbb > /tmp/hoge
+  # echo ccc > /tmp/hoge
+  # rm /tmp/hoge
+
+Then, the trace data will show::
+
+ # tracer: nop
+ #
+ # entries-in-buffer/entries-written: 32/32   #P:8
+ #
+ #                                _-----=> irqs-off/BH-disabled
+ #                               / _----=> need-resched
+ #                              | / _---=> hardirq/softirq
+ #                              || / _--=> preempt-depth
+ #                              ||| / _-=> migrate-disable
+ #                              |||| /     delay
+ #           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
+ #              | |         |   |||||     |         |
+               sh-107     [004] ...1.     9.990418: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+               sh-107     [004] ...1.     9.990914: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b3de78
+               sh-107     [004] ...1.     9.993175: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049ddd40
+               sh-107     [004] .....     9.995198: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
+               sh-107     [004] ...1.     9.995389: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db998
+               sh-107     [004] ..Zff     9.997503: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+               sh-107     [004] ..Zff     9.997509: watch: (path_openat+0x211/0xda0) address=0xffff8880048083a8 value=0x8200080
+               sh-107     [004] ..Zff     9.997514: watch: (path_openat+0xa56/0xda0) address=0xffff8880048083a8 value=0x8200080
+               sh-107     [004] ..Zff     9.997518: watch: (path_openat+0xae2/0xda0) address=0xffff8880048083a8 value=0x8200080
+               sh-107     [004] .....     9.997521: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
+               sh-107     [004] ...1.     9.997582: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004808270
+               sh-107     [004] ...1.     9.999365: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db728
+               sh-107     [004] ...1.     9.999388: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b1c000
+               rm-113     [005] ..Zff    10.000965: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+               rm-113     [005] ..Zff    10.000971: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
+               rm-113     [005] ..Zff    10.000984: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+               rm-113     [005] ..Zff    10.000988: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
+               rm-113     [005] ..Zff    10.001010: watch: (lookup_one_qstr_excl+0x28/0x140) address=0xffff8880048083a8 value=0x8200080
+               rm-113     [005] ..Zff    10.001014: watch: (lookup_one_qstr_excl+0xd1/0x140) address=0xffff8880048083a8 value=0x8200080
+               rm-113     [005] ..Zff    10.001018: watch: (may_delete_dentry+0x1c/0x200) address=0xffff8880048083a8 value=0x8200080
+               rm-113     [005] ..Zff    10.001021: watch: (may_delete_dentry+0x195/0x200) address=0xffff8880048083a8 value=0x8200080
+               rm-113     [005] ..Zff    10.001031: watch: (vfs_unlink+0x5e/0x260) address=0xffff8880048083a8 value=0x8200080
+               rm-113     [005] d.Z..    10.001067: watch: (d_make_discardable+0x1b/0x40) address=0xffff8880048083a8 value=0x8200080
+               rm-113     [005] d.Z..    10.001071: watch: (d_make_discardable+0x29/0x40) address=0xffff8880048083a8 value=0x200080
+               rm-113     [005] ...1.    10.001072: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
+               rm-113     [005] ...1.    10.001218: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
+               sh-107     [004] ...1.    10.001416: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db110
+               sh-107     [004] ...1.    10.001444: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db248
+               sh-107     [004] ...1.    10.001500: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+               sh-107     [004] ...1.    10.002067: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
+               sh-107     [004] ...1.    10.904920: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
+               sh-107     [004] ...1.    10.905129: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+
+You can see the watch event is correctly configured on the dentry.

index 43ffd9a76d88..f81a5308c116 100644
@@ -738,6 +738,7 @@ enum event_trigger_type {
 	ETT_EVENT_HIST		= (1 << 4),
 	ETT_HIST_ENABLE		= (1 << 5),
 	ETT_EVENT_EPROBE	= (1 << 6),
+	ETT_EVENT_WPROBE	= (1 << 7),
 };
 
 extern int filter_match_preds(struct event_filter *filter, void *rec);

index d9b6fa5c35d9..5fd8ed63c516 100644
@@ -876,6 +876,16 @@ config WPROBE_EVENTS
 	  Those events can be inserted wherever hardware breakpoints can be
 	  set, and record accessed memory address and values.
 
+config WPROBE_TRIGGERS
+	depends on WPROBE_EVENTS
+	depends on HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
+	bool
+	default y
+	help
+	  This adds an event trigger which will set the wprobe on a specific
+	  field of an event. This allows user to trace the memory access of
+	  an address pointed by the event field.
+
 config BPF_EVENTS
 	depends on BPF_SYSCALL
 	depends on (KPROBE_EVENTS || UPROBE_EVENTS) && PERF_EVENTS

index 438787e4eec3..9d19307c4196 100644
@@ -4362,6 +4362,10 @@ static const char readme_msg[] =
 	"\t   trigger: traceon, traceoff\n"
 	"\t            enable_event:<system>:<event>\n"
 	"\t            disable_event:<system>:<event>\n"
+#ifdef CONFIG_WPROBE_TRIGGERS
+	"\t            set_wprobe:<wprobe-event>:<field>[+|-<offset>]\n"
+	"\t            clear_wprobe:<wprobe-event>[:<field>[+|-<offset>]]\n"
+#endif
 #ifdef CONFIG_HIST_TRIGGERS
 	"\t            enable_hist:<system>:<event>\n"
 	"\t            disable_hist:<system>:<event>\n"

index e44b5dd325bb..d8b2a34033d6 100644
@@ -1985,6 +1985,7 @@ trigger_data_alloc(struct event_command *cmd_ops, char *cmd, char *param,
 		   void *private_data);
 extern void trigger_data_free(struct event_trigger_data *data);
 extern int event_trigger_init(struct event_trigger_data *data);
+extern void event_trigger_free(struct event_trigger_data *data);
 extern int trace_event_trigger_enable_disable(struct trace_event_file *file,
 					      int trigger_enable);
 extern void update_cond_flag(struct trace_event_file *file);

index 4d2fde996c0f..bb24fca2fb02 100644
@@ -589,7 +589,7 @@ int event_trigger_init(struct event_trigger_data *data)
  * Usually used directly as the @free method in event trigger
  * implementations.
  */
-static void
+void
 event_trigger_free(struct event_trigger_data *data)
 {
 	if (WARN_ON_ONCE(data->ref <= 0))

index b4e7668df589..86bb89a718dc 100644
@@ -20,7 +20,7 @@
 #undef C
 #define C(a, b)		b
 
-static const char *trace_probe_err_text[] = { ERRORS };
+const char *trace_probe_err_text[] = { ERRORS };
 
 static const char *reserved_field_names[] = {
 	"common_type",

index f20cafb87cc6..1e11077ead67 100644
@@ -636,7 +636,12 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
 	C(TYPECAST_SYM_OFFSET,	"@SYM+/-OFFSET with typecast needs parentheses"),	\
 	C(USED_ARG_NAME,	"This argument name is already used"),		\
 	C(WPROBE_NO_MAXACT,	"Watchpoint probe does not support maxactive"),	\
-	C(WPROBE_NO_SIBLING,	"Watchpoint probe does not support sibling probes"),
+	C(WPROBE_NO_SIBLING,	"Watchpoint probe does not support sibling probes"), \
+	C(WPROBE_ON_KPROBE,	"Wprobe trigger is not supported on kprobe event"), \
+	C(WPROBE_NOT_FOUND,	"Target wprobe event is not found"),		\
+	C(WPROBE_BUSY,		"Target wprobe event is already enabled"),	\
+	C(WPROBE_NEED_FIELD,	"Wprobe trigger requires a target field"),	\
+	C(WPROBE_BAD_FIELD,	"Target field must be pointer size"),
 
 #undef C
 #define C(a, b)		TP_ERR_##a

@@ -660,6 +665,8 @@ void __trace_probe_log_err(int offset, int err);
 
 DEFINE_FREE(trace_probe_log_clear, const char *, if (_T) trace_probe_log_clear())
 
+extern const char *trace_probe_err_text[];
+
 #define trace_probe_log_err(offs, err)	\
 	__trace_probe_log_err(offs, TP_ERR_##err)
 

From: Masami Hiramatsu (Google) <[email protected]>

Add testcases for checking wprobe triggers. This sets set_wprobe and clear_wprobe triggers on fprobe event and static trace event to monitor memory accesses within the trace-events-sample kernel module. Also add a testcase for verifying wprobe triggers across CPU hotplug, and a testcase for verifying wprobe trigger syntax error logging.

---
Changes in v18:
- Add "set_wprobe:<wprobe-event>":README to requires line in wprobe trigger tests so they are skipped if CONFIG_WPROBE_TRIGGERS is disabled.
- Clear trace buffer after taking CPU offline in trigger-wprobe-hotplug.tc to avoid matching watchpoint events logged prior to offlining.
Changes in v17:
- Add trigger-wprobe-hotplug.tc to verify wprobe triggers across CPU offline and online transitions.
Changes in v15:
- Add explicit failure checks with fail helper in trigger-wprobe.tc.
- Declare fprobe README requirement in trigger-wprobe.tc.
- Add trigger file check to trigger-wprobe-syntax-errors.tc requires line.
- Fix subject tag to tracing.
Changes in v14:
- Update dummy wprobe event definition to use '-1' instead of '0'.
- Add BTF argument dependency requirement in trigger-wprobe.tc.
- Add delay before clear_trace in trigger-wprobe.tc to avoid race with clear_wprobe trigger execution.
Changes in v13:
- Look up target function dynamically in trigger-wprobe-syntax-errors.tc.
- Disable set_wprobe trigger before verifying clear_wprobe in trigger-wprobe.tc to avoid race condition with sample_timer_cb.
Changes in v12:
- Add trigger-wprobe-syntax-errors.tc for verifying wprobe trigger syntax error logging.
- Fix requires line in trigger-wprobe-syntax-errors.tc so test is not evaluated as unsupported prior to execution.
- Define dfd=$arg1 fetcharg on testevent for trigger syntax checks.
Changes in v11:
- Update testcase to use trace-events-sample kernel module instead of VFS file operations.

---
 tools/testing/selftests/ftrace/config              |    1 
 .../test.d/trigger/trigger-wprobe-hotplug.tc       |  153 ++++++++++++++++++++
 .../test.d/trigger/trigger-wprobe-syntax-errors.tc |   37 +++++
 .../ftrace/test.d/trigger/trigger-wprobe.tc        |  107 ++++++++++++++
 4 files changed, 298 insertions(+)
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc

index c2caea1ba1f5..7e07088464b6 100644
@@ -34,3 +34,4 @@ CONFIG_TRACER_SNAPSHOT=y
 CONFIG_UPROBES=y
 CONFIG_UPROBE_EVENTS=y
 CONFIG_WPROBE_EVENTS=y
+CONFIG_WPROBE_TRIGGERS=y

new file mode 100644
index 000000000000..87170a66e882
@@ -0,0 +1,37 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger syntax errors
+# requires: dynamic_events error_log "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "set_wprobe:<wprobe-event>":README
+
+check_error() { # command-with-error-pos-by-^
+    ftrace_errlog_check "wprobe_trigger" "$1" "events/fprobes/testevent/trigger"
+}
+
+TARGET_FUNC=$(grep -m 1 -E -w "[tT] (vfs_read|do_sys_openat2|do_sys_open)" /proc/kallsyms | awk '{print $3}')
+if [ -z "$TARGET_FUNC" ]; then
+    echo "UNRESOLVED: target function not found"
+    exit_unresolved
+fi
+
+# Add a dummy fprobe event to attach triggers to
+echo "f:fprobes/testevent $TARGET_FUNC dfd=\$arg1" > dynamic_events
+
+# Add a target wprobe event
+echo 'w:watch rw@-1:8' >> dynamic_events
+
+# Test errors on trigger syntax
+check_error 'set_wprobe:^non_exist_wprobe:dfd'	# WPROBE_NOT_FOUND
+check_error 'set_wprobe:^'				# WPROBE_NOT_FOUND
+check_error 'set_wprobe:watch^'				# WPROBE_NEED_FIELD
+check_error 'set_wprobe:watch:^non_exist_field'	# NO_EVENT_FIELD
+
+# Enable target wprobe event and test WPROBE_BUSY error
+echo 1 > events/wprobes/watch/enable
+check_error 'set_wprobe:^watch:dfd'			# WPROBE_BUSY
+echo 0 > events/wprobes/watch/enable
+
+# Cleanup
+echo '-:watch' >> dynamic_events
+echo '-:testevent' >> dynamic_events
+
+exit 0

From: Masami Hiramatsu (Google) <[email protected]>

Allow BTF typecast syntax (STRUCT)FETCHARG->MEMBER in wprobe event fetchargs. Previously, handle_typecast() rejected any probe context that was not a function entry/return or tracepoint event probe.

Wprobe events use $addr (the accessed address) and $value (the value at that address). By enabling BTF typecast, users can now cast these to a concrete struct type and access its fields directly. For example:

echo 'w:watch rw@-1:8 dflag=(dentry)$addr->d_flags' >> dynamic_events

With a set_wprobe trigger pointing the watchpoint at a dentry address, the resulting trace shows d_flags being accessed at that location.

Note that $addr and $value are restricted to kernel-space memory, which is consistent with the existing TPARG_FL_KERNEL flag used when parsing wprobe fetchargs.

---
Changes in v18:
- Add "set_wprobe:<wprobe-event>":README to requires line in trigger-wprobe-btf-typecast.tc so the test is skipped if CONFIG_WPROBE_TRIGGERS is disabled.
Changes in v15:
- Document BTF typecast syntax for wprobe fetchargs in wprobetrace.rst.
- Add explicit failure checks with fail helper in trigger-wprobe-btf-typecast.tc.
- Declare fprobe README requirement in trigger-wprobe-btf-typecast.tc.
Changes in v14:
- Update dummy wprobe event definition to use '-1' instead of '0'.
Changes in v11:
- Update trigger-wprobe-btf-typecast.tc to use trace-events-sample kernel module.
- Fix commit comment.
Changes in v9:
- Newly added.

---
 Documentation/trace/wprobetrace.rst                |    8 ++
 kernel/trace/trace_probe.c                         |   13 +++
 kernel/trace/trace_probe.h                         |    5 +
 tools/testing/selftests/ftrace/config              |    1 
 .../test.d/trigger/trigger-wprobe-btf-typecast.tc  |   85 ++++++++++++++++++++
 5 files changed, 111 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc

index df8985283312..20a11443c6db 100644
@@ -47,6 +47,14 @@ Synopsis of wprobe-events
                   (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types
                   (x8/x16/x32/x64), "char", "string", "ustring", "symbol", "symstr"
                   and bitfield are supported.
+   (STRUCT[,ASGN])FETCHARG->MEMBER[->MEMBER] : If BTF is supported, typecast
+                  FETCHARG to a pointer to STRUCT and then dereference the
+                  pointer defined by ->MEMBER. ASGN can be specified optionally.
+                  If ASGN is specified, FETCHARG will be cast to the same offset
+                  position as the ASGN member, rather than to the beginning of
+                  the STRUCT.
+   (STRUCT[,ASGN])(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above
+                  can also be used with another FETCHARG.
 
    (\*1) This is useful for fetching a field of data structures.
    (\*2) "u" means user-space dereference.

index 86bb89a718dc..a982fc384f0c 100644
@@ -890,6 +890,16 @@ static int query_btf_struct(const char *sname, struct traceprobe_parse_context *
 		ctx->struct_btf = NULL;
 	}
 
+	if (ctx->btf) {
+		id = btf_find_by_name_kind(ctx->btf, sname, BTF_KIND_STRUCT);
+		if (id > 0) {
+			btf_get(ctx->btf);
+			ctx->struct_btf = ctx->btf;
+			ctx->last_struct = btf_type_by_id(ctx->struct_btf, id);
+			return 0;
+		}
+	}
+
 	id = bpf_find_btf_id(sname, BTF_KIND_STRUCT, &btf);
 	if (id < 0)
 		return id;

@@ -965,7 +975,8 @@ static int handle_typecast(char *arg, struct traceprobe_parse_context *ctx)
 
 	if (!(tparg_is_event_probe(ctx->flags) ||
 	      tparg_is_function_entry(ctx->flags) ||
-	      tparg_is_function_return(ctx->flags))) {
+	      tparg_is_function_return(ctx->flags) ||
+	      tparg_is_wprobe(ctx->flags))) {
 		trace_probe_log_err(ctx->offset, NOSUP_BTFARG);
 		return -EOPNOTSUPP;
 	}

index 1e11077ead67..c54c554b1949 100644
@@ -439,6 +439,11 @@ static inline bool tparg_is_event_probe(unsigned int flags)
 	return !!(flags & TPARG_FL_TEVENT);
 }
 
+static inline bool tparg_is_wprobe(unsigned int flags)
+{
+	return !!(flags & TPARG_FL_WPROBE);
+}
+
 /* Each typecast consumes nested level. So the max number of typecast is 8. */
 #define TRACEPROBE_MAX_NESTED_LEVEL 8
 

index 7e07088464b6..05654c1c4ffd 100644
@@ -2,6 +2,7 @@ CONFIG_BOOT_CONFIG=y
 CONFIG_BOOTTIME_TRACING=y
 CONFIG_BPF_SYSCALL=y
 CONFIG_DEBUG_INFO_BTF=y
+CONFIG_DEBUG_INFO_BTF_MODULES=y
 CONFIG_DEBUG_INFO_DWARF4=y
 CONFIG_EPROBE_EVENTS=y
 CONFIG_FPROBE=y

From: Masami Hiramatsu (Google) <[email protected]>

Allow set_wprobe trigger to use BTF struct offset resolution to specify the target address field.

---
Changes in v18:
- In get_offset_of_field(), rename mtype to container and document that it represents the containing structure/union type (handling anonymous structures/unions) rather than member's own type.
- In get_offset_of_field(), verify bit offset is byte-aligned before returning byte offset.
- Add "set_wprobe:<wprobe-event>":README to requires line in trigger-wprobe-btf-offset.tc so the test is skipped if CONFIG_WPROBE_TRIGGERS is disabled.
Changes in v15:
- Duplicate field string to avoid modifying glob in place in wprobe_trigger_typecast_parse() so tracing_log_err() prints pristine command string.
- Remove leftover offset and adjust debug print in wprobe_trigger_print().
- Document BTF struct offset resolution syntax in wprobetrace.rst.
- Add explicit failure checks with fail helper in trigger-wprobe-btf-offset.tc.
- Declare fprobe README requirement in trigger-wprobe-btf-offset.tc.
Changes in v14:
- Update dummy wprobe event definition to use '-1' instead of '0'.
- Pass member_type to btf_find_struct_member() and check its kflag in get_offset_of_field().
Changes in v13:
- Check for field token before calling wprobe_trigger_field_parse() in clear_wprobe to avoid spurious error log entries on numeric counts.
Changes in v12:
- Refactor field parsing logic into wprobe_trigger_field_parse().
- Remove unused variable count_str in wprobe_trigger_cmd_parse().

---
 Documentation/trace/wprobetrace.rst                |    7 
 kernel/trace/trace_wprobe.c                        |  304 +++++++++++++++-----
 .../test.d/trigger/trigger-wprobe-btf-offset.tc    |   85 ++++++
 3 files changed, 318 insertions(+), 78 deletions(-)
 create mode 100644 tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc

index 20a11443c6db..1d9b32d02d53 100644
@@ -90,10 +90,17 @@ Combination with trigger action
 The event trigger action can extend the utilization of this wprobe.
 
 - set_wprobe:WPEVENT:FIELD[+|-ADJUST][:COUNT]
+- set_wprobe:WPEVENT:(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST][:COUNT]
 - clear_wprobe:WPEVENT[:FIELD[+|-ADJUST][:COUNT]]
+- clear_wprobe:WPEVENT[:(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST][:COUNT]]
 
 Set these triggers to the target event, then the WPROBE event will be
 setup to trace the memory access at FIELD[+|-ADJUST] address.
+If BTF is supported, the target address can also be resolved using BTF
+struct offset resolution: `(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST]`.
+Here, EVENT_FIELD is an event field containing a pointer to STRUCT (or to
+the ASGN member if ASGN is specified), and MEMBER is the struct member whose
+offset is resolved automatically via BTF.
 When clear_wprobe is hit, if FIELD is NOT specified, the WPEVENT is
 forcibly cleared. If FIELD[+|-ADJUST] is set, it clears WPEVENT only
 if its watching address is the same as the FIELD[+|-ADJUST] value.