Faultable Tracepoints
Subsystems (26)
Mathieu Desnoyers
reviewerv5 cover Faultable Tracepoints
Wire up the system call tracepoints with Tasks Trace RCU to allow the ftrace, perf, and eBPF tracers to handle page faults.
This series does the initial wire-up allowing tracers to handle page faults, but leaves out the actual handling of said page faults as future work.
I have tested this against a feature branch of lttng-modules which implements handling of page faults for the filename argument of the openat(2) system call.
This v5 addresses comments from the previous round of review [1].
Steven Rostedt suggested separating tracepoints into two separate sections. It is unclear how that approach would prove to be an improvement over the currently proposed approach, so those changes were not incorporated. See [2] for my detailed reply.
In the previous round, Peter Zijlstra suggested use of SRCU rather than Tasks Trace RCU. See my reply about the distinction between SRCU and Tasks Trace RCU [3] and this explanation from Paul E. McKenney about the purpose of Tasks Trace RCU [4].
The macros DEFINE_INACTIVE_GUARD and activate_guard are added to cleanup.h for use in the __DO_TRACE() macro. Those appear to be more flexible than the guard_if() proposed by Peter Zijlstra in the previous round of review [5].
This series is based on kernel v6.9.6.
Thanks,
Mathieu
Mathieu Desnoyers (8):
cleanup.h: Header include guard should match header name cleanup.h guard: Rename DEFINE_ prefix to DECLARE_ cleanup.h: Introduce DEFINE_INACTIVE_GUARD and activate_guard
tracing: Introduce faultable tracepoints tracing/ftrace: Add support for faultable tracepoints tracing/bpf-trace: Add support for faultable tracepoints tracing/perf: Add support for faultable tracepoints
tracing: Convert sys_enter/exit to faultable tracepoints
drivers/cxl/core/cdat.c | 2 +- drivers/cxl/cxl.h | 2 +- drivers/gpio/gpiolib.h | 2 +- drivers/platform/x86/intel/pmc/core_ssram.c | 2 +- fs/fuse/virtio_fs.c | 2 +- fs/pstore/inode.c | 4 +- include/linux/bitmap.h | 2 +- include/linux/cleanup.h | 85 ++++++++++++-------- include/linux/cpu.h | 2 +- include/linux/cpumask.h | 2 +- include/linux/device.h | 6 +- include/linux/file.h | 4 +- include/linux/firmware.h | 2 +- include/linux/gpio/driver.h | 4 +- include/linux/iio/iio.h | 4 +- include/linux/irqflags.h | 4 +- include/linux/mutex.h | 6 +- include/linux/of.h | 2 +- include/linux/pci.h | 4 +- include/linux/percpu.h | 2 +- include/linux/preempt.h | 6 +- include/linux/rcupdate.h | 2 +- include/linux/rwsem.h | 10 +-- include/linux/sched/task.h | 4 +- include/linux/slab.h | 4 +- include/linux/spinlock.h | 38 ++++----- include/linux/srcu.h | 2 +- include/linux/tracepoint-defs.h | 14 ++++ include/linux/tracepoint.h | 88 +++++++++++++++------ include/sound/pcm.h | 6 +- include/trace/bpf_probe.h | 20 ++++- include/trace/define_trace.h | 7 ++ include/trace/events/syscalls.h | 4 +- include/trace/perf.h | 22 +++++- include/trace/trace_events.h | 68 +++++++++++++++- init/Kconfig | 1 + kernel/sched/core.c | 4 +- kernel/sched/sched.h | 16 ++-- kernel/trace/bpf_trace.c | 11 ++- kernel/trace/trace_events.c | 28 +++++-- kernel/trace/trace_fprobe.c | 5 +- kernel/trace/trace_syscalls.c | 52 ++++++++++-- kernel/tracepoint.c | 65 +++++++++------ lib/locking-selftest.c | 12 +-- sound/core/control_led.c | 2 +- 45 files changed, 441 insertions(+), 193 deletions(-)
Signature · 1 line
2.39.2
Mathieu Desnoyers
reviewerPATCH v5 1/8 cleanup.h: Header include guard should match header name
The include guard should match the header name. Rename __LINUX_GUARDS_H to __LINUX_CLEANUP_H.
--- include/linux/cleanup.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
include/linux/cleanup.h +3 -3
@@ -1,6 +1,6 @@/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __LINUX_GUARDS_H -#define __LINUX_GUARDS_H +#ifndef __LINUX_CLEANUP_H +#define __LINUX_CLEANUP_H#include <linux/compiler.h>
@@ -247,4 +247,4 @@ __DEFINE_LOCK_GUARD_0(_name, _lock) { return class_##_name##_lock_ptr(_T); } -#endif /* __LINUX_GUARDS_H */ +#endif /* __LINUX_CLEANUP_H */
Signature · 1 line
2.39.2
Mathieu Desnoyers
reviewerPATCH v5 2/8 cleanup.h guard: Rename DEFINE_ prefix to DECLARE_
In reply to Mathieu DesnoyersThe convention used in other kernel headers (e.g. wait.h, percpu-defs.h) is to use "DECLARE_" prefix for macros emitting externs, static inlines and type definitions.
The "DEFINE_" prefix is used for variable definitions.
In preparation to introduce a "DEFINE_INACTIVE_GUARD()" to actually define a guard variable, rename all the guard "DEFINE_" prefix to "DECLARE_".
--- drivers/cxl/core/cdat.c | 2 +- drivers/cxl/cxl.h | 2 +- drivers/gpio/gpiolib.h | 2 +- drivers/platform/x86/intel/pmc/core_ssram.c | 2 +- fs/fuse/virtio_fs.c | 2 +- fs/pstore/inode.c | 4 +- include/linux/bitmap.h | 2 +- include/linux/cleanup.h | 56 ++++++++++----------- include/linux/cpu.h | 2 +- include/linux/cpumask.h | 2 +- include/linux/device.h | 6 +-- include/linux/file.h | 4 +- include/linux/firmware.h | 2 +- include/linux/gpio/driver.h | 4 +- include/linux/iio/iio.h | 4 +- include/linux/irqflags.h | 4 +- include/linux/mutex.h | 6 +-- include/linux/of.h | 2 +- include/linux/pci.h | 4 +- include/linux/percpu.h | 2 +- include/linux/preempt.h | 6 +-- include/linux/rcupdate.h | 2 +- include/linux/rwsem.h | 10 ++-- include/linux/sched/task.h | 4 +- include/linux/slab.h | 4 +- include/linux/spinlock.h | 38 +++++++------- include/linux/srcu.h | 2 +- include/sound/pcm.h | 6 +-- kernel/sched/core.c | 4 +- kernel/sched/sched.h | 16 +++--- lib/locking-selftest.c | 12 ++--- sound/core/control_led.c | 2 +- 32 files changed, 110 insertions(+), 110 deletions(-)
drivers/cxl/core/cdat.c +1 -1
@@ -385,7 +385,7 @@ static void discard_dsmas(struct xarray *xa) }xa_destroy (xa); } -DEFINE_FREE(dsmas, struct xarray *, if (_T) discard_dsmas(_T)) +DECLARE_FREE(dsmas, struct xarray *, if (_T) discard_dsmas(_T))void cxl_endpoint_parse_cdat (struct cxl_port *port) {
drivers/cxl/cxl.h +1 -1
@@ -737,7 +737,7 @@ struct cxl_root *devm_cxl_add_root(struct device *host, const struct cxl_root_ops *ops);struct cxl_root *find_cxl_root (struct cxl_port *port);void put_cxl_root (struct cxl_root *cxl_root); -DEFINE_FREE(put_cxl_root, struct cxl_root *, if (_T) put_cxl_root(_T)) +DECLARE_FREE(put_cxl_root, struct cxl_root *, if (_T) put_cxl_root(_T))int devm_cxl_enumerate_ports (struct cxl_memdev *cxlmd);void cxl_bus_rescan (void );
drivers/gpio/gpiolib.h +1 -1
@@ -199,7 +199,7 @@ struct gpio_chip_guard {int idx; }; -DEFINE_CLASS(gpio_chip_guard, +DECLARE_CLASS(gpio_chip_guard, struct gpio_chip_guard, srcu_read_unlock(&_T.gdev->srcu, _T.idx), ({
drivers/platform/x86/intel/pmc/core_ssram.c +1 -1
@@ -29,7 +29,7 @@#define LPM_REG_COUNT 28#define LPM_MODE_OFFSET 1 -DEFINE_FREE(pmc_core_iounmap, void __iomem *, iounmap(_T)); +DECLARE_FREE(pmc_core_iounmap, void __iomem *, iounmap(_T));static u32 pmc_core_find_guid (struct pmc_info *list,const struct pmc_reg_map *map) {
fs/fuse/virtio_fs.c +1 -1
@@ -852,7 +852,7 @@ static void virtio_fs_cleanup_dax(void *data)put_dax (dax_dev); } -DEFINE_FREE(cleanup_dax, struct dax_dev *, if (!IS_ERR_OR_NULL(_T)) virtio_fs_cleanup_dax(_T)) +DECLARE_FREE(cleanup_dax, struct dax_dev *, if (!IS_ERR_OR_NULL(_T)) virtio_fs_cleanup_dax(_T))static int virtio_fs_setup_dax (struct virtio_device *vdev,struct virtio_fs *fs) {
fs/pstore/inode.c +2 -2
@@ -35,7 +35,7 @@ static LIST_HEAD(records_list);static DEFINE_MUTEX (pstore_sb_lock);static struct super_block *pstore_sb; -DEFINE_FREE(pstore_iput, struct inode *, if (_T) iput(_T)) +DECLARE_FREE(pstore_iput, struct inode *, if (_T) iput(_T))struct pstore_private {struct list_head list;
@@ -63,7 +63,7 @@ static void free_pstore_private(struct pstore_private *private) }kfree (private); } -DEFINE_FREE(pstore_private, struct pstore_private *, free_pstore_private(_T)); +DECLARE_FREE(pstore_private, struct pstore_private *, free_pstore_private(_T));static void *pstore_ftrace_seq_start (struct seq_file *s,loff_t *pos) {
include/linux/bitmap.h +1 -1
@@ -131,7 +131,7 @@ unsigned long *bitmap_alloc_node(unsigned int nbits, gfp_t flags, int node);unsigned long *bitmap_zalloc_node (unsigned int nbits,gfp_t flags,int node);void bitmap_free (const unsigned long *bitmap); -DEFINE_FREE(bitmap, unsigned long *, if (_T) bitmap_free(_T)) +DECLARE_FREE(bitmap, unsigned long *, if (_T) bitmap_free(_T)) /* Managed variants of the above. */ unsigned long *devm_bitmap_alloc(struct device *dev,
include/linux/cleanup.h +28 -28
147 lines not on this page · raw message
include/linux/cpu.h +1 -1
@@ -162,7 +162,7 @@ static inline int remove_cpu(unsigned int cpu) { return -EPERM; }static inline void smp_shutdown_nonboot_cpus (unsigned int primary_cpu) { }#endif /* !CONFIG_HOTPLUG_CPU */ -DEFINE_LOCK_GUARD_0(cpus_read_lock, cpus_read_lock(), cpus_read_unlock()) +DECLARE_LOCK_GUARD_0(cpus_read_lock, cpus_read_lock(), cpus_read_unlock())#ifdef CONFIG_PM_SLEEP_SMP extern int freeze_secondary_cpus (int primary);
include/linux/cpumask.h +1 -1
@@ -991,7 +991,7 @@ static inline bool cpumask_available(cpumask_var_t mask) }#endif /* CONFIG_CPUMASK_OFFSTACK */ -DEFINE_FREE(free_cpumask_var, struct cpumask *, if (_T) free_cpumask_var(_T)); +DECLARE_FREE(free_cpumask_var, struct cpumask *, if (_T) free_cpumask_var(_T));/* It's common to want to use cpu_all_mask in struct member initializers, * so it has to refer to an address rather than a pointer. */
include/linux/device.h +3 -3
@@ -1005,7 +1005,7 @@ static inline void device_unlock(struct device *dev)mutex_unlock (&dev->mutex); } -DEFINE_GUARD(device, struct device *, device_lock(_T), device_unlock(_T)) +DECLARE_GUARD(device, struct device *, device_lock(_T), device_unlock(_T))static inline void device_lock_assert (struct device *dev) {
@@ -1055,7 +1055,7 @@ void device_initialize(struct device *dev);int __must_check device_add (struct device *dev);void device_del (struct device *dev); -DEFINE_FREE(device_del, struct device *, if (_T) device_del(_T)) +DECLARE_FREE(device_del, struct device *, if (_T) device_del(_T))int device_for_each_child (struct device *dev,void *data,int (*fn)(struct device *dev,void *data));
@@ -1224,7 +1224,7 @@ extern int (*platform_notify_remove)(struct device *dev);struct device *get_device (struct device *dev);void put_device (struct device *dev); -DEFINE_FREE(put_device, struct device *, if (_T) put_device(_T)) +DECLARE_FREE(put_device, struct device *, if (_T) put_device(_T))bool kill_device (struct device *dev);
include/linux/file.h +2 -2
@@ -83,7 +83,7 @@ static inline void fdput_pos(struct fd f)fdput (f); } -DEFINE_CLASS(fd, struct fd, fdput(_T), fdget(fd), int fd) +DECLARE_CLASS(fd, struct fd, fdput(_T), fdget(fd), int fd)extern int f_dupfd (unsigned int from,struct file *file,unsigned flags);extern int replace_fd (unsigned fd,struct file *file,unsigned flags);
@@ -93,7 +93,7 @@ extern int __get_unused_fd_flags(unsigned flags, unsigned long nofile);extern int get_unused_fd_flags (unsigned flags);extern void put_unused_fd (unsigned int fd); -DEFINE_CLASS(get_unused_fd, int, if (_T >= 0) put_unused_fd(_T), +DECLARE_CLASS(get_unused_fd, int, if (_T >= 0) put_unused_fd(_T), get_unused_fd_flags(flags), unsigned flags)extern void fd_install (unsigned int fd,struct file *file);
include/linux/firmware.h +1 -1
@@ -199,6 +199,6 @@ static inline void firmware_upload_unregister(struct fw_upload *fw_upload)int firmware_request_cache (struct device *device,const char *name); -DEFINE_FREE(firmware, struct firmware *, release_firmware(_T)) +DECLARE_FREE(firmware, struct firmware *, release_firmware(_T))#endif
include/linux/gpio/driver.h +2 -2
@@ -541,7 +541,7 @@ struct _gpiochip_for_each_data {unsigned int *i; }; -DEFINE_CLASS(_gpiochip_for_each_data, +DECLARE_CLASS(_gpiochip_for_each_data, struct _gpiochip_for_each_data, if (*_T.label) kfree(*_T.label), ({
@@ -650,7 +650,7 @@ struct gpio_device *gpio_device_find(const void *data,struct gpio_device *gpio_device_get (struct gpio_device *gdev);void gpio_device_put (struct gpio_device *gdev); -DEFINE_FREE(gpio_device_put, struct gpio_device *, +DECLARE_FREE(gpio_device_put, struct gpio_device *, if (!IS_ERR_OR_NULL(_T)) gpio_device_put(_T))struct device *gpio_device_to_device (struct gpio_device *gdev);
include/linux/iio/iio.h +2 -2
@@ -644,10 +644,10 @@ void iio_device_release_direct_mode(struct iio_dev *indio_dev);* This autocleanup logic is normally used via * iio_device_claim_direct_scoped(). */ -DEFINE_GUARD(iio_claim_direct, struct iio_dev *, iio_device_claim_direct_mode(_T), +DECLARE_GUARD(iio_claim_direct, struct iio_dev *, iio_device_claim_direct_mode(_T), iio_device_release_direct_mode(_T)) -DEFINE_GUARD_COND(iio_claim_direct, _try, ({ +DECLARE_GUARD_COND(iio_claim_direct, _try, ({ struct iio_dev *dev; int d = iio_device_claim_direct_mode(_T);
include/linux/irqflags.h +2 -2
@@ -256,8 +256,8 @@ extern void warn_bogus_irq_restore(void);#define irqs_disabled_flags (flags) raw_irqs_disabled_flags(flags) -DEFINE_LOCK_GUARD_0(irq, local_irq_disable(), local_irq_enable()) -DEFINE_LOCK_GUARD_0 (irqsave, +DECLARE_LOCK_GUARD_0(irq, local_irq_disable(), local_irq_enable()) +DECLARE_LOCK_GUARD_0 (irqsave,local_irq_save (_T->flags),local_irq_restore (_T->flags), unsigned long flags)
include/linux/mutex.h +3 -3
@@ -166,8 +166,8 @@ extern void mutex_unlock(struct mutex *lock);extern int atomic_dec_and_mutex_lock (atomic_t *cnt,struct mutex *lock); -DEFINE_GUARD(mutex, struct mutex *, mutex_lock(_T), mutex_unlock(_T)) -DEFINE_GUARD_COND(mutex, _try, mutex_trylock(_T)) -DEFINE_GUARD_COND(mutex, _intr, mutex_lock_interruptible(_T) == 0) +DECLARE_GUARD(mutex, struct mutex *, mutex_lock(_T), mutex_unlock(_T)) +DECLARE_GUARD_COND(mutex, _try, mutex_trylock(_T)) +DECLARE_GUARD_COND(mutex, _intr, mutex_lock_interruptible(_T) == 0)#endif /* __LINUX_MUTEX_H */
include/linux/of.h +1 -1
@@ -135,7 +135,7 @@ static inline struct device_node *of_node_get(struct device_node *node) }static inline void of_node_put (struct device_node *node) { }#endif /* !CONFIG_OF_DYNAMIC */ -DEFINE_FREE(device_node, struct device_node *, if (_T) of_node_put(_T)) +DECLARE_FREE(device_node, struct device_node *, if (_T) of_node_put(_T)) /* Pointer for first entry in chain of all nodes. */extern struct device_node *of_root;
include/linux/pci.h +2 -2
@@ -1171,7 +1171,7 @@ int pci_get_interrupt_pin(struct pci_dev *dev, struct pci_dev **bridge);u8 pci_common_swizzle (struct pci_dev *dev,u8 *pinp);struct pci_dev *pci_dev_get (struct pci_dev *dev);void pci_dev_put (struct pci_dev *dev); -DEFINE_FREE(pci_dev_put, struct pci_dev *, if (_T) pci_dev_put(_T)) +DECLARE_FREE(pci_dev_put, struct pci_dev *, if (_T) pci_dev_put(_T))void pci_remove_bus (struct pci_bus *b);void pci_stop_and_remove_bus_device (struct pci_dev *dev);void pci_stop_and_remove_bus_device_locked (struct pci_dev *dev);
@@ -1882,7 +1882,7 @@ void pci_cfg_access_unlock(struct pci_dev *dev);void pci_dev_lock (struct pci_dev *dev);int pci_dev_trylock (struct pci_dev *dev);void pci_dev_unlock (struct pci_dev *dev); -DEFINE_GUARD(pci_dev, struct pci_dev *, pci_dev_lock(_T), pci_dev_unlock(_T)) +DECLARE_GUARD(pci_dev, struct pci_dev *, pci_dev_lock(_T), pci_dev_unlock(_T))/* * PCI domain support. Sometimes called PCI segment (eg by ACPI),
include/linux/percpu.h +1 -1
@@ -134,7 +134,7 @@ extern void __percpu *__alloc_percpu(size_t size, size_t align) __alloc_size(1);extern void free_percpu (void __percpu *__pdata);extern size_t pcpu_alloc_size (void __percpu *__pdata); -DEFINE_FREE(free_percpu, void __percpu *, free_percpu(_T)) +DECLARE_FREE(free_percpu, void __percpu *, free_percpu(_T))extern phys_addr_t per_cpu_ptr_to_phys (void *addr);
include/linux/preempt.h +3 -3
@@ -477,8 +477,8 @@ static __always_inline void preempt_enable_nested(void)preempt_enable (); } -DEFINE_LOCK_GUARD_0(preempt, preempt_disable(), preempt_enable()) -DEFINE_LOCK_GUARD_0(preempt_notrace, preempt_disable_notrace(), preempt_enable_notrace()) -DEFINE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable()) +DECLARE_LOCK_GUARD_0(preempt, preempt_disable(), preempt_enable()) +DECLARE_LOCK_GUARD_0(preempt_notrace, preempt_disable_notrace(), preempt_enable_notrace()) +DECLARE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable())#endif /* __LINUX_PREEMPT_H */
include/linux/rcupdate.h +1 -1
@@ -1090,6 +1090,6 @@ rcu_head_after_call_rcu(struct rcu_head *rhp, rcu_callback_t f)extern int rcu_expedited;extern int rcu_normal; -DEFINE_LOCK_GUARD_0(rcu, rcu_read_lock(), rcu_read_unlock()) +DECLARE_LOCK_GUARD_0(rcu, rcu_read_lock(), rcu_read_unlock())#endif /* __LINUX_RCUPDATE_H */
include/linux/rwsem.h +5 -5
@@ -238,12 +238,12 @@ extern void up_read(struct rw_semaphore *sem);*/ extern void up_write (struct rw_semaphore *sem); -DEFINE_GUARD(rwsem_read, struct rw_semaphore *, down_read(_T), up_read(_T)) -DEFINE_GUARD_COND(rwsem_read, _try, down_read_trylock(_T)) -DEFINE_GUARD_COND(rwsem_read, _intr, down_read_interruptible(_T) == 0) +DECLARE_GUARD(rwsem_read, struct rw_semaphore *, down_read(_T), up_read(_T)) +DECLARE_GUARD_COND(rwsem_read, _try, down_read_trylock(_T)) +DECLARE_GUARD_COND(rwsem_read, _intr, down_read_interruptible(_T) == 0) -DEFINE_GUARD(rwsem_write, struct rw_semaphore *, down_write(_T), up_write(_T)) -DEFINE_GUARD_COND (rwsem_write, _try,down_write_trylock (_T)) +DECLARE_GUARD(rwsem_write, struct rw_semaphore *, down_write(_T), up_write(_T)) +DECLARE_GUARD_COND (rwsem_write, _try,down_write_trylock (_T))/* * downgrade write lock to read lock
include/linux/sched/task.h +2 -2
@@ -163,7 +163,7 @@ static inline void put_task_struct(struct task_struct *t)call_rcu (&t->rcu, __put_task_struct_rcu_cb); } -DEFINE_FREE(put_task, struct task_struct *, if (_T) put_task_struct(_T)) +DECLARE_FREE(put_task, struct task_struct *, if (_T) put_task_struct(_T))static inline void put_task_struct_many (struct task_struct *t,int nr) {
@@ -228,6 +228,6 @@ static inline void task_unlock(struct task_struct *p)spin_unlock (&p->alloc_lock); } -DEFINE_GUARD(task_lock, struct task_struct *, task_lock(_T), task_unlock(_T)) +DECLARE_GUARD(task_lock, struct task_struct *, task_lock(_T), task_unlock(_T))#endif /* _LINUX_SCHED_TASK_H */
include/linux/slab.h +2 -2
@@ -266,7 +266,7 @@ void kfree(const void *objp);void kfree_sensitive (const void *objp);size_t __ksize (const void *objp); -DEFINE_FREE(kfree, void *, if (!IS_ERR_OR_NULL(_T)) kfree(_T)) +DECLARE_FREE(kfree, void *, if (!IS_ERR_OR_NULL(_T)) kfree(_T))/** * ksize - Report actual allocation size of associated object
@@ -792,7 +792,7 @@ static inline __alloc_size(1, 2) void *kvcalloc(size_t n, size_t size, gfp_t flaextern void *kvrealloc (const void *p,size_t oldsize,size_t newsize,gfp_t flags)__realloc_size (3 );extern void kvfree (const void *addr); -DEFINE_FREE(kvfree, void *, if (!IS_ERR_OR_NULL(_T)) kvfree(_T)) +DECLARE_FREE(kvfree, void *, if (!IS_ERR_OR_NULL(_T)) kvfree(_T))extern void kvfree_sensitive (const void *addr,size_t len);
include/linux/spinlock.h +19 -19
@@ -534,73 +534,73 @@ int __alloc_bucket_spinlocks(spinlock_t **locks, unsigned int *lock_mask,void free_bucket_spinlocks (spinlock_t *locks); -DEFINE_LOCK_GUARD_1 (raw_spinlock, raw_spinlock_t, +DECLARE_LOCK_GUARD_1 (raw_spinlock, raw_spinlock_t,raw_spin_lock (_T->lock), raw_spin_unlock(_T->lock)) -DEFINE_LOCK_GUARD_1_COND(raw_spinlock, _try, raw_spin_trylock(_T->lock)) +DECLARE_LOCK_GUARD_1_COND(raw_spinlock, _try, raw_spin_trylock(_T->lock)) -DEFINE_LOCK_GUARD_1 (raw_spinlock_nested, raw_spinlock_t, +DECLARE_LOCK_GUARD_1 (raw_spinlock_nested, raw_spinlock_t,raw_spin_lock_nested (_T->lock,SINGLE_DEPTH_NESTING ), raw_spin_unlock(_T->lock)) -DEFINE_LOCK_GUARD_1 (raw_spinlock_irq, raw_spinlock_t, +DECLARE_LOCK_GUARD_1 (raw_spinlock_irq, raw_spinlock_t,raw_spin_lock_irq (_T->lock), raw_spin_unlock_irq(_T->lock)) -DEFINE_LOCK_GUARD_1_COND(raw_spinlock_irq, _try, raw_spin_trylock_irq(_T->lock)) +DECLARE_LOCK_GUARD_1_COND(raw_spinlock_irq, _try, raw_spin_trylock_irq(_T->lock)) -DEFINE_LOCK_GUARD_1 (raw_spinlock_irqsave, raw_spinlock_t, +DECLARE_LOCK_GUARD_1 (raw_spinlock_irqsave, raw_spinlock_t,raw_spin_lock_irqsave (_T->lock, _T->flags),raw_spin_unlock_irqrestore (_T->lock, _T->flags), unsigned long flags) -DEFINE_LOCK_GUARD_1_COND (raw_spinlock_irqsave, _try, +DECLARE_LOCK_GUARD_1_COND (raw_spinlock_irqsave, _try, raw_spin_trylock_irqsave(_T->lock, _T->flags)) -DEFINE_LOCK_GUARD_1 (spinlock, spinlock_t, +DECLARE_LOCK_GUARD_1 (spinlock, spinlock_t,spin_lock (_T->lock), spin_unlock(_T->lock)) -DEFINE_LOCK_GUARD_1_COND(spinlock, _try, spin_trylock(_T->lock)) +DECLARE_LOCK_GUARD_1_COND(spinlock, _try, spin_trylock(_T->lock)) -DEFINE_LOCK_GUARD_1 (spinlock_irq, spinlock_t, +DECLARE_LOCK_GUARD_1 (spinlock_irq, spinlock_t,spin_lock_irq (_T->lock), spin_unlock_irq(_T->lock)) -DEFINE_LOCK_GUARD_1_COND (spinlock_irq, _try, +DECLARE_LOCK_GUARD_1_COND (spinlock_irq, _try, spin_trylock_irq(_T->lock)) -DEFINE_LOCK_GUARD_1 (spinlock_irqsave, spinlock_t, +DECLARE_LOCK_GUARD_1 (spinlock_irqsave, spinlock_t,spin_lock_irqsave (_T->lock, _T->flags),spin_unlock_irqrestore (_T->lock, _T->flags), unsigned long flags) -DEFINE_LOCK_GUARD_1_COND (spinlock_irqsave, _try, +DECLARE_LOCK_GUARD_1_COND (spinlock_irqsave, _try, spin_trylock_irqsave(_T->lock, _T->flags)) -DEFINE_LOCK_GUARD_1 (read_lock, rwlock_t, +DECLARE_LOCK_GUARD_1 (read_lock, rwlock_t,read_lock (_T->lock), read_unlock(_T->lock)) -DEFINE_LOCK_GUARD_1 (read_lock_irq, rwlock_t, +DECLARE_LOCK_GUARD_1 (read_lock_irq, rwlock_t,read_lock_irq (_T->lock), read_unlock_irq(_T->lock)) -DEFINE_LOCK_GUARD_1 (read_lock_irqsave, rwlock_t, +DECLARE_LOCK_GUARD_1 (read_lock_irqsave, rwlock_t,read_lock_irqsave (_T->lock, _T->flags),read_unlock_irqrestore (_T->lock, _T->flags), unsigned long flags) -DEFINE_LOCK_GUARD_1 (write_lock, rwlock_t, +DECLARE_LOCK_GUARD_1 (write_lock, rwlock_t,write_lock (_T->lock), write_unlock(_T->lock)) -DEFINE_LOCK_GUARD_1 (write_lock_irq, rwlock_t, +DECLARE_LOCK_GUARD_1 (write_lock_irq, rwlock_t,write_lock_irq (_T->lock), write_unlock_irq(_T->lock)) -DEFINE_LOCK_GUARD_1 (write_lock_irqsave, rwlock_t, +DECLARE_LOCK_GUARD_1 (write_lock_irqsave, rwlock_t,write_lock_irqsave (_T->lock, _T->flags),write_unlock_irqrestore (_T->lock, _T->flags), unsigned long flags)
include/linux/srcu.h +1 -1
@@ -343,7 +343,7 @@ static inline void smp_mb__after_srcu_read_unlock(void)/* __srcu_read_unlock has smp_mb() internally so nothing to do here. */ } -DEFINE_LOCK_GUARD_1(srcu, struct srcu_struct, +DECLARE_LOCK_GUARD_1(srcu, struct srcu_struct, _T->idx =srcu_read_lock (_T->lock),srcu_read_unlock (_T->lock, _T->idx), int idx)
include/sound/pcm.h +3 -3
@@ -660,13 +660,13 @@ void snd_pcm_stream_unlock_irqrestore(struct snd_pcm_substream *substream, }while (0 )/* definitions for guard(); use like guard(pcm_stream_lock) */ -DEFINE_LOCK_GUARD_1(pcm_stream_lock, struct snd_pcm_substream, +DECLARE_LOCK_GUARD_1(pcm_stream_lock, struct snd_pcm_substream,snd_pcm_stream_lock (_T->lock), snd_pcm_stream_unlock(_T->lock)) -DEFINE_LOCK_GUARD_1(pcm_stream_lock_irq, struct snd_pcm_substream, +DECLARE_LOCK_GUARD_1(pcm_stream_lock_irq, struct snd_pcm_substream,snd_pcm_stream_lock_irq (_T->lock), snd_pcm_stream_unlock_irq(_T->lock)) -DEFINE_LOCK_GUARD_1(pcm_stream_lock_irqsave, struct snd_pcm_substream, +DECLARE_LOCK_GUARD_1(pcm_stream_lock_irqsave, struct snd_pcm_substream,snd_pcm_stream_lock_irqsave (_T->lock, _T->flags),snd_pcm_stream_unlock_irqrestore (_T->lock, _T->flags), unsigned long flags)
kernel/sched/core.c +2 -2
@@ -6439,7 +6439,7 @@ static void queue_core_balance(struct rq *rq)queue_balance_callback (rq, &per_cpu (core_balance_head, rq->cpu), sched_core_balance); } -DEFINE_LOCK_GUARD_1 (core_lock, int, +DECLARE_LOCK_GUARD_1 (core_lock, int,sched_core_lock (*_T->lock, &_T->flags),sched_core_unlock (*_T->lock, &_T->flags), unsigned long flags)
@@ -7591,7 +7591,7 @@ static struct task_struct *find_get_task(pid_t pid)return p; } -DEFINE_CLASS(find_get_task, struct task_struct *, if (_T) put_task_struct(_T), +DECLARE_CLASS(find_get_task, struct task_struct *, if (_T) put_task_struct(_T), find_get_task(pid), pid_t pid)/*
kernel/sched/sched.h +8 -8
@@ -1673,7 +1673,7 @@ task_rq_unlock(struct rq *rq, struct task_struct *p, struct rq_flags *rf)raw_spin_unlock_irqrestore (&p->pi_lock, rf->flags); } -DEFINE_LOCK_GUARD_1(task_rq_lock, struct task_struct, +DECLARE_LOCK_GUARD_1(task_rq_lock, struct task_struct, _T->rq =task_rq_lock (_T->lock, &_T->rf),task_rq_unlock (_T->rq, _T->lock, &_T->rf), struct rq *rq; struct rq_flags rf)
@@ -1726,17 +1726,17 @@ rq_unlock(struct rq *rq, struct rq_flags *rf)raw_spin_rq_unlock (rq); } -DEFINE_LOCK_GUARD_1(rq_lock, struct rq, +DECLARE_LOCK_GUARD_1(rq_lock, struct rq,rq_lock (_T->lock, &_T->rf),rq_unlock (_T->lock, &_T->rf), struct rq_flags rf) -DEFINE_LOCK_GUARD_1(rq_lock_irq, struct rq, +DECLARE_LOCK_GUARD_1(rq_lock_irq, struct rq,rq_lock_irq (_T->lock, &_T->rf),rq_unlock_irq (_T->lock, &_T->rf), struct rq_flags rf) -DEFINE_LOCK_GUARD_1(rq_lock_irqsave, struct rq, +DECLARE_LOCK_GUARD_1(rq_lock_irqsave, struct rq,rq_lock_irqsave (_T->lock, &_T->rf),rq_unlock_irqrestore (_T->lock, &_T->rf), struct rq_flags rf)
@@ -2661,8 +2661,8 @@ static inline void double_rq_clock_clear_update(struct rq *rq1, struct rq *rq2)static inline void double_rq_clock_clear_update (struct rq *rq1,struct rq *rq2) {}#endif -#define DEFINE_LOCK_GUARD_2 (name, type, _lock, _unlock, ...) \ -__DEFINE_UNLOCK_GUARD(name, type, _unlock, type *lock2; __VA_ARGS__) \ +#define DECLARE_LOCK_GUARD_2 (name, type, _lock, _unlock, ...) \ +__DECLARE_UNLOCK_GUARD(name, type, _unlock, type *lock2; __VA_ARGS__) \ static inline class_##name##_t class_##name##_constructor(type *lock, type *lock2) \ { class_##name##_t _t = { .lock = lock, .lock2 = lock2 }, *_T = &_t; \ _lock; return _t; }
@@ -2802,7 +2802,7 @@ static inline void double_raw_unlock(raw_spinlock_t *l1, raw_spinlock_t *l2)raw_spin_unlock (l2); } -DEFINE_LOCK_GUARD_2 (double_raw_spinlock, raw_spinlock_t, +DECLARE_LOCK_GUARD_2 (double_raw_spinlock, raw_spinlock_t,double_raw_lock (_T->lock, _T->lock2),double_raw_unlock (_T->lock, _T->lock2))
@@ -2863,7 +2863,7 @@ static inline void double_rq_unlock(struct rq *rq1, struct rq *rq2)#endif -DEFINE_LOCK_GUARD_2(double_rq_lock, struct rq, +DECLARE_LOCK_GUARD_2(double_rq_lock, struct rq,double_rq_lock (_T->lock, _T->lock2),double_rq_unlock (_T->lock, _T->lock2))
lib/locking-selftest.c +6 -6
@@ -2507,19 +2507,19 @@ static void fs_reclaim_tests(void) }/* Defines guard classes to create contexts */ -DEFINE_LOCK_GUARD_0(HARDIRQ, HARDIRQ_ENTER(), HARDIRQ_EXIT()) -DEFINE_LOCK_GUARD_0(NOTTHREADED_HARDIRQ, +DECLARE_LOCK_GUARD_0(HARDIRQ, HARDIRQ_ENTER(), HARDIRQ_EXIT()) +DECLARE_LOCK_GUARD_0(NOTTHREADED_HARDIRQ, do { local_irq_disable(); __irq_enter(); WARN_ON(!in_irq()); } while(0), HARDIRQ_EXIT()) -DEFINE_LOCK_GUARD_0 (SOFTIRQ ,SOFTIRQ_ENTER (),SOFTIRQ_EXIT ()) +DECLARE_LOCK_GUARD_0 (SOFTIRQ ,SOFTIRQ_ENTER (),SOFTIRQ_EXIT ()) /* Define RCU guards, should go away when RCU has its own guard definitions */ -DEFINE_LOCK_GUARD_0(RCU, rcu_read_lock(), rcu_read_unlock()) -DEFINE_LOCK_GUARD_0(RCU_BH, rcu_read_lock_bh(), rcu_read_unlock_bh()) -DEFINE_LOCK_GUARD_0(RCU_SCHED, rcu_read_lock_sched(), rcu_read_unlock_sched()) +DECLARE_LOCK_GUARD_0(RCU, rcu_read_lock(), rcu_read_unlock()) +DECLARE_LOCK_GUARD_0(RCU_BH, rcu_read_lock_bh(), rcu_read_unlock_bh()) +DECLARE_LOCK_GUARD_0(RCU_SCHED, rcu_read_lock_sched(), rcu_read_unlock_sched())#define GENERATE_2_CONTEXT_TESTCASE (outer, outer_lock, inner, inner_lock) \
sound/core/control_led.c +1 -1
@@ -236,7 +236,7 @@ static void snd_ctl_led_notify(struct snd_card *card, unsigned int mask, } } -DEFINE_FREE(snd_card_unref, struct snd_card *, if (_T) snd_card_unref(_T)) +DECLARE_FREE(snd_card_unref, struct snd_card *, if (_T) snd_card_unref(_T))static int snd_ctl_led_set_id (int card_number,struct snd_ctl_elem_id *id,unsigned int group,bool set)
Signature · 1 line
2.39.2
Mathieu Desnoyers
reviewerPATCH v5 3/8 cleanup.h: Introduce DEFINE_INACTIVE_GUARD and activate_guard
In reply to Mathieu DesnoyersIn scenarios where the scope of the guard differs from the scope of its activation, introduce DEFINE_INACTIVE_GUARD() and activate_guard().
Here is an example use for a conditionally activated guard variable:
void func(bool a)
{
DEFINE_INACTIVE_GUARD(preempt_notrace, myguard);
[...]
if (a) {
might_sleep();
activate_guard(preempt_notrace, myguard)();
}
[ protected code ]
}--- include/linux/cleanup.h | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)
include/linux/cleanup.h +22 -1
@@ -146,12 +146,20 @@ static inline class_##_name##_t class_##_name##ext##_constructor(_init_args) \* similar to scoped_guard(), except it does fail when the lock * acquire fails. * +* DEFINE_INACTIVE_GUARD(name, var): +* define an inactive guard variable in a given scope, initialized to NULL. +* +* activate_guard(name, var)(args...): +* activate a guard variable with its constructor, if it is not already +* activated. */ #define DECLARE_GUARD (_name, _type, _lock, _unlock) \ DECLARE_CLASS(_name, _type, if (_T) { _unlock; }, ({ _lock; _T; }), _type _T); \ static inline void * class_##_name##_lock_ptr(class_##_name##_t *_T) \ - { return *_T; } + { return *_T; } \ + static inline class_##_name##_t class_##_name##_null(void) \ + { return NULL; }#define DECLARE_GUARD_COND (_name, _ext, _condlock) \ EXTEND_CLASS(_name, _ext, \
@@ -175,6 +183,14 @@ static inline class_##_name##_t class_##_name##ext##_constructor(_init_args) \if (!__guard_ptr (_name)(&scope)) _fail; \ else +#define DEFINE_INACTIVE_GUARD (_name, _var) \ + class_##_name##_t _var __cleanup(class_##_name##_destructor) = \ + class_##_name##_null() + +#define activate_guard (_name, _var) \ + if (!class_##_name##_lock_ptr(&(_var))) \ + _var = class_##_name##_constructor +/* * Additional helper macros for generating lock guards with types, either for * locks that don't have a native type (eg. RCU, preempt) or those that need a
@@ -209,6 +225,11 @@ static inline void class_##_name##_destructor(class_##_name##_t *_T) \ static inline void *class_##_name##_lock_ptr(class_##_name##_t *_T) \ { \return _T->lock; \ +} \ +static inline class_##_name##_t class_##_name##_null(void) \ +{ \ + class_##_name##_t _t = { .lock = NULL }; \ +return _t; \ }
Signature · 1 line
2.39.2
Mathieu Desnoyers
reviewerPATCH v5 4/8 tracing: Introduce faultable tracepoints
In reply to Mathieu DesnoyersWhen invoked from system call enter/exit instrumentation, accessing user-space data is a common use-case for tracers. However, tracepoints currently disable preemption around iteration on the registered tracepoint probes and invocation of the probe callbacks, which prevents tracers from handling page faults.
Extend the tracepoint and trace event APIs to allow defining a faultable tracepoint which invokes its callback with preemption enabled.
Also extend the tracepoint API to allow tracers to request specific probes to be connected to those faultable tracepoints. When the TRACEPOINT_MAY_FAULT flag is provided on registration, the probe callback will be called with preemption enabled, and is allowed to take page faults. Faultable probes can only be registered on faultable tracepoints and non-faultable probes on non-faultable tracepoints.
The tasks trace rcu mechanism is used to synchronize read-side marshalling of the registered probes with respect to faultable probes unregistration and teardown.
---
Changes since v1:
- Cleanup __DO_TRACE() implementation.
- Rename "sleepable tracepoints" to "faultable tracepoints", MAYSLEEP to MAYFAULT, and use might_fault() rather than might_sleep(), to properly convey that the tracepoints are meant to be able to take a page fault, which requires to be able to sleep *and* to hold the mmap_sem.
Changes since v2:
- Rename MAYFAULT to MAY_FAULT.
- Rebased on 6.5.5.
- Introduce MAY_EXIST tracepoint flag.
Changes since v3:
- Rebased on 6.6.2.
Changes since v4:
- Rebased on 6.9.6.
- Simplify flag check in tracepoint_probe_register_prio_flags().
- Update MAY_EXIST flag description.
--- include/linux/tracepoint-defs.h | 14 ++++++ include/linux/tracepoint.h | 88 +++++++++++++++++++++++---------- include/trace/define_trace.h | 7 +++ include/trace/trace_events.h | 6 +++ init/Kconfig | 1 + kernel/trace/bpf_trace.c | 5 +- kernel/trace/trace_fprobe.c | 5 +- kernel/tracepoint.c | 65 ++++++++++++++---------- 8 files changed, 136 insertions(+), 55 deletions(-)
include/linux/tracepoint-defs.h +14 -0
@@ -29,6 +29,19 @@ struct tracepoint_func {int prio; }; +/** +* enum tracepoint_flags - Tracepoint flags +* @TRACEPOINT_MAY_EXIST: On registration, don't warn if the tracepoint is +* already registered. +* @TRACEPOINT_MAY_FAULT: The tracepoint probe callback will be called with +* preemption enabled, and is allowed to take page +* faults. +*/ +enum tracepoint_flags { +TRACEPOINT_MAY_EXIST = (1 <<0 ), +TRACEPOINT_MAY_FAULT = (1 <<1 ), +}; +struct tracepoint {const char *name;/* Tracepoint name */ struct static_key key;
@@ -39,6 +52,7 @@ struct tracepoint {int (*regfunc)(void );void (*unregfunc)(void );struct tracepoint_func __rcu *funcs; +unsigned int flags; };#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
include/linux/tracepoint.h +63 -25
215 lines not on this page · raw message
include/trace/define_trace.h +7 -0
@@ -41,6 +41,12 @@ assign, print, reg, unreg) \ DEFINE_TRACE_FN(name, reg, unreg, PARAMS(proto), PARAMS(args)) +#undef TRACE_EVENT_FN_MAY_FAULT +#define TRACE_EVENT_FN_MAY_FAULT (name, proto, args, tstruct, \ + assign, print, reg, unreg) \ + DEFINE_TRACE_FN_FLAGS(name, reg, unreg, PARAMS(proto), \ + PARAMS(args), TRACEPOINT_MAY_FAULT) +#undef TRACE_EVENT_FN_COND#define TRACE_EVENT_FN_COND (name, proto, args, cond, tstruct, \ assign, print, reg, unreg) \
@@ -106,6 +112,7 @@#undef TRACE_EVENT#undef TRACE_EVENT_FN +#undef TRACE_EVENT_FN_MAY_FAULT#undef TRACE_EVENT_FN_COND#undef TRACE_EVENT_CONDITION#undef TRACE_EVENT_NOP
include/trace/trace_events.h +6 -0
@@ -77,6 +77,12 @@TRACE_EVENT (name,PARAMS (proto),PARAMS (args), \ PARAMS(tstruct), PARAMS(assign), PARAMS(print)) \ +#undef TRACE_EVENT_FN_MAY_FAULT +#define TRACE_EVENT_FN_MAY_FAULT (name, proto, args, tstruct, \ + assign, print, reg, unreg) \ + TRACE_EVENT(name, PARAMS(proto), PARAMS(args), \ + PARAMS(tstruct), PARAMS(assign), PARAMS(print)) \ +#undef TRACE_EVENT_FN_COND#define TRACE_EVENT_FN_COND (name, proto, args, cond, tstruct, \ assign, print, reg, unreg) \
init/Kconfig +1 -0
@@ -1933,6 +1933,7 @@ config BINDGEN_VERSION_TEXT # config TRACEPOINTS bool + select TASKS_TRACE_RCU source "kernel/Kconfig.kexec"
kernel/trace/bpf_trace.c +3 -2
@@ -2443,8 +2443,9 @@ static int __bpf_probe_register(struct bpf_raw_event_map *btp, struct bpf_prog *if (prog->aux->max_tp_access > btp->writable_size)return -EINVAL ; -return tracepoint_probe_register_may_exist (tp, (void *)btp->bpf_func, - prog); +return tracepoint_probe_register_prio_flags (tp, (void *)btp->bpf_func, + prog,TRACEPOINT_DEFAULT_PRIO , +TRACEPOINT_MAY_EXIST ); }int bpf_probe_register (struct bpf_raw_event_map *btp,struct bpf_prog *prog)
kernel/trace/trace_fprobe.c +3 -2
@@ -705,8 +705,9 @@ static int __register_trace_fprobe(struct trace_fprobe *tf)* At first, put __probestub_##TP function on the tracepoint * and put a fprobe on the stub function. */ - ret =tracepoint_probe_register_prio_may_exist (tpoint, - tpoint->probestub,NULL ,0 ); + ret =tracepoint_probe_register_prio_flags (tpoint, + tpoint->probestub,NULL ,0 , +TRACEPOINT_MAY_EXIST );if (ret <0 )return ret;return register_fprobe_ips (&tf->fp, &ip,1 );
kernel/tracepoint.c +39 -26
134 lines not on this page · raw message
Signature · 1 line
2.39.2
Mathieu Desnoyers
reviewerPATCH v5 5/8 tracing/ftrace: Add support for faultable tracepoints
In reply to Mathieu DesnoyersIn preparation for converting system call enter/exit instrumentation into faultable tracepoints, make sure that ftrace can handle registering to such tracepoints by explicitly disabling preemption within the ftrace tracepoint probes to respect the current expectations within ftrace ring buffer code.
This change does not yet allow ftrace to take page faults per se within its probe, but allows its existing probes to connect to faultable tracepoints.
---
Changes since v4:
- Use DEFINE_INACTIVE_GUARD.
- Add brackets to multiline 'if' statements.
--- include/trace/trace_events.h | 64 ++++++++++++++++++++++++++++++++++-- kernel/trace/trace_events.c | 28 ++++++++++++---- 2 files changed, 83 insertions(+), 9 deletions(-)
include/trace/trace_events.h +61 -3
@@ -45,6 +45,16 @@ PARAMS(print)); \DEFINE_EVENT (name, name,PARAMS (proto),PARAMS (args)); +#undef TRACE_EVENT_MAY_FAULT +#define TRACE_EVENT_MAY_FAULT (name, proto, args, tstruct, assign, print) \ + DECLARE_EVENT_CLASS_MAY_FAULT(name, \ + PARAMS(proto), \ + PARAMS(args), \ + PARAMS(tstruct), \ + PARAMS(assign), \ + PARAMS(print)); \ + DEFINE_EVENT(name, name, PARAMS(proto), PARAMS(args)); +#include "stages/stage1_struct_define.h" #undef DECLARE_EVENT_CLASS
@@ -57,6 +67,11 @@ \ static struct trace_event_class event_class_##name; +#undef DECLARE_EVENT_CLASS_MAY_FAULT +#define DECLARE_EVENT_CLASS_MAY_FAULT (name, proto, args, tstruct, assign, print) \ + DECLARE_EVENT_CLASS(name, PARAMS(proto), PARAMS(args), \ + PARAMS(tstruct), PARAMS(assign), PARAMS(print)) +#undef DEFINE_EVENT#define DEFINE_EVENT (template, name, proto, args) \ static struct trace_event_call __used \
@@ -80,7 +95,7 @@#undef TRACE_EVENT_FN_MAY_FAULT#define TRACE_EVENT_FN_MAY_FAULT (name, proto, args, tstruct, \ assign, print, reg, unreg) \ - TRACE_EVENT(name, PARAMS(proto), PARAMS(args), \ + TRACE_EVENT_MAY_FAULT(name, PARAMS(proto), PARAMS(args), \ PARAMS(tstruct), PARAMS(assign), PARAMS(print)) \#undef TRACE_EVENT_FN_COND
@@ -123,6 +138,11 @@ tstruct; \ }; +#undef DECLARE_EVENT_CLASS_MAY_FAULT +#define DECLARE_EVENT_CLASS_MAY_FAULT (call, proto, args, tstruct, assign, print) \ + DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args), \ + PARAMS(tstruct), PARAMS(assign), PARAMS(print)) +#undef DEFINE_EVENT#define DEFINE_EVENT (template, name, proto, args)
@@ -214,6 +234,11 @@ static struct trace_event_functions trace_event_type_funcs_##call = { \ .trace = trace_raw_output_##call, \ }; +#undef DECLARE_EVENT_CLASS_MAY_FAULT +#define DECLARE_EVENT_CLASS_MAY_FAULT (call, proto, args, tstruct, assign, print) \ + DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args), \ + PARAMS(tstruct), PARAMS(assign), PARAMS(print)) +#undef DEFINE_EVENT_PRINT#define DEFINE_EVENT_PRINT (template, call, proto, args, print) \ static notrace enum print_line_t \
@@ -250,6 +275,11 @@ static struct trace_event_fields trace_event_fields_##call[] = { \ tstruct \ {} }; +#undef DECLARE_EVENT_CLASS_MAY_FAULT +#define DECLARE_EVENT_CLASS_MAY_FAULT (call, proto, args, tstruct, assign, print) \ + DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args), \ + PARAMS(tstruct), PARAMS(assign), PARAMS(print)) +#undef DEFINE_EVENT_PRINT#define DEFINE_EVENT_PRINT (template, name, proto, args, print)
@@ -271,6 +301,11 @@ static inline notrace int trace_event_get_offsets_##call( \return __data_size; \ } +#undef DECLARE_EVENT_CLASS_MAY_FAULT +#define DECLARE_EVENT_CLASS_MAY_FAULT (call, proto, args, tstruct, assign, print) \ + DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args), \ + PARAMS(tstruct), PARAMS(assign), PARAMS(print)) +#include TRACE_INCLUDE (TRACE_INCLUDE_FILE )/*
@@ -380,8 +415,8 @@ static inline notrace int trace_event_get_offsets_##call( \#include "stages/stage6_event_callback.h" -#undef DECLARE_EVENT_CLASS -#define DECLARE_EVENT_CLASS (call, proto, args, tstruct, assign, print) \ +#undef _DECLARE_EVENT_CLASS +#define _DECLARE_EVENT_CLASS (call, proto, args, tstruct, assign, print, tp_flags) \ \ static notrace void \ trace_event_raw_event_##call(void *__data, proto) \
@@ -392,6 +427,13 @@ trace_event_raw_event_##call(void *__data, proto) \ struct trace_event_raw_##call *entry; \ int __data_size; \ \ + DEFINE_INACTIVE_GUARD(preempt_notrace, trace_event_guard); \ + \ + if ((tp_flags) & TRACEPOINT_MAY_FAULT) { \ + might_fault(); \ + activate_guard(preempt_notrace, trace_event_guard)(); \ + } \ + \ if (trace_trigger_soft_disabled(trace_file)) \ return; \ \
@@ -409,6 +451,17 @@ trace_event_raw_event_##call(void *__data, proto) \ \trace_event_buffer_commit (&fbuffer); \ } + +#undef DECLARE_EVENT_CLASS +#define DECLARE_EVENT_CLASS (call, proto, args, tstruct, assign, print) \ + _DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args), \ + PARAMS(tstruct), PARAMS(assign), PARAMS(print), 0) + +#undef DECLARE_EVENT_CLASS_MAY_FAULT +#define DECLARE_EVENT_CLASS_MAY_FAULT (call, proto, args, tstruct, assign, print) \ + _DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args), \ + PARAMS(tstruct), PARAMS(assign), PARAMS(print), TRACEPOINT_MAY_FAULT) +/* * The ftrace_test_probe is compiled out, it is only here as a build time check * to make sure that if the tracepoint handling changes, the ftrace probe will
@@ -440,6 +493,11 @@ static struct trace_event_class __used __refdata event_class_##call = { \ _TRACE_PERF_INIT(call) \ }; +#undef DECLARE_EVENT_CLASS_MAY_FAULT +#define DECLARE_EVENT_CLASS_MAY_FAULT (call, proto, args, tstruct, assign, print) \ + DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args), \ + PARAMS(tstruct), PARAMS(assign), PARAMS(print)) +#undef DEFINE_EVENT#define DEFINE_EVENT (template, call, proto, args) \ \
kernel/trace/trace_events.c +22 -6
@@ -532,9 +532,17 @@ int trace_event_reg(struct trace_event_call *call,WARN_ON (!(call->flags &TRACE_EVENT_FL_TRACEPOINT ));switch (type) {case TRACE_REG_REGISTER : -return tracepoint_probe_register (call->tp, - call->class->probe, - file); +if (call->tp->flags &TRACEPOINT_MAY_FAULT ) { +return tracepoint_probe_register_prio_flags (call->tp, + call->class->probe, + file, +TRACEPOINT_DEFAULT_PRIO , +TRACEPOINT_MAY_FAULT ); + }else { +return tracepoint_probe_register (call->tp, + call->class->probe, + file); + }case TRACE_REG_UNREGISTER : tracepoint_probe_unregister(call->tp, call->class->probe,
@@ -543,9 +551,17 @@ int trace_event_reg(struct trace_event_call *call,#ifdef CONFIG_PERF_EVENTS case TRACE_REG_PERF_REGISTER : -return tracepoint_probe_register (call->tp, - call->class->perf_probe, - call); +if (call->tp->flags &TRACEPOINT_MAY_FAULT ) { +return tracepoint_probe_register_prio_flags (call->tp, + call->class->perf_probe, + call, +TRACEPOINT_DEFAULT_PRIO , +TRACEPOINT_MAY_FAULT ); + }else { +return tracepoint_probe_register (call->tp, + call->class->perf_probe, + call); + }case TRACE_REG_PERF_UNREGISTER : tracepoint_probe_unregister(call->tp, call->class->perf_probe,
Signature · 1 line
2.39.2
Mathieu Desnoyers
reviewerPATCH v5 6/8 tracing/bpf-trace: Add support for faultable tracepoints
In reply to Mathieu DesnoyersIn preparation for converting system call enter/exit instrumentation into faultable tracepoints, make sure that bpf can handle registering to such tracepoints by explicitly disabling preemption within the bpf tracepoint probes to respect the current expectations within bpf tracing code.
This change does not yet allow bpf to take page faults per se within its probe, but allows its existing probes to connect to faultable tracepoints.
---
Changes since v4:
- Use DEFINE_INACTIVE_GUARD.
- Add brackets to multiline 'if' statements.
--- include/trace/bpf_probe.h | 20 ++++++++++++++++---- kernel/trace/bpf_trace.c | 12 +++++++++--- 2 files changed, 25 insertions(+), 7 deletions(-)
include/trace/bpf_probe.h +16 -4
@@ -42,17 +42,29 @@/* tracepoints with more than 12 arguments will hit build error */ #define CAST_TO_U64 (...) CONCATENATE(__CAST, COUNT_ARGS(__VA_ARGS__))(__VA_ARGS__) -#define __BPF_DECLARE_TRACE (call, proto, args) \ +#define __BPF_DECLARE_TRACE (call, proto, args, tp_flags) \ static notrace void \ __bpf_trace_##call(void *__data, proto) \ { \ struct bpf_prog *prog = __data; \ + \ + DEFINE_INACTIVE_GUARD(preempt_notrace, bpf_trace_guard); \ + \ + if ((tp_flags) & TRACEPOINT_MAY_FAULT) { \ + might_fault(); \ + activate_guard(preempt_notrace, bpf_trace_guard)(); \ + } \ + \ CONCATENATE(bpf_trace_run, COUNT_ARGS(args))(prog, CAST_TO_U64(args)); \ }#undef DECLARE_EVENT_CLASS#define DECLARE_EVENT_CLASS (call, proto, args, tstruct, assign, print) \ - __BPF_DECLARE_TRACE(call, PARAMS(proto), PARAMS(args)) + __BPF_DECLARE_TRACE(call, PARAMS(proto), PARAMS(args), 0) + +#undef DECLARE_EVENT_CLASS_MAY_FAULT +#define DECLARE_EVENT_CLASS_MAY_FAULT (call, proto, args, tstruct, assign, print) \ + __BPF_DECLARE_TRACE(call, PARAMS(proto), PARAMS(args), TRACEPOINT_MAY_FAULT)/* * This part is compiled out, it is only here as a build time check
@@ -106,13 +118,13 @@ static inline void bpf_test_buffer_##call(void) \#undef DECLARE_TRACE#define DECLARE_TRACE (call, proto, args) \ - __BPF_DECLARE_TRACE(call, PARAMS(proto), PARAMS(args)) \ + __BPF_DECLARE_TRACE(call, PARAMS(proto), PARAMS(args), 0) \ __DEFINE_EVENT(call, call, PARAMS(proto), PARAMS(args), 0)#undef DECLARE_TRACE_WRITABLE#define DECLARE_TRACE_WRITABLE (call, proto, args, size) \ __CHECK_WRITABLE_BUF_SIZE(call, PARAMS(proto), PARAMS(args), size) \ - __BPF_DECLARE_TRACE(call, PARAMS(proto), PARAMS(args)) \ + __BPF_DECLARE_TRACE(call, PARAMS(proto), PARAMS(args), 0) \ __DEFINE_EVENT(call, call, PARAMS(proto), PARAMS(args), size)#include TRACE_INCLUDE (TRACE_INCLUDE_FILE )
kernel/trace/bpf_trace.c +9 -3
@@ -2443,9 +2443,15 @@ static int __bpf_probe_register(struct bpf_raw_event_map *btp, struct bpf_prog *if (prog->aux->max_tp_access > btp->writable_size)return -EINVAL ; -return tracepoint_probe_register_prio_flags (tp, (void *)btp->bpf_func, - prog,TRACEPOINT_DEFAULT_PRIO , -TRACEPOINT_MAY_EXIST ); +if (tp->flags &TRACEPOINT_MAY_FAULT ) { +return tracepoint_probe_register_prio_flags (tp, (void *)btp->bpf_func, + prog,TRACEPOINT_DEFAULT_PRIO , +TRACEPOINT_MAY_EXIST |TRACEPOINT_MAY_FAULT ); + }else { +return tracepoint_probe_register_prio_flags (tp, (void *)btp->bpf_func, + prog,TRACEPOINT_DEFAULT_PRIO , +TRACEPOINT_MAY_EXIST ); + } }int bpf_probe_register (struct bpf_raw_event_map *btp,struct bpf_prog *prog)
Signature · 1 line
2.39.2
Re: [PATCH v5 6/8] tracing/bpf-trace: Add support for faultable tracepoints
Quoted 6 lines · On Wed, 26 Jun 2024 14:59:39 -0400 Mathieu Desnoyers…
kernel/trace/bpf_trace.c +0
-2@@ -2443,9 +2443,15 @@ static int __bpf_probe_register(struct bpf_raw_event_map *btp, struct bpf_prog *if (prog->aux->max_tp_access > btp->writable_size)return -EINVAL ; - return tracepoint_probe_register_prio_flags(tp, (void *)btp->bpf_func, - prog, TRACEPOINT_DEFAULT_PRIO,
kernel/trace/bpf_trace.c static int __bpf_probe_register(struct bpf_raw_event_map *btp, struct bpf_prog * · line 2454
Quoted 10 lines
kernel/trace/bpf_trace.c +9
-1- TRACEPOINT_MAY_EXIST); +if (tp->flags &TRACEPOINT_MAY_FAULT ) { +return tracepoint_probe_register_prio_flags (tp, (void *)btp->bpf_func, + prog,TRACEPOINT_DEFAULT_PRIO , +TRACEPOINT_MAY_EXIST |TRACEPOINT_MAY_FAULT ); + }else { +return tracepoint_probe_register_prio_flags (tp, (void *)btp->bpf_func, + prog,TRACEPOINT_DEFAULT_PRIO , +TRACEPOINT_MAY_EXIST ); + }
nit: you can also just pass the flag directly,
return tracepoint_probe_register_prio_flags(tp, (void *)btp->bpf_func, prog, TRACEPOINT_DEFAULT_PRIO, TRACEPOINT_MAY_EXIST | (tp->flags & TRACEPOINT_MAY_FAULT));
But others looks good to me.
Thank you,
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
Mathieu Desnoyers
reviewerRe: [PATCH v5 6/8] tracing/bpf-trace: Add support for faultable tracepoints
Quoted 22 lines · Masami Hiramatsu (Google)
Quoted 6 lines · On Wed, 26 Jun 2024 14:59:39 -0400 Mathieu Desnoyers…
kernel/trace/bpf_trace.c +0
-2@@ -2443,9 +2443,15 @@ static int __bpf_probe_register(struct bpf_raw_event_map *btp, struct bpf_prog *if (prog->aux->max_tp_access > btp->writable_size)return -EINVAL ; - return tracepoint_probe_register_prio_flags(tp, (void *)btp->bpf_func, - prog, TRACEPOINT_DEFAULT_PRIO,kernel/trace/bpf_trace.c static int __bpf_probe_register(struct bpf_raw_event_map *btp, struct bpf_prog * · line 2454
Quoted 10 lines
kernel/trace/bpf_trace.c +9
-1- TRACEPOINT_MAY_EXIST); +if (tp->flags &TRACEPOINT_MAY_FAULT ) { +return tracepoint_probe_register_prio_flags (tp, (void *)btp->bpf_func, + prog,TRACEPOINT_DEFAULT_PRIO , +TRACEPOINT_MAY_EXIST |TRACEPOINT_MAY_FAULT ); + }else { +return tracepoint_probe_register_prio_flags (tp, (void *)btp->bpf_func, + prog,TRACEPOINT_DEFAULT_PRIO , +TRACEPOINT_MAY_EXIST ); + }nit: you can also just pass the flag directly,
Quoted 4 lines
return tracepoint_probe_register_prio_flags(tp, (void *)btp->bpf_func, prog, TRACEPOINT_DEFAULT_PRIO, TRACEPOINT_MAY_EXIST | (tp->flags & TRACEPOINT_MAY_FAULT));
I'll do that for both the ftrace and the bpf patches and eliminate this odd duplication. That's a good idea.
I'll add your Reviewed-by to all the "tracing/*" patches.
Thanks,
Mathieu
Quoted 6 lines
But others looks good to me.
Reviewed-by Masami Hiramatsu (Google) <[email protected]>Thank you,
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com
Mathieu Desnoyers
reviewerPATCH v5 7/8 tracing/perf: Add support for faultable tracepoints
In reply to Mathieu DesnoyersIn preparation for converting system call enter/exit instrumentation into faultable tracepoints, make sure that perf can handle registering to such tracepoints by explicitly disabling preemption within the perf tracepoint probes to respect the current expectations within perf ring buffer code.
This change does not yet allow perf to take page faults per se within its probe, but allows its existing probes to connect to faultable tracepoints.
---
Changes since v4:
- Use DEFINE_INACTIVE_GUARD.
--- include/trace/perf.h | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-)
include/trace/perf.h +20 -2
@@ -12,8 +12,8 @@#undef __perf_task#define __perf_task (t) (__task = (t)) -#undef DECLARE_EVENT_CLASS -#define DECLARE_EVENT_CLASS (call, proto, args, tstruct, assign, print) \ +#undef _DECLARE_EVENT_CLASS +#define _DECLARE_EVENT_CLASS (call, proto, args, tstruct, assign, print, tp_flags) \ static notrace void \ perf_trace_##call(void *__data, proto) \ { \
@@ -28,6 +28,13 @@ perf_trace_##call(void *__data, proto) \int __data_size; \int rctx; \ \ +DEFINE_INACTIVE_GUARD (preempt_notrace, trace_event_guard); \ + \ +if ((tp_flags) &TRACEPOINT_MAY_FAULT ) { \ +might_fault (); \ +activate_guard (preempt_notrace, trace_event_guard)(); \ + } \ + \ __data_size = trace_event_get_offsets_##call(&__data_offsets, args); \ \ head = this_cpu_ptr(event_call->perf_events); \
@@ -55,6 +62,17 @@ perf_trace_##call(void *__data, proto) \ head, __task); \ } +#undef DECLARE_EVENT_CLASS +#define DECLARE_EVENT_CLASS (call, proto, args, tstruct, assign, print) \ + _DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args), \ + PARAMS(tstruct), PARAMS(assign), PARAMS(print), 0) + +#undef DECLARE_EVENT_CLASS_MAY_FAULT +#define DECLARE_EVENT_CLASS_MAY_FAULT (call, proto, args, tstruct, assign, print) \ + _DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args), \ + PARAMS(tstruct), PARAMS(assign), PARAMS(print), \ + TRACEPOINT_MAY_FAULT) +/* * This part is compiled out, it is only here as a build time check * to make sure that if the tracepoint handling changes, the
Signature · 1 line
2.39.2
Mathieu Desnoyers
reviewerPATCH v5 8/8 tracing: Convert sys_enter/exit to faultable tracepoints
In reply to Mathieu DesnoyersConvert the definition of the system call enter/exit tracepoints to faultable tracepoints now that all upstream tracers handle it.
This allows tracers to fault-in userspace system call arguments such as path strings within their probe callbacks.
---
Since v4:
- Use 'guard(preempt_notrace)'.
- Add brackets to multiline 'if' statements.
--- include/trace/events/syscalls.h | 4 +-- kernel/trace/trace_syscalls.c | 52 ++++++++++++++++++++++++++++----- 2 files changed, 46 insertions(+), 10 deletions(-)
include/trace/events/syscalls.h +2 -2
@@ -15,7 +15,7 @@#ifdef CONFIG_HAVE_SYSCALL_TRACEPOINTS -TRACE_EVENT_FN(sys_enter, +TRACE_EVENT_FN_MAY_FAULT(sys_enter, TP_PROTO(struct pt_regs *regs, long id),
@@ -41,7 +41,7 @@ TRACE_EVENT_FN(sys_enter, TRACE_EVENT_FLAGS(sys_enter, TRACE_EVENT_FL_CAP_ANY) -TRACE_EVENT_FN(sys_exit, +TRACE_EVENT_FN_MAY_FAULT(sys_exit, TP_PROTO(struct pt_regs *regs, long ret),
kernel/trace/trace_syscalls.c +44 -8
100 lines not on this page · raw message
Signature · 1 line
2.39.2
Steven Rostedt
maintainerQuoted 7 lines · On Wed, 26 Jun 2024 14:59:33 -0400 Mathieu Desnoyers…
Wire up the system call tracepoints with Tasks Trace RCU to allow the ftrace, perf, and eBPF tracers to handle page faults.
This series does the initial wire-up allowing tracers to handle page faults, but leaves out the actual handling of said page faults as future work.
Quoted 5 lines
I have tested this against a feature branch of lttng-modules which implements handling of page faults for the filename argument of the openat(2) system call.
This v5 addresses comments from the previous round of review [1].
Hi Mathieu,
Can you resend this and Cc [email protected]?
That would put it into our patchwork and makes it work with our workflow.
https://patchwork.kernel.org/project/linux-trace-kernel/list/
Thanks,
-- Steve
Quoted 17 lines
Steven Rostedt suggested separating tracepoints into two separate sections. It is unclear how that approach would prove to be an improvement over the currently proposed approach, so those changes were not incorporated. See [2] for my detailed reply.
In the previous round, Peter Zijlstra suggested use of SRCU rather than Tasks Trace RCU. See my reply about the distinction between SRCU and Tasks Trace RCU [3] and this explanation from Paul E. McKenney about the purpose of Tasks Trace RCU [4].
The macros
DEFINE_INACTIVE_GUARDandactivate_guardare added tocleanup.hfor use in the__DO_TRACE()macro. Those appear to be more flexible than theguard_if()proposed by Peter Zijlstra in the previous round of review [5].This series is based on kernel v6.9.6.
Mathieu Desnoyers
reviewerQuoted 14 lines · Steven Rostedt
Quoted 11 lines · On Wed, 26 Jun 2024 14:59:33 -0400 Mathieu Desnoyers…
Wire up the system call tracepoints with Tasks Trace RCU to allow the ftrace, perf, and eBPF tracers to handle page faults.
This series does the initial wire-up allowing tracers to handle page faults, but leaves out the actual handling of said page faults as future work.
I have tested this against a feature branch of lttng-modules which implements handling of page faults for the filename argument of the openat(2) system call.
Quoted 5 lines
Quoted 1 line
This v5 addresses comments from the previous round of review [1].
Hi Mathieu,
Can you resend this and Cc [email protected]?
Sure, will do for v6. Thanks!
Mathieu
Quoted 27 lines
That would put it into our patchwork and makes it work with our workflow.
https://patchwork.kernel.org/project/linux-trace-kernel/list/
Thanks,
-- Steve
Quoted 17 lines
Steven Rostedt suggested separating tracepoints into two separate sections. It is unclear how that approach would prove to be an improvement over the currently proposed approach, so those changes were not incorporated. See [2] for my detailed reply.
In the previous round, Peter Zijlstra suggested use of SRCU rather than Tasks Trace RCU. See my reply about the distinction between SRCU and Tasks Trace RCU [3] and this explanation from Paul E. McKenney about the purpose of Tasks Trace RCU [4].
The macros
DEFINE_INACTIVE_GUARDandactivate_guardare added tocleanup.hfor use in the__DO_TRACE()macro. Those appear to be more flexible than theguard_if()proposed by Peter Zijlstra in the previous round of review [5].This series is based on kernel v6.9.6.
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com