Skip to contentSkip to conversation

tracing/user_events: Remote write ABI

PATCH Superseded Beau Belgrave 2 participants 19 messages Dec 5, 2022

Subsystems (7)
Documentation VFS Kselftest MM core +3 more subsystems

v5 of 10 Superseded Latest is v10 →

lore
Share & tools
lore
#0 lore raw

v5 cover tracing/user_events: Remote write ABI

As part of the discussions for user_events aligned with user space tracers, it was determined that user programs should register a aligned value to set or clear a bit when an event becomes enabled. Currently a shared page is being used that requires mmap(). Remove the shared page implementation and move to a user registered address implementation.

In this new model during the event registration from user programs 3 new values are specified. The first is the address to update when the event is either enabled or disabled. The second is the bit to set/clear to reflect the event being enabled. The third is the size of the value at the specified address.

This allows for a local 32/64-bit value in user programs to support both kernel and user tracers. As an example, setting bit 31 for kernel tracers when the event becomes enabled allows for user tracers to use the other bits for ref counts or other flags. The kernel side updates the bit atomically, user programs need to also update these values atomically.

User provided addresses must be aligned on a natural boundary, this allows for single page checking and prevents odd behaviors such as a enable value straddling 2 pages instead of a single page.

When page faults are encountered they are done asyncly via a workqueue. If the page faults back in, the write update is attempted again. If the page cannot fault-in, then we log and wait until the next time the event is enabled/disabled. This is to prevent possible infinite loops resulting from bad user processes unmapping or changing protection values after registering the address.

Change history

V5:
GFP_NOWAIT is still needed in user_event_enabler_dup(), due to rcu lock.

V4:
Rebase to 6.1-rc7.

Moved user_events_fork() out of task signal lock and dropped use of GFP_NOWAIT. All allocations are now GFP_KERNEL or GFP_KERNEL_ACCOUNT.

Added boot parameter user_events_max= to limit global events.

Added sysctl value kernel.user_events_max to limit global events.

Added cgroup tracking of memory allocated for events.

V3:
Rebase to 6.1-rc6.

Removed RFC tag on series.

Updated documentation to reflect ABI changes.

Added self-test for ABI specific clone/fork cases.

Moved user_event_mm removal into do_exit() to ensure RSS task accounting is done properly in async fault paths. Also lets us remove the delayed mmdrop(), saving memory in each user_event_mm struct.

Fixed timing window where task exits, but write could be in-progress. During exit we now take mmap_write_lock to ensure we drain writes.

V2:
Rebase to 6.1-rc5.

Added various comments based on feedback.

Added enable_size to register struct, allows 32/64 bit addresses as long as the enable_bit fits and the address is naturally aligned.

Changed user_event_enabler_write to accept a new flag indicating if a fault fixup should be done or not. This allows user_event_enabler_create to return back failures to the user ioctl reg call and retry to fault in data.

Added tracking fork/exec/exit of tasks to have the user_event_mm lifetime tied more to the task than the file. This came with extra requirements around when you can lock, such as softirq cases, as well as a RCU pattern to ensure fork/exec/exit take minimal lock times.

Changed enablers to use a single word-aligned value for saving the bit to set and any flags, such as faulting asyncly or being freed. This was required to ensure atomic bit set/test for fork cases where taking the event_mutex is not a good scalability decision.

Added unregister IOCTL, since file lifetime no longer limits the enable time for any events (the mm does).

Updated sample code to reflect the new remote write based ABI.

Updated self-test code to reflect the new remote write based ABI.

Beau Belgrave (11):
tracing/user_events: Split header into uapi and kernel tracing/user_events: Track fork/exec/exit for mm lifetime tracing/user_events: Use remote writes for event enablement tracing/user_events: Fixup enable faults asyncly tracing/user_events: Add ioctl for disabling addresses tracing/user_events: Update self-tests to write ABI tracing/user_events: Add ABI self-test tracing/user_events: Use write ABI in example tracing/user_events: Update documentation for ABI tracing/user_events: Charge event allocs to cgroups tracing/user_events: Limit global user_event count

 Documentation/trace/user_events.rst           | 172 ++--
 fs/exec.c                                     |   2 +
 include/linux/sched.h                         |   5 +
 include/linux/user_events.h                   |  99 +-
 include/uapi/linux/user_events.h              |  81 ++
 kernel/exit.c                                 |   2 +
 kernel/fork.c                                 |   2 +
 kernel/trace/Kconfig                          |   5 +-
 kernel/trace/trace_events_user.c              | 847 +++++++++++++++---
 samples/user_events/example.c                 |  47 +-
 tools/testing/selftests/user_events/Makefile  |   2 +-
 .../testing/selftests/user_events/abi_test.c  | 228 +++++
 .../testing/selftests/user_events/dyn_test.c  |   2 +-
 .../selftests/user_events/ftrace_test.c       | 162 ++--
 .../testing/selftests/user_events/perf_test.c |  39 +-
 15 files changed, 1296 insertions(+), 399 deletions(-)
 create mode 100644 include/uapi/linux/user_events.h
 create mode 100644 tools/testing/selftests/user_events/abi_test.c

base-commit: b7b275e60bcd5f89771e865a8239325f86d9927d

Signature · 1 line
2.25.1
#1 lore raw

PATCH v5 1/11 tracing/user_events: Split header into uapi and kernel

The UAPI parts need to be split out from the kernel parts of user_events now that other parts of the kernel will reference it. Do so by moving the existing include/linux/user_events.h into include/uapi/linux/user_events.h.

---
 include/linux/user_events.h      | 52 +++++---------------------------
 include/uapi/linux/user_events.h | 48 +++++++++++++++++++++++++++++
 kernel/trace/trace_events_user.c |  5 ---
 3 files changed, 56 insertions(+), 49 deletions(-)
 create mode 100644 include/uapi/linux/user_events.h
include/linux/user_events.h +8 -44
index 592a3fbed98e..036b360f3d97 100644
@@ -1,54 +1,18 @@
-/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/* SPDX-License-Identifier: GPL-2.0-only */
 /*
- * Copyright (c) 2021, Microsoft Corporation.
+ * Copyright (c) 2022, Microsoft Corporation.
  *
  * Authors:
  *   Beau Belgrave <[email protected]>
  */
-#ifndef _UAPI_LINUX_USER_EVENTS_H
-#define _UAPI_LINUX_USER_EVENTS_H
 
-#include <linux/types.h>
-#include <linux/ioctl.h>
+#ifndef _LINUX_USER_EVENTS_H
+#define _LINUX_USER_EVENTS_H
 
-#ifdef __KERNEL__
-#include <linux/uio.h>
+#include <uapi/linux/user_events.h>
+
+#ifdef CONFIG_USER_EVENTS
 #else
-#include <sys/uio.h>
 #endif
 
-#define USER_EVENTS_SYSTEM "user_events"
-#define USER_EVENTS_PREFIX "u:"
-
-/* Create dynamic location entry within a 32-bit value */
-#define DYN_LOC(offset, size) ((size) << 16 | (offset))
-
-/*
- * Describes an event registration and stores the results of the registration.
- * This structure is passed to the DIAG_IOCSREG ioctl, callers at a minimum
- * must set the size and name_args before invocation.
- */
-struct user_reg {
-
-	/* Input: Size of the user_reg structure being used */
-	__u32 size;
-
-	/* Input: Pointer to string with event name, description and flags */
-	__u64 name_args;
-
-	/* Output: Bitwise index of the event within the status page */
-	__u32 status_bit;
-
-	/* Output: Index of the event to use when writing data */
-	__u32 write_index;
-} __attribute__((__packed__));
-
-#define DIAG_IOC_MAGIC '*'
-
-/* Requests to register a user_event */
-#define DIAG_IOCSREG _IOWR(DIAG_IOC_MAGIC, 0, struct user_reg*)
-
-/* Requests to delete a user_event */
-#define DIAG_IOCSDEL _IOW(DIAG_IOC_MAGIC, 1, char*)
-
-#endif /* _UAPI_LINUX_USER_EVENTS_H */
+#endif /* _LINUX_USER_EVENTS_H */
include/uapi/linux/user_events.h new file +48 -0

48 lines not on this page · raw message

kernel/trace/trace_events_user.c +0 -5
index ae78c2d53c8a..890357b48c37 100644
@@ -19,12 +19,7 @@
 #include <linux/tracefs.h>
 #include <linux/types.h>
 #include <linux/uaccess.h>
-/* Reminder to move to uapi when everything works */
-#ifdef CONFIG_COMPILE_TEST
 #include <linux/user_events.h>
-#else
-#include <uapi/linux/user_events.h>
-#endif
 #include "trace.h"
 #include "trace_dynevent.h"
 
Signature · 1 line
2.25.1
#2 lore raw

Re: [PATCH v5 01/11] tracing/user_events: Split header into uapi and kernel

Quoted 33 lines · Beau Belgrave

The UAPI parts need to be split out from the kernel parts of user_events now that other parts of the kernel will reference it. Do so by moving the existing include/linux/user_events.h into include/uapi/linux/user_events.h.

---
  include/linux/user_events.h      | 52 +++++---------------------------
  include/uapi/linux/user_events.h | 48 +++++++++++++++++++++++++++++
  kernel/trace/trace_events_user.c |  5 ---
  3 files changed, 56 insertions(+), 49 deletions(-)

create mode 100644 include/uapi/linux/user_events.h

include/linux/user_events.h +3 -6
index 592a3fbed98e..036b360f3d97 100644
@@ -1,54 +1,18 @@
-/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/* SPDX-License-Identifier: GPL-2.0-only */
  /*
- * Copyright (c) 2021, Microsoft Corporation.
+ * Copyright (c) 2022, Microsoft Corporation.
   *
   * Authors:
   *   Beau Belgrave <[email protected]>
   */
-#ifndef _UAPI_LINUX_USER_EVENTS_H
-#define _UAPI_LINUX_USER_EVENTS_H
 
-#include <linux/types.h>
-#include <linux/ioctl.h>
+#ifndef _LINUX_USER_EVENTS_H

include/linux/user_events.h line 16

Quoted 10 lines
include/linux/user_events.h +4 -3
+#define _LINUX_USER_EVENTS_H
 
-#ifdef __KERNEL__
-#include <linux/uio.h>
+#include <uapi/linux/user_events.h>
+
+#ifdef CONFIG_USER_EVENTS
  #else
-#include <sys/uio.h>
  #endif

Not sure why this is left here ?

#ifdef CONFIG_USER_EVENTS
#else
#endif

It seems useless at this stage. Perhaps it's meant to show up in a following patch ?

[...]

Quoted 42 lines
include/linux/user_events.h +1 -2
-
-#endif /* _UAPI_LINUX_USER_EVENTS_H */
+#endif /* _LINUX_USER_EVENTS_H */
include/uapi/linux/user_events.h new file +33 -0
new file mode 100644
index 000000000000..7700759a7cd9
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (c) 2021-2022, Microsoft Corporation.
+ *
+ * Authors:
+ *   Beau Belgrave <[email protected]>
+ */
+#ifndef _UAPI_LINUX_USER_EVENTS_H
+#define _UAPI_LINUX_USER_EVENTS_H
+
+#include <linux/types.h>
+#include <linux/ioctl.h>
+
+#define USER_EVENTS_SYSTEM "user_events"
+#define USER_EVENTS_PREFIX "u:"
+
+/* Create dynamic location entry within a 32-bit value */
+#define DYN_LOC(offset, size) ((size) << 16 | (offset))
+
+/*
+ * Describes an event registration and stores the results of the registration.
+ * This structure is passed to the DIAG_IOCSREG ioctl, callers at a minimum
+ * must set the size and name_args before invocation.
+ */
+struct user_reg {
+
+	/* Input: Size of the user_reg structure being used */
+	__u32 size;
+
+	/* Input: Pointer to string with event name, description and flags */
+	__u64 name_args;
+
+	/* Output: Bitwise index of the event within the status page */

include/uapi/linux/user_events.h line 43

Quoted 10 lines
include/uapi/linux/user_events.h new file +10 -0
+	__u32 status_bit;
+
+	/* Output: Index of the event to use when writing data */
+	__u32 write_index;
+} __attribute__((__packed__));
+
+#define DIAG_IOC_MAGIC '*'
+
+/* Requests to register a user_event */
+#define DIAG_IOCSREG _IOWR(DIAG_IOC_MAGIC, 0, struct user_reg*)

Coding style: "struct user_reg *" (space before '*').

include/uapi/linux/user_events.h line 46

Quoted 3 lines
include/uapi/linux/user_events.h new file +3 -0
+
+/* Requests to delete a user_event */
+#define DIAG_IOCSDEL _IOW(DIAG_IOC_MAGIC, 1, char*)

Coding style: space before '*'.

I notice the use of plural for "Requests". Are those batched requests, or a single request ?

Thanks,

Mathieu

Quoted 19 lines
include/uapi/linux/user_events.h new file +2 -0
+
+#endif /* _UAPI_LINUX_USER_EVENTS_H */
kernel/trace/trace_events_user.c +0 -5
index ae78c2d53c8a..890357b48c37 100644
@@ -19,12 +19,7 @@
  #include <linux/tracefs.h>
  #include <linux/types.h>
  #include <linux/uaccess.h>
-/* Reminder to move to uapi when everything works */
-#ifdef CONFIG_COMPILE_TEST
  #include <linux/user_events.h>
-#else
-#include <uapi/linux/user_events.h>
-#endif
  #include "trace.h"
  #include "trace_dynevent.h"
 
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
#3 lore raw

Re: [PATCH v5 01/11] tracing/user_events: Split header into uapi and kernel

Quoted 45 lines · Mathieu Desnoyers
Quoted 31 lines · Beau Belgrave

The UAPI parts need to be split out from the kernel parts of user_events now that other parts of the kernel will reference it. Do so by moving the existing include/linux/user_events.h into include/uapi/linux/user_events.h.

---
  include/linux/user_events.h      | 52 +++++---------------------------
  include/uapi/linux/user_events.h | 48 +++++++++++++++++++++++++++++
  kernel/trace/trace_events_user.c |  5 ---
  3 files changed, 56 insertions(+), 49 deletions(-)

create mode 100644 include/uapi/linux/user_events.h

include/linux/user_events.h +2 -6
index 592a3fbed98e..036b360f3d97 100644
@@ -1,54 +1,18 @@
-/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/* SPDX-License-Identifier: GPL-2.0-only */
  /*
- * Copyright (c) 2021, Microsoft Corporation.
+ * Copyright (c) 2022, Microsoft Corporation.
   *
   * Authors:
   *   Beau Belgrave <[email protected]>
   */
-#ifndef _UAPI_LINUX_USER_EVENTS_H
-#define _UAPI_LINUX_USER_EVENTS_H
-#include <linux/types.h>
-#include <linux/ioctl.h>

include/linux/user_events.h line 14

Quoted 10 lines
include/linux/user_events.h +5 -3
+#ifndef _LINUX_USER_EVENTS_H
+#define _LINUX_USER_EVENTS_H
-#ifdef __KERNEL__
-#include <linux/uio.h>
+#include <uapi/linux/user_events.h>
+
+#ifdef CONFIG_USER_EVENTS
  #else
-#include <sys/uio.h>
  #endif

Not sure why this is left here ?

Quoted 7 lines

#ifdef CONFIG_USER_EVENTS
#else
#endif

It seems useless at this stage. Perhaps it's meant to show up in a following patch ?

Yes, it's used in later patches. I can fix this up so it's less odd here.

Thanks,
-Beau

Quoted 48 lines

[...]

Quoted 46 lines
include/linux/user_events.h +1 -2
-
-#endif /* _UAPI_LINUX_USER_EVENTS_H */
+#endif /* _LINUX_USER_EVENTS_H */
include/uapi/linux/user_events.h new file +37 -0
new file mode 100644
index 000000000000..7700759a7cd9
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (c) 2021-2022, Microsoft Corporation.
+ *
+ * Authors:
+ *   Beau Belgrave <[email protected]>
+ */
+#ifndef _UAPI_LINUX_USER_EVENTS_H
+#define _UAPI_LINUX_USER_EVENTS_H
+
+#include <linux/types.h>
+#include <linux/ioctl.h>
+
+#define USER_EVENTS_SYSTEM "user_events"
+#define USER_EVENTS_PREFIX "u:"
+
+/* Create dynamic location entry within a 32-bit value */
+#define DYN_LOC(offset, size) ((size) << 16 | (offset))
+
+/*
+ * Describes an event registration and stores the results of the registration.
+ * This structure is passed to the DIAG_IOCSREG ioctl, callers at a minimum
+ * must set the size and name_args before invocation.
+ */
+struct user_reg {
+
+	/* Input: Size of the user_reg structure being used */
+	__u32 size;
+
+	/* Input: Pointer to string with event name, description and flags */
+	__u64 name_args;
+
+	/* Output: Bitwise index of the event within the status page */
+	__u32 status_bit;
+
+	/* Output: Index of the event to use when writing data */
+	__u32 write_index;
Quoted 9 lines

include/uapi/linux/user_events.h line 43

Quoted 6 lines
include/uapi/linux/user_events.h new file +6 -0
+} __attribute__((__packed__));
+
+#define DIAG_IOC_MAGIC '*'
+
+/* Requests to register a user_event */
+#define DIAG_IOCSREG _IOWR(DIAG_IOC_MAGIC, 0, struct user_reg*)

Coding style: "struct user_reg *" (space before '*').

Will do.

Quoted 6 lines

include/uapi/linux/user_events.h line 46

Quoted 3 lines
include/uapi/linux/user_events.h new file +3 -0
+
+/* Requests to delete a user_event */
+#define DIAG_IOCSDEL _IOW(DIAG_IOC_MAGIC, 1, char*)

Coding style: space before '*'.

Will do.

Quoted 3 lines

I notice the use of plural for "Requests". Are those batched requests, or a single request ?

It is a single request, I'll change this to just "Request" to make it clear.

Thanks,
-Beau

Quoted 27 lines

Thanks,

Mathieu

Quoted 18 lines
include/uapi/linux/user_events.h new file +2 -0
+
+#endif /* _UAPI_LINUX_USER_EVENTS_H */
kernel/trace/trace_events_user.c +0 -5
index ae78c2d53c8a..890357b48c37 100644
@@ -19,12 +19,7 @@
  #include <linux/tracefs.h>
  #include <linux/types.h>
  #include <linux/uaccess.h>
-/* Reminder to move to uapi when everything works */
-#ifdef CONFIG_COMPILE_TEST
  #include <linux/user_events.h>
-#else
-#include <uapi/linux/user_events.h>
-#endif
  #include "trace.h"
  #include "trace_dynevent.h"
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
#4 lore raw

PATCH v5 2/11 tracing/user_events: Track fork/exec/exit for mm lifetime

In reply to Beau Belgrave

During tracefs discussions it was decided instead of requiring a mapping within a user-process to track the lifetime of memory descriptors we should hook the appropriate calls. Do this by adding the minimal stubs required for task fork, exec, and exit. Currently this is just a NOP. Future patches will implement these calls fully.

---
 fs/exec.c                   |  2 ++
 include/linux/sched.h       |  5 +++++
 include/linux/user_events.h | 16 +++++++++++++++-
 kernel/exit.c               |  2 ++
 kernel/fork.c               |  2 ++
 5 files changed, 26 insertions(+), 1 deletion(-)
fs/exec.c +2 -0
index a0b1f0337a62..75ca6fbd195c 100644
@@ -64,6 +64,7 @@
 #include <linux/io_uring.h>
 #include <linux/syscall_user_dispatch.h>
 #include <linux/coredump.h>
+#include <linux/user_events.h>
 
 #include <linux/uaccess.h>
 #include <asm/mmu_context.h>
@@ -1842,6 +1843,7 @@ static int bprm_execve(struct linux_binprm *bprm,
 	current->fs->in_exec = 0;
 	current->in_execve = 0;
 	rseq_execve(current);
+	user_events_execve(current);
 	acct_update_integrals(current);
 	task_numa_free(current, false);
 	return retval;
include/linux/sched.h +5 -0
index ffb6eb55cd13..61ed2f9deb26 100644
@@ -69,6 +69,7 @@ struct sighand_struct;
 struct signal_struct;
 struct task_delay_info;
 struct task_group;
+struct user_event_mm;
 
 /*
  * Task state bitmask. NOTE! These bits are also
@@ -1528,6 +1529,10 @@ struct task_struct {
 	union rv_task_monitor		rv[RV_PER_TASK_MONITORS];
 #endif
 
+#ifdef CONFIG_USER_EVENTS
+	struct user_event_mm		*user_event_mm;
+#endif
+
 	/*
 	 * New fields for task_struct should be added above here, so that
 	 * they are included in the randomized portion of task_struct.
include/linux/user_events.h +15 -1
index 036b360f3d97..3d747c45d2fa 100644
@@ -12,7 +12,21 @@
 #include <uapi/linux/user_events.h>
 
 #ifdef CONFIG_USER_EVENTS
-#else
+struct user_event_mm {
+};
 #endif
 
+static inline void user_events_fork(struct task_struct *t,
+				    unsigned long clone_flags)
+{
+}
+
+static inline void user_events_execve(struct task_struct *t)
+{
+}
+
+static inline void user_events_exit(struct task_struct *t)
+{
+}
+
 #endif /* _LINUX_USER_EVENTS_H */
kernel/exit.c +2 -0
index 35e0a31a0315..0bfadc26d278 100644
@@ -67,6 +67,7 @@
 #include <linux/io_uring.h>
 #include <linux/kprobes.h>
 #include <linux/rethook.h>
+#include <linux/user_events.h>
 
 #include <linux/uaccess.h>
 #include <asm/unistd.h>
@@ -769,6 +770,7 @@ void __noreturn do_exit(long code)
 
 	coredump_task_exit(tsk);
 	ptrace_event(PTRACE_EVENT_EXIT, code);
+	user_events_exit(tsk);
 
 	validate_creds_for_do_exit(tsk);
 
kernel/fork.c +2 -0
index 08969f5aa38d..a4be76276ccb 100644
@@ -97,6 +97,7 @@
 #include <linux/scs.h>
 #include <linux/io_uring.h>
 #include <linux/bpf.h>
+#include <linux/user_events.h>
 
 #include <asm/pgalloc.h>
 #include <linux/uaccess.h>
@@ -2487,6 +2488,7 @@ static __latent_entropy struct task_struct *copy_process(
 
 	trace_task_newtask(p, clone_flags);
 	uprobe_copy_process(p, clone_flags);
+	user_events_fork(p, clone_flags);
 
 	copy_oom_score_adj(clone_flags, p);
 
Signature · 1 line
2.25.1
reviewer Reviewed-by on v5 2/11
#5 lore raw

Re: [PATCH v5 02/11] tracing/user_events: Track fork/exec/exit for mm lifetime

Quoted 5 lines · Beau Belgrave

During tracefs discussions it was decided instead of requiring a mapping within a user-process to track the lifetime of memory descriptors we should hook the appropriate calls. Do this by adding the minimal stubs required for task fork, exec, and exit. Currently this is just a NOP. Future patches will implement these calls fully.

Quoted 121 lines
---
  fs/exec.c                   |  2 ++
  include/linux/sched.h       |  5 +++++
  include/linux/user_events.h | 16 +++++++++++++++-
  kernel/exit.c               |  2 ++
  kernel/fork.c               |  2 ++
  5 files changed, 26 insertions(+), 1 deletion(-)
fs/exec.c +2 -0
index a0b1f0337a62..75ca6fbd195c 100644
@@ -64,6 +64,7 @@
  #include <linux/io_uring.h>
  #include <linux/syscall_user_dispatch.h>
  #include <linux/coredump.h>
+#include <linux/user_events.h>
 
  #include <linux/uaccess.h>
  #include <asm/mmu_context.h>
@@ -1842,6 +1843,7 @@ static int bprm_execve(struct linux_binprm *bprm,
  	current->fs->in_exec = 0;
  	current->in_execve = 0;
  	rseq_execve(current);
+	user_events_execve(current);
  	acct_update_integrals(current);
  	task_numa_free(current, false);
  	return retval;
include/linux/sched.h +5 -0
index ffb6eb55cd13..61ed2f9deb26 100644
@@ -69,6 +69,7 @@ struct sighand_struct;
  struct signal_struct;
  struct task_delay_info;
  struct task_group;
+struct user_event_mm;
 
  /*
   * Task state bitmask. NOTE! These bits are also
@@ -1528,6 +1529,10 @@ struct task_struct {
  	union rv_task_monitor		rv[RV_PER_TASK_MONITORS];
  #endif
 
+#ifdef CONFIG_USER_EVENTS
+	struct user_event_mm		*user_event_mm;
+#endif
+
  	/*
  	 * New fields for task_struct should be added above here, so that
  	 * they are included in the randomized portion of task_struct.
include/linux/user_events.h +15 -1
index 036b360f3d97..3d747c45d2fa 100644
@@ -12,7 +12,21 @@
  #include <uapi/linux/user_events.h>
 
  #ifdef CONFIG_USER_EVENTS
-#else
+struct user_event_mm {
+};
  #endif
 
+static inline void user_events_fork(struct task_struct *t,
+				    unsigned long clone_flags)
+{
+}
+
+static inline void user_events_execve(struct task_struct *t)
+{
+}
+
+static inline void user_events_exit(struct task_struct *t)
+{
+}
+
  #endif /* _LINUX_USER_EVENTS_H */
kernel/exit.c +2 -0
index 35e0a31a0315..0bfadc26d278 100644
@@ -67,6 +67,7 @@
  #include <linux/io_uring.h>
  #include <linux/kprobes.h>
  #include <linux/rethook.h>
+#include <linux/user_events.h>
 
  #include <linux/uaccess.h>
  #include <asm/unistd.h>
@@ -769,6 +770,7 @@ void __noreturn do_exit(long code)
 
  	coredump_task_exit(tsk);
  	ptrace_event(PTRACE_EVENT_EXIT, code);
+	user_events_exit(tsk);
 
  	validate_creds_for_do_exit(tsk);
 
kernel/fork.c +2 -0
index 08969f5aa38d..a4be76276ccb 100644
@@ -97,6 +97,7 @@
  #include <linux/scs.h>
  #include <linux/io_uring.h>
  #include <linux/bpf.h>
+#include <linux/user_events.h>
 
  #include <asm/pgalloc.h>
  #include <linux/uaccess.h>
@@ -2487,6 +2488,7 @@ static __latent_entropy struct task_struct *copy_process(
 
  	trace_task_newtask(p, clone_flags);
  	uprobe_copy_process(p, clone_flags);
+	user_events_fork(p, clone_flags);
 
  	copy_oom_score_adj(clone_flags, p);
 
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
#6 lore raw

PATCH v5 3/11 tracing/user_events: Use remote writes for event enablement

In reply to Beau Belgrave

As part of the discussions for user_events aligned with user space tracers, it was determined that user programs should register a aligned value to set or clear a bit when an event becomes enabled. Currently a shared page is being used that requires mmap(). Remove the shared page implementation and move to a user registered address implementation.

In this new model during the event registration from user programs 3 new values are specified. The first is the address to update when the event is either enabled or disabled. The second is the bit to set/clear to reflect the event being enabled. The third is the size of the value at the specified address.

This allows for a local 32/64-bit value in user programs to support both kernel and user tracers. As an example, setting bit 31 for kernel tracers when the event becomes enabled allows for user tracers to use the other bits for ref counts or other flags. The kernel side updates the bit atomically, user programs need to also update these values atomically.

User provided addresses must be aligned on a natural boundary, this allows for single page checking and prevents odd behaviors such as a enable value straddling 2 pages instead of a single page. Currently page faults are only logged, future patches will handle these.

---
 include/linux/user_events.h      |  51 ++-
 include/uapi/linux/user_events.h |  15 +-
 kernel/trace/Kconfig             |   5 +-
 kernel/trace/trace_events_user.c | 572 +++++++++++++++++++++++--------
 4 files changed, 501 insertions(+), 142 deletions(-)
include/linux/user_events.h +50 -1
index 3d747c45d2fa..4b6642899b33 100644
@@ -9,13 +9,61 @@
 #ifndef _LINUX_USER_EVENTS_H
 #define _LINUX_USER_EVENTS_H
 
+#include <linux/list.h>
+#include <linux/refcount.h>
+#include <linux/mm_types.h>
 #include <uapi/linux/user_events.h>
 
 #ifdef CONFIG_USER_EVENTS
 struct user_event_mm {
+	struct list_head link;
+	struct list_head enablers;
+	struct mm_struct *mm;
+	struct user_event_mm *next;
+	refcount_t refcnt;
+	refcount_t tasks;
 };
-#endif
 
+extern void user_event_mm_dup(struct task_struct *t,
+			      struct user_event_mm *old_mm);
+
+extern void user_event_mm_remove(struct task_struct *t);
+
+static inline void user_events_fork(struct task_struct *t,
+				    unsigned long clone_flags)
+{
+	struct user_event_mm *old_mm;
+
+	if (!t || !current->user_event_mm)
+		return;
+
+	old_mm = current->user_event_mm;
+
+	if (clone_flags & CLONE_VM) {
+		t->user_event_mm = old_mm;
+		refcount_inc(&old_mm->tasks);
+		return;
+	}
+
+	user_event_mm_dup(t, old_mm);
+}
+
+static inline void user_events_execve(struct task_struct *t)
+{
+	if (!t || !t->user_event_mm)
+		return;
+
+	user_event_mm_remove(t);
+}
+
+static inline void user_events_exit(struct task_struct *t)
+{
+	if (!t || !t->user_event_mm)
+		return;
+
+	user_event_mm_remove(t);
+}
+#else
 static inline void user_events_fork(struct task_struct *t,
 				    unsigned long clone_flags)
 {
@@ -28,5 +76,6 @@ static inline void user_events_execve(struct task_struct *t)
 static inline void user_events_exit(struct task_struct *t)
 {
 }
+#endif /* CONFIG_USER_EVENTS */
 
 #endif /* _LINUX_USER_EVENTS_H */
include/uapi/linux/user_events.h +12 -3
index 7700759a7cd9..5bee4201dad0 100644
@@ -27,12 +27,21 @@ struct user_reg {
 	/* Input: Size of the user_reg structure being used */
 	__u32 size;
 
+	/* Input: Bit in enable address to use */
+	__u8 enable_bit;
+
+	/* Input: Enable size in bytes at address */
+	__u8 enable_size;
+
+	/* Input: Reserved for future use, set to 0 */
+	__u16 __reserved;
+
+	/* Input: Address to update when enabled */
+	__u64 enable_addr;
+
 	/* Input: Pointer to string with event name, description and flags */
 	__u64 name_args;
 
-	/* Output: Bitwise index of the event within the status page */
-	__u32 status_bit;
-
 	/* Output: Index of the event to use when writing data */
 	__u32 write_index;
 } __attribute__((__packed__));
kernel/trace/Kconfig +3 -2
index e9e95c790b8e..3509deb078c9 100644
@@ -782,9 +782,10 @@ config USER_EVENTS
 	  can be used like an existing kernel trace event.  User trace
 	  events are generated by writing to a tracefs file.  User
 	  processes can determine if their tracing events should be
-	  generated by memory mapping a tracefs file and checking for
-	  an associated byte being non-zero.
+	  generated by registering a value and bit with the kernel
+	  that reflects when it is enabled or not.
 
+	  See Documentation/trace/user_events.rst.
 	  If in doubt, say N.
 
 config HIST_TRIGGERS
kernel/trace/trace_events_user.c +436 -136

752 lines not on this page · raw message

Signature · 1 line
2.25.1
#7 lore raw

Re: [PATCH v5 03/11] tracing/user_events: Use remote writes for event enablement

On 2022-12-05 16:00, Beau Belgrave wrote:
[...]

Quoted 40 lines
  #ifdef CONFIG_USER_EVENTS
  struct user_event_mm {
include/linux/user_events.h +35 -1
+	struct list_head link;
+	struct list_head enablers;
+	struct mm_struct *mm;
+	struct user_event_mm *next;
+	refcount_t refcnt;
+	refcount_t tasks;
  };
-#endif
 
+extern void user_event_mm_dup(struct task_struct *t,
+			      struct user_event_mm *old_mm);
+
+extern void user_event_mm_remove(struct task_struct *t);
+
+static inline void user_events_fork(struct task_struct *t,
+				    unsigned long clone_flags)
+{
+	struct user_event_mm *old_mm;
+
+	if (!t || !current->user_event_mm)
+		return;
+
+	old_mm = current->user_event_mm;
+
+	if (clone_flags & CLONE_VM) {
+		t->user_event_mm = old_mm;
+		refcount_inc(&old_mm->tasks);
+		return;
+	}
+
+	user_event_mm_dup(t, old_mm);
+}
+
+static inline void user_events_execve(struct task_struct *t)
+{
+	if (!t || !t->user_event_mm)
+		return;
+

include/linux/user_events.h line 65

Quoted 10 lines
include/linux/user_events.h +10 -0
+	user_event_mm_remove(t);
+}
+
+static inline void user_events_exit(struct task_struct *t)
+{
+	if (!t || !t->user_event_mm)
+		return;
+
+	user_event_mm_remove(t);
+}

So this is adding user_event_mm_remove() calls on each execve and each process exit, correct ?

[...]

Quoted 24 lines
kernel/trace/trace_events_user.c +24 -0
+
+void user_event_mm_remove(struct task_struct *t)
+{
+	struct user_event_mm *mm;
+	unsigned long flags;
+
+	might_sleep();
+
+	mm = t->user_event_mm;
+	t->user_event_mm = NULL;
+
+	/* Clone will increment the tasks, only remove if last clone */
+	if (!refcount_dec_and_test(&mm->tasks))
+		return;
+
+	/* Remove the mm from the list, so it can no longer be enabled */
+	spin_lock_irqsave(&user_event_mms_lock, flags);
+	list_del_rcu(&mm->link);
+	spin_unlock_irqrestore(&user_event_mms_lock, flags);
+
+	/*
+	 * Put for mm must be done after RCU sync to handle new refs in
+	 * between the list_del_rcu() and now. This ensures any get refs
+	 * during rcu_read_lock() are accounted for during list removal.

kernel/trace/trace_events_user.c static struct user_event_group · line 486

Quoted 10 lines
kernel/trace/trace_events_user.c +10 -0
+	 *
+	 * CPU A			|	CPU B
+	 * ---------------------------------------------------------------
+	 * user_event_mm_remove()	|	rcu_read_lock();
+	 * list_del_rcu()		|	list_for_each_entry_rcu();
+	 * synchronize_rcu()		|	refcount_inc();
+	 * .				|	rcu_read_unlock();
+	 * user_event_mm_put()		|	.
+	 */
+	synchronize_rcu();

This means a synchronize_rcu() is added on each execve and each process exit ? I am really worried about the performance impact of this big hammer synchronization in those key points of process lifetime.

Thanks,

Mathieu

Quoted 43 lines
kernel/trace/trace_events_user.c +37 -2
+
+	/*
+	 * We need to wait for currently occurring writes to stop within
+	 * the mm. This is required since exit_mm() snaps the current rss
+	 * stats and clears them. On the final mmdrop(), check_mm() will
+	 * report a bug if these increment.
+	 *
+	 * All writes/pins are done under mmap_read lock, take the write
+	 * lock to ensure in-progress faults have completed. Faults that
+	 * are pending but yet to run will check the task count and skip
+	 * the fault since the mm is going away.
+	 */
+	mmap_write_lock(mm->mm);
+	mmap_write_unlock(mm->mm);
+
+	/* MM is still alive, but won't be updated anymore */
+	user_event_mm_put(mm);
+}
+
+void user_event_mm_dup(struct task_struct *t, struct user_event_mm *old_mm)
  {
-	int i = user->index;
+	struct user_event_mm *mm = user_event_mm_create(t);
+	struct user_event_enabler *enabler;
+
+	if (!mm)
+		return;
+
+	rcu_read_lock();
 
-	user->group->register_page_data[MAP_STATUS_BYTE(i)] |= MAP_STATUS_MASK(i);
+	list_for_each_entry_rcu(enabler, &old_mm->enablers, link)
+		if (!user_event_enabler_dup(enabler, mm))
+			goto error;
+
+	rcu_read_unlock();
+
+	return;
+error:
+	rcu_read_unlock();
+	user_event_mm_remove(t);
  }
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
#8 lore raw

Re: [PATCH v5 03/11] tracing/user_events: Use remote writes for event enablement

Quoted 46 lines · Mathieu Desnoyers

On 2022-12-05 16:00, Beau Belgrave wrote:
[...]

Quoted 44 lines
  #ifdef CONFIG_USER_EVENTS
  struct user_event_mm {
include/linux/user_events.h +40 -1
+	struct list_head link;
+	struct list_head enablers;
+	struct mm_struct *mm;
+	struct user_event_mm *next;
+	refcount_t refcnt;
+	refcount_t tasks;
  };
-#endif
+extern void user_event_mm_dup(struct task_struct *t,
+			      struct user_event_mm *old_mm);
+
+extern void user_event_mm_remove(struct task_struct *t);
+
+static inline void user_events_fork(struct task_struct *t,
+				    unsigned long clone_flags)
+{
+	struct user_event_mm *old_mm;
+
+	if (!t || !current->user_event_mm)
+		return;
+
+	old_mm = current->user_event_mm;
+
+	if (clone_flags & CLONE_VM) {
+		t->user_event_mm = old_mm;
+		refcount_inc(&old_mm->tasks);
+		return;
+	}
+
+	user_event_mm_dup(t, old_mm);
+}
+
+static inline void user_events_execve(struct task_struct *t)
+{
+	if (!t || !t->user_event_mm)
+		return;
+
+	user_event_mm_remove(t);
+}
+
+static inline void user_events_exit(struct task_struct *t)
+{
Quoted 9 lines

include/linux/user_events.h line 57

Quoted 5 lines
include/linux/user_events.h +5 -0
+	if (!t || !t->user_event_mm)
+		return;
+
+	user_event_mm_remove(t);
+}

So this is adding user_event_mm_remove() calls on each execve and each process exit, correct ?

Yes, as long as the process has registered a user_event. If it has not, nothing happens.

Quoted 33 lines

[...]

Quoted 30 lines
kernel/trace/trace_events_user.c +30 -0
+
+void user_event_mm_remove(struct task_struct *t)
+{
+	struct user_event_mm *mm;
+	unsigned long flags;
+
+	might_sleep();
+
+	mm = t->user_event_mm;
+	t->user_event_mm = NULL;
+
+	/* Clone will increment the tasks, only remove if last clone */
+	if (!refcount_dec_and_test(&mm->tasks))
+		return;
+
+	/* Remove the mm from the list, so it can no longer be enabled */
+	spin_lock_irqsave(&user_event_mms_lock, flags);
+	list_del_rcu(&mm->link);
+	spin_unlock_irqrestore(&user_event_mms_lock, flags);
+
+	/*
+	 * Put for mm must be done after RCU sync to handle new refs in
+	 * between the list_del_rcu() and now. This ensures any get refs
+	 * during rcu_read_lock() are accounted for during list removal.
+	 *
+	 * CPU A			|	CPU B
+	 * ---------------------------------------------------------------
+	 * user_event_mm_remove()	|	rcu_read_lock();
+	 * list_del_rcu()		|	list_for_each_entry_rcu();
+	 * synchronize_rcu()		|	refcount_inc();
Quoted 9 lines

kernel/trace/trace_events_user.c static struct user_event_group · line 486

Quoted 4 lines
kernel/trace/trace_events_user.c +4 -0
+	 * .				|	rcu_read_unlock();
+	 * user_event_mm_put()		|	.
+	 */
+	synchronize_rcu();

This means a synchronize_rcu() is added on each execve and each process exit ? I am really worried about the performance impact of this big hammer synchronization in those key points of process lifetime.

Agreed, I can move this into a call_rcu() at the cost of an alloc. Perhaps that will work better? I could have these in memcaches.

Thanks,
-Beau

Quoted 49 lines

Thanks,

Mathieu

Quoted 41 lines
kernel/trace/trace_events_user.c +37 -2
+
+	/*
+	 * We need to wait for currently occurring writes to stop within
+	 * the mm. This is required since exit_mm() snaps the current rss
+	 * stats and clears them. On the final mmdrop(), check_mm() will
+	 * report a bug if these increment.
+	 *
+	 * All writes/pins are done under mmap_read lock, take the write
+	 * lock to ensure in-progress faults have completed. Faults that
+	 * are pending but yet to run will check the task count and skip
+	 * the fault since the mm is going away.
+	 */
+	mmap_write_lock(mm->mm);
+	mmap_write_unlock(mm->mm);
+
+	/* MM is still alive, but won't be updated anymore */
+	user_event_mm_put(mm);
+}
+
+void user_event_mm_dup(struct task_struct *t, struct user_event_mm *old_mm)
  {
-	int i = user->index;
+	struct user_event_mm *mm = user_event_mm_create(t);
+	struct user_event_enabler *enabler;
+
+	if (!mm)
+		return;
+
+	rcu_read_lock();
-	user->group->register_page_data[MAP_STATUS_BYTE(i)] |= MAP_STATUS_MASK(i);
+	list_for_each_entry_rcu(enabler, &old_mm->enablers, link)
+		if (!user_event_enabler_dup(enabler, mm))
+			goto error;
+
+	rcu_read_unlock();
+
+	return;
+error:
+	rcu_read_unlock();
+	user_event_mm_remove(t);
  }
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
#9 lore raw

PATCH v5 4/11 tracing/user_events: Fixup enable faults asyncly

In reply to Beau Belgrave

When events are enabled within the various tracing facilities, such as ftrace/perf, the event_mutex is held. As events are enabled pages are accessed. We do not want page faults to occur under this lock. Instead queue the fault to a workqueue to be handled in a process context safe way without the lock.

The enable address is marked faulting while the async fault-in occurs. This ensures that we don't attempt to fault-in more than is necessary. Once the page has been faulted in, an address write is re-attempted. If the page couldn't fault-in, then we wait until the next time the event is enabled to prevent any potential infinite loops.

---
 kernel/trace/trace_events_user.c | 120 +++++++++++++++++++++++++++++--
 1 file changed, 114 insertions(+), 6 deletions(-)
kernel/trace/trace_events_user.c +114 -6
index 4b0965365aad..5c061a54086f 100644
@@ -99,9 +99,23 @@ struct user_event_enabler {
 /* Bits 0-5 are for the bit to update upon enable/disable (0-63 allowed) */
 #define ENABLE_VAL_BIT_MASK 0x3F
 
+/* Bit 6 is for faulting status of enablement */
+#define ENABLE_VAL_FAULTING_BIT 6
+
 /* Only duplicate the bit value */
 #define ENABLE_VAL_DUP_MASK ENABLE_VAL_BIT_MASK
 
+#define ENABLE_BITOPS(e) ((unsigned long *)&(e)->values)
+
+/* Used for asynchronous faulting in of pages */
+struct user_event_enabler_fault {
+	struct work_struct work;
+	struct user_event_mm *mm;
+	struct user_event_enabler *enabler;
+};
+
+static struct kmem_cache *fault_cache;
+
 /* Global list of memory descriptors using user_events */
 static LIST_HEAD(user_event_mms);
 static DEFINE_SPINLOCK(user_event_mms_lock);
@@ -263,7 +277,85 @@ static int user_event_mm_fault_in(struct user_event_mm *mm, unsigned long uaddr)
 }
 
 static int user_event_enabler_write(struct user_event_mm *mm,
-				    struct user_event_enabler *enabler)
+				    struct user_event_enabler *enabler,
+				    bool fixup_fault);
+
+static void user_event_enabler_fault_fixup(struct work_struct *work)
+{
+	struct user_event_enabler_fault *fault = container_of(
+		work, struct user_event_enabler_fault, work);
+	struct user_event_enabler *enabler = fault->enabler;
+	struct user_event_mm *mm = fault->mm;
+	unsigned long uaddr = enabler->addr;
+	int ret;
+
+	ret = user_event_mm_fault_in(mm, uaddr);
+
+	if (ret && ret != -ENOENT) {
+		struct user_event *user = enabler->event;
+
+		pr_warn("user_events: Fault for mm: 0x%pK @ 0x%llx event: %s\n",
+			mm->mm, (unsigned long long)uaddr, EVENT_NAME(user));
+	}
+
+	/* Prevent state changes from racing */
+	mutex_lock(&event_mutex);
+
+	/*
+	 * If we managed to get the page, re-issue the write. We do not
+	 * want to get into a possible infinite loop, which is why we only
+	 * attempt again directly if the page came in. If we couldn't get
+	 * the page here, then we will try again the next time the event is
+	 * enabled/disabled.
+	 */
+	clear_bit(ENABLE_VAL_FAULTING_BIT, ENABLE_BITOPS(enabler));
+
+	if (!ret) {
+		mmap_read_lock(mm->mm);
+		user_event_enabler_write(mm, enabler, true);
+		mmap_read_unlock(mm->mm);
+	}
+
+	mutex_unlock(&event_mutex);
+
+	/* In all cases we no longer need the mm or fault */
+	user_event_mm_put(mm);
+	kmem_cache_free(fault_cache, fault);
+}
+
+static bool user_event_enabler_queue_fault(struct user_event_mm *mm,
+					   struct user_event_enabler *enabler)
+{
+	struct user_event_enabler_fault *fault;
+
+	fault = kmem_cache_zalloc(fault_cache, GFP_NOWAIT | __GFP_NOWARN);
+
+	if (!fault)
+		return false;
+
+	INIT_WORK(&fault->work, user_event_enabler_fault_fixup);
+	fault->mm = user_event_mm_get(mm);
+	fault->enabler = enabler;
+
+	/* Don't try to queue in again while we have a pending fault */
+	set_bit(ENABLE_VAL_FAULTING_BIT, ENABLE_BITOPS(enabler));
+
+	if (!schedule_work(&fault->work)) {
+		/* Allow another attempt later */
+		clear_bit(ENABLE_VAL_FAULTING_BIT, ENABLE_BITOPS(enabler));
+
+		user_event_mm_put(mm);
+		kmem_cache_free(fault_cache, fault);
+
+		return false;
+	}
+
+	return true;
+}
+
+static int user_event_enabler_write(struct user_event_mm *mm,
+				    struct user_event_enabler *enabler,
+				    bool fixup_fault)
 {
 	unsigned long uaddr = enabler->addr;
 	unsigned long *ptr;
@@ -278,11 +370,19 @@ static int user_event_enabler_write(struct user_event_mm *mm,
 	if (refcount_read(&mm->tasks) == 0)
 		return -ENOENT;
 
+	if (unlikely(test_bit(ENABLE_VAL_FAULTING_BIT, ENABLE_BITOPS(enabler))))
+		return -EBUSY;
+
 	ret = pin_user_pages_remote(mm->mm, uaddr, 1, FOLL_WRITE | FOLL_NOFAULT,
 				    &page, NULL, NULL);
 
-	if (ret <= 0) {
-		pr_warn("user_events: Enable write failed\n");
+	if (unlikely(ret <= 0)) {
+		if (!fixup_fault)
+			return -EFAULT;
+
+		if (!user_event_enabler_queue_fault(mm, enabler))
+			pr_warn("user_events: Unable to queue fault handler\n");
+
 		return -EFAULT;
 	}
 
@@ -314,7 +414,7 @@ static void user_event_enabler_update(struct user_event *user)
 
 		list_for_each_entry_rcu(enabler, &mm->enablers, link)
 			if (enabler->event == user)
-				user_event_enabler_write(mm, enabler);
+				user_event_enabler_write(mm, enabler, true);
 
 		rcu_read_unlock();
 		mmap_read_unlock(mm->mm);
@@ -552,7 +652,7 @@ static struct user_event_enabler
 
 	/* Attempt to reflect the current state within the process */
 	mmap_read_lock(user_mm->mm);
-	*write_result = user_event_enabler_write(user_mm, enabler);
+	*write_result = user_event_enabler_write(user_mm, enabler, false);
 	mmap_read_unlock(user_mm->mm);
 
 	/*
@@ -2184,15 +2284,23 @@ static int __init trace_events_user_init(void)
 {
 	int ret;
 
+	fault_cache = KMEM_CACHE(user_event_enabler_fault, 0);
+
+	if (!fault_cache)
+		return -ENOMEM;
+
 	init_group = user_event_group_create(&init_user_ns);
 
-	if (!init_group)
+	if (!init_group) {
+		kmem_cache_destroy(fault_cache);
 		return -ENOMEM;
+	}
 
 	ret = create_user_tracefs();
 
 	if (ret) {
 		pr_warn("user_events could not register with tracefs\n");
+		kmem_cache_destroy(fault_cache);
 		user_event_group_destroy(init_group);
 		init_group = NULL;
 		return ret;
Signature · 1 line
2.25.1
#10 lore raw

PATCH v5 5/11 tracing/user_events: Add ioctl for disabling addresses

In reply to Beau Belgrave

Enablements are now tracked by the lifetime of the task/mm. User processes need to be able to disable their addresses if tracing is requested to be turned off. Before unmapping the page would suffice. However, we now need a stronger contract. Add an ioctl to enable this.

A new flag bit is added, freeing, to user_event_enabler to ensure that if the event is attempted to be removed while a fault is being handled that the remove is delayed until after the fault is reattempted.

---
 include/uapi/linux/user_events.h | 24 +++++++++
 kernel/trace/trace_events_user.c | 93 +++++++++++++++++++++++++++++++-
 2 files changed, 115 insertions(+), 2 deletions(-)
include/uapi/linux/user_events.h +24 -0
index 5bee4201dad0..833f1f8a8a4a 100644
@@ -46,6 +46,27 @@ struct user_reg {
 	__u32 write_index;
 } __attribute__((__packed__));
 
+/*
+ * Describes an event unregister, callers must set the size, address and bit.
+ * This structure is passed to the DIAG_IOCSUNREG ioctl to disable bit updates.
+ */
+struct user_unreg {
+	/* Input: Size of the user_unreg structure being used */
+	__u32 size;
+
+	/* Input: Bit to unregister */
+	__u8 disable_bit;
+
+	/* Input: Reserved, set to 0 */
+	__u8 __reserved;
+
+	/* Input: Reserved, set to 0 */
+	__u16 __reserved2;
+
+	/* Input: Address to unregister */
+	__u64 disable_addr;
+} __attribute__((__packed__));
+
 #define DIAG_IOC_MAGIC '*'
 
 /* Requests to register a user_event */
@@ -54,4 +75,7 @@ struct user_reg {
 /* Requests to delete a user_event */
 #define DIAG_IOCSDEL _IOW(DIAG_IOC_MAGIC, 1, char*)
 
+/* Requests to unregister a user_event */
+#define DIAG_IOCSUNREG _IOW(DIAG_IOC_MAGIC, 2, struct user_unreg*)
+
 #endif /* _UAPI_LINUX_USER_EVENTS_H */
kernel/trace/trace_events_user.c +91 -2

135 lines not on this page · raw message

Signature · 1 line
2.25.1
#11 lore raw

PATCH v5 6/11 tracing/user_events: Update self-tests to write ABI

In reply to Beau Belgrave

ABI has been changed to remote writes, update existing test cases to use this new ABI to ensure existing functionality continues to work.

---
 .../testing/selftests/user_events/dyn_test.c  |   2 +-
 .../selftests/user_events/ftrace_test.c       | 162 ++++++++++--------
 .../testing/selftests/user_events/perf_test.c |  39 ++---
 3 files changed, 105 insertions(+), 98 deletions(-)
tools/testing/selftests/user_events/dyn_test.c +1 -1
index d6265d14cd51..8879a7b04c6a 100644
@@ -16,7 +16,7 @@
 
 #include "../kselftest_harness.h"
 
-const char *dyn_file = "/sys/kernel/debug/tracing/dynamic_events";
+const char *dyn_file = "/sys/kernel/tracing/dynamic_events";
 const char *clear = "!u:__test_event";
 
 static int Append(const char *value)
tools/testing/selftests/user_events/ftrace_test.c +88 -74

365 lines not on this page · raw message

tools/testing/selftests/user_events/perf_test.c +16 -23
index 8b4c7879d5a7..a070258d4449 100644
@@ -18,10 +18,9 @@
 
 #include "../kselftest_harness.h"
 
-const char *data_file = "/sys/kernel/debug/tracing/user_events_data";
-const char *status_file = "/sys/kernel/debug/tracing/user_events_status";
-const char *id_file = "/sys/kernel/debug/tracing/events/user_events/__test_event/id";
-const char *fmt_file = "/sys/kernel/debug/tracing/events/user_events/__test_event/format";
+const char *data_file = "/sys/kernel/tracing/user_events_data";
+const char *id_file = "/sys/kernel/tracing/events/user_events/__test_event/id";
+const char *fmt_file = "/sys/kernel/tracing/events/user_events/__test_event/format";
 
 struct event {
 	__u32 index;
@@ -35,11 +34,6 @@ static long perf_event_open(struct perf_event_attr *pe, pid_t pid,
 	return syscall(__NR_perf_event_open, pe, pid, cpu, group_fd, flags);
 }
 
-static inline int status_check(char *status_page, int status_bit)
-{
-	return status_page[status_bit >> 3] & (1 << (status_bit & 7));
-}
-
 static int get_id(void)
 {
 	FILE *fp = fopen(id_file, "r");
@@ -88,45 +82,38 @@ static int get_offset(void)
 }
 
 FIXTURE(user) {
-	int status_fd;
 	int data_fd;
+	int check;
 };
 
 FIXTURE_SETUP(user) {
-	self->status_fd = open(status_file, O_RDONLY);
-	ASSERT_NE(-1, self->status_fd);
-
 	self->data_fd = open(data_file, O_RDWR);
 	ASSERT_NE(-1, self->data_fd);
 }
 
 FIXTURE_TEARDOWN(user) {
-	close(self->status_fd);
 	close(self->data_fd);
 }
 
 TEST_F(user, perf_write) {
 	struct perf_event_attr pe = {0};
 	struct user_reg reg = {0};
-	int page_size = sysconf(_SC_PAGESIZE);
-	char *status_page;
 	struct event event;
 	struct perf_event_mmap_page *perf_page;
+	int page_size = sysconf(_SC_PAGESIZE);
 	int id, fd, offset;
 	__u32 *val;
 
 	reg.size = sizeof(reg);
 	reg.name_args = (__u64)"__test_event u32 field1; u32 field2";
-
-	status_page = mmap(NULL, page_size, PROT_READ, MAP_SHARED,
-			   self->status_fd, 0);
-	ASSERT_NE(MAP_FAILED, status_page);
+	reg.enable_bit = 31;
+	reg.enable_addr = (__u64)&self->check;
+	reg.enable_size = sizeof(self->check);
 
 	/* Register should work */
 	ASSERT_EQ(0, ioctl(self->data_fd, DIAG_IOCSREG, &reg));
 	ASSERT_EQ(0, reg.write_index);
-	ASSERT_NE(0, reg.status_bit);
-	ASSERT_EQ(0, status_check(status_page, reg.status_bit));
+	ASSERT_EQ(0, self->check);
 
 	/* Id should be there */
 	id = get_id();
@@ -149,7 +136,7 @@ TEST_F(user, perf_write) {
 	ASSERT_NE(MAP_FAILED, perf_page);
 
 	/* Status should be updated */
-	ASSERT_NE(0, status_check(status_page, reg.status_bit));
+	ASSERT_EQ(1 << reg.enable_bit, self->check);
 
 	event.index = reg.write_index;
 	event.field1 = 0xc001;
@@ -165,6 +152,12 @@ TEST_F(user, perf_write) {
 	/* Ensure correct */
 	ASSERT_EQ(event.field1, *val++);
 	ASSERT_EQ(event.field2, *val++);
+
+	munmap(perf_page, page_size * 2);
+	close(fd);
+
+	/* Status should be updated */
+	ASSERT_EQ(0, self->check);
 }
 
 int main(int argc, char **argv)
Signature · 1 line
2.25.1
#12 lore raw

PATCH v5 7/11 tracing/user_events: Add ABI self-test

In reply to Beau Belgrave

Add ABI specific self-test to ensure enablements work in various scenarios such as fork, VM_CLONE, and basic event enable/disable. Ensure ABI contracts/limits are also being upheld, such as bit limits and data size limits.

---
 tools/testing/selftests/user_events/Makefile  |   2 +-
 .../testing/selftests/user_events/abi_test.c  | 228 ++++++++++++++++++
 2 files changed, 229 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/user_events/abi_test.c
tools/testing/selftests/user_events/Makefile +1 -1
index c765d8635d9a..d5f64ef93197 100644
@@ -2,7 +2,7 @@
 CFLAGS += -Wl,-no-as-needed -Wall -I../../../../usr/include
 LDLIBS += -lrt -lpthread -lm
 
-TEST_GEN_PROGS = ftrace_test dyn_test perf_test
+TEST_GEN_PROGS = ftrace_test dyn_test perf_test abi_test
 
 TEST_FILES := settings
 
tools/testing/selftests/user_events/abi_test.c new file +228 -0

228 lines not on this page · raw message

Signature · 1 line
2.25.1
#13 lore raw

PATCH v5 8/11 tracing/user_events: Use write ABI in example

In reply to Beau Belgrave

The ABI has changed to use a remote write approach. Update the example to show the expected use of this new ABI. Also remove debugfs path and use tracefs to ensure example works in more environments.

---
 samples/user_events/example.c | 47 +++++++----------------------------
 1 file changed, 9 insertions(+), 38 deletions(-)
samples/user_events/example.c +9 -38
index d06dc24156ec..28165a096697 100644
@@ -9,51 +9,28 @@
 #include <errno.h>
 #include <sys/ioctl.h>
 #include <sys/mman.h>
+#include <sys/uio.h>
 #include <fcntl.h>
 #include <stdio.h>
 #include <unistd.h>
-#include <asm/bitsperlong.h>
-#include <endian.h>
 #include <linux/user_events.h>
 
-#if __BITS_PER_LONG == 64
-#define endian_swap(x) htole64(x)
-#else
-#define endian_swap(x) htole32(x)
-#endif
+const char *data_file = "/sys/kernel/tracing/user_events_data";
+int enabled = 0;
 
-/* Assumes debugfs is mounted */
-const char *data_file = "/sys/kernel/debug/tracing/user_events_data";
-const char *status_file = "/sys/kernel/debug/tracing/user_events_status";
-
-static int event_status(long **status)
-{
-	int fd = open(status_file, O_RDONLY);
-
-	*status = mmap(NULL, sysconf(_SC_PAGESIZE), PROT_READ,
-		       MAP_SHARED, fd, 0);
-
-	close(fd);
-
-	if (*status == MAP_FAILED)
-		return -1;
-
-	return 0;
-}
-
-static int event_reg(int fd, const char *command, long *index, long *mask,
-		     int *write)
+static int event_reg(int fd, const char *command, int *write, int *enabled)
 {
 	struct user_reg reg = {0};
 
 	reg.size = sizeof(reg);
+	reg.enable_bit = 31;
+	reg.enable_size = sizeof(*enabled);
+	reg.enable_addr = (__u64)enabled;
 	reg.name_args = (__u64)command;
 
 	if (ioctl(fd, DIAG_IOCSREG, &reg) == -1)
 		return -1;
 
-	*index = reg.status_bit / __BITS_PER_LONG;
-	*mask = endian_swap(1L << (reg.status_bit % __BITS_PER_LONG));
 	*write = reg.write_index;
 
 	return 0;
@@ -62,17 +39,12 @@ static int event_reg(int fd, const char *command, long *index, long *mask,
 int main(int argc, char **argv)
 {
 	int data_fd, write;
-	long index, mask;
-	long *status_page;
 	struct iovec io[2];
 	__u32 count = 0;
 
-	if (event_status(&status_page) == -1)
-		return errno;
-
 	data_fd = open(data_file, O_RDWR);
 
-	if (event_reg(data_fd, "test u32 count", &index, &mask, &write) == -1)
+	if (event_reg(data_fd, "test u32 count", &write, &enabled) == -1)
 		return errno;
 
 	/* Setup iovec */
@@ -80,13 +52,12 @@ int main(int argc, char **argv)
 	io[0].iov_len = sizeof(write);
 	io[1].iov_base = &count;
 	io[1].iov_len = sizeof(count);
-
 ask:
 	printf("Press enter to check status...\n");
 	getchar();
 
 	/* Check if anyone is listening */
-	if (status_page[index] & mask) {
+	if (enabled) {
 		/* Yep, trace out our data */
 		writev(data_fd, (const struct iovec *)io, 2);
 
Signature · 1 line
2.25.1
#14 lore raw

PATCH v5 9/11 tracing/user_events: Update documentation for ABI

In reply to Beau Belgrave

The ABI for user_events has changed from mmap() based to remote writes. Update the documentation to reflect these changes, add new section for unregistering events since lifetime is now tied to tasks instead of files.

---
 Documentation/trace/user_events.rst | 172 ++++++++++++++++------------
 1 file changed, 97 insertions(+), 75 deletions(-)
Documentation/trace/user_events.rst +97 -75
index 9f181f342a70..c0808addafaa 100644
@@ -11,20 +11,19 @@ that can be viewed via existing tools, such as ftrace and perf.
 To enable this feature, build your kernel with CONFIG_USER_EVENTS=y.
 
 Programs can view status of the events via
-/sys/kernel/debug/tracing/user_events_status and can both register and write
-data out via /sys/kernel/debug/tracing/user_events_data.
+/sys/kernel/tracing/user_events_status and can both register and write
+data out via /sys/kernel/tracing/user_events_data.
 
-Programs can also use /sys/kernel/debug/tracing/dynamic_events to register and
+Programs can also use /sys/kernel/tracing/dynamic_events to register and
 delete user based events via the u: prefix. The format of the command to
 dynamic_events is the same as the ioctl with the u: prefix applied.
 
 Typically programs will register a set of events that they wish to expose to
 tools that can read trace_events (such as ftrace and perf). The registration
-process gives back two ints to the program for each event. The first int is
-the status bit. This describes which bit in little-endian format in the
-/sys/kernel/debug/tracing/user_events_status file represents this event. The
-second int is the write index which describes the data when a write() or
-writev() is called on the /sys/kernel/debug/tracing/user_events_data file.
+process tells the kernel which address and bit to reflect if any tool has
+enabled the event and data should be written. The registration will give back
+a write index which describes the data when a write() or writev() is called
+on the /sys/kernel/tracing/user_events_data file.
 
 The structures referenced in this document are contained within the
 /include/uapi/linux/user_events.h file in the source tree.
@@ -35,29 +34,65 @@ filesystem and may be mounted at different paths than above.*
 Registering
 -----------
 Registering within a user process is done via ioctl() out to the
-/sys/kernel/debug/tracing/user_events_data file. The command to issue is
+/sys/kernel/tracing/user_events_data file. The command to issue is
 DIAG_IOCSREG.
 
 This command takes a packed struct user_reg as an argument::
 
   struct user_reg {
-        u32 size;
-        u64 name_args;
-        u32 status_bit;
-        u32 write_index;
-  };
+        /* Input: Size of the user_reg structure being used */
+        __u32 size;
+
+        /* Input: Bit in enable address to use */
+        __u8 enable_bit;
+
+        /* Input: Enable size in bytes at address */
+        __u8 enable_size;
+
+        /* Input: Reserved for future use, set to 0 */
+        __u16 __reserved;
+
+        /* Input: Address to update when enabled */
+        __u64 enable_addr;
+
+        /* Input: Pointer to string with event name, description and flags */
+        __u64 name_args;
+
+        /* Output: Index of the event to use when writing data */
+        __u32 write_index;
+  } __attribute__((__packed__));
+
+The struct user_reg requires all the above inputs to be set appropriately.
 
-The struct user_reg requires two inputs, the first is the size of the structure
-to ensure forward and backward compatibility. The second is the command string
-to issue for registering. Upon success two outputs are set, the status bit
-and the write index.
++ size: This must be set to sizeof(struct user_reg).
+
++ enable_bit: The bit to reflect the event status at the address specified by
+  enable_addr.
+
++ enable_size: The size of the value specified by enable_addr.
+  This must be 4 (32-bit) or 8 (64-bit). 64-bit values are only allowed to be
+  used on 64-bit kernels, however, 32-bit can be used on all kernels.
+
++ enable_addr: The address of the value to use to reflect event status. This
+  must be naturally aligned and write accessible within the user program.
+
++ name_args: The name and arguments to describe the event, see command format
+  for details.
+
+Upon successful registration the following is set.
+
++ write_index: The index to use for this file descriptor that represents this
+  event when writing out data. The index is unique to this instance of the file
+  descriptor that was used for the registration. See writing data for details.
 
 User based events show up under tracefs like any other event under the
 subsystem named "user_events". This means tools that wish to attach to the
-events need to use /sys/kernel/debug/tracing/events/user_events/[name]/enable
+events need to use /sys/kernel/tracing/events/user_events/[name]/enable
 or perf record -e user_events:[name] when attaching/recording.
 
-**NOTE:** *The write_index returned is only valid for the FD that was used*
+**NOTE:** The event subsystem name by default is "user_events". Callers should
+not assume it will always be "user_events". Operators reserve the right in the
+future to change the subsystem name per-process to accomodate event isolation.
 
 Command Format
 ^^^^^^^^^^^^^^
@@ -94,9 +129,9 @@ Would be represented by the following field::
   struct mytype myname 20
 
 Deleting
------------
+--------
 Deleting an event from within a user process is done via ioctl() out to the
-/sys/kernel/debug/tracing/user_events_data file. The command to issue is
+/sys/kernel/tracing/user_events_data file. The command to issue is
 DIAG_IOCSDEL.
 
 This command only requires a single string specifying the event to delete by
@@ -104,92 +139,79 @@ its name. Delete will only succeed if there are no references left to the
 event (in both user and kernel space). User programs should use a separate file
 to request deletes than the one used for registration due to this.
 
-Status
-------
-When tools attach/record user based events the status of the event is updated
-in realtime. This allows user programs to only incur the cost of the write() or
-writev() calls when something is actively attached to the event.
-
-User programs call mmap() on /sys/kernel/debug/tracing/user_events_status to
-check the status for each event that is registered. The bit to check in the
-file is given back after the register ioctl() via user_reg.status_bit. The bit
-is always in little-endian format. Programs can check if the bit is set either
-using a byte-wise index with a mask or a long-wise index with a little-endian
-mask.
+Unregistering
+-------------
+If after registering an event it is no longer wanted to be updated then it can
+be disabled via ioctl() out to the /sys/kernel/tracing/user_events_data file.
+The command to issue is DIAG_IOCSUNREG. This is different than deleting, where
+deleting actually removes the event from the system. Unregistering simply tells
+the kernel your process is no longer interested in updates to the event.
 
-Currently the size of user_events_status is a single page, however, custom
-kernel configurations can change this size to allow more user based events. In
-all cases the size of the file is a multiple of a page size.
+This command takes a packed struct user_unreg as an argument::
 
-For example, if the register ioctl() gives back a status_bit of 3 you would
-check byte 0 (3 / 8) of the returned mmap data and then AND the result with 8
-(1 << (3 % 8)) to see if anything is attached to that event.
+  struct user_unreg {
+        /* Input: Size of the user_unreg structure being used */
+        __u32 size;
 
-A byte-wise index check is performed as follows::
+        /* Input: Bit to unregister */
+        __u8 disable_bit;
 
-  int index, mask;
-  char *status_page;
+        /* Input: Reserved, set to 0 */
+        __u8 __reserved;
 
-  index = status_bit / 8;
-  mask = 1 << (status_bit % 8);
-
-  ...
+        /* Input: Reserved, set to 0 */
+        __u16 __reserved2;
 
-  if (status_page[index] & mask) {
-        /* Enabled */
-  }
+        /* Input: Address to unregister */
+        __u64 disable_addr;
+  } __attribute__((__packed__));
 
-A long-wise index check is performed as follows::
+The struct user_unreg requires all the above inputs to be set appropriately.
 
-  #include <asm/bitsperlong.h>
-  #include <endian.h>
++ size: This must be set to sizeof(struct user_unreg).
 
-  #if __BITS_PER_LONG == 64
-  #define endian_swap(x) htole64(x)
-  #else
-  #define endian_swap(x) htole32(x)
-  #endif
++ disable_bit: This must be set to the bit to disable (same bit that was
+  previously registered via enable_bit).
 
-  long index, mask, *status_page;
++ disable_addr: This must be set to the address to disable (same address that was
+  previously registered via enable_addr).
 
-  index = status_bit / __BITS_PER_LONG;
-  mask = 1L << (status_bit % __BITS_PER_LONG);
-  mask = endian_swap(mask);
+**NOTE:** Events are automatically unregistered when execve() is invoked. During
+fork() the registered events will be retained and must be unregistered manually
+in each process if wanted.
 
-  ...
+Status
+------
+When tools attach/record user based events the status of the event is updated
+in realtime. This allows user programs to only incur the cost of the write() or
+writev() calls when something is actively attached to the event.
 
-  if (status_page[index] & mask) {
-        /* Enabled */
-  }
+The kernel will update the specified bit that was registered for the event as
+tools attach/detach from the event. User programs simply check if the bit is set
+to see if something is attached or not.
 
 Administrators can easily check the status of all registered events by reading
 the user_events_status file directly via a terminal. The output is as follows::
 
-  Byte:Name [# Comments]
+  Name [# Comments]
   ...
 
   Active: ActiveCount
   Busy: BusyCount
-  Max: MaxCount
 
 For example, on a system that has a single event the output looks like this::
 
-  1:test
+  test
 
   Active: 1
   Busy: 0
-  Max: 32768
 
 If a user enables the user event via ftrace, the output would change to this::
 
-  1:test # Used by ftrace
+  test # Used by ftrace
 
   Active: 1
   Busy: 1
-  Max: 32768
-
-**NOTE:** *A status bit of 0 will never be returned. This allows user programs
-to have a bit that can be used on error cases.*
 
 Writing Data
 ------------
Signature · 1 line
2.25.1
#15 lore raw

PATCH v5 10/11 tracing/user_events: Charge event allocs to cgroups

In reply to Beau Belgrave

Operators need a way to limit how much memory cgroups use. User events need to be included into that accounting. Fix this by using GFP_KERNEL_ACCOUNT for allocations generated by user programs for user_event tracing.

---
 kernel/trace/trace_events_user.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
kernel/trace/trace_events_user.c +10 -10
index ebbde72e1f85..36def244a755 100644
@@ -442,7 +442,7 @@ static bool user_event_enabler_dup(struct user_event_enabler *orig,
 	if (unlikely(test_bit(ENABLE_VAL_FREEING_BIT, ENABLE_BITOPS(orig))))
 		return true;
 
-	enabler = kzalloc(sizeof(*enabler), GFP_NOWAIT);
+	enabler = kzalloc(sizeof(*enabler), GFP_NOWAIT | __GFP_ACCOUNT);
 
 	if (!enabler)
 		return false;
@@ -502,7 +502,7 @@ static struct user_event_mm *user_event_mm_create(struct task_struct *t)
 	struct user_event_mm *user_mm;
 	unsigned long flags;
 
-	user_mm = kmalloc(sizeof(*user_mm), GFP_KERNEL);
+	user_mm = kmalloc(sizeof(*user_mm), GFP_KERNEL_ACCOUNT);
 
 	if (!user_mm)
 		return NULL;
@@ -652,7 +652,7 @@ static struct user_event_enabler
 	if (!user_mm)
 		return NULL;
 
-	enabler = kzalloc(sizeof(*enabler), GFP_KERNEL);
+	enabler = kzalloc(sizeof(*enabler), GFP_KERNEL_ACCOUNT);
 
 	if (!enabler)
 		goto out;
@@ -860,7 +860,7 @@ static int user_event_add_field(struct user_event *user, const char *type,
 	struct ftrace_event_field *field;
 	int validator_flags = 0;
 
-	field = kmalloc(sizeof(*field), GFP_KERNEL);
+	field = kmalloc(sizeof(*field), GFP_KERNEL_ACCOUNT);
 
 	if (!field)
 		return -ENOMEM;
@@ -879,7 +879,7 @@ static int user_event_add_field(struct user_event *user, const char *type,
 	if (strstr(type, "char") != NULL)
 		validator_flags |= VALIDATOR_ENSURE_NULL;
 
-	validator = kmalloc(sizeof(*validator), GFP_KERNEL);
+	validator = kmalloc(sizeof(*validator), GFP_KERNEL_ACCOUNT);
 
 	if (!validator) {
 		kfree(field);
@@ -1165,7 +1165,7 @@ static int user_event_create_print_fmt(struct user_event *user)
 
 	len = user_event_set_print_fmt(user, NULL, 0);
 
-	print_fmt = kmalloc(len, GFP_KERNEL);
+	print_fmt = kmalloc(len, GFP_KERNEL_ACCOUNT);
 
 	if (!print_fmt)
 		return -ENOMEM;
@@ -1498,7 +1498,7 @@ static int user_event_create(const char *raw_command)
 	raw_command += USER_EVENTS_PREFIX_LEN;
 	raw_command = skip_spaces(raw_command);
 
-	name = kstrdup(raw_command, GFP_KERNEL);
+	name = kstrdup(raw_command, GFP_KERNEL_ACCOUNT);
 
 	if (!name)
 		return -ENOMEM;
@@ -1692,7 +1692,7 @@ static int user_event_parse(struct user_event_group *group, char *name,
 		return 0;
 	}
 
-	user = kzalloc(sizeof(*user), GFP_KERNEL);
+	user = kzalloc(sizeof(*user), GFP_KERNEL_ACCOUNT);
 
 	if (!user)
 		return -ENOMEM;
@@ -1861,7 +1861,7 @@ static int user_events_open(struct inode *node, struct file *file)
 	if (!group)
 		return -ENOENT;
 
-	info = kzalloc(sizeof(*info), GFP_KERNEL);
+	info = kzalloc(sizeof(*info), GFP_KERNEL_ACCOUNT);
 
 	if (!info)
 		return -ENOMEM;
@@ -1914,7 +1914,7 @@ static int user_events_ref_add(struct user_event_file_info *info,
 
 	size = struct_size(refs, events, count + 1);
 
-	new_refs = kzalloc(size, GFP_KERNEL);
+	new_refs = kzalloc(size, GFP_KERNEL_ACCOUNT);
 
 	if (!new_refs)
 		return -ENOMEM;
Signature · 1 line
2.25.1
#16 lore raw

PATCH v5 11/11 tracing/user_events: Limit global user_event count

In reply to Beau Belgrave

Operators want to be able to ensure enough tracepoints exist on the system for kernel components as well as for user components. Since there are only up to 64K events, by default allow up to half to be used by user events.

Add a boot parameter (user_events_max=%d) and a kernel sysctl parameter (kernel.user_events_max) to set a global limit that is honored among all groups on the system. This ensures hard limits can be setup to prevent user processes from consuming all event IDs on the system.

---
 kernel/trace/trace_events_user.c | 57 ++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)
kernel/trace/trace_events_user.c +57 -0
index 36def244a755..754942ba92a1 100644
@@ -20,6 +20,7 @@
 #include <linux/types.h>
 #include <linux/uaccess.h>
 #include <linux/highmem.h>
+#include <linux/init.h>
 #include <linux/user_events.h>
 #include "trace.h"
 #include "trace_dynevent.h"
@@ -61,6 +62,12 @@ struct user_event_group {
 /* Group for init_user_ns mapping, top-most group */
 static struct user_event_group *init_group;
 
+/* Max allowed events for the whole system */
+static unsigned int max_user_events = 32768;
+
+/* Current number of events on the whole system */
+static unsigned int current_user_events;
+
 /*
  * Stores per-event properties, as users register events
  * within a file a user_event might be created if it does not
@@ -1247,6 +1254,11 @@ static int destroy_user_event(struct user_event *user)
 	kfree(EVENT_NAME(user));
 	kfree(user);
 
+	if (current_user_events > 0)
+		current_user_events--;
+	else
+		pr_alert("BUG: Bad current_user_events\n");
+
 	return ret;
 }
 
@@ -1732,6 +1744,11 @@ static int user_event_parse(struct user_event_group *group, char *name,
 
 	mutex_lock(&event_mutex);
 
+	if (current_user_events >= max_user_events) {
+		ret = -EMFILE;
+		goto put_user_lock;
+	}
+
 	ret = user_event_trace_register(user);
 
 	if (ret)
@@ -1743,6 +1760,7 @@ static int user_event_parse(struct user_event_group *group, char *name,
 	dyn_event_init(&user->devent, &user_event_dops);
 	dyn_event_add(&user->devent, &user->call);
 	hash_add(group->register_table, &user->node, key);
+	current_user_events++;
 
 	mutex_unlock(&event_mutex);
 
@@ -2369,6 +2387,43 @@ static int create_user_tracefs(void)
 	return -ENODEV;
 }
 
+static int __init set_max_user_events(char *str)
+{
+	if (!str)
+		return 0;
+
+	if (kstrtouint(str, 0, &max_user_events))
+		return 0;
+
+	return 1;
+}
+__setup("user_events_max=", set_max_user_events);
+
+static int set_max_user_events_sysctl(struct ctl_table *table, int write,
+				      void *buffer, size_t *lenp, loff_t *ppos)
+{
+	int ret;
+
+	mutex_lock(&event_mutex);
+
+	ret = proc_douintvec(table, write, buffer, lenp, ppos);
+
+	mutex_unlock(&event_mutex);
+
+	return ret;
+}
+
+static struct ctl_table user_event_sysctls[] = {
+	{
+		.procname	= "user_events_max",
+		.data		= &max_user_events,
+		.maxlen		= sizeof(unsigned int),
+		.mode		= 0644,
+		.proc_handler	= set_max_user_events_sysctl,
+	},
+	{}
+};
+
 static int __init trace_events_user_init(void)
 {
 	int ret;
@@ -2398,6 +2453,8 @@ static int __init trace_events_user_init(void)
 	if (dyn_event_register(&user_event_dops))
 		pr_warn("user_events could not register with dyn_events\n");
 
+	register_sysctl_init("kernel", user_event_sysctls);
+
 	return 0;
 }
 
Signature · 1 line
2.25.1
#17 lore raw

Re: [PATCH v5 11/11] tracing/user_events: Limit global user_event count

Quoted 41 lines · Beau Belgrave

Operators want to be able to ensure enough tracepoints exist on the system for kernel components as well as for user components. Since there are only up to 64K events, by default allow up to half to be used by user events.

Add a boot parameter (user_events_max=%d) and a kernel sysctl parameter (kernel.user_events_max) to set a global limit that is honored among all groups on the system. This ensures hard limits can be setup to prevent user processes from consuming all event IDs on the system.

---
  kernel/trace/trace_events_user.c | 57 ++++++++++++++++++++++++++++++++
  1 file changed, 57 insertions(+)
kernel/trace/trace_events_user.c +7 -0
index 36def244a755..754942ba92a1 100644
@@ -20,6 +20,7 @@
  #include <linux/types.h>
  #include <linux/uaccess.h>
  #include <linux/highmem.h>
+#include <linux/init.h>
  #include <linux/user_events.h>
  #include "trace.h"
  #include "trace_dynevent.h"
@@ -61,6 +62,12 @@ struct user_event_group {
  /* Group for init_user_ns mapping, top-most group */
  static struct user_event_group *init_group;
 
+/* Max allowed events for the whole system */
+static unsigned int max_user_events = 32768;
+
+/* Current number of events on the whole system */
+static unsigned int current_user_events;
+
  /*
   * Stores per-event properties, as users register events
   * within a file a user_event might be created if it does not

kernel/trace/trace_events_user.c static int destroy_user_event(struct user_event *user) · line 1258

Quoted 5 lines
kernel/trace/trace_events_user.c +2 -0
@@ -1247,6 +1254,11 @@ static int destroy_user_event(struct user_event *user)
  	kfree(EVENT_NAME(user));
  	kfree(user);
 
+	if (current_user_events > 0)
+		current_user_events--;

What holds the user_events mutex that guarantees that non-atomic decrement is safe here ?

Thanks,

Mathieu

Quoted 79 lines
kernel/trace/trace_events_user.c +48 -0
+	else
+		pr_alert("BUG: Bad current_user_events\n");
+
  	return ret;
  }
 
@@ -1732,6 +1744,11 @@ static int user_event_parse(struct user_event_group *group, char *name,
 
  	mutex_lock(&event_mutex);
 
+	if (current_user_events >= max_user_events) {
+		ret = -EMFILE;
+		goto put_user_lock;
+	}
+
  	ret = user_event_trace_register(user);
 
  	if (ret)
@@ -1743,6 +1760,7 @@ static int user_event_parse(struct user_event_group *group, char *name,
  	dyn_event_init(&user->devent, &user_event_dops);
  	dyn_event_add(&user->devent, &user->call);
  	hash_add(group->register_table, &user->node, key);
+	current_user_events++;
 
  	mutex_unlock(&event_mutex);
 
@@ -2369,6 +2387,43 @@ static int create_user_tracefs(void)
  	return -ENODEV;
  }
 
+static int __init set_max_user_events(char *str)
+{
+	if (!str)
+		return 0;
+
+	if (kstrtouint(str, 0, &max_user_events))
+		return 0;
+
+	return 1;
+}
+__setup("user_events_max=", set_max_user_events);
+
+static int set_max_user_events_sysctl(struct ctl_table *table, int write,
+				      void *buffer, size_t *lenp, loff_t *ppos)
+{
+	int ret;
+
+	mutex_lock(&event_mutex);
+
+	ret = proc_douintvec(table, write, buffer, lenp, ppos);
+
+	mutex_unlock(&event_mutex);
+
+	return ret;
+}
+
+static struct ctl_table user_event_sysctls[] = {
+	{
+		.procname	= "user_events_max",
+		.data		= &max_user_events,
+		.maxlen		= sizeof(unsigned int),
+		.mode		= 0644,
+		.proc_handler	= set_max_user_events_sysctl,
+	},
+	{}
+};
+
  static int __init trace_events_user_init(void)
  {
  	int ret;
@@ -2398,6 +2453,8 @@ static int __init trace_events_user_init(void)
  	if (dyn_event_register(&user_event_dops))
  		pr_warn("user_events could not register with dyn_events\n");
 
+	register_sysctl_init("kernel", user_event_sysctls);
+
  	return 0;
  }
 
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com
#18 lore raw

Re: [PATCH v5 11/11] tracing/user_events: Limit global user_event count

Quoted 40 lines · Mathieu Desnoyers
Quoted 39 lines · Beau Belgrave

Operators want to be able to ensure enough tracepoints exist on the system for kernel components as well as for user components. Since there are only up to 64K events, by default allow up to half to be used by user events.

Add a boot parameter (user_events_max=%d) and a kernel sysctl parameter (kernel.user_events_max) to set a global limit that is honored among all groups on the system. This ensures hard limits can be setup to prevent user processes from consuming all event IDs on the system.

---
  kernel/trace/trace_events_user.c | 57 ++++++++++++++++++++++++++++++++
  1 file changed, 57 insertions(+)
kernel/trace/trace_events_user.c +7 -0
index 36def244a755..754942ba92a1 100644
@@ -20,6 +20,7 @@
  #include <linux/types.h>
  #include <linux/uaccess.h>
  #include <linux/highmem.h>
+#include <linux/init.h>
  #include <linux/user_events.h>
  #include "trace.h"
  #include "trace_dynevent.h"
@@ -61,6 +62,12 @@ struct user_event_group {
  /* Group for init_user_ns mapping, top-most group */
  static struct user_event_group *init_group;
+/* Max allowed events for the whole system */
+static unsigned int max_user_events = 32768;
+
+/* Current number of events on the whole system */
+static unsigned int current_user_events;
+
  /*
   * Stores per-event properties, as users register events
   * within a file a user_event might be created if it does not
Quoted 9 lines

kernel/trace/trace_events_user.c static int destroy_user_event(struct user_event *user) · line 1257

Quoted 5 lines
kernel/trace/trace_events_user.c +2 -0
@@ -1247,6 +1254,11 @@ static int destroy_user_event(struct user_event *user)
  	kfree(EVENT_NAME(user));
  	kfree(user);
+	if (current_user_events > 0)
+		current_user_events--;

What holds the user_events mutex that guarantees that non-atomic decrement is safe here ?

All callers of destroy_user_event hold the event_mutex, since it removes the call from the system. This is the same for when the current_user_events get incremented.

Maybe add a lock_dep statement here to make it clear?

Thanks,
-Beau

Quoted 79 lines

Thanks,

Mathieu

Quoted 70 lines
kernel/trace/trace_events_user.c +48 -0
+	else
+		pr_alert("BUG: Bad current_user_events\n");
+
  	return ret;
  }
@@ -1732,6 +1744,11 @@ static int user_event_parse(struct user_event_group *group, char *name,
  	mutex_lock(&event_mutex);
+	if (current_user_events >= max_user_events) {
+		ret = -EMFILE;
+		goto put_user_lock;
+	}
+
  	ret = user_event_trace_register(user);
  	if (ret)
@@ -1743,6 +1760,7 @@ static int user_event_parse(struct user_event_group *group, char *name,
  	dyn_event_init(&user->devent, &user_event_dops);
  	dyn_event_add(&user->devent, &user->call);
  	hash_add(group->register_table, &user->node, key);
+	current_user_events++;
  	mutex_unlock(&event_mutex);
@@ -2369,6 +2387,43 @@ static int create_user_tracefs(void)
  	return -ENODEV;
  }
+static int __init set_max_user_events(char *str)
+{
+	if (!str)
+		return 0;
+
+	if (kstrtouint(str, 0, &max_user_events))
+		return 0;
+
+	return 1;
+}
+__setup("user_events_max=", set_max_user_events);
+
+static int set_max_user_events_sysctl(struct ctl_table *table, int write,
+				      void *buffer, size_t *lenp, loff_t *ppos)
+{
+	int ret;
+
+	mutex_lock(&event_mutex);
+
+	ret = proc_douintvec(table, write, buffer, lenp, ppos);
+
+	mutex_unlock(&event_mutex);
+
+	return ret;
+}
+
+static struct ctl_table user_event_sysctls[] = {
+	{
+		.procname	= "user_events_max",
+		.data		= &max_user_events,
+		.maxlen		= sizeof(unsigned int),
+		.mode		= 0644,
+		.proc_handler	= set_max_user_events_sysctl,
+	},
+	{}
+};
+
  static int __init trace_events_user_init(void)
  {
  	int ret;
@@ -2398,6 +2453,8 @@ static int __init trace_events_user_init(void)
  	if (dyn_event_register(&user_event_dops))
  		pr_warn("user_events could not register with dyn_events\n");
+	register_sysctl_init("kernel", user_event_sysctls);
+
  	return 0;
  }
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com